From ioi.lam at oracle.com Sun Mar 1 23:27:24 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Sun, 1 Mar 2020 15:27:24 -0800 Subject: RFR(T) 8240267 VM fails to start with CDS enabled but JVMTI disabled Message-ID: <686e76fc-e287-6044-cbf4-3af6f67a4d64@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8240267 http://cr.openjdk.java.net/~iklam/jdk15/8240267-cds-enabled-jvmti-disabled.v01/ This affects non-product builds only. The fix is to put the JVMTI-related asserts inside JVMTI_ONLY(). Thanks - Ioi From david.holmes at oracle.com Sun Mar 1 23:52:41 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 2 Mar 2020 09:52:41 +1000 Subject: RFR(T) 8240267 VM fails to start with CDS enabled but JVMTI disabled In-Reply-To: <686e76fc-e287-6044-cbf4-3af6f67a4d64@oracle.com> References: <686e76fc-e287-6044-cbf4-3af6f67a4d64@oracle.com> Message-ID: <01ef589a-5dbc-8ec0-06d4-d8bf5929fe3f@oracle.com> Hi Ioi, Looks fine and trivial, but could you fix up the indentation please. Thanks, David On 2/03/2020 9:27 am, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8240267 > http://cr.openjdk.java.net/~iklam/jdk15/8240267-cds-enabled-jvmti-disabled.v01/ > > > This affects non-product builds only. The fix is to put the > JVMTI-related asserts inside JVMTI_ONLY(). > > Thanks > - Ioi From ioi.lam at oracle.com Mon Mar 2 04:31:20 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Sun, 1 Mar 2020 20:31:20 -0800 Subject: RFR(T) 8240267 VM fails to start with CDS enabled but JVMTI disabled In-Reply-To: <01ef589a-5dbc-8ec0-06d4-d8bf5929fe3f@oracle.com> References: <686e76fc-e287-6044-cbf4-3af6f67a4d64@oracle.com> <01ef589a-5dbc-8ec0-06d4-d8bf5929fe3f@oracle.com> Message-ID: <251435e5-431a-d67b-3762-027000f40aaf@oracle.com> Hi David, Thanks for the review. I fixed the indentation and pushed. - Ioi On 3/1/20 3:52 PM, David Holmes wrote: > Hi Ioi, > > Looks fine and trivial, but could you fix up the indentation please. > > Thanks, > David > > On 2/03/2020 9:27 am, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8240267 >> http://cr.openjdk.java.net/~iklam/jdk15/8240267-cds-enabled-jvmti-disabled.v01/ >> >> >> This affects non-product builds only. The fix is to put the >> JVMTI-related asserts inside JVMTI_ONLY(). >> >> Thanks >> - Ioi From david.holmes at oracle.com Mon Mar 2 06:14:41 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 2 Mar 2020 16:14:41 +1000 Subject: RFR(S): 8232081: Try to link all classes during dynamic CDS dump In-Reply-To: <8ac4612b-fae3-bb16-f2ab-fa323c33ca51@oracle.com> References: <9b40145a-db72-6313-3b98-5e5aea754e25@oracle.com> <1167ff6d-3997-6390-d020-d0946fad0af8@oracle.com> <3f373a33-c7ed-0237-1dd5-f8ac8e8794ee@oracle.com> <8ac4612b-fae3-bb16-f2ab-fa323c33ca51@oracle.com> Message-ID: <00090c43-1e0d-2b1c-d2a5-1b5f1b29199d@oracle.com> Hi Calvin, On 28/02/2020 4:12 pm, Calvin Cheung wrote: > Hi David, > > On 2/27/20 5:40 PM, David Holmes wrote: >> Hi Calvin, Ioi, >> >> Looking good - comments below. >> >> A meta-question: normal application classes are rarely loaded but not >> linked so I'm a little surprised this is an issue. What is the main >> source of such classes - generated classes like lambda forms? Also why >> do we need to link them when loading from the archive if they were >> never linked when the application actually executed ?? > > I saw that Ioi already answered the above. > > I'll try to answer your questions inline below.. Responses inline below ... >> >> On 28/02/2020 10:48 am, Calvin Cheung wrote: >>> Hi David, Ioi, >>> >>> Thanks for your review and suggestions. >>> >>> On 2/26/20 9:46 PM, Ioi Lam wrote: >>>> >>>> >>>> On 2/26/20 7:50 PM, David Holmes wrote: >>>>> Hi Calvin, >>>>> >>>>> Adding core-libs-dev as you are messing with their code :) >>>>> >>>>> On 27/02/2020 1:19 pm, Calvin Cheung wrote: >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8232081 >>>>>> webrev: http://cr.openjdk.java.net/~ccheung/jdk15/8232081/webrev.00/ >>>>>> >>>>>> The proposed changeset for this RFE adds a JVM_LinkClassesForCDS() >>>>>> function to be called from java/lang/Shutdown to notify the JVM to >>>>>> link the classes loaded by the builtin class loaders. The >>>>> >>>>> This would be much less disruptive if this was handled purely on >>>>> the VM side once we have started shutdown. No need to make any >>>>> changes to the Java side then, nor jvm.cpp. >>>>> >>>> >>>> Hi David, >>>> >>>> To link the classes, we need to be able to run Java code -- when >>>> linking a class X loaded by the app loader, X will be verified. >>>> During verification of X, we may need to load additional classes >>>> from the app loader, which executes Java code during its class >>>> loading operations. >>>> >>>> We also need to handle all the exit conditions. As far as I can >>>> tell, an app can exit the JVM in two ways: >>>> >>>> (1) Explicitly calling System.exit(). This will call >>>> java.lang.Shutdown.exit() which does this: >>>> >>>> http://hg.openjdk.java.net/jdk/jdk/file/0edc7fd0d7a3/src/java.base/share/classes/java/lang/Shutdown.java#l163 >>>> >>>> >>>> ??????????? beforeHalt(); // native >>>> ??????????? runHooks(); >>>> ??????????? halt(status); >>>> >>>> (2) When all non-daemon threads have died (e.g., falling out of the >>>> bottom of HelloWorld.main()). There's no explicit call to >>>> System.exit(), but the JVM will proactively call >>>> java.lang.Shutdown.shutdown() inside >>>> JavaThread::invoke_shutdown_hooks() >>>> >>>> http://hg.openjdk.java.net/jdk/jdk/file/0edc7fd0d7a3/src/hotspot/share/runtime/thread.cpp#l4331 >>>> >>>> http://hg.openjdk.java.net/jdk/jdk/file/0edc7fd0d7a3/src/java.base/share/classes/java/lang/Shutdown.java#l184 >>>> >>>> >>>> If we want to avoid modifying the Java code, I think we can >>>> intercept JVM_BeforeHalt() and JavaThread::invoke_shutdown_hooks(). >>>> This way we should be able to handle all the classes (except those >>>> that are loaded inside Shutdown.runHooks). >>> >>> I've implemented the above. No changes to the Java side. >>> >>> updated webrev: >>> >>> http://cr.openjdk.java.net/~ccheung/jdk15/8232081/webrev.01/ >> >> This looks much better to me. I wasn't sure if you were that concerned >> about catching the classes loaded by the Shutdown hooks, but it is >> good you are not as it seems problematic to me to trigger class >> loading etc after the shutdown hooks have executed - you may hit >> unexpected conditions. So this approach is good. >> >> A few minor comments: >> >> src/hotspot/share/memory/metaspaceShared.cpp >> src/hotspot/share/memory/metaspaceShared.hpp >> >> Why the change from TRAPS to "Thread* THREAD"? > I forgot why I changed it but I've changed it back and it still works. Ok. >> >> --- >> >> src/hotspot/share/oops/instanceKlass.cpp >> >> I'm not clear how verify_on is used so am unsure how the additional >> error state check may affect code unrelated to dynamic dumping ?? > > Some of the appcds tests by design have some classes which fail > verification. Without the change, the following assert would fail: > > void vtableEntry::verify(klassVtable* vt, outputStream* st) { > ? Klass* vtklass = vt->klass(); > ? if (vtklass->is_instance_klass() && > ???? (InstanceKlass::cast(vtklass)->major_version() >= > klassVtable::VTABLE_TRANSITIVE_OVERRIDE_VERSION)) { > *assert*(method() != NULL, "must have set method"); > ? } Okay so you need to exclude for that case but ... > Here's the call stack during dynamic CDS dump to the above function: > > vtableEntry::verify(klassVtable *, outputStream *) : void > ??? klassVtable::verify(outputStream *, bool) : void > ??????? InstanceKlass::verify_on(outputStream *) : void > ??????????? Klass::verify() : void > ??????????????? ClassLoaderData::verify() : void > ??????????????????? ClassLoaderDataGraph::verify() : void > ??????????????????????? Universe::verify(enum VerifyOption, const char > *) : void > ??????????????????????????? Universe::verify(const char *) : void > DynamicArchiveBuilder::verify_universe(const char *) : void > ??????????????????????????????????? DynamicArchiveBuilder::doit() : > void (2 matches) > VM_PopulateDynamicDumpSharedSpace::doit() : void > VM_Operation::evaluate() : void > > The is_linked() function is implemented as: > > ??? bool is_linked() const?????????????????? { return _init_state >= > linked; } > > which includes 'initialization_error'. ... AFAICS Universe::verify is call from a number of places, all of which would reach this modified code. As that existing code has included classes that are in the error state then it seems to me that unless you can prove otherwise you need to keep the existing code as-is for the non-CDS dump case. > A previous version of the changeset (including the change in question in > instanceKlass.cpp) passed tier1 - 4 testing. Let me know if I should run > other tests to make sure the change is good. There may well not be a test that covers this. Thanks, David >> >> Also can I suggest (as you are touching this code) to delete the >> ancient comment: >> >> 3580???? // $$$ This used to be done only for m/s collections. Doing it >> 3581???? // always seemed a valid generalization.? (DLD -- 6/00) > > I'm glad that you suggested to remove the above comment. I've no clue > what it means either. > > updated webrev: > > http://cr.openjdk.java.net/~ccheung/jdk15/8232081/webrev.02/ > > thanks, > Calvin >> >> Thanks, >> David >> ----- >> >>> I also updated the test to test the shutdown hook and System.exit() >>> scenarios. >>> >>> All appcds tests passed on my linux host. I'll run more tests using >>> mach5. >>> >>> thanks, >>> >>> Calvin >>> >>> >>>> >>>> What do you think? >>>> >>>> - Ioi >>>> >>>>> Thanks, >>>>> David >>>>> >>>>>> MetaspaceShared::link_and_cleanup_shared_classes() has been >>>>>> modified to handle both static and dynamic CDS dump. For dynamic >>>>>> CDS dump, only classes loaded by the builtin class loaders will be >>>>>> linked. Local performance testing using javac on HelloWorld.java >>>>>> shows an improvement of >5%. >>>>>> >>>>>> Passed tier1 - 4 tests. >>>>>> >>>>>> thanks, >>>>>> >>>>>> Calvin >>>>>> >>>> From claes.redestad at oracle.com Mon Mar 2 09:22:32 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 2 Mar 2020 10:22:32 +0100 Subject: RFR: 8240263: Assertion-only call in Method::link_method affecting product builds Message-ID: <6a41ef95-8d49-f694-57b2-8d7ddf3f480b@oracle.com> Hi, in Method::link_method we resolve an entry when linking methods from CDS which is only used for purposes of some assertions. Even though the resolution does not have any side effect compilers can't see through into all parts of this call (specifically method_kind) to prove this and DCE the code. Guarding the resolution by DEBUG_ONLY seem the right thing to do here, which bring a small improvement on startup tests when CDS is enabled. See attached patch[1]. Bug: https://bugs.openjdk.java.net/browse/JDK-8240263 Testing: tier1-3 Thanks! /Claes [1] diff -r acc083236275 src/hotspot/share/oops/method.cpp --- a/src/hotspot/share/oops/method.cpp Mon Mar 02 08:22:48 2020 +0100 +++ b/src/hotspot/share/oops/method.cpp Mon Mar 02 10:15:37 2020 +0100 @@ -1175,7 +1175,7 @@ // If the code cache is full, we may reenter this function for the // leftover methods that weren't linked. if (is_shared()) { - address entry = Interpreter::entry_for_cds_method(h_method); + DEBUG_ONLY(address entry = Interpreter::entry_for_cds_method(h_method);) assert(entry != NULL && entry == _i2i_entry, "should be correctly set during dump time"); if (adapter() != NULL) { From shade at redhat.com Mon Mar 2 09:22:44 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 2 Mar 2020 10:22:44 +0100 Subject: RFR: 8240263: Assertion-only call in Method::link_method affecting product builds In-Reply-To: <6a41ef95-8d49-f694-57b2-8d7ddf3f480b@oracle.com> References: <6a41ef95-8d49-f694-57b2-8d7ddf3f480b@oracle.com> Message-ID: <9d8dae94-f992-6345-58e6-a28fa2789111@redhat.com> On 3/2/20 10:22 AM, Claes Redestad wrote: > Guarding the resolution by DEBUG_ONLY seem the right thing to do here, > which bring a small improvement on startup tests when CDS is enabled. > > See attached patch[1]. I believe we usually do this in such case: #ifdef ASSERT address entry = Interpreter::entry_for_cds_method(h_method); assert(entry != NULL && entry == _i2i_entry, "should be correctly set during dump time"); #endif It keeps the block easily editable, without looking back at whether it interferes with macros. Otherwise looks fine, assuming release builds still pass. -- Thanks, -Aleksey From claes.redestad at oracle.com Mon Mar 2 10:08:32 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 2 Mar 2020 11:08:32 +0100 Subject: RFR: 8240263: Assertion-only call in Method::link_method affecting product builds In-Reply-To: <9d8dae94-f992-6345-58e6-a28fa2789111@redhat.com> References: <6a41ef95-8d49-f694-57b2-8d7ddf3f480b@oracle.com> <9d8dae94-f992-6345-58e6-a28fa2789111@redhat.com> Message-ID: <43f27f02-6ddf-7ad3-754e-97e73280772b@oracle.com> On 2020-03-02 10:22, Aleksey Shipilev wrote: > On 3/2/20 10:22 AM, Claes Redestad wrote: >> Guarding the resolution by DEBUG_ONLY seem the right thing to do here, >> which bring a small improvement on startup tests when CDS is enabled. >> >> See attached patch[1]. > > I believe we usually do this in such case: > > #ifdef ASSERT > address entry = Interpreter::entry_for_cds_method(h_method); > assert(entry != NULL && entry == _i2i_entry, > "should be correctly set during dump time"); > #endif I've seen it done both ways, and have no strong preference. > > It keeps the block easily editable, without looking back at whether it interferes with macros. > > Otherwise looks fine, assuming release builds still pass. Running your suggested variant through tier 1 to make sure. Thanks! /Claes From robbin.ehn at oracle.com Mon Mar 2 10:16:44 2020 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Mon, 2 Mar 2020 11:16:44 +0100 Subject: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents In-Reply-To: References: <1f8a3c7a-fa0f-b5b2-4a8a-7d3d8dbbe1b5@oracle.com> <4b56a45c-a14c-6f74-2bfd-25deaabe8201@oracle.com> <5271429a-481d-ddb9-99dc-b3f6670fcc0b@oracle.com> Message-ID: Hi, On 2/24/20 5:39 PM, Lindenmaier, Goetz wrote: > Hi, > > I had a look at the progress of this change. Nothing > happened since Richard posted his update using more > handshakes [1]. > But we (SAP) would appreciate a lot if this change could > be successfully reviewed and pushed. > > I think there is basic understanding that this > change is helpful. It fixes a number of issues with JVMTI, > and will deliver the same performance benefits as EA > does in current production mode for debugging scenarios. > > This is important for us as we run our VMs prepared > for debugging in production mode. > > I understand that Robbin proposed to replace the usage of > _suspend_flag with handshakes. Apparently, async handshakes > are needed to do so. We have been waiting a while for removal > of the _suspend_flag / introduction of async handshakes [2]. > What is the status here? I have an old prototype which I would like to continue to work on. So do not assume asynch handshakes will make 15. Even if it would, I think there are a lot more investigate work to remove _suspend_flag. > > I think we should no longer wait, but proceed with > this change. We will look into removing the usage of > suspend_flag introduced here once it is possible to implement > it with handshakes. Yes, sure. >> Full: http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.4/ DeoptimizeObjectsALotThread is only used in compileBroker.cpp. You can move both declaration and definition to that file, no need to clobber thread.[c|h]pp. (and the static function deopt_objs_alot_thread_entry) Does JvmtiDeferredUpdates really need to be in thread.hpp, can't be in it's own hpp file? It doesn't seem right to add JVM TI classes into thread.hpp. Note that we also think we may have a bug in deopt: https://bugs.openjdk.java.net/browse/JDK-8238237 I think it would be best, if possible, to push after that is resolved. Not even nearly a full review :) Thanks, Robbin >> Incremental: >> http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.4.inc/ >> >> I was not able to eliminate the additional suspend flag now. I'll take care of this >> as soon as the >> existing suspend-resume-mechanism is reworked. >> >> Testing: >> >> Nightly tests @SAP: >> >> JCK and JTREG, also in Xcomp mode, SPECjvm2008, SPECjbb2015, Renaissance >> Suite, SAP specific tests >> with fastdebug and release builds on all platforms >> >> Stress testing with DeoptimizeObjectsALot running SPECjvm2008 40x parallel >> for 24h >> >> Thanks, Richard. >> >> >> More details on the changes: >> >> * Hide DeoptimizeObjectsALotThread from external view. >> >> * Changed EscapeBarrier_lock to be a _safepoint_check_never lock. >> It used to be _safepoint_check_sometimes, which will be eliminated sooner or >> later. >> I added explicit thread state changes with ThreadBlockInVM to code paths >> where we can wait() >> on EscapeBarrier_lock to become safepoint safe. >> >> * Use handshake EscapeBarrierSuspendHandshake to suspend target threads >> instead of vm operation >> VM_ThreadSuspendAllForObjDeopt. >> >> * Removed uses of Threads_lock. When adding a new thread we suspend it iff >> EA optimizations are >> being reverted. In the previous version we were waiting on Threads_lock >> while EA optimizations >> were reverted. See EscapeBarrier::thread_added(). >> >> * Made tests require Xmixed compilation mode. >> >> * Made tests agnostic regarding tiered compilation. >> I.e. tc isn't disabled anymore, and the tests can be run with tc enabled or >> disabled. >> >> * Exercising EATests.java as well with stress test options >> DeoptimizeObjectsALot* >> Due to the non-deterministic deoptimizations some tests need to be skipped. >> We do this to prevent bit-rot of the stress test code. >> >> * Executing EATests.java as well with graal if available. Driver for this is >> EATestsJVMCI.java. Graal cannot pass all tests, because it does not provide all >> the new debug info >> (namely not_global_escape_in_scope and arg_escape in scopeDesc.hpp). >> And graal does not yet support the JVMTI operations force early return and >> pop frame. >> >> * Removed tracing from new jdi tests in EATests.java. Too much trace output >> before the debugging >> connection is established can cause deadlock because output buffers fill up. >> (See https://bugs.openjdk.java.net/browse/JDK-8173304) >> >> * Many copyright year changes and smaller clean-up changes of testing code >> (trailing white-space and >> the like). >> >> >> -----Original Message----- >> From: David Holmes >> Sent: Donnerstag, 19. Dezember 2019 03:12 >> To: Reingruber, Richard ; serviceability- >> dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot- >> runtime-dev at openjdk.java.net; Vladimir Kozlov (vladimir.kozlov at oracle.com) >> >> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better Performance in >> the Presence of JVMTI Agents >> >> Hi Richard, >> >> I think my issue is with the way EliminateNestedLocks works so I'm going >> to look into that more deeply. >> >> Thanks for the explanations. >> >> David >> >> On 18/12/2019 12:47 am, Reingruber, Richard wrote: >>> Hi David, >>> >>> > > > Some further queries/concerns: >>> > > > >>> > > > src/hotspot/share/runtime/objectMonitor.cpp >>> > > > >>> > > > Can you please explain the changes to ObjectMonitor::wait: >>> > > > >>> > > > ! _recursions = save // restore the old recursion count >>> > > > ! + jt->get_and_reset_relock_count_after_wait(); // >>> > > > increased by the deferred relock count >>> > > > >>> > > > what is the "deferred relock count"? I gather it relates to >>> > > > >>> > > > "The code was extended to be able to deoptimize objects of a >>> > > frame that >>> > > > is not the top frame and to let another thread than the owning >>> > > thread do >>> > > > it." >>> > > >>> > > Yes, these relate. Currently EA based optimizations are reverted, when a >> compiled frame is >>> > > replaced with corresponding interpreter frames. Part of this is relocking >> objects with eliminated >>> > > locking. New with the enhancement is that we do this also just before >> object references are >>> > > acquired through JVMTI. In this case we deoptimize also the owning >> compiled frame C and we >>> > > register deoptimized objects as deferred updates. When control returns >> to C it gets deoptimized, >>> > > we notice that objects are already deoptimized (reallocated and >> relocked), so we don't do it again >>> > > (relocking twice would be incorrect of course). Deferred updates are >> copied into the new >>> > > interpreter frames. >>> > > >>> > > Problem: relocking is not possible if the target thread T is waiting on the >> monitor that needs to >>> > > be relocked. This happens only with non-local objects with >> EliminateNestedLocks. Instead relocking >>> > > is deferred until T owns the monitor again. This is what the piece of >> code above does. >>> > >>> > Sorry I need some more detail here. How can you wait() on an object >>> > monitor if the object allocation and/or locking was optimised away? And >>> > what is a "non-local object" in this context? Isn't EA restricted to >>> > thread-confined objects? >>> >>> "Non-local object" is an object that escapes its thread. The issue I'm >> addressing with the changes >>> in ObjectMonitor::wait are almost unrelated to EA. They are caused by >> EliminateNestedLocks, where C2 >>> eliminates recursive locking of an already owned lock. The lock owning object >> exists on the heap, it >>> is locked and you can call wait() on it. >>> >>> EliminateLocks is the C2 option that controls lock elimination based on EA. >> Both optimizations have >>> in common that objects with eliminated locking need to be relocked when >> deoptimizing a frame, >>> i.e. when replacing a compiled frame with equivalent interpreter >>> frames. Deoptimization::relock_objects does that job for /all/ eliminated >> locks in scope. /All/ can >>> be a mix of eliminated nested locks and locks of not-escaping objects. >>> >>> New with the enhancement: I call relock_objects earlier, just before objects >> pontentially >>> escape. But then later when the owning compiled frame gets deoptimized, I >> must not do it again: >>> >>> See call to EscapeBarrier::objs_are_deoptimized in deoptimization.cpp: >>> >>> 373 if ((jvmci_enabled || ((DoEscapeAnalysis || EliminateNestedLocks) && >> EliminateLocks)) >>> 374 && !EscapeBarrier::objs_are_deoptimized(thread, deoptee.id())) { >>> 375 bool unused; >>> 376 eliminate_locks(thread, chunk, realloc_failures, deoptee, exec_mode, >> unused); >>> 377 } >>> >>> Now when calling relock_objects early it is quiet possible that I have to relock >> an object the >>> target thread currently waits for. Obviously I cannot relock in this case, >> instead I chose to >>> introduce relock_count_after_wait to JavaThread. >>> >>> > Is it just that some of the locking gets optimized away e.g. >>> > >>> > synchronised(obj) { >>> > synchronised(obj) { >>> > synchronised(obj) { >>> > obj.wait(); >>> > } >>> > } >>> > } >>> > >>> > If this is reduced to a form as-if it were a single lock of the monitor >>> > (due to EA) and the wait() triggers a JVM TI event which leads to the >>> > escape of "obj" then we need to reconstruct the true lock state, and so >>> > when the wait() internally unblocks and reacquires the monitor it has to >>> > set the true recursion count to 3, not the 1 that it appeared to be when >>> > wait() was initially called. Is that the scenario? >>> >>> Kind of... except that the locking is not eliminated due to EA and there is no >> JVM TI event >>> triggered by wait. >>> >>> Add >>> >>> LocalObject l1 = new LocalObject(); >>> >>> in front of the synchrnized blocks and assume a JVM TI agent acquires l1. This >> triggers the code in >>> question. >>> >>> See that relocking/reallocating is transactional. If it is done then for /all/ >> objects in scope and it is >>> done at most once. It wouldn't be quite so easy to split this in relocking of >> nested/EA-based >>> eliminated locks. >>> >>> > If so I find this truly awful. Anyone using wait() in a realistic form >>> > requires a notification and so the object cannot be thread confined. In >>> >>> It is not thread confined. >>> >>> > which case I would strongly argue that upon hitting the wait() the deopt >>> > should occur unconditionally and so the lock state is correct before we >>> > wait and so we don't need to mess with the recursion count internally >>> > when we reacquire the monitor. >>> > >>> > > >>> > > > which I don't like the sound of at all when it comes to ObjectMonitor >>> > > > state. So I'd like to understand in detail exactly what is going on here >>> > > > and why. This is a very intrusive change that seems to badly break >>> > > > encapsulation and impacts future changes to ObjectMonitor that are >> under >>> > > > investigation. >>> > > >>> > > I would not regard this as breaking encapsulation. Certainly not badly. >>> > > >>> > > I've added a property relock_count_after_wait to JavaThread. The >> property is well >>> > > encapsulated. Future ObjectMonitor implementations have to deal with >> recursion too. They are free >>> > > in choosing a way to do that as long as that property is taken into >> account. This is hardly a >>> > > limitation. >>> > >>> > I do think this badly breaks encapsulation as you have to add a callout >>> > from the guts of the ObjectMonitor code to reach into the thread to get >>> > this lock count adjustment. I understand why you have had to do this but >>> > I would much rather see a change to the EA optimisation strategy so that >>> > this is not needed. >>> > >>> > > Note also that the property is a straight forward extension of the >> existing concept of deferred >>> > > local updates. It is embedded into the structure holding them. So not >> even the footprint of a >>> > > JavaThread is enlarged if no deferred updates are generated. >>> > >>> > [...] >>> > >>> > > >>> > > I'm actually duplicating the existing external suspend mechanism, >> because a thread can be >>> > > suspended at most once. And hey, and don't like that either! But it >> seems not unlikely that the >>> > > duplicate can be removed together with the original and the new type >> of handshakes that will be >>> > > used for thread suspend can be used for object deoptimization too. See >> today's discussion in >>> > > JDK-8227745 [2]. >>> > >>> > I hope that discussion bears some fruit, at the moment it seems not to >>> > be possible to use handshakes here. :( >>> > >>> > The external suspend mechanism is a royal pain in the proverbial that we >>> > have to carefully live with. The idea that we're duplicating that for >>> > use in another fringe area of functionality does not thrill me at all. >>> > >>> > To be clear, I understand the problem that exists and that you wish to >>> > solve, but for the runtime parts I balk at the complexity cost of >>> > solving it. >>> >>> I know it's complex, but by far no rocket science. >>> >>> Also I find it hard to imagine another fix for JDK-8233915 besides changing >> the JVM TI specification. >>> >>> Thanks, Richard. >>> >>> -----Original Message----- >>> From: David Holmes >>> Sent: Dienstag, 17. Dezember 2019 08:03 >>> To: Reingruber, Richard ; serviceability- >> dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot- >> runtime-dev at openjdk.java.net; Vladimir Kozlov (vladimir.kozlov at oracle.com) >> >>> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better Performance >> in the Presence of JVMTI Agents >>> >>> >>> >>> David >>> >>> On 17/12/2019 4:57 pm, David Holmes wrote: >>>> Hi Richard, >>>> >>>> On 14/12/2019 5:01 am, Reingruber, Richard wrote: >>>>> Hi David, >>>>> >>>>> ?? > Some further queries/concerns: >>>>> ?? > >>>>> ?? > src/hotspot/share/runtime/objectMonitor.cpp >>>>> ?? > >>>>> ?? > Can you please explain the changes to ObjectMonitor::wait: >>>>> ?? > >>>>> ?? > !?? _recursions = save????? // restore the old recursion count >>>>> ?? > !???????????????? + jt->get_and_reset_relock_count_after_wait(); // >>>>> ?? > increased by the deferred relock count >>>>> ?? > >>>>> ?? > what is the "deferred relock count"? I gather it relates to >>>>> ?? > >>>>> ?? > "The code was extended to be able to deoptimize objects of a >>>>> frame that >>>>> ?? > is not the top frame and to let another thread than the owning >>>>> thread do >>>>> ?? > it." >>>>> >>>>> Yes, these relate. Currently EA based optimizations are reverted, when >>>>> a compiled frame is replaced >>>>> with corresponding interpreter frames. Part of this is relocking >>>>> objects with eliminated >>>>> locking. New with the enhancement is that we do this also just before >>>>> object references are acquired >>>>> through JVMTI. In this case we deoptimize also the owning compiled >>>>> frame C and we register >>>>> deoptimized objects as deferred updates. When control returns to C it >>>>> gets deoptimized, we notice >>>>> that objects are already deoptimized (reallocated and relocked), so we >>>>> don't do it again (relocking >>>>> twice would be incorrect of course). Deferred updates are copied into >>>>> the new interpreter frames. >>>>> >>>>> Problem: relocking is not possible if the target thread T is waiting >>>>> on the monitor that needs to be >>>>> relocked. This happens only with non-local objects with >>>>> EliminateNestedLocks. Instead relocking is >>>>> deferred until T owns the monitor again. This is what the piece of >>>>> code above does. >>>> >>>> Sorry I need some more detail here. How can you wait() on an object >>>> monitor if the object allocation and/or locking was optimised away? And >>>> what is a "non-local object" in this context? Isn't EA restricted to >>>> thread-confined objects? >>>> >>>> Is it just that some of the locking gets optimized away e.g. >>>> >>>> synchronised(obj) { >>>> ? synchronised(obj) { >>>> ??? synchronised(obj) { >>>> ????? obj.wait(); >>>> ??? } >>>> ? } >>>> } >>>> >>>> If this is reduced to a form as-if it were a single lock of the monitor >>>> (due to EA) and the wait() triggers a JVM TI event which leads to the >>>> escape of "obj" then we need to reconstruct the true lock state, and so >>>> when the wait() internally unblocks and reacquires the monitor it has to >>>> set the true recursion count to 3, not the 1 that it appeared to be when >>>> wait() was initially called. Is that the scenario? >>>> >>>> If so I find this truly awful. Anyone using wait() in a realistic form >>>> requires a notification and so the object cannot be thread confined. In >>>> which case I would strongly argue that upon hitting the wait() the deopt >>>> should occur unconditionally and so the lock state is correct before we >>>> wait and so we don't need to mess with the recursion count internally >>>> when we reacquire the monitor. >>>> >>>>> >>>>> ?? > which I don't like the sound of at all when it comes to >>>>> ObjectMonitor >>>>> ?? > state. So I'd like to understand in detail exactly what is going >>>>> on here >>>>> ?? > and why.? This is a very intrusive change that seems to badly break >>>>> ?? > encapsulation and impacts future changes to ObjectMonitor that >>>>> are under >>>>> ?? > investigation. >>>>> >>>>> I would not regard this as breaking encapsulation. Certainly not badly. >>>>> >>>>> I've added a property relock_count_after_wait to JavaThread. The >>>>> property is well >>>>> encapsulated. Future ObjectMonitor implementations have to deal with >>>>> recursion too. They are free in >>>>> choosing a way to do that as long as that property is taken into >>>>> account. This is hardly a >>>>> limitation. >>>> >>>> I do think this badly breaks encapsulation as you have to add a callout >>>> from the guts of the ObjectMonitor code to reach into the thread to get >>>> this lock count adjustment. I understand why you have had to do this but >>>> I would much rather see a change to the EA optimisation strategy so that >>>> this is not needed. >>>> >>>>> Note also that the property is a straight forward extension of the >>>>> existing concept of deferred >>>>> local updates. It is embedded into the structure holding them. So not >>>>> even the footprint of a >>>>> JavaThread is enlarged if no deferred updates are generated. >>>>> >>>>> ?? > --- >>>>> ?? > >>>>> ?? > src/hotspot/share/runtime/thread.cpp >>>>> ?? > >>>>> ?? > Can you please explain why >>>>> JavaThread::wait_for_object_deoptimization >>>>> ?? > has to be handcrafted in this way rather than using proper >>>>> transitions. >>>>> ?? > >>>>> >>>>> I wrote wait_for_object_deoptimization taking >>>>> JavaThread::java_suspend_self_with_safepoint_check >>>>> as template. So in short: for the same reasons :) >>>>> >>>>> Threads reach both methods as part of thread state transitions, >>>>> therefore special handling is >>>>> required to change thread state on top of ongoing transitions. >>>>> >>>>> ?? > We got rid of "deopt suspend" some time ago and it is disturbing >>>>> to see >>>>> ?? > it being added back (effectively). This seems like it may be >>>>> something >>>>> ?? > that handshakes could be used for. >>>>> >>>>> Deopt suspend used to be something rather different with a similar >>>>> name[1]. It is not being added back. >>>> >>>> I stand corrected. Despite comments in the code to the contrary >>>> deopt_suspend didn't actually cause a self-suspend. I was doing a lot of >>>> cleanup in this area 13 years ago :) >>>> >>>>> >>>>> I'm actually duplicating the existing external suspend mechanism, >>>>> because a thread can be suspended >>>>> at most once. And hey, and don't like that either! But it seems not >>>>> unlikely that the duplicate can >>>>> be removed together with the original and the new type of handshakes >>>>> that will be used for >>>>> thread suspend can be used for object deoptimization too. See today's >>>>> discussion in JDK-8227745 [2]. >>>> >>>> I hope that discussion bears some fruit, at the moment it seems not to >>>> be possible to use handshakes here. :( >>>> >>>> The external suspend mechanism is a royal pain in the proverbial that we >>>> have to carefully live with. The idea that we're duplicating that for >>>> use in another fringe area of functionality does not thrill me at all. >>>> >>>> To be clear, I understand the problem that exists and that you wish to >>>> solve, but for the runtime parts I balk at the complexity cost of >>>> solving it. >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>>> Thanks, Richard. >>>>> >>>>> [1] Deopt suspend was something like an async. handshake for >>>>> architectures with register windows, >>>>> ???? where patching the return pc for deoptimization of a compiled >>>>> frame was racy if the owner thread >>>>> ???? was in native code. Instead a "deopt" suspend flag was set on >>>>> which the thread patched its own >>>>> ???? frame upon return from native. So no thread was suspended. It got >>>>> its name only from the name of >>>>> ???? the flags. >>>>> >>>>> [2] Discussion about using handshakes to sync. with the target thread: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK- >> 8227745?focusedCommentId=14306727&page=com.atlassian.jira.plugin.syste >> m.issuetabpanels:comment-tabpanel#comment-14306727 >>>>> >>>>> >>>>> -----Original Message----- >>>>> From: David Holmes >>>>> Sent: Freitag, 13. Dezember 2019 00:56 >>>>> To: Reingruber, Richard ; >>>>> serviceability-dev at openjdk.java.net; >>>>> hotspot-compiler-dev at openjdk.java.net; >>>>> hotspot-runtime-dev at openjdk.java.net >>>>> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better >>>>> Performance in the Presence of JVMTI Agents >>>>> >>>>> Hi Richard, >>>>> >>>>> Some further queries/concerns: >>>>> >>>>> src/hotspot/share/runtime/objectMonitor.cpp >>>>> >>>>> Can you please explain the changes to ObjectMonitor::wait: >>>>> >>>>> !?? _recursions = save????? // restore the old recursion count >>>>> !???????????????? + jt->get_and_reset_relock_count_after_wait(); // >>>>> increased by the deferred relock count >>>>> >>>>> what is the "deferred relock count"? I gather it relates to >>>>> >>>>> "The code was extended to be able to deoptimize objects of a frame that >>>>> is not the top frame and to let another thread than the owning thread do >>>>> it." >>>>> >>>>> which I don't like the sound of at all when it comes to ObjectMonitor >>>>> state. So I'd like to understand in detail exactly what is going on here >>>>> and why.? This is a very intrusive change that seems to badly break >>>>> encapsulation and impacts future changes to ObjectMonitor that are under >>>>> investigation. >>>>> >>>>> --- >>>>> >>>>> src/hotspot/share/runtime/thread.cpp >>>>> >>>>> Can you please explain why JavaThread::wait_for_object_deoptimization >>>>> has to be handcrafted in this way rather than using proper transitions. >>>>> >>>>> We got rid of "deopt suspend" some time ago and it is disturbing to see >>>>> it being added back (effectively). This seems like it may be something >>>>> that handshakes could be used for. >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>> On 12/12/2019 7:02 am, David Holmes wrote: >>>>>> On 12/12/2019 1:07 am, Reingruber, Richard wrote: >>>>>>> Hi David, >>>>>>> >>>>>>> ??? > Most of the details here are in areas I can comment on in detail, >>>>>>> but I >>>>>>> ??? > did take an initial general look at things. >>>>>>> >>>>>>> Thanks for taking the time! >>>>>> >>>>>> Apologies the above should read: >>>>>> >>>>>> "Most of the details here are in areas I *can't* comment on in detail >>>>>> ..." >>>>>> >>>>>> David >>>>>> >>>>>>> ??? > The only thing that jumped out at me is that I think the >>>>>>> ??? > DeoptimizeObjectsALotThread should be a hidden thread. >>>>>>> ??? > >>>>>>> ??? > +? bool is_hidden_from_external_view() const { return true; } >>>>>>> >>>>>>> Yes, it should. Will add the method like above. >>>>>>> >>>>>>> ??? > Also I don't see any testing of the DeoptimizeObjectsALotThread. >>>>>>> Without >>>>>>> ??? > active testing this will just bit-rot. >>>>>>> >>>>>>> DeoptimizeObjectsALot is meant for stress testing with a larger >>>>>>> workload. I will add a minimal test >>>>>>> to keep it fresh. >>>>>>> >>>>>>> ??? > Also on the tests I don't understand your @requires clause: >>>>>>> ??? > >>>>>>> ??? >?? @requires ((vm.compMode != "Xcomp") & vm.compiler2.enabled >> & >>>>>>> ??? > (vm.opt.TieredCompilation != true)) >>>>>>> ??? > >>>>>>> ??? > This seems to require that TieredCompilation is disabled, but >>>>>>> tiered is >>>>>>> ??? > our normal mode of operation. ?? >>>>>>> ??? > >>>>>>> >>>>>>> I removed the clause. I guess I wanted to target the tests towards the >>>>>>> code they are supposed to >>>>>>> test, and it's easier to analyze failures w/o tiered compilation and >>>>>>> with just one compiler thread. >>>>>>> >>>>>>> Additionally I will make use of >>>>>>> compiler.whitebox.CompilerWhiteBoxTest.THRESHOLD in the tests. >>>>>>> >>>>>>> Thanks, >>>>>>> Richard. >>>>>>> >>>>>>> -----Original Message----- >>>>>>> From: David Holmes >>>>>>> Sent: Mittwoch, 11. Dezember 2019 08:03 >>>>>>> To: Reingruber, Richard ; >>>>>>> serviceability-dev at openjdk.java.net; >>>>>>> hotspot-compiler-dev at openjdk.java.net; >>>>>>> hotspot-runtime-dev at openjdk.java.net >>>>>>> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better >>>>>>> Performance in the Presence of JVMTI Agents >>>>>>> >>>>>>> Hi Richard, >>>>>>> >>>>>>> On 11/12/2019 7:45 am, Reingruber, Richard wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> I would like to get reviews please for >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.3/ >>>>>>>> >>>>>>>> Corresponding RFE: >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8227745 >>>>>>>> >>>>>>>> Fixes also https://bugs.openjdk.java.net/browse/JDK-8233915 >>>>>>>> And potentially https://bugs.openjdk.java.net/browse/JDK-8214584 [1] >>>>>>>> >>>>>>>> Vladimir Kozlov kindly put webrev.3 through tier1-8 testing without >>>>>>>> issues (thanks!). In addition the >>>>>>>> change is being tested at SAP since I posted the first RFR some >>>>>>>> months ago. >>>>>>>> >>>>>>>> The intention of this enhancement is to benefit performance wise from >>>>>>>> escape analysis even if JVMTI >>>>>>>> agents request capabilities that allow them to access local variable >>>>>>>> values. E.g. if you start-up >>>>>>>> with -agentlib:jdwp=transport=dt_socket,server=y,suspend=n, then >>>>>>>> escape analysis is disabled right >>>>>>>> from the beginning, well before a debugger attaches -- if ever one >>>>>>>> should do so. With the >>>>>>>> enhancement, escape analysis will remain enabled until and after a >>>>>>>> debugger attaches. EA based >>>>>>>> optimizations are reverted just before an agent acquires the >>>>>>>> reference to an object. In the JBS item >>>>>>>> you'll find more details. >>>>>>> >>>>>>> Most of the details here are in areas I can comment on in detail, but I >>>>>>> did take an initial general look at things. >>>>>>> >>>>>>> The only thing that jumped out at me is that I think the >>>>>>> DeoptimizeObjectsALotThread should be a hidden thread. >>>>>>> >>>>>>> +? bool is_hidden_from_external_view() const { return true; } >>>>>>> >>>>>>> Also I don't see any testing of the DeoptimizeObjectsALotThread. >>>>>>> Without >>>>>>> active testing this will just bit-rot. >>>>>>> >>>>>>> Also on the tests I don't understand your @requires clause: >>>>>>> >>>>>>> ??? @requires ((vm.compMode != "Xcomp") & vm.compiler2.enabled & >>>>>>> (vm.opt.TieredCompilation != true)) >>>>>>> >>>>>>> This seems to require that TieredCompilation is disabled, but tiered is >>>>>>> our normal mode of operation. ?? >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> >>>>>>>> Thanks, >>>>>>>> Richard. >>>>>>>> >>>>>>>> [1] Experimental fix for JDK-8214584 based on JDK-8227745 >>>>>>>> >> http://cr.openjdk.java.net/~rrich/webrevs/2019/8214584/experiment_v1.patc >> h >>>>>>>> >>>>>>>> >>>>>>>> From kevin.walls at oracle.com Mon Mar 2 10:47:16 2020 From: kevin.walls at oracle.com (Kevin Walls) Date: Mon, 2 Mar 2020 10:47:16 +0000 Subject: RFR(S): hs_err elapsed time in seconds is not accurate enough Message-ID: <4a3d7c8e-df9e-51de-77e4-775be1480392@oracle.com> Hi, (s11y and runtime opinions both relevant) A few times in the last month I've really wanted to compare the Events logged in the hs_err file, and the time of the JVM's crash. "elapsed time" in hs_err is only accurate to one second, and has been since before jdk5 was created. The diff below changes the format string and uses the non-rounded time value (I don't see a need to change the other integer arithmetic here), and we can enjoy hs_errs with detail like: ... Time: Mon Mar? 2 09:57:13 2020 UTC elapsed time: 5.490135 seconds (0d 0h 0m 5s) ... Thanks Kevin /jdk/open$ hg diff diff --git a/src/hotspot/share/runtime/os.cpp b/src/hotspot/share/runtime/os.cpp --- a/src/hotspot/share/runtime/os.cpp +++ b/src/hotspot/share/runtime/os.cpp @@ -1016,9 +1016,8 @@ ?? } ?? double t = os::elapsedTime(); -? // NOTE: It tends to crash after a SEGV if we want to printf("%f",...) in -? //?????? Linux. Must be a bug in glibc ? Workaround is to round "t" to int -? //?????? before printf. We lost some precision, but who cares? +? // NOTE: a crash using printf("%f",...) on Linux was historically noted here +? //?????? (before the jdk5 repo was created). ?? int eltime = (int)t;? // elapsed time in seconds ?? // print elapsed time in a human-readable format: @@ -1029,7 +1028,7 @@ ?? int elmins = (eltime - day_secs - hour_secs) / secs_per_min; ?? int minute_secs = elmins * secs_per_min; ?? int elsecs = (eltime - day_secs - hour_secs - minute_secs); -? st->print_cr(" elapsed time: %d seconds (%dd %dh %dm %ds)", eltime, eldays, elhours, elmins, elsecs); +? st->print_cr(" elapsed time: %f seconds (%dd %dh %dm %ds)", t, eldays, elhours, elmins, elsecs); ?} From kevin.walls at oracle.com Mon Mar 2 10:48:13 2020 From: kevin.walls at oracle.com (Kevin Walls) Date: Mon, 2 Mar 2020 10:48:13 +0000 Subject: RFR(S): 8240295: hs_err elapsed time in seconds is not accurate enough In-Reply-To: <4a3d7c8e-df9e-51de-77e4-775be1480392@oracle.com> References: <4a3d7c8e-df9e-51de-77e4-775be1480392@oracle.com> Message-ID: Oops, and with the bug ID in the title and JBS link: https://bugs.openjdk.java.net/browse/JDK-8240295 On 02/03/2020 10:47, Kevin Walls wrote: > Hi, > > (s11y and runtime opinions both relevant) > > A few times in the last month I've really wanted to compare the Events > logged in the hs_err file, and the time of the JVM's crash. > > "elapsed time" in hs_err is only accurate to one second, and has been > since before jdk5 was created. > > The diff below changes the format string and uses the non-rounded time > value (I don't see a need to change the other integer arithmetic > here), and we can enjoy hs_errs with detail like: > > ... > Time: Mon Mar? 2 09:57:13 2020 UTC elapsed time: 5.490135 seconds (0d > 0h 0m 5s) > ... > > Thanks > Kevin > > > /jdk/open$ hg diff > diff --git a/src/hotspot/share/runtime/os.cpp > b/src/hotspot/share/runtime/os.cpp > --- a/src/hotspot/share/runtime/os.cpp > +++ b/src/hotspot/share/runtime/os.cpp > @@ -1016,9 +1016,8 @@ > ?? } > > ?? double t = os::elapsedTime(); > -? // NOTE: It tends to crash after a SEGV if we want to > printf("%f",...) in > -? //?????? Linux. Must be a bug in glibc ? Workaround is to round "t" > to int > -? //?????? before printf. We lost some precision, but who cares? > +? // NOTE: a crash using printf("%f",...) on Linux was historically > noted here > +? //?????? (before the jdk5 repo was created). > ?? int eltime = (int)t;? // elapsed time in seconds > > ?? // print elapsed time in a human-readable format: > @@ -1029,7 +1028,7 @@ > ?? int elmins = (eltime - day_secs - hour_secs) / secs_per_min; > ?? int minute_secs = elmins * secs_per_min; > ?? int elsecs = (eltime - day_secs - hour_secs - minute_secs); > -? st->print_cr(" elapsed time: %d seconds (%dd %dh %dm %ds)", eltime, > eldays, elhours, elmins, elsecs); > +? st->print_cr(" elapsed time: %f seconds (%dd %dh %dm %ds)", t, > eldays, elhours, elmins, elsecs); > ?} > > From ralf.schmelter at sap.com Mon Mar 2 13:24:22 2020 From: ralf.schmelter at sap.com (Schmelter, Ralf) Date: Mon, 2 Mar 2020 13:24:22 +0000 Subject: RFR: 8240197: Cannot start JVM when $JAVA_HOME includes CJK characters In-Reply-To: References: <6ec2f649-8999-fdd5-c2df-465925c3663b@oss.nttdata.com> Message-ID: Hi Yasumasa, thanks for your patch. I have some remarks: 1. wcstombs_s creates UTF-16, as does MultiByteToWideChar. That wasn't the problem. The problem was that wcstombs_s uses the current locale, which is "C" initially. One is normally expected to set the locale to the system locale via setlocale(LC_ALL, ""), as it is done in ParseLocale() on Unix. One could fix the original problem by setting the locale (probably not a good idea, since this changes the locale of the whole process). Or one could specify the correct locale to wcstombs like: static _locale_t loc = _create_locale(LC_ALL, ""); err = ::_mbstowcs_s_l(&converted_chars, path_start, buf_len + 1, buf, buf_len, loc); Both fix the problem. But I agree, using the MultiByteToWideChar method is better, since you don't have to create the locale. There are some other places in the code where mbstowcs is used on Windows (canonicalize_md.c for example). They should probably fixed in the same way. 2.I don't think that using strlen(buf) as the number of wchars to allocate for the converted string is really wrong. There are no code pages supported by Java which would map a single byte to a Unicode character > U+ffff. But since one can get the always correct value via MultiByteToWideChar, this is futureproof and obviously right. 3. I would not use the CP_THREAD_ACP code page in MultiByteToWideChar and instead use CP_ACP, because the latter always uses the system locale. And this is the locale used to convert the arguments to bytes in the first place. 4. You replaced the malloc/memcpy for the string which will be used to call native_path() with a strdup. This leads to a memory overwrite, when a path like "C:" is used, since native_path() will convert it to "C:.". And the debug version of the VM will detect this, since the tail guard will be partially overwritten. You can check this by running: make run-test TEST=gtest:os_windows 5. Using WCHAR and LPWSTR seems odd to me. Nobody uses LPCSTR or CHAR for char* and char. But if you think WCHAR is better, I would adjust the return value of the method and the functions which use it too. 6. When you handle a relative path, you now only allocate enough space to hold the relative path, not the generated absolute path. Since a relative path is likely to be smaller than the absolute path, the _wfullpath call later will likely fail. That was the reason we used JVM_MAXPATHLEN for relative paths. You could fix this like in this patch http://cr.openjdk.java.net/~rschmelter/webrevs/tmp/unicode_convert.patch Best regards, Ralf From robbin.ehn at oracle.com Mon Mar 2 13:32:37 2020 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Mon, 2 Mar 2020 14:32:37 +0100 Subject: RFR(L) 8153224 Monitor deflation prolong safepoints (CR10/v2.10/13-for-jdk15) In-Reply-To: References: <62729044-8a22-0e20-0eda-04d47c9ea23c@oracle.com> <313e51c8-b672-bb1c-577a-49868f09e6c1@oracle.com> <1fd54b23-bd35-9b8f-e6f3-6000440d8770@oracle.com> <3fb1a2b5-5aad-eab3-09ba-6f64a2242d30@oracle.com> <38e2d441-11c9-8342-37d5-8030dd06f2f4@oracle.com> <2a8976f7-37e0-03b9-3099-e07464e46512@oracle.com> <5681d640-08c8-3433-0f85-3f23eea69e87@oracle.com> Message-ID: <5a7a7a36-a93c-aa57-93e7-bbd4c58f4d73@oracle.com> Hi Dan, thanks sticking with it and addressing the comments! I think you got all mine, minus those we decided should be handled as follow-ups. I have nothing more, looks good, thanks! /Robbin On 2/26/20 11:22 PM, Daniel D. Daugherty wrote: > Greetings, > > I have made changes to the Async Monitor Deflation code in response to > the CR9/v2.09/12-for-jdk14 code review cycle. Thanks to Robbin and Erik O. > for their comments in this round! > > With the extraction and push of {8235931,8236035,8235795} to JDK15, the > Async Monitor Deflation code is back to "just" async deflation changes! > > I have attached the change list from CR9 to CR10 instead of putting it in > the body of this email. I've also added a link to the CR9-to-CR10-changes > file to the webrevs so it should be easy to find. > > Main bug URL: > > ??? JDK-8153224 Monitor deflation prolong safepoints > ??? https://bugs.openjdk.java.net/browse/JDK-8153224 > > The project is currently baselined on jdk-15+11. > > Here's the full webrev URL for those folks that want to see all of the > current Async Monitor Deflation code in one go (v2.10 full): > > http://cr.openjdk.java.net/~dcubed/8153224-webrev/13-for-jdk15+11.v2.10.full/ > > Some folks might want to see just what has changed since the last review > cycle so here's a webrev for that (v2.10 inc): > > http://cr.openjdk.java.net/~dcubed/8153224-webrev/13-for-jdk15+11.v2.10.inc/ > > Since we backed out the HandshakeAfterDeflateIdleMonitors option and the > C2 ref_count changes and updated the copyright years, the "inc" webrev has > a bit more noise in it than usual. Sorry about that! > > The OpenJDK wiki has been updated for this round of changes: > > https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation > > The jdk-15+11 based v2.10 version of the patch has been thru Mach5 tier[1-7] > testing on Oracle's usual set of platforms. Mach5 tier8 is still running. > I'm running the v2.10 patch through my usual set of stress testing on > Linux-X64 and macOSX. > > I'm planning to do a SPECjbb2015 round on the CR10/v2.20/13-for-jdk15 bits. > > Thanks, in advance, for any questions, comments or suggestions. > > Dan > > > On 2/4/20 9:41 AM, Daniel D. Daugherty wrote: >> Greetings, >> >> This project is no longer targeted to JDK14 so this is NOT an urgent code >> review request. >> >> I've extracted the following three fixes from the Async Monitor Deflation >> project code: >> >> ? ? JDK-8235931 add OM_CACHE_LINE_SIZE and use smaller size on SPARCv9 and X64 >> ? ? https://bugs.openjdk.java.net/browse/JDK-8235931 >> >> ? ? JDK-8236035 refactor ObjectMonitor::set_owner() and _owner field setting >> ? ? https://bugs.openjdk.java.net/browse/JDK-8236035 >> >> ? ? JDK-8235795 replace monitor list mux{Acquire,Release}(&gListLock) with >> spin locks >> ? ? https://bugs.openjdk.java.net/browse/JDK-8235795 >> >> Each of these has been reviewed separately and will be pushed to JDK15 >> in the near future (possibly by the end of this week). Of course, there >> were improvements during these review cycles and the purpose of this >> e-mail is to provided updated webrevs for this fix (CR9/v2.09/12-for-jdk14) >> within the revised context provided by {8235931, 8236035, 8235795}. >> >> Main bug URL: >> >> ??? JDK-8153224 Monitor deflation prolong safepoints >> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >> >> The project is currently baselined on jdk-14+34. >> >> Here's the full webrev URL for those folks that want to see all of the >> current Async Monitor Deflation code along with {8235931, 8236035, 8235795} >> in one go (v2.09b full): >> >> http://cr.openjdk.java.net/~dcubed/8153224-webrev/12-for-jdk14.v2.09b.full/ >> >> Compare the open.patch file in 12-for-jdk14.v2.09.full and >> 12-for-jdk14.v2.09b.full >> using your favorite file comparison/merge tool to see how Async Monitor Deflation >> evolved due to {8235931, 8236035, 8235795}. >> >> Some folks might want to see just the Async Monitor Deflation code on top of >> {8235931, 8236035, 8235795} so here's a webrev for that (v2.09b inc): >> >> http://cr.openjdk.java.net/~dcubed/8153224-webrev/12-for-jdk14.v2.09b.inc/ >> >> These webrevs have gone thru several Mach5 Tier[1-8] runs along with >> my usual stress testing and SPECjbb2015 testing and there aren't any >> surprises relative to CR9/v2.09/12-for-jdk14. >> >> Thanks, in advance, for any questions, comments or suggestions. >> >> Dan >> >> >> On 12/11/19 3:41 PM, Daniel D. Daugherty wrote: >>> Greetings, >>> >>> I have made changes to the Async Monitor Deflation code in response to >>> the CR8/v2.08/11-for-jdk14 code review cycle. Thanks to David H., Robbin >>> and Erik O. for their comments! >>> >>> This project is no longer targeted to JDK14 so this is NOT an urgent code >>> review request. The primary purpose of this webrev is simply to close the >>> CR8/v2.08/11-for-jdk14 code review loop and to let folks see how I resolved >>> the code review comments from that round. >>> >>> Most of the comments in the CR8/v2.08/11-for-jdk14 code review cycle were >>> on the monitor list changes so I'm going to take a look at extracting those >>> changes into a standalone patch. Switching from Thread::muxAcquire(&gListLock) >>> and Thread::muxRelease(&gListLock) to finer grained internal spin locks needs >>> to be thoroughly reviewed and the best way to do that is separately from the >>> Async Monitor Deflation changes. Thanks to Coleen for suggesting doing this >>> extraction earlier. >>> >>> I have attached the change list from CR8 to CR9 instead of putting it in >>> the body of this email. I've also added a link to the CR8-to-CR9-changes >>> file to the webrevs so it should be easy to find. >>> >>> Main bug URL: >>> >>> ??? JDK-8153224 Monitor deflation prolong safepoints >>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>> >>> The project is currently baselined on jdk-14+26. >>> >>> Here's the full webrev URL for those folks that want to see all of the >>> current Async Monitor Deflation code in one go (v2.09 full): >>> >>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/12-for-jdk14.v2.09.full/ >>> >>> Some folks might want to see just what has changed since the last review >>> cycle so here's a webrev for that (v2.09 inc): >>> >>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/12-for-jdk14.v2.09.inc/ >>> >>> The OpenJDK wiki has NOT yet been updated for this round of changes: >>> >>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>> >>> The jdk-14+26 based v2.09 version of the patch has been thru Mach5 tier[1-7] >>> testing on Oracle's usual set of platforms. Mach5 tier8 is still running. >>> A slightly older version of the v2.09 patch has also been through my usual >>> set of stress testing on Linux-X64 and macOSX with the addition of Robbin's >>> "MoCrazy 1024" test running in parallel on Linux-X64 with the other tests in >>> my lab. The "MoCrazy 1024" has been going for > 5 days and 6700+ iterations >>> without any failures. >>> >>> I'm planning to do a SPECjbb2015 round on the CR9/v2.09/12-for-jdk14 bits. >>> >>> Thanks, in advance, for any questions, comments or suggestions. >>> >>> Dan >>> >>> >>> On 11/4/19 4:03 PM, Daniel D. Daugherty wrote: >>>> Greetings, >>>> >>>> I have made changes to the Async Monitor Deflation code in response to >>>> the CR7/v2.07/10-for-jdk14 code review cycle. Thanks to David H., Robbin >>>> and Erik O. for their comments! >>>> >>>> JDK14 Rampdown phase one is coming on Dec. 12, 2019 and the Async Monitor >>>> Deflation project needs to push before Nov. 12, 2019 in order to allow >>>> for sufficient bake time for such a big change. Nov. 12 is _next_ Tuesday >>>> so we have 8 days from today to finish this code review cycle and push >>>> this code for JDK14. >>>> >>>> Carsten and Roman! Time for you guys to chime in again on the code reviews. >>>> >>>> I have attached the change list from CR7 to CR8 instead of putting it in >>>> the body of this email. I've also added a link to the CR7-to-CR8-changes >>>> file to the webrevs so it should be easy to find. >>>> >>>> Main bug URL: >>>> >>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>>> >>>> The project is currently baselined on jdk-14+21. >>>> >>>> Here's the full webrev URL for those folks that want to see all of the >>>> current Async Monitor Deflation code in one go (v2.08 full): >>>> >>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/11-for-jdk14.v2.08.full >>>> >>>> Some folks might want to see just what has changed since the last review >>>> cycle so here's a webrev for that (v2.08 inc): >>>> >>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/11-for-jdk14.v2.08.inc/ >>>> >>>> The OpenJDK wiki did not need any changes for this round: >>>> >>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>> >>>> The jdk-14+21 based v2.08 version of the patch has been thru Mach5 tier[1-8] >>>> testing on Oracle's usual set of platforms. It has also been through my usual >>>> set of stress testing on Linux-X64, macOSX and Solaris-X64 with the addition >>>> of Robbin's "MoCrazy 1024" test running in parallel with the other tests in >>>> my lab. Some testing is still running, but so far there are no new regressions. >>>> >>>> I have not yet done a SPECjbb2015 round on the CR8/v2.08/11-for-jdk14 bits. >>>> >>>> Thanks, in advance, for any questions, comments or suggestions. >>>> >>>> Dan >>>> >>>> >>>> On 10/17/19 5:50 PM, Daniel D. Daugherty wrote: >>>>> Greetings, >>>>> >>>>> The Async Monitor Deflation project is reaching the end game. I have no >>>>> changes planned for the project at this time so all that is left is code >>>>> review and any changes that results from those reviews. >>>>> >>>>> Carsten and Roman! Time for you guys to chime in again on the code reviews. >>>>> >>>>> I have attached the list of fixes from CR6 to CR7 instead of putting it >>>>> in the main body of this email. >>>>> >>>>> Main bug URL: >>>>> >>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>> >>>>> The project is currently baselined on jdk-14+19. >>>>> >>>>> Here's the full webrev URL for those folks that want to see all of the >>>>> current Async Monitor Deflation code in one go (v2.07 full): >>>>> >>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/10-for-jdk14.v2.07.full >>>>> >>>>> Some folks might want to see just what has changed since the last review >>>>> cycle so here's a webrev for that (v2.07 inc): >>>>> >>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/10-for-jdk14.v2.07.inc/ >>>>> >>>>> The OpenJDK wiki has been updated to match the CR7/v2.07/10-for-jdk14 changes: >>>>> >>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>> >>>>> The jdk-14+18 based v2.07 version of the patch has been thru Mach5 tier[1-8] >>>>> testing on Oracle's usual set of platforms. It has also been through my usual >>>>> set of stress testing on Linux-X64, macOSX and Solaris-X64 with the addition >>>>> of Robbin's "MoCrazy 1024" test running in parallel with the other tests in >>>>> my lab. >>>>> >>>>> The jdk-14+19 based v2.07 version of the patch has been thru Mach5 tier[1-3] >>>>> test on Oracle's usual set of platforms. Mach5 tier[4-8] are in process. >>>>> >>>>> I did another round of SPECjbb2015 testing in Oracle's Aurora Performance lab >>>>> using using their tuned SPECjbb2015 Linux-X64 G1 configs: >>>>> >>>>> ??? - "base" is jdk-14+18 >>>>> ??? - "v2.07" is the latest version and includes C2 inc_om_ref_count() support >>>>> ????? on LP64 X64 and the new HandshakeAfterDeflateIdleMonitors option >>>>> ? ? - "off" is with -XX:-AsyncDeflateIdleMonitors specified >>>>> ? ? - "handshake" is with -XX:+HandshakeAfterDeflateIdleMonitors specified >>>>> >>>>> ???????? hbIR?????????? hbIR >>>>> ??? (max attempted)? (settled)? max-jOPS? critical-jOPS runtime >>>>> ??? ---------------? ---------? --------? ------------- ------- >>>>> ?????????? 34282.00?? 30635.90? 28831.30?????? 20969.20 3841.30 base >>>>> ?????????? 34282.00?? 30973.00? 29345.80?????? 21025.20 3964.10 v2.07 >>>>> ?????????? 34282.00?? 31105.60? 29174.30?????? 21074.00 3931.30 >>>>> v2.07_handshake >>>>> ?????????? 34282.00?? 30789.70? 27151.60?????? 19839.10 3850.20 v2.07_off >>>>> >>>>> ??? - The Aurora Perf comparison tool reports: >>>>> >>>>> ??????? Comparison????????????? max-jOPS critical-jOPS >>>>> ??????? ----------------------? -------------------- -------------------- >>>>> ??????? base vs 2.07??????????? +1.78% (s, p=0.000)?? +0.27% (ns, p=0.790) >>>>> ??????? base vs 2.07_handshake? +1.19% (s, p=0.007)?? +0.58% (ns, p=0.536) >>>>> ??????? base vs 2.07_off??????? -5.83% (ns, p=0.394)? -5.39% (ns, p=0.347) >>>>> >>>>> ??????? (s) - significant? (ns) - not-significant >>>>> >>>>> ??? - For historical comparison, the Aurora Perf comparision tool >>>>> ??????? reported for v2.06 with a baseline of jdk-13+31: >>>>> >>>>> ??????? Comparison????????????? max-jOPS critical-jOPS >>>>> ??????? ----------------------? -------------------- -------------------- >>>>> ??????? base vs 2.06??????????? -0.32% (ns, p=0.345)? +0.71% (ns, p=0.646) >>>>> ??????? base vs 2.06_off??????? +0.49% (ns, p=0.292)? -1.21% (ns, p=0.481) >>>>> >>>>> ??????? (s) - significant? (ns) - not-significant >>>>> >>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>> >>>>> Dan >>>>> >>>>> >>>>> On 8/28/19 5:02 PM, Daniel D. Daugherty wrote: >>>>>> Greetings, >>>>>> >>>>>> The Async Monitor Deflation project has rebased to JDK14 so it's time >>>>>> for our first code review in that new context!! >>>>>> >>>>>> I've been focused on changing the monitor list management code to be >>>>>> lock-free in order to make SPECjbb2015 happier. Of course with a change >>>>>> like that, it takes a while to chase down all the new and wonderful >>>>>> races. At this point, I have the code back to the same stability that >>>>>> I had with CR5/v2.05/8-for-jdk13. >>>>>> >>>>>> To lay the ground work for this round of review, I pushed the following >>>>>> two fixes to jdk/jdk earlier today: >>>>>> >>>>>> ??? JDK-8230184 rename, whitespace, indent and comments changes in >>>>>> preparation >>>>>> ? ? ??????????? for lock free Monitor lists >>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8230184 >>>>>> >>>>>> ??? JDK-8230317 serviceability/sa/ClhsdbPrintStatics.java fails after 8230184 >>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8230317 >>>>>> >>>>>> I have attached the list of fixes from CR5 to CR6 instead of putting >>>>>> in the main body of this email. >>>>>> >>>>>> Main bug URL: >>>>>> >>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>> >>>>>> The project is currently baselined on jdk-14+11 plus the fixes for >>>>>> JDK-8230184 and JDK-8230317. >>>>>> >>>>>> Here's the full webrev URL for those folks that want to see all of the >>>>>> current Async Monitor Deflation code in one go (v2.06 full): >>>>>> >>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.06.full/ >>>>>> >>>>>> >>>>>> The primary focus of this review cycle is on the lock-free Monitor List >>>>>> management changes so here's a webrev for just that patch (v2.06c): >>>>>> >>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.06c.inc/ >>>>>> >>>>>> The secondary focus of this review cycle is on the bug fixes that have >>>>>> been made since CR5/v2.05/8-for-jdk13 so here's a webrev for just that >>>>>> patch (v2.06b): >>>>>> >>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.06b.inc/ >>>>>> >>>>>> The third and final bucket for this review cycle is the rename, whitespace, >>>>>> indent and comments changes made in preparation for lock free Monitor list >>>>>> management. Almost all of that was extracted into JDK-8230184 for the >>>>>> baseline so this bucket now has just a few comment changes relative to >>>>>> CR5/v2.05/8-for-jdk13. Here's a webrev for the remainder (v2.06a): >>>>>> >>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.06a.inc/ >>>>>> >>>>>> >>>>>> Some folks might want to see just what has changed since the last review >>>>>> cycle so here's a webrev for that (v2.06 inc): >>>>>> >>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.06.inc/ >>>>>> >>>>>> >>>>>> Last, but not least, some folks might want to see the code before the >>>>>> addition of lock-free Monitor List management so here's a webrev for >>>>>> that (v2.00 -> v2.05): >>>>>> >>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.05.inc/ >>>>>> >>>>>> The OpenJDK wiki will need minor updates to match the CR6 changes: >>>>>> >>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>> >>>>>> but that should only be changes to describe per-thread list async monitor >>>>>> deflation being done by the ServiceThread. >>>>>> >>>>>> (I did update the OpenJDK wiki for the CR5 changes back on 2019.08.14) >>>>>> >>>>>> This version of the patch has been thru Mach5 tier[1-8] testing on >>>>>> Oracle's usual set of platforms. It has also been through my usual set >>>>>> of stress testing on Linux-X64, macOSX and Solaris-X64. >>>>>> >>>>>> I did a bunch of SPECjbb2015 testing in Oracle's Aurora Performance lab >>>>>> using using their tuned SPECjbb2015 Linux-X64 G1 configs. This was using >>>>>> this patch baselined on jdk-13+31 (for stability): >>>>>> >>>>>> ????????? hbIR?????????? hbIR >>>>>> ???? (max attempted)? (settled)? max-jOPS? critical-jOPS runtime >>>>>> ???? ---------------? ---------? --------? ------------- ------- >>>>>> ??????????? 34282.00?? 28837.20? 27905.20?????? 19817.40 3658.10 base >>>>>> ??????????? 34965.70?? 29798.80? 27814.90?????? 19959.00 3514.60 v2.06d >>>>>> ??????????? 34282.00?? 29100.70? 28042.50?????? 19577.00 3701.90 v2.06d_off >>>>>> ??????????? 34282.00?? 29218.50? 27562.80?????? 19397.30 3657.60 >>>>>> v2.06d_ocache >>>>>> ??????????? 34965.70?? 29838.30? 26512.40?????? 19170.60 3569.90 v2.05 >>>>>> ??????????? 34282.00?? 28926.10? 27734.00?????? 19835.10 3588.40 v2.05_off >>>>>> >>>>>> The "off" configs are with -XX:-AsyncDeflateIdleMonitors specified and >>>>>> the "ocache" config is with 128 byte cache line sizes instead of 64 byte >>>>>> cache lines sizes. "v2.06d" is the last set of changes that I made before >>>>>> those changes were distributed into the "v2.06a", "v2.06b" and "v2.06c" >>>>>> buckets for this review recycle. >>>>>> >>>>>> >>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>> >>>>>> Dan >>>>>> >>>>>> >>>>>> On 7/11/19 3:49 PM, Daniel D. Daugherty wrote: >>>>>>> Greetings, >>>>>>> >>>>>>> I've been focused on chasing down and fixing the rare test failures >>>>>>> that only pop up rarely. So this round is primarily fixes for races >>>>>>> with a few additional fixes that came from Karen's review of CR4. >>>>>>> Thanks Karen! >>>>>>> >>>>>>> I have attached the list of fixes from CR4 to CR5 instead of putting >>>>>>> in the main body of this email. >>>>>>> >>>>>>> Main bug URL: >>>>>>> >>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>> >>>>>>> The project is currently baselined on jdk-13+29. This will likely be >>>>>>> the last JDK13 baseline for this project and I'll roll to the JDK14 >>>>>>> (jdk/jdk) repo soon... >>>>>>> >>>>>>> Here's the full webrev URL: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/8-for-jdk13.full/ >>>>>>> >>>>>>> Here's the incremental webrev URL: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/8-for-jdk13.inc/ >>>>>>> >>>>>>> I have not yet checked the OpenJDK wiki to see if it needs any updates >>>>>>> to match the CR5 changes: >>>>>>> >>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>> >>>>>>> (I did update the OpenJDK wiki for the CR4 changes back on 2019.06.26) >>>>>>> >>>>>>> This version of the patch has been thru Mach5 tier[1-3] testing on >>>>>>> Oracle's usual set of platforms. Mach5 tier[4-6] is running now and >>>>>>> Mach5 tier[78] will follow. I'll kick off the usual stress testing >>>>>>> on Linux-X64, macOSX and Solaris-X64 as those machines become available. >>>>>>> Since I haven't made any performance changes in this round, I'll only >>>>>>> be running SPECjbb2015 to gather the latest monitorinflation logs. >>>>>>> >>>>>>> Next up: >>>>>>> >>>>>>> - We're still seeing 4-5% lower performance with SPECjbb2015 on >>>>>>> ? Linux-X64 and we've determined that some of that comes from >>>>>>> ? contention on the gListLock. So I'm going to investigate removing >>>>>>> ? the gListLock. Yes, another lock free set of changes is coming! >>>>>>> - Of course, going lock free often causes new races and new failures >>>>>>> ? so that's a good reason for make those changes isolated in their >>>>>>> ? own round (and not holding up CR5/v2.05/8-for-jdk13 anymore). >>>>>>> - I finally have a potential fix for the Win* failure with >>>>>>> ? ? gc/g1/humongousObjects/TestHumongousClassLoader.java >>>>>>> ? but I haven't run it through Mach5 yet so it'll be in the next round. >>>>>>> - Some RTM tests were recently re-enabled in Mach5 and I'm seeing some >>>>>>> ? monitor related failures there. I suspect that I need to go take a >>>>>>> ? look at the C2 RTM macro assembler code and look for things that might >>>>>>> ? conflict if Async Monitor Deflation. If you're interested in that kind >>>>>>> ? of issue, then see the macroAssembler_x86.cpp sanity check that I >>>>>>> ? added in this round! >>>>>>> >>>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>>> >>>>>>> Dan >>>>>>> >>>>>>> >>>>>>> On 5/26/19 8:30 PM, Daniel D. Daugherty wrote: >>>>>>>> Greetings, >>>>>>>> >>>>>>>> I have a fix for an issue that came up during performance testing. >>>>>>>> Many thanks to Robbin for diagnosing the issue in his SPECjbb2015 >>>>>>>> experiments. >>>>>>>> >>>>>>>> Here's the list of changes from CR3 to CR4. The list is a bit >>>>>>>> verbose due to the complexity of the issue, but the changes >>>>>>>> themselves are not that big. >>>>>>>> >>>>>>>> Functional: >>>>>>>> ? - Change SafepointSynchronize::is_cleanup_needed() from calling >>>>>>>> ??? ObjectSynchronizer::is_cleanup_needed() to calling >>>>>>>> ObjectSynchronizer::is_safepoint_deflation_needed(): >>>>>>>> ??? - is_safepoint_deflation_needed() returns the result of >>>>>>>> ????? monitors_used_above_threshold() for safepoint based >>>>>>>> ????? monitor deflation (!AsyncDeflateIdleMonitors). >>>>>>>> ??? - For AsyncDeflateIdleMonitors, it only returns true if >>>>>>>> ????? there is a special deflation request, e.g., System.gc() >>>>>>>> ????? - This solves a bug where there are a bunch of Cleanup >>>>>>>> ??????? safepoints that simply request async deflation which >>>>>>>> ??????? keeps the async JavaThreads from making progress on >>>>>>>> ??????? their async deflation work. >>>>>>>> ? - Add AsyncDeflationInterval diagnostic option. Description: >>>>>>>> ????? Async deflate idle monitors every so many milliseconds when >>>>>>>> ????? MonitorUsedDeflationThreshold is exceeded (0 is off). >>>>>>>> ? - Replace ObjectSynchronizer::gOmShouldDeflateIdleMonitors() with >>>>>>>> ??? ObjectSynchronizer::is_async_deflation_needed(): >>>>>>>> ??? - is_async_deflation_needed() returns true when >>>>>>>> ????? is_async_cleanup_requested() is true or when >>>>>>>> ????? monitors_used_above_threshold() is true (but no more often than >>>>>>>> ????? AsyncDeflationInterval). >>>>>>>> ??? - if AsyncDeflateIdleMonitors Service_lock->wait() now waits for >>>>>>>> ????? at most GuaranteedSafepointInterval millis: >>>>>>>> ????? - This allows is_async_deflation_needed() to be checked at >>>>>>>> ??????? the same interval as GuaranteedSafepointInterval. >>>>>>>> ??????? (default is 1000 millis/1 second) >>>>>>>> ????? - Once is_async_deflation_needed() has returned true, it >>>>>>>> ??????? generally cannot return true for AsyncDeflationInterval. >>>>>>>> ??????? This is to prevent async deflation from swamping the >>>>>>>> ??????? ServiceThread. >>>>>>>> ? - The ServiceThread still handles async deflation of the global >>>>>>>> ??? in-use list and now it also marks JavaThreads for async deflation >>>>>>>> ??? of their in-use lists. >>>>>>>> ??? - The ServiceThread will check for async deflation work every >>>>>>>> ????? GuaranteedSafepointInterval. >>>>>>>> ??? - A safepoint can still cause the ServiceThread to check for >>>>>>>> ????? async deflation work via is_async_deflation_requested. >>>>>>>> ? - Refactor code from ObjectSynchronizer::is_cleanup_needed() into >>>>>>>> ??? monitors_used_above_threshold() and remove is_cleanup_needed(). >>>>>>>> ? - In addition to System.gc(), the VM_Exit VM op and the final >>>>>>>> ??? VMThread safepoint now set the is_special_deflation_requested >>>>>>>> ??? flag to reduce the in-use monitor population that is reported by >>>>>>>> ??? ObjectSynchronizer::log_in_use_monitor_details() at VM exit. >>>>>>>> >>>>>>>> Test update: >>>>>>>> ? - test/hotspot/gtest/oops/test_markOop.cpp is updated to work with >>>>>>>> ??? AsyncDeflateIdleMonitors. >>>>>>>> >>>>>>>> Collateral: >>>>>>>> ? - Add/clarify/update some logging messages. >>>>>>>> >>>>>>>> Cleanup: >>>>>>>> ? - Updated comments based on Karen's code review. >>>>>>>> ? - Change 'special cleanup' -> 'special deflation' and >>>>>>>> ??? 'async cleanup' -> 'async deflation'. >>>>>>>> ??? - comment and function name changes >>>>>>>> ? - Clarify MonitorUsedDeflationThreshold description; >>>>>>>> >>>>>>>> >>>>>>>> Main bug URL: >>>>>>>> >>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>> >>>>>>>> The project is currently baselined on jdk-13+22. >>>>>>>> >>>>>>>> Here's the full webrev URL: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/7-for-jdk13.full/ >>>>>>>> >>>>>>>> Here's the incremental webrev URL: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/7-for-jdk13.inc/ >>>>>>>> >>>>>>>> I have not updated the OpenJDK wiki to reflect the CR4 changes: >>>>>>>> >>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>> >>>>>>>> The wiki doesn't say a whole lot about the async deflation invocation >>>>>>>> mechanism so I have to figure out how to add that content. >>>>>>>> >>>>>>>> This version of the patch has been thru Mach5 tier[1-8] testing on >>>>>>>> Oracle's usual set of platforms. My Solaris-X64 stress kit run is >>>>>>>> running now. Kitchensink8H on product, fastdebug, and slowdebug bits >>>>>>>> are running on Linux-X64, MacOSX and Solaris-X64. I still have to run >>>>>>>> my stress kit on Linux-X64. I still have to run the SPECjbb2015 >>>>>>>> baseline and CR4 runs on Linux-X64, MacOSX and Solaris-X64. >>>>>>>> >>>>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>>>> >>>>>>>> Dan >>>>>>>> >>>>>>>> On 5/6/19 11:52 AM, Daniel D. Daugherty wrote: >>>>>>>>> Greetings, >>>>>>>>> >>>>>>>>> I had some discussions with Karen about a race that was in the >>>>>>>>> ObjectMonitor::enter() code in CR2/v2.02/5-for-jdk13. This race was >>>>>>>>> theoretical and I had no test failures due to it. The fix is pretty >>>>>>>>> simple: remove the special case code for async deflation in the >>>>>>>>> ObjectMonitor::enter() function and rely solely on the ref_count >>>>>>>>> for ObjectMonitor::enter() protection. >>>>>>>>> >>>>>>>>> During those discussions Karen also floated the idea of using the >>>>>>>>> ref_count field instead of the contentions field for the Async >>>>>>>>> Monitor Deflation protocol. I decided to go ahead and code up that >>>>>>>>> change and I have run it through the usual stress and Mach5 testing >>>>>>>>> with no issues. It's also known as v2.03 (for those for with the >>>>>>>>> patches) and as webrev/6-for-jdk13 (for those with webrev URLs). >>>>>>>>> Sorry for all the names... >>>>>>>>> >>>>>>>>> Main bug URL: >>>>>>>>> >>>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>>> >>>>>>>>> The project is currently baselined on jdk-13+18. >>>>>>>>> >>>>>>>>> Here's the full webrev URL: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/6-for-jdk13.full/ >>>>>>>>> >>>>>>>>> Here's the incremental webrev URL: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/6-for-jdk13.inc/ >>>>>>>>> >>>>>>>>> I have also updated the OpenJDK wiki to reflect the CR3 changes: >>>>>>>>> >>>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>>> >>>>>>>>> This version of the patch has been thru Mach5 tier[1-8] testing on >>>>>>>>> Oracle's usual set of platforms. My Solaris-X64 stress kit run had >>>>>>>>> no issues. Kitchensink8H on product, fastdebug, and slowdebug bits >>>>>>>>> had no failures on Linux-X64; MacOSX fastdebug and slowdebug and >>>>>>>>> Solaris-X64 release had the usual "Too large time diff" complaints. >>>>>>>>> 12 hour Inflate2 runs on product, fastdebug and slowdebug bits on >>>>>>>>> Linux-X64, MacOSX and Solaris-X64 had no failures. My Linux-X64 >>>>>>>>> stress kit is running right now. >>>>>>>>> >>>>>>>>> I've done the SPECjbb2015 baseline and CR3 runs. I need to gather >>>>>>>>> the results and analyze them. >>>>>>>>> >>>>>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>>>>> >>>>>>>>> Dan >>>>>>>>> >>>>>>>>> >>>>>>>>> On 4/25/19 12:38 PM, Daniel D. Daugherty wrote: >>>>>>>>>> Greetings, >>>>>>>>>> >>>>>>>>>> I have a small but important bug fix for the Async Monitor Deflation >>>>>>>>>> project ready to go. It's also known as v2.02 (for those for with the >>>>>>>>>> patches) and as webrev/5-for-jdk13 (for those with webrev URLs). Sorry >>>>>>>>>> for all the names... >>>>>>>>>> >>>>>>>>>> JDK-8222295 was pushed to jdk/jdk two days ago so that baseline patch >>>>>>>>>> is out of our hair. >>>>>>>>>> >>>>>>>>>> Main bug URL: >>>>>>>>>> >>>>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>>>> >>>>>>>>>> The project is currently baselined on jdk-13+17. >>>>>>>>>> >>>>>>>>>> Here's the full webrev URL: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/5-for-jdk13.full/ >>>>>>>>>> >>>>>>>>>> Here's the incremental webrev URL (JDK-8153224): >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/5-for-jdk13.inc/ >>>>>>>>>> >>>>>>>>>> I still have to update the OpenJDK wiki to reflect the CR2 changes: >>>>>>>>>> >>>>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>>>> >>>>>>>>>> This version of the patch has been thru Mach5 tier[1-6] testing on >>>>>>>>>> Oracle's usual set of platforms. Mach5 tier[7-8] is running now. >>>>>>>>>> My stress kit is running on Solaris-X64 now. Kitchensink8H is running >>>>>>>>>> now on product, fastdebug, and slowdebug bits on Linux-X64, MacOSX >>>>>>>>>> and Solaris-X64. 12 hour Inflate2 runs are running now on product, >>>>>>>>>> fastdebug and slowdebug bits on Linux-X64, MacOSX and Solaris-X64. >>>>>>>>>> I'll start my my stress kit on Linux-X64 sometime on Sunday (after >>>>>>>>>> my jdk-13+18 stress run is done). >>>>>>>>>> >>>>>>>>>> I'll do SPECjbb2015 baseline and CR2 runs after all the stress >>>>>>>>>> testing is done. >>>>>>>>>> >>>>>>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>>>>>> >>>>>>>>>> Dan >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 4/19/19 11:58 AM, Daniel D. Daugherty wrote: >>>>>>>>>>> Greetings, >>>>>>>>>>> >>>>>>>>>>> I finally have CR1 for the Async Monitor Deflation project ready to >>>>>>>>>>> go. It's also known as v2.01 (for those for with the patches) and as >>>>>>>>>>> webrev/4-for-jdk13 (for those with webrev URLs). Sorry for all the >>>>>>>>>>> names... >>>>>>>>>>> >>>>>>>>>>> Main bug URL: >>>>>>>>>>> >>>>>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>>>>> >>>>>>>>>>> Baseline bug fixes URL: >>>>>>>>>>> >>>>>>>>>>> ??? JDK-8222295 more baseline cleanups from Async Monitor Deflation >>>>>>>>>>> project >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8222295 >>>>>>>>>>> >>>>>>>>>>> The project is currently baselined on jdk-13+15. >>>>>>>>>>> >>>>>>>>>>> Here's the webrev for the latest baseline changes (JDK-8222295): >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/4-for-jdk13.8222295 >>>>>>>>>>> >>>>>>>>>>> Here's the full webrev URL (JDK-8153224 only): >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/4-for-jdk13.full/ >>>>>>>>>>> >>>>>>>>>>> Here's the incremental webrev URL (JDK-8153224): >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/4-for-jdk13.inc/ >>>>>>>>>>> >>>>>>>>>>> So I'm looking for reviews for both JDK-8222295 and the latest version >>>>>>>>>>> of JDK-8153224... >>>>>>>>>>> >>>>>>>>>>> I still have to update the OpenJDK wiki to reflect the CR changes: >>>>>>>>>>> >>>>>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>>>>> >>>>>>>>>>> This version of the patch has been thru Mach5 tier[1-3] testing on >>>>>>>>>>> Oracle's usual set of platforms. Mach5 tier[4-6] is running now and >>>>>>>>>>> Mach5 tier[78] will be run later today. My stress kit on Solaris-X64 >>>>>>>>>>> is running now. Linux-X64 stress testing will start on Sunday. I'm >>>>>>>>>>> planning to do Kitchensink runs, SPECjbb2015 runs and my monitor >>>>>>>>>>> inflation stress tests on Linux-X64, MacOSX and Solaris-X64. >>>>>>>>>>> >>>>>>>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>>>>>>> >>>>>>>>>>> Dan >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 3/24/19 9:57 AM, Daniel D. Daugherty wrote: >>>>>>>>>>>> Greetings, >>>>>>>>>>>> >>>>>>>>>>>> Welcome to the OpenJDK review thread for my port of Carsten's work on: >>>>>>>>>>>> >>>>>>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>>>>>> >>>>>>>>>>>> Here's a link to the OpenJDK wiki that describes my port: >>>>>>>>>>>> >>>>>>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>>>>>> >>>>>>>>>>>> Here's the webrev URL: >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/3-for-jdk13/ >>>>>>>>>>>> >>>>>>>>>>>> Here's a link to Carsten's original webrev: >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~cvarming/monitor_deflate_conc/0/ >>>>>>>>>>>> >>>>>>>>>>>> Earlier versions of this patch have been through several rounds of >>>>>>>>>>>> preliminary review. Many thanks to Carsten, Coleen, Robbin, and >>>>>>>>>>>> Roman for their preliminary code review comments. A very special >>>>>>>>>>>> thanks to Robbin and Roman for building and testing the patch in >>>>>>>>>>>> their own environments (including specJBB2015). >>>>>>>>>>>> >>>>>>>>>>>> This version of the patch has been thru Mach5 tier[1-8] testing on >>>>>>>>>>>> Oracle's usual set of platforms. Earlier versions have been run >>>>>>>>>>>> through my stress kit on my Linux-X64 and Solaris-X64 servers >>>>>>>>>>>> (product, fastdebug, slowdebug).Earlier versions have run Kitchensink >>>>>>>>>>>> for 12 hours on MacOSX, Linux-X64 and Solaris-X64 (product, fastdebug >>>>>>>>>>>> and slowdebug). Earlier versions have run my monitor inflation stress >>>>>>>>>>>> tests for 12 hours on MacOSX, Linux-X64 and Solaris-X64 (product, >>>>>>>>>>>> fastdebug and slowdebug). >>>>>>>>>>>> >>>>>>>>>>>> All of the testing done on earlier versions will be redone on the >>>>>>>>>>>> latest version of the patch. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>>>>>>>> >>>>>>>>>>>> Dan >>>>>>>>>>>> >>>>>>>>>>>> P.S. >>>>>>>>>>>> One subtest in gc/g1/humongousObjects/TestHumongousClassLoader.java >>>>>>>>>>>> is currently failing in -Xcomp mode on Win* only. I've been trying >>>>>>>>>>>> to characterize/analyze this failure for more than a week now. At >>>>>>>>>>>> this point I'm convinced that Async Monitor Deflation is aggravating >>>>>>>>>>>> an existing bug. However, I plan to have a better handle on that >>>>>>>>>>>> failure before these bits are pushed to the jdk/jdk repo. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From coleen.phillimore at oracle.com Mon Mar 2 14:34:19 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 2 Mar 2020 09:34:19 -0500 Subject: RFR(L) 8153224 Monitor deflation prolong safepoints (CR10/v2.10/13-for-jdk15) In-Reply-To: <5a7a7a36-a93c-aa57-93e7-bbd4c58f4d73@oracle.com> References: <62729044-8a22-0e20-0eda-04d47c9ea23c@oracle.com> <313e51c8-b672-bb1c-577a-49868f09e6c1@oracle.com> <1fd54b23-bd35-9b8f-e6f3-6000440d8770@oracle.com> <3fb1a2b5-5aad-eab3-09ba-6f64a2242d30@oracle.com> <38e2d441-11c9-8342-37d5-8030dd06f2f4@oracle.com> <2a8976f7-37e0-03b9-3099-e07464e46512@oracle.com> <5681d640-08c8-3433-0f85-3f23eea69e87@oracle.com> <5a7a7a36-a93c-aa57-93e7-bbd4c58f4d73@oracle.com> Message-ID: <0ce22f0f-a06b-1b04-9e66-24b5c78653b4@oracle.com> Hi Dan, Can you post a webrev with the result of Erik's comments? thanks, Coleen On 3/2/20 8:32 AM, Robbin Ehn wrote: > Hi Dan, thanks sticking with it and addressing the comments! > > I think you got all mine, minus those we decided should be handled as > follow-ups. > > I have nothing more, looks good, thanks! > > /Robbin > > On 2/26/20 11:22 PM, Daniel D. Daugherty wrote: >> Greetings, >> >> I have made changes to the Async Monitor Deflation code in response to >> the CR9/v2.09/12-for-jdk14 code review cycle. Thanks to Robbin and >> Erik O. >> for their comments in this round! >> >> With the extraction and push of {8235931,8236035,8235795} to JDK15, the >> Async Monitor Deflation code is back to "just" async deflation changes! >> >> I have attached the change list from CR9 to CR10 instead of putting >> it in >> the body of this email. I've also added a link to the >> CR9-to-CR10-changes >> file to the webrevs so it should be easy to find. >> >> Main bug URL: >> >> ???? JDK-8153224 Monitor deflation prolong safepoints >> ???? https://bugs.openjdk.java.net/browse/JDK-8153224 >> >> The project is currently baselined on jdk-15+11. >> >> Here's the full webrev URL for those folks that want to see all of the >> current Async Monitor Deflation code in one go (v2.10 full): >> >> http://cr.openjdk.java.net/~dcubed/8153224-webrev/13-for-jdk15+11.v2.10.full/ >> >> >> Some folks might want to see just what has changed since the last review >> cycle so here's a webrev for that (v2.10 inc): >> >> http://cr.openjdk.java.net/~dcubed/8153224-webrev/13-for-jdk15+11.v2.10.inc/ >> >> >> Since we backed out the HandshakeAfterDeflateIdleMonitors option and the >> C2 ref_count changes and updated the copyright years, the "inc" >> webrev has >> a bit more noise in it than usual. Sorry about that! >> >> The OpenJDK wiki has been updated for this round of changes: >> >> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >> >> The jdk-15+11 based v2.10 version of the patch has been thru Mach5 >> tier[1-7] >> testing on Oracle's usual set of platforms. Mach5 tier8 is still >> running. >> I'm running the v2.10 patch through my usual set of stress testing on >> Linux-X64 and macOSX. >> >> I'm planning to do a SPECjbb2015 round on the CR10/v2.20/13-for-jdk15 >> bits. >> >> Thanks, in advance, for any questions, comments or suggestions. >> >> Dan >> >> >> On 2/4/20 9:41 AM, Daniel D. Daugherty wrote: >>> Greetings, >>> >>> This project is no longer targeted to JDK14 so this is NOT an urgent >>> code >>> review request. >>> >>> I've extracted the following three fixes from the Async Monitor >>> Deflation >>> project code: >>> >>> ? ? JDK-8235931 add OM_CACHE_LINE_SIZE and use smaller size on >>> SPARCv9 and X64 >>> ? ? https://bugs.openjdk.java.net/browse/JDK-8235931 >>> >>> ? ? JDK-8236035 refactor ObjectMonitor::set_owner() and _owner field >>> setting >>> ? ? https://bugs.openjdk.java.net/browse/JDK-8236035 >>> >>> ? ? JDK-8235795 replace monitor list >>> mux{Acquire,Release}(&gListLock) with spin locks >>> ? ? https://bugs.openjdk.java.net/browse/JDK-8235795 >>> >>> Each of these has been reviewed separately and will be pushed to JDK15 >>> in the near future (possibly by the end of this week). Of course, there >>> were improvements during these review cycles and the purpose of this >>> e-mail is to provided updated webrevs for this fix >>> (CR9/v2.09/12-for-jdk14) >>> within the revised context provided by {8235931, 8236035, 8235795}. >>> >>> Main bug URL: >>> >>> ??? JDK-8153224 Monitor deflation prolong safepoints >>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>> >>> The project is currently baselined on jdk-14+34. >>> >>> Here's the full webrev URL for those folks that want to see all of the >>> current Async Monitor Deflation code along with {8235931, 8236035, >>> 8235795} >>> in one go (v2.09b full): >>> >>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/12-for-jdk14.v2.09b.full/ >>> >>> >>> Compare the open.patch file in 12-for-jdk14.v2.09.full and >>> 12-for-jdk14.v2.09b.full >>> using your favorite file comparison/merge tool to see how Async >>> Monitor Deflation >>> evolved due to {8235931, 8236035, 8235795}. >>> >>> Some folks might want to see just the Async Monitor Deflation code >>> on top of >>> {8235931, 8236035, 8235795} so here's a webrev for that (v2.09b inc): >>> >>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/12-for-jdk14.v2.09b.inc/ >>> >>> >>> These webrevs have gone thru several Mach5 Tier[1-8] runs along with >>> my usual stress testing and SPECjbb2015 testing and there aren't any >>> surprises relative to CR9/v2.09/12-for-jdk14. >>> >>> Thanks, in advance, for any questions, comments or suggestions. >>> >>> Dan >>> >>> >>> On 12/11/19 3:41 PM, Daniel D. Daugherty wrote: >>>> Greetings, >>>> >>>> I have made changes to the Async Monitor Deflation code in response to >>>> the CR8/v2.08/11-for-jdk14 code review cycle. Thanks to David H., >>>> Robbin >>>> and Erik O. for their comments! >>>> >>>> This project is no longer targeted to JDK14 so this is NOT an >>>> urgent code >>>> review request. The primary purpose of this webrev is simply to >>>> close the >>>> CR8/v2.08/11-for-jdk14 code review loop and to let folks see how I >>>> resolved >>>> the code review comments from that round. >>>> >>>> Most of the comments in the CR8/v2.08/11-for-jdk14 code review >>>> cycle were >>>> on the monitor list changes so I'm going to take a look at >>>> extracting those >>>> changes into a standalone patch. Switching from >>>> Thread::muxAcquire(&gListLock) >>>> and Thread::muxRelease(&gListLock) to finer grained internal spin >>>> locks needs >>>> to be thoroughly reviewed and the best way to do that is separately >>>> from the >>>> Async Monitor Deflation changes. Thanks to Coleen for suggesting >>>> doing this >>>> extraction earlier. >>>> >>>> I have attached the change list from CR8 to CR9 instead of putting >>>> it in >>>> the body of this email. I've also added a link to the >>>> CR8-to-CR9-changes >>>> file to the webrevs so it should be easy to find. >>>> >>>> Main bug URL: >>>> >>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>>> >>>> The project is currently baselined on jdk-14+26. >>>> >>>> Here's the full webrev URL for those folks that want to see all of the >>>> current Async Monitor Deflation code in one go (v2.09 full): >>>> >>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/12-for-jdk14.v2.09.full/ >>>> >>>> >>>> Some folks might want to see just what has changed since the last >>>> review >>>> cycle so here's a webrev for that (v2.09 inc): >>>> >>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/12-for-jdk14.v2.09.inc/ >>>> >>>> >>>> The OpenJDK wiki has NOT yet been updated for this round of changes: >>>> >>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>> >>>> The jdk-14+26 based v2.09 version of the patch has been thru Mach5 >>>> tier[1-7] >>>> testing on Oracle's usual set of platforms. Mach5 tier8 is still >>>> running. >>>> A slightly older version of the v2.09 patch has also been through >>>> my usual >>>> set of stress testing on Linux-X64 and macOSX with the addition of >>>> Robbin's >>>> "MoCrazy 1024" test running in parallel on Linux-X64 with the other >>>> tests in >>>> my lab. The "MoCrazy 1024" has been going for > 5 days and 6700+ >>>> iterations >>>> without any failures. >>>> >>>> I'm planning to do a SPECjbb2015 round on the >>>> CR9/v2.09/12-for-jdk14 bits. >>>> >>>> Thanks, in advance, for any questions, comments or suggestions. >>>> >>>> Dan >>>> >>>> >>>> On 11/4/19 4:03 PM, Daniel D. Daugherty wrote: >>>>> Greetings, >>>>> >>>>> I have made changes to the Async Monitor Deflation code in >>>>> response to >>>>> the CR7/v2.07/10-for-jdk14 code review cycle. Thanks to David H., >>>>> Robbin >>>>> and Erik O. for their comments! >>>>> >>>>> JDK14 Rampdown phase one is coming on Dec. 12, 2019 and the Async >>>>> Monitor >>>>> Deflation project needs to push before Nov. 12, 2019 in order to >>>>> allow >>>>> for sufficient bake time for such a big change. Nov. 12 is _next_ >>>>> Tuesday >>>>> so we have 8 days from today to finish this code review cycle and >>>>> push >>>>> this code for JDK14. >>>>> >>>>> Carsten and Roman! Time for you guys to chime in again on the code >>>>> reviews. >>>>> >>>>> I have attached the change list from CR7 to CR8 instead of putting >>>>> it in >>>>> the body of this email. I've also added a link to the >>>>> CR7-to-CR8-changes >>>>> file to the webrevs so it should be easy to find. >>>>> >>>>> Main bug URL: >>>>> >>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>> >>>>> The project is currently baselined on jdk-14+21. >>>>> >>>>> Here's the full webrev URL for those folks that want to see all of >>>>> the >>>>> current Async Monitor Deflation code in one go (v2.08 full): >>>>> >>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/11-for-jdk14.v2.08.full >>>>> >>>>> >>>>> Some folks might want to see just what has changed since the last >>>>> review >>>>> cycle so here's a webrev for that (v2.08 inc): >>>>> >>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/11-for-jdk14.v2.08.inc/ >>>>> >>>>> >>>>> The OpenJDK wiki did not need any changes for this round: >>>>> >>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>> >>>>> The jdk-14+21 based v2.08 version of the patch has been thru Mach5 >>>>> tier[1-8] >>>>> testing on Oracle's usual set of platforms. It has also been >>>>> through my usual >>>>> set of stress testing on Linux-X64, macOSX and Solaris-X64 with >>>>> the addition >>>>> of Robbin's "MoCrazy 1024" test running in parallel with the other >>>>> tests in >>>>> my lab. Some testing is still running, but so far there are no new >>>>> regressions. >>>>> >>>>> I have not yet done a SPECjbb2015 round on the >>>>> CR8/v2.08/11-for-jdk14 bits. >>>>> >>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>> >>>>> Dan >>>>> >>>>> >>>>> On 10/17/19 5:50 PM, Daniel D. Daugherty wrote: >>>>>> Greetings, >>>>>> >>>>>> The Async Monitor Deflation project is reaching the end game. I >>>>>> have no >>>>>> changes planned for the project at this time so all that is left >>>>>> is code >>>>>> review and any changes that results from those reviews. >>>>>> >>>>>> Carsten and Roman! Time for you guys to chime in again on the >>>>>> code reviews. >>>>>> >>>>>> I have attached the list of fixes from CR6 to CR7 instead of >>>>>> putting it >>>>>> in the main body of this email. >>>>>> >>>>>> Main bug URL: >>>>>> >>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>> >>>>>> The project is currently baselined on jdk-14+19. >>>>>> >>>>>> Here's the full webrev URL for those folks that want to see all >>>>>> of the >>>>>> current Async Monitor Deflation code in one go (v2.07 full): >>>>>> >>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/10-for-jdk14.v2.07.full >>>>>> >>>>>> >>>>>> Some folks might want to see just what has changed since the last >>>>>> review >>>>>> cycle so here's a webrev for that (v2.07 inc): >>>>>> >>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/10-for-jdk14.v2.07.inc/ >>>>>> >>>>>> >>>>>> The OpenJDK wiki has been updated to match the >>>>>> CR7/v2.07/10-for-jdk14 changes: >>>>>> >>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>> >>>>>> >>>>>> The jdk-14+18 based v2.07 version of the patch has been thru >>>>>> Mach5 tier[1-8] >>>>>> testing on Oracle's usual set of platforms. It has also been >>>>>> through my usual >>>>>> set of stress testing on Linux-X64, macOSX and Solaris-X64 with >>>>>> the addition >>>>>> of Robbin's "MoCrazy 1024" test running in parallel with the >>>>>> other tests in >>>>>> my lab. >>>>>> >>>>>> The jdk-14+19 based v2.07 version of the patch has been thru >>>>>> Mach5 tier[1-3] >>>>>> test on Oracle's usual set of platforms. Mach5 tier[4-8] are in >>>>>> process. >>>>>> >>>>>> I did another round of SPECjbb2015 testing in Oracle's Aurora >>>>>> Performance lab >>>>>> using using their tuned SPECjbb2015 Linux-X64 G1 configs: >>>>>> >>>>>> ??? - "base" is jdk-14+18 >>>>>> ??? - "v2.07" is the latest version and includes C2 >>>>>> inc_om_ref_count() support >>>>>> ????? on LP64 X64 and the new HandshakeAfterDeflateIdleMonitors >>>>>> option >>>>>> ? ? - "off" is with -XX:-AsyncDeflateIdleMonitors specified >>>>>> ? ? - "handshake" is with -XX:+HandshakeAfterDeflateIdleMonitors >>>>>> specified >>>>>> >>>>>> ???????? hbIR?????????? hbIR >>>>>> ??? (max attempted)? (settled)? max-jOPS? critical-jOPS runtime >>>>>> ??? ---------------? ---------? --------? ------------- ------- >>>>>> ?????????? 34282.00?? 30635.90? 28831.30?????? 20969.20 3841.30 base >>>>>> ?????????? 34282.00?? 30973.00? 29345.80?????? 21025.20 3964.10 >>>>>> v2.07 >>>>>> ?????????? 34282.00?? 31105.60? 29174.30?????? 21074.00 3931.30 >>>>>> v2.07_handshake >>>>>> ?????????? 34282.00?? 30789.70? 27151.60?????? 19839.10 3850.20 >>>>>> v2.07_off >>>>>> >>>>>> ??? - The Aurora Perf comparison tool reports: >>>>>> >>>>>> ??????? Comparison????????????? max-jOPS critical-jOPS >>>>>> ??????? ----------------------? -------------------- >>>>>> -------------------- >>>>>> ??????? base vs 2.07??????????? +1.78% (s, p=0.000) +0.27% (ns, >>>>>> p=0.790) >>>>>> ??????? base vs 2.07_handshake? +1.19% (s, p=0.007) +0.58% (ns, >>>>>> p=0.536) >>>>>> ??????? base vs 2.07_off??????? -5.83% (ns, p=0.394) -5.39% (ns, >>>>>> p=0.347) >>>>>> >>>>>> ??????? (s) - significant? (ns) - not-significant >>>>>> >>>>>> ??? - For historical comparison, the Aurora Perf comparision tool >>>>>> ??????? reported for v2.06 with a baseline of jdk-13+31: >>>>>> >>>>>> ??????? Comparison????????????? max-jOPS critical-jOPS >>>>>> ??????? ----------------------? -------------------- >>>>>> -------------------- >>>>>> ??????? base vs 2.06??????????? -0.32% (ns, p=0.345) +0.71% (ns, >>>>>> p=0.646) >>>>>> ??????? base vs 2.06_off??????? +0.49% (ns, p=0.292) -1.21% (ns, >>>>>> p=0.481) >>>>>> >>>>>> ??????? (s) - significant? (ns) - not-significant >>>>>> >>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>> >>>>>> Dan >>>>>> >>>>>> >>>>>> On 8/28/19 5:02 PM, Daniel D. Daugherty wrote: >>>>>>> Greetings, >>>>>>> >>>>>>> The Async Monitor Deflation project has rebased to JDK14 so it's >>>>>>> time >>>>>>> for our first code review in that new context!! >>>>>>> >>>>>>> I've been focused on changing the monitor list management code >>>>>>> to be >>>>>>> lock-free in order to make SPECjbb2015 happier. Of course with a >>>>>>> change >>>>>>> like that, it takes a while to chase down all the new and wonderful >>>>>>> races. At this point, I have the code back to the same stability >>>>>>> that >>>>>>> I had with CR5/v2.05/8-for-jdk13. >>>>>>> >>>>>>> To lay the ground work for this round of review, I pushed the >>>>>>> following >>>>>>> two fixes to jdk/jdk earlier today: >>>>>>> >>>>>>> ??? JDK-8230184 rename, whitespace, indent and comments changes >>>>>>> in preparation >>>>>>> ? ? ??????????? for lock free Monitor lists >>>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8230184 >>>>>>> >>>>>>> ??? JDK-8230317 serviceability/sa/ClhsdbPrintStatics.java fails >>>>>>> after 8230184 >>>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8230317 >>>>>>> >>>>>>> I have attached the list of fixes from CR5 to CR6 instead of >>>>>>> putting >>>>>>> in the main body of this email. >>>>>>> >>>>>>> Main bug URL: >>>>>>> >>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>> >>>>>>> The project is currently baselined on jdk-14+11 plus the fixes for >>>>>>> JDK-8230184 and JDK-8230317. >>>>>>> >>>>>>> Here's the full webrev URL for those folks that want to see all >>>>>>> of the >>>>>>> current Async Monitor Deflation code in one go (v2.06 full): >>>>>>> >>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.06.full/ >>>>>>> >>>>>>> >>>>>>> >>>>>>> The primary focus of this review cycle is on the lock-free >>>>>>> Monitor List >>>>>>> management changes so here's a webrev for just that patch (v2.06c): >>>>>>> >>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.06c.inc/ >>>>>>> >>>>>>> >>>>>>> The secondary focus of this review cycle is on the bug fixes >>>>>>> that have >>>>>>> been made since CR5/v2.05/8-for-jdk13 so here's a webrev for >>>>>>> just that >>>>>>> patch (v2.06b): >>>>>>> >>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.06b.inc/ >>>>>>> >>>>>>> >>>>>>> The third and final bucket for this review cycle is the rename, >>>>>>> whitespace, >>>>>>> indent and comments changes made in preparation for lock free >>>>>>> Monitor list >>>>>>> management. Almost all of that was extracted into JDK-8230184 >>>>>>> for the >>>>>>> baseline so this bucket now has just a few comment changes >>>>>>> relative to >>>>>>> CR5/v2.05/8-for-jdk13. Here's a webrev for the remainder (v2.06a): >>>>>>> >>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.06a.inc/ >>>>>>> >>>>>>> >>>>>>> >>>>>>> Some folks might want to see just what has changed since the >>>>>>> last review >>>>>>> cycle so here's a webrev for that (v2.06 inc): >>>>>>> >>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.06.inc/ >>>>>>> >>>>>>> >>>>>>> >>>>>>> Last, but not least, some folks might want to see the code >>>>>>> before the >>>>>>> addition of lock-free Monitor List management so here's a webrev >>>>>>> for >>>>>>> that (v2.00 -> v2.05): >>>>>>> >>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.05.inc/ >>>>>>> >>>>>>> >>>>>>> The OpenJDK wiki will need minor updates to match the CR6 changes: >>>>>>> >>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>> >>>>>>> >>>>>>> but that should only be changes to describe per-thread list >>>>>>> async monitor >>>>>>> deflation being done by the ServiceThread. >>>>>>> >>>>>>> (I did update the OpenJDK wiki for the CR5 changes back on >>>>>>> 2019.08.14) >>>>>>> >>>>>>> This version of the patch has been thru Mach5 tier[1-8] testing on >>>>>>> Oracle's usual set of platforms. It has also been through my >>>>>>> usual set >>>>>>> of stress testing on Linux-X64, macOSX and Solaris-X64. >>>>>>> >>>>>>> I did a bunch of SPECjbb2015 testing in Oracle's Aurora >>>>>>> Performance lab >>>>>>> using using their tuned SPECjbb2015 Linux-X64 G1 configs. This >>>>>>> was using >>>>>>> this patch baselined on jdk-13+31 (for stability): >>>>>>> >>>>>>> ????????? hbIR?????????? hbIR >>>>>>> ???? (max attempted)? (settled)? max-jOPS critical-jOPS runtime >>>>>>> ???? ---------------? ---------? -------- ------------- ------- >>>>>>> ??????????? 34282.00?? 28837.20? 27905.20 19817.40 3658.10 base >>>>>>> ??????????? 34965.70?? 29798.80? 27814.90 19959.00 3514.60 v2.06d >>>>>>> ??????????? 34282.00?? 29100.70? 28042.50 19577.00 3701.90 >>>>>>> v2.06d_off >>>>>>> ??????????? 34282.00?? 29218.50? 27562.80 19397.30 3657.60 >>>>>>> v2.06d_ocache >>>>>>> ??????????? 34965.70?? 29838.30? 26512.40 19170.60 3569.90 v2.05 >>>>>>> ??????????? 34282.00?? 28926.10? 27734.00 19835.10 3588.40 >>>>>>> v2.05_off >>>>>>> >>>>>>> The "off" configs are with -XX:-AsyncDeflateIdleMonitors >>>>>>> specified and >>>>>>> the "ocache" config is with 128 byte cache line sizes instead of >>>>>>> 64 byte >>>>>>> cache lines sizes. "v2.06d" is the last set of changes that I >>>>>>> made before >>>>>>> those changes were distributed into the "v2.06a", "v2.06b" and >>>>>>> "v2.06c" >>>>>>> buckets for this review recycle. >>>>>>> >>>>>>> >>>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>>> >>>>>>> Dan >>>>>>> >>>>>>> >>>>>>> On 7/11/19 3:49 PM, Daniel D. Daugherty wrote: >>>>>>>> Greetings, >>>>>>>> >>>>>>>> I've been focused on chasing down and fixing the rare test >>>>>>>> failures >>>>>>>> that only pop up rarely. So this round is primarily fixes for >>>>>>>> races >>>>>>>> with a few additional fixes that came from Karen's review of CR4. >>>>>>>> Thanks Karen! >>>>>>>> >>>>>>>> I have attached the list of fixes from CR4 to CR5 instead of >>>>>>>> putting >>>>>>>> in the main body of this email. >>>>>>>> >>>>>>>> Main bug URL: >>>>>>>> >>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>> >>>>>>>> The project is currently baselined on jdk-13+29. This will >>>>>>>> likely be >>>>>>>> the last JDK13 baseline for this project and I'll roll to the >>>>>>>> JDK14 >>>>>>>> (jdk/jdk) repo soon... >>>>>>>> >>>>>>>> Here's the full webrev URL: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/8-for-jdk13.full/ >>>>>>>> >>>>>>>> >>>>>>>> Here's the incremental webrev URL: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/8-for-jdk13.inc/ >>>>>>>> >>>>>>>> I have not yet checked the OpenJDK wiki to see if it needs any >>>>>>>> updates >>>>>>>> to match the CR5 changes: >>>>>>>> >>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>> >>>>>>>> >>>>>>>> (I did update the OpenJDK wiki for the CR4 changes back on >>>>>>>> 2019.06.26) >>>>>>>> >>>>>>>> This version of the patch has been thru Mach5 tier[1-3] testing on >>>>>>>> Oracle's usual set of platforms. Mach5 tier[4-6] is running now >>>>>>>> and >>>>>>>> Mach5 tier[78] will follow. I'll kick off the usual stress testing >>>>>>>> on Linux-X64, macOSX and Solaris-X64 as those machines become >>>>>>>> available. >>>>>>>> Since I haven't made any performance changes in this round, >>>>>>>> I'll only >>>>>>>> be running SPECjbb2015 to gather the latest monitorinflation logs. >>>>>>>> >>>>>>>> Next up: >>>>>>>> >>>>>>>> - We're still seeing 4-5% lower performance with SPECjbb2015 on >>>>>>>> ? Linux-X64 and we've determined that some of that comes from >>>>>>>> ? contention on the gListLock. So I'm going to investigate >>>>>>>> removing >>>>>>>> ? the gListLock. Yes, another lock free set of changes is coming! >>>>>>>> - Of course, going lock free often causes new races and new >>>>>>>> failures >>>>>>>> ? so that's a good reason for make those changes isolated in their >>>>>>>> ? own round (and not holding up CR5/v2.05/8-for-jdk13 anymore). >>>>>>>> - I finally have a potential fix for the Win* failure with >>>>>>>> gc/g1/humongousObjects/TestHumongousClassLoader.java >>>>>>>> ? but I haven't run it through Mach5 yet so it'll be in the >>>>>>>> next round. >>>>>>>> - Some RTM tests were recently re-enabled in Mach5 and I'm >>>>>>>> seeing some >>>>>>>> ? monitor related failures there. I suspect that I need to go >>>>>>>> take a >>>>>>>> ? look at the C2 RTM macro assembler code and look for things >>>>>>>> that might >>>>>>>> ? conflict if Async Monitor Deflation. If you're interested in >>>>>>>> that kind >>>>>>>> ? of issue, then see the macroAssembler_x86.cpp sanity check >>>>>>>> that I >>>>>>>> ? added in this round! >>>>>>>> >>>>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>>>> >>>>>>>> Dan >>>>>>>> >>>>>>>> >>>>>>>> On 5/26/19 8:30 PM, Daniel D. Daugherty wrote: >>>>>>>>> Greetings, >>>>>>>>> >>>>>>>>> I have a fix for an issue that came up during performance >>>>>>>>> testing. >>>>>>>>> Many thanks to Robbin for diagnosing the issue in his SPECjbb2015 >>>>>>>>> experiments. >>>>>>>>> >>>>>>>>> Here's the list of changes from CR3 to CR4. The list is a bit >>>>>>>>> verbose due to the complexity of the issue, but the changes >>>>>>>>> themselves are not that big. >>>>>>>>> >>>>>>>>> Functional: >>>>>>>>> ? - Change SafepointSynchronize::is_cleanup_needed() from calling >>>>>>>>> ??? ObjectSynchronizer::is_cleanup_needed() to calling >>>>>>>>> ObjectSynchronizer::is_safepoint_deflation_needed(): >>>>>>>>> ??? - is_safepoint_deflation_needed() returns the result of >>>>>>>>> ????? monitors_used_above_threshold() for safepoint based >>>>>>>>> ????? monitor deflation (!AsyncDeflateIdleMonitors). >>>>>>>>> ??? - For AsyncDeflateIdleMonitors, it only returns true if >>>>>>>>> ????? there is a special deflation request, e.g., System.gc() >>>>>>>>> ????? - This solves a bug where there are a bunch of Cleanup >>>>>>>>> ??????? safepoints that simply request async deflation which >>>>>>>>> ??????? keeps the async JavaThreads from making progress on >>>>>>>>> ??????? their async deflation work. >>>>>>>>> ? - Add AsyncDeflationInterval diagnostic option. Description: >>>>>>>>> ????? Async deflate idle monitors every so many milliseconds when >>>>>>>>> ????? MonitorUsedDeflationThreshold is exceeded (0 is off). >>>>>>>>> ? - Replace ObjectSynchronizer::gOmShouldDeflateIdleMonitors() >>>>>>>>> with >>>>>>>>> ObjectSynchronizer::is_async_deflation_needed(): >>>>>>>>> ??? - is_async_deflation_needed() returns true when >>>>>>>>> ????? is_async_cleanup_requested() is true or when >>>>>>>>> ????? monitors_used_above_threshold() is true (but no more >>>>>>>>> often than >>>>>>>>> ????? AsyncDeflationInterval). >>>>>>>>> ??? - if AsyncDeflateIdleMonitors Service_lock->wait() now >>>>>>>>> waits for >>>>>>>>> ????? at most GuaranteedSafepointInterval millis: >>>>>>>>> ????? - This allows is_async_deflation_needed() to be checked at >>>>>>>>> ??????? the same interval as GuaranteedSafepointInterval. >>>>>>>>> ??????? (default is 1000 millis/1 second) >>>>>>>>> ????? - Once is_async_deflation_needed() has returned true, it >>>>>>>>> ??????? generally cannot return true for AsyncDeflationInterval. >>>>>>>>> ??????? This is to prevent async deflation from swamping the >>>>>>>>> ??????? ServiceThread. >>>>>>>>> ? - The ServiceThread still handles async deflation of the global >>>>>>>>> ??? in-use list and now it also marks JavaThreads for async >>>>>>>>> deflation >>>>>>>>> ??? of their in-use lists. >>>>>>>>> ??? - The ServiceThread will check for async deflation work every >>>>>>>>> ????? GuaranteedSafepointInterval. >>>>>>>>> ??? - A safepoint can still cause the ServiceThread to check for >>>>>>>>> ????? async deflation work via is_async_deflation_requested. >>>>>>>>> ? - Refactor code from ObjectSynchronizer::is_cleanup_needed() >>>>>>>>> into >>>>>>>>> ??? monitors_used_above_threshold() and remove >>>>>>>>> is_cleanup_needed(). >>>>>>>>> ? - In addition to System.gc(), the VM_Exit VM op and the final >>>>>>>>> ??? VMThread safepoint now set the is_special_deflation_requested >>>>>>>>> ??? flag to reduce the in-use monitor population that is >>>>>>>>> reported by >>>>>>>>> ObjectSynchronizer::log_in_use_monitor_details() at VM exit. >>>>>>>>> >>>>>>>>> Test update: >>>>>>>>> ? - test/hotspot/gtest/oops/test_markOop.cpp is updated to >>>>>>>>> work with >>>>>>>>> ??? AsyncDeflateIdleMonitors. >>>>>>>>> >>>>>>>>> Collateral: >>>>>>>>> ? - Add/clarify/update some logging messages. >>>>>>>>> >>>>>>>>> Cleanup: >>>>>>>>> ? - Updated comments based on Karen's code review. >>>>>>>>> ? - Change 'special cleanup' -> 'special deflation' and >>>>>>>>> ??? 'async cleanup' -> 'async deflation'. >>>>>>>>> ??? - comment and function name changes >>>>>>>>> ? - Clarify MonitorUsedDeflationThreshold description; >>>>>>>>> >>>>>>>>> >>>>>>>>> Main bug URL: >>>>>>>>> >>>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>>> >>>>>>>>> The project is currently baselined on jdk-13+22. >>>>>>>>> >>>>>>>>> Here's the full webrev URL: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/7-for-jdk13.full/ >>>>>>>>> >>>>>>>>> >>>>>>>>> Here's the incremental webrev URL: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/7-for-jdk13.inc/ >>>>>>>>> >>>>>>>>> >>>>>>>>> I have not updated the OpenJDK wiki to reflect the CR4 changes: >>>>>>>>> >>>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>>> >>>>>>>>> >>>>>>>>> The wiki doesn't say a whole lot about the async deflation >>>>>>>>> invocation >>>>>>>>> mechanism so I have to figure out how to add that content. >>>>>>>>> >>>>>>>>> This version of the patch has been thru Mach5 tier[1-8] >>>>>>>>> testing on >>>>>>>>> Oracle's usual set of platforms. My Solaris-X64 stress kit run is >>>>>>>>> running now. Kitchensink8H on product, fastdebug, and >>>>>>>>> slowdebug bits >>>>>>>>> are running on Linux-X64, MacOSX and Solaris-X64. I still have >>>>>>>>> to run >>>>>>>>> my stress kit on Linux-X64. I still have to run the SPECjbb2015 >>>>>>>>> baseline and CR4 runs on Linux-X64, MacOSX and Solaris-X64. >>>>>>>>> >>>>>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>>>>> >>>>>>>>> Dan >>>>>>>>> >>>>>>>>> On 5/6/19 11:52 AM, Daniel D. Daugherty wrote: >>>>>>>>>> Greetings, >>>>>>>>>> >>>>>>>>>> I had some discussions with Karen about a race that was in the >>>>>>>>>> ObjectMonitor::enter() code in CR2/v2.02/5-for-jdk13. This >>>>>>>>>> race was >>>>>>>>>> theoretical and I had no test failures due to it. The fix is >>>>>>>>>> pretty >>>>>>>>>> simple: remove the special case code for async deflation in the >>>>>>>>>> ObjectMonitor::enter() function and rely solely on the ref_count >>>>>>>>>> for ObjectMonitor::enter() protection. >>>>>>>>>> >>>>>>>>>> During those discussions Karen also floated the idea of using >>>>>>>>>> the >>>>>>>>>> ref_count field instead of the contentions field for the Async >>>>>>>>>> Monitor Deflation protocol. I decided to go ahead and code up >>>>>>>>>> that >>>>>>>>>> change and I have run it through the usual stress and Mach5 >>>>>>>>>> testing >>>>>>>>>> with no issues. It's also known as v2.03 (for those for with the >>>>>>>>>> patches) and as webrev/6-for-jdk13 (for those with webrev URLs). >>>>>>>>>> Sorry for all the names... >>>>>>>>>> >>>>>>>>>> Main bug URL: >>>>>>>>>> >>>>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>>>> >>>>>>>>>> The project is currently baselined on jdk-13+18. >>>>>>>>>> >>>>>>>>>> Here's the full webrev URL: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/6-for-jdk13.full/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Here's the incremental webrev URL: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/6-for-jdk13.inc/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> I have also updated the OpenJDK wiki to reflect the CR3 changes: >>>>>>>>>> >>>>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> This version of the patch has been thru Mach5 tier[1-8] >>>>>>>>>> testing on >>>>>>>>>> Oracle's usual set of platforms. My Solaris-X64 stress kit >>>>>>>>>> run had >>>>>>>>>> no issues. Kitchensink8H on product, fastdebug, and slowdebug >>>>>>>>>> bits >>>>>>>>>> had no failures on Linux-X64; MacOSX fastdebug and slowdebug and >>>>>>>>>> Solaris-X64 release had the usual "Too large time diff" >>>>>>>>>> complaints. >>>>>>>>>> 12 hour Inflate2 runs on product, fastdebug and slowdebug >>>>>>>>>> bits on >>>>>>>>>> Linux-X64, MacOSX and Solaris-X64 had no failures. My Linux-X64 >>>>>>>>>> stress kit is running right now. >>>>>>>>>> >>>>>>>>>> I've done the SPECjbb2015 baseline and CR3 runs. I need to >>>>>>>>>> gather >>>>>>>>>> the results and analyze them. >>>>>>>>>> >>>>>>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>>>>>> >>>>>>>>>> Dan >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 4/25/19 12:38 PM, Daniel D. Daugherty wrote: >>>>>>>>>>> Greetings, >>>>>>>>>>> >>>>>>>>>>> I have a small but important bug fix for the Async Monitor >>>>>>>>>>> Deflation >>>>>>>>>>> project ready to go. It's also known as v2.02 (for those for >>>>>>>>>>> with the >>>>>>>>>>> patches) and as webrev/5-for-jdk13 (for those with webrev >>>>>>>>>>> URLs). Sorry >>>>>>>>>>> for all the names... >>>>>>>>>>> >>>>>>>>>>> JDK-8222295 was pushed to jdk/jdk two days ago so that >>>>>>>>>>> baseline patch >>>>>>>>>>> is out of our hair. >>>>>>>>>>> >>>>>>>>>>> Main bug URL: >>>>>>>>>>> >>>>>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>>>>> >>>>>>>>>>> The project is currently baselined on jdk-13+17. >>>>>>>>>>> >>>>>>>>>>> Here's the full webrev URL: >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/5-for-jdk13.full/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Here's the incremental webrev URL (JDK-8153224): >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/5-for-jdk13.inc/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I still have to update the OpenJDK wiki to reflect the CR2 >>>>>>>>>>> changes: >>>>>>>>>>> >>>>>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> This version of the patch has been thru Mach5 tier[1-6] >>>>>>>>>>> testing on >>>>>>>>>>> Oracle's usual set of platforms. Mach5 tier[7-8] is running >>>>>>>>>>> now. >>>>>>>>>>> My stress kit is running on Solaris-X64 now. Kitchensink8H >>>>>>>>>>> is running >>>>>>>>>>> now on product, fastdebug, and slowdebug bits on Linux-X64, >>>>>>>>>>> MacOSX >>>>>>>>>>> and Solaris-X64. 12 hour Inflate2 runs are running now on >>>>>>>>>>> product, >>>>>>>>>>> fastdebug and slowdebug bits on Linux-X64, MacOSX and >>>>>>>>>>> Solaris-X64. >>>>>>>>>>> I'll start my my stress kit on Linux-X64 sometime on Sunday >>>>>>>>>>> (after >>>>>>>>>>> my jdk-13+18 stress run is done). >>>>>>>>>>> >>>>>>>>>>> I'll do SPECjbb2015 baseline and CR2 runs after all the stress >>>>>>>>>>> testing is done. >>>>>>>>>>> >>>>>>>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>>>>>>> >>>>>>>>>>> Dan >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 4/19/19 11:58 AM, Daniel D. Daugherty wrote: >>>>>>>>>>>> Greetings, >>>>>>>>>>>> >>>>>>>>>>>> I finally have CR1 for the Async Monitor Deflation project >>>>>>>>>>>> ready to >>>>>>>>>>>> go. It's also known as v2.01 (for those for with the >>>>>>>>>>>> patches) and as >>>>>>>>>>>> webrev/4-for-jdk13 (for those with webrev URLs). Sorry for >>>>>>>>>>>> all the >>>>>>>>>>>> names... >>>>>>>>>>>> >>>>>>>>>>>> Main bug URL: >>>>>>>>>>>> >>>>>>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>>>>>> >>>>>>>>>>>> Baseline bug fixes URL: >>>>>>>>>>>> >>>>>>>>>>>> ??? JDK-8222295 more baseline cleanups from Async Monitor >>>>>>>>>>>> Deflation project >>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8222295 >>>>>>>>>>>> >>>>>>>>>>>> The project is currently baselined on jdk-13+15. >>>>>>>>>>>> >>>>>>>>>>>> Here's the webrev for the latest baseline changes >>>>>>>>>>>> (JDK-8222295): >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/4-for-jdk13.8222295 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Here's the full webrev URL (JDK-8153224 only): >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/4-for-jdk13.full/ >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Here's the incremental webrev URL (JDK-8153224): >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/4-for-jdk13.inc/ >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> So I'm looking for reviews for both JDK-8222295 and the >>>>>>>>>>>> latest version >>>>>>>>>>>> of JDK-8153224... >>>>>>>>>>>> >>>>>>>>>>>> I still have to update the OpenJDK wiki to reflect the CR >>>>>>>>>>>> changes: >>>>>>>>>>>> >>>>>>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> This version of the patch has been thru Mach5 tier[1-3] >>>>>>>>>>>> testing on >>>>>>>>>>>> Oracle's usual set of platforms. Mach5 tier[4-6] is running >>>>>>>>>>>> now and >>>>>>>>>>>> Mach5 tier[78] will be run later today. My stress kit on >>>>>>>>>>>> Solaris-X64 >>>>>>>>>>>> is running now. Linux-X64 stress testing will start on >>>>>>>>>>>> Sunday. I'm >>>>>>>>>>>> planning to do Kitchensink runs, SPECjbb2015 runs and my >>>>>>>>>>>> monitor >>>>>>>>>>>> inflation stress tests on Linux-X64, MacOSX and Solaris-X64. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, in advance, for any questions, comments or >>>>>>>>>>>> suggestions. >>>>>>>>>>>> >>>>>>>>>>>> Dan >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 3/24/19 9:57 AM, Daniel D. Daugherty wrote: >>>>>>>>>>>>> Greetings, >>>>>>>>>>>>> >>>>>>>>>>>>> Welcome to the OpenJDK review thread for my port of >>>>>>>>>>>>> Carsten's work on: >>>>>>>>>>>>> >>>>>>>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>>>>>>> >>>>>>>>>>>>> Here's a link to the OpenJDK wiki that describes my port: >>>>>>>>>>>>> >>>>>>>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Here's the webrev URL: >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/3-for-jdk13/ >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Here's a link to Carsten's original webrev: >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~cvarming/monitor_deflate_conc/0/ >>>>>>>>>>>>> >>>>>>>>>>>>> Earlier versions of this patch have been through several >>>>>>>>>>>>> rounds of >>>>>>>>>>>>> preliminary review. Many thanks to Carsten, Coleen, >>>>>>>>>>>>> Robbin, and >>>>>>>>>>>>> Roman for their preliminary code review comments. A very >>>>>>>>>>>>> special >>>>>>>>>>>>> thanks to Robbin and Roman for building and testing the >>>>>>>>>>>>> patch in >>>>>>>>>>>>> their own environments (including specJBB2015). >>>>>>>>>>>>> >>>>>>>>>>>>> This version of the patch has been thru Mach5 tier[1-8] >>>>>>>>>>>>> testing on >>>>>>>>>>>>> Oracle's usual set of platforms. Earlier versions have >>>>>>>>>>>>> been run >>>>>>>>>>>>> through my stress kit on my Linux-X64 and Solaris-X64 servers >>>>>>>>>>>>> (product, fastdebug, slowdebug).Earlier versions have run >>>>>>>>>>>>> Kitchensink >>>>>>>>>>>>> for 12 hours on MacOSX, Linux-X64 and Solaris-X64 >>>>>>>>>>>>> (product, fastdebug >>>>>>>>>>>>> and slowdebug). Earlier versions have run my monitor >>>>>>>>>>>>> inflation stress >>>>>>>>>>>>> tests for 12 hours on MacOSX, Linux-X64 and Solaris-X64 >>>>>>>>>>>>> (product, >>>>>>>>>>>>> fastdebug and slowdebug). >>>>>>>>>>>>> >>>>>>>>>>>>> All of the testing done on earlier versions will be redone >>>>>>>>>>>>> on the >>>>>>>>>>>>> latest version of the patch. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, in advance, for any questions, comments or >>>>>>>>>>>>> suggestions. >>>>>>>>>>>>> >>>>>>>>>>>>> Dan >>>>>>>>>>>>> >>>>>>>>>>>>> P.S. >>>>>>>>>>>>> One subtest in >>>>>>>>>>>>> gc/g1/humongousObjects/TestHumongousClassLoader.java >>>>>>>>>>>>> is currently failing in -Xcomp mode on Win* only. I've >>>>>>>>>>>>> been trying >>>>>>>>>>>>> to characterize/analyze this failure for more than a week >>>>>>>>>>>>> now. At >>>>>>>>>>>>> this point I'm convinced that Async Monitor Deflation is >>>>>>>>>>>>> aggravating >>>>>>>>>>>>> an existing bug. However, I plan to have a better handle >>>>>>>>>>>>> on that >>>>>>>>>>>>> failure before these bits are pushed to the jdk/jdk repo. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> From claes.redestad at oracle.com Mon Mar 2 15:07:10 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 2 Mar 2020 16:07:10 +0100 Subject: RFR: 8240302: x64: Assembler::reachable redundantly call Relocation::type() more than once Message-ID: Hi, in the x64-version Assembler::reachable method, various calls to adr.reloc() are not hoisted by the compiler, due calling into the virtual method Relocation::type(). This means each call to reachable typically do several redundant calls into Relocation::type() This patch refactors all these calls into a single one, which has a small, but measurable, effect on startup (and likely some small effect on JIT compiler throughput in general) Webrev: http://cr.openjdk.java.net/~redestad/8240302/open.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8240302 Testing: tier1-3 Thanks! /Claes From anton.tarasov at jetbrains.com Mon Mar 2 15:12:13 2020 From: anton.tarasov at jetbrains.com (Anton Tarasov) Date: Mon, 2 Mar 2020 18:12:13 +0300 Subject: RFR (S): 8238676: jni crashes on accessing it from process exit hook In-Reply-To: <4a511743-152a-dff9-5d19-707dcb53fb03@oracle.com> References: <063252ad-ab02-403a-aa5a-1dcf288ea330@oracle.com> <4a511743-152a-dff9-5d19-707dcb53fb03@oracle.com> Message-ID: Hi David, Thank you for fixing this! Is it possible to backport the fix to jdk11? Thanks, Anton. On Sat, Feb 29, 2020 at 8:16 AM David Holmes wrote: > Hi Gerard, > > Are you okay with this? > > Thanks, > David > > On 26/02/2020 10:11 am, David Holmes wrote: > > Hi Gerard, > > > > Thanks for taking a look at this. > > > > On 26/02/2020 2:45 am, gerard ziemski wrote: > >> hi David, > >> > >> On 2/18/20 8:04 PM, David Holmes wrote: > >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8238676 > >>> webrev: http://cr.openjdk.java.net/~dholmes/8238676/webrev/ > >>> > >>> If an application tries to use JNI from an atexit handler, the > >>> attempt can't succeed (the VM has already logically terminated) but > >>> it should not crash. The JNI Invocation API code was missing some > >>> checks in places and wasn't aware of the possibility of trying to > >>> make calls from the VMThread. > >>> > >>> Testing: > >>> - new test added for the JNI Invocation API > >>> - tiers 1-3 > >>> > >>> Thanks, > >>> David > >> > >> In jni_GetEnv() we have: > >> > >> 4120 if (vm_created == 0) { > >> 4121 *penv = NULL; > >> 4122 ret = JNI_EDETACHED; > >> 4123 return ret; > >> 4124 } > >> > >> but in jni_DetachCurrentThread() we have: > >> > >> 4063 if (vm_created == 0) { > >> 4064 HOTSPOT_JNI_DETACHCURRENTTHREAD_RETURN(JNI_ERR); > >> 4065 return JNI_ERR; > >> 4066 } > >> > >> should lines 4064,4065 perhaps be: > >> > >> 4064 HOTSPOT_JNI_DETACHCURRENTTHREAD_RETURN(JNI_EDETACHED); > >> 4065 return JNI_EDETACHED; > >> > >> to be consistent? > > > > Well ... ideally the JNI spec would have considered the question of > > "what if the JavaVM is no longer live?" and had an JNI_ENOJVM errro > > code. But it doesn't. Nor does it allow for any JNI method to return > > JNI_ERR for unexpected errors situations. So we have to adhere to the > > specifications for each method. > > > > For GetEnv the spec only allows for returning JNI_EDETACHED, > > JNI_EVERSION or JNI_OK. As you can't be attached to a VM that doesn't > > exist then JNI_EDETACHED is the only possible return code - and it isn't > > wrong, it just doesn't tell you why you aren't attached. > > > > If you apply the same logic to DetachCurrentThread, and follow the spec > > then it states > > > > "Trying to detach a thread that is not attached is a no-op." > > > > which suggests we do nothing and report JNI_OK. But I think that would > > be a disservice to the programmer in the case where the JVM is no longer > > live. And we're allowed to return "a suitable JNI error code on failure" > > so that is what I chose to do. And I return JNI_ERR rather than > > JNI_EDETACHED for two reasons: > > > > 1. It is consistent with what we do for attach in the same circumstance: > > > > jint JNICALL jni_AttachCurrentThread(JavaVM *vm, void **penv, void > > *_args) { > > HOTSPOT_JNI_ATTACHCURRENTTHREAD_ENTRY(vm, penv, _args); > > if (vm_created == 0) { > > HOTSPOT_JNI_ATTACHCURRENTTHREAD_RETURN((uint32_t) JNI_ERR); > > return JNI_ERR; > > } > > > > 2. It would be confusing to return JNI_EDETACHED as that means "thread > > detached from the VM" yet the spec says "Trying to detach a thread that > > is not attached is a no-op.". So I need to convey that there is a more > > specific underlying error here. > > > >> Looks good otherwise. > > > > Thanks for the review. > > > > David > > ----- > > > >> > >> cheers > >> > From daniel.daugherty at oracle.com Mon Mar 2 16:33:01 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 2 Mar 2020 11:33:01 -0500 Subject: RFR: 8240263: Assertion-only call in Method::link_method affecting product builds In-Reply-To: <43f27f02-6ddf-7ad3-754e-97e73280772b@oracle.com> References: <6a41ef95-8d49-f694-57b2-8d7ddf3f480b@oracle.com> <9d8dae94-f992-6345-58e6-a28fa2789111@redhat.com> <43f27f02-6ddf-7ad3-754e-97e73280772b@oracle.com> Message-ID: <1f3f2bf0-1c90-1d66-2004-ed5545b63c25@oracle.com> I have a preference for the DEBUG_ONLY style, but that's probably because I just did some of those in the ObjectMonitor subsystem. Either way: Thumbs up! Dan On 3/2/20 5:08 AM, Claes Redestad wrote: > > > On 2020-03-02 10:22, Aleksey Shipilev wrote: >> On 3/2/20 10:22 AM, Claes Redestad wrote: >>> Guarding the resolution by DEBUG_ONLY seem the right thing to do here, >>> which bring a small improvement on startup tests when CDS is enabled. >>> >>> See attached patch[1]. >> >> I believe we usually do this in such case: >> >> #ifdef ASSERT >> ?? address entry = Interpreter::entry_for_cds_method(h_method); >> ?? assert(entry != NULL && entry == _i2i_entry, >> ????????? "should be correctly set during dump time"); >> #endif > > I've seen it done both ways, and have no strong preference. > >> >> It keeps the block easily editable, without looking back at whether >> it interferes with macros. >> >> Otherwise looks fine, assuming release builds still pass. > > Running your suggested variant through tier 1 to make sure. > > Thanks! > > /Claes From daniel.daugherty at oracle.com Mon Mar 2 17:29:03 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 2 Mar 2020 12:29:03 -0500 Subject: RFR(L) 8153224 Monitor deflation prolong safepoints (CR10/v2.10/13-for-jdk15) In-Reply-To: <5a7a7a36-a93c-aa57-93e7-bbd4c58f4d73@oracle.com> References: <62729044-8a22-0e20-0eda-04d47c9ea23c@oracle.com> <313e51c8-b672-bb1c-577a-49868f09e6c1@oracle.com> <1fd54b23-bd35-9b8f-e6f3-6000440d8770@oracle.com> <3fb1a2b5-5aad-eab3-09ba-6f64a2242d30@oracle.com> <38e2d441-11c9-8342-37d5-8030dd06f2f4@oracle.com> <2a8976f7-37e0-03b9-3099-e07464e46512@oracle.com> <5681d640-08c8-3433-0f85-3f23eea69e87@oracle.com> <5a7a7a36-a93c-aa57-93e7-bbd4c58f4d73@oracle.com> Message-ID: Robbin, Thanks for the re-review! Dan On 3/2/20 8:32 AM, Robbin Ehn wrote: > Hi Dan, thanks sticking with it and addressing the comments! > > I think you got all mine, minus those we decided should be handled as > follow-ups. > > I have nothing more, looks good, thanks! > > /Robbin > > On 2/26/20 11:22 PM, Daniel D. Daugherty wrote: >> Greetings, >> >> I have made changes to the Async Monitor Deflation code in response to >> the CR9/v2.09/12-for-jdk14 code review cycle. Thanks to Robbin and >> Erik O. >> for their comments in this round! >> >> With the extraction and push of {8235931,8236035,8235795} to JDK15, the >> Async Monitor Deflation code is back to "just" async deflation changes! >> >> I have attached the change list from CR9 to CR10 instead of putting >> it in >> the body of this email. I've also added a link to the >> CR9-to-CR10-changes >> file to the webrevs so it should be easy to find. >> >> Main bug URL: >> >> ???? JDK-8153224 Monitor deflation prolong safepoints >> ???? https://bugs.openjdk.java.net/browse/JDK-8153224 >> >> The project is currently baselined on jdk-15+11. >> >> Here's the full webrev URL for those folks that want to see all of the >> current Async Monitor Deflation code in one go (v2.10 full): >> >> http://cr.openjdk.java.net/~dcubed/8153224-webrev/13-for-jdk15+11.v2.10.full/ >> >> >> Some folks might want to see just what has changed since the last review >> cycle so here's a webrev for that (v2.10 inc): >> >> http://cr.openjdk.java.net/~dcubed/8153224-webrev/13-for-jdk15+11.v2.10.inc/ >> >> >> Since we backed out the HandshakeAfterDeflateIdleMonitors option and the >> C2 ref_count changes and updated the copyright years, the "inc" >> webrev has >> a bit more noise in it than usual. Sorry about that! >> >> The OpenJDK wiki has been updated for this round of changes: >> >> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >> >> The jdk-15+11 based v2.10 version of the patch has been thru Mach5 >> tier[1-7] >> testing on Oracle's usual set of platforms. Mach5 tier8 is still >> running. >> I'm running the v2.10 patch through my usual set of stress testing on >> Linux-X64 and macOSX. >> >> I'm planning to do a SPECjbb2015 round on the CR10/v2.20/13-for-jdk15 >> bits. >> >> Thanks, in advance, for any questions, comments or suggestions. >> >> Dan >> >> >> On 2/4/20 9:41 AM, Daniel D. Daugherty wrote: >>> Greetings, >>> >>> This project is no longer targeted to JDK14 so this is NOT an urgent >>> code >>> review request. >>> >>> I've extracted the following three fixes from the Async Monitor >>> Deflation >>> project code: >>> >>> ? ? JDK-8235931 add OM_CACHE_LINE_SIZE and use smaller size on >>> SPARCv9 and X64 >>> ? ? https://bugs.openjdk.java.net/browse/JDK-8235931 >>> >>> ? ? JDK-8236035 refactor ObjectMonitor::set_owner() and _owner field >>> setting >>> ? ? https://bugs.openjdk.java.net/browse/JDK-8236035 >>> >>> ? ? JDK-8235795 replace monitor list >>> mux{Acquire,Release}(&gListLock) with spin locks >>> ? ? https://bugs.openjdk.java.net/browse/JDK-8235795 >>> >>> Each of these has been reviewed separately and will be pushed to JDK15 >>> in the near future (possibly by the end of this week). Of course, there >>> were improvements during these review cycles and the purpose of this >>> e-mail is to provided updated webrevs for this fix >>> (CR9/v2.09/12-for-jdk14) >>> within the revised context provided by {8235931, 8236035, 8235795}. >>> >>> Main bug URL: >>> >>> ??? JDK-8153224 Monitor deflation prolong safepoints >>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>> >>> The project is currently baselined on jdk-14+34. >>> >>> Here's the full webrev URL for those folks that want to see all of the >>> current Async Monitor Deflation code along with {8235931, 8236035, >>> 8235795} >>> in one go (v2.09b full): >>> >>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/12-for-jdk14.v2.09b.full/ >>> >>> >>> Compare the open.patch file in 12-for-jdk14.v2.09.full and >>> 12-for-jdk14.v2.09b.full >>> using your favorite file comparison/merge tool to see how Async >>> Monitor Deflation >>> evolved due to {8235931, 8236035, 8235795}. >>> >>> Some folks might want to see just the Async Monitor Deflation code >>> on top of >>> {8235931, 8236035, 8235795} so here's a webrev for that (v2.09b inc): >>> >>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/12-for-jdk14.v2.09b.inc/ >>> >>> >>> These webrevs have gone thru several Mach5 Tier[1-8] runs along with >>> my usual stress testing and SPECjbb2015 testing and there aren't any >>> surprises relative to CR9/v2.09/12-for-jdk14. >>> >>> Thanks, in advance, for any questions, comments or suggestions. >>> >>> Dan >>> >>> >>> On 12/11/19 3:41 PM, Daniel D. Daugherty wrote: >>>> Greetings, >>>> >>>> I have made changes to the Async Monitor Deflation code in response to >>>> the CR8/v2.08/11-for-jdk14 code review cycle. Thanks to David H., >>>> Robbin >>>> and Erik O. for their comments! >>>> >>>> This project is no longer targeted to JDK14 so this is NOT an >>>> urgent code >>>> review request. The primary purpose of this webrev is simply to >>>> close the >>>> CR8/v2.08/11-for-jdk14 code review loop and to let folks see how I >>>> resolved >>>> the code review comments from that round. >>>> >>>> Most of the comments in the CR8/v2.08/11-for-jdk14 code review >>>> cycle were >>>> on the monitor list changes so I'm going to take a look at >>>> extracting those >>>> changes into a standalone patch. Switching from >>>> Thread::muxAcquire(&gListLock) >>>> and Thread::muxRelease(&gListLock) to finer grained internal spin >>>> locks needs >>>> to be thoroughly reviewed and the best way to do that is separately >>>> from the >>>> Async Monitor Deflation changes. Thanks to Coleen for suggesting >>>> doing this >>>> extraction earlier. >>>> >>>> I have attached the change list from CR8 to CR9 instead of putting >>>> it in >>>> the body of this email. I've also added a link to the >>>> CR8-to-CR9-changes >>>> file to the webrevs so it should be easy to find. >>>> >>>> Main bug URL: >>>> >>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>>> >>>> The project is currently baselined on jdk-14+26. >>>> >>>> Here's the full webrev URL for those folks that want to see all of the >>>> current Async Monitor Deflation code in one go (v2.09 full): >>>> >>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/12-for-jdk14.v2.09.full/ >>>> >>>> >>>> Some folks might want to see just what has changed since the last >>>> review >>>> cycle so here's a webrev for that (v2.09 inc): >>>> >>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/12-for-jdk14.v2.09.inc/ >>>> >>>> >>>> The OpenJDK wiki has NOT yet been updated for this round of changes: >>>> >>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>> >>>> The jdk-14+26 based v2.09 version of the patch has been thru Mach5 >>>> tier[1-7] >>>> testing on Oracle's usual set of platforms. Mach5 tier8 is still >>>> running. >>>> A slightly older version of the v2.09 patch has also been through >>>> my usual >>>> set of stress testing on Linux-X64 and macOSX with the addition of >>>> Robbin's >>>> "MoCrazy 1024" test running in parallel on Linux-X64 with the other >>>> tests in >>>> my lab. The "MoCrazy 1024" has been going for > 5 days and 6700+ >>>> iterations >>>> without any failures. >>>> >>>> I'm planning to do a SPECjbb2015 round on the >>>> CR9/v2.09/12-for-jdk14 bits. >>>> >>>> Thanks, in advance, for any questions, comments or suggestions. >>>> >>>> Dan >>>> >>>> >>>> On 11/4/19 4:03 PM, Daniel D. Daugherty wrote: >>>>> Greetings, >>>>> >>>>> I have made changes to the Async Monitor Deflation code in >>>>> response to >>>>> the CR7/v2.07/10-for-jdk14 code review cycle. Thanks to David H., >>>>> Robbin >>>>> and Erik O. for their comments! >>>>> >>>>> JDK14 Rampdown phase one is coming on Dec. 12, 2019 and the Async >>>>> Monitor >>>>> Deflation project needs to push before Nov. 12, 2019 in order to >>>>> allow >>>>> for sufficient bake time for such a big change. Nov. 12 is _next_ >>>>> Tuesday >>>>> so we have 8 days from today to finish this code review cycle and >>>>> push >>>>> this code for JDK14. >>>>> >>>>> Carsten and Roman! Time for you guys to chime in again on the code >>>>> reviews. >>>>> >>>>> I have attached the change list from CR7 to CR8 instead of putting >>>>> it in >>>>> the body of this email. I've also added a link to the >>>>> CR7-to-CR8-changes >>>>> file to the webrevs so it should be easy to find. >>>>> >>>>> Main bug URL: >>>>> >>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>> >>>>> The project is currently baselined on jdk-14+21. >>>>> >>>>> Here's the full webrev URL for those folks that want to see all of >>>>> the >>>>> current Async Monitor Deflation code in one go (v2.08 full): >>>>> >>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/11-for-jdk14.v2.08.full >>>>> >>>>> >>>>> Some folks might want to see just what has changed since the last >>>>> review >>>>> cycle so here's a webrev for that (v2.08 inc): >>>>> >>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/11-for-jdk14.v2.08.inc/ >>>>> >>>>> >>>>> The OpenJDK wiki did not need any changes for this round: >>>>> >>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>> >>>>> The jdk-14+21 based v2.08 version of the patch has been thru Mach5 >>>>> tier[1-8] >>>>> testing on Oracle's usual set of platforms. It has also been >>>>> through my usual >>>>> set of stress testing on Linux-X64, macOSX and Solaris-X64 with >>>>> the addition >>>>> of Robbin's "MoCrazy 1024" test running in parallel with the other >>>>> tests in >>>>> my lab. Some testing is still running, but so far there are no new >>>>> regressions. >>>>> >>>>> I have not yet done a SPECjbb2015 round on the >>>>> CR8/v2.08/11-for-jdk14 bits. >>>>> >>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>> >>>>> Dan >>>>> >>>>> >>>>> On 10/17/19 5:50 PM, Daniel D. Daugherty wrote: >>>>>> Greetings, >>>>>> >>>>>> The Async Monitor Deflation project is reaching the end game. I >>>>>> have no >>>>>> changes planned for the project at this time so all that is left >>>>>> is code >>>>>> review and any changes that results from those reviews. >>>>>> >>>>>> Carsten and Roman! Time for you guys to chime in again on the >>>>>> code reviews. >>>>>> >>>>>> I have attached the list of fixes from CR6 to CR7 instead of >>>>>> putting it >>>>>> in the main body of this email. >>>>>> >>>>>> Main bug URL: >>>>>> >>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>> >>>>>> The project is currently baselined on jdk-14+19. >>>>>> >>>>>> Here's the full webrev URL for those folks that want to see all >>>>>> of the >>>>>> current Async Monitor Deflation code in one go (v2.07 full): >>>>>> >>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/10-for-jdk14.v2.07.full >>>>>> >>>>>> >>>>>> Some folks might want to see just what has changed since the last >>>>>> review >>>>>> cycle so here's a webrev for that (v2.07 inc): >>>>>> >>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/10-for-jdk14.v2.07.inc/ >>>>>> >>>>>> >>>>>> The OpenJDK wiki has been updated to match the >>>>>> CR7/v2.07/10-for-jdk14 changes: >>>>>> >>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>> >>>>>> >>>>>> The jdk-14+18 based v2.07 version of the patch has been thru >>>>>> Mach5 tier[1-8] >>>>>> testing on Oracle's usual set of platforms. It has also been >>>>>> through my usual >>>>>> set of stress testing on Linux-X64, macOSX and Solaris-X64 with >>>>>> the addition >>>>>> of Robbin's "MoCrazy 1024" test running in parallel with the >>>>>> other tests in >>>>>> my lab. >>>>>> >>>>>> The jdk-14+19 based v2.07 version of the patch has been thru >>>>>> Mach5 tier[1-3] >>>>>> test on Oracle's usual set of platforms. Mach5 tier[4-8] are in >>>>>> process. >>>>>> >>>>>> I did another round of SPECjbb2015 testing in Oracle's Aurora >>>>>> Performance lab >>>>>> using using their tuned SPECjbb2015 Linux-X64 G1 configs: >>>>>> >>>>>> ??? - "base" is jdk-14+18 >>>>>> ??? - "v2.07" is the latest version and includes C2 >>>>>> inc_om_ref_count() support >>>>>> ????? on LP64 X64 and the new HandshakeAfterDeflateIdleMonitors >>>>>> option >>>>>> ? ? - "off" is with -XX:-AsyncDeflateIdleMonitors specified >>>>>> ? ? - "handshake" is with -XX:+HandshakeAfterDeflateIdleMonitors >>>>>> specified >>>>>> >>>>>> ???????? hbIR?????????? hbIR >>>>>> ??? (max attempted)? (settled)? max-jOPS? critical-jOPS runtime >>>>>> ??? ---------------? ---------? --------? ------------- ------- >>>>>> ?????????? 34282.00?? 30635.90? 28831.30?????? 20969.20 3841.30 base >>>>>> ?????????? 34282.00?? 30973.00? 29345.80?????? 21025.20 3964.10 >>>>>> v2.07 >>>>>> ?????????? 34282.00?? 31105.60? 29174.30?????? 21074.00 3931.30 >>>>>> v2.07_handshake >>>>>> ?????????? 34282.00?? 30789.70? 27151.60?????? 19839.10 3850.20 >>>>>> v2.07_off >>>>>> >>>>>> ??? - The Aurora Perf comparison tool reports: >>>>>> >>>>>> ??????? Comparison????????????? max-jOPS critical-jOPS >>>>>> ??????? ----------------------? -------------------- >>>>>> -------------------- >>>>>> ??????? base vs 2.07??????????? +1.78% (s, p=0.000) +0.27% (ns, >>>>>> p=0.790) >>>>>> ??????? base vs 2.07_handshake? +1.19% (s, p=0.007) +0.58% (ns, >>>>>> p=0.536) >>>>>> ??????? base vs 2.07_off??????? -5.83% (ns, p=0.394) -5.39% (ns, >>>>>> p=0.347) >>>>>> >>>>>> ??????? (s) - significant? (ns) - not-significant >>>>>> >>>>>> ??? - For historical comparison, the Aurora Perf comparision tool >>>>>> ??????? reported for v2.06 with a baseline of jdk-13+31: >>>>>> >>>>>> ??????? Comparison????????????? max-jOPS critical-jOPS >>>>>> ??????? ----------------------? -------------------- >>>>>> -------------------- >>>>>> ??????? base vs 2.06??????????? -0.32% (ns, p=0.345) +0.71% (ns, >>>>>> p=0.646) >>>>>> ??????? base vs 2.06_off??????? +0.49% (ns, p=0.292) -1.21% (ns, >>>>>> p=0.481) >>>>>> >>>>>> ??????? (s) - significant? (ns) - not-significant >>>>>> >>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>> >>>>>> Dan >>>>>> >>>>>> >>>>>> On 8/28/19 5:02 PM, Daniel D. Daugherty wrote: >>>>>>> Greetings, >>>>>>> >>>>>>> The Async Monitor Deflation project has rebased to JDK14 so it's >>>>>>> time >>>>>>> for our first code review in that new context!! >>>>>>> >>>>>>> I've been focused on changing the monitor list management code >>>>>>> to be >>>>>>> lock-free in order to make SPECjbb2015 happier. Of course with a >>>>>>> change >>>>>>> like that, it takes a while to chase down all the new and wonderful >>>>>>> races. At this point, I have the code back to the same stability >>>>>>> that >>>>>>> I had with CR5/v2.05/8-for-jdk13. >>>>>>> >>>>>>> To lay the ground work for this round of review, I pushed the >>>>>>> following >>>>>>> two fixes to jdk/jdk earlier today: >>>>>>> >>>>>>> ??? JDK-8230184 rename, whitespace, indent and comments changes >>>>>>> in preparation >>>>>>> ? ? ??????????? for lock free Monitor lists >>>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8230184 >>>>>>> >>>>>>> ??? JDK-8230317 serviceability/sa/ClhsdbPrintStatics.java fails >>>>>>> after 8230184 >>>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8230317 >>>>>>> >>>>>>> I have attached the list of fixes from CR5 to CR6 instead of >>>>>>> putting >>>>>>> in the main body of this email. >>>>>>> >>>>>>> Main bug URL: >>>>>>> >>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>> >>>>>>> The project is currently baselined on jdk-14+11 plus the fixes for >>>>>>> JDK-8230184 and JDK-8230317. >>>>>>> >>>>>>> Here's the full webrev URL for those folks that want to see all >>>>>>> of the >>>>>>> current Async Monitor Deflation code in one go (v2.06 full): >>>>>>> >>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.06.full/ >>>>>>> >>>>>>> >>>>>>> >>>>>>> The primary focus of this review cycle is on the lock-free >>>>>>> Monitor List >>>>>>> management changes so here's a webrev for just that patch (v2.06c): >>>>>>> >>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.06c.inc/ >>>>>>> >>>>>>> >>>>>>> The secondary focus of this review cycle is on the bug fixes >>>>>>> that have >>>>>>> been made since CR5/v2.05/8-for-jdk13 so here's a webrev for >>>>>>> just that >>>>>>> patch (v2.06b): >>>>>>> >>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.06b.inc/ >>>>>>> >>>>>>> >>>>>>> The third and final bucket for this review cycle is the rename, >>>>>>> whitespace, >>>>>>> indent and comments changes made in preparation for lock free >>>>>>> Monitor list >>>>>>> management. Almost all of that was extracted into JDK-8230184 >>>>>>> for the >>>>>>> baseline so this bucket now has just a few comment changes >>>>>>> relative to >>>>>>> CR5/v2.05/8-for-jdk13. Here's a webrev for the remainder (v2.06a): >>>>>>> >>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.06a.inc/ >>>>>>> >>>>>>> >>>>>>> >>>>>>> Some folks might want to see just what has changed since the >>>>>>> last review >>>>>>> cycle so here's a webrev for that (v2.06 inc): >>>>>>> >>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.06.inc/ >>>>>>> >>>>>>> >>>>>>> >>>>>>> Last, but not least, some folks might want to see the code >>>>>>> before the >>>>>>> addition of lock-free Monitor List management so here's a webrev >>>>>>> for >>>>>>> that (v2.00 -> v2.05): >>>>>>> >>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.05.inc/ >>>>>>> >>>>>>> >>>>>>> The OpenJDK wiki will need minor updates to match the CR6 changes: >>>>>>> >>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>> >>>>>>> >>>>>>> but that should only be changes to describe per-thread list >>>>>>> async monitor >>>>>>> deflation being done by the ServiceThread. >>>>>>> >>>>>>> (I did update the OpenJDK wiki for the CR5 changes back on >>>>>>> 2019.08.14) >>>>>>> >>>>>>> This version of the patch has been thru Mach5 tier[1-8] testing on >>>>>>> Oracle's usual set of platforms. It has also been through my >>>>>>> usual set >>>>>>> of stress testing on Linux-X64, macOSX and Solaris-X64. >>>>>>> >>>>>>> I did a bunch of SPECjbb2015 testing in Oracle's Aurora >>>>>>> Performance lab >>>>>>> using using their tuned SPECjbb2015 Linux-X64 G1 configs. This >>>>>>> was using >>>>>>> this patch baselined on jdk-13+31 (for stability): >>>>>>> >>>>>>> ????????? hbIR?????????? hbIR >>>>>>> ???? (max attempted)? (settled)? max-jOPS critical-jOPS runtime >>>>>>> ???? ---------------? ---------? -------- ------------- ------- >>>>>>> ??????????? 34282.00?? 28837.20? 27905.20 19817.40 3658.10 base >>>>>>> ??????????? 34965.70?? 29798.80? 27814.90 19959.00 3514.60 v2.06d >>>>>>> ??????????? 34282.00?? 29100.70? 28042.50 19577.00 3701.90 >>>>>>> v2.06d_off >>>>>>> ??????????? 34282.00?? 29218.50? 27562.80 19397.30 3657.60 >>>>>>> v2.06d_ocache >>>>>>> ??????????? 34965.70?? 29838.30? 26512.40 19170.60 3569.90 v2.05 >>>>>>> ??????????? 34282.00?? 28926.10? 27734.00 19835.10 3588.40 >>>>>>> v2.05_off >>>>>>> >>>>>>> The "off" configs are with -XX:-AsyncDeflateIdleMonitors >>>>>>> specified and >>>>>>> the "ocache" config is with 128 byte cache line sizes instead of >>>>>>> 64 byte >>>>>>> cache lines sizes. "v2.06d" is the last set of changes that I >>>>>>> made before >>>>>>> those changes were distributed into the "v2.06a", "v2.06b" and >>>>>>> "v2.06c" >>>>>>> buckets for this review recycle. >>>>>>> >>>>>>> >>>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>>> >>>>>>> Dan >>>>>>> >>>>>>> >>>>>>> On 7/11/19 3:49 PM, Daniel D. Daugherty wrote: >>>>>>>> Greetings, >>>>>>>> >>>>>>>> I've been focused on chasing down and fixing the rare test >>>>>>>> failures >>>>>>>> that only pop up rarely. So this round is primarily fixes for >>>>>>>> races >>>>>>>> with a few additional fixes that came from Karen's review of CR4. >>>>>>>> Thanks Karen! >>>>>>>> >>>>>>>> I have attached the list of fixes from CR4 to CR5 instead of >>>>>>>> putting >>>>>>>> in the main body of this email. >>>>>>>> >>>>>>>> Main bug URL: >>>>>>>> >>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>> >>>>>>>> The project is currently baselined on jdk-13+29. This will >>>>>>>> likely be >>>>>>>> the last JDK13 baseline for this project and I'll roll to the >>>>>>>> JDK14 >>>>>>>> (jdk/jdk) repo soon... >>>>>>>> >>>>>>>> Here's the full webrev URL: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/8-for-jdk13.full/ >>>>>>>> >>>>>>>> >>>>>>>> Here's the incremental webrev URL: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/8-for-jdk13.inc/ >>>>>>>> >>>>>>>> I have not yet checked the OpenJDK wiki to see if it needs any >>>>>>>> updates >>>>>>>> to match the CR5 changes: >>>>>>>> >>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>> >>>>>>>> >>>>>>>> (I did update the OpenJDK wiki for the CR4 changes back on >>>>>>>> 2019.06.26) >>>>>>>> >>>>>>>> This version of the patch has been thru Mach5 tier[1-3] testing on >>>>>>>> Oracle's usual set of platforms. Mach5 tier[4-6] is running now >>>>>>>> and >>>>>>>> Mach5 tier[78] will follow. I'll kick off the usual stress testing >>>>>>>> on Linux-X64, macOSX and Solaris-X64 as those machines become >>>>>>>> available. >>>>>>>> Since I haven't made any performance changes in this round, >>>>>>>> I'll only >>>>>>>> be running SPECjbb2015 to gather the latest monitorinflation logs. >>>>>>>> >>>>>>>> Next up: >>>>>>>> >>>>>>>> - We're still seeing 4-5% lower performance with SPECjbb2015 on >>>>>>>> ? Linux-X64 and we've determined that some of that comes from >>>>>>>> ? contention on the gListLock. So I'm going to investigate >>>>>>>> removing >>>>>>>> ? the gListLock. Yes, another lock free set of changes is coming! >>>>>>>> - Of course, going lock free often causes new races and new >>>>>>>> failures >>>>>>>> ? so that's a good reason for make those changes isolated in their >>>>>>>> ? own round (and not holding up CR5/v2.05/8-for-jdk13 anymore). >>>>>>>> - I finally have a potential fix for the Win* failure with >>>>>>>> gc/g1/humongousObjects/TestHumongousClassLoader.java >>>>>>>> ? but I haven't run it through Mach5 yet so it'll be in the >>>>>>>> next round. >>>>>>>> - Some RTM tests were recently re-enabled in Mach5 and I'm >>>>>>>> seeing some >>>>>>>> ? monitor related failures there. I suspect that I need to go >>>>>>>> take a >>>>>>>> ? look at the C2 RTM macro assembler code and look for things >>>>>>>> that might >>>>>>>> ? conflict if Async Monitor Deflation. If you're interested in >>>>>>>> that kind >>>>>>>> ? of issue, then see the macroAssembler_x86.cpp sanity check >>>>>>>> that I >>>>>>>> ? added in this round! >>>>>>>> >>>>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>>>> >>>>>>>> Dan >>>>>>>> >>>>>>>> >>>>>>>> On 5/26/19 8:30 PM, Daniel D. Daugherty wrote: >>>>>>>>> Greetings, >>>>>>>>> >>>>>>>>> I have a fix for an issue that came up during performance >>>>>>>>> testing. >>>>>>>>> Many thanks to Robbin for diagnosing the issue in his SPECjbb2015 >>>>>>>>> experiments. >>>>>>>>> >>>>>>>>> Here's the list of changes from CR3 to CR4. The list is a bit >>>>>>>>> verbose due to the complexity of the issue, but the changes >>>>>>>>> themselves are not that big. >>>>>>>>> >>>>>>>>> Functional: >>>>>>>>> ? - Change SafepointSynchronize::is_cleanup_needed() from calling >>>>>>>>> ??? ObjectSynchronizer::is_cleanup_needed() to calling >>>>>>>>> ObjectSynchronizer::is_safepoint_deflation_needed(): >>>>>>>>> ??? - is_safepoint_deflation_needed() returns the result of >>>>>>>>> ????? monitors_used_above_threshold() for safepoint based >>>>>>>>> ????? monitor deflation (!AsyncDeflateIdleMonitors). >>>>>>>>> ??? - For AsyncDeflateIdleMonitors, it only returns true if >>>>>>>>> ????? there is a special deflation request, e.g., System.gc() >>>>>>>>> ????? - This solves a bug where there are a bunch of Cleanup >>>>>>>>> ??????? safepoints that simply request async deflation which >>>>>>>>> ??????? keeps the async JavaThreads from making progress on >>>>>>>>> ??????? their async deflation work. >>>>>>>>> ? - Add AsyncDeflationInterval diagnostic option. Description: >>>>>>>>> ????? Async deflate idle monitors every so many milliseconds when >>>>>>>>> ????? MonitorUsedDeflationThreshold is exceeded (0 is off). >>>>>>>>> ? - Replace ObjectSynchronizer::gOmShouldDeflateIdleMonitors() >>>>>>>>> with >>>>>>>>> ObjectSynchronizer::is_async_deflation_needed(): >>>>>>>>> ??? - is_async_deflation_needed() returns true when >>>>>>>>> ????? is_async_cleanup_requested() is true or when >>>>>>>>> ????? monitors_used_above_threshold() is true (but no more >>>>>>>>> often than >>>>>>>>> ????? AsyncDeflationInterval). >>>>>>>>> ??? - if AsyncDeflateIdleMonitors Service_lock->wait() now >>>>>>>>> waits for >>>>>>>>> ????? at most GuaranteedSafepointInterval millis: >>>>>>>>> ????? - This allows is_async_deflation_needed() to be checked at >>>>>>>>> ??????? the same interval as GuaranteedSafepointInterval. >>>>>>>>> ??????? (default is 1000 millis/1 second) >>>>>>>>> ????? - Once is_async_deflation_needed() has returned true, it >>>>>>>>> ??????? generally cannot return true for AsyncDeflationInterval. >>>>>>>>> ??????? This is to prevent async deflation from swamping the >>>>>>>>> ??????? ServiceThread. >>>>>>>>> ? - The ServiceThread still handles async deflation of the global >>>>>>>>> ??? in-use list and now it also marks JavaThreads for async >>>>>>>>> deflation >>>>>>>>> ??? of their in-use lists. >>>>>>>>> ??? - The ServiceThread will check for async deflation work every >>>>>>>>> ????? GuaranteedSafepointInterval. >>>>>>>>> ??? - A safepoint can still cause the ServiceThread to check for >>>>>>>>> ????? async deflation work via is_async_deflation_requested. >>>>>>>>> ? - Refactor code from ObjectSynchronizer::is_cleanup_needed() >>>>>>>>> into >>>>>>>>> ??? monitors_used_above_threshold() and remove >>>>>>>>> is_cleanup_needed(). >>>>>>>>> ? - In addition to System.gc(), the VM_Exit VM op and the final >>>>>>>>> ??? VMThread safepoint now set the is_special_deflation_requested >>>>>>>>> ??? flag to reduce the in-use monitor population that is >>>>>>>>> reported by >>>>>>>>> ObjectSynchronizer::log_in_use_monitor_details() at VM exit. >>>>>>>>> >>>>>>>>> Test update: >>>>>>>>> ? - test/hotspot/gtest/oops/test_markOop.cpp is updated to >>>>>>>>> work with >>>>>>>>> ??? AsyncDeflateIdleMonitors. >>>>>>>>> >>>>>>>>> Collateral: >>>>>>>>> ? - Add/clarify/update some logging messages. >>>>>>>>> >>>>>>>>> Cleanup: >>>>>>>>> ? - Updated comments based on Karen's code review. >>>>>>>>> ? - Change 'special cleanup' -> 'special deflation' and >>>>>>>>> ??? 'async cleanup' -> 'async deflation'. >>>>>>>>> ??? - comment and function name changes >>>>>>>>> ? - Clarify MonitorUsedDeflationThreshold description; >>>>>>>>> >>>>>>>>> >>>>>>>>> Main bug URL: >>>>>>>>> >>>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>>> >>>>>>>>> The project is currently baselined on jdk-13+22. >>>>>>>>> >>>>>>>>> Here's the full webrev URL: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/7-for-jdk13.full/ >>>>>>>>> >>>>>>>>> >>>>>>>>> Here's the incremental webrev URL: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/7-for-jdk13.inc/ >>>>>>>>> >>>>>>>>> >>>>>>>>> I have not updated the OpenJDK wiki to reflect the CR4 changes: >>>>>>>>> >>>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>>> >>>>>>>>> >>>>>>>>> The wiki doesn't say a whole lot about the async deflation >>>>>>>>> invocation >>>>>>>>> mechanism so I have to figure out how to add that content. >>>>>>>>> >>>>>>>>> This version of the patch has been thru Mach5 tier[1-8] >>>>>>>>> testing on >>>>>>>>> Oracle's usual set of platforms. My Solaris-X64 stress kit run is >>>>>>>>> running now. Kitchensink8H on product, fastdebug, and >>>>>>>>> slowdebug bits >>>>>>>>> are running on Linux-X64, MacOSX and Solaris-X64. I still have >>>>>>>>> to run >>>>>>>>> my stress kit on Linux-X64. I still have to run the SPECjbb2015 >>>>>>>>> baseline and CR4 runs on Linux-X64, MacOSX and Solaris-X64. >>>>>>>>> >>>>>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>>>>> >>>>>>>>> Dan >>>>>>>>> >>>>>>>>> On 5/6/19 11:52 AM, Daniel D. Daugherty wrote: >>>>>>>>>> Greetings, >>>>>>>>>> >>>>>>>>>> I had some discussions with Karen about a race that was in the >>>>>>>>>> ObjectMonitor::enter() code in CR2/v2.02/5-for-jdk13. This >>>>>>>>>> race was >>>>>>>>>> theoretical and I had no test failures due to it. The fix is >>>>>>>>>> pretty >>>>>>>>>> simple: remove the special case code for async deflation in the >>>>>>>>>> ObjectMonitor::enter() function and rely solely on the ref_count >>>>>>>>>> for ObjectMonitor::enter() protection. >>>>>>>>>> >>>>>>>>>> During those discussions Karen also floated the idea of using >>>>>>>>>> the >>>>>>>>>> ref_count field instead of the contentions field for the Async >>>>>>>>>> Monitor Deflation protocol. I decided to go ahead and code up >>>>>>>>>> that >>>>>>>>>> change and I have run it through the usual stress and Mach5 >>>>>>>>>> testing >>>>>>>>>> with no issues. It's also known as v2.03 (for those for with the >>>>>>>>>> patches) and as webrev/6-for-jdk13 (for those with webrev URLs). >>>>>>>>>> Sorry for all the names... >>>>>>>>>> >>>>>>>>>> Main bug URL: >>>>>>>>>> >>>>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>>>> >>>>>>>>>> The project is currently baselined on jdk-13+18. >>>>>>>>>> >>>>>>>>>> Here's the full webrev URL: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/6-for-jdk13.full/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Here's the incremental webrev URL: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/6-for-jdk13.inc/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> I have also updated the OpenJDK wiki to reflect the CR3 changes: >>>>>>>>>> >>>>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> This version of the patch has been thru Mach5 tier[1-8] >>>>>>>>>> testing on >>>>>>>>>> Oracle's usual set of platforms. My Solaris-X64 stress kit >>>>>>>>>> run had >>>>>>>>>> no issues. Kitchensink8H on product, fastdebug, and slowdebug >>>>>>>>>> bits >>>>>>>>>> had no failures on Linux-X64; MacOSX fastdebug and slowdebug and >>>>>>>>>> Solaris-X64 release had the usual "Too large time diff" >>>>>>>>>> complaints. >>>>>>>>>> 12 hour Inflate2 runs on product, fastdebug and slowdebug >>>>>>>>>> bits on >>>>>>>>>> Linux-X64, MacOSX and Solaris-X64 had no failures. My Linux-X64 >>>>>>>>>> stress kit is running right now. >>>>>>>>>> >>>>>>>>>> I've done the SPECjbb2015 baseline and CR3 runs. I need to >>>>>>>>>> gather >>>>>>>>>> the results and analyze them. >>>>>>>>>> >>>>>>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>>>>>> >>>>>>>>>> Dan >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 4/25/19 12:38 PM, Daniel D. Daugherty wrote: >>>>>>>>>>> Greetings, >>>>>>>>>>> >>>>>>>>>>> I have a small but important bug fix for the Async Monitor >>>>>>>>>>> Deflation >>>>>>>>>>> project ready to go. It's also known as v2.02 (for those for >>>>>>>>>>> with the >>>>>>>>>>> patches) and as webrev/5-for-jdk13 (for those with webrev >>>>>>>>>>> URLs). Sorry >>>>>>>>>>> for all the names... >>>>>>>>>>> >>>>>>>>>>> JDK-8222295 was pushed to jdk/jdk two days ago so that >>>>>>>>>>> baseline patch >>>>>>>>>>> is out of our hair. >>>>>>>>>>> >>>>>>>>>>> Main bug URL: >>>>>>>>>>> >>>>>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>>>>> >>>>>>>>>>> The project is currently baselined on jdk-13+17. >>>>>>>>>>> >>>>>>>>>>> Here's the full webrev URL: >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/5-for-jdk13.full/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Here's the incremental webrev URL (JDK-8153224): >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/5-for-jdk13.inc/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I still have to update the OpenJDK wiki to reflect the CR2 >>>>>>>>>>> changes: >>>>>>>>>>> >>>>>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> This version of the patch has been thru Mach5 tier[1-6] >>>>>>>>>>> testing on >>>>>>>>>>> Oracle's usual set of platforms. Mach5 tier[7-8] is running >>>>>>>>>>> now. >>>>>>>>>>> My stress kit is running on Solaris-X64 now. Kitchensink8H >>>>>>>>>>> is running >>>>>>>>>>> now on product, fastdebug, and slowdebug bits on Linux-X64, >>>>>>>>>>> MacOSX >>>>>>>>>>> and Solaris-X64. 12 hour Inflate2 runs are running now on >>>>>>>>>>> product, >>>>>>>>>>> fastdebug and slowdebug bits on Linux-X64, MacOSX and >>>>>>>>>>> Solaris-X64. >>>>>>>>>>> I'll start my my stress kit on Linux-X64 sometime on Sunday >>>>>>>>>>> (after >>>>>>>>>>> my jdk-13+18 stress run is done). >>>>>>>>>>> >>>>>>>>>>> I'll do SPECjbb2015 baseline and CR2 runs after all the stress >>>>>>>>>>> testing is done. >>>>>>>>>>> >>>>>>>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>>>>>>> >>>>>>>>>>> Dan >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 4/19/19 11:58 AM, Daniel D. Daugherty wrote: >>>>>>>>>>>> Greetings, >>>>>>>>>>>> >>>>>>>>>>>> I finally have CR1 for the Async Monitor Deflation project >>>>>>>>>>>> ready to >>>>>>>>>>>> go. It's also known as v2.01 (for those for with the >>>>>>>>>>>> patches) and as >>>>>>>>>>>> webrev/4-for-jdk13 (for those with webrev URLs). Sorry for >>>>>>>>>>>> all the >>>>>>>>>>>> names... >>>>>>>>>>>> >>>>>>>>>>>> Main bug URL: >>>>>>>>>>>> >>>>>>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>>>>>> >>>>>>>>>>>> Baseline bug fixes URL: >>>>>>>>>>>> >>>>>>>>>>>> ??? JDK-8222295 more baseline cleanups from Async Monitor >>>>>>>>>>>> Deflation project >>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8222295 >>>>>>>>>>>> >>>>>>>>>>>> The project is currently baselined on jdk-13+15. >>>>>>>>>>>> >>>>>>>>>>>> Here's the webrev for the latest baseline changes >>>>>>>>>>>> (JDK-8222295): >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/4-for-jdk13.8222295 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Here's the full webrev URL (JDK-8153224 only): >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/4-for-jdk13.full/ >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Here's the incremental webrev URL (JDK-8153224): >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/4-for-jdk13.inc/ >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> So I'm looking for reviews for both JDK-8222295 and the >>>>>>>>>>>> latest version >>>>>>>>>>>> of JDK-8153224... >>>>>>>>>>>> >>>>>>>>>>>> I still have to update the OpenJDK wiki to reflect the CR >>>>>>>>>>>> changes: >>>>>>>>>>>> >>>>>>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> This version of the patch has been thru Mach5 tier[1-3] >>>>>>>>>>>> testing on >>>>>>>>>>>> Oracle's usual set of platforms. Mach5 tier[4-6] is running >>>>>>>>>>>> now and >>>>>>>>>>>> Mach5 tier[78] will be run later today. My stress kit on >>>>>>>>>>>> Solaris-X64 >>>>>>>>>>>> is running now. Linux-X64 stress testing will start on >>>>>>>>>>>> Sunday. I'm >>>>>>>>>>>> planning to do Kitchensink runs, SPECjbb2015 runs and my >>>>>>>>>>>> monitor >>>>>>>>>>>> inflation stress tests on Linux-X64, MacOSX and Solaris-X64. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, in advance, for any questions, comments or >>>>>>>>>>>> suggestions. >>>>>>>>>>>> >>>>>>>>>>>> Dan >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 3/24/19 9:57 AM, Daniel D. Daugherty wrote: >>>>>>>>>>>>> Greetings, >>>>>>>>>>>>> >>>>>>>>>>>>> Welcome to the OpenJDK review thread for my port of >>>>>>>>>>>>> Carsten's work on: >>>>>>>>>>>>> >>>>>>>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>>>>>>> >>>>>>>>>>>>> Here's a link to the OpenJDK wiki that describes my port: >>>>>>>>>>>>> >>>>>>>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Here's the webrev URL: >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/3-for-jdk13/ >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Here's a link to Carsten's original webrev: >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~cvarming/monitor_deflate_conc/0/ >>>>>>>>>>>>> >>>>>>>>>>>>> Earlier versions of this patch have been through several >>>>>>>>>>>>> rounds of >>>>>>>>>>>>> preliminary review. Many thanks to Carsten, Coleen, >>>>>>>>>>>>> Robbin, and >>>>>>>>>>>>> Roman for their preliminary code review comments. A very >>>>>>>>>>>>> special >>>>>>>>>>>>> thanks to Robbin and Roman for building and testing the >>>>>>>>>>>>> patch in >>>>>>>>>>>>> their own environments (including specJBB2015). >>>>>>>>>>>>> >>>>>>>>>>>>> This version of the patch has been thru Mach5 tier[1-8] >>>>>>>>>>>>> testing on >>>>>>>>>>>>> Oracle's usual set of platforms. Earlier versions have >>>>>>>>>>>>> been run >>>>>>>>>>>>> through my stress kit on my Linux-X64 and Solaris-X64 servers >>>>>>>>>>>>> (product, fastdebug, slowdebug).Earlier versions have run >>>>>>>>>>>>> Kitchensink >>>>>>>>>>>>> for 12 hours on MacOSX, Linux-X64 and Solaris-X64 >>>>>>>>>>>>> (product, fastdebug >>>>>>>>>>>>> and slowdebug). Earlier versions have run my monitor >>>>>>>>>>>>> inflation stress >>>>>>>>>>>>> tests for 12 hours on MacOSX, Linux-X64 and Solaris-X64 >>>>>>>>>>>>> (product, >>>>>>>>>>>>> fastdebug and slowdebug). >>>>>>>>>>>>> >>>>>>>>>>>>> All of the testing done on earlier versions will be redone >>>>>>>>>>>>> on the >>>>>>>>>>>>> latest version of the patch. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, in advance, for any questions, comments or >>>>>>>>>>>>> suggestions. >>>>>>>>>>>>> >>>>>>>>>>>>> Dan >>>>>>>>>>>>> >>>>>>>>>>>>> P.S. >>>>>>>>>>>>> One subtest in >>>>>>>>>>>>> gc/g1/humongousObjects/TestHumongousClassLoader.java >>>>>>>>>>>>> is currently failing in -Xcomp mode on Win* only. I've >>>>>>>>>>>>> been trying >>>>>>>>>>>>> to characterize/analyze this failure for more than a week >>>>>>>>>>>>> now. At >>>>>>>>>>>>> this point I'm convinced that Async Monitor Deflation is >>>>>>>>>>>>> aggravating >>>>>>>>>>>>> an existing bug. However, I plan to have a better handle >>>>>>>>>>>>> on that >>>>>>>>>>>>> failure before these bits are pushed to the jdk/jdk repo. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> From daniel.daugherty at oracle.com Mon Mar 2 18:02:50 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 2 Mar 2020 13:02:50 -0500 Subject: RFR(L) 8153224 Monitor deflation prolong safepoints (CR10/v2.10/13-for-jdk15) In-Reply-To: <0ce22f0f-a06b-1b04-9e66-24b5c78653b4@oracle.com> References: <62729044-8a22-0e20-0eda-04d47c9ea23c@oracle.com> <313e51c8-b672-bb1c-577a-49868f09e6c1@oracle.com> <1fd54b23-bd35-9b8f-e6f3-6000440d8770@oracle.com> <3fb1a2b5-5aad-eab3-09ba-6f64a2242d30@oracle.com> <38e2d441-11c9-8342-37d5-8030dd06f2f4@oracle.com> <2a8976f7-37e0-03b9-3099-e07464e46512@oracle.com> <5681d640-08c8-3433-0f85-3f23eea69e87@oracle.com> <5a7a7a36-a93c-aa57-93e7-bbd4c58f4d73@oracle.com> <0ce22f0f-a06b-1b04-9e66-24b5c78653b4@oracle.com> Message-ID: <0c0a378f-575f-d1de-373a-b4569c0bfa80@oracle.com> On 3/2/20 9:34 AM, coleen.phillimore at oracle.com wrote: > Hi Dan, Can you post a webrev with the result of Erik's comments? Hi Coleen, Please keep in mind this is preliminary v2.11 and has not had much use or testing yet. These are Erik's three webrevs: > http://cr.openjdk.java.net/~eosterlund/dan_monitors/webrev.00/ ??? I haven't taken the change from this one yet. ??? Aurora perf hasn't run that job set yet. > http://cr.openjdk.java.net/~eosterlund/dan_monitors/webrev.01/ ??? I've taken this change and made some additional changes ??? to the comments. > http://cr.openjdk.java.net/~eosterlund/dan_monitors/webrev.02/ ??? As discussed in my reply to Erik, I took one of the four changes ??? as is. I took the other three changes with changes of my own. ??? My plan to take the first change from this webrev as a conversion ??? from this: ? ? ? ? L438:?? if (object() != obj) { ? ? ? ? L439:???? // ObjectMonitor's object ref no longer refers to the target object ? ? ? ? L440:???? // so the object's header has already been restored. ? ? ? ? L441:???? return; ? ? ? ? L442:?? } ??? to this: ??????????????? ADIM_guarantee(object() == obj, "object=" INTPTR_FORMAT ??????????????? ? ? ? ? ? ?? ? " must equal obj=" INTPTR_FORMAT, p2i(object()), p2i(obj)); ??? did not survive a KitchensinkSanity run. The assertion fired ??? because you can have a race between the deflater thread's call ??? to install_displaced_markword_in_object() followed by clearing ??? the object ref with a thread calling save_om_ptr() which calls ??? install_displaced_markword_in_object() and it can observe the ??? NULL object field compared with 'obj'. ??? The clearing of the object field to NULL happens at ObjectMonitor ??? deflation time _before_ the ObjectMonitor is put on the global ??? free list; see deflate_monitor_using_JT(). The handshake after ??? ObjectMonitors are put on the global free list does not help ??? prevent the race here. ??? So L438-442 are replaced with this code: ??????? L439: ? const oop l_object = (oop)object(); ??????? L440: ? if (l_object == NULL) { ??????? L441: ??? // ObjectMonitor's object ref has already been cleared by async ??????? L442: ??? // deflation so we're done here. ??????? L443: ??? return; ??????? L444: ? } ??????? L445: ? ADIM_guarantee(l_object == obj, "object=" INTPTR_FORMAT " must equal obj=" ??????? L446: ???????????????? INTPTR_FORMAT, p2i(l_object), p2i(obj)); To reiterate: Please keep in mind this is preliminary v2.11 and has not had much use or testing yet. Here's v2.11-prelim1 full: http://cr.openjdk.java.net/~dcubed/8153224-webrev/13-for-jdk15+11.v2.11.prelim1.full/ Here's v2.11-prelim1 inc (relative to v2.10): http://cr.openjdk.java.net/~dcubed/8153224-webrev/13-for-jdk15+11.v2.11.prelim1.inc/ Dan > thanks, > Coleen > > On 3/2/20 8:32 AM, Robbin Ehn wrote: >> Hi Dan, thanks sticking with it and addressing the comments! >> >> I think you got all mine, minus those we decided should be handled as >> follow-ups. >> >> I have nothing more, looks good, thanks! >> >> /Robbin >> >> On 2/26/20 11:22 PM, Daniel D. Daugherty wrote: >>> Greetings, >>> >>> I have made changes to the Async Monitor Deflation code in response to >>> the CR9/v2.09/12-for-jdk14 code review cycle. Thanks to Robbin and >>> Erik O. >>> for their comments in this round! >>> >>> With the extraction and push of {8235931,8236035,8235795} to JDK15, the >>> Async Monitor Deflation code is back to "just" async deflation changes! >>> >>> I have attached the change list from CR9 to CR10 instead of putting >>> it in >>> the body of this email. I've also added a link to the >>> CR9-to-CR10-changes >>> file to the webrevs so it should be easy to find. >>> >>> Main bug URL: >>> >>> ???? JDK-8153224 Monitor deflation prolong safepoints >>> ???? https://bugs.openjdk.java.net/browse/JDK-8153224 >>> >>> The project is currently baselined on jdk-15+11. >>> >>> Here's the full webrev URL for those folks that want to see all of the >>> current Async Monitor Deflation code in one go (v2.10 full): >>> >>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/13-for-jdk15+11.v2.10.full/ >>> >>> >>> Some folks might want to see just what has changed since the last >>> review >>> cycle so here's a webrev for that (v2.10 inc): >>> >>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/13-for-jdk15+11.v2.10.inc/ >>> >>> >>> Since we backed out the HandshakeAfterDeflateIdleMonitors option and >>> the >>> C2 ref_count changes and updated the copyright years, the "inc" >>> webrev has >>> a bit more noise in it than usual. Sorry about that! >>> >>> The OpenJDK wiki has been updated for this round of changes: >>> >>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>> >>> The jdk-15+11 based v2.10 version of the patch has been thru Mach5 >>> tier[1-7] >>> testing on Oracle's usual set of platforms. Mach5 tier8 is still >>> running. >>> I'm running the v2.10 patch through my usual set of stress testing on >>> Linux-X64 and macOSX. >>> >>> I'm planning to do a SPECjbb2015 round on the >>> CR10/v2.20/13-for-jdk15 bits. >>> >>> Thanks, in advance, for any questions, comments or suggestions. >>> >>> Dan >>> >>> >>> On 2/4/20 9:41 AM, Daniel D. Daugherty wrote: >>>> Greetings, >>>> >>>> This project is no longer targeted to JDK14 so this is NOT an >>>> urgent code >>>> review request. >>>> >>>> I've extracted the following three fixes from the Async Monitor >>>> Deflation >>>> project code: >>>> >>>> ? ? JDK-8235931 add OM_CACHE_LINE_SIZE and use smaller size on >>>> SPARCv9 and X64 >>>> ? ? https://bugs.openjdk.java.net/browse/JDK-8235931 >>>> >>>> ? ? JDK-8236035 refactor ObjectMonitor::set_owner() and _owner >>>> field setting >>>> ? ? https://bugs.openjdk.java.net/browse/JDK-8236035 >>>> >>>> ? ? JDK-8235795 replace monitor list >>>> mux{Acquire,Release}(&gListLock) with spin locks >>>> ? ? https://bugs.openjdk.java.net/browse/JDK-8235795 >>>> >>>> Each of these has been reviewed separately and will be pushed to JDK15 >>>> in the near future (possibly by the end of this week). Of course, >>>> there >>>> were improvements during these review cycles and the purpose of this >>>> e-mail is to provided updated webrevs for this fix >>>> (CR9/v2.09/12-for-jdk14) >>>> within the revised context provided by {8235931, 8236035, 8235795}. >>>> >>>> Main bug URL: >>>> >>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>>> >>>> The project is currently baselined on jdk-14+34. >>>> >>>> Here's the full webrev URL for those folks that want to see all of the >>>> current Async Monitor Deflation code along with {8235931, 8236035, >>>> 8235795} >>>> in one go (v2.09b full): >>>> >>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/12-for-jdk14.v2.09b.full/ >>>> >>>> >>>> Compare the open.patch file in 12-for-jdk14.v2.09.full and >>>> 12-for-jdk14.v2.09b.full >>>> using your favorite file comparison/merge tool to see how Async >>>> Monitor Deflation >>>> evolved due to {8235931, 8236035, 8235795}. >>>> >>>> Some folks might want to see just the Async Monitor Deflation code >>>> on top of >>>> {8235931, 8236035, 8235795} so here's a webrev for that (v2.09b inc): >>>> >>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/12-for-jdk14.v2.09b.inc/ >>>> >>>> >>>> These webrevs have gone thru several Mach5 Tier[1-8] runs along with >>>> my usual stress testing and SPECjbb2015 testing and there aren't any >>>> surprises relative to CR9/v2.09/12-for-jdk14. >>>> >>>> Thanks, in advance, for any questions, comments or suggestions. >>>> >>>> Dan >>>> >>>> >>>> On 12/11/19 3:41 PM, Daniel D. Daugherty wrote: >>>>> Greetings, >>>>> >>>>> I have made changes to the Async Monitor Deflation code in >>>>> response to >>>>> the CR8/v2.08/11-for-jdk14 code review cycle. Thanks to David H., >>>>> Robbin >>>>> and Erik O. for their comments! >>>>> >>>>> This project is no longer targeted to JDK14 so this is NOT an >>>>> urgent code >>>>> review request. The primary purpose of this webrev is simply to >>>>> close the >>>>> CR8/v2.08/11-for-jdk14 code review loop and to let folks see how I >>>>> resolved >>>>> the code review comments from that round. >>>>> >>>>> Most of the comments in the CR8/v2.08/11-for-jdk14 code review >>>>> cycle were >>>>> on the monitor list changes so I'm going to take a look at >>>>> extracting those >>>>> changes into a standalone patch. Switching from >>>>> Thread::muxAcquire(&gListLock) >>>>> and Thread::muxRelease(&gListLock) to finer grained internal spin >>>>> locks needs >>>>> to be thoroughly reviewed and the best way to do that is >>>>> separately from the >>>>> Async Monitor Deflation changes. Thanks to Coleen for suggesting >>>>> doing this >>>>> extraction earlier. >>>>> >>>>> I have attached the change list from CR8 to CR9 instead of putting >>>>> it in >>>>> the body of this email. I've also added a link to the >>>>> CR8-to-CR9-changes >>>>> file to the webrevs so it should be easy to find. >>>>> >>>>> Main bug URL: >>>>> >>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>> >>>>> The project is currently baselined on jdk-14+26. >>>>> >>>>> Here's the full webrev URL for those folks that want to see all of >>>>> the >>>>> current Async Monitor Deflation code in one go (v2.09 full): >>>>> >>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/12-for-jdk14.v2.09.full/ >>>>> >>>>> >>>>> Some folks might want to see just what has changed since the last >>>>> review >>>>> cycle so here's a webrev for that (v2.09 inc): >>>>> >>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/12-for-jdk14.v2.09.inc/ >>>>> >>>>> >>>>> The OpenJDK wiki has NOT yet been updated for this round of changes: >>>>> >>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>> >>>>> The jdk-14+26 based v2.09 version of the patch has been thru Mach5 >>>>> tier[1-7] >>>>> testing on Oracle's usual set of platforms. Mach5 tier8 is still >>>>> running. >>>>> A slightly older version of the v2.09 patch has also been through >>>>> my usual >>>>> set of stress testing on Linux-X64 and macOSX with the addition of >>>>> Robbin's >>>>> "MoCrazy 1024" test running in parallel on Linux-X64 with the >>>>> other tests in >>>>> my lab. The "MoCrazy 1024" has been going for > 5 days and 6700+ >>>>> iterations >>>>> without any failures. >>>>> >>>>> I'm planning to do a SPECjbb2015 round on the >>>>> CR9/v2.09/12-for-jdk14 bits. >>>>> >>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>> >>>>> Dan >>>>> >>>>> >>>>> On 11/4/19 4:03 PM, Daniel D. Daugherty wrote: >>>>>> Greetings, >>>>>> >>>>>> I have made changes to the Async Monitor Deflation code in >>>>>> response to >>>>>> the CR7/v2.07/10-for-jdk14 code review cycle. Thanks to David H., >>>>>> Robbin >>>>>> and Erik O. for their comments! >>>>>> >>>>>> JDK14 Rampdown phase one is coming on Dec. 12, 2019 and the Async >>>>>> Monitor >>>>>> Deflation project needs to push before Nov. 12, 2019 in order to >>>>>> allow >>>>>> for sufficient bake time for such a big change. Nov. 12 is _next_ >>>>>> Tuesday >>>>>> so we have 8 days from today to finish this code review cycle and >>>>>> push >>>>>> this code for JDK14. >>>>>> >>>>>> Carsten and Roman! Time for you guys to chime in again on the >>>>>> code reviews. >>>>>> >>>>>> I have attached the change list from CR7 to CR8 instead of >>>>>> putting it in >>>>>> the body of this email. I've also added a link to the >>>>>> CR7-to-CR8-changes >>>>>> file to the webrevs so it should be easy to find. >>>>>> >>>>>> Main bug URL: >>>>>> >>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>> >>>>>> The project is currently baselined on jdk-14+21. >>>>>> >>>>>> Here's the full webrev URL for those folks that want to see all >>>>>> of the >>>>>> current Async Monitor Deflation code in one go (v2.08 full): >>>>>> >>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/11-for-jdk14.v2.08.full >>>>>> >>>>>> >>>>>> Some folks might want to see just what has changed since the last >>>>>> review >>>>>> cycle so here's a webrev for that (v2.08 inc): >>>>>> >>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/11-for-jdk14.v2.08.inc/ >>>>>> >>>>>> >>>>>> The OpenJDK wiki did not need any changes for this round: >>>>>> >>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>> >>>>>> >>>>>> The jdk-14+21 based v2.08 version of the patch has been thru >>>>>> Mach5 tier[1-8] >>>>>> testing on Oracle's usual set of platforms. It has also been >>>>>> through my usual >>>>>> set of stress testing on Linux-X64, macOSX and Solaris-X64 with >>>>>> the addition >>>>>> of Robbin's "MoCrazy 1024" test running in parallel with the >>>>>> other tests in >>>>>> my lab. Some testing is still running, but so far there are no >>>>>> new regressions. >>>>>> >>>>>> I have not yet done a SPECjbb2015 round on the >>>>>> CR8/v2.08/11-for-jdk14 bits. >>>>>> >>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>> >>>>>> Dan >>>>>> >>>>>> >>>>>> On 10/17/19 5:50 PM, Daniel D. Daugherty wrote: >>>>>>> Greetings, >>>>>>> >>>>>>> The Async Monitor Deflation project is reaching the end game. I >>>>>>> have no >>>>>>> changes planned for the project at this time so all that is left >>>>>>> is code >>>>>>> review and any changes that results from those reviews. >>>>>>> >>>>>>> Carsten and Roman! Time for you guys to chime in again on the >>>>>>> code reviews. >>>>>>> >>>>>>> I have attached the list of fixes from CR6 to CR7 instead of >>>>>>> putting it >>>>>>> in the main body of this email. >>>>>>> >>>>>>> Main bug URL: >>>>>>> >>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>> >>>>>>> The project is currently baselined on jdk-14+19. >>>>>>> >>>>>>> Here's the full webrev URL for those folks that want to see all >>>>>>> of the >>>>>>> current Async Monitor Deflation code in one go (v2.07 full): >>>>>>> >>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/10-for-jdk14.v2.07.full >>>>>>> >>>>>>> >>>>>>> Some folks might want to see just what has changed since the >>>>>>> last review >>>>>>> cycle so here's a webrev for that (v2.07 inc): >>>>>>> >>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/10-for-jdk14.v2.07.inc/ >>>>>>> >>>>>>> >>>>>>> The OpenJDK wiki has been updated to match the >>>>>>> CR7/v2.07/10-for-jdk14 changes: >>>>>>> >>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>> >>>>>>> >>>>>>> The jdk-14+18 based v2.07 version of the patch has been thru >>>>>>> Mach5 tier[1-8] >>>>>>> testing on Oracle's usual set of platforms. It has also been >>>>>>> through my usual >>>>>>> set of stress testing on Linux-X64, macOSX and Solaris-X64 with >>>>>>> the addition >>>>>>> of Robbin's "MoCrazy 1024" test running in parallel with the >>>>>>> other tests in >>>>>>> my lab. >>>>>>> >>>>>>> The jdk-14+19 based v2.07 version of the patch has been thru >>>>>>> Mach5 tier[1-3] >>>>>>> test on Oracle's usual set of platforms. Mach5 tier[4-8] are in >>>>>>> process. >>>>>>> >>>>>>> I did another round of SPECjbb2015 testing in Oracle's Aurora >>>>>>> Performance lab >>>>>>> using using their tuned SPECjbb2015 Linux-X64 G1 configs: >>>>>>> >>>>>>> ??? - "base" is jdk-14+18 >>>>>>> ??? - "v2.07" is the latest version and includes C2 >>>>>>> inc_om_ref_count() support >>>>>>> ????? on LP64 X64 and the new HandshakeAfterDeflateIdleMonitors >>>>>>> option >>>>>>> ? ? - "off" is with -XX:-AsyncDeflateIdleMonitors specified >>>>>>> ? ? - "handshake" is with -XX:+HandshakeAfterDeflateIdleMonitors >>>>>>> specified >>>>>>> >>>>>>> ???????? hbIR?????????? hbIR >>>>>>> ??? (max attempted)? (settled)? max-jOPS critical-jOPS runtime >>>>>>> ??? ---------------? ---------? -------- ------------- ------- >>>>>>> ?????????? 34282.00?? 30635.90? 28831.30 20969.20 3841.30 base >>>>>>> ?????????? 34282.00?? 30973.00? 29345.80 21025.20 3964.10 v2.07 >>>>>>> ?????????? 34282.00?? 31105.60? 29174.30 21074.00 3931.30 >>>>>>> v2.07_handshake >>>>>>> ?????????? 34282.00?? 30789.70? 27151.60 19839.10 3850.20 v2.07_off >>>>>>> >>>>>>> ??? - The Aurora Perf comparison tool reports: >>>>>>> >>>>>>> ??????? Comparison????????????? max-jOPS critical-jOPS >>>>>>> ??????? ----------------------? -------------------- >>>>>>> -------------------- >>>>>>> ??????? base vs 2.07??????????? +1.78% (s, p=0.000) +0.27% (ns, >>>>>>> p=0.790) >>>>>>> ??????? base vs 2.07_handshake? +1.19% (s, p=0.007) +0.58% (ns, >>>>>>> p=0.536) >>>>>>> ??????? base vs 2.07_off??????? -5.83% (ns, p=0.394) -5.39% (ns, >>>>>>> p=0.347) >>>>>>> >>>>>>> ??????? (s) - significant? (ns) - not-significant >>>>>>> >>>>>>> ??? - For historical comparison, the Aurora Perf comparision tool >>>>>>> ??????? reported for v2.06 with a baseline of jdk-13+31: >>>>>>> >>>>>>> ??????? Comparison????????????? max-jOPS critical-jOPS >>>>>>> ??????? ----------------------? -------------------- >>>>>>> -------------------- >>>>>>> ??????? base vs 2.06??????????? -0.32% (ns, p=0.345) +0.71% (ns, >>>>>>> p=0.646) >>>>>>> ??????? base vs 2.06_off??????? +0.49% (ns, p=0.292) -1.21% (ns, >>>>>>> p=0.481) >>>>>>> >>>>>>> ??????? (s) - significant? (ns) - not-significant >>>>>>> >>>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>>> >>>>>>> Dan >>>>>>> >>>>>>> >>>>>>> On 8/28/19 5:02 PM, Daniel D. Daugherty wrote: >>>>>>>> Greetings, >>>>>>>> >>>>>>>> The Async Monitor Deflation project has rebased to JDK14 so >>>>>>>> it's time >>>>>>>> for our first code review in that new context!! >>>>>>>> >>>>>>>> I've been focused on changing the monitor list management code >>>>>>>> to be >>>>>>>> lock-free in order to make SPECjbb2015 happier. Of course with >>>>>>>> a change >>>>>>>> like that, it takes a while to chase down all the new and >>>>>>>> wonderful >>>>>>>> races. At this point, I have the code back to the same >>>>>>>> stability that >>>>>>>> I had with CR5/v2.05/8-for-jdk13. >>>>>>>> >>>>>>>> To lay the ground work for this round of review, I pushed the >>>>>>>> following >>>>>>>> two fixes to jdk/jdk earlier today: >>>>>>>> >>>>>>>> ??? JDK-8230184 rename, whitespace, indent and comments changes >>>>>>>> in preparation >>>>>>>> ? ? ??????????? for lock free Monitor lists >>>>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8230184 >>>>>>>> >>>>>>>> ??? JDK-8230317 serviceability/sa/ClhsdbPrintStatics.java fails >>>>>>>> after 8230184 >>>>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8230317 >>>>>>>> >>>>>>>> I have attached the list of fixes from CR5 to CR6 instead of >>>>>>>> putting >>>>>>>> in the main body of this email. >>>>>>>> >>>>>>>> Main bug URL: >>>>>>>> >>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>> >>>>>>>> The project is currently baselined on jdk-14+11 plus the fixes for >>>>>>>> JDK-8230184 and JDK-8230317. >>>>>>>> >>>>>>>> Here's the full webrev URL for those folks that want to see all >>>>>>>> of the >>>>>>>> current Async Monitor Deflation code in one go (v2.06 full): >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.06.full/ >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> The primary focus of this review cycle is on the lock-free >>>>>>>> Monitor List >>>>>>>> management changes so here's a webrev for just that patch >>>>>>>> (v2.06c): >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.06c.inc/ >>>>>>>> >>>>>>>> >>>>>>>> The secondary focus of this review cycle is on the bug fixes >>>>>>>> that have >>>>>>>> been made since CR5/v2.05/8-for-jdk13 so here's a webrev for >>>>>>>> just that >>>>>>>> patch (v2.06b): >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.06b.inc/ >>>>>>>> >>>>>>>> >>>>>>>> The third and final bucket for this review cycle is the rename, >>>>>>>> whitespace, >>>>>>>> indent and comments changes made in preparation for lock free >>>>>>>> Monitor list >>>>>>>> management. Almost all of that was extracted into JDK-8230184 >>>>>>>> for the >>>>>>>> baseline so this bucket now has just a few comment changes >>>>>>>> relative to >>>>>>>> CR5/v2.05/8-for-jdk13. Here's a webrev for the remainder (v2.06a): >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.06a.inc/ >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Some folks might want to see just what has changed since the >>>>>>>> last review >>>>>>>> cycle so here's a webrev for that (v2.06 inc): >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.06.inc/ >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Last, but not least, some folks might want to see the code >>>>>>>> before the >>>>>>>> addition of lock-free Monitor List management so here's a >>>>>>>> webrev for >>>>>>>> that (v2.00 -> v2.05): >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.05.inc/ >>>>>>>> >>>>>>>> >>>>>>>> The OpenJDK wiki will need minor updates to match the CR6 changes: >>>>>>>> >>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>> >>>>>>>> >>>>>>>> but that should only be changes to describe per-thread list >>>>>>>> async monitor >>>>>>>> deflation being done by the ServiceThread. >>>>>>>> >>>>>>>> (I did update the OpenJDK wiki for the CR5 changes back on >>>>>>>> 2019.08.14) >>>>>>>> >>>>>>>> This version of the patch has been thru Mach5 tier[1-8] testing on >>>>>>>> Oracle's usual set of platforms. It has also been through my >>>>>>>> usual set >>>>>>>> of stress testing on Linux-X64, macOSX and Solaris-X64. >>>>>>>> >>>>>>>> I did a bunch of SPECjbb2015 testing in Oracle's Aurora >>>>>>>> Performance lab >>>>>>>> using using their tuned SPECjbb2015 Linux-X64 G1 configs. This >>>>>>>> was using >>>>>>>> this patch baselined on jdk-13+31 (for stability): >>>>>>>> >>>>>>>> ????????? hbIR?????????? hbIR >>>>>>>> ???? (max attempted)? (settled)? max-jOPS critical-jOPS runtime >>>>>>>> ???? ---------------? ---------? -------- ------------- ------- >>>>>>>> ??????????? 34282.00?? 28837.20? 27905.20 19817.40 3658.10 base >>>>>>>> ??????????? 34965.70?? 29798.80? 27814.90 19959.00 3514.60 v2.06d >>>>>>>> ??????????? 34282.00?? 29100.70? 28042.50 19577.00 3701.90 >>>>>>>> v2.06d_off >>>>>>>> ??????????? 34282.00?? 29218.50? 27562.80 19397.30 3657.60 >>>>>>>> v2.06d_ocache >>>>>>>> ??????????? 34965.70?? 29838.30? 26512.40 19170.60 3569.90 v2.05 >>>>>>>> ??????????? 34282.00?? 28926.10? 27734.00 19835.10 3588.40 >>>>>>>> v2.05_off >>>>>>>> >>>>>>>> The "off" configs are with -XX:-AsyncDeflateIdleMonitors >>>>>>>> specified and >>>>>>>> the "ocache" config is with 128 byte cache line sizes instead >>>>>>>> of 64 byte >>>>>>>> cache lines sizes. "v2.06d" is the last set of changes that I >>>>>>>> made before >>>>>>>> those changes were distributed into the "v2.06a", "v2.06b" and >>>>>>>> "v2.06c" >>>>>>>> buckets for this review recycle. >>>>>>>> >>>>>>>> >>>>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>>>> >>>>>>>> Dan >>>>>>>> >>>>>>>> >>>>>>>> On 7/11/19 3:49 PM, Daniel D. Daugherty wrote: >>>>>>>>> Greetings, >>>>>>>>> >>>>>>>>> I've been focused on chasing down and fixing the rare test >>>>>>>>> failures >>>>>>>>> that only pop up rarely. So this round is primarily fixes for >>>>>>>>> races >>>>>>>>> with a few additional fixes that came from Karen's review of CR4. >>>>>>>>> Thanks Karen! >>>>>>>>> >>>>>>>>> I have attached the list of fixes from CR4 to CR5 instead of >>>>>>>>> putting >>>>>>>>> in the main body of this email. >>>>>>>>> >>>>>>>>> Main bug URL: >>>>>>>>> >>>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>>> >>>>>>>>> The project is currently baselined on jdk-13+29. This will >>>>>>>>> likely be >>>>>>>>> the last JDK13 baseline for this project and I'll roll to the >>>>>>>>> JDK14 >>>>>>>>> (jdk/jdk) repo soon... >>>>>>>>> >>>>>>>>> Here's the full webrev URL: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/8-for-jdk13.full/ >>>>>>>>> >>>>>>>>> >>>>>>>>> Here's the incremental webrev URL: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/8-for-jdk13.inc/ >>>>>>>>> >>>>>>>>> >>>>>>>>> I have not yet checked the OpenJDK wiki to see if it needs any >>>>>>>>> updates >>>>>>>>> to match the CR5 changes: >>>>>>>>> >>>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>>> >>>>>>>>> >>>>>>>>> (I did update the OpenJDK wiki for the CR4 changes back on >>>>>>>>> 2019.06.26) >>>>>>>>> >>>>>>>>> This version of the patch has been thru Mach5 tier[1-3] >>>>>>>>> testing on >>>>>>>>> Oracle's usual set of platforms. Mach5 tier[4-6] is running >>>>>>>>> now and >>>>>>>>> Mach5 tier[78] will follow. I'll kick off the usual stress >>>>>>>>> testing >>>>>>>>> on Linux-X64, macOSX and Solaris-X64 as those machines become >>>>>>>>> available. >>>>>>>>> Since I haven't made any performance changes in this round, >>>>>>>>> I'll only >>>>>>>>> be running SPECjbb2015 to gather the latest monitorinflation >>>>>>>>> logs. >>>>>>>>> >>>>>>>>> Next up: >>>>>>>>> >>>>>>>>> - We're still seeing 4-5% lower performance with SPECjbb2015 on >>>>>>>>> ? Linux-X64 and we've determined that some of that comes from >>>>>>>>> ? contention on the gListLock. So I'm going to investigate >>>>>>>>> removing >>>>>>>>> ? the gListLock. Yes, another lock free set of changes is coming! >>>>>>>>> - Of course, going lock free often causes new races and new >>>>>>>>> failures >>>>>>>>> ? so that's a good reason for make those changes isolated in >>>>>>>>> their >>>>>>>>> ? own round (and not holding up CR5/v2.05/8-for-jdk13 anymore). >>>>>>>>> - I finally have a potential fix for the Win* failure with >>>>>>>>> gc/g1/humongousObjects/TestHumongousClassLoader.java >>>>>>>>> ? but I haven't run it through Mach5 yet so it'll be in the >>>>>>>>> next round. >>>>>>>>> - Some RTM tests were recently re-enabled in Mach5 and I'm >>>>>>>>> seeing some >>>>>>>>> ? monitor related failures there. I suspect that I need to go >>>>>>>>> take a >>>>>>>>> ? look at the C2 RTM macro assembler code and look for things >>>>>>>>> that might >>>>>>>>> ? conflict if Async Monitor Deflation. If you're interested in >>>>>>>>> that kind >>>>>>>>> ? of issue, then see the macroAssembler_x86.cpp sanity check >>>>>>>>> that I >>>>>>>>> ? added in this round! >>>>>>>>> >>>>>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>>>>> >>>>>>>>> Dan >>>>>>>>> >>>>>>>>> >>>>>>>>> On 5/26/19 8:30 PM, Daniel D. Daugherty wrote: >>>>>>>>>> Greetings, >>>>>>>>>> >>>>>>>>>> I have a fix for an issue that came up during performance >>>>>>>>>> testing. >>>>>>>>>> Many thanks to Robbin for diagnosing the issue in his >>>>>>>>>> SPECjbb2015 >>>>>>>>>> experiments. >>>>>>>>>> >>>>>>>>>> Here's the list of changes from CR3 to CR4. The list is a bit >>>>>>>>>> verbose due to the complexity of the issue, but the changes >>>>>>>>>> themselves are not that big. >>>>>>>>>> >>>>>>>>>> Functional: >>>>>>>>>> ? - Change SafepointSynchronize::is_cleanup_needed() from >>>>>>>>>> calling >>>>>>>>>> ??? ObjectSynchronizer::is_cleanup_needed() to calling >>>>>>>>>> ObjectSynchronizer::is_safepoint_deflation_needed(): >>>>>>>>>> ??? - is_safepoint_deflation_needed() returns the result of >>>>>>>>>> ????? monitors_used_above_threshold() for safepoint based >>>>>>>>>> ????? monitor deflation (!AsyncDeflateIdleMonitors). >>>>>>>>>> ??? - For AsyncDeflateIdleMonitors, it only returns true if >>>>>>>>>> ????? there is a special deflation request, e.g., System.gc() >>>>>>>>>> ????? - This solves a bug where there are a bunch of Cleanup >>>>>>>>>> ??????? safepoints that simply request async deflation which >>>>>>>>>> ??????? keeps the async JavaThreads from making progress on >>>>>>>>>> ??????? their async deflation work. >>>>>>>>>> ? - Add AsyncDeflationInterval diagnostic option. Description: >>>>>>>>>> ????? Async deflate idle monitors every so many milliseconds >>>>>>>>>> when >>>>>>>>>> ????? MonitorUsedDeflationThreshold is exceeded (0 is off). >>>>>>>>>> ? - Replace >>>>>>>>>> ObjectSynchronizer::gOmShouldDeflateIdleMonitors() with >>>>>>>>>> ObjectSynchronizer::is_async_deflation_needed(): >>>>>>>>>> ??? - is_async_deflation_needed() returns true when >>>>>>>>>> ????? is_async_cleanup_requested() is true or when >>>>>>>>>> ????? monitors_used_above_threshold() is true (but no more >>>>>>>>>> often than >>>>>>>>>> ????? AsyncDeflationInterval). >>>>>>>>>> ??? - if AsyncDeflateIdleMonitors Service_lock->wait() now >>>>>>>>>> waits for >>>>>>>>>> ????? at most GuaranteedSafepointInterval millis: >>>>>>>>>> ????? - This allows is_async_deflation_needed() to be checked at >>>>>>>>>> ??????? the same interval as GuaranteedSafepointInterval. >>>>>>>>>> ??????? (default is 1000 millis/1 second) >>>>>>>>>> ????? - Once is_async_deflation_needed() has returned true, it >>>>>>>>>> ??????? generally cannot return true for AsyncDeflationInterval. >>>>>>>>>> ??????? This is to prevent async deflation from swamping the >>>>>>>>>> ??????? ServiceThread. >>>>>>>>>> ? - The ServiceThread still handles async deflation of the >>>>>>>>>> global >>>>>>>>>> ??? in-use list and now it also marks JavaThreads for async >>>>>>>>>> deflation >>>>>>>>>> ??? of their in-use lists. >>>>>>>>>> ??? - The ServiceThread will check for async deflation work >>>>>>>>>> every >>>>>>>>>> ????? GuaranteedSafepointInterval. >>>>>>>>>> ??? - A safepoint can still cause the ServiceThread to check for >>>>>>>>>> ????? async deflation work via is_async_deflation_requested. >>>>>>>>>> ? - Refactor code from >>>>>>>>>> ObjectSynchronizer::is_cleanup_needed() into >>>>>>>>>> ??? monitors_used_above_threshold() and remove >>>>>>>>>> is_cleanup_needed(). >>>>>>>>>> ? - In addition to System.gc(), the VM_Exit VM op and the final >>>>>>>>>> ??? VMThread safepoint now set the >>>>>>>>>> is_special_deflation_requested >>>>>>>>>> ??? flag to reduce the in-use monitor population that is >>>>>>>>>> reported by >>>>>>>>>> ObjectSynchronizer::log_in_use_monitor_details() at VM exit. >>>>>>>>>> >>>>>>>>>> Test update: >>>>>>>>>> ? - test/hotspot/gtest/oops/test_markOop.cpp is updated to >>>>>>>>>> work with >>>>>>>>>> ??? AsyncDeflateIdleMonitors. >>>>>>>>>> >>>>>>>>>> Collateral: >>>>>>>>>> ? - Add/clarify/update some logging messages. >>>>>>>>>> >>>>>>>>>> Cleanup: >>>>>>>>>> ? - Updated comments based on Karen's code review. >>>>>>>>>> ? - Change 'special cleanup' -> 'special deflation' and >>>>>>>>>> ??? 'async cleanup' -> 'async deflation'. >>>>>>>>>> ??? - comment and function name changes >>>>>>>>>> ? - Clarify MonitorUsedDeflationThreshold description; >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Main bug URL: >>>>>>>>>> >>>>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>>>> >>>>>>>>>> The project is currently baselined on jdk-13+22. >>>>>>>>>> >>>>>>>>>> Here's the full webrev URL: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/7-for-jdk13.full/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Here's the incremental webrev URL: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/7-for-jdk13.inc/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> I have not updated the OpenJDK wiki to reflect the CR4 changes: >>>>>>>>>> >>>>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> The wiki doesn't say a whole lot about the async deflation >>>>>>>>>> invocation >>>>>>>>>> mechanism so I have to figure out how to add that content. >>>>>>>>>> >>>>>>>>>> This version of the patch has been thru Mach5 tier[1-8] >>>>>>>>>> testing on >>>>>>>>>> Oracle's usual set of platforms. My Solaris-X64 stress kit >>>>>>>>>> run is >>>>>>>>>> running now. Kitchensink8H on product, fastdebug, and >>>>>>>>>> slowdebug bits >>>>>>>>>> are running on Linux-X64, MacOSX and Solaris-X64. I still >>>>>>>>>> have to run >>>>>>>>>> my stress kit on Linux-X64. I still have to run the SPECjbb2015 >>>>>>>>>> baseline and CR4 runs on Linux-X64, MacOSX and Solaris-X64. >>>>>>>>>> >>>>>>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>>>>>> >>>>>>>>>> Dan >>>>>>>>>> >>>>>>>>>> On 5/6/19 11:52 AM, Daniel D. Daugherty wrote: >>>>>>>>>>> Greetings, >>>>>>>>>>> >>>>>>>>>>> I had some discussions with Karen about a race that was in the >>>>>>>>>>> ObjectMonitor::enter() code in CR2/v2.02/5-for-jdk13. This >>>>>>>>>>> race was >>>>>>>>>>> theoretical and I had no test failures due to it. The fix is >>>>>>>>>>> pretty >>>>>>>>>>> simple: remove the special case code for async deflation in the >>>>>>>>>>> ObjectMonitor::enter() function and rely solely on the >>>>>>>>>>> ref_count >>>>>>>>>>> for ObjectMonitor::enter() protection. >>>>>>>>>>> >>>>>>>>>>> During those discussions Karen also floated the idea of >>>>>>>>>>> using the >>>>>>>>>>> ref_count field instead of the contentions field for the Async >>>>>>>>>>> Monitor Deflation protocol. I decided to go ahead and code >>>>>>>>>>> up that >>>>>>>>>>> change and I have run it through the usual stress and Mach5 >>>>>>>>>>> testing >>>>>>>>>>> with no issues. It's also known as v2.03 (for those for with >>>>>>>>>>> the >>>>>>>>>>> patches) and as webrev/6-for-jdk13 (for those with webrev >>>>>>>>>>> URLs). >>>>>>>>>>> Sorry for all the names... >>>>>>>>>>> >>>>>>>>>>> Main bug URL: >>>>>>>>>>> >>>>>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>>>>> >>>>>>>>>>> The project is currently baselined on jdk-13+18. >>>>>>>>>>> >>>>>>>>>>> Here's the full webrev URL: >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/6-for-jdk13.full/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Here's the incremental webrev URL: >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/6-for-jdk13.inc/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I have also updated the OpenJDK wiki to reflect the CR3 >>>>>>>>>>> changes: >>>>>>>>>>> >>>>>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> This version of the patch has been thru Mach5 tier[1-8] >>>>>>>>>>> testing on >>>>>>>>>>> Oracle's usual set of platforms. My Solaris-X64 stress kit >>>>>>>>>>> run had >>>>>>>>>>> no issues. Kitchensink8H on product, fastdebug, and >>>>>>>>>>> slowdebug bits >>>>>>>>>>> had no failures on Linux-X64; MacOSX fastdebug and slowdebug >>>>>>>>>>> and >>>>>>>>>>> Solaris-X64 release had the usual "Too large time diff" >>>>>>>>>>> complaints. >>>>>>>>>>> 12 hour Inflate2 runs on product, fastdebug and slowdebug >>>>>>>>>>> bits on >>>>>>>>>>> Linux-X64, MacOSX and Solaris-X64 had no failures. My Linux-X64 >>>>>>>>>>> stress kit is running right now. >>>>>>>>>>> >>>>>>>>>>> I've done the SPECjbb2015 baseline and CR3 runs. I need to >>>>>>>>>>> gather >>>>>>>>>>> the results and analyze them. >>>>>>>>>>> >>>>>>>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>>>>>>> >>>>>>>>>>> Dan >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 4/25/19 12:38 PM, Daniel D. Daugherty wrote: >>>>>>>>>>>> Greetings, >>>>>>>>>>>> >>>>>>>>>>>> I have a small but important bug fix for the Async Monitor >>>>>>>>>>>> Deflation >>>>>>>>>>>> project ready to go. It's also known as v2.02 (for those >>>>>>>>>>>> for with the >>>>>>>>>>>> patches) and as webrev/5-for-jdk13 (for those with webrev >>>>>>>>>>>> URLs). Sorry >>>>>>>>>>>> for all the names... >>>>>>>>>>>> >>>>>>>>>>>> JDK-8222295 was pushed to jdk/jdk two days ago so that >>>>>>>>>>>> baseline patch >>>>>>>>>>>> is out of our hair. >>>>>>>>>>>> >>>>>>>>>>>> Main bug URL: >>>>>>>>>>>> >>>>>>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>>>>>> >>>>>>>>>>>> The project is currently baselined on jdk-13+17. >>>>>>>>>>>> >>>>>>>>>>>> Here's the full webrev URL: >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/5-for-jdk13.full/ >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Here's the incremental webrev URL (JDK-8153224): >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/5-for-jdk13.inc/ >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I still have to update the OpenJDK wiki to reflect the CR2 >>>>>>>>>>>> changes: >>>>>>>>>>>> >>>>>>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> This version of the patch has been thru Mach5 tier[1-6] >>>>>>>>>>>> testing on >>>>>>>>>>>> Oracle's usual set of platforms. Mach5 tier[7-8] is running >>>>>>>>>>>> now. >>>>>>>>>>>> My stress kit is running on Solaris-X64 now. Kitchensink8H >>>>>>>>>>>> is running >>>>>>>>>>>> now on product, fastdebug, and slowdebug bits on Linux-X64, >>>>>>>>>>>> MacOSX >>>>>>>>>>>> and Solaris-X64. 12 hour Inflate2 runs are running now on >>>>>>>>>>>> product, >>>>>>>>>>>> fastdebug and slowdebug bits on Linux-X64, MacOSX and >>>>>>>>>>>> Solaris-X64. >>>>>>>>>>>> I'll start my my stress kit on Linux-X64 sometime on Sunday >>>>>>>>>>>> (after >>>>>>>>>>>> my jdk-13+18 stress run is done). >>>>>>>>>>>> >>>>>>>>>>>> I'll do SPECjbb2015 baseline and CR2 runs after all the stress >>>>>>>>>>>> testing is done. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, in advance, for any questions, comments or >>>>>>>>>>>> suggestions. >>>>>>>>>>>> >>>>>>>>>>>> Dan >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 4/19/19 11:58 AM, Daniel D. Daugherty wrote: >>>>>>>>>>>>> Greetings, >>>>>>>>>>>>> >>>>>>>>>>>>> I finally have CR1 for the Async Monitor Deflation project >>>>>>>>>>>>> ready to >>>>>>>>>>>>> go. It's also known as v2.01 (for those for with the >>>>>>>>>>>>> patches) and as >>>>>>>>>>>>> webrev/4-for-jdk13 (for those with webrev URLs). Sorry for >>>>>>>>>>>>> all the >>>>>>>>>>>>> names... >>>>>>>>>>>>> >>>>>>>>>>>>> Main bug URL: >>>>>>>>>>>>> >>>>>>>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>>>>>>> >>>>>>>>>>>>> Baseline bug fixes URL: >>>>>>>>>>>>> >>>>>>>>>>>>> ??? JDK-8222295 more baseline cleanups from Async Monitor >>>>>>>>>>>>> Deflation project >>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8222295 >>>>>>>>>>>>> >>>>>>>>>>>>> The project is currently baselined on jdk-13+15. >>>>>>>>>>>>> >>>>>>>>>>>>> Here's the webrev for the latest baseline changes >>>>>>>>>>>>> (JDK-8222295): >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/4-for-jdk13.8222295 >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Here's the full webrev URL (JDK-8153224 only): >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/4-for-jdk13.full/ >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Here's the incremental webrev URL (JDK-8153224): >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/4-for-jdk13.inc/ >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> So I'm looking for reviews for both JDK-8222295 and the >>>>>>>>>>>>> latest version >>>>>>>>>>>>> of JDK-8153224... >>>>>>>>>>>>> >>>>>>>>>>>>> I still have to update the OpenJDK wiki to reflect the CR >>>>>>>>>>>>> changes: >>>>>>>>>>>>> >>>>>>>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> This version of the patch has been thru Mach5 tier[1-3] >>>>>>>>>>>>> testing on >>>>>>>>>>>>> Oracle's usual set of platforms. Mach5 tier[4-6] is >>>>>>>>>>>>> running now and >>>>>>>>>>>>> Mach5 tier[78] will be run later today. My stress kit on >>>>>>>>>>>>> Solaris-X64 >>>>>>>>>>>>> is running now. Linux-X64 stress testing will start on >>>>>>>>>>>>> Sunday. I'm >>>>>>>>>>>>> planning to do Kitchensink runs, SPECjbb2015 runs and my >>>>>>>>>>>>> monitor >>>>>>>>>>>>> inflation stress tests on Linux-X64, MacOSX and Solaris-X64. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, in advance, for any questions, comments or >>>>>>>>>>>>> suggestions. >>>>>>>>>>>>> >>>>>>>>>>>>> Dan >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 3/24/19 9:57 AM, Daniel D. Daugherty wrote: >>>>>>>>>>>>>> Greetings, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Welcome to the OpenJDK review thread for my port of >>>>>>>>>>>>>> Carsten's work on: >>>>>>>>>>>>>> >>>>>>>>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>>>>>>>> >>>>>>>>>>>>>> Here's a link to the OpenJDK wiki that describes my port: >>>>>>>>>>>>>> >>>>>>>>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Here's the webrev URL: >>>>>>>>>>>>>> >>>>>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/3-for-jdk13/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Here's a link to Carsten's original webrev: >>>>>>>>>>>>>> >>>>>>>>>>>>>> http://cr.openjdk.java.net/~cvarming/monitor_deflate_conc/0/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> Earlier versions of this patch have been through several >>>>>>>>>>>>>> rounds of >>>>>>>>>>>>>> preliminary review. Many thanks to Carsten, Coleen, >>>>>>>>>>>>>> Robbin, and >>>>>>>>>>>>>> Roman for their preliminary code review comments. A very >>>>>>>>>>>>>> special >>>>>>>>>>>>>> thanks to Robbin and Roman for building and testing the >>>>>>>>>>>>>> patch in >>>>>>>>>>>>>> their own environments (including specJBB2015). >>>>>>>>>>>>>> >>>>>>>>>>>>>> This version of the patch has been thru Mach5 tier[1-8] >>>>>>>>>>>>>> testing on >>>>>>>>>>>>>> Oracle's usual set of platforms. Earlier versions have >>>>>>>>>>>>>> been run >>>>>>>>>>>>>> through my stress kit on my Linux-X64 and Solaris-X64 >>>>>>>>>>>>>> servers >>>>>>>>>>>>>> (product, fastdebug, slowdebug).Earlier versions have run >>>>>>>>>>>>>> Kitchensink >>>>>>>>>>>>>> for 12 hours on MacOSX, Linux-X64 and Solaris-X64 >>>>>>>>>>>>>> (product, fastdebug >>>>>>>>>>>>>> and slowdebug). Earlier versions have run my monitor >>>>>>>>>>>>>> inflation stress >>>>>>>>>>>>>> tests for 12 hours on MacOSX, Linux-X64 and Solaris-X64 >>>>>>>>>>>>>> (product, >>>>>>>>>>>>>> fastdebug and slowdebug). >>>>>>>>>>>>>> >>>>>>>>>>>>>> All of the testing done on earlier versions will be >>>>>>>>>>>>>> redone on the >>>>>>>>>>>>>> latest version of the patch. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, in advance, for any questions, comments or >>>>>>>>>>>>>> suggestions. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Dan >>>>>>>>>>>>>> >>>>>>>>>>>>>> P.S. >>>>>>>>>>>>>> One subtest in >>>>>>>>>>>>>> gc/g1/humongousObjects/TestHumongousClassLoader.java >>>>>>>>>>>>>> is currently failing in -Xcomp mode on Win* only. I've >>>>>>>>>>>>>> been trying >>>>>>>>>>>>>> to characterize/analyze this failure for more than a week >>>>>>>>>>>>>> now. At >>>>>>>>>>>>>> this point I'm convinced that Async Monitor Deflation is >>>>>>>>>>>>>> aggravating >>>>>>>>>>>>>> an existing bug. However, I plan to have a better handle >>>>>>>>>>>>>> on that >>>>>>>>>>>>>> failure before these bits are pushed to the jdk/jdk repo. >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> > From ioi.lam at oracle.com Mon Mar 2 18:36:23 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 2 Mar 2020 10:36:23 -0800 Subject: RFR: 8240263: Assertion-only call in Method::link_method affecting product builds In-Reply-To: <1f3f2bf0-1c90-1d66-2004-ed5545b63c25@oracle.com> References: <6a41ef95-8d49-f694-57b2-8d7ddf3f480b@oracle.com> <9d8dae94-f992-6345-58e6-a28fa2789111@redhat.com> <43f27f02-6ddf-7ad3-754e-97e73280772b@oracle.com> <1f3f2bf0-1c90-1d66-2004-ed5545b63c25@oracle.com> Message-ID: <4cd3d4af-fd37-77cd-f804-381079459d40@oracle.com> Looks good to me too. Thanks for finding this performance problem! Thanks - Ioi On 3/2/20 8:33 AM, Daniel D. Daugherty wrote: > I have a preference for the DEBUG_ONLY style, but that's probably > because I just did some of those in the ObjectMonitor subsystem. > > Either way: Thumbs up! > > Dan > > On 3/2/20 5:08 AM, Claes Redestad wrote: >> >> >> On 2020-03-02 10:22, Aleksey Shipilev wrote: >>> On 3/2/20 10:22 AM, Claes Redestad wrote: >>>> Guarding the resolution by DEBUG_ONLY seem the right thing to do here, >>>> which bring a small improvement on startup tests when CDS is enabled. >>>> >>>> See attached patch[1]. >>> >>> I believe we usually do this in such case: >>> >>> #ifdef ASSERT >>> ?? address entry = Interpreter::entry_for_cds_method(h_method); >>> ?? assert(entry != NULL && entry == _i2i_entry, >>> ????????? "should be correctly set during dump time"); >>> #endif >> >> I've seen it done both ways, and have no strong preference. >> >>> >>> It keeps the block easily editable, without looking back at whether >>> it interferes with macros. >>> >>> Otherwise looks fine, assuming release builds still pass. >> >> Running your suggested variant through tier 1 to make sure. >> >> Thanks! >> >> /Claes > From ioi.lam at oracle.com Mon Mar 2 18:41:49 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 2 Mar 2020 10:41:49 -0800 Subject: RFR: 8240197: Cannot start JVM when $JAVA_HOME includes CJK characters In-Reply-To: References: <6ec2f649-8999-fdd5-c2df-465925c3663b@oss.nttdata.com> Message-ID: On 3/2/20 5:24 AM, Schmelter, Ralf wrote: > Hi Yasumasa, > > thanks for your patch. I have some remarks: > > 1. wcstombs_s creates UTF-16, as does MultiByteToWideChar. That wasn't the problem. The problem was that wcstombs_s uses the current locale, which is "C" initially. One is normally expected to set the locale to the system locale via setlocale(LC_ALL, ""), as it is done in ParseLocale() on Unix. > > One could fix the original problem by setting the locale (probably not a good idea, since this changes the locale of the whole process). Or one could specify the correct locale to wcstombs like: > > static _locale_t loc = _create_locale(LC_ALL, ""); > err = ::_mbstowcs_s_l(&converted_chars, path_start, buf_len + 1, buf, buf_len, loc); > > Both fix the problem. But I agree, using the MultiByteToWideChar method is better, since you don't have to create the locale. > > There are some other places in the code where mbstowcs is used on Windows (canonicalize_md.c for example). They should probably fixed in the same way. > > 2.I don't think that using strlen(buf) as the number of wchars to allocate for the converted string is really wrong. There are no code pages supported by Java which would map a single byte to a Unicode character > U+ffff. But since one can get the always correct value via MultiByteToWideChar, this is futureproof and obviously right. > > 3. I would not use the CP_THREAD_ACP code page in MultiByteToWideChar and instead use CP_ACP, because the latter always uses the system locale. And this is the locale used to convert the arguments to bytes in the first place. > > 4. You replaced the malloc/memcpy for the string which will be used to call native_path() with a strdup. This leads to a memory overwrite, when a path like "C:" is used, since native_path() will convert it to "C:.". And the debug version of the VM will detect this, since the tail guard will be partially overwritten. You can check this by running: > > make run-test TEST=gtest:os_windows > > 5. Using WCHAR and LPWSTR seems odd to me. Nobody uses LPCSTR or CHAR for char* and char. But if you think WCHAR is better, I would adjust the return value of the method and the functions which use it too. > > 6. When you handle a relative path, you now only allocate enough space to hold the relative path, not the generated absolute path. Since a relative path is likely to be smaller than the absolute path, the _wfullpath call later will likely fail. That was the reason we used JVM_MAXPATHLEN for relative paths. You could fix this like in this patch http://cr.openjdk.java.net/~rschmelter/webrevs/tmp/unicode_convert.patch Regarding #6, we have test cases under test/hotspot/jtreg/runtime/cds/appcds that use relative paths. Please run these tests when validating the patch. Thanks - Ioi > > Best regards, > Ralf From david.holmes at oracle.com Mon Mar 2 22:56:15 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 3 Mar 2020 08:56:15 +1000 Subject: RFR (S): 8238676: jni crashes on accessing it from process exit hook In-Reply-To: References: <063252ad-ab02-403a-aa5a-1dcf288ea330@oracle.com> <4a511743-152a-dff9-5d19-707dcb53fb03@oracle.com> Message-ID: <2ac96b12-1010-311e-69e7-fa19c8528e65@oracle.com> Hi Anton, On 3/03/2020 1:12 am, Anton Tarasov wrote: > Hi David, > > Thank you for fixing this! Is it possible to backport the fix to jdk11? I can tag it for backporting once it is pushed, but I can't do the actual backport for OpenJDK 11u. David > Thanks, > Anton. > > On Sat, Feb 29, 2020 at 8:16 AM David Holmes > wrote: > > Hi Gerard, > > Are you okay with this? > > Thanks, > David > > On 26/02/2020 10:11 am, David Holmes wrote: > > Hi Gerard, > > > > Thanks for taking a look at this. > > > > On 26/02/2020 2:45 am, gerard ziemski wrote: > >> hi David, > >> > >> On 2/18/20 8:04 PM, David Holmes wrote: > >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8238676 > >>> webrev: http://cr.openjdk.java.net/~dholmes/8238676/webrev/ > >>> > >>> If an application tries to use JNI from an atexit handler, the > >>> attempt can't succeed (the VM has already logically terminated) > but > >>> it should not crash. The JNI Invocation API code was missing some > >>> checks in places and wasn't aware of the possibility of trying to > >>> make calls from the VMThread. > >>> > >>> Testing: > >>> ?- new test added for the JNI Invocation API > >>> ?- tiers 1-3 > >>> > >>> Thanks, > >>> David > >> > >> In jni_GetEnv() we have: > >> > >> 4120?? if (vm_created == 0) { > >> 4121???? *penv = NULL; > >> 4122???? ret = JNI_EDETACHED; > >> 4123???? return ret; > >> 4124?? } > >> > >> but in jni_DetachCurrentThread() we have: > >> > >> 4063?? if (vm_created == 0) { > >> 4064???? HOTSPOT_JNI_DETACHCURRENTTHREAD_RETURN(JNI_ERR); > >> 4065???? return JNI_ERR; > >> 4066?? } > >> > >> should lines 4064,4065 perhaps be: > >> > >> 4064???? HOTSPOT_JNI_DETACHCURRENTTHREAD_RETURN(JNI_EDETACHED); > >> 4065???? return JNI_EDETACHED; > >> > >> to be consistent? > > > > Well ... ideally the JNI spec would have considered the question of > > "what if the JavaVM is no longer live?" and had an JNI_ENOJVM errro > > code. But it doesn't. Nor does it allow for any JNI method to return > > JNI_ERR for unexpected errors situations. So we have to adhere to > the > > specifications for each method. > > > > For GetEnv the spec only allows for returning JNI_EDETACHED, > > JNI_EVERSION or JNI_OK. As you can't be attached to a VM that > doesn't > > exist then JNI_EDETACHED is the only possible return code - and > it isn't > > wrong, it just doesn't tell you why you aren't attached. > > > > If you apply the same logic to DetachCurrentThread, and follow > the spec > > then it states > > > > "Trying to detach a thread that is not attached is a no-op." > > > > which suggests we do nothing and report JNI_OK. But I think that > would > > be a disservice to the programmer in the case where the JVM is no > longer > > live. And we're allowed to return "a suitable JNI error code on > failure" > > so that is what I chose to do. And I return JNI_ERR rather than > > JNI_EDETACHED for two reasons: > > > > 1. It is consistent with what we do for attach in the same > circumstance: > > > > jint JNICALL jni_AttachCurrentThread(JavaVM *vm, void **penv, void > > *_args) { > >? ? HOTSPOT_JNI_ATTACHCURRENTTHREAD_ENTRY(vm, penv, _args); > >? ? if (vm_created == 0) { > >? ??? HOTSPOT_JNI_ATTACHCURRENTTHREAD_RETURN((uint32_t) JNI_ERR); > >? ??? return JNI_ERR; > >? ? } > > > > 2. It would be confusing to return JNI_EDETACHED as that means > "thread > > detached from the VM" yet the spec says "Trying to detach a > thread that > > is not attached is a no-op.". So I need to convey that there is a > more > > specific underlying error here. > > > >> Looks good otherwise. > > > > Thanks for the review. > > > > David > > ----- > > > >> > >> cheers > >> > From gerard.ziemski at oracle.com Mon Mar 2 23:13:06 2020 From: gerard.ziemski at oracle.com (gerard ziemski) Date: Mon, 2 Mar 2020 17:13:06 -0600 Subject: RFR (S): 8238676: jni crashes on accessing it from process exit hook In-Reply-To: <4a511743-152a-dff9-5d19-707dcb53fb03@oracle.com> References: <063252ad-ab02-403a-aa5a-1dcf288ea330@oracle.com> <4a511743-152a-dff9-5d19-707dcb53fb03@oracle.com> Message-ID: Sorry, yes I am OK with the change. I think there is some room to interpretation here, and I'd have done it slightly differently, but I'm fine with your interpretation. cheers On 2/28/20 11:16 PM, David Holmes wrote: > Hi Gerard, > > Are you okay with this? > > Thanks, > David > > On 26/02/2020 10:11 am, David Holmes wrote: >> Hi Gerard, >> >> Thanks for taking a look at this. >> >> On 26/02/2020 2:45 am, gerard ziemski wrote: >>> hi David, >>> >>> On 2/18/20 8:04 PM, David Holmes wrote: >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8238676 >>>> webrev: http://cr.openjdk.java.net/~dholmes/8238676/webrev/ >>>> >>>> If an application tries to use JNI from an atexit handler, the >>>> attempt can't succeed (the VM has already logically terminated) but >>>> it should not crash. The JNI Invocation API code was missing some >>>> checks in places and wasn't aware of the possibility of trying to >>>> make calls from the VMThread. >>>> >>>> Testing: >>>> ?- new test added for the JNI Invocation API >>>> ?- tiers 1-3 >>>> >>>> Thanks, >>>> David >>> >>> In jni_GetEnv() we have: >>> >>> 4120?? if (vm_created == 0) { >>> 4121???? *penv = NULL; >>> 4122???? ret = JNI_EDETACHED; >>> 4123???? return ret; >>> 4124?? } >>> >>> but in jni_DetachCurrentThread() we have: >>> >>> 4063?? if (vm_created == 0) { >>> 4064???? HOTSPOT_JNI_DETACHCURRENTTHREAD_RETURN(JNI_ERR); >>> 4065???? return JNI_ERR; >>> 4066?? } >>> >>> should lines 4064,4065 perhaps be: >>> >>> 4064 HOTSPOT_JNI_DETACHCURRENTTHREAD_RETURN(JNI_EDETACHED); >>> 4065???? return JNI_EDETACHED; >>> >>> to be consistent? >> >> Well ... ideally the JNI spec would have considered the question of >> "what if the JavaVM is no longer live?" and had an JNI_ENOJVM errro >> code. But it doesn't. Nor does it allow for any JNI method to return >> JNI_ERR for unexpected errors situations. So we have to adhere to the >> specifications for each method. >> >> For GetEnv the spec only allows for returning JNI_EDETACHED, >> JNI_EVERSION or JNI_OK. As you can't be attached to a VM that doesn't >> exist then JNI_EDETACHED is the only possible return code - and it >> isn't wrong, it just doesn't tell you why you aren't attached. >> >> If you apply the same logic to DetachCurrentThread, and follow the >> spec then it states >> >> "Trying to detach a thread that is not attached is a no-op." >> >> which suggests we do nothing and report JNI_OK. But I think that >> would be a disservice to the programmer in the case where the JVM is >> no longer live. And we're allowed to return "a suitable JNI error >> code on failure" so that is what I chose to do. And I return JNI_ERR >> rather than JNI_EDETACHED for two reasons: >> >> 1. It is consistent with what we do for attach in the same circumstance: >> >> jint JNICALL jni_AttachCurrentThread(JavaVM *vm, void **penv, void >> *_args) { >> ?? HOTSPOT_JNI_ATTACHCURRENTTHREAD_ENTRY(vm, penv, _args); >> ?? if (vm_created == 0) { >> ???? HOTSPOT_JNI_ATTACHCURRENTTHREAD_RETURN((uint32_t) JNI_ERR); >> ???? return JNI_ERR; >> ?? } >> >> 2. It would be confusing to return JNI_EDETACHED as that means >> "thread detached from the VM" yet the spec says "Trying to detach a >> thread that is not attached is a no-op.". So I need to convey that >> there is a more specific underlying error here. >> >>> Looks good otherwise. >> >> Thanks for the review. >> >> David >> ----- >> >>> >>> cheers >>> From david.holmes at oracle.com Mon Mar 2 23:28:48 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 3 Mar 2020 09:28:48 +1000 Subject: RFR (S): 8238676: jni crashes on accessing it from process exit hook In-Reply-To: References: <063252ad-ab02-403a-aa5a-1dcf288ea330@oracle.com> <4a511743-152a-dff9-5d19-707dcb53fb03@oracle.com> Message-ID: On 3/03/2020 9:13 am, gerard ziemski wrote: > Sorry, yes I am OK with the change. > > I think there is some room to interpretation here, and I'd have done it > slightly differently, but I'm fine with your interpretation. Thanks Gerard! David > > cheers > > On 2/28/20 11:16 PM, David Holmes wrote: >> Hi Gerard, >> >> Are you okay with this? >> >> Thanks, >> David >> >> On 26/02/2020 10:11 am, David Holmes wrote: >>> Hi Gerard, >>> >>> Thanks for taking a look at this. >>> >>> On 26/02/2020 2:45 am, gerard ziemski wrote: >>>> hi David, >>>> >>>> On 2/18/20 8:04 PM, David Holmes wrote: >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8238676 >>>>> webrev: http://cr.openjdk.java.net/~dholmes/8238676/webrev/ >>>>> >>>>> If an application tries to use JNI from an atexit handler, the >>>>> attempt can't succeed (the VM has already logically terminated) but >>>>> it should not crash. The JNI Invocation API code was missing some >>>>> checks in places and wasn't aware of the possibility of trying to >>>>> make calls from the VMThread. >>>>> >>>>> Testing: >>>>> ?- new test added for the JNI Invocation API >>>>> ?- tiers 1-3 >>>>> >>>>> Thanks, >>>>> David >>>> >>>> In jni_GetEnv() we have: >>>> >>>> 4120?? if (vm_created == 0) { >>>> 4121???? *penv = NULL; >>>> 4122???? ret = JNI_EDETACHED; >>>> 4123???? return ret; >>>> 4124?? } >>>> >>>> but in jni_DetachCurrentThread() we have: >>>> >>>> 4063?? if (vm_created == 0) { >>>> 4064???? HOTSPOT_JNI_DETACHCURRENTTHREAD_RETURN(JNI_ERR); >>>> 4065???? return JNI_ERR; >>>> 4066?? } >>>> >>>> should lines 4064,4065 perhaps be: >>>> >>>> 4064 HOTSPOT_JNI_DETACHCURRENTTHREAD_RETURN(JNI_EDETACHED); >>>> 4065???? return JNI_EDETACHED; >>>> >>>> to be consistent? >>> >>> Well ... ideally the JNI spec would have considered the question of >>> "what if the JavaVM is no longer live?" and had an JNI_ENOJVM errro >>> code. But it doesn't. Nor does it allow for any JNI method to return >>> JNI_ERR for unexpected errors situations. So we have to adhere to the >>> specifications for each method. >>> >>> For GetEnv the spec only allows for returning JNI_EDETACHED, >>> JNI_EVERSION or JNI_OK. As you can't be attached to a VM that doesn't >>> exist then JNI_EDETACHED is the only possible return code - and it >>> isn't wrong, it just doesn't tell you why you aren't attached. >>> >>> If you apply the same logic to DetachCurrentThread, and follow the >>> spec then it states >>> >>> "Trying to detach a thread that is not attached is a no-op." >>> >>> which suggests we do nothing and report JNI_OK. But I think that >>> would be a disservice to the programmer in the case where the JVM is >>> no longer live. And we're allowed to return "a suitable JNI error >>> code on failure" so that is what I chose to do. And I return JNI_ERR >>> rather than JNI_EDETACHED for two reasons: >>> >>> 1. It is consistent with what we do for attach in the same circumstance: >>> >>> jint JNICALL jni_AttachCurrentThread(JavaVM *vm, void **penv, void >>> *_args) { >>> ?? HOTSPOT_JNI_ATTACHCURRENTTHREAD_ENTRY(vm, penv, _args); >>> ?? if (vm_created == 0) { >>> ???? HOTSPOT_JNI_ATTACHCURRENTTHREAD_RETURN((uint32_t) JNI_ERR); >>> ???? return JNI_ERR; >>> ?? } >>> >>> 2. It would be confusing to return JNI_EDETACHED as that means >>> "thread detached from the VM" yet the spec says "Trying to detach a >>> thread that is not attached is a no-op.". So I need to convey that >>> there is a more specific underlying error here. >>> >>>> Looks good otherwise. >>> >>> Thanks for the review. >>> >>> David >>> ----- >>> >>>> >>>> cheers >>>> > From vladimir.kozlov at oracle.com Tue Mar 3 00:11:15 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 2 Mar 2020 16:11:15 -0800 Subject: RFR: 8240302: x64: Assembler::reachable redundantly call Relocation::type() more than once In-Reply-To: References: Message-ID: Good cleanup. Thanks, Vladimir On 3/2/20 7:07 AM, Claes Redestad wrote: > Hi, > > in the x64-version Assembler::reachable method, various calls to > adr.reloc() are not hoisted by the compiler, due calling into the > virtual method Relocation::type(). This means each call to reachable > typically do several redundant calls into Relocation::type() > > This patch refactors all these calls into a single one, which has a > small, but measurable, effect on startup (and likely some small > effect on JIT compiler throughput in general) > > Webrev: http://cr.openjdk.java.net/~redestad/8240302/open.00/ > Bug:??? https://bugs.openjdk.java.net/browse/JDK-8240302 > > Testing: tier1-3 > > Thanks! > > /Claes From david.holmes at oracle.com Tue Mar 3 01:11:02 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 3 Mar 2020 11:11:02 +1000 Subject: RFR(S): 8240295: hs_err elapsed time in seconds is not accurate enough In-Reply-To: References: <4a3d7c8e-df9e-51de-77e4-775be1480392@oracle.com> Message-ID: <92ada19d-7200-6bb7-b174-9da8a166aea7@oracle.com> Hi Kevin, On 2/03/2020 8:48 pm, Kevin Walls wrote: > Oops, and with the bug ID in the title and JBS link: > https://bugs.openjdk.java.net/browse/JDK-8240295 > > > On 02/03/2020 10:47, Kevin Walls wrote: >> Hi, >> >> (s11y and runtime opinions both relevant) >> >> A few times in the last month I've really wanted to compare the Events >> logged in the hs_err file, and the time of the JVM's crash. >> >> "elapsed time" in hs_err is only accurate to one second, and has been >> since before jdk5 was created. >> >> The diff below changes the format string and uses the non-rounded time >> value (I don't see a need to change the other integer arithmetic >> here), and we can enjoy hs_errs with detail like: >> >> ... >> Time: Mon Mar? 2 09:57:13 2020 UTC elapsed time: 5.490135 seconds (0d >> 0h 0m 5s) >> ... >> >> Thanks >> Kevin >> >> >> /jdk/open$ hg diff >> diff --git a/src/hotspot/share/runtime/os.cpp >> b/src/hotspot/share/runtime/os.cpp >> --- a/src/hotspot/share/runtime/os.cpp >> +++ b/src/hotspot/share/runtime/os.cpp >> @@ -1016,9 +1016,8 @@ >> ?? } >> >> ?? double t = os::elapsedTime(); >> -? // NOTE: It tends to crash after a SEGV if we want to >> printf("%f",...) in >> -? //?????? Linux. Must be a bug in glibc ? Workaround is to round "t" >> to int >> -? //?????? before printf. We lost some precision, but who cares? >> +? // NOTE: a crash using printf("%f",...) on Linux was historically >> noted here >> +? //?????? (before the jdk5 repo was created). Just because it is old doesn't mean it no longer applies. printf is not async-signal-safe - we know that but we try to use it anyway. Maybe %f is even less async-signal-safe? This may get through testing okay but cause problems with real crashes in the field. What about breaking the time up into two ints: seconds and nanos? Cheers, David ----- >> ?? int eltime = (int)t;? // elapsed time in seconds >> >> ?? // print elapsed time in a human-readable format: >> @@ -1029,7 +1028,7 @@ >> ?? int elmins = (eltime - day_secs - hour_secs) / secs_per_min; >> ?? int minute_secs = elmins * secs_per_min; >> ?? int elsecs = (eltime - day_secs - hour_secs - minute_secs); >> -? st->print_cr(" elapsed time: %d seconds (%dd %dh %dm %ds)", eltime, >> eldays, elhours, elmins, elsecs); >> +? st->print_cr(" elapsed time: %f seconds (%dd %dh %dm %ds)", t, >> eldays, elhours, elmins, elsecs); >> ?} >> >> From ioi.lam at oracle.com Tue Mar 3 02:25:14 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 2 Mar 2020 18:25:14 -0800 Subject: RFR: 8240302: x64: Assembler::reachable redundantly call Relocation::type() more than once In-Reply-To: References: Message-ID: <79cfd32b-44ca-42f9-e6ef-e0edb6e14f09@oracle.com> Looks good to me, too. I was wondering if the C compiler would be a little smarter if Relocation::type() was declared as a const function, like: - virtual relocInfo::relocType type()?????? { return relocInfo::none; } + virtual relocInfo::relocType type() const { return relocInfo::none; } But apparently GCC will not elide the multiple calls: // test.cpp class X { public: virtual int get() const { return 1; } }; class Y : public X { public:? virtual int get() const { return 2; } }; X* xx = 0; int testfunc(char* argv[]) { ? X* x = xx; ? return x->get() + x->get() + x->get(); } $ gcc -O -S -o - test.cpp | grep call ??? call??? *(%rax) ??? call??? *(%rax) ??? call??? *(%rax) So it looks like we must manually save the return value of type() as done in your patch. Thanks - Ioi On 3/2/20 4:11 PM, Vladimir Kozlov wrote: > Good cleanup. > > Thanks, > Vladimir > > On 3/2/20 7:07 AM, Claes Redestad wrote: >> Hi, >> >> in the x64-version Assembler::reachable method, various calls to >> adr.reloc() are not hoisted by the compiler, due calling into the >> virtual method Relocation::type(). This means each call to reachable >> typically do several redundant calls into Relocation::type() >> >> This patch refactors all these calls into a single one, which has a >> small, but measurable, effect on startup (and likely some small >> effect on JIT compiler throughput in general) >> >> Webrev: http://cr.openjdk.java.net/~redestad/8240302/open.00/ >> Bug:??? https://bugs.openjdk.java.net/browse/JDK-8240302 >> >> Testing: tier1-3 >> >> Thanks! >> >> /Claes From suenaga at oss.nttdata.com Tue Mar 3 03:28:21 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Tue, 3 Mar 2020 12:28:21 +0900 Subject: RFR: 8240197: Cannot start JVM when $JAVA_HOME includes CJK characters In-Reply-To: References: <6ec2f649-8999-fdd5-c2df-465925c3663b@oss.nttdata.com> Message-ID: <2d5148fc-7d71-d94c-d4a3-e6424a38b091@oss.nttdata.com> Hi Ralf, Ioi, I uploaded new webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.02/ My comments are in below. On 2020/03/03 3:41, Ioi Lam wrote: > > > On 3/2/20 5:24 AM, Schmelter, Ralf wrote: >> Hi Yasumasa, >> >> thanks for your patch. I have some remarks: >> >> 1. wcstombs_s creates UTF-16, as does MultiByteToWideChar. That wasn't the problem. The problem was that wcstombs_s uses the current locale, which is "C" initially. One is normally expected to set the locale to the system locale via setlocale(LC_ALL, ""), as it is done in ParseLocale() on Unix. >> >> One could fix the original problem by setting the locale (probably not a good idea, since this changes the locale of the whole process). Or one could specify the correct locale to wcstombs like: >> >> static _locale_t loc = _create_locale(LC_ALL, ""); >> err = ::_mbstowcs_s_l(&converted_chars, path_start, buf_len + 1, buf, buf_len, loc); >> >> Both fix the problem. But I agree, using the MultiByteToWideChar method is better, since you don't have to create the locale. >> >> There are some other places in the code where mbstowcs is used on Windows (canonicalize_md.c for example). They should probably fixed in the same way. I agree with you, We need to fix similar codes in entire of JDK. I will pick up and will file them to JBS in future. >> 2.I don't think that using strlen(buf) as the number of wchars to allocate for the converted string is really wrong. There are no code pages supported by Java which would map a single byte to a Unicode character > U+ffff. But since one can get the always correct value via MultiByteToWideChar, this is futureproof and obviously right. >> >> 3. I would not use the CP_THREAD_ACP code page in MultiByteToWideChar and instead use CP_ACP, because the latter always uses the system locale. And this is the locale used to convert the arguments to bytes in the first place. We should use CP_THREAD_ACP because the user might change locale via `chcp` on their console. >> 4. You replaced the malloc/memcpy for the string which will be used to call native_path() with a strdup. This leads to a memory overwrite, when a path like "C:" is used, since native_path() will convert it to "C:.". And the debug version of the VM will detect this, since the tail guard will be partially overwritten. You can check this by running: >> >> make run-test TEST=gtest:os_windows Ok, I use malloc() instead of strdup() in new webrev. >> 5. Using WCHAR and LPWSTR seems odd to me. Nobody uses LPCSTR or CHAR for char* and char. But if you think WCHAR is better, I would adjust the return value of the method and the functions which use it too. This change is based on the comment from Thomas, and I agree with him because Windows API uses their types. I changed return type of wide_abs_unc_path() to LPWSTR, but I didn't change the caller of it because I want to focus this change to wide_abs_unc_path() and it would not give big effect to caller. >> 6. When you handle a relative path, you now only allocate enough space to hold the relative path, not the generated absolute path. Since a relative path is likely to be smaller than the absolute path, the _wfullpath call later will likely fail. That was the reason we used JVM_MAXPATHLEN for relative paths. You could fix this like in this patch http://cr.openjdk.java.net/~rschmelter/webrevs/tmp/unicode_convert.patch > Regarding #6, we have test cases under test/hotspot/jtreg/runtime/cds/appcds that use relative paths. Please run these tests when validating the patch. I use GetFullPathNameW() to get the length of result and to convert to full path. It resolves your concern. This webrev has passed all tests on submit repo (mach5-one-ysuenaga-JDK-8240197-2-20200303-0158-9122754) and also has passed test/hotspot/jtreg/runtime/cds/appcds jtreg tests on my laptop. Thanks, Yasumasa > Thanks > - Ioi > >> >> Best regards, >> Ralf > From ioi.lam at oracle.com Tue Mar 3 06:34:29 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 2 Mar 2020 22:34:29 -0800 Subject: RFR(S) 8240244 Avoid calling resolve_super_or_fail in SystemDictionary::load_shared_class Message-ID: <5d5c6c4d-0a0f-0c9d-c962-ec9e5e37e077@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8240244 http://cr.openjdk.java.net/~iklam/jdk15/8240244-reduce-resolve-super.v01/ When loading shared classes from the CDS archive, we need to test if the actually loaded super types are the same as those used during CDS dumping. That was done with a pretty expensive call to resolve_super_or_fail. In most cases, the super type has already been loaded (about 350 out of 400 cases when running HelloWorld), so we can quickly determine that by doing a look up in the super type's dictionary. This saves about 250,000 out of about 112M instructions (about 0.22%) when running HelloWorld with the default archive on Linux/x64. I also renamed a few functions in instanceKlass.hpp to _xxx_shared_xxx since they are used only for CDS. Thanks - Ioi From anton.tarasov at jetbrains.com Tue Mar 3 06:36:43 2020 From: anton.tarasov at jetbrains.com (Anton Tarasov) Date: Tue, 3 Mar 2020 09:36:43 +0300 Subject: RFR (S): 8238676: jni crashes on accessing it from process exit hook In-Reply-To: <2ac96b12-1010-311e-69e7-fa19c8528e65@oracle.com> References: <063252ad-ab02-403a-aa5a-1dcf288ea330@oracle.com> <4a511743-152a-dff9-5d19-707dcb53fb03@oracle.com> <2ac96b12-1010-311e-69e7-fa19c8528e65@oracle.com> Message-ID: Hi David, Thanks, please tag it, hope someone else will backport it (or if not I'm ready to do that job). With regards, Anton. On Tue, Mar 3, 2020 at 1:56 AM David Holmes wrote: > Hi Anton, > > On 3/03/2020 1:12 am, Anton Tarasov wrote: > > Hi David, > > > > Thank you for fixing this! Is it possible to backport the fix to jdk11? > > I can tag it for backporting once it is pushed, but I can't do the > actual backport for OpenJDK 11u. > > David > > > Thanks, > > Anton. > > > > On Sat, Feb 29, 2020 at 8:16 AM David Holmes > > wrote: > > > > Hi Gerard, > > > > Are you okay with this? > > > > Thanks, > > David > > > > On 26/02/2020 10:11 am, David Holmes wrote: > > > Hi Gerard, > > > > > > Thanks for taking a look at this. > > > > > > On 26/02/2020 2:45 am, gerard ziemski wrote: > > >> hi David, > > >> > > >> On 2/18/20 8:04 PM, David Holmes wrote: > > >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8238676 > > >>> webrev: http://cr.openjdk.java.net/~dholmes/8238676/webrev/ > > >>> > > >>> If an application tries to use JNI from an atexit handler, the > > >>> attempt can't succeed (the VM has already logically terminated) > > but > > >>> it should not crash. The JNI Invocation API code was missing > some > > >>> checks in places and wasn't aware of the possibility of trying > to > > >>> make calls from the VMThread. > > >>> > > >>> Testing: > > >>> - new test added for the JNI Invocation API > > >>> - tiers 1-3 > > >>> > > >>> Thanks, > > >>> David > > >> > > >> In jni_GetEnv() we have: > > >> > > >> 4120 if (vm_created == 0) { > > >> 4121 *penv = NULL; > > >> 4122 ret = JNI_EDETACHED; > > >> 4123 return ret; > > >> 4124 } > > >> > > >> but in jni_DetachCurrentThread() we have: > > >> > > >> 4063 if (vm_created == 0) { > > >> 4064 HOTSPOT_JNI_DETACHCURRENTTHREAD_RETURN(JNI_ERR); > > >> 4065 return JNI_ERR; > > >> 4066 } > > >> > > >> should lines 4064,4065 perhaps be: > > >> > > >> 4064 HOTSPOT_JNI_DETACHCURRENTTHREAD_RETURN(JNI_EDETACHED); > > >> 4065 return JNI_EDETACHED; > > >> > > >> to be consistent? > > > > > > Well ... ideally the JNI spec would have considered the question > of > > > "what if the JavaVM is no longer live?" and had an JNI_ENOJVM > errro > > > code. But it doesn't. Nor does it allow for any JNI method to > return > > > JNI_ERR for unexpected errors situations. So we have to adhere to > > the > > > specifications for each method. > > > > > > For GetEnv the spec only allows for returning JNI_EDETACHED, > > > JNI_EVERSION or JNI_OK. As you can't be attached to a VM that > > doesn't > > > exist then JNI_EDETACHED is the only possible return code - and > > it isn't > > > wrong, it just doesn't tell you why you aren't attached. > > > > > > If you apply the same logic to DetachCurrentThread, and follow > > the spec > > > then it states > > > > > > "Trying to detach a thread that is not attached is a no-op." > > > > > > which suggests we do nothing and report JNI_OK. But I think that > > would > > > be a disservice to the programmer in the case where the JVM is no > > longer > > > live. And we're allowed to return "a suitable JNI error code on > > failure" > > > so that is what I chose to do. And I return JNI_ERR rather than > > > JNI_EDETACHED for two reasons: > > > > > > 1. It is consistent with what we do for attach in the same > > circumstance: > > > > > > jint JNICALL jni_AttachCurrentThread(JavaVM *vm, void **penv, void > > > *_args) { > > > HOTSPOT_JNI_ATTACHCURRENTTHREAD_ENTRY(vm, penv, _args); > > > if (vm_created == 0) { > > > HOTSPOT_JNI_ATTACHCURRENTTHREAD_RETURN((uint32_t) JNI_ERR); > > > return JNI_ERR; > > > } > > > > > > 2. It would be confusing to return JNI_EDETACHED as that means > > "thread > > > detached from the VM" yet the spec says "Trying to detach a > > thread that > > > is not attached is a no-op.". So I need to convey that there is a > > more > > > specific underlying error here. > > > > > >> Looks good otherwise. > > > > > > Thanks for the review. > > > > > > David > > > ----- > > > > > >> > > >> cheers > > >> > > > From tobias.hartmann at oracle.com Tue Mar 3 07:50:34 2020 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Tue, 3 Mar 2020 08:50:34 +0100 Subject: RFR: 8240302: x64: Assembler::reachable redundantly call Relocation::type() more than once In-Reply-To: References: Message-ID: <4f8bc6a3-afa7-25ed-e7e8-76b86fafbd79@oracle.com> +1 Best regards, Tobias On 03.03.20 01:11, Vladimir Kozlov wrote: > Good cleanup. > > Thanks, > Vladimir > > On 3/2/20 7:07 AM, Claes Redestad wrote: >> Hi, >> >> in the x64-version Assembler::reachable method, various calls to >> adr.reloc() are not hoisted by the compiler, due calling into the >> virtual method Relocation::type(). This means each call to reachable >> typically do several redundant calls into Relocation::type() >> >> This patch refactors all these calls into a single one, which has a >> small, but measurable, effect on startup (and likely some small >> effect on JIT compiler throughput in general) >> >> Webrev: http://cr.openjdk.java.net/~redestad/8240302/open.00/ >> Bug:??? https://bugs.openjdk.java.net/browse/JDK-8240302 >> >> Testing: tier1-3 >> >> Thanks! >> >> /Claes From claes.redestad at oracle.com Tue Mar 3 11:15:48 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 3 Mar 2020 12:15:48 +0100 Subject: RFR(S) 8240244 Avoid calling resolve_super_or_fail in SystemDictionary::load_shared_class In-Reply-To: <5d5c6c4d-0a0f-0c9d-c962-ec9e5e37e077@oracle.com> References: <5d5c6c4d-0a0f-0c9d-c962-ec9e5e37e077@oracle.com> Message-ID: Hi Ioi, overall looks good, thanks! systemDictionary.cpp I'm not sure what you meant to write here: + // + Don't take the lock if loader data is NULL without taking lock. Left-over "!= 0": + if (!super_type->is_shared_unregistered_class() != 0 && super_type->class_loader_data() != NULL) { -- It seems to me that it might be possible to take the SystemDictionary_lock just once in check_shared_class_super_types method, spanning super and all interfaces. That might need some restructuring since we can't hold that lock when going into the class-loading slow path, but I think the slow path could profitably be moved to the outer method outside such a locked section. Thanks! /Claes On 2020-03-03 07:34, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8240244 > http://cr.openjdk.java.net/~iklam/jdk15/8240244-reduce-resolve-super.v01/ > > When loading shared classes from the CDS archive, we need to test if the > actually loaded super types are the same as those used during CDS dumping. > That was done with a pretty expensive call to resolve_super_or_fail. > > In most cases, the super type has already been loaded (about 350 out of 400 > cases when running HelloWorld), so we can quickly determine that by > doing a look up in the super type's dictionary. > > This saves about 250,000 out of about 112M instructions (about 0.22%) when > running HelloWorld with the default archive on Linux/x64. > > I also renamed a few functions in instanceKlass.hpp to _xxx_shared_xxx > since > they are used only for CDS. > > Thanks > - Ioi > > From claes.redestad at oracle.com Tue Mar 3 12:03:25 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 3 Mar 2020 04:03:25 -0800 (PST) Subject: RFR: 8240302: x64: Assembler::reachable redundantly call Relocation::type() more than once In-Reply-To: <79cfd32b-44ca-42f9-e6ef-e0edb6e14f09@oracle.com> References: <79cfd32b-44ca-42f9-e6ef-e0edb6e14f09@oracle.com> Message-ID: <77c7c3fd-7dad-1e88-a03f-f948417d7ce8@oracle.com> Hi Ioi, Vladimir, Tobias, thanks for reviewing! Yes, const seems like it can't convince the compiler that it can elide the redundant calls. Maybe a link-time optimization pass would have allowed the toolchain to do the right thing here..? /Claes On 2020-03-03 03:25, Ioi Lam wrote: > Looks good to me, too. > > I was wondering if the C compiler would be a little smarter if > Relocation::type() was declared as a const function, like: > > - virtual relocInfo::relocType type()?????? { return relocInfo::none; } > + virtual relocInfo::relocType type() const { return relocInfo::none; } > > But apparently GCC will not elide the multiple calls: > > // test.cpp > class X { > public: virtual int get() const { return 1; } > }; > > class Y : public X { > public:? virtual int get() const { return 2; } > }; > > X* xx = 0; > int testfunc(char* argv[]) { > ? X* x = xx; > ? return x->get() + x->get() + x->get(); > } > > $ gcc -O -S -o - test.cpp | grep call > ??? call??? *(%rax) > ??? call??? *(%rax) > ??? call??? *(%rax) > > So it looks like we must manually save the return value of type() as > done in your patch. > > Thanks > - Ioi > > On 3/2/20 4:11 PM, Vladimir Kozlov wrote: >> Good cleanup. >> >> Thanks, >> Vladimir >> >> On 3/2/20 7:07 AM, Claes Redestad wrote: >>> Hi, >>> >>> in the x64-version Assembler::reachable method, various calls to >>> adr.reloc() are not hoisted by the compiler, due calling into the >>> virtual method Relocation::type(). This means each call to reachable >>> typically do several redundant calls into Relocation::type() >>> >>> This patch refactors all these calls into a single one, which has a >>> small, but measurable, effect on startup (and likely some small >>> effect on JIT compiler throughput in general) >>> >>> Webrev: http://cr.openjdk.java.net/~redestad/8240302/open.00/ >>> Bug:??? https://bugs.openjdk.java.net/browse/JDK-8240302 >>> >>> Testing: tier1-3 >>> >>> Thanks! >>> >>> /Claes > From claes.redestad at oracle.com Tue Mar 3 13:37:31 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 3 Mar 2020 14:37:31 +0100 Subject: RFR: 8240263: Assertion-only call in Method::link_method affecting product builds In-Reply-To: <4cd3d4af-fd37-77cd-f804-381079459d40@oracle.com> References: <6a41ef95-8d49-f694-57b2-8d7ddf3f480b@oracle.com> <9d8dae94-f992-6345-58e6-a28fa2789111@redhat.com> <43f27f02-6ddf-7ad3-754e-97e73280772b@oracle.com> <1f3f2bf0-1c90-1d66-2004-ed5545b63c25@oracle.com> <4cd3d4af-fd37-77cd-f804-381079459d40@oracle.com> Message-ID: <4d07fa5f-ab8c-cc47-8dff-9beacf90165b@oracle.com> Thanks Aleksey, Dan and Ioi for reviewing! Hearing no strong opinions the other way I pushed the ASSERT version. /Claes On 2020-03-02 19:36, Ioi Lam wrote: > Looks good to me too. Thanks for finding this performance problem! > > Thanks > - Ioi > > On 3/2/20 8:33 AM, Daniel D. Daugherty wrote: >> I have a preference for the DEBUG_ONLY style, but that's probably >> because I just did some of those in the ObjectMonitor subsystem. >> >> Either way: Thumbs up! >> >> Dan >> >> On 3/2/20 5:08 AM, Claes Redestad wrote: >>> >>> >>> On 2020-03-02 10:22, Aleksey Shipilev wrote: >>>> On 3/2/20 10:22 AM, Claes Redestad wrote: >>>>> Guarding the resolution by DEBUG_ONLY seem the right thing to do here, >>>>> which bring a small improvement on startup tests when CDS is enabled. >>>>> >>>>> See attached patch[1]. >>>> >>>> I believe we usually do this in such case: >>>> >>>> #ifdef ASSERT >>>> ?? address entry = Interpreter::entry_for_cds_method(h_method); >>>> ?? assert(entry != NULL && entry == _i2i_entry, >>>> ????????? "should be correctly set during dump time"); >>>> #endif >>> >>> I've seen it done both ways, and have no strong preference. >>> >>>> >>>> It keeps the block easily editable, without looking back at whether >>>> it interferes with macros. >>>> >>>> Otherwise looks fine, assuming release builds still pass. >>> >>> Running your suggested variant through tier 1 to make sure. >>> >>> Thanks! >>> >>> /Claes >> > From ralf.schmelter at sap.com Tue Mar 3 14:25:23 2020 From: ralf.schmelter at sap.com (Schmelter, Ralf) Date: Tue, 3 Mar 2020 14:25:23 +0000 Subject: RFR: 8240197: Cannot start JVM when $JAVA_HOME includes CJK characters In-Reply-To: <2d5148fc-7d71-d94c-d4a3-e6424a38b091@oss.nttdata.com> References: <6ec2f649-8999-fdd5-c2df-465925c3663b@oss.nttdata.com> <2d5148fc-7d71-d94c-d4a3-e6424a38b091@oss.nttdata.com> Message-ID: Hi Yasumasa, thanks for your work. The code should now be able to handle any path. Two small changes (no new webrev needed): size_t result_len = prefix_len + prefix_off + additional_space should be size_t result_len = prefix_len - prefix_off + additional_space And you should copy the prefix after you copied the filename, since sometimes the prefix is required to overwrite the start of the filename: // This call would be success because it is checked in above GetFullPathNameW(unicode_path, full_path_len, result + prefix_len - prefix_off, NULL); // Copy prefix memcpy(result, prefix, sizeof(WCHAR) * prefix_len); and // Copy unicode path memcpy(result + prefix_len - prefix_off, unicode_path, sizeof(WCHAR) * unicode_path_len); // Copy prefix memcpy(result, prefix, sizeof(WCHAR) * prefix_len); With these changes the os_windows gtest and the appcds jtreg tests run without errors. Best regards, Ralf From claes.redestad at oracle.com Tue Mar 3 15:19:46 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 3 Mar 2020 16:19:46 +0100 Subject: RFR(S) 8240244 Avoid calling resolve_super_or_fail in SystemDictionary::load_shared_class In-Reply-To: References: <5d5c6c4d-0a0f-0c9d-c962-ec9e5e37e077@oracle.com> Message-ID: <4b56f1b9-05b5-c633-faf3-7e2672775bbc@oracle.com> On 2020-03-03 12:15, Claes Redestad wrote: > It seems to me that it might be possible to take the > SystemDictionary_lock just once in check_shared_class_super_types > method, spanning super and all interfaces. That might need some > restructuring since we can't hold that lock when going into > the class-loading slow path, but I think the slow path could profitably > be moved to the outer method outside such a locked section. I took a quick stab at prototyping this idea[1], and it does not appear to be an improvement over the proposed version. /Claes [1] http://cr.openjdk.java.net/~redestad/scratch/8240244_lock.00/ From harold.seigel at oracle.com Tue Mar 3 15:33:27 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Tue, 3 Mar 2020 10:33:27 -0500 Subject: RFR 8240324: Improve is_boot_class_loader_data() by adding simple check Message-ID: <1e54594e-c3cf-9ca3-563b-6b094b8e3fc3@oracle.com> Hi, Please review this trivial fix for JDK-8240324 to potentially avoid accessing the class loader oop. Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8240324/webrev/ JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8240324 The fix was regression tested by running Mach5 tiers 1 and 2 tests and builds on Linux-x64, Solaris, Windows, and Mac OS X, by running Mach5 tiers 3-5 tests on Linux-x64, and JCK lang and VM tests on Linux-x64. Thanks, Harold From coleen.phillimore at oracle.com Tue Mar 3 15:46:52 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 3 Mar 2020 10:46:52 -0500 Subject: RFR 8240324: Improve is_boot_class_loader_data() by adding simple check In-Reply-To: <1e54594e-c3cf-9ca3-563b-6b094b8e3fc3@oracle.com> References: <1e54594e-c3cf-9ca3-563b-6b094b8e3fc3@oracle.com> Message-ID: Looks good plus trivial. Thanks, Coleen On 3/3/20 10:33 AM, Harold Seigel wrote: > Hi, > > Please review this trivial fix for JDK-8240324 to potentially avoid > accessing the class loader oop. > > Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8240324/webrev/ > > JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8240324 > > The fix was regression tested by running Mach5 tiers 1 and 2 tests and > builds on Linux-x64, Solaris, Windows, and Mac OS X, by running Mach5 > tiers 3-5 tests on Linux-x64, and JCK lang and VM tests on Linux-x64. > > Thanks, Harold > From harold.seigel at oracle.com Tue Mar 3 15:48:09 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Tue, 3 Mar 2020 10:48:09 -0500 Subject: RFR 8240324: Improve is_boot_class_loader_data() by adding simple check In-Reply-To: References: <1e54594e-c3cf-9ca3-563b-6b094b8e3fc3@oracle.com> Message-ID: <0ce3b041-5b4f-08e4-24ff-fa241f4b3969@oracle.com> Thanks Coleen! Harold On 3/3/2020 10:46 AM, coleen.phillimore at oracle.com wrote: > Looks good plus trivial. > Thanks, > Coleen > > On 3/3/20 10:33 AM, Harold Seigel wrote: >> Hi, >> >> Please review this trivial fix for JDK-8240324 to potentially avoid >> accessing the class loader oop. >> >> Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8240324/webrev/ >> >> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8240324 >> >> The fix was regression tested by running Mach5 tiers 1 and 2 tests >> and builds on Linux-x64, Solaris, Windows, and Mac OS X, by running >> Mach5 tiers 3-5 tests on Linux-x64, and JCK lang and VM tests on >> Linux-x64. >> >> Thanks, Harold >> > From coleen.phillimore at oracle.com Tue Mar 3 15:48:24 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 3 Mar 2020 10:48:24 -0500 Subject: RFR (T) 8225760: oop::raw_set_obj isn't needed Message-ID: <51797921-3d65-7337-4464-d4e22723275d@oracle.com> Remove the public function and add to set_obj.? Ran tier1 tests locally. open webrev at http://cr.openjdk.java.net/~coleenp/2020/8225760.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8225760 Thanks, Coleen From rkennke at redhat.com Tue Mar 3 15:55:17 2020 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 3 Mar 2020 16:55:17 +0100 Subject: RFR (T) 8225760: oop::raw_set_obj isn't needed In-Reply-To: <51797921-3d65-7337-4464-d4e22723275d@oracle.com> References: <51797921-3d65-7337-4464-d4e22723275d@oracle.com> Message-ID: <49184844-4b4e-385b-13f5-285be78508b8@redhat.com> This is probably a left-over from an earlier Shenandoah. Patch looks good to me. Thanks, Roman > Remove the public function and add to set_obj.? Ran tier1 tests locally. > > open webrev at http://cr.openjdk.java.net/~coleenp/2020/8225760.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8225760 > > Thanks, > Coleen > From harold.seigel at oracle.com Tue Mar 3 15:55:10 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Tue, 3 Mar 2020 07:55:10 -0800 (PST) Subject: RFR (T) 8225760: oop::raw_set_obj isn't needed In-Reply-To: <51797921-3d65-7337-4464-d4e22723275d@oracle.com> References: <51797921-3d65-7337-4464-d4e22723275d@oracle.com> Message-ID: <47c147ed-aa18-05b1-26fb-e89e83e90124@oracle.com> Looks good and trivial. Thanks!? Harold On 3/3/2020 10:48 AM, coleen.phillimore at oracle.com wrote: > Remove the public function and add to set_obj.? Ran tier1 tests locally. > > open webrev at http://cr.openjdk.java.net/~coleenp/2020/8225760.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8225760 > > Thanks, > Coleen From lois.foltan at oracle.com Tue Mar 3 15:57:42 2020 From: lois.foltan at oracle.com (Lois Foltan) Date: Tue, 3 Mar 2020 10:57:42 -0500 Subject: RFR 8240324: Improve is_boot_class_loader_data() by adding simple check In-Reply-To: References: <1e54594e-c3cf-9ca3-563b-6b094b8e3fc3@oracle.com> Message-ID: On 3/3/2020 10:46 AM, coleen.phillimore at oracle.com wrote: > Looks good plus trivial. +1. Lois > Thanks, > Coleen > > On 3/3/20 10:33 AM, Harold Seigel wrote: >> Hi, >> >> Please review this trivial fix for JDK-8240324 to potentially avoid >> accessing the class loader oop. >> >> Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8240324/webrev/ >> >> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8240324 >> >> The fix was regression tested by running Mach5 tiers 1 and 2 tests >> and builds on Linux-x64, Solaris, Windows, and Mac OS X, by running >> Mach5 tiers 3-5 tests on Linux-x64, and JCK lang and VM tests on >> Linux-x64. >> >> Thanks, Harold >> > From m.sundar85 at gmail.com Tue Mar 3 16:02:24 2020 From: m.sundar85 at gmail.com (Sundara Mohan M) Date: Tue, 3 Mar 2020 11:02:24 -0500 Subject: Need help on debugging JVM crash Message-ID: Hi, I am seeing JVM crashes on our system in GC Thread with parallel gc on x86 linux. Observed the same crash happening on JVM-11.0.6/13.0.2/13.0.1 GA builds. Adding some logs lines to give some context. # # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x00007f669c964311, pid=66684, tid=71106 # # JRE version: OpenJDK Runtime Environment (13.0.1+9) (build 13.0.1+9) # Java VM: OpenJDK 64-Bit Server VM (13.0.1+9, mixed mode, tiered, parallel gc, linux-amd64) # Problematic frame: # V [libjvm.so+0xcd3311] PCMarkAndPushClosure::do_oop(oopDesc**)+0x51 # # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again # # If you would like to submit a bug report, please visit: # https://github.com/AdoptOpenJDK/openjdk-build/issues # Host: Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz, 48 cores, 125G, Red Hat Enterprise Linux Server release 6.10 (Santiago) Time: Thu Feb 6 11:43:48 2020 UTC elapsed time: 198626 seconds (2d 7h 10m 26s) Following is the stack trace ex1: Stack: [0x00007fd01cbdb000,0x00007fd01ccdb000], sp=0x00007fd01ccd8890, free space=1014k Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code) *V [libjvm.so+0xcc0121] PCMarkAndPushClosure::do_oop(oopDesc**)+0x51* V [libjvm.so+0xc58c8b] OopMapSet::oops_do(frame const*, RegisterMap const*, OopClosure*)+0x2eb V [libjvm.so+0x7521e9] frame::oops_do_internal(OopClosure*, CodeBlobClosure*, RegisterMap*, bool)+0x99 V [libjvm.so+0xf55757] JavaThread::oops_do(OopClosure*, CodeBlobClosure*)+0x187 V [libjvm.so+0xcbb100] ThreadRootsMarkingTask::do_it(GCTaskManager*, unsigned int)+0xb0 V [libjvm.so+0x7e0f8b] GCTaskThread::run()+0x1eb V [libjvm.so+0xf5d43d] Thread::call_run()+0x10d V [libjvm.so+0xc74337] thread_native_entry(Thread*)+0xe7 JavaThread 0x00007fbeb9209800 (nid = 82380) was being processed Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) v ~RuntimeStub::_new_array_Java J 62465 c2 ch.qos.logback.classic.spi.ThrowableProxy.(Ljava/lang/Throwable;)V (207 bytes) @ 0x00007fd00ad43704 [0x00007fd00ad41420+0x00000000000022e4] J 474206 c2 org.eclipse.jetty.util.log.JettyAwareLogger.log(Lorg/slf4j/Marker;ILjava/lang/String;[Ljava/lang/Object;Ljava/lang/Throwable;)V (134 bytes) @ 0x00007fd00c4e81ec [0x00007fd00c4e7ee0+0x000000000000030c] j org.eclipse.jetty.util.log.JettyAwareLogger.warn(Ljava/lang/String;Ljava/lang/Throwable;)V+7 j org.eclipse.jetty.util.log.Slf4jLog.warn(Ljava/lang/String;Ljava/lang/Throwable;)V+6 j org.eclipse.jetty.server.HttpChannel.handleException(Ljava/lang/Throwable;)V+181 j org.eclipse.jetty.server.HttpChannelOverHttp.handleException(Ljava/lang/Throwable;)V+13 J 64106 c2 org.eclipse.jetty.server.HttpChannel.handle()Z (997 bytes) @ 0x00007fd00c6d2cd4 [0x00007fd00c6cdec0+0x0000000000004e14] J 280430 c2 org.eclipse.jetty.server.HttpConnection.onFillable()V (334 bytes) @ 0x00007fd00da925f0 [0x00007fd00da91e40+0x00000000000007b0] J 41979 c2 org.eclipse.jetty.io.ChannelEndPoint$2.run()V (12 bytes) @ 0x00007fd00a14f604 [0x00007fd00a14f4e0+0x0000000000000124] J 86362 c2 org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run()V (565 bytes) @ 0x00007fd0087d7e34 [0x00007fd0087d7cc0+0x0000000000000174] J 75998 c2 java.lang.Thread.run()V java.base at 13.0.2 (17 bytes) @ 0x00007fd00c93b8d8 [0x00007fd00c93b8a0+0x0000000000000038] v ~StubRoutines::call_stub ex2: Stack: [0x00007f669869f000,0x00007f669879f000], sp=0x00007f669879c890, free space=1014k Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code) *V [libjvm.so+0xcd3311] PCMarkAndPushClosure::do_oop(oopDesc**)+0x51*V [libjvm.so+0xc6bf0b] OopMapSet::oops_do(frame const*, RegisterMap const*, OopClosure*)+0x2eb V [libjvm.so+0x765489] frame::oops_do_internal(OopClosure*, CodeBlobClosure*, RegisterMap*, bool)+0x99 V [libjvm.so+0xf68b17] JavaThread::oops_do(OopClosure*, CodeBlobClosure*)+0x187 V [libjvm.so+0xcce2f0] ThreadRootsMarkingTask::do_it(GCTaskManager*, unsigned int)+0xb0 V [libjvm.so+0x7f422b] GCTaskThread::run()+0x1eb V [libjvm.so+0xf707fd] Thread::call_run()+0x10d V [libjvm.so+0xc875b7] thread_native_entry(Thread*)+0xe7 JavaThread 0x00007f5518004000 (nid = 75659) was being processed Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) v ~RuntimeStub::_new_array_Java J 54174 c2 ch.qos.logback.classic.spi.ThrowableProxy.(Ljava/lang/Throwable;)V (207 bytes) @ 0x00007f6687d92678 [0x00007f6687d8c700+0x0000000000005f78] J 334031 c2 com.xmas.webservice.exception.ExceptionLoggingWrapper.execute()V (1004 bytes) @ 0x00007f6686ede430 [0x00007f6686edd580+0x0000000000000eb0] J 53431 c2 com.xmas.webservice.exception.mapper.AbstractExceptionMapper.toResponse(Lcom/xmas/beans/exceptions/mapper/V3ErrorCode;Ljava/lang/Exception;)Ljavax/ws/rs/core/Response; (105 bytes) @ 0x00007f6687db88b0 [0x00007f6687db8660+0x0000000000000250] J 63819 c2 com.xmas.webservice.exception.mapper.RequestBlockedExceptionMapper.toResponse(Ljava/lang/Exception;)Ljavax/ws/rs/core/Response; (9 bytes) @ 0x00007f6686a6ed9c [0x00007f6686a6ecc0+0x00000000000000dc] J 334032 c2 com.xmas.webservice.filters.ResponseSerializationWorker.processException()Ljava/io/InputStream; (332 bytes) @ 0x00007f668992ad34 [0x00007f668992a840+0x00000000000004f4] J 403918 c2 com.xmas.webservice.filters.ResponseSerializationWorker.execute()Z (272 bytes) @ 0x00007f66869d67fc [0x00007f66869d5e80+0x000000000000097c] J 17530 c2 com.lafaspot.common.concurrent.internal.WorkerWrapper.execute()Z (208 bytes) @ 0x00007f66848b3708 [0x00007f66848b36a0+0x0000000000000068] J 31970% c2 com.lafaspot.common.concurrent.internal.WorkerManagerOneThread.call()Lcom/lafaspot/common/concurrent/internal/WorkerManagerState; (486 bytes) @ 0x00007f668608dcb0 [0x00007f668608d5e0+0x00000000000006d0] j com.lafaspot.common.concurrent.internal.WorkerManagerOneThread.call()Ljava/lang/Object;+1 J 4889 c1 java.util.concurrent.FutureTask.run()V java.base at 13.0.1 (123 bytes) @ 0x00007f667d0be604 [0x00007f667d0bdf80+0x0000000000000684] J 7487 c1 java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V java.base at 13.0.1 (187 bytes) @ 0x00007f667dd45854 [0x00007f667dd44a60+0x0000000000000df4] J 7486 c1 java.util.concurrent.ThreadPoolExecutor$Worker.run()V java.base at 13.0.1 (9 bytes) @ 0x00007f667d1f643c [0x00007f667d1f63c0+0x000000000000007c] J 7078 c1 java.lang.Thread.run()V java.base at 13.0.1 (17 bytes) @ 0x00007f667d1f2d74 [0x00007f667d1f2c40+0x0000000000000134] v ~StubRoutines::call_stub Not very frequent but ~90 days ~120 crashes with following signal siginfo: si_signo: 11 (SIGSEGV), si_code: 128 (SI_KERNEL), si_addr: 0x0000000000000000 This signal is generated when we try to access non canonical address in linux. As suggested by Stefan in another thread i tried to add VerifyAfterGC/VerifyBeforeGC but it seems to increase the latency and applications not surviving our production traffic(timing out and requests are failing). Questions 1. When i looked at source code for printing stack trace i see following https://github.com/openjdk/jdk11u/blob/master/src/hotspot/share/utilities/vmError.cpp#L696 (Prints native stack trace) https://github.com/openjdk/jdk11u/blob/master/src/hotspot/share/utilities/vmError.cpp#L718 (printing Java thread stack trace if it is involved in GC crash) a. How do you know this java thread was involved in jvm crash? b. Can i assume the java thread printed after native stack trace was the culprit? c. Since i am seeing the same frame (~RuntimeStub::_new_array_Java, J 54174 c2 ch.qos.logback.classic.spi.ThrowableProxy...) but different stack trace in both crashes can this be the root cause? 2. Thinking of excluding compilation of ch.qos.logback.classic.spi.ThrowableProxy class and running in production to see if compilation of this method is the cause. Does it make sense? 3. Any other suggestion on debugging this further? TIA Sundar From coleen.phillimore at oracle.com Tue Mar 3 16:17:50 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 3 Mar 2020 11:17:50 -0500 Subject: RFR (T) 8225760: oop::raw_set_obj isn't needed In-Reply-To: <49184844-4b4e-385b-13f5-285be78508b8@redhat.com> References: <51797921-3d65-7337-4464-d4e22723275d@oracle.com> <49184844-4b4e-385b-13f5-285be78508b8@redhat.com> Message-ID: <64252a12-0d7b-b3fa-0037-3f6d4a73d695@oracle.com> On 3/3/20 10:55 AM, Roman Kennke wrote: > This is probably a left-over from an earlier Shenandoah. Patch looks > good to me. I didn't track down it's origin.? Thank you for the code review. Coleen > > Thanks, > Roman > > >> Remove the public function and add to set_obj.? Ran tier1 tests locally. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/2020/8225760.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8225760 >> >> Thanks, >> Coleen >> From coleen.phillimore at oracle.com Tue Mar 3 16:18:02 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 3 Mar 2020 08:18:02 -0800 (PST) Subject: RFR (T) 8225760: oop::raw_set_obj isn't needed In-Reply-To: <47c147ed-aa18-05b1-26fb-e89e83e90124@oracle.com> References: <51797921-3d65-7337-4464-d4e22723275d@oracle.com> <47c147ed-aa18-05b1-26fb-e89e83e90124@oracle.com> Message-ID: <75fc4946-af0a-d04f-9582-7100030ffbd7@oracle.com> Thanks Harold. Coleen On 3/3/20 10:55 AM, Harold Seigel wrote: > Looks good and trivial. > > Thanks!? Harold > > On 3/3/2020 10:48 AM, coleen.phillimore at oracle.com wrote: >> Remove the public function and add to set_obj.? Ran tier1 tests locally. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8225760.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8225760 >> >> Thanks, >> Coleen From yumin.qi at oracle.com Tue Mar 3 16:22:44 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Tue, 3 Mar 2020 08:22:44 -0800 Subject: Need help on debugging JVM crash In-Reply-To: References: Message-ID: HI, Sundara On 3/3/20 8:02 AM, Sundara Mohan M wrote: > Hi, > I am seeing JVM crashes on our system in GC Thread with parallel gc on > x86 linux. Observed the same crash happening on JVM-11.0.6/13.0.2/13.0.1 GA > builds. > Adding some logs lines to give some context. > > # > # A fatal error has been detected by the Java Runtime Environment: > # > # SIGSEGV (0xb) at pc=0x00007f669c964311, pid=66684, tid=71106 > # > # JRE version: OpenJDK Runtime Environment (13.0.1+9) (build 13.0.1+9) > # Java VM: OpenJDK 64-Bit Server VM (13.0.1+9, mixed mode, tiered, parallel > gc, linux-amd64) > # Problematic frame: > # V [libjvm.so+0xcd3311] PCMarkAndPushClosure::do_oop(oopDesc**)+0x51 > # > # No core dump will be written. Core dumps have been disabled. To enable > core dumping, try "ulimit -c unlimited" before starting Java again > # > # If you would like to submit a bug report, please visit: > # https://github.com/AdoptOpenJDK/openjdk-build/issues > # > > Host: Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz, 48 cores, 125G, Red Hat > Enterprise Linux Server release 6.10 (Santiago) > Time: Thu Feb 6 11:43:48 2020 UTC elapsed time: 198626 seconds (2d 7h 10m > 26s) > > > Following is the stack trace > ex1: > Stack: [0x00007fd01cbdb000,0x00007fd01ccdb000], sp=0x00007fd01ccd8890, > free space=1014k > Native frames: (J=compiled Java code, A=aot compiled Java code, > j=interpreted, Vv=VM code, C=native code) > *V [libjvm.so+0xcc0121] PCMarkAndPushClosure::do_oop(oopDesc**)+0x51* > V [libjvm.so+0xc58c8b] OopMapSet::oops_do(frame const*, RegisterMap > const*, OopClosure*)+0x2eb > V [libjvm.so+0x7521e9] frame::oops_do_internal(OopClosure*, > CodeBlobClosure*, RegisterMap*, bool)+0x99 > V [libjvm.so+0xf55757] JavaThread::oops_do(OopClosure*, > CodeBlobClosure*)+0x187 > V [libjvm.so+0xcbb100] ThreadRootsMarkingTask::do_it(GCTaskManager*, > unsigned int)+0xb0 > V [libjvm.so+0x7e0f8b] GCTaskThread::run()+0x1eb > V [libjvm.so+0xf5d43d] Thread::call_run()+0x10d > V [libjvm.so+0xc74337] thread_native_entry(Thread*)+0xe7 > > JavaThread 0x00007fbeb9209800 (nid = 82380) was being processed > Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) > v ~RuntimeStub::_new_array_Java > J 62465 c2 > ch.qos.logback.classic.spi.ThrowableProxy.(Ljava/lang/Throwable;)V > (207 bytes) @ 0x00007fd00ad43704 [0x00007fd00ad41420+0x00000000000022e4] > J 474206 c2 > org.eclipse.jetty.util.log.JettyAwareLogger.log(Lorg/slf4j/Marker;ILjava/lang/String;[Ljava/lang/Object;Ljava/lang/Throwable;)V > (134 bytes) @ 0x00007fd00c4e81ec [0x00007fd00c4e7ee0+0x000000000000030c] > j > org.eclipse.jetty.util.log.JettyAwareLogger.warn(Ljava/lang/String;Ljava/lang/Throwable;)V+7 > j > org.eclipse.jetty.util.log.Slf4jLog.warn(Ljava/lang/String;Ljava/lang/Throwable;)V+6 > j > org.eclipse.jetty.server.HttpChannel.handleException(Ljava/lang/Throwable;)V+181 > j > org.eclipse.jetty.server.HttpChannelOverHttp.handleException(Ljava/lang/Throwable;)V+13 > J 64106 c2 org.eclipse.jetty.server.HttpChannel.handle()Z (997 bytes) @ > 0x00007fd00c6d2cd4 [0x00007fd00c6cdec0+0x0000000000004e14] > J 280430 c2 org.eclipse.jetty.server.HttpConnection.onFillable()V (334 > bytes) @ 0x00007fd00da925f0 [0x00007fd00da91e40+0x00000000000007b0] > J 41979 c2 org.eclipse.jetty.io.ChannelEndPoint$2.run()V (12 bytes) @ > 0x00007fd00a14f604 [0x00007fd00a14f4e0+0x0000000000000124] > J 86362 c2 org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run()V > (565 bytes) @ 0x00007fd0087d7e34 [0x00007fd0087d7cc0+0x0000000000000174] > J 75998 c2 java.lang.Thread.run()V java.base at 13.0.2 (17 bytes) @ > 0x00007fd00c93b8d8 [0x00007fd00c93b8a0+0x0000000000000038] > v ~StubRoutines::call_stub > > ex2: > Stack: [0x00007f669869f000,0x00007f669879f000], sp=0x00007f669879c890, > free space=1014k > Native frames: (J=compiled Java code, A=aot compiled Java code, > j=interpreted, Vv=VM code, C=native code) > > *V [libjvm.so+0xcd3311] PCMarkAndPushClosure::do_oop(oopDesc**)+0x51*V > [libjvm.so+0xc6bf0b] OopMapSet::oops_do(frame const*, RegisterMap const*, > OopClosure*)+0x2eb > V [libjvm.so+0x765489] frame::oops_do_internal(OopClosure*, > CodeBlobClosure*, RegisterMap*, bool)+0x99 > V [libjvm.so+0xf68b17] JavaThread::oops_do(OopClosure*, > CodeBlobClosure*)+0x187 > V [libjvm.so+0xcce2f0] ThreadRootsMarkingTask::do_it(GCTaskManager*, > unsigned int)+0xb0 > V [libjvm.so+0x7f422b] GCTaskThread::run()+0x1eb > V [libjvm.so+0xf707fd] Thread::call_run()+0x10d > V [libjvm.so+0xc875b7] thread_native_entry(Thread*)+0xe7 > > JavaThread 0x00007f5518004000 (nid = 75659) was being processed > Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) > v ~RuntimeStub::_new_array_Java > J 54174 c2 > ch.qos.logback.classic.spi.ThrowableProxy.(Ljava/lang/Throwable;)V > (207 bytes) @ 0x00007f6687d92678 [0x00007f6687d8c700+0x0000000000005f78] > J 334031 c2 > com.xmas.webservice.exception.ExceptionLoggingWrapper.execute()V (1004 > bytes) @ 0x00007f6686ede430 [0x00007f6686edd580+0x0000000000000eb0] > J 53431 c2 > com.xmas.webservice.exception.mapper.AbstractExceptionMapper.toResponse(Lcom/xmas/beans/exceptions/mapper/V3ErrorCode;Ljava/lang/Exception;)Ljavax/ws/rs/core/Response; > (105 bytes) @ 0x00007f6687db88b0 [0x00007f6687db8660+0x0000000000000250] > J 63819 c2 > com.xmas.webservice.exception.mapper.RequestBlockedExceptionMapper.toResponse(Ljava/lang/Exception;)Ljavax/ws/rs/core/Response; > (9 bytes) @ 0x00007f6686a6ed9c [0x00007f6686a6ecc0+0x00000000000000dc] > J 334032 c2 > com.xmas.webservice.filters.ResponseSerializationWorker.processException()Ljava/io/InputStream; > (332 bytes) @ 0x00007f668992ad34 [0x00007f668992a840+0x00000000000004f4] > J 403918 c2 > com.xmas.webservice.filters.ResponseSerializationWorker.execute()Z (272 > bytes) @ 0x00007f66869d67fc [0x00007f66869d5e80+0x000000000000097c] > J 17530 c2 com.lafaspot.common.concurrent.internal.WorkerWrapper.execute()Z > (208 bytes) @ 0x00007f66848b3708 [0x00007f66848b36a0+0x0000000000000068] > J 31970% c2 > com.lafaspot.common.concurrent.internal.WorkerManagerOneThread.call()Lcom/lafaspot/common/concurrent/internal/WorkerManagerState; > (486 bytes) @ 0x00007f668608dcb0 [0x00007f668608d5e0+0x00000000000006d0] > j > com.lafaspot.common.concurrent.internal.WorkerManagerOneThread.call()Ljava/lang/Object;+1 > J 4889 c1 java.util.concurrent.FutureTask.run()V java.base at 13.0.1 (123 > bytes) @ 0x00007f667d0be604 [0x00007f667d0bdf80+0x0000000000000684] > J 7487 c1 > java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V > java.base at 13.0.1 (187 bytes) @ 0x00007f667dd45854 > [0x00007f667dd44a60+0x0000000000000df4] > J 7486 c1 java.util.concurrent.ThreadPoolExecutor$Worker.run()V > java.base at 13.0.1 (9 bytes) @ 0x00007f667d1f643c > [0x00007f667d1f63c0+0x000000000000007c] > J 7078 c1 java.lang.Thread.run()V java.base at 13.0.1 (17 bytes) @ > 0x00007f667d1f2d74 [0x00007f667d1f2c40+0x0000000000000134] > v ~StubRoutines::call_stub > > Not very frequent but ~90 days ~120 crashes with following signal > siginfo: si_signo: 11 (SIGSEGV), si_code: 128 (SI_KERNEL), si_addr: > 0x0000000000000000 > This signal is generated when we try to access non canonical address in > linux. > > As suggested by Stefan in another thread i tried to > add VerifyAfterGC/VerifyBeforeGC but it seems to increase the latency and > applications not surviving our production traffic(timing out and requests > are failing). > > Questions > 1. When i looked at source code for printing stack trace i see following > https://github.com/openjdk/jdk11u/blob/master/src/hotspot/share/utilities/vmError.cpp#L696 > (Prints native stack trace) > https://github.com/openjdk/jdk11u/blob/master/src/hotspot/share/utilities/vmError.cpp#L718 > (printing Java thread stack trace if it is involved in GC crash) > a. How do you know this java thread was involved in jvm crash? When GC processes thread stack as root, the java thread first was recorded. This is why at crash, the java thread was printed out. > b. Can i assume the java thread printed after native stack trace was the > culprit? Please check this thread stack frames, when GC is doing marking work, I think, it encountered a bad oop. Check: If it is a compiled frame, if so, it may related to compiled code. > c. Since i am seeing the same frame (~RuntimeStub::_new_array_Java, J > 54174 c2 ch.qos.logback.classic.spi.ThrowableProxy...) but different > stack trace in both crashes can this be the root cause? It is a C2 compiled frame. The bad oop could be a result of compiler. It also needs detail debug information to make the conclusion. Thanks Yumin > 2. Thinking of excluding compilation > of ch.qos.logback.classic.spi.ThrowableProxy class and running in > production to see if compilation of this method is the cause. Does it make > sense? > > 3. Any other suggestion on debugging this further? > > TIA > Sundar From lois.foltan at oracle.com Tue Mar 3 16:49:49 2020 From: lois.foltan at oracle.com (Lois Foltan) Date: Tue, 3 Mar 2020 11:49:49 -0500 Subject: RFR(S) 8240244 Avoid calling resolve_super_or_fail in SystemDictionary::load_shared_class In-Reply-To: <5d5c6c4d-0a0f-0c9d-c962-ec9e5e37e077@oracle.com> References: <5d5c6c4d-0a0f-0c9d-c962-ec9e5e37e077@oracle.com> Message-ID: <4db2ef25-3073-c3ca-dc1f-72c6806489b2@oracle.com> On 3/3/2020 1:34 AM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8240244 > http://cr.openjdk.java.net/~iklam/jdk15/8240244-reduce-resolve-super.v01/ > > When loading shared classes from the CDS archive, we need to test if the > actually loaded super types are the same as those used during CDS > dumping. > That was done with a pretty expensive call to resolve_super_or_fail. > > In most cases, the super type has already been loaded (about 350 out > of 400 > cases when running HelloWorld), so we can quickly determine that by > doing a look up in the super type's dictionary. > > This saves about 250,000 out of about 112M instructions (about 0.22%) > when > running HelloWorld with the default archive on Linux/x64. > > I also renamed a few functions in instanceKlass.hpp to _xxx_shared_xxx > since > they are used only for CDS. > > Thanks > - Ioi > > Hi Ioi, Looks good, I like the move to consolidate the calls to resolve_super or fail into SystemDictionary::check_shared_class_super_type().? I have a minor comment about checking the loader against NULL. classfile/systemDictionary.cpp line #1266 & #1272, please consider using ClassLoaderData::is_the_boot_class_loader_data() Thanks, Lois From ioi.lam at oracle.com Tue Mar 3 17:04:19 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 3 Mar 2020 09:04:19 -0800 Subject: RFR(S) 8240244 Avoid calling resolve_super_or_fail in SystemDictionary::load_shared_class In-Reply-To: References: <5d5c6c4d-0a0f-0c9d-c962-ec9e5e37e077@oracle.com> Message-ID: <4bed29f2-4995-0d0e-c569-d29ffe597a4e@oracle.com> On 3/3/20 3:15 AM, Claes Redestad wrote: > Hi Ioi, > > overall looks good, thanks! > > systemDictionary.cpp > > I'm not sure what you meant to write here: > > +? // + Don't take the lock if loader data is NULL without taking lock. Sorry I was sloppy when editing :-(. I meant: // + Don't take the lock if loader data is NULL. > > > Left-over "!= 0": > > +? if (!super_type->is_shared_unregistered_class() != 0 && > super_type->class_loader_data() != NULL) { > Fixed. Thanks - Ioi > -- > > It seems to me that it might be possible to take the > SystemDictionary_lock just once in check_shared_class_super_types > method, spanning super and all interfaces. That might need some > restructuring since we can't hold that lock when going into > the class-loading slow path, but I think the slow path could profitably > be moved to the outer method outside such a locked section. > > Thanks! > > /Claes > > On 2020-03-03 07:34, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8240244 >> http://cr.openjdk.java.net/~iklam/jdk15/8240244-reduce-resolve-super.v01/ >> >> >> When loading shared classes from the CDS archive, we need to test if the >> actually loaded super types are the same as those used during CDS >> dumping. >> That was done with a pretty expensive call to resolve_super_or_fail. >> >> In most cases, the super type has already been loaded (about 350 out >> of 400 >> cases when running HelloWorld), so we can quickly determine that by >> doing a look up in the super type's dictionary. >> >> This saves about 250,000 out of about 112M instructions (about 0.22%) >> when >> running HelloWorld with the default archive on Linux/x64. >> >> I also renamed a few functions in instanceKlass.hpp to >> _xxx_shared_xxx since >> they are used only for CDS. >> >> Thanks >> - Ioi >> >> From ioi.lam at oracle.com Tue Mar 3 17:05:36 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 3 Mar 2020 09:05:36 -0800 Subject: RFR(S) 8240244 Avoid calling resolve_super_or_fail in SystemDictionary::load_shared_class In-Reply-To: <4b56f1b9-05b5-c633-faf3-7e2672775bbc@oracle.com> References: <5d5c6c4d-0a0f-0c9d-c962-ec9e5e37e077@oracle.com> <4b56f1b9-05b5-c633-faf3-7e2672775bbc@oracle.com> Message-ID: <60b26e83-456f-3bbb-9095-321a1de3f42b@oracle.com> Hi Claes, Thanks for taking your time to try out the idea! - Ioi On 3/3/20 7:19 AM, Claes Redestad wrote: > On 2020-03-03 12:15, Claes Redestad wrote: >> It seems to me that it might be possible to take the >> SystemDictionary_lock just once in check_shared_class_super_types >> method, spanning super and all interfaces. That might need some >> restructuring since we can't hold that lock when going into >> the class-loading slow path, but I think the slow path could profitably >> be moved to the outer method outside such a locked section. > > I took a quick stab at prototyping this idea[1], and it does not appear > to be an improvement over the proposed version. > > /Claes > > [1] http://cr.openjdk.java.net/~redestad/scratch/8240244_lock.00/ From ioi.lam at oracle.com Tue Mar 3 17:09:09 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 3 Mar 2020 09:09:09 -0800 Subject: RFR(S) 8240244 Avoid calling resolve_super_or_fail in SystemDictionary::load_shared_class In-Reply-To: <4db2ef25-3073-c3ca-dc1f-72c6806489b2@oracle.com> References: <5d5c6c4d-0a0f-0c9d-c962-ec9e5e37e077@oracle.com> <4db2ef25-3073-c3ca-dc1f-72c6806489b2@oracle.com> Message-ID: On 3/3/20 8:49 AM, Lois Foltan wrote: > On 3/3/2020 1:34 AM, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8240244 >> http://cr.openjdk.java.net/~iklam/jdk15/8240244-reduce-resolve-super.v01/ >> >> >> When loading shared classes from the CDS archive, we need to test if the >> actually loaded super types are the same as those used during CDS >> dumping. >> That was done with a pretty expensive call to resolve_super_or_fail. >> >> In most cases, the super type has already been loaded (about 350 out >> of 400 >> cases when running HelloWorld), so we can quickly determine that by >> doing a look up in the super type's dictionary. >> >> This saves about 250,000 out of about 112M instructions (about 0.22%) >> when >> running HelloWorld with the default archive on Linux/x64. >> >> I also renamed a few functions in instanceKlass.hpp to >> _xxx_shared_xxx since >> they are used only for CDS. >> >> Thanks >> - Ioi >> >> > Hi Ioi, > > Looks good, I like the move to consolidate the calls to resolve_super > or fail into SystemDictionary::check_shared_class_super_type().? I > have a minor comment about checking the loader against NULL. > > classfile/systemDictionary.cpp line #1266 & #1272, please consider > using ClassLoaderData::is_the_boot_class_loader_data() > Hi Lois, Thanks for the review. The intention of the code is to read loader_data->dictionary() only if InstanceKlass::class_loader_data() is not NULL. If a class X has been loaded by the boot class loader, X->class_loader_data()? is non-null. It's only X->class_loader() that is NULL. Thanks - Ioi > Thanks, > Lois From lois.foltan at oracle.com Tue Mar 3 17:19:04 2020 From: lois.foltan at oracle.com (Lois Foltan) Date: Tue, 3 Mar 2020 12:19:04 -0500 Subject: RFR(S) 8240244 Avoid calling resolve_super_or_fail in SystemDictionary::load_shared_class In-Reply-To: References: <5d5c6c4d-0a0f-0c9d-c962-ec9e5e37e077@oracle.com> <4db2ef25-3073-c3ca-dc1f-72c6806489b2@oracle.com> Message-ID: <771d0c60-b1f2-3468-29b2-b8a5b0e51a8f@oracle.com> On 3/3/2020 12:09 PM, Ioi Lam wrote: > > > On 3/3/20 8:49 AM, Lois Foltan wrote: >> On 3/3/2020 1:34 AM, Ioi Lam wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8240244 >>> http://cr.openjdk.java.net/~iklam/jdk15/8240244-reduce-resolve-super.v01/ >>> >>> >>> When loading shared classes from the CDS archive, we need to test if >>> the >>> actually loaded super types are the same as those used during CDS >>> dumping. >>> That was done with a pretty expensive call to resolve_super_or_fail. >>> >>> In most cases, the super type has already been loaded (about 350 out >>> of 400 >>> cases when running HelloWorld), so we can quickly determine that by >>> doing a look up in the super type's dictionary. >>> >>> This saves about 250,000 out of about 112M instructions (about >>> 0.22%) when >>> running HelloWorld with the default archive on Linux/x64. >>> >>> I also renamed a few functions in instanceKlass.hpp to >>> _xxx_shared_xxx since >>> they are used only for CDS. >>> >>> Thanks >>> - Ioi >>> >>> >> Hi Ioi, >> >> Looks good, I like the move to consolidate the calls to resolve_super >> or fail into SystemDictionary::check_shared_class_super_type().? I >> have a minor comment about checking the loader against NULL. >> >> classfile/systemDictionary.cpp line #1266 & #1272, please consider >> using ClassLoaderData::is_the_boot_class_loader_data() >> > > Hi Lois, > > Thanks for the review. > > The intention of the code is to read loader_data->dictionary() only if > InstanceKlass::class_loader_data() is not NULL. > > If a class X has been loaded by the boot class loader, > X->class_loader_data()? is non-null. It's only X->class_loader() that > is NULL. Ahhh, thanks for the clarification.? I'm good with the changes! Lois > > Thanks > - Ioi > >> Thanks, >> Lois > From m.sundar85 at gmail.com Tue Mar 3 17:39:05 2020 From: m.sundar85 at gmail.com (Sundara Mohan M) Date: Tue, 3 Mar 2020 12:39:05 -0500 Subject: Need help on debugging JVM crash In-Reply-To: References: Message-ID: Hi Yumin, On Tue, Mar 3, 2020 at 11:23 AM Yumin Qi wrote: > HI, Sundara > On 3/3/20 8:02 AM, Sundara Mohan M wrote: > > Hi, > I am seeing JVM crashes on our system in GC Thread with parallel gc on > x86 linux. Observed the same crash happening on JVM-11.0.6/13.0.2/13.0.1 GA > builds. > Adding some logs lines to give some context. > > # > # A fatal error has been detected by the Java Runtime Environment: > # > # SIGSEGV (0xb) at pc=0x00007f669c964311, pid=66684, tid=71106 > # > # JRE version: OpenJDK Runtime Environment (13.0.1+9) (build 13.0.1+9) > # Java VM: OpenJDK 64-Bit Server VM (13.0.1+9, mixed mode, tiered, parallel > gc, linux-amd64) > # Problematic frame: > # V [libjvm.so+0xcd3311] PCMarkAndPushClosure::do_oop(oopDesc**)+0x51 > # > # No core dump will be written. Core dumps have been disabled. To enable > core dumping, try "ulimit -c unlimited" before starting Java again > # > # If you would like to submit a bug report, please visit: > # https://github.com/AdoptOpenJDK/openjdk-build/issues > # > > Host: Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz, 48 cores, 125G, Red Hat > Enterprise Linux Server release 6.10 (Santiago) > Time: Thu Feb 6 11:43:48 2020 UTC elapsed time: 198626 seconds (2d 7h 10m > 26s) > > > Following is the stack trace > ex1: > Stack: [0x00007fd01cbdb000,0x00007fd01ccdb000], sp=0x00007fd01ccd8890, > free space=1014k > Native frames: (J=compiled Java code, A=aot compiled Java code, > j=interpreted, Vv=VM code, C=native code) > *V [libjvm.so+0xcc0121] PCMarkAndPushClosure::do_oop(oopDesc**)+0x51* > V [libjvm.so+0xc58c8b] OopMapSet::oops_do(frame const*, RegisterMap > const*, OopClosure*)+0x2eb > V [libjvm.so+0x7521e9] frame::oops_do_internal(OopClosure*, > CodeBlobClosure*, RegisterMap*, bool)+0x99 > V [libjvm.so+0xf55757] JavaThread::oops_do(OopClosure*, > CodeBlobClosure*)+0x187 > V [libjvm.so+0xcbb100] ThreadRootsMarkingTask::do_it(GCTaskManager*, > unsigned int)+0xb0 > V [libjvm.so+0x7e0f8b] GCTaskThread::run()+0x1eb > V [libjvm.so+0xf5d43d] Thread::call_run()+0x10d > V [libjvm.so+0xc74337] thread_native_entry(Thread*)+0xe7 > > JavaThread 0x00007fbeb9209800 (nid = 82380) was being processed > Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) > v ~RuntimeStub::_new_array_Java > J 62465 c2 > ch.qos.logback.classic.spi.ThrowableProxy.(Ljava/lang/Throwable;)V > (207 bytes) @ 0x00007fd00ad43704 [0x00007fd00ad41420+0x00000000000022e4] > J 474206 c2 > org.eclipse.jetty.util.log.JettyAwareLogger.log(Lorg/slf4j/Marker;ILjava/lang/String;[Ljava/lang/Object;Ljava/lang/Throwable;)V > (134 bytes) @ 0x00007fd00c4e81ec [0x00007fd00c4e7ee0+0x000000000000030c] > j > org.eclipse.jetty.util.log.JettyAwareLogger.warn(Ljava/lang/String;Ljava/lang/Throwable;)V+7 > j > org.eclipse.jetty.util.log.Slf4jLog.warn(Ljava/lang/String;Ljava/lang/Throwable;)V+6 > j > org.eclipse.jetty.server.HttpChannel.handleException(Ljava/lang/Throwable;)V+181 > j > org.eclipse.jetty.server.HttpChannelOverHttp.handleException(Ljava/lang/Throwable;)V+13 > J 64106 c2 org.eclipse.jetty.server.HttpChannel.handle()Z (997 bytes) @ > 0x00007fd00c6d2cd4 [0x00007fd00c6cdec0+0x0000000000004e14] > J 280430 c2 org.eclipse.jetty.server.HttpConnection.onFillable()V (334 > bytes) @ 0x00007fd00da925f0 [0x00007fd00da91e40+0x00000000000007b0] > J 41979 c2 org.eclipse.jetty.io.ChannelEndPoint$2.run()V (12 bytes) @ > 0x00007fd00a14f604 [0x00007fd00a14f4e0+0x0000000000000124] > J 86362 c2 org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run()V > (565 bytes) @ 0x00007fd0087d7e34 [0x00007fd0087d7cc0+0x0000000000000174] > J 75998 c2 java.lang.Thread.run()V java.base at 13.0.2 (17 bytes) @ > 0x00007fd00c93b8d8 [0x00007fd00c93b8a0+0x0000000000000038] > v ~StubRoutines::call_stub > > ex2: > Stack: [0x00007f669869f000,0x00007f669879f000], sp=0x00007f669879c890, > free space=1014k > Native frames: (J=compiled Java code, A=aot compiled Java code, > j=interpreted, Vv=VM code, C=native code) > > *V [libjvm.so+0xcd3311] PCMarkAndPushClosure::do_oop(oopDesc**)+0x51*V > [libjvm.so+0xc6bf0b] OopMapSet::oops_do(frame const*, RegisterMap const*, > OopClosure*)+0x2eb > V [libjvm.so+0x765489] frame::oops_do_internal(OopClosure*, > CodeBlobClosure*, RegisterMap*, bool)+0x99 > V [libjvm.so+0xf68b17] JavaThread::oops_do(OopClosure*, > CodeBlobClosure*)+0x187 > V [libjvm.so+0xcce2f0] ThreadRootsMarkingTask::do_it(GCTaskManager*, > unsigned int)+0xb0 > V [libjvm.so+0x7f422b] GCTaskThread::run()+0x1eb > V [libjvm.so+0xf707fd] Thread::call_run()+0x10d > V [libjvm.so+0xc875b7] thread_native_entry(Thread*)+0xe7 > > JavaThread 0x00007f5518004000 (nid = 75659) was being processed > Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) > v ~RuntimeStub::_new_array_Java > J 54174 c2 > ch.qos.logback.classic.spi.ThrowableProxy.(Ljava/lang/Throwable;)V > (207 bytes) @ 0x00007f6687d92678 [0x00007f6687d8c700+0x0000000000005f78] > J 334031 c2 > com.xmas.webservice.exception.ExceptionLoggingWrapper.execute()V (1004 > bytes) @ 0x00007f6686ede430 [0x00007f6686edd580+0x0000000000000eb0] > J 53431 c2 > com.xmas.webservice.exception.mapper.AbstractExceptionMapper.toResponse(Lcom/xmas/beans/exceptions/mapper/V3ErrorCode;Ljava/lang/Exception;)Ljavax/ws/rs/core/Response; > (105 bytes) @ 0x00007f6687db88b0 [0x00007f6687db8660+0x0000000000000250] > J 63819 c2 > com.xmas.webservice.exception.mapper.RequestBlockedExceptionMapper.toResponse(Ljava/lang/Exception;)Ljavax/ws/rs/core/Response; > (9 bytes) @ 0x00007f6686a6ed9c [0x00007f6686a6ecc0+0x00000000000000dc] > J 334032 c2 > com.xmas.webservice.filters.ResponseSerializationWorker.processException()Ljava/io/InputStream; > (332 bytes) @ 0x00007f668992ad34 [0x00007f668992a840+0x00000000000004f4] > J 403918 c2 > com.xmas.webservice.filters.ResponseSerializationWorker.execute()Z (272 > bytes) @ 0x00007f66869d67fc [0x00007f66869d5e80+0x000000000000097c] > J 17530 c2 com.lafaspot.common.concurrent.internal.WorkerWrapper.execute()Z > (208 bytes) @ 0x00007f66848b3708 [0x00007f66848b36a0+0x0000000000000068] > J 31970% c2 > com.lafaspot.common.concurrent.internal.WorkerManagerOneThread.call()Lcom/lafaspot/common/concurrent/internal/WorkerManagerState; > (486 bytes) @ 0x00007f668608dcb0 [0x00007f668608d5e0+0x00000000000006d0] > j > com.lafaspot.common.concurrent.internal.WorkerManagerOneThread.call()Ljava/lang/Object;+1 > J 4889 c1 java.util.concurrent.FutureTask.run()V java.base at 13.0.1 (123 > bytes) @ 0x00007f667d0be604 [0x00007f667d0bdf80+0x0000000000000684] > J 7487 c1 > java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)Vjava.base at 13.0.1 (187 bytes) @ 0x00007f667dd45854 > [0x00007f667dd44a60+0x0000000000000df4] > J 7486 c1 java.util.concurrent.ThreadPoolExecutor$Worker.run()Vjava.base at 13.0.1 (9 bytes) @ 0x00007f667d1f643c > [0x00007f667d1f63c0+0x000000000000007c] > J 7078 c1 java.lang.Thread.run()V java.base at 13.0.1 (17 bytes) @ > 0x00007f667d1f2d74 [0x00007f667d1f2c40+0x0000000000000134] > v ~StubRoutines::call_stub > > Not very frequent but ~90 days ~120 crashes with following signal > siginfo: si_signo: 11 (SIGSEGV), si_code: 128 (SI_KERNEL), si_addr: > 0x0000000000000000 > This signal is generated when we try to access non canonical address in > linux. > > As suggested by Stefan in another thread i tried to > add VerifyAfterGC/VerifyBeforeGC but it seems to increase the latency and > applications not surviving our production traffic(timing out and requests > are failing). > > Questions > 1. When i looked at source code for printing stack trace i see followinghttps://github.com/openjdk/jdk11u/blob/master/src/hotspot/share/utilities/vmError.cpp#L696 > (Prints native stack trace)https://github.com/openjdk/jdk11u/blob/master/src/hotspot/share/utilities/vmError.cpp#L718 > (printing Java thread stack trace if it is involved in GC crash) > a. How do you know this java thread was involved in jvm crash? > > When GC processes thread stack as root, the java thread first was > recorded. This is why at crash, the java thread was printed out. > > b. Can i assume the java thread printed after native stack trace was the > culprit? > > Please check this thread stack frames, when GC is doing marking work, I > think, it encountered a bad oop. Check: > > If it is a compiled frame, if so, it may related to compiled code. > > c. Since i am seeing the same frame (~RuntimeStub::_new_array_Java, J > 54174 c2 ch.qos.logback.classic.spi.ThrowableProxy...) but different > stack trace in both crashes can this be the root cause? > > It is a C2 compiled frame. The bad oop could be a result of compiler. > Actually the top two frame are always same in different crashes v ~RuntimeStub::_new_array_Java J 54174 c2 ch.qos.logback.classic.spi.ThrowableProxy.(Ljava/lang/Throwable;)V (207 bytes) @ 0x00007f6687d92678 [0x00007f6687d8c700+0x0000000000005f78] In this case do you think JVM code(frame 1) or C2 compiler code(frame 2) might be issue? Is there any way to identify that and what kind of debug flags/settings might give us this information? > It also needs detail debug information to make the conclusion. > Do you think any of the information dumped in hs_err* file might give us more info (like registers content/Instructions/core file)? Can you please let me know what additional details might help to make the conclusion? Also how to get those information? Thanks > > Yumin > > 2. Thinking of excluding compilation > of ch.qos.logback.classic.spi.ThrowableProxy class and running in > production to see if compilation of this method is the cause. Does it make > sense? > > 3. Any other suggestion on debugging this further? > > TIA > Sundar > > Thanks Sundar From yumin.qi at oracle.com Tue Mar 3 17:42:45 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Tue, 3 Mar 2020 09:42:45 -0800 Subject: RFR(S) 8240244 Avoid calling resolve_super_or_fail in SystemDictionary::load_shared_class In-Reply-To: <5d5c6c4d-0a0f-0c9d-c962-ec9e5e37e077@oracle.com> References: <5d5c6c4d-0a0f-0c9d-c962-ec9e5e37e077@oracle.com> Message-ID: <8e76d935-064a-4cfc-dd19-9bcb78573cd5@oracle.com> Hi, Ioi ?? Looks good to me. ?? One comment: 1276 if (SystemDictionary::find_class(d_hash, name, dictionary) != NULL) { Should be replaced with == super_type? Thanks Yumin On 3/2/20 10:34 PM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8240244 > http://cr.openjdk.java.net/~iklam/jdk15/8240244-reduce-resolve-super.v01/ > > When loading shared classes from the CDS archive, we need to test if the > actually loaded super types are the same as those used during CDS > dumping. > That was done with a pretty expensive call to resolve_super_or_fail. > > In most cases, the super type has already been loaded (about 350 out > of 400 > cases when running HelloWorld), so we can quickly determine that by > doing a look up in the super type's dictionary. > > This saves about 250,000 out of about 112M instructions (about 0.22%) > when > running HelloWorld with the default archive on Linux/x64. > > I also renamed a few functions in instanceKlass.hpp to _xxx_shared_xxx > since > they are used only for CDS. > > Thanks > - Ioi > > From ioi.lam at oracle.com Tue Mar 3 18:01:47 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 3 Mar 2020 10:01:47 -0800 Subject: RFR(S) 8240244 Avoid calling resolve_super_or_fail in SystemDictionary::load_shared_class In-Reply-To: <8e76d935-064a-4cfc-dd19-9bcb78573cd5@oracle.com> References: <5d5c6c4d-0a0f-0c9d-c962-ec9e5e37e077@oracle.com> <8e76d935-064a-4cfc-dd19-9bcb78573cd5@oracle.com> Message-ID: On 3/3/20 9:42 AM, Yumin Qi wrote: > > Hi, Ioi > > ?? Looks good to me. > > ?? One comment: > > 1276 if (SystemDictionary::find_class(d_hash, name, dictionary) != > NULL) { Should be replaced with == super_type? Hi Yumin, Thanks for the review. You're right. I changed it to "== super_type". Re-running the tests. - Ioi > Thanks Yumin > > On 3/2/20 10:34 PM, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8240244 >> http://cr.openjdk.java.net/~iklam/jdk15/8240244-reduce-resolve-super.v01/ >> >> >> When loading shared classes from the CDS archive, we need to test if the >> actually loaded super types are the same as those used during CDS >> dumping. >> That was done with a pretty expensive call to resolve_super_or_fail. >> >> In most cases, the super type has already been loaded (about 350 out >> of 400 >> cases when running HelloWorld), so we can quickly determine that by >> doing a look up in the super type's dictionary. >> >> This saves about 250,000 out of about 112M instructions (about 0.22%) >> when >> running HelloWorld with the default archive on Linux/x64. >> >> I also renamed a few functions in instanceKlass.hpp to >> _xxx_shared_xxx since >> they are used only for CDS. >> >> Thanks >> - Ioi >> >> From aph at redhat.com Tue Mar 3 18:02:59 2020 From: aph at redhat.com (Andrew Haley) Date: Tue, 3 Mar 2020 18:02:59 +0000 Subject: Need help on debugging JVM crash In-Reply-To: References: Message-ID: <442a8045-8ef6-00ae-cd61-2db6f1fdb5fd@redhat.com> On 3/3/20 5:39 PM, Sundara Mohan M wrote: >> Questions >> 1. When i looked at source code for printing stack trace i see followinghttps://github.com/openjdk/jdk11u/blob/master/src/hotspot/share/utilities/vmError.cpp#L696 >> (Prints native stack trace)https://github.com/openjdk/jdk11u/blob/master/src/hotspot/share/utilities/vmError.cpp#L718 >> (printing Java thread stack trace if it is involved in GC crash) >> a. How do you know this java thread was involved in jvm crash? The top thread -- the first in the file -- is the one that crashed. >> When GC processes thread stack as root, the java thread first was >> recorded. This is why at crash, the java thread was printed out. >> >> b. Can i assume the java thread printed after native stack trace was the >> culprit? Certainly not. >> Please check this thread stack frames, when GC is doing marking work, I >> think, it encountered a bad oop. Check: >> >> If it is a compiled frame, if so, it may related to compiled code. >> >> c. Since i am seeing the same frame (~RuntimeStub::_new_array_Java, J >> 54174 c2 ch.qos.logback.classic.spi.ThrowableProxy...) but different >> stack trace in both crashes can this be the root cause? >> >> It is a C2 compiled frame. The bad oop could be a result of compiler. >> > Actually the top two frame are always same in different crashes > v ~RuntimeStub::_new_array_Java > J 54174 c2 > ch.qos.logback.classic.spi.ThrowableProxy.(Ljava/lang/Throwable;)V > (207 bytes) @ 0x00007f6687d92678 [0x00007f6687d8c700+0x0000000000005f78] > In this case do you think JVM code(frame 1) or C2 compiler code(frame 2) > might be issue? Probably not. My money would be on a bad library using Unsafe to do something unwise. But there are many other possibilities. > Is there any way to identify that and what kind of debug flags/settings > might give us this information? > >> It also needs detail debug information to make the conclusion. >> > Do you think any of the information dumped in hs_err* file might give us > more info (like registers content/Instructions/core file)? > > Can you please let me know what additional details might help to make the > conclusion? Also how to get those information? Let's see the complete hs_err file. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From m.sundar85 at gmail.com Tue Mar 3 18:13:24 2020 From: m.sundar85 at gmail.com (Sundara Mohan M) Date: Tue, 3 Mar 2020 13:13:24 -0500 Subject: Need help on debugging JVM crash In-Reply-To: References: <442a8045-8ef6-00ae-cd61-2db6f1fdb5fd@redhat.com> Message-ID: Waiting for moderator approval to get my hs_err* files sent. Is being held until the list moderator can review it for approval. The reason it is being held: Message body is too big: 1048807 bytes with a limit of 500 KB Thanks Sundar On Tue, Mar 3, 2020 at 1:07 PM Sundara Mohan M wrote: > Hi Andrew, > Attaching hs_err* from multiple hosts where both java thread top frame > is same. > > Thanks > Sundar > > On Tue, Mar 3, 2020 at 1:03 PM Andrew Haley wrote: > >> On 3/3/20 5:39 PM, Sundara Mohan M wrote: >> >> Questions >> >> 1. When i looked at source code for printing stack trace i see >> followinghttps:// >> github.com/openjdk/jdk11u/blob/master/src/hotspot/share/utilities/vmError.cpp#L696 >> >> (Prints native stack trace) >> https://github.com/openjdk/jdk11u/blob/master/src/hotspot/share/utilities/vmError.cpp#L718 >> >> (printing Java thread stack trace if it is involved in GC crash) >> >> a. How do you know this java thread was involved in jvm crash? >> >> The top thread -- the first in the file -- is the one that crashed. >> >> >> When GC processes thread stack as root, the java thread first was >> >> recorded. This is why at crash, the java thread was printed out. >> >> >> >> b. Can i assume the java thread printed after native stack trace was >> the >> >> culprit? >> >> Certainly not. >> >> >> Please check this thread stack frames, when GC is doing marking work, I >> >> think, it encountered a bad oop. Check: >> >> >> >> If it is a compiled frame, if so, it may related to compiled code. >> >> >> >> c. Since i am seeing the same frame (~RuntimeStub::_new_array_Java, J >> >> 54174 c2 ch.qos.logback.classic.spi.ThrowableProxy...) but >> different >> >> stack trace in both crashes can this be the root cause? >> >> >> >> It is a C2 compiled frame. The bad oop could be a result of compiler. >> >> >> > Actually the top two frame are always same in different crashes >> > v ~RuntimeStub::_new_array_Java >> > J 54174 c2 >> > ch.qos.logback.classic.spi.ThrowableProxy.(Ljava/lang/Throwable;)V >> > (207 bytes) @ 0x00007f6687d92678 [0x00007f6687d8c700+0x0000000000005f78] >> > In this case do you think JVM code(frame 1) or C2 compiler code(frame 2) >> > might be issue? >> >> Probably not. My money would be on a bad library using Unsafe to do >> something unwise. But there are many other possibilities. >> >> > Is there any way to identify that and what kind of debug flags/settings >> > might give us this information? >> > >> >> It also needs detail debug information to make the conclusion. >> >> >> > Do you think any of the information dumped in hs_err* file might give us >> > more info (like registers content/Instructions/core file)? >> > >> > Can you please let me know what additional details might help to make >> the >> > conclusion? Also how to get those information? >> >> Let's see the complete hs_err file. >> >> -- >> Andrew Haley (he/him) >> Java Platform Lead Engineer >> Red Hat UK Ltd. >> https://keybase.io/andrewhaley >> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 >> >> From mandy.chung at oracle.com Tue Mar 3 18:40:55 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 3 Mar 2020 10:40:55 -0800 Subject: RFR 8240324: Improve is_boot_class_loader_data() by adding simple check In-Reply-To: <1e54594e-c3cf-9ca3-563b-6b094b8e3fc3@oracle.com> References: <1e54594e-c3cf-9ca3-563b-6b094b8e3fc3@oracle.com> Message-ID: <7110c754-2369-02ca-dd2e-ffeebec785b1@oracle.com> Hi Harold, Looks good.? Thanks for fixing it. Mandy On 3/3/20 7:33 AM, Harold Seigel wrote: > Hi, > > Please review this trivial fix for JDK-8240324 to potentially avoid > accessing the class loader oop. > > Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8240324/webrev/ > > JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8240324 > > The fix was regression tested by running Mach5 tiers 1 and 2 tests and > builds on Linux-x64, Solaris, Windows, and Mac OS X, by running Mach5 > tiers 3-5 tests on Linux-x64, and JCK lang and VM tests on Linux-x64. > > Thanks, Harold > From yumin.qi at oracle.com Tue Mar 3 18:49:06 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Tue, 3 Mar 2020 10:49:06 -0800 Subject: Need help on debugging JVM crash In-Reply-To: References: Message-ID: Hi, Sundara As suggested by Stefan in another thread i tried to > >> add VerifyAfterGC/VerifyBeforeGC but it seems to increase the latency and >> applications not surviving our production traffic(timing out and requests >> are failing). >> >> Questions >> 1. When i looked at source code for printing stack trace i see following >> https://github.com/openjdk/jdk11u/blob/master/src/hotspot/share/utilities/vmError.cpp#L696 >> (Prints native stack trace) >> https://github.com/openjdk/jdk11u/blob/master/src/hotspot/share/utilities/vmError.cpp#L718 >> (printing Java thread stack trace if it is involved in GC crash) >> a. How do you know this java thread was involved in jvm crash? > When GC processes thread stack as root, the java thread first was > recorded. This is why at crash, the java thread was printed out. >> b. Can i assume the java thread printed after native stack trace was the >> culprit? > > Please check this thread stack frames, when GC is doing marking > work, I think, it encountered a bad oop. Check: > > If it is a compiled frame, if so, it may related to compiled code. > >> c. Since i am seeing the same frame (~RuntimeStub::_new_array_Java, J >> 54174 c2 ch.qos.logback.classic.spi.ThrowableProxy...) but different >> stack trace in both crashes can this be the root cause? > > It is a C2 compiled frame. The bad oop could be a result of compiler. > > Actually the top two frame are always same in different crashes > v ~RuntimeStub::_new_array_Java > J 54174 c2 > ch.qos.logback.classic.spi.ThrowableProxy.(Ljava/lang/Throwable;)V > (207 bytes) @ 0x00007f6687d92678 [0x00007f6687d8c700+0x0000000000005f78] > In this case do you think JVM code(frame 1) or C2 compiler code(frame > 2) might be issue? > Is there any way to identify that and what kind of debug > flags/settings might give us this information? > > It also needs detail debug information to make the conclusion. > > Do you think any of the information dumped in hs_err* file might give > us more info (like registers content/Instructions/core file)? > > Can you please let me know what additional details might help to make > the conclusion? Also how to get those information? > If it is caused by this compiled java method, excluding the java method from compilation is a workaround. You can switch to the java thread (the printed out java thread at crash), compare the failed frame in GC thread to the frame in the java thread so you will know which frame contained bad oop. Also know what is the frame, compiled, interpreter, or native. Yumin > Thanks > > Yumin > >> 2. Thinking of excluding compilation >> of ch.qos.logback.classic.spi.ThrowableProxy class and running in >> production to see if compilation of this method is the cause. Does it make >> sense? >> >> 3. Any other suggestion on debugging this further? >> >> TIA >> Sundar > > > Thanks > Sundar From stefan.karlsson at oracle.com Tue Mar 3 18:57:40 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 3 Mar 2020 19:57:40 +0100 Subject: Need help on debugging JVM crash In-Reply-To: References: <442a8045-8ef6-00ae-cd61-2db6f1fdb5fd@redhat.com> Message-ID: <7a572768-b73b-9d6e-db63-a39583f0c507@oracle.com> I have approved the message, but it isn't arriving. As a workaround, could you try send one hs_err file at a time, and cut the rest of the message? Each hs_err file is < 500 KB, so maybe that will work. StefanK On 2020-03-03 19:13, Sundara Mohan M wrote: > Waiting for moderator approval to get my hs_err* files sent. > > Is being held until the list moderator can review it for approval. > > The reason it is being held: > > Message body is too big: 1048807 bytes with a limit of 500 KB > > Thanks > Sundar > > On Tue, Mar 3, 2020 at 1:07 PM Sundara Mohan M wrote: > >> Hi Andrew, >> Attaching hs_err* from multiple hosts where both java thread top frame >> is same. >> >> Thanks >> Sundar >> >> On Tue, Mar 3, 2020 at 1:03 PM Andrew Haley wrote: >> >>> On 3/3/20 5:39 PM, Sundara Mohan M wrote: >>>>> Questions >>>>> 1. When i looked at source code for printing stack trace i see >>> followinghttps:// >>> github.com/openjdk/jdk11u/blob/master/src/hotspot/share/utilities/vmError.cpp#L696 >>>>> (Prints native stack trace) >>> https://github.com/openjdk/jdk11u/blob/master/src/hotspot/share/utilities/vmError.cpp#L718 >>>>> (printing Java thread stack trace if it is involved in GC crash) >>>>> a. How do you know this java thread was involved in jvm crash? >>> The top thread -- the first in the file -- is the one that crashed. >>> >>>>> When GC processes thread stack as root, the java thread first was >>>>> recorded. This is why at crash, the java thread was printed out. >>>>> >>>>> b. Can i assume the java thread printed after native stack trace was >>> the >>>>> culprit? >>> Certainly not. >>> >>>>> Please check this thread stack frames, when GC is doing marking work, I >>>>> think, it encountered a bad oop. Check: >>>>> >>>>> If it is a compiled frame, if so, it may related to compiled code. >>>>> >>>>> c. Since i am seeing the same frame (~RuntimeStub::_new_array_Java, J >>>>> 54174 c2 ch.qos.logback.classic.spi.ThrowableProxy...) but >>> different >>>>> stack trace in both crashes can this be the root cause? >>>>> >>>>> It is a C2 compiled frame. The bad oop could be a result of compiler. >>>>> >>>> Actually the top two frame are always same in different crashes >>>> v ~RuntimeStub::_new_array_Java >>>> J 54174 c2 >>>> ch.qos.logback.classic.spi.ThrowableProxy.(Ljava/lang/Throwable;)V >>>> (207 bytes) @ 0x00007f6687d92678 [0x00007f6687d8c700+0x0000000000005f78] >>>> In this case do you think JVM code(frame 1) or C2 compiler code(frame 2) >>>> might be issue? >>> Probably not. My money would be on a bad library using Unsafe to do >>> something unwise. But there are many other possibilities. >>> >>>> Is there any way to identify that and what kind of debug flags/settings >>>> might give us this information? >>>> >>>>> It also needs detail debug information to make the conclusion. >>>>> >>>> Do you think any of the information dumped in hs_err* file might give us >>>> more info (like registers content/Instructions/core file)? >>>> >>>> Can you please let me know what additional details might help to make >>> the >>>> conclusion? Also how to get those information? >>> Let's see the complete hs_err file. >>> >>> -- >>> Andrew Haley (he/him) >>> Java Platform Lead Engineer >>> Red Hat UK Ltd. >>> https://keybase.io/andrewhaley >>> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 >>> >>> From harold.seigel at oracle.com Tue Mar 3 19:00:12 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Tue, 3 Mar 2020 14:00:12 -0500 Subject: RFR 8240324: Improve is_boot_class_loader_data() by adding simple check In-Reply-To: <7110c754-2369-02ca-dd2e-ffeebec785b1@oracle.com> References: <1e54594e-c3cf-9ca3-563b-6b094b8e3fc3@oracle.com> <7110c754-2369-02ca-dd2e-ffeebec785b1@oracle.com> Message-ID: <885c5cf2-d610-e340-1f19-9723f3245c84@oracle.com> Thanks Mandy!? And thanks for suggesting the change. Harold On 3/3/2020 1:40 PM, Mandy Chung wrote: > Hi Harold, > > Looks good.? Thanks for fixing it. > > Mandy > > On 3/3/20 7:33 AM, Harold Seigel wrote: >> Hi, >> >> Please review this trivial fix for JDK-8240324 to potentially avoid >> accessing the class loader oop. >> >> Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8240324/webrev/ >> >> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8240324 >> >> The fix was regression tested by running Mach5 tiers 1 and 2 tests >> and builds on Linux-x64, Solaris, Windows, and Mac OS X, by running >> Mach5 tiers 3-5 tests on Linux-x64, and JCK lang and VM tests on >> Linux-x64. >> >> Thanks, Harold >> > From m.sundar85 at gmail.com Tue Mar 3 19:00:41 2020 From: m.sundar85 at gmail.com (Sundara Mohan M) Date: Tue, 3 Mar 2020 14:00:41 -0500 Subject: Need help on debugging JVM crash In-Reply-To: References: <442a8045-8ef6-00ae-cd61-2db6f1fdb5fd@redhat.com> <7a572768-b73b-9d6e-db63-a39583f0c507@oracle.com> Message-ID: Another crash file. Thanks Sundar On Tue, Mar 3, 2020 at 2:00 PM Sundara Mohan M wrote: > Attaching 1 file as a work around! > > Thanks > Sundar > > On Tue, Mar 3, 2020 at 1:57 PM Stefan Karlsson > wrote: > >> I have approved the message, but it isn't arriving. As a workaround, >> could you try send one hs_err file at a time, and cut the rest of the >> message? Each hs_err file is < 500 KB, so maybe that will work. >> >> StefanK >> >> On 2020-03-03 19:13, Sundara Mohan M wrote: >> > Waiting for moderator approval to get my hs_err* files sent. >> > >> > Is being held until the list moderator can review it for approval. >> > >> > The reason it is being held: >> > >> > Message body is too big: 1048807 bytes with a limit of 500 KB >> > >> > Thanks >> > Sundar >> > >> > On Tue, Mar 3, 2020 at 1:07 PM Sundara Mohan M >> wrote: >> > >> >> Hi Andrew, >> >> Attaching hs_err* from multiple hosts where both java thread top >> frame >> >> is same. >> >> >> >> Thanks >> >> Sundar >> >> >> >> On Tue, Mar 3, 2020 at 1:03 PM Andrew Haley wrote: >> >> >> >>> On 3/3/20 5:39 PM, Sundara Mohan M wrote: >> >>>>> Questions >> >>>>> 1. When i looked at source code for printing stack trace i see >> >>> followinghttps:// >> >>> >> github.com/openjdk/jdk11u/blob/master/src/hotspot/share/utilities/vmError.cpp#L696 >> >>>>> (Prints native stack trace) >> >>> >> https://github.com/openjdk/jdk11u/blob/master/src/hotspot/share/utilities/vmError.cpp#L718 >> >>>>> (printing Java thread stack trace if it is involved in GC crash) >> >>>>> a. How do you know this java thread was involved in jvm crash? >> >>> The top thread -- the first in the file -- is the one that crashed. >> >>> >> >>>>> When GC processes thread stack as root, the java thread first was >> >>>>> recorded. This is why at crash, the java thread was printed out. >> >>>>> >> >>>>> b. Can i assume the java thread printed after native stack trace >> was >> >>> the >> >>>>> culprit? >> >>> Certainly not. >> >>> >> >>>>> Please check this thread stack frames, when GC is doing marking >> work, I >> >>>>> think, it encountered a bad oop. Check: >> >>>>> >> >>>>> If it is a compiled frame, if so, it may related to compiled code. >> >>>>> >> >>>>> c. Since i am seeing the same frame >> (~RuntimeStub::_new_array_Java, J >> >>>>> 54174 c2 ch.qos.logback.classic.spi.ThrowableProxy...) but >> >>> different >> >>>>> stack trace in both crashes can this be the root cause? >> >>>>> >> >>>>> It is a C2 compiled frame. The bad oop could be a result of >> compiler. >> >>>>> >> >>>> Actually the top two frame are always same in different crashes >> >>>> v ~RuntimeStub::_new_array_Java >> >>>> J 54174 c2 >> >>>> >> ch.qos.logback.classic.spi.ThrowableProxy.(Ljava/lang/Throwable;)V >> >>>> (207 bytes) @ 0x00007f6687d92678 >> [0x00007f6687d8c700+0x0000000000005f78] >> >>>> In this case do you think JVM code(frame 1) or C2 compiler >> code(frame 2) >> >>>> might be issue? >> >>> Probably not. My money would be on a bad library using Unsafe to do >> >>> something unwise. But there are many other possibilities. >> >>> >> >>>> Is there any way to identify that and what kind of debug >> flags/settings >> >>>> might give us this information? >> >>>> >> >>>>> It also needs detail debug information to make the conclusion. >> >>>>> >> >>>> Do you think any of the information dumped in hs_err* file might >> give us >> >>>> more info (like registers content/Instructions/core file)? >> >>>> >> >>>> Can you please let me know what additional details might help to make >> >>> the >> >>>> conclusion? Also how to get those information? >> >>> Let's see the complete hs_err file. >> >>> >> >>> -- >> >>> Andrew Haley (he/him) >> >>> Java Platform Lead Engineer >> >>> Red Hat UK Ltd. >> >>> https://keybase.io/andrewhaley >> >>> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 >> >>> >> >>> >> >> From m.sundar85 at gmail.com Tue Mar 3 19:02:50 2020 From: m.sundar85 at gmail.com (Sundara Mohan M) Date: Tue, 3 Mar 2020 14:02:50 -0500 Subject: Need help on debugging JVM crash In-Reply-To: References: <442a8045-8ef6-00ae-cd61-2db6f1fdb5fd@redhat.com> <7a572768-b73b-9d6e-db63-a39583f0c507@oracle.com> Message-ID: Trying to send crash file1 alone. Thanks Sundar From m.sundar85 at gmail.com Tue Mar 3 19:12:03 2020 From: m.sundar85 at gmail.com (Sundara Mohan M) Date: Tue, 3 Mar 2020 14:12:03 -0500 Subject: Need help on debugging JVM crash In-Reply-To: References: <442a8045-8ef6-00ae-cd61-2db6f1fdb5fd@redhat.com> <7a572768-b73b-9d6e-db63-a39583f0c507@oracle.com> Message-ID: Sorry for spamming, can someone confirm if you received 2 crash report files? I tried sending separately but only 1 file went through other still says message body more than 500K. Thanks Sundar On Tue, Mar 3, 2020 at 2:08 PM Sundara Mohan M wrote: > From ioi.lam at oracle.com Tue Mar 3 19:27:21 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 3 Mar 2020 11:27:21 -0800 Subject: Need help on debugging JVM crash In-Reply-To: References: <442a8045-8ef6-00ae-cd61-2db6f1fdb5fd@redhat.com> <7a572768-b73b-9d6e-db63-a39583f0c507@oracle.com> Message-ID: <8c2b3189-797f-a75e-c6de-a14f9bd7264d@oracle.com> For me, at least, your attachment has been filtered out by the mail server. Since this is an AdoptJDK build, I would suggest filing a bug according to the hs_err file # If you would like to submit a bug report, please visit: #?? https://github.com/AdoptOpenJDK/openjdk-build/issues and then attach your hs_err log there, and then post the URL of the bug here. Thanks - Ioi On 3/3/20 11:12 AM, Sundara Mohan M wrote: > Sorry for spamming, can someone confirm if you received 2 crash report > files? I tried sending separately but only 1 file went through other still > says message body more than 500K. > > Thanks > Sundar > > On Tue, Mar 3, 2020 at 2:08 PM Sundara Mohan M wrote: > From m.sundar85 at gmail.com Tue Mar 3 19:30:57 2020 From: m.sundar85 at gmail.com (Sundara Mohan M) Date: Tue, 3 Mar 2020 14:30:57 -0500 Subject: Need help on debugging JVM crash In-Reply-To: <8c2b3189-797f-a75e-c6de-a14f9bd7264d@oracle.com> References: <442a8045-8ef6-00ae-cd61-2db6f1fdb5fd@redhat.com> <7a572768-b73b-9d6e-db63-a39583f0c507@oracle.com> <8c2b3189-797f-a75e-c6de-a14f9bd7264d@oracle.com> Message-ID: Hi Ioi, Thanks for the information. I have uploaded logs here https://github.com/AdoptOpenJDK/openjdk-support/issues/69 Thanks Sundar On Tue, Mar 3, 2020 at 2:27 PM Ioi Lam wrote: > For me, at least, your attachment has been filtered out by the mail server. > > Since this is an AdoptJDK build, I would suggest filing a bug according > to the hs_err file > > # If you would like to submit a bug report, please visit: > # https://github.com/AdoptOpenJDK/openjdk-build/issues > > and then attach your hs_err log there, and then post the URL of the bug > here. > > Thanks > - Ioi > > On 3/3/20 11:12 AM, Sundara Mohan M wrote: > > Sorry for spamming, can someone confirm if you received 2 crash report > > files? I tried sending separately but only 1 file went through other > still > > says message body more than 500K. > > > > Thanks > > Sundar > > > > On Tue, Mar 3, 2020 at 2:08 PM Sundara Mohan M > wrote: > > > > From ioi.lam at oracle.com Tue Mar 3 20:12:04 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 3 Mar 2020 12:12:04 -0800 Subject: Need help on debugging JVM crash In-Reply-To: References: Message-ID: <86c8b74f-9f11-b9b5-00f2-30360d0e8b6f@oracle.com> The crash happened while the GC is running. I tried disasm of the crashing address PCMarkAndPushClosure::do_oop(oopDesc**)+0x51 (from the 13.0.1+9 GA binaries of AdoptJDK) (gdb) x/80i _ZN20PCMarkAndPushClosure6do_oopEPP7oopDesc ?? <>:??? push?? %rbp ?? <+1>:??? mov??? %rsp,%rbp ?? <+4>:??? push?? %r13 ?? <+6>:??? push?? %r12 ?? <+8>:??? push?? %rbx ?? <+9>:??? sub??? $0x8,%rsp ?? <+13>:??? mov??? (%rsi),%rbx? ;;; rbx = oop ?? <+16>:??? test?? %rbx,%rbx??? ;;; oop != null? ?? <+19>:??? je???? 0x7ffff67ca317 <_ZN20PCMarkAndPushClosure6do_oopEPP7oopDesc+87> ?? <+21>:??? lea??? 0x9c0afc(%rip),%rax??????? # 0x7ffff718add8 <_ZN20ParCompactionManager12_mark_bitmapE> ?? <+28>:??? mov??? %rbx,%rcx??? ;;; rcx = oop ?? <+31>:??? mov??? (%rax),%rdx? ;;; rdx = ParCompactionManager::_mark_bitmap ?? <+34>:??? sub??? (%rdx),%rcx? ;;; rcx = oop - _mark_bitmap->_region_start ?? <+37>:??? mov??? 0x10(%rdx),%rdx ;; rdx = _mark_bitmap->_beg_bits->_map ?? <+41>:??? mov??? %rcx,%rax?? ;;;? rax = oop - _mark_bitmap->_region_start ?? <+44>:??? lea??? 0x93b935(%rip),%rcx?????? # 0x7ffff7105c28 ?? <+51>:??? shr??? $0x3,%rax ?? <+55>:??? mov??? (%rcx),%ecx ?? <+57>:??? shr??? %cl,%rax ?? <+60>:??? mov??? %rax,%rcx ?? <+63>:??? mov??? %rax,%rsi????? ;;; rsi = index of oop inside mark_bitmap ?? <+66>:??? mov??? $0x1,%eax ?? <+71>:??? and??? $0x3f,%ecx ?? <+74>:??? shr??? $0x6,%rsi ?? <+78>:??? shl??? %cl,%rax ?? <+81>:??? test?? %rax,(%rdx,%rsi,8) << crash This looks like that the oop that we try to mark is actually outside of the heap range, so trying to mark it in the mark_bitmap causes this: ?? siginfo: si_signo: 11 (SIGSEGV), si_code: 128 (SI_KERNEL), si_addr: ?? 0x0000000000000000 Here are the values of the registers for the "test" instruction above: ??? RAX=0x0000000000000001 is an unknown value ??? RDX=0x00007f55af000000 points into unknown readable memory: 01 00 00 00 01 00 00 04 ??? RSI=0x007fffc05491d000 is an unknown value As you can see, RSI is very large, which means you have an invalid oop in the stack that's probably very large. ??? [libjvm.so+0xcc0121] PCMarkAndPushClosure::do_oop(oopDesc**)+0x51 ??? [libjvm.so+0xc58c8b]? OopMapSet::oops_do() ??? [libjvm.so+0x7521e9]? frame::oops_do_internal()+0x99 <<<< HERE ??? [libjvm.so+0xf55757]? JavaThread::oops_do()+0x187 As others have mentioned, this kind of error is usually caused by invalid use of Unsafe or JNI that leads to heap corruption. However, it's plausible that somehow the VM has messed up the frame and tries to mark an invalid oop. Thanks - Ioi On 3/3/20 8:02 AM, Sundara Mohan M wrote: > Hi, > I am seeing JVM crashes on our system in GC Thread with parallel gc on > x86 linux. Observed the same crash happening on JVM-11.0.6/13.0.2/13.0.1 GA > builds. > Adding some logs lines to give some context. > > # > # A fatal error has been detected by the Java Runtime Environment: > # > # SIGSEGV (0xb) at pc=0x00007f669c964311, pid=66684, tid=71106 > # > # JRE version: OpenJDK Runtime Environment (13.0.1+9) (build 13.0.1+9) > # Java VM: OpenJDK 64-Bit Server VM (13.0.1+9, mixed mode, tiered, parallel > gc, linux-amd64) > # Problematic frame: > # V [libjvm.so+0xcd3311] PCMarkAndPushClosure::do_oop(oopDesc**)+0x51 > # > # No core dump will be written. Core dumps have been disabled. To enable > core dumping, try "ulimit -c unlimited" before starting Java again > # > # If you would like to submit a bug report, please visit: > # https://github.com/AdoptOpenJDK/openjdk-build/issues > # > > Host: Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz, 48 cores, 125G, Red Hat > Enterprise Linux Server release 6.10 (Santiago) > Time: Thu Feb 6 11:43:48 2020 UTC elapsed time: 198626 seconds (2d 7h 10m > 26s) > > > Following is the stack trace > ex1: > Stack: [0x00007fd01cbdb000,0x00007fd01ccdb000], sp=0x00007fd01ccd8890, > free space=1014k > Native frames: (J=compiled Java code, A=aot compiled Java code, > j=interpreted, Vv=VM code, C=native code) > *V [libjvm.so+0xcc0121] PCMarkAndPushClosure::do_oop(oopDesc**)+0x51* > V [libjvm.so+0xc58c8b] OopMapSet::oops_do(frame const*, RegisterMap > const*, OopClosure*)+0x2eb > V [libjvm.so+0x7521e9] frame::oops_do_internal(OopClosure*, > CodeBlobClosure*, RegisterMap*, bool)+0x99 > V [libjvm.so+0xf55757] JavaThread::oops_do(OopClosure*, > CodeBlobClosure*)+0x187 > V [libjvm.so+0xcbb100] ThreadRootsMarkingTask::do_it(GCTaskManager*, > unsigned int)+0xb0 > V [libjvm.so+0x7e0f8b] GCTaskThread::run()+0x1eb > V [libjvm.so+0xf5d43d] Thread::call_run()+0x10d > V [libjvm.so+0xc74337] thread_native_entry(Thread*)+0xe7 > > JavaThread 0x00007fbeb9209800 (nid = 82380) was being processed > Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) > v ~RuntimeStub::_new_array_Java > J 62465 c2 > ch.qos.logback.classic.spi.ThrowableProxy.(Ljava/lang/Throwable;)V > (207 bytes) @ 0x00007fd00ad43704 [0x00007fd00ad41420+0x00000000000022e4] > J 474206 c2 > org.eclipse.jetty.util.log.JettyAwareLogger.log(Lorg/slf4j/Marker;ILjava/lang/String;[Ljava/lang/Object;Ljava/lang/Throwable;)V > (134 bytes) @ 0x00007fd00c4e81ec [0x00007fd00c4e7ee0+0x000000000000030c] > j > org.eclipse.jetty.util.log.JettyAwareLogger.warn(Ljava/lang/String;Ljava/lang/Throwable;)V+7 > j > org.eclipse.jetty.util.log.Slf4jLog.warn(Ljava/lang/String;Ljava/lang/Throwable;)V+6 > j > org.eclipse.jetty.server.HttpChannel.handleException(Ljava/lang/Throwable;)V+181 > j > org.eclipse.jetty.server.HttpChannelOverHttp.handleException(Ljava/lang/Throwable;)V+13 > J 64106 c2 org.eclipse.jetty.server.HttpChannel.handle()Z (997 bytes) @ > 0x00007fd00c6d2cd4 [0x00007fd00c6cdec0+0x0000000000004e14] > J 280430 c2 org.eclipse.jetty.server.HttpConnection.onFillable()V (334 > bytes) @ 0x00007fd00da925f0 [0x00007fd00da91e40+0x00000000000007b0] > J 41979 c2 org.eclipse.jetty.io.ChannelEndPoint$2.run()V (12 bytes) @ > 0x00007fd00a14f604 [0x00007fd00a14f4e0+0x0000000000000124] > J 86362 c2 org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run()V > (565 bytes) @ 0x00007fd0087d7e34 [0x00007fd0087d7cc0+0x0000000000000174] > J 75998 c2 java.lang.Thread.run()V java.base at 13.0.2 (17 bytes) @ > 0x00007fd00c93b8d8 [0x00007fd00c93b8a0+0x0000000000000038] > v ~StubRoutines::call_stub > > ex2: > Stack: [0x00007f669869f000,0x00007f669879f000], sp=0x00007f669879c890, > free space=1014k > Native frames: (J=compiled Java code, A=aot compiled Java code, > j=interpreted, Vv=VM code, C=native code) > > *V [libjvm.so+0xcd3311] PCMarkAndPushClosure::do_oop(oopDesc**)+0x51*V > [libjvm.so+0xc6bf0b] OopMapSet::oops_do(frame const*, RegisterMap const*, > OopClosure*)+0x2eb > V [libjvm.so+0x765489] frame::oops_do_internal(OopClosure*, > CodeBlobClosure*, RegisterMap*, bool)+0x99 > V [libjvm.so+0xf68b17] JavaThread::oops_do(OopClosure*, > CodeBlobClosure*)+0x187 > V [libjvm.so+0xcce2f0] ThreadRootsMarkingTask::do_it(GCTaskManager*, > unsigned int)+0xb0 > V [libjvm.so+0x7f422b] GCTaskThread::run()+0x1eb > V [libjvm.so+0xf707fd] Thread::call_run()+0x10d > V [libjvm.so+0xc875b7] thread_native_entry(Thread*)+0xe7 > > JavaThread 0x00007f5518004000 (nid = 75659) was being processed > Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) > v ~RuntimeStub::_new_array_Java > J 54174 c2 > ch.qos.logback.classic.spi.ThrowableProxy.(Ljava/lang/Throwable;)V > (207 bytes) @ 0x00007f6687d92678 [0x00007f6687d8c700+0x0000000000005f78] > J 334031 c2 > com.xmas.webservice.exception.ExceptionLoggingWrapper.execute()V (1004 > bytes) @ 0x00007f6686ede430 [0x00007f6686edd580+0x0000000000000eb0] > J 53431 c2 > com.xmas.webservice.exception.mapper.AbstractExceptionMapper.toResponse(Lcom/xmas/beans/exceptions/mapper/V3ErrorCode;Ljava/lang/Exception;)Ljavax/ws/rs/core/Response; > (105 bytes) @ 0x00007f6687db88b0 [0x00007f6687db8660+0x0000000000000250] > J 63819 c2 > com.xmas.webservice.exception.mapper.RequestBlockedExceptionMapper.toResponse(Ljava/lang/Exception;)Ljavax/ws/rs/core/Response; > (9 bytes) @ 0x00007f6686a6ed9c [0x00007f6686a6ecc0+0x00000000000000dc] > J 334032 c2 > com.xmas.webservice.filters.ResponseSerializationWorker.processException()Ljava/io/InputStream; > (332 bytes) @ 0x00007f668992ad34 [0x00007f668992a840+0x00000000000004f4] > J 403918 c2 > com.xmas.webservice.filters.ResponseSerializationWorker.execute()Z (272 > bytes) @ 0x00007f66869d67fc [0x00007f66869d5e80+0x000000000000097c] > J 17530 c2 com.lafaspot.common.concurrent.internal.WorkerWrapper.execute()Z > (208 bytes) @ 0x00007f66848b3708 [0x00007f66848b36a0+0x0000000000000068] > J 31970% c2 > com.lafaspot.common.concurrent.internal.WorkerManagerOneThread.call()Lcom/lafaspot/common/concurrent/internal/WorkerManagerState; > (486 bytes) @ 0x00007f668608dcb0 [0x00007f668608d5e0+0x00000000000006d0] > j > com.lafaspot.common.concurrent.internal.WorkerManagerOneThread.call()Ljava/lang/Object;+1 > J 4889 c1 java.util.concurrent.FutureTask.run()V java.base at 13.0.1 (123 > bytes) @ 0x00007f667d0be604 [0x00007f667d0bdf80+0x0000000000000684] > J 7487 c1 > java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V > java.base at 13.0.1 (187 bytes) @ 0x00007f667dd45854 > [0x00007f667dd44a60+0x0000000000000df4] > J 7486 c1 java.util.concurrent.ThreadPoolExecutor$Worker.run()V > java.base at 13.0.1 (9 bytes) @ 0x00007f667d1f643c > [0x00007f667d1f63c0+0x000000000000007c] > J 7078 c1 java.lang.Thread.run()V java.base at 13.0.1 (17 bytes) @ > 0x00007f667d1f2d74 [0x00007f667d1f2c40+0x0000000000000134] > v ~StubRoutines::call_stub > > Not very frequent but ~90 days ~120 crashes with following signal > siginfo: si_signo: 11 (SIGSEGV), si_code: 128 (SI_KERNEL), si_addr: > 0x0000000000000000 > This signal is generated when we try to access non canonical address in > linux. > > As suggested by Stefan in another thread i tried to > add VerifyAfterGC/VerifyBeforeGC but it seems to increase the latency and > applications not surviving our production traffic(timing out and requests > are failing). > > Questions > 1. When i looked at source code for printing stack trace i see following > https://github.com/openjdk/jdk11u/blob/master/src/hotspot/share/utilities/vmError.cpp#L696 > (Prints native stack trace) > https://github.com/openjdk/jdk11u/blob/master/src/hotspot/share/utilities/vmError.cpp#L718 > (printing Java thread stack trace if it is involved in GC crash) > a. How do you know this java thread was involved in jvm crash? > b. Can i assume the java thread printed after native stack trace was the > culprit? > c. Since i am seeing the same frame (~RuntimeStub::_new_array_Java, J > 54174 c2 ch.qos.logback.classic.spi.ThrowableProxy...) but different > stack trace in both crashes can this be the root cause? > > 2. Thinking of excluding compilation > of ch.qos.logback.classic.spi.ThrowableProxy class and running in > production to see if compilation of this method is the cause. Does it make > sense? > > 3. Any other suggestion on debugging this further? > > TIA > Sundar From richard.reingruber at sap.com Tue Mar 3 20:22:46 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Tue, 3 Mar 2020 20:22:46 +0000 Subject: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents In-Reply-To: References: <1f8a3c7a-fa0f-b5b2-4a8a-7d3d8dbbe1b5@oracle.com> <4b56a45c-a14c-6f74-2bfd-25deaabe8201@oracle.com> <5271429a-481d-ddb9-99dc-b3f6670fcc0b@oracle.com> Message-ID: Hi Robbin, > > I understand that Robbin proposed to replace the usage of > > _suspend_flag with handshakes. Apparently, async handshakes > > are needed to do so. We have been waiting a while for removal > > of the _suspend_flag / introduction of async handshakes [2]. > > What is the status here? > I have an old prototype which I would like to continue to work on. > So do not assume asynch handshakes will make 15. > Even if it would, I think there are a lot more investigate work to remove > _suspend_flag. Let us know, if we can be of any help to you and be it only testing. > >> Full: http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.4/ > DeoptimizeObjectsALotThread is only used in compileBroker.cpp. > You can move both declaration and definition to that file, no need to clobber > thread.[c|h]pp. (and the static function deopt_objs_alot_thread_entry) Will do. > Does JvmtiDeferredUpdates really need to be in thread.hpp, can't be in it's own > hpp file? It doesn't seem right to add JVM TI classes into thread.hpp. You are right. It shouldn't be declared in thread.hpp. I will look into that. > Note that we also think we may have a bug in deopt: > https://bugs.openjdk.java.net/browse/JDK-8238237 > I think it would be best, if possible, to push after that is resolved. Sure. > Not even nearly a full review :) I know :) Anyways, thanks a lot, Richard. -----Original Message----- From: Robbin Ehn Sent: Monday, March 2, 2020 11:17 AM To: Lindenmaier, Goetz ; Reingruber, Richard ; David Holmes ; Vladimir Kozlov (vladimir.kozlov at oracle.com) ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents Hi, On 2/24/20 5:39 PM, Lindenmaier, Goetz wrote: > Hi, > > I had a look at the progress of this change. Nothing > happened since Richard posted his update using more > handshakes [1]. > But we (SAP) would appreciate a lot if this change could > be successfully reviewed and pushed. > > I think there is basic understanding that this > change is helpful. It fixes a number of issues with JVMTI, > and will deliver the same performance benefits as EA > does in current production mode for debugging scenarios. > > This is important for us as we run our VMs prepared > for debugging in production mode. > > I understand that Robbin proposed to replace the usage of > _suspend_flag with handshakes. Apparently, async handshakes > are needed to do so. We have been waiting a while for removal > of the _suspend_flag / introduction of async handshakes [2]. > What is the status here? I have an old prototype which I would like to continue to work on. So do not assume asynch handshakes will make 15. Even if it would, I think there are a lot more investigate work to remove _suspend_flag. > > I think we should no longer wait, but proceed with > this change. We will look into removing the usage of > suspend_flag introduced here once it is possible to implement > it with handshakes. Yes, sure. >> Full: http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.4/ DeoptimizeObjectsALotThread is only used in compileBroker.cpp. You can move both declaration and definition to that file, no need to clobber thread.[c|h]pp. (and the static function deopt_objs_alot_thread_entry) Does JvmtiDeferredUpdates really need to be in thread.hpp, can't be in it's own hpp file? It doesn't seem right to add JVM TI classes into thread.hpp. Note that we also think we may have a bug in deopt: https://bugs.openjdk.java.net/browse/JDK-8238237 I think it would be best, if possible, to push after that is resolved. Not even nearly a full review :) Thanks, Robbin >> Incremental: >> http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.4.inc/ >> >> I was not able to eliminate the additional suspend flag now. I'll take care of this >> as soon as the >> existing suspend-resume-mechanism is reworked. >> >> Testing: >> >> Nightly tests @SAP: >> >> JCK and JTREG, also in Xcomp mode, SPECjvm2008, SPECjbb2015, Renaissance >> Suite, SAP specific tests >> with fastdebug and release builds on all platforms >> >> Stress testing with DeoptimizeObjectsALot running SPECjvm2008 40x parallel >> for 24h >> >> Thanks, Richard. >> >> >> More details on the changes: >> >> * Hide DeoptimizeObjectsALotThread from external view. >> >> * Changed EscapeBarrier_lock to be a _safepoint_check_never lock. >> It used to be _safepoint_check_sometimes, which will be eliminated sooner or >> later. >> I added explicit thread state changes with ThreadBlockInVM to code paths >> where we can wait() >> on EscapeBarrier_lock to become safepoint safe. >> >> * Use handshake EscapeBarrierSuspendHandshake to suspend target threads >> instead of vm operation >> VM_ThreadSuspendAllForObjDeopt. >> >> * Removed uses of Threads_lock. When adding a new thread we suspend it iff >> EA optimizations are >> being reverted. In the previous version we were waiting on Threads_lock >> while EA optimizations >> were reverted. See EscapeBarrier::thread_added(). >> >> * Made tests require Xmixed compilation mode. >> >> * Made tests agnostic regarding tiered compilation. >> I.e. tc isn't disabled anymore, and the tests can be run with tc enabled or >> disabled. >> >> * Exercising EATests.java as well with stress test options >> DeoptimizeObjectsALot* >> Due to the non-deterministic deoptimizations some tests need to be skipped. >> We do this to prevent bit-rot of the stress test code. >> >> * Executing EATests.java as well with graal if available. Driver for this is >> EATestsJVMCI.java. Graal cannot pass all tests, because it does not provide all >> the new debug info >> (namely not_global_escape_in_scope and arg_escape in scopeDesc.hpp). >> And graal does not yet support the JVMTI operations force early return and >> pop frame. >> >> * Removed tracing from new jdi tests in EATests.java. Too much trace output >> before the debugging >> connection is established can cause deadlock because output buffers fill up. >> (See https://bugs.openjdk.java.net/browse/JDK-8173304) >> >> * Many copyright year changes and smaller clean-up changes of testing code >> (trailing white-space and >> the like). >> >> >> -----Original Message----- >> From: David Holmes >> Sent: Donnerstag, 19. Dezember 2019 03:12 >> To: Reingruber, Richard ; serviceability- >> dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot- >> runtime-dev at openjdk.java.net; Vladimir Kozlov (vladimir.kozlov at oracle.com) >> >> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better Performance in >> the Presence of JVMTI Agents >> >> Hi Richard, >> >> I think my issue is with the way EliminateNestedLocks works so I'm going >> to look into that more deeply. >> >> Thanks for the explanations. >> >> David >> >> On 18/12/2019 12:47 am, Reingruber, Richard wrote: >>> Hi David, >>> >>> > > > Some further queries/concerns: >>> > > > >>> > > > src/hotspot/share/runtime/objectMonitor.cpp >>> > > > >>> > > > Can you please explain the changes to ObjectMonitor::wait: >>> > > > >>> > > > ! _recursions = save // restore the old recursion count >>> > > > ! + jt->get_and_reset_relock_count_after_wait(); // >>> > > > increased by the deferred relock count >>> > > > >>> > > > what is the "deferred relock count"? I gather it relates to >>> > > > >>> > > > "The code was extended to be able to deoptimize objects of a >>> > > frame that >>> > > > is not the top frame and to let another thread than the owning >>> > > thread do >>> > > > it." >>> > > >>> > > Yes, these relate. Currently EA based optimizations are reverted, when a >> compiled frame is >>> > > replaced with corresponding interpreter frames. Part of this is relocking >> objects with eliminated >>> > > locking. New with the enhancement is that we do this also just before >> object references are >>> > > acquired through JVMTI. In this case we deoptimize also the owning >> compiled frame C and we >>> > > register deoptimized objects as deferred updates. When control returns >> to C it gets deoptimized, >>> > > we notice that objects are already deoptimized (reallocated and >> relocked), so we don't do it again >>> > > (relocking twice would be incorrect of course). Deferred updates are >> copied into the new >>> > > interpreter frames. >>> > > >>> > > Problem: relocking is not possible if the target thread T is waiting on the >> monitor that needs to >>> > > be relocked. This happens only with non-local objects with >> EliminateNestedLocks. Instead relocking >>> > > is deferred until T owns the monitor again. This is what the piece of >> code above does. >>> > >>> > Sorry I need some more detail here. How can you wait() on an object >>> > monitor if the object allocation and/or locking was optimised away? And >>> > what is a "non-local object" in this context? Isn't EA restricted to >>> > thread-confined objects? >>> >>> "Non-local object" is an object that escapes its thread. The issue I'm >> addressing with the changes >>> in ObjectMonitor::wait are almost unrelated to EA. They are caused by >> EliminateNestedLocks, where C2 >>> eliminates recursive locking of an already owned lock. The lock owning object >> exists on the heap, it >>> is locked and you can call wait() on it. >>> >>> EliminateLocks is the C2 option that controls lock elimination based on EA. >> Both optimizations have >>> in common that objects with eliminated locking need to be relocked when >> deoptimizing a frame, >>> i.e. when replacing a compiled frame with equivalent interpreter >>> frames. Deoptimization::relock_objects does that job for /all/ eliminated >> locks in scope. /All/ can >>> be a mix of eliminated nested locks and locks of not-escaping objects. >>> >>> New with the enhancement: I call relock_objects earlier, just before objects >> pontentially >>> escape. But then later when the owning compiled frame gets deoptimized, I >> must not do it again: >>> >>> See call to EscapeBarrier::objs_are_deoptimized in deoptimization.cpp: >>> >>> 373 if ((jvmci_enabled || ((DoEscapeAnalysis || EliminateNestedLocks) && >> EliminateLocks)) >>> 374 && !EscapeBarrier::objs_are_deoptimized(thread, deoptee.id())) { >>> 375 bool unused; >>> 376 eliminate_locks(thread, chunk, realloc_failures, deoptee, exec_mode, >> unused); >>> 377 } >>> >>> Now when calling relock_objects early it is quiet possible that I have to relock >> an object the >>> target thread currently waits for. Obviously I cannot relock in this case, >> instead I chose to >>> introduce relock_count_after_wait to JavaThread. >>> >>> > Is it just that some of the locking gets optimized away e.g. >>> > >>> > synchronised(obj) { >>> > synchronised(obj) { >>> > synchronised(obj) { >>> > obj.wait(); >>> > } >>> > } >>> > } >>> > >>> > If this is reduced to a form as-if it were a single lock of the monitor >>> > (due to EA) and the wait() triggers a JVM TI event which leads to the >>> > escape of "obj" then we need to reconstruct the true lock state, and so >>> > when the wait() internally unblocks and reacquires the monitor it has to >>> > set the true recursion count to 3, not the 1 that it appeared to be when >>> > wait() was initially called. Is that the scenario? >>> >>> Kind of... except that the locking is not eliminated due to EA and there is no >> JVM TI event >>> triggered by wait. >>> >>> Add >>> >>> LocalObject l1 = new LocalObject(); >>> >>> in front of the synchrnized blocks and assume a JVM TI agent acquires l1. This >> triggers the code in >>> question. >>> >>> See that relocking/reallocating is transactional. If it is done then for /all/ >> objects in scope and it is >>> done at most once. It wouldn't be quite so easy to split this in relocking of >> nested/EA-based >>> eliminated locks. >>> >>> > If so I find this truly awful. Anyone using wait() in a realistic form >>> > requires a notification and so the object cannot be thread confined. In >>> >>> It is not thread confined. >>> >>> > which case I would strongly argue that upon hitting the wait() the deopt >>> > should occur unconditionally and so the lock state is correct before we >>> > wait and so we don't need to mess with the recursion count internally >>> > when we reacquire the monitor. >>> > >>> > > >>> > > > which I don't like the sound of at all when it comes to ObjectMonitor >>> > > > state. So I'd like to understand in detail exactly what is going on here >>> > > > and why. This is a very intrusive change that seems to badly break >>> > > > encapsulation and impacts future changes to ObjectMonitor that are >> under >>> > > > investigation. >>> > > >>> > > I would not regard this as breaking encapsulation. Certainly not badly. >>> > > >>> > > I've added a property relock_count_after_wait to JavaThread. The >> property is well >>> > > encapsulated. Future ObjectMonitor implementations have to deal with >> recursion too. They are free >>> > > in choosing a way to do that as long as that property is taken into >> account. This is hardly a >>> > > limitation. >>> > >>> > I do think this badly breaks encapsulation as you have to add a callout >>> > from the guts of the ObjectMonitor code to reach into the thread to get >>> > this lock count adjustment. I understand why you have had to do this but >>> > I would much rather see a change to the EA optimisation strategy so that >>> > this is not needed. >>> > >>> > > Note also that the property is a straight forward extension of the >> existing concept of deferred >>> > > local updates. It is embedded into the structure holding them. So not >> even the footprint of a >>> > > JavaThread is enlarged if no deferred updates are generated. >>> > >>> > [...] >>> > >>> > > >>> > > I'm actually duplicating the existing external suspend mechanism, >> because a thread can be >>> > > suspended at most once. And hey, and don't like that either! But it >> seems not unlikely that the >>> > > duplicate can be removed together with the original and the new type >> of handshakes that will be >>> > > used for thread suspend can be used for object deoptimization too. See >> today's discussion in >>> > > JDK-8227745 [2]. >>> > >>> > I hope that discussion bears some fruit, at the moment it seems not to >>> > be possible to use handshakes here. :( >>> > >>> > The external suspend mechanism is a royal pain in the proverbial that we >>> > have to carefully live with. The idea that we're duplicating that for >>> > use in another fringe area of functionality does not thrill me at all. >>> > >>> > To be clear, I understand the problem that exists and that you wish to >>> > solve, but for the runtime parts I balk at the complexity cost of >>> > solving it. >>> >>> I know it's complex, but by far no rocket science. >>> >>> Also I find it hard to imagine another fix for JDK-8233915 besides changing >> the JVM TI specification. >>> >>> Thanks, Richard. >>> >>> -----Original Message----- >>> From: David Holmes >>> Sent: Dienstag, 17. Dezember 2019 08:03 >>> To: Reingruber, Richard ; serviceability- >> dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot- >> runtime-dev at openjdk.java.net; Vladimir Kozlov (vladimir.kozlov at oracle.com) >> >>> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better Performance >> in the Presence of JVMTI Agents >>> >>> >>> >>> David >>> >>> On 17/12/2019 4:57 pm, David Holmes wrote: >>>> Hi Richard, >>>> >>>> On 14/12/2019 5:01 am, Reingruber, Richard wrote: >>>>> Hi David, >>>>> >>>>> ?? > Some further queries/concerns: >>>>> ?? > >>>>> ?? > src/hotspot/share/runtime/objectMonitor.cpp >>>>> ?? > >>>>> ?? > Can you please explain the changes to ObjectMonitor::wait: >>>>> ?? > >>>>> ?? > !?? _recursions = save????? // restore the old recursion count >>>>> ?? > !???????????????? + jt->get_and_reset_relock_count_after_wait(); // >>>>> ?? > increased by the deferred relock count >>>>> ?? > >>>>> ?? > what is the "deferred relock count"? I gather it relates to >>>>> ?? > >>>>> ?? > "The code was extended to be able to deoptimize objects of a >>>>> frame that >>>>> ?? > is not the top frame and to let another thread than the owning >>>>> thread do >>>>> ?? > it." >>>>> >>>>> Yes, these relate. Currently EA based optimizations are reverted, when >>>>> a compiled frame is replaced >>>>> with corresponding interpreter frames. Part of this is relocking >>>>> objects with eliminated >>>>> locking. New with the enhancement is that we do this also just before >>>>> object references are acquired >>>>> through JVMTI. In this case we deoptimize also the owning compiled >>>>> frame C and we register >>>>> deoptimized objects as deferred updates. When control returns to C it >>>>> gets deoptimized, we notice >>>>> that objects are already deoptimized (reallocated and relocked), so we >>>>> don't do it again (relocking >>>>> twice would be incorrect of course). Deferred updates are copied into >>>>> the new interpreter frames. >>>>> >>>>> Problem: relocking is not possible if the target thread T is waiting >>>>> on the monitor that needs to be >>>>> relocked. This happens only with non-local objects with >>>>> EliminateNestedLocks. Instead relocking is >>>>> deferred until T owns the monitor again. This is what the piece of >>>>> code above does. >>>> >>>> Sorry I need some more detail here. How can you wait() on an object >>>> monitor if the object allocation and/or locking was optimised away? And >>>> what is a "non-local object" in this context? Isn't EA restricted to >>>> thread-confined objects? >>>> >>>> Is it just that some of the locking gets optimized away e.g. >>>> >>>> synchronised(obj) { >>>> ? synchronised(obj) { >>>> ??? synchronised(obj) { >>>> ????? obj.wait(); >>>> ??? } >>>> ? } >>>> } >>>> >>>> If this is reduced to a form as-if it were a single lock of the monitor >>>> (due to EA) and the wait() triggers a JVM TI event which leads to the >>>> escape of "obj" then we need to reconstruct the true lock state, and so >>>> when the wait() internally unblocks and reacquires the monitor it has to >>>> set the true recursion count to 3, not the 1 that it appeared to be when >>>> wait() was initially called. Is that the scenario? >>>> >>>> If so I find this truly awful. Anyone using wait() in a realistic form >>>> requires a notification and so the object cannot be thread confined. In >>>> which case I would strongly argue that upon hitting the wait() the deopt >>>> should occur unconditionally and so the lock state is correct before we >>>> wait and so we don't need to mess with the recursion count internally >>>> when we reacquire the monitor. >>>> >>>>> >>>>> ?? > which I don't like the sound of at all when it comes to >>>>> ObjectMonitor >>>>> ?? > state. So I'd like to understand in detail exactly what is going >>>>> on here >>>>> ?? > and why.? This is a very intrusive change that seems to badly break >>>>> ?? > encapsulation and impacts future changes to ObjectMonitor that >>>>> are under >>>>> ?? > investigation. >>>>> >>>>> I would not regard this as breaking encapsulation. Certainly not badly. >>>>> >>>>> I've added a property relock_count_after_wait to JavaThread. The >>>>> property is well >>>>> encapsulated. Future ObjectMonitor implementations have to deal with >>>>> recursion too. They are free in >>>>> choosing a way to do that as long as that property is taken into >>>>> account. This is hardly a >>>>> limitation. >>>> >>>> I do think this badly breaks encapsulation as you have to add a callout >>>> from the guts of the ObjectMonitor code to reach into the thread to get >>>> this lock count adjustment. I understand why you have had to do this but >>>> I would much rather see a change to the EA optimisation strategy so that >>>> this is not needed. >>>> >>>>> Note also that the property is a straight forward extension of the >>>>> existing concept of deferred >>>>> local updates. It is embedded into the structure holding them. So not >>>>> even the footprint of a >>>>> JavaThread is enlarged if no deferred updates are generated. >>>>> >>>>> ?? > --- >>>>> ?? > >>>>> ?? > src/hotspot/share/runtime/thread.cpp >>>>> ?? > >>>>> ?? > Can you please explain why >>>>> JavaThread::wait_for_object_deoptimization >>>>> ?? > has to be handcrafted in this way rather than using proper >>>>> transitions. >>>>> ?? > >>>>> >>>>> I wrote wait_for_object_deoptimization taking >>>>> JavaThread::java_suspend_self_with_safepoint_check >>>>> as template. So in short: for the same reasons :) >>>>> >>>>> Threads reach both methods as part of thread state transitions, >>>>> therefore special handling is >>>>> required to change thread state on top of ongoing transitions. >>>>> >>>>> ?? > We got rid of "deopt suspend" some time ago and it is disturbing >>>>> to see >>>>> ?? > it being added back (effectively). This seems like it may be >>>>> something >>>>> ?? > that handshakes could be used for. >>>>> >>>>> Deopt suspend used to be something rather different with a similar >>>>> name[1]. It is not being added back. >>>> >>>> I stand corrected. Despite comments in the code to the contrary >>>> deopt_suspend didn't actually cause a self-suspend. I was doing a lot of >>>> cleanup in this area 13 years ago :) >>>> >>>>> >>>>> I'm actually duplicating the existing external suspend mechanism, >>>>> because a thread can be suspended >>>>> at most once. And hey, and don't like that either! But it seems not >>>>> unlikely that the duplicate can >>>>> be removed together with the original and the new type of handshakes >>>>> that will be used for >>>>> thread suspend can be used for object deoptimization too. See today's >>>>> discussion in JDK-8227745 [2]. >>>> >>>> I hope that discussion bears some fruit, at the moment it seems not to >>>> be possible to use handshakes here. :( >>>> >>>> The external suspend mechanism is a royal pain in the proverbial that we >>>> have to carefully live with. The idea that we're duplicating that for >>>> use in another fringe area of functionality does not thrill me at all. >>>> >>>> To be clear, I understand the problem that exists and that you wish to >>>> solve, but for the runtime parts I balk at the complexity cost of >>>> solving it. >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>>> Thanks, Richard. >>>>> >>>>> [1] Deopt suspend was something like an async. handshake for >>>>> architectures with register windows, >>>>> ???? where patching the return pc for deoptimization of a compiled >>>>> frame was racy if the owner thread >>>>> ???? was in native code. Instead a "deopt" suspend flag was set on >>>>> which the thread patched its own >>>>> ???? frame upon return from native. So no thread was suspended. It got >>>>> its name only from the name of >>>>> ???? the flags. >>>>> >>>>> [2] Discussion about using handshakes to sync. with the target thread: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK- >> 8227745?focusedCommentId=14306727&page=com.atlassian.jira.plugin.syste >> m.issuetabpanels:comment-tabpanel#comment-14306727 >>>>> >>>>> >>>>> -----Original Message----- >>>>> From: David Holmes >>>>> Sent: Freitag, 13. Dezember 2019 00:56 >>>>> To: Reingruber, Richard ; >>>>> serviceability-dev at openjdk.java.net; >>>>> hotspot-compiler-dev at openjdk.java.net; >>>>> hotspot-runtime-dev at openjdk.java.net >>>>> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better >>>>> Performance in the Presence of JVMTI Agents >>>>> >>>>> Hi Richard, >>>>> >>>>> Some further queries/concerns: >>>>> >>>>> src/hotspot/share/runtime/objectMonitor.cpp >>>>> >>>>> Can you please explain the changes to ObjectMonitor::wait: >>>>> >>>>> !?? _recursions = save????? // restore the old recursion count >>>>> !???????????????? + jt->get_and_reset_relock_count_after_wait(); // >>>>> increased by the deferred relock count >>>>> >>>>> what is the "deferred relock count"? I gather it relates to >>>>> >>>>> "The code was extended to be able to deoptimize objects of a frame that >>>>> is not the top frame and to let another thread than the owning thread do >>>>> it." >>>>> >>>>> which I don't like the sound of at all when it comes to ObjectMonitor >>>>> state. So I'd like to understand in detail exactly what is going on here >>>>> and why.? This is a very intrusive change that seems to badly break >>>>> encapsulation and impacts future changes to ObjectMonitor that are under >>>>> investigation. >>>>> >>>>> --- >>>>> >>>>> src/hotspot/share/runtime/thread.cpp >>>>> >>>>> Can you please explain why JavaThread::wait_for_object_deoptimization >>>>> has to be handcrafted in this way rather than using proper transitions. >>>>> >>>>> We got rid of "deopt suspend" some time ago and it is disturbing to see >>>>> it being added back (effectively). This seems like it may be something >>>>> that handshakes could be used for. >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>> On 12/12/2019 7:02 am, David Holmes wrote: >>>>>> On 12/12/2019 1:07 am, Reingruber, Richard wrote: >>>>>>> Hi David, >>>>>>> >>>>>>> ??? > Most of the details here are in areas I can comment on in detail, >>>>>>> but I >>>>>>> ??? > did take an initial general look at things. >>>>>>> >>>>>>> Thanks for taking the time! >>>>>> >>>>>> Apologies the above should read: >>>>>> >>>>>> "Most of the details here are in areas I *can't* comment on in detail >>>>>> ..." >>>>>> >>>>>> David >>>>>> >>>>>>> ??? > The only thing that jumped out at me is that I think the >>>>>>> ??? > DeoptimizeObjectsALotThread should be a hidden thread. >>>>>>> ??? > >>>>>>> ??? > +? bool is_hidden_from_external_view() const { return true; } >>>>>>> >>>>>>> Yes, it should. Will add the method like above. >>>>>>> >>>>>>> ??? > Also I don't see any testing of the DeoptimizeObjectsALotThread. >>>>>>> Without >>>>>>> ??? > active testing this will just bit-rot. >>>>>>> >>>>>>> DeoptimizeObjectsALot is meant for stress testing with a larger >>>>>>> workload. I will add a minimal test >>>>>>> to keep it fresh. >>>>>>> >>>>>>> ??? > Also on the tests I don't understand your @requires clause: >>>>>>> ??? > >>>>>>> ??? >?? @requires ((vm.compMode != "Xcomp") & vm.compiler2.enabled >> & >>>>>>> ??? > (vm.opt.TieredCompilation != true)) >>>>>>> ??? > >>>>>>> ??? > This seems to require that TieredCompilation is disabled, but >>>>>>> tiered is >>>>>>> ??? > our normal mode of operation. ?? >>>>>>> ??? > >>>>>>> >>>>>>> I removed the clause. I guess I wanted to target the tests towards the >>>>>>> code they are supposed to >>>>>>> test, and it's easier to analyze failures w/o tiered compilation and >>>>>>> with just one compiler thread. >>>>>>> >>>>>>> Additionally I will make use of >>>>>>> compiler.whitebox.CompilerWhiteBoxTest.THRESHOLD in the tests. >>>>>>> >>>>>>> Thanks, >>>>>>> Richard. >>>>>>> >>>>>>> -----Original Message----- >>>>>>> From: David Holmes >>>>>>> Sent: Mittwoch, 11. Dezember 2019 08:03 >>>>>>> To: Reingruber, Richard ; >>>>>>> serviceability-dev at openjdk.java.net; >>>>>>> hotspot-compiler-dev at openjdk.java.net; >>>>>>> hotspot-runtime-dev at openjdk.java.net >>>>>>> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better >>>>>>> Performance in the Presence of JVMTI Agents >>>>>>> >>>>>>> Hi Richard, >>>>>>> >>>>>>> On 11/12/2019 7:45 am, Reingruber, Richard wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> I would like to get reviews please for >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.3/ >>>>>>>> >>>>>>>> Corresponding RFE: >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8227745 >>>>>>>> >>>>>>>> Fixes also https://bugs.openjdk.java.net/browse/JDK-8233915 >>>>>>>> And potentially https://bugs.openjdk.java.net/browse/JDK-8214584 [1] >>>>>>>> >>>>>>>> Vladimir Kozlov kindly put webrev.3 through tier1-8 testing without >>>>>>>> issues (thanks!). In addition the >>>>>>>> change is being tested at SAP since I posted the first RFR some >>>>>>>> months ago. >>>>>>>> >>>>>>>> The intention of this enhancement is to benefit performance wise from >>>>>>>> escape analysis even if JVMTI >>>>>>>> agents request capabilities that allow them to access local variable >>>>>>>> values. E.g. if you start-up >>>>>>>> with -agentlib:jdwp=transport=dt_socket,server=y,suspend=n, then >>>>>>>> escape analysis is disabled right >>>>>>>> from the beginning, well before a debugger attaches -- if ever one >>>>>>>> should do so. With the >>>>>>>> enhancement, escape analysis will remain enabled until and after a >>>>>>>> debugger attaches. EA based >>>>>>>> optimizations are reverted just before an agent acquires the >>>>>>>> reference to an object. In the JBS item >>>>>>>> you'll find more details. >>>>>>> >>>>>>> Most of the details here are in areas I can comment on in detail, but I >>>>>>> did take an initial general look at things. >>>>>>> >>>>>>> The only thing that jumped out at me is that I think the >>>>>>> DeoptimizeObjectsALotThread should be a hidden thread. >>>>>>> >>>>>>> +? bool is_hidden_from_external_view() const { return true; } >>>>>>> >>>>>>> Also I don't see any testing of the DeoptimizeObjectsALotThread. >>>>>>> Without >>>>>>> active testing this will just bit-rot. >>>>>>> >>>>>>> Also on the tests I don't understand your @requires clause: >>>>>>> >>>>>>> ??? @requires ((vm.compMode != "Xcomp") & vm.compiler2.enabled & >>>>>>> (vm.opt.TieredCompilation != true)) >>>>>>> >>>>>>> This seems to require that TieredCompilation is disabled, but tiered is >>>>>>> our normal mode of operation. ?? >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> >>>>>>>> Thanks, >>>>>>>> Richard. >>>>>>>> >>>>>>>> [1] Experimental fix for JDK-8214584 based on JDK-8227745 >>>>>>>> >> http://cr.openjdk.java.net/~rrich/webrevs/2019/8214584/experiment_v1.patc >> h >>>>>>>> >>>>>>>> >>>>>>>> From kevin.walls at oracle.com Tue Mar 3 22:44:14 2020 From: kevin.walls at oracle.com (Kevin Walls) Date: Tue, 3 Mar 2020 22:44:14 +0000 Subject: RFR(S): 8240295: hs_err elapsed time in seconds is not accurate enough In-Reply-To: <92ada19d-7200-6bb7-b174-9da8a166aea7@oracle.com> References: <4a3d7c8e-df9e-51de-77e4-775be1480392@oracle.com> <92ada19d-7200-6bb7-b174-9da8a166aea7@oracle.com> Message-ID: <053864a8-617f-fcac-fc26-220d161e3e55@oracle.com> Thanks David - Yes there are situations where hs_err fails, and few people are sadder than me when that happens 8-) , so I was thinking about how scared to be by the comment. With the safety net of the error handler for the steps of the hs_err file (which works well, we see it invoked frequently), it looks reasonable to use %f as we might do other slightly questionable things for a signal handler. Corrupting locale information or floating point state might possibly cause problems, but if I cause a fake crash in print_date_and_time the error handler recovers and the report continues. Thinking about printing with two ints, seconds and fractions: I don't see anything already that returns such a time in two components in the JVM, so we might implement a new form of os::javaTimeNanos() or similar that returns the two parts, and do that on each platform. I didn't yet come up with anything to do in os::print_date_and_time() which will take the fractional part of the double, and print just the fraction as an int, without using any library / %f facilities. If you're still concerned I could revisit these or some other idea. Genuine laugh out loud moment for me, I backported the elapsed time logging from 6u4 to 5u19? (https://bugs.openjdk.java.net/browse/JDK-6447157) (2007). (I said before jdk5 was created, I should have said before it was in mercurial.) Thanks Kevin On 03/03/2020 01:11, David Holmes wrote: > Hi Kevin, > > On 2/03/2020 8:48 pm, Kevin Walls wrote: >> Oops, and with the bug ID in the title and JBS link: >> https://bugs.openjdk.java.net/browse/JDK-8240295 >> >> >> On 02/03/2020 10:47, Kevin Walls wrote: >>> Hi, >>> >>> (s11y and runtime opinions both relevant) >>> >>> A few times in the last month I've really wanted to compare the >>> Events logged in the hs_err file, and the time of the JVM's crash. >>> >>> "elapsed time" in hs_err is only accurate to one second, and has >>> been since before jdk5 was created. >>> >>> The diff below changes the format string and uses the non-rounded >>> time value (I don't see a need to change the other integer >>> arithmetic here), and we can enjoy hs_errs with detail like: >>> >>> ... >>> Time: Mon Mar? 2 09:57:13 2020 UTC elapsed time: 5.490135 seconds >>> (0d 0h 0m 5s) >>> ... >>> >>> Thanks >>> Kevin >>> >>> >>> /jdk/open$ hg diff >>> diff --git a/src/hotspot/share/runtime/os.cpp >>> b/src/hotspot/share/runtime/os.cpp >>> --- a/src/hotspot/share/runtime/os.cpp >>> +++ b/src/hotspot/share/runtime/os.cpp >>> @@ -1016,9 +1016,8 @@ >>> ?? } >>> >>> ?? double t = os::elapsedTime(); >>> -? // NOTE: It tends to crash after a SEGV if we want to >>> printf("%f",...) in >>> -? //?????? Linux. Must be a bug in glibc ? Workaround is to round >>> "t" to int >>> -? //?????? before printf. We lost some precision, but who cares? >>> +? // NOTE: a crash using printf("%f",...) on Linux was historically >>> noted here >>> +? //?????? (before the jdk5 repo was created). > > Just because it is old doesn't mean it no longer applies. printf is > not async-signal-safe - we know that but we try to use it anyway. > Maybe %f is even less async-signal-safe? > > This may get through testing okay but cause problems with real crashes > in the field. > > What about breaking the time up into two ints: seconds and nanos? > > Cheers, > David > ----- > >>> ?? int eltime = (int)t;? // elapsed time in seconds >>> >>> ?? // print elapsed time in a human-readable format: >>> @@ -1029,7 +1028,7 @@ >>> ?? int elmins = (eltime - day_secs - hour_secs) / secs_per_min; >>> ?? int minute_secs = elmins * secs_per_min; >>> ?? int elsecs = (eltime - day_secs - hour_secs - minute_secs); >>> -? st->print_cr(" elapsed time: %d seconds (%dd %dh %dm %ds)", >>> eltime, eldays, elhours, elmins, elsecs); >>> +? st->print_cr(" elapsed time: %f seconds (%dd %dh %dm %ds)", t, >>> eldays, elhours, elmins, elsecs); >>> ?} >>> >>> From suenaga at oss.nttdata.com Wed Mar 4 00:19:08 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Wed, 4 Mar 2020 09:19:08 +0900 Subject: RFR: 8240197: Cannot start JVM when $JAVA_HOME includes CJK characters In-Reply-To: References: <6ec2f649-8999-fdd5-c2df-465925c3663b@oss.nttdata.com> <2d5148fc-7d71-d94c-d4a3-e6424a38b091@oss.nttdata.com> Message-ID: <37e2f672-3407-5b16-ab71-aa1e70e928db@oss.nttdata.com> Thanks Ralf! I fixed them. I need a Reviewer to push, so I uploaded new webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.03/ Yasumasa On 2020/03/03 23:25, Schmelter, Ralf wrote: > Hi Yasumasa, > > thanks for your work. The code should now be able to handle any path. > > Two small changes (no new webrev needed): > > size_t result_len = prefix_len + prefix_off + additional_space > should be > size_t result_len = prefix_len - prefix_off + additional_space > > And you should copy the prefix after you copied the filename, since sometimes the prefix is required to overwrite the start of the filename: > > // This call would be success because it is checked in above > GetFullPathNameW(unicode_path, full_path_len, result + prefix_len - prefix_off, NULL); > // Copy prefix > memcpy(result, prefix, sizeof(WCHAR) * prefix_len); > > and > > // Copy unicode path > memcpy(result + prefix_len - prefix_off, unicode_path, sizeof(WCHAR) * unicode_path_len); > // Copy prefix > memcpy(result, prefix, sizeof(WCHAR) * prefix_len); > > With these changes the os_windows gtest and the appcds jtreg tests run without errors. > > Best regards, > Ralf > From m.sundar85 at gmail.com Wed Mar 4 01:01:28 2020 From: m.sundar85 at gmail.com (Sundara Mohan M) Date: Tue, 3 Mar 2020 20:01:28 -0500 Subject: Need help on debugging JVM crash In-Reply-To: <86c8b74f-9f11-b9b5-00f2-30360d0e8b6f@oracle.com> References: <86c8b74f-9f11-b9b5-00f2-30360d0e8b6f@oracle.com> Message-ID: Hi Ioi, Thanks for the analysis. On Tue, Mar 3, 2020 at 3:12 PM Ioi Lam wrote: > The crash happened while the GC is running. I tried disasm of the > crashing address PCMarkAndPushClosure::do_oop(oopDesc**)+0x51 (from the > 13.0.1+9 GA binaries of AdoptJDK) > > (gdb) x/80i _ZN20PCMarkAndPushClosure6do_oopEPP7oopDesc > <>: push %rbp > <+1>: mov %rsp,%rbp > <+4>: push %r13 > <+6>: push %r12 > <+8>: push %rbx > <+9>: sub $0x8,%rsp > <+13>: mov (%rsi),%rbx ;;; rbx = oop > <+16>: test %rbx,%rbx ;;; oop != null? > <+19>: je 0x7ffff67ca317 > <_ZN20PCMarkAndPushClosure6do_oopEPP7oopDesc+87> > <+21>: lea 0x9c0afc(%rip),%rax # 0x7ffff718add8 > <_ZN20ParCompactionManager12_mark_bitmapE> > <+28>: mov %rbx,%rcx ;;; rcx = oop > <+31>: mov (%rax),%rdx ;;; rdx = > ParCompactionManager::_mark_bitmap > <+34>: sub (%rdx),%rcx ;;; rcx = oop - > _mark_bitmap->_region_start > <+37>: mov 0x10(%rdx),%rdx ;; rdx = _mark_bitmap->_beg_bits->_map > <+41>: mov %rcx,%rax ;;; rax = oop - > _mark_bitmap->_region_start > <+44>: lea 0x93b935(%rip),%rcx # 0x7ffff7105c28 > > <+51>: shr $0x3,%rax > <+55>: mov (%rcx),%ecx > <+57>: shr %cl,%rax > <+60>: mov %rax,%rcx > <+63>: mov %rax,%rsi ;;; rsi = index of oop inside > mark_bitmap > <+66>: mov $0x1,%eax > <+71>: and $0x3f,%ecx > <+74>: shr $0x6,%rsi > <+78>: shl %cl,%rax > <+81>: test %rax,(%rdx,%rsi,8) << crash > > > This looks like that the oop that we try to mark is actually outside of > the heap range, so trying to mark it in the mark_bitmap causes this: > > > siginfo: si_signo: 11 (SIGSEGV), si_code: 128 (SI_KERNEL), si_addr: > 0x0000000000000000 > > > Here are the values of the registers for the "test" instruction above: > > > RAX=0x0000000000000001 is an unknown value > RDX=0x00007f55af000000 points into unknown readable memory: 01 00 > 00 00 01 00 00 04 > RSI=0x007fffc05491d000 is an unknown value > > > As you can see, RSI is very large, which means you have an invalid oop > in the stack that's probably very large. > Can you please explain "stack" means here? Is it functions stack variable or some thing which GC internally uses? > > > [libjvm.so+0xcc0121] PCMarkAndPushClosure::do_oop(oopDesc**)+0x51 > [libjvm.so+0xc58c8b] OopMapSet::oops_do() > [libjvm.so+0x7521e9] frame::oops_do_internal()+0x99 <<<< HERE > [libjvm.so+0xf55757] JavaThread::oops_do()+0x187 > > > As others have mentioned, this kind of error is usually caused by > invalid use of Unsafe or JNI that leads to heap corruption. However, > it's plausible that somehow the VM has messed up the frame and tries to > mark an invalid oop. > Was trying to avoid using JNI calls to check if that is the cause but that seems not an option for now. Do you think any other way to get the root cause for this? > Thanks > - Ioi > > > On 3/3/20 8:02 AM, Sundara Mohan M wrote: > > Hi, > > I am seeing JVM crashes on our system in GC Thread with parallel gc > on > > x86 linux. Observed the same crash happening on JVM-11.0.6/13.0.2/13.0.1 > GA > > builds. > > Adding some logs lines to give some context. > > > > # > > # A fatal error has been detected by the Java Runtime Environment: > > # > > # SIGSEGV (0xb) at pc=0x00007f669c964311, pid=66684, tid=71106 > > # > > # JRE version: OpenJDK Runtime Environment (13.0.1+9) (build 13.0.1+9) > > # Java VM: OpenJDK 64-Bit Server VM (13.0.1+9, mixed mode, tiered, > parallel > > gc, linux-amd64) > > # Problematic frame: > > # V [libjvm.so+0xcd3311] PCMarkAndPushClosure::do_oop(oopDesc**)+0x51 > > # > > # No core dump will be written. Core dumps have been disabled. To enable > > core dumping, try "ulimit -c unlimited" before starting Java again > > # > > # If you would like to submit a bug report, please visit: > > # https://github.com/AdoptOpenJDK/openjdk-build/issues > > # > > > > Host: Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz, 48 cores, 125G, Red Hat > > Enterprise Linux Server release 6.10 (Santiago) > > Time: Thu Feb 6 11:43:48 2020 UTC elapsed time: 198626 seconds (2d 7h > 10m > > 26s) > > > > > > Following is the stack trace > > ex1: > > Stack: [0x00007fd01cbdb000,0x00007fd01ccdb000], sp=0x00007fd01ccd8890, > > free space=1014k > > Native frames: (J=compiled Java code, A=aot compiled Java code, > > j=interpreted, Vv=VM code, C=native code) > > *V [libjvm.so+0xcc0121] PCMarkAndPushClosure::do_oop(oopDesc**)+0x51* > > V [libjvm.so+0xc58c8b] OopMapSet::oops_do(frame const*, RegisterMap > > const*, OopClosure*)+0x2eb > > V [libjvm.so+0x7521e9] frame::oops_do_internal(OopClosure*, > > CodeBlobClosure*, RegisterMap*, bool)+0x99 > > V [libjvm.so+0xf55757] JavaThread::oops_do(OopClosure*, > > CodeBlobClosure*)+0x187 > > V [libjvm.so+0xcbb100] ThreadRootsMarkingTask::do_it(GCTaskManager*, > > unsigned int)+0xb0 > > V [libjvm.so+0x7e0f8b] GCTaskThread::run()+0x1eb > > V [libjvm.so+0xf5d43d] Thread::call_run()+0x10d > > V [libjvm.so+0xc74337] thread_native_entry(Thread*)+0xe7 > > > > JavaThread 0x00007fbeb9209800 (nid = 82380) was being processed > > Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) > > v ~RuntimeStub::_new_array_Java > > J 62465 c2 > > ch.qos.logback.classic.spi.ThrowableProxy.(Ljava/lang/Throwable;)V > > (207 bytes) @ 0x00007fd00ad43704 [0x00007fd00ad41420+0x00000000000022e4] > > J 474206 c2 > > > org.eclipse.jetty.util.log.JettyAwareLogger.log(Lorg/slf4j/Marker;ILjava/lang/String;[Ljava/lang/Object;Ljava/lang/Throwable;)V > > (134 bytes) @ 0x00007fd00c4e81ec [0x00007fd00c4e7ee0+0x000000000000030c] > > j > > > org.eclipse.jetty.util.log.JettyAwareLogger.warn(Ljava/lang/String;Ljava/lang/Throwable;)V+7 > > j > > > org.eclipse.jetty.util.log.Slf4jLog.warn(Ljava/lang/String;Ljava/lang/Throwable;)V+6 > > j > > > org.eclipse.jetty.server.HttpChannel.handleException(Ljava/lang/Throwable;)V+181 > > j > > > org.eclipse.jetty.server.HttpChannelOverHttp.handleException(Ljava/lang/Throwable;)V+13 > > J 64106 c2 org.eclipse.jetty.server.HttpChannel.handle()Z (997 bytes) @ > > 0x00007fd00c6d2cd4 [0x00007fd00c6cdec0+0x0000000000004e14] > > J 280430 c2 org.eclipse.jetty.server.HttpConnection.onFillable()V (334 > > bytes) @ 0x00007fd00da925f0 [0x00007fd00da91e40+0x00000000000007b0] > > J 41979 c2 org.eclipse.jetty.io.ChannelEndPoint$2.run()V (12 bytes) @ > > 0x00007fd00a14f604 [0x00007fd00a14f4e0+0x0000000000000124] > > J 86362 c2 org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run()V > > (565 bytes) @ 0x00007fd0087d7e34 [0x00007fd0087d7cc0+0x0000000000000174] > > J 75998 c2 java.lang.Thread.run()V java.base at 13.0.2 (17 bytes) @ > > 0x00007fd00c93b8d8 [0x00007fd00c93b8a0+0x0000000000000038] > > v ~StubRoutines::call_stub > > > > ex2: > > Stack: [0x00007f669869f000,0x00007f669879f000], sp=0x00007f669879c890, > > free space=1014k > > Native frames: (J=compiled Java code, A=aot compiled Java code, > > j=interpreted, Vv=VM code, C=native code) > > > > *V [libjvm.so+0xcd3311] PCMarkAndPushClosure::do_oop(oopDesc**)+0x51*V > > [libjvm.so+0xc6bf0b] OopMapSet::oops_do(frame const*, RegisterMap > const*, > > OopClosure*)+0x2eb > > V [libjvm.so+0x765489] frame::oops_do_internal(OopClosure*, > > CodeBlobClosure*, RegisterMap*, bool)+0x99 > > V [libjvm.so+0xf68b17] JavaThread::oops_do(OopClosure*, > > CodeBlobClosure*)+0x187 > > V [libjvm.so+0xcce2f0] ThreadRootsMarkingTask::do_it(GCTaskManager*, > > unsigned int)+0xb0 > > V [libjvm.so+0x7f422b] GCTaskThread::run()+0x1eb > > V [libjvm.so+0xf707fd] Thread::call_run()+0x10d > > V [libjvm.so+0xc875b7] thread_native_entry(Thread*)+0xe7 > > > > JavaThread 0x00007f5518004000 (nid = 75659) was being processed > > Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) > > v ~RuntimeStub::_new_array_Java > > J 54174 c2 > > ch.qos.logback.classic.spi.ThrowableProxy.(Ljava/lang/Throwable;)V > > (207 bytes) @ 0x00007f6687d92678 [0x00007f6687d8c700+0x0000000000005f78] > > J 334031 c2 > > com.xmas.webservice.exception.ExceptionLoggingWrapper.execute()V (1004 > > bytes) @ 0x00007f6686ede430 [0x00007f6686edd580+0x0000000000000eb0] > > J 53431 c2 > > > com.xmas.webservice.exception.mapper.AbstractExceptionMapper.toResponse(Lcom/xmas/beans/exceptions/mapper/V3ErrorCode;Ljava/lang/Exception;)Ljavax/ws/rs/core/Response; > > (105 bytes) @ 0x00007f6687db88b0 [0x00007f6687db8660+0x0000000000000250] > > J 63819 c2 > > > com.xmas.webservice.exception.mapper.RequestBlockedExceptionMapper.toResponse(Ljava/lang/Exception;)Ljavax/ws/rs/core/Response; > > (9 bytes) @ 0x00007f6686a6ed9c [0x00007f6686a6ecc0+0x00000000000000dc] > > J 334032 c2 > > > com.xmas.webservice.filters.ResponseSerializationWorker.processException()Ljava/io/InputStream; > > (332 bytes) @ 0x00007f668992ad34 [0x00007f668992a840+0x00000000000004f4] > > J 403918 c2 > > com.xmas.webservice.filters.ResponseSerializationWorker.execute()Z (272 > > bytes) @ 0x00007f66869d67fc [0x00007f66869d5e80+0x000000000000097c] > > J 17530 c2 > com.lafaspot.common.concurrent.internal.WorkerWrapper.execute()Z > > (208 bytes) @ 0x00007f66848b3708 [0x00007f66848b36a0+0x0000000000000068] > > J 31970% c2 > > > com.lafaspot.common.concurrent.internal.WorkerManagerOneThread.call()Lcom/lafaspot/common/concurrent/internal/WorkerManagerState; > > (486 bytes) @ 0x00007f668608dcb0 [0x00007f668608d5e0+0x00000000000006d0] > > j > > > com.lafaspot.common.concurrent.internal.WorkerManagerOneThread.call()Ljava/lang/Object;+1 > > J 4889 c1 java.util.concurrent.FutureTask.run()V java.base at 13.0.1 (123 > > bytes) @ 0x00007f667d0be604 [0x00007f667d0bdf80+0x0000000000000684] > > J 7487 c1 > > > java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V > > java.base at 13.0.1 (187 bytes) @ 0x00007f667dd45854 > > [0x00007f667dd44a60+0x0000000000000df4] > > J 7486 c1 java.util.concurrent.ThreadPoolExecutor$Worker.run()V > > java.base at 13.0.1 (9 bytes) @ 0x00007f667d1f643c > > [0x00007f667d1f63c0+0x000000000000007c] > > J 7078 c1 java.lang.Thread.run()V java.base at 13.0.1 (17 bytes) @ > > 0x00007f667d1f2d74 [0x00007f667d1f2c40+0x0000000000000134] > > v ~StubRoutines::call_stub > > > > Not very frequent but ~90 days ~120 crashes with following signal > > siginfo: si_signo: 11 (SIGSEGV), si_code: 128 (SI_KERNEL), si_addr: > > 0x0000000000000000 > > This signal is generated when we try to access non canonical address in > > linux. > > > > As suggested by Stefan in another thread i tried to > > add VerifyAfterGC/VerifyBeforeGC but it seems to increase the latency and > > applications not surviving our production traffic(timing out and requests > > are failing). > > > > Questions > > 1. When i looked at source code for printing stack trace i see following > > > https://github.com/openjdk/jdk11u/blob/master/src/hotspot/share/utilities/vmError.cpp#L696 > > (Prints native stack trace) > > > https://github.com/openjdk/jdk11u/blob/master/src/hotspot/share/utilities/vmError.cpp#L718 > > (printing Java thread stack trace if it is involved in GC crash) > > a. How do you know this java thread was involved in jvm crash? > > b. Can i assume the java thread printed after native stack trace was > the > > culprit? > > c. Since i am seeing the same frame (~RuntimeStub::_new_array_Java, J > > 54174 c2 ch.qos.logback.classic.spi.ThrowableProxy...) but > different > > stack trace in both crashes can this be the root cause? > > > > 2. Thinking of excluding compilation > > of ch.qos.logback.classic.spi.ThrowableProxy class and running in > > production to see if compilation of this method is the cause. Does it > make > > sense? > > > > 3. Any other suggestion on debugging this further? > > > > TIA > > Sundar > > Thanks Sundar From calvin.cheung at oracle.com Wed Mar 4 05:23:09 2020 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Tue, 3 Mar 2020 21:23:09 -0800 Subject: RFR(S) 8240481: Remove CDS usage of InstanceKlass::is_in_error_state Message-ID: JBS: https://bugs.openjdk.java.net/browse/JDK-8240481 webrev: http://cr.openjdk.java.net/~ccheung/jdk15/8240481/webrev.00/ Changes include: 1. A bool _fail_verification will be added to the DumpTimeSharedClassInfo to indicate a class has failed verification. ???Helper functions will be added to SystemDictionaryShared to set and get the above bool field. ???SystemDictionaryShared::set_class_has_failed_verification(ik) will be called instead of ik->set_in_error_state(). ???SystemDictionaryShared::has_class_failed_verification(ik) will be called instead of ik->is_in_error_state(). 2. The class CheckSharedClassesClosure and related code will be removed as similar work is already done in SystemDictionaryShared::check_excluded_classes(). Passed tier1 - 4 tests. thanks, Calvin From ioi.lam at oracle.com Wed Mar 4 05:50:08 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 3 Mar 2020 21:50:08 -0800 Subject: RFR: 8240197: Cannot start JVM when $JAVA_HOME includes CJK characters In-Reply-To: <37e2f672-3407-5b16-ab71-aa1e70e928db@oss.nttdata.com> References: <6ec2f649-8999-fdd5-c2df-465925c3663b@oss.nttdata.com> <2d5148fc-7d71-d94c-d4a3-e6424a38b091@oss.nttdata.com> <37e2f672-3407-5b16-ab71-aa1e70e928db@oss.nttdata.com> Message-ID: Hi Yasumasa, I think this function is getting too long and it's hard to figure out all the error conditions. (I feel sorry for you because the code wasn't pretty messy to begin with, but I think it's at a point that we need to do the cleanup. It took me like 30 minutes to understand what is going on) [0] I noticed you changed the return type to LPWSTR, but all the callers are using wchar_t*. Are these two types the same? ??? If you want to use LPWSTR within this function for consistency, maybe you can return the value as ??? return (wchar_t*)result; // LPWSTR and wchat_t* are the same type on Windows. ??? That way people unfamiliar with Windows (most of us) won't be confused. [1] I think the memory freeing can be done using a destructor, so you can simply return in case of error, without having a lot of "if" checks: ??? class State { ?????? errno_t & err; ? ????? char* buf; ??????? LPWSTR unicode_path; LPWSTR result; ??????? State(errno_t & e) : buf(NULL), unicode_path(NULL), result(NULL), err(e) { ??????????? err = SUCCESS; ???????? } ??????? ~State() { ????????? os::free(buf); ? ? ? ? ? os::free(unicode_path); ????????? if (err != ERROR_SUCCESS) { ???????????? os::free(result); ????????? } ??????? } ??????? LPWSTR as_error(errno_t e) { ??????????? err = e; ??????????? return NULL; ??????? } ??? } st; ??? if ((st.buf = static_cast(os::malloc(buf_size, mtInternal))) == NULL) { ??????? return st.as_error(ENOMEM); ??? } ??? .... ??? return st.result; [2] Will it be possible to break the functon down into smaller pieces, so it's easier to understand what's going on? E.g., Maybe the code that? converts the C string into a unicode string can be moved into a separate function? ? // Get required buffer size to convert to Unicode ? int unicode_path_len = MultiByteToWideChar(CP_THREAD_ACP, ???????????????????????????????????????????? MB_ERR_INVALID_CHARS, ???????????????????????????????????????????? buf, -1, ???????????????????????????????????????????? NULL, 0); ? if (unicode_path_len == 0) { ??? os::free(buf); ??? err = EINVAL; ??? return NULL; ? } ? .... ? LPWSTR unicode_path = static_cast(os::malloc(sizeof(WCHAR) * unicode_path_len, mtInternal)); ? if (unicode_path == NULL) { ??? err = ENOMEM; ? } else { ??? // This call would be success because it is checked in above ??? err = ERROR_SUCCESS; ??? MultiByteToWideChar(CP_THREAD_ACP, ??????????????????????? MB_ERR_INVALID_CHARS, ??????????????????????? buf, -1, ??????????????????????? unicode_path, unicode_path_len); [3] size_t result_len = prefix_len - prefix_off + additional_space; The variable name result_len is confusing. It's not the length of the result. [4] The old copying code was already hard to understand, and the new version now has 2 versions of the copying code. I think the reason is you want to avoid an extra memcpy by calling GetFullPathNameW directly on the result. ??? if (needs_fullpath) { ????? // Get required buffer size to convert to full path ????? DWORD full_path_len = GetFullPathNameW(unicode_path, 0, NULL, NULL); ????? if (full_path_len == 0) { ??????? err = EINVAL; ????? } else { ??????? size_t result_size = sizeof(WCHAR) * (result_len + full_path_len); ??????? result = static_cast(os::malloc(result_size, mtInternal)); ??????? if (result == NULL) { ????????? err = ENOMEM; ??????? } else { ????????? // This call would be success because it is checked in above ????????? GetFullPathNameW(unicode_path, full_path_len, result + prefix_len - prefix_off, NULL); ????????? // Copy prefix ????????? memcpy(result, prefix, sizeof(WCHAR) * prefix_len); ??????? } ????? } ??? } else { ????? size_t result_size = sizeof(WCHAR) * (result_len + unicode_path_len); ????? result = static_cast(os::malloc(result_size, mtInternal)); ????? if (result == NULL) { ??????? err = ENOMEM; ????? } else { ??????? // Copy unicode path ??????? memcpy(result + prefix_len - prefix_off, unicode_path, sizeof(WCHAR) * unicode_path_len); ??????? // Copy prefix ??????? memcpy(result, prefix, sizeof(WCHAR) * prefix_len); ????? } ??? } I think the code can be simplified if you do the fullpath version in a separate function: ??? if (needs_fullpath) { ?????? unicode_path = to_full_path(unicode_path, err, &unicode_path_len); ??? } ??? size_t result_size = ...... This will do one more memcpy, but I think it's not performance critical Thanks - Ioi On 3/3/20 4:19 PM, Yasumasa Suenaga wrote: > Thanks Ralf! > I fixed them. > > I need a Reviewer to push, so I uploaded new webrev: > > ? http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.03/ > > > Yasumasa > > > On 2020/03/03 23:25, Schmelter, Ralf wrote: >> Hi Yasumasa, >> >> thanks for your work. The code should now be able to handle any path. >> >> Two small changes (no new webrev needed): >> >> size_t result_len = prefix_len + prefix_off + additional_space >> should be >> size_t result_len = prefix_len - prefix_off + additional_space >> >> And you should copy the prefix after you copied the filename, since >> sometimes the prefix is required to overwrite the start of the filename: >> >> // This call would be success because it is checked in above >> GetFullPathNameW(unicode_path, full_path_len, result + prefix_len - >> prefix_off, NULL); >> // Copy prefix >> memcpy(result, prefix, sizeof(WCHAR) * prefix_len); >> >> and >> >> // Copy unicode path >> memcpy(result + prefix_len - prefix_off, unicode_path, sizeof(WCHAR) >> * unicode_path_len); >> // Copy prefix >> memcpy(result, prefix, sizeof(WCHAR) * prefix_len); >> >> With these changes the os_windows gtest and the appcds jtreg tests >> run without errors. >> >> Best regards, >> Ralf >> From ioi.lam at oracle.com Wed Mar 4 05:51:42 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 3 Mar 2020 21:51:42 -0800 Subject: RFR: 8240197: Cannot start JVM when $JAVA_HOME includes CJK characters In-Reply-To: References: <6ec2f649-8999-fdd5-c2df-465925c3663b@oss.nttdata.com> <2d5148fc-7d71-d94c-d4a3-e6424a38b091@oss.nttdata.com> <37e2f672-3407-5b16-ab71-aa1e70e928db@oss.nttdata.com> Message-ID: <36c2e0b5-a97b-b91e-9b51-270d43fb17b6@oracle.com> On 3/3/20 9:50 PM, Ioi Lam wrote: > Hi Yasumasa, > > I think this function is getting too long and it's hard to figure out > all the error conditions. > > (I feel sorry for you because the code wasn't pretty messy to begin > with, but I think it's at a point that we need to do the cleanup. It > took me like 30 minutes to understand what is going on) Typo "wasn't" -> "was". - Ioi From suenaga at oss.nttdata.com Wed Mar 4 06:22:10 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Wed, 4 Mar 2020 15:22:10 +0900 Subject: RFR: 8240197: Cannot start JVM when $JAVA_HOME includes CJK characters In-Reply-To: References: <6ec2f649-8999-fdd5-c2df-465925c3663b@oss.nttdata.com> <2d5148fc-7d71-d94c-d4a3-e6424a38b091@oss.nttdata.com> <37e2f672-3407-5b16-ab71-aa1e70e928db@oss.nttdata.com> Message-ID: Hi Ioi, > [1] I think the memory freeing can be done using a destructor, so you can simply return in case of error, without having a lot of "if" checks: What do you think to use `goto` statement like Linux kernel? Yasumasa On 2020/03/04 14:50, Ioi Lam wrote: > Hi Yasumasa, > > I think this function is getting too long and it's hard to figure out all the error conditions. > > (I feel sorry for you because the code wasn't pretty messy to begin with, but I think it's at a point that we need to do the cleanup. It took me like 30 minutes to understand what is going on) > > [0] I noticed you changed the return type to LPWSTR, but all the callers are using wchar_t*. Are these two types the same? > > ??? If you want to use LPWSTR within this function for consistency, maybe you can return the value as > > ??? return (wchar_t*)result; // LPWSTR and wchat_t* are the same type on Windows. > > ??? That way people unfamiliar with Windows (most of us) won't be confused. > > [1] I think the memory freeing can be done using a destructor, so you can simply return in case of error, without having a lot of "if" checks: > > ??? class State { > ?????? errno_t & err; > ? ????? char* buf; > ??????? LPWSTR unicode_path; > LPWSTR result; > > ??????? State(errno_t & e) : buf(NULL), unicode_path(NULL), result(NULL), err(e) { > ??????????? err = SUCCESS; > ???????? } > ??????? ~State() { > ????????? os::free(buf); > ? ? ? ? ? os::free(unicode_path); > ????????? if (err != ERROR_SUCCESS) { > ???????????? os::free(result); > ????????? } > ??????? } > ??????? LPWSTR as_error(errno_t e) { > ??????????? err = e; > ??????????? return NULL; > ??????? } > ??? } st; > > > ??? if ((st.buf = static_cast(os::malloc(buf_size, mtInternal))) == NULL) { > ??????? return st.as_error(ENOMEM); > ??? } > > ??? .... > ??? return st.result; > > [2] Will it be possible to break the functon down into smaller pieces, so it's easier to understand what's going on? > > E.g., Maybe the code that? converts the C string into a unicode string can be moved into a separate function? > > ? // Get required buffer size to convert to Unicode > ? int unicode_path_len = MultiByteToWideChar(CP_THREAD_ACP, > ???????????????????????????????????????????? MB_ERR_INVALID_CHARS, > ???????????????????????????????????????????? buf, -1, > ???????????????????????????????????????????? NULL, 0); > ? if (unicode_path_len == 0) { > ??? os::free(buf); > ??? err = EINVAL; > ??? return NULL; > ? } > > ? .... > > ? LPWSTR unicode_path = static_cast(os::malloc(sizeof(WCHAR) * unicode_path_len, mtInternal)); > ? if (unicode_path == NULL) { > ??? err = ENOMEM; > ? } else { > ??? // This call would be success because it is checked in above > ??? err = ERROR_SUCCESS; > ??? MultiByteToWideChar(CP_THREAD_ACP, > ??????????????????????? MB_ERR_INVALID_CHARS, > ??????????????????????? buf, -1, > ??????????????????????? unicode_path, unicode_path_len); > > > > [3] size_t result_len = prefix_len - prefix_off + additional_space; > > The variable name result_len is confusing. It's not the length of the result. > > [4] The old copying code was already hard to understand, and the new version now has 2 versions of the copying code. I think the reason is you want to avoid an extra memcpy by calling GetFullPathNameW directly on the result. > > ??? if (needs_fullpath) { > ????? // Get required buffer size to convert to full path > ????? DWORD full_path_len = GetFullPathNameW(unicode_path, 0, NULL, NULL); > ????? if (full_path_len == 0) { > ??????? err = EINVAL; > ????? } else { > ??????? size_t result_size = sizeof(WCHAR) * (result_len + full_path_len); > ??????? result = static_cast(os::malloc(result_size, mtInternal)); > ??????? if (result == NULL) { > ????????? err = ENOMEM; > ??????? } else { > ????????? // This call would be success because it is checked in above > ????????? GetFullPathNameW(unicode_path, full_path_len, result + prefix_len - prefix_off, NULL); > ????????? // Copy prefix > ????????? memcpy(result, prefix, sizeof(WCHAR) * prefix_len); > ??????? } > ????? } > ??? } else { > ????? size_t result_size = sizeof(WCHAR) * (result_len + unicode_path_len); > ????? result = static_cast(os::malloc(result_size, mtInternal)); > ????? if (result == NULL) { > ??????? err = ENOMEM; > ????? } else { > ??????? // Copy unicode path > ??????? memcpy(result + prefix_len - prefix_off, unicode_path, sizeof(WCHAR) * unicode_path_len); > ??????? // Copy prefix > ??????? memcpy(result, prefix, sizeof(WCHAR) * prefix_len); > ????? } > ??? } > > I think the code can be simplified if you do the fullpath version in a separate function: > > ??? if (needs_fullpath) { > ?????? unicode_path = to_full_path(unicode_path, err, &unicode_path_len); > ??? } > > ??? size_t result_size = ...... > > This will do one more memcpy, but I think it's not performance critical > > Thanks > - Ioi > > > On 3/3/20 4:19 PM, Yasumasa Suenaga wrote: >> Thanks Ralf! >> I fixed them. >> >> I need a Reviewer to push, so I uploaded new webrev: >> >> ? http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.03/ >> >> >> Yasumasa >> >> >> On 2020/03/03 23:25, Schmelter, Ralf wrote: >>> Hi Yasumasa, >>> >>> thanks for your work. The code should now be able to handle any path. >>> >>> Two small changes (no new webrev needed): >>> >>> size_t result_len = prefix_len + prefix_off + additional_space >>> should be >>> size_t result_len = prefix_len - prefix_off + additional_space >>> >>> And you should copy the prefix after you copied the filename, since sometimes the prefix is required to overwrite the start of the filename: >>> >>> // This call would be success because it is checked in above >>> GetFullPathNameW(unicode_path, full_path_len, result + prefix_len - prefix_off, NULL); >>> // Copy prefix >>> memcpy(result, prefix, sizeof(WCHAR) * prefix_len); >>> >>> and >>> >>> // Copy unicode path >>> memcpy(result + prefix_len - prefix_off, unicode_path, sizeof(WCHAR) * unicode_path_len); >>> // Copy prefix >>> memcpy(result, prefix, sizeof(WCHAR) * prefix_len); >>> >>> With these changes the os_windows gtest and the appcds jtreg tests run without errors. >>> >>> Best regards, >>> Ralf >>> > From ioi.lam at oracle.com Wed Mar 4 07:05:48 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 3 Mar 2020 23:05:48 -0800 Subject: RFR: 8240197: Cannot start JVM when $JAVA_HOME includes CJK characters In-Reply-To: References: <6ec2f649-8999-fdd5-c2df-465925c3663b@oss.nttdata.com> <2d5148fc-7d71-d94c-d4a3-e6424a38b091@oss.nttdata.com> <37e2f672-3407-5b16-ab71-aa1e70e928db@oss.nttdata.com> Message-ID: <954f0bf1-91e2-3a1b-e2cb-e93d8343b0e6@oracle.com> On 3/3/20 10:22 PM, Yasumasa Suenaga wrote: > Hi Ioi, > >> [1] I think the memory freeing can be done using a destructor, so you >> can simply return in case of error, without having a lot of "if" checks: > > What do you think to use `goto` statement like Linux kernel? > > I don't have a strong opinion but I think many/most HotSpot developers are against it. Also with goto, you need to store the return value in a variable, but with destructors, you can just say "return value", so the code is cleaner. Thanks - Ioi > Yasumasa > > > On 2020/03/04 14:50, Ioi Lam wrote: >> Hi Yasumasa, >> >> I think this function is getting too long and it's hard to figure out >> all the error conditions. >> >> (I feel sorry for you because the code wasn't pretty messy to begin >> with, but I think it's at a point that we need to do the cleanup. It >> took me like 30 minutes to understand what is going on) >> >> [0] I noticed you changed the return type to LPWSTR, but all the >> callers are using wchar_t*. Are these two types the same? >> >> ???? If you want to use LPWSTR within this function for consistency, >> maybe you can return the value as >> >> ???? return (wchar_t*)result; // LPWSTR and wchat_t* are the same >> type on Windows. >> >> ???? That way people unfamiliar with Windows (most of us) won't be >> confused. >> >> [1] I think the memory freeing can be done using a destructor, so you >> can simply return in case of error, without having a lot of "if" checks: >> >> ???? class State { >> ??????? errno_t & err; >> ?? ????? char* buf; >> ???????? LPWSTR unicode_path; >> LPWSTR result; >> >> ???????? State(errno_t & e) : buf(NULL), unicode_path(NULL), >> result(NULL), err(e) { >> ???????????? err = SUCCESS; >> ????????? } >> ???????? ~State() { >> ?????????? os::free(buf); >> ?? ? ? ? ? os::free(unicode_path); >> ?????????? if (err != ERROR_SUCCESS) { >> ????????????? os::free(result); >> ?????????? } >> ???????? } >> ???????? LPWSTR as_error(errno_t e) { >> ???????????? err = e; >> ???????????? return NULL; >> ???????? } >> ???? } st; >> >> >> ???? if ((st.buf = static_cast(os::malloc(buf_size, >> mtInternal))) == NULL) { >> ???????? return st.as_error(ENOMEM); >> ???? } >> >> ???? .... >> ???? return st.result; >> >> [2] Will it be possible to break the functon down into smaller >> pieces, so it's easier to understand what's going on? >> >> E.g., Maybe the code that? converts the C string into a unicode >> string can be moved into a separate function? >> >> ?? // Get required buffer size to convert to Unicode >> ?? int unicode_path_len = MultiByteToWideChar(CP_THREAD_ACP, >> MB_ERR_INVALID_CHARS, >> ????????????????????????????????????????????? buf, -1, >> ????????????????????????????????????????????? NULL, 0); >> ?? if (unicode_path_len == 0) { >> ???? os::free(buf); >> ???? err = EINVAL; >> ???? return NULL; >> ?? } >> >> ?? .... >> >> ?? LPWSTR unicode_path = static_cast(os::malloc(sizeof(WCHAR) >> * unicode_path_len, mtInternal)); >> ?? if (unicode_path == NULL) { >> ???? err = ENOMEM; >> ?? } else { >> ???? // This call would be success because it is checked in above >> ???? err = ERROR_SUCCESS; >> ???? MultiByteToWideChar(CP_THREAD_ACP, >> ???????????????????????? MB_ERR_INVALID_CHARS, >> ???????????????????????? buf, -1, >> ???????????????????????? unicode_path, unicode_path_len); >> >> >> >> [3] size_t result_len = prefix_len - prefix_off + additional_space; >> >> The variable name result_len is confusing. It's not the length of the >> result. >> >> [4] The old copying code was already hard to understand, and the new >> version now has 2 versions of the copying code. I think the reason is >> you want to avoid an extra memcpy by calling GetFullPathNameW >> directly on the result. >> >> ???? if (needs_fullpath) { >> ?????? // Get required buffer size to convert to full path >> ?????? DWORD full_path_len = GetFullPathNameW(unicode_path, 0, NULL, >> NULL); >> ?????? if (full_path_len == 0) { >> ???????? err = EINVAL; >> ?????? } else { >> ???????? size_t result_size = sizeof(WCHAR) * (result_len + >> full_path_len); >> ???????? result = static_cast(os::malloc(result_size, >> mtInternal)); >> ???????? if (result == NULL) { >> ?????????? err = ENOMEM; >> ???????? } else { >> ?????????? // This call would be success because it is checked in above >> ?????????? GetFullPathNameW(unicode_path, full_path_len, result + >> prefix_len - prefix_off, NULL); >> ?????????? // Copy prefix >> ?????????? memcpy(result, prefix, sizeof(WCHAR) * prefix_len); >> ???????? } >> ?????? } >> ???? } else { >> ?????? size_t result_size = sizeof(WCHAR) * (result_len + >> unicode_path_len); >> ?????? result = static_cast(os::malloc(result_size, >> mtInternal)); >> ?????? if (result == NULL) { >> ???????? err = ENOMEM; >> ?????? } else { >> ???????? // Copy unicode path >> ???????? memcpy(result + prefix_len - prefix_off, unicode_path, >> sizeof(WCHAR) * unicode_path_len); >> ???????? // Copy prefix >> ???????? memcpy(result, prefix, sizeof(WCHAR) * prefix_len); >> ?????? } >> ???? } >> >> I think the code can be simplified if you do the fullpath version in >> a separate function: >> >> ???? if (needs_fullpath) { >> ??????? unicode_path = to_full_path(unicode_path, err, >> &unicode_path_len); >> ???? } >> >> ???? size_t result_size = ...... >> >> This will do one more memcpy, but I think it's not performance critical >> >> Thanks >> - Ioi >> >> >> On 3/3/20 4:19 PM, Yasumasa Suenaga wrote: >>> Thanks Ralf! >>> I fixed them. >>> >>> I need a Reviewer to push, so I uploaded new webrev: >>> >>> ? http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.03/ >>> >>> >>> Yasumasa >>> >>> >>> On 2020/03/03 23:25, Schmelter, Ralf wrote: >>>> Hi Yasumasa, >>>> >>>> thanks for your work. The code should now be able to handle any path. >>>> >>>> Two small changes (no new webrev needed): >>>> >>>> size_t result_len = prefix_len + prefix_off + additional_space >>>> should be >>>> size_t result_len = prefix_len - prefix_off + additional_space >>>> >>>> And you should copy the prefix after you copied the filename, since >>>> sometimes the prefix is required to overwrite the start of the >>>> filename: >>>> >>>> // This call would be success because it is checked in above >>>> GetFullPathNameW(unicode_path, full_path_len, result + prefix_len - >>>> prefix_off, NULL); >>>> // Copy prefix >>>> memcpy(result, prefix, sizeof(WCHAR) * prefix_len); >>>> >>>> and >>>> >>>> // Copy unicode path >>>> memcpy(result + prefix_len - prefix_off, unicode_path, >>>> sizeof(WCHAR) * unicode_path_len); >>>> // Copy prefix >>>> memcpy(result, prefix, sizeof(WCHAR) * prefix_len); >>>> >>>> With these changes the os_windows gtest and the appcds jtreg tests >>>> run without errors. >>>> >>>> Best regards, >>>> Ralf >>>> >> From suenaga at oss.nttdata.com Wed Mar 4 08:22:07 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Wed, 4 Mar 2020 17:22:07 +0900 Subject: RFR: 8240197: Cannot start JVM when $JAVA_HOME includes CJK characters In-Reply-To: <954f0bf1-91e2-3a1b-e2cb-e93d8343b0e6@oracle.com> References: <6ec2f649-8999-fdd5-c2df-465925c3663b@oss.nttdata.com> <2d5148fc-7d71-d94c-d4a3-e6424a38b091@oss.nttdata.com> <37e2f672-3407-5b16-ab71-aa1e70e928db@oss.nttdata.com> <954f0bf1-91e2-3a1b-e2cb-e93d8343b0e6@oracle.com> Message-ID: <71ba4bad-05c9-042d-73dc-7f492fc1728a@oss.nttdata.com> Hi Ioi, Thanks for your comment. How about this change? http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.04/ This webrev adds the function both converting to unicode and getting full path, they would be called from wide_abs_unc_path(). Also MemoryCleaner class which is introduced in this webrev frees memory auomatically in d'tor. Thanks, Yasumasa On 2020/03/04 16:05, Ioi Lam wrote: > > > On 3/3/20 10:22 PM, Yasumasa Suenaga wrote: >> Hi Ioi, >> >>> [1] I think the memory freeing can be done using a destructor, so you can simply return in case of error, without having a lot of "if" checks: >> >> What do you think to use `goto` statement like Linux kernel? >> >> > I don't have a strong opinion but I think many/most HotSpot developers are against it. Also with goto, you need to store the return value in a variable, but with destructors, you can just say "return value", so the code is cleaner. > > Thanks > - Ioi > >> Yasumasa >> >> >> On 2020/03/04 14:50, Ioi Lam wrote: >>> Hi Yasumasa, >>> >>> I think this function is getting too long and it's hard to figure out all the error conditions. >>> >>> (I feel sorry for you because the code wasn't pretty messy to begin with, but I think it's at a point that we need to do the cleanup. It took me like 30 minutes to understand what is going on) >>> >>> [0] I noticed you changed the return type to LPWSTR, but all the callers are using wchar_t*. Are these two types the same? >>> >>> ???? If you want to use LPWSTR within this function for consistency, maybe you can return the value as >>> >>> ???? return (wchar_t*)result; // LPWSTR and wchat_t* are the same type on Windows. >>> >>> ???? That way people unfamiliar with Windows (most of us) won't be confused. >>> >>> [1] I think the memory freeing can be done using a destructor, so you can simply return in case of error, without having a lot of "if" checks: >>> >>> ???? class State { >>> ??????? errno_t & err; >>> ?? ????? char* buf; >>> ???????? LPWSTR unicode_path; >>> LPWSTR result; >>> >>> ???????? State(errno_t & e) : buf(NULL), unicode_path(NULL), result(NULL), err(e) { >>> ???????????? err = SUCCESS; >>> ????????? } >>> ???????? ~State() { >>> ?????????? os::free(buf); >>> ?? ? ? ? ? os::free(unicode_path); >>> ?????????? if (err != ERROR_SUCCESS) { >>> ????????????? os::free(result); >>> ?????????? } >>> ???????? } >>> ???????? LPWSTR as_error(errno_t e) { >>> ???????????? err = e; >>> ???????????? return NULL; >>> ???????? } >>> ???? } st; >>> >>> >>> ???? if ((st.buf = static_cast(os::malloc(buf_size, mtInternal))) == NULL) { >>> ???????? return st.as_error(ENOMEM); >>> ???? } >>> >>> ???? .... >>> ???? return st.result; >>> >>> [2] Will it be possible to break the functon down into smaller pieces, so it's easier to understand what's going on? >>> >>> E.g., Maybe the code that? converts the C string into a unicode string can be moved into a separate function? >>> >>> ?? // Get required buffer size to convert to Unicode >>> ?? int unicode_path_len = MultiByteToWideChar(CP_THREAD_ACP, >>> MB_ERR_INVALID_CHARS, >>> ????????????????????????????????????????????? buf, -1, >>> ????????????????????????????????????????????? NULL, 0); >>> ?? if (unicode_path_len == 0) { >>> ???? os::free(buf); >>> ???? err = EINVAL; >>> ???? return NULL; >>> ?? } >>> >>> ?? .... >>> >>> ?? LPWSTR unicode_path = static_cast(os::malloc(sizeof(WCHAR) * unicode_path_len, mtInternal)); >>> ?? if (unicode_path == NULL) { >>> ???? err = ENOMEM; >>> ?? } else { >>> ???? // This call would be success because it is checked in above >>> ???? err = ERROR_SUCCESS; >>> ???? MultiByteToWideChar(CP_THREAD_ACP, >>> ???????????????????????? MB_ERR_INVALID_CHARS, >>> ???????????????????????? buf, -1, >>> ???????????????????????? unicode_path, unicode_path_len); >>> >>> >>> >>> [3] size_t result_len = prefix_len - prefix_off + additional_space; >>> >>> The variable name result_len is confusing. It's not the length of the result. >>> >>> [4] The old copying code was already hard to understand, and the new version now has 2 versions of the copying code. I think the reason is you want to avoid an extra memcpy by calling GetFullPathNameW directly on the result. >>> >>> ???? if (needs_fullpath) { >>> ?????? // Get required buffer size to convert to full path >>> ?????? DWORD full_path_len = GetFullPathNameW(unicode_path, 0, NULL, NULL); >>> ?????? if (full_path_len == 0) { >>> ???????? err = EINVAL; >>> ?????? } else { >>> ???????? size_t result_size = sizeof(WCHAR) * (result_len + full_path_len); >>> ???????? result = static_cast(os::malloc(result_size, mtInternal)); >>> ???????? if (result == NULL) { >>> ?????????? err = ENOMEM; >>> ???????? } else { >>> ?????????? // This call would be success because it is checked in above >>> ?????????? GetFullPathNameW(unicode_path, full_path_len, result + prefix_len - prefix_off, NULL); >>> ?????????? // Copy prefix >>> ?????????? memcpy(result, prefix, sizeof(WCHAR) * prefix_len); >>> ???????? } >>> ?????? } >>> ???? } else { >>> ?????? size_t result_size = sizeof(WCHAR) * (result_len + unicode_path_len); >>> ?????? result = static_cast(os::malloc(result_size, mtInternal)); >>> ?????? if (result == NULL) { >>> ???????? err = ENOMEM; >>> ?????? } else { >>> ???????? // Copy unicode path >>> ???????? memcpy(result + prefix_len - prefix_off, unicode_path, sizeof(WCHAR) * unicode_path_len); >>> ???????? // Copy prefix >>> ???????? memcpy(result, prefix, sizeof(WCHAR) * prefix_len); >>> ?????? } >>> ???? } >>> >>> I think the code can be simplified if you do the fullpath version in a separate function: >>> >>> ???? if (needs_fullpath) { >>> ??????? unicode_path = to_full_path(unicode_path, err, &unicode_path_len); >>> ???? } >>> >>> ???? size_t result_size = ...... >>> >>> This will do one more memcpy, but I think it's not performance critical >>> >>> Thanks >>> - Ioi >>> >>> >>> On 3/3/20 4:19 PM, Yasumasa Suenaga wrote: >>>> Thanks Ralf! >>>> I fixed them. >>>> >>>> I need a Reviewer to push, so I uploaded new webrev: >>>> >>>> ? http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.03/ >>>> >>>> >>>> Yasumasa >>>> >>>> >>>> On 2020/03/03 23:25, Schmelter, Ralf wrote: >>>>> Hi Yasumasa, >>>>> >>>>> thanks for your work. The code should now be able to handle any path. >>>>> >>>>> Two small changes (no new webrev needed): >>>>> >>>>> size_t result_len = prefix_len + prefix_off + additional_space >>>>> should be >>>>> size_t result_len = prefix_len - prefix_off + additional_space >>>>> >>>>> And you should copy the prefix after you copied the filename, since sometimes the prefix is required to overwrite the start of the filename: >>>>> >>>>> // This call would be success because it is checked in above >>>>> GetFullPathNameW(unicode_path, full_path_len, result + prefix_len - prefix_off, NULL); >>>>> // Copy prefix >>>>> memcpy(result, prefix, sizeof(WCHAR) * prefix_len); >>>>> >>>>> and >>>>> >>>>> // Copy unicode path >>>>> memcpy(result + prefix_len - prefix_off, unicode_path, sizeof(WCHAR) * unicode_path_len); >>>>> // Copy prefix >>>>> memcpy(result, prefix, sizeof(WCHAR) * prefix_len); >>>>> >>>>> With these changes the os_windows gtest and the appcds jtreg tests run without errors. >>>>> >>>>> Best regards, >>>>> Ralf >>>>> >>> > From stefan.karlsson at oracle.com Wed Mar 4 08:37:03 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 4 Mar 2020 09:37:03 +0100 Subject: Need help on debugging JVM crash In-Reply-To: <86c8b74f-9f11-b9b5-00f2-30360d0e8b6f@oracle.com> References: <86c8b74f-9f11-b9b5-00f2-30360d0e8b6f@oracle.com> Message-ID: <6420763d-43e8-9ee8-0041-c06578911644@oracle.com> FWIW, I see that this is run with -XX:-OmitStackTraceInFastThrowFalse. Maybe there's a problem with that flag? Some more info from the hs_err file that could further clues to the problem: The Java thread the GC is scanning is creating a ThrowableProxy, and is in the process of taking a slow path to allocate an array. Looking at the code it seems like it first calls Thread.getStackTrace(), and then creates an array of proxies to those elements. One of the hs_err files report over > 800 OutOfMemoryErrors. StefanK On 2020-03-03 21:12, Ioi Lam wrote: > The crash happened while the GC is running. I tried disasm of the > crashing address PCMarkAndPushClosure::do_oop(oopDesc**)+0x51 (from > the 13.0.1+9 GA binaries of AdoptJDK) > > (gdb) x/80i _ZN20PCMarkAndPushClosure6do_oopEPP7oopDesc > ?? <>:??? push?? %rbp > ?? <+1>:??? mov??? %rsp,%rbp > ?? <+4>:??? push?? %r13 > ?? <+6>:??? push?? %r12 > ?? <+8>:??? push?? %rbx > ?? <+9>:??? sub??? $0x8,%rsp > ?? <+13>:??? mov??? (%rsi),%rbx? ;;; rbx = oop > ?? <+16>:??? test?? %rbx,%rbx??? ;;; oop != null? > ?? <+19>:??? je???? 0x7ffff67ca317 > <_ZN20PCMarkAndPushClosure6do_oopEPP7oopDesc+87> > ?? <+21>:??? lea??? 0x9c0afc(%rip),%rax??????? # 0x7ffff718add8 > <_ZN20ParCompactionManager12_mark_bitmapE> > ?? <+28>:??? mov??? %rbx,%rcx??? ;;; rcx = oop > ?? <+31>:??? mov??? (%rax),%rdx? ;;; rdx = > ParCompactionManager::_mark_bitmap > ?? <+34>:??? sub??? (%rdx),%rcx? ;;; rcx = oop - > _mark_bitmap->_region_start > ?? <+37>:??? mov??? 0x10(%rdx),%rdx ;; rdx = > _mark_bitmap->_beg_bits->_map > ?? <+41>:??? mov??? %rcx,%rax?? ;;;? rax = oop - > _mark_bitmap->_region_start > ?? <+44>:??? lea??? 0x93b935(%rip),%rcx?????? # 0x7ffff7105c28 > > ?? <+51>:??? shr??? $0x3,%rax > ?? <+55>:??? mov??? (%rcx),%ecx > ?? <+57>:??? shr??? %cl,%rax > ?? <+60>:??? mov??? %rax,%rcx > ?? <+63>:??? mov??? %rax,%rsi????? ;;; rsi = index of oop inside > mark_bitmap > ?? <+66>:??? mov??? $0x1,%eax > ?? <+71>:??? and??? $0x3f,%ecx > ?? <+74>:??? shr??? $0x6,%rsi > ?? <+78>:??? shl??? %cl,%rax > ?? <+81>:??? test?? %rax,(%rdx,%rsi,8) << crash > > > This looks like that the oop that we try to mark is actually outside > of the heap range, so trying to mark it in the mark_bitmap causes this: > > > ?? siginfo: si_signo: 11 (SIGSEGV), si_code: 128 (SI_KERNEL), si_addr: > ?? 0x0000000000000000 > > > Here are the values of the registers for the "test" instruction above: > > > ??? RAX=0x0000000000000001 is an unknown value > ??? RDX=0x00007f55af000000 points into unknown readable memory: 01 00 > 00 00 01 00 00 04 > ??? RSI=0x007fffc05491d000 is an unknown value > > > As you can see, RSI is very large, which means you have an invalid oop > in the stack that's probably very large. > > > ??? [libjvm.so+0xcc0121] PCMarkAndPushClosure::do_oop(oopDesc**)+0x51 > ??? [libjvm.so+0xc58c8b]? OopMapSet::oops_do() > ??? [libjvm.so+0x7521e9]? frame::oops_do_internal()+0x99 <<<< HERE > ??? [libjvm.so+0xf55757]? JavaThread::oops_do()+0x187 > > > As others have mentioned, this kind of error is usually caused by > invalid use of Unsafe or JNI that leads to heap corruption. However, > it's plausible that somehow the VM has messed up the frame and tries > to mark an invalid oop. > > Thanks > - Ioi > > > On 3/3/20 8:02 AM, Sundara Mohan M wrote: >> Hi, >> ???? I am seeing JVM crashes on our system in GC Thread with parallel >> gc on >> x86 linux. Observed the same crash happening on >> JVM-11.0.6/13.0.2/13.0.1 GA >> builds. >> Adding some logs lines to give some context. >> >> # >> # A fatal error has been detected by the Java Runtime Environment: >> # >> #? SIGSEGV (0xb) at pc=0x00007f669c964311, pid=66684, tid=71106 >> # >> # JRE version: OpenJDK Runtime Environment (13.0.1+9) (build 13.0.1+9) >> # Java VM: OpenJDK 64-Bit Server VM (13.0.1+9, mixed mode, tiered, >> parallel >> gc, linux-amd64) >> # Problematic frame: >> # V? [libjvm.so+0xcd3311] PCMarkAndPushClosure::do_oop(oopDesc**)+0x51 >> # >> # No core dump will be written. Core dumps have been disabled. To enable >> core dumping, try "ulimit -c unlimited" before starting Java again >> # >> # If you would like to submit a bug report, please visit: >> #?? https://github.com/AdoptOpenJDK/openjdk-build/issues >> # >> >> Host: Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz, 48 cores, 125G, Red Hat >> Enterprise Linux Server release 6.10 (Santiago) >> Time: Thu Feb? 6 11:43:48 2020 UTC elapsed time: 198626 seconds (2d >> 7h 10m >> 26s) >> >> >> Following is the stack trace >> ex1: >> Stack: [0x00007fd01cbdb000,0x00007fd01ccdb000], sp=0x00007fd01ccd8890, >> ? free space=1014k >> Native frames: (J=compiled Java code, A=aot compiled Java code, >> j=interpreted, Vv=VM code, C=native code) >> *V? [libjvm.so+0xcc0121] PCMarkAndPushClosure::do_oop(oopDesc**)+0x51* >> V? [libjvm.so+0xc58c8b]? OopMapSet::oops_do(frame const*, RegisterMap >> const*, OopClosure*)+0x2eb >> V? [libjvm.so+0x7521e9]? frame::oops_do_internal(OopClosure*, >> CodeBlobClosure*, RegisterMap*, bool)+0x99 >> V? [libjvm.so+0xf55757]? JavaThread::oops_do(OopClosure*, >> CodeBlobClosure*)+0x187 >> V? [libjvm.so+0xcbb100] ThreadRootsMarkingTask::do_it(GCTaskManager*, >> unsigned int)+0xb0 >> V? [libjvm.so+0x7e0f8b]? GCTaskThread::run()+0x1eb >> V? [libjvm.so+0xf5d43d]? Thread::call_run()+0x10d >> V? [libjvm.so+0xc74337]? thread_native_entry(Thread*)+0xe7 >> >> JavaThread 0x00007fbeb9209800 (nid = 82380) was being processed >> Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) >> v? ~RuntimeStub::_new_array_Java >> J 62465 c2 >> ch.qos.logback.classic.spi.ThrowableProxy.(Ljava/lang/Throwable;)V >> (207 bytes) @ 0x00007fd00ad43704 [0x00007fd00ad41420+0x00000000000022e4] >> J 474206 c2 >> org.eclipse.jetty.util.log.JettyAwareLogger.log(Lorg/slf4j/Marker;ILjava/lang/String;[Ljava/lang/Object;Ljava/lang/Throwable;)V >> >> (134 bytes) @ 0x00007fd00c4e81ec [0x00007fd00c4e7ee0+0x000000000000030c] >> j >> org.eclipse.jetty.util.log.JettyAwareLogger.warn(Ljava/lang/String;Ljava/lang/Throwable;)V+7 >> j >> org.eclipse.jetty.util.log.Slf4jLog.warn(Ljava/lang/String;Ljava/lang/Throwable;)V+6 >> j >> org.eclipse.jetty.server.HttpChannel.handleException(Ljava/lang/Throwable;)V+181 >> j >> org.eclipse.jetty.server.HttpChannelOverHttp.handleException(Ljava/lang/Throwable;)V+13 >> J 64106 c2 org.eclipse.jetty.server.HttpChannel.handle()Z (997 bytes) @ >> 0x00007fd00c6d2cd4 [0x00007fd00c6cdec0+0x0000000000004e14] >> J 280430 c2 org.eclipse.jetty.server.HttpConnection.onFillable()V (334 >> bytes) @ 0x00007fd00da925f0 [0x00007fd00da91e40+0x00000000000007b0] >> J 41979 c2 org.eclipse.jetty.io.ChannelEndPoint$2.run()V (12 bytes) @ >> 0x00007fd00a14f604 [0x00007fd00a14f4e0+0x0000000000000124] >> J 86362 c2 org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run()V >> (565 bytes) @ 0x00007fd0087d7e34 [0x00007fd0087d7cc0+0x0000000000000174] >> J 75998 c2 java.lang.Thread.run()V java.base at 13.0.2 (17 bytes) @ >> 0x00007fd00c93b8d8 [0x00007fd00c93b8a0+0x0000000000000038] >> v? ~StubRoutines::call_stub >> >> ex2: >> Stack: [0x00007f669869f000,0x00007f669879f000], sp=0x00007f669879c890, >> ? free space=1014k >> Native frames: (J=compiled Java code, A=aot compiled Java code, >> j=interpreted, Vv=VM code, C=native code) >> >> *V? [libjvm.so+0xcd3311] PCMarkAndPushClosure::do_oop(oopDesc**)+0x51*V >> ? [libjvm.so+0xc6bf0b]? OopMapSet::oops_do(frame const*, RegisterMap >> const*, >> OopClosure*)+0x2eb >> V? [libjvm.so+0x765489]? frame::oops_do_internal(OopClosure*, >> CodeBlobClosure*, RegisterMap*, bool)+0x99 >> V? [libjvm.so+0xf68b17]? JavaThread::oops_do(OopClosure*, >> CodeBlobClosure*)+0x187 >> V? [libjvm.so+0xcce2f0] ThreadRootsMarkingTask::do_it(GCTaskManager*, >> unsigned int)+0xb0 >> V? [libjvm.so+0x7f422b]? GCTaskThread::run()+0x1eb >> V? [libjvm.so+0xf707fd]? Thread::call_run()+0x10d >> V? [libjvm.so+0xc875b7]? thread_native_entry(Thread*)+0xe7 >> >> JavaThread 0x00007f5518004000 (nid = 75659) was being processed >> Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) >> v? ~RuntimeStub::_new_array_Java >> J 54174 c2 >> ch.qos.logback.classic.spi.ThrowableProxy.(Ljava/lang/Throwable;)V >> (207 bytes) @ 0x00007f6687d92678 [0x00007f6687d8c700+0x0000000000005f78] >> J 334031 c2 >> com.xmas.webservice.exception.ExceptionLoggingWrapper.execute()V (1004 >> bytes) @ 0x00007f6686ede430 [0x00007f6686edd580+0x0000000000000eb0] >> J 53431 c2 >> com.xmas.webservice.exception.mapper.AbstractExceptionMapper.toResponse(Lcom/xmas/beans/exceptions/mapper/V3ErrorCode;Ljava/lang/Exception;)Ljavax/ws/rs/core/Response; >> >> (105 bytes) @ 0x00007f6687db88b0 [0x00007f6687db8660+0x0000000000000250] >> J 63819 c2 >> com.xmas.webservice.exception.mapper.RequestBlockedExceptionMapper.toResponse(Ljava/lang/Exception;)Ljavax/ws/rs/core/Response; >> >> (9 bytes) @ 0x00007f6686a6ed9c [0x00007f6686a6ecc0+0x00000000000000dc] >> J 334032 c2 >> com.xmas.webservice.filters.ResponseSerializationWorker.processException()Ljava/io/InputStream; >> >> (332 bytes) @ 0x00007f668992ad34 [0x00007f668992a840+0x00000000000004f4] >> J 403918 c2 >> com.xmas.webservice.filters.ResponseSerializationWorker.execute()Z (272 >> bytes) @ 0x00007f66869d67fc [0x00007f66869d5e80+0x000000000000097c] >> J 17530 c2 >> com.lafaspot.common.concurrent.internal.WorkerWrapper.execute()Z >> (208 bytes) @ 0x00007f66848b3708 [0x00007f66848b36a0+0x0000000000000068] >> J 31970% c2 >> com.lafaspot.common.concurrent.internal.WorkerManagerOneThread.call()Lcom/lafaspot/common/concurrent/internal/WorkerManagerState; >> >> (486 bytes) @ 0x00007f668608dcb0 [0x00007f668608d5e0+0x00000000000006d0] >> j >> com.lafaspot.common.concurrent.internal.WorkerManagerOneThread.call()Ljava/lang/Object;+1 >> J 4889 c1 java.util.concurrent.FutureTask.run()V java.base at 13.0.1 (123 >> bytes) @ 0x00007f667d0be604 [0x00007f667d0bdf80+0x0000000000000684] >> J 7487 c1 >> java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V >> >> java.base at 13.0.1 (187 bytes) @ 0x00007f667dd45854 >> [0x00007f667dd44a60+0x0000000000000df4] >> J 7486 c1 java.util.concurrent.ThreadPoolExecutor$Worker.run()V >> java.base at 13.0.1 (9 bytes) @ 0x00007f667d1f643c >> [0x00007f667d1f63c0+0x000000000000007c] >> J 7078 c1 java.lang.Thread.run()V java.base at 13.0.1 (17 bytes) @ >> 0x00007f667d1f2d74 [0x00007f667d1f2c40+0x0000000000000134] >> v? ~StubRoutines::call_stub >> >> Not very frequent but ~90 days ~120 crashes with following signal >> siginfo: si_signo: 11 (SIGSEGV), si_code: 128 (SI_KERNEL), si_addr: >> 0x0000000000000000 >> This signal is generated when we try to access non canonical address in >> linux. >> >> As suggested by Stefan in another thread i tried to >> add VerifyAfterGC/VerifyBeforeGC but it seems to increase the latency >> and >> applications not surviving our production traffic(timing out and >> requests >> are failing). >> >> Questions >> 1. When i looked at source code for printing stack trace i see following >> https://github.com/openjdk/jdk11u/blob/master/src/hotspot/share/utilities/vmError.cpp#L696 >> >> (Prints native stack trace) >> https://github.com/openjdk/jdk11u/blob/master/src/hotspot/share/utilities/vmError.cpp#L718 >> >> (printing Java thread stack trace if it is involved in GC crash) >> ?? a. How do you know this java thread was involved in jvm crash? >> ?? b. Can i assume the java thread printed after native stack trace >> was the >> culprit? >> ?? c. Since i am seeing the same frame (~RuntimeStub::_new_array_Java, J >> 54174 c2 ch.qos.logback.classic.spi.ThrowableProxy...) but >> different >> stack trace in both crashes can this be the root cause? >> >> 2. Thinking of excluding compilation >> of ch.qos.logback.classic.spi.ThrowableProxy class and running in >> production to see if compilation of this method is the cause. Does it >> make >> sense? >> >> 3. Any other suggestion on debugging this further? >> >> TIA >> Sundar > From suenaga at oss.nttdata.com Wed Mar 4 12:18:56 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Wed, 4 Mar 2020 21:18:56 +0900 Subject: RFR: 8240197: Cannot start JVM when $JAVA_HOME includes CJK characters In-Reply-To: <71ba4bad-05c9-042d-73dc-7f492fc1728a@oss.nttdata.com> References: <6ec2f649-8999-fdd5-c2df-465925c3663b@oss.nttdata.com> <2d5148fc-7d71-d94c-d4a3-e6424a38b091@oss.nttdata.com> <37e2f672-3407-5b16-ab71-aa1e70e928db@oss.nttdata.com> <954f0bf1-91e2-3a1b-e2cb-e93d8343b0e6@oracle.com> <71ba4bad-05c9-042d-73dc-7f492fc1728a@oss.nttdata.com> Message-ID: <5ed449c8-6bc0-b219-3c4e-baaeae1cbfd8@oss.nttdata.com> Hi Ioi, Ralf, How about this change? If we can use macro and alloca(), we can fix more simply. http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.04-2/ Thanks, Yasumasa On 2020/03/04 17:22, Yasumasa Suenaga wrote: > Hi Ioi, > > Thanks for your comment. > How about this change? > > ? http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.04/ > > This webrev adds the function both converting to unicode and getting full path, > they would be called from wide_abs_unc_path(). > Also MemoryCleaner class which is introduced in this webrev frees memory auomatically in d'tor. > > > Thanks, > > Yasumasa > > > On 2020/03/04 16:05, Ioi Lam wrote: >> >> >> On 3/3/20 10:22 PM, Yasumasa Suenaga wrote: >>> Hi Ioi, >>> >>>> [1] I think the memory freeing can be done using a destructor, so you can simply return in case of error, without having a lot of "if" checks: >>> >>> What do you think to use `goto` statement like Linux kernel? >>> >>> >> I don't have a strong opinion but I think many/most HotSpot developers are against it. Also with goto, you need to store the return value in a variable, but with destructors, you can just say "return value", so the code is cleaner. >> >> Thanks >> - Ioi >> >>> Yasumasa >>> >>> >>> On 2020/03/04 14:50, Ioi Lam wrote: >>>> Hi Yasumasa, >>>> >>>> I think this function is getting too long and it's hard to figure out all the error conditions. >>>> >>>> (I feel sorry for you because the code wasn't pretty messy to begin with, but I think it's at a point that we need to do the cleanup. It took me like 30 minutes to understand what is going on) >>>> >>>> [0] I noticed you changed the return type to LPWSTR, but all the callers are using wchar_t*. Are these two types the same? >>>> >>>> ???? If you want to use LPWSTR within this function for consistency, maybe you can return the value as >>>> >>>> ???? return (wchar_t*)result; // LPWSTR and wchat_t* are the same type on Windows. >>>> >>>> ???? That way people unfamiliar with Windows (most of us) won't be confused. >>>> >>>> [1] I think the memory freeing can be done using a destructor, so you can simply return in case of error, without having a lot of "if" checks: >>>> >>>> ???? class State { >>>> ??????? errno_t & err; >>>> ?? ????? char* buf; >>>> ???????? LPWSTR unicode_path; >>>> LPWSTR result; >>>> >>>> ???????? State(errno_t & e) : buf(NULL), unicode_path(NULL), result(NULL), err(e) { >>>> ???????????? err = SUCCESS; >>>> ????????? } >>>> ???????? ~State() { >>>> ?????????? os::free(buf); >>>> ?? ? ? ? ? os::free(unicode_path); >>>> ?????????? if (err != ERROR_SUCCESS) { >>>> ????????????? os::free(result); >>>> ?????????? } >>>> ???????? } >>>> ???????? LPWSTR as_error(errno_t e) { >>>> ???????????? err = e; >>>> ???????????? return NULL; >>>> ???????? } >>>> ???? } st; >>>> >>>> >>>> ???? if ((st.buf = static_cast(os::malloc(buf_size, mtInternal))) == NULL) { >>>> ???????? return st.as_error(ENOMEM); >>>> ???? } >>>> >>>> ???? .... >>>> ???? return st.result; >>>> >>>> [2] Will it be possible to break the functon down into smaller pieces, so it's easier to understand what's going on? >>>> >>>> E.g., Maybe the code that? converts the C string into a unicode string can be moved into a separate function? >>>> >>>> ?? // Get required buffer size to convert to Unicode >>>> ?? int unicode_path_len = MultiByteToWideChar(CP_THREAD_ACP, >>>> MB_ERR_INVALID_CHARS, >>>> ????????????????????????????????????????????? buf, -1, >>>> ????????????????????????????????????????????? NULL, 0); >>>> ?? if (unicode_path_len == 0) { >>>> ???? os::free(buf); >>>> ???? err = EINVAL; >>>> ???? return NULL; >>>> ?? } >>>> >>>> ?? .... >>>> >>>> ?? LPWSTR unicode_path = static_cast(os::malloc(sizeof(WCHAR) * unicode_path_len, mtInternal)); >>>> ?? if (unicode_path == NULL) { >>>> ???? err = ENOMEM; >>>> ?? } else { >>>> ???? // This call would be success because it is checked in above >>>> ???? err = ERROR_SUCCESS; >>>> ???? MultiByteToWideChar(CP_THREAD_ACP, >>>> ???????????????????????? MB_ERR_INVALID_CHARS, >>>> ???????????????????????? buf, -1, >>>> ???????????????????????? unicode_path, unicode_path_len); >>>> >>>> >>>> >>>> [3] size_t result_len = prefix_len - prefix_off + additional_space; >>>> >>>> The variable name result_len is confusing. It's not the length of the result. >>>> >>>> [4] The old copying code was already hard to understand, and the new version now has 2 versions of the copying code. I think the reason is you want to avoid an extra memcpy by calling GetFullPathNameW directly on the result. >>>> >>>> ???? if (needs_fullpath) { >>>> ?????? // Get required buffer size to convert to full path >>>> ?????? DWORD full_path_len = GetFullPathNameW(unicode_path, 0, NULL, NULL); >>>> ?????? if (full_path_len == 0) { >>>> ???????? err = EINVAL; >>>> ?????? } else { >>>> ???????? size_t result_size = sizeof(WCHAR) * (result_len + full_path_len); >>>> ???????? result = static_cast(os::malloc(result_size, mtInternal)); >>>> ???????? if (result == NULL) { >>>> ?????????? err = ENOMEM; >>>> ???????? } else { >>>> ?????????? // This call would be success because it is checked in above >>>> ?????????? GetFullPathNameW(unicode_path, full_path_len, result + prefix_len - prefix_off, NULL); >>>> ?????????? // Copy prefix >>>> ?????????? memcpy(result, prefix, sizeof(WCHAR) * prefix_len); >>>> ???????? } >>>> ?????? } >>>> ???? } else { >>>> ?????? size_t result_size = sizeof(WCHAR) * (result_len + unicode_path_len); >>>> ?????? result = static_cast(os::malloc(result_size, mtInternal)); >>>> ?????? if (result == NULL) { >>>> ???????? err = ENOMEM; >>>> ?????? } else { >>>> ???????? // Copy unicode path >>>> ???????? memcpy(result + prefix_len - prefix_off, unicode_path, sizeof(WCHAR) * unicode_path_len); >>>> ???????? // Copy prefix >>>> ???????? memcpy(result, prefix, sizeof(WCHAR) * prefix_len); >>>> ?????? } >>>> ???? } >>>> >>>> I think the code can be simplified if you do the fullpath version in a separate function: >>>> >>>> ???? if (needs_fullpath) { >>>> ??????? unicode_path = to_full_path(unicode_path, err, &unicode_path_len); >>>> ???? } >>>> >>>> ???? size_t result_size = ...... >>>> >>>> This will do one more memcpy, but I think it's not performance critical >>>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>> On 3/3/20 4:19 PM, Yasumasa Suenaga wrote: >>>>> Thanks Ralf! >>>>> I fixed them. >>>>> >>>>> I need a Reviewer to push, so I uploaded new webrev: >>>>> >>>>> ? http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.03/ >>>>> >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2020/03/03 23:25, Schmelter, Ralf wrote: >>>>>> Hi Yasumasa, >>>>>> >>>>>> thanks for your work. The code should now be able to handle any path. >>>>>> >>>>>> Two small changes (no new webrev needed): >>>>>> >>>>>> size_t result_len = prefix_len + prefix_off + additional_space >>>>>> should be >>>>>> size_t result_len = prefix_len - prefix_off + additional_space >>>>>> >>>>>> And you should copy the prefix after you copied the filename, since sometimes the prefix is required to overwrite the start of the filename: >>>>>> >>>>>> // This call would be success because it is checked in above >>>>>> GetFullPathNameW(unicode_path, full_path_len, result + prefix_len - prefix_off, NULL); >>>>>> // Copy prefix >>>>>> memcpy(result, prefix, sizeof(WCHAR) * prefix_len); >>>>>> >>>>>> and >>>>>> >>>>>> // Copy unicode path >>>>>> memcpy(result + prefix_len - prefix_off, unicode_path, sizeof(WCHAR) * unicode_path_len); >>>>>> // Copy prefix >>>>>> memcpy(result, prefix, sizeof(WCHAR) * prefix_len); >>>>>> >>>>>> With these changes the os_windows gtest and the appcds jtreg tests run without errors. >>>>>> >>>>>> Best regards, >>>>>> Ralf >>>>>> >>>> >> From stefan.karlsson at oracle.com Wed Mar 4 14:25:23 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 4 Mar 2020 15:25:23 +0100 Subject: RFR: 8240529: CheckUnhandledOops breaks NULL check in Modules::define_module Message-ID: Hi all, Please review this small fix to CheckUnhandledOops. https://cr.openjdk.java.net/~stefank/8240529/webrev.01 https://bugs.openjdk.java.net/browse/JDK-8240529 'loader' is an unhandled oop that is alive across a safepoint check and then is used for a NULL check. When the safepoint check executes the value of the oop is changed to 0xff...ff1. If the oop was NULL then the check now starts to fail. When this happens ClassLoader::_exploded_entries isn't populated as expected, causing class loading failures with exploded builds. Thanks, StefanK From harold.seigel at oracle.com Wed Mar 4 14:36:26 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Wed, 4 Mar 2020 09:36:26 -0500 Subject: RFR: 8240529: CheckUnhandledOops breaks NULL check in Modules::define_module In-Reply-To: References: Message-ID: Looks good and trivial.? Thanks for finding and fixing this. Harold On 3/4/2020 9:25 AM, Stefan Karlsson wrote: > Hi all, > > Please review this small fix to CheckUnhandledOops. > > https://cr.openjdk.java.net/~stefank/8240529/webrev.01 > https://bugs.openjdk.java.net/browse/JDK-8240529 > > 'loader' is an unhandled oop that is alive across a safepoint check > and then is used for a NULL check. When the safepoint check executes > the value of the oop is changed to 0xff...ff1. If the oop was NULL > then the check now starts to fail. When this happens > ClassLoader::_exploded_entries isn't populated as expected, causing > class loading failures with exploded builds. > > Thanks, > StefanK From stefan.karlsson at oracle.com Wed Mar 4 14:48:20 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 4 Mar 2020 15:48:20 +0100 Subject: RFR: 8240530: CheckUnhandledOops breaks BacktraceBuilder::set_has_hidden_top_frame Message-ID: <89b3460e-47a2-daec-bbd0-755e095d4c46@oracle.com> Hi all, Please review this patch to get rid of a unhandled oop null check in the BacktraceBuilder. https://cr.openjdk.java.net/~stefank/8240530/webrev.01/ https://bugs.openjdk.java.net/browse/JDK-8240530 The patch changes the type from oop to void* to prevent NULL oops from being overwritten with non-NULL values. I think an alternative fix would be to change the type of _has_hidden_top_frame to bool. Maybe someone familiar with this code knows why this was encoded with an oop NULL check instead of a plain bool? Testing with tier1-3 with CheckUnhandledOops on by default. StefanK From coleen.phillimore at oracle.com Wed Mar 4 14:59:00 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 4 Mar 2020 09:59:00 -0500 Subject: RFR: 8240529: CheckUnhandledOops breaks NULL check in Modules::define_module In-Reply-To: References: Message-ID: +1 Coleen On 3/4/20 9:36 AM, Harold Seigel wrote: > Looks good and trivial.? Thanks for finding and fixing this. > > Harold > > On 3/4/2020 9:25 AM, Stefan Karlsson wrote: >> Hi all, >> >> Please review this small fix to CheckUnhandledOops. >> >> https://cr.openjdk.java.net/~stefank/8240529/webrev.01 >> https://bugs.openjdk.java.net/browse/JDK-8240529 >> >> 'loader' is an unhandled oop that is alive across a safepoint check >> and then is used for a NULL check. When the safepoint check executes >> the value of the oop is changed to 0xff...ff1. If the oop was NULL >> then the check now starts to fail. When this happens >> ClassLoader::_exploded_entries isn't populated as expected, causing >> class loading failures with exploded builds. >> >> Thanks, >> StefanK From coleen.phillimore at oracle.com Wed Mar 4 15:08:12 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 4 Mar 2020 10:08:12 -0500 Subject: RFR: 8240530: CheckUnhandledOops breaks BacktraceBuilder::set_has_hidden_top_frame In-Reply-To: <89b3460e-47a2-daec-bbd0-755e095d4c46@oracle.com> References: <89b3460e-47a2-daec-bbd0-755e095d4c46@oracle.com> Message-ID: <261be2cb-a8df-351f-043c-ea4b4792b754@oracle.com> Hi, I don't like the change to void* for _has_hidden_top_frame. Can you change this line to ??? if (_has_hidden_top_frame == _methods) { instead? Can you add an 'e' to Therefor ????? // It would be nice to add java/lang/Boolean::TRUE here ????? // to indicate that this backtrace has a hidden top frame. ????? // But this code is used before TRUE is allocated. ????? // Therefor let's just use an arbitrary legal oop ????? // available right here. We only test for != null ????? // anyways. _methods is a short[]. The reason that it's an oop is because the _backtrace is an objArrayOop of java.lang.Object[6], and hidden boolean is Object[5].? It's not a nice data structure but it needs to be fast. Coleen On 3/4/20 9:48 AM, Stefan Karlsson wrote: > Hi all, > > Please review this patch to get rid of a unhandled oop null check in > the BacktraceBuilder. > > https://cr.openjdk.java.net/~stefank/8240530/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8240530 > > The patch changes the type from oop to void* to prevent NULL oops from > being overwritten with non-NULL values. > > I think an alternative fix would be to change the type of > _has_hidden_top_frame to bool. Maybe someone familiar with this code > knows why this was encoded with an oop NULL check instead of a plain > bool? > > Testing with tier1-3 with CheckUnhandledOops on by default. > > StefanK From ioi.lam at oracle.com Wed Mar 4 18:15:08 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 4 Mar 2020 10:15:08 -0800 Subject: RFR(S) 8240481: Remove CDS usage of InstanceKlass::is_in_error_state In-Reply-To: References: Message-ID: <7f91e05c-30c6-2b57-6284-b72b032740bd@oracle.com> Hi Calvin, Looks good to me. Just one nit: 1131???? warn_excluded(k, "Failed verification"); I think this should be if (has_class_failed_verification(k)) { ? warn_excluded(k, "Failed verification"); } else { ? warn_excluded(k, "Not linked"); } Thanks On 3/3/20 9:23 PM, Calvin Cheung wrote: > JBS: https://bugs.openjdk.java.net/browse/JDK-8240481 > > webrev: http://cr.openjdk.java.net/~ccheung/jdk15/8240481/webrev.00/ > > Changes include: > > 1. A bool _fail_verification will be added to the > DumpTimeSharedClassInfo to indicate a class has failed verification. > ???Helper functions will be added to SystemDictionaryShared to set and > get the above bool field. > ???SystemDictionaryShared::set_class_has_failed_verification(ik) will > be called instead of ik->set_in_error_state(). > ???SystemDictionaryShared::has_class_failed_verification(ik) will be > called instead of ik->is_in_error_state(). > > > 2. The class CheckSharedClassesClosure and related code will be > removed as similar work is already done in > SystemDictionaryShared::check_excluded_classes(). > > Passed tier1 - 4 tests. > > thanks, > Calvin From calvin.cheung at oracle.com Wed Mar 4 21:47:37 2020 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Wed, 4 Mar 2020 13:47:37 -0800 Subject: RFR(S) 8240481: Remove CDS usage of InstanceKlass::is_in_error_state In-Reply-To: <7f91e05c-30c6-2b57-6284-b72b032740bd@oracle.com> References: <7f91e05c-30c6-2b57-6284-b72b032740bd@oracle.com> Message-ID: <3c78736d-0de9-52d2-5a30-751496e4fadf@oracle.com> Hi Ioi, Thanks for your review. I've updated systemDictionaryShared.cpp with your suggestions: http://cr.openjdk.java.net/~ccheung/jdk15/8240481/webrev.01/src/hotspot/share/classfile/systemDictionaryShared.cpp.sdiff.html I also added 2 asserts to the following: 1207?? if (_dumptime_table == NULL) return false; It now becomes: 1211?? if (_dumptime_table == NULL) { 1212???? assert(DynamicDumpSharedSpaces, "sanity"); 1213???? assert(ik->is_shared(), "must be a shared class in the static archive"); 1214???? return false; 1215?? } thanks, Calvin On 3/4/20 10:15 AM, Ioi Lam wrote: > Hi Calvin, > > Looks good to me. Just one nit: > > > 1131???? warn_excluded(k, "Failed verification"); > > I think this should be > > if (has_class_failed_verification(k)) { > ? warn_excluded(k, "Failed verification"); > } else { > ? warn_excluded(k, "Not linked"); > } > > Thanks > > On 3/3/20 9:23 PM, Calvin Cheung wrote: >> JBS: https://bugs.openjdk.java.net/browse/JDK-8240481 >> >> webrev: http://cr.openjdk.java.net/~ccheung/jdk15/8240481/webrev.00/ >> >> Changes include: >> >> 1. A bool _fail_verification will be added to the >> DumpTimeSharedClassInfo to indicate a class has failed verification. >> ???Helper functions will be added to SystemDictionaryShared to set >> and get the above bool field. >> ???SystemDictionaryShared::set_class_has_failed_verification(ik) will >> be called instead of ik->set_in_error_state(). >> ???SystemDictionaryShared::has_class_failed_verification(ik) will be >> called instead of ik->is_in_error_state(). >> >> >> 2. The class CheckSharedClassesClosure and related code will be >> removed as similar work is already done in >> SystemDictionaryShared::check_excluded_classes(). >> >> Passed tier1 - 4 tests. >> >> thanks, >> Calvin > From ioi.lam at oracle.com Wed Mar 4 21:56:03 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 4 Mar 2020 13:56:03 -0800 Subject: RFR: 8240197: Cannot start JVM when $JAVA_HOME includes CJK characters In-Reply-To: <71ba4bad-05c9-042d-73dc-7f492fc1728a@oss.nttdata.com> References: <6ec2f649-8999-fdd5-c2df-465925c3663b@oss.nttdata.com> <2d5148fc-7d71-d94c-d4a3-e6424a38b091@oss.nttdata.com> <37e2f672-3407-5b16-ab71-aa1e70e928db@oss.nttdata.com> <954f0bf1-91e2-3a1b-e2cb-e93d8343b0e6@oracle.com> <71ba4bad-05c9-042d-73dc-7f492fc1728a@oss.nttdata.com> Message-ID: <44d86ec7-aa6d-daf8-7200-534717cb4edb@oracle.com> Hi Yasumasa, This version looks good! [0] One thing I forgot ... is it possible to use "ResourceMark rm" in this function? If so, you can allocate the temp arrays using NEW_RESOURCE_ARRAY and then don't need to worry about freeing them. [1] for this code 4202?? // This call would be success because it is checked in above 4203?? GetFullPathNameW(unicode_path, full_path_len, full_path, NULL); How about this instead: ????? DWORD len = GetFullPathNameW(unicode_path, full_path_len, full_path, NULL); ????? assert(len <= full_path_len, "length already checked above"); (same for the MultiByteToWideChar call in convert_to_unicode) [2] This is no longer needed 4240???? os::free(buf); [3] I think the following code is not needed .... 4269?? } else { 4270???? size_t converted_path_size = sizeof(WCHAR) * (wcslen(unicode_path) + 1); 4271???? converted_path = reinterpret_cast(os::malloc(converted_path_size, mtInternal)); 4272???? if (converted_path == NULL) { 4273?????? vm_exit_out_of_memory(buf_size, OOM_MALLOC_ERROR, "wide_abs_unc_path"); 4274???? } 4275???? memcpy(converted_path, unicode_path, converted_path_size); ... if you free the original buffer inside get_full_path (or use ResourceMark): static errno_t get_full_path(LPCWSTR& unicode_path) { ?? LPCWSTR full_path = .....; ?? .... ?? free(unicode_path); unicode_path = full_path; ?? return ERROR_SUCCESS; } [4] This looks good! Much more readable than the original code! 4285?? _snwprintf(result, result_len, L"%s%s", prefix, &converted_path[prefix_off]); Thanks - Ioi On 3/4/20 12:22 AM, Yasumasa Suenaga wrote: > Hi Ioi, > > Thanks for your comment. > How about this change? > > ? http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.04/ > > This webrev adds the function both converting to unicode and getting > full path, > they would be called from wide_abs_unc_path(). > Also MemoryCleaner class which is introduced in this webrev frees > memory auomatically in d'tor. > > > Thanks, > > Yasumasa > > > On 2020/03/04 16:05, Ioi Lam wrote: >> >> >> On 3/3/20 10:22 PM, Yasumasa Suenaga wrote: >>> Hi Ioi, >>> >>>> [1] I think the memory freeing can be done using a destructor, so >>>> you can simply return in case of error, without having a lot of >>>> "if" checks: >>> >>> What do you think to use `goto` statement like Linux kernel? >>> >>> >> I don't have a strong opinion but I think many/most HotSpot >> developers are against it. Also with goto, you need to store the >> return value in a variable, but with destructors, you can just say >> "return value", so the code is cleaner. >> >> Thanks >> - Ioi >> >>> Yasumasa >>> >>> >>> On 2020/03/04 14:50, Ioi Lam wrote: >>>> Hi Yasumasa, >>>> >>>> I think this function is getting too long and it's hard to figure >>>> out all the error conditions. >>>> >>>> (I feel sorry for you because the code wasn't pretty messy to begin >>>> with, but I think it's at a point that we need to do the cleanup. >>>> It took me like 30 minutes to understand what is going on) >>>> >>>> [0] I noticed you changed the return type to LPWSTR, but all the >>>> callers are using wchar_t*. Are these two types the same? >>>> >>>> ???? If you want to use LPWSTR within this function for >>>> consistency, maybe you can return the value as >>>> >>>> ???? return (wchar_t*)result; // LPWSTR and wchat_t* are the same >>>> type on Windows. >>>> >>>> ???? That way people unfamiliar with Windows (most of us) won't be >>>> confused. >>>> >>>> [1] I think the memory freeing can be done using a destructor, so >>>> you can simply return in case of error, without having a lot of >>>> "if" checks: >>>> >>>> ???? class State { >>>> ??????? errno_t & err; >>>> ?? ????? char* buf; >>>> ???????? LPWSTR unicode_path; >>>> LPWSTR result; >>>> >>>> ???????? State(errno_t & e) : buf(NULL), unicode_path(NULL), >>>> result(NULL), err(e) { >>>> ???????????? err = SUCCESS; >>>> ????????? } >>>> ???????? ~State() { >>>> ?????????? os::free(buf); >>>> ?? ? ? ? ? os::free(unicode_path); >>>> ?????????? if (err != ERROR_SUCCESS) { >>>> ????????????? os::free(result); >>>> ?????????? } >>>> ???????? } >>>> ???????? LPWSTR as_error(errno_t e) { >>>> ???????????? err = e; >>>> ???????????? return NULL; >>>> ???????? } >>>> ???? } st; >>>> >>>> >>>> ???? if ((st.buf = static_cast(os::malloc(buf_size, >>>> mtInternal))) == NULL) { >>>> ???????? return st.as_error(ENOMEM); >>>> ???? } >>>> >>>> ???? .... >>>> ???? return st.result; >>>> >>>> [2] Will it be possible to break the functon down into smaller >>>> pieces, so it's easier to understand what's going on? >>>> >>>> E.g., Maybe the code that? converts the C string into a unicode >>>> string can be moved into a separate function? >>>> >>>> ?? // Get required buffer size to convert to Unicode >>>> ?? int unicode_path_len = MultiByteToWideChar(CP_THREAD_ACP, >>>> MB_ERR_INVALID_CHARS, >>>> ????????????????????????????????????????????? buf, -1, >>>> ????????????????????????????????????????????? NULL, 0); >>>> ?? if (unicode_path_len == 0) { >>>> ???? os::free(buf); >>>> ???? err = EINVAL; >>>> ???? return NULL; >>>> ?? } >>>> >>>> ?? .... >>>> >>>> ?? LPWSTR unicode_path = >>>> static_cast(os::malloc(sizeof(WCHAR) * unicode_path_len, >>>> mtInternal)); >>>> ?? if (unicode_path == NULL) { >>>> ???? err = ENOMEM; >>>> ?? } else { >>>> ???? // This call would be success because it is checked in above >>>> ???? err = ERROR_SUCCESS; >>>> ???? MultiByteToWideChar(CP_THREAD_ACP, >>>> ???????????????????????? MB_ERR_INVALID_CHARS, >>>> ???????????????????????? buf, -1, >>>> ???????????????????????? unicode_path, unicode_path_len); >>>> >>>> >>>> >>>> [3] size_t result_len = prefix_len - prefix_off + additional_space; >>>> >>>> The variable name result_len is confusing. It's not the length of >>>> the result. >>>> >>>> [4] The old copying code was already hard to understand, and the >>>> new version now has 2 versions of the copying code. I think the >>>> reason is you want to avoid an extra memcpy by calling >>>> GetFullPathNameW directly on the result. >>>> >>>> ???? if (needs_fullpath) { >>>> ?????? // Get required buffer size to convert to full path >>>> ?????? DWORD full_path_len = GetFullPathNameW(unicode_path, 0, >>>> NULL, NULL); >>>> ?????? if (full_path_len == 0) { >>>> ???????? err = EINVAL; >>>> ?????? } else { >>>> ???????? size_t result_size = sizeof(WCHAR) * (result_len + >>>> full_path_len); >>>> ???????? result = static_cast(os::malloc(result_size, >>>> mtInternal)); >>>> ???????? if (result == NULL) { >>>> ?????????? err = ENOMEM; >>>> ???????? } else { >>>> ?????????? // This call would be success because it is checked in >>>> above >>>> ?????????? GetFullPathNameW(unicode_path, full_path_len, result + >>>> prefix_len - prefix_off, NULL); >>>> ?????????? // Copy prefix >>>> ?????????? memcpy(result, prefix, sizeof(WCHAR) * prefix_len); >>>> ???????? } >>>> ?????? } >>>> ???? } else { >>>> ?????? size_t result_size = sizeof(WCHAR) * (result_len + >>>> unicode_path_len); >>>> ?????? result = static_cast(os::malloc(result_size, >>>> mtInternal)); >>>> ?????? if (result == NULL) { >>>> ???????? err = ENOMEM; >>>> ?????? } else { >>>> ???????? // Copy unicode path >>>> ???????? memcpy(result + prefix_len - prefix_off, unicode_path, >>>> sizeof(WCHAR) * unicode_path_len); >>>> ???????? // Copy prefix >>>> ???????? memcpy(result, prefix, sizeof(WCHAR) * prefix_len); >>>> ?????? } >>>> ???? } >>>> >>>> I think the code can be simplified if you do the fullpath version >>>> in a separate function: >>>> >>>> ???? if (needs_fullpath) { >>>> ??????? unicode_path = to_full_path(unicode_path, err, >>>> &unicode_path_len); >>>> ???? } >>>> >>>> ???? size_t result_size = ...... >>>> >>>> This will do one more memcpy, but I think it's not performance >>>> critical >>>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>> On 3/3/20 4:19 PM, Yasumasa Suenaga wrote: >>>>> Thanks Ralf! >>>>> I fixed them. >>>>> >>>>> I need a Reviewer to push, so I uploaded new webrev: >>>>> >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.03/ >>>>> >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2020/03/03 23:25, Schmelter, Ralf wrote: >>>>>> Hi Yasumasa, >>>>>> >>>>>> thanks for your work. The code should now be able to handle any >>>>>> path. >>>>>> >>>>>> Two small changes (no new webrev needed): >>>>>> >>>>>> size_t result_len = prefix_len + prefix_off + additional_space >>>>>> should be >>>>>> size_t result_len = prefix_len - prefix_off + additional_space >>>>>> >>>>>> And you should copy the prefix after you copied the filename, >>>>>> since sometimes the prefix is required to overwrite the start of >>>>>> the filename: >>>>>> >>>>>> // This call would be success because it is checked in above >>>>>> GetFullPathNameW(unicode_path, full_path_len, result + prefix_len >>>>>> - prefix_off, NULL); >>>>>> // Copy prefix >>>>>> memcpy(result, prefix, sizeof(WCHAR) * prefix_len); >>>>>> >>>>>> and >>>>>> >>>>>> // Copy unicode path >>>>>> memcpy(result + prefix_len - prefix_off, unicode_path, >>>>>> sizeof(WCHAR) * unicode_path_len); >>>>>> // Copy prefix >>>>>> memcpy(result, prefix, sizeof(WCHAR) * prefix_len); >>>>>> >>>>>> With these changes the os_windows gtest and the appcds jtreg >>>>>> tests run without errors. >>>>>> >>>>>> Best regards, >>>>>> Ralf >>>>>> >>>> >> From ioi.lam at oracle.com Wed Mar 4 21:57:00 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 4 Mar 2020 13:57:00 -0800 Subject: RFR(S) 8240481: Remove CDS usage of InstanceKlass::is_in_error_state In-Reply-To: <3c78736d-0de9-52d2-5a30-751496e4fadf@oracle.com> References: <7f91e05c-30c6-2b57-6284-b72b032740bd@oracle.com> <3c78736d-0de9-52d2-5a30-751496e4fadf@oracle.com> Message-ID: <6ec1710f-6533-d3f3-145f-ef5dbfb6a79e@oracle.com> Looks good to me. Thanks - Ioi On 3/4/20 1:47 PM, Calvin Cheung wrote: > Hi Ioi, > > Thanks for your review. > > I've updated systemDictionaryShared.cpp with your suggestions: > > http://cr.openjdk.java.net/~ccheung/jdk15/8240481/webrev.01/src/hotspot/share/classfile/systemDictionaryShared.cpp.sdiff.html > > > I also added 2 asserts to the following: > > 1207?? if (_dumptime_table == NULL) return false; > > It now becomes: > > 1211?? if (_dumptime_table == NULL) { > 1212???? assert(DynamicDumpSharedSpaces, "sanity"); > 1213???? assert(ik->is_shared(), "must be a shared class in the static > archive"); > 1214???? return false; > 1215?? } > > thanks, > > Calvin > > On 3/4/20 10:15 AM, Ioi Lam wrote: >> Hi Calvin, >> >> Looks good to me. Just one nit: >> >> >> 1131???? warn_excluded(k, "Failed verification"); >> >> I think this should be >> >> if (has_class_failed_verification(k)) { >> ? warn_excluded(k, "Failed verification"); >> } else { >> ? warn_excluded(k, "Not linked"); >> } >> >> Thanks >> >> On 3/3/20 9:23 PM, Calvin Cheung wrote: >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8240481 >>> >>> webrev: http://cr.openjdk.java.net/~ccheung/jdk15/8240481/webrev.00/ >>> >>> Changes include: >>> >>> 1. A bool _fail_verification will be added to the >>> DumpTimeSharedClassInfo to indicate a class has failed verification. >>> ???Helper functions will be added to SystemDictionaryShared to set >>> and get the above bool field. >>> ???SystemDictionaryShared::set_class_has_failed_verification(ik) >>> will be called instead of ik->set_in_error_state(). >>> ???SystemDictionaryShared::has_class_failed_verification(ik) will be >>> called instead of ik->is_in_error_state(). >>> >>> >>> 2. The class CheckSharedClassesClosure and related code will be >>> removed as similar work is already done in >>> SystemDictionaryShared::check_excluded_classes(). >>> >>> Passed tier1 - 4 tests. >>> >>> thanks, >>> Calvin >> From yumin.qi at oracle.com Wed Mar 4 21:57:58 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Wed, 4 Mar 2020 13:57:58 -0800 Subject: RFR(S) 8240481: Remove CDS usage of InstanceKlass::is_in_error_state In-Reply-To: <3c78736d-0de9-52d2-5a30-751496e4fadf@oracle.com> References: <7f91e05c-30c6-2b57-6284-b72b032740bd@oracle.com> <3c78736d-0de9-52d2-5a30-751496e4fadf@oracle.com> Message-ID: Hi, Calvin ? Looks good to me. Thanks Yumin On 3/4/20 1:47 PM, Calvin Cheung wrote: > Hi Ioi, > > Thanks for your review. > > I've updated systemDictionaryShared.cpp with your suggestions: > > http://cr.openjdk.java.net/~ccheung/jdk15/8240481/webrev.01/src/hotspot/share/classfile/systemDictionaryShared.cpp.sdiff.html > > > I also added 2 asserts to the following: > > 1207?? if (_dumptime_table == NULL) return false; > > It now becomes: > > 1211?? if (_dumptime_table == NULL) { > 1212???? assert(DynamicDumpSharedSpaces, "sanity"); > 1213???? assert(ik->is_shared(), "must be a shared class in the static > archive"); > 1214???? return false; > 1215?? } > > thanks, > > Calvin > > On 3/4/20 10:15 AM, Ioi Lam wrote: >> Hi Calvin, >> >> Looks good to me. Just one nit: >> >> >> 1131???? warn_excluded(k, "Failed verification"); >> >> I think this should be >> >> if (has_class_failed_verification(k)) { >> ? warn_excluded(k, "Failed verification"); >> } else { >> ? warn_excluded(k, "Not linked"); >> } >> >> Thanks >> >> On 3/3/20 9:23 PM, Calvin Cheung wrote: >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8240481 >>> >>> webrev: http://cr.openjdk.java.net/~ccheung/jdk15/8240481/webrev.00/ >>> >>> Changes include: >>> >>> 1. A bool _fail_verification will be added to the >>> DumpTimeSharedClassInfo to indicate a class has failed verification. >>> ???Helper functions will be added to SystemDictionaryShared to set >>> and get the above bool field. >>> ???SystemDictionaryShared::set_class_has_failed_verification(ik) >>> will be called instead of ik->set_in_error_state(). >>> ???SystemDictionaryShared::has_class_failed_verification(ik) will be >>> called instead of ik->is_in_error_state(). >>> >>> >>> 2. The class CheckSharedClassesClosure and related code will be >>> removed as similar work is already done in >>> SystemDictionaryShared::check_excluded_classes(). >>> >>> Passed tier1 - 4 tests. >>> >>> thanks, >>> Calvin >> From calvin.cheung at oracle.com Wed Mar 4 22:07:08 2020 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Wed, 4 Mar 2020 14:07:08 -0800 Subject: RFR(S) 8240481: Remove CDS usage of InstanceKlass::is_in_error_state In-Reply-To: References: <7f91e05c-30c6-2b57-6284-b72b032740bd@oracle.com> <3c78736d-0de9-52d2-5a30-751496e4fadf@oracle.com> Message-ID: Thanks Yumin. Calvin On 3/4/20 1:57 PM, Yumin Qi wrote: > > Hi, Calvin > > > ? Looks good to me. > > Thanks > > Yumin > > On 3/4/20 1:47 PM, Calvin Cheung wrote: >> Hi Ioi, >> >> Thanks for your review. >> >> I've updated systemDictionaryShared.cpp with your suggestions: >> >> http://cr.openjdk.java.net/~ccheung/jdk15/8240481/webrev.01/src/hotspot/share/classfile/systemDictionaryShared.cpp.sdiff.html >> >> >> I also added 2 asserts to the following: >> >> 1207?? if (_dumptime_table == NULL) return false; >> >> It now becomes: >> >> 1211?? if (_dumptime_table == NULL) { >> 1212???? assert(DynamicDumpSharedSpaces, "sanity"); >> 1213???? assert(ik->is_shared(), "must be a shared class in the >> static archive"); >> 1214???? return false; >> 1215?? } >> >> thanks, >> >> Calvin >> >> On 3/4/20 10:15 AM, Ioi Lam wrote: >>> Hi Calvin, >>> >>> Looks good to me. Just one nit: >>> >>> >>> 1131???? warn_excluded(k, "Failed verification"); >>> >>> I think this should be >>> >>> if (has_class_failed_verification(k)) { >>> ? warn_excluded(k, "Failed verification"); >>> } else { >>> ? warn_excluded(k, "Not linked"); >>> } >>> >>> Thanks >>> >>> On 3/3/20 9:23 PM, Calvin Cheung wrote: >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8240481 >>>> >>>> webrev: http://cr.openjdk.java.net/~ccheung/jdk15/8240481/webrev.00/ >>>> >>>> Changes include: >>>> >>>> 1. A bool _fail_verification will be added to the >>>> DumpTimeSharedClassInfo to indicate a class has failed verification. >>>> ???Helper functions will be added to SystemDictionaryShared to set >>>> and get the above bool field. >>>> ???SystemDictionaryShared::set_class_has_failed_verification(ik) >>>> will be called instead of ik->set_in_error_state(). >>>> ???SystemDictionaryShared::has_class_failed_verification(ik) will >>>> be called instead of ik->is_in_error_state(). >>>> >>>> >>>> 2. The class CheckSharedClassesClosure and related code will be >>>> removed as similar work is already done in >>>> SystemDictionaryShared::check_excluded_classes(). >>>> >>>> Passed tier1 - 4 tests. >>>> >>>> thanks, >>>> Calvin >>> From calvin.cheung at oracle.com Wed Mar 4 22:07:58 2020 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Wed, 4 Mar 2020 14:07:58 -0800 Subject: RFR(S) 8240481: Remove CDS usage of InstanceKlass::is_in_error_state In-Reply-To: <6ec1710f-6533-d3f3-145f-ef5dbfb6a79e@oracle.com> References: <7f91e05c-30c6-2b57-6284-b72b032740bd@oracle.com> <3c78736d-0de9-52d2-5a30-751496e4fadf@oracle.com> <6ec1710f-6533-d3f3-145f-ef5dbfb6a79e@oracle.com> Message-ID: <066fbe19-b3a5-e854-3485-d57764c1e788@oracle.com> Thanks Ioi. Calvin On 3/4/20 1:57 PM, Ioi Lam wrote: > Looks good to me. > > Thanks > - Ioi > > On 3/4/20 1:47 PM, Calvin Cheung wrote: >> Hi Ioi, >> >> Thanks for your review. >> >> I've updated systemDictionaryShared.cpp with your suggestions: >> >> http://cr.openjdk.java.net/~ccheung/jdk15/8240481/webrev.01/src/hotspot/share/classfile/systemDictionaryShared.cpp.sdiff.html >> >> >> I also added 2 asserts to the following: >> >> 1207?? if (_dumptime_table == NULL) return false; >> >> It now becomes: >> >> 1211?? if (_dumptime_table == NULL) { >> 1212???? assert(DynamicDumpSharedSpaces, "sanity"); >> 1213???? assert(ik->is_shared(), "must be a shared class in the >> static archive"); >> 1214???? return false; >> 1215?? } >> >> thanks, >> >> Calvin >> >> On 3/4/20 10:15 AM, Ioi Lam wrote: >>> Hi Calvin, >>> >>> Looks good to me. Just one nit: >>> >>> >>> 1131???? warn_excluded(k, "Failed verification"); >>> >>> I think this should be >>> >>> if (has_class_failed_verification(k)) { >>> ? warn_excluded(k, "Failed verification"); >>> } else { >>> ? warn_excluded(k, "Not linked"); >>> } >>> >>> Thanks >>> >>> On 3/3/20 9:23 PM, Calvin Cheung wrote: >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8240481 >>>> >>>> webrev: http://cr.openjdk.java.net/~ccheung/jdk15/8240481/webrev.00/ >>>> >>>> Changes include: >>>> >>>> 1. A bool _fail_verification will be added to the >>>> DumpTimeSharedClassInfo to indicate a class has failed verification. >>>> ???Helper functions will be added to SystemDictionaryShared to set >>>> and get the above bool field. >>>> ???SystemDictionaryShared::set_class_has_failed_verification(ik) >>>> will be called instead of ik->set_in_error_state(). >>>> ???SystemDictionaryShared::has_class_failed_verification(ik) will >>>> be called instead of ik->is_in_error_state(). >>>> >>>> >>>> 2. The class CheckSharedClassesClosure and related code will be >>>> removed as similar work is already done in >>>> SystemDictionaryShared::check_excluded_classes(). >>>> >>>> Passed tier1 - 4 tests. >>>> >>>> thanks, >>>> Calvin >>> > From suenaga at oss.nttdata.com Thu Mar 5 00:18:21 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Thu, 5 Mar 2020 09:18:21 +0900 Subject: RFR: 8240197: Cannot start JVM when $JAVA_HOME includes CJK characters In-Reply-To: <44d86ec7-aa6d-daf8-7200-534717cb4edb@oracle.com> References: <6ec2f649-8999-fdd5-c2df-465925c3663b@oss.nttdata.com> <2d5148fc-7d71-d94c-d4a3-e6424a38b091@oss.nttdata.com> <37e2f672-3407-5b16-ab71-aa1e70e928db@oss.nttdata.com> <954f0bf1-91e2-3a1b-e2cb-e93d8343b0e6@oracle.com> <71ba4bad-05c9-042d-73dc-7f492fc1728a@oss.nttdata.com> <44d86ec7-aa6d-daf8-7200-534717cb4edb@oracle.com> Message-ID: <25aa10eb-6a8f-b416-02c2-b3fcd140076e@oss.nttdata.com> Hi Ioi, How about this change? http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.05/ I already tried to use ResouceMark, but it was crashed because wide_abs_unc_path() would be called from outside of java thread at boot time. This webrev uses os::free() instead of memory cleaner class, but I think it is simple because I tweaked memory lifecycle. Thanks, Yasumasa On 2020/03/05 6:56, Ioi Lam wrote: > Hi Yasumasa, > > This version looks good! > > > [0] One thing I forgot ... is it possible to use "ResourceMark rm" in this function? If so, you can allocate the temp arrays using NEW_RESOURCE_ARRAY and then don't need to worry about freeing them. > > > [1] for this code > > 4202?? // This call would be success because it is checked in above > 4203?? GetFullPathNameW(unicode_path, full_path_len, full_path, NULL); > > How about this instead: > > ????? DWORD len = GetFullPathNameW(unicode_path, full_path_len, full_path, NULL); > ????? assert(len <= full_path_len, "length already checked above"); > > (same for the MultiByteToWideChar call in convert_to_unicode) > > > [2] This is no longer needed > > 4240???? os::free(buf); > > > [3] I think the following code is not needed .... > > 4269?? } else { > 4270???? size_t converted_path_size = sizeof(WCHAR) * (wcslen(unicode_path) + 1); > 4271???? converted_path = reinterpret_cast(os::malloc(converted_path_size, mtInternal)); > 4272???? if (converted_path == NULL) { > 4273?????? vm_exit_out_of_memory(buf_size, OOM_MALLOC_ERROR, "wide_abs_unc_path"); > 4274???? } > 4275???? memcpy(converted_path, unicode_path, converted_path_size); > > > ... if you free the original buffer inside get_full_path (or use ResourceMark): > > static errno_t get_full_path(LPCWSTR& unicode_path) { > ?? LPCWSTR full_path = .....; > ?? .... > ?? free(unicode_path); > unicode_path = full_path; > ?? return ERROR_SUCCESS; > } > > [4] This looks good! Much more readable than the original code! > > 4285?? _snwprintf(result, result_len, L"%s%s", prefix, &converted_path[prefix_off]); > > > Thanks > - Ioi > > > On 3/4/20 12:22 AM, Yasumasa Suenaga wrote: >> Hi Ioi, >> >> Thanks for your comment. >> How about this change? >> >> ? http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.04/ >> >> This webrev adds the function both converting to unicode and getting full path, >> they would be called from wide_abs_unc_path(). >> Also MemoryCleaner class which is introduced in this webrev frees memory auomatically in d'tor. >> >> >> Thanks, >> >> Yasumasa >> >> >> On 2020/03/04 16:05, Ioi Lam wrote: >>> >>> >>> On 3/3/20 10:22 PM, Yasumasa Suenaga wrote: >>>> Hi Ioi, >>>> >>>>> [1] I think the memory freeing can be done using a destructor, so you can simply return in case of error, without having a lot of "if" checks: >>>> >>>> What do you think to use `goto` statement like Linux kernel? >>>> >>>> >>> I don't have a strong opinion but I think many/most HotSpot developers are against it. Also with goto, you need to store the return value in a variable, but with destructors, you can just say "return value", so the code is cleaner. >>> >>> Thanks >>> - Ioi >>> >>>> Yasumasa >>>> >>>> >>>> On 2020/03/04 14:50, Ioi Lam wrote: >>>>> Hi Yasumasa, >>>>> >>>>> I think this function is getting too long and it's hard to figure out all the error conditions. >>>>> >>>>> (I feel sorry for you because the code wasn't pretty messy to begin with, but I think it's at a point that we need to do the cleanup. It took me like 30 minutes to understand what is going on) >>>>> >>>>> [0] I noticed you changed the return type to LPWSTR, but all the callers are using wchar_t*. Are these two types the same? >>>>> >>>>> ???? If you want to use LPWSTR within this function for consistency, maybe you can return the value as >>>>> >>>>> ???? return (wchar_t*)result; // LPWSTR and wchat_t* are the same type on Windows. >>>>> >>>>> ???? That way people unfamiliar with Windows (most of us) won't be confused. >>>>> >>>>> [1] I think the memory freeing can be done using a destructor, so you can simply return in case of error, without having a lot of "if" checks: >>>>> >>>>> ???? class State { >>>>> ??????? errno_t & err; >>>>> ?? ????? char* buf; >>>>> ???????? LPWSTR unicode_path; >>>>> LPWSTR result; >>>>> >>>>> ???????? State(errno_t & e) : buf(NULL), unicode_path(NULL), result(NULL), err(e) { >>>>> ???????????? err = SUCCESS; >>>>> ????????? } >>>>> ???????? ~State() { >>>>> ?????????? os::free(buf); >>>>> ?? ? ? ? ? os::free(unicode_path); >>>>> ?????????? if (err != ERROR_SUCCESS) { >>>>> ????????????? os::free(result); >>>>> ?????????? } >>>>> ???????? } >>>>> ???????? LPWSTR as_error(errno_t e) { >>>>> ???????????? err = e; >>>>> ???????????? return NULL; >>>>> ???????? } >>>>> ???? } st; >>>>> >>>>> >>>>> ???? if ((st.buf = static_cast(os::malloc(buf_size, mtInternal))) == NULL) { >>>>> ???????? return st.as_error(ENOMEM); >>>>> ???? } >>>>> >>>>> ???? .... >>>>> ???? return st.result; >>>>> >>>>> [2] Will it be possible to break the functon down into smaller pieces, so it's easier to understand what's going on? >>>>> >>>>> E.g., Maybe the code that? converts the C string into a unicode string can be moved into a separate function? >>>>> >>>>> ?? // Get required buffer size to convert to Unicode >>>>> ?? int unicode_path_len = MultiByteToWideChar(CP_THREAD_ACP, >>>>> MB_ERR_INVALID_CHARS, >>>>> ????????????????????????????????????????????? buf, -1, >>>>> ????????????????????????????????????????????? NULL, 0); >>>>> ?? if (unicode_path_len == 0) { >>>>> ???? os::free(buf); >>>>> ???? err = EINVAL; >>>>> ???? return NULL; >>>>> ?? } >>>>> >>>>> ?? .... >>>>> >>>>> ?? LPWSTR unicode_path = static_cast(os::malloc(sizeof(WCHAR) * unicode_path_len, mtInternal)); >>>>> ?? if (unicode_path == NULL) { >>>>> ???? err = ENOMEM; >>>>> ?? } else { >>>>> ???? // This call would be success because it is checked in above >>>>> ???? err = ERROR_SUCCESS; >>>>> ???? MultiByteToWideChar(CP_THREAD_ACP, >>>>> ???????????????????????? MB_ERR_INVALID_CHARS, >>>>> ???????????????????????? buf, -1, >>>>> ???????????????????????? unicode_path, unicode_path_len); >>>>> >>>>> >>>>> >>>>> [3] size_t result_len = prefix_len - prefix_off + additional_space; >>>>> >>>>> The variable name result_len is confusing. It's not the length of the result. >>>>> >>>>> [4] The old copying code was already hard to understand, and the new version now has 2 versions of the copying code. I think the reason is you want to avoid an extra memcpy by calling GetFullPathNameW directly on the result. >>>>> >>>>> ???? if (needs_fullpath) { >>>>> ?????? // Get required buffer size to convert to full path >>>>> ?????? DWORD full_path_len = GetFullPathNameW(unicode_path, 0, NULL, NULL); >>>>> ?????? if (full_path_len == 0) { >>>>> ???????? err = EINVAL; >>>>> ?????? } else { >>>>> ???????? size_t result_size = sizeof(WCHAR) * (result_len + full_path_len); >>>>> ???????? result = static_cast(os::malloc(result_size, mtInternal)); >>>>> ???????? if (result == NULL) { >>>>> ?????????? err = ENOMEM; >>>>> ???????? } else { >>>>> ?????????? // This call would be success because it is checked in above >>>>> ?????????? GetFullPathNameW(unicode_path, full_path_len, result + prefix_len - prefix_off, NULL); >>>>> ?????????? // Copy prefix >>>>> ?????????? memcpy(result, prefix, sizeof(WCHAR) * prefix_len); >>>>> ???????? } >>>>> ?????? } >>>>> ???? } else { >>>>> ?????? size_t result_size = sizeof(WCHAR) * (result_len + unicode_path_len); >>>>> ?????? result = static_cast(os::malloc(result_size, mtInternal)); >>>>> ?????? if (result == NULL) { >>>>> ???????? err = ENOMEM; >>>>> ?????? } else { >>>>> ???????? // Copy unicode path >>>>> ???????? memcpy(result + prefix_len - prefix_off, unicode_path, sizeof(WCHAR) * unicode_path_len); >>>>> ???????? // Copy prefix >>>>> ???????? memcpy(result, prefix, sizeof(WCHAR) * prefix_len); >>>>> ?????? } >>>>> ???? } >>>>> >>>>> I think the code can be simplified if you do the fullpath version in a separate function: >>>>> >>>>> ???? if (needs_fullpath) { >>>>> ??????? unicode_path = to_full_path(unicode_path, err, &unicode_path_len); >>>>> ???? } >>>>> >>>>> ???? size_t result_size = ...... >>>>> >>>>> This will do one more memcpy, but I think it's not performance critical >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> >>>>> On 3/3/20 4:19 PM, Yasumasa Suenaga wrote: >>>>>> Thanks Ralf! >>>>>> I fixed them. >>>>>> >>>>>> I need a Reviewer to push, so I uploaded new webrev: >>>>>> >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.03/ >>>>>> >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2020/03/03 23:25, Schmelter, Ralf wrote: >>>>>>> Hi Yasumasa, >>>>>>> >>>>>>> thanks for your work. The code should now be able to handle any path. >>>>>>> >>>>>>> Two small changes (no new webrev needed): >>>>>>> >>>>>>> size_t result_len = prefix_len + prefix_off + additional_space >>>>>>> should be >>>>>>> size_t result_len = prefix_len - prefix_off + additional_space >>>>>>> >>>>>>> And you should copy the prefix after you copied the filename, since sometimes the prefix is required to overwrite the start of the filename: >>>>>>> >>>>>>> // This call would be success because it is checked in above >>>>>>> GetFullPathNameW(unicode_path, full_path_len, result + prefix_len - prefix_off, NULL); >>>>>>> // Copy prefix >>>>>>> memcpy(result, prefix, sizeof(WCHAR) * prefix_len); >>>>>>> >>>>>>> and >>>>>>> >>>>>>> // Copy unicode path >>>>>>> memcpy(result + prefix_len - prefix_off, unicode_path, sizeof(WCHAR) * unicode_path_len); >>>>>>> // Copy prefix >>>>>>> memcpy(result, prefix, sizeof(WCHAR) * prefix_len); >>>>>>> >>>>>>> With these changes the os_windows gtest and the appcds jtreg tests run without errors. >>>>>>> >>>>>>> Best regards, >>>>>>> Ralf >>>>>>> >>>>> >>> > From david.holmes at oracle.com Thu Mar 5 00:57:35 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 5 Mar 2020 10:57:35 +1000 Subject: RFR(S): 8240295: hs_err elapsed time in seconds is not accurate enough In-Reply-To: <053864a8-617f-fcac-fc26-220d161e3e55@oracle.com> References: <4a3d7c8e-df9e-51de-77e4-775be1480392@oracle.com> <92ada19d-7200-6bb7-b174-9da8a166aea7@oracle.com> <053864a8-617f-fcac-fc26-220d161e3e55@oracle.com> Message-ID: On 4/03/2020 8:44 am, Kevin Walls wrote: > > Thanks David - > > Yes there are situations where hs_err fails, and few people are sadder > than me > when that happens 8-) , so I was thinking about how scared to be by the > comment. > > With the safety net of the error handler for the steps of the hs_err file > (which works well, we see it invoked frequently), it looks reasonable to > use > %f as we might do other slightly questionable things for a signal handler. > > Corrupting locale information or floating point state might possibly cause > problems, but if I cause a fake crash in print_date_and_time the error > handler recovers and the report continues. That is good to know. > Thinking about printing with two ints, seconds and fractions: > I don't see anything already that returns such a time in two components > in the > JVM, so we might implement a new form of os::javaTimeNanos() or similar > that > returns the two parts, and do that on each platform. I was thinking of something simple/crude ... > I didn't yet come up with anything to do in os::print_date_and_time() > which will take the fractional part of the double, and print just the > fraction as an int, without using any library / %f facilities. ... just using e.g. (untested) double t = os::elapsedTime(); int secs = (int) t; int micros = (int)((t - secs) * 100000); printf("%d.%d", secs, micros); with appropriate width specifiers to get the formatting right. Cheers, David > > If you're still concerned I could revisit these or some other idea. > > Genuine laugh out loud moment for me, I backported the elapsed time > logging from > 6u4 to 5u19? (https://bugs.openjdk.java.net/browse/JDK-6447157) (2007). > (I said before jdk5 was created, I should have said before it was in > mercurial.) > > Thanks > Kevin > > > On 03/03/2020 01:11, David Holmes wrote: >> Hi Kevin, >> >> On 2/03/2020 8:48 pm, Kevin Walls wrote: >>> Oops, and with the bug ID in the title and JBS link: >>> https://bugs.openjdk.java.net/browse/JDK-8240295 >>> >>> >>> On 02/03/2020 10:47, Kevin Walls wrote: >>>> Hi, >>>> >>>> (s11y and runtime opinions both relevant) >>>> >>>> A few times in the last month I've really wanted to compare the >>>> Events logged in the hs_err file, and the time of the JVM's crash. >>>> >>>> "elapsed time" in hs_err is only accurate to one second, and has >>>> been since before jdk5 was created. >>>> >>>> The diff below changes the format string and uses the non-rounded >>>> time value (I don't see a need to change the other integer >>>> arithmetic here), and we can enjoy hs_errs with detail like: >>>> >>>> ... >>>> Time: Mon Mar? 2 09:57:13 2020 UTC elapsed time: 5.490135 seconds >>>> (0d 0h 0m 5s) >>>> ... >>>> >>>> Thanks >>>> Kevin >>>> >>>> >>>> /jdk/open$ hg diff >>>> diff --git a/src/hotspot/share/runtime/os.cpp >>>> b/src/hotspot/share/runtime/os.cpp >>>> --- a/src/hotspot/share/runtime/os.cpp >>>> +++ b/src/hotspot/share/runtime/os.cpp >>>> @@ -1016,9 +1016,8 @@ >>>> ?? } >>>> >>>> ?? double t = os::elapsedTime(); >>>> -? // NOTE: It tends to crash after a SEGV if we want to >>>> printf("%f",...) in >>>> -? //?????? Linux. Must be a bug in glibc ? Workaround is to round >>>> "t" to int >>>> -? //?????? before printf. We lost some precision, but who cares? >>>> +? // NOTE: a crash using printf("%f",...) on Linux was historically >>>> noted here >>>> +? //?????? (before the jdk5 repo was created). >> >> Just because it is old doesn't mean it no longer applies. printf is >> not async-signal-safe - we know that but we try to use it anyway. >> Maybe %f is even less async-signal-safe? >> >> This may get through testing okay but cause problems with real crashes >> in the field. >> >> What about breaking the time up into two ints: seconds and nanos? >> >> Cheers, >> David >> ----- >> >>>> ?? int eltime = (int)t;? // elapsed time in seconds >>>> >>>> ?? // print elapsed time in a human-readable format: >>>> @@ -1029,7 +1028,7 @@ >>>> ?? int elmins = (eltime - day_secs - hour_secs) / secs_per_min; >>>> ?? int minute_secs = elmins * secs_per_min; >>>> ?? int elsecs = (eltime - day_secs - hour_secs - minute_secs); >>>> -? st->print_cr(" elapsed time: %d seconds (%dd %dh %dm %ds)", >>>> eltime, eldays, elhours, elmins, elsecs); >>>> +? st->print_cr(" elapsed time: %f seconds (%dd %dh %dm %ds)", t, >>>> eldays, elhours, elmins, elsecs); >>>> ?} >>>> >>>> From ioi.lam at oracle.com Thu Mar 5 04:15:30 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 4 Mar 2020 20:15:30 -0800 Subject: RFR: 8240197: Cannot start JVM when $JAVA_HOME includes CJK characters In-Reply-To: <25aa10eb-6a8f-b416-02c2-b3fcd140076e@oss.nttdata.com> References: <6ec2f649-8999-fdd5-c2df-465925c3663b@oss.nttdata.com> <2d5148fc-7d71-d94c-d4a3-e6424a38b091@oss.nttdata.com> <37e2f672-3407-5b16-ab71-aa1e70e928db@oss.nttdata.com> <954f0bf1-91e2-3a1b-e2cb-e93d8343b0e6@oracle.com> <71ba4bad-05c9-042d-73dc-7f492fc1728a@oss.nttdata.com> <44d86ec7-aa6d-daf8-7200-534717cb4edb@oracle.com> <25aa10eb-6a8f-b416-02c2-b3fcd140076e@oss.nttdata.com> Message-ID: <4997774e-f009-a6e0-b4df-8c1dd3a4dd17@oracle.com> Hi Yasumasa, This looks very good. Thanks for bearing with my requests :-) The return value of GetFullPathNameW is a bit weird. I would suggest adding comments so people can understand the code without reading the GetFullPathNameW documentation: 4190?? // Get required buffer size to convert to full path. The return ?????? // value INCLUDES the terminating null character. 4191?? DWORD full_path_len = GetFullPathNameW(unicode_path, 0, NULL, NULL); ?????? // When the buffer has sufficient size, the return value EXCLUDES the ?????? // terminating null character. 4202?? DWORD result = GetFullPathNameW(unicode_path, full_path_len, full_path, NULL); 4203?? assert(result == (full_path_len - 1), "length already checked above"); (No need for new webrev). Thanks - Ioi On 3/4/20 4:18 PM, Yasumasa Suenaga wrote: > Hi Ioi, > > How about this change? > > ? http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.05/ > > I already tried to use ResouceMark, but it was crashed because > wide_abs_unc_path() would be called from outside of java thread at > boot time. > This webrev uses os::free() instead of memory cleaner class, but I > think it is simple because I tweaked memory lifecycle. > > > Thanks, > > Yasumasa > > > On 2020/03/05 6:56, Ioi Lam wrote: >> Hi Yasumasa, >> >> This version looks good! >> >> >> [0] One thing I forgot ... is it possible to use "ResourceMark rm" in >> this function? If so, you can allocate the temp arrays using >> NEW_RESOURCE_ARRAY and then don't need to worry about freeing them. >> >> >> [1] for this code >> >> 4202?? // This call would be success because it is checked in above >> 4203?? GetFullPathNameW(unicode_path, full_path_len, full_path, NULL); >> >> How about this instead: >> >> ?????? DWORD len = GetFullPathNameW(unicode_path, full_path_len, >> full_path, NULL); >> ?????? assert(len <= full_path_len, "length already checked above"); >> >> (same for the MultiByteToWideChar call in convert_to_unicode) >> >> >> [2] This is no longer needed >> >> 4240???? os::free(buf); >> >> >> [3] I think the following code is not needed .... >> >> 4269?? } else { >> 4270???? size_t converted_path_size = sizeof(WCHAR) * >> (wcslen(unicode_path) + 1); >> 4271???? converted_path = >> reinterpret_cast(os::malloc(converted_path_size, mtInternal)); >> 4272???? if (converted_path == NULL) { >> 4273?????? vm_exit_out_of_memory(buf_size, OOM_MALLOC_ERROR, >> "wide_abs_unc_path"); >> 4274???? } >> 4275???? memcpy(converted_path, unicode_path, converted_path_size); >> >> >> ... if you free the original buffer inside get_full_path (or use >> ResourceMark): >> >> static errno_t get_full_path(LPCWSTR& unicode_path) { >> ??? LPCWSTR full_path = .....; >> ??? .... >> ??? free(unicode_path); >> unicode_path = full_path; >> ??? return ERROR_SUCCESS; >> } >> >> [4] This looks good! Much more readable than the original code! >> >> 4285?? _snwprintf(result, result_len, L"%s%s", prefix, >> &converted_path[prefix_off]); >> >> >> Thanks >> - Ioi >> >> >> On 3/4/20 12:22 AM, Yasumasa Suenaga wrote: >>> Hi Ioi, >>> >>> Thanks for your comment. >>> How about this change? >>> >>> ? http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.04/ >>> >>> This webrev adds the function both converting to unicode and getting >>> full path, >>> they would be called from wide_abs_unc_path(). >>> Also MemoryCleaner class which is introduced in this webrev frees >>> memory auomatically in d'tor. >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2020/03/04 16:05, Ioi Lam wrote: >>>> >>>> >>>> On 3/3/20 10:22 PM, Yasumasa Suenaga wrote: >>>>> Hi Ioi, >>>>> >>>>>> [1] I think the memory freeing can be done using a destructor, so >>>>>> you can simply return in case of error, without having a lot of >>>>>> "if" checks: >>>>> >>>>> What do you think to use `goto` statement like Linux kernel? >>>>> >>>>> >>>> I don't have a strong opinion but I think many/most HotSpot >>>> developers are against it. Also with goto, you need to store the >>>> return value in a variable, but with destructors, you can just say >>>> "return value", so the code is cleaner. >>>> >>>> Thanks >>>> - Ioi >>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2020/03/04 14:50, Ioi Lam wrote: >>>>>> Hi Yasumasa, >>>>>> >>>>>> I think this function is getting too long and it's hard to figure >>>>>> out all the error conditions. >>>>>> >>>>>> (I feel sorry for you because the code wasn't pretty messy to >>>>>> begin with, but I think it's at a point that we need to do the >>>>>> cleanup. It took me like 30 minutes to understand what is going on) >>>>>> >>>>>> [0] I noticed you changed the return type to LPWSTR, but all the >>>>>> callers are using wchar_t*. Are these two types the same? >>>>>> >>>>>> ???? If you want to use LPWSTR within this function for >>>>>> consistency, maybe you can return the value as >>>>>> >>>>>> ???? return (wchar_t*)result; // LPWSTR and wchat_t* are the same >>>>>> type on Windows. >>>>>> >>>>>> ???? That way people unfamiliar with Windows (most of us) won't >>>>>> be confused. >>>>>> >>>>>> [1] I think the memory freeing can be done using a destructor, so >>>>>> you can simply return in case of error, without having a lot of >>>>>> "if" checks: >>>>>> >>>>>> ???? class State { >>>>>> ??????? errno_t & err; >>>>>> ?? ????? char* buf; >>>>>> ???????? LPWSTR unicode_path; >>>>>> LPWSTR result; >>>>>> >>>>>> ???????? State(errno_t & e) : buf(NULL), unicode_path(NULL), >>>>>> result(NULL), err(e) { >>>>>> ???????????? err = SUCCESS; >>>>>> ????????? } >>>>>> ???????? ~State() { >>>>>> ?????????? os::free(buf); >>>>>> ?? ? ? ? ? os::free(unicode_path); >>>>>> ?????????? if (err != ERROR_SUCCESS) { >>>>>> ????????????? os::free(result); >>>>>> ?????????? } >>>>>> ???????? } >>>>>> ???????? LPWSTR as_error(errno_t e) { >>>>>> ???????????? err = e; >>>>>> ???????????? return NULL; >>>>>> ???????? } >>>>>> ???? } st; >>>>>> >>>>>> >>>>>> ???? if ((st.buf = static_cast(os::malloc(buf_size, >>>>>> mtInternal))) == NULL) { >>>>>> ???????? return st.as_error(ENOMEM); >>>>>> ???? } >>>>>> >>>>>> ???? .... >>>>>> ???? return st.result; >>>>>> >>>>>> [2] Will it be possible to break the functon down into smaller >>>>>> pieces, so it's easier to understand what's going on? >>>>>> >>>>>> E.g., Maybe the code that? converts the C string into a unicode >>>>>> string can be moved into a separate function? >>>>>> >>>>>> ?? // Get required buffer size to convert to Unicode >>>>>> ?? int unicode_path_len = MultiByteToWideChar(CP_THREAD_ACP, >>>>>> MB_ERR_INVALID_CHARS, >>>>>> ????????????????????????????????????????????? buf, -1, >>>>>> ????????????????????????????????????????????? NULL, 0); >>>>>> ?? if (unicode_path_len == 0) { >>>>>> ???? os::free(buf); >>>>>> ???? err = EINVAL; >>>>>> ???? return NULL; >>>>>> ?? } >>>>>> >>>>>> ?? .... >>>>>> >>>>>> ?? LPWSTR unicode_path = >>>>>> static_cast(os::malloc(sizeof(WCHAR) * unicode_path_len, >>>>>> mtInternal)); >>>>>> ?? if (unicode_path == NULL) { >>>>>> ???? err = ENOMEM; >>>>>> ?? } else { >>>>>> ???? // This call would be success because it is checked in above >>>>>> ???? err = ERROR_SUCCESS; >>>>>> ???? MultiByteToWideChar(CP_THREAD_ACP, >>>>>> ???????????????????????? MB_ERR_INVALID_CHARS, >>>>>> ???????????????????????? buf, -1, >>>>>> ???????????????????????? unicode_path, unicode_path_len); >>>>>> >>>>>> >>>>>> >>>>>> [3] size_t result_len = prefix_len - prefix_off + additional_space; >>>>>> >>>>>> The variable name result_len is confusing. It's not the length of >>>>>> the result. >>>>>> >>>>>> [4] The old copying code was already hard to understand, and the >>>>>> new version now has 2 versions of the copying code. I think the >>>>>> reason is you want to avoid an extra memcpy by calling >>>>>> GetFullPathNameW directly on the result. >>>>>> >>>>>> ???? if (needs_fullpath) { >>>>>> ?????? // Get required buffer size to convert to full path >>>>>> ?????? DWORD full_path_len = GetFullPathNameW(unicode_path, 0, >>>>>> NULL, NULL); >>>>>> ?????? if (full_path_len == 0) { >>>>>> ???????? err = EINVAL; >>>>>> ?????? } else { >>>>>> ???????? size_t result_size = sizeof(WCHAR) * (result_len + >>>>>> full_path_len); >>>>>> ???????? result = static_cast(os::malloc(result_size, >>>>>> mtInternal)); >>>>>> ???????? if (result == NULL) { >>>>>> ?????????? err = ENOMEM; >>>>>> ???????? } else { >>>>>> ?????????? // This call would be success because it is checked in >>>>>> above >>>>>> ?????????? GetFullPathNameW(unicode_path, full_path_len, result + >>>>>> prefix_len - prefix_off, NULL); >>>>>> ?????????? // Copy prefix >>>>>> ?????????? memcpy(result, prefix, sizeof(WCHAR) * prefix_len); >>>>>> ???????? } >>>>>> ?????? } >>>>>> ???? } else { >>>>>> ?????? size_t result_size = sizeof(WCHAR) * (result_len + >>>>>> unicode_path_len); >>>>>> ?????? result = static_cast(os::malloc(result_size, >>>>>> mtInternal)); >>>>>> ?????? if (result == NULL) { >>>>>> ???????? err = ENOMEM; >>>>>> ?????? } else { >>>>>> ???????? // Copy unicode path >>>>>> ???????? memcpy(result + prefix_len - prefix_off, unicode_path, >>>>>> sizeof(WCHAR) * unicode_path_len); >>>>>> ???????? // Copy prefix >>>>>> ???????? memcpy(result, prefix, sizeof(WCHAR) * prefix_len); >>>>>> ?????? } >>>>>> ???? } >>>>>> >>>>>> I think the code can be simplified if you do the fullpath version >>>>>> in a separate function: >>>>>> >>>>>> ???? if (needs_fullpath) { >>>>>> ??????? unicode_path = to_full_path(unicode_path, err, >>>>>> &unicode_path_len); >>>>>> ???? } >>>>>> >>>>>> ???? size_t result_size = ...... >>>>>> >>>>>> This will do one more memcpy, but I think it's not performance >>>>>> critical >>>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>> >>>>>> >>>>>> On 3/3/20 4:19 PM, Yasumasa Suenaga wrote: >>>>>>> Thanks Ralf! >>>>>>> I fixed them. >>>>>>> >>>>>>> I need a Reviewer to push, so I uploaded new webrev: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.03/ >>>>>>> >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2020/03/03 23:25, Schmelter, Ralf wrote: >>>>>>>> Hi Yasumasa, >>>>>>>> >>>>>>>> thanks for your work. The code should now be able to handle any >>>>>>>> path. >>>>>>>> >>>>>>>> Two small changes (no new webrev needed): >>>>>>>> >>>>>>>> size_t result_len = prefix_len + prefix_off + additional_space >>>>>>>> should be >>>>>>>> size_t result_len = prefix_len - prefix_off + additional_space >>>>>>>> >>>>>>>> And you should copy the prefix after you copied the filename, >>>>>>>> since sometimes the prefix is required to overwrite the start >>>>>>>> of the filename: >>>>>>>> >>>>>>>> // This call would be success because it is checked in above >>>>>>>> GetFullPathNameW(unicode_path, full_path_len, result + >>>>>>>> prefix_len - prefix_off, NULL); >>>>>>>> // Copy prefix >>>>>>>> memcpy(result, prefix, sizeof(WCHAR) * prefix_len); >>>>>>>> >>>>>>>> and >>>>>>>> >>>>>>>> // Copy unicode path >>>>>>>> memcpy(result + prefix_len - prefix_off, unicode_path, >>>>>>>> sizeof(WCHAR) * unicode_path_len); >>>>>>>> // Copy prefix >>>>>>>> memcpy(result, prefix, sizeof(WCHAR) * prefix_len); >>>>>>>> >>>>>>>> With these changes the os_windows gtest and the appcds jtreg >>>>>>>> tests run without errors. >>>>>>>> >>>>>>>> Best regards, >>>>>>>> Ralf >>>>>>>> >>>>>> >>>> >> From yumin.qi at oracle.com Thu Mar 5 05:05:00 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Wed, 4 Mar 2020 21:05:00 -0800 Subject: RFR(XS): 8240546: runtime/cds/appcds/TestZGCWithCDS.java fails with Graal Message-ID: <385660b7-9c54-b259-c4c4-be671a2ad5b3@oracle.com> Hi, ? Can I have your review on ? Bug: https://bugs.openjdk.java.net/browse/JDK-8240546 ? Webrev: http://cr.openjdk.java.net/~minqi/8240546/webrev/ ? Summary: TestZGCWithCDS failed graal on ZGC since Graal does not? work with ZGC. Fixed in test code to skip when Graal enabled. ? Thanks ? Yumin From calvin.cheung at oracle.com Thu Mar 5 05:13:02 2020 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Wed, 4 Mar 2020 21:13:02 -0800 Subject: RFR(S): 8232081: Try to link all classes during dynamic CDS dump In-Reply-To: <00090c43-1e0d-2b1c-d2a5-1b5f1b29199d@oracle.com> References: <9b40145a-db72-6313-3b98-5e5aea754e25@oracle.com> <1167ff6d-3997-6390-d020-d0946fad0af8@oracle.com> <3f373a33-c7ed-0237-1dd5-f8ac8e8794ee@oracle.com> <8ac4612b-fae3-bb16-f2ab-fa323c33ca51@oracle.com> <00090c43-1e0d-2b1c-d2a5-1b5f1b29199d@oracle.com> Message-ID: <7a54e00f-c049-a5ef-0afc-bb6c9d0f30ad@oracle.com> Hi David, Ioi, Here's an updated webrev which doesn't involve changing InstanceKlass::verify_on() and made use of the changes for JDK-8240481. ??? http://cr.openjdk.java.net/~ccheung/jdk15/8232081/webrev.03/ thanks, Calvin On 3/1/20 10:14 PM, David Holmes wrote: > Hi Calvin, > > On 28/02/2020 4:12 pm, Calvin Cheung wrote: >> Hi David, >> >> On 2/27/20 5:40 PM, David Holmes wrote: >>> Hi Calvin, Ioi, >>> >>> Looking good - comments below. >>> >>> A meta-question: normal application classes are rarely loaded but >>> not linked so I'm a little surprised this is an issue. What is the >>> main source of such classes - generated classes like lambda forms? >>> Also why do we need to link them when loading from the archive if >>> they were never linked when the application actually executed ?? >> >> I saw that Ioi already answered the above. >> >> I'll try to answer your questions inline below.. > > Responses inline below ... > >>> >>> On 28/02/2020 10:48 am, Calvin Cheung wrote: >>>> Hi David, Ioi, >>>> >>>> Thanks for your review and suggestions. >>>> >>>> On 2/26/20 9:46 PM, Ioi Lam wrote: >>>>> >>>>> >>>>> On 2/26/20 7:50 PM, David Holmes wrote: >>>>>> Hi Calvin, >>>>>> >>>>>> Adding core-libs-dev as you are messing with their code :) >>>>>> >>>>>> On 27/02/2020 1:19 pm, Calvin Cheung wrote: >>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8232081 >>>>>>> webrev: >>>>>>> http://cr.openjdk.java.net/~ccheung/jdk15/8232081/webrev.00/ >>>>>>> >>>>>>> The proposed changeset for this RFE adds a >>>>>>> JVM_LinkClassesForCDS() function to be called from >>>>>>> java/lang/Shutdown to notify the JVM to link the classes loaded >>>>>>> by the builtin class loaders. The >>>>>> >>>>>> This would be much less disruptive if this was handled purely on >>>>>> the VM side once we have started shutdown. No need to make any >>>>>> changes to the Java side then, nor jvm.cpp. >>>>>> >>>>> >>>>> Hi David, >>>>> >>>>> To link the classes, we need to be able to run Java code -- when >>>>> linking a class X loaded by the app loader, X will be verified. >>>>> During verification of X, we may need to load additional classes >>>>> from the app loader, which executes Java code during its class >>>>> loading operations. >>>>> >>>>> We also need to handle all the exit conditions. As far as I can >>>>> tell, an app can exit the JVM in two ways: >>>>> >>>>> (1) Explicitly calling System.exit(). This will call >>>>> java.lang.Shutdown.exit() which does this: >>>>> >>>>> http://hg.openjdk.java.net/jdk/jdk/file/0edc7fd0d7a3/src/java.base/share/classes/java/lang/Shutdown.java#l163 >>>>> >>>>> >>>>> ??????????? beforeHalt(); // native >>>>> ??????????? runHooks(); >>>>> ??????????? halt(status); >>>>> >>>>> (2) When all non-daemon threads have died (e.g., falling out of >>>>> the bottom of HelloWorld.main()). There's no explicit call to >>>>> System.exit(), but the JVM will proactively call >>>>> java.lang.Shutdown.shutdown() inside >>>>> JavaThread::invoke_shutdown_hooks() >>>>> >>>>> http://hg.openjdk.java.net/jdk/jdk/file/0edc7fd0d7a3/src/hotspot/share/runtime/thread.cpp#l4331 >>>>> >>>>> http://hg.openjdk.java.net/jdk/jdk/file/0edc7fd0d7a3/src/java.base/share/classes/java/lang/Shutdown.java#l184 >>>>> >>>>> >>>>> If we want to avoid modifying the Java code, I think we can >>>>> intercept JVM_BeforeHalt() and >>>>> JavaThread::invoke_shutdown_hooks(). This way we should be able to >>>>> handle all the classes (except those that are loaded inside >>>>> Shutdown.runHooks). >>>> >>>> I've implemented the above. No changes to the Java side. >>>> >>>> updated webrev: >>>> >>>> http://cr.openjdk.java.net/~ccheung/jdk15/8232081/webrev.01/ >>> >>> This looks much better to me. I wasn't sure if you were that >>> concerned about catching the classes loaded by the Shutdown hooks, >>> but it is good you are not as it seems problematic to me to trigger >>> class loading etc after the shutdown hooks have executed - you may >>> hit unexpected conditions. So this approach is good. >>> >>> A few minor comments: >>> >>> src/hotspot/share/memory/metaspaceShared.cpp >>> src/hotspot/share/memory/metaspaceShared.hpp >>> >>> Why the change from TRAPS to "Thread* THREAD"? >> I forgot why I changed it but I've changed it back and it still works. > > Ok. > >>> >>> --- >>> >>> src/hotspot/share/oops/instanceKlass.cpp >>> >>> I'm not clear how verify_on is used so am unsure how the additional >>> error state check may affect code unrelated to dynamic dumping ?? >> >> Some of the appcds tests by design have some classes which fail >> verification. Without the change, the following assert would fail: >> >> void vtableEntry::verify(klassVtable* vt, outputStream* st) { >> ?? Klass* vtklass = vt->klass(); >> ?? if (vtklass->is_instance_klass() && >> ????? (InstanceKlass::cast(vtklass)->major_version() >= >> klassVtable::VTABLE_TRANSITIVE_OVERRIDE_VERSION)) { >> *assert*(method() != NULL, "must have set method"); >> ?? } > > Okay so you need to exclude for that case but ... > >> Here's the call stack during dynamic CDS dump to the above function: >> >> vtableEntry::verify(klassVtable *, outputStream *) : void >> ???? klassVtable::verify(outputStream *, bool) : void >> ???????? InstanceKlass::verify_on(outputStream *) : void >> ???????????? Klass::verify() : void >> ???????????????? ClassLoaderData::verify() : void >> ???????????????????? ClassLoaderDataGraph::verify() : void >> ???????????????????????? Universe::verify(enum VerifyOption, const >> char *) : void >> ???????????????????????????? Universe::verify(const char *) : void >> DynamicArchiveBuilder::verify_universe(const char *) : void >> DynamicArchiveBuilder::doit() : void (2 matches) >> VM_PopulateDynamicDumpSharedSpace::doit() : void >> VM_Operation::evaluate() : void >> >> The is_linked() function is implemented as: >> >> ???? bool is_linked() const?????????????????? { return _init_state >= >> linked; } >> >> which includes 'initialization_error'. > > ... AFAICS Universe::verify is call from a number of places, all of > which would reach this modified code. As that existing code has > included classes that are in the error state then it seems to me that > unless you can prove otherwise you need to keep the existing code > as-is for the non-CDS dump case. > >> A previous version of the changeset (including the change in question >> in instanceKlass.cpp) passed tier1 - 4 testing. Let me know if I >> should run other tests to make sure the change is good. > > There may well not be a test that covers this. > > Thanks, > David > >>> >>> Also can I suggest (as you are touching this code) to delete the >>> ancient comment: >>> >>> 3580???? // $$$ This used to be done only for m/s collections. Doing it >>> 3581???? // always seemed a valid generalization.? (DLD -- 6/00) >> >> I'm glad that you suggested to remove the above comment. I've no clue >> what it means either. >> >> updated webrev: >> >> http://cr.openjdk.java.net/~ccheung/jdk15/8232081/webrev.02/ >> >> thanks, >> Calvin >>> >>> Thanks, >>> David >>> ----- >>> >>>> I also updated the test to test the shutdown hook and System.exit() >>>> scenarios. >>>> >>>> All appcds tests passed on my linux host. I'll run more tests using >>>> mach5. >>>> >>>> thanks, >>>> >>>> Calvin >>>> >>>> >>>>> >>>>> What do you think? >>>>> >>>>> - Ioi >>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>>> MetaspaceShared::link_and_cleanup_shared_classes() has been >>>>>>> modified to handle both static and dynamic CDS dump. For dynamic >>>>>>> CDS dump, only classes loaded by the builtin class loaders will >>>>>>> be linked. Local performance testing using javac on >>>>>>> HelloWorld.java shows an improvement of >5%. >>>>>>> >>>>>>> Passed tier1 - 4 tests. >>>>>>> >>>>>>> thanks, >>>>>>> >>>>>>> Calvin >>>>>>> >>>>> From calvin.cheung at oracle.com Thu Mar 5 05:20:10 2020 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Wed, 4 Mar 2020 21:20:10 -0800 Subject: RFR(XS): 8240546: runtime/cds/appcds/TestZGCWithCDS.java fails with Graal In-Reply-To: <385660b7-9c54-b259-c4c4-be671a2ad5b3@oracle.com> References: <385660b7-9c54-b259-c4c4-be671a2ad5b3@oracle.com> Message-ID: <35a586b6-f3d4-995c-a968-88c6c476d3e8@oracle.com> Hi Yumin, Changes look good. thanks, Calvin On 3/4/20 9:05 PM, Yumin Qi wrote: > Hi, > > ? Can I have your review on > > ? Bug: https://bugs.openjdk.java.net/browse/JDK-8240546 > > ? Webrev: http://cr.openjdk.java.net/~minqi/8240546/webrev/ > > ? Summary: TestZGCWithCDS failed graal on ZGC since Graal does not? > work with ZGC. Fixed in test code to skip when Graal enabled. > > > ? Thanks > > ? Yumin > From yumin.qi at oracle.com Thu Mar 5 05:25:24 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Wed, 4 Mar 2020 21:25:24 -0800 Subject: RFR(XS): 8240546: runtime/cds/appcds/TestZGCWithCDS.java fails with Graal In-Reply-To: <35a586b6-f3d4-995c-a968-88c6c476d3e8@oracle.com> References: <385660b7-9c54-b259-c4c4-be671a2ad5b3@oracle.com> <35a586b6-f3d4-995c-a968-88c6c476d3e8@oracle.com> Message-ID: <3cd2fa10-70c1-d538-f5f8-82cbe0b49340@oracle.com> Thanks for quick review! Yumin On 3/4/20 9:20 PM, Calvin Cheung wrote: > Hi Yumin, > > Changes look good. > > thanks, > Calvin > > On 3/4/20 9:05 PM, Yumin Qi wrote: >> Hi, >> >> ? Can I have your review on >> >> ? Bug: https://bugs.openjdk.java.net/browse/JDK-8240546 >> >> ? Webrev: http://cr.openjdk.java.net/~minqi/8240546/webrev/ >> >> ? Summary: TestZGCWithCDS failed graal on ZGC since Graal does not? >> work with ZGC. Fixed in test code to skip when Graal enabled. >> >> >> ? Thanks >> >> ? Yumin >> From suenaga at oss.nttdata.com Thu Mar 5 05:39:26 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Thu, 5 Mar 2020 14:39:26 +0900 Subject: RFR: 8240197: Cannot start JVM when $JAVA_HOME includes CJK characters In-Reply-To: <4997774e-f009-a6e0-b4df-8c1dd3a4dd17@oracle.com> References: <6ec2f649-8999-fdd5-c2df-465925c3663b@oss.nttdata.com> <2d5148fc-7d71-d94c-d4a3-e6424a38b091@oss.nttdata.com> <37e2f672-3407-5b16-ab71-aa1e70e928db@oss.nttdata.com> <954f0bf1-91e2-3a1b-e2cb-e93d8343b0e6@oracle.com> <71ba4bad-05c9-042d-73dc-7f492fc1728a@oss.nttdata.com> <44d86ec7-aa6d-daf8-7200-534717cb4edb@oracle.com> <25aa10eb-6a8f-b416-02c2-b3fcd140076e@oss.nttdata.com> <4997774e-f009-a6e0-b4df-8c1dd3a4dd17@oracle.com> Message-ID: Thanks Ioi! Ralf, are you ok to this change? http://hg.openjdk.java.net/jdk/submit/rev/8f25e02f0432 (all tests on submit repo has passed) Yasumasa On 2020/03/05 13:15, Ioi Lam wrote: > Hi Yasumasa, > > This looks very good. Thanks for bearing with my requests :-) > > The return value of GetFullPathNameW is a bit weird. I would suggest adding comments so people can understand the code without reading the > GetFullPathNameW documentation: > > > 4190?? // Get required buffer size to convert to full path. The return > ?????? // value INCLUDES the terminating null character. > 4191?? DWORD full_path_len = GetFullPathNameW(unicode_path, 0, NULL, NULL); > > ?????? // When the buffer has sufficient size, the return value EXCLUDES the > ?????? // terminating null character. > 4202?? DWORD result = GetFullPathNameW(unicode_path, full_path_len, full_path, NULL); > 4203?? assert(result == (full_path_len - 1), "length already checked above"); > > (No need for new webrev). > > Thanks > - Ioi > > > > > On 3/4/20 4:18 PM, Yasumasa Suenaga wrote: >> Hi Ioi, >> >> How about this change? >> >> ? http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.05/ >> >> I already tried to use ResouceMark, but it was crashed because wide_abs_unc_path() would be called from outside of java thread at boot time. >> This webrev uses os::free() instead of memory cleaner class, but I think it is simple because I tweaked memory lifecycle. >> >> >> Thanks, >> >> Yasumasa >> >> >> On 2020/03/05 6:56, Ioi Lam wrote: >>> Hi Yasumasa, >>> >>> This version looks good! >>> >>> >>> [0] One thing I forgot ... is it possible to use "ResourceMark rm" in this function? If so, you can allocate the temp arrays using NEW_RESOURCE_ARRAY and then don't need to worry about freeing them. >>> >>> >>> [1] for this code >>> >>> 4202?? // This call would be success because it is checked in above >>> 4203?? GetFullPathNameW(unicode_path, full_path_len, full_path, NULL); >>> >>> How about this instead: >>> >>> ?????? DWORD len = GetFullPathNameW(unicode_path, full_path_len, full_path, NULL); >>> ?????? assert(len <= full_path_len, "length already checked above"); >>> >>> (same for the MultiByteToWideChar call in convert_to_unicode) >>> >>> >>> [2] This is no longer needed >>> >>> 4240???? os::free(buf); >>> >>> >>> [3] I think the following code is not needed .... >>> >>> 4269?? } else { >>> 4270???? size_t converted_path_size = sizeof(WCHAR) * (wcslen(unicode_path) + 1); >>> 4271???? converted_path = reinterpret_cast(os::malloc(converted_path_size, mtInternal)); >>> 4272???? if (converted_path == NULL) { >>> 4273?????? vm_exit_out_of_memory(buf_size, OOM_MALLOC_ERROR, "wide_abs_unc_path"); >>> 4274???? } >>> 4275???? memcpy(converted_path, unicode_path, converted_path_size); >>> >>> >>> ... if you free the original buffer inside get_full_path (or use ResourceMark): >>> >>> static errno_t get_full_path(LPCWSTR& unicode_path) { >>> ??? LPCWSTR full_path = .....; >>> ??? .... >>> ??? free(unicode_path); >>> unicode_path = full_path; >>> ??? return ERROR_SUCCESS; >>> } >>> >>> [4] This looks good! Much more readable than the original code! >>> >>> 4285?? _snwprintf(result, result_len, L"%s%s", prefix, &converted_path[prefix_off]); >>> >>> >>> Thanks >>> - Ioi >>> >>> >>> On 3/4/20 12:22 AM, Yasumasa Suenaga wrote: >>>> Hi Ioi, >>>> >>>> Thanks for your comment. >>>> How about this change? >>>> >>>> ? http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.04/ >>>> >>>> This webrev adds the function both converting to unicode and getting full path, >>>> they would be called from wide_abs_unc_path(). >>>> Also MemoryCleaner class which is introduced in this webrev frees memory auomatically in d'tor. >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2020/03/04 16:05, Ioi Lam wrote: >>>>> >>>>> >>>>> On 3/3/20 10:22 PM, Yasumasa Suenaga wrote: >>>>>> Hi Ioi, >>>>>> >>>>>>> [1] I think the memory freeing can be done using a destructor, so you can simply return in case of error, without having a lot of "if" checks: >>>>>> >>>>>> What do you think to use `goto` statement like Linux kernel? >>>>>> >>>>>> >>>>> I don't have a strong opinion but I think many/most HotSpot developers are against it. Also with goto, you need to store the return value in a variable, but with destructors, you can just say "return value", so the code is cleaner. >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2020/03/04 14:50, Ioi Lam wrote: >>>>>>> Hi Yasumasa, >>>>>>> >>>>>>> I think this function is getting too long and it's hard to figure out all the error conditions. >>>>>>> >>>>>>> (I feel sorry for you because the code wasn't pretty messy to begin with, but I think it's at a point that we need to do the cleanup. It took me like 30 minutes to understand what is going on) >>>>>>> >>>>>>> [0] I noticed you changed the return type to LPWSTR, but all the callers are using wchar_t*. Are these two types the same? >>>>>>> >>>>>>> ???? If you want to use LPWSTR within this function for consistency, maybe you can return the value as >>>>>>> >>>>>>> ???? return (wchar_t*)result; // LPWSTR and wchat_t* are the same type on Windows. >>>>>>> >>>>>>> ???? That way people unfamiliar with Windows (most of us) won't be confused. >>>>>>> >>>>>>> [1] I think the memory freeing can be done using a destructor, so you can simply return in case of error, without having a lot of "if" checks: >>>>>>> >>>>>>> ???? class State { >>>>>>> ??????? errno_t & err; >>>>>>> ?? ????? char* buf; >>>>>>> ???????? LPWSTR unicode_path; >>>>>>> LPWSTR result; >>>>>>> >>>>>>> ???????? State(errno_t & e) : buf(NULL), unicode_path(NULL), result(NULL), err(e) { >>>>>>> ???????????? err = SUCCESS; >>>>>>> ????????? } >>>>>>> ???????? ~State() { >>>>>>> ?????????? os::free(buf); >>>>>>> ?? ? ? ? ? os::free(unicode_path); >>>>>>> ?????????? if (err != ERROR_SUCCESS) { >>>>>>> ????????????? os::free(result); >>>>>>> ?????????? } >>>>>>> ???????? } >>>>>>> ???????? LPWSTR as_error(errno_t e) { >>>>>>> ???????????? err = e; >>>>>>> ???????????? return NULL; >>>>>>> ???????? } >>>>>>> ???? } st; >>>>>>> >>>>>>> >>>>>>> ???? if ((st.buf = static_cast(os::malloc(buf_size, mtInternal))) == NULL) { >>>>>>> ???????? return st.as_error(ENOMEM); >>>>>>> ???? } >>>>>>> >>>>>>> ???? .... >>>>>>> ???? return st.result; >>>>>>> >>>>>>> [2] Will it be possible to break the functon down into smaller pieces, so it's easier to understand what's going on? >>>>>>> >>>>>>> E.g., Maybe the code that? converts the C string into a unicode string can be moved into a separate function? >>>>>>> >>>>>>> ?? // Get required buffer size to convert to Unicode >>>>>>> ?? int unicode_path_len = MultiByteToWideChar(CP_THREAD_ACP, >>>>>>> MB_ERR_INVALID_CHARS, >>>>>>> ????????????????????????????????????????????? buf, -1, >>>>>>> ????????????????????????????????????????????? NULL, 0); >>>>>>> ?? if (unicode_path_len == 0) { >>>>>>> ???? os::free(buf); >>>>>>> ???? err = EINVAL; >>>>>>> ???? return NULL; >>>>>>> ?? } >>>>>>> >>>>>>> ?? .... >>>>>>> >>>>>>> ?? LPWSTR unicode_path = static_cast(os::malloc(sizeof(WCHAR) * unicode_path_len, mtInternal)); >>>>>>> ?? if (unicode_path == NULL) { >>>>>>> ???? err = ENOMEM; >>>>>>> ?? } else { >>>>>>> ???? // This call would be success because it is checked in above >>>>>>> ???? err = ERROR_SUCCESS; >>>>>>> ???? MultiByteToWideChar(CP_THREAD_ACP, >>>>>>> ???????????????????????? MB_ERR_INVALID_CHARS, >>>>>>> ???????????????????????? buf, -1, >>>>>>> ???????????????????????? unicode_path, unicode_path_len); >>>>>>> >>>>>>> >>>>>>> >>>>>>> [3] size_t result_len = prefix_len - prefix_off + additional_space; >>>>>>> >>>>>>> The variable name result_len is confusing. It's not the length of the result. >>>>>>> >>>>>>> [4] The old copying code was already hard to understand, and the new version now has 2 versions of the copying code. I think the reason is you want to avoid an extra memcpy by calling GetFullPathNameW directly on the result. >>>>>>> >>>>>>> ???? if (needs_fullpath) { >>>>>>> ?????? // Get required buffer size to convert to full path >>>>>>> ?????? DWORD full_path_len = GetFullPathNameW(unicode_path, 0, NULL, NULL); >>>>>>> ?????? if (full_path_len == 0) { >>>>>>> ???????? err = EINVAL; >>>>>>> ?????? } else { >>>>>>> ???????? size_t result_size = sizeof(WCHAR) * (result_len + full_path_len); >>>>>>> ???????? result = static_cast(os::malloc(result_size, mtInternal)); >>>>>>> ???????? if (result == NULL) { >>>>>>> ?????????? err = ENOMEM; >>>>>>> ???????? } else { >>>>>>> ?????????? // This call would be success because it is checked in above >>>>>>> ?????????? GetFullPathNameW(unicode_path, full_path_len, result + prefix_len - prefix_off, NULL); >>>>>>> ?????????? // Copy prefix >>>>>>> ?????????? memcpy(result, prefix, sizeof(WCHAR) * prefix_len); >>>>>>> ???????? } >>>>>>> ?????? } >>>>>>> ???? } else { >>>>>>> ?????? size_t result_size = sizeof(WCHAR) * (result_len + unicode_path_len); >>>>>>> ?????? result = static_cast(os::malloc(result_size, mtInternal)); >>>>>>> ?????? if (result == NULL) { >>>>>>> ???????? err = ENOMEM; >>>>>>> ?????? } else { >>>>>>> ???????? // Copy unicode path >>>>>>> ???????? memcpy(result + prefix_len - prefix_off, unicode_path, sizeof(WCHAR) * unicode_path_len); >>>>>>> ???????? // Copy prefix >>>>>>> ???????? memcpy(result, prefix, sizeof(WCHAR) * prefix_len); >>>>>>> ?????? } >>>>>>> ???? } >>>>>>> >>>>>>> I think the code can be simplified if you do the fullpath version in a separate function: >>>>>>> >>>>>>> ???? if (needs_fullpath) { >>>>>>> ??????? unicode_path = to_full_path(unicode_path, err, &unicode_path_len); >>>>>>> ???? } >>>>>>> >>>>>>> ???? size_t result_size = ...... >>>>>>> >>>>>>> This will do one more memcpy, but I think it's not performance critical >>>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>>>> >>>>>>> >>>>>>> On 3/3/20 4:19 PM, Yasumasa Suenaga wrote: >>>>>>>> Thanks Ralf! >>>>>>>> I fixed them. >>>>>>>> >>>>>>>> I need a Reviewer to push, so I uploaded new webrev: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.03/ >>>>>>>> >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2020/03/03 23:25, Schmelter, Ralf wrote: >>>>>>>>> Hi Yasumasa, >>>>>>>>> >>>>>>>>> thanks for your work. The code should now be able to handle any path. >>>>>>>>> >>>>>>>>> Two small changes (no new webrev needed): >>>>>>>>> >>>>>>>>> size_t result_len = prefix_len + prefix_off + additional_space >>>>>>>>> should be >>>>>>>>> size_t result_len = prefix_len - prefix_off + additional_space >>>>>>>>> >>>>>>>>> And you should copy the prefix after you copied the filename, since sometimes the prefix is required to overwrite the start of the filename: >>>>>>>>> >>>>>>>>> // This call would be success because it is checked in above >>>>>>>>> GetFullPathNameW(unicode_path, full_path_len, result + prefix_len - prefix_off, NULL); >>>>>>>>> // Copy prefix >>>>>>>>> memcpy(result, prefix, sizeof(WCHAR) * prefix_len); >>>>>>>>> >>>>>>>>> and >>>>>>>>> >>>>>>>>> // Copy unicode path >>>>>>>>> memcpy(result + prefix_len - prefix_off, unicode_path, sizeof(WCHAR) * unicode_path_len); >>>>>>>>> // Copy prefix >>>>>>>>> memcpy(result, prefix, sizeof(WCHAR) * prefix_len); >>>>>>>>> >>>>>>>>> With these changes the os_windows gtest and the appcds jtreg tests run without errors. >>>>>>>>> >>>>>>>>> Best regards, >>>>>>>>> Ralf >>>>>>>>> >>>>>>> >>>>> >>> > From thomas.stuefe at gmail.com Thu Mar 5 05:43:29 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 5 Mar 2020 06:43:29 +0100 Subject: RFR: 8240197: Cannot start JVM when $JAVA_HOME includes CJK characters In-Reply-To: <25aa10eb-6a8f-b416-02c2-b3fcd140076e@oss.nttdata.com> References: <6ec2f649-8999-fdd5-c2df-465925c3663b@oss.nttdata.com> <2d5148fc-7d71-d94c-d4a3-e6424a38b091@oss.nttdata.com> <37e2f672-3407-5b16-ab71-aa1e70e928db@oss.nttdata.com> <954f0bf1-91e2-3a1b-e2cb-e93d8343b0e6@oracle.com> <71ba4bad-05c9-042d-73dc-7f492fc1728a@oss.nttdata.com> <44d86ec7-aa6d-daf8-7200-534717cb4edb@oracle.com> <25aa10eb-6a8f-b416-02c2-b3fcd140076e@oss.nttdata.com> Message-ID: Hi Yasumasa, Had a glance over version 05. Some remarks: -- I dislike the reinterpret_cast<> and do not understand the need for it. We do not do this in other places with malloc. Strictly speaking this would be UB since it is not guaranteed for A=reinterpret_cast that A==B. In any way you should use NEW_C_HEAP_ARRAY and FREE_C_HEAP_ARRAY instead since those do the casting for you. They also exit the VM in case of OOM so no need to handle that manually. -- In wide_abs_unc_path: + // Need to allocate at least room for 3 characters, since os::native_path transforms C: to C:. + size_t buf_size = 1 + MAX2((size_t)3, strlen(path)); + char* buf = reinterpret_cast(os::malloc(buf_size, mtInternal)); + if (buf == NULL) { + vm_exit_out_of_memory(buf_size, OOM_MALLOC_ERROR, "wide_abs_unc_path"); + } + memcpy(buf, path, buf_size); + os::native_path(buf); The memcpy is wrong. You have a rare chance of triggering a read fault since you read over the limit of path. Just use strcpy. -- The comment about "additional_space" confused me until I read the sources, sp I would suggest reforming it as such: -// additional_space is the number of additionally allocated wchars after the terminating L'\0'. +// additional_space is the size of space, in wchar_t, the function will additionally add to the allocation of return buffer (such that the size of the returned buffer is at least wcslen(buf) + 1 + additional_space). -- I would also omit this comment: +// This is based on pathToNTPath() in io_util_md.cpp, but omits the optimizations for +// short paths. I do not think anyone reading this code cares. -- Hmm, small style nit, sometimes we return errno_t and the actual return value is via output parameter, +static errno_t convert_to_unicode(char const* char_path, LPWSTR& unicode_path) { +static errno_t get_full_path(LPCWSTR unicode_path, LPWSTR& full_path) { and here its the other way around: +static wchar_t* wide_abs_unc_path(char const* path, errno_t & err, int additional_space = 0) { I do not care much, but it seems strange. -- +static errno_t convert_to_unicode(char const* char_path, LPWSTR& unicode_path) + LPWSTR unicode_path; + err = convert_to_unicode(buf, unicode_path); + os::free(buf); Sigh - I really do dislike passing output parameters via reference, since at the call site is is indistinguishable from an input parameter. I know C++ programmers love this though so I won't fight this here... Could you please at least initialize unicode_path, since you do this in other places with output parameters too? -- The rest seems fine. Cheers, Thomas On Thu, Mar 5, 2020 at 1:18 AM Yasumasa Suenaga wrote: > Hi Ioi, > > How about this change? > > http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.05/ > > I already tried to use ResouceMark, but it was crashed because > wide_abs_unc_path() would be called from outside of java thread at boot > time. > This webrev uses os::free() instead of memory cleaner class, but I think > it is simple because I tweaked memory lifecycle. > > > Thanks, > > Yasumasa > > > On 2020/03/05 6:56, Ioi Lam wrote: > > Hi Yasumasa, > > > > This version looks good! > > > > > > [0] One thing I forgot ... is it possible to use "ResourceMark rm" in > this function? If so, you can allocate the temp arrays using > NEW_RESOURCE_ARRAY and then don't need to worry about freeing them. > > > > > > [1] for this code > > > > 4202 // This call would be success because it is checked in above > > 4203 GetFullPathNameW(unicode_path, full_path_len, full_path, NULL); > > > > How about this instead: > > > > DWORD len = GetFullPathNameW(unicode_path, full_path_len, > full_path, NULL); > > assert(len <= full_path_len, "length already checked above"); > > > > (same for the MultiByteToWideChar call in convert_to_unicode) > > > > > > [2] This is no longer needed > > > > 4240 os::free(buf); > > > > > > [3] I think the following code is not needed .... > > > > 4269 } else { > > 4270 size_t converted_path_size = sizeof(WCHAR) * > (wcslen(unicode_path) + 1); > > 4271 converted_path = > reinterpret_cast(os::malloc(converted_path_size, mtInternal)); > > 4272 if (converted_path == NULL) { > > 4273 vm_exit_out_of_memory(buf_size, OOM_MALLOC_ERROR, > "wide_abs_unc_path"); > > 4274 } > > 4275 memcpy(converted_path, unicode_path, converted_path_size); > > > > > > ... if you free the original buffer inside get_full_path (or use > ResourceMark): > > > > static errno_t get_full_path(LPCWSTR& unicode_path) { > > LPCWSTR full_path = .....; > > .... > > free(unicode_path); > > unicode_path = full_path; > > return ERROR_SUCCESS; > > } > > > > [4] This looks good! Much more readable than the original code! > > > > 4285 _snwprintf(result, result_len, L"%s%s", prefix, > &converted_path[prefix_off]); > > > > > > Thanks > > - Ioi > > > > > > On 3/4/20 12:22 AM, Yasumasa Suenaga wrote: > >> Hi Ioi, > >> > >> Thanks for your comment. > >> How about this change? > >> > >> http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.04/ > >> > >> This webrev adds the function both converting to unicode and getting > full path, > >> they would be called from wide_abs_unc_path(). > >> Also MemoryCleaner class which is introduced in this webrev frees > memory auomatically in d'tor. > >> > >> > >> Thanks, > >> > >> Yasumasa > >> > >> > >> On 2020/03/04 16:05, Ioi Lam wrote: > >>> > >>> > >>> On 3/3/20 10:22 PM, Yasumasa Suenaga wrote: > >>>> Hi Ioi, > >>>> > >>>>> [1] I think the memory freeing can be done using a destructor, so > you can simply return in case of error, without having a lot of "if" checks: > >>>> > >>>> What do you think to use `goto` statement like Linux kernel? > >>>> > >>>> > >>> I don't have a strong opinion but I think many/most HotSpot developers > are against it. Also with goto, you need to store the return value in a > variable, but with destructors, you can just say "return value", so the > code is cleaner. > >>> > >>> Thanks > >>> - Ioi > >>> > >>>> Yasumasa > >>>> > >>>> > >>>> On 2020/03/04 14:50, Ioi Lam wrote: > >>>>> Hi Yasumasa, > >>>>> > >>>>> I think this function is getting too long and it's hard to figure > out all the error conditions. > >>>>> > >>>>> (I feel sorry for you because the code wasn't pretty messy to begin > with, but I think it's at a point that we need to do the cleanup. It took > me like 30 minutes to understand what is going on) > >>>>> > >>>>> [0] I noticed you changed the return type to LPWSTR, but all the > callers are using wchar_t*. Are these two types the same? > >>>>> > >>>>> If you want to use LPWSTR within this function for consistency, > maybe you can return the value as > >>>>> > >>>>> return (wchar_t*)result; // LPWSTR and wchat_t* are the same > type on Windows. > >>>>> > >>>>> That way people unfamiliar with Windows (most of us) won't be > confused. > >>>>> > >>>>> [1] I think the memory freeing can be done using a destructor, so > you can simply return in case of error, without having a lot of "if" checks: > >>>>> > >>>>> class State { > >>>>> errno_t & err; > >>>>> char* buf; > >>>>> LPWSTR unicode_path; > >>>>> LPWSTR result; > >>>>> > >>>>> State(errno_t & e) : buf(NULL), unicode_path(NULL), > result(NULL), err(e) { > >>>>> err = SUCCESS; > >>>>> } > >>>>> ~State() { > >>>>> os::free(buf); > >>>>> os::free(unicode_path); > >>>>> if (err != ERROR_SUCCESS) { > >>>>> os::free(result); > >>>>> } > >>>>> } > >>>>> LPWSTR as_error(errno_t e) { > >>>>> err = e; > >>>>> return NULL; > >>>>> } > >>>>> } st; > >>>>> > >>>>> > >>>>> if ((st.buf = static_cast(os::malloc(buf_size, > mtInternal))) == NULL) { > >>>>> return st.as_error(ENOMEM); > >>>>> } > >>>>> > >>>>> .... > >>>>> return st.result; > >>>>> > >>>>> [2] Will it be possible to break the functon down into smaller > pieces, so it's easier to understand what's going on? > >>>>> > >>>>> E.g., Maybe the code that converts the C string into a unicode > string can be moved into a separate function? > >>>>> > >>>>> // Get required buffer size to convert to Unicode > >>>>> int unicode_path_len = MultiByteToWideChar(CP_THREAD_ACP, > >>>>> MB_ERR_INVALID_CHARS, > >>>>> buf, -1, > >>>>> NULL, 0); > >>>>> if (unicode_path_len == 0) { > >>>>> os::free(buf); > >>>>> err = EINVAL; > >>>>> return NULL; > >>>>> } > >>>>> > >>>>> .... > >>>>> > >>>>> LPWSTR unicode_path = > static_cast(os::malloc(sizeof(WCHAR) * unicode_path_len, > mtInternal)); > >>>>> if (unicode_path == NULL) { > >>>>> err = ENOMEM; > >>>>> } else { > >>>>> // This call would be success because it is checked in above > >>>>> err = ERROR_SUCCESS; > >>>>> MultiByteToWideChar(CP_THREAD_ACP, > >>>>> MB_ERR_INVALID_CHARS, > >>>>> buf, -1, > >>>>> unicode_path, unicode_path_len); > >>>>> > >>>>> > >>>>> > >>>>> [3] size_t result_len = prefix_len - prefix_off + additional_space; > >>>>> > >>>>> The variable name result_len is confusing. It's not the length of > the result. > >>>>> > >>>>> [4] The old copying code was already hard to understand, and the new > version now has 2 versions of the copying code. I think the reason is you > want to avoid an extra memcpy by calling GetFullPathNameW directly on the > result. > >>>>> > >>>>> if (needs_fullpath) { > >>>>> // Get required buffer size to convert to full path > >>>>> DWORD full_path_len = GetFullPathNameW(unicode_path, 0, NULL, > NULL); > >>>>> if (full_path_len == 0) { > >>>>> err = EINVAL; > >>>>> } else { > >>>>> size_t result_size = sizeof(WCHAR) * (result_len + > full_path_len); > >>>>> result = static_cast(os::malloc(result_size, > mtInternal)); > >>>>> if (result == NULL) { > >>>>> err = ENOMEM; > >>>>> } else { > >>>>> // This call would be success because it is checked in > above > >>>>> GetFullPathNameW(unicode_path, full_path_len, result + > prefix_len - prefix_off, NULL); > >>>>> // Copy prefix > >>>>> memcpy(result, prefix, sizeof(WCHAR) * prefix_len); > >>>>> } > >>>>> } > >>>>> } else { > >>>>> size_t result_size = sizeof(WCHAR) * (result_len + > unicode_path_len); > >>>>> result = static_cast(os::malloc(result_size, > mtInternal)); > >>>>> if (result == NULL) { > >>>>> err = ENOMEM; > >>>>> } else { > >>>>> // Copy unicode path > >>>>> memcpy(result + prefix_len - prefix_off, unicode_path, > sizeof(WCHAR) * unicode_path_len); > >>>>> // Copy prefix > >>>>> memcpy(result, prefix, sizeof(WCHAR) * prefix_len); > >>>>> } > >>>>> } > >>>>> > >>>>> I think the code can be simplified if you do the fullpath version in > a separate function: > >>>>> > >>>>> if (needs_fullpath) { > >>>>> unicode_path = to_full_path(unicode_path, err, > &unicode_path_len); > >>>>> } > >>>>> > >>>>> size_t result_size = ...... > >>>>> > >>>>> This will do one more memcpy, but I think it's not performance > critical > >>>>> > >>>>> Thanks > >>>>> - Ioi > >>>>> > >>>>> > >>>>> On 3/3/20 4:19 PM, Yasumasa Suenaga wrote: > >>>>>> Thanks Ralf! > >>>>>> I fixed them. > >>>>>> > >>>>>> I need a Reviewer to push, so I uploaded new webrev: > >>>>>> > >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.03/ > >>>>>> > >>>>>> > >>>>>> Yasumasa > >>>>>> > >>>>>> > >>>>>> On 2020/03/03 23:25, Schmelter, Ralf wrote: > >>>>>>> Hi Yasumasa, > >>>>>>> > >>>>>>> thanks for your work. The code should now be able to handle any > path. > >>>>>>> > >>>>>>> Two small changes (no new webrev needed): > >>>>>>> > >>>>>>> size_t result_len = prefix_len + prefix_off + additional_space > >>>>>>> should be > >>>>>>> size_t result_len = prefix_len - prefix_off + additional_space > >>>>>>> > >>>>>>> And you should copy the prefix after you copied the filename, > since sometimes the prefix is required to overwrite the start of the > filename: > >>>>>>> > >>>>>>> // This call would be success because it is checked in above > >>>>>>> GetFullPathNameW(unicode_path, full_path_len, result + prefix_len > - prefix_off, NULL); > >>>>>>> // Copy prefix > >>>>>>> memcpy(result, prefix, sizeof(WCHAR) * prefix_len); > >>>>>>> > >>>>>>> and > >>>>>>> > >>>>>>> // Copy unicode path > >>>>>>> memcpy(result + prefix_len - prefix_off, unicode_path, > sizeof(WCHAR) * unicode_path_len); > >>>>>>> // Copy prefix > >>>>>>> memcpy(result, prefix, sizeof(WCHAR) * prefix_len); > >>>>>>> > >>>>>>> With these changes the os_windows gtest and the appcds jtreg tests > run without errors. > >>>>>>> > >>>>>>> Best regards, > >>>>>>> Ralf > >>>>>>> > >>>>> > >>> > > > From suenaga at oss.nttdata.com Thu Mar 5 06:28:18 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Thu, 5 Mar 2020 15:28:18 +0900 Subject: RFR: 8240197: Cannot start JVM when $JAVA_HOME includes CJK characters In-Reply-To: References: <6ec2f649-8999-fdd5-c2df-465925c3663b@oss.nttdata.com> <2d5148fc-7d71-d94c-d4a3-e6424a38b091@oss.nttdata.com> <37e2f672-3407-5b16-ab71-aa1e70e928db@oss.nttdata.com> <954f0bf1-91e2-3a1b-e2cb-e93d8343b0e6@oracle.com> <71ba4bad-05c9-042d-73dc-7f492fc1728a@oss.nttdata.com> <44d86ec7-aa6d-daf8-7200-534717cb4edb@oracle.com> <25aa10eb-6a8f-b416-02c2-b3fcd140076e@oss.nttdata.com> Message-ID: Hi Thomas, Thanks for your comment. I fixed them in new webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.06/ Yasumasa On 2020/03/05 14:43, Thomas St?fe wrote: > Hi Yasumasa, > > Had a glance over version 05. Some remarks: > > -- > > I dislike the reinterpret_cast<> and do not understand the need for it. We do not do this in other places with malloc. Strictly speaking this would be UB since it is not guaranteed for A=reinterpret_cast that A==B. > > In any way you should use NEW_C_HEAP_ARRAY and FREE_C_HEAP_ARRAY instead since those do the casting for you. > > They also exit the VM in case of OOM so no need to handle that manually. > > -- > > In wide_abs_unc_path: > > + ?// Need to allocate at least room for 3 characters, since os::native_path transforms C: to C:. > + ?size_t buf_size = 1 + MAX2((size_t)3, strlen(path)); > + ?char* buf = reinterpret_cast(os::malloc(buf_size, mtInternal)); > + ?if (buf == NULL) { > + ? ?vm_exit_out_of_memory(buf_size, OOM_MALLOC_ERROR, "wide_abs_unc_path"); > + ?} > + ?memcpy(buf, path, buf_size); > + ?os::native_path(buf); > > The memcpy is wrong. You have a rare chance of triggering a read fault since you read over the limit of path. Just use strcpy. > > -- > > The comment about "additional_space" confused me until I read the sources, sp I would suggest reforming it as such: > > -// additional_space is the number of additionally allocated wchars after the terminating L'\0'. > +// additional_space is the size of space, in wchar_t, the function will additionally add to the allocation of return buffer (such that the size of the returned buffer is at least wcslen(buf) + 1 + additional_space). > > -- > > I would also omit this comment: > > +// This is based on pathToNTPath() in io_util_md.cpp, but omits the optimizations for > +// short paths. > > I do not think anyone reading this code cares. > > -- > > Hmm, small style nit, sometimes we return errno_t and the actual return value is via output parameter, > > +static errno_t convert_to_unicode(char const* char_path, LPWSTR& unicode_path) { > > +static errno_t get_full_path(LPCWSTR unicode_path, LPWSTR& full_path) { > > and here its the other way around: > > +static wchar_t* wide_abs_unc_path(char const* path, errno_t & err, int additional_space = 0) { > > I do not care much, but it seems strange. > > -- > > +static errno_t convert_to_unicode(char const* char_path, LPWSTR& unicode_path) > > + ?LPWSTR unicode_path; > + ?err = convert_to_unicode(buf, unicode_path); > + ?os::free(buf); > > Sigh - I really do dislike passing output parameters via reference, since at the call site is is indistinguishable from an input parameter. > I know C++ programmers love this though so I won't fight this here... > > Could you please at least initialize unicode_path, since you do this in other places with output parameters too? > > -- > > The rest seems fine. > > Cheers, Thomas > > On Thu, Mar 5, 2020 at 1:18 AM Yasumasa Suenaga > wrote: > > Hi Ioi, > > How about this change? > > http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.05/ > > I already tried to use ResouceMark, but it was crashed because wide_abs_unc_path() would be called from outside of java thread at boot time. > This webrev uses os::free() instead of memory cleaner class, but I think it is simple because I tweaked memory lifecycle. > > > Thanks, > > Yasumasa > > > On 2020/03/05 6:56, Ioi Lam wrote: > > Hi Yasumasa, > > > > This version looks good! > > > > > > [0] One thing I forgot ... is it possible to use "ResourceMark rm" in this function? If so, you can allocate the temp arrays using NEW_RESOURCE_ARRAY and then don't need to worry about freeing them. > > > > > > [1] for this code > > > > 4202?? // This call would be success because it is checked in above > > 4203?? GetFullPathNameW(unicode_path, full_path_len, full_path, NULL); > > > > How about this instead: > > > >? ????? DWORD len = GetFullPathNameW(unicode_path, full_path_len, full_path, NULL); > >? ????? assert(len <= full_path_len, "length already checked above"); > > > > (same for the MultiByteToWideChar call in convert_to_unicode) > > > > > > [2] This is no longer needed > > > > 4240???? os::free(buf); > > > > > > [3] I think the following code is not needed .... > > > > 4269?? } else { > > 4270???? size_t converted_path_size = sizeof(WCHAR) * (wcslen(unicode_path) + 1); > > 4271???? converted_path = reinterpret_cast(os::malloc(converted_path_size, mtInternal)); > > 4272???? if (converted_path == NULL) { > > 4273?????? vm_exit_out_of_memory(buf_size, OOM_MALLOC_ERROR, "wide_abs_unc_path"); > > 4274???? } > > 4275???? memcpy(converted_path, unicode_path, converted_path_size); > > > > > > ... if you free the original buffer inside get_full_path (or use ResourceMark): > > > > static errno_t get_full_path(LPCWSTR& unicode_path) { > >? ?? LPCWSTR full_path = .....; > >? ?? .... > >? ?? free(unicode_path); > > unicode_path = full_path; > >? ?? return ERROR_SUCCESS; > > } > > > > [4] This looks good! Much more readable than the original code! > > > > 4285?? _snwprintf(result, result_len, L"%s%s", prefix, &converted_path[prefix_off]); > > > > > > Thanks > > - Ioi > > > > > > On 3/4/20 12:22 AM, Yasumasa Suenaga wrote: > >> Hi Ioi, > >> > >> Thanks for your comment. > >> How about this change? > >> > >> http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.04/ > >> > >> This webrev adds the function both converting to unicode and getting full path, > >> they would be called from wide_abs_unc_path(). > >> Also MemoryCleaner class which is introduced in this webrev frees memory auomatically in d'tor. > >> > >> > >> Thanks, > >> > >> Yasumasa > >> > >> > >> On 2020/03/04 16:05, Ioi Lam wrote: > >>> > >>> > >>> On 3/3/20 10:22 PM, Yasumasa Suenaga wrote: > >>>> Hi Ioi, > >>>> > >>>>> [1] I think the memory freeing can be done using a destructor, so you can simply return in case of error, without having a lot of "if" checks: > >>>> > >>>> What do you think to use `goto` statement like Linux kernel? > >>>> > >>>> > >>> I don't have a strong opinion but I think many/most HotSpot developers are against it. Also with goto, you need to store the return value in a variable, but with destructors, you can just say "return value", so the code is cleaner. > >>> > >>> Thanks > >>> - Ioi > >>> > >>>> Yasumasa > >>>> > >>>> > >>>> On 2020/03/04 14:50, Ioi Lam wrote: > >>>>> Hi Yasumasa, > >>>>> > >>>>> I think this function is getting too long and it's hard to figure out all the error conditions. > >>>>> > >>>>> (I feel sorry for you because the code wasn't pretty messy to begin with, but I think it's at a point that we need to do the cleanup. It took me like 30 minutes to understand what is going on) > >>>>> > >>>>> [0] I noticed you changed the return type to LPWSTR, but all the callers are using wchar_t*. Are these two types the same? > >>>>> > >>>>> ???? If you want to use LPWSTR within this function for consistency, maybe you can return the value as > >>>>> > >>>>> ???? return (wchar_t*)result; // LPWSTR and wchat_t* are the same type on Windows. > >>>>> > >>>>> ???? That way people unfamiliar with Windows (most of us) won't be confused. > >>>>> > >>>>> [1] I think the memory freeing can be done using a destructor, so you can simply return in case of error, without having a lot of "if" checks: > >>>>> > >>>>> ???? class State { > >>>>> ??????? errno_t & err; > >>>>> ?? ????? char* buf; > >>>>> ???????? LPWSTR unicode_path; > >>>>> LPWSTR result; > >>>>> > >>>>> ???????? State(errno_t & e) : buf(NULL), unicode_path(NULL), result(NULL), err(e) { > >>>>> ???????????? err = SUCCESS; > >>>>> ????????? } > >>>>> ???????? ~State() { > >>>>> ?????????? os::free(buf); > >>>>> ?? ? ? ? ? os::free(unicode_path); > >>>>> ?????????? if (err != ERROR_SUCCESS) { > >>>>> ????????????? os::free(result); > >>>>> ?????????? } > >>>>> ???????? } > >>>>> ???????? LPWSTR as_error(errno_t e) { > >>>>> ???????????? err = e; > >>>>> ???????????? return NULL; > >>>>> ???????? } > >>>>> ???? } st; > >>>>> > >>>>> > >>>>> ???? if ((st.buf = static_cast(os::malloc(buf_size, mtInternal))) == NULL) { > >>>>> ???????? return st.as_error(ENOMEM); > >>>>> ???? } > >>>>> > >>>>> ???? .... > >>>>> ???? return st.result; > >>>>> > >>>>> [2] Will it be possible to break the functon down into smaller pieces, so it's easier to understand what's going on? > >>>>> > >>>>> E.g., Maybe the code that? converts the C string into a unicode string can be moved into a separate function? > >>>>> > >>>>> ?? // Get required buffer size to convert to Unicode > >>>>> ?? int unicode_path_len = MultiByteToWideChar(CP_THREAD_ACP, > >>>>> MB_ERR_INVALID_CHARS, > >>>>> ????????????????????????????????????????????? buf, -1, > >>>>> ????????????????????????????????????????????? NULL, 0); > >>>>> ?? if (unicode_path_len == 0) { > >>>>> ???? os::free(buf); > >>>>> ???? err = EINVAL; > >>>>> ???? return NULL; > >>>>> ?? } > >>>>> > >>>>> ?? .... > >>>>> > >>>>> ?? LPWSTR unicode_path = static_cast(os::malloc(sizeof(WCHAR) * unicode_path_len, mtInternal)); > >>>>> ?? if (unicode_path == NULL) { > >>>>> ???? err = ENOMEM; > >>>>> ?? } else { > >>>>> ???? // This call would be success because it is checked in above > >>>>> ???? err = ERROR_SUCCESS; > >>>>> ???? MultiByteToWideChar(CP_THREAD_ACP, > >>>>> ???????????????????????? MB_ERR_INVALID_CHARS, > >>>>> ???????????????????????? buf, -1, > >>>>> ???????????????????????? unicode_path, unicode_path_len); > >>>>> > >>>>> > >>>>> > >>>>> [3] size_t result_len = prefix_len - prefix_off + additional_space; > >>>>> > >>>>> The variable name result_len is confusing. It's not the length of the result. > >>>>> > >>>>> [4] The old copying code was already hard to understand, and the new version now has 2 versions of the copying code. I think the reason is you want to avoid an extra memcpy by calling GetFullPathNameW directly on the result. > >>>>> > >>>>> ???? if (needs_fullpath) { > >>>>> ?????? // Get required buffer size to convert to full path > >>>>> ?????? DWORD full_path_len = GetFullPathNameW(unicode_path, 0, NULL, NULL); > >>>>> ?????? if (full_path_len == 0) { > >>>>> ???????? err = EINVAL; > >>>>> ?????? } else { > >>>>> ???????? size_t result_size = sizeof(WCHAR) * (result_len + full_path_len); > >>>>> ???????? result = static_cast(os::malloc(result_size, mtInternal)); > >>>>> ???????? if (result == NULL) { > >>>>> ?????????? err = ENOMEM; > >>>>> ???????? } else { > >>>>> ?????????? // This call would be success because it is checked in above > >>>>> ?????????? GetFullPathNameW(unicode_path, full_path_len, result + prefix_len - prefix_off, NULL); > >>>>> ?????????? // Copy prefix > >>>>> ?????????? memcpy(result, prefix, sizeof(WCHAR) * prefix_len); > >>>>> ???????? } > >>>>> ?????? } > >>>>> ???? } else { > >>>>> ?????? size_t result_size = sizeof(WCHAR) * (result_len + unicode_path_len); > >>>>> ?????? result = static_cast(os::malloc(result_size, mtInternal)); > >>>>> ?????? if (result == NULL) { > >>>>> ???????? err = ENOMEM; > >>>>> ?????? } else { > >>>>> ???????? // Copy unicode path > >>>>> ???????? memcpy(result + prefix_len - prefix_off, unicode_path, sizeof(WCHAR) * unicode_path_len); > >>>>> ???????? // Copy prefix > >>>>> ???????? memcpy(result, prefix, sizeof(WCHAR) * prefix_len); > >>>>> ?????? } > >>>>> ???? } > >>>>> > >>>>> I think the code can be simplified if you do the fullpath version in a separate function: > >>>>> > >>>>> ???? if (needs_fullpath) { > >>>>> ??????? unicode_path = to_full_path(unicode_path, err, &unicode_path_len); > >>>>> ???? } > >>>>> > >>>>> ???? size_t result_size = ...... > >>>>> > >>>>> This will do one more memcpy, but I think it's not performance critical > >>>>> > >>>>> Thanks > >>>>> - Ioi > >>>>> > >>>>> > >>>>> On 3/3/20 4:19 PM, Yasumasa Suenaga wrote: > >>>>>> Thanks Ralf! > >>>>>> I fixed them. > >>>>>> > >>>>>> I need a Reviewer to push, so I uploaded new webrev: > >>>>>> > >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.03/ > >>>>>> > >>>>>> > >>>>>> Yasumasa > >>>>>> > >>>>>> > >>>>>> On 2020/03/03 23:25, Schmelter, Ralf wrote: > >>>>>>> Hi Yasumasa, > >>>>>>> > >>>>>>> thanks for your work. The code should now be able to handle any path. > >>>>>>> > >>>>>>> Two small changes (no new webrev needed): > >>>>>>> > >>>>>>> size_t result_len = prefix_len + prefix_off + additional_space > >>>>>>> should be > >>>>>>> size_t result_len = prefix_len - prefix_off + additional_space > >>>>>>> > >>>>>>> And you should copy the prefix after you copied the filename, since sometimes the prefix is required to overwrite the start of the filename: > >>>>>>> > >>>>>>> // This call would be success because it is checked in above > >>>>>>> GetFullPathNameW(unicode_path, full_path_len, result + prefix_len - prefix_off, NULL); > >>>>>>> // Copy prefix > >>>>>>> memcpy(result, prefix, sizeof(WCHAR) * prefix_len); > >>>>>>> > >>>>>>> and > >>>>>>> > >>>>>>> // Copy unicode path > >>>>>>> memcpy(result + prefix_len - prefix_off, unicode_path, sizeof(WCHAR) * unicode_path_len); > >>>>>>> // Copy prefix > >>>>>>> memcpy(result, prefix, sizeof(WCHAR) * prefix_len); > >>>>>>> > >>>>>>> With these changes the os_windows gtest and the appcds jtreg tests run without errors. > >>>>>>> > >>>>>>> Best regards, > >>>>>>> Ralf > >>>>>>> > >>>>> > >>> > > > From thomas.stuefe at gmail.com Thu Mar 5 06:38:38 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 5 Mar 2020 07:38:38 +0100 Subject: RFR: 8240197: Cannot start JVM when $JAVA_HOME includes CJK characters In-Reply-To: References: <6ec2f649-8999-fdd5-c2df-465925c3663b@oss.nttdata.com> <2d5148fc-7d71-d94c-d4a3-e6424a38b091@oss.nttdata.com> <37e2f672-3407-5b16-ab71-aa1e70e928db@oss.nttdata.com> <954f0bf1-91e2-3a1b-e2cb-e93d8343b0e6@oracle.com> <71ba4bad-05c9-042d-73dc-7f492fc1728a@oss.nttdata.com> <44d86ec7-aa6d-daf8-7200-534717cb4edb@oracle.com> <25aa10eb-6a8f-b416-02c2-b3fcd140076e@oss.nttdata.com> Message-ID: Looks good to me now, thanks for the changes and your perseverance! All good from my side. ..Thomas On Thu, Mar 5, 2020 at 7:28 AM Yasumasa Suenaga wrote: > Hi Thomas, > > Thanks for your comment. > I fixed them in new webrev: > > http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.06/ > > > Yasumasa > > > On 2020/03/05 14:43, Thomas St?fe wrote: > > Hi Yasumasa, > > > > Had a glance over version 05. Some remarks: > > > > -- > > > > I dislike the reinterpret_cast<> and do not understand the need for it. > We do not do this in other places with malloc. Strictly speaking this would > be UB since it is not guaranteed for A=reinterpret_cast that A==B. > > > > In any way you should use NEW_C_HEAP_ARRAY and FREE_C_HEAP_ARRAY instead > since those do the casting for you. > > > > They also exit the VM in case of OOM so no need to handle that manually. > > > > -- > > > > In wide_abs_unc_path: > > > > + // Need to allocate at least room for 3 characters, since > os::native_path transforms C: to C:. > > + size_t buf_size = 1 + MAX2((size_t)3, strlen(path)); > > + char* buf = reinterpret_cast(os::malloc(buf_size, mtInternal)); > > + if (buf == NULL) { > > + vm_exit_out_of_memory(buf_size, OOM_MALLOC_ERROR, > "wide_abs_unc_path"); > > + } > > + memcpy(buf, path, buf_size); > > + os::native_path(buf); > > > > The memcpy is wrong. You have a rare chance of triggering a read fault > since you read over the limit of path. Just use strcpy. > > > > -- > > > > The comment about "additional_space" confused me until I read the > sources, sp I would suggest reforming it as such: > > > > -// additional_space is the number of additionally allocated wchars > after the terminating L'\0'. > > +// additional_space is the size of space, in wchar_t, the function will > additionally add to the allocation of return buffer (such that the size of > the returned buffer is at least wcslen(buf) + 1 + additional_space). > > > > -- > > > > I would also omit this comment: > > > > +// This is based on pathToNTPath() in io_util_md.cpp, but omits the > optimizations for > > +// short paths. > > > > I do not think anyone reading this code cares. > > > > -- > > > > Hmm, small style nit, sometimes we return errno_t and the actual return > value is via output parameter, > > > > +static errno_t convert_to_unicode(char const* char_path, LPWSTR& > unicode_path) { > > > > +static errno_t get_full_path(LPCWSTR unicode_path, LPWSTR& full_path) { > > > > and here its the other way around: > > > > +static wchar_t* wide_abs_unc_path(char const* path, errno_t & err, int > additional_space = 0) { > > > > I do not care much, but it seems strange. > > > > -- > > > > +static errno_t convert_to_unicode(char const* char_path, LPWSTR& > unicode_path) > > > > + LPWSTR unicode_path; > > + err = convert_to_unicode(buf, unicode_path); > > + os::free(buf); > > > > Sigh - I really do dislike passing output parameters via reference, > since at the call site is is indistinguishable from an input parameter. > > I know C++ programmers love this though so I won't fight this here... > > > > Could you please at least initialize unicode_path, since you do this in > other places with output parameters too? > > > > -- > > > > The rest seems fine. > > > > Cheers, Thomas > > > > On Thu, Mar 5, 2020 at 1:18 AM Yasumasa Suenaga > wrote: > > > > Hi Ioi, > > > > How about this change? > > > > http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.05/ > > > > I already tried to use ResouceMark, but it was crashed because > wide_abs_unc_path() would be called from outside of java thread at boot > time. > > This webrev uses os::free() instead of memory cleaner class, but I > think it is simple because I tweaked memory lifecycle. > > > > > > Thanks, > > > > Yasumasa > > > > > > On 2020/03/05 6:56, Ioi Lam wrote: > > > Hi Yasumasa, > > > > > > This version looks good! > > > > > > > > > [0] One thing I forgot ... is it possible to use "ResourceMark > rm" in this function? If so, you can allocate the temp arrays using > NEW_RESOURCE_ARRAY and then don't need to worry about freeing them. > > > > > > > > > [1] for this code > > > > > > 4202 // This call would be success because it is checked in > above > > > 4203 GetFullPathNameW(unicode_path, full_path_len, full_path, > NULL); > > > > > > How about this instead: > > > > > > DWORD len = GetFullPathNameW(unicode_path, full_path_len, > full_path, NULL); > > > assert(len <= full_path_len, "length already checked > above"); > > > > > > (same for the MultiByteToWideChar call in convert_to_unicode) > > > > > > > > > [2] This is no longer needed > > > > > > 4240 os::free(buf); > > > > > > > > > [3] I think the following code is not needed .... > > > > > > 4269 } else { > > > 4270 size_t converted_path_size = sizeof(WCHAR) * > (wcslen(unicode_path) + 1); > > > 4271 converted_path = > reinterpret_cast(os::malloc(converted_path_size, mtInternal)); > > > 4272 if (converted_path == NULL) { > > > 4273 vm_exit_out_of_memory(buf_size, OOM_MALLOC_ERROR, > "wide_abs_unc_path"); > > > 4274 } > > > 4275 memcpy(converted_path, unicode_path, > converted_path_size); > > > > > > > > > ... if you free the original buffer inside get_full_path (or use > ResourceMark): > > > > > > static errno_t get_full_path(LPCWSTR& unicode_path) { > > > LPCWSTR full_path = .....; > > > .... > > > free(unicode_path); > > > unicode_path = full_path; > > > return ERROR_SUCCESS; > > > } > > > > > > [4] This looks good! Much more readable than the original code! > > > > > > 4285 _snwprintf(result, result_len, L"%s%s", prefix, > &converted_path[prefix_off]); > > > > > > > > > Thanks > > > - Ioi > > > > > > > > > On 3/4/20 12:22 AM, Yasumasa Suenaga wrote: > > >> Hi Ioi, > > >> > > >> Thanks for your comment. > > >> How about this change? > > >> > > >> http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.04/ > > >> > > >> This webrev adds the function both converting to unicode and > getting full path, > > >> they would be called from wide_abs_unc_path(). > > >> Also MemoryCleaner class which is introduced in this webrev > frees memory auomatically in d'tor. > > >> > > >> > > >> Thanks, > > >> > > >> Yasumasa > > >> > > >> > > >> On 2020/03/04 16:05, Ioi Lam wrote: > > >>> > > >>> > > >>> On 3/3/20 10:22 PM, Yasumasa Suenaga wrote: > > >>>> Hi Ioi, > > >>>> > > >>>>> [1] I think the memory freeing can be done using a > destructor, so you can simply return in case of error, without having a lot > of "if" checks: > > >>>> > > >>>> What do you think to use `goto` statement like Linux kernel? > > >>>> > > >>>> > > >>> I don't have a strong opinion but I think many/most HotSpot > developers are against it. Also with goto, you need to store the return > value in a variable, but with destructors, you can just say "return value", > so the code is cleaner. > > >>> > > >>> Thanks > > >>> - Ioi > > >>> > > >>>> Yasumasa > > >>>> > > >>>> > > >>>> On 2020/03/04 14:50, Ioi Lam wrote: > > >>>>> Hi Yasumasa, > > >>>>> > > >>>>> I think this function is getting too long and it's hard to > figure out all the error conditions. > > >>>>> > > >>>>> (I feel sorry for you because the code wasn't pretty messy to > begin with, but I think it's at a point that we need to do the cleanup. It > took me like 30 minutes to understand what is going on) > > >>>>> > > >>>>> [0] I noticed you changed the return type to LPWSTR, but all > the callers are using wchar_t*. Are these two types the same? > > >>>>> > > >>>>> If you want to use LPWSTR within this function for > consistency, maybe you can return the value as > > >>>>> > > >>>>> return (wchar_t*)result; // LPWSTR and wchat_t* are the > same type on Windows. > > >>>>> > > >>>>> That way people unfamiliar with Windows (most of us) > won't be confused. > > >>>>> > > >>>>> [1] I think the memory freeing can be done using a > destructor, so you can simply return in case of error, without having a lot > of "if" checks: > > >>>>> > > >>>>> class State { > > >>>>> errno_t & err; > > >>>>> char* buf; > > >>>>> LPWSTR unicode_path; > > >>>>> LPWSTR result; > > >>>>> > > >>>>> State(errno_t & e) : buf(NULL), unicode_path(NULL), > result(NULL), err(e) { > > >>>>> err = SUCCESS; > > >>>>> } > > >>>>> ~State() { > > >>>>> os::free(buf); > > >>>>> os::free(unicode_path); > > >>>>> if (err != ERROR_SUCCESS) { > > >>>>> os::free(result); > > >>>>> } > > >>>>> } > > >>>>> LPWSTR as_error(errno_t e) { > > >>>>> err = e; > > >>>>> return NULL; > > >>>>> } > > >>>>> } st; > > >>>>> > > >>>>> > > >>>>> if ((st.buf = static_cast(os::malloc(buf_size, > mtInternal))) == NULL) { > > >>>>> return st.as_error(ENOMEM); > > >>>>> } > > >>>>> > > >>>>> .... > > >>>>> return st.result; > > >>>>> > > >>>>> [2] Will it be possible to break the functon down into > smaller pieces, so it's easier to understand what's going on? > > >>>>> > > >>>>> E.g., Maybe the code that converts the C string into a > unicode string can be moved into a separate function? > > >>>>> > > >>>>> // Get required buffer size to convert to Unicode > > >>>>> int unicode_path_len = MultiByteToWideChar(CP_THREAD_ACP, > > >>>>> MB_ERR_INVALID_CHARS, > > >>>>> buf, -1, > > >>>>> NULL, 0); > > >>>>> if (unicode_path_len == 0) { > > >>>>> os::free(buf); > > >>>>> err = EINVAL; > > >>>>> return NULL; > > >>>>> } > > >>>>> > > >>>>> .... > > >>>>> > > >>>>> LPWSTR unicode_path = > static_cast(os::malloc(sizeof(WCHAR) * unicode_path_len, > mtInternal)); > > >>>>> if (unicode_path == NULL) { > > >>>>> err = ENOMEM; > > >>>>> } else { > > >>>>> // This call would be success because it is checked in > above > > >>>>> err = ERROR_SUCCESS; > > >>>>> MultiByteToWideChar(CP_THREAD_ACP, > > >>>>> MB_ERR_INVALID_CHARS, > > >>>>> buf, -1, > > >>>>> unicode_path, unicode_path_len); > > >>>>> > > >>>>> > > >>>>> > > >>>>> [3] size_t result_len = prefix_len - prefix_off + > additional_space; > > >>>>> > > >>>>> The variable name result_len is confusing. It's not the > length of the result. > > >>>>> > > >>>>> [4] The old copying code was already hard to understand, and > the new version now has 2 versions of the copying code. I think the reason > is you want to avoid an extra memcpy by calling GetFullPathNameW directly > on the result. > > >>>>> > > >>>>> if (needs_fullpath) { > > >>>>> // Get required buffer size to convert to full path > > >>>>> DWORD full_path_len = GetFullPathNameW(unicode_path, > 0, NULL, NULL); > > >>>>> if (full_path_len == 0) { > > >>>>> err = EINVAL; > > >>>>> } else { > > >>>>> size_t result_size = sizeof(WCHAR) * (result_len + > full_path_len); > > >>>>> result = static_cast(os::malloc(result_size, > mtInternal)); > > >>>>> if (result == NULL) { > > >>>>> err = ENOMEM; > > >>>>> } else { > > >>>>> // This call would be success because it is > checked in above > > >>>>> GetFullPathNameW(unicode_path, full_path_len, > result + prefix_len - prefix_off, NULL); > > >>>>> // Copy prefix > > >>>>> memcpy(result, prefix, sizeof(WCHAR) * prefix_len); > > >>>>> } > > >>>>> } > > >>>>> } else { > > >>>>> size_t result_size = sizeof(WCHAR) * (result_len + > unicode_path_len); > > >>>>> result = static_cast(os::malloc(result_size, > mtInternal)); > > >>>>> if (result == NULL) { > > >>>>> err = ENOMEM; > > >>>>> } else { > > >>>>> // Copy unicode path > > >>>>> memcpy(result + prefix_len - prefix_off, > unicode_path, sizeof(WCHAR) * unicode_path_len); > > >>>>> // Copy prefix > > >>>>> memcpy(result, prefix, sizeof(WCHAR) * prefix_len); > > >>>>> } > > >>>>> } > > >>>>> > > >>>>> I think the code can be simplified if you do the fullpath > version in a separate function: > > >>>>> > > >>>>> if (needs_fullpath) { > > >>>>> unicode_path = to_full_path(unicode_path, err, > &unicode_path_len); > > >>>>> } > > >>>>> > > >>>>> size_t result_size = ...... > > >>>>> > > >>>>> This will do one more memcpy, but I think it's not > performance critical > > >>>>> > > >>>>> Thanks > > >>>>> - Ioi > > >>>>> > > >>>>> > > >>>>> On 3/3/20 4:19 PM, Yasumasa Suenaga wrote: > > >>>>>> Thanks Ralf! > > >>>>>> I fixed them. > > >>>>>> > > >>>>>> I need a Reviewer to push, so I uploaded new webrev: > > >>>>>> > > >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.03/ > > >>>>>> > > >>>>>> > > >>>>>> Yasumasa > > >>>>>> > > >>>>>> > > >>>>>> On 2020/03/03 23:25, Schmelter, Ralf wrote: > > >>>>>>> Hi Yasumasa, > > >>>>>>> > > >>>>>>> thanks for your work. The code should now be able to handle > any path. > > >>>>>>> > > >>>>>>> Two small changes (no new webrev needed): > > >>>>>>> > > >>>>>>> size_t result_len = prefix_len + prefix_off + > additional_space > > >>>>>>> should be > > >>>>>>> size_t result_len = prefix_len - prefix_off + > additional_space > > >>>>>>> > > >>>>>>> And you should copy the prefix after you copied the > filename, since sometimes the prefix is required to overwrite the start of > the filename: > > >>>>>>> > > >>>>>>> // This call would be success because it is checked in above > > >>>>>>> GetFullPathNameW(unicode_path, full_path_len, result + > prefix_len - prefix_off, NULL); > > >>>>>>> // Copy prefix > > >>>>>>> memcpy(result, prefix, sizeof(WCHAR) * prefix_len); > > >>>>>>> > > >>>>>>> and > > >>>>>>> > > >>>>>>> // Copy unicode path > > >>>>>>> memcpy(result + prefix_len - prefix_off, unicode_path, > sizeof(WCHAR) * unicode_path_len); > > >>>>>>> // Copy prefix > > >>>>>>> memcpy(result, prefix, sizeof(WCHAR) * prefix_len); > > >>>>>>> > > >>>>>>> With these changes the os_windows gtest and the appcds > jtreg tests run without errors. > > >>>>>>> > > >>>>>>> Best regards, > > >>>>>>> Ralf > > >>>>>>> > > >>>>> > > >>> > > > > > > From suenaga at oss.nttdata.com Thu Mar 5 06:41:06 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Thu, 5 Mar 2020 15:41:06 +0900 Subject: RFR: 8240197: Cannot start JVM when $JAVA_HOME includes CJK characters In-Reply-To: References: <6ec2f649-8999-fdd5-c2df-465925c3663b@oss.nttdata.com> <2d5148fc-7d71-d94c-d4a3-e6424a38b091@oss.nttdata.com> <37e2f672-3407-5b16-ab71-aa1e70e928db@oss.nttdata.com> <954f0bf1-91e2-3a1b-e2cb-e93d8343b0e6@oracle.com> <71ba4bad-05c9-042d-73dc-7f492fc1728a@oss.nttdata.com> <44d86ec7-aa6d-daf8-7200-534717cb4edb@oracle.com> <25aa10eb-6a8f-b416-02c2-b3fcd140076e@oss.nttdata.com> Message-ID: Thanks Thomas! I will push it if Ralf is ok to this change (webrev.06). Yasumasa On 2020/03/05 15:38, Thomas St?fe wrote: > Looks good to me now, thanks for the changes and your perseverance! > > All good from my side. > > ..Thomas > > On Thu, Mar 5, 2020 at 7:28 AM Yasumasa Suenaga > wrote: > > Hi Thomas, > > Thanks for your comment. > I fixed them in new webrev: > > http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.06/ > > > Yasumasa > > > On 2020/03/05 14:43, Thomas St?fe wrote: > > Hi Yasumasa, > > > > Had a glance over version 05. Some remarks: > > > > -- > > > > I dislike the reinterpret_cast<> and do not understand the need for it. We do not do this in other places with malloc. Strictly speaking this would be UB since it is not guaranteed for A=reinterpret_cast that A==B. > > > > In any way you should use NEW_C_HEAP_ARRAY and FREE_C_HEAP_ARRAY instead since those do the casting for you. > > > > They also exit the VM in case of OOM so no need to handle that manually. > > > > -- > > > > In wide_abs_unc_path: > > > > + ?// Need to allocate at least room for 3 characters, since os::native_path transforms C: to C:. > > + ?size_t buf_size = 1 + MAX2((size_t)3, strlen(path)); > > + ?char* buf = reinterpret_cast(os::malloc(buf_size, mtInternal)); > > + ?if (buf == NULL) { > > + ? ?vm_exit_out_of_memory(buf_size, OOM_MALLOC_ERROR, "wide_abs_unc_path"); > > + ?} > > + ?memcpy(buf, path, buf_size); > > + ?os::native_path(buf); > > > > The memcpy is wrong. You have a rare chance of triggering a read fault since you read over the limit of path. Just use strcpy. > > > > -- > > > > The comment about "additional_space" confused me until I read the sources, sp I would suggest reforming it as such: > > > > -// additional_space is the number of additionally allocated wchars after the terminating L'\0'. > > +// additional_space is the size of space, in wchar_t, the function will additionally add to the allocation of return buffer (such that the size of the returned buffer is at least wcslen(buf) + 1 + additional_space). > > > > -- > > > > I would also omit this comment: > > > > +// This is based on pathToNTPath() in io_util_md.cpp, but omits the optimizations for > > +// short paths. > > > > I do not think anyone reading this code cares. > > > > -- > > > > Hmm, small style nit, sometimes we return errno_t and the actual return value is via output parameter, > > > > +static errno_t convert_to_unicode(char const* char_path, LPWSTR& unicode_path) { > > > > +static errno_t get_full_path(LPCWSTR unicode_path, LPWSTR& full_path) { > > > > and here its the other way around: > > > > +static wchar_t* wide_abs_unc_path(char const* path, errno_t & err, int additional_space = 0) { > > > > I do not care much, but it seems strange. > > > > -- > > > > +static errno_t convert_to_unicode(char const* char_path, LPWSTR& unicode_path) > > > > + ?LPWSTR unicode_path; > > + ?err = convert_to_unicode(buf, unicode_path); > > + ?os::free(buf); > > > > Sigh - I really do dislike passing output parameters via reference, since at the call site is is indistinguishable from an input parameter. > > I know C++ programmers love this though so I won't fight this here... > > > > Could you please at least initialize unicode_path, since you do this in other places with output parameters too? > > > > -- > > > > The rest seems fine. > > > > Cheers, Thomas > > > > On Thu, Mar 5, 2020 at 1:18 AM Yasumasa Suenaga >> wrote: > > > >? ? ?Hi Ioi, > > > >? ? ?How about this change? > > > > http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.05/ > > > >? ? ?I already tried to use ResouceMark, but it was crashed because wide_abs_unc_path() would be called from outside of java thread at boot time. > >? ? ?This webrev uses os::free() instead of memory cleaner class, but I think it is simple because I tweaked memory lifecycle. > > > > > >? ? ?Thanks, > > > >? ? ?Yasumasa > > > > > >? ? ?On 2020/03/05 6:56, Ioi Lam wrote: > >? ? ? > Hi Yasumasa, > >? ? ? > > >? ? ? > This version looks good! > >? ? ? > > >? ? ? > > >? ? ? > [0] One thing I forgot ... is it possible to use "ResourceMark rm" in this function? If so, you can allocate the temp arrays using NEW_RESOURCE_ARRAY and then don't need to worry about freeing them. > >? ? ? > > >? ? ? > > >? ? ? > [1] for this code > >? ? ? > > >? ? ? > 4202?? // This call would be success because it is checked in above > >? ? ? > 4203?? GetFullPathNameW(unicode_path, full_path_len, full_path, NULL); > >? ? ? > > >? ? ? > How about this instead: > >? ? ? > > >? ? ? >? ????? DWORD len = GetFullPathNameW(unicode_path, full_path_len, full_path, NULL); > >? ? ? >? ????? assert(len <= full_path_len, "length already checked above"); > >? ? ? > > >? ? ? > (same for the MultiByteToWideChar call in convert_to_unicode) > >? ? ? > > >? ? ? > > >? ? ? > [2] This is no longer needed > >? ? ? > > >? ? ? > 4240???? os::free(buf); > >? ? ? > > >? ? ? > > >? ? ? > [3] I think the following code is not needed .... > >? ? ? > > >? ? ? > 4269?? } else { > >? ? ? > 4270???? size_t converted_path_size = sizeof(WCHAR) * (wcslen(unicode_path) + 1); > >? ? ? > 4271???? converted_path = reinterpret_cast(os::malloc(converted_path_size, mtInternal)); > >? ? ? > 4272???? if (converted_path == NULL) { > >? ? ? > 4273?????? vm_exit_out_of_memory(buf_size, OOM_MALLOC_ERROR, "wide_abs_unc_path"); > >? ? ? > 4274???? } > >? ? ? > 4275???? memcpy(converted_path, unicode_path, converted_path_size); > >? ? ? > > >? ? ? > > >? ? ? > ... if you free the original buffer inside get_full_path (or use ResourceMark): > >? ? ? > > >? ? ? > static errno_t get_full_path(LPCWSTR& unicode_path) { > >? ? ? >? ?? LPCWSTR full_path = .....; > >? ? ? >? ?? .... > >? ? ? >? ?? free(unicode_path); > >? ? ? > unicode_path = full_path; > >? ? ? >? ?? return ERROR_SUCCESS; > >? ? ? > } > >? ? ? > > >? ? ? > [4] This looks good! Much more readable than the original code! > >? ? ? > > >? ? ? > 4285?? _snwprintf(result, result_len, L"%s%s", prefix, &converted_path[prefix_off]); > >? ? ? > > >? ? ? > > >? ? ? > Thanks > >? ? ? > - Ioi > >? ? ? > > >? ? ? > > >? ? ? > On 3/4/20 12:22 AM, Yasumasa Suenaga wrote: > >? ? ? >> Hi Ioi, > >? ? ? >> > >? ? ? >> Thanks for your comment. > >? ? ? >> How about this change? > >? ? ? >> > >? ? ? >> http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.04/ > >? ? ? >> > >? ? ? >> This webrev adds the function both converting to unicode and getting full path, > >? ? ? >> they would be called from wide_abs_unc_path(). > >? ? ? >> Also MemoryCleaner class which is introduced in this webrev frees memory auomatically in d'tor. > >? ? ? >> > >? ? ? >> > >? ? ? >> Thanks, > >? ? ? >> > >? ? ? >> Yasumasa > >? ? ? >> > >? ? ? >> > >? ? ? >> On 2020/03/04 16:05, Ioi Lam wrote: > >? ? ? >>> > >? ? ? >>> > >? ? ? >>> On 3/3/20 10:22 PM, Yasumasa Suenaga wrote: > >? ? ? >>>> Hi Ioi, > >? ? ? >>>> > >? ? ? >>>>> [1] I think the memory freeing can be done using a destructor, so you can simply return in case of error, without having a lot of "if" checks: > >? ? ? >>>> > >? ? ? >>>> What do you think to use `goto` statement like Linux kernel? > >? ? ? >>>> > >? ? ? >>>> > >? ? ? >>> I don't have a strong opinion but I think many/most HotSpot developers are against it. Also with goto, you need to store the return value in a variable, but with destructors, you can just say "return value", so the code is cleaner. > >? ? ? >>> > >? ? ? >>> Thanks > >? ? ? >>> - Ioi > >? ? ? >>> > >? ? ? >>>> Yasumasa > >? ? ? >>>> > >? ? ? >>>> > >? ? ? >>>> On 2020/03/04 14:50, Ioi Lam wrote: > >? ? ? >>>>> Hi Yasumasa, > >? ? ? >>>>> > >? ? ? >>>>> I think this function is getting too long and it's hard to figure out all the error conditions. > >? ? ? >>>>> > >? ? ? >>>>> (I feel sorry for you because the code wasn't pretty messy to begin with, but I think it's at a point that we need to do the cleanup. It took me like 30 minutes to understand what is going on) > >? ? ? >>>>> > >? ? ? >>>>> [0] I noticed you changed the return type to LPWSTR, but all the callers are using wchar_t*. Are these two types the same? > >? ? ? >>>>> > >? ? ? >>>>> ???? If you want to use LPWSTR within this function for consistency, maybe you can return the value as > >? ? ? >>>>> > >? ? ? >>>>> ???? return (wchar_t*)result; // LPWSTR and wchat_t* are the same type on Windows. > >? ? ? >>>>> > >? ? ? >>>>> ???? That way people unfamiliar with Windows (most of us) won't be confused. > >? ? ? >>>>> > >? ? ? >>>>> [1] I think the memory freeing can be done using a destructor, so you can simply return in case of error, without having a lot of "if" checks: > >? ? ? >>>>> > >? ? ? >>>>> ???? class State { > >? ? ? >>>>> ??????? errno_t & err; > >? ? ? >>>>> ?? ????? char* buf; > >? ? ? >>>>> ???????? LPWSTR unicode_path; > >? ? ? >>>>> LPWSTR result; > >? ? ? >>>>> > >? ? ? >>>>> ???????? State(errno_t & e) : buf(NULL), unicode_path(NULL), result(NULL), err(e) { > >? ? ? >>>>> ???????????? err = SUCCESS; > >? ? ? >>>>> ????????? } > >? ? ? >>>>> ???????? ~State() { > >? ? ? >>>>> ?????????? os::free(buf); > >? ? ? >>>>> ?? ? ? ? ? os::free(unicode_path); > >? ? ? >>>>> ?????????? if (err != ERROR_SUCCESS) { > >? ? ? >>>>> ????????????? os::free(result); > >? ? ? >>>>> ?????????? } > >? ? ? >>>>> ???????? } > >? ? ? >>>>> ???????? LPWSTR as_error(errno_t e) { > >? ? ? >>>>> ???????????? err = e; > >? ? ? >>>>> ???????????? return NULL; > >? ? ? >>>>> ???????? } > >? ? ? >>>>> ???? } st; > >? ? ? >>>>> > >? ? ? >>>>> > >? ? ? >>>>> ???? if ((st.buf = static_cast(os::malloc(buf_size, mtInternal))) == NULL) { > >? ? ? >>>>> ???????? return st.as_error(ENOMEM); > >? ? ? >>>>> ???? } > >? ? ? >>>>> > >? ? ? >>>>> ???? .... > >? ? ? >>>>> ???? return st.result; > >? ? ? >>>>> > >? ? ? >>>>> [2] Will it be possible to break the functon down into smaller pieces, so it's easier to understand what's going on? > >? ? ? >>>>> > >? ? ? >>>>> E.g., Maybe the code that? converts the C string into a unicode string can be moved into a separate function? > >? ? ? >>>>> > >? ? ? >>>>> ?? // Get required buffer size to convert to Unicode > >? ? ? >>>>> ?? int unicode_path_len = MultiByteToWideChar(CP_THREAD_ACP, > >? ? ? >>>>> MB_ERR_INVALID_CHARS, > >? ? ? >>>>> ????????????????????????????????????????????? buf, -1, > >? ? ? >>>>> ????????????????????????????????????????????? NULL, 0); > >? ? ? >>>>> ?? if (unicode_path_len == 0) { > >? ? ? >>>>> ???? os::free(buf); > >? ? ? >>>>> ???? err = EINVAL; > >? ? ? >>>>> ???? return NULL; > >? ? ? >>>>> ?? } > >? ? ? >>>>> > >? ? ? >>>>> ?? .... > >? ? ? >>>>> > >? ? ? >>>>> ?? LPWSTR unicode_path = static_cast(os::malloc(sizeof(WCHAR) * unicode_path_len, mtInternal)); > >? ? ? >>>>> ?? if (unicode_path == NULL) { > >? ? ? >>>>> ???? err = ENOMEM; > >? ? ? >>>>> ?? } else { > >? ? ? >>>>> ???? // This call would be success because it is checked in above > >? ? ? >>>>> ???? err = ERROR_SUCCESS; > >? ? ? >>>>> ???? MultiByteToWideChar(CP_THREAD_ACP, > >? ? ? >>>>> ???????????????????????? MB_ERR_INVALID_CHARS, > >? ? ? >>>>> ???????????????????????? buf, -1, > >? ? ? >>>>> ???????????????????????? unicode_path, unicode_path_len); > >? ? ? >>>>> > >? ? ? >>>>> > >? ? ? >>>>> > >? ? ? >>>>> [3] size_t result_len = prefix_len - prefix_off + additional_space; > >? ? ? >>>>> > >? ? ? >>>>> The variable name result_len is confusing. It's not the length of the result. > >? ? ? >>>>> > >? ? ? >>>>> [4] The old copying code was already hard to understand, and the new version now has 2 versions of the copying code. I think the reason is you want to avoid an extra memcpy by calling GetFullPathNameW directly on the result. > >? ? ? >>>>> > >? ? ? >>>>> ???? if (needs_fullpath) { > >? ? ? >>>>> ?????? // Get required buffer size to convert to full path > >? ? ? >>>>> ?????? DWORD full_path_len = GetFullPathNameW(unicode_path, 0, NULL, NULL); > >? ? ? >>>>> ?????? if (full_path_len == 0) { > >? ? ? >>>>> ???????? err = EINVAL; > >? ? ? >>>>> ?????? } else { > >? ? ? >>>>> ???????? size_t result_size = sizeof(WCHAR) * (result_len + full_path_len); > >? ? ? >>>>> ???????? result = static_cast(os::malloc(result_size, mtInternal)); > >? ? ? >>>>> ???????? if (result == NULL) { > >? ? ? >>>>> ?????????? err = ENOMEM; > >? ? ? >>>>> ???????? } else { > >? ? ? >>>>> ?????????? // This call would be success because it is checked in above > >? ? ? >>>>> ?????????? GetFullPathNameW(unicode_path, full_path_len, result + prefix_len - prefix_off, NULL); > >? ? ? >>>>> ?????????? // Copy prefix > >? ? ? >>>>> ?????????? memcpy(result, prefix, sizeof(WCHAR) * prefix_len); > >? ? ? >>>>> ???????? } > >? ? ? >>>>> ?????? } > >? ? ? >>>>> ???? } else { > >? ? ? >>>>> ?????? size_t result_size = sizeof(WCHAR) * (result_len + unicode_path_len); > >? ? ? >>>>> ?????? result = static_cast(os::malloc(result_size, mtInternal)); > >? ? ? >>>>> ?????? if (result == NULL) { > >? ? ? >>>>> ???????? err = ENOMEM; > >? ? ? >>>>> ?????? } else { > >? ? ? >>>>> ???????? // Copy unicode path > >? ? ? >>>>> ???????? memcpy(result + prefix_len - prefix_off, unicode_path, sizeof(WCHAR) * unicode_path_len); > >? ? ? >>>>> ???????? // Copy prefix > >? ? ? >>>>> ???????? memcpy(result, prefix, sizeof(WCHAR) * prefix_len); > >? ? ? >>>>> ?????? } > >? ? ? >>>>> ???? } > >? ? ? >>>>> > >? ? ? >>>>> I think the code can be simplified if you do the fullpath version in a separate function: > >? ? ? >>>>> > >? ? ? >>>>> ???? if (needs_fullpath) { > >? ? ? >>>>> ??????? unicode_path = to_full_path(unicode_path, err, &unicode_path_len); > >? ? ? >>>>> ???? } > >? ? ? >>>>> > >? ? ? >>>>> ???? size_t result_size = ...... > >? ? ? >>>>> > >? ? ? >>>>> This will do one more memcpy, but I think it's not performance critical > >? ? ? >>>>> > >? ? ? >>>>> Thanks > >? ? ? >>>>> - Ioi > >? ? ? >>>>> > >? ? ? >>>>> > >? ? ? >>>>> On 3/3/20 4:19 PM, Yasumasa Suenaga wrote: > >? ? ? >>>>>> Thanks Ralf! > >? ? ? >>>>>> I fixed them. > >? ? ? >>>>>> > >? ? ? >>>>>> I need a Reviewer to push, so I uploaded new webrev: > >? ? ? >>>>>> > >? ? ? >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8240197/webrev.03/ > >? ? ? >>>>>> > >? ? ? >>>>>> > >? ? ? >>>>>> Yasumasa > >? ? ? >>>>>> > >? ? ? >>>>>> > >? ? ? >>>>>> On 2020/03/03 23:25, Schmelter, Ralf wrote: > >? ? ? >>>>>>> Hi Yasumasa, > >? ? ? >>>>>>> > >? ? ? >>>>>>> thanks for your work. The code should now be able to handle any path. > >? ? ? >>>>>>> > >? ? ? >>>>>>> Two small changes (no new webrev needed): > >? ? ? >>>>>>> > >? ? ? >>>>>>> size_t result_len = prefix_len + prefix_off + additional_space > >? ? ? >>>>>>> should be > >? ? ? >>>>>>> size_t result_len = prefix_len - prefix_off + additional_space > >? ? ? >>>>>>> > >? ? ? >>>>>>> And you should copy the prefix after you copied the filename, since sometimes the prefix is required to overwrite the start of the filename: > >? ? ? >>>>>>> > >? ? ? >>>>>>> // This call would be success because it is checked in above > >? ? ? >>>>>>> GetFullPathNameW(unicode_path, full_path_len, result + prefix_len - prefix_off, NULL); > >? ? ? >>>>>>> // Copy prefix > >? ? ? >>>>>>> memcpy(result, prefix, sizeof(WCHAR) * prefix_len); > >? ? ? >>>>>>> > >? ? ? >>>>>>> and > >? ? ? >>>>>>> > >? ? ? >>>>>>> // Copy unicode path > >? ? ? >>>>>>> memcpy(result + prefix_len - prefix_off, unicode_path, sizeof(WCHAR) * unicode_path_len); > >? ? ? >>>>>>> // Copy prefix > >? ? ? >>>>>>> memcpy(result, prefix, sizeof(WCHAR) * prefix_len); > >? ? ? >>>>>>> > >? ? ? >>>>>>> With these changes the os_windows gtest and the appcds jtreg tests run without errors. > >? ? ? >>>>>>> > >? ? ? >>>>>>> Best regards, > >? ? ? >>>>>>> Ralf > >? ? ? >>>>>>> > >? ? ? >>>>> > >? ? ? >>> > >? ? ? > > > > From david.holmes at oracle.com Thu Mar 5 07:34:40 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 5 Mar 2020 17:34:40 +1000 Subject: RFR(S): 8232081: Try to link all classes during dynamic CDS dump In-Reply-To: <7a54e00f-c049-a5ef-0afc-bb6c9d0f30ad@oracle.com> References: <9b40145a-db72-6313-3b98-5e5aea754e25@oracle.com> <1167ff6d-3997-6390-d020-d0946fad0af8@oracle.com> <3f373a33-c7ed-0237-1dd5-f8ac8e8794ee@oracle.com> <8ac4612b-fae3-bb16-f2ab-fa323c33ca51@oracle.com> <00090c43-1e0d-2b1c-d2a5-1b5f1b29199d@oracle.com> <7a54e00f-c049-a5ef-0afc-bb6c9d0f30ad@oracle.com> Message-ID: On 5/03/2020 3:13 pm, Calvin Cheung wrote: > Hi David, Ioi, > > Here's an updated webrev which doesn't involve changing > InstanceKlass::verify_on() and made use of the changes for JDK-8240481. > > ??? http://cr.openjdk.java.net/~ccheung/jdk15/8232081/webrev.03/ No further comments from me. Thanks for cleaning that up. David > thanks, > Calvin > > On 3/1/20 10:14 PM, David Holmes wrote: >> Hi Calvin, >> >> On 28/02/2020 4:12 pm, Calvin Cheung wrote: >>> Hi David, >>> >>> On 2/27/20 5:40 PM, David Holmes wrote: >>>> Hi Calvin, Ioi, >>>> >>>> Looking good - comments below. >>>> >>>> A meta-question: normal application classes are rarely loaded but >>>> not linked so I'm a little surprised this is an issue. What is the >>>> main source of such classes - generated classes like lambda forms? >>>> Also why do we need to link them when loading from the archive if >>>> they were never linked when the application actually executed ?? >>> >>> I saw that Ioi already answered the above. >>> >>> I'll try to answer your questions inline below.. >> >> Responses inline below ... >> >>>> >>>> On 28/02/2020 10:48 am, Calvin Cheung wrote: >>>>> Hi David, Ioi, >>>>> >>>>> Thanks for your review and suggestions. >>>>> >>>>> On 2/26/20 9:46 PM, Ioi Lam wrote: >>>>>> >>>>>> >>>>>> On 2/26/20 7:50 PM, David Holmes wrote: >>>>>>> Hi Calvin, >>>>>>> >>>>>>> Adding core-libs-dev as you are messing with their code :) >>>>>>> >>>>>>> On 27/02/2020 1:19 pm, Calvin Cheung wrote: >>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8232081 >>>>>>>> webrev: >>>>>>>> http://cr.openjdk.java.net/~ccheung/jdk15/8232081/webrev.00/ >>>>>>>> >>>>>>>> The proposed changeset for this RFE adds a >>>>>>>> JVM_LinkClassesForCDS() function to be called from >>>>>>>> java/lang/Shutdown to notify the JVM to link the classes loaded >>>>>>>> by the builtin class loaders. The >>>>>>> >>>>>>> This would be much less disruptive if this was handled purely on >>>>>>> the VM side once we have started shutdown. No need to make any >>>>>>> changes to the Java side then, nor jvm.cpp. >>>>>>> >>>>>> >>>>>> Hi David, >>>>>> >>>>>> To link the classes, we need to be able to run Java code -- when >>>>>> linking a class X loaded by the app loader, X will be verified. >>>>>> During verification of X, we may need to load additional classes >>>>>> from the app loader, which executes Java code during its class >>>>>> loading operations. >>>>>> >>>>>> We also need to handle all the exit conditions. As far as I can >>>>>> tell, an app can exit the JVM in two ways: >>>>>> >>>>>> (1) Explicitly calling System.exit(). This will call >>>>>> java.lang.Shutdown.exit() which does this: >>>>>> >>>>>> http://hg.openjdk.java.net/jdk/jdk/file/0edc7fd0d7a3/src/java.base/share/classes/java/lang/Shutdown.java#l163 >>>>>> >>>>>> >>>>>> ??????????? beforeHalt(); // native >>>>>> ??????????? runHooks(); >>>>>> ??????????? halt(status); >>>>>> >>>>>> (2) When all non-daemon threads have died (e.g., falling out of >>>>>> the bottom of HelloWorld.main()). There's no explicit call to >>>>>> System.exit(), but the JVM will proactively call >>>>>> java.lang.Shutdown.shutdown() inside >>>>>> JavaThread::invoke_shutdown_hooks() >>>>>> >>>>>> http://hg.openjdk.java.net/jdk/jdk/file/0edc7fd0d7a3/src/hotspot/share/runtime/thread.cpp#l4331 >>>>>> >>>>>> http://hg.openjdk.java.net/jdk/jdk/file/0edc7fd0d7a3/src/java.base/share/classes/java/lang/Shutdown.java#l184 >>>>>> >>>>>> >>>>>> If we want to avoid modifying the Java code, I think we can >>>>>> intercept JVM_BeforeHalt() and >>>>>> JavaThread::invoke_shutdown_hooks(). This way we should be able to >>>>>> handle all the classes (except those that are loaded inside >>>>>> Shutdown.runHooks). >>>>> >>>>> I've implemented the above. No changes to the Java side. >>>>> >>>>> updated webrev: >>>>> >>>>> http://cr.openjdk.java.net/~ccheung/jdk15/8232081/webrev.01/ >>>> >>>> This looks much better to me. I wasn't sure if you were that >>>> concerned about catching the classes loaded by the Shutdown hooks, >>>> but it is good you are not as it seems problematic to me to trigger >>>> class loading etc after the shutdown hooks have executed - you may >>>> hit unexpected conditions. So this approach is good. >>>> >>>> A few minor comments: >>>> >>>> src/hotspot/share/memory/metaspaceShared.cpp >>>> src/hotspot/share/memory/metaspaceShared.hpp >>>> >>>> Why the change from TRAPS to "Thread* THREAD"? >>> I forgot why I changed it but I've changed it back and it still works. >> >> Ok. >> >>>> >>>> --- >>>> >>>> src/hotspot/share/oops/instanceKlass.cpp >>>> >>>> I'm not clear how verify_on is used so am unsure how the additional >>>> error state check may affect code unrelated to dynamic dumping ?? >>> >>> Some of the appcds tests by design have some classes which fail >>> verification. Without the change, the following assert would fail: >>> >>> void vtableEntry::verify(klassVtable* vt, outputStream* st) { >>> ?? Klass* vtklass = vt->klass(); >>> ?? if (vtklass->is_instance_klass() && >>> ????? (InstanceKlass::cast(vtklass)->major_version() >= >>> klassVtable::VTABLE_TRANSITIVE_OVERRIDE_VERSION)) { >>> *assert*(method() != NULL, "must have set method"); >>> ?? } >> >> Okay so you need to exclude for that case but ... >> >>> Here's the call stack during dynamic CDS dump to the above function: >>> >>> vtableEntry::verify(klassVtable *, outputStream *) : void >>> ???? klassVtable::verify(outputStream *, bool) : void >>> ???????? InstanceKlass::verify_on(outputStream *) : void >>> ???????????? Klass::verify() : void >>> ???????????????? ClassLoaderData::verify() : void >>> ???????????????????? ClassLoaderDataGraph::verify() : void >>> ???????????????????????? Universe::verify(enum VerifyOption, const >>> char *) : void >>> ???????????????????????????? Universe::verify(const char *) : void >>> DynamicArchiveBuilder::verify_universe(const char *) : void >>> DynamicArchiveBuilder::doit() : void (2 matches) >>> VM_PopulateDynamicDumpSharedSpace::doit() : void >>> VM_Operation::evaluate() : void >>> >>> The is_linked() function is implemented as: >>> >>> ???? bool is_linked() const?????????????????? { return _init_state >= >>> linked; } >>> >>> which includes 'initialization_error'. >> >> ... AFAICS Universe::verify is call from a number of places, all of >> which would reach this modified code. As that existing code has >> included classes that are in the error state then it seems to me that >> unless you can prove otherwise you need to keep the existing code >> as-is for the non-CDS dump case. >> >>> A previous version of the changeset (including the change in question >>> in instanceKlass.cpp) passed tier1 - 4 testing. Let me know if I >>> should run other tests to make sure the change is good. >> >> There may well not be a test that covers this. >> >> Thanks, >> David >> >>>> >>>> Also can I suggest (as you are touching this code) to delete the >>>> ancient comment: >>>> >>>> 3580???? // $$$ This used to be done only for m/s collections. Doing it >>>> 3581???? // always seemed a valid generalization.? (DLD -- 6/00) >>> >>> I'm glad that you suggested to remove the above comment. I've no clue >>> what it means either. >>> >>> updated webrev: >>> >>> http://cr.openjdk.java.net/~ccheung/jdk15/8232081/webrev.02/ >>> >>> thanks, >>> Calvin >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>>> I also updated the test to test the shutdown hook and System.exit() >>>>> scenarios. >>>>> >>>>> All appcds tests passed on my linux host. I'll run more tests using >>>>> mach5. >>>>> >>>>> thanks, >>>>> >>>>> Calvin >>>>> >>>>> >>>>>> >>>>>> What do you think? >>>>>> >>>>>> - Ioi >>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> >>>>>>>> MetaspaceShared::link_and_cleanup_shared_classes() has been >>>>>>>> modified to handle both static and dynamic CDS dump. For dynamic >>>>>>>> CDS dump, only classes loaded by the builtin class loaders will >>>>>>>> be linked. Local performance testing using javac on >>>>>>>> HelloWorld.java shows an improvement of >5%. >>>>>>>> >>>>>>>> Passed tier1 - 4 tests. >>>>>>>> >>>>>>>> thanks, >>>>>>>> >>>>>>>> Calvin >>>>>>>> >>>>>> From kevin.walls at oracle.com Thu Mar 5 10:00:24 2020 From: kevin.walls at oracle.com (Kevin Walls) Date: Thu, 5 Mar 2020 10:00:24 +0000 Subject: RFR(S): 8240295: hs_err elapsed time in seconds is not accurate enough In-Reply-To: References: <4a3d7c8e-df9e-51de-77e4-775be1480392@oracle.com> <92ada19d-7200-6bb7-b174-9da8a166aea7@oracle.com> <053864a8-617f-fcac-fc26-220d161e3e55@oracle.com> Message-ID: Thanks - I had tried some ideas in the simple fashion, and we can use %06d formatting.... OK maybe such formatting is not as "bad" as %f... (glibc parses the int width specified without allocation.? We provide the output buffer, I don't think we will cause? vfprintf code to alloca or malloc.) I can offer a second version below that uses %d only.? Testing alongside %f in the same line, it retains the same value and position, e.g. Time: Thu Mar? 5 08:57:50 2020 UTC elapsed time: f: 2.001065 int: 2.001065 (raw int: 1065) seconds (0d 0h 0m 2s) Output example from the hg diff below (not from the same run): Time: Thu Mar? 5 09:28:01 2020 UTC elapsed time: 2.000611 seconds (0d 0h 0m 2s) Thanks! Kevin $ hg diff diff --git a/src/hotspot/share/runtime/os.cpp b/src/hotspot/share/runtime/os.cpp --- a/src/hotspot/share/runtime/os.cpp +++ b/src/hotspot/share/runtime/os.cpp @@ -1016,10 +1016,9 @@ ?? } ?? double t = os::elapsedTime(); -? // NOTE: It tends to crash after a SEGV if we want to printf("%f",...) in -? //?????? Linux. Must be a bug in glibc ? Workaround is to round "t" to int -? //?????? before printf. We lost some precision, but who cares? +? // NOTE: a crash using printf("%f",...) on Linux was historically noted here. ?? int eltime = (int)t;? // elapsed time in seconds +? int eltimeFraction = (int) ((t - eltime) * 1000000); ?? // print elapsed time in a human-readable format: ?? int eldays = eltime / secs_per_day; @@ -1029,7 +1028,7 @@ ?? int elmins = (eltime - day_secs - hour_secs) / secs_per_min; ?? int minute_secs = elmins * secs_per_min; ?? int elsecs = (eltime - day_secs - hour_secs - minute_secs); -? st->print_cr(" elapsed time: %d seconds (%dd %dh %dm %ds)", eltime, eldays, elhours, elmins, elsecs); +? st->print_cr(" elapsed time: %d.%06d seconds (%dd %dh %dm %ds)", eltime, eltimeFraction, eldays, elhours, elmins, elsecs); ?} On 05/03/2020 00:57, David Holmes wrote: > On 4/03/2020 8:44 am, Kevin Walls wrote: >> >> Thanks David - >> >> Yes there are situations where hs_err fails, and few people are >> sadder than me >> when that happens 8-) , so I was thinking about how scared to be by >> the comment. >> >> With the safety net of the error handler for the steps of the hs_err >> file >> (which works well, we see it invoked frequently), it looks reasonable >> to use >> %f as we might do other slightly questionable things for a signal >> handler. >> >> Corrupting locale information or floating point state might possibly >> cause >> problems, but if I cause a fake crash in print_date_and_time the error >> handler recovers and the report continues. > > That is good to know. > >> Thinking about printing with two ints, seconds and fractions: >> I don't see anything already that returns such a time in two >> components in the >> JVM, so we might implement a new form of os::javaTimeNanos() or >> similar that >> returns the two parts, and do that on each platform. > > I was thinking of something simple/crude ... > >> I didn't yet come up with anything to do in os::print_date_and_time() >> which will take the fractional part of the double, and print just the >> fraction as an int, without using any library / %f facilities. > > ... just using e.g. (untested) > > double t = os::elapsedTime(); > int secs =? (int) t; > int micros =? (int)((t - secs) * 100000); > printf("%d.%d", secs, micros); > > with appropriate width specifiers to get the formatting right. > > Cheers, > David > >> >> If you're still concerned I could revisit these or some other idea. >> >> Genuine laugh out loud moment for me, I backported the elapsed time >> logging from >> 6u4 to 5u19? (https://bugs.openjdk.java.net/browse/JDK-6447157) (2007). >> (I said before jdk5 was created, I should have said before it was in >> mercurial.) >> >> Thanks >> Kevin >> >> >> On 03/03/2020 01:11, David Holmes wrote: >>> Hi Kevin, >>> >>> On 2/03/2020 8:48 pm, Kevin Walls wrote: >>>> Oops, and with the bug ID in the title and JBS link: >>>> https://bugs.openjdk.java.net/browse/JDK-8240295 >>>> >>>> >>>> On 02/03/2020 10:47, Kevin Walls wrote: >>>>> Hi, >>>>> >>>>> (s11y and runtime opinions both relevant) >>>>> >>>>> A few times in the last month I've really wanted to compare the >>>>> Events logged in the hs_err file, and the time of the JVM's crash. >>>>> >>>>> "elapsed time" in hs_err is only accurate to one second, and has >>>>> been since before jdk5 was created. >>>>> >>>>> The diff below changes the format string and uses the non-rounded >>>>> time value (I don't see a need to change the other integer >>>>> arithmetic here), and we can enjoy hs_errs with detail like: >>>>> >>>>> ... >>>>> Time: Mon Mar? 2 09:57:13 2020 UTC elapsed time: 5.490135 seconds >>>>> (0d 0h 0m 5s) >>>>> ... >>>>> >>>>> Thanks >>>>> Kevin >>>>> >>>>> >>>>> /jdk/open$ hg diff >>>>> diff --git a/src/hotspot/share/runtime/os.cpp >>>>> b/src/hotspot/share/runtime/os.cpp >>>>> --- a/src/hotspot/share/runtime/os.cpp >>>>> +++ b/src/hotspot/share/runtime/os.cpp >>>>> @@ -1016,9 +1016,8 @@ >>>>> ?? } >>>>> >>>>> ?? double t = os::elapsedTime(); >>>>> -? // NOTE: It tends to crash after a SEGV if we want to >>>>> printf("%f",...) in >>>>> -? //?????? Linux. Must be a bug in glibc ? Workaround is to round >>>>> "t" to int >>>>> -? //?????? before printf. We lost some precision, but who cares? >>>>> +? // NOTE: a crash using printf("%f",...) on Linux was >>>>> historically noted here >>>>> +? //?????? (before the jdk5 repo was created). >>> >>> Just because it is old doesn't mean it no longer applies. printf is >>> not async-signal-safe - we know that but we try to use it anyway. >>> Maybe %f is even less async-signal-safe? >>> >>> This may get through testing okay but cause problems with real >>> crashes in the field. >>> >>> What about breaking the time up into two ints: seconds and nanos? >>> >>> Cheers, >>> David >>> ----- >>> >>>>> ?? int eltime = (int)t;? // elapsed time in seconds >>>>> >>>>> ?? // print elapsed time in a human-readable format: >>>>> @@ -1029,7 +1028,7 @@ >>>>> ?? int elmins = (eltime - day_secs - hour_secs) / secs_per_min; >>>>> ?? int minute_secs = elmins * secs_per_min; >>>>> ?? int elsecs = (eltime - day_secs - hour_secs - minute_secs); >>>>> -? st->print_cr(" elapsed time: %d seconds (%dd %dh %dm %ds)", >>>>> eltime, eldays, elhours, elmins, elsecs); >>>>> +? st->print_cr(" elapsed time: %f seconds (%dd %dh %dm %ds)", t, >>>>> eldays, elhours, elmins, elsecs); >>>>> ?} >>>>> >>>>> From stefan.karlsson at oracle.com Thu Mar 5 10:01:49 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 5 Mar 2020 11:01:49 +0100 Subject: RFR: 8240530: CheckUnhandledOops breaks BacktraceBuilder::set_has_hidden_top_frame In-Reply-To: <261be2cb-a8df-351f-043c-ea4b4792b754@oracle.com> References: <89b3460e-47a2-daec-bbd0-755e095d4c46@oracle.com> <261be2cb-a8df-351f-043c-ea4b4792b754@oracle.com> Message-ID: <210af131-15e4-efb4-ac74-c6d8f8411ebc@oracle.com> On 2020-03-04 16:08, coleen.phillimore at oracle.com wrote: > > Hi, > > I don't like the change to void* for _has_hidden_top_frame. > > Can you change this line to > > ??? if (_has_hidden_top_frame == _methods) { > > instead? I think you mean? if (_has_hidden_top_frame != _methods) { That doesn't work for at least two reason: 1) _has_hidden_top_frame is still being clobbered by CheckUnhandledOops 2) _methods are being reinitialized and changed value in BacktraceBuilder::expand > > Can you add an 'e' to Therefor > > ????? // It would be nice to add java/lang/Boolean::TRUE here > ????? // to indicate that this backtrace has a hidden top frame. > ????? // But this code is used before TRUE is allocated. > ????? // Therefor let's just use an arbitrary legal oop > ????? // available right here. We only test for != null > ????? // anyways. _methods is a short[]. Sure. > > The reason that it's an oop is because the _backtrace is an > objArrayOop of java.lang.Object[6], and hidden boolean is Object[5].? > It's not a nice data structure but it needs to be fast. That's the reason why we put an object in trace_hidden_offset: ????? _head->obj_at_put(trace_hidden_offset, ); but I'm specifically asking about the field _has_hidden_top_frame. I created a patch to show what I mean, to show that that field can simply be a bool: ?https://cr.openjdk.java.net/~stefank/8240530/webrev.02.delta ?https://cr.openjdk.java.net/~stefank/8240530/webrev.02 It compiles and seems to work. StefanK > Coleen > > > On 3/4/20 9:48 AM, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to get rid of a unhandled oop null check in >> the BacktraceBuilder. >> >> https://cr.openjdk.java.net/~stefank/8240530/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8240530 >> >> The patch changes the type from oop to void* to prevent NULL oops >> from being overwritten with non-NULL values. >> >> I think an alternative fix would be to change the type of >> _has_hidden_top_frame to bool. Maybe someone familiar with this code >> knows why this was encoded with an oop NULL check instead of a plain >> bool? >> >> Testing with tier1-3 with CheckUnhandledOops on by default. >> >> StefanK > From stefan.karlsson at oracle.com Thu Mar 5 10:03:07 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 5 Mar 2020 11:03:07 +0100 Subject: RFR: 8240529: CheckUnhandledOops breaks NULL check in Modules::define_module In-Reply-To: References: Message-ID: <17f69c4c-ceb7-c0ac-2295-10c2a82ef041@oracle.com> Thanks for reviewing, Harold. StefanK On 2020-03-04 15:36, Harold Seigel wrote: > Looks good and trivial.? Thanks for finding and fixing this. > > Harold > > On 3/4/2020 9:25 AM, Stefan Karlsson wrote: >> Hi all, >> >> Please review this small fix to CheckUnhandledOops. >> >> https://cr.openjdk.java.net/~stefank/8240529/webrev.01 >> https://bugs.openjdk.java.net/browse/JDK-8240529 >> >> 'loader' is an unhandled oop that is alive across a safepoint check >> and then is used for a NULL check. When the safepoint check executes >> the value of the oop is changed to 0xff...ff1. If the oop was NULL >> then the check now starts to fail. When this happens >> ClassLoader::_exploded_entries isn't populated as expected, causing >> class loading failures with exploded builds. >> >> Thanks, >> StefanK From stefan.karlsson at oracle.com Thu Mar 5 10:03:22 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 5 Mar 2020 11:03:22 +0100 Subject: RFR: 8240529: CheckUnhandledOops breaks NULL check in Modules::define_module In-Reply-To: References: Message-ID: <9f598b3a-454e-1ac7-0679-d188347e2b56@oracle.com> Thanks, Coleen. StefanK On 2020-03-04 15:59, coleen.phillimore at oracle.com wrote: > +1 > Coleen > > On 3/4/20 9:36 AM, Harold Seigel wrote: >> Looks good and trivial.? Thanks for finding and fixing this. >> >> Harold >> >> On 3/4/2020 9:25 AM, Stefan Karlsson wrote: >>> Hi all, >>> >>> Please review this small fix to CheckUnhandledOops. >>> >>> https://cr.openjdk.java.net/~stefank/8240529/webrev.01 >>> https://bugs.openjdk.java.net/browse/JDK-8240529 >>> >>> 'loader' is an unhandled oop that is alive across a safepoint check >>> and then is used for a NULL check. When the safepoint check executes >>> the value of the oop is changed to 0xff...ff1. If the oop was NULL >>> then the check now starts to fail. When this happens >>> ClassLoader::_exploded_entries isn't populated as expected, causing >>> class loading failures with exploded builds. >>> >>> Thanks, >>> StefanK > From ralf.schmelter at sap.com Thu Mar 5 10:03:35 2020 From: ralf.schmelter at sap.com (Schmelter, Ralf) Date: Thu, 5 Mar 2020 10:03:35 +0000 Subject: RFR: 8240197: Cannot start JVM when $JAVA_HOME includes CJK characters In-Reply-To: References: <6ec2f649-8999-fdd5-c2df-465925c3663b@oss.nttdata.com> <2d5148fc-7d71-d94c-d4a3-e6424a38b091@oss.nttdata.com> <37e2f672-3407-5b16-ab71-aa1e70e928db@oss.nttdata.com> <954f0bf1-91e2-3a1b-e2cb-e93d8343b0e6@oracle.com> <71ba4bad-05c9-042d-73dc-7f492fc1728a@oss.nttdata.com> <44d86ec7-aa6d-daf8-7200-534717cb4edb@oracle.com> <25aa10eb-6a8f-b416-02c2-b3fcd140076e@oss.nttdata.com> Message-ID: Thanks Yasumasa, webrev.06 looks good to me. Both correct and readable. Best regards, Ralf -----Original Message----- From: Yasumasa Suenaga Sent: Donnerstag, 5. M?rz 2020 07:41 To: Thomas St?fe Cc: Ioi Lam ; Schmelter, Ralf ; hotspot-runtime-dev at openjdk.java.net; yasuenag at gmail.com Subject: Re: RFR: 8240197: Cannot start JVM when $JAVA_HOME includes CJK characters Thanks Thomas! I will push it if Ralf is ok to this change (webrev.06). Yasumasa From xxinliu at amazon.com Thu Mar 5 10:04:21 2020 From: xxinliu at amazon.com (Liu, Xin) Date: Thu, 5 Mar 2020 10:04:21 +0000 Subject: RFR(L): JDK-8229517: Support for optional asynchronous/buffered logging Message-ID: <77FA4BF1-DC2E-4AC8-987B-56578E5D33B9@amazon.com> Hi, hotspot reviewers, Could you take a look at my webrev? Bug: https://bugs.openjdk.java.net/browse/JDK-8229517 Webrev: https://cr.openjdk.java.net/~xliu/8229517/01/webrev/ Previously, we had discussion at runtime-dev. https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-February/038211.html I would like to continue it because I redo my implementation on the tip of jdk and have made it a general solution. This webrev provides on apar with . Even though it doesn?t touch any existing logging calls, it?s very possible to offload any callsite of logging to the asynchronous buffer. My rationale is that we only need to redirect some time-critical logs to asynchronous buffer to dodge unexpected filesystem delay. Eg. g1gc has some loggings at safepoint. We mitigate those long-tailed GC latencies rooted by filesystem writing to gclog. I got feedback from my our internal users. They have a concern about my approach of dynamic memory. Indeed, I gave up ringbuffer design in previous version and turn to use a dynamic allocated linkedlist. If writing to filesystem stalls infinitely, JVM may implode because of memory usage. I plan to solve it by adding a memory quota to the FIFO of logAsyncFlusher.hpp. It will start dropping early messages of the FIFO if it has consumed too many free memories. Another potential improvement is changing the FIFO from mutex protected to lock-free. It make sense if we redirect high volume of logging messages to it and mutex becomes hot. Thanks, --lx From david.holmes at oracle.com Thu Mar 5 10:38:58 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 5 Mar 2020 20:38:58 +1000 Subject: RFR(S): 8240295: hs_err elapsed time in seconds is not accurate enough In-Reply-To: References: <4a3d7c8e-df9e-51de-77e4-775be1480392@oracle.com> <92ada19d-7200-6bb7-b174-9da8a166aea7@oracle.com> <053864a8-617f-fcac-fc26-220d161e3e55@oracle.com> Message-ID: <20045e23-c736-5289-866e-9df5a09101a8@oracle.com> Thanks Kevin. I think this is the less risky change and achieves the goal. David On 5/03/2020 8:00 pm, Kevin Walls wrote: > Thanks - > > I had tried some ideas in the simple fashion, and we can use %06d > formatting.... OK maybe such formatting is not as "bad" as %f... > > (glibc parses the int width specified without allocation.? We provide > the output buffer, I don't think we will cause? vfprintf code to alloca > or malloc.) > > I can offer a second version below that uses %d only.? Testing alongside > %f in the same line, it retains the same value and position, e.g. > > Time: Thu Mar? 5 08:57:50 2020 UTC elapsed time: f: 2.001065 int: > 2.001065 (raw int: 1065) seconds (0d 0h 0m 2s) > > Output example from the hg diff below (not from the same run): > > Time: Thu Mar? 5 09:28:01 2020 UTC elapsed time: 2.000611 seconds (0d 0h > 0m 2s) > > > Thanks! > Kevin > > > $ hg diff > diff --git a/src/hotspot/share/runtime/os.cpp > b/src/hotspot/share/runtime/os.cpp > --- a/src/hotspot/share/runtime/os.cpp > +++ b/src/hotspot/share/runtime/os.cpp > @@ -1016,10 +1016,9 @@ > ?? } > > ?? double t = os::elapsedTime(); > -? // NOTE: It tends to crash after a SEGV if we want to > printf("%f",...) in > -? //?????? Linux. Must be a bug in glibc ? Workaround is to round "t" > to int > -? //?????? before printf. We lost some precision, but who cares? > +? // NOTE: a crash using printf("%f",...) on Linux was historically > noted here. > ?? int eltime = (int)t;? // elapsed time in seconds > +? int eltimeFraction = (int) ((t - eltime) * 1000000); > > ?? // print elapsed time in a human-readable format: > ?? int eldays = eltime / secs_per_day; > @@ -1029,7 +1028,7 @@ > ?? int elmins = (eltime - day_secs - hour_secs) / secs_per_min; > ?? int minute_secs = elmins * secs_per_min; > ?? int elsecs = (eltime - day_secs - hour_secs - minute_secs); > -? st->print_cr(" elapsed time: %d seconds (%dd %dh %dm %ds)", eltime, > eldays, elhours, elmins, elsecs); > +? st->print_cr(" elapsed time: %d.%06d seconds (%dd %dh %dm %ds)", > eltime, eltimeFraction, eldays, elhours, elmins, elsecs); > ?} > > > > On 05/03/2020 00:57, David Holmes wrote: >> On 4/03/2020 8:44 am, Kevin Walls wrote: >>> >>> Thanks David - >>> >>> Yes there are situations where hs_err fails, and few people are >>> sadder than me >>> when that happens 8-) , so I was thinking about how scared to be by >>> the comment. >>> >>> With the safety net of the error handler for the steps of the hs_err >>> file >>> (which works well, we see it invoked frequently), it looks reasonable >>> to use >>> %f as we might do other slightly questionable things for a signal >>> handler. >>> >>> Corrupting locale information or floating point state might possibly >>> cause >>> problems, but if I cause a fake crash in print_date_and_time the error >>> handler recovers and the report continues. >> >> That is good to know. >> >>> Thinking about printing with two ints, seconds and fractions: >>> I don't see anything already that returns such a time in two >>> components in the >>> JVM, so we might implement a new form of os::javaTimeNanos() or >>> similar that >>> returns the two parts, and do that on each platform. >> >> I was thinking of something simple/crude ... >> >>> I didn't yet come up with anything to do in os::print_date_and_time() >>> which will take the fractional part of the double, and print just the >>> fraction as an int, without using any library / %f facilities. >> >> ... just using e.g. (untested) >> >> double t = os::elapsedTime(); >> int secs =? (int) t; >> int micros =? (int)((t - secs) * 100000); >> printf("%d.%d", secs, micros); >> >> with appropriate width specifiers to get the formatting right. >> >> Cheers, >> David >> >>> >>> If you're still concerned I could revisit these or some other idea. >>> >>> Genuine laugh out loud moment for me, I backported the elapsed time >>> logging from >>> 6u4 to 5u19? (https://bugs.openjdk.java.net/browse/JDK-6447157) (2007). >>> (I said before jdk5 was created, I should have said before it was in >>> mercurial.) >>> >>> Thanks >>> Kevin >>> >>> >>> On 03/03/2020 01:11, David Holmes wrote: >>>> Hi Kevin, >>>> >>>> On 2/03/2020 8:48 pm, Kevin Walls wrote: >>>>> Oops, and with the bug ID in the title and JBS link: >>>>> https://bugs.openjdk.java.net/browse/JDK-8240295 >>>>> >>>>> >>>>> On 02/03/2020 10:47, Kevin Walls wrote: >>>>>> Hi, >>>>>> >>>>>> (s11y and runtime opinions both relevant) >>>>>> >>>>>> A few times in the last month I've really wanted to compare the >>>>>> Events logged in the hs_err file, and the time of the JVM's crash. >>>>>> >>>>>> "elapsed time" in hs_err is only accurate to one second, and has >>>>>> been since before jdk5 was created. >>>>>> >>>>>> The diff below changes the format string and uses the non-rounded >>>>>> time value (I don't see a need to change the other integer >>>>>> arithmetic here), and we can enjoy hs_errs with detail like: >>>>>> >>>>>> ... >>>>>> Time: Mon Mar? 2 09:57:13 2020 UTC elapsed time: 5.490135 seconds >>>>>> (0d 0h 0m 5s) >>>>>> ... >>>>>> >>>>>> Thanks >>>>>> Kevin >>>>>> >>>>>> >>>>>> /jdk/open$ hg diff >>>>>> diff --git a/src/hotspot/share/runtime/os.cpp >>>>>> b/src/hotspot/share/runtime/os.cpp >>>>>> --- a/src/hotspot/share/runtime/os.cpp >>>>>> +++ b/src/hotspot/share/runtime/os.cpp >>>>>> @@ -1016,9 +1016,8 @@ >>>>>> ?? } >>>>>> >>>>>> ?? double t = os::elapsedTime(); >>>>>> -? // NOTE: It tends to crash after a SEGV if we want to >>>>>> printf("%f",...) in >>>>>> -? //?????? Linux. Must be a bug in glibc ? Workaround is to round >>>>>> "t" to int >>>>>> -? //?????? before printf. We lost some precision, but who cares? >>>>>> +? // NOTE: a crash using printf("%f",...) on Linux was >>>>>> historically noted here >>>>>> +? //?????? (before the jdk5 repo was created). >>>> >>>> Just because it is old doesn't mean it no longer applies. printf is >>>> not async-signal-safe - we know that but we try to use it anyway. >>>> Maybe %f is even less async-signal-safe? >>>> >>>> This may get through testing okay but cause problems with real >>>> crashes in the field. >>>> >>>> What about breaking the time up into two ints: seconds and nanos? >>>> >>>> Cheers, >>>> David >>>> ----- >>>> >>>>>> ?? int eltime = (int)t;? // elapsed time in seconds >>>>>> >>>>>> ?? // print elapsed time in a human-readable format: >>>>>> @@ -1029,7 +1028,7 @@ >>>>>> ?? int elmins = (eltime - day_secs - hour_secs) / secs_per_min; >>>>>> ?? int minute_secs = elmins * secs_per_min; >>>>>> ?? int elsecs = (eltime - day_secs - hour_secs - minute_secs); >>>>>> -? st->print_cr(" elapsed time: %d seconds (%dd %dh %dm %ds)", >>>>>> eltime, eldays, elhours, elmins, elsecs); >>>>>> +? st->print_cr(" elapsed time: %f seconds (%dd %dh %dm %ds)", t, >>>>>> eldays, elhours, elmins, elsecs); >>>>>> ?} >>>>>> >>>>>> From suenaga at oss.nttdata.com Thu Mar 5 10:42:21 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Thu, 5 Mar 2020 19:42:21 +0900 Subject: RFR: 8240197: Cannot start JVM when $JAVA_HOME includes CJK characters In-Reply-To: References: <6ec2f649-8999-fdd5-c2df-465925c3663b@oss.nttdata.com> <2d5148fc-7d71-d94c-d4a3-e6424a38b091@oss.nttdata.com> <37e2f672-3407-5b16-ab71-aa1e70e928db@oss.nttdata.com> <954f0bf1-91e2-3a1b-e2cb-e93d8343b0e6@oracle.com> <71ba4bad-05c9-042d-73dc-7f492fc1728a@oss.nttdata.com> <44d86ec7-aa6d-daf8-7200-534717cb4edb@oracle.com> <25aa10eb-6a8f-b416-02c2-b3fcd140076e@oss.nttdata.com> Message-ID: <7f37cc99-f723-283b-7aab-4cf6c291925b@oss.nttdata.com> Thanks Ralf! Yasumasa On 2020/03/05 19:03, Schmelter, Ralf wrote: > Thanks Yasumasa, webrev.06 looks good to me. Both correct and readable. > > Best regards, > Ralf > > -----Original Message----- > From: Yasumasa Suenaga > Sent: Donnerstag, 5. M?rz 2020 07:41 > To: Thomas St?fe > Cc: Ioi Lam ; Schmelter, Ralf ; hotspot-runtime-dev at openjdk.java.net; yasuenag at gmail.com > Subject: Re: RFR: 8240197: Cannot start JVM when $JAVA_HOME includes CJK characters > > Thanks Thomas! > > I will push it if Ralf is ok to this change (webrev.06). > > > Yasumasa > From coleen.phillimore at oracle.com Thu Mar 5 12:21:52 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 5 Mar 2020 07:21:52 -0500 Subject: RFR: 8240530: CheckUnhandledOops breaks BacktraceBuilder::set_has_hidden_top_frame In-Reply-To: <210af131-15e4-efb4-ac74-c6d8f8411ebc@oracle.com> References: <89b3460e-47a2-daec-bbd0-755e095d4c46@oracle.com> <261be2cb-a8df-351f-043c-ea4b4792b754@oracle.com> <210af131-15e4-efb4-ac74-c6d8f8411ebc@oracle.com> Message-ID: <8a5ad3a5-75e9-228b-a005-de4c12bf77c4@oracle.com> On 3/5/20 5:01 AM, Stefan Karlsson wrote: > On 2020-03-04 16:08, coleen.phillimore at oracle.com wrote: >> >> Hi, >> >> I don't like the change to void* for _has_hidden_top_frame. >> >> Can you change this line to >> >> ??? if (_has_hidden_top_frame == _methods) { >> >> instead? > > I think you mean? > if (_has_hidden_top_frame != _methods) { > > That doesn't work for at least two reason: > 1) _has_hidden_top_frame is still being clobbered by CheckUnhandledOops > 2) _methods are being reinitialized and changed value in > BacktraceBuilder::expand > >> >> Can you add an 'e' to Therefor >> >> ????? // It would be nice to add java/lang/Boolean::TRUE here >> ????? // to indicate that this backtrace has a hidden top frame. >> ????? // But this code is used before TRUE is allocated. >> ????? // Therefor let's just use an arbitrary legal oop >> ????? // available right here. We only test for != null >> ????? // anyways. _methods is a short[]. > > Sure. > >> >> The reason that it's an oop is because the _backtrace is an >> objArrayOop of java.lang.Object[6], and hidden boolean is Object[5].? >> It's not a nice data structure but it needs to be fast. > > That's the reason why we put an object in trace_hidden_offset: > ????? _head->obj_at_put(trace_hidden_offset, ); > > but I'm specifically asking about the field _has_hidden_top_frame. > > I created a patch to show what I mean, to show that that field can > simply be a bool: > ?https://cr.openjdk.java.net/~stefank/8240530/webrev.02.delta > ?https://cr.openjdk.java.net/~stefank/8240530/webrev.02 > > It compiles and seems to work. Yes. That looks great.?? We should have done that in the first place. Thanks, Coleen > > StefanK > >> Coleen >> >> >> On 3/4/20 9:48 AM, Stefan Karlsson wrote: >>> Hi all, >>> >>> Please review this patch to get rid of a unhandled oop null check in >>> the BacktraceBuilder. >>> >>> https://cr.openjdk.java.net/~stefank/8240530/webrev.01/ >>> https://bugs.openjdk.java.net/browse/JDK-8240530 >>> >>> The patch changes the type from oop to void* to prevent NULL oops >>> from being overwritten with non-NULL values. >>> >>> I think an alternative fix would be to change the type of >>> _has_hidden_top_frame to bool. Maybe someone familiar with this code >>> knows why this was encoded with an oop NULL check instead of a plain >>> bool? >>> >>> Testing with tier1-3 with CheckUnhandledOops on by default. >>> >>> StefanK >> > From stefan.karlsson at oracle.com Thu Mar 5 12:22:28 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 5 Mar 2020 13:22:28 +0100 Subject: RFR: 8240530: CheckUnhandledOops breaks BacktraceBuilder::set_has_hidden_top_frame In-Reply-To: <8a5ad3a5-75e9-228b-a005-de4c12bf77c4@oracle.com> References: <89b3460e-47a2-daec-bbd0-755e095d4c46@oracle.com> <261be2cb-a8df-351f-043c-ea4b4792b754@oracle.com> <210af131-15e4-efb4-ac74-c6d8f8411ebc@oracle.com> <8a5ad3a5-75e9-228b-a005-de4c12bf77c4@oracle.com> Message-ID: <671bda0b-ee08-1ee3-59ba-9a29e1be5bc6@oracle.com> Thanks for reviewing, Coleen. StefanK On 2020-03-05 13:21, coleen.phillimore at oracle.com wrote: > > > On 3/5/20 5:01 AM, Stefan Karlsson wrote: >> On 2020-03-04 16:08, coleen.phillimore at oracle.com wrote: >>> >>> Hi, >>> >>> I don't like the change to void* for _has_hidden_top_frame. >>> >>> Can you change this line to >>> >>> ??? if (_has_hidden_top_frame == _methods) { >>> >>> instead? >> >> I think you mean? >> if (_has_hidden_top_frame != _methods) { >> >> That doesn't work for at least two reason: >> 1) _has_hidden_top_frame is still being clobbered by CheckUnhandledOops >> 2) _methods are being reinitialized and changed value in >> BacktraceBuilder::expand >> >>> >>> Can you add an 'e' to Therefor >>> >>> ????? // It would be nice to add java/lang/Boolean::TRUE here >>> ????? // to indicate that this backtrace has a hidden top frame. >>> ????? // But this code is used before TRUE is allocated. >>> ????? // Therefor let's just use an arbitrary legal oop >>> ????? // available right here. We only test for != null >>> ????? // anyways. _methods is a short[]. >> >> Sure. >> >>> >>> The reason that it's an oop is because the _backtrace is an >>> objArrayOop of java.lang.Object[6], and hidden boolean is >>> Object[5].? It's not a nice data structure but it needs to be fast. >> >> That's the reason why we put an object in trace_hidden_offset: >> ????? _head->obj_at_put(trace_hidden_offset, ); >> >> but I'm specifically asking about the field _has_hidden_top_frame. >> >> I created a patch to show what I mean, to show that that field can >> simply be a bool: >> ?https://cr.openjdk.java.net/~stefank/8240530/webrev.02.delta >> ?https://cr.openjdk.java.net/~stefank/8240530/webrev.02 >> >> It compiles and seems to work. > > Yes. That looks great.?? We should have done that in the first place. > Thanks, > Coleen >> >> StefanK >> >>> Coleen >>> >>> >>> On 3/4/20 9:48 AM, Stefan Karlsson wrote: >>>> Hi all, >>>> >>>> Please review this patch to get rid of a unhandled oop null check >>>> in the BacktraceBuilder. >>>> >>>> https://cr.openjdk.java.net/~stefank/8240530/webrev.01/ >>>> https://bugs.openjdk.java.net/browse/JDK-8240530 >>>> >>>> The patch changes the type from oop to void* to prevent NULL oops >>>> from being overwritten with non-NULL values. >>>> >>>> I think an alternative fix would be to change the type of >>>> _has_hidden_top_frame to bool. Maybe someone familiar with this >>>> code knows why this was encoded with an oop NULL check instead of a >>>> plain bool? >>>> >>>> Testing with tier1-3 with CheckUnhandledOops on by default. >>>> >>>> StefanK >>> >> > From ioi.lam at oracle.com Thu Mar 5 17:17:29 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 5 Mar 2020 09:17:29 -0800 Subject: RFR(S): 8232081: Try to link all classes during dynamic CDS dump In-Reply-To: <7a54e00f-c049-a5ef-0afc-bb6c9d0f30ad@oracle.com> References: <9b40145a-db72-6313-3b98-5e5aea754e25@oracle.com> <1167ff6d-3997-6390-d020-d0946fad0af8@oracle.com> <3f373a33-c7ed-0237-1dd5-f8ac8e8794ee@oracle.com> <8ac4612b-fae3-bb16-f2ab-fa323c33ca51@oracle.com> <00090c43-1e0d-2b1c-d2a5-1b5f1b29199d@oracle.com> <7a54e00f-c049-a5ef-0afc-bb6c9d0f30ad@oracle.com> Message-ID: <5fa95d9f-47a7-c1c7-90e5-87db0173e3b7@oracle.com> Hi Calvin, Looks good overall. I just have two comment: I think this is not needed: ?306 void ConstantPool::resolve_class_constants(TRAPS) { ?307?? Arguments::assert_is_dumping_archive(); because this function is used to resolve all Strings in the statically dumped classes to archive all the Strings. However, the archive heap is not supported for the dynamic archive. So I think it's better to avoid calling this function from LinkSharedClassesClosure for dynamic dump. LinkSharedClassesClosure::_is_static -- maybe we can avoid adding this field and just check "if (DumpSharedSpaces)" instead? Thanks - Ioi On 3/4/20 9:13 PM, Calvin Cheung wrote: > Hi David, Ioi, > > Here's an updated webrev which doesn't involve changing > InstanceKlass::verify_on() and made use of the changes for JDK-8240481. > > ??? http://cr.openjdk.java.net/~ccheung/jdk15/8232081/webrev.03/ > > thanks, > Calvin > > On 3/1/20 10:14 PM, David Holmes wrote: >> Hi Calvin, >> >> On 28/02/2020 4:12 pm, Calvin Cheung wrote: >>> Hi David, >>> >>> On 2/27/20 5:40 PM, David Holmes wrote: >>>> Hi Calvin, Ioi, >>>> >>>> Looking good - comments below. >>>> >>>> A meta-question: normal application classes are rarely loaded but >>>> not linked so I'm a little surprised this is an issue. What is the >>>> main source of such classes - generated classes like lambda forms? >>>> Also why do we need to link them when loading from the archive if >>>> they were never linked when the application actually executed ?? >>> >>> I saw that Ioi already answered the above. >>> >>> I'll try to answer your questions inline below.. >> >> Responses inline below ... >> >>>> >>>> On 28/02/2020 10:48 am, Calvin Cheung wrote: >>>>> Hi David, Ioi, >>>>> >>>>> Thanks for your review and suggestions. >>>>> >>>>> On 2/26/20 9:46 PM, Ioi Lam wrote: >>>>>> >>>>>> >>>>>> On 2/26/20 7:50 PM, David Holmes wrote: >>>>>>> Hi Calvin, >>>>>>> >>>>>>> Adding core-libs-dev as you are messing with their code :) >>>>>>> >>>>>>> On 27/02/2020 1:19 pm, Calvin Cheung wrote: >>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8232081 >>>>>>>> webrev: >>>>>>>> http://cr.openjdk.java.net/~ccheung/jdk15/8232081/webrev.00/ >>>>>>>> >>>>>>>> The proposed changeset for this RFE adds a >>>>>>>> JVM_LinkClassesForCDS() function to be called from >>>>>>>> java/lang/Shutdown to notify the JVM to link the classes loaded >>>>>>>> by the builtin class loaders. The >>>>>>> >>>>>>> This would be much less disruptive if this was handled purely on >>>>>>> the VM side once we have started shutdown. No need to make any >>>>>>> changes to the Java side then, nor jvm.cpp. >>>>>>> >>>>>> >>>>>> Hi David, >>>>>> >>>>>> To link the classes, we need to be able to run Java code -- when >>>>>> linking a class X loaded by the app loader, X will be verified. >>>>>> During verification of X, we may need to load additional classes >>>>>> from the app loader, which executes Java code during its class >>>>>> loading operations. >>>>>> >>>>>> We also need to handle all the exit conditions. As far as I can >>>>>> tell, an app can exit the JVM in two ways: >>>>>> >>>>>> (1) Explicitly calling System.exit(). This will call >>>>>> java.lang.Shutdown.exit() which does this: >>>>>> >>>>>> http://hg.openjdk.java.net/jdk/jdk/file/0edc7fd0d7a3/src/java.base/share/classes/java/lang/Shutdown.java#l163 >>>>>> >>>>>> >>>>>> ??????????? beforeHalt(); // native >>>>>> ??????????? runHooks(); >>>>>> ??????????? halt(status); >>>>>> >>>>>> (2) When all non-daemon threads have died (e.g., falling out of >>>>>> the bottom of HelloWorld.main()). There's no explicit call to >>>>>> System.exit(), but the JVM will proactively call >>>>>> java.lang.Shutdown.shutdown() inside >>>>>> JavaThread::invoke_shutdown_hooks() >>>>>> >>>>>> http://hg.openjdk.java.net/jdk/jdk/file/0edc7fd0d7a3/src/hotspot/share/runtime/thread.cpp#l4331 >>>>>> >>>>>> http://hg.openjdk.java.net/jdk/jdk/file/0edc7fd0d7a3/src/java.base/share/classes/java/lang/Shutdown.java#l184 >>>>>> >>>>>> >>>>>> If we want to avoid modifying the Java code, I think we can >>>>>> intercept JVM_BeforeHalt() and >>>>>> JavaThread::invoke_shutdown_hooks(). This way we should be able >>>>>> to handle all the classes (except those that are loaded inside >>>>>> Shutdown.runHooks). >>>>> >>>>> I've implemented the above. No changes to the Java side. >>>>> >>>>> updated webrev: >>>>> >>>>> http://cr.openjdk.java.net/~ccheung/jdk15/8232081/webrev.01/ >>>> >>>> This looks much better to me. I wasn't sure if you were that >>>> concerned about catching the classes loaded by the Shutdown hooks, >>>> but it is good you are not as it seems problematic to me to trigger >>>> class loading etc after the shutdown hooks have executed - you may >>>> hit unexpected conditions. So this approach is good. >>>> >>>> A few minor comments: >>>> >>>> src/hotspot/share/memory/metaspaceShared.cpp >>>> src/hotspot/share/memory/metaspaceShared.hpp >>>> >>>> Why the change from TRAPS to "Thread* THREAD"? >>> I forgot why I changed it but I've changed it back and it still works. >> >> Ok. >> >>>> >>>> --- >>>> >>>> src/hotspot/share/oops/instanceKlass.cpp >>>> >>>> I'm not clear how verify_on is used so am unsure how the additional >>>> error state check may affect code unrelated to dynamic dumping ?? >>> >>> Some of the appcds tests by design have some classes which fail >>> verification. Without the change, the following assert would fail: >>> >>> void vtableEntry::verify(klassVtable* vt, outputStream* st) { >>> ?? Klass* vtklass = vt->klass(); >>> ?? if (vtklass->is_instance_klass() && >>> ????? (InstanceKlass::cast(vtklass)->major_version() >= >>> klassVtable::VTABLE_TRANSITIVE_OVERRIDE_VERSION)) { >>> *assert*(method() != NULL, "must have set method"); >>> ?? } >> >> Okay so you need to exclude for that case but ... >> >>> Here's the call stack during dynamic CDS dump to the above function: >>> >>> vtableEntry::verify(klassVtable *, outputStream *) : void >>> ???? klassVtable::verify(outputStream *, bool) : void >>> ???????? InstanceKlass::verify_on(outputStream *) : void >>> ???????????? Klass::verify() : void >>> ???????????????? ClassLoaderData::verify() : void >>> ???????????????????? ClassLoaderDataGraph::verify() : void >>> ???????????????????????? Universe::verify(enum VerifyOption, const >>> char *) : void >>> ???????????????????????????? Universe::verify(const char *) : void >>> DynamicArchiveBuilder::verify_universe(const char *) : void >>> DynamicArchiveBuilder::doit() : void (2 matches) >>> VM_PopulateDynamicDumpSharedSpace::doit() : void >>> VM_Operation::evaluate() : void >>> >>> The is_linked() function is implemented as: >>> >>> ???? bool is_linked() const?????????????????? { return _init_state >>> >= linked; } >>> >>> which includes 'initialization_error'. >> >> ... AFAICS Universe::verify is call from a number of places, all of >> which would reach this modified code. As that existing code has >> included classes that are in the error state then it seems to me that >> unless you can prove otherwise you need to keep the existing code >> as-is for the non-CDS dump case. >> >>> A previous version of the changeset (including the change in >>> question in instanceKlass.cpp) passed tier1 - 4 testing. Let me know >>> if I should run other tests to make sure the change is good. >> >> There may well not be a test that covers this. >> >> Thanks, >> David >> >>>> >>>> Also can I suggest (as you are touching this code) to delete the >>>> ancient comment: >>>> >>>> 3580???? // $$$ This used to be done only for m/s collections. >>>> Doing it >>>> 3581???? // always seemed a valid generalization.? (DLD -- 6/00) >>> >>> I'm glad that you suggested to remove the above comment. I've no >>> clue what it means either. >>> >>> updated webrev: >>> >>> http://cr.openjdk.java.net/~ccheung/jdk15/8232081/webrev.02/ >>> >>> thanks, >>> Calvin >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>>> I also updated the test to test the shutdown hook and >>>>> System.exit() scenarios. >>>>> >>>>> All appcds tests passed on my linux host. I'll run more tests >>>>> using mach5. >>>>> >>>>> thanks, >>>>> >>>>> Calvin >>>>> >>>>> >>>>>> >>>>>> What do you think? >>>>>> >>>>>> - Ioi >>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> >>>>>>>> MetaspaceShared::link_and_cleanup_shared_classes() has been >>>>>>>> modified to handle both static and dynamic CDS dump. For >>>>>>>> dynamic CDS dump, only classes loaded by the builtin class >>>>>>>> loaders will be linked. Local performance testing using javac >>>>>>>> on HelloWorld.java shows an improvement of >5%. >>>>>>>> >>>>>>>> Passed tier1 - 4 tests. >>>>>>>> >>>>>>>> thanks, >>>>>>>> >>>>>>>> Calvin >>>>>>>> >>>>>> From aph at redhat.com Thu Mar 5 18:06:49 2020 From: aph at redhat.com (Andrew Haley) Date: Thu, 5 Mar 2020 18:06:49 +0000 Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is inconsistent Message-ID: is_power_of_2() has always behaved oddly, and it still behaves oddly but in a different way since 8183574: Unify the is_power_of_2 functions. And this change breaks TCK on AArch64 systems. Previously, is_power_of_2(long) returned true for LONG_MIN, the most -ve signed long, but returned false for INT_MIN, the most -ve signed int. Also, is_power_of_2(long) had Undefined Behaviour (UB) for LONG_MIN: it overflowed from the most -ve long to the most +ve long. Now, is_power_of_2() has Undefined Behaviour for both int and long arguments when applied to their respective most negative values. In testing it returns true for both, but as this is UB it is not guaranteed. previous definition: inline bool is_power_of_2(intptr_t x) { return ((x != NoBits) && (mask_bits(x, x - 1) == NoBits)); } since JDK-8240615: template bool is_power_of_2(T x) { return (x != T(0)) && ((x & (x - 1)) == T(0)); } I do not believe that this behavioural change was deliberate. I suggest that we should change the definition of is_power_of_2() for all signed types such that it returns false for negative arguments. This makes the behaviour consistent and also avoids undefined behaviour. This is important because is_power_of_2 is sometimes used as a guard in ADfile patterns: instruct cmpI_branch_bit(cmpOpEqNe cmp, iRegIorL2I op1, immI op2, immI0 op3, label labl) %{ match(If cmp (CmpI (AndI op1 op2) op3)); predicate(is_power_of_2(n->in(2)->in(1)->in(2)->get_int())); effect(USE labl); ins_cost(BRANCH_COST); format %{ "tb$cmp $op1, $op2, $labl" %} ins_encode %{ Label* L = $labl$$label; Assembler::Condition cond = (Assembler::Condition)$cmp$$cmpcode; int bit = exact_log2($op2$$constant); ... This has a runtime error since JDK-8240615 because the initial is_power_of_2 test succeeds, but the later exact_log2 fails. exact_log2() takes an intptr_t, not an int. On 64-bit systems, intptr_t is a 64-bit signed type, so the first call to is_power_of_2 succeeds, whereas exact_log2 fails because its arg is sign-extended to 64 bits. http://cr.openjdk.java.net/~aph/8240615-1/ -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From stefan.karlsson at oracle.com Thu Mar 5 18:23:31 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 5 Mar 2020 19:23:31 +0100 Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is inconsistent In-Reply-To: References: Message-ID: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com> Hi Andrew, The implementation of is_power_of_2 looks good to me. Since this changes the value of is_power_of_2(LONG_MIN), did you try to figure out if any of the existing code relies on this returning true? It might be prudent to run this patch through testing with an assert catching any places where LONG_MIN is used as the argument. I can run that on our side, but need help for full platform coverage. You used std::numeric_limits::min() in the test. There's a comment in powerOfTwo.hpp stating: // Helper function to get the maximum positive value. Implemented here // since using std::numeric_limits::max() seems problematic on some // platforms. I don't know if this means that min() is problematic as well? Thanks for fixing this. StefanK On 2020-03-05 19:06, Andrew Haley wrote: > is_power_of_2() has always behaved oddly, and it still behaves oddly > but in a different way since 8183574: Unify the is_power_of_2 > functions. And this change breaks TCK on AArch64 systems. > > Previously, is_power_of_2(long) returned true for LONG_MIN, the most > -ve signed long, but returned false for INT_MIN, the most -ve signed > int. > > Also, is_power_of_2(long) had Undefined Behaviour (UB) for LONG_MIN: > it overflowed from the most -ve long to the most +ve long. > > Now, is_power_of_2() has Undefined Behaviour for both int and long > arguments when applied to their respective most negative values. In > testing it returns true for both, but as this is UB it is not > guaranteed. > > previous definition: > > inline bool is_power_of_2(intptr_t x) { > return ((x != NoBits) && (mask_bits(x, x - 1) == NoBits)); > } > > since JDK-8240615: > > template > bool is_power_of_2(T x) { > return (x != T(0)) && ((x & (x - 1)) == T(0)); > } > > I do not believe that this behavioural change was deliberate. > > I suggest that we should change the definition of is_power_of_2() for > all signed types such that it returns false for negative > arguments. This makes the behaviour consistent and also avoids > undefined behaviour. > > This is important because is_power_of_2 is sometimes used as a guard > in ADfile patterns: > > instruct cmpI_branch_bit(cmpOpEqNe cmp, iRegIorL2I op1, immI op2, immI0 op3, label labl) %{ > match(If cmp (CmpI (AndI op1 op2) op3)); > predicate(is_power_of_2(n->in(2)->in(1)->in(2)->get_int())); > effect(USE labl); > > ins_cost(BRANCH_COST); > format %{ "tb$cmp $op1, $op2, $labl" %} > ins_encode %{ > Label* L = $labl$$label; > Assembler::Condition cond = (Assembler::Condition)$cmp$$cmpcode; > int bit = exact_log2($op2$$constant); > ... > > This has a runtime error since JDK-8240615 because the initial > is_power_of_2 test succeeds, but the later exact_log2 > fails. exact_log2() takes an intptr_t, not an int. On 64-bit systems, > intptr_t is a 64-bit signed type, so the first call to is_power_of_2 > succeeds, whereas exact_log2 fails because its arg is sign-extended > to 64 bits. > > http://cr.openjdk.java.net/~aph/8240615-1/ > From claes.redestad at oracle.com Thu Mar 5 18:42:30 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Thu, 5 Mar 2020 19:42:30 +0100 Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is inconsistent In-Reply-To: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com> References: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com> Message-ID: <702314ab-17bc-1a3d-7384-6810beaec7b2@oracle.com> On 2020-03-05 19:23, Stefan Karlsson wrote: > You used std::numeric_limits::min() in the test. There's a comment in > powerOfTwo.hpp stating: > > // Helper function to get the maximum positive value. Implemented here > // since using std::numeric_limits::max() seems problematic on some > // platforms. > > I don't know if this means that min() is problematic as well? Me neither, but I think for test use it should be fine. There are some uses of ::max() in a few gtests already. For context I saw Solaris slowdebug builds fail when implementing powerOfTwo with use of std::numeric_limits. The build issue appeared only in slowdebug builds when numeric_limits was used in code that get heavily inlined, causing some linking issue. /Claes From stefan.karlsson at oracle.com Thu Mar 5 19:46:00 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 5 Mar 2020 20:46:00 +0100 Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is inconsistent In-Reply-To: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com> References: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com> Message-ID: I ran with this patch: diff --git a/src/hotspot/share/utilities/powerOfTwo.hpp b/src/hotspot/share/utilities/powerOfTwo.hpp --- a/src/hotspot/share/utilities/powerOfTwo.hpp +++ b/src/hotspot/share/utilities/powerOfTwo.hpp @@ -36,6 +36,7 @@ ?template ?bool is_power_of_2(T x) { +? assert(sizeof(x) != 8 || !IsSigned::value || x != std::numeric_limits::min(), "Caught min value"); ?? return (x > T(0)) && ((x & (x - 1)) == T(0)); ?} and it asserts in DaCapo in immL_Pow2 below: 0x00007f33727e26ea in State::_sub_Op_ConL (this=0x7f3340842f60, n=0x7f336cbb04b8) at src/hotspot/cpu/x86/x86_64.ad:3123 operand immL_Pow2() %{ ? predicate(is_power_of_2(n->get_long())); ? match(ConL); ? op_cost(15); ? format %{ %} ? interface(CONST_INTER); %} operand immL_NotPow2() %{ ? predicate(is_power_of_2(~n->get_long())); ? match(ConL); ? op_cost(15); ? format %{ %} ? interface(CONST_INTER); %} The predicate for Pow2 and NotPow2 will both be false because of this change. Is that a problem? StefanK On 2020-03-05 19:23, Stefan Karlsson wrote: > Hi Andrew, > > The implementation of is_power_of_2 looks good to me. > > Since this changes the value of is_power_of_2(LONG_MIN), did you try > to figure out if any of the existing code relies on this returning > true? It might be prudent to run this patch through testing with an > assert catching any places where LONG_MIN is used as the argument. I > can run that on our side, but need help for full platform coverage. > > You used std::numeric_limits::min() in the test. There's a comment > in powerOfTwo.hpp stating: > > // Helper function to get the maximum positive value. Implemented here > // since using std::numeric_limits::max() seems problematic on some > // platforms. > > I don't know if this means that min() is problematic as well? > > Thanks for fixing this. > > StefanK > > On 2020-03-05 19:06, Andrew Haley wrote: >> is_power_of_2() has always behaved oddly, and it still behaves oddly >> but in a different way since 8183574: Unify the is_power_of_2 >> functions. And this change breaks TCK on AArch64 systems. >> >> Previously, is_power_of_2(long) returned true for LONG_MIN, the most >> -ve signed long, but returned false for INT_MIN, the most -ve signed >> int. >> >> Also, is_power_of_2(long) had Undefined Behaviour (UB) for LONG_MIN: >> it overflowed from the most -ve long to the most +ve long. >> >> Now, is_power_of_2() has Undefined Behaviour for both int and long >> arguments when applied to their respective most negative values. In >> testing it returns true for both, but as this is UB it is not >> guaranteed. >> >> previous definition: >> >> inline bool is_power_of_2(intptr_t x) { >> ?? return ((x != NoBits) && (mask_bits(x, x - 1) == NoBits)); >> } >> >> since JDK-8240615: >> >> template >> bool is_power_of_2(T x) { >> ?? return (x != T(0)) && ((x & (x - 1)) == T(0)); >> } >> >> I do not believe that this behavioural change was deliberate. >> >> I suggest that we should change the definition of is_power_of_2() for >> all signed types such that it returns false for negative >> arguments. This makes the behaviour consistent and also avoids >> undefined behaviour. >> >> This is important because is_power_of_2 is sometimes used as a guard >> in ADfile patterns: >> >> instruct cmpI_branch_bit(cmpOpEqNe cmp, iRegIorL2I op1, immI op2, >> immI0 op3, label labl) %{ >> ?? match(If cmp (CmpI (AndI op1 op2) op3)); >> predicate(is_power_of_2(n->in(2)->in(1)->in(2)->get_int())); >> ?? effect(USE labl); >> >> ?? ins_cost(BRANCH_COST); >> ?? format %{ "tb$cmp $op1, $op2, $labl" %} >> ?? ins_encode %{ >> ???? Label* L = $labl$$label; >> ???? Assembler::Condition cond = (Assembler::Condition)$cmp$$cmpcode; >> ???? int bit = exact_log2($op2$$constant); >> ??? ... >> >> This has a runtime error since JDK-8240615 because the initial >> is_power_of_2 test succeeds, but the later exact_log2 >> fails. exact_log2() takes an intptr_t, not an int. On 64-bit systems, >> intptr_t is a 64-bit signed type, so the first call to is_power_of_2 >> succeeds, whereas exact_log2 fails because its arg is sign-extended >> to 64 bits. >> >> http://cr.openjdk.java.net/~aph/8240615-1/ >> > From calvin.cheung at oracle.com Thu Mar 5 19:48:49 2020 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Thu, 5 Mar 2020 11:48:49 -0800 Subject: RFR(S): 8232081: Try to link all classes during dynamic CDS dump In-Reply-To: <5fa95d9f-47a7-c1c7-90e5-87db0173e3b7@oracle.com> References: <9b40145a-db72-6313-3b98-5e5aea754e25@oracle.com> <1167ff6d-3997-6390-d020-d0946fad0af8@oracle.com> <3f373a33-c7ed-0237-1dd5-f8ac8e8794ee@oracle.com> <8ac4612b-fae3-bb16-f2ab-fa323c33ca51@oracle.com> <00090c43-1e0d-2b1c-d2a5-1b5f1b29199d@oracle.com> <7a54e00f-c049-a5ef-0afc-bb6c9d0f30ad@oracle.com> <5fa95d9f-47a7-c1c7-90e5-87db0173e3b7@oracle.com> Message-ID: <35a07978-29f7-8e16-c50a-0c97ee575871@oracle.com> On 3/5/20 9:17 AM, Ioi Lam wrote: > Hi Calvin, > > Looks good overall. I just have two comment: > > I think this is not needed: > > ?306 void ConstantPool::resolve_class_constants(TRAPS) { > ?307?? Arguments::assert_is_dumping_archive(); I've reverted this change. > > because this function is used to resolve all Strings in the statically > dumped classes to archive all the Strings. However, the archive heap > is not supported for the dynamic archive. So I think it's better to > avoid calling this function from LinkSharedClassesClosure for dynamic > dump. Now, the function will not be called with dynamic dump: 1714???????? if (DumpSharedSpaces) { 1715?????????? // The following function is used to resolve all Strings in the statically 1716?????????? // dumped classes to archive all the Strings. The archive heap is not supported 1717?????????? // for the dynamic archive. 1718 ik->constants()->resolve_class_constants(THREAD); 1719???????? } > > LinkSharedClassesClosure::_is_static -- maybe we can avoid adding this > field and just check "if (DumpSharedSpaces)" instead? This is a good simplification. btw, you've removed loader_type() from instanceKlass.hpp when you pushed the change for JDK-8240244. I've added it back as shared_loader_type(). updated webrev: ??? http://cr.openjdk.java.net/~ccheung/jdk15/8232081/webrev.04/ thanks, Calvin > > Thanks > - Ioi > > > > > > On 3/4/20 9:13 PM, Calvin Cheung wrote: >> Hi David, Ioi, >> >> Here's an updated webrev which doesn't involve changing >> InstanceKlass::verify_on() and made use of the changes for JDK-8240481. >> >> ??? http://cr.openjdk.java.net/~ccheung/jdk15/8232081/webrev.03/ >> >> thanks, >> Calvin >> >> On 3/1/20 10:14 PM, David Holmes wrote: >>> Hi Calvin, >>> >>> On 28/02/2020 4:12 pm, Calvin Cheung wrote: >>>> Hi David, >>>> >>>> On 2/27/20 5:40 PM, David Holmes wrote: >>>>> Hi Calvin, Ioi, >>>>> >>>>> Looking good - comments below. >>>>> >>>>> A meta-question: normal application classes are rarely loaded but >>>>> not linked so I'm a little surprised this is an issue. What is the >>>>> main source of such classes - generated classes like lambda forms? >>>>> Also why do we need to link them when loading from the archive if >>>>> they were never linked when the application actually executed ?? >>>> >>>> I saw that Ioi already answered the above. >>>> >>>> I'll try to answer your questions inline below.. >>> >>> Responses inline below ... >>> >>>>> >>>>> On 28/02/2020 10:48 am, Calvin Cheung wrote: >>>>>> Hi David, Ioi, >>>>>> >>>>>> Thanks for your review and suggestions. >>>>>> >>>>>> On 2/26/20 9:46 PM, Ioi Lam wrote: >>>>>>> >>>>>>> >>>>>>> On 2/26/20 7:50 PM, David Holmes wrote: >>>>>>>> Hi Calvin, >>>>>>>> >>>>>>>> Adding core-libs-dev as you are messing with their code :) >>>>>>>> >>>>>>>> On 27/02/2020 1:19 pm, Calvin Cheung wrote: >>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8232081 >>>>>>>>> webrev: >>>>>>>>> http://cr.openjdk.java.net/~ccheung/jdk15/8232081/webrev.00/ >>>>>>>>> >>>>>>>>> The proposed changeset for this RFE adds a >>>>>>>>> JVM_LinkClassesForCDS() function to be called from >>>>>>>>> java/lang/Shutdown to notify the JVM to link the classes >>>>>>>>> loaded by the builtin class loaders. The >>>>>>>> >>>>>>>> This would be much less disruptive if this was handled purely >>>>>>>> on the VM side once we have started shutdown. No need to make >>>>>>>> any changes to the Java side then, nor jvm.cpp. >>>>>>>> >>>>>>> >>>>>>> Hi David, >>>>>>> >>>>>>> To link the classes, we need to be able to run Java code -- when >>>>>>> linking a class X loaded by the app loader, X will be verified. >>>>>>> During verification of X, we may need to load additional classes >>>>>>> from the app loader, which executes Java code during its class >>>>>>> loading operations. >>>>>>> >>>>>>> We also need to handle all the exit conditions. As far as I can >>>>>>> tell, an app can exit the JVM in two ways: >>>>>>> >>>>>>> (1) Explicitly calling System.exit(). This will call >>>>>>> java.lang.Shutdown.exit() which does this: >>>>>>> >>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/0edc7fd0d7a3/src/java.base/share/classes/java/lang/Shutdown.java#l163 >>>>>>> >>>>>>> >>>>>>> ??????????? beforeHalt(); // native >>>>>>> ??????????? runHooks(); >>>>>>> ??????????? halt(status); >>>>>>> >>>>>>> (2) When all non-daemon threads have died (e.g., falling out of >>>>>>> the bottom of HelloWorld.main()). There's no explicit call to >>>>>>> System.exit(), but the JVM will proactively call >>>>>>> java.lang.Shutdown.shutdown() inside >>>>>>> JavaThread::invoke_shutdown_hooks() >>>>>>> >>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/0edc7fd0d7a3/src/hotspot/share/runtime/thread.cpp#l4331 >>>>>>> >>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/0edc7fd0d7a3/src/java.base/share/classes/java/lang/Shutdown.java#l184 >>>>>>> >>>>>>> >>>>>>> If we want to avoid modifying the Java code, I think we can >>>>>>> intercept JVM_BeforeHalt() and >>>>>>> JavaThread::invoke_shutdown_hooks(). This way we should be able >>>>>>> to handle all the classes (except those that are loaded inside >>>>>>> Shutdown.runHooks). >>>>>> >>>>>> I've implemented the above. No changes to the Java side. >>>>>> >>>>>> updated webrev: >>>>>> >>>>>> http://cr.openjdk.java.net/~ccheung/jdk15/8232081/webrev.01/ >>>>> >>>>> This looks much better to me. I wasn't sure if you were that >>>>> concerned about catching the classes loaded by the Shutdown hooks, >>>>> but it is good you are not as it seems problematic to me to >>>>> trigger class loading etc after the shutdown hooks have executed - >>>>> you may hit unexpected conditions. So this approach is good. >>>>> >>>>> A few minor comments: >>>>> >>>>> src/hotspot/share/memory/metaspaceShared.cpp >>>>> src/hotspot/share/memory/metaspaceShared.hpp >>>>> >>>>> Why the change from TRAPS to "Thread* THREAD"? >>>> I forgot why I changed it but I've changed it back and it still works. >>> >>> Ok. >>> >>>>> >>>>> --- >>>>> >>>>> src/hotspot/share/oops/instanceKlass.cpp >>>>> >>>>> I'm not clear how verify_on is used so am unsure how the >>>>> additional error state check may affect code unrelated to dynamic >>>>> dumping ?? >>>> >>>> Some of the appcds tests by design have some classes which fail >>>> verification. Without the change, the following assert would fail: >>>> >>>> void vtableEntry::verify(klassVtable* vt, outputStream* st) { >>>> ?? Klass* vtklass = vt->klass(); >>>> ?? if (vtklass->is_instance_klass() && >>>> ????? (InstanceKlass::cast(vtklass)->major_version() >= >>>> klassVtable::VTABLE_TRANSITIVE_OVERRIDE_VERSION)) { >>>> *assert*(method() != NULL, "must have set method"); >>>> ?? } >>> >>> Okay so you need to exclude for that case but ... >>> >>>> Here's the call stack during dynamic CDS dump to the above function: >>>> >>>> vtableEntry::verify(klassVtable *, outputStream *) : void >>>> ???? klassVtable::verify(outputStream *, bool) : void >>>> ???????? InstanceKlass::verify_on(outputStream *) : void >>>> ???????????? Klass::verify() : void >>>> ???????????????? ClassLoaderData::verify() : void >>>> ???????????????????? ClassLoaderDataGraph::verify() : void >>>> ???????????????????????? Universe::verify(enum VerifyOption, const >>>> char *) : void >>>> ???????????????????????????? Universe::verify(const char *) : void >>>> DynamicArchiveBuilder::verify_universe(const char *) : void >>>> DynamicArchiveBuilder::doit() : void (2 matches) >>>> VM_PopulateDynamicDumpSharedSpace::doit() : void >>>> VM_Operation::evaluate() : void >>>> >>>> The is_linked() function is implemented as: >>>> >>>> ???? bool is_linked() const?????????????????? { return _init_state >>>> >= linked; } >>>> >>>> which includes 'initialization_error'. >>> >>> ... AFAICS Universe::verify is call from a number of places, all of >>> which would reach this modified code. As that existing code has >>> included classes that are in the error state then it seems to me >>> that unless you can prove otherwise you need to keep the existing >>> code as-is for the non-CDS dump case. >>> >>>> A previous version of the changeset (including the change in >>>> question in instanceKlass.cpp) passed tier1 - 4 testing. Let me >>>> know if I should run other tests to make sure the change is good. >>> >>> There may well not be a test that covers this. >>> >>> Thanks, >>> David >>> >>>>> >>>>> Also can I suggest (as you are touching this code) to delete the >>>>> ancient comment: >>>>> >>>>> 3580???? // $$$ This used to be done only for m/s collections. >>>>> Doing it >>>>> 3581???? // always seemed a valid generalization.? (DLD -- 6/00) >>>> >>>> I'm glad that you suggested to remove the above comment. I've no >>>> clue what it means either. >>>> >>>> updated webrev: >>>> >>>> http://cr.openjdk.java.net/~ccheung/jdk15/8232081/webrev.02/ >>>> >>>> thanks, >>>> Calvin >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>>> I also updated the test to test the shutdown hook and >>>>>> System.exit() scenarios. >>>>>> >>>>>> All appcds tests passed on my linux host. I'll run more tests >>>>>> using mach5. >>>>>> >>>>>> thanks, >>>>>> >>>>>> Calvin >>>>>> >>>>>> >>>>>>> >>>>>>> What do you think? >>>>>>> >>>>>>> - Ioi >>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> >>>>>>>>> MetaspaceShared::link_and_cleanup_shared_classes() has been >>>>>>>>> modified to handle both static and dynamic CDS dump. For >>>>>>>>> dynamic CDS dump, only classes loaded by the builtin class >>>>>>>>> loaders will be linked. Local performance testing using javac >>>>>>>>> on HelloWorld.java shows an improvement of >5%. >>>>>>>>> >>>>>>>>> Passed tier1 - 4 tests. >>>>>>>>> >>>>>>>>> thanks, >>>>>>>>> >>>>>>>>> Calvin >>>>>>>>> >>>>>>> > From kevin.walls at oracle.com Thu Mar 5 21:01:53 2020 From: kevin.walls at oracle.com (Kevin Walls) Date: Thu, 5 Mar 2020 21:01:53 +0000 Subject: RFR(S): 8240295: hs_err elapsed time in seconds is not accurate enough In-Reply-To: <20045e23-c736-5289-866e-9df5a09101a8@oracle.com> References: <4a3d7c8e-df9e-51de-77e4-775be1480392@oracle.com> <92ada19d-7200-6bb7-b174-9da8a166aea7@oracle.com> <053864a8-617f-fcac-fc26-220d161e3e55@oracle.com> <20045e23-c736-5289-866e-9df5a09101a8@oracle.com> Message-ID: Great, thanks David. On 05/03/2020 10:38, David Holmes wrote: > Thanks Kevin. I think this is the less risky change and achieves the > goal. > > David > > On 5/03/2020 8:00 pm, Kevin Walls wrote: >> Thanks - >> >> I had tried some ideas in the simple fashion, and we can use %06d >> formatting.... OK maybe such formatting is not as "bad" as %f... >> >> (glibc parses the int width specified without allocation.? We provide >> the output buffer, I don't think we will cause? vfprintf code to >> alloca or malloc.) >> >> I can offer a second version below that uses %d only.? Testing >> alongside %f in the same line, it retains the same value and >> position, e.g. >> >> Time: Thu Mar? 5 08:57:50 2020 UTC elapsed time: f: 2.001065 int: >> 2.001065 (raw int: 1065) seconds (0d 0h 0m 2s) >> >> Output example from the hg diff below (not from the same run): >> >> Time: Thu Mar? 5 09:28:01 2020 UTC elapsed time: 2.000611 seconds (0d >> 0h 0m 2s) >> >> >> Thanks! >> Kevin >> >> >> $ hg diff >> diff --git a/src/hotspot/share/runtime/os.cpp >> b/src/hotspot/share/runtime/os.cpp >> --- a/src/hotspot/share/runtime/os.cpp >> +++ b/src/hotspot/share/runtime/os.cpp >> @@ -1016,10 +1016,9 @@ >> ??? } >> >> ??? double t = os::elapsedTime(); >> -? // NOTE: It tends to crash after a SEGV if we want to >> printf("%f",...) in >> -? //?????? Linux. Must be a bug in glibc ? Workaround is to round >> "t" to int >> -? //?????? before printf. We lost some precision, but who cares? >> +? // NOTE: a crash using printf("%f",...) on Linux was historically >> noted here. >> ??? int eltime = (int)t;? // elapsed time in seconds >> +? int eltimeFraction = (int) ((t - eltime) * 1000000); >> >> ??? // print elapsed time in a human-readable format: >> ??? int eldays = eltime / secs_per_day; >> @@ -1029,7 +1028,7 @@ >> ??? int elmins = (eltime - day_secs - hour_secs) / secs_per_min; >> ??? int minute_secs = elmins * secs_per_min; >> ??? int elsecs = (eltime - day_secs - hour_secs - minute_secs); >> -? st->print_cr(" elapsed time: %d seconds (%dd %dh %dm %ds)", >> eltime, eldays, elhours, elmins, elsecs); >> +? st->print_cr(" elapsed time: %d.%06d seconds (%dd %dh %dm %ds)", >> eltime, eltimeFraction, eldays, elhours, elmins, elsecs); >> ??} >> >> >> >> On 05/03/2020 00:57, David Holmes wrote: >>> On 4/03/2020 8:44 am, Kevin Walls wrote: >>>> >>>> Thanks David - >>>> >>>> Yes there are situations where hs_err fails, and few people are >>>> sadder than me >>>> when that happens 8-) , so I was thinking about how scared to be by >>>> the comment. >>>> >>>> With the safety net of the error handler for the steps of the >>>> hs_err file >>>> (which works well, we see it invoked frequently), it looks >>>> reasonable to use >>>> %f as we might do other slightly questionable things for a signal >>>> handler. >>>> >>>> Corrupting locale information or floating point state might >>>> possibly cause >>>> problems, but if I cause a fake crash in print_date_and_time the error >>>> handler recovers and the report continues. >>> >>> That is good to know. >>> >>>> Thinking about printing with two ints, seconds and fractions: >>>> I don't see anything already that returns such a time in two >>>> components in the >>>> JVM, so we might implement a new form of os::javaTimeNanos() or >>>> similar that >>>> returns the two parts, and do that on each platform. >>> >>> I was thinking of something simple/crude ... >>> >>>> I didn't yet come up with anything to do in os::print_date_and_time() >>>> which will take the fractional part of the double, and print just >>>> the fraction as an int, without using any library / %f facilities. >>> >>> ... just using e.g. (untested) >>> >>> double t = os::elapsedTime(); >>> int secs =? (int) t; >>> int micros =? (int)((t - secs) * 100000); >>> printf("%d.%d", secs, micros); >>> >>> with appropriate width specifiers to get the formatting right. >>> >>> Cheers, >>> David >>> >>>> >>>> If you're still concerned I could revisit these or some other idea. >>>> >>>> Genuine laugh out loud moment for me, I backported the elapsed time >>>> logging from >>>> 6u4 to 5u19 (https://bugs.openjdk.java.net/browse/JDK-6447157) (2007). >>>> (I said before jdk5 was created, I should have said before it was >>>> in mercurial.) >>>> >>>> Thanks >>>> Kevin >>>> >>>> >>>> On 03/03/2020 01:11, David Holmes wrote: >>>>> Hi Kevin, >>>>> >>>>> On 2/03/2020 8:48 pm, Kevin Walls wrote: >>>>>> Oops, and with the bug ID in the title and JBS link: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8240295 >>>>>> >>>>>> >>>>>> On 02/03/2020 10:47, Kevin Walls wrote: >>>>>>> Hi, >>>>>>> >>>>>>> (s11y and runtime opinions both relevant) >>>>>>> >>>>>>> A few times in the last month I've really wanted to compare the >>>>>>> Events logged in the hs_err file, and the time of the JVM's crash. >>>>>>> >>>>>>> "elapsed time" in hs_err is only accurate to one second, and has >>>>>>> been since before jdk5 was created. >>>>>>> >>>>>>> The diff below changes the format string and uses the >>>>>>> non-rounded time value (I don't see a need to change the other >>>>>>> integer arithmetic here), and we can enjoy hs_errs with detail >>>>>>> like: >>>>>>> >>>>>>> ... >>>>>>> Time: Mon Mar? 2 09:57:13 2020 UTC elapsed time: 5.490135 >>>>>>> seconds (0d 0h 0m 5s) >>>>>>> ... >>>>>>> >>>>>>> Thanks >>>>>>> Kevin >>>>>>> >>>>>>> >>>>>>> /jdk/open$ hg diff >>>>>>> diff --git a/src/hotspot/share/runtime/os.cpp >>>>>>> b/src/hotspot/share/runtime/os.cpp >>>>>>> --- a/src/hotspot/share/runtime/os.cpp >>>>>>> +++ b/src/hotspot/share/runtime/os.cpp >>>>>>> @@ -1016,9 +1016,8 @@ >>>>>>> ?? } >>>>>>> >>>>>>> ?? double t = os::elapsedTime(); >>>>>>> -? // NOTE: It tends to crash after a SEGV if we want to >>>>>>> printf("%f",...) in >>>>>>> -? //?????? Linux. Must be a bug in glibc ? Workaround is to >>>>>>> round "t" to int >>>>>>> -? //?????? before printf. We lost some precision, but who cares? >>>>>>> +? // NOTE: a crash using printf("%f",...) on Linux was >>>>>>> historically noted here >>>>>>> +? //?????? (before the jdk5 repo was created). >>>>> >>>>> Just because it is old doesn't mean it no longer applies. printf >>>>> is not async-signal-safe - we know that but we try to use it >>>>> anyway. Maybe %f is even less async-signal-safe? >>>>> >>>>> This may get through testing okay but cause problems with real >>>>> crashes in the field. >>>>> >>>>> What about breaking the time up into two ints: seconds and nanos? >>>>> >>>>> Cheers, >>>>> David >>>>> ----- >>>>> >>>>>>> ?? int eltime = (int)t;? // elapsed time in seconds >>>>>>> >>>>>>> ?? // print elapsed time in a human-readable format: >>>>>>> @@ -1029,7 +1028,7 @@ >>>>>>> ?? int elmins = (eltime - day_secs - hour_secs) / secs_per_min; >>>>>>> ?? int minute_secs = elmins * secs_per_min; >>>>>>> ?? int elsecs = (eltime - day_secs - hour_secs - minute_secs); >>>>>>> -? st->print_cr(" elapsed time: %d seconds (%dd %dh %dm %ds)", >>>>>>> eltime, eldays, elhours, elmins, elsecs); >>>>>>> +? st->print_cr(" elapsed time: %f seconds (%dd %dh %dm %ds)", >>>>>>> t, eldays, elhours, elmins, elsecs); >>>>>>> ?} >>>>>>> >>>>>>> From ioi.lam at oracle.com Fri Mar 6 05:16:33 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 5 Mar 2020 21:16:33 -0800 Subject: RFR(S): 8232081: Try to link all classes during dynamic CDS dump In-Reply-To: <35a07978-29f7-8e16-c50a-0c97ee575871@oracle.com> References: <9b40145a-db72-6313-3b98-5e5aea754e25@oracle.com> <1167ff6d-3997-6390-d020-d0946fad0af8@oracle.com> <3f373a33-c7ed-0237-1dd5-f8ac8e8794ee@oracle.com> <8ac4612b-fae3-bb16-f2ab-fa323c33ca51@oracle.com> <00090c43-1e0d-2b1c-d2a5-1b5f1b29199d@oracle.com> <7a54e00f-c049-a5ef-0afc-bb6c9d0f30ad@oracle.com> <5fa95d9f-47a7-c1c7-90e5-87db0173e3b7@oracle.com> <35a07978-29f7-8e16-c50a-0c97ee575871@oracle.com> Message-ID: <209d74da-fdb9-1ca1-4410-75c530cfd575@oracle.com> Hi Calvin, Looks good. In JDK-8240244, I removed "ik->loader_type() != 0" because it's unclear what it means. I replaced with a new method InstanceKlass::is_shared_unregistered_class(). I think you can use this in your patch instead of adding loader_type() back. No need for new webrev. Thanks - Ioi On 3/5/20 11:48 AM, Calvin Cheung wrote: > > On 3/5/20 9:17 AM, Ioi Lam wrote: >> Hi Calvin, >> >> Looks good overall. I just have two comment: >> >> I think this is not needed: >> >> ?306 void ConstantPool::resolve_class_constants(TRAPS) { >> ?307?? Arguments::assert_is_dumping_archive(); > I've reverted this change. >> >> because this function is used to resolve all Strings in the >> statically dumped classes to archive all the Strings. However, the >> archive heap is not supported for the dynamic archive. So I think >> it's better to avoid calling this function from >> LinkSharedClassesClosure for dynamic dump. > Now, the function will not be called with dynamic dump: > 1714???????? if (DumpSharedSpaces) { > 1715?????????? // The following function is used to resolve all > Strings in the statically > 1716?????????? // dumped classes to archive all the Strings. The > archive heap is not supported > 1717?????????? // for the dynamic archive. > 1718 ik->constants()->resolve_class_constants(THREAD); > 1719???????? } >> >> LinkSharedClassesClosure::_is_static -- maybe we can avoid adding >> this field and just check "if (DumpSharedSpaces)" instead? > > This is a good simplification. > > btw, you've removed loader_type() from instanceKlass.hpp when you > pushed the change for JDK-8240244. I've added it back as > shared_loader_type(). > > updated webrev: > > ??? http://cr.openjdk.java.net/~ccheung/jdk15/8232081/webrev.04/ > > thanks, > > Calvin > >> >> Thanks >> - Ioi >> >> >> >> >> >> On 3/4/20 9:13 PM, Calvin Cheung wrote: >>> Hi David, Ioi, >>> >>> Here's an updated webrev which doesn't involve changing >>> InstanceKlass::verify_on() and made use of the changes for JDK-8240481. >>> >>> http://cr.openjdk.java.net/~ccheung/jdk15/8232081/webrev.03/ >>> >>> thanks, >>> Calvin >>> >>> On 3/1/20 10:14 PM, David Holmes wrote: >>>> Hi Calvin, >>>> >>>> On 28/02/2020 4:12 pm, Calvin Cheung wrote: >>>>> Hi David, >>>>> >>>>> On 2/27/20 5:40 PM, David Holmes wrote: >>>>>> Hi Calvin, Ioi, >>>>>> >>>>>> Looking good - comments below. >>>>>> >>>>>> A meta-question: normal application classes are rarely loaded but >>>>>> not linked so I'm a little surprised this is an issue. What is >>>>>> the main source of such classes - generated classes like lambda >>>>>> forms? Also why do we need to link them when loading from the >>>>>> archive if they were never linked when the application actually >>>>>> executed ?? >>>>> >>>>> I saw that Ioi already answered the above. >>>>> >>>>> I'll try to answer your questions inline below.. >>>> >>>> Responses inline below ... >>>> >>>>>> >>>>>> On 28/02/2020 10:48 am, Calvin Cheung wrote: >>>>>>> Hi David, Ioi, >>>>>>> >>>>>>> Thanks for your review and suggestions. >>>>>>> >>>>>>> On 2/26/20 9:46 PM, Ioi Lam wrote: >>>>>>>> >>>>>>>> >>>>>>>> On 2/26/20 7:50 PM, David Holmes wrote: >>>>>>>>> Hi Calvin, >>>>>>>>> >>>>>>>>> Adding core-libs-dev as you are messing with their code :) >>>>>>>>> >>>>>>>>> On 27/02/2020 1:19 pm, Calvin Cheung wrote: >>>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8232081 >>>>>>>>>> webrev: >>>>>>>>>> http://cr.openjdk.java.net/~ccheung/jdk15/8232081/webrev.00/ >>>>>>>>>> >>>>>>>>>> The proposed changeset for this RFE adds a >>>>>>>>>> JVM_LinkClassesForCDS() function to be called from >>>>>>>>>> java/lang/Shutdown to notify the JVM to link the classes >>>>>>>>>> loaded by the builtin class loaders. The >>>>>>>>> >>>>>>>>> This would be much less disruptive if this was handled purely >>>>>>>>> on the VM side once we have started shutdown. No need to make >>>>>>>>> any changes to the Java side then, nor jvm.cpp. >>>>>>>>> >>>>>>>> >>>>>>>> Hi David, >>>>>>>> >>>>>>>> To link the classes, we need to be able to run Java code -- >>>>>>>> when linking a class X loaded by the app loader, X will be >>>>>>>> verified. During verification of X, we may need to load >>>>>>>> additional classes from the app loader, which executes Java >>>>>>>> code during its class loading operations. >>>>>>>> >>>>>>>> We also need to handle all the exit conditions. As far as I can >>>>>>>> tell, an app can exit the JVM in two ways: >>>>>>>> >>>>>>>> (1) Explicitly calling System.exit(). This will call >>>>>>>> java.lang.Shutdown.exit() which does this: >>>>>>>> >>>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/0edc7fd0d7a3/src/java.base/share/classes/java/lang/Shutdown.java#l163 >>>>>>>> >>>>>>>> >>>>>>>> ??????????? beforeHalt(); // native >>>>>>>> ??????????? runHooks(); >>>>>>>> ??????????? halt(status); >>>>>>>> >>>>>>>> (2) When all non-daemon threads have died (e.g., falling out of >>>>>>>> the bottom of HelloWorld.main()). There's no explicit call to >>>>>>>> System.exit(), but the JVM will proactively call >>>>>>>> java.lang.Shutdown.shutdown() inside >>>>>>>> JavaThread::invoke_shutdown_hooks() >>>>>>>> >>>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/0edc7fd0d7a3/src/hotspot/share/runtime/thread.cpp#l4331 >>>>>>>> >>>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/0edc7fd0d7a3/src/java.base/share/classes/java/lang/Shutdown.java#l184 >>>>>>>> >>>>>>>> >>>>>>>> If we want to avoid modifying the Java code, I think we can >>>>>>>> intercept JVM_BeforeHalt() and >>>>>>>> JavaThread::invoke_shutdown_hooks(). This way we should be able >>>>>>>> to handle all the classes (except those that are loaded inside >>>>>>>> Shutdown.runHooks). >>>>>>> >>>>>>> I've implemented the above. No changes to the Java side. >>>>>>> >>>>>>> updated webrev: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~ccheung/jdk15/8232081/webrev.01/ >>>>>> >>>>>> This looks much better to me. I wasn't sure if you were that >>>>>> concerned about catching the classes loaded by the Shutdown >>>>>> hooks, but it is good you are not as it seems problematic to me >>>>>> to trigger class loading etc after the shutdown hooks have >>>>>> executed - you may hit unexpected conditions. So this approach is >>>>>> good. >>>>>> >>>>>> A few minor comments: >>>>>> >>>>>> src/hotspot/share/memory/metaspaceShared.cpp >>>>>> src/hotspot/share/memory/metaspaceShared.hpp >>>>>> >>>>>> Why the change from TRAPS to "Thread* THREAD"? >>>>> I forgot why I changed it but I've changed it back and it still >>>>> works. >>>> >>>> Ok. >>>> >>>>>> >>>>>> --- >>>>>> >>>>>> src/hotspot/share/oops/instanceKlass.cpp >>>>>> >>>>>> I'm not clear how verify_on is used so am unsure how the >>>>>> additional error state check may affect code unrelated to dynamic >>>>>> dumping ?? >>>>> >>>>> Some of the appcds tests by design have some classes which fail >>>>> verification. Without the change, the following assert would fail: >>>>> >>>>> void vtableEntry::verify(klassVtable* vt, outputStream* st) { >>>>> ?? Klass* vtklass = vt->klass(); >>>>> ?? if (vtklass->is_instance_klass() && >>>>> ????? (InstanceKlass::cast(vtklass)->major_version() >= >>>>> klassVtable::VTABLE_TRANSITIVE_OVERRIDE_VERSION)) { >>>>> *assert*(method() != NULL, "must have set method"); >>>>> ?? } >>>> >>>> Okay so you need to exclude for that case but ... >>>> >>>>> Here's the call stack during dynamic CDS dump to the above function: >>>>> >>>>> vtableEntry::verify(klassVtable *, outputStream *) : void >>>>> ???? klassVtable::verify(outputStream *, bool) : void >>>>> ???????? InstanceKlass::verify_on(outputStream *) : void >>>>> ???????????? Klass::verify() : void >>>>> ???????????????? ClassLoaderData::verify() : void >>>>> ???????????????????? ClassLoaderDataGraph::verify() : void >>>>> ???????????????????????? Universe::verify(enum VerifyOption, const >>>>> char *) : void >>>>> ???????????????????????????? Universe::verify(const char *) : void >>>>> DynamicArchiveBuilder::verify_universe(const char *) : void >>>>> DynamicArchiveBuilder::doit() : void (2 matches) >>>>> VM_PopulateDynamicDumpSharedSpace::doit() : void >>>>> VM_Operation::evaluate() : void >>>>> >>>>> The is_linked() function is implemented as: >>>>> >>>>> ???? bool is_linked() const?????????????????? { return _init_state >>>>> >= linked; } >>>>> >>>>> which includes 'initialization_error'. >>>> >>>> ... AFAICS Universe::verify is call from a number of places, all of >>>> which would reach this modified code. As that existing code has >>>> included classes that are in the error state then it seems to me >>>> that unless you can prove otherwise you need to keep the existing >>>> code as-is for the non-CDS dump case. >>>> >>>>> A previous version of the changeset (including the change in >>>>> question in instanceKlass.cpp) passed tier1 - 4 testing. Let me >>>>> know if I should run other tests to make sure the change is good. >>>> >>>> There may well not be a test that covers this. >>>> >>>> Thanks, >>>> David >>>> >>>>>> >>>>>> Also can I suggest (as you are touching this code) to delete the >>>>>> ancient comment: >>>>>> >>>>>> 3580???? // $$$ This used to be done only for m/s collections. >>>>>> Doing it >>>>>> 3581???? // always seemed a valid generalization.? (DLD -- 6/00) >>>>> >>>>> I'm glad that you suggested to remove the above comment. I've no >>>>> clue what it means either. >>>>> >>>>> updated webrev: >>>>> >>>>> http://cr.openjdk.java.net/~ccheung/jdk15/8232081/webrev.02/ >>>>> >>>>> thanks, >>>>> Calvin >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>> >>>>>>> I also updated the test to test the shutdown hook and >>>>>>> System.exit() scenarios. >>>>>>> >>>>>>> All appcds tests passed on my linux host. I'll run more tests >>>>>>> using mach5. >>>>>>> >>>>>>> thanks, >>>>>>> >>>>>>> Calvin >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> What do you think? >>>>>>>> >>>>>>>> - Ioi >>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> David >>>>>>>>> >>>>>>>>>> MetaspaceShared::link_and_cleanup_shared_classes() has been >>>>>>>>>> modified to handle both static and dynamic CDS dump. For >>>>>>>>>> dynamic CDS dump, only classes loaded by the builtin class >>>>>>>>>> loaders will be linked. Local performance testing using javac >>>>>>>>>> on HelloWorld.java shows an improvement of >5%. >>>>>>>>>> >>>>>>>>>> Passed tier1 - 4 tests. >>>>>>>>>> >>>>>>>>>> thanks, >>>>>>>>>> >>>>>>>>>> Calvin >>>>>>>>>> >>>>>>>> >> From aph at redhat.com Fri Mar 6 14:48:42 2020 From: aph at redhat.com (Andrew Haley) Date: Fri, 6 Mar 2020 14:48:42 +0000 Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is inconsistent In-Reply-To: References: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com> Message-ID: <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com> On 3/5/20 7:46 PM, Stefan Karlsson wrote: > I ran with this patch: > diff --git a/src/hotspot/share/utilities/powerOfTwo.hpp > b/src/hotspot/share/utilities/powerOfTwo.hpp > --- a/src/hotspot/share/utilities/powerOfTwo.hpp > +++ b/src/hotspot/share/utilities/powerOfTwo.hpp > @@ -36,6 +36,7 @@ > > ?template > ?bool is_power_of_2(T x) { > +? assert(sizeof(x) != 8 || !IsSigned::value || x != > std::numeric_limits::min(), "Caught min value"); > ?? return (x > T(0)) && ((x & (x - 1)) == T(0)); > ?} > > and it asserts in DaCapo in immL_Pow2 below: > > 0x00007f33727e26ea in State::_sub_Op_ConL (this=0x7f3340842f60, > n=0x7f336cbb04b8) at src/hotspot/cpu/x86/x86_64.ad:3123 > > operand immL_Pow2() > %{ > ? predicate(is_power_of_2(n->get_long())); > ? match(ConL); > > ? op_cost(15); > ? format %{ %} > ? interface(CONST_INTER); > %} > > operand immL_NotPow2() > %{ > ? predicate(is_power_of_2(~n->get_long())); > ? match(ConL); > > ? op_cost(15); > ? format %{ %} > ? interface(CONST_INTER); > %} > > The predicate for Pow2 and NotPow2 will both be false because of this > change. Is that a problem? Probably. I'll do some more digging. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From mark.reinhold at oracle.com Fri Mar 6 16:37:54 2020 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Fri, 6 Mar 2020 17:37:54 +0100 (CET) Subject: New candidate JEP: 374: Disable and Deprecate Biased Locking Message-ID: <20200306163754.841713194E1@eggemoggin.niobe.net> https://openjdk.java.net/jeps/374 - Mark From calvin.cheung at oracle.com Fri Mar 6 16:44:21 2020 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Fri, 6 Mar 2020 08:44:21 -0800 Subject: RFR(S): 8232081: Try to link all classes during dynamic CDS dump In-Reply-To: <209d74da-fdb9-1ca1-4410-75c530cfd575@oracle.com> References: <9b40145a-db72-6313-3b98-5e5aea754e25@oracle.com> <1167ff6d-3997-6390-d020-d0946fad0af8@oracle.com> <3f373a33-c7ed-0237-1dd5-f8ac8e8794ee@oracle.com> <8ac4612b-fae3-bb16-f2ab-fa323c33ca51@oracle.com> <00090c43-1e0d-2b1c-d2a5-1b5f1b29199d@oracle.com> <7a54e00f-c049-a5ef-0afc-bb6c9d0f30ad@oracle.com> <5fa95d9f-47a7-c1c7-90e5-87db0173e3b7@oracle.com> <35a07978-29f7-8e16-c50a-0c97ee575871@oracle.com> <209d74da-fdb9-1ca1-4410-75c530cfd575@oracle.com> Message-ID: <916938b1-b7f2-22da-786f-f1a772984d80@oracle.com> Hi Ioi, On 3/5/20 9:16 PM, Ioi Lam wrote: > Hi Calvin, > > Looks good. Thanks for taking another look. > > In JDK-8240244, I removed "ik->loader_type() != 0" because it's > unclear what it means. I replaced with a new method > InstanceKlass::is_shared_unregistered_class(). I think you can use > this in your patch instead of adding loader_type() back. I'll make the change and do more testing before push. thanks, Calvin > > No need for new webrev. > > Thanks > - Ioi > > > On 3/5/20 11:48 AM, Calvin Cheung wrote: >> >> On 3/5/20 9:17 AM, Ioi Lam wrote: >>> Hi Calvin, >>> >>> Looks good overall. I just have two comment: >>> >>> I think this is not needed: >>> >>> ?306 void ConstantPool::resolve_class_constants(TRAPS) { >>> ?307?? Arguments::assert_is_dumping_archive(); >> I've reverted this change. >>> >>> because this function is used to resolve all Strings in the >>> statically dumped classes to archive all the Strings. However, the >>> archive heap is not supported for the dynamic archive. So I think >>> it's better to avoid calling this function from >>> LinkSharedClassesClosure for dynamic dump. >> Now, the function will not be called with dynamic dump: >> 1714???????? if (DumpSharedSpaces) { >> 1715?????????? // The following function is used to resolve all >> Strings in the statically >> 1716?????????? // dumped classes to archive all the Strings. The >> archive heap is not supported >> 1717?????????? // for the dynamic archive. >> 1718 ik->constants()->resolve_class_constants(THREAD); >> 1719???????? } >>> >>> LinkSharedClassesClosure::_is_static -- maybe we can avoid adding >>> this field and just check "if (DumpSharedSpaces)" instead? >> >> This is a good simplification. >> >> btw, you've removed loader_type() from instanceKlass.hpp when you >> pushed the change for JDK-8240244. I've added it back as >> shared_loader_type(). >> >> updated webrev: >> >> ??? http://cr.openjdk.java.net/~ccheung/jdk15/8232081/webrev.04/ >> >> thanks, >> >> Calvin >> >>> >>> Thanks >>> - Ioi >>> >>> >>> >>> >>> >>> On 3/4/20 9:13 PM, Calvin Cheung wrote: >>>> Hi David, Ioi, >>>> >>>> Here's an updated webrev which doesn't involve changing >>>> InstanceKlass::verify_on() and made use of the changes for >>>> JDK-8240481. >>>> >>>> http://cr.openjdk.java.net/~ccheung/jdk15/8232081/webrev.03/ >>>> >>>> thanks, >>>> Calvin >>>> >>>> On 3/1/20 10:14 PM, David Holmes wrote: >>>>> Hi Calvin, >>>>> >>>>> On 28/02/2020 4:12 pm, Calvin Cheung wrote: >>>>>> Hi David, >>>>>> >>>>>> On 2/27/20 5:40 PM, David Holmes wrote: >>>>>>> Hi Calvin, Ioi, >>>>>>> >>>>>>> Looking good - comments below. >>>>>>> >>>>>>> A meta-question: normal application classes are rarely loaded >>>>>>> but not linked so I'm a little surprised this is an issue. What >>>>>>> is the main source of such classes - generated classes like >>>>>>> lambda forms? Also why do we need to link them when loading from >>>>>>> the archive if they were never linked when the application >>>>>>> actually executed ?? >>>>>> >>>>>> I saw that Ioi already answered the above. >>>>>> >>>>>> I'll try to answer your questions inline below.. >>>>> >>>>> Responses inline below ... >>>>> >>>>>>> >>>>>>> On 28/02/2020 10:48 am, Calvin Cheung wrote: >>>>>>>> Hi David, Ioi, >>>>>>>> >>>>>>>> Thanks for your review and suggestions. >>>>>>>> >>>>>>>> On 2/26/20 9:46 PM, Ioi Lam wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2/26/20 7:50 PM, David Holmes wrote: >>>>>>>>>> Hi Calvin, >>>>>>>>>> >>>>>>>>>> Adding core-libs-dev as you are messing with their code :) >>>>>>>>>> >>>>>>>>>> On 27/02/2020 1:19 pm, Calvin Cheung wrote: >>>>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8232081 >>>>>>>>>>> webrev: >>>>>>>>>>> http://cr.openjdk.java.net/~ccheung/jdk15/8232081/webrev.00/ >>>>>>>>>>> >>>>>>>>>>> The proposed changeset for this RFE adds a >>>>>>>>>>> JVM_LinkClassesForCDS() function to be called from >>>>>>>>>>> java/lang/Shutdown to notify the JVM to link the classes >>>>>>>>>>> loaded by the builtin class loaders. The >>>>>>>>>> >>>>>>>>>> This would be much less disruptive if this was handled purely >>>>>>>>>> on the VM side once we have started shutdown. No need to make >>>>>>>>>> any changes to the Java side then, nor jvm.cpp. >>>>>>>>>> >>>>>>>>> >>>>>>>>> Hi David, >>>>>>>>> >>>>>>>>> To link the classes, we need to be able to run Java code -- >>>>>>>>> when linking a class X loaded by the app loader, X will be >>>>>>>>> verified. During verification of X, we may need to load >>>>>>>>> additional classes from the app loader, which executes Java >>>>>>>>> code during its class loading operations. >>>>>>>>> >>>>>>>>> We also need to handle all the exit conditions. As far as I >>>>>>>>> can tell, an app can exit the JVM in two ways: >>>>>>>>> >>>>>>>>> (1) Explicitly calling System.exit(). This will call >>>>>>>>> java.lang.Shutdown.exit() which does this: >>>>>>>>> >>>>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/0edc7fd0d7a3/src/java.base/share/classes/java/lang/Shutdown.java#l163 >>>>>>>>> >>>>>>>>> >>>>>>>>> ??????????? beforeHalt(); // native >>>>>>>>> ??????????? runHooks(); >>>>>>>>> ??????????? halt(status); >>>>>>>>> >>>>>>>>> (2) When all non-daemon threads have died (e.g., falling out >>>>>>>>> of the bottom of HelloWorld.main()). There's no explicit call >>>>>>>>> to System.exit(), but the JVM will proactively call >>>>>>>>> java.lang.Shutdown.shutdown() inside >>>>>>>>> JavaThread::invoke_shutdown_hooks() >>>>>>>>> >>>>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/0edc7fd0d7a3/src/hotspot/share/runtime/thread.cpp#l4331 >>>>>>>>> >>>>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/0edc7fd0d7a3/src/java.base/share/classes/java/lang/Shutdown.java#l184 >>>>>>>>> >>>>>>>>> >>>>>>>>> If we want to avoid modifying the Java code, I think we can >>>>>>>>> intercept JVM_BeforeHalt() and >>>>>>>>> JavaThread::invoke_shutdown_hooks(). This way we should be >>>>>>>>> able to handle all the classes (except those that are loaded >>>>>>>>> inside Shutdown.runHooks). >>>>>>>> >>>>>>>> I've implemented the above. No changes to the Java side. >>>>>>>> >>>>>>>> updated webrev: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~ccheung/jdk15/8232081/webrev.01/ >>>>>>> >>>>>>> This looks much better to me. I wasn't sure if you were that >>>>>>> concerned about catching the classes loaded by the Shutdown >>>>>>> hooks, but it is good you are not as it seems problematic to me >>>>>>> to trigger class loading etc after the shutdown hooks have >>>>>>> executed - you may hit unexpected conditions. So this approach >>>>>>> is good. >>>>>>> >>>>>>> A few minor comments: >>>>>>> >>>>>>> src/hotspot/share/memory/metaspaceShared.cpp >>>>>>> src/hotspot/share/memory/metaspaceShared.hpp >>>>>>> >>>>>>> Why the change from TRAPS to "Thread* THREAD"? >>>>>> I forgot why I changed it but I've changed it back and it still >>>>>> works. >>>>> >>>>> Ok. >>>>> >>>>>>> >>>>>>> --- >>>>>>> >>>>>>> src/hotspot/share/oops/instanceKlass.cpp >>>>>>> >>>>>>> I'm not clear how verify_on is used so am unsure how the >>>>>>> additional error state check may affect code unrelated to >>>>>>> dynamic dumping ?? >>>>>> >>>>>> Some of the appcds tests by design have some classes which fail >>>>>> verification. Without the change, the following assert would fail: >>>>>> >>>>>> void vtableEntry::verify(klassVtable* vt, outputStream* st) { >>>>>> ?? Klass* vtklass = vt->klass(); >>>>>> ?? if (vtklass->is_instance_klass() && >>>>>> ????? (InstanceKlass::cast(vtklass)->major_version() >= >>>>>> klassVtable::VTABLE_TRANSITIVE_OVERRIDE_VERSION)) { >>>>>> *assert*(method() != NULL, "must have set method"); >>>>>> ?? } >>>>> >>>>> Okay so you need to exclude for that case but ... >>>>> >>>>>> Here's the call stack during dynamic CDS dump to the above function: >>>>>> >>>>>> vtableEntry::verify(klassVtable *, outputStream *) : void >>>>>> ???? klassVtable::verify(outputStream *, bool) : void >>>>>> ???????? InstanceKlass::verify_on(outputStream *) : void >>>>>> ???????????? Klass::verify() : void >>>>>> ???????????????? ClassLoaderData::verify() : void >>>>>> ???????????????????? ClassLoaderDataGraph::verify() : void >>>>>> ???????????????????????? Universe::verify(enum VerifyOption, >>>>>> const char *) : void >>>>>> ???????????????????????????? Universe::verify(const char *) : void >>>>>> DynamicArchiveBuilder::verify_universe(const char *) : void >>>>>> DynamicArchiveBuilder::doit() : void (2 matches) >>>>>> VM_PopulateDynamicDumpSharedSpace::doit() : void >>>>>> VM_Operation::evaluate() : void >>>>>> >>>>>> The is_linked() function is implemented as: >>>>>> >>>>>> ???? bool is_linked() const?????????????????? { return >>>>>> _init_state >= linked; } >>>>>> >>>>>> which includes 'initialization_error'. >>>>> >>>>> ... AFAICS Universe::verify is call from a number of places, all >>>>> of which would reach this modified code. As that existing code has >>>>> included classes that are in the error state then it seems to me >>>>> that unless you can prove otherwise you need to keep the existing >>>>> code as-is for the non-CDS dump case. >>>>> >>>>>> A previous version of the changeset (including the change in >>>>>> question in instanceKlass.cpp) passed tier1 - 4 testing. Let me >>>>>> know if I should run other tests to make sure the change is good. >>>>> >>>>> There may well not be a test that covers this. >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>>>> >>>>>>> Also can I suggest (as you are touching this code) to delete the >>>>>>> ancient comment: >>>>>>> >>>>>>> 3580???? // $$$ This used to be done only for m/s collections. >>>>>>> Doing it >>>>>>> 3581???? // always seemed a valid generalization. (DLD -- 6/00) >>>>>> >>>>>> I'm glad that you suggested to remove the above comment. I've no >>>>>> clue what it means either. >>>>>> >>>>>> updated webrev: >>>>>> >>>>>> http://cr.openjdk.java.net/~ccheung/jdk15/8232081/webrev.02/ >>>>>> >>>>>> thanks, >>>>>> Calvin >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>>> I also updated the test to test the shutdown hook and >>>>>>>> System.exit() scenarios. >>>>>>>> >>>>>>>> All appcds tests passed on my linux host. I'll run more tests >>>>>>>> using mach5. >>>>>>>> >>>>>>>> thanks, >>>>>>>> >>>>>>>> Calvin >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> What do you think? >>>>>>>>> >>>>>>>>> - Ioi >>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> David >>>>>>>>>> >>>>>>>>>>> MetaspaceShared::link_and_cleanup_shared_classes() has been >>>>>>>>>>> modified to handle both static and dynamic CDS dump. For >>>>>>>>>>> dynamic CDS dump, only classes loaded by the builtin class >>>>>>>>>>> loaders will be linked. Local performance testing using >>>>>>>>>>> javac on HelloWorld.java shows an improvement of >5%. >>>>>>>>>>> >>>>>>>>>>> Passed tier1 - 4 tests. >>>>>>>>>>> >>>>>>>>>>> thanks, >>>>>>>>>>> >>>>>>>>>>> Calvin >>>>>>>>>>> >>>>>>>>> >>> > From aph at redhat.com Fri Mar 6 17:11:20 2020 From: aph at redhat.com (Andrew Haley) Date: Fri, 6 Mar 2020 17:11:20 +0000 Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is inconsistent In-Reply-To: <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com> References: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com> <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com> Message-ID: <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com> On 3/6/20 2:48 PM, Andrew Haley wrote: >> The predicate for Pow2 and NotPow2 will both be false because of this >> change. Is that a problem? > Probably. I'll do some more digging. The pattern genertates a bitset instruction. It'll generate slightly less efficient code after my change. Maybe. It's pretty damm hard to persuade C2 actually to use this pattern, but this works: @Benchmark public void test_btsL_mem_imm(BenchmarkState state) { for (int i = 0; i < 1000; i += 3) { state.array[i] |= 0x4000_0000_0000_0000l; state.array[i+1] |= 0x8000_0000_0000_0000l; state.array[i+2] |= 0x1000_0000_0000_0000l; } } These three |= ops turn into: 0x00007fe65cc58f40: bts QWORD PTR [r8+r10*8+0x10],0x3e;*lastore {reexecute=0 rethrow=0 return_oop=0} 0x00007fe65cc58f47: or QWORD PTR [r8+r10*8+0x18],rdi;*lastore {reexecute=0 rethrow=0 return_oop=0} 0x00007fe65cc58f4c: bts QWORD PTR [r8+r10*8+0x20],0x3c;*lastore {reexecute=0 rethrow=0 return_oop=0} There is a very slight pessimization in that RDI is now occupied, so the register pressure is slightly increased, and there's the cost of a 64-bit load immediate operation. We can get back to the status quo with this patch: --- a/src/hotspot/cpu/x86/x86_64.ad Thu Mar 05 17:56:08 2020 +0000 +++ b/src/hotspot/cpu/x86/x86_64.ad Fri Mar 06 17:06:20 2020 +0000 @@ -3120,7 +3120,7 @@ operand immL_Pow2() %{ - predicate(is_power_of_2(n->get_long())); + predicate(is_power_of_2((julong)n->get_long())); match(ConL); op_cost(15); @@ -3130,7 +3130,7 @@ operand immL_NotPow2() %{ - predicate(is_power_of_2(~n->get_long())); + predicate(is_power_of_2((julong)~n->get_long())); match(ConL); op_cost(15); 0x00007fc688bf8010: bts QWORD PTR [r10+r8*8+0x10],0x3e;*lastore {reexecute=0 rethrow=0 return_oop=0} 0x00007fc688bf8017: bts QWORD PTR [r10+r8*8+0x18],0x3f;*lastore {reexecute=0 rethrow=0 return_oop=0} 0x00007fc688bf801e: bts QWORD PTR [r10+r8*8+0x20],0x3c;*lastore {reexecute=0 rethrow=0 return_oop=0} There will be more examples of this, stuff that just works "by accident". If we want the previous weird behaviour we could explictly special-case that behaviour and test it. I'd rather not... -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From john.r.rose at oracle.com Fri Mar 6 19:48:58 2020 From: john.r.rose at oracle.com (John Rose) Date: Fri, 6 Mar 2020 11:48:58 -0800 Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is inconsistent In-Reply-To: <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com> References: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com> <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com> <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com> Message-ID: On Mar 6, 2020, at 9:11 AM, Andrew Haley wrote: > > + predicate(is_power_of_2((julong)n->get_long())); To me this is an easy ?yes?. The hardware instruction in question uses the unsigned convention. The harder part is (as you say) figuring out where we were implicitly relying on unsigned conventions and behavior will change. But I think we?ve had plenty of eyeballs on this, and it?s easy to fix more stuff like this if we find it later, right? ? John From yumin.qi at oracle.com Fri Mar 6 22:26:00 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Fri, 6 Mar 2020 14:26:00 -0800 Subject: RFR(XS): 8240691: serviceability/sa/ClhsdbCDSJstackPrintAll.java and serviceability/sa/ClhsdbCDSCore.java should be excluded with ZGC Message-ID: <86878457-3386-ea0e-f23e-7ad1bdff64a6@oracle.com> HI, ? Can I have your reviews on ? Bug: https://bugs.openjdk.java.net/browse/JDK-8240691 ? webrev: http://cr.openjdk.java.net/~minqi/8240691/webrev/ ? Summary: With JDK-8232069 (https://bugs.openjdk.java.net/browse/JDK-8232069), CDS works with UseCompressedOop and UseCompressedClassPointers off. The code for detecting CDS will report false with ZGC since ZGC turned off those two flags. The detection code will be changed in JDK-8240563(https://bugs.openjdk.java.net/browse/JDK-8240563), this will cause those two tests fail. ?? Tests: jtreg to show the tests not selected with ZGC. ?? Thanks ?? Yumin From chris.plummer at oracle.com Fri Mar 6 23:01:28 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Fri, 6 Mar 2020 15:01:28 -0800 Subject: RFR(XS): 8240691: serviceability/sa/ClhsdbCDSJstackPrintAll.java and serviceability/sa/ClhsdbCDSCore.java should be excluded with ZGC In-Reply-To: <86878457-3386-ea0e-f23e-7ad1bdff64a6@oracle.com> References: <86878457-3386-ea0e-f23e-7ad1bdff64a6@oracle.com> Message-ID: <038c6af5-f7e4-cf93-9c22-729f1db03e05@oracle.com> Hi Yumin, Why do these tests not work with ZGC? Is there a bug filed for that? thanks, Chris On 3/6/20 2:26 PM, Yumin Qi wrote: > HI, > > ? Can I have your reviews on > > ? Bug: https://bugs.openjdk.java.net/browse/JDK-8240691 > > ? webrev: http://cr.openjdk.java.net/~minqi/8240691/webrev/ > > ? Summary: With JDK-8232069 > (https://bugs.openjdk.java.net/browse/JDK-8232069), CDS works with > UseCompressedOop and UseCompressedClassPointers off. The code for > detecting CDS will report false with ZGC since ZGC turned off those > two flags. The detection code will be changed in > JDK-8240563(https://bugs.openjdk.java.net/browse/JDK-8240563), this > will cause those two tests fail. > > > ?? Tests: jtreg to show the tests not selected with ZGC. > > ?? Thanks > > ?? Yumin > From yumin.qi at oracle.com Sat Mar 7 00:28:04 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Fri, 6 Mar 2020 16:28:04 -0800 Subject: RFR(XS): 8240691: serviceability/sa/ClhsdbCDSJstackPrintAll.java and serviceability/sa/ClhsdbCDSCore.java should be excluded with ZGC In-Reply-To: <038c6af5-f7e4-cf93-9c22-729f1db03e05@oracle.com> References: <86878457-3386-ea0e-f23e-7ad1bdff64a6@oracle.com> <038c6af5-f7e4-cf93-9c22-729f1db03e05@oracle.com> Message-ID: <8c057888-5a75-c9c7-cb55-9352d07b3013@oracle.com> HI, Chris ?After discussed with Ioi, we found there are bugs in the test lib and this test case: ?1) The test lib, LingeredApp did not pass getTestJavaOp0ts()? to the children process if vmArguments is already specified. http://hg.openjdk.java.net/jdk/jdk/file/3465ed78d670/test/lib/jdk/test/lib/apps/LingeredApp.java#l307 ?2) The check in the test code: (ClhsdbCDSJstackPrintAll.java) should be changed: 72 if (!useSharedSpacesOutput.contains("true")) { 73 // CDS archive is not mapped. Skip the rest of the test. 74 LingeredApp.stopApp(theApp); 75 throw new SkippedException("The CDS archive is not mapped"); 76 } 77 ??? if (useSharedSpacesOutput.contains("UseSharedSpaces = false")) { ? This will cause the test skipped. I will file a bug for 1) and will post webrev for this bug next. ? The java opts not passed into children will cause the parameters used in dump/run inconsistent. Thanks Yumin 3/6/20 3:01 PM, Chris Plummer wrote: > Hi Yumin, > > Why do these tests not work with ZGC? Is there a bug filed for that? > > thanks, > > Chris > > On 3/6/20 2:26 PM, Yumin Qi wrote: >> HI, >> >> ? Can I have your reviews on >> >> ? Bug: https://bugs.openjdk.java.net/browse/JDK-8240691 >> >> ? webrev: http://cr.openjdk.java.net/~minqi/8240691/webrev/ >> >> ? Summary: With JDK-8232069 >> (https://bugs.openjdk.java.net/browse/JDK-8232069), CDS works with >> UseCompressedOop and UseCompressedClassPointers off. The code for >> detecting CDS will report false with ZGC since ZGC turned off those >> two flags. The detection code will be changed in >> JDK-8240563(https://bugs.openjdk.java.net/browse/JDK-8240563), this >> will cause those two tests fail. >> >> >> ?? Tests: jtreg to show the tests not selected with ZGC. >> >> ?? Thanks >> >> ?? Yumin >> > > From jiefu at tencent.com Sat Mar 7 00:40:25 2020 From: jiefu at tencent.com (=?utf-8?B?amllZnUo5YKF5p2wKQ==?=) Date: Sat, 7 Mar 2020 00:40:25 +0000 Subject: RFR: 8240695: Build is broken when cds is disabled after JDK-8232081 Message-ID: <0655BD96-79CA-4F81-AD61-B99FA154052E@tencent.com> Hi all, JBS: https://bugs.openjdk.java.net/browse/JDK-8240695 Build is broken when cds is disabled. It might be fixed by ------------------------------------ diff -r ba031902229b src/hotspot/share/memory/metaspaceShared.hpp --- a/src/hotspot/share/memory/metaspaceShared.hpp Fri Mar 06 18:42:13 2020 -0500 +++ b/src/hotspot/share/memory/metaspaceShared.hpp Sat Mar 07 08:25:01 2020 +0800 @@ -296,7 +296,7 @@ } static bool try_link_class(InstanceKlass* ik, TRAPS); - static void link_and_cleanup_shared_classes(TRAPS); + static void link_and_cleanup_shared_classes(TRAPS) NOT_CDS_RETURN; #if INCLUDE_CDS static ReservedSpace reserve_shared_space(size_t size, char* requested_address = NULL); ------------------------------------ Could you please review it and give me some advice? Thanks a lot. Best regards, Jie From ioi.lam at oracle.com Sat Mar 7 04:27:34 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 6 Mar 2020 20:27:34 -0800 Subject: RFR: 8240695: Build is broken when cds is disabled after JDK-8232081 In-Reply-To: <0655BD96-79CA-4F81-AD61-B99FA154052E@tencent.com> References: <0655BD96-79CA-4F81-AD61-B99FA154052E@tencent.com> Message-ID: Hi Jie, this looks like the correct fix. Thanks - Ioi On 3/6/20 4:40 PM, jiefu(??) wrote: > Hi all, > > > > JBS: https://bugs.openjdk.java.net/browse/JDK-8240695 > > > > Build is broken when cds is disabled. > > > > It might be fixed by > > ------------------------------------ > > diff -r ba031902229b src/hotspot/share/memory/metaspaceShared.hpp > > --- a/src/hotspot/share/memory/metaspaceShared.hpp Fri Mar 06 18:42:13 2020 -0500 > > +++ b/src/hotspot/share/memory/metaspaceShared.hpp Sat Mar 07 08:25:01 2020 +0800 > > @@ -296,7 +296,7 @@ > > } > > > > static bool try_link_class(InstanceKlass* ik, TRAPS); > > - static void link_and_cleanup_shared_classes(TRAPS); > > + static void link_and_cleanup_shared_classes(TRAPS) NOT_CDS_RETURN; > > > > #if INCLUDE_CDS > > static ReservedSpace reserve_shared_space(size_t size, char* requested_address = NULL); > > ------------------------------------ > > > > Could you please review it and give me some advice? > > > > Thanks a lot. > > Best regards, > > Jie > From jiefu at tencent.com Sat Mar 7 06:48:21 2020 From: jiefu at tencent.com (=?utf-8?B?amllZnUo5YKF5p2wKQ==?=) Date: Sat, 7 Mar 2020 06:48:21 +0000 Subject: RFR: 8240695: Build is broken when cds is disabled after JDK-8232081 Message-ID: Thanks Ioi for your review. Pushed. Best regards, Jie ?On 2020/3/7, 12:29 PM, "hotspot-runtime-dev on behalf of Ioi Lam" wrote: Hi Jie, this looks like the correct fix. Thanks - Ioi On 3/6/20 4:40 PM, jiefu(??) wrote: > Hi all, > > > > JBS: https://bugs.openjdk.java.net/browse/JDK-8240695 > > > > Build is broken when cds is disabled. > > > > It might be fixed by > > ------------------------------------ > > diff -r ba031902229b src/hotspot/share/memory/metaspaceShared.hpp > > --- a/src/hotspot/share/memory/metaspaceShared.hpp Fri Mar 06 18:42:13 2020 -0500 > > +++ b/src/hotspot/share/memory/metaspaceShared.hpp Sat Mar 07 08:25:01 2020 +0800 > > @@ -296,7 +296,7 @@ > > } > > > > static bool try_link_class(InstanceKlass* ik, TRAPS); > > - static void link_and_cleanup_shared_classes(TRAPS); > > + static void link_and_cleanup_shared_classes(TRAPS) NOT_CDS_RETURN; > > > > #if INCLUDE_CDS > > static ReservedSpace reserve_shared_space(size_t size, char* requested_address = NULL); > > ------------------------------------ > > > > Could you please review it and give me some advice? > > > > Thanks a lot. > > Best regards, > > Jie > From serguei.spitsyn at oracle.com Sat Mar 7 07:53:21 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 6 Mar 2020 23:53:21 -0800 Subject: RFR(S): 8240295: hs_err elapsed time in seconds is not accurate enough In-Reply-To: <20045e23-c736-5289-866e-9df5a09101a8@oracle.com> References: <4a3d7c8e-df9e-51de-77e4-775be1480392@oracle.com> <92ada19d-7200-6bb7-b174-9da8a166aea7@oracle.com> <053864a8-617f-fcac-fc26-220d161e3e55@oracle.com> <20045e23-c736-5289-866e-9df5a09101a8@oracle.com> Message-ID: <80fc495b-5169-2ec8-559c-8ba8c9f4939d@oracle.com> Hi Kevin, This looks okay to me as well. Thanks, Serguei On 3/5/20 02:38, David Holmes wrote: > Thanks Kevin. I think this is the less risky change and achieves the > goal. > > David > > On 5/03/2020 8:00 pm, Kevin Walls wrote: >> Thanks - >> >> I had tried some ideas in the simple fashion, and we can use %06d >> formatting.... OK maybe such formatting is not as "bad" as %f... >> >> (glibc parses the int width specified without allocation.? We provide >> the output buffer, I don't think we will cause? vfprintf code to >> alloca or malloc.) >> >> I can offer a second version below that uses %d only.? Testing >> alongside %f in the same line, it retains the same value and >> position, e.g. >> >> Time: Thu Mar? 5 08:57:50 2020 UTC elapsed time: f: 2.001065 int: >> 2.001065 (raw int: 1065) seconds (0d 0h 0m 2s) >> >> Output example from the hg diff below (not from the same run): >> >> Time: Thu Mar? 5 09:28:01 2020 UTC elapsed time: 2.000611 seconds (0d >> 0h 0m 2s) >> >> >> Thanks! >> Kevin >> >> >> $ hg diff >> diff --git a/src/hotspot/share/runtime/os.cpp >> b/src/hotspot/share/runtime/os.cpp >> --- a/src/hotspot/share/runtime/os.cpp >> +++ b/src/hotspot/share/runtime/os.cpp >> @@ -1016,10 +1016,9 @@ >> ??? } >> >> ??? double t = os::elapsedTime(); >> -? // NOTE: It tends to crash after a SEGV if we want to >> printf("%f",...) in >> -? //?????? Linux. Must be a bug in glibc ? Workaround is to round >> "t" to int >> -? //?????? before printf. We lost some precision, but who cares? >> +? // NOTE: a crash using printf("%f",...) on Linux was historically >> noted here. >> ??? int eltime = (int)t;? // elapsed time in seconds >> +? int eltimeFraction = (int) ((t - eltime) * 1000000); >> >> ??? // print elapsed time in a human-readable format: >> ??? int eldays = eltime / secs_per_day; >> @@ -1029,7 +1028,7 @@ >> ??? int elmins = (eltime - day_secs - hour_secs) / secs_per_min; >> ??? int minute_secs = elmins * secs_per_min; >> ??? int elsecs = (eltime - day_secs - hour_secs - minute_secs); >> -? st->print_cr(" elapsed time: %d seconds (%dd %dh %dm %ds)", >> eltime, eldays, elhours, elmins, elsecs); >> +? st->print_cr(" elapsed time: %d.%06d seconds (%dd %dh %dm %ds)", >> eltime, eltimeFraction, eldays, elhours, elmins, elsecs); >> ??} >> >> >> >> On 05/03/2020 00:57, David Holmes wrote: >>> On 4/03/2020 8:44 am, Kevin Walls wrote: >>>> >>>> Thanks David - >>>> >>>> Yes there are situations where hs_err fails, and few people are >>>> sadder than me >>>> when that happens 8-) , so I was thinking about how scared to be by >>>> the comment. >>>> >>>> With the safety net of the error handler for the steps of the >>>> hs_err file >>>> (which works well, we see it invoked frequently), it looks >>>> reasonable to use >>>> %f as we might do other slightly questionable things for a signal >>>> handler. >>>> >>>> Corrupting locale information or floating point state might >>>> possibly cause >>>> problems, but if I cause a fake crash in print_date_and_time the error >>>> handler recovers and the report continues. >>> >>> That is good to know. >>> >>>> Thinking about printing with two ints, seconds and fractions: >>>> I don't see anything already that returns such a time in two >>>> components in the >>>> JVM, so we might implement a new form of os::javaTimeNanos() or >>>> similar that >>>> returns the two parts, and do that on each platform. >>> >>> I was thinking of something simple/crude ... >>> >>>> I didn't yet come up with anything to do in os::print_date_and_time() >>>> which will take the fractional part of the double, and print just >>>> the fraction as an int, without using any library / %f facilities. >>> >>> ... just using e.g. (untested) >>> >>> double t = os::elapsedTime(); >>> int secs =? (int) t; >>> int micros =? (int)((t - secs) * 100000); >>> printf("%d.%d", secs, micros); >>> >>> with appropriate width specifiers to get the formatting right. >>> >>> Cheers, >>> David >>> >>>> >>>> If you're still concerned I could revisit these or some other idea. >>>> >>>> Genuine laugh out loud moment for me, I backported the elapsed time >>>> logging from >>>> 6u4 to 5u19 (https://bugs.openjdk.java.net/browse/JDK-6447157) (2007). >>>> (I said before jdk5 was created, I should have said before it was >>>> in mercurial.) >>>> >>>> Thanks >>>> Kevin >>>> >>>> >>>> On 03/03/2020 01:11, David Holmes wrote: >>>>> Hi Kevin, >>>>> >>>>> On 2/03/2020 8:48 pm, Kevin Walls wrote: >>>>>> Oops, and with the bug ID in the title and JBS link: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8240295 >>>>>> >>>>>> >>>>>> On 02/03/2020 10:47, Kevin Walls wrote: >>>>>>> Hi, >>>>>>> >>>>>>> (s11y and runtime opinions both relevant) >>>>>>> >>>>>>> A few times in the last month I've really wanted to compare the >>>>>>> Events logged in the hs_err file, and the time of the JVM's crash. >>>>>>> >>>>>>> "elapsed time" in hs_err is only accurate to one second, and has >>>>>>> been since before jdk5 was created. >>>>>>> >>>>>>> The diff below changes the format string and uses the >>>>>>> non-rounded time value (I don't see a need to change the other >>>>>>> integer arithmetic here), and we can enjoy hs_errs with detail >>>>>>> like: >>>>>>> >>>>>>> ... >>>>>>> Time: Mon Mar? 2 09:57:13 2020 UTC elapsed time: 5.490135 >>>>>>> seconds (0d 0h 0m 5s) >>>>>>> ... >>>>>>> >>>>>>> Thanks >>>>>>> Kevin >>>>>>> >>>>>>> >>>>>>> /jdk/open$ hg diff >>>>>>> diff --git a/src/hotspot/share/runtime/os.cpp >>>>>>> b/src/hotspot/share/runtime/os.cpp >>>>>>> --- a/src/hotspot/share/runtime/os.cpp >>>>>>> +++ b/src/hotspot/share/runtime/os.cpp >>>>>>> @@ -1016,9 +1016,8 @@ >>>>>>> ?? } >>>>>>> >>>>>>> ?? double t = os::elapsedTime(); >>>>>>> -? // NOTE: It tends to crash after a SEGV if we want to >>>>>>> printf("%f",...) in >>>>>>> -? //?????? Linux. Must be a bug in glibc ? Workaround is to >>>>>>> round "t" to int >>>>>>> -? //?????? before printf. We lost some precision, but who cares? >>>>>>> +? // NOTE: a crash using printf("%f",...) on Linux was >>>>>>> historically noted here >>>>>>> +? //?????? (before the jdk5 repo was created). >>>>> >>>>> Just because it is old doesn't mean it no longer applies. printf >>>>> is not async-signal-safe - we know that but we try to use it >>>>> anyway. Maybe %f is even less async-signal-safe? >>>>> >>>>> This may get through testing okay but cause problems with real >>>>> crashes in the field. >>>>> >>>>> What about breaking the time up into two ints: seconds and nanos? >>>>> >>>>> Cheers, >>>>> David >>>>> ----- >>>>> >>>>>>> ?? int eltime = (int)t;? // elapsed time in seconds >>>>>>> >>>>>>> ?? // print elapsed time in a human-readable format: >>>>>>> @@ -1029,7 +1028,7 @@ >>>>>>> ?? int elmins = (eltime - day_secs - hour_secs) / secs_per_min; >>>>>>> ?? int minute_secs = elmins * secs_per_min; >>>>>>> ?? int elsecs = (eltime - day_secs - hour_secs - minute_secs); >>>>>>> -? st->print_cr(" elapsed time: %d seconds (%dd %dh %dm %ds)", >>>>>>> eltime, eldays, elhours, elmins, elsecs); >>>>>>> +? st->print_cr(" elapsed time: %f seconds (%dd %dh %dm %ds)", >>>>>>> t, eldays, elhours, elmins, elsecs); >>>>>>> ?} >>>>>>> >>>>>>> From kevin.walls at oracle.com Sat Mar 7 07:57:48 2020 From: kevin.walls at oracle.com (Kevin Walls) Date: Fri, 6 Mar 2020 23:57:48 -0800 (PST) Subject: RFR(S): 8240295: hs_err elapsed time in seconds is not accurate enough In-Reply-To: <80fc495b-5169-2ec8-559c-8ba8c9f4939d@oracle.com> References: <4a3d7c8e-df9e-51de-77e4-775be1480392@oracle.com> <92ada19d-7200-6bb7-b174-9da8a166aea7@oracle.com> <053864a8-617f-fcac-fc26-220d161e3e55@oracle.com> <20045e23-c736-5289-866e-9df5a09101a8@oracle.com> <80fc495b-5169-2ec8-559c-8ba8c9f4939d@oracle.com> Message-ID: <67d1cbc7-8102-09ac-7322-9f545089a45c@oracle.com> Great, thanks! On 07/03/2020 07:53, serguei.spitsyn at oracle.com wrote: > Hi Kevin, > > This looks okay to me as well. > > Thanks, > Serguei From aph at redhat.com Sat Mar 7 10:54:32 2020 From: aph at redhat.com (Andrew Haley) Date: Sat, 7 Mar 2020 10:54:32 +0000 Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is inconsistent In-Reply-To: References: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com> <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com> <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com> Message-ID: On 3/6/20 7:48 PM, John Rose wrote: > On Mar 6, 2020, at 9:11 AM, Andrew Haley wrote: >> >> + predicate(is_power_of_2((julong)n->get_long())); > > To me this is an easy ?yes?. The hardware instruction in question > uses the unsigned convention. > > The harder part is (as you say) figuring out where we were implicitly > relying on unsigned conventions and behavior will change. But I > think we?ve had plenty of eyeballs on this, and it?s easy to fix more > stuff like this if we find it later, right? I think we'll have to do that. For the record: This is a Hard Problem, for a reason that is not obvious. In C2, you often have patterns like this: match(If cmp (CmpI (AndI op1 op2) op3)); predicate(is_power_of_2(n->in(2)->in(1)->in(2)->get_int())); ... ins_encode %{ ... int bit = exact_log2($op2$$constant); Very unfortunately, (n->in(2)->in(1)->in(2)->get_int()) is a jint, but $op2$$constant is an intptr_t. This is problematic if we're running on a 64-bit machine because intptr_t is signed long. At first I wanted to define exact_log2() and is_power_of_2() in a way that treats all arguments as unsigned -- I wrote a patch which does that -- but it doesn't help. Because of the above, even with a jint operand representing 0x8000_0000, $op2$$constant returns 0xffff_ffff_8000_0000. This is a historic mistake in the design of C2, IMO, but very hard to fix today. So, AFAICS the only way to fix this is to treat all negative integer values, whatever their size, as non-powers of 2. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From ioi.lam at oracle.com Sun Mar 8 07:17:13 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Sat, 7 Mar 2020 23:17:13 -0800 Subject: RFR(T) 8240613 InstanceKlass::set_init_state failed with assert(good_state || state == allocated) Message-ID: <761d8b0e-9ca9-f43a-1177-2074b34691b3@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8240613 http://cr.openjdk.java.net/~iklam/jdk15/8240613-set-init-state-graal.v01/ In JDK-8240244 (Avoid calling resolve_super_or_fail in SystemDictionary::load_shared_class) I added code to check if a super type of a CDS shared class is already loaded. Apparently this check, which takes the SystemDictionary_lock, causes a race condition during class loading, and leads to the assert. So far the assert has happened only when graal is used as the JIT, but I think that's just because when graal starts up, it initializes many system classes in parallel to the main app, which increases the likelihood of the race condition. I am not backing out JDK-8240244 entirely, because new code has been pushed which depends on some of the cleanups done in JDK-8240244. Instead, I am removing the most suspicious part of JDK-8240244. The assert would happen intermittently, but at least once in a couple of hs-tier3-graal.js I am running hs-tier3-graal.js six times and the assert no longer happens, so I think the fix is good. Thanks - Ioi From daniel.daugherty at oracle.com Sun Mar 8 13:04:11 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Sun, 8 Mar 2020 09:04:11 -0400 Subject: RFR(T) 8240613 InstanceKlass::set_init_state failed with assert(good_state || state == allocated) In-Reply-To: <761d8b0e-9ca9-f43a-1177-2074b34691b3@oracle.com> References: <761d8b0e-9ca9-f43a-1177-2074b34691b3@oracle.com> Message-ID: On 3/8/20 3:17 AM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8240613 > http://cr.openjdk.java.net/~iklam/jdk15/8240613-set-init-state-graal.v01/ src/hotspot/share/classfile/systemDictionary.cpp ??? Looks like your intended partial backout of JDK-8240244 is correct. Please do a Tier1 sanity check run before you push. Thumbs up. Dan > > In JDK-8240244 (Avoid calling resolve_super_or_fail in > SystemDictionary::load_shared_class) I added code to check if a super > type of a CDS shared class is already loaded. Apparently this check, > which takes the SystemDictionary_lock, causes a race condition during > class loading, and leads to the assert. > > So far the assert has happened only when graal is used as the JIT, but > I think that's just because when graal starts up, it initializes many > system classes in parallel to the main app, which increases the > likelihood of the race condition. > > > I am not backing out JDK-8240244 entirely, because new code has been > pushed which depends on some of the cleanups done in JDK-8240244. > Instead, I am removing the most suspicious part of JDK-8240244. > > The assert would happen intermittently, but at least once in a couple > of hs-tier3-graal.js > > I am running hs-tier3-graal.js six times and the assert no longer > happens, so I think the fix is good. > > Thanks > - Ioi From daniel.daugherty at oracle.com Sun Mar 8 13:06:47 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Sun, 8 Mar 2020 09:06:47 -0400 Subject: RFR(T) 8240613 InstanceKlass::set_init_state failed with assert(good_state || state == allocated) In-Reply-To: References: <761d8b0e-9ca9-f43a-1177-2074b34691b3@oracle.com> Message-ID: <6af4a8f8-b551-5667-232e-1f220bac3052@oracle.com> On 3/8/20 9:04 AM, Daniel D. Daugherty wrote: > On 3/8/20 3:17 AM, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8240613 >> http://cr.openjdk.java.net/~iklam/jdk15/8240613-set-init-state-graal.v01/ >> > > src/hotspot/share/classfile/systemDictionary.cpp > ??? Looks like your intended partial backout of JDK-8240244 is correct. > > Please do a Tier1 sanity check run before you push. > > Thumbs up. And I agree that this is a trivial fix. Dan > > Dan > > >> >> In JDK-8240244 (Avoid calling resolve_super_or_fail in >> SystemDictionary::load_shared_class) I added code to check if a super >> type of a CDS shared class is already loaded. Apparently this check, >> which takes the SystemDictionary_lock, causes a race condition during >> class loading, and leads to the assert. >> >> So far the assert has happened only when graal is used as the JIT, >> but I think that's just because when graal starts up, it initializes >> many system classes in parallel to the main app, which increases the >> likelihood of the race condition. >> >> >> I am not backing out JDK-8240244 entirely, because new code has been >> pushed which depends on some of the cleanups done in JDK-8240244. >> Instead, I am removing the most suspicious part of JDK-8240244. >> >> The assert would happen intermittently, but at least once in a couple >> of hs-tier3-graal.js >> >> I am running hs-tier3-graal.js six times and the assert no longer >> happens, so I think the fix is good. >> >> Thanks >> - Ioi > From claes.redestad at oracle.com Sun Mar 8 14:12:29 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Sun, 8 Mar 2020 15:12:29 +0100 Subject: RFR: 8240669: Devirtualize Relocation::type Message-ID: <5e26fc09-be3e-8009-3248-2fcaa17e1f1f@oracle.com> Hi, devirtualizing Relocation::type() means a throughput increase in places such as Assembler::emit_operand (-15% instructions, fewer branches), due enabling more aggressive inlining. Several of these places are sensitive to startup/warmup. For example paths taken when generating the interpreter, runtime adapters and by C1 improve, and the refactoring here has a measurable win on #instructions, #branches (-0.4% on Hello World) etc, on a range of startup tests. Bug: https://bugs.openjdk.java.net/browse/JDK-8240302 Webrev: http://cr.openjdk.java.net/~redestad/8240669/open.00/ Testing: tier1-5 Thanks! /Claes From richard.reingruber at sap.com Sun Mar 8 21:32:35 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Sun, 8 Mar 2020 21:32:35 +0000 Subject: RFC(S) 8234146: [TESTBUG] compiler/jsr292/ContinuousCallSiteTargetChange.java times out on SPARC Message-ID: Hi, I'd like to get comments on two alternative fixes for Bug: https://bugs.openjdk.java.net/browse/JDK-8234146 The timeouts occur when the system is under load. Then the list of dependent nmethods of the dynamic call site of the PingPongTest gets as long as 1500 nmethods. The list is iterated in DependencyContext::mark_dependent_nmethods() every time the call site target is changed. And this takes longer and longer as the list grows. Note that dependents are removed only when they are made zombie, but this doesn't happen here. The list of dependent nmethods is virtually empty if the system is idle, because then the target is changed faster than the method holding the call site gets compiled. When the jit finishes a compilation it discards the new nmethod, because the call site target was changed already. To reproduce long runs I'd suggest to run ContinuousCallSiteTargetChange$PingPongTest manually taking the command from ContinuousCallSiteTargetChange.jtr of a previous run. Under load [1] the test will take about 10x as long as without load. 30s vs 3s on my linux x86_64 box for the server config. With the fixes below the test will take under load approximately as long as without load. There are 2 possible fixes I'd like to get comments for: Option A: Force the NMethodSweeper using the whitebox api. Webrev A: http://cr.openjdk.java.net/~rrich/webrevs/8234146/A/webrev.0/ Option B: Specialize DependencyContext::mark_dependent_nmethods() for call site target changes, i.e. flag list elements that are marked for deoptimization. Upon the next target change only new dependents are marked for deoptimization. The list iteration is ended as soon as an element is encountered that is already flagged. Webrev B: http://cr.openjdk.java.net/~rrich/webrevs/8234146/B/webrev.0/ Personally I like B better, because I think it is simple and straight forward: if the target changes, then all dependents need to be invalidated. By construction of the list we can stop the invalidation if we find an element that was invalidated before. This specialization is of course less generic, and there might be new types of call site dependencies for which it is illegal, but currently I can't think of any. Curious about your comments... Thanks, Richard. [1] To simulate load you can pin the jvm process to one cpu (e.g. on Linux with "taskset 1 java ...") or wget http://cr.openjdk.java.net/~rrich/webrevs/8234146/CPULoadGenerator.java && java CPULoadGenerator.java N where N is the number of CPUs of your box From ioi.lam at oracle.com Sun Mar 8 22:05:25 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Sun, 8 Mar 2020 15:05:25 -0700 Subject: RFR(T) 8240613 InstanceKlass::set_init_state failed with assert(good_state || state == allocated) In-Reply-To: <6af4a8f8-b551-5667-232e-1f220bac3052@oracle.com> References: <761d8b0e-9ca9-f43a-1177-2074b34691b3@oracle.com> <6af4a8f8-b551-5667-232e-1f220bac3052@oracle.com> Message-ID: <12f4f744-437e-ef9d-07e8-6c3d660a1cbf@oracle.com> Thanks Dan for the review. I ran my changes with hs-tier1/2/3 and there are no failures. I am pushing now. Thanks - Ioi On 3/8/20 6:06 AM, Daniel D. Daugherty wrote: > On 3/8/20 9:04 AM, Daniel D. Daugherty wrote: >> On 3/8/20 3:17 AM, Ioi Lam wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8240613 >>> http://cr.openjdk.java.net/~iklam/jdk15/8240613-set-init-state-graal.v01/ >>> >> >> src/hotspot/share/classfile/systemDictionary.cpp >> ??? Looks like your intended partial backout of JDK-8240244 is correct. >> >> Please do a Tier1 sanity check run before you push. >> >> Thumbs up. > > And I agree that this is a trivial fix. > > Dan > > >> >> Dan >> >> >>> >>> In JDK-8240244 (Avoid calling resolve_super_or_fail in >>> SystemDictionary::load_shared_class) I added code to check if a >>> super type of a CDS shared class is already loaded. Apparently this >>> check, which takes the SystemDictionary_lock, causes a race >>> condition during class loading, and leads to the assert. >>> >>> So far the assert has happened only when graal is used as the JIT, >>> but I think that's just because when graal starts up, it initializes >>> many system classes in parallel to the main app, which increases the >>> likelihood of the race condition. >>> >>> >>> I am not backing out JDK-8240244 entirely, because new code has been >>> pushed which depends on some of the cleanups done in JDK-8240244. >>> Instead, I am removing the most suspicious part of JDK-8240244. >>> >>> The assert would happen intermittently, but at least once in a >>> couple of hs-tier3-graal.js >>> >>> I am running hs-tier3-graal.js six times and the assert no longer >>> happens, so I think the fix is good. >>> >>> Thanks >>> - Ioi >> > From shiyue.xw at alibaba-inc.com Mon Mar 9 03:18:52 2020 From: shiyue.xw at alibaba-inc.com (=?UTF-8?B?5ZCR5LyfKOivhuaciCk=?=) Date: Mon, 09 Mar 2020 11:18:52 +0800 Subject: =?UTF-8?B?5Zue5aSN77yaVk0gY3Jhc2hlZCBhdCBTdHJpbmdUYWJsZSBleHBhbnNpb24=?= In-Reply-To: <524d0407-1ee2-97f6-f0e6-8cc76770043f@oracle.com> References: <40f49c75-93a8-4be3-ba8c-1dd2fbcae2c2.shiyue.xw@alibaba-inc.com> <8184d9b3-a4d1-3b17-c048-e574d744a62b@oracle.com>, <524d0407-1ee2-97f6-f0e6-8cc76770043f@oracle.com> Message-ID: Hi? To avoid the crash, I have a proposal: 1. add a new field "size_t _hash" in class Node. 2. when insert the new Node into ConcurrentHashTable, set the hash in the node. 3. For stringtable: Find an invalid node --> check whether the aux_hash is equal to _hash in the node. --> if no, it means that the value in the node is changed. The node can be considered as an dead node? and take care of the invalid node following the rule for the node whose dead_hash is true. --> if yes, still regard it as fatal error. please review the proposal. If it is feasible, I will propose a patch to solve the problem. Thanks, Wei Xiang ------------------------------------------------------------------ ????coleen.phillimore ?????2020?2?26?(???) 21:04 ????hotspot-runtime-dev ????Re: VM crashed at StringTable expansion On 2/26/20 1:07 AM, David Holmes wrote: > Hi, > > cc'ing core-libs-dev as this seems to me like a case of "don't do > that!". Reflection can be used to "shoot oneself in the foot" and I > think that is the case here. > > That said, if we can make the VM more resilient without penalizing all > the well behaved code, then we should probably do that. I reopened bug https://bugs.openjdk.java.net/browse/JDK-8233525 with this test case. I agree that we should try not to crash. Coleen > > Thanks, > David > > On 26/02/2020 3:43 pm, ??(??) wrote: >> Hi?all, >> >> When we migrate an application from JDK8 to JDK11, we meet the same >> failure with >> ?Test fails with Internal Error (concurrentHashTable.inline.hpp:679), >> pid=15229, tid=15260 >> #fatal error: aux_index does not match even or odd >> indices(https://bugs.openjdk.java.net/browse/JDK-8233525)? >> After investigating the issue, we found the root cause as follow: >> The customer used reflect to modify the object with the result value >> of ?String.intern()?. That broke the assumption of StringTable. >> For StringTable in JDK11, it is implemented by the concurrent hash >> table, and uses the string value as key. >> After reflecting, the Node in StringTable is invalid. >> When the StringTable needs to grow up, the invalid node is found in >> unzip_bucket and reports fatal error as below: >> # >> # A fatal error has been detected by the Java Runtime Environment: >> # >> # Internal Error (concurrentHashTable.inline.hpp:684), pid=110176, >> tid=110200 >> # fatal error: aux_index does not match even or odd indices >> We create a simple case to reproduce the issue: >> >> import java.lang.reflect.Field; >> import java.lang.reflect.Modifier; >> public class StringTableTest { >> public static void main(String[] args) throws Exception { >> Field f = String.class.getDeclaredField("value"); >> f.setAccessible(true); >> f.set("s1".intern(), f.get("s2")); >> for (int i = 0; i < 4_000_000; i++) { >> ("s_" + i).intern(); >> } >> } >> } >> >> To solve the issue, we have two proposals: >> 1. when we find the invalid node in unzip_bucket for StringTable, we >> discard the node and give a warning. >> This doesn?t require modification of any user code, and doesn?t >> change the existing implementation logic.2. forbid the user to modify >> the return value of String.intern(). >> >> Thanks, >> Wei Xiang >> From shiyue.xw at alibaba-inc.com Mon Mar 9 03:51:14 2020 From: shiyue.xw at alibaba-inc.com (=?UTF-8?B?5ZCR5LyfKOivhuaciCk=?=) Date: Mon, 09 Mar 2020 11:51:14 +0800 Subject: =?UTF-8?B?TmVlZCBJbmZvcm1hdGlvbiBmb3IgbWVtb3J5IHNoYXJpbmcgd2l0aCBBcHBDRFM=?= Message-ID: <4c513dcc-ed4e-43ec-a607-764ded395058.shiyue.xw@alibaba-inc.com> Hi, We am trying to take advantage of the characteristics of memory sharing with AppCDS, and we expected that the JVM internal packages and common packages can be shared with AppCDS. In the user scenario, there are some common base packages for different Java processes with "-cp common_path" Process A: -cp common_path:pathA Process B: -cp common_path:pathA:pathB Process C: -cp common_path:pathC I check the code in JDK11, and it uses SharedPathsMiscInfo to do the consistency check for class path. The below usage can work in JDK11 Dump stage: -cp common_path Runtime stage: -cp common_path:new_path But I find that SharedPathsMiscInfo has been removed in the latest JDK code. I need the information: 1. the new flow to do the consistency check. 2. constraint condition about the memory sharing among the different Java processes. Is there any user guide to use the memory sharing feature for AppCDS? Thanks, Wei Xiang From david.holmes at oracle.com Mon Mar 9 04:10:49 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 9 Mar 2020 14:10:49 +1000 Subject: RFR(L) 8153224 Monitor deflation prolong safepoints (CR10/v2.10/13-for-jdk15) In-Reply-To: References: <62729044-8a22-0e20-0eda-04d47c9ea23c@oracle.com> <313e51c8-b672-bb1c-577a-49868f09e6c1@oracle.com> <1fd54b23-bd35-9b8f-e6f3-6000440d8770@oracle.com> <3fb1a2b5-5aad-eab3-09ba-6f64a2242d30@oracle.com> <38e2d441-11c9-8342-37d5-8030dd06f2f4@oracle.com> <2a8976f7-37e0-03b9-3099-e07464e46512@oracle.com> <5681d640-08c8-3433-0f85-3f23eea69e87@oracle.com> Message-ID: Hi Dan, Sorry for the delay in looking at latest updates. No further comments from me at this stage. Thanks, David On 27/02/2020 8:22 am, Daniel D. Daugherty wrote: > Greetings, > > I have made changes to the Async Monitor Deflation code in response to > the CR9/v2.09/12-for-jdk14 code review cycle. Thanks to Robbin and Erik O. > for their comments in this round! > > With the extraction and push of {8235931,8236035,8235795} to JDK15, the > Async Monitor Deflation code is back to "just" async deflation changes! > > I have attached the change list from CR9 to CR10 instead of putting it in > the body of this email. I've also added a link to the CR9-to-CR10-changes > file to the webrevs so it should be easy to find. > > Main bug URL: > > ??? JDK-8153224 Monitor deflation prolong safepoints > ??? https://bugs.openjdk.java.net/browse/JDK-8153224 > > The project is currently baselined on jdk-15+11. > > Here's the full webrev URL for those folks that want to see all of the > current Async Monitor Deflation code in one go (v2.10 full): > > http://cr.openjdk.java.net/~dcubed/8153224-webrev/13-for-jdk15+11.v2.10.full/ > > > Some folks might want to see just what has changed since the last review > cycle so here's a webrev for that (v2.10 inc): > > http://cr.openjdk.java.net/~dcubed/8153224-webrev/13-for-jdk15+11.v2.10.inc/ > > > Since we backed out the HandshakeAfterDeflateIdleMonitors option and the > C2 ref_count changes and updated the copyright years, the "inc" webrev has > a bit more noise in it than usual. Sorry about that! > > The OpenJDK wiki has been updated for this round of changes: > > https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation > > The jdk-15+11 based v2.10 version of the patch has been thru Mach5 > tier[1-7] > testing on Oracle's usual set of platforms. Mach5 tier8 is still running. > I'm running the v2.10 patch through my usual set of stress testing on > Linux-X64 and macOSX. > > I'm planning to do a SPECjbb2015 round on the CR10/v2.20/13-for-jdk15 bits. > > Thanks, in advance, for any questions, comments or suggestions. > > Dan > > > On 2/4/20 9:41 AM, Daniel D. Daugherty wrote: >> Greetings, >> >> This project is no longer targeted to JDK14 so this is NOT an urgent code >> review request. >> >> I've extracted the following three fixes from the Async Monitor Deflation >> project code: >> >> ? ? JDK-8235931 add OM_CACHE_LINE_SIZE and use smaller size on SPARCv9 >> and X64 >> ? ? https://bugs.openjdk.java.net/browse/JDK-8235931 >> >> ? ? JDK-8236035 refactor ObjectMonitor::set_owner() and _owner field >> setting >> ? ? https://bugs.openjdk.java.net/browse/JDK-8236035 >> >> ? ? JDK-8235795 replace monitor list mux{Acquire,Release}(&gListLock) >> with spin locks >> ? ? https://bugs.openjdk.java.net/browse/JDK-8235795 >> >> Each of these has been reviewed separately and will be pushed to JDK15 >> in the near future (possibly by the end of this week). Of course, there >> were improvements during these review cycles and the purpose of this >> e-mail is to provided updated webrevs for this fix >> (CR9/v2.09/12-for-jdk14) >> within the revised context provided by {8235931, 8236035, 8235795}. >> >> Main bug URL: >> >> ??? JDK-8153224 Monitor deflation prolong safepoints >> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >> >> The project is currently baselined on jdk-14+34. >> >> Here's the full webrev URL for those folks that want to see all of the >> current Async Monitor Deflation code along with {8235931, 8236035, >> 8235795} >> in one go (v2.09b full): >> >> http://cr.openjdk.java.net/~dcubed/8153224-webrev/12-for-jdk14.v2.09b.full/ >> >> >> Compare the open.patch file in 12-for-jdk14.v2.09.full and >> 12-for-jdk14.v2.09b.full >> using your favorite file comparison/merge tool to see how Async >> Monitor Deflation >> evolved due to {8235931, 8236035, 8235795}. >> >> Some folks might want to see just the Async Monitor Deflation code on >> top of >> {8235931, 8236035, 8235795} so here's a webrev for that (v2.09b inc): >> >> http://cr.openjdk.java.net/~dcubed/8153224-webrev/12-for-jdk14.v2.09b.inc/ >> >> >> These webrevs have gone thru several Mach5 Tier[1-8] runs along with >> my usual stress testing and SPECjbb2015 testing and there aren't any >> surprises relative to CR9/v2.09/12-for-jdk14. >> >> Thanks, in advance, for any questions, comments or suggestions. >> >> Dan >> >> >> On 12/11/19 3:41 PM, Daniel D. Daugherty wrote: >>> Greetings, >>> >>> I have made changes to the Async Monitor Deflation code in response to >>> the CR8/v2.08/11-for-jdk14 code review cycle. Thanks to David H., Robbin >>> and Erik O. for their comments! >>> >>> This project is no longer targeted to JDK14 so this is NOT an urgent >>> code >>> review request. The primary purpose of this webrev is simply to close >>> the >>> CR8/v2.08/11-for-jdk14 code review loop and to let folks see how I >>> resolved >>> the code review comments from that round. >>> >>> Most of the comments in the CR8/v2.08/11-for-jdk14 code review cycle >>> were >>> on the monitor list changes so I'm going to take a look at extracting >>> those >>> changes into a standalone patch. Switching from >>> Thread::muxAcquire(&gListLock) >>> and Thread::muxRelease(&gListLock) to finer grained internal spin >>> locks needs >>> to be thoroughly reviewed and the best way to do that is separately >>> from the >>> Async Monitor Deflation changes. Thanks to Coleen for suggesting >>> doing this >>> extraction earlier. >>> >>> I have attached the change list from CR8 to CR9 instead of putting it in >>> the body of this email. I've also added a link to the CR8-to-CR9-changes >>> file to the webrevs so it should be easy to find. >>> >>> Main bug URL: >>> >>> ??? JDK-8153224 Monitor deflation prolong safepoints >>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>> >>> The project is currently baselined on jdk-14+26. >>> >>> Here's the full webrev URL for those folks that want to see all of the >>> current Async Monitor Deflation code in one go (v2.09 full): >>> >>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/12-for-jdk14.v2.09.full/ >>> >>> >>> Some folks might want to see just what has changed since the last review >>> cycle so here's a webrev for that (v2.09 inc): >>> >>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/12-for-jdk14.v2.09.inc/ >>> >>> >>> The OpenJDK wiki has NOT yet been updated for this round of changes: >>> >>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>> >>> The jdk-14+26 based v2.09 version of the patch has been thru Mach5 >>> tier[1-7] >>> testing on Oracle's usual set of platforms. Mach5 tier8 is still >>> running. >>> A slightly older version of the v2.09 patch has also been through my >>> usual >>> set of stress testing on Linux-X64 and macOSX with the addition of >>> Robbin's >>> "MoCrazy 1024" test running in parallel on Linux-X64 with the other >>> tests in >>> my lab. The "MoCrazy 1024" has been going for > 5 days and 6700+ >>> iterations >>> without any failures. >>> >>> I'm planning to do a SPECjbb2015 round on the CR9/v2.09/12-for-jdk14 >>> bits. >>> >>> Thanks, in advance, for any questions, comments or suggestions. >>> >>> Dan >>> >>> >>> On 11/4/19 4:03 PM, Daniel D. Daugherty wrote: >>>> Greetings, >>>> >>>> I have made changes to the Async Monitor Deflation code in response to >>>> the CR7/v2.07/10-for-jdk14 code review cycle. Thanks to David H., >>>> Robbin >>>> and Erik O. for their comments! >>>> >>>> JDK14 Rampdown phase one is coming on Dec. 12, 2019 and the Async >>>> Monitor >>>> Deflation project needs to push before Nov. 12, 2019 in order to allow >>>> for sufficient bake time for such a big change. Nov. 12 is _next_ >>>> Tuesday >>>> so we have 8 days from today to finish this code review cycle and push >>>> this code for JDK14. >>>> >>>> Carsten and Roman! Time for you guys to chime in again on the code >>>> reviews. >>>> >>>> I have attached the change list from CR7 to CR8 instead of putting >>>> it in >>>> the body of this email. I've also added a link to the >>>> CR7-to-CR8-changes >>>> file to the webrevs so it should be easy to find. >>>> >>>> Main bug URL: >>>> >>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>>> >>>> The project is currently baselined on jdk-14+21. >>>> >>>> Here's the full webrev URL for those folks that want to see all of the >>>> current Async Monitor Deflation code in one go (v2.08 full): >>>> >>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/11-for-jdk14.v2.08.full >>>> >>>> >>>> Some folks might want to see just what has changed since the last >>>> review >>>> cycle so here's a webrev for that (v2.08 inc): >>>> >>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/11-for-jdk14.v2.08.inc/ >>>> >>>> >>>> The OpenJDK wiki did not need any changes for this round: >>>> >>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>> >>>> The jdk-14+21 based v2.08 version of the patch has been thru Mach5 >>>> tier[1-8] >>>> testing on Oracle's usual set of platforms. It has also been through >>>> my usual >>>> set of stress testing on Linux-X64, macOSX and Solaris-X64 with the >>>> addition >>>> of Robbin's "MoCrazy 1024" test running in parallel with the other >>>> tests in >>>> my lab. Some testing is still running, but so far there are no new >>>> regressions. >>>> >>>> I have not yet done a SPECjbb2015 round on the >>>> CR8/v2.08/11-for-jdk14 bits. >>>> >>>> Thanks, in advance, for any questions, comments or suggestions. >>>> >>>> Dan >>>> >>>> >>>> On 10/17/19 5:50 PM, Daniel D. Daugherty wrote: >>>>> Greetings, >>>>> >>>>> The Async Monitor Deflation project is reaching the end game. I >>>>> have no >>>>> changes planned for the project at this time so all that is left is >>>>> code >>>>> review and any changes that results from those reviews. >>>>> >>>>> Carsten and Roman! Time for you guys to chime in again on the code >>>>> reviews. >>>>> >>>>> I have attached the list of fixes from CR6 to CR7 instead of >>>>> putting it >>>>> in the main body of this email. >>>>> >>>>> Main bug URL: >>>>> >>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>> >>>>> The project is currently baselined on jdk-14+19. >>>>> >>>>> Here's the full webrev URL for those folks that want to see all of the >>>>> current Async Monitor Deflation code in one go (v2.07 full): >>>>> >>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/10-for-jdk14.v2.07.full >>>>> >>>>> >>>>> Some folks might want to see just what has changed since the last >>>>> review >>>>> cycle so here's a webrev for that (v2.07 inc): >>>>> >>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/10-for-jdk14.v2.07.inc/ >>>>> >>>>> >>>>> The OpenJDK wiki has been updated to match the >>>>> CR7/v2.07/10-for-jdk14 changes: >>>>> >>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>> >>>>> The jdk-14+18 based v2.07 version of the patch has been thru Mach5 >>>>> tier[1-8] >>>>> testing on Oracle's usual set of platforms. It has also been >>>>> through my usual >>>>> set of stress testing on Linux-X64, macOSX and Solaris-X64 with the >>>>> addition >>>>> of Robbin's "MoCrazy 1024" test running in parallel with the other >>>>> tests in >>>>> my lab. >>>>> >>>>> The jdk-14+19 based v2.07 version of the patch has been thru Mach5 >>>>> tier[1-3] >>>>> test on Oracle's usual set of platforms. Mach5 tier[4-8] are in >>>>> process. >>>>> >>>>> I did another round of SPECjbb2015 testing in Oracle's Aurora >>>>> Performance lab >>>>> using using their tuned SPECjbb2015 Linux-X64 G1 configs: >>>>> >>>>> ??? - "base" is jdk-14+18 >>>>> ??? - "v2.07" is the latest version and includes C2 >>>>> inc_om_ref_count() support >>>>> ????? on LP64 X64 and the new HandshakeAfterDeflateIdleMonitors option >>>>> ? ? - "off" is with -XX:-AsyncDeflateIdleMonitors specified >>>>> ? ? - "handshake" is with -XX:+HandshakeAfterDeflateIdleMonitors >>>>> specified >>>>> >>>>> ???????? hbIR?????????? hbIR >>>>> ??? (max attempted)? (settled)? max-jOPS? critical-jOPS runtime >>>>> ??? ---------------? ---------? --------? ------------- ------- >>>>> ?????????? 34282.00?? 30635.90? 28831.30?????? 20969.20 3841.30 base >>>>> ?????????? 34282.00?? 30973.00? 29345.80?????? 21025.20 3964.10 v2.07 >>>>> ?????????? 34282.00?? 31105.60? 29174.30?????? 21074.00 3931.30 >>>>> v2.07_handshake >>>>> ?????????? 34282.00?? 30789.70? 27151.60?????? 19839.10 3850.20 >>>>> v2.07_off >>>>> >>>>> ??? - The Aurora Perf comparison tool reports: >>>>> >>>>> ??????? Comparison????????????? max-jOPS critical-jOPS >>>>> ??????? ----------------------? -------------------- >>>>> -------------------- >>>>> ??????? base vs 2.07??????????? +1.78% (s, p=0.000)?? +0.27% (ns, >>>>> p=0.790) >>>>> ??????? base vs 2.07_handshake? +1.19% (s, p=0.007)?? +0.58% (ns, >>>>> p=0.536) >>>>> ??????? base vs 2.07_off??????? -5.83% (ns, p=0.394)? -5.39% (ns, >>>>> p=0.347) >>>>> >>>>> ??????? (s) - significant? (ns) - not-significant >>>>> >>>>> ??? - For historical comparison, the Aurora Perf comparision tool >>>>> ??????? reported for v2.06 with a baseline of jdk-13+31: >>>>> >>>>> ??????? Comparison????????????? max-jOPS critical-jOPS >>>>> ??????? ----------------------? -------------------- >>>>> -------------------- >>>>> ??????? base vs 2.06??????????? -0.32% (ns, p=0.345)? +0.71% (ns, >>>>> p=0.646) >>>>> ??????? base vs 2.06_off??????? +0.49% (ns, p=0.292)? -1.21% (ns, >>>>> p=0.481) >>>>> >>>>> ??????? (s) - significant? (ns) - not-significant >>>>> >>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>> >>>>> Dan >>>>> >>>>> >>>>> On 8/28/19 5:02 PM, Daniel D. Daugherty wrote: >>>>>> Greetings, >>>>>> >>>>>> The Async Monitor Deflation project has rebased to JDK14 so it's time >>>>>> for our first code review in that new context!! >>>>>> >>>>>> I've been focused on changing the monitor list management code to be >>>>>> lock-free in order to make SPECjbb2015 happier. Of course with a >>>>>> change >>>>>> like that, it takes a while to chase down all the new and wonderful >>>>>> races. At this point, I have the code back to the same stability that >>>>>> I had with CR5/v2.05/8-for-jdk13. >>>>>> >>>>>> To lay the ground work for this round of review, I pushed the >>>>>> following >>>>>> two fixes to jdk/jdk earlier today: >>>>>> >>>>>> ??? JDK-8230184 rename, whitespace, indent and comments changes in >>>>>> preparation >>>>>> ? ? ??????????? for lock free Monitor lists >>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8230184 >>>>>> >>>>>> ??? JDK-8230317 serviceability/sa/ClhsdbPrintStatics.java fails >>>>>> after 8230184 >>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8230317 >>>>>> >>>>>> I have attached the list of fixes from CR5 to CR6 instead of putting >>>>>> in the main body of this email. >>>>>> >>>>>> Main bug URL: >>>>>> >>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>> >>>>>> The project is currently baselined on jdk-14+11 plus the fixes for >>>>>> JDK-8230184 and JDK-8230317. >>>>>> >>>>>> Here's the full webrev URL for those folks that want to see all of >>>>>> the >>>>>> current Async Monitor Deflation code in one go (v2.06 full): >>>>>> >>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.06.full/ >>>>>> >>>>>> >>>>>> >>>>>> The primary focus of this review cycle is on the lock-free Monitor >>>>>> List >>>>>> management changes so here's a webrev for just that patch (v2.06c): >>>>>> >>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.06c.inc/ >>>>>> >>>>>> >>>>>> The secondary focus of this review cycle is on the bug fixes that >>>>>> have >>>>>> been made since CR5/v2.05/8-for-jdk13 so here's a webrev for just >>>>>> that >>>>>> patch (v2.06b): >>>>>> >>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.06b.inc/ >>>>>> >>>>>> >>>>>> The third and final bucket for this review cycle is the rename, >>>>>> whitespace, >>>>>> indent and comments changes made in preparation for lock free >>>>>> Monitor list >>>>>> management. Almost all of that was extracted into JDK-8230184 for the >>>>>> baseline so this bucket now has just a few comment changes >>>>>> relative to >>>>>> CR5/v2.05/8-for-jdk13. Here's a webrev for the remainder (v2.06a): >>>>>> >>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.06a.inc/ >>>>>> >>>>>> >>>>>> >>>>>> Some folks might want to see just what has changed since the last >>>>>> review >>>>>> cycle so here's a webrev for that (v2.06 inc): >>>>>> >>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.06.inc/ >>>>>> >>>>>> >>>>>> >>>>>> Last, but not least, some folks might want to see the code before the >>>>>> addition of lock-free Monitor List management so here's a webrev for >>>>>> that (v2.00 -> v2.05): >>>>>> >>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.05.inc/ >>>>>> >>>>>> >>>>>> The OpenJDK wiki will need minor updates to match the CR6 changes: >>>>>> >>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>> >>>>>> but that should only be changes to describe per-thread list async >>>>>> monitor >>>>>> deflation being done by the ServiceThread. >>>>>> >>>>>> (I did update the OpenJDK wiki for the CR5 changes back on >>>>>> 2019.08.14) >>>>>> >>>>>> This version of the patch has been thru Mach5 tier[1-8] testing on >>>>>> Oracle's usual set of platforms. It has also been through my usual >>>>>> set >>>>>> of stress testing on Linux-X64, macOSX and Solaris-X64. >>>>>> >>>>>> I did a bunch of SPECjbb2015 testing in Oracle's Aurora >>>>>> Performance lab >>>>>> using using their tuned SPECjbb2015 Linux-X64 G1 configs. This was >>>>>> using >>>>>> this patch baselined on jdk-13+31 (for stability): >>>>>> >>>>>> ????????? hbIR?????????? hbIR >>>>>> ???? (max attempted)? (settled)? max-jOPS? critical-jOPS runtime >>>>>> ???? ---------------? ---------? --------? ------------- ------- >>>>>> ??????????? 34282.00?? 28837.20? 27905.20?????? 19817.40 3658.10 base >>>>>> ??????????? 34965.70?? 29798.80? 27814.90?????? 19959.00 3514.60 >>>>>> v2.06d >>>>>> ??????????? 34282.00?? 29100.70? 28042.50?????? 19577.00 3701.90 >>>>>> v2.06d_off >>>>>> ??????????? 34282.00?? 29218.50? 27562.80?????? 19397.30 3657.60 >>>>>> v2.06d_ocache >>>>>> ??????????? 34965.70?? 29838.30? 26512.40?????? 19170.60 3569.90 >>>>>> v2.05 >>>>>> ??????????? 34282.00?? 28926.10? 27734.00?????? 19835.10 3588.40 >>>>>> v2.05_off >>>>>> >>>>>> The "off" configs are with -XX:-AsyncDeflateIdleMonitors specified >>>>>> and >>>>>> the "ocache" config is with 128 byte cache line sizes instead of >>>>>> 64 byte >>>>>> cache lines sizes. "v2.06d" is the last set of changes that I made >>>>>> before >>>>>> those changes were distributed into the "v2.06a", "v2.06b" and >>>>>> "v2.06c" >>>>>> buckets for this review recycle. >>>>>> >>>>>> >>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>> >>>>>> Dan >>>>>> >>>>>> >>>>>> On 7/11/19 3:49 PM, Daniel D. Daugherty wrote: >>>>>>> Greetings, >>>>>>> >>>>>>> I've been focused on chasing down and fixing the rare test failures >>>>>>> that only pop up rarely. So this round is primarily fixes for races >>>>>>> with a few additional fixes that came from Karen's review of CR4. >>>>>>> Thanks Karen! >>>>>>> >>>>>>> I have attached the list of fixes from CR4 to CR5 instead of putting >>>>>>> in the main body of this email. >>>>>>> >>>>>>> Main bug URL: >>>>>>> >>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>> >>>>>>> The project is currently baselined on jdk-13+29. This will likely be >>>>>>> the last JDK13 baseline for this project and I'll roll to the JDK14 >>>>>>> (jdk/jdk) repo soon... >>>>>>> >>>>>>> Here's the full webrev URL: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/8-for-jdk13.full/ >>>>>>> >>>>>>> Here's the incremental webrev URL: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/8-for-jdk13.inc/ >>>>>>> >>>>>>> I have not yet checked the OpenJDK wiki to see if it needs any >>>>>>> updates >>>>>>> to match the CR5 changes: >>>>>>> >>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>> >>>>>>> >>>>>>> (I did update the OpenJDK wiki for the CR4 changes back on >>>>>>> 2019.06.26) >>>>>>> >>>>>>> This version of the patch has been thru Mach5 tier[1-3] testing on >>>>>>> Oracle's usual set of platforms. Mach5 tier[4-6] is running now and >>>>>>> Mach5 tier[78] will follow. I'll kick off the usual stress testing >>>>>>> on Linux-X64, macOSX and Solaris-X64 as those machines become >>>>>>> available. >>>>>>> Since I haven't made any performance changes in this round, I'll >>>>>>> only >>>>>>> be running SPECjbb2015 to gather the latest monitorinflation logs. >>>>>>> >>>>>>> Next up: >>>>>>> >>>>>>> - We're still seeing 4-5% lower performance with SPECjbb2015 on >>>>>>> ? Linux-X64 and we've determined that some of that comes from >>>>>>> ? contention on the gListLock. So I'm going to investigate removing >>>>>>> ? the gListLock. Yes, another lock free set of changes is coming! >>>>>>> - Of course, going lock free often causes new races and new failures >>>>>>> ? so that's a good reason for make those changes isolated in their >>>>>>> ? own round (and not holding up CR5/v2.05/8-for-jdk13 anymore). >>>>>>> - I finally have a potential fix for the Win* failure with >>>>>>> ? ? gc/g1/humongousObjects/TestHumongousClassLoader.java >>>>>>> ? but I haven't run it through Mach5 yet so it'll be in the next >>>>>>> round. >>>>>>> - Some RTM tests were recently re-enabled in Mach5 and I'm seeing >>>>>>> some >>>>>>> ? monitor related failures there. I suspect that I need to go take a >>>>>>> ? look at the C2 RTM macro assembler code and look for things >>>>>>> that might >>>>>>> ? conflict if Async Monitor Deflation. If you're interested in >>>>>>> that kind >>>>>>> ? of issue, then see the macroAssembler_x86.cpp sanity check that I >>>>>>> ? added in this round! >>>>>>> >>>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>>> >>>>>>> Dan >>>>>>> >>>>>>> >>>>>>> On 5/26/19 8:30 PM, Daniel D. Daugherty wrote: >>>>>>>> Greetings, >>>>>>>> >>>>>>>> I have a fix for an issue that came up during performance testing. >>>>>>>> Many thanks to Robbin for diagnosing the issue in his SPECjbb2015 >>>>>>>> experiments. >>>>>>>> >>>>>>>> Here's the list of changes from CR3 to CR4. The list is a bit >>>>>>>> verbose due to the complexity of the issue, but the changes >>>>>>>> themselves are not that big. >>>>>>>> >>>>>>>> Functional: >>>>>>>> ? - Change SafepointSynchronize::is_cleanup_needed() from calling >>>>>>>> ??? ObjectSynchronizer::is_cleanup_needed() to calling >>>>>>>> ObjectSynchronizer::is_safepoint_deflation_needed(): >>>>>>>> ??? - is_safepoint_deflation_needed() returns the result of >>>>>>>> ????? monitors_used_above_threshold() for safepoint based >>>>>>>> ????? monitor deflation (!AsyncDeflateIdleMonitors). >>>>>>>> ??? - For AsyncDeflateIdleMonitors, it only returns true if >>>>>>>> ????? there is a special deflation request, e.g., System.gc() >>>>>>>> ????? - This solves a bug where there are a bunch of Cleanup >>>>>>>> ??????? safepoints that simply request async deflation which >>>>>>>> ??????? keeps the async JavaThreads from making progress on >>>>>>>> ??????? their async deflation work. >>>>>>>> ? - Add AsyncDeflationInterval diagnostic option. Description: >>>>>>>> ????? Async deflate idle monitors every so many milliseconds when >>>>>>>> ????? MonitorUsedDeflationThreshold is exceeded (0 is off). >>>>>>>> ? - Replace ObjectSynchronizer::gOmShouldDeflateIdleMonitors() with >>>>>>>> ??? ObjectSynchronizer::is_async_deflation_needed(): >>>>>>>> ??? - is_async_deflation_needed() returns true when >>>>>>>> ????? is_async_cleanup_requested() is true or when >>>>>>>> ????? monitors_used_above_threshold() is true (but no more often >>>>>>>> than >>>>>>>> ????? AsyncDeflationInterval). >>>>>>>> ??? - if AsyncDeflateIdleMonitors Service_lock->wait() now waits >>>>>>>> for >>>>>>>> ????? at most GuaranteedSafepointInterval millis: >>>>>>>> ????? - This allows is_async_deflation_needed() to be checked at >>>>>>>> ??????? the same interval as GuaranteedSafepointInterval. >>>>>>>> ??????? (default is 1000 millis/1 second) >>>>>>>> ????? - Once is_async_deflation_needed() has returned true, it >>>>>>>> ??????? generally cannot return true for AsyncDeflationInterval. >>>>>>>> ??????? This is to prevent async deflation from swamping the >>>>>>>> ??????? ServiceThread. >>>>>>>> ? - The ServiceThread still handles async deflation of the global >>>>>>>> ??? in-use list and now it also marks JavaThreads for async >>>>>>>> deflation >>>>>>>> ??? of their in-use lists. >>>>>>>> ??? - The ServiceThread will check for async deflation work every >>>>>>>> ????? GuaranteedSafepointInterval. >>>>>>>> ??? - A safepoint can still cause the ServiceThread to check for >>>>>>>> ????? async deflation work via is_async_deflation_requested. >>>>>>>> ? - Refactor code from ObjectSynchronizer::is_cleanup_needed() into >>>>>>>> ??? monitors_used_above_threshold() and remove is_cleanup_needed(). >>>>>>>> ? - In addition to System.gc(), the VM_Exit VM op and the final >>>>>>>> ??? VMThread safepoint now set the is_special_deflation_requested >>>>>>>> ??? flag to reduce the in-use monitor population that is >>>>>>>> reported by >>>>>>>> ??? ObjectSynchronizer::log_in_use_monitor_details() at VM exit. >>>>>>>> >>>>>>>> Test update: >>>>>>>> ? - test/hotspot/gtest/oops/test_markOop.cpp is updated to work >>>>>>>> with >>>>>>>> ??? AsyncDeflateIdleMonitors. >>>>>>>> >>>>>>>> Collateral: >>>>>>>> ? - Add/clarify/update some logging messages. >>>>>>>> >>>>>>>> Cleanup: >>>>>>>> ? - Updated comments based on Karen's code review. >>>>>>>> ? - Change 'special cleanup' -> 'special deflation' and >>>>>>>> ??? 'async cleanup' -> 'async deflation'. >>>>>>>> ??? - comment and function name changes >>>>>>>> ? - Clarify MonitorUsedDeflationThreshold description; >>>>>>>> >>>>>>>> >>>>>>>> Main bug URL: >>>>>>>> >>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>> >>>>>>>> The project is currently baselined on jdk-13+22. >>>>>>>> >>>>>>>> Here's the full webrev URL: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/7-for-jdk13.full/ >>>>>>>> >>>>>>>> Here's the incremental webrev URL: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/7-for-jdk13.inc/ >>>>>>>> >>>>>>>> I have not updated the OpenJDK wiki to reflect the CR4 changes: >>>>>>>> >>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>> >>>>>>>> >>>>>>>> The wiki doesn't say a whole lot about the async deflation >>>>>>>> invocation >>>>>>>> mechanism so I have to figure out how to add that content. >>>>>>>> >>>>>>>> This version of the patch has been thru Mach5 tier[1-8] testing on >>>>>>>> Oracle's usual set of platforms. My Solaris-X64 stress kit run is >>>>>>>> running now. Kitchensink8H on product, fastdebug, and slowdebug >>>>>>>> bits >>>>>>>> are running on Linux-X64, MacOSX and Solaris-X64. I still have >>>>>>>> to run >>>>>>>> my stress kit on Linux-X64. I still have to run the SPECjbb2015 >>>>>>>> baseline and CR4 runs on Linux-X64, MacOSX and Solaris-X64. >>>>>>>> >>>>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>>>> >>>>>>>> Dan >>>>>>>> >>>>>>>> On 5/6/19 11:52 AM, Daniel D. Daugherty wrote: >>>>>>>>> Greetings, >>>>>>>>> >>>>>>>>> I had some discussions with Karen about a race that was in the >>>>>>>>> ObjectMonitor::enter() code in CR2/v2.02/5-for-jdk13. This race >>>>>>>>> was >>>>>>>>> theoretical and I had no test failures due to it. The fix is >>>>>>>>> pretty >>>>>>>>> simple: remove the special case code for async deflation in the >>>>>>>>> ObjectMonitor::enter() function and rely solely on the ref_count >>>>>>>>> for ObjectMonitor::enter() protection. >>>>>>>>> >>>>>>>>> During those discussions Karen also floated the idea of using the >>>>>>>>> ref_count field instead of the contentions field for the Async >>>>>>>>> Monitor Deflation protocol. I decided to go ahead and code up that >>>>>>>>> change and I have run it through the usual stress and Mach5 >>>>>>>>> testing >>>>>>>>> with no issues. It's also known as v2.03 (for those for with the >>>>>>>>> patches) and as webrev/6-for-jdk13 (for those with webrev URLs). >>>>>>>>> Sorry for all the names... >>>>>>>>> >>>>>>>>> Main bug URL: >>>>>>>>> >>>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>>> >>>>>>>>> The project is currently baselined on jdk-13+18. >>>>>>>>> >>>>>>>>> Here's the full webrev URL: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/6-for-jdk13.full/ >>>>>>>>> >>>>>>>>> >>>>>>>>> Here's the incremental webrev URL: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/6-for-jdk13.inc/ >>>>>>>>> >>>>>>>>> I have also updated the OpenJDK wiki to reflect the CR3 changes: >>>>>>>>> >>>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>>> >>>>>>>>> >>>>>>>>> This version of the patch has been thru Mach5 tier[1-8] testing on >>>>>>>>> Oracle's usual set of platforms. My Solaris-X64 stress kit run had >>>>>>>>> no issues. Kitchensink8H on product, fastdebug, and slowdebug bits >>>>>>>>> had no failures on Linux-X64; MacOSX fastdebug and slowdebug and >>>>>>>>> Solaris-X64 release had the usual "Too large time diff" >>>>>>>>> complaints. >>>>>>>>> 12 hour Inflate2 runs on product, fastdebug and slowdebug bits on >>>>>>>>> Linux-X64, MacOSX and Solaris-X64 had no failures. My Linux-X64 >>>>>>>>> stress kit is running right now. >>>>>>>>> >>>>>>>>> I've done the SPECjbb2015 baseline and CR3 runs. I need to gather >>>>>>>>> the results and analyze them. >>>>>>>>> >>>>>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>>>>> >>>>>>>>> Dan >>>>>>>>> >>>>>>>>> >>>>>>>>> On 4/25/19 12:38 PM, Daniel D. Daugherty wrote: >>>>>>>>>> Greetings, >>>>>>>>>> >>>>>>>>>> I have a small but important bug fix for the Async Monitor >>>>>>>>>> Deflation >>>>>>>>>> project ready to go. It's also known as v2.02 (for those for >>>>>>>>>> with the >>>>>>>>>> patches) and as webrev/5-for-jdk13 (for those with webrev >>>>>>>>>> URLs). Sorry >>>>>>>>>> for all the names... >>>>>>>>>> >>>>>>>>>> JDK-8222295 was pushed to jdk/jdk two days ago so that >>>>>>>>>> baseline patch >>>>>>>>>> is out of our hair. >>>>>>>>>> >>>>>>>>>> Main bug URL: >>>>>>>>>> >>>>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>>>> >>>>>>>>>> The project is currently baselined on jdk-13+17. >>>>>>>>>> >>>>>>>>>> Here's the full webrev URL: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/5-for-jdk13.full/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Here's the incremental webrev URL (JDK-8153224): >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/5-for-jdk13.inc/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> I still have to update the OpenJDK wiki to reflect the CR2 >>>>>>>>>> changes: >>>>>>>>>> >>>>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> This version of the patch has been thru Mach5 tier[1-6] >>>>>>>>>> testing on >>>>>>>>>> Oracle's usual set of platforms. Mach5 tier[7-8] is running now. >>>>>>>>>> My stress kit is running on Solaris-X64 now. Kitchensink8H is >>>>>>>>>> running >>>>>>>>>> now on product, fastdebug, and slowdebug bits on Linux-X64, >>>>>>>>>> MacOSX >>>>>>>>>> and Solaris-X64. 12 hour Inflate2 runs are running now on >>>>>>>>>> product, >>>>>>>>>> fastdebug and slowdebug bits on Linux-X64, MacOSX and >>>>>>>>>> Solaris-X64. >>>>>>>>>> I'll start my my stress kit on Linux-X64 sometime on Sunday >>>>>>>>>> (after >>>>>>>>>> my jdk-13+18 stress run is done). >>>>>>>>>> >>>>>>>>>> I'll do SPECjbb2015 baseline and CR2 runs after all the stress >>>>>>>>>> testing is done. >>>>>>>>>> >>>>>>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>>>>>> >>>>>>>>>> Dan >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 4/19/19 11:58 AM, Daniel D. Daugherty wrote: >>>>>>>>>>> Greetings, >>>>>>>>>>> >>>>>>>>>>> I finally have CR1 for the Async Monitor Deflation project >>>>>>>>>>> ready to >>>>>>>>>>> go. It's also known as v2.01 (for those for with the patches) >>>>>>>>>>> and as >>>>>>>>>>> webrev/4-for-jdk13 (for those with webrev URLs). Sorry for >>>>>>>>>>> all the >>>>>>>>>>> names... >>>>>>>>>>> >>>>>>>>>>> Main bug URL: >>>>>>>>>>> >>>>>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>>>>> >>>>>>>>>>> Baseline bug fixes URL: >>>>>>>>>>> >>>>>>>>>>> ??? JDK-8222295 more baseline cleanups from Async Monitor >>>>>>>>>>> Deflation project >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8222295 >>>>>>>>>>> >>>>>>>>>>> The project is currently baselined on jdk-13+15. >>>>>>>>>>> >>>>>>>>>>> Here's the webrev for the latest baseline changes (JDK-8222295): >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/4-for-jdk13.8222295 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Here's the full webrev URL (JDK-8153224 only): >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/4-for-jdk13.full/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Here's the incremental webrev URL (JDK-8153224): >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/4-for-jdk13.inc/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> So I'm looking for reviews for both JDK-8222295 and the >>>>>>>>>>> latest version >>>>>>>>>>> of JDK-8153224... >>>>>>>>>>> >>>>>>>>>>> I still have to update the OpenJDK wiki to reflect the CR >>>>>>>>>>> changes: >>>>>>>>>>> >>>>>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> This version of the patch has been thru Mach5 tier[1-3] >>>>>>>>>>> testing on >>>>>>>>>>> Oracle's usual set of platforms. Mach5 tier[4-6] is running >>>>>>>>>>> now and >>>>>>>>>>> Mach5 tier[78] will be run later today. My stress kit on >>>>>>>>>>> Solaris-X64 >>>>>>>>>>> is running now. Linux-X64 stress testing will start on >>>>>>>>>>> Sunday. I'm >>>>>>>>>>> planning to do Kitchensink runs, SPECjbb2015 runs and my monitor >>>>>>>>>>> inflation stress tests on Linux-X64, MacOSX and Solaris-X64. >>>>>>>>>>> >>>>>>>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>>>>>>> >>>>>>>>>>> Dan >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 3/24/19 9:57 AM, Daniel D. Daugherty wrote: >>>>>>>>>>>> Greetings, >>>>>>>>>>>> >>>>>>>>>>>> Welcome to the OpenJDK review thread for my port of >>>>>>>>>>>> Carsten's work on: >>>>>>>>>>>> >>>>>>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>>>>>> >>>>>>>>>>>> Here's a link to the OpenJDK wiki that describes my port: >>>>>>>>>>>> >>>>>>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Here's the webrev URL: >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/3-for-jdk13/ >>>>>>>>>>>> >>>>>>>>>>>> Here's a link to Carsten's original webrev: >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~cvarming/monitor_deflate_conc/0/ >>>>>>>>>>>> >>>>>>>>>>>> Earlier versions of this patch have been through several >>>>>>>>>>>> rounds of >>>>>>>>>>>> preliminary review. Many thanks to Carsten, Coleen, Robbin, and >>>>>>>>>>>> Roman for their preliminary code review comments. A very >>>>>>>>>>>> special >>>>>>>>>>>> thanks to Robbin and Roman for building and testing the >>>>>>>>>>>> patch in >>>>>>>>>>>> their own environments (including specJBB2015). >>>>>>>>>>>> >>>>>>>>>>>> This version of the patch has been thru Mach5 tier[1-8] >>>>>>>>>>>> testing on >>>>>>>>>>>> Oracle's usual set of platforms. Earlier versions have been run >>>>>>>>>>>> through my stress kit on my Linux-X64 and Solaris-X64 servers >>>>>>>>>>>> (product, fastdebug, slowdebug).Earlier versions have run >>>>>>>>>>>> Kitchensink >>>>>>>>>>>> for 12 hours on MacOSX, Linux-X64 and Solaris-X64 (product, >>>>>>>>>>>> fastdebug >>>>>>>>>>>> and slowdebug). Earlier versions have run my monitor >>>>>>>>>>>> inflation stress >>>>>>>>>>>> tests for 12 hours on MacOSX, Linux-X64 and Solaris-X64 >>>>>>>>>>>> (product, >>>>>>>>>>>> fastdebug and slowdebug). >>>>>>>>>>>> >>>>>>>>>>>> All of the testing done on earlier versions will be redone >>>>>>>>>>>> on the >>>>>>>>>>>> latest version of the patch. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>>>>>>>> >>>>>>>>>>>> Dan >>>>>>>>>>>> >>>>>>>>>>>> P.S. >>>>>>>>>>>> One subtest in >>>>>>>>>>>> gc/g1/humongousObjects/TestHumongousClassLoader.java >>>>>>>>>>>> is currently failing in -Xcomp mode on Win* only. I've been >>>>>>>>>>>> trying >>>>>>>>>>>> to characterize/analyze this failure for more than a week >>>>>>>>>>>> now. At >>>>>>>>>>>> this point I'm convinced that Async Monitor Deflation is >>>>>>>>>>>> aggravating >>>>>>>>>>>> an existing bug. However, I plan to have a better handle on >>>>>>>>>>>> that >>>>>>>>>>>> failure before these bits are pushed to the jdk/jdk repo. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From ioi.lam at oracle.com Mon Mar 9 04:52:45 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Sun, 8 Mar 2020 21:52:45 -0700 Subject: RFR(XS) 8240548 [TESTBUG] CDS NoClassToArchive.java fails with Graal Message-ID: <9fd602d1-0e2c-5920-d677-8f2d0a1616be@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8240548 http://cr.openjdk.java.net/~iklam/jdk15/8240548-NoClassToArchive.graal.v01/ This test should not be executed with graal. This test started failing in valhalla/nestmate because the string "unsafe anonymous" is no longer printed by -Xlog:class+load=trace. However, the underlying problem is unrelated to nestmate. So I will fix the test in the mainline jdk/jdk and it can be synced down to nestmate. The test assumes that when "java -version" is executed, only a very limited number of classes are loaded, and all of those are loaded from the default shared archive. However, when graal is used as the JIT, many extra classes are loaded during VM start-up. Some of those are loaded dynamically from jrt:/. Some classes are also defined by LambdaMetafactory. This causes complexity that cannot be easily handled by this test. The VM code covered by this test can be sufficiently tested with C1/C2. So there's no need to bend over backwards to run this test with graal. Thanks - Ioi From ioi.lam at oracle.com Mon Mar 9 05:03:03 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Sun, 8 Mar 2020 22:03:03 -0700 Subject: Need Information for memory sharing with AppCDS In-Reply-To: <4c513dcc-ed4e-43ec-a607-764ded395058.shiyue.xw@alibaba-inc.com> References: <4c513dcc-ed4e-43ec-a607-764ded395058.shiyue.xw@alibaba-inc.com> Message-ID: <86cccf4b-e497-bd03-155a-c51fbcc9ae77@oracle.com> On 3/8/20 8:51 PM, ??(??) wrote: > Hi, > > We am trying to take advantage of the characteristics of memory sharing with AppCDS, and we expected that the > JVM internal packages and common packages can be shared with AppCDS. > > In the user scenario, there are some common base packages for different Java processes with "-cp common_path" > Process A: -cp common_path:pathA > Process B: -cp common_path:pathA:pathB > Process C: -cp common_path:pathC > > I check the code in JDK11, and it uses SharedPathsMiscInfo to do the consistency check for class path. > The below usage can work in JDK11 > Dump stage: > -cp common_path > > Runtime stage: > -cp common_path:new_path > > But I find that SharedPathsMiscInfo has been removed in the latest JDK code. > > I need the information: > 1. the new flow to do the consistency check. > 2. constraint condition about the memory sharing among the different Java processes. > > Is there any user guide to use the memory sharing feature for AppCDS? Hi Wei, The topic of sharing the same CDS archive between different apps that have the same classpath prefix can be found here: https://docs.oracle.com/en/java/javase/13/docs/specs/man/java.html#application-class-data-sharing See the section "Sharing a Shared Archive Across Multiple Application Processes". SharedPathsMiscInfo was removed in this RFE in JDK 14: https://bugs.openjdk.java.net/browse/JDK-8227370 However, we still enforce the same set of consistency checks as previous JDK versions. Please let me know if you have further questions. Thanks - Ioi > Thanks, > Wei Xiang From david.holmes at oracle.com Mon Mar 9 05:33:44 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 9 Mar 2020 15:33:44 +1000 Subject: RFR(L): JDK-8229517: Support for optional asynchronous/buffered logging In-Reply-To: <77FA4BF1-DC2E-4AC8-987B-56578E5D33B9@amazon.com> References: <77FA4BF1-DC2E-4AC8-987B-56578E5D33B9@amazon.com> Message-ID: <614b4e2d-3d66-2731-c35b-53a3c96de71f@oracle.com> Hi, On 5/03/2020 8:04 pm, Liu, Xin wrote: > Hi, hotspot reviewers, > > Could you take a look at my webrev? > Bug: https://bugs.openjdk.java.net/browse/JDK-8229517 > Webrev: https://cr.openjdk.java.net/~xliu/8229517/01/webrev/ > > Previously, we had discussion at runtime-dev. https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-February/038211.html > I would like to continue it because I redo my implementation on the tip of jdk and have made it a general solution. Not a review as such but a couple of comments/queries. The async logging is done as a periodic task that will be executed the Watcher thread. This task requires a mutex that is also used by the other threads doing the log operations. So potentially the watcher thread could experience a significant delay waiting for the mutex. I'm not clear what the actual lifecycle of the LogAsyncFlusher instance is. I would have expected a single instance initialized at VM startup and cleaned up at VM exit. The code gives the appearance that this can actually be much more dynamic with multiple instances coming and going through the lifetime of the VM. Deletion of the instance is particularly problematic due to the Mutex, which has to be guaranteed not to be accessed when it gets deleted. > This webrev provides on apar with . Even though it doesn?t touch any existing logging calls, it?s very possible to offload any callsite of logging to the asynchronous buffer. > My rationale is that we only need to redirect some time-critical logs to asynchronous buffer to dodge unexpected filesystem delay. Eg. g1gc has some loggings at safepoint. We mitigate those long-tailed GC latencies rooted by filesystem writing to gclog. So this isn't an opt-in facility at runtime, but rather something that has to be decided a-priori by the developer. How does this interact with the synchronous logging that may be used for a given set of log tags? Thanks, David ----- > I got feedback from my our internal users. They have a concern about my approach of dynamic memory. Indeed, I gave up ringbuffer design in previous version and turn to use a dynamic allocated linkedlist. If writing to filesystem stalls infinitely, JVM may implode because of memory usage. I plan to solve it by adding a memory quota to the FIFO of logAsyncFlusher.hpp. It will start dropping early messages of the FIFO if it has consumed too many free memories. Another potential improvement is changing the FIFO from mutex protected to lock-free. It make sense if we redirect high volume of logging messages to it and mutex becomes hot. > > Thanks, > --lx > > > From david.holmes at oracle.com Mon Mar 9 05:41:40 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 9 Mar 2020 15:41:40 +1000 Subject: RFR(XS) 8240548 [TESTBUG] CDS NoClassToArchive.java fails with Graal In-Reply-To: <9fd602d1-0e2c-5920-d677-8f2d0a1616be@oracle.com> References: <9fd602d1-0e2c-5920-d677-8f2d0a1616be@oracle.com> Message-ID: <40b3e05c-ef1d-fab6-b76b-f2db33056f73@oracle.com> Hi Ioi, On 9/03/2020 2:52 pm, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8240548 > http://cr.openjdk.java.net/~iklam/jdk15/8240548-NoClassToArchive.graal.v01/ > > This test should not be executed with graal. > > This test started failing in valhalla/nestmate because the string > "unsafe anonymous" is no longer printed by -Xlog:class+load=trace. However, > the underlying problem is unrelated to nestmate. So I will fix the test in > the mainline jdk/jdk and it can be synced down to nestmate. > > The test assumes that when "java -version" is executed, only a very limited > number of classes are loaded, and all of those are loaded from the default > shared archive. > > However, when graal is used as the JIT, many extra classes are loaded > during VM start-up. Some of those are loaded dynamically from jrt:/. > Some classes are also defined by LambdaMetafactory. This causes complexity > that cannot be easily handled by this test. > > The VM code covered by this test can be sufficiently tested with C1/C2. So > there's no need to bend over backwards to run this test with graal. Okay. I'd probably have that explanatory comment up the top of the file rather than within doTest(), but that's just a nit. Thanks, David > Thanks > - Ioi From shiyue.xw at alibaba-inc.com Mon Mar 9 06:55:26 2020 From: shiyue.xw at alibaba-inc.com (=?UTF-8?B?5ZCR5LyfKOivhuaciCk=?=) Date: Mon, 09 Mar 2020 14:55:26 +0800 Subject: =?UTF-8?B?5Zue5aSN77yaTmVlZCBJbmZvcm1hdGlvbiBmb3IgbWVtb3J5IHNoYXJpbmcgd2l0aCBBcHBD?= =?UTF-8?B?RFM=?= In-Reply-To: <86cccf4b-e497-bd03-155a-c51fbcc9ae77@oracle.com> References: <4c513dcc-ed4e-43ec-a607-764ded395058.shiyue.xw@alibaba-inc.com>, <86cccf4b-e497-bd03-155a-c51fbcc9ae77@oracle.com> Message-ID: <7d348b97-161b-43d2-bd7b-896a50287a57.shiyue.xw@alibaba-inc.com> Hi Ioi, Thanks a lot for your information. I have the question about the usage in the document: For common class path prefix: --------------------------------------- Run the Hello and Hi applications with the same shared archive: java -XX:SharedArchiveFile=common.jsa -cp common.jar:hello.jar:hi.jar Hello java -XX:SharedArchiveFile=common.jsa -cp common.jar:hello.jar:hi.jar Hi If a class with name ABC is both in hello.jar and hi.jar, the content is different. With the class path -cp common.jar:hello.jar:hi.jar, the second run "java .. Hi" can't load the correct class ABC in hi.jar. Is there any solution for this usage? Thanks, Wei Xiang ------------------------------------------------------------------ ????Ioi Lam ?????2020?3?9?(???) 13:03 ????hotspot-runtime-dev ????Re: Need Information for memory sharing with AppCDS On 3/8/20 8:51 PM, ??(??) wrote: > Hi, > > We am trying to take advantage of the characteristics of memory sharing with AppCDS, and we expected that the > JVM internal packages and common packages can be shared with AppCDS. > > In the user scenario, there are some common base packages for different Java processes with "-cp common_path" > Process A: -cp common_path:pathA > Process B: -cp common_path:pathA:pathB > Process C: -cp common_path:pathC > > I check the code in JDK11, and it uses SharedPathsMiscInfo to do the consistency check for class path. > The below usage can work in JDK11 > Dump stage: > -cp common_path > > Runtime stage: > -cp common_path:new_path > > But I find that SharedPathsMiscInfo has been removed in the latest JDK code. > > I need the information: > 1. the new flow to do the consistency check. > 2. constraint condition about the memory sharing among the different Java processes. > > Is there any user guide to use the memory sharing feature for AppCDS? Hi Wei, The topic of sharing the same CDS archive between different apps that have the same classpath prefix can be found here: https://docs.oracle.com/en/java/javase/13/docs/specs/man/java.html#application-class-data-sharing See the section "Sharing a Shared Archive Across Multiple Application Processes". SharedPathsMiscInfo was removed in this RFE in JDK 14: https://bugs.openjdk.java.net/browse/JDK-8227370 However, we still enforce the same set of consistency checks as previous JDK versions. Please let me know if you have further questions. Thanks - Ioi > Thanks, > Wei Xiang From thomas.schatzl at oracle.com Mon Mar 9 09:47:52 2020 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 9 Mar 2020 10:47:52 +0100 Subject: =?UTF-8?B?UmU6IOWbnuWkje+8mlZNIGNyYXNoZWQgYXQgU3RyaW5nVGFibGUgZXhw?= =?UTF-8?Q?ansion?= In-Reply-To: References: <40f49c75-93a8-4be3-ba8c-1dd2fbcae2c2.shiyue.xw@alibaba-inc.com> <8184d9b3-a4d1-3b17-c048-e574d744a62b@oracle.com> <524d0407-1ee2-97f6-f0e6-8cc76770043f@oracle.com> Message-ID: Hi, On 09.03.20 04:18, ??(??) wrote: > Hi? > > To avoid the crash, I have a proposal: > 1. add a new field "size_t _hash" in class Node. > 2. when insert the new Node into ConcurrentHashTable, set the hash in the node. > 3. For stringtable: > Find an invalid node > --> check whether the aux_hash is equal to _hash in the node. > --> if no, it means that the value in the node is changed. The node can be considered as an dead node? > and take care of the invalid node following the rule for the node whose dead_hash is true. > --> if yes, still regard it as fatal error. > > please review the proposal. If it is feasible, I will propose a patch to solve the problem. > Since there are other users of ConcurrentHashTable than the stringtabale I would prefer if this mitigation were limited to cases that may be affected by this issue, i.e. StringTable. I am sure your description meant to, but just in case: I.e. I have a prototype that uses CHT for remembered sets, and increasing a Node by a size_t (8 bytes) would increase memory usage of a Node significantly (+25%) which is a problem because remembered sets can consist mostly of Nodes, encoding its data into its VALUE without additional memory use. For such internal cases I would also prefer that the code just errors out instead of potentially (mostly silently) dropping nodes, i.e. in this case remembered sets. Thanks, Thomas From shiyue.xw at alibaba-inc.com Mon Mar 9 10:39:31 2020 From: shiyue.xw at alibaba-inc.com (=?UTF-8?B?5ZCR5LyfKOivhuaciCk=?=) Date: Mon, 09 Mar 2020 18:39:31 +0800 Subject: =?UTF-8?B?5Zue5aSN77ya5Zue5aSN77yaVk0gY3Jhc2hlZCBhdCBTdHJpbmdUYWJsZSBleHBhbnNpb24=?= In-Reply-To: References: <40f49c75-93a8-4be3-ba8c-1dd2fbcae2c2.shiyue.xw@alibaba-inc.com> <8184d9b3-a4d1-3b17-c048-e574d744a62b@oracle.com> <524d0407-1ee2-97f6-f0e6-8cc76770043f@oracle.com> , Message-ID: <92e3f33c-d7ca-4b85-8e82-7b828079b73e.shiyue.xw@alibaba-inc.com> Hi Thomas, The proposal is only limited to stringtable, and the code still errors out for other cases which use ConcurrentHashTable. I change the proposal as below: For StringTable, find an invalid node --> No matter what causes it?regard the node as an dead node, and take care of the invalid node following the rule for the node whose dead_hash is true. For other cases using ConcurrentHashTable, find an invalid node -> directly error out It doesn't increase the memory, and avoid the crash for Stringtable. Thanks, Wei Xiang ------------------------------------------------------------------ ????Thomas Schatzl ?????2020?3?9?(???) 17:48 ????hotspot-runtime-dev ????Re: ???VM crashed at StringTable expansion Hi, On 09.03.20 04:18, ??(??) wrote: > Hi? > > To avoid the crash, I have a proposal: > 1. add a new field "size_t _hash" in class Node. > 2. when insert the new Node into ConcurrentHashTable, set the hash in the node. > 3. For stringtable: > Find an invalid node > --> check whether the aux_hash is equal to _hash in the node. > --> if no, it means that the value in the node is changed. The node can be considered as an dead node? > and take care of the invalid node following the rule for the node whose dead_hash is true. > --> if yes, still regard it as fatal error. > > please review the proposal. If it is feasible, I will propose a patch to solve the problem. > Since there are other users of ConcurrentHashTable than the stringtabale I would prefer if this mitigation were limited to cases that may be affected by this issue, i.e. StringTable. I am sure your description meant to, but just in case: I.e. I have a prototype that uses CHT for remembered sets, and increasing a Node by a size_t (8 bytes) would increase memory usage of a Node significantly (+25%) which is a problem because remembered sets can consist mostly of Nodes, encoding its data into its VALUE without additional memory use. For such internal cases I would also prefer that the code just errors out instead of potentially (mostly silently) dropping nodes, i.e. in this case remembered sets. Thanks, Thomas From aph at redhat.com Mon Mar 9 10:54:19 2020 From: aph at redhat.com (Andrew Haley) Date: Mon, 9 Mar 2020 10:54:19 +0000 Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is inconsistent In-Reply-To: References: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com> <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com> <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com> Message-ID: So, my plan is to push http://cr.openjdk.java.net/~aph/8240615-1/ and subsequently patch a few AArch64 cases plus the x86_64 immL_Pow2 and immL_NotPow2 cases that we already identified. I do not intend to do bulk changes to any other clients of is_power_of_2() because it's often unclear exactly what behaviour is required, and I'd risk breaking something. OK? -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From stefan.karlsson at oracle.com Mon Mar 9 12:20:56 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 9 Mar 2020 13:20:56 +0100 Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is inconsistent In-Reply-To: References: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com> <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com> <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com> Message-ID: On 2020-03-09 11:54, Andrew Haley wrote: > So, my plan is to push http://cr.openjdk.java.net/~aph/8240615-1/ and > subsequently patch a few AArch64 cases plus the x86_64 immL_Pow2 and > immL_NotPow2 cases that we already identified. > > I do not intend to do bulk changes to any other clients of > is_power_of_2() because it's often unclear exactly what behaviour is > required, and I'd risk breaking something. > > OK? I would prefer if the x86_64 and aarch64 patches were pushed first, or folded into this patch. Otherwise, we knowingly introduce a regression by pushing the proposed patch above. However, I'll leave it up to the compiler devs/maintainers to decide if this is OK. StefanK > From aph at redhat.com Mon Mar 9 12:37:05 2020 From: aph at redhat.com (Andrew Haley) Date: Mon, 9 Mar 2020 12:37:05 +0000 Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is inconsistent In-Reply-To: References: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com> <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com> <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com> Message-ID: On 3/9/20 12:20 PM, Stefan Karlsson wrote: > On 2020-03-09 11:54, Andrew Haley wrote: >> So, my plan is to push http://cr.openjdk.java.net/~aph/8240615-1/ and >> subsequently patch a few AArch64 cases plus the x86_64 immL_Pow2 and >> immL_NotPow2 cases that we already identified. >> >> I do not intend to do bulk changes to any other clients of >> is_power_of_2() because it's often unclear exactly what behaviour is >> required, and I'd risk breaking something. >> >> OK? > > I would prefer if the x86_64 and aarch64 patches were pushed first, or > folded into this patch. Otherwise, we knowingly introduce a regression > by pushing the proposed patch above. However, I'll leave it up to the > compiler devs/maintainers to decide if this is OK. We already have a regression in that AArch64 generates incorrect code. IMO it's likely other targets do too. However, I'm happy to push the x86_64 and aarch64 patches first or as part of this patch. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From martin.doerr at sap.com Mon Mar 9 14:33:50 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 9 Mar 2020 14:33:50 +0000 Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is inconsistent In-Reply-To: References: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com> <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com> <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com> Message-ID: Hi Andrew, I'm ok with your proposal (regardless of which change gets pushed first), too. In addition to that we should also fix: log2_int(int x) log2_jint(jint x) which use a wrong type conversion violating "If x < 0, the function returns 31 on a 32-bit machine and 63 on a 64-bit machine.". I guess I should open a new issue for that. Best regards, Martin > -----Original Message----- > From: hotspot-dev On Behalf Of > Andrew Haley > Sent: Montag, 9. M?rz 2020 13:37 > To: Stefan Karlsson ; John Rose > > Cc: hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is > inconsistent > > On 3/9/20 12:20 PM, Stefan Karlsson wrote: > > On 2020-03-09 11:54, Andrew Haley wrote: > >> So, my plan is to push http://cr.openjdk.java.net/~aph/8240615-1/ and > >> subsequently patch a few AArch64 cases plus the x86_64 immL_Pow2 and > >> immL_NotPow2 cases that we already identified. > >> > >> I do not intend to do bulk changes to any other clients of > >> is_power_of_2() because it's often unclear exactly what behaviour is > >> required, and I'd risk breaking something. > >> > >> OK? > > > > I would prefer if the x86_64 and aarch64 patches were pushed first, or > > folded into this patch. Otherwise, we knowingly introduce a regression > > by pushing the proposed patch above. However, I'll leave it up to the > > compiler devs/maintainers to decide if this is OK. > > We already have a regression in that AArch64 generates incorrect code. > IMO it's likely other targets do too. > > However, I'm happy to push the x86_64 and aarch64 patches first or > as part of this patch. > > -- > Andrew Haley (he/him) > Java Platform Lead Engineer > Red Hat UK Ltd. > https://keybase.io/andrewhaley > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From martin.doerr at sap.com Mon Mar 9 14:46:56 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 9 Mar 2020 14:46:56 +0000 Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is inconsistent In-Reply-To: References: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com> <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com> <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com> Message-ID: Sorry, my comment about log2_int + log2_jint was not precise. The comment "If x < 0, the function returns 31 on a 32-bit machine and 63 on a 64-bit machine." belongs to log2_intptr, but it also applies to log2_int + log2_jint which is not desirable IMHO. I guess using zero extend would be better for these 2 functions: "If x < 0, the function returns 31 on all machines." Best regards, Martin > -----Original Message----- > From: hotspot-runtime-dev bounces at openjdk.java.net> On Behalf Of Doerr, Martin > Sent: Montag, 9. M?rz 2020 15:34 > To: Andrew Haley ; Stefan Karlsson > ; John Rose > Cc: hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net > Subject: RE: RFR: 8240615: is_power_of_2() has Undefined > Behaviour and is inconsistent > > Hi Andrew, > > I'm ok with your proposal (regardless of which change gets pushed first), too. > > In addition to that we should also fix: > log2_int(int x) > log2_jint(jint x) > which use a wrong type conversion violating > "If x < 0, the function returns 31 on a 32-bit machine and 63 on a 64-bit > machine.". > > I guess I should open a new issue for that. > > Best regards, > Martin > > > > -----Original Message----- > > From: hotspot-dev On Behalf > Of > > Andrew Haley > > Sent: Montag, 9. M?rz 2020 13:37 > > To: Stefan Karlsson ; John Rose > > > > Cc: hotspot-dev at openjdk.java.net; hotspot-runtime- > dev at openjdk.java.net > > Subject: Re: RFR: 8240615: is_power_of_2() has Undefined Behaviour and > is > > inconsistent > > > > On 3/9/20 12:20 PM, Stefan Karlsson wrote: > > > On 2020-03-09 11:54, Andrew Haley wrote: > > >> So, my plan is to push http://cr.openjdk.java.net/~aph/8240615-1/ and > > >> subsequently patch a few AArch64 cases plus the x86_64 immL_Pow2 > and > > >> immL_NotPow2 cases that we already identified. > > >> > > >> I do not intend to do bulk changes to any other clients of > > >> is_power_of_2() because it's often unclear exactly what behaviour is > > >> required, and I'd risk breaking something. > > >> > > >> OK? > > > > > > I would prefer if the x86_64 and aarch64 patches were pushed first, or > > > folded into this patch. Otherwise, we knowingly introduce a regression > > > by pushing the proposed patch above. However, I'll leave it up to the > > > compiler devs/maintainers to decide if this is OK. > > > > We already have a regression in that AArch64 generates incorrect code. > > IMO it's likely other targets do too. > > > > However, I'm happy to push the x86_64 and aarch64 patches first or > > as part of this patch. > > > > -- > > Andrew Haley (he/him) > > Java Platform Lead Engineer > > Red Hat UK Ltd. > > https://keybase.io/andrewhaley > > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From daniel.daugherty at oracle.com Mon Mar 9 14:59:00 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 9 Mar 2020 10:59:00 -0400 Subject: RFR(L) 8153224 Monitor deflation prolong safepoints (CR10/v2.10/13-for-jdk15) In-Reply-To: References: <62729044-8a22-0e20-0eda-04d47c9ea23c@oracle.com> <313e51c8-b672-bb1c-577a-49868f09e6c1@oracle.com> <1fd54b23-bd35-9b8f-e6f3-6000440d8770@oracle.com> <3fb1a2b5-5aad-eab3-09ba-6f64a2242d30@oracle.com> <38e2d441-11c9-8342-37d5-8030dd06f2f4@oracle.com> <2a8976f7-37e0-03b9-3099-e07464e46512@oracle.com> <5681d640-08c8-3433-0f85-3f23eea69e87@oracle.com> Message-ID: <7b5ad2a5-ce90-74fc-b81a-06d8e8c7c6b5@oracle.com> Hi David, Thanks for the re-review! Dan On 3/9/20 12:10 AM, David Holmes wrote: > Hi Dan, > > Sorry for the delay in looking at latest updates. No further comments > from me at this stage. > > Thanks, > David > > On 27/02/2020 8:22 am, Daniel D. Daugherty wrote: >> Greetings, >> >> I have made changes to the Async Monitor Deflation code in response to >> the CR9/v2.09/12-for-jdk14 code review cycle. Thanks to Robbin and >> Erik O. >> for their comments in this round! >> >> With the extraction and push of {8235931,8236035,8235795} to JDK15, the >> Async Monitor Deflation code is back to "just" async deflation changes! >> >> I have attached the change list from CR9 to CR10 instead of putting >> it in >> the body of this email. I've also added a link to the >> CR9-to-CR10-changes >> file to the webrevs so it should be easy to find. >> >> Main bug URL: >> >> ???? JDK-8153224 Monitor deflation prolong safepoints >> ???? https://bugs.openjdk.java.net/browse/JDK-8153224 >> >> The project is currently baselined on jdk-15+11. >> >> Here's the full webrev URL for those folks that want to see all of the >> current Async Monitor Deflation code in one go (v2.10 full): >> >> http://cr.openjdk.java.net/~dcubed/8153224-webrev/13-for-jdk15+11.v2.10.full/ >> >> >> Some folks might want to see just what has changed since the last review >> cycle so here's a webrev for that (v2.10 inc): >> >> http://cr.openjdk.java.net/~dcubed/8153224-webrev/13-for-jdk15+11.v2.10.inc/ >> >> >> Since we backed out the HandshakeAfterDeflateIdleMonitors option and the >> C2 ref_count changes and updated the copyright years, the "inc" >> webrev has >> a bit more noise in it than usual. Sorry about that! >> >> The OpenJDK wiki has been updated for this round of changes: >> >> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >> >> The jdk-15+11 based v2.10 version of the patch has been thru Mach5 >> tier[1-7] >> testing on Oracle's usual set of platforms. Mach5 tier8 is still >> running. >> I'm running the v2.10 patch through my usual set of stress testing on >> Linux-X64 and macOSX. >> >> I'm planning to do a SPECjbb2015 round on the CR10/v2.20/13-for-jdk15 >> bits. >> >> Thanks, in advance, for any questions, comments or suggestions. >> >> Dan >> >> >> On 2/4/20 9:41 AM, Daniel D. Daugherty wrote: >>> Greetings, >>> >>> This project is no longer targeted to JDK14 so this is NOT an urgent >>> code >>> review request. >>> >>> I've extracted the following three fixes from the Async Monitor >>> Deflation >>> project code: >>> >>> ? ? JDK-8235931 add OM_CACHE_LINE_SIZE and use smaller size on >>> SPARCv9 and X64 >>> ? ? https://bugs.openjdk.java.net/browse/JDK-8235931 >>> >>> ? ? JDK-8236035 refactor ObjectMonitor::set_owner() and _owner field >>> setting >>> ? ? https://bugs.openjdk.java.net/browse/JDK-8236035 >>> >>> ? ? JDK-8235795 replace monitor list >>> mux{Acquire,Release}(&gListLock) with spin locks >>> ? ? https://bugs.openjdk.java.net/browse/JDK-8235795 >>> >>> Each of these has been reviewed separately and will be pushed to JDK15 >>> in the near future (possibly by the end of this week). Of course, there >>> were improvements during these review cycles and the purpose of this >>> e-mail is to provided updated webrevs for this fix >>> (CR9/v2.09/12-for-jdk14) >>> within the revised context provided by {8235931, 8236035, 8235795}. >>> >>> Main bug URL: >>> >>> ??? JDK-8153224 Monitor deflation prolong safepoints >>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>> >>> The project is currently baselined on jdk-14+34. >>> >>> Here's the full webrev URL for those folks that want to see all of the >>> current Async Monitor Deflation code along with {8235931, 8236035, >>> 8235795} >>> in one go (v2.09b full): >>> >>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/12-for-jdk14.v2.09b.full/ >>> >>> >>> Compare the open.patch file in 12-for-jdk14.v2.09.full and >>> 12-for-jdk14.v2.09b.full >>> using your favorite file comparison/merge tool to see how Async >>> Monitor Deflation >>> evolved due to {8235931, 8236035, 8235795}. >>> >>> Some folks might want to see just the Async Monitor Deflation code >>> on top of >>> {8235931, 8236035, 8235795} so here's a webrev for that (v2.09b inc): >>> >>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/12-for-jdk14.v2.09b.inc/ >>> >>> >>> These webrevs have gone thru several Mach5 Tier[1-8] runs along with >>> my usual stress testing and SPECjbb2015 testing and there aren't any >>> surprises relative to CR9/v2.09/12-for-jdk14. >>> >>> Thanks, in advance, for any questions, comments or suggestions. >>> >>> Dan >>> >>> >>> On 12/11/19 3:41 PM, Daniel D. Daugherty wrote: >>>> Greetings, >>>> >>>> I have made changes to the Async Monitor Deflation code in response to >>>> the CR8/v2.08/11-for-jdk14 code review cycle. Thanks to David H., >>>> Robbin >>>> and Erik O. for their comments! >>>> >>>> This project is no longer targeted to JDK14 so this is NOT an >>>> urgent code >>>> review request. The primary purpose of this webrev is simply to >>>> close the >>>> CR8/v2.08/11-for-jdk14 code review loop and to let folks see how I >>>> resolved >>>> the code review comments from that round. >>>> >>>> Most of the comments in the CR8/v2.08/11-for-jdk14 code review >>>> cycle were >>>> on the monitor list changes so I'm going to take a look at >>>> extracting those >>>> changes into a standalone patch. Switching from >>>> Thread::muxAcquire(&gListLock) >>>> and Thread::muxRelease(&gListLock) to finer grained internal spin >>>> locks needs >>>> to be thoroughly reviewed and the best way to do that is separately >>>> from the >>>> Async Monitor Deflation changes. Thanks to Coleen for suggesting >>>> doing this >>>> extraction earlier. >>>> >>>> I have attached the change list from CR8 to CR9 instead of putting >>>> it in >>>> the body of this email. I've also added a link to the >>>> CR8-to-CR9-changes >>>> file to the webrevs so it should be easy to find. >>>> >>>> Main bug URL: >>>> >>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>>> >>>> The project is currently baselined on jdk-14+26. >>>> >>>> Here's the full webrev URL for those folks that want to see all of the >>>> current Async Monitor Deflation code in one go (v2.09 full): >>>> >>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/12-for-jdk14.v2.09.full/ >>>> >>>> >>>> Some folks might want to see just what has changed since the last >>>> review >>>> cycle so here's a webrev for that (v2.09 inc): >>>> >>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/12-for-jdk14.v2.09.inc/ >>>> >>>> >>>> The OpenJDK wiki has NOT yet been updated for this round of changes: >>>> >>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>> >>>> The jdk-14+26 based v2.09 version of the patch has been thru Mach5 >>>> tier[1-7] >>>> testing on Oracle's usual set of platforms. Mach5 tier8 is still >>>> running. >>>> A slightly older version of the v2.09 patch has also been through >>>> my usual >>>> set of stress testing on Linux-X64 and macOSX with the addition of >>>> Robbin's >>>> "MoCrazy 1024" test running in parallel on Linux-X64 with the other >>>> tests in >>>> my lab. The "MoCrazy 1024" has been going for > 5 days and 6700+ >>>> iterations >>>> without any failures. >>>> >>>> I'm planning to do a SPECjbb2015 round on the >>>> CR9/v2.09/12-for-jdk14 bits. >>>> >>>> Thanks, in advance, for any questions, comments or suggestions. >>>> >>>> Dan >>>> >>>> >>>> On 11/4/19 4:03 PM, Daniel D. Daugherty wrote: >>>>> Greetings, >>>>> >>>>> I have made changes to the Async Monitor Deflation code in >>>>> response to >>>>> the CR7/v2.07/10-for-jdk14 code review cycle. Thanks to David H., >>>>> Robbin >>>>> and Erik O. for their comments! >>>>> >>>>> JDK14 Rampdown phase one is coming on Dec. 12, 2019 and the Async >>>>> Monitor >>>>> Deflation project needs to push before Nov. 12, 2019 in order to >>>>> allow >>>>> for sufficient bake time for such a big change. Nov. 12 is _next_ >>>>> Tuesday >>>>> so we have 8 days from today to finish this code review cycle and >>>>> push >>>>> this code for JDK14. >>>>> >>>>> Carsten and Roman! Time for you guys to chime in again on the code >>>>> reviews. >>>>> >>>>> I have attached the change list from CR7 to CR8 instead of putting >>>>> it in >>>>> the body of this email. I've also added a link to the >>>>> CR7-to-CR8-changes >>>>> file to the webrevs so it should be easy to find. >>>>> >>>>> Main bug URL: >>>>> >>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>> >>>>> The project is currently baselined on jdk-14+21. >>>>> >>>>> Here's the full webrev URL for those folks that want to see all of >>>>> the >>>>> current Async Monitor Deflation code in one go (v2.08 full): >>>>> >>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/11-for-jdk14.v2.08.full >>>>> >>>>> >>>>> Some folks might want to see just what has changed since the last >>>>> review >>>>> cycle so here's a webrev for that (v2.08 inc): >>>>> >>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/11-for-jdk14.v2.08.inc/ >>>>> >>>>> >>>>> The OpenJDK wiki did not need any changes for this round: >>>>> >>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>> >>>>> The jdk-14+21 based v2.08 version of the patch has been thru Mach5 >>>>> tier[1-8] >>>>> testing on Oracle's usual set of platforms. It has also been >>>>> through my usual >>>>> set of stress testing on Linux-X64, macOSX and Solaris-X64 with >>>>> the addition >>>>> of Robbin's "MoCrazy 1024" test running in parallel with the other >>>>> tests in >>>>> my lab. Some testing is still running, but so far there are no new >>>>> regressions. >>>>> >>>>> I have not yet done a SPECjbb2015 round on the >>>>> CR8/v2.08/11-for-jdk14 bits. >>>>> >>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>> >>>>> Dan >>>>> >>>>> >>>>> On 10/17/19 5:50 PM, Daniel D. Daugherty wrote: >>>>>> Greetings, >>>>>> >>>>>> The Async Monitor Deflation project is reaching the end game. I >>>>>> have no >>>>>> changes planned for the project at this time so all that is left >>>>>> is code >>>>>> review and any changes that results from those reviews. >>>>>> >>>>>> Carsten and Roman! Time for you guys to chime in again on the >>>>>> code reviews. >>>>>> >>>>>> I have attached the list of fixes from CR6 to CR7 instead of >>>>>> putting it >>>>>> in the main body of this email. >>>>>> >>>>>> Main bug URL: >>>>>> >>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>> >>>>>> The project is currently baselined on jdk-14+19. >>>>>> >>>>>> Here's the full webrev URL for those folks that want to see all >>>>>> of the >>>>>> current Async Monitor Deflation code in one go (v2.07 full): >>>>>> >>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/10-for-jdk14.v2.07.full >>>>>> >>>>>> >>>>>> Some folks might want to see just what has changed since the last >>>>>> review >>>>>> cycle so here's a webrev for that (v2.07 inc): >>>>>> >>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/10-for-jdk14.v2.07.inc/ >>>>>> >>>>>> >>>>>> The OpenJDK wiki has been updated to match the >>>>>> CR7/v2.07/10-for-jdk14 changes: >>>>>> >>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>> >>>>>> >>>>>> The jdk-14+18 based v2.07 version of the patch has been thru >>>>>> Mach5 tier[1-8] >>>>>> testing on Oracle's usual set of platforms. It has also been >>>>>> through my usual >>>>>> set of stress testing on Linux-X64, macOSX and Solaris-X64 with >>>>>> the addition >>>>>> of Robbin's "MoCrazy 1024" test running in parallel with the >>>>>> other tests in >>>>>> my lab. >>>>>> >>>>>> The jdk-14+19 based v2.07 version of the patch has been thru >>>>>> Mach5 tier[1-3] >>>>>> test on Oracle's usual set of platforms. Mach5 tier[4-8] are in >>>>>> process. >>>>>> >>>>>> I did another round of SPECjbb2015 testing in Oracle's Aurora >>>>>> Performance lab >>>>>> using using their tuned SPECjbb2015 Linux-X64 G1 configs: >>>>>> >>>>>> ??? - "base" is jdk-14+18 >>>>>> ??? - "v2.07" is the latest version and includes C2 >>>>>> inc_om_ref_count() support >>>>>> ????? on LP64 X64 and the new HandshakeAfterDeflateIdleMonitors >>>>>> option >>>>>> ? ? - "off" is with -XX:-AsyncDeflateIdleMonitors specified >>>>>> ? ? - "handshake" is with -XX:+HandshakeAfterDeflateIdleMonitors >>>>>> specified >>>>>> >>>>>> ???????? hbIR?????????? hbIR >>>>>> ??? (max attempted)? (settled)? max-jOPS? critical-jOPS runtime >>>>>> ??? ---------------? ---------? --------? ------------- ------- >>>>>> ?????????? 34282.00?? 30635.90? 28831.30?????? 20969.20 3841.30 base >>>>>> ?????????? 34282.00?? 30973.00? 29345.80?????? 21025.20 3964.10 >>>>>> v2.07 >>>>>> ?????????? 34282.00?? 31105.60? 29174.30?????? 21074.00 3931.30 >>>>>> v2.07_handshake >>>>>> ?????????? 34282.00?? 30789.70? 27151.60?????? 19839.10 3850.20 >>>>>> v2.07_off >>>>>> >>>>>> ??? - The Aurora Perf comparison tool reports: >>>>>> >>>>>> ??????? Comparison????????????? max-jOPS critical-jOPS >>>>>> ??????? ----------------------? -------------------- >>>>>> -------------------- >>>>>> ??????? base vs 2.07??????????? +1.78% (s, p=0.000) +0.27% (ns, >>>>>> p=0.790) >>>>>> ??????? base vs 2.07_handshake? +1.19% (s, p=0.007) +0.58% (ns, >>>>>> p=0.536) >>>>>> ??????? base vs 2.07_off??????? -5.83% (ns, p=0.394) -5.39% (ns, >>>>>> p=0.347) >>>>>> >>>>>> ??????? (s) - significant? (ns) - not-significant >>>>>> >>>>>> ??? - For historical comparison, the Aurora Perf comparision tool >>>>>> ??????? reported for v2.06 with a baseline of jdk-13+31: >>>>>> >>>>>> ??????? Comparison????????????? max-jOPS critical-jOPS >>>>>> ??????? ----------------------? -------------------- >>>>>> -------------------- >>>>>> ??????? base vs 2.06??????????? -0.32% (ns, p=0.345) +0.71% (ns, >>>>>> p=0.646) >>>>>> ??????? base vs 2.06_off??????? +0.49% (ns, p=0.292) -1.21% (ns, >>>>>> p=0.481) >>>>>> >>>>>> ??????? (s) - significant? (ns) - not-significant >>>>>> >>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>> >>>>>> Dan >>>>>> >>>>>> >>>>>> On 8/28/19 5:02 PM, Daniel D. Daugherty wrote: >>>>>>> Greetings, >>>>>>> >>>>>>> The Async Monitor Deflation project has rebased to JDK14 so it's >>>>>>> time >>>>>>> for our first code review in that new context!! >>>>>>> >>>>>>> I've been focused on changing the monitor list management code >>>>>>> to be >>>>>>> lock-free in order to make SPECjbb2015 happier. Of course with a >>>>>>> change >>>>>>> like that, it takes a while to chase down all the new and wonderful >>>>>>> races. At this point, I have the code back to the same stability >>>>>>> that >>>>>>> I had with CR5/v2.05/8-for-jdk13. >>>>>>> >>>>>>> To lay the ground work for this round of review, I pushed the >>>>>>> following >>>>>>> two fixes to jdk/jdk earlier today: >>>>>>> >>>>>>> ??? JDK-8230184 rename, whitespace, indent and comments changes >>>>>>> in preparation >>>>>>> ? ? ??????????? for lock free Monitor lists >>>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8230184 >>>>>>> >>>>>>> ??? JDK-8230317 serviceability/sa/ClhsdbPrintStatics.java fails >>>>>>> after 8230184 >>>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8230317 >>>>>>> >>>>>>> I have attached the list of fixes from CR5 to CR6 instead of >>>>>>> putting >>>>>>> in the main body of this email. >>>>>>> >>>>>>> Main bug URL: >>>>>>> >>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>> >>>>>>> The project is currently baselined on jdk-14+11 plus the fixes for >>>>>>> JDK-8230184 and JDK-8230317. >>>>>>> >>>>>>> Here's the full webrev URL for those folks that want to see all >>>>>>> of the >>>>>>> current Async Monitor Deflation code in one go (v2.06 full): >>>>>>> >>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.06.full/ >>>>>>> >>>>>>> >>>>>>> >>>>>>> The primary focus of this review cycle is on the lock-free >>>>>>> Monitor List >>>>>>> management changes so here's a webrev for just that patch (v2.06c): >>>>>>> >>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.06c.inc/ >>>>>>> >>>>>>> >>>>>>> The secondary focus of this review cycle is on the bug fixes >>>>>>> that have >>>>>>> been made since CR5/v2.05/8-for-jdk13 so here's a webrev for >>>>>>> just that >>>>>>> patch (v2.06b): >>>>>>> >>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.06b.inc/ >>>>>>> >>>>>>> >>>>>>> The third and final bucket for this review cycle is the rename, >>>>>>> whitespace, >>>>>>> indent and comments changes made in preparation for lock free >>>>>>> Monitor list >>>>>>> management. Almost all of that was extracted into JDK-8230184 >>>>>>> for the >>>>>>> baseline so this bucket now has just a few comment changes >>>>>>> relative to >>>>>>> CR5/v2.05/8-for-jdk13. Here's a webrev for the remainder (v2.06a): >>>>>>> >>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.06a.inc/ >>>>>>> >>>>>>> >>>>>>> >>>>>>> Some folks might want to see just what has changed since the >>>>>>> last review >>>>>>> cycle so here's a webrev for that (v2.06 inc): >>>>>>> >>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.06.inc/ >>>>>>> >>>>>>> >>>>>>> >>>>>>> Last, but not least, some folks might want to see the code >>>>>>> before the >>>>>>> addition of lock-free Monitor List management so here's a webrev >>>>>>> for >>>>>>> that (v2.00 -> v2.05): >>>>>>> >>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/9-for-jdk14.v2.05.inc/ >>>>>>> >>>>>>> >>>>>>> The OpenJDK wiki will need minor updates to match the CR6 changes: >>>>>>> >>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>> >>>>>>> >>>>>>> but that should only be changes to describe per-thread list >>>>>>> async monitor >>>>>>> deflation being done by the ServiceThread. >>>>>>> >>>>>>> (I did update the OpenJDK wiki for the CR5 changes back on >>>>>>> 2019.08.14) >>>>>>> >>>>>>> This version of the patch has been thru Mach5 tier[1-8] testing on >>>>>>> Oracle's usual set of platforms. It has also been through my >>>>>>> usual set >>>>>>> of stress testing on Linux-X64, macOSX and Solaris-X64. >>>>>>> >>>>>>> I did a bunch of SPECjbb2015 testing in Oracle's Aurora >>>>>>> Performance lab >>>>>>> using using their tuned SPECjbb2015 Linux-X64 G1 configs. This >>>>>>> was using >>>>>>> this patch baselined on jdk-13+31 (for stability): >>>>>>> >>>>>>> ????????? hbIR?????????? hbIR >>>>>>> ???? (max attempted)? (settled)? max-jOPS critical-jOPS runtime >>>>>>> ???? ---------------? ---------? -------- ------------- ------- >>>>>>> ??????????? 34282.00?? 28837.20? 27905.20 19817.40 3658.10 base >>>>>>> ??????????? 34965.70?? 29798.80? 27814.90 19959.00 3514.60 v2.06d >>>>>>> ??????????? 34282.00?? 29100.70? 28042.50 19577.00 3701.90 >>>>>>> v2.06d_off >>>>>>> ??????????? 34282.00?? 29218.50? 27562.80 19397.30 3657.60 >>>>>>> v2.06d_ocache >>>>>>> ??????????? 34965.70?? 29838.30? 26512.40 19170.60 3569.90 v2.05 >>>>>>> ??????????? 34282.00?? 28926.10? 27734.00 19835.10 3588.40 >>>>>>> v2.05_off >>>>>>> >>>>>>> The "off" configs are with -XX:-AsyncDeflateIdleMonitors >>>>>>> specified and >>>>>>> the "ocache" config is with 128 byte cache line sizes instead of >>>>>>> 64 byte >>>>>>> cache lines sizes. "v2.06d" is the last set of changes that I >>>>>>> made before >>>>>>> those changes were distributed into the "v2.06a", "v2.06b" and >>>>>>> "v2.06c" >>>>>>> buckets for this review recycle. >>>>>>> >>>>>>> >>>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>>> >>>>>>> Dan >>>>>>> >>>>>>> >>>>>>> On 7/11/19 3:49 PM, Daniel D. Daugherty wrote: >>>>>>>> Greetings, >>>>>>>> >>>>>>>> I've been focused on chasing down and fixing the rare test >>>>>>>> failures >>>>>>>> that only pop up rarely. So this round is primarily fixes for >>>>>>>> races >>>>>>>> with a few additional fixes that came from Karen's review of CR4. >>>>>>>> Thanks Karen! >>>>>>>> >>>>>>>> I have attached the list of fixes from CR4 to CR5 instead of >>>>>>>> putting >>>>>>>> in the main body of this email. >>>>>>>> >>>>>>>> Main bug URL: >>>>>>>> >>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>> >>>>>>>> The project is currently baselined on jdk-13+29. This will >>>>>>>> likely be >>>>>>>> the last JDK13 baseline for this project and I'll roll to the >>>>>>>> JDK14 >>>>>>>> (jdk/jdk) repo soon... >>>>>>>> >>>>>>>> Here's the full webrev URL: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/8-for-jdk13.full/ >>>>>>>> >>>>>>>> >>>>>>>> Here's the incremental webrev URL: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/8-for-jdk13.inc/ >>>>>>>> >>>>>>>> I have not yet checked the OpenJDK wiki to see if it needs any >>>>>>>> updates >>>>>>>> to match the CR5 changes: >>>>>>>> >>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>> >>>>>>>> >>>>>>>> (I did update the OpenJDK wiki for the CR4 changes back on >>>>>>>> 2019.06.26) >>>>>>>> >>>>>>>> This version of the patch has been thru Mach5 tier[1-3] testing on >>>>>>>> Oracle's usual set of platforms. Mach5 tier[4-6] is running now >>>>>>>> and >>>>>>>> Mach5 tier[78] will follow. I'll kick off the usual stress testing >>>>>>>> on Linux-X64, macOSX and Solaris-X64 as those machines become >>>>>>>> available. >>>>>>>> Since I haven't made any performance changes in this round, >>>>>>>> I'll only >>>>>>>> be running SPECjbb2015 to gather the latest monitorinflation logs. >>>>>>>> >>>>>>>> Next up: >>>>>>>> >>>>>>>> - We're still seeing 4-5% lower performance with SPECjbb2015 on >>>>>>>> ? Linux-X64 and we've determined that some of that comes from >>>>>>>> ? contention on the gListLock. So I'm going to investigate >>>>>>>> removing >>>>>>>> ? the gListLock. Yes, another lock free set of changes is coming! >>>>>>>> - Of course, going lock free often causes new races and new >>>>>>>> failures >>>>>>>> ? so that's a good reason for make those changes isolated in their >>>>>>>> ? own round (and not holding up CR5/v2.05/8-for-jdk13 anymore). >>>>>>>> - I finally have a potential fix for the Win* failure with >>>>>>>> gc/g1/humongousObjects/TestHumongousClassLoader.java >>>>>>>> ? but I haven't run it through Mach5 yet so it'll be in the >>>>>>>> next round. >>>>>>>> - Some RTM tests were recently re-enabled in Mach5 and I'm >>>>>>>> seeing some >>>>>>>> ? monitor related failures there. I suspect that I need to go >>>>>>>> take a >>>>>>>> ? look at the C2 RTM macro assembler code and look for things >>>>>>>> that might >>>>>>>> ? conflict if Async Monitor Deflation. If you're interested in >>>>>>>> that kind >>>>>>>> ? of issue, then see the macroAssembler_x86.cpp sanity check >>>>>>>> that I >>>>>>>> ? added in this round! >>>>>>>> >>>>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>>>> >>>>>>>> Dan >>>>>>>> >>>>>>>> >>>>>>>> On 5/26/19 8:30 PM, Daniel D. Daugherty wrote: >>>>>>>>> Greetings, >>>>>>>>> >>>>>>>>> I have a fix for an issue that came up during performance >>>>>>>>> testing. >>>>>>>>> Many thanks to Robbin for diagnosing the issue in his SPECjbb2015 >>>>>>>>> experiments. >>>>>>>>> >>>>>>>>> Here's the list of changes from CR3 to CR4. The list is a bit >>>>>>>>> verbose due to the complexity of the issue, but the changes >>>>>>>>> themselves are not that big. >>>>>>>>> >>>>>>>>> Functional: >>>>>>>>> ? - Change SafepointSynchronize::is_cleanup_needed() from calling >>>>>>>>> ??? ObjectSynchronizer::is_cleanup_needed() to calling >>>>>>>>> ObjectSynchronizer::is_safepoint_deflation_needed(): >>>>>>>>> ??? - is_safepoint_deflation_needed() returns the result of >>>>>>>>> ????? monitors_used_above_threshold() for safepoint based >>>>>>>>> ????? monitor deflation (!AsyncDeflateIdleMonitors). >>>>>>>>> ??? - For AsyncDeflateIdleMonitors, it only returns true if >>>>>>>>> ????? there is a special deflation request, e.g., System.gc() >>>>>>>>> ????? - This solves a bug where there are a bunch of Cleanup >>>>>>>>> ??????? safepoints that simply request async deflation which >>>>>>>>> ??????? keeps the async JavaThreads from making progress on >>>>>>>>> ??????? their async deflation work. >>>>>>>>> ? - Add AsyncDeflationInterval diagnostic option. Description: >>>>>>>>> ????? Async deflate idle monitors every so many milliseconds when >>>>>>>>> ????? MonitorUsedDeflationThreshold is exceeded (0 is off). >>>>>>>>> ? - Replace ObjectSynchronizer::gOmShouldDeflateIdleMonitors() >>>>>>>>> with >>>>>>>>> ObjectSynchronizer::is_async_deflation_needed(): >>>>>>>>> ??? - is_async_deflation_needed() returns true when >>>>>>>>> ????? is_async_cleanup_requested() is true or when >>>>>>>>> ????? monitors_used_above_threshold() is true (but no more >>>>>>>>> often than >>>>>>>>> ????? AsyncDeflationInterval). >>>>>>>>> ??? - if AsyncDeflateIdleMonitors Service_lock->wait() now >>>>>>>>> waits for >>>>>>>>> ????? at most GuaranteedSafepointInterval millis: >>>>>>>>> ????? - This allows is_async_deflation_needed() to be checked at >>>>>>>>> ??????? the same interval as GuaranteedSafepointInterval. >>>>>>>>> ??????? (default is 1000 millis/1 second) >>>>>>>>> ????? - Once is_async_deflation_needed() has returned true, it >>>>>>>>> ??????? generally cannot return true for AsyncDeflationInterval. >>>>>>>>> ??????? This is to prevent async deflation from swamping the >>>>>>>>> ??????? ServiceThread. >>>>>>>>> ? - The ServiceThread still handles async deflation of the global >>>>>>>>> ??? in-use list and now it also marks JavaThreads for async >>>>>>>>> deflation >>>>>>>>> ??? of their in-use lists. >>>>>>>>> ??? - The ServiceThread will check for async deflation work every >>>>>>>>> ????? GuaranteedSafepointInterval. >>>>>>>>> ??? - A safepoint can still cause the ServiceThread to check for >>>>>>>>> ????? async deflation work via is_async_deflation_requested. >>>>>>>>> ? - Refactor code from ObjectSynchronizer::is_cleanup_needed() >>>>>>>>> into >>>>>>>>> ??? monitors_used_above_threshold() and remove >>>>>>>>> is_cleanup_needed(). >>>>>>>>> ? - In addition to System.gc(), the VM_Exit VM op and the final >>>>>>>>> ??? VMThread safepoint now set the is_special_deflation_requested >>>>>>>>> ??? flag to reduce the in-use monitor population that is >>>>>>>>> reported by >>>>>>>>> ObjectSynchronizer::log_in_use_monitor_details() at VM exit. >>>>>>>>> >>>>>>>>> Test update: >>>>>>>>> ? - test/hotspot/gtest/oops/test_markOop.cpp is updated to >>>>>>>>> work with >>>>>>>>> ??? AsyncDeflateIdleMonitors. >>>>>>>>> >>>>>>>>> Collateral: >>>>>>>>> ? - Add/clarify/update some logging messages. >>>>>>>>> >>>>>>>>> Cleanup: >>>>>>>>> ? - Updated comments based on Karen's code review. >>>>>>>>> ? - Change 'special cleanup' -> 'special deflation' and >>>>>>>>> ??? 'async cleanup' -> 'async deflation'. >>>>>>>>> ??? - comment and function name changes >>>>>>>>> ? - Clarify MonitorUsedDeflationThreshold description; >>>>>>>>> >>>>>>>>> >>>>>>>>> Main bug URL: >>>>>>>>> >>>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>>> >>>>>>>>> The project is currently baselined on jdk-13+22. >>>>>>>>> >>>>>>>>> Here's the full webrev URL: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/7-for-jdk13.full/ >>>>>>>>> >>>>>>>>> >>>>>>>>> Here's the incremental webrev URL: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/7-for-jdk13.inc/ >>>>>>>>> >>>>>>>>> >>>>>>>>> I have not updated the OpenJDK wiki to reflect the CR4 changes: >>>>>>>>> >>>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>>> >>>>>>>>> >>>>>>>>> The wiki doesn't say a whole lot about the async deflation >>>>>>>>> invocation >>>>>>>>> mechanism so I have to figure out how to add that content. >>>>>>>>> >>>>>>>>> This version of the patch has been thru Mach5 tier[1-8] >>>>>>>>> testing on >>>>>>>>> Oracle's usual set of platforms. My Solaris-X64 stress kit run is >>>>>>>>> running now. Kitchensink8H on product, fastdebug, and >>>>>>>>> slowdebug bits >>>>>>>>> are running on Linux-X64, MacOSX and Solaris-X64. I still have >>>>>>>>> to run >>>>>>>>> my stress kit on Linux-X64. I still have to run the SPECjbb2015 >>>>>>>>> baseline and CR4 runs on Linux-X64, MacOSX and Solaris-X64. >>>>>>>>> >>>>>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>>>>> >>>>>>>>> Dan >>>>>>>>> >>>>>>>>> On 5/6/19 11:52 AM, Daniel D. Daugherty wrote: >>>>>>>>>> Greetings, >>>>>>>>>> >>>>>>>>>> I had some discussions with Karen about a race that was in the >>>>>>>>>> ObjectMonitor::enter() code in CR2/v2.02/5-for-jdk13. This >>>>>>>>>> race was >>>>>>>>>> theoretical and I had no test failures due to it. The fix is >>>>>>>>>> pretty >>>>>>>>>> simple: remove the special case code for async deflation in the >>>>>>>>>> ObjectMonitor::enter() function and rely solely on the ref_count >>>>>>>>>> for ObjectMonitor::enter() protection. >>>>>>>>>> >>>>>>>>>> During those discussions Karen also floated the idea of using >>>>>>>>>> the >>>>>>>>>> ref_count field instead of the contentions field for the Async >>>>>>>>>> Monitor Deflation protocol. I decided to go ahead and code up >>>>>>>>>> that >>>>>>>>>> change and I have run it through the usual stress and Mach5 >>>>>>>>>> testing >>>>>>>>>> with no issues. It's also known as v2.03 (for those for with the >>>>>>>>>> patches) and as webrev/6-for-jdk13 (for those with webrev URLs). >>>>>>>>>> Sorry for all the names... >>>>>>>>>> >>>>>>>>>> Main bug URL: >>>>>>>>>> >>>>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>>>> >>>>>>>>>> The project is currently baselined on jdk-13+18. >>>>>>>>>> >>>>>>>>>> Here's the full webrev URL: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/6-for-jdk13.full/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Here's the incremental webrev URL: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/6-for-jdk13.inc/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> I have also updated the OpenJDK wiki to reflect the CR3 changes: >>>>>>>>>> >>>>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> This version of the patch has been thru Mach5 tier[1-8] >>>>>>>>>> testing on >>>>>>>>>> Oracle's usual set of platforms. My Solaris-X64 stress kit >>>>>>>>>> run had >>>>>>>>>> no issues. Kitchensink8H on product, fastdebug, and slowdebug >>>>>>>>>> bits >>>>>>>>>> had no failures on Linux-X64; MacOSX fastdebug and slowdebug and >>>>>>>>>> Solaris-X64 release had the usual "Too large time diff" >>>>>>>>>> complaints. >>>>>>>>>> 12 hour Inflate2 runs on product, fastdebug and slowdebug >>>>>>>>>> bits on >>>>>>>>>> Linux-X64, MacOSX and Solaris-X64 had no failures. My Linux-X64 >>>>>>>>>> stress kit is running right now. >>>>>>>>>> >>>>>>>>>> I've done the SPECjbb2015 baseline and CR3 runs. I need to >>>>>>>>>> gather >>>>>>>>>> the results and analyze them. >>>>>>>>>> >>>>>>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>>>>>> >>>>>>>>>> Dan >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 4/25/19 12:38 PM, Daniel D. Daugherty wrote: >>>>>>>>>>> Greetings, >>>>>>>>>>> >>>>>>>>>>> I have a small but important bug fix for the Async Monitor >>>>>>>>>>> Deflation >>>>>>>>>>> project ready to go. It's also known as v2.02 (for those for >>>>>>>>>>> with the >>>>>>>>>>> patches) and as webrev/5-for-jdk13 (for those with webrev >>>>>>>>>>> URLs). Sorry >>>>>>>>>>> for all the names... >>>>>>>>>>> >>>>>>>>>>> JDK-8222295 was pushed to jdk/jdk two days ago so that >>>>>>>>>>> baseline patch >>>>>>>>>>> is out of our hair. >>>>>>>>>>> >>>>>>>>>>> Main bug URL: >>>>>>>>>>> >>>>>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>>>>> >>>>>>>>>>> The project is currently baselined on jdk-13+17. >>>>>>>>>>> >>>>>>>>>>> Here's the full webrev URL: >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/5-for-jdk13.full/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Here's the incremental webrev URL (JDK-8153224): >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/5-for-jdk13.inc/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I still have to update the OpenJDK wiki to reflect the CR2 >>>>>>>>>>> changes: >>>>>>>>>>> >>>>>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> This version of the patch has been thru Mach5 tier[1-6] >>>>>>>>>>> testing on >>>>>>>>>>> Oracle's usual set of platforms. Mach5 tier[7-8] is running >>>>>>>>>>> now. >>>>>>>>>>> My stress kit is running on Solaris-X64 now. Kitchensink8H >>>>>>>>>>> is running >>>>>>>>>>> now on product, fastdebug, and slowdebug bits on Linux-X64, >>>>>>>>>>> MacOSX >>>>>>>>>>> and Solaris-X64. 12 hour Inflate2 runs are running now on >>>>>>>>>>> product, >>>>>>>>>>> fastdebug and slowdebug bits on Linux-X64, MacOSX and >>>>>>>>>>> Solaris-X64. >>>>>>>>>>> I'll start my my stress kit on Linux-X64 sometime on Sunday >>>>>>>>>>> (after >>>>>>>>>>> my jdk-13+18 stress run is done). >>>>>>>>>>> >>>>>>>>>>> I'll do SPECjbb2015 baseline and CR2 runs after all the stress >>>>>>>>>>> testing is done. >>>>>>>>>>> >>>>>>>>>>> Thanks, in advance, for any questions, comments or suggestions. >>>>>>>>>>> >>>>>>>>>>> Dan >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 4/19/19 11:58 AM, Daniel D. Daugherty wrote: >>>>>>>>>>>> Greetings, >>>>>>>>>>>> >>>>>>>>>>>> I finally have CR1 for the Async Monitor Deflation project >>>>>>>>>>>> ready to >>>>>>>>>>>> go. It's also known as v2.01 (for those for with the >>>>>>>>>>>> patches) and as >>>>>>>>>>>> webrev/4-for-jdk13 (for those with webrev URLs). Sorry for >>>>>>>>>>>> all the >>>>>>>>>>>> names... >>>>>>>>>>>> >>>>>>>>>>>> Main bug URL: >>>>>>>>>>>> >>>>>>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>>>>>> >>>>>>>>>>>> Baseline bug fixes URL: >>>>>>>>>>>> >>>>>>>>>>>> ??? JDK-8222295 more baseline cleanups from Async Monitor >>>>>>>>>>>> Deflation project >>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8222295 >>>>>>>>>>>> >>>>>>>>>>>> The project is currently baselined on jdk-13+15. >>>>>>>>>>>> >>>>>>>>>>>> Here's the webrev for the latest baseline changes >>>>>>>>>>>> (JDK-8222295): >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/4-for-jdk13.8222295 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Here's the full webrev URL (JDK-8153224 only): >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/4-for-jdk13.full/ >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Here's the incremental webrev URL (JDK-8153224): >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/4-for-jdk13.inc/ >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> So I'm looking for reviews for both JDK-8222295 and the >>>>>>>>>>>> latest version >>>>>>>>>>>> of JDK-8153224... >>>>>>>>>>>> >>>>>>>>>>>> I still have to update the OpenJDK wiki to reflect the CR >>>>>>>>>>>> changes: >>>>>>>>>>>> >>>>>>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> This version of the patch has been thru Mach5 tier[1-3] >>>>>>>>>>>> testing on >>>>>>>>>>>> Oracle's usual set of platforms. Mach5 tier[4-6] is running >>>>>>>>>>>> now and >>>>>>>>>>>> Mach5 tier[78] will be run later today. My stress kit on >>>>>>>>>>>> Solaris-X64 >>>>>>>>>>>> is running now. Linux-X64 stress testing will start on >>>>>>>>>>>> Sunday. I'm >>>>>>>>>>>> planning to do Kitchensink runs, SPECjbb2015 runs and my >>>>>>>>>>>> monitor >>>>>>>>>>>> inflation stress tests on Linux-X64, MacOSX and Solaris-X64. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, in advance, for any questions, comments or >>>>>>>>>>>> suggestions. >>>>>>>>>>>> >>>>>>>>>>>> Dan >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 3/24/19 9:57 AM, Daniel D. Daugherty wrote: >>>>>>>>>>>>> Greetings, >>>>>>>>>>>>> >>>>>>>>>>>>> Welcome to the OpenJDK review thread for my port of >>>>>>>>>>>>> Carsten's work on: >>>>>>>>>>>>> >>>>>>>>>>>>> ??? JDK-8153224 Monitor deflation prolong safepoints >>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8153224 >>>>>>>>>>>>> >>>>>>>>>>>>> Here's a link to the OpenJDK wiki that describes my port: >>>>>>>>>>>>> >>>>>>>>>>>>> https://wiki.openjdk.java.net/display/HotSpot/Async+Monitor+Deflation >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Here's the webrev URL: >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~dcubed/8153224-webrev/3-for-jdk13/ >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Here's a link to Carsten's original webrev: >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~cvarming/monitor_deflate_conc/0/ >>>>>>>>>>>>> >>>>>>>>>>>>> Earlier versions of this patch have been through several >>>>>>>>>>>>> rounds of >>>>>>>>>>>>> preliminary review. Many thanks to Carsten, Coleen, >>>>>>>>>>>>> Robbin, and >>>>>>>>>>>>> Roman for their preliminary code review comments. A very >>>>>>>>>>>>> special >>>>>>>>>>>>> thanks to Robbin and Roman for building and testing the >>>>>>>>>>>>> patch in >>>>>>>>>>>>> their own environments (including specJBB2015). >>>>>>>>>>>>> >>>>>>>>>>>>> This version of the patch has been thru Mach5 tier[1-8] >>>>>>>>>>>>> testing on >>>>>>>>>>>>> Oracle's usual set of platforms. Earlier versions have >>>>>>>>>>>>> been run >>>>>>>>>>>>> through my stress kit on my Linux-X64 and Solaris-X64 servers >>>>>>>>>>>>> (product, fastdebug, slowdebug).Earlier versions have run >>>>>>>>>>>>> Kitchensink >>>>>>>>>>>>> for 12 hours on MacOSX, Linux-X64 and Solaris-X64 >>>>>>>>>>>>> (product, fastdebug >>>>>>>>>>>>> and slowdebug). Earlier versions have run my monitor >>>>>>>>>>>>> inflation stress >>>>>>>>>>>>> tests for 12 hours on MacOSX, Linux-X64 and Solaris-X64 >>>>>>>>>>>>> (product, >>>>>>>>>>>>> fastdebug and slowdebug). >>>>>>>>>>>>> >>>>>>>>>>>>> All of the testing done on earlier versions will be redone >>>>>>>>>>>>> on the >>>>>>>>>>>>> latest version of the patch. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, in advance, for any questions, comments or >>>>>>>>>>>>> suggestions. >>>>>>>>>>>>> >>>>>>>>>>>>> Dan >>>>>>>>>>>>> >>>>>>>>>>>>> P.S. >>>>>>>>>>>>> One subtest in >>>>>>>>>>>>> gc/g1/humongousObjects/TestHumongousClassLoader.java >>>>>>>>>>>>> is currently failing in -Xcomp mode on Win* only. I've >>>>>>>>>>>>> been trying >>>>>>>>>>>>> to characterize/analyze this failure for more than a week >>>>>>>>>>>>> now. At >>>>>>>>>>>>> this point I'm convinced that Async Monitor Deflation is >>>>>>>>>>>>> aggravating >>>>>>>>>>>>> an existing bug. However, I plan to have a better handle >>>>>>>>>>>>> on that >>>>>>>>>>>>> failure before these bits are pushed to the jdk/jdk repo. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> From aph at redhat.com Mon Mar 9 15:42:12 2020 From: aph at redhat.com (Andrew Haley) Date: Mon, 9 Mar 2020 15:42:12 +0000 Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is inconsistent In-Reply-To: References: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com> <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com> <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com> Message-ID: <1c40a7d1-fa36-b5d2-6072-eaa243cc9397@redhat.com> On 3/9/20 2:46 PM, Doerr, Martin wrote: > Sorry, my comment about log2_int + log2_jint was not precise. > > The comment > "If x < 0, the function returns 31 on a 32-bit machine and 63 on a 64-bit machine." > belongs to log2_intptr, Indeed. > but it also applies to log2_int + log2_jint which is not desirable IMHO. > I guess using zero extend would be better for these 2 functions: > "If x < 0, the function returns 31 on all machines." As far as I can see, log2_jint() does the right thing: it returns 31 for 0x8000_0000-0xffff_ffff, and it does this with no undefined behaviour. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From martin.doerr at sap.com Mon Mar 9 15:56:33 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 9 Mar 2020 15:56:33 +0000 Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is inconsistent In-Reply-To: <1c40a7d1-fa36-b5d2-6072-eaa243cc9397@redhat.com> References: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com> <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com> <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com> <1c40a7d1-fa36-b5d2-6072-eaa243cc9397@redhat.com> Message-ID: Hi Andrew, it doesn't run into the UB case, but log2_jint(x) returns 63 for negative x. I'd treat this as bug. A jint has 32 bit and log2 should never return 63. Best regards, Martin > -----Original Message----- > From: Andrew Haley > Sent: Montag, 9. M?rz 2020 16:42 > To: Doerr, Martin ; Stefan Karlsson > ; John Rose > Cc: hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is > inconsistent > > On 3/9/20 2:46 PM, Doerr, Martin wrote: > > Sorry, my comment about log2_int + log2_jint was not precise. > > > > The comment > > "If x < 0, the function returns 31 on a 32-bit machine and 63 on a 64-bit > machine." > > belongs to log2_intptr, > > Indeed. > > > but it also applies to log2_int + log2_jint which is not desirable IMHO. > > I guess using zero extend would be better for these 2 functions: > > "If x < 0, the function returns 31 on all machines." > > As far as I can see, log2_jint() does the right thing: it returns > 31 for 0x8000_0000-0xffff_ffff, and it does this with no undefined > behaviour. > > -- > Andrew Haley (he/him) > Java Platform Lead Engineer > Red Hat UK Ltd. > https://keybase.io/andrewhaley > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From mandy.chung at oracle.com Mon Mar 9 16:43:59 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 9 Mar 2020 09:43:59 -0700 Subject: RFR(XS) 8240548 [TESTBUG] CDS NoClassToArchive.java fails with Graal In-Reply-To: <9fd602d1-0e2c-5920-d677-8f2d0a1616be@oracle.com> References: <9fd602d1-0e2c-5920-d677-8f2d0a1616be@oracle.com> Message-ID: Hi Ioi, On 3/8/20 9:52 PM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8240548 > http://cr.openjdk.java.net/~iklam/jdk15/8240548-NoClassToArchive.graal.v01/ > > 65 if (output.getStdout().contains("jrt:/") || 66 output.getStdout().contains("unsafe anonymous") || 67 output.getStdout().contains("Unsafe anonymous")) { Do you mean to filter hidden classes? there are duplicated checks for unsafe anonymous. LMF will be defining hidden classes instead of VM anonymous class by JEP 371. Mandy > This test should not be executed with graal. > > This test started failing in valhalla/nestmate because the string > "unsafe anonymous" is no longer printed by -Xlog:class+load=trace. > However, > the underlying problem is unrelated to nestmate. So I will fix the > test in > the mainline jdk/jdk and it can be synced down to nestmate. > > The test assumes that when "java -version" is executed, only a very > limited > number of classes are loaded, and all of those are loaded from the > default > shared archive. > > However, when graal is used as the JIT, many extra classes are loaded > during VM start-up. Some of those are loaded dynamically from jrt:/. > Some classes are also defined by LambdaMetafactory. This causes > complexity > that cannot be easily handled by this test. > > The VM code covered by this test can be sufficiently tested with > C1/C2. So > there's no need to bend over backwards to run this test with graal. > > Thanks > - Ioi From jianglizhou at google.com Mon Mar 9 16:44:36 2020 From: jianglizhou at google.com (Jiangli Zhou) Date: Mon, 9 Mar 2020 09:44:36 -0700 Subject: Need Information for memory sharing with AppCDS In-Reply-To: <4c513dcc-ed4e-43ec-a607-764ded395058.shiyue.xw@alibaba-inc.com> References: <4c513dcc-ed4e-43ec-a607-764ded395058.shiyue.xw@alibaba-inc.com> Message-ID: Hi, On Sun, Mar 8, 2020 at 8:51 PM ??(??) wrote: > > Hi, > > We am trying to take advantage of the characteristics of memory sharing with AppCDS, and we expected that the > JVM internal packages and common packages can be shared with AppCDS. > > In the user scenario, there are some common base packages for different Java processes with "-cp common_path" > Process A: -cp common_path:pathA > Process B: -cp common_path:pathA:pathB > Process C: -cp common_path:pathC > > I check the code in JDK11, and it uses SharedPathsMiscInfo to do the consistency check for class path. > The below usage can work in JDK11 > Dump stage: > -cp common_path > > Runtime stage: > -cp common_path:new_path > > But I find that SharedPathsMiscInfo has been removed in the latest JDK code. > > I need the information: > 1. the new flow to do the consistency check. The check now uses the shared classpath table only, FileMapInfo::validate_shared_path_table(). SharedPathsMiscInfo was a redundant data structure, henenced removed. > 2. constraint condition about the memory sharing among the different Java processes. Memory sharing can only be achieved when the same archive file is memory mapped in different JVM processes running the same JVM binaries. What I've found was that in some cloud environments, different copies of the same shared archive file might be used for different containers and therefore disabled the sharing. Oftentimes, that's intentional. Best, Jiangli > > Is there any user guide to use the memory sharing feature for AppCDS? > > Thanks, > Wei Xiang From jianglizhou at google.com Mon Mar 9 16:50:34 2020 From: jianglizhou at google.com (Jiangli Zhou) Date: Mon, 9 Mar 2020 09:50:34 -0700 Subject: Need Information for memory sharing with AppCDS In-Reply-To: <7d348b97-161b-43d2-bd7b-896a50287a57.shiyue.xw@alibaba-inc.com> References: <4c513dcc-ed4e-43ec-a607-764ded395058.shiyue.xw@alibaba-inc.com> <86cccf4b-e497-bd03-155a-c51fbcc9ae77@oracle.com> <7d348b97-161b-43d2-bd7b-896a50287a57.shiyue.xw@alibaba-inc.com> Message-ID: Hi, On Sun, Mar 8, 2020 at 11:56 PM ??(??) wrote: > > Hi Ioi, > > Thanks a lot for your information. > I have the question about the usage in the document: > For common class path prefix: > --------------------------------------- > Run the Hello and Hi applications with the same shared archive: > java -XX:SharedArchiveFile=common.jsa -cp common.jar:hello.jar:hi.jar Hello > java -XX:SharedArchiveFile=common.jsa -cp common.jar:hello.jar:hi.jar Hi > > If a class with name ABC is both in hello.jar and hi.jar, the content is different. > With the class path -cp common.jar:hello.jar:hi.jar, the second run "java .. Hi" can't load the correct class ABC in hi.jar. > For the above usage, I'd recommend to only archive the classes from common.jar to create a common.jsa. The common.jsa can be used for both applications for sharing. As described in the earlier email, the follow two conditions must be true to achieve sharing: - Both processes must be running with the same JVM binary - Both processes must be memory mapping using the same copy of common.jsa. If each container uses a different copy of the common.jsa, then there is no memory sharing. > Is there any solution for this usage? I'd recommend to not include ABC in the common.jsa in this case. Hope that helps. Best, Jiangli > > Thanks, > Wei Xiang > ------------------------------------------------------------------ > ????Ioi Lam > ?????2020?3?9?(???) 13:03 > ????hotspot-runtime-dev > ????Re: Need Information for memory sharing with AppCDS > > On 3/8/20 8:51 PM, ??(??) wrote: > > Hi, > > > > We am trying to take advantage of the characteristics of memory sharing with AppCDS, and we expected that the > > JVM internal packages and common packages can be shared with AppCDS. > > > > In the user scenario, there are some common base packages for different Java processes with "-cp common_path" > > Process A: -cp common_path:pathA > > Process B: -cp common_path:pathA:pathB > > Process C: -cp common_path:pathC > > > > I check the code in JDK11, and it uses SharedPathsMiscInfo to do the consistency check for class path. > > The below usage can work in JDK11 > > Dump stage: > > -cp common_path > > > > Runtime stage: > > -cp common_path:new_path > > > > But I find that SharedPathsMiscInfo has been removed in the latest JDK code. > > > > I need the information: > > 1. the new flow to do the consistency check. > > 2. constraint condition about the memory sharing among the different Java processes. > > > > Is there any user guide to use the memory sharing feature for AppCDS? > > Hi Wei, > > The topic of sharing the same CDS archive between different apps that > have the same classpath prefix can be found here: > > https://docs.oracle.com/en/java/javase/13/docs/specs/man/java.html#application-class-data-sharing > > See the section "Sharing a Shared Archive Across Multiple Application > Processes". > > SharedPathsMiscInfo was removed in this RFE in JDK 14: > https://bugs.openjdk.java.net/browse/JDK-8227370 > > However, we still enforce the same set of consistency checks as previous > JDK versions. > > Please let me know if you have further questions. > > Thanks > - Ioi > > > Thanks, > > Wei Xiang > From mandy.chung at oracle.com Mon Mar 9 23:33:23 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 9 Mar 2020 16:33:23 -0700 Subject: Review Request JDK-8228336: Refactor native library loading implementation Message-ID: <00d2029d-ae16-6a9b-5942-cd67aba7c141@oracle.com> This patch refactors the native library loading implementation out of ClassLoader and move them to jdk.internal.loader package. This introduces a new NativeLibraries abstraction which loads and registers the loaded native libraries.? Previously it was maintained in a nativeLibrary map in each ClassLoader instance and one systemNativeLibraries for the null loader case.? With this change, each ClassLoader and BootLoader have its own NativeLibraries instead. NativeLibraries.java and NativeLibraries.c are mostly refactored from the existing code.? Only minor cleanups are applied. This refactoring enables Panama LookupLibrary to further experiment and allow loading of a native library without the restriction that a native library can only be loaded by one class loader.? The restriction is important for System::loadLibrary in loading JNI libraries that implements JNI native methods and so remain unchanged. This patch follows up the change introduced by JDK-8227587 which adds BootLoader::loadLibrary.? It should belong a helper classes instead of BootLoader and this helper method is to avoid doPrivileged call if no security manager is installed for startup performance improvement.?? Several `privilegedXXX` methods were added for the same purpose for example sun.security.action.GetPropertyAction::privilegedGetProperty and privilegedGetProperties.? So I think sun.security.action.LoadLibraryAction is a proper place. Webrev: http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8228336/webrev.00/ Thanks Mandy From maurizio.cimadamore at oracle.com Mon Mar 9 23:44:37 2020 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 9 Mar 2020 23:44:37 +0000 Subject: Review Request JDK-8228336: Refactor native library loading implementation In-Reply-To: <00d2029d-ae16-6a9b-5942-cd67aba7c141@oracle.com> References: <00d2029d-ae16-6a9b-5942-cd67aba7c141@oracle.com> Message-ID: Thanks Mandy, this refactoring would indeed enable Panama library loading to get rid of classloader-related restriction which are JNI-based and have little to do with the way Panama does things. I'm looking forward to be able to use something like this from my side of the fence :-) Cheers Maurizio On 09/03/2020 23:33, Mandy Chung wrote: > This refactoring enables Panama LookupLibrary to further experiment > and allow loading of a native library without the restriction that a > native library can only be loaded by one class loader.? The > restriction is important for System::loadLibrary in loading JNI > libraries that implements JNI native methods and so remain unchanged. From shiyue.xw at alibaba-inc.com Tue Mar 10 02:39:52 2020 From: shiyue.xw at alibaba-inc.com (=?UTF-8?B?5ZCR5LyfKOivhuaciCk=?=) Date: Tue, 10 Mar 2020 10:39:52 +0800 Subject: =?UTF-8?B?5Zue5aSN77yaTmVlZCBJbmZvcm1hdGlvbiBmb3IgbWVtb3J5IHNoYXJpbmcgd2l0aCBBcHBD?= =?UTF-8?B?RFM=?= In-Reply-To: References: <4c513dcc-ed4e-43ec-a607-764ded395058.shiyue.xw@alibaba-inc.com> <86cccf4b-e497-bd03-155a-c51fbcc9ae77@oracle.com> <7d348b97-161b-43d2-bd7b-896a50287a57.shiyue.xw@alibaba-inc.com>, Message-ID: Hi Jiangli, Thanks a lot for the detailed information. It's clear to me. Thanks, Wei Xiang ------------------------------------------------------------------ ????Jiangli Zhou ?????2020?3?10?(???) 00:50 ??????(??) ????hotspot-runtime-dev ; hotspot-runtime-dev ????Re: Need Information for memory sharing with AppCDS Hi, On Sun, Mar 8, 2020 at 11:56 PM ??(??) wrote: > > Hi Ioi, > > Thanks a lot for your information. > I have the question about the usage in the document: > For common class path prefix: > --------------------------------------- > Run the Hello and Hi applications with the same shared archive: > java -XX:SharedArchiveFile=common.jsa -cp common.jar:hello.jar:hi.jar Hello > java -XX:SharedArchiveFile=common.jsa -cp common.jar:hello.jar:hi.jar Hi > > If a class with name ABC is both in hello.jar and hi.jar, the content is different. > With the class path -cp common.jar:hello.jar:hi.jar, the second run "java .. Hi" can't load the correct class ABC in hi.jar. > For the above usage, I'd recommend to only archive the classes from common.jar to create a common.jsa. The common.jsa can be used for both applications for sharing. As described in the earlier email, the follow two conditions must be true to achieve sharing: - Both processes must be running with the same JVM binary - Both processes must be memory mapping using the same copy of common.jsa. If each container uses a different copy of the common.jsa, then there is no memory sharing. > Is there any solution for this usage? I'd recommend to not include ABC in the common.jsa in this case. Hope that helps. Best, Jiangli > > Thanks, > Wei Xiang > ------------------------------------------------------------------ > ????Ioi Lam > ?????2020?3?9?(???) 13:03 > ????hotspot-runtime-dev > ? ??Re: Need Information for memory sharing with AppCDS > > On 3/8/20 8:51 PM, ??(??) wrote: > > Hi, > > > > We am trying to take advantage of the characteristics of memory sharing with AppCDS, and we expected that the > > JVM internal packages and common packages can be shared with AppCDS. > > > > In the user scenario, there are some common base packages for different Java processes with "-cp common_path" > > Process A: -cp common_path:pathA > > Process B: -cp common_path:pathA:pathB > > Process C: -cp common_path:pathC > > > > I check the code in JDK11, and it uses SharedPathsMiscInfo to do the consistency check for class path. > > The below usage can work in JDK11 > > Dump stage: > > -cp common_path > > > > Runtime stage: > > -cp common_path:new_path > > > > But I find that SharedPathsMiscInfo has been removed in the latest JDK code. > > > > I need the information: > > 1. the new flow to do the consistency check. > > 2. constraint condition about the memory sharing among the different Java processes. > > > > Is there any user guide to use the memory sharing feature for AppCDS? > > Hi Wei, > > The topic of sharing the same CDS archive between different apps that > have the same classpath prefix can be found here: > > https://docs.oracle.com/en/java/javase/13/docs/specs/man/java.html#application-class-data-sharing > > See the section "Sharing a Shared Archive Across Multiple Application > Processes". > > SharedPathsMiscInfo was removed in this RFE in JDK 14: > https://bugs.openjdk.java.net/browse/JDK-8227370 > > However, we still enforce the same set of consistency checks as previous > JDK versions. > > Please let me know if you have further questions. > > Thanks > - Ioi > > > Thanks, > > Wei Xiang > From david.holmes at oracle.com Tue Mar 10 02:41:26 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 10 Mar 2020 12:41:26 +1000 Subject: Review Request JDK-8228336: Refactor native library loading implementation In-Reply-To: <00d2029d-ae16-6a9b-5942-cd67aba7c141@oracle.com> References: <00d2029d-ae16-6a9b-5942-cd67aba7c141@oracle.com> Message-ID: Hi Mandy, On 10/03/2020 9:33 am, Mandy Chung wrote: > This patch refactors the native library loading implementation out of > ClassLoader and move them to jdk.internal.loader package. This > introduces a new NativeLibraries abstraction which loads and registers > the loaded native libraries.? Previously it was maintained in a > nativeLibrary map in each ClassLoader instance and one > systemNativeLibraries for the null loader case.? With this change, each > ClassLoader and BootLoader have its own NativeLibraries instead. > > NativeLibraries.java and NativeLibraries.c are mostly refactored from > the existing code.? Only minor cleanups are applied. > > This refactoring enables Panama LookupLibrary to further experiment and > allow loading of a native library without the restriction that a native > library can only be loaded by one class loader.? The restriction is > important for System::loadLibrary in loading JNI libraries that > implements JNI native methods and so remain unchanged. > > This patch follows up the change introduced by JDK-8227587 which adds > BootLoader::loadLibrary.? It should belong a helper classes instead of > BootLoader and this helper method is to avoid doPrivileged call if no > security manager is installed for startup performance improvement. > Several `privilegedXXX` methods were added for the same purpose for > example sun.security.action.GetPropertyAction::privilegedGetProperty and > privilegedGetProperties.? So I think > sun.security.action.LoadLibraryAction is a proper place. That's a core-libs decision but I'm not sure that's a namespace we want to increase usage of. > Webrev: > http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8228336/webrev.00/ Overall refactoring looks okay to me. A couple of minor things I noticed: src/java.base/share/classes/java/lang/ClassLoader.java ! " in java.library.path: " + StaticProperty.sunBootLibraryPath()); s/java/sun.boot/ src/java.base/share/classes/java/lang/Runtime.java I don't see the point in changing load0 and loadLibrary0 to return NativeLibrary when it is unused. Is this to allow for direct use of these methods in the future? Thanks, David ----- > Thanks > Mandy From yumin.qi at oracle.com Tue Mar 10 03:06:30 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Mon, 9 Mar 2020 20:06:30 -0700 Subject: RFR(XS): 8240691: serviceability/sa/ClhsdbCDSJstackPrintAll.java and serviceability/sa/ClhsdbCDSCore.java not work with ZGC In-Reply-To: <86878457-3386-ea0e-f23e-7ad1bdff64a6@oracle.com> References: <86878457-3386-ea0e-f23e-7ad1bdff64a6@oracle.com> Message-ID: <090873f5-91da-5c96-bc0c-3fb9992ccd5e@oracle.com> Hi, ??? I made a change after discussed with Ioi, the test should check against UseSharedSpaces instead, it passed with/without change of 8240563 (tested in local). Also passed tier1,2,3. ???? URL: http://cr.openjdk.java.net/~minqi/8240691/webrev/ ??? Thanks ???? Yumin On 3/6/20 2:26 PM, Yumin Qi wrote: > HI, > > ? Can I have your reviews on > > ? Bug: https://bugs.openjdk.java.net/browse/JDK-8240691 > > ? webrev: http://cr.openjdk.java.net/~minqi/8240691/webrev/ > > ? Summary: With JDK-8232069 > (https://bugs.openjdk.java.net/browse/JDK-8232069), CDS works with > UseCompressedOop and UseCompressedClassPointers off. The code for > detecting CDS will report false with ZGC since ZGC turned off those > two flags. The detection code will be changed in > JDK-8240563(https://bugs.openjdk.java.net/browse/JDK-8240563), this > will cause those two tests fail. > > > ?? Tests: jtreg to show the tests not selected with ZGC. > > ?? Thanks > > ?? Yumin > From mandy.chung at oracle.com Tue Mar 10 04:53:32 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 9 Mar 2020 21:53:32 -0700 Subject: Review Request JDK-8228336: Refactor native library loading implementation In-Reply-To: References: <00d2029d-ae16-6a9b-5942-cd67aba7c141@oracle.com> Message-ID: <1030f82a-6d7d-daa1-788f-f39848a5022b@oracle.com> Hi David, On 3/9/20 7:41 PM, David Holmes wrote: > That's a core-libs decision but I'm not sure that's a namespace we > want to increase usage of. > I'm open to other suggestion.? This helper method avoids the call to doPrivileged when security manager is enabled and I think it's okay to add this helper method in sun.security.action package. >> Webrev: >> http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8228336/webrev.00/ > > Overall refactoring looks okay to me. A couple of minor things I noticed: > > src/java.base/share/classes/java/lang/ClassLoader.java > > !???????????????????? " in java.library.path: " + > StaticProperty.sunBootLibraryPath()); > s/java/sun.boot/ > Thanks for catching this. > src/java.base/share/classes/java/lang/Runtime.java > > I don't see the point in changing load0 and loadLibrary0 to return > NativeLibrary when it is unused. Is this to allow for direct use of > these methods in the future? > It was leftover from my early prototype.?? It's not intended to change.? I have fixed it. This new webrev includes the above minor fixes and a few windows/solaris specific files I missed from webrev.00: http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8228336/webrev.01/index.html Thanks Mandy From david.holmes at oracle.com Tue Mar 10 06:11:16 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 10 Mar 2020 16:11:16 +1000 Subject: Review Request JDK-8228336: Refactor native library loading implementation In-Reply-To: <1030f82a-6d7d-daa1-788f-f39848a5022b@oracle.com> References: <00d2029d-ae16-6a9b-5942-cd67aba7c141@oracle.com> <1030f82a-6d7d-daa1-788f-f39848a5022b@oracle.com> Message-ID: Hi Mandy, Updates seem fine to me. Thanks, David On 10/03/2020 2:53 pm, Mandy Chung wrote: > Hi David, > > On 3/9/20 7:41 PM, David Holmes wrote: >> That's a core-libs decision but I'm not sure that's a namespace we >> want to increase usage of. >> > > I'm open to other suggestion.? This helper method avoids the call to > doPrivileged when security manager is enabled and I think it's okay to > add this helper method in sun.security.action package. > >>> Webrev: >>> http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8228336/webrev.00/ >> >> Overall refactoring looks okay to me. A couple of minor things I noticed: >> >> src/java.base/share/classes/java/lang/ClassLoader.java >> >> !???????????????????? " in java.library.path: " + >> StaticProperty.sunBootLibraryPath()); >> s/java/sun.boot/ >> > > Thanks for catching this. > >> src/java.base/share/classes/java/lang/Runtime.java >> >> I don't see the point in changing load0 and loadLibrary0 to return >> NativeLibrary when it is unused. Is this to allow for direct use of >> these methods in the future? >> > > It was leftover from my early prototype.?? It's not intended to change. > I have fixed it. > > This new webrev includes the above minor fixes and a few windows/solaris > specific files I missed from webrev.00: > http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8228336/webrev.01/index.html > > Thanks > Mandy From stefan.karlsson at oracle.com Tue Mar 10 06:32:35 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 10 Mar 2020 07:32:35 +0100 Subject: RFR: 8240530: CheckUnhandledOops breaks BacktraceBuilder::set_has_hidden_top_frame In-Reply-To: <8a5ad3a5-75e9-228b-a005-de4c12bf77c4@oracle.com> References: <89b3460e-47a2-daec-bbd0-755e095d4c46@oracle.com> <261be2cb-a8df-351f-043c-ea4b4792b754@oracle.com> <210af131-15e4-efb4-ac74-c6d8f8411ebc@oracle.com> <8a5ad3a5-75e9-228b-a005-de4c12bf77c4@oracle.com> Message-ID: Could I get a second review of the the patch: https://cr.openjdk.java.net/~stefank/8240530/webrev.02 Thanks! StefanK On 2020-03-05 13:21, coleen.phillimore at oracle.com wrote: > > > On 3/5/20 5:01 AM, Stefan Karlsson wrote: >> On 2020-03-04 16:08, coleen.phillimore at oracle.com wrote: >>> >>> Hi, >>> >>> I don't like the change to void* for _has_hidden_top_frame. >>> >>> Can you change this line to >>> >>> ??? if (_has_hidden_top_frame == _methods) { >>> >>> instead? >> >> I think you mean? >> if (_has_hidden_top_frame != _methods) { >> >> That doesn't work for at least two reason: >> 1) _has_hidden_top_frame is still being clobbered by CheckUnhandledOops >> 2) _methods are being reinitialized and changed value in >> BacktraceBuilder::expand >> >>> >>> Can you add an 'e' to Therefor >>> >>> ????? // It would be nice to add java/lang/Boolean::TRUE here >>> ????? // to indicate that this backtrace has a hidden top frame. >>> ????? // But this code is used before TRUE is allocated. >>> ????? // Therefor let's just use an arbitrary legal oop >>> ????? // available right here. We only test for != null >>> ????? // anyways. _methods is a short[]. >> >> Sure. >> >>> >>> The reason that it's an oop is because the _backtrace is an >>> objArrayOop of java.lang.Object[6], and hidden boolean is Object[5]. >>> It's not a nice data structure but it needs to be fast. >> >> That's the reason why we put an object in trace_hidden_offset: >> ????? _head->obj_at_put(trace_hidden_offset, ); >> >> but I'm specifically asking about the field _has_hidden_top_frame. >> >> I created a patch to show what I mean, to show that that field can >> simply be a bool: >> ?https://cr.openjdk.java.net/~stefank/8240530/webrev.02.delta >> ?https://cr.openjdk.java.net/~stefank/8240530/webrev.02 >> >> It compiles and seems to work. > > Yes. That looks great.?? We should have done that in the first place. > Thanks, > Coleen >> >> StefanK >> >>> Coleen >>> >>> >>> On 3/4/20 9:48 AM, Stefan Karlsson wrote: >>>> Hi all, >>>> >>>> Please review this patch to get rid of a unhandled oop null check in >>>> the BacktraceBuilder. >>>> >>>> https://cr.openjdk.java.net/~stefank/8240530/webrev.01/ >>>> https://bugs.openjdk.java.net/browse/JDK-8240530 >>>> >>>> The patch changes the type from oop to void* to prevent NULL oops >>>> from being overwritten with non-NULL values. >>>> >>>> I think an alternative fix would be to change the type of >>>> _has_hidden_top_frame to bool. Maybe someone familiar with this code >>>> knows why this was encoded with an oop NULL check instead of a plain >>>> bool? >>>> >>>> Testing with tier1-3 with CheckUnhandledOops on by default. >>>> >>>> StefanK >>> >> > From rickard.backman at oracle.com Tue Mar 10 06:53:19 2020 From: rickard.backman at oracle.com (Rickard =?iso-8859-1?Q?B=E4ckman?=) Date: Tue, 10 Mar 2020 07:53:19 +0100 Subject: RFR: 8240669: Devirtualize Relocation::type In-Reply-To: <5e26fc09-be3e-8009-3248-2fcaa17e1f1f@oracle.com> References: <5e26fc09-be3e-8009-3248-2fcaa17e1f1f@oracle.com> Message-ID: <20200310065319.GQ23956@rbackman> Looks good! /R On 03/08, Claes Redestad wrote: > Hi, > > devirtualizing Relocation::type() means a throughput increase in places such > as Assembler::emit_operand (-15% instructions, fewer branches), due > enabling more aggressive inlining. > > Several of these places are sensitive to startup/warmup. For example > paths taken when generating the interpreter, runtime adapters and by C1 > improve, and the refactoring here has a measurable win on #instructions, > #branches (-0.4% on Hello World) etc, on a range of startup tests. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8240302 > Webrev: http://cr.openjdk.java.net/~redestad/8240669/open.00/ > > Testing: tier1-5 > > Thanks! > > /Claes From xxinliu at amazon.com Tue Mar 10 07:06:14 2020 From: xxinliu at amazon.com (Liu, Xin) Date: Tue, 10 Mar 2020 07:06:14 +0000 Subject: RFR(L): JDK-8229517: Support for optional asynchronous/buffered logging Message-ID: <49CE3CD3-37D9-458E-9B8A-B12F3A478EBF@amazon.com> Hi, David, Thanks for commenting the RFR. Indeed, LogAsyncFlusher needs to wait async loggers to release the RW mutex, but please note that LogAsyncFlusher releases the RW mutex before it traps into filesystem writing. It's not a blocker( refer to LogAsyncFlusher::task). Ie. The current implementation is blocking-free but not lock-free. I mentioned it in early email. I don't want to make thing complicated in early stage. Lock-free FIFO requires extensive testing. For a FIFO queue, there's a broad design space for it. If we only choose selective logging sites and offload them to the async buffer, it won't be contended situation. If it turns out to be a contented lock, we could change it to spinlock, priority lock or even upgrade the FIFO to lock-free implementation. LogAsyncFlusher is singleton. Its constructor is private. Currently, I manually initialize and delete the only instance of it. It's very subtle because there's timing issue alongside of the initialization JVM. If we enables C++11 in the future, we can have Meyer's singleton idiom. It will be more elegant because the instance can be lazy-initialized. Speaking of tagset, I think it's advantage of Unified logging framework. The framework separates frond-end and backend, interface and implementation. I love it and make use of it. A concrete class called 'LogTagSet' represents a set of tags. They are all singleton and created by LogTagSetMapping. How to emit log messages is defined in LogTagSet. They are even configurable in the runtime. and are just two set of compatible interfaces. Under the hook, they shared the same LogTagset infrastructure. Yes, developers have to choose go conventional log or asynclog in build-time. I think developers have insights of their code which may cause long latency from logging. In order to have drop-in replacement api, I didn't go the direction that provide a new tag for async buffer. So far, tags are all component names of hotspot. Thanks, --lx ?On 3/8/20, 10:34 PM, "David Holmes" wrote: CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. Hi, On 5/03/2020 8:04 pm, Liu, Xin wrote: > Hi, hotspot reviewers, > > Could you take a look at my webrev? > Bug: https://bugs.openjdk.java.net/browse/JDK-8229517 > Webrev: https://cr.openjdk.java.net/~xliu/8229517/01/webrev/ > > Previously, we had discussion at runtime-dev. https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-February/038211.html > I would like to continue it because I redo my implementation on the tip of jdk and have made it a general solution. Not a review as such but a couple of comments/queries. The async logging is done as a periodic task that will be executed the Watcher thread. This task requires a mutex that is also used by the other threads doing the log operations. So potentially the watcher thread could experience a significant delay waiting for the mutex. I'm not clear what the actual lifecycle of the LogAsyncFlusher instance is. I would have expected a single instance initialized at VM startup and cleaned up at VM exit. The code gives the appearance that this can actually be much more dynamic with multiple instances coming and going through the lifetime of the VM. Deletion of the instance is particularly problematic due to the Mutex, which has to be guaranteed not to be accessed when it gets deleted. > This webrev provides on apar with . Even though it doesn?t touch any existing logging calls, it?s very possible to offload any callsite of logging to the asynchronous buffer. > My rationale is that we only need to redirect some time-critical logs to asynchronous buffer to dodge unexpected filesystem delay. Eg. g1gc has some loggings at safepoint. We mitigate those long-tailed GC latencies rooted by filesystem writing to gclog. So this isn't an opt-in facility at runtime, but rather something that has to be decided a-priori by the developer. How does this interact with the synchronous logging that may be used for a given set of log tags? Thanks, David ----- > I got feedback from my our internal users. They have a concern about my approach of dynamic memory. Indeed, I gave up ringbuffer design in previous version and turn to use a dynamic allocated linkedlist. If writing to filesystem stalls infinitely, JVM may implode because of memory usage. I plan to solve it by adding a memory quota to the FIFO of logAsyncFlusher.hpp. It will start dropping early messages of the FIFO if it has consumed too many free memories. Another potential improvement is changing the FIFO from mutex protected to lock-free. It make sense if we redirect high volume of logging messages to it and mutex becomes hot. > > Thanks, > --lx > > > From david.holmes at oracle.com Tue Mar 10 07:09:58 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 10 Mar 2020 17:09:58 +1000 Subject: RFR: 8240530: CheckUnhandledOops breaks BacktraceBuilder::set_has_hidden_top_frame In-Reply-To: References: <89b3460e-47a2-daec-bbd0-755e095d4c46@oracle.com> <261be2cb-a8df-351f-043c-ea4b4792b754@oracle.com> <210af131-15e4-efb4-ac74-c6d8f8411ebc@oracle.com> <8a5ad3a5-75e9-228b-a005-de4c12bf77c4@oracle.com> Message-ID: LGTM! Thanks, David On 10/03/2020 4:32 pm, Stefan Karlsson wrote: > Could I get a second review of the the patch: > https://cr.openjdk.java.net/~stefank/8240530/webrev.02 > > Thanks! > StefanK > > On 2020-03-05 13:21, coleen.phillimore at oracle.com wrote: >> >> >> On 3/5/20 5:01 AM, Stefan Karlsson wrote: >>> On 2020-03-04 16:08, coleen.phillimore at oracle.com wrote: >>>> >>>> Hi, >>>> >>>> I don't like the change to void* for _has_hidden_top_frame. >>>> >>>> Can you change this line to >>>> >>>> ??? if (_has_hidden_top_frame == _methods) { >>>> >>>> instead? >>> >>> I think you mean? >>> if (_has_hidden_top_frame != _methods) { >>> >>> That doesn't work for at least two reason: >>> 1) _has_hidden_top_frame is still being clobbered by CheckUnhandledOops >>> 2) _methods are being reinitialized and changed value in >>> BacktraceBuilder::expand >>> >>>> >>>> Can you add an 'e' to Therefor >>>> >>>> ????? // It would be nice to add java/lang/Boolean::TRUE here >>>> ????? // to indicate that this backtrace has a hidden top frame. >>>> ????? // But this code is used before TRUE is allocated. >>>> ????? // Therefor let's just use an arbitrary legal oop >>>> ????? // available right here. We only test for != null >>>> ????? // anyways. _methods is a short[]. >>> >>> Sure. >>> >>>> >>>> The reason that it's an oop is because the _backtrace is an >>>> objArrayOop of java.lang.Object[6], and hidden boolean is Object[5]. >>>> It's not a nice data structure but it needs to be fast. >>> >>> That's the reason why we put an object in trace_hidden_offset: >>> ????? _head->obj_at_put(trace_hidden_offset, ); >>> >>> but I'm specifically asking about the field _has_hidden_top_frame. >>> >>> I created a patch to show what I mean, to show that that field can >>> simply be a bool: >>> ?https://cr.openjdk.java.net/~stefank/8240530/webrev.02.delta >>> ?https://cr.openjdk.java.net/~stefank/8240530/webrev.02 >>> >>> It compiles and seems to work. >> >> Yes. That looks great.?? We should have done that in the first place. >> Thanks, >> Coleen >>> >>> StefanK >>> >>>> Coleen >>>> >>>> >>>> On 3/4/20 9:48 AM, Stefan Karlsson wrote: >>>>> Hi all, >>>>> >>>>> Please review this patch to get rid of a unhandled oop null check >>>>> in the BacktraceBuilder. >>>>> >>>>> https://cr.openjdk.java.net/~stefank/8240530/webrev.01/ >>>>> https://bugs.openjdk.java.net/browse/JDK-8240530 >>>>> >>>>> The patch changes the type from oop to void* to prevent NULL oops >>>>> from being overwritten with non-NULL values. >>>>> >>>>> I think an alternative fix would be to change the type of >>>>> _has_hidden_top_frame to bool. Maybe someone familiar with this >>>>> code knows why this was encoded with an oop NULL check instead of a >>>>> plain bool? >>>>> >>>>> Testing with tier1-3 with CheckUnhandledOops on by default. >>>>> >>>>> StefanK >>>> >>> >> From stefan.karlsson at oracle.com Tue Mar 10 07:29:30 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 10 Mar 2020 08:29:30 +0100 Subject: RFR: 8240530: CheckUnhandledOops breaks BacktraceBuilder::set_has_hidden_top_frame In-Reply-To: References: <89b3460e-47a2-daec-bbd0-755e095d4c46@oracle.com> <261be2cb-a8df-351f-043c-ea4b4792b754@oracle.com> <210af131-15e4-efb4-ac74-c6d8f8411ebc@oracle.com> <8a5ad3a5-75e9-228b-a005-de4c12bf77c4@oracle.com> Message-ID: <7182b8b6-78cf-1b08-c8c0-b62ca064311b@oracle.com> Thanks, David! StefanK On 2020-03-10 08:09, David Holmes wrote: > LGTM! > > Thanks, > David > > On 10/03/2020 4:32 pm, Stefan Karlsson wrote: >> Could I get a second review of the the patch: >> https://cr.openjdk.java.net/~stefank/8240530/webrev.02 >> >> Thanks! >> StefanK >> >> On 2020-03-05 13:21, coleen.phillimore at oracle.com wrote: >>> >>> >>> On 3/5/20 5:01 AM, Stefan Karlsson wrote: >>>> On 2020-03-04 16:08, coleen.phillimore at oracle.com wrote: >>>>> >>>>> Hi, >>>>> >>>>> I don't like the change to void* for _has_hidden_top_frame. >>>>> >>>>> Can you change this line to >>>>> >>>>> ??? if (_has_hidden_top_frame == _methods) { >>>>> >>>>> instead? >>>> >>>> I think you mean? >>>> if (_has_hidden_top_frame != _methods) { >>>> >>>> That doesn't work for at least two reason: >>>> 1) _has_hidden_top_frame is still being clobbered by CheckUnhandledOops >>>> 2) _methods are being reinitialized and changed value in >>>> BacktraceBuilder::expand >>>> >>>>> >>>>> Can you add an 'e' to Therefor >>>>> >>>>> ????? // It would be nice to add java/lang/Boolean::TRUE here >>>>> ????? // to indicate that this backtrace has a hidden top frame. >>>>> ????? // But this code is used before TRUE is allocated. >>>>> ????? // Therefor let's just use an arbitrary legal oop >>>>> ????? // available right here. We only test for != null >>>>> ????? // anyways. _methods is a short[]. >>>> >>>> Sure. >>>> >>>>> >>>>> The reason that it's an oop is because the _backtrace is an >>>>> objArrayOop of java.lang.Object[6], and hidden boolean is >>>>> Object[5]. It's not a nice data structure but it needs to be fast. >>>> >>>> That's the reason why we put an object in trace_hidden_offset: >>>> ????? _head->obj_at_put(trace_hidden_offset, ); >>>> >>>> but I'm specifically asking about the field _has_hidden_top_frame. >>>> >>>> I created a patch to show what I mean, to show that that field can >>>> simply be a bool: >>>> ?https://cr.openjdk.java.net/~stefank/8240530/webrev.02.delta >>>> ?https://cr.openjdk.java.net/~stefank/8240530/webrev.02 >>>> >>>> It compiles and seems to work. >>> >>> Yes. That looks great.?? We should have done that in the first place. >>> Thanks, >>> Coleen >>>> >>>> StefanK >>>> >>>>> Coleen >>>>> >>>>> >>>>> On 3/4/20 9:48 AM, Stefan Karlsson wrote: >>>>>> Hi all, >>>>>> >>>>>> Please review this patch to get rid of a unhandled oop null check >>>>>> in the BacktraceBuilder. >>>>>> >>>>>> https://cr.openjdk.java.net/~stefank/8240530/webrev.01/ >>>>>> https://bugs.openjdk.java.net/browse/JDK-8240530 >>>>>> >>>>>> The patch changes the type from oop to void* to prevent NULL oops >>>>>> from being overwritten with non-NULL values. >>>>>> >>>>>> I think an alternative fix would be to change the type of >>>>>> _has_hidden_top_frame to bool. Maybe someone familiar with this >>>>>> code knows why this was encoded with an oop NULL check instead of >>>>>> a plain bool? >>>>>> >>>>>> Testing with tier1-3 with CheckUnhandledOops on by default. >>>>>> >>>>>> StefanK >>>>> >>>> >>> From tobias.hartmann at oracle.com Tue Mar 10 09:36:34 2020 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Tue, 10 Mar 2020 10:36:34 +0100 Subject: RFR: 8235908: omit ThreadPriorityPolicy warning when value is set from image In-Reply-To: <48754173-0e4e-736e-6895-6d3051225c7c@oracle.com> References: <48754173-0e4e-736e-6895-6d3051225c7c@oracle.com> Message-ID: <8b87a5a8-93e1-ad0d-37e1-ec3254dadb71@oracle.com> [adding hotspot-runtime-dev] On 06.03.20 09:56, Matthias Neugschwandtner wrote: > Hello everyone, > > Please review this change, which omits the warning that ThreadPriorityPolicy=1 requires system level > permission if the value is set via jlink, the same way it is omitted if the value is set by default. > > https://bugs.openjdk.java.net/browse/JDK-8235908 > http://cr.openjdk.java.net/~mneugschwand/8235908 > > Thank you, > Matthias From tobias.hartmann at oracle.com Tue Mar 10 09:41:50 2020 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Tue, 10 Mar 2020 10:41:50 +0100 Subject: RFR: 8240669: Devirtualize Relocation::type In-Reply-To: <5e26fc09-be3e-8009-3248-2fcaa17e1f1f@oracle.com> References: <5e26fc09-be3e-8009-3248-2fcaa17e1f1f@oracle.com> Message-ID: Hi Claes, On 08.03.20 15:12, Claes Redestad wrote: > Bug:??? https://bugs.openjdk.java.net/browse/JDK-8240302 > Webrev: http://cr.openjdk.java.net/~redestad/8240669/open.00/ Looks good to me too (you've linked the wrong bug). Best regards, Tobias From aph at redhat.com Tue Mar 10 09:44:54 2020 From: aph at redhat.com (Andrew Haley) Date: Tue, 10 Mar 2020 09:44:54 +0000 Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is inconsistent In-Reply-To: References: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com> <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com> <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com> <1c40a7d1-fa36-b5d2-6072-eaa243cc9397@redhat.com> Message-ID: <57e26fa3-6c8f-a7db-f6fc-9d4b58cf4126@redhat.com> On 3/9/20 3:56 PM, Doerr, Martin wrote: > it doesn't run into the UB case, but log2_jint(x) returns 63 for negative x. > I'd treat this as bug. A jint has 32 bit and log2 should never return 63. OK. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From claes.redestad at oracle.com Tue Mar 10 10:33:41 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 10 Mar 2020 11:33:41 +0100 Subject: RFR: 8240669: Devirtualize Relocation::type In-Reply-To: References: <5e26fc09-be3e-8009-3248-2fcaa17e1f1f@oracle.com> Message-ID: <0ed7e8d1-278a-7ef4-0232-fa9cb219097a@oracle.com> On 2020-03-10 10:41, Tobias Hartmann wrote: > Hi Claes, > > On 08.03.20 15:12, Claes Redestad wrote: >> Bug:??? https://bugs.openjdk.java.net/browse/JDK-8240302 >> Webrev: http://cr.openjdk.java.net/~redestad/8240669/open.00/ > > Looks good to me too Thanks! > (you've linked the wrong bug). D'oh, intended to link to that one with a note that the performance gain from that RFE ought to be made redundant by this devirtualization (still a reasonable cleanup) /Claes From claes.redestad at oracle.com Tue Mar 10 10:33:56 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 10 Mar 2020 11:33:56 +0100 Subject: RFR: 8240669: Devirtualize Relocation::type In-Reply-To: <20200310065319.GQ23956@rbackman> References: <5e26fc09-be3e-8009-3248-2fcaa17e1f1f@oracle.com> <20200310065319.GQ23956@rbackman> Message-ID: Thanks, Rickard! /Claes On 2020-03-10 07:53, Rickard B?ckman wrote: > Looks good! > > /R > > On 03/08, Claes Redestad wrote: >> Hi, >> >> devirtualizing Relocation::type() means a throughput increase in places such >> as Assembler::emit_operand (-15% instructions, fewer branches), due >> enabling more aggressive inlining. >> >> Several of these places are sensitive to startup/warmup. For example >> paths taken when generating the interpreter, runtime adapters and by C1 >> improve, and the refactoring here has a measurable win on #instructions, >> #branches (-0.4% on Hello World) etc, on a range of startup tests. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8240302 >> Webrev: http://cr.openjdk.java.net/~redestad/8240669/open.00/ >> >> Testing: tier1-5 >> >> Thanks! >> >> /Claes From claes.redestad at oracle.com Tue Mar 10 13:46:10 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 10 Mar 2020 14:46:10 +0100 Subject: RFR: 8240772: x86_64: Pre-generate Assembler::popa, pusha and vzeroupper Message-ID: <9fc3ea60-ccfb-7250-c1e0-06c40088fe7e@oracle.com> Hi, calculate some invariant Assembler routines at bootstrap, copy on subsequent invocations. For popa and pusha this means an overhead reduction of around 98% (from ~2500 instructions to emit a pusha to ~50). For vzeroupper an overhead reduction of ~65% (117 -> 42). Together these add up to about a 1% reduction of instructions executed on a Hello World - with some (smaller) scaling impact on larger applications. The initialization is very simple/naive, i.e., lacks any kind of synchronization protocol. But as this setup is guaranteed to happen very early during bootstrap this should be fine. Thanks Ioi for some helpful suggestions here! Bug: https://bugs.openjdk.java.net/browse/JDK-8240772 Webrev: http://cr.openjdk.java.net/~redestad/8240772/open.00/ Testing: tier1-3 Thanks! /Claes From aph at redhat.com Tue Mar 10 15:04:24 2020 From: aph at redhat.com (Andrew Haley) Date: Tue, 10 Mar 2020 15:04:24 +0000 Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is inconsistent In-Reply-To: References: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com> <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com> <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com> Message-ID: On 3/9/20 12:20 PM, Stefan Karlsson wrote: > On 2020-03-09 11:54, Andrew Haley wrote: >> So, my plan is to push http://cr.openjdk.java.net/~aph/8240615-1/ and >> subsequently patch a few AArch64 cases plus the x86_64 immL_Pow2 and >> immL_NotPow2 cases that we already identified. >> >> I do not intend to do bulk changes to any other clients of >> is_power_of_2() because it's often unclear exactly what behaviour is >> required, and I'd risk breaking something. >> >> OK? > > I would prefer if the x86_64 and aarch64 patches were pushed first, or > folded into this patch. Otherwise, we knowingly introduce a regression > by pushing the proposed patch above. However, I'll leave it up to the > compiler devs/maintainers to decide if this is OK. http://cr.openjdk.java.net/~aph/8240615-2/ OK? -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From stefan.karlsson at oracle.com Tue Mar 10 15:12:46 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 10 Mar 2020 16:12:46 +0100 Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is inconsistent In-Reply-To: References: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com> <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com> <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com> Message-ID: <51dfbbb3-f0cc-2210-a4f1-558a3a6a9ac7@oracle.com> On 2020-03-10 16:04, Andrew Haley wrote: > On 3/9/20 12:20 PM, Stefan Karlsson wrote: >> On 2020-03-09 11:54, Andrew Haley wrote: >>> So, my plan is to push http://cr.openjdk.java.net/~aph/8240615-1/ and >>> subsequently patch a few AArch64 cases plus the x86_64 immL_Pow2 and >>> immL_NotPow2 cases that we already identified. >>> >>> I do not intend to do bulk changes to any other clients of >>> is_power_of_2() because it's often unclear exactly what behaviour is >>> required, and I'd risk breaking something. >>> >>> OK? >> I would prefer if the x86_64 and aarch64 patches were pushed first, or >> folded into this patch. Otherwise, we knowingly introduce a regression >> by pushing the proposed patch above. However, I'll leave it up to the >> compiler devs/maintainers to decide if this is OK. > http://cr.openjdk.java.net/~aph/8240615-2/ > > OK? Looks good to me, but compiler devs need to review as well. StefanK > From yumin.qi at oracle.com Tue Mar 10 15:26:40 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Tue, 10 Mar 2020 08:26:40 -0700 Subject: RFR(XS): 8240691: serviceability/sa/ClhsdbCDSJstackPrintAll.java and serviceability/sa/ClhsdbCDSCore.java not work with ZGC In-Reply-To: References: <86878457-3386-ea0e-f23e-7ad1bdff64a6@oracle.com> <090873f5-91da-5c96-bc0c-3fb9992ccd5e@oracle.com> Message-ID: <4007dbab-cfa9-32a7-72bc-67be966a3561@oracle.com> Hi, Chris ??? The original failure is caused by fixing bug https://bugs.openjdk.java.net/browse/JDK-8240563 ?? The removal of checking UseCompressedOops and UseCompressedClassPointers since CDS now is working with them on/off. ?? The testing will be skipped ZGC since the old code detecting CDS on 64 bit is checking if those two flags turned off --- CDS only works with compressed oops before (https://bugs.openjdk.java.net/browse/JDK-8232069 pushed. With ZGC, those two flags will be turned off so the detection of CDS will return false. This makes the tests skipped, it will never tested with ZGC. Now the detection will return true if built with CDS enabled. So the test will go through with CDS. Unfortunately sa does not work with ZGC heap I think so it failed. ?? Thanks ?? Yumin On 3/9/20 9:33 PM, Chris Plummer wrote: > Hi Yumin, > > While your fix is a more bullet proof way of checking if CDS is on, > what was broken with the original version (was there a stray "false" > in the output"), and what does this have to do with ZGC? > > thanks, > > Chris > > On 3/9/20 8:06 PM, Yumin Qi wrote: >> >> Hi, >> >> ??? I made a change after discussed with Ioi, the test should check >> against UseSharedSpaces instead, it passed with/without change of >> 8240563 (tested in local). Also passed tier1,2,3. >> >> ???? URL: http://cr.openjdk.java.net/~minqi/8240691/webrev/ >> >> >> ??? Thanks >> >> ???? Yumin >> >> >> On 3/6/20 2:26 PM, Yumin Qi wrote: >>> HI, >>> >>> ? Can I have your reviews on >>> >>> ? Bug: https://bugs.openjdk.java.net/browse/JDK-8240691 >>> >>> ? webrev: http://cr.openjdk.java.net/~minqi/8240691/webrev/ >>> >>> ? Summary: With JDK-8232069 >>> (https://bugs.openjdk.java.net/browse/JDK-8232069), CDS works with >>> UseCompressedOop and UseCompressedClassPointers off. The code for >>> detecting CDS will report false with ZGC since ZGC turned off those >>> two flags. The detection code will be changed in >>> JDK-8240563(https://bugs.openjdk.java.net/browse/JDK-8240563), this >>> will cause those two tests fail. >>> >>> >>> ?? Tests: jtreg to show the tests not selected with ZGC. >>> >>> ?? Thanks >>> >>> ?? Yumin >>> > From richard.reingruber at sap.com Tue Mar 10 16:00:21 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Tue, 10 Mar 2020 16:00:21 +0000 Subject: RFC(S) 8234146: [TESTBUG] compiler/jsr292/ContinuousCallSiteTargetChange.java times out on SPARC In-Reply-To: References: Message-ID: Giving steps how to reproduce the slowdown and test the fixes might be helpful: * do a release build of the jdk repo without the fix and change into the build directory * create symbolic link to the jdk repository named jdk-repo * compile ContinuousCallSiteTargetChange.java images/jdk/bin/javac -d classes --source-path jdk-repo/test/lib jdk-repo/test/hotspot/jtreg/compiler/jsr292/ContinuousCallSiteTargetChange.java * execute PingPongTest with relevant vm options used in ContinuousCallSiteTargetChange::testServer time images/jdk/bin/java -server -XX:-TieredCompilation -cp classes 'compiler.jsr292.ContinuousCallSiteTargetChange$PingPongTest' 50 real 0m2.158s user 0m4.068s sys 0m0.292s Linux Only: * simulate high load by pinning the test to one cpu time taskset 1 images/jdk/bin/java -server -XX:-TieredCompilation -cp classes 'compiler.jsr292.ContinuousCallSiteTargetChange$PingPongTest' 50 real 0m23.734s user 0m23.620s sys 0m0.108s * Apply http://cr.openjdk.java.net/~rrich/webrevs/8234146/B/webrev.0/jdk.changeset * Rebuild and run test again time taskset 1 images/jdk/bin/java -server -XX:-TieredCompilation -cp classes 'compiler.jsr292.ContinuousCallSiteTargetChange$PingPongTest' 50 real 0m2.076s user 0m2.024s sys 0m0.048s With higher load (20 cores Xeon(R) CPU E5-2660 v3 @ 2.60GHz): wget http://cr.openjdk.java.net/~rrich/webrevs/8234146/CPULoadGenerator.java images/jdk/bin/java CPULoadGenerator.java 240 # without fix time images/jdk/bin/java -server -XX:-TieredCompilation -cp classes 'compiler.jsr292.ContinuousCallSiteTargetChange$PingPongTest' 50 real 1m53.832s user 0m30.648s sys 0m0.128s # with http://cr.openjdk.java.net/~rrich/webrevs/8234146/B/webrev.0/jdk.changeset time images/jdk/bin/java -server -XX:-TieredCompilation -cp classes 'compiler.jsr292.ContinuousCallSiteTargetChange$PingPongTest' 50 real 0m8.241s user 0m3.856s sys 0m0.100s Thanks, Richard. -----Original Message----- From: hotspot-compiler-dev On Behalf Of Reingruber, Richard Sent: Sonntag, 8. M?rz 2020 22:33 To: hotspot-compiler-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net Subject: [CAUTION] RFC(S) 8234146: [TESTBUG] compiler/jsr292/ContinuousCallSiteTargetChange.java times out on SPARC Hi, I'd like to get comments on two alternative fixes for Bug: https://bugs.openjdk.java.net/browse/JDK-8234146 The timeouts occur when the system is under load. Then the list of dependent nmethods of the dynamic call site of the PingPongTest gets as long as 1500 nmethods. The list is iterated in DependencyContext::mark_dependent_nmethods() every time the call site target is changed. And this takes longer and longer as the list grows. Note that dependents are removed only when they are made zombie, but this doesn't happen here. The list of dependent nmethods is virtually empty if the system is idle, because then the target is changed faster than the method holding the call site gets compiled. When the jit finishes a compilation it discards the new nmethod, because the call site target was changed already. To reproduce long runs I'd suggest to run ContinuousCallSiteTargetChange$PingPongTest manually taking the command from ContinuousCallSiteTargetChange.jtr of a previous run. Under load [1] the test will take about 10x as long as without load. 30s vs 3s on my linux x86_64 box for the server config. With the fixes below the test will take under load approximately as long as without load. There are 2 possible fixes I'd like to get comments for: Option A: Force the NMethodSweeper using the whitebox api. Webrev A: http://cr.openjdk.java.net/~rrich/webrevs/8234146/A/webrev.0/ Option B: Specialize DependencyContext::mark_dependent_nmethods() for call site target changes, i.e. flag list elements that are marked for deoptimization. Upon the next target change only new dependents are marked for deoptimization. The list iteration is ended as soon as an element is encountered that is already flagged. Webrev B: http://cr.openjdk.java.net/~rrich/webrevs/8234146/B/webrev.0/ Personally I like B better, because I think it is simple and straight forward: if the target changes, then all dependents need to be invalidated. By construction of the list we can stop the invalidation if we find an element that was invalidated before. This specialization is of course less generic, and there might be new types of call site dependencies for which it is illegal, but currently I can't think of any. Curious about your comments... Thanks, Richard. [1] To simulate load you can pin the jvm process to one cpu (e.g. on Linux with "taskset 1 java ...") or wget http://cr.openjdk.java.net/~rrich/webrevs/8234146/CPULoadGenerator.java && java CPULoadGenerator.java N where N is the number of CPUs of your box From yumin.qi at oracle.com Tue Mar 10 16:24:12 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Tue, 10 Mar 2020 09:24:12 -0700 Subject: RFR(XS): 8240691: serviceability/sa/ClhsdbCDSJstackPrintAll.java and serviceability/sa/ClhsdbCDSCore.java not work with ZGC In-Reply-To: <4007dbab-cfa9-32a7-72bc-67be966a3561@oracle.com> References: <86878457-3386-ea0e-f23e-7ad1bdff64a6@oracle.com> <090873f5-91da-5c96-bc0c-3fb9992ccd5e@oracle.com> <4007dbab-cfa9-32a7-72bc-67be966a3561@oracle.com> Message-ID: Sorry the last sentence is not right: The test went through but due to the other bug https://bugs.openjdk.java.net/browse/JDK-8240698 The options are not consistent for dump/run this caused CDS was disabled for the run. This led this test failed to find _nofast_aload0. Yumin On 3/10/20 8:26 AM, Yumin Qi wrote: > Hi, Chris > > > ??? The original failure is caused by fixing bug > https://bugs.openjdk.java.net/browse/JDK-8240563 > > ?? The removal of checking UseCompressedOops and > UseCompressedClassPointers since CDS now is working with them on/off. > > ?? The testing will be skipped ZGC since the old code detecting CDS on > 64 bit is checking if those two flags turned off --- CDS only works > with compressed oops before > (https://bugs.openjdk.java.net/browse/JDK-8232069 pushed. With ZGC, > those two flags will be turned off so the detection of CDS will return > false. This makes the tests skipped, it will never tested with ZGC. > Now the detection will return true if built with CDS enabled. So the > test will go through with CDS. Unfortunately sa does not work with ZGC > heap I think so it failed. > > ?? Thanks > > ?? Yumin > > On 3/9/20 9:33 PM, Chris Plummer wrote: >> Hi Yumin, >> >> While your fix is a more bullet proof way of checking if CDS is on, >> what was broken with the original version (was there a stray "false" >> in the output"), and what does this have to do with ZGC? >> >> thanks, >> >> Chris >> >> On 3/9/20 8:06 PM, Yumin Qi wrote: >>> >>> Hi, >>> >>> ??? I made a change after discussed with Ioi, the test should check >>> against UseSharedSpaces instead, it passed with/without change of >>> 8240563 (tested in local). Also passed tier1,2,3. >>> >>> ???? URL: http://cr.openjdk.java.net/~minqi/8240691/webrev/ >>> >>> >>> ??? Thanks >>> >>> ???? Yumin >>> >>> >>> On 3/6/20 2:26 PM, Yumin Qi wrote: >>>> HI, >>>> >>>> ? Can I have your reviews on >>>> >>>> ? Bug: https://bugs.openjdk.java.net/browse/JDK-8240691 >>>> >>>> ? webrev: http://cr.openjdk.java.net/~minqi/8240691/webrev/ >>>> >>>> ? Summary: With JDK-8232069 >>>> (https://bugs.openjdk.java.net/browse/JDK-8232069), CDS works with >>>> UseCompressedOop and UseCompressedClassPointers off. The code for >>>> detecting CDS will report false with ZGC since ZGC turned off those >>>> two flags. The detection code will be changed in >>>> JDK-8240563(https://bugs.openjdk.java.net/browse/JDK-8240563), this >>>> will cause those two tests fail. >>>> >>>> >>>> ?? Tests: jtreg to show the tests not selected with ZGC. >>>> >>>> ?? Thanks >>>> >>>> ?? Yumin >>>> >> From ioi.lam at oracle.com Tue Mar 10 16:29:58 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 10 Mar 2020 09:29:58 -0700 Subject: RFR(XS) 8240548 [TESTBUG] CDS NoClassToArchive.java fails with Graal In-Reply-To: References: <9fd602d1-0e2c-5920-d677-8f2d0a1616be@oracle.com> Message-ID: <92fa6d77-4896-d322-2d10-09ebab728bb4@oracle.com> On 3/9/20 9:43 AM, Mandy Chung wrote: > Hi Ioi, > > On 3/8/20 9:52 PM, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8240548 >> http://cr.openjdk.java.net/~iklam/jdk15/8240548-NoClassToArchive.graal.v01/ >> >> > > 65 if (output.getStdout().contains("jrt:/") || > 66 output.getStdout().contains("unsafe anonymous") || > 67 output.getStdout().contains("Unsafe anonymous")) { Hi Mandy, thanks for the review. These lines are supposed to look for classes that are not loaded from CDS, with patterns like this: [0.050s][info ][class,load] LambHello$$Lambda$1/0x0000000800b6d840 source: LambHello [0.050s][debug][class,load]? klass: 0x0000000800b6d840 super: 0x0000000800003440 interfaces: 0x00000008000e2b30 loader: [loader data: 0x00007f851c30bb50 for instance a 'jdk/internal/loader/ClassLoaders$AppClassLoader'{0x0000000451017380} unsafe anonymous] bytes: 315 checksum: aa68e916 but the "unsafe anonymous" part is no longer printed out by the nestmate branch due to the hidden class changes. In the latest fix, I changed the code to test for the "bytes: 315 checksum: aa68e916" part of the log, which is more reliable in finding out any non-CDS classes. http://cr.openjdk.java.net/~iklam/jdk15/8240548-NoClassToArchive.graal.v02/ Thanks - Ioi > Do you mean to filter hidden classes? there are duplicated checks for > unsafe anonymous. LMF will be defining hidden classes instead of VM > anonymous class by JEP 371. Mandy >> This test should not be executed with graal. >> >> This test started failing in valhalla/nestmate because the string >> "unsafe anonymous" is no longer printed by -Xlog:class+load=trace. >> However, >> the underlying problem is unrelated to nestmate. So I will fix the >> test in >> the mainline jdk/jdk and it can be synced down to nestmate. >> >> The test assumes that when "java -version" is executed, only a very >> limited >> number of classes are loaded, and all of those are loaded from the >> default >> shared archive. >> >> However, when graal is used as the JIT, many extra classes are loaded >> during VM start-up. Some of those are loaded dynamically from jrt:/. >> Some classes are also defined by LambdaMetafactory. This causes >> complexity >> that cannot be easily handled by this test. >> >> The VM code covered by this test can be sufficiently tested with >> C1/C2. So >> there's no need to bend over backwards to run this test with graal. >> >> Thanks >> - Ioi > From ioi.lam at oracle.com Tue Mar 10 16:30:30 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 10 Mar 2020 09:30:30 -0700 Subject: RFR(XS) 8240548 [TESTBUG] CDS NoClassToArchive.java fails with Graal In-Reply-To: <40b3e05c-ef1d-fab6-b76b-f2db33056f73@oracle.com> References: <9fd602d1-0e2c-5920-d677-8f2d0a1616be@oracle.com> <40b3e05c-ef1d-fab6-b76b-f2db33056f73@oracle.com> Message-ID: <9ca00863-c388-5b18-f962-9f9bcf73ca96@oracle.com> On 3/8/20 10:41 PM, David Holmes wrote: > Hi Ioi, > > On 9/03/2020 2:52 pm, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8240548 >> http://cr.openjdk.java.net/~iklam/jdk15/8240548-NoClassToArchive.graal.v01/ >> >> >> This test should not be executed with graal. >> >> This test started failing in valhalla/nestmate because the string >> "unsafe anonymous" is no longer printed by -Xlog:class+load=trace. >> However, >> the underlying problem is unrelated to nestmate. So I will fix the >> test in >> the mainline jdk/jdk and it can be synced down to nestmate. >> >> The test assumes that when "java -version" is executed, only a very >> limited >> number of classes are loaded, and all of those are loaded from the >> default >> shared archive. >> >> However, when graal is used as the JIT, many extra classes are loaded >> during VM start-up. Some of those are loaded dynamically from jrt:/. >> Some classes are also defined by LambdaMetafactory. This causes >> complexity >> that cannot be easily handled by this test. >> >> The VM code covered by this test can be sufficiently tested with >> C1/C2. So >> there's no need to bend over backwards to run this test with graal. > > Okay. I'd probably have that explanatory comment up the top of the > file rather than within doTest(), but that's just a nit. > Hi David, I moved the comments into the top of the file: http://cr.openjdk.java.net/~iklam/jdk15/8240548-NoClassToArchive.graal.v02/ Thanks - Ioi > Thanks, > David > >> Thanks >> - Ioi From ioi.lam at oracle.com Tue Mar 10 16:38:00 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 10 Mar 2020 09:38:00 -0700 Subject: RFR(XS): 8240691: serviceability/sa/ClhsdbCDSJstackPrintAll.java and serviceability/sa/ClhsdbCDSCore.java not work with ZGC In-Reply-To: References: <86878457-3386-ea0e-f23e-7ad1bdff64a6@oracle.com> <090873f5-91da-5c96-bc0c-3fb9992ccd5e@oracle.com> Message-ID: <68008ddf-7838-a6a4-f85a-06ac1aca6dcc@oracle.com> Hi Chris, The test uses this to check if CDS is on: ??? List cmds = List.of("flags UseSharedSpaces"); ??? String useSharedSpacesOutput = test.run(theApp.getPid(), cmds, ??????????????????????????????????????????????????? null, null); ??? .... ??? if (!useSharedSpacesOutput.contains("true")) { ??????? // CDS archive is not mapped. Skip the rest of the test. The output looks like this: ??? hsdb> hsdb> + verbose true ??? hsdb> + flags UseSharedSpaces ??? UseSharedSpaces = false command line ??? hsdb> + quit I think at one point the "if" worked, but the "+ verbose true" part of the log must have been added recently to cause the "if" to determine the wrong result. Thanks - Ioi On 3/9/20 9:33 PM, Chris Plummer wrote: > Hi Yumin, > > While your fix is a more bullet proof way of checking if CDS is on, > what was broken with the original version (was there a stray "false" > in the output"), and what does this have to do with ZGC? > > thanks, > > Chris > > On 3/9/20 8:06 PM, Yumin Qi wrote: >> >> Hi, >> >> ??? I made a change after discussed with Ioi, the test should check >> against UseSharedSpaces instead, it passed with/without change of >> 8240563 (tested in local). Also passed tier1,2,3. >> >> ???? URL: http://cr.openjdk.java.net/~minqi/8240691/webrev/ >> >> >> ??? Thanks >> >> ???? Yumin >> >> >> On 3/6/20 2:26 PM, Yumin Qi wrote: >>> HI, >>> >>> ? Can I have your reviews on >>> >>> ? Bug: https://bugs.openjdk.java.net/browse/JDK-8240691 >>> >>> ? webrev: http://cr.openjdk.java.net/~minqi/8240691/webrev/ >>> >>> ? Summary: With JDK-8232069 >>> (https://bugs.openjdk.java.net/browse/JDK-8232069), CDS works with >>> UseCompressedOop and UseCompressedClassPointers off. The code for >>> detecting CDS will report false with ZGC since ZGC turned off those >>> two flags. The detection code will be changed in >>> JDK-8240563(https://bugs.openjdk.java.net/browse/JDK-8240563), this >>> will cause those two tests fail. >>> >>> >>> ?? Tests: jtreg to show the tests not selected with ZGC. >>> >>> ?? Thanks >>> >>> ?? Yumin >>> > From vladimir.kozlov at oracle.com Tue Mar 10 18:06:43 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 10 Mar 2020 11:06:43 -0700 Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is inconsistent In-Reply-To: References: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com> <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com> <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com> Message-ID: <1ee27254-5fe9-9876-248d-3c8f166f7de8@oracle.com> On 3/10/20 8:04 AM, Andrew Haley wrote: > On 3/9/20 12:20 PM, Stefan Karlsson wrote: >> On 2020-03-09 11:54, Andrew Haley wrote: >>> So, my plan is to push http://cr.openjdk.java.net/~aph/8240615-1/ and >>> subsequently patch a few AArch64 cases plus the x86_64 immL_Pow2 and >>> immL_NotPow2 cases that we already identified. >>> >>> I do not intend to do bulk changes to any other clients of >>> is_power_of_2() because it's often unclear exactly what behaviour is >>> required, and I'd risk breaking something. >>> >>> OK? >> >> I would prefer if the x86_64 and aarch64 patches were pushed first, or >> folded into this patch. Otherwise, we knowingly introduce a regression >> by pushing the proposed patch above. However, I'll leave it up to the >> compiler devs/maintainers to decide if this is OK. > > http://cr.openjdk.java.net/~aph/8240615-2/ > > OK? > From vladimir.kozlov at oracle.com Tue Mar 10 18:06:40 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 10 Mar 2020 11:06:40 -0700 Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is inconsistent In-Reply-To: References: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com> <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com> <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com> Message-ID: <4d400304-d5cb-12ba-d772-3f594308355a@oracle.com> Why you did not cast constant to (julong) in btrL_mem_imm() x86_64.ad? I looked through C1 and C2 code usage of is_power_of_2(). And it is not clear to me if we handle MAX and MIN values correctly in some cases (mulnode.cpp, divnode.cpp). But it is a different issue. I think your is_power_of_2() change should be fine for C2 and C1 code. Thanks, Vladimir On 3/10/20 8:04 AM, Andrew Haley wrote: > On 3/9/20 12:20 PM, Stefan Karlsson wrote: >> On 2020-03-09 11:54, Andrew Haley wrote: >>> So, my plan is to push http://cr.openjdk.java.net/~aph/8240615-1/ and >>> subsequently patch a few AArch64 cases plus the x86_64 immL_Pow2 and >>> immL_NotPow2 cases that we already identified. >>> >>> I do not intend to do bulk changes to any other clients of >>> is_power_of_2() because it's often unclear exactly what behaviour is >>> required, and I'd risk breaking something. >>> >>> OK? >> >> I would prefer if the x86_64 and aarch64 patches were pushed first, or >> folded into this patch. Otherwise, we knowingly introduce a regression >> by pushing the proposed patch above. However, I'll leave it up to the >> compiler devs/maintainers to decide if this is OK. > > http://cr.openjdk.java.net/~aph/8240615-2/ > > OK? > From vladimir.x.ivanov at oracle.com Tue Mar 10 18:07:36 2020 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 10 Mar 2020 21:07:36 +0300 Subject: RFC(S) 8234146: [TESTBUG] compiler/jsr292/ContinuousCallSiteTargetChange.java times out on SPARC In-Reply-To: References: Message-ID: <38086d99-43fe-c88d-a483-82b934eb4a81@oracle.com> Hi Richard, Nice analysis! I'd prefer to go with the test fix (option #A) for now. The idea to short-circuit repeated traversals over CallSite context looks clever, but I'm not yet persuaded it's worth the added complexity for such a pathological case. So, I propose to fix the test case and file an RFE for the inefficiency you spotted in dependency invalidation. [...] > Option A: Force the NMethodSweeper using the whitebox api. > Webrev A: http://cr.openjdk.java.net/~rrich/webrevs/8234146/A/webrev.0/ Looks good. Best regards, Vladimir Ivanov > > Option B: Specialize DependencyContext::mark_dependent_nmethods() for call site target changes, > i.e. flag list elements that are marked for deoptimization. Upon the next target change only new > dependents are marked for deoptimization. The list iteration is ended as soon as an element is > encountered that is already flagged. > Webrev B: http://cr.openjdk.java.net/~rrich/webrevs/8234146/B/webrev.0/ > > Personally I like B better, because I think it is simple and straight forward: if the target > changes, then all dependents need to be invalidated. By construction of the list we can stop the > invalidation if we find an element that was invalidated before. This specialization is of course > less generic, and there might be new types of call site dependencies for which it is illegal, but > currently I can't think of any. > > Curious about your comments... > > Thanks, > Richard. > > [1] To simulate load you can pin the jvm process to one cpu (e.g. on Linux with "taskset 1 java ...") > or > wget http://cr.openjdk.java.net/~rrich/webrevs/8234146/CPULoadGenerator.java && java CPULoadGenerator.java N > where N is the number of CPUs of your box > From ioi.lam at oracle.com Tue Mar 10 18:11:03 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 10 Mar 2020 11:11:03 -0700 Subject: RFR(XS): 8240691: serviceability/sa/ClhsdbCDSJstackPrintAll.java and serviceability/sa/ClhsdbCDSCore.java not work with ZGC In-Reply-To: <9861adc9-6060-2538-6267-a0c4e8e4ab18@oracle.com> References: <86878457-3386-ea0e-f23e-7ad1bdff64a6@oracle.com> <090873f5-91da-5c96-bc0c-3fb9992ccd5e@oracle.com> <68008ddf-7838-a6a4-f85a-06ac1aca6dcc@oracle.com> <9861adc9-6060-2538-6267-a0c4e8e4ab18@oracle.com> Message-ID: On 3/10/20 10:26 AM, Chris Plummer wrote: > Ok, that makes sense. I committed a change recently that issues an > "echo true" command so all commands that follow are echoed, which is > why you now see "verbose true" in the output. So this resulted in it > always thinking that shared spaces was enabled, even when not. But > this did not cause any difference in behavior I think because of > JDK-8240698 (I'm still not real clear on that). > > And after fixing this issue and JDK-8240698 we will then start to see > issues because ZGC does not support SA, and therefore need a 3rd bug > to fix that? > Hi Chris, I think we need to fix things in the following order. The goal is to increase test coverage for CDS. [1] 8240691 - (I renamed the bug to) "ClhsdbCDSJstackPrintAll incorrectly thinks CDS is in use") After the above is fixed, we still won't run ClhsdbCDSJstackPrintAll for jtreg -vmoption:-XX:+UseZGC yet, because this test has "@requires vm.cds", which reports false when ZGC is selected. [2] 8240563 - WB_IsCDSIncludedInVmBuild should support uncompressed oops/klasses After the above is fixed, we can run a lot of *other* CDS tests with CDS+ZGC. However, for ClhsdbCDSJstackPrintAll, when jtreg -vmoption:-XX:+UseZGC is used + the test dumps a CDS archive with ZGC enabled (use compressed oops = false) + the test tries to use the CDS archive without enabling ZGC (due to 8240698), so compressed oops = true. As a result, the VM cannot load this CDS archive because the oop compression mode doesn't match. [3] 8240698 - LingeredApp did not pass getTestJavaOpts() to the children process if vmArguments is already specified After the above is fixed, ClhsdbCDSJstackPrintAll will dump and run with the archive with -XX+UseZGC, so we can test the behavior of SA when ZGC is enabled. But as Chris said, SA doesn't support ZGC yet (JDK-8220624), so we probably don't need to fix 8240698 until SA supports ZGC. As a precaution, we should disable this test in ProblemList-zgc.txt as a part of 8240563. Also, I think the behavior of 8240698 may be intentional -- in many cases, especially when with "jtreg -vmoption:-Xcomp", you probably don't want the LingerApp to run with -Xcomp. So we should have an API in LingerApp.startApp() to explicitly allow the passing of getTestJavaOpts() to the child process. Thanks - Ioi > Chris > > On 3/10/20 9:38 AM, Ioi Lam wrote: >> Hi Chris, >> >> >> The test uses this to check if CDS is on: >> >> ??? List cmds = List.of("flags UseSharedSpaces"); >> ??? String useSharedSpacesOutput = test.run(theApp.getPid(), cmds, >> ??????????????????????????????????????????????????? null, null); >> ??? .... >> ??? if (!useSharedSpacesOutput.contains("true")) { >> ??????? // CDS archive is not mapped. Skip the rest of the test. >> >> The output looks like this: >> >> ??? hsdb> hsdb> + verbose true >> ??? hsdb> + flags UseSharedSpaces >> ??? UseSharedSpaces = false command line >> ??? hsdb> + quit >> >> I think at one point the "if" worked, but the "+ verbose true" part >> of the log must have been added recently to cause the "if" to >> determine the wrong result. >> >> Thanks >> - Ioi >> >> >> >> On 3/9/20 9:33 PM, Chris Plummer wrote: >>> Hi Yumin, >>> >>> While your fix is a more bullet proof way of checking if CDS is on, >>> what was broken with the original version (was there a stray "false" >>> in the output"), and what does this have to do with ZGC? >>> >>> thanks, >>> >>> Chris >>> >>> On 3/9/20 8:06 PM, Yumin Qi wrote: >>>> >>>> Hi, >>>> >>>> ??? I made a change after discussed with Ioi, the test should check >>>> against UseSharedSpaces instead, it passed with/without change of >>>> 8240563 (tested in local). Also passed tier1,2,3. >>>> >>>> ???? URL: http://cr.openjdk.java.net/~minqi/8240691/webrev/ >>>> >>>> >>>> ??? Thanks >>>> >>>> ???? Yumin >>>> >>>> >>>> On 3/6/20 2:26 PM, Yumin Qi wrote: >>>>> HI, >>>>> >>>>> ? Can I have your reviews on >>>>> >>>>> ? Bug: https://bugs.openjdk.java.net/browse/JDK-8240691 >>>>> >>>>> ? webrev: http://cr.openjdk.java.net/~minqi/8240691/webrev/ >>>>> >>>>> ? Summary: With JDK-8232069 >>>>> (https://bugs.openjdk.java.net/browse/JDK-8232069), CDS works with >>>>> UseCompressedOop and UseCompressedClassPointers off. The code for >>>>> detecting CDS will report false with ZGC since ZGC turned off >>>>> those two flags. The detection code will be changed in >>>>> JDK-8240563(https://bugs.openjdk.java.net/browse/JDK-8240563), >>>>> this will cause those two tests fail. >>>>> >>>>> >>>>> ?? Tests: jtreg to show the tests not selected with ZGC. >>>>> >>>>> ?? Thanks >>>>> >>>>> ?? Yumin >>>>> >>> >> > From aph at redhat.com Tue Mar 10 18:25:10 2020 From: aph at redhat.com (Andrew Haley) Date: Tue, 10 Mar 2020 18:25:10 +0000 Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is inconsistent In-Reply-To: <4d400304-d5cb-12ba-d772-3f594308355a@oracle.com> References: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com> <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com> <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com> <4d400304-d5cb-12ba-d772-3f594308355a@oracle.com> Message-ID: On 3/10/20 6:06 PM, Vladimir Kozlov wrote: > Why you did not cast constant to (julong) in btrL_mem_imm() x86_64.ad? I don't think that it's necessary. (~$con$$constant) returns a jlong which gets passed to log2_long(julong x), so it gets converted to from julong to julong at the point of the call. > I looked through C1 and C2 code usage of is_power_of_2(). And it is not clear to me if we handle MAX and MIN values > correctly in some cases (mulnode.cpp, divnode.cpp). But it is a different issue. > > I think your is_power_of_2() change should be fine for C2 and C1 code. OK, thanks. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Tue Mar 10 18:28:34 2020 From: aph at redhat.com (Andrew Haley) Date: Tue, 10 Mar 2020 18:28:34 +0000 Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is inconsistent In-Reply-To: References: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com> <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com> <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com> <4d400304-d5cb-12ba-d772-3f594308355a@oracle.com> Message-ID: <0d9a51b2-f366-70e0-b0ac-aab185b6081a@redhat.com> On 3/10/20 6:25 PM, Andrew Haley wrote: > I don't think that it's necessary. (~$con$$constant) returns a jlong > which gets passed to log2_long(julong x), so it gets converted to > from julong to julong at the point of the call. Uh, from jlong to julong. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From yumin.qi at oracle.com Tue Mar 10 18:31:52 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Tue, 10 Mar 2020 11:31:52 -0700 Subject: RFR(XS): 8240691: serviceability/sa/ClhsdbCDSJstackPrintAll.java and serviceability/sa/ClhsdbCDSCore.java not work with ZGC In-Reply-To: References: <86878457-3386-ea0e-f23e-7ad1bdff64a6@oracle.com> <090873f5-91da-5c96-bc0c-3fb9992ccd5e@oracle.com> <68008ddf-7838-a6a4-f85a-06ac1aca6dcc@oracle.com> <9861adc9-6060-2538-6267-a0c4e8e4ab18@oracle.com> Message-ID: <50955cca-0c28-3d73-abc1-25a933fa33f1@oracle.com> Hi, Ioi ? Thanks for clearing the whole things. ? Since it is a trivial fix, I will change the synopsis of the bug at push using your updated version: ClhsdbCDSJstackPrintAll incorrectly thinks CDS is in use ?? Let me know if it is OK for push. Thanks Yumin On 3/10/20 11:11 AM, Ioi Lam wrote: > > > On 3/10/20 10:26 AM, Chris Plummer wrote: >> Ok, that makes sense. I committed a change recently that issues an >> "echo true" command so all commands that follow are echoed, which is >> why you now see "verbose true" in the output. So this resulted in it >> always thinking that shared spaces was enabled, even when not. But >> this did not cause any difference in behavior I think because of >> JDK-8240698 (I'm still not real clear on that). >> >> And after fixing this issue and JDK-8240698 we will then start to see >> issues because ZGC does not support SA, and therefore need a 3rd bug >> to fix that? >> > > Hi Chris, > > > I think we need to fix things in the following order. The goal is to > increase test coverage for CDS. > > > > [1] 8240691 - (I renamed the bug to) "ClhsdbCDSJstackPrintAll > incorrectly thinks CDS is in use") > > After the above is fixed, we still won't run ClhsdbCDSJstackPrintAll > for jtreg -vmoption:-XX:+UseZGC yet, because this test has "@requires > vm.cds", which reports false when ZGC is selected. > > > > [2] 8240563 - WB_IsCDSIncludedInVmBuild should support uncompressed > oops/klasses > > After the above is fixed, we can run a lot of *other* CDS tests with > CDS+ZGC. > > However, for ClhsdbCDSJstackPrintAll, when jtreg -vmoption:-XX:+UseZGC > is used > > + the test dumps a CDS archive with ZGC enabled (use compressed oops = > false) > > + the test tries to use the CDS archive without enabling ZGC (due to > 8240698), so compressed oops = true. As a result, the VM cannot load > this CDS archive because the oop compression mode doesn't match. > > > > [3] 8240698 - LingeredApp did not pass getTestJavaOpts() to the > children process if vmArguments is already specified > > After the above is fixed, ClhsdbCDSJstackPrintAll will dump and run > with the archive with -XX+UseZGC, so we can test the behavior of SA > when ZGC is enabled. > > > > But as Chris said, SA doesn't support ZGC yet (JDK-8220624), so we > probably don't need to fix 8240698 until SA supports ZGC. As a > precaution, we should disable this test in ProblemList-zgc.txt as a > part of 8240563. > > Also, I think the behavior of 8240698 may be intentional -- in many > cases, especially when with "jtreg -vmoption:-Xcomp", you probably > don't want the LingerApp to run with -Xcomp. So we should have an API > in LingerApp.startApp() to explicitly allow the passing of > getTestJavaOpts() to the child process. > > Thanks > - Ioi > > >> Chris >> >> On 3/10/20 9:38 AM, Ioi Lam wrote: >>> Hi Chris, >>> >>> >>> The test uses this to check if CDS is on: >>> >>> ??? List cmds = List.of("flags UseSharedSpaces"); >>> ??? String useSharedSpacesOutput = test.run(theApp.getPid(), cmds, >>> ??????????????????????????????????????????????????? null, null); >>> ??? .... >>> ??? if (!useSharedSpacesOutput.contains("true")) { >>> ??????? // CDS archive is not mapped. Skip the rest of the test. >>> >>> The output looks like this: >>> >>> ??? hsdb> hsdb> + verbose true >>> ??? hsdb> + flags UseSharedSpaces >>> ??? UseSharedSpaces = false command line >>> ??? hsdb> + quit >>> >>> I think at one point the "if" worked, but the "+ verbose true" part >>> of the log must have been added recently to cause the "if" to >>> determine the wrong result. >>> >>> Thanks >>> - Ioi >>> >>> >>> >>> On 3/9/20 9:33 PM, Chris Plummer wrote: >>>> Hi Yumin, >>>> >>>> While your fix is a more bullet proof way of checking if CDS is on, >>>> what was broken with the original version (was there a stray >>>> "false" in the output"), and what does this have to do with ZGC? >>>> >>>> thanks, >>>> >>>> Chris >>>> >>>> On 3/9/20 8:06 PM, Yumin Qi wrote: >>>>> >>>>> Hi, >>>>> >>>>> ??? I made a change after discussed with Ioi, the test should >>>>> check against UseSharedSpaces instead, it passed with/without >>>>> change of 8240563 (tested in local). Also passed tier1,2,3. >>>>> >>>>> ???? URL: http://cr.openjdk.java.net/~minqi/8240691/webrev/ >>>>> >>>>> >>>>> ??? Thanks >>>>> >>>>> ???? Yumin >>>>> >>>>> >>>>> On 3/6/20 2:26 PM, Yumin Qi wrote: >>>>>> HI, >>>>>> >>>>>> ? Can I have your reviews on >>>>>> >>>>>> ? Bug: https://bugs.openjdk.java.net/browse/JDK-8240691 >>>>>> >>>>>> ? webrev: http://cr.openjdk.java.net/~minqi/8240691/webrev/ >>>>>> >>>>>> ? Summary: With JDK-8232069 >>>>>> (https://bugs.openjdk.java.net/browse/JDK-8232069), CDS works >>>>>> with UseCompressedOop and UseCompressedClassPointers off. The >>>>>> code for detecting CDS will report false with ZGC since ZGC >>>>>> turned off those two flags. The detection code will be changed in >>>>>> JDK-8240563(https://bugs.openjdk.java.net/browse/JDK-8240563), >>>>>> this will cause those two tests fail. >>>>>> >>>>>> >>>>>> ?? Tests: jtreg to show the tests not selected with ZGC. >>>>>> >>>>>> ?? Thanks >>>>>> >>>>>> ?? Yumin >>>>>> >>>> >>> >> > From ioi.lam at oracle.com Tue Mar 10 18:40:04 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 10 Mar 2020 11:40:04 -0700 Subject: RFR: 8240772: x86_64: Pre-generate Assembler::popa, pusha and vzeroupper In-Reply-To: <9fc3ea60-ccfb-7250-c1e0-06c40088fe7e@oracle.com> References: <9fc3ea60-ccfb-7250-c1e0-06c40088fe7e@oracle.com> Message-ID: <87ca51f4-6d98-8d71-d00c-3815367b3e1f@oracle.com> Hi Claes, This is a really good optimization! Small bang for big bucks! I have a suggestion code coding style: Rename Assembler::popa to Assembler::popa_slow(); void Assembler::popa() { // 64bit ? if (!precomputed) { ??? precompute_instructions(); ? } ? copy_precomputed_instructions(popa_code, popa_len); } static void precompute_instructions() { ? ... ? MacroAssembler masm(&buffer); ? address begin_popa? = masm.code_section()->end(); ? masm.popa_slow(); ? address end_popa??? = masm.code_section()->end(); ? ... } ---- Also, maybe you can add this assert after generating the code for all 3 macros: ? assert(masm->code()->total_relocation_size() == 0 && masm->code()->total_oop_size() == 0 && masm->code()->total_metadata_size() == 0, ???????? "precomputed code cannot have any of these"); Thanks! - Ioi On 3/10/20 6:46 AM, Claes Redestad wrote: > Hi, > > calculate some invariant Assembler routines at bootstrap, copy on > subsequent invocations. > > For popa and pusha this means an overhead reduction of around 98% (from > ~2500 instructions to emit a pusha to ~50). For vzeroupper an overhead > reduction of ~65% (117 -> 42). Together these add up to about a 1% > reduction of instructions executed on a Hello World - with some > (smaller) scaling impact on larger applications. > > The initialization is very simple/naive, i.e., lacks any kind of > synchronization protocol. But as this setup is guaranteed to happen very > early during bootstrap this should be fine. Thanks Ioi for some helpful > suggestions here! > > Bug:??? https://bugs.openjdk.java.net/browse/JDK-8240772 > Webrev: http://cr.openjdk.java.net/~redestad/8240772/open.00/ > > Testing: tier1-3 > > Thanks! > > /Claes From ioi.lam at oracle.com Tue Mar 10 18:40:43 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 10 Mar 2020 11:40:43 -0700 Subject: RFR(XS): 8240691: serviceability/sa/ClhsdbCDSJstackPrintAll.java and serviceability/sa/ClhsdbCDSCore.java not work with ZGC In-Reply-To: <50955cca-0c28-3d73-abc1-25a933fa33f1@oracle.com> References: <86878457-3386-ea0e-f23e-7ad1bdff64a6@oracle.com> <090873f5-91da-5c96-bc0c-3fb9992ccd5e@oracle.com> <68008ddf-7838-a6a4-f85a-06ac1aca6dcc@oracle.com> <9861adc9-6060-2538-6267-a0c4e8e4ab18@oracle.com> <50955cca-0c28-3d73-abc1-25a933fa33f1@oracle.com> Message-ID: <6c07e87c-82ec-823b-3fd5-4c5d5a347340@oracle.com> Looks good to me. Thanks! - Ioi On 3/10/20 11:31 AM, Yumin Qi wrote: > Hi, Ioi > ? Thanks for clearing the whole things. > ? Since it is a trivial fix, I will change the synopsis of the bug at > push using your updated version: ClhsdbCDSJstackPrintAll incorrectly > thinks CDS is in use > > ?? Let me know if it is OK for push. > > Thanks > Yumin > > On 3/10/20 11:11 AM, Ioi Lam wrote: >> >> >> On 3/10/20 10:26 AM, Chris Plummer wrote: >>> Ok, that makes sense. I committed a change recently that issues an >>> "echo true" command so all commands that follow are echoed, which is >>> why you now see "verbose true" in the output. So this resulted in it >>> always thinking that shared spaces was enabled, even when not. But >>> this did not cause any difference in behavior I think because of >>> JDK-8240698 (I'm still not real clear on that). >>> >>> And after fixing this issue and JDK-8240698 we will then start to >>> see issues because ZGC does not support SA, and therefore need a 3rd >>> bug to fix that? >>> >> >> Hi Chris, >> >> >> I think we need to fix things in the following order. The goal is to >> increase test coverage for CDS. >> >> >> >> [1] 8240691 - (I renamed the bug to) "ClhsdbCDSJstackPrintAll >> incorrectly thinks CDS is in use") >> >> After the above is fixed, we still won't run ClhsdbCDSJstackPrintAll >> for jtreg -vmoption:-XX:+UseZGC yet, because this test has "@requires >> vm.cds", which reports false when ZGC is selected. >> >> >> >> [2] 8240563 - WB_IsCDSIncludedInVmBuild should support uncompressed >> oops/klasses >> >> After the above is fixed, we can run a lot of *other* CDS tests with >> CDS+ZGC. >> >> However, for ClhsdbCDSJstackPrintAll, when jtreg >> -vmoption:-XX:+UseZGC is used >> >> + the test dumps a CDS archive with ZGC enabled (use compressed oops >> = false) >> >> + the test tries to use the CDS archive without enabling ZGC (due to >> 8240698), so compressed oops = true. As a result, the VM cannot load >> this CDS archive because the oop compression mode doesn't match. >> >> >> >> [3] 8240698 - LingeredApp did not pass getTestJavaOpts() to the >> children process if vmArguments is already specified >> >> After the above is fixed, ClhsdbCDSJstackPrintAll will dump and run >> with the archive with -XX+UseZGC, so we can test the behavior of SA >> when ZGC is enabled. >> >> >> >> But as Chris said, SA doesn't support ZGC yet (JDK-8220624), so we >> probably don't need to fix 8240698 until SA supports ZGC. As a >> precaution, we should disable this test in ProblemList-zgc.txt as a >> part of 8240563. >> >> Also, I think the behavior of 8240698 may be intentional -- in many >> cases, especially when with "jtreg -vmoption:-Xcomp", you probably >> don't want the LingerApp to run with -Xcomp. So we should have an API >> in LingerApp.startApp() to explicitly allow the passing of >> getTestJavaOpts() to the child process. >> >> Thanks >> - Ioi >> >> >>> Chris >>> >>> On 3/10/20 9:38 AM, Ioi Lam wrote: >>>> Hi Chris, >>>> >>>> >>>> The test uses this to check if CDS is on: >>>> >>>> ??? List cmds = List.of("flags UseSharedSpaces"); >>>> ??? String useSharedSpacesOutput = test.run(theApp.getPid(), cmds, >>>> ??????????????????????????????????????????????????? null, null); >>>> ??? .... >>>> ??? if (!useSharedSpacesOutput.contains("true")) { >>>> ??????? // CDS archive is not mapped. Skip the rest of the test. >>>> >>>> The output looks like this: >>>> >>>> ??? hsdb> hsdb> + verbose true >>>> ??? hsdb> + flags UseSharedSpaces >>>> ??? UseSharedSpaces = false command line >>>> ??? hsdb> + quit >>>> >>>> I think at one point the "if" worked, but the "+ verbose true" part >>>> of the log must have been added recently to cause the "if" to >>>> determine the wrong result. >>>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>> >>>> On 3/9/20 9:33 PM, Chris Plummer wrote: >>>>> Hi Yumin, >>>>> >>>>> While your fix is a more bullet proof way of checking if CDS is >>>>> on, what was broken with the original version (was there a stray >>>>> "false" in the output"), and what does this have to do with ZGC? >>>>> >>>>> thanks, >>>>> >>>>> Chris >>>>> >>>>> On 3/9/20 8:06 PM, Yumin Qi wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> ??? I made a change after discussed with Ioi, the test should >>>>>> check against UseSharedSpaces instead, it passed with/without >>>>>> change of 8240563 (tested in local). Also passed tier1,2,3. >>>>>> >>>>>> ???? URL: http://cr.openjdk.java.net/~minqi/8240691/webrev/ >>>>>> >>>>>> >>>>>> ??? Thanks >>>>>> >>>>>> ???? Yumin >>>>>> >>>>>> >>>>>> On 3/6/20 2:26 PM, Yumin Qi wrote: >>>>>>> HI, >>>>>>> >>>>>>> ? Can I have your reviews on >>>>>>> >>>>>>> ? Bug: https://bugs.openjdk.java.net/browse/JDK-8240691 >>>>>>> >>>>>>> ? webrev: http://cr.openjdk.java.net/~minqi/8240691/webrev/ >>>>>>> >>>>>>> ? Summary: With JDK-8232069 >>>>>>> (https://bugs.openjdk.java.net/browse/JDK-8232069), CDS works >>>>>>> with UseCompressedOop and UseCompressedClassPointers off. The >>>>>>> code for detecting CDS will report false with ZGC since ZGC >>>>>>> turned off those two flags. The detection code will be changed >>>>>>> in >>>>>>> JDK-8240563(https://bugs.openjdk.java.net/browse/JDK-8240563), >>>>>>> this will cause those two tests fail. >>>>>>> >>>>>>> >>>>>>> ?? Tests: jtreg to show the tests not selected with ZGC. >>>>>>> >>>>>>> ?? Thanks >>>>>>> >>>>>>> ?? Yumin >>>>>>> >>>>> >>>> >>> >> > From yumin.qi at oracle.com Tue Mar 10 18:53:58 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Tue, 10 Mar 2020 11:53:58 -0700 Subject: RFR(XS): 8240691: serviceability/sa/ClhsdbCDSJstackPrintAll.java and serviceability/sa/ClhsdbCDSCore.java not work with ZGC In-Reply-To: <6c07e87c-82ec-823b-3fd5-4c5d5a347340@oracle.com> References: <86878457-3386-ea0e-f23e-7ad1bdff64a6@oracle.com> <090873f5-91da-5c96-bc0c-3fb9992ccd5e@oracle.com> <68008ddf-7838-a6a4-f85a-06ac1aca6dcc@oracle.com> <9861adc9-6060-2538-6267-a0c4e8e4ab18@oracle.com> <50955cca-0c28-3d73-abc1-25a933fa33f1@oracle.com> <6c07e87c-82ec-823b-3fd5-4c5d5a347340@oracle.com> Message-ID: Thanks. Pushed. On 3/10/20 11:40 AM, Ioi Lam wrote: > Looks good to me. Thanks! > > - Ioi > > On 3/10/20 11:31 AM, Yumin Qi wrote: >> Hi, Ioi >> ? Thanks for clearing the whole things. >> ? Since it is a trivial fix, I will change the synopsis of the bug at >> push using your updated version: ClhsdbCDSJstackPrintAll incorrectly >> thinks CDS is in use >> >> ?? Let me know if it is OK for push. >> >> Thanks >> Yumin >> >> On 3/10/20 11:11 AM, Ioi Lam wrote: >>> >>> >>> On 3/10/20 10:26 AM, Chris Plummer wrote: >>>> Ok, that makes sense. I committed a change recently that issues an >>>> "echo true" command so all commands that follow are echoed, which >>>> is why you now see "verbose true" in the output. So this resulted >>>> in it always thinking that shared spaces was enabled, even when >>>> not. But this did not cause any difference in behavior I think >>>> because of JDK-8240698 (I'm still not real clear on that). >>>> >>>> And after fixing this issue and JDK-8240698 we will then start to >>>> see issues because ZGC does not support SA, and therefore need a >>>> 3rd bug to fix that? >>>> >>> >>> Hi Chris, >>> >>> >>> I think we need to fix things in the following order. The goal is to >>> increase test coverage for CDS. >>> >>> >>> >>> [1] 8240691 - (I renamed the bug to) "ClhsdbCDSJstackPrintAll >>> incorrectly thinks CDS is in use") >>> >>> After the above is fixed, we still won't run ClhsdbCDSJstackPrintAll >>> for jtreg -vmoption:-XX:+UseZGC yet, because this test has >>> "@requires vm.cds", which reports false when ZGC is selected. >>> >>> >>> >>> [2] 8240563 - WB_IsCDSIncludedInVmBuild should support uncompressed >>> oops/klasses >>> >>> After the above is fixed, we can run a lot of *other* CDS tests with >>> CDS+ZGC. >>> >>> However, for ClhsdbCDSJstackPrintAll, when jtreg >>> -vmoption:-XX:+UseZGC is used >>> >>> + the test dumps a CDS archive with ZGC enabled (use compressed oops >>> = false) >>> >>> + the test tries to use the CDS archive without enabling ZGC (due to >>> 8240698), so compressed oops = true. As a result, the VM cannot load >>> this CDS archive because the oop compression mode doesn't match. >>> >>> >>> >>> [3] 8240698 - LingeredApp did not pass getTestJavaOpts() to the >>> children process if vmArguments is already specified >>> >>> After the above is fixed, ClhsdbCDSJstackPrintAll will dump and run >>> with the archive with -XX+UseZGC, so we can test the behavior of SA >>> when ZGC is enabled. >>> >>> >>> >>> But as Chris said, SA doesn't support ZGC yet (JDK-8220624), so we >>> probably don't need to fix 8240698 until SA supports ZGC. As a >>> precaution, we should disable this test in ProblemList-zgc.txt as a >>> part of 8240563. >>> >>> Also, I think the behavior of 8240698 may be intentional -- in many >>> cases, especially when with "jtreg -vmoption:-Xcomp", you probably >>> don't want the LingerApp to run with -Xcomp. So we should have an >>> API in LingerApp.startApp() to explicitly allow the passing of >>> getTestJavaOpts() to the child process. >>> >>> Thanks >>> - Ioi >>> >>> >>>> Chris >>>> >>>> On 3/10/20 9:38 AM, Ioi Lam wrote: >>>>> Hi Chris, >>>>> >>>>> >>>>> The test uses this to check if CDS is on: >>>>> >>>>> ??? List cmds = List.of("flags UseSharedSpaces"); >>>>> ??? String useSharedSpacesOutput = test.run(theApp.getPid(), cmds, >>>>> ??????????????????????????????????????????????????? null, null); >>>>> ??? .... >>>>> ??? if (!useSharedSpacesOutput.contains("true")) { >>>>> ??????? // CDS archive is not mapped. Skip the rest of the test. >>>>> >>>>> The output looks like this: >>>>> >>>>> ??? hsdb> hsdb> + verbose true >>>>> ??? hsdb> + flags UseSharedSpaces >>>>> ??? UseSharedSpaces = false command line >>>>> ??? hsdb> + quit >>>>> >>>>> I think at one point the "if" worked, but the "+ verbose true" >>>>> part of the log must have been added recently to cause the "if" to >>>>> determine the wrong result. >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> >>>>> >>>>> On 3/9/20 9:33 PM, Chris Plummer wrote: >>>>>> Hi Yumin, >>>>>> >>>>>> While your fix is a more bullet proof way of checking if CDS is >>>>>> on, what was broken with the original version (was there a stray >>>>>> "false" in the output"), and what does this have to do with ZGC? >>>>>> >>>>>> thanks, >>>>>> >>>>>> Chris >>>>>> >>>>>> On 3/9/20 8:06 PM, Yumin Qi wrote: >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> ??? I made a change after discussed with Ioi, the test should >>>>>>> check against UseSharedSpaces instead, it passed with/without >>>>>>> change of 8240563 (tested in local). Also passed tier1,2,3. >>>>>>> >>>>>>> ???? URL: http://cr.openjdk.java.net/~minqi/8240691/webrev/ >>>>>>> >>>>>>> >>>>>>> ??? Thanks >>>>>>> >>>>>>> ???? Yumin >>>>>>> >>>>>>> >>>>>>> On 3/6/20 2:26 PM, Yumin Qi wrote: >>>>>>>> HI, >>>>>>>> >>>>>>>> ? Can I have your reviews on >>>>>>>> >>>>>>>> ? Bug: https://bugs.openjdk.java.net/browse/JDK-8240691 >>>>>>>> >>>>>>>> ? webrev: http://cr.openjdk.java.net/~minqi/8240691/webrev/ >>>>>>>> >>>>>>>> ? Summary: With JDK-8232069 >>>>>>>> (https://bugs.openjdk.java.net/browse/JDK-8232069), CDS works >>>>>>>> with UseCompressedOop and UseCompressedClassPointers off. The >>>>>>>> code for detecting CDS will report false with ZGC since ZGC >>>>>>>> turned off those two flags. The detection code will be changed >>>>>>>> in >>>>>>>> JDK-8240563(https://bugs.openjdk.java.net/browse/JDK-8240563), >>>>>>>> this will cause those two tests fail. >>>>>>>> >>>>>>>> >>>>>>>> ?? Tests: jtreg to show the tests not selected with ZGC. >>>>>>>> >>>>>>>> ?? Thanks >>>>>>>> >>>>>>>> ?? Yumin >>>>>>>> >>>>>> >>>>> >>>> >>> >> > From vladimir.kozlov at oracle.com Tue Mar 10 18:55:39 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 10 Mar 2020 11:55:39 -0700 Subject: RFR: 8240772: x86_64: Pre-generate Assembler::popa, pusha and vzeroupper In-Reply-To: <9fc3ea60-ccfb-7250-c1e0-06c40088fe7e@oracle.com> References: <9fc3ea60-ccfb-7250-c1e0-06c40088fe7e@oracle.com> Message-ID: <7ae72563-8fc3-0af4-cfc3-8a6e14ec4168@oracle.com> Hi Claes, I don't think you can guaranty that these instructions will be called during bootstrap. I was thinking about calling precompute_instructions() from generate_initial() in stubGenerator_x86_64.cpp so it is definitely called during bootstrap. thanks, Vladimir K On 3/10/20 6:46 AM, Claes Redestad wrote: > Hi, > > calculate some invariant Assembler routines at bootstrap, copy on > subsequent invocations. > > For popa and pusha this means an overhead reduction of around 98% (from > ~2500 instructions to emit a pusha to ~50). For vzeroupper an overhead > reduction of ~65% (117 -> 42). Together these add up to about a 1% > reduction of instructions executed on a Hello World - with some > (smaller) scaling impact on larger applications. > > The initialization is very simple/naive, i.e., lacks any kind of synchronization protocol. But as this setup is > guaranteed to happen very > early during bootstrap this should be fine. Thanks Ioi for some helpful > suggestions here! > > Bug:??? https://bugs.openjdk.java.net/browse/JDK-8240772 > Webrev: http://cr.openjdk.java.net/~redestad/8240772/open.00/ > > Testing: tier1-3 > > Thanks! > > /Claes From vladimir.kozlov at oracle.com Tue Mar 10 18:57:57 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 10 Mar 2020 11:57:57 -0700 Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is inconsistent In-Reply-To: <0d9a51b2-f366-70e0-b0ac-aab185b6081a@redhat.com> References: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com> <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com> <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com> <4d400304-d5cb-12ba-d772-3f594308355a@oracle.com> <0d9a51b2-f366-70e0-b0ac-aab185b6081a@redhat.com> Message-ID: Okay. Thanks, Vladimir On 3/10/20 11:28 AM, Andrew Haley wrote: > On 3/10/20 6:25 PM, Andrew Haley wrote: >> I don't think that it's necessary. (~$con$$constant) returns a jlong >> which gets passed to log2_long(julong x), so it gets converted to >> from julong to julong at the point of the call. > > Uh, from jlong to julong. > From claes.redestad at oracle.com Tue Mar 10 19:19:39 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 10 Mar 2020 20:19:39 +0100 Subject: RFR: 8240772: x86_64: Pre-generate Assembler::popa, pusha and vzeroupper In-Reply-To: <87ca51f4-6d98-8d71-d00c-3815367b3e1f@oracle.com> References: <9fc3ea60-ccfb-7250-c1e0-06c40088fe7e@oracle.com> <87ca51f4-6d98-8d71-d00c-3815367b3e1f@oracle.com> Message-ID: <1ac200e6-1d8f-7476-5cb8-6d44c77f3170@oracle.com> Hi Ioi, good suggestions. I will rework this tomorrow, along with Vladimir's suggestion to add an explicit call to precompute_instructions from the stubGenerator. Thanks! /Claes On 2020-03-10 19:40, Ioi Lam wrote: > Hi Claes, > > This is a really good optimization! Small bang for big bucks! > > I have a suggestion code coding style: > > Rename Assembler::popa to Assembler::popa_slow(); > > void Assembler::popa() { // 64bit > ? if (!precomputed) { > ??? precompute_instructions(); > ? } > ? copy_precomputed_instructions(popa_code, popa_len); > } > > static void precompute_instructions() { > ? ... > ? MacroAssembler masm(&buffer); > > ? address begin_popa? = masm.code_section()->end(); > ? masm.popa_slow(); > ? address end_popa??? = masm.code_section()->end(); > ? ... > } > > ---- > > Also, maybe you can add this assert after generating the code for all 3 > macros: > > ? assert(masm->code()->total_relocation_size() == 0 && > masm->code()->total_oop_size() == 0 && > masm->code()->total_metadata_size() == 0, > ???????? "precomputed code cannot have any of these"); > > > Thanks! > - Ioi > > > > On 3/10/20 6:46 AM, Claes Redestad wrote: >> Hi, >> >> calculate some invariant Assembler routines at bootstrap, copy on >> subsequent invocations. >> >> For popa and pusha this means an overhead reduction of around 98% (from >> ~2500 instructions to emit a pusha to ~50). For vzeroupper an overhead >> reduction of ~65% (117 -> 42). Together these add up to about a 1% >> reduction of instructions executed on a Hello World - with some >> (smaller) scaling impact on larger applications. >> >> The initialization is very simple/naive, i.e., lacks any kind of >> synchronization protocol. But as this setup is guaranteed to happen very >> early during bootstrap this should be fine. Thanks Ioi for some helpful >> suggestions here! >> >> Bug:??? https://bugs.openjdk.java.net/browse/JDK-8240772 >> Webrev: http://cr.openjdk.java.net/~redestad/8240772/open.00/ >> >> Testing: tier1-3 >> >> Thanks! >> >> /Claes > From yumin.qi at oracle.com Tue Mar 10 21:11:14 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Tue, 10 Mar 2020 14:11:14 -0700 Subject: RFR(XS): https://bugs.openjdk.java.net/browse/JDK-8240840 Message-ID: <07c09c54-3378-6450-b120-40543f852020@oracle.com> Hi, ? I just pushed 8240691: https://bugs.openjdk.java.net/browse/JDK-8240691 but wrongly included changes in whitbox.cpp which should not change in this bug. Instead it is for bug 8240563. --- a/src/hotspot/share/prims/whitebox.cpp??? Tue Mar 10 20:51:09 2020 +0300 +++ b/src/hotspot/share/prims/whitebox.cpp??? Tue Mar 10 11:52:53 2020 -0700 @@ -1990,12 +1990,6 @@ ?WB_ENTRY(jboolean, WB_IsCDSIncludedInVmBuild(JNIEnv* env)) ?#if INCLUDE_CDS -# ifdef _LP64 -??? if (!UseCompressedOops || !UseCompressedClassPointers) { -????? // On 64-bit VMs, CDS is supported only with compressed oops/pointers -????? return false; -??? } -# endif // _LP64 ?? return true; ?#else ?? return false; Now I need add it back since it will fail other two test cases: test/hotspot/jtreg/runtime/cds/appcds/CommandLineFlagComboNegative.java test/hotspot/jtreg/runtime/cds/appcds/TestZGCWithCDS.java The diff for roll back is: ?WB_ENTRY(jboolean, WB_IsCDSIncludedInVmBuild(JNIEnv* env)) ?#if INCLUDE_CDS +# ifdef _LP64 +??? if (!UseCompressedOops || !UseCompressedClassPointers) { +????? // On 64-bit VMs, CDS is supported only with compressed oops/pointers +????? return false; +??? } +# endif // _LP64 ?? return true; ?#else ?? return false; Seek approval for this accidental change. Thanks Yumin From ioi.lam at oracle.com Tue Mar 10 21:20:08 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 10 Mar 2020 14:20:08 -0700 Subject: RFR(XS): https://bugs.openjdk.java.net/browse/JDK-8240840 In-Reply-To: <07c09c54-3378-6450-b120-40543f852020@oracle.com> References: <07c09c54-3378-6450-b120-40543f852020@oracle.com> Message-ID: Hi Yumin, Looks good and trivial. thanks - Ioi On 3/10/20 2:11 PM, Yumin Qi wrote: > Hi, > ? I just pushed 8240691: > https://bugs.openjdk.java.net/browse/JDK-8240691 but wrongly included > changes in whitbox.cpp which should not change in this bug. Instead it > is for bug 8240563. > > --- a/src/hotspot/share/prims/whitebox.cpp??? Tue Mar 10 20:51:09 2020 > +0300 > +++ b/src/hotspot/share/prims/whitebox.cpp??? Tue Mar 10 11:52:53 2020 > -0700 > @@ -1990,12 +1990,6 @@ > > ?WB_ENTRY(jboolean, WB_IsCDSIncludedInVmBuild(JNIEnv* env)) > ?#if INCLUDE_CDS > -# ifdef _LP64 > -??? if (!UseCompressedOops || !UseCompressedClassPointers) { > -????? // On 64-bit VMs, CDS is supported only with compressed > oops/pointers > -????? return false; > -??? } > -# endif // _LP64 > ?? return true; > ?#else > ?? return false; > > Now I need add it back since it will fail other two test cases: > test/hotspot/jtreg/runtime/cds/appcds/CommandLineFlagComboNegative.java > test/hotspot/jtreg/runtime/cds/appcds/TestZGCWithCDS.java > > The diff for roll back is: > ?WB_ENTRY(jboolean, WB_IsCDSIncludedInVmBuild(JNIEnv* env)) > ?#if INCLUDE_CDS > +# ifdef _LP64 > +??? if (!UseCompressedOops || !UseCompressedClassPointers) { > +????? // On 64-bit VMs, CDS is supported only with compressed > oops/pointers > +????? return false; > +??? } > +# endif // _LP64 > ?? return true; > ?#else > ?? return false; > > Seek approval for this accidental change. > > Thanks > Yumin From calvin.cheung at oracle.com Tue Mar 10 21:20:30 2020 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Tue, 10 Mar 2020 14:20:30 -0700 Subject: RFR(XS): https://bugs.openjdk.java.net/browse/JDK-8240840 In-Reply-To: <07c09c54-3378-6450-b120-40543f852020@oracle.com> References: <07c09c54-3378-6450-b120-40543f852020@oracle.com> Message-ID: Looks good. thanks, Calvin On 3/10/20 2:11 PM, Yumin Qi wrote: > Hi, > ? I just pushed 8240691: > https://bugs.openjdk.java.net/browse/JDK-8240691 but wrongly included > changes in whitbox.cpp which should not change in this bug. Instead it > is for bug 8240563. > > --- a/src/hotspot/share/prims/whitebox.cpp??? Tue Mar 10 20:51:09 2020 > +0300 > +++ b/src/hotspot/share/prims/whitebox.cpp??? Tue Mar 10 11:52:53 2020 > -0700 > @@ -1990,12 +1990,6 @@ > > ?WB_ENTRY(jboolean, WB_IsCDSIncludedInVmBuild(JNIEnv* env)) > ?#if INCLUDE_CDS > -# ifdef _LP64 > -??? if (!UseCompressedOops || !UseCompressedClassPointers) { > -????? // On 64-bit VMs, CDS is supported only with compressed > oops/pointers > -????? return false; > -??? } > -# endif // _LP64 > ?? return true; > ?#else > ?? return false; > > Now I need add it back since it will fail other two test cases: > test/hotspot/jtreg/runtime/cds/appcds/CommandLineFlagComboNegative.java > test/hotspot/jtreg/runtime/cds/appcds/TestZGCWithCDS.java > > The diff for roll back is: > ?WB_ENTRY(jboolean, WB_IsCDSIncludedInVmBuild(JNIEnv* env)) > ?#if INCLUDE_CDS > +# ifdef _LP64 > +??? if (!UseCompressedOops || !UseCompressedClassPointers) { > +????? // On 64-bit VMs, CDS is supported only with compressed > oops/pointers > +????? return false; > +??? } > +# endif // _LP64 > ?? return true; > ?#else > ?? return false; > > Seek approval for this accidental change. > > Thanks > Yumin From yumin.qi at oracle.com Tue Mar 10 21:26:34 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Tue, 10 Mar 2020 14:26:34 -0700 Subject: RFR(XS): https://bugs.openjdk.java.net/browse/JDK-8240840 In-Reply-To: References: <07c09c54-3378-6450-b120-40543f852020@oracle.com> Message-ID: <8c128fb0-86d4-2c25-20fe-d6e5b551f4e2@oracle.com> Thanks for quick review. Yumin On 3/10/20 2:20 PM, Ioi Lam wrote: > Hi Yumin, > > Looks good and trivial. > > thanks > - Ioi > > On 3/10/20 2:11 PM, Yumin Qi wrote: >> Hi, >> ? I just pushed 8240691: >> https://bugs.openjdk.java.net/browse/JDK-8240691 but wrongly included >> changes in whitbox.cpp which should not change in this bug. Instead >> it is for bug 8240563. >> >> --- a/src/hotspot/share/prims/whitebox.cpp??? Tue Mar 10 20:51:09 >> 2020 +0300 >> +++ b/src/hotspot/share/prims/whitebox.cpp??? Tue Mar 10 11:52:53 >> 2020 -0700 >> @@ -1990,12 +1990,6 @@ >> >> ?WB_ENTRY(jboolean, WB_IsCDSIncludedInVmBuild(JNIEnv* env)) >> ?#if INCLUDE_CDS >> -# ifdef _LP64 >> -??? if (!UseCompressedOops || !UseCompressedClassPointers) { >> -????? // On 64-bit VMs, CDS is supported only with compressed >> oops/pointers >> -????? return false; >> -??? } >> -# endif // _LP64 >> ?? return true; >> ?#else >> ?? return false; >> >> Now I need add it back since it will fail other two test cases: >> test/hotspot/jtreg/runtime/cds/appcds/CommandLineFlagComboNegative.java >> test/hotspot/jtreg/runtime/cds/appcds/TestZGCWithCDS.java >> >> The diff for roll back is: >> ?WB_ENTRY(jboolean, WB_IsCDSIncludedInVmBuild(JNIEnv* env)) >> ?#if INCLUDE_CDS >> +# ifdef _LP64 >> +??? if (!UseCompressedOops || !UseCompressedClassPointers) { >> +????? // On 64-bit VMs, CDS is supported only with compressed >> oops/pointers >> +????? return false; >> +??? } >> +# endif // _LP64 >> ?? return true; >> ?#else >> ?? return false; >> >> Seek approval for this accidental change. >> >> Thanks >> Yumin > From yumin.qi at oracle.com Tue Mar 10 21:26:58 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Tue, 10 Mar 2020 14:26:58 -0700 Subject: RFR(XS): https://bugs.openjdk.java.net/browse/JDK-8240840 In-Reply-To: References: <07c09c54-3378-6450-b120-40543f852020@oracle.com> Message-ID: Calvin ? Thanks! Yumin On 3/10/20 2:20 PM, Calvin Cheung wrote: > Looks good. > > thanks, > Calvin > > On 3/10/20 2:11 PM, Yumin Qi wrote: >> Hi, >> ? I just pushed 8240691: >> https://bugs.openjdk.java.net/browse/JDK-8240691 but wrongly included >> changes in whitbox.cpp which should not change in this bug. Instead >> it is for bug 8240563. >> >> --- a/src/hotspot/share/prims/whitebox.cpp??? Tue Mar 10 20:51:09 >> 2020 +0300 >> +++ b/src/hotspot/share/prims/whitebox.cpp??? Tue Mar 10 11:52:53 >> 2020 -0700 >> @@ -1990,12 +1990,6 @@ >> >> ?WB_ENTRY(jboolean, WB_IsCDSIncludedInVmBuild(JNIEnv* env)) >> ?#if INCLUDE_CDS >> -# ifdef _LP64 >> -??? if (!UseCompressedOops || !UseCompressedClassPointers) { >> -????? // On 64-bit VMs, CDS is supported only with compressed >> oops/pointers >> -????? return false; >> -??? } >> -# endif // _LP64 >> ?? return true; >> ?#else >> ?? return false; >> >> Now I need add it back since it will fail other two test cases: >> test/hotspot/jtreg/runtime/cds/appcds/CommandLineFlagComboNegative.java >> test/hotspot/jtreg/runtime/cds/appcds/TestZGCWithCDS.java >> >> The diff for roll back is: >> ?WB_ENTRY(jboolean, WB_IsCDSIncludedInVmBuild(JNIEnv* env)) >> ?#if INCLUDE_CDS >> +# ifdef _LP64 >> +??? if (!UseCompressedOops || !UseCompressedClassPointers) { >> +????? // On 64-bit VMs, CDS is supported only with compressed >> oops/pointers >> +????? return false; >> +??? } >> +# endif // _LP64 >> ?? return true; >> ?#else >> ?? return false; >> >> Seek approval for this accidental change. >> >> Thanks >> Yumin From david.holmes at oracle.com Tue Mar 10 23:24:23 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 11 Mar 2020 09:24:23 +1000 Subject: RFR(XS) 8240548 [TESTBUG] CDS NoClassToArchive.java fails with Graal In-Reply-To: <9ca00863-c388-5b18-f962-9f9bcf73ca96@oracle.com> References: <9fd602d1-0e2c-5920-d677-8f2d0a1616be@oracle.com> <40b3e05c-ef1d-fab6-b76b-f2db33056f73@oracle.com> <9ca00863-c388-5b18-f962-9f9bcf73ca96@oracle.com> Message-ID: <6f642d47-f7be-b4b4-1925-26465e7bd147@oracle.com> Looks good. Thanks, David On 11/03/2020 2:30 am, Ioi Lam wrote: > > > On 3/8/20 10:41 PM, David Holmes wrote: >> Hi Ioi, >> >> On 9/03/2020 2:52 pm, Ioi Lam wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8240548 >>> http://cr.openjdk.java.net/~iklam/jdk15/8240548-NoClassToArchive.graal.v01/ >>> >>> >>> This test should not be executed with graal. >>> >>> This test started failing in valhalla/nestmate because the string >>> "unsafe anonymous" is no longer printed by -Xlog:class+load=trace. >>> However, >>> the underlying problem is unrelated to nestmate. So I will fix the >>> test in >>> the mainline jdk/jdk and it can be synced down to nestmate. >>> >>> The test assumes that when "java -version" is executed, only a very >>> limited >>> number of classes are loaded, and all of those are loaded from the >>> default >>> shared archive. >>> >>> However, when graal is used as the JIT, many extra classes are loaded >>> during VM start-up. Some of those are loaded dynamically from jrt:/. >>> Some classes are also defined by LambdaMetafactory. This causes >>> complexity >>> that cannot be easily handled by this test. >>> >>> The VM code covered by this test can be sufficiently tested with >>> C1/C2. So >>> there's no need to bend over backwards to run this test with graal. >> >> Okay. I'd probably have that explanatory comment up the top of the >> file rather than within doTest(), but that's just a nit. >> > > Hi David, > > I moved the comments into the top of the file: > > http://cr.openjdk.java.net/~iklam/jdk15/8240548-NoClassToArchive.graal.v02/ > > Thanks > - Ioi > >> Thanks, >> David >> >>> Thanks >>> - Ioi > From ioi.lam at oracle.com Tue Mar 10 23:56:59 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 10 Mar 2020 16:56:59 -0700 Subject: RFR: 8240772: x86_64: Pre-generate Assembler::popa, pusha and vzeroupper In-Reply-To: <87ca51f4-6d98-8d71-d00c-3815367b3e1f@oracle.com> References: <9fc3ea60-ccfb-7250-c1e0-06c40088fe7e@oracle.com> <87ca51f4-6d98-8d71-d00c-3815367b3e1f@oracle.com> Message-ID: On 3/10/20 11:40 AM, Ioi Lam wrote: > Hi Claes, > > This is a really good optimization! Small bang for big bucks! Sorry my grammar has failed me again. I meant the opposite -- big bang for small bucks! - Ioi > > I have a suggestion code coding style: > > Rename Assembler::popa to Assembler::popa_slow(); > > void Assembler::popa() { // 64bit > ? if (!precomputed) { > ??? precompute_instructions(); > ? } > ? copy_precomputed_instructions(popa_code, popa_len); > } > > static void precompute_instructions() { > ? ... > ? MacroAssembler masm(&buffer); > > ? address begin_popa? = masm.code_section()->end(); > ? masm.popa_slow(); > ? address end_popa??? = masm.code_section()->end(); > ? ... > } > > ---- > > Also, maybe you can add this assert after generating the code for all > 3 macros: > > ? assert(masm->code()->total_relocation_size() == 0 && > masm->code()->total_oop_size() == 0 && > masm->code()->total_metadata_size() == 0, > ???????? "precomputed code cannot have any of these"); > > > Thanks! > - Ioi > > > > On 3/10/20 6:46 AM, Claes Redestad wrote: >> Hi, >> >> calculate some invariant Assembler routines at bootstrap, copy on >> subsequent invocations. >> >> For popa and pusha this means an overhead reduction of around 98% (from >> ~2500 instructions to emit a pusha to ~50). For vzeroupper an overhead >> reduction of ~65% (117 -> 42). Together these add up to about a 1% >> reduction of instructions executed on a Hello World - with some >> (smaller) scaling impact on larger applications. >> >> The initialization is very simple/naive, i.e., lacks any kind of >> synchronization protocol. But as this setup is guaranteed to happen very >> early during bootstrap this should be fine. Thanks Ioi for some helpful >> suggestions here! >> >> Bug:??? https://bugs.openjdk.java.net/browse/JDK-8240772 >> Webrev: http://cr.openjdk.java.net/~redestad/8240772/open.00/ >> >> Testing: tier1-3 >> >> Thanks! >> >> /Claes > From david.holmes at oracle.com Wed Mar 11 02:08:45 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 11 Mar 2020 12:08:45 +1000 Subject: RFR (S): 8239593: Bogus warning "Re-registering of platform native method" for a JVMTI agent Message-ID: <7fc24e19-736c-423e-c8e5-bcf945c1b12a@oracle.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8239593 webrev: http://cr.openjdk.java.net/~dholmes/8239593/webrev/ The original intent was to warn if changes were being made to platform classes by non-platform classes. That was weakened slightly to warn if classes loaded by the boot/platform loaders were modified by a class not loaded by the boot/platform loader. But that formulation also catches non-platform classes that have been added to the bootclasspath. So we augment the check to also check that the target class is in a named module** (loaded by the boot/platform loader) - this ensures we catch changes to platform files whilst excluding classes incidentally on the bootclasspath. ** Note that you cannot add a named module to the boot/platform loaders at runtime - they are defined at build time. Tested against Yourkit reproducer in the bug report, and with existing registerNatives test. Test builds on tier1 platforms. Thanks, David From mandy.chung at oracle.com Wed Mar 11 05:25:53 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 10 Mar 2020 22:25:53 -0700 Subject: RFR(XS) 8240548 [TESTBUG] CDS NoClassToArchive.java fails with Graal In-Reply-To: <92fa6d77-4896-d322-2d10-09ebab728bb4@oracle.com> References: <9fd602d1-0e2c-5920-d677-8f2d0a1616be@oracle.com> <92fa6d77-4896-d322-2d10-09ebab728bb4@oracle.com> Message-ID: <2ef35bed-c8f8-c0d7-5df6-2eea3fbfc18e@oracle.com> On 3/10/20 9:29 AM, Ioi Lam wrote: > > In the latest fix, I changed the code to test for the "bytes: 315 > checksum: aa68e916" part of the log, which is more reliable in finding > out any non-CDS classes. > > http://cr.openjdk.java.net/~iklam/jdk15/8240548-NoClassToArchive.graal.v02/ > This version looks okay. Mandy From ioi.lam at oracle.com Wed Mar 11 06:46:33 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 10 Mar 2020 23:46:33 -0700 Subject: RFR(XS) 8240548 [TESTBUG] CDS NoClassToArchive.java fails with Graal In-Reply-To: <2ef35bed-c8f8-c0d7-5df6-2eea3fbfc18e@oracle.com> References: <9fd602d1-0e2c-5920-d677-8f2d0a1616be@oracle.com> <92fa6d77-4896-d322-2d10-09ebab728bb4@oracle.com> <2ef35bed-c8f8-c0d7-5df6-2eea3fbfc18e@oracle.com> Message-ID: Hi Mandy, thanks for the review! - Ioi On 3/10/20 10:25 PM, Mandy Chung wrote: > > > On 3/10/20 9:29 AM, Ioi Lam wrote: >> >> In the latest fix, I changed the code to test for the "bytes: 315 >> checksum: aa68e916" part of the log, which is more reliable in >> finding out any non-CDS classes. >> >> http://cr.openjdk.java.net/~iklam/jdk15/8240548-NoClassToArchive.graal.v02/ >> > > This version looks okay. > > Mandy From tobias.hartmann at oracle.com Wed Mar 11 07:00:38 2020 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Wed, 11 Mar 2020 08:00:38 +0100 Subject: RFC(S) 8234146: [TESTBUG] compiler/jsr292/ContinuousCallSiteTargetChange.java times out on SPARC In-Reply-To: <38086d99-43fe-c88d-a483-82b934eb4a81@oracle.com> References: <38086d99-43fe-c88d-a483-82b934eb4a81@oracle.com> Message-ID: <31ec30ff-241a-3a94-3fd8-144d3b3b9643@oracle.com> On 10.03.20 19:07, Vladimir Ivanov wrote: >> Option A: Force the NMethodSweeper using the whitebox api. >> Webrev A: http://cr.openjdk.java.net/~rrich/webrevs/8234146/A/webrev.0/ > > Looks good. +1 Best regards, Tobias From richard.reingruber at sap.com Wed Mar 11 08:23:30 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Wed, 11 Mar 2020 08:23:30 +0000 Subject: RFC(S) 8234146: [TESTBUG] compiler/jsr292/ContinuousCallSiteTargetChange.java times out on SPARC In-Reply-To: <38086d99-43fe-c88d-a483-82b934eb4a81@oracle.com> References: <38086d99-43fe-c88d-a483-82b934eb4a81@oracle.com> Message-ID: Hi Vladimir and Tobias, thanks for your reviews. I agree. Regarding option #B: I'd say the current implementation is very generic but it obfuscates the simple/intuitive rule that all dependent nmethods have to be invalidated when the call site target is changed. So in my eyes #B is less complex. Still: new code => potentially new bugs. So I agree to fix the issue with option #B. Thanks, Richard. -----Original Message----- From: Vladimir Ivanov Sent: Dienstag, 10. M?rz 2020 19:08 To: Reingruber, Richard ; hotspot-compiler-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net Subject: Re: RFC(S) 8234146: [TESTBUG] compiler/jsr292/ContinuousCallSiteTargetChange.java times out on SPARC Hi Richard, Nice analysis! I'd prefer to go with the test fix (option #A) for now. The idea to short-circuit repeated traversals over CallSite context looks clever, but I'm not yet persuaded it's worth the added complexity for such a pathological case. So, I propose to fix the test case and file an RFE for the inefficiency you spotted in dependency invalidation. [...] > Option A: Force the NMethodSweeper using the whitebox api. > Webrev A: http://cr.openjdk.java.net/~rrich/webrevs/8234146/A/webrev.0/ Looks good. Best regards, Vladimir Ivanov > > Option B: Specialize DependencyContext::mark_dependent_nmethods() for call site target changes, > i.e. flag list elements that are marked for deoptimization. Upon the next target change only new > dependents are marked for deoptimization. The list iteration is ended as soon as an element is > encountered that is already flagged. > Webrev B: http://cr.openjdk.java.net/~rrich/webrevs/8234146/B/webrev.0/ > > Personally I like B better, because I think it is simple and straight forward: if the target > changes, then all dependents need to be invalidated. By construction of the list we can stop the > invalidation if we find an element that was invalidated before. This specialization is of course > less generic, and there might be new types of call site dependencies for which it is illegal, but > currently I can't think of any. > > Curious about your comments... > > Thanks, > Richard. > > [1] To simulate load you can pin the jvm process to one cpu (e.g. on Linux with "taskset 1 java ...") > or > wget http://cr.openjdk.java.net/~rrich/webrevs/8234146/CPULoadGenerator.java && java CPULoadGenerator.java N > where N is the number of CPUs of your box > From claes.redestad at oracle.com Wed Mar 11 13:20:21 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 11 Mar 2020 13:20:21 +0000 (UTC) Subject: RFR: 8240772: x86_64: Pre-generate Assembler::popa, pusha and vzeroupper In-Reply-To: <1ac200e6-1d8f-7476-5cb8-6d44c77f3170@oracle.com> References: <9fc3ea60-ccfb-7250-c1e0-06c40088fe7e@oracle.com> <87ca51f4-6d98-8d71-d00c-3815367b3e1f@oracle.com> <1ac200e6-1d8f-7476-5cb8-6d44c77f3170@oracle.com> Message-ID: <7fe23978-ad6b-f381-efc2-78bce21857df@oracle.com> New webrev: http://cr.openjdk.java.net/~redestad/8240772/open.01/ Rather than *_slow I went with *_uncached. Reworked initialization, and discovered a bug in open.00: vzeroupper is speculatively emitted in the VM_Version stub with the CPU feature flag explicitly set. This meant we pre-computed the vzeroupper as always enabled, before CPU capabilities had been determined. This caused an intermittent test issue. I now expose an uncached version of vzeroupper that the VM_Version stub can use, then trigger the pre-compute right after running the CPU feature checks. Testing: re-running tier1-3, verified 32-bit builds locally /Claes On 2020-03-10 20:19, Claes Redestad wrote: > Hi Ioi, > > good suggestions. I will rework this tomorrow, along with Vladimir's > suggestion to add an explicit call to precompute_instructions from > the stubGenerator. > > Thanks! > > /Claes > > On 2020-03-10 19:40, Ioi Lam wrote: >> Hi Claes, >> >> This is a really good optimization! Small bang for big bucks! >> >> I have a suggestion code coding style: >> >> Rename Assembler::popa to Assembler::popa_slow(); >> >> void Assembler::popa() { // 64bit >> ?? if (!precomputed) { >> ???? precompute_instructions(); >> ?? } >> ?? copy_precomputed_instructions(popa_code, popa_len); >> } >> >> static void precompute_instructions() { >> ?? ... >> ?? MacroAssembler masm(&buffer); >> >> ?? address begin_popa? = masm.code_section()->end(); >> ?? masm.popa_slow(); >> ?? address end_popa??? = masm.code_section()->end(); >> ?? ... >> } >> >> ---- >> >> Also, maybe you can add this assert after generating the code for all >> 3 macros: >> >> ?? assert(masm->code()->total_relocation_size() == 0 && >> masm->code()->total_oop_size() == 0 && >> masm->code()->total_metadata_size() == 0, >> ????????? "precomputed code cannot have any of these"); >> >> >> Thanks! >> - Ioi >> >> >> >> On 3/10/20 6:46 AM, Claes Redestad wrote: >>> Hi, >>> >>> calculate some invariant Assembler routines at bootstrap, copy on >>> subsequent invocations. >>> >>> For popa and pusha this means an overhead reduction of around 98% (from >>> ~2500 instructions to emit a pusha to ~50). For vzeroupper an overhead >>> reduction of ~65% (117 -> 42). Together these add up to about a 1% >>> reduction of instructions executed on a Hello World - with some >>> (smaller) scaling impact on larger applications. >>> >>> The initialization is very simple/naive, i.e., lacks any kind of >>> synchronization protocol. But as this setup is guaranteed to happen very >>> early during bootstrap this should be fine. Thanks Ioi for some helpful >>> suggestions here! >>> >>> Bug:??? https://bugs.openjdk.java.net/browse/JDK-8240772 >>> Webrev: http://cr.openjdk.java.net/~redestad/8240772/open.00/ >>> >>> Testing: tier1-3 >>> >>> Thanks! >>> >>> /Claes >> From stuart.monteith at arm.com Wed Mar 11 13:53:54 2020 From: stuart.monteith at arm.com (Stuart Monteith) Date: Wed, 11 Mar 2020 13:53:54 +0000 Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is inconsistent In-Reply-To: References: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com> <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com> <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com> Message-ID: <8529c051-8eb0-7fa4-97bb-876149aeb092@arm.com> Look good, tests ok on aarch64. However, the case for "0" isn't passed in the test, or handled by the function. Is "count_leading_zeros" sufficiently well defined that we can expect it to produce "64" or "32" in this case, or should there be an assertion in exact_log2* to trip when 0 is passed? On 10/03/2020 15:04, Andrew Haley wrote: > On 3/9/20 12:20 PM, Stefan Karlsson wrote: >> On 2020-03-09 11:54, Andrew Haley wrote: >>> So, my plan is to push http://cr.openjdk.java.net/~aph/8240615-1/ and >>> subsequently patch a few AArch64 cases plus the x86_64 immL_Pow2 and >>> immL_NotPow2 cases that we already identified. >>> >>> I do not intend to do bulk changes to any other clients of >>> is_power_of_2() because it's often unclear exactly what behaviour is >>> required, and I'd risk breaking something. >>> >>> OK? >> >> I would prefer if the x86_64 and aarch64 patches were pushed first, or >> folded into this patch. Otherwise, we knowingly introduce a regression >> by pushing the proposed patch above. However, I'll leave it up to the >> compiler devs/maintainers to decide if this is OK. > > http://cr.openjdk.java.net/~aph/8240615-2/ > > OK? > From aph at redhat.com Wed Mar 11 14:22:38 2020 From: aph at redhat.com (Andrew Haley) Date: Wed, 11 Mar 2020 14:22:38 +0000 Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is inconsistent In-Reply-To: <8529c051-8eb0-7fa4-97bb-876149aeb092@arm.com> References: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com> <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com> <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com> <8529c051-8eb0-7fa4-97bb-876149aeb092@arm.com> Message-ID: <2c06bbb3-404e-452d-1dfa-c56c06aa92fc@redhat.com> On 3/11/20 1:53 PM, Stuart Monteith wrote: > Look good, tests ok on aarch64. However, the case for "0" isn't passed > in the test, or handled by the function. Is "count_leading_zeros" > sufficiently well defined that we can expect it to produce "64" or "32" > in this case, or should there be an assertion in exact_log2* to trip > when 0 is passed? Hmm. Am I missing something? There is, surely, everything you need to reject 0 here: bool is_power_of_2(T x) { return (x > T(0)) && ((x & (x - 1)) == T(0)); } inline int exact_log2(intptr_t x) { assert(is_power_of_2((uintptr_t)x), "x must be a power of 2: " INTPTR_FORMAT, x); -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From stuart.monteith at arm.com Wed Mar 11 14:26:35 2020 From: stuart.monteith at arm.com (Stuart Monteith) Date: Wed, 11 Mar 2020 14:26:35 +0000 Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is inconsistent In-Reply-To: <2c06bbb3-404e-452d-1dfa-c56c06aa92fc@redhat.com> References: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com> <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com> <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com> <8529c051-8eb0-7fa4-97bb-876149aeb092@arm.com> <2c06bbb3-404e-452d-1dfa-c56c06aa92fc@redhat.com> Message-ID: <54a62b81-0ade-662e-559d-2253a5834880@arm.com> Oh for goodness sake... Sorry, that's just me not paying attention... On 11/03/2020 14:22, Andrew Haley wrote: > On 3/11/20 1:53 PM, Stuart Monteith wrote: >> Look good, tests ok on aarch64. However, the case for "0" isn't passed >> in the test, or handled by the function. Is "count_leading_zeros" >> sufficiently well defined that we can expect it to produce "64" or "32" >> in this case, or should there be an assertion in exact_log2* to trip >> when 0 is passed? > > Hmm. Am I missing something? > > There is, surely, everything you need to reject 0 here: > > bool is_power_of_2(T x) { > return (x > T(0)) && ((x & (x - 1)) == T(0)); > } > > inline int exact_log2(intptr_t x) { > assert(is_power_of_2((uintptr_t)x), "x must be a power of 2: " INTPTR_FORMAT, x); > From aph at redhat.com Wed Mar 11 15:46:55 2020 From: aph at redhat.com (Andrew Haley) Date: Wed, 11 Mar 2020 15:46:55 +0000 Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is inconsistent In-Reply-To: References: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com> <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com> <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com> <1c40a7d1-fa36-b5d2-6072-eaa243cc9397@redhat.com> Message-ID: <2343a628-d5d0-5042-649f-150ab649d1e3@redhat.com> On 3/9/20 3:56 PM, Doerr, Martin wrote: > it doesn't run into the UB case, but log2_jint(x) returns 63 for negative x. > I'd treat this as bug. A jint has 32 bit and log2 should never return 63. I guess this bug dates from the days when uintptr_t was 32 bits. I think you're right, but this has never been a problem because all uses are either guarded by is_power_of_2() or they occur in a context (such as, say, os::active_processor_count()) where > 2**31 is extremely unlikely. I'm happy to fix this, though, if anyone thinks I should. The fix is: diff -r f9893c227e12 src/hotspot/share/utilities/globalDefinitions.hpp --- a/src/hotspot/share/utilities/globalDefinitions.hpp Wed Mar 11 15:02:09 2020 +0000 +++ b/src/hotspot/share/utilities/globalDefinitions.hpp Wed Mar 11 15:45:26 2020 +0000 @@ -1022,12 +1022,12 @@ inline int log2_int(int x) { STATIC_ASSERT(sizeof(int) <= sizeof(uintptr_t)); - return log2_intptr((uintptr_t)x); + return log2_intptr((uintptr_t)(unsigned int)x); } inline int log2_jint(jint x) { STATIC_ASSERT(sizeof(jint) <= sizeof(uintptr_t)); - return log2_intptr((uintptr_t)x); + return log2_intptr((uintptr_t)(juint)x); } inline int log2_uint(uint x) { -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From Alan.Bateman at oracle.com Wed Mar 11 16:10:30 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 11 Mar 2020 16:10:30 +0000 Subject: Review Request JDK-8228336: Refactor native library loading implementation In-Reply-To: <1030f82a-6d7d-daa1-788f-f39848a5022b@oracle.com> References: <00d2029d-ae16-6a9b-5942-cd67aba7c141@oracle.com> <1030f82a-6d7d-daa1-788f-f39848a5022b@oracle.com> Message-ID: <3b892f87-625f-2c10-f56c-ec39d7877027@oracle.com> On 10/03/2020 04:53, Mandy Chung wrote: > Hi David, > > On 3/9/20 7:41 PM, David Holmes wrote: >> That's a core-libs decision but I'm not sure that's a namespace we >> want to increase usage of. >> > > I'm open to other suggestion.? This helper method avoids the call to > doPrivileged when security manager is enabled and I think it's okay to > add this helper method in sun.security.action package. BootLoader defines methods to correspond to the methods defined by ClassLoader so I don't think the loadLibrary method was too bad. Moreover it was very clear when calling BootLoader.loadLibrary that it would attempt to load it from the system path. I don't object to putting this in sun.security.action but it doesn't feel. If it is moved but I think we'll need to find better names so it's clear in the 20+ places where it used (and also avoid the temptation to use it in the modules that aren't mapped to the boot loader). -Alan From martin.doerr at sap.com Wed Mar 11 16:27:24 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Wed, 11 Mar 2020 16:27:24 +0000 Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is inconsistent In-Reply-To: <2343a628-d5d0-5042-649f-150ab649d1e3@redhat.com> References: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com> <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com> <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com> <1c40a7d1-fa36-b5d2-6072-eaa243cc9397@redhat.com> <2343a628-d5d0-5042-649f-150ab649d1e3@redhat.com> Message-ID: Hi Andrew, I think it's worth fixing as you suggest. Current usages don't have a problem, but who knows about future usages? Thanks and best regards, Martin > -----Original Message----- > From: Andrew Haley > Sent: Mittwoch, 11. M?rz 2020 16:47 > To: Doerr, Martin ; Stefan Karlsson > ; John Rose > Cc: hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is > inconsistent > > On 3/9/20 3:56 PM, Doerr, Martin wrote: > > it doesn't run into the UB case, but log2_jint(x) returns 63 for negative x. > > I'd treat this as bug. A jint has 32 bit and log2 should never return 63. > > I guess this bug dates from the days when uintptr_t was 32 bits. > > I think you're right, but this has never been a problem because all uses > are either guarded by is_power_of_2() or they occur in a context (such as, > say, os::active_processor_count()) where > 2**31 is extremely unlikely. > > I'm happy to fix this, though, if anyone thinks I should. The fix is: > > diff -r f9893c227e12 src/hotspot/share/utilities/globalDefinitions.hpp > --- a/src/hotspot/share/utilities/globalDefinitions.hpp Wed Mar 11 > 15:02:09 2020 +0000 > +++ b/src/hotspot/share/utilities/globalDefinitions.hpp Wed Mar 11 > 15:45:26 2020 +0000 > @@ -1022,12 +1022,12 @@ > > inline int log2_int(int x) { > STATIC_ASSERT(sizeof(int) <= sizeof(uintptr_t)); > - return log2_intptr((uintptr_t)x); > + return log2_intptr((uintptr_t)(unsigned int)x); > } > > inline int log2_jint(jint x) { > STATIC_ASSERT(sizeof(jint) <= sizeof(uintptr_t)); > - return log2_intptr((uintptr_t)x); > + return log2_intptr((uintptr_t)(juint)x); > } > > inline int log2_uint(uint x) { > > -- > Andrew Haley (he/him) > Java Platform Lead Engineer > Red Hat UK Ltd. > https://keybase.io/andrewhaley > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From daniel.fuchs at oracle.com Wed Mar 11 17:25:32 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 11 Mar 2020 17:25:32 +0000 Subject: Review Request JDK-8228336: Refactor native library loading implementation In-Reply-To: <3b892f87-625f-2c10-f56c-ec39d7877027@oracle.com> References: <00d2029d-ae16-6a9b-5942-cd67aba7c141@oracle.com> <1030f82a-6d7d-daa1-788f-f39848a5022b@oracle.com> <3b892f87-625f-2c10-f56c-ec39d7877027@oracle.com> Message-ID: <93da1c6a-8038-3677-2368-a2a31b307cb8@oracle.com> Hi Mandy, On 11/03/2020 16:10, Alan Bateman wrote: >> I'm open to other suggestion.? This helper method avoids the call to >> doPrivileged when security manager is enabled and I think it's okay to >> add this helper method in sun.security.action package. > BootLoader defines methods to correspond to the methods defined by > ClassLoader so I don't think the loadLibrary method was too bad. > Moreover it was very clear when calling BootLoader.loadLibrary that it > would attempt to load it from the system path. I don't object to putting > this in sun.security.action but it doesn't feel. If it is moved but I > think we'll need to find better names so it's clear in the 20+ places > where it used (and also avoid the temptation to use it in the modules > that aren't mapped to the boot loader). That's probably a stupid question but wouldn't it have been simpler to redefine BootLoader::loadLibrary(String) to call the new LoadLibraryAction rather than removing the method altogether? best regards, -- daniel > > -Alan From ioi.lam at oracle.com Wed Mar 11 18:04:53 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 11 Mar 2020 11:04:53 -0700 Subject: RFR: 8240772: x86_64: Pre-generate Assembler::popa, pusha and vzeroupper In-Reply-To: <7fe23978-ad6b-f381-efc2-78bce21857df@oracle.com> References: <9fc3ea60-ccfb-7250-c1e0-06c40088fe7e@oracle.com> <87ca51f4-6d98-8d71-d00c-3815367b3e1f@oracle.com> <1ac200e6-1d8f-7476-5cb8-6d44c77f3170@oracle.com> <7fe23978-ad6b-f381-efc2-78bce21857df@oracle.com> Message-ID: <59b44684-242a-a1e5-730b-454518287c47@oracle.com> Hi Claes, Looks good. I would suggest moving these into an inline function to avoid repetition. ? assert(pusha_code != NULL, "must be pregenerated"); ? assert(code_section()->limit() - code_section()->end() > pusha_len, "code buffer not large enough"); ? address end = code_section()->end(); ? memcpy((char*)end, pusha_code, pusha_len); ? code_section()->set_end(end + pusha_len); Thanks - Ioi On 3/11/20 6:20 AM, Claes Redestad wrote: > New webrev: http://cr.openjdk.java.net/~redestad/8240772/open.01/ > > Rather than *_slow I went with *_uncached. > > Reworked initialization, and discovered a bug in open.00: > > vzeroupper is speculatively emitted in the VM_Version stub with the CPU > feature flag explicitly set. This meant we pre-computed the vzeroupper > as always enabled, before CPU capabilities had been determined. This > caused an intermittent test issue. > > I now expose an uncached version of vzeroupper that the VM_Version stub > can use, then trigger the pre-compute right after running the CPU > feature checks. > > Testing: re-running tier1-3, verified 32-bit builds locally > > /Claes > > On 2020-03-10 20:19, Claes Redestad wrote: >> Hi Ioi, >> >> good suggestions. I will rework this tomorrow, along with Vladimir's >> suggestion to add an explicit call to precompute_instructions from >> the stubGenerator. >> >> Thanks! >> >> /Claes >> >> On 2020-03-10 19:40, Ioi Lam wrote: >>> Hi Claes, >>> >>> This is a really good optimization! Small bang for big bucks! >>> >>> I have a suggestion code coding style: >>> >>> Rename Assembler::popa to Assembler::popa_slow(); >>> >>> void Assembler::popa() { // 64bit >>> ?? if (!precomputed) { >>> ???? precompute_instructions(); >>> ?? } >>> ?? copy_precomputed_instructions(popa_code, popa_len); >>> } >>> >>> static void precompute_instructions() { >>> ?? ... >>> ?? MacroAssembler masm(&buffer); >>> >>> ?? address begin_popa? = masm.code_section()->end(); >>> ?? masm.popa_slow(); >>> ?? address end_popa??? = masm.code_section()->end(); >>> ?? ... >>> } >>> >>> ---- >>> >>> Also, maybe you can add this assert after generating the code for >>> all 3 macros: >>> >>> ?? assert(masm->code()->total_relocation_size() == 0 && >>> masm->code()->total_oop_size() == 0 && >>> masm->code()->total_metadata_size() == 0, >>> ????????? "precomputed code cannot have any of these"); >>> >>> >>> Thanks! >>> - Ioi >>> >>> >>> >>> On 3/10/20 6:46 AM, Claes Redestad wrote: >>>> Hi, >>>> >>>> calculate some invariant Assembler routines at bootstrap, copy on >>>> subsequent invocations. >>>> >>>> For popa and pusha this means an overhead reduction of around 98% >>>> (from >>>> ~2500 instructions to emit a pusha to ~50). For vzeroupper an overhead >>>> reduction of ~65% (117 -> 42). Together these add up to about a 1% >>>> reduction of instructions executed on a Hello World - with some >>>> (smaller) scaling impact on larger applications. >>>> >>>> The initialization is very simple/naive, i.e., lacks any kind of >>>> synchronization protocol. But as this setup is guaranteed to happen >>>> very >>>> early during bootstrap this should be fine. Thanks Ioi for some >>>> helpful >>>> suggestions here! >>>> >>>> Bug:??? https://bugs.openjdk.java.net/browse/JDK-8240772 >>>> Webrev: http://cr.openjdk.java.net/~redestad/8240772/open.00/ >>>> >>>> Testing: tier1-3 >>>> >>>> Thanks! >>>> >>>> /Claes >>> From vladimir.kozlov at oracle.com Wed Mar 11 18:29:07 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 11 Mar 2020 11:29:07 -0700 Subject: RFR: 8240772: x86_64: Pre-generate Assembler::popa, pusha and vzeroupper In-Reply-To: <59b44684-242a-a1e5-730b-454518287c47@oracle.com> References: <9fc3ea60-ccfb-7250-c1e0-06c40088fe7e@oracle.com> <87ca51f4-6d98-8d71-d00c-3815367b3e1f@oracle.com> <1ac200e6-1d8f-7476-5cb8-6d44c77f3170@oracle.com> <7fe23978-ad6b-f381-efc2-78bce21857df@oracle.com> <59b44684-242a-a1e5-730b-454518287c47@oracle.com> Message-ID: Yes, I also thought about factoring this code into one function. The only drawback is yuo would need call stack to see which instruction failed. But I doubt we will see such failure ;) That leads to question. Claes, what was the reason you used loop instead of memcpy for vzeroupper? The logic of changes is good. Thanks, Vladimir On 3/11/20 11:04 AM, Ioi Lam wrote: > Hi Claes, > > Looks good. I would suggest moving these into an inline function to avoid repetition. > > ? assert(pusha_code != NULL, "must be pregenerated"); > ? assert(code_section()->limit() - code_section()->end() > pusha_len, "code buffer not large enough"); > ? address end = code_section()->end(); > ? memcpy((char*)end, pusha_code, pusha_len); > ? code_section()->set_end(end + pusha_len); > > > Thanks > - Ioi > > On 3/11/20 6:20 AM, Claes Redestad wrote: >> New webrev: http://cr.openjdk.java.net/~redestad/8240772/open.01/ >> >> Rather than *_slow I went with *_uncached. >> >> Reworked initialization, and discovered a bug in open.00: >> >> vzeroupper is speculatively emitted in the VM_Version stub with the CPU >> feature flag explicitly set. This meant we pre-computed the vzeroupper >> as always enabled, before CPU capabilities had been determined. This >> caused an intermittent test issue. >> >> I now expose an uncached version of vzeroupper that the VM_Version stub >> can use, then trigger the pre-compute right after running the CPU >> feature checks. >> >> Testing: re-running tier1-3, verified 32-bit builds locally >> >> /Claes >> >> On 2020-03-10 20:19, Claes Redestad wrote: >>> Hi Ioi, >>> >>> good suggestions. I will rework this tomorrow, along with Vladimir's >>> suggestion to add an explicit call to precompute_instructions from >>> the stubGenerator. >>> >>> Thanks! >>> >>> /Claes >>> >>> On 2020-03-10 19:40, Ioi Lam wrote: >>>> Hi Claes, >>>> >>>> This is a really good optimization! Small bang for big bucks! >>>> >>>> I have a suggestion code coding style: >>>> >>>> Rename Assembler::popa to Assembler::popa_slow(); >>>> >>>> void Assembler::popa() { // 64bit >>>> ?? if (!precomputed) { >>>> ???? precompute_instructions(); >>>> ?? } >>>> ?? copy_precomputed_instructions(popa_code, popa_len); >>>> } >>>> >>>> static void precompute_instructions() { >>>> ?? ... >>>> ?? MacroAssembler masm(&buffer); >>>> >>>> ?? address begin_popa? = masm.code_section()->end(); >>>> ?? masm.popa_slow(); >>>> ?? address end_popa??? = masm.code_section()->end(); >>>> ?? ... >>>> } >>>> >>>> ---- >>>> >>>> Also, maybe you can add this assert after generating the code for all 3 macros: >>>> >>>> ?? assert(masm->code()->total_relocation_size() == 0 && >>>> masm->code()->total_oop_size() == 0 && >>>> masm->code()->total_metadata_size() == 0, >>>> ????????? "precomputed code cannot have any of these"); >>>> >>>> >>>> Thanks! >>>> - Ioi >>>> >>>> >>>> >>>> On 3/10/20 6:46 AM, Claes Redestad wrote: >>>>> Hi, >>>>> >>>>> calculate some invariant Assembler routines at bootstrap, copy on >>>>> subsequent invocations. >>>>> >>>>> For popa and pusha this means an overhead reduction of around 98% (from >>>>> ~2500 instructions to emit a pusha to ~50). For vzeroupper an overhead >>>>> reduction of ~65% (117 -> 42). Together these add up to about a 1% >>>>> reduction of instructions executed on a Hello World - with some >>>>> (smaller) scaling impact on larger applications. >>>>> >>>>> The initialization is very simple/naive, i.e., lacks any kind of synchronization protocol. But as this setup is >>>>> guaranteed to happen very >>>>> early during bootstrap this should be fine. Thanks Ioi for some helpful >>>>> suggestions here! >>>>> >>>>> Bug:??? https://bugs.openjdk.java.net/browse/JDK-8240772 >>>>> Webrev: http://cr.openjdk.java.net/~redestad/8240772/open.00/ >>>>> >>>>> Testing: tier1-3 >>>>> >>>>> Thanks! >>>>> >>>>> /Claes >>>> > From vladimir.kozlov at oracle.com Wed Mar 11 18:37:02 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 11 Mar 2020 11:37:02 -0700 Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is inconsistent In-Reply-To: References: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com> <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com> <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com> <1c40a7d1-fa36-b5d2-6072-eaa243cc9397@redhat.com> <2343a628-d5d0-5042-649f-150ab649d1e3@redhat.com> Message-ID: <6603cedc-fe50-7d9a-aba4-8bce876937cb@oracle.com> +1 But please test it. Thanks, Vladimir On 3/11/20 9:27 AM, Doerr, Martin wrote: > Hi Andrew, > > I think it's worth fixing as you suggest. > Current usages don't have a problem, but who knows about future usages? > > Thanks and best regards, > Martin > > >> -----Original Message----- >> From: Andrew Haley >> Sent: Mittwoch, 11. M?rz 2020 16:47 >> To: Doerr, Martin ; Stefan Karlsson >> ; John Rose >> Cc: hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net >> Subject: Re: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is >> inconsistent >> >> On 3/9/20 3:56 PM, Doerr, Martin wrote: >>> it doesn't run into the UB case, but log2_jint(x) returns 63 for negative x. >>> I'd treat this as bug. A jint has 32 bit and log2 should never return 63. >> >> I guess this bug dates from the days when uintptr_t was 32 bits. >> >> I think you're right, but this has never been a problem because all uses >> are either guarded by is_power_of_2() or they occur in a context (such as, >> say, os::active_processor_count()) where > 2**31 is extremely unlikely. >> >> I'm happy to fix this, though, if anyone thinks I should. The fix is: >> >> diff -r f9893c227e12 src/hotspot/share/utilities/globalDefinitions.hpp >> --- a/src/hotspot/share/utilities/globalDefinitions.hpp Wed Mar 11 >> 15:02:09 2020 +0000 >> +++ b/src/hotspot/share/utilities/globalDefinitions.hpp Wed Mar 11 >> 15:45:26 2020 +0000 >> @@ -1022,12 +1022,12 @@ >> >> inline int log2_int(int x) { >> STATIC_ASSERT(sizeof(int) <= sizeof(uintptr_t)); >> - return log2_intptr((uintptr_t)x); >> + return log2_intptr((uintptr_t)(unsigned int)x); >> } >> >> inline int log2_jint(jint x) { >> STATIC_ASSERT(sizeof(jint) <= sizeof(uintptr_t)); >> - return log2_intptr((uintptr_t)x); >> + return log2_intptr((uintptr_t)(juint)x); >> } >> >> inline int log2_uint(uint x) { >> >> -- >> Andrew Haley (he/him) >> Java Platform Lead Engineer >> Red Hat UK Ltd. >> https://keybase.io/andrewhaley >> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 > From claes.redestad at oracle.com Wed Mar 11 21:36:27 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 11 Mar 2020 22:36:27 +0100 Subject: RFR: 8240772: x86_64: Pre-generate Assembler::popa, pusha and vzeroupper In-Reply-To: References: <9fc3ea60-ccfb-7250-c1e0-06c40088fe7e@oracle.com> <87ca51f4-6d98-8d71-d00c-3815367b3e1f@oracle.com> <1ac200e6-1d8f-7476-5cb8-6d44c77f3170@oracle.com> <7fe23978-ad6b-f381-efc2-78bce21857df@oracle.com> <59b44684-242a-a1e5-730b-454518287c47@oracle.com> Message-ID: On 2020-03-11 19:29, Vladimir Kozlov wrote: > > That leads to question. Claes, what was the reason you used loop instead > of memcpy for vzeroupper? For UseAXV <= 2 then vzeroupper is 3 bytes, so the loop simply used fewer instructions. It doesn't seem to matter much in the end, so let's favor simplicity. Final(?) version: http://cr.openjdk.java.net/~redestad/8240772/open.02/ /Claes From mandy.chung at oracle.com Wed Mar 11 21:43:28 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 11 Mar 2020 14:43:28 -0700 (PDT) Subject: Review Request JDK-8228336: Refactor native library loading implementation In-Reply-To: <93da1c6a-8038-3677-2368-a2a31b307cb8@oracle.com> References: <00d2029d-ae16-6a9b-5942-cd67aba7c141@oracle.com> <1030f82a-6d7d-daa1-788f-f39848a5022b@oracle.com> <3b892f87-625f-2c10-f56c-ec39d7877027@oracle.com> <93da1c6a-8038-3677-2368-a2a31b307cb8@oracle.com> Message-ID: <48764f71-93b7-002f-f113-e57bd6c62567@oracle.com> On 3/11/20 10:25 AM, Daniel Fuchs wrote: > Hi Mandy, > > On 11/03/2020 16:10, Alan Bateman wrote: >>> I'm open to other suggestion.? This helper method avoids the call to >>> doPrivileged when security manager is enabled and I think it's okay >>> to add this helper method in sun.security.action package. >> BootLoader defines methods to correspond to the methods defined by >> ClassLoader so I don't think the loadLibrary method was too bad. >> Moreover it was very clear when calling BootLoader.loadLibrary that >> it would attempt to load it from the system path. I don't object to >> putting this in sun.security.action but it doesn't feel. If it is >> moved but I think we'll need to find better names so it's clear in >> the 20+ places where it used (and also avoid the temptation to use it >> in the modules that aren't mapped to the boot loader). > > That's probably a stupid question but wouldn't it have been simpler to > redefine BootLoader::loadLibrary(String) to call the new > LoadLibraryAction rather than removing the method altogether? Thanks for the feedback. I discussed further with Alan.?? Other class and names are no better than BootLoader::loadLibrary.? We agree to keep that helper method as BootLoader::loadLibrary.? The changes due to the renaming is reverted. Updated patch: http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8228336/webrev.02/ Mandy From vladimir.kozlov at oracle.com Wed Mar 11 22:35:36 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 11 Mar 2020 15:35:36 -0700 Subject: RFR: 8240772: x86_64: Pre-generate Assembler::popa, pusha and vzeroupper In-Reply-To: References: <9fc3ea60-ccfb-7250-c1e0-06c40088fe7e@oracle.com> <87ca51f4-6d98-8d71-d00c-3815367b3e1f@oracle.com> <1ac200e6-1d8f-7476-5cb8-6d44c77f3170@oracle.com> <7fe23978-ad6b-f381-efc2-78bce21857df@oracle.com> <59b44684-242a-a1e5-730b-454518287c47@oracle.com> Message-ID: Nice! Thanks, Vladimir On 3/11/20 2:36 PM, Claes Redestad wrote: > On 2020-03-11 19:29, Vladimir Kozlov wrote: >> >> That leads to question. Claes, what was the reason you used loop instead of memcpy for vzeroupper? > > For UseAXV <= 2 then vzeroupper is 3 bytes, so the loop simply used > fewer instructions. It doesn't seem to matter much in the end, so let's > favor simplicity. > > Final(?) version: http://cr.openjdk.java.net/~redestad/8240772/open.02/ > > /Claes From yumin.qi at oracle.com Wed Mar 11 23:47:06 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Wed, 11 Mar 2020 16:47:06 -0700 Subject: RFR(S): 8240563: [TESTBUG] WB_IsCDSIncludedInVmBuild should support uncompressed oops/klasses Message-ID: <3dba8465-370d-6de5-530e-afe3573ba480@oracle.com> Hi, ? Please review: ? bug: https://bugs.openjdk.java.net/browse/JDK-8240563 ? url: http://cr.openjdk.java.net/~minqi/8240563/webrev-00/ ? Summary: With bug fix of 8232069 (https://bugs.openjdk.java.net/browse/JDK-8232069) pushed, CDS now works with UserCompressedOops on/off (also UseCompressedClassPointers). The detecting of CDS should not rely on the two flags. ?test/hotspot/jtreg/runtime/cds/appcds/CommandLineFlagComboNegative.java : ???? Mark for non-ZGC since ZGC will turn off UseCompressedOops and UseCompressedClassPointers so will mess up the flag combination. ?serviceability/sa/ClhsdbCDSCore.java will fail with the fix: ? ?? The core file generated with UseZGC but currently SA does not work with ZGC. so add it to test/hotspot/jtreg/ProblemList-zgc.txt ? test/hotspot/jtreg/runtime/cds/appcds/TestZGCWithCDS.java: ???? Explicitly add options which could be changed by ZGC. ?? Tests: hs-tier1,hs-tier2, hs-tier3 Thanks Yumin From ioi.lam at oracle.com Thu Mar 12 04:52:00 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 11 Mar 2020 21:52:00 -0700 Subject: RFR(S): 8240563: [TESTBUG] WB_IsCDSIncludedInVmBuild should support uncompressed oops/klasses In-Reply-To: <3dba8465-370d-6de5-530e-afe3573ba480@oracle.com> References: <3dba8465-370d-6de5-530e-afe3573ba480@oracle.com> Message-ID: Hi Yumin, Looks good to me. Thanks! - Ioi On 3/11/20 4:47 PM, Yumin Qi wrote: > Hi, > ? Please review: > ? bug: https://bugs.openjdk.java.net/browse/JDK-8240563 > ? url: http://cr.openjdk.java.net/~minqi/8240563/webrev-00/ > > ? Summary: With bug fix of 8232069 > (https://bugs.openjdk.java.net/browse/JDK-8232069) > pushed, CDS now > works with UserCompressedOops on/off (also > UseCompressedClassPointers). The detecting of CDS should not rely on > the two flags. > > ?test/hotspot/jtreg/runtime/cds/appcds/CommandLineFlagComboNegative.java > : > ???? Mark for non-ZGC since ZGC will turn off UseCompressedOops and > UseCompressedClassPointers so will mess up the flag combination. > ?serviceability/sa/ClhsdbCDSCore.java will fail with the fix: > ? ?? The core file generated with UseZGC but currently SA does not > work with ZGC. so add it to test/hotspot/jtreg/ProblemList-zgc.txt > ? test/hotspot/jtreg/runtime/cds/appcds/TestZGCWithCDS.java: > ???? Explicitly add options which could be changed by ZGC. > > ?? Tests: hs-tier1,hs-tier2, hs-tier3 > > Thanks > Yumin From ioi.lam at oracle.com Thu Mar 12 05:09:23 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 11 Mar 2020 22:09:23 -0700 Subject: RFR: 8240772: x86_64: Pre-generate Assembler::popa, pusha and vzeroupper In-Reply-To: References: <9fc3ea60-ccfb-7250-c1e0-06c40088fe7e@oracle.com> <87ca51f4-6d98-8d71-d00c-3815367b3e1f@oracle.com> <1ac200e6-1d8f-7476-5cb8-6d44c77f3170@oracle.com> <7fe23978-ad6b-f381-efc2-78bce21857df@oracle.com> <59b44684-242a-a1e5-730b-454518287c47@oracle.com> Message-ID: Looks good! - Ioi On 3/11/20 2:36 PM, Claes Redestad wrote: > On 2020-03-11 19:29, Vladimir Kozlov wrote: >> >> That leads to question. Claes, what was the reason you used loop >> instead of memcpy for vzeroupper? > > For UseAXV <= 2 then vzeroupper is 3 bytes, so the loop simply used > fewer instructions. It doesn't seem to matter much in the end, so let's > favor simplicity. > > Final(?) version: http://cr.openjdk.java.net/~redestad/8240772/open.02/ > > /Claes From Alan.Bateman at oracle.com Thu Mar 12 08:55:51 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 12 Mar 2020 08:55:51 +0000 Subject: Review Request JDK-8228336: Refactor native library loading implementation In-Reply-To: <48764f71-93b7-002f-f113-e57bd6c62567@oracle.com> References: <00d2029d-ae16-6a9b-5942-cd67aba7c141@oracle.com> <1030f82a-6d7d-daa1-788f-f39848a5022b@oracle.com> <3b892f87-625f-2c10-f56c-ec39d7877027@oracle.com> <93da1c6a-8038-3677-2368-a2a31b307cb8@oracle.com> <48764f71-93b7-002f-f113-e57bd6c62567@oracle.com> Message-ID: <1566d410-5f58-b1af-04e8-4b3fcc1a36ef@oracle.com> On 11/03/2020 21:43, Mandy Chung wrote: > : > > > I discussed further with Alan.?? Other class and names are no better > than BootLoader::loadLibrary.? We agree to keep that helper method as > BootLoader::loadLibrary.? The changes due to the renaming is reverted. > > Updated patch: > http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8228336/webrev.02/ This version also renames a method in NativeLibrary so the name is consistent. Looks good to me. -Alan From yumin.qi at oracle.com Thu Mar 12 15:59:18 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Thu, 12 Mar 2020 08:59:18 -0700 Subject: RFR(S): 8240563: [TESTBUG] WB_IsCDSIncludedInVmBuild should support uncompressed oops/klasses In-Reply-To: References: <3dba8465-370d-6de5-530e-afe3573ba480@oracle.com> Message-ID: Hi, Ioi ? Thanks for the review! Yumin On 3/11/20 9:52 PM, Ioi Lam wrote: > Hi Yumin, > > Looks good to me. > > Thanks! > > - Ioi > > On 3/11/20 4:47 PM, Yumin Qi wrote: >> Hi, >> ? Please review: >> ? bug: https://bugs.openjdk.java.net/browse/JDK-8240563 >> ? url: http://cr.openjdk.java.net/~minqi/8240563/webrev-00/ >> >> ? Summary: With bug fix of 8232069 >> (https://bugs.openjdk.java.net/browse/JDK-8232069) >> pushed, CDS now >> works with UserCompressedOops on/off (also >> UseCompressedClassPointers). The detecting of CDS should not rely on >> the two flags. >> >> ?test/hotspot/jtreg/runtime/cds/appcds/CommandLineFlagComboNegative.java >> : >> ???? Mark for non-ZGC since ZGC will turn off UseCompressedOops and >> UseCompressedClassPointers so will mess up the flag combination. >> ?serviceability/sa/ClhsdbCDSCore.java will fail with the fix: >> ? ?? The core file generated with UseZGC but currently SA does not >> work with ZGC. so add it to test/hotspot/jtreg/ProblemList-zgc.txt >> ? test/hotspot/jtreg/runtime/cds/appcds/TestZGCWithCDS.java: >> ???? Explicitly add options which could be changed by ZGC. >> >> ?? Tests: hs-tier1,hs-tier2, hs-tier3 >> >> Thanks >> Yumin > From martin.doerr at sap.com Thu Mar 12 16:28:29 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Thu, 12 Mar 2020 16:28:29 +0000 Subject: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents In-Reply-To: References: <1f8a3c7a-fa0f-b5b2-4a8a-7d3d8dbbe1b5@oracle.com> <4b56a45c-a14c-6f74-2bfd-25deaabe8201@oracle.com> <5271429a-481d-ddb9-99dc-b3f6670fcc0b@oracle.com> Message-ID: Hi Richard, I managed to find time for a (almost) complete review of webrev.4. (I'll review the tests separately.) First of all, the change seems to be in pretty good quality for its significant complexity. I couldn't find any real bugs. But I'd like to propose minor improvements. I'm convinced that it's mature because we did substantial testing. I like the new functionality for object deoptimization. It can possibly be reused for future escape analysis based optimizations. So I appreciate having it available in the code base. In addition to that, your change makes the JVMTI implementation better integrated into the VM. Now to the details: src/hotspot/share/c1/c1_IR.hpp describe_scope parameters. Ok. src/hotspot/share/ci/ciEnv.cpp src/hotspot/share/ci/ciEnv.hpp Fix for JvmtiExport::can_walk_any_space() capability. Ok. src/hotspot/share/code/compiledMethod.cpp Nice cleanup! src/hotspot/share/code/debugInfoRec.cpp src/hotspot/share/code/debugInfoRec.hpp Additional parmeters. (Remark: I think "non_global_escape_in_scope" would read better than "not_global_escape_in_scope", but your version is consistent with existing code, so no change request from my side.) Ok. src/hotspot/share/code/nmethod.cpp Nice cleanup! src/hotspot/share/code/pcDesc.hpp Additional parameters. Ok. src/hotspot/share/code/scopeDesc.cpp src/hotspot/share/code/scopeDesc.hpp Improved implementation + additional parameters. Ok. src/hotspot/share/compiler/compileBroker.cpp src/hotspot/share/compiler/compileBroker.hpp Extra thread for DeoptimizeObjectsALot. (Remark: I would have put it into a follow up change together with the test in order to make this webrev smaller, but since it is included, I'm reviewing everything at once. Not a big deal.) Ok. src/hotspot/share/jvmci/jvmciCodeInstaller.cpp Additional parameters. Ok. src/hotspot/share/opto/c2compiler.cpp Make do_escape_analysis independent of JVMCI capabilities. Nice! src/hotspot/share/opto/callnode.hpp Additional fields for MachSafePointNodes. Ok. src/hotspot/share/opto/escape.cpp Annotation for MachSafePointNodes. Your added functionality looks correct. But I'd prefer to move the bulky code out of the large function. I suggest to factor out something like has_not_global_escape and has_arg_escape. So the code could look like this: SafePointNode* sfn = sfn_worklist.at(next); sfn->set_not_global_escape_in_scope(has_not_global_escape(sfn)); if (sfn->is_CallJava()) { CallJavaNode* call = sfn->as_CallJava(); call->set_arg_escape(has_arg_escape(call)); } This would also allow us to get rid of the found_..._escape_in_args variables making the loops better readable. It's kind of ugly to use strcmp to recognize uncommon trap, but that seems to be the way to do it (there are more such places). So it's ok. src/hotspot/share/opto/machnode.hpp Additional fields for MachSafePointNodes. Ok. src/hotspot/share/opto/macro.cpp Allow elimination of non-escaping allocations. Ok. src/hotspot/share/opto/matcher.cpp src/hotspot/share/opto/output.cpp Copy attribute / pass parameters. Ok. src/hotspot/share/prims/jvmtiCodeBlobEvents.cpp Nice cleanup! src/hotspot/share/prims/jvmtiEnv.cpp src/hotspot/share/prims/jvmtiEnvBase.cpp Escape barriers + deoptimize objects for target thread. Good. src/hotspot/share/prims/jvmtiImpl.cpp src/hotspot/share/prims/jvmtiImpl.hpp The sequence is pretty complex: VM_GetOrSetLocal element initialization executes EscapeBarrier code which suspends the target thread (extra VM Operation). VM_GetOrSetLocal::doit_prologue performs object deoptimization (by VM Thread to prepare VM Operation with frame deoptimization). VM_GetOrSetLocal destructor implicitly calls EscapeBarrier destructor which resumes the target thread. But I don't have any improvement proposal. Performance is probably not a concern, here. So it's ok. VM_GetOrSetLocal::deoptimize_objects deoptimizes the top frame if it has non-globally escaping objects and other frames if they have arg escaping ones. Good. src/hotspot/share/prims/jvmtiTagMap.cpp Escape barriers + deoptimize objects for all threads. Ok. src/hotspot/share/prims/whitebox.cpp Added WB_IsFrameDeoptimized to API. Ok. src/hotspot/share/runtime/deoptimization.cpp Object deoptimization. I have more comments and proposals, here. First of all, handling recursive and waiting locks in relock_objects is tricky, but looks correct. Comments are sufficient to understand why things are done as they are implemented. BiasedLocking related parts are complex, but we may get rid of them in the future (with BiasedLocking removal). Anyway, looks correct, too. Typo in comment: "regularily" => "regularly" Deoptimization::fetch_unroll_info_helper is the only place where _jvmti_deferred_updates get deallocated (except JavaThread destructor). But I think we always go through it, so I can't see a memory leak or such kind of issues. EscapeBarrier::deoptimize_objects: ResourceMark should use calling_thread(). You can use MutexLocker and MonitorLocker with Thread* to save the Thread::current() call. I'd make set_objs_are_deoptimized static and remove it from the EscapeBarrier interface because I think it shouldn't be used outside of EscapeBarrier::deoptimize_objects. Typo in comment: "we must only deoptimize" => "we only have to deoptimize" "bool EscapeBarrier::deoptimize_objects(intptr_t* fr_id)" is trivial and barrier_active() is redundant. Implementation can get moved to hpp file. I'll get back to suspend flags, later. There are weird cases regarding _self_deoptimization_in_progress. Assume we have 3 threads A, B and C. A deopts C, B deopts C, C deopts C. C can set _self_deoptimization_in_progress while A performs the handshake for suspending C. I think this doesn't lead to errors, but it's probably not desired. I think it would be better to use only one "wait" call in sync_and_suspend_one and sync_and_suspend_all. I first thought it'd be better to move ThreadBlockInVM before wait() to reduce thread state transitions, but that seems to be problematic because ThreadBlockInVM destructor contains a safepoint check which we shouldn't do while holding EscapeBarrier_lock. So no change request. Change in thred_added: I think the sequence would be more comprehensive if we waited for deopt_all_threads in Thread::start and all other places where a new thread can run into Java code (e.g. JVMTI attach). Your version makes new threads come up with suspend flag set. That looks correct, too. Advantage is that you only have to change one place (thread_added). It'll be interesting to see how it will look like when we use async handshakes instead of suspend flags. For now, I'm ok with your version. I'd only move MutexLocker ml(EscapeBarrier_lock...) after if (!jt->is_hidden_from_external_view()). Having 4 different deoptimize_objects functions makes it a little hard to keep an overview of which one is used for what. Maybe adding suffixes would help a little bit, but I can also live with what you have. Implementation looks correct to me. src/hotspot/share/runtime/deoptimization.hpp Escape barriers and object deoptimization functions. Typo in comment: "helt" => "held" src/hotspot/share/runtime/globals.hpp Addition of develop flag DeoptimizeObjectsALotInterval. Ok. src/hotspot/share/runtime/interfaceSupport.cpp InterfaceSupport::deoptimizeAllObjects() is only used for DeoptimizeObjectsALot = 1. I think DeoptimizeObjectsALot = 2 is more important, but I think it's not bad to have DeoptimizeObjectsALot = 1 in addition. Ok. src/hotspot/share/runtime/interfaceSupport.inline.hpp Addition of deoptimizeAllObjects. Ok. src/hotspot/share/runtime/mutexLocker.cpp src/hotspot/share/runtime/mutexLocker.hpp Addition of EscapeBarrier_lock. Ok. src/hotspot/share/runtime/objectMonitor.cpp Make recursion count relock aware. Ok. src/hotspot/share/runtime/stackValue.hpp Better reinitilization in StackValue. Good. src/hotspot/share/runtime/thread.cpp src/hotspot/share/runtime/thread.hpp src/hotspot/share/runtime/thread.inline.hpp wait_for_object_deoptimization, suspend flag, deferred updates and test feature to deoptimize objects. In the long term, we want to get rid of suspend flags, so it's not so nice to introduce a new one. But I agree with G?tz that it should be acceptable as temporary solution until async handshakes are available (which takes more time). So I'm ok with your change. You can use MutexLocker with Thread*. JVMTIDeferredUpdates: I agree with Robin. It'd be nice to move the class out of thread.hpp. src/hotspot/share/runtime/vframe.cpp Added support for entry frame to new_vframe. Ok. src/hotspot/share/runtime/vframe_hp.cpp src/hotspot/share/runtime/vframe_hp.hpp I think code()->as_nmethod() in not_global_escape_in_scope() and arg_escape() should better be under #ifdef ASSERT or inside the assert statement (no need for code cache walking in product build). jvmtiDeferredLocalVariableSet::update_monitors: Please add a comment explaining that owner referenced by original info may be scalar replaced, but it is deoptimized in the vframe. src/hotspot/share/utilities/macros.hpp Addition of NOT_COMPILER2_OR_JVMCI_RETURN macros. Ok. test/hotspot/jtreg/serviceability/jvmti/Heap/IterateHeapWithEscapeAnalysisEnabled.java test/hotspot/jtreg/serviceability/jvmti/Heap/libIterateHeapWithEscapeAnalysisEnabled.c New test. Will review separately. test/jdk/TEST.ROOT Addition of vm.jvmci as required property. Ok. test/jdk/com/sun/jdi/EATests.java test/jdk/com/sun/jdi/EATestsJVMCI.java New test. Will review separately. test/lib/sun/hotspot/WhiteBox.java Added isFrameDeoptimized to API. Ok. That was it. Best regards, Martin > -----Original Message----- > From: hotspot-compiler-dev bounces at openjdk.java.net> On Behalf Of Reingruber, Richard > Sent: Dienstag, 3. M?rz 2020 21:23 > To: 'Robbin Ehn' ; Lindenmaier, Goetz > ; David Holmes ; > Vladimir Kozlov (vladimir.kozlov at oracle.com) > ; serviceability-dev at openjdk.java.net; > hotspot-compiler-dev at openjdk.java.net; hotspot-runtime- > dev at openjdk.java.net > Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better > Performance in the Presence of JVMTI Agents > > Hi Robbin, > > > > I understand that Robbin proposed to replace the usage of > > > _suspend_flag with handshakes. Apparently, async handshakes > > > are needed to do so. We have been waiting a while for removal > > > of the _suspend_flag / introduction of async handshakes [2]. > > > What is the status here? > > > I have an old prototype which I would like to continue to work on. > > So do not assume asynch handshakes will make 15. > > Even if it would, I think there are a lot more investigate work to remove > > _suspend_flag. > > Let us know, if we can be of any help to you and be it only testing. > > > >> Full: > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.4/ > > > DeoptimizeObjectsALotThread is only used in compileBroker.cpp. > > You can move both declaration and definition to that file, no need to > clobber > > thread.[c|h]pp. (and the static function deopt_objs_alot_thread_entry) > > Will do. > > > Does JvmtiDeferredUpdates really need to be in thread.hpp, can't be in it's > own > > hpp file? It doesn't seem right to add JVM TI classes into thread.hpp. > > You are right. It shouldn't be declared in thread.hpp. I will look into that. > > > Note that we also think we may have a bug in deopt: > > https://bugs.openjdk.java.net/browse/JDK-8238237 > > > I think it would be best, if possible, to push after that is resolved. > > Sure. > > > Not even nearly a full review :) > > I know :) > > Anyways, thanks a lot, > Richard. > > > -----Original Message----- > From: Robbin Ehn > Sent: Monday, March 2, 2020 11:17 AM > To: Lindenmaier, Goetz ; Reingruber, Richard > ; David Holmes ; > Vladimir Kozlov (vladimir.kozlov at oracle.com) > ; serviceability-dev at openjdk.java.net; > hotspot-compiler-dev at openjdk.java.net; hotspot-runtime- > dev at openjdk.java.net > Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better Performance > in the Presence of JVMTI Agents > > Hi, > > On 2/24/20 5:39 PM, Lindenmaier, Goetz wrote: > > Hi, > > > > I had a look at the progress of this change. Nothing > > happened since Richard posted his update using more > > handshakes [1]. > > But we (SAP) would appreciate a lot if this change could > > be successfully reviewed and pushed. > > > > I think there is basic understanding that this > > change is helpful. It fixes a number of issues with JVMTI, > > and will deliver the same performance benefits as EA > > does in current production mode for debugging scenarios. > > > > This is important for us as we run our VMs prepared > > for debugging in production mode. > > > > I understand that Robbin proposed to replace the usage of > > _suspend_flag with handshakes. Apparently, async handshakes > > are needed to do so. We have been waiting a while for removal > > of the _suspend_flag / introduction of async handshakes [2]. > > What is the status here? > > I have an old prototype which I would like to continue to work on. > So do not assume asynch handshakes will make 15. > Even if it would, I think there are a lot more investigate work to remove > _suspend_flag. > > > > > I think we should no longer wait, but proceed with > > this change. We will look into removing the usage of > > suspend_flag introduced here once it is possible to implement > > it with handshakes. > > Yes, sure. > > >> Full: http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.4/ > > DeoptimizeObjectsALotThread is only used in compileBroker.cpp. > You can move both declaration and definition to that file, no need to clobber > thread.[c|h]pp. (and the static function deopt_objs_alot_thread_entry) > > Does JvmtiDeferredUpdates really need to be in thread.hpp, can't be in it's > own > hpp file? It doesn't seem right to add JVM TI classes into thread.hpp. > > Note that we also think we may have a bug in deopt: > https://bugs.openjdk.java.net/browse/JDK-8238237 > > I think it would be best, if possible, to push after that is resolved. > > Not even nearly a full review :) > > Thanks, Robbin > > > >> Incremental: > >> http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.4.inc/ > >> > >> I was not able to eliminate the additional suspend flag now. I'll take care > of this > >> as soon as the > >> existing suspend-resume-mechanism is reworked. > >> > >> Testing: > >> > >> Nightly tests @SAP: > >> > >> JCK and JTREG, also in Xcomp mode, SPECjvm2008, SPECjbb2015, > Renaissance > >> Suite, SAP specific tests > >> with fastdebug and release builds on all platforms > >> > >> Stress testing with DeoptimizeObjectsALot running SPECjvm2008 40x > parallel > >> for 24h > >> > >> Thanks, Richard. > >> > >> > >> More details on the changes: > >> > >> * Hide DeoptimizeObjectsALotThread from external view. > >> > >> * Changed EscapeBarrier_lock to be a _safepoint_check_never lock. > >> It used to be _safepoint_check_sometimes, which will be eliminated > sooner or > >> later. > >> I added explicit thread state changes with ThreadBlockInVM to code > paths > >> where we can wait() > >> on EscapeBarrier_lock to become safepoint safe. > >> > >> * Use handshake EscapeBarrierSuspendHandshake to suspend target > threads > >> instead of vm operation > >> VM_ThreadSuspendAllForObjDeopt. > >> > >> * Removed uses of Threads_lock. When adding a new thread we suspend > it iff > >> EA optimizations are > >> being reverted. In the previous version we were waiting on > Threads_lock > >> while EA optimizations > >> were reverted. See EscapeBarrier::thread_added(). > >> > >> * Made tests require Xmixed compilation mode. > >> > >> * Made tests agnostic regarding tiered compilation. > >> I.e. tc isn't disabled anymore, and the tests can be run with tc enabled or > >> disabled. > >> > >> * Exercising EATests.java as well with stress test options > >> DeoptimizeObjectsALot* > >> Due to the non-deterministic deoptimizations some tests need to be > skipped. > >> We do this to prevent bit-rot of the stress test code. > >> > >> * Executing EATests.java as well with graal if available. Driver for this is > >> EATestsJVMCI.java. Graal cannot pass all tests, because it does not > provide all > >> the new debug info > >> (namely not_global_escape_in_scope and arg_escape in > scopeDesc.hpp). > >> And graal does not yet support the JVMTI operations force early return > and > >> pop frame. > >> > >> * Removed tracing from new jdi tests in EATests.java. Too much trace > output > >> before the debugging > >> connection is established can cause deadlock because output buffers fill > up. > >> (See https://bugs.openjdk.java.net/browse/JDK-8173304) > >> > >> * Many copyright year changes and smaller clean-up changes of testing > code > >> (trailing white-space and > >> the like). > >> > >> > >> -----Original Message----- > >> From: David Holmes > >> Sent: Donnerstag, 19. Dezember 2019 03:12 > >> To: Reingruber, Richard ; serviceability- > >> dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; > hotspot- > >> runtime-dev at openjdk.java.net; Vladimir Kozlov > (vladimir.kozlov at oracle.com) > >> > >> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better > Performance in > >> the Presence of JVMTI Agents > >> > >> Hi Richard, > >> > >> I think my issue is with the way EliminateNestedLocks works so I'm going > >> to look into that more deeply. > >> > >> Thanks for the explanations. > >> > >> David > >> > >> On 18/12/2019 12:47 am, Reingruber, Richard wrote: > >>> Hi David, > >>> > >>> > > > Some further queries/concerns: > >>> > > > > >>> > > > src/hotspot/share/runtime/objectMonitor.cpp > >>> > > > > >>> > > > Can you please explain the changes to ObjectMonitor::wait: > >>> > > > > >>> > > > ! _recursions = save // restore the old recursion count > >>> > > > ! + jt->get_and_reset_relock_count_after_wait(); // > >>> > > > increased by the deferred relock count > >>> > > > > >>> > > > what is the "deferred relock count"? I gather it relates to > >>> > > > > >>> > > > "The code was extended to be able to deoptimize objects of a > >>> > > frame that > >>> > > > is not the top frame and to let another thread than the owning > >>> > > thread do > >>> > > > it." > >>> > > > >>> > > Yes, these relate. Currently EA based optimizations are reverted, > when a > >> compiled frame is > >>> > > replaced with corresponding interpreter frames. Part of this is > relocking > >> objects with eliminated > >>> > > locking. New with the enhancement is that we do this also just > before > >> object references are > >>> > > acquired through JVMTI. In this case we deoptimize also the > owning > >> compiled frame C and we > >>> > > register deoptimized objects as deferred updates. When control > returns > >> to C it gets deoptimized, > >>> > > we notice that objects are already deoptimized (reallocated and > >> relocked), so we don't do it again > >>> > > (relocking twice would be incorrect of course). Deferred updates > are > >> copied into the new > >>> > > interpreter frames. > >>> > > > >>> > > Problem: relocking is not possible if the target thread T is waiting > on the > >> monitor that needs to > >>> > > be relocked. This happens only with non-local objects with > >> EliminateNestedLocks. Instead relocking > >>> > > is deferred until T owns the monitor again. This is what the piece of > >> code above does. > >>> > > >>> > Sorry I need some more detail here. How can you wait() on an > object > >>> > monitor if the object allocation and/or locking was optimised away? > And > >>> > what is a "non-local object" in this context? Isn't EA restricted to > >>> > thread-confined objects? > >>> > >>> "Non-local object" is an object that escapes its thread. The issue I'm > >> addressing with the changes > >>> in ObjectMonitor::wait are almost unrelated to EA. They are caused by > >> EliminateNestedLocks, where C2 > >>> eliminates recursive locking of an already owned lock. The lock owning > object > >> exists on the heap, it > >>> is locked and you can call wait() on it. > >>> > >>> EliminateLocks is the C2 option that controls lock elimination based on > EA. > >> Both optimizations have > >>> in common that objects with eliminated locking need to be relocked > when > >> deoptimizing a frame, > >>> i.e. when replacing a compiled frame with equivalent interpreter > >>> frames. Deoptimization::relock_objects does that job for /all/ eliminated > >> locks in scope. /All/ can > >>> be a mix of eliminated nested locks and locks of not-escaping objects. > >>> > >>> New with the enhancement: I call relock_objects earlier, just before > objects > >> pontentially > >>> escape. But then later when the owning compiled frame gets > deoptimized, I > >> must not do it again: > >>> > >>> See call to EscapeBarrier::objs_are_deoptimized in deoptimization.cpp: > >>> > >>> 373 if ((jvmci_enabled || ((DoEscapeAnalysis || > EliminateNestedLocks) && > >> EliminateLocks)) > >>> 374 && !EscapeBarrier::objs_are_deoptimized(thread, > deoptee.id())) { > >>> 375 bool unused; > >>> 376 eliminate_locks(thread, chunk, realloc_failures, deoptee, > exec_mode, > >> unused); > >>> 377 } > >>> > >>> Now when calling relock_objects early it is quiet possible that I have to > relock > >> an object the > >>> target thread currently waits for. Obviously I cannot relock in this case, > >> instead I chose to > >>> introduce relock_count_after_wait to JavaThread. > >>> > >>> > Is it just that some of the locking gets optimized away e.g. > >>> > > >>> > synchronised(obj) { > >>> > synchronised(obj) { > >>> > synchronised(obj) { > >>> > obj.wait(); > >>> > } > >>> > } > >>> > } > >>> > > >>> > If this is reduced to a form as-if it were a single lock of the monitor > >>> > (due to EA) and the wait() triggers a JVM TI event which leads to the > >>> > escape of "obj" then we need to reconstruct the true lock state, and > so > >>> > when the wait() internally unblocks and reacquires the monitor it > has to > >>> > set the true recursion count to 3, not the 1 that it appeared to be > when > >>> > wait() was initially called. Is that the scenario? > >>> > >>> Kind of... except that the locking is not eliminated due to EA and there is > no > >> JVM TI event > >>> triggered by wait. > >>> > >>> Add > >>> > >>> LocalObject l1 = new LocalObject(); > >>> > >>> in front of the synchrnized blocks and assume a JVM TI agent acquires l1. > This > >> triggers the code in > >>> question. > >>> > >>> See that relocking/reallocating is transactional. If it is done then for /all/ > >> objects in scope and it is > >>> done at most once. It wouldn't be quite so easy to split this in relocking > of > >> nested/EA-based > >>> eliminated locks. > >>> > >>> > If so I find this truly awful. Anyone using wait() in a realistic form > >>> > requires a notification and so the object cannot be thread confined. > In > >>> > >>> It is not thread confined. > >>> > >>> > which case I would strongly argue that upon hitting the wait() the > deopt > >>> > should occur unconditionally and so the lock state is correct before > we > >>> > wait and so we don't need to mess with the recursion count > internally > >>> > when we reacquire the monitor. > >>> > > >>> > > > >>> > > > which I don't like the sound of at all when it comes to > ObjectMonitor > >>> > > > state. So I'd like to understand in detail exactly what is going on > here > >>> > > > and why. This is a very intrusive change that seems to badly > break > >>> > > > encapsulation and impacts future changes to ObjectMonitor > that are > >> under > >>> > > > investigation. > >>> > > > >>> > > I would not regard this as breaking encapsulation. Certainly not > badly. > >>> > > > >>> > > I've added a property relock_count_after_wait to JavaThread. The > >> property is well > >>> > > encapsulated. Future ObjectMonitor implementations have to deal > with > >> recursion too. They are free > >>> > > in choosing a way to do that as long as that property is taken into > >> account. This is hardly a > >>> > > limitation. > >>> > > >>> > I do think this badly breaks encapsulation as you have to add a > callout > >>> > from the guts of the ObjectMonitor code to reach into the thread to > get > >>> > this lock count adjustment. I understand why you have had to do > this but > >>> > I would much rather see a change to the EA optimisation strategy so > that > >>> > this is not needed. > >>> > > >>> > > Note also that the property is a straight forward extension of the > >> existing concept of deferred > >>> > > local updates. It is embedded into the structure holding them. So > not > >> even the footprint of a > >>> > > JavaThread is enlarged if no deferred updates are generated. > >>> > > >>> > [...] > >>> > > >>> > > > >>> > > I'm actually duplicating the existing external suspend mechanism, > >> because a thread can be > >>> > > suspended at most once. And hey, and don't like that either! But it > >> seems not unlikely that the > >>> > > duplicate can be removed together with the original and the new > type > >> of handshakes that will be > >>> > > used for thread suspend can be used for object deoptimization > too. See > >> today's discussion in > >>> > > JDK-8227745 [2]. > >>> > > >>> > I hope that discussion bears some fruit, at the moment it seems not > to > >>> > be possible to use handshakes here. :( > >>> > > >>> > The external suspend mechanism is a royal pain in the proverbial > that we > >>> > have to carefully live with. The idea that we're duplicating that for > >>> > use in another fringe area of functionality does not thrill me at all. > >>> > > >>> > To be clear, I understand the problem that exists and that you wish > to > >>> > solve, but for the runtime parts I balk at the complexity cost of > >>> > solving it. > >>> > >>> I know it's complex, but by far no rocket science. > >>> > >>> Also I find it hard to imagine another fix for JDK-8233915 besides > changing > >> the JVM TI specification. > >>> > >>> Thanks, Richard. > >>> > >>> -----Original Message----- > >>> From: David Holmes > >>> Sent: Dienstag, 17. Dezember 2019 08:03 > >>> To: Reingruber, Richard ; serviceability- > >> dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; > hotspot- > >> runtime-dev at openjdk.java.net; Vladimir Kozlov > (vladimir.kozlov at oracle.com) > >> > >>> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better > Performance > >> in the Presence of JVMTI Agents > >>> > >>> > >>> > >>> David > >>> > >>> On 17/12/2019 4:57 pm, David Holmes wrote: > >>>> Hi Richard, > >>>> > >>>> On 14/12/2019 5:01 am, Reingruber, Richard wrote: > >>>>> Hi David, > >>>>> > >>>>> ?? > Some further queries/concerns: > >>>>> ?? > > >>>>> ?? > src/hotspot/share/runtime/objectMonitor.cpp > >>>>> ?? > > >>>>> ?? > Can you please explain the changes to ObjectMonitor::wait: > >>>>> ?? > > >>>>> ?? > !?? _recursions = save????? // restore the old recursion count > >>>>> ?? > !???????????????? + jt->get_and_reset_relock_count_after_wait(); // > >>>>> ?? > increased by the deferred relock count > >>>>> ?? > > >>>>> ?? > what is the "deferred relock count"? I gather it relates to > >>>>> ?? > > >>>>> ?? > "The code was extended to be able to deoptimize objects of a > >>>>> frame that > >>>>> ?? > is not the top frame and to let another thread than the owning > >>>>> thread do > >>>>> ?? > it." > >>>>> > >>>>> Yes, these relate. Currently EA based optimizations are reverted, > when > >>>>> a compiled frame is replaced > >>>>> with corresponding interpreter frames. Part of this is relocking > >>>>> objects with eliminated > >>>>> locking. New with the enhancement is that we do this also just before > >>>>> object references are acquired > >>>>> through JVMTI. In this case we deoptimize also the owning compiled > >>>>> frame C and we register > >>>>> deoptimized objects as deferred updates. When control returns to C > it > >>>>> gets deoptimized, we notice > >>>>> that objects are already deoptimized (reallocated and relocked), so > we > >>>>> don't do it again (relocking > >>>>> twice would be incorrect of course). Deferred updates are copied into > >>>>> the new interpreter frames. > >>>>> > >>>>> Problem: relocking is not possible if the target thread T is waiting > >>>>> on the monitor that needs to be > >>>>> relocked. This happens only with non-local objects with > >>>>> EliminateNestedLocks. Instead relocking is > >>>>> deferred until T owns the monitor again. This is what the piece of > >>>>> code above does. > >>>> > >>>> Sorry I need some more detail here. How can you wait() on an object > >>>> monitor if the object allocation and/or locking was optimised away? > And > >>>> what is a "non-local object" in this context? Isn't EA restricted to > >>>> thread-confined objects? > >>>> > >>>> Is it just that some of the locking gets optimized away e.g. > >>>> > >>>> synchronised(obj) { > >>>> ? synchronised(obj) { > >>>> ??? synchronised(obj) { > >>>> ????? obj.wait(); > >>>> ??? } > >>>> ? } > >>>> } > >>>> > >>>> If this is reduced to a form as-if it were a single lock of the monitor > >>>> (due to EA) and the wait() triggers a JVM TI event which leads to the > >>>> escape of "obj" then we need to reconstruct the true lock state, and so > >>>> when the wait() internally unblocks and reacquires the monitor it has to > >>>> set the true recursion count to 3, not the 1 that it appeared to be when > >>>> wait() was initially called. Is that the scenario? > >>>> > >>>> If so I find this truly awful. Anyone using wait() in a realistic form > >>>> requires a notification and so the object cannot be thread confined. In > >>>> which case I would strongly argue that upon hitting the wait() the > deopt > >>>> should occur unconditionally and so the lock state is correct before we > >>>> wait and so we don't need to mess with the recursion count internally > >>>> when we reacquire the monitor. > >>>> > >>>>> > >>>>> ?? > which I don't like the sound of at all when it comes to > >>>>> ObjectMonitor > >>>>> ?? > state. So I'd like to understand in detail exactly what is going > >>>>> on here > >>>>> ?? > and why.? This is a very intrusive change that seems to badly > break > >>>>> ?? > encapsulation and impacts future changes to ObjectMonitor that > >>>>> are under > >>>>> ?? > investigation. > >>>>> > >>>>> I would not regard this as breaking encapsulation. Certainly not badly. > >>>>> > >>>>> I've added a property relock_count_after_wait to JavaThread. The > >>>>> property is well > >>>>> encapsulated. Future ObjectMonitor implementations have to deal > with > >>>>> recursion too. They are free in > >>>>> choosing a way to do that as long as that property is taken into > >>>>> account. This is hardly a > >>>>> limitation. > >>>> > >>>> I do think this badly breaks encapsulation as you have to add a callout > >>>> from the guts of the ObjectMonitor code to reach into the thread to > get > >>>> this lock count adjustment. I understand why you have had to do this > but > >>>> I would much rather see a change to the EA optimisation strategy so > that > >>>> this is not needed. > >>>> > >>>>> Note also that the property is a straight forward extension of the > >>>>> existing concept of deferred > >>>>> local updates. It is embedded into the structure holding them. So not > >>>>> even the footprint of a > >>>>> JavaThread is enlarged if no deferred updates are generated. > >>>>> > >>>>> ?? > --- > >>>>> ?? > > >>>>> ?? > src/hotspot/share/runtime/thread.cpp > >>>>> ?? > > >>>>> ?? > Can you please explain why > >>>>> JavaThread::wait_for_object_deoptimization > >>>>> ?? > has to be handcrafted in this way rather than using proper > >>>>> transitions. > >>>>> ?? > > >>>>> > >>>>> I wrote wait_for_object_deoptimization taking > >>>>> JavaThread::java_suspend_self_with_safepoint_check > >>>>> as template. So in short: for the same reasons :) > >>>>> > >>>>> Threads reach both methods as part of thread state transitions, > >>>>> therefore special handling is > >>>>> required to change thread state on top of ongoing transitions. > >>>>> > >>>>> ?? > We got rid of "deopt suspend" some time ago and it is disturbing > >>>>> to see > >>>>> ?? > it being added back (effectively). This seems like it may be > >>>>> something > >>>>> ?? > that handshakes could be used for. > >>>>> > >>>>> Deopt suspend used to be something rather different with a similar > >>>>> name[1]. It is not being added back. > >>>> > >>>> I stand corrected. Despite comments in the code to the contrary > >>>> deopt_suspend didn't actually cause a self-suspend. I was doing a lot of > >>>> cleanup in this area 13 years ago :) > >>>> > >>>>> > >>>>> I'm actually duplicating the existing external suspend mechanism, > >>>>> because a thread can be suspended > >>>>> at most once. And hey, and don't like that either! But it seems not > >>>>> unlikely that the duplicate can > >>>>> be removed together with the original and the new type of > handshakes > >>>>> that will be used for > >>>>> thread suspend can be used for object deoptimization too. See > today's > >>>>> discussion in JDK-8227745 [2]. > >>>> > >>>> I hope that discussion bears some fruit, at the moment it seems not to > >>>> be possible to use handshakes here. :( > >>>> > >>>> The external suspend mechanism is a royal pain in the proverbial that > we > >>>> have to carefully live with. The idea that we're duplicating that for > >>>> use in another fringe area of functionality does not thrill me at all. > >>>> > >>>> To be clear, I understand the problem that exists and that you wish to > >>>> solve, but for the runtime parts I balk at the complexity cost of > >>>> solving it. > >>>> > >>>> Thanks, > >>>> David > >>>> ----- > >>>> > >>>>> Thanks, Richard. > >>>>> > >>>>> [1] Deopt suspend was something like an async. handshake for > >>>>> architectures with register windows, > >>>>> ???? where patching the return pc for deoptimization of a compiled > >>>>> frame was racy if the owner thread > >>>>> ???? was in native code. Instead a "deopt" suspend flag was set on > >>>>> which the thread patched its own > >>>>> ???? frame upon return from native. So no thread was suspended. It > got > >>>>> its name only from the name of > >>>>> ???? the flags. > >>>>> > >>>>> [2] Discussion about using handshakes to sync. with the target thread: > >>>>> > >>>>> https://bugs.openjdk.java.net/browse/JDK- > >> > 8227745?focusedCommentId=14306727&page=com.atlassian.jira.plugin.syst > e > >> m.issuetabpanels:comment-tabpanel#comment-14306727 > >>>>> > >>>>> > >>>>> -----Original Message----- > >>>>> From: David Holmes > >>>>> Sent: Freitag, 13. Dezember 2019 00:56 > >>>>> To: Reingruber, Richard ; > >>>>> serviceability-dev at openjdk.java.net; > >>>>> hotspot-compiler-dev at openjdk.java.net; > >>>>> hotspot-runtime-dev at openjdk.java.net > >>>>> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better > >>>>> Performance in the Presence of JVMTI Agents > >>>>> > >>>>> Hi Richard, > >>>>> > >>>>> Some further queries/concerns: > >>>>> > >>>>> src/hotspot/share/runtime/objectMonitor.cpp > >>>>> > >>>>> Can you please explain the changes to ObjectMonitor::wait: > >>>>> > >>>>> !?? _recursions = save????? // restore the old recursion count > >>>>> !???????????????? + jt->get_and_reset_relock_count_after_wait(); // > >>>>> increased by the deferred relock count > >>>>> > >>>>> what is the "deferred relock count"? I gather it relates to > >>>>> > >>>>> "The code was extended to be able to deoptimize objects of a frame > that > >>>>> is not the top frame and to let another thread than the owning thread > do > >>>>> it." > >>>>> > >>>>> which I don't like the sound of at all when it comes to ObjectMonitor > >>>>> state. So I'd like to understand in detail exactly what is going on here > >>>>> and why.? This is a very intrusive change that seems to badly break > >>>>> encapsulation and impacts future changes to ObjectMonitor that are > under > >>>>> investigation. > >>>>> > >>>>> --- > >>>>> > >>>>> src/hotspot/share/runtime/thread.cpp > >>>>> > >>>>> Can you please explain why > JavaThread::wait_for_object_deoptimization > >>>>> has to be handcrafted in this way rather than using proper transitions. > >>>>> > >>>>> We got rid of "deopt suspend" some time ago and it is disturbing to > see > >>>>> it being added back (effectively). This seems like it may be something > >>>>> that handshakes could be used for. > >>>>> > >>>>> Thanks, > >>>>> David > >>>>> ----- > >>>>> > >>>>> On 12/12/2019 7:02 am, David Holmes wrote: > >>>>>> On 12/12/2019 1:07 am, Reingruber, Richard wrote: > >>>>>>> Hi David, > >>>>>>> > >>>>>>> ??? > Most of the details here are in areas I can comment on in > detail, > >>>>>>> but I > >>>>>>> ??? > did take an initial general look at things. > >>>>>>> > >>>>>>> Thanks for taking the time! > >>>>>> > >>>>>> Apologies the above should read: > >>>>>> > >>>>>> "Most of the details here are in areas I *can't* comment on in detail > >>>>>> ..." > >>>>>> > >>>>>> David > >>>>>> > >>>>>>> ??? > The only thing that jumped out at me is that I think the > >>>>>>> ??? > DeoptimizeObjectsALotThread should be a hidden thread. > >>>>>>> ??? > > >>>>>>> ??? > +? bool is_hidden_from_external_view() const { return true; } > >>>>>>> > >>>>>>> Yes, it should. Will add the method like above. > >>>>>>> > >>>>>>> ??? > Also I don't see any testing of the > DeoptimizeObjectsALotThread. > >>>>>>> Without > >>>>>>> ??? > active testing this will just bit-rot. > >>>>>>> > >>>>>>> DeoptimizeObjectsALot is meant for stress testing with a larger > >>>>>>> workload. I will add a minimal test > >>>>>>> to keep it fresh. > >>>>>>> > >>>>>>> ??? > Also on the tests I don't understand your @requires clause: > >>>>>>> ??? > > >>>>>>> ??? >?? @requires ((vm.compMode != "Xcomp") & > vm.compiler2.enabled > >> & > >>>>>>> ??? > (vm.opt.TieredCompilation != true)) > >>>>>>> ??? > > >>>>>>> ??? > This seems to require that TieredCompilation is disabled, but > >>>>>>> tiered is > >>>>>>> ??? > our normal mode of operation. ?? > >>>>>>> ??? > > >>>>>>> > >>>>>>> I removed the clause. I guess I wanted to target the tests towards > the > >>>>>>> code they are supposed to > >>>>>>> test, and it's easier to analyze failures w/o tiered compilation and > >>>>>>> with just one compiler thread. > >>>>>>> > >>>>>>> Additionally I will make use of > >>>>>>> compiler.whitebox.CompilerWhiteBoxTest.THRESHOLD in the tests. > >>>>>>> > >>>>>>> Thanks, > >>>>>>> Richard. > >>>>>>> > >>>>>>> -----Original Message----- > >>>>>>> From: David Holmes > >>>>>>> Sent: Mittwoch, 11. Dezember 2019 08:03 > >>>>>>> To: Reingruber, Richard ; > >>>>>>> serviceability-dev at openjdk.java.net; > >>>>>>> hotspot-compiler-dev at openjdk.java.net; > >>>>>>> hotspot-runtime-dev at openjdk.java.net > >>>>>>> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better > >>>>>>> Performance in the Presence of JVMTI Agents > >>>>>>> > >>>>>>> Hi Richard, > >>>>>>> > >>>>>>> On 11/12/2019 7:45 am, Reingruber, Richard wrote: > >>>>>>>> Hi, > >>>>>>>> > >>>>>>>> I would like to get reviews please for > >>>>>>>> > >>>>>>>> > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.3/ > >>>>>>>> > >>>>>>>> Corresponding RFE: > >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8227745 > >>>>>>>> > >>>>>>>> Fixes also https://bugs.openjdk.java.net/browse/JDK-8233915 > >>>>>>>> And potentially https://bugs.openjdk.java.net/browse/JDK- > 8214584 [1] > >>>>>>>> > >>>>>>>> Vladimir Kozlov kindly put webrev.3 through tier1-8 testing > without > >>>>>>>> issues (thanks!). In addition the > >>>>>>>> change is being tested at SAP since I posted the first RFR some > >>>>>>>> months ago. > >>>>>>>> > >>>>>>>> The intention of this enhancement is to benefit performance wise > from > >>>>>>>> escape analysis even if JVMTI > >>>>>>>> agents request capabilities that allow them to access local variable > >>>>>>>> values. E.g. if you start-up > >>>>>>>> with -agentlib:jdwp=transport=dt_socket,server=y,suspend=n, > then > >>>>>>>> escape analysis is disabled right > >>>>>>>> from the beginning, well before a debugger attaches -- if ever one > >>>>>>>> should do so. With the > >>>>>>>> enhancement, escape analysis will remain enabled until and after > a > >>>>>>>> debugger attaches. EA based > >>>>>>>> optimizations are reverted just before an agent acquires the > >>>>>>>> reference to an object. In the JBS item > >>>>>>>> you'll find more details. > >>>>>>> > >>>>>>> Most of the details here are in areas I can comment on in detail, but > I > >>>>>>> did take an initial general look at things. > >>>>>>> > >>>>>>> The only thing that jumped out at me is that I think the > >>>>>>> DeoptimizeObjectsALotThread should be a hidden thread. > >>>>>>> > >>>>>>> +? bool is_hidden_from_external_view() const { return true; } > >>>>>>> > >>>>>>> Also I don't see any testing of the DeoptimizeObjectsALotThread. > >>>>>>> Without > >>>>>>> active testing this will just bit-rot. > >>>>>>> > >>>>>>> Also on the tests I don't understand your @requires clause: > >>>>>>> > >>>>>>> ??? @requires ((vm.compMode != "Xcomp") & > vm.compiler2.enabled & > >>>>>>> (vm.opt.TieredCompilation != true)) > >>>>>>> > >>>>>>> This seems to require that TieredCompilation is disabled, but tiered > is > >>>>>>> our normal mode of operation. ?? > >>>>>>> > >>>>>>> Thanks, > >>>>>>> David > >>>>>>> > >>>>>>>> Thanks, > >>>>>>>> Richard. > >>>>>>>> > >>>>>>>> [1] Experimental fix for JDK-8214584 based on JDK-8227745 > >>>>>>>> > >> > http://cr.openjdk.java.net/~rrich/webrevs/2019/8214584/experiment_v1.pa > tc > >> h > >>>>>>>> > >>>>>>>> > >>>>>>>> From calvin.cheung at oracle.com Thu Mar 12 16:32:26 2020 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Thu, 12 Mar 2020 09:32:26 -0700 Subject: RFR(S): 8240563: [TESTBUG] WB_IsCDSIncludedInVmBuild should support uncompressed oops/klasses In-Reply-To: References: <3dba8465-370d-6de5-530e-afe3573ba480@oracle.com> Message-ID: <1464bd9f-b7c2-6abf-8cb1-eff8382989bd@oracle.com> +1 thanks, Calvin On 3/11/20 9:52 PM, Ioi Lam wrote: > Hi Yumin, > > Looks good to me. > > Thanks! > > - Ioi > > On 3/11/20 4:47 PM, Yumin Qi wrote: >> Hi, >> ? Please review: >> ? bug: https://bugs.openjdk.java.net/browse/JDK-8240563 >> ? url: http://cr.openjdk.java.net/~minqi/8240563/webrev-00/ >> >> ? Summary: With bug fix of 8232069 >> (https://bugs.openjdk.java.net/browse/JDK-8232069) >> pushed, CDS now >> works with UserCompressedOops on/off (also >> UseCompressedClassPointers). The detecting of CDS should not rely on >> the two flags. >> >> ?test/hotspot/jtreg/runtime/cds/appcds/CommandLineFlagComboNegative.java >> : >> ???? Mark for non-ZGC since ZGC will turn off UseCompressedOops and >> UseCompressedClassPointers so will mess up the flag combination. >> ?serviceability/sa/ClhsdbCDSCore.java will fail with the fix: >> ? ?? The core file generated with UseZGC but currently SA does not >> work with ZGC. so add it to test/hotspot/jtreg/ProblemList-zgc.txt >> ? test/hotspot/jtreg/runtime/cds/appcds/TestZGCWithCDS.java: >> ???? Explicitly add options which could be changed by ZGC. >> >> ?? Tests: hs-tier1,hs-tier2, hs-tier3 >> >> Thanks >> Yumin > From calvin.cheung at oracle.com Thu Mar 12 18:59:24 2020 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Thu, 12 Mar 2020 11:59:24 -0700 Subject: RFR(S): 8238000: Crash in ClassLoader::record_result while dynamic dumping netty Message-ID: <3a1773cf-bd7d-fbc1-9f4b-bc9f0a335f72@oracle.com> bug: https://bugs.openjdk.java.net/browse/JDK-8238000 webrev: http://cr.openjdk.java.net/~ccheung/jdk15/8238000/webrev.00/ Make a copy of the _shared_path_table for use during dynamic CDS dump. It is needed because some Java code continues to execute after dynamic dump has finished. However, during dynamic dump, we have modified FileMapInfo::_shared_path_table so FileMapInfo::shared_path(i) returns incorrect information in ClassLoader::record_result(). Passed tier1 - 4 testing. thanks, Calvin From ioi.lam at oracle.com Thu Mar 12 20:50:23 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 12 Mar 2020 13:50:23 -0700 Subject: RFR(S): 8238000: Crash in ClassLoader::record_result while dynamic dumping netty In-Reply-To: <3a1773cf-bd7d-fbc1-9f4b-bc9f0a335f72@oracle.com> References: <3a1773cf-bd7d-fbc1-9f4b-bc9f0a335f72@oracle.com> Message-ID: <07db4c7b-cb04-bca3-7f23-8e8af586867e@oracle.com> Hi Calvin, The fix looks good to me. Thanks - Ioi On 3/12/20 11:59 AM, Calvin Cheung wrote: > bug: https://bugs.openjdk.java.net/browse/JDK-8238000 > > webrev: http://cr.openjdk.java.net/~ccheung/jdk15/8238000/webrev.00/ > > Make a copy of the _shared_path_table for use during dynamic CDS dump. > It is needed because some Java code continues to execute after dynamic > dump has finished. However, during dynamic dump, we have modified > FileMapInfo::_shared_path_table so FileMapInfo::shared_path(i) returns > incorrect information in ClassLoader::record_result(). > > Passed tier1 - 4 testing. > > thanks, > > Calvin > From yumin.qi at oracle.com Thu Mar 12 21:17:28 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Thu, 12 Mar 2020 14:17:28 -0700 Subject: RFR(S): 8238000: Crash in ClassLoader::record_result while dynamic dumping netty In-Reply-To: <3a1773cf-bd7d-fbc1-9f4b-bc9f0a335f72@oracle.com> References: <3a1773cf-bd7d-fbc1-9f4b-bc9f0a335f72@oracle.com> Message-ID: Hi, Calvin ? Looks good to me. Thanks Yumin On 3/12/20 11:59 AM, Calvin Cheung wrote: > bug: https://bugs.openjdk.java.net/browse/JDK-8238000 > > webrev: http://cr.openjdk.java.net/~ccheung/jdk15/8238000/webrev.00/ > > Make a copy of the _shared_path_table for use during dynamic CDS dump. > It is needed because some Java code continues to execute after dynamic > dump has finished. However, during dynamic dump, we have modified > FileMapInfo::_shared_path_table so FileMapInfo::shared_path(i) returns > incorrect information in ClassLoader::record_result(). > > Passed tier1 - 4 testing. > > thanks, > > Calvin > From calvin.cheung at oracle.com Thu Mar 12 21:35:24 2020 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Thu, 12 Mar 2020 14:35:24 -0700 Subject: RFR(S): 8238000: Crash in ClassLoader::record_result while dynamic dumping netty In-Reply-To: <07db4c7b-cb04-bca3-7f23-8e8af586867e@oracle.com> References: <3a1773cf-bd7d-fbc1-9f4b-bc9f0a335f72@oracle.com> <07db4c7b-cb04-bca3-7f23-8e8af586867e@oracle.com> Message-ID: Thanks for your quick review, Ioi. Calvin On 3/12/20 1:50 PM, Ioi Lam wrote: > Hi Calvin, > > The fix looks good to me. > > Thanks > - Ioi > > On 3/12/20 11:59 AM, Calvin Cheung wrote: >> bug: https://bugs.openjdk.java.net/browse/JDK-8238000 >> >> webrev: http://cr.openjdk.java.net/~ccheung/jdk15/8238000/webrev.00/ >> >> Make a copy of the _shared_path_table for use during dynamic CDS >> dump. It is needed because some Java code continues to execute after >> dynamic dump has finished. However, during dynamic dump, we have >> modified FileMapInfo::_shared_path_table so >> FileMapInfo::shared_path(i) returns incorrect information in >> ClassLoader::record_result(). >> >> Passed tier1 - 4 testing. >> >> thanks, >> >> Calvin >> > From calvin.cheung at oracle.com Thu Mar 12 21:35:56 2020 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Thu, 12 Mar 2020 14:35:56 -0700 Subject: RFR(S): 8238000: Crash in ClassLoader::record_result while dynamic dumping netty In-Reply-To: References: <3a1773cf-bd7d-fbc1-9f4b-bc9f0a335f72@oracle.com> Message-ID: <4d4ad798-2671-0e11-1ee2-b28fa5f31524@oracle.com> Thanks for your quick review, Yumin. Calvin On 3/12/20 2:17 PM, Yumin Qi wrote: > Hi, Calvin > > ? Looks good to me. > > Thanks > Yumin > > On 3/12/20 11:59 AM, Calvin Cheung wrote: >> bug: https://bugs.openjdk.java.net/browse/JDK-8238000 >> >> webrev: http://cr.openjdk.java.net/~ccheung/jdk15/8238000/webrev.00/ >> >> Make a copy of the _shared_path_table for use during dynamic CDS >> dump. It is needed because some Java code continues to execute after >> dynamic dump has finished. However, during dynamic dump, we have >> modified FileMapInfo::_shared_path_table so >> FileMapInfo::shared_path(i) returns incorrect information in >> ClassLoader::record_result(). >> >> Passed tier1 - 4 testing. >> >> thanks, >> >> Calvin >> > From david.holmes at oracle.com Thu Mar 12 23:08:55 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 13 Mar 2020 09:08:55 +1000 Subject: RFR (S): 8239593: Bogus warning "Re-registering of platform native method" for a JVMTI agent In-Reply-To: <7fc24e19-736c-423e-c8e5-bcf945c1b12a@oracle.com> References: <7fc24e19-736c-423e-c8e5-bcf945c1b12a@oracle.com> Message-ID: <8a729883-8928-5364-0f56-533364789d5f@oracle.com> Anyone? David On 11/03/2020 12:08 pm, David Holmes wrote: > Bug: https://bugs.openjdk.java.net/browse/JDK-8239593 > webrev: http://cr.openjdk.java.net/~dholmes/8239593/webrev/ > > The original intent was to warn if changes were being made to platform > classes by non-platform classes. That was weakened slightly to warn if > classes loaded by the boot/platform loaders were modified by a class not > loaded by the boot/platform loader. But that formulation also catches > non-platform classes that have been added to the bootclasspath. So we > augment the check to also check that the target class is in a named > module** (loaded by the boot/platform loader) - this ensures we catch > changes to platform files whilst excluding classes incidentally on the > bootclasspath. > > ** Note that you cannot add a named module to the boot/platform loaders > at runtime - they are defined at build time. > > Tested against Yourkit reproducer in the bug report, and with existing > registerNatives test. > > Test builds on tier1 platforms. > > Thanks, > David From daniel.daugherty at oracle.com Thu Mar 12 23:38:28 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 12 Mar 2020 19:38:28 -0400 Subject: RFR (S): 8239593: Bogus warning "Re-registering of platform native method" for a JVMTI agent In-Reply-To: <7fc24e19-736c-423e-c8e5-bcf945c1b12a@oracle.com> References: <7fc24e19-736c-423e-c8e5-bcf945c1b12a@oracle.com> Message-ID: Hi David, I started and stopped a couple of times due to other distractions. Sorry about that... On 3/10/20 10:08 PM, David Holmes wrote: > Bug: https://bugs.openjdk.java.net/browse/JDK-8239593 > webrev: http://cr.openjdk.java.net/~dholmes/8239593/webrev/ src/hotspot/share/prims/jni.cpp ??? L2818:?? .... That will catch changes ??? L2819: ? // to platform classes while excluding classes added to the bootclasspath. ??????? I'm having trouble with the "excluding classes added to the ??????? bootclasspath" part. Perhaps this: ??? ? ? ? ? .... That will catch changes ??? ? ? ? ? // to platform classes while permitting changes to classes added to the bootclasspath. ??????? Maybe it's just me, but "excluding classes" sounds like those ??????? classes won't be allowed or something. ??? L2828: ??? if ((cl ==? NULL || SystemDictionary::is_platform_class_loader(cl)) && ??????? nit - not yours, but there's an extra space after '=='. ??? No comments on the code itself, although I did have to read it a ??? couple of times to convince myself the cases were right/caught. Thumbs up. Your call on changing the comments or fixing the nit. I don't need to see another webrev. Thanks for mentioning the testing done. Dan > > The original intent was to warn if changes were being made to platform > classes by non-platform classes. That was weakened slightly to warn if > classes loaded by the boot/platform loaders were modified by a class > not loaded by the boot/platform loader. But that formulation also > catches non-platform classes that have been added to the > bootclasspath. So we augment the check to also check that the target > class is in a named module** (loaded by the boot/platform loader) - > this ensures we catch changes to platform files whilst excluding > classes incidentally on the bootclasspath. > > ** Note that you cannot add a named module to the boot/platform > loaders at runtime - they are defined at build time. > > Tested against Yourkit reproducer in the bug report, and with existing > registerNatives test. > > Test builds on tier1 platforms. > > Thanks, > David From vladimir.kozlov at oracle.com Fri Mar 13 02:08:57 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 12 Mar 2020 19:08:57 -0700 Subject: [15] RFR(S) 8237045: JVM uses excessive memory with -XX:+EnableJVMCI -XX:JVMCICounterSize=2147483648 Message-ID: <35735d94-7ccf-158e-a2be-a2b4c968afd9@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8237045 http://cr.openjdk.java.net/~kvn/8237045/webrev.00/ After discussion JVMCICounterSize flag's value range is limited to 1M. I also added other missing ranges for JVMCI flags. Fixed TestOptionsWithRanges.java test to add -XX:+EnableJVMCI for JVMCI flags which requires it. And do not test max value for JVMCI threads counts values as we do for other threads counts flags. Tested hs-tier1-3. Thanks, Vladimir From david.holmes at oracle.com Fri Mar 13 07:23:39 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 13 Mar 2020 17:23:39 +1000 Subject: RFR (S): 8239593: Bogus warning "Re-registering of platform native method" for a JVMTI agent In-Reply-To: References: <7fc24e19-736c-423e-c8e5-bcf945c1b12a@oracle.com> Message-ID: Thanks for the Review Dan! I'll fix up the two nits. David On 13/03/2020 9:38 am, Daniel D. Daugherty wrote: > Hi David, > > I started and stopped a couple of times due to other distractions. > Sorry about that... > > > On 3/10/20 10:08 PM, David Holmes wrote: >> Bug: https://bugs.openjdk.java.net/browse/JDK-8239593 >> webrev: http://cr.openjdk.java.net/~dholmes/8239593/webrev/ > > src/hotspot/share/prims/jni.cpp > ??? L2818:?? .... That will catch changes > ??? L2819: ? // to platform classes while excluding classes added to > the bootclasspath. > ??????? I'm having trouble with the "excluding classes added to the > ??????? bootclasspath" part. Perhaps this: > > ??? ? ? ? ? .... That will catch changes > ??? ? ? ? ? // to platform classes while permitting changes to classes > added to the bootclasspath. > > ??????? Maybe it's just me, but "excluding classes" sounds like those > ??????? classes won't be allowed or something. > > ??? L2828: ??? if ((cl ==? NULL || > SystemDictionary::is_platform_class_loader(cl)) && > ??????? nit - not yours, but there's an extra space after '=='. > > ??? No comments on the code itself, although I did have to read it a > ??? couple of times to convince myself the cases were right/caught. > > Thumbs up. Your call on changing the comments or fixing the nit. > I don't need to see another webrev. > > Thanks for mentioning the testing done. > > Dan > > >> >> The original intent was to warn if changes were being made to platform >> classes by non-platform classes. That was weakened slightly to warn if >> classes loaded by the boot/platform loaders were modified by a class >> not loaded by the boot/platform loader. But that formulation also >> catches non-platform classes that have been added to the >> bootclasspath. So we augment the check to also check that the target >> class is in a named module** (loaded by the boot/platform loader) - >> this ensures we catch changes to platform files whilst excluding >> classes incidentally on the bootclasspath. >> >> ** Note that you cannot add a named module to the boot/platform >> loaders at runtime - they are defined at build time. >> >> Tested against Yourkit reproducer in the bug report, and with existing >> registerNatives test. >> >> Test builds on tier1 platforms. >> >> Thanks, >> David > From shiyue.xw at alibaba-inc.com Fri Mar 13 07:28:18 2020 From: shiyue.xw at alibaba-inc.com (=?UTF-8?B?5ZCR5LyfKOivhuaciCk=?=) Date: Fri, 13 Mar 2020 15:28:18 +0800 Subject: =?UTF-8?B?5Zue5aSN77yaTmVlZCBJbmZvcm1hdGlvbiBmb3IgbWVtb3J5IHNoYXJpbmcgd2l0aCBBcHBD?= =?UTF-8?B?RFM=?= In-Reply-To: References: <4c513dcc-ed4e-43ec-a607-764ded395058.shiyue.xw@alibaba-inc.com> <86cccf4b-e497-bd03-155a-c51fbcc9ae77@oracle.com> <7d348b97-161b-43d2-bd7b-896a50287a57.shiyue.xw@alibaba-inc.com>, , Message-ID: <54d454bb-b712-48c9-87cc-27bade3f47f6.shiyue.xw@alibaba-inc.com> Hi Jiangli, Ioi, Another question about the memory sharing with AppCDS: How to evaluate memory savings when multiple processes use the same archiving file? Thanks, Wei Xiang ------------------------------------------------------------------ ??????(??) ?????2020?3?10?(???) 10:39 ????Jiangli Zhou ????hotspot-runtime-dev ; hotspot-runtime-dev ???????Need Information for memory sharing with AppCDS Hi Jiangli, Thanks a lot for the detailed information. It's clear to me. Thanks, Wei Xiang ------------------------------------------------------------------ ????Jiangli Zhou ?????2020?3?10?(???) 00:50 ??????(??) ????hotspot-runtime-dev ; hotspot-runtime-dev ????Re: Need Information for memory sharing with AppCDS Hi, On Sun, Mar 8, 2020 at 11:56 PM ??(??) wrote: > > Hi Ioi, > > Thanks a lot for your information. > I have the question about the usage in the document: > For common class path prefix: > --------------------------------------- > Run the Hello and Hi applications with the same shared archive: > java -XX:SharedArchiveFile=common.jsa -cp common.jar:hello.jar:hi.jar Hello > java -XX:SharedArchiveFile=common.jsa -cp common.jar:hello.jar:hi.jar Hi > > If a class with name ABC is both in hello.jar and hi.jar, the content is different. > With the class path -cp common.jar:hello.jar:hi.jar, the second run "java .. Hi" can't load the correct class ABC in hi.jar. > For the above usage, I'd recommend to only archive the classes from common.jar to create a common.jsa. The common.jsa can be used for both applications for sharing. As described in the earlier email, the follow two conditions must be true to achieve sharing: - Both processes must be running with the same JVM binary - Both processes must be memory mapping using the same copy of common.jsa. If each container uses a different copy of the common.jsa, then there is no memory sharing. > Is there any solution for this usage? I'd recommend to not include ABC in the common.jsa in this case. Hope that helps. Best, Jiangli > > Thanks, > Wei Xiang > ------------------------------------------------------------------ > ????Ioi Lam > ?????2020?3?9?(???) 13:03 > ????hotspot-runtime-dev > ? ??Re: Need Information for memory sharing with AppCDS > > On 3/8/20 8:51 PM, ??(??) wrote: > > Hi, > > > > We am trying to take advantage of the characteristics of memory sharing with AppCDS, and we expected that the > > JVM internal packages and common packages can be shared with AppCDS. > > > > In the user scenario, there are some common base packages for different Java processes with "-cp common_path" > > Process A: -cp common_path:pathA > > Process B: -cp common_path:pathA:pathB > > Process C: -cp common_path:pathC > > > > I check the code in JDK11, and it uses SharedPathsMiscInfo to do the consistency check for class path. > > The below usage can work in JDK11 > > Dump stage: > > -cp common_path > > > > Runtime stage: > > -cp common_path:new_path > > > > But I find that SharedPathsMiscInfo has been removed in the latest JDK code. > > > > I need the information: > > 1. the new flow to do the consistency check. > > 2. constraint condition about the memory sharing among the different Java processes. > > > > Is there any user guide to use the memory sharing feature for AppCDS? > > Hi Wei, > > The topic of sharing the same CDS archive between different apps that > have the same classpath prefix can be found here: > > https://docs.oracle.com/en/java/javase/13/docs/specs/man/java.html#application-class-data-sharing > > See the section "Sharing a Shared Archive Across Multiple Application > Processes". > > SharedPathsMiscInfo was removed in this RFE in JDK 14: > https://bugs.openjdk.java.net/browse/JDK-8227370 > > However, we still enforce the same set of consistency checks as previous > JDK versions. > > Please let me know if you have further questions. > > Thanks > - Ioi > > > Thanks, > > Wei Xiang > From richard.reingruber at sap.com Fri Mar 13 09:08:51 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Fri, 13 Mar 2020 09:08:51 +0000 Subject: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents In-Reply-To: References: <1f8a3c7a-fa0f-b5b2-4a8a-7d3d8dbbe1b5@oracle.com> <4b56a45c-a14c-6f74-2bfd-25deaabe8201@oracle.com> <5271429a-481d-ddb9-99dc-b3f6670fcc0b@oracle.com> Message-ID: Hi Martin, thanks a lot for reviewing and the feedback. I'll dig into the details as soon as possible. Looking forward to it :) Thanks, Richard. -----Original Message----- From: Doerr, Martin Sent: Donnerstag, 12. M?rz 2020 17:28 To: Reingruber, Richard ; 'Robbin Ehn' ; Lindenmaier, Goetz ; David Holmes ; Vladimir Kozlov (vladimir.kozlov at oracle.com) ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents Hi Richard, I managed to find time for a (almost) complete review of webrev.4. (I'll review the tests separately.) First of all, the change seems to be in pretty good quality for its significant complexity. I couldn't find any real bugs. But I'd like to propose minor improvements. I'm convinced that it's mature because we did substantial testing. I like the new functionality for object deoptimization. It can possibly be reused for future escape analysis based optimizations. So I appreciate having it available in the code base. In addition to that, your change makes the JVMTI implementation better integrated into the VM. Now to the details: src/hotspot/share/c1/c1_IR.hpp describe_scope parameters. Ok. src/hotspot/share/ci/ciEnv.cpp src/hotspot/share/ci/ciEnv.hpp Fix for JvmtiExport::can_walk_any_space() capability. Ok. src/hotspot/share/code/compiledMethod.cpp Nice cleanup! src/hotspot/share/code/debugInfoRec.cpp src/hotspot/share/code/debugInfoRec.hpp Additional parmeters. (Remark: I think "non_global_escape_in_scope" would read better than "not_global_escape_in_scope", but your version is consistent with existing code, so no change request from my side.) Ok. src/hotspot/share/code/nmethod.cpp Nice cleanup! src/hotspot/share/code/pcDesc.hpp Additional parameters. Ok. src/hotspot/share/code/scopeDesc.cpp src/hotspot/share/code/scopeDesc.hpp Improved implementation + additional parameters. Ok. src/hotspot/share/compiler/compileBroker.cpp src/hotspot/share/compiler/compileBroker.hpp Extra thread for DeoptimizeObjectsALot. (Remark: I would have put it into a follow up change together with the test in order to make this webrev smaller, but since it is included, I'm reviewing everything at once. Not a big deal.) Ok. src/hotspot/share/jvmci/jvmciCodeInstaller.cpp Additional parameters. Ok. src/hotspot/share/opto/c2compiler.cpp Make do_escape_analysis independent of JVMCI capabilities. Nice! src/hotspot/share/opto/callnode.hpp Additional fields for MachSafePointNodes. Ok. src/hotspot/share/opto/escape.cpp Annotation for MachSafePointNodes. Your added functionality looks correct. But I'd prefer to move the bulky code out of the large function. I suggest to factor out something like has_not_global_escape and has_arg_escape. So the code could look like this: SafePointNode* sfn = sfn_worklist.at(next); sfn->set_not_global_escape_in_scope(has_not_global_escape(sfn)); if (sfn->is_CallJava()) { CallJavaNode* call = sfn->as_CallJava(); call->set_arg_escape(has_arg_escape(call)); } This would also allow us to get rid of the found_..._escape_in_args variables making the loops better readable. It's kind of ugly to use strcmp to recognize uncommon trap, but that seems to be the way to do it (there are more such places). So it's ok. src/hotspot/share/opto/machnode.hpp Additional fields for MachSafePointNodes. Ok. src/hotspot/share/opto/macro.cpp Allow elimination of non-escaping allocations. Ok. src/hotspot/share/opto/matcher.cpp src/hotspot/share/opto/output.cpp Copy attribute / pass parameters. Ok. src/hotspot/share/prims/jvmtiCodeBlobEvents.cpp Nice cleanup! src/hotspot/share/prims/jvmtiEnv.cpp src/hotspot/share/prims/jvmtiEnvBase.cpp Escape barriers + deoptimize objects for target thread. Good. src/hotspot/share/prims/jvmtiImpl.cpp src/hotspot/share/prims/jvmtiImpl.hpp The sequence is pretty complex: VM_GetOrSetLocal element initialization executes EscapeBarrier code which suspends the target thread (extra VM Operation). VM_GetOrSetLocal::doit_prologue performs object deoptimization (by VM Thread to prepare VM Operation with frame deoptimization). VM_GetOrSetLocal destructor implicitly calls EscapeBarrier destructor which resumes the target thread. But I don't have any improvement proposal. Performance is probably not a concern, here. So it's ok. VM_GetOrSetLocal::deoptimize_objects deoptimizes the top frame if it has non-globally escaping objects and other frames if they have arg escaping ones. Good. src/hotspot/share/prims/jvmtiTagMap.cpp Escape barriers + deoptimize objects for all threads. Ok. src/hotspot/share/prims/whitebox.cpp Added WB_IsFrameDeoptimized to API. Ok. src/hotspot/share/runtime/deoptimization.cpp Object deoptimization. I have more comments and proposals, here. First of all, handling recursive and waiting locks in relock_objects is tricky, but looks correct. Comments are sufficient to understand why things are done as they are implemented. BiasedLocking related parts are complex, but we may get rid of them in the future (with BiasedLocking removal). Anyway, looks correct, too. Typo in comment: "regularily" => "regularly" Deoptimization::fetch_unroll_info_helper is the only place where _jvmti_deferred_updates get deallocated (except JavaThread destructor). But I think we always go through it, so I can't see a memory leak or such kind of issues. EscapeBarrier::deoptimize_objects: ResourceMark should use calling_thread(). You can use MutexLocker and MonitorLocker with Thread* to save the Thread::current() call. I'd make set_objs_are_deoptimized static and remove it from the EscapeBarrier interface because I think it shouldn't be used outside of EscapeBarrier::deoptimize_objects. Typo in comment: "we must only deoptimize" => "we only have to deoptimize" "bool EscapeBarrier::deoptimize_objects(intptr_t* fr_id)" is trivial and barrier_active() is redundant. Implementation can get moved to hpp file. I'll get back to suspend flags, later. There are weird cases regarding _self_deoptimization_in_progress. Assume we have 3 threads A, B and C. A deopts C, B deopts C, C deopts C. C can set _self_deoptimization_in_progress while A performs the handshake for suspending C. I think this doesn't lead to errors, but it's probably not desired. I think it would be better to use only one "wait" call in sync_and_suspend_one and sync_and_suspend_all. I first thought it'd be better to move ThreadBlockInVM before wait() to reduce thread state transitions, but that seems to be problematic because ThreadBlockInVM destructor contains a safepoint check which we shouldn't do while holding EscapeBarrier_lock. So no change request. Change in thred_added: I think the sequence would be more comprehensive if we waited for deopt_all_threads in Thread::start and all other places where a new thread can run into Java code (e.g. JVMTI attach). Your version makes new threads come up with suspend flag set. That looks correct, too. Advantage is that you only have to change one place (thread_added). It'll be interesting to see how it will look like when we use async handshakes instead of suspend flags. For now, I'm ok with your version. I'd only move MutexLocker ml(EscapeBarrier_lock...) after if (!jt->is_hidden_from_external_view()). Having 4 different deoptimize_objects functions makes it a little hard to keep an overview of which one is used for what. Maybe adding suffixes would help a little bit, but I can also live with what you have. Implementation looks correct to me. src/hotspot/share/runtime/deoptimization.hpp Escape barriers and object deoptimization functions. Typo in comment: "helt" => "held" src/hotspot/share/runtime/globals.hpp Addition of develop flag DeoptimizeObjectsALotInterval. Ok. src/hotspot/share/runtime/interfaceSupport.cpp InterfaceSupport::deoptimizeAllObjects() is only used for DeoptimizeObjectsALot = 1. I think DeoptimizeObjectsALot = 2 is more important, but I think it's not bad to have DeoptimizeObjectsALot = 1 in addition. Ok. src/hotspot/share/runtime/interfaceSupport.inline.hpp Addition of deoptimizeAllObjects. Ok. src/hotspot/share/runtime/mutexLocker.cpp src/hotspot/share/runtime/mutexLocker.hpp Addition of EscapeBarrier_lock. Ok. src/hotspot/share/runtime/objectMonitor.cpp Make recursion count relock aware. Ok. src/hotspot/share/runtime/stackValue.hpp Better reinitilization in StackValue. Good. src/hotspot/share/runtime/thread.cpp src/hotspot/share/runtime/thread.hpp src/hotspot/share/runtime/thread.inline.hpp wait_for_object_deoptimization, suspend flag, deferred updates and test feature to deoptimize objects. In the long term, we want to get rid of suspend flags, so it's not so nice to introduce a new one. But I agree with G?tz that it should be acceptable as temporary solution until async handshakes are available (which takes more time). So I'm ok with your change. You can use MutexLocker with Thread*. JVMTIDeferredUpdates: I agree with Robin. It'd be nice to move the class out of thread.hpp. src/hotspot/share/runtime/vframe.cpp Added support for entry frame to new_vframe. Ok. src/hotspot/share/runtime/vframe_hp.cpp src/hotspot/share/runtime/vframe_hp.hpp I think code()->as_nmethod() in not_global_escape_in_scope() and arg_escape() should better be under #ifdef ASSERT or inside the assert statement (no need for code cache walking in product build). jvmtiDeferredLocalVariableSet::update_monitors: Please add a comment explaining that owner referenced by original info may be scalar replaced, but it is deoptimized in the vframe. src/hotspot/share/utilities/macros.hpp Addition of NOT_COMPILER2_OR_JVMCI_RETURN macros. Ok. test/hotspot/jtreg/serviceability/jvmti/Heap/IterateHeapWithEscapeAnalysisEnabled.java test/hotspot/jtreg/serviceability/jvmti/Heap/libIterateHeapWithEscapeAnalysisEnabled.c New test. Will review separately. test/jdk/TEST.ROOT Addition of vm.jvmci as required property. Ok. test/jdk/com/sun/jdi/EATests.java test/jdk/com/sun/jdi/EATestsJVMCI.java New test. Will review separately. test/lib/sun/hotspot/WhiteBox.java Added isFrameDeoptimized to API. Ok. That was it. Best regards, Martin > -----Original Message----- > From: hotspot-compiler-dev bounces at openjdk.java.net> On Behalf Of Reingruber, Richard > Sent: Dienstag, 3. M?rz 2020 21:23 > To: 'Robbin Ehn' ; Lindenmaier, Goetz > ; David Holmes ; > Vladimir Kozlov (vladimir.kozlov at oracle.com) > ; serviceability-dev at openjdk.java.net; > hotspot-compiler-dev at openjdk.java.net; hotspot-runtime- > dev at openjdk.java.net > Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better > Performance in the Presence of JVMTI Agents > > Hi Robbin, > > > > I understand that Robbin proposed to replace the usage of > > > _suspend_flag with handshakes. Apparently, async handshakes > > > are needed to do so. We have been waiting a while for removal > > > of the _suspend_flag / introduction of async handshakes [2]. > > > What is the status here? > > > I have an old prototype which I would like to continue to work on. > > So do not assume asynch handshakes will make 15. > > Even if it would, I think there are a lot more investigate work to remove > > _suspend_flag. > > Let us know, if we can be of any help to you and be it only testing. > > > >> Full: > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.4/ > > > DeoptimizeObjectsALotThread is only used in compileBroker.cpp. > > You can move both declaration and definition to that file, no need to > clobber > > thread.[c|h]pp. (and the static function deopt_objs_alot_thread_entry) > > Will do. > > > Does JvmtiDeferredUpdates really need to be in thread.hpp, can't be in it's > own > > hpp file? It doesn't seem right to add JVM TI classes into thread.hpp. > > You are right. It shouldn't be declared in thread.hpp. I will look into that. > > > Note that we also think we may have a bug in deopt: > > https://bugs.openjdk.java.net/browse/JDK-8238237 > > > I think it would be best, if possible, to push after that is resolved. > > Sure. > > > Not even nearly a full review :) > > I know :) > > Anyways, thanks a lot, > Richard. > > > -----Original Message----- > From: Robbin Ehn > Sent: Monday, March 2, 2020 11:17 AM > To: Lindenmaier, Goetz ; Reingruber, Richard > ; David Holmes ; > Vladimir Kozlov (vladimir.kozlov at oracle.com) > ; serviceability-dev at openjdk.java.net; > hotspot-compiler-dev at openjdk.java.net; hotspot-runtime- > dev at openjdk.java.net > Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better Performance > in the Presence of JVMTI Agents > > Hi, > > On 2/24/20 5:39 PM, Lindenmaier, Goetz wrote: > > Hi, > > > > I had a look at the progress of this change. Nothing > > happened since Richard posted his update using more > > handshakes [1]. > > But we (SAP) would appreciate a lot if this change could > > be successfully reviewed and pushed. > > > > I think there is basic understanding that this > > change is helpful. It fixes a number of issues with JVMTI, > > and will deliver the same performance benefits as EA > > does in current production mode for debugging scenarios. > > > > This is important for us as we run our VMs prepared > > for debugging in production mode. > > > > I understand that Robbin proposed to replace the usage of > > _suspend_flag with handshakes. Apparently, async handshakes > > are needed to do so. We have been waiting a while for removal > > of the _suspend_flag / introduction of async handshakes [2]. > > What is the status here? > > I have an old prototype which I would like to continue to work on. > So do not assume asynch handshakes will make 15. > Even if it would, I think there are a lot more investigate work to remove > _suspend_flag. > > > > > I think we should no longer wait, but proceed with > > this change. We will look into removing the usage of > > suspend_flag introduced here once it is possible to implement > > it with handshakes. > > Yes, sure. > > >> Full: http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.4/ > > DeoptimizeObjectsALotThread is only used in compileBroker.cpp. > You can move both declaration and definition to that file, no need to clobber > thread.[c|h]pp. (and the static function deopt_objs_alot_thread_entry) > > Does JvmtiDeferredUpdates really need to be in thread.hpp, can't be in it's > own > hpp file? It doesn't seem right to add JVM TI classes into thread.hpp. > > Note that we also think we may have a bug in deopt: > https://bugs.openjdk.java.net/browse/JDK-8238237 > > I think it would be best, if possible, to push after that is resolved. > > Not even nearly a full review :) > > Thanks, Robbin > > > >> Incremental: > >> http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.4.inc/ > >> > >> I was not able to eliminate the additional suspend flag now. I'll take care > of this > >> as soon as the > >> existing suspend-resume-mechanism is reworked. > >> > >> Testing: > >> > >> Nightly tests @SAP: > >> > >> JCK and JTREG, also in Xcomp mode, SPECjvm2008, SPECjbb2015, > Renaissance > >> Suite, SAP specific tests > >> with fastdebug and release builds on all platforms > >> > >> Stress testing with DeoptimizeObjectsALot running SPECjvm2008 40x > parallel > >> for 24h > >> > >> Thanks, Richard. > >> > >> > >> More details on the changes: > >> > >> * Hide DeoptimizeObjectsALotThread from external view. > >> > >> * Changed EscapeBarrier_lock to be a _safepoint_check_never lock. > >> It used to be _safepoint_check_sometimes, which will be eliminated > sooner or > >> later. > >> I added explicit thread state changes with ThreadBlockInVM to code > paths > >> where we can wait() > >> on EscapeBarrier_lock to become safepoint safe. > >> > >> * Use handshake EscapeBarrierSuspendHandshake to suspend target > threads > >> instead of vm operation > >> VM_ThreadSuspendAllForObjDeopt. > >> > >> * Removed uses of Threads_lock. When adding a new thread we suspend > it iff > >> EA optimizations are > >> being reverted. In the previous version we were waiting on > Threads_lock > >> while EA optimizations > >> were reverted. See EscapeBarrier::thread_added(). > >> > >> * Made tests require Xmixed compilation mode. > >> > >> * Made tests agnostic regarding tiered compilation. > >> I.e. tc isn't disabled anymore, and the tests can be run with tc enabled or > >> disabled. > >> > >> * Exercising EATests.java as well with stress test options > >> DeoptimizeObjectsALot* > >> Due to the non-deterministic deoptimizations some tests need to be > skipped. > >> We do this to prevent bit-rot of the stress test code. > >> > >> * Executing EATests.java as well with graal if available. Driver for this is > >> EATestsJVMCI.java. Graal cannot pass all tests, because it does not > provide all > >> the new debug info > >> (namely not_global_escape_in_scope and arg_escape in > scopeDesc.hpp). > >> And graal does not yet support the JVMTI operations force early return > and > >> pop frame. > >> > >> * Removed tracing from new jdi tests in EATests.java. Too much trace > output > >> before the debugging > >> connection is established can cause deadlock because output buffers fill > up. > >> (See https://bugs.openjdk.java.net/browse/JDK-8173304) > >> > >> * Many copyright year changes and smaller clean-up changes of testing > code > >> (trailing white-space and > >> the like). > >> > >> > >> -----Original Message----- > >> From: David Holmes > >> Sent: Donnerstag, 19. Dezember 2019 03:12 > >> To: Reingruber, Richard ; serviceability- > >> dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; > hotspot- > >> runtime-dev at openjdk.java.net; Vladimir Kozlov > (vladimir.kozlov at oracle.com) > >> > >> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better > Performance in > >> the Presence of JVMTI Agents > >> > >> Hi Richard, > >> > >> I think my issue is with the way EliminateNestedLocks works so I'm going > >> to look into that more deeply. > >> > >> Thanks for the explanations. > >> > >> David > >> > >> On 18/12/2019 12:47 am, Reingruber, Richard wrote: > >>> Hi David, > >>> > >>> > > > Some further queries/concerns: > >>> > > > > >>> > > > src/hotspot/share/runtime/objectMonitor.cpp > >>> > > > > >>> > > > Can you please explain the changes to ObjectMonitor::wait: > >>> > > > > >>> > > > ! _recursions = save // restore the old recursion count > >>> > > > ! + jt->get_and_reset_relock_count_after_wait(); // > >>> > > > increased by the deferred relock count > >>> > > > > >>> > > > what is the "deferred relock count"? I gather it relates to > >>> > > > > >>> > > > "The code was extended to be able to deoptimize objects of a > >>> > > frame that > >>> > > > is not the top frame and to let another thread than the owning > >>> > > thread do > >>> > > > it." > >>> > > > >>> > > Yes, these relate. Currently EA based optimizations are reverted, > when a > >> compiled frame is > >>> > > replaced with corresponding interpreter frames. Part of this is > relocking > >> objects with eliminated > >>> > > locking. New with the enhancement is that we do this also just > before > >> object references are > >>> > > acquired through JVMTI. In this case we deoptimize also the > owning > >> compiled frame C and we > >>> > > register deoptimized objects as deferred updates. When control > returns > >> to C it gets deoptimized, > >>> > > we notice that objects are already deoptimized (reallocated and > >> relocked), so we don't do it again > >>> > > (relocking twice would be incorrect of course). Deferred updates > are > >> copied into the new > >>> > > interpreter frames. > >>> > > > >>> > > Problem: relocking is not possible if the target thread T is waiting > on the > >> monitor that needs to > >>> > > be relocked. This happens only with non-local objects with > >> EliminateNestedLocks. Instead relocking > >>> > > is deferred until T owns the monitor again. This is what the piece of > >> code above does. > >>> > > >>> > Sorry I need some more detail here. How can you wait() on an > object > >>> > monitor if the object allocation and/or locking was optimised away? > And > >>> > what is a "non-local object" in this context? Isn't EA restricted to > >>> > thread-confined objects? > >>> > >>> "Non-local object" is an object that escapes its thread. The issue I'm > >> addressing with the changes > >>> in ObjectMonitor::wait are almost unrelated to EA. They are caused by > >> EliminateNestedLocks, where C2 > >>> eliminates recursive locking of an already owned lock. The lock owning > object > >> exists on the heap, it > >>> is locked and you can call wait() on it. > >>> > >>> EliminateLocks is the C2 option that controls lock elimination based on > EA. > >> Both optimizations have > >>> in common that objects with eliminated locking need to be relocked > when > >> deoptimizing a frame, > >>> i.e. when replacing a compiled frame with equivalent interpreter > >>> frames. Deoptimization::relock_objects does that job for /all/ eliminated > >> locks in scope. /All/ can > >>> be a mix of eliminated nested locks and locks of not-escaping objects. > >>> > >>> New with the enhancement: I call relock_objects earlier, just before > objects > >> pontentially > >>> escape. But then later when the owning compiled frame gets > deoptimized, I > >> must not do it again: > >>> > >>> See call to EscapeBarrier::objs_are_deoptimized in deoptimization.cpp: > >>> > >>> 373 if ((jvmci_enabled || ((DoEscapeAnalysis || > EliminateNestedLocks) && > >> EliminateLocks)) > >>> 374 && !EscapeBarrier::objs_are_deoptimized(thread, > deoptee.id())) { > >>> 375 bool unused; > >>> 376 eliminate_locks(thread, chunk, realloc_failures, deoptee, > exec_mode, > >> unused); > >>> 377 } > >>> > >>> Now when calling relock_objects early it is quiet possible that I have to > relock > >> an object the > >>> target thread currently waits for. Obviously I cannot relock in this case, > >> instead I chose to > >>> introduce relock_count_after_wait to JavaThread. > >>> > >>> > Is it just that some of the locking gets optimized away e.g. > >>> > > >>> > synchronised(obj) { > >>> > synchronised(obj) { > >>> > synchronised(obj) { > >>> > obj.wait(); > >>> > } > >>> > } > >>> > } > >>> > > >>> > If this is reduced to a form as-if it were a single lock of the monitor > >>> > (due to EA) and the wait() triggers a JVM TI event which leads to the > >>> > escape of "obj" then we need to reconstruct the true lock state, and > so > >>> > when the wait() internally unblocks and reacquires the monitor it > has to > >>> > set the true recursion count to 3, not the 1 that it appeared to be > when > >>> > wait() was initially called. Is that the scenario? > >>> > >>> Kind of... except that the locking is not eliminated due to EA and there is > no > >> JVM TI event > >>> triggered by wait. > >>> > >>> Add > >>> > >>> LocalObject l1 = new LocalObject(); > >>> > >>> in front of the synchrnized blocks and assume a JVM TI agent acquires l1. > This > >> triggers the code in > >>> question. > >>> > >>> See that relocking/reallocating is transactional. If it is done then for /all/ > >> objects in scope and it is > >>> done at most once. It wouldn't be quite so easy to split this in relocking > of > >> nested/EA-based > >>> eliminated locks. > >>> > >>> > If so I find this truly awful. Anyone using wait() in a realistic form > >>> > requires a notification and so the object cannot be thread confined. > In > >>> > >>> It is not thread confined. > >>> > >>> > which case I would strongly argue that upon hitting the wait() the > deopt > >>> > should occur unconditionally and so the lock state is correct before > we > >>> > wait and so we don't need to mess with the recursion count > internally > >>> > when we reacquire the monitor. > >>> > > >>> > > > >>> > > > which I don't like the sound of at all when it comes to > ObjectMonitor > >>> > > > state. So I'd like to understand in detail exactly what is going on > here > >>> > > > and why. This is a very intrusive change that seems to badly > break > >>> > > > encapsulation and impacts future changes to ObjectMonitor > that are > >> under > >>> > > > investigation. > >>> > > > >>> > > I would not regard this as breaking encapsulation. Certainly not > badly. > >>> > > > >>> > > I've added a property relock_count_after_wait to JavaThread. The > >> property is well > >>> > > encapsulated. Future ObjectMonitor implementations have to deal > with > >> recursion too. They are free > >>> > > in choosing a way to do that as long as that property is taken into > >> account. This is hardly a > >>> > > limitation. > >>> > > >>> > I do think this badly breaks encapsulation as you have to add a > callout > >>> > from the guts of the ObjectMonitor code to reach into the thread to > get > >>> > this lock count adjustment. I understand why you have had to do > this but > >>> > I would much rather see a change to the EA optimisation strategy so > that > >>> > this is not needed. > >>> > > >>> > > Note also that the property is a straight forward extension of the > >> existing concept of deferred > >>> > > local updates. It is embedded into the structure holding them. So > not > >> even the footprint of a > >>> > > JavaThread is enlarged if no deferred updates are generated. > >>> > > >>> > [...] > >>> > > >>> > > > >>> > > I'm actually duplicating the existing external suspend mechanism, > >> because a thread can be > >>> > > suspended at most once. And hey, and don't like that either! But it > >> seems not unlikely that the > >>> > > duplicate can be removed together with the original and the new > type > >> of handshakes that will be > >>> > > used for thread suspend can be used for object deoptimization > too. See > >> today's discussion in > >>> > > JDK-8227745 [2]. > >>> > > >>> > I hope that discussion bears some fruit, at the moment it seems not > to > >>> > be possible to use handshakes here. :( > >>> > > >>> > The external suspend mechanism is a royal pain in the proverbial > that we > >>> > have to carefully live with. The idea that we're duplicating that for > >>> > use in another fringe area of functionality does not thrill me at all. > >>> > > >>> > To be clear, I understand the problem that exists and that you wish > to > >>> > solve, but for the runtime parts I balk at the complexity cost of > >>> > solving it. > >>> > >>> I know it's complex, but by far no rocket science. > >>> > >>> Also I find it hard to imagine another fix for JDK-8233915 besides > changing > >> the JVM TI specification. > >>> > >>> Thanks, Richard. > >>> > >>> -----Original Message----- > >>> From: David Holmes > >>> Sent: Dienstag, 17. Dezember 2019 08:03 > >>> To: Reingruber, Richard ; serviceability- > >> dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; > hotspot- > >> runtime-dev at openjdk.java.net; Vladimir Kozlov > (vladimir.kozlov at oracle.com) > >> > >>> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better > Performance > >> in the Presence of JVMTI Agents > >>> > >>> > >>> > >>> David > >>> > >>> On 17/12/2019 4:57 pm, David Holmes wrote: > >>>> Hi Richard, > >>>> > >>>> On 14/12/2019 5:01 am, Reingruber, Richard wrote: > >>>>> Hi David, > >>>>> > >>>>> ?? > Some further queries/concerns: > >>>>> ?? > > >>>>> ?? > src/hotspot/share/runtime/objectMonitor.cpp > >>>>> ?? > > >>>>> ?? > Can you please explain the changes to ObjectMonitor::wait: > >>>>> ?? > > >>>>> ?? > !?? _recursions = save????? // restore the old recursion count > >>>>> ?? > !???????????????? + jt->get_and_reset_relock_count_after_wait(); // > >>>>> ?? > increased by the deferred relock count > >>>>> ?? > > >>>>> ?? > what is the "deferred relock count"? I gather it relates to > >>>>> ?? > > >>>>> ?? > "The code was extended to be able to deoptimize objects of a > >>>>> frame that > >>>>> ?? > is not the top frame and to let another thread than the owning > >>>>> thread do > >>>>> ?? > it." > >>>>> > >>>>> Yes, these relate. Currently EA based optimizations are reverted, > when > >>>>> a compiled frame is replaced > >>>>> with corresponding interpreter frames. Part of this is relocking > >>>>> objects with eliminated > >>>>> locking. New with the enhancement is that we do this also just before > >>>>> object references are acquired > >>>>> through JVMTI. In this case we deoptimize also the owning compiled > >>>>> frame C and we register > >>>>> deoptimized objects as deferred updates. When control returns to C > it > >>>>> gets deoptimized, we notice > >>>>> that objects are already deoptimized (reallocated and relocked), so > we > >>>>> don't do it again (relocking > >>>>> twice would be incorrect of course). Deferred updates are copied into > >>>>> the new interpreter frames. > >>>>> > >>>>> Problem: relocking is not possible if the target thread T is waiting > >>>>> on the monitor that needs to be > >>>>> relocked. This happens only with non-local objects with > >>>>> EliminateNestedLocks. Instead relocking is > >>>>> deferred until T owns the monitor again. This is what the piece of > >>>>> code above does. > >>>> > >>>> Sorry I need some more detail here. How can you wait() on an object > >>>> monitor if the object allocation and/or locking was optimised away? > And > >>>> what is a "non-local object" in this context? Isn't EA restricted to > >>>> thread-confined objects? > >>>> > >>>> Is it just that some of the locking gets optimized away e.g. > >>>> > >>>> synchronised(obj) { > >>>> ? synchronised(obj) { > >>>> ??? synchronised(obj) { > >>>> ????? obj.wait(); > >>>> ??? } > >>>> ? } > >>>> } > >>>> > >>>> If this is reduced to a form as-if it were a single lock of the monitor > >>>> (due to EA) and the wait() triggers a JVM TI event which leads to the > >>>> escape of "obj" then we need to reconstruct the true lock state, and so > >>>> when the wait() internally unblocks and reacquires the monitor it has to > >>>> set the true recursion count to 3, not the 1 that it appeared to be when > >>>> wait() was initially called. Is that the scenario? > >>>> > >>>> If so I find this truly awful. Anyone using wait() in a realistic form > >>>> requires a notification and so the object cannot be thread confined. In > >>>> which case I would strongly argue that upon hitting the wait() the > deopt > >>>> should occur unconditionally and so the lock state is correct before we > >>>> wait and so we don't need to mess with the recursion count internally > >>>> when we reacquire the monitor. > >>>> > >>>>> > >>>>> ?? > which I don't like the sound of at all when it comes to > >>>>> ObjectMonitor > >>>>> ?? > state. So I'd like to understand in detail exactly what is going > >>>>> on here > >>>>> ?? > and why.? This is a very intrusive change that seems to badly > break > >>>>> ?? > encapsulation and impacts future changes to ObjectMonitor that > >>>>> are under > >>>>> ?? > investigation. > >>>>> > >>>>> I would not regard this as breaking encapsulation. Certainly not badly. > >>>>> > >>>>> I've added a property relock_count_after_wait to JavaThread. The > >>>>> property is well > >>>>> encapsulated. Future ObjectMonitor implementations have to deal > with > >>>>> recursion too. They are free in > >>>>> choosing a way to do that as long as that property is taken into > >>>>> account. This is hardly a > >>>>> limitation. > >>>> > >>>> I do think this badly breaks encapsulation as you have to add a callout > >>>> from the guts of the ObjectMonitor code to reach into the thread to > get > >>>> this lock count adjustment. I understand why you have had to do this > but > >>>> I would much rather see a change to the EA optimisation strategy so > that > >>>> this is not needed. > >>>> > >>>>> Note also that the property is a straight forward extension of the > >>>>> existing concept of deferred > >>>>> local updates. It is embedded into the structure holding them. So not > >>>>> even the footprint of a > >>>>> JavaThread is enlarged if no deferred updates are generated. > >>>>> > >>>>> ?? > --- > >>>>> ?? > > >>>>> ?? > src/hotspot/share/runtime/thread.cpp > >>>>> ?? > > >>>>> ?? > Can you please explain why > >>>>> JavaThread::wait_for_object_deoptimization > >>>>> ?? > has to be handcrafted in this way rather than using proper > >>>>> transitions. > >>>>> ?? > > >>>>> > >>>>> I wrote wait_for_object_deoptimization taking > >>>>> JavaThread::java_suspend_self_with_safepoint_check > >>>>> as template. So in short: for the same reasons :) > >>>>> > >>>>> Threads reach both methods as part of thread state transitions, > >>>>> therefore special handling is > >>>>> required to change thread state on top of ongoing transitions. > >>>>> > >>>>> ?? > We got rid of "deopt suspend" some time ago and it is disturbing > >>>>> to see > >>>>> ?? > it being added back (effectively). This seems like it may be > >>>>> something > >>>>> ?? > that handshakes could be used for. > >>>>> > >>>>> Deopt suspend used to be something rather different with a similar > >>>>> name[1]. It is not being added back. > >>>> > >>>> I stand corrected. Despite comments in the code to the contrary > >>>> deopt_suspend didn't actually cause a self-suspend. I was doing a lot of > >>>> cleanup in this area 13 years ago :) > >>>> > >>>>> > >>>>> I'm actually duplicating the existing external suspend mechanism, > >>>>> because a thread can be suspended > >>>>> at most once. And hey, and don't like that either! But it seems not > >>>>> unlikely that the duplicate can > >>>>> be removed together with the original and the new type of > handshakes > >>>>> that will be used for > >>>>> thread suspend can be used for object deoptimization too. See > today's > >>>>> discussion in JDK-8227745 [2]. > >>>> > >>>> I hope that discussion bears some fruit, at the moment it seems not to > >>>> be possible to use handshakes here. :( > >>>> > >>>> The external suspend mechanism is a royal pain in the proverbial that > we > >>>> have to carefully live with. The idea that we're duplicating that for > >>>> use in another fringe area of functionality does not thrill me at all. > >>>> > >>>> To be clear, I understand the problem that exists and that you wish to > >>>> solve, but for the runtime parts I balk at the complexity cost of > >>>> solving it. > >>>> > >>>> Thanks, > >>>> David > >>>> ----- > >>>> > >>>>> Thanks, Richard. > >>>>> > >>>>> [1] Deopt suspend was something like an async. handshake for > >>>>> architectures with register windows, > >>>>> ???? where patching the return pc for deoptimization of a compiled > >>>>> frame was racy if the owner thread > >>>>> ???? was in native code. Instead a "deopt" suspend flag was set on > >>>>> which the thread patched its own > >>>>> ???? frame upon return from native. So no thread was suspended. It > got > >>>>> its name only from the name of > >>>>> ???? the flags. > >>>>> > >>>>> [2] Discussion about using handshakes to sync. with the target thread: > >>>>> > >>>>> https://bugs.openjdk.java.net/browse/JDK- > >> > 8227745?focusedCommentId=14306727&page=com.atlassian.jira.plugin.syst > e > >> m.issuetabpanels:comment-tabpanel#comment-14306727 > >>>>> > >>>>> > >>>>> -----Original Message----- > >>>>> From: David Holmes > >>>>> Sent: Freitag, 13. Dezember 2019 00:56 > >>>>> To: Reingruber, Richard ; > >>>>> serviceability-dev at openjdk.java.net; > >>>>> hotspot-compiler-dev at openjdk.java.net; > >>>>> hotspot-runtime-dev at openjdk.java.net > >>>>> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better > >>>>> Performance in the Presence of JVMTI Agents > >>>>> > >>>>> Hi Richard, > >>>>> > >>>>> Some further queries/concerns: > >>>>> > >>>>> src/hotspot/share/runtime/objectMonitor.cpp > >>>>> > >>>>> Can you please explain the changes to ObjectMonitor::wait: > >>>>> > >>>>> !?? _recursions = save????? // restore the old recursion count > >>>>> !???????????????? + jt->get_and_reset_relock_count_after_wait(); // > >>>>> increased by the deferred relock count > >>>>> > >>>>> what is the "deferred relock count"? I gather it relates to > >>>>> > >>>>> "The code was extended to be able to deoptimize objects of a frame > that > >>>>> is not the top frame and to let another thread than the owning thread > do > >>>>> it." > >>>>> > >>>>> which I don't like the sound of at all when it comes to ObjectMonitor > >>>>> state. So I'd like to understand in detail exactly what is going on here > >>>>> and why.? This is a very intrusive change that seems to badly break > >>>>> encapsulation and impacts future changes to ObjectMonitor that are > under > >>>>> investigation. > >>>>> > >>>>> --- > >>>>> > >>>>> src/hotspot/share/runtime/thread.cpp > >>>>> > >>>>> Can you please explain why > JavaThread::wait_for_object_deoptimization > >>>>> has to be handcrafted in this way rather than using proper transitions. > >>>>> > >>>>> We got rid of "deopt suspend" some time ago and it is disturbing to > see > >>>>> it being added back (effectively). This seems like it may be something > >>>>> that handshakes could be used for. > >>>>> > >>>>> Thanks, > >>>>> David > >>>>> ----- > >>>>> > >>>>> On 12/12/2019 7:02 am, David Holmes wrote: > >>>>>> On 12/12/2019 1:07 am, Reingruber, Richard wrote: > >>>>>>> Hi David, > >>>>>>> > >>>>>>> ??? > Most of the details here are in areas I can comment on in > detail, > >>>>>>> but I > >>>>>>> ??? > did take an initial general look at things. > >>>>>>> > >>>>>>> Thanks for taking the time! > >>>>>> > >>>>>> Apologies the above should read: > >>>>>> > >>>>>> "Most of the details here are in areas I *can't* comment on in detail > >>>>>> ..." > >>>>>> > >>>>>> David > >>>>>> > >>>>>>> ??? > The only thing that jumped out at me is that I think the > >>>>>>> ??? > DeoptimizeObjectsALotThread should be a hidden thread. > >>>>>>> ??? > > >>>>>>> ??? > +? bool is_hidden_from_external_view() const { return true; } > >>>>>>> > >>>>>>> Yes, it should. Will add the method like above. > >>>>>>> > >>>>>>> ??? > Also I don't see any testing of the > DeoptimizeObjectsALotThread. > >>>>>>> Without > >>>>>>> ??? > active testing this will just bit-rot. > >>>>>>> > >>>>>>> DeoptimizeObjectsALot is meant for stress testing with a larger > >>>>>>> workload. I will add a minimal test > >>>>>>> to keep it fresh. > >>>>>>> > >>>>>>> ??? > Also on the tests I don't understand your @requires clause: > >>>>>>> ??? > > >>>>>>> ??? >?? @requires ((vm.compMode != "Xcomp") & > vm.compiler2.enabled > >> & > >>>>>>> ??? > (vm.opt.TieredCompilation != true)) > >>>>>>> ??? > > >>>>>>> ??? > This seems to require that TieredCompilation is disabled, but > >>>>>>> tiered is > >>>>>>> ??? > our normal mode of operation. ?? > >>>>>>> ??? > > >>>>>>> > >>>>>>> I removed the clause. I guess I wanted to target the tests towards > the > >>>>>>> code they are supposed to > >>>>>>> test, and it's easier to analyze failures w/o tiered compilation and > >>>>>>> with just one compiler thread. > >>>>>>> > >>>>>>> Additionally I will make use of > >>>>>>> compiler.whitebox.CompilerWhiteBoxTest.THRESHOLD in the tests. > >>>>>>> > >>>>>>> Thanks, > >>>>>>> Richard. > >>>>>>> > >>>>>>> -----Original Message----- > >>>>>>> From: David Holmes > >>>>>>> Sent: Mittwoch, 11. Dezember 2019 08:03 > >>>>>>> To: Reingruber, Richard ; > >>>>>>> serviceability-dev at openjdk.java.net; > >>>>>>> hotspot-compiler-dev at openjdk.java.net; > >>>>>>> hotspot-runtime-dev at openjdk.java.net > >>>>>>> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better > >>>>>>> Performance in the Presence of JVMTI Agents > >>>>>>> > >>>>>>> Hi Richard, > >>>>>>> > >>>>>>> On 11/12/2019 7:45 am, Reingruber, Richard wrote: > >>>>>>>> Hi, > >>>>>>>> > >>>>>>>> I would like to get reviews please for > >>>>>>>> > >>>>>>>> > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.3/ > >>>>>>>> > >>>>>>>> Corresponding RFE: > >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8227745 > >>>>>>>> > >>>>>>>> Fixes also https://bugs.openjdk.java.net/browse/JDK-8233915 > >>>>>>>> And potentially https://bugs.openjdk.java.net/browse/JDK- > 8214584 [1] > >>>>>>>> > >>>>>>>> Vladimir Kozlov kindly put webrev.3 through tier1-8 testing > without > >>>>>>>> issues (thanks!). In addition the > >>>>>>>> change is being tested at SAP since I posted the first RFR some > >>>>>>>> months ago. > >>>>>>>> > >>>>>>>> The intention of this enhancement is to benefit performance wise > from > >>>>>>>> escape analysis even if JVMTI > >>>>>>>> agents request capabilities that allow them to access local variable > >>>>>>>> values. E.g. if you start-up > >>>>>>>> with -agentlib:jdwp=transport=dt_socket,server=y,suspend=n, > then > >>>>>>>> escape analysis is disabled right > >>>>>>>> from the beginning, well before a debugger attaches -- if ever one > >>>>>>>> should do so. With the > >>>>>>>> enhancement, escape analysis will remain enabled until and after > a > >>>>>>>> debugger attaches. EA based > >>>>>>>> optimizations are reverted just before an agent acquires the > >>>>>>>> reference to an object. In the JBS item > >>>>>>>> you'll find more details. > >>>>>>> > >>>>>>> Most of the details here are in areas I can comment on in detail, but > I > >>>>>>> did take an initial general look at things. > >>>>>>> > >>>>>>> The only thing that jumped out at me is that I think the > >>>>>>> DeoptimizeObjectsALotThread should be a hidden thread. > >>>>>>> > >>>>>>> +? bool is_hidden_from_external_view() const { return true; } > >>>>>>> > >>>>>>> Also I don't see any testing of the DeoptimizeObjectsALotThread. > >>>>>>> Without > >>>>>>> active testing this will just bit-rot. > >>>>>>> > >>>>>>> Also on the tests I don't understand your @requires clause: > >>>>>>> > >>>>>>> ??? @requires ((vm.compMode != "Xcomp") & > vm.compiler2.enabled & > >>>>>>> (vm.opt.TieredCompilation != true)) > >>>>>>> > >>>>>>> This seems to require that TieredCompilation is disabled, but tiered > is > >>>>>>> our normal mode of operation. ?? > >>>>>>> > >>>>>>> Thanks, > >>>>>>> David > >>>>>>> > >>>>>>>> Thanks, > >>>>>>>> Richard. > >>>>>>>> > >>>>>>>> [1] Experimental fix for JDK-8214584 based on JDK-8227745 > >>>>>>>> > >> > http://cr.openjdk.java.net/~rrich/webrevs/2019/8214584/experiment_v1.pa > tc > >> h > >>>>>>>> > >>>>>>>> > >>>>>>>> From ralf.schmelter at sap.com Fri Mar 13 11:43:08 2020 From: ralf.schmelter at sap.com (Schmelter, Ralf) Date: Fri, 13 Mar 2020 11:43:08 +0000 Subject: RFR(L) 8237354: Add option to jcmd to write a gzipped heap dump In-Reply-To: References: <01361a9d-2855-db67-a176-73731fada08f@oracle.com> <0c687e55-ed91-e606-28a7-f9aef745ed8d@oracle.com> <490d58f7-7adc-00aa-b504-0ac284fe7eb5@oracle.com> <0343dfac-61f7-1b1c-ee96-bdee130578ad@oracle.com> Message-ID: Hi, I have updated the webrev: http://cr.openjdk.java.net/~rschmelter/webrevs/8237354/webrev.1/ It has the following significant changes: - The jcmd now uses two separate flags. The -gz flag is now a boolean flag which toggles the compression on/off. And the new -gz-level flag can be used to change the compression level. If tried to change the jlong flag coding to allow the old behavior (only one flag, which acts both as a boolean flag and a jlong flag), but decided against it, since it changes the semantic of a jlong flag. And I don't expect the -gz-level flag to be used all that much. - I no longer use my own threads. Instead I use the WorkGang returned from CollectedHeap:: get_safepoint_workers(). This works fine, apart from Shenandoah GC, which runs into assertions when calling the CollectedHeap::object_iterate() method from a worker thread. I'm not sure if the assertion is too strong, but since the GC is currently experimental, I switch back to single threading in this case (as would be the case for serial GC or epsilon GC). Using the worker threads removes the problems the original code had regarding destruction of the monitor used. - The reported number of bytes is now the one written to disk. Best regards, Ralf -----Original Message----- From: Ioi Lam Sent: Dienstag, 25. Februar 2020 18:03 To: Langer, Christoph ; Schmelter, Ralf ; Yasumasa Suenaga ; serguei.spitsyn at oracle.com; hotspot-runtime-dev at openjdk.java.net runtime Cc: serviceability-dev at openjdk.java.net Subject: Re: RFR(L) 8237354: Add option to jcmd to write a gzipped heap dump Hi Christoph, This sounds fair. I will remove my objection :-) Thanks - Ioi From claes.redestad at oracle.com Fri Mar 13 13:56:15 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Fri, 13 Mar 2020 14:56:15 +0100 Subject: RFR(T?): 8241006: Cleanup TemplateInterpreter initialization Message-ID: <3a25716d-d527-7b9c-b54a-55233f34a3bb@oracle.com> Hi, some initialization is unnecessarily called twice. Also clean up some unused things in the neighborhood, and a TraceTime around what is effectively compiled down to ~1200 instructions. bug: https://bugs.openjdk.java.net/browse/JDK-8241006 webrev: http://cr.openjdk.java.net/~redestad/8241006/open.00/index.html Testing: tier1-2 Thanks! /Claes From coleen.phillimore at oracle.com Fri Mar 13 14:27:42 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 13 Mar 2020 10:27:42 -0400 Subject: RFR(T?): 8241006: Cleanup TemplateInterpreter initialization In-Reply-To: <3a25716d-d527-7b9c-b54a-55233f34a3bb@oracle.com> References: <3a25716d-d527-7b9c-b54a-55233f34a3bb@oracle.com> Message-ID: This looks good to me.? And trivial.? Can you remove the extra scope here: http://cr.openjdk.java.net/~redestad/8241006/open.00/src/hotspot/share/memory/universe.cpp.udiff.html Around if (!UseSharedSpaces) {} thanks, Coleen On 3/13/20 9:56 AM, Claes Redestad wrote: > Hi, > > some initialization is unnecessarily called twice. Also clean up some > unused things in the neighborhood, and a TraceTime around what is > effectively compiled down to ~1200 instructions. > > bug:??? https://bugs.openjdk.java.net/browse/JDK-8241006 > webrev: http://cr.openjdk.java.net/~redestad/8241006/open.00/index.html > > Testing: tier1-2 > > Thanks! > > /Claes From claes.redestad at oracle.com Fri Mar 13 15:10:58 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Fri, 13 Mar 2020 16:10:58 +0100 Subject: RFR(T?): 8241006: Cleanup TemplateInterpreter initialization In-Reply-To: References: <3a25716d-d527-7b9c-b54a-55233f34a3bb@oracle.com> Message-ID: On 2020-03-13 15:27, coleen.phillimore at oracle.com wrote: > This looks good to me.? And trivial.? Can you remove the extra scope here: Thanks! Removing scope and sanity testing before push. /Claes From claes.redestad at oracle.com Fri Mar 13 15:52:40 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Fri, 13 Mar 2020 16:52:40 +0100 Subject: RFR(T): 8241010: Unnecessarily resolving some well-known classes Message-ID: Hi, for well-known classes, we can replace a SystemDictionary resolve with corresponding Klass_klass() accessor. Bug: https://bugs.openjdk.java.net/browse/JDK-8241010 Webrev: http://cr.openjdk.java.net/~redestad/8241010/open.00/index.html Testing: tier1+2 Thanks! /Claes From coleen.phillimore at oracle.com Fri Mar 13 15:51:49 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 13 Mar 2020 11:51:49 -0400 Subject: RFR(T): 8241010: Unnecessarily resolving some well-known classes In-Reply-To: References: Message-ID: Yes, also trivial.? Looks good! Coleen On 3/13/20 11:52 AM, Claes Redestad wrote: > Hi, > > for well-known classes, we can replace a SystemDictionary resolve with > corresponding Klass_klass() accessor. > > Bug:??? https://bugs.openjdk.java.net/browse/JDK-8241010 > Webrev: http://cr.openjdk.java.net/~redestad/8241010/open.00/index.html > > Testing: tier1+2 > > Thanks! > > /Claes From jianglizhou at google.com Fri Mar 13 16:32:16 2020 From: jianglizhou at google.com (Jiangli Zhou) Date: Fri, 13 Mar 2020 09:32:16 -0700 Subject: Need Information for memory sharing with AppCDS In-Reply-To: <54d454bb-b712-48c9-87cc-27bade3f47f6.shiyue.xw@alibaba-inc.com> References: <4c513dcc-ed4e-43ec-a607-764ded395058.shiyue.xw@alibaba-inc.com> <86cccf4b-e497-bd03-155a-c51fbcc9ae77@oracle.com> <7d348b97-161b-43d2-bd7b-896a50287a57.shiyue.xw@alibaba-inc.com> <54d454bb-b712-48c9-87cc-27bade3f47f6.shiyue.xw@alibaba-inc.com> Message-ID: Hi, One method is to use 'top'. You can run multiple JVM instances with and without enabling the shared archive, and compare the 'RES' and 'SHR' memory reported by 'top'. 'RES' is the resident size. 'SHR' is the shared memory size, which is the amount of shared memory (can be potentially shared with other processes) used by a process. /proc//smaps also gives the memory consumption of a process. Best, Jiangli On Fri, Mar 13, 2020 at 12:38 AM ??(??) wrote: > > Hi Jiangli, Ioi, > > Another question about the memory sharing with AppCDS: > How to evaluate memory savings when multiple processes use the same archiving file? > > Thanks, > Wei Xiang > > ------------------------------------------------------------------ > ??????(??) > ?????2020?3?10?(???) 10:39 > ????Jiangli Zhou > ????hotspot-runtime-dev ; hotspot-runtime-dev > ???????Need Information for memory sharing with AppCDS > > Hi Jiangli, > > Thanks a lot for the detailed information. It's clear to me. > > Thanks, > Wei Xiang > ------------------------------------------------------------------ > ????Jiangli Zhou > ?????2020?3?10?(???) 00:50 > ??????(??) > ????hotspot-runtime-dev ; hotspot-runtime-dev > ????Re: Need Information for memory sharing with AppCDS > > Hi, > > On Sun, Mar 8, 2020 at 11:56 PM ??(??) wrote: > > > > Hi Ioi, > > > > Thanks a lot for your information. > > I have the question about the usage in the document: > > For common class path prefix: > > --------------------------------------- > > Run the Hello and Hi applications with the same shared archive: > > java -XX:SharedArchiveFile=common.jsa -cp common.jar:hello.jar:hi.jar Hello > > java -XX:SharedArchiveFile=common.jsa -cp common.jar:hello.jar:hi.jar Hi > > > > If a class with name ABC is both in hello.jar and hi.jar, the content is different. > > With the class path -cp common.jar:hello.jar:hi.jar, the second run "java .. Hi" can't load the correct class ABC in hi.jar. > > > > For the above usage, I'd recommend to only archive the classes from > common.jar to create a common.jsa. The common.jsa can be used for both > applications for sharing. As described in the earlier email, the > follow two conditions must be true to achieve sharing: > > - Both processes must be running with the same JVM binary > - Both processes must be memory mapping using the same copy of common.jsa. > > If each container uses a different copy of the common.jsa, then there > is no memory sharing. > > > Is there any solution for this usage? > > I'd recommend to not include ABC in the common.jsa in this case. > > Hope that helps. > > Best, > Jiangli > > > > Thanks, > > Wei Xiang > > ------------------------------------------------------------------ > > ????Ioi Lam > > ?????2020?3?9?(???) 13:03 > > ????hotspot-runtime-dev > > ? ??Re: Need Information for memory sharing with AppCDS > > > > On 3/8/20 8:51 PM, ??(??) wrote: > > > Hi, > > > > > > We am trying to take advantage of the characteristics of memory sharing with AppCDS, and we expected that the > > > JVM internal packages and common packages can be shared with AppCDS. > > > > > > In the user scenario, there are some common base packages for different Java processes with "-cp common_path" > > > Process A: -cp common_path:pathA > > > Process B: -cp common_path:pathA:pathB > > > Process C: -cp common_path:pathC > > > > > > I check the code in JDK11, and it uses SharedPathsMiscInfo to do the consistency check for class path. > > > The below usage can work in JDK11 > > > Dump stage: > > > -cp common_path > > > > > > Runtime stage: > > > -cp common_path:new_path > > > > > > But I find that SharedPathsMiscInfo has been removed in the latest JDK code. > > > > > > I need the information: > > > 1. the new flow to do the consistency check. > > > 2. constraint condition about the memory sharing among the different Java processes. > > > > > > Is there any user guide to use the memory sharing feature for AppCDS? > > > > Hi Wei, > > > > The topic of sharing the same CDS archive between different apps that > > have the same classpath prefix can be found here: > > > > https://docs.oracle.com/en/java/javase/13/docs/specs/man/java.html#application-class-data-sharing > > > > See the section "Sharing a Shared Archive Across Multiple Application > > Processes". > > > > SharedPathsMiscInfo was removed in this RFE in JDK 14: > > https://bugs.openjdk.java.net/browse/JDK-8227370 > > > > However, we still enforce the same set of consistency checks as previous > > JDK versions. > > > > Please let me know if you have further questions. > > > > Thanks > > - Ioi > > > > > Thanks, > > > Wei Xiang > > > > From ioi.lam at oracle.com Fri Mar 13 18:16:39 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 13 Mar 2020 11:16:39 -0700 Subject: =?UTF-8?B?UmU6IOWbnuWkje+8mk5lZWQgSW5mb3JtYXRpb24gZm9yIG1lbW9yeSBz?= =?UTF-8?Q?haring_with_AppCDS?= In-Reply-To: <54d454bb-b712-48c9-87cc-27bade3f47f6.shiyue.xw@alibaba-inc.com> References: <4c513dcc-ed4e-43ec-a607-764ded395058.shiyue.xw@alibaba-inc.com> <86cccf4b-e497-bd03-155a-c51fbcc9ae77@oracle.com> <7d348b97-161b-43d2-bd7b-896a50287a57.shiyue.xw@alibaba-inc.com> <54d454bb-b712-48c9-87cc-27bade3f47f6.shiyue.xw@alibaba-inc.com> Message-ID: <2c3dd710-8a40-6f77-4800-44c79f7f6dd8@oracle.com> I usually use ps_mem.py https://github.com/pixelb/ps_mem/blob/master/ps_mem.py java -cp ... app1 & java -cp ... app2 & ps_mem.py | grep java Thanks - Ioi On 3/13/20 12:28 AM, ??(??) wrote: > Hi Jiangli, Ioi, > > Another question about the memory sharing with AppCDS: > How?to?evaluate?memory?savings when multiple processes use the same > archiving file? > > Thanks, > Wei Xiang > > ------------------------------------------------------------------ > ??????(??) > ?????2020?3?10?(???) 10:39 > ????Jiangli Zhou > ????hotspot-runtime-dev > ; > hotspot-runtime-dev > ???????Need Information for memory sharing with AppCDS > > Hi Jiangli, > > Thanks a lot for the detailed?information. It's?clear?to?me. > > Thanks, > Wei Xiang > ------------------------------------------------------------------ > ????Jiangli Zhou > ?????2020?3?10?(???) 00:50 > ??????(??) > ????hotspot-runtime-dev > ; > hotspot-runtime-dev > ????Re: Need Information for memory sharing with AppCDS > > Hi, > > On?Sun,?Mar?8,?2020?at?11:56?PM???(??)??wrote: > > > >?Hi?Ioi, > > > >?Thanks?a?lot?for?your?information. > >?I?have?the?question?about?the?usage?in?the?document: > >?For?common?class?path?prefix: > >?--------------------------------------- > >?????Run?the?Hello?and?Hi?applications?with?the?same?shared?archive: > >?java?-XX:SharedArchiveFile=common.jsa?-cp?common.jar:hello.jar:hi.jar?Hello > >?java?-XX:SharedArchiveFile=common.jsa?-cp?common.jar:hello.jar:hi.jar?Hi > > > >?If?a?class?with?name?ABC?is?both?in?hello.jar?and?hi.jar,?the?content?is?different. > >?With?the?class?path?-cp?common.jar:hello.jar:hi.jar,?the?second?run?"java?..?Hi"?can't?load?the?correct?class?ABC?in?hi.jar. > > > > For?the?above?usage,?I'd?recommend?to?only?archive?the?classes?from > common.jar?to?create?a?common.jsa.?The?common.jsa?can?be?used?for?both > applications?for?sharing.?As?described?in?the?earlier?email,?the > follow?two?conditions?must?be?true?to?achieve?sharing: > > -?Both?processes?must?be?running?with?the?same?JVM?binary > -?Both?processes?must?be?memory?mapping?using?the?same?copy?of?common.jsa. > > If?each?container?uses?a?different?copy?of?the?common.jsa,?then?there > is?no?memory?sharing. > > >?Is?there?any?solution?for?this?usage? > > I'd?recommend?to?not?include?ABC?in?the?common.jsa?in?this?case. > > Hope?that?helps. > > Best, > Jiangli > > > >?Thanks, > >?Wei?Xiang > >?------------------------------------------------------------------ > >?????Ioi?Lam? > >??????2020?3?9?(???)?13:03 > >?????hotspot-runtime-dev? > >?????Re:?Need?Information?for?memory?sharing?with?AppCDS > > > >?On?3/8/20?8:51?PM,???(??)?wrote: > >?>?Hi, > >?> > >?>?We?am?trying?to?take?advantage?of?the?characteristics?of?memory?sharing?with?AppCDS,?and?we?expected?that?the > >?>?JVM?internal?packages?and?common?packages?can?be?shared?with?AppCDS. > >?> > >?>?In?the?user?scenario,?there?are?some?common?base?packages?for?different?Java?processes?with?"-cp?common_path" > >?>?Process?A:?-cp?common_path:pathA > >?>?Process?B:?-cp?common_path:pathA:pathB > >?>?Process?C:?-cp?common_path:pathC > >?> > >?>?I?check?the?code?in?JDK11,?and?it?uses?SharedPathsMiscInfo?to?do?the?consistency?check?for?class?path. > >?>?The?below?usage?can?work?in?JDK11 > >?>?Dump?stage: > >?>?-cp?common_path > >?> > >?>?Runtime?stage: > >?>?-cp??common_path:new_path > >?> > >?>?But?I?find?that?SharedPathsMiscInfo?has?been?removed?in?the?latest?JDK?code. > >?> > >?>?I?need?the?information: > >?>?1.?the?new?flow?to?do?the?consistency?check. > >?>?2.?constraint?condition?about?the?memory?sharing?among?the?different?Java?processes. > >?> > >?>?Is?there?any?user?guide?to?use?the?memory?sharing?feature?for?AppCDS? > > > >?Hi?Wei, > > > >?The?topic?of?sharing?the?same?CDS?archive?between?different?apps?that > >?have?the?same?classpath?prefix?can?be?found?here: > > > > > https://docs.oracle.com/en/java/javase/13/docs/specs/man/java.html#application-class-data-sharing > > > >?See?the?section?"Sharing?a?Shared?Archive?Across?Multiple?Application > >?Processes". > > > >?SharedPathsMiscInfo?was?removed?in?this?RFE?in?JDK?14: > > https://bugs.openjdk.java.net/browse/JDK-8227370 > > > >?However,?we?still?enforce?the?same?set?of?consistency?checks?as?previous > >?JDK?versions. > > > >?Please?let?me?know?if?you?have?further?questions. > > > >?Thanks > >?-?Ioi > > > >?>?Thanks, > >?>?Wei?Xiang > > > > From coleen.phillimore at oracle.com Fri Mar 13 19:17:31 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 13 Mar 2020 15:17:31 -0400 Subject: =?UTF-8?B?UmU6IOWbnuWkje+8mk5lZWQgSW5mb3JtYXRpb24gZm9yIG1lbW9yeSBz?= =?UTF-8?Q?haring_with_AppCDS?= In-Reply-To: <2c3dd710-8a40-6f77-4800-44c79f7f6dd8@oracle.com> References: <4c513dcc-ed4e-43ec-a607-764ded395058.shiyue.xw@alibaba-inc.com> <86cccf4b-e497-bd03-155a-c51fbcc9ae77@oracle.com> <7d348b97-161b-43d2-bd7b-896a50287a57.shiyue.xw@alibaba-inc.com> <54d454bb-b712-48c9-87cc-27bade3f47f6.shiyue.xw@alibaba-inc.com> <2c3dd710-8a40-6f77-4800-44c79f7f6dd8@oracle.com> Message-ID: Hi, Can you remove hotspot-runtime-dev-bounces at openjdk.java.net from this thread, please? thanks, Coleen On 3/13/20 2:16 PM, Ioi Lam wrote: > I usually use ps_mem.py > > https://github.com/pixelb/ps_mem/blob/master/ps_mem.py > > java -cp ... app1 & > java -cp ... app2 & > ps_mem.py | grep java > > Thanks > - Ioi > > > > On 3/13/20 12:28 AM, ??(??) wrote: >> Hi Jiangli, Ioi, >> >> Another question about the memory sharing with AppCDS: >> How?to?evaluate?memory?savings when multiple processes use the same >> archiving file? >> >> Thanks, >> Wei Xiang >> >> ------------------------------------------------------------------ >> ??? ??????(??) >> ??? ?????2020?3?10?(???) 10:39 >> ??? ????Jiangli Zhou >> ??? ????hotspot-runtime-dev >> ??? ; >> ??? hotspot-runtime-dev >> ??? ???????Need Information for memory sharing with AppCDS >> >> ??? Hi Jiangli, >> >> ??? Thanks a lot for the detailed?information. It's?clear?to?me. >> >> ??? Thanks, >> ??? Wei Xiang >> ------------------------------------------------------------------ >> ??? ????Jiangli Zhou >> ??? ?????2020?3?10?(???) 00:50 >> ??? ??????(??) >> ??? ????hotspot-runtime-dev >> ??? ; >> ??? hotspot-runtime-dev >> ??? ????Re: Need Information for memory sharing with AppCDS >> >> ??? Hi, >> >> On?Sun,?Mar?8,?2020?at?11:56?PM???(??)??wrote: >> ??? > >> ??? >?Hi?Ioi, >> ??? > >> ??? >?Thanks?a?lot?for?your?information. >> ??? >?I?have?the?question?about?the?usage?in?the?document: >> ??? >?For?common?class?path?prefix: >> ??? >?--------------------------------------- >> >?????Run?the?Hello?and?Hi?applications?with?the?same?shared?archive: >> >?java?-XX:SharedArchiveFile=common.jsa?-cp?common.jar:hello.jar:hi.jar?Hello >> >?java?-XX:SharedArchiveFile=common.jsa?-cp?common.jar:hello.jar:hi.jar?Hi >> ??? > >> >?If?a?class?with?name?ABC?is?both?in?hello.jar?and?hi.jar,?the?content?is?different. >> >?With?the?class?path?-cp?common.jar:hello.jar:hi.jar,?the?second?run?"java?..?Hi"?can't?load?the?correct?class?ABC?in?hi.jar. >> ??? > >> >> For?the?above?usage,?I'd?recommend?to?only?archive?the?classes?from >> common.jar?to?create?a?common.jsa.?The?common.jsa?can?be?used?for?both >> applications?for?sharing.?As?described?in?the?earlier?email,?the >> ??? follow?two?conditions?must?be?true?to?achieve?sharing: >> >> ??? -?Both?processes?must?be?running?with?the?same?JVM?binary >> -?Both?processes?must?be?memory?mapping?using?the?same?copy?of?common.jsa. >> >> If?each?container?uses?a?different?copy?of?the?common.jsa,?then?there >> ??? is?no?memory?sharing. >> >> ??? >?Is?there?any?solution?for?this?usage? >> >> I'd?recommend?to?not?include?ABC?in?the?common.jsa?in?this?case. >> >> ??? Hope?that?helps. >> >> ??? Best, >> ??? Jiangli >> ??? > >> ??? >?Thanks, >> ??? >?Wei?Xiang >> >?------------------------------------------------------------------ >> ??? >?????Ioi?Lam? >> ??? >??????2020?3?9?(???)?13:03 >> >?????hotspot-runtime-dev? >> ??? >?????Re:?Need?Information?for?memory?sharing?with?AppCDS >> ??? > >> ??? >?On?3/8/20?8:51?PM,???(??)?wrote: >> ??? >?>?Hi, >> ??? >?> >> >?>?We?am?trying?to?take?advantage?of?the?characteristics?of?memory?sharing?with?AppCDS,?and?we?expected?that?the >> >?>?JVM?internal?packages?and?common?packages?can?be?shared?with?AppCDS. >> ??? >?> >> >?>?In?the?user?scenario,?there?are?some?common?base?packages?for?different?Java?processes?with?"-cp?common_path" >> ??? >?>?Process?A:?-cp?common_path:pathA >> ??? >?>?Process?B:?-cp?common_path:pathA:pathB >> ??? >?>?Process?C:?-cp?common_path:pathC >> ??? >?> >> >?>?I?check?the?code?in?JDK11,?and?it?uses?SharedPathsMiscInfo?to?do?the?consistency?check?for?class?path. >> ??? >?>?The?below?usage?can?work?in?JDK11 >> ??? >?>?Dump?stage: >> ??? >?>?-cp?common_path >> ??? >?> >> ??? >?>?Runtime?stage: >> ??? >?>?-cp??common_path:new_path >> ??? >?> >> >?>?But?I?find?that?SharedPathsMiscInfo?has?been?removed?in?the?latest?JDK?code. >> ??? >?> >> ??? >?>?I?need?the?information: >> ??? >?>?1.?the?new?flow?to?do?the?consistency?check. >> >?>?2.?constraint?condition?about?the?memory?sharing?among?the?different?Java?processes. >> ??? >?> >> >?>?Is?there?any?user?guide?to?use?the?memory?sharing?feature?for?AppCDS? >> ??? > >> ??? >?Hi?Wei, >> ??? > >> >?The?topic?of?sharing?the?same?CDS?archive?between?different?apps?that >> ??? >?have?the?same?classpath?prefix?can?be?found?here: >> ??? > >> ??? > >> https://docs.oracle.com/en/java/javase/13/docs/specs/man/java.html#application-class-data-sharing >> ??? > >> >?See?the?section?"Sharing?a?Shared?Archive?Across?Multiple?Application >> ??? >?Processes". >> ??? > >> ??? >?SharedPathsMiscInfo?was?removed?in?this?RFE?in?JDK?14: >> ??? > https://bugs.openjdk.java.net/browse/JDK-8227370 >> ??? > >> >?However,?we?still?enforce?the?same?set?of?consistency?checks?as?previous >> ??? >?JDK?versions. >> ??? > >> ??? >?Please?let?me?know?if?you?have?further?questions. >> ??? > >> ??? >?Thanks >> ??? >?-?Ioi >> ??? > >> ??? >?>?Thanks, >> ??? >?>?Wei?Xiang >> ??? > >> >> > From david.holmes at oracle.com Mon Mar 16 01:59:34 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 16 Mar 2020 11:59:34 +1000 Subject: RFR (trivial): 8241043: Expand assertions to identify thread with errant _stack_base Message-ID: webrev: http://cr.openjdk.java.net/~dholmes/8241043/webrev/ bug: https://bugs.openjdk.java.net/browse/JDK-8241043 This is a subtask of JDK-8239895 as I've been unable to reproduce the failures seen there. So this subtask augments a couple of assertions to try and assist with pinning down the actual bug. I think this counts as trivial. Testing: tiers 1-3 Thanks, David From mikael.vidstedt at oracle.com Mon Mar 16 05:22:45 2020 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Sun, 15 Mar 2020 22:22:45 -0700 Subject: RFR (trivial): 8241043: Expand assertions to identify thread with errant _stack_base In-Reply-To: References: Message-ID: <9E27DF95-37F1-490F-881F-7F00B2052BB6@oracle.com> Looks good. Cheers, Mikael > On Mar 15, 2020, at 6:59 PM, David Holmes wrote: > > webrev: http://cr.openjdk.java.net/~dholmes/8241043/webrev/ > bug: https://bugs.openjdk.java.net/browse/JDK-8241043 > > This is a subtask of JDK-8239895 as I've been unable to reproduce the failures seen there. So this subtask augments a couple of assertions to try and assist with pinning down the actual bug. > > I think this counts as trivial. > > Testing: tiers 1-3 > > Thanks, > David From david.holmes at oracle.com Mon Mar 16 06:14:48 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 16 Mar 2020 16:14:48 +1000 Subject: RFR (trivial): 8241043: Expand assertions to identify thread with errant _stack_base In-Reply-To: <9E27DF95-37F1-490F-881F-7F00B2052BB6@oracle.com> References: <9E27DF95-37F1-490F-881F-7F00B2052BB6@oracle.com> Message-ID: <1a8f4814-1526-99f1-3234-dcbffe4cef90@oracle.com> Thanks Mikael! Pushing now. David On 16/03/2020 3:22 pm, Mikael Vidstedt wrote: > > Looks good. > > Cheers, > Mikael > >> On Mar 15, 2020, at 6:59 PM, David Holmes wrote: >> >> webrev: http://cr.openjdk.java.net/~dholmes/8241043/webrev/ >> bug: https://bugs.openjdk.java.net/browse/JDK-8241043 >> >> This is a subtask of JDK-8239895 as I've been unable to reproduce the failures seen there. So this subtask augments a couple of assertions to try and assist with pinning down the actual bug. >> >> I think this counts as trivial. >> >> Testing: tiers 1-3 >> >> Thanks, >> David > From claes.redestad at oracle.com Mon Mar 16 15:40:43 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 16 Mar 2020 16:40:43 +0100 Subject: RFR: 8241042: x86_64: Improve Assembler generation Message-ID: <3563eb44-cb69-90ab-f4b9-9fba8e635836@oracle.com> Hi, on x64, folding consecutive bytes emitted by Assembler routines into byte-wise variants of emit_int16, emit_int24 and emit_32 turns out to be profitable, since gcc can't seem to fold the back to back stores to the end() pointer into one. The new methods are also a bit convenient. Together with a few other small fixes, this adds up to around a 0.4% reduction in instructions retired on Hello World. Bug: https://bugs.openjdk.java.net/browse/JDK-8241042 Webrev: http://cr.openjdk.java.net/~redestad/8241042/open.00/ Testing: tier1-5 (earlier version with no semantic differences), tier1-2 (current patch) Thanks! /Claes From vladimir.kozlov at oracle.com Mon Mar 16 17:25:34 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 16 Mar 2020 10:25:34 -0700 Subject: RFR: 8241042: x86_64: Improve Assembler generation In-Reply-To: <3563eb44-cb69-90ab-f4b9-9fba8e635836@oracle.com> References: <3563eb44-cb69-90ab-f4b9-9fba8e635836@oracle.com> Message-ID: <587c943c-c330-a2b6-4aa0-0f07f525aca1@oracle.com> On 3/16/20 8:40 AM, Claes Redestad wrote: > Hi, > > on x64, folding consecutive bytes emitted by Assembler routines into > byte-wise variants of emit_int16, emit_int24 and emit_32 turns out to > be profitable, since gcc can't seem to fold the back to back stores > to the end() pointer into one. The new methods are also a bit > convenient. Yes, I agree with this. Can you put () around arguments like next? It would be easy see them: emit_int24((0x44 | regenc), (scale << 6 | indexenc | baseenc), (disp & 0xFF)); Or better put them on separate lines as you did in some cases. Can we avoid cast (unsigned char) or should we do it always (when sign bit may be set)? I seems we added it to avoid additional instructions generated by gcc. > > Together with a few other small fixes, this adds up to around a 0.4% > reduction in instructions retired on Hello World. I wish you did these small fixes as separate RFE. Changes are already big. > > Bug:??? https://bugs.openjdk.java.net/browse/JDK-8241042 > Webrev: http://cr.openjdk.java.net/~redestad/8241042/open.00/ Thanks, Vladimir > > Testing: tier1-5 (earlier version with no semantic differences), tier1-2 > (current patch) > > Thanks! > > /Claes From claes.redestad at oracle.com Mon Mar 16 17:51:17 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 16 Mar 2020 18:51:17 +0100 Subject: RFR: 8241042: x86_64: Improve Assembler generation In-Reply-To: <587c943c-c330-a2b6-4aa0-0f07f525aca1@oracle.com> References: <3563eb44-cb69-90ab-f4b9-9fba8e635836@oracle.com> <587c943c-c330-a2b6-4aa0-0f07f525aca1@oracle.com> Message-ID: <6610007b-5fa4-4259-9245-3a21a0e0bc10@oracle.com> On 2020-03-16 18:25, Vladimir Kozlov wrote: > On 3/16/20 8:40 AM, Claes Redestad wrote: >> Hi, >> >> on x64, folding consecutive bytes emitted by Assembler routines into >> byte-wise variants of emit_int16, emit_int24 and emit_32 turns out to >> be profitable, since gcc can't seem to fold the back to back stores >> to the end() pointer into one. The new methods are also a bit >> convenient. > > Yes, I agree with this. > > Can you put () around arguments like next? It would be easy see them: > > emit_int24((0x44 | regenc), (scale << 6 | indexenc | baseenc), (disp & > 0xFF)); Sure. > > Or better put them on separate lines as you did in some cases. I prefer keeping the expressions more compact, unless there's a need to comment on what's going on. > > Can we avoid cast (unsigned char) or should we do it always (when sign > bit may be set)? I seems we added it to avoid additional instructions > generated by gcc. I did some testing and couldn't see any difference from removing them on linux, but the Windows compiler starts complaining if you pass a 0x80 or higher literal to a method taking an int8_t. I wonder if there's any reason to keep these signatures as int8_t and not go with uint8_t/u_char, and started playing with that. But yeah, this changeset is already getting a bit out of control so I dialed back a bit. > >> >> Together with a few other small fixes, this adds up to around a 0.4% >> reduction in instructions retired on Hello World. > > I wish you did these small fixes as separate RFE. Changes are already big. Ok. Splitting out the InstructionAttr changes to: https://bugs.openjdk.java.net/browse/JDK-8241077 Be right back... /Claes From claes.redestad at oracle.com Mon Mar 16 20:09:10 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 16 Mar 2020 21:09:10 +0100 Subject: RFR: 8241042: x86_64: Improve Assembler generation In-Reply-To: <6610007b-5fa4-4259-9245-3a21a0e0bc10@oracle.com> References: <3563eb44-cb69-90ab-f4b9-9fba8e635836@oracle.com> <587c943c-c330-a2b6-4aa0-0f07f525aca1@oracle.com> <6610007b-5fa4-4259-9245-3a21a0e0bc10@oracle.com> Message-ID: <36b8d04b-9256-9a0a-0007-8a8894479ef7@oracle.com> On 2020-03-16 18:51, Claes Redestad wrote: >>> >>> on x64, folding consecutive bytes emitted by Assembler routines into >>> byte-wise variants of emit_int16, emit_int24 and emit_32 turns out to >>> be profitable, since gcc can't seem to fold the back to back stores >>> to the end() pointer into one. The new methods are also a bit >>> convenient. >> >> Yes, I agree with this. >> >> Can you put () around arguments like next? It would be easy see them: >> >> emit_int24((0x44 | regenc), (scale << 6 | indexenc | baseenc), (disp & >> 0xFF)); > > Sure. > >> >> Or better put them on separate lines as you did in some cases. > > I prefer keeping the expressions more compact, unless there's a need to > comment on what's going on. > >> >> Can we avoid cast (unsigned char) or should we do it always (when sign >> bit may be set)? I seems we added it to avoid additional instructions >> generated by gcc. I've made a number of edits that should help with readability in places where it got a bit muddied, especially in emit_operand: http://cr.openjdk.java.net/~redestad/8241042/open.01/ Also made use of (unsigned char) a bit more consistent. I think a follow up to examine if we can get rid of these casts by changing arguments to uint8_t might be worthwhile. Rerunning tier1+2. /Claes From claes.redestad at oracle.com Mon Mar 16 20:28:01 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 16 Mar 2020 21:28:01 +0100 Subject: RFR: 8241077: x86_64: Minor Assembler improvements Message-ID: Hi, a few small improvements extracted from 8241042 to keep that RFE more focused: - make InstructionAttr more compact by reordering some bool and int fields leads to slightly better (and more compact) code generation. - _is_managed only used on 32-bit - code gen for *_needs_rex improve if replacing reg != (x)noreg with reg->is_valid(), since the expression compiles down into a range checks (fewer branches) Bug: https://bugs.openjdk.java.net/browse/JDK-8241077 Webrev: http://cr.openjdk.java.net/~redestad/8241077/open.00/ Testing: tier1-5 (together with JDK-8241042) /Claes From vladimir.kozlov at oracle.com Mon Mar 16 21:09:24 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 16 Mar 2020 14:09:24 -0700 Subject: RFR: 8241077: x86_64: Minor Assembler improvements In-Reply-To: References: Message-ID: Good. I think it is trivial. Thanks, Vladimir On 3/16/20 1:28 PM, Claes Redestad wrote: > Hi, > > a few small improvements extracted from 8241042 to keep that RFE > more focused: > > - make InstructionAttr more compact by reordering some bool and int > fields leads to slightly better (and more compact) code generation. > > - _is_managed only used on 32-bit > > - code gen for *_needs_rex improve if replacing reg != (x)noreg > ? with reg->is_valid(), since the expression compiles down into a > ? range checks (fewer branches) > > Bug:??? https://bugs.openjdk.java.net/browse/JDK-8241077 > Webrev: http://cr.openjdk.java.net/~redestad/8241077/open.00/ > > Testing: tier1-5 (together with JDK-8241042) > > /Claes From claes.redestad at oracle.com Mon Mar 16 21:25:55 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 16 Mar 2020 22:25:55 +0100 Subject: RFR: 8241077: x86_64: Minor Assembler improvements In-Reply-To: References: Message-ID: On 2020-03-16 22:09, Vladimir Kozlov wrote: > Good. I think it is trivial. Thanks! /Claes From vladimir.kozlov at oracle.com Mon Mar 16 22:29:41 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 16 Mar 2020 15:29:41 -0700 Subject: RFR: 8241042: x86_64: Improve Assembler generation In-Reply-To: <36b8d04b-9256-9a0a-0007-8a8894479ef7@oracle.com> References: <3563eb44-cb69-90ab-f4b9-9fba8e635836@oracle.com> <587c943c-c330-a2b6-4aa0-0f07f525aca1@oracle.com> <6610007b-5fa4-4259-9245-3a21a0e0bc10@oracle.com> <36b8d04b-9256-9a0a-0007-8a8894479ef7@oracle.com> Message-ID: <4e5037fd-718e-d955-ba62-55f290444ef7@oracle.com> Okay, emit_int* code change seems fine (I did not verify each and every line - hope testing will show correctness). About get_prefixq() change. 1. Would be nice to have common prefixes table and method get_prefixq(). Could be done by passing noreg (-1): void Assembler::prefixq(Address adr) { emit_int8(get_prefixq(adr, noreg)); } 2. Use asserts for them as you did in first get_prefixq() variant. 3. May swap values in table by reversion (src->encoding() < 8) condition. Thanks, Vladimir On 3/16/20 1:09 PM, Claes Redestad wrote: > > > On 2020-03-16 18:51, Claes Redestad wrote: >>>> >>>> on x64, folding consecutive bytes emitted by Assembler routines into >>>> byte-wise variants of emit_int16, emit_int24 and emit_32 turns out to >>>> be profitable, since gcc can't seem to fold the back to back stores >>>> to the end() pointer into one. The new methods are also a bit >>>> convenient. >>> >>> Yes, I agree with this. >>> >>> Can you put () around arguments like next? It would be easy see them: >>> >>> emit_int24((0x44 | regenc), (scale << 6 | indexenc | baseenc), (disp & 0xFF)); >> >> Sure. >> >>> >>> Or better put them on separate lines as you did in some cases. >> >> I prefer keeping the expressions more compact, unless there's a need to >> comment on what's going on. >> >>> >>> Can we avoid cast (unsigned char) or should we do it always (when sign bit may be set)? I seems we added it to avoid >>> additional instructions generated by gcc. > > I've made a number of edits that should help with readability in places > where it got a bit muddied, especially in emit_operand: > > http://cr.openjdk.java.net/~redestad/8241042/open.01/ > > Also made use of (unsigned char) a bit more consistent. > > I think a follow up to examine if we can get rid of these casts by > changing arguments to uint8_t might be worthwhile. > > Rerunning tier1+2. > > /Claes From claes.redestad at oracle.com Mon Mar 16 22:35:15 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 16 Mar 2020 23:35:15 +0100 Subject: [15] RFR(S) 8237045: JVM uses excessive memory with -XX:+EnableJVMCI -XX:JVMCICounterSize=2147483648 In-Reply-To: <35735d94-7ccf-158e-a2be-a2b4c968afd9@oracle.com> References: <35735d94-7ccf-158e-a2be-a2b4c968afd9@oracle.com> Message-ID: <39608c76-f723-dbd2-c77e-b65256e5aa3b@oracle.com> Hi, + case "MethodProfileWidth": + option.addPrepend("-XX:+EnableJVMCI"); + break; given that you're removing MethodProfileWidth in JDK-8240976 I'm a bit surprised to see this here. Otherwise looks good to me. /Claes On 2020-03-13 03:08, Vladimir Kozlov wrote: > https://bugs.openjdk.java.net/browse/JDK-8237045 > http://cr.openjdk.java.net/~kvn/8237045/webrev.00/ > > After discussion JVMCICounterSize flag's value range is limited to 1M. > I also added other missing ranges for JVMCI flags. > > Fixed TestOptionsWithRanges.java test to add -XX:+EnableJVMCI for JVMCI > flags which requires it. > And do not test max value for JVMCI threads counts values as we do for > other threads counts flags. > > Tested hs-tier1-3. > > Thanks, > Vladimir From david.holmes at oracle.com Mon Mar 16 23:12:10 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 17 Mar 2020 09:12:10 +1000 Subject: RFR (S): 8239593: Bogus warning "Re-registering of platform native method" for a JVMTI agent In-Reply-To: References: <7fc24e19-736c-423e-c8e5-bcf945c1b12a@oracle.com> Message-ID: Can I get a second review please. Thanks, David On 13/03/2020 5:23 pm, David Holmes wrote: > Thanks for the Review Dan! I'll fix up the two nits. > > David > > On 13/03/2020 9:38 am, Daniel D. Daugherty wrote: >> Hi David, >> >> I started and stopped a couple of times due to other distractions. >> Sorry about that... >> >> >> On 3/10/20 10:08 PM, David Holmes wrote: >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8239593 >>> webrev: http://cr.openjdk.java.net/~dholmes/8239593/webrev/ >> >> src/hotspot/share/prims/jni.cpp >> ???? L2818:?? .... That will catch changes >> ???? L2819: ? // to platform classes while excluding classes added to >> the bootclasspath. >> ???????? I'm having trouble with the "excluding classes added to the >> ???????? bootclasspath" part. Perhaps this: >> >> ???? ? ? ? ? .... That will catch changes >> ???? ? ? ? ? // to platform classes while permitting changes to >> classes added to the bootclasspath. >> >> ???????? Maybe it's just me, but "excluding classes" sounds like those >> ???????? classes won't be allowed or something. >> >> ???? L2828: ??? if ((cl ==? NULL || >> SystemDictionary::is_platform_class_loader(cl)) && >> ???????? nit - not yours, but there's an extra space after '=='. >> >> ???? No comments on the code itself, although I did have to read it a >> ???? couple of times to convince myself the cases were right/caught. >> >> Thumbs up. Your call on changing the comments or fixing the nit. >> I don't need to see another webrev. >> >> Thanks for mentioning the testing done. >> >> Dan >> >> >>> >>> The original intent was to warn if changes were being made to >>> platform classes by non-platform classes. That was weakened slightly >>> to warn if classes loaded by the boot/platform loaders were modified >>> by a class not loaded by the boot/platform loader. But that >>> formulation also catches non-platform classes that have been added to >>> the bootclasspath. So we augment the check to also check that the >>> target class is in a named module** (loaded by the boot/platform >>> loader) - this ensures we catch changes to platform files whilst >>> excluding classes incidentally on the bootclasspath. >>> >>> ** Note that you cannot add a named module to the boot/platform >>> loaders at runtime - they are defined at build time. >>> >>> Tested against Yourkit reproducer in the bug report, and with >>> existing registerNatives test. >>> >>> Test builds on tier1 platforms. >>> >>> Thanks, >>> David >> From vladimir.kozlov at oracle.com Mon Mar 16 23:24:04 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 16 Mar 2020 16:24:04 -0700 Subject: [15] RFR(S) 8237045: JVM uses excessive memory with -XX:+EnableJVMCI -XX:JVMCICounterSize=2147483648 In-Reply-To: <39608c76-f723-dbd2-c77e-b65256e5aa3b@oracle.com> References: <35735d94-7ccf-158e-a2be-a2b4c968afd9@oracle.com> <39608c76-f723-dbd2-c77e-b65256e5aa3b@oracle.com> Message-ID: Thank you, Claes I finished these changes before I start working on 8240976. Depending which one get 2 reviews first I will adjust changes. Thanks, Vladimir On 3/16/20 3:35 PM, Claes Redestad wrote: > Hi, > > +???????????? case "MethodProfileWidth": > +???????????????? option.addPrepend("-XX:+EnableJVMCI"); > +???????????????? break; > > given that you're removing MethodProfileWidth in JDK-8240976 I'm a bit > surprised to see this here. > > Otherwise looks good to me. > > /Claes > > On 2020-03-13 03:08, Vladimir Kozlov wrote: >> https://bugs.openjdk.java.net/browse/JDK-8237045 >> http://cr.openjdk.java.net/~kvn/8237045/webrev.00/ >> >> After discussion JVMCICounterSize flag's value range is limited to 1M. >> I also added other missing ranges for JVMCI flags. >> >> Fixed TestOptionsWithRanges.java test to add -XX:+EnableJVMCI for JVMCI flags which requires it. >> And do not test max value for JVMCI threads counts values as we do for other threads counts flags. >> >> Tested hs-tier1-3. >> >> Thanks, >> Vladimir From claes.redestad at oracle.com Mon Mar 16 23:43:22 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 17 Mar 2020 00:43:22 +0100 Subject: RFR: 8241042: x86_64: Improve Assembler generation In-Reply-To: <4e5037fd-718e-d955-ba62-55f290444ef7@oracle.com> References: <3563eb44-cb69-90ab-f4b9-9fba8e635836@oracle.com> <587c943c-c330-a2b6-4aa0-0f07f525aca1@oracle.com> <6610007b-5fa4-4259-9245-3a21a0e0bc10@oracle.com> <36b8d04b-9256-9a0a-0007-8a8894479ef7@oracle.com> <4e5037fd-718e-d955-ba62-55f290444ef7@oracle.com> Message-ID: On 2020-03-16 23:29, Vladimir Kozlov wrote: > Okay, emit_int* code change seems fine (I did not verify each and every > line - hope testing will show correctness). Thanks! > > About get_prefixq() change. > > 1. Would be nice to have common prefixes table and method get_prefixq(). > Could be done by passing noreg (-1) > > void Assembler::prefixq(Address adr) { > ? emit_int8(get_prefixq(adr, noreg)); > } Hmm, get_prefixq(adr, reg) calls reg->encoding(), which will assert on noreg (assert(reg->is_valid(), ...)). > > 2. Use asserts for them as you did in first get_prefixq() variant. This is a good idea, in fact I thought I already had the corresponding assert block in both places. > > 3. May swap values in table by reversion (src->encoding() < 8) condition. Hmm, maybe we should just exploit the fact that the REX_W* values returned in both cases are consecutive from REX_W to REX_WXB and REX_WRXB respectively, and ordered in a way that makes it possible to get rid of the lookup tables entirely: int8_t Assembler::get_prefixq(Address adr) { int8_t prfx = (int8_t)(REX_W + ((int)adr.base_needs_rex()) + ((int)adr.index_needs_rex() << 1)); .. int8_t Assembler::get_prefixq(Address adr, Register src) { int8_t prfx = (int8_t)(REX_W + ((int)adr.base_needs_rex()) + ((int)adr.index_needs_rex() << 1) + ((int)(src->encoding() >= 8) << 2)); .. http://cr.openjdk.java.net/~redestad/8241042/open.02/ While this is really a bit too clever, the assertions make for a very strong fail-safe, and I think all the possibilities should be checked rather promply by our regular testing. I've sanity tested locally and verified the code gen look a bit better without the lookup tables. Running tier1-4 over night. Thanks! /Claes From vladimir.kozlov at oracle.com Mon Mar 16 23:56:01 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 16 Mar 2020 16:56:01 -0700 Subject: RFR: 8241042: x86_64: Improve Assembler generation In-Reply-To: References: <3563eb44-cb69-90ab-f4b9-9fba8e635836@oracle.com> <587c943c-c330-a2b6-4aa0-0f07f525aca1@oracle.com> <6610007b-5fa4-4259-9245-3a21a0e0bc10@oracle.com> <36b8d04b-9256-9a0a-0007-8a8894479ef7@oracle.com> <4e5037fd-718e-d955-ba62-55f290444ef7@oracle.com> Message-ID: Something messed up. Move prefixq(Address adr, Register src) near corresponding get_prefixq(). Currently an other method Assembler::prefix(Address adr, Register reg, bool byteinst) mixed up in changes. Which brings question: why you did not do the same for prefix() methods? May be you can move get_prefixq() changes into separate RFE so we can continue discussion? Changes emit_int* can be pushed first. Thanks, Vladimir On 3/16/20 4:43 PM, Claes Redestad wrote: > > > On 2020-03-16 23:29, Vladimir Kozlov wrote: >> Okay, emit_int* code change seems fine (I did not verify each and every line - hope testing will show correctness). > > Thanks! > >> >> About get_prefixq() change. >> >> 1. Would be nice to have common prefixes table and method get_prefixq(). Could be done by passing noreg (-1) > >> void Assembler::prefixq(Address adr) { >> ?? emit_int8(get_prefixq(adr, noreg)); >> } > > Hmm, get_prefixq(adr, reg) calls reg->encoding(), which will assert on > noreg (assert(reg->is_valid(), ...)). > >> >> 2. Use asserts for them as you did in first get_prefixq() variant. > > This is a good idea, in fact I thought I already had the > corresponding assert block in both places. > >> >> 3. May swap values in table by reversion (src->encoding() < 8) condition. > > Hmm, maybe we should just exploit the fact that the REX_W* values > returned in both cases are consecutive from REX_W to REX_WXB and > REX_WRXB respectively, and ordered in a way that makes it possible to > get rid of the lookup tables entirely: > > int8_t Assembler::get_prefixq(Address adr) { > ? int8_t prfx = (int8_t)(REX_W + ((int)adr.base_needs_rex()) + ((int)adr.index_needs_rex() << 1)); > ? .. > > int8_t Assembler::get_prefixq(Address adr, Register src) { > ? int8_t prfx = (int8_t)(REX_W + ((int)adr.base_needs_rex()) + ((int)adr.index_needs_rex() << 1) + > ((int)(src->encoding() >= 8) << 2)); > ? .. > > http://cr.openjdk.java.net/~redestad/8241042/open.02/ > > While this is really a bit too clever, the assertions make for a very > strong fail-safe, and I think all the possibilities should be checked > rather promply by our regular testing. I've sanity tested locally and > verified the code gen look a bit better without the lookup tables. > > Running tier1-4 over night. > > Thanks! > > /Claes From martinrb at google.com Tue Mar 17 03:46:23 2020 From: martinrb at google.com (Martin Buchholz) Date: Mon, 16 Mar 2020 20:46:23 -0700 Subject: RFR: 8186780: clang fastdebug assertion failure in os_linux_x86:os::verify_stack_alignment() Message-ID: We finally came up with a solution for this that more than one engineer can agree on: https://cr.openjdk.java.net/~martin/webrevs/jdk/clang-stack-alignment-builtin-frame-address/ https://bugs.openjdk.java.net/browse/JDK-8186780 From tobias.hartmann at oracle.com Tue Mar 17 07:40:38 2020 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Tue, 17 Mar 2020 08:40:38 +0100 Subject: [15] RFR(S) 8237045: JVM uses excessive memory with -XX:+EnableJVMCI -XX:JVMCICounterSize=2147483648 In-Reply-To: <35735d94-7ccf-158e-a2be-a2b4c968afd9@oracle.com> References: <35735d94-7ccf-158e-a2be-a2b4c968afd9@oracle.com> Message-ID: <8db92ae4-f396-2a69-1ee0-3b3e54fb41d1@oracle.com> Hi Vladimir, On 13.03.20 03:08, Vladimir Kozlov wrote: > http://cr.openjdk.java.net/~kvn/8237045/webrev.00/ Looks good to me. Best regards, Tobias From Alan.Bateman at oracle.com Tue Mar 17 08:35:36 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 17 Mar 2020 08:35:36 +0000 Subject: RFR (S): 8239593: Bogus warning "Re-registering of platform native method" for a JVMTI agent In-Reply-To: References: <7fc24e19-736c-423e-c8e5-bcf945c1b12a@oracle.com> Message-ID: <52a139f0-72aa-d9c9-9b1e-3ce2e4312cde@oracle.com> On 16/03/2020 23:12, David Holmes wrote: > Can I get a second review please. The approach and changes look okay to me. I see in your reply to Dan that you tweaking the comment. One additional nit is is the existing "But we issue ..." is an unusual sentence as the reason for emitting the warning isn't explained in the sentence. Might be simplest to just drop the "But". -Alan From david.holmes at oracle.com Tue Mar 17 08:48:08 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 17 Mar 2020 18:48:08 +1000 Subject: RFR (S): 8239593: Bogus warning "Re-registering of platform native method" for a JVMTI agent In-Reply-To: <52a139f0-72aa-d9c9-9b1e-3ce2e4312cde@oracle.com> References: <7fc24e19-736c-423e-c8e5-bcf945c1b12a@oracle.com> <52a139f0-72aa-d9c9-9b1e-3ce2e4312cde@oracle.com> Message-ID: On 17/03/2020 6:35 pm, Alan Bateman wrote: > On 16/03/2020 23:12, David Holmes wrote: >> Can I get a second review please. > The approach and changes look okay to me. I see in your reply to Dan > that you tweaking the comment. One additional nit is is the existing > "But we issue ..." is an unusual sentence as the reason for emitting the > warning isn't explained in the sentence. Might be simplest to just drop > the "But". Thanks Alan. The "But ..." serves as a special condition in relation to the "There are no restrictions ...". The grammar is probably not the best there. This: " ... native methods. But we issue ..." should probably be: " ...native methods, however we issue ..." Thanks, David > -Alan From Alan.Bateman at oracle.com Tue Mar 17 08:57:06 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 17 Mar 2020 08:57:06 +0000 Subject: RFR (S): 8239593: Bogus warning "Re-registering of platform native method" for a JVMTI agent In-Reply-To: References: <7fc24e19-736c-423e-c8e5-bcf945c1b12a@oracle.com> <52a139f0-72aa-d9c9-9b1e-3ce2e4312cde@oracle.com> Message-ID: <513806c0-931c-62fc-22b7-87bc34b0d054@oracle.com> On 17/03/2020 08:48, David Holmes wrote: > On 17/03/2020 6:35 pm, Alan Bateman wrote: >> On 16/03/2020 23:12, David Holmes wrote: >>> Can I get a second review please. >> The approach and changes look okay to me. I see in your reply to Dan >> that you tweaking the comment. One additional nit is is the existing >> "But we issue ..." is an unusual sentence as the reason for emitting >> the warning isn't explained in the sentence. Might be simplest to >> just drop the "But". > > Thanks Alan. > > The "But ..." serves as a special condition in relation to the "There > are no restrictions ...". The grammar is probably not the best there. > This: > > " ... native methods. But we issue ..." > > should probably be: > > " ...native methods, however we issue ..." That works for me. -Alan. From claes.redestad at oracle.com Tue Mar 17 10:55:08 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 17 Mar 2020 11:55:08 +0100 Subject: RFR: 8241042: x86_64: Improve Assembler generation In-Reply-To: References: <3563eb44-cb69-90ab-f4b9-9fba8e635836@oracle.com> <587c943c-c330-a2b6-4aa0-0f07f525aca1@oracle.com> <6610007b-5fa4-4259-9245-3a21a0e0bc10@oracle.com> <36b8d04b-9256-9a0a-0007-8a8894479ef7@oracle.com> <4e5037fd-718e-d955-ba62-55f290444ef7@oracle.com> Message-ID: On 2020-03-17 00:56, Vladimir Kozlov wrote: > Something messed up. Move prefixq(Address adr, Register src) near > corresponding get_prefixq(). > Currently an other method Assembler::prefix(Address adr, Register reg, > bool byteinst) mixed up in changes. > Which brings question: why you did not do the same for prefix() methods? > > May be you can move get_prefixq() changes into separate RFE so we can > continue discussion? Ok, let's tease these apart. > > Changes emit_int* can be pushed first. Here's a version with only the emit_intxx changes: http://cr.openjdk.java.net/~redestad/8241042/open.03/ I'll file an RFE and keep polishing the prefixq improvements there. /Claes From harold.seigel at oracle.com Tue Mar 17 12:02:09 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Tue, 17 Mar 2020 08:02:09 -0400 Subject: RFR (S): 8239593: Bogus warning "Re-registering of platform native method" for a JVMTI agent In-Reply-To: References: <7fc24e19-736c-423e-c8e5-bcf945c1b12a@oracle.com> Message-ID: <529a715d-9e32-617f-10c6-c7207270a2d0@oracle.com> Looks good. One nit is that the assignment to 'caller' could be moved inside the 'if' statement starting at line 2828. Thanks, Harold On 3/16/2020 7:12 PM, David Holmes wrote: > Can I get a second review please. > > Thanks, > David > > On 13/03/2020 5:23 pm, David Holmes wrote: >> Thanks for the Review Dan! I'll fix up the two nits. >> >> David >> >> On 13/03/2020 9:38 am, Daniel D. Daugherty wrote: >>> Hi David, >>> >>> I started and stopped a couple of times due to other distractions. >>> Sorry about that... >>> >>> >>> On 3/10/20 10:08 PM, David Holmes wrote: >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8239593 >>>> webrev: http://cr.openjdk.java.net/~dholmes/8239593/webrev/ >>> >>> src/hotspot/share/prims/jni.cpp >>> ???? L2818:?? .... That will catch changes >>> ???? L2819: ? // to platform classes while excluding classes added >>> to the bootclasspath. >>> ???????? I'm having trouble with the "excluding classes added to the >>> ???????? bootclasspath" part. Perhaps this: >>> >>> ???? ? ? ? ? .... That will catch changes >>> ???? ? ? ? ? // to platform classes while permitting changes to >>> classes added to the bootclasspath. >>> >>> ???????? Maybe it's just me, but "excluding classes" sounds like those >>> ???????? classes won't be allowed or something. >>> >>> ???? L2828: ??? if ((cl ==? NULL || >>> SystemDictionary::is_platform_class_loader(cl)) && >>> ???????? nit - not yours, but there's an extra space after '=='. >>> >>> ???? No comments on the code itself, although I did have to read it a >>> ???? couple of times to convince myself the cases were right/caught. >>> >>> Thumbs up. Your call on changing the comments or fixing the nit. >>> I don't need to see another webrev. >>> >>> Thanks for mentioning the testing done. >>> >>> Dan >>> >>> >>>> >>>> The original intent was to warn if changes were being made to >>>> platform classes by non-platform classes. That was weakened >>>> slightly to warn if classes loaded by the boot/platform loaders >>>> were modified by a class not loaded by the boot/platform loader. >>>> But that formulation also catches non-platform classes that have >>>> been added to the bootclasspath. So we augment the check to also >>>> check that the target class is in a named module** (loaded by the >>>> boot/platform loader) - this ensures we catch changes to platform >>>> files whilst excluding classes incidentally on the bootclasspath. >>>> >>>> ** Note that you cannot add a named module to the boot/platform >>>> loaders at runtime - they are defined at build time. >>>> >>>> Tested against Yourkit reproducer in the bug report, and with >>>> existing registerNatives test. >>>> >>>> Test builds on tier1 platforms. >>>> >>>> Thanks, >>>> David >>> From david.holmes at oracle.com Tue Mar 17 12:18:04 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 17 Mar 2020 22:18:04 +1000 Subject: RFR (S): 8239593: Bogus warning "Re-registering of platform native method" for a JVMTI agent In-Reply-To: <529a715d-9e32-617f-10c6-c7207270a2d0@oracle.com> References: <7fc24e19-736c-423e-c8e5-bcf945c1b12a@oracle.com> <529a715d-9e32-617f-10c6-c7207270a2d0@oracle.com> Message-ID: Hi Harold, On 17/03/2020 10:02 pm, Harold Seigel wrote: > Looks good. Thanks for the review. > One nit is that the assignment to 'caller' could be moved inside the > 'if' statement starting at line 2828. Yes - good point - no need to do that unnecessarily! Will update and push. Thanks, David > Thanks, Harold > > On 3/16/2020 7:12 PM, David Holmes wrote: >> Can I get a second review please. >> >> Thanks, >> David >> >> On 13/03/2020 5:23 pm, David Holmes wrote: >>> Thanks for the Review Dan! I'll fix up the two nits. >>> >>> David >>> >>> On 13/03/2020 9:38 am, Daniel D. Daugherty wrote: >>>> Hi David, >>>> >>>> I started and stopped a couple of times due to other distractions. >>>> Sorry about that... >>>> >>>> >>>> On 3/10/20 10:08 PM, David Holmes wrote: >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8239593 >>>>> webrev: http://cr.openjdk.java.net/~dholmes/8239593/webrev/ >>>> >>>> src/hotspot/share/prims/jni.cpp >>>> ???? L2818:?? .... That will catch changes >>>> ???? L2819: ? // to platform classes while excluding classes added >>>> to the bootclasspath. >>>> ???????? I'm having trouble with the "excluding classes added to the >>>> ???????? bootclasspath" part. Perhaps this: >>>> >>>> ???? ? ? ? ? .... That will catch changes >>>> ???? ? ? ? ? // to platform classes while permitting changes to >>>> classes added to the bootclasspath. >>>> >>>> ???????? Maybe it's just me, but "excluding classes" sounds like those >>>> ???????? classes won't be allowed or something. >>>> >>>> ???? L2828: ??? if ((cl ==? NULL || >>>> SystemDictionary::is_platform_class_loader(cl)) && >>>> ???????? nit - not yours, but there's an extra space after '=='. >>>> >>>> ???? No comments on the code itself, although I did have to read it a >>>> ???? couple of times to convince myself the cases were right/caught. >>>> >>>> Thumbs up. Your call on changing the comments or fixing the nit. >>>> I don't need to see another webrev. >>>> >>>> Thanks for mentioning the testing done. >>>> >>>> Dan >>>> >>>> >>>>> >>>>> The original intent was to warn if changes were being made to >>>>> platform classes by non-platform classes. That was weakened >>>>> slightly to warn if classes loaded by the boot/platform loaders >>>>> were modified by a class not loaded by the boot/platform loader. >>>>> But that formulation also catches non-platform classes that have >>>>> been added to the bootclasspath. So we augment the check to also >>>>> check that the target class is in a named module** (loaded by the >>>>> boot/platform loader) - this ensures we catch changes to platform >>>>> files whilst excluding classes incidentally on the bootclasspath. >>>>> >>>>> ** Note that you cannot add a named module to the boot/platform >>>>> loaders at runtime - they are defined at build time. >>>>> >>>>> Tested against Yourkit reproducer in the bug report, and with >>>>> existing registerNatives test. >>>>> >>>>> Test builds on tier1 platforms. >>>>> >>>>> Thanks, >>>>> David >>>> From vladimir.kozlov at oracle.com Tue Mar 17 16:09:08 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 17 Mar 2020 09:09:08 -0700 Subject: [15] RFR(S) 8237045: JVM uses excessive memory with -XX:+EnableJVMCI -XX:JVMCICounterSize=2147483648 In-Reply-To: <8db92ae4-f396-2a69-1ee0-3b3e54fb41d1@oracle.com> References: <35735d94-7ccf-158e-a2be-a2b4c968afd9@oracle.com> <8db92ae4-f396-2a69-1ee0-3b3e54fb41d1@oracle.com> Message-ID: <115a88db-3046-df08-d3ce-7ed4f49ae092@oracle.com> Thank you, Tobias Vladimir K On 3/17/20 12:40 AM, Tobias Hartmann wrote: > Hi Vladimir, > > On 13.03.20 03:08, Vladimir Kozlov wrote: >> http://cr.openjdk.java.net/~kvn/8237045/webrev.00/ > > Looks good to me. > > Best regards, > Tobias > From vladimir.x.ivanov at oracle.com Tue Mar 17 16:36:39 2020 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 17 Mar 2020 19:36:39 +0300 Subject: RFR: 8241042: x86_64: Improve Assembler generation In-Reply-To: References: <3563eb44-cb69-90ab-f4b9-9fba8e635836@oracle.com> <587c943c-c330-a2b6-4aa0-0f07f525aca1@oracle.com> <6610007b-5fa4-4259-9245-3a21a0e0bc10@oracle.com> <36b8d04b-9256-9a0a-0007-8a8894479ef7@oracle.com> <4e5037fd-718e-d955-ba62-55f290444ef7@oracle.com> Message-ID: > http://cr.openjdk.java.net/~redestad/8241042/open.03/ Looks good. Best regards, Vladimir Ivanov From jianglizhou at google.com Tue Mar 17 16:41:15 2020 From: jianglizhou at google.com (Jiangli Zhou) Date: Tue, 17 Mar 2020 09:41:15 -0700 Subject: RFR: 8186780: clang fastdebug assertion failure in os_linux_x86:os::verify_stack_alignment() In-Reply-To: References: Message-ID: Yes, the solution looks good to me. (Duplicating one of the comment from JDK-8186780 here)The Clang generated code for a fastdebug build looks like this when __builtin_frame_address() is used: (gdb) x/i $pc-4 0x7ffff748d820 : push %rbp (gdb) 0x7ffff748d821 : mov %rsp,%rbp (gdb) => 0x7ffff748d824 : test $0xf,%bpl (gdb) %bpl is the lower 8-bit of %rbp. With the __builtin_frame_address() call, it forces the compiler to emit 'push %rbp; mov %rsp, %rbp', which does the necessary stack adjustment before the test. Earlier I had a concern about using __builtin_frame_address(0) for gcc case. According to https://github.com/google/glog/blob/master/src/stacktrace_x86-inl.h#L113, __builtin_frame_address(0) can return the wrong address on gcc-4.1.0-k8. However, gcc-4.1.0 is very old (>=13 year old). Martin also pointed out that __builtin_frame_address(0) is already being used unconditionally elsewhere in hotspot, for example in os_linux_zero.cpp. So the change seems to be safe. Best regards, Jiangli On Mon, Mar 16, 2020 at 8:46 PM Martin Buchholz wrote: > > We finally came up with a solution for this that more than one engineer can agree on: > > https://cr.openjdk.java.net/~martin/webrevs/jdk/clang-stack-alignment-builtin-frame-address/ > https://bugs.openjdk.java.net/browse/JDK-8186780 From vladimir.kozlov at oracle.com Tue Mar 17 16:54:28 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 17 Mar 2020 09:54:28 -0700 Subject: RFR: 8241042: x86_64: Improve Assembler generation In-Reply-To: References: <3563eb44-cb69-90ab-f4b9-9fba8e635836@oracle.com> <587c943c-c330-a2b6-4aa0-0f07f525aca1@oracle.com> <6610007b-5fa4-4259-9245-3a21a0e0bc10@oracle.com> <36b8d04b-9256-9a0a-0007-8a8894479ef7@oracle.com> <4e5037fd-718e-d955-ba62-55f290444ef7@oracle.com> Message-ID: <1dcc6cf2-fbb7-6232-dd14-532cd5777a47@oracle.com> +1 Thanks, Vladimir K On 3/17/20 9:36 AM, Vladimir Ivanov wrote: > >> http://cr.openjdk.java.net/~redestad/8241042/open.03/ > > Looks good. > > Best regards, > Vladimir Ivanov From claes.redestad at oracle.com Tue Mar 17 17:45:55 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 17 Mar 2020 18:45:55 +0100 Subject: RFR: 8241042: x86_64: Improve Assembler generation In-Reply-To: <1dcc6cf2-fbb7-6232-dd14-532cd5777a47@oracle.com> References: <3563eb44-cb69-90ab-f4b9-9fba8e635836@oracle.com> <587c943c-c330-a2b6-4aa0-0f07f525aca1@oracle.com> <6610007b-5fa4-4259-9245-3a21a0e0bc10@oracle.com> <36b8d04b-9256-9a0a-0007-8a8894479ef7@oracle.com> <4e5037fd-718e-d955-ba62-55f290444ef7@oracle.com> <1dcc6cf2-fbb7-6232-dd14-532cd5777a47@oracle.com> Message-ID: Thanks for the reviews! /Claes On 2020-03-17 17:54, Vladimir Kozlov wrote: > +1 > > Thanks, > Vladimir K > > On 3/17/20 9:36 AM, Vladimir Ivanov wrote: >> >>> http://cr.openjdk.java.net/~redestad/8241042/open.03/ >> >> Looks good. >> >> Best regards, >> Vladimir Ivanov From coleen.phillimore at oracle.com Tue Mar 17 18:46:46 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 17 Mar 2020 14:46:46 -0400 Subject: RFR: 8241042: x86_64: Improve Assembler generation In-Reply-To: References: <3563eb44-cb69-90ab-f4b9-9fba8e635836@oracle.com> <587c943c-c330-a2b6-4aa0-0f07f525aca1@oracle.com> <6610007b-5fa4-4259-9245-3a21a0e0bc10@oracle.com> <36b8d04b-9256-9a0a-0007-8a8894479ef7@oracle.com> <4e5037fd-718e-d955-ba62-55f290444ef7@oracle.com> <1dcc6cf2-fbb7-6232-dd14-532cd5777a47@oracle.com> Message-ID: <03a22667-f032-cbed-a8ea-8d39320f26bb@oracle.com> Hi Claes, I like this 03 version than the first one.? emit_int16(int8, int8) makes more sense than emit_int8(int8, int8). thanks, Coleen On 3/17/20 1:45 PM, Claes Redestad wrote: > Thanks for the reviews! > > /Claes > > On 2020-03-17 17:54, Vladimir Kozlov wrote: >> +1 >> >> Thanks, >> Vladimir K >> >> On 3/17/20 9:36 AM, Vladimir Ivanov wrote: >>> >>>> http://cr.openjdk.java.net/~redestad/8241042/open.03/ >>> >>> Looks good. >>> >>> Best regards, >>> Vladimir Ivanov From claes.redestad at oracle.com Tue Mar 17 18:55:23 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 17 Mar 2020 19:55:23 +0100 Subject: RFR: 8241042: x86_64: Improve Assembler generation In-Reply-To: <03a22667-f032-cbed-a8ea-8d39320f26bb@oracle.com> References: <3563eb44-cb69-90ab-f4b9-9fba8e635836@oracle.com> <587c943c-c330-a2b6-4aa0-0f07f525aca1@oracle.com> <6610007b-5fa4-4259-9245-3a21a0e0bc10@oracle.com> <36b8d04b-9256-9a0a-0007-8a8894479ef7@oracle.com> <4e5037fd-718e-d955-ba62-55f290444ef7@oracle.com> <1dcc6cf2-fbb7-6232-dd14-532cd5777a47@oracle.com> <03a22667-f032-cbed-a8ea-8d39320f26bb@oracle.com> Message-ID: On 2020-03-17 19:46, coleen.phillimore at oracle.com wrote: > Hi Claes, > I like this 03 version than the first one.? emit_int16(int8, int8) makes > more sense than emit_int8(int8, int8). Glad you like it, because I pushed it a few minutes ago. :-) /Claes From yumin.qi at oracle.com Tue Mar 17 23:01:32 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Tue, 17 Mar 2020 16:01:32 -0700 Subject: RFR(XS): 8241009: CommandLineFlagComboNegative.java fails after JDK-8240563 Message-ID: Hi, please review ? Bug:https://bugs.openjdk.java.net/browse/JDK-8241009 ? Webrev: http://cr.openjdk.java.net/~minqi/8241009/webrev/ ? The vmoptions from jtreg still affect the launched processes and mess up the parameters for launched java processes in the test. ? Fix skip the test if -XX:-UseCompressedOops or -XX:-UseCompressedClassPointers in jtreg vmoptions (or javaoptions). ? Tests: tested with explicitly set -XX:-UseCompressedOops or -XX:-UseCompressedClassPointers or both in jtreg vmoptions. ? tier1,2,4,7 test pending. Thanks Yumin From david.holmes at oracle.com Wed Mar 18 00:47:07 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 18 Mar 2020 10:47:07 +1000 Subject: URGENT (trivial) RFR: 8241124: Aarch64 build broken by JDK-8239593 Message-ID: <7ad384ab-6af7-a15d-c22a-07af27080fac@oracle.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8241124 webrev: http://cr.openjdk.java.net/~dholmes/8241124/webrev/ Missing include file. --- old/src/hotspot/share/prims/jni.cpp 2020-03-17 20:42:10.342771597 -0400 +++ new/src/hotspot/share/prims/jni.cpp 2020-03-17 20:42:08.606752168 -0400 @@ -32,6 +32,7 @@ #include "classfile/classLoader.hpp" #include "classfile/javaClasses.hpp" #include "classfile/javaClasses.inline.hpp" +#include "classfile/moduleEntry.hpp" #include "classfile/modules.hpp" #include "classfile/symbolTable.hpp" #include "classfile/systemDictionary.hpp" Tested with local aarch64 and x64 build. Thanks, David From mikael.vidstedt at oracle.com Wed Mar 18 00:47:47 2020 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Tue, 17 Mar 2020 17:47:47 -0700 Subject: URGENT (trivial) RFR: 8241124: Aarch64 build broken by JDK-8239593 In-Reply-To: <7ad384ab-6af7-a15d-c22a-07af27080fac@oracle.com> References: <7ad384ab-6af7-a15d-c22a-07af27080fac@oracle.com> Message-ID: <5369ACB8-3725-4C1F-A943-2B32EE2FF05B@oracle.com> Looks good, thanks for fixing! Cheers, Mikael > On Mar 17, 2020, at 5:47 PM, David Holmes wrote: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8241124 > webrev: http://cr.openjdk.java.net/~dholmes/8241124/webrev/ > > Missing include file. > > --- old/src/hotspot/share/prims/jni.cpp 2020-03-17 20:42:10.342771597 -0400 > +++ new/src/hotspot/share/prims/jni.cpp 2020-03-17 20:42:08.606752168 -0400 > @@ -32,6 +32,7 @@ > #include "classfile/classLoader.hpp" > #include "classfile/javaClasses.hpp" > #include "classfile/javaClasses.inline.hpp" > +#include "classfile/moduleEntry.hpp" > #include "classfile/modules.hpp" > #include "classfile/symbolTable.hpp" > #include "classfile/systemDictionary.hpp" > > Tested with local aarch64 and x64 build. > > Thanks, > David From mikhailo.seledtsov at oracle.com Wed Mar 18 01:24:07 2020 From: mikhailo.seledtsov at oracle.com (mikhailo.seledtsov at oracle.com) Date: Tue, 17 Mar 2020 18:24:07 -0700 Subject: RFR(S): 8240634: event/runtime/TestMetaspaceAllocationFailure.java times out Message-ID: This test timed out occasionally, about 1/100 runs, on Linux only. Please review this change that mitigates occasional test failure by limiting the number of iterations attempting to fill up metaspace, and throwing SkippedException when the max number of iterations is reached. ??? JBS: https://bugs.openjdk.java.net/browse/JDK-8240634 ??? Webrev: http://cr.openjdk.java.net/~mseledtsov/8240634.00/ ??? Testing: ??????? Ran 100 times on Linux, 1 skipped, rest PASS Thank you, Misha From david.holmes at oracle.com Wed Mar 18 04:52:40 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 17 Mar 2020 21:52:40 -0700 (PDT) Subject: RFR(XS): 8241009: CommandLineFlagComboNegative.java fails after JDK-8240563 In-Reply-To: References: Message-ID: Hi Yumin, On 18/03/2020 9:01 am, Yumin Qi wrote: > Hi, please review > > ? Bug:https://bugs.openjdk.java.net/browse/JDK-8241009 > > ? Webrev: http://cr.openjdk.java.net/~minqi/8241009/webrev/ > > ? The vmoptions from jtreg still affect the launched processes and mess > up the parameters for launched java processes in the test. > ? Fix skip the test if -XX:-UseCompressedOops or > -XX:-UseCompressedClassPointers in jtreg vmoptions (or javaoptions). I can think of a couple of different ways to try and account for this problem, none of which stands out to me as the best way, so using whitebox to skip the test seems okay. Ideally we would have a simple way to filter the passed through options so that we could remove problematic ones for a given test, or add new flags to override the problematic ones. Thanks, David ----- > ? Tests: tested with explicitly set -XX:-UseCompressedOops or > -XX:-UseCompressedClassPointers or both in jtreg vmoptions. > ? tier1,2,4,7 test pending. > > Thanks > Yumin From david.holmes at oracle.com Wed Mar 18 05:11:27 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 18 Mar 2020 15:11:27 +1000 Subject: RFR(L) 8237354: Add option to jcmd to write a gzipped heap dump In-Reply-To: References: <01361a9d-2855-db67-a176-73731fada08f@oracle.com> <0c687e55-ed91-e606-28a7-f9aef745ed8d@oracle.com> <490d58f7-7adc-00aa-b504-0ac284fe7eb5@oracle.com> <0343dfac-61f7-1b1c-ee96-bdee130578ad@oracle.com> Message-ID: Hi Ralf, On 13/03/2020 9:43 pm, Schmelter, Ralf wrote: > Hi, > > I have updated the webrev: http://cr.openjdk.java.net/~rschmelter/webrevs/8237354/webrev.1/ > > It has the following significant changes: > > - The jcmd now uses two separate flags. The -gz flag is now a boolean flag which toggles the compression on/off. And the new -gz-level flag can be used to change the compression level. If tried to change the jlong flag coding to allow the old behavior (only one flag, which acts both as a boolean flag and a jlong flag), but decided against it, since it changes the semantic of a jlong flag. And I don't expect the -gz-level flag to be used all that much. > > - I no longer use my own threads. Instead I use the WorkGang returned from CollectedHeap:: get_safepoint_workers(). This works fine, apart from Shenandoah GC, which runs into assertions when calling the CollectedHeap::object_iterate() method from a worker thread. I'm not sure if the assertion is too strong, but since the GC is currently experimental, I switch back to single threading in this case (as would be the case for serial GC or epsilon GC). Using the worker threads removes the problems the original code had regarding destruction of the monitor used. I'm glad to see you are no longer using your own threads, and I apologise that I have not yet been able to look further into the thread lifecycle issues you encountered. However I'm not clear how this solves the problem of destroying the monitor while it can still be being accessed - is the dumping occurring at a safepoint in the WorkGang threads? Thanks, David ----- > - The reported number of bytes is now the one written to disk. > > Best regards, > Ralf > > -----Original Message----- > From: Ioi Lam > Sent: Dienstag, 25. Februar 2020 18:03 > To: Langer, Christoph ; Schmelter, Ralf ; Yasumasa Suenaga ; serguei.spitsyn at oracle.com; hotspot-runtime-dev at openjdk.java.net runtime > Cc: serviceability-dev at openjdk.java.net > Subject: Re: RFR(L) 8237354: Add option to jcmd to write a gzipped heap dump > > Hi Christoph, > > This sounds fair. I will remove my objection :-) > > Thanks > - Ioi > From shade at redhat.com Wed Mar 18 05:44:22 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 18 Mar 2020 06:44:22 +0100 Subject: RFR (XS) 8241128: x86_32 build failure after JDK-8241042 Message-ID: <7aa093ae-3e9b-e929-a589-7735a4fbe730@redhat.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8241128 JDK-8241042 changed the lines below incorrectly. There is no multi-arg emit_int8, and it must be emit_int16 or emit_int24. Fix: diff -r 82d11846109a -r 3d86b4b4701b src/hotspot/cpu/x86/assembler_x86.cpp --- a/src/hotspot/cpu/x86/assembler_x86.cpp Wed Mar 18 06:28:50 2020 +0100 +++ b/src/hotspot/cpu/x86/assembler_x86.cpp Wed Mar 18 06:39:55 2020 +0100 @@ -6838,7 +6838,7 @@ void Assembler::cmp_literal32(Register src1, int32_t imm32, RelocationHolder const& rspec) { // NO PREFIX AS NEVER 64BIT InstructionMark im(this); - emit_int8((unsigned char)0x81, (0xF8 | src1->encoding())); + emit_int16((unsigned char)0x81, (0xF8 | src1->encoding())); emit_data(imm32, rspec, 0); } @@ -7638,7 +7638,7 @@ } void Assembler::set_byte_if_not_zero(Register dst) { - emit_int8(0x0F, (unsigned char)0x95, (0xE0 | dst->encoding())); + emit_int24(0x0F, (unsigned char)0x95, (0xE0 | dst->encoding())); } #else // LP64 Testing: x86_32 build -- Thanks, -Aleksey From ralf.schmelter at sap.com Wed Mar 18 06:39:36 2020 From: ralf.schmelter at sap.com (Schmelter, Ralf) Date: Wed, 18 Mar 2020 06:39:36 +0000 Subject: RFR(L) 8237354: Add option to jcmd to write a gzipped heap dump In-Reply-To: References: <01361a9d-2855-db67-a176-73731fada08f@oracle.com> <0c687e55-ed91-e606-28a7-f9aef745ed8d@oracle.com> <490d58f7-7adc-00aa-b504-0ac284fe7eb5@oracle.com> <0343dfac-61f7-1b1c-ee96-bdee130578ad@oracle.com> , Message-ID: Hi David, >?However I'm not clear how this solves??the problem of destroying > the monitor while it can still be being?accessed - is the dumping > occurring at a safepoint in the WorkGang threads? Because when the run_task() method returns, I can be sure none of the work gang threads still use the mutex. They have to exit the thread_loop() method to finish the task. And by exiting the method they have released the mutex. Best regards, Ralf From: David Holmes Sent: Wednesday, March 18, 2020 6:11 AM To: Schmelter, Ralf ; Ioi Lam ; Langer, Christoph ; Yasumasa Suenaga ; serguei.spitsyn at oracle.com ; hotspot-runtime-dev at openjdk.java.net runtime Cc: serviceability-dev at openjdk.java.net Subject: Re: RFR(L) 8237354: Add option to jcmd to write a gzipped heap dump ? Hi Ralf, On 13/03/2020 9:43 pm, Schmelter, Ralf wrote: > Hi, > > I have updated the webrev: http://cr.openjdk.java.net/~rschmelter/webrevs/8237354/webrev.1/ > > It has the following significant changes: > > - The jcmd now uses two separate flags. The -gz flag is now a boolean flag which toggles the compression? on/off. And the new -gz-level flag can be used to change the compression level. If tried to change the jlong flag coding to allow the old behavior (only one flag, which acts both as a boolean flag and a jlong flag), but decided against it, since it changes the semantic of a jlong flag. And I don't expect the -gz-level flag to be used all that much. > > - I no longer use my own threads. Instead I use the WorkGang returned from CollectedHeap:: get_safepoint_workers(). This works fine, apart from Shenandoah GC, which runs into assertions when calling the CollectedHeap::object_iterate() method from a worker thread. I'm not sure if the assertion is too strong, but since the GC is currently experimental, I switch back to single threading in this case (as would be the case for serial GC or epsilon GC). Using the worker threads removes the problems the original code had regarding destruction of the monitor used. I'm glad to see you are no longer using your own threads, and I apologise that I have not yet been able to look further into the thread lifecycle issues you encountered. However I'm not clear how this solves the problem of destroying the monitor while it can still be being accessed - is the dumping occurring at a safepoint in the WorkGang threads? Thanks, David ----- From david.holmes at oracle.com Wed Mar 18 06:43:22 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 18 Mar 2020 16:43:22 +1000 Subject: RFR(L) 8237354: Add option to jcmd to write a gzipped heap dump In-Reply-To: References: <0c687e55-ed91-e606-28a7-f9aef745ed8d@oracle.com> <490d58f7-7adc-00aa-b504-0ac284fe7eb5@oracle.com> <0343dfac-61f7-1b1c-ee96-bdee130578ad@oracle.com> Message-ID: <05ae5818-f1b7-2e86-e4dd-c09ff240748e@oracle.com> On 18/03/2020 4:39 pm, Schmelter, Ralf wrote: > Hi David, > >> ?However I'm not clear how this solves??the problem of destroying >> the monitor while it can still be being?accessed - is the dumping >> occurring at a safepoint in the WorkGang threads? > > Because when the run_task() method returns, I can be sure none > of the work gang threads still use the mutex. They have to exit the > thread_loop() method to finish the task. And by exiting the method > they have released the mutex. All of which is happening via VM_HeapDumper::doit(). Got it. Thanks, David > Best regards, > Ralf > > > > > > > From: David Holmes > > Sent: Wednesday, March 18, 2020 6:11 AM > > To: Schmelter, Ralf ; Ioi Lam ; Langer, Christoph ; Yasumasa Suenaga ; serguei.spitsyn at oracle.com ; hotspot-runtime-dev at openjdk.java.net > runtime > > Cc: serviceability-dev at openjdk.java.net > > Subject: Re: RFR(L) 8237354: Add option to jcmd to write a gzipped heap dump > > > > > Hi Ralf, > > > > On 13/03/2020 9:43 pm, Schmelter, Ralf wrote: > >> Hi, > >> > >> I have updated the webrev: > http://cr.openjdk.java.net/~rschmelter/webrevs/8237354/webrev.1/ > >> > >> It has the following significant changes: > >> > >> - The jcmd now uses two separate flags. The -gz flag is now a boolean flag which toggles the compression? on/off. And the new -gz-level flag can be used to change the compression level. If tried to change the jlong flag coding to allow the old behavior (only > one flag, which acts both as a boolean flag and a jlong flag), but decided against it, since it changes the semantic of a jlong flag. And I don't expect the -gz-level flag to be used all that much. > >> > >> - I no longer use my own threads. Instead I use the WorkGang returned from CollectedHeap:: get_safepoint_workers(). This works fine, apart from Shenandoah GC, which runs into assertions when calling the CollectedHeap::object_iterate() method from a worker > thread. I'm not sure if the assertion is too strong, but since the GC is currently experimental, I switch back to single threading in this case (as would be the case for serial GC or epsilon GC). Using the worker threads removes the problems the original code > had regarding destruction of the monitor used. > > > > I'm glad to see you are no longer using your own threads, and I > > apologise that I have not yet been able to look further into the thread > > lifecycle issues you encountered. However I'm not clear how this solves > > the problem of destroying the monitor while it can still be being > > accessed - is the dumping occurring at a safepoint in the WorkGang threads? > > > > Thanks, > > David > > ----- > > From stefan.karlsson at oracle.com Wed Mar 18 08:58:07 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 18 Mar 2020 09:58:07 +0100 Subject: RFR(XS): 8241009: CommandLineFlagComboNegative.java fails after JDK-8240563 In-Reply-To: References: Message-ID: <0d2552b2-155f-278e-2437-848599917792@oracle.com> Hi Yumin, On 2020-03-18 00:01, Yumin Qi wrote: > Hi, please review > > ? Bug:https://bugs.openjdk.java.net/browse/JDK-8241009 > > ? Webrev: http://cr.openjdk.java.net/~minqi/8241009/webrev/ > > ? The vmoptions from jtreg still affect the launched processes and > mess up the parameters for launched java processes in the test. > ? Fix skip the test if -XX:-UseCompressedOops or > -XX:-UseCompressedClassPointers in jtreg vmoptions (or javaoptions). We already have a way to use @require to disable tests when incompatible flags are passed. Take a look at: test/hotspot/jtreg/runtime/CompressedOops/CompressedClassSpaceSize.java ?* @requires vm.bits == 64 & vm.opt.final.UseCompressedOops == true Could this be used instead? Note that this is using vm.opt.final.UseCompressedOops and not vm.opt.UseCompressedOops. The former actually checks the value of the flag, just like your adhoc version does. The list of supported flags are currently hard-coded, so if you need a flag that's not already exposed, you'll have to add it yourself. See: test/jtreg-ext/requires/VMProps.java ??? /** ???? * Selected final flag. ???? * ???? * @param map - property-value pairs ???? * @param flagName - flag name ???? */ ??? private void vmOptFinalFlag(SafeMap map, String flagName) { ??????? map.put("vm.opt.final." + flagName, ??????????????? () -> String.valueOf(WB.getBooleanVMFlag(flagName))); ??? } ??? /** ???? * Selected sets of final flags. ???? * ???? * @param map - property-value pairs ???? */ ??? protected void vmOptFinalFlags(SafeMap map) { ??????? vmOptFinalFlag(map, "ClassUnloading"); ??????? vmOptFinalFlag(map, "ClassUnloadingWithConcurrentMark"); ??????? vmOptFinalFlag(map, "UseCompressedOops"); ??????? vmOptFinalFlag(map, "EnableJVMCI"); ??????? vmOptFinalFlag(map, "EliminateAllocations"); ??? } Thanks, StefanK > > ? Tests: tested with explicitly set -XX:-UseCompressedOops or > -XX:-UseCompressedClassPointers or both in jtreg vmoptions. > ? tier1,2,4,7 test pending. > > Thanks > Yumin From claes.redestad at oracle.com Wed Mar 18 13:46:09 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 18 Mar 2020 14:46:09 +0100 Subject: RFR: 8241095: x86: Improve prefix handling in Assembler Message-ID: <40d97c7c-9d94-7ba1-ab97-da80e424100c@oracle.com> Hej, this patch enable folding emitted prefixq bytes into multi-byte emitter functions (emit_int16(int8, int8) etc). Only those functions where there is always a prefix being emitted have been adjusted. Other variants could improve slightly by doing similar tricks, but with a smaller payoff and more complexity. Webrev: http://cr.openjdk.java.net/~redestad/8241095/open.00 Bug: https://bugs.openjdk.java.net/browse/JDK-8241095 Testing: tier1-4 (ongoing) Thanks, /Claes From claes.redestad at oracle.com Wed Mar 18 17:01:42 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 18 Mar 2020 18:01:42 +0100 Subject: RFR (XS) 8241128: x86_32 build failure after JDK-8241042 In-Reply-To: <7aa093ae-3e9b-e929-a589-7735a4fbe730@redhat.com> References: <7aa093ae-3e9b-e929-a589-7735a4fbe730@redhat.com> Message-ID: <78db5c28-cb2b-3510-4a4f-10627f0cafff@oracle.com> Looks good. /Claes On 2020-03-18 06:44, Aleksey Shipilev wrote: > Bug: > https://bugs.openjdk.java.net/browse/JDK-8241128 > > JDK-8241042 changed the lines below incorrectly. There is no multi-arg emit_int8, and it must be > emit_int16 or emit_int24. > > Fix: > > diff -r 82d11846109a -r 3d86b4b4701b src/hotspot/cpu/x86/assembler_x86.cpp > --- a/src/hotspot/cpu/x86/assembler_x86.cpp Wed Mar 18 06:28:50 2020 +0100 > +++ b/src/hotspot/cpu/x86/assembler_x86.cpp Wed Mar 18 06:39:55 2020 +0100 > @@ -6838,7 +6838,7 @@ > void Assembler::cmp_literal32(Register src1, int32_t imm32, RelocationHolder const& rspec) { > // NO PREFIX AS NEVER 64BIT > InstructionMark im(this); > - emit_int8((unsigned char)0x81, (0xF8 | src1->encoding())); > + emit_int16((unsigned char)0x81, (0xF8 | src1->encoding())); > emit_data(imm32, rspec, 0); > } > > @@ -7638,7 +7638,7 @@ > } > > void Assembler::set_byte_if_not_zero(Register dst) { > - emit_int8(0x0F, (unsigned char)0x95, (0xE0 | dst->encoding())); > + emit_int24(0x0F, (unsigned char)0x95, (0xE0 | dst->encoding())); > } > > #else // LP64 > > Testing: x86_32 build > From claes.redestad at oracle.com Wed Mar 18 17:02:09 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 18 Mar 2020 18:02:09 +0100 Subject: RFR (XS) 8241128: x86_32 build failure after JDK-8241042 In-Reply-To: <78db5c28-cb2b-3510-4a4f-10627f0cafff@oracle.com> References: <7aa093ae-3e9b-e929-a589-7735a4fbe730@redhat.com> <78db5c28-cb2b-3510-4a4f-10627f0cafff@oracle.com> Message-ID: <0d2527c7-912c-ad41-a8d9-0238342760c2@oracle.com> And trivial! On 2020-03-18 18:01, Claes Redestad wrote: > Looks good. > > /Claes > > On 2020-03-18 06:44, Aleksey Shipilev wrote: >> Bug: >> ?? https://bugs.openjdk.java.net/browse/JDK-8241128 >> >> JDK-8241042 changed the lines below incorrectly. There is no multi-arg >> emit_int8, and it must be >> emit_int16 or emit_int24. >> >> Fix: >> >> diff -r 82d11846109a -r 3d86b4b4701b >> src/hotspot/cpu/x86/assembler_x86.cpp >> --- a/src/hotspot/cpu/x86/assembler_x86.cpp???? Wed Mar 18 06:28:50 >> 2020 +0100 >> +++ b/src/hotspot/cpu/x86/assembler_x86.cpp???? Wed Mar 18 06:39:55 >> 2020 +0100 >> @@ -6838,7 +6838,7 @@ >> ? void Assembler::cmp_literal32(Register src1, int32_t imm32, >> RelocationHolder const& rspec) { >> ??? // NO PREFIX AS NEVER 64BIT >> ??? InstructionMark im(this); >> -? emit_int8((unsigned char)0x81, (0xF8 | src1->encoding())); >> +? emit_int16((unsigned char)0x81, (0xF8 | src1->encoding())); >> ??? emit_data(imm32, rspec, 0); >> ? } >> >> @@ -7638,7 +7638,7 @@ >> ? } >> >> ? void Assembler::set_byte_if_not_zero(Register dst) { >> -? emit_int8(0x0F, (unsigned char)0x95, (0xE0 | dst->encoding())); >> +? emit_int24(0x0F, (unsigned char)0x95, (0xE0 | dst->encoding())); >> ? } >> >> ? #else // LP64 >> >> Testing: x86_32 build >> From vladimir.kozlov at oracle.com Wed Mar 18 17:12:34 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 18 Mar 2020 10:12:34 -0700 Subject: RFR (XS) 8241128: x86_32 build failure after JDK-8241042 In-Reply-To: <78db5c28-cb2b-3510-4a4f-10627f0cafff@oracle.com> References: <7aa093ae-3e9b-e929-a589-7735a4fbe730@redhat.com> <78db5c28-cb2b-3510-4a4f-10627f0cafff@oracle.com> Message-ID: <71152836-0022-9fd0-fbfa-24ebfe26ff56@oracle.com> +1 Thanks, Vladimir On 3/18/20 10:01 AM, Claes Redestad wrote: > Looks good. > > /Claes > > On 2020-03-18 06:44, Aleksey Shipilev wrote: >> Bug: >> ?? https://bugs.openjdk.java.net/browse/JDK-8241128 >> >> JDK-8241042 changed the lines below incorrectly. There is no multi-arg emit_int8, and it must be >> emit_int16 or emit_int24. >> >> Fix: >> >> diff -r 82d11846109a -r 3d86b4b4701b src/hotspot/cpu/x86/assembler_x86.cpp >> --- a/src/hotspot/cpu/x86/assembler_x86.cpp???? Wed Mar 18 06:28:50 2020 +0100 >> +++ b/src/hotspot/cpu/x86/assembler_x86.cpp???? Wed Mar 18 06:39:55 2020 +0100 >> @@ -6838,7 +6838,7 @@ >> ? void Assembler::cmp_literal32(Register src1, int32_t imm32, RelocationHolder const& rspec) { >> ??? // NO PREFIX AS NEVER 64BIT >> ??? InstructionMark im(this); >> -? emit_int8((unsigned char)0x81, (0xF8 | src1->encoding())); >> +? emit_int16((unsigned char)0x81, (0xF8 | src1->encoding())); >> ??? emit_data(imm32, rspec, 0); >> ? } >> >> @@ -7638,7 +7638,7 @@ >> ? } >> >> ? void Assembler::set_byte_if_not_zero(Register dst) { >> -? emit_int8(0x0F, (unsigned char)0x95, (0xE0 | dst->encoding())); >> +? emit_int24(0x0F, (unsigned char)0x95, (0xE0 | dst->encoding())); >> ? } >> >> ? #else // LP64 >> >> Testing: x86_32 build >> From yumin.qi at oracle.com Wed Mar 18 17:18:21 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Wed, 18 Mar 2020 10:18:21 -0700 Subject: RFR(XS): 8241009: CommandLineFlagComboNegative.java fails after JDK-8240563 In-Reply-To: References: Message-ID: <421aeb8d-b3a5-c331-1458-1f3a69da3708@oracle.com> Hi, David ? I have updated the web, I missed -XX:+UnlockDiagnosticVMOptions -XX:+WhiteBoxAPI --- this will cause on some platform(Windows) linkage Error. ? It passed locally due to !vm.gc.Z which will not be selected when run with ZGC. Now it will skip ZGC with checkSkip(). On 3/17/20 9:52 PM, David Holmes wrote: > Hi Yumin, > > On 18/03/2020 9:01 am, Yumin Qi wrote: >> Hi, please review >> >> ?? Bug:https://bugs.openjdk.java.net/browse/JDK-8241009 >> >> ?? Webrev: http://cr.openjdk.java.net/~minqi/8241009/webrev/ >> >> ?? The vmoptions from jtreg still affect the launched processes and >> mess up the parameters for launched java processes in the test. >> ?? Fix skip the test if -XX:-UseCompressedOops or >> -XX:-UseCompressedClassPointers in jtreg vmoptions (or javaoptions). > > I can think of a couple of different ways to try and account for this > problem, none of which stands out to me as the best way, so using > whitebox to skip the test seems okay. > > Ideally we would have a simple way to filter the passed through > options so that we could remove problematic ones for a given test, or > add new flags to override the problematic ones. > Yes, agree. Should we suggest test coding with some rules such like: if the test not launching process, like @run main/.. TestCaste, the options specified in by the author. if the test launching process, the test frame filter out the vmoptions. The cases in appcds are more like this. The children processes launched with their own vm options, jtreg options will mess them up. I think bug https://bugs.openjdk.java.net/browse/JDK-8240698 should handle this. The best solution maybe, just run jtreg without vmoptions, let the author name them in the tests either @run or supplying options to children processes. Thanks Yumin > Thanks, > David > ----- > >> ?? Tests: tested with explicitly set -XX:-UseCompressedOops or >> -XX:-UseCompressedClassPointers or both in jtreg vmoptions. >> ?? tier1,2,4,7 test pending. >> >> Thanks >> Yumin From yumin.qi at oracle.com Wed Mar 18 17:21:46 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Wed, 18 Mar 2020 10:21:46 -0700 Subject: RFR(XS): 8241009: CommandLineFlagComboNegative.java fails after JDK-8240563 In-Reply-To: <0d2552b2-155f-278e-2437-848599917792@oracle.com> References: <0d2552b2-155f-278e-2437-848599917792@oracle.com> Message-ID: <387010be-9e36-bde3-9372-de98092b2c89@oracle.com> Hi, Karlsson ? Thanks. that is more concise version. Will update. Yumin On 3/18/20 1:58 AM, Stefan Karlsson wrote: > Hi Yumin, > > On 2020-03-18 00:01, Yumin Qi wrote: >> Hi, please review >> >> ? Bug:https://bugs.openjdk.java.net/browse/JDK-8241009 >> >> ? Webrev: http://cr.openjdk.java.net/~minqi/8241009/webrev/ >> >> ? The vmoptions from jtreg still affect the launched processes and >> mess up the parameters for launched java processes in the test. >> ? Fix skip the test if -XX:-UseCompressedOops or >> -XX:-UseCompressedClassPointers in jtreg vmoptions (or javaoptions). > > We already have a way to use @require to disable tests when > incompatible flags are passed. > > Take a look at: > test/hotspot/jtreg/runtime/CompressedOops/CompressedClassSpaceSize.java > > ?* @requires vm.bits == 64 & vm.opt.final.UseCompressedOops == true > > Could this be used instead? Note that this is using > vm.opt.final.UseCompressedOops and not vm.opt.UseCompressedOops. The > former actually checks the value of the flag, just like your adhoc > version does. > > The list of supported flags are currently hard-coded, so if you need a > flag that's not already exposed, you'll have to add it yourself. See: > test/jtreg-ext/requires/VMProps.java > > ??? /** > ???? * Selected final flag. > ???? * > ???? * @param map - property-value pairs > ???? * @param flagName - flag name > ???? */ > ??? private void vmOptFinalFlag(SafeMap map, String flagName) { > ??????? map.put("vm.opt.final." + flagName, > ??????????????? () -> String.valueOf(WB.getBooleanVMFlag(flagName))); > ??? } > > ??? /** > ???? * Selected sets of final flags. > ???? * > ???? * @param map - property-value pairs > ???? */ > ??? protected void vmOptFinalFlags(SafeMap map) { > ??????? vmOptFinalFlag(map, "ClassUnloading"); > ??????? vmOptFinalFlag(map, "ClassUnloadingWithConcurrentMark"); > ??????? vmOptFinalFlag(map, "UseCompressedOops"); > ??????? vmOptFinalFlag(map, "EnableJVMCI"); > ??????? vmOptFinalFlag(map, "EliminateAllocations"); > ??? } > > Thanks, > StefanK > >> >> ? Tests: tested with explicitly set -XX:-UseCompressedOops or >> -XX:-UseCompressedClassPointers or both in jtreg vmoptions. >> ? tier1,2,4,7 test pending. >> >> Thanks >> Yumin > From shade at redhat.com Wed Mar 18 18:25:10 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 18 Mar 2020 19:25:10 +0100 Subject: RFR (XS) 8241128: x86_32 build failure after JDK-8241042 In-Reply-To: <71152836-0022-9fd0-fbfa-24ebfe26ff56@oracle.com> References: <7aa093ae-3e9b-e929-a589-7735a4fbe730@redhat.com> <78db5c28-cb2b-3510-4a4f-10627f0cafff@oracle.com> <71152836-0022-9fd0-fbfa-24ebfe26ff56@oracle.com> Message-ID: Thanks, pushed. On 3/18/20 6:12 PM, Vladimir Kozlov wrote: > +1 > > Thanks, > Vladimir > > On 3/18/20 10:01 AM, Claes Redestad wrote: >> Looks good. >> >> /Claes -Aleksey From vladimir.kozlov at oracle.com Wed Mar 18 19:30:12 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 18 Mar 2020 12:30:12 -0700 Subject: RFR: 8241095: x86: Improve prefix handling in Assembler In-Reply-To: <40d97c7c-9d94-7ba1-ab97-da80e424100c@oracle.com> References: <40d97c7c-9d94-7ba1-ab97-da80e424100c@oracle.com> Message-ID: Looks good. I checked that moved methods stay the same. You only modified prefixq(Address adr) and prefixq(Address adr, Register src) as we discussed. Good. Thanks, Vladimir On 3/18/20 6:46 AM, Claes Redestad wrote: > Hej, > > this patch enable folding emitted prefixq bytes into multi-byte emitter > functions (emit_int16(int8, int8) etc). > > Only those functions where there is always a prefix being emitted have > been adjusted. Other variants could improve slightly by doing similar > tricks, but with a smaller payoff and more complexity. > > Webrev: http://cr.openjdk.java.net/~redestad/8241095/open.00 > Bug:??? https://bugs.openjdk.java.net/browse/JDK-8241095 > > Testing: tier1-4 (ongoing) > > Thanks, > > /Claes From claes.redestad at oracle.com Wed Mar 18 21:13:26 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 18 Mar 2020 22:13:26 +0100 Subject: RFR: 8241095: x86: Improve prefix handling in Assembler In-Reply-To: References: <40d97c7c-9d94-7ba1-ab97-da80e424100c@oracle.com> Message-ID: On 2020-03-18 20:30, Vladimir Kozlov wrote: > Looks good. Thanks! > > I checked that moved methods stay the same. You only modified > prefixq(Address adr) and prefixq(Address adr, Register src) as we > discussed. Good. Yep. Also: By passing in rax (or any other valid register constant in the 0-7 encoding range) I can reuse get_prefixq(Address adr, Register src) from prefixq(Address adr) without any overhead in compiled code. I'll add a short comment on that. /Claes > > Thanks, > Vladimir > > On 3/18/20 6:46 AM, Claes Redestad wrote: >> Hej, >> >> this patch enable folding emitted prefixq bytes into multi-byte emitter >> functions (emit_int16(int8, int8) etc). >> >> Only those functions where there is always a prefix being emitted have >> been adjusted. Other variants could improve slightly by doing similar >> tricks, but with a smaller payoff and more complexity. >> >> Webrev: http://cr.openjdk.java.net/~redestad/8241095/open.00 >> Bug:??? https://bugs.openjdk.java.net/browse/JDK-8241095 >> >> Testing: tier1-4 (ongoing) >> >> Thanks, >> >> /Claes From yumin.qi at oracle.com Wed Mar 18 21:43:46 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Wed, 18 Mar 2020 14:43:46 -0700 Subject: RFR(XS): 8241009: CommandLineFlagComboNegative.java fails after JDK-8240563 In-Reply-To: <387010be-9e36-bde3-9372-de98092b2c89@oracle.com> References: <0d2552b2-155f-278e-2437-848599917792@oracle.com> <387010be-9e36-bde3-9372-de98092b2c89@oracle.com> Message-ID: <8d8d06ca-a788-7b37-6c5e-d05d61df0bd0@oracle.com> Hi, Stefan and all ? (Sorry last email spell your last name first). ? Update using your suggestion. Also remove the checking for 64bit code since after the change, checking against 64bit no longer needed. http://cr.openjdk.java.net/~minqi/8241009/webrev/ Thanks Yumin On 3/18/20 10:21 AM, Yumin Qi wrote: > Hi, Karlsson > ? Thanks. that is more concise version. Will update. > > Yumin > > On 3/18/20 1:58 AM, Stefan Karlsson wrote: >> Hi Yumin, >> >> On 2020-03-18 00:01, Yumin Qi wrote: >>> Hi, please review >>> >>> ? Bug:https://bugs.openjdk.java.net/browse/JDK-8241009 >>> >>> ? Webrev: http://cr.openjdk.java.net/~minqi/8241009/webrev/ >>> >>> ? The vmoptions from jtreg still affect the launched processes and >>> mess up the parameters for launched java processes in the test. >>> ? Fix skip the test if -XX:-UseCompressedOops or >>> -XX:-UseCompressedClassPointers in jtreg vmoptions (or javaoptions). >> >> We already have a way to use @require to disable tests when >> incompatible flags are passed. >> >> Take a look at: >> test/hotspot/jtreg/runtime/CompressedOops/CompressedClassSpaceSize.java >> >> ?* @requires vm.bits == 64 & vm.opt.final.UseCompressedOops == true >> >> Could this be used instead? Note that this is using >> vm.opt.final.UseCompressedOops and not vm.opt.UseCompressedOops. The >> former actually checks the value of the flag, just like your adhoc >> version does. >> >> The list of supported flags are currently hard-coded, so if you need >> a flag that's not already exposed, you'll have to add it yourself. >> See: test/jtreg-ext/requires/VMProps.java >> >> ??? /** >> ???? * Selected final flag. >> ???? * >> ???? * @param map - property-value pairs >> ???? * @param flagName - flag name >> ???? */ >> ??? private void vmOptFinalFlag(SafeMap map, String flagName) { >> ??????? map.put("vm.opt.final." + flagName, >> ??????????????? () -> String.valueOf(WB.getBooleanVMFlag(flagName))); >> ??? } >> >> ??? /** >> ???? * Selected sets of final flags. >> ???? * >> ???? * @param map - property-value pairs >> ???? */ >> ??? protected void vmOptFinalFlags(SafeMap map) { >> ??????? vmOptFinalFlag(map, "ClassUnloading"); >> ??????? vmOptFinalFlag(map, "ClassUnloadingWithConcurrentMark"); >> ??????? vmOptFinalFlag(map, "UseCompressedOops"); >> ??????? vmOptFinalFlag(map, "EnableJVMCI"); >> ??????? vmOptFinalFlag(map, "EliminateAllocations"); >> ??? } >> >> Thanks, >> StefanK >> >>> >>> ? Tests: tested with explicitly set -XX:-UseCompressedOops or >>> -XX:-UseCompressedClassPointers or both in jtreg vmoptions. >>> ? tier1,2,4,7 test pending. >>> >>> Thanks >>> Yumin >> > From david.holmes at oracle.com Wed Mar 18 22:54:39 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 19 Mar 2020 08:54:39 +1000 Subject: RFR(XS): 8241009: CommandLineFlagComboNegative.java fails after JDK-8240563 In-Reply-To: <8d8d06ca-a788-7b37-6c5e-d05d61df0bd0@oracle.com> References: <0d2552b2-155f-278e-2437-848599917792@oracle.com> <387010be-9e36-bde3-9372-de98092b2c89@oracle.com> <8d8d06ca-a788-7b37-6c5e-d05d61df0bd0@oracle.com> Message-ID: <5ee7c5f8-2eb6-4a7c-a412-5934607f4cc3@oracle.com> Hi Yumin, On 19/03/2020 7:43 am, Yumin Qi wrote: > Hi, Stefan and all > > ? (Sorry last email spell your last name first). > ? Update using your suggestion. Also remove the checking for 64bit code > since after the change, checking against 64bit no longer needed. > > http://cr.openjdk.java.net/~minqi/8241009/webrev/ This doesn't seem ideal but it does seem sufficient. Ideally the test would explicitly set all the flags it depends upon rather than assuming/requiring the flags have a specific value. Thanks, David > Thanks > Yumin > > On 3/18/20 10:21 AM, Yumin Qi wrote: >> Hi, Karlsson >> ? Thanks. that is more concise version. Will update. >> >> Yumin >> >> On 3/18/20 1:58 AM, Stefan Karlsson wrote: >>> Hi Yumin, >>> >>> On 2020-03-18 00:01, Yumin Qi wrote: >>>> Hi, please review >>>> >>>> ? Bug:https://bugs.openjdk.java.net/browse/JDK-8241009 >>>> >>>> ? Webrev: http://cr.openjdk.java.net/~minqi/8241009/webrev/ >>>> >>>> ? The vmoptions from jtreg still affect the launched processes and >>>> mess up the parameters for launched java processes in the test. >>>> ? Fix skip the test if -XX:-UseCompressedOops or >>>> -XX:-UseCompressedClassPointers in jtreg vmoptions (or javaoptions). >>> >>> We already have a way to use @require to disable tests when >>> incompatible flags are passed. >>> >>> Take a look at: >>> test/hotspot/jtreg/runtime/CompressedOops/CompressedClassSpaceSize.java >>> >>> ?* @requires vm.bits == 64 & vm.opt.final.UseCompressedOops == true >>> >>> Could this be used instead? Note that this is using >>> vm.opt.final.UseCompressedOops and not vm.opt.UseCompressedOops. The >>> former actually checks the value of the flag, just like your adhoc >>> version does. >>> >>> The list of supported flags are currently hard-coded, so if you need >>> a flag that's not already exposed, you'll have to add it yourself. >>> See: test/jtreg-ext/requires/VMProps.java >>> >>> ??? /** >>> ???? * Selected final flag. >>> ???? * >>> ???? * @param map - property-value pairs >>> ???? * @param flagName - flag name >>> ???? */ >>> ??? private void vmOptFinalFlag(SafeMap map, String flagName) { >>> ??????? map.put("vm.opt.final." + flagName, >>> ??????????????? () -> String.valueOf(WB.getBooleanVMFlag(flagName))); >>> ??? } >>> >>> ??? /** >>> ???? * Selected sets of final flags. >>> ???? * >>> ???? * @param map - property-value pairs >>> ???? */ >>> ??? protected void vmOptFinalFlags(SafeMap map) { >>> ??????? vmOptFinalFlag(map, "ClassUnloading"); >>> ??????? vmOptFinalFlag(map, "ClassUnloadingWithConcurrentMark"); >>> ??????? vmOptFinalFlag(map, "UseCompressedOops"); >>> ??????? vmOptFinalFlag(map, "EnableJVMCI"); >>> ??????? vmOptFinalFlag(map, "EliminateAllocations"); >>> ??? } >>> >>> Thanks, >>> StefanK >>> >>>> >>>> ? Tests: tested with explicitly set -XX:-UseCompressedOops or >>>> -XX:-UseCompressedClassPointers or both in jtreg vmoptions. >>>> ? tier1,2,4,7 test pending. >>>> >>>> Thanks >>>> Yumin >>> >> > From yumin.qi at oracle.com Wed Mar 18 23:07:27 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Wed, 18 Mar 2020 16:07:27 -0700 Subject: RFR(XS): 8241009: CommandLineFlagComboNegative.java fails after JDK-8240563 In-Reply-To: <5ee7c5f8-2eb6-4a7c-a412-5934607f4cc3@oracle.com> References: <0d2552b2-155f-278e-2437-848599917792@oracle.com> <387010be-9e36-bde3-9372-de98092b2c89@oracle.com> <8d8d06ca-a788-7b37-6c5e-d05d61df0bd0@oracle.com> <5ee7c5f8-2eb6-4a7c-a412-5934607f4cc3@oracle.com> Message-ID: <2817c25a-c308-57a9-154a-785e90659a20@oracle.com> Thanks for review. Yumin On 3/18/20 3:54 PM, David Holmes wrote: > Hi Yumin, > > On 19/03/2020 7:43 am, Yumin Qi wrote: >> Hi, Stefan and all >> >> ?? (Sorry last email spell your last name first). >> ?? Update using your suggestion. Also remove the checking for 64bit >> code since after the change, checking against 64bit no longer needed. >> >> http://cr.openjdk.java.net/~minqi/8241009/webrev/ > > This doesn't seem ideal but it does seem sufficient. > > Ideally the test would explicitly set all the flags it depends upon > rather than assuming/requiring the flags have a specific value. > > Thanks, > David > >> Thanks >> Yumin >> >> On 3/18/20 10:21 AM, Yumin Qi wrote: >>> Hi, Karlsson >>> ? Thanks. that is more concise version. Will update. >>> >>> Yumin >>> >>> On 3/18/20 1:58 AM, Stefan Karlsson wrote: >>>> Hi Yumin, >>>> >>>> On 2020-03-18 00:01, Yumin Qi wrote: >>>>> Hi, please review >>>>> >>>>> ? Bug:https://bugs.openjdk.java.net/browse/JDK-8241009 >>>>> >>>>> ? Webrev: http://cr.openjdk.java.net/~minqi/8241009/webrev/ >>>>> >>>>> ? The vmoptions from jtreg still affect the launched processes and >>>>> mess up the parameters for launched java processes in the test. >>>>> ? Fix skip the test if -XX:-UseCompressedOops or >>>>> -XX:-UseCompressedClassPointers in jtreg vmoptions (or javaoptions). >>>> >>>> We already have a way to use @require to disable tests when >>>> incompatible flags are passed. >>>> >>>> Take a look at: >>>> test/hotspot/jtreg/runtime/CompressedOops/CompressedClassSpaceSize.java >>>> >>>> >>>> ?* @requires vm.bits == 64 & vm.opt.final.UseCompressedOops == true >>>> >>>> Could this be used instead? Note that this is using >>>> vm.opt.final.UseCompressedOops and not vm.opt.UseCompressedOops. >>>> The former actually checks the value of the flag, just like your >>>> adhoc version does. >>>> >>>> The list of supported flags are currently hard-coded, so if you >>>> need a flag that's not already exposed, you'll have to add it >>>> yourself. See: test/jtreg-ext/requires/VMProps.java >>>> >>>> ??? /** >>>> ???? * Selected final flag. >>>> ???? * >>>> ???? * @param map - property-value pairs >>>> ???? * @param flagName - flag name >>>> ???? */ >>>> ??? private void vmOptFinalFlag(SafeMap map, String flagName) { >>>> ??????? map.put("vm.opt.final." + flagName, >>>> ??????????????? () -> String.valueOf(WB.getBooleanVMFlag(flagName))); >>>> ??? } >>>> >>>> ??? /** >>>> ???? * Selected sets of final flags. >>>> ???? * >>>> ???? * @param map - property-value pairs >>>> ???? */ >>>> ??? protected void vmOptFinalFlags(SafeMap map) { >>>> ??????? vmOptFinalFlag(map, "ClassUnloading"); >>>> ??????? vmOptFinalFlag(map, "ClassUnloadingWithConcurrentMark"); >>>> ??????? vmOptFinalFlag(map, "UseCompressedOops"); >>>> ??????? vmOptFinalFlag(map, "EnableJVMCI"); >>>> ??????? vmOptFinalFlag(map, "EliminateAllocations"); >>>> ??? } >>>> >>>> Thanks, >>>> StefanK >>>> >>>>> >>>>> ? Tests: tested with explicitly set -XX:-UseCompressedOops or >>>>> -XX:-UseCompressedClassPointers or both in jtreg vmoptions. >>>>> ? tier1,2,4,7 test pending. >>>>> >>>>> Thanks >>>>> Yumin >>>> >>> >> From nick.gasson at arm.com Thu Mar 19 02:24:06 2020 From: nick.gasson at arm.com (Nick Gasson) Date: Thu, 19 Mar 2020 10:24:06 +0800 Subject: RFR: 8186780: clang fastdebug assertion failure in os_linux_x86:os::verify_stack_alignment() In-Reply-To: References: Message-ID: <85fte53wvt.fsf@arm.com> On 03/17/20 11:46 am, Martin Buchholz wrote: > We finally came up with a solution for this that more than one engineer can > agree on: > > https://cr.openjdk.java.net/~martin/webrevs/jdk/clang-stack-alignment-builtin-frame-address/ > https://bugs.openjdk.java.net/browse/JDK-8186780 We made the same change in the AArch64 port last year without any issues. Thanks, Nick From jianglizhou at google.com Thu Mar 19 02:34:19 2020 From: jianglizhou at google.com (Jiangli Zhou) Date: Wed, 18 Mar 2020 19:34:19 -0700 Subject: RFR: 8186780: clang fastdebug assertion failure in os_linux_x86:os::verify_stack_alignment() In-Reply-To: <85fte53wvt.fsf@arm.com> References: <85fte53wvt.fsf@arm.com> Message-ID: Thanks, Nick! I was going to suggest Martin ask you to help test for AArch64. Best, Jiangli On Wed, Mar 18, 2020 at 7:24 PM Nick Gasson wrote: > > > On 03/17/20 11:46 am, Martin Buchholz wrote: > > We finally came up with a solution for this that more than one engineer can > > agree on: > > > > https://cr.openjdk.java.net/~martin/webrevs/jdk/clang-stack-alignment-builtin-frame-address/ > > https://bugs.openjdk.java.net/browse/JDK-8186780 > > We made the same change in the AArch64 port last year without any > issues. > > > Thanks, > Nick From stefan.karlsson at oracle.com Thu Mar 19 08:12:31 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 19 Mar 2020 09:12:31 +0100 Subject: RFR(XS): 8241009: CommandLineFlagComboNegative.java fails after JDK-8240563 In-Reply-To: <5ee7c5f8-2eb6-4a7c-a412-5934607f4cc3@oracle.com> References: <0d2552b2-155f-278e-2437-848599917792@oracle.com> <387010be-9e36-bde3-9372-de98092b2c89@oracle.com> <8d8d06ca-a788-7b37-6c5e-d05d61df0bd0@oracle.com> <5ee7c5f8-2eb6-4a7c-a412-5934607f4cc3@oracle.com> Message-ID: <1e842d40-3306-2781-1464-9e9f6093467c@oracle.com> On 2020-03-18 23:54, David Holmes wrote: > Hi Yumin, > > On 19/03/2020 7:43 am, Yumin Qi wrote: >> Hi, Stefan and all >> >> ?? (Sorry last email spell your last name first). >> ?? Update using your suggestion. Also remove the checking for 64bit >> code since after the change, checking against 64bit no longer needed. >> >> http://cr.openjdk.java.net/~minqi/8241009/webrev/ > > This doesn't seem ideal but it does seem sufficient. > > Ideally the test would explicitly set all the flags it depends upon > rather than assuming/requiring the flags have a specific value. Hmm. Yes, I might have misunderstood the reason why the filtering was done, and maybe agree with David here. I was trying to look at the original webrev again, but it has been overwritten with the new version. Could you add back the original version, and put the updated webrev in a separate directory? I'm also fine with leaving this as is if we think this is good enough for now. Thanks, StefanK > > Thanks, > David > >> Thanks >> Yumin >> >> On 3/18/20 10:21 AM, Yumin Qi wrote: >>> Hi, Karlsson >>> ? Thanks. that is more concise version. Will update. >>> >>> Yumin >>> >>> On 3/18/20 1:58 AM, Stefan Karlsson wrote: >>>> Hi Yumin, >>>> >>>> On 2020-03-18 00:01, Yumin Qi wrote: >>>>> Hi, please review >>>>> >>>>> ? Bug:https://bugs.openjdk.java.net/browse/JDK-8241009 >>>>> >>>>> ? Webrev: http://cr.openjdk.java.net/~minqi/8241009/webrev/ >>>>> >>>>> ? The vmoptions from jtreg still affect the launched processes and >>>>> mess up the parameters for launched java processes in the test. >>>>> ? Fix skip the test if -XX:-UseCompressedOops or >>>>> -XX:-UseCompressedClassPointers in jtreg vmoptions (or javaoptions). >>>> >>>> We already have a way to use @require to disable tests when >>>> incompatible flags are passed. >>>> >>>> Take a look at: >>>> test/hotspot/jtreg/runtime/CompressedOops/CompressedClassSpaceSize.java >>>> >>>> >>>> ?* @requires vm.bits == 64 & vm.opt.final.UseCompressedOops == true >>>> >>>> Could this be used instead? Note that this is using >>>> vm.opt.final.UseCompressedOops and not vm.opt.UseCompressedOops. >>>> The former actually checks the value of the flag, just like your >>>> adhoc version does. >>>> >>>> The list of supported flags are currently hard-coded, so if you >>>> need a flag that's not already exposed, you'll have to add it >>>> yourself. See: test/jtreg-ext/requires/VMProps.java >>>> >>>> ??? /** >>>> ???? * Selected final flag. >>>> ???? * >>>> ???? * @param map - property-value pairs >>>> ???? * @param flagName - flag name >>>> ???? */ >>>> ??? private void vmOptFinalFlag(SafeMap map, String flagName) { >>>> ??????? map.put("vm.opt.final." + flagName, >>>> ??????????????? () -> String.valueOf(WB.getBooleanVMFlag(flagName))); >>>> ??? } >>>> >>>> ??? /** >>>> ???? * Selected sets of final flags. >>>> ???? * >>>> ???? * @param map - property-value pairs >>>> ???? */ >>>> ??? protected void vmOptFinalFlags(SafeMap map) { >>>> ??????? vmOptFinalFlag(map, "ClassUnloading"); >>>> ??????? vmOptFinalFlag(map, "ClassUnloadingWithConcurrentMark"); >>>> ??????? vmOptFinalFlag(map, "UseCompressedOops"); >>>> ??????? vmOptFinalFlag(map, "EnableJVMCI"); >>>> ??????? vmOptFinalFlag(map, "EliminateAllocations"); >>>> ??? } >>>> >>>> Thanks, >>>> StefanK >>>> >>>>> >>>>> ? Tests: tested with explicitly set -XX:-UseCompressedOops or >>>>> -XX:-UseCompressedClassPointers or both in jtreg vmoptions. >>>>> ? tier1,2,4,7 test pending. >>>>> >>>>> Thanks >>>>> Yumin >>>> >>> >> From tobias.hartmann at oracle.com Thu Mar 19 08:23:56 2020 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Thu, 19 Mar 2020 09:23:56 +0100 Subject: RFR: 8241095: x86: Improve prefix handling in Assembler In-Reply-To: References: <40d97c7c-9d94-7ba1-ab97-da80e424100c@oracle.com> Message-ID: <7e1ce826-f303-fefa-6798-ed37e1ada9cb@oracle.com> On 18.03.20 20:30, Vladimir Kozlov wrote: > Looks good. +1 Best regards, Tobias From yumin.qi at oracle.com Thu Mar 19 16:59:20 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Thu, 19 Mar 2020 09:59:20 -0700 Subject: RFR(XS): 8241009: CommandLineFlagComboNegative.java fails after JDK-8240563 In-Reply-To: <1e842d40-3306-2781-1464-9e9f6093467c@oracle.com> References: <0d2552b2-155f-278e-2437-848599917792@oracle.com> <387010be-9e36-bde3-9372-de98092b2c89@oracle.com> <8d8d06ca-a788-7b37-6c5e-d05d61df0bd0@oracle.com> <5ee7c5f8-2eb6-4a7c-a412-5934607f4cc3@oracle.com> <1e842d40-3306-2781-1464-9e9f6093467c@oracle.com> Message-ID: <0a903ae6-39d0-811d-5d9e-b8dc2ca6a18e@oracle.com> HI, Stefan ? Thanks. I upload the first version at: http://cr.openjdk.java.net/~minqi/8241009/webrev-00/ ? if @requires condition failed, the test won't be selected and result won't be listed. With SkippedException, the result shows what reason the test skipped. ? Thanks ? Yumin On 3/19/20 1:12 AM, Stefan Karlsson wrote: > On 2020-03-18 23:54, David Holmes wrote: >> Hi Yumin, >> >> On 19/03/2020 7:43 am, Yumin Qi wrote: >>> Hi, Stefan and all >>> >>> ?? (Sorry last email spell your last name first). >>> ?? Update using your suggestion. Also remove the checking for 64bit >>> code since after the change, checking against 64bit no longer needed. >>> >>> http://cr.openjdk.java.net/~minqi/8241009/webrev/ >> >> This doesn't seem ideal but it does seem sufficient. >> >> Ideally the test would explicitly set all the flags it depends upon >> rather than assuming/requiring the flags have a specific value. > > Hmm. Yes, I might have misunderstood the reason why the filtering was > done, and maybe agree with David here. I was trying to look at the > original webrev again, but it has been overwritten with the new > version. Could you add back the original version, and put the updated > webrev in a separate directory? > > I'm also fine with leaving this as is if we think this is good enough > for now. > > Thanks, > StefanK > >> >> Thanks, >> David >> >>> Thanks >>> Yumin >>> >>> On 3/18/20 10:21 AM, Yumin Qi wrote: >>>> Hi, Karlsson >>>> ? Thanks. that is more concise version. Will update. >>>> >>>> Yumin >>>> >>>> On 3/18/20 1:58 AM, Stefan Karlsson wrote: >>>>> Hi Yumin, >>>>> >>>>> On 2020-03-18 00:01, Yumin Qi wrote: >>>>>> Hi, please review >>>>>> >>>>>> ? Bug:https://bugs.openjdk.java.net/browse/JDK-8241009 >>>>>> >>>>>> ? Webrev: http://cr.openjdk.java.net/~minqi/8241009/webrev/ >>>>>> >>>>>> ? The vmoptions from jtreg still affect the launched processes >>>>>> and mess up the parameters for launched java processes in the test. >>>>>> ? Fix skip the test if -XX:-UseCompressedOops or >>>>>> -XX:-UseCompressedClassPointers in jtreg vmoptions (or javaoptions). >>>>> >>>>> We already have a way to use @require to disable tests when >>>>> incompatible flags are passed. >>>>> >>>>> Take a look at: >>>>> test/hotspot/jtreg/runtime/CompressedOops/CompressedClassSpaceSize.java >>>>> >>>>> >>>>> ?* @requires vm.bits == 64 & vm.opt.final.UseCompressedOops == true >>>>> >>>>> Could this be used instead? Note that this is using >>>>> vm.opt.final.UseCompressedOops and not vm.opt.UseCompressedOops. >>>>> The former actually checks the value of the flag, just like your >>>>> adhoc version does. >>>>> >>>>> The list of supported flags are currently hard-coded, so if you >>>>> need a flag that's not already exposed, you'll have to add it >>>>> yourself. See: test/jtreg-ext/requires/VMProps.java >>>>> >>>>> ??? /** >>>>> ???? * Selected final flag. >>>>> ???? * >>>>> ???? * @param map - property-value pairs >>>>> ???? * @param flagName - flag name >>>>> ???? */ >>>>> ??? private void vmOptFinalFlag(SafeMap map, String flagName) { >>>>> ??????? map.put("vm.opt.final." + flagName, >>>>> ??????????????? () -> String.valueOf(WB.getBooleanVMFlag(flagName))); >>>>> ??? } >>>>> >>>>> ??? /** >>>>> ???? * Selected sets of final flags. >>>>> ???? * >>>>> ???? * @param map - property-value pairs >>>>> ???? */ >>>>> ??? protected void vmOptFinalFlags(SafeMap map) { >>>>> ??????? vmOptFinalFlag(map, "ClassUnloading"); >>>>> ??????? vmOptFinalFlag(map, "ClassUnloadingWithConcurrentMark"); >>>>> ??????? vmOptFinalFlag(map, "UseCompressedOops"); >>>>> ??????? vmOptFinalFlag(map, "EnableJVMCI"); >>>>> ??????? vmOptFinalFlag(map, "EliminateAllocations"); >>>>> ??? } >>>>> >>>>> Thanks, >>>>> StefanK >>>>> >>>>>> >>>>>> ? Tests: tested with explicitly set -XX:-UseCompressedOops or >>>>>> -XX:-UseCompressedClassPointers or both in jtreg vmoptions. >>>>>> ? tier1,2,4,7 test pending. >>>>>> >>>>>> Thanks >>>>>> Yumin >>>>> >>>> >>> > From yudi.zheng at oracle.com Fri Mar 20 15:03:28 2020 From: yudi.zheng at oracle.com (Yudi Zheng) Date: Fri, 20 Mar 2020 16:03:28 +0100 Subject: RFR: 8241234: Unify monitor enter/exit runtime entries Message-ID: <222D2846-F6AE-4D5B-B41F-F976D90E329C@oracle.com> Hello, Please review this patch that unifies the monitor enter/exit runtime entries used by C1, C2, and JVMCI. https://bugs.openjdk.java.net/browse/JDK-8061553 added support for faster "slow" locking paths. These paths are not currently used by JVMCI (or C1) leading to slower performance on programs that have a lot of short lived locked sections (such as dacapo:h2). This patch creates helper methods based on C2?s monitor enter/exit entries, and shares with the C1, JVMCI runtimes. It will prevent these runtimes from missing future updates. https://bugs.openjdk.java.net/browse/JDK-8241234 http://cr.openjdk.java.net/~yzheng/8241234/webrev.00/ Many thanks, Yudi From coleen.phillimore at oracle.com Fri Mar 20 15:35:00 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 20 Mar 2020 11:35:00 -0400 Subject: RFR: 8241234: Unify monitor enter/exit runtime entries In-Reply-To: <222D2846-F6AE-4D5B-B41F-F976D90E329C@oracle.com> References: <222D2846-F6AE-4D5B-B41F-F976D90E329C@oracle.com> Message-ID: <91eeada8-e05f-bc73-b029-94e169216a56@oracle.com> This seems like a good thing to do: http://cr.openjdk.java.net/~yzheng/8241234/webrev.00/src/hotspot/share/runtime/sharedRuntime.cpp.udiff.html I think you should have monitor_enter_helper and maybe monitor_exi_helper both take Handle as arguments.? This looks very strange. +void SharedRuntime::monitor_enter_helper(oopDesc* _obj, BasicLock* lock, JavaThread* thread) { Because it creates a Handle anyway.? We try to eagerly pass Handle so that we don't have to search through the code to make sure there aren't any unhandled oops. The _obj parameter name doesn't follow coding conventions (I was looking for it to be a field in something). + oop obj(_obj); Is there not already a implicit conversion from oopDesc* to oop? Thanks, Coleen On 3/20/20 11:03 AM, Yudi Zheng wrote: > Hello, > > Please review this patch that unifies the monitor enter/exit runtime entries used by C1, C2, and JVMCI. > > https://bugs.openjdk.java.net/browse/JDK-8061553 added support for faster "slow" locking paths. > These paths are not currently used by JVMCI (or C1) leading to slower performance on programs > that have a lot of short lived locked sections (such as dacapo:h2). > > This patch creates helper methods based on C2?s monitor enter/exit entries, and shares with the C1, > JVMCI runtimes. It will prevent these runtimes from missing future updates. > > https://bugs.openjdk.java.net/browse/JDK-8241234 > http://cr.openjdk.java.net/~yzheng/8241234/webrev.00/ > > Many thanks, > Yudi From vladimir.kozlov at oracle.com Fri Mar 20 17:01:45 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 20 Mar 2020 10:01:45 -0700 Subject: [15] RFR(M) 8237497: vmStructs_jvmci.cpp does not check that the correct field type is specified Message-ID: <64e0f4db-7e5f-039a-84a7-b014d1a523f2@oracle.com> http://cr.openjdk.java.net/~kvn/8237497/webrev.00/ https://bugs.openjdk.java.net/browse/JDK-8237497 JVMCI missed checks for declarations in vmStructs_jvmci.cpp. I copied check logic from vmStruct to fix that. It will be called during JVMCI initialization in debug build (as for vmStruct). I can't call it from vmStructs_init() because vmStruct code is guarded by INCLUDE_VM_STRUCTS variable which can be switched off with configure option --with-jvm-features=-vm-structs. On other hand JVMCIVMStruct have to be initialized always when JVMCI is used - it is how Graal accesses HotSpot's internal data. The fix found several (4) cases of incorrect types in JVMCI declarations. 2 of them require update Graal code as well which I will upstream later. I also did cleanup in both, vmStructs_jvmci and vmStructs. In vmStructs_jvmci I removed macros which are not used (we can add them back if they needed). In vmStructs I moved all check code under #ifdef ASSERT because it is called only in debug build. I removed VMStructs::test() declaration which we forgot to remove when its body was removed in JDK-8171090. I removed duplicated lines which existed from "day one" (before Mercurial). I did local build and testing with and without --with-jvm-features=-vm-structs to make sure everything (inluding Graal) worked. I ran tier1 (which runs HotSpot gtests), hs-tier2, hs-tier3-graal. Thanks, Vladimir From vladimir.kozlov at oracle.com Fri Mar 20 19:32:32 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 20 Mar 2020 12:32:32 -0700 Subject: [15] RFR(M) 8237497: vmStructs_jvmci.cpp does not check that the correct field type is specified In-Reply-To: <5a40a0ed-7e3f-72a2-e707-09bd21dd9b6f@oracle.com> References: <64e0f4db-7e5f-039a-84a7-b014d1a523f2@oracle.com> <5a40a0ed-7e3f-72a2-e707-09bd21dd9b6f@oracle.com> Message-ID: <62546416-20fa-c4bc-46ce-192eb19e18f7@oracle.com> Thank you, Ioi, for looking on changes. On 3/20/20 11:56 AM, Ioi Lam wrote: > Hi Vladimir, > > Why do you need jvmci_vmStructs_init()? Is it possible to call JVMCIVMStructs::init() directly? For that I have to #include vmStructs_jvmci.hpp which will pull other headers as well. Which would increase parsing time of this C++ code even for product build. And I followed what we do for vmStruct - it calls vmStructs_init(). > > Also, I think you can change the declaration as > > class JVMCIVMStructs { > ?? void init() NOT_DEBUG_RETURN; > } > > So the caller can avoid using DEBUG_ONLY(). I don't like hiding that a called code is executed only in debug more. And why do we need to keep empty methods in product builds? It takes a lot of time to compile HotSpot VM. > > The rest looks good to me. Thank you, Vladimir K > > Thanks > - Ioi > > On 3/20/20 10:01 AM, Vladimir Kozlov wrote: >> http://cr.openjdk.java.net/~kvn/8237497/webrev.00/ >> https://bugs.openjdk.java.net/browse/JDK-8237497 >> >> JVMCI missed checks for declarations in vmStructs_jvmci.cpp. I copied check logic from vmStruct to fix that. It will >> be called during JVMCI initialization in debug build (as for vmStruct). I can't call it from vmStructs_init() because >> vmStruct code is guarded by INCLUDE_VM_STRUCTS variable which can be switched off with configure option >> --with-jvm-features=-vm-structs. On other hand JVMCIVMStruct have to be initialized always when JVMCI is used - it is >> how Graal accesses HotSpot's internal data. >> >> The fix found several (4) cases of incorrect types in JVMCI declarations. 2 of them require update Graal code as well >> which I will upstream later. >> >> I also did cleanup in both, vmStructs_jvmci and vmStructs. >> In vmStructs_jvmci I removed macros which are not used (we can add them back if they needed). >> In vmStructs I moved all check code under #ifdef ASSERT because it is called only in debug build. >> I removed VMStructs::test() declaration which we forgot to remove when its body was removed in JDK-8171090. >> I removed duplicated lines which existed from "day one" (before Mercurial). >> >> I did local build and testing with and without --with-jvm-features=-vm-structs to make sure everything (inluding >> Graal) worked. I ran tier1 (which runs HotSpot gtests), hs-tier2, hs-tier3-graal. >> >> Thanks, >> Vladimir > From ioi.lam at oracle.com Fri Mar 20 20:00:55 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 20 Mar 2020 13:00:55 -0700 Subject: [15] RFR(M) 8237497: vmStructs_jvmci.cpp does not check that the correct field type is specified In-Reply-To: <62546416-20fa-c4bc-46ce-192eb19e18f7@oracle.com> References: <64e0f4db-7e5f-039a-84a7-b014d1a523f2@oracle.com> <5a40a0ed-7e3f-72a2-e707-09bd21dd9b6f@oracle.com> <62546416-20fa-c4bc-46ce-192eb19e18f7@oracle.com> Message-ID: <9dcb2133-072f-05d7-d2ed-dee7327143c3@oracle.com> On 3/20/20 12:32 PM, Vladimir Kozlov wrote: > Thank you, Ioi, for looking on changes. > > On 3/20/20 11:56 AM, Ioi Lam wrote: >> Hi Vladimir, >> >> Why do you need jvmci_vmStructs_init()? Is it possible to call >> JVMCIVMStructs::init() directly? > > For that I have to #include vmStructs_jvmci.hpp which will pull other > headers as well. Which would increase parsing time of this C++ code > even for product build. And I followed what we do for vmStruct - it > calls vmStructs_init(). > OK. >> >> Also, I think you can change the declaration as >> >> class JVMCIVMStructs { >> ??? void init() NOT_DEBUG_RETURN; >> } >> >> So the caller can avoid using DEBUG_ONLY(). > > I don't like hiding that a called code is executed only in debug more. > And why do we need to keep empty methods in product builds? > The "XXX_RETURN;" pattern is widely used. E.g., class nmethod : public CompiledMethod { ? void print_dependencies()?????????????????????? PRODUCT_RETURN; } These functions are all inlined and elided by the C++ compiler. They don't take up space in the product build. $ nm -S debug/images/jdk/lib/server/libjvm.so | c++filt | grep nmethod::print_dependencies 000000000173fdb2 000000000000017f t nmethod::print_dependencies() $ nm -S product/images/jdk/lib/server/libjvm.so | c++filt | grep nmethod::print_dependencies | wc ????? 0?????? 0?????? 0 We only use this style of declaration for class member functions, though. I believe it's not strictly compliant to have a stray ";" in the global scope until some newer version of C++ that we have not adopted yet. Thanks - Ioi > It takes a lot of time to compile HotSpot VM. > >> >> The rest looks good to me. > > Thank you, > Vladimir K > >> >> Thanks >> - Ioi >> >> On 3/20/20 10:01 AM, Vladimir Kozlov wrote: >>> http://cr.openjdk.java.net/~kvn/8237497/webrev.00/ >>> https://bugs.openjdk.java.net/browse/JDK-8237497 >>> >>> JVMCI missed checks for declarations in vmStructs_jvmci.cpp. I >>> copied check logic from vmStruct to fix that. It will be called >>> during JVMCI initialization in debug build (as for vmStruct). I >>> can't call it from vmStructs_init() because vmStruct code is guarded >>> by INCLUDE_VM_STRUCTS variable which can be switched off with >>> configure option --with-jvm-features=-vm-structs. On other hand >>> JVMCIVMStruct have to be initialized always when JVMCI is used - it >>> is how Graal accesses HotSpot's internal data. >>> >>> The fix found several (4) cases of incorrect types in JVMCI >>> declarations. 2 of them require update Graal code as well which I >>> will upstream later. >>> >>> I also did cleanup in both, vmStructs_jvmci and vmStructs. >>> In vmStructs_jvmci I removed macros which are not used (we can add >>> them back if they needed). >>> In vmStructs I moved all check code under #ifdef ASSERT because it >>> is called only in debug build. >>> I removed VMStructs::test() declaration which we forgot to remove >>> when its body was removed in JDK-8171090. >>> I removed duplicated lines which existed from "day one" (before >>> Mercurial). >>> >>> I did local build and testing with and without >>> --with-jvm-features=-vm-structs to make sure everything (inluding >>> Graal) worked. I ran tier1 (which runs HotSpot gtests), hs-tier2, >>> hs-tier3-graal. >>> >>> Thanks, >>> Vladimir >> From vladimir.kozlov at oracle.com Sat Mar 21 00:47:16 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 20 Mar 2020 17:47:16 -0700 Subject: [15] RFR(M) 8237497: vmStructs_jvmci.cpp does not check that the correct field type is specified In-Reply-To: <9dcb2133-072f-05d7-d2ed-dee7327143c3@oracle.com> References: <64e0f4db-7e5f-039a-84a7-b014d1a523f2@oracle.com> <5a40a0ed-7e3f-72a2-e707-09bd21dd9b6f@oracle.com> <62546416-20fa-c4bc-46ce-192eb19e18f7@oracle.com> <9dcb2133-072f-05d7-d2ed-dee7327143c3@oracle.com> Message-ID: Okay, as you suggested. Methods JVMCIVMStructs::init() and jvmci_vmStructs_init() declaration in jvmci.cpp use NOT_DEBUG_RETURN. The same for vmStructs_init() in init.cpp. I also added NOT_VM_STRUCTS_RETURN_(0) to methods in VMStructs because vmStructs.cpp file is excluded from compilation when vm-structs is excluded in configure. http://cr.openjdk.java.net/~kvn/8237497/webrev.01/ Thanks, Vladimir On 3/20/20 1:00 PM, Ioi Lam wrote: > > > On 3/20/20 12:32 PM, Vladimir Kozlov wrote: >> Thank you, Ioi, for looking on changes. >> >> On 3/20/20 11:56 AM, Ioi Lam wrote: >>> Hi Vladimir, >>> >>> Why do you need jvmci_vmStructs_init()? Is it possible to call JVMCIVMStructs::init() directly? >> >> For that I have to #include vmStructs_jvmci.hpp which will pull other headers as well. Which would increase parsing >> time of this C++ code even for product build. And I followed what we do for vmStruct - it calls vmStructs_init(). >> > > OK. > >>> >>> Also, I think you can change the declaration as >>> >>> class JVMCIVMStructs { >>> ??? void init() NOT_DEBUG_RETURN; >>> } >>> >>> So the caller can avoid using DEBUG_ONLY(). >> >> I don't like hiding that a called code is executed only in debug more. >> And why do we need to keep empty methods in product builds? >> > The "XXX_RETURN;" pattern is widely used. E.g., > > class nmethod : public CompiledMethod { > ? void print_dependencies()?????????????????????? PRODUCT_RETURN; > } > > These functions are all inlined and elided by the C++ compiler. They don't take up space in the product build. > > $ nm -S debug/images/jdk/lib/server/libjvm.so | c++filt | grep nmethod::print_dependencies > 000000000173fdb2 000000000000017f t nmethod::print_dependencies() > > $ nm -S product/images/jdk/lib/server/libjvm.so | c++filt | grep nmethod::print_dependencies | wc > ????? 0?????? 0?????? 0 > > We only use this style of declaration for class member functions, though. I believe it's not strictly compliant to have > a stray ";" in the global scope until some newer version of C++ that we have not adopted yet. > > Thanks > - Ioi > >> It takes a lot of time to compile HotSpot VM. >> >>> >>> The rest looks good to me. >> >> Thank you, >> Vladimir K >> >>> >>> Thanks >>> - Ioi >>> >>> On 3/20/20 10:01 AM, Vladimir Kozlov wrote: >>>> http://cr.openjdk.java.net/~kvn/8237497/webrev.00/ >>>> https://bugs.openjdk.java.net/browse/JDK-8237497 >>>> >>>> JVMCI missed checks for declarations in vmStructs_jvmci.cpp. I copied check logic from vmStruct to fix that. It will >>>> be called during JVMCI initialization in debug build (as for vmStruct). I can't call it from vmStructs_init() >>>> because vmStruct code is guarded by INCLUDE_VM_STRUCTS variable which can be switched off with configure option >>>> --with-jvm-features=-vm-structs. On other hand JVMCIVMStruct have to be initialized always when JVMCI is used - it >>>> is how Graal accesses HotSpot's internal data. >>>> >>>> The fix found several (4) cases of incorrect types in JVMCI declarations. 2 of them require update Graal code as >>>> well which I will upstream later. >>>> >>>> I also did cleanup in both, vmStructs_jvmci and vmStructs. >>>> In vmStructs_jvmci I removed macros which are not used (we can add them back if they needed). >>>> In vmStructs I moved all check code under #ifdef ASSERT because it is called only in debug build. >>>> I removed VMStructs::test() declaration which we forgot to remove when its body was removed in JDK-8171090. >>>> I removed duplicated lines which existed from "day one" (before Mercurial). >>>> >>>> I did local build and testing with and without --with-jvm-features=-vm-structs to make sure everything (inluding >>>> Graal) worked. I ran tier1 (which runs HotSpot gtests), hs-tier2, hs-tier3-graal. >>>> >>>> Thanks, >>>> Vladimir >>> > From thomas.stuefe at gmail.com Sat Mar 21 12:40:48 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 21 Mar 2020 13:40:48 +0100 Subject: RFR(xxs): 8241395: Factor out platform independent code for os::xxx_memory_special() Message-ID: Hi, may I please have reviews for this little cleanup. A while ago we factored out platform independent behavior for os::reserve_memory() and friends to os.cpp and relegated platform dependent code to os::pd_reserve_memory() etc. This change does the same for os::reserve_memory_special() and os::release_memory_special(). Only Linux and Windows are affected since only they have meaningful implementations for UseLargePages. It also adds a small comment about the NMT Tracker object. In addition it asserts - now for all platforms, it used to do this only on Linux - that if both requested address and alignment are given to os::reserve_memory_special() the requested address should at least be aligned to the given alignment (the arguments should be mutually exclusive and I plan to clean this up further). Issue: https://bugs.openjdk.java.net/browse/JDK-8241395 webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8241395-refactor-reserve-memory-special/webrev.00/webrev/ Thanks, Thomas From ioi.lam at oracle.com Sat Mar 21 18:20:59 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Sat, 21 Mar 2020 11:20:59 -0700 Subject: [15] RFR(M) 8237497: vmStructs_jvmci.cpp does not check that the correct field type is specified In-Reply-To: References: <64e0f4db-7e5f-039a-84a7-b014d1a523f2@oracle.com> <5a40a0ed-7e3f-72a2-e707-09bd21dd9b6f@oracle.com> <62546416-20fa-c4bc-46ce-192eb19e18f7@oracle.com> <9dcb2133-072f-05d7-d2ed-dee7327143c3@oracle.com> Message-ID: Looks good. Thanks - Ioi On 3/20/20 5:47 PM, Vladimir Kozlov wrote: > Okay, as you suggested. > > Methods JVMCIVMStructs::init() and jvmci_vmStructs_init() declaration > in jvmci.cpp use NOT_DEBUG_RETURN. > The same for vmStructs_init() in init.cpp. I also added > NOT_VM_STRUCTS_RETURN_(0) to methods in VMStructs because > vmStructs.cpp file is excluded from compilation when vm-structs is > excluded in configure. > > http://cr.openjdk.java.net/~kvn/8237497/webrev.01/ > > Thanks, > Vladimir > > On 3/20/20 1:00 PM, Ioi Lam wrote: >> >> >> On 3/20/20 12:32 PM, Vladimir Kozlov wrote: >>> Thank you, Ioi, for looking on changes. >>> >>> On 3/20/20 11:56 AM, Ioi Lam wrote: >>>> Hi Vladimir, >>>> >>>> Why do you need jvmci_vmStructs_init()? Is it possible to call >>>> JVMCIVMStructs::init() directly? >>> >>> For that I have to #include vmStructs_jvmci.hpp which will pull >>> other headers as well. Which would increase parsing time of this C++ >>> code even for product build. And I followed what we do for vmStruct >>> - it calls vmStructs_init(). >>> >> >> OK. >> >>>> >>>> Also, I think you can change the declaration as >>>> >>>> class JVMCIVMStructs { >>>> ??? void init() NOT_DEBUG_RETURN; >>>> } >>>> >>>> So the caller can avoid using DEBUG_ONLY(). >>> >>> I don't like hiding that a called code is executed only in debug more. >>> And why do we need to keep empty methods in product builds? >>> >> The "XXX_RETURN;" pattern is widely used. E.g., >> >> class nmethod : public CompiledMethod { >> ?? void print_dependencies() PRODUCT_RETURN; >> } >> >> These functions are all inlined and elided by the C++ compiler. They >> don't take up space in the product build. >> >> $ nm -S debug/images/jdk/lib/server/libjvm.so | c++filt | grep >> nmethod::print_dependencies >> 000000000173fdb2 000000000000017f t nmethod::print_dependencies() >> >> $ nm -S product/images/jdk/lib/server/libjvm.so | c++filt | grep >> nmethod::print_dependencies | wc >> ?????? 0?????? 0?????? 0 >> >> We only use this style of declaration for class member functions, >> though. I believe it's not strictly compliant to have a stray ";" in >> the global scope until some newer version of C++ that we have not >> adopted yet. >> >> Thanks >> - Ioi >> >>> It takes a lot of time to compile HotSpot VM. >>> >>>> >>>> The rest looks good to me. >>> >>> Thank you, >>> Vladimir K >>> >>>> >>>> Thanks >>>> - Ioi >>>> >>>> On 3/20/20 10:01 AM, Vladimir Kozlov wrote: >>>>> http://cr.openjdk.java.net/~kvn/8237497/webrev.00/ >>>>> https://bugs.openjdk.java.net/browse/JDK-8237497 >>>>> >>>>> JVMCI missed checks for declarations in vmStructs_jvmci.cpp. I >>>>> copied check logic from vmStruct to fix that. It will be called >>>>> during JVMCI initialization in debug build (as for vmStruct). I >>>>> can't call it from vmStructs_init() because vmStruct code is >>>>> guarded by INCLUDE_VM_STRUCTS variable which can be switched off >>>>> with configure option --with-jvm-features=-vm-structs. On other >>>>> hand JVMCIVMStruct have to be initialized always when JVMCI is >>>>> used - it is how Graal accesses HotSpot's internal data. >>>>> >>>>> The fix found several (4) cases of incorrect types in JVMCI >>>>> declarations. 2 of them require update Graal code as well which I >>>>> will upstream later. >>>>> >>>>> I also did cleanup in both, vmStructs_jvmci and vmStructs. >>>>> In vmStructs_jvmci I removed macros which are not used (we can add >>>>> them back if they needed). >>>>> In vmStructs I moved all check code under #ifdef ASSERT because it >>>>> is called only in debug build. >>>>> I removed VMStructs::test() declaration which we forgot to remove >>>>> when its body was removed in JDK-8171090. >>>>> I removed duplicated lines which existed from "day one" (before >>>>> Mercurial). >>>>> >>>>> I did local build and testing with and without >>>>> --with-jvm-features=-vm-structs to make sure everything (inluding >>>>> Graal) worked. I ran tier1 (which runs HotSpot gtests), hs-tier2, >>>>> hs-tier3-graal. >>>>> >>>>> Thanks, >>>>> Vladimir >>>> >> From claes.redestad at oracle.com Sun Mar 22 16:15:15 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Sun, 22 Mar 2020 17:15:15 +0100 Subject: RFR: 8241371: Refactor and consolidate package_from_name Message-ID: <828623c1-177c-16ef-281c-1a1cec4ecef7@oracle.com> Hi, JDK-8153858 partially consolidated the logic shared between InstanceKlass::package_from_name and ClassLoader::package_from_name. Unfortunately it also caused an undetected regression in some paths due need to copy from Symbol -> resource arrays and back again. By refactoring package_from_name into a single implementation which operates on a Symbol* rather than a char* we can both simplify a bit and regain some lost performance: http://cr.openjdk.java.net/~redestad/scratch/pkg_lookups.00/ Testing: tier1-5, verified a ~0.5% improvement on startup Thanks! /Claes From vladimir.kozlov at oracle.com Sun Mar 22 16:55:09 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Sun, 22 Mar 2020 09:55:09 -0700 Subject: [15] RFR(M) 8237497: vmStructs_jvmci.cpp does not check that the correct field type is specified In-Reply-To: References: Message-ID: Thank you, Ioi Thanks Vladimir > On Mar 21, 2020, at 11:21 AM, Ioi Lam wrote: > > ?Looks good. > > Thanks > - Ioi > >> On 3/20/20 5:47 PM, Vladimir Kozlov wrote: >> Okay, as you suggested. >> >> Methods JVMCIVMStructs::init() and jvmci_vmStructs_init() declaration in jvmci.cpp use NOT_DEBUG_RETURN. >> The same for vmStructs_init() in init.cpp. I also added NOT_VM_STRUCTS_RETURN_(0) to methods in VMStructs because vmStructs.cpp file is excluded from compilation when vm-structs is excluded in configure. >> >> http://cr.openjdk.java.net/~kvn/8237497/webrev.01/ >> >> Thanks, >> Vladimir >> >>> On 3/20/20 1:00 PM, Ioi Lam wrote: >>> >>> >>> On 3/20/20 12:32 PM, Vladimir Kozlov wrote: >>>> Thank you, Ioi, for looking on changes. >>>> >>>> On 3/20/20 11:56 AM, Ioi Lam wrote: >>>>> Hi Vladimir, >>>>> >>>>> Why do you need jvmci_vmStructs_init()? Is it possible to call JVMCIVMStructs::init() directly? >>>> >>>> For that I have to #include vmStructs_jvmci.hpp which will pull other headers as well. Which would increase parsing time of this C++ code even for product build. And I followed what we do for vmStruct - it calls vmStructs_init(). >>>> >>> >>> OK. >>> >>>>> >>>>> Also, I think you can change the declaration as >>>>> >>>>> class JVMCIVMStructs { >>>>> void init() NOT_DEBUG_RETURN; >>>>> } >>>>> >>>>> So the caller can avoid using DEBUG_ONLY(). >>>> >>>> I don't like hiding that a called code is executed only in debug more. >>>> And why do we need to keep empty methods in product builds? >>>> >>> The "XXX_RETURN;" pattern is widely used. E.g., >>> >>> class nmethod : public CompiledMethod { >>> void print_dependencies() PRODUCT_RETURN; >>> } >>> >>> These functions are all inlined and elided by the C++ compiler. They don't take up space in the product build. >>> >>> $ nm -S debug/images/jdk/lib/server/libjvm.so | c++filt | grep nmethod::print_dependencies >>> 000000000173fdb2 000000000000017f t nmethod::print_dependencies() >>> >>> $ nm -S product/images/jdk/lib/server/libjvm.so | c++filt | grep nmethod::print_dependencies | wc >>> 0 0 0 >>> >>> We only use this style of declaration for class member functions, though. I believe it's not strictly compliant to have a stray ";" in the global scope until some newer version of C++ that we have not adopted yet. >>> >>> Thanks >>> - Ioi >>> >>>> It takes a lot of time to compile HotSpot VM. >>>> >>>>> >>>>> The rest looks good to me. >>>> >>>> Thank you, >>>> Vladimir K >>>> >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> On 3/20/20 10:01 AM, Vladimir Kozlov wrote: >>>>>> http://cr.openjdk.java.net/~kvn/8237497/webrev.00/ >>>>>> https://bugs.openjdk.java.net/browse/JDK-8237497 >>>>>> >>>>>> JVMCI missed checks for declarations in vmStructs_jvmci.cpp. I copied check logic from vmStruct to fix that. It will be called during JVMCI initialization in debug build (as for vmStruct). I can't call it from vmStructs_init() because vmStruct code is guarded by INCLUDE_VM_STRUCTS variable which can be switched off with configure option --with-jvm-features=-vm-structs. On other hand JVMCIVMStruct have to be initialized always when JVMCI is used - it is how Graal accesses HotSpot's internal data. >>>>>> >>>>>> The fix found several (4) cases of incorrect types in JVMCI declarations. 2 of them require update Graal code as well which I will upstream later. >>>>>> >>>>>> I also did cleanup in both, vmStructs_jvmci and vmStructs. >>>>>> In vmStructs_jvmci I removed macros which are not used (we can add them back if they needed). >>>>>> In vmStructs I moved all check code under #ifdef ASSERT because it is called only in debug build. >>>>>> I removed VMStructs::test() declaration which we forgot to remove when its body was removed in JDK-8171090. >>>>>> I removed duplicated lines which existed from "day one" (before Mercurial). >>>>>> >>>>>> I did local build and testing with and without --with-jvm-features=-vm-structs to make sure everything (inluding Graal) worked. I ran tier1 (which runs HotSpot gtests), hs-tier2, hs-tier3-graal. >>>>>> >>>>>> Thanks, >>>>>> Vladimir >>>>> >>> > From claes.redestad at oracle.com Sun Mar 22 21:22:40 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Sun, 22 Mar 2020 22:22:40 +0100 Subject: RFR: 8241371: Refactor and consolidate package_from_name In-Reply-To: <828623c1-177c-16ef-281c-1a1cec4ecef7@oracle.com> References: <828623c1-177c-16ef-281c-1a1cec4ecef7@oracle.com> Message-ID: Correct webrev: http://cr.openjdk.java.net/~redestad/8241371/open.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8241371 On 2020-03-22 17:15, Claes Redestad wrote: > Hi, > > JDK-8153858 partially consolidated the logic shared between > InstanceKlass::package_from_name and ClassLoader::package_from_name. > Unfortunately it also caused an undetected regression in some paths due > need to copy from Symbol -> resource arrays and back again. > > By refactoring package_from_name into a single implementation which > operates on a Symbol* rather than a char* we can both simplify a bit > and regain some lost performance: > > Testing: tier1-5, verified a ~0.5% improvement on startup > > Thanks! > > /Claes From ioi.lam at oracle.com Sun Mar 22 22:45:17 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Sun, 22 Mar 2020 15:45:17 -0700 Subject: RFR: 8241371: Refactor and consolidate package_from_name In-Reply-To: References: <828623c1-177c-16ef-281c-1a1cec4ecef7@oracle.com> Message-ID: Hi Claes, This is a good optimization. Looks good to me. Here are some minor nits: [1] ClassLoader::search_module_entries may be called twice by the same caller, ? ? I think it's better to compute the mod_entry in the caller, so you can avoid ??? the pkg_entry lookup twice. [2] For tracking the code changes, I think it's better to move your new version ??? of InstanceKlass::package_from_name() to ClassLoader::package_from_name(). ??? I did a diff of the new version of InstanceKlass::package_from_name() and ??? the old version of ClassLoader::package_from_name(), and I can see that the ? ? logic has not changed (the only changes are how we allocate the buffers). ??? With the current patch, it's hard to see what has changed and difficult to ??? validate its correctness. ??? Also, it's better to rename the function to ??? ClassLoader::package_from_class_name() Thanks - Ioi On 3/22/20 2:22 PM, Claes Redestad wrote: > Correct webrev: http://cr.openjdk.java.net/~redestad/8241371/open.00/ > Bug:??????????? https://bugs.openjdk.java.net/browse/JDK-8241371 > > On 2020-03-22 17:15, Claes Redestad wrote: >> Hi, >> >> JDK-8153858 partially consolidated the logic shared between >> InstanceKlass::package_from_name and ClassLoader::package_from_name. >> Unfortunately it also caused an undetected regression in some paths due >> need to copy from Symbol -> resource arrays and back again. >> >> By refactoring package_from_name into a single implementation which >> operates on a Symbol* rather than a char* we can both simplify a bit >> and regain some lost performance: >> >> Testing: tier1-5, verified a ~0.5% improvement on startup >> >> Thanks! >> >> /Claes From claes.redestad at oracle.com Sun Mar 22 23:02:29 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 23 Mar 2020 00:02:29 +0100 Subject: RFR: 8241371: Refactor and consolidate package_from_name In-Reply-To: References: <828623c1-177c-16ef-281c-1a1cec4ecef7@oracle.com> Message-ID: Hi Ioi, On 2020-03-22 23:45, Ioi Lam wrote: > Hi Claes, > > This is a good optimization. Looks good to me. Here are some minor nits: > > [1] ClassLoader::search_module_entries may be called twice by the same > caller, > ? ? I think it's better to compute the mod_entry in the caller, so you > can avoid > ??? the pkg_entry lookup twice. ClassLoader::search_module_entries would only be called twice if running with --patch-module on an exploded build, which doesn't seem like a case we should bother to optimize for. > > [2] For tracking the code changes, I think it's better to move your new > version > ??? of InstanceKlass::package_from_name() to > ClassLoader::package_from_name(). > > ??? I did a diff of the new version of > InstanceKlass::package_from_name() and > ??? the old version of ClassLoader::package_from_name(), and I can see > that the > ? ? logic has not changed (the only changes are how we allocate the > buffers). > > ??? With the current patch, it's hard to see what has changed and > difficult to > ??? validate its correctness. Ok, I just felt that as a static utility it's more coupled with a class/InstanceKlass than its class loader. If you insist I'll move it to ClassLoader instead. > > ??? Also, it's better to rename the function to > ??? ClassLoader::package_from_class_name() Sure, that sounds better! /Claes > > Thanks > - Ioi > > > > On 3/22/20 2:22 PM, Claes Redestad wrote: >> Correct webrev: http://cr.openjdk.java.net/~redestad/8241371/open.00/ >> Bug:??????????? https://bugs.openjdk.java.net/browse/JDK-8241371 >> >> On 2020-03-22 17:15, Claes Redestad wrote: >>> Hi, >>> >>> JDK-8153858 partially consolidated the logic shared between >>> InstanceKlass::package_from_name and ClassLoader::package_from_name. >>> Unfortunately it also caused an undetected regression in some paths due >>> need to copy from Symbol -> resource arrays and back again. >>> >>> By refactoring package_from_name into a single implementation which >>> operates on a Symbol* rather than a char* we can both simplify a bit >>> and regain some lost performance: >>> >>> Testing: tier1-5, verified a ~0.5% improvement on startup >>> >>> Thanks! >>> >>> /Claes > From ioi.lam at oracle.com Mon Mar 23 03:36:14 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Sun, 22 Mar 2020 20:36:14 -0700 Subject: RFR: 8241371: Refactor and consolidate package_from_name In-Reply-To: References: <828623c1-177c-16ef-281c-1a1cec4ecef7@oracle.com> Message-ID: <9de7ebfd-e234-4312-dc29-451096ff9426@oracle.com> On 3/22/20 4:02 PM, Claes Redestad wrote: > Hi Ioi, > > On 2020-03-22 23:45, Ioi Lam wrote: >> Hi Claes, >> >> This is a good optimization. Looks good to me. Here are some minor nits: >> >> [1] ClassLoader::search_module_entries may be called twice by the >> same caller, >> ?? ? I think it's better to compute the mod_entry in the caller, so >> you can avoid >> ???? the pkg_entry lookup twice. > > ClassLoader::search_module_entries would only be called twice if > running with --patch-module on an exploded build, which doesn't seem > like a case we should bother to optimize for. > > >> >> [2] For tracking the code changes, I think it's better to move your >> new version >> ???? of InstanceKlass::package_from_name() to >> ClassLoader::package_from_name(). >> >> ???? I did a diff of the new version of >> InstanceKlass::package_from_name() and >> ???? the old version of ClassLoader::package_from_name(), and I can >> see that the >> ?? ? logic has not changed (the only changes are how we allocate the >> buffers). >> >> ???? With the current patch, it's hard to see what has changed and >> difficult to >> ???? validate its correctness. > > Ok, I just felt that as a static utility it's more coupled with a > class/InstanceKlass than its class loader. If you insist I'll move it > to ClassLoader instead. > classLoader.cpp also has other methods related to packages, so I think it's a good place for this method, too. Plus, you don't need to move the test cases, so there's less code churn. Thanks - Ioi >> >> ???? Also, it's better to rename the function to >> ???? ClassLoader::package_from_class_name() > > Sure, that sounds better! > > /Claes > >> >> Thanks >> - Ioi >> >> >> >> On 3/22/20 2:22 PM, Claes Redestad wrote: >>> Correct webrev: http://cr.openjdk.java.net/~redestad/8241371/open.00/ >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8241371 >>> >>> On 2020-03-22 17:15, Claes Redestad wrote: >>>> Hi, >>>> >>>> JDK-8153858 partially consolidated the logic shared between >>>> InstanceKlass::package_from_name and ClassLoader::package_from_name. >>>> Unfortunately it also caused an undetected regression in some paths >>>> due >>>> need to copy from Symbol -> resource arrays and back again. >>>> >>>> By refactoring package_from_name into a single implementation which >>>> operates on a Symbol* rather than a char* we can both simplify a bit >>>> and regain some lost performance: >>>> >>>> Testing: tier1-5, verified a ~0.5% improvement on startup >>>> >>>> Thanks! >>>> >>>> /Claes >> From ioi.lam at oracle.com Mon Mar 23 03:49:23 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Sun, 22 Mar 2020 20:49:23 -0700 Subject: RFR(XXS) 8241244 CDS dynamic dump asserts in ArchivePtrBitmapCleaner::do_bit Message-ID: <4c091d2d-b3c8-15f2-26b9-fb50bac47a20@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8241244 http://cr.openjdk.java.net/~iklam/jdk15/8241244-dynamic-cds-redefined-class.v01/ In the failing test case, some classes in the base archive (e.g., java.lang.Throwable) are redefined by JFR before the dynamic archive is dumped. The assert happens when java/lang/InternalError is destined for the dynamic archive, but its vtable contains a redefined method of the Throwable class. This method is neither in the base archive, nor in the dynamic archive, causing the assert to fail. The fix is: if a class (or any of its super-types) has been redefined, do not dump this class into the dynamic CDS archive. Note - this fix is not needed for the static archive dumping (java -Xshare:dump), as class redefinition cannot happen. Thanks - Ioi From yumin.qi at oracle.com Mon Mar 23 04:51:40 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Sun, 22 Mar 2020 21:51:40 -0700 Subject: RFR(XXS) 8241244 CDS dynamic dump asserts in ArchivePtrBitmapCleaner::do_bit In-Reply-To: <4c091d2d-b3c8-15f2-26b9-fb50bac47a20@oracle.com> References: <4c091d2d-b3c8-15f2-26b9-fb50bac47a20@oracle.com> Message-ID: <780aca74-b013-63e1-4814-0795cabde128@oracle.com> Ioi, ?The change looks good to me. Thanks Yumin On 3/22/20 8:49 PM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8241244 > http://cr.openjdk.java.net/~iklam/jdk15/8241244-dynamic-cds-redefined-class.v01/ > > > In the failing test case, some classes in the base archive (e.g., > java.lang.Throwable) are redefined by JFR before the dynamic archive > is dumped. The assert happens when java/lang/InternalError is destined > for the dynamic archive, but its vtable contains a redefined method of > the Throwable class. This method is neither in the base archive, nor > in the dynamic archive, causing the assert to fail. > > The fix is: if a class (or any of its super-types) has been redefined, > do not dump this class into the dynamic CDS archive. > > Note - this fix is not needed for the static archive dumping (java > -Xshare:dump), as class redefinition cannot happen. > > Thanks > - Ioi > > > From tobias.hartmann at oracle.com Mon Mar 23 07:51:42 2020 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Mon, 23 Mar 2020 08:51:42 +0100 Subject: RFR: 8241234: Unify monitor enter/exit runtime entries In-Reply-To: <222D2846-F6AE-4D5B-B41F-F976D90E329C@oracle.com> References: <222D2846-F6AE-4D5B-B41F-F976D90E329C@oracle.com> Message-ID: <7b9bdfbc-98da-dc84-71a6-386397cf1313@oracle.com> Hi Yudi, looks good to me. Best regards, Tobias On 20.03.20 16:03, Yudi Zheng wrote: > Hello, > > Please review this patch that unifies the monitor enter/exit runtime entries used by C1, C2, and JVMCI. > > https://bugs.openjdk.java.net/browse/JDK-8061553 added support for faster "slow" locking paths. > These paths are not currently used by JVMCI (or C1) leading to slower performance on programs > that have a lot of short lived locked sections (such as dacapo:h2). > > This patch creates helper methods based on C2?s monitor enter/exit entries, and shares with the C1, > JVMCI runtimes. It will prevent these runtimes from missing future updates. > > https://bugs.openjdk.java.net/browse/JDK-8241234 > http://cr.openjdk.java.net/~yzheng/8241234/webrev.00/ > > Many thanks, > Yudi > From claes.redestad at oracle.com Mon Mar 23 10:44:59 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 23 Mar 2020 11:44:59 +0100 Subject: RFR: 8241371: Refactor and consolidate package_from_name In-Reply-To: <9de7ebfd-e234-4312-dc29-451096ff9426@oracle.com> References: <828623c1-177c-16ef-281c-1a1cec4ecef7@oracle.com> <9de7ebfd-e234-4312-dc29-451096ff9426@oracle.com> Message-ID: <79369f75-cc8f-3658-873f-42f83e8e6fda@oracle.com> On 2020-03-23 04:36, Ioi Lam wrote: >>> >> >> Ok, I just felt that as a static utility it's more coupled with a >> class/InstanceKlass than its class loader. If you insist I'll move it >> to ClassLoader instead. >> > classLoader.cpp also has other methods related to packages, so I think > it's a good place for this method, too. Plus, you don't need to move the > test cases, so there's less code churn. Ok: http://cr.openjdk.java.net/~redestad/8241371/open.01/ /Claes From claes.redestad at oracle.com Mon Mar 23 11:27:35 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 23 Mar 2020 12:27:35 +0100 Subject: RFR: 8241427: Coarsen locking in Modules::add_module_exports Message-ID: <8121467e-74a3-91dd-faba-0ef3303321cc@oracle.com> Hi, in some places we first lookup a PackageEntry - taking and releasing Module_lock - only to then, after some trivial checks - take the Module_lock again to update some state in the PackageEntry. Coarsening the locking so we only take the mutex once per call brings a small improvement to bootstrap (reduction of ~200-250k instructions) Webrev: http://cr.openjdk.java.net/~redestad/8241427/open.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8241427 Testing: tier1-3 Thanks! /Claes From coleen.phillimore at oracle.com Mon Mar 23 12:53:17 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 23 Mar 2020 08:53:17 -0400 Subject: RFR(XXS) 8241244 CDS dynamic dump asserts in ArchivePtrBitmapCleaner::do_bit In-Reply-To: <4c091d2d-b3c8-15f2-26b9-fb50bac47a20@oracle.com> References: <4c091d2d-b3c8-15f2-26b9-fb50bac47a20@oracle.com> Message-ID: Ioi, This looks good and reasonable. Coleen On 3/22/20 11:49 PM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8241244 > http://cr.openjdk.java.net/~iklam/jdk15/8241244-dynamic-cds-redefined-class.v01/ > > > In the failing test case, some classes in the base archive (e.g., > java.lang.Throwable) are redefined by JFR before the dynamic archive > is dumped. The assert happens when java/lang/InternalError is destined > for the dynamic archive, but its vtable contains a redefined method of > the Throwable class. This method is neither in the base archive, nor > in the dynamic archive, causing the assert to fail. > > The fix is: if a class (or any of its super-types) has been redefined, > do not dump this class into the dynamic CDS archive. > > Note - this fix is not needed for the static archive dumping (java > -Xshare:dump), as class redefinition cannot happen. > > Thanks > - Ioi > > > From lois.foltan at oracle.com Mon Mar 23 13:41:57 2020 From: lois.foltan at oracle.com (Lois Foltan) Date: Mon, 23 Mar 2020 09:41:57 -0400 Subject: RFR: 8241371: Refactor and consolidate package_from_name In-Reply-To: <79369f75-cc8f-3658-873f-42f83e8e6fda@oracle.com> References: <828623c1-177c-16ef-281c-1a1cec4ecef7@oracle.com> <9de7ebfd-e234-4312-dc29-451096ff9426@oracle.com> <79369f75-cc8f-3658-873f-42f83e8e6fda@oracle.com> Message-ID: <7a447393-549a-c44a-0845-86ecdd0cda0d@oracle.com> On 3/23/2020 6:44 AM, Claes Redestad wrote: > On 2020-03-23 04:36, Ioi Lam wrote: >>>> >>> >>> Ok, I just felt that as a static utility it's more coupled with a >>> class/InstanceKlass than its class loader. If you insist I'll move it >>> to ClassLoader instead. >>> >> classLoader.cpp also has other methods related to packages, so I >> think it's a good place for this method, too. Plus, you don't need to >> move the test cases, so there's less code churn. > > Ok: > > http://cr.openjdk.java.net/~redestad/8241371/open.01/ > > /Claes Hi Claes, This looks great, thank you for looking into this improvement!? Few review comments: - classfile/classLoader.cpp line #181: ClassLoader::package_from_class_name(), could we remove all the checks for "if (bad_class_name != NULL)" in favor of an assert ahead of line #182 that just "assert(bad_class_name != NULL, "caller did not set bad_class_name correctly");" line #197: I still wish we could get rid of that do while loop. Would searching for the last '[' be better performance wise?? Like UTF8::strrchr(start, utf_len, JVM_SIGNATURE_ARRAY)? - gtest/runtime/test_classLoader.cpp please include a test that has a multi-dimensional array Thanks! Lois From claes.redestad at oracle.com Mon Mar 23 14:07:21 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 23 Mar 2020 15:07:21 +0100 Subject: RFR: 8241371: Refactor and consolidate package_from_name In-Reply-To: <7a447393-549a-c44a-0845-86ecdd0cda0d@oracle.com> References: <828623c1-177c-16ef-281c-1a1cec4ecef7@oracle.com> <9de7ebfd-e234-4312-dc29-451096ff9426@oracle.com> <79369f75-cc8f-3658-873f-42f83e8e6fda@oracle.com> <7a447393-549a-c44a-0845-86ecdd0cda0d@oracle.com> Message-ID: On 2020-03-23 14:41, Lois Foltan wrote: >> >> >> http://cr.openjdk.java.net/~redestad/8241371/open.01/ >> >> /Claes > > Hi Claes, > > This looks great, thank you for looking into this improvement! Thanks for reviewing, Lois! > Few review comments: > > - classfile/classLoader.cpp > line #181: ClassLoader::package_from_class_name(), could we remove all > the checks for "if (bad_class_name != NULL)" in favor of an assert ahead > of line #182 that just "assert(bad_class_name != NULL, "caller did not > set bad_class_name correctly");" Hmm, passing NULL for bad_class_name is the common case, since only one caller is testing for malformed class names. As a future simplification/micro-optimization I think we could get rid of the first test (if (name == NULL)) and replace that and a few other NULL-checks with asserts since they seem to be impossible. I've gotten that through tier1-3, but decided I did not want to overburden this RFE. > line #197: I still wish we could get rid of that do while loop. Would > searching for the last '[' be better performance wise?? Like > UTF8::strrchr(start, utf_len, JVM_SIGNATURE_ARRAY)? I think that could have some cost since we'll scan the entire string. Perhaps an UTF8::mismatch(start, end, JVM_SIGNATURE_ARRAY) could be added, but the if-do-while construct is nice here since the check for JVM_SIGNATURE_CLASS is moved off the critical path. Either way I think this is out of scope for this RFE, since the do-while loop is pre- existing? > > - gtest/runtime/test_classLoader.cpp > please include a test that has a multi-dimensional array Adding: TEST_VM(ClassLoader, class_multiarray) { bool bad_class_name = false; TempNewSymbol name = SymbolTable::new_symbol("[[package/class"); TempNewSymbol retval = ClassLoader::package_from_class_name(name, &bad_class_name); ASSERT_FALSE(bad_class_name) << "Function set bad_class_name with class array"; ASSERT_TRUE(retval->equals("package")) << "Wrong package for class with leading bracket"; } OK, or do you need a new webrev? /Claes From lois.foltan at oracle.com Mon Mar 23 14:13:02 2020 From: lois.foltan at oracle.com (Lois Foltan) Date: Mon, 23 Mar 2020 10:13:02 -0400 Subject: RFR: 8241371: Refactor and consolidate package_from_name In-Reply-To: References: <828623c1-177c-16ef-281c-1a1cec4ecef7@oracle.com> <9de7ebfd-e234-4312-dc29-451096ff9426@oracle.com> <79369f75-cc8f-3658-873f-42f83e8e6fda@oracle.com> <7a447393-549a-c44a-0845-86ecdd0cda0d@oracle.com> Message-ID: <95248311-86db-8287-ee9a-902774642daa@oracle.com> On 3/23/2020 10:07 AM, Claes Redestad wrote: > > > On 2020-03-23 14:41, Lois Foltan wrote: >>> >>> >>> http://cr.openjdk.java.net/~redestad/8241371/open.01/ >>> >>> /Claes >> >> Hi Claes, >> >> This looks great, thank you for looking into this improvement! > > Thanks for reviewing, Lois! > >> Few review comments: >> >> - classfile/classLoader.cpp >> line #181: ClassLoader::package_from_class_name(), could we remove >> all the checks for "if (bad_class_name != NULL)" in favor of an >> assert ahead of line #182 that just "assert(bad_class_name != NULL, >> "caller did not set bad_class_name correctly");" > > Hmm, passing NULL for bad_class_name is the common case, since only one > caller is testing for malformed class names. > > As a future simplification/micro-optimization I think we could get rid > of the first test (if (name == NULL)) and replace that and a few other > NULL-checks with asserts since they seem to be impossible. I've gotten > that through tier1-3, but decided I did not want to overburden this RFE. Ideally I wish we could get rid of the parameter all together, but that is for another time. > >> line #197: I still wish we could get rid of that do while loop. Would >> searching for the last '[' be better performance wise?? Like >> UTF8::strrchr(start, utf_len, JVM_SIGNATURE_ARRAY)? > > I think that could have some cost since we'll scan the entire string. > > Perhaps an UTF8::mismatch(start, end, JVM_SIGNATURE_ARRAY) could be > added, but the if-do-while construct is nice here since the check for > JVM_SIGNATURE_CLASS is moved off the critical path. Either way I think > this is out of scope for this RFE, since the do-while loop is pre- > existing? Agree, thanks for considering it. > >> >> - gtest/runtime/test_classLoader.cpp >> please include a test that has a multi-dimensional array > > Adding: > > TEST_VM(ClassLoader, class_multiarray) { > ? bool bad_class_name = false; > ? TempNewSymbol name = SymbolTable::new_symbol("[[package/class"); > ? TempNewSymbol retval = ClassLoader::package_from_class_name(name, > &bad_class_name); > ? ASSERT_FALSE(bad_class_name) << "Function set bad_class_name with > class array"; > ? ASSERT_TRUE(retval->equals("package")) << "Wrong package for class > with leading bracket"; > } Thank you! > > OK, or do you need a new webrev? No, thanks! > > /Claes From claes.redestad at oracle.com Mon Mar 23 14:26:26 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 23 Mar 2020 15:26:26 +0100 Subject: RFR: 8241371: Refactor and consolidate package_from_name In-Reply-To: <95248311-86db-8287-ee9a-902774642daa@oracle.com> References: <828623c1-177c-16ef-281c-1a1cec4ecef7@oracle.com> <9de7ebfd-e234-4312-dc29-451096ff9426@oracle.com> <79369f75-cc8f-3658-873f-42f83e8e6fda@oracle.com> <7a447393-549a-c44a-0845-86ecdd0cda0d@oracle.com> <95248311-86db-8287-ee9a-902774642daa@oracle.com> Message-ID: <19e89298-0604-9abf-c4c6-4a267c964336@oracle.com> On 2020-03-23 15:13, Lois Foltan wrote: > Ideally I wish we could get rid of the parameter all together, but that > is for another time. Right, we could return some bad_class_name symbol instead of NULL, but need to make sure all callers deal properly with that (e.g. assert they never see such a value) >> OK, or do you need a new webrev? > No, thanks! Thanks! I'll run a quick test and wait for Ioi to see the latest version before pushing. /Claes From ioi.lam at oracle.com Mon Mar 23 18:31:46 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 23 Mar 2020 11:31:46 -0700 Subject: RFR: 8241371: Refactor and consolidate package_from_name In-Reply-To: <79369f75-cc8f-3658-873f-42f83e8e6fda@oracle.com> References: <828623c1-177c-16ef-281c-1a1cec4ecef7@oracle.com> <9de7ebfd-e234-4312-dc29-451096ff9426@oracle.com> <79369f75-cc8f-3658-873f-42f83e8e6fda@oracle.com> Message-ID: Hi Claes, Thanks for moving the code back to classLoader.cpp. Now I can compare the difference more easily and found some issues: ? if (*start == JVM_SIGNATURE_ARRAY) { ??? do { ????? start++; ??? } while (*start == JVM_SIGNATURE_ARRAY); The utf8 in the symbol is not 0-terminated, so it's better to change it to ??? do { ????? start++; ??? } while (start < end && *start == JVM_SIGNATURE_ARRAY); ======== ??? if (start != base && *start == JVM_SIGNATURE_CLASS) { We know that start will be at least (base+1), so the (start != base) is not necessary. ========= Also, do we ever call ClassLoader::package_from_class_name() with a NULL name? If not, I think this can be removed ? if (name == NULL) { ??? if (bad_class_name != NULL) { ????? *bad_class_name = true; ??? } ??? return NULL; ? } ========= ??? // A package or class name could have just the slash character in the name. This comment is ambiguous about whether "could have" is legal or not. I think it's better to change the comment to. ??? // It's illegal for a package or class name to have just the slash character in the name. Thanks - Ioi On 3/23/20 3:44 AM, Claes Redestad wrote: > On 2020-03-23 04:36, Ioi Lam wrote: >>>> >>> >>> Ok, I just felt that as a static utility it's more coupled with a >>> class/InstanceKlass than its class loader. If you insist I'll move it >>> to ClassLoader instead. >>> >> classLoader.cpp also has other methods related to packages, so I >> think it's a good place for this method, too. Plus, you don't need to >> move the test cases, so there's less code churn. > > Ok: > > http://cr.openjdk.java.net/~redestad/8241371/open.01/ > > /Claes From martin.doerr at sap.com Mon Mar 23 19:06:54 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 23 Mar 2020 19:06:54 +0000 Subject: RFR(XS): 8241464: [11u] Backport: make rehashing be a needed guaranteed safepoint cleanup action Message-ID: Hi, I'd like to backport a part of JDK-8221967. I'm using a new bug because it's only a part of the original one. Original change has the summary "Move redundant table lookup and make rehashing be a needed guaranteed safepoint cleanup action." The first part of it doesn't apply to 11u, because the SymbolTable and StringTable parts are different in 11u and I don't see the need to change them. The second part of it does apply and makes perfect sense. Especially if BiasedLocking is disabled (11u BiasedLocking implementation requests more safepoints making this fix less important). Original bug: https://bugs.openjdk.java.net/browse/JDK-8221967 Original issue: http://hg.openjdk.java.net/jdk/jdk/rev/2523496f5107 New webrev: http://cr.openjdk.java.net/~mdoerr/8221967_string_table_partial_backport_11u/webrev.00/ Please review. Best regards, Martin From claes.redestad at oracle.com Mon Mar 23 19:24:20 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 23 Mar 2020 20:24:20 +0100 Subject: RFR: 8241371: Refactor and consolidate package_from_name In-Reply-To: References: <828623c1-177c-16ef-281c-1a1cec4ecef7@oracle.com> <9de7ebfd-e234-4312-dc29-451096ff9426@oracle.com> <79369f75-cc8f-3658-873f-42f83e8e6fda@oracle.com> Message-ID: Hi, fixed and cleaned up: http://cr.openjdk.java.net/~redestad/8241371/open.02/ I mentioned the unnecessary name == NULL check elsewhere in this review thread and suggested we fix that in a follow-up. /Claes On 2020-03-23 19:31, Ioi Lam wrote: > Hi Claes, > > Thanks for moving the code back to classLoader.cpp. Now I can compare > the difference more easily and found some issues: > > ? if (*start == JVM_SIGNATURE_ARRAY) { > ??? do { > ????? start++; > ??? } while (*start == JVM_SIGNATURE_ARRAY); > > The utf8 in the symbol is not 0-terminated, so it's better to change it to > > ??? do { > ????? start++; > ??? } while (start < end && *start == JVM_SIGNATURE_ARRAY); > > ======== > > ??? if (start != base && *start == JVM_SIGNATURE_CLASS) { > > We know that start will be at least (base+1), so the (start != base) is > not necessary. > > ========= > > Also, do we ever call ClassLoader::package_from_class_name() with a NULL > name? If not, I think this can be removed > > ? if (name == NULL) { > ??? if (bad_class_name != NULL) { > ????? *bad_class_name = true; > ??? } > ??? return NULL; > ? } > > ========= > > ??? // A package or class name could have just the slash character in > the name. > > This comment is ambiguous about whether "could have" is legal or not. I > think it's better to change the comment to. > > ??? // It's illegal for a package or class name to have just the slash > character in the name. > > > Thanks > - Ioi > > On 3/23/20 3:44 AM, Claes Redestad wrote: >> On 2020-03-23 04:36, Ioi Lam wrote: >>>>> >>>> >>>> Ok, I just felt that as a static utility it's more coupled with a >>>> class/InstanceKlass than its class loader. If you insist I'll move it >>>> to ClassLoader instead. >>>> >>> classLoader.cpp also has other methods related to packages, so I >>> think it's a good place for this method, too. Plus, you don't need to >>> move the test cases, so there's less code churn. >> >> Ok: >> >> http://cr.openjdk.java.net/~redestad/8241371/open.01/ >> >> /Claes > From ioi.lam at oracle.com Mon Mar 23 19:41:49 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 23 Mar 2020 12:41:49 -0700 Subject: RFR: 8241371: Refactor and consolidate package_from_name In-Reply-To: References: <828623c1-177c-16ef-281c-1a1cec4ecef7@oracle.com> <9de7ebfd-e234-4312-dc29-451096ff9426@oracle.com> <79369f75-cc8f-3658-873f-42f83e8e6fda@oracle.com> Message-ID: Looks good to me. thanks! - Ioi On 3/23/20 12:24 PM, Claes Redestad wrote: > Hi, > > fixed and cleaned up: > > http://cr.openjdk.java.net/~redestad/8241371/open.02/ > > I mentioned the unnecessary name == NULL check elsewhere in this review > thread and suggested we fix that in a follow-up. > > /Claes > > On 2020-03-23 19:31, Ioi Lam wrote: >> Hi Claes, >> >> Thanks for moving the code back to classLoader.cpp. Now I can compare >> the difference more easily and found some issues: >> >> ?? if (*start == JVM_SIGNATURE_ARRAY) { >> ???? do { >> ?????? start++; >> ???? } while (*start == JVM_SIGNATURE_ARRAY); >> >> The utf8 in the symbol is not 0-terminated, so it's better to change >> it to >> >> ???? do { >> ?????? start++; >> ???? } while (start < end && *start == JVM_SIGNATURE_ARRAY); >> >> ======== >> >> ???? if (start != base && *start == JVM_SIGNATURE_CLASS) { >> >> We know that start will be at least (base+1), so the (start != base) >> is not necessary. >> >> ========= >> >> Also, do we ever call ClassLoader::package_from_class_name() with a >> NULL name? If not, I think this can be removed >> >> ?? if (name == NULL) { >> ???? if (bad_class_name != NULL) { >> ?????? *bad_class_name = true; >> ???? } >> ???? return NULL; >> ?? } >> >> ========= >> >> ???? // A package or class name could have just the slash character >> in the name. >> >> This comment is ambiguous about whether "could have" is legal or not. >> I think it's better to change the comment to. >> >> ???? // It's illegal for a package or class name to have just the >> slash character in the name. >> >> >> Thanks >> - Ioi >> >> On 3/23/20 3:44 AM, Claes Redestad wrote: >>> On 2020-03-23 04:36, Ioi Lam wrote: >>>>>> >>>>> >>>>> Ok, I just felt that as a static utility it's more coupled with a >>>>> class/InstanceKlass than its class loader. If you insist I'll move it >>>>> to ClassLoader instead. >>>>> >>>> classLoader.cpp also has other methods related to packages, so I >>>> think it's a good place for this method, too. Plus, you don't need >>>> to move the test cases, so there's less code churn. >>> >>> Ok: >>> >>> http://cr.openjdk.java.net/~redestad/8241371/open.01/ >>> >>> /Claes >> From ioi.lam at oracle.com Mon Mar 23 21:02:22 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 23 Mar 2020 14:02:22 -0700 Subject: RFR(XXS) 8241244 CDS dynamic dump asserts in ArchivePtrBitmapCleaner::do_bit In-Reply-To: References: <4c091d2d-b3c8-15f2-26b9-fb50bac47a20@oracle.com> Message-ID: Hi Coleen & Yumin, Thanks for the review. I've pushed the changes. - Ioi On 3/23/20 5:53 AM, coleen.phillimore at oracle.com wrote: > > Ioi, This looks good and reasonable. > > Coleen > > > > On 3/22/20 11:49 PM, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8241244 >> http://cr.openjdk.java.net/~iklam/jdk15/8241244-dynamic-cds-redefined-class.v01/ >> >> >> In the failing test case, some classes in the base archive (e.g., >> java.lang.Throwable) are redefined by JFR before the dynamic archive >> is dumped. The assert happens when java/lang/InternalError is >> destined for the dynamic archive, but its vtable contains a redefined >> method of the Throwable class. This method is neither in the base >> archive, nor in the dynamic archive, causing the assert to fail. >> >> The fix is: if a class (or any of its super-types) has been >> redefined, do not dump this class into the dynamic CDS archive. >> >> Note - this fix is not needed for the static archive dumping (java >> -Xshare:dump), as class redefinition cannot happen. >> >> Thanks >> - Ioi >> >> >> > From tobias.hartmann at oracle.com Tue Mar 24 08:52:08 2020 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Tue, 24 Mar 2020 09:52:08 +0100 Subject: [15] RFR(M) 8237497: vmStructs_jvmci.cpp does not check that the correct field type is specified In-Reply-To: References: <64e0f4db-7e5f-039a-84a7-b014d1a523f2@oracle.com> <5a40a0ed-7e3f-72a2-e707-09bd21dd9b6f@oracle.com> <62546416-20fa-c4bc-46ce-192eb19e18f7@oracle.com> <9dcb2133-072f-05d7-d2ed-dee7327143c3@oracle.com> Message-ID: Hi Vladimir, On 21.03.20 01:47, Vladimir Kozlov wrote: > http://cr.openjdk.java.net/~kvn/8237497/webrev.01/ Looks good to me. Best regards, Tobias From yudi.zheng at oracle.com Tue Mar 24 08:55:34 2020 From: yudi.zheng at oracle.com (Yudi Zheng) Date: Tue, 24 Mar 2020 09:55:34 +0100 Subject: RFR: 8241234: Unify monitor enter/exit runtime entries In-Reply-To: <7b9bdfbc-98da-dc84-71a6-386397cf1313@oracle.com> References: <222D2846-F6AE-4D5B-B41F-F976D90E329C@oracle.com> <7b9bdfbc-98da-dc84-71a6-386397cf1313@oracle.com> Message-ID: <51530CDB-F0F7-4EF7-AEAA-08E901F48966@oracle.com> Hi Tobias, Thanks for the review! -Yudi > On 23 Mar 2020, at 08:51, Tobias Hartmann wrote: > > Hi Yudi, > > looks good to me. > > Best regards, > Tobias > > On 20.03.20 16:03, Yudi Zheng wrote: >> Hello, >> >> Please review this patch that unifies the monitor enter/exit runtime entries used by C1, C2, and JVMCI. >> >> https://bugs.openjdk.java.net/browse/JDK-8061553 added support for faster "slow" locking paths. >> These paths are not currently used by JVMCI (or C1) leading to slower performance on programs >> that have a lot of short lived locked sections (such as dacapo:h2). >> >> This patch creates helper methods based on C2?s monitor enter/exit entries, and shares with the C1, >> JVMCI runtimes. It will prevent these runtimes from missing future updates. >> >> https://bugs.openjdk.java.net/browse/JDK-8241234 >> http://cr.openjdk.java.net/~yzheng/8241234/webrev.00/ >> >> Many thanks, >> Yudi >> From shade at redhat.com Tue Mar 24 10:54:35 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 24 Mar 2020 11:54:35 +0100 Subject: RFR (XS) 8241500: FieldLayout/OldLayoutCheck.java fails in 32-bit VMs Message-ID: Test bug: https://bugs.openjdk.java.net/browse/JDK-8241500 See the brief discussion and JOL dump in the bug. Fix: diff -r cc739b0abc44 test/hotspot/jtreg/runtime/FieldLayout/OldLayoutCheck.java --- a/test/hotspot/jtreg/runtime/FieldLayout/OldLayoutCheck.java Tue Mar 24 06:13:39 2020 +0100 +++ b/test/hotspot/jtreg/runtime/FieldLayout/OldLayoutCheck.java Tue Mar 24 11:52:36 2020 +0100 @@ -45,28 +45,32 @@ import java.util.Arrays; import java.util.Comparator; import jdk.internal.misc.Unsafe; import jdk.test.lib.Asserts; +import jdk.test.lib.Platform; public class OldLayoutCheck { static class LIClass { public long l; public int i; } + static final long INT_OFFSET = Platform.is64bit() ? 12L : 16L; + static final long LONG_OFFSET = Platform.is64bit() ? 16L : 8L; + static public void main(String[] args) { Unsafe unsafe = Unsafe.getUnsafe(); Class c = LIClass.class; Field[] fields = c.getFields(); for (int i = 0; i < fields.length; i++) { long offset = unsafe.objectFieldOffset(fields[i]); if (fields[i].getType() == int.class) { - Asserts.assertEquals(offset, 12L, "Misplaced int field"); + Asserts.assertEquals(offset, INT_OFFSET, "Misplaced int field"); } else if (fields[i].getType() == long.class) { - Asserts.assertEquals(offset, 16L, "Misplaced long field"); + Asserts.assertEquals(offset, LONG_OFFSET, "Misplaced long field"); } else { Asserts.fail("Unexpected field type"); } } } Testing: affected test on Linux {x86_64, x86_32} -- Thanks, -Aleksey From frederic.parain at oracle.com Tue Mar 24 13:19:17 2020 From: frederic.parain at oracle.com (Frederic Parain) Date: Tue, 24 Mar 2020 09:19:17 -0400 Subject: RFR (XS) 8241500: FieldLayout/OldLayoutCheck.java fails in 32-bit VMs In-Reply-To: References: Message-ID: <78DB5B79-4C4D-42D3-ADCE-5F588B67AFB1@oracle.com> Aleksey, Looks good to me, thank you for fixing this. Fred > On Mar 24, 2020, at 06:54, Aleksey Shipilev wrote: > > Test bug: > https://bugs.openjdk.java.net/browse/JDK-8241500 > > See the brief discussion and JOL dump in the bug. > > Fix: > > diff -r cc739b0abc44 test/hotspot/jtreg/runtime/FieldLayout/OldLayoutCheck.java > --- a/test/hotspot/jtreg/runtime/FieldLayout/OldLayoutCheck.java Tue Mar 24 06:13:39 2020 +0100 > +++ b/test/hotspot/jtreg/runtime/FieldLayout/OldLayoutCheck.java Tue Mar 24 11:52:36 2020 +0100 > @@ -45,28 +45,32 @@ > import java.util.Arrays; > import java.util.Comparator; > import jdk.internal.misc.Unsafe; > > import jdk.test.lib.Asserts; > +import jdk.test.lib.Platform; > > public class OldLayoutCheck { > > static class LIClass { > public long l; > public int i; > } > > + static final long INT_OFFSET = Platform.is64bit() ? 12L : 16L; > + static final long LONG_OFFSET = Platform.is64bit() ? 16L : 8L; > + > static public void main(String[] args) { > Unsafe unsafe = Unsafe.getUnsafe(); > Class c = LIClass.class; > Field[] fields = c.getFields(); > for (int i = 0; i < fields.length; i++) { > long offset = unsafe.objectFieldOffset(fields[i]); > if (fields[i].getType() == int.class) { > - Asserts.assertEquals(offset, 12L, "Misplaced int field"); > + Asserts.assertEquals(offset, INT_OFFSET, "Misplaced int field"); > } else if (fields[i].getType() == long.class) { > - Asserts.assertEquals(offset, 16L, "Misplaced long field"); > + Asserts.assertEquals(offset, LONG_OFFSET, "Misplaced long field"); > } else { > Asserts.fail("Unexpected field type"); > } > } > } > > Testing: affected test on Linux {x86_64, x86_32} > > -- > Thanks, > -Aleksey > From david.holmes at oracle.com Tue Mar 24 13:51:51 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 24 Mar 2020 23:51:51 +1000 Subject: RFR (XS) 8241500: FieldLayout/OldLayoutCheck.java fails in 32-bit VMs In-Reply-To: <78DB5B79-4C4D-42D3-ADCE-5F588B67AFB1@oracle.com> References: <78DB5B79-4C4D-42D3-ADCE-5F588B67AFB1@oracle.com> Message-ID: On 24/03/2020 11:19 pm, Frederic Parain wrote: > Aleksey, > > Looks good to me, thank you for fixing this. I'm somewhat surprised that it has taken 4 weeks for this problem on 32-bit to be noticed! :( A comment as to why INT_OFFSET and LONG_OFFSET have those values would be good. Otherwise fix is fine by me too. Thanks, David > Fred > > >> On Mar 24, 2020, at 06:54, Aleksey Shipilev wrote: >> >> Test bug: >> https://bugs.openjdk.java.net/browse/JDK-8241500 >> >> See the brief discussion and JOL dump in the bug. >> >> Fix: >> >> diff -r cc739b0abc44 test/hotspot/jtreg/runtime/FieldLayout/OldLayoutCheck.java >> --- a/test/hotspot/jtreg/runtime/FieldLayout/OldLayoutCheck.java Tue Mar 24 06:13:39 2020 +0100 >> +++ b/test/hotspot/jtreg/runtime/FieldLayout/OldLayoutCheck.java Tue Mar 24 11:52:36 2020 +0100 >> @@ -45,28 +45,32 @@ >> import java.util.Arrays; >> import java.util.Comparator; >> import jdk.internal.misc.Unsafe; >> >> import jdk.test.lib.Asserts; >> +import jdk.test.lib.Platform; >> >> public class OldLayoutCheck { >> >> static class LIClass { >> public long l; >> public int i; >> } >> >> + static final long INT_OFFSET = Platform.is64bit() ? 12L : 16L; >> + static final long LONG_OFFSET = Platform.is64bit() ? 16L : 8L; >> + >> static public void main(String[] args) { >> Unsafe unsafe = Unsafe.getUnsafe(); >> Class c = LIClass.class; >> Field[] fields = c.getFields(); >> for (int i = 0; i < fields.length; i++) { >> long offset = unsafe.objectFieldOffset(fields[i]); >> if (fields[i].getType() == int.class) { >> - Asserts.assertEquals(offset, 12L, "Misplaced int field"); >> + Asserts.assertEquals(offset, INT_OFFSET, "Misplaced int field"); >> } else if (fields[i].getType() == long.class) { >> - Asserts.assertEquals(offset, 16L, "Misplaced long field"); >> + Asserts.assertEquals(offset, LONG_OFFSET, "Misplaced long field"); >> } else { >> Asserts.fail("Unexpected field type"); >> } >> } >> } >> >> Testing: affected test on Linux {x86_64, x86_32} >> >> -- >> Thanks, >> -Aleksey >> > From shade at redhat.com Tue Mar 24 14:00:10 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 24 Mar 2020 15:00:10 +0100 Subject: RFR (XS) 8241500: FieldLayout/OldLayoutCheck.java fails in 32-bit VMs In-Reply-To: References: <78DB5B79-4C4D-42D3-ADCE-5F588B67AFB1@oracle.com> Message-ID: <1ca66561-7a4e-da58-f6d3-8f06f97e9f07@redhat.com> On 3/24/20 2:51 PM, David Holmes wrote: > On 24/03/2020 11:19 pm, Frederic Parain wrote: > I'm somewhat surprised that it has taken 4 weeks for this problem on > 32-bit to be noticed! :( Nobody has time to look at auxiliary CIs in the midst of pandemic! > A comment as to why INT_OFFSET and LONG_OFFSET have those values would > be good. Otherwise fix is fine by me too. I'll do this: + // 32-bit VMs: @0: 8 byte header, @8: long field, @16: int field + // 64-bit VMs: @0: 12 byte header, @12: int field, @16: long field + static final long INT_OFFSET = Platform.is64bit() ? 12L : 16L; + static final long LONG_OFFSET = Platform.is64bit() ? 16L : 8L; Reasonable? -- Thanks, -Aleksey From matthias.baesken at sap.com Tue Mar 24 15:24:32 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 24 Mar 2020 15:24:32 +0000 Subject: RFR(xxs): 8241395: Factor out platform independent code for os::xxx_memory_special() In-Reply-To: References: Message-ID: Hi Thomas, looks good to me. But you might want to change the copyright year in http://cr.openjdk.java.net/~stuefe/webrevs/8241395-refactor-reserve-memory-special/webrev.00/webrev/src/hotspot/share/runtime/os.hpp.frames.html Thanks, Matthias ---------- Forwarded message --------- From: Thomas St?fe > Date: Sat, Mar 21, 2020 at 1:40 PM Subject: RFR(xxs): 8241395: Factor out platform independent code for os::xxx_memory_special() To: Hotspot dev runtime > Hi, may I please have reviews for this little cleanup. A while ago we factored out platform independent behavior for os::reserve_memory() and friends to os.cpp and relegated platform dependent code to os::pd_reserve_memory() etc. This change does the same for os::reserve_memory_special() and os::release_memory_special(). Only Linux and Windows are affected since only they have meaningful implementations for UseLargePages. It also adds a small comment about the NMT Tracker object. In addition it asserts - now for all platforms, it used to do this only on Linux - that if both requested address and alignment are given to os::reserve_memory_special() the requested address should at least be aligned to the given alignment (the arguments should be mutually exclusive and I plan to clean this up further). Issue: https://bugs.openjdk.java.net/browse/JDK-8241395 webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8241395-refactor-reserve-memory-special/webrev.00/webrev/ Thanks, Thomas From thomas.stuefe at gmail.com Tue Mar 24 15:28:28 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 24 Mar 2020 16:28:28 +0100 Subject: RFR(xxs): 8241395: Factor out platform independent code for os::xxx_memory_special() In-Reply-To: References: Message-ID: Thank you Matthias. Will change copyright year before pushing. Cheers, Thomas On Tue, Mar 24, 2020 at 4:24 PM Baesken, Matthias wrote: > Hi Thomas, looks good to me. > > But you might want to change the copyright year in > > > > > http://cr.openjdk.java.net/~stuefe/webrevs/8241395-refactor-reserve-memory-special/webrev.00/webrev/src/hotspot/share/runtime/os.hpp.frames.html > > > > Thanks, Matthias > > > > ---------- Forwarded message --------- > From: *Thomas St?fe* > Date: Sat, Mar 21, 2020 at 1:40 PM > Subject: RFR(xxs): 8241395: Factor out platform independent code for > os::xxx_memory_special() > To: Hotspot dev runtime > > > > Hi, > > > > may I please have reviews for this little cleanup. > > > > A while ago we factored out platform independent behavior for > os::reserve_memory() and friends to os.cpp and relegated platform > dependent code to os::pd_reserve_memory() etc. This change does the same > for os::reserve_memory_special() and os::release_memory_special(). > > > > Only Linux and Windows are affected since only they have meaningful > implementations for UseLargePages. > > > > It also adds a small comment about the NMT Tracker object. > > > > In addition it asserts - now for all platforms, it used to do this only on > Linux - that if both requested address and alignment are given to > os::reserve_memory_special() the requested address should at least be > aligned to the given alignment (the arguments should be mutually exclusive > and I plan to clean this up further). > > > > Issue: https://bugs.openjdk.java.net/browse/JDK-8241395 > > webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/8241395-refactor-reserve-memory-special/webrev.00/webrev/ > > > > Thanks, Thomas > From martin.doerr at sap.com Tue Mar 24 15:31:40 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 24 Mar 2020 15:31:40 +0000 Subject: RFR(xxs): 8241395: Factor out platform independent code for os::xxx_memory_special() In-Reply-To: References: Message-ID: Hi Thomas, looks good to me, too. In addition to the copyright, please improve indentation for the 2nd argument line in os_linux.cpp and os_windows.cpp: +char* os::pd_reserve_memory_special(size_t bytes, size_t alignment, char* req_addr, bool exec) { and remove extra newline in os.cpp: + return result; + + } No need for a new webrev. Best regards, Martin > -----Original Message----- > From: hotspot-runtime-dev bounces at openjdk.java.net> On Behalf Of Baesken, Matthias > Sent: Dienstag, 24. M?rz 2020 16:25 > To: Thomas St?fe ; hotspot-runtime- > dev at openjdk.java.net > Subject: RE: RFR(xxs): 8241395: Factor out platform independent > code for os::xxx_memory_special() > > Hi Thomas, looks good to me. > But you might want to change the copyright year in > > http://cr.openjdk.java.net/~stuefe/webrevs/8241395-refactor-reserve- > memory- > special/webrev.00/webrev/src/hotspot/share/runtime/os.hpp.frames.html > > Thanks, Matthias > > ---------- Forwarded message --------- > From: Thomas St?fe > > > Date: Sat, Mar 21, 2020 at 1:40 PM > Subject: RFR(xxs): 8241395: Factor out platform independent code for > os::xxx_memory_special() > To: Hotspot dev runtime dev at openjdk.java.net> > > Hi, > > may I please have reviews for this little cleanup. > > A while ago we factored out platform independent behavior for > os::reserve_memory() and friends to os.cpp and relegated platform > dependent code to os::pd_reserve_memory() etc. This change does the > same for os::reserve_memory_special() and os::release_memory_special(). > > Only Linux and Windows are affected since only they have meaningful > implementations for UseLargePages. > > It also adds a small comment about the NMT Tracker object. > > In addition it asserts - now for all platforms, it used to do this only on Linux - > that if both requested address and alignment are given to > os::reserve_memory_special() the requested address should at least be > aligned to the given alignment (the arguments should be mutually exclusive > and I plan to clean this up further). > > Issue: https://bugs.openjdk.java.net/browse/JDK-8241395 > webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8241395-refactor- > reserve-memory-special/webrev.00/webrev/ > > Thanks, Thomas From vladimir.kozlov at oracle.com Tue Mar 24 15:43:02 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 24 Mar 2020 08:43:02 -0700 Subject: [15] RFR(M) 8237497: vmStructs_jvmci.cpp does not check that the correct field type is specified In-Reply-To: References: <64e0f4db-7e5f-039a-84a7-b014d1a523f2@oracle.com> <5a40a0ed-7e3f-72a2-e707-09bd21dd9b6f@oracle.com> <62546416-20fa-c4bc-46ce-192eb19e18f7@oracle.com> <9dcb2133-072f-05d7-d2ed-dee7327143c3@oracle.com> Message-ID: <8c5a1943-35a6-432a-2d3c-7401a200f328@oracle.com> Thank you, Tobias Vladimir K On 3/24/20 1:52 AM, Tobias Hartmann wrote: > Hi Vladimir, > > On 21.03.20 01:47, Vladimir Kozlov wrote: >> http://cr.openjdk.java.net/~kvn/8237497/webrev.01/ > > Looks good to me. > > Best regards, > Tobias > From vladimir.kozlov at oracle.com Tue Mar 24 15:51:10 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 24 Mar 2020 08:51:10 -0700 Subject: RFR: 8241234: Unify monitor enter/exit runtime entries In-Reply-To: <91eeada8-e05f-bc73-b029-94e169216a56@oracle.com> References: <222D2846-F6AE-4D5B-B41F-F976D90E329C@oracle.com> <91eeada8-e05f-bc73-b029-94e169216a56@oracle.com> Message-ID: Thank you, Coleen It seems Yudi did not see your comments :( CCing him and compiler group Regards, Vladimir On 3/20/20 8:35 AM, coleen.phillimore at oracle.com wrote: > > This seems like a good thing to do: > > http://cr.openjdk.java.net/~yzheng/8241234/webrev.00/src/hotspot/share/runtime/sharedRuntime.cpp.udiff.html > > I think you should have monitor_enter_helper and maybe monitor_exi_helper both take Handle as arguments.? This looks > very strange. > > +void SharedRuntime::monitor_enter_helper(oopDesc* _obj, BasicLock* lock, JavaThread* thread) { > > Because it creates a Handle anyway.? We try to eagerly pass Handle so that we don't have to search through the code to > make sure there aren't any unhandled oops. > > The _obj parameter name doesn't follow coding conventions (I was looking for it to be a field in something). > > + oop obj(_obj); > > > Is there not already a implicit conversion from oopDesc* to oop? > > Thanks, > Coleen > > > On 3/20/20 11:03 AM, Yudi Zheng wrote: >> Hello, >> >> Please review this patch that unifies the monitor enter/exit runtime entries used by C1, C2, and JVMCI. >> >> https://bugs.openjdk.java.net/browse/JDK-8061553 added support for faster "slow" locking paths. >> These paths are not currently used by JVMCI (or C1) leading to slower performance on programs >> that have a lot of short lived locked sections (such as dacapo:h2). >> >> This patch creates helper methods based on C2?s monitor enter/exit entries, and shares with the C1, >> JVMCI runtimes. It will prevent these runtimes from missing future updates. >> >> https://bugs.openjdk.java.net/browse/JDK-8241234 >> http://cr.openjdk.java.net/~yzheng/8241234/webrev.00/ >> >> Many thanks, >> Yudi > From lutz.schmidt at sap.com Tue Mar 24 16:03:46 2020 From: lutz.schmidt at sap.com (Schmidt, Lutz) Date: Tue, 24 Mar 2020 16:03:46 +0000 Subject: [CAUTION] RE: RFR(xxs): 8241395: Factor out platform independent code for os::xxx_memory_special() Message-ID: <22288384-68F5-4896-B832-8986D76549AE@sap.com> Hi Thomas, looks good to me in general. Is there a reason why in os_aix.cpp there is an assert(false) in pd_reserve* and an unimplemented() in pd_release*? For the other OSes, there is a fatal(), if applicable. In addition, I'd like to see the same failure text in either case. No new webrev required. Thanks, Lutz ?On 24.03.20, 16:31, "hotspot-runtime-dev on behalf of Doerr, Martin" wrote: Hi Thomas, looks good to me, too. In addition to the copyright, please improve indentation for the 2nd argument line in os_linux.cpp and os_windows.cpp: +char* os::pd_reserve_memory_special(size_t bytes, size_t alignment, char* req_addr, bool exec) { and remove extra newline in os.cpp: + return result; + + } No need for a new webrev. Best regards, Martin > -----Original Message----- > From: hotspot-runtime-dev bounces at openjdk.java.net> On Behalf Of Baesken, Matthias > Sent: Dienstag, 24. M?rz 2020 16:25 > To: Thomas St?fe ; hotspot-runtime- > dev at openjdk.java.net > Subject: RE: RFR(xxs): 8241395: Factor out platform independent > code for os::xxx_memory_special() > > Hi Thomas, looks good to me. > But you might want to change the copyright year in > > http://cr.openjdk.java.net/~stuefe/webrevs/8241395-refactor-reserve- > memory- > special/webrev.00/webrev/src/hotspot/share/runtime/os.hpp.frames.html > > Thanks, Matthias > > ---------- Forwarded message --------- > From: Thomas St?fe > > > Date: Sat, Mar 21, 2020 at 1:40 PM > Subject: RFR(xxs): 8241395: Factor out platform independent code for > os::xxx_memory_special() > To: Hotspot dev runtime dev at openjdk.java.net> > > Hi, > > may I please have reviews for this little cleanup. > > A while ago we factored out platform independent behavior for > os::reserve_memory() and friends to os.cpp and relegated platform > dependent code to os::pd_reserve_memory() etc. This change does the > same for os::reserve_memory_special() and os::release_memory_special(). > > Only Linux and Windows are affected since only they have meaningful > implementations for UseLargePages. > > It also adds a small comment about the NMT Tracker object. > > In addition it asserts - now for all platforms, it used to do this only on Linux - > that if both requested address and alignment are given to > os::reserve_memory_special() the requested address should at least be > aligned to the given alignment (the arguments should be mutually exclusive > and I plan to clean this up further). > > Issue: https://bugs.openjdk.java.net/browse/JDK-8241395 > webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8241395-refactor- > reserve-memory-special/webrev.00/webrev/ > > Thanks, Thomas From daniel.daugherty at oracle.com Tue Mar 24 17:01:29 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 24 Mar 2020 13:01:29 -0400 Subject: RFR(T): 8241532: ProblemList tests from 8241530 on OSX Message-ID: <59631faa-ee70-c95f-7c9f-91f24cd0d1de@oracle.com> Greetings, I have a trivial review for ProblemListing some tests. We're having some network issues with the new OSX 10.15 machines that are being addressed. In the mean time, I'm trying to reduce the noise in the CI in Tier5 and Tier6 so I'm ProblemListing the affected tests: $ hg diff diff -r 23dab0354eb0 test/jdk/ProblemList.txt --- a/test/jdk/ProblemList.txt??? Tue Mar 24 17:39:52 2020 +0100 +++ b/test/jdk/ProblemList.txt??? Tue Mar 24 12:57:43 2020 -0400 @@ -604,6 +604,10 @@ ?com/sun/management/OperatingSystemMXBean/GetProcessCpuLoad.java 8030957 aix-all ?com/sun/management/OperatingSystemMXBean/GetSystemCpuLoad.java 8030957 aix-all +sun/management/jdp/JdpDefaultsTest.java 8241530 macosx-all +sun/management/jdp/JdpJmxRemoteDynamicPortTest.java 8241530 macosx-all +sun/management/jdp/JdpSpecificAddressTest.java 8241530 macosx-all + ?############################################################################ ?# jdk_jmx @@ -924,6 +928,9 @@ ?com/sun/jdi/InvokeHangTest.java 8218463 linux-all +com/sun/jdi/JdwpAttachTest.java 8241530 macosx-all +com/sun/jdi/JdwpListenTest.java 8241530 macosx-all + ?############################################################################ ?# jdk_time Thanks, in advance, for any comments, questions or suggestions. Dan From christian.tornqvist at oracle.com Tue Mar 24 17:03:31 2020 From: christian.tornqvist at oracle.com (Christian Tornqvist) Date: Tue, 24 Mar 2020 10:03:31 -0700 Subject: RFR(T): 8241532: ProblemList tests from 8241530 on OSX In-Reply-To: <59631faa-ee70-c95f-7c9f-91f24cd0d1de@oracle.com> References: <59631faa-ee70-c95f-7c9f-91f24cd0d1de@oracle.com> Message-ID: Looks good, thanks for doing this. Thanks, Christian > On Mar 24, 2020, at 10:01 AM, Daniel D. Daugherty wrote: > > Greetings, > > I have a trivial review for ProblemListing some tests. > > We're having some network issues with the new OSX 10.15 machines that > are being addressed. In the mean time, I'm trying to reduce the noise > in the CI in Tier5 and Tier6 so I'm ProblemListing the affected tests: > > $ hg diff > diff -r 23dab0354eb0 test/jdk/ProblemList.txt > --- a/test/jdk/ProblemList.txt Tue Mar 24 17:39:52 2020 +0100 > +++ b/test/jdk/ProblemList.txt Tue Mar 24 12:57:43 2020 -0400 > @@ -604,6 +604,10 @@ > com/sun/management/OperatingSystemMXBean/GetProcessCpuLoad.java 8030957 aix-all > com/sun/management/OperatingSystemMXBean/GetSystemCpuLoad.java 8030957 aix-all > > +sun/management/jdp/JdpDefaultsTest.java 8241530 macosx-all > +sun/management/jdp/JdpJmxRemoteDynamicPortTest.java 8241530 macosx-all > +sun/management/jdp/JdpSpecificAddressTest.java 8241530 macosx-all > + > ############################################################################ > > # jdk_jmx > @@ -924,6 +928,9 @@ > > com/sun/jdi/InvokeHangTest.java 8218463 linux-all > > +com/sun/jdi/JdwpAttachTest.java 8241530 macosx-all > +com/sun/jdi/JdwpListenTest.java 8241530 macosx-all > + > ############################################################################ > > # jdk_time > > Thanks, in advance, for any comments, questions or suggestions. > > Dan > From daniel.daugherty at oracle.com Tue Mar 24 17:04:23 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 24 Mar 2020 13:04:23 -0400 Subject: RFR(T): 8241532: ProblemList tests from 8241530 on OSX In-Reply-To: References: <59631faa-ee70-c95f-7c9f-91f24cd0d1de@oracle.com> Message-ID: <5e7bf2c8-ad77-949d-c984-b63cf0ace03a@oracle.com> Thanks for the fast review! Dan On 3/24/20 1:03 PM, Christian Tornqvist wrote: > Looks good, thanks for doing this. > > Thanks, > Christian > >> On Mar 24, 2020, at 10:01 AM, Daniel D. Daugherty wrote: >> >> Greetings, >> >> I have a trivial review for ProblemListing some tests. >> >> We're having some network issues with the new OSX 10.15 machines that >> are being addressed. In the mean time, I'm trying to reduce the noise >> in the CI in Tier5 and Tier6 so I'm ProblemListing the affected tests: >> >> $ hg diff >> diff -r 23dab0354eb0 test/jdk/ProblemList.txt >> --- a/test/jdk/ProblemList.txt Tue Mar 24 17:39:52 2020 +0100 >> +++ b/test/jdk/ProblemList.txt Tue Mar 24 12:57:43 2020 -0400 >> @@ -604,6 +604,10 @@ >> com/sun/management/OperatingSystemMXBean/GetProcessCpuLoad.java 8030957 aix-all >> com/sun/management/OperatingSystemMXBean/GetSystemCpuLoad.java 8030957 aix-all >> >> +sun/management/jdp/JdpDefaultsTest.java 8241530 macosx-all >> +sun/management/jdp/JdpJmxRemoteDynamicPortTest.java 8241530 macosx-all >> +sun/management/jdp/JdpSpecificAddressTest.java 8241530 macosx-all >> + >> ############################################################################ >> >> # jdk_jmx >> @@ -924,6 +928,9 @@ >> >> com/sun/jdi/InvokeHangTest.java 8218463 linux-all >> >> +com/sun/jdi/JdwpAttachTest.java 8241530 macosx-all >> +com/sun/jdi/JdwpListenTest.java 8241530 macosx-all >> + >> ############################################################################ >> >> # jdk_time >> >> Thanks, in advance, for any comments, questions or suggestions. >> >> Dan >> From yudi.zheng at oracle.com Tue Mar 24 17:14:55 2020 From: yudi.zheng at oracle.com (Yudi Zheng) Date: Tue, 24 Mar 2020 18:14:55 +0100 Subject: RFR: 8241234: Unify monitor enter/exit runtime entries In-Reply-To: References: <222D2846-F6AE-4D5B-B41F-F976D90E329C@oracle.com> <91eeada8-e05f-bc73-b029-94e169216a56@oracle.com> Message-ID: Hi Coleen, Thanks for the review! Sorry I missed your email. And thanks Vladimir for the reminder. >> I think you should have monitor_enter_helper and maybe monitor_exi_helper both take Handle as arguments. This looks very strange. Shall we change ObjectSynchronizer::exit to accept Handle as argument as well? >> +void SharedRuntime::monitor_enter_helper(oopDesc* _obj, BasicLock* lock, JavaThread* thread) { >> Because it creates a Handle anyway. We try to eagerly pass Handle so that we don't have to search through the code to make sure there aren't any unhandled oops. >> The _obj parameter name doesn't follow coding conventions (I was looking for it to be a field in something). I took the parameter name from the original methods: JRT_BLOCK_ENTRY(void, SharedRuntime::complete_monitor_locking_C(oopDesc* _obj, BasicLock* lock, JavaThread* thread)) JRT_LEAF(void, SharedRuntime::complete_monitor_unlocking_C(oopDesc* _obj, BasicLock* lock, JavaThread * THREAD)) Will Change that. >> + oop obj(_obj); >> Is there not already a implicit conversion from oopDesc* to oop? Also took from the original methods. Will get rid of that. Thanks, Yudi >> Thanks, >> Coleen >> On 3/20/20 11:03 AM, Yudi Zheng wrote: >>> Hello, >>> >>> Please review this patch that unifies the monitor enter/exit runtime entries used by C1, C2, and JVMCI. >>> >>> https://bugs.openjdk.java.net/browse/JDK-8061553 added support for faster "slow" locking paths. >>> These paths are not currently used by JVMCI (or C1) leading to slower performance on programs >>> that have a lot of short lived locked sections (such as dacapo:h2). >>> >>> This patch creates helper methods based on C2?s monitor enter/exit entries, and shares with the C1, >>> JVMCI runtimes. It will prevent these runtimes from missing future updates. >>> >>> https://bugs.openjdk.java.net/browse/JDK-8241234 >>> http://cr.openjdk.java.net/~yzheng/8241234/webrev.00/ >>> >>> Many thanks, >>> Yudi From thomas.stuefe at gmail.com Tue Mar 24 17:28:49 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 24 Mar 2020 18:28:49 +0100 Subject: RFR(xxs): 8241395: Factor out platform independent code for os::xxx_memory_special() In-Reply-To: References: Message-ID: Thanks Martin! On Tue, Mar 24, 2020 at 4:31 PM Doerr, Martin wrote: > Hi Thomas, > > looks good to me, too. > > In addition to the copyright, please improve indentation for the 2nd > argument line in os_linux.cpp and os_windows.cpp: > +char* os::pd_reserve_memory_special(size_t bytes, size_t alignment, > char* req_addr, bool exec) { > > and remove extra newline in os.cpp: > + return result; > + > + } > > No need for a new webrev. > > Best regards, > Martin > > > > -----Original Message----- > > From: hotspot-runtime-dev > bounces at openjdk.java.net> On Behalf Of Baesken, Matthias > > Sent: Dienstag, 24. M?rz 2020 16:25 > > To: Thomas St?fe ; hotspot-runtime- > > dev at openjdk.java.net > > Subject: RE: RFR(xxs): 8241395: Factor out platform independent > > code for os::xxx_memory_special() > > > > Hi Thomas, looks good to me. > > But you might want to change the copyright year in > > > > http://cr.openjdk.java.net/~stuefe/webrevs/8241395-refactor-reserve- > > memory- > > special/webrev.00/webrev/src/hotspot/share/runtime/os.hpp.frames.html > > > > Thanks, Matthias > > > > ---------- Forwarded message --------- > > From: Thomas St?fe > > > > > Date: Sat, Mar 21, 2020 at 1:40 PM > > Subject: RFR(xxs): 8241395: Factor out platform independent code for > > os::xxx_memory_special() > > To: Hotspot dev runtime > dev at openjdk.java.net> > > > > Hi, > > > > may I please have reviews for this little cleanup. > > > > A while ago we factored out platform independent behavior for > > os::reserve_memory() and friends to os.cpp and relegated platform > > dependent code to os::pd_reserve_memory() etc. This change does the > > same for os::reserve_memory_special() and os::release_memory_special(). > > > > Only Linux and Windows are affected since only they have meaningful > > implementations for UseLargePages. > > > > It also adds a small comment about the NMT Tracker object. > > > > In addition it asserts - now for all platforms, it used to do this only > on Linux - > > that if both requested address and alignment are given to > > os::reserve_memory_special() the requested address should at least be > > aligned to the given alignment (the arguments should be mutually > exclusive > > and I plan to clean this up further). > > > > Issue: https://bugs.openjdk.java.net/browse/JDK-8241395 > > webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8241395-refactor- > > reserve-memory-special/webrev.00/webrev/ > > > > Thanks, Thomas > From thomas.stuefe at gmail.com Tue Mar 24 17:33:19 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 24 Mar 2020 18:33:19 +0100 Subject: [CAUTION] RE: RFR(xxs): 8241395: Factor out platform independent code for os::xxx_memory_special() In-Reply-To: <22288384-68F5-4896-B832-8986D76549AE@sap.com> References: <22288384-68F5-4896-B832-8986D76549AE@sap.com> Message-ID: Hi Lutz, no special reason for the AIX case, I think our code there predated the other versions. I'll unify the messages as you suggested. Thanks for the review. ..Thomas On Tue, Mar 24, 2020 at 5:03 PM Schmidt, Lutz wrote: > Hi Thomas, > > looks good to me in general. > > Is there a reason why in os_aix.cpp there is an assert(false) in > pd_reserve* and an unimplemented() in pd_release*? For the other OSes, > there is a fatal(), if applicable. > > In addition, I'd like to see the same failure text in either case. > > No new webrev required. > > Thanks, > Lutz > > ?On 24.03.20, 16:31, "hotspot-runtime-dev on behalf of Doerr, Martin" < > hotspot-runtime-dev-bounces at openjdk.java.net on behalf of > martin.doerr at sap.com> wrote: > > Hi Thomas, > > looks good to me, too. > > In addition to the copyright, please improve indentation for the 2nd > argument line in os_linux.cpp and os_windows.cpp: > +char* os::pd_reserve_memory_special(size_t bytes, size_t alignment, > char* req_addr, bool exec) { > > and remove extra newline in os.cpp: > + return result; > + > + } > > No need for a new webrev. > > Best regards, > Martin > > > > -----Original Message----- > > From: hotspot-runtime-dev > bounces at openjdk.java.net> On Behalf Of Baesken, Matthias > > Sent: Dienstag, 24. M?rz 2020 16:25 > > To: Thomas St?fe ; hotspot-runtime- > > dev at openjdk.java.net > > Subject: RE: RFR(xxs): 8241395: Factor out platform independent > > code for os::xxx_memory_special() > > > > Hi Thomas, looks good to me. > > But you might want to change the copyright year in > > > > http://cr.openjdk.java.net/~stuefe/webrevs/8241395-refactor-reserve- > > memory- > > special/webrev.00/webrev/src/hotspot/share/runtime/os.hpp.frames.html > > > > Thanks, Matthias > > > > ---------- Forwarded message --------- > > From: Thomas St?fe > > > > > Date: Sat, Mar 21, 2020 at 1:40 PM > > Subject: RFR(xxs): 8241395: Factor out platform independent code for > > os::xxx_memory_special() > > To: Hotspot dev runtime > dev at openjdk.java.net> > > > > Hi, > > > > may I please have reviews for this little cleanup. > > > > A while ago we factored out platform independent behavior for > > os::reserve_memory() and friends to os.cpp and relegated platform > > dependent code to os::pd_reserve_memory() etc. This change does the > > same for os::reserve_memory_special() and > os::release_memory_special(). > > > > Only Linux and Windows are affected since only they have meaningful > > implementations for UseLargePages. > > > > It also adds a small comment about the NMT Tracker object. > > > > In addition it asserts - now for all platforms, it used to do this > only on Linux - > > that if both requested address and alignment are given to > > os::reserve_memory_special() the requested address should at least be > > aligned to the given alignment (the arguments should be mutually > exclusive > > and I plan to clean this up further). > > > > Issue: https://bugs.openjdk.java.net/browse/JDK-8241395 > > webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8241395-refactor- > > reserve-memory-special/webrev.00/webrev/ > > > > Thanks, Thomas > > > From lutz.schmidt at sap.com Tue Mar 24 19:52:09 2020 From: lutz.schmidt at sap.com (Schmidt, Lutz) Date: Tue, 24 Mar 2020 19:52:09 +0000 Subject: [CAUTION] RE: RFR(xxs): 8241395: Factor out platform independent code for os::xxx_memory_special() In-Reply-To: References: <22288384-68F5-4896-B832-8986D76549AE@sap.com> Message-ID: <4F2864B5-80DE-497D-9276-467D46A8ACFA@sap.com> Perfect. Thank you, Thomas. Lutz From: Thomas St?fe Date: Tuesday, 24. March 2020 at 18:33 To: Lutz Schmidt Cc: "Doerr, Martin (martin.doerr at sap.com)" , "Baesken, Matthias" , "hotspot-runtime-dev at openjdk.java.net" Subject: Re: [CAUTION] RE: RFR(xxs): 8241395: Factor out platform independent code for os::xxx_memory_special() Hi Lutz, no special reason for the AIX case, I think our code there predated the other versions. I'll unify the messages as you suggested. Thanks for the review. ..Thomas On Tue, Mar 24, 2020 at 5:03 PM Schmidt, Lutz > wrote: Hi Thomas, looks good to me in general. Is there a reason why in os_aix.cpp there is an assert(false) in pd_reserve* and an unimplemented() in pd_release*? For the other OSes, there is a fatal(), if applicable. In addition, I'd like to see the same failure text in either case. No new webrev required. Thanks, Lutz On 24.03.20, 16:31, "hotspot-runtime-dev on behalf of Doerr, Martin" on behalf of martin.doerr at sap.com> wrote: Hi Thomas, looks good to me, too. In addition to the copyright, please improve indentation for the 2nd argument line in os_linux.cpp and os_windows.cpp: +char* os::pd_reserve_memory_special(size_t bytes, size_t alignment, char* req_addr, bool exec) { and remove extra newline in os.cpp: + return result; + + } No need for a new webrev. Best regards, Martin > -----Original Message----- > From: hotspot-runtime-dev bounces at openjdk.java.net> On Behalf Of Baesken, Matthias > Sent: Dienstag, 24. M?rz 2020 16:25 > To: Thomas St?fe >; hotspot-runtime- > dev at openjdk.java.net > Subject: RE: RFR(xxs): 8241395: Factor out platform independent > code for os::xxx_memory_special() > > Hi Thomas, looks good to me. > But you might want to change the copyright year in > > http://cr.openjdk.java.net/~stuefe/webrevs/8241395-refactor-reserve- > memory- > special/webrev.00/webrev/src/hotspot/share/runtime/os.hpp.frames.html > > Thanks, Matthias > > ---------- Forwarded message --------- > From: Thomas St?fe > >> > Date: Sat, Mar 21, 2020 at 1:40 PM > Subject: RFR(xxs): 8241395: Factor out platform independent code for > os::xxx_memory_special() > To: Hotspot dev runtime dev at openjdk.java.net>> > > Hi, > > may I please have reviews for this little cleanup. > > A while ago we factored out platform independent behavior for > os::reserve_memory() and friends to os.cpp and relegated platform > dependent code to os::pd_reserve_memory() etc. This change does the > same for os::reserve_memory_special() and os::release_memory_special(). > > Only Linux and Windows are affected since only they have meaningful > implementations for UseLargePages. > > It also adds a small comment about the NMT Tracker object. > > In addition it asserts - now for all platforms, it used to do this only on Linux - > that if both requested address and alignment are given to > os::reserve_memory_special() the requested address should at least be > aligned to the given alignment (the arguments should be mutually exclusive > and I plan to clean this up further). > > Issue: https://bugs.openjdk.java.net/browse/JDK-8241395 > webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8241395-refactor- > reserve-memory-special/webrev.00/webrev/ > > Thanks, Thomas From lutz.schmidt at sap.com Tue Mar 24 20:00:16 2020 From: lutz.schmidt at sap.com (Schmidt, Lutz) Date: Tue, 24 Mar 2020 20:00:16 +0000 Subject: [CAUTION] RFR(XS): 8241464: [11u] Backport: make rehashing be a needed guaranteed safepoint cleanup action Message-ID: <6401890E-B4B8-406B-A33D-6FBE6216228C@sap.com> Hi Martin, your change looks good to me. Maybe you want to add a small comment like "// Heal unbalanced hash (done at safepoint)." Thanks, Lutz ?On 23.03.20, 20:06, "hotspot-runtime-dev on behalf of Doerr, Martin" wrote: Hi, I'd like to backport a part of JDK-8221967. I'm using a new bug because it's only a part of the original one. Original change has the summary "Move redundant table lookup and make rehashing be a needed guaranteed safepoint cleanup action." The first part of it doesn't apply to 11u, because the SymbolTable and StringTable parts are different in 11u and I don't see the need to change them. The second part of it does apply and makes perfect sense. Especially if BiasedLocking is disabled (11u BiasedLocking implementation requests more safepoints making this fix less important). Original bug: https://bugs.openjdk.java.net/browse/JDK-8221967 Original issue: http://hg.openjdk.java.net/jdk/jdk/rev/2523496f5107 New webrev: http://cr.openjdk.java.net/~mdoerr/8221967_string_table_partial_backport_11u/webrev.00/ Please review. Best regards, Martin From martin.doerr at sap.com Tue Mar 24 20:07:05 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 24 Mar 2020 20:07:05 +0000 Subject: [CAUTION] RFR(XS): 8241464: [11u] Backport: make rehashing be a needed guaranteed safepoint cleanup action In-Reply-To: <6401890E-B4B8-406B-A33D-6FBE6216228C@sap.com> References: <6401890E-B4B8-406B-A33D-6FBE6216228C@sap.com> Message-ID: Hi Lutz, > your change looks good to me. Thanks for the review! > Maybe you want to add a small comment like "// Heal unbalanced hash > (done at safepoint)." Well, it is still a clean (partial) backport, not a new change, so I don't want to introduce a diff to the jdk/jdk version. Best regards, Martin > -----Original Message----- > From: Schmidt, Lutz > Sent: Dienstag, 24. M?rz 2020 21:00 > To: Doerr, Martin ; jdk-updates- > dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR(XS): 8241464: [11u] Backport: make rehashing be > a needed guaranteed safepoint cleanup action > > Hi Martin, > > your change looks good to me. > > Maybe you want to add a small comment like "// Heal unbalanced hash > (done at safepoint)." > > Thanks, > Lutz > > ?On 23.03.20, 20:06, "hotspot-runtime-dev on behalf of Doerr, Martin" > martin.doerr at sap.com> wrote: > > Hi, > > I'd like to backport a part of JDK-8221967. I'm using a new bug because it's > only a part of the original one. > Original change has the summary "Move redundant table lookup and make > rehashing be a needed guaranteed safepoint cleanup action." > The first part of it doesn't apply to 11u, because the SymbolTable and > StringTable parts are different in 11u and I don't see the need to change > them. > > The second part of it does apply and makes perfect sense. Especially if > BiasedLocking is disabled (11u BiasedLocking implementation requests more > safepoints making this fix less important). > > Original bug: > https://bugs.openjdk.java.net/browse/JDK-8221967 > > Original issue: > http://hg.openjdk.java.net/jdk/jdk/rev/2523496f5107 > > New webrev: > > http://cr.openjdk.java.net/~mdoerr/8221967_string_table_partial_backport > _11u/webrev.00/ > > Please review. > > Best regards, > Martin > > From coleen.phillimore at oracle.com Tue Mar 24 20:09:18 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 24 Mar 2020 16:09:18 -0400 Subject: [CAUTION] RFR(XS): 8241464: [11u] Backport: make rehashing be a needed guaranteed safepoint cleanup action In-Reply-To: References: <6401890E-B4B8-406B-A33D-6FBE6216228C@sap.com> Message-ID: This change looks good. On 3/24/20 4:07 PM, Doerr, Martin wrote: > Hi Lutz, > >> your change looks good to me. > Thanks for the review! > >> Maybe you want to add a small comment like "// Heal unbalanced hash >> (done at safepoint)." > Well, it is still a clean (partial) backport, not a new change, so I don't want to introduce a diff to the jdk/jdk version. I agree. If you want to add this comment, we should have done it in the current version. Coleen > > Best regards, > Martin > > >> -----Original Message----- >> From: Schmidt, Lutz >> Sent: Dienstag, 24. M?rz 2020 21:00 >> To: Doerr, Martin ; jdk-updates- >> dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net >> Subject: Re: RFR(XS): 8241464: [11u] Backport: make rehashing be >> a needed guaranteed safepoint cleanup action >> >> Hi Martin, >> >> your change looks good to me. >> >> Maybe you want to add a small comment like "// Heal unbalanced hash >> (done at safepoint)." >> >> Thanks, >> Lutz >> >> ?On 23.03.20, 20:06, "hotspot-runtime-dev on behalf of Doerr, Martin" >> > martin.doerr at sap.com> wrote: >> >> Hi, >> >> I'd like to backport a part of JDK-8221967. I'm using a new bug because it's >> only a part of the original one. >> Original change has the summary "Move redundant table lookup and make >> rehashing be a needed guaranteed safepoint cleanup action." >> The first part of it doesn't apply to 11u, because the SymbolTable and >> StringTable parts are different in 11u and I don't see the need to change >> them. >> >> The second part of it does apply and makes perfect sense. Especially if >> BiasedLocking is disabled (11u BiasedLocking implementation requests more >> safepoints making this fix less important). >> >> Original bug: >> https://bugs.openjdk.java.net/browse/JDK-8221967 >> >> Original issue: >> http://hg.openjdk.java.net/jdk/jdk/rev/2523496f5107 >> >> New webrev: >> >> http://cr.openjdk.java.net/~mdoerr/8221967_string_table_partial_backport >> _11u/webrev.00/ >> >> Please review. >> >> Best regards, >> Martin >> >> From christoph.langer at sap.com Tue Mar 24 20:11:26 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Tue, 24 Mar 2020 20:11:26 +0000 Subject: [CAUTION] RFR(XS): 8241464: [11u] Backport: make rehashing be a needed guaranteed safepoint cleanup action In-Reply-To: References: <6401890E-B4B8-406B-A33D-6FBE6216228C@sap.com> Message-ID: Hi Martin, the backport looks good to me as well. > > Maybe you want to add a small comment like "// Heal unbalanced hash > > (done at safepoint)." > Well, it is still a clean (partial) backport, not a new change, so I don't want to > introduce a diff to the jdk/jdk version. I agree. As this is not new code, the backport should be as clean as possible. Best regards Christoph > > Best regards, > Martin > > > > -----Original Message----- > > From: Schmidt, Lutz > > Sent: Dienstag, 24. M?rz 2020 21:00 > > To: Doerr, Martin ; jdk-updates- > > dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net > > Subject: Re: RFR(XS): 8241464: [11u] Backport: make rehashing be > > a needed guaranteed safepoint cleanup action > > > > Hi Martin, > > > > your change looks good to me. > > > > Maybe you want to add a small comment like "// Heal unbalanced hash > > (done at safepoint)." > > > > Thanks, > > Lutz > > > > ?On 23.03.20, 20:06, "hotspot-runtime-dev on behalf of Doerr, Martin" > > > martin.doerr at sap.com> wrote: > > > > Hi, > > > > I'd like to backport a part of JDK-8221967. I'm using a new bug because it's > > only a part of the original one. > > Original change has the summary "Move redundant table lookup and > make > > rehashing be a needed guaranteed safepoint cleanup action." > > The first part of it doesn't apply to 11u, because the SymbolTable and > > StringTable parts are different in 11u and I don't see the need to change > > them. > > > > The second part of it does apply and makes perfect sense. Especially if > > BiasedLocking is disabled (11u BiasedLocking implementation requests > more > > safepoints making this fix less important). > > > > Original bug: > > https://bugs.openjdk.java.net/browse/JDK-8221967 > > > > Original issue: > > http://hg.openjdk.java.net/jdk/jdk/rev/2523496f5107 > > > > New webrev: > > > > > http://cr.openjdk.java.net/~mdoerr/8221967_string_table_partial_backport > > _11u/webrev.00/ > > > > Please review. > > > > Best regards, > > Martin > > > > From martin.doerr at sap.com Tue Mar 24 22:14:32 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 24 Mar 2020 22:14:32 +0000 Subject: RFR(XS): 8241464: [11u] Backport: make rehashing be a needed guaranteed safepoint cleanup action Message-ID: Thanks for the review! Best regards, Martin > -----Original Message----- > From: Langer, Christoph > Sent: Dienstag, 24. M?rz 2020 21:11 > To: Doerr, Martin ; Schmidt, Lutz > ; jdk-updates-dev at openjdk.java.net; hotspot- > runtime-dev at openjdk.java.net > Subject: RE: RFR(XS): 8241464: [11u] Backport: make rehashing be > a needed guaranteed safepoint cleanup action > > Hi Martin, > > the backport looks good to me as well. > > > > Maybe you want to add a small comment like "// Heal unbalanced hash > > > (done at safepoint)." > > Well, it is still a clean (partial) backport, not a new change, so I don't want to > > introduce a diff to the jdk/jdk version. > > I agree. As this is not new code, the backport should be as clean as possible. > > Best regards > Christoph > > > > > Best regards, > > Martin > > > > > > > -----Original Message----- > > > From: Schmidt, Lutz > > > Sent: Dienstag, 24. M?rz 2020 21:00 > > > To: Doerr, Martin ; jdk-updates- > > > dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net > > > Subject: Re: RFR(XS): 8241464: [11u] Backport: make rehashing be > > > a needed guaranteed safepoint cleanup action > > > > > > Hi Martin, > > > > > > your change looks good to me. > > > > > > Maybe you want to add a small comment like "// Heal unbalanced hash > > > (done at safepoint)." > > > > > > Thanks, > > > Lutz > > > > > > ?On 23.03.20, 20:06, "hotspot-runtime-dev on behalf of Doerr, Martin" > > > > > martin.doerr at sap.com> wrote: > > > > > > Hi, > > > > > > I'd like to backport a part of JDK-8221967. I'm using a new bug because > it's > > > only a part of the original one. > > > Original change has the summary "Move redundant table lookup and > > make > > > rehashing be a needed guaranteed safepoint cleanup action." > > > The first part of it doesn't apply to 11u, because the SymbolTable and > > > StringTable parts are different in 11u and I don't see the need to change > > > them. > > > > > > The second part of it does apply and makes perfect sense. Especially if > > > BiasedLocking is disabled (11u BiasedLocking implementation requests > > more > > > safepoints making this fix less important). > > > > > > Original bug: > > > https://bugs.openjdk.java.net/browse/JDK-8221967 > > > > > > Original issue: > > > http://hg.openjdk.java.net/jdk/jdk/rev/2523496f5107 > > > > > > New webrev: > > > > > > > > > http://cr.openjdk.java.net/~mdoerr/8221967_string_table_partial_backport > > > _11u/webrev.00/ > > > > > > Please review. > > > > > > Best regards, > > > Martin > > > > > > From david.holmes at oracle.com Tue Mar 24 23:08:04 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 25 Mar 2020 09:08:04 +1000 Subject: RFR (XS) 8241500: FieldLayout/OldLayoutCheck.java fails in 32-bit VMs In-Reply-To: <1ca66561-7a4e-da58-f6d3-8f06f97e9f07@redhat.com> References: <78DB5B79-4C4D-42D3-ADCE-5F588B67AFB1@oracle.com> <1ca66561-7a4e-da58-f6d3-8f06f97e9f07@redhat.com> Message-ID: <7f958a73-82cb-ee37-0d71-50982dec7013@oracle.com> On 25/03/2020 12:00 am, Aleksey Shipilev wrote: > On 3/24/20 2:51 PM, David Holmes wrote: >> On 24/03/2020 11:19 pm, Frederic Parain wrote: >> I'm somewhat surprised that it has taken 4 weeks for this problem on >> 32-bit to be noticed! :( > > Nobody has time to look at auxiliary CIs in the midst of pandemic! > >> A comment as to why INT_OFFSET and LONG_OFFSET have those values would >> be good. Otherwise fix is fine by me too. > > I'll do this: > > + // 32-bit VMs: @0: 8 byte header, @8: long field, @16: int field > + // 64-bit VMs: @0: 12 byte header, @12: int field, @16: long field > + static final long INT_OFFSET = Platform.is64bit() ? 12L : 16L; > + static final long LONG_OFFSET = Platform.is64bit() ? 16L : 8L; > > Reasonable? Very. Thanks. David From Jie.He at arm.com Wed Mar 25 09:13:57 2020 From: Jie.He at arm.com (Jie He) Date: Wed, 25 Mar 2020 09:13:57 +0000 Subject: RFR(xxs): 8241419: Remove unused InterfaceSupport::_number_of_calls Message-ID: Hi May I please have reviews for this cleanup. The variable InterfaceSupport::_number_of_calls is actually not used. The place to set the variable is in macro TRACE_CALL, as below: ifdef ASSERT #define TRACE_CALL(result_type, header) \ InterfaceSupport::_number_of_calls++; \ if (CountRuntimeCalls) { \ static RuntimeHistogramElement* e = new RuntimeHistogramElement(#header); \ if (e != NULL) e->increment_count(); \ } #define TRACE_CALL(result_type, header) \ /* do nothing */ #endif // ASSERT But can't find the use place of this variable, so assume it's a useless variable. After check the history, found a patch in jdk9 https://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/fb4a19e4f7dc which said, "Removed TraceRuntimeCalls, TraceJNICalls, and TraceJVMCalls options because their output is excessive, not useful, and covered in a more useful way by other options." so I think it's the leftover of the patch, should be removed. Issue: https://bugs.openjdk.java.net/browse/JDK-8241419 webrev: http://cr.openjdk.java.net/~yzhang/8241419/webrev.00/ Thanks Jie He From david.holmes at oracle.com Wed Mar 25 09:43:26 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 25 Mar 2020 19:43:26 +1000 Subject: RFR(xxs): 8241419: Remove unused InterfaceSupport::_number_of_calls In-Reply-To: References: Message-ID: <04e1d4d4-2a69-f779-1cdc-80e912eeec64@oracle.com> Looks good to me. And trivial. Thanks, David On 25/03/2020 7:13 pm, Jie He wrote: > Hi > > May I please have reviews for this cleanup. > > The variable InterfaceSupport::_number_of_calls is actually not used. > > The place to set the variable is in macro TRACE_CALL, as below: > > ifdef ASSERT > #define TRACE_CALL(result_type, header) \ > InterfaceSupport::_number_of_calls++; \ > if (CountRuntimeCalls) { \ > static RuntimeHistogramElement* e = new RuntimeHistogramElement(#header); \ > if (e != NULL) e->increment_count(); \ > } > #define TRACE_CALL(result_type, header) \ > /* do nothing */ > #endif // ASSERT > > But can't find the use place of this variable, so assume it's a useless variable. > > After check the history, found a patch in jdk9 https://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/fb4a19e4f7dc > which said, "Removed TraceRuntimeCalls, TraceJNICalls, and TraceJVMCalls options because their output is excessive, not useful, and covered in a more useful way by other options." > so I think it's the leftover of the patch, should be removed. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8241419 > webrev: http://cr.openjdk.java.net/~yzhang/8241419/webrev.00/ > > Thanks > Jie He > From martin.doerr at sap.com Wed Mar 25 10:09:37 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Wed, 25 Mar 2020 10:09:37 +0000 Subject: RFR(XS): 8241464: [11u] Backport: make rehashing be a needed guaranteed safepoint cleanup action Message-ID: Hi Coleen, thank you for the review. Pushed to jdk11u-dev. Best regards, Martin > -----Original Message----- > From: hotspot-runtime-dev bounces at openjdk.java.net> On Behalf Of coleen.phillimore at oracle.com > Sent: Dienstag, 24. M?rz 2020 21:09 > To: hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR(XS): 8241464: [11u] Backport: make rehashing be > a needed guaranteed safepoint cleanup action > > > This change looks good. > > On 3/24/20 4:07 PM, Doerr, Martin wrote: > > Hi Lutz, > > > >> your change looks good to me. > > Thanks for the review! > > > >> Maybe you want to add a small comment like "// Heal unbalanced hash > >> (done at safepoint)." > > Well, it is still a clean (partial) backport, not a new change, so I don't want to > introduce a diff to the jdk/jdk version. > > I agree. If you want to add this comment, we should have done it in the > current version. > > Coleen > > > > > Best regards, > > Martin > > > > > >> -----Original Message----- > >> From: Schmidt, Lutz > >> Sent: Dienstag, 24. M?rz 2020 21:00 > >> To: Doerr, Martin ; jdk-updates- > >> dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net > >> Subject: Re: RFR(XS): 8241464: [11u] Backport: make rehashing be > >> a needed guaranteed safepoint cleanup action > >> > >> Hi Martin, > >> > >> your change looks good to me. > >> > >> Maybe you want to add a small comment like "// Heal unbalanced hash > >> (done at safepoint)." > >> > >> Thanks, > >> Lutz > >> > >> ?On 23.03.20, 20:06, "hotspot-runtime-dev on behalf of Doerr, Martin" > >> >> martin.doerr at sap.com> wrote: > >> > >> Hi, > >> > >> I'd like to backport a part of JDK-8221967. I'm using a new bug because > it's > >> only a part of the original one. > >> Original change has the summary "Move redundant table lookup and > make > >> rehashing be a needed guaranteed safepoint cleanup action." > >> The first part of it doesn't apply to 11u, because the SymbolTable and > >> StringTable parts are different in 11u and I don't see the need to change > >> them. > >> > >> The second part of it does apply and makes perfect sense. Especially if > >> BiasedLocking is disabled (11u BiasedLocking implementation requests > more > >> safepoints making this fix less important). > >> > >> Original bug: > >> https://bugs.openjdk.java.net/browse/JDK-8221967 > >> > >> Original issue: > >> http://hg.openjdk.java.net/jdk/jdk/rev/2523496f5107 > >> > >> New webrev: > >> > >> > http://cr.openjdk.java.net/~mdoerr/8221967_string_table_partial_backport > >> _11u/webrev.00/ > >> > >> Please review. > >> > >> Best regards, > >> Martin > >> > >> From claes.redestad at oracle.com Wed Mar 25 11:52:37 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 25 Mar 2020 12:52:37 +0100 Subject: RFR[T]: 8241584: Remove unused classLoader perf counters Message-ID: Hi, removing two perf counters that are completely unused: http://cr.openjdk.java.net/~redestad/8241584/open.00/ Thanks! /Claes From harold.seigel at oracle.com Wed Mar 25 12:27:02 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Wed, 25 Mar 2020 08:27:02 -0400 Subject: RFR[T]: 8241584: Remove unused classLoader perf counters In-Reply-To: References: Message-ID: <2d58780c-19ec-981a-89e6-d9c025da85ce@oracle.com> Looks good and trivial. Thanks, Harold On 3/25/2020 7:52 AM, Claes Redestad wrote: > Hi, > > removing two perf counters that are completely unused: > > http://cr.openjdk.java.net/~redestad/8241584/open.00/ > > Thanks! > > /Claes From claes.redestad at oracle.com Wed Mar 25 12:34:17 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 25 Mar 2020 13:34:17 +0100 Subject: RFR[T]: 8241584: Remove unused classLoader perf counters In-Reply-To: <2d58780c-19ec-981a-89e6-d9c025da85ce@oracle.com> References: <2d58780c-19ec-981a-89e6-d9c025da85ce@oracle.com> Message-ID: <1f6e3e15-2339-cada-28a3-5d3acc8df5a6@oracle.com> Thanks, Harold! /Claes On 2020-03-25 13:27, Harold Seigel wrote: > Looks good and trivial. > > Thanks, Harold > > On 3/25/2020 7:52 AM, Claes Redestad wrote: >> Hi, >> >> removing two perf counters that are completely unused: >> >> http://cr.openjdk.java.net/~redestad/8241584/open.00/ >> >> Thanks! >> >> /Claes From thomas.stuefe at gmail.com Wed Mar 25 12:35:41 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 25 Mar 2020 13:35:41 +0100 Subject: RFR(xs): 8241581: Add BitMap::count_one_bits variant for arbitrary lengths Message-ID: Greetings, could I have reviews for this small enhancement please: JBS: https://bugs.openjdk.java.net/browse/JDK-8241581 Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/bitmap-arbitrary-range-popcount/webrev.00/webrev/ This adds the ability to call BitMap::count_one_bits on an arbitrary bit region, not just on the whole BitMap. I need this for the new Metaspace prototype. Note that while it would be cool to use platform dependent popcnt instructions here, that is not the goal of this fix. This just expands on the existing implementation of BitMap::count_one_bits() using the algorithm in population:count.hpp. It comes with quite thorough gtests. I ran the gtests on little and big endian machines and all is well. Submit tests also ran through. Thanks, Thomas From claes.redestad at oracle.com Wed Mar 25 13:16:43 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 25 Mar 2020 14:16:43 +0100 Subject: RFR(xs): 8241581: Add BitMap::count_one_bits variant for arbitrary lengths In-Reply-To: References: Message-ID: <84330583-0b43-a71e-8cef-d6013668131a@oracle.com> Hi, On 2020-03-25 13:35, Thomas St?fe wrote: > Greetings, > > could I have reviews for this small enhancement please: > > JBS: https://bugs.openjdk.java.net/browse/JDK-8241581 > Webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/bitmap-arbitrary-range-popcount/webrev.00/webrev/ looks reasonable. > > This adds the ability to call BitMap::count_one_bits on an arbitrary bit > region, not just on the whole BitMap. I need this for the new Metaspace > prototype. > > Note that while it would be cool to use platform dependent popcnt > instructions here, that is not the goal of this fix. This just expands on > the existing implementation of BitMap::count_one_bits() using the algorithm > in population:count.hpp. Yeah, H/W-enabled popcnt would be great, and population_count.hpp intends to be the place to add that similarly to what's been done for other intrinsics such as count_leading_zeros. However, on x86 we compile for a H/W target that doesn't guarantee popcnt exists.. so intrinsics like gcc's __builtin_popcount are likely to emit something similar to what's currently in population_count.hpp. And last I checked that turned out to be a pessimization in the places where we do 32-bit population counts (C2!). > > It comes with quite thorough gtests. I ran the gtests on little and big > endian machines and all is well. Submit tests also ran through. Good! /Claes From thomas.stuefe at gmail.com Wed Mar 25 13:21:43 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 25 Mar 2020 14:21:43 +0100 Subject: RFR(xs): 8241581: Add BitMap::count_one_bits variant for arbitrary lengths In-Reply-To: <84330583-0b43-a71e-8cef-d6013668131a@oracle.com> References: <84330583-0b43-a71e-8cef-d6013668131a@oracle.com> Message-ID: On Wed, Mar 25, 2020 at 2:14 PM Claes Redestad wrote: > Hi, > > On 2020-03-25 13:35, Thomas St?fe wrote: > > Greetings, > > > > could I have reviews for this small enhancement please: > > > > JBS: https://bugs.openjdk.java.net/browse/JDK-8241581 > > Webrev: > > > http://cr.openjdk.java.net/~stuefe/webrevs/bitmap-arbitrary-range-popcount/webrev.00/webrev/ > > looks reasonable. > > Thanks! > > > > This adds the ability to call BitMap::count_one_bits on an arbitrary bit > > region, not just on the whole BitMap. I need this for the new Metaspace > > prototype. > > > > Note that while it would be cool to use platform dependent popcnt > > instructions here, that is not the goal of this fix. This just expands on > > the existing implementation of BitMap::count_one_bits() using the > algorithm > > in population:count.hpp. > > Yeah, H/W-enabled popcnt would be great, and population_count.hpp > intends to be the place to add that similarly to what's been done for > other intrinsics such as count_leading_zeros. > > However, on x86 we compile for a H/W target that doesn't guarantee > popcnt exists.. so intrinsics like gcc's __builtin_popcount are likely > to emit something similar to what's currently in population_count.hpp. > And last I checked that turned out to be a pessimization in the places > where we do 32-bit population counts (C2!). > I noticed this too when playing around with the builtin. Seems to depend on the march setting we use. Anything below nehalem does not use popcnt. > > > > It comes with quite thorough gtests. I ran the gtests on little and big > > endian machines and all is well. Submit tests also ran through. > > Good! > > /Claes > Thank you! ..Thomas From coleen.phillimore at oracle.com Wed Mar 25 14:49:10 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 25 Mar 2020 10:49:10 -0400 Subject: RFR: 8241234: Unify monitor enter/exit runtime entries In-Reply-To: References: <222D2846-F6AE-4D5B-B41F-F976D90E329C@oracle.com> <91eeada8-e05f-bc73-b029-94e169216a56@oracle.com> Message-ID: On 3/24/20 1:14 PM, Yudi Zheng wrote: > Hi Coleen, > > Thanks for the review! Sorry I missed your email. And thanks Vladimir > for the reminder. > >>> I think you should have monitor_enter_helper and maybe >>> monitor_exi_helper both take Handle as arguments.? This looks very >>> strange. > > Shall we change?ObjectSynchronizer::exit to accept Handle as argument > as well? I think you should. > >>> +void SharedRuntime::monitor_enter_helper(oopDesc* _obj, BasicLock* >>> lock, JavaThread* thread) { >>> Because it creates a Handle anyway.? We try to eagerly pass Handle >>> so that we don't have to search through the code to make sure there >>> aren't any unhandled oops. >>> The _obj parameter name doesn't follow coding conventions (I was >>> looking for it to be a field in something). > > I took the parameter name from the original methods: > JRT_BLOCK_ENTRY(void, > SharedRuntime::complete_monitor_locking_C(oopDesc* _obj, BasicLock* > lock, JavaThread* thread)) > JRT_LEAF(void, SharedRuntime::complete_monitor_unlocking_C(oopDesc* > _obj, BasicLock* lock, JavaThread * THREAD)) > Will Change that. > >>> + oop obj(_obj); >>> Is there not already a implicit conversion from oopDesc* to oop? > > Also took from the original methods. Will get rid of that. Great. thank you. Coleen > > Thanks, > Yudi > >>> Thanks, >>> Coleen >>> On 3/20/20 11:03 AM, Yudi Zheng wrote: >>>> Hello, >>>> >>>> Please review this patch that unifies the monitor enter/exit >>>> runtime entries used by C1, C2, and JVMCI. >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8061553 added support for >>>> faster "slow" locking paths. >>>> These paths are not currently used by JVMCI (or C1) leading to >>>> slower performance on programs >>>> that have a lot of short lived locked sections (such as dacapo:h2). >>>> >>>> This patch creates helper methods based on C2?s monitor enter/exit >>>> entries, and shares with the C1, >>>> JVMCI runtimes. It will prevent these runtimes from missing future >>>> updates. >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8241234 >>>> http://cr.openjdk.java.net/~yzheng/8241234/webrev.00/ >>>> >>>> Many thanks, >>>> Yudi > From claes.redestad at oracle.com Wed Mar 25 16:23:18 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 25 Mar 2020 17:23:18 +0100 Subject: RFR: 8241427: Coarsen locking in Modules::add_module_exports In-Reply-To: <8121467e-74a3-91dd-faba-0ef3303321cc@oracle.com> References: <8121467e-74a3-91dd-faba-0ef3303321cc@oracle.com> Message-ID: <7a3848ba-cd6c-2775-b33a-cb65a416fd87@oracle.com> Hi again, updated after offline discussion with Lois: http://cr.openjdk.java.net/~redestad/8241427/open.01 - replaced package_entry->module() != module_entry checks with an assert in get_locked_package_entry. - removed corresponding throwing of IAEs - refactored logic in add_module_exports_to_all_unnamed to be similar to add_module_exports Thanks! /Claes On 2020-03-23 12:27, Claes Redestad wrote: > Hi, > > in some places we first lookup a PackageEntry - taking and releasing > Module_lock - only to then, after some trivial checks - take the > Module_lock again to update some state in the PackageEntry. > > Coarsening the locking so we only take the mutex once per call brings a > small improvement to bootstrap (reduction of ~200-250k instructions) > > Webrev: http://cr.openjdk.java.net/~redestad/8241427/open.00/ > Bug:??? https://bugs.openjdk.java.net/browse/JDK-8241427 > > Testing: tier1-3 > > Thanks! > > /Claes From calvin.cheung at oracle.com Wed Mar 25 16:39:01 2020 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Wed, 25 Mar 2020 09:39:01 -0700 Subject: RFR(S) 8233093: Move CDS heap oopmaps into new MetaspaceShared::bm region Message-ID: <04266548-45ab-ba45-bb58-f068017a45da@oracle.com> RFE: https://bugs.openjdk.java.net/browse/JDK-8233093 webrev: http://cr.openjdk.java.net/~ccheung/jdk15/8233093/webrev.00/ The MetaspaceShared::bm region was introduced via JDK-8231610 (Relocate the CDS archive if it cannot be mapped to the requested address) and only used during archive loading. The region will be unmapped afterwards to save memory footprint. Since the heap oopmaps are also only used during archive loading, it makes sense for them to reside in the "bm" region as well. Before: [0.753s][debug][cds] ro space: 7552120 [ 59.7% of total] out of 7553024 bytes [100.0% used] at 0x00000008003e1000 [0.753s][debug][cds] bm space: 184320 [ 1.5% of total] out of 184320 bytes [100.0% used] at 0x0000000000000000 After: [0.766s][debug][cds] ro space: 7525856 [ 59.5% of total] out of 7528448 bytes [100.0% used] at 0x00000008003e1000 [0.766s][debug][cds] bm space: 208896 [ 1.7% of total] out of 208896 bytes [100.0% used] at 0x0000000000000000 Passed tiers 1 - 4 testing. thanks, Calvin From harold.seigel at oracle.com Wed Mar 25 17:48:43 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Wed, 25 Mar 2020 13:48:43 -0400 Subject: RFR: 8241427: Coarsen locking in Modules::add_module_exports In-Reply-To: <7a3848ba-cd6c-2775-b33a-cb65a416fd87@oracle.com> References: <8121467e-74a3-91dd-faba-0ef3303321cc@oracle.com> <7a3848ba-cd6c-2775-b33a-cb65a416fd87@oracle.com> Message-ID: <39ff3f8a-8bb9-2297-5d3e-2c8bc2cec042@oracle.com> Hi Claes, It looks like you removed this from modules.cpp.? Can you also remove it from modules.hpp? Modules::get_module(Symbol* package_name, Handle h_loader, TRAPS) Otherwise, the change looks good. Thanks, Harold On 3/25/2020 12:23 PM, Claes Redestad wrote: > Hi again, > > updated after offline discussion with Lois: > > http://cr.openjdk.java.net/~redestad/8241427/open.01 > > - replaced package_entry->module() != module_entry checks with > ? an assert in get_locked_package_entry. > - removed corresponding throwing of IAEs > - refactored logic in add_module_exports_to_all_unnamed to be > ? similar to add_module_exports > > Thanks! > > /Claes > > On 2020-03-23 12:27, Claes Redestad wrote: >> Hi, >> >> in some places we first lookup a PackageEntry - taking and releasing >> Module_lock - only to then, after some trivial checks - take the >> Module_lock again to update some state in the PackageEntry. >> >> Coarsening the locking so we only take the mutex once per call brings a >> small improvement to bootstrap (reduction of ~200-250k instructions) >> >> Webrev: http://cr.openjdk.java.net/~redestad/8241427/open.00/ >> Bug:??? https://bugs.openjdk.java.net/browse/JDK-8241427 >> >> Testing: tier1-3 >> >> Thanks! >> >> /Claes From gerard.ziemski at oracle.com Wed Mar 25 18:03:46 2020 From: gerard.ziemski at oracle.com (gerard ziemski) Date: Wed, 25 Mar 2020 13:03:46 -0500 Subject: RFR (XS): 8236177: assert(status == 0) failed: error ETIMEDOUT(60), cond_wait In-Reply-To: References: Message-ID: <64362c99-c360-1935-b9ac-e7d32e00528d@oracle.com> hi all, Please review this "workaround" for now, which can not be called an actual fix just yet, designed to figure out why on Mac OS X, we get (very rarely) ETIMEDOUT when calling pthread_cond_wait() API. On other hand, it might actually fix it. According to man pages, only EINVAL is allowed as an error code here, but an inspection of the Darwin source code https://opensource.apple.com/source/libpthread/libpthread-301.1.6/src/pthread_cond.c.auto.html reveals that under some circumstances ETIMEDOUT may be returned. The workaround for now is to try the operation again, after a short delay (via yield() API). This will hopefully let us know whether that's enough. If not, we may try to either keep trying the same operation until it succeeds or return an error and let the code above handle it. The workaround applies only to OS X. bug link at https://bugs.openjdk.java.net/browse/JDK-8236177 open webrev at http://cr.openjdk.java.net/~gziemski/8236177_rev1 passes Mach hs-tier1,2,3,4,5 cheers From claes.redestad at oracle.com Wed Mar 25 18:17:28 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 25 Mar 2020 19:17:28 +0100 Subject: RFR: 8241427: Coarsen locking in Modules::add_module_exports In-Reply-To: <39ff3f8a-8bb9-2297-5d3e-2c8bc2cec042@oracle.com> References: <8121467e-74a3-91dd-faba-0ef3303321cc@oracle.com> <7a3848ba-cd6c-2775-b33a-cb65a416fd87@oracle.com> <39ff3f8a-8bb9-2297-5d3e-2c8bc2cec042@oracle.com> Message-ID: Hi, On 2020-03-25 18:48, Harold Seigel wrote: > Hi Claes, > > It looks like you removed this from modules.cpp.? Can you also remove it > from modules.hpp? > > ?? Modules::get_module(Symbol* package_name, Handle h_loader, TRAPS) Done. > > Otherwise, the change looks good. I also updated the webrev to remove 2 tests that poke at the internal API to provoke the IAE we'll no longer be throwing. I'll need to check if there's any other fallout from higher tiers - or if we should dial this back a bit... /Claes From ioi.lam at oracle.com Wed Mar 25 18:26:55 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 25 Mar 2020 11:26:55 -0700 Subject: RFR(S) 8233093: Move CDS heap oopmaps into new MetaspaceShared::bm region In-Reply-To: <04266548-45ab-ba45-bb58-f068017a45da@oracle.com> References: <04266548-45ab-ba45-bb58-f068017a45da@oracle.com> Message-ID: <817fe526-ca3d-ea58-30d7-779ee5133ad4@oracle.com> Hi Calvin, Looks good. Just some minor nits for filemap.cpp: 1244?? if (closed_oopmaps != NULL && open_oopmaps != NULL) { 1245???? size_t curr_offset = write_oopmaps(closed_oopmaps, ptrmap->size_in_bytes(), buffer); 1246???? curr_offset = write_oopmaps(open_oopmaps, curr_offset, buffer); There's no need to write to curr_offset on line 1246. ---- Unnecessary new line at 1897. ---- 1599???? current_info()->unmap_region(MetaspaceShared::bm); This call can be removed, because the "bm" region will be eventually unmapped by the new code in MetaspaceShared::initialize_shared_spaces(). This way, if we later need to relocate the archived heap relocation, there's no need to map the "bm" region again. Thanks - Ioi On 3/25/20 9:39 AM, Calvin Cheung wrote: > RFE: https://bugs.openjdk.java.net/browse/JDK-8233093 > > webrev: http://cr.openjdk.java.net/~ccheung/jdk15/8233093/webrev.00/ > > The MetaspaceShared::bm region was introduced via JDK-8231610 > (Relocate the CDS archive if it cannot be mapped to the requested > address) and only used during archive loading. The region will be > unmapped afterwards to save memory footprint. Since the heap oopmaps > are also only used during archive loading, it makes sense for them to > reside in the "bm" region as well. > > Before: > [0.753s][debug][cds] ro space: 7552120 [ 59.7% of total] out of > 7553024 bytes [100.0% used] at 0x00000008003e1000 > [0.753s][debug][cds] bm space: 184320 [ 1.5% of total] out of 184320 > bytes [100.0% used] at 0x0000000000000000 > > After: > [0.766s][debug][cds] ro space: 7525856 [ 59.5% of total] out of > 7528448 bytes [100.0% used] at 0x00000008003e1000 > [0.766s][debug][cds] bm space: 208896 [ 1.7% of total] out of 208896 > bytes [100.0% used] at 0x0000000000000000 > > Passed tiers 1 - 4 testing. > > thanks, > > Calvin > From harold.seigel at oracle.com Wed Mar 25 18:33:31 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Wed, 25 Mar 2020 14:33:31 -0400 Subject: RFR: 8241427: Coarsen locking in Modules::add_module_exports In-Reply-To: References: <8121467e-74a3-91dd-faba-0ef3303321cc@oracle.com> <7a3848ba-cd6c-2775-b33a-cb65a416fd87@oracle.com> <39ff3f8a-8bb9-2297-5d3e-2c8bc2cec042@oracle.com> Message-ID: <42c50fb2-7019-4a68-79fa-543aa9970dd1@oracle.com> How difficult would it be to restore the IAE? Thanks, Harold On 3/25/2020 2:17 PM, Claes Redestad wrote: > Hi, > > On 2020-03-25 18:48, Harold Seigel wrote: >> Hi Claes, >> >> It looks like you removed this from modules.cpp.? Can you also remove >> it from modules.hpp? >> >> ??? Modules::get_module(Symbol* package_name, Handle h_loader, TRAPS) > > Done. > >> >> Otherwise, the change looks good. > > I also updated the webrev to remove 2 tests that poke at the internal > API to provoke the IAE we'll no longer be throwing. I'll need to check > if there's any other fallout from higher tiers - or if we should dial > this back a bit... > > /Claes From claes.redestad at oracle.com Wed Mar 25 18:49:46 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 25 Mar 2020 19:49:46 +0100 Subject: RFR: 8241427: Coarsen locking in Modules::add_module_exports In-Reply-To: <42c50fb2-7019-4a68-79fa-543aa9970dd1@oracle.com> References: <8121467e-74a3-91dd-faba-0ef3303321cc@oracle.com> <7a3848ba-cd6c-2775-b33a-cb65a416fd87@oracle.com> <39ff3f8a-8bb9-2297-5d3e-2c8bc2cec042@oracle.com> <42c50fb2-7019-4a68-79fa-543aa9970dd1@oracle.com> Message-ID: Not at all difficult, basically go back to open.00 and fix the things that doesn't change the interatction. But this difference is only observable when using the WhiteBox API to call into JVM_AddModuleExports* - all public APIs like Module.addExports(String, Module) will throw an IAE as usual (with tests to check that). I'm ok with dialing this RFE back a notch and file a follow-up for that, though. /Claes On 2020-03-25 19:33, Harold Seigel wrote: > How difficult would it be to restore the IAE? > > Thanks, Harold > > On 3/25/2020 2:17 PM, Claes Redestad wrote: >> Hi, >> >> On 2020-03-25 18:48, Harold Seigel wrote: >>> Hi Claes, >>> >>> It looks like you removed this from modules.cpp.? Can you also remove >>> it from modules.hpp? >>> >>> ??? Modules::get_module(Symbol* package_name, Handle h_loader, TRAPS) >> >> Done. >> >>> >>> Otherwise, the change looks good. >> >> I also updated the webrev to remove 2 tests that poke at the internal >> API to provoke the IAE we'll no longer be throwing. I'll need to check >> if there's any other fallout from higher tiers - or if we should dial >> this back a bit... >> >> /Claes From harold.seigel at oracle.com Wed Mar 25 18:51:39 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Wed, 25 Mar 2020 14:51:39 -0400 Subject: RFR: 8241427: Coarsen locking in Modules::add_module_exports In-Reply-To: References: <8121467e-74a3-91dd-faba-0ef3303321cc@oracle.com> <7a3848ba-cd6c-2775-b33a-cb65a416fd87@oracle.com> <39ff3f8a-8bb9-2297-5d3e-2c8bc2cec042@oracle.com> <42c50fb2-7019-4a68-79fa-543aa9970dd1@oracle.com> Message-ID: I'm okay with the change as is. Thanks, Harold On 3/25/2020 2:49 PM, Claes Redestad wrote: > Not at all difficult, basically go back to open.00 and fix the things > that doesn't change the interatction. But this difference is only > observable when using the WhiteBox API to call into > JVM_AddModuleExports* - all public APIs like Module.addExports(String, > Module) will throw an IAE as usual (with tests to check that). > > I'm ok with dialing this RFE back a notch and file a follow-up for that, > though. > > /Claes > > On 2020-03-25 19:33, Harold Seigel wrote: >> How difficult would it be to restore the IAE? >> >> Thanks, Harold >> >> On 3/25/2020 2:17 PM, Claes Redestad wrote: >>> Hi, >>> >>> On 2020-03-25 18:48, Harold Seigel wrote: >>>> Hi Claes, >>>> >>>> It looks like you removed this from modules.cpp.? Can you also >>>> remove it from modules.hpp? >>>> >>>> ??? Modules::get_module(Symbol* package_name, Handle h_loader, TRAPS) >>> >>> Done. >>> >>>> >>>> Otherwise, the change looks good. >>> >>> I also updated the webrev to remove 2 tests that poke at the internal >>> API to provoke the IAE we'll no longer be throwing. I'll need to check >>> if there's any other fallout from higher tiers - or if we should dial >>> this back a bit... >>> >>> /Claes From lois.foltan at oracle.com Wed Mar 25 19:33:13 2020 From: lois.foltan at oracle.com (Lois Foltan) Date: Wed, 25 Mar 2020 15:33:13 -0400 Subject: RFR: 8241427: Coarsen locking in Modules::add_module_exports In-Reply-To: References: <8121467e-74a3-91dd-faba-0ef3303321cc@oracle.com> <7a3848ba-cd6c-2775-b33a-cb65a416fd87@oracle.com> <39ff3f8a-8bb9-2297-5d3e-2c8bc2cec042@oracle.com> <42c50fb2-7019-4a68-79fa-543aa9970dd1@oracle.com> Message-ID: On 3/25/2020 2:51 PM, Harold Seigel wrote: > I'm okay with the change as is. I'm good with .01 webrev as well.? Thanks for making those changes Claes! Lois > > Thanks, Harold > > On 3/25/2020 2:49 PM, Claes Redestad wrote: >> Not at all difficult, basically go back to open.00 and fix the things >> that doesn't change the interatction. But this difference is only >> observable when using the WhiteBox API to call into >> JVM_AddModuleExports* - all public APIs like Module.addExports(String, >> Module) will throw an IAE as usual (with tests to check that). >> >> I'm ok with dialing this RFE back a notch and file a follow-up for that, >> though. >> >> /Claes >> >> On 2020-03-25 19:33, Harold Seigel wrote: >>> How difficult would it be to restore the IAE? >>> >>> Thanks, Harold >>> >>> On 3/25/2020 2:17 PM, Claes Redestad wrote: >>>> Hi, >>>> >>>> On 2020-03-25 18:48, Harold Seigel wrote: >>>>> Hi Claes, >>>>> >>>>> It looks like you removed this from modules.cpp.? Can you also >>>>> remove it from modules.hpp? >>>>> >>>>> ??? Modules::get_module(Symbol* package_name, Handle h_loader, TRAPS) >>>> >>>> Done. >>>> >>>>> >>>>> Otherwise, the change looks good. >>>> >>>> I also updated the webrev to remove 2 tests that poke at the internal >>>> API to provoke the IAE we'll no longer be throwing. I'll need to check >>>> if there's any other fallout from higher tiers - or if we should dial >>>> this back a bit... >>>> >>>> /Claes From claes.redestad at oracle.com Wed Mar 25 19:42:19 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 25 Mar 2020 20:42:19 +0100 Subject: RFR: 8241427: Coarsen locking in Modules::add_module_exports In-Reply-To: References: <8121467e-74a3-91dd-faba-0ef3303321cc@oracle.com> <7a3848ba-cd6c-2775-b33a-cb65a416fd87@oracle.com> <39ff3f8a-8bb9-2297-5d3e-2c8bc2cec042@oracle.com> <42c50fb2-7019-4a68-79fa-543aa9970dd1@oracle.com> Message-ID: <8c39d4ec-bbae-3ca7-cab5-56cb66f90521@oracle.com> Thanks, I'm running some more tests before push to see that there aren't more tests that hit the assert via WB. /Claes On 2020-03-25 20:33, Lois Foltan wrote: > > > On 3/25/2020 2:51 PM, Harold Seigel wrote: >> I'm okay with the change as is. > > I'm good with .01 webrev as well.? Thanks for making those changes Claes! > Lois > > >> >> Thanks, Harold >> >> On 3/25/2020 2:49 PM, Claes Redestad wrote: >>> Not at all difficult, basically go back to open.00 and fix the things >>> that doesn't change the interatction. But this difference is only >>> observable when using the WhiteBox API to call into >>> JVM_AddModuleExports* - all public APIs like Module.addExports(String, >>> Module) will throw an IAE as usual (with tests to check that). >>> >>> I'm ok with dialing this RFE back a notch and file a follow-up for that, >>> though. >>> >>> /Claes >>> >>> On 2020-03-25 19:33, Harold Seigel wrote: >>>> How difficult would it be to restore the IAE? >>>> >>>> Thanks, Harold >>>> >>>> On 3/25/2020 2:17 PM, Claes Redestad wrote: >>>>> Hi, >>>>> >>>>> On 2020-03-25 18:48, Harold Seigel wrote: >>>>>> Hi Claes, >>>>>> >>>>>> It looks like you removed this from modules.cpp.? Can you also >>>>>> remove it from modules.hpp? >>>>>> >>>>>> ??? Modules::get_module(Symbol* package_name, Handle h_loader, TRAPS) >>>>> >>>>> Done. >>>>> >>>>>> >>>>>> Otherwise, the change looks good. >>>>> >>>>> I also updated the webrev to remove 2 tests that poke at the internal >>>>> API to provoke the IAE we'll no longer be throwing. I'll need to check >>>>> if there's any other fallout from higher tiers - or if we should dial >>>>> this back a bit... >>>>> >>>>> /Claes > From calvin.cheung at oracle.com Wed Mar 25 20:56:32 2020 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Wed, 25 Mar 2020 13:56:32 -0700 Subject: RFR(S) 8233093: Move CDS heap oopmaps into new MetaspaceShared::bm region In-Reply-To: <817fe526-ca3d-ea58-30d7-779ee5133ad4@oracle.com> References: <04266548-45ab-ba45-bb58-f068017a45da@oracle.com> <817fe526-ca3d-ea58-30d7-779ee5133ad4@oracle.com> Message-ID: Hi Ioi, Thanks for your review. On 3/25/20 11:26 AM, Ioi Lam wrote: > Hi Calvin, > > Looks good. Just some minor nits for filemap.cpp: > > 1244?? if (closed_oopmaps != NULL && open_oopmaps != NULL) { > 1245???? size_t curr_offset = write_oopmaps(closed_oopmaps, > ptrmap->size_in_bytes(), buffer); > 1246???? curr_offset = write_oopmaps(open_oopmaps, curr_offset, buffer); > > There's no need to write to curr_offset on line 1246. > Fixed. > ---- > > Unnecessary new line at 1897. > Removed. > ---- > > 1599???? current_info()->unmap_region(MetaspaceShared::bm); > > This call can be removed, because the "bm" region will be eventually > unmapped by the new code in > MetaspaceShared::initialize_shared_spaces(). This way, if we later > need to relocate the archived heap relocation, there's no need to map > the "bm" region again. I've replaced it with a comment. updated webrev: ??? http://cr.openjdk.java.net/~ccheung/jdk15/8233093/webrev.01/ thanks, Calvin > > Thanks > - Ioi > > > On 3/25/20 9:39 AM, Calvin Cheung wrote: >> RFE: https://bugs.openjdk.java.net/browse/JDK-8233093 >> >> webrev: http://cr.openjdk.java.net/~ccheung/jdk15/8233093/webrev.00/ >> >> The MetaspaceShared::bm region was introduced via JDK-8231610 >> (Relocate the CDS archive if it cannot be mapped to the requested >> address) and only used during archive loading. The region will be >> unmapped afterwards to save memory footprint. Since the heap oopmaps >> are also only used during archive loading, it makes sense for them to >> reside in the "bm" region as well. >> >> Before: >> [0.753s][debug][cds] ro space: 7552120 [ 59.7% of total] out of >> 7553024 bytes [100.0% used] at 0x00000008003e1000 >> [0.753s][debug][cds] bm space: 184320 [ 1.5% of total] out of 184320 >> bytes [100.0% used] at 0x0000000000000000 >> >> After: >> [0.766s][debug][cds] ro space: 7525856 [ 59.5% of total] out of >> 7528448 bytes [100.0% used] at 0x00000008003e1000 >> [0.766s][debug][cds] bm space: 208896 [ 1.7% of total] out of 208896 >> bytes [100.0% used] at 0x0000000000000000 >> >> Passed tiers 1 - 4 testing. >> >> thanks, >> >> Calvin >> > From david.holmes at oracle.com Wed Mar 25 23:43:09 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 26 Mar 2020 09:43:09 +1000 Subject: RFR (XS): 8236177: assert(status == 0) failed: error ETIMEDOUT(60), cond_wait In-Reply-To: <64362c99-c360-1935-b9ac-e7d32e00528d@oracle.com> References: <64362c99-c360-1935-b9ac-e7d32e00528d@oracle.com> Message-ID: Hi Gerard, On 26/03/2020 4:03 am, gerard ziemski wrote: > hi all, > > Please review this "workaround" for now, which can not be called an > actual fix just yet, designed to figure out why on Mac OS X, we get > (very rarely) ETIMEDOUT when calling pthread_cond_wait() API. On other > hand, it might actually fix it. The ETIMEDOUT should be treated as a "spurious wakeup" and we will naturally retry the wait if the condition is not yet met. All we have to do to our code is adjust the assert so that ETIMEDOUT doesn't cause it to fail. Thanks, David > According to man pages, only EINVAL is allowed as an error code here, > but an inspection of the Darwin source code > https://opensource.apple.com/source/libpthread/libpthread-301.1.6/src/pthread_cond.c.auto.html > reveals that under some circumstances ETIMEDOUT may be returned. > > The workaround for now is to try the operation again, after a short > delay (via yield() API). This will hopefully let us know whether that's > enough. If not, we may try to either keep trying the same operation > until it succeeds or return an error and let the code above handle it. > > The workaround applies only to OS X. > > bug link at https://bugs.openjdk.java.net/browse/JDK-8236177 > open webrev at http://cr.openjdk.java.net/~gziemski/8236177_rev1 > passes Mach hs-tier1,2,3,4,5 > > > cheers From david.holmes at oracle.com Wed Mar 25 23:54:25 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 26 Mar 2020 09:54:25 +1000 Subject: RFR (XS): 8236177: assert(status == 0) failed: error ETIMEDOUT(60), cond_wait In-Reply-To: References: <64362c99-c360-1935-b9ac-e7d32e00528d@oracle.com> Message-ID: <4b505a62-b125-b8a1-a2d4-c3fb15ca4df7@oracle.com> On 26/03/2020 9:43 am, David Holmes wrote: > Hi Gerard, > > On 26/03/2020 4:03 am, gerard ziemski wrote: >> hi all, >> >> Please review this "workaround" for now, which can not be called an >> actual fix just yet, designed to figure out why on Mac OS X, we get >> (very rarely) ETIMEDOUT when calling pthread_cond_wait() API. On other >> hand, it might actually fix it. > > The ETIMEDOUT should be treated as a "spurious wakeup" and we will > naturally retry the wait if the condition is not yet met. All we have to > do to our code is adjust the assert so that ETIMEDOUT doesn't cause it > to fail. And I'm happy to explicitly show this is a macOS issue (though we don't have an MACOS_ONLY macro) assert_status(status == 0 BSD_ONLY(|| status == ETIMEDOUT), status, "cond_wait"); David > Thanks, > David > >> According to man pages, only EINVAL is allowed as an error code here, >> but an inspection of the Darwin source code >> https://opensource.apple.com/source/libpthread/libpthread-301.1.6/src/pthread_cond.c.auto.html >> reveals that under some circumstances ETIMEDOUT may be returned. >> >> The workaround for now is to try the operation again, after a short >> delay (via yield() API). This will hopefully let us know whether >> that's enough. If not, we may try to either keep trying the same >> operation until it succeeds or return an error and let the code above >> handle it. >> >> The workaround applies only to OS X. >> >> bug link at https://bugs.openjdk.java.net/browse/JDK-8236177 >> open webrev at http://cr.openjdk.java.net/~gziemski/8236177_rev1 >> passes Mach hs-tier1,2,3,4,5 >> >> >> cheers From ioi.lam at oracle.com Thu Mar 26 05:21:01 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 25 Mar 2020 22:21:01 -0700 Subject: RFR(S) 8233093: Move CDS heap oopmaps into new MetaspaceShared::bm region In-Reply-To: References: <04266548-45ab-ba45-bb58-f068017a45da@oracle.com> <817fe526-ca3d-ea58-30d7-779ee5133ad4@oracle.com> Message-ID: Look good! Thanks - Ioi On 3/25/20 1:56 PM, Calvin Cheung wrote: > Hi Ioi, > > Thanks for your review. > > On 3/25/20 11:26 AM, Ioi Lam wrote: >> Hi Calvin, >> >> Looks good. Just some minor nits for filemap.cpp: >> >> 1244?? if (closed_oopmaps != NULL && open_oopmaps != NULL) { >> 1245???? size_t curr_offset = write_oopmaps(closed_oopmaps, >> ptrmap->size_in_bytes(), buffer); >> 1246???? curr_offset = write_oopmaps(open_oopmaps, curr_offset, buffer); >> >> There's no need to write to curr_offset on line 1246. >> > Fixed. >> ---- >> >> Unnecessary new line at 1897. >> > Removed. >> ---- >> >> 1599???? current_info()->unmap_region(MetaspaceShared::bm); >> >> This call can be removed, because the "bm" region will be eventually >> unmapped by the new code in >> MetaspaceShared::initialize_shared_spaces(). This way, if we later >> need to relocate the archived heap relocation, there's no need to map >> the "bm" region again. > > I've replaced it with a comment. > > updated webrev: > > ??? http://cr.openjdk.java.net/~ccheung/jdk15/8233093/webrev.01/ > > thanks, > > Calvin > >> >> Thanks >> - Ioi >> >> >> On 3/25/20 9:39 AM, Calvin Cheung wrote: >>> RFE: https://bugs.openjdk.java.net/browse/JDK-8233093 >>> >>> webrev: http://cr.openjdk.java.net/~ccheung/jdk15/8233093/webrev.00/ >>> >>> The MetaspaceShared::bm region was introduced via JDK-8231610 >>> (Relocate the CDS archive if it cannot be mapped to the requested >>> address) and only used during archive loading. The region will be >>> unmapped afterwards to save memory footprint. Since the heap oopmaps >>> are also only used during archive loading, it makes sense for them >>> to reside in the "bm" region as well. >>> >>> Before: >>> [0.753s][debug][cds] ro space: 7552120 [ 59.7% of total] out of >>> 7553024 bytes [100.0% used] at 0x00000008003e1000 >>> [0.753s][debug][cds] bm space: 184320 [ 1.5% of total] out of 184320 >>> bytes [100.0% used] at 0x0000000000000000 >>> >>> After: >>> [0.766s][debug][cds] ro space: 7525856 [ 59.5% of total] out of >>> 7528448 bytes [100.0% used] at 0x00000008003e1000 >>> [0.766s][debug][cds] bm space: 208896 [ 1.7% of total] out of 208896 >>> bytes [100.0% used] at 0x0000000000000000 >>> >>> Passed tiers 1 - 4 testing. >>> >>> thanks, >>> >>> Calvin >>> >> From coleen.phillimore at oracle.com Thu Mar 26 12:15:51 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 26 Mar 2020 08:15:51 -0400 Subject: RFR (T) 8172485: [TESTBUG] RedefineLeak.java runs out of metaspace memory Message-ID: <9b63315b-2c23-a6fb-8a9a-e457edb3bcac@oracle.com> Summary: Check for zero exit value. It doesn't run out of memory anymore.? Bugs appears down for maintenance, but that's what I said there too. Ran tier2 on all Oracle platforms, where the test is run. open webrev at http://cr.openjdk.java.net/~coleenp/2020/8172485.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8172485 Thanks, Coleen From lois.foltan at oracle.com Thu Mar 26 12:26:31 2020 From: lois.foltan at oracle.com (Lois Foltan) Date: Thu, 26 Mar 2020 08:26:31 -0400 Subject: RFR (T) 8172485: [TESTBUG] RedefineLeak.java runs out of metaspace memory In-Reply-To: <9b63315b-2c23-a6fb-8a9a-e457edb3bcac@oracle.com> References: <9b63315b-2c23-a6fb-8a9a-e457edb3bcac@oracle.com> Message-ID: <05c0ebf6-19c5-d00e-9926-067e933eb13b@oracle.com> Looks good & trivial. Lois On 3/26/2020 8:15 AM, coleen.phillimore at oracle.com wrote: > Summary: Check for zero exit value. > > It doesn't run out of memory anymore.? Bugs appears down for > maintenance, but that's what I said there too. > > Ran tier2 on all Oracle platforms, where the test is run. > > open webrev at http://cr.openjdk.java.net/~coleenp/2020/8172485.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8172485 > > Thanks, > Coleen From yudi.zheng at oracle.com Thu Mar 26 13:48:38 2020 From: yudi.zheng at oracle.com (Yudi Zheng) Date: Thu, 26 Mar 2020 14:48:38 +0100 Subject: RFR: 8241234: Unify monitor enter/exit runtime entries In-Reply-To: References: <222D2846-F6AE-4D5B-B41F-F976D90E329C@oracle.com> <91eeada8-e05f-bc73-b029-94e169216a56@oracle.com> Message-ID: Hi Coleen, The monitor exit entries are marked with JRT_LEAF which contains > debug_only(NoHandleMark __hm;) \ This disallows the creation of Handle within the monitor exit entries. > # A fatal error has been detected by the Java Runtime Environment: > # > # Internal Error (open/src/hotspot/share/runtime/handles.cpp:35), pid=28631, tid=28687 > # assert(_no_handle_mark_nesting == 0) failed: allocating handle inside NoHandleMark I have uploaded a version that addresses the following comments: 1. Have monitor_enter_helper take Handle as arguments. 2. Avoid using _obj as parameter name as it indicates a field according to the coding conventions. 3. oop obj(_obj); is not necessary as there is an implicit conversion from oopDesc* to oop See http://cr.openjdk.java.net/~yzheng/8241234/webrev.02/ Let me know if there is any other problem. Many thanks, Yudi > On 25 Mar 2020, at 15:49, coleen.phillimore at oracle.com wrote: > > > > On 3/24/20 1:14 PM, Yudi Zheng wrote: >> Hi Coleen, >> >> Thanks for the review! Sorry I missed your email. And thanks Vladimir for the reminder. >> >>>> I think you should have monitor_enter_helper and maybe monitor_exi_helper both take Handle as arguments. This looks very strange. >> >> Shall we change ObjectSynchronizer::exit to accept Handle as argument as well? > > I think you should. >> >>>> +void SharedRuntime::monitor_enter_helper(oopDesc* _obj, BasicLock* lock, JavaThread* thread) { >>>> Because it creates a Handle anyway. We try to eagerly pass Handle so that we don't have to search through the code to make sure there aren't any unhandled oops. >>>> The _obj parameter name doesn't follow coding conventions (I was looking for it to be a field in something). >> >> I took the parameter name from the original methods: >> JRT_BLOCK_ENTRY(void, SharedRuntime::complete_monitor_locking_C(oopDesc* _obj, BasicLock* lock, JavaThread* thread)) >> JRT_LEAF(void, SharedRuntime::complete_monitor_unlocking_C(oopDesc* _obj, BasicLock* lock, JavaThread * THREAD)) >> Will Change that. >> >>>> + oop obj(_obj); >>>> Is there not already a implicit conversion from oopDesc* to oop? >> >> Also took from the original methods. Will get rid of that. > > Great. thank you. > Coleen >> >> Thanks, >> Yudi >> >>>> Thanks, >>>> Coleen >>>> On 3/20/20 11:03 AM, Yudi Zheng wrote: >>>>> Hello, >>>>> >>>>> Please review this patch that unifies the monitor enter/exit runtime entries used by C1, C2, and JVMCI. >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8061553 added support for faster "slow" locking paths. >>>>> These paths are not currently used by JVMCI (or C1) leading to slower performance on programs >>>>> that have a lot of short lived locked sections (such as dacapo:h2). >>>>> >>>>> This patch creates helper methods based on C2?s monitor enter/exit entries, and shares with the C1, >>>>> JVMCI runtimes. It will prevent these runtimes from missing future updates. >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8241234 >>>>> http://cr.openjdk.java.net/~yzheng/8241234/webrev.00/ >>>>> >>>>> Many thanks, >>>>> Yudi From coleen.phillimore at oracle.com Thu Mar 26 14:25:02 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 26 Mar 2020 10:25:02 -0400 Subject: RFR: 8241234: Unify monitor enter/exit runtime entries In-Reply-To: References: <222D2846-F6AE-4D5B-B41F-F976D90E329C@oracle.com> <91eeada8-e05f-bc73-b029-94e169216a56@oracle.com> Message-ID: <534b8cf7-cd8c-565b-5163-09a216d4f94e@oracle.com> On 3/26/20 9:48 AM, Yudi Zheng wrote: > Hi Coleen, > > The monitor exit entries are marked with?JRT_LEAF which contains > > ?debug_only(NoHandleMark __hm;) ? ? ? ? ? ? ? ? ? ? ? \ > > This disallows the creation of Handle within the monitor exit entries. > > # A fatal error has been detected by the Java Runtime Environment: > > # > > # Internal Error (open/src/hotspot/share/runtime/handles.cpp:35), pid=28631, tid=28687 > > # assert(_no_handle_mark_nesting == 0) failed: allocating handle inside NoHandleMark You are right.? Thanks for pointing this out. > I have uploaded a version that addresses the following comments: > 1. Have monitor_enter_helper take Handle as arguments. > 2. Avoid using _obj as parameter name as it indicates a field > according to the coding conventions. > 3. oop obj(_obj); is not necessary as there is an implicit conversion > from oopDesc* to oop > > See http://cr.openjdk.java.net/~yzheng/8241234/webrev.02/ > Actually, I apologize that I think I am wrong about making monitor_enter_helper take a Handle because I don't think you can create Handle from a thread that isn't _thread_in_vm. +// Handles the uncommon case in locking, i.e., contention or an inflated lock. +JRT_BLOCK_ENTRY(void, SharedRuntime::complete_monitor_locking_C(oopDesc* obj, BasicLock* lock, JavaThread* thread)) + Handle h_obj(thread, obj); + SharedRuntime::monitor_enter_helper(h_obj, lock, thread); +JRT_END So I think SharedRuntime::monitor_enter_helper() shouldn't take a Handle and that would allow all paths to have this fast path: if (!SafepointSynchronize::is_synchronizing()) { // Only try quick_enter() if we're not trying to reach a safepoint // so that the calling thread reaches the safepoint more quickly. - if (ObjectSynchronizer::quick_enter(_obj, thread, lock)) return; + if (ObjectSynchronizer::quick_enter(h_obj(), thread, lock)) return; } Then create the handle in the JRT_BLOCK_NO_ASYNC where it was. Maybe this gets you back to your original webrev.? Sorry about this. Thanks, Coleen > Let me know if there is any other problem. > > Many thanks, > Yudi > >> On 25 Mar 2020, at 15:49, coleen.phillimore at oracle.com >> wrote: >> >> >> >> On 3/24/20 1:14 PM, Yudi Zheng wrote: >>> Hi Coleen, >>> >>> Thanks for the review! Sorry I missed your email. And thanks >>> Vladimir for the reminder. >>> >>>>> I think you should have monitor_enter_helper and maybe >>>>> monitor_exi_helper both take Handle as arguments.? This looks very >>>>> strange. >>> >>> Shall we change?ObjectSynchronizer::exit to accept Handle as >>> argument as well? >> >> I think you should. >>> >>>>> +void SharedRuntime::monitor_enter_helper(oopDesc* _obj, >>>>> BasicLock* lock, JavaThread* thread) { >>>>> Because it creates a Handle anyway.? We try to eagerly pass Handle >>>>> so that we don't have to search through the code to make sure >>>>> there aren't any unhandled oops. >>>>> The _obj parameter name doesn't follow coding conventions (I was >>>>> looking for it to be a field in something). >>> >>> I took the parameter name from the original methods: >>> JRT_BLOCK_ENTRY(void, >>> SharedRuntime::complete_monitor_locking_C(oopDesc* _obj, BasicLock* >>> lock, JavaThread* thread)) >>> JRT_LEAF(void, SharedRuntime::complete_monitor_unlocking_C(oopDesc* >>> _obj, BasicLock* lock, JavaThread * THREAD)) >>> Will Change that. >>> >>>>> + oop obj(_obj); >>>>> Is there not already a implicit conversion from oopDesc* to oop? >>> >>> Also took from the original methods. Will get rid of that. >> >> Great. thank you. >> Coleen >>> >>> Thanks, >>> Yudi >>> >>>>> Thanks, >>>>> Coleen >>>>> On 3/20/20 11:03 AM, Yudi Zheng wrote: >>>>>> Hello, >>>>>> >>>>>> Please review this patch that unifies the monitor enter/exit >>>>>> runtime entries used by C1, C2, and JVMCI. >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8061553added support for >>>>>> faster "slow" locking paths. >>>>>> These paths are not currently used by JVMCI (or C1) leading to >>>>>> slower performance on programs >>>>>> that have a lot of short lived locked sections (such as dacapo:h2). >>>>>> >>>>>> This patch creates helper methods based on C2?s monitor >>>>>> enter/exit entries, and shares with the C1, >>>>>> JVMCI runtimes. It will prevent these runtimes from missing >>>>>> future updates. >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8241234 >>>>>> http://cr.openjdk.java.net/~yzheng/8241234/webrev.00/ >>>>>> >>>>>> Many thanks, >>>>>> Yudi > From coleen.phillimore at oracle.com Thu Mar 26 14:28:48 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 26 Mar 2020 10:28:48 -0400 Subject: RFR (T) 8172485: [TESTBUG] RedefineLeak.java runs out of metaspace memory In-Reply-To: <05c0ebf6-19c5-d00e-9926-067e933eb13b@oracle.com> References: <9b63315b-2c23-a6fb-8a9a-e457edb3bcac@oracle.com> <05c0ebf6-19c5-d00e-9926-067e933eb13b@oracle.com> Message-ID: <479b7000-17c0-c1dd-4dc2-675b89321cd8@oracle.com> Thanks Lois! Coleen On 3/26/20 8:26 AM, Lois Foltan wrote: > Looks good & trivial. > Lois > > On 3/26/2020 8:15 AM, coleen.phillimore at oracle.com wrote: >> Summary: Check for zero exit value. >> >> It doesn't run out of memory anymore.? Bugs appears down for >> maintenance, but that's what I said there too. >> >> Ran tier2 on all Oracle platforms, where the test is run. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8172485.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8172485 >> >> Thanks, >> Coleen > From yudi.zheng at oracle.com Thu Mar 26 15:22:21 2020 From: yudi.zheng at oracle.com (Yudi Zheng) Date: Thu, 26 Mar 2020 16:22:21 +0100 Subject: RFR: 8241234: Unify monitor enter/exit runtime entries In-Reply-To: <534b8cf7-cd8c-565b-5163-09a216d4f94e@oracle.com> References: <222D2846-F6AE-4D5B-B41F-F976D90E329C@oracle.com> <91eeada8-e05f-bc73-b029-94e169216a56@oracle.com> <534b8cf7-cd8c-565b-5163-09a216d4f94e@oracle.com> Message-ID: Thanks for the comment! Here is the new version: http://cr.openjdk.java.net/~yzheng/8241234/webrev.03/ Differences w.r.t. webrev.00 are that: 1. parameter _obj is now renamed to obj 2. oop obj(_obj) is removed Many thanks, Yudi > On 26 Mar 2020, at 15:25, coleen.phillimore at oracle.com wrote: > > > > On 3/26/20 9:48 AM, Yudi Zheng wrote: >> Hi Coleen, >> >> The monitor exit entries are marked with JRT_LEAF which contains >> > debug_only(NoHandleMark __hm;) \ >> >> This disallows the creation of Handle within the monitor exit entries. >> > # A fatal error has been detected by the Java Runtime Environment: >> > # >> > # Internal Error (open/src/hotspot/share/runtime/handles.cpp:35), pid=28631, tid=28687 >> > # assert(_no_handle_mark_nesting == 0) failed: allocating handle inside NoHandleMark > > You are right. Thanks for pointing this out. >> I have uploaded a version that addresses the following comments: >> 1. Have monitor_enter_helper take Handle as arguments. >> 2. Avoid using _obj as parameter name as it indicates a field according to the coding conventions. >> 3. oop obj(_obj); is not necessary as there is an implicit conversion from oopDesc* to oop >> >> See http://cr.openjdk.java.net/~yzheng/8241234/webrev.02/ >> > > Actually, I apologize that I think I am wrong about making monitor_enter_helper take a Handle because I don't think you can create Handle from a thread that isn't _thread_in_vm. > > +// Handles the uncommon case in locking, i.e., contention or an inflated lock. > +JRT_BLOCK_ENTRY(void, SharedRuntime::complete_monitor_locking_C(oopDesc* obj, BasicLock* lock, JavaThread* thread)) > + Handle h_obj(thread, obj); > + SharedRuntime::monitor_enter_helper(h_obj, lock, thread); > +JRT_END > > So I think SharedRuntime::monitor_enter_helper() shouldn't take a Handle and that would allow all paths to have this fast path: > > if (!SafepointSynchronize::is_synchronizing()) { > // Only try quick_enter() if we're not trying to reach a safepoint > // so that the calling thread reaches the safepoint more quickly. > - if (ObjectSynchronizer::quick_enter(_obj, thread, lock)) return; > + if (ObjectSynchronizer::quick_enter(h_obj(), thread, lock)) return; > } > > Then create the handle in the JRT_BLOCK_NO_ASYNC where it was. Maybe this gets you back to your original webrev. Sorry about this. > > Thanks, > Coleen > >> Let me know if there is any other problem. >> >> Many thanks, >> Yudi >> >>> On 25 Mar 2020, at 15:49, coleen.phillimore at oracle.com wrote: >>> >>> >>> >>> On 3/24/20 1:14 PM, Yudi Zheng wrote: >>>> Hi Coleen, >>>> >>>> Thanks for the review! Sorry I missed your email. And thanks Vladimir for the reminder. >>>> >>>>>> I think you should have monitor_enter_helper and maybe monitor_exi_helper both take Handle as arguments. This looks very strange. >>>> >>>> Shall we change ObjectSynchronizer::exit to accept Handle as argument as well? >>> >>> I think you should. >>>> >>>>>> +void SharedRuntime::monitor_enter_helper(oopDesc* _obj, BasicLock* lock, JavaThread* thread) { >>>>>> Because it creates a Handle anyway. We try to eagerly pass Handle so that we don't have to search through the code to make sure there aren't any unhandled oops. >>>>>> The _obj parameter name doesn't follow coding conventions (I was looking for it to be a field in something). >>>> >>>> I took the parameter name from the original methods: >>>> JRT_BLOCK_ENTRY(void, SharedRuntime::complete_monitor_locking_C(oopDesc* _obj, BasicLock* lock, JavaThread* thread)) >>>> JRT_LEAF(void, SharedRuntime::complete_monitor_unlocking_C(oopDesc* _obj, BasicLock* lock, JavaThread * THREAD)) >>>> Will Change that. >>>> >>>>>> + oop obj(_obj); >>>>>> Is there not already a implicit conversion from oopDesc* to oop? >>>> >>>> Also took from the original methods. Will get rid of that. >>> >>> Great. thank you. >>> Coleen >>>> >>>> Thanks, >>>> Yudi >>>> >>>>>> Thanks, >>>>>> Coleen >>>>>> On 3/20/20 11:03 AM, Yudi Zheng wrote: >>>>>>> Hello, >>>>>>> >>>>>>> Please review this patch that unifies the monitor enter/exit runtime entries used by C1, C2, and JVMCI. >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8061553 added support for faster "slow" locking paths. >>>>>>> These paths are not currently used by JVMCI (or C1) leading to slower performance on programs >>>>>>> that have a lot of short lived locked sections (such as dacapo:h2). >>>>>>> >>>>>>> This patch creates helper methods based on C2?s monitor enter/exit entries, and shares with the C1, >>>>>>> JVMCI runtimes. It will prevent these runtimes from missing future updates. >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8241234 >>>>>>> http://cr.openjdk.java.net/~yzheng/8241234/webrev.00/ >>>>>>> >>>>>>> Many thanks, >>>>>>> Yudi >> > From yumin.qi at oracle.com Thu Mar 26 16:36:37 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Thu, 26 Mar 2020 09:36:37 -0700 Subject: RFR: 8241234: Unify monitor enter/exit runtime entries In-Reply-To: References: <222D2846-F6AE-4D5B-B41F-F976D90E329C@oracle.com> <91eeada8-e05f-bc73-b029-94e169216a56@oracle.com> <534b8cf7-cd8c-565b-5163-09a216d4f94e@oracle.com> Message-ID: <1a2bd9bd-3e9c-e6ba-aaae-2990493c3e23@oracle.com> HI, Yudi and Coleen jvmciRuntime.cpp: 385 JRT_BLOCK_ENTRY(void, JVMCIRuntime::monitorenter(JavaThread* thread, oopDesc* obj, BasicLock* lock)) 386 IF_TRACE_jvmci_3 { 387 char type[O_BUFLEN]; 388 obj->klass()->name()->as_C_string(type, O_BUFLEN); 389 markWord mark = obj->mark(); 390 TRACE_jvmci_3("%s: entered locking slow case with obj=" INTPTR_FORMAT ", type=%s, mark=" INTPTR_FORMAT ", lock=" INTPTR_FORMAT, thread->name(), p2i(obj), type, mark.value(), p2i(lock)); 391 tty->flush(); 392 } 393 SharedRuntime::monitor_enter_helper(obj, lock, thread); 394 TRACE_jvmci_3("%s: exiting locking slow with obj=" INTPTR_FORMAT, thread->name(), p2i(obj)); 395 JRT_END line 394: first it is not guarded in IF_TRACE_jvmci_3 second I think it should be removed (I am not sure what it means here). Thanks Yumin On 3/26/20 8:22 AM, Yudi Zheng wrote: > Thanks for the comment! > > Here is the new version: http://cr.openjdk.java.net/~yzheng/8241234/webrev.03/ > > Differences w.r.t. webrev.00 are that: > 1. parameter _obj is now renamed to obj > 2. oop obj(_obj) is removed > > Many thanks, > Yudi > > >> On 26 Mar 2020, at 15:25, coleen.phillimore at oracle.com wrote: >> >> >> >> On 3/26/20 9:48 AM, Yudi Zheng wrote: >>> Hi Coleen, >>> >>> The monitor exit entries are marked with JRT_LEAF which contains >>>> debug_only(NoHandleMark __hm;) \ >>> This disallows the creation of Handle within the monitor exit entries. >>>> # A fatal error has been detected by the Java Runtime Environment: >>>> # >>>> # Internal Error (open/src/hotspot/share/runtime/handles.cpp:35), pid=28631, tid=28687 >>>> # assert(_no_handle_mark_nesting == 0) failed: allocating handle inside NoHandleMark >> You are right. Thanks for pointing this out. >>> I have uploaded a version that addresses the following comments: >>> 1. Have monitor_enter_helper take Handle as arguments. >>> 2. Avoid using _obj as parameter name as it indicates a field according to the coding conventions. >>> 3. oop obj(_obj); is not necessary as there is an implicit conversion from oopDesc* to oop >>> >>> See http://cr.openjdk.java.net/~yzheng/8241234/webrev.02/ >>> >> Actually, I apologize that I think I am wrong about making monitor_enter_helper take a Handle because I don't think you can create Handle from a thread that isn't _thread_in_vm. >> >> +// Handles the uncommon case in locking, i.e., contention or an inflated lock. >> +JRT_BLOCK_ENTRY(void, SharedRuntime::complete_monitor_locking_C(oopDesc* obj, BasicLock* lock, JavaThread* thread)) >> + Handle h_obj(thread, obj); >> + SharedRuntime::monitor_enter_helper(h_obj, lock, thread); >> +JRT_END >> >> So I think SharedRuntime::monitor_enter_helper() shouldn't take a Handle and that would allow all paths to have this fast path: >> >> if (!SafepointSynchronize::is_synchronizing()) { >> // Only try quick_enter() if we're not trying to reach a safepoint >> // so that the calling thread reaches the safepoint more quickly. >> - if (ObjectSynchronizer::quick_enter(_obj, thread, lock)) return; >> + if (ObjectSynchronizer::quick_enter(h_obj(), thread, lock)) return; >> } >> >> Then create the handle in the JRT_BLOCK_NO_ASYNC where it was. Maybe this gets you back to your original webrev. Sorry about this. >> >> Thanks, >> Coleen >> >>> Let me know if there is any other problem. >>> >>> Many thanks, >>> Yudi >>> >>>> On 25 Mar 2020, at 15:49, coleen.phillimore at oracle.com wrote: >>>> >>>> >>>> >>>> On 3/24/20 1:14 PM, Yudi Zheng wrote: >>>>> Hi Coleen, >>>>> >>>>> Thanks for the review! Sorry I missed your email. And thanks Vladimir for the reminder. >>>>> >>>>>>> I think you should have monitor_enter_helper and maybe monitor_exi_helper both take Handle as arguments. This looks very strange. >>>>> Shall we change ObjectSynchronizer::exit to accept Handle as argument as well? >>>> I think you should. >>>>>>> +void SharedRuntime::monitor_enter_helper(oopDesc* _obj, BasicLock* lock, JavaThread* thread) { >>>>>>> Because it creates a Handle anyway. We try to eagerly pass Handle so that we don't have to search through the code to make sure there aren't any unhandled oops. >>>>>>> The _obj parameter name doesn't follow coding conventions (I was looking for it to be a field in something). >>>>> I took the parameter name from the original methods: >>>>> JRT_BLOCK_ENTRY(void, SharedRuntime::complete_monitor_locking_C(oopDesc* _obj, BasicLock* lock, JavaThread* thread)) >>>>> JRT_LEAF(void, SharedRuntime::complete_monitor_unlocking_C(oopDesc* _obj, BasicLock* lock, JavaThread * THREAD)) >>>>> Will Change that. >>>>> >>>>>>> + oop obj(_obj); >>>>>>> Is there not already a implicit conversion from oopDesc* to oop? >>>>> Also took from the original methods. Will get rid of that. >>>> Great. thank you. >>>> Coleen >>>>> Thanks, >>>>> Yudi >>>>> >>>>>>> Thanks, >>>>>>> Coleen >>>>>>> On 3/20/20 11:03 AM, Yudi Zheng wrote: >>>>>>>> Hello, >>>>>>>> >>>>>>>> Please review this patch that unifies the monitor enter/exit runtime entries used by C1, C2, and JVMCI. >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8061553 added support for faster "slow" locking paths. >>>>>>>> These paths are not currently used by JVMCI (or C1) leading to slower performance on programs >>>>>>>> that have a lot of short lived locked sections (such as dacapo:h2). >>>>>>>> >>>>>>>> This patch creates helper methods based on C2?s monitor enter/exit entries, and shares with the C1, >>>>>>>> JVMCI runtimes. It will prevent these runtimes from missing future updates. >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8241234 >>>>>>>> http://cr.openjdk.java.net/~yzheng/8241234/webrev.00/ >>>>>>>> >>>>>>>> Many thanks, >>>>>>>> Yudi From coleen.phillimore at oracle.com Thu Mar 26 16:39:18 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 26 Mar 2020 12:39:18 -0400 Subject: RFR: 8241234: Unify monitor enter/exit runtime entries In-Reply-To: References: <222D2846-F6AE-4D5B-B41F-F976D90E329C@oracle.com> <91eeada8-e05f-bc73-b029-94e169216a56@oracle.com> <534b8cf7-cd8c-565b-5163-09a216d4f94e@oracle.com> Message-ID: This looks good to me. Thank you. Coleen On 3/26/20 11:22 AM, Yudi Zheng wrote: > Thanks for the comment! > > Here is the new version: > http://cr.openjdk.java.net/~yzheng/8241234/webrev.03/ > > Differences w.r.t. webrev.00 are that: > 1. parameter _obj is now renamed to obj > 2. oop obj(_obj) is removed > > Many thanks, > Yudi > > >> On 26 Mar 2020, at 15:25, coleen.phillimore at oracle.com >> wrote: >> >> >> >> On 3/26/20 9:48 AM, Yudi Zheng wrote: >>> Hi Coleen, >>> >>> The monitor exit entries are marked with?JRT_LEAF which contains >>> > ?debug_only(NoHandleMark __hm;) ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? \ >>> >>> This disallows the creation of Handle within the monitor exit entries. >>> > # A fatal error has been detected by the Java Runtime Environment: >>> > # >>> > # Internal Error (open/src/hotspot/share/runtime/handles.cpp:35), pid=28631, tid=28687 >>> > # assert(_no_handle_mark_nesting == 0) failed: allocating handle inside NoHandleMark >> >> You are right.? Thanks for pointing this out. >>> I have uploaded a version that addresses the following comments: >>> 1. Have monitor_enter_helper take Handle as arguments. >>> 2. Avoid using _obj as parameter name as it indicates a field >>> according to the coding conventions. >>> 3. oop obj(_obj); is not necessary as there is an implicit >>> conversion from oopDesc* to oop >>> >>> See http://cr.openjdk.java.net/~yzheng/8241234/webrev.02/ >>> >> >> Actually, I apologize that I think I am wrong about making >> monitor_enter_helper take a Handle because I don't think you can >> create Handle from a thread that isn't _thread_in_vm. >> >> +// Handles the uncommon case in locking, i.e., contention or an >> inflated lock. >> +JRT_BLOCK_ENTRY(void, >> SharedRuntime::complete_monitor_locking_C(oopDesc* obj, BasicLock* >> lock, JavaThread* thread)) >> + Handle h_obj(thread, obj); >> + SharedRuntime::monitor_enter_helper(h_obj, lock, thread); >> +JRT_END >> >> So I think SharedRuntime::monitor_enter_helper() shouldn't take a >> Handle and that would allow all paths to have this fast path: >> >> if (!SafepointSynchronize::is_synchronizing()) { >> // Only try quick_enter() if we're not trying to reach a safepoint >> // so that the calling thread reaches the safepoint more quickly. >> - if (ObjectSynchronizer::quick_enter(_obj, thread, lock)) return; >> + if (ObjectSynchronizer::quick_enter(h_obj(), thread, lock)) return; >> } >> >> Then create the handle in the JRT_BLOCK_NO_ASYNC where it was. Maybe >> this gets you back to your original webrev.? Sorry about this. >> >> Thanks, >> Coleen >> >>> Let me know if there is any other problem. >>> >>> Many thanks, >>> Yudi >>> >>>> On 25 Mar 2020, at 15:49, coleen.phillimore at oracle.com >>>> wrote: >>>> >>>> >>>> >>>> On 3/24/20 1:14 PM, Yudi Zheng wrote: >>>>> Hi Coleen, >>>>> >>>>> Thanks for the review! Sorry I missed your email. And thanks >>>>> Vladimir for the reminder. >>>>> >>>>>>> I think you should have monitor_enter_helper and maybe >>>>>>> monitor_exi_helper both take Handle as arguments.? This looks >>>>>>> very strange. >>>>> >>>>> Shall we change?ObjectSynchronizer::exit to accept Handle as >>>>> argument as well? >>>> >>>> I think you should. >>>>> >>>>>>> +void SharedRuntime::monitor_enter_helper(oopDesc* _obj, >>>>>>> BasicLock* lock, JavaThread* thread) { >>>>>>> Because it creates a Handle anyway.? We try to eagerly pass >>>>>>> Handle so that we don't have to search through the code to make >>>>>>> sure there aren't any unhandled oops. >>>>>>> The _obj parameter name doesn't follow coding conventions (I was >>>>>>> looking for it to be a field in something). >>>>> >>>>> I took the parameter name from the original methods: >>>>> JRT_BLOCK_ENTRY(void, >>>>> SharedRuntime::complete_monitor_locking_C(oopDesc* _obj, >>>>> BasicLock* lock, JavaThread* thread)) >>>>> JRT_LEAF(void, >>>>> SharedRuntime::complete_monitor_unlocking_C(oopDesc* _obj, >>>>> BasicLock* lock, JavaThread * THREAD)) >>>>> Will Change that. >>>>> >>>>>>> + oop obj(_obj); >>>>>>> Is there not already a implicit conversion from oopDesc* to oop? >>>>> >>>>> Also took from the original methods. Will get rid of that. >>>> >>>> Great. thank you. >>>> Coleen >>>>> >>>>> Thanks, >>>>> Yudi >>>>> >>>>>>> Thanks, >>>>>>> Coleen >>>>>>> On 3/20/20 11:03 AM, Yudi Zheng wrote: >>>>>>>> Hello, >>>>>>>> >>>>>>>> Please review this patch that unifies the monitor enter/exit >>>>>>>> runtime entries used by C1, C2, and JVMCI. >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8061553added support >>>>>>>> for faster "slow" locking paths. >>>>>>>> These paths are not currently used by JVMCI (or C1) leading to >>>>>>>> slower performance on programs >>>>>>>> that have a lot of short lived locked sections (such as dacapo:h2). >>>>>>>> >>>>>>>> This patch creates helper methods based on C2?s monitor >>>>>>>> enter/exit entries, and shares with the C1, >>>>>>>> JVMCI runtimes. It will prevent these runtimes from missing >>>>>>>> future updates. >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8241234 >>>>>>>> http://cr.openjdk.java.net/~yzheng/8241234/webrev.00/ >>>>>>>> >>>>>>>> Many thanks, >>>>>>>> Yudi >>> >> > From coleen.phillimore at oracle.com Thu Mar 26 16:44:22 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 26 Mar 2020 12:44:22 -0400 Subject: RFR: 8241234: Unify monitor enter/exit runtime entries In-Reply-To: <1a2bd9bd-3e9c-e6ba-aaae-2990493c3e23@oracle.com> References: <222D2846-F6AE-4D5B-B41F-F976D90E329C@oracle.com> <91eeada8-e05f-bc73-b029-94e169216a56@oracle.com> <534b8cf7-cd8c-565b-5163-09a216d4f94e@oracle.com> <1a2bd9bd-3e9c-e6ba-aaae-2990493c3e23@oracle.com> Message-ID: <3f8ba3f8-66ca-c56c-b4b5-0234097de72c@oracle.com> On 3/26/20 12:36 PM, Yumin Qi wrote: > HI, Yudi and Coleen > > jvmciRuntime.cpp: > > ?385 JRT_BLOCK_ENTRY(void, JVMCIRuntime::monitorenter(JavaThread* > thread, oopDesc* obj, BasicLock* lock)) > ?386?? IF_TRACE_jvmci_3 { > ?387???? char type[O_BUFLEN]; > ?388???? obj->klass()->name()->as_C_string(type, O_BUFLEN); > ?389???? markWord mark = obj->mark(); > ?390???? TRACE_jvmci_3("%s: entered locking slow case with obj=" > INTPTR_FORMAT ", type=%s, mark=" INTPTR_FORMAT ", lock=" > INTPTR_FORMAT, thread->name(), p2i(obj), type, mark.value(), p2i(lock)); > ?391???? tty->flush(); > ?392?? } > ?393?? SharedRuntime::monitor_enter_helper(obj, lock, thread); > ?394?? TRACE_jvmci_3("%s: exiting locking slow with obj=" > INTPTR_FORMAT, thread->name(), p2i(obj)); > ?395 JRT_END > > line 394: > first it is not guarded in IF_TRACE_jvmci_3 > second I think it should be removed (I am not sure what it means here). I don't know what this jvmci code does either, but decided not to comment on it because it was pre-existing.?? I am not sure if this code can access metadata like obj->klass() with the _thread_in_java state.? Maybe it can.? It seems like something that can be investigated separately. Coleen > > Thanks > Yumin > > On 3/26/20 8:22 AM, Yudi Zheng wrote: >> Thanks for the comment! >> >> Here is the new version: >> http://cr.openjdk.java.net/~yzheng/8241234/webrev.03/ >> >> >> Differences w.r.t. webrev.00 are that: >> 1. parameter _obj is now renamed to obj >> 2. oop obj(_obj) is removed >> >> Many thanks, >> Yudi >> >> >>> On 26 Mar 2020, at 15:25, coleen.phillimore at oracle.com wrote: >>> >>> >>> >>> On 3/26/20 9:48 AM, Yudi Zheng wrote: >>>> Hi Coleen, >>>> >>>> The monitor exit entries are marked with JRT_LEAF which contains >>>>> ? debug_only(NoHandleMark >>>>> __hm;)???????????????????????????????????? \ >>>> This disallows the creation of Handle within the monitor exit entries. >>>>> # A fatal error has been detected by the Java Runtime Environment: >>>>> # >>>>> #? Internal Error (open/src/hotspot/share/runtime/handles.cpp:35), >>>>> pid=28631, tid=28687 >>>>> #? assert(_no_handle_mark_nesting == 0) failed: allocating handle >>>>> inside NoHandleMark >>> You are right.? Thanks for pointing this out. >>>> I have uploaded a version that addresses the following comments: >>>> 1. Have monitor_enter_helper take Handle as arguments. >>>> 2. Avoid using _obj as parameter name as it indicates a field >>>> according to the coding conventions. >>>> 3. oop obj(_obj); is not necessary as there is an implicit >>>> conversion from oopDesc* to oop >>>> >>>> See http://cr.openjdk.java.net/~yzheng/8241234/webrev.02/ >>>> >>>> >>> Actually, I apologize that I think I am wrong about making >>> monitor_enter_helper take a Handle because I don't think you can >>> create Handle from a thread that isn't _thread_in_vm. >>> >>> +// Handles the uncommon case in locking, i.e., contention or an >>> inflated lock. >>> +JRT_BLOCK_ENTRY(void, >>> SharedRuntime::complete_monitor_locking_C(oopDesc* obj, BasicLock* >>> lock, JavaThread* thread)) >>> +? Handle h_obj(thread, obj); >>> +? SharedRuntime::monitor_enter_helper(h_obj, lock, thread); >>> +JRT_END >>> ? So I think SharedRuntime::monitor_enter_helper() shouldn't take a >>> Handle and that would allow all paths to have this fast path: >>> >>> ??? if (!SafepointSynchronize::is_synchronizing()) { >>> ????? // Only try quick_enter() if we're not trying to reach a >>> safepoint >>> ????? // so that the calling thread reaches the safepoint more quickly. >>> -??? if (ObjectSynchronizer::quick_enter(_obj, thread, lock)) return; >>> +??? if (ObjectSynchronizer::quick_enter(h_obj(), thread, lock)) >>> return; >>> ??? } >>> >>> Then create the handle in the JRT_BLOCK_NO_ASYNC where it was. Maybe >>> this gets you back to your original webrev.? Sorry about this. >>> >>> Thanks, >>> Coleen >>> >>>> Let me know if there is any other problem. >>>> >>>> Many thanks, >>>> Yudi >>>> >>>>> On 25 Mar 2020, at 15:49, coleen.phillimore at oracle.com >>>>> wrote: >>>>> >>>>> >>>>> >>>>> On 3/24/20 1:14 PM, Yudi Zheng wrote: >>>>>> Hi Coleen, >>>>>> >>>>>> Thanks for the review! Sorry I missed your email. And thanks >>>>>> Vladimir for the reminder. >>>>>> >>>>>>>> I think you should have monitor_enter_helper and maybe >>>>>>>> monitor_exi_helper both take Handle as arguments.? This looks >>>>>>>> very strange. >>>>>> Shall we change ObjectSynchronizer::exit to accept Handle as >>>>>> argument as well? >>>>> I think you should. >>>>>>>> +void SharedRuntime::monitor_enter_helper(oopDesc* _obj, >>>>>>>> BasicLock* lock, JavaThread* thread) { >>>>>>>> Because it creates a Handle anyway.? We try to eagerly pass >>>>>>>> Handle so that we don't have to search through the code to make >>>>>>>> sure there aren't any unhandled oops. >>>>>>>> The _obj parameter name doesn't follow coding conventions (I >>>>>>>> was looking for it to be a field in something). >>>>>> I took the parameter name from the original methods: >>>>>> JRT_BLOCK_ENTRY(void, >>>>>> SharedRuntime::complete_monitor_locking_C(oopDesc* _obj, >>>>>> BasicLock* lock, JavaThread* thread)) >>>>>> JRT_LEAF(void, >>>>>> SharedRuntime::complete_monitor_unlocking_C(oopDesc* _obj, >>>>>> BasicLock* lock, JavaThread * THREAD)) >>>>>> Will Change that. >>>>>> >>>>>>>> + oop obj(_obj); >>>>>>>> Is there not already a implicit conversion from oopDesc* to oop? >>>>>> Also took from the original methods. Will get rid of that. >>>>> Great. thank you. >>>>> Coleen >>>>>> Thanks, >>>>>> Yudi >>>>>> >>>>>>>> Thanks, >>>>>>>> Coleen >>>>>>>> On 3/20/20 11:03 AM, Yudi Zheng wrote: >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> Please review this patch that unifies the monitor enter/exit >>>>>>>>> runtime entries used by C1, C2, and JVMCI. >>>>>>>>> >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8061553 >>>>>>>>> added >>>>>>>>> support for faster "slow" locking paths. >>>>>>>>> These paths are not currently used by JVMCI (or C1) leading to >>>>>>>>> slower performance on programs >>>>>>>>> that have a lot of short lived locked sections (such as >>>>>>>>> dacapo:h2). >>>>>>>>> >>>>>>>>> This patch creates helper methods based on C2?s monitor >>>>>>>>> enter/exit entries, and shares with the C1, >>>>>>>>> JVMCI runtimes. It will prevent these runtimes from missing >>>>>>>>> future updates. >>>>>>>>> >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8241234 >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~yzheng/8241234/webrev.00/ >>>>>>>>> >>>>>>>>> >>>>>>>>> Many thanks, >>>>>>>>> Yudi > From yumin.qi at oracle.com Thu Mar 26 16:49:19 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Thu, 26 Mar 2020 09:49:19 -0700 Subject: RFR(S) 8233093: Move CDS heap oopmaps into new MetaspaceShared::bm region In-Reply-To: References: <04266548-45ab-ba45-bb58-f068017a45da@oracle.com> <817fe526-ca3d-ea58-30d7-779ee5133ad4@oracle.com> Message-ID: <46a11738-6563-ebe6-6142-6c4d77e9a15b@oracle.com> Hi, Calvin Looks good to me. Thanks Yumin On 3/25/20 1:56 PM, Calvin Cheung wrote: > Hi Ioi, > > Thanks for your review. > > On 3/25/20 11:26 AM, Ioi Lam wrote: >> Hi Calvin, >> >> Looks good. Just some minor nits for filemap.cpp: >> >> 1244?? if (closed_oopmaps != NULL && open_oopmaps != NULL) { >> 1245???? size_t curr_offset = write_oopmaps(closed_oopmaps, >> ptrmap->size_in_bytes(), buffer); >> 1246???? curr_offset = write_oopmaps(open_oopmaps, curr_offset, buffer); >> >> There's no need to write to curr_offset on line 1246. >> > Fixed. >> ---- >> >> Unnecessary new line at 1897. >> > Removed. >> ---- >> >> 1599???? current_info()->unmap_region(MetaspaceShared::bm); >> >> This call can be removed, because the "bm" region will be eventually >> unmapped by the new code in >> MetaspaceShared::initialize_shared_spaces(). This way, if we later >> need to relocate the archived heap relocation, there's no need to map >> the "bm" region again. > > I've replaced it with a comment. > > updated webrev: > > ??? http://cr.openjdk.java.net/~ccheung/jdk15/8233093/webrev.01/ > > thanks, > > Calvin > >> >> Thanks >> - Ioi >> >> >> On 3/25/20 9:39 AM, Calvin Cheung wrote: >>> RFE: https://bugs.openjdk.java.net/browse/JDK-8233093 >>> >>> webrev: http://cr.openjdk.java.net/~ccheung/jdk15/8233093/webrev.00/ >>> >>> The MetaspaceShared::bm region was introduced via JDK-8231610 >>> (Relocate the CDS archive if it cannot be mapped to the requested >>> address) and only used during archive loading. The region will be >>> unmapped afterwards to save memory footprint. Since the heap oopmaps >>> are also only used during archive loading, it makes sense for them >>> to reside in the "bm" region as well. >>> >>> Before: >>> [0.753s][debug][cds] ro space: 7552120 [ 59.7% of total] out of >>> 7553024 bytes [100.0% used] at 0x00000008003e1000 >>> [0.753s][debug][cds] bm space: 184320 [ 1.5% of total] out of 184320 >>> bytes [100.0% used] at 0x0000000000000000 >>> >>> After: >>> [0.766s][debug][cds] ro space: 7525856 [ 59.5% of total] out of >>> 7528448 bytes [100.0% used] at 0x00000008003e1000 >>> [0.766s][debug][cds] bm space: 208896 [ 1.7% of total] out of 208896 >>> bytes [100.0% used] at 0x0000000000000000 >>> >>> Passed tiers 1 - 4 testing. >>> >>> thanks, >>> >>> Calvin >>> >> From calvin.cheung at oracle.com Thu Mar 26 17:17:33 2020 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Thu, 26 Mar 2020 10:17:33 -0700 Subject: RFR(S) 8233093: Move CDS heap oopmaps into new MetaspaceShared::bm region In-Reply-To: <46a11738-6563-ebe6-6142-6c4d77e9a15b@oracle.com> References: <04266548-45ab-ba45-bb58-f068017a45da@oracle.com> <817fe526-ca3d-ea58-30d7-779ee5133ad4@oracle.com> <46a11738-6563-ebe6-6142-6c4d77e9a15b@oracle.com> Message-ID: Thanks Yumin and Ioi. I've pushed the changeset. Calvin On 3/26/20 9:49 AM, Yumin Qi wrote: > Hi, Calvin > > Looks good to me. > > Thanks > Yumin > > > > On 3/25/20 1:56 PM, Calvin Cheung wrote: >> Hi Ioi, >> >> Thanks for your review. >> >> On 3/25/20 11:26 AM, Ioi Lam wrote: >>> Hi Calvin, >>> >>> Looks good. Just some minor nits for filemap.cpp: >>> >>> 1244?? if (closed_oopmaps != NULL && open_oopmaps != NULL) { >>> 1245???? size_t curr_offset = write_oopmaps(closed_oopmaps, >>> ptrmap->size_in_bytes(), buffer); >>> 1246???? curr_offset = write_oopmaps(open_oopmaps, curr_offset, >>> buffer); >>> >>> There's no need to write to curr_offset on line 1246. >>> >> Fixed. >>> ---- >>> >>> Unnecessary new line at 1897. >>> >> Removed. >>> ---- >>> >>> 1599???? current_info()->unmap_region(MetaspaceShared::bm); >>> >>> This call can be removed, because the "bm" region will be eventually >>> unmapped by the new code in >>> MetaspaceShared::initialize_shared_spaces(). This way, if we later >>> need to relocate the archived heap relocation, there's no need to >>> map the "bm" region again. >> >> I've replaced it with a comment. >> >> updated webrev: >> >> ??? http://cr.openjdk.java.net/~ccheung/jdk15/8233093/webrev.01/ >> >> thanks, >> >> Calvin >> >>> >>> Thanks >>> - Ioi >>> >>> >>> On 3/25/20 9:39 AM, Calvin Cheung wrote: >>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8233093 >>>> >>>> webrev: http://cr.openjdk.java.net/~ccheung/jdk15/8233093/webrev.00/ >>>> >>>> The MetaspaceShared::bm region was introduced via JDK-8231610 >>>> (Relocate the CDS archive if it cannot be mapped to the requested >>>> address) and only used during archive loading. The region will be >>>> unmapped afterwards to save memory footprint. Since the heap >>>> oopmaps are also only used during archive loading, it makes sense >>>> for them to reside in the "bm" region as well. >>>> >>>> Before: >>>> [0.753s][debug][cds] ro space: 7552120 [ 59.7% of total] out of >>>> 7553024 bytes [100.0% used] at 0x00000008003e1000 >>>> [0.753s][debug][cds] bm space: 184320 [ 1.5% of total] out of >>>> 184320 bytes [100.0% used] at 0x0000000000000000 >>>> >>>> After: >>>> [0.766s][debug][cds] ro space: 7525856 [ 59.5% of total] out of >>>> 7528448 bytes [100.0% used] at 0x00000008003e1000 >>>> [0.766s][debug][cds] bm space: 208896 [ 1.7% of total] out of >>>> 208896 bytes [100.0% used] at 0x0000000000000000 >>>> >>>> Passed tiers 1 - 4 testing. >>>> >>>> thanks, >>>> >>>> Calvin >>>> >>> > From claes.redestad at oracle.com Thu Mar 26 17:40:13 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Thu, 26 Mar 2020 18:40:13 +0100 Subject: RFR: 8241585: Remove unused _recursion_counter facility from PerfTraceTime Message-ID: <91a8ebbc-522d-bd67-6304-c9e097bd8366@oracle.com> Hi, PerfTraceTime::_recursion_counter is unused, and removing it gets rid of some branchy (but well-predicted) code in paths that is somewhat startup sensitive. http://cr.openjdk.java.net/~redestad/8241585/open.00/ Also added some trace logging to determine the number of perf data counter or each type along with a tune-up to exactly match the defaults. Testing: tier1+2 Thanks! /Claes From ioi.lam at oracle.com Thu Mar 26 19:12:34 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 26 Mar 2020 12:12:34 -0700 Subject: RFR: 8241585: Remove unused _recursion_counter facility from PerfTraceTime In-Reply-To: <91a8ebbc-522d-bd67-6304-c9e097bd8366@oracle.com> References: <91a8ebbc-522d-bd67-6304-c9e097bd8366@oracle.com> Message-ID: <1c544a24-838c-6e67-242e-03fd7cf04fbc@oracle.com> Hi Claes, This looks good to me. I did a casual search of the function that you removed, and couldn't find anyone that actually calls them. Thanks - Ioi On 3/26/20 10:40 AM, Claes Redestad wrote: > Hi, > > PerfTraceTime::_recursion_counter is unused, and removing it > gets rid of some branchy (but well-predicted) code in paths that is > somewhat startup sensitive. > > http://cr.openjdk.java.net/~redestad/8241585/open.00/ > > Also added some trace logging to determine the number of perf > data counter or each type along with a tune-up to exactly match > the defaults. > > Testing: tier1+2 > > Thanks! > > /Claes From patricio.chilano.mateo at oracle.com Thu Mar 26 19:23:56 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Thu, 26 Mar 2020 16:23:56 -0300 Subject: RFR 8231264: Implementation of JEP 374: Disable biased-locking and deprecate all flags related to biased-locking Message-ID: <589a66e6-82c0-435f-cf01-7a82fe7a3b69@oracle.com> Hi all, Please review the following implementation for JEP 374: Disable and Deprecate Biased Locking. As already discussed we will change the default value for UseBiasedLocking to false. Deprecating the flags will also give more visibility to the change and should increase our changes to potentially get more feedback.? After a transition/evaluation period and based on collected evidence that will complement the results of our own benchmarks we can decide if we want to keep this feature or not. I modified test VMDeprecatedOptions.java to include the deprecated flags, except for the two diagnostic ones (PrintBiasedLockingStatistics, PrintPreciseBiasedLockingStatistics). Tested in mach5, tiers1-7 on all platforms (Linux-x64, macOS-x64, Windows-x64, Solaris-sparcv9). Let me know if there are more tests I should run. Bug: ? https://bugs.openjdk.java.net/browse/JDK-8231264 Webrev: ? http://cr.openjdk.java.net/~pchilanomate/8231264/v1/webrev/ Thanks, Patricio From kim.barrett at oracle.com Thu Mar 26 19:42:31 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 26 Mar 2020 15:42:31 -0400 Subject: RFR(xs): 8241581: Add BitMap::count_one_bits variant for arbitrary lengths In-Reply-To: References: Message-ID: <1FA2C1F8-4C25-49C4-AA83-7C92702119C4@oracle.com> > On Mar 25, 2020, at 8:35 AM, Thomas St?fe wrote: > > Greetings, > > could I have reviews for this small enhancement please: > > JBS: https://bugs.openjdk.java.net/browse/JDK-8241581 > Webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/bitmap-arbitrary-range-popcount/webrev.00/webrev/ > > This adds the ability to call BitMap::count_one_bits on an arbitrary bit > region, not just on the whole BitMap. I need this for the new Metaspace > prototype. > > Note that while it would be cool to use platform dependent popcnt > instructions here, that is not the goal of this fix. This just expands on > the existing implementation of BitMap::count_one_bits() using the algorithm > in population:count.hpp. > > It comes with quite thorough gtests. I ran the gtests on little and big > endian machines and all is well. Submit tests also ran through. > > Thanks, Thomas Looks good. From claes.redestad at oracle.com Thu Mar 26 20:17:54 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Thu, 26 Mar 2020 21:17:54 +0100 Subject: RFR: 8241585: Remove unused _recursion_counter facility from PerfTraceTime In-Reply-To: <1c544a24-838c-6e67-242e-03fd7cf04fbc@oracle.com> References: <91a8ebbc-522d-bd67-6304-c9e097bd8366@oracle.com> <1c544a24-838c-6e67-242e-03fd7cf04fbc@oracle.com> Message-ID: On 2020-03-26 20:12, Ioi Lam wrote: > Hi Claes, > > This looks good to me. Thanks, Ioi! /Claes From thomas.stuefe at gmail.com Thu Mar 26 20:19:05 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 26 Mar 2020 21:19:05 +0100 Subject: RFR(xs): 8241581: Add BitMap::count_one_bits variant for arbitrary lengths In-Reply-To: <1FA2C1F8-4C25-49C4-AA83-7C92702119C4@oracle.com> References: <1FA2C1F8-4C25-49C4-AA83-7C92702119C4@oracle.com> Message-ID: Thanks Kim. On Thu 26. Mar 2020 at 20:42, Kim Barrett wrote: > > On Mar 25, 2020, at 8:35 AM, Thomas St?fe > wrote: > > > > Greetings, > > > > could I have reviews for this small enhancement please: > > > > JBS: https://bugs.openjdk.java.net/browse/JDK-8241581 > > Webrev: > > > http://cr.openjdk.java.net/~stuefe/webrevs/bitmap-arbitrary-range-popcount/webrev.00/webrev/ > > > > This adds the ability to call BitMap::count_one_bits on an arbitrary bit > > region, not just on the whole BitMap. I need this for the new Metaspace > > prototype. > > > > Note that while it would be cool to use platform dependent popcnt > > instructions here, that is not the goal of this fix. This just expands on > > the existing implementation of BitMap::count_one_bits() using the > algorithm > > in population:count.hpp. > > > > It comes with quite thorough gtests. I ran the gtests on little and big > > endian machines and all is well. Submit tests also ran through. > > > > Thanks, Thomas > > Looks good. > > From david.holmes at oracle.com Thu Mar 26 22:50:18 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 27 Mar 2020 08:50:18 +1000 Subject: RFR 8231264: Implementation of JEP 374: Disable biased-locking and deprecate all flags related to biased-locking In-Reply-To: <589a66e6-82c0-435f-cf01-7a82fe7a3b69@oracle.com> References: <589a66e6-82c0-435f-cf01-7a82fe7a3b69@oracle.com> Message-ID: <60f2691b-210a-d337-37a7-226e7313d633@oracle.com> Hi Patricio, This looks good to me. Ready to go once the JEP is targeted. Thanks, David On 27/03/2020 5:23 am, Patricio Chilano wrote: > Hi all, > > Please review the following implementation for JEP 374: Disable and > Deprecate Biased Locking. > > As already discussed we will change the default value for > UseBiasedLocking to false. Deprecating the flags will also give more > visibility to the change and should increase our changes to potentially > get more feedback.? After a transition/evaluation period and based on > collected evidence that will complement the results of our own > benchmarks we can decide if we want to keep this feature or not. > > I modified test VMDeprecatedOptions.java to include the deprecated > flags, except for the two diagnostic ones (PrintBiasedLockingStatistics, > PrintPreciseBiasedLockingStatistics). > > Tested in mach5, tiers1-7 on all platforms (Linux-x64, macOS-x64, > Windows-x64, Solaris-sparcv9). > > Let me know if there are more tests I should run. > > Bug: > ? https://bugs.openjdk.java.net/browse/JDK-8231264 > > Webrev: > ? http://cr.openjdk.java.net/~pchilanomate/8231264/v1/webrev/ > > > Thanks, > > Patricio From david.holmes at oracle.com Thu Mar 26 22:58:29 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 27 Mar 2020 08:58:29 +1000 Subject: RFR: 8241234: Unify monitor enter/exit runtime entries In-Reply-To: <1a2bd9bd-3e9c-e6ba-aaae-2990493c3e23@oracle.com> References: <222D2846-F6AE-4D5B-B41F-F976D90E329C@oracle.com> <91eeada8-e05f-bc73-b029-94e169216a56@oracle.com> <534b8cf7-cd8c-565b-5163-09a216d4f94e@oracle.com> <1a2bd9bd-3e9c-e6ba-aaae-2990493c3e23@oracle.com> Message-ID: On 27/03/2020 2:36 am, Yumin Qi wrote: > HI, Yudi and Coleen > > jvmciRuntime.cpp: > > ?385 JRT_BLOCK_ENTRY(void, JVMCIRuntime::monitorenter(JavaThread* > thread, oopDesc* obj, BasicLock* lock)) > ?386?? IF_TRACE_jvmci_3 { > ?387???? char type[O_BUFLEN]; > ?388???? obj->klass()->name()->as_C_string(type, O_BUFLEN); > ?389???? markWord mark = obj->mark(); > ?390???? TRACE_jvmci_3("%s: entered locking slow case with obj=" > INTPTR_FORMAT ", type=%s, mark=" INTPTR_FORMAT ", lock=" INTPTR_FORMAT, > thread->name(), p2i(obj), type, mark.value(), p2i(lock)); > ?391???? tty->flush(); > ?392?? } > ?393?? SharedRuntime::monitor_enter_helper(obj, lock, thread); > ?394?? TRACE_jvmci_3("%s: exiting locking slow with obj=" > INTPTR_FORMAT, thread->name(), p2i(obj)); > ?395 JRT_END > > line 394: > first it is not guarded in IF_TRACE_jvmci_3 It doesn't need to be. An explicit IF_TRACE_jcmvi is only needed to guard code that you don't want to execute unless tracing e.g. setting up the temp buffer as per the preceding code. > second I think it should be removed (I am not sure what it means here). It is just a logging statement showing the monitor enter has completed - it matches the previous log statement. David ----- > Thanks > Yumin > > On 3/26/20 8:22 AM, Yudi Zheng wrote: >> Thanks for the comment! >> >> Here is the new version: >> http://cr.openjdk.java.net/~yzheng/8241234/webrev.03/ >> >> >> Differences w.r.t. webrev.00 are that: >> 1. parameter _obj is now renamed to obj >> 2. oop obj(_obj) is removed >> >> Many thanks, >> Yudi >> >> >>> On 26 Mar 2020, at 15:25, coleen.phillimore at oracle.com wrote: >>> >>> >>> >>> On 3/26/20 9:48 AM, Yudi Zheng wrote: >>>> Hi Coleen, >>>> >>>> The monitor exit entries are marked with JRT_LEAF which contains >>>>> ? debug_only(NoHandleMark __hm;)???????????????????????????????????? \ >>>> This disallows the creation of Handle within the monitor exit entries. >>>>> # A fatal error has been detected by the Java Runtime Environment: >>>>> # >>>>> #? Internal Error (open/src/hotspot/share/runtime/handles.cpp:35), >>>>> pid=28631, tid=28687 >>>>> #? assert(_no_handle_mark_nesting == 0) failed: allocating handle >>>>> inside NoHandleMark >>> You are right.? Thanks for pointing this out. >>>> I have uploaded a version that addresses the following comments: >>>> 1. Have monitor_enter_helper take Handle as arguments. >>>> 2. Avoid using _obj as parameter name as it indicates a field >>>> according to the coding conventions. >>>> 3. oop obj(_obj); is not necessary as there is an implicit >>>> conversion from oopDesc* to oop >>>> >>>> See http://cr.openjdk.java.net/~yzheng/8241234/webrev.02/ >>>> >>>> >>> Actually, I apologize that I think I am wrong about making >>> monitor_enter_helper take a Handle because I don't think you can >>> create Handle from a thread that isn't _thread_in_vm. >>> >>> +// Handles the uncommon case in locking, i.e., contention or an >>> inflated lock. >>> +JRT_BLOCK_ENTRY(void, >>> SharedRuntime::complete_monitor_locking_C(oopDesc* obj, BasicLock* >>> lock, JavaThread* thread)) >>> +? Handle h_obj(thread, obj); >>> +? SharedRuntime::monitor_enter_helper(h_obj, lock, thread); >>> +JRT_END >>> So I think SharedRuntime::monitor_enter_helper() shouldn't take a >>> Handle and that would allow all paths to have this fast path: >>> >>> ??? if (!SafepointSynchronize::is_synchronizing()) { >>> ????? // Only try quick_enter() if we're not trying to reach a safepoint >>> ????? // so that the calling thread reaches the safepoint more quickly. >>> -??? if (ObjectSynchronizer::quick_enter(_obj, thread, lock)) return; >>> +??? if (ObjectSynchronizer::quick_enter(h_obj(), thread, lock)) return; >>> ??? } >>> >>> Then create the handle in the JRT_BLOCK_NO_ASYNC where it was. Maybe >>> this gets you back to your original webrev.? Sorry about this. >>> >>> Thanks, >>> Coleen >>> >>>> Let me know if there is any other problem. >>>> >>>> Many thanks, >>>> Yudi >>>> >>>>> On 25 Mar 2020, at 15:49, coleen.phillimore at oracle.com >>>>> wrote: >>>>> >>>>> >>>>> >>>>> On 3/24/20 1:14 PM, Yudi Zheng wrote: >>>>>> Hi Coleen, >>>>>> >>>>>> Thanks for the review! Sorry I missed your email. And thanks >>>>>> Vladimir for the reminder. >>>>>> >>>>>>>> I think you should have monitor_enter_helper and maybe >>>>>>>> monitor_exi_helper both take Handle as arguments.? This looks >>>>>>>> very strange. >>>>>> Shall we change ObjectSynchronizer::exit to accept Handle as >>>>>> argument as well? >>>>> I think you should. >>>>>>>> +void SharedRuntime::monitor_enter_helper(oopDesc* _obj, >>>>>>>> BasicLock* lock, JavaThread* thread) { >>>>>>>> Because it creates a Handle anyway.? We try to eagerly pass >>>>>>>> Handle so that we don't have to search through the code to make >>>>>>>> sure there aren't any unhandled oops. >>>>>>>> The _obj parameter name doesn't follow coding conventions (I was >>>>>>>> looking for it to be a field in something). >>>>>> I took the parameter name from the original methods: >>>>>> JRT_BLOCK_ENTRY(void, >>>>>> SharedRuntime::complete_monitor_locking_C(oopDesc* _obj, >>>>>> BasicLock* lock, JavaThread* thread)) >>>>>> JRT_LEAF(void, >>>>>> SharedRuntime::complete_monitor_unlocking_C(oopDesc* _obj, >>>>>> BasicLock* lock, JavaThread * THREAD)) >>>>>> Will Change that. >>>>>> >>>>>>>> + oop obj(_obj); >>>>>>>> Is there not already a implicit conversion from oopDesc* to oop? >>>>>> Also took from the original methods. Will get rid of that. >>>>> Great. thank you. >>>>> Coleen >>>>>> Thanks, >>>>>> Yudi >>>>>> >>>>>>>> Thanks, >>>>>>>> Coleen >>>>>>>> On 3/20/20 11:03 AM, Yudi Zheng wrote: >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> Please review this patch that unifies the monitor enter/exit >>>>>>>>> runtime entries used by C1, C2, and JVMCI. >>>>>>>>> >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8061553 >>>>>>>>> added >>>>>>>>> support for faster "slow" locking paths. >>>>>>>>> These paths are not currently used by JVMCI (or C1) leading to >>>>>>>>> slower performance on programs >>>>>>>>> that have a lot of short lived locked sections (such as >>>>>>>>> dacapo:h2). >>>>>>>>> >>>>>>>>> This patch creates helper methods based on C2?s monitor >>>>>>>>> enter/exit entries, and shares with the C1, >>>>>>>>> JVMCI runtimes. It will prevent these runtimes from missing >>>>>>>>> future updates. >>>>>>>>> >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8241234 >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~yzheng/8241234/webrev.00/ >>>>>>>>> >>>>>>>>> >>>>>>>>> Many thanks, >>>>>>>>> Yudi > From david.holmes at oracle.com Thu Mar 26 23:36:35 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 27 Mar 2020 09:36:35 +1000 Subject: RFR: 8241585: Remove unused _recursion_counter facility from PerfTraceTime In-Reply-To: <91a8ebbc-522d-bd67-6304-c9e097bd8366@oracle.com> References: <91a8ebbc-522d-bd67-6304-c9e097bd8366@oracle.com> Message-ID: <31c866a4-e135-adbc-cb8c-81fbd77bb59e@oracle.com> Hi Claes, Adding serviceability as they are the consumers of this IIUC. On 27/03/2020 3:40 am, Claes Redestad wrote: > Hi, > > PerfTraceTime::_recursion_counter is unused, and removing it > gets rid of some branchy (but well-predicted) code in paths that is > somewhat startup sensitive. Okay. > http://cr.openjdk.java.net/~redestad/8241585/open.00/ > > Also added some trace logging to determine the number of perf > data counter or each type along with a tune-up to exactly match > the defaults. Okay so can you change the bug synopsis and description to cover this more general cleanup and tuneup please. I'm never very clear on the uses of these PerfCounters. It seems SUN_NS is unused after this change. The references to jvmstat seem no longer correct - these are read via jstat ? > Testing: tier1+2 I think serviceability testing is mainly in tier3. Thanks, David ----- > > Thanks! > > /Claes From chris.plummer at oracle.com Thu Mar 26 23:46:12 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 26 Mar 2020 16:46:12 -0700 Subject: RFR: 8241585: Remove unused _recursion_counter facility from PerfTraceTime In-Reply-To: <31c866a4-e135-adbc-cb8c-81fbd77bb59e@oracle.com> References: <91a8ebbc-522d-bd67-6304-c9e097bd8366@oracle.com> <31c866a4-e135-adbc-cb8c-81fbd77bb59e@oracle.com> Message-ID: On 3/26/20 4:36 PM, David Holmes wrote: > Hi Claes, > > Adding serviceability as they are the consumers of this IIUC. > > On 27/03/2020 3:40 am, Claes Redestad wrote: >> Hi, >> >> PerfTraceTime::_recursion_counter is unused, and removing it >> gets rid of some branchy (but well-predicted) code in paths that is >> somewhat startup sensitive. > > Okay. > >> http://cr.openjdk.java.net/~redestad/8241585/open.00/ >> >> Also added some trace logging to determine the number of perf >> data counter or each type along with a tune-up to exactly match >> the defaults. > > Okay so can you change the bug synopsis and description to cover this > more general cleanup and tuneup please. > > I'm never very clear on the uses of these PerfCounters. It seems > SUN_NS is unused after this change. The references to jvmstat seem no > longer correct - these are read via jstat ? jstat uses jvmstat. Chris > >> Testing: tier1+2 > > I think serviceability testing is mainly in tier3. > > Thanks, > David > ----- > >> >> Thanks! >> >> /Claes From david.holmes at oracle.com Thu Mar 26 23:49:16 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 27 Mar 2020 09:49:16 +1000 Subject: RFR: 8241585: Remove unused _recursion_counter facility from PerfTraceTime In-Reply-To: References: <91a8ebbc-522d-bd67-6304-c9e097bd8366@oracle.com> <31c866a4-e135-adbc-cb8c-81fbd77bb59e@oracle.com> Message-ID: <3a4ac731-c79c-8094-3dd6-b6f94a8f6df4@oracle.com> On 27/03/2020 9:46 am, Chris Plummer wrote: > On 3/26/20 4:36 PM, David Holmes wrote: >> Hi Claes, >> >> Adding serviceability as they are the consumers of this IIUC. >> >> On 27/03/2020 3:40 am, Claes Redestad wrote: >>> Hi, >>> >>> PerfTraceTime::_recursion_counter is unused, and removing it >>> gets rid of some branchy (but well-predicted) code in paths that is >>> somewhat startup sensitive. >> >> Okay. >> >>> http://cr.openjdk.java.net/~redestad/8241585/open.00/ >>> >>> Also added some trace logging to determine the number of perf >>> data counter or each type along with a tune-up to exactly match >>> the defaults. >> >> Okay so can you change the bug synopsis and description to cover this >> more general cleanup and tuneup please. >> >> I'm never very clear on the uses of these PerfCounters. It seems >> SUN_NS is unused after this change. The references to jvmstat seem no >> longer correct - these are read via jstat ? > jstat uses jvmstat. Thanks Chris, I was grepping C++ code not realizing jvmstat is a Java API. David > Chris >> >>> Testing: tier1+2 >> >> I think serviceability testing is mainly in tier3. >> >> Thanks, >> David >> ----- >> >>> >>> Thanks! >>> >>> /Claes > From claes.redestad at oracle.com Fri Mar 27 00:11:34 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Fri, 27 Mar 2020 01:11:34 +0100 Subject: RFR: 8241585: Remove unused _recursion_counter facility from PerfTraceTime In-Reply-To: <31c866a4-e135-adbc-cb8c-81fbd77bb59e@oracle.com> References: <91a8ebbc-522d-bd67-6304-c9e097bd8366@oracle.com> <31c866a4-e135-adbc-cb8c-81fbd77bb59e@oracle.com> Message-ID: On 2020-03-27 00:36, David Holmes wrote: >> > > Okay so can you change the bug synopsis and description to cover this > more general cleanup and tuneup please. I filed an addendum RFE and will add this RFE bug id to the single changeset push: https://bugs.openjdk.java.net/browse/JDK-8241705 > > I'm never very clear on the uses of these PerfCounters. It seems SUN_NS > is unused after this change. The references to jvmstat seem no longer > correct - these are read via jstat ? The general confusion about PerfData/-Counters and what they're for is why I'm trying to untangle this. Generally I think we should pull the plug on it, but the perfdata shared file is tangled up with functionality to detect running JVMs used by jcmd etc, so it might take a few iterations to get there. /Claes From david.holmes at oracle.com Fri Mar 27 02:18:28 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 27 Mar 2020 12:18:28 +1000 Subject: RFR: 8241585: Remove unused _recursion_counter facility from PerfTraceTime In-Reply-To: References: <91a8ebbc-522d-bd67-6304-c9e097bd8366@oracle.com> <31c866a4-e135-adbc-cb8c-81fbd77bb59e@oracle.com> Message-ID: <880cc6fc-de02-0891-9f66-46bb6185b04f@oracle.com> Hi Claes, On 27/03/2020 10:11 am, Claes Redestad wrote: > > > On 2020-03-27 00:36, David Holmes wrote: >>> >> >> Okay so can you change the bug synopsis and description to cover this >> more general cleanup and tuneup please. > > I filed an addendum RFE and will add this RFE bug id to the single > changeset push: > https://bugs.openjdk.java.net/browse/JDK-8241705 That works too :) Thanks. >> >> I'm never very clear on the uses of these PerfCounters. It seems >> SUN_NS is unused after this change. The references to jvmstat seem no >> longer correct - these are read via jstat ? > > The general confusion about PerfData/-Counters and what they're for is > why I'm trying to untangle this. Generally I think we should pull the > plug on it, but the perfdata shared file is tangled up with > functionality to detect running JVMs used by jcmd etc, so it might > take a few iterations to get there. Yeah they confuse me. Which makes it hard to see what impact your changes may have. Hopefully serviceability folk are more familiar with how things hook together. Thanks, David > /Claes From thomas.stuefe at gmail.com Fri Mar 27 07:25:31 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 27 Mar 2020 08:25:31 +0100 Subject: RFR[T, urgent]: 8241723: Build error after 8241581 Message-ID: Hi, not my proudest day. Gtest for 8241581 got added accidentally after running the tests but before the push. Immediate result is an escaped build error in release: /shared/projects/openjdk/jdk-jdk/source/test/hotspot/gtest/utilities/test_bitMap_popcnt.cpp: In destructor 'ErrorDetailsPrinter::~ErrorDetailsPrinter()': /shared/projects/openjdk/jdk-jdk/source/test/hotspot/gtest/utilities/test_bitMap_popcnt.cpp:96:11: error: 'class BitMap' has no member named 'print_on' but since there may be other errors in this gtest as well, I'd like to remove the test, re-run all tests, then re-add it. The tested functionality is still unused, so we need not to roll-back the whole patch. https://bugs.openjdk.java.net/browse/JDK-8241723 Fix consists of "hg remove test/hotspot/gtest/utilities/test_bitMap_popcnt.cpp", so there is no webrev. Thanks, Thomas From thomas.stuefe at gmail.com Fri Mar 27 07:34:12 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 27 Mar 2020 08:34:12 +0100 Subject: RFR[T, urgent]: 8241723: Build error after 8241581 In-Reply-To: References: Message-ID: Crosspost to hs-dev. On Fri, Mar 27, 2020 at 8:25 AM Thomas St?fe wrote: > Hi, > > not my proudest day. > > > Gtest for 8241581 got added accidentally after running the tests but > before the push. Immediate result is an escaped build error in release: > > /shared/projects/openjdk/jdk-jdk/source/test/hotspot/gtest/utilities/test_bitMap_popcnt.cpp: > In destructor 'ErrorDetailsPrinter::~ErrorDetailsPrinter()': > /shared/projects/openjdk/jdk-jdk/source/test/hotspot/gtest/utilities/test_bitMap_popcnt.cpp:96:11: > error: 'class BitMap' has no member named 'print_on' > > but since there may be other errors in this gtest as well, I'd like to > remove the test, re-run all tests, then re-add it. > > The tested functionality is still unused, so we need not to roll-back the > whole patch. > > https://bugs.openjdk.java.net/browse/JDK-8241723 > > Fix consists of "hg > remove test/hotspot/gtest/utilities/test_bitMap_popcnt.cpp", so there is no > webrev. > > Thanks, Thomas > > From kim.barrett at oracle.com Fri Mar 27 07:41:22 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 27 Mar 2020 03:41:22 -0400 Subject: RFR[T, urgent]: 8241723: Build error after 8241581 In-Reply-To: References: Message-ID: > On Mar 27, 2020, at 3:25 AM, Thomas St?fe wrote: > > Hi, > > not my proudest day. > > > Gtest for 8241581 got added accidentally after running the tests but before > the push. Immediate result is an escaped build error in release: > > /shared/projects/openjdk/jdk-jdk/source/test/hotspot/gtest/utilities/test_bitMap_popcnt.cpp: > In destructor 'ErrorDetailsPrinter::~ErrorDetailsPrinter()': > /shared/projects/openjdk/jdk-jdk/source/test/hotspot/gtest/utilities/test_bitMap_popcnt.cpp:96:11: > error: 'class BitMap' has no member named 'print_on' > > but since there may be other errors in this gtest as well, I'd like to > remove the test, re-run all tests, then re-add it. > > The tested functionality is still unused, so we need not to roll-back the > whole patch. > > https://bugs.openjdk.java.net/browse/JDK-8241723 > > Fix consists of "hg > remove test/hotspot/gtest/utilities/test_bitMap_popcnt.cpp", so there is no > webrev. > > Thanks, Thomas Looks good, and trivial. From christoph.langer at sap.com Fri Mar 27 07:43:27 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 27 Mar 2020 07:43:27 +0000 Subject: RFR[T, urgent]: 8241723: Build error after 8241581 In-Reply-To: References: Message-ID: Hi Thomas, looks good to me. Best regards Christoph > -----Original Message----- > From: hotspot-dev On Behalf Of > Thomas St?fe > Sent: Freitag, 27. M?rz 2020 08:34 > To: Hotspot dev runtime ; > HotSpot Open Source Developers > Subject: Re: RFR[T, urgent]: 8241723: Build error after 8241581 > > Crosspost to hs-dev. > > On Fri, Mar 27, 2020 at 8:25 AM Thomas St?fe > wrote: > > > Hi, > > > > not my proudest day. > > > > > > Gtest for 8241581 got added accidentally after running the tests but > > before the push. Immediate result is an escaped build error in release: > > > > /shared/projects/openjdk/jdk- > jdk/source/test/hotspot/gtest/utilities/test_bitMap_popcnt.cpp: > > In destructor 'ErrorDetailsPrinter::~ErrorDetailsPrinter()': > > /shared/projects/openjdk/jdk- > jdk/source/test/hotspot/gtest/utilities/test_bitMap_popcnt.cpp:96:11: > > error: 'class BitMap' has no member named 'print_on' > > > > but since there may be other errors in this gtest as well, I'd like to > > remove the test, re-run all tests, then re-add it. > > > > The tested functionality is still unused, so we need not to roll-back the > > whole patch. > > > > https://bugs.openjdk.java.net/browse/JDK-8241723 > > > > Fix consists of "hg > > remove test/hotspot/gtest/utilities/test_bitMap_popcnt.cpp", so there is > no > > webrev. > > > > Thanks, Thomas > > > > From thomas.stuefe at gmail.com Fri Mar 27 07:49:23 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 27 Mar 2020 08:49:23 +0100 Subject: RFR[T, urgent]: 8241723: Build error after 8241581 In-Reply-To: References: Message-ID: Thanks guys, pushed. On Fri, Mar 27, 2020 at 8:41 AM Kim Barrett wrote: > > On Mar 27, 2020, at 3:25 AM, Thomas St?fe > wrote: > > > > Hi, > > > > not my proudest day. > > > > > > Gtest for 8241581 got added accidentally after running the tests but > before > > the push. Immediate result is an escaped build error in release: > > > > > /shared/projects/openjdk/jdk-jdk/source/test/hotspot/gtest/utilities/test_bitMap_popcnt.cpp: > > In destructor 'ErrorDetailsPrinter::~ErrorDetailsPrinter()': > > > /shared/projects/openjdk/jdk-jdk/source/test/hotspot/gtest/utilities/test_bitMap_popcnt.cpp:96:11: > > error: 'class BitMap' has no member named 'print_on' > > > > but since there may be other errors in this gtest as well, I'd like to > > remove the test, re-run all tests, then re-add it. > > > > The tested functionality is still unused, so we need not to roll-back the > > whole patch. > > > > https://bugs.openjdk.java.net/browse/JDK-8241723 > > > > Fix consists of "hg > > remove test/hotspot/gtest/utilities/test_bitMap_popcnt.cpp", so there is > no > > webrev. > > > > Thanks, Thomas > > Looks good, and trivial. > > From yudi.zheng at oracle.com Fri Mar 27 10:13:18 2020 From: yudi.zheng at oracle.com (Yudi Zheng) Date: Fri, 27 Mar 2020 11:13:18 +0100 Subject: RFR: 8241234: Unify monitor enter/exit runtime entries In-Reply-To: References: <222D2846-F6AE-4D5B-B41F-F976D90E329C@oracle.com> <91eeada8-e05f-bc73-b029-94e169216a56@oracle.com> <534b8cf7-cd8c-565b-5163-09a216d4f94e@oracle.com> <1a2bd9bd-3e9c-e6ba-aaae-2990493c3e23@oracle.com> Message-ID: Thanks David! @Yumin, as David said, TRACE_jvmci_3 is implicitly guarded. I personally don?t have problem dropping it. I think back to then it was for diagnosing some lock issue. -Yudi > On 26 Mar 2020, at 23:58, David Holmes wrote: > > On 27/03/2020 2:36 am, Yumin Qi wrote: >> HI, Yudi and Coleen >> jvmciRuntime.cpp: >> 385 JRT_BLOCK_ENTRY(void, JVMCIRuntime::monitorenter(JavaThread* thread, oopDesc* obj, BasicLock* lock)) >> 386 IF_TRACE_jvmci_3 { >> 387 char type[O_BUFLEN]; >> 388 obj->klass()->name()->as_C_string(type, O_BUFLEN); >> 389 markWord mark = obj->mark(); >> 390 TRACE_jvmci_3("%s: entered locking slow case with obj=" INTPTR_FORMAT ", type=%s, mark=" INTPTR_FORMAT ", lock=" INTPTR_FORMAT, thread->name(), p2i(obj), type, mark.value(), p2i(lock)); >> 391 tty->flush(); >> 392 } >> 393 SharedRuntime::monitor_enter_helper(obj, lock, thread); >> 394 TRACE_jvmci_3("%s: exiting locking slow with obj=" INTPTR_FORMAT, thread->name(), p2i(obj)); >> 395 JRT_END >> line 394: >> first it is not guarded in IF_TRACE_jvmci_3 > > It doesn't need to be. An explicit IF_TRACE_jcmvi is only needed to guard code that you don't want to execute unless tracing e.g. setting up the temp buffer as per the preceding code. > >> second I think it should be removed (I am not sure what it means here). > > It is just a logging statement showing the monitor enter has completed - it matches the previous log statement. > > David > ----- > >> Thanks >> Yumin >> On 3/26/20 8:22 AM, Yudi Zheng wrote: >>> Thanks for the comment! >>> >>> Here is the new version: http://cr.openjdk.java.net/~yzheng/8241234/webrev.03/ >>> >>> Differences w.r.t. webrev.00 are that: >>> 1. parameter _obj is now renamed to obj >>> 2. oop obj(_obj) is removed >>> >>> Many thanks, >>> Yudi >>> >>> >>>> On 26 Mar 2020, at 15:25, coleen.phillimore at oracle.com wrote: >>>> >>>> >>>> >>>> On 3/26/20 9:48 AM, Yudi Zheng wrote: >>>>> Hi Coleen, >>>>> >>>>> The monitor exit entries are marked with JRT_LEAF which contains >>>>>> debug_only(NoHandleMark __hm;) \ >>>>> This disallows the creation of Handle within the monitor exit entries. >>>>>> # A fatal error has been detected by the Java Runtime Environment: >>>>>> # >>>>>> # Internal Error (open/src/hotspot/share/runtime/handles.cpp:35), pid=28631, tid=28687 >>>>>> # assert(_no_handle_mark_nesting == 0) failed: allocating handle inside NoHandleMark >>>> You are right. Thanks for pointing this out. >>>>> I have uploaded a version that addresses the following comments: >>>>> 1. Have monitor_enter_helper take Handle as arguments. >>>>> 2. Avoid using _obj as parameter name as it indicates a field according to the coding conventions. >>>>> 3. oop obj(_obj); is not necessary as there is an implicit conversion from oopDesc* to oop >>>>> >>>>> See http://cr.openjdk.java.net/~yzheng/8241234/webrev.02/ >>>>> >>>> Actually, I apologize that I think I am wrong about making monitor_enter_helper take a Handle because I don't think you can create Handle from a thread that isn't _thread_in_vm. >>>> >>>> +// Handles the uncommon case in locking, i.e., contention or an inflated lock. >>>> +JRT_BLOCK_ENTRY(void, SharedRuntime::complete_monitor_locking_C(oopDesc* obj, BasicLock* lock, JavaThread* thread)) >>>> + Handle h_obj(thread, obj); >>>> + SharedRuntime::monitor_enter_helper(h_obj, lock, thread); >>>> +JRT_END >>>> So I think SharedRuntime::monitor_enter_helper() shouldn't take a Handle and that would allow all paths to have this fast path: >>>> >>>> if (!SafepointSynchronize::is_synchronizing()) { >>>> // Only try quick_enter() if we're not trying to reach a safepoint >>>> // so that the calling thread reaches the safepoint more quickly. >>>> - if (ObjectSynchronizer::quick_enter(_obj, thread, lock)) return; >>>> + if (ObjectSynchronizer::quick_enter(h_obj(), thread, lock)) return; >>>> } >>>> >>>> Then create the handle in the JRT_BLOCK_NO_ASYNC where it was. Maybe this gets you back to your original webrev. Sorry about this. >>>> >>>> Thanks, >>>> Coleen >>>> >>>>> Let me know if there is any other problem. >>>>> >>>>> Many thanks, >>>>> Yudi >>>>> >>>>>> On 25 Mar 2020, at 15:49, coleen.phillimore at oracle.com wrote: >>>>>> >>>>>> >>>>>> >>>>>> On 3/24/20 1:14 PM, Yudi Zheng wrote: >>>>>>> Hi Coleen, >>>>>>> >>>>>>> Thanks for the review! Sorry I missed your email. And thanks Vladimir for the reminder. >>>>>>> >>>>>>>>> I think you should have monitor_enter_helper and maybe monitor_exi_helper both take Handle as arguments. This looks very strange. >>>>>>> Shall we change ObjectSynchronizer::exit to accept Handle as argument as well? >>>>>> I think you should. >>>>>>>>> +void SharedRuntime::monitor_enter_helper(oopDesc* _obj, BasicLock* lock, JavaThread* thread) { >>>>>>>>> Because it creates a Handle anyway. We try to eagerly pass Handle so that we don't have to search through the code to make sure there aren't any unhandled oops. >>>>>>>>> The _obj parameter name doesn't follow coding conventions (I was looking for it to be a field in something). >>>>>>> I took the parameter name from the original methods: >>>>>>> JRT_BLOCK_ENTRY(void, SharedRuntime::complete_monitor_locking_C(oopDesc* _obj, BasicLock* lock, JavaThread* thread)) >>>>>>> JRT_LEAF(void, SharedRuntime::complete_monitor_unlocking_C(oopDesc* _obj, BasicLock* lock, JavaThread * THREAD)) >>>>>>> Will Change that. >>>>>>> >>>>>>>>> + oop obj(_obj); >>>>>>>>> Is there not already a implicit conversion from oopDesc* to oop? >>>>>>> Also took from the original methods. Will get rid of that. >>>>>> Great. thank you. >>>>>> Coleen >>>>>>> Thanks, >>>>>>> Yudi >>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Coleen >>>>>>>>> On 3/20/20 11:03 AM, Yudi Zheng wrote: >>>>>>>>>> Hello, >>>>>>>>>> >>>>>>>>>> Please review this patch that unifies the monitor enter/exit runtime entries used by C1, C2, and JVMCI. >>>>>>>>>> >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8061553 added support for faster "slow" locking paths. >>>>>>>>>> These paths are not currently used by JVMCI (or C1) leading to slower performance on programs >>>>>>>>>> that have a lot of short lived locked sections (such as dacapo:h2). >>>>>>>>>> >>>>>>>>>> This patch creates helper methods based on C2?s monitor enter/exit entries, and shares with the C1, >>>>>>>>>> JVMCI runtimes. It will prevent these runtimes from missing future updates. >>>>>>>>>> >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8241234 >>>>>>>>>> http://cr.openjdk.java.net/~yzheng/8241234/webrev.00/ >>>>>>>>>> >>>>>>>>>> Many thanks, >>>>>>>>>> Yudi From claes.redestad at oracle.com Fri Mar 27 10:26:36 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Fri, 27 Mar 2020 11:26:36 +0100 Subject: RFR: 8241585: Remove unused _recursion_counter facility from PerfTraceTime In-Reply-To: <880cc6fc-de02-0891-9f66-46bb6185b04f@oracle.com> References: <91a8ebbc-522d-bd67-6304-c9e097bd8366@oracle.com> <31c866a4-e135-adbc-cb8c-81fbd77bb59e@oracle.com> <880cc6fc-de02-0891-9f66-46bb6185b04f@oracle.com> Message-ID: <2a064a78-09cb-4f0e-6bef-4a74ca9712f9@oracle.com> On 2020-03-27 03:18, David Holmes wrote: > > Yeah they confuse me. Which makes it hard to see what impact your > changes may have. This patch removes some internal, unused code on the JVM end that is not observable via jstat / jvmstat. I'm happy if serviceability can weigh in though. The other RFE[1] I've filed to remove StatSampler[1] might be more contentious since it changes what gets periodically stored in the perfdata shared file. I've not yet decided if it's worth the trouble to move ahead with that at this point. /Claes [1] https://bugs.openjdk.java.net/browse/JDK-8241701 From doug.simon at oracle.com Fri Mar 27 11:07:44 2020 From: doug.simon at oracle.com (Doug Simon) Date: Fri, 27 Mar 2020 12:07:44 +0100 Subject: RFR: 8241234: Unify monitor enter/exit runtime entries In-Reply-To: References: <222D2846-F6AE-4D5B-B41F-F976D90E329C@oracle.com> <91eeada8-e05f-bc73-b029-94e169216a56@oracle.com> <534b8cf7-cd8c-565b-5163-09a216d4f94e@oracle.com> <1a2bd9bd-3e9c-e6ba-aaae-2990493c3e23@oracle.com> Message-ID: > On 27 Mar 2020, at 11:13, Yudi Zheng wrote: > > Thanks David! > > @Yumin, as David said, TRACE_jvmci_3 is implicitly guarded. I personally don?t have problem dropping it. > I think back to then it was for diagnosing some lock issue. I think that was indeed the case. If considered problematic, this tracing code can be removed. -Doug > >> On 26 Mar 2020, at 23:58, David Holmes wrote: >> >> On 27/03/2020 2:36 am, Yumin Qi wrote: >>> HI, Yudi and Coleen >>> jvmciRuntime.cpp: >>> 385 JRT_BLOCK_ENTRY(void, JVMCIRuntime::monitorenter(JavaThread* thread, oopDesc* obj, BasicLock* lock)) >>> 386 IF_TRACE_jvmci_3 { >>> 387 char type[O_BUFLEN]; >>> 388 obj->klass()->name()->as_C_string(type, O_BUFLEN); >>> 389 markWord mark = obj->mark(); >>> 390 TRACE_jvmci_3("%s: entered locking slow case with obj=" INTPTR_FORMAT ", type=%s, mark=" INTPTR_FORMAT ", lock=" INTPTR_FORMAT, thread->name(), p2i(obj), type, mark.value(), p2i(lock)); >>> 391 tty->flush(); >>> 392 } >>> 393 SharedRuntime::monitor_enter_helper(obj, lock, thread); >>> 394 TRACE_jvmci_3("%s: exiting locking slow with obj=" INTPTR_FORMAT, thread->name(), p2i(obj)); >>> 395 JRT_END >>> line 394: >>> first it is not guarded in IF_TRACE_jvmci_3 >> >> It doesn't need to be. An explicit IF_TRACE_jcmvi is only needed to guard code that you don't want to execute unless tracing e.g. setting up the temp buffer as per the preceding code. >> >>> second I think it should be removed (I am not sure what it means here). >> >> It is just a logging statement showing the monitor enter has completed - it matches the previous log statement. >> >> David >> ----- >> >>> Thanks >>> Yumin >>> On 3/26/20 8:22 AM, Yudi Zheng wrote: >>>> Thanks for the comment! >>>> >>>> Here is the new version: http://cr.openjdk.java.net/~yzheng/8241234/webrev.03/ >>>> >>>> Differences w.r.t. webrev.00 are that: >>>> 1. parameter _obj is now renamed to obj >>>> 2. oop obj(_obj) is removed >>>> >>>> Many thanks, >>>> Yudi >>>> >>>> >>>>> On 26 Mar 2020, at 15:25, coleen.phillimore at oracle.com wrote: >>>>> >>>>> >>>>> >>>>> On 3/26/20 9:48 AM, Yudi Zheng wrote: >>>>>> Hi Coleen, >>>>>> >>>>>> The monitor exit entries are marked with JRT_LEAF which contains >>>>>>> debug_only(NoHandleMark __hm;) \ >>>>>> This disallows the creation of Handle within the monitor exit entries. >>>>>>> # A fatal error has been detected by the Java Runtime Environment: >>>>>>> # >>>>>>> # Internal Error (open/src/hotspot/share/runtime/handles.cpp:35), pid=28631, tid=28687 >>>>>>> # assert(_no_handle_mark_nesting == 0) failed: allocating handle inside NoHandleMark >>>>> You are right. Thanks for pointing this out. >>>>>> I have uploaded a version that addresses the following comments: >>>>>> 1. Have monitor_enter_helper take Handle as arguments. >>>>>> 2. Avoid using _obj as parameter name as it indicates a field according to the coding conventions. >>>>>> 3. oop obj(_obj); is not necessary as there is an implicit conversion from oopDesc* to oop >>>>>> >>>>>> See http://cr.openjdk.java.net/~yzheng/8241234/webrev.02/ >>>>>> >>>>> Actually, I apologize that I think I am wrong about making monitor_enter_helper take a Handle because I don't think you can create Handle from a thread that isn't _thread_in_vm. >>>>> >>>>> +// Handles the uncommon case in locking, i.e., contention or an inflated lock. >>>>> +JRT_BLOCK_ENTRY(void, SharedRuntime::complete_monitor_locking_C(oopDesc* obj, BasicLock* lock, JavaThread* thread)) >>>>> + Handle h_obj(thread, obj); >>>>> + SharedRuntime::monitor_enter_helper(h_obj, lock, thread); >>>>> +JRT_END >>>>> So I think SharedRuntime::monitor_enter_helper() shouldn't take a Handle and that would allow all paths to have this fast path: >>>>> >>>>> if (!SafepointSynchronize::is_synchronizing()) { >>>>> // Only try quick_enter() if we're not trying to reach a safepoint >>>>> // so that the calling thread reaches the safepoint more quickly. >>>>> - if (ObjectSynchronizer::quick_enter(_obj, thread, lock)) return; >>>>> + if (ObjectSynchronizer::quick_enter(h_obj(), thread, lock)) return; >>>>> } >>>>> >>>>> Then create the handle in the JRT_BLOCK_NO_ASYNC where it was. Maybe this gets you back to your original webrev. Sorry about this. >>>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>>> Let me know if there is any other problem. >>>>>> >>>>>> Many thanks, >>>>>> Yudi >>>>>> >>>>>>> On 25 Mar 2020, at 15:49, coleen.phillimore at oracle.com wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 3/24/20 1:14 PM, Yudi Zheng wrote: >>>>>>>> Hi Coleen, >>>>>>>> >>>>>>>> Thanks for the review! Sorry I missed your email. And thanks Vladimir for the reminder. >>>>>>>> >>>>>>>>>> I think you should have monitor_enter_helper and maybe monitor_exi_helper both take Handle as arguments. This looks very strange. >>>>>>>> Shall we change ObjectSynchronizer::exit to accept Handle as argument as well? >>>>>>> I think you should. >>>>>>>>>> +void SharedRuntime::monitor_enter_helper(oopDesc* _obj, BasicLock* lock, JavaThread* thread) { >>>>>>>>>> Because it creates a Handle anyway. We try to eagerly pass Handle so that we don't have to search through the code to make sure there aren't any unhandled oops. >>>>>>>>>> The _obj parameter name doesn't follow coding conventions (I was looking for it to be a field in something). >>>>>>>> I took the parameter name from the original methods: >>>>>>>> JRT_BLOCK_ENTRY(void, SharedRuntime::complete_monitor_locking_C(oopDesc* _obj, BasicLock* lock, JavaThread* thread)) >>>>>>>> JRT_LEAF(void, SharedRuntime::complete_monitor_unlocking_C(oopDesc* _obj, BasicLock* lock, JavaThread * THREAD)) >>>>>>>> Will Change that. >>>>>>>> >>>>>>>>>> + oop obj(_obj); >>>>>>>>>> Is there not already a implicit conversion from oopDesc* to oop? >>>>>>>> Also took from the original methods. Will get rid of that. >>>>>>> Great. thank you. >>>>>>> Coleen >>>>>>>> Thanks, >>>>>>>> Yudi >>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Coleen >>>>>>>>>> On 3/20/20 11:03 AM, Yudi Zheng wrote: >>>>>>>>>>> Hello, >>>>>>>>>>> >>>>>>>>>>> Please review this patch that unifies the monitor enter/exit runtime entries used by C1, C2, and JVMCI. >>>>>>>>>>> >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8061553 added support for faster "slow" locking paths. >>>>>>>>>>> These paths are not currently used by JVMCI (or C1) leading to slower performance on programs >>>>>>>>>>> that have a lot of short lived locked sections (such as dacapo:h2). >>>>>>>>>>> >>>>>>>>>>> This patch creates helper methods based on C2?s monitor enter/exit entries, and shares with the C1, >>>>>>>>>>> JVMCI runtimes. It will prevent these runtimes from missing future updates. >>>>>>>>>>> >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8241234 >>>>>>>>>>> http://cr.openjdk.java.net/~yzheng/8241234/webrev.00/ >>>>>>>>>>> >>>>>>>>>>> Many thanks, >>>>>>>>>>> Yudi > From claes.redestad at oracle.com Fri Mar 27 13:55:45 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Fri, 27 Mar 2020 14:55:45 +0100 Subject: RFR: 8241294: Examine input checking in ClassLoader::package_from_class_name Message-ID: Hi, in ClassLoader::package_from_class_name, testing if the class name input is NULL is redundant, and testing for and skipping ['s is wrong, since fully qualified array class names doesn't start with [. We can also get rid of the bad_class_name parameter, and move the complexity of that checking to the only call site that distinguishes between an empty package NULL and a bad class name NULL Webrev: http://cr.openjdk.java.net/~redestad/8241294/open.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8241294 Testing: tier1-4 (ongoing) Thanks! /Claes From daniel.daugherty at oracle.com Fri Mar 27 16:21:36 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 27 Mar 2020 12:21:36 -0400 Subject: RFR: 8241234: Unify monitor enter/exit runtime entries In-Reply-To: References: <222D2846-F6AE-4D5B-B41F-F976D90E329C@oracle.com> <91eeada8-e05f-bc73-b029-94e169216a56@oracle.com> <534b8cf7-cd8c-565b-5163-09a216d4f94e@oracle.com> Message-ID: <904faf68-4fff-f1b8-2fb8-48d65f282fa2@oracle.com> Hi Yudi, On 3/26/20 11:22 AM, Yudi Zheng wrote: > Thanks for the comment! > > Here is the new version: http://cr.openjdk.java.net/~yzheng/8241234/webrev.03/ src/hotspot/share/runtime/sharedRuntime.hpp ??? Please don't forget to update the copyright year before you push. src/hotspot/share/runtime/sharedRuntime.cpp ??? L2104: ? ObjectSynchronizer::exit(obj, lock, THREAD); ??????? The use of 'THREAD' here and 'TRAPS' in the function itself ??????? standout more now, but that's something for me to cleanup. src/hotspot/share/c1/c1_Runtime1.cpp ??? old L718: ? assert(thread == JavaThread::current(), "threads must correspond"); ??????? Removed in favor of the assert in SharedRuntime::monitor_enter_helper(). ??????? Okay that makes sense. ??? old L721: ? EXCEPTION_MARK; ??????? Removed in favor of the same in SharedRuntime::monitor_enter_helper(). ??????? Okay that makes sense. src/hotspot/share/jvmci/jvmciRuntime.cpp ??? old L403: ? assert(thread == JavaThread::current(), "threads must correspond"); ??? old L406: ? EXCEPTION_MARK; ??????? Same as for c1_Runtime1.cpp ??? L390: ??? TRACE_jvmci_3("%s: entered locking slow case with obj="... ??? L394: ? TRACE_jvmci_3("%s: exiting locking slow with obj=" ??? L417: ??? TRACE_jvmci_3("%s: exited locking slow case with obj=" ??????? But this is no longer the "slow" case so I'm a bit confused. ??????? Update: I see there's a comment about the tracing being removed. ??????? I have no opinion on that since it is JVM/CI code, but the word ??????? "slow" needs to be adjusted if you keep it. Thumbs up! Dan P.S. For those reviewers tracking Async Monitor Deflation (8153224)... Yes, I'll have to make small updates to adjust. > > Differences w.r.t. webrev.00 are that: > 1. parameter _obj is now renamed to obj > 2. oop obj(_obj) is removed > > Many thanks, > Yudi > > >> On 26 Mar 2020, at 15:25, coleen.phillimore at oracle.com wrote: >> >> >> >> On 3/26/20 9:48 AM, Yudi Zheng wrote: >>> Hi Coleen, >>> >>> The monitor exit entries are marked with JRT_LEAF which contains >>>> debug_only(NoHandleMark __hm;) \ >>> This disallows the creation of Handle within the monitor exit entries. >>>> # A fatal error has been detected by the Java Runtime Environment: >>>> # >>>> # Internal Error (open/src/hotspot/share/runtime/handles.cpp:35), pid=28631, tid=28687 >>>> # assert(_no_handle_mark_nesting == 0) failed: allocating handle inside NoHandleMark >> You are right. Thanks for pointing this out. >>> I have uploaded a version that addresses the following comments: >>> 1. Have monitor_enter_helper take Handle as arguments. >>> 2. Avoid using _obj as parameter name as it indicates a field according to the coding conventions. >>> 3. oop obj(_obj); is not necessary as there is an implicit conversion from oopDesc* to oop >>> >>> See http://cr.openjdk.java.net/~yzheng/8241234/webrev.02/ >>> >> Actually, I apologize that I think I am wrong about making monitor_enter_helper take a Handle because I don't think you can create Handle from a thread that isn't _thread_in_vm. >> >> +// Handles the uncommon case in locking, i.e., contention or an inflated lock. >> +JRT_BLOCK_ENTRY(void, SharedRuntime::complete_monitor_locking_C(oopDesc* obj, BasicLock* lock, JavaThread* thread)) >> + Handle h_obj(thread, obj); >> + SharedRuntime::monitor_enter_helper(h_obj, lock, thread); >> +JRT_END >> >> So I think SharedRuntime::monitor_enter_helper() shouldn't take a Handle and that would allow all paths to have this fast path: >> >> if (!SafepointSynchronize::is_synchronizing()) { >> // Only try quick_enter() if we're not trying to reach a safepoint >> // so that the calling thread reaches the safepoint more quickly. >> - if (ObjectSynchronizer::quick_enter(_obj, thread, lock)) return; >> + if (ObjectSynchronizer::quick_enter(h_obj(), thread, lock)) return; >> } >> >> Then create the handle in the JRT_BLOCK_NO_ASYNC where it was. Maybe this gets you back to your original webrev. Sorry about this. >> >> Thanks, >> Coleen >> >>> Let me know if there is any other problem. >>> >>> Many thanks, >>> Yudi >>> >>>> On 25 Mar 2020, at 15:49, coleen.phillimore at oracle.com wrote: >>>> >>>> >>>> >>>> On 3/24/20 1:14 PM, Yudi Zheng wrote: >>>>> Hi Coleen, >>>>> >>>>> Thanks for the review! Sorry I missed your email. And thanks Vladimir for the reminder. >>>>> >>>>>>> I think you should have monitor_enter_helper and maybe monitor_exi_helper both take Handle as arguments. This looks very strange. >>>>> Shall we change ObjectSynchronizer::exit to accept Handle as argument as well? >>>> I think you should. >>>>>>> +void SharedRuntime::monitor_enter_helper(oopDesc* _obj, BasicLock* lock, JavaThread* thread) { >>>>>>> Because it creates a Handle anyway. We try to eagerly pass Handle so that we don't have to search through the code to make sure there aren't any unhandled oops. >>>>>>> The _obj parameter name doesn't follow coding conventions (I was looking for it to be a field in something). >>>>> I took the parameter name from the original methods: >>>>> JRT_BLOCK_ENTRY(void, SharedRuntime::complete_monitor_locking_C(oopDesc* _obj, BasicLock* lock, JavaThread* thread)) >>>>> JRT_LEAF(void, SharedRuntime::complete_monitor_unlocking_C(oopDesc* _obj, BasicLock* lock, JavaThread * THREAD)) >>>>> Will Change that. >>>>> >>>>>>> + oop obj(_obj); >>>>>>> Is there not already a implicit conversion from oopDesc* to oop? >>>>> Also took from the original methods. Will get rid of that. >>>> Great. thank you. >>>> Coleen >>>>> Thanks, >>>>> Yudi >>>>> >>>>>>> Thanks, >>>>>>> Coleen >>>>>>> On 3/20/20 11:03 AM, Yudi Zheng wrote: >>>>>>>> Hello, >>>>>>>> >>>>>>>> Please review this patch that unifies the monitor enter/exit runtime entries used by C1, C2, and JVMCI. >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8061553 added support for faster "slow" locking paths. >>>>>>>> These paths are not currently used by JVMCI (or C1) leading to slower performance on programs >>>>>>>> that have a lot of short lived locked sections (such as dacapo:h2). >>>>>>>> >>>>>>>> This patch creates helper methods based on C2?s monitor enter/exit entries, and shares with the C1, >>>>>>>> JVMCI runtimes. It will prevent these runtimes from missing future updates. >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8241234 >>>>>>>> http://cr.openjdk.java.net/~yzheng/8241234/webrev.00/ >>>>>>>> >>>>>>>> Many thanks, >>>>>>>> Yudi From gerard.ziemski at oracle.com Fri Mar 27 16:23:41 2020 From: gerard.ziemski at oracle.com (gerard ziemski) Date: Fri, 27 Mar 2020 11:23:41 -0500 Subject: RFR (XS): 8236177: assert(status == 0) failed: error ETIMEDOUT(60), cond_wait In-Reply-To: References: <64362c99-c360-1935-b9ac-e7d32e00528d@oracle.com> Message-ID: Thnak you David for your feedback! On 3/25/20 6:43 PM, David Holmes wrote: > Hi Gerard, > > On 26/03/2020 4:03 am, gerard ziemski wrote: >> hi all, >> >> Please review this "workaround" for now, which can not be called an >> actual fix just yet, designed to figure out why on Mac OS X, we get >> (very rarely) ETIMEDOUT when calling pthread_cond_wait() API. On >> other hand, it might actually fix it. > > The ETIMEDOUT should be treated as a "spurious wakeup" and we will > naturally retry the wait if the condition is not yet met. All we have > to do to our code is adjust the assert so that ETIMEDOUT doesn't cause > it to fail. My initial approach, as I noted in the bug, was to do exactly that, but when you commented in the bug, that we used to run into these kinds of issues before, and used to have workarounds in place for that, it made me worry that we will do again the workaround, and at some point in the future we remove it again, and we will back to square one. I was hoping that now we do something to get a better sense of what the underlying issue is, which is why I proposed this change instead. Is that OK? If you really want us to do what you suggest, then would you mind sharing what you remember about the previous workaround, and why it was removed? If anyone else remembers anything about it, hen please share here as well. cheers From gerard.ziemski at oracle.com Fri Mar 27 16:26:35 2020 From: gerard.ziemski at oracle.com (gerard ziemski) Date: Fri, 27 Mar 2020 11:26:35 -0500 Subject: RFR (XS): 8236177: assert(status == 0) failed: error ETIMEDOUT(60), cond_wait In-Reply-To: <4b505a62-b125-b8a1-a2d4-c3fb15ca4df7@oracle.com> References: <64362c99-c360-1935-b9ac-e7d32e00528d@oracle.com> <4b505a62-b125-b8a1-a2d4-c3fb15ca4df7@oracle.com> Message-ID: hi David, On 3/25/20 6:54 PM, David Holmes wrote: > On 26/03/2020 9:43 am, David Holmes wrote: >> Hi Gerard, >> >> On 26/03/2020 4:03 am, gerard ziemski wrote: >>> hi all, >>> >>> Please review this "workaround" for now, which can not be called an >>> actual fix just yet, designed to figure out why on Mac OS X, we get >>> (very rarely) ETIMEDOUT when calling pthread_cond_wait() API. On >>> other hand, it might actually fix it. >> >> The ETIMEDOUT should be treated as a "spurious wakeup" and we will >> naturally retry the wait if the condition is not yet met. All we have >> to do to our code is adjust the assert so that ETIMEDOUT doesn't >> cause it to fail. > > And I'm happy to explicitly show this is a macOS issue (though we > don't have an MACOS_ONLY macro) > > assert_status(status == 0 BSD_ONLY(|| status == ETIMEDOUT), status, > "cond_wait"); Just to make sure, you are saying that you are OK with using "#ifdef __APPLE__" here, right? You are not suggesting we introduce "MACOS_ONLY" macro yet, correct? cheers From daniel.daugherty at oracle.com Fri Mar 27 16:41:11 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 27 Mar 2020 12:41:11 -0400 Subject: RFR 8231264: Implementation of JEP 374: Disable biased-locking and deprecate all flags related to biased-locking In-Reply-To: <589a66e6-82c0-435f-cf01-7a82fe7a3b69@oracle.com> References: <589a66e6-82c0-435f-cf01-7a82fe7a3b69@oracle.com> Message-ID: <89f287d7-26e7-619f-a0cf-223da300cc55@oracle.com> On 3/26/20 3:23 PM, Patricio Chilano wrote: > Hi all, > > Please review the following implementation for JEP 374: Disable and > Deprecate Biased Locking. > > As already discussed we will change the default value for > UseBiasedLocking to false. Deprecating the flags will also give more > visibility to the change and should increase our changes to > potentially get more feedback.? After a transition/evaluation period > and based on collected evidence that will complement the results of > our own benchmarks we can decide if we want to keep this feature or not. > > I modified test VMDeprecatedOptions.java to include the deprecated > flags, except for the two diagnostic ones > (PrintBiasedLockingStatistics, PrintPreciseBiasedLockingStatistics). > > Tested in mach5, tiers1-7 on all platforms (Linux-x64, macOS-x64, > Windows-x64, Solaris-sparcv9). > > Let me know if there are more tests I should run. > > Bug: > ? https://bugs.openjdk.java.net/browse/JDK-8231264 > > Webrev: > ? http://cr.openjdk.java.net/~pchilanomate/8231264/v1/webrev/ src/hotspot/share/opto/c2_globals.hpp ??? No comments. src/hotspot/share/runtime/arguments.cpp ??? No comments. src/hotspot/share/runtime/globals.hpp ??? No comments. test/hotspot/gtest/oops/test_markWord.cpp ??? No comments. test/hotspot/jtreg/runtime/CommandLine/VMDeprecatedOptions.java ??? Is there a C2 specific test for checking deprecation of ??? PrintPreciseBiasedLockingStatistics and UseOptoBiasInlining? ??? I don't remember there being one, but I figured I would ask... Thumbs up! Dan > > > Thanks, > > Patricio From patricio.chilano.mateo at oracle.com Fri Mar 27 17:42:45 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Fri, 27 Mar 2020 14:42:45 -0300 Subject: RFR 8231264: Implementation of JEP 374: Disable biased-locking and deprecate all flags related to biased-locking In-Reply-To: <60f2691b-210a-d337-37a7-226e7313d633@oracle.com> References: <589a66e6-82c0-435f-cf01-7a82fe7a3b69@oracle.com> <60f2691b-210a-d337-37a7-226e7313d633@oracle.com> Message-ID: Hi David, On 3/26/20 7:50 PM, David Holmes wrote: > Hi Patricio, > > This looks good to me. Ready to go once the JEP is targeted. Thanks for looking into this! Yes, I'll wait for that. Patricio > Thanks, > David > > On 27/03/2020 5:23 am, Patricio Chilano wrote: >> Hi all, >> >> Please review the following implementation for JEP 374: Disable and >> Deprecate Biased Locking. >> >> As already discussed we will change the default value for >> UseBiasedLocking to false. Deprecating the flags will also give more >> visibility to the change and should increase our changes to >> potentially get more feedback.? After a transition/evaluation period >> and based on collected evidence that will complement the results of >> our own benchmarks we can decide if we want to keep this feature or not. >> >> I modified test VMDeprecatedOptions.java to include the deprecated >> flags, except for the two diagnostic ones >> (PrintBiasedLockingStatistics, PrintPreciseBiasedLockingStatistics). >> >> Tested in mach5, tiers1-7 on all platforms (Linux-x64, macOS-x64, >> Windows-x64, Solaris-sparcv9). >> >> Let me know if there are more tests I should run. >> >> Bug: >> ?? https://bugs.openjdk.java.net/browse/JDK-8231264 >> >> Webrev: >> ?? http://cr.openjdk.java.net/~pchilanomate/8231264/v1/webrev/ >> >> >> Thanks, >> >> Patricio From patricio.chilano.mateo at oracle.com Fri Mar 27 18:02:16 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Fri, 27 Mar 2020 15:02:16 -0300 Subject: RFR 8231264: Implementation of JEP 374: Disable biased-locking and deprecate all flags related to biased-locking In-Reply-To: <89f287d7-26e7-619f-a0cf-223da300cc55@oracle.com> References: <589a66e6-82c0-435f-cf01-7a82fe7a3b69@oracle.com> <89f287d7-26e7-619f-a0cf-223da300cc55@oracle.com> Message-ID: Hi Dan, On 3/27/20 1:41 PM, Daniel D. Daugherty wrote: > On 3/26/20 3:23 PM, Patricio Chilano wrote: >> Hi all, >> >> Please review the following implementation for JEP 374: Disable and >> Deprecate Biased Locking. >> >> As already discussed we will change the default value for >> UseBiasedLocking to false. Deprecating the flags will also give more >> visibility to the change and should increase our changes to >> potentially get more feedback.? After a transition/evaluation period >> and based on collected evidence that will complement the results of >> our own benchmarks we can decide if we want to keep this feature or not. >> >> I modified test VMDeprecatedOptions.java to include the deprecated >> flags, except for the two diagnostic ones >> (PrintBiasedLockingStatistics, PrintPreciseBiasedLockingStatistics). >> >> Tested in mach5, tiers1-7 on all platforms (Linux-x64, macOS-x64, >> Windows-x64, Solaris-sparcv9). >> >> Let me know if there are more tests I should run. >> >> Bug: >> ? https://bugs.openjdk.java.net/browse/JDK-8231264 >> >> Webrev: >> ? http://cr.openjdk.java.net/~pchilanomate/8231264/v1/webrev/ > > src/hotspot/share/opto/c2_globals.hpp > ??? No comments. > > src/hotspot/share/runtime/arguments.cpp > ??? No comments. > > src/hotspot/share/runtime/globals.hpp > ??? No comments. > > test/hotspot/gtest/oops/test_markWord.cpp > ??? No comments. > > test/hotspot/jtreg/runtime/CommandLine/VMDeprecatedOptions.java > ??? Is there a C2 specific test for checking deprecation of > ??? PrintPreciseBiasedLockingStatistics and UseOptoBiasInlining? > ??? I don't remember there being one, but I figured I would ask... I didn't find any other test to check that besides VMDeprecatedOptions.java. > Thumbs up! Thanks for reviewing this Dan! Patricio > Dan > >> >> >> Thanks, >> >> Patricio > From lois.foltan at oracle.com Fri Mar 27 20:47:58 2020 From: lois.foltan at oracle.com (Lois Foltan) Date: Fri, 27 Mar 2020 16:47:58 -0400 Subject: RFR: 8241294: Examine input checking in ClassLoader::package_from_class_name In-Reply-To: References: Message-ID: <40fd8bc2-8e26-9d50-c743-561f5e451533@oracle.com> On 3/27/2020 9:55 AM, Claes Redestad wrote: > Hi, > > in ClassLoader::package_from_class_name, testing if the class name input > is NULL is redundant, and testing for and skipping ['s is wrong, since > fully qualified array class names doesn't start with [. > > We can also get rid of the bad_class_name parameter, and move > the complexity of that checking to the only call site that distinguishes > between an empty package NULL and a bad class name NULL > > Webrev: http://cr.openjdk.java.net/~redestad/8241294/open.00/ > Bug:??? https://bugs.openjdk.java.net/browse/JDK-8241294 > > Testing: tier1-4 (ongoing) > > Thanks! > > /Claes Hi Claes, Looks good.? I'm glad to be rid of that "bad_class_name" parameter. Thank you for doing this.? Minor comment: - classfile/classLoader.cpp line #180 & #188 - plase consider adding an improved comment over "Can't be" in the asserts.? Maybe line #180 "unexpected null class name" line #188 "class name's first character contains an array opening bracket" Also, I was able to write a .jasm file with a class whose name is "[FOO" for example.? With your patch, I did try executing the resulting .class file and am receiving a CNFE, "Could not find or load main class [FOO" exception.? If you would like the .jasm file I can mail it to you to experiment more with. Thanks, Lois From david.holmes at oracle.com Fri Mar 27 22:53:49 2020 From: david.holmes at oracle.com (David Holmes) Date: Sat, 28 Mar 2020 08:53:49 +1000 Subject: RFR (XS): 8236177: assert(status == 0) failed: error ETIMEDOUT(60), cond_wait In-Reply-To: References: <64362c99-c360-1935-b9ac-e7d32e00528d@oracle.com> Message-ID: <0f250221-dd1e-979b-45df-7f64c6741af3@oracle.com> Hi Gerard, On 28/03/2020 2:23 am, gerard ziemski wrote: > Thnak you David for your feedback! > > On 3/25/20 6:43 PM, David Holmes wrote: >> Hi Gerard, >> >> On 26/03/2020 4:03 am, gerard ziemski wrote: >>> hi all, >>> >>> Please review this "workaround" for now, which can not be called an >>> actual fix just yet, designed to figure out why on Mac OS X, we get >>> (very rarely) ETIMEDOUT when calling pthread_cond_wait() API. On >>> other hand, it might actually fix it. >> >> The ETIMEDOUT should be treated as a "spurious wakeup" and we will >> naturally retry the wait if the condition is not yet met. All we have >> to do to our code is adjust the assert so that ETIMEDOUT doesn't cause >> it to fail. > > My initial approach, as I noted in the bug, was to do exactly that, but > when you commented in the bug, that we used to run into these kinds of > issues before, and used to have workarounds in place for that, it made > me worry that we will do again the workaround, and at some point in the > future we remove it again, and we will back to square one. I'm sorry my comment misled you. > I was hoping that now we do something to get a better sense of what the > underlying issue is, which is why I proposed this change instead. > > Is that OK? There is no need for that kind of workaround. The effect of the bug is at worst a spurious wakeup (and we can't tell if it is spurious or not without more investigation - not that it matters.) We just need to fix the assert. > If you really want us to do what you suggest, then would you mind > sharing what you remember about the previous workaround, and why it was > removed? If anyone else remembers anything about it, hen please share > here as well. There have been a number of issues as NPTL evolved. Here's a key one: // Beware -- Some versions of NPTL embody a flaw where pthread_cond_timedwait() can // hang indefinitely. For instance NPTL 0.60 on 2.4.21-4ELsmp is vulnerable. // For specifics regarding the bug see GLIBC BUGID 261237 : // http://www.mail-archive.com/debian-glibc at lists.debian.org/msg10837.html. // Briefly, pthread_cond_timedwait() calls with an expiry time that's not in the future // will either hang or corrupt the condvar, resulting in subsequent hangs if the condvar // is used. This one was addressed via the WorkAroundNTPLTimedWaitHang flag. For which we did this hack: status = os::Linux::safe_cond_timedwait(_cond, _mutex, &abst); if (status != 0 && WorkAroundNPTLTimedWaitHang) { pthread_cond_destroy (_cond); pthread_cond_init (_cond, NULL) ; } Note however I was mis-remembering the structure here. The workaround was in park() etc not inside os::Linux::safe_cond_timedwait (which was need to hide the selection of NPTL or LinuxThreads logic). These workarounds eventually get removed when we no longer have any concerns about running on a platform that would still have the bug. Thanks, David > > cheers > From david.holmes at oracle.com Fri Mar 27 22:56:08 2020 From: david.holmes at oracle.com (David Holmes) Date: Sat, 28 Mar 2020 08:56:08 +1000 Subject: RFR (XS): 8236177: assert(status == 0) failed: error ETIMEDOUT(60), cond_wait In-Reply-To: References: <64362c99-c360-1935-b9ac-e7d32e00528d@oracle.com> <4b505a62-b125-b8a1-a2d4-c3fb15ca4df7@oracle.com> Message-ID: <808e1c5b-94ea-fb92-4fd2-80d4627607be@oracle.com> On 28/03/2020 2:26 am, gerard ziemski wrote: > hi David, > > On 3/25/20 6:54 PM, David Holmes wrote: >> On 26/03/2020 9:43 am, David Holmes wrote: >>> Hi Gerard, >>> >>> On 26/03/2020 4:03 am, gerard ziemski wrote: >>>> hi all, >>>> >>>> Please review this "workaround" for now, which can not be called an >>>> actual fix just yet, designed to figure out why on Mac OS X, we get >>>> (very rarely) ETIMEDOUT when calling pthread_cond_wait() API. On >>>> other hand, it might actually fix it. >>> >>> The ETIMEDOUT should be treated as a "spurious wakeup" and we will >>> naturally retry the wait if the condition is not yet met. All we have >>> to do to our code is adjust the assert so that ETIMEDOUT doesn't >>> cause it to fail. >> >> And I'm happy to explicitly show this is a macOS issue (though we >> don't have an MACOS_ONLY macro) >> >> assert_status(status == 0 BSD_ONLY(|| status == ETIMEDOUT), status, >> "cond_wait"); > > Just to make sure, you are saying that you are OK with using "#ifdef > __APPLE__" here, right? You are not suggesting we introduce "MACOS_ONLY" > macro yet, correct? I wouldn't object to introducing MACOS_ONLY :) it would be slightly cleaner than assert_status(status == 0 #ifdef __APPLE__ || status == ETIMEDOUT #endif , status, "cond_wait"); Cheers, David > > cheers From claes.redestad at oracle.com Sat Mar 28 01:07:07 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Sat, 28 Mar 2020 02:07:07 +0100 Subject: RFR: 8241294: Examine input checking in ClassLoader::package_from_class_name In-Reply-To: <40fd8bc2-8e26-9d50-c743-561f5e451533@oracle.com> References: <40fd8bc2-8e26-9d50-c743-561f5e451533@oracle.com> Message-ID: <6399a38f-5777-72d5-fbd4-2a63f45b534f@oracle.com> On 2020-03-27 21:47, Lois Foltan wrote: > On 3/27/2020 9:55 AM, Claes Redestad wrote: >> Hi, >> >> in ClassLoader::package_from_class_name, testing if the class name input >> is NULL is redundant, and testing for and skipping ['s is wrong, since >> fully qualified array class names doesn't start with [. >> >> We can also get rid of the bad_class_name parameter, and move >> the complexity of that checking to the only call site that distinguishes >> between an empty package NULL and a bad class name NULL >> >> Webrev: http://cr.openjdk.java.net/~redestad/8241294/open.00/ >> Bug:??? https://bugs.openjdk.java.net/browse/JDK-8241294 >> >> Testing: tier1-4 (ongoing) >> >> Thanks! >> >> /Claes > > Hi Claes, > Looks good.? I'm glad to be rid of that "bad_class_name" parameter. > Thank you for doing this. Thanks! > Minor comment: > > - classfile/classLoader.cpp > line #180 & #188 - plase consider adding an improved comment over "Can't > be" in the asserts.? Maybe > line #180 "unexpected null class name" > line #188 "class name's first character contains an array opening bracket" Fixed. > > Also, I was able to write a .jasm file with a class whose name is "[FOO" > for example.? With your patch, I did try executing the resulting .class > file and am receiving a CNFE, "Could not find or load main class [FOO" > exception.? If you would like the .jasm file I can mail it to you to > experiment more with. Added a sanity test based on runtime/ClassFile/FormatCheckingTest (thanks for the pointer!): http://cr.openjdk.java.net/~redestad/8241294/open.01/ The test outputs the same ClassFormatError as the old test case (LBadHelloWorld; vs [BadHelloWorld2). Running BadHelloWorld2 fails with a CNFE if I run without verification, both with and without my patch, so I think we have verified that such classes are rejected before ever hitting the package_for_class_name. Thanks, /Claes > > Thanks, > Lois From ioi.lam at oracle.com Sat Mar 28 05:16:48 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 27 Mar 2020 22:16:48 -0700 Subject: RFR: 8241294: Examine input checking in ClassLoader::package_from_class_name In-Reply-To: <6399a38f-5777-72d5-fbd4-2a63f45b534f@oracle.com> References: <40fd8bc2-8e26-9d50-c743-561f5e451533@oracle.com> <6399a38f-5777-72d5-fbd4-2a63f45b534f@oracle.com> Message-ID: <37286faa-5243-6508-4efd-25c14bdfaf63@oracle.com> Hi Claes, I think you should change the RFE title to something like "clean up ClassLoader::package_from_class_name API". =============== Since you are removing checks over strings that can potentially have arbitrary contents, we need to be extra careful. You mentioned that "testing for and skipping ['s is wrong, since fully qualified array class names doesn't start with [." I think you are assuming that this function is always called with a valid class name. I checked casually and that seems to be the case, but there are a lot of call sites and I have not validated them exhaustively. InstanceKlass::is_override ?? -> InstanceKlass::is_same_class_package() ?????? -> ClassLoader::package_from_class_name() java.lang.Class.forName() ?? -> ... ????? -> SystemDictionary::load_instance_class() ???????? -> ClassLoader::package_from_class_name() ..... Class.forName() can be called by user code with arbitrary strings, but it's validated by a call to verifyFixClassname/verifyClassname: http://hg.openjdk.java.net/jdk/jdk/file/5ac19bd3a1e2/src/java.base/share/native/libjava/Class.c#l129 All other cases seem to be names that comes from class names, or references from constant pool entries, which should have been checked during class file parsing to contain valid names. Since you're going down this path, I think we should add a stronger assertion than just (name[0] != '['). Instead, you should assert with ??? (name[0] != '[' && ClassFileParser::verify_legal_class_name(...)) (you would need to refactor the verify_legal_class_name function). This will make sure all future users of this function won't try to abuse it. And then, run the JCK to make sure we are indeed catching all cases. =============== Also, I think you can get rid of this, as it will never happen: ? // in which case start > end ? if (base >= end) { ??? return NULL; ? } ================ Then, you can tighten up the comments in the hpp file about the input and output (something like): input -- must be a valid, binary name of an InstanceKlass (array names are not allowed) output -- package of the class, or NULL iff the class is in the default package (i.e., the name contains no "/"). Thanks - Ioi On 3/27/20 6:07 PM, Claes Redestad wrote: > > > On 2020-03-27 21:47, Lois Foltan wrote: >> On 3/27/2020 9:55 AM, Claes Redestad wrote: >>> Hi, >>> >>> in ClassLoader::package_from_class_name, testing if the class name >>> input >>> is NULL is redundant, and testing for and skipping ['s is wrong, since >>> fully qualified array class names doesn't start with [. >>> >>> We can also get rid of the bad_class_name parameter, and move >>> the complexity of that checking to the only call site that >>> distinguishes >>> between an empty package NULL and a bad class name NULL >>> >>> Webrev: http://cr.openjdk.java.net/~redestad/8241294/open.00/ >>> Bug:??? https://bugs.openjdk.java.net/browse/JDK-8241294 >>> >>> Testing: tier1-4 (ongoing) >>> >>> Thanks! >>> >>> /Claes >> >> Hi Claes, >> Looks good.? I'm glad to be rid of that "bad_class_name" parameter. >> Thank you for doing this. > > Thanks! > >> Minor comment: >> >> - classfile/classLoader.cpp >> line #180 & #188 - plase consider adding an improved comment over >> "Can't be" in the asserts.? Maybe >> line #180 "unexpected null class name" >> line #188 "class name's first character contains an array opening >> bracket" > > Fixed. > >> >> Also, I was able to write a .jasm file with a class whose name is >> "[FOO" for example.? With your patch, I did try executing the >> resulting .class file and am receiving a CNFE, "Could not find or >> load main class [FOO" exception.? If you would like the .jasm file I >> can mail it to you to experiment more with. > > Added a sanity test based on runtime/ClassFile/FormatCheckingTest > (thanks for the pointer!): > > http://cr.openjdk.java.net/~redestad/8241294/open.01/ > > The test outputs the same ClassFormatError as the old test case > (LBadHelloWorld; vs [BadHelloWorld2). Running BadHelloWorld2 fails with > a CNFE if I run without verification, both with and without my patch, > so I think we have verified that such classes are rejected before ever > hitting the package_for_class_name. > > Thanks, > > /Claes > >> >> Thanks, >> Lois From richard.reingruber at sap.com Mon Mar 30 08:10:42 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Mon, 30 Mar 2020 08:10:42 +0000 Subject: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents In-Reply-To: References: <1f8a3c7a-fa0f-b5b2-4a8a-7d3d8dbbe1b5@oracle.com> <4b56a45c-a14c-6f74-2bfd-25deaabe8201@oracle.com> <5271429a-481d-ddb9-99dc-b3f6670fcc0b@oracle.com> Message-ID: Hi, this is webrev.5 based on Robbin's feedback and Martin's review - thanks! :) The change affects jvmti, hotspot and c2. Partial reviews are very welcome too. Full: http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.5/ Delta: http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.5.inc/ Robbin, Martin, please let me know, if anything shouldn't be quite as you wanted it. Also find my comments on your feedback below. Robbin, can I count you as Reviewer for the runtime part? Thanks, Richard. -----Original Message----- From: Doerr, Martin Sent: Donnerstag, 12. M?rz 2020 17:28 To: Reingruber, Richard ; 'Robbin Ehn' ; Lindenmaier, Goetz ; David Holmes ; Vladimir Kozlov (vladimir.kozlov at oracle.com) ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents Hi Richard, I managed to find time for a (almost) complete review of webrev.4. (I'll review the tests separately.) First of all, the change seems to be in pretty good quality for its significant complexity. I couldn't find any real bugs. But I'd like to propose minor improvements. I'm convinced that it's mature because we did substantial testing. I like the new functionality for object deoptimization. It can possibly be reused for future escape analysis based optimizations. So I appreciate having it available in the code base. In addition to that, your change makes the JVMTI implementation better integrated into the VM. Now to the details: src/hotspot/share/c1/c1_IR.hpp describe_scope parameters. Ok. src/hotspot/share/ci/ciEnv.cpp src/hotspot/share/ci/ciEnv.hpp Fix for JvmtiExport::can_walk_any_space() capability. Ok. src/hotspot/share/code/compiledMethod.cpp Nice cleanup! src/hotspot/share/code/debugInfoRec.cpp src/hotspot/share/code/debugInfoRec.hpp Additional parmeters. (Remark: I think "non_global_escape_in_scope" would read better than "not_global_escape_in_scope", but your version is consistent with existing code, so no change request from my side.) Ok. src/hotspot/share/code/nmethod.cpp Nice cleanup! src/hotspot/share/code/pcDesc.hpp Additional parameters. Ok. src/hotspot/share/code/scopeDesc.cpp src/hotspot/share/code/scopeDesc.hpp Improved implementation + additional parameters. Ok. src/hotspot/share/compiler/compileBroker.cpp src/hotspot/share/compiler/compileBroker.hpp Extra thread for DeoptimizeObjectsALot. (Remark: I would have put it into a follow up change together with the test in order to make this webrev smaller, but since it is included, I'm reviewing everything at once. Not a big deal.) Ok. src/hotspot/share/jvmci/jvmciCodeInstaller.cpp Additional parameters. Ok. src/hotspot/share/opto/c2compiler.cpp Make do_escape_analysis independent of JVMCI capabilities. Nice! src/hotspot/share/opto/callnode.hpp Additional fields for MachSafePointNodes. Ok. src/hotspot/share/opto/escape.cpp Annotation for MachSafePointNodes. Your added functionality looks correct. But I'd prefer to move the bulky code out of the large function. I suggest to factor out something like has_not_global_escape and has_arg_escape. So the code could look like this: SafePointNode* sfn = sfn_worklist.at(next); sfn->set_not_global_escape_in_scope(has_not_global_escape(sfn)); if (sfn->is_CallJava()) { CallJavaNode* call = sfn->as_CallJava(); call->set_arg_escape(has_arg_escape(call)); } This would also allow us to get rid of the found_..._escape_in_args variables making the loops better readable. It's kind of ugly to use strcmp to recognize uncommon trap, but that seems to be the way to do it (there are more such places). So it's ok. src/hotspot/share/opto/machnode.hpp Additional fields for MachSafePointNodes. Ok. src/hotspot/share/opto/macro.cpp Allow elimination of non-escaping allocations. Ok. src/hotspot/share/opto/matcher.cpp src/hotspot/share/opto/output.cpp Copy attribute / pass parameters. Ok. src/hotspot/share/prims/jvmtiCodeBlobEvents.cpp Nice cleanup! src/hotspot/share/prims/jvmtiEnv.cpp src/hotspot/share/prims/jvmtiEnvBase.cpp Escape barriers + deoptimize objects for target thread. Good. src/hotspot/share/prims/jvmtiImpl.cpp src/hotspot/share/prims/jvmtiImpl.hpp The sequence is pretty complex: VM_GetOrSetLocal element initialization executes EscapeBarrier code which suspends the target thread (extra VM Operation). VM_GetOrSetLocal::doit_prologue performs object deoptimization (by VM Thread to prepare VM Operation with frame deoptimization). VM_GetOrSetLocal destructor implicitly calls EscapeBarrier destructor which resumes the target thread. But I don't have any improvement proposal. Performance is probably not a concern, here. So it's ok. VM_GetOrSetLocal::deoptimize_objects deoptimizes the top frame if it has non-globally escaping objects and other frames if they have arg escaping ones. Good. src/hotspot/share/prims/jvmtiTagMap.cpp Escape barriers + deoptimize objects for all threads. Ok. src/hotspot/share/prims/whitebox.cpp Added WB_IsFrameDeoptimized to API. Ok. src/hotspot/share/runtime/deoptimization.cpp Object deoptimization. I have more comments and proposals, here. First of all, handling recursive and waiting locks in relock_objects is tricky, but looks correct. Comments are sufficient to understand why things are done as they are implemented. BiasedLocking related parts are complex, but we may get rid of them in the future (with BiasedLocking removal). Anyway, looks correct, too. Typo in comment: "regularily" => "regularly" Deoptimization::fetch_unroll_info_helper is the only place where _jvmti_deferred_updates get deallocated (except JavaThread destructor). But I think we always go through it, so I can't see a memory leak or such kind of issues. EscapeBarrier::deoptimize_objects: ResourceMark should use calling_thread(). You can use MutexLocker and MonitorLocker with Thread* to save the Thread::current() call. I'd make set_objs_are_deoptimized static and remove it from the EscapeBarrier interface because I think it shouldn't be used outside of EscapeBarrier::deoptimize_objects. Typo in comment: "we must only deoptimize" => "we only have to deoptimize" "bool EscapeBarrier::deoptimize_objects(intptr_t* fr_id)" is trivial and barrier_active() is redundant. Implementation can get moved to hpp file. I'll get back to suspend flags, later. There are weird cases regarding _self_deoptimization_in_progress. Assume we have 3 threads A, B and C. A deopts C, B deopts C, C deopts C. C can set _self_deoptimization_in_progress while A performs the handshake for suspending C. I think this doesn't lead to errors, but it's probably not desired. I think it would be better to use only one "wait" call in sync_and_suspend_one and sync_and_suspend_all. I first thought it'd be better to move ThreadBlockInVM before wait() to reduce thread state transitions, but that seems to be problematic because ThreadBlockInVM destructor contains a safepoint check which we shouldn't do while holding EscapeBarrier_lock. So no change request. Change in thred_added: I think the sequence would be more comprehensive if we waited for deopt_all_threads in Thread::start and all other places where a new thread can run into Java code (e.g. JVMTI attach). Your version makes new threads come up with suspend flag set. That looks correct, too. Advantage is that you only have to change one place (thread_added). It'll be interesting to see how it will look like when we use async handshakes instead of suspend flags. For now, I'm ok with your version. I'd only move MutexLocker ml(EscapeBarrier_lock...) after if (!jt->is_hidden_from_external_view()). Having 4 different deoptimize_objects functions makes it a little hard to keep an overview of which one is used for what. Maybe adding suffixes would help a little bit, but I can also live with what you have. Implementation looks correct to me. src/hotspot/share/runtime/deoptimization.hpp Escape barriers and object deoptimization functions. Typo in comment: "helt" => "held" src/hotspot/share/runtime/globals.hpp Addition of develop flag DeoptimizeObjectsALotInterval. Ok. src/hotspot/share/runtime/interfaceSupport.cpp InterfaceSupport::deoptimizeAllObjects() is only used for DeoptimizeObjectsALot = 1. I think DeoptimizeObjectsALot = 2 is more important, but I think it's not bad to have DeoptimizeObjectsALot = 1 in addition. Ok. src/hotspot/share/runtime/interfaceSupport.inline.hpp Addition of deoptimizeAllObjects. Ok. src/hotspot/share/runtime/mutexLocker.cpp src/hotspot/share/runtime/mutexLocker.hpp Addition of EscapeBarrier_lock. Ok. src/hotspot/share/runtime/objectMonitor.cpp Make recursion count relock aware. Ok. src/hotspot/share/runtime/stackValue.hpp Better reinitilization in StackValue. Good. src/hotspot/share/runtime/thread.cpp src/hotspot/share/runtime/thread.hpp src/hotspot/share/runtime/thread.inline.hpp wait_for_object_deoptimization, suspend flag, deferred updates and test feature to deoptimize objects. In the long term, we want to get rid of suspend flags, so it's not so nice to introduce a new one. But I agree with G?tz that it should be acceptable as temporary solution until async handshakes are available (which takes more time). So I'm ok with your change. You can use MutexLocker with Thread*. JVMTIDeferredUpdates: I agree with Robin. It'd be nice to move the class out of thread.hpp. src/hotspot/share/runtime/vframe.cpp Added support for entry frame to new_vframe. Ok. src/hotspot/share/runtime/vframe_hp.cpp src/hotspot/share/runtime/vframe_hp.hpp I think code()->as_nmethod() in not_global_escape_in_scope() and arg_escape() should better be under #ifdef ASSERT or inside the assert statement (no need for code cache walking in product build). jvmtiDeferredLocalVariableSet::update_monitors: Please add a comment explaining that owner referenced by original info may be scalar replaced, but it is deoptimized in the vframe. src/hotspot/share/utilities/macros.hpp Addition of NOT_COMPILER2_OR_JVMCI_RETURN macros. Ok. test/hotspot/jtreg/serviceability/jvmti/Heap/IterateHeapWithEscapeAnalysisEnabled.java test/hotspot/jtreg/serviceability/jvmti/Heap/libIterateHeapWithEscapeAnalysisEnabled.c New test. Will review separately. test/jdk/TEST.ROOT Addition of vm.jvmci as required property. Ok. test/jdk/com/sun/jdi/EATests.java test/jdk/com/sun/jdi/EATestsJVMCI.java New test. Will review separately. test/lib/sun/hotspot/WhiteBox.java Added isFrameDeoptimized to API. Ok. That was it. Best regards, Martin > -----Original Message----- > From: hotspot-compiler-dev bounces at openjdk.java.net> On Behalf Of Reingruber, Richard > Sent: Dienstag, 3. M?rz 2020 21:23 > To: 'Robbin Ehn' ; Lindenmaier, Goetz > ; David Holmes ; > Vladimir Kozlov (vladimir.kozlov at oracle.com) > ; serviceability-dev at openjdk.java.net; > hotspot-compiler-dev at openjdk.java.net; hotspot-runtime- > dev at openjdk.java.net > Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better > Performance in the Presence of JVMTI Agents > > Hi Robbin, > > > > I understand that Robbin proposed to replace the usage of > > > _suspend_flag with handshakes. Apparently, async handshakes > > > are needed to do so. We have been waiting a while for removal > > > of the _suspend_flag / introduction of async handshakes [2]. > > > What is the status here? > > > I have an old prototype which I would like to continue to work on. > > So do not assume asynch handshakes will make 15. > > Even if it would, I think there are a lot more investigate work to remove > > _suspend_flag. > > Let us know, if we can be of any help to you and be it only testing. > > > >> Full: > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.4/ > > > DeoptimizeObjectsALotThread is only used in compileBroker.cpp. > > You can move both declaration and definition to that file, no need to > clobber > > thread.[c|h]pp. (and the static function deopt_objs_alot_thread_entry) > > Will do. > > > Does JvmtiDeferredUpdates really need to be in thread.hpp, can't be in it's > own > > hpp file? It doesn't seem right to add JVM TI classes into thread.hpp. > > You are right. It shouldn't be declared in thread.hpp. I will look into that. > > > Note that we also think we may have a bug in deopt: > > https://bugs.openjdk.java.net/browse/JDK-8238237 > > > I think it would be best, if possible, to push after that is resolved. > > Sure. > > > Not even nearly a full review :) > > I know :) > > Anyways, thanks a lot, > Richard. > > > -----Original Message----- > From: Robbin Ehn > Sent: Monday, March 2, 2020 11:17 AM > To: Lindenmaier, Goetz ; Reingruber, Richard > ; David Holmes ; > Vladimir Kozlov (vladimir.kozlov at oracle.com) > ; serviceability-dev at openjdk.java.net; > hotspot-compiler-dev at openjdk.java.net; hotspot-runtime- > dev at openjdk.java.net > Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better Performance > in the Presence of JVMTI Agents > > Hi, > > On 2/24/20 5:39 PM, Lindenmaier, Goetz wrote: > > Hi, > > > > I had a look at the progress of this change. Nothing > > happened since Richard posted his update using more > > handshakes [1]. > > But we (SAP) would appreciate a lot if this change could > > be successfully reviewed and pushed. > > > > I think there is basic understanding that this > > change is helpful. It fixes a number of issues with JVMTI, > > and will deliver the same performance benefits as EA > > does in current production mode for debugging scenarios. > > > > This is important for us as we run our VMs prepared > > for debugging in production mode. > > > > I understand that Robbin proposed to replace the usage of > > _suspend_flag with handshakes. Apparently, async handshakes > > are needed to do so. We have been waiting a while for removal > > of the _suspend_flag / introduction of async handshakes [2]. > > What is the status here? > > I have an old prototype which I would like to continue to work on. > So do not assume asynch handshakes will make 15. > Even if it would, I think there are a lot more investigate work to remove > _suspend_flag. > > > > > I think we should no longer wait, but proceed with > > this change. We will look into removing the usage of > > suspend_flag introduced here once it is possible to implement > > it with handshakes. > > Yes, sure. > > >> Full: http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.4/ > > DeoptimizeObjectsALotThread is only used in compileBroker.cpp. > You can move both declaration and definition to that file, no need to clobber > thread.[c|h]pp. (and the static function deopt_objs_alot_thread_entry) > > Does JvmtiDeferredUpdates really need to be in thread.hpp, can't be in it's > own > hpp file? It doesn't seem right to add JVM TI classes into thread.hpp. > > Note that we also think we may have a bug in deopt: > https://bugs.openjdk.java.net/browse/JDK-8238237 > > I think it would be best, if possible, to push after that is resolved. > > Not even nearly a full review :) > > Thanks, Robbin > > > >> Incremental: > >> http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.4.inc/ > >> > >> I was not able to eliminate the additional suspend flag now. I'll take care > of this > >> as soon as the > >> existing suspend-resume-mechanism is reworked. > >> > >> Testing: > >> > >> Nightly tests @SAP: > >> > >> JCK and JTREG, also in Xcomp mode, SPECjvm2008, SPECjbb2015, > Renaissance > >> Suite, SAP specific tests > >> with fastdebug and release builds on all platforms > >> > >> Stress testing with DeoptimizeObjectsALot running SPECjvm2008 40x > parallel > >> for 24h > >> > >> Thanks, Richard. > >> > >> > >> More details on the changes: > >> > >> * Hide DeoptimizeObjectsALotThread from external view. > >> > >> * Changed EscapeBarrier_lock to be a _safepoint_check_never lock. > >> It used to be _safepoint_check_sometimes, which will be eliminated > sooner or > >> later. > >> I added explicit thread state changes with ThreadBlockInVM to code > paths > >> where we can wait() > >> on EscapeBarrier_lock to become safepoint safe. > >> > >> * Use handshake EscapeBarrierSuspendHandshake to suspend target > threads > >> instead of vm operation > >> VM_ThreadSuspendAllForObjDeopt. > >> > >> * Removed uses of Threads_lock. When adding a new thread we suspend > it iff > >> EA optimizations are > >> being reverted. In the previous version we were waiting on > Threads_lock > >> while EA optimizations > >> were reverted. See EscapeBarrier::thread_added(). > >> > >> * Made tests require Xmixed compilation mode. > >> > >> * Made tests agnostic regarding tiered compilation. > >> I.e. tc isn't disabled anymore, and the tests can be run with tc enabled or > >> disabled. > >> > >> * Exercising EATests.java as well with stress test options > >> DeoptimizeObjectsALot* > >> Due to the non-deterministic deoptimizations some tests need to be > skipped. > >> We do this to prevent bit-rot of the stress test code. > >> > >> * Executing EATests.java as well with graal if available. Driver for this is > >> EATestsJVMCI.java. Graal cannot pass all tests, because it does not > provide all > >> the new debug info > >> (namely not_global_escape_in_scope and arg_escape in > scopeDesc.hpp). > >> And graal does not yet support the JVMTI operations force early return > and > >> pop frame. > >> > >> * Removed tracing from new jdi tests in EATests.java. Too much trace > output > >> before the debugging > >> connection is established can cause deadlock because output buffers fill > up. > >> (See https://bugs.openjdk.java.net/browse/JDK-8173304) > >> > >> * Many copyright year changes and smaller clean-up changes of testing > code > >> (trailing white-space and > >> the like). > >> > >> > >> -----Original Message----- > >> From: David Holmes > >> Sent: Donnerstag, 19. Dezember 2019 03:12 > >> To: Reingruber, Richard ; serviceability- > >> dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; > hotspot- > >> runtime-dev at openjdk.java.net; Vladimir Kozlov > (vladimir.kozlov at oracle.com) > >> > >> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better > Performance in > >> the Presence of JVMTI Agents > >> > >> Hi Richard, > >> > >> I think my issue is with the way EliminateNestedLocks works so I'm going > >> to look into that more deeply. > >> > >> Thanks for the explanations. > >> > >> David > >> > >> On 18/12/2019 12:47 am, Reingruber, Richard wrote: > >>> Hi David, > >>> > >>> > > > Some further queries/concerns: > >>> > > > > >>> > > > src/hotspot/share/runtime/objectMonitor.cpp > >>> > > > > >>> > > > Can you please explain the changes to ObjectMonitor::wait: > >>> > > > > >>> > > > ! _recursions = save // restore the old recursion count > >>> > > > ! + jt->get_and_reset_relock_count_after_wait(); // > >>> > > > increased by the deferred relock count > >>> > > > > >>> > > > what is the "deferred relock count"? I gather it relates to > >>> > > > > >>> > > > "The code was extended to be able to deoptimize objects of a > >>> > > frame that > >>> > > > is not the top frame and to let another thread than the owning > >>> > > thread do > >>> > > > it." > >>> > > > >>> > > Yes, these relate. Currently EA based optimizations are reverted, > when a > >> compiled frame is > >>> > > replaced with corresponding interpreter frames. Part of this is > relocking > >> objects with eliminated > >>> > > locking. New with the enhancement is that we do this also just > before > >> object references are > >>> > > acquired through JVMTI. In this case we deoptimize also the > owning > >> compiled frame C and we > >>> > > register deoptimized objects as deferred updates. When control > returns > >> to C it gets deoptimized, > >>> > > we notice that objects are already deoptimized (reallocated and > >> relocked), so we don't do it again > >>> > > (relocking twice would be incorrect of course). Deferred updates > are > >> copied into the new > >>> > > interpreter frames. > >>> > > > >>> > > Problem: relocking is not possible if the target thread T is waiting > on the > >> monitor that needs to > >>> > > be relocked. This happens only with non-local objects with > >> EliminateNestedLocks. Instead relocking > >>> > > is deferred until T owns the monitor again. This is what the piece of > >> code above does. > >>> > > >>> > Sorry I need some more detail here. How can you wait() on an > object > >>> > monitor if the object allocation and/or locking was optimised away? > And > >>> > what is a "non-local object" in this context? Isn't EA restricted to > >>> > thread-confined objects? > >>> > >>> "Non-local object" is an object that escapes its thread. The issue I'm > >> addressing with the changes > >>> in ObjectMonitor::wait are almost unrelated to EA. They are caused by > >> EliminateNestedLocks, where C2 > >>> eliminates recursive locking of an already owned lock. The lock owning > object > >> exists on the heap, it > >>> is locked and you can call wait() on it. > >>> > >>> EliminateLocks is the C2 option that controls lock elimination based on > EA. > >> Both optimizations have > >>> in common that objects with eliminated locking need to be relocked > when > >> deoptimizing a frame, > >>> i.e. when replacing a compiled frame with equivalent interpreter > >>> frames. Deoptimization::relock_objects does that job for /all/ eliminated > >> locks in scope. /All/ can > >>> be a mix of eliminated nested locks and locks of not-escaping objects. > >>> > >>> New with the enhancement: I call relock_objects earlier, just before > objects > >> pontentially > >>> escape. But then later when the owning compiled frame gets > deoptimized, I > >> must not do it again: > >>> > >>> See call to EscapeBarrier::objs_are_deoptimized in deoptimization.cpp: > >>> > >>> 373 if ((jvmci_enabled || ((DoEscapeAnalysis || > EliminateNestedLocks) && > >> EliminateLocks)) > >>> 374 && !EscapeBarrier::objs_are_deoptimized(thread, > deoptee.id())) { > >>> 375 bool unused; > >>> 376 eliminate_locks(thread, chunk, realloc_failures, deoptee, > exec_mode, > >> unused); > >>> 377 } > >>> > >>> Now when calling relock_objects early it is quiet possible that I have to > relock > >> an object the > >>> target thread currently waits for. Obviously I cannot relock in this case, > >> instead I chose to > >>> introduce relock_count_after_wait to JavaThread. > >>> > >>> > Is it just that some of the locking gets optimized away e.g. > >>> > > >>> > synchronised(obj) { > >>> > synchronised(obj) { > >>> > synchronised(obj) { > >>> > obj.wait(); > >>> > } > >>> > } > >>> > } > >>> > > >>> > If this is reduced to a form as-if it were a single lock of the monitor > >>> > (due to EA) and the wait() triggers a JVM TI event which leads to the > >>> > escape of "obj" then we need to reconstruct the true lock state, and > so > >>> > when the wait() internally unblocks and reacquires the monitor it > has to > >>> > set the true recursion count to 3, not the 1 that it appeared to be > when > >>> > wait() was initially called. Is that the scenario? > >>> > >>> Kind of... except that the locking is not eliminated due to EA and there is > no > >> JVM TI event > >>> triggered by wait. > >>> > >>> Add > >>> > >>> LocalObject l1 = new LocalObject(); > >>> > >>> in front of the synchrnized blocks and assume a JVM TI agent acquires l1. > This > >> triggers the code in > >>> question. > >>> > >>> See that relocking/reallocating is transactional. If it is done then for /all/ > >> objects in scope and it is > >>> done at most once. It wouldn't be quite so easy to split this in relocking > of > >> nested/EA-based > >>> eliminated locks. > >>> > >>> > If so I find this truly awful. Anyone using wait() in a realistic form > >>> > requires a notification and so the object cannot be thread confined. > In > >>> > >>> It is not thread confined. > >>> > >>> > which case I would strongly argue that upon hitting the wait() the > deopt > >>> > should occur unconditionally and so the lock state is correct before > we > >>> > wait and so we don't need to mess with the recursion count > internally > >>> > when we reacquire the monitor. > >>> > > >>> > > > >>> > > > which I don't like the sound of at all when it comes to > ObjectMonitor > >>> > > > state. So I'd like to understand in detail exactly what is going on > here > >>> > > > and why. This is a very intrusive change that seems to badly > break > >>> > > > encapsulation and impacts future changes to ObjectMonitor > that are > >> under > >>> > > > investigation. > >>> > > > >>> > > I would not regard this as breaking encapsulation. Certainly not > badly. > >>> > > > >>> > > I've added a property relock_count_after_wait to JavaThread. The > >> property is well > >>> > > encapsulated. Future ObjectMonitor implementations have to deal > with > >> recursion too. They are free > >>> > > in choosing a way to do that as long as that property is taken into > >> account. This is hardly a > >>> > > limitation. > >>> > > >>> > I do think this badly breaks encapsulation as you have to add a > callout > >>> > from the guts of the ObjectMonitor code to reach into the thread to > get > >>> > this lock count adjustment. I understand why you have had to do > this but > >>> > I would much rather see a change to the EA optimisation strategy so > that > >>> > this is not needed. > >>> > > >>> > > Note also that the property is a straight forward extension of the > >> existing concept of deferred > >>> > > local updates. It is embedded into the structure holding them. So > not > >> even the footprint of a > >>> > > JavaThread is enlarged if no deferred updates are generated. > >>> > > >>> > [...] > >>> > > >>> > > > >>> > > I'm actually duplicating the existing external suspend mechanism, > >> because a thread can be > >>> > > suspended at most once. And hey, and don't like that either! But it > >> seems not unlikely that the > >>> > > duplicate can be removed together with the original and the new > type > >> of handshakes that will be > >>> > > used for thread suspend can be used for object deoptimization > too. See > >> today's discussion in > >>> > > JDK-8227745 [2]. > >>> > > >>> > I hope that discussion bears some fruit, at the moment it seems not > to > >>> > be possible to use handshakes here. :( > >>> > > >>> > The external suspend mechanism is a royal pain in the proverbial > that we > >>> > have to carefully live with. The idea that we're duplicating that for > >>> > use in another fringe area of functionality does not thrill me at all. > >>> > > >>> > To be clear, I understand the problem that exists and that you wish > to > >>> > solve, but for the runtime parts I balk at the complexity cost of > >>> > solving it. > >>> > >>> I know it's complex, but by far no rocket science. > >>> > >>> Also I find it hard to imagine another fix for JDK-8233915 besides > changing > >> the JVM TI specification. > >>> > >>> Thanks, Richard. > >>> > >>> -----Original Message----- > >>> From: David Holmes > >>> Sent: Dienstag, 17. Dezember 2019 08:03 > >>> To: Reingruber, Richard ; serviceability- > >> dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; > hotspot- > >> runtime-dev at openjdk.java.net; Vladimir Kozlov > (vladimir.kozlov at oracle.com) > >> > >>> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better > Performance > >> in the Presence of JVMTI Agents > >>> > >>> > >>> > >>> David > >>> > >>> On 17/12/2019 4:57 pm, David Holmes wrote: > >>>> Hi Richard, > >>>> > >>>> On 14/12/2019 5:01 am, Reingruber, Richard wrote: > >>>>> Hi David, > >>>>> > >>>>> ?? > Some further queries/concerns: > >>>>> ?? > > >>>>> ?? > src/hotspot/share/runtime/objectMonitor.cpp > >>>>> ?? > > >>>>> ?? > Can you please explain the changes to ObjectMonitor::wait: > >>>>> ?? > > >>>>> ?? > !?? _recursions = save????? // restore the old recursion count > >>>>> ?? > !???????????????? + jt->get_and_reset_relock_count_after_wait(); // > >>>>> ?? > increased by the deferred relock count > >>>>> ?? > > >>>>> ?? > what is the "deferred relock count"? I gather it relates to > >>>>> ?? > > >>>>> ?? > "The code was extended to be able to deoptimize objects of a > >>>>> frame that > >>>>> ?? > is not the top frame and to let another thread than the owning > >>>>> thread do > >>>>> ?? > it." > >>>>> > >>>>> Yes, these relate. Currently EA based optimizations are reverted, > when > >>>>> a compiled frame is replaced > >>>>> with corresponding interpreter frames. Part of this is relocking > >>>>> objects with eliminated > >>>>> locking. New with the enhancement is that we do this also just before > >>>>> object references are acquired > >>>>> through JVMTI. In this case we deoptimize also the owning compiled > >>>>> frame C and we register > >>>>> deoptimized objects as deferred updates. When control returns to C > it > >>>>> gets deoptimized, we notice > >>>>> that objects are already deoptimized (reallocated and relocked), so > we > >>>>> don't do it again (relocking > >>>>> twice would be incorrect of course). Deferred updates are copied into > >>>>> the new interpreter frames. > >>>>> > >>>>> Problem: relocking is not possible if the target thread T is waiting > >>>>> on the monitor that needs to be > >>>>> relocked. This happens only with non-local objects with > >>>>> EliminateNestedLocks. Instead relocking is > >>>>> deferred until T owns the monitor again. This is what the piece of > >>>>> code above does. > >>>> > >>>> Sorry I need some more detail here. How can you wait() on an object > >>>> monitor if the object allocation and/or locking was optimised away? > And > >>>> what is a "non-local object" in this context? Isn't EA restricted to > >>>> thread-confined objects? > >>>> > >>>> Is it just that some of the locking gets optimized away e.g. > >>>> > >>>> synchronised(obj) { > >>>> ? synchronised(obj) { > >>>> ??? synchronised(obj) { > >>>> ????? obj.wait(); > >>>> ??? } > >>>> ? } > >>>> } > >>>> > >>>> If this is reduced to a form as-if it were a single lock of the monitor > >>>> (due to EA) and the wait() triggers a JVM TI event which leads to the > >>>> escape of "obj" then we need to reconstruct the true lock state, and so > >>>> when the wait() internally unblocks and reacquires the monitor it has to > >>>> set the true recursion count to 3, not the 1 that it appeared to be when > >>>> wait() was initially called. Is that the scenario? > >>>> > >>>> If so I find this truly awful. Anyone using wait() in a realistic form > >>>> requires a notification and so the object cannot be thread confined. In > >>>> which case I would strongly argue that upon hitting the wait() the > deopt > >>>> should occur unconditionally and so the lock state is correct before we > >>>> wait and so we don't need to mess with the recursion count internally > >>>> when we reacquire the monitor. > >>>> > >>>>> > >>>>> ?? > which I don't like the sound of at all when it comes to > >>>>> ObjectMonitor > >>>>> ?? > state. So I'd like to understand in detail exactly what is going > >>>>> on here > >>>>> ?? > and why.? This is a very intrusive change that seems to badly > break > >>>>> ?? > encapsulation and impacts future changes to ObjectMonitor that > >>>>> are under > >>>>> ?? > investigation. > >>>>> > >>>>> I would not regard this as breaking encapsulation. Certainly not badly. > >>>>> > >>>>> I've added a property relock_count_after_wait to JavaThread. The > >>>>> property is well > >>>>> encapsulated. Future ObjectMonitor implementations have to deal > with > >>>>> recursion too. They are free in > >>>>> choosing a way to do that as long as that property is taken into > >>>>> account. This is hardly a > >>>>> limitation. > >>>> > >>>> I do think this badly breaks encapsulation as you have to add a callout > >>>> from the guts of the ObjectMonitor code to reach into the thread to > get > >>>> this lock count adjustment. I understand why you have had to do this > but > >>>> I would much rather see a change to the EA optimisation strategy so > that > >>>> this is not needed. > >>>> > >>>>> Note also that the property is a straight forward extension of the > >>>>> existing concept of deferred > >>>>> local updates. It is embedded into the structure holding them. So not > >>>>> even the footprint of a > >>>>> JavaThread is enlarged if no deferred updates are generated. > >>>>> > >>>>> ?? > --- > >>>>> ?? > > >>>>> ?? > src/hotspot/share/runtime/thread.cpp > >>>>> ?? > > >>>>> ?? > Can you please explain why > >>>>> JavaThread::wait_for_object_deoptimization > >>>>> ?? > has to be handcrafted in this way rather than using proper > >>>>> transitions. > >>>>> ?? > > >>>>> > >>>>> I wrote wait_for_object_deoptimization taking > >>>>> JavaThread::java_suspend_self_with_safepoint_check > >>>>> as template. So in short: for the same reasons :) > >>>>> > >>>>> Threads reach both methods as part of thread state transitions, > >>>>> therefore special handling is > >>>>> required to change thread state on top of ongoing transitions. > >>>>> > >>>>> ?? > We got rid of "deopt suspend" some time ago and it is disturbing > >>>>> to see > >>>>> ?? > it being added back (effectively). This seems like it may be > >>>>> something > >>>>> ?? > that handshakes could be used for. > >>>>> > >>>>> Deopt suspend used to be something rather different with a similar > >>>>> name[1]. It is not being added back. > >>>> > >>>> I stand corrected. Despite comments in the code to the contrary > >>>> deopt_suspend didn't actually cause a self-suspend. I was doing a lot of > >>>> cleanup in this area 13 years ago :) > >>>> > >>>>> > >>>>> I'm actually duplicating the existing external suspend mechanism, > >>>>> because a thread can be suspended > >>>>> at most once. And hey, and don't like that either! But it seems not > >>>>> unlikely that the duplicate can > >>>>> be removed together with the original and the new type of > handshakes > >>>>> that will be used for > >>>>> thread suspend can be used for object deoptimization too. See > today's > >>>>> discussion in JDK-8227745 [2]. > >>>> > >>>> I hope that discussion bears some fruit, at the moment it seems not to > >>>> be possible to use handshakes here. :( > >>>> > >>>> The external suspend mechanism is a royal pain in the proverbial that > we > >>>> have to carefully live with. The idea that we're duplicating that for > >>>> use in another fringe area of functionality does not thrill me at all. > >>>> > >>>> To be clear, I understand the problem that exists and that you wish to > >>>> solve, but for the runtime parts I balk at the complexity cost of > >>>> solving it. > >>>> > >>>> Thanks, > >>>> David > >>>> ----- > >>>> > >>>>> Thanks, Richard. > >>>>> > >>>>> [1] Deopt suspend was something like an async. handshake for > >>>>> architectures with register windows, > >>>>> ???? where patching the return pc for deoptimization of a compiled > >>>>> frame was racy if the owner thread > >>>>> ???? was in native code. Instead a "deopt" suspend flag was set on > >>>>> which the thread patched its own > >>>>> ???? frame upon return from native. So no thread was suspended. It > got > >>>>> its name only from the name of > >>>>> ???? the flags. > >>>>> > >>>>> [2] Discussion about using handshakes to sync. with the target thread: > >>>>> > >>>>> https://bugs.openjdk.java.net/browse/JDK- > >> > 8227745?focusedCommentId=14306727&page=com.atlassian.jira.plugin.syst > e > >> m.issuetabpanels:comment-tabpanel#comment-14306727 > >>>>> > >>>>> > >>>>> -----Original Message----- > >>>>> From: David Holmes > >>>>> Sent: Freitag, 13. Dezember 2019 00:56 > >>>>> To: Reingruber, Richard ; > >>>>> serviceability-dev at openjdk.java.net; > >>>>> hotspot-compiler-dev at openjdk.java.net; > >>>>> hotspot-runtime-dev at openjdk.java.net > >>>>> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better > >>>>> Performance in the Presence of JVMTI Agents > >>>>> > >>>>> Hi Richard, > >>>>> > >>>>> Some further queries/concerns: > >>>>> > >>>>> src/hotspot/share/runtime/objectMonitor.cpp > >>>>> > >>>>> Can you please explain the changes to ObjectMonitor::wait: > >>>>> > >>>>> !?? _recursions = save????? // restore the old recursion count > >>>>> !???????????????? + jt->get_and_reset_relock_count_after_wait(); // > >>>>> increased by the deferred relock count > >>>>> > >>>>> what is the "deferred relock count"? I gather it relates to > >>>>> > >>>>> "The code was extended to be able to deoptimize objects of a frame > that > >>>>> is not the top frame and to let another thread than the owning thread > do > >>>>> it." > >>>>> > >>>>> which I don't like the sound of at all when it comes to ObjectMonitor > >>>>> state. So I'd like to understand in detail exactly what is going on here > >>>>> and why.? This is a very intrusive change that seems to badly break > >>>>> encapsulation and impacts future changes to ObjectMonitor that are > under > >>>>> investigation. > >>>>> > >>>>> --- > >>>>> > >>>>> src/hotspot/share/runtime/thread.cpp > >>>>> > >>>>> Can you please explain why > JavaThread::wait_for_object_deoptimization > >>>>> has to be handcrafted in this way rather than using proper transitions. > >>>>> > >>>>> We got rid of "deopt suspend" some time ago and it is disturbing to > see > >>>>> it being added back (effectively). This seems like it may be something > >>>>> that handshakes could be used for. > >>>>> > >>>>> Thanks, > >>>>> David > >>>>> ----- > >>>>> > >>>>> On 12/12/2019 7:02 am, David Holmes wrote: > >>>>>> On 12/12/2019 1:07 am, Reingruber, Richard wrote: > >>>>>>> Hi David, > >>>>>>> > >>>>>>> ??? > Most of the details here are in areas I can comment on in > detail, > >>>>>>> but I > >>>>>>> ??? > did take an initial general look at things. > >>>>>>> > >>>>>>> Thanks for taking the time! > >>>>>> > >>>>>> Apologies the above should read: > >>>>>> > >>>>>> "Most of the details here are in areas I *can't* comment on in detail > >>>>>> ..." > >>>>>> > >>>>>> David > >>>>>> > >>>>>>> ??? > The only thing that jumped out at me is that I think the > >>>>>>> ??? > DeoptimizeObjectsALotThread should be a hidden thread. > >>>>>>> ??? > > >>>>>>> ??? > +? bool is_hidden_from_external_view() const { return true; } > >>>>>>> > >>>>>>> Yes, it should. Will add the method like above. > >>>>>>> > >>>>>>> ??? > Also I don't see any testing of the > DeoptimizeObjectsALotThread. > >>>>>>> Without > >>>>>>> ??? > active testing this will just bit-rot. > >>>>>>> > >>>>>>> DeoptimizeObjectsALot is meant for stress testing with a larger > >>>>>>> workload. I will add a minimal test > >>>>>>> to keep it fresh. > >>>>>>> > >>>>>>> ??? > Also on the tests I don't understand your @requires clause: > >>>>>>> ??? > > >>>>>>> ??? >?? @requires ((vm.compMode != "Xcomp") & > vm.compiler2.enabled > >> & > >>>>>>> ??? > (vm.opt.TieredCompilation != true)) > >>>>>>> ??? > > >>>>>>> ??? > This seems to require that TieredCompilation is disabled, but > >>>>>>> tiered is > >>>>>>> ??? > our normal mode of operation. ?? > >>>>>>> ??? > > >>>>>>> > >>>>>>> I removed the clause. I guess I wanted to target the tests towards > the > >>>>>>> code they are supposed to > >>>>>>> test, and it's easier to analyze failures w/o tiered compilation and > >>>>>>> with just one compiler thread. > >>>>>>> > >>>>>>> Additionally I will make use of > >>>>>>> compiler.whitebox.CompilerWhiteBoxTest.THRESHOLD in the tests. > >>>>>>> > >>>>>>> Thanks, > >>>>>>> Richard. > >>>>>>> > >>>>>>> -----Original Message----- > >>>>>>> From: David Holmes > >>>>>>> Sent: Mittwoch, 11. Dezember 2019 08:03 > >>>>>>> To: Reingruber, Richard ; > >>>>>>> serviceability-dev at openjdk.java.net; > >>>>>>> hotspot-compiler-dev at openjdk.java.net; > >>>>>>> hotspot-runtime-dev at openjdk.java.net > >>>>>>> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better > >>>>>>> Performance in the Presence of JVMTI Agents > >>>>>>> > >>>>>>> Hi Richard, > >>>>>>> > >>>>>>> On 11/12/2019 7:45 am, Reingruber, Richard wrote: > >>>>>>>> Hi, > >>>>>>>> > >>>>>>>> I would like to get reviews please for > >>>>>>>> > >>>>>>>> > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.3/ > >>>>>>>> > >>>>>>>> Corresponding RFE: > >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8227745 > >>>>>>>> > >>>>>>>> Fixes also https://bugs.openjdk.java.net/browse/JDK-8233915 > >>>>>>>> And potentially https://bugs.openjdk.java.net/browse/JDK- > 8214584 [1] > >>>>>>>> > >>>>>>>> Vladimir Kozlov kindly put webrev.3 through tier1-8 testing > without > >>>>>>>> issues (thanks!). In addition the > >>>>>>>> change is being tested at SAP since I posted the first RFR some > >>>>>>>> months ago. > >>>>>>>> > >>>>>>>> The intention of this enhancement is to benefit performance wise > from > >>>>>>>> escape analysis even if JVMTI > >>>>>>>> agents request capabilities that allow them to access local variable > >>>>>>>> values. E.g. if you start-up > >>>>>>>> with -agentlib:jdwp=transport=dt_socket,server=y,suspend=n, > then > >>>>>>>> escape analysis is disabled right > >>>>>>>> from the beginning, well before a debugger attaches -- if ever one > >>>>>>>> should do so. With the > >>>>>>>> enhancement, escape analysis will remain enabled until and after > a > >>>>>>>> debugger attaches. EA based > >>>>>>>> optimizations are reverted just before an agent acquires the > >>>>>>>> reference to an object. In the JBS item > >>>>>>>> you'll find more details. > >>>>>>> > >>>>>>> Most of the details here are in areas I can comment on in detail, but > I > >>>>>>> did take an initial general look at things. > >>>>>>> > >>>>>>> The only thing that jumped out at me is that I think the > >>>>>>> DeoptimizeObjectsALotThread should be a hidden thread. > >>>>>>> > >>>>>>> +? bool is_hidden_from_external_view() const { return true; } > >>>>>>> > >>>>>>> Also I don't see any testing of the DeoptimizeObjectsALotThread. > >>>>>>> Without > >>>>>>> active testing this will just bit-rot. > >>>>>>> > >>>>>>> Also on the tests I don't understand your @requires clause: > >>>>>>> > >>>>>>> ??? @requires ((vm.compMode != "Xcomp") & > vm.compiler2.enabled & > >>>>>>> (vm.opt.TieredCompilation != true)) > >>>>>>> > >>>>>>> This seems to require that TieredCompilation is disabled, but tiered > is > >>>>>>> our normal mode of operation. ?? > >>>>>>> > >>>>>>> Thanks, > >>>>>>> David > >>>>>>> > >>>>>>>> Thanks, > >>>>>>>> Richard. > >>>>>>>> > >>>>>>>> [1] Experimental fix for JDK-8214584 based on JDK-8227745 > >>>>>>>> > >> > http://cr.openjdk.java.net/~rrich/webrevs/2019/8214584/experiment_v1.pa > tc > >> h > >>>>>>>> > >>>>>>>> > >>>>>>>> From richard.reingruber at sap.com Mon Mar 30 08:31:30 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Mon, 30 Mar 2020 08:31:30 +0000 Subject: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents In-Reply-To: References: <1f8a3c7a-fa0f-b5b2-4a8a-7d3d8dbbe1b5@oracle.com> <4b56a45c-a14c-6f74-2bfd-25deaabe8201@oracle.com> <5271429a-481d-ddb9-99dc-b3f6670fcc0b@oracle.com> Message-ID: Hi, this is webrev.5 based on Robbin's feedback and Martin's review - thanks! :) The change affects jvmti, hotspot and c2. Partial reviews are very welcome too. Full: http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.5/ Delta: http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.5.inc/ Robbin, Martin, please let me know, if anything shouldn't be quite as you wanted it. Also find my comments on your feedback below. Robbin, can I count you as Reviewer for the runtime part? Thanks, Richard. -- > DeoptimizeObjectsALotThread is only used in compileBroker.cpp. > You can move both declaration and definition to that file, no need to clobber > thread.[c|h]pp. (and the static function deopt_objs_alot_thread_entry) Done. > Does JvmtiDeferredUpdates really need to be in thread.hpp, can't be in it's own > hpp file? It doesn't seem right to add JVM TI classes into thread.hpp. I moved JvmtiDeferredUpdates to vframe_hp.hpp where preexisting jvmtiDeferredLocalVariableSet is declared. > src/hotspot/share/code/compiledMethod.cpp > Nice cleanup! Thanks :) > src/hotspot/share/code/debugInfoRec.cpp > src/hotspot/share/code/debugInfoRec.hpp > Additional parmeters. (Remark: I think "non_global_escape_in_scope" would read better than "not_global_escape_in_scope", but your version is consistent with existing code, so no change request from my side.) Ok. I've been thinking about this too and finally stayed with not_global_escape_in_scope. It's supposed to mean an object whose escape state is not GlobalEscape is in scope. > src/hotspot/share/compiler/compileBroker.cpp > src/hotspot/share/compiler/compileBroker.hpp > Extra thread for DeoptimizeObjectsALot. (Remark: I would have put it into a follow up change together with the test in order to make this webrev smaller, but since it is included, I'm reviewing everything at once. Not a big deal.) Ok. Yes the change would be a little smaller. And if it helps I'll split it off. In general I prefer patches that bring along a suitable amount of tests. > src/hotspot/share/opto/c2compiler.cpp > Make do_escape_analysis independent of JVMCI capabilities. Nice! It is the main goal of the enhancement. It is done for C2, but could be done for JVMCI compilers with just a small effort as well. > src/hotspot/share/opto/escape.cpp > Annotation for MachSafePointNodes. Your added functionality looks correct. > But I'd prefer to move the bulky code out of the large function. > I suggest to factor out something like has_not_global_escape and has_arg_escape. So the code could look like this: > SafePointNode* sfn = sfn_worklist.at(next); > sfn->set_not_global_escape_in_scope(has_not_global_escape(sfn)); > if (sfn->is_CallJava()) { > CallJavaNode* call = sfn->as_CallJava(); > call->set_arg_escape(has_arg_escape(call)); > } > This would also allow us to get rid of the found_..._escape_in_args variables making the loops better readable. Done. > It's kind of ugly to use strcmp to recognize uncommon trap, but that seems to be the way to do it (there are more such places). So it's ok. Yeah. I copied the snippet. > src/hotspot/share/prims/jvmtiImpl.cpp > src/hotspot/share/prims/jvmtiImpl.hpp > The sequence is pretty complex: > VM_GetOrSetLocal element initialization executes EscapeBarrier code which suspends the target thread (extra VM Operation). Note that the target threads have to be suspended already for VM_GetOrSetLocal*. So it's mainly the synchronization effect of EscapeBarrier::sync_and_suspend_one() that is required here. Also no extra _handshake_ is executed, since sync_and_suspend_one() will find the target threads already suspended. > VM_GetOrSetLocal::doit_prologue performs object deoptimization (by VM Thread to prepare VM Operation with frame deoptimization). > VM_GetOrSetLocal destructor implicitly calls EscapeBarrier destructor which resumes the target thread. > But I don't have any improvement proposal. Performance is probably not a concern, here. So it's ok. > VM_GetOrSetLocal::deoptimize_objects deoptimizes the top frame if it has non-globally escaping objects and other frames if they have arg escaping ones. Good. It's not specifically the top frame, but the frame that is accessed. > src/hotspot/share/runtime/deoptimization.cpp > Object deoptimization. I have more comments and proposals, here. > First of all, handling recursive and waiting locks in relock_objects is tricky, but looks correct. > Comments are sufficient to understand why things are done as they are implemented. > BiasedLocking related parts are complex, but we may get rid of them in the future (with BiasedLocking removal). > Anyway, looks correct, too. > Typo in comment: "regularily" => "regularly" > Deoptimization::fetch_unroll_info_helper is the only place where _jvmti_deferred_updates get deallocated (except JavaThread destructor). But I think we always go through it, so I can't see a memory leak or such kind of issues. That's correct. The compiled frame for which deferred updates are allocated is always deoptimized before (see EscapeBarrier::deoptimize_objects()). This is also asserted in compiledVFrame::update_deferred_value(). I've added the same assertion to Deoptimization::relock_objects(). So we can be sure that _jvmti_deferred_updates are deallocated again in fetch_unroll_info_helper(). > EscapeBarrier::deoptimize_objects: ResourceMark should use calling_thread(). Sure, well spotted! > You can use MutexLocker and MonitorLocker with Thread* to save the Thread::current() call. Right, good hint. This was recently introduced with 8235678. I even had to resolve conflicts. Should have done this then. > I'd make set_objs_are_deoptimized static and remove it from the EscapeBarrier interface because I think it shouldn't be used outside of EscapeBarrier::deoptimize_objects. Done. > Typo in comment: "we must only deoptimize" => "we only have to deoptimize" Replaced with "[...] we deoptimize iff local objects are passed as args" > "bool EscapeBarrier::deoptimize_objects(intptr_t* fr_id)" is trivial and barrier_active() is redundant. Implementation can get moved to hpp file. Ok. Done. > I'll get back to suspend flags, later. > There are weird cases regarding _self_deoptimization_in_progress. > Assume we have 3 threads A, B and C. A deopts C, B deopts C, C deopts C. C can set _self_deoptimization_in_progress while A performs the handshake for suspending C. I think this doesn't lead to errors, but it's probably not desired. > I think it would be better to use only one "wait" call in sync_and_suspend_one and sync_and_suspend_all. You're right. We've discussed that face-to-face, but couldn't find a real issue. But now, thinking again, a reckon I found one: 2808 // Sync with other threads that might be doing deoptimizations 2809 { 2810 // Need to switch to _thread_blocked for the wait() call 2811 ThreadBlockInVM tbivm(_calling_thread); 2812 MonitorLocker ml(EscapeBarrier_lock, Mutex::_no_safepoint_check_flag); 2813 while (_self_deoptimization_in_progress) { 2814 ml.wait(); 2815 } 2816 2817 if (self_deopt()) { 2818 _self_deoptimization_in_progress = true; 2819 } 2820 2821 while (_deoptee_thread->is_ea_obj_deopt_suspend()) { 2822 ml.wait(); 2823 } 2824 2825 if (self_deopt()) { 2826 return; 2827 } 2828 2829 // set suspend flag for target thread 2830 _deoptee_thread->set_ea_obj_deopt_flag(); 2831 } - A waits in 2822 - C is suspended - B notifies all in resume_one() - A and C wake up - C wins over A and sets _self_deoptimization_in_progress = true in 2818 - C does the self deoptimization - A executes 2830 _deoptee_thread->set_ea_obj_deopt_flag() C will self suspend at some undefined point. The resulting state is illegal. > I first thought it'd be better to move ThreadBlockInVM before wait() to reduce thread state transitions, but that seems to be problematic because ThreadBlockInVM destructor contains a safepoint check which we shouldn't do while holding EscapeBarrier_lock. So no change request. Yes, would be nice to have the state change only if needed, but for the reason you mentioned it is not quite as easy as it seems to be. I experimented as well with a second lock, but did not succeed. > Change in thred_added: > I think the sequence would be more comprehensive if we waited for deopt_all_threads in Thread::start and all other places where a new thread can run into Java code (e.g. JVMTI attach). > Your version makes new threads come up with suspend flag set. That looks correct, too. Advantage is that you only have to change one place (thread_added). It'll be interesting to see how it will look like when we use async handshakes instead of suspend flags. > For now, I'm ok with your version. I had a version that did what you are suggesting. The current version also has the advantage, that there are fewer places where a thread has to wait for ongoing object deoptimization. This means viewer places where you have to worry about correct thread state transitions, possible deadlocks, and if all oops are properly Handle'ed. > I'd only move MutexLocker ml(EscapeBarrier_lock...) after if (!jt->is_hidden_from_external_view()). Done. > Having 4 different deoptimize_objects functions makes it a little hard to keep an overview of which one is used for what. > Maybe adding suffixes would help a little bit, but I can also live with what you have. > Implementation looks correct to me. 2 are internal. I added the suffix _internal to them. This leaves 2 to choose from. > src/hotspot/share/runtime/deoptimization.hpp > Escape barriers and object deoptimization functions. > Typo in comment: "helt" => "held" Done in place already. > src/hotspot/share/runtime/interfaceSupport.cpp > InterfaceSupport::deoptimizeAllObjects() is only used for DeoptimizeObjectsALot = 1. > I think DeoptimizeObjectsALot = 2 is more important, but I think it's not bad to have DeoptimizeObjectsALot = 1 in addition. Ok. I never used DeoptimizeObjectsALot = 1 that much. It could be more deterministic in single threaded scenarios. I wouldn't object to get rid of it though. > src/hotspot/share/runtime/stackValue.hpp > Better reinitilization in StackValue. Good. StackValue::obj_is_scalar_replaced() should not return true after calling set_obj(). > src/hotspot/share/runtime/thread.cpp > src/hotspot/share/runtime/thread.hpp > src/hotspot/share/runtime/thread.inline.hpp > wait_for_object_deoptimization, suspend flag, deferred updates and test feature to deoptimize objects. > In the long term, we want to get rid of suspend flags, so it's not so nice to introduce a new one. But I agree with G?tz that it should be acceptable as temporary solution until async handshakes are available (which takes more time). So I'm ok with your change. I'm keen to build the feature on async handshakes when the arive. > You can use MutexLocker with Thread*. Done. > JVMTIDeferredUpdates: I agree with Robin. It'd be nice to move the class out of thread.hpp. Done. > src/hotspot/share/runtime/vframe.cpp > Added support for entry frame to new_vframe. Ok. > src/hotspot/share/runtime/vframe_hp.cpp > src/hotspot/share/runtime/vframe_hp.hpp > I think code()->as_nmethod() in not_global_escape_in_scope() and arg_escape() should better be under #ifdef ASSERT or inside the assert statement (no need for code cache walking in product build). Done. > jvmtiDeferredLocalVariableSet::update_monitors: > Please add a comment explaining that owner referenced by original info may be scalar replaced, but it is deoptimized in the vframe. Done. -----Original Message----- From: Doerr, Martin Sent: Donnerstag, 12. M?rz 2020 17:28 To: Reingruber, Richard ; 'Robbin Ehn' ; Lindenmaier, Goetz ; David Holmes ; Vladimir Kozlov (vladimir.kozlov at oracle.com) ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents Hi Richard, I managed to find time for a (almost) complete review of webrev.4. (I'll review the tests separately.) First of all, the change seems to be in pretty good quality for its significant complexity. I couldn't find any real bugs. But I'd like to propose minor improvements. I'm convinced that it's mature because we did substantial testing. I like the new functionality for object deoptimization. It can possibly be reused for future escape analysis based optimizations. So I appreciate having it available in the code base. In addition to that, your change makes the JVMTI implementation better integrated into the VM. Now to the details: src/hotspot/share/c1/c1_IR.hpp describe_scope parameters. Ok. src/hotspot/share/ci/ciEnv.cpp src/hotspot/share/ci/ciEnv.hpp Fix for JvmtiExport::can_walk_any_space() capability. Ok. src/hotspot/share/code/compiledMethod.cpp Nice cleanup! src/hotspot/share/code/debugInfoRec.cpp src/hotspot/share/code/debugInfoRec.hpp Additional parmeters. (Remark: I think "non_global_escape_in_scope" would read better than "not_global_escape_in_scope", but your version is consistent with existing code, so no change request from my side.) Ok. src/hotspot/share/code/nmethod.cpp Nice cleanup! src/hotspot/share/code/pcDesc.hpp Additional parameters. Ok. src/hotspot/share/code/scopeDesc.cpp src/hotspot/share/code/scopeDesc.hpp Improved implementation + additional parameters. Ok. src/hotspot/share/compiler/compileBroker.cpp src/hotspot/share/compiler/compileBroker.hpp Extra thread for DeoptimizeObjectsALot. (Remark: I would have put it into a follow up change together with the test in order to make this webrev smaller, but since it is included, I'm reviewing everything at once. Not a big deal.) Ok. src/hotspot/share/jvmci/jvmciCodeInstaller.cpp Additional parameters. Ok. src/hotspot/share/opto/c2compiler.cpp Make do_escape_analysis independent of JVMCI capabilities. Nice! src/hotspot/share/opto/callnode.hpp Additional fields for MachSafePointNodes. Ok. src/hotspot/share/opto/escape.cpp Annotation for MachSafePointNodes. Your added functionality looks correct. But I'd prefer to move the bulky code out of the large function. I suggest to factor out something like has_not_global_escape and has_arg_escape. So the code could look like this: SafePointNode* sfn = sfn_worklist.at(next); sfn->set_not_global_escape_in_scope(has_not_global_escape(sfn)); if (sfn->is_CallJava()) { CallJavaNode* call = sfn->as_CallJava(); call->set_arg_escape(has_arg_escape(call)); } This would also allow us to get rid of the found_..._escape_in_args variables making the loops better readable. It's kind of ugly to use strcmp to recognize uncommon trap, but that seems to be the way to do it (there are more such places). So it's ok. src/hotspot/share/opto/machnode.hpp Additional fields for MachSafePointNodes. Ok. src/hotspot/share/opto/macro.cpp Allow elimination of non-escaping allocations. Ok. src/hotspot/share/opto/matcher.cpp src/hotspot/share/opto/output.cpp Copy attribute / pass parameters. Ok. src/hotspot/share/prims/jvmtiCodeBlobEvents.cpp Nice cleanup! src/hotspot/share/prims/jvmtiEnv.cpp src/hotspot/share/prims/jvmtiEnvBase.cpp Escape barriers + deoptimize objects for target thread. Good. src/hotspot/share/prims/jvmtiImpl.cpp src/hotspot/share/prims/jvmtiImpl.hpp The sequence is pretty complex: VM_GetOrSetLocal element initialization executes EscapeBarrier code which suspends the target thread (extra VM Operation). VM_GetOrSetLocal::doit_prologue performs object deoptimization (by VM Thread to prepare VM Operation with frame deoptimization). VM_GetOrSetLocal destructor implicitly calls EscapeBarrier destructor which resumes the target thread. But I don't have any improvement proposal. Performance is probably not a concern, here. So it's ok. VM_GetOrSetLocal::deoptimize_objects deoptimizes the top frame if it has non-globally escaping objects and other frames if they have arg escaping ones. Good. src/hotspot/share/prims/jvmtiTagMap.cpp Escape barriers + deoptimize objects for all threads. Ok. src/hotspot/share/prims/whitebox.cpp Added WB_IsFrameDeoptimized to API. Ok. src/hotspot/share/runtime/deoptimization.cpp Object deoptimization. I have more comments and proposals, here. First of all, handling recursive and waiting locks in relock_objects is tricky, but looks correct. Comments are sufficient to understand why things are done as they are implemented. BiasedLocking related parts are complex, but we may get rid of them in the future (with BiasedLocking removal). Anyway, looks correct, too. Typo in comment: "regularily" => "regularly" Deoptimization::fetch_unroll_info_helper is the only place where _jvmti_deferred_updates get deallocated (except JavaThread destructor). But I think we always go through it, so I can't see a memory leak or such kind of issues. EscapeBarrier::deoptimize_objects: ResourceMark should use calling_thread(). You can use MutexLocker and MonitorLocker with Thread* to save the Thread::current() call. I'd make set_objs_are_deoptimized static and remove it from the EscapeBarrier interface because I think it shouldn't be used outside of EscapeBarrier::deoptimize_objects. Typo in comment: "we must only deoptimize" => "we only have to deoptimize" "bool EscapeBarrier::deoptimize_objects(intptr_t* fr_id)" is trivial and barrier_active() is redundant. Implementation can get moved to hpp file. I'll get back to suspend flags, later. There are weird cases regarding _self_deoptimization_in_progress. Assume we have 3 threads A, B and C. A deopts C, B deopts C, C deopts C. C can set _self_deoptimization_in_progress while A performs the handshake for suspending C. I think this doesn't lead to errors, but it's probably not desired. I think it would be better to use only one "wait" call in sync_and_suspend_one and sync_and_suspend_all. I first thought it'd be better to move ThreadBlockInVM before wait() to reduce thread state transitions, but that seems to be problematic because ThreadBlockInVM destructor contains a safepoint check which we shouldn't do while holding EscapeBarrier_lock. So no change request. Change in thred_added: I think the sequence would be more comprehensive if we waited for deopt_all_threads in Thread::start and all other places where a new thread can run into Java code (e.g. JVMTI attach). Your version makes new threads come up with suspend flag set. That looks correct, too. Advantage is that you only have to change one place (thread_added). It'll be interesting to see how it will look like when we use async handshakes instead of suspend flags. For now, I'm ok with your version. I'd only move MutexLocker ml(EscapeBarrier_lock...) after if (!jt->is_hidden_from_external_view()). Having 4 different deoptimize_objects functions makes it a little hard to keep an overview of which one is used for what. Maybe adding suffixes would help a little bit, but I can also live with what you have. Implementation looks correct to me. src/hotspot/share/runtime/deoptimization.hpp Escape barriers and object deoptimization functions. Typo in comment: "helt" => "held" src/hotspot/share/runtime/globals.hpp Addition of develop flag DeoptimizeObjectsALotInterval. Ok. src/hotspot/share/runtime/interfaceSupport.cpp InterfaceSupport::deoptimizeAllObjects() is only used for DeoptimizeObjectsALot = 1. I think DeoptimizeObjectsALot = 2 is more important, but I think it's not bad to have DeoptimizeObjectsALot = 1 in addition. Ok. src/hotspot/share/runtime/interfaceSupport.inline.hpp Addition of deoptimizeAllObjects. Ok. src/hotspot/share/runtime/mutexLocker.cpp src/hotspot/share/runtime/mutexLocker.hpp Addition of EscapeBarrier_lock. Ok. src/hotspot/share/runtime/objectMonitor.cpp Make recursion count relock aware. Ok. src/hotspot/share/runtime/stackValue.hpp Better reinitilization in StackValue. Good. src/hotspot/share/runtime/thread.cpp src/hotspot/share/runtime/thread.hpp src/hotspot/share/runtime/thread.inline.hpp wait_for_object_deoptimization, suspend flag, deferred updates and test feature to deoptimize objects. In the long term, we want to get rid of suspend flags, so it's not so nice to introduce a new one. But I agree with G?tz that it should be acceptable as temporary solution until async handshakes are available (which takes more time). So I'm ok with your change. You can use MutexLocker with Thread*. JVMTIDeferredUpdates: I agree with Robin. It'd be nice to move the class out of thread.hpp. src/hotspot/share/runtime/vframe.cpp Added support for entry frame to new_vframe. Ok. src/hotspot/share/runtime/vframe_hp.cpp src/hotspot/share/runtime/vframe_hp.hpp I think code()->as_nmethod() in not_global_escape_in_scope() and arg_escape() should better be under #ifdef ASSERT or inside the assert statement (no need for code cache walking in product build). jvmtiDeferredLocalVariableSet::update_monitors: Please add a comment explaining that owner referenced by original info may be scalar replaced, but it is deoptimized in the vframe. src/hotspot/share/utilities/macros.hpp Addition of NOT_COMPILER2_OR_JVMCI_RETURN macros. Ok. test/hotspot/jtreg/serviceability/jvmti/Heap/IterateHeapWithEscapeAnalysisEnabled.java test/hotspot/jtreg/serviceability/jvmti/Heap/libIterateHeapWithEscapeAnalysisEnabled.c New test. Will review separately. test/jdk/TEST.ROOT Addition of vm.jvmci as required property. Ok. test/jdk/com/sun/jdi/EATests.java test/jdk/com/sun/jdi/EATestsJVMCI.java New test. Will review separately. test/lib/sun/hotspot/WhiteBox.java Added isFrameDeoptimized to API. Ok. That was it. Best regards, Martin > -----Original Message----- > From: hotspot-compiler-dev bounces at openjdk.java.net> On Behalf Of Reingruber, Richard > Sent: Dienstag, 3. M?rz 2020 21:23 > To: 'Robbin Ehn' ; Lindenmaier, Goetz > ; David Holmes ; > Vladimir Kozlov (vladimir.kozlov at oracle.com) > ; serviceability-dev at openjdk.java.net; > hotspot-compiler-dev at openjdk.java.net; hotspot-runtime- > dev at openjdk.java.net > Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better > Performance in the Presence of JVMTI Agents > > Hi Robbin, > > > > I understand that Robbin proposed to replace the usage of > > > _suspend_flag with handshakes. Apparently, async handshakes > > > are needed to do so. We have been waiting a while for removal > > > of the _suspend_flag / introduction of async handshakes [2]. > > > What is the status here? > > > I have an old prototype which I would like to continue to work on. > > So do not assume asynch handshakes will make 15. > > Even if it would, I think there are a lot more investigate work to remove > > _suspend_flag. > > Let us know, if we can be of any help to you and be it only testing. > > > >> Full: > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.4/ > > > DeoptimizeObjectsALotThread is only used in compileBroker.cpp. > > You can move both declaration and definition to that file, no need to > clobber > > thread.[c|h]pp. (and the static function deopt_objs_alot_thread_entry) > > Will do. > > > Does JvmtiDeferredUpdates really need to be in thread.hpp, can't be in it's > own > > hpp file? It doesn't seem right to add JVM TI classes into thread.hpp. > > You are right. It shouldn't be declared in thread.hpp. I will look into that. > > > Note that we also think we may have a bug in deopt: > > https://bugs.openjdk.java.net/browse/JDK-8238237 > > > I think it would be best, if possible, to push after that is resolved. > > Sure. > > > Not even nearly a full review :) > > I know :) > > Anyways, thanks a lot, > Richard. > > > -----Original Message----- > From: Robbin Ehn > Sent: Monday, March 2, 2020 11:17 AM > To: Lindenmaier, Goetz ; Reingruber, Richard > ; David Holmes ; > Vladimir Kozlov (vladimir.kozlov at oracle.com) > ; serviceability-dev at openjdk.java.net; > hotspot-compiler-dev at openjdk.java.net; hotspot-runtime- > dev at openjdk.java.net > Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better Performance > in the Presence of JVMTI Agents > > Hi, > > On 2/24/20 5:39 PM, Lindenmaier, Goetz wrote: > > Hi, > > > > I had a look at the progress of this change. Nothing > > happened since Richard posted his update using more > > handshakes [1]. > > But we (SAP) would appreciate a lot if this change could > > be successfully reviewed and pushed. > > > > I think there is basic understanding that this > > change is helpful. It fixes a number of issues with JVMTI, > > and will deliver the same performance benefits as EA > > does in current production mode for debugging scenarios. > > > > This is important for us as we run our VMs prepared > > for debugging in production mode. > > > > I understand that Robbin proposed to replace the usage of > > _suspend_flag with handshakes. Apparently, async handshakes > > are needed to do so. We have been waiting a while for removal > > of the _suspend_flag / introduction of async handshakes [2]. > > What is the status here? > > I have an old prototype which I would like to continue to work on. > So do not assume asynch handshakes will make 15. > Even if it would, I think there are a lot more investigate work to remove > _suspend_flag. > > > > > I think we should no longer wait, but proceed with > > this change. We will look into removing the usage of > > suspend_flag introduced here once it is possible to implement > > it with handshakes. > > Yes, sure. > > >> Full: http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.4/ > > DeoptimizeObjectsALotThread is only used in compileBroker.cpp. > You can move both declaration and definition to that file, no need to clobber > thread.[c|h]pp. (and the static function deopt_objs_alot_thread_entry) > > Does JvmtiDeferredUpdates really need to be in thread.hpp, can't be in it's > own > hpp file? It doesn't seem right to add JVM TI classes into thread.hpp. > > Note that we also think we may have a bug in deopt: > https://bugs.openjdk.java.net/browse/JDK-8238237 > > I think it would be best, if possible, to push after that is resolved. > > Not even nearly a full review :) > > Thanks, Robbin > > > >> Incremental: > >> http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.4.inc/ > >> > >> I was not able to eliminate the additional suspend flag now. I'll take care > of this > >> as soon as the > >> existing suspend-resume-mechanism is reworked. > >> > >> Testing: > >> > >> Nightly tests @SAP: > >> > >> JCK and JTREG, also in Xcomp mode, SPECjvm2008, SPECjbb2015, > Renaissance > >> Suite, SAP specific tests > >> with fastdebug and release builds on all platforms > >> > >> Stress testing with DeoptimizeObjectsALot running SPECjvm2008 40x > parallel > >> for 24h > >> > >> Thanks, Richard. > >> > >> > >> More details on the changes: > >> > >> * Hide DeoptimizeObjectsALotThread from external view. > >> > >> * Changed EscapeBarrier_lock to be a _safepoint_check_never lock. > >> It used to be _safepoint_check_sometimes, which will be eliminated > sooner or > >> later. > >> I added explicit thread state changes with ThreadBlockInVM to code > paths > >> where we can wait() > >> on EscapeBarrier_lock to become safepoint safe. > >> > >> * Use handshake EscapeBarrierSuspendHandshake to suspend target > threads > >> instead of vm operation > >> VM_ThreadSuspendAllForObjDeopt. > >> > >> * Removed uses of Threads_lock. When adding a new thread we suspend > it iff > >> EA optimizations are > >> being reverted. In the previous version we were waiting on > Threads_lock > >> while EA optimizations > >> were reverted. See EscapeBarrier::thread_added(). > >> > >> * Made tests require Xmixed compilation mode. > >> > >> * Made tests agnostic regarding tiered compilation. > >> I.e. tc isn't disabled anymore, and the tests can be run with tc enabled or > >> disabled. > >> > >> * Exercising EATests.java as well with stress test options > >> DeoptimizeObjectsALot* > >> Due to the non-deterministic deoptimizations some tests need to be > skipped. > >> We do this to prevent bit-rot of the stress test code. > >> > >> * Executing EATests.java as well with graal if available. Driver for this is > >> EATestsJVMCI.java. Graal cannot pass all tests, because it does not > provide all > >> the new debug info > >> (namely not_global_escape_in_scope and arg_escape in > scopeDesc.hpp). > >> And graal does not yet support the JVMTI operations force early return > and > >> pop frame. > >> > >> * Removed tracing from new jdi tests in EATests.java. Too much trace > output > >> before the debugging > >> connection is established can cause deadlock because output buffers fill > up. > >> (See https://bugs.openjdk.java.net/browse/JDK-8173304) > >> > >> * Many copyright year changes and smaller clean-up changes of testing > code > >> (trailing white-space and > >> the like). > >> > >> > >> -----Original Message----- > >> From: David Holmes > >> Sent: Donnerstag, 19. Dezember 2019 03:12 > >> To: Reingruber, Richard ; serviceability- > >> dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; > hotspot- > >> runtime-dev at openjdk.java.net; Vladimir Kozlov > (vladimir.kozlov at oracle.com) > >> > >> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better > Performance in > >> the Presence of JVMTI Agents > >> > >> Hi Richard, > >> > >> I think my issue is with the way EliminateNestedLocks works so I'm going > >> to look into that more deeply. > >> > >> Thanks for the explanations. > >> > >> David > >> > >> On 18/12/2019 12:47 am, Reingruber, Richard wrote: > >>> Hi David, > >>> > >>> > > > Some further queries/concerns: > >>> > > > > >>> > > > src/hotspot/share/runtime/objectMonitor.cpp > >>> > > > > >>> > > > Can you please explain the changes to ObjectMonitor::wait: > >>> > > > > >>> > > > ! _recursions = save // restore the old recursion count > >>> > > > ! + jt->get_and_reset_relock_count_after_wait(); // > >>> > > > increased by the deferred relock count > >>> > > > > >>> > > > what is the "deferred relock count"? I gather it relates to > >>> > > > > >>> > > > "The code was extended to be able to deoptimize objects of a > >>> > > frame that > >>> > > > is not the top frame and to let another thread than the owning > >>> > > thread do > >>> > > > it." > >>> > > > >>> > > Yes, these relate. Currently EA based optimizations are reverted, > when a > >> compiled frame is > >>> > > replaced with corresponding interpreter frames. Part of this is > relocking > >> objects with eliminated > >>> > > locking. New with the enhancement is that we do this also just > before > >> object references are > >>> > > acquired through JVMTI. In this case we deoptimize also the > owning > >> compiled frame C and we > >>> > > register deoptimized objects as deferred updates. When control > returns > >> to C it gets deoptimized, > >>> > > we notice that objects are already deoptimized (reallocated and > >> relocked), so we don't do it again > >>> > > (relocking twice would be incorrect of course). Deferred updates > are > >> copied into the new > >>> > > interpreter frames. > >>> > > > >>> > > Problem: relocking is not possible if the target thread T is waiting > on the > >> monitor that needs to > >>> > > be relocked. This happens only with non-local objects with > >> EliminateNestedLocks. Instead relocking > >>> > > is deferred until T owns the monitor again. This is what the piece of > >> code above does. > >>> > > >>> > Sorry I need some more detail here. How can you wait() on an > object > >>> > monitor if the object allocation and/or locking was optimised away? > And > >>> > what is a "non-local object" in this context? Isn't EA restricted to > >>> > thread-confined objects? > >>> > >>> "Non-local object" is an object that escapes its thread. The issue I'm > >> addressing with the changes > >>> in ObjectMonitor::wait are almost unrelated to EA. They are caused by > >> EliminateNestedLocks, where C2 > >>> eliminates recursive locking of an already owned lock. The lock owning > object > >> exists on the heap, it > >>> is locked and you can call wait() on it. > >>> > >>> EliminateLocks is the C2 option that controls lock elimination based on > EA. > >> Both optimizations have > >>> in common that objects with eliminated locking need to be relocked > when > >> deoptimizing a frame, > >>> i.e. when replacing a compiled frame with equivalent interpreter > >>> frames. Deoptimization::relock_objects does that job for /all/ eliminated > >> locks in scope. /All/ can > >>> be a mix of eliminated nested locks and locks of not-escaping objects. > >>> > >>> New with the enhancement: I call relock_objects earlier, just before > objects > >> pontentially > >>> escape. But then later when the owning compiled frame gets > deoptimized, I > >> must not do it again: > >>> > >>> See call to EscapeBarrier::objs_are_deoptimized in deoptimization.cpp: > >>> > >>> 373 if ((jvmci_enabled || ((DoEscapeAnalysis || > EliminateNestedLocks) && > >> EliminateLocks)) > >>> 374 && !EscapeBarrier::objs_are_deoptimized(thread, > deoptee.id())) { > >>> 375 bool unused; > >>> 376 eliminate_locks(thread, chunk, realloc_failures, deoptee, > exec_mode, > >> unused); > >>> 377 } > >>> > >>> Now when calling relock_objects early it is quiet possible that I have to > relock > >> an object the > >>> target thread currently waits for. Obviously I cannot relock in this case, > >> instead I chose to > >>> introduce relock_count_after_wait to JavaThread. > >>> > >>> > Is it just that some of the locking gets optimized away e.g. > >>> > > >>> > synchronised(obj) { > >>> > synchronised(obj) { > >>> > synchronised(obj) { > >>> > obj.wait(); > >>> > } > >>> > } > >>> > } > >>> > > >>> > If this is reduced to a form as-if it were a single lock of the monitor > >>> > (due to EA) and the wait() triggers a JVM TI event which leads to the > >>> > escape of "obj" then we need to reconstruct the true lock state, and > so > >>> > when the wait() internally unblocks and reacquires the monitor it > has to > >>> > set the true recursion count to 3, not the 1 that it appeared to be > when > >>> > wait() was initially called. Is that the scenario? > >>> > >>> Kind of... except that the locking is not eliminated due to EA and there is > no > >> JVM TI event > >>> triggered by wait. > >>> > >>> Add > >>> > >>> LocalObject l1 = new LocalObject(); > >>> > >>> in front of the synchrnized blocks and assume a JVM TI agent acquires l1. > This > >> triggers the code in > >>> question. > >>> > >>> See that relocking/reallocating is transactional. If it is done then for /all/ > >> objects in scope and it is > >>> done at most once. It wouldn't be quite so easy to split this in relocking > of > >> nested/EA-based > >>> eliminated locks. > >>> > >>> > If so I find this truly awful. Anyone using wait() in a realistic form > >>> > requires a notification and so the object cannot be thread confined. > In > >>> > >>> It is not thread confined. > >>> > >>> > which case I would strongly argue that upon hitting the wait() the > deopt > >>> > should occur unconditionally and so the lock state is correct before > we > >>> > wait and so we don't need to mess with the recursion count > internally > >>> > when we reacquire the monitor. > >>> > > >>> > > > >>> > > > which I don't like the sound of at all when it comes to > ObjectMonitor > >>> > > > state. So I'd like to understand in detail exactly what is going on > here > >>> > > > and why. This is a very intrusive change that seems to badly > break > >>> > > > encapsulation and impacts future changes to ObjectMonitor > that are > >> under > >>> > > > investigation. > >>> > > > >>> > > I would not regard this as breaking encapsulation. Certainly not > badly. > >>> > > > >>> > > I've added a property relock_count_after_wait to JavaThread. The > >> property is well > >>> > > encapsulated. Future ObjectMonitor implementations have to deal > with > >> recursion too. They are free > >>> > > in choosing a way to do that as long as that property is taken into > >> account. This is hardly a > >>> > > limitation. > >>> > > >>> > I do think this badly breaks encapsulation as you have to add a > callout > >>> > from the guts of the ObjectMonitor code to reach into the thread to > get > >>> > this lock count adjustment. I understand why you have had to do > this but > >>> > I would much rather see a change to the EA optimisation strategy so > that > >>> > this is not needed. > >>> > > >>> > > Note also that the property is a straight forward extension of the > >> existing concept of deferred > >>> > > local updates. It is embedded into the structure holding them. So > not > >> even the footprint of a > >>> > > JavaThread is enlarged if no deferred updates are generated. > >>> > > >>> > [...] > >>> > > >>> > > > >>> > > I'm actually duplicating the existing external suspend mechanism, > >> because a thread can be > >>> > > suspended at most once. And hey, and don't like that either! But it > >> seems not unlikely that the > >>> > > duplicate can be removed together with the original and the new > type > >> of handshakes that will be > >>> > > used for thread suspend can be used for object deoptimization > too. See > >> today's discussion in > >>> > > JDK-8227745 [2]. > >>> > > >>> > I hope that discussion bears some fruit, at the moment it seems not > to > >>> > be possible to use handshakes here. :( > >>> > > >>> > The external suspend mechanism is a royal pain in the proverbial > that we > >>> > have to carefully live with. The idea that we're duplicating that for > >>> > use in another fringe area of functionality does not thrill me at all. > >>> > > >>> > To be clear, I understand the problem that exists and that you wish > to > >>> > solve, but for the runtime parts I balk at the complexity cost of > >>> > solving it. > >>> > >>> I know it's complex, but by far no rocket science. > >>> > >>> Also I find it hard to imagine another fix for JDK-8233915 besides > changing > >> the JVM TI specification. > >>> > >>> Thanks, Richard. > >>> > >>> -----Original Message----- > >>> From: David Holmes > >>> Sent: Dienstag, 17. Dezember 2019 08:03 > >>> To: Reingruber, Richard ; serviceability- > >> dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; > hotspot- > >> runtime-dev at openjdk.java.net; Vladimir Kozlov > (vladimir.kozlov at oracle.com) > >> > >>> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better > Performance > >> in the Presence of JVMTI Agents > >>> > >>> > >>> > >>> David > >>> > >>> On 17/12/2019 4:57 pm, David Holmes wrote: > >>>> Hi Richard, > >>>> > >>>> On 14/12/2019 5:01 am, Reingruber, Richard wrote: > >>>>> Hi David, > >>>>> > >>>>> ?? > Some further queries/concerns: > >>>>> ?? > > >>>>> ?? > src/hotspot/share/runtime/objectMonitor.cpp > >>>>> ?? > > >>>>> ?? > Can you please explain the changes to ObjectMonitor::wait: > >>>>> ?? > > >>>>> ?? > !?? _recursions = save????? // restore the old recursion count > >>>>> ?? > !???????????????? + jt->get_and_reset_relock_count_after_wait(); // > >>>>> ?? > increased by the deferred relock count > >>>>> ?? > > >>>>> ?? > what is the "deferred relock count"? I gather it relates to > >>>>> ?? > > >>>>> ?? > "The code was extended to be able to deoptimize objects of a > >>>>> frame that > >>>>> ?? > is not the top frame and to let another thread than the owning > >>>>> thread do > >>>>> ?? > it." > >>>>> > >>>>> Yes, these relate. Currently EA based optimizations are reverted, > when > >>>>> a compiled frame is replaced > >>>>> with corresponding interpreter frames. Part of this is relocking > >>>>> objects with eliminated > >>>>> locking. New with the enhancement is that we do this also just before > >>>>> object references are acquired > >>>>> through JVMTI. In this case we deoptimize also the owning compiled > >>>>> frame C and we register > >>>>> deoptimized objects as deferred updates. When control returns to C > it > >>>>> gets deoptimized, we notice > >>>>> that objects are already deoptimized (reallocated and relocked), so > we > >>>>> don't do it again (relocking > >>>>> twice would be incorrect of course). Deferred updates are copied into > >>>>> the new interpreter frames. > >>>>> > >>>>> Problem: relocking is not possible if the target thread T is waiting > >>>>> on the monitor that needs to be > >>>>> relocked. This happens only with non-local objects with > >>>>> EliminateNestedLocks. Instead relocking is > >>>>> deferred until T owns the monitor again. This is what the piece of > >>>>> code above does. > >>>> > >>>> Sorry I need some more detail here. How can you wait() on an object > >>>> monitor if the object allocation and/or locking was optimised away? > And > >>>> what is a "non-local object" in this context? Isn't EA restricted to > >>>> thread-confined objects? > >>>> > >>>> Is it just that some of the locking gets optimized away e.g. > >>>> > >>>> synchronised(obj) { > >>>> ? synchronised(obj) { > >>>> ??? synchronised(obj) { > >>>> ????? obj.wait(); > >>>> ??? } > >>>> ? } > >>>> } > >>>> > >>>> If this is reduced to a form as-if it were a single lock of the monitor > >>>> (due to EA) and the wait() triggers a JVM TI event which leads to the > >>>> escape of "obj" then we need to reconstruct the true lock state, and so > >>>> when the wait() internally unblocks and reacquires the monitor it has to > >>>> set the true recursion count to 3, not the 1 that it appeared to be when > >>>> wait() was initially called. Is that the scenario? > >>>> > >>>> If so I find this truly awful. Anyone using wait() in a realistic form > >>>> requires a notification and so the object cannot be thread confined. In > >>>> which case I would strongly argue that upon hitting the wait() the > deopt > >>>> should occur unconditionally and so the lock state is correct before we > >>>> wait and so we don't need to mess with the recursion count internally > >>>> when we reacquire the monitor. > >>>> > >>>>> > >>>>> ?? > which I don't like the sound of at all when it comes to > >>>>> ObjectMonitor > >>>>> ?? > state. So I'd like to understand in detail exactly what is going > >>>>> on here > >>>>> ?? > and why.? This is a very intrusive change that seems to badly > break > >>>>> ?? > encapsulation and impacts future changes to ObjectMonitor that > >>>>> are under > >>>>> ?? > investigation. > >>>>> > >>>>> I would not regard this as breaking encapsulation. Certainly not badly. > >>>>> > >>>>> I've added a property relock_count_after_wait to JavaThread. The > >>>>> property is well > >>>>> encapsulated. Future ObjectMonitor implementations have to deal > with > >>>>> recursion too. They are free in > >>>>> choosing a way to do that as long as that property is taken into > >>>>> account. This is hardly a > >>>>> limitation. > >>>> > >>>> I do think this badly breaks encapsulation as you have to add a callout > >>>> from the guts of the ObjectMonitor code to reach into the thread to > get > >>>> this lock count adjustment. I understand why you have had to do this > but > >>>> I would much rather see a change to the EA optimisation strategy so > that > >>>> this is not needed. > >>>> > >>>>> Note also that the property is a straight forward extension of the > >>>>> existing concept of deferred > >>>>> local updates. It is embedded into the structure holding them. So not > >>>>> even the footprint of a > >>>>> JavaThread is enlarged if no deferred updates are generated. > >>>>> > >>>>> ?? > --- > >>>>> ?? > > >>>>> ?? > src/hotspot/share/runtime/thread.cpp > >>>>> ?? > > >>>>> ?? > Can you please explain why > >>>>> JavaThread::wait_for_object_deoptimization > >>>>> ?? > has to be handcrafted in this way rather than using proper > >>>>> transitions. > >>>>> ?? > > >>>>> > >>>>> I wrote wait_for_object_deoptimization taking > >>>>> JavaThread::java_suspend_self_with_safepoint_check > >>>>> as template. So in short: for the same reasons :) > >>>>> > >>>>> Threads reach both methods as part of thread state transitions, > >>>>> therefore special handling is > >>>>> required to change thread state on top of ongoing transitions. > >>>>> > >>>>> ?? > We got rid of "deopt suspend" some time ago and it is disturbing > >>>>> to see > >>>>> ?? > it being added back (effectively). This seems like it may be > >>>>> something > >>>>> ?? > that handshakes could be used for. > >>>>> > >>>>> Deopt suspend used to be something rather different with a similar > >>>>> name[1]. It is not being added back. > >>>> > >>>> I stand corrected. Despite comments in the code to the contrary > >>>> deopt_suspend didn't actually cause a self-suspend. I was doing a lot of > >>>> cleanup in this area 13 years ago :) > >>>> > >>>>> > >>>>> I'm actually duplicating the existing external suspend mechanism, > >>>>> because a thread can be suspended > >>>>> at most once. And hey, and don't like that either! But it seems not > >>>>> unlikely that the duplicate can > >>>>> be removed together with the original and the new type of > handshakes > >>>>> that will be used for > >>>>> thread suspend can be used for object deoptimization too. See > today's > >>>>> discussion in JDK-8227745 [2]. > >>>> > >>>> I hope that discussion bears some fruit, at the moment it seems not to > >>>> be possible to use handshakes here. :( > >>>> > >>>> The external suspend mechanism is a royal pain in the proverbial that > we > >>>> have to carefully live with. The idea that we're duplicating that for > >>>> use in another fringe area of functionality does not thrill me at all. > >>>> > >>>> To be clear, I understand the problem that exists and that you wish to > >>>> solve, but for the runtime parts I balk at the complexity cost of > >>>> solving it. > >>>> > >>>> Thanks, > >>>> David > >>>> ----- > >>>> > >>>>> Thanks, Richard. > >>>>> > >>>>> [1] Deopt suspend was something like an async. handshake for > >>>>> architectures with register windows, > >>>>> ???? where patching the return pc for deoptimization of a compiled > >>>>> frame was racy if the owner thread > >>>>> ???? was in native code. Instead a "deopt" suspend flag was set on > >>>>> which the thread patched its own > >>>>> ???? frame upon return from native. So no thread was suspended. It > got > >>>>> its name only from the name of > >>>>> ???? the flags. > >>>>> > >>>>> [2] Discussion about using handshakes to sync. with the target thread: > >>>>> > >>>>> https://bugs.openjdk.java.net/browse/JDK- > >> > 8227745?focusedCommentId=14306727&page=com.atlassian.jira.plugin.syst > e > >> m.issuetabpanels:comment-tabpanel#comment-14306727 > >>>>> > >>>>> > >>>>> -----Original Message----- > >>>>> From: David Holmes > >>>>> Sent: Freitag, 13. Dezember 2019 00:56 > >>>>> To: Reingruber, Richard ; > >>>>> serviceability-dev at openjdk.java.net; > >>>>> hotspot-compiler-dev at openjdk.java.net; > >>>>> hotspot-runtime-dev at openjdk.java.net > >>>>> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better > >>>>> Performance in the Presence of JVMTI Agents > >>>>> > >>>>> Hi Richard, > >>>>> > >>>>> Some further queries/concerns: > >>>>> > >>>>> src/hotspot/share/runtime/objectMonitor.cpp > >>>>> > >>>>> Can you please explain the changes to ObjectMonitor::wait: > >>>>> > >>>>> !?? _recursions = save????? // restore the old recursion count > >>>>> !???????????????? + jt->get_and_reset_relock_count_after_wait(); // > >>>>> increased by the deferred relock count > >>>>> > >>>>> what is the "deferred relock count"? I gather it relates to > >>>>> > >>>>> "The code was extended to be able to deoptimize objects of a frame > that > >>>>> is not the top frame and to let another thread than the owning thread > do > >>>>> it." > >>>>> > >>>>> which I don't like the sound of at all when it comes to ObjectMonitor > >>>>> state. So I'd like to understand in detail exactly what is going on here > >>>>> and why.? This is a very intrusive change that seems to badly break > >>>>> encapsulation and impacts future changes to ObjectMonitor that are > under > >>>>> investigation. > >>>>> > >>>>> --- > >>>>> > >>>>> src/hotspot/share/runtime/thread.cpp > >>>>> > >>>>> Can you please explain why > JavaThread::wait_for_object_deoptimization > >>>>> has to be handcrafted in this way rather than using proper transitions. > >>>>> > >>>>> We got rid of "deopt suspend" some time ago and it is disturbing to > see > >>>>> it being added back (effectively). This seems like it may be something > >>>>> that handshakes could be used for. > >>>>> > >>>>> Thanks, > >>>>> David > >>>>> ----- > >>>>> > >>>>> On 12/12/2019 7:02 am, David Holmes wrote: > >>>>>> On 12/12/2019 1:07 am, Reingruber, Richard wrote: > >>>>>>> Hi David, > >>>>>>> > >>>>>>> ??? > Most of the details here are in areas I can comment on in > detail, > >>>>>>> but I > >>>>>>> ??? > did take an initial general look at things. > >>>>>>> > >>>>>>> Thanks for taking the time! > >>>>>> > >>>>>> Apologies the above should read: > >>>>>> > >>>>>> "Most of the details here are in areas I *can't* comment on in detail > >>>>>> ..." > >>>>>> > >>>>>> David > >>>>>> > >>>>>>> ??? > The only thing that jumped out at me is that I think the > >>>>>>> ??? > DeoptimizeObjectsALotThread should be a hidden thread. > >>>>>>> ??? > > >>>>>>> ??? > +? bool is_hidden_from_external_view() const { return true; } > >>>>>>> > >>>>>>> Yes, it should. Will add the method like above. > >>>>>>> > >>>>>>> ??? > Also I don't see any testing of the > DeoptimizeObjectsALotThread. > >>>>>>> Without > >>>>>>> ??? > active testing this will just bit-rot. > >>>>>>> > >>>>>>> DeoptimizeObjectsALot is meant for stress testing with a larger > >>>>>>> workload. I will add a minimal test > >>>>>>> to keep it fresh. > >>>>>>> > >>>>>>> ??? > Also on the tests I don't understand your @requires clause: > >>>>>>> ??? > > >>>>>>> ??? >?? @requires ((vm.compMode != "Xcomp") & > vm.compiler2.enabled > >> & > >>>>>>> ??? > (vm.opt.TieredCompilation != true)) > >>>>>>> ??? > > >>>>>>> ??? > This seems to require that TieredCompilation is disabled, but > >>>>>>> tiered is > >>>>>>> ??? > our normal mode of operation. ?? > >>>>>>> ??? > > >>>>>>> > >>>>>>> I removed the clause. I guess I wanted to target the tests towards > the > >>>>>>> code they are supposed to > >>>>>>> test, and it's easier to analyze failures w/o tiered compilation and > >>>>>>> with just one compiler thread. > >>>>>>> > >>>>>>> Additionally I will make use of > >>>>>>> compiler.whitebox.CompilerWhiteBoxTest.THRESHOLD in the tests. > >>>>>>> > >>>>>>> Thanks, > >>>>>>> Richard. > >>>>>>> > >>>>>>> -----Original Message----- > >>>>>>> From: David Holmes > >>>>>>> Sent: Mittwoch, 11. Dezember 2019 08:03 > >>>>>>> To: Reingruber, Richard ; > >>>>>>> serviceability-dev at openjdk.java.net; > >>>>>>> hotspot-compiler-dev at openjdk.java.net; > >>>>>>> hotspot-runtime-dev at openjdk.java.net > >>>>>>> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better > >>>>>>> Performance in the Presence of JVMTI Agents > >>>>>>> > >>>>>>> Hi Richard, > >>>>>>> > >>>>>>> On 11/12/2019 7:45 am, Reingruber, Richard wrote: > >>>>>>>> Hi, > >>>>>>>> > >>>>>>>> I would like to get reviews please for > >>>>>>>> > >>>>>>>> > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.3/ > >>>>>>>> > >>>>>>>> Corresponding RFE: > >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8227745 > >>>>>>>> > >>>>>>>> Fixes also https://bugs.openjdk.java.net/browse/JDK-8233915 > >>>>>>>> And potentially https://bugs.openjdk.java.net/browse/JDK- > 8214584 [1] > >>>>>>>> > >>>>>>>> Vladimir Kozlov kindly put webrev.3 through tier1-8 testing > without > >>>>>>>> issues (thanks!). In addition the > >>>>>>>> change is being tested at SAP since I posted the first RFR some > >>>>>>>> months ago. > >>>>>>>> > >>>>>>>> The intention of this enhancement is to benefit performance wise > from > >>>>>>>> escape analysis even if JVMTI > >>>>>>>> agents request capabilities that allow them to access local variable > >>>>>>>> values. E.g. if you start-up > >>>>>>>> with -agentlib:jdwp=transport=dt_socket,server=y,suspend=n, > then > >>>>>>>> escape analysis is disabled right > >>>>>>>> from the beginning, well before a debugger attaches -- if ever one > >>>>>>>> should do so. With the > >>>>>>>> enhancement, escape analysis will remain enabled until and after > a > >>>>>>>> debugger attaches. EA based > >>>>>>>> optimizations are reverted just before an agent acquires the > >>>>>>>> reference to an object. In the JBS item > >>>>>>>> you'll find more details. > >>>>>>> > >>>>>>> Most of the details here are in areas I can comment on in detail, but > I > >>>>>>> did take an initial general look at things. > >>>>>>> > >>>>>>> The only thing that jumped out at me is that I think the > >>>>>>> DeoptimizeObjectsALotThread should be a hidden thread. > >>>>>>> > >>>>>>> +? bool is_hidden_from_external_view() const { return true; } > >>>>>>> > >>>>>>> Also I don't see any testing of the DeoptimizeObjectsALotThread. > >>>>>>> Without > >>>>>>> active testing this will just bit-rot. > >>>>>>> > >>>>>>> Also on the tests I don't understand your @requires clause: > >>>>>>> > >>>>>>> ??? @requires ((vm.compMode != "Xcomp") & > vm.compiler2.enabled & > >>>>>>> (vm.opt.TieredCompilation != true)) > >>>>>>> > >>>>>>> This seems to require that TieredCompilation is disabled, but tiered > is > >>>>>>> our normal mode of operation. ?? > >>>>>>> > >>>>>>> Thanks, > >>>>>>> David > >>>>>>> > >>>>>>>> Thanks, > >>>>>>>> Richard. > >>>>>>>> > >>>>>>>> [1] Experimental fix for JDK-8214584 based on JDK-8227745 > >>>>>>>> > >> > http://cr.openjdk.java.net/~rrich/webrevs/2019/8214584/experiment_v1.pa > tc > >> h > >>>>>>>> > >>>>>>>> > >>>>>>>> From yudi.zheng at oracle.com Mon Mar 30 14:15:19 2020 From: yudi.zheng at oracle.com (Yudi Zheng) Date: Mon, 30 Mar 2020 16:15:19 +0200 Subject: RFR: 8241234: Unify monitor enter/exit runtime entries In-Reply-To: <904faf68-4fff-f1b8-2fb8-48d65f282fa2@oracle.com> References: <222D2846-F6AE-4D5B-B41F-F976D90E329C@oracle.com> <91eeada8-e05f-bc73-b029-94e169216a56@oracle.com> <534b8cf7-cd8c-565b-5163-09a216d4f94e@oracle.com> <904faf68-4fff-f1b8-2fb8-48d65f282fa2@oracle.com> Message-ID: Hi Daniel, Thanks for the review! I have uploaded a new version with your comments addressed: http://cr.openjdk.java.net/~yzheng/8241234/webrev.04/ > src/hotspot/share/runtime/sharedRuntime.hpp > Please don't forget to update the copyright year before you push. Fixed. > src/hotspot/share/runtime/sharedRuntime.cpp > L2104: ObjectSynchronizer::exit(obj, lock, THREAD); > The use of 'THREAD' here and 'TRAPS' in the function itself > standout more now, but that's something for me to cleanup. Also, I noticed that C2 was using CHECK > ObjectSynchronizer::enter(h_obj, lock, CHECK); While C1 and JVMCI were using THREAD: > ObjectSynchronizer::enter(h_obj, lock->lock(), THREAD); I have no idea when to use what, and hope unifying to the C2 entries would help. Let me know if there is something I should address in this patch. Otherwise, I would rather leave it to the expert, i.e., you ;) > src/hotspot/share/c1/c1_Runtime1.cpp > old L718: assert(thread == JavaThread::current(), "threads must correspond"); > Removed in favor of the assert in SharedRuntime::monitor_enter_helper(). > Okay that makes sense. > > old L721: EXCEPTION_MARK; > Removed in favor of the same in SharedRuntime::monitor_enter_helper(). > Okay that makes sense. > > src/hotspot/share/jvmci/jvmciRuntime.cpp > old L403: assert(thread == JavaThread::current(), "threads must correspond"); > old L406: EXCEPTION_MARK; > Same as for c1_Runtime1.cpp I assume I don?t need to do anything regarding the comments above. > > L390: TRACE_jvmci_3("%s: entered locking slow case with obj="... > L394: TRACE_jvmci_3("%s: exiting locking slow with obj=" > L417: TRACE_jvmci_3("%s: exited locking slow case with obj=" > But this is no longer the "slow" case so I'm a bit confused. > > Update: I see there's a comment about the tracing being removed. > I have no opinion on that since it is JVM/CI code, but the word > "slow" needs to be adjusted if you keep it. I removed all the tracing code. Many thanks, Yudi From daniel.daugherty at oracle.com Mon Mar 30 14:20:40 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 30 Mar 2020 10:20:40 -0400 Subject: RFR: 8241234: Unify monitor enter/exit runtime entries In-Reply-To: References: <222D2846-F6AE-4D5B-B41F-F976D90E329C@oracle.com> <91eeada8-e05f-bc73-b029-94e169216a56@oracle.com> <534b8cf7-cd8c-565b-5163-09a216d4f94e@oracle.com> <904faf68-4fff-f1b8-2fb8-48d65f282fa2@oracle.com> Message-ID: On 3/30/20 10:15 AM, Yudi Zheng wrote: > Hi Daniel, > > Thanks for the review! I have uploaded a new version with your comments addressed: > http://cr.openjdk.java.net/~yzheng/8241234/webrev.04/ > >> src/hotspot/share/runtime/sharedRuntime.hpp >> Please don't forget to update the copyright year before you push. > Fixed. > >> src/hotspot/share/runtime/sharedRuntime.cpp >> L2104: ObjectSynchronizer::exit(obj, lock, THREAD); >> The use of 'THREAD' here and 'TRAPS' in the function itself >> standout more now, but that's something for me to cleanup. > Also, I noticed that C2 was using CHECK >> ObjectSynchronizer::enter(h_obj, lock, CHECK); > While C1 and JVMCI were using THREAD: >> ObjectSynchronizer::enter(h_obj, lock->lock(), THREAD); > I have no idea when to use what, and hope unifying to the C2 entries would help. > Let me know if there is something I should address in this patch. Otherwise, I would > rather leave it to the expert, i.e., you ;) Yes, please leave it for me to clean up. >> src/hotspot/share/c1/c1_Runtime1.cpp >> old L718: assert(thread == JavaThread::current(), "threads must correspond"); >> Removed in favor of the assert in SharedRuntime::monitor_enter_helper(). >> Okay that makes sense. >> >> old L721: EXCEPTION_MARK; >> Removed in favor of the same in SharedRuntime::monitor_enter_helper(). >> Okay that makes sense. >> >> src/hotspot/share/jvmci/jvmciRuntime.cpp >> old L403: assert(thread == JavaThread::current(), "threads must correspond"); >> old L406: EXCEPTION_MARK; >> Same as for c1_Runtime1.cpp > I assume I don?t need to do anything regarding the comments above. Correct. Just observations on the old code. >> L390: TRACE_jvmci_3("%s: entered locking slow case with obj="... >> L394: TRACE_jvmci_3("%s: exiting locking slow with obj=" >> L417: TRACE_jvmci_3("%s: exited locking slow case with obj=" >> But this is no longer the "slow" case so I'm a bit confused. >> >> Update: I see there's a comment about the tracing being removed. >> I have no opinion on that since it is JVM/CI code, but the word >> "slow" needs to be adjusted if you keep it. > I removed all the tracing code. Thanks for cleaning that up. Dan > > Many thanks, > Yudi From coleen.phillimore at oracle.com Mon Mar 30 14:27:00 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 30 Mar 2020 10:27:00 -0400 Subject: RFR: 8241234: Unify monitor enter/exit runtime entries In-Reply-To: References: <222D2846-F6AE-4D5B-B41F-F976D90E329C@oracle.com> <91eeada8-e05f-bc73-b029-94e169216a56@oracle.com> <534b8cf7-cd8c-565b-5163-09a216d4f94e@oracle.com> <904faf68-4fff-f1b8-2fb8-48d65f282fa2@oracle.com> Message-ID: <3fc16b46-7fdb-9a02-85aa-b517640160d3@oracle.com> On 3/30/20 10:20 AM, Daniel D. Daugherty wrote: > On 3/30/20 10:15 AM, Yudi Zheng wrote: >> Hi Daniel, >> >> Thanks for the review! I have uploaded a new version with your >> comments addressed: >> http://cr.openjdk.java.net/~yzheng/8241234/webrev.04/ >> >>> src/hotspot/share/runtime/sharedRuntime.hpp >>> ???? Please don't forget to update the copyright year before you push. >> Fixed. >> >>> src/hotspot/share/runtime/sharedRuntime.cpp >>> ???? L2104:?? ObjectSynchronizer::exit(obj, lock, THREAD); >>> ???????? The use of 'THREAD' here and 'TRAPS' in the function itself >>> ???????? standout more now, but that's something for me to cleanup. >> Also, I noticed that C2 was using CHECK >>> ??? ObjectSynchronizer::enter(h_obj, lock, CHECK); >> While C1 and JVMCI were using THREAD: >>> ??? ObjectSynchronizer::enter(h_obj, lock->lock(), THREAD); >> I have no idea when to use what, and hope unifying to the C2 entries >> would help. >> Let me know if there is something I should address in this patch. >> Otherwise, I would >> rather leave it to the expert, i.e., you ;) > > Yes, please leave it for me to clean up. It appears that both of these places (monitor_enter_helper and monitor_exit_helper) should be using CATCH. Coleen > > >>> src/hotspot/share/c1/c1_Runtime1.cpp >>> ???? old L718:?? assert(thread == JavaThread::current(), "threads >>> must correspond"); >>> ???????? Removed in favor of the assert in >>> SharedRuntime::monitor_enter_helper(). >>> ???????? Okay that makes sense. >>> >>> ???? old L721:?? EXCEPTION_MARK; >>> ???????? Removed in favor of the same in >>> SharedRuntime::monitor_enter_helper(). >>> ???????? Okay that makes sense. >>> >>> src/hotspot/share/jvmci/jvmciRuntime.cpp >>> ???? old L403:?? assert(thread == JavaThread::current(), "threads >>> must correspond"); >>> ???? old L406:?? EXCEPTION_MARK; >>> ???????? Same as for c1_Runtime1.cpp >> I assume I don?t need to do anything regarding the comments above. > > Correct. Just observations on the old code. > > >>> ???? L390:???? TRACE_jvmci_3("%s: entered locking slow case with >>> obj="... >>> ???? L394:?? TRACE_jvmci_3("%s: exiting locking slow with obj=" >>> ???? L417:???? TRACE_jvmci_3("%s: exited locking slow case with obj=" >>> ???????? But this is no longer the "slow" case so I'm a bit confused. >>> >>> ???????? Update: I see there's a comment about the tracing being >>> removed. >>> ???????? I have no opinion on that since it is JVM/CI code, but the >>> word >>> ???????? "slow" needs to be adjusted if you keep it. >> I removed all the tracing code. > > Thanks for cleaning that up. > > Dan > >> >> Many thanks, >> Yudi > From daniel.daugherty at oracle.com Mon Mar 30 15:20:48 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 30 Mar 2020 11:20:48 -0400 Subject: RFR: 8241234: Unify monitor enter/exit runtime entries In-Reply-To: <3fc16b46-7fdb-9a02-85aa-b517640160d3@oracle.com> References: <222D2846-F6AE-4D5B-B41F-F976D90E329C@oracle.com> <91eeada8-e05f-bc73-b029-94e169216a56@oracle.com> <534b8cf7-cd8c-565b-5163-09a216d4f94e@oracle.com> <904faf68-4fff-f1b8-2fb8-48d65f282fa2@oracle.com> <3fc16b46-7fdb-9a02-85aa-b517640160d3@oracle.com> Message-ID: <748209ee-5e5f-7a3d-3713-0ed835e710aa@oracle.com> On 3/30/20 10:27 AM, coleen.phillimore at oracle.com wrote: > > > On 3/30/20 10:20 AM, Daniel D. Daugherty wrote: >> On 3/30/20 10:15 AM, Yudi Zheng wrote: >>> Hi Daniel, >>> >>> Thanks for the review! I have uploaded a new version with your >>> comments addressed: >>> http://cr.openjdk.java.net/~yzheng/8241234/webrev.04/ >>> >>>> src/hotspot/share/runtime/sharedRuntime.hpp >>>> ???? Please don't forget to update the copyright year before you push. >>> Fixed. >>> >>>> src/hotspot/share/runtime/sharedRuntime.cpp >>>> ???? L2104:?? ObjectSynchronizer::exit(obj, lock, THREAD); >>>> ???????? The use of 'THREAD' here and 'TRAPS' in the function itself >>>> ???????? standout more now, but that's something for me to cleanup. >>> Also, I noticed that C2 was using CHECK >>>> ??? ObjectSynchronizer::enter(h_obj, lock, CHECK); >>> While C1 and JVMCI were using THREAD: >>>> ??? ObjectSynchronizer::enter(h_obj, lock->lock(), THREAD); >>> I have no idea when to use what, and hope unifying to the C2 entries >>> would help. >>> Let me know if there is something I should address in this patch. >>> Otherwise, I would >>> rather leave it to the expert, i.e., you ;) >> >> Yes, please leave it for me to clean up. > > It appears that both of these places (monitor_enter_helper and > monitor_exit_helper) should be using CATCH. Here's CATCH: src/hotspot/share/utilities/exceptions.hpp: // The CATCH macro checks that no exception has been thrown by a function; it is used at // call sites about which is statically known that the callee cannot throw an exception // even though it is declared with TRAPS. #define CATCH????????????????????????????? \ ? THREAD); if (HAS_PENDING_EXCEPTION) {??? \ ??? oop ex = PENDING_EXCEPTION;??????????? \ ??? CLEAR_PENDING_EXCEPTION;?????????????? \ ??? ex->print();?????????????????????????? \ ??? ShouldNotReachHere();????????????????? \ ? } (void)(0 I don't think you want either monitor_enter_helper() or monitor_exit_helper() crashing the VM via a ShouldNotReachHere() if someone happens to throw an exception at a thread during a 'monitorenter' or 'monitorexit' bytecode. What am I missing here? Dan > > Coleen >> >> >>>> src/hotspot/share/c1/c1_Runtime1.cpp >>>> ???? old L718:?? assert(thread == JavaThread::current(), "threads >>>> must correspond"); >>>> ???????? Removed in favor of the assert in >>>> SharedRuntime::monitor_enter_helper(). >>>> ???????? Okay that makes sense. >>>> >>>> ???? old L721:?? EXCEPTION_MARK; >>>> ???????? Removed in favor of the same in >>>> SharedRuntime::monitor_enter_helper(). >>>> ???????? Okay that makes sense. >>>> >>>> src/hotspot/share/jvmci/jvmciRuntime.cpp >>>> ???? old L403:?? assert(thread == JavaThread::current(), "threads >>>> must correspond"); >>>> ???? old L406:?? EXCEPTION_MARK; >>>> ???????? Same as for c1_Runtime1.cpp >>> I assume I don?t need to do anything regarding the comments above. >> >> Correct. Just observations on the old code. >> >> >>>> ???? L390:???? TRACE_jvmci_3("%s: entered locking slow case with >>>> obj="... >>>> ???? L394:?? TRACE_jvmci_3("%s: exiting locking slow with obj=" >>>> ???? L417:???? TRACE_jvmci_3("%s: exited locking slow case with obj=" >>>> ???????? But this is no longer the "slow" case so I'm a bit confused. >>>> >>>> ???????? Update: I see there's a comment about the tracing being >>>> removed. >>>> ???????? I have no opinion on that since it is JVM/CI code, but the >>>> word >>>> ???????? "slow" needs to be adjusted if you keep it. >>> I removed all the tracing code. >> >> Thanks for cleaning that up. >> >> Dan >> >>> >>> Many thanks, >>> Yudi >> > From coleen.phillimore at oracle.com Mon Mar 30 15:41:45 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 30 Mar 2020 11:41:45 -0400 Subject: RFR: 8241234: Unify monitor enter/exit runtime entries In-Reply-To: <748209ee-5e5f-7a3d-3713-0ed835e710aa@oracle.com> References: <222D2846-F6AE-4D5B-B41F-F976D90E329C@oracle.com> <91eeada8-e05f-bc73-b029-94e169216a56@oracle.com> <534b8cf7-cd8c-565b-5163-09a216d4f94e@oracle.com> <904faf68-4fff-f1b8-2fb8-48d65f282fa2@oracle.com> <3fc16b46-7fdb-9a02-85aa-b517640160d3@oracle.com> <748209ee-5e5f-7a3d-3713-0ed835e710aa@oracle.com> Message-ID: <9c330297-d2a4-df6e-4f32-3c26d72d0526@oracle.com> On 3/30/20 11:20 AM, Daniel D. Daugherty wrote: > On 3/30/20 10:27 AM, coleen.phillimore at oracle.com wrote: >> >> >> On 3/30/20 10:20 AM, Daniel D. Daugherty wrote: >>> On 3/30/20 10:15 AM, Yudi Zheng wrote: >>>> Hi Daniel, >>>> >>>> Thanks for the review! I have uploaded a new version with your >>>> comments addressed: >>>> http://cr.openjdk.java.net/~yzheng/8241234/webrev.04/ >>>> >>>>> src/hotspot/share/runtime/sharedRuntime.hpp >>>>> ???? Please don't forget to update the copyright year before you >>>>> push. >>>> Fixed. >>>> >>>>> src/hotspot/share/runtime/sharedRuntime.cpp >>>>> ???? L2104:?? ObjectSynchronizer::exit(obj, lock, THREAD); >>>>> ???????? The use of 'THREAD' here and 'TRAPS' in the function itself >>>>> ???????? standout more now, but that's something for me to cleanup. >>>> Also, I noticed that C2 was using CHECK >>>>> ??? ObjectSynchronizer::enter(h_obj, lock, CHECK); >>>> While C1 and JVMCI were using THREAD: >>>>> ??? ObjectSynchronizer::enter(h_obj, lock->lock(), THREAD); >>>> I have no idea when to use what, and hope unifying to the C2 >>>> entries would help. >>>> Let me know if there is something I should address in this patch. >>>> Otherwise, I would >>>> rather leave it to the expert, i.e., you ;) >>> >>> Yes, please leave it for me to clean up. >> >> It appears that both of these places (monitor_enter_helper and >> monitor_exit_helper) should be using CATCH. > > Here's CATCH: > > src/hotspot/share/utilities/exceptions.hpp: > > // The CATCH macro checks that no exception has been thrown by a > function; it is used at > // call sites about which is statically known that the callee cannot > throw an exception > // even though it is declared with TRAPS. > > #define CATCH????????????????????????????? \ > ? THREAD); if (HAS_PENDING_EXCEPTION) {??? \ > ??? oop ex = PENDING_EXCEPTION;??????????? \ > ??? CLEAR_PENDING_EXCEPTION;?????????????? \ > ??? ex->print();?????????????????????????? \ > ??? ShouldNotReachHere();????????????????? \ > ? } (void)(0 > > > I don't think you want either monitor_enter_helper() or > monitor_exit_helper() crashing the VM via a ShouldNotReachHere() > if someone happens to throw an exception at a thread during > a 'monitorenter' or 'monitorexit' bytecode. Of course, it's more complicated.? I was looking at this: ObjectSynchronizer::enter(h_obj, lock, CHECK); assert(!HAS_PENDING_EXCEPTION, "Should have no exception here"); JRT_BLOCK_END Which essentially wants to, but fails to, assert that there's no exceptions possible (it is enclosed with JRT_BLOCK_NO_ASYNC). I can't see whether ObjectSynchronizer::enter() has code to exclude async exceptions from being installed (possibly biased locking revoking at a handshake?). And this: +void SharedRuntime::monitor_exit_helper(oopDesc* obj, BasicLock* lock, JavaThread* thread) { + assert(JavaThread::current() == thread, "invariant"); // Exit must be non-blocking, and therefore no exceptions can be thrown. EXCEPTION_MARK; ObjectSynchronizer::exit(obj, lock, THREAD); - } +} Which has ExceptionMark whose destructor has: fatal("ExceptionMark destructor expects no pending exceptions"); Which appears to be enabled in production. So either way, you want the functionality of CHECK, just not have it assert in production (which was a surprise to me). Actually, I agree with you that you want to clean this up. Coleen > > What am I missing here? > > Dan > > >> >> Coleen >>> >>> >>>>> src/hotspot/share/c1/c1_Runtime1.cpp >>>>> ???? old L718:?? assert(thread == JavaThread::current(), "threads >>>>> must correspond"); >>>>> ???????? Removed in favor of the assert in >>>>> SharedRuntime::monitor_enter_helper(). >>>>> ???????? Okay that makes sense. >>>>> >>>>> ???? old L721:?? EXCEPTION_MARK; >>>>> ???????? Removed in favor of the same in >>>>> SharedRuntime::monitor_enter_helper(). >>>>> ???????? Okay that makes sense. >>>>> >>>>> src/hotspot/share/jvmci/jvmciRuntime.cpp >>>>> ???? old L403:?? assert(thread == JavaThread::current(), "threads >>>>> must correspond"); >>>>> ???? old L406:?? EXCEPTION_MARK; >>>>> ???????? Same as for c1_Runtime1.cpp >>>> I assume I don?t need to do anything regarding the comments above. >>> >>> Correct. Just observations on the old code. >>> >>> >>>>> ???? L390:???? TRACE_jvmci_3("%s: entered locking slow case with >>>>> obj="... >>>>> ???? L394:?? TRACE_jvmci_3("%s: exiting locking slow with obj=" >>>>> ???? L417:???? TRACE_jvmci_3("%s: exited locking slow case with obj=" >>>>> ???????? But this is no longer the "slow" case so I'm a bit confused. >>>>> >>>>> ???????? Update: I see there's a comment about the tracing being >>>>> removed. >>>>> ???????? I have no opinion on that since it is JVM/CI code, but >>>>> the word >>>>> ???????? "slow" needs to be adjusted if you keep it. >>>> I removed all the tracing code. >>> >>> Thanks for cleaning that up. >>> >>> Dan >>> >>>> >>>> Many thanks, >>>> Yudi >>> >> > From claes.redestad at oracle.com Mon Mar 30 16:27:24 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 30 Mar 2020 18:27:24 +0200 Subject: RFR (T): 8241852: Cleanup error message generation in LinkResolver::resolve_field Message-ID: <4035d995-f33f-8ecb-4801-1b472c003f16@oracle.com> Hi, a couple of places in LinkResolver::resolve_field where we allocate ResourceMark and stringStream in a wider scope than necessary. Narrowing down the scope where they are declared in a similar way as done consistently in all other places in LinkResolver is a small and trivial optimization: http://cr.openjdk.java.net/~redestad/8241852/open.00/ Thanks! /Claes From daniel.daugherty at oracle.com Mon Mar 30 16:34:18 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 30 Mar 2020 12:34:18 -0400 Subject: RFR (T): 8241852: Cleanup error message generation in LinkResolver::resolve_field In-Reply-To: <4035d995-f33f-8ecb-4801-1b472c003f16@oracle.com> References: <4035d995-f33f-8ecb-4801-1b472c003f16@oracle.com> Message-ID: <49dfc4d7-1c39-3218-0e06-4e8e7b6a75a1@oracle.com> On 3/30/20 12:27 PM, Claes Redestad wrote: > Hi, > > a couple of places in LinkResolver::resolve_field where we allocate > ResourceMark and stringStream in a wider scope than necessary. Narrowing > down the scope where they are declared in a similar way as done > consistently in all other places in LinkResolver is a small and trivial > optimization: > > http://cr.openjdk.java.net/~redestad/8241852/open.00/ src/hotspot/share/interpreter/linkResolver.cpp ??? Please update the copyright year before you push. Thumbs up! I concur that this is a trivial fix. Dan > > Thanks! > > /Claes > > > From claes.redestad at oracle.com Mon Mar 30 16:45:13 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 30 Mar 2020 18:45:13 +0200 Subject: RFR (T): 8241852: Cleanup error message generation in LinkResolver::resolve_field In-Reply-To: <49dfc4d7-1c39-3218-0e06-4e8e7b6a75a1@oracle.com> References: <4035d995-f33f-8ecb-4801-1b472c003f16@oracle.com> <49dfc4d7-1c39-3218-0e06-4e8e7b6a75a1@oracle.com> Message-ID: <7a03b072-c3fd-b901-e351-5e2981f6aa4d@oracle.com> On 2020-03-30 18:34, Daniel D. Daugherty wrote: > On 3/30/20 12:27 PM, Claes Redestad wrote: >> Hi, >> >> a couple of places in LinkResolver::resolve_field where we allocate >> ResourceMark and stringStream in a wider scope than necessary. Narrowing >> down the scope where they are declared in a similar way as done >> consistently in all other places in LinkResolver is a small and trivial >> optimization: >> >> http://cr.openjdk.java.net/~redestad/8241852/open.00/ > > src/hotspot/share/interpreter/linkResolver.cpp > ??? Please update the copyright year before you push. > Will do. > > Thumbs up! I concur that this is a trivial fix. Thanks! /Claes From claes.redestad at oracle.com Mon Mar 30 23:02:30 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 31 Mar 2020 01:02:30 +0200 Subject: RFR: 8241837: Cleanup stringStream usage in ObjectSynchronizer Message-ID: <4873f9b5-d9ec-242a-fed5-46a3b7905b1a@oracle.com> Hi, this removes one unused stringStream, and refactors two other so that they're only defined in the slow path error where they are used: http://cr.openjdk.java.net/~redestad/8241837/open.00/ While a very small startup performance gain (removes ~50k instructions on a Hello World), these stringStreams mallocs on init, which could be problematic at times in this particular area. Thanks! /Claes From david.holmes at oracle.com Mon Mar 30 23:49:11 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 31 Mar 2020 09:49:11 +1000 Subject: RFR: 8241234: Unify monitor enter/exit runtime entries In-Reply-To: <9c330297-d2a4-df6e-4f32-3c26d72d0526@oracle.com> References: <222D2846-F6AE-4D5B-B41F-F976D90E329C@oracle.com> <91eeada8-e05f-bc73-b029-94e169216a56@oracle.com> <534b8cf7-cd8c-565b-5163-09a216d4f94e@oracle.com> <904faf68-4fff-f1b8-2fb8-48d65f282fa2@oracle.com> <3fc16b46-7fdb-9a02-85aa-b517640160d3@oracle.com> <748209ee-5e5f-7a3d-3713-0ed835e710aa@oracle.com> <9c330297-d2a4-df6e-4f32-3c26d72d0526@oracle.com> Message-ID: On 31/03/2020 1:41 am, coleen.phillimore at oracle.com wrote: > On 3/30/20 11:20 AM, Daniel D. Daugherty wrote: >> On 3/30/20 10:27 AM, coleen.phillimore at oracle.com wrote: >>> >>> >>> On 3/30/20 10:20 AM, Daniel D. Daugherty wrote: >>>> On 3/30/20 10:15 AM, Yudi Zheng wrote: >>>>> Hi Daniel, >>>>> >>>>> Thanks for the review! I have uploaded a new version with your >>>>> comments addressed: >>>>> http://cr.openjdk.java.net/~yzheng/8241234/webrev.04/ >>>>> >>>>>> src/hotspot/share/runtime/sharedRuntime.hpp >>>>>> ???? Please don't forget to update the copyright year before you >>>>>> push. >>>>> Fixed. >>>>> >>>>>> src/hotspot/share/runtime/sharedRuntime.cpp >>>>>> ???? L2104:?? ObjectSynchronizer::exit(obj, lock, THREAD); >>>>>> ???????? The use of 'THREAD' here and 'TRAPS' in the function itself >>>>>> ???????? standout more now, but that's something for me to cleanup. >>>>> Also, I noticed that C2 was using CHECK >>>>>> ??? ObjectSynchronizer::enter(h_obj, lock, CHECK); >>>>> While C1 and JVMCI were using THREAD: >>>>>> ??? ObjectSynchronizer::enter(h_obj, lock->lock(), THREAD); >>>>> I have no idea when to use what, and hope unifying to the C2 >>>>> entries would help. >>>>> Let me know if there is something I should address in this patch. >>>>> Otherwise, I would >>>>> rather leave it to the expert, i.e., you ;) >>>> >>>> Yes, please leave it for me to clean up. >>> >>> It appears that both of these places (monitor_enter_helper and >>> monitor_exit_helper) should be using CATCH. >> >> Here's CATCH: >> >> src/hotspot/share/utilities/exceptions.hpp: >> >> // The CATCH macro checks that no exception has been thrown by a >> function; it is used at >> // call sites about which is statically known that the callee cannot >> throw an exception >> // even though it is declared with TRAPS. >> >> #define CATCH????????????????????????????? \ >> ? THREAD); if (HAS_PENDING_EXCEPTION) {??? \ >> ??? oop ex = PENDING_EXCEPTION;??????????? \ >> ??? CLEAR_PENDING_EXCEPTION;?????????????? \ >> ??? ex->print();?????????????????????????? \ >> ??? ShouldNotReachHere();????????????????? \ >> ? } (void)(0 >> >> >> I don't think you want either monitor_enter_helper() or >> monitor_exit_helper() crashing the VM via a ShouldNotReachHere() >> if someone happens to throw an exception at a thread during >> a 'monitorenter' or 'monitorexit' bytecode. > > Of course, it's more complicated.? I was looking at this: > > ?? ObjectSynchronizer::enter(h_obj, lock, CHECK); > ?? assert(!HAS_PENDING_EXCEPTION, "Should have no exception here"); > ?? JRT_BLOCK_END > > Which essentially wants to, but fails to, assert that there's no > exceptions possible (it is enclosed with JRT_BLOCK_NO_ASYNC). > > I can't see whether ObjectSynchronizer::enter() has code to exclude > async exceptions from being installed (possibly biased locking revoking > at a handshake?). > > And this: > > +void SharedRuntime::monitor_exit_helper(oopDesc* obj, BasicLock* lock, > JavaThread* thread) { > + assert(JavaThread::current() == thread, "invariant"); > ???? // Exit must be non-blocking, and therefore no exceptions can be > thrown. > ???? EXCEPTION_MARK; > ???? ObjectSynchronizer::exit(obj, lock, THREAD); > - } > +} > > Which has ExceptionMark whose destructor has: > > fatal("ExceptionMark destructor expects no pending exceptions"); > > Which appears to be enabled in production. > > So either way, you want the functionality of CHECK, just not have it > assert in production (which was a surprise to me). > > Actually, I agree with you that you want to clean this up. +1 for Dan to clean it up :) Conceptually monitor entry should not generate any exceptions, but detection of async exceptions needs to be done very carefully. Monitor exit can trigger IllegalMonitorStateException but the devil is in the detail as this feeds into the "balanced locking" rules. So whether or not it is possible depends on exactly where this code fits in. David > Coleen >> >> What am I missing here? >> >> Dan >> >> >>> >>> Coleen >>>> >>>> >>>>>> src/hotspot/share/c1/c1_Runtime1.cpp >>>>>> ???? old L718:?? assert(thread == JavaThread::current(), "threads >>>>>> must correspond"); >>>>>> ???????? Removed in favor of the assert in >>>>>> SharedRuntime::monitor_enter_helper(). >>>>>> ???????? Okay that makes sense. >>>>>> >>>>>> ???? old L721:?? EXCEPTION_MARK; >>>>>> ???????? Removed in favor of the same in >>>>>> SharedRuntime::monitor_enter_helper(). >>>>>> ???????? Okay that makes sense. >>>>>> >>>>>> src/hotspot/share/jvmci/jvmciRuntime.cpp >>>>>> ???? old L403:?? assert(thread == JavaThread::current(), "threads >>>>>> must correspond"); >>>>>> ???? old L406:?? EXCEPTION_MARK; >>>>>> ???????? Same as for c1_Runtime1.cpp >>>>> I assume I don?t need to do anything regarding the comments above. >>>> >>>> Correct. Just observations on the old code. >>>> >>>> >>>>>> ???? L390:???? TRACE_jvmci_3("%s: entered locking slow case with >>>>>> obj="... >>>>>> ???? L394:?? TRACE_jvmci_3("%s: exiting locking slow with obj=" >>>>>> ???? L417:???? TRACE_jvmci_3("%s: exited locking slow case with obj=" >>>>>> ???????? But this is no longer the "slow" case so I'm a bit confused. >>>>>> >>>>>> ???????? Update: I see there's a comment about the tracing being >>>>>> removed. >>>>>> ???????? I have no opinion on that since it is JVM/CI code, but >>>>>> the word >>>>>> ???????? "slow" needs to be adjusted if you keep it. >>>>> I removed all the tracing code. >>>> >>>> Thanks for cleaning that up. >>>> >>>> Dan >>>> >>>>> >>>>> Many thanks, >>>>> Yudi >>>> >>> >> > From david.holmes at oracle.com Mon Mar 30 23:55:10 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 31 Mar 2020 09:55:10 +1000 Subject: RFR: 8241837: Cleanup stringStream usage in ObjectSynchronizer In-Reply-To: <4873f9b5-d9ec-242a-fed5-46a3b7905b1a@oracle.com> References: <4873f9b5-d9ec-242a-fed5-46a3b7905b1a@oracle.com> Message-ID: <579fa4c7-7dbb-74a9-7010-4400bdc9980a@oracle.com> Hi Claes, On 31/03/2020 9:02 am, Claes Redestad wrote: > Hi, > > this removes one unused stringStream, and refactors two other > so that they're only defined in the slow path error where they are used: > > http://cr.openjdk.java.net/~redestad/8241837/open.00/ Looks good. Does make me wonder where the ResourceMarks are that should be needed to use the stringStream ?? Thanks, David > While a very small startup performance gain (removes ~50k instructions > on a Hello World), these stringStreams mallocs on init, which could be > problematic at times in this particular area. > > Thanks! > > /Claes From claes.redestad at oracle.com Tue Mar 31 00:03:14 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 31 Mar 2020 02:03:14 +0200 Subject: RFR: 8241837: Cleanup stringStream usage in ObjectSynchronizer In-Reply-To: <579fa4c7-7dbb-74a9-7010-4400bdc9980a@oracle.com> References: <4873f9b5-d9ec-242a-fed5-46a3b7905b1a@oracle.com> <579fa4c7-7dbb-74a9-7010-4400bdc9980a@oracle.com> Message-ID: <174bf722-7093-da5b-a6d7-4a29809c9a99@oracle.com> On 2020-03-31 01:55, David Holmes wrote: > Hi Claes, > > On 31/03/2020 9:02 am, Claes Redestad wrote: >> Hi, >> >> this removes one unused stringStream, and refactors two other >> so that they're only defined in the slow path error where they are used: >> >> http://cr.openjdk.java.net/~redestad/8241837/open.00/ > > Looks good. Thanks! > > Does make me wonder where the ResourceMarks are that should be needed to > use the stringStream ?? I think you'll find that you only need a RM if you call ss.as_string() - the ss itself is allocated on stack and its backing storage malloc'd on C heap. /Claes > > Thanks, > David > >> While a very small startup performance gain (removes ~50k instructions >> on a Hello World), these stringStreams mallocs on init, which could be >> problematic at times in this particular area. >> >> Thanks! >> >> /Claes From david.holmes at oracle.com Tue Mar 31 00:15:00 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 31 Mar 2020 10:15:00 +1000 Subject: RFR: 8241837: Cleanup stringStream usage in ObjectSynchronizer In-Reply-To: <174bf722-7093-da5b-a6d7-4a29809c9a99@oracle.com> References: <4873f9b5-d9ec-242a-fed5-46a3b7905b1a@oracle.com> <579fa4c7-7dbb-74a9-7010-4400bdc9980a@oracle.com> <174bf722-7093-da5b-a6d7-4a29809c9a99@oracle.com> Message-ID: On 31/03/2020 10:03 am, Claes Redestad wrote: > On 2020-03-31 01:55, David Holmes wrote: >> Hi Claes, >> >> On 31/03/2020 9:02 am, Claes Redestad wrote: >>> Hi, >>> >>> this removes one unused stringStream, and refactors two other >>> so that they're only defined in the slow path error where they are used: >>> >>> http://cr.openjdk.java.net/~redestad/8241837/open.00/ >> >> Looks good. > > Thanks! > >> >> Does make me wonder where the ResourceMarks are that should be needed >> to use the stringStream ?? > > I think you'll find that you only need a RM if you call ss.as_string() - > the ss itself is allocated on stack and its backing storage malloc'd on > C heap. Ah! Usage is more subtle than I thought. There's a modification coming in via JEP-371 that allows as_string() to optionally return C-Heap copy of internal buffer - for when you need the string to live more permanently. That gives three usage patterns: - use base() when the string can be deleted by the stringStream destructor - use as_string(false) (default) when you need the string live longer than the stringStream, but within an enclosing ResourceMark scope. - use as_string(true) to get a "permanent" copy (that you have to free) Cheers, David > /Claes > >> >> Thanks, >> David >> >>> While a very small startup performance gain (removes ~50k instructions >>> on a Hello World), these stringStreams mallocs on init, which could be >>> problematic at times in this particular area. >>> >>> Thanks! >>> >>> /Claes From claes.redestad at oracle.com Tue Mar 31 00:53:19 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 31 Mar 2020 02:53:19 +0200 Subject: RFR: 8241837: Cleanup stringStream usage in ObjectSynchronizer In-Reply-To: References: <4873f9b5-d9ec-242a-fed5-46a3b7905b1a@oracle.com> <579fa4c7-7dbb-74a9-7010-4400bdc9980a@oracle.com> <174bf722-7093-da5b-a6d7-4a29809c9a99@oracle.com> Message-ID: <9fb5aa39-2adc-2310-656b-026dcebe9243@oracle.com> On 2020-03-31 02:15, David Holmes wrote: > There's a modification coming in via JEP-371 that allows as_string() to > optionally return C-Heap copy of internal buffer - for when you need the > string to live more permanently. That gives three usage patterns: > > - use base() when the string can be deleted by the stringStream destructor > - use as_string(false) (default) when you need the string live longer > than the stringStream, but within an enclosing ResourceMark scope. > - use as_string(true) to get a "permanent" copy (that you have to free) Interesting - doesn't sound very JEP-371-specific, but a welcome improvement. /Claes From david.holmes at oracle.com Tue Mar 31 05:18:24 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 31 Mar 2020 15:18:24 +1000 Subject: RFR (S): 8239895: assert(_stack_base != 0LL) failed: Sanity check Message-ID: <04e25b32-69e4-8740-68b7-f02b9a713690@oracle.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8239895 webrev: http://cr.openjdk.java.net/~dholmes/8239895/webrev/ Prior to JDK-8238988 there were uses of stack_base() which checked it was initialized, and there was a raw use of _stack_base in on_local_stack() that did not need it to be initialized (because it may not be). After JDK-8238988 both cases call is_in_stack_range() which uses stack_base() and so asserts that the stack base is initialized in all cases. This leads to the assertion failures when the _stack_base is not initialised. The fix has three parts: 1. Rename is_in_full_stack to is_in_full_stack_checked - as it checks _stack_base is initialized via an assertion. 2. Add a new is_in_full_stack which doesn't use any assertions. 3. Update all the uses of stack_base() prior to JDK-8238988 that were changed to call is_in_full_stack, to now call is_in_full_stack_checked. There are not many of them. (The corollary to that is that all old calls to on_local_stack() call the new unchecked is_in_full_stack.) Here's the webrev for JDK-8238988 for comparison if desired: http://cr.openjdk.java.net/~dholmes/8238988/webrev/ I also backed out the assertion changes that I made under: https://bugs.openjdk.java.net/browse/JDK-8241043 as they were failing due to the use of get_thread_name(). I've filed a separate RFE for that issue: https://bugs.openjdk.java.net/browse/JDK-8241403 Testing: tiers 1 - 3 Thanks, David From thomas.stuefe at gmail.com Tue Mar 31 06:55:00 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 31 Mar 2020 08:55:00 +0200 Subject: RFR(xs): 8241726: Re-enable gtest for BitMap::count_one_bits() Message-ID: Hi, please review this small patch. It adds a gtest to test the BitMap::count_one_bit() variants, including the new form which works over arbitrary bit ranges. That and this gtest were introduced with [1] but the gtest showed errors and I removed it with [2]. It is now fixed and retested on all our platforms, including big endian and 32bit machines. So it can be re-added. Issue: https://bugs.openjdk.java.net/browse/JDK-8241726 Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8241726-reenable-gtest-for-count-one-bits/webrev.00/webrev/ Thanks, Thomas [1] https://bugs.openjdk.java.net/browse/JDK-8241581 [2] https://bugs.openjdk.java.net/browse/JDK-8241723 From lutz.schmidt at sap.com Tue Mar 31 07:21:11 2020 From: lutz.schmidt at sap.com (Schmidt, Lutz) Date: Tue, 31 Mar 2020 07:21:11 +0000 Subject: RFR(S): 8241101: [s390] jtreg test failure after JDK-8238696: not conformant features string Message-ID: Dear All, may I please request reviews for this fix, adjusting the CPU features string such that jtreg tests, in particular a newly introduced one, pass OK. Bug: https://bugs.openjdk.java.net/browse/JDK-8241101 Webrev: https://cr.openjdk.java.net/~lucy/webrevs/8241101.00/ Thank you! Lutz From thomas.stuefe at gmail.com Tue Mar 31 08:01:18 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 31 Mar 2020 10:01:18 +0200 Subject: RFR(S): 8241101: [s390] jtreg test failure after JDK-8238696: not conformant features string In-Reply-To: References: Message-ID: HI Lutz, Should we not better adjust the jtreg test? Cheers, Thomas On Tue, Mar 31, 2020 at 9:22 AM Schmidt, Lutz wrote: > Dear All, > > may I please request reviews for this fix, adjusting the CPU features > string such that jtreg tests, in particular a newly introduced one, pass OK. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8241101 > Webrev: https://cr.openjdk.java.net/~lucy/webrevs/8241101.00/ > > Thank you! > Lutz > > > > From martin.doerr at sap.com Tue Mar 31 09:33:27 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 31 Mar 2020 09:33:27 +0000 Subject: RFR(S): 8241101: [s390] jtreg test failure after JDK-8238696: not conformant features string In-Reply-To: References: Message-ID: Hi Thomas, I agree with that. test/lib/sun/hotspot/cpuinfo/CPUInfo.java uses return features.contains(feature.toLowerCase()); in "hasFeature" which is not nicely designed. If it is required to lower the case (for some reason), I think it should better use features.contains(feature) || features.contains(feature.toLowerCase()) Or even better if we don't need toLowerCase at all and fix the strings. Anyway, the 2nd part of the change (SHA, AES, ...) looks good to me. Best regards, Martin > -----Original Message----- > From: hotspot-runtime-dev bounces at openjdk.java.net> On Behalf Of Thomas St?fe > Sent: Dienstag, 31. M?rz 2020 10:01 > To: Schmidt, Lutz > Cc: hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR(S): 8241101: [s390] jtreg test failure after JDK-8238696: not > conformant features string > > HI Lutz, > > Should we not better adjust the jtreg test? > > Cheers, Thomas > > On Tue, Mar 31, 2020 at 9:22 AM Schmidt, Lutz > wrote: > > > Dear All, > > > > may I please request reviews for this fix, adjusting the CPU features > > string such that jtreg tests, in particular a newly introduced one, pass OK. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8241101 > > Webrev: https://cr.openjdk.java.net/~lucy/webrevs/8241101.00/ > > > > Thank you! > > Lutz > > > > > > > > From denghui.ddh at alibaba-inc.com Tue Mar 31 10:01:53 2020 From: denghui.ddh at alibaba-inc.com (Denghui Dong) Date: Tue, 31 Mar 2020 18:01:53 +0800 Subject: =?UTF-8?B?UmU6IFBJTkc6IFJGUihTKTogODI0MDM2MDogTmF0aXZlTGlicmFyeUV2ZW50IGhhcyB3cm9u?= =?UTF-8?B?ZyBsaWJyYXJ5IG5hbWUgb24gTGludXg=?= In-Reply-To: References: <0741e423-3bfc-4bbc-8929-54500f387cb7.denghui.ddh@alibaba-inc.com> <9f586a0e-11ff-1dce-575a-508686a3427f@oss.nttdata.com> , <774cc959-d635-0699-34d4-f85bb83de764@oss.nttdata.com>, Message-ID: <872c2924-db54-452d-aa73-1a52e4919b5d.denghui.ddh@alibaba-inc.com> Hi, Could anyone review it, need one more reviewer. Webrev: http://cr.openjdk.java.net/~ddong/8240360/webrev.01/ JBS: https://bugs.openjdk.java.net/browse/JDK-8240360 Thanks Denghui ------------------------------------------------------------------ From:???(??) Send Time:2020?3?23?(???) 21:26 To:hotspot-jfr-dev ; Yasumasa Suenaga Subject:Re: PING: RFR(S): 8240360: NativeLibraryEvent has wrong library name on Linux Thank you Yasumasa. Can I get one more reviewer please? Cheers, Denghui Dong ------------------------------------------------------------------ From:Yasumasa Suenaga Send Time:2020?3?23?(???) 20:25 To:???(??) ; hotspot-jfr-dev Subject:Re: PING: RFR(S): 8240360: NativeLibraryEvent has wrong library name on Linux Hi Denghui, On 2020/03/23 18:08, Denghui Dong wrote: > Hi Yasumasa, > > Thanks for the review. > > I agree with you that it's necessary to filter those lines in which the file is not a shared object by check the exec bit. > I think we should create another bug to trace the problem that you mentioned. I filed it: https://bugs.openjdk.java.net/browse/JDK-8241439 I think we might use dl_iterate_phdr() for the fix. I will try it. > BTW, Could you sponsor it? Sure, but you need to get one more reviewer before pushing. Thanks, Yasumasa > Thanks, > Denghui Dong > > ------------------------------------------------------------------ > From:Yasumasa Suenaga > Send Time:2020?3?23?(???) 16:31 > To:???(??) ; hotspot-jfr-dev > Subject:Re: PING: RFR(S): 8240360: NativeLibraryEvent has wrong library name on Linux > > Hi Denghui, > > Your change looks good. > > However, we might need to update os::get_loaded_modules_info() more. > Currently, we would get multiple jdk.NativeLibrary events from 1 library as below: > > ``` > jdk.NativeLibrary { > startTime = 16:59:06.621 > name = "/usr/lib64/libc-2.30.so" > baseAddress = 0x7FDB06ACC000 > topAddress = 0x7FDB06AF1000 > } > > jdk.NativeLibrary { > startTime = 16:59:06.621 > name = "/usr/lib64/libc-2.30.so" > baseAddress = 0x7FDB06AF1000 > topAddress = 0x7FDB06C40000 > } > > : > > ``` > > In addition, we would get the events even if they are not shared library as below: > > ``` > jdk.NativeLibrary { > startTime = 16:59:06.621 > name = "/home/ysuenaga/OpenJDK/jdk/build/linux-x86_64-server-fastdebug/images/jdk/lib/server/cl > asses.jsa" > baseAddress = 0x800000000 > topAddress = 0x800007000 > } > ``` > > os::get_loaded_modules_info() has introduced in JDK-8056242, and it says this function is for dll/so/dylibs. > In fact, os::get_loaded_modules_info() uses EnumProcessModules() in Windows. > > So I think we need to check exec bit, and merge same library to one event. > But I think it is another issue, and I'm not sure my think is correct. > > Thus I agree with your change now. > > > Thanks, > > Yasumasa (ysuenaga) > > > On 2020/03/20 11:12, Denghui Dong wrote: > > PING: Could you review it? > > > > Webrev: http://cr.openjdk.java.net/~ddong/8240360/webrev.01/ > > JBS: https://bugs.openjdk.java.net/browse/JDK-8240360 > > > > In Linux, EventNativeLibrary has a wrong name if the target library is located in a device whose device number (major or minor) exceeds the range of two digits. > > > > My fix is small, use "%*s" to ignore perms, offset and device. > > > > --- old/src/hotspot/os/linux/os_linux.cpp 2020-03-10 14:35:41.224134041 +0800 > > +++ new/src/hotspot/os/linux/os_linux.cpp 2020-03-10 14:35:41.090129104 +0800 > > @@ -2078,20 +2078,18 @@ > > > > // Read line by line from 'file' > > while (fgets(line, sizeof(line), procmapsFile) != NULL) { > > - u8 base, top, offset, inode; > > - char permissions[5]; > > - char device[6]; > > + u8 base, top, inode; > > char name[sizeof(line)]; > > > > - // Parse fields from line > > - int matches = sscanf(line, UINT64_FORMAT_X "-" UINT64_FORMAT_X " %4s " UINT64_FORMAT_X " %5s " INT64_FORMAT " %s", > > - &base, &top, permissions, &offset, device, &inode, name); > > - // the last entry 'name' is empty for some entries, so we might have 6 matches instead of 7 for some lines > > - if (matches < 6) continue; > > - if (matches == 6) name[0] = '\0'; > > + // Parse fields from line, discard perms, offset and device > > + int matches = sscanf(line, UINT64_FORMAT_X "-" UINT64_FORMAT_X " %*s %*s %*s " INT64_FORMAT " %s", > > + &base, &top, &inode, name); > > + // the last entry 'name' is empty for some entries, so we might have 3 matches instead of 4 for some lines > > + if (matches < 3) continue; > > + if (matches == 3) name[0] = '\0'; > > > > - // Filter by device id '00:00' so that we only get file system mapped files. > > - if (strcmp(device, "00:00") != 0) { > > + // Filter by inode 0 so that we only get file system mapped files. > > + if (inode != 0) { > > > > // Call callback with the fields of interest > > if(callback(name, (address)base, (address)top, param)) { > > > > > > Thanks, > > Denghui Dong > > > > > > > > ------------------------------------------------------------------ > > From:???(??) > > Send Time:2020?3?10?(???) 14:16 > > To:hotspot-jfr-dev > > Subject:Re: RFR(S): 8240360: NativeLibraryEvent has wrong library name on Linux > > > > Hi, > > I used a new way to fix this problem. > > Ignore "perms", "offset" and "device" fields by "%*s" so that we don't need to be concerned about the length of the device, > > and use "inode" instead of "device" for filtering. > > > > Webrev: http://cr.openjdk.java.net/~ddong/8240360/webrev.01/ > > > > Could you review it? > > > > Thanks, > > Denghui Dong > > ------------------------------------------------------------------ > > From:???(??) > > Send Time:2020?3?4?(???) 12:05 > > To:hotspot-jfr-dev > > Subject:Re: RFR(S): 8240360: NativeLibraryEvent has wrong library name on Linux > > > > Testing: jdk/jfr all passed. > > ------------------------------------------------------------------ > > From:???(??) > > Send Time:2020?3?3?(???) 22:25 > > To:hotspot-jfr-dev > > Subject:RFR(S): 8240360: NativeLibraryEvent has wrong library name on Linux > > > > Hi team, > > > > I found TestNativeLibrariesEvent is always failed in my Linux environment. > > > > The root cause is that os::get_loaded_modules_info in os_linux.cpp can not read library name correctly, > > because the device number in my Linux environment is 103:01, the format passed to sscanf can't identify > > it. > > > > I found the limit of major and minor device number in Linux from > > https://github.com/torvalds/linux/blob/master/include/linux/kdev_t.h, and made this patch. > > > > Please review this small change that fixes this problem. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8240360 > > Webrev: http://cr.openjdk.java.net/~ddong/8240360/webrev.00/ > > > > > > Thanks > > Denghui Dong > > > From martin.doerr at sap.com Tue Mar 31 11:59:47 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 31 Mar 2020 11:59:47 +0000 Subject: fatal error: must own lock Threads_lock Message-ID: Hi Dan, we have observed the error (see subject) when running jck test vm/jvmti/GetObjectMonitorUsage/gomu001/gomu00102/gomu00102.html on Windows. Stack: V [jvm.dll+0x413e10] report_fatal+0x80 (debug.cpp:286) V [jvm.dll+0x990eb6] assert_locked_or_safepoint_or_handshake+0xf6 (mutexlocker.cpp:194) V [jvm.dll+0xc14d95] JavaThread::get_thread_name+0x85 (thread.cpp:3191) V [jvm.dll+0x17c07] Thread::stack_end+0x27 (thread.hpp:762) V [jvm.dll+0xc160e1] JavaThread::is_lock_owned+0x21 (thread.cpp:2246) V [jvm.dll+0xc17cf4] Threads::owning_thread_from_monitor_owner+0x74 (thread.cpp:4752) V [jvm.dll+0x7e6d43] JvmtiEnvBase::get_object_monitor_usage+0x1e3 (jvmtienvbase.cpp:996) V [jvm.dll+0x7ddc2c] JvmtiEnv::GetObjectMonitorUsage+0x4c (jvmtienv.cpp:2870) V [jvm.dll+0x7ad650] jvmti_GetObjectMonitorUsage+0x130 (jvmtienter.cpp:4105) C [jckjvmti.dll+0x3a56a] Threads_lock was taken before the following change: 8167108: inconsistent handling of SR_lock can lead to crashes Summary: Add Thread Safe Memory Reclamation (Thread-SMR) mechanism. http://hg.openjdk.java.net/jdk/jdk/rev/8d15b1369c7a I guess we need to add it back. Should I create an issue for that? Best regards, Martin From david.holmes at oracle.com Tue Mar 31 12:13:24 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 31 Mar 2020 22:13:24 +1000 Subject: fatal error: must own lock Threads_lock In-Reply-To: References: Message-ID: Hi Martin, Sorry for the noise with this one. The new assertion failure masks the underlying one which relates to: 8239895: assert(_stack_base != 0LL) failed: Sanity check which is out for review right now. On 31/03/2020 9:59 pm, Doerr, Martin wrote: > Hi Dan, > > we have observed the error (see subject) when running jck test > vm/jvmti/GetObjectMonitorUsage/gomu001/gomu00102/gomu00102.html > on Windows. > > Stack: > V [jvm.dll+0x413e10] report_fatal+0x80 (debug.cpp:286) > V [jvm.dll+0x990eb6] assert_locked_or_safepoint_or_handshake+0xf6 (mutexlocker.cpp:194) > V [jvm.dll+0xc14d95] JavaThread::get_thread_name+0x85 (thread.cpp:3191) > V [jvm.dll+0x17c07] Thread::stack_end+0x27 (thread.hpp:762) > V [jvm.dll+0xc160e1] JavaThread::is_lock_owned+0x21 (thread.cpp:2246) > V [jvm.dll+0xc17cf4] Threads::owning_thread_from_monitor_owner+0x74 (thread.cpp:4752) > V [jvm.dll+0x7e6d43] JvmtiEnvBase::get_object_monitor_usage+0x1e3 (jvmtienvbase.cpp:996) > V [jvm.dll+0x7ddc2c] JvmtiEnv::GetObjectMonitorUsage+0x4c (jvmtienv.cpp:2870) > V [jvm.dll+0x7ad650] jvmti_GetObjectMonitorUsage+0x130 (jvmtienter.cpp:4105) > C [jckjvmti.dll+0x3a56a] > > Threads_lock was taken before the following change: > 8167108: inconsistent handling of SR_lock can lead to crashes > Summary: Add Thread Safe Memory Reclamation (Thread-SMR) mechanism. > http://hg.openjdk.java.net/jdk/jdk/rev/8d15b1369c7a > > I guess we need to add it back. No, the problem is that I used get_thread_name() in a context that isn't supported at the moment. I filed an additional RFE to fix it properly: https://bugs.openjdk.java.net/browse/JDK-8241043 Cheers, David ----- > Should I create an issue for that? > > Best regards, > Martin > From david.holmes at oracle.com Tue Mar 31 12:18:23 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 31 Mar 2020 22:18:23 +1000 Subject: fatal error: must own lock Threads_lock In-Reply-To: References: Message-ID: <41c615a6-e60a-d7e0-810d-0eb1d4c0985b@oracle.com> Correction at end ... On 31/03/2020 10:13 pm, David Holmes wrote: > Hi Martin, > > Sorry for the noise with this one. The new assertion failure masks the > underlying one which relates to: > > ?8239895: assert(_stack_base != 0LL) failed: Sanity check > > which is out for review right now. > > > On 31/03/2020 9:59 pm, Doerr, Martin wrote: >> Hi Dan, >> >> we have observed the error (see subject) when running jck test >> vm/jvmti/GetObjectMonitorUsage/gomu001/gomu00102/gomu00102.html >> on Windows. >> >> Stack: >> V? [jvm.dll+0x413e10]? report_fatal+0x80? (debug.cpp:286) >> V? [jvm.dll+0x990eb6]? assert_locked_or_safepoint_or_handshake+0xf6 >> (mutexlocker.cpp:194) >> V? [jvm.dll+0xc14d95]? JavaThread::get_thread_name+0x85 >> (thread.cpp:3191) >> V? [jvm.dll+0x17c07]? Thread::stack_end+0x27? (thread.hpp:762) >> V? [jvm.dll+0xc160e1]? JavaThread::is_lock_owned+0x21? (thread.cpp:2246) >> V? [jvm.dll+0xc17cf4]? Threads::owning_thread_from_monitor_owner+0x74 >> (thread.cpp:4752) >> V? [jvm.dll+0x7e6d43]? JvmtiEnvBase::get_object_monitor_usage+0x1e3 >> (jvmtienvbase.cpp:996) >> V? [jvm.dll+0x7ddc2c]? JvmtiEnv::GetObjectMonitorUsage+0x4c >> (jvmtienv.cpp:2870) >> V? [jvm.dll+0x7ad650]? jvmti_GetObjectMonitorUsage+0x130 >> (jvmtienter.cpp:4105) >> C? [jckjvmti.dll+0x3a56a] >> >> Threads_lock was taken before the following change: >> 8167108: inconsistent handling of SR_lock can lead to crashes >> Summary: Add Thread Safe Memory Reclamation (Thread-SMR) mechanism. >> http://hg.openjdk.java.net/jdk/jdk/rev/8d15b1369c7a >> >> I guess we need to add it back. > > No, the problem is that I used get_thread_name() in a context that isn't > supported at the moment. I filed an additional RFE to fix it properly: > > https://bugs.openjdk.java.net/browse/JDK-8241043 https://bugs.openjdk.java.net/browse/JDK-8241403 David ----- > Cheers, > David > ----- > >> Should I create an issue for that? >> >> Best regards, >> Martin >> From martin.doerr at sap.com Tue Mar 31 12:47:24 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 31 Mar 2020 12:47:24 +0000 Subject: fatal error: must own lock Threads_lock In-Reply-To: <41c615a6-e60a-d7e0-810d-0eb1d4c0985b@oracle.com> References: <41c615a6-e60a-d7e0-810d-0eb1d4c0985b@oracle.com> Message-ID: Thanks a lot, David! Best regards, Martin > -----Original Message----- > From: David Holmes > Sent: Dienstag, 31. M?rz 2020 14:18 > To: Doerr, Martin ; daniel.daugherty at oracle.com > Cc: hotspot-runtime-dev at openjdk.java.net > Subject: Re: fatal error: must own lock Threads_lock > > Correction at end ... > > On 31/03/2020 10:13 pm, David Holmes wrote: > > Hi Martin, > > > > Sorry for the noise with this one. The new assertion failure masks the > > underlying one which relates to: > > > > ?8239895: assert(_stack_base != 0LL) failed: Sanity check > > > > which is out for review right now. > > > > > > On 31/03/2020 9:59 pm, Doerr, Martin wrote: > >> Hi Dan, > >> > >> we have observed the error (see subject) when running jck test > >> > vm/jvmti/GetObjectMonitorUsage/gomu001/gomu00102/gomu00102.html > >> on Windows. > >> > >> Stack: > >> V? [jvm.dll+0x413e10]? report_fatal+0x80? (debug.cpp:286) > >> V? [jvm.dll+0x990eb6]? assert_locked_or_safepoint_or_handshake+0xf6 > >> (mutexlocker.cpp:194) > >> V? [jvm.dll+0xc14d95]? JavaThread::get_thread_name+0x85 > >> (thread.cpp:3191) > >> V? [jvm.dll+0x17c07]? Thread::stack_end+0x27? (thread.hpp:762) > >> > V? [jvm.dll+0xc160e1]? JavaThread::is_lock_owned+0x21? (thread.cpp:2246) > >> > V? [jvm.dll+0xc17cf4]? Threads::owning_thread_from_monitor_owner+0x74 > >> (thread.cpp:4752) > >> V? [jvm.dll+0x7e6d43]? JvmtiEnvBase::get_object_monitor_usage+0x1e3 > >> (jvmtienvbase.cpp:996) > >> V? [jvm.dll+0x7ddc2c]? JvmtiEnv::GetObjectMonitorUsage+0x4c > >> (jvmtienv.cpp:2870) > >> V? [jvm.dll+0x7ad650]? jvmti_GetObjectMonitorUsage+0x130 > >> (jvmtienter.cpp:4105) > >> C? [jckjvmti.dll+0x3a56a] > >> > >> Threads_lock was taken before the following change: > >> 8167108: inconsistent handling of SR_lock can lead to crashes > >> Summary: Add Thread Safe Memory Reclamation (Thread-SMR) > mechanism. > >> http://hg.openjdk.java.net/jdk/jdk/rev/8d15b1369c7a > >> > >> I guess we need to add it back. > > > > No, the problem is that I used get_thread_name() in a context that isn't > > supported at the moment. I filed an additional RFE to fix it properly: > > > > https://bugs.openjdk.java.net/browse/JDK-8241043 > > https://bugs.openjdk.java.net/browse/JDK-8241403 > > David > ----- > > > Cheers, > > David > > ----- > > > >> Should I create an issue for that? > >> > >> Best regards, > >> Martin > >> From lutz.schmidt at sap.com Tue Mar 31 13:37:56 2020 From: lutz.schmidt at sap.com (Schmidt, Lutz) Date: Tue, 31 Mar 2020 13:37:56 +0000 Subject: RFR(S): 8241101: [s390] jtreg test failure after JDK-8238696: not conformant features string In-Reply-To: References: Message-ID: <30182338-771E-4A43-849C-977C9B7C27F9@sap.com> Thomas, Martin, thank you for looking into this. Generally, I'm very much in favor of the "fix the root cause" approach. In this particular case, however, I'm afraid of side effects and excessive effort. CPUInfo.hasFeature() is called in quite a few places, see the attached text file for reference. My proposed change, on the other hand, is very constrained in scope: just s390, just a few string edits. The risk to break anything is extremely low. Therefore, I would like to stick with the proposed change. Thanks, Lutz ?On 31.03.20, 11:33, "Doerr, Martin" wrote: Hi Thomas, I agree with that. test/lib/sun/hotspot/cpuinfo/CPUInfo.java uses return features.contains(feature.toLowerCase()); in "hasFeature" which is not nicely designed. If it is required to lower the case (for some reason), I think it should better use features.contains(feature) || features.contains(feature.toLowerCase()) Or even better if we don't need toLowerCase at all and fix the strings. Anyway, the 2nd part of the change (SHA, AES, ...) looks good to me. Best regards, Martin > -----Original Message----- > From: hotspot-runtime-dev bounces at openjdk.java.net> On Behalf Of Thomas St?fe > Sent: Dienstag, 31. M?rz 2020 10:01 > To: Schmidt, Lutz > Cc: hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR(S): 8241101: [s390] jtreg test failure after JDK-8238696: not > conformant features string > > HI Lutz, > > Should we not better adjust the jtreg test? > > Cheers, Thomas > > On Tue, Mar 31, 2020 at 9:22 AM Schmidt, Lutz > wrote: > > > Dear All, > > > > may I please request reviews for this fix, adjusting the CPU features > > string such that jtreg tests, in particular a newly introduced one, pass OK. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8241101 > > Webrev: https://cr.openjdk.java.net/~lucy/webrevs/8241101.00/ > > > > Thank you! > > Lutz > > > > > > > > -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: grep_hasFeature.txt URL: From thomas.stuefe at gmail.com Tue Mar 31 13:50:47 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 31 Mar 2020 15:50:47 +0200 Subject: RFR(S): 8241101: [s390] jtreg test failure after JDK-8238696: not conformant features string In-Reply-To: <30182338-771E-4A43-849C-977C9B7C27F9@sap.com> References: <30182338-771E-4A43-849C-977C9B7C27F9@sap.com> Message-ID: Okay. The change looks good to me. Cheers, Thomas On Tue, Mar 31, 2020 at 3:38 PM Schmidt, Lutz wrote: > Thomas, Martin, > > thank you for looking into this. > > Generally, I'm very much in favor of the "fix the root cause" approach. In > this particular case, however, I'm afraid of side effects and excessive > effort. CPUInfo.hasFeature() is called in quite a few places, see the > attached text file for reference. > > My proposed change, on the other hand, is very constrained in scope: just > s390, just a few string edits. The risk to break anything is extremely low. > Therefore, I would like to stick with the proposed change. > > Thanks, > Lutz > > > ?On 31.03.20, 11:33, "Doerr, Martin" wrote: > > Hi Thomas, > > I agree with that. > > test/lib/sun/hotspot/cpuinfo/CPUInfo.java uses > return features.contains(feature.toLowerCase()); > in "hasFeature" which is not nicely designed. > > If it is required to lower the case (for some reason), I think it > should better use > features.contains(feature) || features.contains(feature.toLowerCase()) > > Or even better if we don't need toLowerCase at all and fix the strings. > > Anyway, the 2nd part of the change (SHA, AES, ...) looks good to me. > > Best regards, > Martin > > > > -----Original Message----- > > From: hotspot-runtime-dev > bounces at openjdk.java.net> On Behalf Of Thomas St?fe > > Sent: Dienstag, 31. M?rz 2020 10:01 > > To: Schmidt, Lutz > > Cc: hotspot-runtime-dev at openjdk.java.net > > Subject: Re: RFR(S): 8241101: [s390] jtreg test failure after > JDK-8238696: not > > conformant features string > > > > HI Lutz, > > > > Should we not better adjust the jtreg test? > > > > Cheers, Thomas > > > > On Tue, Mar 31, 2020 at 9:22 AM Schmidt, Lutz > > wrote: > > > > > Dear All, > > > > > > may I please request reviews for this fix, adjusting the CPU > features > > > string such that jtreg tests, in particular a newly introduced > one, pass OK. > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8241101 > > > Webrev: https://cr.openjdk.java.net/~lucy/webrevs/8241101.00/ > > > > > > Thank you! > > > Lutz > > > > > > > > > > > > > > > From martin.doerr at sap.com Tue Mar 31 13:54:31 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 31 Mar 2020 13:54:31 +0000 Subject: RFR(S): 8241101: [s390] jtreg test failure after JDK-8238696: not conformant features string In-Reply-To: References: <30182338-771E-4A43-849C-977C9B7C27F9@sap.com> Message-ID: But what about features.contains(feature) || features.contains(feature.toLowerCase()) ? Is it still problematic if we just support both, lower case and exact match? Best regards, Martin From: Thomas St?fe Sent: Dienstag, 31. M?rz 2020 15:51 To: Schmidt, Lutz Cc: Doerr, Martin ; Vladimir Ivanov ; hotspot-runtime-dev at openjdk.java.net Subject: Re: RFR(S): 8241101: [s390] jtreg test failure after JDK-8238696: not conformant features string Okay. The change looks good to me. Cheers, Thomas On Tue, Mar 31, 2020 at 3:38 PM Schmidt, Lutz > wrote: Thomas, Martin, thank you for looking into this. Generally, I'm very much in favor of the "fix the root cause" approach. In this particular case, however, I'm afraid of side effects and excessive effort. CPUInfo.hasFeature() is called in quite a few places, see the attached text file for reference. My proposed change, on the other hand, is very constrained in scope: just s390, just a few string edits. The risk to break anything is extremely low. Therefore, I would like to stick with the proposed change. Thanks, Lutz ?On 31.03.20, 11:33, "Doerr, Martin" > wrote: Hi Thomas, I agree with that. test/lib/sun/hotspot/cpuinfo/CPUInfo.java uses return features.contains(feature.toLowerCase()); in "hasFeature" which is not nicely designed. If it is required to lower the case (for some reason), I think it should better use features.contains(feature) || features.contains(feature.toLowerCase()) Or even better if we don't need toLowerCase at all and fix the strings. Anyway, the 2nd part of the change (SHA, AES, ...) looks good to me. Best regards, Martin > -----Original Message----- > From: hotspot-runtime-dev bounces at openjdk.java.net> On Behalf Of Thomas St?fe > Sent: Dienstag, 31. M?rz 2020 10:01 > To: Schmidt, Lutz > > Cc: hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR(S): 8241101: [s390] jtreg test failure after JDK-8238696: not > conformant features string > > HI Lutz, > > Should we not better adjust the jtreg test? > > Cheers, Thomas > > On Tue, Mar 31, 2020 at 9:22 AM Schmidt, Lutz > > wrote: > > > Dear All, > > > > may I please request reviews for this fix, adjusting the CPU features > > string such that jtreg tests, in particular a newly introduced one, pass OK. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8241101 > > Webrev: https://cr.openjdk.java.net/~lucy/webrevs/8241101.00/ > > > > Thank you! > > Lutz > > > > > > > > From lutz.schmidt at sap.com Tue Mar 31 13:56:35 2020 From: lutz.schmidt at sap.com (Schmidt, Lutz) Date: Tue, 31 Mar 2020 13:56:35 +0000 Subject: RFR(S): 8241101: [s390] jtreg test failure after JDK-8238696: not conformant features string In-Reply-To: References: <30182338-771E-4A43-849C-977C9B7C27F9@sap.com> Message-ID: Thank you, Thomas, for the review. Regards, Lutz From: Thomas St?fe Date: Tuesday, 31. March 2020 at 15:50 To: Lutz Schmidt Cc: "Doerr, Martin (martin.doerr at sap.com)" , Vladimir Ivanov , "hotspot-runtime-dev at openjdk.java.net" Subject: Re: RFR(S): 8241101: [s390] jtreg test failure after JDK-8238696: not conformant features string Okay. The change looks good to me. Cheers, Thomas On Tue, Mar 31, 2020 at 3:38 PM Schmidt, Lutz > wrote: Thomas, Martin, thank you for looking into this. Generally, I'm very much in favor of the "fix the root cause" approach. In this particular case, however, I'm afraid of side effects and excessive effort. CPUInfo.hasFeature() is called in quite a few places, see the attached text file for reference. My proposed change, on the other hand, is very constrained in scope: just s390, just a few string edits. The risk to break anything is extremely low. Therefore, I would like to stick with the proposed change. Thanks, Lutz On 31.03.20, 11:33, "Doerr, Martin" > wrote: Hi Thomas, I agree with that. test/lib/sun/hotspot/cpuinfo/CPUInfo.java uses return features.contains(feature.toLowerCase()); in "hasFeature" which is not nicely designed. If it is required to lower the case (for some reason), I think it should better use features.contains(feature) || features.contains(feature.toLowerCase()) Or even better if we don't need toLowerCase at all and fix the strings. Anyway, the 2nd part of the change (SHA, AES, ...) looks good to me. Best regards, Martin > -----Original Message----- > From: hotspot-runtime-dev bounces at openjdk.java.net> On Behalf Of Thomas St?fe > Sent: Dienstag, 31. M?rz 2020 10:01 > To: Schmidt, Lutz > > Cc: hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR(S): 8241101: [s390] jtreg test failure after JDK-8238696: not > conformant features string > > HI Lutz, > > Should we not better adjust the jtreg test? > > Cheers, Thomas > > On Tue, Mar 31, 2020 at 9:22 AM Schmidt, Lutz > > wrote: > > > Dear All, > > > > may I please request reviews for this fix, adjusting the CPU features > > string such that jtreg tests, in particular a newly introduced one, pass OK. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8241101 > > Webrev: https://cr.openjdk.java.net/~lucy/webrevs/8241101.00/ > > > > Thank you! > > Lutz > > > > > > > > From lutz.schmidt at sap.com Tue Mar 31 13:59:08 2020 From: lutz.schmidt at sap.com (Schmidt, Lutz) Date: Tue, 31 Mar 2020 13:59:08 +0000 Subject: RFR(S): 8241101: [s390] jtreg test failure after JDK-8238696: not conformant features string In-Reply-To: References: <30182338-771E-4A43-849C-977C9B7C27F9@sap.com> Message-ID: <6C62F39C-87D9-4C29-A7EA-5EFD9321C8FC@sap.com> Hi Martin, your extended check would heal the issue we saw with the original feature string on s390. So far, so good. I can?t judge if other tests can live with that. Regards, Lutz From: "Doerr, Martin (martin.doerr at sap.com)" Date: Tuesday, 31. March 2020 at 15:54 To: Thomas St?fe , Lutz Schmidt Cc: Vladimir Ivanov , "hotspot-runtime-dev at openjdk.java.net" Subject: RE: RFR(S): 8241101: [s390] jtreg test failure after JDK-8238696: not conformant features string But what about features.contains(feature) || features.contains(feature.toLowerCase()) ? Is it still problematic if we just support both, lower case and exact match? Best regards, Martin From: Thomas St?fe Sent: Dienstag, 31. M?rz 2020 15:51 To: Schmidt, Lutz Cc: Doerr, Martin ; Vladimir Ivanov ; hotspot-runtime-dev at openjdk.java.net Subject: Re: RFR(S): 8241101: [s390] jtreg test failure after JDK-8238696: not conformant features string Okay. The change looks good to me. Cheers, Thomas On Tue, Mar 31, 2020 at 3:38 PM Schmidt, Lutz > wrote: Thomas, Martin, thank you for looking into this. Generally, I'm very much in favor of the "fix the root cause" approach. In this particular case, however, I'm afraid of side effects and excessive effort. CPUInfo.hasFeature() is called in quite a few places, see the attached text file for reference. My proposed change, on the other hand, is very constrained in scope: just s390, just a few string edits. The risk to break anything is extremely low. Therefore, I would like to stick with the proposed change. Thanks, Lutz On 31.03.20, 11:33, "Doerr, Martin" > wrote: Hi Thomas, I agree with that. test/lib/sun/hotspot/cpuinfo/CPUInfo.java uses return features.contains(feature.toLowerCase()); in "hasFeature" which is not nicely designed. If it is required to lower the case (for some reason), I think it should better use features.contains(feature) || features.contains(feature.toLowerCase()) Or even better if we don't need toLowerCase at all and fix the strings. Anyway, the 2nd part of the change (SHA, AES, ...) looks good to me. Best regards, Martin > -----Original Message----- > From: hotspot-runtime-dev bounces at openjdk.java.net> On Behalf Of Thomas St?fe > Sent: Dienstag, 31. M?rz 2020 10:01 > To: Schmidt, Lutz > > Cc: hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR(S): 8241101: [s390] jtreg test failure after JDK-8238696: not > conformant features string > > HI Lutz, > > Should we not better adjust the jtreg test? > > Cheers, Thomas > > On Tue, Mar 31, 2020 at 9:22 AM Schmidt, Lutz > > wrote: > > > Dear All, > > > > may I please request reviews for this fix, adjusting the CPU features > > string such that jtreg tests, in particular a newly introduced one, pass OK. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8241101 > > Webrev: https://cr.openjdk.java.net/~lucy/webrevs/8241101.00/ > > > > Thank you! > > Lutz > > > > > > > > From martin.doerr at sap.com Tue Mar 31 14:01:02 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 31 Mar 2020 14:01:02 +0000 Subject: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents In-Reply-To: References: <1f8a3c7a-fa0f-b5b2-4a8a-7d3d8dbbe1b5@oracle.com> <4b56a45c-a14c-6f74-2bfd-25deaabe8201@oracle.com> <5271429a-481d-ddb9-99dc-b3f6670fcc0b@oracle.com> Message-ID: Hi Richard, thanks for addressing all my points. I've looked over webrev.5 and I'm satisfied with your changes. I had also promised to review the tests. test/hotspot/jtreg/serviceability/jvmti/Heap/IterateHeapWithEscapeAnalysisEnabled.java Thanks for updating the @summary comment. Looks good in webrev.5. test/hotspot/jtreg/serviceability/jvmti/Heap/libIterateHeapWithEscapeAnalysisEnabled.c JVMTI agent for object tagging and heap iteration. Good. test/jdk/com/sun/jdi/EATests.java This is a substantial amount of tests which is appropriate for a such a large change. Skipping some subtests with UseJVMCICompiler makes sense because it doesn't provide the necessary JVMTI functionality, yet. Nice work! I also like that you test with and without BiasedLocking. Your tests will still be fine after BiasedLocking deprecation. Very minor nits: - 2 typos in comment above EARelockingNestedInflatedTarget: "lockes are ommitted" (sounds funny) - You sometimes write "graal" and sometimes "Graal". I guess the capital G is better. (Also in EATestsJVMCI.java.) test/jdk/com/sun/jdi/EATestsJVMCI.java EATests with Graal enabled. Nice that you support Graal to some extent. Maybe Graal folks want to enhance them in the future. I think this is a good starting point. Conclusion: Looks good and not trivial :-) Now, you have one full review. I'd be ok with covering 2nd review by partial reviews. Compiler and JVMTI parts are not too complicated IMHO. Runtime part should get at least one additional careful review. Best regards, Martin > -----Original Message----- > From: Reingruber, Richard > Sent: Montag, 30. M?rz 2020 10:32 > To: Doerr, Martin ; 'Robbin Ehn' > ; Lindenmaier, Goetz > ; David Holmes ; > Vladimir Kozlov (vladimir.kozlov at oracle.com) > ; serviceability-dev at openjdk.java.net; > hotspot-compiler-dev at openjdk.java.net; hotspot-runtime- > dev at openjdk.java.net > Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better Performance > in the Presence of JVMTI Agents > > Hi, > > this is webrev.5 based on Robbin's feedback and Martin's review - thanks! :) > > The change affects jvmti, hotspot and c2. Partial reviews are very welcome > too. > > Full: http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.5/ > Delta: > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.5.inc/ > > Robbin, Martin, please let me know, if anything shouldn't be quite as you > wanted it. Also find my > comments on your feedback below. > > Robbin, can I count you as Reviewer for the runtime part? > > Thanks, Richard. > > -- > > > DeoptimizeObjectsALotThread is only used in compileBroker.cpp. > > You can move both declaration and definition to that file, no need to > clobber > > thread.[c|h]pp. (and the static function deopt_objs_alot_thread_entry) > > Done. > > > Does JvmtiDeferredUpdates really need to be in thread.hpp, can't be in it's > own > > hpp file? It doesn't seem right to add JVM TI classes into thread.hpp. > > I moved JvmtiDeferredUpdates to vframe_hp.hpp where preexisting > jvmtiDeferredLocalVariableSet is > declared. > > > src/hotspot/share/code/compiledMethod.cpp > > Nice cleanup! > > Thanks :) > > > src/hotspot/share/code/debugInfoRec.cpp > > src/hotspot/share/code/debugInfoRec.hpp > > Additional parmeters. (Remark: I think "non_global_escape_in_scope" > would read better than "not_global_escape_in_scope", but your version is > consistent with existing code, so no change request from my side.) Ok. > > I've been thinking about this too and finally stayed with > not_global_escape_in_scope. It's supposed > to mean an object whose escape state is not GlobalEscape is in scope. > > > src/hotspot/share/compiler/compileBroker.cpp > > src/hotspot/share/compiler/compileBroker.hpp > > Extra thread for DeoptimizeObjectsALot. (Remark: I would have put it into > a follow up change together with the test in order to make this webrev > smaller, but since it is included, I'm reviewing everything at once. Not a big > deal.) Ok. > > Yes the change would be a little smaller. And if it helps I'll split it off. In > general I prefer > patches that bring along a suitable amount of tests. > > > src/hotspot/share/opto/c2compiler.cpp > > Make do_escape_analysis independent of JVMCI capabilities. Nice! > > It is the main goal of the enhancement. It is done for C2, but could be done > for JVMCI compilers > with just a small effort as well. > > > src/hotspot/share/opto/escape.cpp > > Annotation for MachSafePointNodes. Your added functionality looks > correct. > > But I'd prefer to move the bulky code out of the large function. > > I suggest to factor out something like has_not_global_escape and > has_arg_escape. So the code could look like this: > > SafePointNode* sfn = sfn_worklist.at(next); > > sfn->set_not_global_escape_in_scope(has_not_global_escape(sfn)); > > if (sfn->is_CallJava()) { > > CallJavaNode* call = sfn->as_CallJava(); > > call->set_arg_escape(has_arg_escape(call)); > > } > > This would also allow us to get rid of the found_..._escape_in_args > variables making the loops better readable. > > Done. > > > It's kind of ugly to use strcmp to recognize uncommon trap, but that seems > to be the way to do it (there are more such places). So it's ok. > > Yeah. I copied the snippet. > > > src/hotspot/share/prims/jvmtiImpl.cpp > > src/hotspot/share/prims/jvmtiImpl.hpp > > The sequence is pretty complex: > > VM_GetOrSetLocal element initialization executes EscapeBarrier code > which suspends the target thread (extra VM Operation). > > Note that the target threads have to be suspended already for > VM_GetOrSetLocal*. So it's mainly the > synchronization effect of EscapeBarrier::sync_and_suspend_one() that is > required here. Also no extra > _handshake_ is executed, since sync_and_suspend_one() will find the > target threads already > suspended. > > > VM_GetOrSetLocal::doit_prologue performs object deoptimization (by VM > Thread to prepare VM Operation with frame deoptimization). > > VM_GetOrSetLocal destructor implicitly calls EscapeBarrier destructor > which resumes the target thread. > > But I don't have any improvement proposal. Performance is probably not a > concern, here. So it's ok. > > > VM_GetOrSetLocal::deoptimize_objects deoptimizes the top frame if it > has non-globally escaping objects and other frames if they have arg escaping > ones. Good. > > It's not specifically the top frame, but the frame that is accessed. > > > src/hotspot/share/runtime/deoptimization.cpp > > Object deoptimization. I have more comments and proposals, here. > > First of all, handling recursive and waiting locks in relock_objects is tricky, > but looks correct. > > Comments are sufficient to understand why things are done as they are > implemented. > > > BiasedLocking related parts are complex, but we may get rid of them in the > future (with BiasedLocking removal). > > Anyway, looks correct, too. > > > Typo in comment: "regularily" => "regularly" > > > Deoptimization::fetch_unroll_info_helper is the only place where > _jvmti_deferred_updates get deallocated (except JavaThread destructor). > But I think we always go through it, so I can't see a memory leak or such kind > of issues. > > That's correct. The compiled frame for which deferred updates are allocated > is always deoptimized > before (see EscapeBarrier::deoptimize_objects()). This is also asserted in > compiledVFrame::update_deferred_value(). I've added the same assertion > to > Deoptimization::relock_objects(). So we can be sure that > _jvmti_deferred_updates are deallocated > again in fetch_unroll_info_helper(). > > > EscapeBarrier::deoptimize_objects: ResourceMark should use > calling_thread(). > > Sure, well spotted! > > > You can use MutexLocker and MonitorLocker with Thread* to save the > Thread::current() call. > > Right, good hint. This was recently introduced with 8235678. I even had to > resolve conflicts. Should > have done this then. > > > I'd make set_objs_are_deoptimized static and remove it from the > EscapeBarrier interface because I think it shouldn't be used outside of > EscapeBarrier::deoptimize_objects. > > Done. > > > Typo in comment: "we must only deoptimize" => "we only have to > deoptimize" > > Replaced with "[...] we deoptimize iff local objects are passed as args" > > > "bool EscapeBarrier::deoptimize_objects(intptr_t* fr_id)" is trivial and > barrier_active() is redundant. Implementation can get moved to hpp file. > > Ok. Done. > > > I'll get back to suspend flags, later. > > > There are weird cases regarding _self_deoptimization_in_progress. > > Assume we have 3 threads A, B and C. A deopts C, B deopts C, C deopts C. > C can set _self_deoptimization_in_progress while A performs the handshake > for suspending C. I think this doesn't lead to errors, but it's probably not > desired. > > I think it would be better to use only one "wait" call in > sync_and_suspend_one and sync_and_suspend_all. > > You're right. We've discussed that face-to-face, but couldn't find a real issue. > But now, thinking again, a reckon I found one: > > 2808 // Sync with other threads that might be doing deoptimizations > 2809 { > 2810 // Need to switch to _thread_blocked for the wait() call > 2811 ThreadBlockInVM tbivm(_calling_thread); > 2812 MonitorLocker ml(EscapeBarrier_lock, > Mutex::_no_safepoint_check_flag); > 2813 while (_self_deoptimization_in_progress) { > 2814 ml.wait(); > 2815 } > 2816 > 2817 if (self_deopt()) { > 2818 _self_deoptimization_in_progress = true; > 2819 } > 2820 > 2821 while (_deoptee_thread->is_ea_obj_deopt_suspend()) { > 2822 ml.wait(); > 2823 } > 2824 > 2825 if (self_deopt()) { > 2826 return; > 2827 } > 2828 > 2829 // set suspend flag for target thread > 2830 _deoptee_thread->set_ea_obj_deopt_flag(); > 2831 } > > - A waits in 2822 > - C is suspended > - B notifies all in resume_one() > - A and C wake up > - C wins over A and sets _self_deoptimization_in_progress = true in 2818 > - C does the self deoptimization > - A executes 2830 _deoptee_thread->set_ea_obj_deopt_flag() > > C will self suspend at some undefined point. The resulting state is illegal. > > > I first thought it'd be better to move ThreadBlockInVM before wait() to > reduce thread state transitions, but that seems to be problematic because > ThreadBlockInVM destructor contains a safepoint check which we shouldn't > do while holding EscapeBarrier_lock. So no change request. > > Yes, would be nice to have the state change only if needed, but for the > reason you mentioned it is > not quite as easy as it seems to be. I experimented as well with a second > lock, but did not succeed. > > > Change in thred_added: > > I think the sequence would be more comprehensive if we waited for > deopt_all_threads in Thread::start and all other places where a new thread > can run into Java code (e.g. JVMTI attach). > > Your version makes new threads come up with suspend flag set. That looks > correct, too. Advantage is that you only have to change one place > (thread_added). It'll be interesting to see how it will look like when we use > async handshakes instead of suspend flags. > > For now, I'm ok with your version. > > I had a version that did what you are suggesting. The current version also has > the advantage, that > there are fewer places where a thread has to wait for ongoing object > deoptimization. This means > viewer places where you have to worry about correct thread state > transitions, possible deadlocks, > and if all oops are properly Handle'ed. > > > I'd only move MutexLocker ml(EscapeBarrier_lock...) after if (!jt- > >is_hidden_from_external_view()). > > Done. > > > Having 4 different deoptimize_objects functions makes it a little hard to > keep an overview of which one is used for what. > > Maybe adding suffixes would help a little bit, but I can also live with what > you have. > > Implementation looks correct to me. > > 2 are internal. I added the suffix _internal to them. This leaves 2 to choose > from. > > > src/hotspot/share/runtime/deoptimization.hpp > > Escape barriers and object deoptimization functions. > > Typo in comment: "helt" => "held" > > Done in place already. > > > src/hotspot/share/runtime/interfaceSupport.cpp > > InterfaceSupport::deoptimizeAllObjects() is only used for > DeoptimizeObjectsALot = 1. > > I think DeoptimizeObjectsALot = 2 is more important, but I think it's not bad > to have DeoptimizeObjectsALot = 1 in addition. Ok. > > I never used DeoptimizeObjectsALot = 1 that much. It could be more > deterministic in single threaded > scenarios. I wouldn't object to get rid of it though. > > > src/hotspot/share/runtime/stackValue.hpp > > Better reinitilization in StackValue. Good. > > StackValue::obj_is_scalar_replaced() should not return true after calling > set_obj(). > > > src/hotspot/share/runtime/thread.cpp > > src/hotspot/share/runtime/thread.hpp > > src/hotspot/share/runtime/thread.inline.hpp > > wait_for_object_deoptimization, suspend flag, deferred updates and test > feature to deoptimize objects. > > > In the long term, we want to get rid of suspend flags, so it's not so nice to > introduce a new one. But I agree with G?tz that it should be acceptable as > temporary solution until async handshakes are available (which takes more > time). So I'm ok with your change. > > I'm keen to build the feature on async handshakes when the arive. > > > You can use MutexLocker with Thread*. > > Done. > > > JVMTIDeferredUpdates: I agree with Robin. It'd be nice to move the class > out of thread.hpp. > > Done. > > > src/hotspot/share/runtime/vframe.cpp > > Added support for entry frame to new_vframe. Ok. > > > > src/hotspot/share/runtime/vframe_hp.cpp > > src/hotspot/share/runtime/vframe_hp.hpp > > > I think code()->as_nmethod() in not_global_escape_in_scope() and > arg_escape() should better be under #ifdef ASSERT or inside the assert > statement (no need for code cache walking in product build). > > Done. > > > jvmtiDeferredLocalVariableSet::update_monitors: > > Please add a comment explaining that owner referenced by original info > may be scalar replaced, but it is deoptimized in the vframe. > > Done. > > -----Original Message----- > From: Doerr, Martin > Sent: Donnerstag, 12. M?rz 2020 17:28 > To: Reingruber, Richard ; 'Robbin Ehn' > ; Lindenmaier, Goetz > ; David Holmes ; > Vladimir Kozlov (vladimir.kozlov at oracle.com) > ; serviceability-dev at openjdk.java.net; > hotspot-compiler-dev at openjdk.java.net; hotspot-runtime- > dev at openjdk.java.net > Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better Performance > in the Presence of JVMTI Agents > > Hi Richard, > > > I managed to find time for a (almost) complete review of webrev.4. (I'll > review the tests separately.) > > First of all, the change seems to be in pretty good quality for its significant > complexity. I couldn't find any real bugs. But I'd like to propose minor > improvements. > I'm convinced that it's mature because we did substantial testing. > > I like the new functionality for object deoptimization. It can possibly be > reused for future escape analysis based optimizations. So I appreciate having > it available in the code base. > In addition to that, your change makes the JVMTI implementation better > integrated into the VM. > > > Now to the details: > > > src/hotspot/share/c1/c1_IR.hpp > describe_scope parameters. Ok. > > > src/hotspot/share/ci/ciEnv.cpp > src/hotspot/share/ci/ciEnv.hpp > Fix for JvmtiExport::can_walk_any_space() capability. Ok. > > > src/hotspot/share/code/compiledMethod.cpp > Nice cleanup! > > > src/hotspot/share/code/debugInfoRec.cpp > src/hotspot/share/code/debugInfoRec.hpp > Additional parmeters. (Remark: I think "non_global_escape_in_scope" > would read better than "not_global_escape_in_scope", but your version is > consistent with existing code, so no change request from my side.) Ok. > > > src/hotspot/share/code/nmethod.cpp > Nice cleanup! > > > src/hotspot/share/code/pcDesc.hpp > Additional parameters. Ok. > > > src/hotspot/share/code/scopeDesc.cpp > src/hotspot/share/code/scopeDesc.hpp > Improved implementation + additional parameters. Ok. > > > src/hotspot/share/compiler/compileBroker.cpp > src/hotspot/share/compiler/compileBroker.hpp > Extra thread for DeoptimizeObjectsALot. (Remark: I would have put it into a > follow up change together with the test in order to make this webrev > smaller, but since it is included, I'm reviewing everything at once. Not a big > deal.) Ok. > > > src/hotspot/share/jvmci/jvmciCodeInstaller.cpp > Additional parameters. Ok. > > > src/hotspot/share/opto/c2compiler.cpp > Make do_escape_analysis independent of JVMCI capabilities. Nice! > > > src/hotspot/share/opto/callnode.hpp > Additional fields for MachSafePointNodes. Ok. > > > src/hotspot/share/opto/escape.cpp > Annotation for MachSafePointNodes. Your added functionality looks correct. > But I'd prefer to move the bulky code out of the large function. > I suggest to factor out something like has_not_global_escape and > has_arg_escape. So the code could look like this: > SafePointNode* sfn = sfn_worklist.at(next); > sfn->set_not_global_escape_in_scope(has_not_global_escape(sfn)); > if (sfn->is_CallJava()) { > CallJavaNode* call = sfn->as_CallJava(); > call->set_arg_escape(has_arg_escape(call)); > } > This would also allow us to get rid of the found_..._escape_in_args variables > making the loops better readable. > > It's kind of ugly to use strcmp to recognize uncommon trap, but that seems > to be the way to do it (there are more such places). So it's ok. > > > src/hotspot/share/opto/machnode.hpp > Additional fields for MachSafePointNodes. Ok. > > > src/hotspot/share/opto/macro.cpp > Allow elimination of non-escaping allocations. Ok. > > > src/hotspot/share/opto/matcher.cpp > src/hotspot/share/opto/output.cpp > Copy attribute / pass parameters. Ok. > > > src/hotspot/share/prims/jvmtiCodeBlobEvents.cpp > Nice cleanup! > > > src/hotspot/share/prims/jvmtiEnv.cpp > src/hotspot/share/prims/jvmtiEnvBase.cpp > Escape barriers + deoptimize objects for target thread. Good. > > > src/hotspot/share/prims/jvmtiImpl.cpp > src/hotspot/share/prims/jvmtiImpl.hpp > The sequence is pretty complex: > VM_GetOrSetLocal element initialization executes EscapeBarrier code which > suspends the target thread (extra VM Operation). > VM_GetOrSetLocal::doit_prologue performs object deoptimization (by VM > Thread to prepare VM Operation with frame deoptimization). > VM_GetOrSetLocal destructor implicitly calls EscapeBarrier destructor which > resumes the target thread. > But I don't have any improvement proposal. Performance is probably not a > concern, here. So it's ok. > > VM_GetOrSetLocal::deoptimize_objects deoptimizes the top frame if it has > non-globally escaping objects and other frames if they have arg escaping > ones. Good. > > > src/hotspot/share/prims/jvmtiTagMap.cpp > Escape barriers + deoptimize objects for all threads. Ok. > > > src/hotspot/share/prims/whitebox.cpp > Added WB_IsFrameDeoptimized to API. Ok. > > > src/hotspot/share/runtime/deoptimization.cpp > Object deoptimization. I have more comments and proposals, here. > First of all, handling recursive and waiting locks in relock_objects is tricky, but > looks correct. > Comments are sufficient to understand why things are done as they are > implemented. > > BiasedLocking related parts are complex, but we may get rid of them in the > future (with BiasedLocking removal). > Anyway, looks correct, too. > > Typo in comment: "regularily" => "regularly" > > Deoptimization::fetch_unroll_info_helper is the only place where > _jvmti_deferred_updates get deallocated (except JavaThread destructor). > But I think we always go through it, so I can't see a memory leak or such kind > of issues. > > EscapeBarrier::deoptimize_objects: ResourceMark should use > calling_thread(). > > You can use MutexLocker and MonitorLocker with Thread* to save the > Thread::current() call. > > I'd make set_objs_are_deoptimized static and remove it from the > EscapeBarrier interface because I think it shouldn't be used outside of > EscapeBarrier::deoptimize_objects. > > Typo in comment: "we must only deoptimize" => "we only have to > deoptimize" > > "bool EscapeBarrier::deoptimize_objects(intptr_t* fr_id)" is trivial and > barrier_active() is redundant. Implementation can get moved to hpp file. > > I'll get back to suspend flags, later. > > There are weird cases regarding _self_deoptimization_in_progress. > Assume we have 3 threads A, B and C. A deopts C, B deopts C, C deopts C. C > can set _self_deoptimization_in_progress while A performs the handshake > for suspending C. I think this doesn't lead to errors, but it's probably not > desired. > I think it would be better to use only one "wait" call in > sync_and_suspend_one and sync_and_suspend_all. > > I first thought it'd be better to move ThreadBlockInVM before wait() to > reduce thread state transitions, but that seems to be problematic because > ThreadBlockInVM destructor contains a safepoint check which we shouldn't > do while holding EscapeBarrier_lock. So no change request. > > Change in thred_added: > I think the sequence would be more comprehensive if we waited for > deopt_all_threads in Thread::start and all other places where a new thread > can run into Java code (e.g. JVMTI attach). > Your version makes new threads come up with suspend flag set. That looks > correct, too. Advantage is that you only have to change one place > (thread_added). It'll be interesting to see how it will look like when we use > async handshakes instead of suspend flags. > For now, I'm ok with your version. > > I'd only move MutexLocker ml(EscapeBarrier_lock...) after if (!jt- > >is_hidden_from_external_view()). > > Having 4 different deoptimize_objects functions makes it a little hard to keep > an overview of which one is used for what. > Maybe adding suffixes would help a little bit, but I can also live with what you > have. > Implementation looks correct to me. > > > src/hotspot/share/runtime/deoptimization.hpp > Escape barriers and object deoptimization functions. > Typo in comment: "helt" => "held" > > > src/hotspot/share/runtime/globals.hpp > Addition of develop flag DeoptimizeObjectsALotInterval. Ok. > > > src/hotspot/share/runtime/interfaceSupport.cpp > InterfaceSupport::deoptimizeAllObjects() is only used for > DeoptimizeObjectsALot = 1. > I think DeoptimizeObjectsALot = 2 is more important, but I think it's not bad > to have DeoptimizeObjectsALot = 1 in addition. Ok. > > > src/hotspot/share/runtime/interfaceSupport.inline.hpp > Addition of deoptimizeAllObjects. Ok. > > > src/hotspot/share/runtime/mutexLocker.cpp > src/hotspot/share/runtime/mutexLocker.hpp > Addition of EscapeBarrier_lock. Ok. > > > src/hotspot/share/runtime/objectMonitor.cpp > Make recursion count relock aware. Ok. > > > src/hotspot/share/runtime/stackValue.hpp > Better reinitilization in StackValue. Good. > > > src/hotspot/share/runtime/thread.cpp > src/hotspot/share/runtime/thread.hpp > src/hotspot/share/runtime/thread.inline.hpp > wait_for_object_deoptimization, suspend flag, deferred updates and test > feature to deoptimize objects. > > In the long term, we want to get rid of suspend flags, so it's not so nice to > introduce a new one. But I agree with G?tz that it should be acceptable as > temporary solution until async handshakes are available (which takes more > time). So I'm ok with your change. > > You can use MutexLocker with Thread*. > > JVMTIDeferredUpdates: I agree with Robin. It'd be nice to move the class out > of thread.hpp. > > > src/hotspot/share/runtime/vframe.cpp > Added support for entry frame to new_vframe. Ok. > > > src/hotspot/share/runtime/vframe_hp.cpp > src/hotspot/share/runtime/vframe_hp.hpp > > I think code()->as_nmethod() in not_global_escape_in_scope() and > arg_escape() should better be under #ifdef ASSERT or inside the assert > statement (no need for code cache walking in product build). > > jvmtiDeferredLocalVariableSet::update_monitors: > Please add a comment explaining that owner referenced by original info may > be scalar replaced, but it is deoptimized in the vframe. > > > src/hotspot/share/utilities/macros.hpp > Addition of NOT_COMPILER2_OR_JVMCI_RETURN macros. Ok. > > > test/hotspot/jtreg/serviceability/jvmti/Heap/IterateHeapWithEscapeAnalysi > sEnabled.java > test/hotspot/jtreg/serviceability/jvmti/Heap/libIterateHeapWithEscapeAnal > ysisEnabled.c > New test. Will review separately. > > > test/jdk/TEST.ROOT > Addition of vm.jvmci as required property. Ok. > > > test/jdk/com/sun/jdi/EATests.java > test/jdk/com/sun/jdi/EATestsJVMCI.java > New test. Will review separately. > > > test/lib/sun/hotspot/WhiteBox.java > Added isFrameDeoptimized to API. Ok. > > > That was it. Best regards, > Martin > > > > -----Original Message----- > > From: hotspot-compiler-dev > bounces at openjdk.java.net> On Behalf Of Reingruber, Richard > > Sent: Dienstag, 3. M?rz 2020 21:23 > > To: 'Robbin Ehn' ; Lindenmaier, Goetz > > ; David Holmes > ; > > Vladimir Kozlov (vladimir.kozlov at oracle.com) > > ; serviceability-dev at openjdk.java.net; > > hotspot-compiler-dev at openjdk.java.net; hotspot-runtime- > > dev at openjdk.java.net > > Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better > > Performance in the Presence of JVMTI Agents > > > > Hi Robbin, > > > > > > I understand that Robbin proposed to replace the usage of > > > > _suspend_flag with handshakes. Apparently, async handshakes > > > > are needed to do so. We have been waiting a while for removal > > > > of the _suspend_flag / introduction of async handshakes [2]. > > > > What is the status here? > > > > > I have an old prototype which I would like to continue to work on. > > > So do not assume asynch handshakes will make 15. > > > Even if it would, I think there are a lot more investigate work to remove > > > _suspend_flag. > > > > Let us know, if we can be of any help to you and be it only testing. > > > > > >> Full: > > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.4/ > > > > > DeoptimizeObjectsALotThread is only used in compileBroker.cpp. > > > You can move both declaration and definition to that file, no need to > > clobber > > > thread.[c|h]pp. (and the static function deopt_objs_alot_thread_entry) > > > > Will do. > > > > > Does JvmtiDeferredUpdates really need to be in thread.hpp, can't be in > it's > > own > > > hpp file? It doesn't seem right to add JVM TI classes into thread.hpp. > > > > You are right. It shouldn't be declared in thread.hpp. I will look into that. > > > > > Note that we also think we may have a bug in deopt: > > > https://bugs.openjdk.java.net/browse/JDK-8238237 > > > > > I think it would be best, if possible, to push after that is resolved. > > > > Sure. > > > > > Not even nearly a full review :) > > > > I know :) > > > > Anyways, thanks a lot, > > Richard. > > > > > > -----Original Message----- > > From: Robbin Ehn > > Sent: Monday, March 2, 2020 11:17 AM > > To: Lindenmaier, Goetz ; Reingruber, > Richard > > ; David Holmes > ; > > Vladimir Kozlov (vladimir.kozlov at oracle.com) > > ; serviceability-dev at openjdk.java.net; > > hotspot-compiler-dev at openjdk.java.net; hotspot-runtime- > > dev at openjdk.java.net > > Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better Performance > > in the Presence of JVMTI Agents > > > > Hi, > > > > On 2/24/20 5:39 PM, Lindenmaier, Goetz wrote: > > > Hi, > > > > > > I had a look at the progress of this change. Nothing > > > happened since Richard posted his update using more > > > handshakes [1]. > > > But we (SAP) would appreciate a lot if this change could > > > be successfully reviewed and pushed. > > > > > > I think there is basic understanding that this > > > change is helpful. It fixes a number of issues with JVMTI, > > > and will deliver the same performance benefits as EA > > > does in current production mode for debugging scenarios. > > > > > > This is important for us as we run our VMs prepared > > > for debugging in production mode. > > > > > > I understand that Robbin proposed to replace the usage of > > > _suspend_flag with handshakes. Apparently, async handshakes > > > are needed to do so. We have been waiting a while for removal > > > of the _suspend_flag / introduction of async handshakes [2]. > > > What is the status here? > > > > I have an old prototype which I would like to continue to work on. > > So do not assume asynch handshakes will make 15. > > Even if it would, I think there are a lot more investigate work to remove > > _suspend_flag. > > > > > > > > I think we should no longer wait, but proceed with > > > this change. We will look into removing the usage of > > > suspend_flag introduced here once it is possible to implement > > > it with handshakes. > > > > Yes, sure. > > > > >> Full: > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.4/ > > > > DeoptimizeObjectsALotThread is only used in compileBroker.cpp. > > You can move both declaration and definition to that file, no need to > clobber > > thread.[c|h]pp. (and the static function deopt_objs_alot_thread_entry) > > > > Does JvmtiDeferredUpdates really need to be in thread.hpp, can't be in it's > > own > > hpp file? It doesn't seem right to add JVM TI classes into thread.hpp. > > > > Note that we also think we may have a bug in deopt: > > https://bugs.openjdk.java.net/browse/JDK-8238237 > > > > I think it would be best, if possible, to push after that is resolved. > > > > Not even nearly a full review :) > > > > Thanks, Robbin > > > > > > >> Incremental: > > >> > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.4.inc/ > > >> > > >> I was not able to eliminate the additional suspend flag now. I'll take care > > of this > > >> as soon as the > > >> existing suspend-resume-mechanism is reworked. > > >> > > >> Testing: > > >> > > >> Nightly tests @SAP: > > >> > > >> JCK and JTREG, also in Xcomp mode, SPECjvm2008, SPECjbb2015, > > Renaissance > > >> Suite, SAP specific tests > > >> with fastdebug and release builds on all platforms > > >> > > >> Stress testing with DeoptimizeObjectsALot running SPECjvm2008 40x > > parallel > > >> for 24h > > >> > > >> Thanks, Richard. > > >> > > >> > > >> More details on the changes: > > >> > > >> * Hide DeoptimizeObjectsALotThread from external view. > > >> > > >> * Changed EscapeBarrier_lock to be a _safepoint_check_never lock. > > >> It used to be _safepoint_check_sometimes, which will be eliminated > > sooner or > > >> later. > > >> I added explicit thread state changes with ThreadBlockInVM to code > > paths > > >> where we can wait() > > >> on EscapeBarrier_lock to become safepoint safe. > > >> > > >> * Use handshake EscapeBarrierSuspendHandshake to suspend target > > threads > > >> instead of vm operation > > >> VM_ThreadSuspendAllForObjDeopt. > > >> > > >> * Removed uses of Threads_lock. When adding a new thread we > suspend > > it iff > > >> EA optimizations are > > >> being reverted. In the previous version we were waiting on > > Threads_lock > > >> while EA optimizations > > >> were reverted. See EscapeBarrier::thread_added(). > > >> > > >> * Made tests require Xmixed compilation mode. > > >> > > >> * Made tests agnostic regarding tiered compilation. > > >> I.e. tc isn't disabled anymore, and the tests can be run with tc enabled > or > > >> disabled. > > >> > > >> * Exercising EATests.java as well with stress test options > > >> DeoptimizeObjectsALot* > > >> Due to the non-deterministic deoptimizations some tests need to be > > skipped. > > >> We do this to prevent bit-rot of the stress test code. > > >> > > >> * Executing EATests.java as well with graal if available. Driver for this is > > >> EATestsJVMCI.java. Graal cannot pass all tests, because it does not > > provide all > > >> the new debug info > > >> (namely not_global_escape_in_scope and arg_escape in > > scopeDesc.hpp). > > >> And graal does not yet support the JVMTI operations force early > return > > and > > >> pop frame. > > >> > > >> * Removed tracing from new jdi tests in EATests.java. Too much trace > > output > > >> before the debugging > > >> connection is established can cause deadlock because output buffers > fill > > up. > > >> (See https://bugs.openjdk.java.net/browse/JDK-8173304) > > >> > > >> * Many copyright year changes and smaller clean-up changes of testing > > code > > >> (trailing white-space and > > >> the like). > > >> > > >> > > >> -----Original Message----- > > >> From: David Holmes > > >> Sent: Donnerstag, 19. Dezember 2019 03:12 > > >> To: Reingruber, Richard ; serviceability- > > >> dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; > > hotspot- > > >> runtime-dev at openjdk.java.net; Vladimir Kozlov > > (vladimir.kozlov at oracle.com) > > >> > > >> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better > > Performance in > > >> the Presence of JVMTI Agents > > >> > > >> Hi Richard, > > >> > > >> I think my issue is with the way EliminateNestedLocks works so I'm going > > >> to look into that more deeply. > > >> > > >> Thanks for the explanations. > > >> > > >> David > > >> > > >> On 18/12/2019 12:47 am, Reingruber, Richard wrote: > > >>> Hi David, > > >>> > > >>> > > > Some further queries/concerns: > > >>> > > > > > >>> > > > src/hotspot/share/runtime/objectMonitor.cpp > > >>> > > > > > >>> > > > Can you please explain the changes to ObjectMonitor::wait: > > >>> > > > > > >>> > > > ! _recursions = save // restore the old recursion count > > >>> > > > ! + jt->get_and_reset_relock_count_after_wait(); // > > >>> > > > increased by the deferred relock count > > >>> > > > > > >>> > > > what is the "deferred relock count"? I gather it relates to > > >>> > > > > > >>> > > > "The code was extended to be able to deoptimize objects of a > > >>> > > frame that > > >>> > > > is not the top frame and to let another thread than the > owning > > >>> > > thread do > > >>> > > > it." > > >>> > > > > >>> > > Yes, these relate. Currently EA based optimizations are reverted, > > when a > > >> compiled frame is > > >>> > > replaced with corresponding interpreter frames. Part of this is > > relocking > > >> objects with eliminated > > >>> > > locking. New with the enhancement is that we do this also just > > before > > >> object references are > > >>> > > acquired through JVMTI. In this case we deoptimize also the > > owning > > >> compiled frame C and we > > >>> > > register deoptimized objects as deferred updates. When control > > returns > > >> to C it gets deoptimized, > > >>> > > we notice that objects are already deoptimized (reallocated and > > >> relocked), so we don't do it again > > >>> > > (relocking twice would be incorrect of course). Deferred updates > > are > > >> copied into the new > > >>> > > interpreter frames. > > >>> > > > > >>> > > Problem: relocking is not possible if the target thread T is waiting > > on the > > >> monitor that needs to > > >>> > > be relocked. This happens only with non-local objects with > > >> EliminateNestedLocks. Instead relocking > > >>> > > is deferred until T owns the monitor again. This is what the piece > of > > >> code above does. > > >>> > > > >>> > Sorry I need some more detail here. How can you wait() on an > > object > > >>> > monitor if the object allocation and/or locking was optimised > away? > > And > > >>> > what is a "non-local object" in this context? Isn't EA restricted to > > >>> > thread-confined objects? > > >>> > > >>> "Non-local object" is an object that escapes its thread. The issue I'm > > >> addressing with the changes > > >>> in ObjectMonitor::wait are almost unrelated to EA. They are caused by > > >> EliminateNestedLocks, where C2 > > >>> eliminates recursive locking of an already owned lock. The lock owning > > object > > >> exists on the heap, it > > >>> is locked and you can call wait() on it. > > >>> > > >>> EliminateLocks is the C2 option that controls lock elimination based on > > EA. > > >> Both optimizations have > > >>> in common that objects with eliminated locking need to be relocked > > when > > >> deoptimizing a frame, > > >>> i.e. when replacing a compiled frame with equivalent interpreter > > >>> frames. Deoptimization::relock_objects does that job for /all/ > eliminated > > >> locks in scope. /All/ can > > >>> be a mix of eliminated nested locks and locks of not-escaping objects. > > >>> > > >>> New with the enhancement: I call relock_objects earlier, just before > > objects > > >> pontentially > > >>> escape. But then later when the owning compiled frame gets > > deoptimized, I > > >> must not do it again: > > >>> > > >>> See call to EscapeBarrier::objs_are_deoptimized in > deoptimization.cpp: > > >>> > > >>> 373 if ((jvmci_enabled || ((DoEscapeAnalysis || > > EliminateNestedLocks) && > > >> EliminateLocks)) > > >>> 374 && !EscapeBarrier::objs_are_deoptimized(thread, > > deoptee.id())) { > > >>> 375 bool unused; > > >>> 376 eliminate_locks(thread, chunk, realloc_failures, deoptee, > > exec_mode, > > >> unused); > > >>> 377 } > > >>> > > >>> Now when calling relock_objects early it is quiet possible that I have to > > relock > > >> an object the > > >>> target thread currently waits for. Obviously I cannot relock in this case, > > >> instead I chose to > > >>> introduce relock_count_after_wait to JavaThread. > > >>> > > >>> > Is it just that some of the locking gets optimized away e.g. > > >>> > > > >>> > synchronised(obj) { > > >>> > synchronised(obj) { > > >>> > synchronised(obj) { > > >>> > obj.wait(); > > >>> > } > > >>> > } > > >>> > } > > >>> > > > >>> > If this is reduced to a form as-if it were a single lock of the monitor > > >>> > (due to EA) and the wait() triggers a JVM TI event which leads to > the > > >>> > escape of "obj" then we need to reconstruct the true lock state, > and > > so > > >>> > when the wait() internally unblocks and reacquires the monitor it > > has to > > >>> > set the true recursion count to 3, not the 1 that it appeared to be > > when > > >>> > wait() was initially called. Is that the scenario? > > >>> > > >>> Kind of... except that the locking is not eliminated due to EA and there > is > > no > > >> JVM TI event > > >>> triggered by wait. > > >>> > > >>> Add > > >>> > > >>> LocalObject l1 = new LocalObject(); > > >>> > > >>> in front of the synchrnized blocks and assume a JVM TI agent acquires > l1. > > This > > >> triggers the code in > > >>> question. > > >>> > > >>> See that relocking/reallocating is transactional. If it is done then for > /all/ > > >> objects in scope and it is > > >>> done at most once. It wouldn't be quite so easy to split this in relocking > > of > > >> nested/EA-based > > >>> eliminated locks. > > >>> > > >>> > If so I find this truly awful. Anyone using wait() in a realistic form > > >>> > requires a notification and so the object cannot be thread > confined. > > In > > >>> > > >>> It is not thread confined. > > >>> > > >>> > which case I would strongly argue that upon hitting the wait() the > > deopt > > >>> > should occur unconditionally and so the lock state is correct before > > we > > >>> > wait and so we don't need to mess with the recursion count > > internally > > >>> > when we reacquire the monitor. > > >>> > > > >>> > > > > >>> > > > which I don't like the sound of at all when it comes to > > ObjectMonitor > > >>> > > > state. So I'd like to understand in detail exactly what is going > on > > here > > >>> > > > and why. This is a very intrusive change that seems to badly > > break > > >>> > > > encapsulation and impacts future changes to ObjectMonitor > > that are > > >> under > > >>> > > > investigation. > > >>> > > > > >>> > > I would not regard this as breaking encapsulation. Certainly not > > badly. > > >>> > > > > >>> > > I've added a property relock_count_after_wait to JavaThread. > The > > >> property is well > > >>> > > encapsulated. Future ObjectMonitor implementations have to > deal > > with > > >> recursion too. They are free > > >>> > > in choosing a way to do that as long as that property is taken into > > >> account. This is hardly a > > >>> > > limitation. > > >>> > > > >>> > I do think this badly breaks encapsulation as you have to add a > > callout > > >>> > from the guts of the ObjectMonitor code to reach into the thread > to > > get > > >>> > this lock count adjustment. I understand why you have had to do > > this but > > >>> > I would much rather see a change to the EA optimisation strategy > so > > that > > >>> > this is not needed. > > >>> > > > >>> > > Note also that the property is a straight forward extension of the > > >> existing concept of deferred > > >>> > > local updates. It is embedded into the structure holding them. So > > not > > >> even the footprint of a > > >>> > > JavaThread is enlarged if no deferred updates are generated. > > >>> > > > >>> > [...] > > >>> > > > >>> > > > > >>> > > I'm actually duplicating the existing external suspend mechanism, > > >> because a thread can be > > >>> > > suspended at most once. And hey, and don't like that either! But > it > > >> seems not unlikely that the > > >>> > > duplicate can be removed together with the original and the new > > type > > >> of handshakes that will be > > >>> > > used for thread suspend can be used for object deoptimization > > too. See > > >> today's discussion in > > >>> > > JDK-8227745 [2]. > > >>> > > > >>> > I hope that discussion bears some fruit, at the moment it seems > not > > to > > >>> > be possible to use handshakes here. :( > > >>> > > > >>> > The external suspend mechanism is a royal pain in the proverbial > > that we > > >>> > have to carefully live with. The idea that we're duplicating that for > > >>> > use in another fringe area of functionality does not thrill me at all. > > >>> > > > >>> > To be clear, I understand the problem that exists and that you > wish > > to > > >>> > solve, but for the runtime parts I balk at the complexity cost of > > >>> > solving it. > > >>> > > >>> I know it's complex, but by far no rocket science. > > >>> > > >>> Also I find it hard to imagine another fix for JDK-8233915 besides > > changing > > >> the JVM TI specification. > > >>> > > >>> Thanks, Richard. > > >>> > > >>> -----Original Message----- > > >>> From: David Holmes > > >>> Sent: Dienstag, 17. Dezember 2019 08:03 > > >>> To: Reingruber, Richard ; serviceability- > > >> dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; > > hotspot- > > >> runtime-dev at openjdk.java.net; Vladimir Kozlov > > (vladimir.kozlov at oracle.com) > > >> > > >>> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better > > Performance > > >> in the Presence of JVMTI Agents > > >>> > > >>> > > >>> > > >>> David > > >>> > > >>> On 17/12/2019 4:57 pm, David Holmes wrote: > > >>>> Hi Richard, > > >>>> > > >>>> On 14/12/2019 5:01 am, Reingruber, Richard wrote: > > >>>>> Hi David, > > >>>>> > > >>>>> ?? > Some further queries/concerns: > > >>>>> ?? > > > >>>>> ?? > src/hotspot/share/runtime/objectMonitor.cpp > > >>>>> ?? > > > >>>>> ?? > Can you please explain the changes to ObjectMonitor::wait: > > >>>>> ?? > > > >>>>> ?? > !?? _recursions = save????? // restore the old recursion count > > >>>>> ?? > !???????????????? + jt->get_and_reset_relock_count_after_wait(); // > > >>>>> ?? > increased by the deferred relock count > > >>>>> ?? > > > >>>>> ?? > what is the "deferred relock count"? I gather it relates to > > >>>>> ?? > > > >>>>> ?? > "The code was extended to be able to deoptimize objects of a > > >>>>> frame that > > >>>>> ?? > is not the top frame and to let another thread than the owning > > >>>>> thread do > > >>>>> ?? > it." > > >>>>> > > >>>>> Yes, these relate. Currently EA based optimizations are reverted, > > when > > >>>>> a compiled frame is replaced > > >>>>> with corresponding interpreter frames. Part of this is relocking > > >>>>> objects with eliminated > > >>>>> locking. New with the enhancement is that we do this also just > before > > >>>>> object references are acquired > > >>>>> through JVMTI. In this case we deoptimize also the owning compiled > > >>>>> frame C and we register > > >>>>> deoptimized objects as deferred updates. When control returns to > C > > it > > >>>>> gets deoptimized, we notice > > >>>>> that objects are already deoptimized (reallocated and relocked), so > > we > > >>>>> don't do it again (relocking > > >>>>> twice would be incorrect of course). Deferred updates are copied > into > > >>>>> the new interpreter frames. > > >>>>> > > >>>>> Problem: relocking is not possible if the target thread T is waiting > > >>>>> on the monitor that needs to be > > >>>>> relocked. This happens only with non-local objects with > > >>>>> EliminateNestedLocks. Instead relocking is > > >>>>> deferred until T owns the monitor again. This is what the piece of > > >>>>> code above does. > > >>>> > > >>>> Sorry I need some more detail here. How can you wait() on an object > > >>>> monitor if the object allocation and/or locking was optimised away? > > And > > >>>> what is a "non-local object" in this context? Isn't EA restricted to > > >>>> thread-confined objects? > > >>>> > > >>>> Is it just that some of the locking gets optimized away e.g. > > >>>> > > >>>> synchronised(obj) { > > >>>> ? synchronised(obj) { > > >>>> ??? synchronised(obj) { > > >>>> ????? obj.wait(); > > >>>> ??? } > > >>>> ? } > > >>>> } > > >>>> > > >>>> If this is reduced to a form as-if it were a single lock of the monitor > > >>>> (due to EA) and the wait() triggers a JVM TI event which leads to the > > >>>> escape of "obj" then we need to reconstruct the true lock state, and > so > > >>>> when the wait() internally unblocks and reacquires the monitor it has > to > > >>>> set the true recursion count to 3, not the 1 that it appeared to be > when > > >>>> wait() was initially called. Is that the scenario? > > >>>> > > >>>> If so I find this truly awful. Anyone using wait() in a realistic form > > >>>> requires a notification and so the object cannot be thread confined. > In > > >>>> which case I would strongly argue that upon hitting the wait() the > > deopt > > >>>> should occur unconditionally and so the lock state is correct before > we > > >>>> wait and so we don't need to mess with the recursion count internally > > >>>> when we reacquire the monitor. > > >>>> > > >>>>> > > >>>>> ?? > which I don't like the sound of at all when it comes to > > >>>>> ObjectMonitor > > >>>>> ?? > state. So I'd like to understand in detail exactly what is going > > >>>>> on here > > >>>>> ?? > and why.? This is a very intrusive change that seems to badly > > break > > >>>>> ?? > encapsulation and impacts future changes to ObjectMonitor > that > > >>>>> are under > > >>>>> ?? > investigation. > > >>>>> > > >>>>> I would not regard this as breaking encapsulation. Certainly not > badly. > > >>>>> > > >>>>> I've added a property relock_count_after_wait to JavaThread. The > > >>>>> property is well > > >>>>> encapsulated. Future ObjectMonitor implementations have to deal > > with > > >>>>> recursion too. They are free in > > >>>>> choosing a way to do that as long as that property is taken into > > >>>>> account. This is hardly a > > >>>>> limitation. > > >>>> > > >>>> I do think this badly breaks encapsulation as you have to add a callout > > >>>> from the guts of the ObjectMonitor code to reach into the thread to > > get > > >>>> this lock count adjustment. I understand why you have had to do this > > but > > >>>> I would much rather see a change to the EA optimisation strategy so > > that > > >>>> this is not needed. > > >>>> > > >>>>> Note also that the property is a straight forward extension of the > > >>>>> existing concept of deferred > > >>>>> local updates. It is embedded into the structure holding them. So > not > > >>>>> even the footprint of a > > >>>>> JavaThread is enlarged if no deferred updates are generated. > > >>>>> > > >>>>> ?? > --- > > >>>>> ?? > > > >>>>> ?? > src/hotspot/share/runtime/thread.cpp > > >>>>> ?? > > > >>>>> ?? > Can you please explain why > > >>>>> JavaThread::wait_for_object_deoptimization > > >>>>> ?? > has to be handcrafted in this way rather than using proper > > >>>>> transitions. > > >>>>> ?? > > > >>>>> > > >>>>> I wrote wait_for_object_deoptimization taking > > >>>>> JavaThread::java_suspend_self_with_safepoint_check > > >>>>> as template. So in short: for the same reasons :) > > >>>>> > > >>>>> Threads reach both methods as part of thread state transitions, > > >>>>> therefore special handling is > > >>>>> required to change thread state on top of ongoing transitions. > > >>>>> > > >>>>> ?? > We got rid of "deopt suspend" some time ago and it is > disturbing > > >>>>> to see > > >>>>> ?? > it being added back (effectively). This seems like it may be > > >>>>> something > > >>>>> ?? > that handshakes could be used for. > > >>>>> > > >>>>> Deopt suspend used to be something rather different with a similar > > >>>>> name[1]. It is not being added back. > > >>>> > > >>>> I stand corrected. Despite comments in the code to the contrary > > >>>> deopt_suspend didn't actually cause a self-suspend. I was doing a lot > of > > >>>> cleanup in this area 13 years ago :) > > >>>> > > >>>>> > > >>>>> I'm actually duplicating the existing external suspend mechanism, > > >>>>> because a thread can be suspended > > >>>>> at most once. And hey, and don't like that either! But it seems not > > >>>>> unlikely that the duplicate can > > >>>>> be removed together with the original and the new type of > > handshakes > > >>>>> that will be used for > > >>>>> thread suspend can be used for object deoptimization too. See > > today's > > >>>>> discussion in JDK-8227745 [2]. > > >>>> > > >>>> I hope that discussion bears some fruit, at the moment it seems not > to > > >>>> be possible to use handshakes here. :( > > >>>> > > >>>> The external suspend mechanism is a royal pain in the proverbial that > > we > > >>>> have to carefully live with. The idea that we're duplicating that for > > >>>> use in another fringe area of functionality does not thrill me at all. > > >>>> > > >>>> To be clear, I understand the problem that exists and that you wish to > > >>>> solve, but for the runtime parts I balk at the complexity cost of > > >>>> solving it. > > >>>> > > >>>> Thanks, > > >>>> David > > >>>> ----- > > >>>> > > >>>>> Thanks, Richard. > > >>>>> > > >>>>> [1] Deopt suspend was something like an async. handshake for > > >>>>> architectures with register windows, > > >>>>> ???? where patching the return pc for deoptimization of a compiled > > >>>>> frame was racy if the owner thread > > >>>>> ???? was in native code. Instead a "deopt" suspend flag was set on > > >>>>> which the thread patched its own > > >>>>> ???? frame upon return from native. So no thread was suspended. It > > got > > >>>>> its name only from the name of > > >>>>> ???? the flags. > > >>>>> > > >>>>> [2] Discussion about using handshakes to sync. with the target > thread: > > >>>>> > > >>>>> https://bugs.openjdk.java.net/browse/JDK- > > >> > > > 8227745?focusedCommentId=14306727&page=com.atlassian.jira.plugin.syst > > e > > >> m.issuetabpanels:comment-tabpanel#comment-14306727 > > >>>>> > > >>>>> > > >>>>> -----Original Message----- > > >>>>> From: David Holmes > > >>>>> Sent: Freitag, 13. Dezember 2019 00:56 > > >>>>> To: Reingruber, Richard ; > > >>>>> serviceability-dev at openjdk.java.net; > > >>>>> hotspot-compiler-dev at openjdk.java.net; > > >>>>> hotspot-runtime-dev at openjdk.java.net > > >>>>> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better > > >>>>> Performance in the Presence of JVMTI Agents > > >>>>> > > >>>>> Hi Richard, > > >>>>> > > >>>>> Some further queries/concerns: > > >>>>> > > >>>>> src/hotspot/share/runtime/objectMonitor.cpp > > >>>>> > > >>>>> Can you please explain the changes to ObjectMonitor::wait: > > >>>>> > > >>>>> !?? _recursions = save????? // restore the old recursion count > > >>>>> !???????????????? + jt->get_and_reset_relock_count_after_wait(); // > > >>>>> increased by the deferred relock count > > >>>>> > > >>>>> what is the "deferred relock count"? I gather it relates to > > >>>>> > > >>>>> "The code was extended to be able to deoptimize objects of a > frame > > that > > >>>>> is not the top frame and to let another thread than the owning > thread > > do > > >>>>> it." > > >>>>> > > >>>>> which I don't like the sound of at all when it comes to ObjectMonitor > > >>>>> state. So I'd like to understand in detail exactly what is going on here > > >>>>> and why.? This is a very intrusive change that seems to badly break > > >>>>> encapsulation and impacts future changes to ObjectMonitor that > are > > under > > >>>>> investigation. > > >>>>> > > >>>>> --- > > >>>>> > > >>>>> src/hotspot/share/runtime/thread.cpp > > >>>>> > > >>>>> Can you please explain why > > JavaThread::wait_for_object_deoptimization > > >>>>> has to be handcrafted in this way rather than using proper > transitions. > > >>>>> > > >>>>> We got rid of "deopt suspend" some time ago and it is disturbing to > > see > > >>>>> it being added back (effectively). This seems like it may be > something > > >>>>> that handshakes could be used for. > > >>>>> > > >>>>> Thanks, > > >>>>> David > > >>>>> ----- > > >>>>> > > >>>>> On 12/12/2019 7:02 am, David Holmes wrote: > > >>>>>> On 12/12/2019 1:07 am, Reingruber, Richard wrote: > > >>>>>>> Hi David, > > >>>>>>> > > >>>>>>> ??? > Most of the details here are in areas I can comment on in > > detail, > > >>>>>>> but I > > >>>>>>> ??? > did take an initial general look at things. > > >>>>>>> > > >>>>>>> Thanks for taking the time! > > >>>>>> > > >>>>>> Apologies the above should read: > > >>>>>> > > >>>>>> "Most of the details here are in areas I *can't* comment on in > detail > > >>>>>> ..." > > >>>>>> > > >>>>>> David > > >>>>>> > > >>>>>>> ??? > The only thing that jumped out at me is that I think the > > >>>>>>> ??? > DeoptimizeObjectsALotThread should be a hidden thread. > > >>>>>>> ??? > > > >>>>>>> ??? > +? bool is_hidden_from_external_view() const { return true; > } > > >>>>>>> > > >>>>>>> Yes, it should. Will add the method like above. > > >>>>>>> > > >>>>>>> ??? > Also I don't see any testing of the > > DeoptimizeObjectsALotThread. > > >>>>>>> Without > > >>>>>>> ??? > active testing this will just bit-rot. > > >>>>>>> > > >>>>>>> DeoptimizeObjectsALot is meant for stress testing with a larger > > >>>>>>> workload. I will add a minimal test > > >>>>>>> to keep it fresh. > > >>>>>>> > > >>>>>>> ??? > Also on the tests I don't understand your @requires clause: > > >>>>>>> ??? > > > >>>>>>> ??? >?? @requires ((vm.compMode != "Xcomp") & > > vm.compiler2.enabled > > >> & > > >>>>>>> ??? > (vm.opt.TieredCompilation != true)) > > >>>>>>> ??? > > > >>>>>>> ??? > This seems to require that TieredCompilation is disabled, but > > >>>>>>> tiered is > > >>>>>>> ??? > our normal mode of operation. ?? > > >>>>>>> ??? > > > >>>>>>> > > >>>>>>> I removed the clause. I guess I wanted to target the tests towards > > the > > >>>>>>> code they are supposed to > > >>>>>>> test, and it's easier to analyze failures w/o tiered compilation and > > >>>>>>> with just one compiler thread. > > >>>>>>> > > >>>>>>> Additionally I will make use of > > >>>>>>> compiler.whitebox.CompilerWhiteBoxTest.THRESHOLD in the > tests. > > >>>>>>> > > >>>>>>> Thanks, > > >>>>>>> Richard. > > >>>>>>> > > >>>>>>> -----Original Message----- > > >>>>>>> From: David Holmes > > >>>>>>> Sent: Mittwoch, 11. Dezember 2019 08:03 > > >>>>>>> To: Reingruber, Richard ; > > >>>>>>> serviceability-dev at openjdk.java.net; > > >>>>>>> hotspot-compiler-dev at openjdk.java.net; > > >>>>>>> hotspot-runtime-dev at openjdk.java.net > > >>>>>>> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better > > >>>>>>> Performance in the Presence of JVMTI Agents > > >>>>>>> > > >>>>>>> Hi Richard, > > >>>>>>> > > >>>>>>> On 11/12/2019 7:45 am, Reingruber, Richard wrote: > > >>>>>>>> Hi, > > >>>>>>>> > > >>>>>>>> I would like to get reviews please for > > >>>>>>>> > > >>>>>>>> > > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.3/ > > >>>>>>>> > > >>>>>>>> Corresponding RFE: > > >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8227745 > > >>>>>>>> > > >>>>>>>> Fixes also https://bugs.openjdk.java.net/browse/JDK-8233915 > > >>>>>>>> And potentially https://bugs.openjdk.java.net/browse/JDK- > > 8214584 [1] > > >>>>>>>> > > >>>>>>>> Vladimir Kozlov kindly put webrev.3 through tier1-8 testing > > without > > >>>>>>>> issues (thanks!). In addition the > > >>>>>>>> change is being tested at SAP since I posted the first RFR some > > >>>>>>>> months ago. > > >>>>>>>> > > >>>>>>>> The intention of this enhancement is to benefit performance > wise > > from > > >>>>>>>> escape analysis even if JVMTI > > >>>>>>>> agents request capabilities that allow them to access local > variable > > >>>>>>>> values. E.g. if you start-up > > >>>>>>>> with -agentlib:jdwp=transport=dt_socket,server=y,suspend=n, > > then > > >>>>>>>> escape analysis is disabled right > > >>>>>>>> from the beginning, well before a debugger attaches -- if ever > one > > >>>>>>>> should do so. With the > > >>>>>>>> enhancement, escape analysis will remain enabled until and > after > > a > > >>>>>>>> debugger attaches. EA based > > >>>>>>>> optimizations are reverted just before an agent acquires the > > >>>>>>>> reference to an object. In the JBS item > > >>>>>>>> you'll find more details. > > >>>>>>> > > >>>>>>> Most of the details here are in areas I can comment on in detail, > but > > I > > >>>>>>> did take an initial general look at things. > > >>>>>>> > > >>>>>>> The only thing that jumped out at me is that I think the > > >>>>>>> DeoptimizeObjectsALotThread should be a hidden thread. > > >>>>>>> > > >>>>>>> +? bool is_hidden_from_external_view() const { return true; } > > >>>>>>> > > >>>>>>> Also I don't see any testing of the DeoptimizeObjectsALotThread. > > >>>>>>> Without > > >>>>>>> active testing this will just bit-rot. > > >>>>>>> > > >>>>>>> Also on the tests I don't understand your @requires clause: > > >>>>>>> > > >>>>>>> ??? @requires ((vm.compMode != "Xcomp") & > > vm.compiler2.enabled & > > >>>>>>> (vm.opt.TieredCompilation != true)) > > >>>>>>> > > >>>>>>> This seems to require that TieredCompilation is disabled, but > tiered > > is > > >>>>>>> our normal mode of operation. ?? > > >>>>>>> > > >>>>>>> Thanks, > > >>>>>>> David > > >>>>>>> > > >>>>>>>> Thanks, > > >>>>>>>> Richard. > > >>>>>>>> > > >>>>>>>> [1] Experimental fix for JDK-8214584 based on JDK-8227745 > > >>>>>>>> > > >> > > > http://cr.openjdk.java.net/~rrich/webrevs/2019/8214584/experiment_v1.pa > > tc > > >> h > > >>>>>>>> > > >>>>>>>> > > >>>>>>>> From daniel.daugherty at oracle.com Tue Mar 31 14:09:01 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 31 Mar 2020 10:09:01 -0400 Subject: RFR: 8241837: Cleanup stringStream usage in ObjectSynchronizer In-Reply-To: <4873f9b5-d9ec-242a-fed5-46a3b7905b1a@oracle.com> References: <4873f9b5-d9ec-242a-fed5-46a3b7905b1a@oracle.com> Message-ID: <22ad475a-52a7-9778-42a7-6d184c0c78ac@oracle.com> On 3/30/20 7:02 PM, Claes Redestad wrote: > Hi, > > this removes one unused stringStream, and refactors two other > so that they're only defined in the slow path error where they are used: > > http://cr.openjdk.java.net/~redestad/8241837/open.00/ src/hotspot/share/runtime/synchronizer.cpp ??? No comments. Just FYI: ObjectMonitor::is_busy_to_string() calls stringStream::base() so the cleanup is handled by the stringStream destructor. I didn't want ResourceMarks. Thumbs up. Dan > > While a very small startup performance gain (removes ~50k instructions > on a Hello World), these stringStreams mallocs on init, which could be > problematic at times in this particular area. > > Thanks! > > /Claes From robbin.ehn at oracle.com Tue Mar 31 14:20:41 2020 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 31 Mar 2020 16:20:41 +0200 Subject: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents In-Reply-To: References: <1f8a3c7a-fa0f-b5b2-4a8a-7d3d8dbbe1b5@oracle.com> <4b56a45c-a14c-6f74-2bfd-25deaabe8201@oracle.com> <5271429a-481d-ddb9-99dc-b3f6670fcc0b@oracle.com> Message-ID: <0a07f87e-ede1-edbd-c754-e7df884e0545@oracle.com> Thanks for cleaning up thread.hpp! /Robbin On 2020-03-30 10:31, Reingruber, Richard wrote: > Hi, > > this is webrev.5 based on Robbin's feedback and Martin's review - thanks! :) > > The change affects jvmti, hotspot and c2. Partial reviews are very welcome too. > > Full: http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.5/ > Delta: http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.5.inc/ > > Robbin, Martin, please let me know, if anything shouldn't be quite as you wanted it. Also find my > comments on your feedback below. > > Robbin, can I count you as Reviewer for the runtime part? > > Thanks, Richard. > > -- > >> DeoptimizeObjectsALotThread is only used in compileBroker.cpp. >> You can move both declaration and definition to that file, no need to clobber >> thread.[c|h]pp. (and the static function deopt_objs_alot_thread_entry) > > Done. > >> Does JvmtiDeferredUpdates really need to be in thread.hpp, can't be in it's own >> hpp file? It doesn't seem right to add JVM TI classes into thread.hpp. > > I moved JvmtiDeferredUpdates to vframe_hp.hpp where preexisting jvmtiDeferredLocalVariableSet is > declared. > >> src/hotspot/share/code/compiledMethod.cpp >> Nice cleanup! > > Thanks :) > >> src/hotspot/share/code/debugInfoRec.cpp >> src/hotspot/share/code/debugInfoRec.hpp >> Additional parmeters. (Remark: I think "non_global_escape_in_scope" would read better than "not_global_escape_in_scope", but your version is consistent with existing code, so no change request from my side.) Ok. > > I've been thinking about this too and finally stayed with not_global_escape_in_scope. It's supposed > to mean an object whose escape state is not GlobalEscape is in scope. > >> src/hotspot/share/compiler/compileBroker.cpp >> src/hotspot/share/compiler/compileBroker.hpp >> Extra thread for DeoptimizeObjectsALot. (Remark: I would have put it into a follow up change together with the test in order to make this webrev smaller, but since it is included, I'm reviewing everything at once. Not a big deal.) Ok. > > Yes the change would be a little smaller. And if it helps I'll split it off. In general I prefer > patches that bring along a suitable amount of tests. > >> src/hotspot/share/opto/c2compiler.cpp >> Make do_escape_analysis independent of JVMCI capabilities. Nice! > > It is the main goal of the enhancement. It is done for C2, but could be done for JVMCI compilers > with just a small effort as well. > >> src/hotspot/share/opto/escape.cpp >> Annotation for MachSafePointNodes. Your added functionality looks correct. >> But I'd prefer to move the bulky code out of the large function. >> I suggest to factor out something like has_not_global_escape and has_arg_escape. So the code could look like this: >> SafePointNode* sfn = sfn_worklist.at(next); >> sfn->set_not_global_escape_in_scope(has_not_global_escape(sfn)); >> if (sfn->is_CallJava()) { >> CallJavaNode* call = sfn->as_CallJava(); >> call->set_arg_escape(has_arg_escape(call)); >> } >> This would also allow us to get rid of the found_..._escape_in_args variables making the loops better readable. > > Done. > >> It's kind of ugly to use strcmp to recognize uncommon trap, but that seems to be the way to do it (there are more such places). So it's ok. > > Yeah. I copied the snippet. > >> src/hotspot/share/prims/jvmtiImpl.cpp >> src/hotspot/share/prims/jvmtiImpl.hpp >> The sequence is pretty complex: >> VM_GetOrSetLocal element initialization executes EscapeBarrier code which suspends the target thread (extra VM Operation). > > Note that the target threads have to be suspended already for VM_GetOrSetLocal*. So it's mainly the > synchronization effect of EscapeBarrier::sync_and_suspend_one() that is required here. Also no extra > _handshake_ is executed, since sync_and_suspend_one() will find the target threads already > suspended. > >> VM_GetOrSetLocal::doit_prologue performs object deoptimization (by VM Thread to prepare VM Operation with frame deoptimization). >> VM_GetOrSetLocal destructor implicitly calls EscapeBarrier destructor which resumes the target thread. >> But I don't have any improvement proposal. Performance is probably not a concern, here. So it's ok. > >> VM_GetOrSetLocal::deoptimize_objects deoptimizes the top frame if it has non-globally escaping objects and other frames if they have arg escaping ones. Good. > > It's not specifically the top frame, but the frame that is accessed. > >> src/hotspot/share/runtime/deoptimization.cpp >> Object deoptimization. I have more comments and proposals, here. >> First of all, handling recursive and waiting locks in relock_objects is tricky, but looks correct. >> Comments are sufficient to understand why things are done as they are implemented. > >> BiasedLocking related parts are complex, but we may get rid of them in the future (with BiasedLocking removal). >> Anyway, looks correct, too. > >> Typo in comment: "regularily" => "regularly" > >> Deoptimization::fetch_unroll_info_helper is the only place where _jvmti_deferred_updates get deallocated (except JavaThread destructor). But I think we always go through it, so I can't see a memory leak or such kind of issues. > > That's correct. The compiled frame for which deferred updates are allocated is always deoptimized > before (see EscapeBarrier::deoptimize_objects()). This is also asserted in > compiledVFrame::update_deferred_value(). I've added the same assertion to > Deoptimization::relock_objects(). So we can be sure that _jvmti_deferred_updates are deallocated > again in fetch_unroll_info_helper(). > >> EscapeBarrier::deoptimize_objects: ResourceMark should use calling_thread(). > > Sure, well spotted! > >> You can use MutexLocker and MonitorLocker with Thread* to save the Thread::current() call. > > Right, good hint. This was recently introduced with 8235678. I even had to resolve conflicts. Should > have done this then. > >> I'd make set_objs_are_deoptimized static and remove it from the EscapeBarrier interface because I think it shouldn't be used outside of EscapeBarrier::deoptimize_objects. > > Done. > >> Typo in comment: "we must only deoptimize" => "we only have to deoptimize" > > Replaced with "[...] we deoptimize iff local objects are passed as args" > >> "bool EscapeBarrier::deoptimize_objects(intptr_t* fr_id)" is trivial and barrier_active() is redundant. Implementation can get moved to hpp file. > > Ok. Done. > >> I'll get back to suspend flags, later. > >> There are weird cases regarding _self_deoptimization_in_progress. >> Assume we have 3 threads A, B and C. A deopts C, B deopts C, C deopts C. C can set _self_deoptimization_in_progress while A performs the handshake for suspending C. I think this doesn't lead to errors, but it's probably not desired. >> I think it would be better to use only one "wait" call in sync_and_suspend_one and sync_and_suspend_all. > > You're right. We've discussed that face-to-face, but couldn't find a real issue. But now, thinking again, a reckon I found one: > > 2808 // Sync with other threads that might be doing deoptimizations > 2809 { > 2810 // Need to switch to _thread_blocked for the wait() call > 2811 ThreadBlockInVM tbivm(_calling_thread); > 2812 MonitorLocker ml(EscapeBarrier_lock, Mutex::_no_safepoint_check_flag); > 2813 while (_self_deoptimization_in_progress) { > 2814 ml.wait(); > 2815 } > 2816 > 2817 if (self_deopt()) { > 2818 _self_deoptimization_in_progress = true; > 2819 } > 2820 > 2821 while (_deoptee_thread->is_ea_obj_deopt_suspend()) { > 2822 ml.wait(); > 2823 } > 2824 > 2825 if (self_deopt()) { > 2826 return; > 2827 } > 2828 > 2829 // set suspend flag for target thread > 2830 _deoptee_thread->set_ea_obj_deopt_flag(); > 2831 } > > - A waits in 2822 > - C is suspended > - B notifies all in resume_one() > - A and C wake up > - C wins over A and sets _self_deoptimization_in_progress = true in 2818 > - C does the self deoptimization > - A executes 2830 _deoptee_thread->set_ea_obj_deopt_flag() > > C will self suspend at some undefined point. The resulting state is illegal. > >> I first thought it'd be better to move ThreadBlockInVM before wait() to reduce thread state transitions, but that seems to be problematic because ThreadBlockInVM destructor contains a safepoint check which we shouldn't do while holding EscapeBarrier_lock. So no change request. > > Yes, would be nice to have the state change only if needed, but for the reason you mentioned it is > not quite as easy as it seems to be. I experimented as well with a second lock, but did not succeed. > >> Change in thred_added: >> I think the sequence would be more comprehensive if we waited for deopt_all_threads in Thread::start and all other places where a new thread can run into Java code (e.g. JVMTI attach). >> Your version makes new threads come up with suspend flag set. That looks correct, too. Advantage is that you only have to change one place (thread_added). It'll be interesting to see how it will look like when we use async handshakes instead of suspend flags. >> For now, I'm ok with your version. > > I had a version that did what you are suggesting. The current version also has the advantage, that > there are fewer places where a thread has to wait for ongoing object deoptimization. This means > viewer places where you have to worry about correct thread state transitions, possible deadlocks, > and if all oops are properly Handle'ed. > >> I'd only move MutexLocker ml(EscapeBarrier_lock...) after if (!jt->is_hidden_from_external_view()). > > Done. > >> Having 4 different deoptimize_objects functions makes it a little hard to keep an overview of which one is used for what. >> Maybe adding suffixes would help a little bit, but I can also live with what you have. >> Implementation looks correct to me. > > 2 are internal. I added the suffix _internal to them. This leaves 2 to choose from. > >> src/hotspot/share/runtime/deoptimization.hpp >> Escape barriers and object deoptimization functions. >> Typo in comment: "helt" => "held" > > Done in place already. > >> src/hotspot/share/runtime/interfaceSupport.cpp >> InterfaceSupport::deoptimizeAllObjects() is only used for DeoptimizeObjectsALot = 1. >> I think DeoptimizeObjectsALot = 2 is more important, but I think it's not bad to have DeoptimizeObjectsALot = 1 in addition. Ok. > > I never used DeoptimizeObjectsALot = 1 that much. It could be more deterministic in single threaded > scenarios. I wouldn't object to get rid of it though. > >> src/hotspot/share/runtime/stackValue.hpp >> Better reinitilization in StackValue. Good. > > StackValue::obj_is_scalar_replaced() should not return true after calling set_obj(). > >> src/hotspot/share/runtime/thread.cpp >> src/hotspot/share/runtime/thread.hpp >> src/hotspot/share/runtime/thread.inline.hpp >> wait_for_object_deoptimization, suspend flag, deferred updates and test feature to deoptimize objects. > >> In the long term, we want to get rid of suspend flags, so it's not so nice to introduce a new one. But I agree with G?tz that it should be acceptable as temporary solution until async handshakes are available (which takes more time). So I'm ok with your change. > > I'm keen to build the feature on async handshakes when the arive. > >> You can use MutexLocker with Thread*. > > Done. > >> JVMTIDeferredUpdates: I agree with Robin. It'd be nice to move the class out of thread.hpp. > > Done. > >> src/hotspot/share/runtime/vframe.cpp >> Added support for entry frame to new_vframe. Ok. > > >> src/hotspot/share/runtime/vframe_hp.cpp >> src/hotspot/share/runtime/vframe_hp.hpp > >> I think code()->as_nmethod() in not_global_escape_in_scope() and arg_escape() should better be under #ifdef ASSERT or inside the assert statement (no need for code cache walking in product build). > > Done. > >> jvmtiDeferredLocalVariableSet::update_monitors: >> Please add a comment explaining that owner referenced by original info may be scalar replaced, but it is deoptimized in the vframe. > > Done. > > -----Original Message----- > From: Doerr, Martin > Sent: Donnerstag, 12. M?rz 2020 17:28 > To: Reingruber, Richard ; 'Robbin Ehn' ; Lindenmaier, Goetz ; David Holmes ; Vladimir Kozlov (vladimir.kozlov at oracle.com) ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net > Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents > > Hi Richard, > > > I managed to find time for a (almost) complete review of webrev.4. (I'll review the tests separately.) > > First of all, the change seems to be in pretty good quality for its significant complexity. I couldn't find any real bugs. But I'd like to propose minor improvements. > I'm convinced that it's mature because we did substantial testing. > > I like the new functionality for object deoptimization. It can possibly be reused for future escape analysis based optimizations. So I appreciate having it available in the code base. > In addition to that, your change makes the JVMTI implementation better integrated into the VM. > > > Now to the details: > > > src/hotspot/share/c1/c1_IR.hpp > describe_scope parameters. Ok. > > > src/hotspot/share/ci/ciEnv.cpp > src/hotspot/share/ci/ciEnv.hpp > Fix for JvmtiExport::can_walk_any_space() capability. Ok. > > > src/hotspot/share/code/compiledMethod.cpp > Nice cleanup! > > > src/hotspot/share/code/debugInfoRec.cpp > src/hotspot/share/code/debugInfoRec.hpp > Additional parmeters. (Remark: I think "non_global_escape_in_scope" would read better than "not_global_escape_in_scope", but your version is consistent with existing code, so no change request from my side.) Ok. > > > src/hotspot/share/code/nmethod.cpp > Nice cleanup! > > > src/hotspot/share/code/pcDesc.hpp > Additional parameters. Ok. > > > src/hotspot/share/code/scopeDesc.cpp > src/hotspot/share/code/scopeDesc.hpp > Improved implementation + additional parameters. Ok. > > > src/hotspot/share/compiler/compileBroker.cpp > src/hotspot/share/compiler/compileBroker.hpp > Extra thread for DeoptimizeObjectsALot. (Remark: I would have put it into a follow up change together with the test in order to make this webrev smaller, but since it is included, I'm reviewing everything at once. Not a big deal.) Ok. > > > src/hotspot/share/jvmci/jvmciCodeInstaller.cpp > Additional parameters. Ok. > > > src/hotspot/share/opto/c2compiler.cpp > Make do_escape_analysis independent of JVMCI capabilities. Nice! > > > src/hotspot/share/opto/callnode.hpp > Additional fields for MachSafePointNodes. Ok. > > > src/hotspot/share/opto/escape.cpp > Annotation for MachSafePointNodes. Your added functionality looks correct. > But I'd prefer to move the bulky code out of the large function. > I suggest to factor out something like has_not_global_escape and has_arg_escape. So the code could look like this: > SafePointNode* sfn = sfn_worklist.at(next); > sfn->set_not_global_escape_in_scope(has_not_global_escape(sfn)); > if (sfn->is_CallJava()) { > CallJavaNode* call = sfn->as_CallJava(); > call->set_arg_escape(has_arg_escape(call)); > } > This would also allow us to get rid of the found_..._escape_in_args variables making the loops better readable. > > It's kind of ugly to use strcmp to recognize uncommon trap, but that seems to be the way to do it (there are more such places). So it's ok. > > > src/hotspot/share/opto/machnode.hpp > Additional fields for MachSafePointNodes. Ok. > > > src/hotspot/share/opto/macro.cpp > Allow elimination of non-escaping allocations. Ok. > > > src/hotspot/share/opto/matcher.cpp > src/hotspot/share/opto/output.cpp > Copy attribute / pass parameters. Ok. > > > src/hotspot/share/prims/jvmtiCodeBlobEvents.cpp > Nice cleanup! > > > src/hotspot/share/prims/jvmtiEnv.cpp > src/hotspot/share/prims/jvmtiEnvBase.cpp > Escape barriers + deoptimize objects for target thread. Good. > > > src/hotspot/share/prims/jvmtiImpl.cpp > src/hotspot/share/prims/jvmtiImpl.hpp > The sequence is pretty complex: > VM_GetOrSetLocal element initialization executes EscapeBarrier code which suspends the target thread (extra VM Operation). > VM_GetOrSetLocal::doit_prologue performs object deoptimization (by VM Thread to prepare VM Operation with frame deoptimization). > VM_GetOrSetLocal destructor implicitly calls EscapeBarrier destructor which resumes the target thread. > But I don't have any improvement proposal. Performance is probably not a concern, here. So it's ok. > > VM_GetOrSetLocal::deoptimize_objects deoptimizes the top frame if it has non-globally escaping objects and other frames if they have arg escaping ones. Good. > > > src/hotspot/share/prims/jvmtiTagMap.cpp > Escape barriers + deoptimize objects for all threads. Ok. > > > src/hotspot/share/prims/whitebox.cpp > Added WB_IsFrameDeoptimized to API. Ok. > > > src/hotspot/share/runtime/deoptimization.cpp > Object deoptimization. I have more comments and proposals, here. > First of all, handling recursive and waiting locks in relock_objects is tricky, but looks correct. > Comments are sufficient to understand why things are done as they are implemented. > > BiasedLocking related parts are complex, but we may get rid of them in the future (with BiasedLocking removal). > Anyway, looks correct, too. > > Typo in comment: "regularily" => "regularly" > > Deoptimization::fetch_unroll_info_helper is the only place where _jvmti_deferred_updates get deallocated (except JavaThread destructor). But I think we always go through it, so I can't see a memory leak or such kind of issues. > > EscapeBarrier::deoptimize_objects: ResourceMark should use calling_thread(). > > You can use MutexLocker and MonitorLocker with Thread* to save the Thread::current() call. > > I'd make set_objs_are_deoptimized static and remove it from the EscapeBarrier interface because I think it shouldn't be used outside of EscapeBarrier::deoptimize_objects. > > Typo in comment: "we must only deoptimize" => "we only have to deoptimize" > > "bool EscapeBarrier::deoptimize_objects(intptr_t* fr_id)" is trivial and barrier_active() is redundant. Implementation can get moved to hpp file. > > I'll get back to suspend flags, later. > > There are weird cases regarding _self_deoptimization_in_progress. > Assume we have 3 threads A, B and C. A deopts C, B deopts C, C deopts C. C can set _self_deoptimization_in_progress while A performs the handshake for suspending C. I think this doesn't lead to errors, but it's probably not desired. > I think it would be better to use only one "wait" call in sync_and_suspend_one and sync_and_suspend_all. > > I first thought it'd be better to move ThreadBlockInVM before wait() to reduce thread state transitions, but that seems to be problematic because ThreadBlockInVM destructor contains a safepoint check which we shouldn't do while holding EscapeBarrier_lock. So no change request. > > Change in thred_added: > I think the sequence would be more comprehensive if we waited for deopt_all_threads in Thread::start and all other places where a new thread can run into Java code (e.g. JVMTI attach). > Your version makes new threads come up with suspend flag set. That looks correct, too. Advantage is that you only have to change one place (thread_added). It'll be interesting to see how it will look like when we use async handshakes instead of suspend flags. > For now, I'm ok with your version. > > I'd only move MutexLocker ml(EscapeBarrier_lock...) after if (!jt->is_hidden_from_external_view()). > > Having 4 different deoptimize_objects functions makes it a little hard to keep an overview of which one is used for what. > Maybe adding suffixes would help a little bit, but I can also live with what you have. > Implementation looks correct to me. > > > src/hotspot/share/runtime/deoptimization.hpp > Escape barriers and object deoptimization functions. > Typo in comment: "helt" => "held" > > > src/hotspot/share/runtime/globals.hpp > Addition of develop flag DeoptimizeObjectsALotInterval. Ok. > > > src/hotspot/share/runtime/interfaceSupport.cpp > InterfaceSupport::deoptimizeAllObjects() is only used for DeoptimizeObjectsALot = 1. > I think DeoptimizeObjectsALot = 2 is more important, but I think it's not bad to have DeoptimizeObjectsALot = 1 in addition. Ok. > > > src/hotspot/share/runtime/interfaceSupport.inline.hpp > Addition of deoptimizeAllObjects. Ok. > > > src/hotspot/share/runtime/mutexLocker.cpp > src/hotspot/share/runtime/mutexLocker.hpp > Addition of EscapeBarrier_lock. Ok. > > > src/hotspot/share/runtime/objectMonitor.cpp > Make recursion count relock aware. Ok. > > > src/hotspot/share/runtime/stackValue.hpp > Better reinitilization in StackValue. Good. > > > src/hotspot/share/runtime/thread.cpp > src/hotspot/share/runtime/thread.hpp > src/hotspot/share/runtime/thread.inline.hpp > wait_for_object_deoptimization, suspend flag, deferred updates and test feature to deoptimize objects. > > In the long term, we want to get rid of suspend flags, so it's not so nice to introduce a new one. But I agree with G?tz that it should be acceptable as temporary solution until async handshakes are available (which takes more time). So I'm ok with your change. > > You can use MutexLocker with Thread*. > > JVMTIDeferredUpdates: I agree with Robin. It'd be nice to move the class out of thread.hpp. > > > src/hotspot/share/runtime/vframe.cpp > Added support for entry frame to new_vframe. Ok. > > > src/hotspot/share/runtime/vframe_hp.cpp > src/hotspot/share/runtime/vframe_hp.hpp > > I think code()->as_nmethod() in not_global_escape_in_scope() and arg_escape() should better be under #ifdef ASSERT or inside the assert statement (no need for code cache walking in product build). > > jvmtiDeferredLocalVariableSet::update_monitors: > Please add a comment explaining that owner referenced by original info may be scalar replaced, but it is deoptimized in the vframe. > > > src/hotspot/share/utilities/macros.hpp > Addition of NOT_COMPILER2_OR_JVMCI_RETURN macros. Ok. > > > test/hotspot/jtreg/serviceability/jvmti/Heap/IterateHeapWithEscapeAnalysisEnabled.java > test/hotspot/jtreg/serviceability/jvmti/Heap/libIterateHeapWithEscapeAnalysisEnabled.c > New test. Will review separately. > > > test/jdk/TEST.ROOT > Addition of vm.jvmci as required property. Ok. > > > test/jdk/com/sun/jdi/EATests.java > test/jdk/com/sun/jdi/EATestsJVMCI.java > New test. Will review separately. > > > test/lib/sun/hotspot/WhiteBox.java > Added isFrameDeoptimized to API. Ok. > > > That was it. Best regards, > Martin > > >> -----Original Message----- >> From: hotspot-compiler-dev > bounces at openjdk.java.net> On Behalf Of Reingruber, Richard >> Sent: Dienstag, 3. M?rz 2020 21:23 >> To: 'Robbin Ehn' ; Lindenmaier, Goetz >> ; David Holmes ; >> Vladimir Kozlov (vladimir.kozlov at oracle.com) >> ; serviceability-dev at openjdk.java.net; >> hotspot-compiler-dev at openjdk.java.net; hotspot-runtime- >> dev at openjdk.java.net >> Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better >> Performance in the Presence of JVMTI Agents >> >> Hi Robbin, >> >>>> I understand that Robbin proposed to replace the usage of >>>> _suspend_flag with handshakes. Apparently, async handshakes >>>> are needed to do so. We have been waiting a while for removal >>>> of the _suspend_flag / introduction of async handshakes [2]. >>>> What is the status here? >> >>> I have an old prototype which I would like to continue to work on. >>> So do not assume asynch handshakes will make 15. >>> Even if it would, I think there are a lot more investigate work to remove >>> _suspend_flag. >> >> Let us know, if we can be of any help to you and be it only testing. >> >>>>> Full: >> http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.4/ >> >>> DeoptimizeObjectsALotThread is only used in compileBroker.cpp. >>> You can move both declaration and definition to that file, no need to >> clobber >>> thread.[c|h]pp. (and the static function deopt_objs_alot_thread_entry) >> >> Will do. >> >>> Does JvmtiDeferredUpdates really need to be in thread.hpp, can't be in it's >> own >>> hpp file? It doesn't seem right to add JVM TI classes into thread.hpp. >> >> You are right. It shouldn't be declared in thread.hpp. I will look into that. >> >>> Note that we also think we may have a bug in deopt: >>> https://bugs.openjdk.java.net/browse/JDK-8238237 >> >>> I think it would be best, if possible, to push after that is resolved. >> >> Sure. >> >>> Not even nearly a full review :) >> >> I know :) >> >> Anyways, thanks a lot, >> Richard. >> >> >> -----Original Message----- >> From: Robbin Ehn >> Sent: Monday, March 2, 2020 11:17 AM >> To: Lindenmaier, Goetz ; Reingruber, Richard >> ; David Holmes ; >> Vladimir Kozlov (vladimir.kozlov at oracle.com) >> ; serviceability-dev at openjdk.java.net; >> hotspot-compiler-dev at openjdk.java.net; hotspot-runtime- >> dev at openjdk.java.net >> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better Performance >> in the Presence of JVMTI Agents >> >> Hi, >> >> On 2/24/20 5:39 PM, Lindenmaier, Goetz wrote: >>> Hi, >>> >>> I had a look at the progress of this change. Nothing >>> happened since Richard posted his update using more >>> handshakes [1]. >>> But we (SAP) would appreciate a lot if this change could >>> be successfully reviewed and pushed. >>> >>> I think there is basic understanding that this >>> change is helpful. It fixes a number of issues with JVMTI, >>> and will deliver the same performance benefits as EA >>> does in current production mode for debugging scenarios. >>> >>> This is important for us as we run our VMs prepared >>> for debugging in production mode. >>> >>> I understand that Robbin proposed to replace the usage of >>> _suspend_flag with handshakes. Apparently, async handshakes >>> are needed to do so. We have been waiting a while for removal >>> of the _suspend_flag / introduction of async handshakes [2]. >>> What is the status here? >> >> I have an old prototype which I would like to continue to work on. >> So do not assume asynch handshakes will make 15. >> Even if it would, I think there are a lot more investigate work to remove >> _suspend_flag. >> >>> >>> I think we should no longer wait, but proceed with >>> this change. We will look into removing the usage of >>> suspend_flag introduced here once it is possible to implement >>> it with handshakes. >> >> Yes, sure. >> >>>> Full: http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.4/ >> >> DeoptimizeObjectsALotThread is only used in compileBroker.cpp. >> You can move both declaration and definition to that file, no need to clobber >> thread.[c|h]pp. (and the static function deopt_objs_alot_thread_entry) >> >> Does JvmtiDeferredUpdates really need to be in thread.hpp, can't be in it's >> own >> hpp file? It doesn't seem right to add JVM TI classes into thread.hpp. >> >> Note that we also think we may have a bug in deopt: >> https://bugs.openjdk.java.net/browse/JDK-8238237 >> >> I think it would be best, if possible, to push after that is resolved. >> >> Not even nearly a full review :) >> >> Thanks, Robbin >> >> >>>> Incremental: >>>> http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.4.inc/ >>>> >>>> I was not able to eliminate the additional suspend flag now. I'll take care >> of this >>>> as soon as the >>>> existing suspend-resume-mechanism is reworked. >>>> >>>> Testing: >>>> >>>> Nightly tests @SAP: >>>> >>>> JCK and JTREG, also in Xcomp mode, SPECjvm2008, SPECjbb2015, >> Renaissance >>>> Suite, SAP specific tests >>>> with fastdebug and release builds on all platforms >>>> >>>> Stress testing with DeoptimizeObjectsALot running SPECjvm2008 40x >> parallel >>>> for 24h >>>> >>>> Thanks, Richard. >>>> >>>> >>>> More details on the changes: >>>> >>>> * Hide DeoptimizeObjectsALotThread from external view. >>>> >>>> * Changed EscapeBarrier_lock to be a _safepoint_check_never lock. >>>> It used to be _safepoint_check_sometimes, which will be eliminated >> sooner or >>>> later. >>>> I added explicit thread state changes with ThreadBlockInVM to code >> paths >>>> where we can wait() >>>> on EscapeBarrier_lock to become safepoint safe. >>>> >>>> * Use handshake EscapeBarrierSuspendHandshake to suspend target >> threads >>>> instead of vm operation >>>> VM_ThreadSuspendAllForObjDeopt. >>>> >>>> * Removed uses of Threads_lock. When adding a new thread we suspend >> it iff >>>> EA optimizations are >>>> being reverted. In the previous version we were waiting on >> Threads_lock >>>> while EA optimizations >>>> were reverted. See EscapeBarrier::thread_added(). >>>> >>>> * Made tests require Xmixed compilation mode. >>>> >>>> * Made tests agnostic regarding tiered compilation. >>>> I.e. tc isn't disabled anymore, and the tests can be run with tc enabled or >>>> disabled. >>>> >>>> * Exercising EATests.java as well with stress test options >>>> DeoptimizeObjectsALot* >>>> Due to the non-deterministic deoptimizations some tests need to be >> skipped. >>>> We do this to prevent bit-rot of the stress test code. >>>> >>>> * Executing EATests.java as well with graal if available. Driver for this is >>>> EATestsJVMCI.java. Graal cannot pass all tests, because it does not >> provide all >>>> the new debug info >>>> (namely not_global_escape_in_scope and arg_escape in >> scopeDesc.hpp). >>>> And graal does not yet support the JVMTI operations force early return >> and >>>> pop frame. >>>> >>>> * Removed tracing from new jdi tests in EATests.java. Too much trace >> output >>>> before the debugging >>>> connection is established can cause deadlock because output buffers fill >> up. >>>> (See https://bugs.openjdk.java.net/browse/JDK-8173304) >>>> >>>> * Many copyright year changes and smaller clean-up changes of testing >> code >>>> (trailing white-space and >>>> the like). >>>> >>>> >>>> -----Original Message----- >>>> From: David Holmes >>>> Sent: Donnerstag, 19. Dezember 2019 03:12 >>>> To: Reingruber, Richard ; serviceability- >>>> dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; >> hotspot- >>>> runtime-dev at openjdk.java.net; Vladimir Kozlov >> (vladimir.kozlov at oracle.com) >>>> >>>> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better >> Performance in >>>> the Presence of JVMTI Agents >>>> >>>> Hi Richard, >>>> >>>> I think my issue is with the way EliminateNestedLocks works so I'm going >>>> to look into that more deeply. >>>> >>>> Thanks for the explanations. >>>> >>>> David >>>> >>>> On 18/12/2019 12:47 am, Reingruber, Richard wrote: >>>>> Hi David, >>>>> >>>>> > > > Some further queries/concerns: >>>>> > > > >>>>> > > > src/hotspot/share/runtime/objectMonitor.cpp >>>>> > > > >>>>> > > > Can you please explain the changes to ObjectMonitor::wait: >>>>> > > > >>>>> > > > ! _recursions = save // restore the old recursion count >>>>> > > > ! + jt->get_and_reset_relock_count_after_wait(); // >>>>> > > > increased by the deferred relock count >>>>> > > > >>>>> > > > what is the "deferred relock count"? I gather it relates to >>>>> > > > >>>>> > > > "The code was extended to be able to deoptimize objects of a >>>>> > > frame that >>>>> > > > is not the top frame and to let another thread than the owning >>>>> > > thread do >>>>> > > > it." >>>>> > > >>>>> > > Yes, these relate. Currently EA based optimizations are reverted, >> when a >>>> compiled frame is >>>>> > > replaced with corresponding interpreter frames. Part of this is >> relocking >>>> objects with eliminated >>>>> > > locking. New with the enhancement is that we do this also just >> before >>>> object references are >>>>> > > acquired through JVMTI. In this case we deoptimize also the >> owning >>>> compiled frame C and we >>>>> > > register deoptimized objects as deferred updates. When control >> returns >>>> to C it gets deoptimized, >>>>> > > we notice that objects are already deoptimized (reallocated and >>>> relocked), so we don't do it again >>>>> > > (relocking twice would be incorrect of course). Deferred updates >> are >>>> copied into the new >>>>> > > interpreter frames. >>>>> > > >>>>> > > Problem: relocking is not possible if the target thread T is waiting >> on the >>>> monitor that needs to >>>>> > > be relocked. This happens only with non-local objects with >>>> EliminateNestedLocks. Instead relocking >>>>> > > is deferred until T owns the monitor again. This is what the piece of >>>> code above does. >>>>> > >>>>> > Sorry I need some more detail here. How can you wait() on an >> object >>>>> > monitor if the object allocation and/or locking was optimised away? >> And >>>>> > what is a "non-local object" in this context? Isn't EA restricted to >>>>> > thread-confined objects? >>>>> >>>>> "Non-local object" is an object that escapes its thread. The issue I'm >>>> addressing with the changes >>>>> in ObjectMonitor::wait are almost unrelated to EA. They are caused by >>>> EliminateNestedLocks, where C2 >>>>> eliminates recursive locking of an already owned lock. The lock owning >> object >>>> exists on the heap, it >>>>> is locked and you can call wait() on it. >>>>> >>>>> EliminateLocks is the C2 option that controls lock elimination based on >> EA. >>>> Both optimizations have >>>>> in common that objects with eliminated locking need to be relocked >> when >>>> deoptimizing a frame, >>>>> i.e. when replacing a compiled frame with equivalent interpreter >>>>> frames. Deoptimization::relock_objects does that job for /all/ eliminated >>>> locks in scope. /All/ can >>>>> be a mix of eliminated nested locks and locks of not-escaping objects. >>>>> >>>>> New with the enhancement: I call relock_objects earlier, just before >> objects >>>> pontentially >>>>> escape. But then later when the owning compiled frame gets >> deoptimized, I >>>> must not do it again: >>>>> >>>>> See call to EscapeBarrier::objs_are_deoptimized in deoptimization.cpp: >>>>> >>>>> 373 if ((jvmci_enabled || ((DoEscapeAnalysis || >> EliminateNestedLocks) && >>>> EliminateLocks)) >>>>> 374 && !EscapeBarrier::objs_are_deoptimized(thread, >> deoptee.id())) { >>>>> 375 bool unused; >>>>> 376 eliminate_locks(thread, chunk, realloc_failures, deoptee, >> exec_mode, >>>> unused); >>>>> 377 } >>>>> >>>>> Now when calling relock_objects early it is quiet possible that I have to >> relock >>>> an object the >>>>> target thread currently waits for. Obviously I cannot relock in this case, >>>> instead I chose to >>>>> introduce relock_count_after_wait to JavaThread. >>>>> >>>>> > Is it just that some of the locking gets optimized away e.g. >>>>> > >>>>> > synchronised(obj) { >>>>> > synchronised(obj) { >>>>> > synchronised(obj) { >>>>> > obj.wait(); >>>>> > } >>>>> > } >>>>> > } >>>>> > >>>>> > If this is reduced to a form as-if it were a single lock of the monitor >>>>> > (due to EA) and the wait() triggers a JVM TI event which leads to the >>>>> > escape of "obj" then we need to reconstruct the true lock state, and >> so >>>>> > when the wait() internally unblocks and reacquires the monitor it >> has to >>>>> > set the true recursion count to 3, not the 1 that it appeared to be >> when >>>>> > wait() was initially called. Is that the scenario? >>>>> >>>>> Kind of... except that the locking is not eliminated due to EA and there is >> no >>>> JVM TI event >>>>> triggered by wait. >>>>> >>>>> Add >>>>> >>>>> LocalObject l1 = new LocalObject(); >>>>> >>>>> in front of the synchrnized blocks and assume a JVM TI agent acquires l1. >> This >>>> triggers the code in >>>>> question. >>>>> >>>>> See that relocking/reallocating is transactional. If it is done then for /all/ >>>> objects in scope and it is >>>>> done at most once. It wouldn't be quite so easy to split this in relocking >> of >>>> nested/EA-based >>>>> eliminated locks. >>>>> >>>>> > If so I find this truly awful. Anyone using wait() in a realistic form >>>>> > requires a notification and so the object cannot be thread confined. >> In >>>>> >>>>> It is not thread confined. >>>>> >>>>> > which case I would strongly argue that upon hitting the wait() the >> deopt >>>>> > should occur unconditionally and so the lock state is correct before >> we >>>>> > wait and so we don't need to mess with the recursion count >> internally >>>>> > when we reacquire the monitor. >>>>> > >>>>> > > >>>>> > > > which I don't like the sound of at all when it comes to >> ObjectMonitor >>>>> > > > state. So I'd like to understand in detail exactly what is going on >> here >>>>> > > > and why. This is a very intrusive change that seems to badly >> break >>>>> > > > encapsulation and impacts future changes to ObjectMonitor >> that are >>>> under >>>>> > > > investigation. >>>>> > > >>>>> > > I would not regard this as breaking encapsulation. Certainly not >> badly. >>>>> > > >>>>> > > I've added a property relock_count_after_wait to JavaThread. The >>>> property is well >>>>> > > encapsulated. Future ObjectMonitor implementations have to deal >> with >>>> recursion too. They are free >>>>> > > in choosing a way to do that as long as that property is taken into >>>> account. This is hardly a >>>>> > > limitation. >>>>> > >>>>> > I do think this badly breaks encapsulation as you have to add a >> callout >>>>> > from the guts of the ObjectMonitor code to reach into the thread to >> get >>>>> > this lock count adjustment. I understand why you have had to do >> this but >>>>> > I would much rather see a change to the EA optimisation strategy so >> that >>>>> > this is not needed. >>>>> > >>>>> > > Note also that the property is a straight forward extension of the >>>> existing concept of deferred >>>>> > > local updates. It is embedded into the structure holding them. So >> not >>>> even the footprint of a >>>>> > > JavaThread is enlarged if no deferred updates are generated. >>>>> > >>>>> > [...] >>>>> > >>>>> > > >>>>> > > I'm actually duplicating the existing external suspend mechanism, >>>> because a thread can be >>>>> > > suspended at most once. And hey, and don't like that either! But it >>>> seems not unlikely that the >>>>> > > duplicate can be removed together with the original and the new >> type >>>> of handshakes that will be >>>>> > > used for thread suspend can be used for object deoptimization >> too. See >>>> today's discussion in >>>>> > > JDK-8227745 [2]. >>>>> > >>>>> > I hope that discussion bears some fruit, at the moment it seems not >> to >>>>> > be possible to use handshakes here. :( >>>>> > >>>>> > The external suspend mechanism is a royal pain in the proverbial >> that we >>>>> > have to carefully live with. The idea that we're duplicating that for >>>>> > use in another fringe area of functionality does not thrill me at all. >>>>> > >>>>> > To be clear, I understand the problem that exists and that you wish >> to >>>>> > solve, but for the runtime parts I balk at the complexity cost of >>>>> > solving it. >>>>> >>>>> I know it's complex, but by far no rocket science. >>>>> >>>>> Also I find it hard to imagine another fix for JDK-8233915 besides >> changing >>>> the JVM TI specification. >>>>> >>>>> Thanks, Richard. >>>>> >>>>> -----Original Message----- >>>>> From: David Holmes >>>>> Sent: Dienstag, 17. Dezember 2019 08:03 >>>>> To: Reingruber, Richard ; serviceability- >>>> dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; >> hotspot- >>>> runtime-dev at openjdk.java.net; Vladimir Kozlov >> (vladimir.kozlov at oracle.com) >>>> >>>>> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better >> Performance >>>> in the Presence of JVMTI Agents >>>>> >>>>> >>>>> >>>>> David >>>>> >>>>> On 17/12/2019 4:57 pm, David Holmes wrote: >>>>>> Hi Richard, >>>>>> >>>>>> On 14/12/2019 5:01 am, Reingruber, Richard wrote: >>>>>>> Hi David, >>>>>>> >>>>>>> ?? > Some further queries/concerns: >>>>>>> ?? > >>>>>>> ?? > src/hotspot/share/runtime/objectMonitor.cpp >>>>>>> ?? > >>>>>>> ?? > Can you please explain the changes to ObjectMonitor::wait: >>>>>>> ?? > >>>>>>> ?? > !?? _recursions = save????? // restore the old recursion count >>>>>>> ?? > !???????????????? + jt->get_and_reset_relock_count_after_wait(); // >>>>>>> ?? > increased by the deferred relock count >>>>>>> ?? > >>>>>>> ?? > what is the "deferred relock count"? I gather it relates to >>>>>>> ?? > >>>>>>> ?? > "The code was extended to be able to deoptimize objects of a >>>>>>> frame that >>>>>>> ?? > is not the top frame and to let another thread than the owning >>>>>>> thread do >>>>>>> ?? > it." >>>>>>> >>>>>>> Yes, these relate. Currently EA based optimizations are reverted, >> when >>>>>>> a compiled frame is replaced >>>>>>> with corresponding interpreter frames. Part of this is relocking >>>>>>> objects with eliminated >>>>>>> locking. New with the enhancement is that we do this also just before >>>>>>> object references are acquired >>>>>>> through JVMTI. In this case we deoptimize also the owning compiled >>>>>>> frame C and we register >>>>>>> deoptimized objects as deferred updates. When control returns to C >> it >>>>>>> gets deoptimized, we notice >>>>>>> that objects are already deoptimized (reallocated and relocked), so >> we >>>>>>> don't do it again (relocking >>>>>>> twice would be incorrect of course). Deferred updates are copied into >>>>>>> the new interpreter frames. >>>>>>> >>>>>>> Problem: relocking is not possible if the target thread T is waiting >>>>>>> on the monitor that needs to be >>>>>>> relocked. This happens only with non-local objects with >>>>>>> EliminateNestedLocks. Instead relocking is >>>>>>> deferred until T owns the monitor again. This is what the piece of >>>>>>> code above does. >>>>>> >>>>>> Sorry I need some more detail here. How can you wait() on an object >>>>>> monitor if the object allocation and/or locking was optimised away? >> And >>>>>> what is a "non-local object" in this context? Isn't EA restricted to >>>>>> thread-confined objects? >>>>>> >>>>>> Is it just that some of the locking gets optimized away e.g. >>>>>> >>>>>> synchronised(obj) { >>>>>> ? synchronised(obj) { >>>>>> ??? synchronised(obj) { >>>>>> ????? obj.wait(); >>>>>> ??? } >>>>>> ? } >>>>>> } >>>>>> >>>>>> If this is reduced to a form as-if it were a single lock of the monitor >>>>>> (due to EA) and the wait() triggers a JVM TI event which leads to the >>>>>> escape of "obj" then we need to reconstruct the true lock state, and so >>>>>> when the wait() internally unblocks and reacquires the monitor it has to >>>>>> set the true recursion count to 3, not the 1 that it appeared to be when >>>>>> wait() was initially called. Is that the scenario? >>>>>> >>>>>> If so I find this truly awful. Anyone using wait() in a realistic form >>>>>> requires a notification and so the object cannot be thread confined. In >>>>>> which case I would strongly argue that upon hitting the wait() the >> deopt >>>>>> should occur unconditionally and so the lock state is correct before we >>>>>> wait and so we don't need to mess with the recursion count internally >>>>>> when we reacquire the monitor. >>>>>> >>>>>>> >>>>>>> ?? > which I don't like the sound of at all when it comes to >>>>>>> ObjectMonitor >>>>>>> ?? > state. So I'd like to understand in detail exactly what is going >>>>>>> on here >>>>>>> ?? > and why.? This is a very intrusive change that seems to badly >> break >>>>>>> ?? > encapsulation and impacts future changes to ObjectMonitor that >>>>>>> are under >>>>>>> ?? > investigation. >>>>>>> >>>>>>> I would not regard this as breaking encapsulation. Certainly not badly. >>>>>>> >>>>>>> I've added a property relock_count_after_wait to JavaThread. The >>>>>>> property is well >>>>>>> encapsulated. Future ObjectMonitor implementations have to deal >> with >>>>>>> recursion too. They are free in >>>>>>> choosing a way to do that as long as that property is taken into >>>>>>> account. This is hardly a >>>>>>> limitation. >>>>>> >>>>>> I do think this badly breaks encapsulation as you have to add a callout >>>>>> from the guts of the ObjectMonitor code to reach into the thread to >> get >>>>>> this lock count adjustment. I understand why you have had to do this >> but >>>>>> I would much rather see a change to the EA optimisation strategy so >> that >>>>>> this is not needed. >>>>>> >>>>>>> Note also that the property is a straight forward extension of the >>>>>>> existing concept of deferred >>>>>>> local updates. It is embedded into the structure holding them. So not >>>>>>> even the footprint of a >>>>>>> JavaThread is enlarged if no deferred updates are generated. >>>>>>> >>>>>>> ?? > --- >>>>>>> ?? > >>>>>>> ?? > src/hotspot/share/runtime/thread.cpp >>>>>>> ?? > >>>>>>> ?? > Can you please explain why >>>>>>> JavaThread::wait_for_object_deoptimization >>>>>>> ?? > has to be handcrafted in this way rather than using proper >>>>>>> transitions. >>>>>>> ?? > >>>>>>> >>>>>>> I wrote wait_for_object_deoptimization taking >>>>>>> JavaThread::java_suspend_self_with_safepoint_check >>>>>>> as template. So in short: for the same reasons :) >>>>>>> >>>>>>> Threads reach both methods as part of thread state transitions, >>>>>>> therefore special handling is >>>>>>> required to change thread state on top of ongoing transitions. >>>>>>> >>>>>>> ?? > We got rid of "deopt suspend" some time ago and it is disturbing >>>>>>> to see >>>>>>> ?? > it being added back (effectively). This seems like it may be >>>>>>> something >>>>>>> ?? > that handshakes could be used for. >>>>>>> >>>>>>> Deopt suspend used to be something rather different with a similar >>>>>>> name[1]. It is not being added back. >>>>>> >>>>>> I stand corrected. Despite comments in the code to the contrary >>>>>> deopt_suspend didn't actually cause a self-suspend. I was doing a lot of >>>>>> cleanup in this area 13 years ago :) >>>>>> >>>>>>> >>>>>>> I'm actually duplicating the existing external suspend mechanism, >>>>>>> because a thread can be suspended >>>>>>> at most once. And hey, and don't like that either! But it seems not >>>>>>> unlikely that the duplicate can >>>>>>> be removed together with the original and the new type of >> handshakes >>>>>>> that will be used for >>>>>>> thread suspend can be used for object deoptimization too. See >> today's >>>>>>> discussion in JDK-8227745 [2]. >>>>>> >>>>>> I hope that discussion bears some fruit, at the moment it seems not to >>>>>> be possible to use handshakes here. :( >>>>>> >>>>>> The external suspend mechanism is a royal pain in the proverbial that >> we >>>>>> have to carefully live with. The idea that we're duplicating that for >>>>>> use in another fringe area of functionality does not thrill me at all. >>>>>> >>>>>> To be clear, I understand the problem that exists and that you wish to >>>>>> solve, but for the runtime parts I balk at the complexity cost of >>>>>> solving it. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>> >>>>>>> Thanks, Richard. >>>>>>> >>>>>>> [1] Deopt suspend was something like an async. handshake for >>>>>>> architectures with register windows, >>>>>>> ???? where patching the return pc for deoptimization of a compiled >>>>>>> frame was racy if the owner thread >>>>>>> ???? was in native code. Instead a "deopt" suspend flag was set on >>>>>>> which the thread patched its own >>>>>>> ???? frame upon return from native. So no thread was suspended. It >> got >>>>>>> its name only from the name of >>>>>>> ???? the flags. >>>>>>> >>>>>>> [2] Discussion about using handshakes to sync. with the target thread: >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK- >>>> >> 8227745?focusedCommentId=14306727&page=com.atlassian.jira.plugin.syst >> e >>>> m.issuetabpanels:comment-tabpanel#comment-14306727 >>>>>>> >>>>>>> >>>>>>> -----Original Message----- >>>>>>> From: David Holmes >>>>>>> Sent: Freitag, 13. Dezember 2019 00:56 >>>>>>> To: Reingruber, Richard ; >>>>>>> serviceability-dev at openjdk.java.net; >>>>>>> hotspot-compiler-dev at openjdk.java.net; >>>>>>> hotspot-runtime-dev at openjdk.java.net >>>>>>> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better >>>>>>> Performance in the Presence of JVMTI Agents >>>>>>> >>>>>>> Hi Richard, >>>>>>> >>>>>>> Some further queries/concerns: >>>>>>> >>>>>>> src/hotspot/share/runtime/objectMonitor.cpp >>>>>>> >>>>>>> Can you please explain the changes to ObjectMonitor::wait: >>>>>>> >>>>>>> !?? _recursions = save????? // restore the old recursion count >>>>>>> !???????????????? + jt->get_and_reset_relock_count_after_wait(); // >>>>>>> increased by the deferred relock count >>>>>>> >>>>>>> what is the "deferred relock count"? I gather it relates to >>>>>>> >>>>>>> "The code was extended to be able to deoptimize objects of a frame >> that >>>>>>> is not the top frame and to let another thread than the owning thread >> do >>>>>>> it." >>>>>>> >>>>>>> which I don't like the sound of at all when it comes to ObjectMonitor >>>>>>> state. So I'd like to understand in detail exactly what is going on here >>>>>>> and why.? This is a very intrusive change that seems to badly break >>>>>>> encapsulation and impacts future changes to ObjectMonitor that are >> under >>>>>>> investigation. >>>>>>> >>>>>>> --- >>>>>>> >>>>>>> src/hotspot/share/runtime/thread.cpp >>>>>>> >>>>>>> Can you please explain why >> JavaThread::wait_for_object_deoptimization >>>>>>> has to be handcrafted in this way rather than using proper transitions. >>>>>>> >>>>>>> We got rid of "deopt suspend" some time ago and it is disturbing to >> see >>>>>>> it being added back (effectively). This seems like it may be something >>>>>>> that handshakes could be used for. >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>> On 12/12/2019 7:02 am, David Holmes wrote: >>>>>>>> On 12/12/2019 1:07 am, Reingruber, Richard wrote: >>>>>>>>> Hi David, >>>>>>>>> >>>>>>>>> ??? > Most of the details here are in areas I can comment on in >> detail, >>>>>>>>> but I >>>>>>>>> ??? > did take an initial general look at things. >>>>>>>>> >>>>>>>>> Thanks for taking the time! >>>>>>>> >>>>>>>> Apologies the above should read: >>>>>>>> >>>>>>>> "Most of the details here are in areas I *can't* comment on in detail >>>>>>>> ..." >>>>>>>> >>>>>>>> David >>>>>>>> >>>>>>>>> ??? > The only thing that jumped out at me is that I think the >>>>>>>>> ??? > DeoptimizeObjectsALotThread should be a hidden thread. >>>>>>>>> ??? > >>>>>>>>> ??? > +? bool is_hidden_from_external_view() const { return true; } >>>>>>>>> >>>>>>>>> Yes, it should. Will add the method like above. >>>>>>>>> >>>>>>>>> ??? > Also I don't see any testing of the >> DeoptimizeObjectsALotThread. >>>>>>>>> Without >>>>>>>>> ??? > active testing this will just bit-rot. >>>>>>>>> >>>>>>>>> DeoptimizeObjectsALot is meant for stress testing with a larger >>>>>>>>> workload. I will add a minimal test >>>>>>>>> to keep it fresh. >>>>>>>>> >>>>>>>>> ??? > Also on the tests I don't understand your @requires clause: >>>>>>>>> ??? > >>>>>>>>> ??? >?? @requires ((vm.compMode != "Xcomp") & >> vm.compiler2.enabled >>>> & >>>>>>>>> ??? > (vm.opt.TieredCompilation != true)) >>>>>>>>> ??? > >>>>>>>>> ??? > This seems to require that TieredCompilation is disabled, but >>>>>>>>> tiered is >>>>>>>>> ??? > our normal mode of operation. ?? >>>>>>>>> ??? > >>>>>>>>> >>>>>>>>> I removed the clause. I guess I wanted to target the tests towards >> the >>>>>>>>> code they are supposed to >>>>>>>>> test, and it's easier to analyze failures w/o tiered compilation and >>>>>>>>> with just one compiler thread. >>>>>>>>> >>>>>>>>> Additionally I will make use of >>>>>>>>> compiler.whitebox.CompilerWhiteBoxTest.THRESHOLD in the tests. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Richard. >>>>>>>>> >>>>>>>>> -----Original Message----- >>>>>>>>> From: David Holmes >>>>>>>>> Sent: Mittwoch, 11. Dezember 2019 08:03 >>>>>>>>> To: Reingruber, Richard ; >>>>>>>>> serviceability-dev at openjdk.java.net; >>>>>>>>> hotspot-compiler-dev at openjdk.java.net; >>>>>>>>> hotspot-runtime-dev at openjdk.java.net >>>>>>>>> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better >>>>>>>>> Performance in the Presence of JVMTI Agents >>>>>>>>> >>>>>>>>> Hi Richard, >>>>>>>>> >>>>>>>>> On 11/12/2019 7:45 am, Reingruber, Richard wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> I would like to get reviews please for >>>>>>>>>> >>>>>>>>>> >> http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.3/ >>>>>>>>>> >>>>>>>>>> Corresponding RFE: >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8227745 >>>>>>>>>> >>>>>>>>>> Fixes also https://bugs.openjdk.java.net/browse/JDK-8233915 >>>>>>>>>> And potentially https://bugs.openjdk.java.net/browse/JDK- >> 8214584 [1] >>>>>>>>>> >>>>>>>>>> Vladimir Kozlov kindly put webrev.3 through tier1-8 testing >> without >>>>>>>>>> issues (thanks!). In addition the >>>>>>>>>> change is being tested at SAP since I posted the first RFR some >>>>>>>>>> months ago. >>>>>>>>>> >>>>>>>>>> The intention of this enhancement is to benefit performance wise >> from >>>>>>>>>> escape analysis even if JVMTI >>>>>>>>>> agents request capabilities that allow them to access local variable >>>>>>>>>> values. E.g. if you start-up >>>>>>>>>> with -agentlib:jdwp=transport=dt_socket,server=y,suspend=n, >> then >>>>>>>>>> escape analysis is disabled right >>>>>>>>>> from the beginning, well before a debugger attaches -- if ever one >>>>>>>>>> should do so. With the >>>>>>>>>> enhancement, escape analysis will remain enabled until and after >> a >>>>>>>>>> debugger attaches. EA based >>>>>>>>>> optimizations are reverted just before an agent acquires the >>>>>>>>>> reference to an object. In the JBS item >>>>>>>>>> you'll find more details. >>>>>>>>> >>>>>>>>> Most of the details here are in areas I can comment on in detail, but >> I >>>>>>>>> did take an initial general look at things. >>>>>>>>> >>>>>>>>> The only thing that jumped out at me is that I think the >>>>>>>>> DeoptimizeObjectsALotThread should be a hidden thread. >>>>>>>>> >>>>>>>>> +? bool is_hidden_from_external_view() const { return true; } >>>>>>>>> >>>>>>>>> Also I don't see any testing of the DeoptimizeObjectsALotThread. >>>>>>>>> Without >>>>>>>>> active testing this will just bit-rot. >>>>>>>>> >>>>>>>>> Also on the tests I don't understand your @requires clause: >>>>>>>>> >>>>>>>>> ??? @requires ((vm.compMode != "Xcomp") & >> vm.compiler2.enabled & >>>>>>>>> (vm.opt.TieredCompilation != true)) >>>>>>>>> >>>>>>>>> This seems to require that TieredCompilation is disabled, but tiered >> is >>>>>>>>> our normal mode of operation. ?? >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> David >>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Richard. >>>>>>>>>> >>>>>>>>>> [1] Experimental fix for JDK-8214584 based on JDK-8227745 >>>>>>>>>> >>>> >> http://cr.openjdk.java.net/~rrich/webrevs/2019/8214584/experiment_v1.pa >> tc >>>> h >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> From claes.redestad at oracle.com Tue Mar 31 14:30:32 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 31 Mar 2020 16:30:32 +0200 Subject: RFR: 8241837: Cleanup stringStream usage in ObjectSynchronizer In-Reply-To: <22ad475a-52a7-9778-42a7-6d184c0c78ac@oracle.com> References: <4873f9b5-d9ec-242a-fed5-46a3b7905b1a@oracle.com> <22ad475a-52a7-9778-42a7-6d184c0c78ac@oracle.com> Message-ID: On 2020-03-31 16:09, Daniel D. Daugherty wrote: > On 3/30/20 7:02 PM, Claes Redestad wrote: >> Hi, >> >> this removes one unused stringStream, and refactors two other >> so that they're only defined in the slow path error where they are used: >> >> http://cr.openjdk.java.net/~redestad/8241837/open.00/ > > src/hotspot/share/runtime/synchronizer.cpp > ??? No comments. > > Just FYI: ObjectMonitor::is_busy_to_string() calls stringStream::base() > so the cleanup is handled by the stringStream destructor. I didn't want > ResourceMarks. > > Thumbs up. Thanks! /Claes > > Dan From daniel.daugherty at oracle.com Tue Mar 31 14:26:56 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 31 Mar 2020 10:26:56 -0400 Subject: RFR (S): 8239895: assert(_stack_base != 0LL) failed: Sanity check In-Reply-To: <04e25b32-69e4-8740-68b7-f02b9a713690@oracle.com> References: <04e25b32-69e4-8740-68b7-f02b9a713690@oracle.com> Message-ID: <490be892-73e7-13cd-3070-f073d86cec07@oracle.com> On 3/31/20 1:18 AM, David Holmes wrote: > Bug: https://bugs.openjdk.java.net/browse/JDK-8239895 > webrev: http://cr.openjdk.java.net/~dholmes/8239895/webrev/ src/hotspot/cpu/aarch64/frame_aarch64.cpp ??? No comments. src/hotspot/cpu/arm/frame_arm.cpp ??? No comments. src/hotspot/cpu/ppc/frame_ppc.cpp ??? No comments. src/hotspot/cpu/s390/frame_s390.cpp ??? No comments. src/hotspot/cpu/x86/frame_x86.cpp ??? No comments. src/hotspot/os_cpu/solaris_sparc/os_solaris_sparc.cpp ??? No comments. src/hotspot/os_cpu/solaris_x86/os_solaris_x86.cpp ??? No comments. src/hotspot/share/runtime/thread.cpp ??? No comments (8241043 backout). src/hotspot/share/runtime/thread.hpp ??? No comments (new func and 8241043 backout). Thumbs up. More below... > Prior to JDK-8238988 there were uses of stack_base() which checked it > was initialized, and there was a raw use of _stack_base in > on_local_stack() that did not need it to be initialized (because it > may not be). After JDK-8238988 both cases call is_in_stack_range() > which uses stack_base() and so asserts that the stack base is > initialized in all cases. This leads to the assertion failures when > the _stack_base is not initialised. The fix has three parts: > > 1. Rename is_in_full_stack to is_in_full_stack_checked - as it checks > _stack_base is initialized via an assertion. > > 2. Add a new is_in_full_stack which doesn't use any assertions. > > 3. Update all the uses of stack_base() prior to JDK-8238988 that were > changed to call is_in_full_stack, to now call > is_in_full_stack_checked. There are not many of them. (The corollary > to that is that all old calls to on_local_stack() call the new > unchecked is_in_full_stack.) > > Here's the webrev for JDK-8238988 for comparison if desired: > > http://cr.openjdk.java.net/~dholmes/8238988/webrev/ The above link doesn't work. I used this one: http://cr.openjdk.java.net/~dholmes/8238988/webrev.v3/ > > I also backed out the assertion changes that I made under: > > https://bugs.openjdk.java.net/browse/JDK-8241043 > > as they were failing due to the use of get_thread_name(). I've filed a > separate RFE for that issue: > > https://bugs.openjdk.java.net/browse/JDK-8241403 > > Testing: tiers 1 - 3 The test failures that we've been seeing are happening in Tier5 and Tier6. Dan > > Thanks, > David > From goetz.lindenmaier at sap.com Tue Mar 31 14:28:26 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 31 Mar 2020 14:28:26 +0000 Subject: RFR (T): 8241852: Cleanup error message generation in LinkResolver::resolve_field In-Reply-To: <7a03b072-c3fd-b901-e351-5e2981f6aa4d@oracle.com> References: <4035d995-f33f-8ecb-4801-1b472c003f16@oracle.com> <49dfc4d7-1c39-3218-0e06-4e8e7b6a75a1@oracle.com> <7a03b072-c3fd-b901-e351-5e2981f6aa4d@oracle.com> Message-ID: Hi, I think you can remove the ResourceMark altogether. Instead of ss.as_string() use ss.base(). The char* is allocated with malloc (this was changed previously). The internal string is zero terminated. The StringStream object itself is on the stack. Best regards, Goetz. > -----Original Message----- > From: hotspot-runtime-dev bounces at openjdk.java.net> On Behalf Of Claes Redestad > Sent: Monday, March 30, 2020 6:45 PM > To: daniel.daugherty at oracle.com; Hotspot dev runtime dev at openjdk.java.net> > Subject: Re: RFR (T): 8241852: Cleanup error message generation in > LinkResolver::resolve_field > > On 2020-03-30 18:34, Daniel D. Daugherty wrote: > > On 3/30/20 12:27 PM, Claes Redestad wrote: > >> Hi, > >> > >> a couple of places in LinkResolver::resolve_field where we allocate > >> ResourceMark and stringStream in a wider scope than necessary. > Narrowing > >> down the scope where they are declared in a similar way as done > >> consistently in all other places in LinkResolver is a small and trivial > >> optimization: > >> > >> http://cr.openjdk.java.net/~redestad/8241852/open.00/ > > > > src/hotspot/share/interpreter/linkResolver.cpp > > ??? Please update the copyright year before you push. > > > > Will do. > > > > > Thumbs up! I concur that this is a trivial fix. > > Thanks! > > /Claes From claes.redestad at oracle.com Tue Mar 31 14:58:57 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 31 Mar 2020 16:58:57 +0200 Subject: RFR (T): 8241852: Cleanup error message generation in LinkResolver::resolve_field In-Reply-To: References: <4035d995-f33f-8ecb-4801-1b472c003f16@oracle.com> <49dfc4d7-1c39-3218-0e06-4e8e7b6a75a1@oracle.com> <7a03b072-c3fd-b901-e351-5e2981f6aa4d@oracle.com> Message-ID: Hi Goetz, On 2020-03-31 16:28, Lindenmaier, Goetz wrote: > Hi, > > I think you can remove the ResourceMark altogether. > Instead of ss.as_string() use ss.base(). > The char* is allocated with malloc (this was changed previously). > The internal string is zero terminated. > The StringStream object itself is on the stack. yes, if we think it's cleaner we could go that route. While these ResourceMarks and stringStreams are all off the hot path, compilers might generate more compact code if we removed the ResourceMarks and used ss.base(). I've already pushed this enhancement, though, but such an cleanup sounds like something that should be evaluated and applied in more places as a standalone RFE. Thanks! /Claes > > Best regards, > Goetz. > >> -----Original Message----- >> From: hotspot-runtime-dev > bounces at openjdk.java.net> On Behalf Of Claes Redestad >> Sent: Monday, March 30, 2020 6:45 PM >> To: daniel.daugherty at oracle.com; Hotspot dev runtime > dev at openjdk.java.net> >> Subject: Re: RFR (T): 8241852: Cleanup error message generation in >> LinkResolver::resolve_field >> >> On 2020-03-30 18:34, Daniel D. Daugherty wrote: >>> On 3/30/20 12:27 PM, Claes Redestad wrote: >>>> Hi, >>>> >>>> a couple of places in LinkResolver::resolve_field where we allocate >>>> ResourceMark and stringStream in a wider scope than necessary. >> Narrowing >>>> down the scope where they are declared in a similar way as done >>>> consistently in all other places in LinkResolver is a small and trivial >>>> optimization: >>>> >>>> http://cr.openjdk.java.net/~redestad/8241852/open.00/ >>> >>> src/hotspot/share/interpreter/linkResolver.cpp >>> ??? Please update the copyright year before you push. >>> >> >> Will do. >> >>> >>> Thumbs up! I concur that this is a trivial fix. >> >> Thanks! >> >> /Claes From christoph.langer at sap.com Tue Mar 31 19:23:52 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Tue, 31 Mar 2020 19:23:52 +0000 Subject: RFR(S): 8241101: [s390] jtreg test failure after JDK-8238696: not conformant features string In-Reply-To: <6C62F39C-87D9-4C29-A7EA-5EFD9321C8FC@sap.com> References: <30182338-771E-4A43-849C-977C9B7C27F9@sap.com> <6C62F39C-87D9-4C29-A7EA-5EFD9321C8FC@sap.com> Message-ID: Hi Lutz, I also agree with Martin's idea of fixing this. I think it's better than changing existing feature strings just for the sake of quiescing a test. I would even suggest to have a second Array of features in CPUInfo that contains all values in lowercase to be queried by CPUInfo::hasFeature. We should give it a try. Best regards Christoph > -----Original Message----- > From: hotspot-runtime-dev bounces at openjdk.java.net> On Behalf Of Schmidt, Lutz > Sent: Dienstag, 31. M?rz 2020 15:59 > To: Doerr, Martin ; Thomas St?fe > > Cc: hotspot-runtime-dev at openjdk.java.net > Subject: [CAUTION] Re: RFR(S): 8241101: [s390] jtreg test failure after JDK- > 8238696: not conformant features string > > Hi Martin, > your extended check would heal the issue we saw with the original feature > string on s390. So far, so good. I can?t judge if other tests can live with that. > Regards, > Lutz > > From: "Doerr, Martin (martin.doerr at sap.com)" > Date: Tuesday, 31. March 2020 at 15:54 > To: Thomas St?fe , Lutz Schmidt > > Cc: Vladimir Ivanov , "hotspot-runtime- > dev at openjdk.java.net" > Subject: RE: RFR(S): 8241101: [s390] jtreg test failure after JDK-8238696: not > conformant features string > > But what about > features.contains(feature) || features.contains(feature.toLowerCase()) > ? > > Is it still problematic if we just support both, lower case and exact match? > > Best regards, > Martin > > > From: Thomas St?fe > Sent: Dienstag, 31. M?rz 2020 15:51 > To: Schmidt, Lutz > Cc: Doerr, Martin ; Vladimir Ivanov > ; hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR(S): 8241101: [s390] jtreg test failure after JDK-8238696: not > conformant features string > > Okay. The change looks good to me. > > Cheers, Thomas > > On Tue, Mar 31, 2020 at 3:38 PM Schmidt, Lutz > > wrote: > Thomas, Martin, > > thank you for looking into this. > > Generally, I'm very much in favor of the "fix the root cause" approach. In this > particular case, however, I'm afraid of side effects and excessive effort. > CPUInfo.hasFeature() is called in quite a few places, see the attached text > file for reference. > > My proposed change, on the other hand, is very constrained in scope: just > s390, just a few string edits. The risk to break anything is extremely low. > Therefore, I would like to stick with the proposed change. > > Thanks, > Lutz > > > On 31.03.20, 11:33, "Doerr, Martin" > > wrote: > > Hi Thomas, > > I agree with that. > > test/lib/sun/hotspot/cpuinfo/CPUInfo.java uses > return features.contains(feature.toLowerCase()); > in "hasFeature" which is not nicely designed. > > If it is required to lower the case (for some reason), I think it should better > use > features.contains(feature) || features.contains(feature.toLowerCase()) > > Or even better if we don't need toLowerCase at all and fix the strings. > > Anyway, the 2nd part of the change (SHA, AES, ...) looks good to me. > > Best regards, > Martin > > > > -----Original Message----- > > From: hotspot-runtime-dev > bounces at openjdk.java.net> On > Behalf Of Thomas St?fe > > Sent: Dienstag, 31. M?rz 2020 10:01 > > To: Schmidt, Lutz > > > > Cc: hotspot-runtime-dev at openjdk.java.net dev at openjdk.java.net> > > Subject: Re: RFR(S): 8241101: [s390] jtreg test failure after JDK-8238696: > not > > conformant features string > > > > HI Lutz, > > > > Should we not better adjust the jtreg test? > > > > Cheers, Thomas > > > > On Tue, Mar 31, 2020 at 9:22 AM Schmidt, Lutz > > > > wrote: > > > > > Dear All, > > > > > > may I please request reviews for this fix, adjusting the CPU features > > > string such that jtreg tests, in particular a newly introduced one, pass > OK. > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8241101 > > > Webrev: https://cr.openjdk.java.net/~lucy/webrevs/8241101.00/ > > > > > > Thank you! > > > Lutz > > > > > > > > > > > > > From lutz.schmidt at sap.com Tue Mar 31 20:13:45 2020 From: lutz.schmidt at sap.com (Schmidt, Lutz) Date: Tue, 31 Mar 2020 20:13:45 +0000 Subject: RFR(S): 8241101: [s390] jtreg test failure after JDK-8238696: not conformant features string In-Reply-To: References: <30182338-771E-4A43-849C-977C9B7C27F9@sap.com> <6C62F39C-87D9-4C29-A7EA-5EFD9321C8FC@sap.com> Message-ID: <36E440E1-FB96-478F-AF07-2AB4953514E8@sap.com> Hi Christoph, I'm sorry but I strongly disagree. This feature string is solely used as supplemental information in the hs_err* file. At least I couldn't find any other usage. And for that use case, it simply doesn't matter if the string is all lowercase. Best, Lutz ?On 31.03.20, 21:23, "Langer, Christoph" wrote: Hi Lutz, I also agree with Martin's idea of fixing this. I think it's better than changing existing feature strings just for the sake of quiescing a test. I would even suggest to have a second Array of features in CPUInfo that contains all values in lowercase to be queried by CPUInfo::hasFeature. We should give it a try. Best regards Christoph > -----Original Message----- > From: hotspot-runtime-dev bounces at openjdk.java.net> On Behalf Of Schmidt, Lutz > Sent: Dienstag, 31. M?rz 2020 15:59 > To: Doerr, Martin ; Thomas St?fe > > Cc: hotspot-runtime-dev at openjdk.java.net > Subject: [CAUTION] Re: RFR(S): 8241101: [s390] jtreg test failure after JDK- > 8238696: not conformant features string > > Hi Martin, > your extended check would heal the issue we saw with the original feature > string on s390. So far, so good. I can?t judge if other tests can live with that. > Regards, > Lutz > > From: "Doerr, Martin (martin.doerr at sap.com)" > Date: Tuesday, 31. March 2020 at 15:54 > To: Thomas St?fe , Lutz Schmidt > > Cc: Vladimir Ivanov , "hotspot-runtime- > dev at openjdk.java.net" > Subject: RE: RFR(S): 8241101: [s390] jtreg test failure after JDK-8238696: not > conformant features string > > But what about > features.contains(feature) || features.contains(feature.toLowerCase()) > ? > > Is it still problematic if we just support both, lower case and exact match? > > Best regards, > Martin > > > From: Thomas St?fe > Sent: Dienstag, 31. M?rz 2020 15:51 > To: Schmidt, Lutz > Cc: Doerr, Martin ; Vladimir Ivanov > ; hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR(S): 8241101: [s390] jtreg test failure after JDK-8238696: not > conformant features string > > Okay. The change looks good to me. > > Cheers, Thomas > > On Tue, Mar 31, 2020 at 3:38 PM Schmidt, Lutz > > wrote: > Thomas, Martin, > > thank you for looking into this. > > Generally, I'm very much in favor of the "fix the root cause" approach. In this > particular case, however, I'm afraid of side effects and excessive effort. > CPUInfo.hasFeature() is called in quite a few places, see the attached text > file for reference. > > My proposed change, on the other hand, is very constrained in scope: just > s390, just a few string edits. The risk to break anything is extremely low. > Therefore, I would like to stick with the proposed change. > > Thanks, > Lutz > > > On 31.03.20, 11:33, "Doerr, Martin" > > wrote: > > Hi Thomas, > > I agree with that. > > test/lib/sun/hotspot/cpuinfo/CPUInfo.java uses > return features.contains(feature.toLowerCase()); > in "hasFeature" which is not nicely designed. > > If it is required to lower the case (for some reason), I think it should better > use > features.contains(feature) || features.contains(feature.toLowerCase()) > > Or even better if we don't need toLowerCase at all and fix the strings. > > Anyway, the 2nd part of the change (SHA, AES, ...) looks good to me. > > Best regards, > Martin > > > > -----Original Message----- > > From: hotspot-runtime-dev > bounces at openjdk.java.net> On > Behalf Of Thomas St?fe > > Sent: Dienstag, 31. M?rz 2020 10:01 > > To: Schmidt, Lutz > > > > Cc: hotspot-runtime-dev at openjdk.java.net dev at openjdk.java.net> > > Subject: Re: RFR(S): 8241101: [s390] jtreg test failure after JDK-8238696: > not > > conformant features string > > > > HI Lutz, > > > > Should we not better adjust the jtreg test? > > > > Cheers, Thomas > > > > On Tue, Mar 31, 2020 at 9:22 AM Schmidt, Lutz > > > > wrote: > > > > > Dear All, > > > > > > may I please request reviews for this fix, adjusting the CPU features > > > string such that jtreg tests, in particular a newly introduced one, pass > OK. > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8241101 > > > Webrev: https://cr.openjdk.java.net/~lucy/webrevs/8241101.00/ > > > > > > Thank you! > > > Lutz > > > > > > > > > > > > > From david.holmes at oracle.com Tue Mar 31 21:34:20 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 1 Apr 2020 07:34:20 +1000 Subject: RFR (S): 8239895: assert(_stack_base != 0LL) failed: Sanity check In-Reply-To: <490be892-73e7-13cd-3070-f073d86cec07@oracle.com> References: <04e25b32-69e4-8740-68b7-f02b9a713690@oracle.com> <490be892-73e7-13cd-3070-f073d86cec07@oracle.com> Message-ID: <2423bf9a-ee1c-4eac-3fc2-2ac5efb9afa9@oracle.com> Hi Dan, On 1/04/2020 12:26 am, Daniel D. Daugherty wrote: > On 3/31/20 1:18 AM, David Holmes wrote: >> Bug: https://bugs.openjdk.java.net/browse/JDK-8239895 >> webrev: http://cr.openjdk.java.net/~dholmes/8239895/webrev/ > > src/hotspot/cpu/aarch64/frame_aarch64.cpp > ??? No comments. > > src/hotspot/cpu/arm/frame_arm.cpp > ??? No comments. > > src/hotspot/cpu/ppc/frame_ppc.cpp > ??? No comments. > > src/hotspot/cpu/s390/frame_s390.cpp > ??? No comments. > > src/hotspot/cpu/x86/frame_x86.cpp > ??? No comments. > > src/hotspot/os_cpu/solaris_sparc/os_solaris_sparc.cpp > ??? No comments. > > src/hotspot/os_cpu/solaris_x86/os_solaris_x86.cpp > ??? No comments. > > src/hotspot/share/runtime/thread.cpp > ??? No comments (8241043 backout). > > src/hotspot/share/runtime/thread.hpp > ??? No comments (new func and 8241043 backout). > > > Thumbs up. Thanks for the review! > More below... > > >> Prior to JDK-8238988 there were uses of stack_base() which checked it >> was initialized, and there was a raw use of _stack_base in >> on_local_stack() that did not need it to be initialized (because it >> may not be). After JDK-8238988 both cases call is_in_stack_range() >> which uses stack_base() and so asserts that the stack base is >> initialized in all cases. This leads to the assertion failures when >> the _stack_base is not initialised. The fix has three parts: >> >> 1. Rename is_in_full_stack to is_in_full_stack_checked - as it checks >> _stack_base is initialized via an assertion. >> >> 2. Add a new is_in_full_stack which doesn't use any assertions. >> >> 3. Update all the uses of stack_base() prior to JDK-8238988 that were >> changed to call is_in_full_stack, to now call >> is_in_full_stack_checked. There are not many of them. (The corollary >> to that is that all old calls to on_local_stack() call the new >> unchecked is_in_full_stack.) >> >> Here's the webrev for JDK-8238988 for comparison if desired: >> >> http://cr.openjdk.java.net/~dholmes/8238988/webrev/ > > The above link doesn't work. I used this one: > > http://cr.openjdk.java.net/~dholmes/8238988/webrev.v3/ Oops sorry about that. >> >> I also backed out the assertion changes that I made under: >> >> https://bugs.openjdk.java.net/browse/JDK-8241043 >> >> as they were failing due to the use of get_thread_name(). I've filed a >> separate RFE for that issue: >> >> https://bugs.openjdk.java.net/browse/JDK-8241403 >> >> Testing: tiers 1 - 3 > > The test failures that we've been seeing are happening in Tier5 and Tier6. My testing was to check nothing new was unexpectedly broken. The test failures we occasionally see in tiers 5 and 6 do not reproduce easily and I've never had a failure occur in numerous such runs when trying to reproduce this originally. I will submit a tier 5/6 run while waiting for a second review. Thanks, David > Dan > > >> >> Thanks, >> David >> >