From suenaga at oss.nttdata.com Wed Jul 1 01:53:55 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Wed, 1 Jul 2020 10:53:55 +0900 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <2ea2f5d8-5c43-a7d3-d525-cd44c4009d25@oss.nttdata.com> <65889ab4-6b7e-86fb-59cc-012520b83138@oracle.com> <4b7df6f8-c8f7-fedf-6366-b885d2235b58@oss.nttdata.com> Message-ID: <74c05dfd-ffb2-b73d-66e5-dc98752729f5@oss.nttdata.com> Hi, I uploaded new webrev. Could review again? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>> >>>>> 820 assert(SafepointSynchronize::is_at_safepoint() || >>>>> 821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>> 822 current_thread == java_thread->active_handshaker(), >>>>> 823 "at safepoint / handshake or target thread is suspended"); >>>>> >>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? This function (JvmtiEnvBase::get_stack_trace()) can be called to get own stack trace. For example, we can call GetStackTrace() for current thread at JVMTI event. So I changed assert as below: ``` 820 assert(current_thread == java_thread || 821 SafepointSynchronize::is_at_safepoint() || 822 current_thread == java_thread->active_handshaker(), 823 "call by myself / at safepoint / at handshake"); ``` Thanks, Yasumasa On 2020/07/01 8:48, David Holmes wrote: > Hi Yasumasa, > > On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >> Hi David, >> >>>>> 1271???? ResourceMark rm; >>>>> >>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>> >>>>> ???? ResourceMark rm(_calling_thread); >> >> If so, we can call make_local() in L1272 without JavaThread (or we can pass current thread to make_local()). Is it right? >> >> ``` >> 1271???? ResourceMark rm; >> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >> 1273??????????????????????????? jt, thread_oop); >> ``` > > Sorry I got confused, _calling_thread may not be the current thread as we could be executing the handshake in the target thread itself. So the ResourceMark is correct as-is (implicitly for current thread). > > The argument to fill_frames will be used in the jvmtiStackInfo and passed back to the _calling_thread, so it must be created via make_local(_calling_thread, ...) as you presently have. > > Thanks, > David > >> Thanks, >> >> Yasumasa >> >> >> On 2020/07/01 7:05, David Holmes wrote: >>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>> Hi David, >>>> >>>> Thank you for reviewing! I will update new webrev tomorrow. >>>> >>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>> >>>>> ? 498 class VM_GetAllStackTraces : public VM_Operation { >>>>> ? 499 private: >>>>> ? 500?? JavaThread *_calling_thread; >>>>> ? 501?? jint _final_thread_count; >>>>> ? 502?? MultipleStackTracesCollector _collector; >>>>> >>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>> >>>> I'm not sure what does mean "embedded". >>>> Is it ok as below? >>>> >>>> ``` >>>> class MultipleStackTracesCollector { >>>> ??? : >>>> } >>>> >>>> class GetAllStackTraces : public VM_Operation { >>>> ?? private: >>>> ???? MultipleStackTracesCollector _collector; >>>> } >>>> ``` >>> >>> Yes that I what I meant. >>> >>> Thanks, >>> David >>> ----- >>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2020/06/30 22:22, David Holmes wrote: >>>>> Hi Yasumasa, >>>>> >>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>> Hi David, Serguei, >>>>>> >>>>>> I updated webrev for 8242428. Could you review again? >>>>>> This change migrate to use direct handshake for GetStackTrace() and GetThreadListStackTraces() (when thread_count == 1). >>>>>> >>>>>> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>> >>>>> This looks really good now! I only have a few nits below. There is one thing I don't like about it but it requires a change to the main Handshake logic to address - in JvmtiEnv::GetThreadListStackTraces you have to create a ThreadsListHandle to convert the jthread to a JavaThread, but then the Handshake::execute_direct creates another ThreadsListHandle internally. That's a waste. I will discuss with Robbin and file a RFE to have an overload of execute_direct that takes an existing TLH. Actually it's worse than that because we have another TLH in use at the entry point for the JVMTI functions, so I think there may be some scope for simplifying the use of TLH instances - future RFE. >>>>> >>>>> --- >>>>> >>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>> >>>>> ??451?? GetStackTraceClosure(JvmtiEnv *env, jint start_depth, jint max_count, >>>>> ??452??????????????????????? jvmtiFrameInfo* frame_buffer, jint* count_ptr) >>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>> ??454?????? _env(env), _start_depth(start_depth), _max_count(max_count), >>>>> ??455?????? _frame_buffer(frame_buffer), _count_ptr(count_ptr), >>>>> ??456?????? _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>> >>>>> Nit: can you do one initializer per line please. >>>>> >>>>> This looks wrong: >>>>> >>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>> >>>>> ??498 class VM_GetAllStackTraces : public VM_Operation { >>>>> ??499 private: >>>>> ??500?? JavaThread *_calling_thread; >>>>> ??501?? jint _final_thread_count; >>>>> ??502?? MultipleStackTracesCollector _collector; >>>>> >>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>> >>>>> 481?? MultipleStackTracesCollector(JvmtiEnv *env, jint max_frame_count) { >>>>> ??482???? _env = env; >>>>> ??483???? _max_frame_count = max_frame_count; >>>>> ??484???? _frame_count_total = 0; >>>>> ??485???? _head = NULL; >>>>> ??486???? _stack_info = NULL; >>>>> ??487???? _result = JVMTI_ERROR_NONE; >>>>> ??488?? } >>>>> >>>>> As you are touching this can you change it to use an initializer list as you did for the HandshakeClosure, and please keep one item per line. >>>>> >>>>> --- >>>>> >>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>> >>>>> ??820?? assert(SafepointSynchronize::is_at_safepoint() || >>>>> ??821????????? java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>> ??822????????? current_thread == java_thread->active_handshaker(), >>>>> ??823????????? "at safepoint / handshake or target thread is suspended"); >>>>> >>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>> >>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>> 1269 >>>>> 1270?? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>> >>>>> You can use thread_oop in line 1270. >>>>> >>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>> 1273??????????????????????????? jt, thread_oop); >>>>> >>>>> It is frustrating that this entire call chain started with a jthread reference, which we converted to a JavaThread, only to eventually need to convert it back to a jthread! I think there is some scope for simplification here but not as part of this change. >>>>> >>>>> 1271???? ResourceMark rm; >>>>> >>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>> >>>>> ???? ResourceMark rm(_calling_thread); >>>>> >>>>> --- >>>>> >>>>> Please add @bug lines to the tests. >>>>> >>>>> I'm still pondering the test logic but wanted to send this now. >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>>> VM_GetThreadListStackTrace (for GetThreadListStackTraces) and VM_GetAllStackTraces (for GetAllStackTraces) have inherited VM_GetMultipleStackTraces VM operation which provides the feature to generate jvmtiStackInfo. I modified? VM_GetMultipleStackTraces to a normal C++ class to share with HandshakeClosure for GetThreadListStackTraces (GetSingleStackTraceClosure). >>>>>> >>>>>> Also I added new testcases which test GetThreadListStackTraces() with thread_count == 1 and with all threads. >>>>>> >>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/nsk/jdwp. >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> Please review this change: >>>>>>> >>>>>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>> >>>>>>> This change replace following VM operations to direct handshake. >>>>>>> >>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>> ??- VM_GetThreadListStackTraces (GetThreadListStackTrace()) >>>>>>> ??- VM_GetCurrentLocation >>>>>>> >>>>>>> GetThreadListStackTrace() uses direct handshake if thread count == 1. In other case (thread count > 1), it would be performed as VM operation (VM_GetThreadListStackTraces). >>>>>>> Caller of VM_GetCurrentLocation (JvmtiEnvThreadState::reset_current_location()) might be called at safepoint. So I added safepoint check in its caller. >>>>>>> >>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/ns >>>>>>> k/jdwp. >>>>>>> >>>>>>> Also I tested it on submit repo, then it has execution error (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) due to dependency error. So I think it does not occur by this change. >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa From daniil.x.titov at oracle.com Wed Jul 1 03:00:22 2020 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Tue, 30 Jun 2020 20:00:22 -0700 Subject: RFR(S): 8247533: SA stack walking sometimes fails with sun.jvm.hotspot.debugger.DebuggerException: get_thread_regs failed for a lwp In-Reply-To: <4B569840-8BA6-481C-B76A-533C59C93FE5@oracle.com> References: <5e3727cd-dee9-abdf-fe7a-657fe6a4c845@oracle.com> <4B569840-8BA6-481C-B76A-533C59C93FE5@oracle.com> Message-ID: <9942F460-797D-4C5C-98D3-53057EBA2FCA@oracle.com> Hi Chris, The fix looks good to me. Best regards, Daniil On 6/25/20 13:29, Chris Plummer wrote: > Ping. I still need one more review for this. There was one updated > webev. I list it below so you don't need to dig it up in the long > email thread: >> I've updated with webrev based on the new finding that a JavaThread >> cannot be on the ThreadList after its OS thread has been destroyed >> since the JavaThread removes itself from the ThreadList, and >> therefore must be running on its OS thread. The logic of the fix is >> unchanged from the first webrev, but I updated the comments to better >> reflect what is going on. I also updated the CR: >> >> https://bugs.openjdk.java.net/browse/JDK-8247533 >> http://cr.openjdk.java.net/~cjplummer/8247533/webrev.01/index.html > > thanks, > > Chris > > On 6/17/20 1:34 PM, Chris Plummer wrote: >> Hello, >> >> Please help review the following: >> >> https://bugs.openjdk.java.net/browse/JDK-8247533 >> http://cr.openjdk.java.net/~cjplummer/8247533/webrev.00/index.html >> >> The CR contains all the needed details. Here's a summary of changes >> in each file: >> >> src/jdk.hotspot.agent/linux/native/libsaproc/LinuxDebuggerLocal.cpp >> src/jdk.hotspot.agent/macosx/native/libsaproc/MacosxDebuggerLocal.m >> src/jdk.hotspot.agent/windows/native/libsaproc/sawindbg.cpp >> -Instead of throwing an exception when the OS ThreadID is invalid, >> print a warning. >> >> src/jdk.hotspot.agent/linux/native/libsaproc/ps_proc.c >> -Improve a print_debug message >> >> src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/debugger/bsd/BsdThread.java >> >> src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/debugger/linux/LinuxThread.java >> >> src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/debugger/windbg/amd64/WindbgAMD64Thread.java >> >> -Deal with the array of registers read in being null due to the OS >> ThreadID not being valid. >> >> src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/debugger/bsd/BsdDebuggerLocal.java >> >> src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/debugger/linux/LinuxDebuggerLocal.java >> >> -Fix issue with "sun.jvm.hotspot.debugger.DebuggerException" >> appearing twice when printing the exception. >> >> thanks, >> >> Chris > > From daniil.x.titov at oracle.com Wed Jul 1 05:24:46 2020 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Tue, 30 Jun 2020 22:24:46 -0700 Subject: RFR(M): 8244383: jhsdb/HeapDumpTestWithActiveProcess.java fails with "AssertionFailure: illegal bci" In-Reply-To: References: <28e1b453-e1ea-0a1c-0ae0-0494b52f4b71@oracle.com> <6efbc900-732f-ee8b-5561-f9a813ebfeca@oracle.com> Message-ID: Hi Chris, The fix, in general, looks good to me. Some small comments for src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/amd64/AMD64CurrentFrameGuess.java: 1) Imports at lines 30 and 35 are not used and could be removed. 30 import sun.jvm.hotspot.interpreter.*; 35 import sun.jvm.hotspot.utilities.*; 2) Local variable "end" defined at line 189 is not used. 189 Address end = sp.addOffsetTo(regionInBytesToSearch); No new webrev is required. Thanks, Daniil ?On 6/25/20, 1:55 PM, "serviceability-dev on behalf of Chris Plummer" wrote: Ping #2. I still need one more reviewer (Thanks for the review, Dan). I updated the webrev based on Dan's comments: http://cr.openjdk.java.net/~cjplummer/8244383/webrev.01/ I can still make the simplification mentioned below if necessary. thanks, Chris On 6/23/20 11:29 AM, Chris Plummer wrote: > Ping! > > If this fix is too complicated, there is a simplification I can make, > but at the cost of abandoning some attempts to determine the current > frame when this error condition pops up. At the start of > validateInterpreterFrame() it attempts to verify that the frame is > valid by verifying that frame->method and frame->bcp are valid. This > part is pretty simple. The complicated part is everything that follows > if the verification fails. It attempts to error correct the situation > by looking at various register contents and stack contents. I could > just abandon this complicated code and return false if frame->method > and frame->bcp don't check out. Upon return, the caller's code would > be simplified to: > > if (validateInterpreterFrame(sp, fp, pc)) { > return true; // We're done. setValues() has been called > for valid interpreter frame. > } else { > return checkLastJavaSP(); > } > > So there's still a chance we can determine a valid current frame if > "last java frame" has been setup. However, if not setup we would not > be able to. This is where the complicated code in > validateInterpreterFrame() is useful because it can usually determine > the current frame, even if "last java frame" is not setup, but it's > rare enough that we run into this situation that I think failing to > get the current frame is ok. > > So if I can get a couple promises for reviews if I make this change, > I'll go ahead and do it and send out a new RFR. > > thanks, > > Chris > > On 6/18/20 5:54 PM, Chris Plummer wrote: >> [I've added runtime-dev to this SA review since understanding >> interpreter invokes (code generated by >> TemplateInterpreterGenerator::generate_normal_entry()) and stack >> walking is probably more important than understanding SA.] >> >> Hello, >> >> Please help review the following: >> >> https://bugs.openjdk.java.net/browse/JDK-8244383 >> http://cr.openjdk.java.net/~cjplummer/8244383/webrev.00/index.html >> >> The crux of the bug is when doing stack walking the topmost frame is >> in an inconsistent state because we are in the middle of pushing a >> new interpreter frame. Basically we are executing code generated by >> TemplateInterpreterGenerator::generate_normal_entry(). Since the PC >> register is in this code, SA assumes the topmost frame is an >> interpreter frame. >> >> The first issue with this interpreter frame assumption is if we >> haven't actually pushed the frame yet, then the current frame is the >> caller's frame, and could be compiled. But since SA thinks it's >> interpreted, later on it tries to convert the frame->bcp to a BCI, >> but frame->bcp is only valid for interpreter frames. Thus the >> "illegal BCI" failures. If the previous frame happened to be >> interpreted, then the existing SA code works fine. >> >> The other state of frame pushing that was problematic was when the >> new frame had been pushed, but frame->method and frame->bcp were not >> setup yet. This also would lead to "illegal BCI" later on because >> garbage would be stored in these locations. >> >> Fixing the above problems requires trying to determine the state of >> the frame push through a series of checks, and then adapting what is >> considered to be the current frame based on the outcome of the >> checks. The first things checked is that frame->method is valid (we >> can successfully instantiate a wrapper for the Method* without >> failure) and that frame->bcp is within the method. If both these pass >> then we can use the frame as-is. >> >> If the above checks fail, then we try to determine whether the issue >> is that the frame is not yet pushed and the current frame is actually >> compiled, or the frame has been pushed but not yet initialized. This >> is done by first getting the return address from the stack or RAX >> (it's location depends on how far along we are in the entry code) and >> comparing this to what is stored in frame->return_addr. If they are >> the same, then we have pushed the frame but not yet initialized it. >> In this case we use the previous frame (senderSP() and senderFP()) as >> the current frame since the current frame is not yet initialized. If >> the return address check fails, then we assume the new frame is not >> yet pushed, and and treat the current frame as compiled, even though >> PC points into the interpreter (we replace PC with RAX in this case). >> >> Comments in the code pretty well explain all the above, so it is >> probably easier to follow the logic in the code along with the >> comments rather than apply my above description to the code. >> >> I should add that it's very rare that we ever get into this special >> error handling code. This bug was very hard to reproduce initially. I >> was only able to make progress with reproducing and debugging by >> inserting delay loops in various spots in the code generated by >> TemplateInterpreterGenerator::generate_normal_entry(). By doing this >> I was able to reproduce the issue quite easily and hit all the logic >> in the new code I've added. >> >> The fix is basically entirely contained within >> AMD64CurrentFrameGuess.java. The rest of the changes are minor: >> >> src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/amd64/AMD64CurrentFrameGuess.java >> >> -Main fix for CR >> >> src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/x86/X86Frame.java >> >> -Added getInterpreterFrameBCP(), which is now needed by >> AMD64CurrentFrameGuess.java >> -I also simplified some code by using the existing >> getInterpreterFrameMethod() >> rather than replicating inline what it does. >> >> src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/debugger/bsd/amd64/BsdAMD64CFrame.java >> >> -I noticed the windows version of this code had some extra checks >> that were missing >> from the bsd version. I then looked at the linux version, but it had >> been heavily modified >> a short while back to leverage DWARF info to determine frames. So I >> looked at the previous >> rev and it too had these extra checks. I decided to add them to the >> BSD port. I'm not sure >> if it helps at all, but it certainly doesn't seem to do any harm. >> >> thanks, >> >> Chris >> > > From daniil.x.titov at oracle.com Wed Jul 1 17:05:45 2020 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Wed, 01 Jul 2020 10:05:45 -0700 Subject: RFR: 8227337: javax/management/remote/mandatory/connection/ReconnectTest.java NoSuchObjectException no such object in table Message-ID: <119571EF-7387-4DD1-9284-24821E98135C@oracle.com> Hi Paul, Thank you for reviewing this change. You are right, in most test configurations test.timeout.factor is greater than 1.0. Best regards, Daniil ?On 6/30/20, 8:50 AM, "Hohensee, Paul" wrote: The JBS issue is non-public, but this looks fine. I assume you set test.timeout.factor to something larger than 1.0 when you run MultiThreadDeadLockTest. Thanks, Paul ?On 6/29/20, 12:43 PM, "serviceability-dev on behalf of Daniil Titov" wrote: Please review the change that fixes an intermittent tests failure. The tests javax/management/remote/mandatory/connection/ReconnectTest.java and javax/management/remote/mandatory/connection/MultiThreadDeadLockTest.java use specific settings for server timeout that in some cases (e.g. when the test is run with -Xcomp) result in JMX server connection timeout thread unexports the remote object while the client connection is still in the progress. Below is an example of a such stacktrace: java.rmi.NoSuchObjectException: no such object in table at java.rmi/sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:303) at java.rmi/sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:279) at java.rmi/sun.rmi.server.UnicastRef.invoke(UnicastRef.java:164) at jdk.remoteref/jdk.jmx.remote.internal.rmi.PRef.invoke(Unknown Source) at java.management.rmi/javax.management.remote.rmi.RMIConnectionImpl_Stub.getConnectionId(RMIConnectionImpl_Stub.java:318) at java.management.rmi/javax.management.remote.rmi.RMIConnector.getConnectionId(RMIConnector.java:385) at java.management.rmi/javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:347) at java.management/javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270) at MultiThreadDeadLockTest.main(MultiThreadDeadLockTest.java:87) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:564) at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127) at java.base/java.lang.Thread.run(Thread.java:832) The fix adjusts the server timeout the tests use for "test.timeout.factor" system property. Testing: Mach5 tests are in the progress. [1] https://cr.openjdk.java.net/~dtitov/8227337/webrev.01/ [2] https://bugs.openjdk.java.net/browse/JDK-8227337 Thanks, Daniil From leonid.mesnik at oracle.com Wed Jul 1 17:48:32 2020 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Wed, 1 Jul 2020 10:48:32 -0700 Subject: RFR (Preliminary): 8248194: Need better support for running SA tests on core files In-Reply-To: <84c27b68-d90b-c0c6-4e65-1a97aa9f2126@oracle.com> References: <04c841a1-1e38-155a-5d94-5d0e5a32c708@oracle.com> <8BD2E0F8-EE7B-4308-A058-2EC235D5212F@oracle.com> <2ac19e8a-0f0d-a215-6f92-748fd8c0c5ab@oracle.com> <6980c0f3-5ae7-a813-3adb-16cac40fe7f6@oracle.com> <84c27b68-d90b-c0c6-4e65-1a97aa9f2126@oracle.com> Message-ID: Good idea, I think. So we will have core utils which allow to ensure that core is dumped and help to find them. Leonid > On Jun 29, 2020, at 5:10 PM, Chris Plummer wrote: > > Hi Leonid, > > I'm starting to think that this should all go in a new CoreUtils.java file. I experimented with moving getCoreFileLocation() to OutputAnalyzer. It worked, but one adjustment I had to make is also moving SATestUtils.unzipCores() there also and make it private (no one else calls it). But that just got me thinking that maybe CoreUtils.java would be a better solution. I think I would put the addCoreUlimitCommand() API discussed below there also. What do you think? > > thanks, > > Chris > > On 6/28/20 5:29 PM, Chris Plummer wrote: >> Hi Leonid, >> >> I think getCoreFileLocation() can simply move to OutputAnalyzer. No need for it to be in SAUtils and be passed the String argument that comes from OutputAnalyzer.getOutput(). >> >> For the ulimit support, how about if in ProcessTools I add: >> >> public static ProcessBuilder addCoreUlimitCommand(ProcessBuilder pb); >> >> All the ulimit logic would move there from SATestUtils. It's straight forward to use this API in LingeredApp and TestJmapCore. For ClhsdbCDSCore I'll need to rework the getTestJvmCommandlineWithPrefix() code a bit, since it creates a pb, but doesn't save it. It only uses it to get the cmd String. >> >> Also, there's one new finding since I sent out the review. I found the following in CiReplayBase.java: >> >> // lets search few possible locations using process output and return existing location >> private String getCoreFileLocation(String crashOutputString) { >> >> This is identical to the code I pulled from ClhsdbCDSCore and is now in SATestUtils.parseCoreFileLocationFromOutput(). Although this is in the compiler directory, it is in fact an SA test that uses clhsdb, although directly via the CLHSDB class rather than through "jhsdb clhsdb". >> >> This also explains why ClhsdbCDSCore had some logic to move and rename the core file to "cds_core_file". I removed this logic because it seemed unnecessary, but for CiReplayBase.java it needs to be in a known location so SABase.java can find it. It's still fine for ClhsdbCDSCore to not do the rename, and renaming is independent of any code that locates the core file. >> >> I'm not going to update CiReplayBase.java as part of these changes because the two tests that use it both have issues. TestSAServer is problem listed, and when I removed it from the problem list it failed with every run on every platform. There's also TestSAClient, but it relies on client VM, which we don't support anymore. So with neither of these tests running, I'd rather not introduce changes I can't really test. >> >> However, there was something good that came out of the CiReplayBase.java discovery. I had previously noted that ClhsdbCDSCore is excluded from running on windows. When I removed the @requires for this, it failed for a reason I didn't quite understand. The complaint was about the path to java.exe when running the process that was suppose to crash, although the path looked fine. However, I found that TestSAServer ran fine on Windows, even though it was basically the process launching code for causing the crash. I looked closer and found one difference. In getTestJvmCommandlineWithPrefix(), which both tests have, the CiReplayBase version had some extra code for Windows: >> >> return new String[]{"sh", "-c", prefix >> + (Platform.isWindows() ? cmd.replace('\\', '/').replace(";", "\\;").replace("|", "\\|") : cmd)}; >> >> So on Windows it's doing a path conversion. Once I started doing the same with ClhsdbCDSCore, it started to run fine on Windwos also. >> >> thanks, >> >> Chris >> >> On 6/26/20 8:42 PM, Chris Plummer wrote: >>> Hi Leonid, >>> >>> On 6/26/20 7:51 PM, Leonid Mesnik wrote: >>>> Hi >>>> >>>> The idea basically looks good. I think it just make a sense to polish it a little bit to hide "sh" usage from test and get core from OutputAnalyzer. >>> Ok, I'll look into both of those. >>>> Also, there is a 'CrashApp' in ClhsdbCDSCore.java. Makes it sense to unify it with LingeredApp crasher? Currently, it uses Unsafe to crash application. >>> Yes, I purposely didn't not make that change. My main goal with the LingeredApp changes is to make it easier to make existing LingeredApp SA tests run on both a live process and on a core, and my main goal with ClhsdbCDSCore and TestJmapCore was to move the core finding code and ulimit code to a common location that could be reused by other tests. >>> >>> Keep in mind that ClhsdbLauncher and LingeredApp are independent of each other. You can have a LingeredApp tests that use or don't use ClhsdbLauncher, and you can have a non-LingeredApp tests that use or don't use ClhsdbLauncher. So I didn't want to go down the path of changing ClhsdbCDSCore (a non LingeredApp test) to use LingeredApp. Likewise I did not change TestJmapCore to use LingeredApp or ClhsdbLauncher. Possibly there is good reason to convert some of the tests to start using LingeredApp and/or ClhsdbLauncher, but that should be done under a separate RFE. >>> >>>> >>>> Also, crashes are used in other tests, I see some implementations in >>>> open/test/hotspot/jtreg/vmTestbase/vm/share/vmcrasher >>> I don't see vmcrasher being used by any tests. In any case, my first attempt went down the Unsafe path to produce a crash. The issue is that it forces every user of LingeredApp to include the @module for Unsafe. I also tried using a WhiteBox API. That was worse, also requiring every user of LingeredApp to include an @module, plus the tests that actually want to cause a crash need to @build WhiteBox.java and then do the classfile install. It also required additional module related hacks in LingeredApp. The issue with my current solution is how to get libLingeredApp.c to compile has not been settled on. I'm still waiting for an answer from the build team. >>>> >>>> So it would be nice to have some common way to crash hotspot. >>> I can see possibly moving the crashing code out of LingeredApp and into a native lib that non-LingeredApp tests can use, although that really is just a very small part of the changes to LingeredApp. For the most part the changes would look the same except you would call a different API to cause the crash. >>>> >>>> Leonid >>>> >>> Thanks for having a look. >>> >>> Chris >>>>> On Jun 25, 2020, at 2:41 PM, Chris Plummer wrote: >>>>> >>>>> Hello, >>>>> >>>>> Please help with a preliminary review of changes to add better support for writing SA tests that work on core files: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8248194 >>>>> http://cr.openjdk.java.net/~cjplummer/8248194/webrev.00/index.html >>>>> >>>>> As pointed out, this is a preliminary review. I suspect there will be some feedback for changes/improvements. Also, I still need to work out a final solution for how to get LingeredApp to produce a crash. What I currently have works but is somewhat of a hack w.r.t. the makefile change, so you can ignore the makefiile change for now. I'm working on a more proper solution with the build team. >>>>> >>>>> As outlined in the CR, these are the 3 main goals of this CR: >>>>> >>>>> 1. SATestUtils should include support for finding the core file. This includes parsing the output of the crashed process to locate where the core file was saved, and returning this location to the user. >>>>> >>>>> 2. SATestUtils should include support for adding the "ulimit -c unlimited" prefix to the command that will produce the core file, allowing the overriding of any lower limit so we can be sure the core file will be produced. >>>>> >>>>> 3. LingeredApp should include support for producing a core file. >>>>> >>>>> As proof of concept for these 3 changes in test library support, I'm updating the following 3 tests: >>>>> >>>>> ClhsdbCDSCore.java: Use the SATestUtils support listed above. This test does not use LingeredApp, so those improvements don't apply. >>>>> >>>>> TestJmapCore.java: Use the SATestUtils support listed above. This test does not use LingeredApp, so those improvements don't apply. >>>>> >>>>> ClhsdbFindPC.java: Use all the above features, including having LingeredApp produce a core file. This is the only test modified to start testing on core files that didn't previously do so. It still also tests on a live process. >>>>> >>>>> In the future more Clhsdb tests will be converted to work on core files in a manner similar to ClhsdbFindPC. >>>>> >>>>> The new SATestUtils code is borrowed from (more like ripped out of) ClhsdbCDSCore.java and TestJmapCore.java. They both had a lot of code dedicated to finding the core file and also applying "ulimit -c unlimitted" if necessary, but didn't do so in quite the same way. Now both these tests share code in SATestUtils.java. One thing I did drop is TestJmapCore.java use of ":KILLED_PID" in the output to help find the core file. It's no longer necessary based on the smarter core locating code I pulled from ClhsdbCDSCore.java. >>>>> >>>>> thanks, >>>>> >>>>> Chris >>> >> >> > > From leonid.mesnik at oracle.com Wed Jul 1 18:05:50 2020 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Wed, 1 Jul 2020 11:05:50 -0700 Subject: RFR: 8248658: Remove vmTestbase/vm/share/vmcrasher Message-ID: <94402391-8422-474F-980C-56E7D6DC2A42@oracle.com> Hi Could you please review fix which removes unused code. webrev: http://cr.openjdk.java.net/~lmesnik/8248658/webrev.00/ bug: https://bugs.openjdk.java.net/browse/JDK-8248658 -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Wed Jul 1 18:32:23 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 1 Jul 2020 11:32:23 -0700 Subject: RFR: 8248658: Remove vmTestbase/vm/share/vmcrasher In-Reply-To: <94402391-8422-474F-980C-56E7D6DC2A42@oracle.com> References: <94402391-8422-474F-980C-56E7D6DC2A42@oracle.com> Message-ID: <03971bd5-0f4f-492c-2a10-222807e61247@oracle.com> Looks good. Chris On 7/1/20 11:05 AM, Leonid Mesnik wrote: > Hi > > Could you please review fix which removes unused code. > > webrev: http://cr.openjdk.java.net/~lmesnik/8248658/webrev.00/ > bug: https://bugs.openjdk.java.net/browse/JDK-8248658 From serguei.spitsyn at oracle.com Wed Jul 1 19:56:08 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 1 Jul 2020 12:56:08 -0700 Subject: RFR: 8248658: Remove vmTestbase/vm/share/vmcrasher In-Reply-To: <03971bd5-0f4f-492c-2a10-222807e61247@oracle.com> References: <94402391-8422-474F-980C-56E7D6DC2A42@oracle.com> <03971bd5-0f4f-492c-2a10-222807e61247@oracle.com> Message-ID: Hi Leonid, LGTM++ Thanks, Serguei On 7/1/20 11:32, Chris Plummer wrote: > Looks good. > > Chris > > On 7/1/20 11:05 AM, Leonid Mesnik wrote: >> Hi >> >> Could you please review fix which removes unused code. >> >> webrev: http://cr.openjdk.java.net/~lmesnik/8248658/webrev.00/ >> bug: https://bugs.openjdk.java.net/browse/JDK-8248658 > From david.holmes at oracle.com Thu Jul 2 06:05:25 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 2 Jul 2020 16:05:25 +1000 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: <74c05dfd-ffb2-b73d-66e5-dc98752729f5@oss.nttdata.com> References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <2ea2f5d8-5c43-a7d3-d525-cd44c4009d25@oss.nttdata.com> <65889ab4-6b7e-86fb-59cc-012520b83138@oracle.com> <4b7df6f8-c8f7-fedf-6366-b885d2235b58@oss.nttdata.com> <74c05dfd-ffb2-b73d-66e5-dc98752729f5@oss.nttdata.com> Message-ID: Hi Yasumasa, On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: > Hi, > > I uploaded new webrev. Could review again? > > ? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ Updates look fine - thanks. One minor nit: 1274 _collector.allocate_and_fill_stacks(1); 1275 _collector.set_result(JVMTI_ERROR_NONE); In the other places where you use _collector you rely on result being initialized to JVMTI_ERROR_NONE, rather than setting it directly after allocate_and_fill_stacks(). > >>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>> >>>>>> ? 820?? assert(SafepointSynchronize::is_at_safepoint() || >>>>>> ? 821????????? java_thread->is_thread_fully_suspended(false, >>>>>> &debug_bits) || >>>>>> ? 822????????? current_thread == java_thread->active_handshaker(), >>>>>> ? 823????????? "at safepoint / handshake or target thread is >>>>>> suspended"); >>>>>> >>>>>> I don't think the suspension check is necessary, as even if the >>>>>> target is suspended we must still be at a safepoint or in a >>>>>> handshake with it. Makes me wonder if we used to allow a racy >>>>>> stacktrace operation on a suspended thread, assuming it would >>>>>> remain suspended? > > This function (JvmtiEnvBase::get_stack_trace()) can be called to get own > stack trace. For example, we can call GetStackTrace() for current thread > at JVMTI event. > So I changed assert as below: > > ``` > ?820?? assert(current_thread == java_thread || > ?821????????? SafepointSynchronize::is_at_safepoint() || > ?822????????? current_thread == java_thread->active_handshaker(), > ?823????????? "call by myself / at safepoint / at handshake"); > ``` Yep good catch. I hope current tests caught that. Speaking of tests ... In the native code I think you need to check the success of all JNI methods that can throw exceptions - otherwise I believe the tests may trigger warnings if -Xcheck:jni is used with them. See for example: test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp In the Java code the target thread: 45 public void run() { 46 try { 47 synchronized (lock) { 48 lock.wait(); 49 System.out.println("OK"); 50 } is potentially susceptible to spurious wakeups. Using a CountDownLatch would be robust. Thanks, David ----- > > Thanks, > > Yasumasa > > > On 2020/07/01 8:48, David Holmes wrote: >> Hi Yasumasa, >> >> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>> Hi David, >>> >>>>>> 1271???? ResourceMark rm; >>>>>> >>>>>> IIUC at this point the _calling_thread is the current thread, so >>>>>> we can use: >>>>>> >>>>>> ???? ResourceMark rm(_calling_thread); >>> >>> If so, we can call make_local() in L1272 without JavaThread (or we >>> can pass current thread to make_local()). Is it right? >>> >>> ``` >>> 1271???? ResourceMark rm; >>> 1272 >>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>> thread_oop), >>> 1273??????????????????????????? jt, thread_oop); >>> ``` >> >> Sorry I got confused, _calling_thread may not be the current thread as >> we could be executing the handshake in the target thread itself. So >> the ResourceMark is correct as-is (implicitly for current thread). >> >> The argument to fill_frames will be used in the jvmtiStackInfo and >> passed back to the _calling_thread, so it must be created via >> make_local(_calling_thread, ...) as you presently have. >> >> Thanks, >> David >> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2020/07/01 7:05, David Holmes wrote: >>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>> Hi David, >>>>> >>>>> Thank you for reviewing! I will update new webrev tomorrow. >>>>> >>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>> >>>>>> ? 498 class VM_GetAllStackTraces : public VM_Operation { >>>>>> ? 499 private: >>>>>> ? 500?? JavaThread *_calling_thread; >>>>>> ? 501?? jint _final_thread_count; >>>>>> ? 502?? MultipleStackTracesCollector _collector; >>>>>> >>>>>> You can't have a StackObj as a member of another class like that >>>>>> as it may not be on the stack. I think >>>>>> MultipleStackTracesCollector should not extend any allocation >>>>>> class, and should always be embedded directly in another class. >>>>> >>>>> I'm not sure what does mean "embedded". >>>>> Is it ok as below? >>>>> >>>>> ``` >>>>> class MultipleStackTracesCollector { >>>>> ??? : >>>>> } >>>>> >>>>> class GetAllStackTraces : public VM_Operation { >>>>> ?? private: >>>>> ???? MultipleStackTracesCollector _collector; >>>>> } >>>>> ``` >>>> >>>> Yes that I what I meant. >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>> Hi Yasumasa, >>>>>> >>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>> Hi David, Serguei, >>>>>>> >>>>>>> I updated webrev for 8242428. Could you review again? >>>>>>> This change migrate to use direct handshake for GetStackTrace() >>>>>>> and GetThreadListStackTraces() (when thread_count == 1). >>>>>>> >>>>>>> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>> >>>>>> This looks really good now! I only have a few nits below. There is >>>>>> one thing I don't like about it but it requires a change to the >>>>>> main Handshake logic to address - in >>>>>> JvmtiEnv::GetThreadListStackTraces you have to create a >>>>>> ThreadsListHandle to convert the jthread to a JavaThread, but then >>>>>> the Handshake::execute_direct creates another ThreadsListHandle >>>>>> internally. That's a waste. I will discuss with Robbin and file a >>>>>> RFE to have an overload of execute_direct that takes an existing >>>>>> TLH. Actually it's worse than that because we have another TLH in >>>>>> use at the entry point for the JVMTI functions, so I think there >>>>>> may be some scope for simplifying the use of TLH instances - >>>>>> future RFE. >>>>>> >>>>>> --- >>>>>> >>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>> >>>>>> ??451?? GetStackTraceClosure(JvmtiEnv *env, jint start_depth, jint >>>>>> max_count, >>>>>> ??452??????????????????????? jvmtiFrameInfo* frame_buffer, jint* >>>>>> count_ptr) >>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>> ??454?????? _env(env), _start_depth(start_depth), >>>>>> _max_count(max_count), >>>>>> ??455?????? _frame_buffer(frame_buffer), _count_ptr(count_ptr), >>>>>> ??456?????? _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>> >>>>>> Nit: can you do one initializer per line please. >>>>>> >>>>>> This looks wrong: >>>>>> >>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>> >>>>>> ??498 class VM_GetAllStackTraces : public VM_Operation { >>>>>> ??499 private: >>>>>> ??500?? JavaThread *_calling_thread; >>>>>> ??501?? jint _final_thread_count; >>>>>> ??502?? MultipleStackTracesCollector _collector; >>>>>> >>>>>> You can't have a StackObj as a member of another class like that >>>>>> as it may not be on the stack. I think >>>>>> MultipleStackTracesCollector should not extend any allocation >>>>>> class, and should always be embedded directly in another class. >>>>>> >>>>>> 481?? MultipleStackTracesCollector(JvmtiEnv *env, jint >>>>>> max_frame_count) { >>>>>> ??482???? _env = env; >>>>>> ??483???? _max_frame_count = max_frame_count; >>>>>> ??484???? _frame_count_total = 0; >>>>>> ??485???? _head = NULL; >>>>>> ??486???? _stack_info = NULL; >>>>>> ??487???? _result = JVMTI_ERROR_NONE; >>>>>> ??488?? } >>>>>> >>>>>> As you are touching this can you change it to use an initializer >>>>>> list as you did for the HandshakeClosure, and please keep one item >>>>>> per line. >>>>>> >>>>>> --- >>>>>> >>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>> >>>>>> ??820?? assert(SafepointSynchronize::is_at_safepoint() || >>>>>> ??821????????? java_thread->is_thread_fully_suspended(false, >>>>>> &debug_bits) || >>>>>> ??822????????? current_thread == java_thread->active_handshaker(), >>>>>> ??823????????? "at safepoint / handshake or target thread is >>>>>> suspended"); >>>>>> >>>>>> I don't think the suspension check is necessary, as even if the >>>>>> target is suspended we must still be at a safepoint or in a >>>>>> handshake with it. Makes me wonder if we used to allow a racy >>>>>> stacktrace operation on a suspended thread, assuming it would >>>>>> remain suspended? >>>>>> >>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>> 1269 >>>>>> 1270?? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>> >>>>>> You can use thread_oop in line 1270. >>>>>> >>>>>> 1272 >>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>> thread_oop), >>>>>> 1273??????????????????????????? jt, thread_oop); >>>>>> >>>>>> It is frustrating that this entire call chain started with a >>>>>> jthread reference, which we converted to a JavaThread, only to >>>>>> eventually need to convert it back to a jthread! I think there is >>>>>> some scope for simplification here but not as part of this change. >>>>>> >>>>>> 1271???? ResourceMark rm; >>>>>> >>>>>> IIUC at this point the _calling_thread is the current thread, so >>>>>> we can use: >>>>>> >>>>>> ???? ResourceMark rm(_calling_thread); >>>>>> >>>>>> --- >>>>>> >>>>>> Please add @bug lines to the tests. >>>>>> >>>>>> I'm still pondering the test logic but wanted to send this now. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>>> VM_GetThreadListStackTrace (for GetThreadListStackTraces) and >>>>>>> VM_GetAllStackTraces (for GetAllStackTraces) have inherited >>>>>>> VM_GetMultipleStackTraces VM operation which provides the feature >>>>>>> to generate jvmtiStackInfo. I modified? VM_GetMultipleStackTraces >>>>>>> to a normal C++ class to share with HandshakeClosure for >>>>>>> GetThreadListStackTraces (GetSingleStackTraceClosure). >>>>>>> >>>>>>> Also I added new testcases which test GetThreadListStackTraces() >>>>>>> with thread_count == 1 and with all threads. >>>>>>> >>>>>>> This change has been tested in serviceability/jvmti >>>>>>> serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi >>>>>>> vmTestbase/nsk/jdwp. >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>> Hi all, >>>>>>>> >>>>>>>> Please review this change: >>>>>>>> >>>>>>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>> ?? webrev: >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>> >>>>>>>> This change replace following VM operations to direct handshake. >>>>>>>> >>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>> ??- VM_GetThreadListStackTraces (GetThreadListStackTrace()) >>>>>>>> ??- VM_GetCurrentLocation >>>>>>>> >>>>>>>> GetThreadListStackTrace() uses direct handshake if thread count >>>>>>>> == 1. In other case (thread count > 1), it would be performed as >>>>>>>> VM operation (VM_GetThreadListStackTraces). >>>>>>>> Caller of VM_GetCurrentLocation >>>>>>>> (JvmtiEnvThreadState::reset_current_location()) might be called >>>>>>>> at safepoint. So I added safepoint check in its caller. >>>>>>>> >>>>>>>> This change has been tested in serviceability/jvmti >>>>>>>> serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi >>>>>>>> vmTestbase/ns >>>>>>>> k/jdwp. >>>>>>>> >>>>>>>> Also I tested it on submit repo, then it has execution error >>>>>>>> (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) due to >>>>>>>> dependency error. So I think it does not occur by this change. >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa From suenaga at oss.nttdata.com Thu Jul 2 09:19:55 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Thu, 2 Jul 2020 18:19:55 +0900 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <2ea2f5d8-5c43-a7d3-d525-cd44c4009d25@oss.nttdata.com> <65889ab4-6b7e-86fb-59cc-012520b83138@oracle.com> <4b7df6f8-c8f7-fedf-6366-b885d2235b58@oss.nttdata.com> <74c05dfd-ffb2-b73d-66e5-dc98752729f5@oss.nttdata.com> Message-ID: <763793b1-9506-44c6-0681-1c71c44a44fd@oss.nttdata.com> Hi David, I upload new webrev. Could you review again? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ On 2020/07/02 15:05, David Holmes wrote: > Hi Yasumasa, > > On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >> Hi, >> >> I uploaded new webrev. Could review again? >> >> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ > > Updates look fine - thanks. > > One minor nit: > > 1274???? _collector.allocate_and_fill_stacks(1); > 1275???? _collector.set_result(JVMTI_ERROR_NONE); > > In the other places where you use _collector you rely on result being initialized to JVMTI_ERROR_NONE, rather than setting it directly after allocate_and_fill_stacks(). Fixed. >>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>> >>>>>>> ? 820?? assert(SafepointSynchronize::is_at_safepoint() || >>>>>>> ? 821????????? java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>> ? 822????????? current_thread == java_thread->active_handshaker(), >>>>>>> ? 823????????? "at safepoint / handshake or target thread is suspended"); >>>>>>> >>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >> >> This function (JvmtiEnvBase::get_stack_trace()) can be called to get own stack trace. For example, we can call GetStackTrace() for current thread at JVMTI event. >> So I changed assert as below: >> >> ``` >> ??820?? assert(current_thread == java_thread || >> ??821????????? SafepointSynchronize::is_at_safepoint() || >> ??822????????? current_thread == java_thread->active_handshaker(), >> ??823????????? "call by myself / at safepoint / at handshake"); >> ``` > > Yep good catch. I hope current tests caught that. They would be tested in vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ (own call stacks), and getstacktr003 (call stacks in other thread). > Speaking of tests ... > > In the native code I think you need to check the success of all JNI methods that can throw exceptions - otherwise I believe the tests may trigger warnings if -Xcheck:jni is used with them. See for example: > > test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp I updated testcases to check JNI and JVMTI function calls. > In the Java code the target thread: > > ? 45???? public void run() { > ? 46?????? try { > ? 47?????????? synchronized (lock) { > ? 48?????????????? lock.wait(); > ? 49?????????????? System.out.println("OK"); > ? 50?????????? } > > is potentially susceptible to spurious wakeups. Using a CountDownLatch would be robust. Fixed. Thanks, Yasumasa > Thanks, > David > ----- > >> >> Thanks, >> >> Yasumasa >> >> >> On 2020/07/01 8:48, David Holmes wrote: >>> Hi Yasumasa, >>> >>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>> Hi David, >>>> >>>>>>> 1271???? ResourceMark rm; >>>>>>> >>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>> >>>>>>> ???? ResourceMark rm(_calling_thread); >>>> >>>> If so, we can call make_local() in L1272 without JavaThread (or we can pass current thread to make_local()). Is it right? >>>> >>>> ``` >>>> 1271???? ResourceMark rm; >>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>> 1273??????????????????????????? jt, thread_oop); >>>> ``` >>> >>> Sorry I got confused, _calling_thread may not be the current thread as we could be executing the handshake in the target thread itself. So the ResourceMark is correct as-is (implicitly for current thread). >>> >>> The argument to fill_frames will be used in the jvmtiStackInfo and passed back to the _calling_thread, so it must be created via make_local(_calling_thread, ...) as you presently have. >>> >>> Thanks, >>> David >>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2020/07/01 7:05, David Holmes wrote: >>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>> Hi David, >>>>>> >>>>>> Thank you for reviewing! I will update new webrev tomorrow. >>>>>> >>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>> >>>>>>> ? 498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>> ? 499 private: >>>>>>> ? 500?? JavaThread *_calling_thread; >>>>>>> ? 501?? jint _final_thread_count; >>>>>>> ? 502?? MultipleStackTracesCollector _collector; >>>>>>> >>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>> >>>>>> I'm not sure what does mean "embedded". >>>>>> Is it ok as below? >>>>>> >>>>>> ``` >>>>>> class MultipleStackTracesCollector { >>>>>> ??? : >>>>>> } >>>>>> >>>>>> class GetAllStackTraces : public VM_Operation { >>>>>> ?? private: >>>>>> ???? MultipleStackTracesCollector _collector; >>>>>> } >>>>>> ``` >>>>> >>>>> Yes that I what I meant. >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>> Hi Yasumasa, >>>>>>> >>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>>> Hi David, Serguei, >>>>>>>> >>>>>>>> I updated webrev for 8242428. Could you review again? >>>>>>>> This change migrate to use direct handshake for GetStackTrace() and GetThreadListStackTraces() (when thread_count == 1). >>>>>>>> >>>>>>>> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>> >>>>>>> This looks really good now! I only have a few nits below. There is one thing I don't like about it but it requires a change to the main Handshake logic to address - in JvmtiEnv::GetThreadListStackTraces you have to create a ThreadsListHandle to convert the jthread to a JavaThread, but then the Handshake::execute_direct creates another ThreadsListHandle internally. That's a waste. I will discuss with Robbin and file a RFE to have an overload of execute_direct that takes an existing TLH. Actually it's worse than that because we have another TLH in use at the entry point for the JVMTI functions, so I think there may be some scope for simplifying the use of TLH instances - future RFE. >>>>>>> >>>>>>> --- >>>>>>> >>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>> >>>>>>> ??451?? GetStackTraceClosure(JvmtiEnv *env, jint start_depth, jint max_count, >>>>>>> ??452??????????????????????? jvmtiFrameInfo* frame_buffer, jint* count_ptr) >>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>> ??454?????? _env(env), _start_depth(start_depth), _max_count(max_count), >>>>>>> ??455?????? _frame_buffer(frame_buffer), _count_ptr(count_ptr), >>>>>>> ??456?????? _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>> >>>>>>> Nit: can you do one initializer per line please. >>>>>>> >>>>>>> This looks wrong: >>>>>>> >>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>> >>>>>>> ??498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>> ??499 private: >>>>>>> ??500?? JavaThread *_calling_thread; >>>>>>> ??501?? jint _final_thread_count; >>>>>>> ??502?? MultipleStackTracesCollector _collector; >>>>>>> >>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>> >>>>>>> 481?? MultipleStackTracesCollector(JvmtiEnv *env, jint max_frame_count) { >>>>>>> ??482???? _env = env; >>>>>>> ??483???? _max_frame_count = max_frame_count; >>>>>>> ??484???? _frame_count_total = 0; >>>>>>> ??485???? _head = NULL; >>>>>>> ??486???? _stack_info = NULL; >>>>>>> ??487???? _result = JVMTI_ERROR_NONE; >>>>>>> ??488?? } >>>>>>> >>>>>>> As you are touching this can you change it to use an initializer list as you did for the HandshakeClosure, and please keep one item per line. >>>>>>> >>>>>>> --- >>>>>>> >>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>> >>>>>>> ??820?? assert(SafepointSynchronize::is_at_safepoint() || >>>>>>> ??821????????? java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>> ??822????????? current_thread == java_thread->active_handshaker(), >>>>>>> ??823????????? "at safepoint / handshake or target thread is suspended"); >>>>>>> >>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>> >>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>> 1269 >>>>>>> 1270?? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>> >>>>>>> You can use thread_oop in line 1270. >>>>>>> >>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>> 1273??????????????????????????? jt, thread_oop); >>>>>>> >>>>>>> It is frustrating that this entire call chain started with a jthread reference, which we converted to a JavaThread, only to eventually need to convert it back to a jthread! I think there is some scope for simplification here but not as part of this change. >>>>>>> >>>>>>> 1271???? ResourceMark rm; >>>>>>> >>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>> >>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>> >>>>>>> --- >>>>>>> >>>>>>> Please add @bug lines to the tests. >>>>>>> >>>>>>> I'm still pondering the test logic but wanted to send this now. >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> ----- >>>>>>>> VM_GetThreadListStackTrace (for GetThreadListStackTraces) and VM_GetAllStackTraces (for GetAllStackTraces) have inherited VM_GetMultipleStackTraces VM operation which provides the feature to generate jvmtiStackInfo. I modified? VM_GetMultipleStackTraces to a normal C++ class to share with HandshakeClosure for GetThreadListStackTraces (GetSingleStackTraceClosure). >>>>>>>> >>>>>>>> Also I added new testcases which test GetThreadListStackTraces() with thread_count == 1 and with all threads. >>>>>>>> >>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/nsk/jdwp. >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>> Hi all, >>>>>>>>> >>>>>>>>> Please review this change: >>>>>>>>> >>>>>>>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>> >>>>>>>>> This change replace following VM operations to direct handshake. >>>>>>>>> >>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>> ??- VM_GetThreadListStackTraces (GetThreadListStackTrace()) >>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>> >>>>>>>>> GetThreadListStackTrace() uses direct handshake if thread count == 1. In other case (thread count > 1), it would be performed as VM operation (VM_GetThreadListStackTraces). >>>>>>>>> Caller of VM_GetCurrentLocation (JvmtiEnvThreadState::reset_current_location()) might be called at safepoint. So I added safepoint check in its caller. >>>>>>>>> >>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/ns >>>>>>>>> k/jdwp. >>>>>>>>> >>>>>>>>> Also I tested it on submit repo, then it has execution error (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) due to dependency error. So I think it does not occur by this change. >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa From chris.plummer at oracle.com Thu Jul 2 14:21:14 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 2 Jul 2020 07:21:14 -0700 Subject: RFR (L): 8248194: Need better support for running SA tests on core files Message-ID: <5760b18e-7ba2-2fdd-94af-1e555cef74cc@oracle.com> [Note, the following changes only impact serviceability tests, but I am adding some test library code to assist with creating and finding core files, and I thought others on hotspot-dev might have an interest in that.] Hello, Please help review the following changes to add better support for writing SA tests that work on core files: https://bugs.openjdk.java.net/browse/JDK-8248194 http://cr.openjdk.java.net/~cjplummer/8248194/webrev.01/index.html As outlined in the CR, these are the 3 main goals of this CR: 1. Add a shared API for locating the path to the core file. This includes parsing the output of the crashed process to locate where the core file was saved, and returning this location to the user. This API will be placed in the new CoreUtils class. 2. Add a shared API to support adding the "ulimit -c unlimited" prefix to the command that will produce the core file, allowing the overriding of any lower limit so we can be sure the core file will be produced. This API will also be placed in the new CoreUtils class. 3. LingeredApp should include support for producing a core file. As proof of concept for these improvements, I'm updating the following 3 tests to make use of them: ClhsdbCDSCore.java and TestJmapCore.java: Use the CoreUtils support listed above. These tests do not use LingeredApp, so those improvements don't apply. ClhsdbFindPC.java: Use all the above new features, including having LingeredApp produce a core file. This is the only test modified to start testing on core files that didn't previously do so. It still also tests on a live process. In the future more Clhsdb tests will be converted to work on core files in a manner similar to ClhsdbFindPC. The new CoreUtils code is borrowed from (more like ripped out of) ClhsdbCDSCore.java and TestJmapCore.java. They both had a lot of code dedicated to finding the core file and also applying "ulimit -c unlimited", but didn't do so in quite the same way. Now both these tests share code in CoreUtils.java. One thing I did drop is TestJmapCore.java use of ":KILLED_PID" in the output to help find the core file. It's no longer necessary based on the smarter core locating code I pulled from ClhsdbCDSCore.java. One other improvement was to enable windows support for ClhsdbCDSCore. The only reason it was not enabled previously is because the author couldn't figure out how to properly generate the command for the process that produces the core. Since it is launched using "sh -c", the path has to be converted to use forward slashes. This is now done in CoreUtils. One other change in ClhsdbCDSCore is that it no longer renames the core file to a well known name in the cwd. This was unnecessary. It originated in code from ciReplayBase.java, which does have a reason to do the rename, but ClhsdbCDSCore does not. The new libLingeredApp.c relies on JDK-8248667 [1] being in place in order to have the build system properly compile it. JDK-8248667 will be reviewed separately on build-dev and pushed just before the changes for this CR. thanks, Chris [1] https://bugs.openjdk.java.net/browse/JDK-8248667 From daniel.daugherty at oracle.com Thu Jul 2 22:16:54 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 2 Jul 2020 18:16:54 -0400 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: <763793b1-9506-44c6-0681-1c71c44a44fd@oss.nttdata.com> References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <2ea2f5d8-5c43-a7d3-d525-cd44c4009d25@oss.nttdata.com> <65889ab4-6b7e-86fb-59cc-012520b83138@oracle.com> <4b7df6f8-c8f7-fedf-6366-b885d2235b58@oss.nttdata.com> <74c05dfd-ffb2-b73d-66e5-dc98752729f5@oss.nttdata.com> <763793b1-9506-44c6-0681-1c71c44a44fd@oss.nttdata.com> Message-ID: <5107c478-1de2-7a49-ee71-dfad6a99a667@oracle.com> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: > Hi David, > > I upload new webrev. Could you review again? > > http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ src/hotspot/share/prims/jvmtiEnv.cpp ??? L1542: ??? // Get stack trace with handshake ??????? nit - please add a period at the end. ??? L1591: ??? *stack_info_ptr = op.stack_info(); ??????? The return parameter should not be touched unless the return ??????? code in 'err' == JVMTI_ERROR_NONE. ??? old L1582: ? if (err == JVMTI_ERROR_NONE) { ??????? Please restore this check. The return parameter should not ??????? be touched unless the return code in 'err' == JVMTI_ERROR_NONE. ??? L1272: ? if (!jt->is_exiting() && (thread_oop != NULL)) { ??????? nit - extra parens around the second expression. src/hotspot/share/prims/jvmtiEnvBase.cpp ??? old L1532: ? _result = JVMTI_ERROR_THREAD_NOT_ALIVE; ??????? This deletion of the _result field threw me for a minute and then ??? ? ? I figured out that the field is init to JVMTI_ERROR_THREAD_NOT_ALIVE ??????? in the constructor. ??? L1553: ? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { ??????? nit - extra parens around the second expression. src/hotspot/share/prims/jvmtiEnvBase.hpp ??? No comments. src/hotspot/share/runtime/vmOperations.hpp ??? No comments. test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java ??? No comments. test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java ??? L64: ??????? startSignal.countDown(); ??????? I was expecting this to be a call to await() instead of ??????? countDown(). What am I missing here? ??????? I think this test might be passing by accident right now, but... test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c ??? L92: ? jthreads = (jthread *)malloc(sizeof(jthread) * num_threads); ??????? You don't check for malloc() failure. ??????? 'jthreads' is allocated but never freed. test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c ??? L91: ? result = (*jvmti)->SuspendThread(jvmti, thread); ??????? Why are you suspending the thread? GetAllStackTraces() and ??????? GetThreadListStackTraces() do not require the target thread(s) ??????? to be suspend. ??????? If you decide not to SuspendThread, then you don't need the ??????? AddCapabilities or the ResumeThread calls. Dan > > On 2020/07/02 15:05, David Holmes wrote: >> Hi Yasumasa, >> >> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>> Hi, >>> >>> I uploaded new webrev. Could review again? >>> >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >> >> Updates look fine - thanks. >> >> One minor nit: >> >> 1274???? _collector.allocate_and_fill_stacks(1); >> 1275???? _collector.set_result(JVMTI_ERROR_NONE); >> >> In the other places where you use _collector you rely on result being >> initialized to JVMTI_ERROR_NONE, rather than setting it directly >> after allocate_and_fill_stacks(). > > Fixed. > > >>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>> >>>>>>>> ? 820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>> ? 821 java_thread->is_thread_fully_suspended(false, >>>>>>>> &debug_bits) || >>>>>>>> ? 822????????? current_thread == java_thread->active_handshaker(), >>>>>>>> ? 823????????? "at safepoint / handshake or target thread is >>>>>>>> suspended"); >>>>>>>> >>>>>>>> I don't think the suspension check is necessary, as even if the >>>>>>>> target is suspended we must still be at a safepoint or in a >>>>>>>> handshake with it. Makes me wonder if we used to allow a racy >>>>>>>> stacktrace operation on a suspended thread, assuming it would >>>>>>>> remain suspended? >>> >>> This function (JvmtiEnvBase::get_stack_trace()) can be called to get >>> own stack trace. For example, we can call GetStackTrace() for >>> current thread at JVMTI event. >>> So I changed assert as below: >>> >>> ``` >>> ??820?? assert(current_thread == java_thread || >>> ??821????????? SafepointSynchronize::is_at_safepoint() || >>> ??822????????? current_thread == java_thread->active_handshaker(), >>> ??823????????? "call by myself / at safepoint / at handshake"); >>> ``` >> >> Yep good catch. I hope current tests caught that. > > They would be tested in > vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ (own call stacks), > and getstacktr003 (call stacks in other thread). > > >> Speaking of tests ... >> >> In the native code I think you need to check the success of all JNI >> methods that can throw exceptions - otherwise I believe the tests may >> trigger warnings if -Xcheck:jni is used with them. See for example: >> >> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >> > > I updated testcases to check JNI and JVMTI function calls. > > >> In the Java code the target thread: >> >> ?? 45???? public void run() { >> ?? 46?????? try { >> ?? 47?????????? synchronized (lock) { >> ?? 48?????????????? lock.wait(); >> ?? 49?????????????? System.out.println("OK"); >> ?? 50?????????? } >> >> is potentially susceptible to spurious wakeups. Using a >> CountDownLatch would be robust. > > Fixed. > > > Thanks, > > Yasumasa > > >> Thanks, >> David >> ----- >> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2020/07/01 8:48, David Holmes wrote: >>>> Hi Yasumasa, >>>> >>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>> Hi David, >>>>> >>>>>>>> 1271???? ResourceMark rm; >>>>>>>> >>>>>>>> IIUC at this point the _calling_thread is the current thread, >>>>>>>> so we can use: >>>>>>>> >>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>> >>>>> If so, we can call make_local() in L1272 without JavaThread (or we >>>>> can pass current thread to make_local()). Is it right? >>>>> >>>>> ``` >>>>> 1271???? ResourceMark rm; >>>>> 1272 >>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>> thread_oop), >>>>> 1273??????????????????????????? jt, thread_oop); >>>>> ``` >>>> >>>> Sorry I got confused, _calling_thread may not be the current thread >>>> as we could be executing the handshake in the target thread itself. >>>> So the ResourceMark is correct as-is (implicitly for current thread). >>>> >>>> The argument to fill_frames will be used in the jvmtiStackInfo and >>>> passed back to the _calling_thread, so it must be created via >>>> make_local(_calling_thread, ...) as you presently have. >>>> >>>> Thanks, >>>> David >>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>> Hi David, >>>>>>> >>>>>>> Thank you for reviewing! I will update new webrev tomorrow. >>>>>>> >>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>> >>>>>>>> ? 498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>> ? 499 private: >>>>>>>> ? 500?? JavaThread *_calling_thread; >>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>> ? 502?? MultipleStackTracesCollector _collector; >>>>>>>> >>>>>>>> You can't have a StackObj as a member of another class like >>>>>>>> that as it may not be on the stack. I think >>>>>>>> MultipleStackTracesCollector should not extend any allocation >>>>>>>> class, and should always be embedded directly in another class. >>>>>>> >>>>>>> I'm not sure what does mean "embedded". >>>>>>> Is it ok as below? >>>>>>> >>>>>>> ``` >>>>>>> class MultipleStackTracesCollector { >>>>>>> ??? : >>>>>>> } >>>>>>> >>>>>>> class GetAllStackTraces : public VM_Operation { >>>>>>> ?? private: >>>>>>> ???? MultipleStackTracesCollector _collector; >>>>>>> } >>>>>>> ``` >>>>>> >>>>>> Yes that I what I meant. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>> Hi Yasumasa, >>>>>>>> >>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>>>> Hi David, Serguei, >>>>>>>>> >>>>>>>>> I updated webrev for 8242428. Could you review again? >>>>>>>>> This change migrate to use direct handshake for >>>>>>>>> GetStackTrace() and GetThreadListStackTraces() (when >>>>>>>>> thread_count == 1). >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>> >>>>>>>> This looks really good now! I only have a few nits below. There >>>>>>>> is one thing I don't like about it but it requires a change to >>>>>>>> the main Handshake logic to address - in >>>>>>>> JvmtiEnv::GetThreadListStackTraces you have to create a >>>>>>>> ThreadsListHandle to convert the jthread to a JavaThread, but >>>>>>>> then the Handshake::execute_direct creates another >>>>>>>> ThreadsListHandle internally. That's a waste. I will discuss >>>>>>>> with Robbin and file a RFE to have an overload of >>>>>>>> execute_direct that takes an existing TLH. Actually it's worse >>>>>>>> than that because we have another TLH in use at the entry point >>>>>>>> for the JVMTI functions, so I think there may be some scope for >>>>>>>> simplifying the use of TLH instances - future RFE. >>>>>>>> >>>>>>>> --- >>>>>>>> >>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>> >>>>>>>> ??451?? GetStackTraceClosure(JvmtiEnv *env, jint start_depth, >>>>>>>> jint max_count, >>>>>>>> ??452??????????????????????? jvmtiFrameInfo* frame_buffer, >>>>>>>> jint* count_ptr) >>>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>>> ??454?????? _env(env), _start_depth(start_depth), >>>>>>>> _max_count(max_count), >>>>>>>> ??455?????? _frame_buffer(frame_buffer), _count_ptr(count_ptr), >>>>>>>> ??456?????? _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>> >>>>>>>> Nit: can you do one initializer per line please. >>>>>>>> >>>>>>>> This looks wrong: >>>>>>>> >>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>> >>>>>>>> ??498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>> ??499 private: >>>>>>>> ??500?? JavaThread *_calling_thread; >>>>>>>> ??501?? jint _final_thread_count; >>>>>>>> ??502?? MultipleStackTracesCollector _collector; >>>>>>>> >>>>>>>> You can't have a StackObj as a member of another class like >>>>>>>> that as it may not be on the stack. I think >>>>>>>> MultipleStackTracesCollector should not extend any allocation >>>>>>>> class, and should always be embedded directly in another class. >>>>>>>> >>>>>>>> 481?? MultipleStackTracesCollector(JvmtiEnv *env, jint >>>>>>>> max_frame_count) { >>>>>>>> ??482???? _env = env; >>>>>>>> ??483???? _max_frame_count = max_frame_count; >>>>>>>> ??484???? _frame_count_total = 0; >>>>>>>> ??485???? _head = NULL; >>>>>>>> ??486???? _stack_info = NULL; >>>>>>>> ??487???? _result = JVMTI_ERROR_NONE; >>>>>>>> ??488?? } >>>>>>>> >>>>>>>> As you are touching this can you change it to use an >>>>>>>> initializer list as you did for the HandshakeClosure, and >>>>>>>> please keep one item per line. >>>>>>>> >>>>>>>> --- >>>>>>>> >>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>> >>>>>>>> ??820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>> ??821 java_thread->is_thread_fully_suspended(false, >>>>>>>> &debug_bits) || >>>>>>>> ??822????????? current_thread == java_thread->active_handshaker(), >>>>>>>> ??823????????? "at safepoint / handshake or target thread is >>>>>>>> suspended"); >>>>>>>> >>>>>>>> I don't think the suspension check is necessary, as even if the >>>>>>>> target is suspended we must still be at a safepoint or in a >>>>>>>> handshake with it. Makes me wonder if we used to allow a racy >>>>>>>> stacktrace operation on a suspended thread, assuming it would >>>>>>>> remain suspended? >>>>>>>> >>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>> 1269 >>>>>>>> 1270?? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>> >>>>>>>> You can use thread_oop in line 1270. >>>>>>>> >>>>>>>> 1272 >>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>> thread_oop), >>>>>>>> 1273??????????????????????????? jt, thread_oop); >>>>>>>> >>>>>>>> It is frustrating that this entire call chain started with a >>>>>>>> jthread reference, which we converted to a JavaThread, only to >>>>>>>> eventually need to convert it back to a jthread! I think there >>>>>>>> is some scope for simplification here but not as part of this >>>>>>>> change. >>>>>>>> >>>>>>>> 1271???? ResourceMark rm; >>>>>>>> >>>>>>>> IIUC at this point the _calling_thread is the current thread, >>>>>>>> so we can use: >>>>>>>> >>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>> >>>>>>>> --- >>>>>>>> >>>>>>>> Please add @bug lines to the tests. >>>>>>>> >>>>>>>> I'm still pondering the test logic but wanted to send this now. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> ----- >>>>>>>>> VM_GetThreadListStackTrace (for GetThreadListStackTraces) and >>>>>>>>> VM_GetAllStackTraces (for GetAllStackTraces) have inherited >>>>>>>>> VM_GetMultipleStackTraces VM operation which provides the >>>>>>>>> feature to generate jvmtiStackInfo. I modified >>>>>>>>> VM_GetMultipleStackTraces to a normal C++ class to share with >>>>>>>>> HandshakeClosure for GetThreadListStackTraces >>>>>>>>> (GetSingleStackTraceClosure). >>>>>>>>> >>>>>>>>> Also I added new testcases which test >>>>>>>>> GetThreadListStackTraces() with thread_count == 1 and with all >>>>>>>>> threads. >>>>>>>>> >>>>>>>>> This change has been tested in serviceability/jvmti >>>>>>>>> serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi >>>>>>>>> vmTestbase/nsk/jdwp. >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>> Hi all, >>>>>>>>>> >>>>>>>>>> Please review this change: >>>>>>>>>> >>>>>>>>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>> ?? webrev: >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>> >>>>>>>>>> This change replace following VM operations to direct handshake. >>>>>>>>>> >>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>>> ??- VM_GetThreadListStackTraces (GetThreadListStackTrace()) >>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>> >>>>>>>>>> GetThreadListStackTrace() uses direct handshake if thread >>>>>>>>>> count == 1. In other case (thread count > 1), it would be >>>>>>>>>> performed as VM operation (VM_GetThreadListStackTraces). >>>>>>>>>> Caller of VM_GetCurrentLocation >>>>>>>>>> (JvmtiEnvThreadState::reset_current_location()) might be >>>>>>>>>> called at safepoint. So I added safepoint check in its caller. >>>>>>>>>> >>>>>>>>>> This change has been tested in serviceability/jvmti >>>>>>>>>> serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi >>>>>>>>>> vmTestbase/ns >>>>>>>>>> k/jdwp. >>>>>>>>>> >>>>>>>>>> Also I tested it on submit repo, then it has execution error >>>>>>>>>> (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) due >>>>>>>>>> to dependency error. So I think it does not occur by this >>>>>>>>>> change. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa From suenaga at oss.nttdata.com Thu Jul 2 23:14:17 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Fri, 3 Jul 2020 08:14:17 +0900 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: <5107c478-1de2-7a49-ee71-dfad6a99a667@oracle.com> References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <2ea2f5d8-5c43-a7d3-d525-cd44c4009d25@oss.nttdata.com> <65889ab4-6b7e-86fb-59cc-012520b83138@oracle.com> <4b7df6f8-c8f7-fedf-6366-b885d2235b58@oss.nttdata.com> <74c05dfd-ffb2-b73d-66e5-dc98752729f5@oss.nttdata.com> <763793b1-9506-44c6-0681-1c71c44a44fd@oss.nttdata.com> <5107c478-1de2-7a49-ee71-dfad6a99a667@oracle.com> Message-ID: <90cdfef9-e558-0d25-33f5-25b7d65bb83b@oss.nttdata.com> Hi Dan, Thanks for your comment! On 2020/07/03 7:16, Daniel D. Daugherty wrote: > On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >> Hi David, >> >> I upload new webrev. Could you review again? >> >> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ > > src/hotspot/share/prims/jvmtiEnv.cpp > ??? L1542: ??? // Get stack trace with handshake > ??????? nit - please add a period at the end. I will fix it. > ??? L1591: ??? *stack_info_ptr = op.stack_info(); > ??????? The return parameter should not be touched unless the return > ??????? code in 'err' == JVMTI_ERROR_NONE. > > ??? old L1582: ? if (err == JVMTI_ERROR_NONE) { > ??????? Please restore this check. The return parameter should not > ??????? be touched unless the return code in 'err' == JVMTI_ERROR_NONE. I will fix it. > ??? L1272: ? if (!jt->is_exiting() && (thread_oop != NULL)) { > ??????? nit - extra parens around the second expression. I will fix it. > src/hotspot/share/prims/jvmtiEnvBase.cpp > ??? old L1532: ? _result = JVMTI_ERROR_THREAD_NOT_ALIVE; > ??????? This deletion of the _result field threw me for a minute and then > ??? ? ? I figured out that the field is init to JVMTI_ERROR_THREAD_NOT_ALIVE > ??????? in the constructor. > > ??? L1553: ? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { > ??????? nit - extra parens around the second expression. I will fix it. > src/hotspot/share/prims/jvmtiEnvBase.hpp > ??? No comments. > > src/hotspot/share/runtime/vmOperations.hpp > ??? No comments. > > test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java > ??? No comments. > > test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java > ??? L64: ??????? startSignal.countDown(); > ??????? I was expecting this to be a call to await() instead of > ??????? countDown(). What am I missing here? > > ??????? I think this test might be passing by accident right now, but... Main thread (which call JVMTI functions to test) should wait until test thread is ready. So main thread would wait startSignal, and test thread would count down. > test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c > ??? L92: ? jthreads = (jthread *)malloc(sizeof(jthread) * num_threads); > ??????? You don't check for malloc() failure. > ??????? 'jthreads' is allocated but never freed. I will fix it. > test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c > ??? L91: ? result = (*jvmti)->SuspendThread(jvmti, thread); > ??????? Why are you suspending the thread? GetAllStackTraces() and > ??????? GetThreadListStackTraces() do not require the target thread(s) > ??????? to be suspend. > > ??????? If you decide not to SuspendThread, then you don't need the > ??????? AddCapabilities or the ResumeThread calls. Test thread might not be entered following code (stopSignal.await()). We might see deferent call stack between GetAllStackTraces() and GetThreadListStackTraces(). We cannot control to freeze call stack of test thread in Java code. (I didn't use SuspendThread() at first, but I saw some errors which causes in above.) So we need to call SuspendThread() to ensure we can see same call stack. Thanks, Yasumasa > Dan > >> >> On 2020/07/02 15:05, David Holmes wrote: >>> Hi Yasumasa, >>> >>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>> Hi, >>>> >>>> I uploaded new webrev. Could review again? >>>> >>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>> >>> Updates look fine - thanks. >>> >>> One minor nit: >>> >>> 1274???? _collector.allocate_and_fill_stacks(1); >>> 1275???? _collector.set_result(JVMTI_ERROR_NONE); >>> >>> In the other places where you use _collector you rely on result being initialized to JVMTI_ERROR_NONE, rather than setting it directly after allocate_and_fill_stacks(). >> >> Fixed. >> >> >>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>> >>>>>>>>> ? 820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>> ? 821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>> ? 822????????? current_thread == java_thread->active_handshaker(), >>>>>>>>> ? 823????????? "at safepoint / handshake or target thread is suspended"); >>>>>>>>> >>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>> >>>> This function (JvmtiEnvBase::get_stack_trace()) can be called to get own stack trace. For example, we can call GetStackTrace() for current thread at JVMTI event. >>>> So I changed assert as below: >>>> >>>> ``` >>>> ??820?? assert(current_thread == java_thread || >>>> ??821????????? SafepointSynchronize::is_at_safepoint() || >>>> ??822????????? current_thread == java_thread->active_handshaker(), >>>> ??823????????? "call by myself / at safepoint / at handshake"); >>>> ``` >>> >>> Yep good catch. I hope current tests caught that. >> >> They would be tested in vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ (own call stacks), and getstacktr003 (call stacks in other thread). >> >> >>> Speaking of tests ... >>> >>> In the native code I think you need to check the success of all JNI methods that can throw exceptions - otherwise I believe the tests may trigger warnings if -Xcheck:jni is used with them. See for example: >>> >>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >> >> I updated testcases to check JNI and JVMTI function calls. >> >> >>> In the Java code the target thread: >>> >>> ?? 45???? public void run() { >>> ?? 46?????? try { >>> ?? 47?????????? synchronized (lock) { >>> ?? 48?????????????? lock.wait(); >>> ?? 49?????????????? System.out.println("OK"); >>> ?? 50?????????? } >>> >>> is potentially susceptible to spurious wakeups. Using a CountDownLatch would be robust. >> >> Fixed. >> >> >> Thanks, >> >> Yasumasa >> >> >>> Thanks, >>> David >>> ----- >>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2020/07/01 8:48, David Holmes wrote: >>>>> Hi Yasumasa, >>>>> >>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>> Hi David, >>>>>> >>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>> >>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>> >>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>> >>>>>> If so, we can call make_local() in L1272 without JavaThread (or we can pass current thread to make_local()). Is it right? >>>>>> >>>>>> ``` >>>>>> 1271???? ResourceMark rm; >>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>> 1273??????????????????????????? jt, thread_oop); >>>>>> ``` >>>>> >>>>> Sorry I got confused, _calling_thread may not be the current thread as we could be executing the handshake in the target thread itself. So the ResourceMark is correct as-is (implicitly for current thread). >>>>> >>>>> The argument to fill_frames will be used in the jvmtiStackInfo and passed back to the _calling_thread, so it must be created via make_local(_calling_thread, ...) as you presently have. >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>> Hi David, >>>>>>>> >>>>>>>> Thank you for reviewing! I will update new webrev tomorrow. >>>>>>>> >>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>> >>>>>>>>> ? 498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>> ? 499 private: >>>>>>>>> ? 500?? JavaThread *_calling_thread; >>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>> ? 502?? MultipleStackTracesCollector _collector; >>>>>>>>> >>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>> >>>>>>>> I'm not sure what does mean "embedded". >>>>>>>> Is it ok as below? >>>>>>>> >>>>>>>> ``` >>>>>>>> class MultipleStackTracesCollector { >>>>>>>> ??? : >>>>>>>> } >>>>>>>> >>>>>>>> class GetAllStackTraces : public VM_Operation { >>>>>>>> ?? private: >>>>>>>> ???? MultipleStackTracesCollector _collector; >>>>>>>> } >>>>>>>> ``` >>>>>>> >>>>>>> Yes that I what I meant. >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>> Hi Yasumasa, >>>>>>>>> >>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>>>>> Hi David, Serguei, >>>>>>>>>> >>>>>>>>>> I updated webrev for 8242428. Could you review again? >>>>>>>>>> This change migrate to use direct handshake for GetStackTrace() and GetThreadListStackTraces() (when thread_count == 1). >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>> >>>>>>>>> This looks really good now! I only have a few nits below. There is one thing I don't like about it but it requires a change to the main Handshake logic to address - in JvmtiEnv::GetThreadListStackTraces you have to create a ThreadsListHandle to convert the jthread to a JavaThread, but then the Handshake::execute_direct creates another ThreadsListHandle internally. That's a waste. I will discuss with Robbin and file a RFE to have an overload of execute_direct that takes an existing TLH. Actually it's worse than that because we have another TLH in use at the entry point for the JVMTI functions, so I think there may be some scope for simplifying the use of TLH instances - future RFE. >>>>>>>>> >>>>>>>>> --- >>>>>>>>> >>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>> >>>>>>>>> ??451?? GetStackTraceClosure(JvmtiEnv *env, jint start_depth, jint max_count, >>>>>>>>> ??452??????????????????????? jvmtiFrameInfo* frame_buffer, jint* count_ptr) >>>>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>>>> ??454?????? _env(env), _start_depth(start_depth), _max_count(max_count), >>>>>>>>> ??455?????? _frame_buffer(frame_buffer), _count_ptr(count_ptr), >>>>>>>>> ??456?????? _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>> >>>>>>>>> Nit: can you do one initializer per line please. >>>>>>>>> >>>>>>>>> This looks wrong: >>>>>>>>> >>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>> >>>>>>>>> ??498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>> ??499 private: >>>>>>>>> ??500?? JavaThread *_calling_thread; >>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>> ??502?? MultipleStackTracesCollector _collector; >>>>>>>>> >>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>> >>>>>>>>> 481?? MultipleStackTracesCollector(JvmtiEnv *env, jint max_frame_count) { >>>>>>>>> ??482???? _env = env; >>>>>>>>> ??483???? _max_frame_count = max_frame_count; >>>>>>>>> ??484???? _frame_count_total = 0; >>>>>>>>> ??485???? _head = NULL; >>>>>>>>> ??486???? _stack_info = NULL; >>>>>>>>> ??487???? _result = JVMTI_ERROR_NONE; >>>>>>>>> ??488?? } >>>>>>>>> >>>>>>>>> As you are touching this can you change it to use an initializer list as you did for the HandshakeClosure, and please keep one item per line. >>>>>>>>> >>>>>>>>> --- >>>>>>>>> >>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>> >>>>>>>>> ??820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>> ??821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>> ??822????????? current_thread == java_thread->active_handshaker(), >>>>>>>>> ??823????????? "at safepoint / handshake or target thread is suspended"); >>>>>>>>> >>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>> >>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>> 1269 >>>>>>>>> 1270?? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>> >>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>> >>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>> 1273??????????????????????????? jt, thread_oop); >>>>>>>>> >>>>>>>>> It is frustrating that this entire call chain started with a jthread reference, which we converted to a JavaThread, only to eventually need to convert it back to a jthread! I think there is some scope for simplification here but not as part of this change. >>>>>>>>> >>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>> >>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>> >>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>> >>>>>>>>> --- >>>>>>>>> >>>>>>>>> Please add @bug lines to the tests. >>>>>>>>> >>>>>>>>> I'm still pondering the test logic but wanted to send this now. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> David >>>>>>>>> ----- >>>>>>>>>> VM_GetThreadListStackTrace (for GetThreadListStackTraces) and VM_GetAllStackTraces (for GetAllStackTraces) have inherited VM_GetMultipleStackTraces VM operation which provides the feature to generate jvmtiStackInfo. I modified VM_GetMultipleStackTraces to a normal C++ class to share with HandshakeClosure for GetThreadListStackTraces (GetSingleStackTraceClosure). >>>>>>>>>> >>>>>>>>>> Also I added new testcases which test GetThreadListStackTraces() with thread_count == 1 and with all threads. >>>>>>>>>> >>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/nsk/jdwp. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi all, >>>>>>>>>>> >>>>>>>>>>> Please review this change: >>>>>>>>>>> >>>>>>>>>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>> >>>>>>>>>>> This change replace following VM operations to direct handshake. >>>>>>>>>>> >>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>>>> ??- VM_GetThreadListStackTraces (GetThreadListStackTrace()) >>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>> >>>>>>>>>>> GetThreadListStackTrace() uses direct handshake if thread count == 1. In other case (thread count > 1), it would be performed as VM operation (VM_GetThreadListStackTraces). >>>>>>>>>>> Caller of VM_GetCurrentLocation (JvmtiEnvThreadState::reset_current_location()) might be called at safepoint. So I added safepoint check in its caller. >>>>>>>>>>> >>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/ns >>>>>>>>>>> k/jdwp. >>>>>>>>>>> >>>>>>>>>>> Also I tested it on submit repo, then it has execution error (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) due to dependency error. So I think it does not occur by this change. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa > From daniel.daugherty at oracle.com Fri Jul 3 00:56:39 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 2 Jul 2020 20:56:39 -0400 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: <90cdfef9-e558-0d25-33f5-25b7d65bb83b@oss.nttdata.com> References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <2ea2f5d8-5c43-a7d3-d525-cd44c4009d25@oss.nttdata.com> <65889ab4-6b7e-86fb-59cc-012520b83138@oracle.com> <4b7df6f8-c8f7-fedf-6366-b885d2235b58@oss.nttdata.com> <74c05dfd-ffb2-b73d-66e5-dc98752729f5@oss.nttdata.com> <763793b1-9506-44c6-0681-1c71c44a44fd@oss.nttdata.com> <5107c478-1de2-7a49-ee71-dfad6a99a667@oracle.com> <90cdfef9-e558-0d25-33f5-25b7d65bb83b@oss.nttdata.com> Message-ID: On 7/2/20 7:14 PM, Yasumasa Suenaga wrote: > Hi Dan, > > Thanks for your comment! > > On 2020/07/03 7:16, Daniel D. Daugherty wrote: >> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>> Hi David, >>> >>> I upload new webrev. Could you review again? >>> >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >> >> src/hotspot/share/prims/jvmtiEnv.cpp >> ???? L1542: ??? // Get stack trace with handshake >> ???????? nit - please add a period at the end. > > I will fix it. > > >> ???? L1591: ??? *stack_info_ptr = op.stack_info(); >> ???????? The return parameter should not be touched unless the return >> ???????? code in 'err' == JVMTI_ERROR_NONE. >> >> ???? old L1582: ? if (err == JVMTI_ERROR_NONE) { >> ???????? Please restore this check. The return parameter should not >> ???????? be touched unless the return code in 'err' == JVMTI_ERROR_NONE. > > I will fix it. > > >> ???? L1272: ? if (!jt->is_exiting() && (thread_oop != NULL)) { >> ???????? nit - extra parens around the second expression. > > I will fix it. > > >> src/hotspot/share/prims/jvmtiEnvBase.cpp >> ???? old L1532: ? _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >> ???????? This deletion of the _result field threw me for a minute and >> then >> ???? ? ? I figured out that the field is init to >> JVMTI_ERROR_THREAD_NOT_ALIVE >> ???????? in the constructor. >> >> ???? L1553: ? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >> ???????? nit - extra parens around the second expression. > > I will fix it. > > >> src/hotspot/share/prims/jvmtiEnvBase.hpp >> ???? No comments. >> >> src/hotspot/share/runtime/vmOperations.hpp >> ???? No comments. >> >> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >> >> ???? No comments. >> >> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >> >> ???? L64: ??????? startSignal.countDown(); >> ???????? I was expecting this to be a call to await() instead of >> ???????? countDown(). What am I missing here? >> >> ???????? I think this test might be passing by accident right now, >> but... > > Main thread (which call JVMTI functions to test) should wait until > test thread is ready. > So main thread would wait startSignal, and test thread would count down. That's my point. L64 is the main thread so it should be a call to startSignal.await(). > > >> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >> >> ???? L92: ? jthreads = (jthread *)malloc(sizeof(jthread) * num_threads); >> ???????? You don't check for malloc() failure. >> ???????? 'jthreads' is allocated but never freed. > > I will fix it. > > >> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >> >> ???? L91: ? result = (*jvmti)->SuspendThread(jvmti, thread); >> ???????? Why are you suspending the thread? GetAllStackTraces() and >> ???????? GetThreadListStackTraces() do not require the target thread(s) >> ???????? to be suspend. >> >> ???????? If you decide not to SuspendThread, then you don't need the >> ???????? AddCapabilities or the ResumeThread calls. > > Test thread might not be entered following code (stopSignal.await()). > We might see deferent call stack between GetAllStackTraces() and > GetThreadListStackTraces(). We cannot control to freeze call stack of > test thread in Java code. > (I didn't use SuspendThread() at first, but I saw some errors which > causes in above.) > > So we need to call SuspendThread() to ensure we can see same call stack. That sounds like a good reason. Please add a comment near the SuspendThread() call so that other readers won't wonder... Dan > > > Thanks, > > Yasumasa > > >> Dan >> >>> >>> On 2020/07/02 15:05, David Holmes wrote: >>>> Hi Yasumasa, >>>> >>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>> Hi, >>>>> >>>>> I uploaded new webrev. Could review again? >>>>> >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>> >>>> Updates look fine - thanks. >>>> >>>> One minor nit: >>>> >>>> 1274???? _collector.allocate_and_fill_stacks(1); >>>> 1275???? _collector.set_result(JVMTI_ERROR_NONE); >>>> >>>> In the other places where you use _collector you rely on result >>>> being initialized to JVMTI_ERROR_NONE, rather than setting it >>>> directly after allocate_and_fill_stacks(). >>> >>> Fixed. >>> >>> >>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>> >>>>>>>>>> ? 820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>> ? 821 java_thread->is_thread_fully_suspended(false, >>>>>>>>>> &debug_bits) || >>>>>>>>>> ? 822????????? current_thread == >>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>> ? 823????????? "at safepoint / handshake or target thread is >>>>>>>>>> suspended"); >>>>>>>>>> >>>>>>>>>> I don't think the suspension check is necessary, as even if >>>>>>>>>> the target is suspended we must still be at a safepoint or in >>>>>>>>>> a handshake with it. Makes me wonder if we used to allow a >>>>>>>>>> racy stacktrace operation on a suspended thread, assuming it >>>>>>>>>> would remain suspended? >>>>> >>>>> This function (JvmtiEnvBase::get_stack_trace()) can be called to >>>>> get own stack trace. For example, we can call GetStackTrace() for >>>>> current thread at JVMTI event. >>>>> So I changed assert as below: >>>>> >>>>> ``` >>>>> ??820?? assert(current_thread == java_thread || >>>>> ??821????????? SafepointSynchronize::is_at_safepoint() || >>>>> ??822????????? current_thread == java_thread->active_handshaker(), >>>>> ??823????????? "call by myself / at safepoint / at handshake"); >>>>> ``` >>>> >>>> Yep good catch. I hope current tests caught that. >>> >>> They would be tested in >>> vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ (own call stacks), >>> and getstacktr003 (call stacks in other thread). >>> >>> >>>> Speaking of tests ... >>>> >>>> In the native code I think you need to check the success of all JNI >>>> methods that can throw exceptions - otherwise I believe the tests >>>> may trigger warnings if -Xcheck:jni is used with them. See for >>>> example: >>>> >>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>> >>> >>> I updated testcases to check JNI and JVMTI function calls. >>> >>> >>>> In the Java code the target thread: >>>> >>>> ?? 45???? public void run() { >>>> ?? 46?????? try { >>>> ?? 47?????????? synchronized (lock) { >>>> ?? 48?????????????? lock.wait(); >>>> ?? 49?????????????? System.out.println("OK"); >>>> ?? 50?????????? } >>>> >>>> is potentially susceptible to spurious wakeups. Using a >>>> CountDownLatch would be robust. >>> >>> Fixed. >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>> Hi Yasumasa, >>>>>> >>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>> Hi David, >>>>>>> >>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>> >>>>>>>>>> IIUC at this point the _calling_thread is the current thread, >>>>>>>>>> so we can use: >>>>>>>>>> >>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>> >>>>>>> If so, we can call make_local() in L1272 without JavaThread (or >>>>>>> we can pass current thread to make_local()). Is it right? >>>>>>> >>>>>>> ``` >>>>>>> 1271???? ResourceMark rm; >>>>>>> 1272 >>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>> thread_oop), >>>>>>> 1273??????????????????????????? jt, thread_oop); >>>>>>> ``` >>>>>> >>>>>> Sorry I got confused, _calling_thread may not be the current >>>>>> thread as we could be executing the handshake in the target >>>>>> thread itself. So the ResourceMark is correct as-is (implicitly >>>>>> for current thread). >>>>>> >>>>>> The argument to fill_frames will be used in the jvmtiStackInfo >>>>>> and passed back to the _calling_thread, so it must be created via >>>>>> make_local(_calling_thread, ...) as you presently have. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>>> Hi David, >>>>>>>>> >>>>>>>>> Thank you for reviewing! I will update new webrev tomorrow. >>>>>>>>> >>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>> >>>>>>>>>> ? 498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>> ? 499 private: >>>>>>>>>> ? 500?? JavaThread *_calling_thread; >>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>> ? 502?? MultipleStackTracesCollector _collector; >>>>>>>>>> >>>>>>>>>> You can't have a StackObj as a member of another class like >>>>>>>>>> that as it may not be on the stack. I think >>>>>>>>>> MultipleStackTracesCollector should not extend any allocation >>>>>>>>>> class, and should always be embedded directly in another class. >>>>>>>>> >>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>> Is it ok as below? >>>>>>>>> >>>>>>>>> ``` >>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>> ??? : >>>>>>>>> } >>>>>>>>> >>>>>>>>> class GetAllStackTraces : public VM_Operation { >>>>>>>>> ?? private: >>>>>>>>> ???? MultipleStackTracesCollector _collector; >>>>>>>>> } >>>>>>>>> ``` >>>>>>>> >>>>>>>> Yes that I what I meant. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> ----- >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>> Hi Yasumasa, >>>>>>>>>> >>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>> >>>>>>>>>>> I updated webrev for 8242428. Could you review again? >>>>>>>>>>> This change migrate to use direct handshake for >>>>>>>>>>> GetStackTrace() and GetThreadListStackTraces() (when >>>>>>>>>>> thread_count == 1). >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>> >>>>>>>>>> This looks really good now! I only have a few nits below. >>>>>>>>>> There is one thing I don't like about it but it requires a >>>>>>>>>> change to the main Handshake logic to address - in >>>>>>>>>> JvmtiEnv::GetThreadListStackTraces you have to create a >>>>>>>>>> ThreadsListHandle to convert the jthread to a JavaThread, but >>>>>>>>>> then the Handshake::execute_direct creates another >>>>>>>>>> ThreadsListHandle internally. That's a waste. I will discuss >>>>>>>>>> with Robbin and file a RFE to have an overload of >>>>>>>>>> execute_direct that takes an existing TLH. Actually it's >>>>>>>>>> worse than that because we have another TLH in use at the >>>>>>>>>> entry point for the JVMTI functions, so I think there may be >>>>>>>>>> some scope for simplifying the use of TLH instances - future >>>>>>>>>> RFE. >>>>>>>>>> >>>>>>>>>> --- >>>>>>>>>> >>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>> >>>>>>>>>> ??451?? GetStackTraceClosure(JvmtiEnv *env, jint start_depth, >>>>>>>>>> jint max_count, >>>>>>>>>> ??452??????????????????????? jvmtiFrameInfo* frame_buffer, >>>>>>>>>> jint* count_ptr) >>>>>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>>>>> ??454?????? _env(env), _start_depth(start_depth), >>>>>>>>>> _max_count(max_count), >>>>>>>>>> ??455?????? _frame_buffer(frame_buffer), _count_ptr(count_ptr), >>>>>>>>>> ??456 _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>> >>>>>>>>>> Nit: can you do one initializer per line please. >>>>>>>>>> >>>>>>>>>> This looks wrong: >>>>>>>>>> >>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>> >>>>>>>>>> ??498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>> ??499 private: >>>>>>>>>> ??500?? JavaThread *_calling_thread; >>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>> ??502?? MultipleStackTracesCollector _collector; >>>>>>>>>> >>>>>>>>>> You can't have a StackObj as a member of another class like >>>>>>>>>> that as it may not be on the stack. I think >>>>>>>>>> MultipleStackTracesCollector should not extend any allocation >>>>>>>>>> class, and should always be embedded directly in another class. >>>>>>>>>> >>>>>>>>>> 481?? MultipleStackTracesCollector(JvmtiEnv *env, jint >>>>>>>>>> max_frame_count) { >>>>>>>>>> ??482???? _env = env; >>>>>>>>>> ??483???? _max_frame_count = max_frame_count; >>>>>>>>>> ??484???? _frame_count_total = 0; >>>>>>>>>> ??485???? _head = NULL; >>>>>>>>>> ??486???? _stack_info = NULL; >>>>>>>>>> ??487???? _result = JVMTI_ERROR_NONE; >>>>>>>>>> ??488?? } >>>>>>>>>> >>>>>>>>>> As you are touching this can you change it to use an >>>>>>>>>> initializer list as you did for the HandshakeClosure, and >>>>>>>>>> please keep one item per line. >>>>>>>>>> >>>>>>>>>> --- >>>>>>>>>> >>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>> >>>>>>>>>> ??820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>> ??821 java_thread->is_thread_fully_suspended(false, >>>>>>>>>> &debug_bits) || >>>>>>>>>> ??822????????? current_thread == >>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>> ??823????????? "at safepoint / handshake or target thread is >>>>>>>>>> suspended"); >>>>>>>>>> >>>>>>>>>> I don't think the suspension check is necessary, as even if >>>>>>>>>> the target is suspended we must still be at a safepoint or in >>>>>>>>>> a handshake with it. Makes me wonder if we used to allow a >>>>>>>>>> racy stacktrace operation on a suspended thread, assuming it >>>>>>>>>> would remain suspended? >>>>>>>>>> >>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>> 1269 >>>>>>>>>> 1270?? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>> >>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>> >>>>>>>>>> 1272 >>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>> thread_oop), >>>>>>>>>> 1273??????????????????????????? jt, thread_oop); >>>>>>>>>> >>>>>>>>>> It is frustrating that this entire call chain started with a >>>>>>>>>> jthread reference, which we converted to a JavaThread, only >>>>>>>>>> to eventually need to convert it back to a jthread! I think >>>>>>>>>> there is some scope for simplification here but not as part >>>>>>>>>> of this change. >>>>>>>>>> >>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>> >>>>>>>>>> IIUC at this point the _calling_thread is the current thread, >>>>>>>>>> so we can use: >>>>>>>>>> >>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>> >>>>>>>>>> --- >>>>>>>>>> >>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>> >>>>>>>>>> I'm still pondering the test logic but wanted to send this now. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> David >>>>>>>>>> ----- >>>>>>>>>>> VM_GetThreadListStackTrace (for GetThreadListStackTraces) >>>>>>>>>>> and VM_GetAllStackTraces (for GetAllStackTraces) have >>>>>>>>>>> inherited VM_GetMultipleStackTraces VM operation which >>>>>>>>>>> provides the feature to generate jvmtiStackInfo. I modified >>>>>>>>>>> VM_GetMultipleStackTraces to a normal C++ class to share >>>>>>>>>>> with HandshakeClosure for GetThreadListStackTraces >>>>>>>>>>> (GetSingleStackTraceClosure). >>>>>>>>>>> >>>>>>>>>>> Also I added new testcases which test >>>>>>>>>>> GetThreadListStackTraces() with thread_count == 1 and with >>>>>>>>>>> all threads. >>>>>>>>>>> >>>>>>>>>>> This change has been tested in serviceability/jvmti >>>>>>>>>>> serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi >>>>>>>>>>> vmTestbase/nsk/jdwp. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>>>> Hi all, >>>>>>>>>>>> >>>>>>>>>>>> Please review this change: >>>>>>>>>>>> >>>>>>>>>>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>> ?? webrev: >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>> >>>>>>>>>>>> This change replace following VM operations to direct >>>>>>>>>>>> handshake. >>>>>>>>>>>> >>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>>>>> ??- VM_GetThreadListStackTraces (GetThreadListStackTrace()) >>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>> >>>>>>>>>>>> GetThreadListStackTrace() uses direct handshake if thread >>>>>>>>>>>> count == 1. In other case (thread count > 1), it would be >>>>>>>>>>>> performed as VM operation (VM_GetThreadListStackTraces). >>>>>>>>>>>> Caller of VM_GetCurrentLocation >>>>>>>>>>>> (JvmtiEnvThreadState::reset_current_location()) might be >>>>>>>>>>>> called at safepoint. So I added safepoint check in its caller. >>>>>>>>>>>> >>>>>>>>>>>> This change has been tested in serviceability/jvmti >>>>>>>>>>>> serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi >>>>>>>>>>>> vmTestbase/ns >>>>>>>>>>>> k/jdwp. >>>>>>>>>>>> >>>>>>>>>>>> Also I tested it on submit repo, then it has execution >>>>>>>>>>>> error >>>>>>>>>>>> (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) due >>>>>>>>>>>> to dependency error. So I think it does not occur by this >>>>>>>>>>>> change. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >> From vladimir.kozlov at oracle.com Fri Jul 3 02:02:06 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 2 Jul 2020 19:02:06 -0700 Subject: [15] RFR(T) 8247527: serviceability/dcmd/gc/HeapDumpCompressedTest.java fails with Graal + ZGC Message-ID: https://cr.openjdk.java.net/~kvn/8247527/webrev.00/ https://bugs.openjdk.java.net/browse/JDK-8247527 Test should have @requires which excludes running Graal with GC which it does not support. Testing: hs-tier1,hs-tier4-graal Thanks, Vladimir From igor.ignatyev at oracle.com Fri Jul 3 02:24:15 2020 From: igor.ignatyev at oracle.com (igor.ignatyev at oracle.com) Date: Thu, 2 Jul 2020 19:24:15 -0700 Subject: [15] RFR(T) 8247527: serviceability/dcmd/gc/HeapDumpCompressedTest.java fails with Graal + ZGC In-Reply-To: References: Message-ID: <5E33E613-882E-400A-886A-EA4FAD85F2EA@oracle.com> LGTM ? Igor > On Jul 2, 2020, at 7:03 PM, Vladimir Kozlov wrote: > > ?https://cr.openjdk.java.net/~kvn/8247527/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-8247527 > > Test should have @requires which excludes running Graal with GC which it does not support. > > Testing: hs-tier1,hs-tier4-graal > > Thanks, > Vladimir From david.holmes at oracle.com Fri Jul 3 02:25:45 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 3 Jul 2020 12:25:45 +1000 Subject: [15] RFR(T) 8247527: serviceability/dcmd/gc/HeapDumpCompressedTest.java fails with Graal + ZGC In-Reply-To: References: Message-ID: Hi Vladimir, On 3/07/2020 12:02 pm, Vladimir Kozlov wrote: > https://cr.openjdk.java.net/~kvn/8247527/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-8247527 > > Test should have @requires which excludes running Graal with GC which it > does not support. I find it somewhat disturbing that a generic test has to know about the limitations between GCs and Graal! I would have been more inclined to just exclude this test when running with Graal, even if that theoretically reduced the test coverage in a ting way. If/When Graal supports these other GCs who will remember to re-enable these test cases? Thanks, David > Testing: hs-tier1,hs-tier4-graal > > Thanks, > Vladimir From david.holmes at oracle.com Fri Jul 3 02:50:03 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 3 Jul 2020 12:50:03 +1000 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: <90cdfef9-e558-0d25-33f5-25b7d65bb83b@oss.nttdata.com> References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <2ea2f5d8-5c43-a7d3-d525-cd44c4009d25@oss.nttdata.com> <65889ab4-6b7e-86fb-59cc-012520b83138@oracle.com> <4b7df6f8-c8f7-fedf-6366-b885d2235b58@oss.nttdata.com> <74c05dfd-ffb2-b73d-66e5-dc98752729f5@oss.nttdata.com> <763793b1-9506-44c6-0681-1c71c44a44fd@oss.nttdata.com> <5107c478-1de2-7a49-ee71-dfad6a99a667@oracle.com> <90cdfef9-e558-0d25-33f5-25b7d65bb83b@oss.nttdata.com> Message-ID: <0e359819-618f-c17b-a427-bb7e4daa2c58@oracle.com> Sorry I'm responding here without seeing latest webrev but there is enough context I think ... On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: > Hi Dan, > > Thanks for your comment! > > On 2020/07/03 7:16, Daniel D. Daugherty wrote: >> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>> Hi David, >>> >>> I upload new webrev. Could you review again? >>> >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >> >> src/hotspot/share/prims/jvmtiEnv.cpp >> ???? L1542: ??? // Get stack trace with handshake >> ???????? nit - please add a period at the end. > > I will fix it. > > >> ???? L1591: ??? *stack_info_ptr = op.stack_info(); >> ???????? The return parameter should not be touched unless the return >> ???????? code in 'err' == JVMTI_ERROR_NONE. >> >> ???? old L1582: ? if (err == JVMTI_ERROR_NONE) { >> ???????? Please restore this check. The return parameter should not >> ???????? be touched unless the return code in 'err' == JVMTI_ERROR_NONE. > > I will fix it. But op.stack_info() will return NULL if the error is not JVMTI_ERROR_NONE. Are you (Dan) concerned about someone passing in a non-null/initialized out-pointer that will be reset to NULL if there was an error? > >> ???? L1272: ? if (!jt->is_exiting() && (thread_oop != NULL)) { >> ???????? nit - extra parens around the second expression. > > I will fix it. > > >> src/hotspot/share/prims/jvmtiEnvBase.cpp >> ???? old L1532: ? _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >> ???????? This deletion of the _result field threw me for a minute and >> then >> ???? ? ? I figured out that the field is init to >> JVMTI_ERROR_THREAD_NOT_ALIVE >> ???????? in the constructor. >> >> ???? L1553: ? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >> ???????? nit - extra parens around the second expression. > > I will fix it. > > >> src/hotspot/share/prims/jvmtiEnvBase.hpp >> ???? No comments. >> >> src/hotspot/share/runtime/vmOperations.hpp >> ???? No comments. >> >> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >> >> ???? No comments. >> >> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >> >> ???? L64: ??????? startSignal.countDown(); >> ???????? I was expecting this to be a call to await() instead of >> ???????? countDown(). What am I missing here? >> >> ???????? I think this test might be passing by accident right now, but... > > Main thread (which call JVMTI functions to test) should wait until test > thread is ready. > So main thread would wait startSignal, and test thread would count down. No! The test thread that previously called obj.wait() now calls latch.await(). The main thread that previously called obj.notify() now calls latch.countDown(). The main thread continues to spin until it sees the target is WAITING before proceeding with the test. > >> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >> >> ???? L92: ? jthreads = (jthread *)malloc(sizeof(jthread) * num_threads); >> ???????? You don't check for malloc() failure. >> ???????? 'jthreads' is allocated but never freed. > > I will fix it. > > >> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >> >> ???? L91: ? result = (*jvmti)->SuspendThread(jvmti, thread); >> ???????? Why are you suspending the thread? GetAllStackTraces() and >> ???????? GetThreadListStackTraces() do not require the target thread(s) >> ???????? to be suspend. >> >> ???????? If you decide not to SuspendThread, then you don't need the >> ???????? AddCapabilities or the ResumeThread calls. > > Test thread might not be entered following code (stopSignal.await()). We > might see deferent call stack between GetAllStackTraces() and > GetThreadListStackTraces(). We cannot control to freeze call stack of > test thread in Java code. > (I didn't use SuspendThread() at first, but I saw some errors which > causes in above.) > > So we need to call SuspendThread() to ensure we can see same call stack. If you are checking that the thread is in state WAITING then it cannot escape from that state and you can sample the stack multiple times from any API and get the same result. I suspect the errors you saw were from the apparent incorrect use of the CountDownLatch. Cheers, David ----- > > Thanks, > > Yasumasa > > >> Dan >> >>> >>> On 2020/07/02 15:05, David Holmes wrote: >>>> Hi Yasumasa, >>>> >>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>> Hi, >>>>> >>>>> I uploaded new webrev. Could review again? >>>>> >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>> >>>> Updates look fine - thanks. >>>> >>>> One minor nit: >>>> >>>> 1274???? _collector.allocate_and_fill_stacks(1); >>>> 1275???? _collector.set_result(JVMTI_ERROR_NONE); >>>> >>>> In the other places where you use _collector you rely on result >>>> being initialized to JVMTI_ERROR_NONE, rather than setting it >>>> directly after allocate_and_fill_stacks(). >>> >>> Fixed. >>> >>> >>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>> >>>>>>>>>> ? 820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>> ? 821 java_thread->is_thread_fully_suspended(false, >>>>>>>>>> &debug_bits) || >>>>>>>>>> ? 822????????? current_thread == >>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>> ? 823????????? "at safepoint / handshake or target thread is >>>>>>>>>> suspended"); >>>>>>>>>> >>>>>>>>>> I don't think the suspension check is necessary, as even if >>>>>>>>>> the target is suspended we must still be at a safepoint or in >>>>>>>>>> a handshake with it. Makes me wonder if we used to allow a >>>>>>>>>> racy stacktrace operation on a suspended thread, assuming it >>>>>>>>>> would remain suspended? >>>>> >>>>> This function (JvmtiEnvBase::get_stack_trace()) can be called to >>>>> get own stack trace. For example, we can call GetStackTrace() for >>>>> current thread at JVMTI event. >>>>> So I changed assert as below: >>>>> >>>>> ``` >>>>> ??820?? assert(current_thread == java_thread || >>>>> ??821????????? SafepointSynchronize::is_at_safepoint() || >>>>> ??822????????? current_thread == java_thread->active_handshaker(), >>>>> ??823????????? "call by myself / at safepoint / at handshake"); >>>>> ``` >>>> >>>> Yep good catch. I hope current tests caught that. >>> >>> They would be tested in >>> vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ (own call stacks), >>> and getstacktr003 (call stacks in other thread). >>> >>> >>>> Speaking of tests ... >>>> >>>> In the native code I think you need to check the success of all JNI >>>> methods that can throw exceptions - otherwise I believe the tests >>>> may trigger warnings if -Xcheck:jni is used with them. See for example: >>>> >>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>> >>> >>> I updated testcases to check JNI and JVMTI function calls. >>> >>> >>>> In the Java code the target thread: >>>> >>>> ?? 45???? public void run() { >>>> ?? 46?????? try { >>>> ?? 47?????????? synchronized (lock) { >>>> ?? 48?????????????? lock.wait(); >>>> ?? 49?????????????? System.out.println("OK"); >>>> ?? 50?????????? } >>>> >>>> is potentially susceptible to spurious wakeups. Using a >>>> CountDownLatch would be robust. >>> >>> Fixed. >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>> Hi Yasumasa, >>>>>> >>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>> Hi David, >>>>>>> >>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>> >>>>>>>>>> IIUC at this point the _calling_thread is the current thread, >>>>>>>>>> so we can use: >>>>>>>>>> >>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>> >>>>>>> If so, we can call make_local() in L1272 without JavaThread (or >>>>>>> we can pass current thread to make_local()). Is it right? >>>>>>> >>>>>>> ``` >>>>>>> 1271???? ResourceMark rm; >>>>>>> 1272 >>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>> thread_oop), >>>>>>> 1273??????????????????????????? jt, thread_oop); >>>>>>> ``` >>>>>> >>>>>> Sorry I got confused, _calling_thread may not be the current >>>>>> thread as we could be executing the handshake in the target thread >>>>>> itself. So the ResourceMark is correct as-is (implicitly for >>>>>> current thread). >>>>>> >>>>>> The argument to fill_frames will be used in the jvmtiStackInfo and >>>>>> passed back to the _calling_thread, so it must be created via >>>>>> make_local(_calling_thread, ...) as you presently have. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>>> Hi David, >>>>>>>>> >>>>>>>>> Thank you for reviewing! I will update new webrev tomorrow. >>>>>>>>> >>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>> >>>>>>>>>> ? 498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>> ? 499 private: >>>>>>>>>> ? 500?? JavaThread *_calling_thread; >>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>> ? 502?? MultipleStackTracesCollector _collector; >>>>>>>>>> >>>>>>>>>> You can't have a StackObj as a member of another class like >>>>>>>>>> that as it may not be on the stack. I think >>>>>>>>>> MultipleStackTracesCollector should not extend any allocation >>>>>>>>>> class, and should always be embedded directly in another class. >>>>>>>>> >>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>> Is it ok as below? >>>>>>>>> >>>>>>>>> ``` >>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>> ??? : >>>>>>>>> } >>>>>>>>> >>>>>>>>> class GetAllStackTraces : public VM_Operation { >>>>>>>>> ?? private: >>>>>>>>> ???? MultipleStackTracesCollector _collector; >>>>>>>>> } >>>>>>>>> ``` >>>>>>>> >>>>>>>> Yes that I what I meant. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> ----- >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>> Hi Yasumasa, >>>>>>>>>> >>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>> >>>>>>>>>>> I updated webrev for 8242428. Could you review again? >>>>>>>>>>> This change migrate to use direct handshake for >>>>>>>>>>> GetStackTrace() and GetThreadListStackTraces() (when >>>>>>>>>>> thread_count == 1). >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>> >>>>>>>>>> This looks really good now! I only have a few nits below. >>>>>>>>>> There is one thing I don't like about it but it requires a >>>>>>>>>> change to the main Handshake logic to address - in >>>>>>>>>> JvmtiEnv::GetThreadListStackTraces you have to create a >>>>>>>>>> ThreadsListHandle to convert the jthread to a JavaThread, but >>>>>>>>>> then the Handshake::execute_direct creates another >>>>>>>>>> ThreadsListHandle internally. That's a waste. I will discuss >>>>>>>>>> with Robbin and file a RFE to have an overload of >>>>>>>>>> execute_direct that takes an existing TLH. Actually it's worse >>>>>>>>>> than that because we have another TLH in use at the entry >>>>>>>>>> point for the JVMTI functions, so I think there may be some >>>>>>>>>> scope for simplifying the use of TLH instances - future RFE. >>>>>>>>>> >>>>>>>>>> --- >>>>>>>>>> >>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>> >>>>>>>>>> ??451?? GetStackTraceClosure(JvmtiEnv *env, jint start_depth, >>>>>>>>>> jint max_count, >>>>>>>>>> ??452??????????????????????? jvmtiFrameInfo* frame_buffer, >>>>>>>>>> jint* count_ptr) >>>>>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>>>>> ??454?????? _env(env), _start_depth(start_depth), >>>>>>>>>> _max_count(max_count), >>>>>>>>>> ??455?????? _frame_buffer(frame_buffer), _count_ptr(count_ptr), >>>>>>>>>> ??456?????? _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>> >>>>>>>>>> Nit: can you do one initializer per line please. >>>>>>>>>> >>>>>>>>>> This looks wrong: >>>>>>>>>> >>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>> >>>>>>>>>> ??498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>> ??499 private: >>>>>>>>>> ??500?? JavaThread *_calling_thread; >>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>> ??502?? MultipleStackTracesCollector _collector; >>>>>>>>>> >>>>>>>>>> You can't have a StackObj as a member of another class like >>>>>>>>>> that as it may not be on the stack. I think >>>>>>>>>> MultipleStackTracesCollector should not extend any allocation >>>>>>>>>> class, and should always be embedded directly in another class. >>>>>>>>>> >>>>>>>>>> 481?? MultipleStackTracesCollector(JvmtiEnv *env, jint >>>>>>>>>> max_frame_count) { >>>>>>>>>> ??482???? _env = env; >>>>>>>>>> ??483???? _max_frame_count = max_frame_count; >>>>>>>>>> ??484???? _frame_count_total = 0; >>>>>>>>>> ??485???? _head = NULL; >>>>>>>>>> ??486???? _stack_info = NULL; >>>>>>>>>> ??487???? _result = JVMTI_ERROR_NONE; >>>>>>>>>> ??488?? } >>>>>>>>>> >>>>>>>>>> As you are touching this can you change it to use an >>>>>>>>>> initializer list as you did for the HandshakeClosure, and >>>>>>>>>> please keep one item per line. >>>>>>>>>> >>>>>>>>>> --- >>>>>>>>>> >>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>> >>>>>>>>>> ??820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>> ??821 java_thread->is_thread_fully_suspended(false, >>>>>>>>>> &debug_bits) || >>>>>>>>>> ??822????????? current_thread == >>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>> ??823????????? "at safepoint / handshake or target thread is >>>>>>>>>> suspended"); >>>>>>>>>> >>>>>>>>>> I don't think the suspension check is necessary, as even if >>>>>>>>>> the target is suspended we must still be at a safepoint or in >>>>>>>>>> a handshake with it. Makes me wonder if we used to allow a >>>>>>>>>> racy stacktrace operation on a suspended thread, assuming it >>>>>>>>>> would remain suspended? >>>>>>>>>> >>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>> 1269 >>>>>>>>>> 1270?? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>> >>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>> >>>>>>>>>> 1272 >>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>> thread_oop), >>>>>>>>>> 1273??????????????????????????? jt, thread_oop); >>>>>>>>>> >>>>>>>>>> It is frustrating that this entire call chain started with a >>>>>>>>>> jthread reference, which we converted to a JavaThread, only to >>>>>>>>>> eventually need to convert it back to a jthread! I think there >>>>>>>>>> is some scope for simplification here but not as part of this >>>>>>>>>> change. >>>>>>>>>> >>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>> >>>>>>>>>> IIUC at this point the _calling_thread is the current thread, >>>>>>>>>> so we can use: >>>>>>>>>> >>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>> >>>>>>>>>> --- >>>>>>>>>> >>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>> >>>>>>>>>> I'm still pondering the test logic but wanted to send this now. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> David >>>>>>>>>> ----- >>>>>>>>>>> VM_GetThreadListStackTrace (for GetThreadListStackTraces) and >>>>>>>>>>> VM_GetAllStackTraces (for GetAllStackTraces) have inherited >>>>>>>>>>> VM_GetMultipleStackTraces VM operation which provides the >>>>>>>>>>> feature to generate jvmtiStackInfo. I modified >>>>>>>>>>> VM_GetMultipleStackTraces to a normal C++ class to share with >>>>>>>>>>> HandshakeClosure for GetThreadListStackTraces >>>>>>>>>>> (GetSingleStackTraceClosure). >>>>>>>>>>> >>>>>>>>>>> Also I added new testcases which test >>>>>>>>>>> GetThreadListStackTraces() with thread_count == 1 and with >>>>>>>>>>> all threads. >>>>>>>>>>> >>>>>>>>>>> This change has been tested in serviceability/jvmti >>>>>>>>>>> serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi >>>>>>>>>>> vmTestbase/nsk/jdwp. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>>>> Hi all, >>>>>>>>>>>> >>>>>>>>>>>> Please review this change: >>>>>>>>>>>> >>>>>>>>>>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>> ?? webrev: >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>> >>>>>>>>>>>> This change replace following VM operations to direct >>>>>>>>>>>> handshake. >>>>>>>>>>>> >>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>>>>> ??- VM_GetThreadListStackTraces (GetThreadListStackTrace()) >>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>> >>>>>>>>>>>> GetThreadListStackTrace() uses direct handshake if thread >>>>>>>>>>>> count == 1. In other case (thread count > 1), it would be >>>>>>>>>>>> performed as VM operation (VM_GetThreadListStackTraces). >>>>>>>>>>>> Caller of VM_GetCurrentLocation >>>>>>>>>>>> (JvmtiEnvThreadState::reset_current_location()) might be >>>>>>>>>>>> called at safepoint. So I added safepoint check in its caller. >>>>>>>>>>>> >>>>>>>>>>>> This change has been tested in serviceability/jvmti >>>>>>>>>>>> serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi >>>>>>>>>>>> vmTestbase/ns >>>>>>>>>>>> k/jdwp. >>>>>>>>>>>> >>>>>>>>>>>> Also I tested it on submit repo, then it has execution error >>>>>>>>>>>> (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) due >>>>>>>>>>>> to dependency error. So I think it does not occur by this >>>>>>>>>>>> change. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >> From igor.ignatyev at oracle.com Fri Jul 3 02:59:35 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 2 Jul 2020 19:59:35 -0700 Subject: [15] RFR(T) 8247527: serviceability/dcmd/gc/HeapDumpCompressedTest.java fails with Graal + ZGC In-Reply-To: References: Message-ID: Hi David, it's in my todo list to improve this situation and have vm.gc.X to take selected JIT into account; and update existing (>200) occurrences of 'vm.gc.X & !vm.graal.enabled' -- Igor > On Jul 2, 2020, at 7:25 PM, David Holmes wrote: > > Hi Vladimir, > > On 3/07/2020 12:02 pm, Vladimir Kozlov wrote: >> https://cr.openjdk.java.net/~kvn/8247527/webrev.00/ >> https://bugs.openjdk.java.net/browse/JDK-8247527 >> Test should have @requires which excludes running Graal with GC which it does not support. > > I find it somewhat disturbing that a generic test has to know about the limitations between GCs and Graal! > > I would have been more inclined to just exclude this test when running with Graal, even if that theoretically reduced the test coverage in a ting way. > > If/When Graal supports these other GCs who will remember to re-enable these test cases? > > Thanks, > David > >> Testing: hs-tier1,hs-tier4-graal >> Thanks, >> Vladimir From daniel.daugherty at oracle.com Fri Jul 3 03:24:06 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 2 Jul 2020 23:24:06 -0400 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: <0e359819-618f-c17b-a427-bb7e4daa2c58@oracle.com> References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <2ea2f5d8-5c43-a7d3-d525-cd44c4009d25@oss.nttdata.com> <65889ab4-6b7e-86fb-59cc-012520b83138@oracle.com> <4b7df6f8-c8f7-fedf-6366-b885d2235b58@oss.nttdata.com> <74c05dfd-ffb2-b73d-66e5-dc98752729f5@oss.nttdata.com> <763793b1-9506-44c6-0681-1c71c44a44fd@oss.nttdata.com> <5107c478-1de2-7a49-ee71-dfad6a99a667@oracle.com> <90cdfef9-e558-0d25-33f5-25b7d65bb83b@oss.nttdata.com> <0e359819-618f-c17b-a427-bb7e4daa2c58@oracle.com> Message-ID: <3f890bed-6892-e64d-b21b-ca0cbe11784e@oracle.com> On 7/2/20 10:50 PM, David Holmes wrote: > Sorry I'm responding here without seeing latest webrev but there is > enough context I think ... > > On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >> Hi Dan, >> >> Thanks for your comment! >> >> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>> Hi David, >>>> >>>> I upload new webrev. Could you review again? >>>> >>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>> >>> src/hotspot/share/prims/jvmtiEnv.cpp >>> ???? L1542: ??? // Get stack trace with handshake >>> ???????? nit - please add a period at the end. >> >> I will fix it. >> >> >>> ???? L1591: ??? *stack_info_ptr = op.stack_info(); >>> ???????? The return parameter should not be touched unless the return >>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>> >>> ???? old L1582: ? if (err == JVMTI_ERROR_NONE) { >>> ???????? Please restore this check. The return parameter should not >>> ???????? be touched unless the return code in 'err' == >>> JVMTI_ERROR_NONE. >> >> I will fix it. > > But op.stack_info() will return NULL if the error is not > JVMTI_ERROR_NONE. Are you (Dan) concerned about someone passing in a > non-null/initialized out-pointer that will be reset to NULL if there > was an error? Actually the way we used to test this in POSIX tests is to call an API with known bad parameters and the return parameter ptr set to NULL. If the return parameter ptr was touched when an error should have been detected on an earlier parameter, then the test failed. > >> >>> ???? L1272: ? if (!jt->is_exiting() && (thread_oop != NULL)) { >>> ???????? nit - extra parens around the second expression. >> >> I will fix it. >> >> >>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>> ???? old L1532: ? _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>> ???????? This deletion of the _result field threw me for a minute >>> and then >>> ???? ? ? I figured out that the field is init to >>> JVMTI_ERROR_THREAD_NOT_ALIVE >>> ???????? in the constructor. >>> >>> ???? L1553: ? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>> ???????? nit - extra parens around the second expression. >> >> I will fix it. >> >> >>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>> ???? No comments. >>> >>> src/hotspot/share/runtime/vmOperations.hpp >>> ???? No comments. >>> >>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>> >>> ???? No comments. >>> >>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>> >>> ???? L64: ??????? startSignal.countDown(); >>> ???????? I was expecting this to be a call to await() instead of >>> ???????? countDown(). What am I missing here? >>> >>> ???????? I think this test might be passing by accident right now, >>> but... >> >> Main thread (which call JVMTI functions to test) should wait until >> test thread is ready. >> So main thread would wait startSignal, and test thread would count down. > > No! > > The test thread that previously called obj.wait() now calls > latch.await(). > > The main thread that previously called obj.notify() now calls > latch.countDown(). > > The main thread continues to spin until it sees the target is WAITING > before proceeding with the test. Here's the flow as I see it: main thread ? - start worker thread ? - startSignal.await() ??? - main is now blocked worker thread ? - startSignal.countDown() ??? - main is now unblocked ? - stopSignal.await() ??? - worker is now blocked main thread ? - checkCallStacks(th) ? - stopSignal.countDown() ??? - worker is now unblocked ? - th.join ??? - main is now blocked worker thread ? - runs off the end of run() ??? - main is now unblocked main thread ? - run off the end of main() > >> >>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>> >>> ???? L92: ? jthreads = (jthread *)malloc(sizeof(jthread) * >>> num_threads); >>> ???????? You don't check for malloc() failure. >>> ???????? 'jthreads' is allocated but never freed. >> >> I will fix it. >> >> >>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>> >>> ???? L91: ? result = (*jvmti)->SuspendThread(jvmti, thread); >>> ???????? Why are you suspending the thread? GetAllStackTraces() and >>> ???????? GetThreadListStackTraces() do not require the target thread(s) >>> ???????? to be suspend. >>> >>> ???????? If you decide not to SuspendThread, then you don't need the >>> ???????? AddCapabilities or the ResumeThread calls. >> >> Test thread might not be entered following code (stopSignal.await()). >> We might see deferent call stack between GetAllStackTraces() and >> GetThreadListStackTraces(). We cannot control to freeze call stack of >> test thread in Java code. >> (I didn't use SuspendThread() at first, but I saw some errors which >> causes in above.) >> >> So we need to call SuspendThread() to ensure we can see same call stack. > > If you are checking that the thread is in state WAITING then it cannot > escape from that state and you can sample the stack multiple times > from any API and get the same result. > > I suspect the errors you saw were from the apparent incorrect use of > the CountDownLatch. With the flow outlined above, the worker thread should be nicely blocked in stopSignal.await() when stuff is sampled. Dan > > Cheers, > David > ----- > >> >> Thanks, >> >> Yasumasa >> >> >>> Dan >>> >>>> >>>> On 2020/07/02 15:05, David Holmes wrote: >>>>> Hi Yasumasa, >>>>> >>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>>> Hi, >>>>>> >>>>>> I uploaded new webrev. Could review again? >>>>>> >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>> >>>>> Updates look fine - thanks. >>>>> >>>>> One minor nit: >>>>> >>>>> 1274???? _collector.allocate_and_fill_stacks(1); >>>>> 1275???? _collector.set_result(JVMTI_ERROR_NONE); >>>>> >>>>> In the other places where you use _collector you rely on result >>>>> being initialized to JVMTI_ERROR_NONE, rather than setting it >>>>> directly after allocate_and_fill_stacks(). >>>> >>>> Fixed. >>>> >>>> >>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>> >>>>>>>>>>> ? 820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>> ? 821 java_thread->is_thread_fully_suspended(false, >>>>>>>>>>> &debug_bits) || >>>>>>>>>>> ? 822????????? current_thread == >>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>> ? 823????????? "at safepoint / handshake or target thread is >>>>>>>>>>> suspended"); >>>>>>>>>>> >>>>>>>>>>> I don't think the suspension check is necessary, as even if >>>>>>>>>>> the target is suspended we must still be at a safepoint or >>>>>>>>>>> in a handshake with it. Makes me wonder if we used to allow >>>>>>>>>>> a racy stacktrace operation on a suspended thread, assuming >>>>>>>>>>> it would remain suspended? >>>>>> >>>>>> This function (JvmtiEnvBase::get_stack_trace()) can be called to >>>>>> get own stack trace. For example, we can call GetStackTrace() for >>>>>> current thread at JVMTI event. >>>>>> So I changed assert as below: >>>>>> >>>>>> ``` >>>>>> ??820?? assert(current_thread == java_thread || >>>>>> ??821????????? SafepointSynchronize::is_at_safepoint() || >>>>>> ??822????????? current_thread == java_thread->active_handshaker(), >>>>>> ??823????????? "call by myself / at safepoint / at handshake"); >>>>>> ``` >>>>> >>>>> Yep good catch. I hope current tests caught that. >>>> >>>> They would be tested in >>>> vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ (own call >>>> stacks), and getstacktr003 (call stacks in other thread). >>>> >>>> >>>>> Speaking of tests ... >>>>> >>>>> In the native code I think you need to check the success of all >>>>> JNI methods that can throw exceptions - otherwise I believe the >>>>> tests may trigger warnings if -Xcheck:jni is used with them. See >>>>> for example: >>>>> >>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>> >>>> >>>> I updated testcases to check JNI and JVMTI function calls. >>>> >>>> >>>>> In the Java code the target thread: >>>>> >>>>> ?? 45???? public void run() { >>>>> ?? 46?????? try { >>>>> ?? 47?????????? synchronized (lock) { >>>>> ?? 48?????????????? lock.wait(); >>>>> ?? 49?????????????? System.out.println("OK"); >>>>> ?? 50?????????? } >>>>> >>>>> is potentially susceptible to spurious wakeups. Using a >>>>> CountDownLatch would be robust. >>>> >>>> Fixed. >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>> Hi Yasumasa, >>>>>>> >>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>>> Hi David, >>>>>>>> >>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>> >>>>>>>>>>> IIUC at this point the _calling_thread is the current >>>>>>>>>>> thread, so we can use: >>>>>>>>>>> >>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>> >>>>>>>> If so, we can call make_local() in L1272 without JavaThread (or >>>>>>>> we can pass current thread to make_local()). Is it right? >>>>>>>> >>>>>>>> ``` >>>>>>>> 1271???? ResourceMark rm; >>>>>>>> 1272 >>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>> thread_oop), >>>>>>>> 1273??????????????????????????? jt, thread_oop); >>>>>>>> ``` >>>>>>> >>>>>>> Sorry I got confused, _calling_thread may not be the current >>>>>>> thread as we could be executing the handshake in the target >>>>>>> thread itself. So the ResourceMark is correct as-is (implicitly >>>>>>> for current thread). >>>>>>> >>>>>>> The argument to fill_frames will be used in the jvmtiStackInfo >>>>>>> and passed back to the _calling_thread, so it must be created >>>>>>> via make_local(_calling_thread, ...) as you presently have. >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>>>> Hi David, >>>>>>>>>> >>>>>>>>>> Thank you for reviewing! I will update new webrev tomorrow. >>>>>>>>>> >>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>> >>>>>>>>>>> ? 498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>> ? 499 private: >>>>>>>>>>> ? 500?? JavaThread *_calling_thread; >>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>> ? 502?? MultipleStackTracesCollector _collector; >>>>>>>>>>> >>>>>>>>>>> You can't have a StackObj as a member of another class like >>>>>>>>>>> that as it may not be on the stack. I think >>>>>>>>>>> MultipleStackTracesCollector should not extend any >>>>>>>>>>> allocation class, and should always be embedded directly in >>>>>>>>>>> another class. >>>>>>>>>> >>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>> Is it ok as below? >>>>>>>>>> >>>>>>>>>> ``` >>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>> ??? : >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> class GetAllStackTraces : public VM_Operation { >>>>>>>>>> ?? private: >>>>>>>>>> ???? MultipleStackTracesCollector _collector; >>>>>>>>>> } >>>>>>>>>> ``` >>>>>>>>> >>>>>>>>> Yes that I what I meant. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> David >>>>>>>>> ----- >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>> >>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>> >>>>>>>>>>>> I updated webrev for 8242428. Could you review again? >>>>>>>>>>>> This change migrate to use direct handshake for >>>>>>>>>>>> GetStackTrace() and GetThreadListStackTraces() (when >>>>>>>>>>>> thread_count == 1). >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>> >>>>>>>>>>> This looks really good now! I only have a few nits below. >>>>>>>>>>> There is one thing I don't like about it but it requires a >>>>>>>>>>> change to the main Handshake logic to address - in >>>>>>>>>>> JvmtiEnv::GetThreadListStackTraces you have to create a >>>>>>>>>>> ThreadsListHandle to convert the jthread to a JavaThread, >>>>>>>>>>> but then the Handshake::execute_direct creates another >>>>>>>>>>> ThreadsListHandle internally. That's a waste. I will discuss >>>>>>>>>>> with Robbin and file a RFE to have an overload of >>>>>>>>>>> execute_direct that takes an existing TLH. Actually it's >>>>>>>>>>> worse than that because we have another TLH in use at the >>>>>>>>>>> entry point for the JVMTI functions, so I think there may be >>>>>>>>>>> some scope for simplifying the use of TLH instances - future >>>>>>>>>>> RFE. >>>>>>>>>>> >>>>>>>>>>> --- >>>>>>>>>>> >>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>> >>>>>>>>>>> ??451?? GetStackTraceClosure(JvmtiEnv *env, jint >>>>>>>>>>> start_depth, jint max_count, >>>>>>>>>>> ??452??????????????????????? jvmtiFrameInfo* frame_buffer, >>>>>>>>>>> jint* count_ptr) >>>>>>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>>>>>> ??454?????? _env(env), _start_depth(start_depth), >>>>>>>>>>> _max_count(max_count), >>>>>>>>>>> ??455?????? _frame_buffer(frame_buffer), _count_ptr(count_ptr), >>>>>>>>>>> ??456 _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>>> >>>>>>>>>>> Nit: can you do one initializer per line please. >>>>>>>>>>> >>>>>>>>>>> This looks wrong: >>>>>>>>>>> >>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>> >>>>>>>>>>> ??498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>> ??499 private: >>>>>>>>>>> ??500?? JavaThread *_calling_thread; >>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>> ??502?? MultipleStackTracesCollector _collector; >>>>>>>>>>> >>>>>>>>>>> You can't have a StackObj as a member of another class like >>>>>>>>>>> that as it may not be on the stack. I think >>>>>>>>>>> MultipleStackTracesCollector should not extend any >>>>>>>>>>> allocation class, and should always be embedded directly in >>>>>>>>>>> another class. >>>>>>>>>>> >>>>>>>>>>> 481?? MultipleStackTracesCollector(JvmtiEnv *env, jint >>>>>>>>>>> max_frame_count) { >>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>> ??483???? _max_frame_count = max_frame_count; >>>>>>>>>>> ??484???? _frame_count_total = 0; >>>>>>>>>>> ??485???? _head = NULL; >>>>>>>>>>> ??486???? _stack_info = NULL; >>>>>>>>>>> ??487???? _result = JVMTI_ERROR_NONE; >>>>>>>>>>> ??488?? } >>>>>>>>>>> >>>>>>>>>>> As you are touching this can you change it to use an >>>>>>>>>>> initializer list as you did for the HandshakeClosure, and >>>>>>>>>>> please keep one item per line. >>>>>>>>>>> >>>>>>>>>>> --- >>>>>>>>>>> >>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>> >>>>>>>>>>> ??820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>> ??821 java_thread->is_thread_fully_suspended(false, >>>>>>>>>>> &debug_bits) || >>>>>>>>>>> ??822????????? current_thread == >>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>> ??823????????? "at safepoint / handshake or target thread is >>>>>>>>>>> suspended"); >>>>>>>>>>> >>>>>>>>>>> I don't think the suspension check is necessary, as even if >>>>>>>>>>> the target is suspended we must still be at a safepoint or >>>>>>>>>>> in a handshake with it. Makes me wonder if we used to allow >>>>>>>>>>> a racy stacktrace operation on a suspended thread, assuming >>>>>>>>>>> it would remain suspended? >>>>>>>>>>> >>>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>>> 1269 >>>>>>>>>>> 1270?? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>> >>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>> >>>>>>>>>>> 1272 >>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>> thread_oop), >>>>>>>>>>> 1273??????????????????????????? jt, thread_oop); >>>>>>>>>>> >>>>>>>>>>> It is frustrating that this entire call chain started with a >>>>>>>>>>> jthread reference, which we converted to a JavaThread, only >>>>>>>>>>> to eventually need to convert it back to a jthread! I think >>>>>>>>>>> there is some scope for simplification here but not as part >>>>>>>>>>> of this change. >>>>>>>>>>> >>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>> >>>>>>>>>>> IIUC at this point the _calling_thread is the current >>>>>>>>>>> thread, so we can use: >>>>>>>>>>> >>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>> >>>>>>>>>>> --- >>>>>>>>>>> >>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>> >>>>>>>>>>> I'm still pondering the test logic but wanted to send this now. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> David >>>>>>>>>>> ----- >>>>>>>>>>>> VM_GetThreadListStackTrace (for GetThreadListStackTraces) >>>>>>>>>>>> and VM_GetAllStackTraces (for GetAllStackTraces) have >>>>>>>>>>>> inherited VM_GetMultipleStackTraces VM operation which >>>>>>>>>>>> provides the feature to generate jvmtiStackInfo. I modified >>>>>>>>>>>> VM_GetMultipleStackTraces to a normal C++ class to share >>>>>>>>>>>> with HandshakeClosure for GetThreadListStackTraces >>>>>>>>>>>> (GetSingleStackTraceClosure). >>>>>>>>>>>> >>>>>>>>>>>> Also I added new testcases which test >>>>>>>>>>>> GetThreadListStackTraces() with thread_count == 1 and with >>>>>>>>>>>> all threads. >>>>>>>>>>>> >>>>>>>>>>>> This change has been tested in serviceability/jvmti >>>>>>>>>>>> serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi >>>>>>>>>>>> vmTestbase/nsk/jdwp. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>>>>> Hi all, >>>>>>>>>>>>> >>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>> >>>>>>>>>>>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>> ?? webrev: >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>> >>>>>>>>>>>>> This change replace following VM operations to direct >>>>>>>>>>>>> handshake. >>>>>>>>>>>>> >>>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>>>>>> ??- VM_GetThreadListStackTraces (GetThreadListStackTrace()) >>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>> >>>>>>>>>>>>> GetThreadListStackTrace() uses direct handshake if thread >>>>>>>>>>>>> count == 1. In other case (thread count > 1), it would be >>>>>>>>>>>>> performed as VM operation (VM_GetThreadListStackTraces). >>>>>>>>>>>>> Caller of VM_GetCurrentLocation >>>>>>>>>>>>> (JvmtiEnvThreadState::reset_current_location()) might be >>>>>>>>>>>>> called at safepoint. So I added safepoint check in its >>>>>>>>>>>>> caller. >>>>>>>>>>>>> >>>>>>>>>>>>> This change has been tested in serviceability/jvmti >>>>>>>>>>>>> serviceability/jdwp vmTestbase/nsk/jvmti >>>>>>>>>>>>> vmTestbase/nsk/jdi vmTestbase/ns >>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>> >>>>>>>>>>>>> Also I tested it on submit repo, then it has execution >>>>>>>>>>>>> error >>>>>>>>>>>>> (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) >>>>>>>>>>>>> due to dependency error. So I think it does not occur by >>>>>>>>>>>>> this change. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>> From suenaga at oss.nttdata.com Fri Jul 3 04:27:15 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Fri, 3 Jul 2020 13:27:15 +0900 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: <3f890bed-6892-e64d-b21b-ca0cbe11784e@oracle.com> References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <2ea2f5d8-5c43-a7d3-d525-cd44c4009d25@oss.nttdata.com> <65889ab4-6b7e-86fb-59cc-012520b83138@oracle.com> <4b7df6f8-c8f7-fedf-6366-b885d2235b58@oss.nttdata.com> <74c05dfd-ffb2-b73d-66e5-dc98752729f5@oss.nttdata.com> <763793b1-9506-44c6-0681-1c71c44a44fd@oss.nttdata.com> <5107c478-1de2-7a49-ee71-dfad6a99a667@oracle.com> <90cdfef9-e558-0d25-33f5-25b7d65bb83b@oss.nttdata.com> <0e359819-618f-c17b-a427-bb7e4daa2c58@oracle.com> <3f890bed-6892-e64d-b21b-ca0cbe11784e@oracle.com> Message-ID: <7dc1903b-6295-3864-6301-4c02aacdb971@oss.nttdata.com> On 2020/07/03 12:24, Daniel D. Daugherty wrote: > On 7/2/20 10:50 PM, David Holmes wrote: >> Sorry I'm responding here without seeing latest webrev but there is enough context I think ... >> >> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>> Hi Dan, >>> >>> Thanks for your comment! >>> >>> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>> Hi David, >>>>> >>>>> I upload new webrev. Could you review again? >>>>> >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>> >>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>> ???? L1542: ??? // Get stack trace with handshake >>>> ???????? nit - please add a period at the end. >>> >>> I will fix it. >>> >>> >>>> ???? L1591: ??? *stack_info_ptr = op.stack_info(); >>>> ???????? The return parameter should not be touched unless the return >>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>> >>>> ???? old L1582: ? if (err == JVMTI_ERROR_NONE) { >>>> ???????? Please restore this check. The return parameter should not >>>> ???????? be touched unless the return code in 'err' == JVMTI_ERROR_NONE. >>> >>> I will fix it. >> >> But op.stack_info() will return NULL if the error is not JVMTI_ERROR_NONE. Are you (Dan) concerned about someone passing in a non-null/initialized out-pointer that will be reset to NULL if there was an error? > > Actually the way we used to test this in POSIX tests is to call > an API with known bad parameters and the return parameter ptr > set to NULL. If the return parameter ptr was touched when an > error should have been detected on an earlier parameter, then > the test failed. > > >> >>> >>>> ???? L1272: ? if (!jt->is_exiting() && (thread_oop != NULL)) { >>>> ???????? nit - extra parens around the second expression. >>> >>> I will fix it. >>> >>> >>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>> ???? old L1532: ? _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>>> ???????? This deletion of the _result field threw me for a minute and then >>>> ???? ? ? I figured out that the field is init to JVMTI_ERROR_THREAD_NOT_ALIVE >>>> ???????? in the constructor. >>>> >>>> ???? L1553: ? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>> ???????? nit - extra parens around the second expression. >>> >>> I will fix it. >>> >>> >>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>> ???? No comments. >>>> >>>> src/hotspot/share/runtime/vmOperations.hpp >>>> ???? No comments. >>>> >>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>> ???? No comments. >>>> >>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>> ???? L64: ??????? startSignal.countDown(); >>>> ???????? I was expecting this to be a call to await() instead of >>>> ???????? countDown(). What am I missing here? >>>> >>>> ???????? I think this test might be passing by accident right now, but... >>> >>> Main thread (which call JVMTI functions to test) should wait until test thread is ready. >>> So main thread would wait startSignal, and test thread would count down. >> >> No! >> >> The test thread that previously called obj.wait() now calls latch.await(). >> >> The main thread that previously called obj.notify() now calls latch.countDown(). >> >> The main thread continues to spin until it sees the target is WAITING before proceeding with the test. If I add spin wait to wait until transit target thread state is WAITING (as following), we don't need to call SuspendThread(). Which is better? ``` /* Wait until the thread state transits to "waiting" */ while (th.getState() != Thread.State.WAITING) { Thread.onSpinWait(); } ``` For simplify, spin wait is prefer to OneGetThreadListStackTraces.java in webrev.03. Thanks, Yasumasa > Here's the flow as I see it: > > main thread > ? - start worker thread > ? - startSignal.await() > ??? - main is now blocked > > worker thread > ? - startSignal.countDown() > ??? - main is now unblocked > ? - stopSignal.await() > ??? - worker is now blocked > > main thread > ? - checkCallStacks(th) > ? - stopSignal.countDown() > ??? - worker is now unblocked > ? - th.join > ??? - main is now blocked > > worker thread > ? - runs off the end of run() > ??? - main is now unblocked > > main thread > ? - run off the end of main() > > >> >>> >>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>> ???? L92: ? jthreads = (jthread *)malloc(sizeof(jthread) * num_threads); >>>> ???????? You don't check for malloc() failure. >>>> ???????? 'jthreads' is allocated but never freed. >>> >>> I will fix it. >>> >>> >>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>> ???? L91: ? result = (*jvmti)->SuspendThread(jvmti, thread); >>>> ???????? Why are you suspending the thread? GetAllStackTraces() and >>>> ???????? GetThreadListStackTraces() do not require the target thread(s) >>>> ???????? to be suspend. >>>> >>>> ???????? If you decide not to SuspendThread, then you don't need the >>>> ???????? AddCapabilities or the ResumeThread calls. >>> >>> Test thread might not be entered following code (stopSignal.await()). We might see deferent call stack between GetAllStackTraces() and GetThreadListStackTraces(). We cannot control to freeze call stack of test thread in Java code. >>> (I didn't use SuspendThread() at first, but I saw some errors which causes in above.) >>> >>> So we need to call SuspendThread() to ensure we can see same call stack. >> >> If you are checking that the thread is in state WAITING then it cannot escape from that state and you can sample the stack multiple times from any API and get the same result. >> >> I suspect the errors you saw were from the apparent incorrect use of the CountDownLatch. > > With the flow outlined above, the worker thread should be > nicely blocked in stopSignal.await() when stuff is sampled. > > Dan > > >> >> Cheers, >> David >> ----- >> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>>> Dan >>>> >>>>> >>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>> Hi Yasumasa, >>>>>> >>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>>>> Hi, >>>>>>> >>>>>>> I uploaded new webrev. Could review again? >>>>>>> >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>> >>>>>> Updates look fine - thanks. >>>>>> >>>>>> One minor nit: >>>>>> >>>>>> 1274???? _collector.allocate_and_fill_stacks(1); >>>>>> 1275???? _collector.set_result(JVMTI_ERROR_NONE); >>>>>> >>>>>> In the other places where you use _collector you rely on result being initialized to JVMTI_ERROR_NONE, rather than setting it directly after allocate_and_fill_stacks(). >>>>> >>>>> Fixed. >>>>> >>>>> >>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>> >>>>>>>>>>>> ? 820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>> ? 821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>> ? 822????????? current_thread == java_thread->active_handshaker(), >>>>>>>>>>>> ? 823????????? "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>> >>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>> >>>>>>> This function (JvmtiEnvBase::get_stack_trace()) can be called to get own stack trace. For example, we can call GetStackTrace() for current thread at JVMTI event. >>>>>>> So I changed assert as below: >>>>>>> >>>>>>> ``` >>>>>>> ??820?? assert(current_thread == java_thread || >>>>>>> ??821????????? SafepointSynchronize::is_at_safepoint() || >>>>>>> ??822????????? current_thread == java_thread->active_handshaker(), >>>>>>> ??823????????? "call by myself / at safepoint / at handshake"); >>>>>>> ``` >>>>>> >>>>>> Yep good catch. I hope current tests caught that. >>>>> >>>>> They would be tested in vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ (own call stacks), and getstacktr003 (call stacks in other thread). >>>>> >>>>> >>>>>> Speaking of tests ... >>>>>> >>>>>> In the native code I think you need to check the success of all JNI methods that can throw exceptions - otherwise I believe the tests may trigger warnings if -Xcheck:jni is used with them. See for example: >>>>>> >>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>> >>>>> I updated testcases to check JNI and JVMTI function calls. >>>>> >>>>> >>>>>> In the Java code the target thread: >>>>>> >>>>>> ?? 45???? public void run() { >>>>>> ?? 46?????? try { >>>>>> ?? 47?????????? synchronized (lock) { >>>>>> ?? 48?????????????? lock.wait(); >>>>>> ?? 49?????????????? System.out.println("OK"); >>>>>> ?? 50?????????? } >>>>>> >>>>>> is potentially susceptible to spurious wakeups. Using a CountDownLatch would be robust. >>>>> >>>>> Fixed. >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>> Hi Yasumasa, >>>>>>>> >>>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>>>> Hi David, >>>>>>>>> >>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>> >>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>> >>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>> >>>>>>>>> If so, we can call make_local() in L1272 without JavaThread (or we can pass current thread to make_local()). Is it right? >>>>>>>>> >>>>>>>>> ``` >>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>> 1273??????????????????????????? jt, thread_oop); >>>>>>>>> ``` >>>>>>>> >>>>>>>> Sorry I got confused, _calling_thread may not be the current thread as we could be executing the handshake in the target thread itself. So the ResourceMark is correct as-is (implicitly for current thread). >>>>>>>> >>>>>>>> The argument to fill_frames will be used in the jvmtiStackInfo and passed back to the _calling_thread, so it must be created via make_local(_calling_thread, ...) as you presently have. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi David, >>>>>>>>>>> >>>>>>>>>>> Thank you for reviewing! I will update new webrev tomorrow. >>>>>>>>>>> >>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>> >>>>>>>>>>>> ? 498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>> ? 499 private: >>>>>>>>>>>> ? 500?? JavaThread *_calling_thread; >>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>> ? 502?? MultipleStackTracesCollector _collector; >>>>>>>>>>>> >>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>> >>>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>>> Is it ok as below? >>>>>>>>>>> >>>>>>>>>>> ``` >>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>> ??? : >>>>>>>>>>> } >>>>>>>>>>> >>>>>>>>>>> class GetAllStackTraces : public VM_Operation { >>>>>>>>>>> ?? private: >>>>>>>>>>> ???? MultipleStackTracesCollector _collector; >>>>>>>>>>> } >>>>>>>>>>> ``` >>>>>>>>>> >>>>>>>>>> Yes that I what I meant. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> David >>>>>>>>>> ----- >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>> >>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>> >>>>>>>>>>>>> I updated webrev for 8242428. Could you review again? >>>>>>>>>>>>> This change migrate to use direct handshake for GetStackTrace() and GetThreadListStackTraces() (when thread_count == 1). >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>> >>>>>>>>>>>> This looks really good now! I only have a few nits below. There is one thing I don't like about it but it requires a change to the main Handshake logic to address - in JvmtiEnv::GetThreadListStackTraces you have to create a ThreadsListHandle to convert the jthread to a JavaThread, but then the Handshake::execute_direct creates another ThreadsListHandle internally. That's a waste. I will discuss with Robbin and file a RFE to have an overload of execute_direct that takes an existing TLH. Actually it's worse than that because we have another TLH in use at the entry point for the JVMTI functions, so I think there may be some scope for simplifying the use of TLH instances - future RFE. >>>>>>>>>>>> >>>>>>>>>>>> --- >>>>>>>>>>>> >>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>> >>>>>>>>>>>> ??451?? GetStackTraceClosure(JvmtiEnv *env, jint start_depth, jint max_count, >>>>>>>>>>>> ??452??????????????????????? jvmtiFrameInfo* frame_buffer, jint* count_ptr) >>>>>>>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>>>>>>> ??454?????? _env(env), _start_depth(start_depth), _max_count(max_count), >>>>>>>>>>>> ??455?????? _frame_buffer(frame_buffer), _count_ptr(count_ptr), >>>>>>>>>>>> ??456 _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>>>> >>>>>>>>>>>> Nit: can you do one initializer per line please. >>>>>>>>>>>> >>>>>>>>>>>> This looks wrong: >>>>>>>>>>>> >>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>> >>>>>>>>>>>> ??498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>> ??499 private: >>>>>>>>>>>> ??500?? JavaThread *_calling_thread; >>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>> ??502?? MultipleStackTracesCollector _collector; >>>>>>>>>>>> >>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>> >>>>>>>>>>>> 481?? MultipleStackTracesCollector(JvmtiEnv *env, jint max_frame_count) { >>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>> ??483???? _max_frame_count = max_frame_count; >>>>>>>>>>>> ??484???? _frame_count_total = 0; >>>>>>>>>>>> ??485???? _head = NULL; >>>>>>>>>>>> ??486???? _stack_info = NULL; >>>>>>>>>>>> ??487???? _result = JVMTI_ERROR_NONE; >>>>>>>>>>>> ??488?? } >>>>>>>>>>>> >>>>>>>>>>>> As you are touching this can you change it to use an initializer list as you did for the HandshakeClosure, and please keep one item per line. >>>>>>>>>>>> >>>>>>>>>>>> --- >>>>>>>>>>>> >>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>> >>>>>>>>>>>> ??820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>> ??821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>> ??822????????? current_thread == java_thread->active_handshaker(), >>>>>>>>>>>> ??823????????? "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>> >>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>> >>>>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>>>> 1269 >>>>>>>>>>>> 1270?? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>> >>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>> >>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>> 1273??????????????????????????? jt, thread_oop); >>>>>>>>>>>> >>>>>>>>>>>> It is frustrating that this entire call chain started with a jthread reference, which we converted to a JavaThread, only to eventually need to convert it back to a jthread! I think there is some scope for simplification here but not as part of this change. >>>>>>>>>>>> >>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>> >>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>> >>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>> >>>>>>>>>>>> --- >>>>>>>>>>>> >>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>> >>>>>>>>>>>> I'm still pondering the test logic but wanted to send this now. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> David >>>>>>>>>>>> ----- >>>>>>>>>>>>> VM_GetThreadListStackTrace (for GetThreadListStackTraces) and VM_GetAllStackTraces (for GetAllStackTraces) have inherited VM_GetMultipleStackTraces VM operation which provides the feature to generate jvmtiStackInfo. I modified VM_GetMultipleStackTraces to a normal C++ class to share with HandshakeClosure for GetThreadListStackTraces (GetSingleStackTraceClosure). >>>>>>>>>>>>> >>>>>>>>>>>>> Also I added new testcases which test GetThreadListStackTraces() with thread_count == 1 and with all threads. >>>>>>>>>>>>> >>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/nsk/jdwp. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>> >>>>>>>>>>>>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> This change replace following VM operations to direct handshake. >>>>>>>>>>>>>> >>>>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces (GetThreadListStackTrace()) >>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>> >>>>>>>>>>>>>> GetThreadListStackTrace() uses direct handshake if thread count == 1. In other case (thread count > 1), it would be performed as VM operation (VM_GetThreadListStackTraces). >>>>>>>>>>>>>> Caller of VM_GetCurrentLocation (JvmtiEnvThreadState::reset_current_location()) might be called at safepoint. So I added safepoint check in its caller. >>>>>>>>>>>>>> >>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/ns >>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Also I tested it on submit repo, then it has execution error (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) due to dependency error. So I think it does not occur by this change. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yasumasa >>>> > From david.holmes at oracle.com Fri Jul 3 05:09:28 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 3 Jul 2020 15:09:28 +1000 Subject: [15] RFR(T) 8247527: serviceability/dcmd/gc/HeapDumpCompressedTest.java fails with Graal + ZGC In-Reply-To: References: Message-ID: Hi Igor, On 3/07/2020 12:59 pm, Igor Ignatyev wrote: > Hi David, > > it's in my todo list to improve this situation and have vm.gc.X to take selected JIT into account; and update existing (>200) occurrences of 'vm.gc.X & !vm.graal.enabled' 200+ ouch! :( I guess this fix doesn't make the situation any worse in a practical sense. Thanks, David ----- > -- Igor > >> On Jul 2, 2020, at 7:25 PM, David Holmes wrote: >> >> Hi Vladimir, >> >> On 3/07/2020 12:02 pm, Vladimir Kozlov wrote: >>> https://cr.openjdk.java.net/~kvn/8247527/webrev.00/ >>> https://bugs.openjdk.java.net/browse/JDK-8247527 >>> Test should have @requires which excludes running Graal with GC which it does not support. >> >> I find it somewhat disturbing that a generic test has to know about the limitations between GCs and Graal! >> >> I would have been more inclined to just exclude this test when running with Graal, even if that theoretically reduced the test coverage in a ting way. >> >> If/When Graal supports these other GCs who will remember to re-enable these test cases? >> >> Thanks, >> David >> >>> Testing: hs-tier1,hs-tier4-graal >>> Thanks, >>> Vladimir > From david.holmes at oracle.com Fri Jul 3 05:12:47 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 3 Jul 2020 15:12:47 +1000 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: <3f890bed-6892-e64d-b21b-ca0cbe11784e@oracle.com> References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <2ea2f5d8-5c43-a7d3-d525-cd44c4009d25@oss.nttdata.com> <65889ab4-6b7e-86fb-59cc-012520b83138@oracle.com> <4b7df6f8-c8f7-fedf-6366-b885d2235b58@oss.nttdata.com> <74c05dfd-ffb2-b73d-66e5-dc98752729f5@oss.nttdata.com> <763793b1-9506-44c6-0681-1c71c44a44fd@oss.nttdata.com> <5107c478-1de2-7a49-ee71-dfad6a99a667@oracle.com> <90cdfef9-e558-0d25-33f5-25b7d65bb83b@oss.nttdata.com> <0e359819-618f-c17b-a427-bb7e4daa2c58@oracle.com> <3f890bed-6892-e64d-b21b-ca0cbe11784e@oracle.com> Message-ID: On 3/07/2020 1:24 pm, Daniel D. Daugherty wrote: > On 7/2/20 10:50 PM, David Holmes wrote: >> Sorry I'm responding here without seeing latest webrev but there is >> enough context I think ... >> >> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>> Hi Dan, >>> >>> Thanks for your comment! >>> >>> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>> Hi David, >>>>> >>>>> I upload new webrev. Could you review again? >>>>> >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>> >>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>> ???? L1542: ??? // Get stack trace with handshake >>>> ???????? nit - please add a period at the end. >>> >>> I will fix it. >>> >>> >>>> ???? L1591: ??? *stack_info_ptr = op.stack_info(); >>>> ???????? The return parameter should not be touched unless the return >>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>> >>>> ???? old L1582: ? if (err == JVMTI_ERROR_NONE) { >>>> ???????? Please restore this check. The return parameter should not >>>> ???????? be touched unless the return code in 'err' == >>>> JVMTI_ERROR_NONE. >>> >>> I will fix it. >> >> But op.stack_info() will return NULL if the error is not >> JVMTI_ERROR_NONE. Are you (Dan) concerned about someone passing in a >> non-null/initialized out-pointer that will be reset to NULL if there >> was an error? > > Actually the way we used to test this in POSIX tests is to call > an API with known bad parameters and the return parameter ptr > set to NULL. If the return parameter ptr was touched when an > error should have been detected on an earlier parameter, then > the test failed. Okay. > >> >>> >>>> ???? L1272: ? if (!jt->is_exiting() && (thread_oop != NULL)) { >>>> ???????? nit - extra parens around the second expression. >>> >>> I will fix it. >>> >>> >>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>> ???? old L1532: ? _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>>> ???????? This deletion of the _result field threw me for a minute >>>> and then >>>> ???? ? ? I figured out that the field is init to >>>> JVMTI_ERROR_THREAD_NOT_ALIVE >>>> ???????? in the constructor. >>>> >>>> ???? L1553: ? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>> ???????? nit - extra parens around the second expression. >>> >>> I will fix it. >>> >>> >>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>> ???? No comments. >>>> >>>> src/hotspot/share/runtime/vmOperations.hpp >>>> ???? No comments. >>>> >>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>> >>>> ???? No comments. >>>> >>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>> >>>> ???? L64: ??????? startSignal.countDown(); >>>> ???????? I was expecting this to be a call to await() instead of >>>> ???????? countDown(). What am I missing here? >>>> >>>> ???????? I think this test might be passing by accident right now, >>>> but... >>> >>> Main thread (which call JVMTI functions to test) should wait until >>> test thread is ready. >>> So main thread would wait startSignal, and test thread would count down. >> >> No! >> >> The test thread that previously called obj.wait() now calls >> latch.await(). >> >> The main thread that previously called obj.notify() now calls >> latch.countDown(). >> >> The main thread continues to spin until it sees the target is WAITING >> before proceeding with the test. > > Here's the flow as I see it: > > main thread > ? - start worker thread > ? - startSignal.await() > ??? - main is now blocked > > worker thread > ? - startSignal.countDown() > ??? - main is now unblocked > ? - stopSignal.await() > ??? - worker is now blocked > > main thread > ? - checkCallStacks(th) > ? - stopSignal.countDown() > ??? - worker is now unblocked > ? - th.join > ??? - main is now blocked > > worker thread > ? - runs off the end of run() > ??? - main is now unblocked > > main thread > ? - run off the end of main() That is not all what I wanted when I said to use the CountDownLatch and its broken because it is now racy. :( Cheers, David ----- > >> >>> >>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>> >>>> ???? L92: ? jthreads = (jthread *)malloc(sizeof(jthread) * >>>> num_threads); >>>> ???????? You don't check for malloc() failure. >>>> ???????? 'jthreads' is allocated but never freed. >>> >>> I will fix it. >>> >>> >>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>> >>>> ???? L91: ? result = (*jvmti)->SuspendThread(jvmti, thread); >>>> ???????? Why are you suspending the thread? GetAllStackTraces() and >>>> ???????? GetThreadListStackTraces() do not require the target thread(s) >>>> ???????? to be suspend. >>>> >>>> ???????? If you decide not to SuspendThread, then you don't need the >>>> ???????? AddCapabilities or the ResumeThread calls. >>> >>> Test thread might not be entered following code (stopSignal.await()). >>> We might see deferent call stack between GetAllStackTraces() and >>> GetThreadListStackTraces(). We cannot control to freeze call stack of >>> test thread in Java code. >>> (I didn't use SuspendThread() at first, but I saw some errors which >>> causes in above.) >>> >>> So we need to call SuspendThread() to ensure we can see same call stack. >> >> If you are checking that the thread is in state WAITING then it cannot >> escape from that state and you can sample the stack multiple times >> from any API and get the same result. >> >> I suspect the errors you saw were from the apparent incorrect use of >> the CountDownLatch. > > With the flow outlined above, the worker thread should be > nicely blocked in stopSignal.await() when stuff is sampled. > > Dan > > >> >> Cheers, >> David >> ----- >> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>>> Dan >>>> >>>>> >>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>> Hi Yasumasa, >>>>>> >>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>>>> Hi, >>>>>>> >>>>>>> I uploaded new webrev. Could review again? >>>>>>> >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>> >>>>>> Updates look fine - thanks. >>>>>> >>>>>> One minor nit: >>>>>> >>>>>> 1274???? _collector.allocate_and_fill_stacks(1); >>>>>> 1275???? _collector.set_result(JVMTI_ERROR_NONE); >>>>>> >>>>>> In the other places where you use _collector you rely on result >>>>>> being initialized to JVMTI_ERROR_NONE, rather than setting it >>>>>> directly after allocate_and_fill_stacks(). >>>>> >>>>> Fixed. >>>>> >>>>> >>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>> >>>>>>>>>>>> ? 820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>> ? 821 java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>> ? 822????????? current_thread == >>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>> ? 823????????? "at safepoint / handshake or target thread is >>>>>>>>>>>> suspended"); >>>>>>>>>>>> >>>>>>>>>>>> I don't think the suspension check is necessary, as even if >>>>>>>>>>>> the target is suspended we must still be at a safepoint or >>>>>>>>>>>> in a handshake with it. Makes me wonder if we used to allow >>>>>>>>>>>> a racy stacktrace operation on a suspended thread, assuming >>>>>>>>>>>> it would remain suspended? >>>>>>> >>>>>>> This function (JvmtiEnvBase::get_stack_trace()) can be called to >>>>>>> get own stack trace. For example, we can call GetStackTrace() for >>>>>>> current thread at JVMTI event. >>>>>>> So I changed assert as below: >>>>>>> >>>>>>> ``` >>>>>>> ??820?? assert(current_thread == java_thread || >>>>>>> ??821????????? SafepointSynchronize::is_at_safepoint() || >>>>>>> ??822????????? current_thread == java_thread->active_handshaker(), >>>>>>> ??823????????? "call by myself / at safepoint / at handshake"); >>>>>>> ``` >>>>>> >>>>>> Yep good catch. I hope current tests caught that. >>>>> >>>>> They would be tested in >>>>> vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ (own call >>>>> stacks), and getstacktr003 (call stacks in other thread). >>>>> >>>>> >>>>>> Speaking of tests ... >>>>>> >>>>>> In the native code I think you need to check the success of all >>>>>> JNI methods that can throw exceptions - otherwise I believe the >>>>>> tests may trigger warnings if -Xcheck:jni is used with them. See >>>>>> for example: >>>>>> >>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>> >>>>> >>>>> I updated testcases to check JNI and JVMTI function calls. >>>>> >>>>> >>>>>> In the Java code the target thread: >>>>>> >>>>>> ?? 45???? public void run() { >>>>>> ?? 46?????? try { >>>>>> ?? 47?????????? synchronized (lock) { >>>>>> ?? 48?????????????? lock.wait(); >>>>>> ?? 49?????????????? System.out.println("OK"); >>>>>> ?? 50?????????? } >>>>>> >>>>>> is potentially susceptible to spurious wakeups. Using a >>>>>> CountDownLatch would be robust. >>>>> >>>>> Fixed. >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>> Hi Yasumasa, >>>>>>>> >>>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>>>> Hi David, >>>>>>>>> >>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>> >>>>>>>>>>>> IIUC at this point the _calling_thread is the current >>>>>>>>>>>> thread, so we can use: >>>>>>>>>>>> >>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>> >>>>>>>>> If so, we can call make_local() in L1272 without JavaThread (or >>>>>>>>> we can pass current thread to make_local()). Is it right? >>>>>>>>> >>>>>>>>> ``` >>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>> 1272 >>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>> thread_oop), >>>>>>>>> 1273??????????????????????????? jt, thread_oop); >>>>>>>>> ``` >>>>>>>> >>>>>>>> Sorry I got confused, _calling_thread may not be the current >>>>>>>> thread as we could be executing the handshake in the target >>>>>>>> thread itself. So the ResourceMark is correct as-is (implicitly >>>>>>>> for current thread). >>>>>>>> >>>>>>>> The argument to fill_frames will be used in the jvmtiStackInfo >>>>>>>> and passed back to the _calling_thread, so it must be created >>>>>>>> via make_local(_calling_thread, ...) as you presently have. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi David, >>>>>>>>>>> >>>>>>>>>>> Thank you for reviewing! I will update new webrev tomorrow. >>>>>>>>>>> >>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>> >>>>>>>>>>>> ? 498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>> ? 499 private: >>>>>>>>>>>> ? 500?? JavaThread *_calling_thread; >>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>> ? 502?? MultipleStackTracesCollector _collector; >>>>>>>>>>>> >>>>>>>>>>>> You can't have a StackObj as a member of another class like >>>>>>>>>>>> that as it may not be on the stack. I think >>>>>>>>>>>> MultipleStackTracesCollector should not extend any >>>>>>>>>>>> allocation class, and should always be embedded directly in >>>>>>>>>>>> another class. >>>>>>>>>>> >>>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>>> Is it ok as below? >>>>>>>>>>> >>>>>>>>>>> ``` >>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>> ??? : >>>>>>>>>>> } >>>>>>>>>>> >>>>>>>>>>> class GetAllStackTraces : public VM_Operation { >>>>>>>>>>> ?? private: >>>>>>>>>>> ???? MultipleStackTracesCollector _collector; >>>>>>>>>>> } >>>>>>>>>>> ``` >>>>>>>>>> >>>>>>>>>> Yes that I what I meant. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> David >>>>>>>>>> ----- >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>> >>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>> >>>>>>>>>>>>> I updated webrev for 8242428. Could you review again? >>>>>>>>>>>>> This change migrate to use direct handshake for >>>>>>>>>>>>> GetStackTrace() and GetThreadListStackTraces() (when >>>>>>>>>>>>> thread_count == 1). >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>> >>>>>>>>>>>> This looks really good now! I only have a few nits below. >>>>>>>>>>>> There is one thing I don't like about it but it requires a >>>>>>>>>>>> change to the main Handshake logic to address - in >>>>>>>>>>>> JvmtiEnv::GetThreadListStackTraces you have to create a >>>>>>>>>>>> ThreadsListHandle to convert the jthread to a JavaThread, >>>>>>>>>>>> but then the Handshake::execute_direct creates another >>>>>>>>>>>> ThreadsListHandle internally. That's a waste. I will discuss >>>>>>>>>>>> with Robbin and file a RFE to have an overload of >>>>>>>>>>>> execute_direct that takes an existing TLH. Actually it's >>>>>>>>>>>> worse than that because we have another TLH in use at the >>>>>>>>>>>> entry point for the JVMTI functions, so I think there may be >>>>>>>>>>>> some scope for simplifying the use of TLH instances - future >>>>>>>>>>>> RFE. >>>>>>>>>>>> >>>>>>>>>>>> --- >>>>>>>>>>>> >>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>> >>>>>>>>>>>> ??451?? GetStackTraceClosure(JvmtiEnv *env, jint >>>>>>>>>>>> start_depth, jint max_count, >>>>>>>>>>>> ??452??????????????????????? jvmtiFrameInfo* frame_buffer, >>>>>>>>>>>> jint* count_ptr) >>>>>>>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>>>>>>> ??454?????? _env(env), _start_depth(start_depth), >>>>>>>>>>>> _max_count(max_count), >>>>>>>>>>>> ??455?????? _frame_buffer(frame_buffer), _count_ptr(count_ptr), >>>>>>>>>>>> ??456 _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>>>> >>>>>>>>>>>> Nit: can you do one initializer per line please. >>>>>>>>>>>> >>>>>>>>>>>> This looks wrong: >>>>>>>>>>>> >>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>> >>>>>>>>>>>> ??498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>> ??499 private: >>>>>>>>>>>> ??500?? JavaThread *_calling_thread; >>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>> ??502?? MultipleStackTracesCollector _collector; >>>>>>>>>>>> >>>>>>>>>>>> You can't have a StackObj as a member of another class like >>>>>>>>>>>> that as it may not be on the stack. I think >>>>>>>>>>>> MultipleStackTracesCollector should not extend any >>>>>>>>>>>> allocation class, and should always be embedded directly in >>>>>>>>>>>> another class. >>>>>>>>>>>> >>>>>>>>>>>> 481?? MultipleStackTracesCollector(JvmtiEnv *env, jint >>>>>>>>>>>> max_frame_count) { >>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>> ??483???? _max_frame_count = max_frame_count; >>>>>>>>>>>> ??484???? _frame_count_total = 0; >>>>>>>>>>>> ??485???? _head = NULL; >>>>>>>>>>>> ??486???? _stack_info = NULL; >>>>>>>>>>>> ??487???? _result = JVMTI_ERROR_NONE; >>>>>>>>>>>> ??488?? } >>>>>>>>>>>> >>>>>>>>>>>> As you are touching this can you change it to use an >>>>>>>>>>>> initializer list as you did for the HandshakeClosure, and >>>>>>>>>>>> please keep one item per line. >>>>>>>>>>>> >>>>>>>>>>>> --- >>>>>>>>>>>> >>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>> >>>>>>>>>>>> ??820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>> ??821 java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>> ??822????????? current_thread == >>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>> ??823????????? "at safepoint / handshake or target thread is >>>>>>>>>>>> suspended"); >>>>>>>>>>>> >>>>>>>>>>>> I don't think the suspension check is necessary, as even if >>>>>>>>>>>> the target is suspended we must still be at a safepoint or >>>>>>>>>>>> in a handshake with it. Makes me wonder if we used to allow >>>>>>>>>>>> a racy stacktrace operation on a suspended thread, assuming >>>>>>>>>>>> it would remain suspended? >>>>>>>>>>>> >>>>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>>>> 1269 >>>>>>>>>>>> 1270?? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>> >>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>> >>>>>>>>>>>> 1272 >>>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>>> thread_oop), >>>>>>>>>>>> 1273??????????????????????????? jt, thread_oop); >>>>>>>>>>>> >>>>>>>>>>>> It is frustrating that this entire call chain started with a >>>>>>>>>>>> jthread reference, which we converted to a JavaThread, only >>>>>>>>>>>> to eventually need to convert it back to a jthread! I think >>>>>>>>>>>> there is some scope for simplification here but not as part >>>>>>>>>>>> of this change. >>>>>>>>>>>> >>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>> >>>>>>>>>>>> IIUC at this point the _calling_thread is the current >>>>>>>>>>>> thread, so we can use: >>>>>>>>>>>> >>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>> >>>>>>>>>>>> --- >>>>>>>>>>>> >>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>> >>>>>>>>>>>> I'm still pondering the test logic but wanted to send this now. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> David >>>>>>>>>>>> ----- >>>>>>>>>>>>> VM_GetThreadListStackTrace (for GetThreadListStackTraces) >>>>>>>>>>>>> and VM_GetAllStackTraces (for GetAllStackTraces) have >>>>>>>>>>>>> inherited VM_GetMultipleStackTraces VM operation which >>>>>>>>>>>>> provides the feature to generate jvmtiStackInfo. I modified >>>>>>>>>>>>> VM_GetMultipleStackTraces to a normal C++ class to share >>>>>>>>>>>>> with HandshakeClosure for GetThreadListStackTraces >>>>>>>>>>>>> (GetSingleStackTraceClosure). >>>>>>>>>>>>> >>>>>>>>>>>>> Also I added new testcases which test >>>>>>>>>>>>> GetThreadListStackTraces() with thread_count == 1 and with >>>>>>>>>>>>> all threads. >>>>>>>>>>>>> >>>>>>>>>>>>> This change has been tested in serviceability/jvmti >>>>>>>>>>>>> serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi >>>>>>>>>>>>> vmTestbase/nsk/jdwp. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>> >>>>>>>>>>>>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>> ?? webrev: >>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> This change replace following VM operations to direct >>>>>>>>>>>>>> handshake. >>>>>>>>>>>>>> >>>>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces (GetThreadListStackTrace()) >>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>> >>>>>>>>>>>>>> GetThreadListStackTrace() uses direct handshake if thread >>>>>>>>>>>>>> count == 1. In other case (thread count > 1), it would be >>>>>>>>>>>>>> performed as VM operation (VM_GetThreadListStackTraces). >>>>>>>>>>>>>> Caller of VM_GetCurrentLocation >>>>>>>>>>>>>> (JvmtiEnvThreadState::reset_current_location()) might be >>>>>>>>>>>>>> called at safepoint. So I added safepoint check in its >>>>>>>>>>>>>> caller. >>>>>>>>>>>>>> >>>>>>>>>>>>>> This change has been tested in serviceability/jvmti >>>>>>>>>>>>>> serviceability/jdwp vmTestbase/nsk/jvmti >>>>>>>>>>>>>> vmTestbase/nsk/jdi vmTestbase/ns >>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Also I tested it on submit repo, then it has execution >>>>>>>>>>>>>> error >>>>>>>>>>>>>> (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) >>>>>>>>>>>>>> due to dependency error. So I think it does not occur by >>>>>>>>>>>>>> this change. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yasumasa >>>> > From david.holmes at oracle.com Fri Jul 3 05:15:02 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 3 Jul 2020 15:15:02 +1000 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: <7dc1903b-6295-3864-6301-4c02aacdb971@oss.nttdata.com> References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <2ea2f5d8-5c43-a7d3-d525-cd44c4009d25@oss.nttdata.com> <65889ab4-6b7e-86fb-59cc-012520b83138@oracle.com> <4b7df6f8-c8f7-fedf-6366-b885d2235b58@oss.nttdata.com> <74c05dfd-ffb2-b73d-66e5-dc98752729f5@oss.nttdata.com> <763793b1-9506-44c6-0681-1c71c44a44fd@oss.nttdata.com> <5107c478-1de2-7a49-ee71-dfad6a99a667@oracle.com> <90cdfef9-e558-0d25-33f5-25b7d65bb83b@oss.nttdata.com> <0e359819-618f-c17b-a427-bb7e4daa2c58@oracle.com> <3f890bed-6892-e64d-b21b-ca0cbe11784e@oracle.com> <7dc1903b-6295-3864-6301-4c02aacdb971@oss.nttdata.com> Message-ID: <04e09eb7-9483-8f63-4bb4-01c9e814802e@oracle.com> On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: > On 2020/07/03 12:24, Daniel D. Daugherty wrote: >> On 7/2/20 10:50 PM, David Holmes wrote: >>> Sorry I'm responding here without seeing latest webrev but there is >>> enough context I think ... >>> >>> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>>> Hi Dan, >>>> >>>> Thanks for your comment! >>>> >>>> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>> Hi David, >>>>>> >>>>>> I upload new webrev. Could you review again? >>>>>> >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>> >>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>> ???? L1542: ??? // Get stack trace with handshake >>>>> ???????? nit - please add a period at the end. >>>> >>>> I will fix it. >>>> >>>> >>>>> ???? L1591: ??? *stack_info_ptr = op.stack_info(); >>>>> ???????? The return parameter should not be touched unless the return >>>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>>> >>>>> ???? old L1582: ? if (err == JVMTI_ERROR_NONE) { >>>>> ???????? Please restore this check. The return parameter should not >>>>> ???????? be touched unless the return code in 'err' == >>>>> JVMTI_ERROR_NONE. >>>> >>>> I will fix it. >>> >>> But op.stack_info() will return NULL if the error is not >>> JVMTI_ERROR_NONE. Are you (Dan) concerned about someone passing in a >>> non-null/initialized out-pointer that will be reset to NULL if there >>> was an error? >> >> Actually the way we used to test this in POSIX tests is to call >> an API with known bad parameters and the return parameter ptr >> set to NULL. If the return parameter ptr was touched when an >> error should have been detected on an earlier parameter, then >> the test failed. >> >> >>> >>>> >>>>> ???? L1272: ? if (!jt->is_exiting() && (thread_oop != NULL)) { >>>>> ???????? nit - extra parens around the second expression. >>>> >>>> I will fix it. >>>> >>>> >>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>> ???? old L1532: ? _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>> ???????? This deletion of the _result field threw me for a minute >>>>> and then >>>>> ???? ? ? I figured out that the field is init to >>>>> JVMTI_ERROR_THREAD_NOT_ALIVE >>>>> ???????? in the constructor. >>>>> >>>>> ???? L1553: ? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>> ???????? nit - extra parens around the second expression. >>>> >>>> I will fix it. >>>> >>>> >>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>> ???? No comments. >>>>> >>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>> ???? No comments. >>>>> >>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>> >>>>> ???? No comments. >>>>> >>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>> >>>>> ???? L64: ??????? startSignal.countDown(); >>>>> ???????? I was expecting this to be a call to await() instead of >>>>> ???????? countDown(). What am I missing here? >>>>> >>>>> ???????? I think this test might be passing by accident right now, >>>>> but... >>>> >>>> Main thread (which call JVMTI functions to test) should wait until >>>> test thread is ready. >>>> So main thread would wait startSignal, and test thread would count >>>> down. >>> >>> No! >>> >>> The test thread that previously called obj.wait() now calls >>> latch.await(). >>> >>> The main thread that previously called obj.notify() now calls >>> latch.countDown(). >>> >>> The main thread continues to spin until it sees the target is WAITING >>> before proceeding with the test. > > If I add spin wait to wait until transit target thread state is WAITING > (as following), we don't need to call SuspendThread(). > Which is better? The original spin-wait loop checking for WAITING is better because it is the only guarantee that the target thread is blocked where you need it to be. suspending the thread is racy as you don't know exactly where the suspend will hit. Thanks, David ----- > > ``` > /* Wait until the thread state transits to "waiting" */ > while (th.getState() != Thread.State.WAITING) { > ??? Thread.onSpinWait(); > } > ``` > > For simplify, spin wait is prefer to OneGetThreadListStackTraces.java in > webrev.03. > > > Thanks, > > Yasumasa > > >> Here's the flow as I see it: >> >> main thread >> ?? - start worker thread >> ?? - startSignal.await() >> ???? - main is now blocked >> >> worker thread >> ?? - startSignal.countDown() >> ???? - main is now unblocked >> ?? - stopSignal.await() >> ???? - worker is now blocked >> >> main thread >> ?? - checkCallStacks(th) >> ?? - stopSignal.countDown() >> ???? - worker is now unblocked >> ?? - th.join >> ???? - main is now blocked >> >> worker thread >> ?? - runs off the end of run() >> ???? - main is now unblocked >> >> main thread >> ?? - run off the end of main() >> >> >>> >>>> >>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>> >>>>> ???? L92: ? jthreads = (jthread *)malloc(sizeof(jthread) * >>>>> num_threads); >>>>> ???????? You don't check for malloc() failure. >>>>> ???????? 'jthreads' is allocated but never freed. >>>> >>>> I will fix it. >>>> >>>> >>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>> >>>>> ???? L91: ? result = (*jvmti)->SuspendThread(jvmti, thread); >>>>> ???????? Why are you suspending the thread? GetAllStackTraces() and >>>>> ???????? GetThreadListStackTraces() do not require the target >>>>> thread(s) >>>>> ???????? to be suspend. >>>>> >>>>> ???????? If you decide not to SuspendThread, then you don't need the >>>>> ???????? AddCapabilities or the ResumeThread calls. >>>> >>>> Test thread might not be entered following code >>>> (stopSignal.await()). We might see deferent call stack between >>>> GetAllStackTraces() and GetThreadListStackTraces(). We cannot >>>> control to freeze call stack of test thread in Java code. >>>> (I didn't use SuspendThread() at first, but I saw some errors which >>>> causes in above.) >>>> >>>> So we need to call SuspendThread() to ensure we can see same call >>>> stack. >>> >>> If you are checking that the thread is in state WAITING then it >>> cannot escape from that state and you can sample the stack multiple >>> times from any API and get the same result. >>> >>> I suspect the errors you saw were from the apparent incorrect use of >>> the CountDownLatch. >> >> With the flow outlined above, the worker thread should be >> nicely blocked in stopSignal.await() when stuff is sampled. >> >> Dan >> >> >>> >>> Cheers, >>> David >>> ----- >>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>>> Dan >>>>> >>>>>> >>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>> Hi Yasumasa, >>>>>>> >>>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> I uploaded new webrev. Could review again? >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>> >>>>>>> Updates look fine - thanks. >>>>>>> >>>>>>> One minor nit: >>>>>>> >>>>>>> 1274???? _collector.allocate_and_fill_stacks(1); >>>>>>> 1275???? _collector.set_result(JVMTI_ERROR_NONE); >>>>>>> >>>>>>> In the other places where you use _collector you rely on result >>>>>>> being initialized to JVMTI_ERROR_NONE, rather than setting it >>>>>>> directly after allocate_and_fill_stacks(). >>>>>> >>>>>> Fixed. >>>>>> >>>>>> >>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>> >>>>>>>>>>>>> ? 820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>> ? 821 java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>>> ? 822????????? current_thread == >>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>> ? 823????????? "at safepoint / handshake or target thread >>>>>>>>>>>>> is suspended"); >>>>>>>>>>>>> >>>>>>>>>>>>> I don't think the suspension check is necessary, as even if >>>>>>>>>>>>> the target is suspended we must still be at a safepoint or >>>>>>>>>>>>> in a handshake with it. Makes me wonder if we used to allow >>>>>>>>>>>>> a racy stacktrace operation on a suspended thread, assuming >>>>>>>>>>>>> it would remain suspended? >>>>>>>> >>>>>>>> This function (JvmtiEnvBase::get_stack_trace()) can be called to >>>>>>>> get own stack trace. For example, we can call GetStackTrace() >>>>>>>> for current thread at JVMTI event. >>>>>>>> So I changed assert as below: >>>>>>>> >>>>>>>> ``` >>>>>>>> ??820?? assert(current_thread == java_thread || >>>>>>>> ??821????????? SafepointSynchronize::is_at_safepoint() || >>>>>>>> ??822????????? current_thread == java_thread->active_handshaker(), >>>>>>>> ??823????????? "call by myself / at safepoint / at handshake"); >>>>>>>> ``` >>>>>>> >>>>>>> Yep good catch. I hope current tests caught that. >>>>>> >>>>>> They would be tested in >>>>>> vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ (own call >>>>>> stacks), and getstacktr003 (call stacks in other thread). >>>>>> >>>>>> >>>>>>> Speaking of tests ... >>>>>>> >>>>>>> In the native code I think you need to check the success of all >>>>>>> JNI methods that can throw exceptions - otherwise I believe the >>>>>>> tests may trigger warnings if -Xcheck:jni is used with them. See >>>>>>> for example: >>>>>>> >>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>> >>>>>> >>>>>> I updated testcases to check JNI and JVMTI function calls. >>>>>> >>>>>> >>>>>>> In the Java code the target thread: >>>>>>> >>>>>>> ?? 45???? public void run() { >>>>>>> ?? 46?????? try { >>>>>>> ?? 47?????????? synchronized (lock) { >>>>>>> ?? 48?????????????? lock.wait(); >>>>>>> ?? 49?????????????? System.out.println("OK"); >>>>>>> ?? 50?????????? } >>>>>>> >>>>>>> is potentially susceptible to spurious wakeups. Using a >>>>>>> CountDownLatch would be robust. >>>>>> >>>>>> Fixed. >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>> Hi Yasumasa, >>>>>>>>> >>>>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>>>>> Hi David, >>>>>>>>>> >>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>> >>>>>>>>>>>>> IIUC at this point the _calling_thread is the current >>>>>>>>>>>>> thread, so we can use: >>>>>>>>>>>>> >>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>> >>>>>>>>>> If so, we can call make_local() in L1272 without JavaThread >>>>>>>>>> (or we can pass current thread to make_local()). Is it right? >>>>>>>>>> >>>>>>>>>> ``` >>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>> 1272 >>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>> thread_oop), >>>>>>>>>> 1273??????????????????????????? jt, thread_oop); >>>>>>>>>> ``` >>>>>>>>> >>>>>>>>> Sorry I got confused, _calling_thread may not be the current >>>>>>>>> thread as we could be executing the handshake in the target >>>>>>>>> thread itself. So the ResourceMark is correct as-is (implicitly >>>>>>>>> for current thread). >>>>>>>>> >>>>>>>>> The argument to fill_frames will be used in the jvmtiStackInfo >>>>>>>>> and passed back to the _calling_thread, so it must be created >>>>>>>>> via make_local(_calling_thread, ...) as you presently have. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> David >>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>>>>>> Hi David, >>>>>>>>>>>> >>>>>>>>>>>> Thank you for reviewing! I will update new webrev tomorrow. >>>>>>>>>>>> >>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>> >>>>>>>>>>>>> ? 498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>> ? 500?? JavaThread *_calling_thread; >>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>> ? 502?? MultipleStackTracesCollector _collector; >>>>>>>>>>>>> >>>>>>>>>>>>> You can't have a StackObj as a member of another class like >>>>>>>>>>>>> that as it may not be on the stack. I think >>>>>>>>>>>>> MultipleStackTracesCollector should not extend any >>>>>>>>>>>>> allocation class, and should always be embedded directly in >>>>>>>>>>>>> another class. >>>>>>>>>>>> >>>>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>> >>>>>>>>>>>> ``` >>>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>>> ??? : >>>>>>>>>>>> } >>>>>>>>>>>> >>>>>>>>>>>> class GetAllStackTraces : public VM_Operation { >>>>>>>>>>>> ?? private: >>>>>>>>>>>> ???? MultipleStackTracesCollector _collector; >>>>>>>>>>>> } >>>>>>>>>>>> ``` >>>>>>>>>>> >>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> David >>>>>>>>>>> ----- >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>> >>>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>> >>>>>>>>>>>>>> I updated webrev for 8242428. Could you review again? >>>>>>>>>>>>>> This change migrate to use direct handshake for >>>>>>>>>>>>>> GetStackTrace() and GetThreadListStackTraces() (when >>>>>>>>>>>>>> thread_count == 1). >>>>>>>>>>>>>> >>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>> >>>>>>>>>>>>> This looks really good now! I only have a few nits below. >>>>>>>>>>>>> There is one thing I don't like about it but it requires a >>>>>>>>>>>>> change to the main Handshake logic to address - in >>>>>>>>>>>>> JvmtiEnv::GetThreadListStackTraces you have to create a >>>>>>>>>>>>> ThreadsListHandle to convert the jthread to a JavaThread, >>>>>>>>>>>>> but then the Handshake::execute_direct creates another >>>>>>>>>>>>> ThreadsListHandle internally. That's a waste. I will >>>>>>>>>>>>> discuss with Robbin and file a RFE to have an overload of >>>>>>>>>>>>> execute_direct that takes an existing TLH. Actually it's >>>>>>>>>>>>> worse than that because we have another TLH in use at the >>>>>>>>>>>>> entry point for the JVMTI functions, so I think there may >>>>>>>>>>>>> be some scope for simplifying the use of TLH instances - >>>>>>>>>>>>> future RFE. >>>>>>>>>>>>> >>>>>>>>>>>>> --- >>>>>>>>>>>>> >>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>> >>>>>>>>>>>>> ??451?? GetStackTraceClosure(JvmtiEnv *env, jint >>>>>>>>>>>>> start_depth, jint max_count, >>>>>>>>>>>>> ??452??????????????????????? jvmtiFrameInfo* frame_buffer, >>>>>>>>>>>>> jint* count_ptr) >>>>>>>>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>> ??454?????? _env(env), _start_depth(start_depth), >>>>>>>>>>>>> _max_count(max_count), >>>>>>>>>>>>> ??455?????? _frame_buffer(frame_buffer), >>>>>>>>>>>>> _count_ptr(count_ptr), >>>>>>>>>>>>> ??456 _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>>>>> >>>>>>>>>>>>> Nit: can you do one initializer per line please. >>>>>>>>>>>>> >>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>> >>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>> >>>>>>>>>>>>> ??498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>> ??500?? JavaThread *_calling_thread; >>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>> ??502?? MultipleStackTracesCollector _collector; >>>>>>>>>>>>> >>>>>>>>>>>>> You can't have a StackObj as a member of another class like >>>>>>>>>>>>> that as it may not be on the stack. I think >>>>>>>>>>>>> MultipleStackTracesCollector should not extend any >>>>>>>>>>>>> allocation class, and should always be embedded directly in >>>>>>>>>>>>> another class. >>>>>>>>>>>>> >>>>>>>>>>>>> 481?? MultipleStackTracesCollector(JvmtiEnv *env, jint >>>>>>>>>>>>> max_frame_count) { >>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>> ??483???? _max_frame_count = max_frame_count; >>>>>>>>>>>>> ??484???? _frame_count_total = 0; >>>>>>>>>>>>> ??485???? _head = NULL; >>>>>>>>>>>>> ??486???? _stack_info = NULL; >>>>>>>>>>>>> ??487???? _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>> >>>>>>>>>>>>> As you are touching this can you change it to use an >>>>>>>>>>>>> initializer list as you did for the HandshakeClosure, and >>>>>>>>>>>>> please keep one item per line. >>>>>>>>>>>>> >>>>>>>>>>>>> --- >>>>>>>>>>>>> >>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>> >>>>>>>>>>>>> ??820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>> ??821 java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>>> ??822????????? current_thread == >>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>> ??823????????? "at safepoint / handshake or target thread >>>>>>>>>>>>> is suspended"); >>>>>>>>>>>>> >>>>>>>>>>>>> I don't think the suspension check is necessary, as even if >>>>>>>>>>>>> the target is suspended we must still be at a safepoint or >>>>>>>>>>>>> in a handshake with it. Makes me wonder if we used to allow >>>>>>>>>>>>> a racy stacktrace operation on a suspended thread, assuming >>>>>>>>>>>>> it would remain suspended? >>>>>>>>>>>>> >>>>>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>>>>> 1269 >>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>>> >>>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>>> >>>>>>>>>>>>> 1272 >>>>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>>>> thread_oop), >>>>>>>>>>>>> 1273??????????????????????????? jt, thread_oop); >>>>>>>>>>>>> >>>>>>>>>>>>> It is frustrating that this entire call chain started with >>>>>>>>>>>>> a jthread reference, which we converted to a JavaThread, >>>>>>>>>>>>> only to eventually need to convert it back to a jthread! I >>>>>>>>>>>>> think there is some scope for simplification here but not >>>>>>>>>>>>> as part of this change. >>>>>>>>>>>>> >>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>> >>>>>>>>>>>>> IIUC at this point the _calling_thread is the current >>>>>>>>>>>>> thread, so we can use: >>>>>>>>>>>>> >>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>>> >>>>>>>>>>>>> --- >>>>>>>>>>>>> >>>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>>> >>>>>>>>>>>>> I'm still pondering the test logic but wanted to send this >>>>>>>>>>>>> now. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> David >>>>>>>>>>>>> ----- >>>>>>>>>>>>>> VM_GetThreadListStackTrace (for GetThreadListStackTraces) >>>>>>>>>>>>>> and VM_GetAllStackTraces (for GetAllStackTraces) have >>>>>>>>>>>>>> inherited VM_GetMultipleStackTraces VM operation which >>>>>>>>>>>>>> provides the feature to generate jvmtiStackInfo. I >>>>>>>>>>>>>> modified VM_GetMultipleStackTraces to a normal C++ class >>>>>>>>>>>>>> to share with HandshakeClosure for >>>>>>>>>>>>>> GetThreadListStackTraces (GetSingleStackTraceClosure). >>>>>>>>>>>>>> >>>>>>>>>>>>>> Also I added new testcases which test >>>>>>>>>>>>>> GetThreadListStackTraces() with thread_count == 1 and with >>>>>>>>>>>>>> all threads. >>>>>>>>>>>>>> >>>>>>>>>>>>>> This change has been tested in serviceability/jvmti >>>>>>>>>>>>>> serviceability/jdwp vmTestbase/nsk/jvmti >>>>>>>>>>>>>> vmTestbase/nsk/jdi vmTestbase/nsk/jdwp. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>> ?? webrev: >>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> This change replace following VM operations to direct >>>>>>>>>>>>>>> handshake. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces (GetThreadListStackTrace()) >>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> GetThreadListStackTrace() uses direct handshake if thread >>>>>>>>>>>>>>> count == 1. In other case (thread count > 1), it would be >>>>>>>>>>>>>>> performed as VM operation (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>> Caller of VM_GetCurrentLocation >>>>>>>>>>>>>>> (JvmtiEnvThreadState::reset_current_location()) might be >>>>>>>>>>>>>>> called at safepoint. So I added safepoint check in its >>>>>>>>>>>>>>> caller. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti >>>>>>>>>>>>>>> serviceability/jdwp vmTestbase/nsk/jvmti >>>>>>>>>>>>>>> vmTestbase/nsk/jdi vmTestbase/ns >>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Also I tested it on submit repo, then it has execution >>>>>>>>>>>>>>> error >>>>>>>>>>>>>>> (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) >>>>>>>>>>>>>>> due to dependency error. So I think it does not occur by >>>>>>>>>>>>>>> this change. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yasumasa >>>>> >> From chris.plummer at oracle.com Fri Jul 3 05:36:38 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 2 Jul 2020 22:36:38 -0700 Subject: RFR(T): 8248746: Undo jhsdb related exclusiveAccess.dirs changes that were done for JDK-8220295 Message-ID: <6d2e0e0e-fda7-b346-1f1c-4e6383f58c1e@oracle.com> Hello, Please review the following trivial change: https://bugs.openjdk.java.net/browse/JDK-8248746 diff --git a/test/jdk/TEST.ROOT b/test/jdk/TEST.ROOT --- a/test/jdk/TEST.ROOT +++ b/test/jdk/TEST.ROOT @@ -24,7 +24,7 @@ ?# Tests that cannot run concurrently ?exclusiveAccess.dirs=java/math/BigInteger/largeMemory \ ?java/rmi/Naming java/util/prefs sun/management/jmxremote \ -sun/tools/jstatd sun/tools/jcmd sun/tools/jhsdb sun/tools/jhsdb/heapconfig \ +sun/tools/jstatd sun/tools/jcmd \ ?sun/tools/jinfo sun/tools/jmap sun/tools/jps sun/tools/jstack sun/tools/jstat \ ?com/sun/tools/attach sun/security/mscapi java/util/stream java/util/Arrays/largeMemory \ ?java/util/BitSet/stream javax/rmi Due to some issues with running multiple Attach API tests at the same time, it was decided to add them all to exclusiveAccess.dirs. This was done under JDK-8220295. However, the jhsdb tests should never have been added since they don't use the Attach API. They use SA Attach, which is completely unrelated. I've been testing with this change for the past 6 weeks or so, and haven't seen any issues, nor would I expect any. thanks, Chris From igor.ignatyev at oracle.com Fri Jul 3 05:40:17 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 2 Jul 2020 22:40:17 -0700 Subject: RFR(T): 8248746: Undo jhsdb related exclusiveAccess.dirs changes that were done for JDK-8220295 In-Reply-To: <6d2e0e0e-fda7-b346-1f1c-4e6383f58c1e@oracle.com> References: <6d2e0e0e-fda7-b346-1f1c-4e6383f58c1e@oracle.com> Message-ID: <641570A2-6CE3-4E3B-AA10-D55386689A58@oracle.com> Hi Chris, looks good and trivial. would you consider pushing it into jdk/jdk15? -- Igor > On Jul 2, 2020, at 10:36 PM, Chris Plummer wrote: > > Hello, > > Please review the following trivial change: > > https://bugs.openjdk.java.net/browse/JDK-8248746 > > diff --git a/test/jdk/TEST.ROOT b/test/jdk/TEST.ROOT > --- a/test/jdk/TEST.ROOT > +++ b/test/jdk/TEST.ROOT > @@ -24,7 +24,7 @@ > # Tests that cannot run concurrently > exclusiveAccess.dirs=java/math/BigInteger/largeMemory \ > java/rmi/Naming java/util/prefs sun/management/jmxremote \ > -sun/tools/jstatd sun/tools/jcmd sun/tools/jhsdb sun/tools/jhsdb/heapconfig \ > +sun/tools/jstatd sun/tools/jcmd \ > sun/tools/jinfo sun/tools/jmap sun/tools/jps sun/tools/jstack sun/tools/jstat \ > com/sun/tools/attach sun/security/mscapi java/util/stream java/util/Arrays/largeMemory \ > java/util/BitSet/stream javax/rmi > > Due to some issues with running multiple Attach API tests at the same time, it was decided to add them all to exclusiveAccess.dirs. This was done under JDK-8220295. However, the jhsdb tests should never have been added since they don't use the Attach API. They use SA Attach, which is completely unrelated. I've been testing with this change for the past 6 weeks or so, and haven't seen any issues, nor would I expect any. > > thanks, > > Chris > From chris.plummer at oracle.com Fri Jul 3 06:10:08 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 2 Jul 2020 23:10:08 -0700 Subject: RFR(T): 8248746: Undo jhsdb related exclusiveAccess.dirs changes that were done for JDK-8220295 In-Reply-To: <641570A2-6CE3-4E3B-AA10-D55386689A58@oracle.com> References: <6d2e0e0e-fda7-b346-1f1c-4e6383f58c1e@oracle.com> <641570A2-6CE3-4E3B-AA10-D55386689A58@oracle.com> Message-ID: <59f57ba9-9304-3253-1ee4-5e758ceffd42@oracle.com> I can do that. Chris On 7/2/20 10:40 PM, Igor Ignatyev wrote: > Hi Chris, > > looks good and trivial. would you consider pushing it into jdk/jdk15? > > -- Igor > >> On Jul 2, 2020, at 10:36 PM, Chris Plummer wrote: >> >> Hello, >> >> Please review the following trivial change: >> >> https://bugs.openjdk.java.net/browse/JDK-8248746 >> >> diff --git a/test/jdk/TEST.ROOT b/test/jdk/TEST.ROOT >> --- a/test/jdk/TEST.ROOT >> +++ b/test/jdk/TEST.ROOT >> @@ -24,7 +24,7 @@ >> # Tests that cannot run concurrently >> exclusiveAccess.dirs=java/math/BigInteger/largeMemory \ >> java/rmi/Naming java/util/prefs sun/management/jmxremote \ >> -sun/tools/jstatd sun/tools/jcmd sun/tools/jhsdb sun/tools/jhsdb/heapconfig \ >> +sun/tools/jstatd sun/tools/jcmd \ >> sun/tools/jinfo sun/tools/jmap sun/tools/jps sun/tools/jstack sun/tools/jstat \ >> com/sun/tools/attach sun/security/mscapi java/util/stream java/util/Arrays/largeMemory \ >> java/util/BitSet/stream javax/rmi >> >> Due to some issues with running multiple Attach API tests at the same time, it was decided to add them all to exclusiveAccess.dirs. This was done under JDK-8220295. However, the jhsdb tests should never have been added since they don't use the Attach API. They use SA Attach, which is completely unrelated. I've been testing with this change for the past 6 weeks or so, and haven't seen any issues, nor would I expect any. >> >> thanks, >> >> Chris >> From suenaga at oss.nttdata.com Fri Jul 3 06:29:43 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Fri, 3 Jul 2020 15:29:43 +0900 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: <04e09eb7-9483-8f63-4bb4-01c9e814802e@oracle.com> References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <2ea2f5d8-5c43-a7d3-d525-cd44c4009d25@oss.nttdata.com> <65889ab4-6b7e-86fb-59cc-012520b83138@oracle.com> <4b7df6f8-c8f7-fedf-6366-b885d2235b58@oss.nttdata.com> <74c05dfd-ffb2-b73d-66e5-dc98752729f5@oss.nttdata.com> <763793b1-9506-44c6-0681-1c71c44a44fd@oss.nttdata.com> <5107c478-1de2-7a49-ee71-dfad6a99a667@oracle.com> <90cdfef9-e558-0d25-33f5-25b7d65bb83b@oss.nttdata.com> <0e359819-618f-c17b-a427-bb7e4daa2c58@oracle.com> <3f890bed-6892-e64d-b21b-ca0cbe11784e@oracle.com> <7dc1903b-6295-3864-6301-4c02aacdb971@oss.nttdata.com> <04e09eb7-9483-8f63-4bb4-01c9e814802e@oracle.com> Message-ID: Hi Dan, David, I uploaded new webrev. Could you review again? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/ OneGetThreadListStackTraces.java in this webrev would wait until thread state is transited to "waiting" with spin wait. CountDownLatch::await call as Dan pointed is fixed in it :) Diff from webrev.03: http://hg.openjdk.java.net/jdk/submit/rev/c9aeb7001e50 Thanks, Yasumasa On 2020/07/03 14:15, David Holmes wrote: > > > On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: >> On 2020/07/03 12:24, Daniel D. Daugherty wrote: >>> On 7/2/20 10:50 PM, David Holmes wrote: >>>> Sorry I'm responding here without seeing latest webrev but there is enough context I think ... >>>> >>>> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>>>> Hi Dan, >>>>> >>>>> Thanks for your comment! >>>>> >>>>> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>>> Hi David, >>>>>>> >>>>>>> I upload new webrev. Could you review again? >>>>>>> >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>>> >>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>> ???? L1542: ??? // Get stack trace with handshake >>>>>> ???????? nit - please add a period at the end. >>>>> >>>>> I will fix it. >>>>> >>>>> >>>>>> ???? L1591: ??? *stack_info_ptr = op.stack_info(); >>>>>> ???????? The return parameter should not be touched unless the return >>>>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>>>> >>>>>> ???? old L1582: ? if (err == JVMTI_ERROR_NONE) { >>>>>> ???????? Please restore this check. The return parameter should not >>>>>> ???????? be touched unless the return code in 'err' == JVMTI_ERROR_NONE. >>>>> >>>>> I will fix it. >>>> >>>> But op.stack_info() will return NULL if the error is not JVMTI_ERROR_NONE. Are you (Dan) concerned about someone passing in a non-null/initialized out-pointer that will be reset to NULL if there was an error? >>> >>> Actually the way we used to test this in POSIX tests is to call >>> an API with known bad parameters and the return parameter ptr >>> set to NULL. If the return parameter ptr was touched when an >>> error should have been detected on an earlier parameter, then >>> the test failed. >>> >>> >>>> >>>>> >>>>>> ???? L1272: ? if (!jt->is_exiting() && (thread_oop != NULL)) { >>>>>> ???????? nit - extra parens around the second expression. >>>>> >>>>> I will fix it. >>>>> >>>>> >>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>> ???? old L1532: ? _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>>> ???????? This deletion of the _result field threw me for a minute and then >>>>>> ???? ? ? I figured out that the field is init to JVMTI_ERROR_THREAD_NOT_ALIVE >>>>>> ???????? in the constructor. >>>>>> >>>>>> ???? L1553: ? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>> ???????? nit - extra parens around the second expression. >>>>> >>>>> I will fix it. >>>>> >>>>> >>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>> ???? No comments. >>>>>> >>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>> ???? No comments. >>>>>> >>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>>> ???? No comments. >>>>>> >>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>> ???? L64: ??????? startSignal.countDown(); >>>>>> ???????? I was expecting this to be a call to await() instead of >>>>>> ???????? countDown(). What am I missing here? >>>>>> >>>>>> ???????? I think this test might be passing by accident right now, but... >>>>> >>>>> Main thread (which call JVMTI functions to test) should wait until test thread is ready. >>>>> So main thread would wait startSignal, and test thread would count down. >>>> >>>> No! >>>> >>>> The test thread that previously called obj.wait() now calls latch.await(). >>>> >>>> The main thread that previously called obj.notify() now calls latch.countDown(). >>>> >>>> The main thread continues to spin until it sees the target is WAITING before proceeding with the test. >> >> If I add spin wait to wait until transit target thread state is WAITING (as following), we don't need to call SuspendThread(). >> Which is better? > > The original spin-wait loop checking for? WAITING is better because it is the only guarantee that the target thread is blocked where you need it to be. suspending the thread is racy as you don't know exactly where the suspend will hit. > > Thanks, > David > ----- > >> >> ``` >> /* Wait until the thread state transits to "waiting" */ >> while (th.getState() != Thread.State.WAITING) { >> ???? Thread.onSpinWait(); >> } >> ``` >> >> For simplify, spin wait is prefer to OneGetThreadListStackTraces.java in webrev.03. >> >> >> Thanks, >> >> Yasumasa >> >> >>> Here's the flow as I see it: >>> >>> main thread >>> ?? - start worker thread >>> ?? - startSignal.await() >>> ???? - main is now blocked >>> >>> worker thread >>> ?? - startSignal.countDown() >>> ???? - main is now unblocked >>> ?? - stopSignal.await() >>> ???? - worker is now blocked >>> >>> main thread >>> ?? - checkCallStacks(th) >>> ?? - stopSignal.countDown() >>> ???? - worker is now unblocked >>> ?? - th.join >>> ???? - main is now blocked >>> >>> worker thread >>> ?? - runs off the end of run() >>> ???? - main is now unblocked >>> >>> main thread >>> ?? - run off the end of main() >>> >>> >>>> >>>>> >>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>>> ???? L92: ? jthreads = (jthread *)malloc(sizeof(jthread) * num_threads); >>>>>> ???????? You don't check for malloc() failure. >>>>>> ???????? 'jthreads' is allocated but never freed. >>>>> >>>>> I will fix it. >>>>> >>>>> >>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>>> ???? L91: ? result = (*jvmti)->SuspendThread(jvmti, thread); >>>>>> ???????? Why are you suspending the thread? GetAllStackTraces() and >>>>>> ???????? GetThreadListStackTraces() do not require the target thread(s) >>>>>> ???????? to be suspend. >>>>>> >>>>>> ???????? If you decide not to SuspendThread, then you don't need the >>>>>> ???????? AddCapabilities or the ResumeThread calls. >>>>> >>>>> Test thread might not be entered following code (stopSignal.await()). We might see deferent call stack between GetAllStackTraces() and GetThreadListStackTraces(). We cannot control to freeze call stack of test thread in Java code. >>>>> (I didn't use SuspendThread() at first, but I saw some errors which causes in above.) >>>>> >>>>> So we need to call SuspendThread() to ensure we can see same call stack. >>>> >>>> If you are checking that the thread is in state WAITING then it cannot escape from that state and you can sample the stack multiple times from any API and get the same result. >>>> >>>> I suspect the errors you saw were from the apparent incorrect use of the CountDownLatch. >>> >>> With the flow outlined above, the worker thread should be >>> nicely blocked in stopSignal.await() when stuff is sampled. >>> >>> Dan >>> >>> >>>> >>>> Cheers, >>>> David >>>> ----- >>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>>> Dan >>>>>> >>>>>>> >>>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>>> Hi Yasumasa, >>>>>>>> >>>>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I uploaded new webrev. Could review again? >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>>> >>>>>>>> Updates look fine - thanks. >>>>>>>> >>>>>>>> One minor nit: >>>>>>>> >>>>>>>> 1274???? _collector.allocate_and_fill_stacks(1); >>>>>>>> 1275???? _collector.set_result(JVMTI_ERROR_NONE); >>>>>>>> >>>>>>>> In the other places where you use _collector you rely on result being initialized to JVMTI_ERROR_NONE, rather than setting it directly after allocate_and_fill_stacks(). >>>>>>> >>>>>>> Fixed. >>>>>>> >>>>>>> >>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>> >>>>>>>>>>>>>> ? 820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>> ? 821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>> ? 822????????? current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>> ? 823????????? "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>> >>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>> >>>>>>>>> This function (JvmtiEnvBase::get_stack_trace()) can be called to get own stack trace. For example, we can call GetStackTrace() for current thread at JVMTI event. >>>>>>>>> So I changed assert as below: >>>>>>>>> >>>>>>>>> ``` >>>>>>>>> ??820?? assert(current_thread == java_thread || >>>>>>>>> ??821????????? SafepointSynchronize::is_at_safepoint() || >>>>>>>>> ??822????????? current_thread == java_thread->active_handshaker(), >>>>>>>>> ??823????????? "call by myself / at safepoint / at handshake"); >>>>>>>>> ``` >>>>>>>> >>>>>>>> Yep good catch. I hope current tests caught that. >>>>>>> >>>>>>> They would be tested in vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ (own call stacks), and getstacktr003 (call stacks in other thread). >>>>>>> >>>>>>> >>>>>>>> Speaking of tests ... >>>>>>>> >>>>>>>> In the native code I think you need to check the success of all JNI methods that can throw exceptions - otherwise I believe the tests may trigger warnings if -Xcheck:jni is used with them. See for example: >>>>>>>> >>>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>> >>>>>>> I updated testcases to check JNI and JVMTI function calls. >>>>>>> >>>>>>> >>>>>>>> In the Java code the target thread: >>>>>>>> >>>>>>>> ?? 45???? public void run() { >>>>>>>> ?? 46?????? try { >>>>>>>> ?? 47?????????? synchronized (lock) { >>>>>>>> ?? 48?????????????? lock.wait(); >>>>>>>> ?? 49?????????????? System.out.println("OK"); >>>>>>>> ?? 50?????????? } >>>>>>>> >>>>>>>> is potentially susceptible to spurious wakeups. Using a CountDownLatch would be robust. >>>>>>> >>>>>>> Fixed. >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> ----- >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>>> Hi Yasumasa, >>>>>>>>>> >>>>>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi David, >>>>>>>>>>> >>>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>>> >>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>> >>>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>> >>>>>>>>>>> If so, we can call make_local() in L1272 without JavaThread (or we can pass current thread to make_local()). Is it right? >>>>>>>>>>> >>>>>>>>>>> ``` >>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>> 1273??????????????????????????? jt, thread_oop); >>>>>>>>>>> ``` >>>>>>>>>> >>>>>>>>>> Sorry I got confused, _calling_thread may not be the current thread as we could be executing the handshake in the target thread itself. So the ResourceMark is correct as-is (implicitly for current thread). >>>>>>>>>> >>>>>>>>>> The argument to fill_frames will be used in the jvmtiStackInfo and passed back to the _calling_thread, so it must be created via make_local(_calling_thread, ...) as you presently have. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> David >>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>> Hi David, >>>>>>>>>>>>> >>>>>>>>>>>>> Thank you for reviewing! I will update new webrev tomorrow. >>>>>>>>>>>>> >>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>> >>>>>>>>>>>>>> ? 498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>>> ? 500?? JavaThread *_calling_thread; >>>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>>> ? 502?? MultipleStackTracesCollector _collector; >>>>>>>>>>>>>> >>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>> >>>>>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>>> >>>>>>>>>>>>> ``` >>>>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>>>> ??? : >>>>>>>>>>>>> } >>>>>>>>>>>>> >>>>>>>>>>>>> class GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>> ?? private: >>>>>>>>>>>>> ???? MultipleStackTracesCollector _collector; >>>>>>>>>>>>> } >>>>>>>>>>>>> ``` >>>>>>>>>>>> >>>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> David >>>>>>>>>>>> ----- >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I updated webrev for 8242428. Could you review again? >>>>>>>>>>>>>>> This change migrate to use direct handshake for GetStackTrace() and GetThreadListStackTraces() (when thread_count == 1). >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> This looks really good now! I only have a few nits below. There is one thing I don't like about it but it requires a change to the main Handshake logic to address - in JvmtiEnv::GetThreadListStackTraces you have to create a ThreadsListHandle to convert the jthread to a JavaThread, but then the Handshake::execute_direct creates another ThreadsListHandle internally. That's a waste. I will discuss with Robbin and file a RFE to have an overload of execute_direct that takes an existing TLH. Actually it's worse than that because we have another TLH in use at the entry point for the JVMTI functions, so I think there may be some scope for simplifying the use of TLH instances - future RFE. >>>>>>>>>>>>>> >>>>>>>>>>>>>> --- >>>>>>>>>>>>>> >>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>> >>>>>>>>>>>>>> ??451?? GetStackTraceClosure(JvmtiEnv *env, jint start_depth, jint max_count, >>>>>>>>>>>>>> ??452??????????????????????? jvmtiFrameInfo* frame_buffer, jint* count_ptr) >>>>>>>>>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>>> ??454?????? _env(env), _start_depth(start_depth), _max_count(max_count), >>>>>>>>>>>>>> ??455?????? _frame_buffer(frame_buffer), _count_ptr(count_ptr), >>>>>>>>>>>>>> ??456 _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>>>>>> >>>>>>>>>>>>>> Nit: can you do one initializer per line please. >>>>>>>>>>>>>> >>>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>>> >>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>> >>>>>>>>>>>>>> ??498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>>> ??500?? JavaThread *_calling_thread; >>>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>>> ??502?? MultipleStackTracesCollector _collector; >>>>>>>>>>>>>> >>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>> >>>>>>>>>>>>>> 481?? MultipleStackTracesCollector(JvmtiEnv *env, jint max_frame_count) { >>>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>>> ??483???? _max_frame_count = max_frame_count; >>>>>>>>>>>>>> ??484???? _frame_count_total = 0; >>>>>>>>>>>>>> ??485???? _head = NULL; >>>>>>>>>>>>>> ??486???? _stack_info = NULL; >>>>>>>>>>>>>> ??487???? _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>>> >>>>>>>>>>>>>> As you are touching this can you change it to use an initializer list as you did for the HandshakeClosure, and please keep one item per line. >>>>>>>>>>>>>> >>>>>>>>>>>>>> --- >>>>>>>>>>>>>> >>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>> >>>>>>>>>>>>>> ??820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>> ??821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>> ??822????????? current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>> ??823????????? "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>> >>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>>>> >>>>>>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>>>>>> 1269 >>>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>>>> >>>>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>>>> >>>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>>> 1273??????????????????????????? jt, thread_oop); >>>>>>>>>>>>>> >>>>>>>>>>>>>> It is frustrating that this entire call chain started with a jthread reference, which we converted to a JavaThread, only to eventually need to convert it back to a jthread! I think there is some scope for simplification here but not as part of this change. >>>>>>>>>>>>>> >>>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>>> >>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>> >>>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>>>> >>>>>>>>>>>>>> --- >>>>>>>>>>>>>> >>>>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I'm still pondering the test logic but wanted to send this now. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> David >>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>> VM_GetThreadListStackTrace (for GetThreadListStackTraces) and VM_GetAllStackTraces (for GetAllStackTraces) have inherited VM_GetMultipleStackTraces VM operation which provides the feature to generate jvmtiStackInfo. I modified VM_GetMultipleStackTraces to a normal C++ class to share with HandshakeClosure for GetThreadListStackTraces (GetSingleStackTraceClosure). >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Also I added new testcases which test GetThreadListStackTraces() with thread_count == 1 and with all threads. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/nsk/jdwp. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> This change replace following VM operations to direct handshake. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces (GetThreadListStackTrace()) >>>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> GetThreadListStackTrace() uses direct handshake if thread count == 1. In other case (thread count > 1), it would be performed as VM operation (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>>> Caller of VM_GetCurrentLocation (JvmtiEnvThreadState::reset_current_location()) might be called at safepoint. So I added safepoint check in its caller. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/ns >>>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Also I tested it on submit repo, then it has execution error (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) due to dependency error. So I think it does not occur by this change. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Yasumasa >>>>>> >>> From serguei.spitsyn at oracle.com Fri Jul 3 07:45:40 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 3 Jul 2020 00:45:40 -0700 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <2ea2f5d8-5c43-a7d3-d525-cd44c4009d25@oss.nttdata.com> <65889ab4-6b7e-86fb-59cc-012520b83138@oracle.com> <4b7df6f8-c8f7-fedf-6366-b885d2235b58@oss.nttdata.com> <74c05dfd-ffb2-b73d-66e5-dc98752729f5@oss.nttdata.com> <763793b1-9506-44c6-0681-1c71c44a44fd@oss.nttdata.com> <5107c478-1de2-7a49-ee71-dfad6a99a667@oracle.com> <90cdfef9-e558-0d25-33f5-25b7d65bb83b@oss.nttdata.com> <0e359819-618f-c17b-a427-bb7e4daa2c58@oracle.com> <3f890bed-6892-e64d-b21b-ca0cbe11784e@oracle.com> <7dc1903b-6295-3864-6301-4c02aacdb971@oss.nttdata.com> <04e09eb7-9483-8f63-4bb4-01c9e814802e@oracle.com> Message-ID: <238a89a3-06b1-90e4-d5a4-8dc2f1ed12ab@oracle.com> An HTML attachment was scrubbed... URL: From suenaga at oss.nttdata.com Fri Jul 3 08:16:22 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Fri, 3 Jul 2020 17:16:22 +0900 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: <238a89a3-06b1-90e4-d5a4-8dc2f1ed12ab@oracle.com> References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <2ea2f5d8-5c43-a7d3-d525-cd44c4009d25@oss.nttdata.com> <65889ab4-6b7e-86fb-59cc-012520b83138@oracle.com> <4b7df6f8-c8f7-fedf-6366-b885d2235b58@oss.nttdata.com> <74c05dfd-ffb2-b73d-66e5-dc98752729f5@oss.nttdata.com> <763793b1-9506-44c6-0681-1c71c44a44fd@oss.nttdata.com> <5107c478-1de2-7a49-ee71-dfad6a99a667@oracle.com> <90cdfef9-e558-0d25-33f5-25b7d65bb83b@oss.nttdata.com> <0e359819-618f-c17b-a427-bb7e4daa2c58@oracle.com> <3f890bed-6892-e64d-b21b-ca0cbe11784e@oracle.com> <7dc1903b-6295-3864-6301-4c02aacdb971@oss.nttdata.com> <04e09eb7-9483-8f63-4bb4-01c9e814802e@oracle.com> <238a89a3-06b1-90e4-d5a4-8dc2f1ed12ab@oracle.com> Message-ID: Hi Serguei, Generally I agree with you, but I have concern about the difference of the result of GetStackTrace() and GetThreadListStackTraces(). GetStackTrace: jvmtiFrameInfo GetThreadListStackTraces: jvmtiStackInfo jvmtiStackInfo contains thread state, and it is ensured it is the state of the call stack. If we want to get both call stack and thread state, we need to suspend target thread, and call both GetStackTrace() and GetThreadState(). Is it ok? I was wondering if JDK-8201641 (parent ticket of this change) needed them for profiling (dynatrace?) If it is responsibility of JVMTI agent implementor, I remove this closure. Thanks, Yasumasa On 2020/07/03 16:45, serguei.spitsyn at oracle.com wrote: > Hi Yasumasa, > > After some thinking I've concluded that I do not like this optimization > of the GetThreadListStackTraces with GetSingleStackTraceClosure. > > We may need more opinions on this but these are my points: > ?- it adds some complexity and ugliness > ?- a win is doubtful because it has to be a rare case, so that total overhead should not be high > ?- if it is really high for some use cases then it is up to the user > ?? to optimize it with using GetStackTrace instead > > In such cases with doubtful overhead I usually prefer the simplicity. > > Good examples where it makes sense to optimize are checks for target thread to be current thread. > In such cases there is no need to suspend the target thread, or use a VMop/HandshakeClosure. > For instance, please, see the Monitor functions with the check: (java_thread == calling_thread). > Getting information for current thread is frequently used case, e.g. to get info at an event point. > > Thanks, > Serguei > > > On 7/2/20 23:29, Yasumasa Suenaga wrote: >> Hi Dan, David, >> >> I uploaded new webrev. Could you review again? >> >> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/ >> >> OneGetThreadListStackTraces.java in this webrev would wait until thread state is transited to "waiting" with spin wait. >> CountDownLatch::await call as Dan pointed is fixed in it :) >> >> Diff from webrev.03: >> http://hg.openjdk.java.net/jdk/submit/rev/c9aeb7001e50 >> >> >> Thanks, >> >> Yasumasa >> >> >> On 2020/07/03 14:15, David Holmes wrote: >>> >>> >>> On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: >>>> On 2020/07/03 12:24, Daniel D. Daugherty wrote: >>>>> On 7/2/20 10:50 PM, David Holmes wrote: >>>>>> Sorry I'm responding here without seeing latest webrev but there is enough context I think ... >>>>>> >>>>>> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>>>>>> Hi Dan, >>>>>>> >>>>>>> Thanks for your comment! >>>>>>> >>>>>>> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>>>>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>>>>> Hi David, >>>>>>>>> >>>>>>>>> I upload new webrev. Could you review again? >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>>>>> >>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>> ???? L1542: ??? // Get stack trace with handshake >>>>>>>> ???????? nit - please add a period at the end. >>>>>>> >>>>>>> I will fix it. >>>>>>> >>>>>>> >>>>>>>> ???? L1591: ??? *stack_info_ptr = op.stack_info(); >>>>>>>> ???????? The return parameter should not be touched unless the return >>>>>>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>>>>>> >>>>>>>> ???? old L1582: ? if (err == JVMTI_ERROR_NONE) { >>>>>>>> ???????? Please restore this check. The return parameter should not >>>>>>>> ???????? be touched unless the return code in 'err' == JVMTI_ERROR_NONE. >>>>>>> >>>>>>> I will fix it. >>>>>> >>>>>> But op.stack_info() will return NULL if the error is not JVMTI_ERROR_NONE. Are you (Dan) concerned about someone passing in a non-null/initialized out-pointer that will be reset to NULL if there was an error? >>>>> >>>>> Actually the way we used to test this in POSIX tests is to call >>>>> an API with known bad parameters and the return parameter ptr >>>>> set to NULL. If the return parameter ptr was touched when an >>>>> error should have been detected on an earlier parameter, then >>>>> the test failed. >>>>> >>>>> >>>>>> >>>>>>> >>>>>>>> ???? L1272: ? if (!jt->is_exiting() && (thread_oop != NULL)) { >>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>> >>>>>>> I will fix it. >>>>>>> >>>>>>> >>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>> ???? old L1532: ? _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>>>>> ???????? This deletion of the _result field threw me for a minute and then >>>>>>>> ???? ? ? I figured out that the field is init to JVMTI_ERROR_THREAD_NOT_ALIVE >>>>>>>> ???????? in the constructor. >>>>>>>> >>>>>>>> ???? L1553: ? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>> >>>>>>> I will fix it. >>>>>>> >>>>>>> >>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>> ???? No comments. >>>>>>>> >>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>> ???? No comments. >>>>>>>> >>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>>>>> ???? No comments. >>>>>>>> >>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>> ???? L64: ??????? startSignal.countDown(); >>>>>>>> ???????? I was expecting this to be a call to await() instead of >>>>>>>> ???????? countDown(). What am I missing here? >>>>>>>> >>>>>>>> ???????? I think this test might be passing by accident right now, but... >>>>>>> >>>>>>> Main thread (which call JVMTI functions to test) should wait until test thread is ready. >>>>>>> So main thread would wait startSignal, and test thread would count down. >>>>>> >>>>>> No! >>>>>> >>>>>> The test thread that previously called obj.wait() now calls latch.await(). >>>>>> >>>>>> The main thread that previously called obj.notify() now calls latch.countDown(). >>>>>> >>>>>> The main thread continues to spin until it sees the target is WAITING before proceeding with the test. >>>> >>>> If I add spin wait to wait until transit target thread state is WAITING (as following), we don't need to call SuspendThread(). >>>> Which is better? >>> >>> The original spin-wait loop checking for? WAITING is better because it is the only guarantee that the target thread is blocked where you need it to be. suspending the thread is racy as you don't know exactly where the suspend will hit. >>> >>> Thanks, >>> David >>> ----- >>> >>>> >>>> ``` >>>> /* Wait until the thread state transits to "waiting" */ >>>> while (th.getState() != Thread.State.WAITING) { >>>> ???? Thread.onSpinWait(); >>>> } >>>> ``` >>>> >>>> For simplify, spin wait is prefer to OneGetThreadListStackTraces.java in webrev.03. >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>>> Here's the flow as I see it: >>>>> >>>>> main thread >>>>> ?? - start worker thread >>>>> ?? - startSignal.await() >>>>> ???? - main is now blocked >>>>> >>>>> worker thread >>>>> ?? - startSignal.countDown() >>>>> ???? - main is now unblocked >>>>> ?? - stopSignal.await() >>>>> ???? - worker is now blocked >>>>> >>>>> main thread >>>>> ?? - checkCallStacks(th) >>>>> ?? - stopSignal.countDown() >>>>> ???? - worker is now unblocked >>>>> ?? - th.join >>>>> ???? - main is now blocked >>>>> >>>>> worker thread >>>>> ?? - runs off the end of run() >>>>> ???? - main is now unblocked >>>>> >>>>> main thread >>>>> ?? - run off the end of main() >>>>> >>>>> >>>>>> >>>>>>> >>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>>>>> ???? L92: ? jthreads = (jthread *)malloc(sizeof(jthread) * num_threads); >>>>>>>> ???????? You don't check for malloc() failure. >>>>>>>> ???????? 'jthreads' is allocated but never freed. >>>>>>> >>>>>>> I will fix it. >>>>>>> >>>>>>> >>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>>>>> ???? L91: ? result = (*jvmti)->SuspendThread(jvmti, thread); >>>>>>>> ???????? Why are you suspending the thread? GetAllStackTraces() and >>>>>>>> ???????? GetThreadListStackTraces() do not require the target thread(s) >>>>>>>> ???????? to be suspend. >>>>>>>> >>>>>>>> ???????? If you decide not to SuspendThread, then you don't need the >>>>>>>> ???????? AddCapabilities or the ResumeThread calls. >>>>>>> >>>>>>> Test thread might not be entered following code (stopSignal.await()). We might see deferent call stack between GetAllStackTraces() and GetThreadListStackTraces(). We cannot control to freeze call stack of test thread in Java code. >>>>>>> (I didn't use SuspendThread() at first, but I saw some errors which causes in above.) >>>>>>> >>>>>>> So we need to call SuspendThread() to ensure we can see same call stack. >>>>>> >>>>>> If you are checking that the thread is in state WAITING then it cannot escape from that state and you can sample the stack multiple times from any API and get the same result. >>>>>> >>>>>> I suspect the errors you saw were from the apparent incorrect use of the CountDownLatch. >>>>> >>>>> With the flow outlined above, the worker thread should be >>>>> nicely blocked in stopSignal.await() when stuff is sampled. >>>>> >>>>> Dan >>>>> >>>>> >>>>>> >>>>>> Cheers, >>>>>> David >>>>>> ----- >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>>> Dan >>>>>>>> >>>>>>>>> >>>>>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>>>>> Hi Yasumasa, >>>>>>>>>> >>>>>>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> I uploaded new webrev. Could review again? >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>>>>> >>>>>>>>>> Updates look fine - thanks. >>>>>>>>>> >>>>>>>>>> One minor nit: >>>>>>>>>> >>>>>>>>>> 1274???? _collector.allocate_and_fill_stacks(1); >>>>>>>>>> 1275???? _collector.set_result(JVMTI_ERROR_NONE); >>>>>>>>>> >>>>>>>>>> In the other places where you use _collector you rely on result being initialized to JVMTI_ERROR_NONE, rather than setting it directly after allocate_and_fill_stacks(). >>>>>>>>> >>>>>>>>> Fixed. >>>>>>>>> >>>>>>>>> >>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> ? 820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>> ? 821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>>>> ? 822????????? current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>> ? 823????????? "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>> >>>>>>>>>>> This function (JvmtiEnvBase::get_stack_trace()) can be called to get own stack trace. For example, we can call GetStackTrace() for current thread at JVMTI event. >>>>>>>>>>> So I changed assert as below: >>>>>>>>>>> >>>>>>>>>>> ``` >>>>>>>>>>> ??820?? assert(current_thread == java_thread || >>>>>>>>>>> ??821 SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>> ??822????????? current_thread == java_thread->active_handshaker(), >>>>>>>>>>> ??823????????? "call by myself / at safepoint / at handshake"); >>>>>>>>>>> ``` >>>>>>>>>> >>>>>>>>>> Yep good catch. I hope current tests caught that. >>>>>>>>> >>>>>>>>> They would be tested in vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ (own call stacks), and getstacktr003 (call stacks in other thread). >>>>>>>>> >>>>>>>>> >>>>>>>>>> Speaking of tests ... >>>>>>>>>> >>>>>>>>>> In the native code I think you need to check the success of all JNI methods that can throw exceptions - otherwise I believe the tests may trigger warnings if -Xcheck:jni is used with them. See for example: >>>>>>>>>> >>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>>>> >>>>>>>>> I updated testcases to check JNI and JVMTI function calls. >>>>>>>>> >>>>>>>>> >>>>>>>>>> In the Java code the target thread: >>>>>>>>>> >>>>>>>>>> ?? 45???? public void run() { >>>>>>>>>> ?? 46?????? try { >>>>>>>>>> ?? 47?????????? synchronized (lock) { >>>>>>>>>> ?? 48?????????????? lock.wait(); >>>>>>>>>> ?? 49?????????????? System.out.println("OK"); >>>>>>>>>> ?? 50?????????? } >>>>>>>>>> >>>>>>>>>> is potentially susceptible to spurious wakeups. Using a CountDownLatch would be robust. >>>>>>>>> >>>>>>>>> Fixed. >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> David >>>>>>>>>> ----- >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>> >>>>>>>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>> Hi David, >>>>>>>>>>>>> >>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>>> >>>>>>>>>>>>> If so, we can call make_local() in L1272 without JavaThread (or we can pass current thread to make_local()). Is it right? >>>>>>>>>>>>> >>>>>>>>>>>>> ``` >>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>> 1273??????????????????????????? jt, thread_oop); >>>>>>>>>>>>> ``` >>>>>>>>>>>> >>>>>>>>>>>> Sorry I got confused, _calling_thread may not be the current thread as we could be executing the handshake in the target thread itself. So the ResourceMark is correct as-is (implicitly for current thread). >>>>>>>>>>>> >>>>>>>>>>>> The argument to fill_frames will be used in the jvmtiStackInfo and passed back to the _calling_thread, so it must be created via make_local(_calling_thread, ...) as you presently have. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> David >>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thank you for reviewing! I will update new webrev tomorrow. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> ? 498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>>>>> ? 500?? JavaThread *_calling_thread; >>>>>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>>>>> ? 502?? MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>>>>>> ??? : >>>>>>>>>>>>>>> } >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> class GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>> ?? private: >>>>>>>>>>>>>>> ???? MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>> } >>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> David >>>>>>>>>>>>>> ----- >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I updated webrev for 8242428. Could you review again? >>>>>>>>>>>>>>>>> This change migrate to use direct handshake for GetStackTrace() and GetThreadListStackTraces() (when thread_count == 1). >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> This looks really good now! I only have a few nits below. There is one thing I don't like about it but it requires a change to the main Handshake logic to address - in JvmtiEnv::GetThreadListStackTraces you have to create a ThreadsListHandle to convert the jthread to a JavaThread, but then the Handshake::execute_direct creates another ThreadsListHandle internally. That's a waste. I will discuss with Robbin and file a RFE to have an overload of execute_direct that takes an existing TLH. Actually it's worse than that because we have another TLH in use at the entry point for the JVMTI functions, so I think there may be some scope for simplifying the use of TLH instances - future RFE. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> ??451?? GetStackTraceClosure(JvmtiEnv *env, jint start_depth, jint max_count, >>>>>>>>>>>>>>>> ??452 jvmtiFrameInfo* frame_buffer, jint* count_ptr) >>>>>>>>>>>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>>>>> ??454?????? _env(env), _start_depth(start_depth), _max_count(max_count), >>>>>>>>>>>>>>>> ??455 _frame_buffer(frame_buffer), _count_ptr(count_ptr), >>>>>>>>>>>>>>>> ??456 _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Nit: can you do one initializer per line please. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> ??498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>>>>> ??500?? JavaThread *_calling_thread; >>>>>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>>>>> ??502?? MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 481 MultipleStackTracesCollector(JvmtiEnv *env, jint max_frame_count) { >>>>>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>>>>> ??483???? _max_frame_count = max_frame_count; >>>>>>>>>>>>>>>> ??484???? _frame_count_total = 0; >>>>>>>>>>>>>>>> ??485???? _head = NULL; >>>>>>>>>>>>>>>> ??486???? _stack_info = NULL; >>>>>>>>>>>>>>>> ??487???? _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> As you are touching this can you change it to use an initializer list as you did for the HandshakeClosure, and please keep one item per line. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> ??820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>> ??821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>>>> ??822????????? current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>> ??823????????? "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>>>>>>>> 1269 >>>>>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>>>>> 1273??????????????????????????? jt, thread_oop); >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> It is frustrating that this entire call chain started with a jthread reference, which we converted to a JavaThread, only to eventually need to convert it back to a jthread! I think there is some scope for simplification here but not as part of this change. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I'm still pondering the test logic but wanted to send this now. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>> VM_GetThreadListStackTrace (for GetThreadListStackTraces) and VM_GetAllStackTraces (for GetAllStackTraces) have inherited VM_GetMultipleStackTraces VM operation which provides the feature to generate jvmtiStackInfo. I modified VM_GetMultipleStackTraces to a normal C++ class to share with HandshakeClosure for GetThreadListStackTraces (GetSingleStackTraceClosure). >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Also I added new testcases which test GetThreadListStackTraces() with thread_count == 1 and with all threads. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/nsk/jdwp. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>>>>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> This change replace following VM operations to direct handshake. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces (GetThreadListStackTrace()) >>>>>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> GetThreadListStackTrace() uses direct handshake if thread count == 1. In other case (thread count > 1), it would be performed as VM operation (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>>>>> Caller of VM_GetCurrentLocation (JvmtiEnvThreadState::reset_current_location()) might be called at safepoint. So I added safepoint check in its caller. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/ns >>>>>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Also I tested it on submit repo, then it has execution error (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) due to dependency error. So I think it does not occur by this change. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>> >>>>> > From vladimir.kozlov at oracle.com Fri Jul 3 18:09:26 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 3 Jul 2020 11:09:26 -0700 Subject: [15] RFR(T) 8247527: serviceability/dcmd/gc/HeapDumpCompressedTest.java fails with Graal + ZGC In-Reply-To: <5E33E613-882E-400A-886A-EA4FAD85F2EA@oracle.com> References: <5E33E613-882E-400A-886A-EA4FAD85F2EA@oracle.com> Message-ID: <2bde8004-4ed8-8ca3-b387-05240f423e3f@oracle.com> Thank you, Igor Vladimir K On 7/2/20 7:24 PM, igor.ignatyev at oracle.com wrote: > LGTM > > ? Igor > >> On Jul 2, 2020, at 7:03 PM, Vladimir Kozlov wrote: >> >> ?https://cr.openjdk.java.net/~kvn/8247527/webrev.00/ >> https://bugs.openjdk.java.net/browse/JDK-8247527 >> >> Test should have @requires which excludes running Graal with GC which it does not support. >> >> Testing: hs-tier1,hs-tier4-graal >> >> Thanks, >> Vladimir > From vladimir.kozlov at oracle.com Fri Jul 3 18:30:31 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 3 Jul 2020 11:30:31 -0700 Subject: [15] RFR(T) 8247527: serviceability/dcmd/gc/HeapDumpCompressedTest.java fails with Graal + ZGC In-Reply-To: References: Message-ID: Thank you, David, for looking on changes. I will remember to update tests. I filed RFE 8248815 [1] for tracking. Can you approve this fix now? Thanks, Vladimir K [1] https://bugs.openjdk.java.net/browse/JDK-8248815 On 7/2/20 10:09 PM, David Holmes wrote: > Hi Igor, > > On 3/07/2020 12:59 pm, Igor Ignatyev wrote: >> Hi David, >> >> it's in my todo list to improve this situation and have vm.gc.X to take selected JIT into account; and update existing >> (>200) occurrences of 'vm.gc.X & !vm.graal.enabled' > > 200+ ouch! :( > > I guess this fix doesn't make the situation any worse in a practical sense. > > Thanks, > David > ----- > >> -- Igor >> >>> On Jul 2, 2020, at 7:25 PM, David Holmes wrote: >>> >>> Hi Vladimir, >>> >>> On 3/07/2020 12:02 pm, Vladimir Kozlov wrote: >>>> https://cr.openjdk.java.net/~kvn/8247527/webrev.00/ >>>> https://bugs.openjdk.java.net/browse/JDK-8247527 >>>> Test should have @requires which excludes running Graal with GC which it does not support. >>> >>> I find it somewhat disturbing that a generic test has to know about the limitations between GCs and Graal! >>> >>> I would have been more inclined to just exclude this test when running with Graal, even if that theoretically reduced >>> the test coverage in a ting way. >>> >>> If/When Graal supports these other GCs who will remember to re-enable these test cases? >>> >>> Thanks, >>> David >>> >>>> Testing: hs-tier1,hs-tier4-graal >>>> Thanks, >>>> Vladimir >> From serguei.spitsyn at oracle.com Fri Jul 3 19:32:29 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 3 Jul 2020 12:32:29 -0700 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <65889ab4-6b7e-86fb-59cc-012520b83138@oracle.com> <4b7df6f8-c8f7-fedf-6366-b885d2235b58@oss.nttdata.com> <74c05dfd-ffb2-b73d-66e5-dc98752729f5@oss.nttdata.com> <763793b1-9506-44c6-0681-1c71c44a44fd@oss.nttdata.com> <5107c478-1de2-7a49-ee71-dfad6a99a667@oracle.com> <90cdfef9-e558-0d25-33f5-25b7d65bb83b@oss.nttdata.com> <0e359819-618f-c17b-a427-bb7e4daa2c58@oracle.com> <3f890bed-6892-e64d-b21b-ca0cbe11784e@oracle.com> <7dc1903b-6295-3864-6301-4c02aacdb971@oss.nttdata.com> <04e09eb7-9483-8f63-4bb4-01c9e814802e@oracle.com> <238a89a3-06b1-90e4-d5a4-8dc2f1ed12ab@oracle.com> Message-ID: Hi Yasumasa, This difference is not that big to care about. I feel this is really rare case and so, does not worth these complications. Do we have a real request from customers to optimize it? Thanks, Serguei On 7/3/20 01:16, Yasumasa Suenaga wrote: > Hi Serguei, > > Generally I agree with you, but I have concern about the difference of > the result of GetStackTrace() and GetThreadListStackTraces(). > > ? GetStackTrace: jvmtiFrameInfo > ? GetThreadListStackTraces: jvmtiStackInfo > > jvmtiStackInfo contains thread state, and it is ensured it is the > state of the call stack. > If we want to get both call stack and thread state, we need to suspend > target thread, and call both GetStackTrace() and GetThreadState(). Is > it ok? > > I was wondering if JDK-8201641 (parent ticket of this change) needed > them for profiling (dynatrace?) > If it is responsibility of JVMTI agent implementor, I remove this > closure. > > > Thanks, > > Yasumasa > > > On 2020/07/03 16:45, serguei.spitsyn at oracle.com wrote: >> Hi Yasumasa, >> >> After some thinking I've concluded that I do not like this optimization >> of the GetThreadListStackTraces with GetSingleStackTraceClosure. >> >> We may need more opinions on this but these are my points: >> ??- it adds some complexity and ugliness >> ??- a win is doubtful because it has to be a rare case, so that total >> overhead should not be high >> ??- if it is really high for some use cases then it is up to the user >> ??? to optimize it with using GetStackTrace instead >> >> In such cases with doubtful overhead I usually prefer the simplicity. >> >> Good examples where it makes sense to optimize are checks for target >> thread to be current thread. >> In such cases there is no need to suspend the target thread, or use a >> VMop/HandshakeClosure. >> For instance, please, see the Monitor functions with the check: >> (java_thread == calling_thread). >> Getting information for current thread is frequently used case, e.g. >> to get info at an event point. >> >> Thanks, >> Serguei >> >> >> On 7/2/20 23:29, Yasumasa Suenaga wrote: >>> Hi Dan, David, >>> >>> I uploaded new webrev. Could you review again? >>> >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/ >>> >>> OneGetThreadListStackTraces.java in this webrev would wait until >>> thread state is transited to "waiting" with spin wait. >>> CountDownLatch::await call as Dan pointed is fixed in it :) >>> >>> Diff from webrev.03: >>> http://hg.openjdk.java.net/jdk/submit/rev/c9aeb7001e50 >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2020/07/03 14:15, David Holmes wrote: >>>> >>>> >>>> On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: >>>>> On 2020/07/03 12:24, Daniel D. Daugherty wrote: >>>>>> On 7/2/20 10:50 PM, David Holmes wrote: >>>>>>> Sorry I'm responding here without seeing latest webrev but there >>>>>>> is enough context I think ... >>>>>>> >>>>>>> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>>>>>>> Hi Dan, >>>>>>>> >>>>>>>> Thanks for your comment! >>>>>>>> >>>>>>>> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>>>>>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>>>>>> Hi David, >>>>>>>>>> >>>>>>>>>> I upload new webrev. Could you review again? >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>>>>>> >>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>> ???? L1542: ??? // Get stack trace with handshake >>>>>>>>> ???????? nit - please add a period at the end. >>>>>>>> >>>>>>>> I will fix it. >>>>>>>> >>>>>>>> >>>>>>>>> ???? L1591: *stack_info_ptr = op.stack_info(); >>>>>>>>> ???????? The return parameter should not be touched unless the >>>>>>>>> return >>>>>>>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>> >>>>>>>>> ???? old L1582: ? if (err == JVMTI_ERROR_NONE) { >>>>>>>>> ???????? Please restore this check. The return parameter >>>>>>>>> should not >>>>>>>>> ???????? be touched unless the return code in 'err' == >>>>>>>>> JVMTI_ERROR_NONE. >>>>>>>> >>>>>>>> I will fix it. >>>>>>> >>>>>>> But op.stack_info() will return NULL if the error is not >>>>>>> JVMTI_ERROR_NONE. Are you (Dan) concerned about someone passing >>>>>>> in a non-null/initialized out-pointer that will be reset to NULL >>>>>>> if there was an error? >>>>>> >>>>>> Actually the way we used to test this in POSIX tests is to call >>>>>> an API with known bad parameters and the return parameter ptr >>>>>> set to NULL. If the return parameter ptr was touched when an >>>>>> error should have been detected on an earlier parameter, then >>>>>> the test failed. >>>>>> >>>>>> >>>>>>> >>>>>>>> >>>>>>>>> ???? L1272: ? if (!jt->is_exiting() && (thread_oop != NULL)) { >>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>> >>>>>>>> I will fix it. >>>>>>>> >>>>>>>> >>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>> ???? old L1532: ? _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>>>>>> ???????? This deletion of the _result field threw me for a >>>>>>>>> minute and then >>>>>>>>> ???? ? ? I figured out that the field is init to >>>>>>>>> JVMTI_ERROR_THREAD_NOT_ALIVE >>>>>>>>> ???????? in the constructor. >>>>>>>>> >>>>>>>>> ???? L1553: ? if (!jt->is_exiting() && (jt->threadObj() != >>>>>>>>> NULL)) { >>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>> >>>>>>>> I will fix it. >>>>>>>> >>>>>>>> >>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>> ???? No comments. >>>>>>>>> >>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>> ???? No comments. >>>>>>>>> >>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>>>>>> >>>>>>>>> ???? No comments. >>>>>>>>> >>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>> >>>>>>>>> ???? L64: ??????? startSignal.countDown(); >>>>>>>>> ???????? I was expecting this to be a call to await() instead of >>>>>>>>> ???????? countDown(). What am I missing here? >>>>>>>>> >>>>>>>>> ???????? I think this test might be passing by accident right >>>>>>>>> now, but... >>>>>>>> >>>>>>>> Main thread (which call JVMTI functions to test) should wait >>>>>>>> until test thread is ready. >>>>>>>> So main thread would wait startSignal, and test thread would >>>>>>>> count down. >>>>>>> >>>>>>> No! >>>>>>> >>>>>>> The test thread that previously called obj.wait() now calls >>>>>>> latch.await(). >>>>>>> >>>>>>> The main thread that previously called obj.notify() now calls >>>>>>> latch.countDown(). >>>>>>> >>>>>>> The main thread continues to spin until it sees the target is >>>>>>> WAITING before proceeding with the test. >>>>> >>>>> If I add spin wait to wait until transit target thread state is >>>>> WAITING (as following), we don't need to call SuspendThread(). >>>>> Which is better? >>>> >>>> The original spin-wait loop checking for? WAITING is better because >>>> it is the only guarantee that the target thread is blocked where >>>> you need it to be. suspending the thread is racy as you don't know >>>> exactly where the suspend will hit. >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>>> >>>>> ``` >>>>> /* Wait until the thread state transits to "waiting" */ >>>>> while (th.getState() != Thread.State.WAITING) { >>>>> ???? Thread.onSpinWait(); >>>>> } >>>>> ``` >>>>> >>>>> For simplify, spin wait is prefer to >>>>> OneGetThreadListStackTraces.java in webrev.03. >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>>> Here's the flow as I see it: >>>>>> >>>>>> main thread >>>>>> ?? - start worker thread >>>>>> ?? - startSignal.await() >>>>>> ???? - main is now blocked >>>>>> >>>>>> worker thread >>>>>> ?? - startSignal.countDown() >>>>>> ???? - main is now unblocked >>>>>> ?? - stopSignal.await() >>>>>> ???? - worker is now blocked >>>>>> >>>>>> main thread >>>>>> ?? - checkCallStacks(th) >>>>>> ?? - stopSignal.countDown() >>>>>> ???? - worker is now unblocked >>>>>> ?? - th.join >>>>>> ???? - main is now blocked >>>>>> >>>>>> worker thread >>>>>> ?? - runs off the end of run() >>>>>> ???? - main is now unblocked >>>>>> >>>>>> main thread >>>>>> ?? - run off the end of main() >>>>>> >>>>>> >>>>>>> >>>>>>>> >>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>>>>>> >>>>>>>>> ???? L92: ? jthreads = (jthread *)malloc(sizeof(jthread) * >>>>>>>>> num_threads); >>>>>>>>> ???????? You don't check for malloc() failure. >>>>>>>>> ???????? 'jthreads' is allocated but never freed. >>>>>>>> >>>>>>>> I will fix it. >>>>>>>> >>>>>>>> >>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>>>>>> >>>>>>>>> ???? L91: ? result = (*jvmti)->SuspendThread(jvmti, thread); >>>>>>>>> ???????? Why are you suspending the thread? >>>>>>>>> GetAllStackTraces() and >>>>>>>>> ???????? GetThreadListStackTraces() do not require the target >>>>>>>>> thread(s) >>>>>>>>> ???????? to be suspend. >>>>>>>>> >>>>>>>>> ???????? If you decide not to SuspendThread, then you don't >>>>>>>>> need the >>>>>>>>> ???????? AddCapabilities or the ResumeThread calls. >>>>>>>> >>>>>>>> Test thread might not be entered following code >>>>>>>> (stopSignal.await()). We might see deferent call stack between >>>>>>>> GetAllStackTraces() and GetThreadListStackTraces(). We cannot >>>>>>>> control to freeze call stack of test thread in Java code. >>>>>>>> (I didn't use SuspendThread() at first, but I saw some errors >>>>>>>> which causes in above.) >>>>>>>> >>>>>>>> So we need to call SuspendThread() to ensure we can see same >>>>>>>> call stack. >>>>>>> >>>>>>> If you are checking that the thread is in state WAITING then it >>>>>>> cannot escape from that state and you can sample the stack >>>>>>> multiple times from any API and get the same result. >>>>>>> >>>>>>> I suspect the errors you saw were from the apparent incorrect >>>>>>> use of the CountDownLatch. >>>>>> >>>>>> With the flow outlined above, the worker thread should be >>>>>> nicely blocked in stopSignal.await() when stuff is sampled. >>>>>> >>>>>> Dan >>>>>> >>>>>> >>>>>>> >>>>>>> Cheers, >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>>> Dan >>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>> >>>>>>>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> I uploaded new webrev. Could review again? >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>>>>>> >>>>>>>>>>> Updates look fine - thanks. >>>>>>>>>>> >>>>>>>>>>> One minor nit: >>>>>>>>>>> >>>>>>>>>>> 1274 _collector.allocate_and_fill_stacks(1); >>>>>>>>>>> 1275 _collector.set_result(JVMTI_ERROR_NONE); >>>>>>>>>>> >>>>>>>>>>> In the other places where you use _collector you rely on >>>>>>>>>>> result being initialized to JVMTI_ERROR_NONE, rather than >>>>>>>>>>> setting it directly after allocate_and_fill_stacks(). >>>>>>>>>> >>>>>>>>>> Fixed. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> ? 820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>> ? 821 java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>>>>>>> ? 822????????? current_thread == >>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>> ? 823????????? "at safepoint / handshake or target >>>>>>>>>>>>>>>>> thread is suspended"); >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as >>>>>>>>>>>>>>>>> even if the target is suspended we must still be at a >>>>>>>>>>>>>>>>> safepoint or in a handshake with it. Makes me wonder >>>>>>>>>>>>>>>>> if we used to allow a racy stacktrace operation on a >>>>>>>>>>>>>>>>> suspended thread, assuming it would remain suspended? >>>>>>>>>>>> >>>>>>>>>>>> This function (JvmtiEnvBase::get_stack_trace()) can be >>>>>>>>>>>> called to get own stack trace. For example, we can call >>>>>>>>>>>> GetStackTrace() for current thread at JVMTI event. >>>>>>>>>>>> So I changed assert as below: >>>>>>>>>>>> >>>>>>>>>>>> ``` >>>>>>>>>>>> ??820?? assert(current_thread == java_thread || >>>>>>>>>>>> ??821 SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>> ??822????????? current_thread == >>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>> ??823????????? "call by myself / at safepoint / at >>>>>>>>>>>> handshake"); >>>>>>>>>>>> ``` >>>>>>>>>>> >>>>>>>>>>> Yep good catch. I hope current tests caught that. >>>>>>>>>> >>>>>>>>>> They would be tested in >>>>>>>>>> vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ (own call >>>>>>>>>> stacks), and getstacktr003 (call stacks in other thread). >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> Speaking of tests ... >>>>>>>>>>> >>>>>>>>>>> In the native code I think you need to check the success of >>>>>>>>>>> all JNI methods that can throw exceptions - otherwise I >>>>>>>>>>> believe the tests may trigger warnings if -Xcheck:jni is >>>>>>>>>>> used with them. See for example: >>>>>>>>>>> >>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> I updated testcases to check JNI and JVMTI function calls. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> In the Java code the target thread: >>>>>>>>>>> >>>>>>>>>>> ?? 45???? public void run() { >>>>>>>>>>> ?? 46?????? try { >>>>>>>>>>> ?? 47?????????? synchronized (lock) { >>>>>>>>>>> ?? 48?????????????? lock.wait(); >>>>>>>>>>> ?? 49?????????????? System.out.println("OK"); >>>>>>>>>>> ?? 50?????????? } >>>>>>>>>>> >>>>>>>>>>> is potentially susceptible to spurious wakeups. Using a >>>>>>>>>>> CountDownLatch would be robust. >>>>>>>>>> >>>>>>>>>> Fixed. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> David >>>>>>>>>>> ----- >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>> >>>>>>>>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current >>>>>>>>>>>>>>>>> thread, so we can use: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>>>> >>>>>>>>>>>>>> If so, we can call make_local() in L1272 without >>>>>>>>>>>>>> JavaThread (or we can pass current thread to >>>>>>>>>>>>>> make_local()). Is it right? >>>>>>>>>>>>>> >>>>>>>>>>>>>> ``` >>>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>>> 1272 >>>>>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>>>>> thread_oop), >>>>>>>>>>>>>> 1273??????????????????????????? jt, thread_oop); >>>>>>>>>>>>>> ``` >>>>>>>>>>>>> >>>>>>>>>>>>> Sorry I got confused, _calling_thread may not be the >>>>>>>>>>>>> current thread as we could be executing the handshake in >>>>>>>>>>>>> the target thread itself. So the ResourceMark is correct >>>>>>>>>>>>> as-is (implicitly for current thread). >>>>>>>>>>>>> >>>>>>>>>>>>> The argument to fill_frames will be used in the >>>>>>>>>>>>> jvmtiStackInfo and passed back to the _calling_thread, so >>>>>>>>>>>>> it must be created via make_local(_calling_thread, ...) as >>>>>>>>>>>>> you presently have. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> David >>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thank you for reviewing! I will update new webrev >>>>>>>>>>>>>>>> tomorrow. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public >>>>>>>>>>>>>>>>> StackObj { >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> ? 498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>>>>>> ? 500?? JavaThread *_calling_thread; >>>>>>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>>>>>> ? 502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class >>>>>>>>>>>>>>>>> like that as it may not be on the stack. I think >>>>>>>>>>>>>>>>> MultipleStackTracesCollector should not extend any >>>>>>>>>>>>>>>>> allocation class, and should always be embedded >>>>>>>>>>>>>>>>> directly in another class. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>>>>>>> ??? : >>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> class GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>> ?? private: >>>>>>>>>>>>>>>> ???? MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> David >>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I updated webrev for 8242428. Could you review again? >>>>>>>>>>>>>>>>>> This change migrate to use direct handshake for >>>>>>>>>>>>>>>>>> GetStackTrace() and GetThreadListStackTraces() (when >>>>>>>>>>>>>>>>>> thread_count == 1). >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> This looks really good now! I only have a few nits >>>>>>>>>>>>>>>>> below. There is one thing I don't like about it but it >>>>>>>>>>>>>>>>> requires a change to the main Handshake logic to >>>>>>>>>>>>>>>>> address - in JvmtiEnv::GetThreadListStackTraces you >>>>>>>>>>>>>>>>> have to create a ThreadsListHandle to convert the >>>>>>>>>>>>>>>>> jthread to a JavaThread, but then the >>>>>>>>>>>>>>>>> Handshake::execute_direct creates another >>>>>>>>>>>>>>>>> ThreadsListHandle internally. That's a waste. I will >>>>>>>>>>>>>>>>> discuss with Robbin and file a RFE to have an overload >>>>>>>>>>>>>>>>> of execute_direct that takes an existing TLH. Actually >>>>>>>>>>>>>>>>> it's worse than that because we have another TLH in >>>>>>>>>>>>>>>>> use at the entry point for the JVMTI functions, so I >>>>>>>>>>>>>>>>> think there may be some scope for simplifying the use >>>>>>>>>>>>>>>>> of TLH instances - future RFE. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> ??451 GetStackTraceClosure(JvmtiEnv *env, jint >>>>>>>>>>>>>>>>> start_depth, jint max_count, >>>>>>>>>>>>>>>>> ??452 jvmtiFrameInfo* frame_buffer, jint* count_ptr) >>>>>>>>>>>>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>>>>>> ??454?????? _env(env), _start_depth(start_depth), >>>>>>>>>>>>>>>>> _max_count(max_count), >>>>>>>>>>>>>>>>> ??455 _frame_buffer(frame_buffer), _count_ptr(count_ptr), >>>>>>>>>>>>>>>>> ??456 _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Nit: can you do one initializer per line please. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public >>>>>>>>>>>>>>>>> StackObj { >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> ??498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>>>>>> ??500?? JavaThread *_calling_thread; >>>>>>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>>>>>> ??502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class >>>>>>>>>>>>>>>>> like that as it may not be on the stack. I think >>>>>>>>>>>>>>>>> MultipleStackTracesCollector should not extend any >>>>>>>>>>>>>>>>> allocation class, and should always be embedded >>>>>>>>>>>>>>>>> directly in another class. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 481 MultipleStackTracesCollector(JvmtiEnv *env, jint >>>>>>>>>>>>>>>>> max_frame_count) { >>>>>>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>>>>>> ??483???? _max_frame_count = max_frame_count; >>>>>>>>>>>>>>>>> ??484???? _frame_count_total = 0; >>>>>>>>>>>>>>>>> ??485???? _head = NULL; >>>>>>>>>>>>>>>>> ??486???? _stack_info = NULL; >>>>>>>>>>>>>>>>> ??487???? _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> As you are touching this can you change it to use an >>>>>>>>>>>>>>>>> initializer list as you did for the HandshakeClosure, >>>>>>>>>>>>>>>>> and please keep one item per line. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> ??820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>> ??821 java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>>>>>>> ??822????????? current_thread == >>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>> ??823????????? "at safepoint / handshake or target >>>>>>>>>>>>>>>>> thread is suspended"); >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as >>>>>>>>>>>>>>>>> even if the target is suspended we must still be at a >>>>>>>>>>>>>>>>> safepoint or in a handshake with it. Makes me wonder >>>>>>>>>>>>>>>>> if we used to allow a racy stacktrace operation on a >>>>>>>>>>>>>>>>> suspended thread, assuming it would remain suspended? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>>>>>>>>> 1269 >>>>>>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && (jt->threadObj() != >>>>>>>>>>>>>>>>> NULL)) { >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 1272 >>>>>>>>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>>>>>>>> thread_oop), >>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> It is frustrating that this entire call chain started >>>>>>>>>>>>>>>>> with a jthread reference, which we converted to a >>>>>>>>>>>>>>>>> JavaThread, only to eventually need to convert it back >>>>>>>>>>>>>>>>> to a jthread! I think there is some scope for >>>>>>>>>>>>>>>>> simplification here but not as part of this change. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current >>>>>>>>>>>>>>>>> thread, so we can use: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I'm still pondering the test logic but wanted to send >>>>>>>>>>>>>>>>> this now. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>> VM_GetThreadListStackTrace (for >>>>>>>>>>>>>>>>>> GetThreadListStackTraces) and VM_GetAllStackTraces >>>>>>>>>>>>>>>>>> (for GetAllStackTraces) have inherited >>>>>>>>>>>>>>>>>> VM_GetMultipleStackTraces VM operation which provides >>>>>>>>>>>>>>>>>> the feature to generate jvmtiStackInfo. I modified >>>>>>>>>>>>>>>>>> VM_GetMultipleStackTraces to a normal C++ class to >>>>>>>>>>>>>>>>>> share with HandshakeClosure for >>>>>>>>>>>>>>>>>> GetThreadListStackTraces (GetSingleStackTraceClosure). >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Also I added new testcases which test >>>>>>>>>>>>>>>>>> GetThreadListStackTraces() with thread_count == 1 and >>>>>>>>>>>>>>>>>> with all threads. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti >>>>>>>>>>>>>>>>>> serviceability/jdwp vmTestbase/nsk/jvmti >>>>>>>>>>>>>>>>>> vmTestbase/nsk/jdi vmTestbase/nsk/jdwp. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ?? JBS: >>>>>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>>>>>> ?? webrev: >>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> This change replace following VM operations to >>>>>>>>>>>>>>>>>>> direct handshake. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces >>>>>>>>>>>>>>>>>>> (GetThreadListStackTrace()) >>>>>>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> GetThreadListStackTrace() uses direct handshake if >>>>>>>>>>>>>>>>>>> thread count == 1. In other case (thread count > 1), >>>>>>>>>>>>>>>>>>> it would be performed as VM operation >>>>>>>>>>>>>>>>>>> (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>>>>>> Caller of VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>> (JvmtiEnvThreadState::reset_current_location()) >>>>>>>>>>>>>>>>>>> might be called at safepoint. So I added safepoint >>>>>>>>>>>>>>>>>>> check in its caller. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti >>>>>>>>>>>>>>>>>>> serviceability/jdwp vmTestbase/nsk/jvmti >>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jdi vmTestbase/ns >>>>>>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Also I tested it on submit repo, then it has >>>>>>>>>>>>>>>>>>> execution error >>>>>>>>>>>>>>>>>>> (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) >>>>>>>>>>>>>>>>>>> due to dependency error. So I think it does not >>>>>>>>>>>>>>>>>>> occur by this change. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>> >>>>>> >> From david.holmes at oracle.com Fri Jul 3 22:18:53 2020 From: david.holmes at oracle.com (David Holmes) Date: Sat, 4 Jul 2020 08:18:53 +1000 Subject: [15] RFR(T) 8247527: serviceability/dcmd/gc/HeapDumpCompressedTest.java fails with Graal + ZGC In-Reply-To: References: Message-ID: <030c19da-616e-3b05-da33-5add5e6da747@oracle.com> On 4/07/2020 4:30 am, Vladimir Kozlov wrote: > Thank you, David, for looking on changes. > > I will remember to update tests. I filed RFE 8248815 [1] for tracking. > > Can you approve this fix now? Yes - thanks. David > Thanks, > Vladimir K > > [1] https://bugs.openjdk.java.net/browse/JDK-8248815 > > On 7/2/20 10:09 PM, David Holmes wrote: >> Hi Igor, >> >> On 3/07/2020 12:59 pm, Igor Ignatyev wrote: >>> Hi David, >>> >>> it's in my todo list to improve this situation and have vm.gc.X to >>> take selected JIT into account; and update existing (>200) >>> occurrences of 'vm.gc.X & !vm.graal.enabled' >> >> 200+ ouch! :( >> >> I guess this fix doesn't make the situation any worse in a practical >> sense. >> >> Thanks, >> David >> ----- >> >>> -- Igor >>> >>>> On Jul 2, 2020, at 7:25 PM, David Holmes >>>> wrote: >>>> >>>> Hi Vladimir, >>>> >>>> On 3/07/2020 12:02 pm, Vladimir Kozlov wrote: >>>>> https://cr.openjdk.java.net/~kvn/8247527/webrev.00/ >>>>> https://bugs.openjdk.java.net/browse/JDK-8247527 >>>>> Test should have @requires which excludes running Graal with GC >>>>> which it does not support. >>>> >>>> I find it somewhat disturbing that a generic test has to know about >>>> the limitations between GCs and Graal! >>>> >>>> I would have been more inclined to just exclude this test when >>>> running with Graal, even if that theoretically reduced the test >>>> coverage in a ting way. >>>> >>>> If/When Graal supports these other GCs who will remember to >>>> re-enable these test cases? >>>> >>>> Thanks, >>>> David >>>> >>>>> Testing: hs-tier1,hs-tier4-graal >>>>> Thanks, >>>>> Vladimir >>> From vladimir.kozlov at oracle.com Fri Jul 3 22:47:24 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 3 Jul 2020 15:47:24 -0700 Subject: [15] RFR(T) 8247527: serviceability/dcmd/gc/HeapDumpCompressedTest.java fails with Graal + ZGC In-Reply-To: <030c19da-616e-3b05-da33-5add5e6da747@oracle.com> References: <030c19da-616e-3b05-da33-5add5e6da747@oracle.com> Message-ID: Thank you, David Vladimir K On 7/3/20 3:18 PM, David Holmes wrote: > On 4/07/2020 4:30 am, Vladimir Kozlov wrote: >> Thank you, David, for looking on changes. >> >> I will remember to update tests. I filed RFE 8248815 [1] for tracking. >> >> Can you approve this fix now? > > Yes - thanks. > > David > >> Thanks, >> Vladimir K >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8248815 >> >> On 7/2/20 10:09 PM, David Holmes wrote: >>> Hi Igor, >>> >>> On 3/07/2020 12:59 pm, Igor Ignatyev wrote: >>>> Hi David, >>>> >>>> it's in my todo list to improve this situation and have vm.gc.X to take selected JIT into account; and update >>>> existing (>200) occurrences of 'vm.gc.X & !vm.graal.enabled' >>> >>> 200+ ouch! :( >>> >>> I guess this fix doesn't make the situation any worse in a practical sense. >>> >>> Thanks, >>> David >>> ----- >>> >>>> -- Igor >>>> >>>>> On Jul 2, 2020, at 7:25 PM, David Holmes wrote: >>>>> >>>>> Hi Vladimir, >>>>> >>>>> On 3/07/2020 12:02 pm, Vladimir Kozlov wrote: >>>>>> https://cr.openjdk.java.net/~kvn/8247527/webrev.00/ >>>>>> https://bugs.openjdk.java.net/browse/JDK-8247527 >>>>>> Test should have @requires which excludes running Graal with GC which it does not support. >>>>> >>>>> I find it somewhat disturbing that a generic test has to know about the limitations between GCs and Graal! >>>>> >>>>> I would have been more inclined to just exclude this test when running with Graal, even if that theoretically >>>>> reduced the test coverage in a ting way. >>>>> >>>>> If/When Graal supports these other GCs who will remember to re-enable these test cases? >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>>> Testing: hs-tier1,hs-tier4-graal >>>>>> Thanks, >>>>>> Vladimir >>>> From suenaga at oss.nttdata.com Fri Jul 3 22:55:55 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Sat, 4 Jul 2020 07:55:55 +0900 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <4b7df6f8-c8f7-fedf-6366-b885d2235b58@oss.nttdata.com> <74c05dfd-ffb2-b73d-66e5-dc98752729f5@oss.nttdata.com> <763793b1-9506-44c6-0681-1c71c44a44fd@oss.nttdata.com> <5107c478-1de2-7a49-ee71-dfad6a99a667@oracle.com> <90cdfef9-e558-0d25-33f5-25b7d65bb83b@oss.nttdata.com> <0e359819-618f-c17b-a427-bb7e4daa2c58@oracle.com> <3f890bed-6892-e64d-b21b-ca0cbe11784e@oracle.com> <7dc1903b-6295-3864-6301-4c02aacdb971@oss.nttdata.com> <04e09eb7-9483-8f63-4bb4-01c9e814802e@oracle.com> <238a89a3-06b1-90e4-d5a4-8dc2f1ed12ab@oracle.com> Message-ID: Hi Serguei, I'm not an Oracle employee, so I cannot know real request(s) from your customers. However JDK-8201641 says Dynatrace has requested this enhancement. BTW I haven't heared any request from my customers about this. Thanks, Yasumasa On 2020/07/04 4:32, serguei.spitsyn at oracle.com wrote: > Hi Yasumasa, > > This difference is not that big to care about. > I feel this is really rare case and so, does not worth these complications. > Do we have a real request from customers to optimize it? > > Thanks, > Serguei > > > On 7/3/20 01:16, Yasumasa Suenaga wrote: >> Hi Serguei, >> >> Generally I agree with you, but I have concern about the difference of the result of GetStackTrace() and GetThreadListStackTraces(). >> >> ? GetStackTrace: jvmtiFrameInfo >> ? GetThreadListStackTraces: jvmtiStackInfo >> >> jvmtiStackInfo contains thread state, and it is ensured it is the state of the call stack. >> If we want to get both call stack and thread state, we need to suspend target thread, and call both GetStackTrace() and GetThreadState(). Is it ok? >> >> I was wondering if JDK-8201641 (parent ticket of this change) needed them for profiling (dynatrace?) >> If it is responsibility of JVMTI agent implementor, I remove this closure. >> >> >> Thanks, >> >> Yasumasa >> >> >> On 2020/07/03 16:45, serguei.spitsyn at oracle.com wrote: >>> Hi Yasumasa, >>> >>> After some thinking I've concluded that I do not like this optimization >>> of the GetThreadListStackTraces with GetSingleStackTraceClosure. >>> >>> We may need more opinions on this but these are my points: >>> ??- it adds some complexity and ugliness >>> ??- a win is doubtful because it has to be a rare case, so that total overhead should not be high >>> ??- if it is really high for some use cases then it is up to the user >>> ??? to optimize it with using GetStackTrace instead >>> >>> In such cases with doubtful overhead I usually prefer the simplicity. >>> >>> Good examples where it makes sense to optimize are checks for target thread to be current thread. >>> In such cases there is no need to suspend the target thread, or use a VMop/HandshakeClosure. >>> For instance, please, see the Monitor functions with the check: (java_thread == calling_thread). >>> Getting information for current thread is frequently used case, e.g. to get info at an event point. >>> >>> Thanks, >>> Serguei >>> >>> >>> On 7/2/20 23:29, Yasumasa Suenaga wrote: >>>> Hi Dan, David, >>>> >>>> I uploaded new webrev. Could you review again? >>>> >>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/ >>>> >>>> OneGetThreadListStackTraces.java in this webrev would wait until thread state is transited to "waiting" with spin wait. >>>> CountDownLatch::await call as Dan pointed is fixed in it :) >>>> >>>> Diff from webrev.03: >>>> http://hg.openjdk.java.net/jdk/submit/rev/c9aeb7001e50 >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2020/07/03 14:15, David Holmes wrote: >>>>> >>>>> >>>>> On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: >>>>>> On 2020/07/03 12:24, Daniel D. Daugherty wrote: >>>>>>> On 7/2/20 10:50 PM, David Holmes wrote: >>>>>>>> Sorry I'm responding here without seeing latest webrev but there is enough context I think ... >>>>>>>> >>>>>>>> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>>>>>>>> Hi Dan, >>>>>>>>> >>>>>>>>> Thanks for your comment! >>>>>>>>> >>>>>>>>> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>>>>>>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi David, >>>>>>>>>>> >>>>>>>>>>> I upload new webrev. Could you review again? >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>>>>>>> >>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>> ???? L1542: ??? // Get stack trace with handshake >>>>>>>>>> ???????? nit - please add a period at the end. >>>>>>>>> >>>>>>>>> I will fix it. >>>>>>>>> >>>>>>>>> >>>>>>>>>> ???? L1591: *stack_info_ptr = op.stack_info(); >>>>>>>>>> ???????? The return parameter should not be touched unless the return >>>>>>>>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>> >>>>>>>>>> ???? old L1582: ? if (err == JVMTI_ERROR_NONE) { >>>>>>>>>> ???????? Please restore this check. The return parameter should not >>>>>>>>>> ???????? be touched unless the return code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>> >>>>>>>>> I will fix it. >>>>>>>> >>>>>>>> But op.stack_info() will return NULL if the error is not JVMTI_ERROR_NONE. Are you (Dan) concerned about someone passing in a non-null/initialized out-pointer that will be reset to NULL if there was an error? >>>>>>> >>>>>>> Actually the way we used to test this in POSIX tests is to call >>>>>>> an API with known bad parameters and the return parameter ptr >>>>>>> set to NULL. If the return parameter ptr was touched when an >>>>>>> error should have been detected on an earlier parameter, then >>>>>>> the test failed. >>>>>>> >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> ???? L1272: ? if (!jt->is_exiting() && (thread_oop != NULL)) { >>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>> >>>>>>>>> I will fix it. >>>>>>>>> >>>>>>>>> >>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>> ???? old L1532: ? _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>>>>>>> ???????? This deletion of the _result field threw me for a minute and then >>>>>>>>>> ???? ? ? I figured out that the field is init to JVMTI_ERROR_THREAD_NOT_ALIVE >>>>>>>>>> ???????? in the constructor. >>>>>>>>>> >>>>>>>>>> ???? L1553: ? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>> >>>>>>>>> I will fix it. >>>>>>>>> >>>>>>>>> >>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>> ???? No comments. >>>>>>>>>> >>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>> ???? No comments. >>>>>>>>>> >>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>>>>>>> ???? No comments. >>>>>>>>>> >>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>> ???? L64: ??????? startSignal.countDown(); >>>>>>>>>> ???????? I was expecting this to be a call to await() instead of >>>>>>>>>> ???????? countDown(). What am I missing here? >>>>>>>>>> >>>>>>>>>> ???????? I think this test might be passing by accident right now, but... >>>>>>>>> >>>>>>>>> Main thread (which call JVMTI functions to test) should wait until test thread is ready. >>>>>>>>> So main thread would wait startSignal, and test thread would count down. >>>>>>>> >>>>>>>> No! >>>>>>>> >>>>>>>> The test thread that previously called obj.wait() now calls latch.await(). >>>>>>>> >>>>>>>> The main thread that previously called obj.notify() now calls latch.countDown(). >>>>>>>> >>>>>>>> The main thread continues to spin until it sees the target is WAITING before proceeding with the test. >>>>>> >>>>>> If I add spin wait to wait until transit target thread state is WAITING (as following), we don't need to call SuspendThread(). >>>>>> Which is better? >>>>> >>>>> The original spin-wait loop checking for? WAITING is better because it is the only guarantee that the target thread is blocked where you need it to be. suspending the thread is racy as you don't know exactly where the suspend will hit. >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>>> >>>>>> ``` >>>>>> /* Wait until the thread state transits to "waiting" */ >>>>>> while (th.getState() != Thread.State.WAITING) { >>>>>> ???? Thread.onSpinWait(); >>>>>> } >>>>>> ``` >>>>>> >>>>>> For simplify, spin wait is prefer to OneGetThreadListStackTraces.java in webrev.03. >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>>> Here's the flow as I see it: >>>>>>> >>>>>>> main thread >>>>>>> ?? - start worker thread >>>>>>> ?? - startSignal.await() >>>>>>> ???? - main is now blocked >>>>>>> >>>>>>> worker thread >>>>>>> ?? - startSignal.countDown() >>>>>>> ???? - main is now unblocked >>>>>>> ?? - stopSignal.await() >>>>>>> ???? - worker is now blocked >>>>>>> >>>>>>> main thread >>>>>>> ?? - checkCallStacks(th) >>>>>>> ?? - stopSignal.countDown() >>>>>>> ???? - worker is now unblocked >>>>>>> ?? - th.join >>>>>>> ???? - main is now blocked >>>>>>> >>>>>>> worker thread >>>>>>> ?? - runs off the end of run() >>>>>>> ???? - main is now unblocked >>>>>>> >>>>>>> main thread >>>>>>> ?? - run off the end of main() >>>>>>> >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>>>>>>> ???? L92: ? jthreads = (jthread *)malloc(sizeof(jthread) * num_threads); >>>>>>>>>> ???????? You don't check for malloc() failure. >>>>>>>>>> ???????? 'jthreads' is allocated but never freed. >>>>>>>>> >>>>>>>>> I will fix it. >>>>>>>>> >>>>>>>>> >>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>>>>>>> ???? L91: ? result = (*jvmti)->SuspendThread(jvmti, thread); >>>>>>>>>> ???????? Why are you suspending the thread? GetAllStackTraces() and >>>>>>>>>> ???????? GetThreadListStackTraces() do not require the target thread(s) >>>>>>>>>> ???????? to be suspend. >>>>>>>>>> >>>>>>>>>> ???????? If you decide not to SuspendThread, then you don't need the >>>>>>>>>> ???????? AddCapabilities or the ResumeThread calls. >>>>>>>>> >>>>>>>>> Test thread might not be entered following code (stopSignal.await()). We might see deferent call stack between GetAllStackTraces() and GetThreadListStackTraces(). We cannot control to freeze call stack of test thread in Java code. >>>>>>>>> (I didn't use SuspendThread() at first, but I saw some errors which causes in above.) >>>>>>>>> >>>>>>>>> So we need to call SuspendThread() to ensure we can see same call stack. >>>>>>>> >>>>>>>> If you are checking that the thread is in state WAITING then it cannot escape from that state and you can sample the stack multiple times from any API and get the same result. >>>>>>>> >>>>>>>> I suspect the errors you saw were from the apparent incorrect use of the CountDownLatch. >>>>>>> >>>>>>> With the flow outlined above, the worker thread should be >>>>>>> nicely blocked in stopSignal.await() when stuff is sampled. >>>>>>> >>>>>>> Dan >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> Cheers, >>>>>>>> David >>>>>>>> ----- >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>>> Dan >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>> >>>>>>>>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> I uploaded new webrev. Could review again? >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>>>>>>> >>>>>>>>>>>> Updates look fine - thanks. >>>>>>>>>>>> >>>>>>>>>>>> One minor nit: >>>>>>>>>>>> >>>>>>>>>>>> 1274 _collector.allocate_and_fill_stacks(1); >>>>>>>>>>>> 1275 _collector.set_result(JVMTI_ERROR_NONE); >>>>>>>>>>>> >>>>>>>>>>>> In the other places where you use _collector you rely on result being initialized to JVMTI_ERROR_NONE, rather than setting it directly after allocate_and_fill_stacks(). >>>>>>>>>>> >>>>>>>>>>> Fixed. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> ? 820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>> ? 821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>>>>>> ? 822????????? current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>> ? 823????????? "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>>> >>>>>>>>>>>>> This function (JvmtiEnvBase::get_stack_trace()) can be called to get own stack trace. For example, we can call GetStackTrace() for current thread at JVMTI event. >>>>>>>>>>>>> So I changed assert as below: >>>>>>>>>>>>> >>>>>>>>>>>>> ``` >>>>>>>>>>>>> ??820?? assert(current_thread == java_thread || >>>>>>>>>>>>> ??821 SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>> ??822????????? current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>> ??823????????? "call by myself / at safepoint / at handshake"); >>>>>>>>>>>>> ``` >>>>>>>>>>>> >>>>>>>>>>>> Yep good catch. I hope current tests caught that. >>>>>>>>>>> >>>>>>>>>>> They would be tested in vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ (own call stacks), and getstacktr003 (call stacks in other thread). >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> Speaking of tests ... >>>>>>>>>>>> >>>>>>>>>>>> In the native code I think you need to check the success of all JNI methods that can throw exceptions - otherwise I believe the tests may trigger warnings if -Xcheck:jni is used with them. See for example: >>>>>>>>>>>> >>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>>>>>> >>>>>>>>>>> I updated testcases to check JNI and JVMTI function calls. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> In the Java code the target thread: >>>>>>>>>>>> >>>>>>>>>>>> ?? 45???? public void run() { >>>>>>>>>>>> ?? 46?????? try { >>>>>>>>>>>> ?? 47?????????? synchronized (lock) { >>>>>>>>>>>> ?? 48?????????????? lock.wait(); >>>>>>>>>>>> ?? 49?????????????? System.out.println("OK"); >>>>>>>>>>>> ?? 50?????????? } >>>>>>>>>>>> >>>>>>>>>>>> is potentially susceptible to spurious wakeups. Using a CountDownLatch would be robust. >>>>>>>>>>> >>>>>>>>>>> Fixed. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> David >>>>>>>>>>>> ----- >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> If so, we can call make_local() in L1272 without JavaThread (or we can pass current thread to make_local()). Is it right? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>>>> 1273??????????????????????????? jt, thread_oop); >>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>> >>>>>>>>>>>>>> Sorry I got confused, _calling_thread may not be the current thread as we could be executing the handshake in the target thread itself. So the ResourceMark is correct as-is (implicitly for current thread). >>>>>>>>>>>>>> >>>>>>>>>>>>>> The argument to fill_frames will be used in the jvmtiStackInfo and passed back to the _calling_thread, so it must be created via make_local(_calling_thread, ...) as you presently have. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> David >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thank you for reviewing! I will update new webrev tomorrow. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> ? 498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>>>>>>> ? 500?? JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>> ? 502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>>>>>>>> ??? : >>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> class GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>> ?? private: >>>>>>>>>>>>>>>>> ???? MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I updated webrev for 8242428. Could you review again? >>>>>>>>>>>>>>>>>>> This change migrate to use direct handshake for GetStackTrace() and GetThreadListStackTraces() (when thread_count == 1). >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> This looks really good now! I only have a few nits below. There is one thing I don't like about it but it requires a change to the main Handshake logic to address - in JvmtiEnv::GetThreadListStackTraces you have to create a ThreadsListHandle to convert the jthread to a JavaThread, but then the Handshake::execute_direct creates another ThreadsListHandle internally. That's a waste. I will discuss with Robbin and file a RFE to have an overload of execute_direct that takes an existing TLH. Actually it's worse than that because we have another TLH in use at the entry point for the JVMTI functions, so I think there may be some scope for simplifying the use of TLH instances - future RFE. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> ??451 GetStackTraceClosure(JvmtiEnv *env, jint start_depth, jint max_count, >>>>>>>>>>>>>>>>>> ??452 jvmtiFrameInfo* frame_buffer, jint* count_ptr) >>>>>>>>>>>>>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>>>>>>> ??454?????? _env(env), _start_depth(start_depth), _max_count(max_count), >>>>>>>>>>>>>>>>>> ??455 _frame_buffer(frame_buffer), _count_ptr(count_ptr), >>>>>>>>>>>>>>>>>> ??456 _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Nit: can you do one initializer per line please. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> ??498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>>>>>>> ??500?? JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>> ??502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> 481 MultipleStackTracesCollector(JvmtiEnv *env, jint max_frame_count) { >>>>>>>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>>>>>>> ??483???? _max_frame_count = max_frame_count; >>>>>>>>>>>>>>>>>> ??484???? _frame_count_total = 0; >>>>>>>>>>>>>>>>>> ??485???? _head = NULL; >>>>>>>>>>>>>>>>>> ??486???? _stack_info = NULL; >>>>>>>>>>>>>>>>>> ??487???? _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> As you are touching this can you change it to use an initializer list as you did for the HandshakeClosure, and please keep one item per line. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> ??820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>> ??821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>>>>>> ??822????????? current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>> ??823????????? "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>>>>>>>>>> 1269 >>>>>>>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> It is frustrating that this entire call chain started with a jthread reference, which we converted to a JavaThread, only to eventually need to convert it back to a jthread! I think there is some scope for simplification here but not as part of this change. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I'm still pondering the test logic but wanted to send this now. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>> VM_GetThreadListStackTrace (for GetThreadListStackTraces) and VM_GetAllStackTraces (for GetAllStackTraces) have inherited VM_GetMultipleStackTraces VM operation which provides the feature to generate jvmtiStackInfo. I modified VM_GetMultipleStackTraces to a normal C++ class to share with HandshakeClosure for GetThreadListStackTraces (GetSingleStackTraceClosure). >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Also I added new testcases which test GetThreadListStackTraces() with thread_count == 1 and with all threads. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/nsk/jdwp. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>>>>>>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> This change replace following VM operations to direct handshake. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces (GetThreadListStackTrace()) >>>>>>>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> GetThreadListStackTrace() uses direct handshake if thread count == 1. In other case (thread count > 1), it would be performed as VM operation (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>>>>>>> Caller of VM_GetCurrentLocation (JvmtiEnvThreadState::reset_current_location()) might be called at safepoint. So I added safepoint check in its caller. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/ns >>>>>>>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Also I tested it on submit repo, then it has execution error (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) due to dependency error. So I think it does not occur by this change. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>> >>> > From serguei.spitsyn at oracle.com Sun Jul 5 05:46:53 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Sat, 4 Jul 2020 22:46:53 -0700 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <74c05dfd-ffb2-b73d-66e5-dc98752729f5@oss.nttdata.com> <763793b1-9506-44c6-0681-1c71c44a44fd@oss.nttdata.com> <5107c478-1de2-7a49-ee71-dfad6a99a667@oracle.com> <90cdfef9-e558-0d25-33f5-25b7d65bb83b@oss.nttdata.com> <0e359819-618f-c17b-a427-bb7e4daa2c58@oracle.com> <3f890bed-6892-e64d-b21b-ca0cbe11784e@oracle.com> <7dc1903b-6295-3864-6301-4c02aacdb971@oss.nttdata.com> <04e09eb7-9483-8f63-4bb4-01c9e814802e@oracle.com> <238a89a3-06b1-90e4-d5a4-8dc2f1ed12ab@oracle.com> Message-ID: <4c43bc2c-551e-d042-8fa7-b63e755888d1@oracle.com> Hi Yasumasa, Okay, thanks. Then I'm okay to keep the GetSingleStackTraceClosure. http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c.html http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c.html I'm not sure the function 'is_same_thread() is needed. Why do not use the JNI IsSameObject instead? It seems to be a typo at L132 and L137. You, probably. did not want to print the same information for stack_info_1[i].frame_buffer[j].XXX twice. The code at lines 112-142 is not readable. I'd suggest to make a couple of refactoring steps. First step to simplify this a little bit would be with some renaming and getting rid of indexes: ? 71 char err_msg[EXCEPTION_MSG_LEN] = {0}; ?... ?112?? /* Iterate all jvmtiStackInfo to check */ ?113?? for (i = 0; i < num_threads, *exception_msg != '\0'; i++) { ???????? jvmtiStackInfo *si1 = stack_info_1[i]; ???????? jvmtiStackInfo *si2 = stack_info_2[i]; ?114???? if (!IsSameObject(env, si1.thread, si2.thread)) { /* jvmtiStackInfo::thread */ ?115?????? snprintf(err_msg, sizeof(err_msg), ?116??????????????? "thread[%d] is different: stack_info_1 = %p, stack_info_2 = %p", ?117??????????????? i, sinfo1.thread, sinfo2.thread); ?118???? } else if (si1.state != si2.state) { /* jvmtiStackInfo::state */ ?119?????? snprintf(err_msg, sizeof(err_msg), ?120??????????????? "state[%d] is different: stack_info_1 = %d, stack_info_2 = %d", ?121??????????????? i, si1.state, si2.state); ?122???? } else if (si1.frame_count != si2.frame_count) { /* jvmtiStackInfo::frame_count */ ?123?????? snprintf(err_msg, sizeof(err_msg), ?124??????????????? "frame_count[%d] is different: stack_info_1 = %d, stack_info_2 = %d", ?125??????????????? i, si1.frame_count, si2.frame_count); ?126???? } else { ?127?????? /* Iterate all jvmtiFrameInfo to check */ ?128?????? for (j = 0; j < si1.frame_count; j++) { ?129???????? if (si1.frame_buffer[j].method != si1.frame_buffer[j].method) { /* jvmtiFrameInfo::method */ ?130?????????? snprintf(err_msg, sizeof(err_msg), ?131??????????????????? "thread [%d] frame_buffer[%d].method is different: stack_info_1 = %lx, stack_info_2 = %lx", ?132??????????????????? i, j, si1.frame_buffer[j].method, si2.frame_buffer[j].method); ?133?????????? break; ?134???????? } else if (si1.frame_buffer[j].location != si1.frame_buffer[j].location) { /* jvmtiFrameInfo::location */ ?135?????????? snprintf(err_msg, sizeof(err_msg), ?136??????????????????? "thread [%d] frame_buffer[%d].location is different: stack_info_1 = %ld, stack_info_2 = %ld", ?137??????????????????? i, j, si1.frame_buffer[j].location, si2.frame_buffer[j].location); ?138?????????? break; ?139???????? } ?140?????? } ?141???? } ?142?? } Another step would be to create functions that implement a body of each loop. You can use the same techniques to simplify similar place (L127-L138) in the libOneGetThreadListStackTraces.c. Thanks, Serguei On 7/3/20 15:55, Yasumasa Suenaga wrote: > Hi Serguei, > > I'm not an Oracle employee, so I cannot know real request(s) from your > customers. > However JDK-8201641 says Dynatrace has requested this enhancement. > > BTW I haven't heared any request from my customers about this. > > > Thanks, > > Yasumasa > > > On 2020/07/04 4:32, serguei.spitsyn at oracle.com wrote: >> Hi Yasumasa, >> >> This difference is not that big to care about. >> I feel this is really rare case and so, does not worth these >> complications. >> Do we have a real request from customers to optimize it? >> >> Thanks, >> Serguei >> >> >> On 7/3/20 01:16, Yasumasa Suenaga wrote: >>> Hi Serguei, >>> >>> Generally I agree with you, but I have concern about the difference >>> of the result of GetStackTrace() and GetThreadListStackTraces(). >>> >>> ? GetStackTrace: jvmtiFrameInfo >>> ? GetThreadListStackTraces: jvmtiStackInfo >>> >>> jvmtiStackInfo contains thread state, and it is ensured it is the >>> state of the call stack. >>> If we want to get both call stack and thread state, we need to >>> suspend target thread, and call both GetStackTrace() and >>> GetThreadState(). Is it ok? >>> >>> I was wondering if JDK-8201641 (parent ticket of this change) needed >>> them for profiling (dynatrace?) >>> If it is responsibility of JVMTI agent implementor, I remove this >>> closure. >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2020/07/03 16:45, serguei.spitsyn at oracle.com wrote: >>>> Hi Yasumasa, >>>> >>>> After some thinking I've concluded that I do not like this >>>> optimization >>>> of the GetThreadListStackTraces with GetSingleStackTraceClosure. >>>> >>>> We may need more opinions on this but these are my points: >>>> ??- it adds some complexity and ugliness >>>> ??- a win is doubtful because it has to be a rare case, so that >>>> total overhead should not be high >>>> ??- if it is really high for some use cases then it is up to the user >>>> ??? to optimize it with using GetStackTrace instead >>>> >>>> In such cases with doubtful overhead I usually prefer the simplicity. >>>> >>>> Good examples where it makes sense to optimize are checks for >>>> target thread to be current thread. >>>> In such cases there is no need to suspend the target thread, or use >>>> a VMop/HandshakeClosure. >>>> For instance, please, see the Monitor functions with the check: >>>> (java_thread == calling_thread). >>>> Getting information for current thread is frequently used case, >>>> e.g. to get info at an event point. >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 7/2/20 23:29, Yasumasa Suenaga wrote: >>>>> Hi Dan, David, >>>>> >>>>> I uploaded new webrev. Could you review again? >>>>> >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/ >>>>> >>>>> OneGetThreadListStackTraces.java in this webrev would wait until >>>>> thread state is transited to "waiting" with spin wait. >>>>> CountDownLatch::await call as Dan pointed is fixed in it :) >>>>> >>>>> Diff from webrev.03: >>>>> http://hg.openjdk.java.net/jdk/submit/rev/c9aeb7001e50 >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2020/07/03 14:15, David Holmes wrote: >>>>>> >>>>>> >>>>>> On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: >>>>>>> On 2020/07/03 12:24, Daniel D. Daugherty wrote: >>>>>>>> On 7/2/20 10:50 PM, David Holmes wrote: >>>>>>>>> Sorry I'm responding here without seeing latest webrev but >>>>>>>>> there is enough context I think ... >>>>>>>>> >>>>>>>>> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>>>>>>>>> Hi Dan, >>>>>>>>>> >>>>>>>>>> Thanks for your comment! >>>>>>>>>> >>>>>>>>>> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>>>>>>>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>> Hi David, >>>>>>>>>>>> >>>>>>>>>>>> I upload new webrev. Could you review again? >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>>>>>>>> >>>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>>> ???? L1542: ??? // Get stack trace with handshake >>>>>>>>>>> ???????? nit - please add a period at the end. >>>>>>>>>> >>>>>>>>>> I will fix it. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> ???? L1591: *stack_info_ptr = op.stack_info(); >>>>>>>>>>> ???????? The return parameter should not be touched unless >>>>>>>>>>> the return >>>>>>>>>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>> >>>>>>>>>>> ???? old L1582: ? if (err == JVMTI_ERROR_NONE) { >>>>>>>>>>> ???????? Please restore this check. The return parameter >>>>>>>>>>> should not >>>>>>>>>>> ???????? be touched unless the return code in 'err' == >>>>>>>>>>> JVMTI_ERROR_NONE. >>>>>>>>>> >>>>>>>>>> I will fix it. >>>>>>>>> >>>>>>>>> But op.stack_info() will return NULL if the error is not >>>>>>>>> JVMTI_ERROR_NONE. Are you (Dan) concerned about someone >>>>>>>>> passing in a non-null/initialized out-pointer that will be >>>>>>>>> reset to NULL if there was an error? >>>>>>>> >>>>>>>> Actually the way we used to test this in POSIX tests is to call >>>>>>>> an API with known bad parameters and the return parameter ptr >>>>>>>> set to NULL. If the return parameter ptr was touched when an >>>>>>>> error should have been detected on an earlier parameter, then >>>>>>>> the test failed. >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> ???? L1272: ? if (!jt->is_exiting() && (thread_oop != NULL)) { >>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>> >>>>>>>>>> I will fix it. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>> ???? old L1532: ? _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>>>>>>>> ???????? This deletion of the _result field threw me for a >>>>>>>>>>> minute and then >>>>>>>>>>> ???? ? ? I figured out that the field is init to >>>>>>>>>>> JVMTI_ERROR_THREAD_NOT_ALIVE >>>>>>>>>>> ???????? in the constructor. >>>>>>>>>>> >>>>>>>>>>> ???? L1553: ? if (!jt->is_exiting() && (jt->threadObj() != >>>>>>>>>>> NULL)) { >>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>> >>>>>>>>>> I will fix it. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>> ???? No comments. >>>>>>>>>>> >>>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>>> ???? No comments. >>>>>>>>>>> >>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>>>>>>>> >>>>>>>>>>> ???? No comments. >>>>>>>>>>> >>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>>> >>>>>>>>>>> ???? L64: ??????? startSignal.countDown(); >>>>>>>>>>> ???????? I was expecting this to be a call to await() >>>>>>>>>>> instead of >>>>>>>>>>> ???????? countDown(). What am I missing here? >>>>>>>>>>> >>>>>>>>>>> ???????? I think this test might be passing by accident >>>>>>>>>>> right now, but... >>>>>>>>>> >>>>>>>>>> Main thread (which call JVMTI functions to test) should wait >>>>>>>>>> until test thread is ready. >>>>>>>>>> So main thread would wait startSignal, and test thread would >>>>>>>>>> count down. >>>>>>>>> >>>>>>>>> No! >>>>>>>>> >>>>>>>>> The test thread that previously called obj.wait() now calls >>>>>>>>> latch.await(). >>>>>>>>> >>>>>>>>> The main thread that previously called obj.notify() now calls >>>>>>>>> latch.countDown(). >>>>>>>>> >>>>>>>>> The main thread continues to spin until it sees the target is >>>>>>>>> WAITING before proceeding with the test. >>>>>>> >>>>>>> If I add spin wait to wait until transit target thread state is >>>>>>> WAITING (as following), we don't need to call SuspendThread(). >>>>>>> Which is better? >>>>>> >>>>>> The original spin-wait loop checking for? WAITING is better >>>>>> because it is the only guarantee that the target thread is >>>>>> blocked where you need it to be. suspending the thread is racy as >>>>>> you don't know exactly where the suspend will hit. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>> >>>>>>> >>>>>>> ``` >>>>>>> /* Wait until the thread state transits to "waiting" */ >>>>>>> while (th.getState() != Thread.State.WAITING) { >>>>>>> ???? Thread.onSpinWait(); >>>>>>> } >>>>>>> ``` >>>>>>> >>>>>>> For simplify, spin wait is prefer to >>>>>>> OneGetThreadListStackTraces.java in webrev.03. >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>>> Here's the flow as I see it: >>>>>>>> >>>>>>>> main thread >>>>>>>> ?? - start worker thread >>>>>>>> ?? - startSignal.await() >>>>>>>> ???? - main is now blocked >>>>>>>> >>>>>>>> worker thread >>>>>>>> ?? - startSignal.countDown() >>>>>>>> ???? - main is now unblocked >>>>>>>> ?? - stopSignal.await() >>>>>>>> ???? - worker is now blocked >>>>>>>> >>>>>>>> main thread >>>>>>>> ?? - checkCallStacks(th) >>>>>>>> ?? - stopSignal.countDown() >>>>>>>> ???? - worker is now unblocked >>>>>>>> ?? - th.join >>>>>>>> ???? - main is now blocked >>>>>>>> >>>>>>>> worker thread >>>>>>>> ?? - runs off the end of run() >>>>>>>> ???? - main is now unblocked >>>>>>>> >>>>>>>> main thread >>>>>>>> ?? - run off the end of main() >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>>>>>>>> >>>>>>>>>>> ???? L92: ? jthreads = (jthread *)malloc(sizeof(jthread) * >>>>>>>>>>> num_threads); >>>>>>>>>>> ???????? You don't check for malloc() failure. >>>>>>>>>>> ???????? 'jthreads' is allocated but never freed. >>>>>>>>>> >>>>>>>>>> I will fix it. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>>>>>>>> >>>>>>>>>>> ???? L91: ? result = (*jvmti)->SuspendThread(jvmti, thread); >>>>>>>>>>> ???????? Why are you suspending the thread? >>>>>>>>>>> GetAllStackTraces() and >>>>>>>>>>> ???????? GetThreadListStackTraces() do not require the >>>>>>>>>>> target thread(s) >>>>>>>>>>> ???????? to be suspend. >>>>>>>>>>> >>>>>>>>>>> ???????? If you decide not to SuspendThread, then you don't >>>>>>>>>>> need the >>>>>>>>>>> ???????? AddCapabilities or the ResumeThread calls. >>>>>>>>>> >>>>>>>>>> Test thread might not be entered following code >>>>>>>>>> (stopSignal.await()). We might see deferent call stack >>>>>>>>>> between GetAllStackTraces() and GetThreadListStackTraces(). >>>>>>>>>> We cannot control to freeze call stack of test thread in Java >>>>>>>>>> code. >>>>>>>>>> (I didn't use SuspendThread() at first, but I saw some errors >>>>>>>>>> which causes in above.) >>>>>>>>>> >>>>>>>>>> So we need to call SuspendThread() to ensure we can see same >>>>>>>>>> call stack. >>>>>>>>> >>>>>>>>> If you are checking that the thread is in state WAITING then >>>>>>>>> it cannot escape from that state and you can sample the stack >>>>>>>>> multiple times from any API and get the same result. >>>>>>>>> >>>>>>>>> I suspect the errors you saw were from the apparent incorrect >>>>>>>>> use of the CountDownLatch. >>>>>>>> >>>>>>>> With the flow outlined above, the worker thread should be >>>>>>>> nicely blocked in stopSignal.await() when stuff is sampled. >>>>>>>> >>>>>>>> Dan >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> Cheers, >>>>>>>>> David >>>>>>>>> ----- >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> Dan >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>> >>>>>>>>>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> >>>>>>>>>>>>>> I uploaded new webrev. Could review again? >>>>>>>>>>>>>> >>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>>>>>>>> >>>>>>>>>>>>> Updates look fine - thanks. >>>>>>>>>>>>> >>>>>>>>>>>>> One minor nit: >>>>>>>>>>>>> >>>>>>>>>>>>> 1274 _collector.allocate_and_fill_stacks(1); >>>>>>>>>>>>> 1275 _collector.set_result(JVMTI_ERROR_NONE); >>>>>>>>>>>>> >>>>>>>>>>>>> In the other places where you use _collector you rely on >>>>>>>>>>>>> result being initialized to JVMTI_ERROR_NONE, rather than >>>>>>>>>>>>> setting it directly after allocate_and_fill_stacks(). >>>>>>>>>>>> >>>>>>>>>>>> Fixed. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ? 820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>> ? 821 java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>>>>>>>>> ? 822????????? current_thread == >>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>> ? 823????????? "at safepoint / handshake or target >>>>>>>>>>>>>>>>>>> thread is suspended"); >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as >>>>>>>>>>>>>>>>>>> even if the target is suspended we must still be at >>>>>>>>>>>>>>>>>>> a safepoint or in a handshake with it. Makes me >>>>>>>>>>>>>>>>>>> wonder if we used to allow a racy stacktrace >>>>>>>>>>>>>>>>>>> operation on a suspended thread, assuming it would >>>>>>>>>>>>>>>>>>> remain suspended? >>>>>>>>>>>>>> >>>>>>>>>>>>>> This function (JvmtiEnvBase::get_stack_trace()) can be >>>>>>>>>>>>>> called to get own stack trace. For example, we can call >>>>>>>>>>>>>> GetStackTrace() for current thread at JVMTI event. >>>>>>>>>>>>>> So I changed assert as below: >>>>>>>>>>>>>> >>>>>>>>>>>>>> ``` >>>>>>>>>>>>>> ??820?? assert(current_thread == java_thread || >>>>>>>>>>>>>> ??821 SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>> ??822????????? current_thread == >>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>> ??823????????? "call by myself / at safepoint / at >>>>>>>>>>>>>> handshake"); >>>>>>>>>>>>>> ``` >>>>>>>>>>>>> >>>>>>>>>>>>> Yep good catch. I hope current tests caught that. >>>>>>>>>>>> >>>>>>>>>>>> They would be tested in >>>>>>>>>>>> vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ (own call >>>>>>>>>>>> stacks), and getstacktr003 (call stacks in other thread). >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> Speaking of tests ... >>>>>>>>>>>>> >>>>>>>>>>>>> In the native code I think you need to check the success >>>>>>>>>>>>> of all JNI methods that can throw exceptions - otherwise I >>>>>>>>>>>>> believe the tests may trigger warnings if -Xcheck:jni is >>>>>>>>>>>>> used with them. See for example: >>>>>>>>>>>>> >>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I updated testcases to check JNI and JVMTI function calls. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> In the Java code the target thread: >>>>>>>>>>>>> >>>>>>>>>>>>> ?? 45???? public void run() { >>>>>>>>>>>>> ?? 46?????? try { >>>>>>>>>>>>> ?? 47?????????? synchronized (lock) { >>>>>>>>>>>>> ?? 48?????????????? lock.wait(); >>>>>>>>>>>>> ?? 49 System.out.println("OK"); >>>>>>>>>>>>> ?? 50?????????? } >>>>>>>>>>>>> >>>>>>>>>>>>> is potentially susceptible to spurious wakeups. Using a >>>>>>>>>>>>> CountDownLatch would be robust. >>>>>>>>>>>> >>>>>>>>>>>> Fixed. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> David >>>>>>>>>>>>> ----- >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the >>>>>>>>>>>>>>>>>>> current thread, so we can use: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> If so, we can call make_local() in L1272 without >>>>>>>>>>>>>>>> JavaThread (or we can pass current thread to >>>>>>>>>>>>>>>> make_local()). Is it right? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>>>>> 1272 >>>>>>>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>>>>>>> thread_oop), >>>>>>>>>>>>>>>> 1273??????????????????????????? jt, thread_oop); >>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Sorry I got confused, _calling_thread may not be the >>>>>>>>>>>>>>> current thread as we could be executing the handshake in >>>>>>>>>>>>>>> the target thread itself. So the ResourceMark is correct >>>>>>>>>>>>>>> as-is (implicitly for current thread). >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The argument to fill_frames will be used in the >>>>>>>>>>>>>>> jvmtiStackInfo and passed back to the _calling_thread, >>>>>>>>>>>>>>> so it must be created via make_local(_calling_thread, >>>>>>>>>>>>>>> ...) as you presently have. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> David >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thank you for reviewing! I will update new webrev >>>>>>>>>>>>>>>>>> tomorrow. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public >>>>>>>>>>>>>>>>>>> StackObj { >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ? 498 class VM_GetAllStackTraces : public >>>>>>>>>>>>>>>>>>> VM_Operation { >>>>>>>>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>>>>>>>> ? 500?? JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>> ? 502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another >>>>>>>>>>>>>>>>>>> class like that as it may not be on the stack. I >>>>>>>>>>>>>>>>>>> think MultipleStackTracesCollector should not extend >>>>>>>>>>>>>>>>>>> any allocation class, and should always be embedded >>>>>>>>>>>>>>>>>>> directly in another class. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>>>>>>>>> ??? : >>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> class GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>> ?? private: >>>>>>>>>>>>>>>>>> MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I updated webrev for 8242428. Could you review again? >>>>>>>>>>>>>>>>>>>> This change migrate to use direct handshake for >>>>>>>>>>>>>>>>>>>> GetStackTrace() and GetThreadListStackTraces() >>>>>>>>>>>>>>>>>>>> (when thread_count == 1). >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> This looks really good now! I only have a few nits >>>>>>>>>>>>>>>>>>> below. There is one thing I don't like about it but >>>>>>>>>>>>>>>>>>> it requires a change to the main Handshake logic to >>>>>>>>>>>>>>>>>>> address - in JvmtiEnv::GetThreadListStackTraces you >>>>>>>>>>>>>>>>>>> have to create a ThreadsListHandle to convert the >>>>>>>>>>>>>>>>>>> jthread to a JavaThread, but then the >>>>>>>>>>>>>>>>>>> Handshake::execute_direct creates another >>>>>>>>>>>>>>>>>>> ThreadsListHandle internally. That's a waste. I will >>>>>>>>>>>>>>>>>>> discuss with Robbin and file a RFE to have an >>>>>>>>>>>>>>>>>>> overload of execute_direct that takes an existing >>>>>>>>>>>>>>>>>>> TLH. Actually it's worse than that because we have >>>>>>>>>>>>>>>>>>> another TLH in use at the entry point for the JVMTI >>>>>>>>>>>>>>>>>>> functions, so I think there may be some scope for >>>>>>>>>>>>>>>>>>> simplifying the use of TLH instances - future RFE. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ??451 GetStackTraceClosure(JvmtiEnv *env, jint >>>>>>>>>>>>>>>>>>> start_depth, jint max_count, >>>>>>>>>>>>>>>>>>> ??452 jvmtiFrameInfo* frame_buffer, jint* count_ptr) >>>>>>>>>>>>>>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>>>>>>>> ??454?????? _env(env), _start_depth(start_depth), >>>>>>>>>>>>>>>>>>> _max_count(max_count), >>>>>>>>>>>>>>>>>>> ??455 _frame_buffer(frame_buffer), >>>>>>>>>>>>>>>>>>> _count_ptr(count_ptr), >>>>>>>>>>>>>>>>>>> ??456 _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Nit: can you do one initializer per line please. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public >>>>>>>>>>>>>>>>>>> StackObj { >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ??498 class VM_GetAllStackTraces : public >>>>>>>>>>>>>>>>>>> VM_Operation { >>>>>>>>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>>>>>>>> ??500?? JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>> ??502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another >>>>>>>>>>>>>>>>>>> class like that as it may not be on the stack. I >>>>>>>>>>>>>>>>>>> think MultipleStackTracesCollector should not extend >>>>>>>>>>>>>>>>>>> any allocation class, and should always be embedded >>>>>>>>>>>>>>>>>>> directly in another class. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> 481 MultipleStackTracesCollector(JvmtiEnv *env, jint >>>>>>>>>>>>>>>>>>> max_frame_count) { >>>>>>>>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>>>>>>>> ??483???? _max_frame_count = max_frame_count; >>>>>>>>>>>>>>>>>>> ??484???? _frame_count_total = 0; >>>>>>>>>>>>>>>>>>> ??485???? _head = NULL; >>>>>>>>>>>>>>>>>>> ??486???? _stack_info = NULL; >>>>>>>>>>>>>>>>>>> ??487???? _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> As you are touching this can you change it to use an >>>>>>>>>>>>>>>>>>> initializer list as you did for the >>>>>>>>>>>>>>>>>>> HandshakeClosure, and please keep one item per line. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ??820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>> ??821 java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>>>>>>>>> ??822????????? current_thread == >>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>> ??823????????? "at safepoint / handshake or target >>>>>>>>>>>>>>>>>>> thread is suspended"); >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as >>>>>>>>>>>>>>>>>>> even if the target is suspended we must still be at >>>>>>>>>>>>>>>>>>> a safepoint or in a handshake with it. Makes me >>>>>>>>>>>>>>>>>>> wonder if we used to allow a racy stacktrace >>>>>>>>>>>>>>>>>>> operation on a suspended thread, assuming it would >>>>>>>>>>>>>>>>>>> remain suspended? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>>>>>>>>>>> 1269 >>>>>>>>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && (jt->threadObj() != >>>>>>>>>>>>>>>>>>> NULL)) { >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> 1272 >>>>>>>>>>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>>>>>>>>>> thread_oop), >>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> It is frustrating that this entire call chain >>>>>>>>>>>>>>>>>>> started with a jthread reference, which we converted >>>>>>>>>>>>>>>>>>> to a JavaThread, only to eventually need to convert >>>>>>>>>>>>>>>>>>> it back to a jthread! I think there is some scope >>>>>>>>>>>>>>>>>>> for simplification here but not as part of this change. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the >>>>>>>>>>>>>>>>>>> current thread, so we can use: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I'm still pondering the test logic but wanted to >>>>>>>>>>>>>>>>>>> send this now. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>> VM_GetThreadListStackTrace (for >>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces) and VM_GetAllStackTraces >>>>>>>>>>>>>>>>>>>> (for GetAllStackTraces) have inherited >>>>>>>>>>>>>>>>>>>> VM_GetMultipleStackTraces VM operation which >>>>>>>>>>>>>>>>>>>> provides the feature to generate jvmtiStackInfo. I >>>>>>>>>>>>>>>>>>>> modified VM_GetMultipleStackTraces to a normal C++ >>>>>>>>>>>>>>>>>>>> class to share with HandshakeClosure for >>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces (GetSingleStackTraceClosure). >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Also I added new testcases which test >>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() with thread_count == 1 >>>>>>>>>>>>>>>>>>>> and with all threads. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti >>>>>>>>>>>>>>>>>>>> serviceability/jdwp vmTestbase/nsk/jvmti >>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jdi vmTestbase/nsk/jdwp. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ?? JBS: >>>>>>>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>>>>>>>> ?? webrev: >>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> This change replace following VM operations to >>>>>>>>>>>>>>>>>>>>> direct handshake. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces >>>>>>>>>>>>>>>>>>>>> (GetThreadListStackTrace()) >>>>>>>>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> GetThreadListStackTrace() uses direct handshake if >>>>>>>>>>>>>>>>>>>>> thread count == 1. In other case (thread count > >>>>>>>>>>>>>>>>>>>>> 1), it would be performed as VM operation >>>>>>>>>>>>>>>>>>>>> (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>>>>>>>> Caller of VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>> (JvmtiEnvThreadState::reset_current_location()) >>>>>>>>>>>>>>>>>>>>> might be called at safepoint. So I added safepoint >>>>>>>>>>>>>>>>>>>>> check in its caller. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> This change has been tested in >>>>>>>>>>>>>>>>>>>>> serviceability/jvmti serviceability/jdwp >>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/ns >>>>>>>>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Also I tested it on submit repo, then it has >>>>>>>>>>>>>>>>>>>>> execution error >>>>>>>>>>>>>>>>>>>>> (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) >>>>>>>>>>>>>>>>>>>>> due to dependency error. So I think it does not >>>>>>>>>>>>>>>>>>>>> occur by this change. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>> >>>> >> From suenaga at oss.nttdata.com Sun Jul 5 13:13:03 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Sun, 5 Jul 2020 22:13:03 +0900 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: <4c43bc2c-551e-d042-8fa7-b63e755888d1@oracle.com> References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <74c05dfd-ffb2-b73d-66e5-dc98752729f5@oss.nttdata.com> <763793b1-9506-44c6-0681-1c71c44a44fd@oss.nttdata.com> <5107c478-1de2-7a49-ee71-dfad6a99a667@oracle.com> <90cdfef9-e558-0d25-33f5-25b7d65bb83b@oss.nttdata.com> <0e359819-618f-c17b-a427-bb7e4daa2c58@oracle.com> <3f890bed-6892-e64d-b21b-ca0cbe11784e@oracle.com> <7dc1903b-6295-3864-6301-4c02aacdb971@oss.nttdata.com> <04e09eb7-9483-8f63-4bb4-01c9e814802e@oracle.com> <238a89a3-06b1-90e4-d5a4-8dc2f1ed12ab@oracle.com> <4c43bc2c-551e-d042-8fa7-b63e755888d1@oracle.com> Message-ID: Hi Serguei, Thanks for your comment! I refactored testcase. Could you review again? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.05/ It would check Java exception after IsSameObject() call. Does it need? Any exceptions are not described in JNI document[1], and JNI implementation (jni_IsSameObject()) does not seem to throw it. Thanks, Yasumasa [1] https://docs.oracle.com/en/java/javase/14/docs/specs/jni/functions.html#issameobject On 2020/07/05 14:46, serguei.spitsyn at oracle.com wrote: > Hi Yasumasa, > > > Okay, thanks. > Then I'm okay to keep the GetSingleStackTraceClosure. > > > http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c.html > http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c.html > > I'm not sure the function 'is_same_thread() is needed. > Why do not use the JNI IsSameObject instead? > > It seems to be a typo at L132 and L137. > You, probably. did not want to print the same information for stack_info_1[i].frame_buffer[j].XXX twice. > > The code at lines 112-142 is not readable. > I'd suggest to make a couple of refactoring steps. > > First step to simplify this a little bit would be with some renaming and getting rid of indexes: > > ? 71 char err_msg[EXCEPTION_MSG_LEN] = {0}; > ?... > ?112?? /* Iterate all jvmtiStackInfo to check */ > ?113?? for (i = 0; i < num_threads, *exception_msg != '\0'; i++) { > ???????? jvmtiStackInfo *si1 = stack_info_1[i]; > ???????? jvmtiStackInfo *si2 = stack_info_2[i]; > ?114???? if (!IsSameObject(env, si1.thread, si2.thread)) { /* jvmtiStackInfo::thread */ > ?115?????? snprintf(err_msg, sizeof(err_msg), > ?116??????????????? "thread[%d] is different: stack_info_1 = %p, stack_info_2 = %p", > ?117??????????????? i, sinfo1.thread, sinfo2.thread); > ?118???? } else if (si1.state != si2.state) { /* jvmtiStackInfo::state */ > ?119?????? snprintf(err_msg, sizeof(err_msg), > ?120??????????????? "state[%d] is different: stack_info_1 = %d, stack_info_2 = %d", > ?121??????????????? i, si1.state, si2.state); > ?122???? } else if (si1.frame_count != si2.frame_count) { /* jvmtiStackInfo::frame_count */ > ?123?????? snprintf(err_msg, sizeof(err_msg), > ?124??????????????? "frame_count[%d] is different: stack_info_1 = %d, stack_info_2 = %d", > ?125??????????????? i, si1.frame_count, si2.frame_count); > ?126???? } else { > ?127?????? /* Iterate all jvmtiFrameInfo to check */ > ?128?????? for (j = 0; j < si1.frame_count; j++) { > ?129???????? if (si1.frame_buffer[j].method != si1.frame_buffer[j].method) { /* jvmtiFrameInfo::method */ > ?130?????????? snprintf(err_msg, sizeof(err_msg), > ?131??????????????????? "thread [%d] frame_buffer[%d].method is different: stack_info_1 = %lx, stack_info_2 = %lx", > ?132??????????????????? i, j, si1.frame_buffer[j].method, si2.frame_buffer[j].method); > ?133?????????? break; > ?134???????? } else if (si1.frame_buffer[j].location != si1.frame_buffer[j].location) { /* jvmtiFrameInfo::location */ > ?135?????????? snprintf(err_msg, sizeof(err_msg), > ?136??????????????????? "thread [%d] frame_buffer[%d].location is different: stack_info_1 = %ld, stack_info_2 = %ld", > ?137??????????????????? i, j, si1.frame_buffer[j].location, si2.frame_buffer[j].location); > ?138?????????? break; > ?139???????? } > ?140?????? } > ?141???? } > ?142?? } > > Another step would be to create functions that implement a body of each loop. > You can use the same techniques to simplify similar place (L127-L138) in the libOneGetThreadListStackTraces.c. > > Thanks, > Serguei > > > On 7/3/20 15:55, Yasumasa Suenaga wrote: >> Hi Serguei, >> >> I'm not an Oracle employee, so I cannot know real request(s) from your customers. >> However JDK-8201641 says Dynatrace has requested this enhancement. >> >> BTW I haven't heared any request from my customers about this. >> >> >> Thanks, >> >> Yasumasa >> >> >> On 2020/07/04 4:32, serguei.spitsyn at oracle.com wrote: >>> Hi Yasumasa, >>> >>> This difference is not that big to care about. >>> I feel this is really rare case and so, does not worth these complications. >>> Do we have a real request from customers to optimize it? >>> >>> Thanks, >>> Serguei >>> >>> >>> On 7/3/20 01:16, Yasumasa Suenaga wrote: >>>> Hi Serguei, >>>> >>>> Generally I agree with you, but I have concern about the difference of the result of GetStackTrace() and GetThreadListStackTraces(). >>>> >>>> ? GetStackTrace: jvmtiFrameInfo >>>> ? GetThreadListStackTraces: jvmtiStackInfo >>>> >>>> jvmtiStackInfo contains thread state, and it is ensured it is the state of the call stack. >>>> If we want to get both call stack and thread state, we need to suspend target thread, and call both GetStackTrace() and GetThreadState(). Is it ok? >>>> >>>> I was wondering if JDK-8201641 (parent ticket of this change) needed them for profiling (dynatrace?) >>>> If it is responsibility of JVMTI agent implementor, I remove this closure. >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2020/07/03 16:45, serguei.spitsyn at oracle.com wrote: >>>>> Hi Yasumasa, >>>>> >>>>> After some thinking I've concluded that I do not like this optimization >>>>> of the GetThreadListStackTraces with GetSingleStackTraceClosure. >>>>> >>>>> We may need more opinions on this but these are my points: >>>>> ??- it adds some complexity and ugliness >>>>> ??- a win is doubtful because it has to be a rare case, so that total overhead should not be high >>>>> ??- if it is really high for some use cases then it is up to the user >>>>> ??? to optimize it with using GetStackTrace instead >>>>> >>>>> In such cases with doubtful overhead I usually prefer the simplicity. >>>>> >>>>> Good examples where it makes sense to optimize are checks for target thread to be current thread. >>>>> In such cases there is no need to suspend the target thread, or use a VMop/HandshakeClosure. >>>>> For instance, please, see the Monitor functions with the check: (java_thread == calling_thread). >>>>> Getting information for current thread is frequently used case, e.g. to get info at an event point. >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>> On 7/2/20 23:29, Yasumasa Suenaga wrote: >>>>>> Hi Dan, David, >>>>>> >>>>>> I uploaded new webrev. Could you review again? >>>>>> >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/ >>>>>> >>>>>> OneGetThreadListStackTraces.java in this webrev would wait until thread state is transited to "waiting" with spin wait. >>>>>> CountDownLatch::await call as Dan pointed is fixed in it :) >>>>>> >>>>>> Diff from webrev.03: >>>>>> http://hg.openjdk.java.net/jdk/submit/rev/c9aeb7001e50 >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2020/07/03 14:15, David Holmes wrote: >>>>>>> >>>>>>> >>>>>>> On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: >>>>>>>> On 2020/07/03 12:24, Daniel D. Daugherty wrote: >>>>>>>>> On 7/2/20 10:50 PM, David Holmes wrote: >>>>>>>>>> Sorry I'm responding here without seeing latest webrev but there is enough context I think ... >>>>>>>>>> >>>>>>>>>> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi Dan, >>>>>>>>>>> >>>>>>>>>>> Thanks for your comment! >>>>>>>>>>> >>>>>>>>>>> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>>>>>>>>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>> Hi David, >>>>>>>>>>>>> >>>>>>>>>>>>> I upload new webrev. Could you review again? >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>>>>>>>>> >>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>>>> ???? L1542: ??? // Get stack trace with handshake >>>>>>>>>>>> ???????? nit - please add a period at the end. >>>>>>>>>>> >>>>>>>>>>> I will fix it. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> ???? L1591: *stack_info_ptr = op.stack_info(); >>>>>>>>>>>> ???????? The return parameter should not be touched unless the return >>>>>>>>>>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>> >>>>>>>>>>>> ???? old L1582: ? if (err == JVMTI_ERROR_NONE) { >>>>>>>>>>>> ???????? Please restore this check. The return parameter should not >>>>>>>>>>>> ???????? be touched unless the return code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>> >>>>>>>>>>> I will fix it. >>>>>>>>>> >>>>>>>>>> But op.stack_info() will return NULL if the error is not JVMTI_ERROR_NONE. Are you (Dan) concerned about someone passing in a non-null/initialized out-pointer that will be reset to NULL if there was an error? >>>>>>>>> >>>>>>>>> Actually the way we used to test this in POSIX tests is to call >>>>>>>>> an API with known bad parameters and the return parameter ptr >>>>>>>>> set to NULL. If the return parameter ptr was touched when an >>>>>>>>> error should have been detected on an earlier parameter, then >>>>>>>>> the test failed. >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> ???? L1272: ? if (!jt->is_exiting() && (thread_oop != NULL)) { >>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>> >>>>>>>>>>> I will fix it. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>> ???? old L1532: ? _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>>>>>>>>> ???????? This deletion of the _result field threw me for a minute and then >>>>>>>>>>>> ???? ? ? I figured out that the field is init to JVMTI_ERROR_THREAD_NOT_ALIVE >>>>>>>>>>>> ???????? in the constructor. >>>>>>>>>>>> >>>>>>>>>>>> ???? L1553: ? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>> >>>>>>>>>>> I will fix it. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>> ???? No comments. >>>>>>>>>>>> >>>>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>>>> ???? No comments. >>>>>>>>>>>> >>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>>>>>>>>> ???? No comments. >>>>>>>>>>>> >>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>>>> ???? L64: ??????? startSignal.countDown(); >>>>>>>>>>>> ???????? I was expecting this to be a call to await() instead of >>>>>>>>>>>> ???????? countDown(). What am I missing here? >>>>>>>>>>>> >>>>>>>>>>>> ???????? I think this test might be passing by accident right now, but... >>>>>>>>>>> >>>>>>>>>>> Main thread (which call JVMTI functions to test) should wait until test thread is ready. >>>>>>>>>>> So main thread would wait startSignal, and test thread would count down. >>>>>>>>>> >>>>>>>>>> No! >>>>>>>>>> >>>>>>>>>> The test thread that previously called obj.wait() now calls latch.await(). >>>>>>>>>> >>>>>>>>>> The main thread that previously called obj.notify() now calls latch.countDown(). >>>>>>>>>> >>>>>>>>>> The main thread continues to spin until it sees the target is WAITING before proceeding with the test. >>>>>>>> >>>>>>>> If I add spin wait to wait until transit target thread state is WAITING (as following), we don't need to call SuspendThread(). >>>>>>>> Which is better? >>>>>>> >>>>>>> The original spin-wait loop checking for? WAITING is better because it is the only guarantee that the target thread is blocked where you need it to be. suspending the thread is racy as you don't know exactly where the suspend will hit. >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>>> >>>>>>>> ``` >>>>>>>> /* Wait until the thread state transits to "waiting" */ >>>>>>>> while (th.getState() != Thread.State.WAITING) { >>>>>>>> ???? Thread.onSpinWait(); >>>>>>>> } >>>>>>>> ``` >>>>>>>> >>>>>>>> For simplify, spin wait is prefer to OneGetThreadListStackTraces.java in webrev.03. >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>>> Here's the flow as I see it: >>>>>>>>> >>>>>>>>> main thread >>>>>>>>> ?? - start worker thread >>>>>>>>> ?? - startSignal.await() >>>>>>>>> ???? - main is now blocked >>>>>>>>> >>>>>>>>> worker thread >>>>>>>>> ?? - startSignal.countDown() >>>>>>>>> ???? - main is now unblocked >>>>>>>>> ?? - stopSignal.await() >>>>>>>>> ???? - worker is now blocked >>>>>>>>> >>>>>>>>> main thread >>>>>>>>> ?? - checkCallStacks(th) >>>>>>>>> ?? - stopSignal.countDown() >>>>>>>>> ???? - worker is now unblocked >>>>>>>>> ?? - th.join >>>>>>>>> ???? - main is now blocked >>>>>>>>> >>>>>>>>> worker thread >>>>>>>>> ?? - runs off the end of run() >>>>>>>>> ???? - main is now unblocked >>>>>>>>> >>>>>>>>> main thread >>>>>>>>> ?? - run off the end of main() >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>>>>>>>>> ???? L92: ? jthreads = (jthread *)malloc(sizeof(jthread) * num_threads); >>>>>>>>>>>> ???????? You don't check for malloc() failure. >>>>>>>>>>>> ???????? 'jthreads' is allocated but never freed. >>>>>>>>>>> >>>>>>>>>>> I will fix it. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>>>>>>>>> ???? L91: ? result = (*jvmti)->SuspendThread(jvmti, thread); >>>>>>>>>>>> ???????? Why are you suspending the thread? GetAllStackTraces() and >>>>>>>>>>>> ???????? GetThreadListStackTraces() do not require the target thread(s) >>>>>>>>>>>> ???????? to be suspend. >>>>>>>>>>>> >>>>>>>>>>>> ???????? If you decide not to SuspendThread, then you don't need the >>>>>>>>>>>> ???????? AddCapabilities or the ResumeThread calls. >>>>>>>>>>> >>>>>>>>>>> Test thread might not be entered following code (stopSignal.await()). We might see deferent call stack between GetAllStackTraces() and GetThreadListStackTraces(). We cannot control to freeze call stack of test thread in Java code. >>>>>>>>>>> (I didn't use SuspendThread() at first, but I saw some errors which causes in above.) >>>>>>>>>>> >>>>>>>>>>> So we need to call SuspendThread() to ensure we can see same call stack. >>>>>>>>>> >>>>>>>>>> If you are checking that the thread is in state WAITING then it cannot escape from that state and you can sample the stack multiple times from any API and get the same result. >>>>>>>>>> >>>>>>>>>> I suspect the errors you saw were from the apparent incorrect use of the CountDownLatch. >>>>>>>>> >>>>>>>>> With the flow outlined above, the worker thread should be >>>>>>>>> nicely blocked in stopSignal.await() when stuff is sampled. >>>>>>>>> >>>>>>>>> Dan >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Cheers, >>>>>>>>>> David >>>>>>>>>> ----- >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> Dan >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I uploaded new webrev. Could review again? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> Updates look fine - thanks. >>>>>>>>>>>>>> >>>>>>>>>>>>>> One minor nit: >>>>>>>>>>>>>> >>>>>>>>>>>>>> 1274 _collector.allocate_and_fill_stacks(1); >>>>>>>>>>>>>> 1275 _collector.set_result(JVMTI_ERROR_NONE); >>>>>>>>>>>>>> >>>>>>>>>>>>>> In the other places where you use _collector you rely on result being initialized to JVMTI_ERROR_NONE, rather than setting it directly after allocate_and_fill_stacks(). >>>>>>>>>>>>> >>>>>>>>>>>>> Fixed. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> ? 820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>> ? 821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>>>>>>>> ? 822????????? current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>> ? 823????????? "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> This function (JvmtiEnvBase::get_stack_trace()) can be called to get own stack trace. For example, we can call GetStackTrace() for current thread at JVMTI event. >>>>>>>>>>>>>>> So I changed assert as below: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>> ??820?? assert(current_thread == java_thread || >>>>>>>>>>>>>>> ??821 SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>> ??822????????? current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>> ??823????????? "call by myself / at safepoint / at handshake"); >>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yep good catch. I hope current tests caught that. >>>>>>>>>>>>> >>>>>>>>>>>>> They would be tested in vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ (own call stacks), and getstacktr003 (call stacks in other thread). >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> Speaking of tests ... >>>>>>>>>>>>>> >>>>>>>>>>>>>> In the native code I think you need to check the success of all JNI methods that can throw exceptions - otherwise I believe the tests may trigger warnings if -Xcheck:jni is used with them. See for example: >>>>>>>>>>>>>> >>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>>>>>>>> >>>>>>>>>>>>> I updated testcases to check JNI and JVMTI function calls. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> In the Java code the target thread: >>>>>>>>>>>>>> >>>>>>>>>>>>>> ?? 45???? public void run() { >>>>>>>>>>>>>> ?? 46?????? try { >>>>>>>>>>>>>> ?? 47?????????? synchronized (lock) { >>>>>>>>>>>>>> ?? 48?????????????? lock.wait(); >>>>>>>>>>>>>> ?? 49 System.out.println("OK"); >>>>>>>>>>>>>> ?? 50?????????? } >>>>>>>>>>>>>> >>>>>>>>>>>>>> is potentially susceptible to spurious wakeups. Using a CountDownLatch would be robust. >>>>>>>>>>>>> >>>>>>>>>>>>> Fixed. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> David >>>>>>>>>>>>>> ----- >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> If so, we can call make_local() in L1272 without JavaThread (or we can pass current thread to make_local()). Is it right? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>>>>>> 1273??????????????????????????? jt, thread_oop); >>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Sorry I got confused, _calling_thread may not be the current thread as we could be executing the handshake in the target thread itself. So the ResourceMark is correct as-is (implicitly for current thread). >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> The argument to fill_frames will be used in the jvmtiStackInfo and passed back to the _calling_thread, so it must be created via make_local(_calling_thread, ...) as you presently have. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thank you for reviewing! I will update new webrev tomorrow. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> ? 498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>>>>>>>>> ? 500?? JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>> ? 502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>>>>>>>>>> ??? : >>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> class GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>> ?? private: >>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> I updated webrev for 8242428. Could you review again? >>>>>>>>>>>>>>>>>>>>> This change migrate to use direct handshake for GetStackTrace() and GetThreadListStackTraces() (when thread_count == 1). >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> This looks really good now! I only have a few nits below. There is one thing I don't like about it but it requires a change to the main Handshake logic to address - in JvmtiEnv::GetThreadListStackTraces you have to create a ThreadsListHandle to convert the jthread to a JavaThread, but then the Handshake::execute_direct creates another ThreadsListHandle internally. That's a waste. I will discuss with Robbin and file a RFE to have an overload of execute_direct that takes an existing TLH. Actually it's worse than that because we have another TLH in use at the entry point for the JVMTI functions, so I think there may be some scope for simplifying the use of TLH instances - future RFE. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> ??451 GetStackTraceClosure(JvmtiEnv *env, jint start_depth, jint max_count, >>>>>>>>>>>>>>>>>>>> ??452 jvmtiFrameInfo* frame_buffer, jint* count_ptr) >>>>>>>>>>>>>>>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>>>>>>>>> ??454?????? _env(env), _start_depth(start_depth), _max_count(max_count), >>>>>>>>>>>>>>>>>>>> ??455 _frame_buffer(frame_buffer), _count_ptr(count_ptr), >>>>>>>>>>>>>>>>>>>> ??456 _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Nit: can you do one initializer per line please. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> ??498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>>>>>>>>> ??500?? JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>> ??502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> 481 MultipleStackTracesCollector(JvmtiEnv *env, jint max_frame_count) { >>>>>>>>>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>>>>>>>>> ??483???? _max_frame_count = max_frame_count; >>>>>>>>>>>>>>>>>>>> ??484???? _frame_count_total = 0; >>>>>>>>>>>>>>>>>>>> ??485???? _head = NULL; >>>>>>>>>>>>>>>>>>>> ??486???? _stack_info = NULL; >>>>>>>>>>>>>>>>>>>> ??487???? _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> As you are touching this can you change it to use an initializer list as you did for the HandshakeClosure, and please keep one item per line. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> ??820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>> ??821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>>>>>>>> ??822????????? current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>> ??823????????? "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>>>>>>>>>>>> 1269 >>>>>>>>>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> It is frustrating that this entire call chain started with a jthread reference, which we converted to a JavaThread, only to eventually need to convert it back to a jthread! I think there is some scope for simplification here but not as part of this change. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I'm still pondering the test logic but wanted to send this now. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>> VM_GetThreadListStackTrace (for GetThreadListStackTraces) and VM_GetAllStackTraces (for GetAllStackTraces) have inherited VM_GetMultipleStackTraces VM operation which provides the feature to generate jvmtiStackInfo. I modified VM_GetMultipleStackTraces to a normal C++ class to share with HandshakeClosure for GetThreadListStackTraces (GetSingleStackTraceClosure). >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Also I added new testcases which test GetThreadListStackTraces() with thread_count == 1 and with all threads. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/nsk/jdwp. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>>>>>>>>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> This change replace following VM operations to direct handshake. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>>>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces (GetThreadListStackTrace()) >>>>>>>>>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTrace() uses direct handshake if thread count == 1. In other case (thread count > 1), it would be performed as VM operation (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>>>>>>>>> Caller of VM_GetCurrentLocation (JvmtiEnvThreadState::reset_current_location()) might be called at safepoint. So I added safepoint check in its caller. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/ns >>>>>>>>>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Also I tested it on submit repo, then it has execution error (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) due to dependency error. So I think it does not occur by this change. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>> >>>>>>>>> >>>>> >>> > From patric.hedlin at oracle.com Sun Jul 5 15:26:48 2020 From: patric.hedlin at oracle.com (Patric Hedlin) Date: Sun, 5 Jul 2020 17:26:48 +0200 Subject: RFR(S): 8247762: [aarch64] Timeout in .../HeapDumpTestWithActiveProcess.java due to inf. loop in AARCH64CurrentFrameGuess.run() Message-ID: Dear all, I would like to ask for help to review the following change/update: Issue:? https://bugs.openjdk.java.net/browse/JDK-8247762 Webrev: http://cr.openjdk.java.net/~phedlin/tr8247762/ AARCH64CurrentFrameGuess.run() may loop indefinitely in a bad stack-walk. This is JDK-8231635 applied to AArch64. NOTE: This patch is only a partial solution to the various /stack-walk/timeout/ issues in this particular test-case. Testing: HeapDumpTestWithActiveProcess.java Best regards, Patric From serguei.spitsyn at oracle.com Mon Jul 6 07:32:12 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 6 Jul 2020 00:32:12 -0700 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <74c05dfd-ffb2-b73d-66e5-dc98752729f5@oss.nttdata.com> <763793b1-9506-44c6-0681-1c71c44a44fd@oss.nttdata.com> <5107c478-1de2-7a49-ee71-dfad6a99a667@oracle.com> <90cdfef9-e558-0d25-33f5-25b7d65bb83b@oss.nttdata.com> <0e359819-618f-c17b-a427-bb7e4daa2c58@oracle.com> <3f890bed-6892-e64d-b21b-ca0cbe11784e@oracle.com> <7dc1903b-6295-3864-6301-4c02aacdb971@oss.nttdata.com> <04e09eb7-9483-8f63-4bb4-01c9e814802e@oracle.com> <238a89a3-06b1-90e4-d5a4-8dc2f1ed12ab@oracle.com> <4c43bc2c-551e-d042-8fa7-b63e755888d1@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From aph at redhat.com Mon Jul 6 08:37:35 2020 From: aph at redhat.com (Andrew Haley) Date: Mon, 6 Jul 2020 09:37:35 +0100 Subject: RFR(S): 8247762: [aarch64] Timeout in .../HeapDumpTestWithActiveProcess.java due to inf. loop in AARCH64CurrentFrameGuess.run() In-Reply-To: References: Message-ID: <36c12487-cfc9-3823-c4a3-7605629beca5@redhat.com> On 05/07/2020 16:26, Patric Hedlin wrote: > Issue:? https://bugs.openjdk.java.net/browse/JDK-8247762 > Webrev: http://cr.openjdk.java.net/~phedlin/tr8247762/ > > > AARCH64CurrentFrameGuess.run() may loop indefinitely in a bad > stack-walk. This is JDK-8231635 applied to AArch64. 141 Frame oldFrame = frame; 142 frame = frame.sender(map); 143 if (frame.getSP().lessThanOrEqual(oldFrame.getSP())) { 144 // Frame points to itself or to a location in the wrong direction. 145 // Break the loop and move on to next offset. 146 if (DEBUG) { 147 System.out.println("CurrentFrameGuess: frame <= oldFrame: " + frame); 148 } 149 break; 150 } 151 } OK, that looks like a reasonable thing to do, but I would wonder how the stack got into that mess. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From suenaga at oss.nttdata.com Mon Jul 6 13:29:23 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Mon, 6 Jul 2020 22:29:23 +0900 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <74c05dfd-ffb2-b73d-66e5-dc98752729f5@oss.nttdata.com> <763793b1-9506-44c6-0681-1c71c44a44fd@oss.nttdata.com> <5107c478-1de2-7a49-ee71-dfad6a99a667@oracle.com> <90cdfef9-e558-0d25-33f5-25b7d65bb83b@oss.nttdata.com> <0e359819-618f-c17b-a427-bb7e4daa2c58@oracle.com> <3f890bed-6892-e64d-b21b-ca0cbe11784e@oracle.com> <7dc1903b-6295-3864-6301-4c02aacdb971@oss.nttdata.com> <04e09eb7-9483-8f63-4bb4-01c9e814802e@oracle.com> <238a89a3-06b1-90e4-d5a4-8dc2f1ed12ab@oracle.com> <4c43bc2c-551e-d042-8fa7-b63e755888d1@oracle.com> Message-ID: <8c387550-a06e-6320-f453-0e898486befb@oss.nttdata.com> Hi Serguei, Thanks for your comment! I think C++ is more simple to implement the test agent as you said. So I implement it in C++ in new webrev. Could you review again? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.06/ Also I refactored libGetThreadListStackTraces.cpp, and I've kept exception check after IsSameObject(). Thanks, Yasumasa On 2020/07/06 16:32, serguei.spitsyn at oracle.com wrote: > Hi Yasumasa, > > Thank you for the update. > I think, a pending exception after IsSameObject needs to be checked. > > The checkStackInfo() needs one more refactoring as I've already suggested. > The body of the loop at L68-L78 should be converted to a function check_frame_info. > The si1->frame_buffer[i] and si2->frame_buffer[i] will be passed as fi1 and fi2. > The index can be passed as well. > I'm still suggesting to simplify the local exception_msg to something shorter like err_msg or exc_msg. > > I'm not sure using fatal is right here: > > This fragment looks strange: > > 152 if ((*env)->IsSameObject(env, stack_info[i].thread, thread)) { > 153 target_info = &stack_info[i]; > 154 break; > 155 } else if ((*env)->ExceptionOccurred(env)) { > 156 (*env)->ExceptionDescribe(env); > 157 (*env)->FatalError(env, __FILE__); > 158 } > > I expected it to be: > > jboolean same = (*env)->IsSameObject(env, stack_info[i].thread, thread); > if ((*env)->ExceptionOccurred(env)) { > (*env)->ExceptionDescribe(env); > (*env)->FatalError(env, __FILE__); > } > if (same) { > target_info = &stack_info[i]; > ? break; > } > > Would it better to port this agent to C++ to simplify this code nicer? > > Thanks, > Serguei > > > On 7/5/20 06:13, Yasumasa Suenaga wrote: >> Hi Serguei, >> >> Thanks for your comment! >> I refactored testcase. Could you review again? >> >> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.05/ >> >> It would check Java exception after IsSameObject() call. Does it need? >> Any exceptions are not described in JNI document[1], and JNI implementation (jni_IsSameObject()) does not seem to throw it. >> >> >> Thanks, >> >> Yasumasa >> >> >> [1] https://docs.oracle.com/en/java/javase/14/docs/specs/jni/functions.html#issameobject >> >> >> On 2020/07/05 14:46, serguei.spitsyn at oracle.com wrote: >>> Hi Yasumasa, >>> >>> >>> Okay, thanks. >>> Then I'm okay to keep the GetSingleStackTraceClosure. >>> >>> >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c.html >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c.html >>> >>> I'm not sure the function 'is_same_thread() is needed. >>> Why do not use the JNI IsSameObject instead? >>> >>> It seems to be a typo at L132 and L137. >>> You, probably. did not want to print the same information for stack_info_1[i].frame_buffer[j].XXX twice. >>> >>> The code at lines 112-142 is not readable. >>> I'd suggest to make a couple of refactoring steps. >>> >>> First step to simplify this a little bit would be with some renaming and getting rid of indexes: >>> >>> ?? 71 char err_msg[EXCEPTION_MSG_LEN] = {0}; >>> ??... >>> ??112?? /* Iterate all jvmtiStackInfo to check */ >>> ??113?? for (i = 0; i < num_threads, *exception_msg != '\0'; i++) { >>> ????????? jvmtiStackInfo *si1 = stack_info_1[i]; >>> ????????? jvmtiStackInfo *si2 = stack_info_2[i]; >>> ??114???? if (!IsSameObject(env, si1.thread, si2.thread)) { /* jvmtiStackInfo::thread */ >>> ??115?????? snprintf(err_msg, sizeof(err_msg), >>> ??116??????????????? "thread[%d] is different: stack_info_1 = %p, stack_info_2 = %p", >>> ??117??????????????? i, sinfo1.thread, sinfo2.thread); >>> ??118???? } else if (si1.state != si2.state) { /* jvmtiStackInfo::state */ >>> ??119?????? snprintf(err_msg, sizeof(err_msg), >>> ??120??????????????? "state[%d] is different: stack_info_1 = %d, stack_info_2 = %d", >>> ??121??????????????? i, si1.state, si2.state); >>> ??122???? } else if (si1.frame_count != si2.frame_count) { /* jvmtiStackInfo::frame_count */ >>> ??123?????? snprintf(err_msg, sizeof(err_msg), >>> ??124??????????????? "frame_count[%d] is different: stack_info_1 = %d, stack_info_2 = %d", >>> ??125??????????????? i, si1.frame_count, si2.frame_count); >>> ??126???? } else { >>> ??127?????? /* Iterate all jvmtiFrameInfo to check */ >>> ??128?????? for (j = 0; j < si1.frame_count; j++) { >>> ??129???????? if (si1.frame_buffer[j].method != si1.frame_buffer[j].method) { /* jvmtiFrameInfo::method */ >>> ??130?????????? snprintf(err_msg, sizeof(err_msg), >>> ??131??????????????????? "thread [%d] frame_buffer[%d].method is different: stack_info_1 = %lx, stack_info_2 = %lx", >>> ??132??????????????????? i, j, si1.frame_buffer[j].method, si2.frame_buffer[j].method); >>> ??133?????????? break; >>> ??134???????? } else if (si1.frame_buffer[j].location != si1.frame_buffer[j].location) { /* jvmtiFrameInfo::location */ >>> ??135?????????? snprintf(err_msg, sizeof(err_msg), >>> ??136??????????????????? "thread [%d] frame_buffer[%d].location is different: stack_info_1 = %ld, stack_info_2 = %ld", >>> ??137??????????????????? i, j, si1.frame_buffer[j].location, si2.frame_buffer[j].location); >>> ??138?????????? break; >>> ??139???????? } >>> ??140?????? } >>> ??141???? } >>> ??142?? } >>> >>> Another step would be to create functions that implement a body of each loop. >>> You can use the same techniques to simplify similar place (L127-L138) in the libOneGetThreadListStackTraces.c. >>> >>> Thanks, >>> Serguei >>> >>> >>> On 7/3/20 15:55, Yasumasa Suenaga wrote: >>>> Hi Serguei, >>>> >>>> I'm not an Oracle employee, so I cannot know real request(s) from your customers. >>>> However JDK-8201641 says Dynatrace has requested this enhancement. >>>> >>>> BTW I haven't heared any request from my customers about this. >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2020/07/04 4:32, serguei.spitsyn at oracle.com wrote: >>>>> Hi Yasumasa, >>>>> >>>>> This difference is not that big to care about. >>>>> I feel this is really rare case and so, does not worth these complications. >>>>> Do we have a real request from customers to optimize it? >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>> On 7/3/20 01:16, Yasumasa Suenaga wrote: >>>>>> Hi Serguei, >>>>>> >>>>>> Generally I agree with you, but I have concern about the difference of the result of GetStackTrace() and GetThreadListStackTraces(). >>>>>> >>>>>> ? GetStackTrace: jvmtiFrameInfo >>>>>> ? GetThreadListStackTraces: jvmtiStackInfo >>>>>> >>>>>> jvmtiStackInfo contains thread state, and it is ensured it is the state of the call stack. >>>>>> If we want to get both call stack and thread state, we need to suspend target thread, and call both GetStackTrace() and GetThreadState(). Is it ok? >>>>>> >>>>>> I was wondering if JDK-8201641 (parent ticket of this change) needed them for profiling (dynatrace?) >>>>>> If it is responsibility of JVMTI agent implementor, I remove this closure. >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2020/07/03 16:45, serguei.spitsyn at oracle.com wrote: >>>>>>> Hi Yasumasa, >>>>>>> >>>>>>> After some thinking I've concluded that I do not like this optimization >>>>>>> of the GetThreadListStackTraces with GetSingleStackTraceClosure. >>>>>>> >>>>>>> We may need more opinions on this but these are my points: >>>>>>> ??- it adds some complexity and ugliness >>>>>>> ??- a win is doubtful because it has to be a rare case, so that total overhead should not be high >>>>>>> ??- if it is really high for some use cases then it is up to the user >>>>>>> ??? to optimize it with using GetStackTrace instead >>>>>>> >>>>>>> In such cases with doubtful overhead I usually prefer the simplicity. >>>>>>> >>>>>>> Good examples where it makes sense to optimize are checks for target thread to be current thread. >>>>>>> In such cases there is no need to suspend the target thread, or use a VMop/HandshakeClosure. >>>>>>> For instance, please, see the Monitor functions with the check: (java_thread == calling_thread). >>>>>>> Getting information for current thread is frequently used case, e.g. to get info at an event point. >>>>>>> >>>>>>> Thanks, >>>>>>> Serguei >>>>>>> >>>>>>> >>>>>>> On 7/2/20 23:29, Yasumasa Suenaga wrote: >>>>>>>> Hi Dan, David, >>>>>>>> >>>>>>>> I uploaded new webrev. Could you review again? >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/ >>>>>>>> >>>>>>>> OneGetThreadListStackTraces.java in this webrev would wait until thread state is transited to "waiting" with spin wait. >>>>>>>> CountDownLatch::await call as Dan pointed is fixed in it :) >>>>>>>> >>>>>>>> Diff from webrev.03: >>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/c9aeb7001e50 >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2020/07/03 14:15, David Holmes wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: >>>>>>>>>> On 2020/07/03 12:24, Daniel D. Daugherty wrote: >>>>>>>>>>> On 7/2/20 10:50 PM, David Holmes wrote: >>>>>>>>>>>> Sorry I'm responding here without seeing latest webrev but there is enough context I think ... >>>>>>>>>>>> >>>>>>>>>>>> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>> Hi Dan, >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>> >>>>>>>>>>>>> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>>>>>>>>>>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I upload new webrev. Could you review again? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>>>>>> ???? L1542: ??? // Get stack trace with handshake >>>>>>>>>>>>>> ???????? nit - please add a period at the end. >>>>>>>>>>>>> >>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> ???? L1591: *stack_info_ptr = op.stack_info(); >>>>>>>>>>>>>> ???????? The return parameter should not be touched unless the return >>>>>>>>>>>>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>> >>>>>>>>>>>>>> ???? old L1582: ? if (err == JVMTI_ERROR_NONE) { >>>>>>>>>>>>>> ???????? Please restore this check. The return parameter should not >>>>>>>>>>>>>> ???????? be touched unless the return code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>> >>>>>>>>>>>>> I will fix it. >>>>>>>>>>>> >>>>>>>>>>>> But op.stack_info() will return NULL if the error is not JVMTI_ERROR_NONE. Are you (Dan) concerned about someone passing in a non-null/initialized out-pointer that will be reset to NULL if there was an error? >>>>>>>>>>> >>>>>>>>>>> Actually the way we used to test this in POSIX tests is to call >>>>>>>>>>> an API with known bad parameters and the return parameter ptr >>>>>>>>>>> set to NULL. If the return parameter ptr was touched when an >>>>>>>>>>> error should have been detected on an earlier parameter, then >>>>>>>>>>> the test failed. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> ???? L1272: ? if (!jt->is_exiting() && (thread_oop != NULL)) { >>>>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>>>> >>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>> ???? old L1532: ? _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>>>>>>>>>>> ???????? This deletion of the _result field threw me for a minute and then >>>>>>>>>>>>>> ???? ? ? I figured out that the field is init to JVMTI_ERROR_THREAD_NOT_ALIVE >>>>>>>>>>>>>> ???????? in the constructor. >>>>>>>>>>>>>> >>>>>>>>>>>>>> ???? L1553: ? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>>>> >>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>> >>>>>>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>> >>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>> >>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>>>>>> ???? L64: ??????? startSignal.countDown(); >>>>>>>>>>>>>> ???????? I was expecting this to be a call to await() instead of >>>>>>>>>>>>>> ???????? countDown(). What am I missing here? >>>>>>>>>>>>>> >>>>>>>>>>>>>> ???????? I think this test might be passing by accident right now, but... >>>>>>>>>>>>> >>>>>>>>>>>>> Main thread (which call JVMTI functions to test) should wait until test thread is ready. >>>>>>>>>>>>> So main thread would wait startSignal, and test thread would count down. >>>>>>>>>>>> >>>>>>>>>>>> No! >>>>>>>>>>>> >>>>>>>>>>>> The test thread that previously called obj.wait() now calls latch.await(). >>>>>>>>>>>> >>>>>>>>>>>> The main thread that previously called obj.notify() now calls latch.countDown(). >>>>>>>>>>>> >>>>>>>>>>>> The main thread continues to spin until it sees the target is WAITING before proceeding with the test. >>>>>>>>>> >>>>>>>>>> If I add spin wait to wait until transit target thread state is WAITING (as following), we don't need to call SuspendThread(). >>>>>>>>>> Which is better? >>>>>>>>> >>>>>>>>> The original spin-wait loop checking for? WAITING is better because it is the only guarantee that the target thread is blocked where you need it to be. suspending the thread is racy as you don't know exactly where the suspend will hit. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> David >>>>>>>>> ----- >>>>>>>>> >>>>>>>>>> >>>>>>>>>> ``` >>>>>>>>>> /* Wait until the thread state transits to "waiting" */ >>>>>>>>>> while (th.getState() != Thread.State.WAITING) { >>>>>>>>>> ???? Thread.onSpinWait(); >>>>>>>>>> } >>>>>>>>>> ``` >>>>>>>>>> >>>>>>>>>> For simplify, spin wait is prefer to OneGetThreadListStackTraces.java in webrev.03. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> Here's the flow as I see it: >>>>>>>>>>> >>>>>>>>>>> main thread >>>>>>>>>>> ?? - start worker thread >>>>>>>>>>> ?? - startSignal.await() >>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>> >>>>>>>>>>> worker thread >>>>>>>>>>> ?? - startSignal.countDown() >>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>> ?? - stopSignal.await() >>>>>>>>>>> ???? - worker is now blocked >>>>>>>>>>> >>>>>>>>>>> main thread >>>>>>>>>>> ?? - checkCallStacks(th) >>>>>>>>>>> ?? - stopSignal.countDown() >>>>>>>>>>> ???? - worker is now unblocked >>>>>>>>>>> ?? - th.join >>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>> >>>>>>>>>>> worker thread >>>>>>>>>>> ?? - runs off the end of run() >>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>> >>>>>>>>>>> main thread >>>>>>>>>>> ?? - run off the end of main() >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>>>>>>>>>>> ???? L92: ? jthreads = (jthread *)malloc(sizeof(jthread) * num_threads); >>>>>>>>>>>>>> ???????? You don't check for malloc() failure. >>>>>>>>>>>>>> ???????? 'jthreads' is allocated but never freed. >>>>>>>>>>>>> >>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>>>>>>>>>>> ???? L91: ? result = (*jvmti)->SuspendThread(jvmti, thread); >>>>>>>>>>>>>> ???????? Why are you suspending the thread? GetAllStackTraces() and >>>>>>>>>>>>>> ???????? GetThreadListStackTraces() do not require the target thread(s) >>>>>>>>>>>>>> ???????? to be suspend. >>>>>>>>>>>>>> >>>>>>>>>>>>>> ???????? If you decide not to SuspendThread, then you don't need the >>>>>>>>>>>>>> ???????? AddCapabilities or the ResumeThread calls. >>>>>>>>>>>>> >>>>>>>>>>>>> Test thread might not be entered following code (stopSignal.await()). We might see deferent call stack between GetAllStackTraces() and GetThreadListStackTraces(). We cannot control to freeze call stack of test thread in Java code. >>>>>>>>>>>>> (I didn't use SuspendThread() at first, but I saw some errors which causes in above.) >>>>>>>>>>>>> >>>>>>>>>>>>> So we need to call SuspendThread() to ensure we can see same call stack. >>>>>>>>>>>> >>>>>>>>>>>> If you are checking that the thread is in state WAITING then it cannot escape from that state and you can sample the stack multiple times from any API and get the same result. >>>>>>>>>>>> >>>>>>>>>>>> I suspect the errors you saw were from the apparent incorrect use of the CountDownLatch. >>>>>>>>>>> >>>>>>>>>>> With the flow outlined above, the worker thread should be >>>>>>>>>>> nicely blocked in stopSignal.await() when stuff is sampled. >>>>>>>>>>> >>>>>>>>>>> Dan >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Cheers, >>>>>>>>>>>> David >>>>>>>>>>>> ----- >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> Dan >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I uploaded new webrev. Could review again? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Updates look fine - thanks. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> One minor nit: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 1274 _collector.allocate_and_fill_stacks(1); >>>>>>>>>>>>>>>> 1275 _collector.set_result(JVMTI_ERROR_NONE); >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> In the other places where you use _collector you rely on result being initialized to JVMTI_ERROR_NONE, rather than setting it directly after allocate_and_fill_stacks(). >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> ? 820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>> ? 821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>>>>>>>>>> ? 822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>> ? 823????????? "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> This function (JvmtiEnvBase::get_stack_trace()) can be called to get own stack trace. For example, we can call GetStackTrace() for current thread at JVMTI event. >>>>>>>>>>>>>>>>> So I changed assert as below: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>> ??820?? assert(current_thread == java_thread || >>>>>>>>>>>>>>>>> ??821 SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>> ??822????????? current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>> ??823????????? "call by myself / at safepoint / at handshake"); >>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Yep good catch. I hope current tests caught that. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> They would be tested in vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ (own call stacks), and getstacktr003 (call stacks in other thread). >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Speaking of tests ... >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> In the native code I think you need to check the success of all JNI methods that can throw exceptions - otherwise I believe the tests may trigger warnings if -Xcheck:jni is used with them. See for example: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I updated testcases to check JNI and JVMTI function calls. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> In the Java code the target thread: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> ?? 45???? public void run() { >>>>>>>>>>>>>>>> ?? 46?????? try { >>>>>>>>>>>>>>>> ?? 47?????????? synchronized (lock) { >>>>>>>>>>>>>>>> ?? 48?????????????? lock.wait(); >>>>>>>>>>>>>>>> ?? 49 System.out.println("OK"); >>>>>>>>>>>>>>>> ?? 50?????????? } >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> is potentially susceptible to spurious wakeups. Using a CountDownLatch would be robust. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> If so, we can call make_local() in L1272 without JavaThread (or we can pass current thread to make_local()). Is it right? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Sorry I got confused, _calling_thread may not be the current thread as we could be executing the handshake in the target thread itself. So the ResourceMark is correct as-is (implicitly for current thread). >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> The argument to fill_frames will be used in the jvmtiStackInfo and passed back to the _calling_thread, so it must be created via make_local(_calling_thread, ...) as you presently have. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thank you for reviewing! I will update new webrev tomorrow. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> ? 498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>>>>>>>>>>> ? 500?? JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>> ? 502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>>>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>>>>>>>>>>>> ??? : >>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> class GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>> ?? private: >>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I updated webrev for 8242428. Could you review again? >>>>>>>>>>>>>>>>>>>>>>> This change migrate to use direct handshake for GetStackTrace() and GetThreadListStackTraces() (when thread_count == 1). >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> This looks really good now! I only have a few nits below. There is one thing I don't like about it but it requires a change to the main Handshake logic to address - in JvmtiEnv::GetThreadListStackTraces you have to create a ThreadsListHandle to convert the jthread to a JavaThread, but then the Handshake::execute_direct creates another ThreadsListHandle internally. That's a waste. I will discuss with Robbin and file a RFE to have an overload of execute_direct that takes an existing TLH. Actually it's worse than that because we have another TLH in use at the entry point for the JVMTI functions, so I think there may be some scope for simplifying the use of TLH instances - future RFE. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> ??451 GetStackTraceClosure(JvmtiEnv *env, jint start_depth, jint max_count, >>>>>>>>>>>>>>>>>>>>>> ??452 jvmtiFrameInfo* frame_buffer, jint* count_ptr) >>>>>>>>>>>>>>>>>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>>>>>>>>>>> ??454?????? _env(env), _start_depth(start_depth), _max_count(max_count), >>>>>>>>>>>>>>>>>>>>>> ??455 _frame_buffer(frame_buffer), _count_ptr(count_ptr), >>>>>>>>>>>>>>>>>>>>>> ??456 _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Nit: can you do one initializer per line please. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> ??498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>>>>>>>>>>> ??500?? JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>> ??502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> 481 MultipleStackTracesCollector(JvmtiEnv *env, jint max_frame_count) { >>>>>>>>>>>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>>>>>>>>>>> ??483???? _max_frame_count = max_frame_count; >>>>>>>>>>>>>>>>>>>>>> ??484 _frame_count_total = 0; >>>>>>>>>>>>>>>>>>>>>> ??485???? _head = NULL; >>>>>>>>>>>>>>>>>>>>>> ??486???? _stack_info = NULL; >>>>>>>>>>>>>>>>>>>>>> ??487???? _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> As you are touching this can you change it to use an initializer list as you did for the HandshakeClosure, and please keep one item per line. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> ??820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>> ??821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>> ??823????????? "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>>>>>>>>>>>>>> 1269 >>>>>>>>>>>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> It is frustrating that this entire call chain started with a jthread reference, which we converted to a JavaThread, only to eventually need to convert it back to a jthread! I think there is some scope for simplification here but not as part of this change. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I'm still pondering the test logic but wanted to send this now. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>> VM_GetThreadListStackTrace (for GetThreadListStackTraces) and VM_GetAllStackTraces (for GetAllStackTraces) have inherited VM_GetMultipleStackTraces VM operation which provides the feature to generate jvmtiStackInfo. I modified VM_GetMultipleStackTraces to a normal C++ class to share with HandshakeClosure for GetThreadListStackTraces (GetSingleStackTraceClosure). >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Also I added new testcases which test GetThreadListStackTraces() with thread_count == 1 and with all threads. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/nsk/jdwp. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>>>>>>>>>>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> This change replace following VM operations to direct handshake. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces (GetThreadListStackTrace()) >>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTrace() uses direct handshake if thread count == 1. In other case (thread count > 1), it would be performed as VM operation (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>>>>>>>>>>> Caller of VM_GetCurrentLocation (JvmtiEnvThreadState::reset_current_location()) might be called at safepoint. So I added safepoint check in its caller. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/ns >>>>>>>>>>>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Also I tested it on submit repo, then it has execution error (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) due to dependency error. So I think it does not occur by this change. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>> >>>>>>>>>>> >>>>>>> >>>>> >>> > From chris.plummer at oracle.com Mon Jul 6 15:52:00 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 6 Jul 2020 08:52:00 -0700 Subject: RFR(S): 8247762: [aarch64] Timeout in .../HeapDumpTestWithActiveProcess.java due to inf. loop in AARCH64CurrentFrameGuess.run() In-Reply-To: <36c12487-cfc9-3823-c4a3-7605629beca5@redhat.com> References: <36c12487-cfc9-3823-c4a3-7605629beca5@redhat.com> Message-ID: <78a1d527-bd3f-6bc2-7880-ca5f34215267@oracle.com> On 7/6/20 1:37 AM, Andrew Haley wrote: > On 05/07/2020 16:26, Patric Hedlin wrote: >> Issue:? https://bugs.openjdk.java.net/browse/JDK-8247762 >> Webrev: http://cr.openjdk.java.net/~phedlin/tr8247762/ >> >> >> AARCH64CurrentFrameGuess.run() may loop indefinitely in a bad >> stack-walk. This is JDK-8231635 applied to AArch64. > 141 Frame oldFrame = frame; > 142 frame = frame.sender(map); > 143 if (frame.getSP().lessThanOrEqual(oldFrame.getSP())) { > 144 // Frame points to itself or to a location in the wrong direction. > 145 // Break the loop and move on to next offset. > 146 if (DEBUG) { > 147 System.out.println("CurrentFrameGuess: frame <= oldFrame: " + frame); > 148 } > 149 break; > 150 } > 151 } > > OK, that looks like a reasonable thing to do, but I would wonder how the stack got > into that mess. > Hi Patric, The changes look good to me. Andrew, The problem is not the stack per se. AARCH64CurrentFrameGuess.run() tries to find the "current frame". It starts with the specified SP (which I believe comes from the SP register), and validates that it represents the current frame by using it to walk the stack until the first entry frame is found. If it doesn't find it, then it increments SP by a word and tries again. It does this until it either can successfully walk to the first entry frame, or SP leaves the range it is willing to search, at which point it gives up. During this search all manner of bad addresses can be accessed. This is why there is an exception handler that when triggered simply moves on to the next SP to check. So it's not at all surprising that on occasion a bad SP results in frame->sender() pointing to a frame that was already visited. thanks, Chris From hohensee at amazon.com Mon Jul 6 19:03:18 2020 From: hohensee at amazon.com (Hohensee, Paul) Date: Mon, 6 Jul 2020 19:03:18 +0000 Subject: [11u] RFR/A: 8231209: [REDO] JDK-8207266 ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: <09229907-C812-4974-BF5D-8619FEA083BC@amazon.com> References: <09229907-C812-4974-BF5D-8619FEA083BC@amazon.com> Message-ID: The CSRs have been approved, and the JBS issues tagged (8231209, 8231968 and the corresponding backport issues 8247806 and 8247809). Also, Oracle has now committed to backport these to their 11.0.9. Please approve. Thanks, Paul From: serviceability-dev on behalf of "Hohensee, Paul" Date: Thursday, June 18, 2020 at 12:29 PM To: "jdk-updates-dev at openjdk.java.net" Cc: serviceability-dev Subject: [11u] RFR/A: 8231209: [REDO] JDK-8207266 ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread This request is for a pair of backports. 8231209 is the first (and primary), 8231968 is a minor cleanup. There are CSR?s for both. The effect is to add a convenience method getCurrentThreadAllocatedBytes() to com.sun.management.ThreadMXBean that can be implemented more efficiently than the equivalent getThreadAllocatedBytes(long id), and to make the implementation of getThreadAllocatedBytes(long id) and getThreadAllocatedBytes(long[] id) more efficient. These methods are heavily used by heap profiling tools, including Amazon?s, and their efficiency is important to us. There is no effect on the TCK because com.sun.management is a platform-specific package. See the CSRs for more detail. The patches apply cleanly (in sequence) to 11u, but I?m posting a review/approval request because the backport CSRs need approval. Once the backport CSRs are reviewed, finalized, and approved, l can tag 8231209 and 8231968. Tested with hotspot/jtreg/vmTestbase/nsk/monitoring jdk/com/sun/management jdk/jdk/jfr/event/runtime The same tests pass/fail as with unpatched jdk11u-dev. JDK-8231209: [REDO] JDK-8207266 ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread Original RFE: https://bugs.openjdk.java.net/browse/JDK-8231209 Original Patch: https://hg.openjdk.java.net/jdk/jdk/rev/c29e49148be7 Original CSR: https://bugs.openjdk.java.net/browse/JDK-8231374 Original review thread: https://mail.openjdk.java.net/pipermail/serviceability-dev/2019-September/029208.html Backport RFE: https://bugs.openjdk.java.net/browse/JDK-8247806 Backport CSR: https://bugs.openjdk.java.net/browse/JDK-8247807 JDK-8231968: getCurrentThreadAllocatedBytes default implementation s/b getThreadAllocatedBytes Original RFE: https://bugs.openjdk.java.net/browse/JDK-8231968 Original Patch: https://hg.openjdk.java.net/jdk/jdk/rev/5bb426e9acc4 Original CSR: https://bugs.openjdk.java.net/browse/JDK-8232072 Original review thread: https://mail.openjdk.java.net/pipermail/serviceability-dev/2019-October/029659.html Backport RFE: https://bugs.openjdk.java.net/browse/JDK-8247809 Backport CSR: https://bugs.openjdk.java.net/browse/JDK-8247810 Thanks, Paul -------------- next part -------------- An HTML attachment was scrubbed... URL: From patric.hedlin at oracle.com Mon Jul 6 19:54:40 2020 From: patric.hedlin at oracle.com (Patric Hedlin) Date: Mon, 6 Jul 2020 21:54:40 +0200 Subject: RFR(S): 8247762: [aarch64] Timeout in .../HeapDumpTestWithActiveProcess.java due to inf. loop in AARCH64CurrentFrameGuess.run() In-Reply-To: <78a1d527-bd3f-6bc2-7880-ca5f34215267@oracle.com> References: <36c12487-cfc9-3823-c4a3-7605629beca5@redhat.com> <78a1d527-bd3f-6bc2-7880-ca5f34215267@oracle.com> Message-ID: <6322edbb-ab38-4956-6a29-5b1c3d48c132@oracle.com> Thanks Chris, for review and laying out the text. Andrew, Something you can live with? /Patric On 2020-07-06 17:52, Chris Plummer wrote: > On 7/6/20 1:37 AM, Andrew Haley wrote: >> On 05/07/2020 16:26, Patric Hedlin wrote: >>> Issue: https://bugs.openjdk.java.net/browse/JDK-8247762 >>> Webrev: http://cr.openjdk.java.net/~phedlin/tr8247762/ >>> >>> >>> AARCH64CurrentFrameGuess.run() may loop indefinitely in a bad >>> stack-walk. This is JDK-8231635 applied to AArch64. >> ? 141?????????????? Frame oldFrame = frame; >> ? 142?????????????? frame = frame.sender(map); >> ? 143?????????????? if >> (frame.getSP().lessThanOrEqual(oldFrame.getSP())) { >> ? 144???????????????? // Frame points to itself or to a location in >> the wrong direction. >> ? 145???????????????? // Break the loop and move on to next offset. >> ? 146???????????????? if (DEBUG) { >> ? 147?????????????????? System.out.println("CurrentFrameGuess: frame >> <= oldFrame: " + frame); >> ? 148???????????????? } >> ? 149???????????????? break; >> ? 150?????????????? } >> ? 151???????????? } >> >> OK, that looks like a reasonable thing to do, but I would wonder how >> the stack got >> into that mess. >> > Hi Patric, > > The changes look good to me. > > Andrew, > > The problem is not the stack per se. AARCH64CurrentFrameGuess.run() > tries to find the "current frame". It starts with the specified SP > (which I believe comes from the SP register), and validates that it > represents the current frame by using it to walk the stack until the > first entry frame is found. If it doesn't find it, then it increments > SP by a word and tries again. It does this until it either can > successfully walk to the first entry frame, or SP leaves the range it > is willing to search, at which point it gives up. During this search > all manner of bad addresses can be accessed. This is why there is an > exception handler that when triggered simply moves on to the next SP > to check. So it's not at all surprising that on occasion a bad SP > results in frame->sender() pointing to a frame that was already visited. > > thanks, > > Chris > From hohensee at amazon.com Mon Jul 6 21:57:10 2020 From: hohensee at amazon.com (Hohensee, Paul) Date: Mon, 6 Jul 2020 21:57:10 +0000 Subject: RFR(L): 8215624: add parallel heap inspection support for jmap histo(G1)(Internet mail) Message-ID: <02415944-2238-4430-A7C2-3625264A5505@amazon.com> I'd like to see this feature added. :) The CSR looks good, as does the basic parallel inspection algorithm. Stefan's done the GC part, so I'll stick to the non-GC part (fwiw, the GC part lgtm). I'd move all the argument parsing code to JMap.java and just pass the results to Hotspot. Both histo() in JMap.java and code in attachListener.* parse the command line arguments, though the code in histo() doesn't parse the argument to "parallel". I'd upgrade the code in histo() to do a complete parse and pass the option values to executeCommandForPid as before: there would just be more of them now. That would allow you to eliminate all the parsing code in attachListener.cpp as well as the change to arguments.hpp. heapInspection.hpp: _shared_miss_count (s/b _missed_count, see below) isn't a size, so it should be a uint instead of a size_t. Same with the new parallel_thread_num argument to heap_inspection() and populate_table(). Comment copy-edit: +// Parallel heap inspection task. Parallel inspection can fail due to +// a native OOM when allocating memory for TL-KlassInfoTable. +// _success will be set false on an OOM, and serial inspection tried. _shared_miss_count should be _missed_count to match the missed_count() getter, or rename missed_count() to be shared_miss_count(). Whichever way you go, the field type should match the getter result type: uint is reasonable. heapInspection.cpp: You might use ResourceMark twice in populate_table, separately for the parallel attempt and the serial code. If the parallel attempt fails and available memory is low, it would be good to clean up the memory used by the parallel attempt before doing the serial code. Style nit in KlassInfoTable::merge_entry(). I'd line up the definitions of k and elt, so "k" is even with "elt". And, because it's two lines shorter, I'd replace + } else { + return false; + } with + return false; KlassInfoTableMergeClosure.is_success() should be just success() (i.e., no "is_" prefix) because it's a getter. I'd reorganize the code in populate_table() to make it more clear, vis (I changed _shared_missed_count to _missed_count) + if (cit.allocation_failed()) { + // fail to allocate memory, stop parallel mode + Atomic::store(&_success, false); + return; + } + RecordInstanceClosure ric(&cit, _filter); + _poi->object_iterate(&ric, worker_id); + missed_count = ric.missed_count(); + { + MutexLocker x(&_mutex); + merge_success = _shared_cit->merge(&cit); + } + if (merge_success) { + Atomic::add(&_missed_count, missed_count); + else { + Atomic::store(&_success, false); + } Thanks, Paul ?On 6/29/20, 7:20 PM, "linzang(??)" wrote: Dear All, Sorry to bother again, I just want to make sure that is this change worth to be continue to work on? If decision is made to not. I think I can drop this work and stop asking for help reviewing... Thanks for all your help about reviewing this previously. BRs, Lin On 2020/5/9, 3:47 PM, "linzang(??)" wrote: Dear All, May I ask your help again for review the latest change? Thanks! BRs, Lin On 2020/4/28, 1:54 PM, "linzang(??)" wrote: Hi Stefan, >> - Adding Atomic::load/store. >> - Removing the time measurement in the run_task. I renamed G1's function >> to run_task_timed. If we need this outside of G1, we can rethink the API >> at that point. >> - ZGC style cleanups Thanks for revising the patch, they are all good to me, and I have made a tiny change based on it: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04/ http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04-delta/ it reduce the scope of mutex in ParHeapInspectTask, and delete unnecessary comments. BRs, Lin On 2020/4/27, 4:34 PM, "Stefan Karlsson" wrote: Hi Lin, On 2020-04-26 05:10, linzang(??) wrote: > Hi Stefan and Paul? > I have made a new patch based on your comments and Stefan's Poc code: > Webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03/ > Delta(based on Stefan's change:) : http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03-delta/webrev_03-delta/ Thanks for providing a delta patch. It makes it much easier to look at, and more likely for reviewers to continue reviewing. I'm going to continue focusing on the GC parts, and leave the rest to others to review. > > And Here are main changed I made and want to discuss with you: > 1. changed"parallelThreadNum=" to "parallel=" for jmap -histo options. > 2. Add logic to test where parallelHeapInspection is fail, in heapInspection.cpp > This is because the parHeapInspectTask create thread local KlassInfoTable in it's work() method, and this may fail because of native OOM, in this case, the parallel should fail and serial heap inspection can be tried. > One more thing I want discuss with you is about the member "_success" of parHeapInspectTask, when native OOM happenes, it is set to false. And since this "set" operation can be conducted in multiple threads, should it be atomic ops? IMO, this is not necessary because "_success" can only be set to false, and there is no way to change it from back to true after the ParHeapInspectTask instance is created, so it is save to be non-atomic, do you agree with that? In these situations you should be using the Atomic::load/store primitives. We're moving toward a later C++ standard were data races are considered undefined behavior. > 3. make CollectedHeap::run_task() be an abstract virtual func, so that every subclass of collectedHeap should support it, so later implementation of new collectedHeap will not miss the "parallel" features. > The problem I want to discuss with you is about epsilonHeap and SerialHeap, as they may not need parallel heap iteration, so I only make task->work(0), in case the run_task() is invoked someway in future. Another way is to left run_task() unimplemented, which one do you think is better? I don't have a strong opinion about this. And also please help take a look at the zHeap, as there is a class zTask that wrap the abstractGangTask, and the collectedHeap::run_task() only accept AbstraceGangTask* as argument, so I made a delegate class to adapt it , please see src/hotspot/share/gc/z/zHeap.cpp. > > There maybe other better ways to sovle the above problems, welcome for any comments, Thanks! I've created a few cleanups and changes on top of your latest patch: https://cr.openjdk.java.net/~stefank/8215624/webrev.02.delta https://cr.openjdk.java.net/~stefank/8215624/webrev.02 - Adding Atomic::load/store. - Removing the time measurement in the run_task. I renamed G1's function to run_task_timed. If we need this outside of G1, we can rethink the API at that point. - ZGC style cleanups Thanks, StefanK > > BRs, > Lin > > On 2020/4/23, 11:08 AM, "linzang(??)" wrote: > > Thanks Paul! I agree with using "parallel", will make the update in next patch, Thanks for help update the CSR. > > BRs, > Lin > > On 2020/4/23, 4:42 AM, "Hohensee, Paul" wrote: > > For the interface, I'd use "parallel" instead of "parallelThreadNum". All the other options are lower case, and it's a lot easier to type "parallel". I took the liberty of updating the CSR. If you're ok with it, you might want to change variable names and such, plus of course JMap.usage. > > Thanks, > Paul > > On 4/22/20, 2:29 AM, "serviceability-dev on behalf of linzang(??)" wrote: > > Dear Stefan, > > Thanks a lot! I agree with you to decouple the heap inspection code with GC's. > I will start from your POC code, may discuss with you later. > > > BRs, > Lin > > On 2020/4/22, 5:14 PM, "Stefan Karlsson" wrote: > > Hi Lin, > > I took a look at this earlier and saw that the heap inspection code is > strongly coupled with the CollectedHeap and G1CollectedHeap. I'd prefer > if we'd abstract this away, so that the GCs only provide a "parallel > object iteration" interface, and the heap inspection code is kept elsewhere. > > I started experimenting with doing that, but other higher-priority (to > me) tasks have had to take precedence. > > I've uploaded my work-in-progress / proof-of-concept: > https://cr.openjdk.java.net/~stefank/8215624/webrev.01.delta/ > https://cr.openjdk.java.net/~stefank/8215624/webrev.01/ > > The current code doesn't handle the lifecycle (deletion) of the > ParallelObjectIterators. There's also code left unimplemented in around > CollectedHeap::run_task. However, I think this could work as a basis to > pull out the heap inspection code out of the GCs. > > Thanks, > StefanK > > On 2020-04-22 02:21, linzang(??) wrote: > > Dear all, > > May I ask you help to review? This RFR has been there for quite a while. > > Thanks! > > > > BRs, > > Lin > > > > > On 2020/3/16, 5:18 PM, "linzang(??)" wrote:> > > > >> Just update a new path, my preliminary measure show about 3.5x speedup of jmap histo on a nearly full 4GB G1 heap (8-core platform with parallel thread number set to 4). > >> webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_02/ > >> bug: https://bugs.openjdk.java.net/browse/JDK-8215624 > >> CSR: https://bugs.openjdk.java.net/browse/JDK-8239290 > >> BRs, > >> Lin > >> > On 2020/3/2, 9:56 PM, "linzang(??)" wrote: > >> > > >> > Dear all, > >> > Let me try to ease the reviewing work by some explanation :P > >> > The patch's target is to speed up jmap -histo for heap iteration, from my experience it is necessary for large heap investigation. E.g in bigData scenario I have tried to conduct jmap -histo against 180GB heap, it does take quite a while. > >> > And if my understanding is corrent, even the jmap -histo without "live" option does heap inspection with heap lock acquired. so it is very likely to block mutator thread in allocation-sensitive scenario. I would say the faster the heap inspection does, the shorter the mutator be blocked. This is parallel iteration for jmap is necessary. > >> > I think the parallel heap inspection should be applied to all kind of heap. However, consider the heap layout are different for GCs, much time is required to understand all kinds of the heap layout to make the whole change. IMO, It is not wise to have a huge patch for the whole solution at once, and it is even harder to review it. So I plan to implement it incrementally, the first patch (this one) is going to confirm the implemention detail of how jmap accept the new option, passes it to attachListener of the jvm process and then how to make the parallel inspection closure be generic enough to make it easy to extend to different heap layout. And also how to implement the heap inspection in specific gc's heap. This patch use G1's heap as the begining. > >> > This patch actually do several things: > >> > 1. Add an option "parallelThreadNum=" to jmap -histo, the default behavior is to set N to 0, means let's JVM decide how many threads to use for heap inspection. Set this option to 1 will disable parallel heap inspection. (more details in CSR: https://bugs.openjdk.java.net/browse/JDK-8239290) > >> > 2. Make a change in how Jmap passing arguments, changes in http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java.udiff.html, originally it pass options as separate arguments to attachListener, this patch change to that all options be compose to a single string. So the arg_count_max in attachListener.hpp do not need to be changed, and hence avoid the compatibility issue, as disscussed at https://mail.openjdk.java.net/pipermail/serviceability-dev/2019-March/027334.html > >> > 3. Add an abstract class ParHeapInspectTask in heapInspection.hpp / heapInspection.cpp, It's work(uint worker_id) method prepares the data structure (KlassInfoTable) need for every parallel worker thread, and then call do_object_iterate_parallel() which is heap specific implementation. I also added some machenism in KlassInfoTable to support parallel iteration, such as merge(). > >> > 4. In specific heap (G1 in this patch), create a subclass of ParHeapInspectTask, implement the do_object_iterate_parallel() for parallel heap inspection. For G1, it simply invoke g1CollectedHeap's object_iterate_parallel(). > >> > 5. Add related test. > >> > 6. it may be easy to extend this patch for other kinds of heap by creating subclass of ParHeapInspectTask and implement the do_object_iterate_parallel(). > >> > > >> > Hope these info could help on code review and initate the discussion :-) > >> > Thanks! > >> > > >> > BRs, > >> > Lin > >> > >On 2020/2/19, 9:40 AM, "linzang(??)" wrote:. > >> > > > >> > > Re-post this RFR with correct enhancement number to make it trackable. > >> > > please ignore the previous wrong post. sorry for troubles. > >> > > > >> > > webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/ > >> > > Hi bug: https://bugs.openjdk.java.net/browse/JDK-8215624 > >> > > CSR: https://bugs.openjdk.java.net/browse/JDK-8239290 > >> > > -------------- > >> > > Lin > >> > > >Hi Lin, > > > > > > > >> > > >Could you, please, re-post your RFR with the right enhancement number in > >> > > >the message subject? > >> > > >It will be more trackable this way. > >> > > > > >> > > >Thanks, > >> > > >Serguei > >> > > > > >> > > > > >> > > >On 2/17/20 10:29 PM, linzang(??) wrote: > >> > > >> Dear David, > >> > > >> Thanks a lot! > >> > > >> I have updated the refined code to http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_01/. > >> > > >> IMHO the parallel heap inspection can be extended to all kinds of heap as long as the heap layout can support parallel iteration. > >> > > >> Maybe we can firstly use this webrev to discuss how to implement it, because I am not sure my current implementation is an appropriate way to communicate with collectedHeap, then we can extend the solution to other kinds of heap. > >> > > >> > >> > > >> Thanks, > >> > > >> -------------- > >> > > >> Lin > >> > > >>> Hi Lin, > >> > > >>> > >> > > >>> Adding in hotspot-gc-dev as they need to see how this interacts with GC > >> > > >>> worker threads, and whether it needs to be extended beyond G1. > >> > > >>> > >> > > >>> I happened to spot one nit when browsing: > >> > > >>> > >> > > >>> src/hotspot/share/gc/shared/collectedHeap.hpp > >> > > >>> > >> > > >>> + virtual bool run_par_heap_inspect_task(KlassInfoTable* cit, > >> > > >>> + BoolObjectClosure* filter, > >> > > >>> + size_t* missed_count, > >> > > >>> + size_t thread_num) { > >> > > >>> + return NULL; > >> > > >>> > >> > > >>> s/NULL/false/ > >> > > >>> > >> > > >>> Cheers, > >> > > >>> David > > > > > >>> > >> > > >>> On 18/02/2020 2:15 pm, linzang(??) wrote: > >> > > >>>> Dear All, > >> > > >>>> May I ask your help to review the follow changes: > >> > > >>>> webrev: > >> > > >>>> http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_00/ > >> > > >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8215624 > >> > > >>>> related CSR: https://bugs.openjdk.java.net/browse/JDK-8239290 > >> > > >>>> This patch enable parallel heap inspection of G1 for jmap histo. > >> > > >>>> my simple test shown it can speed up 2x of jmap -histo with > >> > > >>>> parallelThreadNum set to 2 for heap at ~500M on 4-core platform. > >> > > >>>> > >> > > >>>> ------------------------------------------------------------------------ > >> > > >>>> BRs, > >> > > >>>> Lin > >> > > >> > > >> > > > > > > > > > > > > > > > From leonid.mesnik at oracle.com Mon Jul 6 22:27:36 2020 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Mon, 6 Jul 2020 15:27:36 -0700 Subject: RFR (L): 8248194: Need better support for running SA tests on core files In-Reply-To: <5760b18e-7ba2-2fdd-94af-1e555cef74cc@oracle.com> References: <5760b18e-7ba2-2fdd-94af-1e555cef74cc@oracle.com> Message-ID: Hi The changes look good for me. Leonid > On Jul 2, 2020, at 7:21 AM, Chris Plummer wrote: > > [Note, the following changes only impact serviceability tests, but I am adding some test library code to assist with creating and finding core files, and I thought others on hotspot-dev might have an interest in that.] > > Hello, > > Please help review the following changes to add better support for writing SA tests that work on core files: > > https://bugs.openjdk.java.net/browse/JDK-8248194 > http://cr.openjdk.java.net/~cjplummer/8248194/webrev.01/index.html > > As outlined in the CR, these are the 3 main goals of this CR: > > 1. Add a shared API for locating the path to the core file. This includes parsing the output of the crashed process to locate where the core file was saved, and returning this location to the user. This API will be placed in the new CoreUtils class. > > 2. Add a shared API to support adding the "ulimit -c unlimited" prefix to the command that will produce the core file, allowing the overriding of any lower limit so we can be sure the core file will be produced. This API will also be placed in the new CoreUtils class. > > 3. LingeredApp should include support for producing a core file. > > As proof of concept for these improvements, I'm updating the following 3 tests to make use of them: > > ClhsdbCDSCore.java and TestJmapCore.java: Use the CoreUtils support listed above. These tests do not use LingeredApp, so those improvements don't apply. > > ClhsdbFindPC.java: Use all the above new features, including having LingeredApp produce a core file. This is the only test modified to start testing on core files that didn't previously do so. It still also tests on a live process. > > In the future more Clhsdb tests will be converted to work on core files in a manner similar to ClhsdbFindPC. > > The new CoreUtils code is borrowed from (more like ripped out of) ClhsdbCDSCore.java and TestJmapCore.java. They both had a lot of code dedicated to finding the core file and also applying "ulimit -c unlimited", but didn't do so in quite the same way. Now both these tests share code in CoreUtils.java. One thing I did drop is TestJmapCore.java use of ":KILLED_PID" in the output to help find the core file. It's no longer necessary based on the smarter core locating code I pulled from ClhsdbCDSCore.java. > > One other improvement was to enable windows support for ClhsdbCDSCore. The only reason it was not enabled previously is because the author couldn't figure out how to properly generate the command for the process that produces the core. Since it is launched using "sh -c", the path has to be converted to use forward slashes. This is now done in CoreUtils. > > One other change in ClhsdbCDSCore is that it no longer renames the core file to a well known name in the cwd. This was unnecessary. It originated in code from ciReplayBase.java, which does have a reason to do the rename, but ClhsdbCDSCore does not. > > The new libLingeredApp.c relies on JDK-8248667 [1] being in place in order to have the build system properly compile it. JDK-8248667 will be reviewed separately on build-dev and pushed just before the changes for this CR. > > thanks, > > Chris > > [1] https://bugs.openjdk.java.net/browse/JDK-8248667 From chris.plummer at oracle.com Mon Jul 6 22:41:21 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 6 Jul 2020 15:41:21 -0700 Subject: RFR (L): 8248194: Need better support for running SA tests on core files In-Reply-To: References: <5760b18e-7ba2-2fdd-94af-1e555cef74cc@oracle.com> Message-ID: Thanks, Leonid! On 7/6/20 3:27 PM, Leonid Mesnik wrote: > Hi > > The changes look good for me. > > Leonid > >> On Jul 2, 2020, at 7:21 AM, Chris Plummer wrote: >> >> [Note, the following changes only impact serviceability tests, but I am adding some test library code to assist with creating and finding core files, and I thought others on hotspot-dev might have an interest in that.] >> >> Hello, >> >> Please help review the following changes to add better support for writing SA tests that work on core files: >> >> https://bugs.openjdk.java.net/browse/JDK-8248194 >> http://cr.openjdk.java.net/~cjplummer/8248194/webrev.01/index.html >> >> As outlined in the CR, these are the 3 main goals of this CR: >> >> 1. Add a shared API for locating the path to the core file. This includes parsing the output of the crashed process to locate where the core file was saved, and returning this location to the user. This API will be placed in the new CoreUtils class. >> >> 2. Add a shared API to support adding the "ulimit -c unlimited" prefix to the command that will produce the core file, allowing the overriding of any lower limit so we can be sure the core file will be produced. This API will also be placed in the new CoreUtils class. >> >> 3. LingeredApp should include support for producing a core file. >> >> As proof of concept for these improvements, I'm updating the following 3 tests to make use of them: >> >> ClhsdbCDSCore.java and TestJmapCore.java: Use the CoreUtils support listed above. These tests do not use LingeredApp, so those improvements don't apply. >> >> ClhsdbFindPC.java: Use all the above new features, including having LingeredApp produce a core file. This is the only test modified to start testing on core files that didn't previously do so. It still also tests on a live process. >> >> In the future more Clhsdb tests will be converted to work on core files in a manner similar to ClhsdbFindPC. >> >> The new CoreUtils code is borrowed from (more like ripped out of) ClhsdbCDSCore.java and TestJmapCore.java. They both had a lot of code dedicated to finding the core file and also applying "ulimit -c unlimited", but didn't do so in quite the same way. Now both these tests share code in CoreUtils.java. One thing I did drop is TestJmapCore.java use of ":KILLED_PID" in the output to help find the core file. It's no longer necessary based on the smarter core locating code I pulled from ClhsdbCDSCore.java. >> >> One other improvement was to enable windows support for ClhsdbCDSCore. The only reason it was not enabled previously is because the author couldn't figure out how to properly generate the command for the process that produces the core. Since it is launched using "sh -c", the path has to be converted to use forward slashes. This is now done in CoreUtils. >> >> One other change in ClhsdbCDSCore is that it no longer renames the core file to a well known name in the cwd. This was unnecessary. It originated in code from ciReplayBase.java, which does have a reason to do the rename, but ClhsdbCDSCore does not. >> >> The new libLingeredApp.c relies on JDK-8248667 [1] being in place in order to have the build system properly compile it. JDK-8248667 will be reviewed separately on build-dev and pushed just before the changes for this CR. >> >> thanks, >> >> Chris >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8248667 From alexey.menkov at oracle.com Mon Jul 6 23:16:02 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Mon, 6 Jul 2020 16:16:02 -0700 Subject: RFR (L): 8248194: Need better support for running SA tests on core files In-Reply-To: <5760b18e-7ba2-2fdd-94af-1e555cef74cc@oracle.com> References: <5760b18e-7ba2-2fdd-94af-1e555cef74cc@oracle.com> Message-ID: <2f2d3b21-8f8e-3c01-1e33-ad68bd39254b@oracle.com> Hi Chris, Generally looks good to me. CoreUtils.java: 86 String[] cmds; - unused var And one question. You use OutputAnalyzer.getStdout() get extract core file location. I thought it should be printed in stderr. Or we have duplicated stdout and stderr in the case? --alex On 07/02/2020 07:21, Chris Plummer wrote: > [Note, the following changes only impact serviceability tests, but I am > adding some test library code to assist with creating and finding core > files, and I thought others on hotspot-dev might have an interest in that.] > > Hello, > > Please help review the following changes to add better support for > writing SA tests that work on core files: > > https://bugs.openjdk.java.net/browse/JDK-8248194 > http://cr.openjdk.java.net/~cjplummer/8248194/webrev.01/index.html > > As outlined in the CR, these are the 3 main goals of this CR: > > 1. Add a shared API for locating the path to the core file. This > includes parsing the output of the crashed process to locate where the > core file was saved, and returning this location to the user. This API > will be placed in the new CoreUtils class. > > 2. Add a shared API to support adding the "ulimit -c unlimited" prefix > to the command that will produce the core file, allowing the overriding > of any lower limit so we can be sure the core file will be produced. > This API will also be placed in the new CoreUtils class. > > 3. LingeredApp should include support for producing a core file. > > As proof of concept for these improvements, I'm updating the following 3 > tests to make use of them: > > ClhsdbCDSCore.java and TestJmapCore.java: Use the CoreUtils support > listed above. These tests do not use LingeredApp, so those improvements > don't apply. > > ClhsdbFindPC.java: Use all the above new features, including having > LingeredApp produce a core file. This is the only test modified to start > testing on core files that didn't previously do so. It still also tests > on a live process. > > In the future more Clhsdb tests will be converted to work on core files > in a manner similar to ClhsdbFindPC. > > The new CoreUtils code is borrowed from (more like ripped out of) > ClhsdbCDSCore.java and TestJmapCore.java. They both had a lot of code > dedicated to finding the core file and also applying "ulimit -c > unlimited", but didn't do so in quite the same way. Now both these tests > share code in CoreUtils.java. One thing I did drop is TestJmapCore.java > use of ":KILLED_PID" in the output to help find the core file. It's no > longer necessary based on the smarter core locating code I pulled from > ClhsdbCDSCore.java. > > One other improvement was to enable windows support for ClhsdbCDSCore. > The only reason it was not enabled previously is because the author > couldn't figure out how to properly generate the command for the process > that produces the core. Since it is launched using "sh -c", the path has > to be converted to use forward slashes. This is now done in CoreUtils. > > One other change in ClhsdbCDSCore is that it no longer renames the core > file to a well known name in the cwd. This was unnecessary. It > originated in code from ciReplayBase.java, which does have a reason to > do the rename, but ClhsdbCDSCore does not. > > The new libLingeredApp.c relies on JDK-8248667 [1] being in place in > order to have the build system properly compile it. JDK-8248667 will be > reviewed separately on build-dev and pushed just before the changes for > this CR. > > thanks, > > Chris > > [1] https://bugs.openjdk.java.net/browse/JDK-8248667 From chris.plummer at oracle.com Tue Jul 7 00:06:09 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 6 Jul 2020 17:06:09 -0700 Subject: RFR (L): 8248194: Need better support for running SA tests on core files In-Reply-To: <2f2d3b21-8f8e-3c01-1e33-ad68bd39254b@oracle.com> References: <5760b18e-7ba2-2fdd-94af-1e555cef74cc@oracle.com> <2f2d3b21-8f8e-3c01-1e33-ad68bd39254b@oracle.com> Message-ID: <8bd159c8-1ba4-6b59-6f5d-64fd7b3b392f@oracle.com> Hi Alex, Thanks for the review. I'll cleanup the unused var. The crash output is on stdout, not stderr. From VMError::report_and_die(): ? // File descriptor to tty to print an error summary to. ? // Hard wired to stdout; see JDK-8215004 (compatibility concerns). ? static const int fd_out = 1; // stdout thanks, Chris On 7/6/20 4:16 PM, Alex Menkov wrote: > Hi Chris, > > Generally looks good to me. > > CoreUtils.java: > > 86???????? String[] cmds; > - unused var > > And one question. > You use OutputAnalyzer.getStdout() get extract core file location. > I thought it should be printed in stderr. > Or we have duplicated stdout and stderr in the case? > > --alex > > On 07/02/2020 07:21, Chris Plummer wrote: >> [Note, the following changes only impact serviceability tests, but I >> am adding some test library code to assist with creating and finding >> core files, and I thought others on hotspot-dev might have an >> interest in that.] >> >> Hello, >> >> Please help review the following changes to add better support for >> writing SA tests that work on core files: >> >> https://bugs.openjdk.java.net/browse/JDK-8248194 >> http://cr.openjdk.java.net/~cjplummer/8248194/webrev.01/index.html >> >> As outlined in the CR, these are the 3 main goals of this CR: >> >> 1. Add a shared API for locating the path to the core file. This >> includes parsing the output of the crashed process to locate where >> the core file was saved, and returning this location to the user. >> This API will be placed in the new CoreUtils class. >> >> 2. Add a shared API to support adding the "ulimit -c unlimited" >> prefix to the command that will produce the core file, allowing the >> overriding of any lower limit so we can be sure the core file will be >> produced. This API will also be placed in the new CoreUtils class. >> >> 3. LingeredApp should include support for producing a core file. >> >> As proof of concept for these improvements, I'm updating the >> following 3 tests to make use of them: >> >> ClhsdbCDSCore.java and TestJmapCore.java: Use the CoreUtils support >> listed above. These tests do not use LingeredApp, so those >> improvements don't apply. >> >> ClhsdbFindPC.java: Use all the above new features, including having >> LingeredApp produce a core file. This is the only test modified to >> start testing on core files that didn't previously do so. It still >> also tests on a live process. >> >> In the future more Clhsdb tests will be converted to work on core >> files in a manner similar to ClhsdbFindPC. >> >> The new CoreUtils code is borrowed from (more like ripped out of) >> ClhsdbCDSCore.java and TestJmapCore.java. They both had a lot of code >> dedicated to finding the core file and also applying "ulimit -c >> unlimited", but didn't do so in quite the same way. Now both these >> tests share code in CoreUtils.java. One thing I did drop is >> TestJmapCore.java use of ":KILLED_PID" in the output to help find the >> core file. It's no longer necessary based on the smarter core >> locating code I pulled from ClhsdbCDSCore.java. >> >> One other improvement was to enable windows support for >> ClhsdbCDSCore. The only reason it was not enabled previously is >> because the author couldn't figure out how to properly generate the >> command for the process that produces the core. Since it is launched >> using "sh -c", the path has to be converted to use forward slashes. >> This is now done in CoreUtils. >> >> One other change in ClhsdbCDSCore is that it no longer renames the >> core file to a well known name in the cwd. This was unnecessary. It >> originated in code from ciReplayBase.java, which does have a reason >> to do the rename, but ClhsdbCDSCore does not. >> >> The new libLingeredApp.c relies on JDK-8248667 [1] being in place in >> order to have the build system properly compile it. JDK-8248667 will >> be reviewed separately on build-dev and pushed just before the >> changes for this CR. >> >> thanks, >> >> Chris >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8248667 From david.holmes at oracle.com Tue Jul 7 02:31:15 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 7 Jul 2020 12:31:15 +1000 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: <8c387550-a06e-6320-f453-0e898486befb@oss.nttdata.com> References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <763793b1-9506-44c6-0681-1c71c44a44fd@oss.nttdata.com> <5107c478-1de2-7a49-ee71-dfad6a99a667@oracle.com> <90cdfef9-e558-0d25-33f5-25b7d65bb83b@oss.nttdata.com> <0e359819-618f-c17b-a427-bb7e4daa2c58@oracle.com> <3f890bed-6892-e64d-b21b-ca0cbe11784e@oracle.com> <7dc1903b-6295-3864-6301-4c02aacdb971@oss.nttdata.com> <04e09eb7-9483-8f63-4bb4-01c9e814802e@oracle.com> <238a89a3-06b1-90e4-d5a4-8dc2f1ed12ab@oracle.com> <4c43bc2c-551e-d042-8fa7-b63e755888d1@oracle.com> <8c387550-a06e-6320-f453-0e898486befb@oss.nttdata.com> Message-ID: Hi Yasumasa, Hard to keep up with the changes - especially without incremental webrevs. If GetSingleStackTraceClosure also took the jthread as a constructor arg, then you wouldn't need to recreate a JNI local handle when calling _collector.fill_frames. It's a small simplification and not essential at this stage. For the test ... I don't see how Java_GetThreadListStackTraces_checkCallStacks is a valid test. It gets the stacks of all live threads, then uses that information to use GetThreadListStackTraces to get the stack for the same set of threads through a different API. It then compares the two sets of stacks for each thread expecting them to be the same, but that need only be the case for the main thread. Other threads could potentially have a different stack (e.g. if this test is run with JFR enabled there will be additional threads found.) Further I would have expected that there already exist tests that check that, for a given thread (which may be suspended or known to be blocked) the same stack is found through the two different APIs. Thanks, David ----- On 6/07/2020 11:29 pm, Yasumasa Suenaga wrote: > Hi Serguei, > > Thanks for your comment! > > I think C++ is more simple to implement the test agent as you said. > So I implement it in C++ in new webrev. Could you review again? > > ? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.06/ > > Also I refactored libGetThreadListStackTraces.cpp, and I've kept > exception check after IsSameObject(). > > > Thanks, > > Yasumasa > > > On 2020/07/06 16:32, serguei.spitsyn at oracle.com wrote: >> Hi Yasumasa, >> >> Thank you for the update. >> I think, a pending exception after IsSameObject needs to be checked. >> >> The checkStackInfo() needs one more refactoring as I've already >> suggested. >> The body of the loop at L68-L78 should be converted to a function >> check_frame_info. >> The si1->frame_buffer[i] and si2->frame_buffer[i] will be passed as >> fi1 and fi2. >> The index can be passed as well. >> I'm still suggesting to simplify the local exception_msg to something >> shorter like err_msg or exc_msg. >> >> I'm not sure using fatal is right here: >> >> This fragment looks strange: >> >> ? 152???? if ((*env)->IsSameObject(env, stack_info[i].thread, thread)) { >> ? 153?????? target_info = &stack_info[i]; >> ? 154?????? break; >> ? 155???? } else if ((*env)->ExceptionOccurred(env)) { >> ? 156?????? (*env)->ExceptionDescribe(env); >> ? 157?????? (*env)->FatalError(env, __FILE__); >> ? 158???? } >> >> I expected it to be: >> >> ??? jboolean same = (*env)->IsSameObject(env, stack_info[i].thread, >> thread); >> ??? if ((*env)->ExceptionOccurred(env)) { >> ????? (*env)->ExceptionDescribe(env); >> ????? (*env)->FatalError(env, __FILE__); >> ??? } >> ??? if (same) { >> ????? target_info = &stack_info[i]; >> ????? break; >> ??? } >> >> Would it better to port this agent to C++ to simplify this code nicer? >> >> Thanks, >> Serguei >> >> >> On 7/5/20 06:13, Yasumasa Suenaga wrote: >>> Hi Serguei, >>> >>> Thanks for your comment! >>> I refactored testcase. Could you review again? >>> >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.05/ >>> >>> It would check Java exception after IsSameObject() call. Does it need? >>> Any exceptions are not described in JNI document[1], and JNI >>> implementation (jni_IsSameObject()) does not seem to throw it. >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> [1] >>> https://docs.oracle.com/en/java/javase/14/docs/specs/jni/functions.html#issameobject >>> >>> >>> >>> On 2020/07/05 14:46, serguei.spitsyn at oracle.com wrote: >>>> Hi Yasumasa, >>>> >>>> >>>> Okay, thanks. >>>> Then I'm okay to keep the GetSingleStackTraceClosure. >>>> >>>> >>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c.html >>>> >>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c.html >>>> >>>> >>>> I'm not sure the function 'is_same_thread() is needed. >>>> Why do not use the JNI IsSameObject instead? >>>> >>>> It seems to be a typo at L132 and L137. >>>> You, probably. did not want to print the same information for >>>> stack_info_1[i].frame_buffer[j].XXX twice. >>>> >>>> The code at lines 112-142 is not readable. >>>> I'd suggest to make a couple of refactoring steps. >>>> >>>> First step to simplify this a little bit would be with some renaming >>>> and getting rid of indexes: >>>> >>>> ?? 71 char err_msg[EXCEPTION_MSG_LEN] = {0}; >>>> ??... >>>> ??112?? /* Iterate all jvmtiStackInfo to check */ >>>> ??113?? for (i = 0; i < num_threads, *exception_msg != '\0'; i++) { >>>> ????????? jvmtiStackInfo *si1 = stack_info_1[i]; >>>> ????????? jvmtiStackInfo *si2 = stack_info_2[i]; >>>> ??114???? if (!IsSameObject(env, si1.thread, si2.thread)) { /* >>>> jvmtiStackInfo::thread */ >>>> ??115?????? snprintf(err_msg, sizeof(err_msg), >>>> ??116??????????????? "thread[%d] is different: stack_info_1 = %p, >>>> stack_info_2 = %p", >>>> ??117??????????????? i, sinfo1.thread, sinfo2.thread); >>>> ??118???? } else if (si1.state != si2.state) { /* >>>> jvmtiStackInfo::state */ >>>> ??119?????? snprintf(err_msg, sizeof(err_msg), >>>> ??120??????????????? "state[%d] is different: stack_info_1 = %d, >>>> stack_info_2 = %d", >>>> ??121??????????????? i, si1.state, si2.state); >>>> ??122???? } else if (si1.frame_count != si2.frame_count) { /* >>>> jvmtiStackInfo::frame_count */ >>>> ??123?????? snprintf(err_msg, sizeof(err_msg), >>>> ??124??????????????? "frame_count[%d] is different: stack_info_1 = >>>> %d, stack_info_2 = %d", >>>> ??125??????????????? i, si1.frame_count, si2.frame_count); >>>> ??126???? } else { >>>> ??127?????? /* Iterate all jvmtiFrameInfo to check */ >>>> ??128?????? for (j = 0; j < si1.frame_count; j++) { >>>> ??129???????? if (si1.frame_buffer[j].method != >>>> si1.frame_buffer[j].method) { /* jvmtiFrameInfo::method */ >>>> ??130?????????? snprintf(err_msg, sizeof(err_msg), >>>> ??131??????????????????? "thread [%d] frame_buffer[%d].method is >>>> different: stack_info_1 = %lx, stack_info_2 = %lx", >>>> ??132??????????????????? i, j, si1.frame_buffer[j].method, >>>> si2.frame_buffer[j].method); >>>> ??133?????????? break; >>>> ??134???????? } else if (si1.frame_buffer[j].location != >>>> si1.frame_buffer[j].location) { /* jvmtiFrameInfo::location */ >>>> ??135?????????? snprintf(err_msg, sizeof(err_msg), >>>> ??136??????????????????? "thread [%d] frame_buffer[%d].location is >>>> different: stack_info_1 = %ld, stack_info_2 = %ld", >>>> ??137??????????????????? i, j, si1.frame_buffer[j].location, >>>> si2.frame_buffer[j].location); >>>> ??138?????????? break; >>>> ??139???????? } >>>> ??140?????? } >>>> ??141???? } >>>> ??142?? } >>>> >>>> Another step would be to create functions that implement a body of >>>> each loop. >>>> You can use the same techniques to simplify similar place >>>> (L127-L138) in the libOneGetThreadListStackTraces.c. >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 7/3/20 15:55, Yasumasa Suenaga wrote: >>>>> Hi Serguei, >>>>> >>>>> I'm not an Oracle employee, so I cannot know real request(s) from >>>>> your customers. >>>>> However JDK-8201641 says Dynatrace has requested this enhancement. >>>>> >>>>> BTW I haven't heared any request from my customers about this. >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2020/07/04 4:32, serguei.spitsyn at oracle.com wrote: >>>>>> Hi Yasumasa, >>>>>> >>>>>> This difference is not that big to care about. >>>>>> I feel this is really rare case and so, does not worth these >>>>>> complications. >>>>>> Do we have a real request from customers to optimize it? >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>>> >>>>>> On 7/3/20 01:16, Yasumasa Suenaga wrote: >>>>>>> Hi Serguei, >>>>>>> >>>>>>> Generally I agree with you, but I have concern about the >>>>>>> difference of the result of GetStackTrace() and >>>>>>> GetThreadListStackTraces(). >>>>>>> >>>>>>> ? GetStackTrace: jvmtiFrameInfo >>>>>>> ? GetThreadListStackTraces: jvmtiStackInfo >>>>>>> >>>>>>> jvmtiStackInfo contains thread state, and it is ensured it is the >>>>>>> state of the call stack. >>>>>>> If we want to get both call stack and thread state, we need to >>>>>>> suspend target thread, and call both GetStackTrace() and >>>>>>> GetThreadState(). Is it ok? >>>>>>> >>>>>>> I was wondering if JDK-8201641 (parent ticket of this change) >>>>>>> needed them for profiling (dynatrace?) >>>>>>> If it is responsibility of JVMTI agent implementor, I remove this >>>>>>> closure. >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2020/07/03 16:45, serguei.spitsyn at oracle.com wrote: >>>>>>>> Hi Yasumasa, >>>>>>>> >>>>>>>> After some thinking I've concluded that I do not like this >>>>>>>> optimization >>>>>>>> of the GetThreadListStackTraces with GetSingleStackTraceClosure. >>>>>>>> >>>>>>>> We may need more opinions on this but these are my points: >>>>>>>> ??- it adds some complexity and ugliness >>>>>>>> ??- a win is doubtful because it has to be a rare case, so that >>>>>>>> total overhead should not be high >>>>>>>> ??- if it is really high for some use cases then it is up to the >>>>>>>> user >>>>>>>> ??? to optimize it with using GetStackTrace instead >>>>>>>> >>>>>>>> In such cases with doubtful overhead I usually prefer the >>>>>>>> simplicity. >>>>>>>> >>>>>>>> Good examples where it makes sense to optimize are checks for >>>>>>>> target thread to be current thread. >>>>>>>> In such cases there is no need to suspend the target thread, or >>>>>>>> use a VMop/HandshakeClosure. >>>>>>>> For instance, please, see the Monitor functions with the check: >>>>>>>> (java_thread == calling_thread). >>>>>>>> Getting information for current thread is frequently used case, >>>>>>>> e.g. to get info at an event point. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Serguei >>>>>>>> >>>>>>>> >>>>>>>> On 7/2/20 23:29, Yasumasa Suenaga wrote: >>>>>>>>> Hi Dan, David, >>>>>>>>> >>>>>>>>> I uploaded new webrev. Could you review again? >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/ >>>>>>>>> >>>>>>>>> OneGetThreadListStackTraces.java in this webrev would wait >>>>>>>>> until thread state is transited to "waiting" with spin wait. >>>>>>>>> CountDownLatch::await call as Dan pointed is fixed in it :) >>>>>>>>> >>>>>>>>> Diff from webrev.03: >>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/c9aeb7001e50 >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2020/07/03 14:15, David Holmes wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: >>>>>>>>>>> On 2020/07/03 12:24, Daniel D. Daugherty wrote: >>>>>>>>>>>> On 7/2/20 10:50 PM, David Holmes wrote: >>>>>>>>>>>>> Sorry I'm responding here without seeing latest webrev but >>>>>>>>>>>>> there is enough context I think ... >>>>>>>>>>>>> >>>>>>>>>>>>> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>> Hi Dan, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I upload new webrev. Could you review again? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>>>>>>> ???? L1542: ??? // Get stack trace with handshake >>>>>>>>>>>>>>> ???????? nit - please add a period at the end. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> ???? L1591: *stack_info_ptr = op.stack_info(); >>>>>>>>>>>>>>> ???????? The return parameter should not be touched >>>>>>>>>>>>>>> unless the return >>>>>>>>>>>>>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ???? old L1582: ? if (err == JVMTI_ERROR_NONE) { >>>>>>>>>>>>>>> ???????? Please restore this check. The return parameter >>>>>>>>>>>>>>> should not >>>>>>>>>>>>>>> ???????? be touched unless the return code in 'err' == >>>>>>>>>>>>>>> JVMTI_ERROR_NONE. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>> >>>>>>>>>>>>> But op.stack_info() will return NULL if the error is not >>>>>>>>>>>>> JVMTI_ERROR_NONE. Are you (Dan) concerned about someone >>>>>>>>>>>>> passing in a non-null/initialized out-pointer that will be >>>>>>>>>>>>> reset to NULL if there was an error? >>>>>>>>>>>> >>>>>>>>>>>> Actually the way we used to test this in POSIX tests is to call >>>>>>>>>>>> an API with known bad parameters and the return parameter ptr >>>>>>>>>>>> set to NULL. If the return parameter ptr was touched when an >>>>>>>>>>>> error should have been detected on an earlier parameter, then >>>>>>>>>>>> the test failed. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> ???? L1272: ? if (!jt->is_exiting() && (thread_oop != >>>>>>>>>>>>>>> NULL)) { >>>>>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>> ???? old L1532: ? _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>>>>>>>>>>>> ???????? This deletion of the _result field threw me for >>>>>>>>>>>>>>> a minute and then >>>>>>>>>>>>>>> ???? ? ? I figured out that the field is init to >>>>>>>>>>>>>>> JVMTI_ERROR_THREAD_NOT_ALIVE >>>>>>>>>>>>>>> ???????? in the constructor. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ???? L1553: ? if (!jt->is_exiting() && (jt->threadObj() >>>>>>>>>>>>>>> != NULL)) { >>>>>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ???? L64: ??????? startSignal.countDown(); >>>>>>>>>>>>>>> ???????? I was expecting this to be a call to await() >>>>>>>>>>>>>>> instead of >>>>>>>>>>>>>>> ???????? countDown(). What am I missing here? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ???????? I think this test might be passing by accident >>>>>>>>>>>>>>> right now, but... >>>>>>>>>>>>>> >>>>>>>>>>>>>> Main thread (which call JVMTI functions to test) should >>>>>>>>>>>>>> wait until test thread is ready. >>>>>>>>>>>>>> So main thread would wait startSignal, and test thread >>>>>>>>>>>>>> would count down. >>>>>>>>>>>>> >>>>>>>>>>>>> No! >>>>>>>>>>>>> >>>>>>>>>>>>> The test thread that previously called obj.wait() now calls >>>>>>>>>>>>> latch.await(). >>>>>>>>>>>>> >>>>>>>>>>>>> The main thread that previously called obj.notify() now >>>>>>>>>>>>> calls latch.countDown(). >>>>>>>>>>>>> >>>>>>>>>>>>> The main thread continues to spin until it sees the target >>>>>>>>>>>>> is WAITING before proceeding with the test. >>>>>>>>>>> >>>>>>>>>>> If I add spin wait to wait until transit target thread state >>>>>>>>>>> is WAITING (as following), we don't need to call >>>>>>>>>>> SuspendThread(). >>>>>>>>>>> Which is better? >>>>>>>>>> >>>>>>>>>> The original spin-wait loop checking for? WAITING is better >>>>>>>>>> because it is the only guarantee that the target thread is >>>>>>>>>> blocked where you need it to be. suspending the thread is racy >>>>>>>>>> as you don't know exactly where the suspend will hit. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> David >>>>>>>>>> ----- >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> ``` >>>>>>>>>>> /* Wait until the thread state transits to "waiting" */ >>>>>>>>>>> while (th.getState() != Thread.State.WAITING) { >>>>>>>>>>> ???? Thread.onSpinWait(); >>>>>>>>>>> } >>>>>>>>>>> ``` >>>>>>>>>>> >>>>>>>>>>> For simplify, spin wait is prefer to >>>>>>>>>>> OneGetThreadListStackTraces.java in webrev.03. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> Here's the flow as I see it: >>>>>>>>>>>> >>>>>>>>>>>> main thread >>>>>>>>>>>> ?? - start worker thread >>>>>>>>>>>> ?? - startSignal.await() >>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>> >>>>>>>>>>>> worker thread >>>>>>>>>>>> ?? - startSignal.countDown() >>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>> ?? - stopSignal.await() >>>>>>>>>>>> ???? - worker is now blocked >>>>>>>>>>>> >>>>>>>>>>>> main thread >>>>>>>>>>>> ?? - checkCallStacks(th) >>>>>>>>>>>> ?? - stopSignal.countDown() >>>>>>>>>>>> ???? - worker is now unblocked >>>>>>>>>>>> ?? - th.join >>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>> >>>>>>>>>>>> worker thread >>>>>>>>>>>> ?? - runs off the end of run() >>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>> >>>>>>>>>>>> main thread >>>>>>>>>>>> ?? - run off the end of main() >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ???? L92: ? jthreads = (jthread *)malloc(sizeof(jthread) >>>>>>>>>>>>>>> * num_threads); >>>>>>>>>>>>>>> ???????? You don't check for malloc() failure. >>>>>>>>>>>>>>> ???????? 'jthreads' is allocated but never freed. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ???? L91: ? result = (*jvmti)->SuspendThread(jvmti, thread); >>>>>>>>>>>>>>> ???????? Why are you suspending the thread? >>>>>>>>>>>>>>> GetAllStackTraces() and >>>>>>>>>>>>>>> ???????? GetThreadListStackTraces() do not require the >>>>>>>>>>>>>>> target thread(s) >>>>>>>>>>>>>>> ???????? to be suspend. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ???????? If you decide not to SuspendThread, then you >>>>>>>>>>>>>>> don't need the >>>>>>>>>>>>>>> ???????? AddCapabilities or the ResumeThread calls. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Test thread might not be entered following code >>>>>>>>>>>>>> (stopSignal.await()). We might see deferent call stack >>>>>>>>>>>>>> between GetAllStackTraces() and >>>>>>>>>>>>>> GetThreadListStackTraces(). We cannot control to freeze >>>>>>>>>>>>>> call stack of test thread in Java code. >>>>>>>>>>>>>> (I didn't use SuspendThread() at first, but I saw some >>>>>>>>>>>>>> errors which causes in above.) >>>>>>>>>>>>>> >>>>>>>>>>>>>> So we need to call SuspendThread() to ensure we can see >>>>>>>>>>>>>> same call stack. >>>>>>>>>>>>> >>>>>>>>>>>>> If you are checking that the thread is in state WAITING >>>>>>>>>>>>> then it cannot escape from that state and you can sample >>>>>>>>>>>>> the stack multiple times from any API and get the same result. >>>>>>>>>>>>> >>>>>>>>>>>>> I suspect the errors you saw were from the apparent >>>>>>>>>>>>> incorrect use of the CountDownLatch. >>>>>>>>>>>> >>>>>>>>>>>> With the flow outlined above, the worker thread should be >>>>>>>>>>>> nicely blocked in stopSignal.await() when stuff is sampled. >>>>>>>>>>>> >>>>>>>>>>>> Dan >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Cheers, >>>>>>>>>>>>> David >>>>>>>>>>>>> ----- >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I uploaded new webrev. Could review again? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Updates look fine - thanks. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> One minor nit: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 1274 _collector.allocate_and_fill_stacks(1); >>>>>>>>>>>>>>>>> 1275 _collector.set_result(JVMTI_ERROR_NONE); >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> In the other places where you use _collector you rely >>>>>>>>>>>>>>>>> on result being initialized to JVMTI_ERROR_NONE, rather >>>>>>>>>>>>>>>>> than setting it directly after allocate_and_fill_stacks(). >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ? 820 >>>>>>>>>>>>>>>>>>>>>>> assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>> ? 821 >>>>>>>>>>>>>>>>>>>>>>> java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>>>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>> ? 822 current_thread == >>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>> ? 823????????? "at safepoint / handshake or >>>>>>>>>>>>>>>>>>>>>>> target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, >>>>>>>>>>>>>>>>>>>>>>> as even if the target is suspended we must still >>>>>>>>>>>>>>>>>>>>>>> be at a safepoint or in a handshake with it. >>>>>>>>>>>>>>>>>>>>>>> Makes me wonder if we used to allow a racy >>>>>>>>>>>>>>>>>>>>>>> stacktrace operation on a suspended thread, >>>>>>>>>>>>>>>>>>>>>>> assuming it would remain suspended? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> This function (JvmtiEnvBase::get_stack_trace()) can be >>>>>>>>>>>>>>>>>> called to get own stack trace. For example, we can >>>>>>>>>>>>>>>>>> call GetStackTrace() for current thread at JVMTI event. >>>>>>>>>>>>>>>>>> So I changed assert as below: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>> ??820?? assert(current_thread == java_thread || >>>>>>>>>>>>>>>>>> ??821 SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>> ??822????????? current_thread == >>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>> ??823????????? "call by myself / at safepoint / at >>>>>>>>>>>>>>>>>> handshake"); >>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Yep good catch. I hope current tests caught that. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> They would be tested in >>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ (own >>>>>>>>>>>>>>>> call stacks), and getstacktr003 (call stacks in other >>>>>>>>>>>>>>>> thread). >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Speaking of tests ... >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> In the native code I think you need to check the >>>>>>>>>>>>>>>>> success of all JNI methods that can throw exceptions - >>>>>>>>>>>>>>>>> otherwise I believe the tests may trigger warnings if >>>>>>>>>>>>>>>>> -Xcheck:jni is used with them. See for example: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I updated testcases to check JNI and JVMTI function calls. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> In the Java code the target thread: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> ?? 45???? public void run() { >>>>>>>>>>>>>>>>> ?? 46?????? try { >>>>>>>>>>>>>>>>> ?? 47?????????? synchronized (lock) { >>>>>>>>>>>>>>>>> ?? 48?????????????? lock.wait(); >>>>>>>>>>>>>>>>> ?? 49 System.out.println("OK"); >>>>>>>>>>>>>>>>> ?? 50?????????? } >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> is potentially susceptible to spurious wakeups. Using a >>>>>>>>>>>>>>>>> CountDownLatch would be robust. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the >>>>>>>>>>>>>>>>>>>>>>> current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> If so, we can call make_local() in L1272 without >>>>>>>>>>>>>>>>>>>> JavaThread (or we can pass current thread to >>>>>>>>>>>>>>>>>>>> make_local()). Is it right? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>>>>>>>>> 1272 >>>>>>>>>>>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>>>>>>>>>>> thread_oop), >>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Sorry I got confused, _calling_thread may not be the >>>>>>>>>>>>>>>>>>> current thread as we could be executing the handshake >>>>>>>>>>>>>>>>>>> in the target thread itself. So the ResourceMark is >>>>>>>>>>>>>>>>>>> correct as-is (implicitly for current thread). >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> The argument to fill_frames will be used in the >>>>>>>>>>>>>>>>>>> jvmtiStackInfo and passed back to the >>>>>>>>>>>>>>>>>>> _calling_thread, so it must be created via >>>>>>>>>>>>>>>>>>> make_local(_calling_thread, ...) as you presently have. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Thank you for reviewing! I will update new webrev >>>>>>>>>>>>>>>>>>>>>> tomorrow. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public >>>>>>>>>>>>>>>>>>>>>>> StackObj { >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ? 498 class VM_GetAllStackTraces : public >>>>>>>>>>>>>>>>>>>>>>> VM_Operation { >>>>>>>>>>>>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>>>>>>>>>>>> ? 500?? JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>> ? 502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another >>>>>>>>>>>>>>>>>>>>>>> class like that as it may not be on the stack. I >>>>>>>>>>>>>>>>>>>>>>> think MultipleStackTracesCollector should not >>>>>>>>>>>>>>>>>>>>>>> extend any allocation class, and should always be >>>>>>>>>>>>>>>>>>>>>>> embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>>>>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>>>>>>>>>>>>> ??? : >>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> class GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>> ?? private: >>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> I updated webrev for 8242428. Could you review >>>>>>>>>>>>>>>>>>>>>>>> again? >>>>>>>>>>>>>>>>>>>>>>>> This change migrate to use direct handshake for >>>>>>>>>>>>>>>>>>>>>>>> GetStackTrace() and GetThreadListStackTraces() >>>>>>>>>>>>>>>>>>>>>>>> (when thread_count == 1). >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> This looks really good now! I only have a few >>>>>>>>>>>>>>>>>>>>>>> nits below. There is one thing I don't like about >>>>>>>>>>>>>>>>>>>>>>> it but it requires a change to the main Handshake >>>>>>>>>>>>>>>>>>>>>>> logic to address - in >>>>>>>>>>>>>>>>>>>>>>> JvmtiEnv::GetThreadListStackTraces you have to >>>>>>>>>>>>>>>>>>>>>>> create a ThreadsListHandle to convert the jthread >>>>>>>>>>>>>>>>>>>>>>> to a JavaThread, but then the >>>>>>>>>>>>>>>>>>>>>>> Handshake::execute_direct creates another >>>>>>>>>>>>>>>>>>>>>>> ThreadsListHandle internally. That's a waste. I >>>>>>>>>>>>>>>>>>>>>>> will discuss with Robbin and file a RFE to have >>>>>>>>>>>>>>>>>>>>>>> an overload of execute_direct that takes an >>>>>>>>>>>>>>>>>>>>>>> existing TLH. Actually it's worse than that >>>>>>>>>>>>>>>>>>>>>>> because we have another TLH in use at the entry >>>>>>>>>>>>>>>>>>>>>>> point for the JVMTI functions, so I think there >>>>>>>>>>>>>>>>>>>>>>> may be some scope for simplifying the use of TLH >>>>>>>>>>>>>>>>>>>>>>> instances - future RFE. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ??451 GetStackTraceClosure(JvmtiEnv *env, jint >>>>>>>>>>>>>>>>>>>>>>> start_depth, jint max_count, >>>>>>>>>>>>>>>>>>>>>>> ??452 jvmtiFrameInfo* frame_buffer, jint* count_ptr) >>>>>>>>>>>>>>>>>>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>>>>>>>>>>>> ??454?????? _env(env), _start_depth(start_depth), >>>>>>>>>>>>>>>>>>>>>>> _max_count(max_count), >>>>>>>>>>>>>>>>>>>>>>> ??455 _frame_buffer(frame_buffer), >>>>>>>>>>>>>>>>>>>>>>> _count_ptr(count_ptr), >>>>>>>>>>>>>>>>>>>>>>> ??456 _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Nit: can you do one initializer per line please. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public >>>>>>>>>>>>>>>>>>>>>>> StackObj { >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ??498 class VM_GetAllStackTraces : public >>>>>>>>>>>>>>>>>>>>>>> VM_Operation { >>>>>>>>>>>>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>>>>>>>>>>>> ??500?? JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>> ??502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another >>>>>>>>>>>>>>>>>>>>>>> class like that as it may not be on the stack. I >>>>>>>>>>>>>>>>>>>>>>> think MultipleStackTracesCollector should not >>>>>>>>>>>>>>>>>>>>>>> extend any allocation class, and should always be >>>>>>>>>>>>>>>>>>>>>>> embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> 481 MultipleStackTracesCollector(JvmtiEnv *env, >>>>>>>>>>>>>>>>>>>>>>> jint max_frame_count) { >>>>>>>>>>>>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>>>>>>>>>>>> ??483???? _max_frame_count = max_frame_count; >>>>>>>>>>>>>>>>>>>>>>> ??484 _frame_count_total = 0; >>>>>>>>>>>>>>>>>>>>>>> ??485???? _head = NULL; >>>>>>>>>>>>>>>>>>>>>>> ??486???? _stack_info = NULL; >>>>>>>>>>>>>>>>>>>>>>> ??487???? _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> As you are touching this can you change it to use >>>>>>>>>>>>>>>>>>>>>>> an initializer list as you did for the >>>>>>>>>>>>>>>>>>>>>>> HandshakeClosure, and please keep one item per line. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ??820 >>>>>>>>>>>>>>>>>>>>>>> assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>> ??821 >>>>>>>>>>>>>>>>>>>>>>> java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>>>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == >>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>> ??823????????? "at safepoint / handshake or >>>>>>>>>>>>>>>>>>>>>>> target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, >>>>>>>>>>>>>>>>>>>>>>> as even if the target is suspended we must still >>>>>>>>>>>>>>>>>>>>>>> be at a safepoint or in a handshake with it. >>>>>>>>>>>>>>>>>>>>>>> Makes me wonder if we used to allow a racy >>>>>>>>>>>>>>>>>>>>>>> stacktrace operation on a suspended thread, >>>>>>>>>>>>>>>>>>>>>>> assuming it would remain suspended? >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>>>>>>>>>>>>>>> 1269 >>>>>>>>>>>>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && (jt->threadObj() >>>>>>>>>>>>>>>>>>>>>>> != NULL)) { >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> 1272 >>>>>>>>>>>>>>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>>>>>>>>>>>>>> thread_oop), >>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> It is frustrating that this entire call chain >>>>>>>>>>>>>>>>>>>>>>> started with a jthread reference, which we >>>>>>>>>>>>>>>>>>>>>>> converted to a JavaThread, only to eventually >>>>>>>>>>>>>>>>>>>>>>> need to convert it back to a jthread! I think >>>>>>>>>>>>>>>>>>>>>>> there is some scope for simplification here but >>>>>>>>>>>>>>>>>>>>>>> not as part of this change. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the >>>>>>>>>>>>>>>>>>>>>>> current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I'm still pondering the test logic but wanted to >>>>>>>>>>>>>>>>>>>>>>> send this now. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>> VM_GetThreadListStackTrace (for >>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces) and >>>>>>>>>>>>>>>>>>>>>>>> VM_GetAllStackTraces (for GetAllStackTraces) >>>>>>>>>>>>>>>>>>>>>>>> have inherited VM_GetMultipleStackTraces VM >>>>>>>>>>>>>>>>>>>>>>>> operation which provides the feature to generate >>>>>>>>>>>>>>>>>>>>>>>> jvmtiStackInfo. I modified >>>>>>>>>>>>>>>>>>>>>>>> VM_GetMultipleStackTraces to a normal C++ class >>>>>>>>>>>>>>>>>>>>>>>> to share with HandshakeClosure for >>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces >>>>>>>>>>>>>>>>>>>>>>>> (GetSingleStackTraceClosure). >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Also I added new testcases which test >>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() with thread_count == >>>>>>>>>>>>>>>>>>>>>>>> 1 and with all threads. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in >>>>>>>>>>>>>>>>>>>>>>>> serviceability/jvmti serviceability/jdwp >>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti vmTestbase/nsk/jdi >>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jdwp. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> ?? JBS: >>>>>>>>>>>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>>>>>>>>>>>> ?? webrev: >>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> This change replace following VM operations to >>>>>>>>>>>>>>>>>>>>>>>>> direct handshake. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces >>>>>>>>>>>>>>>>>>>>>>>>> (GetThreadListStackTrace()) >>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTrace() uses direct handshake >>>>>>>>>>>>>>>>>>>>>>>>> if thread count == 1. In other case (thread >>>>>>>>>>>>>>>>>>>>>>>>> count > 1), it would be performed as VM >>>>>>>>>>>>>>>>>>>>>>>>> operation (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>>>>>>>>>>>> Caller of VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>> (JvmtiEnvThreadState::reset_current_location()) >>>>>>>>>>>>>>>>>>>>>>>>> might be called at safepoint. So I added >>>>>>>>>>>>>>>>>>>>>>>>> safepoint check in its caller. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in >>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jvmti serviceability/jdwp >>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti vmTestbase/nsk/jdi >>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/ns >>>>>>>>>>>>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Also I tested it on submit repo, then it has >>>>>>>>>>>>>>>>>>>>>>>>> execution error >>>>>>>>>>>>>>>>>>>>>>>>> (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) >>>>>>>>>>>>>>>>>>>>>>>>> due to dependency error. So I think it does not >>>>>>>>>>>>>>>>>>>>>>>>> occur by this change. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>> >>>>>> >>>> >> From chris.plummer at oracle.com Tue Jul 7 04:18:15 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 6 Jul 2020 21:18:15 -0700 Subject: RFR(XS): 8248878: SA: Implement simple workaround for JDK-8248876 Message-ID: <39ac40d6-a738-3214-d86c-412152a631d4@oracle.com> Hello, Please help review the following: http://cr.openjdk.java.net/~cjplummer/8248878/webrev.00/index.html https://bugs.openjdk.java.net/browse/JDK-8248878 The explanation of the fix is in the CR. The parent CR, JDK-8248876 [1], explains the issue being addressed. There's no test for this fix yet. It requires the changes I'm making for JDK-8247514 [2], which include changes to "findpc" support and the ClhsdbFindPC.java test that trigger this issue. [1] https://bugs.openjdk.java.net/browse/JDK-8248876 [2] https://bugs.openjdk.java.net/browse/JDK-8247514 thanks, Chris From chris.plummer at oracle.com Tue Jul 7 04:31:34 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 6 Jul 2020 21:31:34 -0700 Subject: RFR(XS): 8248879: SA core file support on OSX has some bugs trying to locate the jvm libraries Message-ID: <6089f692-152b-8615-dfca-779f2b5028a0@oracle.com> Hello, Please help review the following: http://cr.openjdk.java.net/~cjplummer/8248879/webrev.00/index.html https://bugs.openjdk.java.net/browse/JDK-8248879 The description of the problem and the fix are both in the CR. thanks, Chris From suenaga at oss.nttdata.com Tue Jul 7 04:57:14 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Tue, 7 Jul 2020 13:57:14 +0900 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <763793b1-9506-44c6-0681-1c71c44a44fd@oss.nttdata.com> <5107c478-1de2-7a49-ee71-dfad6a99a667@oracle.com> <90cdfef9-e558-0d25-33f5-25b7d65bb83b@oss.nttdata.com> <0e359819-618f-c17b-a427-bb7e4daa2c58@oracle.com> <3f890bed-6892-e64d-b21b-ca0cbe11784e@oracle.com> <7dc1903b-6295-3864-6301-4c02aacdb971@oss.nttdata.com> <04e09eb7-9483-8f63-4bb4-01c9e814802e@oracle.com> <238a89a3-06b1-90e4-d5a4-8dc2f1ed12ab@oracle.com> <4c43bc2c-551e-d042-8fa7-b63e755888d1@oracle.com> <8c387550-a06e-6320-f453-0e898486befb@oss.nttdata.com> Message-ID: <19cfa2a7-f441-425d-5623-1c11c6a870b6@oss.nttdata.com> Hi David, On 2020/07/07 11:31, David Holmes wrote: > Hi Yasumasa, > > Hard to keep up with the changes - especially without incremental webrevs. Sorry, I will upload diff from previous webrev in the next. > If GetSingleStackTraceClosure also took the jthread as a constructor arg, then you wouldn't need to recreate a JNI local handle when calling _collector.fill_frames. It's a small simplification and not essential at this stage. I think we should get jthread from an argument of do_thread() because do_thread() would pass the thread which are stopped certainly. It might be simplification if we pass _calling_thread to MultipleStackTracesCollector. `jthread` is only needed to store jvmtiStackInfo.thread . What do you think? > For the test ... I don't see how Java_GetThreadListStackTraces_checkCallStacks is a valid test. It gets the stacks of all live threads, then uses that information to use GetThreadListStackTraces to get the stack for the same set of threads through a different API. It then compares the two sets of stacks for each thread expecting them to be the same, but that need only be the case for the main thread. Other threads could potentially have a different stack (e.g. if this test is run with JFR enabled there will be additional threads found.) Further I would have expected that there already exist tests that check that, for a given thread (which may be suspended or known to be blocked) the same stack is found through the two different APIs. vmTestbase/nsk/jvmti/unit/GetAllStackTraces/getallstktr001 would check all of threads via GetThreadListStackTraces() and GetAllStackTraces(), so we might be able to remove GetThreadListStackTraces.java from this webrev. OTOH we don't have testcase for GetThreadListStackTraces() with thread_count == 1, so we need to testcase for it (it is OneGetThreadListStackTraces.java) It would check whether the state of target thread is "waiting" before JNI call to call GetThreadListStackTraces(), and also I expect it would not be run with JFR. (it is not described @run) Of course we can check GetThreadListStackTraces() with main thread, but it is not the test for direct handshake for other thread. Thanks, Yasumasa > Thanks, > David > ----- > > On 6/07/2020 11:29 pm, Yasumasa Suenaga wrote: >> Hi Serguei, >> >> Thanks for your comment! >> >> I think C++ is more simple to implement the test agent as you said. >> So I implement it in C++ in new webrev. Could you review again? >> >> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.06/ >> >> Also I refactored libGetThreadListStackTraces.cpp, and I've kept exception check after IsSameObject(). >> >> >> Thanks, >> >> Yasumasa >> >> >> On 2020/07/06 16:32, serguei.spitsyn at oracle.com wrote: >>> Hi Yasumasa, >>> >>> Thank you for the update. >>> I think, a pending exception after IsSameObject needs to be checked. >>> >>> The checkStackInfo() needs one more refactoring as I've already suggested. >>> The body of the loop at L68-L78 should be converted to a function check_frame_info. >>> The si1->frame_buffer[i] and si2->frame_buffer[i] will be passed as fi1 and fi2. >>> The index can be passed as well. >>> I'm still suggesting to simplify the local exception_msg to something shorter like err_msg or exc_msg. >>> >>> I'm not sure using fatal is right here: >>> >>> This fragment looks strange: >>> >>> ? 152???? if ((*env)->IsSameObject(env, stack_info[i].thread, thread)) { >>> ? 153?????? target_info = &stack_info[i]; >>> ? 154?????? break; >>> ? 155???? } else if ((*env)->ExceptionOccurred(env)) { >>> ? 156?????? (*env)->ExceptionDescribe(env); >>> ? 157?????? (*env)->FatalError(env, __FILE__); >>> ? 158???? } >>> >>> I expected it to be: >>> >>> ??? jboolean same = (*env)->IsSameObject(env, stack_info[i].thread, thread); >>> ??? if ((*env)->ExceptionOccurred(env)) { >>> ????? (*env)->ExceptionDescribe(env); >>> ????? (*env)->FatalError(env, __FILE__); >>> ??? } >>> ??? if (same) { >>> ????? target_info = &stack_info[i]; >>> ????? break; >>> ??? } >>> >>> Would it better to port this agent to C++ to simplify this code nicer? >>> >>> Thanks, >>> Serguei >>> >>> >>> On 7/5/20 06:13, Yasumasa Suenaga wrote: >>>> Hi Serguei, >>>> >>>> Thanks for your comment! >>>> I refactored testcase. Could you review again? >>>> >>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.05/ >>>> >>>> It would check Java exception after IsSameObject() call. Does it need? >>>> Any exceptions are not described in JNI document[1], and JNI implementation (jni_IsSameObject()) does not seem to throw it. >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> [1] https://docs.oracle.com/en/java/javase/14/docs/specs/jni/functions.html#issameobject >>>> >>>> >>>> On 2020/07/05 14:46, serguei.spitsyn at oracle.com wrote: >>>>> Hi Yasumasa, >>>>> >>>>> >>>>> Okay, thanks. >>>>> Then I'm okay to keep the GetSingleStackTraceClosure. >>>>> >>>>> >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c.html >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c.html >>>>> >>>>> I'm not sure the function 'is_same_thread() is needed. >>>>> Why do not use the JNI IsSameObject instead? >>>>> >>>>> It seems to be a typo at L132 and L137. >>>>> You, probably. did not want to print the same information for stack_info_1[i].frame_buffer[j].XXX twice. >>>>> >>>>> The code at lines 112-142 is not readable. >>>>> I'd suggest to make a couple of refactoring steps. >>>>> >>>>> First step to simplify this a little bit would be with some renaming and getting rid of indexes: >>>>> >>>>> ?? 71 char err_msg[EXCEPTION_MSG_LEN] = {0}; >>>>> ??... >>>>> ??112?? /* Iterate all jvmtiStackInfo to check */ >>>>> ??113?? for (i = 0; i < num_threads, *exception_msg != '\0'; i++) { >>>>> ????????? jvmtiStackInfo *si1 = stack_info_1[i]; >>>>> ????????? jvmtiStackInfo *si2 = stack_info_2[i]; >>>>> ??114???? if (!IsSameObject(env, si1.thread, si2.thread)) { /* jvmtiStackInfo::thread */ >>>>> ??115?????? snprintf(err_msg, sizeof(err_msg), >>>>> ??116??????????????? "thread[%d] is different: stack_info_1 = %p, stack_info_2 = %p", >>>>> ??117??????????????? i, sinfo1.thread, sinfo2.thread); >>>>> ??118???? } else if (si1.state != si2.state) { /* jvmtiStackInfo::state */ >>>>> ??119?????? snprintf(err_msg, sizeof(err_msg), >>>>> ??120??????????????? "state[%d] is different: stack_info_1 = %d, stack_info_2 = %d", >>>>> ??121??????????????? i, si1.state, si2.state); >>>>> ??122???? } else if (si1.frame_count != si2.frame_count) { /* jvmtiStackInfo::frame_count */ >>>>> ??123?????? snprintf(err_msg, sizeof(err_msg), >>>>> ??124??????????????? "frame_count[%d] is different: stack_info_1 = %d, stack_info_2 = %d", >>>>> ??125??????????????? i, si1.frame_count, si2.frame_count); >>>>> ??126???? } else { >>>>> ??127?????? /* Iterate all jvmtiFrameInfo to check */ >>>>> ??128?????? for (j = 0; j < si1.frame_count; j++) { >>>>> ??129???????? if (si1.frame_buffer[j].method != si1.frame_buffer[j].method) { /* jvmtiFrameInfo::method */ >>>>> ??130?????????? snprintf(err_msg, sizeof(err_msg), >>>>> ??131??????????????????? "thread [%d] frame_buffer[%d].method is different: stack_info_1 = %lx, stack_info_2 = %lx", >>>>> ??132??????????????????? i, j, si1.frame_buffer[j].method, si2.frame_buffer[j].method); >>>>> ??133?????????? break; >>>>> ??134???????? } else if (si1.frame_buffer[j].location != si1.frame_buffer[j].location) { /* jvmtiFrameInfo::location */ >>>>> ??135?????????? snprintf(err_msg, sizeof(err_msg), >>>>> ??136??????????????????? "thread [%d] frame_buffer[%d].location is different: stack_info_1 = %ld, stack_info_2 = %ld", >>>>> ??137??????????????????? i, j, si1.frame_buffer[j].location, si2.frame_buffer[j].location); >>>>> ??138?????????? break; >>>>> ??139???????? } >>>>> ??140?????? } >>>>> ??141???? } >>>>> ??142?? } >>>>> >>>>> Another step would be to create functions that implement a body of each loop. >>>>> You can use the same techniques to simplify similar place (L127-L138) in the libOneGetThreadListStackTraces.c. >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>> On 7/3/20 15:55, Yasumasa Suenaga wrote: >>>>>> Hi Serguei, >>>>>> >>>>>> I'm not an Oracle employee, so I cannot know real request(s) from your customers. >>>>>> However JDK-8201641 says Dynatrace has requested this enhancement. >>>>>> >>>>>> BTW I haven't heared any request from my customers about this. >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2020/07/04 4:32, serguei.spitsyn at oracle.com wrote: >>>>>>> Hi Yasumasa, >>>>>>> >>>>>>> This difference is not that big to care about. >>>>>>> I feel this is really rare case and so, does not worth these complications. >>>>>>> Do we have a real request from customers to optimize it? >>>>>>> >>>>>>> Thanks, >>>>>>> Serguei >>>>>>> >>>>>>> >>>>>>> On 7/3/20 01:16, Yasumasa Suenaga wrote: >>>>>>>> Hi Serguei, >>>>>>>> >>>>>>>> Generally I agree with you, but I have concern about the difference of the result of GetStackTrace() and GetThreadListStackTraces(). >>>>>>>> >>>>>>>> ? GetStackTrace: jvmtiFrameInfo >>>>>>>> ? GetThreadListStackTraces: jvmtiStackInfo >>>>>>>> >>>>>>>> jvmtiStackInfo contains thread state, and it is ensured it is the state of the call stack. >>>>>>>> If we want to get both call stack and thread state, we need to suspend target thread, and call both GetStackTrace() and GetThreadState(). Is it ok? >>>>>>>> >>>>>>>> I was wondering if JDK-8201641 (parent ticket of this change) needed them for profiling (dynatrace?) >>>>>>>> If it is responsibility of JVMTI agent implementor, I remove this closure. >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2020/07/03 16:45, serguei.spitsyn at oracle.com wrote: >>>>>>>>> Hi Yasumasa, >>>>>>>>> >>>>>>>>> After some thinking I've concluded that I do not like this optimization >>>>>>>>> of the GetThreadListStackTraces with GetSingleStackTraceClosure. >>>>>>>>> >>>>>>>>> We may need more opinions on this but these are my points: >>>>>>>>> ??- it adds some complexity and ugliness >>>>>>>>> ??- a win is doubtful because it has to be a rare case, so that total overhead should not be high >>>>>>>>> ??- if it is really high for some use cases then it is up to the user >>>>>>>>> ??? to optimize it with using GetStackTrace instead >>>>>>>>> >>>>>>>>> In such cases with doubtful overhead I usually prefer the simplicity. >>>>>>>>> >>>>>>>>> Good examples where it makes sense to optimize are checks for target thread to be current thread. >>>>>>>>> In such cases there is no need to suspend the target thread, or use a VMop/HandshakeClosure. >>>>>>>>> For instance, please, see the Monitor functions with the check: (java_thread == calling_thread). >>>>>>>>> Getting information for current thread is frequently used case, e.g. to get info at an event point. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Serguei >>>>>>>>> >>>>>>>>> >>>>>>>>> On 7/2/20 23:29, Yasumasa Suenaga wrote: >>>>>>>>>> Hi Dan, David, >>>>>>>>>> >>>>>>>>>> I uploaded new webrev. Could you review again? >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/ >>>>>>>>>> >>>>>>>>>> OneGetThreadListStackTraces.java in this webrev would wait until thread state is transited to "waiting" with spin wait. >>>>>>>>>> CountDownLatch::await call as Dan pointed is fixed in it :) >>>>>>>>>> >>>>>>>>>> Diff from webrev.03: >>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/c9aeb7001e50 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2020/07/03 14:15, David Holmes wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>> On 2020/07/03 12:24, Daniel D. Daugherty wrote: >>>>>>>>>>>>> On 7/2/20 10:50 PM, David Holmes wrote: >>>>>>>>>>>>>> Sorry I'm responding here without seeing latest webrev but there is enough context I think ... >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>> Hi Dan, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I upload new webrev. Could you review again? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>>>>>>>> ???? L1542: ??? // Get stack trace with handshake >>>>>>>>>>>>>>>> ???????? nit - please add a period at the end. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> ???? L1591: *stack_info_ptr = op.stack_info(); >>>>>>>>>>>>>>>> ???????? The return parameter should not be touched unless the return >>>>>>>>>>>>>>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> ???? old L1582: ? if (err == JVMTI_ERROR_NONE) { >>>>>>>>>>>>>>>> ???????? Please restore this check. The return parameter should not >>>>>>>>>>>>>>>> ???????? be touched unless the return code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>> >>>>>>>>>>>>>> But op.stack_info() will return NULL if the error is not JVMTI_ERROR_NONE. Are you (Dan) concerned about someone passing in a non-null/initialized out-pointer that will be reset to NULL if there was an error? >>>>>>>>>>>>> >>>>>>>>>>>>> Actually the way we used to test this in POSIX tests is to call >>>>>>>>>>>>> an API with known bad parameters and the return parameter ptr >>>>>>>>>>>>> set to NULL. If the return parameter ptr was touched when an >>>>>>>>>>>>> error should have been detected on an earlier parameter, then >>>>>>>>>>>>> the test failed. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> ???? L1272: ? if (!jt->is_exiting() && (thread_oop != NULL)) { >>>>>>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>> ???? old L1532: ? _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>>>>>>>>>>>>> ???????? This deletion of the _result field threw me for a minute and then >>>>>>>>>>>>>>>> ???? ? ? I figured out that the field is init to JVMTI_ERROR_THREAD_NOT_ALIVE >>>>>>>>>>>>>>>> ???????? in the constructor. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> ???? L1553: ? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>>>>>>>> ???? L64: ??????? startSignal.countDown(); >>>>>>>>>>>>>>>> ???????? I was expecting this to be a call to await() instead of >>>>>>>>>>>>>>>> ???????? countDown(). What am I missing here? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> ???????? I think this test might be passing by accident right now, but... >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Main thread (which call JVMTI functions to test) should wait until test thread is ready. >>>>>>>>>>>>>>> So main thread would wait startSignal, and test thread would count down. >>>>>>>>>>>>>> >>>>>>>>>>>>>> No! >>>>>>>>>>>>>> >>>>>>>>>>>>>> The test thread that previously called obj.wait() now calls latch.await(). >>>>>>>>>>>>>> >>>>>>>>>>>>>> The main thread that previously called obj.notify() now calls latch.countDown(). >>>>>>>>>>>>>> >>>>>>>>>>>>>> The main thread continues to spin until it sees the target is WAITING before proceeding with the test. >>>>>>>>>>>> >>>>>>>>>>>> If I add spin wait to wait until transit target thread state is WAITING (as following), we don't need to call SuspendThread(). >>>>>>>>>>>> Which is better? >>>>>>>>>>> >>>>>>>>>>> The original spin-wait loop checking for? WAITING is better because it is the only guarantee that the target thread is blocked where you need it to be. suspending the thread is racy as you don't know exactly where the suspend will hit. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> David >>>>>>>>>>> ----- >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> ``` >>>>>>>>>>>> /* Wait until the thread state transits to "waiting" */ >>>>>>>>>>>> while (th.getState() != Thread.State.WAITING) { >>>>>>>>>>>> ???? Thread.onSpinWait(); >>>>>>>>>>>> } >>>>>>>>>>>> ``` >>>>>>>>>>>> >>>>>>>>>>>> For simplify, spin wait is prefer to OneGetThreadListStackTraces.java in webrev.03. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> Here's the flow as I see it: >>>>>>>>>>>>> >>>>>>>>>>>>> main thread >>>>>>>>>>>>> ?? - start worker thread >>>>>>>>>>>>> ?? - startSignal.await() >>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>> >>>>>>>>>>>>> worker thread >>>>>>>>>>>>> ?? - startSignal.countDown() >>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>> ?? - stopSignal.await() >>>>>>>>>>>>> ???? - worker is now blocked >>>>>>>>>>>>> >>>>>>>>>>>>> main thread >>>>>>>>>>>>> ?? - checkCallStacks(th) >>>>>>>>>>>>> ?? - stopSignal.countDown() >>>>>>>>>>>>> ???? - worker is now unblocked >>>>>>>>>>>>> ?? - th.join >>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>> >>>>>>>>>>>>> worker thread >>>>>>>>>>>>> ?? - runs off the end of run() >>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>> >>>>>>>>>>>>> main thread >>>>>>>>>>>>> ?? - run off the end of main() >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>>>>>>>>>>>>> ???? L92: ? jthreads = (jthread *)malloc(sizeof(jthread) * num_threads); >>>>>>>>>>>>>>>> ???????? You don't check for malloc() failure. >>>>>>>>>>>>>>>> ???????? 'jthreads' is allocated but never freed. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>>>>>>>>>>>>> ???? L91: ? result = (*jvmti)->SuspendThread(jvmti, thread); >>>>>>>>>>>>>>>> ???????? Why are you suspending the thread? GetAllStackTraces() and >>>>>>>>>>>>>>>> ???????? GetThreadListStackTraces() do not require the target thread(s) >>>>>>>>>>>>>>>> ???????? to be suspend. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> ???????? If you decide not to SuspendThread, then you don't need the >>>>>>>>>>>>>>>> ???????? AddCapabilities or the ResumeThread calls. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Test thread might not be entered following code (stopSignal.await()). We might see deferent call stack between GetAllStackTraces() and GetThreadListStackTraces(). We cannot control to freeze call stack of test thread in Java code. >>>>>>>>>>>>>>> (I didn't use SuspendThread() at first, but I saw some errors which causes in above.) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> So we need to call SuspendThread() to ensure we can see same call stack. >>>>>>>>>>>>>> >>>>>>>>>>>>>> If you are checking that the thread is in state WAITING then it cannot escape from that state and you can sample the stack multiple times from any API and get the same result. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I suspect the errors you saw were from the apparent incorrect use of the CountDownLatch. >>>>>>>>>>>>> >>>>>>>>>>>>> With the flow outlined above, the worker thread should be >>>>>>>>>>>>> nicely blocked in stopSignal.await() when stuff is sampled. >>>>>>>>>>>>> >>>>>>>>>>>>> Dan >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>> David >>>>>>>>>>>>>> ----- >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could review again? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Updates look fine - thanks. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> One minor nit: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> 1274 _collector.allocate_and_fill_stacks(1); >>>>>>>>>>>>>>>>>> 1275 _collector.set_result(JVMTI_ERROR_NONE); >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> In the other places where you use _collector you rely on result being initialized to JVMTI_ERROR_NONE, rather than setting it directly after allocate_and_fill_stacks(). >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> ? 820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>> ? 821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>> ? 822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>> ? 823????????? "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> This function (JvmtiEnvBase::get_stack_trace()) can be called to get own stack trace. For example, we can call GetStackTrace() for current thread at JVMTI event. >>>>>>>>>>>>>>>>>>> So I changed assert as below: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>> ??820?? assert(current_thread == java_thread || >>>>>>>>>>>>>>>>>>> ??821 SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>> ??822????????? current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>> ??823????????? "call by myself / at safepoint / at handshake"); >>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Yep good catch. I hope current tests caught that. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> They would be tested in vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ (own call stacks), and getstacktr003 (call stacks in other thread). >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Speaking of tests ... >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> In the native code I think you need to check the success of all JNI methods that can throw exceptions - otherwise I believe the tests may trigger warnings if -Xcheck:jni is used with them. See for example: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I updated testcases to check JNI and JVMTI function calls. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> In the Java code the target thread: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> ?? 45???? public void run() { >>>>>>>>>>>>>>>>>> ?? 46?????? try { >>>>>>>>>>>>>>>>>> ?? 47?????????? synchronized (lock) { >>>>>>>>>>>>>>>>>> ?? 48?????????????? lock.wait(); >>>>>>>>>>>>>>>>>> ?? 49 System.out.println("OK"); >>>>>>>>>>>>>>>>>> ?? 50?????????? } >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> is potentially susceptible to spurious wakeups. Using a CountDownLatch would be robust. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> If so, we can call make_local() in L1272 without JavaThread (or we can pass current thread to make_local()). Is it right? >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Sorry I got confused, _calling_thread may not be the current thread as we could be executing the handshake in the target thread itself. So the ResourceMark is correct as-is (implicitly for current thread). >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> The argument to fill_frames will be used in the jvmtiStackInfo and passed back to the _calling_thread, so it must be created via make_local(_calling_thread, ...) as you presently have. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thank you for reviewing! I will update new webrev tomorrow. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> ? 498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>>>>>>>>>>>>> ? 500?? JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>> ? 502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>>>>>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>>>>>>>>>>>>>> ??? : >>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> class GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>> ?? private: >>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> I updated webrev for 8242428. Could you review again? >>>>>>>>>>>>>>>>>>>>>>>>> This change migrate to use direct handshake for GetStackTrace() and GetThreadListStackTraces() (when thread_count == 1). >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> This looks really good now! I only have a few nits below. There is one thing I don't like about it but it requires a change to the main Handshake logic to address - in JvmtiEnv::GetThreadListStackTraces you have to create a ThreadsListHandle to convert the jthread to a JavaThread, but then the Handshake::execute_direct creates another ThreadsListHandle internally. That's a waste. I will discuss with Robbin and file a RFE to have an overload of execute_direct that takes an existing TLH. Actually it's worse than that because we have another TLH in use at the entry point for the JVMTI functions, so I think there may be some scope for simplifying the use of TLH instances - future RFE. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> ??451 GetStackTraceClosure(JvmtiEnv *env, jint start_depth, jint max_count, >>>>>>>>>>>>>>>>>>>>>>>> ??452 jvmtiFrameInfo* frame_buffer, jint* count_ptr) >>>>>>>>>>>>>>>>>>>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>>>>>>>>>>>>> ??454?????? _env(env), _start_depth(start_depth), _max_count(max_count), >>>>>>>>>>>>>>>>>>>>>>>> ??455 _frame_buffer(frame_buffer), _count_ptr(count_ptr), >>>>>>>>>>>>>>>>>>>>>>>> ??456 _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Nit: can you do one initializer per line please. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> ??498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>>>>>>>>>>>>> ??500?? JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>> ??502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> 481 MultipleStackTracesCollector(JvmtiEnv *env, jint max_frame_count) { >>>>>>>>>>>>>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>>>>>>>>>>>>> ??483???? _max_frame_count = max_frame_count; >>>>>>>>>>>>>>>>>>>>>>>> ??484 _frame_count_total = 0; >>>>>>>>>>>>>>>>>>>>>>>> ??485???? _head = NULL; >>>>>>>>>>>>>>>>>>>>>>>> ??486???? _stack_info = NULL; >>>>>>>>>>>>>>>>>>>>>>>> ??487???? _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> As you are touching this can you change it to use an initializer list as you did for the HandshakeClosure, and please keep one item per line. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> ??820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>> ??821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>> ??823????????? "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>>>>>>>>>>>>>>>> 1269 >>>>>>>>>>>>>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> It is frustrating that this entire call chain started with a jthread reference, which we converted to a JavaThread, only to eventually need to convert it back to a jthread! I think there is some scope for simplification here but not as part of this change. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> I'm still pondering the test logic but wanted to send this now. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>> VM_GetThreadListStackTrace (for GetThreadListStackTraces) and VM_GetAllStackTraces (for GetAllStackTraces) have inherited VM_GetMultipleStackTraces VM operation which provides the feature to generate jvmtiStackInfo. I modified VM_GetMultipleStackTraces to a normal C++ class to share with HandshakeClosure for GetThreadListStackTraces (GetSingleStackTraceClosure). >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Also I added new testcases which test GetThreadListStackTraces() with thread_count == 1 and with all threads. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/nsk/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>>>>>>>>>>>>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> This change replace following VM operations to direct handshake. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces (GetThreadListStackTrace()) >>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTrace() uses direct handshake if thread count == 1. In other case (thread count > 1), it would be performed as VM operation (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>>>>>>>>>>>>> Caller of VM_GetCurrentLocation (JvmtiEnvThreadState::reset_current_location()) might be called at safepoint. So I added safepoint check in its caller. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/ns >>>>>>>>>>>>>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Also I tested it on submit repo, then it has execution error (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) due to dependency error. So I think it does not occur by this change. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>> >>>>>>> >>>>> >>> From chris.plummer at oracle.com Tue Jul 7 05:10:09 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 6 Jul 2020 22:10:09 -0700 Subject: RFR(M): 8247515: OSX pc_to_symbol() lookup does not work with core files Message-ID: Hello, Please help review the following: http://cr.openjdk.java.net/~cjplummer/8247515/webrev.00/index.html https://bugs.openjdk.java.net/browse/JDK-8247515 The CR contains a description of the issues being addressed. There is also no test for this symbol lookup support yet. It will be there after I push JDK-8247516 and? JDK-8247514, which are both blocked by the CR. [1] https://bugs.openjdk.java.net/browse/JDK-8247516 [2] https://bugs.openjdk.java.net/browse/JDK-8247514 thanks, Chris From suenaga at oss.nttdata.com Tue Jul 7 05:46:08 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Tue, 7 Jul 2020 14:46:08 +0900 Subject: RFR(XS): 8248878: SA: Implement simple workaround for JDK-8248876 In-Reply-To: <39ac40d6-a738-3214-d86c-412152a631d4@oracle.com> References: <39ac40d6-a738-3214-d86c-412152a631d4@oracle.com> Message-ID: Hi Chris, Your change looks good. BTW I saw JDK-8248876. I'm not sure, but I guess we can fix this issue if we allow PT_INTERP in L118: ``` 105 uintptr_t find_base_address(int fd, ELF_EHDR* ehdr) { : 115 // the base address of a shared object is the lowest vaddr of 116 // its loadable segments (PT_LOAD) 117 for (phdr = phbuf, cnt = 0; cnt < ehdr->e_phnum; cnt++, phdr++) { 118 if (phdr->p_type == PT_LOAD && phdr->p_vaddr < baseaddr) { 119 baseaddr = phdr->p_vaddr; 120 } 121 } ``` /proc//maps shows top of `java` is 0x56543b9df000: 56543b9df000-56543b9e0000 r--p 00000000 08:10 55770 /usr/lib/jvm/java-11-openjdk-amd64/bin/java `i target` on GDB shows 0x56543b9df000 is .interp section: Local exec file: `/usr/lib/jvm/java-11-openjdk-amd64/bin/java', file type elf64-x86-64. Entry point: 0x56543b9e0330 0x000056543b9df318 - 0x000056543b9df334 is .interp Thanks, Yasumasa On 2020/07/07 13:18, Chris Plummer wrote: > Hello, > > Please help review the following: > > http://cr.openjdk.java.net/~cjplummer/8248878/webrev.00/index.html > https://bugs.openjdk.java.net/browse/JDK-8248878 > > The explanation of the fix is in the CR. The parent CR, JDK-8248876 [1], explains the issue being addressed. > > There's no test for this fix yet. It requires the changes I'm making for JDK-8247514 [2], which include changes to "findpc" support and the ClhsdbFindPC.java test that trigger this issue. > > [1] https://bugs.openjdk.java.net/browse/JDK-8248876 > [2] https://bugs.openjdk.java.net/browse/JDK-8247514 > > thanks, > > Chris From david.holmes at oracle.com Tue Jul 7 06:13:46 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 7 Jul 2020 16:13:46 +1000 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: <19cfa2a7-f441-425d-5623-1c11c6a870b6@oss.nttdata.com> References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <5107c478-1de2-7a49-ee71-dfad6a99a667@oracle.com> <90cdfef9-e558-0d25-33f5-25b7d65bb83b@oss.nttdata.com> <0e359819-618f-c17b-a427-bb7e4daa2c58@oracle.com> <3f890bed-6892-e64d-b21b-ca0cbe11784e@oracle.com> <7dc1903b-6295-3864-6301-4c02aacdb971@oss.nttdata.com> <04e09eb7-9483-8f63-4bb4-01c9e814802e@oracle.com> <238a89a3-06b1-90e4-d5a4-8dc2f1ed12ab@oracle.com> <4c43bc2c-551e-d042-8fa7-b63e755888d1@oracle.com> <8c387550-a06e-6320-f453-0e898486befb@oss.nttdata.com> <19cfa2a7-f441-425d-5623-1c11c6a870b6@oss.nttdata.com> Message-ID: On 7/07/2020 2:57 pm, Yasumasa Suenaga wrote: > Hi David, > > On 2020/07/07 11:31, David Holmes wrote: >> Hi Yasumasa, >> >> Hard to keep up with the changes - especially without incremental >> webrevs. > > Sorry, I will upload diff from previous webrev in the next. > > >> If GetSingleStackTraceClosure also took the jthread as a constructor >> arg, then you wouldn't need to recreate a JNI local handle when >> calling _collector.fill_frames. It's a small simplification and not >> essential at this stage. > > I think we should get jthread from an argument of do_thread() because > do_thread() would pass the thread which are stopped certainly. > It might be simplification if we pass _calling_thread to > MultipleStackTracesCollector. `jthread` is only needed to store > jvmtiStackInfo.thread . What do you think? I'm not quite sure what you mean. I think there is a bit of a design wart with direct handshakes in that do_thread takes the target JavaThread as an argument. That's useful in a case where you want a HandshakeClosure that can be applied to multiple threads, but that's not typically what is needed with direct handshakes - there is only a single target. With a single-target HandshakeClosure you can capture all the "target" information for the operation in the closure instance. So if the actual do_thread operation wants the jthread corresponding to the target thread then we can store that in the closure rather than recomputing it (you could assert it is the same but that seems overkill to me). > >> For the test ... I don't see how >> Java_GetThreadListStackTraces_checkCallStacks is a valid test. It gets >> the stacks of all live threads, then uses that information to use >> GetThreadListStackTraces to get the stack for the same set of threads >> through a different API. It then compares the two sets of stacks for >> each thread expecting them to be the same, but that need only be the >> case for the main thread. Other threads could potentially have a >> different stack (e.g. if this test is run with JFR enabled there will >> be additional threads found.) Further I would have expected that there >> already exist tests that check that, for a given thread (which may be >> suspended or known to be blocked) the same stack is found through the >> two different APIs. > > vmTestbase/nsk/jvmti/unit/GetAllStackTraces/getallstktr001 would check > all of threads via GetThreadListStackTraces() and GetAllStackTraces(), > so we might be able to remove GetThreadListStackTraces.java from this > webrev. Yes. The existing test only examines a set of test threads that are all blocked on a raw monitor. You do not need to duplicate that test. > OTOH we don't have testcase for GetThreadListStackTraces() with > thread_count == 1, so we need to testcase for it (it is > OneGetThreadListStackTraces.java) It would check whether the state of > target thread is "waiting" before JNI call to call > GetThreadListStackTraces(), Yes we need to test the special cases introduced by your changes - totally agree - and OneGetThreadListStackTraces.java is a good test for that. > and also I expect it would not be run with > JFR. (it is not described @run) The arguments to run with JFR (or a bunch of other things) can be passed to the jtreg test harness to be applied to all tests. > Of course we can check GetThreadListStackTraces() with main thread, but > it is not the test for direct handshake for other thread. Right - that test already exists as per the above. Thanks, David > > Thanks, > > Yasumasa > > >> Thanks, >> David >> ----- >> >> On 6/07/2020 11:29 pm, Yasumasa Suenaga wrote: >>> Hi Serguei, >>> >>> Thanks for your comment! >>> >>> I think C++ is more simple to implement the test agent as you said. >>> So I implement it in C++ in new webrev. Could you review again? >>> >>> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.06/ >>> >>> Also I refactored libGetThreadListStackTraces.cpp, and I've kept >>> exception check after IsSameObject(). >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2020/07/06 16:32, serguei.spitsyn at oracle.com wrote: >>>> Hi Yasumasa, >>>> >>>> Thank you for the update. >>>> I think, a pending exception after IsSameObject needs to be checked. >>>> >>>> The checkStackInfo() needs one more refactoring as I've already >>>> suggested. >>>> The body of the loop at L68-L78 should be converted to a function >>>> check_frame_info. >>>> The si1->frame_buffer[i] and si2->frame_buffer[i] will be passed as >>>> fi1 and fi2. >>>> The index can be passed as well. >>>> I'm still suggesting to simplify the local exception_msg to >>>> something shorter like err_msg or exc_msg. >>>> >>>> I'm not sure using fatal is right here: >>>> >>>> This fragment looks strange: >>>> >>>> ? 152???? if ((*env)->IsSameObject(env, stack_info[i].thread, >>>> thread)) { >>>> ? 153?????? target_info = &stack_info[i]; >>>> ? 154?????? break; >>>> ? 155???? } else if ((*env)->ExceptionOccurred(env)) { >>>> ? 156?????? (*env)->ExceptionDescribe(env); >>>> ? 157?????? (*env)->FatalError(env, __FILE__); >>>> ? 158???? } >>>> >>>> I expected it to be: >>>> >>>> ??? jboolean same = (*env)->IsSameObject(env, stack_info[i].thread, >>>> thread); >>>> ??? if ((*env)->ExceptionOccurred(env)) { >>>> ????? (*env)->ExceptionDescribe(env); >>>> ????? (*env)->FatalError(env, __FILE__); >>>> ??? } >>>> ??? if (same) { >>>> ????? target_info = &stack_info[i]; >>>> ????? break; >>>> ??? } >>>> >>>> Would it better to port this agent to C++ to simplify this code nicer? >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 7/5/20 06:13, Yasumasa Suenaga wrote: >>>>> Hi Serguei, >>>>> >>>>> Thanks for your comment! >>>>> I refactored testcase. Could you review again? >>>>> >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.05/ >>>>> >>>>> It would check Java exception after IsSameObject() call. Does it need? >>>>> Any exceptions are not described in JNI document[1], and JNI >>>>> implementation (jni_IsSameObject()) does not seem to throw it. >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> [1] >>>>> https://docs.oracle.com/en/java/javase/14/docs/specs/jni/functions.html#issameobject >>>>> >>>>> >>>>> >>>>> On 2020/07/05 14:46, serguei.spitsyn at oracle.com wrote: >>>>>> Hi Yasumasa, >>>>>> >>>>>> >>>>>> Okay, thanks. >>>>>> Then I'm okay to keep the GetSingleStackTraceClosure. >>>>>> >>>>>> >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c.html >>>>>> >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c.html >>>>>> >>>>>> >>>>>> I'm not sure the function 'is_same_thread() is needed. >>>>>> Why do not use the JNI IsSameObject instead? >>>>>> >>>>>> It seems to be a typo at L132 and L137. >>>>>> You, probably. did not want to print the same information for >>>>>> stack_info_1[i].frame_buffer[j].XXX twice. >>>>>> >>>>>> The code at lines 112-142 is not readable. >>>>>> I'd suggest to make a couple of refactoring steps. >>>>>> >>>>>> First step to simplify this a little bit would be with some >>>>>> renaming and getting rid of indexes: >>>>>> >>>>>> ?? 71 char err_msg[EXCEPTION_MSG_LEN] = {0}; >>>>>> ??... >>>>>> ??112?? /* Iterate all jvmtiStackInfo to check */ >>>>>> ??113?? for (i = 0; i < num_threads, *exception_msg != '\0'; i++) { >>>>>> ????????? jvmtiStackInfo *si1 = stack_info_1[i]; >>>>>> ????????? jvmtiStackInfo *si2 = stack_info_2[i]; >>>>>> ??114???? if (!IsSameObject(env, si1.thread, si2.thread)) { /* >>>>>> jvmtiStackInfo::thread */ >>>>>> ??115?????? snprintf(err_msg, sizeof(err_msg), >>>>>> ??116??????????????? "thread[%d] is different: stack_info_1 = %p, >>>>>> stack_info_2 = %p", >>>>>> ??117??????????????? i, sinfo1.thread, sinfo2.thread); >>>>>> ??118???? } else if (si1.state != si2.state) { /* >>>>>> jvmtiStackInfo::state */ >>>>>> ??119?????? snprintf(err_msg, sizeof(err_msg), >>>>>> ??120??????????????? "state[%d] is different: stack_info_1 = %d, >>>>>> stack_info_2 = %d", >>>>>> ??121??????????????? i, si1.state, si2.state); >>>>>> ??122???? } else if (si1.frame_count != si2.frame_count) { /* >>>>>> jvmtiStackInfo::frame_count */ >>>>>> ??123?????? snprintf(err_msg, sizeof(err_msg), >>>>>> ??124??????????????? "frame_count[%d] is different: stack_info_1 = >>>>>> %d, stack_info_2 = %d", >>>>>> ??125??????????????? i, si1.frame_count, si2.frame_count); >>>>>> ??126???? } else { >>>>>> ??127?????? /* Iterate all jvmtiFrameInfo to check */ >>>>>> ??128?????? for (j = 0; j < si1.frame_count; j++) { >>>>>> ??129???????? if (si1.frame_buffer[j].method != >>>>>> si1.frame_buffer[j].method) { /* jvmtiFrameInfo::method */ >>>>>> ??130?????????? snprintf(err_msg, sizeof(err_msg), >>>>>> ??131??????????????????? "thread [%d] frame_buffer[%d].method is >>>>>> different: stack_info_1 = %lx, stack_info_2 = %lx", >>>>>> ??132??????????????????? i, j, si1.frame_buffer[j].method, >>>>>> si2.frame_buffer[j].method); >>>>>> ??133?????????? break; >>>>>> ??134???????? } else if (si1.frame_buffer[j].location != >>>>>> si1.frame_buffer[j].location) { /* jvmtiFrameInfo::location */ >>>>>> ??135?????????? snprintf(err_msg, sizeof(err_msg), >>>>>> ??136??????????????????? "thread [%d] frame_buffer[%d].location is >>>>>> different: stack_info_1 = %ld, stack_info_2 = %ld", >>>>>> ??137??????????????????? i, j, si1.frame_buffer[j].location, >>>>>> si2.frame_buffer[j].location); >>>>>> ??138?????????? break; >>>>>> ??139???????? } >>>>>> ??140?????? } >>>>>> ??141???? } >>>>>> ??142?? } >>>>>> >>>>>> Another step would be to create functions that implement a body of >>>>>> each loop. >>>>>> You can use the same techniques to simplify similar place >>>>>> (L127-L138) in the libOneGetThreadListStackTraces.c. >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>>> >>>>>> On 7/3/20 15:55, Yasumasa Suenaga wrote: >>>>>>> Hi Serguei, >>>>>>> >>>>>>> I'm not an Oracle employee, so I cannot know real request(s) from >>>>>>> your customers. >>>>>>> However JDK-8201641 says Dynatrace has requested this enhancement. >>>>>>> >>>>>>> BTW I haven't heared any request from my customers about this. >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2020/07/04 4:32, serguei.spitsyn at oracle.com wrote: >>>>>>>> Hi Yasumasa, >>>>>>>> >>>>>>>> This difference is not that big to care about. >>>>>>>> I feel this is really rare case and so, does not worth these >>>>>>>> complications. >>>>>>>> Do we have a real request from customers to optimize it? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Serguei >>>>>>>> >>>>>>>> >>>>>>>> On 7/3/20 01:16, Yasumasa Suenaga wrote: >>>>>>>>> Hi Serguei, >>>>>>>>> >>>>>>>>> Generally I agree with you, but I have concern about the >>>>>>>>> difference of the result of GetStackTrace() and >>>>>>>>> GetThreadListStackTraces(). >>>>>>>>> >>>>>>>>> ? GetStackTrace: jvmtiFrameInfo >>>>>>>>> ? GetThreadListStackTraces: jvmtiStackInfo >>>>>>>>> >>>>>>>>> jvmtiStackInfo contains thread state, and it is ensured it is >>>>>>>>> the state of the call stack. >>>>>>>>> If we want to get both call stack and thread state, we need to >>>>>>>>> suspend target thread, and call both GetStackTrace() and >>>>>>>>> GetThreadState(). Is it ok? >>>>>>>>> >>>>>>>>> I was wondering if JDK-8201641 (parent ticket of this change) >>>>>>>>> needed them for profiling (dynatrace?) >>>>>>>>> If it is responsibility of JVMTI agent implementor, I remove >>>>>>>>> this closure. >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2020/07/03 16:45, serguei.spitsyn at oracle.com wrote: >>>>>>>>>> Hi Yasumasa, >>>>>>>>>> >>>>>>>>>> After some thinking I've concluded that I do not like this >>>>>>>>>> optimization >>>>>>>>>> of the GetThreadListStackTraces with GetSingleStackTraceClosure. >>>>>>>>>> >>>>>>>>>> We may need more opinions on this but these are my points: >>>>>>>>>> ??- it adds some complexity and ugliness >>>>>>>>>> ??- a win is doubtful because it has to be a rare case, so >>>>>>>>>> that total overhead should not be high >>>>>>>>>> ??- if it is really high for some use cases then it is up to >>>>>>>>>> the user >>>>>>>>>> ??? to optimize it with using GetStackTrace instead >>>>>>>>>> >>>>>>>>>> In such cases with doubtful overhead I usually prefer the >>>>>>>>>> simplicity. >>>>>>>>>> >>>>>>>>>> Good examples where it makes sense to optimize are checks for >>>>>>>>>> target thread to be current thread. >>>>>>>>>> In such cases there is no need to suspend the target thread, >>>>>>>>>> or use a VMop/HandshakeClosure. >>>>>>>>>> For instance, please, see the Monitor functions with the >>>>>>>>>> check: (java_thread == calling_thread). >>>>>>>>>> Getting information for current thread is frequently used >>>>>>>>>> case, e.g. to get info at an event point. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Serguei >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 7/2/20 23:29, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi Dan, David, >>>>>>>>>>> >>>>>>>>>>> I uploaded new webrev. Could you review again? >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/ >>>>>>>>>>> >>>>>>>>>>> OneGetThreadListStackTraces.java in this webrev would wait >>>>>>>>>>> until thread state is transited to "waiting" with spin wait. >>>>>>>>>>> CountDownLatch::await call as Dan pointed is fixed in it :) >>>>>>>>>>> >>>>>>>>>>> Diff from webrev.03: >>>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/c9aeb7001e50 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2020/07/03 14:15, David Holmes wrote: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>> On 2020/07/03 12:24, Daniel D. Daugherty wrote: >>>>>>>>>>>>>> On 7/2/20 10:50 PM, David Holmes wrote: >>>>>>>>>>>>>>> Sorry I'm responding here without seeing latest webrev >>>>>>>>>>>>>>> but there is enough context I think ... >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>> Hi Dan, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I upload new webrev. Could you review again? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>>>>>>>>> ???? L1542: ??? // Get stack trace with handshake >>>>>>>>>>>>>>>>> ???????? nit - please add a period at the end. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> ???? L1591: *stack_info_ptr = op.stack_info(); >>>>>>>>>>>>>>>>> ???????? The return parameter should not be touched >>>>>>>>>>>>>>>>> unless the return >>>>>>>>>>>>>>>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> ???? old L1582: ? if (err == JVMTI_ERROR_NONE) { >>>>>>>>>>>>>>>>> ???????? Please restore this check. The return >>>>>>>>>>>>>>>>> parameter should not >>>>>>>>>>>>>>>>> ???????? be touched unless the return code in 'err' == >>>>>>>>>>>>>>>>> JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> But op.stack_info() will return NULL if the error is not >>>>>>>>>>>>>>> JVMTI_ERROR_NONE. Are you (Dan) concerned about someone >>>>>>>>>>>>>>> passing in a non-null/initialized out-pointer that will >>>>>>>>>>>>>>> be reset to NULL if there was an error? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Actually the way we used to test this in POSIX tests is to >>>>>>>>>>>>>> call >>>>>>>>>>>>>> an API with known bad parameters and the return parameter ptr >>>>>>>>>>>>>> set to NULL. If the return parameter ptr was touched when an >>>>>>>>>>>>>> error should have been detected on an earlier parameter, then >>>>>>>>>>>>>> the test failed. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> ???? L1272: ? if (!jt->is_exiting() && (thread_oop != >>>>>>>>>>>>>>>>> NULL)) { >>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>> ???? old L1532: ? _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>>>>>>>>>>>>>> ???????? This deletion of the _result field threw me >>>>>>>>>>>>>>>>> for a minute and then >>>>>>>>>>>>>>>>> ???? ? ? I figured out that the field is init to >>>>>>>>>>>>>>>>> JVMTI_ERROR_THREAD_NOT_ALIVE >>>>>>>>>>>>>>>>> ???????? in the constructor. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> ???? L1553: ? if (!jt->is_exiting() && (jt->threadObj() >>>>>>>>>>>>>>>>> != NULL)) { >>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> ???? L64: ??????? startSignal.countDown(); >>>>>>>>>>>>>>>>> ???????? I was expecting this to be a call to await() >>>>>>>>>>>>>>>>> instead of >>>>>>>>>>>>>>>>> ???????? countDown(). What am I missing here? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> ???????? I think this test might be passing by accident >>>>>>>>>>>>>>>>> right now, but... >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Main thread (which call JVMTI functions to test) should >>>>>>>>>>>>>>>> wait until test thread is ready. >>>>>>>>>>>>>>>> So main thread would wait startSignal, and test thread >>>>>>>>>>>>>>>> would count down. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> No! >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The test thread that previously called obj.wait() now >>>>>>>>>>>>>>> calls latch.await(). >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The main thread that previously called obj.notify() now >>>>>>>>>>>>>>> calls latch.countDown(). >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The main thread continues to spin until it sees the >>>>>>>>>>>>>>> target is WAITING before proceeding with the test. >>>>>>>>>>>>> >>>>>>>>>>>>> If I add spin wait to wait until transit target thread >>>>>>>>>>>>> state is WAITING (as following), we don't need to call >>>>>>>>>>>>> SuspendThread(). >>>>>>>>>>>>> Which is better? >>>>>>>>>>>> >>>>>>>>>>>> The original spin-wait loop checking for? WAITING is better >>>>>>>>>>>> because it is the only guarantee that the target thread is >>>>>>>>>>>> blocked where you need it to be. suspending the thread is >>>>>>>>>>>> racy as you don't know exactly where the suspend will hit. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> David >>>>>>>>>>>> ----- >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> ``` >>>>>>>>>>>>> /* Wait until the thread state transits to "waiting" */ >>>>>>>>>>>>> while (th.getState() != Thread.State.WAITING) { >>>>>>>>>>>>> ???? Thread.onSpinWait(); >>>>>>>>>>>>> } >>>>>>>>>>>>> ``` >>>>>>>>>>>>> >>>>>>>>>>>>> For simplify, spin wait is prefer to >>>>>>>>>>>>> OneGetThreadListStackTraces.java in webrev.03. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> Here's the flow as I see it: >>>>>>>>>>>>>> >>>>>>>>>>>>>> main thread >>>>>>>>>>>>>> ?? - start worker thread >>>>>>>>>>>>>> ?? - startSignal.await() >>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>> >>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>> ?? - startSignal.countDown() >>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>> ?? - stopSignal.await() >>>>>>>>>>>>>> ???? - worker is now blocked >>>>>>>>>>>>>> >>>>>>>>>>>>>> main thread >>>>>>>>>>>>>> ?? - checkCallStacks(th) >>>>>>>>>>>>>> ?? - stopSignal.countDown() >>>>>>>>>>>>>> ???? - worker is now unblocked >>>>>>>>>>>>>> ?? - th.join >>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>> >>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>> ?? - runs off the end of run() >>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>> >>>>>>>>>>>>>> main thread >>>>>>>>>>>>>> ?? - run off the end of main() >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> ???? L92: ? jthreads = (jthread >>>>>>>>>>>>>>>>> *)malloc(sizeof(jthread) * num_threads); >>>>>>>>>>>>>>>>> ???????? You don't check for malloc() failure. >>>>>>>>>>>>>>>>> ???????? 'jthreads' is allocated but never freed. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> ???? L91: ? result = (*jvmti)->SuspendThread(jvmti, >>>>>>>>>>>>>>>>> thread); >>>>>>>>>>>>>>>>> ???????? Why are you suspending the thread? >>>>>>>>>>>>>>>>> GetAllStackTraces() and >>>>>>>>>>>>>>>>> ???????? GetThreadListStackTraces() do not require the >>>>>>>>>>>>>>>>> target thread(s) >>>>>>>>>>>>>>>>> ???????? to be suspend. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> ???????? If you decide not to SuspendThread, then you >>>>>>>>>>>>>>>>> don't need the >>>>>>>>>>>>>>>>> ???????? AddCapabilities or the ResumeThread calls. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Test thread might not be entered following code >>>>>>>>>>>>>>>> (stopSignal.await()). We might see deferent call stack >>>>>>>>>>>>>>>> between GetAllStackTraces() and >>>>>>>>>>>>>>>> GetThreadListStackTraces(). We cannot control to freeze >>>>>>>>>>>>>>>> call stack of test thread in Java code. >>>>>>>>>>>>>>>> (I didn't use SuspendThread() at first, but I saw some >>>>>>>>>>>>>>>> errors which causes in above.) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> So we need to call SuspendThread() to ensure we can see >>>>>>>>>>>>>>>> same call stack. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> If you are checking that the thread is in state WAITING >>>>>>>>>>>>>>> then it cannot escape from that state and you can sample >>>>>>>>>>>>>>> the stack multiple times from any API and get the same >>>>>>>>>>>>>>> result. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I suspect the errors you saw were from the apparent >>>>>>>>>>>>>>> incorrect use of the CountDownLatch. >>>>>>>>>>>>>> >>>>>>>>>>>>>> With the flow outlined above, the worker thread should be >>>>>>>>>>>>>> nicely blocked in stopSignal.await() when stuff is sampled. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Dan >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>> David >>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could review again? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Updates look fine - thanks. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> One minor nit: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> 1274 _collector.allocate_and_fill_stacks(1); >>>>>>>>>>>>>>>>>>> 1275 _collector.set_result(JVMTI_ERROR_NONE); >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> In the other places where you use _collector you rely >>>>>>>>>>>>>>>>>>> on result being initialized to JVMTI_ERROR_NONE, >>>>>>>>>>>>>>>>>>> rather than setting it directly after >>>>>>>>>>>>>>>>>>> allocate_and_fill_stacks(). >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> ? 820 >>>>>>>>>>>>>>>>>>>>>>>>> assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>> ? 821 >>>>>>>>>>>>>>>>>>>>>>>>> java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>>>>>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>> ? 822 current_thread == >>>>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>> ? 823????????? "at safepoint / handshake or >>>>>>>>>>>>>>>>>>>>>>>>> target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is >>>>>>>>>>>>>>>>>>>>>>>>> necessary, as even if the target is suspended >>>>>>>>>>>>>>>>>>>>>>>>> we must still be at a safepoint or in a >>>>>>>>>>>>>>>>>>>>>>>>> handshake with it. Makes me wonder if we used >>>>>>>>>>>>>>>>>>>>>>>>> to allow a racy stacktrace operation on a >>>>>>>>>>>>>>>>>>>>>>>>> suspended thread, assuming it would remain >>>>>>>>>>>>>>>>>>>>>>>>> suspended? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> This function (JvmtiEnvBase::get_stack_trace()) can >>>>>>>>>>>>>>>>>>>> be called to get own stack trace. For example, we >>>>>>>>>>>>>>>>>>>> can call GetStackTrace() for current thread at JVMTI >>>>>>>>>>>>>>>>>>>> event. >>>>>>>>>>>>>>>>>>>> So I changed assert as below: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>> ??820?? assert(current_thread == java_thread || >>>>>>>>>>>>>>>>>>>> ??821 SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>> ??822????????? current_thread == >>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>> ??823????????? "call by myself / at safepoint / at >>>>>>>>>>>>>>>>>>>> handshake"); >>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Yep good catch. I hope current tests caught that. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> They would be tested in >>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ (own >>>>>>>>>>>>>>>>>> call stacks), and getstacktr003 (call stacks in other >>>>>>>>>>>>>>>>>> thread). >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Speaking of tests ... >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> In the native code I think you need to check the >>>>>>>>>>>>>>>>>>> success of all JNI methods that can throw exceptions >>>>>>>>>>>>>>>>>>> - otherwise I believe the tests may trigger warnings >>>>>>>>>>>>>>>>>>> if -Xcheck:jni is used with them. See for example: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I updated testcases to check JNI and JVMTI function >>>>>>>>>>>>>>>>>> calls. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> In the Java code the target thread: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ?? 45???? public void run() { >>>>>>>>>>>>>>>>>>> ?? 46?????? try { >>>>>>>>>>>>>>>>>>> ?? 47?????????? synchronized (lock) { >>>>>>>>>>>>>>>>>>> ?? 48?????????????? lock.wait(); >>>>>>>>>>>>>>>>>>> ?? 49 System.out.println("OK"); >>>>>>>>>>>>>>>>>>> ?? 50?????????? } >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> is potentially susceptible to spurious wakeups. Using >>>>>>>>>>>>>>>>>>> a CountDownLatch would be robust. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the >>>>>>>>>>>>>>>>>>>>>>>>> current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> If so, we can call make_local() in L1272 without >>>>>>>>>>>>>>>>>>>>>> JavaThread (or we can pass current thread to >>>>>>>>>>>>>>>>>>>>>> make_local()). Is it right? >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>> 1272 >>>>>>>>>>>>>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>>>>>>>>>>>>> thread_oop), >>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Sorry I got confused, _calling_thread may not be >>>>>>>>>>>>>>>>>>>>> the current thread as we could be executing the >>>>>>>>>>>>>>>>>>>>> handshake in the target thread itself. So the >>>>>>>>>>>>>>>>>>>>> ResourceMark is correct as-is (implicitly for >>>>>>>>>>>>>>>>>>>>> current thread). >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> The argument to fill_frames will be used in the >>>>>>>>>>>>>>>>>>>>> jvmtiStackInfo and passed back to the >>>>>>>>>>>>>>>>>>>>> _calling_thread, so it must be created via >>>>>>>>>>>>>>>>>>>>> make_local(_calling_thread, ...) as you presently >>>>>>>>>>>>>>>>>>>>> have. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Thank you for reviewing! I will update new >>>>>>>>>>>>>>>>>>>>>>>> webrev tomorrow. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public >>>>>>>>>>>>>>>>>>>>>>>>> StackObj { >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> ? 498 class VM_GetAllStackTraces : public >>>>>>>>>>>>>>>>>>>>>>>>> VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>>>>>>>>>>>>>> ? 500?? JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>> ? 502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of >>>>>>>>>>>>>>>>>>>>>>>>> another class like that as it may not be on the >>>>>>>>>>>>>>>>>>>>>>>>> stack. I think MultipleStackTracesCollector >>>>>>>>>>>>>>>>>>>>>>>>> should not extend any allocation class, and >>>>>>>>>>>>>>>>>>>>>>>>> should always be embedded directly in another >>>>>>>>>>>>>>>>>>>>>>>>> class. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>>>>>>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>>>>>>>>>>>>>>> ??? : >>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> class GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>> ?? private: >>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> I updated webrev for 8242428. Could you review >>>>>>>>>>>>>>>>>>>>>>>>>> again? >>>>>>>>>>>>>>>>>>>>>>>>>> This change migrate to use direct handshake >>>>>>>>>>>>>>>>>>>>>>>>>> for GetStackTrace() and >>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() (when thread_count >>>>>>>>>>>>>>>>>>>>>>>>>> == 1). >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> This looks really good now! I only have a few >>>>>>>>>>>>>>>>>>>>>>>>> nits below. There is one thing I don't like >>>>>>>>>>>>>>>>>>>>>>>>> about it but it requires a change to the main >>>>>>>>>>>>>>>>>>>>>>>>> Handshake logic to address - in >>>>>>>>>>>>>>>>>>>>>>>>> JvmtiEnv::GetThreadListStackTraces you have to >>>>>>>>>>>>>>>>>>>>>>>>> create a ThreadsListHandle to convert the >>>>>>>>>>>>>>>>>>>>>>>>> jthread to a JavaThread, but then the >>>>>>>>>>>>>>>>>>>>>>>>> Handshake::execute_direct creates another >>>>>>>>>>>>>>>>>>>>>>>>> ThreadsListHandle internally. That's a waste. I >>>>>>>>>>>>>>>>>>>>>>>>> will discuss with Robbin and file a RFE to have >>>>>>>>>>>>>>>>>>>>>>>>> an overload of execute_direct that takes an >>>>>>>>>>>>>>>>>>>>>>>>> existing TLH. Actually it's worse than that >>>>>>>>>>>>>>>>>>>>>>>>> because we have another TLH in use at the entry >>>>>>>>>>>>>>>>>>>>>>>>> point for the JVMTI functions, so I think there >>>>>>>>>>>>>>>>>>>>>>>>> may be some scope for simplifying the use of >>>>>>>>>>>>>>>>>>>>>>>>> TLH instances - future RFE. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> ??451 GetStackTraceClosure(JvmtiEnv *env, jint >>>>>>>>>>>>>>>>>>>>>>>>> start_depth, jint max_count, >>>>>>>>>>>>>>>>>>>>>>>>> ??452 jvmtiFrameInfo* frame_buffer, jint* >>>>>>>>>>>>>>>>>>>>>>>>> count_ptr) >>>>>>>>>>>>>>>>>>>>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>>>>>>>>>>>>>> ??454?????? _env(env), >>>>>>>>>>>>>>>>>>>>>>>>> _start_depth(start_depth), _max_count(max_count), >>>>>>>>>>>>>>>>>>>>>>>>> ??455 _frame_buffer(frame_buffer), >>>>>>>>>>>>>>>>>>>>>>>>> _count_ptr(count_ptr), >>>>>>>>>>>>>>>>>>>>>>>>> ??456 _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Nit: can you do one initializer per line please. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public >>>>>>>>>>>>>>>>>>>>>>>>> StackObj { >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> ??498 class VM_GetAllStackTraces : public >>>>>>>>>>>>>>>>>>>>>>>>> VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>>>>>>>>>>>>>> ??500?? JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>> ??502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of >>>>>>>>>>>>>>>>>>>>>>>>> another class like that as it may not be on the >>>>>>>>>>>>>>>>>>>>>>>>> stack. I think MultipleStackTracesCollector >>>>>>>>>>>>>>>>>>>>>>>>> should not extend any allocation class, and >>>>>>>>>>>>>>>>>>>>>>>>> should always be embedded directly in another >>>>>>>>>>>>>>>>>>>>>>>>> class. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> 481 MultipleStackTracesCollector(JvmtiEnv *env, >>>>>>>>>>>>>>>>>>>>>>>>> jint max_frame_count) { >>>>>>>>>>>>>>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>>>>>>>>>>>>>> ??483???? _max_frame_count = max_frame_count; >>>>>>>>>>>>>>>>>>>>>>>>> ??484 _frame_count_total = 0; >>>>>>>>>>>>>>>>>>>>>>>>> ??485???? _head = NULL; >>>>>>>>>>>>>>>>>>>>>>>>> ??486???? _stack_info = NULL; >>>>>>>>>>>>>>>>>>>>>>>>> ??487???? _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>>>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> As you are touching this can you change it to >>>>>>>>>>>>>>>>>>>>>>>>> use an initializer list as you did for the >>>>>>>>>>>>>>>>>>>>>>>>> HandshakeClosure, and please keep one item per >>>>>>>>>>>>>>>>>>>>>>>>> line. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> ??820 >>>>>>>>>>>>>>>>>>>>>>>>> assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>> ??821 >>>>>>>>>>>>>>>>>>>>>>>>> java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>>>>>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == >>>>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>> ??823????????? "at safepoint / handshake or >>>>>>>>>>>>>>>>>>>>>>>>> target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is >>>>>>>>>>>>>>>>>>>>>>>>> necessary, as even if the target is suspended >>>>>>>>>>>>>>>>>>>>>>>>> we must still be at a safepoint or in a >>>>>>>>>>>>>>>>>>>>>>>>> handshake with it. Makes me wonder if we used >>>>>>>>>>>>>>>>>>>>>>>>> to allow a racy stacktrace operation on a >>>>>>>>>>>>>>>>>>>>>>>>> suspended thread, assuming it would remain >>>>>>>>>>>>>>>>>>>>>>>>> suspended? >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>>>>>>>>>>>>>>>>> 1269 >>>>>>>>>>>>>>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && >>>>>>>>>>>>>>>>>>>>>>>>> (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> 1272 >>>>>>>>>>>>>>>>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>>>>>>>>>>>>>>>> thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> It is frustrating that this entire call chain >>>>>>>>>>>>>>>>>>>>>>>>> started with a jthread reference, which we >>>>>>>>>>>>>>>>>>>>>>>>> converted to a JavaThread, only to eventually >>>>>>>>>>>>>>>>>>>>>>>>> need to convert it back to a jthread! I think >>>>>>>>>>>>>>>>>>>>>>>>> there is some scope for simplification here but >>>>>>>>>>>>>>>>>>>>>>>>> not as part of this change. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the >>>>>>>>>>>>>>>>>>>>>>>>> current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> I'm still pondering the test logic but wanted >>>>>>>>>>>>>>>>>>>>>>>>> to send this now. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetThreadListStackTrace (for >>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces) and >>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetAllStackTraces (for GetAllStackTraces) >>>>>>>>>>>>>>>>>>>>>>>>>> have inherited VM_GetMultipleStackTraces VM >>>>>>>>>>>>>>>>>>>>>>>>>> operation which provides the feature to >>>>>>>>>>>>>>>>>>>>>>>>>> generate jvmtiStackInfo. I modified >>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetMultipleStackTraces to a normal C++ >>>>>>>>>>>>>>>>>>>>>>>>>> class to share with HandshakeClosure for >>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces >>>>>>>>>>>>>>>>>>>>>>>>>> (GetSingleStackTraceClosure). >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Also I added new testcases which test >>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() with thread_count >>>>>>>>>>>>>>>>>>>>>>>>>> == 1 and with all threads. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in >>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jvmti serviceability/jdwp >>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti vmTestbase/nsk/jdi >>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> ?? JBS: >>>>>>>>>>>>>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>>>>>>>>>>>>>> ?? webrev: >>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> This change replace following VM operations >>>>>>>>>>>>>>>>>>>>>>>>>>> to direct handshake. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces >>>>>>>>>>>>>>>>>>>>>>>>>>> (GetThreadListStackTrace()) >>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTrace() uses direct >>>>>>>>>>>>>>>>>>>>>>>>>>> handshake if thread count == 1. In other case >>>>>>>>>>>>>>>>>>>>>>>>>>> (thread count > 1), it would be performed as >>>>>>>>>>>>>>>>>>>>>>>>>>> VM operation (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>>>>>>>>>>>>>> Caller of VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>> (JvmtiEnvThreadState::reset_current_location()) >>>>>>>>>>>>>>>>>>>>>>>>>>> might be called at safepoint. So I added >>>>>>>>>>>>>>>>>>>>>>>>>>> safepoint check in its caller. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in >>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jvmti serviceability/jdwp >>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti vmTestbase/nsk/jdi >>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/ns >>>>>>>>>>>>>>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Also I tested it on submit repo, then it has >>>>>>>>>>>>>>>>>>>>>>>>>>> execution error >>>>>>>>>>>>>>>>>>>>>>>>>>> (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) >>>>>>>>>>>>>>>>>>>>>>>>>>> due to dependency error. So I think it does >>>>>>>>>>>>>>>>>>>>>>>>>>> not occur by this change. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>> >>>> From chris.plummer at oracle.com Tue Jul 7 06:38:27 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 6 Jul 2020 23:38:27 -0700 Subject: RFR(XS): 8248878: SA: Implement simple workaround for JDK-8248876 In-Reply-To: References: <39ac40d6-a738-3214-d86c-412152a631d4@oracle.com> Message-ID: <8e89d223-0f65-f26e-3bd7-2ca7421a940b@oracle.com> Hi Yasumasa, Thanks for the review. I tried the following for line 188: ??? if ((phdr->p_type == PT_LOAD || phdr->p_type == PT_INTERP) && phdr->p_vaddr < baseaddr) { However, "base" still ended up being 0. I added some printfs. For the exec file there is both a PT_INTER with p_vaddr of 0x238 and a PT_LOAD with p_vaddr 0. I'm not sure which to use, but in either case that won't be the proper base when added to 0: ? if (add_lib_info_fd(ph, exec_file, ph->core->exec_fd, ????????????????????? (uintptr_t)0 + find_base_address(ph->core->exec_fd, &exec_ehdr)) == NULL) { ??? goto err; ? } So maybe it's the (uintptr_t)0 that is the problem here. For shared libs instead of 0 it computes the value to add: ?????????????? if (lib_base_diff == ZERO_LOAD_ADDRESS ) { ???????????????? lib_base_diff = calc_prelinked_load_address(ph, lib_fd, &elf_ehdr, link_map_addr); ???????????????? if (lib_base_diff == INVALID_LOAD_ADDRESS) { ?????????????????? close(lib_fd); ?????????????????? return false; ???????????????? } ?????????????? } ?????????????? lib_base = lib_base_diff + find_base_address(lib_fd, &elf_ehdr); So in this case we've actually computed lib_base_diff rather than just assumed 0. Chris On 7/6/20 10:46 PM, Yasumasa Suenaga wrote: > Hi Chris, > > Your change looks good. > > > BTW I saw JDK-8248876. I'm not sure, but I guess we can fix this issue > if we allow PT_INTERP in L118: > > ``` > 105 uintptr_t find_base_address(int fd, ELF_EHDR* ehdr) { > ????????????????? : > 115?? // the base address of a shared object is the lowest vaddr of > 116?? // its loadable segments (PT_LOAD) > 117?? for (phdr = phbuf, cnt = 0; cnt < ehdr->e_phnum; cnt++, phdr++) { > 118???? if (phdr->p_type == PT_LOAD && phdr->p_vaddr < baseaddr) { > 119?????? baseaddr = phdr->p_vaddr; > 120???? } > 121?? } > ``` > > /proc//maps shows top of `java` is 0x56543b9df000: > > 56543b9df000-56543b9e0000 r--p 00000000 08:10 55770 > /usr/lib/jvm/java-11-openjdk-amd64/bin/java > > > `i target` on GDB shows 0x56543b9df000 is .interp section: > > Local exec file: > ??????? `/usr/lib/jvm/java-11-openjdk-amd64/bin/java', file type > elf64-x86-64. > ??????? Entry point: 0x56543b9e0330 > ??????? 0x000056543b9df318 - 0x000056543b9df334 is .interp > > > Thanks, > > Yasumasa > > > On 2020/07/07 13:18, Chris Plummer wrote: >> Hello, >> >> Please help review the following: >> >> http://cr.openjdk.java.net/~cjplummer/8248878/webrev.00/index.html >> https://bugs.openjdk.java.net/browse/JDK-8248878 >> >> The explanation of the fix is in the CR. The parent CR, JDK-8248876 >> [1], explains the issue being addressed. >> >> There's no test for this fix yet. It requires the changes I'm making >> for JDK-8247514 [2], which include changes to "findpc" support and >> the ClhsdbFindPC.java test that trigger this issue. >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8248876 >> [2] https://bugs.openjdk.java.net/browse/JDK-8247514 >> >> thanks, >> >> Chris From serguei.spitsyn at oracle.com Tue Jul 7 07:43:29 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 7 Jul 2020 00:43:29 -0700 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: <8c387550-a06e-6320-f453-0e898486befb@oss.nttdata.com> References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <763793b1-9506-44c6-0681-1c71c44a44fd@oss.nttdata.com> <5107c478-1de2-7a49-ee71-dfad6a99a667@oracle.com> <90cdfef9-e558-0d25-33f5-25b7d65bb83b@oss.nttdata.com> <0e359819-618f-c17b-a427-bb7e4daa2c58@oracle.com> <3f890bed-6892-e64d-b21b-ca0cbe11784e@oracle.com> <7dc1903b-6295-3864-6301-4c02aacdb971@oss.nttdata.com> <04e09eb7-9483-8f63-4bb4-01c9e814802e@oracle.com> <238a89a3-06b1-90e4-d5a4-8dc2f1ed12ab@oracle.com> <4c43bc2c-551e-d042-8fa7-b63e755888d1@oracle.com> <8c387550-a06e-6320-f453-0e898486befb@oss.nttdata.com> Message-ID: Hi Yasumasa, Thank you for this update - the test looks much better and is readable now. I'm on vacation this week but will try to look at your fixes a little bit more. As I understand you are going to post one more update. Thanks, Serguei On 7/6/20 06:29, Yasumasa Suenaga wrote: > Hi Serguei, > > Thanks for your comment! > > I think C++ is more simple to implement the test agent as you said. > So I implement it in C++ in new webrev. Could you review again? > > ? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.06/ > > Also I refactored libGetThreadListStackTraces.cpp, and I've kept > exception check after IsSameObject(). > > > Thanks, > > Yasumasa > > > On 2020/07/06 16:32, serguei.spitsyn at oracle.com wrote: >> Hi Yasumasa, >> >> Thank you for the update. >> I think, a pending exception after IsSameObject needs to be checked. >> >> The checkStackInfo() needs one more refactoring as I've already >> suggested. >> The body of the loop at L68-L78 should be converted to a function >> check_frame_info. >> The si1->frame_buffer[i] and si2->frame_buffer[i] will be passed as >> fi1 and fi2. >> The index can be passed as well. >> I'm still suggesting to simplify the local exception_msg to something >> shorter like err_msg or exc_msg. >> >> I'm not sure using fatal is right here: >> >> This fragment looks strange: >> >> ? 152???? if ((*env)->IsSameObject(env, stack_info[i].thread, thread)) { >> ? 153?????? target_info = &stack_info[i]; >> ? 154?????? break; >> ? 155???? } else if ((*env)->ExceptionOccurred(env)) { >> ? 156?????? (*env)->ExceptionDescribe(env); >> ? 157?????? (*env)->FatalError(env, __FILE__); >> ? 158???? } >> >> I expected it to be: >> >> ??? jboolean same = (*env)->IsSameObject(env, stack_info[i].thread, >> thread); >> ??? if ((*env)->ExceptionOccurred(env)) { >> ????? (*env)->ExceptionDescribe(env); >> ????? (*env)->FatalError(env, __FILE__); >> ??? } >> ??? if (same) { >> ????? target_info = &stack_info[i]; >> ????? break; >> ??? } >> >> Would it better to port this agent to C++ to simplify this code nicer? >> >> Thanks, >> Serguei >> >> >> On 7/5/20 06:13, Yasumasa Suenaga wrote: >>> Hi Serguei, >>> >>> Thanks for your comment! >>> I refactored testcase. Could you review again? >>> >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.05/ >>> >>> It would check Java exception after IsSameObject() call. Does it need? >>> Any exceptions are not described in JNI document[1], and JNI >>> implementation (jni_IsSameObject()) does not seem to throw it. >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> [1] >>> https://docs.oracle.com/en/java/javase/14/docs/specs/jni/functions.html#issameobject >>> >>> >>> On 2020/07/05 14:46, serguei.spitsyn at oracle.com wrote: >>>> Hi Yasumasa, >>>> >>>> >>>> Okay, thanks. >>>> Then I'm okay to keep the GetSingleStackTraceClosure. >>>> >>>> >>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c.html >>>> >>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c.html >>>> >>>> >>>> I'm not sure the function 'is_same_thread() is needed. >>>> Why do not use the JNI IsSameObject instead? >>>> >>>> It seems to be a typo at L132 and L137. >>>> You, probably. did not want to print the same information for >>>> stack_info_1[i].frame_buffer[j].XXX twice. >>>> >>>> The code at lines 112-142 is not readable. >>>> I'd suggest to make a couple of refactoring steps. >>>> >>>> First step to simplify this a little bit would be with some >>>> renaming and getting rid of indexes: >>>> >>>> ?? 71 char err_msg[EXCEPTION_MSG_LEN] = {0}; >>>> ??... >>>> ??112?? /* Iterate all jvmtiStackInfo to check */ >>>> ??113?? for (i = 0; i < num_threads, *exception_msg != '\0'; i++) { >>>> ????????? jvmtiStackInfo *si1 = stack_info_1[i]; >>>> ????????? jvmtiStackInfo *si2 = stack_info_2[i]; >>>> ??114???? if (!IsSameObject(env, si1.thread, si2.thread)) { /* >>>> jvmtiStackInfo::thread */ >>>> ??115?????? snprintf(err_msg, sizeof(err_msg), >>>> ??116??????????????? "thread[%d] is different: stack_info_1 = %p, >>>> stack_info_2 = %p", >>>> ??117??????????????? i, sinfo1.thread, sinfo2.thread); >>>> ??118???? } else if (si1.state != si2.state) { /* >>>> jvmtiStackInfo::state */ >>>> ??119?????? snprintf(err_msg, sizeof(err_msg), >>>> ??120??????????????? "state[%d] is different: stack_info_1 = %d, >>>> stack_info_2 = %d", >>>> ??121??????????????? i, si1.state, si2.state); >>>> ??122???? } else if (si1.frame_count != si2.frame_count) { /* >>>> jvmtiStackInfo::frame_count */ >>>> ??123?????? snprintf(err_msg, sizeof(err_msg), >>>> ??124??????????????? "frame_count[%d] is different: stack_info_1 = >>>> %d, stack_info_2 = %d", >>>> ??125??????????????? i, si1.frame_count, si2.frame_count); >>>> ??126???? } else { >>>> ??127?????? /* Iterate all jvmtiFrameInfo to check */ >>>> ??128?????? for (j = 0; j < si1.frame_count; j++) { >>>> ??129???????? if (si1.frame_buffer[j].method != >>>> si1.frame_buffer[j].method) { /* jvmtiFrameInfo::method */ >>>> ??130?????????? snprintf(err_msg, sizeof(err_msg), >>>> ??131??????????????????? "thread [%d] frame_buffer[%d].method is >>>> different: stack_info_1 = %lx, stack_info_2 = %lx", >>>> ??132??????????????????? i, j, si1.frame_buffer[j].method, >>>> si2.frame_buffer[j].method); >>>> ??133?????????? break; >>>> ??134???????? } else if (si1.frame_buffer[j].location != >>>> si1.frame_buffer[j].location) { /* jvmtiFrameInfo::location */ >>>> ??135?????????? snprintf(err_msg, sizeof(err_msg), >>>> ??136??????????????????? "thread [%d] frame_buffer[%d].location is >>>> different: stack_info_1 = %ld, stack_info_2 = %ld", >>>> ??137??????????????????? i, j, si1.frame_buffer[j].location, >>>> si2.frame_buffer[j].location); >>>> ??138?????????? break; >>>> ??139???????? } >>>> ??140?????? } >>>> ??141???? } >>>> ??142?? } >>>> >>>> Another step would be to create functions that implement a body of >>>> each loop. >>>> You can use the same techniques to simplify similar place >>>> (L127-L138) in the libOneGetThreadListStackTraces.c. >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 7/3/20 15:55, Yasumasa Suenaga wrote: >>>>> Hi Serguei, >>>>> >>>>> I'm not an Oracle employee, so I cannot know real request(s) from >>>>> your customers. >>>>> However JDK-8201641 says Dynatrace has requested this enhancement. >>>>> >>>>> BTW I haven't heared any request from my customers about this. >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2020/07/04 4:32, serguei.spitsyn at oracle.com wrote: >>>>>> Hi Yasumasa, >>>>>> >>>>>> This difference is not that big to care about. >>>>>> I feel this is really rare case and so, does not worth these >>>>>> complications. >>>>>> Do we have a real request from customers to optimize it? >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>>> >>>>>> On 7/3/20 01:16, Yasumasa Suenaga wrote: >>>>>>> Hi Serguei, >>>>>>> >>>>>>> Generally I agree with you, but I have concern about the >>>>>>> difference of the result of GetStackTrace() and >>>>>>> GetThreadListStackTraces(). >>>>>>> >>>>>>> ? GetStackTrace: jvmtiFrameInfo >>>>>>> ? GetThreadListStackTraces: jvmtiStackInfo >>>>>>> >>>>>>> jvmtiStackInfo contains thread state, and it is ensured it is >>>>>>> the state of the call stack. >>>>>>> If we want to get both call stack and thread state, we need to >>>>>>> suspend target thread, and call both GetStackTrace() and >>>>>>> GetThreadState(). Is it ok? >>>>>>> >>>>>>> I was wondering if JDK-8201641 (parent ticket of this change) >>>>>>> needed them for profiling (dynatrace?) >>>>>>> If it is responsibility of JVMTI agent implementor, I remove >>>>>>> this closure. >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2020/07/03 16:45, serguei.spitsyn at oracle.com wrote: >>>>>>>> Hi Yasumasa, >>>>>>>> >>>>>>>> After some thinking I've concluded that I do not like this >>>>>>>> optimization >>>>>>>> of the GetThreadListStackTraces with GetSingleStackTraceClosure. >>>>>>>> >>>>>>>> We may need more opinions on this but these are my points: >>>>>>>> ??- it adds some complexity and ugliness >>>>>>>> ??- a win is doubtful because it has to be a rare case, so that >>>>>>>> total overhead should not be high >>>>>>>> ??- if it is really high for some use cases then it is up to >>>>>>>> the user >>>>>>>> ??? to optimize it with using GetStackTrace instead >>>>>>>> >>>>>>>> In such cases with doubtful overhead I usually prefer the >>>>>>>> simplicity. >>>>>>>> >>>>>>>> Good examples where it makes sense to optimize are checks for >>>>>>>> target thread to be current thread. >>>>>>>> In such cases there is no need to suspend the target thread, or >>>>>>>> use a VMop/HandshakeClosure. >>>>>>>> For instance, please, see the Monitor functions with the check: >>>>>>>> (java_thread == calling_thread). >>>>>>>> Getting information for current thread is frequently used case, >>>>>>>> e.g. to get info at an event point. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Serguei >>>>>>>> >>>>>>>> >>>>>>>> On 7/2/20 23:29, Yasumasa Suenaga wrote: >>>>>>>>> Hi Dan, David, >>>>>>>>> >>>>>>>>> I uploaded new webrev. Could you review again? >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/ >>>>>>>>> >>>>>>>>> OneGetThreadListStackTraces.java in this webrev would wait >>>>>>>>> until thread state is transited to "waiting" with spin wait. >>>>>>>>> CountDownLatch::await call as Dan pointed is fixed in it :) >>>>>>>>> >>>>>>>>> Diff from webrev.03: >>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/c9aeb7001e50 >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2020/07/03 14:15, David Holmes wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: >>>>>>>>>>> On 2020/07/03 12:24, Daniel D. Daugherty wrote: >>>>>>>>>>>> On 7/2/20 10:50 PM, David Holmes wrote: >>>>>>>>>>>>> Sorry I'm responding here without seeing latest webrev but >>>>>>>>>>>>> there is enough context I think ... >>>>>>>>>>>>> >>>>>>>>>>>>> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>> Hi Dan, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I upload new webrev. Could you review again? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>>>>>>> ???? L1542: ??? // Get stack trace with handshake >>>>>>>>>>>>>>> ???????? nit - please add a period at the end. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> ???? L1591: *stack_info_ptr = op.stack_info(); >>>>>>>>>>>>>>> ???????? The return parameter should not be touched >>>>>>>>>>>>>>> unless the return >>>>>>>>>>>>>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ???? old L1582: ? if (err == JVMTI_ERROR_NONE) { >>>>>>>>>>>>>>> ???????? Please restore this check. The return parameter >>>>>>>>>>>>>>> should not >>>>>>>>>>>>>>> ???????? be touched unless the return code in 'err' == >>>>>>>>>>>>>>> JVMTI_ERROR_NONE. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>> >>>>>>>>>>>>> But op.stack_info() will return NULL if the error is not >>>>>>>>>>>>> JVMTI_ERROR_NONE. Are you (Dan) concerned about someone >>>>>>>>>>>>> passing in a non-null/initialized out-pointer that will be >>>>>>>>>>>>> reset to NULL if there was an error? >>>>>>>>>>>> >>>>>>>>>>>> Actually the way we used to test this in POSIX tests is to >>>>>>>>>>>> call >>>>>>>>>>>> an API with known bad parameters and the return parameter ptr >>>>>>>>>>>> set to NULL. If the return parameter ptr was touched when an >>>>>>>>>>>> error should have been detected on an earlier parameter, then >>>>>>>>>>>> the test failed. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> ???? L1272: ? if (!jt->is_exiting() && (thread_oop != >>>>>>>>>>>>>>> NULL)) { >>>>>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>> ???? old L1532: ? _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>>>>>>>>>>>> ???????? This deletion of the _result field threw me for >>>>>>>>>>>>>>> a minute and then >>>>>>>>>>>>>>> ???? ? ? I figured out that the field is init to >>>>>>>>>>>>>>> JVMTI_ERROR_THREAD_NOT_ALIVE >>>>>>>>>>>>>>> ???????? in the constructor. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ???? L1553: ? if (!jt->is_exiting() && (jt->threadObj() >>>>>>>>>>>>>>> != NULL)) { >>>>>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ???? L64: startSignal.countDown(); >>>>>>>>>>>>>>> ???????? I was expecting this to be a call to await() >>>>>>>>>>>>>>> instead of >>>>>>>>>>>>>>> ???????? countDown(). What am I missing here? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ???????? I think this test might be passing by accident >>>>>>>>>>>>>>> right now, but... >>>>>>>>>>>>>> >>>>>>>>>>>>>> Main thread (which call JVMTI functions to test) should >>>>>>>>>>>>>> wait until test thread is ready. >>>>>>>>>>>>>> So main thread would wait startSignal, and test thread >>>>>>>>>>>>>> would count down. >>>>>>>>>>>>> >>>>>>>>>>>>> No! >>>>>>>>>>>>> >>>>>>>>>>>>> The test thread that previously called obj.wait() now >>>>>>>>>>>>> calls latch.await(). >>>>>>>>>>>>> >>>>>>>>>>>>> The main thread that previously called obj.notify() now >>>>>>>>>>>>> calls latch.countDown(). >>>>>>>>>>>>> >>>>>>>>>>>>> The main thread continues to spin until it sees the target >>>>>>>>>>>>> is WAITING before proceeding with the test. >>>>>>>>>>> >>>>>>>>>>> If I add spin wait to wait until transit target thread state >>>>>>>>>>> is WAITING (as following), we don't need to call >>>>>>>>>>> SuspendThread(). >>>>>>>>>>> Which is better? >>>>>>>>>> >>>>>>>>>> The original spin-wait loop checking for WAITING is better >>>>>>>>>> because it is the only guarantee that the target thread is >>>>>>>>>> blocked where you need it to be. suspending the thread is >>>>>>>>>> racy as you don't know exactly where the suspend will hit. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> David >>>>>>>>>> ----- >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> ``` >>>>>>>>>>> /* Wait until the thread state transits to "waiting" */ >>>>>>>>>>> while (th.getState() != Thread.State.WAITING) { >>>>>>>>>>> ???? Thread.onSpinWait(); >>>>>>>>>>> } >>>>>>>>>>> ``` >>>>>>>>>>> >>>>>>>>>>> For simplify, spin wait is prefer to >>>>>>>>>>> OneGetThreadListStackTraces.java in webrev.03. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> Here's the flow as I see it: >>>>>>>>>>>> >>>>>>>>>>>> main thread >>>>>>>>>>>> ?? - start worker thread >>>>>>>>>>>> ?? - startSignal.await() >>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>> >>>>>>>>>>>> worker thread >>>>>>>>>>>> ?? - startSignal.countDown() >>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>> ?? - stopSignal.await() >>>>>>>>>>>> ???? - worker is now blocked >>>>>>>>>>>> >>>>>>>>>>>> main thread >>>>>>>>>>>> ?? - checkCallStacks(th) >>>>>>>>>>>> ?? - stopSignal.countDown() >>>>>>>>>>>> ???? - worker is now unblocked >>>>>>>>>>>> ?? - th.join >>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>> >>>>>>>>>>>> worker thread >>>>>>>>>>>> ?? - runs off the end of run() >>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>> >>>>>>>>>>>> main thread >>>>>>>>>>>> ?? - run off the end of main() >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ???? L92: ? jthreads = (jthread *)malloc(sizeof(jthread) >>>>>>>>>>>>>>> * num_threads); >>>>>>>>>>>>>>> ???????? You don't check for malloc() failure. >>>>>>>>>>>>>>> ???????? 'jthreads' is allocated but never freed. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ???? L91: ? result = (*jvmti)->SuspendThread(jvmti, >>>>>>>>>>>>>>> thread); >>>>>>>>>>>>>>> ???????? Why are you suspending the thread? >>>>>>>>>>>>>>> GetAllStackTraces() and >>>>>>>>>>>>>>> ???????? GetThreadListStackTraces() do not require the >>>>>>>>>>>>>>> target thread(s) >>>>>>>>>>>>>>> ???????? to be suspend. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ???????? If you decide not to SuspendThread, then you >>>>>>>>>>>>>>> don't need the >>>>>>>>>>>>>>> ???????? AddCapabilities or the ResumeThread calls. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Test thread might not be entered following code >>>>>>>>>>>>>> (stopSignal.await()). We might see deferent call stack >>>>>>>>>>>>>> between GetAllStackTraces() and >>>>>>>>>>>>>> GetThreadListStackTraces(). We cannot control to freeze >>>>>>>>>>>>>> call stack of test thread in Java code. >>>>>>>>>>>>>> (I didn't use SuspendThread() at first, but I saw some >>>>>>>>>>>>>> errors which causes in above.) >>>>>>>>>>>>>> >>>>>>>>>>>>>> So we need to call SuspendThread() to ensure we can see >>>>>>>>>>>>>> same call stack. >>>>>>>>>>>>> >>>>>>>>>>>>> If you are checking that the thread is in state WAITING >>>>>>>>>>>>> then it cannot escape from that state and you can sample >>>>>>>>>>>>> the stack multiple times from any API and get the same >>>>>>>>>>>>> result. >>>>>>>>>>>>> >>>>>>>>>>>>> I suspect the errors you saw were from the apparent >>>>>>>>>>>>> incorrect use of the CountDownLatch. >>>>>>>>>>>> >>>>>>>>>>>> With the flow outlined above, the worker thread should be >>>>>>>>>>>> nicely blocked in stopSignal.await() when stuff is sampled. >>>>>>>>>>>> >>>>>>>>>>>> Dan >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Cheers, >>>>>>>>>>>>> David >>>>>>>>>>>>> ----- >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I uploaded new webrev. Could review again? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Updates look fine - thanks. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> One minor nit: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 1274 _collector.allocate_and_fill_stacks(1); >>>>>>>>>>>>>>>>> 1275 _collector.set_result(JVMTI_ERROR_NONE); >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> In the other places where you use _collector you rely >>>>>>>>>>>>>>>>> on result being initialized to JVMTI_ERROR_NONE, >>>>>>>>>>>>>>>>> rather than setting it directly after >>>>>>>>>>>>>>>>> allocate_and_fill_stacks(). >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ? 820 >>>>>>>>>>>>>>>>>>>>>>> assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>> ? 821 >>>>>>>>>>>>>>>>>>>>>>> java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>>>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>> ? 822 current_thread == >>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>> ? 823????????? "at safepoint / handshake or >>>>>>>>>>>>>>>>>>>>>>> target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, >>>>>>>>>>>>>>>>>>>>>>> as even if the target is suspended we must still >>>>>>>>>>>>>>>>>>>>>>> be at a safepoint or in a handshake with it. >>>>>>>>>>>>>>>>>>>>>>> Makes me wonder if we used to allow a racy >>>>>>>>>>>>>>>>>>>>>>> stacktrace operation on a suspended thread, >>>>>>>>>>>>>>>>>>>>>>> assuming it would remain suspended? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> This function (JvmtiEnvBase::get_stack_trace()) can >>>>>>>>>>>>>>>>>> be called to get own stack trace. For example, we can >>>>>>>>>>>>>>>>>> call GetStackTrace() for current thread at JVMTI event. >>>>>>>>>>>>>>>>>> So I changed assert as below: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>> ??820?? assert(current_thread == java_thread || >>>>>>>>>>>>>>>>>> ??821 SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>> ??822????????? current_thread == >>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>> ??823????????? "call by myself / at safepoint / at >>>>>>>>>>>>>>>>>> handshake"); >>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Yep good catch. I hope current tests caught that. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> They would be tested in >>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ (own >>>>>>>>>>>>>>>> call stacks), and getstacktr003 (call stacks in other >>>>>>>>>>>>>>>> thread). >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Speaking of tests ... >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> In the native code I think you need to check the >>>>>>>>>>>>>>>>> success of all JNI methods that can throw exceptions - >>>>>>>>>>>>>>>>> otherwise I believe the tests may trigger warnings if >>>>>>>>>>>>>>>>> -Xcheck:jni is used with them. See for example: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I updated testcases to check JNI and JVMTI function calls. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> In the Java code the target thread: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> ?? 45???? public void run() { >>>>>>>>>>>>>>>>> ?? 46?????? try { >>>>>>>>>>>>>>>>> ?? 47?????????? synchronized (lock) { >>>>>>>>>>>>>>>>> ?? 48?????????????? lock.wait(); >>>>>>>>>>>>>>>>> ?? 49 System.out.println("OK"); >>>>>>>>>>>>>>>>> ?? 50?????????? } >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> is potentially susceptible to spurious wakeups. Using >>>>>>>>>>>>>>>>> a CountDownLatch would be robust. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the >>>>>>>>>>>>>>>>>>>>>>> current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> If so, we can call make_local() in L1272 without >>>>>>>>>>>>>>>>>>>> JavaThread (or we can pass current thread to >>>>>>>>>>>>>>>>>>>> make_local()). Is it right? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>>>>>>>>> 1272 >>>>>>>>>>>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>>>>>>>>>>> thread_oop), >>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Sorry I got confused, _calling_thread may not be the >>>>>>>>>>>>>>>>>>> current thread as we could be executing the >>>>>>>>>>>>>>>>>>> handshake in the target thread itself. So the >>>>>>>>>>>>>>>>>>> ResourceMark is correct as-is (implicitly for >>>>>>>>>>>>>>>>>>> current thread). >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> The argument to fill_frames will be used in the >>>>>>>>>>>>>>>>>>> jvmtiStackInfo and passed back to the >>>>>>>>>>>>>>>>>>> _calling_thread, so it must be created via >>>>>>>>>>>>>>>>>>> make_local(_calling_thread, ...) as you presently have. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Thank you for reviewing! I will update new webrev >>>>>>>>>>>>>>>>>>>>>> tomorrow. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public >>>>>>>>>>>>>>>>>>>>>>> StackObj { >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ? 498 class VM_GetAllStackTraces : public >>>>>>>>>>>>>>>>>>>>>>> VM_Operation { >>>>>>>>>>>>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>>>>>>>>>>>> ? 500?? JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>> ? 502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another >>>>>>>>>>>>>>>>>>>>>>> class like that as it may not be on the stack. I >>>>>>>>>>>>>>>>>>>>>>> think MultipleStackTracesCollector should not >>>>>>>>>>>>>>>>>>>>>>> extend any allocation class, and should always >>>>>>>>>>>>>>>>>>>>>>> be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>>>>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>>>>>>>>>>>>> ??? : >>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> class GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>> ?? private: >>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> I updated webrev for 8242428. Could you review >>>>>>>>>>>>>>>>>>>>>>>> again? >>>>>>>>>>>>>>>>>>>>>>>> This change migrate to use direct handshake for >>>>>>>>>>>>>>>>>>>>>>>> GetStackTrace() and GetThreadListStackTraces() >>>>>>>>>>>>>>>>>>>>>>>> (when thread_count == 1). >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> This looks really good now! I only have a few >>>>>>>>>>>>>>>>>>>>>>> nits below. There is one thing I don't like >>>>>>>>>>>>>>>>>>>>>>> about it but it requires a change to the main >>>>>>>>>>>>>>>>>>>>>>> Handshake logic to address - in >>>>>>>>>>>>>>>>>>>>>>> JvmtiEnv::GetThreadListStackTraces you have to >>>>>>>>>>>>>>>>>>>>>>> create a ThreadsListHandle to convert the >>>>>>>>>>>>>>>>>>>>>>> jthread to a JavaThread, but then the >>>>>>>>>>>>>>>>>>>>>>> Handshake::execute_direct creates another >>>>>>>>>>>>>>>>>>>>>>> ThreadsListHandle internally. That's a waste. I >>>>>>>>>>>>>>>>>>>>>>> will discuss with Robbin and file a RFE to have >>>>>>>>>>>>>>>>>>>>>>> an overload of execute_direct that takes an >>>>>>>>>>>>>>>>>>>>>>> existing TLH. Actually it's worse than that >>>>>>>>>>>>>>>>>>>>>>> because we have another TLH in use at the entry >>>>>>>>>>>>>>>>>>>>>>> point for the JVMTI functions, so I think there >>>>>>>>>>>>>>>>>>>>>>> may be some scope for simplifying the use of TLH >>>>>>>>>>>>>>>>>>>>>>> instances - future RFE. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ??451 GetStackTraceClosure(JvmtiEnv *env, jint >>>>>>>>>>>>>>>>>>>>>>> start_depth, jint max_count, >>>>>>>>>>>>>>>>>>>>>>> ??452 jvmtiFrameInfo* frame_buffer, jint* >>>>>>>>>>>>>>>>>>>>>>> count_ptr) >>>>>>>>>>>>>>>>>>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>>>>>>>>>>>> ??454?????? _env(env), >>>>>>>>>>>>>>>>>>>>>>> _start_depth(start_depth), _max_count(max_count), >>>>>>>>>>>>>>>>>>>>>>> ??455 _frame_buffer(frame_buffer), >>>>>>>>>>>>>>>>>>>>>>> _count_ptr(count_ptr), >>>>>>>>>>>>>>>>>>>>>>> ??456 _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Nit: can you do one initializer per line please. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public >>>>>>>>>>>>>>>>>>>>>>> StackObj { >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ??498 class VM_GetAllStackTraces : public >>>>>>>>>>>>>>>>>>>>>>> VM_Operation { >>>>>>>>>>>>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>>>>>>>>>>>> ??500?? JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>> ??502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another >>>>>>>>>>>>>>>>>>>>>>> class like that as it may not be on the stack. I >>>>>>>>>>>>>>>>>>>>>>> think MultipleStackTracesCollector should not >>>>>>>>>>>>>>>>>>>>>>> extend any allocation class, and should always >>>>>>>>>>>>>>>>>>>>>>> be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> 481 MultipleStackTracesCollector(JvmtiEnv *env, >>>>>>>>>>>>>>>>>>>>>>> jint max_frame_count) { >>>>>>>>>>>>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>>>>>>>>>>>> ??483 _max_frame_count = max_frame_count; >>>>>>>>>>>>>>>>>>>>>>> ??484 _frame_count_total = 0; >>>>>>>>>>>>>>>>>>>>>>> ??485???? _head = NULL; >>>>>>>>>>>>>>>>>>>>>>> ??486???? _stack_info = NULL; >>>>>>>>>>>>>>>>>>>>>>> ??487???? _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> As you are touching this can you change it to >>>>>>>>>>>>>>>>>>>>>>> use an initializer list as you did for the >>>>>>>>>>>>>>>>>>>>>>> HandshakeClosure, and please keep one item per >>>>>>>>>>>>>>>>>>>>>>> line. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ??820 >>>>>>>>>>>>>>>>>>>>>>> assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>> ??821 >>>>>>>>>>>>>>>>>>>>>>> java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>>>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == >>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>> ??823????????? "at safepoint / handshake or >>>>>>>>>>>>>>>>>>>>>>> target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, >>>>>>>>>>>>>>>>>>>>>>> as even if the target is suspended we must still >>>>>>>>>>>>>>>>>>>>>>> be at a safepoint or in a handshake with it. >>>>>>>>>>>>>>>>>>>>>>> Makes me wonder if we used to allow a racy >>>>>>>>>>>>>>>>>>>>>>> stacktrace operation on a suspended thread, >>>>>>>>>>>>>>>>>>>>>>> assuming it would remain suspended? >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>>>>>>>>>>>>>>> 1269 >>>>>>>>>>>>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && (jt->threadObj() >>>>>>>>>>>>>>>>>>>>>>> != NULL)) { >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> 1272 >>>>>>>>>>>>>>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>>>>>>>>>>>>>> thread_oop), >>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> It is frustrating that this entire call chain >>>>>>>>>>>>>>>>>>>>>>> started with a jthread reference, which we >>>>>>>>>>>>>>>>>>>>>>> converted to a JavaThread, only to eventually >>>>>>>>>>>>>>>>>>>>>>> need to convert it back to a jthread! I think >>>>>>>>>>>>>>>>>>>>>>> there is some scope for simplification here but >>>>>>>>>>>>>>>>>>>>>>> not as part of this change. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the >>>>>>>>>>>>>>>>>>>>>>> current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I'm still pondering the test logic but wanted to >>>>>>>>>>>>>>>>>>>>>>> send this now. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>> VM_GetThreadListStackTrace (for >>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces) and >>>>>>>>>>>>>>>>>>>>>>>> VM_GetAllStackTraces (for GetAllStackTraces) >>>>>>>>>>>>>>>>>>>>>>>> have inherited VM_GetMultipleStackTraces VM >>>>>>>>>>>>>>>>>>>>>>>> operation which provides the feature to >>>>>>>>>>>>>>>>>>>>>>>> generate jvmtiStackInfo. I modified >>>>>>>>>>>>>>>>>>>>>>>> VM_GetMultipleStackTraces to a normal C++ class >>>>>>>>>>>>>>>>>>>>>>>> to share with HandshakeClosure for >>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces >>>>>>>>>>>>>>>>>>>>>>>> (GetSingleStackTraceClosure). >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Also I added new testcases which test >>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() with thread_count == >>>>>>>>>>>>>>>>>>>>>>>> 1 and with all threads. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in >>>>>>>>>>>>>>>>>>>>>>>> serviceability/jvmti serviceability/jdwp >>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti vmTestbase/nsk/jdi >>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jdwp. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> ?? JBS: >>>>>>>>>>>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>>>>>>>>>>>> ?? webrev: >>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> This change replace following VM operations to >>>>>>>>>>>>>>>>>>>>>>>>> direct handshake. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces >>>>>>>>>>>>>>>>>>>>>>>>> (GetThreadListStackTrace()) >>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTrace() uses direct >>>>>>>>>>>>>>>>>>>>>>>>> handshake if thread count == 1. In other case >>>>>>>>>>>>>>>>>>>>>>>>> (thread count > 1), it would be performed as >>>>>>>>>>>>>>>>>>>>>>>>> VM operation (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>>>>>>>>>>>> Caller of VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>> (JvmtiEnvThreadState::reset_current_location()) >>>>>>>>>>>>>>>>>>>>>>>>> might be called at safepoint. So I added >>>>>>>>>>>>>>>>>>>>>>>>> safepoint check in its caller. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in >>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jvmti serviceability/jdwp >>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti vmTestbase/nsk/jdi >>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/ns >>>>>>>>>>>>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Also I tested it on submit repo, then it has >>>>>>>>>>>>>>>>>>>>>>>>> execution error >>>>>>>>>>>>>>>>>>>>>>>>> (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) >>>>>>>>>>>>>>>>>>>>>>>>> due to dependency error. So I think it does >>>>>>>>>>>>>>>>>>>>>>>>> not occur by this change. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>> >>>>>> >>>> >> From suenaga at oss.nttdata.com Tue Jul 7 08:54:13 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Tue, 7 Jul 2020 17:54:13 +0900 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <90cdfef9-e558-0d25-33f5-25b7d65bb83b@oss.nttdata.com> <0e359819-618f-c17b-a427-bb7e4daa2c58@oracle.com> <3f890bed-6892-e64d-b21b-ca0cbe11784e@oracle.com> <7dc1903b-6295-3864-6301-4c02aacdb971@oss.nttdata.com> <04e09eb7-9483-8f63-4bb4-01c9e814802e@oracle.com> <238a89a3-06b1-90e4-d5a4-8dc2f1ed12ab@oracle.com> <4c43bc2c-551e-d042-8fa7-b63e755888d1@oracle.com> <8c387550-a06e-6320-f453-0e898486befb@oss.nttdata.com> <19cfa2a7-f441-425d-5623-1c11c6a870b6@oss.nttdata.com> Message-ID: Hi David, Serguei, Serguei, thank you for replying even though you are on vacaiton! I uploaded new webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.07/ Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/77243b1dcbfe c'tor of GetSingleStackTraceClosure has jthread argument in this webrev. Also it does not contain testcase for GetThreadListStackTraces with all threads, and OneGetThreadListStackTraces would test main thread only. Thanks, Yasumasa On 2020/07/07 15:13, David Holmes wrote: > On 7/07/2020 2:57 pm, Yasumasa Suenaga wrote: >> Hi David, >> >> On 2020/07/07 11:31, David Holmes wrote: >>> Hi Yasumasa, >>> >>> Hard to keep up with the changes - especially without incremental webrevs. >> >> Sorry, I will upload diff from previous webrev in the next. >> >> >>> If GetSingleStackTraceClosure also took the jthread as a constructor arg, then you wouldn't need to recreate a JNI local handle when calling _collector.fill_frames. It's a small simplification and not essential at this stage. >> >> I think we should get jthread from an argument of do_thread() because do_thread() would pass the thread which are stopped certainly. >> It might be simplification if we pass _calling_thread to MultipleStackTracesCollector. `jthread` is only needed to store jvmtiStackInfo.thread . What do you think? > > I'm not quite sure what you mean. > > I think there is a bit of a design wart with direct handshakes in that do_thread takes the target JavaThread as an argument. That's useful in a case where you want a HandshakeClosure that can be applied to multiple threads, but that's not typically what is needed with direct handshakes - there is only a single target. With a single-target HandshakeClosure you can capture all the "target" information for the operation in the closure instance. So if the actual do_thread operation wants the jthread corresponding to the target thread then we can store that in the closure rather than recomputing it (you could assert it is the same but that seems overkill to me). > >> >>> For the test ... I don't see how Java_GetThreadListStackTraces_checkCallStacks is a valid test. It gets the stacks of all live threads, then uses that information to use GetThreadListStackTraces to get the stack for the same set of threads through a different API. It then compares the two sets of stacks for each thread expecting them to be the same, but that need only be the case for the main thread. Other threads could potentially have a different stack (e.g. if this test is run with JFR enabled there will be additional threads found.) Further I would have expected that there already exist tests that check that, for a given thread (which may be suspended or known to be blocked) the same stack is found through the two different APIs. >> >> vmTestbase/nsk/jvmti/unit/GetAllStackTraces/getallstktr001 would check all of threads via GetThreadListStackTraces() and GetAllStackTraces(), so we might be able to remove GetThreadListStackTraces.java from this webrev. > > Yes. The existing test only examines a set of test threads that are all blocked on a raw monitor. You do not need to duplicate that test. > >> OTOH we don't have testcase for GetThreadListStackTraces() with thread_count == 1, so we need to testcase for it (it is OneGetThreadListStackTraces.java) It would check whether the state of target thread is "waiting" before JNI call to call GetThreadListStackTraces(), > > Yes we need to test the special cases introduced by your changes - totally agree - and OneGetThreadListStackTraces.java is a good test for that. > >> and also I expect it would not be run with JFR. (it is not described @run) > > The arguments to run with JFR (or a bunch of other things) can be passed to the jtreg test harness to be applied to all tests. > >> Of course we can check GetThreadListStackTraces() with main thread, but it is not the test for direct handshake for other thread. > > Right - that test already exists as per the above. > > Thanks, > David > >> >> Thanks, >> >> Yasumasa >> >> >>> Thanks, >>> David >>> ----- >>> >>> On 6/07/2020 11:29 pm, Yasumasa Suenaga wrote: >>>> Hi Serguei, >>>> >>>> Thanks for your comment! >>>> >>>> I think C++ is more simple to implement the test agent as you said. >>>> So I implement it in C++ in new webrev. Could you review again? >>>> >>>> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.06/ >>>> >>>> Also I refactored libGetThreadListStackTraces.cpp, and I've kept exception check after IsSameObject(). >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2020/07/06 16:32, serguei.spitsyn at oracle.com wrote: >>>>> Hi Yasumasa, >>>>> >>>>> Thank you for the update. >>>>> I think, a pending exception after IsSameObject needs to be checked. >>>>> >>>>> The checkStackInfo() needs one more refactoring as I've already suggested. >>>>> The body of the loop at L68-L78 should be converted to a function check_frame_info. >>>>> The si1->frame_buffer[i] and si2->frame_buffer[i] will be passed as fi1 and fi2. >>>>> The index can be passed as well. >>>>> I'm still suggesting to simplify the local exception_msg to something shorter like err_msg or exc_msg. >>>>> >>>>> I'm not sure using fatal is right here: >>>>> >>>>> This fragment looks strange: >>>>> >>>>> ? 152???? if ((*env)->IsSameObject(env, stack_info[i].thread, thread)) { >>>>> ? 153?????? target_info = &stack_info[i]; >>>>> ? 154?????? break; >>>>> ? 155???? } else if ((*env)->ExceptionOccurred(env)) { >>>>> ? 156?????? (*env)->ExceptionDescribe(env); >>>>> ? 157?????? (*env)->FatalError(env, __FILE__); >>>>> ? 158???? } >>>>> >>>>> I expected it to be: >>>>> >>>>> ??? jboolean same = (*env)->IsSameObject(env, stack_info[i].thread, thread); >>>>> ??? if ((*env)->ExceptionOccurred(env)) { >>>>> ????? (*env)->ExceptionDescribe(env); >>>>> ????? (*env)->FatalError(env, __FILE__); >>>>> ??? } >>>>> ??? if (same) { >>>>> ????? target_info = &stack_info[i]; >>>>> ????? break; >>>>> ??? } >>>>> >>>>> Would it better to port this agent to C++ to simplify this code nicer? >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>> On 7/5/20 06:13, Yasumasa Suenaga wrote: >>>>>> Hi Serguei, >>>>>> >>>>>> Thanks for your comment! >>>>>> I refactored testcase. Could you review again? >>>>>> >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.05/ >>>>>> >>>>>> It would check Java exception after IsSameObject() call. Does it need? >>>>>> Any exceptions are not described in JNI document[1], and JNI implementation (jni_IsSameObject()) does not seem to throw it. >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> [1] https://docs.oracle.com/en/java/javase/14/docs/specs/jni/functions.html#issameobject >>>>>> >>>>>> >>>>>> On 2020/07/05 14:46, serguei.spitsyn at oracle.com wrote: >>>>>>> Hi Yasumasa, >>>>>>> >>>>>>> >>>>>>> Okay, thanks. >>>>>>> Then I'm okay to keep the GetSingleStackTraceClosure. >>>>>>> >>>>>>> >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c.html >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c.html >>>>>>> >>>>>>> I'm not sure the function 'is_same_thread() is needed. >>>>>>> Why do not use the JNI IsSameObject instead? >>>>>>> >>>>>>> It seems to be a typo at L132 and L137. >>>>>>> You, probably. did not want to print the same information for stack_info_1[i].frame_buffer[j].XXX twice. >>>>>>> >>>>>>> The code at lines 112-142 is not readable. >>>>>>> I'd suggest to make a couple of refactoring steps. >>>>>>> >>>>>>> First step to simplify this a little bit would be with some renaming and getting rid of indexes: >>>>>>> >>>>>>> ?? 71 char err_msg[EXCEPTION_MSG_LEN] = {0}; >>>>>>> ??... >>>>>>> ??112?? /* Iterate all jvmtiStackInfo to check */ >>>>>>> ??113?? for (i = 0; i < num_threads, *exception_msg != '\0'; i++) { >>>>>>> ????????? jvmtiStackInfo *si1 = stack_info_1[i]; >>>>>>> ????????? jvmtiStackInfo *si2 = stack_info_2[i]; >>>>>>> ??114???? if (!IsSameObject(env, si1.thread, si2.thread)) { /* jvmtiStackInfo::thread */ >>>>>>> ??115?????? snprintf(err_msg, sizeof(err_msg), >>>>>>> ??116??????????????? "thread[%d] is different: stack_info_1 = %p, stack_info_2 = %p", >>>>>>> ??117??????????????? i, sinfo1.thread, sinfo2.thread); >>>>>>> ??118???? } else if (si1.state != si2.state) { /* jvmtiStackInfo::state */ >>>>>>> ??119?????? snprintf(err_msg, sizeof(err_msg), >>>>>>> ??120??????????????? "state[%d] is different: stack_info_1 = %d, stack_info_2 = %d", >>>>>>> ??121??????????????? i, si1.state, si2.state); >>>>>>> ??122???? } else if (si1.frame_count != si2.frame_count) { /* jvmtiStackInfo::frame_count */ >>>>>>> ??123?????? snprintf(err_msg, sizeof(err_msg), >>>>>>> ??124??????????????? "frame_count[%d] is different: stack_info_1 = %d, stack_info_2 = %d", >>>>>>> ??125??????????????? i, si1.frame_count, si2.frame_count); >>>>>>> ??126???? } else { >>>>>>> ??127?????? /* Iterate all jvmtiFrameInfo to check */ >>>>>>> ??128?????? for (j = 0; j < si1.frame_count; j++) { >>>>>>> ??129???????? if (si1.frame_buffer[j].method != si1.frame_buffer[j].method) { /* jvmtiFrameInfo::method */ >>>>>>> ??130?????????? snprintf(err_msg, sizeof(err_msg), >>>>>>> ??131??????????????????? "thread [%d] frame_buffer[%d].method is different: stack_info_1 = %lx, stack_info_2 = %lx", >>>>>>> ??132??????????????????? i, j, si1.frame_buffer[j].method, si2.frame_buffer[j].method); >>>>>>> ??133?????????? break; >>>>>>> ??134???????? } else if (si1.frame_buffer[j].location != si1.frame_buffer[j].location) { /* jvmtiFrameInfo::location */ >>>>>>> ??135?????????? snprintf(err_msg, sizeof(err_msg), >>>>>>> ??136??????????????????? "thread [%d] frame_buffer[%d].location is different: stack_info_1 = %ld, stack_info_2 = %ld", >>>>>>> ??137??????????????????? i, j, si1.frame_buffer[j].location, si2.frame_buffer[j].location); >>>>>>> ??138?????????? break; >>>>>>> ??139???????? } >>>>>>> ??140?????? } >>>>>>> ??141???? } >>>>>>> ??142?? } >>>>>>> >>>>>>> Another step would be to create functions that implement a body of each loop. >>>>>>> You can use the same techniques to simplify similar place (L127-L138) in the libOneGetThreadListStackTraces.c. >>>>>>> >>>>>>> Thanks, >>>>>>> Serguei >>>>>>> >>>>>>> >>>>>>> On 7/3/20 15:55, Yasumasa Suenaga wrote: >>>>>>>> Hi Serguei, >>>>>>>> >>>>>>>> I'm not an Oracle employee, so I cannot know real request(s) from your customers. >>>>>>>> However JDK-8201641 says Dynatrace has requested this enhancement. >>>>>>>> >>>>>>>> BTW I haven't heared any request from my customers about this. >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2020/07/04 4:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>> Hi Yasumasa, >>>>>>>>> >>>>>>>>> This difference is not that big to care about. >>>>>>>>> I feel this is really rare case and so, does not worth these complications. >>>>>>>>> Do we have a real request from customers to optimize it? >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Serguei >>>>>>>>> >>>>>>>>> >>>>>>>>> On 7/3/20 01:16, Yasumasa Suenaga wrote: >>>>>>>>>> Hi Serguei, >>>>>>>>>> >>>>>>>>>> Generally I agree with you, but I have concern about the difference of the result of GetStackTrace() and GetThreadListStackTraces(). >>>>>>>>>> >>>>>>>>>> ? GetStackTrace: jvmtiFrameInfo >>>>>>>>>> ? GetThreadListStackTraces: jvmtiStackInfo >>>>>>>>>> >>>>>>>>>> jvmtiStackInfo contains thread state, and it is ensured it is the state of the call stack. >>>>>>>>>> If we want to get both call stack and thread state, we need to suspend target thread, and call both GetStackTrace() and GetThreadState(). Is it ok? >>>>>>>>>> >>>>>>>>>> I was wondering if JDK-8201641 (parent ticket of this change) needed them for profiling (dynatrace?) >>>>>>>>>> If it is responsibility of JVMTI agent implementor, I remove this closure. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2020/07/03 16:45, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>> >>>>>>>>>>> After some thinking I've concluded that I do not like this optimization >>>>>>>>>>> of the GetThreadListStackTraces with GetSingleStackTraceClosure. >>>>>>>>>>> >>>>>>>>>>> We may need more opinions on this but these are my points: >>>>>>>>>>> ??- it adds some complexity and ugliness >>>>>>>>>>> ??- a win is doubtful because it has to be a rare case, so that total overhead should not be high >>>>>>>>>>> ??- if it is really high for some use cases then it is up to the user >>>>>>>>>>> ??? to optimize it with using GetStackTrace instead >>>>>>>>>>> >>>>>>>>>>> In such cases with doubtful overhead I usually prefer the simplicity. >>>>>>>>>>> >>>>>>>>>>> Good examples where it makes sense to optimize are checks for target thread to be current thread. >>>>>>>>>>> In such cases there is no need to suspend the target thread, or use a VMop/HandshakeClosure. >>>>>>>>>>> For instance, please, see the Monitor functions with the check: (java_thread == calling_thread). >>>>>>>>>>> Getting information for current thread is frequently used case, e.g. to get info at an event point. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Serguei >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 7/2/20 23:29, Yasumasa Suenaga wrote: >>>>>>>>>>>> Hi Dan, David, >>>>>>>>>>>> >>>>>>>>>>>> I uploaded new webrev. Could you review again? >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/ >>>>>>>>>>>> >>>>>>>>>>>> OneGetThreadListStackTraces.java in this webrev would wait until thread state is transited to "waiting" with spin wait. >>>>>>>>>>>> CountDownLatch::await call as Dan pointed is fixed in it :) >>>>>>>>>>>> >>>>>>>>>>>> Diff from webrev.03: >>>>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/c9aeb7001e50 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 2020/07/03 14:15, David Holmes wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>> On 2020/07/03 12:24, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>> On 7/2/20 10:50 PM, David Holmes wrote: >>>>>>>>>>>>>>>> Sorry I'm responding here without seeing latest webrev but there is enough context I think ... >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>> Hi Dan, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I upload new webrev. Could you review again? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>>>>>>>>>> ???? L1542: ??? // Get stack trace with handshake >>>>>>>>>>>>>>>>>> ???????? nit - please add a period at the end. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> ???? L1591: *stack_info_ptr = op.stack_info(); >>>>>>>>>>>>>>>>>> ???????? The return parameter should not be touched unless the return >>>>>>>>>>>>>>>>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> ???? old L1582: ? if (err == JVMTI_ERROR_NONE) { >>>>>>>>>>>>>>>>>> ???????? Please restore this check. The return parameter should not >>>>>>>>>>>>>>>>>> ???????? be touched unless the return code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> But op.stack_info() will return NULL if the error is not JVMTI_ERROR_NONE. Are you (Dan) concerned about someone passing in a non-null/initialized out-pointer that will be reset to NULL if there was an error? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Actually the way we used to test this in POSIX tests is to call >>>>>>>>>>>>>>> an API with known bad parameters and the return parameter ptr >>>>>>>>>>>>>>> set to NULL. If the return parameter ptr was touched when an >>>>>>>>>>>>>>> error should have been detected on an earlier parameter, then >>>>>>>>>>>>>>> the test failed. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> ???? L1272: ? if (!jt->is_exiting() && (thread_oop != NULL)) { >>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>> ???? old L1532: ? _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>>>>>>>>>>>>>>> ???????? This deletion of the _result field threw me for a minute and then >>>>>>>>>>>>>>>>>> ???? ? ? I figured out that the field is init to JVMTI_ERROR_THREAD_NOT_ALIVE >>>>>>>>>>>>>>>>>> ???????? in the constructor. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> ???? L1553: ? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>>>>>>>>>> ???? L64: ??????? startSignal.countDown(); >>>>>>>>>>>>>>>>>> ???????? I was expecting this to be a call to await() instead of >>>>>>>>>>>>>>>>>> ???????? countDown(). What am I missing here? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> ???????? I think this test might be passing by accident right now, but... >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Main thread (which call JVMTI functions to test) should wait until test thread is ready. >>>>>>>>>>>>>>>>> So main thread would wait startSignal, and test thread would count down. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> No! >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> The test thread that previously called obj.wait() now calls latch.await(). >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> The main thread that previously called obj.notify() now calls latch.countDown(). >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> The main thread continues to spin until it sees the target is WAITING before proceeding with the test. >>>>>>>>>>>>>> >>>>>>>>>>>>>> If I add spin wait to wait until transit target thread state is WAITING (as following), we don't need to call SuspendThread(). >>>>>>>>>>>>>> Which is better? >>>>>>>>>>>>> >>>>>>>>>>>>> The original spin-wait loop checking for? WAITING is better because it is the only guarantee that the target thread is blocked where you need it to be. suspending the thread is racy as you don't know exactly where the suspend will hit. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> David >>>>>>>>>>>>> ----- >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> ``` >>>>>>>>>>>>>> /* Wait until the thread state transits to "waiting" */ >>>>>>>>>>>>>> while (th.getState() != Thread.State.WAITING) { >>>>>>>>>>>>>> ???? Thread.onSpinWait(); >>>>>>>>>>>>>> } >>>>>>>>>>>>>> ``` >>>>>>>>>>>>>> >>>>>>>>>>>>>> For simplify, spin wait is prefer to OneGetThreadListStackTraces.java in webrev.03. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Here's the flow as I see it: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>> ?? - start worker thread >>>>>>>>>>>>>>> ?? - startSignal.await() >>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>> ?? - startSignal.countDown() >>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>> ?? - stopSignal.await() >>>>>>>>>>>>>>> ???? - worker is now blocked >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>> ?? - checkCallStacks(th) >>>>>>>>>>>>>>> ?? - stopSignal.countDown() >>>>>>>>>>>>>>> ???? - worker is now unblocked >>>>>>>>>>>>>>> ?? - th.join >>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>> ?? - runs off the end of run() >>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>> ?? - run off the end of main() >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>> ???? L92: ? jthreads = (jthread *)malloc(sizeof(jthread) * num_threads); >>>>>>>>>>>>>>>>>> ???????? You don't check for malloc() failure. >>>>>>>>>>>>>>>>>> ???????? 'jthreads' is allocated but never freed. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>> ???? L91: ? result = (*jvmti)->SuspendThread(jvmti, thread); >>>>>>>>>>>>>>>>>> ???????? Why are you suspending the thread? GetAllStackTraces() and >>>>>>>>>>>>>>>>>> ???????? GetThreadListStackTraces() do not require the target thread(s) >>>>>>>>>>>>>>>>>> ???????? to be suspend. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> ???????? If you decide not to SuspendThread, then you don't need the >>>>>>>>>>>>>>>>>> ???????? AddCapabilities or the ResumeThread calls. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Test thread might not be entered following code (stopSignal.await()). We might see deferent call stack between GetAllStackTraces() and GetThreadListStackTraces(). We cannot control to freeze call stack of test thread in Java code. >>>>>>>>>>>>>>>>> (I didn't use SuspendThread() at first, but I saw some errors which causes in above.) >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> So we need to call SuspendThread() to ensure we can see same call stack. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> If you are checking that the thread is in state WAITING then it cannot escape from that state and you can sample the stack multiple times from any API and get the same result. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I suspect the errors you saw were from the apparent incorrect use of the CountDownLatch. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> With the flow outlined above, the worker thread should be >>>>>>>>>>>>>>> nicely blocked in stopSignal.await() when stuff is sampled. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could review again? >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Updates look fine - thanks. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> One minor nit: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> 1274 _collector.allocate_and_fill_stacks(1); >>>>>>>>>>>>>>>>>>>> 1275 _collector.set_result(JVMTI_ERROR_NONE); >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> In the other places where you use _collector you rely on result being initialized to JVMTI_ERROR_NONE, rather than setting it directly after allocate_and_fill_stacks(). >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> ? 820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>> ? 821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>> ? 822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>> ? 823????????? "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> This function (JvmtiEnvBase::get_stack_trace()) can be called to get own stack trace. For example, we can call GetStackTrace() for current thread at JVMTI event. >>>>>>>>>>>>>>>>>>>>> So I changed assert as below: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>> ??820?? assert(current_thread == java_thread || >>>>>>>>>>>>>>>>>>>>> ??821 SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>> ??822????????? current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>> ??823????????? "call by myself / at safepoint / at handshake"); >>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Yep good catch. I hope current tests caught that. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> They would be tested in vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ (own call stacks), and getstacktr003 (call stacks in other thread). >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Speaking of tests ... >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> In the native code I think you need to check the success of all JNI methods that can throw exceptions - otherwise I believe the tests may trigger warnings if -Xcheck:jni is used with them. See for example: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I updated testcases to check JNI and JVMTI function calls. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> In the Java code the target thread: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> ?? 45???? public void run() { >>>>>>>>>>>>>>>>>>>> ?? 46?????? try { >>>>>>>>>>>>>>>>>>>> ?? 47?????????? synchronized (lock) { >>>>>>>>>>>>>>>>>>>> ?? 48?????????????? lock.wait(); >>>>>>>>>>>>>>>>>>>> ?? 49 System.out.println("OK"); >>>>>>>>>>>>>>>>>>>> ?? 50?????????? } >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> is potentially susceptible to spurious wakeups. Using a CountDownLatch would be robust. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> If so, we can call make_local() in L1272 without JavaThread (or we can pass current thread to make_local()). Is it right? >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Sorry I got confused, _calling_thread may not be the current thread as we could be executing the handshake in the target thread itself. So the ResourceMark is correct as-is (implicitly for current thread). >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> The argument to fill_frames will be used in the jvmtiStackInfo and passed back to the _calling_thread, so it must be created via make_local(_calling_thread, ...) as you presently have. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thank you for reviewing! I will update new webrev tomorrow. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> ? 498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>>>>>>>>>>>>>>> ? 500?? JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>> ? 502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>>>>>>>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>>>>>>>>>>>>>>>> ??? : >>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> class GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>> ?? private: >>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> I updated webrev for 8242428. Could you review again? >>>>>>>>>>>>>>>>>>>>>>>>>>> This change migrate to use direct handshake for GetStackTrace() and GetThreadListStackTraces() (when thread_count == 1). >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> This looks really good now! I only have a few nits below. There is one thing I don't like about it but it requires a change to the main Handshake logic to address - in JvmtiEnv::GetThreadListStackTraces you have to create a ThreadsListHandle to convert the jthread to a JavaThread, but then the Handshake::execute_direct creates another ThreadsListHandle internally. That's a waste. I will discuss with Robbin and file a RFE to have an overload of execute_direct that takes an existing TLH. Actually it's worse than that because we have another TLH in use at the entry point for the JVMTI functions, so I think there may be some scope for simplifying the use of TLH instances - future RFE. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> ??451 GetStackTraceClosure(JvmtiEnv *env, jint start_depth, jint max_count, >>>>>>>>>>>>>>>>>>>>>>>>>> ??452 jvmtiFrameInfo* frame_buffer, jint* count_ptr) >>>>>>>>>>>>>>>>>>>>>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>>>>>>>>>>>>>>> ??454?????? _env(env), _start_depth(start_depth), _max_count(max_count), >>>>>>>>>>>>>>>>>>>>>>>>>> ??455 _frame_buffer(frame_buffer), _count_ptr(count_ptr), >>>>>>>>>>>>>>>>>>>>>>>>>> ??456 _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Nit: can you do one initializer per line please. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> ??498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>>>>>>>>>>>>>>> ??500?? JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>> ??502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> 481 MultipleStackTracesCollector(JvmtiEnv *env, jint max_frame_count) { >>>>>>>>>>>>>>>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>>>>>>>>>>>>>>> ??483???? _max_frame_count = max_frame_count; >>>>>>>>>>>>>>>>>>>>>>>>>> ??484 _frame_count_total = 0; >>>>>>>>>>>>>>>>>>>>>>>>>> ??485???? _head = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>> ??486???? _stack_info = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>> ??487???? _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>>>>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> As you are touching this can you change it to use an initializer list as you did for the HandshakeClosure, and please keep one item per line. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> ??820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>> ??821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>> ??823????????? "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>>>>>>>>>>>>>>>>>> 1269 >>>>>>>>>>>>>>>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> It is frustrating that this entire call chain started with a jthread reference, which we converted to a JavaThread, only to eventually need to convert it back to a jthread! I think there is some scope for simplification here but not as part of this change. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> I'm still pondering the test logic but wanted to send this now. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetThreadListStackTrace (for GetThreadListStackTraces) and VM_GetAllStackTraces (for GetAllStackTraces) have inherited VM_GetMultipleStackTraces VM operation which provides the feature to generate jvmtiStackInfo. I modified VM_GetMultipleStackTraces to a normal C++ class to share with HandshakeClosure for GetThreadListStackTraces (GetSingleStackTraceClosure). >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Also I added new testcases which test GetThreadListStackTraces() with thread_count == 1 and with all threads. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/nsk/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> This change replace following VM operations to direct handshake. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces (GetThreadListStackTrace()) >>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTrace() uses direct handshake if thread count == 1. In other case (thread count > 1), it would be performed as VM operation (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>>>>>>>>>>>>>>> Caller of VM_GetCurrentLocation (JvmtiEnvThreadState::reset_current_location()) might be called at safepoint. So I added safepoint check in its caller. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/ns >>>>>>>>>>>>>>>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I tested it on submit repo, then it has execution error (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) due to dependency error. So I think it does not occur by this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>> >>>>> From kevin.walls at oracle.com Tue Jul 7 11:09:13 2020 From: kevin.walls at oracle.com (Kevin Walls) Date: Tue, 7 Jul 2020 12:09:13 +0100 Subject: RFR(M): 8247515: OSX pc_to_symbol() lookup does not work with core files In-Reply-To: References: Message-ID: <092ef5a8-c973-e377-de91-b2002a1b8fe7@oracle.com> Hi Chris, Yes I think this looks good. Question: In nearest_symbol, do we need to initialize lowest_offset_from_sym to something impossibly high, as if it defaults to zero we never find a better/nearer result? Thanks Kevin On 07/07/2020 06:10, Chris Plummer wrote: > Hello, > > Please help review the following: > > http://cr.openjdk.java.net/~cjplummer/8247515/webrev.00/index.html > https://bugs.openjdk.java.net/browse/JDK-8247515 > > The CR contains a description of the issues being addressed. There is > also no test for this symbol lookup support yet. It will be there > after I push JDK-8247516 and? JDK-8247514, which are both blocked by > the CR. > > [1] https://bugs.openjdk.java.net/browse/JDK-8247516 > [2] https://bugs.openjdk.java.net/browse/JDK-8247514 > > thanks, > > Chris > From fairoz.matte at oracle.com Tue Jul 7 14:49:11 2020 From: fairoz.matte at oracle.com (Fairoz Matte) Date: Tue, 7 Jul 2020 07:49:11 -0700 (PDT) Subject: RFR(s): 8236042: [TESTBUG] serviceability/sa/ClhsdbCDSCore.java fails with -Xcomp -XX:TieredStopAtLevel=1 Message-ID: <2abe9fba-e958-4b34-9f92-6bb8d8478f4e@default> Hi, Please review this small test change to consider the scenario when there is no "printmdo" output JBS - https://bugs.openjdk.java.net/browse/JDK-8236042 Webrev - http://cr.openjdk.java.net/~fmatte/8236042/webrev.00/ Thanks, Fairoz From chris.plummer at oracle.com Tue Jul 7 20:17:39 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 7 Jul 2020 13:17:39 -0700 Subject: RFR(M): 8247515: OSX pc_to_symbol() lookup does not work with core files In-Reply-To: <092ef5a8-c973-e377-de91-b2002a1b8fe7@oracle.com> References: <092ef5a8-c973-e377-de91-b2002a1b8fe7@oracle.com> Message-ID: <38e1887e-a646-abc0-0b7e-53339af1d684@oracle.com> Hi Kevin, Thanks for the review. Yes, that lack of initialization of lowest_offset_from_sym is a bug. I'm real surprised the compiler didn't catch it as it will be uninitialized garbage the first time it is referenced. Fortunately usually the eventual offset is very small if not 0, so probably this never prevented a proper match. I think there's also another bug: ?415?????? uintptr_t offset_from_sym = offset - sym->offset; "offset" is the passed in offset, essentially the address of the symbol we are interested in, but given as an offset from the start of the DSO. "sym->offset" is also an offset from the start of the DSO. It could be located before or after "offset". This means the math could result in a negative number, which when converted to unsigned would be a very large positive number. This happens whenever you check a symbol that is actually located after the address you are looking up. The end result is harmless, because it just means there's no way we will match that symbol, which is what you want, but it would be good to clean this up. I think what is best is to use ptrdiff_t and initialize lowest_offset_from_sym to -1. I've updated the webrev: http://cr.openjdk.java.net/~cjplummer/8247515/webrev.01/index.html thanks, Chris On 7/7/20 4:09 AM, Kevin Walls wrote: > Hi Chris, > > Yes I think this looks good. > > Question: In nearest_symbol, do we need to initialize > lowest_offset_from_sym to something impossibly high, as if it defaults > to zero we never find a better/nearer result? > > Thanks > Kevin > > > On 07/07/2020 06:10, Chris Plummer wrote: >> Hello, >> >> Please help review the following: >> >> http://cr.openjdk.java.net/~cjplummer/8247515/webrev.00/index.html >> https://bugs.openjdk.java.net/browse/JDK-8247515 >> >> The CR contains a description of the issues being addressed. There is >> also no test for this symbol lookup support yet. It will be there >> after I push JDK-8247516 and? JDK-8247514, which are both blocked by >> the CR. >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8247516 >> [2] https://bugs.openjdk.java.net/browse/JDK-8247514 >> >> thanks, >> >> Chris >> From chris.plummer at oracle.com Tue Jul 7 22:07:49 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 7 Jul 2020 15:07:49 -0700 Subject: RFR(s): 8236042: [TESTBUG] serviceability/sa/ClhsdbCDSCore.java fails with -Xcomp -XX:TieredStopAtLevel=1 In-Reply-To: <2abe9fba-e958-4b34-9f92-6bb8d8478f4e@default> References: <2abe9fba-e958-4b34-9f92-6bb8d8478f4e@default> Message-ID: <70057c31-e535-f03a-391d-d181b2ec150b@oracle.com> Hi Fairoz, Looks good, except for the missing space in "if(testJavaOpts...". thanks, Chris On 7/7/20 7:49 AM, Fairoz Matte wrote: > Hi, > > Please review this small test change to consider the scenario when there is no "printmdo" output > > JBS - https://bugs.openjdk.java.net/browse/JDK-8236042 > Webrev - http://cr.openjdk.java.net/~fmatte/8236042/webrev.00/ > > Thanks, > Fairoz From chris.plummer at oracle.com Wed Jul 8 00:32:56 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 7 Jul 2020 17:32:56 -0700 Subject: RFR(T): 8249002: New serviceability/sa/ClhsdbFindPC.java #id2 and #id3 tests are failing with ZGC Message-ID: Hello, Please help review this trivial change to problem list a couple of new SA subtests so they are not run with ZGC: diff --git a/test/hotspot/jtreg/ProblemList-zgc.txt b/test/hotspot/jtreg/ProblemList-zgc.txt --- a/test/hotspot/jtreg/ProblemList-zgc.txt +++ b/test/hotspot/jtreg/ProblemList-zgc.txt @@ -34,6 +34,8 @@ ?serviceability/sa/ClhsdbCDSJstackPrintAll.java 8220624?? generic-all ?serviceability/sa/ClhsdbFindPC.java#id0 8220624?? generic-all ?serviceability/sa/ClhsdbFindPC.java#id1 8220624?? generic-all +serviceability/sa/ClhsdbFindPC.java#id2 8220624?? generic-all +serviceability/sa/ClhsdbFindPC.java#id3 8220624?? generic-all ?serviceability/sa/ClhsdbInspect.java 8220624?? generic-all ?serviceability/sa/ClhsdbJdis.java 8220624?? generic-all ?serviceability/sa/ClhsdbJhisto.java 8220624?? generic-all thanks, Chris From chris.plummer at oracle.com Wed Jul 8 00:34:53 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 7 Jul 2020 17:34:53 -0700 Subject: RFR(T): 8249002: New serviceability/sa/ClhsdbFindPC.java #id2 and #id3 tests are failing with ZGC In-Reply-To: References: Message-ID: <6cf90b18-52b9-03f3-d62b-a6f179b1a067@oracle.com> Sorry, forgot the link to the CR: https://bugs.openjdk.java.net/browse/JDK-8249002 Chris On 7/7/20 5:32 PM, Chris Plummer wrote: > Hello, > > Please help review this trivial change to problem list a couple of new > SA subtests so they are not run with ZGC: > > diff --git a/test/hotspot/jtreg/ProblemList-zgc.txt > b/test/hotspot/jtreg/ProblemList-zgc.txt > --- a/test/hotspot/jtreg/ProblemList-zgc.txt > +++ b/test/hotspot/jtreg/ProblemList-zgc.txt > @@ -34,6 +34,8 @@ > ?serviceability/sa/ClhsdbCDSJstackPrintAll.java 8220624 generic-all > ?serviceability/sa/ClhsdbFindPC.java#id0 8220624?? generic-all > ?serviceability/sa/ClhsdbFindPC.java#id1 8220624?? generic-all > +serviceability/sa/ClhsdbFindPC.java#id2 8220624?? generic-all > +serviceability/sa/ClhsdbFindPC.java#id3 8220624?? generic-all > ?serviceability/sa/ClhsdbInspect.java 8220624?? generic-all > ?serviceability/sa/ClhsdbJdis.java 8220624?? generic-all > ?serviceability/sa/ClhsdbJhisto.java 8220624?? generic-all > > thanks, > > Chris > From igor.ignatyev at oracle.com Wed Jul 8 00:38:58 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 7 Jul 2020 17:38:58 -0700 Subject: RFR(T): 8249002: New serviceability/sa/ClhsdbFindPC.java #id2 and #id3 tests are failing with ZGC In-Reply-To: <6cf90b18-52b9-03f3-d62b-a6f179b1a067@oracle.com> References: <6cf90b18-52b9-03f3-d62b-a6f179b1a067@oracle.com> Message-ID: <7B15FCA3-97F1-4A51-888F-468C57EF6B7D@oracle.com> Hi Chris, LGTM -- Igor > On Jul 7, 2020, at 5:34 PM, Chris Plummer wrote: > > Sorry, forgot the link to the CR: > > https://bugs.openjdk.java.net/browse/JDK-8249002 > > Chris > > On 7/7/20 5:32 PM, Chris Plummer wrote: >> Hello, >> >> Please help review this trivial change to problem list a couple of new SA subtests so they are not run with ZGC: >> >> diff --git a/test/hotspot/jtreg/ProblemList-zgc.txt b/test/hotspot/jtreg/ProblemList-zgc.txt >> --- a/test/hotspot/jtreg/ProblemList-zgc.txt >> +++ b/test/hotspot/jtreg/ProblemList-zgc.txt >> @@ -34,6 +34,8 @@ >> serviceability/sa/ClhsdbCDSJstackPrintAll.java 8220624 generic-all >> serviceability/sa/ClhsdbFindPC.java#id0 8220624 generic-all >> serviceability/sa/ClhsdbFindPC.java#id1 8220624 generic-all >> +serviceability/sa/ClhsdbFindPC.java#id2 8220624 generic-all >> +serviceability/sa/ClhsdbFindPC.java#id3 8220624 generic-all >> serviceability/sa/ClhsdbInspect.java 8220624 generic-all >> serviceability/sa/ClhsdbJdis.java 8220624 generic-all >> serviceability/sa/ClhsdbJhisto.java 8220624 generic-all >> >> thanks, >> >> Chris >> > > From suenaga at oss.nttdata.com Wed Jul 8 01:18:13 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Wed, 8 Jul 2020 10:18:13 +0900 Subject: RFR(XS): 8248878: SA: Implement simple workaround for JDK-8248876 In-Reply-To: <8e89d223-0f65-f26e-3bd7-2ca7421a940b@oracle.com> References: <39ac40d6-a738-3214-d86c-412152a631d4@oracle.com> <8e89d223-0f65-f26e-3bd7-2ca7421a940b@oracle.com> Message-ID: Hi Chris, SA would use `link_map` to decide to load address, but it does not seem to contain executable. I set breakpoint to pathmap_open() and I watched the argument of it, then I didn't see any executable (`java`) on it. Maybe current implementation is broken. I guess we can use note section in the core for deciding loading address. I can see valid address (includes executable) from `readelf -n`. Of course it might be big change for SA... Thanks, Yasumasa On 2020/07/07 15:38, Chris Plummer wrote: > Hi Yasumasa, > > Thanks for the review. I tried the following for line 188: > > ??? if ((phdr->p_type == PT_LOAD || phdr->p_type == PT_INTERP) && phdr->p_vaddr < baseaddr) { > > However, "base" still ended up being 0. I added some printfs. For the exec file there is both a PT_INTER with p_vaddr of 0x238 and a PT_LOAD with p_vaddr 0. I'm not sure which to use, but in either case that won't be the proper base when added to 0: > > ? if (add_lib_info_fd(ph, exec_file, ph->core->exec_fd, > ????????????????????? (uintptr_t)0 + find_base_address(ph->core->exec_fd, &exec_ehdr)) == NULL) { > ??? goto err; > ? } > > So maybe it's the (uintptr_t)0 that is the problem here. For shared libs instead of 0 it computes the value to add: > > ?????????????? if (lib_base_diff == ZERO_LOAD_ADDRESS ) { > ???????????????? lib_base_diff = calc_prelinked_load_address(ph, lib_fd, &elf_ehdr, link_map_addr); > ???????????????? if (lib_base_diff == INVALID_LOAD_ADDRESS) { > ?????????????????? close(lib_fd); > ?????????????????? return false; > ???????????????? } > ?????????????? } > > ?????????????? lib_base = lib_base_diff + find_base_address(lib_fd, &elf_ehdr); > > So in this case we've actually computed lib_base_diff rather than just assumed 0. > > Chris > > On 7/6/20 10:46 PM, Yasumasa Suenaga wrote: >> Hi Chris, >> >> Your change looks good. >> >> >> BTW I saw JDK-8248876. I'm not sure, but I guess we can fix this issue if we allow PT_INTERP in L118: >> >> ``` >> 105 uintptr_t find_base_address(int fd, ELF_EHDR* ehdr) { >> ????????????????? : >> 115?? // the base address of a shared object is the lowest vaddr of >> 116?? // its loadable segments (PT_LOAD) >> 117?? for (phdr = phbuf, cnt = 0; cnt < ehdr->e_phnum; cnt++, phdr++) { >> 118???? if (phdr->p_type == PT_LOAD && phdr->p_vaddr < baseaddr) { >> 119?????? baseaddr = phdr->p_vaddr; >> 120???? } >> 121?? } >> ``` >> >> /proc//maps shows top of `java` is 0x56543b9df000: >> >> 56543b9df000-56543b9e0000 r--p 00000000 08:10 55770 /usr/lib/jvm/java-11-openjdk-amd64/bin/java >> >> >> `i target` on GDB shows 0x56543b9df000 is .interp section: >> >> Local exec file: >> ??????? `/usr/lib/jvm/java-11-openjdk-amd64/bin/java', file type elf64-x86-64. >> ??????? Entry point: 0x56543b9e0330 >> ??????? 0x000056543b9df318 - 0x000056543b9df334 is .interp >> >> >> Thanks, >> >> Yasumasa >> >> >> On 2020/07/07 13:18, Chris Plummer wrote: >>> Hello, >>> >>> Please help review the following: >>> >>> http://cr.openjdk.java.net/~cjplummer/8248878/webrev.00/index.html >>> https://bugs.openjdk.java.net/browse/JDK-8248878 >>> >>> The explanation of the fix is in the CR. The parent CR, JDK-8248876 [1], explains the issue being addressed. >>> >>> There's no test for this fix yet. It requires the changes I'm making for JDK-8247514 [2], which include changes to "findpc" support and the ClhsdbFindPC.java test that trigger this issue. >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8248876 >>> [2] https://bugs.openjdk.java.net/browse/JDK-8247514 >>> >>> thanks, >>> >>> Chris > From chris.plummer at oracle.com Wed Jul 8 02:17:23 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 7 Jul 2020 19:17:23 -0700 Subject: RFR(T): 8249002: New serviceability/sa/ClhsdbFindPC.java #id2 and #id3 tests are failing with ZGC In-Reply-To: <7B15FCA3-97F1-4A51-888F-468C57EF6B7D@oracle.com> References: <6cf90b18-52b9-03f3-d62b-a6f179b1a067@oracle.com> <7B15FCA3-97F1-4A51-888F-468C57EF6B7D@oracle.com> Message-ID: <7fc7dbd3-b86e-0afc-ca30-a52259f53476@oracle.com> Thanks! On 7/7/20 5:38 PM, Igor Ignatyev wrote: > Hi Chris, > > LGTM > > -- Igor > >> On Jul 7, 2020, at 5:34 PM, Chris Plummer wrote: >> >> Sorry, forgot the link to the CR: >> >> https://bugs.openjdk.java.net/browse/JDK-8249002 >> >> Chris >> >> On 7/7/20 5:32 PM, Chris Plummer wrote: >>> Hello, >>> >>> Please help review this trivial change to problem list a couple of new SA subtests so they are not run with ZGC: >>> >>> diff --git a/test/hotspot/jtreg/ProblemList-zgc.txt b/test/hotspot/jtreg/ProblemList-zgc.txt >>> --- a/test/hotspot/jtreg/ProblemList-zgc.txt >>> +++ b/test/hotspot/jtreg/ProblemList-zgc.txt >>> @@ -34,6 +34,8 @@ >>> serviceability/sa/ClhsdbCDSJstackPrintAll.java 8220624 generic-all >>> serviceability/sa/ClhsdbFindPC.java#id0 8220624 generic-all >>> serviceability/sa/ClhsdbFindPC.java#id1 8220624 generic-all >>> +serviceability/sa/ClhsdbFindPC.java#id2 8220624 generic-all >>> +serviceability/sa/ClhsdbFindPC.java#id3 8220624 generic-all >>> serviceability/sa/ClhsdbInspect.java 8220624 generic-all >>> serviceability/sa/ClhsdbJdis.java 8220624 generic-all >>> serviceability/sa/ClhsdbJhisto.java 8220624 generic-all >>> >>> thanks, >>> >>> Chris >>> >> From chris.plummer at oracle.com Wed Jul 8 02:29:16 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 7 Jul 2020 19:29:16 -0700 Subject: RFR(XS): 8248878: SA: Implement simple workaround for JDK-8248876 In-Reply-To: References: <39ac40d6-a738-3214-d86c-412152a631d4@oracle.com> <8e89d223-0f65-f26e-3bd7-2ca7421a940b@oracle.com> Message-ID: <6d2610d6-5ca1-8455-ddc6-d7f02b21c0d5@oracle.com> Hi Yasumasa, The executable is not opened with pathmap_open: ? if ((ph->core->exec_fd = open(exec_file, O_RDONLY)) < 0) { I think pathmap_open() is just used for libraries. thanks, Chris On 7/7/20 6:18 PM, Yasumasa Suenaga wrote: > Hi Chris, > > SA would use `link_map` to decide to load address, but it does not > seem to contain executable. > I set breakpoint to pathmap_open() and I watched the argument of it, > then I didn't see any executable (`java`) on it. > Maybe current implementation is broken. > > I guess we can use note section in the core for deciding loading address. > I can see valid address (includes executable) from `readelf -n`. > Of course it might be big change for SA... > > > Thanks, > > Yasumasa > > > On 2020/07/07 15:38, Chris Plummer wrote: >> Hi Yasumasa, >> >> Thanks for the review. I tried the following for line 188: >> >> ???? if ((phdr->p_type == PT_LOAD || phdr->p_type == PT_INTERP) && >> phdr->p_vaddr < baseaddr) { >> >> However, "base" still ended up being 0. I added some printfs. For the >> exec file there is both a PT_INTER with p_vaddr of 0x238 and a >> PT_LOAD with p_vaddr 0. I'm not sure which to use, but in either case >> that won't be the proper base when added to 0: >> >> ?? if (add_lib_info_fd(ph, exec_file, ph->core->exec_fd, >> ?????????????????????? (uintptr_t)0 + >> find_base_address(ph->core->exec_fd, &exec_ehdr)) == NULL) { >> ???? goto err; >> ?? } >> >> So maybe it's the (uintptr_t)0 that is the problem here. For shared >> libs instead of 0 it computes the value to add: >> >> ??????????????? if (lib_base_diff == ZERO_LOAD_ADDRESS ) { >> ????????????????? lib_base_diff = calc_prelinked_load_address(ph, >> lib_fd, &elf_ehdr, link_map_addr); >> ????????????????? if (lib_base_diff == INVALID_LOAD_ADDRESS) { >> ??????????????????? close(lib_fd); >> ??????????????????? return false; >> ????????????????? } >> ??????????????? } >> >> ??????????????? lib_base = lib_base_diff + find_base_address(lib_fd, >> &elf_ehdr); >> >> So in this case we've actually computed lib_base_diff rather than >> just assumed 0. >> >> Chris >> >> On 7/6/20 10:46 PM, Yasumasa Suenaga wrote: >>> Hi Chris, >>> >>> Your change looks good. >>> >>> >>> BTW I saw JDK-8248876. I'm not sure, but I guess we can fix this >>> issue if we allow PT_INTERP in L118: >>> >>> ``` >>> 105 uintptr_t find_base_address(int fd, ELF_EHDR* ehdr) { >>> ????????????????? : >>> 115?? // the base address of a shared object is the lowest vaddr of >>> 116?? // its loadable segments (PT_LOAD) >>> 117?? for (phdr = phbuf, cnt = 0; cnt < ehdr->e_phnum; cnt++, phdr++) { >>> 118???? if (phdr->p_type == PT_LOAD && phdr->p_vaddr < baseaddr) { >>> 119?????? baseaddr = phdr->p_vaddr; >>> 120???? } >>> 121?? } >>> ``` >>> >>> /proc//maps shows top of `java` is 0x56543b9df000: >>> >>> 56543b9df000-56543b9e0000 r--p 00000000 08:10 55770 >>> /usr/lib/jvm/java-11-openjdk-amd64/bin/java >>> >>> >>> `i target` on GDB shows 0x56543b9df000 is .interp section: >>> >>> Local exec file: >>> ??????? `/usr/lib/jvm/java-11-openjdk-amd64/bin/java', file type >>> elf64-x86-64. >>> ??????? Entry point: 0x56543b9e0330 >>> ??????? 0x000056543b9df318 - 0x000056543b9df334 is .interp >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2020/07/07 13:18, Chris Plummer wrote: >>>> Hello, >>>> >>>> Please help review the following: >>>> >>>> http://cr.openjdk.java.net/~cjplummer/8248878/webrev.00/index.html >>>> https://bugs.openjdk.java.net/browse/JDK-8248878 >>>> >>>> The explanation of the fix is in the CR. The parent CR, JDK-8248876 >>>> [1], explains the issue being addressed. >>>> >>>> There's no test for this fix yet. It requires the changes I'm >>>> making for JDK-8247514 [2], which include changes to "findpc" >>>> support and the ClhsdbFindPC.java test that trigger this issue. >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8248876 >>>> [2] https://bugs.openjdk.java.net/browse/JDK-8247514 >>>> >>>> thanks, >>>> >>>> Chris >> From fairoz.matte at oracle.com Wed Jul 8 03:47:48 2020 From: fairoz.matte at oracle.com (Fairoz Matte) Date: Tue, 7 Jul 2020 20:47:48 -0700 (PDT) Subject: RFR(s): 8236042: [TESTBUG] serviceability/sa/ClhsdbCDSCore.java fails with -Xcomp -XX:TieredStopAtLevel=1 In-Reply-To: <70057c31-e535-f03a-391d-d181b2ec150b@oracle.com> References: <2abe9fba-e958-4b34-9f92-6bb8d8478f4e@default> <70057c31-e535-f03a-391d-d181b2ec150b@oracle.com> Message-ID: <958fecdf-d7a1-4b22-835e-a75fadda0a84@default> Thanks Chris, for the review comments. I have updated the suggested change. Thanks, Fairoz > -----Original Message----- > From: Chris Plummer > Sent: Wednesday, July 8, 2020 3:38 AM > To: Fairoz Matte ; hotspot-compiler- > dev at openjdk.java.net; serviceability-dev at openjdk.java.net > Subject: Re: RFR(s): 8236042: [TESTBUG] serviceability/sa/ClhsdbCDSCore.java > fails with -Xcomp -XX:TieredStopAtLevel=1 > > Hi Fairoz, > > Looks good, except for the missing space in "if(testJavaOpts...". > > thanks, > > Chris > > On 7/7/20 7:49 AM, Fairoz Matte wrote: > > Hi, > > > > Please review this small test change to consider the scenario when there is no > "printmdo" output > > > > JBS - https://bugs.openjdk.java.net/browse/JDK-8236042 > > Webrev - http://cr.openjdk.java.net/~fmatte/8236042/webrev.00/ > > > > Thanks, > > Fairoz > From chris.plummer at oracle.com Wed Jul 8 06:20:59 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 7 Jul 2020 23:20:59 -0700 Subject: RFR(M): 8247272: SA ELF file support has never worked for 64-bit causing address to symbol name mapping to fail Message-ID: <338118cb-37ff-9d03-f22e-c96322bfd7a6@oracle.com> Hello, Please help review the following: http://cr.openjdk.java.net/~cjplummer/8247272/webrev.00/index.html https://bugs.openjdk.java.net/browse/JDK-8247272 The short story is that SA address to native symbol name mapping/lookup has never worked on 64-bit, and this is due to the java level ELF file support only supporting 32-bit. This CR fixes that, and I believe also maintains 32-bit compatibility, although I have no way of testing that. There is more to the story however on how we got here. Before going into the gory detail below, I just want to point out that currently nothing is using this support, and therefore it is technically not fixing anything, although I did verify that the fixes work (see details below). Also, I intend to remove all the java level ELF file support as part of JDK-8247516 [1]. The only reason I want to push these changes first is because I already did the work to get it working with 64-bit, and would like to get it archived before removing it in case for some reason it is revived in the future. Now for the ugly details on how we got here (and you really don't need to read this unless you have any concerns with what I stated above). It starts with the clhsdb "whatis" command, which was the only (indirect) user of this java level ELF file support. It's implementation is in javascript, so we have not had access to it ever since JDK9 module support broke the SA javascript support (and javascript support is now removed). I started the process of converting "whatis" to java. It is basically the same as the clhsdb "findpc" command, except it also checks for native symbols, which it does with the following code: ? var dso = loadObjectContainingPC(addr); ? var sym = dso.closestSymbolToPC(addr); ? return sym.name + '+' + sym.offset; Converting this to java was trivial. I just stuck support for it in the PointerFinder class, which is what findpc relies on. However, it always failed to successfully lookup a symbol. I found that DSO.closestSymbolToPC() called into the java level ELF support, and that was failing badly. After some debugging I noticed that the values read in for various ELF headers were mostly garbage. It then occurred to me that it was reading in 32-bit values that probably needed to be 64-bit. Sure enough, this code was never converted to 64-bit support. I then went and tried "whatis" on JDK8, the last version where it was available, and it failed there also with 64-bit binaries. So this is why I initially fixed it to work with 64-bit, and also how I tested it (using the modified findpc on a native symbol). But the story continues... DSO.java, and as a consequence the java ELF file support, is used by all our posix ports to do address to symbol lookups. So I figured that after fixing the java level ELF file support for 64-bit, my improved findpc would start working on OSX also. No such luck, and for obvious reasons. OSX uses mach-o files. This ELF code should never have been used for it, and of course has never worked. So I was left trying to figure out how to do OSX address to native symbol lookups. I then recalled that there was a CFrame.closestSymbolToPC() API that did address to native symbol lookups for native stack traces, and wondered how it was ever working (even on linux with the broken ELF 64-bit support). It turns out this takes a very different path to do the lookups, ending up in native code in libsaproc, where we also have ELF file support. I then converted DSO.closestSymbolToPC(addr) to use this libsaproc code instead, and it worked fine. So now there was no need for the java level ELF file support since its only user was DSO.closestSymbolToPC(addr). I should also add that this is the approach that has always been used on windows, with both CFrame.closestSymbolToPC() and DSO.closestSymbolToPC(addr) using the same libsaproc support. There is still a bit more to the story. After diverting DSO.closestSymbolToPC(addr) to the libsaproc lookup code, it still didn't work for OSX. I thought it would just work since the native BsdDebuggerLocal.lookupByName0() is implemented, and it seems to trickle down to the proper lower level APIs to find the symbol, but there were two issues. The first is that for processes there is no support for looking up all the libraries and populating the list of ps_prochandle structures that are used to do the symbol lookups. This was just never implemented (also is why PMap does not work for OSX processes). For core files the ps_prochandle structs are there, but the lookup code was badly broken. That has now been fixed by JDK-8247515 [2], currently out for review. So the end result is we'll have address to native symbol lookup for everything but OSX processes. If? your still here, thanks for listening! Chris [1] https://bugs.openjdk.java.net/browse/JDK-8247516 [2] https://bugs.openjdk.java.net/browse/JDK-8247515 From david.holmes at oracle.com Wed Jul 8 06:27:51 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 8 Jul 2020 16:27:51 +1000 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <0e359819-618f-c17b-a427-bb7e4daa2c58@oracle.com> <3f890bed-6892-e64d-b21b-ca0cbe11784e@oracle.com> <7dc1903b-6295-3864-6301-4c02aacdb971@oss.nttdata.com> <04e09eb7-9483-8f63-4bb4-01c9e814802e@oracle.com> <238a89a3-06b1-90e4-d5a4-8dc2f1ed12ab@oracle.com> <4c43bc2c-551e-d042-8fa7-b63e755888d1@oracle.com> <8c387550-a06e-6320-f453-0e898486befb@oss.nttdata.com> <19cfa2a7-f441-425d-5623-1c11c6a870b6@oss.nttdata.com> Message-ID: <237e8d26-d185-1008-6e07-5b9f36314534@oracle.com> Hi Yasumasa, On 7/07/2020 6:54 pm, Yasumasa Suenaga wrote: > Hi David, Serguei, > > Serguei, thank you for replying even though you are on vacaiton! > > I uploaded new webrev: > > ? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.07/ > ? Diff from previous webrev: > http://hg.openjdk.java.net/jdk/submit/rev/77243b1dcbfe > > c'tor of GetSingleStackTraceClosure has jthread argument in this webrev. > Also it does not contain testcase for GetThreadListStackTraces with all > threads, and OneGetThreadListStackTraces would test main thread only. All those changes are fine in principle for me. One nit/suggestion: src/hotspot/share/prims/jvmtiEnvBase.hpp 544 jthread _java_thread; elsewhere "java_thread" refers to a JavaThread, so to avoid confusion may I suggest this member be named _jthread. I'm going to be away for the next couple of days - sorry - but will try to check email on this if I can. Thanks, David ----- > > Thanks, > > Yasumasa > > > On 2020/07/07 15:13, David Holmes wrote: >> On 7/07/2020 2:57 pm, Yasumasa Suenaga wrote: >>> Hi David, >>> >>> On 2020/07/07 11:31, David Holmes wrote: >>>> Hi Yasumasa, >>>> >>>> Hard to keep up with the changes - especially without incremental >>>> webrevs. >>> >>> Sorry, I will upload diff from previous webrev in the next. >>> >>> >>>> If GetSingleStackTraceClosure also took the jthread as a constructor >>>> arg, then you wouldn't need to recreate a JNI local handle when >>>> calling _collector.fill_frames. It's a small simplification and not >>>> essential at this stage. >>> >>> I think we should get jthread from an argument of do_thread() because >>> do_thread() would pass the thread which are stopped certainly. >>> It might be simplification if we pass _calling_thread to >>> MultipleStackTracesCollector. `jthread` is only needed to store >>> jvmtiStackInfo.thread . What do you think? >> >> I'm not quite sure what you mean. >> >> I think there is a bit of a design wart with direct handshakes in that >> do_thread takes the target JavaThread as an argument. That's useful in >> a case where you want a HandshakeClosure that can be applied to >> multiple threads, but that's not typically what is needed with direct >> handshakes - there is only a single target. With a single-target >> HandshakeClosure you can capture all the "target" information for the >> operation in the closure instance. So if the actual do_thread >> operation wants the jthread corresponding to the target thread then we >> can store that in the closure rather than recomputing it (you could >> assert it is the same but that seems overkill to me). >> >>> >>>> For the test ... I don't see how >>>> Java_GetThreadListStackTraces_checkCallStacks is a valid test. It >>>> gets the stacks of all live threads, then uses that information to >>>> use GetThreadListStackTraces to get the stack for the same set of >>>> threads through a different API. It then compares the two sets of >>>> stacks for each thread expecting them to be the same, but that need >>>> only be the case for the main thread. Other threads could >>>> potentially have a different stack (e.g. if this test is run with >>>> JFR enabled there will be additional threads found.) Further I would >>>> have expected that there already exist tests that check that, for a >>>> given thread (which may be suspended or known to be blocked) the >>>> same stack is found through the two different APIs. >>> >>> vmTestbase/nsk/jvmti/unit/GetAllStackTraces/getallstktr001 would >>> check all of threads via GetThreadListStackTraces() and >>> GetAllStackTraces(), so we might be able to remove >>> GetThreadListStackTraces.java from this webrev. >> >> Yes. The existing test only examines a set of test threads that are >> all blocked on a raw monitor. You do not need to duplicate that test. >> >>> OTOH we don't have testcase for GetThreadListStackTraces() with >>> thread_count == 1, so we need to testcase for it (it is >>> OneGetThreadListStackTraces.java) It would check whether the state of >>> target thread is "waiting" before JNI call to call >>> GetThreadListStackTraces(), >> >> Yes we need to test the special cases introduced by your changes - >> totally agree - and OneGetThreadListStackTraces.java is a good test >> for that. >> >>> and also I expect it would not be run with JFR. (it is not described >>> @run) >> >> The arguments to run with JFR (or a bunch of other things) can be >> passed to the jtreg test harness to be applied to all tests. >> >>> Of course we can check GetThreadListStackTraces() with main thread, >>> but it is not the test for direct handshake for other thread. >> >> Right - that test already exists as per the above. >> >> Thanks, >> David >> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>> On 6/07/2020 11:29 pm, Yasumasa Suenaga wrote: >>>>> Hi Serguei, >>>>> >>>>> Thanks for your comment! >>>>> >>>>> I think C++ is more simple to implement the test agent as you said. >>>>> So I implement it in C++ in new webrev. Could you review again? >>>>> >>>>> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.06/ >>>>> >>>>> Also I refactored libGetThreadListStackTraces.cpp, and I've kept >>>>> exception check after IsSameObject(). >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2020/07/06 16:32, serguei.spitsyn at oracle.com wrote: >>>>>> Hi Yasumasa, >>>>>> >>>>>> Thank you for the update. >>>>>> I think, a pending exception after IsSameObject needs to be checked. >>>>>> >>>>>> The checkStackInfo() needs one more refactoring as I've already >>>>>> suggested. >>>>>> The body of the loop at L68-L78 should be converted to a function >>>>>> check_frame_info. >>>>>> The si1->frame_buffer[i] and si2->frame_buffer[i] will be passed >>>>>> as fi1 and fi2. >>>>>> The index can be passed as well. >>>>>> I'm still suggesting to simplify the local exception_msg to >>>>>> something shorter like err_msg or exc_msg. >>>>>> >>>>>> I'm not sure using fatal is right here: >>>>>> >>>>>> This fragment looks strange: >>>>>> >>>>>> ? 152???? if ((*env)->IsSameObject(env, stack_info[i].thread, >>>>>> thread)) { >>>>>> ? 153?????? target_info = &stack_info[i]; >>>>>> ? 154?????? break; >>>>>> ? 155???? } else if ((*env)->ExceptionOccurred(env)) { >>>>>> ? 156?????? (*env)->ExceptionDescribe(env); >>>>>> ? 157?????? (*env)->FatalError(env, __FILE__); >>>>>> ? 158???? } >>>>>> >>>>>> I expected it to be: >>>>>> >>>>>> ??? jboolean same = (*env)->IsSameObject(env, >>>>>> stack_info[i].thread, thread); >>>>>> ??? if ((*env)->ExceptionOccurred(env)) { >>>>>> ????? (*env)->ExceptionDescribe(env); >>>>>> ????? (*env)->FatalError(env, __FILE__); >>>>>> ??? } >>>>>> ??? if (same) { >>>>>> ????? target_info = &stack_info[i]; >>>>>> ????? break; >>>>>> ??? } >>>>>> >>>>>> Would it better to port this agent to C++ to simplify this code >>>>>> nicer? >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>>> >>>>>> On 7/5/20 06:13, Yasumasa Suenaga wrote: >>>>>>> Hi Serguei, >>>>>>> >>>>>>> Thanks for your comment! >>>>>>> I refactored testcase. Could you review again? >>>>>>> >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.05/ >>>>>>> >>>>>>> It would check Java exception after IsSameObject() call. Does it >>>>>>> need? >>>>>>> Any exceptions are not described in JNI document[1], and JNI >>>>>>> implementation (jni_IsSameObject()) does not seem to throw it. >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> [1] >>>>>>> https://docs.oracle.com/en/java/javase/14/docs/specs/jni/functions.html#issameobject >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 2020/07/05 14:46, serguei.spitsyn at oracle.com wrote: >>>>>>>> Hi Yasumasa, >>>>>>>> >>>>>>>> >>>>>>>> Okay, thanks. >>>>>>>> Then I'm okay to keep the GetSingleStackTraceClosure. >>>>>>>> >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c.html >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c.html >>>>>>>> >>>>>>>> >>>>>>>> I'm not sure the function 'is_same_thread() is needed. >>>>>>>> Why do not use the JNI IsSameObject instead? >>>>>>>> >>>>>>>> It seems to be a typo at L132 and L137. >>>>>>>> You, probably. did not want to print the same information for >>>>>>>> stack_info_1[i].frame_buffer[j].XXX twice. >>>>>>>> >>>>>>>> The code at lines 112-142 is not readable. >>>>>>>> I'd suggest to make a couple of refactoring steps. >>>>>>>> >>>>>>>> First step to simplify this a little bit would be with some >>>>>>>> renaming and getting rid of indexes: >>>>>>>> >>>>>>>> ?? 71 char err_msg[EXCEPTION_MSG_LEN] = {0}; >>>>>>>> ??... >>>>>>>> ??112?? /* Iterate all jvmtiStackInfo to check */ >>>>>>>> ??113?? for (i = 0; i < num_threads, *exception_msg != '\0'; i++) { >>>>>>>> ????????? jvmtiStackInfo *si1 = stack_info_1[i]; >>>>>>>> ????????? jvmtiStackInfo *si2 = stack_info_2[i]; >>>>>>>> ??114???? if (!IsSameObject(env, si1.thread, si2.thread)) { /* >>>>>>>> jvmtiStackInfo::thread */ >>>>>>>> ??115?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>> ??116??????????????? "thread[%d] is different: stack_info_1 = >>>>>>>> %p, stack_info_2 = %p", >>>>>>>> ??117??????????????? i, sinfo1.thread, sinfo2.thread); >>>>>>>> ??118???? } else if (si1.state != si2.state) { /* >>>>>>>> jvmtiStackInfo::state */ >>>>>>>> ??119?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>> ??120??????????????? "state[%d] is different: stack_info_1 = %d, >>>>>>>> stack_info_2 = %d", >>>>>>>> ??121??????????????? i, si1.state, si2.state); >>>>>>>> ??122???? } else if (si1.frame_count != si2.frame_count) { /* >>>>>>>> jvmtiStackInfo::frame_count */ >>>>>>>> ??123?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>> ??124??????????????? "frame_count[%d] is different: stack_info_1 >>>>>>>> = %d, stack_info_2 = %d", >>>>>>>> ??125??????????????? i, si1.frame_count, si2.frame_count); >>>>>>>> ??126???? } else { >>>>>>>> ??127?????? /* Iterate all jvmtiFrameInfo to check */ >>>>>>>> ??128?????? for (j = 0; j < si1.frame_count; j++) { >>>>>>>> ??129???????? if (si1.frame_buffer[j].method != >>>>>>>> si1.frame_buffer[j].method) { /* jvmtiFrameInfo::method */ >>>>>>>> ??130?????????? snprintf(err_msg, sizeof(err_msg), >>>>>>>> ??131??????????????????? "thread [%d] frame_buffer[%d].method is >>>>>>>> different: stack_info_1 = %lx, stack_info_2 = %lx", >>>>>>>> ??132??????????????????? i, j, si1.frame_buffer[j].method, >>>>>>>> si2.frame_buffer[j].method); >>>>>>>> ??133?????????? break; >>>>>>>> ??134???????? } else if (si1.frame_buffer[j].location != >>>>>>>> si1.frame_buffer[j].location) { /* jvmtiFrameInfo::location */ >>>>>>>> ??135?????????? snprintf(err_msg, sizeof(err_msg), >>>>>>>> ??136??????????????????? "thread [%d] frame_buffer[%d].location >>>>>>>> is different: stack_info_1 = %ld, stack_info_2 = %ld", >>>>>>>> ??137??????????????????? i, j, si1.frame_buffer[j].location, >>>>>>>> si2.frame_buffer[j].location); >>>>>>>> ??138?????????? break; >>>>>>>> ??139???????? } >>>>>>>> ??140?????? } >>>>>>>> ??141???? } >>>>>>>> ??142?? } >>>>>>>> >>>>>>>> Another step would be to create functions that implement a body >>>>>>>> of each loop. >>>>>>>> You can use the same techniques to simplify similar place >>>>>>>> (L127-L138) in the libOneGetThreadListStackTraces.c. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Serguei >>>>>>>> >>>>>>>> >>>>>>>> On 7/3/20 15:55, Yasumasa Suenaga wrote: >>>>>>>>> Hi Serguei, >>>>>>>>> >>>>>>>>> I'm not an Oracle employee, so I cannot know real request(s) >>>>>>>>> from your customers. >>>>>>>>> However JDK-8201641 says Dynatrace has requested this enhancement. >>>>>>>>> >>>>>>>>> BTW I haven't heared any request from my customers about this. >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2020/07/04 4:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>> Hi Yasumasa, >>>>>>>>>> >>>>>>>>>> This difference is not that big to care about. >>>>>>>>>> I feel this is really rare case and so, does not worth these >>>>>>>>>> complications. >>>>>>>>>> Do we have a real request from customers to optimize it? >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Serguei >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 7/3/20 01:16, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi Serguei, >>>>>>>>>>> >>>>>>>>>>> Generally I agree with you, but I have concern about the >>>>>>>>>>> difference of the result of GetStackTrace() and >>>>>>>>>>> GetThreadListStackTraces(). >>>>>>>>>>> >>>>>>>>>>> ? GetStackTrace: jvmtiFrameInfo >>>>>>>>>>> ? GetThreadListStackTraces: jvmtiStackInfo >>>>>>>>>>> >>>>>>>>>>> jvmtiStackInfo contains thread state, and it is ensured it is >>>>>>>>>>> the state of the call stack. >>>>>>>>>>> If we want to get both call stack and thread state, we need >>>>>>>>>>> to suspend target thread, and call both GetStackTrace() and >>>>>>>>>>> GetThreadState(). Is it ok? >>>>>>>>>>> >>>>>>>>>>> I was wondering if JDK-8201641 (parent ticket of this change) >>>>>>>>>>> needed them for profiling (dynatrace?) >>>>>>>>>>> If it is responsibility of JVMTI agent implementor, I remove >>>>>>>>>>> this closure. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2020/07/03 16:45, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>> >>>>>>>>>>>> After some thinking I've concluded that I do not like this >>>>>>>>>>>> optimization >>>>>>>>>>>> of the GetThreadListStackTraces with >>>>>>>>>>>> GetSingleStackTraceClosure. >>>>>>>>>>>> >>>>>>>>>>>> We may need more opinions on this but these are my points: >>>>>>>>>>>> ??- it adds some complexity and ugliness >>>>>>>>>>>> ??- a win is doubtful because it has to be a rare case, so >>>>>>>>>>>> that total overhead should not be high >>>>>>>>>>>> ??- if it is really high for some use cases then it is up to >>>>>>>>>>>> the user >>>>>>>>>>>> ??? to optimize it with using GetStackTrace instead >>>>>>>>>>>> >>>>>>>>>>>> In such cases with doubtful overhead I usually prefer the >>>>>>>>>>>> simplicity. >>>>>>>>>>>> >>>>>>>>>>>> Good examples where it makes sense to optimize are checks >>>>>>>>>>>> for target thread to be current thread. >>>>>>>>>>>> In such cases there is no need to suspend the target thread, >>>>>>>>>>>> or use a VMop/HandshakeClosure. >>>>>>>>>>>> For instance, please, see the Monitor functions with the >>>>>>>>>>>> check: (java_thread == calling_thread). >>>>>>>>>>>> Getting information for current thread is frequently used >>>>>>>>>>>> case, e.g. to get info at an event point. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Serguei >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 7/2/20 23:29, Yasumasa Suenaga wrote: >>>>>>>>>>>>> Hi Dan, David, >>>>>>>>>>>>> >>>>>>>>>>>>> I uploaded new webrev. Could you review again? >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/ >>>>>>>>>>>>> >>>>>>>>>>>>> OneGetThreadListStackTraces.java in this webrev would wait >>>>>>>>>>>>> until thread state is transited to "waiting" with spin wait. >>>>>>>>>>>>> CountDownLatch::await call as Dan pointed is fixed in it :) >>>>>>>>>>>>> >>>>>>>>>>>>> Diff from webrev.03: >>>>>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/c9aeb7001e50 >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 2020/07/03 14:15, David Holmes wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>> On 2020/07/03 12:24, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>> On 7/2/20 10:50 PM, David Holmes wrote: >>>>>>>>>>>>>>>>> Sorry I'm responding here without seeing latest webrev >>>>>>>>>>>>>>>>> but there is enough context I think ... >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>> Hi Dan, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I upload new webrev. Could you review again? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>>>>>>>>>>> ???? L1542: ??? // Get stack trace with handshake >>>>>>>>>>>>>>>>>>> ???????? nit - please add a period at the end. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ???? L1591: *stack_info_ptr = op.stack_info(); >>>>>>>>>>>>>>>>>>> ???????? The return parameter should not be touched >>>>>>>>>>>>>>>>>>> unless the return >>>>>>>>>>>>>>>>>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ???? old L1582: ? if (err == JVMTI_ERROR_NONE) { >>>>>>>>>>>>>>>>>>> ???????? Please restore this check. The return >>>>>>>>>>>>>>>>>>> parameter should not >>>>>>>>>>>>>>>>>>> ???????? be touched unless the return code in 'err' >>>>>>>>>>>>>>>>>>> == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> But op.stack_info() will return NULL if the error is >>>>>>>>>>>>>>>>> not JVMTI_ERROR_NONE. Are you (Dan) concerned about >>>>>>>>>>>>>>>>> someone passing in a non-null/initialized out-pointer >>>>>>>>>>>>>>>>> that will be reset to NULL if there was an error? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Actually the way we used to test this in POSIX tests is >>>>>>>>>>>>>>>> to call >>>>>>>>>>>>>>>> an API with known bad parameters and the return >>>>>>>>>>>>>>>> parameter ptr >>>>>>>>>>>>>>>> set to NULL. If the return parameter ptr was touched >>>>>>>>>>>>>>>> when an >>>>>>>>>>>>>>>> error should have been detected on an earlier parameter, >>>>>>>>>>>>>>>> then >>>>>>>>>>>>>>>> the test failed. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ???? L1272: ? if (!jt->is_exiting() && (thread_oop != >>>>>>>>>>>>>>>>>>> NULL)) { >>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second >>>>>>>>>>>>>>>>>>> expression. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>> ???? old L1532: ? _result = >>>>>>>>>>>>>>>>>>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>>>>>>>>>>>>>>>> ???????? This deletion of the _result field threw me >>>>>>>>>>>>>>>>>>> for a minute and then >>>>>>>>>>>>>>>>>>> ???? ? ? I figured out that the field is init to >>>>>>>>>>>>>>>>>>> JVMTI_ERROR_THREAD_NOT_ALIVE >>>>>>>>>>>>>>>>>>> ???????? in the constructor. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ???? L1553: ? if (!jt->is_exiting() && >>>>>>>>>>>>>>>>>>> (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second >>>>>>>>>>>>>>>>>>> expression. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ???? L64: ??????? startSignal.countDown(); >>>>>>>>>>>>>>>>>>> ???????? I was expecting this to be a call to await() >>>>>>>>>>>>>>>>>>> instead of >>>>>>>>>>>>>>>>>>> ???????? countDown(). What am I missing here? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ???????? I think this test might be passing by >>>>>>>>>>>>>>>>>>> accident right now, but... >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Main thread (which call JVMTI functions to test) >>>>>>>>>>>>>>>>>> should wait until test thread is ready. >>>>>>>>>>>>>>>>>> So main thread would wait startSignal, and test thread >>>>>>>>>>>>>>>>>> would count down. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> No! >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> The test thread that previously called obj.wait() now >>>>>>>>>>>>>>>>> calls latch.await(). >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> The main thread that previously called obj.notify() now >>>>>>>>>>>>>>>>> calls latch.countDown(). >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> The main thread continues to spin until it sees the >>>>>>>>>>>>>>>>> target is WAITING before proceeding with the test. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> If I add spin wait to wait until transit target thread >>>>>>>>>>>>>>> state is WAITING (as following), we don't need to call >>>>>>>>>>>>>>> SuspendThread(). >>>>>>>>>>>>>>> Which is better? >>>>>>>>>>>>>> >>>>>>>>>>>>>> The original spin-wait loop checking for? WAITING is >>>>>>>>>>>>>> better because it is the only guarantee that the target >>>>>>>>>>>>>> thread is blocked where you need it to be. suspending the >>>>>>>>>>>>>> thread is racy as you don't know exactly where the suspend >>>>>>>>>>>>>> will hit. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> David >>>>>>>>>>>>>> ----- >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>> /* Wait until the thread state transits to "waiting" */ >>>>>>>>>>>>>>> while (th.getState() != Thread.State.WAITING) { >>>>>>>>>>>>>>> ???? Thread.onSpinWait(); >>>>>>>>>>>>>>> } >>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> For simplify, spin wait is prefer to >>>>>>>>>>>>>>> OneGetThreadListStackTraces.java in webrev.03. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Here's the flow as I see it: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>> ?? - start worker thread >>>>>>>>>>>>>>>> ?? - startSignal.await() >>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>> ?? - startSignal.countDown() >>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>> ?? - stopSignal.await() >>>>>>>>>>>>>>>> ???? - worker is now blocked >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>> ?? - checkCallStacks(th) >>>>>>>>>>>>>>>> ?? - stopSignal.countDown() >>>>>>>>>>>>>>>> ???? - worker is now unblocked >>>>>>>>>>>>>>>> ?? - th.join >>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>> ?? - runs off the end of run() >>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>> ?? - run off the end of main() >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ???? L92: ? jthreads = (jthread >>>>>>>>>>>>>>>>>>> *)malloc(sizeof(jthread) * num_threads); >>>>>>>>>>>>>>>>>>> ???????? You don't check for malloc() failure. >>>>>>>>>>>>>>>>>>> ???????? 'jthreads' is allocated but never freed. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ???? L91: ? result = (*jvmti)->SuspendThread(jvmti, >>>>>>>>>>>>>>>>>>> thread); >>>>>>>>>>>>>>>>>>> ???????? Why are you suspending the thread? >>>>>>>>>>>>>>>>>>> GetAllStackTraces() and >>>>>>>>>>>>>>>>>>> ???????? GetThreadListStackTraces() do not require >>>>>>>>>>>>>>>>>>> the target thread(s) >>>>>>>>>>>>>>>>>>> ???????? to be suspend. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ???????? If you decide not to SuspendThread, then you >>>>>>>>>>>>>>>>>>> don't need the >>>>>>>>>>>>>>>>>>> ???????? AddCapabilities or the ResumeThread calls. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Test thread might not be entered following code >>>>>>>>>>>>>>>>>> (stopSignal.await()). We might see deferent call stack >>>>>>>>>>>>>>>>>> between GetAllStackTraces() and >>>>>>>>>>>>>>>>>> GetThreadListStackTraces(). We cannot control to >>>>>>>>>>>>>>>>>> freeze call stack of test thread in Java code. >>>>>>>>>>>>>>>>>> (I didn't use SuspendThread() at first, but I saw some >>>>>>>>>>>>>>>>>> errors which causes in above.) >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> So we need to call SuspendThread() to ensure we can >>>>>>>>>>>>>>>>>> see same call stack. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> If you are checking that the thread is in state WAITING >>>>>>>>>>>>>>>>> then it cannot escape from that state and you can >>>>>>>>>>>>>>>>> sample the stack multiple times from any API and get >>>>>>>>>>>>>>>>> the same result. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I suspect the errors you saw were from the apparent >>>>>>>>>>>>>>>>> incorrect use of the CountDownLatch. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> With the flow outlined above, the worker thread should be >>>>>>>>>>>>>>>> nicely blocked in stopSignal.await() when stuff is sampled. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could review again? >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Updates look fine - thanks. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> One minor nit: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> 1274 _collector.allocate_and_fill_stacks(1); >>>>>>>>>>>>>>>>>>>>> 1275 _collector.set_result(JVMTI_ERROR_NONE); >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> In the other places where you use _collector you >>>>>>>>>>>>>>>>>>>>> rely on result being initialized to >>>>>>>>>>>>>>>>>>>>> JVMTI_ERROR_NONE, rather than setting it directly >>>>>>>>>>>>>>>>>>>>> after allocate_and_fill_stacks(). >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> ? 820 >>>>>>>>>>>>>>>>>>>>>>>>>>> assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> ? 821 >>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>>>>>>>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>> ? 822 current_thread == >>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>> ? 823????????? "at safepoint / handshake or >>>>>>>>>>>>>>>>>>>>>>>>>>> target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is >>>>>>>>>>>>>>>>>>>>>>>>>>> necessary, as even if the target is suspended >>>>>>>>>>>>>>>>>>>>>>>>>>> we must still be at a safepoint or in a >>>>>>>>>>>>>>>>>>>>>>>>>>> handshake with it. Makes me wonder if we used >>>>>>>>>>>>>>>>>>>>>>>>>>> to allow a racy stacktrace operation on a >>>>>>>>>>>>>>>>>>>>>>>>>>> suspended thread, assuming it would remain >>>>>>>>>>>>>>>>>>>>>>>>>>> suspended? >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> This function (JvmtiEnvBase::get_stack_trace()) >>>>>>>>>>>>>>>>>>>>>> can be called to get own stack trace. For example, >>>>>>>>>>>>>>>>>>>>>> we can call GetStackTrace() for current thread at >>>>>>>>>>>>>>>>>>>>>> JVMTI event. >>>>>>>>>>>>>>>>>>>>>> So I changed assert as below: >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>> ??820?? assert(current_thread == java_thread || >>>>>>>>>>>>>>>>>>>>>> ??821 SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>> ??822????????? current_thread == >>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>> ??823????????? "call by myself / at safepoint / at >>>>>>>>>>>>>>>>>>>>>> handshake"); >>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Yep good catch. I hope current tests caught that. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> They would be tested in >>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ >>>>>>>>>>>>>>>>>>>> (own call stacks), and getstacktr003 (call stacks in >>>>>>>>>>>>>>>>>>>> other thread). >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Speaking of tests ... >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> In the native code I think you need to check the >>>>>>>>>>>>>>>>>>>>> success of all JNI methods that can throw >>>>>>>>>>>>>>>>>>>>> exceptions - otherwise I believe the tests may >>>>>>>>>>>>>>>>>>>>> trigger warnings if -Xcheck:jni is used with them. >>>>>>>>>>>>>>>>>>>>> See for example: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I updated testcases to check JNI and JVMTI function >>>>>>>>>>>>>>>>>>>> calls. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> In the Java code the target thread: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ?? 45???? public void run() { >>>>>>>>>>>>>>>>>>>>> ?? 46?????? try { >>>>>>>>>>>>>>>>>>>>> ?? 47?????????? synchronized (lock) { >>>>>>>>>>>>>>>>>>>>> ?? 48?????????????? lock.wait(); >>>>>>>>>>>>>>>>>>>>> ?? 49 System.out.println("OK"); >>>>>>>>>>>>>>>>>>>>> ?? 50?????????? } >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> is potentially susceptible to spurious wakeups. >>>>>>>>>>>>>>>>>>>>> Using a CountDownLatch would be robust. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the >>>>>>>>>>>>>>>>>>>>>>>>>>> current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> If so, we can call make_local() in L1272 without >>>>>>>>>>>>>>>>>>>>>>>> JavaThread (or we can pass current thread to >>>>>>>>>>>>>>>>>>>>>>>> make_local()). Is it right? >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>> 1272 >>>>>>>>>>>>>>>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>>>>>>>>>>>>>>> thread_oop), >>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Sorry I got confused, _calling_thread may not be >>>>>>>>>>>>>>>>>>>>>>> the current thread as we could be executing the >>>>>>>>>>>>>>>>>>>>>>> handshake in the target thread itself. So the >>>>>>>>>>>>>>>>>>>>>>> ResourceMark is correct as-is (implicitly for >>>>>>>>>>>>>>>>>>>>>>> current thread). >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> The argument to fill_frames will be used in the >>>>>>>>>>>>>>>>>>>>>>> jvmtiStackInfo and passed back to the >>>>>>>>>>>>>>>>>>>>>>> _calling_thread, so it must be created via >>>>>>>>>>>>>>>>>>>>>>> make_local(_calling_thread, ...) as you presently >>>>>>>>>>>>>>>>>>>>>>> have. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thank you for reviewing! I will update new >>>>>>>>>>>>>>>>>>>>>>>>>> webrev tomorrow. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : >>>>>>>>>>>>>>>>>>>>>>>>>>> public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> ? 498 class VM_GetAllStackTraces : public >>>>>>>>>>>>>>>>>>>>>>>>>>> VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>> ? 500?? JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>> ? 502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of >>>>>>>>>>>>>>>>>>>>>>>>>>> another class like that as it may not be on >>>>>>>>>>>>>>>>>>>>>>>>>>> the stack. I think >>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector should not >>>>>>>>>>>>>>>>>>>>>>>>>>> extend any allocation class, and should >>>>>>>>>>>>>>>>>>>>>>>>>>> always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>>>>>>>>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>>>>>>>>>>>>>>>>> ??? : >>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> class GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>> ?? private: >>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I updated webrev for 8242428. Could you >>>>>>>>>>>>>>>>>>>>>>>>>>>> review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>> This change migrate to use direct handshake >>>>>>>>>>>>>>>>>>>>>>>>>>>> for GetStackTrace() and >>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() (when >>>>>>>>>>>>>>>>>>>>>>>>>>>> thread_count == 1). >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> This looks really good now! I only have a few >>>>>>>>>>>>>>>>>>>>>>>>>>> nits below. There is one thing I don't like >>>>>>>>>>>>>>>>>>>>>>>>>>> about it but it requires a change to the main >>>>>>>>>>>>>>>>>>>>>>>>>>> Handshake logic to address - in >>>>>>>>>>>>>>>>>>>>>>>>>>> JvmtiEnv::GetThreadListStackTraces you have >>>>>>>>>>>>>>>>>>>>>>>>>>> to create a ThreadsListHandle to convert the >>>>>>>>>>>>>>>>>>>>>>>>>>> jthread to a JavaThread, but then the >>>>>>>>>>>>>>>>>>>>>>>>>>> Handshake::execute_direct creates another >>>>>>>>>>>>>>>>>>>>>>>>>>> ThreadsListHandle internally. That's a waste. >>>>>>>>>>>>>>>>>>>>>>>>>>> I will discuss with Robbin and file a RFE to >>>>>>>>>>>>>>>>>>>>>>>>>>> have an overload of execute_direct that takes >>>>>>>>>>>>>>>>>>>>>>>>>>> an existing TLH. Actually it's worse than >>>>>>>>>>>>>>>>>>>>>>>>>>> that because we have another TLH in use at >>>>>>>>>>>>>>>>>>>>>>>>>>> the entry point for the JVMTI functions, so I >>>>>>>>>>>>>>>>>>>>>>>>>>> think there may be some scope for simplifying >>>>>>>>>>>>>>>>>>>>>>>>>>> the use of TLH instances - future RFE. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> ??451 GetStackTraceClosure(JvmtiEnv *env, >>>>>>>>>>>>>>>>>>>>>>>>>>> jint start_depth, jint max_count, >>>>>>>>>>>>>>>>>>>>>>>>>>> ??452 jvmtiFrameInfo* frame_buffer, jint* >>>>>>>>>>>>>>>>>>>>>>>>>>> count_ptr) >>>>>>>>>>>>>>>>>>>>>>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>>>>>>>>>>>>>>>> ??454?????? _env(env), >>>>>>>>>>>>>>>>>>>>>>>>>>> _start_depth(start_depth), >>>>>>>>>>>>>>>>>>>>>>>>>>> _max_count(max_count), >>>>>>>>>>>>>>>>>>>>>>>>>>> ??455 _frame_buffer(frame_buffer), >>>>>>>>>>>>>>>>>>>>>>>>>>> _count_ptr(count_ptr), >>>>>>>>>>>>>>>>>>>>>>>>>>> ??456 _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Nit: can you do one initializer per line please. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : >>>>>>>>>>>>>>>>>>>>>>>>>>> public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> ??498 class VM_GetAllStackTraces : public >>>>>>>>>>>>>>>>>>>>>>>>>>> VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>> ??500?? JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>> ??502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of >>>>>>>>>>>>>>>>>>>>>>>>>>> another class like that as it may not be on >>>>>>>>>>>>>>>>>>>>>>>>>>> the stack. I think >>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector should not >>>>>>>>>>>>>>>>>>>>>>>>>>> extend any allocation class, and should >>>>>>>>>>>>>>>>>>>>>>>>>>> always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> 481 MultipleStackTracesCollector(JvmtiEnv >>>>>>>>>>>>>>>>>>>>>>>>>>> *env, jint max_frame_count) { >>>>>>>>>>>>>>>>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>>>>>>>>>>>>>>>> ??483???? _max_frame_count = max_frame_count; >>>>>>>>>>>>>>>>>>>>>>>>>>> ??484 _frame_count_total = 0; >>>>>>>>>>>>>>>>>>>>>>>>>>> ??485???? _head = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>> ??486???? _stack_info = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>> ??487???? _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>>>>>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> As you are touching this can you change it to >>>>>>>>>>>>>>>>>>>>>>>>>>> use an initializer list as you did for the >>>>>>>>>>>>>>>>>>>>>>>>>>> HandshakeClosure, and please keep one item >>>>>>>>>>>>>>>>>>>>>>>>>>> per line. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> ??820 >>>>>>>>>>>>>>>>>>>>>>>>>>> assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> ??821 >>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>>>>>>>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == >>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>> ??823????????? "at safepoint / handshake or >>>>>>>>>>>>>>>>>>>>>>>>>>> target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is >>>>>>>>>>>>>>>>>>>>>>>>>>> necessary, as even if the target is suspended >>>>>>>>>>>>>>>>>>>>>>>>>>> we must still be at a safepoint or in a >>>>>>>>>>>>>>>>>>>>>>>>>>> handshake with it. Makes me wonder if we used >>>>>>>>>>>>>>>>>>>>>>>>>>> to allow a racy stacktrace operation on a >>>>>>>>>>>>>>>>>>>>>>>>>>> suspended thread, assuming it would remain >>>>>>>>>>>>>>>>>>>>>>>>>>> suspended? >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>>>>>>>>>>>>>>>>>>> 1269 >>>>>>>>>>>>>>>>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && >>>>>>>>>>>>>>>>>>>>>>>>>>> (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 >>>>>>>>>>>>>>>>>>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>>>>>>>>>>>>>>>>>> thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> It is frustrating that this entire call chain >>>>>>>>>>>>>>>>>>>>>>>>>>> started with a jthread reference, which we >>>>>>>>>>>>>>>>>>>>>>>>>>> converted to a JavaThread, only to eventually >>>>>>>>>>>>>>>>>>>>>>>>>>> need to convert it back to a jthread! I think >>>>>>>>>>>>>>>>>>>>>>>>>>> there is some scope for simplification here >>>>>>>>>>>>>>>>>>>>>>>>>>> but not as part of this change. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the >>>>>>>>>>>>>>>>>>>>>>>>>>> current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> I'm still pondering the test logic but wanted >>>>>>>>>>>>>>>>>>>>>>>>>>> to send this now. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetThreadListStackTrace (for >>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces) and >>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetAllStackTraces (for GetAllStackTraces) >>>>>>>>>>>>>>>>>>>>>>>>>>>> have inherited VM_GetMultipleStackTraces VM >>>>>>>>>>>>>>>>>>>>>>>>>>>> operation which provides the feature to >>>>>>>>>>>>>>>>>>>>>>>>>>>> generate jvmtiStackInfo. I modified >>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetMultipleStackTraces to a normal C++ >>>>>>>>>>>>>>>>>>>>>>>>>>>> class to share with HandshakeClosure for >>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces >>>>>>>>>>>>>>>>>>>>>>>>>>>> (GetSingleStackTraceClosure). >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I added new testcases which test >>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() with thread_count >>>>>>>>>>>>>>>>>>>>>>>>>>>> == 1 and with all threads. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in >>>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jvmti serviceability/jdwp >>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti vmTestbase/nsk/jdi >>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? JBS: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? webrev: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change replace following VM operations >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to direct handshake. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces >>>>>>>>>>>>>>>>>>>>>>>>>>>>> (GetThreadListStackTrace()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTrace() uses direct >>>>>>>>>>>>>>>>>>>>>>>>>>>>> handshake if thread count == 1. In other >>>>>>>>>>>>>>>>>>>>>>>>>>>>> case (thread count > 1), it would be >>>>>>>>>>>>>>>>>>>>>>>>>>>>> performed as VM operation >>>>>>>>>>>>>>>>>>>>>>>>>>>>> (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Caller of VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>> (JvmtiEnvThreadState::reset_current_location()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>> might be called at safepoint. So I added >>>>>>>>>>>>>>>>>>>>>>>>>>>>> safepoint check in its caller. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in >>>>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jvmti serviceability/jdwp >>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti vmTestbase/nsk/jdi >>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/ns >>>>>>>>>>>>>>>>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I tested it on submit repo, then it >>>>>>>>>>>>>>>>>>>>>>>>>>>>> has execution error >>>>>>>>>>>>>>>>>>>>>>>>>>>>> (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) >>>>>>>>>>>>>>>>>>>>>>>>>>>>> due to dependency error. So I think it does >>>>>>>>>>>>>>>>>>>>>>>>>>>>> not occur by this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>> From suenaga at oss.nttdata.com Wed Jul 8 08:04:09 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Wed, 8 Jul 2020 17:04:09 +0900 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: <237e8d26-d185-1008-6e07-5b9f36314534@oracle.com> References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <3f890bed-6892-e64d-b21b-ca0cbe11784e@oracle.com> <7dc1903b-6295-3864-6301-4c02aacdb971@oss.nttdata.com> <04e09eb7-9483-8f63-4bb4-01c9e814802e@oracle.com> <238a89a3-06b1-90e4-d5a4-8dc2f1ed12ab@oracle.com> <4c43bc2c-551e-d042-8fa7-b63e755888d1@oracle.com> <8c387550-a06e-6320-f453-0e898486befb@oss.nttdata.com> <19cfa2a7-f441-425d-5623-1c11c6a870b6@oss.nttdata.com> <237e8d26-d185-1008-6e07-5b9f36314534@oracle.com> Message-ID: Hi David, On 2020/07/08 15:27, David Holmes wrote: > Hi Yasumasa, > > On 7/07/2020 6:54 pm, Yasumasa Suenaga wrote: >> Hi David, Serguei, >> >> Serguei, thank you for replying even though you are on vacaiton! >> >> I uploaded new webrev: >> >> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.07/ >> ?? Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/77243b1dcbfe >> >> c'tor of GetSingleStackTraceClosure has jthread argument in this webrev. >> Also it does not contain testcase for GetThreadListStackTraces with all threads, and OneGetThreadListStackTraces would test main thread only. > > All those changes are fine in principle for me. One nit/suggestion: > > src/hotspot/share/prims/jvmtiEnvBase.hpp > > ?544?? jthread _java_thread; > > elsewhere "java_thread" refers to a JavaThread, so to avoid confusion may I suggest this member be named _jthread. I uploaded new webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/ca6263dbdc87 > I'm going to be away for the next couple of days - sorry - but will try to check email on this if I can. Thanks! Yasumasa > Thanks, > David > ----- > >> >> Thanks, >> >> Yasumasa >> >> >> On 2020/07/07 15:13, David Holmes wrote: >>> On 7/07/2020 2:57 pm, Yasumasa Suenaga wrote: >>>> Hi David, >>>> >>>> On 2020/07/07 11:31, David Holmes wrote: >>>>> Hi Yasumasa, >>>>> >>>>> Hard to keep up with the changes - especially without incremental webrevs. >>>> >>>> Sorry, I will upload diff from previous webrev in the next. >>>> >>>> >>>>> If GetSingleStackTraceClosure also took the jthread as a constructor arg, then you wouldn't need to recreate a JNI local handle when calling _collector.fill_frames. It's a small simplification and not essential at this stage. >>>> >>>> I think we should get jthread from an argument of do_thread() because do_thread() would pass the thread which are stopped certainly. >>>> It might be simplification if we pass _calling_thread to MultipleStackTracesCollector. `jthread` is only needed to store jvmtiStackInfo.thread . What do you think? >>> >>> I'm not quite sure what you mean. >>> >>> I think there is a bit of a design wart with direct handshakes in that do_thread takes the target JavaThread as an argument. That's useful in a case where you want a HandshakeClosure that can be applied to multiple threads, but that's not typically what is needed with direct handshakes - there is only a single target. With a single-target HandshakeClosure you can capture all the "target" information for the operation in the closure instance. So if the actual do_thread operation wants the jthread corresponding to the target thread then we can store that in the closure rather than recomputing it (you could assert it is the same but that seems overkill to me). >>> >>>> >>>>> For the test ... I don't see how Java_GetThreadListStackTraces_checkCallStacks is a valid test. It gets the stacks of all live threads, then uses that information to use GetThreadListStackTraces to get the stack for the same set of threads through a different API. It then compares the two sets of stacks for each thread expecting them to be the same, but that need only be the case for the main thread. Other threads could potentially have a different stack (e.g. if this test is run with JFR enabled there will be additional threads found.) Further I would have expected that there already exist tests that check that, for a given thread (which may be suspended or known to be blocked) the same stack is found through the two different APIs. >>>> >>>> vmTestbase/nsk/jvmti/unit/GetAllStackTraces/getallstktr001 would check all of threads via GetThreadListStackTraces() and GetAllStackTraces(), so we might be able to remove GetThreadListStackTraces.java from this webrev. >>> >>> Yes. The existing test only examines a set of test threads that are all blocked on a raw monitor. You do not need to duplicate that test. >>> >>>> OTOH we don't have testcase for GetThreadListStackTraces() with thread_count == 1, so we need to testcase for it (it is OneGetThreadListStackTraces.java) It would check whether the state of target thread is "waiting" before JNI call to call GetThreadListStackTraces(), >>> >>> Yes we need to test the special cases introduced by your changes - totally agree - and OneGetThreadListStackTraces.java is a good test for that. >>> >>>> and also I expect it would not be run with JFR. (it is not described @run) >>> >>> The arguments to run with JFR (or a bunch of other things) can be passed to the jtreg test harness to be applied to all tests. >>> >>>> Of course we can check GetThreadListStackTraces() with main thread, but it is not the test for direct handshake for other thread. >>> >>> Right - that test already exists as per the above. >>> >>> Thanks, >>> David >>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>> On 6/07/2020 11:29 pm, Yasumasa Suenaga wrote: >>>>>> Hi Serguei, >>>>>> >>>>>> Thanks for your comment! >>>>>> >>>>>> I think C++ is more simple to implement the test agent as you said. >>>>>> So I implement it in C++ in new webrev. Could you review again? >>>>>> >>>>>> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.06/ >>>>>> >>>>>> Also I refactored libGetThreadListStackTraces.cpp, and I've kept exception check after IsSameObject(). >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2020/07/06 16:32, serguei.spitsyn at oracle.com wrote: >>>>>>> Hi Yasumasa, >>>>>>> >>>>>>> Thank you for the update. >>>>>>> I think, a pending exception after IsSameObject needs to be checked. >>>>>>> >>>>>>> The checkStackInfo() needs one more refactoring as I've already suggested. >>>>>>> The body of the loop at L68-L78 should be converted to a function check_frame_info. >>>>>>> The si1->frame_buffer[i] and si2->frame_buffer[i] will be passed as fi1 and fi2. >>>>>>> The index can be passed as well. >>>>>>> I'm still suggesting to simplify the local exception_msg to something shorter like err_msg or exc_msg. >>>>>>> >>>>>>> I'm not sure using fatal is right here: >>>>>>> >>>>>>> This fragment looks strange: >>>>>>> >>>>>>> ? 152???? if ((*env)->IsSameObject(env, stack_info[i].thread, thread)) { >>>>>>> ? 153?????? target_info = &stack_info[i]; >>>>>>> ? 154?????? break; >>>>>>> ? 155???? } else if ((*env)->ExceptionOccurred(env)) { >>>>>>> ? 156?????? (*env)->ExceptionDescribe(env); >>>>>>> ? 157?????? (*env)->FatalError(env, __FILE__); >>>>>>> ? 158???? } >>>>>>> >>>>>>> I expected it to be: >>>>>>> >>>>>>> ??? jboolean same = (*env)->IsSameObject(env, stack_info[i].thread, thread); >>>>>>> ??? if ((*env)->ExceptionOccurred(env)) { >>>>>>> ????? (*env)->ExceptionDescribe(env); >>>>>>> ????? (*env)->FatalError(env, __FILE__); >>>>>>> ??? } >>>>>>> ??? if (same) { >>>>>>> ????? target_info = &stack_info[i]; >>>>>>> ????? break; >>>>>>> ??? } >>>>>>> >>>>>>> Would it better to port this agent to C++ to simplify this code nicer? >>>>>>> >>>>>>> Thanks, >>>>>>> Serguei >>>>>>> >>>>>>> >>>>>>> On 7/5/20 06:13, Yasumasa Suenaga wrote: >>>>>>>> Hi Serguei, >>>>>>>> >>>>>>>> Thanks for your comment! >>>>>>>> I refactored testcase. Could you review again? >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.05/ >>>>>>>> >>>>>>>> It would check Java exception after IsSameObject() call. Does it need? >>>>>>>> Any exceptions are not described in JNI document[1], and JNI implementation (jni_IsSameObject()) does not seem to throw it. >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> [1] https://docs.oracle.com/en/java/javase/14/docs/specs/jni/functions.html#issameobject >>>>>>>> >>>>>>>> >>>>>>>> On 2020/07/05 14:46, serguei.spitsyn at oracle.com wrote: >>>>>>>>> Hi Yasumasa, >>>>>>>>> >>>>>>>>> >>>>>>>>> Okay, thanks. >>>>>>>>> Then I'm okay to keep the GetSingleStackTraceClosure. >>>>>>>>> >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c.html >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c.html >>>>>>>>> >>>>>>>>> I'm not sure the function 'is_same_thread() is needed. >>>>>>>>> Why do not use the JNI IsSameObject instead? >>>>>>>>> >>>>>>>>> It seems to be a typo at L132 and L137. >>>>>>>>> You, probably. did not want to print the same information for stack_info_1[i].frame_buffer[j].XXX twice. >>>>>>>>> >>>>>>>>> The code at lines 112-142 is not readable. >>>>>>>>> I'd suggest to make a couple of refactoring steps. >>>>>>>>> >>>>>>>>> First step to simplify this a little bit would be with some renaming and getting rid of indexes: >>>>>>>>> >>>>>>>>> ?? 71 char err_msg[EXCEPTION_MSG_LEN] = {0}; >>>>>>>>> ??... >>>>>>>>> ??112?? /* Iterate all jvmtiStackInfo to check */ >>>>>>>>> ??113?? for (i = 0; i < num_threads, *exception_msg != '\0'; i++) { >>>>>>>>> ????????? jvmtiStackInfo *si1 = stack_info_1[i]; >>>>>>>>> ????????? jvmtiStackInfo *si2 = stack_info_2[i]; >>>>>>>>> ??114???? if (!IsSameObject(env, si1.thread, si2.thread)) { /* jvmtiStackInfo::thread */ >>>>>>>>> ??115?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>> ??116??????????????? "thread[%d] is different: stack_info_1 = %p, stack_info_2 = %p", >>>>>>>>> ??117??????????????? i, sinfo1.thread, sinfo2.thread); >>>>>>>>> ??118???? } else if (si1.state != si2.state) { /* jvmtiStackInfo::state */ >>>>>>>>> ??119?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>> ??120??????????????? "state[%d] is different: stack_info_1 = %d, stack_info_2 = %d", >>>>>>>>> ??121??????????????? i, si1.state, si2.state); >>>>>>>>> ??122???? } else if (si1.frame_count != si2.frame_count) { /* jvmtiStackInfo::frame_count */ >>>>>>>>> ??123?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>> ??124??????????????? "frame_count[%d] is different: stack_info_1 = %d, stack_info_2 = %d", >>>>>>>>> ??125??????????????? i, si1.frame_count, si2.frame_count); >>>>>>>>> ??126???? } else { >>>>>>>>> ??127?????? /* Iterate all jvmtiFrameInfo to check */ >>>>>>>>> ??128?????? for (j = 0; j < si1.frame_count; j++) { >>>>>>>>> ??129???????? if (si1.frame_buffer[j].method != si1.frame_buffer[j].method) { /* jvmtiFrameInfo::method */ >>>>>>>>> ??130?????????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>> ??131??????????????????? "thread [%d] frame_buffer[%d].method is different: stack_info_1 = %lx, stack_info_2 = %lx", >>>>>>>>> ??132??????????????????? i, j, si1.frame_buffer[j].method, si2.frame_buffer[j].method); >>>>>>>>> ??133?????????? break; >>>>>>>>> ??134???????? } else if (si1.frame_buffer[j].location != si1.frame_buffer[j].location) { /* jvmtiFrameInfo::location */ >>>>>>>>> ??135?????????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>> ??136??????????????????? "thread [%d] frame_buffer[%d].location is different: stack_info_1 = %ld, stack_info_2 = %ld", >>>>>>>>> ??137??????????????????? i, j, si1.frame_buffer[j].location, si2.frame_buffer[j].location); >>>>>>>>> ??138?????????? break; >>>>>>>>> ??139???????? } >>>>>>>>> ??140?????? } >>>>>>>>> ??141???? } >>>>>>>>> ??142?? } >>>>>>>>> >>>>>>>>> Another step would be to create functions that implement a body of each loop. >>>>>>>>> You can use the same techniques to simplify similar place (L127-L138) in the libOneGetThreadListStackTraces.c. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Serguei >>>>>>>>> >>>>>>>>> >>>>>>>>> On 7/3/20 15:55, Yasumasa Suenaga wrote: >>>>>>>>>> Hi Serguei, >>>>>>>>>> >>>>>>>>>> I'm not an Oracle employee, so I cannot know real request(s) from your customers. >>>>>>>>>> However JDK-8201641 says Dynatrace has requested this enhancement. >>>>>>>>>> >>>>>>>>>> BTW I haven't heared any request from my customers about this. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2020/07/04 4:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>> >>>>>>>>>>> This difference is not that big to care about. >>>>>>>>>>> I feel this is really rare case and so, does not worth these complications. >>>>>>>>>>> Do we have a real request from customers to optimize it? >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Serguei >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 7/3/20 01:16, Yasumasa Suenaga wrote: >>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>> >>>>>>>>>>>> Generally I agree with you, but I have concern about the difference of the result of GetStackTrace() and GetThreadListStackTraces(). >>>>>>>>>>>> >>>>>>>>>>>> ? GetStackTrace: jvmtiFrameInfo >>>>>>>>>>>> ? GetThreadListStackTraces: jvmtiStackInfo >>>>>>>>>>>> >>>>>>>>>>>> jvmtiStackInfo contains thread state, and it is ensured it is the state of the call stack. >>>>>>>>>>>> If we want to get both call stack and thread state, we need to suspend target thread, and call both GetStackTrace() and GetThreadState(). Is it ok? >>>>>>>>>>>> >>>>>>>>>>>> I was wondering if JDK-8201641 (parent ticket of this change) needed them for profiling (dynatrace?) >>>>>>>>>>>> If it is responsibility of JVMTI agent implementor, I remove this closure. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 2020/07/03 16:45, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>> >>>>>>>>>>>>> After some thinking I've concluded that I do not like this optimization >>>>>>>>>>>>> of the GetThreadListStackTraces with GetSingleStackTraceClosure. >>>>>>>>>>>>> >>>>>>>>>>>>> We may need more opinions on this but these are my points: >>>>>>>>>>>>> ??- it adds some complexity and ugliness >>>>>>>>>>>>> ??- a win is doubtful because it has to be a rare case, so that total overhead should not be high >>>>>>>>>>>>> ??- if it is really high for some use cases then it is up to the user >>>>>>>>>>>>> ??? to optimize it with using GetStackTrace instead >>>>>>>>>>>>> >>>>>>>>>>>>> In such cases with doubtful overhead I usually prefer the simplicity. >>>>>>>>>>>>> >>>>>>>>>>>>> Good examples where it makes sense to optimize are checks for target thread to be current thread. >>>>>>>>>>>>> In such cases there is no need to suspend the target thread, or use a VMop/HandshakeClosure. >>>>>>>>>>>>> For instance, please, see the Monitor functions with the check: (java_thread == calling_thread). >>>>>>>>>>>>> Getting information for current thread is frequently used case, e.g. to get info at an event point. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Serguei >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 7/2/20 23:29, Yasumasa Suenaga wrote: >>>>>>>>>>>>>> Hi Dan, David, >>>>>>>>>>>>>> >>>>>>>>>>>>>> I uploaded new webrev. Could you review again? >>>>>>>>>>>>>> >>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> OneGetThreadListStackTraces.java in this webrev would wait until thread state is transited to "waiting" with spin wait. >>>>>>>>>>>>>> CountDownLatch::await call as Dan pointed is fixed in it :) >>>>>>>>>>>>>> >>>>>>>>>>>>>> Diff from webrev.03: >>>>>>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/c9aeb7001e50 >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 2020/07/03 14:15, David Holmes wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>> On 2020/07/03 12:24, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>> On 7/2/20 10:50 PM, David Holmes wrote: >>>>>>>>>>>>>>>>>> Sorry I'm responding here without seeing latest webrev but there is enough context I think ... >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>> Hi Dan, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> I upload new webrev. Could you review again? >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>>>>>>>>>>>> ???? L1542: ??? // Get stack trace with handshake >>>>>>>>>>>>>>>>>>>> ???????? nit - please add a period at the end. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> ???? L1591: *stack_info_ptr = op.stack_info(); >>>>>>>>>>>>>>>>>>>> ???????? The return parameter should not be touched unless the return >>>>>>>>>>>>>>>>>>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> ???? old L1582: ? if (err == JVMTI_ERROR_NONE) { >>>>>>>>>>>>>>>>>>>> ???????? Please restore this check. The return parameter should not >>>>>>>>>>>>>>>>>>>> ???????? be touched unless the return code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> But op.stack_info() will return NULL if the error is not JVMTI_ERROR_NONE. Are you (Dan) concerned about someone passing in a non-null/initialized out-pointer that will be reset to NULL if there was an error? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Actually the way we used to test this in POSIX tests is to call >>>>>>>>>>>>>>>>> an API with known bad parameters and the return parameter ptr >>>>>>>>>>>>>>>>> set to NULL. If the return parameter ptr was touched when an >>>>>>>>>>>>>>>>> error should have been detected on an earlier parameter, then >>>>>>>>>>>>>>>>> the test failed. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> ???? L1272: ? if (!jt->is_exiting() && (thread_oop != NULL)) { >>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>> ???? old L1532: ? _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>>>>>>>>>>>>>>>>> ???????? This deletion of the _result field threw me for a minute and then >>>>>>>>>>>>>>>>>>>> ???? ? ? I figured out that the field is init to JVMTI_ERROR_THREAD_NOT_ALIVE >>>>>>>>>>>>>>>>>>>> ???????? in the constructor. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> ???? L1553: ? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>> ???? L64: ??????? startSignal.countDown(); >>>>>>>>>>>>>>>>>>>> ???????? I was expecting this to be a call to await() instead of >>>>>>>>>>>>>>>>>>>> ???????? countDown(). What am I missing here? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> ???????? I think this test might be passing by accident right now, but... >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Main thread (which call JVMTI functions to test) should wait until test thread is ready. >>>>>>>>>>>>>>>>>>> So main thread would wait startSignal, and test thread would count down. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> No! >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> The test thread that previously called obj.wait() now calls latch.await(). >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> The main thread that previously called obj.notify() now calls latch.countDown(). >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> The main thread continues to spin until it sees the target is WAITING before proceeding with the test. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> If I add spin wait to wait until transit target thread state is WAITING (as following), we don't need to call SuspendThread(). >>>>>>>>>>>>>>>> Which is better? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The original spin-wait loop checking for? WAITING is better because it is the only guarantee that the target thread is blocked where you need it to be. suspending the thread is racy as you don't know exactly where the suspend will hit. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> David >>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>> /* Wait until the thread state transits to "waiting" */ >>>>>>>>>>>>>>>> while (th.getState() != Thread.State.WAITING) { >>>>>>>>>>>>>>>> ???? Thread.onSpinWait(); >>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> For simplify, spin wait is prefer to OneGetThreadListStackTraces.java in webrev.03. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Here's the flow as I see it: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>> ?? - start worker thread >>>>>>>>>>>>>>>>> ?? - startSignal.await() >>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>> ?? - startSignal.countDown() >>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>> ?? - stopSignal.await() >>>>>>>>>>>>>>>>> ???? - worker is now blocked >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>> ?? - checkCallStacks(th) >>>>>>>>>>>>>>>>> ?? - stopSignal.countDown() >>>>>>>>>>>>>>>>> ???? - worker is now unblocked >>>>>>>>>>>>>>>>> ?? - th.join >>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>> ?? - runs off the end of run() >>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>> ?? - run off the end of main() >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>> ???? L92: ? jthreads = (jthread *)malloc(sizeof(jthread) * num_threads); >>>>>>>>>>>>>>>>>>>> ???????? You don't check for malloc() failure. >>>>>>>>>>>>>>>>>>>> ???????? 'jthreads' is allocated but never freed. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>> ???? L91: ? result = (*jvmti)->SuspendThread(jvmti, thread); >>>>>>>>>>>>>>>>>>>> ???????? Why are you suspending the thread? GetAllStackTraces() and >>>>>>>>>>>>>>>>>>>> ???????? GetThreadListStackTraces() do not require the target thread(s) >>>>>>>>>>>>>>>>>>>> ???????? to be suspend. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> ???????? If you decide not to SuspendThread, then you don't need the >>>>>>>>>>>>>>>>>>>> ???????? AddCapabilities or the ResumeThread calls. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Test thread might not be entered following code (stopSignal.await()). We might see deferent call stack between GetAllStackTraces() and GetThreadListStackTraces(). We cannot control to freeze call stack of test thread in Java code. >>>>>>>>>>>>>>>>>>> (I didn't use SuspendThread() at first, but I saw some errors which causes in above.) >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> So we need to call SuspendThread() to ensure we can see same call stack. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> If you are checking that the thread is in state WAITING then it cannot escape from that state and you can sample the stack multiple times from any API and get the same result. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I suspect the errors you saw were from the apparent incorrect use of the CountDownLatch. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> With the flow outlined above, the worker thread should be >>>>>>>>>>>>>>>>> nicely blocked in stopSignal.await() when stuff is sampled. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could review again? >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Updates look fine - thanks. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> One minor nit: >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> 1274 _collector.allocate_and_fill_stacks(1); >>>>>>>>>>>>>>>>>>>>>> 1275 _collector.set_result(JVMTI_ERROR_NONE); >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> In the other places where you use _collector you rely on result being initialized to JVMTI_ERROR_NONE, rather than setting it directly after allocate_and_fill_stacks(). >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 823????????? "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> This function (JvmtiEnvBase::get_stack_trace()) can be called to get own stack trace. For example, we can call GetStackTrace() for current thread at JVMTI event. >>>>>>>>>>>>>>>>>>>>>>> So I changed assert as below: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>> ??820?? assert(current_thread == java_thread || >>>>>>>>>>>>>>>>>>>>>>> ??821 SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>> ??822????????? current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>> ??823????????? "call by myself / at safepoint / at handshake"); >>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Yep good catch. I hope current tests caught that. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> They would be tested in vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ (own call stacks), and getstacktr003 (call stacks in other thread). >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Speaking of tests ... >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> In the native code I think you need to check the success of all JNI methods that can throw exceptions - otherwise I believe the tests may trigger warnings if -Xcheck:jni is used with them. See for example: >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> I updated testcases to check JNI and JVMTI function calls. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> In the Java code the target thread: >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> ?? 45???? public void run() { >>>>>>>>>>>>>>>>>>>>>> ?? 46?????? try { >>>>>>>>>>>>>>>>>>>>>> ?? 47?????????? synchronized (lock) { >>>>>>>>>>>>>>>>>>>>>> ?? 48?????????????? lock.wait(); >>>>>>>>>>>>>>>>>>>>>> ?? 49 System.out.println("OK"); >>>>>>>>>>>>>>>>>>>>>> ?? 50?????????? } >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> is potentially susceptible to spurious wakeups. Using a CountDownLatch would be robust. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> If so, we can call make_local() in L1272 without JavaThread (or we can pass current thread to make_local()). Is it right? >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Sorry I got confused, _calling_thread may not be the current thread as we could be executing the handshake in the target thread itself. So the ResourceMark is correct as-is (implicitly for current thread). >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> The argument to fill_frames will be used in the jvmtiStackInfo and passed back to the _calling_thread, so it must be created via make_local(_calling_thread, ...) as you presently have. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you for reviewing! I will update new webrev tomorrow. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 500?? JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>>>>>>>>>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>>>>>>>>>>>>>>>>>> ??? : >>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> class GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>> ?? private: >>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I updated webrev for 8242428. Could you review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change migrate to use direct handshake for GetStackTrace() and GetThreadListStackTraces() (when thread_count == 1). >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks really good now! I only have a few nits below. There is one thing I don't like about it but it requires a change to the main Handshake logic to address - in JvmtiEnv::GetThreadListStackTraces you have to create a ThreadsListHandle to convert the jthread to a JavaThread, but then the Handshake::execute_direct creates another ThreadsListHandle internally. That's a waste. I will discuss with Robbin and file a RFE to have an overload of execute_direct that takes an existing TLH. Actually it's worse than that because we have another TLH in use at the entry point for the JVMTI functions, so I think there may be some scope for simplifying the use of TLH instances - future RFE. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> ??451 GetStackTraceClosure(JvmtiEnv *env, jint start_depth, jint max_count, >>>>>>>>>>>>>>>>>>>>>>>>>>>> ??452 jvmtiFrameInfo* frame_buffer, jint* count_ptr) >>>>>>>>>>>>>>>>>>>>>>>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>>>>>>>>>>>>>>>>> ??454?????? _env(env), _start_depth(start_depth), _max_count(max_count), >>>>>>>>>>>>>>>>>>>>>>>>>>>> ??455 _frame_buffer(frame_buffer), _count_ptr(count_ptr), >>>>>>>>>>>>>>>>>>>>>>>>>>>> ??456 _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Nit: can you do one initializer per line please. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> ??498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>> ??500?? JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>> ??502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> 481 MultipleStackTracesCollector(JvmtiEnv *env, jint max_frame_count) { >>>>>>>>>>>>>>>>>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>>>>>>>>>>>>>>>>> ??483???? _max_frame_count = max_frame_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>> ??484 _frame_count_total = 0; >>>>>>>>>>>>>>>>>>>>>>>>>>>> ??485???? _head = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>> ??486???? _stack_info = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>> ??487???? _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>>>>>>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> As you are touching this can you change it to use an initializer list as you did for the HandshakeClosure, and please keep one item per line. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> ??820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>>>> ??821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>> ??823????????? "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>>>>>>>>>>>>>>>>>>>> 1269 >>>>>>>>>>>>>>>>>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> It is frustrating that this entire call chain started with a jthread reference, which we converted to a JavaThread, only to eventually need to convert it back to a jthread! I think there is some scope for simplification here but not as part of this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm still pondering the test logic but wanted to send this now. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetThreadListStackTrace (for GetThreadListStackTraces) and VM_GetAllStackTraces (for GetAllStackTraces) have inherited VM_GetMultipleStackTraces VM operation which provides the feature to generate jvmtiStackInfo. I modified VM_GetMultipleStackTraces to a normal C++ class to share with HandshakeClosure for GetThreadListStackTraces (GetSingleStackTraceClosure). >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I added new testcases which test GetThreadListStackTraces() with thread_count == 1 and with all threads. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/nsk/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change replace following VM operations to direct handshake. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces (GetThreadListStackTrace()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTrace() uses direct handshake if thread count == 1. In other case (thread count > 1), it would be performed as VM operation (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Caller of VM_GetCurrentLocation (JvmtiEnvThreadState::reset_current_location()) might be called at safepoint. So I added safepoint check in its caller. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/ns >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I tested it on submit repo, then it has execution error (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) due to dependency error. So I think it does not occur by this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>> From aph at redhat.com Wed Jul 8 09:49:35 2020 From: aph at redhat.com (Andrew Haley) Date: Wed, 8 Jul 2020 10:49:35 +0100 Subject: RFR(S): 8247762: [aarch64] Timeout in .../HeapDumpTestWithActiveProcess.java due to inf. loop in AARCH64CurrentFrameGuess.run() In-Reply-To: <6322edbb-ab38-4956-6a29-5b1c3d48c132@oracle.com> References: <36c12487-cfc9-3823-c4a3-7605629beca5@redhat.com> <78a1d527-bd3f-6bc2-7880-ca5f34215267@oracle.com> <6322edbb-ab38-4956-6a29-5b1c3d48c132@oracle.com> Message-ID: <24bcb11a-ae3e-a29c-bcc0-a377eb90e19b@redhat.com> On 06/07/2020 20:54, Patric Hedlin wrote: > Andrew, > > Something you can live with? Totally. Great explanation too. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From patric.hedlin at oracle.com Wed Jul 8 10:07:14 2020 From: patric.hedlin at oracle.com (Patric Hedlin) Date: Wed, 8 Jul 2020 12:07:14 +0200 Subject: RFR(S): 8247762: [aarch64] Timeout in .../HeapDumpTestWithActiveProcess.java due to inf. loop in AARCH64CurrentFrameGuess.run() In-Reply-To: <24bcb11a-ae3e-a29c-bcc0-a377eb90e19b@redhat.com> References: <36c12487-cfc9-3823-c4a3-7605629beca5@redhat.com> <78a1d527-bd3f-6bc2-7880-ca5f34215267@oracle.com> <6322edbb-ab38-4956-6a29-5b1c3d48c132@oracle.com> <24bcb11a-ae3e-a29c-bcc0-a377eb90e19b@redhat.com> Message-ID: <0d569e36-5a8e-df1a-ab34-63ebe864f9ec@oracle.com> Thanks for reviewing Andrew. /Patric On 2020-07-08 11:49, Andrew Haley wrote: > On 06/07/2020 20:54, Patric Hedlin wrote: >> Andrew, >> >> Something you can live with? > Totally. Great explanation too. > From kevin.walls at oracle.com Wed Jul 8 10:23:57 2020 From: kevin.walls at oracle.com (Kevin Walls) Date: Wed, 8 Jul 2020 11:23:57 +0100 Subject: RFR(M): 8247515: OSX pc_to_symbol() lookup does not work with core files In-Reply-To: <38e1887e-a646-abc0-0b7e-53339af1d684@oracle.com> References: <092ef5a8-c973-e377-de91-b2002a1b8fe7@oracle.com> <38e1887e-a646-abc0-0b7e-53339af1d684@oracle.com> Message-ID: <3c31622b-831e-d46f-daa6-d641cbf755b1@oracle.com> Sure -- I was thinking lowest_offset_from_sym initialising starting at a high positive integer (that would now be PTRDIFF_MAX I think) to save a comparison with e.g. -1, you can just check if the new offset is less than lowest_offset_from_sym With the ptrdiff_t change you made, this all looks good to me however you decide. 8-) On 07/07/2020 21:17, Chris Plummer wrote: > Hi Kevin, > > Thanks for the review. Yes, that lack of initialization of > lowest_offset_from_sym is a bug. I'm real surprised the compiler > didn't catch it as it will be uninitialized garbage the first time it > is referenced. Fortunately usually the eventual offset is very small > if not 0, so probably this never prevented a proper match. I think > there's also another bug: > > ?415?????? uintptr_t offset_from_sym = offset - sym->offset; > > "offset" is the passed in offset, essentially the address of the > symbol we are interested in, but given as an offset from the start of > the DSO. "sym->offset" is also an offset from the start of the DSO. It > could be located before or after "offset". This means the math could > result in a negative number, which when converted to unsigned would be > a very large positive number. This happens whenever you check a symbol > that is actually located after the address you are looking up. The end > result is harmless, because it just means there's no way we will match > that symbol, which is what you want, but it would be good to clean > this up. > > I think what is best is to use ptrdiff_t and initialize > lowest_offset_from_sym to -1. I've updated the webrev: > > http://cr.openjdk.java.net/~cjplummer/8247515/webrev.01/index.html > > thanks, > > Chris > > On 7/7/20 4:09 AM, Kevin Walls wrote: >> Hi Chris, >> >> Yes I think this looks good. >> >> Question: In nearest_symbol, do we need to initialize >> lowest_offset_from_sym to something impossibly high, as if it >> defaults to zero we never find a better/nearer result? >> >> Thanks >> Kevin >> >> >> On 07/07/2020 06:10, Chris Plummer wrote: >>> Hello, >>> >>> Please help review the following: >>> >>> http://cr.openjdk.java.net/~cjplummer/8247515/webrev.00/index.html >>> https://bugs.openjdk.java.net/browse/JDK-8247515 >>> >>> The CR contains a description of the issues being addressed. There >>> is also no test for this symbol lookup support yet. It will be there >>> after I push JDK-8247516 and? JDK-8247514, which are both blocked by >>> the CR. >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8247516 >>> [2] https://bugs.openjdk.java.net/browse/JDK-8247514 >>> >>> thanks, >>> >>> Chris >>> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kevin.walls at oracle.com Wed Jul 8 10:47:30 2020 From: kevin.walls at oracle.com (Kevin Walls) Date: Wed, 8 Jul 2020 11:47:30 +0100 Subject: RFR(M): 8247272: SA ELF file support has never worked for 64-bit causing address to symbol name mapping to fail In-Reply-To: <338118cb-37ff-9d03-f22e-c96322bfd7a6@oracle.com> References: <338118cb-37ff-9d03-f22e-c96322bfd7a6@oracle.com> Message-ID: Hi Chris -- This is a great story/history lesson. You could if you like, edit those comments in ElfFileParser.java so "Elf32_Addr" as they will contain either "Elf64_Addr or Elf32_Addr", similarly Elf64_Off.? The other Elf64 fields are the same as the 32 bit ones. Yes, the symbol fields are ordered differently. So all looks good to me! Thanks Kevin On 08/07/2020 07:20, Chris Plummer wrote: > Hello, > > Please help review the following: > > http://cr.openjdk.java.net/~cjplummer/8247272/webrev.00/index.html > https://bugs.openjdk.java.net/browse/JDK-8247272 > > The short story is that SA address to native symbol name > mapping/lookup has never worked on 64-bit, and this is due to the java > level ELF file support only supporting 32-bit. This CR fixes that, and > I believe also maintains 32-bit compatibility, although I have no way > of testing that. > > There is more to the story however on how we got here. Before going > into the gory detail below, I just want to point out that currently > nothing is using this support, and therefore it is technically not > fixing anything, although I did verify that the fixes work (see > details below). Also, I intend to remove all the java level ELF file > support as part of JDK-8247516 [1]. The only reason I want to push > these changes first is because I already did the work to get it > working with 64-bit, and would like to get it archived before removing > it in case for some reason it is revived in the future. > > Now for the ugly details on how we got here (and you really don't need > to read this unless you have any concerns with what I stated above). > It starts with the clhsdb "whatis" command, which was the only > (indirect) user of this java level ELF file support. It's > implementation is in javascript, so we have not had access to it ever > since JDK9 module support broke the SA javascript support (and > javascript support is now removed). I started the process of > converting "whatis" to java. It is basically the same as the clhsdb > "findpc" command, except it also checks for native symbols, which it > does with the following code: > > ? var dso = loadObjectContainingPC(addr); > ? var sym = dso.closestSymbolToPC(addr); > ? return sym.name + '+' + sym.offset; > > Converting this to java was trivial. I just stuck support for it in > the PointerFinder class, which is what findpc relies on. However, it > always failed to successfully lookup a symbol. I found that > DSO.closestSymbolToPC() called into the java level ELF support, and > that was failing badly. After some debugging I noticed that the values > read in for various ELF headers were mostly garbage. It then occurred > to me that it was reading in 32-bit values that probably needed to be > 64-bit. Sure enough, this code was never converted to 64-bit support. > I then went and tried "whatis" on JDK8, the last version where it was > available, and it failed there also with 64-bit binaries. So this is > why I initially fixed it to work with 64-bit, and also how I tested it > (using the modified findpc on a native symbol). But the story > continues... > > DSO.java, and as a consequence the java ELF file support, is used by > all our posix ports to do address to symbol lookups. So I figured that > after fixing the java level ELF file support for 64-bit, my improved > findpc would start working on OSX also. No such luck, and for obvious > reasons. OSX uses mach-o files. This ELF code should never have been > used for it, and of course has never worked. > > So I was left trying to figure out how to do OSX address to native > symbol lookups. I then recalled that there was a > CFrame.closestSymbolToPC() API that did address to native symbol > lookups for native stack traces, and wondered how it was ever working > (even on linux with the broken ELF 64-bit support). It turns out this > takes a very different path to do the lookups, ending up in native > code in libsaproc, where we also have ELF file support. I then > converted DSO.closestSymbolToPC(addr) to use this libsaproc code > instead, and it worked fine. So now there was no need for the java > level ELF file support since its only user was > DSO.closestSymbolToPC(addr). I should also add that this is the > approach that has always been used on windows, with both > CFrame.closestSymbolToPC() and DSO.closestSymbolToPC(addr) using the > same libsaproc support. > > There is still a bit more to the story. After diverting > DSO.closestSymbolToPC(addr) to the libsaproc lookup code, it still > didn't work for OSX. I thought it would just work since the native > BsdDebuggerLocal.lookupByName0() is implemented, and it seems to > trickle down to the proper lower level APIs to find the symbol, but > there were two issues. The first is that for processes there is no > support for looking up all the libraries and populating the list of > ps_prochandle structures that are used to do the symbol lookups. This > was just never implemented (also is why PMap does not work for OSX > processes). For core files the ps_prochandle structs are there, but > the lookup code was badly broken. That has now been fixed by > JDK-8247515 [2], currently out for review. So the end result is we'll > have address to native symbol lookup for everything but OSX processes. > > If? your still here, thanks for listening! > > Chris > > [1] https://bugs.openjdk.java.net/browse/JDK-8247516 > [2] https://bugs.openjdk.java.net/browse/JDK-8247515 > > > > From david.holmes at oracle.com Wed Jul 8 12:29:23 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 8 Jul 2020 22:29:23 +1000 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <7dc1903b-6295-3864-6301-4c02aacdb971@oss.nttdata.com> <04e09eb7-9483-8f63-4bb4-01c9e814802e@oracle.com> <238a89a3-06b1-90e4-d5a4-8dc2f1ed12ab@oracle.com> <4c43bc2c-551e-d042-8fa7-b63e755888d1@oracle.com> <8c387550-a06e-6320-f453-0e898486befb@oss.nttdata.com> <19cfa2a7-f441-425d-5623-1c11c6a870b6@oss.nttdata.com> <237e8d26-d185-1008-6e07-5b9f36314534@oracle.com> Message-ID: <0fb1d65e-d888-7fcf-548b-58172d507ea8@oracle.com> On 8/07/2020 6:04 pm, Yasumasa Suenaga wrote: > Hi David, > > On 2020/07/08 15:27, David Holmes wrote: >> Hi Yasumasa, >> >> On 7/07/2020 6:54 pm, Yasumasa Suenaga wrote: >>> Hi David, Serguei, >>> >>> Serguei, thank you for replying even though you are on vacaiton! >>> >>> I uploaded new webrev: >>> >>> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.07/ >>> ?? Diff from previous webrev: >>> http://hg.openjdk.java.net/jdk/submit/rev/77243b1dcbfe >>> >>> c'tor of GetSingleStackTraceClosure has jthread argument in this webrev. >>> Also it does not contain testcase for GetThreadListStackTraces with >>> all threads, and OneGetThreadListStackTraces would test main thread >>> only. >> >> All those changes are fine in principle for me. One nit/suggestion: >> >> src/hotspot/share/prims/jvmtiEnvBase.hpp >> >> ??544?? jthread _java_thread; >> >> elsewhere "java_thread" refers to a JavaThread, so to avoid confusion >> may I suggest this member be named _jthread. > > I uploaded new webrev: > > ? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ > ? Diff from previous webrev: > http://hg.openjdk.java.net/jdk/submit/rev/ca6263dbdc87 That looks fine. Thanks, David ----- > >> I'm going to be away for the next couple of days - sorry - but will >> try to check email on this if I can. > > Thanks! > > > Yasumasa > > >> Thanks, >> David >> ----- >> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2020/07/07 15:13, David Holmes wrote: >>>> On 7/07/2020 2:57 pm, Yasumasa Suenaga wrote: >>>>> Hi David, >>>>> >>>>> On 2020/07/07 11:31, David Holmes wrote: >>>>>> Hi Yasumasa, >>>>>> >>>>>> Hard to keep up with the changes - especially without incremental >>>>>> webrevs. >>>>> >>>>> Sorry, I will upload diff from previous webrev in the next. >>>>> >>>>> >>>>>> If GetSingleStackTraceClosure also took the jthread as a >>>>>> constructor arg, then you wouldn't need to recreate a JNI local >>>>>> handle when calling _collector.fill_frames. It's a small >>>>>> simplification and not essential at this stage. >>>>> >>>>> I think we should get jthread from an argument of do_thread() >>>>> because do_thread() would pass the thread which are stopped certainly. >>>>> It might be simplification if we pass _calling_thread to >>>>> MultipleStackTracesCollector. `jthread` is only needed to store >>>>> jvmtiStackInfo.thread . What do you think? >>>> >>>> I'm not quite sure what you mean. >>>> >>>> I think there is a bit of a design wart with direct handshakes in >>>> that do_thread takes the target JavaThread as an argument. That's >>>> useful in a case where you want a HandshakeClosure that can be >>>> applied to multiple threads, but that's not typically what is needed >>>> with direct handshakes - there is only a single target. With a >>>> single-target HandshakeClosure you can capture all the "target" >>>> information for the operation in the closure instance. So if the >>>> actual do_thread operation wants the jthread corresponding to the >>>> target thread then we can store that in the closure rather than >>>> recomputing it (you could assert it is the same but that seems >>>> overkill to me). >>>> >>>>> >>>>>> For the test ... I don't see how >>>>>> Java_GetThreadListStackTraces_checkCallStacks is a valid test. It >>>>>> gets the stacks of all live threads, then uses that information to >>>>>> use GetThreadListStackTraces to get the stack for the same set of >>>>>> threads through a different API. It then compares the two sets of >>>>>> stacks for each thread expecting them to be the same, but that >>>>>> need only be the case for the main thread. Other threads could >>>>>> potentially have a different stack (e.g. if this test is run with >>>>>> JFR enabled there will be additional threads found.) Further I >>>>>> would have expected that there already exist tests that check >>>>>> that, for a given thread (which may be suspended or known to be >>>>>> blocked) the same stack is found through the two different APIs. >>>>> >>>>> vmTestbase/nsk/jvmti/unit/GetAllStackTraces/getallstktr001 would >>>>> check all of threads via GetThreadListStackTraces() and >>>>> GetAllStackTraces(), so we might be able to remove >>>>> GetThreadListStackTraces.java from this webrev. >>>> >>>> Yes. The existing test only examines a set of test threads that are >>>> all blocked on a raw monitor. You do not need to duplicate that test. >>>> >>>>> OTOH we don't have testcase for GetThreadListStackTraces() with >>>>> thread_count == 1, so we need to testcase for it (it is >>>>> OneGetThreadListStackTraces.java) It would check whether the state >>>>> of target thread is "waiting" before JNI call to call >>>>> GetThreadListStackTraces(), >>>> >>>> Yes we need to test the special cases introduced by your changes - >>>> totally agree - and OneGetThreadListStackTraces.java is a good test >>>> for that. >>>> >>>>> and also I expect it would not be run with JFR. (it is not >>>>> described @run) >>>> >>>> The arguments to run with JFR (or a bunch of other things) can be >>>> passed to the jtreg test harness to be applied to all tests. >>>> >>>>> Of course we can check GetThreadListStackTraces() with main thread, >>>>> but it is not the test for direct handshake for other thread. >>>> >>>> Right - that test already exists as per the above. >>>> >>>> Thanks, >>>> David >>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>> >>>>>> On 6/07/2020 11:29 pm, Yasumasa Suenaga wrote: >>>>>>> Hi Serguei, >>>>>>> >>>>>>> Thanks for your comment! >>>>>>> >>>>>>> I think C++ is more simple to implement the test agent as you said. >>>>>>> So I implement it in C++ in new webrev. Could you review again? >>>>>>> >>>>>>> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.06/ >>>>>>> >>>>>>> Also I refactored libGetThreadListStackTraces.cpp, and I've kept >>>>>>> exception check after IsSameObject(). >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2020/07/06 16:32, serguei.spitsyn at oracle.com wrote: >>>>>>>> Hi Yasumasa, >>>>>>>> >>>>>>>> Thank you for the update. >>>>>>>> I think, a pending exception after IsSameObject needs to be >>>>>>>> checked. >>>>>>>> >>>>>>>> The checkStackInfo() needs one more refactoring as I've already >>>>>>>> suggested. >>>>>>>> The body of the loop at L68-L78 should be converted to a >>>>>>>> function check_frame_info. >>>>>>>> The si1->frame_buffer[i] and si2->frame_buffer[i] will be passed >>>>>>>> as fi1 and fi2. >>>>>>>> The index can be passed as well. >>>>>>>> I'm still suggesting to simplify the local exception_msg to >>>>>>>> something shorter like err_msg or exc_msg. >>>>>>>> >>>>>>>> I'm not sure using fatal is right here: >>>>>>>> >>>>>>>> This fragment looks strange: >>>>>>>> >>>>>>>> ? 152???? if ((*env)->IsSameObject(env, stack_info[i].thread, >>>>>>>> thread)) { >>>>>>>> ? 153?????? target_info = &stack_info[i]; >>>>>>>> ? 154?????? break; >>>>>>>> ? 155???? } else if ((*env)->ExceptionOccurred(env)) { >>>>>>>> ? 156?????? (*env)->ExceptionDescribe(env); >>>>>>>> ? 157?????? (*env)->FatalError(env, __FILE__); >>>>>>>> ? 158???? } >>>>>>>> >>>>>>>> I expected it to be: >>>>>>>> >>>>>>>> ??? jboolean same = (*env)->IsSameObject(env, >>>>>>>> stack_info[i].thread, thread); >>>>>>>> ??? if ((*env)->ExceptionOccurred(env)) { >>>>>>>> ????? (*env)->ExceptionDescribe(env); >>>>>>>> ????? (*env)->FatalError(env, __FILE__); >>>>>>>> ??? } >>>>>>>> ??? if (same) { >>>>>>>> ????? target_info = &stack_info[i]; >>>>>>>> ????? break; >>>>>>>> ??? } >>>>>>>> >>>>>>>> Would it better to port this agent to C++ to simplify this code >>>>>>>> nicer? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Serguei >>>>>>>> >>>>>>>> >>>>>>>> On 7/5/20 06:13, Yasumasa Suenaga wrote: >>>>>>>>> Hi Serguei, >>>>>>>>> >>>>>>>>> Thanks for your comment! >>>>>>>>> I refactored testcase. Could you review again? >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.05/ >>>>>>>>> >>>>>>>>> It would check Java exception after IsSameObject() call. Does >>>>>>>>> it need? >>>>>>>>> Any exceptions are not described in JNI document[1], and JNI >>>>>>>>> implementation (jni_IsSameObject()) does not seem to throw it. >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> [1] >>>>>>>>> https://docs.oracle.com/en/java/javase/14/docs/specs/jni/functions.html#issameobject >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2020/07/05 14:46, serguei.spitsyn at oracle.com wrote: >>>>>>>>>> Hi Yasumasa, >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Okay, thanks. >>>>>>>>>> Then I'm okay to keep the GetSingleStackTraceClosure. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c.html >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c.html >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> I'm not sure the function 'is_same_thread() is needed. >>>>>>>>>> Why do not use the JNI IsSameObject instead? >>>>>>>>>> >>>>>>>>>> It seems to be a typo at L132 and L137. >>>>>>>>>> You, probably. did not want to print the same information for >>>>>>>>>> stack_info_1[i].frame_buffer[j].XXX twice. >>>>>>>>>> >>>>>>>>>> The code at lines 112-142 is not readable. >>>>>>>>>> I'd suggest to make a couple of refactoring steps. >>>>>>>>>> >>>>>>>>>> First step to simplify this a little bit would be with some >>>>>>>>>> renaming and getting rid of indexes: >>>>>>>>>> >>>>>>>>>> ?? 71 char err_msg[EXCEPTION_MSG_LEN] = {0}; >>>>>>>>>> ??... >>>>>>>>>> ??112?? /* Iterate all jvmtiStackInfo to check */ >>>>>>>>>> ??113?? for (i = 0; i < num_threads, *exception_msg != '\0'; >>>>>>>>>> i++) { >>>>>>>>>> ????????? jvmtiStackInfo *si1 = stack_info_1[i]; >>>>>>>>>> ????????? jvmtiStackInfo *si2 = stack_info_2[i]; >>>>>>>>>> ??114???? if (!IsSameObject(env, si1.thread, si2.thread)) { /* >>>>>>>>>> jvmtiStackInfo::thread */ >>>>>>>>>> ??115?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>> ??116??????????????? "thread[%d] is different: stack_info_1 = >>>>>>>>>> %p, stack_info_2 = %p", >>>>>>>>>> ??117??????????????? i, sinfo1.thread, sinfo2.thread); >>>>>>>>>> ??118???? } else if (si1.state != si2.state) { /* >>>>>>>>>> jvmtiStackInfo::state */ >>>>>>>>>> ??119?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>> ??120??????????????? "state[%d] is different: stack_info_1 = >>>>>>>>>> %d, stack_info_2 = %d", >>>>>>>>>> ??121??????????????? i, si1.state, si2.state); >>>>>>>>>> ??122???? } else if (si1.frame_count != si2.frame_count) { /* >>>>>>>>>> jvmtiStackInfo::frame_count */ >>>>>>>>>> ??123?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>> ??124??????????????? "frame_count[%d] is different: >>>>>>>>>> stack_info_1 = %d, stack_info_2 = %d", >>>>>>>>>> ??125??????????????? i, si1.frame_count, si2.frame_count); >>>>>>>>>> ??126???? } else { >>>>>>>>>> ??127?????? /* Iterate all jvmtiFrameInfo to check */ >>>>>>>>>> ??128?????? for (j = 0; j < si1.frame_count; j++) { >>>>>>>>>> ??129???????? if (si1.frame_buffer[j].method != >>>>>>>>>> si1.frame_buffer[j].method) { /* jvmtiFrameInfo::method */ >>>>>>>>>> ??130?????????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>> ??131??????????????????? "thread [%d] frame_buffer[%d].method >>>>>>>>>> is different: stack_info_1 = %lx, stack_info_2 = %lx", >>>>>>>>>> ??132??????????????????? i, j, si1.frame_buffer[j].method, >>>>>>>>>> si2.frame_buffer[j].method); >>>>>>>>>> ??133?????????? break; >>>>>>>>>> ??134???????? } else if (si1.frame_buffer[j].location != >>>>>>>>>> si1.frame_buffer[j].location) { /* jvmtiFrameInfo::location */ >>>>>>>>>> ??135?????????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>> ??136??????????????????? "thread [%d] >>>>>>>>>> frame_buffer[%d].location is different: stack_info_1 = %ld, >>>>>>>>>> stack_info_2 = %ld", >>>>>>>>>> ??137??????????????????? i, j, si1.frame_buffer[j].location, >>>>>>>>>> si2.frame_buffer[j].location); >>>>>>>>>> ??138?????????? break; >>>>>>>>>> ??139???????? } >>>>>>>>>> ??140?????? } >>>>>>>>>> ??141???? } >>>>>>>>>> ??142?? } >>>>>>>>>> >>>>>>>>>> Another step would be to create functions that implement a >>>>>>>>>> body of each loop. >>>>>>>>>> You can use the same techniques to simplify similar place >>>>>>>>>> (L127-L138) in the libOneGetThreadListStackTraces.c. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Serguei >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 7/3/20 15:55, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi Serguei, >>>>>>>>>>> >>>>>>>>>>> I'm not an Oracle employee, so I cannot know real request(s) >>>>>>>>>>> from your customers. >>>>>>>>>>> However JDK-8201641 says Dynatrace has requested this >>>>>>>>>>> enhancement. >>>>>>>>>>> >>>>>>>>>>> BTW I haven't heared any request from my customers about this. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2020/07/04 4:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>> >>>>>>>>>>>> This difference is not that big to care about. >>>>>>>>>>>> I feel this is really rare case and so, does not worth these >>>>>>>>>>>> complications. >>>>>>>>>>>> Do we have a real request from customers to optimize it? >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Serguei >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 7/3/20 01:16, Yasumasa Suenaga wrote: >>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>> >>>>>>>>>>>>> Generally I agree with you, but I have concern about the >>>>>>>>>>>>> difference of the result of GetStackTrace() and >>>>>>>>>>>>> GetThreadListStackTraces(). >>>>>>>>>>>>> >>>>>>>>>>>>> ? GetStackTrace: jvmtiFrameInfo >>>>>>>>>>>>> ? GetThreadListStackTraces: jvmtiStackInfo >>>>>>>>>>>>> >>>>>>>>>>>>> jvmtiStackInfo contains thread state, and it is ensured it >>>>>>>>>>>>> is the state of the call stack. >>>>>>>>>>>>> If we want to get both call stack and thread state, we need >>>>>>>>>>>>> to suspend target thread, and call both GetStackTrace() and >>>>>>>>>>>>> GetThreadState(). Is it ok? >>>>>>>>>>>>> >>>>>>>>>>>>> I was wondering if JDK-8201641 (parent ticket of this >>>>>>>>>>>>> change) needed them for profiling (dynatrace?) >>>>>>>>>>>>> If it is responsibility of JVMTI agent implementor, I >>>>>>>>>>>>> remove this closure. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 2020/07/03 16:45, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>> >>>>>>>>>>>>>> After some thinking I've concluded that I do not like this >>>>>>>>>>>>>> optimization >>>>>>>>>>>>>> of the GetThreadListStackTraces with >>>>>>>>>>>>>> GetSingleStackTraceClosure. >>>>>>>>>>>>>> >>>>>>>>>>>>>> We may need more opinions on this but these are my points: >>>>>>>>>>>>>> ??- it adds some complexity and ugliness >>>>>>>>>>>>>> ??- a win is doubtful because it has to be a rare case, so >>>>>>>>>>>>>> that total overhead should not be high >>>>>>>>>>>>>> ??- if it is really high for some use cases then it is up >>>>>>>>>>>>>> to the user >>>>>>>>>>>>>> ??? to optimize it with using GetStackTrace instead >>>>>>>>>>>>>> >>>>>>>>>>>>>> In such cases with doubtful overhead I usually prefer the >>>>>>>>>>>>>> simplicity. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Good examples where it makes sense to optimize are checks >>>>>>>>>>>>>> for target thread to be current thread. >>>>>>>>>>>>>> In such cases there is no need to suspend the target >>>>>>>>>>>>>> thread, or use a VMop/HandshakeClosure. >>>>>>>>>>>>>> For instance, please, see the Monitor functions with the >>>>>>>>>>>>>> check: (java_thread == calling_thread). >>>>>>>>>>>>>> Getting information for current thread is frequently used >>>>>>>>>>>>>> case, e.g. to get info at an event point. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 7/2/20 23:29, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>> Hi Dan, David, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I uploaded new webrev. Could you review again? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> OneGetThreadListStackTraces.java in this webrev would >>>>>>>>>>>>>>> wait until thread state is transited to "waiting" with >>>>>>>>>>>>>>> spin wait. >>>>>>>>>>>>>>> CountDownLatch::await call as Dan pointed is fixed in it :) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Diff from webrev.03: >>>>>>>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/c9aeb7001e50 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 2020/07/03 14:15, David Holmes wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>> On 2020/07/03 12:24, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>> On 7/2/20 10:50 PM, David Holmes wrote: >>>>>>>>>>>>>>>>>>> Sorry I'm responding here without seeing latest >>>>>>>>>>>>>>>>>>> webrev but there is enough context I think ... >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>> Hi Dan, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I upload new webrev. Could you review again? >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>>>>>>>>>>>>> ???? L1542: ??? // Get stack trace with handshake >>>>>>>>>>>>>>>>>>>>> ???????? nit - please add a period at the end. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ???? L1591: *stack_info_ptr = op.stack_info(); >>>>>>>>>>>>>>>>>>>>> ???????? The return parameter should not be touched >>>>>>>>>>>>>>>>>>>>> unless the return >>>>>>>>>>>>>>>>>>>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ???? old L1582: ? if (err == JVMTI_ERROR_NONE) { >>>>>>>>>>>>>>>>>>>>> ???????? Please restore this check. The return >>>>>>>>>>>>>>>>>>>>> parameter should not >>>>>>>>>>>>>>>>>>>>> ???????? be touched unless the return code in 'err' >>>>>>>>>>>>>>>>>>>>> == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> But op.stack_info() will return NULL if the error is >>>>>>>>>>>>>>>>>>> not JVMTI_ERROR_NONE. Are you (Dan) concerned about >>>>>>>>>>>>>>>>>>> someone passing in a non-null/initialized out-pointer >>>>>>>>>>>>>>>>>>> that will be reset to NULL if there was an error? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Actually the way we used to test this in POSIX tests >>>>>>>>>>>>>>>>>> is to call >>>>>>>>>>>>>>>>>> an API with known bad parameters and the return >>>>>>>>>>>>>>>>>> parameter ptr >>>>>>>>>>>>>>>>>> set to NULL. If the return parameter ptr was touched >>>>>>>>>>>>>>>>>> when an >>>>>>>>>>>>>>>>>> error should have been detected on an earlier >>>>>>>>>>>>>>>>>> parameter, then >>>>>>>>>>>>>>>>>> the test failed. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ???? L1272: ? if (!jt->is_exiting() && (thread_oop >>>>>>>>>>>>>>>>>>>>> != NULL)) { >>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second >>>>>>>>>>>>>>>>>>>>> expression. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>> ???? old L1532: ? _result = >>>>>>>>>>>>>>>>>>>>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>>>>>>>>>>>>>>>>>> ???????? This deletion of the _result field threw >>>>>>>>>>>>>>>>>>>>> me for a minute and then >>>>>>>>>>>>>>>>>>>>> ???? ? ? I figured out that the field is init to >>>>>>>>>>>>>>>>>>>>> JVMTI_ERROR_THREAD_NOT_ALIVE >>>>>>>>>>>>>>>>>>>>> ???????? in the constructor. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ???? L1553: ? if (!jt->is_exiting() && >>>>>>>>>>>>>>>>>>>>> (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second >>>>>>>>>>>>>>>>>>>>> expression. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ???? L64: ??????? startSignal.countDown(); >>>>>>>>>>>>>>>>>>>>> ???????? I was expecting this to be a call to >>>>>>>>>>>>>>>>>>>>> await() instead of >>>>>>>>>>>>>>>>>>>>> ???????? countDown(). What am I missing here? >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ???????? I think this test might be passing by >>>>>>>>>>>>>>>>>>>>> accident right now, but... >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Main thread (which call JVMTI functions to test) >>>>>>>>>>>>>>>>>>>> should wait until test thread is ready. >>>>>>>>>>>>>>>>>>>> So main thread would wait startSignal, and test >>>>>>>>>>>>>>>>>>>> thread would count down. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> No! >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> The test thread that previously called obj.wait() now >>>>>>>>>>>>>>>>>>> calls latch.await(). >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> The main thread that previously called obj.notify() >>>>>>>>>>>>>>>>>>> now calls latch.countDown(). >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> The main thread continues to spin until it sees the >>>>>>>>>>>>>>>>>>> target is WAITING before proceeding with the test. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> If I add spin wait to wait until transit target thread >>>>>>>>>>>>>>>>> state is WAITING (as following), we don't need to call >>>>>>>>>>>>>>>>> SuspendThread(). >>>>>>>>>>>>>>>>> Which is better? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> The original spin-wait loop checking for? WAITING is >>>>>>>>>>>>>>>> better because it is the only guarantee that the target >>>>>>>>>>>>>>>> thread is blocked where you need it to be. suspending >>>>>>>>>>>>>>>> the thread is racy as you don't know exactly where the >>>>>>>>>>>>>>>> suspend will hit. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>> /* Wait until the thread state transits to "waiting" */ >>>>>>>>>>>>>>>>> while (th.getState() != Thread.State.WAITING) { >>>>>>>>>>>>>>>>> ???? Thread.onSpinWait(); >>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> For simplify, spin wait is prefer to >>>>>>>>>>>>>>>>> OneGetThreadListStackTraces.java in webrev.03. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Here's the flow as I see it: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>> ?? - start worker thread >>>>>>>>>>>>>>>>>> ?? - startSignal.await() >>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>> ?? - startSignal.countDown() >>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>> ?? - stopSignal.await() >>>>>>>>>>>>>>>>>> ???? - worker is now blocked >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>> ?? - checkCallStacks(th) >>>>>>>>>>>>>>>>>> ?? - stopSignal.countDown() >>>>>>>>>>>>>>>>>> ???? - worker is now unblocked >>>>>>>>>>>>>>>>>> ?? - th.join >>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>> ?? - runs off the end of run() >>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>> ?? - run off the end of main() >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ???? L92: ? jthreads = (jthread >>>>>>>>>>>>>>>>>>>>> *)malloc(sizeof(jthread) * num_threads); >>>>>>>>>>>>>>>>>>>>> ???????? You don't check for malloc() failure. >>>>>>>>>>>>>>>>>>>>> ???????? 'jthreads' is allocated but never freed. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ???? L91: ? result = (*jvmti)->SuspendThread(jvmti, >>>>>>>>>>>>>>>>>>>>> thread); >>>>>>>>>>>>>>>>>>>>> ???????? Why are you suspending the thread? >>>>>>>>>>>>>>>>>>>>> GetAllStackTraces() and >>>>>>>>>>>>>>>>>>>>> ???????? GetThreadListStackTraces() do not require >>>>>>>>>>>>>>>>>>>>> the target thread(s) >>>>>>>>>>>>>>>>>>>>> ???????? to be suspend. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ???????? If you decide not to SuspendThread, then >>>>>>>>>>>>>>>>>>>>> you don't need the >>>>>>>>>>>>>>>>>>>>> ???????? AddCapabilities or the ResumeThread calls. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Test thread might not be entered following code >>>>>>>>>>>>>>>>>>>> (stopSignal.await()). We might see deferent call >>>>>>>>>>>>>>>>>>>> stack between GetAllStackTraces() and >>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces(). We cannot control to >>>>>>>>>>>>>>>>>>>> freeze call stack of test thread in Java code. >>>>>>>>>>>>>>>>>>>> (I didn't use SuspendThread() at first, but I saw >>>>>>>>>>>>>>>>>>>> some errors which causes in above.) >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> So we need to call SuspendThread() to ensure we can >>>>>>>>>>>>>>>>>>>> see same call stack. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> If you are checking that the thread is in state >>>>>>>>>>>>>>>>>>> WAITING then it cannot escape from that state and you >>>>>>>>>>>>>>>>>>> can sample the stack multiple times from any API and >>>>>>>>>>>>>>>>>>> get the same result. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I suspect the errors you saw were from the apparent >>>>>>>>>>>>>>>>>>> incorrect use of the CountDownLatch. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> With the flow outlined above, the worker thread should be >>>>>>>>>>>>>>>>>> nicely blocked in stopSignal.await() when stuff is >>>>>>>>>>>>>>>>>> sampled. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could review again? >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Updates look fine - thanks. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> One minor nit: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> 1274 _collector.allocate_and_fill_stacks(1); >>>>>>>>>>>>>>>>>>>>>>> 1275 _collector.set_result(JVMTI_ERROR_NONE); >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> In the other places where you use _collector you >>>>>>>>>>>>>>>>>>>>>>> rely on result being initialized to >>>>>>>>>>>>>>>>>>>>>>> JVMTI_ERROR_NONE, rather than setting it directly >>>>>>>>>>>>>>>>>>>>>>> after allocate_and_fill_stacks(). >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 820 >>>>>>>>>>>>>>>>>>>>>>>>>>>>> assert(SafepointSynchronize::is_at_safepoint() >>>>>>>>>>>>>>>>>>>>>>>>>>>>> || >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 821 >>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 822 current_thread == >>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 823????????? "at safepoint / handshake or >>>>>>>>>>>>>>>>>>>>>>>>>>>>> target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is >>>>>>>>>>>>>>>>>>>>>>>>>>>>> necessary, as even if the target is >>>>>>>>>>>>>>>>>>>>>>>>>>>>> suspended we must still be at a safepoint >>>>>>>>>>>>>>>>>>>>>>>>>>>>> or in a handshake with it. Makes me wonder >>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we used to allow a racy stacktrace >>>>>>>>>>>>>>>>>>>>>>>>>>>>> operation on a suspended thread, assuming >>>>>>>>>>>>>>>>>>>>>>>>>>>>> it would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> This function (JvmtiEnvBase::get_stack_trace()) >>>>>>>>>>>>>>>>>>>>>>>> can be called to get own stack trace. For >>>>>>>>>>>>>>>>>>>>>>>> example, we can call GetStackTrace() for current >>>>>>>>>>>>>>>>>>>>>>>> thread at JVMTI event. >>>>>>>>>>>>>>>>>>>>>>>> So I changed assert as below: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>> ??820?? assert(current_thread == java_thread || >>>>>>>>>>>>>>>>>>>>>>>> ??821 SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>> ??822????????? current_thread == >>>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>> ??823????????? "call by myself / at safepoint / >>>>>>>>>>>>>>>>>>>>>>>> at handshake"); >>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Yep good catch. I hope current tests caught that. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> They would be tested in >>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ >>>>>>>>>>>>>>>>>>>>>> (own call stacks), and getstacktr003 (call stacks >>>>>>>>>>>>>>>>>>>>>> in other thread). >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Speaking of tests ... >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> In the native code I think you need to check the >>>>>>>>>>>>>>>>>>>>>>> success of all JNI methods that can throw >>>>>>>>>>>>>>>>>>>>>>> exceptions - otherwise I believe the tests may >>>>>>>>>>>>>>>>>>>>>>> trigger warnings if -Xcheck:jni is used with >>>>>>>>>>>>>>>>>>>>>>> them. See for example: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I updated testcases to check JNI and JVMTI >>>>>>>>>>>>>>>>>>>>>> function calls. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> In the Java code the target thread: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ?? 45???? public void run() { >>>>>>>>>>>>>>>>>>>>>>> ?? 46?????? try { >>>>>>>>>>>>>>>>>>>>>>> ?? 47?????????? synchronized (lock) { >>>>>>>>>>>>>>>>>>>>>>> ?? 48?????????????? lock.wait(); >>>>>>>>>>>>>>>>>>>>>>> ?? 49 System.out.println("OK"); >>>>>>>>>>>>>>>>>>>>>>> ?? 50?????????? } >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> is potentially susceptible to spurious wakeups. >>>>>>>>>>>>>>>>>>>>>>> Using a CountDownLatch would be robust. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> If so, we can call make_local() in L1272 >>>>>>>>>>>>>>>>>>>>>>>>>> without JavaThread (or we can pass current >>>>>>>>>>>>>>>>>>>>>>>>>> thread to make_local()). Is it right? >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>> 1272 >>>>>>>>>>>>>>>>>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>>>>>>>>>>>>>>>>> thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Sorry I got confused, _calling_thread may not >>>>>>>>>>>>>>>>>>>>>>>>> be the current thread as we could be executing >>>>>>>>>>>>>>>>>>>>>>>>> the handshake in the target thread itself. So >>>>>>>>>>>>>>>>>>>>>>>>> the ResourceMark is correct as-is (implicitly >>>>>>>>>>>>>>>>>>>>>>>>> for current thread). >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> The argument to fill_frames will be used in the >>>>>>>>>>>>>>>>>>>>>>>>> jvmtiStackInfo and passed back to the >>>>>>>>>>>>>>>>>>>>>>>>> _calling_thread, so it must be created via >>>>>>>>>>>>>>>>>>>>>>>>> make_local(_calling_thread, ...) as you >>>>>>>>>>>>>>>>>>>>>>>>> presently have. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you for reviewing! I will update new >>>>>>>>>>>>>>>>>>>>>>>>>>>> webrev tomorrow. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : >>>>>>>>>>>>>>>>>>>>>>>>>>>>> public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 498 class VM_GetAllStackTraces : public >>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 500?? JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of >>>>>>>>>>>>>>>>>>>>>>>>>>>>> another class like that as it may not be on >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the stack. I think >>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector should not >>>>>>>>>>>>>>>>>>>>>>>>>>>>> extend any allocation class, and should >>>>>>>>>>>>>>>>>>>>>>>>>>>>> always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>>>>>>>>>>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>>>>>>>>>>>>>>>>>>> ??? : >>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> class GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? private: >>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I updated webrev for 8242428. Could you >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change migrate to use direct >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> handshake for GetStackTrace() and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() (when >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread_count == 1). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks really good now! I only have a >>>>>>>>>>>>>>>>>>>>>>>>>>>>> few nits below. There is one thing I don't >>>>>>>>>>>>>>>>>>>>>>>>>>>>> like about it but it requires a change to >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the main Handshake logic to address - in >>>>>>>>>>>>>>>>>>>>>>>>>>>>> JvmtiEnv::GetThreadListStackTraces you have >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to create a ThreadsListHandle to convert >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the jthread to a JavaThread, but then the >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Handshake::execute_direct creates another >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ThreadsListHandle internally. That's a >>>>>>>>>>>>>>>>>>>>>>>>>>>>> waste. I will discuss with Robbin and file >>>>>>>>>>>>>>>>>>>>>>>>>>>>> a RFE to have an overload of execute_direct >>>>>>>>>>>>>>>>>>>>>>>>>>>>> that takes an existing TLH. Actually it's >>>>>>>>>>>>>>>>>>>>>>>>>>>>> worse than that because we have another TLH >>>>>>>>>>>>>>>>>>>>>>>>>>>>> in use at the entry point for the JVMTI >>>>>>>>>>>>>>>>>>>>>>>>>>>>> functions, so I think there may be some >>>>>>>>>>>>>>>>>>>>>>>>>>>>> scope for simplifying the use of TLH >>>>>>>>>>>>>>>>>>>>>>>>>>>>> instances - future RFE. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??451 GetStackTraceClosure(JvmtiEnv *env, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> jint start_depth, jint max_count, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??452 jvmtiFrameInfo* frame_buffer, jint* >>>>>>>>>>>>>>>>>>>>>>>>>>>>> count_ptr) >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??454?????? _env(env), >>>>>>>>>>>>>>>>>>>>>>>>>>>>> _start_depth(start_depth), >>>>>>>>>>>>>>>>>>>>>>>>>>>>> _max_count(max_count), >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??455 _frame_buffer(frame_buffer), >>>>>>>>>>>>>>>>>>>>>>>>>>>>> _count_ptr(count_ptr), >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??456 _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Nit: can you do one initializer per line >>>>>>>>>>>>>>>>>>>>>>>>>>>>> please. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : >>>>>>>>>>>>>>>>>>>>>>>>>>>>> public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??498 class VM_GetAllStackTraces : public >>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??500?? JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of >>>>>>>>>>>>>>>>>>>>>>>>>>>>> another class like that as it may not be on >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the stack. I think >>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector should not >>>>>>>>>>>>>>>>>>>>>>>>>>>>> extend any allocation class, and should >>>>>>>>>>>>>>>>>>>>>>>>>>>>> always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 481 MultipleStackTracesCollector(JvmtiEnv >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *env, jint max_frame_count) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??483???? _max_frame_count = max_frame_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??484 _frame_count_total = 0; >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??485???? _head = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??486???? _stack_info = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??487???? _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> As you are touching this can you change it >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to use an initializer list as you did for >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the HandshakeClosure, and please keep one >>>>>>>>>>>>>>>>>>>>>>>>>>>>> item per line. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??820 >>>>>>>>>>>>>>>>>>>>>>>>>>>>> assert(SafepointSynchronize::is_at_safepoint() >>>>>>>>>>>>>>>>>>>>>>>>>>>>> || >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??821 >>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == >>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??823????????? "at safepoint / handshake or >>>>>>>>>>>>>>>>>>>>>>>>>>>>> target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is >>>>>>>>>>>>>>>>>>>>>>>>>>>>> necessary, as even if the target is >>>>>>>>>>>>>>>>>>>>>>>>>>>>> suspended we must still be at a safepoint >>>>>>>>>>>>>>>>>>>>>>>>>>>>> or in a handshake with it. Makes me wonder >>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we used to allow a racy stacktrace >>>>>>>>>>>>>>>>>>>>>>>>>>>>> operation on a suspended thread, assuming >>>>>>>>>>>>>>>>>>>>>>>>>>>>> it would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1269 >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && >>>>>>>>>>>>>>>>>>>>>>>>>>>>> (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 >>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is frustrating that this entire call >>>>>>>>>>>>>>>>>>>>>>>>>>>>> chain started with a jthread reference, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> which we converted to a JavaThread, only to >>>>>>>>>>>>>>>>>>>>>>>>>>>>> eventually need to convert it back to a >>>>>>>>>>>>>>>>>>>>>>>>>>>>> jthread! I think there is some scope for >>>>>>>>>>>>>>>>>>>>>>>>>>>>> simplification here but not as part of this >>>>>>>>>>>>>>>>>>>>>>>>>>>>> change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm still pondering the test logic but >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wanted to send this now. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetThreadListStackTrace (for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces) and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetAllStackTraces (for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetAllStackTraces) have inherited >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetMultipleStackTraces VM operation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which provides the feature to generate >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jvmtiStackInfo. I modified >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetMultipleStackTraces to a normal C++ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class to share with HandshakeClosure for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (GetSingleStackTraceClosure). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I added new testcases which test >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread_count == 1 and with all threads. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jvmti serviceability/jdwp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti vmTestbase/nsk/jdi >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? JBS: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? webrev: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change replace following VM >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operations to direct handshake. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (GetThreadListStackTrace()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTrace() uses direct >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> handshake if thread count == 1. In other >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> case (thread count > 1), it would be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performed as VM operation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Caller of VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (JvmtiEnvThreadState::reset_current_location()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> might be called at safepoint. So I added >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> safepoint check in its caller. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jvmti serviceability/jdwp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti vmTestbase/nsk/jdi >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/ns >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I tested it on submit repo, then it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has execution error >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> due to dependency error. So I think it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> does not occur by this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>>>>> From suenaga at oss.nttdata.com Wed Jul 8 14:00:46 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Wed, 8 Jul 2020 23:00:46 +0900 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: <0fb1d65e-d888-7fcf-548b-58172d507ea8@oracle.com> References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <04e09eb7-9483-8f63-4bb4-01c9e814802e@oracle.com> <238a89a3-06b1-90e4-d5a4-8dc2f1ed12ab@oracle.com> <4c43bc2c-551e-d042-8fa7-b63e755888d1@oracle.com> <8c387550-a06e-6320-f453-0e898486befb@oss.nttdata.com> <19cfa2a7-f441-425d-5623-1c11c6a870b6@oss.nttdata.com> <237e8d26-d185-1008-6e07-5b9f36314534@oracle.com> <0fb1d65e-d888-7fcf-548b-58172d507ea8@oracle.com> Message-ID: Thanks David! Yasumasa On 2020/07/08 21:29, David Holmes wrote: > On 8/07/2020 6:04 pm, Yasumasa Suenaga wrote: >> Hi David, >> >> On 2020/07/08 15:27, David Holmes wrote: >>> Hi Yasumasa, >>> >>> On 7/07/2020 6:54 pm, Yasumasa Suenaga wrote: >>>> Hi David, Serguei, >>>> >>>> Serguei, thank you for replying even though you are on vacaiton! >>>> >>>> I uploaded new webrev: >>>> >>>> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.07/ >>>> ?? Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/77243b1dcbfe >>>> >>>> c'tor of GetSingleStackTraceClosure has jthread argument in this webrev. >>>> Also it does not contain testcase for GetThreadListStackTraces with all threads, and OneGetThreadListStackTraces would test main thread only. >>> >>> All those changes are fine in principle for me. One nit/suggestion: >>> >>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>> >>> ??544?? jthread _java_thread; >>> >>> elsewhere "java_thread" refers to a JavaThread, so to avoid confusion may I suggest this member be named _jthread. >> >> I uploaded new webrev: >> >> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ >> ?? Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/ca6263dbdc87 > > That looks fine. > > Thanks, > David > ----- > >> >>> I'm going to be away for the next couple of days - sorry - but will try to check email on this if I can. >> >> Thanks! >> >> >> Yasumasa >> >> >>> Thanks, >>> David >>> ----- >>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2020/07/07 15:13, David Holmes wrote: >>>>> On 7/07/2020 2:57 pm, Yasumasa Suenaga wrote: >>>>>> Hi David, >>>>>> >>>>>> On 2020/07/07 11:31, David Holmes wrote: >>>>>>> Hi Yasumasa, >>>>>>> >>>>>>> Hard to keep up with the changes - especially without incremental webrevs. >>>>>> >>>>>> Sorry, I will upload diff from previous webrev in the next. >>>>>> >>>>>> >>>>>>> If GetSingleStackTraceClosure also took the jthread as a constructor arg, then you wouldn't need to recreate a JNI local handle when calling _collector.fill_frames. It's a small simplification and not essential at this stage. >>>>>> >>>>>> I think we should get jthread from an argument of do_thread() because do_thread() would pass the thread which are stopped certainly. >>>>>> It might be simplification if we pass _calling_thread to MultipleStackTracesCollector. `jthread` is only needed to store jvmtiStackInfo.thread . What do you think? >>>>> >>>>> I'm not quite sure what you mean. >>>>> >>>>> I think there is a bit of a design wart with direct handshakes in that do_thread takes the target JavaThread as an argument. That's useful in a case where you want a HandshakeClosure that can be applied to multiple threads, but that's not typically what is needed with direct handshakes - there is only a single target. With a single-target HandshakeClosure you can capture all the "target" information for the operation in the closure instance. So if the actual do_thread operation wants the jthread corresponding to the target thread then we can store that in the closure rather than recomputing it (you could assert it is the same but that seems overkill to me). >>>>> >>>>>> >>>>>>> For the test ... I don't see how Java_GetThreadListStackTraces_checkCallStacks is a valid test. It gets the stacks of all live threads, then uses that information to use GetThreadListStackTraces to get the stack for the same set of threads through a different API. It then compares the two sets of stacks for each thread expecting them to be the same, but that need only be the case for the main thread. Other threads could potentially have a different stack (e.g. if this test is run with JFR enabled there will be additional threads found.) Further I would have expected that there already exist tests that check that, for a given thread (which may be suspended or known to be blocked) the same stack is found through the two different APIs. >>>>>> >>>>>> vmTestbase/nsk/jvmti/unit/GetAllStackTraces/getallstktr001 would check all of threads via GetThreadListStackTraces() and GetAllStackTraces(), so we might be able to remove GetThreadListStackTraces.java from this webrev. >>>>> >>>>> Yes. The existing test only examines a set of test threads that are all blocked on a raw monitor. You do not need to duplicate that test. >>>>> >>>>>> OTOH we don't have testcase for GetThreadListStackTraces() with thread_count == 1, so we need to testcase for it (it is OneGetThreadListStackTraces.java) It would check whether the state of target thread is "waiting" before JNI call to call GetThreadListStackTraces(), >>>>> >>>>> Yes we need to test the special cases introduced by your changes - totally agree - and OneGetThreadListStackTraces.java is a good test for that. >>>>> >>>>>> and also I expect it would not be run with JFR. (it is not described @run) >>>>> >>>>> The arguments to run with JFR (or a bunch of other things) can be passed to the jtreg test harness to be applied to all tests. >>>>> >>>>>> Of course we can check GetThreadListStackTraces() with main thread, but it is not the test for direct handshake for other thread. >>>>> >>>>> Right - that test already exists as per the above. >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>> On 6/07/2020 11:29 pm, Yasumasa Suenaga wrote: >>>>>>>> Hi Serguei, >>>>>>>> >>>>>>>> Thanks for your comment! >>>>>>>> >>>>>>>> I think C++ is more simple to implement the test agent as you said. >>>>>>>> So I implement it in C++ in new webrev. Could you review again? >>>>>>>> >>>>>>>> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.06/ >>>>>>>> >>>>>>>> Also I refactored libGetThreadListStackTraces.cpp, and I've kept exception check after IsSameObject(). >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2020/07/06 16:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>> Hi Yasumasa, >>>>>>>>> >>>>>>>>> Thank you for the update. >>>>>>>>> I think, a pending exception after IsSameObject needs to be checked. >>>>>>>>> >>>>>>>>> The checkStackInfo() needs one more refactoring as I've already suggested. >>>>>>>>> The body of the loop at L68-L78 should be converted to a function check_frame_info. >>>>>>>>> The si1->frame_buffer[i] and si2->frame_buffer[i] will be passed as fi1 and fi2. >>>>>>>>> The index can be passed as well. >>>>>>>>> I'm still suggesting to simplify the local exception_msg to something shorter like err_msg or exc_msg. >>>>>>>>> >>>>>>>>> I'm not sure using fatal is right here: >>>>>>>>> >>>>>>>>> This fragment looks strange: >>>>>>>>> >>>>>>>>> ? 152???? if ((*env)->IsSameObject(env, stack_info[i].thread, thread)) { >>>>>>>>> ? 153?????? target_info = &stack_info[i]; >>>>>>>>> ? 154?????? break; >>>>>>>>> ? 155???? } else if ((*env)->ExceptionOccurred(env)) { >>>>>>>>> ? 156?????? (*env)->ExceptionDescribe(env); >>>>>>>>> ? 157?????? (*env)->FatalError(env, __FILE__); >>>>>>>>> ? 158???? } >>>>>>>>> >>>>>>>>> I expected it to be: >>>>>>>>> >>>>>>>>> ??? jboolean same = (*env)->IsSameObject(env, stack_info[i].thread, thread); >>>>>>>>> ??? if ((*env)->ExceptionOccurred(env)) { >>>>>>>>> ????? (*env)->ExceptionDescribe(env); >>>>>>>>> ????? (*env)->FatalError(env, __FILE__); >>>>>>>>> ??? } >>>>>>>>> ??? if (same) { >>>>>>>>> ????? target_info = &stack_info[i]; >>>>>>>>> ????? break; >>>>>>>>> ??? } >>>>>>>>> >>>>>>>>> Would it better to port this agent to C++ to simplify this code nicer? >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Serguei >>>>>>>>> >>>>>>>>> >>>>>>>>> On 7/5/20 06:13, Yasumasa Suenaga wrote: >>>>>>>>>> Hi Serguei, >>>>>>>>>> >>>>>>>>>> Thanks for your comment! >>>>>>>>>> I refactored testcase. Could you review again? >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.05/ >>>>>>>>>> >>>>>>>>>> It would check Java exception after IsSameObject() call. Does it need? >>>>>>>>>> Any exceptions are not described in JNI document[1], and JNI implementation (jni_IsSameObject()) does not seem to throw it. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> [1] https://docs.oracle.com/en/java/javase/14/docs/specs/jni/functions.html#issameobject >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2020/07/05 14:46, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Okay, thanks. >>>>>>>>>>> Then I'm okay to keep the GetSingleStackTraceClosure. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c.html >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c.html >>>>>>>>>>> >>>>>>>>>>> I'm not sure the function 'is_same_thread() is needed. >>>>>>>>>>> Why do not use the JNI IsSameObject instead? >>>>>>>>>>> >>>>>>>>>>> It seems to be a typo at L132 and L137. >>>>>>>>>>> You, probably. did not want to print the same information for stack_info_1[i].frame_buffer[j].XXX twice. >>>>>>>>>>> >>>>>>>>>>> The code at lines 112-142 is not readable. >>>>>>>>>>> I'd suggest to make a couple of refactoring steps. >>>>>>>>>>> >>>>>>>>>>> First step to simplify this a little bit would be with some renaming and getting rid of indexes: >>>>>>>>>>> >>>>>>>>>>> ?? 71 char err_msg[EXCEPTION_MSG_LEN] = {0}; >>>>>>>>>>> ??... >>>>>>>>>>> ??112?? /* Iterate all jvmtiStackInfo to check */ >>>>>>>>>>> ??113?? for (i = 0; i < num_threads, *exception_msg != '\0'; i++) { >>>>>>>>>>> ????????? jvmtiStackInfo *si1 = stack_info_1[i]; >>>>>>>>>>> ????????? jvmtiStackInfo *si2 = stack_info_2[i]; >>>>>>>>>>> ??114???? if (!IsSameObject(env, si1.thread, si2.thread)) { /* jvmtiStackInfo::thread */ >>>>>>>>>>> ??115?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>> ??116??????????????? "thread[%d] is different: stack_info_1 = %p, stack_info_2 = %p", >>>>>>>>>>> ??117??????????????? i, sinfo1.thread, sinfo2.thread); >>>>>>>>>>> ??118???? } else if (si1.state != si2.state) { /* jvmtiStackInfo::state */ >>>>>>>>>>> ??119?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>> ??120??????????????? "state[%d] is different: stack_info_1 = %d, stack_info_2 = %d", >>>>>>>>>>> ??121??????????????? i, si1.state, si2.state); >>>>>>>>>>> ??122???? } else if (si1.frame_count != si2.frame_count) { /* jvmtiStackInfo::frame_count */ >>>>>>>>>>> ??123?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>> ??124??????????????? "frame_count[%d] is different: stack_info_1 = %d, stack_info_2 = %d", >>>>>>>>>>> ??125??????????????? i, si1.frame_count, si2.frame_count); >>>>>>>>>>> ??126???? } else { >>>>>>>>>>> ??127?????? /* Iterate all jvmtiFrameInfo to check */ >>>>>>>>>>> ??128?????? for (j = 0; j < si1.frame_count; j++) { >>>>>>>>>>> ??129???????? if (si1.frame_buffer[j].method != si1.frame_buffer[j].method) { /* jvmtiFrameInfo::method */ >>>>>>>>>>> ??130?????????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>> ??131??????????????????? "thread [%d] frame_buffer[%d].method is different: stack_info_1 = %lx, stack_info_2 = %lx", >>>>>>>>>>> ??132??????????????????? i, j, si1.frame_buffer[j].method, si2.frame_buffer[j].method); >>>>>>>>>>> ??133?????????? break; >>>>>>>>>>> ??134???????? } else if (si1.frame_buffer[j].location != si1.frame_buffer[j].location) { /* jvmtiFrameInfo::location */ >>>>>>>>>>> ??135?????????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>> ??136??????????????????? "thread [%d] frame_buffer[%d].location is different: stack_info_1 = %ld, stack_info_2 = %ld", >>>>>>>>>>> ??137??????????????????? i, j, si1.frame_buffer[j].location, si2.frame_buffer[j].location); >>>>>>>>>>> ??138?????????? break; >>>>>>>>>>> ??139???????? } >>>>>>>>>>> ??140?????? } >>>>>>>>>>> ??141???? } >>>>>>>>>>> ??142?? } >>>>>>>>>>> >>>>>>>>>>> Another step would be to create functions that implement a body of each loop. >>>>>>>>>>> You can use the same techniques to simplify similar place (L127-L138) in the libOneGetThreadListStackTraces.c. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Serguei >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 7/3/20 15:55, Yasumasa Suenaga wrote: >>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>> >>>>>>>>>>>> I'm not an Oracle employee, so I cannot know real request(s) from your customers. >>>>>>>>>>>> However JDK-8201641 says Dynatrace has requested this enhancement. >>>>>>>>>>>> >>>>>>>>>>>> BTW I haven't heared any request from my customers about this. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 2020/07/04 4:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>> >>>>>>>>>>>>> This difference is not that big to care about. >>>>>>>>>>>>> I feel this is really rare case and so, does not worth these complications. >>>>>>>>>>>>> Do we have a real request from customers to optimize it? >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Serguei >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 7/3/20 01:16, Yasumasa Suenaga wrote: >>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Generally I agree with you, but I have concern about the difference of the result of GetStackTrace() and GetThreadListStackTraces(). >>>>>>>>>>>>>> >>>>>>>>>>>>>> ? GetStackTrace: jvmtiFrameInfo >>>>>>>>>>>>>> ? GetThreadListStackTraces: jvmtiStackInfo >>>>>>>>>>>>>> >>>>>>>>>>>>>> jvmtiStackInfo contains thread state, and it is ensured it is the state of the call stack. >>>>>>>>>>>>>> If we want to get both call stack and thread state, we need to suspend target thread, and call both GetStackTrace() and GetThreadState(). Is it ok? >>>>>>>>>>>>>> >>>>>>>>>>>>>> I was wondering if JDK-8201641 (parent ticket of this change) needed them for profiling (dynatrace?) >>>>>>>>>>>>>> If it is responsibility of JVMTI agent implementor, I remove this closure. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 2020/07/03 16:45, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> After some thinking I've concluded that I do not like this optimization >>>>>>>>>>>>>>> of the GetThreadListStackTraces with GetSingleStackTraceClosure. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> We may need more opinions on this but these are my points: >>>>>>>>>>>>>>> ??- it adds some complexity and ugliness >>>>>>>>>>>>>>> ??- a win is doubtful because it has to be a rare case, so that total overhead should not be high >>>>>>>>>>>>>>> ??- if it is really high for some use cases then it is up to the user >>>>>>>>>>>>>>> ??? to optimize it with using GetStackTrace instead >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> In such cases with doubtful overhead I usually prefer the simplicity. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Good examples where it makes sense to optimize are checks for target thread to be current thread. >>>>>>>>>>>>>>> In such cases there is no need to suspend the target thread, or use a VMop/HandshakeClosure. >>>>>>>>>>>>>>> For instance, please, see the Monitor functions with the check: (java_thread == calling_thread). >>>>>>>>>>>>>>> Getting information for current thread is frequently used case, e.g. to get info at an event point. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 7/2/20 23:29, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>> Hi Dan, David, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I uploaded new webrev. Could you review again? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/ >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> OneGetThreadListStackTraces.java in this webrev would wait until thread state is transited to "waiting" with spin wait. >>>>>>>>>>>>>>>> CountDownLatch::await call as Dan pointed is fixed in it :) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Diff from webrev.03: >>>>>>>>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/c9aeb7001e50 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 2020/07/03 14:15, David Holmes wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>> On 2020/07/03 12:24, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>>> On 7/2/20 10:50 PM, David Holmes wrote: >>>>>>>>>>>>>>>>>>>> Sorry I'm responding here without seeing latest webrev but there is enough context I think ... >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>> Hi Dan, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>>>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I upload new webrev. Could you review again? >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>>>>>>>>>>>>>> ???? L1542: ??? // Get stack trace with handshake >>>>>>>>>>>>>>>>>>>>>> ???????? nit - please add a period at the end. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> ???? L1591: *stack_info_ptr = op.stack_info(); >>>>>>>>>>>>>>>>>>>>>> ???????? The return parameter should not be touched unless the return >>>>>>>>>>>>>>>>>>>>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> ???? old L1582: ? if (err == JVMTI_ERROR_NONE) { >>>>>>>>>>>>>>>>>>>>>> ???????? Please restore this check. The return parameter should not >>>>>>>>>>>>>>>>>>>>>> ???????? be touched unless the return code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> But op.stack_info() will return NULL if the error is not JVMTI_ERROR_NONE. Are you (Dan) concerned about someone passing in a non-null/initialized out-pointer that will be reset to NULL if there was an error? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Actually the way we used to test this in POSIX tests is to call >>>>>>>>>>>>>>>>>>> an API with known bad parameters and the return parameter ptr >>>>>>>>>>>>>>>>>>> set to NULL. If the return parameter ptr was touched when an >>>>>>>>>>>>>>>>>>> error should have been detected on an earlier parameter, then >>>>>>>>>>>>>>>>>>> the test failed. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> ???? L1272: ? if (!jt->is_exiting() && (thread_oop != NULL)) { >>>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>> ???? old L1532: ? _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>>>>>>>>>>>>>>>>>>> ???????? This deletion of the _result field threw me for a minute and then >>>>>>>>>>>>>>>>>>>>>> ???? ? ? I figured out that the field is init to JVMTI_ERROR_THREAD_NOT_ALIVE >>>>>>>>>>>>>>>>>>>>>> ???????? in the constructor. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> ???? L1553: ? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>>> ???? L64: ??????? startSignal.countDown(); >>>>>>>>>>>>>>>>>>>>>> ???????? I was expecting this to be a call to await() instead of >>>>>>>>>>>>>>>>>>>>>> ???????? countDown(). What am I missing here? >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> ???????? I think this test might be passing by accident right now, but... >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Main thread (which call JVMTI functions to test) should wait until test thread is ready. >>>>>>>>>>>>>>>>>>>>> So main thread would wait startSignal, and test thread would count down. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> No! >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> The test thread that previously called obj.wait() now calls latch.await(). >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> The main thread that previously called obj.notify() now calls latch.countDown(). >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> The main thread continues to spin until it sees the target is WAITING before proceeding with the test. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> If I add spin wait to wait until transit target thread state is WAITING (as following), we don't need to call SuspendThread(). >>>>>>>>>>>>>>>>>> Which is better? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> The original spin-wait loop checking for? WAITING is better because it is the only guarantee that the target thread is blocked where you need it to be. suspending the thread is racy as you don't know exactly where the suspend will hit. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>> /* Wait until the thread state transits to "waiting" */ >>>>>>>>>>>>>>>>>> while (th.getState() != Thread.State.WAITING) { >>>>>>>>>>>>>>>>>> ???? Thread.onSpinWait(); >>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> For simplify, spin wait is prefer to OneGetThreadListStackTraces.java in webrev.03. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Here's the flow as I see it: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>> ?? - start worker thread >>>>>>>>>>>>>>>>>>> ?? - startSignal.await() >>>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>>> ?? - startSignal.countDown() >>>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>>> ?? - stopSignal.await() >>>>>>>>>>>>>>>>>>> ???? - worker is now blocked >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>> ?? - checkCallStacks(th) >>>>>>>>>>>>>>>>>>> ?? - stopSignal.countDown() >>>>>>>>>>>>>>>>>>> ???? - worker is now unblocked >>>>>>>>>>>>>>>>>>> ?? - th.join >>>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>>> ?? - runs off the end of run() >>>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>> ?? - run off the end of main() >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>>> ???? L92: ? jthreads = (jthread *)malloc(sizeof(jthread) * num_threads); >>>>>>>>>>>>>>>>>>>>>> ???????? You don't check for malloc() failure. >>>>>>>>>>>>>>>>>>>>>> ???????? 'jthreads' is allocated but never freed. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>>> ???? L91: ? result = (*jvmti)->SuspendThread(jvmti, thread); >>>>>>>>>>>>>>>>>>>>>> ???????? Why are you suspending the thread? GetAllStackTraces() and >>>>>>>>>>>>>>>>>>>>>> ???????? GetThreadListStackTraces() do not require the target thread(s) >>>>>>>>>>>>>>>>>>>>>> ???????? to be suspend. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> ???????? If you decide not to SuspendThread, then you don't need the >>>>>>>>>>>>>>>>>>>>>> ???????? AddCapabilities or the ResumeThread calls. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Test thread might not be entered following code (stopSignal.await()). We might see deferent call stack between GetAllStackTraces() and GetThreadListStackTraces(). We cannot control to freeze call stack of test thread in Java code. >>>>>>>>>>>>>>>>>>>>> (I didn't use SuspendThread() at first, but I saw some errors which causes in above.) >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> So we need to call SuspendThread() to ensure we can see same call stack. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> If you are checking that the thread is in state WAITING then it cannot escape from that state and you can sample the stack multiple times from any API and get the same result. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I suspect the errors you saw were from the apparent incorrect use of the CountDownLatch. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> With the flow outlined above, the worker thread should be >>>>>>>>>>>>>>>>>>> nicely blocked in stopSignal.await() when stuff is sampled. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could review again? >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Updates look fine - thanks. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> One minor nit: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> 1274 _collector.allocate_and_fill_stacks(1); >>>>>>>>>>>>>>>>>>>>>>>> 1275 _collector.set_result(JVMTI_ERROR_NONE); >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> In the other places where you use _collector you rely on result being initialized to JVMTI_ERROR_NONE, rather than setting it directly after allocate_and_fill_stacks(). >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 823????????? "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> This function (JvmtiEnvBase::get_stack_trace()) can be called to get own stack trace. For example, we can call GetStackTrace() for current thread at JVMTI event. >>>>>>>>>>>>>>>>>>>>>>>>> So I changed assert as below: >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>> ??820?? assert(current_thread == java_thread || >>>>>>>>>>>>>>>>>>>>>>>>> ??821 SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>> ??822????????? current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>> ??823????????? "call by myself / at safepoint / at handshake"); >>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Yep good catch. I hope current tests caught that. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> They would be tested in vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ (own call stacks), and getstacktr003 (call stacks in other thread). >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Speaking of tests ... >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> In the native code I think you need to check the success of all JNI methods that can throw exceptions - otherwise I believe the tests may trigger warnings if -Xcheck:jni is used with them. See for example: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I updated testcases to check JNI and JVMTI function calls. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> In the Java code the target thread: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> ?? 45???? public void run() { >>>>>>>>>>>>>>>>>>>>>>>> ?? 46?????? try { >>>>>>>>>>>>>>>>>>>>>>>> ?? 47?????????? synchronized (lock) { >>>>>>>>>>>>>>>>>>>>>>>> ?? 48?????????????? lock.wait(); >>>>>>>>>>>>>>>>>>>>>>>> ?? 49 System.out.println("OK"); >>>>>>>>>>>>>>>>>>>>>>>> ?? 50?????????? } >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> is potentially susceptible to spurious wakeups. Using a CountDownLatch would be robust. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> If so, we can call make_local() in L1272 without JavaThread (or we can pass current thread to make_local()). Is it right? >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Sorry I got confused, _calling_thread may not be the current thread as we could be executing the handshake in the target thread itself. So the ResourceMark is correct as-is (implicitly for current thread). >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> The argument to fill_frames will be used in the jvmtiStackInfo and passed back to the _calling_thread, so it must be created via make_local(_calling_thread, ...) as you presently have. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you for reviewing! I will update new webrev tomorrow. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 500?? JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??? : >>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> class GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I updated webrev for 8242428. Could you review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change migrate to use direct handshake for GetStackTrace() and GetThreadListStackTraces() (when thread_count == 1). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks really good now! I only have a few nits below. There is one thing I don't like about it but it requires a change to the main Handshake logic to address - in JvmtiEnv::GetThreadListStackTraces you have to create a ThreadsListHandle to convert the jthread to a JavaThread, but then the Handshake::execute_direct creates another ThreadsListHandle internally. That's a waste. I will discuss with Robbin and file a RFE to have an overload of execute_direct that takes an existing TLH. Actually it's worse than that because we have another TLH in use at the entry point for the JVMTI functions, so I think there may be some scope for simplifying the use of TLH instances - future RFE. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??451 GetStackTraceClosure(JvmtiEnv *env, jint start_depth, jint max_count, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??452 jvmtiFrameInfo* frame_buffer, jint* count_ptr) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??454?????? _env(env), _start_depth(start_depth), _max_count(max_count), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??455 _frame_buffer(frame_buffer), _count_ptr(count_ptr), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??456 _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Nit: can you do one initializer per line please. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??500?? JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 481 MultipleStackTracesCollector(JvmtiEnv *env, jint max_frame_count) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??483???? _max_frame_count = max_frame_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??484 _frame_count_total = 0; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??485???? _head = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??486???? _stack_info = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??487???? _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As you are touching this can you change it to use an initializer list as you did for the HandshakeClosure, and please keep one item per line. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??823????????? "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1269 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is frustrating that this entire call chain started with a jthread reference, which we converted to a JavaThread, only to eventually need to convert it back to a jthread! I think there is some scope for simplification here but not as part of this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271???? ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm still pondering the test logic but wanted to send this now. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetThreadListStackTrace (for GetThreadListStackTraces) and VM_GetAllStackTraces (for GetAllStackTraces) have inherited VM_GetMultipleStackTraces VM operation which provides the feature to generate jvmtiStackInfo. I modified VM_GetMultipleStackTraces to a normal C++ class to share with HandshakeClosure for GetThreadListStackTraces (GetSingleStackTraceClosure). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I added new testcases which test GetThreadListStackTraces() with thread_count == 1 and with all threads. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/nsk/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change replace following VM operations to direct handshake. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces (GetThreadListStackTrace()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTrace() uses direct handshake if thread count == 1. In other case (thread count > 1), it would be performed as VM operation (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Caller of VM_GetCurrentLocation (JvmtiEnvThreadState::reset_current_location()) might be called at safepoint. So I added safepoint check in its caller. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/ns >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I tested it on submit repo, then it has execution error (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) due to dependency error. So I think it does not occur by this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> From daniel.daugherty at oracle.com Wed Jul 8 16:42:42 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 8 Jul 2020 12:42:42 -0400 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <7dc1903b-6295-3864-6301-4c02aacdb971@oss.nttdata.com> <04e09eb7-9483-8f63-4bb4-01c9e814802e@oracle.com> <238a89a3-06b1-90e4-d5a4-8dc2f1ed12ab@oracle.com> <4c43bc2c-551e-d042-8fa7-b63e755888d1@oracle.com> <8c387550-a06e-6320-f453-0e898486befb@oss.nttdata.com> <19cfa2a7-f441-425d-5623-1c11c6a870b6@oss.nttdata.com> <237e8d26-d185-1008-6e07-5b9f36314534@oracle.com> Message-ID: > http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ src/hotspot/share/prims/jvmtiEnv.cpp ??? No comments. src/hotspot/share/prims/jvmtiEnvBase.cpp ??? L1159: ? Thread *current_thread = Thread::current(); ??????? Please add "#ifdef ASSERT" above and "#endif" below since ??????? current_thread is only used for the assert() in this function. src/hotspot/share/prims/jvmtiEnvBase.hpp ??? L549: ???????????????????????????? jthread java_thread, jint max_frame_count) ??? L552: ????? _jthread(java_thread), ??????? Please: s/java_thread/thread/ on both lines. src/hotspot/share/runtime/vmOperations.hpp ??? No comments. test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java ??? No comments. test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp ??? L27: #include ??????? This include is out of order; should be first in the list. ??? This file doesn't compile on my MBP13: ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:49:14: error: format specifies type 'unsigned long' but the argument has type 'jmethodID' (aka '_jmethodID *') [-Werror,-Wformat] ???????????? fi1->method, fi2->method); ???????????? ^~~~~~~~~~~ ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:49:27: error: format specifies type 'unsigned long' but the argument has type 'jmethodID' (aka '_jmethodID *') [-Werror,-Wformat] ???????????? fi1->method, fi2->method); ????????????????????????? ^~~~~~~~~~~ ??? 2 errors generated. ??? This change made it compile on my MBP13, but that may break it on ??? other platforms: ??? $ hg diff ??? diff -r 560847c69fbe test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp ??? --- a/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp Wed Jul 08 12:13:32 2020 -0400 ??? +++ b/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp Wed Jul 08 12:40:42 2020 -0400 ??? @@ -46,7 +46,7 @@ ? ?? ? if (fi1->method != fi2->method) { /* jvmtiFrameInfo::method */ ? ?? ??? snprintf(err_msg, sizeof(err_msg), ? ? ? ??????????? "method is different: fi1 = %lx, fi2 = %lx", ??? -???????????? fi1->method, fi2->method); ??? +???????????? (unsigned long) fi1->method, (unsigned long) fi2->method); ? ?? ??? env->FatalError(err_msg); ? ?? ? } else if (fi1->location != fi2->location) { /* jvmtiFrameInfo::location */ ? ?? ??? snprintf(err_msg, sizeof(err_msg), ??? I'm not sure of the right platform independent way to output ??? the 'method' field. Dan On 7/8/20 4:04 AM, Yasumasa Suenaga wrote: > Hi David, > > On 2020/07/08 15:27, David Holmes wrote: >> Hi Yasumasa, >> >> On 7/07/2020 6:54 pm, Yasumasa Suenaga wrote: >>> Hi David, Serguei, >>> >>> Serguei, thank you for replying even though you are on vacaiton! >>> >>> I uploaded new webrev: >>> >>> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.07/ >>> ?? Diff from previous webrev: >>> http://hg.openjdk.java.net/jdk/submit/rev/77243b1dcbfe >>> >>> c'tor of GetSingleStackTraceClosure has jthread argument in this >>> webrev. >>> Also it does not contain testcase for GetThreadListStackTraces with >>> all threads, and OneGetThreadListStackTraces would test main thread >>> only. >> >> All those changes are fine in principle for me. One nit/suggestion: >> >> src/hotspot/share/prims/jvmtiEnvBase.hpp >> >> ??544?? jthread _java_thread; >> >> elsewhere "java_thread" refers to a JavaThread, so to avoid confusion >> may I suggest this member be named _jthread. > > I uploaded new webrev: > > ? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ > ? Diff from previous webrev: > http://hg.openjdk.java.net/jdk/submit/rev/ca6263dbdc87 > > >> I'm going to be away for the next couple of days - sorry - but will >> try to check email on this if I can. > > Thanks! > > > Yasumasa > > >> Thanks, >> David >> ----- >> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2020/07/07 15:13, David Holmes wrote: >>>> On 7/07/2020 2:57 pm, Yasumasa Suenaga wrote: >>>>> Hi David, >>>>> >>>>> On 2020/07/07 11:31, David Holmes wrote: >>>>>> Hi Yasumasa, >>>>>> >>>>>> Hard to keep up with the changes - especially without incremental >>>>>> webrevs. >>>>> >>>>> Sorry, I will upload diff from previous webrev in the next. >>>>> >>>>> >>>>>> If GetSingleStackTraceClosure also took the jthread as a >>>>>> constructor arg, then you wouldn't need to recreate a JNI local >>>>>> handle when calling _collector.fill_frames. It's a small >>>>>> simplification and not essential at this stage. >>>>> >>>>> I think we should get jthread from an argument of do_thread() >>>>> because do_thread() would pass the thread which are stopped >>>>> certainly. >>>>> It might be simplification if we pass _calling_thread to >>>>> MultipleStackTracesCollector. `jthread` is only needed to store >>>>> jvmtiStackInfo.thread . What do you think? >>>> >>>> I'm not quite sure what you mean. >>>> >>>> I think there is a bit of a design wart with direct handshakes in >>>> that do_thread takes the target JavaThread as an argument. That's >>>> useful in a case where you want a HandshakeClosure that can be >>>> applied to multiple threads, but that's not typically what is >>>> needed with direct handshakes - there is only a single target. With >>>> a single-target HandshakeClosure you can capture all the "target" >>>> information for the operation in the closure instance. So if the >>>> actual do_thread operation wants the jthread corresponding to the >>>> target thread then we can store that in the closure rather than >>>> recomputing it (you could assert it is the same but that seems >>>> overkill to me). >>>> >>>>> >>>>>> For the test ... I don't see how >>>>>> Java_GetThreadListStackTraces_checkCallStacks is a valid test. It >>>>>> gets the stacks of all live threads, then uses that information >>>>>> to use GetThreadListStackTraces to get the stack for the same set >>>>>> of threads through a different API. It then compares the two sets >>>>>> of stacks for each thread expecting them to be the same, but that >>>>>> need only be the case for the main thread. Other threads could >>>>>> potentially have a different stack (e.g. if this test is run with >>>>>> JFR enabled there will be additional threads found.) Further I >>>>>> would have expected that there already exist tests that check >>>>>> that, for a given thread (which may be suspended or known to be >>>>>> blocked) the same stack is found through the two different APIs. >>>>> >>>>> vmTestbase/nsk/jvmti/unit/GetAllStackTraces/getallstktr001 would >>>>> check all of threads via GetThreadListStackTraces() and >>>>> GetAllStackTraces(), so we might be able to remove >>>>> GetThreadListStackTraces.java from this webrev. >>>> >>>> Yes. The existing test only examines a set of test threads that are >>>> all blocked on a raw monitor. You do not need to duplicate that test. >>>> >>>>> OTOH we don't have testcase for GetThreadListStackTraces() with >>>>> thread_count == 1, so we need to testcase for it (it is >>>>> OneGetThreadListStackTraces.java) It would check whether the state >>>>> of target thread is "waiting" before JNI call to call >>>>> GetThreadListStackTraces(), >>>> >>>> Yes we need to test the special cases introduced by your changes - >>>> totally agree - and OneGetThreadListStackTraces.java is a good test >>>> for that. >>>> >>>>> and also I expect it would not be run with JFR. (it is not >>>>> described @run) >>>> >>>> The arguments to run with JFR (or a bunch of other things) can be >>>> passed to the jtreg test harness to be applied to all tests. >>>> >>>>> Of course we can check GetThreadListStackTraces() with main >>>>> thread, but it is not the test for direct handshake for other thread. >>>> >>>> Right - that test already exists as per the above. >>>> >>>> Thanks, >>>> David >>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>> >>>>>> On 6/07/2020 11:29 pm, Yasumasa Suenaga wrote: >>>>>>> Hi Serguei, >>>>>>> >>>>>>> Thanks for your comment! >>>>>>> >>>>>>> I think C++ is more simple to implement the test agent as you said. >>>>>>> So I implement it in C++ in new webrev. Could you review again? >>>>>>> >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.06/ >>>>>>> >>>>>>> Also I refactored libGetThreadListStackTraces.cpp, and I've kept >>>>>>> exception check after IsSameObject(). >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2020/07/06 16:32, serguei.spitsyn at oracle.com wrote: >>>>>>>> Hi Yasumasa, >>>>>>>> >>>>>>>> Thank you for the update. >>>>>>>> I think, a pending exception after IsSameObject needs to be >>>>>>>> checked. >>>>>>>> >>>>>>>> The checkStackInfo() needs one more refactoring as I've already >>>>>>>> suggested. >>>>>>>> The body of the loop at L68-L78 should be converted to a >>>>>>>> function check_frame_info. >>>>>>>> The si1->frame_buffer[i] and si2->frame_buffer[i] will be >>>>>>>> passed as fi1 and fi2. >>>>>>>> The index can be passed as well. >>>>>>>> I'm still suggesting to simplify the local exception_msg to >>>>>>>> something shorter like err_msg or exc_msg. >>>>>>>> >>>>>>>> I'm not sure using fatal is right here: >>>>>>>> >>>>>>>> This fragment looks strange: >>>>>>>> >>>>>>>> ? 152???? if ((*env)->IsSameObject(env, stack_info[i].thread, >>>>>>>> thread)) { >>>>>>>> ? 153?????? target_info = &stack_info[i]; >>>>>>>> ? 154?????? break; >>>>>>>> ? 155???? } else if ((*env)->ExceptionOccurred(env)) { >>>>>>>> ? 156?????? (*env)->ExceptionDescribe(env); >>>>>>>> ? 157?????? (*env)->FatalError(env, __FILE__); >>>>>>>> ? 158???? } >>>>>>>> >>>>>>>> I expected it to be: >>>>>>>> >>>>>>>> ??? jboolean same = (*env)->IsSameObject(env, >>>>>>>> stack_info[i].thread, thread); >>>>>>>> ??? if ((*env)->ExceptionOccurred(env)) { >>>>>>>> ????? (*env)->ExceptionDescribe(env); >>>>>>>> ????? (*env)->FatalError(env, __FILE__); >>>>>>>> ??? } >>>>>>>> ??? if (same) { >>>>>>>> ????? target_info = &stack_info[i]; >>>>>>>> ????? break; >>>>>>>> ??? } >>>>>>>> >>>>>>>> Would it better to port this agent to C++ to simplify this code >>>>>>>> nicer? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Serguei >>>>>>>> >>>>>>>> >>>>>>>> On 7/5/20 06:13, Yasumasa Suenaga wrote: >>>>>>>>> Hi Serguei, >>>>>>>>> >>>>>>>>> Thanks for your comment! >>>>>>>>> I refactored testcase. Could you review again? >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.05/ >>>>>>>>> >>>>>>>>> It would check Java exception after IsSameObject() call. Does >>>>>>>>> it need? >>>>>>>>> Any exceptions are not described in JNI document[1], and JNI >>>>>>>>> implementation (jni_IsSameObject()) does not seem to throw it. >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> [1] >>>>>>>>> https://docs.oracle.com/en/java/javase/14/docs/specs/jni/functions.html#issameobject >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2020/07/05 14:46, serguei.spitsyn at oracle.com wrote: >>>>>>>>>> Hi Yasumasa, >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Okay, thanks. >>>>>>>>>> Then I'm okay to keep the GetSingleStackTraceClosure. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c.html >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c.html >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> I'm not sure the function 'is_same_thread() is needed. >>>>>>>>>> Why do not use the JNI IsSameObject instead? >>>>>>>>>> >>>>>>>>>> It seems to be a typo at L132 and L137. >>>>>>>>>> You, probably. did not want to print the same information for >>>>>>>>>> stack_info_1[i].frame_buffer[j].XXX twice. >>>>>>>>>> >>>>>>>>>> The code at lines 112-142 is not readable. >>>>>>>>>> I'd suggest to make a couple of refactoring steps. >>>>>>>>>> >>>>>>>>>> First step to simplify this a little bit would be with some >>>>>>>>>> renaming and getting rid of indexes: >>>>>>>>>> >>>>>>>>>> ?? 71 char err_msg[EXCEPTION_MSG_LEN] = {0}; >>>>>>>>>> ??... >>>>>>>>>> ??112?? /* Iterate all jvmtiStackInfo to check */ >>>>>>>>>> ??113?? for (i = 0; i < num_threads, *exception_msg != '\0'; >>>>>>>>>> i++) { >>>>>>>>>> ????????? jvmtiStackInfo *si1 = stack_info_1[i]; >>>>>>>>>> ????????? jvmtiStackInfo *si2 = stack_info_2[i]; >>>>>>>>>> ??114???? if (!IsSameObject(env, si1.thread, si2.thread)) { >>>>>>>>>> /* jvmtiStackInfo::thread */ >>>>>>>>>> ??115?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>> ??116??????????????? "thread[%d] is different: stack_info_1 = >>>>>>>>>> %p, stack_info_2 = %p", >>>>>>>>>> ??117??????????????? i, sinfo1.thread, sinfo2.thread); >>>>>>>>>> ??118???? } else if (si1.state != si2.state) { /* >>>>>>>>>> jvmtiStackInfo::state */ >>>>>>>>>> ??119?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>> ??120??????????????? "state[%d] is different: stack_info_1 = >>>>>>>>>> %d, stack_info_2 = %d", >>>>>>>>>> ??121??????????????? i, si1.state, si2.state); >>>>>>>>>> ??122???? } else if (si1.frame_count != si2.frame_count) { /* >>>>>>>>>> jvmtiStackInfo::frame_count */ >>>>>>>>>> ??123?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>> ??124??????????????? "frame_count[%d] is different: >>>>>>>>>> stack_info_1 = %d, stack_info_2 = %d", >>>>>>>>>> ??125??????????????? i, si1.frame_count, si2.frame_count); >>>>>>>>>> ??126???? } else { >>>>>>>>>> ??127?????? /* Iterate all jvmtiFrameInfo to check */ >>>>>>>>>> ??128?????? for (j = 0; j < si1.frame_count; j++) { >>>>>>>>>> ??129???????? if (si1.frame_buffer[j].method != >>>>>>>>>> si1.frame_buffer[j].method) { /* jvmtiFrameInfo::method */ >>>>>>>>>> ??130?????????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>> ??131??????????????????? "thread [%d] frame_buffer[%d].method >>>>>>>>>> is different: stack_info_1 = %lx, stack_info_2 = %lx", >>>>>>>>>> ??132??????????????????? i, j, si1.frame_buffer[j].method, >>>>>>>>>> si2.frame_buffer[j].method); >>>>>>>>>> ??133?????????? break; >>>>>>>>>> ??134???????? } else if (si1.frame_buffer[j].location != >>>>>>>>>> si1.frame_buffer[j].location) { /* jvmtiFrameInfo::location */ >>>>>>>>>> ??135?????????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>> ??136??????????????????? "thread [%d] >>>>>>>>>> frame_buffer[%d].location is different: stack_info_1 = %ld, >>>>>>>>>> stack_info_2 = %ld", >>>>>>>>>> ??137??????????????????? i, j, si1.frame_buffer[j].location, >>>>>>>>>> si2.frame_buffer[j].location); >>>>>>>>>> ??138?????????? break; >>>>>>>>>> ??139???????? } >>>>>>>>>> ??140?????? } >>>>>>>>>> ??141???? } >>>>>>>>>> ??142?? } >>>>>>>>>> >>>>>>>>>> Another step would be to create functions that implement a >>>>>>>>>> body of each loop. >>>>>>>>>> You can use the same techniques to simplify similar place >>>>>>>>>> (L127-L138) in the libOneGetThreadListStackTraces.c. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Serguei >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 7/3/20 15:55, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi Serguei, >>>>>>>>>>> >>>>>>>>>>> I'm not an Oracle employee, so I cannot know real request(s) >>>>>>>>>>> from your customers. >>>>>>>>>>> However JDK-8201641 says Dynatrace has requested this >>>>>>>>>>> enhancement. >>>>>>>>>>> >>>>>>>>>>> BTW I haven't heared any request from my customers about this. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2020/07/04 4:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>> >>>>>>>>>>>> This difference is not that big to care about. >>>>>>>>>>>> I feel this is really rare case and so, does not worth >>>>>>>>>>>> these complications. >>>>>>>>>>>> Do we have a real request from customers to optimize it? >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Serguei >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 7/3/20 01:16, Yasumasa Suenaga wrote: >>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>> >>>>>>>>>>>>> Generally I agree with you, but I have concern about the >>>>>>>>>>>>> difference of the result of GetStackTrace() and >>>>>>>>>>>>> GetThreadListStackTraces(). >>>>>>>>>>>>> >>>>>>>>>>>>> ? GetStackTrace: jvmtiFrameInfo >>>>>>>>>>>>> ? GetThreadListStackTraces: jvmtiStackInfo >>>>>>>>>>>>> >>>>>>>>>>>>> jvmtiStackInfo contains thread state, and it is ensured it >>>>>>>>>>>>> is the state of the call stack. >>>>>>>>>>>>> If we want to get both call stack and thread state, we >>>>>>>>>>>>> need to suspend target thread, and call both >>>>>>>>>>>>> GetStackTrace() and GetThreadState(). Is it ok? >>>>>>>>>>>>> >>>>>>>>>>>>> I was wondering if JDK-8201641 (parent ticket of this >>>>>>>>>>>>> change) needed them for profiling (dynatrace?) >>>>>>>>>>>>> If it is responsibility of JVMTI agent implementor, I >>>>>>>>>>>>> remove this closure. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 2020/07/03 16:45, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>> >>>>>>>>>>>>>> After some thinking I've concluded that I do not like >>>>>>>>>>>>>> this optimization >>>>>>>>>>>>>> of the GetThreadListStackTraces with >>>>>>>>>>>>>> GetSingleStackTraceClosure. >>>>>>>>>>>>>> >>>>>>>>>>>>>> We may need more opinions on this but these are my points: >>>>>>>>>>>>>> ??- it adds some complexity and ugliness >>>>>>>>>>>>>> ??- a win is doubtful because it has to be a rare case, >>>>>>>>>>>>>> so that total overhead should not be high >>>>>>>>>>>>>> ??- if it is really high for some use cases then it is up >>>>>>>>>>>>>> to the user >>>>>>>>>>>>>> ??? to optimize it with using GetStackTrace instead >>>>>>>>>>>>>> >>>>>>>>>>>>>> In such cases with doubtful overhead I usually prefer the >>>>>>>>>>>>>> simplicity. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Good examples where it makes sense to optimize are checks >>>>>>>>>>>>>> for target thread to be current thread. >>>>>>>>>>>>>> In such cases there is no need to suspend the target >>>>>>>>>>>>>> thread, or use a VMop/HandshakeClosure. >>>>>>>>>>>>>> For instance, please, see the Monitor functions with the >>>>>>>>>>>>>> check: (java_thread == calling_thread). >>>>>>>>>>>>>> Getting information for current thread is frequently used >>>>>>>>>>>>>> case, e.g. to get info at an event point. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 7/2/20 23:29, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>> Hi Dan, David, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I uploaded new webrev. Could you review again? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> OneGetThreadListStackTraces.java in this webrev would >>>>>>>>>>>>>>> wait until thread state is transited to "waiting" with >>>>>>>>>>>>>>> spin wait. >>>>>>>>>>>>>>> CountDownLatch::await call as Dan pointed is fixed in it :) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Diff from webrev.03: >>>>>>>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/c9aeb7001e50 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 2020/07/03 14:15, David Holmes wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>> On 2020/07/03 12:24, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>> On 7/2/20 10:50 PM, David Holmes wrote: >>>>>>>>>>>>>>>>>>> Sorry I'm responding here without seeing latest >>>>>>>>>>>>>>>>>>> webrev but there is enough context I think ... >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>> Hi Dan, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I upload new webrev. Could you review again? >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>>>>>>>>>>>>> ???? L1542: ??? // Get stack trace with handshake >>>>>>>>>>>>>>>>>>>>> ???????? nit - please add a period at the end. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> L1591: *stack_info_ptr = op.stack_info(); >>>>>>>>>>>>>>>>>>>>> ???????? The return parameter should not be >>>>>>>>>>>>>>>>>>>>> touched unless the return >>>>>>>>>>>>>>>>>>>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ???? old L1582: ? if (err == JVMTI_ERROR_NONE) { >>>>>>>>>>>>>>>>>>>>> ???????? Please restore this check. The return >>>>>>>>>>>>>>>>>>>>> parameter should not >>>>>>>>>>>>>>>>>>>>> ???????? be touched unless the return code in >>>>>>>>>>>>>>>>>>>>> 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> But op.stack_info() will return NULL if the error is >>>>>>>>>>>>>>>>>>> not JVMTI_ERROR_NONE. Are you (Dan) concerned about >>>>>>>>>>>>>>>>>>> someone passing in a non-null/initialized >>>>>>>>>>>>>>>>>>> out-pointer that will be reset to NULL if there was >>>>>>>>>>>>>>>>>>> an error? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Actually the way we used to test this in POSIX tests >>>>>>>>>>>>>>>>>> is to call >>>>>>>>>>>>>>>>>> an API with known bad parameters and the return >>>>>>>>>>>>>>>>>> parameter ptr >>>>>>>>>>>>>>>>>> set to NULL. If the return parameter ptr was touched >>>>>>>>>>>>>>>>>> when an >>>>>>>>>>>>>>>>>> error should have been detected on an earlier >>>>>>>>>>>>>>>>>> parameter, then >>>>>>>>>>>>>>>>>> the test failed. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> L1272: ? if (!jt->is_exiting() && (thread_oop != >>>>>>>>>>>>>>>>>>>>> NULL)) { >>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second >>>>>>>>>>>>>>>>>>>>> expression. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>> ???? old L1532: ? _result = >>>>>>>>>>>>>>>>>>>>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>>>>>>>>>>>>>>>>>> ???????? This deletion of the _result field threw >>>>>>>>>>>>>>>>>>>>> me for a minute and then >>>>>>>>>>>>>>>>>>>>> ???? ? ? I figured out that the field is init to >>>>>>>>>>>>>>>>>>>>> JVMTI_ERROR_THREAD_NOT_ALIVE >>>>>>>>>>>>>>>>>>>>> ???????? in the constructor. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ???? L1553: ? if (!jt->is_exiting() && >>>>>>>>>>>>>>>>>>>>> (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second >>>>>>>>>>>>>>>>>>>>> expression. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ???? L64: startSignal.countDown(); >>>>>>>>>>>>>>>>>>>>> ???????? I was expecting this to be a call to >>>>>>>>>>>>>>>>>>>>> await() instead of >>>>>>>>>>>>>>>>>>>>> ???????? countDown(). What am I missing here? >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ???????? I think this test might be passing by >>>>>>>>>>>>>>>>>>>>> accident right now, but... >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Main thread (which call JVMTI functions to test) >>>>>>>>>>>>>>>>>>>> should wait until test thread is ready. >>>>>>>>>>>>>>>>>>>> So main thread would wait startSignal, and test >>>>>>>>>>>>>>>>>>>> thread would count down. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> No! >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> The test thread that previously called obj.wait() >>>>>>>>>>>>>>>>>>> now calls latch.await(). >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> The main thread that previously called obj.notify() >>>>>>>>>>>>>>>>>>> now calls latch.countDown(). >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> The main thread continues to spin until it sees the >>>>>>>>>>>>>>>>>>> target is WAITING before proceeding with the test. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> If I add spin wait to wait until transit target thread >>>>>>>>>>>>>>>>> state is WAITING (as following), we don't need to call >>>>>>>>>>>>>>>>> SuspendThread(). >>>>>>>>>>>>>>>>> Which is better? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> The original spin-wait loop checking for? WAITING is >>>>>>>>>>>>>>>> better because it is the only guarantee that the target >>>>>>>>>>>>>>>> thread is blocked where you need it to be. suspending >>>>>>>>>>>>>>>> the thread is racy as you don't know exactly where the >>>>>>>>>>>>>>>> suspend will hit. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>> /* Wait until the thread state transits to "waiting" */ >>>>>>>>>>>>>>>>> while (th.getState() != Thread.State.WAITING) { >>>>>>>>>>>>>>>>> ???? Thread.onSpinWait(); >>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> For simplify, spin wait is prefer to >>>>>>>>>>>>>>>>> OneGetThreadListStackTraces.java in webrev.03. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Here's the flow as I see it: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>> ?? - start worker thread >>>>>>>>>>>>>>>>>> ?? - startSignal.await() >>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>> ?? - startSignal.countDown() >>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>> ?? - stopSignal.await() >>>>>>>>>>>>>>>>>> ???? - worker is now blocked >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>> ?? - checkCallStacks(th) >>>>>>>>>>>>>>>>>> ?? - stopSignal.countDown() >>>>>>>>>>>>>>>>>> ???? - worker is now unblocked >>>>>>>>>>>>>>>>>> ?? - th.join >>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>> ?? - runs off the end of run() >>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>> ?? - run off the end of main() >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ???? L92: ? jthreads = (jthread >>>>>>>>>>>>>>>>>>>>> *)malloc(sizeof(jthread) * num_threads); >>>>>>>>>>>>>>>>>>>>> ???????? You don't check for malloc() failure. >>>>>>>>>>>>>>>>>>>>> ???????? 'jthreads' is allocated but never freed. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ???? L91: ? result = >>>>>>>>>>>>>>>>>>>>> (*jvmti)->SuspendThread(jvmti, thread); >>>>>>>>>>>>>>>>>>>>> ???????? Why are you suspending the thread? >>>>>>>>>>>>>>>>>>>>> GetAllStackTraces() and >>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() do not require the >>>>>>>>>>>>>>>>>>>>> target thread(s) >>>>>>>>>>>>>>>>>>>>> ???????? to be suspend. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ???????? If you decide not to SuspendThread, then >>>>>>>>>>>>>>>>>>>>> you don't need the >>>>>>>>>>>>>>>>>>>>> ???????? AddCapabilities or the ResumeThread calls. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Test thread might not be entered following code >>>>>>>>>>>>>>>>>>>> (stopSignal.await()). We might see deferent call >>>>>>>>>>>>>>>>>>>> stack between GetAllStackTraces() and >>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces(). We cannot control to >>>>>>>>>>>>>>>>>>>> freeze call stack of test thread in Java code. >>>>>>>>>>>>>>>>>>>> (I didn't use SuspendThread() at first, but I saw >>>>>>>>>>>>>>>>>>>> some errors which causes in above.) >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> So we need to call SuspendThread() to ensure we can >>>>>>>>>>>>>>>>>>>> see same call stack. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> If you are checking that the thread is in state >>>>>>>>>>>>>>>>>>> WAITING then it cannot escape from that state and >>>>>>>>>>>>>>>>>>> you can sample the stack multiple times from any API >>>>>>>>>>>>>>>>>>> and get the same result. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I suspect the errors you saw were from the apparent >>>>>>>>>>>>>>>>>>> incorrect use of the CountDownLatch. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> With the flow outlined above, the worker thread >>>>>>>>>>>>>>>>>> should be >>>>>>>>>>>>>>>>>> nicely blocked in stopSignal.await() when stuff is >>>>>>>>>>>>>>>>>> sampled. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could review again? >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Updates look fine - thanks. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> One minor nit: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> 1274 _collector.allocate_and_fill_stacks(1); >>>>>>>>>>>>>>>>>>>>>>> 1275 _collector.set_result(JVMTI_ERROR_NONE); >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> In the other places where you use _collector you >>>>>>>>>>>>>>>>>>>>>>> rely on result being initialized to >>>>>>>>>>>>>>>>>>>>>>> JVMTI_ERROR_NONE, rather than setting it >>>>>>>>>>>>>>>>>>>>>>> directly after allocate_and_fill_stacks(). >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 820 >>>>>>>>>>>>>>>>>>>>>>>>>>>>> assert(SafepointSynchronize::is_at_safepoint() >>>>>>>>>>>>>>>>>>>>>>>>>>>>> || >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 821 >>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 822 current_thread == >>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 823 "at safepoint / handshake or target >>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is >>>>>>>>>>>>>>>>>>>>>>>>>>>>> necessary, as even if the target is >>>>>>>>>>>>>>>>>>>>>>>>>>>>> suspended we must still be at a safepoint >>>>>>>>>>>>>>>>>>>>>>>>>>>>> or in a handshake with it. Makes me wonder >>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we used to allow a racy stacktrace >>>>>>>>>>>>>>>>>>>>>>>>>>>>> operation on a suspended thread, assuming >>>>>>>>>>>>>>>>>>>>>>>>>>>>> it would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> This function (JvmtiEnvBase::get_stack_trace()) >>>>>>>>>>>>>>>>>>>>>>>> can be called to get own stack trace. For >>>>>>>>>>>>>>>>>>>>>>>> example, we can call GetStackTrace() for >>>>>>>>>>>>>>>>>>>>>>>> current thread at JVMTI event. >>>>>>>>>>>>>>>>>>>>>>>> So I changed assert as below: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>> ??820 assert(current_thread == java_thread || >>>>>>>>>>>>>>>>>>>>>>>> ??821 SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == >>>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>> ??823????????? "call by myself / at safepoint / >>>>>>>>>>>>>>>>>>>>>>>> at handshake"); >>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Yep good catch. I hope current tests caught that. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> They would be tested in >>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ >>>>>>>>>>>>>>>>>>>>>> (own call stacks), and getstacktr003 (call stacks >>>>>>>>>>>>>>>>>>>>>> in other thread). >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Speaking of tests ... >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> In the native code I think you need to check the >>>>>>>>>>>>>>>>>>>>>>> success of all JNI methods that can throw >>>>>>>>>>>>>>>>>>>>>>> exceptions - otherwise I believe the tests may >>>>>>>>>>>>>>>>>>>>>>> trigger warnings if -Xcheck:jni is used with >>>>>>>>>>>>>>>>>>>>>>> them. See for example: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I updated testcases to check JNI and JVMTI >>>>>>>>>>>>>>>>>>>>>> function calls. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> In the Java code the target thread: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ?? 45???? public void run() { >>>>>>>>>>>>>>>>>>>>>>> ?? 46?????? try { >>>>>>>>>>>>>>>>>>>>>>> ?? 47 synchronized (lock) { >>>>>>>>>>>>>>>>>>>>>>> ?? 48 lock.wait(); >>>>>>>>>>>>>>>>>>>>>>> ?? 49 System.out.println("OK"); >>>>>>>>>>>>>>>>>>>>>>> ?? 50?????????? } >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> is potentially susceptible to spurious wakeups. >>>>>>>>>>>>>>>>>>>>>>> Using a CountDownLatch would be robust. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> If so, we can call make_local() in L1272 >>>>>>>>>>>>>>>>>>>>>>>>>> without JavaThread (or we can pass current >>>>>>>>>>>>>>>>>>>>>>>>>> thread to make_local()). Is it right? >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>> 1272 >>>>>>>>>>>>>>>>>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>>>>>>>>>>>>>>>>> thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Sorry I got confused, _calling_thread may not >>>>>>>>>>>>>>>>>>>>>>>>> be the current thread as we could be executing >>>>>>>>>>>>>>>>>>>>>>>>> the handshake in the target thread itself. So >>>>>>>>>>>>>>>>>>>>>>>>> the ResourceMark is correct as-is (implicitly >>>>>>>>>>>>>>>>>>>>>>>>> for current thread). >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> The argument to fill_frames will be used in >>>>>>>>>>>>>>>>>>>>>>>>> the jvmtiStackInfo and passed back to the >>>>>>>>>>>>>>>>>>>>>>>>> _calling_thread, so it must be created via >>>>>>>>>>>>>>>>>>>>>>>>> make_local(_calling_thread, ...) as you >>>>>>>>>>>>>>>>>>>>>>>>> presently have. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you for reviewing! I will update new >>>>>>>>>>>>>>>>>>>>>>>>>>>> webrev tomorrow. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : >>>>>>>>>>>>>>>>>>>>>>>>>>>>> public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 498 class VM_GetAllStackTraces : public >>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 500 JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 502 MultipleStackTracesCollector >>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of >>>>>>>>>>>>>>>>>>>>>>>>>>>>> another class like that as it may not be >>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the stack. I think >>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector should not >>>>>>>>>>>>>>>>>>>>>>>>>>>>> extend any allocation class, and should >>>>>>>>>>>>>>>>>>>>>>>>>>>>> always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>>>>>>>>>>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>>>>>>>>>>>>>>>>>>> ??? : >>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> class GetAllStackTraces : public >>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? private: >>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I updated webrev for 8242428. Could you >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change migrate to use direct >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> handshake for GetStackTrace() and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() (when >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread_count == 1). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks really good now! I only have a >>>>>>>>>>>>>>>>>>>>>>>>>>>>> few nits below. There is one thing I don't >>>>>>>>>>>>>>>>>>>>>>>>>>>>> like about it but it requires a change to >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the main Handshake logic to address - in >>>>>>>>>>>>>>>>>>>>>>>>>>>>> JvmtiEnv::GetThreadListStackTraces you >>>>>>>>>>>>>>>>>>>>>>>>>>>>> have to create a ThreadsListHandle to >>>>>>>>>>>>>>>>>>>>>>>>>>>>> convert the jthread to a JavaThread, but >>>>>>>>>>>>>>>>>>>>>>>>>>>>> then the Handshake::execute_direct creates >>>>>>>>>>>>>>>>>>>>>>>>>>>>> another ThreadsListHandle internally. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> That's a waste. I will discuss with Robbin >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and file a RFE to have an overload of >>>>>>>>>>>>>>>>>>>>>>>>>>>>> execute_direct that takes an existing TLH. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Actually it's worse than that because we >>>>>>>>>>>>>>>>>>>>>>>>>>>>> have another TLH in use at the entry point >>>>>>>>>>>>>>>>>>>>>>>>>>>>> for the JVMTI functions, so I think there >>>>>>>>>>>>>>>>>>>>>>>>>>>>> may be some scope for simplifying the use >>>>>>>>>>>>>>>>>>>>>>>>>>>>> of TLH instances - future RFE. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??451 GetStackTraceClosure(JvmtiEnv *env, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> jint start_depth, jint max_count, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??452 jvmtiFrameInfo* frame_buffer, jint* >>>>>>>>>>>>>>>>>>>>>>>>>>>>> count_ptr) >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??453???? : >>>>>>>>>>>>>>>>>>>>>>>>>>>>> HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??454 _env(env), >>>>>>>>>>>>>>>>>>>>>>>>>>>>> _start_depth(start_depth), >>>>>>>>>>>>>>>>>>>>>>>>>>>>> _max_count(max_count), >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??455 _frame_buffer(frame_buffer), >>>>>>>>>>>>>>>>>>>>>>>>>>>>> _count_ptr(count_ptr), >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??456 _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Nit: can you do one initializer per line >>>>>>>>>>>>>>>>>>>>>>>>>>>>> please. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : >>>>>>>>>>>>>>>>>>>>>>>>>>>>> public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??498 class VM_GetAllStackTraces : public >>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??500 JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??502 MultipleStackTracesCollector >>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of >>>>>>>>>>>>>>>>>>>>>>>>>>>>> another class like that as it may not be >>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the stack. I think >>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector should not >>>>>>>>>>>>>>>>>>>>>>>>>>>>> extend any allocation class, and should >>>>>>>>>>>>>>>>>>>>>>>>>>>>> always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 481 MultipleStackTracesCollector(JvmtiEnv >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *env, jint max_frame_count) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??483 _max_frame_count = max_frame_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??484 _frame_count_total = 0; >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??485 _head = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??486 _stack_info = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??487 _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> As you are touching this can you change it >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to use an initializer list as you did for >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the HandshakeClosure, and please keep one >>>>>>>>>>>>>>>>>>>>>>>>>>>>> item per line. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??820 >>>>>>>>>>>>>>>>>>>>>>>>>>>>> assert(SafepointSynchronize::is_at_safepoint() >>>>>>>>>>>>>>>>>>>>>>>>>>>>> || >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??821 >>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == >>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??823 "at safepoint / handshake or target >>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is >>>>>>>>>>>>>>>>>>>>>>>>>>>>> necessary, as even if the target is >>>>>>>>>>>>>>>>>>>>>>>>>>>>> suspended we must still be at a safepoint >>>>>>>>>>>>>>>>>>>>>>>>>>>>> or in a handshake with it. Makes me wonder >>>>>>>>>>>>>>>>>>>>>>>>>>>>> if we used to allow a racy stacktrace >>>>>>>>>>>>>>>>>>>>>>>>>>>>> operation on a suspended thread, assuming >>>>>>>>>>>>>>>>>>>>>>>>>>>>> it would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1269 >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && >>>>>>>>>>>>>>>>>>>>>>>>>>>>> (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 >>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is frustrating that this entire call >>>>>>>>>>>>>>>>>>>>>>>>>>>>> chain started with a jthread reference, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> which we converted to a JavaThread, only >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to eventually need to convert it back to a >>>>>>>>>>>>>>>>>>>>>>>>>>>>> jthread! I think there is some scope for >>>>>>>>>>>>>>>>>>>>>>>>>>>>> simplification here but not as part of >>>>>>>>>>>>>>>>>>>>>>>>>>>>> this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm still pondering the test logic but >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wanted to send this now. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetThreadListStackTrace (for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces) and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetAllStackTraces (for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetAllStackTraces) have inherited >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetMultipleStackTraces VM operation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which provides the feature to generate >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jvmtiStackInfo. I modified >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetMultipleStackTraces to a normal C++ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class to share with HandshakeClosure for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (GetSingleStackTraceClosure). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I added new testcases which test >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread_count == 1 and with all threads. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jvmti serviceability/jdwp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti vmTestbase/nsk/jdi >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? JBS: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? webrev: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change replace following VM >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operations to direct handshake. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (GetFrameLocation()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (GetThreadListStackTrace()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTrace() uses direct >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> handshake if thread count == 1. In other >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> case (thread count > 1), it would be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performed as VM operation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Caller of VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (JvmtiEnvThreadState::reset_current_location()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> might be called at safepoint. So I added >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> safepoint check in its caller. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jvmti serviceability/jdwp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti vmTestbase/nsk/jdi >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/ns >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I tested it on submit repo, then it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> has execution error >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> due to dependency error. So I think it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> does not occur by this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>>>>> From igor.ignatyev at oracle.com Wed Jul 8 17:21:11 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 8 Jul 2020 10:21:11 -0700 Subject: RFR [15] : 8249028 : clean up FileInstaller $test.src $cwd in vmTestbase_nsk_monitoring tests Message-ID: http://cr.openjdk.java.net/~iignatyev/8249028/webrev.00/ > 547 lines changed: 0 ins; 361 del; 186 mod; Hi all, could you please review the patch which removes `FileInstaller . .` jtreg action from :vmTestbase_nsk_monitoring tests? from the main issue(8204985): > all vmTestbase tests have '@run driver jdk.test.lib.FileInstaller . .' to mimic old test harness behavior and copy all files from a test source directory to a current work directory. some tests depend on this step, so we need 1st identify such tests and then either rewrite them not to have this dependency or leave FileInstaller only in these tests. effectively, the patch is just `ag -l '@run driver jdk.test.lib.FileInstaller . .' vmTestbase/nsk/monitoring | xargs -I{} gsed -i '/@run driver jdk.test.lib.FileInstaller \. \./d' {}` testing: :vmTestbase_nsk_monitoring on linux-x64 webrev: http://cr.openjdk.java.net/~iignatyev//8249018/webrev.00 JBS: https://bugs.openjdk.java.net/browse/JDK-8249028 Thanks, -- Igor From chris.plummer at oracle.com Wed Jul 8 18:04:12 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 8 Jul 2020 11:04:12 -0700 Subject: RFR(M): 8247272: SA ELF file support has never worked for 64-bit causing address to symbol name mapping to fail In-Reply-To: References: <338118cb-37ff-9d03-f22e-c96322bfd7a6@oracle.com> Message-ID: <54ca0720-61f2-4580-80e5-b2a0da966cc0@oracle.com> Hi Kevin, Thanks for the review. I'll add the additional Elf64_Addr and Elf64_Off comments. Probably the others should be updated too. Although they are the same size, they do have different names. For example: /* Type for a 16-bit quantity.? */ typedef uint16_t Elf32_Half; typedef uint16_t Elf64_Half; thanks, Chris On 7/8/20 3:47 AM, Kevin Walls wrote: > Hi Chris -- > > This is a great story/history lesson. > > You could if you like, edit those comments in ElfFileParser.java so > "Elf32_Addr" as they will contain either "Elf64_Addr or Elf32_Addr", > similarly Elf64_Off.? The other Elf64 fields are the same as the 32 > bit ones. > > Yes, the symbol fields are ordered differently. > > So all looks good to me! > > Thanks > Kevin > > > > On 08/07/2020 07:20, Chris Plummer wrote: >> Hello, >> >> Please help review the following: >> >> http://cr.openjdk.java.net/~cjplummer/8247272/webrev.00/index.html >> https://bugs.openjdk.java.net/browse/JDK-8247272 >> >> The short story is that SA address to native symbol name >> mapping/lookup has never worked on 64-bit, and this is due to the >> java level ELF file support only supporting 32-bit. This CR fixes >> that, and I believe also maintains 32-bit compatibility, although I >> have no way of testing that. >> >> There is more to the story however on how we got here. Before going >> into the gory detail below, I just want to point out that currently >> nothing is using this support, and therefore it is technically not >> fixing anything, although I did verify that the fixes work (see >> details below). Also, I intend to remove all the java level ELF file >> support as part of JDK-8247516 [1]. The only reason I want to push >> these changes first is because I already did the work to get it >> working with 64-bit, and would like to get it archived before >> removing it in case for some reason it is revived in the future. >> >> Now for the ugly details on how we got here (and you really don't >> need to read this unless you have any concerns with what I stated >> above). It starts with the clhsdb "whatis" command, which was the >> only (indirect) user of this java level ELF file support. It's >> implementation is in javascript, so we have not had access to it ever >> since JDK9 module support broke the SA javascript support (and >> javascript support is now removed). I started the process of >> converting "whatis" to java. It is basically the same as the clhsdb >> "findpc" command, except it also checks for native symbols, which it >> does with the following code: >> >> ? var dso = loadObjectContainingPC(addr); >> ? var sym = dso.closestSymbolToPC(addr); >> ? return sym.name + '+' + sym.offset; >> >> Converting this to java was trivial. I just stuck support for it in >> the PointerFinder class, which is what findpc relies on. However, it >> always failed to successfully lookup a symbol. I found that >> DSO.closestSymbolToPC() called into the java level ELF support, and >> that was failing badly. After some debugging I noticed that the >> values read in for various ELF headers were mostly garbage. It then >> occurred to me that it was reading in 32-bit values that probably >> needed to be 64-bit. Sure enough, this code was never converted to >> 64-bit support. I then went and tried "whatis" on JDK8, the last >> version where it was available, and it failed there also with 64-bit >> binaries. So this is why I initially fixed it to work with 64-bit, >> and also how I tested it (using the modified findpc on a native >> symbol). But the story continues... >> >> DSO.java, and as a consequence the java ELF file support, is used by >> all our posix ports to do address to symbol lookups. So I figured >> that after fixing the java level ELF file support for 64-bit, my >> improved findpc would start working on OSX also. No such luck, and >> for obvious reasons. OSX uses mach-o files. This ELF code should >> never have been used for it, and of course has never worked. >> >> So I was left trying to figure out how to do OSX address to native >> symbol lookups. I then recalled that there was a >> CFrame.closestSymbolToPC() API that did address to native symbol >> lookups for native stack traces, and wondered how it was ever working >> (even on linux with the broken ELF 64-bit support). It turns out this >> takes a very different path to do the lookups, ending up in native >> code in libsaproc, where we also have ELF file support. I then >> converted DSO.closestSymbolToPC(addr) to use this libsaproc code >> instead, and it worked fine. So now there was no need for the java >> level ELF file support since its only user was >> DSO.closestSymbolToPC(addr). I should also add that this is the >> approach that has always been used on windows, with both >> CFrame.closestSymbolToPC() and DSO.closestSymbolToPC(addr) using the >> same libsaproc support. >> >> There is still a bit more to the story. After diverting >> DSO.closestSymbolToPC(addr) to the libsaproc lookup code, it still >> didn't work for OSX. I thought it would just work since the native >> BsdDebuggerLocal.lookupByName0() is implemented, and it seems to >> trickle down to the proper lower level APIs to find the symbol, but >> there were two issues. The first is that for processes there is no >> support for looking up all the libraries and populating the list of >> ps_prochandle structures that are used to do the symbol lookups. This >> was just never implemented (also is why PMap does not work for OSX >> processes). For core files the ps_prochandle structs are there, but >> the lookup code was badly broken. That has now been fixed by >> JDK-8247515 [2], currently out for review. So the end result is we'll >> have address to native symbol lookup for everything but OSX processes. >> >> If? your still here, thanks for listening! >> >> Chris >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8247516 >> [2] https://bugs.openjdk.java.net/browse/JDK-8247515 >> >> >> >> From chris.plummer at oracle.com Wed Jul 8 19:26:27 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 8 Jul 2020 12:26:27 -0700 Subject: RFR(M): 8247272: SA ELF file support has never worked for 64-bit causing address to symbol name mapping to fail In-Reply-To: <54ca0720-61f2-4580-80e5-b2a0da966cc0@oracle.com> References: <338118cb-37ff-9d03-f22e-c96322bfd7a6@oracle.com> <54ca0720-61f2-4580-80e5-b2a0da966cc0@oracle.com> Message-ID: <9d743570-7e55-df53-abfc-77a62af4d136@oracle.com> Webrev has been updated with the suggested comment changes. Note to new reviewers, look in webrev.00 first since it doesn't have the clutter of the comment changes, making it easier to see which lines actually have code changes. http://cr.openjdk.java.net/~cjplummer/8247272/webrev.01/index.html thanks, Chris On 7/8/20 11:04 AM, Chris Plummer wrote: > Hi Kevin, > > Thanks for the review. I'll add the additional Elf64_Addr and > Elf64_Off comments. Probably the others should be updated too. > Although they are the same size, they do have different names. For > example: > > /* Type for a 16-bit quantity.? */ > typedef uint16_t Elf32_Half; > typedef uint16_t Elf64_Half; > > thanks, > > Chris > > On 7/8/20 3:47 AM, Kevin Walls wrote: >> Hi Chris -- >> >> This is a great story/history lesson. >> >> You could if you like, edit those comments in ElfFileParser.java so >> "Elf32_Addr" as they will contain either "Elf64_Addr or Elf32_Addr", >> similarly Elf64_Off.? The other Elf64 fields are the same as the 32 >> bit ones. >> >> Yes, the symbol fields are ordered differently. >> >> So all looks good to me! >> >> Thanks >> Kevin >> >> >> >> On 08/07/2020 07:20, Chris Plummer wrote: >>> Hello, >>> >>> Please help review the following: >>> >>> http://cr.openjdk.java.net/~cjplummer/8247272/webrev.00/index.html >>> https://bugs.openjdk.java.net/browse/JDK-8247272 >>> >>> The short story is that SA address to native symbol name >>> mapping/lookup has never worked on 64-bit, and this is due to the >>> java level ELF file support only supporting 32-bit. This CR fixes >>> that, and I believe also maintains 32-bit compatibility, although I >>> have no way of testing that. >>> >>> There is more to the story however on how we got here. Before going >>> into the gory detail below, I just want to point out that currently >>> nothing is using this support, and therefore it is technically not >>> fixing anything, although I did verify that the fixes work (see >>> details below). Also, I intend to remove all the java level ELF file >>> support as part of JDK-8247516 [1]. The only reason I want to push >>> these changes first is because I already did the work to get it >>> working with 64-bit, and would like to get it archived before >>> removing it in case for some reason it is revived in the future. >>> >>> Now for the ugly details on how we got here (and you really don't >>> need to read this unless you have any concerns with what I stated >>> above). It starts with the clhsdb "whatis" command, which was the >>> only (indirect) user of this java level ELF file support. It's >>> implementation is in javascript, so we have not had access to it >>> ever since JDK9 module support broke the SA javascript support (and >>> javascript support is now removed). I started the process of >>> converting "whatis" to java. It is basically the same as the clhsdb >>> "findpc" command, except it also checks for native symbols, which it >>> does with the following code: >>> >>> ? var dso = loadObjectContainingPC(addr); >>> ? var sym = dso.closestSymbolToPC(addr); >>> ? return sym.name + '+' + sym.offset; >>> >>> Converting this to java was trivial. I just stuck support for it in >>> the PointerFinder class, which is what findpc relies on. However, it >>> always failed to successfully lookup a symbol. I found that >>> DSO.closestSymbolToPC() called into the java level ELF support, and >>> that was failing badly. After some debugging I noticed that the >>> values read in for various ELF headers were mostly garbage. It then >>> occurred to me that it was reading in 32-bit values that probably >>> needed to be 64-bit. Sure enough, this code was never converted to >>> 64-bit support. I then went and tried "whatis" on JDK8, the last >>> version where it was available, and it failed there also with 64-bit >>> binaries. So this is why I initially fixed it to work with 64-bit, >>> and also how I tested it (using the modified findpc on a native >>> symbol). But the story continues... >>> >>> DSO.java, and as a consequence the java ELF file support, is used by >>> all our posix ports to do address to symbol lookups. So I figured >>> that after fixing the java level ELF file support for 64-bit, my >>> improved findpc would start working on OSX also. No such luck, and >>> for obvious reasons. OSX uses mach-o files. This ELF code should >>> never have been used for it, and of course has never worked. >>> >>> So I was left trying to figure out how to do OSX address to native >>> symbol lookups. I then recalled that there was a >>> CFrame.closestSymbolToPC() API that did address to native symbol >>> lookups for native stack traces, and wondered how it was ever >>> working (even on linux with the broken ELF 64-bit support). It turns >>> out this takes a very different path to do the lookups, ending up in >>> native code in libsaproc, where we also have ELF file support. I >>> then converted DSO.closestSymbolToPC(addr) to use this libsaproc >>> code instead, and it worked fine. So now there was no need for the >>> java level ELF file support since its only user was >>> DSO.closestSymbolToPC(addr). I should also add that this is the >>> approach that has always been used on windows, with both >>> CFrame.closestSymbolToPC() and DSO.closestSymbolToPC(addr) using the >>> same libsaproc support. >>> >>> There is still a bit more to the story. After diverting >>> DSO.closestSymbolToPC(addr) to the libsaproc lookup code, it still >>> didn't work for OSX. I thought it would just work since the native >>> BsdDebuggerLocal.lookupByName0() is implemented, and it seems to >>> trickle down to the proper lower level APIs to find the symbol, but >>> there were two issues. The first is that for processes there is no >>> support for looking up all the libraries and populating the list of >>> ps_prochandle structures that are used to do the symbol lookups. >>> This was just never implemented (also is why PMap does not work for >>> OSX processes). For core files the ps_prochandle structs are there, >>> but the lookup code was badly broken. That has now been fixed by >>> JDK-8247515 [2], currently out for review. So the end result is >>> we'll have address to native symbol lookup for everything but OSX >>> processes. >>> >>> If? your still here, thanks for listening! >>> >>> Chris >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8247516 >>> [2] https://bugs.openjdk.java.net/browse/JDK-8247515 >>> >>> >>> >>> > > From kevin.walls at oracle.com Wed Jul 8 21:07:24 2020 From: kevin.walls at oracle.com (Kevin Walls) Date: Wed, 8 Jul 2020 22:07:24 +0100 Subject: RFR(M): 8247272: SA ELF file support has never worked for 64-bit causing address to symbol name mapping to fail In-Reply-To: <9d743570-7e55-df53-abfc-77a62af4d136@oracle.com> References: <338118cb-37ff-9d03-f22e-c96322bfd7a6@oracle.com> <54ca0720-61f2-4580-80e5-b2a0da966cc0@oracle.com> <9d743570-7e55-df53-abfc-77a62af4d136@oracle.com> Message-ID: Thanks Chris, it's a bit of clutter, but truthful clutter. 8-) On 08/07/2020 20:26, Chris Plummer wrote: > Webrev has been updated with the suggested comment changes. Note to > new reviewers, look in webrev.00 first since it doesn't have the > clutter of the comment changes, making it easier to see which lines > actually have code changes. > > http://cr.openjdk.java.net/~cjplummer/8247272/webrev.01/index.html > > thanks, > > Chris > > On 7/8/20 11:04 AM, Chris Plummer wrote: >> Hi Kevin, >> >> Thanks for the review. I'll add the additional Elf64_Addr and >> Elf64_Off comments. Probably the others should be updated too. >> Although they are the same size, they do have different names. For >> example: >> >> /* Type for a 16-bit quantity.? */ >> typedef uint16_t Elf32_Half; >> typedef uint16_t Elf64_Half; >> >> thanks, >> >> Chris >> >> On 7/8/20 3:47 AM, Kevin Walls wrote: >>> Hi Chris -- >>> >>> This is a great story/history lesson. >>> >>> You could if you like, edit those comments in ElfFileParser.java so >>> "Elf32_Addr" as they will contain either "Elf64_Addr or Elf32_Addr", >>> similarly Elf64_Off.? The other Elf64 fields are the same as the 32 >>> bit ones. >>> >>> Yes, the symbol fields are ordered differently. >>> >>> So all looks good to me! >>> >>> Thanks >>> Kevin >>> >>> >>> >>> On 08/07/2020 07:20, Chris Plummer wrote: >>>> Hello, >>>> >>>> Please help review the following: >>>> >>>> http://cr.openjdk.java.net/~cjplummer/8247272/webrev.00/index.html >>>> https://bugs.openjdk.java.net/browse/JDK-8247272 >>>> >>>> The short story is that SA address to native symbol name >>>> mapping/lookup has never worked on 64-bit, and this is due to the >>>> java level ELF file support only supporting 32-bit. This CR fixes >>>> that, and I believe also maintains 32-bit compatibility, although I >>>> have no way of testing that. >>>> >>>> There is more to the story however on how we got here. Before going >>>> into the gory detail below, I just want to point out that currently >>>> nothing is using this support, and therefore it is technically not >>>> fixing anything, although I did verify that the fixes work (see >>>> details below). Also, I intend to remove all the java level ELF >>>> file support as part of JDK-8247516 [1]. The only reason I want to >>>> push these changes first is because I already did the work to get >>>> it working with 64-bit, and would like to get it archived before >>>> removing it in case for some reason it is revived in the future. >>>> >>>> Now for the ugly details on how we got here (and you really don't >>>> need to read this unless you have any concerns with what I stated >>>> above). It starts with the clhsdb "whatis" command, which was the >>>> only (indirect) user of this java level ELF file support. It's >>>> implementation is in javascript, so we have not had access to it >>>> ever since JDK9 module support broke the SA javascript support (and >>>> javascript support is now removed). I started the process of >>>> converting "whatis" to java. It is basically the same as the clhsdb >>>> "findpc" command, except it also checks for native symbols, which >>>> it does with the following code: >>>> >>>> ? var dso = loadObjectContainingPC(addr); >>>> ? var sym = dso.closestSymbolToPC(addr); >>>> ? return sym.name + '+' + sym.offset; >>>> >>>> Converting this to java was trivial. I just stuck support for it in >>>> the PointerFinder class, which is what findpc relies on. However, >>>> it always failed to successfully lookup a symbol. I found that >>>> DSO.closestSymbolToPC() called into the java level ELF support, and >>>> that was failing badly. After some debugging I noticed that the >>>> values read in for various ELF headers were mostly garbage. It then >>>> occurred to me that it was reading in 32-bit values that probably >>>> needed to be 64-bit. Sure enough, this code was never converted to >>>> 64-bit support. I then went and tried "whatis" on JDK8, the last >>>> version where it was available, and it failed there also with >>>> 64-bit binaries. So this is why I initially fixed it to work with >>>> 64-bit, and also how I tested it (using the modified findpc on a >>>> native symbol). But the story continues... >>>> >>>> DSO.java, and as a consequence the java ELF file support, is used >>>> by all our posix ports to do address to symbol lookups. So I >>>> figured that after fixing the java level ELF file support for >>>> 64-bit, my improved findpc would start working on OSX also. No such >>>> luck, and for obvious reasons. OSX uses mach-o files. This ELF code >>>> should never have been used for it, and of course has never worked. >>>> >>>> So I was left trying to figure out how to do OSX address to native >>>> symbol lookups. I then recalled that there was a >>>> CFrame.closestSymbolToPC() API that did address to native symbol >>>> lookups for native stack traces, and wondered how it was ever >>>> working (even on linux with the broken ELF 64-bit support). It >>>> turns out this takes a very different path to do the lookups, >>>> ending up in native code in libsaproc, where we also have ELF file >>>> support. I then converted DSO.closestSymbolToPC(addr) to use this >>>> libsaproc code instead, and it worked fine. So now there was no >>>> need for the java level ELF file support since its only user was >>>> DSO.closestSymbolToPC(addr). I should also add that this is the >>>> approach that has always been used on windows, with both >>>> CFrame.closestSymbolToPC() and DSO.closestSymbolToPC(addr) using >>>> the same libsaproc support. >>>> >>>> There is still a bit more to the story. After diverting >>>> DSO.closestSymbolToPC(addr) to the libsaproc lookup code, it still >>>> didn't work for OSX. I thought it would just work since the native >>>> BsdDebuggerLocal.lookupByName0() is implemented, and it seems to >>>> trickle down to the proper lower level APIs to find the symbol, but >>>> there were two issues. The first is that for processes there is no >>>> support for looking up all the libraries and populating the list of >>>> ps_prochandle structures that are used to do the symbol lookups. >>>> This was just never implemented (also is why PMap does not work for >>>> OSX processes). For core files the ps_prochandle structs are there, >>>> but the lookup code was badly broken. That has now been fixed by >>>> JDK-8247515 [2], currently out for review. So the end result is >>>> we'll have address to native symbol lookup for everything but OSX >>>> processes. >>>> >>>> If? your still here, thanks for listening! >>>> >>>> Chris >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8247516 >>>> [2] https://bugs.openjdk.java.net/browse/JDK-8247515 >>>> >>>> >>>> >>>> >> >> > > From suenaga at oss.nttdata.com Thu Jul 9 00:25:38 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Thu, 9 Jul 2020 09:25:38 +0900 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <04e09eb7-9483-8f63-4bb4-01c9e814802e@oracle.com> <238a89a3-06b1-90e4-d5a4-8dc2f1ed12ab@oracle.com> <4c43bc2c-551e-d042-8fa7-b63e755888d1@oracle.com> <8c387550-a06e-6320-f453-0e898486befb@oss.nttdata.com> <19cfa2a7-f441-425d-5623-1c11c6a870b6@oss.nttdata.com> <237e8d26-d185-1008-6e07-5b9f36314534@oracle.com> Message-ID: Hi Dan, Thanks for your comment! I uploaded new webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.09/ Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/5d167adf8524 I saw similar build errors in libOneGetThreadListStackTraces.cpp on Windows. This webrev fixes them. Thanks, Yasumasa On 2020/07/09 1:42, Daniel D. Daugherty wrote: > > http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ > > src/hotspot/share/prims/jvmtiEnv.cpp > ??? No comments. > > src/hotspot/share/prims/jvmtiEnvBase.cpp > ??? L1159: ? Thread *current_thread = Thread::current(); > ??????? Please add "#ifdef ASSERT" above and "#endif" below since > ??????? current_thread is only used for the assert() in this function. > > src/hotspot/share/prims/jvmtiEnvBase.hpp > ??? L549: ???????????????????????????? jthread java_thread, jint max_frame_count) > ??? L552: ????? _jthread(java_thread), > ??????? Please: s/java_thread/thread/ on both lines. > > src/hotspot/share/runtime/vmOperations.hpp > ??? No comments. > > test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java > ??? No comments. > > test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp > ??? L27: #include > ??????? This include is out of order; should be first in the list. > > ??? This file doesn't compile on my MBP13: > > ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:49:14: error: format specifies type 'unsigned long' but the argument has type 'jmethodID' (aka '_jmethodID *') [-Werror,-Wformat] > ???????????? fi1->method, fi2->method); > ???????????? ^~~~~~~~~~~ > ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:49:27: error: format specifies type 'unsigned long' but the argument has type 'jmethodID' (aka '_jmethodID *') [-Werror,-Wformat] > ???????????? fi1->method, fi2->method); > ????????????????????????? ^~~~~~~~~~~ > ??? 2 errors generated. > > ??? This change made it compile on my MBP13, but that may break it on > ??? other platforms: > > ??? $ hg diff > ??? diff -r 560847c69fbe test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp > ??? --- a/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp Wed Jul 08 12:13:32 2020 -0400 > ??? +++ b/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp Wed Jul 08 12:40:42 2020 -0400 > ??? @@ -46,7 +46,7 @@ > ? ?? ? if (fi1->method != fi2->method) { /* jvmtiFrameInfo::method */ > ? ?? ??? snprintf(err_msg, sizeof(err_msg), > ? ? ? ??????????? "method is different: fi1 = %lx, fi2 = %lx", > ??? -???????????? fi1->method, fi2->method); > ??? +???????????? (unsigned long) fi1->method, (unsigned long) fi2->method); > ? ?? ??? env->FatalError(err_msg); > ? ?? ? } else if (fi1->location != fi2->location) { /* jvmtiFrameInfo::location */ > ? ?? ??? snprintf(err_msg, sizeof(err_msg), > > ??? I'm not sure of the right platform independent way to output > ??? the 'method' field. > > Dan > > > On 7/8/20 4:04 AM, Yasumasa Suenaga wrote: >> Hi David, >> >> On 2020/07/08 15:27, David Holmes wrote: >>> Hi Yasumasa, >>> >>> On 7/07/2020 6:54 pm, Yasumasa Suenaga wrote: >>>> Hi David, Serguei, >>>> >>>> Serguei, thank you for replying even though you are on vacaiton! >>>> >>>> I uploaded new webrev: >>>> >>>> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.07/ >>>> ?? Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/77243b1dcbfe >>>> >>>> c'tor of GetSingleStackTraceClosure has jthread argument in this webrev. >>>> Also it does not contain testcase for GetThreadListStackTraces with all threads, and OneGetThreadListStackTraces would test main thread only. >>> >>> All those changes are fine in principle for me. One nit/suggestion: >>> >>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>> >>> ??544?? jthread _java_thread; >>> >>> elsewhere "java_thread" refers to a JavaThread, so to avoid confusion may I suggest this member be named _jthread. >> >> I uploaded new webrev: >> >> ? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ >> ? Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/ca6263dbdc87 >> >> >>> I'm going to be away for the next couple of days - sorry - but will try to check email on this if I can. >> >> Thanks! >> >> >> Yasumasa >> >> >>> Thanks, >>> David >>> ----- >>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2020/07/07 15:13, David Holmes wrote: >>>>> On 7/07/2020 2:57 pm, Yasumasa Suenaga wrote: >>>>>> Hi David, >>>>>> >>>>>> On 2020/07/07 11:31, David Holmes wrote: >>>>>>> Hi Yasumasa, >>>>>>> >>>>>>> Hard to keep up with the changes - especially without incremental webrevs. >>>>>> >>>>>> Sorry, I will upload diff from previous webrev in the next. >>>>>> >>>>>> >>>>>>> If GetSingleStackTraceClosure also took the jthread as a constructor arg, then you wouldn't need to recreate a JNI local handle when calling _collector.fill_frames. It's a small simplification and not essential at this stage. >>>>>> >>>>>> I think we should get jthread from an argument of do_thread() because do_thread() would pass the thread which are stopped certainly. >>>>>> It might be simplification if we pass _calling_thread to MultipleStackTracesCollector. `jthread` is only needed to store jvmtiStackInfo.thread . What do you think? >>>>> >>>>> I'm not quite sure what you mean. >>>>> >>>>> I think there is a bit of a design wart with direct handshakes in that do_thread takes the target JavaThread as an argument. That's useful in a case where you want a HandshakeClosure that can be applied to multiple threads, but that's not typically what is needed with direct handshakes - there is only a single target. With a single-target HandshakeClosure you can capture all the "target" information for the operation in the closure instance. So if the actual do_thread operation wants the jthread corresponding to the target thread then we can store that in the closure rather than recomputing it (you could assert it is the same but that seems overkill to me). >>>>> >>>>>> >>>>>>> For the test ... I don't see how Java_GetThreadListStackTraces_checkCallStacks is a valid test. It gets the stacks of all live threads, then uses that information to use GetThreadListStackTraces to get the stack for the same set of threads through a different API. It then compares the two sets of stacks for each thread expecting them to be the same, but that need only be the case for the main thread. Other threads could potentially have a different stack (e.g. if this test is run with JFR enabled there will be additional threads found.) Further I would have expected that there already exist tests that check that, for a given thread (which may be suspended or known to be blocked) the same stack is found through the two different APIs. >>>>>> >>>>>> vmTestbase/nsk/jvmti/unit/GetAllStackTraces/getallstktr001 would check all of threads via GetThreadListStackTraces() and GetAllStackTraces(), so we might be able to remove GetThreadListStackTraces.java from this webrev. >>>>> >>>>> Yes. The existing test only examines a set of test threads that are all blocked on a raw monitor. You do not need to duplicate that test. >>>>> >>>>>> OTOH we don't have testcase for GetThreadListStackTraces() with thread_count == 1, so we need to testcase for it (it is OneGetThreadListStackTraces.java) It would check whether the state of target thread is "waiting" before JNI call to call GetThreadListStackTraces(), >>>>> >>>>> Yes we need to test the special cases introduced by your changes - totally agree - and OneGetThreadListStackTraces.java is a good test for that. >>>>> >>>>>> and also I expect it would not be run with JFR. (it is not described @run) >>>>> >>>>> The arguments to run with JFR (or a bunch of other things) can be passed to the jtreg test harness to be applied to all tests. >>>>> >>>>>> Of course we can check GetThreadListStackTraces() with main thread, but it is not the test for direct handshake for other thread. >>>>> >>>>> Right - that test already exists as per the above. >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>> On 6/07/2020 11:29 pm, Yasumasa Suenaga wrote: >>>>>>>> Hi Serguei, >>>>>>>> >>>>>>>> Thanks for your comment! >>>>>>>> >>>>>>>> I think C++ is more simple to implement the test agent as you said. >>>>>>>> So I implement it in C++ in new webrev. Could you review again? >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.06/ >>>>>>>> >>>>>>>> Also I refactored libGetThreadListStackTraces.cpp, and I've kept exception check after IsSameObject(). >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2020/07/06 16:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>> Hi Yasumasa, >>>>>>>>> >>>>>>>>> Thank you for the update. >>>>>>>>> I think, a pending exception after IsSameObject needs to be checked. >>>>>>>>> >>>>>>>>> The checkStackInfo() needs one more refactoring as I've already suggested. >>>>>>>>> The body of the loop at L68-L78 should be converted to a function check_frame_info. >>>>>>>>> The si1->frame_buffer[i] and si2->frame_buffer[i] will be passed as fi1 and fi2. >>>>>>>>> The index can be passed as well. >>>>>>>>> I'm still suggesting to simplify the local exception_msg to something shorter like err_msg or exc_msg. >>>>>>>>> >>>>>>>>> I'm not sure using fatal is right here: >>>>>>>>> >>>>>>>>> This fragment looks strange: >>>>>>>>> >>>>>>>>> ? 152???? if ((*env)->IsSameObject(env, stack_info[i].thread, thread)) { >>>>>>>>> ? 153?????? target_info = &stack_info[i]; >>>>>>>>> ? 154?????? break; >>>>>>>>> ? 155???? } else if ((*env)->ExceptionOccurred(env)) { >>>>>>>>> ? 156?????? (*env)->ExceptionDescribe(env); >>>>>>>>> ? 157?????? (*env)->FatalError(env, __FILE__); >>>>>>>>> ? 158???? } >>>>>>>>> >>>>>>>>> I expected it to be: >>>>>>>>> >>>>>>>>> ??? jboolean same = (*env)->IsSameObject(env, stack_info[i].thread, thread); >>>>>>>>> ??? if ((*env)->ExceptionOccurred(env)) { >>>>>>>>> ????? (*env)->ExceptionDescribe(env); >>>>>>>>> ????? (*env)->FatalError(env, __FILE__); >>>>>>>>> ??? } >>>>>>>>> ??? if (same) { >>>>>>>>> ????? target_info = &stack_info[i]; >>>>>>>>> ????? break; >>>>>>>>> ??? } >>>>>>>>> >>>>>>>>> Would it better to port this agent to C++ to simplify this code nicer? >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Serguei >>>>>>>>> >>>>>>>>> >>>>>>>>> On 7/5/20 06:13, Yasumasa Suenaga wrote: >>>>>>>>>> Hi Serguei, >>>>>>>>>> >>>>>>>>>> Thanks for your comment! >>>>>>>>>> I refactored testcase. Could you review again? >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.05/ >>>>>>>>>> >>>>>>>>>> It would check Java exception after IsSameObject() call. Does it need? >>>>>>>>>> Any exceptions are not described in JNI document[1], and JNI implementation (jni_IsSameObject()) does not seem to throw it. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> [1] https://docs.oracle.com/en/java/javase/14/docs/specs/jni/functions.html#issameobject >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2020/07/05 14:46, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Okay, thanks. >>>>>>>>>>> Then I'm okay to keep the GetSingleStackTraceClosure. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c.html >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c.html >>>>>>>>>>> >>>>>>>>>>> I'm not sure the function 'is_same_thread() is needed. >>>>>>>>>>> Why do not use the JNI IsSameObject instead? >>>>>>>>>>> >>>>>>>>>>> It seems to be a typo at L132 and L137. >>>>>>>>>>> You, probably. did not want to print the same information for stack_info_1[i].frame_buffer[j].XXX twice. >>>>>>>>>>> >>>>>>>>>>> The code at lines 112-142 is not readable. >>>>>>>>>>> I'd suggest to make a couple of refactoring steps. >>>>>>>>>>> >>>>>>>>>>> First step to simplify this a little bit would be with some renaming and getting rid of indexes: >>>>>>>>>>> >>>>>>>>>>> ?? 71 char err_msg[EXCEPTION_MSG_LEN] = {0}; >>>>>>>>>>> ??... >>>>>>>>>>> ??112?? /* Iterate all jvmtiStackInfo to check */ >>>>>>>>>>> ??113?? for (i = 0; i < num_threads, *exception_msg != '\0'; i++) { >>>>>>>>>>> ????????? jvmtiStackInfo *si1 = stack_info_1[i]; >>>>>>>>>>> ????????? jvmtiStackInfo *si2 = stack_info_2[i]; >>>>>>>>>>> ??114???? if (!IsSameObject(env, si1.thread, si2.thread)) { /* jvmtiStackInfo::thread */ >>>>>>>>>>> ??115?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>> ??116??????????????? "thread[%d] is different: stack_info_1 = %p, stack_info_2 = %p", >>>>>>>>>>> ??117??????????????? i, sinfo1.thread, sinfo2.thread); >>>>>>>>>>> ??118???? } else if (si1.state != si2.state) { /* jvmtiStackInfo::state */ >>>>>>>>>>> ??119?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>> ??120??????????????? "state[%d] is different: stack_info_1 = %d, stack_info_2 = %d", >>>>>>>>>>> ??121??????????????? i, si1.state, si2.state); >>>>>>>>>>> ??122???? } else if (si1.frame_count != si2.frame_count) { /* jvmtiStackInfo::frame_count */ >>>>>>>>>>> ??123?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>> ??124??????????????? "frame_count[%d] is different: stack_info_1 = %d, stack_info_2 = %d", >>>>>>>>>>> ??125??????????????? i, si1.frame_count, si2.frame_count); >>>>>>>>>>> ??126???? } else { >>>>>>>>>>> ??127?????? /* Iterate all jvmtiFrameInfo to check */ >>>>>>>>>>> ??128?????? for (j = 0; j < si1.frame_count; j++) { >>>>>>>>>>> ??129???????? if (si1.frame_buffer[j].method != si1.frame_buffer[j].method) { /* jvmtiFrameInfo::method */ >>>>>>>>>>> ??130?????????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>> ??131??????????????????? "thread [%d] frame_buffer[%d].method is different: stack_info_1 = %lx, stack_info_2 = %lx", >>>>>>>>>>> ??132??????????????????? i, j, si1.frame_buffer[j].method, si2.frame_buffer[j].method); >>>>>>>>>>> ??133?????????? break; >>>>>>>>>>> ??134???????? } else if (si1.frame_buffer[j].location != si1.frame_buffer[j].location) { /* jvmtiFrameInfo::location */ >>>>>>>>>>> ??135?????????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>> ??136??????????????????? "thread [%d] frame_buffer[%d].location is different: stack_info_1 = %ld, stack_info_2 = %ld", >>>>>>>>>>> ??137??????????????????? i, j, si1.frame_buffer[j].location, si2.frame_buffer[j].location); >>>>>>>>>>> ??138?????????? break; >>>>>>>>>>> ??139???????? } >>>>>>>>>>> ??140?????? } >>>>>>>>>>> ??141???? } >>>>>>>>>>> ??142?? } >>>>>>>>>>> >>>>>>>>>>> Another step would be to create functions that implement a body of each loop. >>>>>>>>>>> You can use the same techniques to simplify similar place (L127-L138) in the libOneGetThreadListStackTraces.c. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Serguei >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 7/3/20 15:55, Yasumasa Suenaga wrote: >>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>> >>>>>>>>>>>> I'm not an Oracle employee, so I cannot know real request(s) from your customers. >>>>>>>>>>>> However JDK-8201641 says Dynatrace has requested this enhancement. >>>>>>>>>>>> >>>>>>>>>>>> BTW I haven't heared any request from my customers about this. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 2020/07/04 4:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>> >>>>>>>>>>>>> This difference is not that big to care about. >>>>>>>>>>>>> I feel this is really rare case and so, does not worth these complications. >>>>>>>>>>>>> Do we have a real request from customers to optimize it? >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Serguei >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 7/3/20 01:16, Yasumasa Suenaga wrote: >>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Generally I agree with you, but I have concern about the difference of the result of GetStackTrace() and GetThreadListStackTraces(). >>>>>>>>>>>>>> >>>>>>>>>>>>>> ? GetStackTrace: jvmtiFrameInfo >>>>>>>>>>>>>> ? GetThreadListStackTraces: jvmtiStackInfo >>>>>>>>>>>>>> >>>>>>>>>>>>>> jvmtiStackInfo contains thread state, and it is ensured it is the state of the call stack. >>>>>>>>>>>>>> If we want to get both call stack and thread state, we need to suspend target thread, and call both GetStackTrace() and GetThreadState(). Is it ok? >>>>>>>>>>>>>> >>>>>>>>>>>>>> I was wondering if JDK-8201641 (parent ticket of this change) needed them for profiling (dynatrace?) >>>>>>>>>>>>>> If it is responsibility of JVMTI agent implementor, I remove this closure. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 2020/07/03 16:45, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> After some thinking I've concluded that I do not like this optimization >>>>>>>>>>>>>>> of the GetThreadListStackTraces with GetSingleStackTraceClosure. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> We may need more opinions on this but these are my points: >>>>>>>>>>>>>>> ??- it adds some complexity and ugliness >>>>>>>>>>>>>>> ??- a win is doubtful because it has to be a rare case, so that total overhead should not be high >>>>>>>>>>>>>>> ??- if it is really high for some use cases then it is up to the user >>>>>>>>>>>>>>> ??? to optimize it with using GetStackTrace instead >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> In such cases with doubtful overhead I usually prefer the simplicity. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Good examples where it makes sense to optimize are checks for target thread to be current thread. >>>>>>>>>>>>>>> In such cases there is no need to suspend the target thread, or use a VMop/HandshakeClosure. >>>>>>>>>>>>>>> For instance, please, see the Monitor functions with the check: (java_thread == calling_thread). >>>>>>>>>>>>>>> Getting information for current thread is frequently used case, e.g. to get info at an event point. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 7/2/20 23:29, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>> Hi Dan, David, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I uploaded new webrev. Could you review again? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/ >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> OneGetThreadListStackTraces.java in this webrev would wait until thread state is transited to "waiting" with spin wait. >>>>>>>>>>>>>>>> CountDownLatch::await call as Dan pointed is fixed in it :) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Diff from webrev.03: >>>>>>>>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/c9aeb7001e50 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 2020/07/03 14:15, David Holmes wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>> On 2020/07/03 12:24, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>>> On 7/2/20 10:50 PM, David Holmes wrote: >>>>>>>>>>>>>>>>>>>> Sorry I'm responding here without seeing latest webrev but there is enough context I think ... >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>> Hi Dan, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>>>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I upload new webrev. Could you review again? >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>>>>>>>>>>>>>> ???? L1542: ??? // Get stack trace with handshake >>>>>>>>>>>>>>>>>>>>>> ???????? nit - please add a period at the end. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> L1591: *stack_info_ptr = op.stack_info(); >>>>>>>>>>>>>>>>>>>>>> ???????? The return parameter should not be touched unless the return >>>>>>>>>>>>>>>>>>>>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> ???? old L1582: ? if (err == JVMTI_ERROR_NONE) { >>>>>>>>>>>>>>>>>>>>>> ???????? Please restore this check. The return parameter should not >>>>>>>>>>>>>>>>>>>>>> ???????? be touched unless the return code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> But op.stack_info() will return NULL if the error is not JVMTI_ERROR_NONE. Are you (Dan) concerned about someone passing in a non-null/initialized out-pointer that will be reset to NULL if there was an error? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Actually the way we used to test this in POSIX tests is to call >>>>>>>>>>>>>>>>>>> an API with known bad parameters and the return parameter ptr >>>>>>>>>>>>>>>>>>> set to NULL. If the return parameter ptr was touched when an >>>>>>>>>>>>>>>>>>> error should have been detected on an earlier parameter, then >>>>>>>>>>>>>>>>>>> the test failed. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> L1272: ? if (!jt->is_exiting() && (thread_oop != NULL)) { >>>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>> ???? old L1532: ? _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>>>>>>>>>>>>>>>>>>> ???????? This deletion of the _result field threw me for a minute and then >>>>>>>>>>>>>>>>>>>>>> ???? ? ? I figured out that the field is init to JVMTI_ERROR_THREAD_NOT_ALIVE >>>>>>>>>>>>>>>>>>>>>> ???????? in the constructor. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> ???? L1553: ? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>>> ???? L64: startSignal.countDown(); >>>>>>>>>>>>>>>>>>>>>> ???????? I was expecting this to be a call to await() instead of >>>>>>>>>>>>>>>>>>>>>> ???????? countDown(). What am I missing here? >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> ???????? I think this test might be passing by accident right now, but... >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Main thread (which call JVMTI functions to test) should wait until test thread is ready. >>>>>>>>>>>>>>>>>>>>> So main thread would wait startSignal, and test thread would count down. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> No! >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> The test thread that previously called obj.wait() now calls latch.await(). >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> The main thread that previously called obj.notify() now calls latch.countDown(). >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> The main thread continues to spin until it sees the target is WAITING before proceeding with the test. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> If I add spin wait to wait until transit target thread state is WAITING (as following), we don't need to call SuspendThread(). >>>>>>>>>>>>>>>>>> Which is better? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> The original spin-wait loop checking for? WAITING is better because it is the only guarantee that the target thread is blocked where you need it to be. suspending the thread is racy as you don't know exactly where the suspend will hit. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>> /* Wait until the thread state transits to "waiting" */ >>>>>>>>>>>>>>>>>> while (th.getState() != Thread.State.WAITING) { >>>>>>>>>>>>>>>>>> ???? Thread.onSpinWait(); >>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> For simplify, spin wait is prefer to OneGetThreadListStackTraces.java in webrev.03. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Here's the flow as I see it: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>> ?? - start worker thread >>>>>>>>>>>>>>>>>>> ?? - startSignal.await() >>>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>>> ?? - startSignal.countDown() >>>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>>> ?? - stopSignal.await() >>>>>>>>>>>>>>>>>>> ???? - worker is now blocked >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>> ?? - checkCallStacks(th) >>>>>>>>>>>>>>>>>>> ?? - stopSignal.countDown() >>>>>>>>>>>>>>>>>>> ???? - worker is now unblocked >>>>>>>>>>>>>>>>>>> ?? - th.join >>>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>>> ?? - runs off the end of run() >>>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>> ?? - run off the end of main() >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>>> ???? L92: ? jthreads = (jthread *)malloc(sizeof(jthread) * num_threads); >>>>>>>>>>>>>>>>>>>>>> ???????? You don't check for malloc() failure. >>>>>>>>>>>>>>>>>>>>>> ???????? 'jthreads' is allocated but never freed. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>>> ???? L91: ? result = (*jvmti)->SuspendThread(jvmti, thread); >>>>>>>>>>>>>>>>>>>>>> ???????? Why are you suspending the thread? GetAllStackTraces() and >>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() do not require the target thread(s) >>>>>>>>>>>>>>>>>>>>>> ???????? to be suspend. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> ???????? If you decide not to SuspendThread, then you don't need the >>>>>>>>>>>>>>>>>>>>>> ???????? AddCapabilities or the ResumeThread calls. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Test thread might not be entered following code (stopSignal.await()). We might see deferent call stack between GetAllStackTraces() and GetThreadListStackTraces(). We cannot control to freeze call stack of test thread in Java code. >>>>>>>>>>>>>>>>>>>>> (I didn't use SuspendThread() at first, but I saw some errors which causes in above.) >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> So we need to call SuspendThread() to ensure we can see same call stack. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> If you are checking that the thread is in state WAITING then it cannot escape from that state and you can sample the stack multiple times from any API and get the same result. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I suspect the errors you saw were from the apparent incorrect use of the CountDownLatch. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> With the flow outlined above, the worker thread should be >>>>>>>>>>>>>>>>>>> nicely blocked in stopSignal.await() when stuff is sampled. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could review again? >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Updates look fine - thanks. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> One minor nit: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> 1274 _collector.allocate_and_fill_stacks(1); >>>>>>>>>>>>>>>>>>>>>>>> 1275 _collector.set_result(JVMTI_ERROR_NONE); >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> In the other places where you use _collector you rely on result being initialized to JVMTI_ERROR_NONE, rather than setting it directly after allocate_and_fill_stacks(). >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 823 "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> This function (JvmtiEnvBase::get_stack_trace()) can be called to get own stack trace. For example, we can call GetStackTrace() for current thread at JVMTI event. >>>>>>>>>>>>>>>>>>>>>>>>> So I changed assert as below: >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>> ??820 assert(current_thread == java_thread || >>>>>>>>>>>>>>>>>>>>>>>>> ??821 SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>> ??823????????? "call by myself / at safepoint / at handshake"); >>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Yep good catch. I hope current tests caught that. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> They would be tested in vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ (own call stacks), and getstacktr003 (call stacks in other thread). >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Speaking of tests ... >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> In the native code I think you need to check the success of all JNI methods that can throw exceptions - otherwise I believe the tests may trigger warnings if -Xcheck:jni is used with them. See for example: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I updated testcases to check JNI and JVMTI function calls. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> In the Java code the target thread: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> ?? 45???? public void run() { >>>>>>>>>>>>>>>>>>>>>>>> ?? 46?????? try { >>>>>>>>>>>>>>>>>>>>>>>> ?? 47 synchronized (lock) { >>>>>>>>>>>>>>>>>>>>>>>> ?? 48 lock.wait(); >>>>>>>>>>>>>>>>>>>>>>>> ?? 49 System.out.println("OK"); >>>>>>>>>>>>>>>>>>>>>>>> ?? 50?????????? } >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> is potentially susceptible to spurious wakeups. Using a CountDownLatch would be robust. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> If so, we can call make_local() in L1272 without JavaThread (or we can pass current thread to make_local()). Is it right? >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Sorry I got confused, _calling_thread may not be the current thread as we could be executing the handshake in the target thread itself. So the ResourceMark is correct as-is (implicitly for current thread). >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> The argument to fill_frames will be used in the jvmtiStackInfo and passed back to the _calling_thread, so it must be created via make_local(_calling_thread, ...) as you presently have. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you for reviewing! I will update new webrev tomorrow. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 500 JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??? : >>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> class GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I updated webrev for 8242428. Could you review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change migrate to use direct handshake for GetStackTrace() and GetThreadListStackTraces() (when thread_count == 1). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks really good now! I only have a few nits below. There is one thing I don't like about it but it requires a change to the main Handshake logic to address - in JvmtiEnv::GetThreadListStackTraces you have to create a ThreadsListHandle to convert the jthread to a JavaThread, but then the Handshake::execute_direct creates another ThreadsListHandle internally. That's a waste. I will discuss with Robbin and file a RFE to have an overload of execute_direct that takes an existing TLH. Actually it's worse than that because we have another TLH in use at the entry point for the JVMTI functions, so I think there may be some scope for simplifying the use of TLH instances - future RFE. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??451 GetStackTraceClosure(JvmtiEnv *env, jint start_depth, jint max_count, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??452 jvmtiFrameInfo* frame_buffer, jint* count_ptr) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??454 _env(env), _start_depth(start_depth), _max_count(max_count), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??455 _frame_buffer(frame_buffer), _count_ptr(count_ptr), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??456 _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Nit: can you do one initializer per line please. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??500 JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 481 MultipleStackTracesCollector(JvmtiEnv *env, jint max_frame_count) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??483 _max_frame_count = max_frame_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??484 _frame_count_total = 0; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??485 _head = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??486 _stack_info = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??487 _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As you are touching this can you change it to use an initializer list as you did for the HandshakeClosure, and please keep one item per line. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??823 "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1269 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is frustrating that this entire call chain started with a jthread reference, which we converted to a JavaThread, only to eventually need to convert it back to a jthread! I think there is some scope for simplification here but not as part of this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm still pondering the test logic but wanted to send this now. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetThreadListStackTrace (for GetThreadListStackTraces) and VM_GetAllStackTraces (for GetAllStackTraces) have inherited VM_GetMultipleStackTraces VM operation which provides the feature to generate jvmtiStackInfo. I modified VM_GetMultipleStackTraces to a normal C++ class to share with HandshakeClosure for GetThreadListStackTraces (GetSingleStackTraceClosure). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I added new testcases which test GetThreadListStackTraces() with thread_count == 1 and with all threads. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/nsk/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change replace following VM operations to direct handshake. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces (GetThreadListStackTrace()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTrace() uses direct handshake if thread count == 1. In other case (thread count > 1), it would be performed as VM operation (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Caller of VM_GetCurrentLocation (JvmtiEnvThreadState::reset_current_location()) might be called at safepoint. So I added safepoint check in its caller. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/ns >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I tested it on submit repo, then it has execution error (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) due to dependency error. So I think it does not occur by this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> > From suenaga at oss.nttdata.com Thu Jul 9 02:36:20 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Thu, 9 Jul 2020 11:36:20 +0900 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <238a89a3-06b1-90e4-d5a4-8dc2f1ed12ab@oracle.com> <4c43bc2c-551e-d042-8fa7-b63e755888d1@oracle.com> <8c387550-a06e-6320-f453-0e898486befb@oss.nttdata.com> <19cfa2a7-f441-425d-5623-1c11c6a870b6@oss.nttdata.com> <237e8d26-d185-1008-6e07-5b9f36314534@oracle.com> Message-ID: On 2020/07/09 9:25, Yasumasa Suenaga wrote: > Hi Dan, > > Thanks for your comment! > I uploaded new webrev: > > ? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.09/ > ? Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/5d167adf8524 Submit repo reported some build errors on macOS. Can you share details? mach5-one-ysuenaga-JDK-8242428-20200709-0029-12486625 Thanks, Yasumasa > I saw similar build errors in libOneGetThreadListStackTraces.cpp on Windows. > This webrev fixes them. > > > Thanks, > > Yasumasa > > > On 2020/07/09 1:42, Daniel D. Daugherty wrote: >> ?> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ >> >> src/hotspot/share/prims/jvmtiEnv.cpp >> ???? No comments. >> >> src/hotspot/share/prims/jvmtiEnvBase.cpp >> ???? L1159: ? Thread *current_thread = Thread::current(); >> ???????? Please add "#ifdef ASSERT" above and "#endif" below since >> ???????? current_thread is only used for the assert() in this function. >> >> src/hotspot/share/prims/jvmtiEnvBase.hpp >> ???? L549: ???????????????????????????? jthread java_thread, jint max_frame_count) >> ???? L552: ????? _jthread(java_thread), >> ???????? Please: s/java_thread/thread/ on both lines. >> >> src/hotspot/share/runtime/vmOperations.hpp >> ???? No comments. >> >> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >> ???? No comments. >> >> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >> ???? L27: #include >> ???????? This include is out of order; should be first in the list. >> >> ???? This file doesn't compile on my MBP13: >> >> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:49:14: error: format specifies type 'unsigned long' but the argument has type 'jmethodID' (aka '_jmethodID *') [-Werror,-Wformat] >> ????????????? fi1->method, fi2->method); >> ????????????? ^~~~~~~~~~~ >> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:49:27: error: format specifies type 'unsigned long' but the argument has type 'jmethodID' (aka '_jmethodID *') [-Werror,-Wformat] >> ????????????? fi1->method, fi2->method); >> ?????????????????????????? ^~~~~~~~~~~ >> ???? 2 errors generated. >> >> ???? This change made it compile on my MBP13, but that may break it on >> ???? other platforms: >> >> ???? $ hg diff >> ???? diff -r 560847c69fbe test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >> ???? --- a/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp Wed Jul 08 12:13:32 2020 -0400 >> ???? +++ b/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp Wed Jul 08 12:40:42 2020 -0400 >> ???? @@ -46,7 +46,7 @@ >> ?? ?? ? if (fi1->method != fi2->method) { /* jvmtiFrameInfo::method */ >> ?? ?? ??? snprintf(err_msg, sizeof(err_msg), >> ?? ? ? ??????????? "method is different: fi1 = %lx, fi2 = %lx", >> ???? -???????????? fi1->method, fi2->method); >> ???? +???????????? (unsigned long) fi1->method, (unsigned long) fi2->method); >> ?? ?? ??? env->FatalError(err_msg); >> ?? ?? ? } else if (fi1->location != fi2->location) { /* jvmtiFrameInfo::location */ >> ?? ?? ??? snprintf(err_msg, sizeof(err_msg), >> >> ???? I'm not sure of the right platform independent way to output >> ???? the 'method' field. >> >> Dan >> >> >> On 7/8/20 4:04 AM, Yasumasa Suenaga wrote: >>> Hi David, >>> >>> On 2020/07/08 15:27, David Holmes wrote: >>>> Hi Yasumasa, >>>> >>>> On 7/07/2020 6:54 pm, Yasumasa Suenaga wrote: >>>>> Hi David, Serguei, >>>>> >>>>> Serguei, thank you for replying even though you are on vacaiton! >>>>> >>>>> I uploaded new webrev: >>>>> >>>>> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.07/ >>>>> ?? Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/77243b1dcbfe >>>>> >>>>> c'tor of GetSingleStackTraceClosure has jthread argument in this webrev. >>>>> Also it does not contain testcase for GetThreadListStackTraces with all threads, and OneGetThreadListStackTraces would test main thread only. >>>> >>>> All those changes are fine in principle for me. One nit/suggestion: >>>> >>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>> >>>> ??544?? jthread _java_thread; >>>> >>>> elsewhere "java_thread" refers to a JavaThread, so to avoid confusion may I suggest this member be named _jthread. >>> >>> I uploaded new webrev: >>> >>> ? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ >>> ? Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/ca6263dbdc87 >>> >>> >>>> I'm going to be away for the next couple of days - sorry - but will try to check email on this if I can. >>> >>> Thanks! >>> >>> >>> Yasumasa >>> >>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2020/07/07 15:13, David Holmes wrote: >>>>>> On 7/07/2020 2:57 pm, Yasumasa Suenaga wrote: >>>>>>> Hi David, >>>>>>> >>>>>>> On 2020/07/07 11:31, David Holmes wrote: >>>>>>>> Hi Yasumasa, >>>>>>>> >>>>>>>> Hard to keep up with the changes - especially without incremental webrevs. >>>>>>> >>>>>>> Sorry, I will upload diff from previous webrev in the next. >>>>>>> >>>>>>> >>>>>>>> If GetSingleStackTraceClosure also took the jthread as a constructor arg, then you wouldn't need to recreate a JNI local handle when calling _collector.fill_frames. It's a small simplification and not essential at this stage. >>>>>>> >>>>>>> I think we should get jthread from an argument of do_thread() because do_thread() would pass the thread which are stopped certainly. >>>>>>> It might be simplification if we pass _calling_thread to MultipleStackTracesCollector. `jthread` is only needed to store jvmtiStackInfo.thread . What do you think? >>>>>> >>>>>> I'm not quite sure what you mean. >>>>>> >>>>>> I think there is a bit of a design wart with direct handshakes in that do_thread takes the target JavaThread as an argument. That's useful in a case where you want a HandshakeClosure that can be applied to multiple threads, but that's not typically what is needed with direct handshakes - there is only a single target. With a single-target HandshakeClosure you can capture all the "target" information for the operation in the closure instance. So if the actual do_thread operation wants the jthread corresponding to the target thread then we can store that in the closure rather than recomputing it (you could assert it is the same but that seems overkill to me). >>>>>> >>>>>>> >>>>>>>> For the test ... I don't see how Java_GetThreadListStackTraces_checkCallStacks is a valid test. It gets the stacks of all live threads, then uses that information to use GetThreadListStackTraces to get the stack for the same set of threads through a different API. It then compares the two sets of stacks for each thread expecting them to be the same, but that need only be the case for the main thread. Other threads could potentially have a different stack (e.g. if this test is run with JFR enabled there will be additional threads found.) Further I would have expected that there already exist tests that check that, for a given thread (which may be suspended or known to be blocked) the same stack is found through the two different APIs. >>>>>>> >>>>>>> vmTestbase/nsk/jvmti/unit/GetAllStackTraces/getallstktr001 would check all of threads via GetThreadListStackTraces() and GetAllStackTraces(), so we might be able to remove GetThreadListStackTraces.java from this webrev. >>>>>> >>>>>> Yes. The existing test only examines a set of test threads that are all blocked on a raw monitor. You do not need to duplicate that test. >>>>>> >>>>>>> OTOH we don't have testcase for GetThreadListStackTraces() with thread_count == 1, so we need to testcase for it (it is OneGetThreadListStackTraces.java) It would check whether the state of target thread is "waiting" before JNI call to call GetThreadListStackTraces(), >>>>>> >>>>>> Yes we need to test the special cases introduced by your changes - totally agree - and OneGetThreadListStackTraces.java is a good test for that. >>>>>> >>>>>>> and also I expect it would not be run with JFR. (it is not described @run) >>>>>> >>>>>> The arguments to run with JFR (or a bunch of other things) can be passed to the jtreg test harness to be applied to all tests. >>>>>> >>>>>>> Of course we can check GetThreadListStackTraces() with main thread, but it is not the test for direct handshake for other thread. >>>>>> >>>>>> Right - that test already exists as per the above. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> ----- >>>>>>>> >>>>>>>> On 6/07/2020 11:29 pm, Yasumasa Suenaga wrote: >>>>>>>>> Hi Serguei, >>>>>>>>> >>>>>>>>> Thanks for your comment! >>>>>>>>> >>>>>>>>> I think C++ is more simple to implement the test agent as you said. >>>>>>>>> So I implement it in C++ in new webrev. Could you review again? >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.06/ >>>>>>>>> >>>>>>>>> Also I refactored libGetThreadListStackTraces.cpp, and I've kept exception check after IsSameObject(). >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2020/07/06 16:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>> Hi Yasumasa, >>>>>>>>>> >>>>>>>>>> Thank you for the update. >>>>>>>>>> I think, a pending exception after IsSameObject needs to be checked. >>>>>>>>>> >>>>>>>>>> The checkStackInfo() needs one more refactoring as I've already suggested. >>>>>>>>>> The body of the loop at L68-L78 should be converted to a function check_frame_info. >>>>>>>>>> The si1->frame_buffer[i] and si2->frame_buffer[i] will be passed as fi1 and fi2. >>>>>>>>>> The index can be passed as well. >>>>>>>>>> I'm still suggesting to simplify the local exception_msg to something shorter like err_msg or exc_msg. >>>>>>>>>> >>>>>>>>>> I'm not sure using fatal is right here: >>>>>>>>>> >>>>>>>>>> This fragment looks strange: >>>>>>>>>> >>>>>>>>>> ? 152???? if ((*env)->IsSameObject(env, stack_info[i].thread, thread)) { >>>>>>>>>> ? 153?????? target_info = &stack_info[i]; >>>>>>>>>> ? 154?????? break; >>>>>>>>>> ? 155???? } else if ((*env)->ExceptionOccurred(env)) { >>>>>>>>>> ? 156?????? (*env)->ExceptionDescribe(env); >>>>>>>>>> ? 157?????? (*env)->FatalError(env, __FILE__); >>>>>>>>>> ? 158???? } >>>>>>>>>> >>>>>>>>>> I expected it to be: >>>>>>>>>> >>>>>>>>>> ??? jboolean same = (*env)->IsSameObject(env, stack_info[i].thread, thread); >>>>>>>>>> ??? if ((*env)->ExceptionOccurred(env)) { >>>>>>>>>> ????? (*env)->ExceptionDescribe(env); >>>>>>>>>> ????? (*env)->FatalError(env, __FILE__); >>>>>>>>>> ??? } >>>>>>>>>> ??? if (same) { >>>>>>>>>> ????? target_info = &stack_info[i]; >>>>>>>>>> ????? break; >>>>>>>>>> ??? } >>>>>>>>>> >>>>>>>>>> Would it better to port this agent to C++ to simplify this code nicer? >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Serguei >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 7/5/20 06:13, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi Serguei, >>>>>>>>>>> >>>>>>>>>>> Thanks for your comment! >>>>>>>>>>> I refactored testcase. Could you review again? >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.05/ >>>>>>>>>>> >>>>>>>>>>> It would check Java exception after IsSameObject() call. Does it need? >>>>>>>>>>> Any exceptions are not described in JNI document[1], and JNI implementation (jni_IsSameObject()) does not seem to throw it. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> [1] https://docs.oracle.com/en/java/javase/14/docs/specs/jni/functions.html#issameobject >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2020/07/05 14:46, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Okay, thanks. >>>>>>>>>>>> Then I'm okay to keep the GetSingleStackTraceClosure. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c.html >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c.html >>>>>>>>>>>> >>>>>>>>>>>> I'm not sure the function 'is_same_thread() is needed. >>>>>>>>>>>> Why do not use the JNI IsSameObject instead? >>>>>>>>>>>> >>>>>>>>>>>> It seems to be a typo at L132 and L137. >>>>>>>>>>>> You, probably. did not want to print the same information for stack_info_1[i].frame_buffer[j].XXX twice. >>>>>>>>>>>> >>>>>>>>>>>> The code at lines 112-142 is not readable. >>>>>>>>>>>> I'd suggest to make a couple of refactoring steps. >>>>>>>>>>>> >>>>>>>>>>>> First step to simplify this a little bit would be with some renaming and getting rid of indexes: >>>>>>>>>>>> >>>>>>>>>>>> ?? 71 char err_msg[EXCEPTION_MSG_LEN] = {0}; >>>>>>>>>>>> ??... >>>>>>>>>>>> ??112?? /* Iterate all jvmtiStackInfo to check */ >>>>>>>>>>>> ??113?? for (i = 0; i < num_threads, *exception_msg != '\0'; i++) { >>>>>>>>>>>> ????????? jvmtiStackInfo *si1 = stack_info_1[i]; >>>>>>>>>>>> ????????? jvmtiStackInfo *si2 = stack_info_2[i]; >>>>>>>>>>>> ??114???? if (!IsSameObject(env, si1.thread, si2.thread)) { /* jvmtiStackInfo::thread */ >>>>>>>>>>>> ??115?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>> ??116??????????????? "thread[%d] is different: stack_info_1 = %p, stack_info_2 = %p", >>>>>>>>>>>> ??117??????????????? i, sinfo1.thread, sinfo2.thread); >>>>>>>>>>>> ??118???? } else if (si1.state != si2.state) { /* jvmtiStackInfo::state */ >>>>>>>>>>>> ??119?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>> ??120??????????????? "state[%d] is different: stack_info_1 = %d, stack_info_2 = %d", >>>>>>>>>>>> ??121??????????????? i, si1.state, si2.state); >>>>>>>>>>>> ??122???? } else if (si1.frame_count != si2.frame_count) { /* jvmtiStackInfo::frame_count */ >>>>>>>>>>>> ??123?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>> ??124??????????????? "frame_count[%d] is different: stack_info_1 = %d, stack_info_2 = %d", >>>>>>>>>>>> ??125??????????????? i, si1.frame_count, si2.frame_count); >>>>>>>>>>>> ??126???? } else { >>>>>>>>>>>> ??127?????? /* Iterate all jvmtiFrameInfo to check */ >>>>>>>>>>>> ??128?????? for (j = 0; j < si1.frame_count; j++) { >>>>>>>>>>>> ??129???????? if (si1.frame_buffer[j].method != si1.frame_buffer[j].method) { /* jvmtiFrameInfo::method */ >>>>>>>>>>>> ??130?????????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>> ??131??????????????????? "thread [%d] frame_buffer[%d].method is different: stack_info_1 = %lx, stack_info_2 = %lx", >>>>>>>>>>>> ??132??????????????????? i, j, si1.frame_buffer[j].method, si2.frame_buffer[j].method); >>>>>>>>>>>> ??133?????????? break; >>>>>>>>>>>> ??134???????? } else if (si1.frame_buffer[j].location != si1.frame_buffer[j].location) { /* jvmtiFrameInfo::location */ >>>>>>>>>>>> ??135?????????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>> ??136??????????????????? "thread [%d] frame_buffer[%d].location is different: stack_info_1 = %ld, stack_info_2 = %ld", >>>>>>>>>>>> ??137??????????????????? i, j, si1.frame_buffer[j].location, si2.frame_buffer[j].location); >>>>>>>>>>>> ??138?????????? break; >>>>>>>>>>>> ??139???????? } >>>>>>>>>>>> ??140?????? } >>>>>>>>>>>> ??141???? } >>>>>>>>>>>> ??142?? } >>>>>>>>>>>> >>>>>>>>>>>> Another step would be to create functions that implement a body of each loop. >>>>>>>>>>>> You can use the same techniques to simplify similar place (L127-L138) in the libOneGetThreadListStackTraces.c. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Serguei >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 7/3/20 15:55, Yasumasa Suenaga wrote: >>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>> >>>>>>>>>>>>> I'm not an Oracle employee, so I cannot know real request(s) from your customers. >>>>>>>>>>>>> However JDK-8201641 says Dynatrace has requested this enhancement. >>>>>>>>>>>>> >>>>>>>>>>>>> BTW I haven't heared any request from my customers about this. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 2020/07/04 4:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>> >>>>>>>>>>>>>> This difference is not that big to care about. >>>>>>>>>>>>>> I feel this is really rare case and so, does not worth these complications. >>>>>>>>>>>>>> Do we have a real request from customers to optimize it? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 7/3/20 01:16, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Generally I agree with you, but I have concern about the difference of the result of GetStackTrace() and GetThreadListStackTraces(). >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ? GetStackTrace: jvmtiFrameInfo >>>>>>>>>>>>>>> ? GetThreadListStackTraces: jvmtiStackInfo >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> jvmtiStackInfo contains thread state, and it is ensured it is the state of the call stack. >>>>>>>>>>>>>>> If we want to get both call stack and thread state, we need to suspend target thread, and call both GetStackTrace() and GetThreadState(). Is it ok? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I was wondering if JDK-8201641 (parent ticket of this change) needed them for profiling (dynatrace?) >>>>>>>>>>>>>>> If it is responsibility of JVMTI agent implementor, I remove this closure. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 2020/07/03 16:45, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> After some thinking I've concluded that I do not like this optimization >>>>>>>>>>>>>>>> of the GetThreadListStackTraces with GetSingleStackTraceClosure. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> We may need more opinions on this but these are my points: >>>>>>>>>>>>>>>> ??- it adds some complexity and ugliness >>>>>>>>>>>>>>>> ??- a win is doubtful because it has to be a rare case, so that total overhead should not be high >>>>>>>>>>>>>>>> ??- if it is really high for some use cases then it is up to the user >>>>>>>>>>>>>>>> ??? to optimize it with using GetStackTrace instead >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> In such cases with doubtful overhead I usually prefer the simplicity. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Good examples where it makes sense to optimize are checks for target thread to be current thread. >>>>>>>>>>>>>>>> In such cases there is no need to suspend the target thread, or use a VMop/HandshakeClosure. >>>>>>>>>>>>>>>> For instance, please, see the Monitor functions with the check: (java_thread == calling_thread). >>>>>>>>>>>>>>>> Getting information for current thread is frequently used case, e.g. to get info at an event point. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 7/2/20 23:29, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>> Hi Dan, David, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I uploaded new webrev. Could you review again? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/ >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> OneGetThreadListStackTraces.java in this webrev would wait until thread state is transited to "waiting" with spin wait. >>>>>>>>>>>>>>>>> CountDownLatch::await call as Dan pointed is fixed in it :) >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Diff from webrev.03: >>>>>>>>>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/c9aeb7001e50 >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 2020/07/03 14:15, David Holmes wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>> On 2020/07/03 12:24, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>>>> On 7/2/20 10:50 PM, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>> Sorry I'm responding here without seeing latest webrev but there is enough context I think ... >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>> Hi Dan, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>>>>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> I upload new webrev. Could you review again? >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>>>>>>>>>>>>>>> ???? L1542: ??? // Get stack trace with handshake >>>>>>>>>>>>>>>>>>>>>>> ???????? nit - please add a period at the end. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> L1591: *stack_info_ptr = op.stack_info(); >>>>>>>>>>>>>>>>>>>>>>> ???????? The return parameter should not be touched unless the return >>>>>>>>>>>>>>>>>>>>>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ???? old L1582: ? if (err == JVMTI_ERROR_NONE) { >>>>>>>>>>>>>>>>>>>>>>> ???????? Please restore this check. The return parameter should not >>>>>>>>>>>>>>>>>>>>>>> ???????? be touched unless the return code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> But op.stack_info() will return NULL if the error is not JVMTI_ERROR_NONE. Are you (Dan) concerned about someone passing in a non-null/initialized out-pointer that will be reset to NULL if there was an error? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Actually the way we used to test this in POSIX tests is to call >>>>>>>>>>>>>>>>>>>> an API with known bad parameters and the return parameter ptr >>>>>>>>>>>>>>>>>>>> set to NULL. If the return parameter ptr was touched when an >>>>>>>>>>>>>>>>>>>> error should have been detected on an earlier parameter, then >>>>>>>>>>>>>>>>>>>> the test failed. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> L1272: ? if (!jt->is_exiting() && (thread_oop != NULL)) { >>>>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>> ???? old L1532: ? _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>>>>>>>>>>>>>>>>>>>> ???????? This deletion of the _result field threw me for a minute and then >>>>>>>>>>>>>>>>>>>>>>> ???? ? ? I figured out that the field is init to JVMTI_ERROR_THREAD_NOT_ALIVE >>>>>>>>>>>>>>>>>>>>>>> ???????? in the constructor. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ???? L1553: ? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>>>> ???? L64: startSignal.countDown(); >>>>>>>>>>>>>>>>>>>>>>> ???????? I was expecting this to be a call to await() instead of >>>>>>>>>>>>>>>>>>>>>>> ???????? countDown(). What am I missing here? >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ???????? I think this test might be passing by accident right now, but... >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Main thread (which call JVMTI functions to test) should wait until test thread is ready. >>>>>>>>>>>>>>>>>>>>>> So main thread would wait startSignal, and test thread would count down. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> No! >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> The test thread that previously called obj.wait() now calls latch.await(). >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> The main thread that previously called obj.notify() now calls latch.countDown(). >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> The main thread continues to spin until it sees the target is WAITING before proceeding with the test. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> If I add spin wait to wait until transit target thread state is WAITING (as following), we don't need to call SuspendThread(). >>>>>>>>>>>>>>>>>>> Which is better? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> The original spin-wait loop checking for? WAITING is better because it is the only guarantee that the target thread is blocked where you need it to be. suspending the thread is racy as you don't know exactly where the suspend will hit. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>> /* Wait until the thread state transits to "waiting" */ >>>>>>>>>>>>>>>>>>> while (th.getState() != Thread.State.WAITING) { >>>>>>>>>>>>>>>>>>> ???? Thread.onSpinWait(); >>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> For simplify, spin wait is prefer to OneGetThreadListStackTraces.java in webrev.03. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Here's the flow as I see it: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>> ?? - start worker thread >>>>>>>>>>>>>>>>>>>> ?? - startSignal.await() >>>>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>>>> ?? - startSignal.countDown() >>>>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>>>> ?? - stopSignal.await() >>>>>>>>>>>>>>>>>>>> ???? - worker is now blocked >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>> ?? - checkCallStacks(th) >>>>>>>>>>>>>>>>>>>> ?? - stopSignal.countDown() >>>>>>>>>>>>>>>>>>>> ???? - worker is now unblocked >>>>>>>>>>>>>>>>>>>> ?? - th.join >>>>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>>>> ?? - runs off the end of run() >>>>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>> ?? - run off the end of main() >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>>>> ???? L92: ? jthreads = (jthread *)malloc(sizeof(jthread) * num_threads); >>>>>>>>>>>>>>>>>>>>>>> ???????? You don't check for malloc() failure. >>>>>>>>>>>>>>>>>>>>>>> ???????? 'jthreads' is allocated but never freed. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>>>> ???? L91: ? result = (*jvmti)->SuspendThread(jvmti, thread); >>>>>>>>>>>>>>>>>>>>>>> ???????? Why are you suspending the thread? GetAllStackTraces() and >>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() do not require the target thread(s) >>>>>>>>>>>>>>>>>>>>>>> ???????? to be suspend. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ???????? If you decide not to SuspendThread, then you don't need the >>>>>>>>>>>>>>>>>>>>>>> ???????? AddCapabilities or the ResumeThread calls. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Test thread might not be entered following code (stopSignal.await()). We might see deferent call stack between GetAllStackTraces() and GetThreadListStackTraces(). We cannot control to freeze call stack of test thread in Java code. >>>>>>>>>>>>>>>>>>>>>> (I didn't use SuspendThread() at first, but I saw some errors which causes in above.) >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> So we need to call SuspendThread() to ensure we can see same call stack. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> If you are checking that the thread is in state WAITING then it cannot escape from that state and you can sample the stack multiple times from any API and get the same result. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> I suspect the errors you saw were from the apparent incorrect use of the CountDownLatch. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> With the flow outlined above, the worker thread should be >>>>>>>>>>>>>>>>>>>> nicely blocked in stopSignal.await() when stuff is sampled. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could review again? >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Updates look fine - thanks. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> One minor nit: >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> 1274 _collector.allocate_and_fill_stacks(1); >>>>>>>>>>>>>>>>>>>>>>>>> 1275 _collector.set_result(JVMTI_ERROR_NONE); >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> In the other places where you use _collector you rely on result being initialized to JVMTI_ERROR_NONE, rather than setting it directly after allocate_and_fill_stacks(). >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 823 "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> This function (JvmtiEnvBase::get_stack_trace()) can be called to get own stack trace. For example, we can call GetStackTrace() for current thread at JVMTI event. >>>>>>>>>>>>>>>>>>>>>>>>>> So I changed assert as below: >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>> ??820 assert(current_thread == java_thread || >>>>>>>>>>>>>>>>>>>>>>>>>> ??821 SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>> ??823????????? "call by myself / at safepoint / at handshake"); >>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Yep good catch. I hope current tests caught that. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> They would be tested in vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ (own call stacks), and getstacktr003 (call stacks in other thread). >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Speaking of tests ... >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> In the native code I think you need to check the success of all JNI methods that can throw exceptions - otherwise I believe the tests may trigger warnings if -Xcheck:jni is used with them. See for example: >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> I updated testcases to check JNI and JVMTI function calls. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> In the Java code the target thread: >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> ?? 45???? public void run() { >>>>>>>>>>>>>>>>>>>>>>>>> ?? 46?????? try { >>>>>>>>>>>>>>>>>>>>>>>>> ?? 47 synchronized (lock) { >>>>>>>>>>>>>>>>>>>>>>>>> ?? 48 lock.wait(); >>>>>>>>>>>>>>>>>>>>>>>>> ?? 49 System.out.println("OK"); >>>>>>>>>>>>>>>>>>>>>>>>> ?? 50?????????? } >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> is potentially susceptible to spurious wakeups. Using a CountDownLatch would be robust. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> If so, we can call make_local() in L1272 without JavaThread (or we can pass current thread to make_local()). Is it right? >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry I got confused, _calling_thread may not be the current thread as we could be executing the handshake in the target thread itself. So the ResourceMark is correct as-is (implicitly for current thread). >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> The argument to fill_frames will be used in the jvmtiStackInfo and passed back to the _calling_thread, so it must be created via make_local(_calling_thread, ...) as you presently have. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you for reviewing! I will update new webrev tomorrow. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 500 JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??? : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I updated webrev for 8242428. Could you review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change migrate to use direct handshake for GetStackTrace() and GetThreadListStackTraces() (when thread_count == 1). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks really good now! I only have a few nits below. There is one thing I don't like about it but it requires a change to the main Handshake logic to address - in JvmtiEnv::GetThreadListStackTraces you have to create a ThreadsListHandle to convert the jthread to a JavaThread, but then the Handshake::execute_direct creates another ThreadsListHandle internally. That's a waste. I will discuss with Robbin and file a RFE to have an overload of execute_direct that takes an existing TLH. Actually it's worse than that because we have another TLH in use at the entry point for the JVMTI functions, so I think there may be some scope for simplifying the use of TLH instances - future RFE. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??451 GetStackTraceClosure(JvmtiEnv *env, jint start_depth, jint max_count, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??452 jvmtiFrameInfo* frame_buffer, jint* count_ptr) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??454 _env(env), _start_depth(start_depth), _max_count(max_count), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??455 _frame_buffer(frame_buffer), _count_ptr(count_ptr), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??456 _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Nit: can you do one initializer per line please. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??500 JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 481 MultipleStackTracesCollector(JvmtiEnv *env, jint max_frame_count) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??483 _max_frame_count = max_frame_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??484 _frame_count_total = 0; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??485 _head = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??486 _stack_info = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??487 _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As you are touching this can you change it to use an initializer list as you did for the HandshakeClosure, and please keep one item per line. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??823 "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1269 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is frustrating that this entire call chain started with a jthread reference, which we converted to a JavaThread, only to eventually need to convert it back to a jthread! I think there is some scope for simplification here but not as part of this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm still pondering the test logic but wanted to send this now. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetThreadListStackTrace (for GetThreadListStackTraces) and VM_GetAllStackTraces (for GetAllStackTraces) have inherited VM_GetMultipleStackTraces VM operation which provides the feature to generate jvmtiStackInfo. I modified VM_GetMultipleStackTraces to a normal C++ class to share with HandshakeClosure for GetThreadListStackTraces (GetSingleStackTraceClosure). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I added new testcases which test GetThreadListStackTraces() with thread_count == 1 and with all threads. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/nsk/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change replace following VM operations to direct handshake. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces (GetThreadListStackTrace()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTrace() uses direct handshake if thread count == 1. In other case (thread count > 1), it would be performed as VM operation (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Caller of VM_GetCurrentLocation (JvmtiEnvThreadState::reset_current_location()) might be called at safepoint. So I added safepoint check in its caller. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/ns >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I tested it on submit repo, then it has execution error (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) due to dependency error. So I think it does not occur by this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >> From jcbeyler at google.com Thu Jul 9 04:03:13 2020 From: jcbeyler at google.com (Jean Christophe Beyler) Date: Wed, 8 Jul 2020 21:03:13 -0700 Subject: RFR (S) 8247615: Initialize the bytes left for the heap sampler In-Reply-To: References: Message-ID: Hello all, Any way to get two reviews of the new version: http://cr.openjdk.java.net/~jcbeyler/8247615/webrev.01/ Thanks and have a great evening! Jc On Tue, Jun 30, 2020 at 12:47 AM Martin Buchholz wrote: > Looks good to me, but of course I'm not qualified to review. > > On Mon, Jun 29, 2020 at 3:26 PM Jean Christophe Beyler > wrote: > > > > Agreed; missed a hg qrefresh; link is now updated: > > http://cr.openjdk.java.net/~jcbeyler/8247615/webrev.01/ > > > > :) > > Jc > > > > On Mon, Jun 29, 2020 at 2:28 PM Man Cao wrote: > >> > >> Looks good. > >> > >> > though adding the change that Man wants might make it more flaky so I > added your numThreads / 2 in case > >> I don't see the "numThreads / 2" in webrev.01 though. No need for a > webrev for this fix. > >> > >> -Man > >> > >> > >> On Mon, Jun 29, 2020 at 1:10 PM Jean Christophe Beyler < > jcbeyler at google.com> wrote: > >>> > >>> Hi all, > >>> > >>> Sorry it took time to get back to this; could I get a new review from: > >>> http://cr.openjdk.java.net/~jcbeyler/8247615/webrev.01/ > >>> > >>> The bug is here: > >>> https://bugs.openjdk.java.net/browse/JDK-8247615 > >>> > >>> Note, this passed the submit repo testing. > >>> > >>> Thanks and have a great day! > >>> Jc > >>> > >>> Ps: explicit inlined Acks/Done are below: > >>> > >>> Sorry it took time to get back to this: > >>> @Martin: > >>> - done the typo > >>> - about the sampling test: No you won't get samples due to how the > system is done, since we know we only will be allocating one object for the > thread, it dies out before a sample is required... though adding the change > that Man wants might make it more flaky so I added your numThreads / 2 in > case > >>> - done for the always in the description > >>> > >>> > >>> On Thu, Jun 25, 2020 at 6:54 PM Derek Thomson > wrote: > >>>> > >>>> > It could also avoid the problem where every thread > deterministically allocates the same object at 512K, although this is > unlikely. > >>>> > >>>> I've recently discovered that with certain server frameworks that > this actually becomes quite likely! So I'd strongly recommend using > pick_next_sample. > >>> > >>> > >>> Ack, done :) > >>> > >>>> > >>>> > >>>> On Thu, Jun 25, 2020 at 4:56 PM Man Cao wrote: > >>>>> > >>>>> Thanks for fixing this! > >>>>> > >>>>> > 53 ThreadHeapSampler() : > _bytes_until_sample(get_sampling_interval()) { > >>>>> > >>>>> Does this work better? (It has to be done after the initialization > of _rnd.) > >>>>> _bytes_until_sample = pick_next_sample(); > >>>>> > >>>>> It could avoid completely missing to sample the first 512K > allocation. > >>>>> It could also avoid the problem where every thread > >>> > >>> > >>> Done. > >>> > >>> > >>>>> > >>>>> deterministically allocates the same object at 512K, although this > is unlikely. > >>>>> > >>>>> -Man > >>> > >>> > >>> > >>> -- > >>> > >>> Thanks, > >>> Jc > > > > > > > > -- > > > > Thanks, > > Jc > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From linzang at tencent.com Thu Jul 9 07:22:22 2020 From: linzang at tencent.com (=?utf-8?B?bGluemFuZyjoh6fnkLMp?=) Date: Thu, 9 Jul 2020 07:22:22 +0000 Subject: RFR(L): 8215624: add parallel heap inspection support for jmap histo(G1)(Internet mail) In-Reply-To: <02415944-2238-4430-A7C2-3625264A5505@amazon.com> References: <02415944-2238-4430-A7C2-3625264A5505@amazon.com> Message-ID: Hi Paul, Thanks for reviewing! >> >> I'd move all the argument parsing code to JMap.java and just pass the results to Hotspot. Both histo() in JMap.java and code in attachListener.* parse the command line arguments, though the code in histo() doesn't parse the argument to "parallel". I'd upgrade the code in histo() to do a complete parse and pass the option values to executeCommandForPid as before: there would just be more of them now. That would allow you to eliminate all the parsing code in attachListener.cpp as well as the change to arguments.hpp. >> The reason I made the change in Jmap.java that compose all arguments as 1 string , instead of passing 3 argments, is to avoid the compatibility issue, as we discussed in http://mail.openjdk.java.net/pipermail/serviceability-dev/2019-February/thread.html#27240. The root cause of the compatibility issue is because max argument count in HotspotVirtualMachineImpl.java and attachlistener.cpp need to be enlarged (changes like http://hg.openjdk.java.net/jdk/jdk/rev/e7cf035682e3#l2.1) when jmap has more than 3 arguments. But if user use an old jcmd/jmap tool, it may stuck at socket read(), because the "max argument count" don't match. I re-checked this change, the argument count of jmap histo is equal to 3 (live, file, parallel), so it can work normally even without the change of passing argument. But I think we have to face the problem if more arguments is added in jcmd alike tools later, not sure whether it should be sloved (or a workaround) in this changeset. And here are the lastest webrev and delta: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_06/ http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_06-delta/ Cheers, Lin ?On 2020/7/7, 5:57 AM, "Hohensee, Paul" wrote: I'd like to see this feature added. :) The CSR looks good, as does the basic parallel inspection algorithm. Stefan's done the GC part, so I'll stick to the non-GC part (fwiw, the GC part lgtm). I'd move all the argument parsing code to JMap.java and just pass the results to Hotspot. Both histo() in JMap.java and code in attachListener.* parse the command line arguments, though the code in histo() doesn't parse the argument to "parallel". I'd upgrade the code in histo() to do a complete parse and pass the option values to executeCommandForPid as before: there would just be more of them now. That would allow you to eliminate all the parsing code in attachListener.cpp as well as the change to arguments.hpp. heapInspection.hpp: _shared_miss_count (s/b _missed_count, see below) isn't a size, so it should be a uint instead of a size_t. Same with the new parallel_thread_num argument to heap_inspection() and populate_table(). Comment copy-edit: +// Parallel heap inspection task. Parallel inspection can fail due to +// a native OOM when allocating memory for TL-KlassInfoTable. +// _success will be set false on an OOM, and serial inspection tried. _shared_miss_count should be _missed_count to match the missed_count() getter, or rename missed_count() to be shared_miss_count(). Whichever way you go, the field type should match the getter result type: uint is reasonable. heapInspection.cpp: You might use ResourceMark twice in populate_table, separately for the parallel attempt and the serial code. If the parallel attempt fails and available memory is low, it would be good to clean up the memory used by the parallel attempt before doing the serial code. Style nit in KlassInfoTable::merge_entry(). I'd line up the definitions of k and elt, so "k" is even with "elt". And, because it's two lines shorter, I'd replace + } else { + return false; + } with + return false; KlassInfoTableMergeClosure.is_success() should be just success() (i.e., no "is_" prefix) because it's a getter. I'd reorganize the code in populate_table() to make it more clear, vis (I changed _shared_missed_count to _missed_count) + if (cit.allocation_failed()) { + // fail to allocate memory, stop parallel mode + Atomic::store(&_success, false); + return; + } + RecordInstanceClosure ric(&cit, _filter); + _poi->object_iterate(&ric, worker_id); + missed_count = ric.missed_count(); + { + MutexLocker x(&_mutex); + merge_success = _shared_cit->merge(&cit); + } + if (merge_success) { + Atomic::add(&_missed_count, missed_count); + else { + Atomic::store(&_success, false); + } Thanks, Paul On 6/29/20, 7:20 PM, "linzang(??)" wrote: Dear All, Sorry to bother again, I just want to make sure that is this change worth to be continue to work on? If decision is made to not. I think I can drop this work and stop asking for help reviewing... Thanks for all your help about reviewing this previously. BRs, Lin On 2020/5/9, 3:47 PM, "linzang(??)" wrote: Dear All, May I ask your help again for review the latest change? Thanks! BRs, Lin On 2020/4/28, 1:54 PM, "linzang(??)" wrote: Hi Stefan, >> - Adding Atomic::load/store. >> - Removing the time measurement in the run_task. I renamed G1's function >> to run_task_timed. If we need this outside of G1, we can rethink the API >> at that point. >> - ZGC style cleanups Thanks for revising the patch, they are all good to me, and I have made a tiny change based on it: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04/ http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04-delta/ it reduce the scope of mutex in ParHeapInspectTask, and delete unnecessary comments. BRs, Lin On 2020/4/27, 4:34 PM, "Stefan Karlsson" wrote: Hi Lin, On 2020-04-26 05:10, linzang(??) wrote: > Hi Stefan and Paul? > I have made a new patch based on your comments and Stefan's Poc code: > Webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03/ > Delta(based on Stefan's change:) : http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03-delta/webrev_03-delta/ Thanks for providing a delta patch. It makes it much easier to look at, and more likely for reviewers to continue reviewing. I'm going to continue focusing on the GC parts, and leave the rest to others to review. > > And Here are main changed I made and want to discuss with you: > 1. changed"parallelThreadNum=" to "parallel=" for jmap -histo options. > 2. Add logic to test where parallelHeapInspection is fail, in heapInspection.cpp > This is because the parHeapInspectTask create thread local KlassInfoTable in it's work() method, and this may fail because of native OOM, in this case, the parallel should fail and serial heap inspection can be tried. > One more thing I want discuss with you is about the member "_success" of parHeapInspectTask, when native OOM happenes, it is set to false. And since this "set" operation can be conducted in multiple threads, should it be atomic ops? IMO, this is not necessary because "_success" can only be set to false, and there is no way to change it from back to true after the ParHeapInspectTask instance is created, so it is save to be non-atomic, do you agree with that? In these situations you should be using the Atomic::load/store primitives. We're moving toward a later C++ standard were data races are considered undefined behavior. > 3. make CollectedHeap::run_task() be an abstract virtual func, so that every subclass of collectedHeap should support it, so later implementation of new collectedHeap will not miss the "parallel" features. > The problem I want to discuss with you is about epsilonHeap and SerialHeap, as they may not need parallel heap iteration, so I only make task->work(0), in case the run_task() is invoked someway in future. Another way is to left run_task() unimplemented, which one do you think is better? I don't have a strong opinion about this. And also please help take a look at the zHeap, as there is a class zTask that wrap the abstractGangTask, and the collectedHeap::run_task() only accept AbstraceGangTask* as argument, so I made a delegate class to adapt it , please see src/hotspot/share/gc/z/zHeap.cpp. > > There maybe other better ways to sovle the above problems, welcome for any comments, Thanks! I've created a few cleanups and changes on top of your latest patch: https://cr.openjdk.java.net/~stefank/8215624/webrev.02.delta https://cr.openjdk.java.net/~stefank/8215624/webrev.02 - Adding Atomic::load/store. - Removing the time measurement in the run_task. I renamed G1's function to run_task_timed. If we need this outside of G1, we can rethink the API at that point. - ZGC style cleanups Thanks, StefanK > > BRs, > Lin > > On 2020/4/23, 11:08 AM, "linzang(??)" wrote: > > Thanks Paul! I agree with using "parallel", will make the update in next patch, Thanks for help update the CSR. > > BRs, > Lin > > On 2020/4/23, 4:42 AM, "Hohensee, Paul" wrote: > > For the interface, I'd use "parallel" instead of "parallelThreadNum". All the other options are lower case, and it's a lot easier to type "parallel". I took the liberty of updating the CSR. If you're ok with it, you might want to change variable names and such, plus of course JMap.usage. > > Thanks, > Paul > > On 4/22/20, 2:29 AM, "serviceability-dev on behalf of linzang(??)" wrote: > > Dear Stefan, > > Thanks a lot! I agree with you to decouple the heap inspection code with GC's. > I will start from your POC code, may discuss with you later. > > > BRs, > Lin > > On 2020/4/22, 5:14 PM, "Stefan Karlsson" wrote: > > Hi Lin, > > I took a look at this earlier and saw that the heap inspection code is > strongly coupled with the CollectedHeap and G1CollectedHeap. I'd prefer > if we'd abstract this away, so that the GCs only provide a "parallel > object iteration" interface, and the heap inspection code is kept elsewhere. > > I started experimenting with doing that, but other higher-priority (to > me) tasks have had to take precedence. > > I've uploaded my work-in-progress / proof-of-concept: > https://cr.openjdk.java.net/~stefank/8215624/webrev.01.delta/ > https://cr.openjdk.java.net/~stefank/8215624/webrev.01/ > > The current code doesn't handle the lifecycle (deletion) of the > ParallelObjectIterators. There's also code left unimplemented in around > CollectedHeap::run_task. However, I think this could work as a basis to > pull out the heap inspection code out of the GCs. > > Thanks, > StefanK > > On 2020-04-22 02:21, linzang(??) wrote: > > Dear all, > > May I ask you help to review? This RFR has been there for quite a while. > > Thanks! > > > > BRs, > > Lin > > > > > On 2020/3/16, 5:18 PM, "linzang(??)" wrote:> > > > >> Just update a new path, my preliminary measure show about 3.5x speedup of jmap histo on a nearly full 4GB G1 heap (8-core platform with parallel thread number set to 4). > >> webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_02/ > >> bug: https://bugs.openjdk.java.net/browse/JDK-8215624 > >> CSR: https://bugs.openjdk.java.net/browse/JDK-8239290 > >> BRs, > >> Lin > >> > On 2020/3/2, 9:56 PM, "linzang(??)" wrote: > >> > > >> > Dear all, > >> > Let me try to ease the reviewing work by some explanation :P > >> > The patch's target is to speed up jmap -histo for heap iteration, from my experience it is necessary for large heap investigation. E.g in bigData scenario I have tried to conduct jmap -histo against 180GB heap, it does take quite a while. > >> > And if my understanding is corrent, even the jmap -histo without "live" option does heap inspection with heap lock acquired. so it is very likely to block mutator thread in allocation-sensitive scenario. I would say the faster the heap inspection does, the shorter the mutator be blocked. This is parallel iteration for jmap is necessary. > >> > I think the parallel heap inspection should be applied to all kind of heap. However, consider the heap layout are different for GCs, much time is required to understand all kinds of the heap layout to make the whole change. IMO, It is not wise to have a huge patch for the whole solution at once, and it is even harder to review it. So I plan to implement it incrementally, the first patch (this one) is going to confirm the implemention detail of how jmap accept the new option, passes it to attachListener of the jvm process and then how to make the parallel inspection closure be generic enough to make it easy to extend to different heap layout. And also how to implement the heap inspection in specific gc's heap. This patch use G1's heap as the begining. > >> > This patch actually do several things: > >> > 1. Add an option "parallelThreadNum=" to jmap -histo, the default behavior is to set N to 0, means let's JVM decide how many threads to use for heap inspection. Set this option to 1 will disable parallel heap inspection. (more details in CSR: https://bugs.openjdk.java.net/browse/JDK-8239290) > >> > 2. Make a change in how Jmap passing arguments, changes in http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java.udiff.html, originally it pass options as separate arguments to attachListener, this patch change to that all options be compose to a single string. So the arg_count_max in attachListener.hpp do not need to be changed, and hence avoid the compatibility issue, as disscussed at https://mail.openjdk.java.net/pipermail/serviceability-dev/2019-March/027334.html > >> > 3. Add an abstract class ParHeapInspectTask in heapInspection.hpp / heapInspection.cpp, It's work(uint worker_id) method prepares the data structure (KlassInfoTable) need for every parallel worker thread, and then call do_object_iterate_parallel() which is heap specific implementation. I also added some machenism in KlassInfoTable to support parallel iteration, such as merge(). > >> > 4. In specific heap (G1 in this patch), create a subclass of ParHeapInspectTask, implement the do_object_iterate_parallel() for parallel heap inspection. For G1, it simply invoke g1CollectedHeap's object_iterate_parallel(). > >> > 5. Add related test. > >> > 6. it may be easy to extend this patch for other kinds of heap by creating subclass of ParHeapInspectTask and implement the do_object_iterate_parallel(). > >> > > >> > Hope these info could help on code review and initate the discussion :-) > >> > Thanks! > >> > > >> > BRs, > >> > Lin > >> > >On 2020/2/19, 9:40 AM, "linzang(??)" wrote:. > >> > > > >> > > Re-post this RFR with correct enhancement number to make it trackable. > >> > > please ignore the previous wrong post. sorry for troubles. > >> > > > >> > > webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/ > >> > > Hi bug: https://bugs.openjdk.java.net/browse/JDK-8215624 > >> > > CSR: https://bugs.openjdk.java.net/browse/JDK-8239290 > >> > > -------------- > >> > > Lin > >> > > >Hi Lin, > > > > > > > >> > > >Could you, please, re-post your RFR with the right enhancement number in > >> > > >the message subject? > >> > > >It will be more trackable this way. > >> > > > > >> > > >Thanks, > >> > > >Serguei > >> > > > > >> > > > > >> > > >On 2/17/20 10:29 PM, linzang(??) wrote: > >> > > >> Dear David, > >> > > >> Thanks a lot! > >> > > >> I have updated the refined code to http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_01/. > >> > > >> IMHO the parallel heap inspection can be extended to all kinds of heap as long as the heap layout can support parallel iteration. > >> > > >> Maybe we can firstly use this webrev to discuss how to implement it, because I am not sure my current implementation is an appropriate way to communicate with collectedHeap, then we can extend the solution to other kinds of heap. > >> > > >> > >> > > >> Thanks, > >> > > >> -------------- > >> > > >> Lin > >> > > >>> Hi Lin, > >> > > >>> > >> > > >>> Adding in hotspot-gc-dev as they need to see how this interacts with GC > >> > > >>> worker threads, and whether it needs to be extended beyond G1. > >> > > >>> > >> > > >>> I happened to spot one nit when browsing: > >> > > >>> > >> > > >>> src/hotspot/share/gc/shared/collectedHeap.hpp > >> > > >>> > >> > > >>> + virtual bool run_par_heap_inspect_task(KlassInfoTable* cit, > >> > > >>> + BoolObjectClosure* filter, > >> > > >>> + size_t* missed_count, > >> > > >>> + size_t thread_num) { > >> > > >>> + return NULL; > >> > > >>> > >> > > >>> s/NULL/false/ > >> > > >>> > >> > > >>> Cheers, > >> > > >>> David > > > > > >>> > >> > > >>> On 18/02/2020 2:15 pm, linzang(??) wrote: > >> > > >>>> Dear All, > >> > > >>>> May I ask your help to review the follow changes: > >> > > >>>> webrev: > >> > > >>>> http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_00/ > >> > > >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8215624 > >> > > >>>> related CSR: https://bugs.openjdk.java.net/browse/JDK-8239290 > >> > > >>>> This patch enable parallel heap inspection of G1 for jmap histo. > >> > > >>>> my simple test shown it can speed up 2x of jmap -histo with > >> > > >>>> parallelThreadNum set to 2 for heap at ~500M on 4-core platform. > >> > > >>>> > >> > > >>>> ------------------------------------------------------------------------ > >> > > >>>> BRs, > >> > > >>>> Lin > >> > > >> > > >> > > > > > > > > > > > > > > > From david.holmes at oracle.com Thu Jul 9 08:58:03 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 9 Jul 2020 18:58:03 +1000 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <238a89a3-06b1-90e4-d5a4-8dc2f1ed12ab@oracle.com> <4c43bc2c-551e-d042-8fa7-b63e755888d1@oracle.com> <8c387550-a06e-6320-f453-0e898486befb@oss.nttdata.com> <19cfa2a7-f441-425d-5623-1c11c6a870b6@oss.nttdata.com> <237e8d26-d185-1008-6e07-5b9f36314534@oracle.com> Message-ID: <2a9ba1f8-b1f7-cb8c-fe11-56dd174d9d58@oracle.com> Hi Yasumasa, On 9/07/2020 10:25 am, Yasumasa Suenaga wrote: > Hi Dan, > > Thanks for your comment! > I uploaded new webrev: > > ? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.09/ > ? Diff from previous webrev: > http://hg.openjdk.java.net/jdk/submit/rev/5d167adf8524 > > I saw similar build errors in libOneGetThreadListStackTraces.cpp on > Windows. > This webrev fixes them. You shouldn't use %p as it may not be portable. In the VM we use INTPTR_FORMAT and convert the arg using p2i. I don't know what exists in the testing code. David ----- > > Thanks, > > Yasumasa > > > On 2020/07/09 1:42, Daniel D. Daugherty wrote: >> ?> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ >> >> src/hotspot/share/prims/jvmtiEnv.cpp >> ???? No comments. >> >> src/hotspot/share/prims/jvmtiEnvBase.cpp >> ???? L1159: ? Thread *current_thread = Thread::current(); >> ???????? Please add "#ifdef ASSERT" above and "#endif" below since >> ???????? current_thread is only used for the assert() in this function. >> >> src/hotspot/share/prims/jvmtiEnvBase.hpp >> ???? L549: ???????????????????????????? jthread java_thread, jint >> max_frame_count) >> ???? L552: ????? _jthread(java_thread), >> ???????? Please: s/java_thread/thread/ on both lines. >> >> src/hotspot/share/runtime/vmOperations.hpp >> ???? No comments. >> >> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >> >> ???? No comments. >> >> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >> >> ???? L27: #include >> ???????? This include is out of order; should be first in the list. >> >> ???? This file doesn't compile on my MBP13: >> >> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:49:14: >> error: format specifies type 'unsigned long' but the argument has type >> 'jmethodID' (aka '_jmethodID *') [-Werror,-Wformat] >> ????????????? fi1->method, fi2->method); >> ????????????? ^~~~~~~~~~~ >> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:49:27: >> error: format specifies type 'unsigned long' but the argument has type >> 'jmethodID' (aka '_jmethodID *') [-Werror,-Wformat] >> ????????????? fi1->method, fi2->method); >> ?????????????????????????? ^~~~~~~~~~~ >> ???? 2 errors generated. >> >> ???? This change made it compile on my MBP13, but that may break it on >> ???? other platforms: >> >> ???? $ hg diff >> ???? diff -r 560847c69fbe >> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >> >> ???? --- >> a/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >> Wed Jul 08 12:13:32 2020 -0400 >> ???? +++ >> b/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >> Wed Jul 08 12:40:42 2020 -0400 >> ???? @@ -46,7 +46,7 @@ >> ?? ?? ? if (fi1->method != fi2->method) { /* jvmtiFrameInfo::method */ >> ?? ?? ??? snprintf(err_msg, sizeof(err_msg), >> ?? ? ? ??????????? "method is different: fi1 = %lx, fi2 = %lx", >> ???? -???????????? fi1->method, fi2->method); >> ???? +???????????? (unsigned long) fi1->method, (unsigned long) >> fi2->method); >> ?? ?? ??? env->FatalError(err_msg); >> ?? ?? ? } else if (fi1->location != fi2->location) { /* >> jvmtiFrameInfo::location */ >> ?? ?? ??? snprintf(err_msg, sizeof(err_msg), >> >> ???? I'm not sure of the right platform independent way to output >> ???? the 'method' field. >> >> Dan >> >> >> On 7/8/20 4:04 AM, Yasumasa Suenaga wrote: >>> Hi David, >>> >>> On 2020/07/08 15:27, David Holmes wrote: >>>> Hi Yasumasa, >>>> >>>> On 7/07/2020 6:54 pm, Yasumasa Suenaga wrote: >>>>> Hi David, Serguei, >>>>> >>>>> Serguei, thank you for replying even though you are on vacaiton! >>>>> >>>>> I uploaded new webrev: >>>>> >>>>> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.07/ >>>>> ?? Diff from previous webrev: >>>>> http://hg.openjdk.java.net/jdk/submit/rev/77243b1dcbfe >>>>> >>>>> c'tor of GetSingleStackTraceClosure has jthread argument in this >>>>> webrev. >>>>> Also it does not contain testcase for GetThreadListStackTraces with >>>>> all threads, and OneGetThreadListStackTraces would test main thread >>>>> only. >>>> >>>> All those changes are fine in principle for me. One nit/suggestion: >>>> >>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>> >>>> ??544?? jthread _java_thread; >>>> >>>> elsewhere "java_thread" refers to a JavaThread, so to avoid >>>> confusion may I suggest this member be named _jthread. >>> >>> I uploaded new webrev: >>> >>> ? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ >>> ? Diff from previous webrev: >>> http://hg.openjdk.java.net/jdk/submit/rev/ca6263dbdc87 >>> >>> >>>> I'm going to be away for the next couple of days - sorry - but will >>>> try to check email on this if I can. >>> >>> Thanks! >>> >>> >>> Yasumasa >>> >>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2020/07/07 15:13, David Holmes wrote: >>>>>> On 7/07/2020 2:57 pm, Yasumasa Suenaga wrote: >>>>>>> Hi David, >>>>>>> >>>>>>> On 2020/07/07 11:31, David Holmes wrote: >>>>>>>> Hi Yasumasa, >>>>>>>> >>>>>>>> Hard to keep up with the changes - especially without >>>>>>>> incremental webrevs. >>>>>>> >>>>>>> Sorry, I will upload diff from previous webrev in the next. >>>>>>> >>>>>>> >>>>>>>> If GetSingleStackTraceClosure also took the jthread as a >>>>>>>> constructor arg, then you wouldn't need to recreate a JNI local >>>>>>>> handle when calling _collector.fill_frames. It's a small >>>>>>>> simplification and not essential at this stage. >>>>>>> >>>>>>> I think we should get jthread from an argument of do_thread() >>>>>>> because do_thread() would pass the thread which are stopped >>>>>>> certainly. >>>>>>> It might be simplification if we pass _calling_thread to >>>>>>> MultipleStackTracesCollector. `jthread` is only needed to store >>>>>>> jvmtiStackInfo.thread . What do you think? >>>>>> >>>>>> I'm not quite sure what you mean. >>>>>> >>>>>> I think there is a bit of a design wart with direct handshakes in >>>>>> that do_thread takes the target JavaThread as an argument. That's >>>>>> useful in a case where you want a HandshakeClosure that can be >>>>>> applied to multiple threads, but that's not typically what is >>>>>> needed with direct handshakes - there is only a single target. >>>>>> With a single-target HandshakeClosure you can capture all the >>>>>> "target" information for the operation in the closure instance. So >>>>>> if the actual do_thread operation wants the jthread corresponding >>>>>> to the target thread then we can store that in the closure rather >>>>>> than recomputing it (you could assert it is the same but that >>>>>> seems overkill to me). >>>>>> >>>>>>> >>>>>>>> For the test ... I don't see how >>>>>>>> Java_GetThreadListStackTraces_checkCallStacks is a valid test. >>>>>>>> It gets the stacks of all live threads, then uses that >>>>>>>> information to use GetThreadListStackTraces to get the stack for >>>>>>>> the same set of threads through a different API. It then >>>>>>>> compares the two sets of stacks for each thread expecting them >>>>>>>> to be the same, but that need only be the case for the main >>>>>>>> thread. Other threads could potentially have a different stack >>>>>>>> (e.g. if this test is run with JFR enabled there will be >>>>>>>> additional threads found.) Further I would have expected that >>>>>>>> there already exist tests that check that, for a given thread >>>>>>>> (which may be suspended or known to be blocked) the same stack >>>>>>>> is found through the two different APIs. >>>>>>> >>>>>>> vmTestbase/nsk/jvmti/unit/GetAllStackTraces/getallstktr001 would >>>>>>> check all of threads via GetThreadListStackTraces() and >>>>>>> GetAllStackTraces(), so we might be able to remove >>>>>>> GetThreadListStackTraces.java from this webrev. >>>>>> >>>>>> Yes. The existing test only examines a set of test threads that >>>>>> are all blocked on a raw monitor. You do not need to duplicate >>>>>> that test. >>>>>> >>>>>>> OTOH we don't have testcase for GetThreadListStackTraces() with >>>>>>> thread_count == 1, so we need to testcase for it (it is >>>>>>> OneGetThreadListStackTraces.java) It would check whether the >>>>>>> state of target thread is "waiting" before JNI call to call >>>>>>> GetThreadListStackTraces(), >>>>>> >>>>>> Yes we need to test the special cases introduced by your changes - >>>>>> totally agree - and OneGetThreadListStackTraces.java is a good >>>>>> test for that. >>>>>> >>>>>>> and also I expect it would not be run with JFR. (it is not >>>>>>> described @run) >>>>>> >>>>>> The arguments to run with JFR (or a bunch of other things) can be >>>>>> passed to the jtreg test harness to be applied to all tests. >>>>>> >>>>>>> Of course we can check GetThreadListStackTraces() with main >>>>>>> thread, but it is not the test for direct handshake for other >>>>>>> thread. >>>>>> >>>>>> Right - that test already exists as per the above. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> ----- >>>>>>>> >>>>>>>> On 6/07/2020 11:29 pm, Yasumasa Suenaga wrote: >>>>>>>>> Hi Serguei, >>>>>>>>> >>>>>>>>> Thanks for your comment! >>>>>>>>> >>>>>>>>> I think C++ is more simple to implement the test agent as you >>>>>>>>> said. >>>>>>>>> So I implement it in C++ in new webrev. Could you review again? >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.06/ >>>>>>>>> >>>>>>>>> Also I refactored libGetThreadListStackTraces.cpp, and I've >>>>>>>>> kept exception check after IsSameObject(). >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2020/07/06 16:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>> Hi Yasumasa, >>>>>>>>>> >>>>>>>>>> Thank you for the update. >>>>>>>>>> I think, a pending exception after IsSameObject needs to be >>>>>>>>>> checked. >>>>>>>>>> >>>>>>>>>> The checkStackInfo() needs one more refactoring as I've >>>>>>>>>> already suggested. >>>>>>>>>> The body of the loop at L68-L78 should be converted to a >>>>>>>>>> function check_frame_info. >>>>>>>>>> The si1->frame_buffer[i] and si2->frame_buffer[i] will be >>>>>>>>>> passed as fi1 and fi2. >>>>>>>>>> The index can be passed as well. >>>>>>>>>> I'm still suggesting to simplify the local exception_msg to >>>>>>>>>> something shorter like err_msg or exc_msg. >>>>>>>>>> >>>>>>>>>> I'm not sure using fatal is right here: >>>>>>>>>> >>>>>>>>>> This fragment looks strange: >>>>>>>>>> >>>>>>>>>> ? 152???? if ((*env)->IsSameObject(env, stack_info[i].thread, >>>>>>>>>> thread)) { >>>>>>>>>> ? 153?????? target_info = &stack_info[i]; >>>>>>>>>> ? 154?????? break; >>>>>>>>>> ? 155???? } else if ((*env)->ExceptionOccurred(env)) { >>>>>>>>>> ? 156?????? (*env)->ExceptionDescribe(env); >>>>>>>>>> ? 157?????? (*env)->FatalError(env, __FILE__); >>>>>>>>>> ? 158???? } >>>>>>>>>> >>>>>>>>>> I expected it to be: >>>>>>>>>> >>>>>>>>>> ??? jboolean same = (*env)->IsSameObject(env, >>>>>>>>>> stack_info[i].thread, thread); >>>>>>>>>> ??? if ((*env)->ExceptionOccurred(env)) { >>>>>>>>>> ????? (*env)->ExceptionDescribe(env); >>>>>>>>>> ????? (*env)->FatalError(env, __FILE__); >>>>>>>>>> ??? } >>>>>>>>>> ??? if (same) { >>>>>>>>>> ????? target_info = &stack_info[i]; >>>>>>>>>> ????? break; >>>>>>>>>> ??? } >>>>>>>>>> >>>>>>>>>> Would it better to port this agent to C++ to simplify this >>>>>>>>>> code nicer? >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Serguei >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 7/5/20 06:13, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi Serguei, >>>>>>>>>>> >>>>>>>>>>> Thanks for your comment! >>>>>>>>>>> I refactored testcase. Could you review again? >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.05/ >>>>>>>>>>> >>>>>>>>>>> It would check Java exception after IsSameObject() call. Does >>>>>>>>>>> it need? >>>>>>>>>>> Any exceptions are not described in JNI document[1], and JNI >>>>>>>>>>> implementation (jni_IsSameObject()) does not seem to throw it. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> [1] >>>>>>>>>>> https://docs.oracle.com/en/java/javase/14/docs/specs/jni/functions.html#issameobject >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2020/07/05 14:46, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Okay, thanks. >>>>>>>>>>>> Then I'm okay to keep the GetSingleStackTraceClosure. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c.html >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c.html >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I'm not sure the function 'is_same_thread() is needed. >>>>>>>>>>>> Why do not use the JNI IsSameObject instead? >>>>>>>>>>>> >>>>>>>>>>>> It seems to be a typo at L132 and L137. >>>>>>>>>>>> You, probably. did not want to print the same information >>>>>>>>>>>> for stack_info_1[i].frame_buffer[j].XXX twice. >>>>>>>>>>>> >>>>>>>>>>>> The code at lines 112-142 is not readable. >>>>>>>>>>>> I'd suggest to make a couple of refactoring steps. >>>>>>>>>>>> >>>>>>>>>>>> First step to simplify this a little bit would be with some >>>>>>>>>>>> renaming and getting rid of indexes: >>>>>>>>>>>> >>>>>>>>>>>> ?? 71 char err_msg[EXCEPTION_MSG_LEN] = {0}; >>>>>>>>>>>> ??... >>>>>>>>>>>> ??112?? /* Iterate all jvmtiStackInfo to check */ >>>>>>>>>>>> ??113?? for (i = 0; i < num_threads, *exception_msg != '\0'; >>>>>>>>>>>> i++) { >>>>>>>>>>>> ????????? jvmtiStackInfo *si1 = stack_info_1[i]; >>>>>>>>>>>> ????????? jvmtiStackInfo *si2 = stack_info_2[i]; >>>>>>>>>>>> ??114???? if (!IsSameObject(env, si1.thread, si2.thread)) { >>>>>>>>>>>> /* jvmtiStackInfo::thread */ >>>>>>>>>>>> ??115?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>> ??116??????????????? "thread[%d] is different: stack_info_1 >>>>>>>>>>>> = %p, stack_info_2 = %p", >>>>>>>>>>>> ??117??????????????? i, sinfo1.thread, sinfo2.thread); >>>>>>>>>>>> ??118???? } else if (si1.state != si2.state) { /* >>>>>>>>>>>> jvmtiStackInfo::state */ >>>>>>>>>>>> ??119?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>> ??120??????????????? "state[%d] is different: stack_info_1 = >>>>>>>>>>>> %d, stack_info_2 = %d", >>>>>>>>>>>> ??121??????????????? i, si1.state, si2.state); >>>>>>>>>>>> ??122???? } else if (si1.frame_count != si2.frame_count) { >>>>>>>>>>>> /* jvmtiStackInfo::frame_count */ >>>>>>>>>>>> ??123?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>> ??124??????????????? "frame_count[%d] is different: >>>>>>>>>>>> stack_info_1 = %d, stack_info_2 = %d", >>>>>>>>>>>> ??125??????????????? i, si1.frame_count, si2.frame_count); >>>>>>>>>>>> ??126???? } else { >>>>>>>>>>>> ??127?????? /* Iterate all jvmtiFrameInfo to check */ >>>>>>>>>>>> ??128?????? for (j = 0; j < si1.frame_count; j++) { >>>>>>>>>>>> ??129???????? if (si1.frame_buffer[j].method != >>>>>>>>>>>> si1.frame_buffer[j].method) { /* jvmtiFrameInfo::method */ >>>>>>>>>>>> ??130?????????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>> ??131??????????????????? "thread [%d] >>>>>>>>>>>> frame_buffer[%d].method is different: stack_info_1 = %lx, >>>>>>>>>>>> stack_info_2 = %lx", >>>>>>>>>>>> ??132??????????????????? i, j, si1.frame_buffer[j].method, >>>>>>>>>>>> si2.frame_buffer[j].method); >>>>>>>>>>>> ??133?????????? break; >>>>>>>>>>>> ??134???????? } else if (si1.frame_buffer[j].location != >>>>>>>>>>>> si1.frame_buffer[j].location) { /* jvmtiFrameInfo::location */ >>>>>>>>>>>> ??135?????????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>> ??136??????????????????? "thread [%d] >>>>>>>>>>>> frame_buffer[%d].location is different: stack_info_1 = %ld, >>>>>>>>>>>> stack_info_2 = %ld", >>>>>>>>>>>> ??137??????????????????? i, j, si1.frame_buffer[j].location, >>>>>>>>>>>> si2.frame_buffer[j].location); >>>>>>>>>>>> ??138?????????? break; >>>>>>>>>>>> ??139???????? } >>>>>>>>>>>> ??140?????? } >>>>>>>>>>>> ??141???? } >>>>>>>>>>>> ??142?? } >>>>>>>>>>>> >>>>>>>>>>>> Another step would be to create functions that implement a >>>>>>>>>>>> body of each loop. >>>>>>>>>>>> You can use the same techniques to simplify similar place >>>>>>>>>>>> (L127-L138) in the libOneGetThreadListStackTraces.c. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Serguei >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 7/3/20 15:55, Yasumasa Suenaga wrote: >>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>> >>>>>>>>>>>>> I'm not an Oracle employee, so I cannot know real >>>>>>>>>>>>> request(s) from your customers. >>>>>>>>>>>>> However JDK-8201641 says Dynatrace has requested this >>>>>>>>>>>>> enhancement. >>>>>>>>>>>>> >>>>>>>>>>>>> BTW I haven't heared any request from my customers about this. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 2020/07/04 4:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>> >>>>>>>>>>>>>> This difference is not that big to care about. >>>>>>>>>>>>>> I feel this is really rare case and so, does not worth >>>>>>>>>>>>>> these complications. >>>>>>>>>>>>>> Do we have a real request from customers to optimize it? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 7/3/20 01:16, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Generally I agree with you, but I have concern about the >>>>>>>>>>>>>>> difference of the result of GetStackTrace() and >>>>>>>>>>>>>>> GetThreadListStackTraces(). >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ? GetStackTrace: jvmtiFrameInfo >>>>>>>>>>>>>>> ? GetThreadListStackTraces: jvmtiStackInfo >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> jvmtiStackInfo contains thread state, and it is ensured >>>>>>>>>>>>>>> it is the state of the call stack. >>>>>>>>>>>>>>> If we want to get both call stack and thread state, we >>>>>>>>>>>>>>> need to suspend target thread, and call both >>>>>>>>>>>>>>> GetStackTrace() and GetThreadState(). Is it ok? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I was wondering if JDK-8201641 (parent ticket of this >>>>>>>>>>>>>>> change) needed them for profiling (dynatrace?) >>>>>>>>>>>>>>> If it is responsibility of JVMTI agent implementor, I >>>>>>>>>>>>>>> remove this closure. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 2020/07/03 16:45, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> After some thinking I've concluded that I do not like >>>>>>>>>>>>>>>> this optimization >>>>>>>>>>>>>>>> of the GetThreadListStackTraces with >>>>>>>>>>>>>>>> GetSingleStackTraceClosure. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> We may need more opinions on this but these are my points: >>>>>>>>>>>>>>>> ??- it adds some complexity and ugliness >>>>>>>>>>>>>>>> ??- a win is doubtful because it has to be a rare case, >>>>>>>>>>>>>>>> so that total overhead should not be high >>>>>>>>>>>>>>>> ??- if it is really high for some use cases then it is >>>>>>>>>>>>>>>> up to the user >>>>>>>>>>>>>>>> ??? to optimize it with using GetStackTrace instead >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> In such cases with doubtful overhead I usually prefer >>>>>>>>>>>>>>>> the simplicity. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Good examples where it makes sense to optimize are >>>>>>>>>>>>>>>> checks for target thread to be current thread. >>>>>>>>>>>>>>>> In such cases there is no need to suspend the target >>>>>>>>>>>>>>>> thread, or use a VMop/HandshakeClosure. >>>>>>>>>>>>>>>> For instance, please, see the Monitor functions with the >>>>>>>>>>>>>>>> check: (java_thread == calling_thread). >>>>>>>>>>>>>>>> Getting information for current thread is frequently >>>>>>>>>>>>>>>> used case, e.g. to get info at an event point. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 7/2/20 23:29, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>> Hi Dan, David, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I uploaded new webrev. Could you review again? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/ >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> OneGetThreadListStackTraces.java in this webrev would >>>>>>>>>>>>>>>>> wait until thread state is transited to "waiting" with >>>>>>>>>>>>>>>>> spin wait. >>>>>>>>>>>>>>>>> CountDownLatch::await call as Dan pointed is fixed in >>>>>>>>>>>>>>>>> it :) >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Diff from webrev.03: >>>>>>>>>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/c9aeb7001e50 >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 2020/07/03 14:15, David Holmes wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>> On 2020/07/03 12:24, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>>>> On 7/2/20 10:50 PM, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>> Sorry I'm responding here without seeing latest >>>>>>>>>>>>>>>>>>>>> webrev but there is enough context I think ... >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>> Hi Dan, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>>>>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> I upload new webrev. Could you review again? >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>>>>>>>>>>>>>>> ???? L1542: ??? // Get stack trace with handshake >>>>>>>>>>>>>>>>>>>>>>> ???????? nit - please add a period at the end. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> L1591: *stack_info_ptr = op.stack_info(); >>>>>>>>>>>>>>>>>>>>>>> ???????? The return parameter should not be >>>>>>>>>>>>>>>>>>>>>>> touched unless the return >>>>>>>>>>>>>>>>>>>>>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ???? old L1582: ? if (err == JVMTI_ERROR_NONE) { >>>>>>>>>>>>>>>>>>>>>>> ???????? Please restore this check. The return >>>>>>>>>>>>>>>>>>>>>>> parameter should not >>>>>>>>>>>>>>>>>>>>>>> ???????? be touched unless the return code in >>>>>>>>>>>>>>>>>>>>>>> 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> But op.stack_info() will return NULL if the error >>>>>>>>>>>>>>>>>>>>> is not JVMTI_ERROR_NONE. Are you (Dan) concerned >>>>>>>>>>>>>>>>>>>>> about someone passing in a non-null/initialized >>>>>>>>>>>>>>>>>>>>> out-pointer that will be reset to NULL if there was >>>>>>>>>>>>>>>>>>>>> an error? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Actually the way we used to test this in POSIX tests >>>>>>>>>>>>>>>>>>>> is to call >>>>>>>>>>>>>>>>>>>> an API with known bad parameters and the return >>>>>>>>>>>>>>>>>>>> parameter ptr >>>>>>>>>>>>>>>>>>>> set to NULL. If the return parameter ptr was touched >>>>>>>>>>>>>>>>>>>> when an >>>>>>>>>>>>>>>>>>>> error should have been detected on an earlier >>>>>>>>>>>>>>>>>>>> parameter, then >>>>>>>>>>>>>>>>>>>> the test failed. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> L1272: ? if (!jt->is_exiting() && (thread_oop != >>>>>>>>>>>>>>>>>>>>>>> NULL)) { >>>>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second >>>>>>>>>>>>>>>>>>>>>>> expression. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>> ???? old L1532: ? _result = >>>>>>>>>>>>>>>>>>>>>>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>>>>>>>>>>>>>>>>>>>> ???????? This deletion of the _result field threw >>>>>>>>>>>>>>>>>>>>>>> me for a minute and then >>>>>>>>>>>>>>>>>>>>>>> ???? ? ? I figured out that the field is init to >>>>>>>>>>>>>>>>>>>>>>> JVMTI_ERROR_THREAD_NOT_ALIVE >>>>>>>>>>>>>>>>>>>>>>> ???????? in the constructor. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ???? L1553: ? if (!jt->is_exiting() && >>>>>>>>>>>>>>>>>>>>>>> (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second >>>>>>>>>>>>>>>>>>>>>>> expression. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ???? L64: startSignal.countDown(); >>>>>>>>>>>>>>>>>>>>>>> ???????? I was expecting this to be a call to >>>>>>>>>>>>>>>>>>>>>>> await() instead of >>>>>>>>>>>>>>>>>>>>>>> ???????? countDown(). What am I missing here? >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ???????? I think this test might be passing by >>>>>>>>>>>>>>>>>>>>>>> accident right now, but... >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Main thread (which call JVMTI functions to test) >>>>>>>>>>>>>>>>>>>>>> should wait until test thread is ready. >>>>>>>>>>>>>>>>>>>>>> So main thread would wait startSignal, and test >>>>>>>>>>>>>>>>>>>>>> thread would count down. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> No! >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> The test thread that previously called obj.wait() >>>>>>>>>>>>>>>>>>>>> now calls latch.await(). >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> The main thread that previously called obj.notify() >>>>>>>>>>>>>>>>>>>>> now calls latch.countDown(). >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> The main thread continues to spin until it sees the >>>>>>>>>>>>>>>>>>>>> target is WAITING before proceeding with the test. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> If I add spin wait to wait until transit target >>>>>>>>>>>>>>>>>>> thread state is WAITING (as following), we don't need >>>>>>>>>>>>>>>>>>> to call SuspendThread(). >>>>>>>>>>>>>>>>>>> Which is better? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> The original spin-wait loop checking for? WAITING is >>>>>>>>>>>>>>>>>> better because it is the only guarantee that the >>>>>>>>>>>>>>>>>> target thread is blocked where you need it to be. >>>>>>>>>>>>>>>>>> suspending the thread is racy as you don't know >>>>>>>>>>>>>>>>>> exactly where the suspend will hit. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>> /* Wait until the thread state transits to "waiting" */ >>>>>>>>>>>>>>>>>>> while (th.getState() != Thread.State.WAITING) { >>>>>>>>>>>>>>>>>>> ???? Thread.onSpinWait(); >>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> For simplify, spin wait is prefer to >>>>>>>>>>>>>>>>>>> OneGetThreadListStackTraces.java in webrev.03. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Here's the flow as I see it: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>> ?? - start worker thread >>>>>>>>>>>>>>>>>>>> ?? - startSignal.await() >>>>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>>>> ?? - startSignal.countDown() >>>>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>>>> ?? - stopSignal.await() >>>>>>>>>>>>>>>>>>>> ???? - worker is now blocked >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>> ?? - checkCallStacks(th) >>>>>>>>>>>>>>>>>>>> ?? - stopSignal.countDown() >>>>>>>>>>>>>>>>>>>> ???? - worker is now unblocked >>>>>>>>>>>>>>>>>>>> ?? - th.join >>>>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>>>> ?? - runs off the end of run() >>>>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>> ?? - run off the end of main() >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ???? L92: ? jthreads = (jthread >>>>>>>>>>>>>>>>>>>>>>> *)malloc(sizeof(jthread) * num_threads); >>>>>>>>>>>>>>>>>>>>>>> ???????? You don't check for malloc() failure. >>>>>>>>>>>>>>>>>>>>>>> ???????? 'jthreads' is allocated but never freed. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ???? L91: ? result = >>>>>>>>>>>>>>>>>>>>>>> (*jvmti)->SuspendThread(jvmti, thread); >>>>>>>>>>>>>>>>>>>>>>> ???????? Why are you suspending the thread? >>>>>>>>>>>>>>>>>>>>>>> GetAllStackTraces() and >>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() do not require the >>>>>>>>>>>>>>>>>>>>>>> target thread(s) >>>>>>>>>>>>>>>>>>>>>>> ???????? to be suspend. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ???????? If you decide not to SuspendThread, then >>>>>>>>>>>>>>>>>>>>>>> you don't need the >>>>>>>>>>>>>>>>>>>>>>> ???????? AddCapabilities or the ResumeThread calls. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Test thread might not be entered following code >>>>>>>>>>>>>>>>>>>>>> (stopSignal.await()). We might see deferent call >>>>>>>>>>>>>>>>>>>>>> stack between GetAllStackTraces() and >>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces(). We cannot control to >>>>>>>>>>>>>>>>>>>>>> freeze call stack of test thread in Java code. >>>>>>>>>>>>>>>>>>>>>> (I didn't use SuspendThread() at first, but I saw >>>>>>>>>>>>>>>>>>>>>> some errors which causes in above.) >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> So we need to call SuspendThread() to ensure we >>>>>>>>>>>>>>>>>>>>>> can see same call stack. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> If you are checking that the thread is in state >>>>>>>>>>>>>>>>>>>>> WAITING then it cannot escape from that state and >>>>>>>>>>>>>>>>>>>>> you can sample the stack multiple times from any >>>>>>>>>>>>>>>>>>>>> API and get the same result. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> I suspect the errors you saw were from the apparent >>>>>>>>>>>>>>>>>>>>> incorrect use of the CountDownLatch. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> With the flow outlined above, the worker thread >>>>>>>>>>>>>>>>>>>> should be >>>>>>>>>>>>>>>>>>>> nicely blocked in stopSignal.await() when stuff is >>>>>>>>>>>>>>>>>>>> sampled. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could review again? >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Updates look fine - thanks. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> One minor nit: >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> 1274 _collector.allocate_and_fill_stacks(1); >>>>>>>>>>>>>>>>>>>>>>>>> 1275 _collector.set_result(JVMTI_ERROR_NONE); >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> In the other places where you use _collector >>>>>>>>>>>>>>>>>>>>>>>>> you rely on result being initialized to >>>>>>>>>>>>>>>>>>>>>>>>> JVMTI_ERROR_NONE, rather than setting it >>>>>>>>>>>>>>>>>>>>>>>>> directly after allocate_and_fill_stacks(). >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 820 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> assert(SafepointSynchronize::is_at_safepoint() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 821 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 822 current_thread == >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 823 "at safepoint / handshake or target >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> necessary, as even if the target is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> suspended we must still be at a safepoint >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or in a handshake with it. Makes me >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wonder if we used to allow a racy >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stacktrace operation on a suspended >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread, assuming it would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> This function >>>>>>>>>>>>>>>>>>>>>>>>>> (JvmtiEnvBase::get_stack_trace()) can be >>>>>>>>>>>>>>>>>>>>>>>>>> called to get own stack trace. For example, we >>>>>>>>>>>>>>>>>>>>>>>>>> can call GetStackTrace() for current thread at >>>>>>>>>>>>>>>>>>>>>>>>>> JVMTI event. >>>>>>>>>>>>>>>>>>>>>>>>>> So I changed assert as below: >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>> ??820 assert(current_thread == java_thread || >>>>>>>>>>>>>>>>>>>>>>>>>> ??821 SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == >>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>> ??823????????? "call by myself / at safepoint >>>>>>>>>>>>>>>>>>>>>>>>>> / at handshake"); >>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Yep good catch. I hope current tests caught that. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> They would be tested in >>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ (own >>>>>>>>>>>>>>>>>>>>>>>> call stacks), and getstacktr003 (call stacks in >>>>>>>>>>>>>>>>>>>>>>>> other thread). >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Speaking of tests ... >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> In the native code I think you need to check >>>>>>>>>>>>>>>>>>>>>>>>> the success of all JNI methods that can throw >>>>>>>>>>>>>>>>>>>>>>>>> exceptions - otherwise I believe the tests may >>>>>>>>>>>>>>>>>>>>>>>>> trigger warnings if -Xcheck:jni is used with >>>>>>>>>>>>>>>>>>>>>>>>> them. See for example: >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> I updated testcases to check JNI and JVMTI >>>>>>>>>>>>>>>>>>>>>>>> function calls. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> In the Java code the target thread: >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> ?? 45???? public void run() { >>>>>>>>>>>>>>>>>>>>>>>>> ?? 46?????? try { >>>>>>>>>>>>>>>>>>>>>>>>> ?? 47 synchronized (lock) { >>>>>>>>>>>>>>>>>>>>>>>>> ?? 48 lock.wait(); >>>>>>>>>>>>>>>>>>>>>>>>> ?? 49 System.out.println("OK"); >>>>>>>>>>>>>>>>>>>>>>>>> ?? 50?????????? } >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> is potentially susceptible to spurious wakeups. >>>>>>>>>>>>>>>>>>>>>>>>> Using a CountDownLatch would be robust. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> If so, we can call make_local() in L1272 >>>>>>>>>>>>>>>>>>>>>>>>>>>> without JavaThread (or we can pass current >>>>>>>>>>>>>>>>>>>>>>>>>>>> thread to make_local()). Is it right? >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 >>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>>>>>>>>>>>>>>>>>>> thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry I got confused, _calling_thread may not >>>>>>>>>>>>>>>>>>>>>>>>>>> be the current thread as we could be >>>>>>>>>>>>>>>>>>>>>>>>>>> executing the handshake in the target thread >>>>>>>>>>>>>>>>>>>>>>>>>>> itself. So the ResourceMark is correct as-is >>>>>>>>>>>>>>>>>>>>>>>>>>> (implicitly for current thread). >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> The argument to fill_frames will be used in >>>>>>>>>>>>>>>>>>>>>>>>>>> the jvmtiStackInfo and passed back to the >>>>>>>>>>>>>>>>>>>>>>>>>>> _calling_thread, so it must be created via >>>>>>>>>>>>>>>>>>>>>>>>>>> make_local(_calling_thread, ...) as you >>>>>>>>>>>>>>>>>>>>>>>>>>> presently have. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you for reviewing! I will update new >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> webrev tomorrow. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 498 class VM_GetAllStackTraces : public >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 500 JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 502 MultipleStackTracesCollector >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> another class like that as it may not be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the stack. I think >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector should not >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> extend any allocation class, and should >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> always be embedded directly in another >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??? : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class GetAllStackTraces : public >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I updated webrev for 8242428. Could you >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change migrate to use direct >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> handshake for GetStackTrace() and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() (when >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread_count == 1). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks really good now! I only have a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few nits below. There is one thing I >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't like about it but it requires a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change to the main Handshake logic to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> address - in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JvmtiEnv::GetThreadListStackTraces you >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have to create a ThreadsListHandle to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> convert the jthread to a JavaThread, but >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then the Handshake::execute_direct >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates another ThreadsListHandle >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> internally. That's a waste. I will >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> discuss with Robbin and file a RFE to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have an overload of execute_direct that >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> takes an existing TLH. Actually it's >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worse than that because we have another >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> TLH in use at the entry point for the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JVMTI functions, so I think there may be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> some scope for simplifying the use of TLH >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> instances - future RFE. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??451 GetStackTraceClosure(JvmtiEnv *env, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jint start_depth, jint max_count, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??452 jvmtiFrameInfo* frame_buffer, jint* >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> count_ptr) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??453???? : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??454 _env(env), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _start_depth(start_depth), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _max_count(max_count), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??455 _frame_buffer(frame_buffer), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _count_ptr(count_ptr), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??456 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Nit: can you do one initializer per line >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> please. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??498 class VM_GetAllStackTraces : public >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??500 JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??502 MultipleStackTracesCollector >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> another class like that as it may not be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on the stack. I think >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector should not >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> extend any allocation class, and should >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> always be embedded directly in another >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 481 MultipleStackTracesCollector(JvmtiEnv >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *env, jint max_frame_count) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??483 _max_frame_count = max_frame_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??484 _frame_count_total = 0; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??485 _head = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??486 _stack_info = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??487 _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As you are touching this can you change >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it to use an initializer list as you did >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for the HandshakeClosure, and please keep >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one item per line. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??820 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> assert(SafepointSynchronize::is_at_safepoint() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??821 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??823 "at safepoint / handshake or target >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> necessary, as even if the target is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> suspended we must still be at a safepoint >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or in a handshake with it. Makes me >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wonder if we used to allow a racy >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stacktrace operation on a suspended >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread, assuming it would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1269 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is frustrating that this entire call >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> chain started with a jthread reference, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which we converted to a JavaThread, only >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to eventually need to convert it back to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a jthread! I think there is some scope >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for simplification here but not as part >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm still pondering the test logic but >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wanted to send this now. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetThreadListStackTrace (for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces) and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetAllStackTraces (for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetAllStackTraces) have inherited >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetMultipleStackTraces VM operation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which provides the feature to generate >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jvmtiStackInfo. I modified >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetMultipleStackTraces to a normal >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> C++ class to share with HandshakeClosure >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for GetThreadListStackTraces >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (GetSingleStackTraceClosure). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I added new testcases which test >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread_count == 1 and with all threads. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jvmti serviceability/jdwp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti vmTestbase/nsk/jdi >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? JBS: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? webrev: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change replace following VM >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operations to direct handshake. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (GetFrameLocation()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (GetThreadListStackTrace()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTrace() uses direct >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> handshake if thread count == 1. In >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other case (thread count > 1), it would >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be performed as VM operation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Caller of VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (JvmtiEnvThreadState::reset_current_location()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> might be called at safepoint. So I >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> added safepoint check in its caller. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jvmti >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jdwp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti vmTestbase/nsk/jdi >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/ns >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I tested it on submit repo, then >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it has execution error >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> due to dependency error. So I think it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> does not occur by this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >> From suenaga at oss.nttdata.com Thu Jul 9 12:30:04 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Thu, 9 Jul 2020 21:30:04 +0900 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: <2a9ba1f8-b1f7-cb8c-fe11-56dd174d9d58@oracle.com> References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <238a89a3-06b1-90e4-d5a4-8dc2f1ed12ab@oracle.com> <4c43bc2c-551e-d042-8fa7-b63e755888d1@oracle.com> <8c387550-a06e-6320-f453-0e898486befb@oss.nttdata.com> <19cfa2a7-f441-425d-5623-1c11c6a870b6@oss.nttdata.com> <237e8d26-d185-1008-6e07-5b9f36314534@oracle.com> <2a9ba1f8-b1f7-cb8c-fe11-56dd174d9d58@oracle.com> Message-ID: On 2020/07/09 17:58, David Holmes wrote: > Hi Yasumasa, > > On 9/07/2020 10:25 am, Yasumasa Suenaga wrote: >> Hi Dan, >> >> Thanks for your comment! >> I uploaded new webrev: >> >> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.09/ >> ?? Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/5d167adf8524 >> >> I saw similar build errors in libOneGetThreadListStackTraces.cpp on Windows. >> This webrev fixes them. > > You shouldn't use %p as it may not be portable. In the VM we use INTPTR_FORMAT and convert the arg using p2i. I don't know what exists in the testing code. I replaced %p to %lx, and also cast values to unsigned long [1] [2], but the test on submit repo was failed. Can anyone share details of mach5-one-ysuenaga-JDK-8242428-20200709-1030-12500928 ? Thanks, Yasumasa [1] http://hg.openjdk.java.net/jdk/submit/rev/dfca51958217 [2] http://hg.openjdk.java.net/jdk/submit/rev/3665361fa91b > David > ----- > >> >> Thanks, >> >> Yasumasa >> >> >> On 2020/07/09 1:42, Daniel D. Daugherty wrote: >>> ?> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ >>> >>> src/hotspot/share/prims/jvmtiEnv.cpp >>> ???? No comments. >>> >>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>> ???? L1159: ? Thread *current_thread = Thread::current(); >>> ???????? Please add "#ifdef ASSERT" above and "#endif" below since >>> ???????? current_thread is only used for the assert() in this function. >>> >>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>> ???? L549: ???????????????????????????? jthread java_thread, jint max_frame_count) >>> ???? L552: ????? _jthread(java_thread), >>> ???????? Please: s/java_thread/thread/ on both lines. >>> >>> src/hotspot/share/runtime/vmOperations.hpp >>> ???? No comments. >>> >>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>> ???? No comments. >>> >>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>> ???? L27: #include >>> ???????? This include is out of order; should be first in the list. >>> >>> ???? This file doesn't compile on my MBP13: >>> >>> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:49:14: error: format specifies type 'unsigned long' but the argument has type 'jmethodID' (aka '_jmethodID *') [-Werror,-Wformat] >>> ????????????? fi1->method, fi2->method); >>> ????????????? ^~~~~~~~~~~ >>> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:49:27: error: format specifies type 'unsigned long' but the argument has type 'jmethodID' (aka '_jmethodID *') [-Werror,-Wformat] >>> ????????????? fi1->method, fi2->method); >>> ?????????????????????????? ^~~~~~~~~~~ >>> ???? 2 errors generated. >>> >>> ???? This change made it compile on my MBP13, but that may break it on >>> ???? other platforms: >>> >>> ???? $ hg diff >>> ???? diff -r 560847c69fbe test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>> ???? --- a/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp Wed Jul 08 12:13:32 2020 -0400 >>> ???? +++ b/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp Wed Jul 08 12:40:42 2020 -0400 >>> ???? @@ -46,7 +46,7 @@ >>> ?? ?? ? if (fi1->method != fi2->method) { /* jvmtiFrameInfo::method */ >>> ?? ?? ??? snprintf(err_msg, sizeof(err_msg), >>> ?? ? ? ??????????? "method is different: fi1 = %lx, fi2 = %lx", >>> ???? -???????????? fi1->method, fi2->method); >>> ???? +???????????? (unsigned long) fi1->method, (unsigned long) fi2->method); >>> ?? ?? ??? env->FatalError(err_msg); >>> ?? ?? ? } else if (fi1->location != fi2->location) { /* jvmtiFrameInfo::location */ >>> ?? ?? ??? snprintf(err_msg, sizeof(err_msg), >>> >>> ???? I'm not sure of the right platform independent way to output >>> ???? the 'method' field. >>> >>> Dan >>> >>> >>> On 7/8/20 4:04 AM, Yasumasa Suenaga wrote: >>>> Hi David, >>>> >>>> On 2020/07/08 15:27, David Holmes wrote: >>>>> Hi Yasumasa, >>>>> >>>>> On 7/07/2020 6:54 pm, Yasumasa Suenaga wrote: >>>>>> Hi David, Serguei, >>>>>> >>>>>> Serguei, thank you for replying even though you are on vacaiton! >>>>>> >>>>>> I uploaded new webrev: >>>>>> >>>>>> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.07/ >>>>>> ?? Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/77243b1dcbfe >>>>>> >>>>>> c'tor of GetSingleStackTraceClosure has jthread argument in this webrev. >>>>>> Also it does not contain testcase for GetThreadListStackTraces with all threads, and OneGetThreadListStackTraces would test main thread only. >>>>> >>>>> All those changes are fine in principle for me. One nit/suggestion: >>>>> >>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>> >>>>> ??544?? jthread _java_thread; >>>>> >>>>> elsewhere "java_thread" refers to a JavaThread, so to avoid confusion may I suggest this member be named _jthread. >>>> >>>> I uploaded new webrev: >>>> >>>> ? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ >>>> ? Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/ca6263dbdc87 >>>> >>>> >>>>> I'm going to be away for the next couple of days - sorry - but will try to check email on this if I can. >>>> >>>> Thanks! >>>> >>>> >>>> Yasumasa >>>> >>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2020/07/07 15:13, David Holmes wrote: >>>>>>> On 7/07/2020 2:57 pm, Yasumasa Suenaga wrote: >>>>>>>> Hi David, >>>>>>>> >>>>>>>> On 2020/07/07 11:31, David Holmes wrote: >>>>>>>>> Hi Yasumasa, >>>>>>>>> >>>>>>>>> Hard to keep up with the changes - especially without incremental webrevs. >>>>>>>> >>>>>>>> Sorry, I will upload diff from previous webrev in the next. >>>>>>>> >>>>>>>> >>>>>>>>> If GetSingleStackTraceClosure also took the jthread as a constructor arg, then you wouldn't need to recreate a JNI local handle when calling _collector.fill_frames. It's a small simplification and not essential at this stage. >>>>>>>> >>>>>>>> I think we should get jthread from an argument of do_thread() because do_thread() would pass the thread which are stopped certainly. >>>>>>>> It might be simplification if we pass _calling_thread to MultipleStackTracesCollector. `jthread` is only needed to store jvmtiStackInfo.thread . What do you think? >>>>>>> >>>>>>> I'm not quite sure what you mean. >>>>>>> >>>>>>> I think there is a bit of a design wart with direct handshakes in that do_thread takes the target JavaThread as an argument. That's useful in a case where you want a HandshakeClosure that can be applied to multiple threads, but that's not typically what is needed with direct handshakes - there is only a single target. With a single-target HandshakeClosure you can capture all the "target" information for the operation in the closure instance. So if the actual do_thread operation wants the jthread corresponding to the target thread then we can store that in the closure rather than recomputing it (you could assert it is the same but that seems overkill to me). >>>>>>> >>>>>>>> >>>>>>>>> For the test ... I don't see how Java_GetThreadListStackTraces_checkCallStacks is a valid test. It gets the stacks of all live threads, then uses that information to use GetThreadListStackTraces to get the stack for the same set of threads through a different API. It then compares the two sets of stacks for each thread expecting them to be the same, but that need only be the case for the main thread. Other threads could potentially have a different stack (e.g. if this test is run with JFR enabled there will be additional threads found.) Further I would have expected that there already exist tests that check that, for a given thread (which may be suspended or known to be blocked) the same stack is found through the two different APIs. >>>>>>>> >>>>>>>> vmTestbase/nsk/jvmti/unit/GetAllStackTraces/getallstktr001 would check all of threads via GetThreadListStackTraces() and GetAllStackTraces(), so we might be able to remove GetThreadListStackTraces.java from this webrev. >>>>>>> >>>>>>> Yes. The existing test only examines a set of test threads that are all blocked on a raw monitor. You do not need to duplicate that test. >>>>>>> >>>>>>>> OTOH we don't have testcase for GetThreadListStackTraces() with thread_count == 1, so we need to testcase for it (it is OneGetThreadListStackTraces.java) It would check whether the state of target thread is "waiting" before JNI call to call GetThreadListStackTraces(), >>>>>>> >>>>>>> Yes we need to test the special cases introduced by your changes - totally agree - and OneGetThreadListStackTraces.java is a good test for that. >>>>>>> >>>>>>>> and also I expect it would not be run with JFR. (it is not described @run) >>>>>>> >>>>>>> The arguments to run with JFR (or a bunch of other things) can be passed to the jtreg test harness to be applied to all tests. >>>>>>> >>>>>>>> Of course we can check GetThreadListStackTraces() with main thread, but it is not the test for direct handshake for other thread. >>>>>>> >>>>>>> Right - that test already exists as per the above. >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> David >>>>>>>>> ----- >>>>>>>>> >>>>>>>>> On 6/07/2020 11:29 pm, Yasumasa Suenaga wrote: >>>>>>>>>> Hi Serguei, >>>>>>>>>> >>>>>>>>>> Thanks for your comment! >>>>>>>>>> >>>>>>>>>> I think C++ is more simple to implement the test agent as you said. >>>>>>>>>> So I implement it in C++ in new webrev. Could you review again? >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.06/ >>>>>>>>>> >>>>>>>>>> Also I refactored libGetThreadListStackTraces.cpp, and I've kept exception check after IsSameObject(). >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2020/07/06 16:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>> >>>>>>>>>>> Thank you for the update. >>>>>>>>>>> I think, a pending exception after IsSameObject needs to be checked. >>>>>>>>>>> >>>>>>>>>>> The checkStackInfo() needs one more refactoring as I've already suggested. >>>>>>>>>>> The body of the loop at L68-L78 should be converted to a function check_frame_info. >>>>>>>>>>> The si1->frame_buffer[i] and si2->frame_buffer[i] will be passed as fi1 and fi2. >>>>>>>>>>> The index can be passed as well. >>>>>>>>>>> I'm still suggesting to simplify the local exception_msg to something shorter like err_msg or exc_msg. >>>>>>>>>>> >>>>>>>>>>> I'm not sure using fatal is right here: >>>>>>>>>>> >>>>>>>>>>> This fragment looks strange: >>>>>>>>>>> >>>>>>>>>>> ? 152???? if ((*env)->IsSameObject(env, stack_info[i].thread, thread)) { >>>>>>>>>>> ? 153?????? target_info = &stack_info[i]; >>>>>>>>>>> ? 154?????? break; >>>>>>>>>>> ? 155???? } else if ((*env)->ExceptionOccurred(env)) { >>>>>>>>>>> ? 156?????? (*env)->ExceptionDescribe(env); >>>>>>>>>>> ? 157?????? (*env)->FatalError(env, __FILE__); >>>>>>>>>>> ? 158???? } >>>>>>>>>>> >>>>>>>>>>> I expected it to be: >>>>>>>>>>> >>>>>>>>>>> ??? jboolean same = (*env)->IsSameObject(env, stack_info[i].thread, thread); >>>>>>>>>>> ??? if ((*env)->ExceptionOccurred(env)) { >>>>>>>>>>> ????? (*env)->ExceptionDescribe(env); >>>>>>>>>>> ????? (*env)->FatalError(env, __FILE__); >>>>>>>>>>> ??? } >>>>>>>>>>> ??? if (same) { >>>>>>>>>>> ????? target_info = &stack_info[i]; >>>>>>>>>>> ????? break; >>>>>>>>>>> ??? } >>>>>>>>>>> >>>>>>>>>>> Would it better to port this agent to C++ to simplify this code nicer? >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Serguei >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 7/5/20 06:13, Yasumasa Suenaga wrote: >>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>> >>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>> I refactored testcase. Could you review again? >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.05/ >>>>>>>>>>>> >>>>>>>>>>>> It would check Java exception after IsSameObject() call. Does it need? >>>>>>>>>>>> Any exceptions are not described in JNI document[1], and JNI implementation (jni_IsSameObject()) does not seem to throw it. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> [1] https://docs.oracle.com/en/java/javase/14/docs/specs/jni/functions.html#issameobject >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 2020/07/05 14:46, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Okay, thanks. >>>>>>>>>>>>> Then I'm okay to keep the GetSingleStackTraceClosure. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c.html >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c.html >>>>>>>>>>>>> >>>>>>>>>>>>> I'm not sure the function 'is_same_thread() is needed. >>>>>>>>>>>>> Why do not use the JNI IsSameObject instead? >>>>>>>>>>>>> >>>>>>>>>>>>> It seems to be a typo at L132 and L137. >>>>>>>>>>>>> You, probably. did not want to print the same information for stack_info_1[i].frame_buffer[j].XXX twice. >>>>>>>>>>>>> >>>>>>>>>>>>> The code at lines 112-142 is not readable. >>>>>>>>>>>>> I'd suggest to make a couple of refactoring steps. >>>>>>>>>>>>> >>>>>>>>>>>>> First step to simplify this a little bit would be with some renaming and getting rid of indexes: >>>>>>>>>>>>> >>>>>>>>>>>>> ?? 71 char err_msg[EXCEPTION_MSG_LEN] = {0}; >>>>>>>>>>>>> ??... >>>>>>>>>>>>> ??112?? /* Iterate all jvmtiStackInfo to check */ >>>>>>>>>>>>> ??113?? for (i = 0; i < num_threads, *exception_msg != '\0'; i++) { >>>>>>>>>>>>> ????????? jvmtiStackInfo *si1 = stack_info_1[i]; >>>>>>>>>>>>> ????????? jvmtiStackInfo *si2 = stack_info_2[i]; >>>>>>>>>>>>> ??114???? if (!IsSameObject(env, si1.thread, si2.thread)) { /* jvmtiStackInfo::thread */ >>>>>>>>>>>>> ??115?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>> ??116??????????????? "thread[%d] is different: stack_info_1 = %p, stack_info_2 = %p", >>>>>>>>>>>>> ??117??????????????? i, sinfo1.thread, sinfo2.thread); >>>>>>>>>>>>> ??118???? } else if (si1.state != si2.state) { /* jvmtiStackInfo::state */ >>>>>>>>>>>>> ??119?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>> ??120??????????????? "state[%d] is different: stack_info_1 = %d, stack_info_2 = %d", >>>>>>>>>>>>> ??121??????????????? i, si1.state, si2.state); >>>>>>>>>>>>> ??122???? } else if (si1.frame_count != si2.frame_count) { /* jvmtiStackInfo::frame_count */ >>>>>>>>>>>>> ??123?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>> ??124??????????????? "frame_count[%d] is different: stack_info_1 = %d, stack_info_2 = %d", >>>>>>>>>>>>> ??125??????????????? i, si1.frame_count, si2.frame_count); >>>>>>>>>>>>> ??126???? } else { >>>>>>>>>>>>> ??127?????? /* Iterate all jvmtiFrameInfo to check */ >>>>>>>>>>>>> ??128?????? for (j = 0; j < si1.frame_count; j++) { >>>>>>>>>>>>> ??129???????? if (si1.frame_buffer[j].method != si1.frame_buffer[j].method) { /* jvmtiFrameInfo::method */ >>>>>>>>>>>>> ??130?????????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>> ??131??????????????????? "thread [%d] frame_buffer[%d].method is different: stack_info_1 = %lx, stack_info_2 = %lx", >>>>>>>>>>>>> ??132??????????????????? i, j, si1.frame_buffer[j].method, si2.frame_buffer[j].method); >>>>>>>>>>>>> ??133?????????? break; >>>>>>>>>>>>> ??134???????? } else if (si1.frame_buffer[j].location != si1.frame_buffer[j].location) { /* jvmtiFrameInfo::location */ >>>>>>>>>>>>> ??135?????????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>> ??136??????????????????? "thread [%d] frame_buffer[%d].location is different: stack_info_1 = %ld, stack_info_2 = %ld", >>>>>>>>>>>>> ??137??????????????????? i, j, si1.frame_buffer[j].location, si2.frame_buffer[j].location); >>>>>>>>>>>>> ??138?????????? break; >>>>>>>>>>>>> ??139???????? } >>>>>>>>>>>>> ??140?????? } >>>>>>>>>>>>> ??141???? } >>>>>>>>>>>>> ??142?? } >>>>>>>>>>>>> >>>>>>>>>>>>> Another step would be to create functions that implement a body of each loop. >>>>>>>>>>>>> You can use the same techniques to simplify similar place (L127-L138) in the libOneGetThreadListStackTraces.c. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Serguei >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 7/3/20 15:55, Yasumasa Suenaga wrote: >>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>> >>>>>>>>>>>>>> I'm not an Oracle employee, so I cannot know real request(s) from your customers. >>>>>>>>>>>>>> However JDK-8201641 says Dynatrace has requested this enhancement. >>>>>>>>>>>>>> >>>>>>>>>>>>>> BTW I haven't heared any request from my customers about this. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 2020/07/04 4:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> This difference is not that big to care about. >>>>>>>>>>>>>>> I feel this is really rare case and so, does not worth these complications. >>>>>>>>>>>>>>> Do we have a real request from customers to optimize it? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 7/3/20 01:16, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Generally I agree with you, but I have concern about the difference of the result of GetStackTrace() and GetThreadListStackTraces(). >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> ? GetStackTrace: jvmtiFrameInfo >>>>>>>>>>>>>>>> ? GetThreadListStackTraces: jvmtiStackInfo >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> jvmtiStackInfo contains thread state, and it is ensured it is the state of the call stack. >>>>>>>>>>>>>>>> If we want to get both call stack and thread state, we need to suspend target thread, and call both GetStackTrace() and GetThreadState(). Is it ok? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I was wondering if JDK-8201641 (parent ticket of this change) needed them for profiling (dynatrace?) >>>>>>>>>>>>>>>> If it is responsibility of JVMTI agent implementor, I remove this closure. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 2020/07/03 16:45, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> After some thinking I've concluded that I do not like this optimization >>>>>>>>>>>>>>>>> of the GetThreadListStackTraces with GetSingleStackTraceClosure. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> We may need more opinions on this but these are my points: >>>>>>>>>>>>>>>>> ??- it adds some complexity and ugliness >>>>>>>>>>>>>>>>> ??- a win is doubtful because it has to be a rare case, so that total overhead should not be high >>>>>>>>>>>>>>>>> ??- if it is really high for some use cases then it is up to the user >>>>>>>>>>>>>>>>> ??? to optimize it with using GetStackTrace instead >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> In such cases with doubtful overhead I usually prefer the simplicity. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Good examples where it makes sense to optimize are checks for target thread to be current thread. >>>>>>>>>>>>>>>>> In such cases there is no need to suspend the target thread, or use a VMop/HandshakeClosure. >>>>>>>>>>>>>>>>> For instance, please, see the Monitor functions with the check: (java_thread == calling_thread). >>>>>>>>>>>>>>>>> Getting information for current thread is frequently used case, e.g. to get info at an event point. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 7/2/20 23:29, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>> Hi Dan, David, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I uploaded new webrev. Could you review again? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/ >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> OneGetThreadListStackTraces.java in this webrev would wait until thread state is transited to "waiting" with spin wait. >>>>>>>>>>>>>>>>>> CountDownLatch::await call as Dan pointed is fixed in it :) >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Diff from webrev.03: >>>>>>>>>>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/c9aeb7001e50 >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 2020/07/03 14:15, David Holmes wrote: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>> On 2020/07/03 12:24, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>>>>> On 7/2/20 10:50 PM, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>> Sorry I'm responding here without seeing latest webrev but there is enough context I think ... >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>> Hi Dan, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>>>>>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> I upload new webrev. Could you review again? >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>>>>>>>>>>>>>>>> ???? L1542: ??? // Get stack trace with handshake >>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - please add a period at the end. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> L1591: *stack_info_ptr = op.stack_info(); >>>>>>>>>>>>>>>>>>>>>>>> ???????? The return parameter should not be touched unless the return >>>>>>>>>>>>>>>>>>>>>>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> ???? old L1582: ? if (err == JVMTI_ERROR_NONE) { >>>>>>>>>>>>>>>>>>>>>>>> ???????? Please restore this check. The return parameter should not >>>>>>>>>>>>>>>>>>>>>>>> ???????? be touched unless the return code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> But op.stack_info() will return NULL if the error is not JVMTI_ERROR_NONE. Are you (Dan) concerned about someone passing in a non-null/initialized out-pointer that will be reset to NULL if there was an error? >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Actually the way we used to test this in POSIX tests is to call >>>>>>>>>>>>>>>>>>>>> an API with known bad parameters and the return parameter ptr >>>>>>>>>>>>>>>>>>>>> set to NULL. If the return parameter ptr was touched when an >>>>>>>>>>>>>>>>>>>>> error should have been detected on an earlier parameter, then >>>>>>>>>>>>>>>>>>>>> the test failed. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> L1272: ? if (!jt->is_exiting() && (thread_oop != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>> ???? old L1532: ? _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>>>>>>>>>>>>>>>>>>>>> ???????? This deletion of the _result field threw me for a minute and then >>>>>>>>>>>>>>>>>>>>>>>> ???? ? ? I figured out that the field is init to JVMTI_ERROR_THREAD_NOT_ALIVE >>>>>>>>>>>>>>>>>>>>>>>> ???????? in the constructor. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> ???? L1553: ? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>>>>> ???? L64: startSignal.countDown(); >>>>>>>>>>>>>>>>>>>>>>>> ???????? I was expecting this to be a call to await() instead of >>>>>>>>>>>>>>>>>>>>>>>> ???????? countDown(). What am I missing here? >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> ???????? I think this test might be passing by accident right now, but... >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Main thread (which call JVMTI functions to test) should wait until test thread is ready. >>>>>>>>>>>>>>>>>>>>>>> So main thread would wait startSignal, and test thread would count down. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> No! >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> The test thread that previously called obj.wait() now calls latch.await(). >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> The main thread that previously called obj.notify() now calls latch.countDown(). >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> The main thread continues to spin until it sees the target is WAITING before proceeding with the test. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> If I add spin wait to wait until transit target thread state is WAITING (as following), we don't need to call SuspendThread(). >>>>>>>>>>>>>>>>>>>> Which is better? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> The original spin-wait loop checking for? WAITING is better because it is the only guarantee that the target thread is blocked where you need it to be. suspending the thread is racy as you don't know exactly where the suspend will hit. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>> /* Wait until the thread state transits to "waiting" */ >>>>>>>>>>>>>>>>>>>> while (th.getState() != Thread.State.WAITING) { >>>>>>>>>>>>>>>>>>>> ???? Thread.onSpinWait(); >>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> For simplify, spin wait is prefer to OneGetThreadListStackTraces.java in webrev.03. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Here's the flow as I see it: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>> ?? - start worker thread >>>>>>>>>>>>>>>>>>>>> ?? - startSignal.await() >>>>>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>>>>> ?? - startSignal.countDown() >>>>>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>>>>> ?? - stopSignal.await() >>>>>>>>>>>>>>>>>>>>> ???? - worker is now blocked >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>> ?? - checkCallStacks(th) >>>>>>>>>>>>>>>>>>>>> ?? - stopSignal.countDown() >>>>>>>>>>>>>>>>>>>>> ???? - worker is now unblocked >>>>>>>>>>>>>>>>>>>>> ?? - th.join >>>>>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>>>>> ?? - runs off the end of run() >>>>>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>> ?? - run off the end of main() >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>>>>> ???? L92: ? jthreads = (jthread *)malloc(sizeof(jthread) * num_threads); >>>>>>>>>>>>>>>>>>>>>>>> ???????? You don't check for malloc() failure. >>>>>>>>>>>>>>>>>>>>>>>> ???????? 'jthreads' is allocated but never freed. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>>>>> ???? L91: ? result = (*jvmti)->SuspendThread(jvmti, thread); >>>>>>>>>>>>>>>>>>>>>>>> ???????? Why are you suspending the thread? GetAllStackTraces() and >>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() do not require the target thread(s) >>>>>>>>>>>>>>>>>>>>>>>> ???????? to be suspend. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> ???????? If you decide not to SuspendThread, then you don't need the >>>>>>>>>>>>>>>>>>>>>>>> ???????? AddCapabilities or the ResumeThread calls. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Test thread might not be entered following code (stopSignal.await()). We might see deferent call stack between GetAllStackTraces() and GetThreadListStackTraces(). We cannot control to freeze call stack of test thread in Java code. >>>>>>>>>>>>>>>>>>>>>>> (I didn't use SuspendThread() at first, but I saw some errors which causes in above.) >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> So we need to call SuspendThread() to ensure we can see same call stack. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> If you are checking that the thread is in state WAITING then it cannot escape from that state and you can sample the stack multiple times from any API and get the same result. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I suspect the errors you saw were from the apparent incorrect use of the CountDownLatch. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> With the flow outlined above, the worker thread should be >>>>>>>>>>>>>>>>>>>>> nicely blocked in stopSignal.await() when stuff is sampled. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could review again? >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Updates look fine - thanks. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> One minor nit: >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> 1274 _collector.allocate_and_fill_stacks(1); >>>>>>>>>>>>>>>>>>>>>>>>>> 1275 _collector.set_result(JVMTI_ERROR_NONE); >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> In the other places where you use _collector you rely on result being initialized to JVMTI_ERROR_NONE, rather than setting it directly after allocate_and_fill_stacks(). >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 823 "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> This function (JvmtiEnvBase::get_stack_trace()) can be called to get own stack trace. For example, we can call GetStackTrace() for current thread at JVMTI event. >>>>>>>>>>>>>>>>>>>>>>>>>>> So I changed assert as below: >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>> ??820 assert(current_thread == java_thread || >>>>>>>>>>>>>>>>>>>>>>>>>>> ??821 SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>> ??823????????? "call by myself / at safepoint / at handshake"); >>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Yep good catch. I hope current tests caught that. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> They would be tested in vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ (own call stacks), and getstacktr003 (call stacks in other thread). >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Speaking of tests ... >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> In the native code I think you need to check the success of all JNI methods that can throw exceptions - otherwise I believe the tests may trigger warnings if -Xcheck:jni is used with them. See for example: >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> I updated testcases to check JNI and JVMTI function calls. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> In the Java code the target thread: >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> ?? 45???? public void run() { >>>>>>>>>>>>>>>>>>>>>>>>>> ?? 46?????? try { >>>>>>>>>>>>>>>>>>>>>>>>>> ?? 47 synchronized (lock) { >>>>>>>>>>>>>>>>>>>>>>>>>> ?? 48 lock.wait(); >>>>>>>>>>>>>>>>>>>>>>>>>> ?? 49 System.out.println("OK"); >>>>>>>>>>>>>>>>>>>>>>>>>> ?? 50?????????? } >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> is potentially susceptible to spurious wakeups. Using a CountDownLatch would be robust. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> If so, we can call make_local() in L1272 without JavaThread (or we can pass current thread to make_local()). Is it right? >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry I got confused, _calling_thread may not be the current thread as we could be executing the handshake in the target thread itself. So the ResourceMark is correct as-is (implicitly for current thread). >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> The argument to fill_frames will be used in the jvmtiStackInfo and passed back to the _calling_thread, so it must be created via make_local(_calling_thread, ...) as you presently have. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you for reviewing! I will update new webrev tomorrow. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 500 JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??? : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I updated webrev for 8242428. Could you review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change migrate to use direct handshake for GetStackTrace() and GetThreadListStackTraces() (when thread_count == 1). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks really good now! I only have a few nits below. There is one thing I don't like about it but it requires a change to the main Handshake logic to address - in JvmtiEnv::GetThreadListStackTraces you have to create a ThreadsListHandle to convert the jthread to a JavaThread, but then the Handshake::execute_direct creates another ThreadsListHandle internally. That's a waste. I will discuss with Robbin and file a RFE to have an overload of execute_direct that takes an existing TLH. Actually it's worse than that because we have another TLH in use at the entry point for the JVMTI functions, so I think there may be some scope for simplifying the use of TLH instances - future RFE. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??451 GetStackTraceClosure(JvmtiEnv *env, jint start_depth, jint max_count, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??452 jvmtiFrameInfo* frame_buffer, jint* count_ptr) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??454 _env(env), _start_depth(start_depth), _max_count(max_count), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??455 _frame_buffer(frame_buffer), _count_ptr(count_ptr), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??456 _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Nit: can you do one initializer per line please. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??500 JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 481 MultipleStackTracesCollector(JvmtiEnv *env, jint max_frame_count) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??483 _max_frame_count = max_frame_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??484 _frame_count_total = 0; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??485 _head = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??486 _stack_info = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??487 _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As you are touching this can you change it to use an initializer list as you did for the HandshakeClosure, and please keep one item per line. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??823 "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1269 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is frustrating that this entire call chain started with a jthread reference, which we converted to a JavaThread, only to eventually need to convert it back to a jthread! I think there is some scope for simplification here but not as part of this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm still pondering the test logic but wanted to send this now. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetThreadListStackTrace (for GetThreadListStackTraces) and VM_GetAllStackTraces (for GetAllStackTraces) have inherited VM_GetMultipleStackTraces VM operation which provides the feature to generate jvmtiStackInfo. I modified VM_GetMultipleStackTraces to a normal C++ class to share with HandshakeClosure for GetThreadListStackTraces (GetSingleStackTraceClosure). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I added new testcases which test GetThreadListStackTraces() with thread_count == 1 and with all threads. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/nsk/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change replace following VM operations to direct handshake. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces (GetThreadListStackTrace()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTrace() uses direct handshake if thread count == 1. In other case (thread count > 1), it would be performed as VM operation (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Caller of VM_GetCurrentLocation (JvmtiEnvThreadState::reset_current_location()) might be called at safepoint. So I added safepoint check in its caller. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/ns >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I tested it on submit repo, then it has execution error (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) due to dependency error. So I think it does not occur by this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>> From jaroslav.bachorik at datadoghq.com Thu Jul 9 13:07:30 2020 From: jaroslav.bachorik at datadoghq.com (=?UTF-8?Q?Jaroslav_Bachor=C3=ADk?=) Date: Thu, 9 Jul 2020 15:07:30 +0200 Subject: Some clarification about JVMTI_EVENT_THREAD_START and PRIMORDIAL phase Message-ID: Hello, Recently, after backporting JDK-8233197 to JDK8u I received a report from Sergey Nazarkin (cc) that the backport broke JVMTI spec basically emitting JVMTI_EVENT_THREAD_START events in primordial phase while the spec states that it should be emitted only in either start or live phase. But if I am reading this correctly (https://hg.openjdk.java.net/jdk8u/jdk8udev/hotspot/file/02b4fd2f9041/src/share/vm/prims/jvmtiEventController.cpp#l418) the spec is also violated there. EARLY_EVENT_BITS are including THREAD_START_BIT and it is not filtered out by THREAD_FILTERED_EVENT_BITS so it seems perfectly valid for JVMTI_EVENT_THREAD_START to appear during PRIMORDIAL phase. So I would like to get clarification before going in and start tinkering with the JVM initialization code order. Thanks! -JB- From daniel.daugherty at oracle.com Thu Jul 9 13:34:15 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 9 Jul 2020 09:34:15 -0400 Subject: RFR (S) 8249137: Remove CollectedHeap::obj_size In-Reply-To: References: Message-ID: <70984ebf-2717-c7b3-7076-12e2c8c7515c@oracle.com> Adding serviceability-dev at ... since a couple of JVM/TI files are changed in this RFR. Also, I moved the bug from hotspot/runtime -> hotspot/gc. Dan On 7/9/20 2:36 AM, Aleksey Shipilev wrote: > RFE: > https://bugs.openjdk.java.net/browse/JDK-8249137 > > It was added by JDK-8211270 to support old-style Shenandoah that needed a separate fwdptr slot. > After JDK-8224584 it does not need this anymore. Additionally, CH::obj_size may disagree with other > code that pokes at layout helper directly, for example GraphKit::new_instance. > > This also avoids a virtual call on some paths, although those paths are not very performance-sensitive. > > The patch is a simple series of few-liners: > > diff -r 9cc348ebdc82 src/hotspot/share/gc/shared/collectedHeap.cpp > --- a/src/hotspot/share/gc/shared/collectedHeap.cpp Thu Jul 09 04:32:30 2020 +0200 > +++ b/src/hotspot/share/gc/shared/collectedHeap.cpp Thu Jul 09 08:05:46 2020 +0200 > @@ -578,6 +578,2 @@ > > -size_t CollectedHeap::obj_size(oop obj) const { > - return obj->size(); > -} > - > uint32_t CollectedHeap::hash_oop(oop obj) const { > diff -r 9cc348ebdc82 src/hotspot/share/gc/shared/collectedHeap.hpp > --- a/src/hotspot/share/gc/shared/collectedHeap.hpp Thu Jul 09 04:32:30 2020 +0200 > +++ b/src/hotspot/share/gc/shared/collectedHeap.hpp Thu Jul 09 08:05:46 2020 +0200 > @@ -495,4 +495,2 @@ > > - virtual size_t obj_size(oop obj) const; > - > // Non product verification and debugging. > diff -r 9cc348ebdc82 src/hotspot/share/prims/jvmtiEnv.cpp > --- a/src/hotspot/share/prims/jvmtiEnv.cpp Thu Jul 09 04:32:30 2020 +0200 > +++ b/src/hotspot/share/prims/jvmtiEnv.cpp Thu Jul 09 08:05:46 2020 +0200 > @@ -488,3 +488,3 @@ > NULL_CHECK(mirror, JVMTI_ERROR_INVALID_OBJECT); > - *size_ptr = (jlong)Universe::heap()->obj_size(mirror) * wordSize; > + *size_ptr = (jlong)mirror->size() * wordSize; > return JVMTI_ERROR_NONE; > diff -r 9cc348ebdc82 src/hotspot/share/prims/jvmtiExport.cpp > --- a/src/hotspot/share/prims/jvmtiExport.cpp Thu Jul 09 04:32:30 2020 +0200 > +++ b/src/hotspot/share/prims/jvmtiExport.cpp Thu Jul 09 08:05:46 2020 +0200 > @@ -1067,3 +1067,3 @@ > _jobj = (jobject)to_jobject(obj); > - _size = Universe::heap()->obj_size(obj) * wordSize; > + _size = obj->size() * wordSize; > }; > diff -r 9cc348ebdc82 src/hotspot/share/prims/whitebox.cpp > --- a/src/hotspot/share/prims/whitebox.cpp Thu Jul 09 04:32:30 2020 +0200 > +++ b/src/hotspot/share/prims/whitebox.cpp Thu Jul 09 08:05:46 2020 +0200 > @@ -389,3 +389,3 @@ > oop p = JNIHandles::resolve(obj); > - return Universe::heap()->obj_size(p) * HeapWordSize; > + return p->size() * HeapWordSize; > WB_END > > Testing: tier{1,2}; jdk-submit (running) > From patricio.chilano.mateo at oracle.com Thu Jul 9 15:00:23 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Thu, 9 Jul 2020 12:00:23 -0300 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <4c43bc2c-551e-d042-8fa7-b63e755888d1@oracle.com> <8c387550-a06e-6320-f453-0e898486befb@oss.nttdata.com> <19cfa2a7-f441-425d-5623-1c11c6a870b6@oss.nttdata.com> <237e8d26-d185-1008-6e07-5b9f36314534@oracle.com> <2a9ba1f8-b1f7-cb8c-fe11-56dd174d9d58@oracle.com> Message-ID: Hi Yasumasa, On 7/9/20 9:30 AM, Yasumasa Suenaga wrote: > On 2020/07/09 17:58, David Holmes wrote: >> Hi Yasumasa, >> >> On 9/07/2020 10:25 am, Yasumasa Suenaga wrote: >>> Hi Dan, >>> >>> Thanks for your comment! >>> I uploaded new webrev: >>> >>> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.09/ >>> ?? Diff from previous webrev: >>> http://hg.openjdk.java.net/jdk/submit/rev/5d167adf8524 >>> >>> I saw similar build errors in libOneGetThreadListStackTraces.cpp on >>> Windows. >>> This webrev fixes them. >> >> You shouldn't use %p as it may not be portable. In the VM we use >> INTPTR_FORMAT and convert the arg using p2i. I don't know what exists >> in the testing code. > > I replaced %p to %lx, and also cast values to unsigned long [1] [2], > but the test on submit repo was failed. > Can anyone share details of > mach5-one-ysuenaga-JDK-8242428-20200709-1030-12500928 ? These are the errors I see for the macOS build: ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:53:14: error: format specifies type 'long long' but the argument has type 'jlocation' (aka 'long') [-Werror,-Wformat] ????????????? fi1->location, fi2->location); ????????????? ^~~~~~~~~~~~~ ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:53:29: error: format specifies type 'long long' but the argument has type 'jlocation' (aka 'long') [-Werror,-Wformat] ????????????? fi1->location, fi2->location); ???????????????????????????? ^~~~~~~~~~~~~ These are the ones I see for the Windows build: ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): warning C4311: 'type cast': pointer truncation from 'jmethodID' to 'unsigned long' ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): warning C4302: 'type cast': truncation from 'jmethodID' to 'unsigned long' ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): warning C4311: 'type cast': pointer truncation from 'jmethodID' to 'unsigned long' ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): warning C4302: 'type cast': truncation from 'jmethodID' to 'unsigned long' ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): warning C4311: 'type cast': pointer truncation from 'jthread' to 'unsigned long' ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): warning C4302: 'type cast': truncation from 'jthread' to 'unsigned long' ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): warning C4311: 'type cast': pointer truncation from 'jthread' to 'unsigned long' ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): warning C4302: 'type cast': truncation from 'jthread' to 'unsigned long' You will probably want to use the macros defined in src/hotspot/share/utilities/globalDefinitions.hpp. Let me know if you need me to test something. Thanks, Patricio > Thanks, > > Yasumasa > > > [1] http://hg.openjdk.java.net/jdk/submit/rev/dfca51958217 > [2] http://hg.openjdk.java.net/jdk/submit/rev/3665361fa91b > > >> David >> ----- >> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2020/07/09 1:42, Daniel D. Daugherty wrote: >>>> ?> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ >>>> >>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>> ???? No comments. >>>> >>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>> ???? L1159: ? Thread *current_thread = Thread::current(); >>>> ???????? Please add "#ifdef ASSERT" above and "#endif" below since >>>> ???????? current_thread is only used for the assert() in this >>>> function. >>>> >>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>> ???? L549: ???????????????????????????? jthread java_thread, jint >>>> max_frame_count) >>>> ???? L552: ????? _jthread(java_thread), >>>> ???????? Please: s/java_thread/thread/ on both lines. >>>> >>>> src/hotspot/share/runtime/vmOperations.hpp >>>> ???? No comments. >>>> >>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>> >>>> ???? No comments. >>>> >>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>> >>>> ???? L27: #include >>>> ???????? This include is out of order; should be first in the list. >>>> >>>> ???? This file doesn't compile on my MBP13: >>>> >>>> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:49:14: >>>> error: format specifies type 'unsigned long' but the argument has >>>> type 'jmethodID' (aka '_jmethodID *') [-Werror,-Wformat] >>>> ????????????? fi1->method, fi2->method); >>>> ????????????? ^~~~~~~~~~~ >>>> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:49:27: >>>> error: format specifies type 'unsigned long' but the argument has >>>> type 'jmethodID' (aka '_jmethodID *') [-Werror,-Wformat] >>>> ????????????? fi1->method, fi2->method); >>>> ?????????????????????????? ^~~~~~~~~~~ >>>> ???? 2 errors generated. >>>> >>>> ???? This change made it compile on my MBP13, but that may break it on >>>> ???? other platforms: >>>> >>>> ???? $ hg diff >>>> ???? diff -r 560847c69fbe >>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>> ???? --- >>>> a/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>> Wed Jul 08 12:13:32 2020 -0400 >>>> ???? +++ >>>> b/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>> Wed Jul 08 12:40:42 2020 -0400 >>>> ???? @@ -46,7 +46,7 @@ >>>> ?? ?? ? if (fi1->method != fi2->method) { /* jvmtiFrameInfo::method */ >>>> ?? ?? ??? snprintf(err_msg, sizeof(err_msg), >>>> ?? ? ? ??????????? "method is different: fi1 = %lx, fi2 = %lx", >>>> ???? -???????????? fi1->method, fi2->method); >>>> ???? +???????????? (unsigned long) fi1->method, (unsigned long) >>>> fi2->method); >>>> ?? ?? ??? env->FatalError(err_msg); >>>> ?? ?? ? } else if (fi1->location != fi2->location) { /* >>>> jvmtiFrameInfo::location */ >>>> ?? ?? ??? snprintf(err_msg, sizeof(err_msg), >>>> >>>> ???? I'm not sure of the right platform independent way to output >>>> ???? the 'method' field. >>>> >>>> Dan >>>> >>>> >>>> On 7/8/20 4:04 AM, Yasumasa Suenaga wrote: >>>>> Hi David, >>>>> >>>>> On 2020/07/08 15:27, David Holmes wrote: >>>>>> Hi Yasumasa, >>>>>> >>>>>> On 7/07/2020 6:54 pm, Yasumasa Suenaga wrote: >>>>>>> Hi David, Serguei, >>>>>>> >>>>>>> Serguei, thank you for replying even though you are on vacaiton! >>>>>>> >>>>>>> I uploaded new webrev: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.07/ >>>>>>> ?? Diff from previous webrev: >>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/77243b1dcbfe >>>>>>> >>>>>>> c'tor of GetSingleStackTraceClosure has jthread argument in this >>>>>>> webrev. >>>>>>> Also it does not contain testcase for GetThreadListStackTraces >>>>>>> with all threads, and OneGetThreadListStackTraces would test >>>>>>> main thread only. >>>>>> >>>>>> All those changes are fine in principle for me. One nit/suggestion: >>>>>> >>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>> >>>>>> ??544?? jthread _java_thread; >>>>>> >>>>>> elsewhere "java_thread" refers to a JavaThread, so to avoid >>>>>> confusion may I suggest this member be named _jthread. >>>>> >>>>> I uploaded new webrev: >>>>> >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ >>>>> ? Diff from previous webrev: >>>>> http://hg.openjdk.java.net/jdk/submit/rev/ca6263dbdc87 >>>>> >>>>> >>>>>> I'm going to be away for the next couple of days - sorry - but >>>>>> will try to check email on this if I can. >>>>> >>>>> Thanks! >>>>> >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2020/07/07 15:13, David Holmes wrote: >>>>>>>> On 7/07/2020 2:57 pm, Yasumasa Suenaga wrote: >>>>>>>>> Hi David, >>>>>>>>> >>>>>>>>> On 2020/07/07 11:31, David Holmes wrote: >>>>>>>>>> Hi Yasumasa, >>>>>>>>>> >>>>>>>>>> Hard to keep up with the changes - especially without >>>>>>>>>> incremental webrevs. >>>>>>>>> >>>>>>>>> Sorry, I will upload diff from previous webrev in the next. >>>>>>>>> >>>>>>>>> >>>>>>>>>> If GetSingleStackTraceClosure also took the jthread as a >>>>>>>>>> constructor arg, then you wouldn't need to recreate a JNI >>>>>>>>>> local handle when calling _collector.fill_frames. It's a >>>>>>>>>> small simplification and not essential at this stage. >>>>>>>>> >>>>>>>>> I think we should get jthread from an argument of do_thread() >>>>>>>>> because do_thread() would pass the thread which are stopped >>>>>>>>> certainly. >>>>>>>>> It might be simplification if we pass _calling_thread to >>>>>>>>> MultipleStackTracesCollector. `jthread` is only needed to >>>>>>>>> store jvmtiStackInfo.thread . What do you think? >>>>>>>> >>>>>>>> I'm not quite sure what you mean. >>>>>>>> >>>>>>>> I think there is a bit of a design wart with direct handshakes >>>>>>>> in that do_thread takes the target JavaThread as an argument. >>>>>>>> That's useful in a case where you want a HandshakeClosure that >>>>>>>> can be applied to multiple threads, but that's not typically >>>>>>>> what is needed with direct handshakes - there is only a single >>>>>>>> target. With a single-target HandshakeClosure you can capture >>>>>>>> all the "target" information for the operation in the closure >>>>>>>> instance. So if the actual do_thread operation wants the >>>>>>>> jthread corresponding to the target thread then we can store >>>>>>>> that in the closure rather than recomputing it (you could >>>>>>>> assert it is the same but that seems overkill to me). >>>>>>>> >>>>>>>>> >>>>>>>>>> For the test ... I don't see how >>>>>>>>>> Java_GetThreadListStackTraces_checkCallStacks is a valid >>>>>>>>>> test. It gets the stacks of all live threads, then uses that >>>>>>>>>> information to use GetThreadListStackTraces to get the stack >>>>>>>>>> for the same set of threads through a different API. It then >>>>>>>>>> compares the two sets of stacks for each thread expecting >>>>>>>>>> them to be the same, but that need only be the case for the >>>>>>>>>> main thread. Other threads could potentially have a different >>>>>>>>>> stack (e.g. if this test is run with JFR enabled there will >>>>>>>>>> be additional threads found.) Further I would have expected >>>>>>>>>> that there already exist tests that check that, for a given >>>>>>>>>> thread (which may be suspended or known to be blocked) the >>>>>>>>>> same stack is found through the two different APIs. >>>>>>>>> >>>>>>>>> vmTestbase/nsk/jvmti/unit/GetAllStackTraces/getallstktr001 >>>>>>>>> would check all of threads via GetThreadListStackTraces() and >>>>>>>>> GetAllStackTraces(), so we might be able to remove >>>>>>>>> GetThreadListStackTraces.java from this webrev. >>>>>>>> >>>>>>>> Yes. The existing test only examines a set of test threads that >>>>>>>> are all blocked on a raw monitor. You do not need to duplicate >>>>>>>> that test. >>>>>>>> >>>>>>>>> OTOH we don't have testcase for GetThreadListStackTraces() >>>>>>>>> with thread_count == 1, so we need to testcase for it (it is >>>>>>>>> OneGetThreadListStackTraces.java) It would check whether the >>>>>>>>> state of target thread is "waiting" before JNI call to call >>>>>>>>> GetThreadListStackTraces(), >>>>>>>> >>>>>>>> Yes we need to test the special cases introduced by your >>>>>>>> changes - totally agree - and OneGetThreadListStackTraces.java >>>>>>>> is a good test for that. >>>>>>>> >>>>>>>>> and also I expect it would not be run with JFR. (it is not >>>>>>>>> described @run) >>>>>>>> >>>>>>>> The arguments to run with JFR (or a bunch of other things) can >>>>>>>> be passed to the jtreg test harness to be applied to all tests. >>>>>>>> >>>>>>>>> Of course we can check GetThreadListStackTraces() with main >>>>>>>>> thread, but it is not the test for direct handshake for other >>>>>>>>> thread. >>>>>>>> >>>>>>>> Right - that test already exists as per the above. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> David >>>>>>>>>> ----- >>>>>>>>>> >>>>>>>>>> On 6/07/2020 11:29 pm, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi Serguei, >>>>>>>>>>> >>>>>>>>>>> Thanks for your comment! >>>>>>>>>>> >>>>>>>>>>> I think C++ is more simple to implement the test agent as >>>>>>>>>>> you said. >>>>>>>>>>> So I implement it in C++ in new webrev. Could you review again? >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.06/ >>>>>>>>>>> >>>>>>>>>>> Also I refactored libGetThreadListStackTraces.cpp, and I've >>>>>>>>>>> kept exception check after IsSameObject(). >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2020/07/06 16:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>> >>>>>>>>>>>> Thank you for the update. >>>>>>>>>>>> I think, a pending exception after IsSameObject needs to be >>>>>>>>>>>> checked. >>>>>>>>>>>> >>>>>>>>>>>> The checkStackInfo() needs one more refactoring as I've >>>>>>>>>>>> already suggested. >>>>>>>>>>>> The body of the loop at L68-L78 should be converted to a >>>>>>>>>>>> function check_frame_info. >>>>>>>>>>>> The si1->frame_buffer[i] and si2->frame_buffer[i] will be >>>>>>>>>>>> passed as fi1 and fi2. >>>>>>>>>>>> The index can be passed as well. >>>>>>>>>>>> I'm still suggesting to simplify the local exception_msg to >>>>>>>>>>>> something shorter like err_msg or exc_msg. >>>>>>>>>>>> >>>>>>>>>>>> I'm not sure using fatal is right here: >>>>>>>>>>>> >>>>>>>>>>>> This fragment looks strange: >>>>>>>>>>>> >>>>>>>>>>>> ? 152???? if ((*env)->IsSameObject(env, >>>>>>>>>>>> stack_info[i].thread, thread)) { >>>>>>>>>>>> ? 153?????? target_info = &stack_info[i]; >>>>>>>>>>>> ? 154?????? break; >>>>>>>>>>>> ? 155???? } else if ((*env)->ExceptionOccurred(env)) { >>>>>>>>>>>> ? 156 (*env)->ExceptionDescribe(env); >>>>>>>>>>>> ? 157?????? (*env)->FatalError(env, __FILE__); >>>>>>>>>>>> ? 158???? } >>>>>>>>>>>> >>>>>>>>>>>> I expected it to be: >>>>>>>>>>>> >>>>>>>>>>>> ??? jboolean same = (*env)->IsSameObject(env, >>>>>>>>>>>> stack_info[i].thread, thread); >>>>>>>>>>>> ??? if ((*env)->ExceptionOccurred(env)) { >>>>>>>>>>>> ????? (*env)->ExceptionDescribe(env); >>>>>>>>>>>> ????? (*env)->FatalError(env, __FILE__); >>>>>>>>>>>> ??? } >>>>>>>>>>>> ??? if (same) { >>>>>>>>>>>> ????? target_info = &stack_info[i]; >>>>>>>>>>>> ????? break; >>>>>>>>>>>> ??? } >>>>>>>>>>>> >>>>>>>>>>>> Would it better to port this agent to C++ to simplify this >>>>>>>>>>>> code nicer? >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Serguei >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 7/5/20 06:13, Yasumasa Suenaga wrote: >>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>> I refactored testcase. Could you review again? >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.05/ >>>>>>>>>>>>> >>>>>>>>>>>>> It would check Java exception after IsSameObject() call. >>>>>>>>>>>>> Does it need? >>>>>>>>>>>>> Any exceptions are not described in JNI document[1], and >>>>>>>>>>>>> JNI implementation (jni_IsSameObject()) does not seem to >>>>>>>>>>>>> throw it. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> [1] >>>>>>>>>>>>> https://docs.oracle.com/en/java/javase/14/docs/specs/jni/functions.html#issameobject >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 2020/07/05 14:46, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Okay, thanks. >>>>>>>>>>>>>> Then I'm okay to keep the GetSingleStackTraceClosure. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c.html >>>>>>>>>>>>>> >>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c.html >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> I'm not sure the function 'is_same_thread() is needed. >>>>>>>>>>>>>> Why do not use the JNI IsSameObject instead? >>>>>>>>>>>>>> >>>>>>>>>>>>>> It seems to be a typo at L132 and L137. >>>>>>>>>>>>>> You, probably. did not want to print the same information >>>>>>>>>>>>>> for stack_info_1[i].frame_buffer[j].XXX twice. >>>>>>>>>>>>>> >>>>>>>>>>>>>> The code at lines 112-142 is not readable. >>>>>>>>>>>>>> I'd suggest to make a couple of refactoring steps. >>>>>>>>>>>>>> >>>>>>>>>>>>>> First step to simplify this a little bit would be with >>>>>>>>>>>>>> some renaming and getting rid of indexes: >>>>>>>>>>>>>> >>>>>>>>>>>>>> ?? 71 char err_msg[EXCEPTION_MSG_LEN] = {0}; >>>>>>>>>>>>>> ??... >>>>>>>>>>>>>> ??112?? /* Iterate all jvmtiStackInfo to check */ >>>>>>>>>>>>>> ??113?? for (i = 0; i < num_threads, *exception_msg != >>>>>>>>>>>>>> '\0'; i++) { >>>>>>>>>>>>>> ????????? jvmtiStackInfo *si1 = stack_info_1[i]; >>>>>>>>>>>>>> ????????? jvmtiStackInfo *si2 = stack_info_2[i]; >>>>>>>>>>>>>> ??114???? if (!IsSameObject(env, si1.thread, si2.thread)) >>>>>>>>>>>>>> { /* jvmtiStackInfo::thread */ >>>>>>>>>>>>>> ??115?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>> ??116??????????????? "thread[%d] is different: >>>>>>>>>>>>>> stack_info_1 = %p, stack_info_2 = %p", >>>>>>>>>>>>>> ??117??????????????? i, sinfo1.thread, sinfo2.thread); >>>>>>>>>>>>>> ??118???? } else if (si1.state != si2.state) { /* >>>>>>>>>>>>>> jvmtiStackInfo::state */ >>>>>>>>>>>>>> ??119?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>> ??120??????????????? "state[%d] is different: >>>>>>>>>>>>>> stack_info_1 = %d, stack_info_2 = %d", >>>>>>>>>>>>>> ??121??????????????? i, si1.state, si2.state); >>>>>>>>>>>>>> ??122???? } else if (si1.frame_count != si2.frame_count) >>>>>>>>>>>>>> { /* jvmtiStackInfo::frame_count */ >>>>>>>>>>>>>> ??123?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>> ??124??????????????? "frame_count[%d] is different: >>>>>>>>>>>>>> stack_info_1 = %d, stack_info_2 = %d", >>>>>>>>>>>>>> ??125??????????????? i, si1.frame_count, si2.frame_count); >>>>>>>>>>>>>> ??126???? } else { >>>>>>>>>>>>>> ??127?????? /* Iterate all jvmtiFrameInfo to check */ >>>>>>>>>>>>>> ??128?????? for (j = 0; j < si1.frame_count; j++) { >>>>>>>>>>>>>> ??129???????? if (si1.frame_buffer[j].method != >>>>>>>>>>>>>> si1.frame_buffer[j].method) { /* jvmtiFrameInfo::method */ >>>>>>>>>>>>>> ??130?????????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>> ??131??????????????????? "thread [%d] >>>>>>>>>>>>>> frame_buffer[%d].method is different: stack_info_1 = %lx, >>>>>>>>>>>>>> stack_info_2 = %lx", >>>>>>>>>>>>>> ??132??????????????????? i, j, >>>>>>>>>>>>>> si1.frame_buffer[j].method, si2.frame_buffer[j].method); >>>>>>>>>>>>>> ??133?????????? break; >>>>>>>>>>>>>> ??134???????? } else if (si1.frame_buffer[j].location != >>>>>>>>>>>>>> si1.frame_buffer[j].location) { /* >>>>>>>>>>>>>> jvmtiFrameInfo::location */ >>>>>>>>>>>>>> ??135?????????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>> ??136??????????????????? "thread [%d] >>>>>>>>>>>>>> frame_buffer[%d].location is different: stack_info_1 = >>>>>>>>>>>>>> %ld, stack_info_2 = %ld", >>>>>>>>>>>>>> ??137??????????????????? i, j, >>>>>>>>>>>>>> si1.frame_buffer[j].location, si2.frame_buffer[j].location); >>>>>>>>>>>>>> ??138?????????? break; >>>>>>>>>>>>>> ??139???????? } >>>>>>>>>>>>>> ??140?????? } >>>>>>>>>>>>>> ??141???? } >>>>>>>>>>>>>> ??142?? } >>>>>>>>>>>>>> >>>>>>>>>>>>>> Another step would be to create functions that implement >>>>>>>>>>>>>> a body of each loop. >>>>>>>>>>>>>> You can use the same techniques to simplify similar place >>>>>>>>>>>>>> (L127-L138) in the libOneGetThreadListStackTraces.c. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 7/3/20 15:55, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I'm not an Oracle employee, so I cannot know real >>>>>>>>>>>>>>> request(s) from your customers. >>>>>>>>>>>>>>> However JDK-8201641 says Dynatrace has requested this >>>>>>>>>>>>>>> enhancement. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> BTW I haven't heared any request from my customers about >>>>>>>>>>>>>>> this. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 2020/07/04 4:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> This difference is not that big to care about. >>>>>>>>>>>>>>>> I feel this is really rare case and so, does not worth >>>>>>>>>>>>>>>> these complications. >>>>>>>>>>>>>>>> Do we have a real request from customers to optimize it? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 7/3/20 01:16, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Generally I agree with you, but I have concern about >>>>>>>>>>>>>>>>> the difference of the result of GetStackTrace() and >>>>>>>>>>>>>>>>> GetThreadListStackTraces(). >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> ? GetStackTrace: jvmtiFrameInfo >>>>>>>>>>>>>>>>> ? GetThreadListStackTraces: jvmtiStackInfo >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> jvmtiStackInfo contains thread state, and it is >>>>>>>>>>>>>>>>> ensured it is the state of the call stack. >>>>>>>>>>>>>>>>> If we want to get both call stack and thread state, we >>>>>>>>>>>>>>>>> need to suspend target thread, and call both >>>>>>>>>>>>>>>>> GetStackTrace() and GetThreadState(). Is it ok? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I was wondering if JDK-8201641 (parent ticket of this >>>>>>>>>>>>>>>>> change) needed them for profiling (dynatrace?) >>>>>>>>>>>>>>>>> If it is responsibility of JVMTI agent implementor, I >>>>>>>>>>>>>>>>> remove this closure. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 2020/07/03 16:45, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> After some thinking I've concluded that I do not like >>>>>>>>>>>>>>>>>> this optimization >>>>>>>>>>>>>>>>>> of the GetThreadListStackTraces with >>>>>>>>>>>>>>>>>> GetSingleStackTraceClosure. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> We may need more opinions on this but these are my >>>>>>>>>>>>>>>>>> points: >>>>>>>>>>>>>>>>>> ??- it adds some complexity and ugliness >>>>>>>>>>>>>>>>>> ??- a win is doubtful because it has to be a rare >>>>>>>>>>>>>>>>>> case, so that total overhead should not be high >>>>>>>>>>>>>>>>>> ??- if it is really high for some use cases then it >>>>>>>>>>>>>>>>>> is up to the user >>>>>>>>>>>>>>>>>> ??? to optimize it with using GetStackTrace instead >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> In such cases with doubtful overhead I usually prefer >>>>>>>>>>>>>>>>>> the simplicity. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Good examples where it makes sense to optimize are >>>>>>>>>>>>>>>>>> checks for target thread to be current thread. >>>>>>>>>>>>>>>>>> In such cases there is no need to suspend the target >>>>>>>>>>>>>>>>>> thread, or use a VMop/HandshakeClosure. >>>>>>>>>>>>>>>>>> For instance, please, see the Monitor functions with >>>>>>>>>>>>>>>>>> the check: (java_thread == calling_thread). >>>>>>>>>>>>>>>>>> Getting information for current thread is frequently >>>>>>>>>>>>>>>>>> used case, e.g. to get info at an event point. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 7/2/20 23:29, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>> Hi Dan, David, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could you review again? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/ >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> OneGetThreadListStackTraces.java in this webrev >>>>>>>>>>>>>>>>>>> would wait until thread state is transited to >>>>>>>>>>>>>>>>>>> "waiting" with spin wait. >>>>>>>>>>>>>>>>>>> CountDownLatch::await call as Dan pointed is fixed >>>>>>>>>>>>>>>>>>> in it :) >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Diff from webrev.03: >>>>>>>>>>>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/c9aeb7001e50 >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On 2020/07/03 14:15, David Holmes wrote: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>> On 2020/07/03 12:24, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>>>>>> On 7/2/20 10:50 PM, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>> Sorry I'm responding here without seeing latest >>>>>>>>>>>>>>>>>>>>>>> webrev but there is enough context I think ... >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>> Hi Dan, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>>>>>>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> I upload new webrev. Could you review again? >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>>>>>>>>>>>>>>>>> ???? L1542: ??? // Get stack trace with handshake >>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - please add a period at the end. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> L1591: *stack_info_ptr = op.stack_info(); >>>>>>>>>>>>>>>>>>>>>>>>> ???????? The return parameter should not be >>>>>>>>>>>>>>>>>>>>>>>>> touched unless the return >>>>>>>>>>>>>>>>>>>>>>>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> ???? old L1582: if (err == JVMTI_ERROR_NONE) { >>>>>>>>>>>>>>>>>>>>>>>>> ???????? Please restore this check. The return >>>>>>>>>>>>>>>>>>>>>>>>> parameter should not >>>>>>>>>>>>>>>>>>>>>>>>> ???????? be touched unless the return code in >>>>>>>>>>>>>>>>>>>>>>>>> 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> But op.stack_info() will return NULL if the >>>>>>>>>>>>>>>>>>>>>>> error is not JVMTI_ERROR_NONE. Are you (Dan) >>>>>>>>>>>>>>>>>>>>>>> concerned about someone passing in a >>>>>>>>>>>>>>>>>>>>>>> non-null/initialized out-pointer that will be >>>>>>>>>>>>>>>>>>>>>>> reset to NULL if there was an error? >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Actually the way we used to test this in POSIX >>>>>>>>>>>>>>>>>>>>>> tests is to call >>>>>>>>>>>>>>>>>>>>>> an API with known bad parameters and the return >>>>>>>>>>>>>>>>>>>>>> parameter ptr >>>>>>>>>>>>>>>>>>>>>> set to NULL. If the return parameter ptr was >>>>>>>>>>>>>>>>>>>>>> touched when an >>>>>>>>>>>>>>>>>>>>>> error should have been detected on an earlier >>>>>>>>>>>>>>>>>>>>>> parameter, then >>>>>>>>>>>>>>>>>>>>>> the test failed. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> L1272: ? if (!jt->is_exiting() && (thread_oop >>>>>>>>>>>>>>>>>>>>>>>>> != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second >>>>>>>>>>>>>>>>>>>>>>>>> expression. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>> ???? old L1532: _result = >>>>>>>>>>>>>>>>>>>>>>>>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>>>>>>>>>>>>>>>>>>>>>> ???????? This deletion of the _result field >>>>>>>>>>>>>>>>>>>>>>>>> threw me for a minute and then >>>>>>>>>>>>>>>>>>>>>>>>> ???? ? ? I figured out that the field is init >>>>>>>>>>>>>>>>>>>>>>>>> to JVMTI_ERROR_THREAD_NOT_ALIVE >>>>>>>>>>>>>>>>>>>>>>>>> ???????? in the constructor. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> ???? L1553: ? if (!jt->is_exiting() && >>>>>>>>>>>>>>>>>>>>>>>>> (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second >>>>>>>>>>>>>>>>>>>>>>>>> expression. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> ???? L64: startSignal.countDown(); >>>>>>>>>>>>>>>>>>>>>>>>> ???????? I was expecting this to be a call to >>>>>>>>>>>>>>>>>>>>>>>>> await() instead of >>>>>>>>>>>>>>>>>>>>>>>>> countDown(). What am I missing here? >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> ???????? I think this test might be passing by >>>>>>>>>>>>>>>>>>>>>>>>> accident right now, but... >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Main thread (which call JVMTI functions to >>>>>>>>>>>>>>>>>>>>>>>> test) should wait until test thread is ready. >>>>>>>>>>>>>>>>>>>>>>>> So main thread would wait startSignal, and test >>>>>>>>>>>>>>>>>>>>>>>> thread would count down. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> No! >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> The test thread that previously called >>>>>>>>>>>>>>>>>>>>>>> obj.wait() now calls latch.await(). >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> The main thread that previously called >>>>>>>>>>>>>>>>>>>>>>> obj.notify() now calls latch.countDown(). >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> The main thread continues to spin until it sees >>>>>>>>>>>>>>>>>>>>>>> the target is WAITING before proceeding with the >>>>>>>>>>>>>>>>>>>>>>> test. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> If I add spin wait to wait until transit target >>>>>>>>>>>>>>>>>>>>> thread state is WAITING (as following), we don't >>>>>>>>>>>>>>>>>>>>> need to call SuspendThread(). >>>>>>>>>>>>>>>>>>>>> Which is better? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> The original spin-wait loop checking for? WAITING >>>>>>>>>>>>>>>>>>>> is better because it is the only guarantee that the >>>>>>>>>>>>>>>>>>>> target thread is blocked where you need it to be. >>>>>>>>>>>>>>>>>>>> suspending the thread is racy as you don't know >>>>>>>>>>>>>>>>>>>> exactly where the suspend will hit. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>> /* Wait until the thread state transits to >>>>>>>>>>>>>>>>>>>>> "waiting" */ >>>>>>>>>>>>>>>>>>>>> while (th.getState() != Thread.State.WAITING) { >>>>>>>>>>>>>>>>>>>>> ???? Thread.onSpinWait(); >>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> For simplify, spin wait is prefer to >>>>>>>>>>>>>>>>>>>>> OneGetThreadListStackTraces.java in webrev.03. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Here's the flow as I see it: >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>> ?? - start worker thread >>>>>>>>>>>>>>>>>>>>>> ?? - startSignal.await() >>>>>>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>>>>>> ?? - startSignal.countDown() >>>>>>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>>>>>> ?? - stopSignal.await() >>>>>>>>>>>>>>>>>>>>>> ???? - worker is now blocked >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>> ?? - checkCallStacks(th) >>>>>>>>>>>>>>>>>>>>>> ?? - stopSignal.countDown() >>>>>>>>>>>>>>>>>>>>>> ???? - worker is now unblocked >>>>>>>>>>>>>>>>>>>>>> ?? - th.join >>>>>>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>>>>>> ?? - runs off the end of run() >>>>>>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>> ?? - run off the end of main() >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> ???? L92: jthreads = (jthread >>>>>>>>>>>>>>>>>>>>>>>>> *)malloc(sizeof(jthread) * num_threads); >>>>>>>>>>>>>>>>>>>>>>>>> ???????? You don't check for malloc() failure. >>>>>>>>>>>>>>>>>>>>>>>>> 'jthreads' is allocated but never freed. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> ???? L91: ? result = >>>>>>>>>>>>>>>>>>>>>>>>> (*jvmti)->SuspendThread(jvmti, thread); >>>>>>>>>>>>>>>>>>>>>>>>> ???????? Why are you suspending the thread? >>>>>>>>>>>>>>>>>>>>>>>>> GetAllStackTraces() and >>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() do not require the >>>>>>>>>>>>>>>>>>>>>>>>> target thread(s) >>>>>>>>>>>>>>>>>>>>>>>>> ???????? to be suspend. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> ???????? If you decide not to SuspendThread, >>>>>>>>>>>>>>>>>>>>>>>>> then you don't need the >>>>>>>>>>>>>>>>>>>>>>>>> AddCapabilities or the ResumeThread calls. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Test thread might not be entered following code >>>>>>>>>>>>>>>>>>>>>>>> (stopSignal.await()). We might see deferent >>>>>>>>>>>>>>>>>>>>>>>> call stack between GetAllStackTraces() and >>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces(). We cannot control >>>>>>>>>>>>>>>>>>>>>>>> to freeze call stack of test thread in Java code. >>>>>>>>>>>>>>>>>>>>>>>> (I didn't use SuspendThread() at first, but I >>>>>>>>>>>>>>>>>>>>>>>> saw some errors which causes in above.) >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> So we need to call SuspendThread() to ensure we >>>>>>>>>>>>>>>>>>>>>>>> can see same call stack. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> If you are checking that the thread is in state >>>>>>>>>>>>>>>>>>>>>>> WAITING then it cannot escape from that state >>>>>>>>>>>>>>>>>>>>>>> and you can sample the stack multiple times from >>>>>>>>>>>>>>>>>>>>>>> any API and get the same result. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I suspect the errors you saw were from the >>>>>>>>>>>>>>>>>>>>>>> apparent incorrect use of the CountDownLatch. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> With the flow outlined above, the worker thread >>>>>>>>>>>>>>>>>>>>>> should be >>>>>>>>>>>>>>>>>>>>>> nicely blocked in stopSignal.await() when stuff >>>>>>>>>>>>>>>>>>>>>> is sampled. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Updates look fine - thanks. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> One minor nit: >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> 1274 _collector.allocate_and_fill_stacks(1); >>>>>>>>>>>>>>>>>>>>>>>>>>> 1275 _collector.set_result(JVMTI_ERROR_NONE); >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> In the other places where you use _collector >>>>>>>>>>>>>>>>>>>>>>>>>>> you rely on result being initialized to >>>>>>>>>>>>>>>>>>>>>>>>>>> JVMTI_ERROR_NONE, rather than setting it >>>>>>>>>>>>>>>>>>>>>>>>>>> directly after allocate_and_fill_stacks(). >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 820 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> assert(SafepointSynchronize::is_at_safepoint() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 821 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 822 current_thread == >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 823 "at safepoint / handshake or >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> necessary, as even if the target is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> suspended we must still be at a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> safepoint or in a handshake with it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Makes me wonder if we used to allow a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> racy stacktrace operation on a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> suspended thread, assuming it would >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> This function >>>>>>>>>>>>>>>>>>>>>>>>>>>> (JvmtiEnvBase::get_stack_trace()) can be >>>>>>>>>>>>>>>>>>>>>>>>>>>> called to get own stack trace. For example, >>>>>>>>>>>>>>>>>>>>>>>>>>>> we can call GetStackTrace() for current >>>>>>>>>>>>>>>>>>>>>>>>>>>> thread at JVMTI event. >>>>>>>>>>>>>>>>>>>>>>>>>>>> So I changed assert as below: >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>> ??820 assert(current_thread == java_thread || >>>>>>>>>>>>>>>>>>>>>>>>>>>> ??821 >>>>>>>>>>>>>>>>>>>>>>>>>>>> SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == >>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>> ??823 "call by myself / at safepoint / at >>>>>>>>>>>>>>>>>>>>>>>>>>>> handshake"); >>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Yep good catch. I hope current tests caught >>>>>>>>>>>>>>>>>>>>>>>>>>> that. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> They would be tested in >>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ >>>>>>>>>>>>>>>>>>>>>>>>>> (own call stacks), and getstacktr003 (call >>>>>>>>>>>>>>>>>>>>>>>>>> stacks in other thread). >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Speaking of tests ... >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> In the native code I think you need to check >>>>>>>>>>>>>>>>>>>>>>>>>>> the success of all JNI methods that can >>>>>>>>>>>>>>>>>>>>>>>>>>> throw exceptions - otherwise I believe the >>>>>>>>>>>>>>>>>>>>>>>>>>> tests may trigger warnings if -Xcheck:jni is >>>>>>>>>>>>>>>>>>>>>>>>>>> used with them. See for example: >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> I updated testcases to check JNI and JVMTI >>>>>>>>>>>>>>>>>>>>>>>>>> function calls. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> In the Java code the target thread: >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 45 public void run() { >>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 46 try { >>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 47 synchronized (lock) { >>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 48 lock.wait(); >>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 49 System.out.println("OK"); >>>>>>>>>>>>>>>>>>>>>>>>>>> 50?????????? } >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> is potentially susceptible to spurious >>>>>>>>>>>>>>>>>>>>>>>>>>> wakeups. Using a CountDownLatch would be >>>>>>>>>>>>>>>>>>>>>>>>>>> robust. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If so, we can call make_local() in L1272 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> without JavaThread (or we can pass >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> current thread to make_local()). Is it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> right? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry I got confused, _calling_thread may >>>>>>>>>>>>>>>>>>>>>>>>>>>>> not be the current thread as we could be >>>>>>>>>>>>>>>>>>>>>>>>>>>>> executing the handshake in the target >>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread itself. So the ResourceMark is >>>>>>>>>>>>>>>>>>>>>>>>>>>>> correct as-is (implicitly for current >>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread). >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> The argument to fill_frames will be used >>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the jvmtiStackInfo and passed back to >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the _calling_thread, so it must be created >>>>>>>>>>>>>>>>>>>>>>>>>>>>> via make_local(_calling_thread, ...) as >>>>>>>>>>>>>>>>>>>>>>>>>>>>> you presently have. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you for reviewing! I will update >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new webrev tomorrow. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> : public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 498 class VM_GetAllStackTraces : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 500 JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 502 MultipleStackTracesCollector >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of another class like that as it may >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not be on the stack. I think >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector should >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not extend any allocation class, and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should always be embedded directly in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??? : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class GetAllStackTraces : public >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I updated webrev for 8242428. Could >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change migrate to use direct >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> handshake for GetStackTrace() and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() (when >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread_count == 1). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks really good now! I only >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a few nits below. There is one >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thing I don't like about it but it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> requires a change to the main >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Handshake logic to address - in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JvmtiEnv::GetThreadListStackTraces you >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have to create a ThreadsListHandle to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> convert the jthread to a JavaThread, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but then the Handshake::execute_direct >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates another ThreadsListHandle >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> internally. That's a waste. I will >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> discuss with Robbin and file a RFE to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have an overload of execute_direct >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that takes an existing TLH. Actually >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it's worse than that because we have >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> another TLH in use at the entry point >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for the JVMTI functions, so I think >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there may be some scope for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> simplifying the use of TLH instances - >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> future RFE. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??451 GetStackTraceClosure(JvmtiEnv >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *env, jint start_depth, jint max_count, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??452 jvmtiFrameInfo* frame_buffer, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jint* count_ptr) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??453???? : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??454 _env(env), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _start_depth(start_depth), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _max_count(max_count), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??455 _frame_buffer(frame_buffer), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _count_ptr(count_ptr), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??456 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Nit: can you do one initializer per >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line please. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> : public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??498 class VM_GetAllStackTraces : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??500 JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??502 MultipleStackTracesCollector >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of another class like that as it may >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not be on the stack. I think >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector should >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not extend any allocation class, and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should always be embedded directly in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 481 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector(JvmtiEnv >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *env, jint max_frame_count) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??483 _max_frame_count = max_frame_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??484 _frame_count_total = 0; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??485 _head = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??486 _stack_info = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??487 _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As you are touching this can you >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change it to use an initializer list >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as you did for the HandshakeClosure, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and please keep one item per line. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??820 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> assert(SafepointSynchronize::is_at_safepoint() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??821 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??823 "at safepoint / handshake or >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> necessary, as even if the target is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> suspended we must still be at a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> safepoint or in a handshake with it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Makes me wonder if we used to allow a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> racy stacktrace operation on a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> suspended thread, assuming it would >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1269 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is frustrating that this entire >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call chain started with a jthread >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reference, which we converted to a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JavaThread, only to eventually need to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> convert it back to a jthread! I think >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there is some scope for simplification >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> here but not as part of this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm still pondering the test logic but >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wanted to send this now. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetThreadListStackTrace (for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces) and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetAllStackTraces (for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetAllStackTraces) have inherited >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetMultipleStackTraces VM >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operation which provides the feature >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to generate jvmtiStackInfo. I >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> modified VM_GetMultipleStackTraces to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a normal C++ class to share with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HandshakeClosure for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (GetSingleStackTraceClosure). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I added new testcases which test >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread_count == 1 and with all threads. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jvmti >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jdwp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jdi vmTestbase/nsk/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? JBS: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? webrev: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change replace following VM >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operations to direct handshake. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (GetFrameLocation()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (GetThreadListStackTrace()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTrace() uses >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> direct handshake if thread count == >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1. In other case (thread count > 1), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it would be performed as VM >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Caller of VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (JvmtiEnvThreadState::reset_current_location()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> might be called at safepoint. So I >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> added safepoint check in its caller. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jvmti >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jdwp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jdi vmTestbase/ns >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I tested it on submit repo, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then it has execution error >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> due to dependency error. So I think >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it does not occur by this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>> From daniel.daugherty at oracle.com Thu Jul 9 16:25:41 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 9 Jul 2020 12:25:41 -0400 Subject: Some clarification about JVMTI_EVENT_THREAD_START and PRIMORDIAL phase In-Reply-To: References: Message-ID: <6287d398-5828-762d-3609-2767b684186d@oracle.com> On 7/9/20 9:07 AM, Jaroslav Bachor?k wrote: > Hello, > > Recently, after backporting JDK-8233197 to JDK8u I received a report > from Sergey Nazarkin (cc) that the backport broke JVMTI spec basically > emitting JVMTI_EVENT_THREAD_START events in primordial phase while the > spec states that it should be emitted only in either start or live > phase. > > But if I am reading this correctly > (https://hg.openjdk.java.net/jdk8u/jdk8udev/hotspot/file/02b4fd2f9041/src/share/vm/prims/jvmtiEventController.cpp#l418) > the spec is also violated there. > > EARLY_EVENT_BITS are including THREAD_START_BIT and it is not filtered > out by THREAD_FILTERED_EVENT_BITS so it seems perfectly valid for > JVMTI_EVENT_THREAD_START to appear during PRIMORDIAL phase. > > So I would like to get clarification before going in and start > tinkering with the JVM initialization code order. > > Thanks! > > -JB- Hi Jaroslav! It has been a long time my friend... :-) It looks like that change was made by this Jigsaw change: ??? JDK-8072745 Re-examine JVM TI execution phases and expectations ??????????????? on what is allowed in each phase ??? https://bugs.openjdk.java.net/browse/JDK-8072745 These diffs in the bug reported jumped out at me: @@ -1022,6 +1032,10 @@ ?void JvmtiExport::post_thread_start(JavaThread *thread) { ???assert(thread->thread_state() == _thread_in_vm, "must be in vm state"); + if (JvmtiEnv::get_phase() <= JVMTI_PHASE_PRIMORDIAL) { + return; + } + ???EVT_TRIG_TRACE(JVMTI_EVENT_THREAD_START, ("JVMTI [%s] Trg Thread Start event triggered", ???????????????????????JvmtiTrace::safe_get_thread_name(thread))); @@ -1050,6 +1064,10 @@ ?void JvmtiExport::post_thread_end(JavaThread *thread) { + if (JvmtiEnv::get_phase() <= JVMTI_PHASE_PRIMORDIAL) { + return; + } + ???EVT_TRIG_TRACE(JVMTI_EVENT_THREAD_END, ("JVMTI [%s] Trg Thread End event triggered", ???????????????????????JvmtiTrace::safe_get_thread_name(thread))); Here's the order of phases from jvmti.h: ??? /* Phases of execution */ typedef enum { ??? JVMTI_PHASE_ONLOAD = 1, ??? JVMTI_PHASE_PRIMORDIAL = 2, ??? JVMTI_PHASE_START = 6, ??? JVMTI_PHASE_LIVE = 4, ??? JVMTI_PHASE_DEAD = 8 } jvmtiPhase; However, the above diffs prevent the JVMTI_EVENT_THREAD_START and JVMTI_EVENT_THREAD_END events from being posted earlier than the JVMTI_PHASE_START phase... Unfortunately, I can't find the actual changeset pushed to the Jigsaw report for JDK-8072745. When you look at the current JvmtiExport::post_thread_start() and JvmtiExport::post_thread_end() functions, you can see this: ??? 1: void JvmtiExport::post_thread_start(JavaThread *thread) { 36508:?? if (JvmtiEnv::get_phase() < JVMTI_PHASE_PRIMORDIAL) { 36508:???? return; 36508:?? } ??? 1: void JvmtiExport::post_thread_end(JavaThread *thread) { 36508:?? if (JvmtiEnv::get_phase() < JVMTI_PHASE_PRIMORDIAL) { 36508:???? return; 36508:?? } $ hg log -r 36508 changeset:?? 36508:5f9eee6b383b user:??????? alanb date:??????? Thu Mar 17 19:04:01 2016 +0000 summary:???? 8142968: Module System implementation So the Jigsaw integration pushed a slightly different change than is shown in JDK-8072745. I don't know whether the fix for JDK-8072745 evolved before being pushed to the Jigsaw repos or whether another JBS issue was used to modify the code after JDK-8072745 was pushed. Short version: The semantics of the JVMTI_EVENT_THREAD_START and JVMTI_EVENT_THREAD_END were changed by Jigsaw, but the JVM/TI spec was not updated to reflect those changes. I've filed a new bug to track this issue: ??? JDK-8249158 THREAD_START and THREAD_END event posting changed ??????????????? by modules without updating the spec ??? https://bugs.openjdk.java.net/browse/JDK-8249158 Dan From patricio.chilano.mateo at oracle.com Thu Jul 9 17:18:31 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Thu, 9 Jul 2020 14:18:31 -0300 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <4c43bc2c-551e-d042-8fa7-b63e755888d1@oracle.com> <8c387550-a06e-6320-f453-0e898486befb@oss.nttdata.com> <19cfa2a7-f441-425d-5623-1c11c6a870b6@oss.nttdata.com> <237e8d26-d185-1008-6e07-5b9f36314534@oracle.com> <2a9ba1f8-b1f7-cb8c-fe11-56dd174d9d58@oracle.com> Message-ID: <7ad907cb-24ef-aa55-972a-0a3f63576568@oracle.com> On 7/9/20 12:00 PM, Patricio Chilano wrote: > Hi Yasumasa, > > On 7/9/20 9:30 AM, Yasumasa Suenaga wrote: >> On 2020/07/09 17:58, David Holmes wrote: >>> Hi Yasumasa, >>> >>> On 9/07/2020 10:25 am, Yasumasa Suenaga wrote: >>>> Hi Dan, >>>> >>>> Thanks for your comment! >>>> I uploaded new webrev: >>>> >>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.09/ >>>> ?? Diff from previous webrev: >>>> http://hg.openjdk.java.net/jdk/submit/rev/5d167adf8524 >>>> >>>> I saw similar build errors in libOneGetThreadListStackTraces.cpp on >>>> Windows. >>>> This webrev fixes them. >>> >>> You shouldn't use %p as it may not be portable. In the VM we use >>> INTPTR_FORMAT and convert the arg using p2i. I don't know what >>> exists in the testing code. >> >> I replaced %p to %lx, and also cast values to unsigned long [1] [2], >> but the test on submit repo was failed. >> Can anyone share details of >> mach5-one-ysuenaga-JDK-8242428-20200709-1030-12500928 ? > These are the errors I see for the macOS build: > > ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:53:14: > error: format specifies type 'long long' but the argument has type > 'jlocation' (aka 'long') [-Werror,-Wformat] > ????????????? fi1->location, fi2->location); > ????????????? ^~~~~~~~~~~~~ > ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:53:29: > error: format specifies type 'long long' but the argument has type > 'jlocation' (aka 'long') [-Werror,-Wformat] > ????????????? fi1->location, fi2->location); > ???????????????????????????? ^~~~~~~~~~~~~ > > These are the ones I see for the Windows build: > > ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): > warning C4311: 'type cast': pointer truncation from 'jmethodID' to > 'unsigned long' > ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): > warning C4302: 'type cast': truncation from 'jmethodID' to 'unsigned > long' > ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): > warning C4311: 'type cast': pointer truncation from 'jmethodID' to > 'unsigned long' > ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): > warning C4302: 'type cast': truncation from 'jmethodID' to 'unsigned > long' > ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): > warning C4311: 'type cast': pointer truncation from 'jthread' to > 'unsigned long' > ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): > warning C4302: 'type cast': truncation from 'jthread' to 'unsigned long' > ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): > warning C4311: 'type cast': pointer truncation from 'jthread' to > 'unsigned long' > ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): > warning C4302: 'type cast': truncation from 'jthread' to 'unsigned long' > > > You will probably want to use the macros defined in > src/hotspot/share/utilities/globalDefinitions.hpp. Let me know if you > need me to test something. With these changes the build works okay on Linux, Windows and macOS: --- a/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp +++ b/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp @@ -27,4 +27,5 @@ ?#include ?#include +#include ?#define MAX_FRAMES 100 @@ -45,11 +46,11 @@ ?? if (fi1->method != fi2->method) { /* jvmtiFrameInfo::method */ ???? snprintf(err_msg, sizeof(err_msg), -???????????? "method is different: fi1 = %p, fi2 = %p", -???????????? fi1->method, fi2->method); +???????????? "method is different: fi1 = 0x%016" PRIxPTR " , fi2 = 0x%016" PRIxPTR, +???????????? (intptr_t)fi1->method, (intptr_t)fi2->method); ???? env->FatalError(err_msg); ?? } else if (fi1->location != fi2->location) { /* jvmtiFrameInfo::location */ ???? snprintf(err_msg, sizeof(err_msg), -???????????? "location is different: fi1 = %lld, fi2 = %lld", -???????????? fi1->location, fi2->location); +???????????? "location is different: fi1 = %" PRId64 " , fi2 = %" PRId64, +???????????? (int64_t)fi1->location, (int64_t)fi2->location); ???? env->FatalError(err_msg); ?? } @@ -67,5 +68,5 @@ ?? if (!is_same) { /* jvmtiStackInfo::thread */ ???? snprintf(err_msg, sizeof(err_msg), -???????????? "thread is different: si1 = %p, si2 = %p", si1->thread, si2->thread); +???????????? "thread is different: si1 = 0x%016" PRIxPTR " , si2 = 0x%016" PRIxPTR, (intptr_t)si1->thread, (intptr_t)si2->thread); ???? env->FatalError(err_msg); ?? } else if (si1->state != si2->state) { /* jvmtiStackInfo::state */ Maybe you can use something like that. Thanks, Patricio > Thanks, > Patricio >> Thanks, >> >> Yasumasa >> >> >> [1] http://hg.openjdk.java.net/jdk/submit/rev/dfca51958217 >> [2] http://hg.openjdk.java.net/jdk/submit/rev/3665361fa91b >> >> >>> David >>> ----- >>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2020/07/09 1:42, Daniel D. Daugherty wrote: >>>>> ?> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ >>>>> >>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>> ???? No comments. >>>>> >>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>> ???? L1159: ? Thread *current_thread = Thread::current(); >>>>> ???????? Please add "#ifdef ASSERT" above and "#endif" below since >>>>> ???????? current_thread is only used for the assert() in this >>>>> function. >>>>> >>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>> ???? L549: ???????????????????????????? jthread java_thread, jint >>>>> max_frame_count) >>>>> ???? L552: ????? _jthread(java_thread), >>>>> ???????? Please: s/java_thread/thread/ on both lines. >>>>> >>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>> ???? No comments. >>>>> >>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>> >>>>> ???? No comments. >>>>> >>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>> >>>>> ???? L27: #include >>>>> ???????? This include is out of order; should be first in the list. >>>>> >>>>> ???? This file doesn't compile on my MBP13: >>>>> >>>>> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:49:14: >>>>> error: format specifies type 'unsigned long' but the argument has >>>>> type 'jmethodID' (aka '_jmethodID *') [-Werror,-Wformat] >>>>> ????????????? fi1->method, fi2->method); >>>>> ????????????? ^~~~~~~~~~~ >>>>> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:49:27: >>>>> error: format specifies type 'unsigned long' but the argument has >>>>> type 'jmethodID' (aka '_jmethodID *') [-Werror,-Wformat] >>>>> ????????????? fi1->method, fi2->method); >>>>> ?????????????????????????? ^~~~~~~~~~~ >>>>> ???? 2 errors generated. >>>>> >>>>> ???? This change made it compile on my MBP13, but that may break >>>>> it on >>>>> ???? other platforms: >>>>> >>>>> ???? $ hg diff >>>>> ???? diff -r 560847c69fbe >>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>> ???? --- >>>>> a/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>> Wed Jul 08 12:13:32 2020 -0400 >>>>> ???? +++ >>>>> b/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>> Wed Jul 08 12:40:42 2020 -0400 >>>>> ???? @@ -46,7 +46,7 @@ >>>>> ?? ?? ? if (fi1->method != fi2->method) { /* >>>>> jvmtiFrameInfo::method */ >>>>> ?? ?? ??? snprintf(err_msg, sizeof(err_msg), >>>>> ?? ? ? ??????????? "method is different: fi1 = %lx, fi2 = %lx", >>>>> ???? -???????????? fi1->method, fi2->method); >>>>> ???? +???????????? (unsigned long) fi1->method, (unsigned long) >>>>> fi2->method); >>>>> ?? ?? ??? env->FatalError(err_msg); >>>>> ?? ?? ? } else if (fi1->location != fi2->location) { /* >>>>> jvmtiFrameInfo::location */ >>>>> ?? ?? ??? snprintf(err_msg, sizeof(err_msg), >>>>> >>>>> ???? I'm not sure of the right platform independent way to output >>>>> ???? the 'method' field. >>>>> >>>>> Dan >>>>> >>>>> >>>>> On 7/8/20 4:04 AM, Yasumasa Suenaga wrote: >>>>>> Hi David, >>>>>> >>>>>> On 2020/07/08 15:27, David Holmes wrote: >>>>>>> Hi Yasumasa, >>>>>>> >>>>>>> On 7/07/2020 6:54 pm, Yasumasa Suenaga wrote: >>>>>>>> Hi David, Serguei, >>>>>>>> >>>>>>>> Serguei, thank you for replying even though you are on vacaiton! >>>>>>>> >>>>>>>> I uploaded new webrev: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.07/ >>>>>>>> ?? Diff from previous webrev: >>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/77243b1dcbfe >>>>>>>> >>>>>>>> c'tor of GetSingleStackTraceClosure has jthread argument in >>>>>>>> this webrev. >>>>>>>> Also it does not contain testcase for GetThreadListStackTraces >>>>>>>> with all threads, and OneGetThreadListStackTraces would test >>>>>>>> main thread only. >>>>>>> >>>>>>> All those changes are fine in principle for me. One nit/suggestion: >>>>>>> >>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>> >>>>>>> ??544?? jthread _java_thread; >>>>>>> >>>>>>> elsewhere "java_thread" refers to a JavaThread, so to avoid >>>>>>> confusion may I suggest this member be named _jthread. >>>>>> >>>>>> I uploaded new webrev: >>>>>> >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ >>>>>> ? Diff from previous webrev: >>>>>> http://hg.openjdk.java.net/jdk/submit/rev/ca6263dbdc87 >>>>>> >>>>>> >>>>>>> I'm going to be away for the next couple of days - sorry - but >>>>>>> will try to check email on this if I can. >>>>>> >>>>>> Thanks! >>>>>> >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2020/07/07 15:13, David Holmes wrote: >>>>>>>>> On 7/07/2020 2:57 pm, Yasumasa Suenaga wrote: >>>>>>>>>> Hi David, >>>>>>>>>> >>>>>>>>>> On 2020/07/07 11:31, David Holmes wrote: >>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>> >>>>>>>>>>> Hard to keep up with the changes - especially without >>>>>>>>>>> incremental webrevs. >>>>>>>>>> >>>>>>>>>> Sorry, I will upload diff from previous webrev in the next. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> If GetSingleStackTraceClosure also took the jthread as a >>>>>>>>>>> constructor arg, then you wouldn't need to recreate a JNI >>>>>>>>>>> local handle when calling _collector.fill_frames. It's a >>>>>>>>>>> small simplification and not essential at this stage. >>>>>>>>>> >>>>>>>>>> I think we should get jthread from an argument of do_thread() >>>>>>>>>> because do_thread() would pass the thread which are stopped >>>>>>>>>> certainly. >>>>>>>>>> It might be simplification if we pass _calling_thread to >>>>>>>>>> MultipleStackTracesCollector. `jthread` is only needed to >>>>>>>>>> store jvmtiStackInfo.thread . What do you think? >>>>>>>>> >>>>>>>>> I'm not quite sure what you mean. >>>>>>>>> >>>>>>>>> I think there is a bit of a design wart with direct handshakes >>>>>>>>> in that do_thread takes the target JavaThread as an argument. >>>>>>>>> That's useful in a case where you want a HandshakeClosure that >>>>>>>>> can be applied to multiple threads, but that's not typically >>>>>>>>> what is needed with direct handshakes - there is only a single >>>>>>>>> target. With a single-target HandshakeClosure you can capture >>>>>>>>> all the "target" information for the operation in the closure >>>>>>>>> instance. So if the actual do_thread operation wants the >>>>>>>>> jthread corresponding to the target thread then we can store >>>>>>>>> that in the closure rather than recomputing it (you could >>>>>>>>> assert it is the same but that seems overkill to me). >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> For the test ... I don't see how >>>>>>>>>>> Java_GetThreadListStackTraces_checkCallStacks is a valid >>>>>>>>>>> test. It gets the stacks of all live threads, then uses that >>>>>>>>>>> information to use GetThreadListStackTraces to get the stack >>>>>>>>>>> for the same set of threads through a different API. It then >>>>>>>>>>> compares the two sets of stacks for each thread expecting >>>>>>>>>>> them to be the same, but that need only be the case for the >>>>>>>>>>> main thread. Other threads could potentially have a >>>>>>>>>>> different stack (e.g. if this test is run with JFR enabled >>>>>>>>>>> there will be additional threads found.) Further I would >>>>>>>>>>> have expected that there already exist tests that check >>>>>>>>>>> that, for a given thread (which may be suspended or known to >>>>>>>>>>> be blocked) the same stack is found through the two >>>>>>>>>>> different APIs. >>>>>>>>>> >>>>>>>>>> vmTestbase/nsk/jvmti/unit/GetAllStackTraces/getallstktr001 >>>>>>>>>> would check all of threads via GetThreadListStackTraces() and >>>>>>>>>> GetAllStackTraces(), so we might be able to remove >>>>>>>>>> GetThreadListStackTraces.java from this webrev. >>>>>>>>> >>>>>>>>> Yes. The existing test only examines a set of test threads >>>>>>>>> that are all blocked on a raw monitor. You do not need to >>>>>>>>> duplicate that test. >>>>>>>>> >>>>>>>>>> OTOH we don't have testcase for GetThreadListStackTraces() >>>>>>>>>> with thread_count == 1, so we need to testcase for it (it is >>>>>>>>>> OneGetThreadListStackTraces.java) It would check whether the >>>>>>>>>> state of target thread is "waiting" before JNI call to call >>>>>>>>>> GetThreadListStackTraces(), >>>>>>>>> >>>>>>>>> Yes we need to test the special cases introduced by your >>>>>>>>> changes - totally agree - and OneGetThreadListStackTraces.java >>>>>>>>> is a good test for that. >>>>>>>>> >>>>>>>>>> and also I expect it would not be run with JFR. (it is not >>>>>>>>>> described @run) >>>>>>>>> >>>>>>>>> The arguments to run with JFR (or a bunch of other things) can >>>>>>>>> be passed to the jtreg test harness to be applied to all tests. >>>>>>>>> >>>>>>>>>> Of course we can check GetThreadListStackTraces() with main >>>>>>>>>> thread, but it is not the test for direct handshake for other >>>>>>>>>> thread. >>>>>>>>> >>>>>>>>> Right - that test already exists as per the above. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> David >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> David >>>>>>>>>>> ----- >>>>>>>>>>> >>>>>>>>>>> On 6/07/2020 11:29 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>> >>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>> >>>>>>>>>>>> I think C++ is more simple to implement the test agent as >>>>>>>>>>>> you said. >>>>>>>>>>>> So I implement it in C++ in new webrev. Could you review >>>>>>>>>>>> again? >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.06/ >>>>>>>>>>>> >>>>>>>>>>>> Also I refactored libGetThreadListStackTraces.cpp, and I've >>>>>>>>>>>> kept exception check after IsSameObject(). >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 2020/07/06 16:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>> >>>>>>>>>>>>> Thank you for the update. >>>>>>>>>>>>> I think, a pending exception after IsSameObject needs to >>>>>>>>>>>>> be checked. >>>>>>>>>>>>> >>>>>>>>>>>>> The checkStackInfo() needs one more refactoring as I've >>>>>>>>>>>>> already suggested. >>>>>>>>>>>>> The body of the loop at L68-L78 should be converted to a >>>>>>>>>>>>> function check_frame_info. >>>>>>>>>>>>> The si1->frame_buffer[i] and si2->frame_buffer[i] will be >>>>>>>>>>>>> passed as fi1 and fi2. >>>>>>>>>>>>> The index can be passed as well. >>>>>>>>>>>>> I'm still suggesting to simplify the local exception_msg >>>>>>>>>>>>> to something shorter like err_msg or exc_msg. >>>>>>>>>>>>> >>>>>>>>>>>>> I'm not sure using fatal is right here: >>>>>>>>>>>>> >>>>>>>>>>>>> This fragment looks strange: >>>>>>>>>>>>> >>>>>>>>>>>>> ? 152???? if ((*env)->IsSameObject(env, >>>>>>>>>>>>> stack_info[i].thread, thread)) { >>>>>>>>>>>>> ? 153?????? target_info = &stack_info[i]; >>>>>>>>>>>>> ? 154?????? break; >>>>>>>>>>>>> ? 155???? } else if ((*env)->ExceptionOccurred(env)) { >>>>>>>>>>>>> ? 156 (*env)->ExceptionDescribe(env); >>>>>>>>>>>>> ? 157?????? (*env)->FatalError(env, __FILE__); >>>>>>>>>>>>> ? 158???? } >>>>>>>>>>>>> >>>>>>>>>>>>> I expected it to be: >>>>>>>>>>>>> >>>>>>>>>>>>> ??? jboolean same = (*env)->IsSameObject(env, >>>>>>>>>>>>> stack_info[i].thread, thread); >>>>>>>>>>>>> ??? if ((*env)->ExceptionOccurred(env)) { >>>>>>>>>>>>> ????? (*env)->ExceptionDescribe(env); >>>>>>>>>>>>> ????? (*env)->FatalError(env, __FILE__); >>>>>>>>>>>>> ??? } >>>>>>>>>>>>> ??? if (same) { >>>>>>>>>>>>> ????? target_info = &stack_info[i]; >>>>>>>>>>>>> ????? break; >>>>>>>>>>>>> ??? } >>>>>>>>>>>>> >>>>>>>>>>>>> Would it better to port this agent to C++ to simplify this >>>>>>>>>>>>> code nicer? >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Serguei >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 7/5/20 06:13, Yasumasa Suenaga wrote: >>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>> I refactored testcase. Could you review again? >>>>>>>>>>>>>> >>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.05/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> It would check Java exception after IsSameObject() call. >>>>>>>>>>>>>> Does it need? >>>>>>>>>>>>>> Any exceptions are not described in JNI document[1], and >>>>>>>>>>>>>> JNI implementation (jni_IsSameObject()) does not seem to >>>>>>>>>>>>>> throw it. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> [1] >>>>>>>>>>>>>> https://docs.oracle.com/en/java/javase/14/docs/specs/jni/functions.html#issameobject >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 2020/07/05 14:46, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Okay, thanks. >>>>>>>>>>>>>>> Then I'm okay to keep the GetSingleStackTraceClosure. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c.html >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c.html >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I'm not sure the function 'is_same_thread() is needed. >>>>>>>>>>>>>>> Why do not use the JNI IsSameObject instead? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> It seems to be a typo at L132 and L137. >>>>>>>>>>>>>>> You, probably. did not want to print the same >>>>>>>>>>>>>>> information for stack_info_1[i].frame_buffer[j].XXX twice. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The code at lines 112-142 is not readable. >>>>>>>>>>>>>>> I'd suggest to make a couple of refactoring steps. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> First step to simplify this a little bit would be with >>>>>>>>>>>>>>> some renaming and getting rid of indexes: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ?? 71 char err_msg[EXCEPTION_MSG_LEN] = {0}; >>>>>>>>>>>>>>> ??... >>>>>>>>>>>>>>> ??112?? /* Iterate all jvmtiStackInfo to check */ >>>>>>>>>>>>>>> ??113?? for (i = 0; i < num_threads, *exception_msg != >>>>>>>>>>>>>>> '\0'; i++) { >>>>>>>>>>>>>>> ????????? jvmtiStackInfo *si1 = stack_info_1[i]; >>>>>>>>>>>>>>> ????????? jvmtiStackInfo *si2 = stack_info_2[i]; >>>>>>>>>>>>>>> ??114???? if (!IsSameObject(env, si1.thread, >>>>>>>>>>>>>>> si2.thread)) { /* jvmtiStackInfo::thread */ >>>>>>>>>>>>>>> ??115?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>> ??116??????????????? "thread[%d] is different: >>>>>>>>>>>>>>> stack_info_1 = %p, stack_info_2 = %p", >>>>>>>>>>>>>>> ??117??????????????? i, sinfo1.thread, sinfo2.thread); >>>>>>>>>>>>>>> ??118???? } else if (si1.state != si2.state) { /* >>>>>>>>>>>>>>> jvmtiStackInfo::state */ >>>>>>>>>>>>>>> ??119?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>> ??120??????????????? "state[%d] is different: >>>>>>>>>>>>>>> stack_info_1 = %d, stack_info_2 = %d", >>>>>>>>>>>>>>> ??121??????????????? i, si1.state, si2.state); >>>>>>>>>>>>>>> ??122???? } else if (si1.frame_count != si2.frame_count) >>>>>>>>>>>>>>> { /* jvmtiStackInfo::frame_count */ >>>>>>>>>>>>>>> ??123?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>> ??124??????????????? "frame_count[%d] is different: >>>>>>>>>>>>>>> stack_info_1 = %d, stack_info_2 = %d", >>>>>>>>>>>>>>> ??125??????????????? i, si1.frame_count, si2.frame_count); >>>>>>>>>>>>>>> ??126???? } else { >>>>>>>>>>>>>>> ??127?????? /* Iterate all jvmtiFrameInfo to check */ >>>>>>>>>>>>>>> ??128?????? for (j = 0; j < si1.frame_count; j++) { >>>>>>>>>>>>>>> ??129???????? if (si1.frame_buffer[j].method != >>>>>>>>>>>>>>> si1.frame_buffer[j].method) { /* jvmtiFrameInfo::method */ >>>>>>>>>>>>>>> ??130?????????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>> ??131??????????????????? "thread [%d] >>>>>>>>>>>>>>> frame_buffer[%d].method is different: stack_info_1 = >>>>>>>>>>>>>>> %lx, stack_info_2 = %lx", >>>>>>>>>>>>>>> ??132??????????????????? i, j, >>>>>>>>>>>>>>> si1.frame_buffer[j].method, si2.frame_buffer[j].method); >>>>>>>>>>>>>>> ??133?????????? break; >>>>>>>>>>>>>>> ??134???????? } else if (si1.frame_buffer[j].location != >>>>>>>>>>>>>>> si1.frame_buffer[j].location) { /* >>>>>>>>>>>>>>> jvmtiFrameInfo::location */ >>>>>>>>>>>>>>> ??135?????????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>> ??136??????????????????? "thread [%d] >>>>>>>>>>>>>>> frame_buffer[%d].location is different: stack_info_1 = >>>>>>>>>>>>>>> %ld, stack_info_2 = %ld", >>>>>>>>>>>>>>> ??137??????????????????? i, j, >>>>>>>>>>>>>>> si1.frame_buffer[j].location, >>>>>>>>>>>>>>> si2.frame_buffer[j].location); >>>>>>>>>>>>>>> ??138?????????? break; >>>>>>>>>>>>>>> ??139???????? } >>>>>>>>>>>>>>> ??140?????? } >>>>>>>>>>>>>>> ??141???? } >>>>>>>>>>>>>>> ??142?? } >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Another step would be to create functions that implement >>>>>>>>>>>>>>> a body of each loop. >>>>>>>>>>>>>>> You can use the same techniques to simplify similar >>>>>>>>>>>>>>> place (L127-L138) in the libOneGetThreadListStackTraces.c. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 7/3/20 15:55, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I'm not an Oracle employee, so I cannot know real >>>>>>>>>>>>>>>> request(s) from your customers. >>>>>>>>>>>>>>>> However JDK-8201641 says Dynatrace has requested this >>>>>>>>>>>>>>>> enhancement. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> BTW I haven't heared any request from my customers >>>>>>>>>>>>>>>> about this. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 2020/07/04 4:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> This difference is not that big to care about. >>>>>>>>>>>>>>>>> I feel this is really rare case and so, does not worth >>>>>>>>>>>>>>>>> these complications. >>>>>>>>>>>>>>>>> Do we have a real request from customers to optimize it? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 7/3/20 01:16, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Generally I agree with you, but I have concern about >>>>>>>>>>>>>>>>>> the difference of the result of GetStackTrace() and >>>>>>>>>>>>>>>>>> GetThreadListStackTraces(). >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> ? GetStackTrace: jvmtiFrameInfo >>>>>>>>>>>>>>>>>> ? GetThreadListStackTraces: jvmtiStackInfo >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> jvmtiStackInfo contains thread state, and it is >>>>>>>>>>>>>>>>>> ensured it is the state of the call stack. >>>>>>>>>>>>>>>>>> If we want to get both call stack and thread state, >>>>>>>>>>>>>>>>>> we need to suspend target thread, and call both >>>>>>>>>>>>>>>>>> GetStackTrace() and GetThreadState(). Is it ok? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I was wondering if JDK-8201641 (parent ticket of this >>>>>>>>>>>>>>>>>> change) needed them for profiling (dynatrace?) >>>>>>>>>>>>>>>>>> If it is responsibility of JVMTI agent implementor, I >>>>>>>>>>>>>>>>>> remove this closure. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 2020/07/03 16:45, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> After some thinking I've concluded that I do not >>>>>>>>>>>>>>>>>>> like this optimization >>>>>>>>>>>>>>>>>>> of the GetThreadListStackTraces with >>>>>>>>>>>>>>>>>>> GetSingleStackTraceClosure. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> We may need more opinions on this but these are my >>>>>>>>>>>>>>>>>>> points: >>>>>>>>>>>>>>>>>>> ??- it adds some complexity and ugliness >>>>>>>>>>>>>>>>>>> ??- a win is doubtful because it has to be a rare >>>>>>>>>>>>>>>>>>> case, so that total overhead should not be high >>>>>>>>>>>>>>>>>>> ??- if it is really high for some use cases then it >>>>>>>>>>>>>>>>>>> is up to the user >>>>>>>>>>>>>>>>>>> ??? to optimize it with using GetStackTrace instead >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> In such cases with doubtful overhead I usually >>>>>>>>>>>>>>>>>>> prefer the simplicity. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Good examples where it makes sense to optimize are >>>>>>>>>>>>>>>>>>> checks for target thread to be current thread. >>>>>>>>>>>>>>>>>>> In such cases there is no need to suspend the target >>>>>>>>>>>>>>>>>>> thread, or use a VMop/HandshakeClosure. >>>>>>>>>>>>>>>>>>> For instance, please, see the Monitor functions with >>>>>>>>>>>>>>>>>>> the check: (java_thread == calling_thread). >>>>>>>>>>>>>>>>>>> Getting information for current thread is frequently >>>>>>>>>>>>>>>>>>> used case, e.g. to get info at an event point. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On 7/2/20 23:29, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>> Hi Dan, David, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could you review again? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/ >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> OneGetThreadListStackTraces.java in this webrev >>>>>>>>>>>>>>>>>>>> would wait until thread state is transited to >>>>>>>>>>>>>>>>>>>> "waiting" with spin wait. >>>>>>>>>>>>>>>>>>>> CountDownLatch::await call as Dan pointed is fixed >>>>>>>>>>>>>>>>>>>> in it :) >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Diff from webrev.03: >>>>>>>>>>>>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/c9aeb7001e50 >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On 2020/07/03 14:15, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 12:24, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>>>>>>> On 7/2/20 10:50 PM, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>> Sorry I'm responding here without seeing latest >>>>>>>>>>>>>>>>>>>>>>>> webrev but there is enough context I think ... >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>> Hi Dan, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> I upload new webrev. Could you review again? >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>>>>>>>>>>>>>>>>>> ???? L1542: // Get stack trace with handshake >>>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - please add a period at the end. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> L1591: *stack_info_ptr = op.stack_info(); >>>>>>>>>>>>>>>>>>>>>>>>>> ???????? The return parameter should not be >>>>>>>>>>>>>>>>>>>>>>>>>> touched unless the return >>>>>>>>>>>>>>>>>>>>>>>>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> ???? old L1582: if (err == JVMTI_ERROR_NONE) { >>>>>>>>>>>>>>>>>>>>>>>>>> ???????? Please restore this check. The >>>>>>>>>>>>>>>>>>>>>>>>>> return parameter should not >>>>>>>>>>>>>>>>>>>>>>>>>> ???????? be touched unless the return code in >>>>>>>>>>>>>>>>>>>>>>>>>> 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> But op.stack_info() will return NULL if the >>>>>>>>>>>>>>>>>>>>>>>> error is not JVMTI_ERROR_NONE. Are you (Dan) >>>>>>>>>>>>>>>>>>>>>>>> concerned about someone passing in a >>>>>>>>>>>>>>>>>>>>>>>> non-null/initialized out-pointer that will be >>>>>>>>>>>>>>>>>>>>>>>> reset to NULL if there was an error? >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Actually the way we used to test this in POSIX >>>>>>>>>>>>>>>>>>>>>>> tests is to call >>>>>>>>>>>>>>>>>>>>>>> an API with known bad parameters and the return >>>>>>>>>>>>>>>>>>>>>>> parameter ptr >>>>>>>>>>>>>>>>>>>>>>> set to NULL. If the return parameter ptr was >>>>>>>>>>>>>>>>>>>>>>> touched when an >>>>>>>>>>>>>>>>>>>>>>> error should have been detected on an earlier >>>>>>>>>>>>>>>>>>>>>>> parameter, then >>>>>>>>>>>>>>>>>>>>>>> the test failed. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> L1272: ? if (!jt->is_exiting() && (thread_oop >>>>>>>>>>>>>>>>>>>>>>>>>> != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second >>>>>>>>>>>>>>>>>>>>>>>>>> expression. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>> ???? old L1532: _result = >>>>>>>>>>>>>>>>>>>>>>>>>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>>>>>>>>>>>>>>>>>>>>>>> ???????? This deletion of the _result field >>>>>>>>>>>>>>>>>>>>>>>>>> threw me for a minute and then >>>>>>>>>>>>>>>>>>>>>>>>>> ???? ? ? I figured out that the field is init >>>>>>>>>>>>>>>>>>>>>>>>>> to JVMTI_ERROR_THREAD_NOT_ALIVE >>>>>>>>>>>>>>>>>>>>>>>>>> ???????? in the constructor. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> ???? L1553: ? if (!jt->is_exiting() && >>>>>>>>>>>>>>>>>>>>>>>>>> (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second >>>>>>>>>>>>>>>>>>>>>>>>>> expression. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> ???? L64: startSignal.countDown(); >>>>>>>>>>>>>>>>>>>>>>>>>> ???????? I was expecting this to be a call to >>>>>>>>>>>>>>>>>>>>>>>>>> await() instead of >>>>>>>>>>>>>>>>>>>>>>>>>> countDown(). What am I missing here? >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> ???????? I think this test might be passing >>>>>>>>>>>>>>>>>>>>>>>>>> by accident right now, but... >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Main thread (which call JVMTI functions to >>>>>>>>>>>>>>>>>>>>>>>>> test) should wait until test thread is ready. >>>>>>>>>>>>>>>>>>>>>>>>> So main thread would wait startSignal, and >>>>>>>>>>>>>>>>>>>>>>>>> test thread would count down. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> No! >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> The test thread that previously called >>>>>>>>>>>>>>>>>>>>>>>> obj.wait() now calls latch.await(). >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> The main thread that previously called >>>>>>>>>>>>>>>>>>>>>>>> obj.notify() now calls latch.countDown(). >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> The main thread continues to spin until it sees >>>>>>>>>>>>>>>>>>>>>>>> the target is WAITING before proceeding with >>>>>>>>>>>>>>>>>>>>>>>> the test. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> If I add spin wait to wait until transit target >>>>>>>>>>>>>>>>>>>>>> thread state is WAITING (as following), we don't >>>>>>>>>>>>>>>>>>>>>> need to call SuspendThread(). >>>>>>>>>>>>>>>>>>>>>> Which is better? >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> The original spin-wait loop checking for? WAITING >>>>>>>>>>>>>>>>>>>>> is better because it is the only guarantee that >>>>>>>>>>>>>>>>>>>>> the target thread is blocked where you need it to >>>>>>>>>>>>>>>>>>>>> be. suspending the thread is racy as you don't >>>>>>>>>>>>>>>>>>>>> know exactly where the suspend will hit. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>> /* Wait until the thread state transits to >>>>>>>>>>>>>>>>>>>>>> "waiting" */ >>>>>>>>>>>>>>>>>>>>>> while (th.getState() != Thread.State.WAITING) { >>>>>>>>>>>>>>>>>>>>>> Thread.onSpinWait(); >>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> For simplify, spin wait is prefer to >>>>>>>>>>>>>>>>>>>>>> OneGetThreadListStackTraces.java in webrev.03. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Here's the flow as I see it: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>>> ?? - start worker thread >>>>>>>>>>>>>>>>>>>>>>> ?? - startSignal.await() >>>>>>>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>>>>>>> ?? - startSignal.countDown() >>>>>>>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>>>>>>> ?? - stopSignal.await() >>>>>>>>>>>>>>>>>>>>>>> ???? - worker is now blocked >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>>> ?? - checkCallStacks(th) >>>>>>>>>>>>>>>>>>>>>>> ?? - stopSignal.countDown() >>>>>>>>>>>>>>>>>>>>>>> ???? - worker is now unblocked >>>>>>>>>>>>>>>>>>>>>>> ?? - th.join >>>>>>>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>>>>>>> ?? - runs off the end of run() >>>>>>>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>>> ?? - run off the end of main() >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> ???? L92: jthreads = (jthread >>>>>>>>>>>>>>>>>>>>>>>>>> *)malloc(sizeof(jthread) * num_threads); >>>>>>>>>>>>>>>>>>>>>>>>>> ???????? You don't check for malloc() failure. >>>>>>>>>>>>>>>>>>>>>>>>>> 'jthreads' is allocated but never freed. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> ???? L91: result = >>>>>>>>>>>>>>>>>>>>>>>>>> (*jvmti)->SuspendThread(jvmti, thread); >>>>>>>>>>>>>>>>>>>>>>>>>> ???????? Why are you suspending the thread? >>>>>>>>>>>>>>>>>>>>>>>>>> GetAllStackTraces() and >>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() do not require the >>>>>>>>>>>>>>>>>>>>>>>>>> target thread(s) >>>>>>>>>>>>>>>>>>>>>>>>>> ???????? to be suspend. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> ???????? If you decide not to SuspendThread, >>>>>>>>>>>>>>>>>>>>>>>>>> then you don't need the >>>>>>>>>>>>>>>>>>>>>>>>>> AddCapabilities or the ResumeThread calls. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Test thread might not be entered following >>>>>>>>>>>>>>>>>>>>>>>>> code (stopSignal.await()). We might see >>>>>>>>>>>>>>>>>>>>>>>>> deferent call stack between >>>>>>>>>>>>>>>>>>>>>>>>> GetAllStackTraces() and >>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces(). We cannot control >>>>>>>>>>>>>>>>>>>>>>>>> to freeze call stack of test thread in Java code. >>>>>>>>>>>>>>>>>>>>>>>>> (I didn't use SuspendThread() at first, but I >>>>>>>>>>>>>>>>>>>>>>>>> saw some errors which causes in above.) >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> So we need to call SuspendThread() to ensure >>>>>>>>>>>>>>>>>>>>>>>>> we can see same call stack. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> If you are checking that the thread is in state >>>>>>>>>>>>>>>>>>>>>>>> WAITING then it cannot escape from that state >>>>>>>>>>>>>>>>>>>>>>>> and you can sample the stack multiple times >>>>>>>>>>>>>>>>>>>>>>>> from any API and get the same result. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> I suspect the errors you saw were from the >>>>>>>>>>>>>>>>>>>>>>>> apparent incorrect use of the CountDownLatch. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> With the flow outlined above, the worker thread >>>>>>>>>>>>>>>>>>>>>>> should be >>>>>>>>>>>>>>>>>>>>>>> nicely blocked in stopSignal.await() when stuff >>>>>>>>>>>>>>>>>>>>>>> is sampled. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Updates look fine - thanks. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> One minor nit: >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> 1274 _collector.allocate_and_fill_stacks(1); >>>>>>>>>>>>>>>>>>>>>>>>>>>> 1275 _collector.set_result(JVMTI_ERROR_NONE); >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> In the other places where you use >>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector you rely on result being >>>>>>>>>>>>>>>>>>>>>>>>>>>> initialized to JVMTI_ERROR_NONE, rather >>>>>>>>>>>>>>>>>>>>>>>>>>>> than setting it directly after >>>>>>>>>>>>>>>>>>>>>>>>>>>> allocate_and_fill_stacks(). >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 820 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> assert(SafepointSynchronize::is_at_safepoint() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 821 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 822 current_thread == >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 823 "at safepoint / handshake or >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> necessary, as even if the target is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> suspended we must still be at a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> safepoint or in a handshake with it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Makes me wonder if we used to allow a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> racy stacktrace operation on a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> suspended thread, assuming it would >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> This function >>>>>>>>>>>>>>>>>>>>>>>>>>>>> (JvmtiEnvBase::get_stack_trace()) can be >>>>>>>>>>>>>>>>>>>>>>>>>>>>> called to get own stack trace. For >>>>>>>>>>>>>>>>>>>>>>>>>>>>> example, we can call GetStackTrace() for >>>>>>>>>>>>>>>>>>>>>>>>>>>>> current thread at JVMTI event. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> So I changed assert as below: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??820 assert(current_thread == java_thread || >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??821 >>>>>>>>>>>>>>>>>>>>>>>>>>>>> SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == >>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??823 "call by myself / at safepoint / at >>>>>>>>>>>>>>>>>>>>>>>>>>>>> handshake"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Yep good catch. I hope current tests caught >>>>>>>>>>>>>>>>>>>>>>>>>>>> that. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> They would be tested in >>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ >>>>>>>>>>>>>>>>>>>>>>>>>>> (own call stacks), and getstacktr003 (call >>>>>>>>>>>>>>>>>>>>>>>>>>> stacks in other thread). >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Speaking of tests ... >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> In the native code I think you need to >>>>>>>>>>>>>>>>>>>>>>>>>>>> check the success of all JNI methods that >>>>>>>>>>>>>>>>>>>>>>>>>>>> can throw exceptions - otherwise I believe >>>>>>>>>>>>>>>>>>>>>>>>>>>> the tests may trigger warnings if >>>>>>>>>>>>>>>>>>>>>>>>>>>> -Xcheck:jni is used with them. See for >>>>>>>>>>>>>>>>>>>>>>>>>>>> example: >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> I updated testcases to check JNI and JVMTI >>>>>>>>>>>>>>>>>>>>>>>>>>> function calls. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> In the Java code the target thread: >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 45 public void run() { >>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 46 try { >>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 47 synchronized (lock) { >>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 48 lock.wait(); >>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 49 System.out.println("OK"); >>>>>>>>>>>>>>>>>>>>>>>>>>>> 50?????????? } >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> is potentially susceptible to spurious >>>>>>>>>>>>>>>>>>>>>>>>>>>> wakeups. Using a CountDownLatch would be >>>>>>>>>>>>>>>>>>>>>>>>>>>> robust. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _calling_thread is the current >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If so, we can call make_local() in L1272 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> without JavaThread (or we can pass >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> current thread to make_local()). Is it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> right? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry I got confused, _calling_thread may >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not be the current thread as we could be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> executing the handshake in the target >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread itself. So the ResourceMark is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correct as-is (implicitly for current >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The argument to fill_frames will be used >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the jvmtiStackInfo and passed back to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the _calling_thread, so it must be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created via make_local(_calling_thread, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ...) as you presently have. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you for reviewing! I will update >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new webrev tomorrow. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector : public >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 498 class VM_GetAllStackTraces : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 500 JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 502 MultipleStackTracesCollector >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of another class like that as it may >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not be on the stack. I think >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector should >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not extend any allocation class, and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should always be embedded directly in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??? : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class GetAllStackTraces : public >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I updated webrev for 8242428. Could >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change migrate to use direct >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> handshake for GetStackTrace() and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() (when >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread_count == 1). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks really good now! I only >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a few nits below. There is one >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thing I don't like about it but it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> requires a change to the main >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Handshake logic to address - in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JvmtiEnv::GetThreadListStackTraces >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you have to create a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ThreadsListHandle to convert the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jthread to a JavaThread, but then the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Handshake::execute_direct creates >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> another ThreadsListHandle internally. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That's a waste. I will discuss with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Robbin and file a RFE to have an >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> overload of execute_direct that takes >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an existing TLH. Actually it's worse >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than that because we have another TLH >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in use at the entry point for the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JVMTI functions, so I think there may >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be some scope for simplifying the use >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of TLH instances - future RFE. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??451 GetStackTraceClosure(JvmtiEnv >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *env, jint start_depth, jint max_count, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??452 jvmtiFrameInfo* frame_buffer, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jint* count_ptr) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??453???? : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??454 _env(env), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _start_depth(start_depth), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _max_count(max_count), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??455 _frame_buffer(frame_buffer), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _count_ptr(count_ptr), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??456 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Nit: can you do one initializer per >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line please. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector : public >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??498 class VM_GetAllStackTraces : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??500 JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??502 MultipleStackTracesCollector >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of another class like that as it may >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not be on the stack. I think >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector should >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not extend any allocation class, and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should always be embedded directly in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 481 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector(JvmtiEnv >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *env, jint max_frame_count) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??483 _max_frame_count = >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max_frame_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??484 _frame_count_total = 0; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??485 _head = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??486 _stack_info = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??487 _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As you are touching this can you >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change it to use an initializer list >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as you did for the HandshakeClosure, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and please keep one item per line. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??820 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> assert(SafepointSynchronize::is_at_safepoint() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??821 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??823 "at safepoint / handshake or >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> necessary, as even if the target is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> suspended we must still be at a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> safepoint or in a handshake with it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Makes me wonder if we used to allow a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> racy stacktrace operation on a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> suspended thread, assuming it would >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1269 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is frustrating that this entire >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call chain started with a jthread >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reference, which we converted to a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JavaThread, only to eventually need >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to convert it back to a jthread! I >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think there is some scope for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> simplification here but not as part >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _calling_thread is the current >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm still pondering the test logic >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but wanted to send this now. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetThreadListStackTrace (for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces) and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetAllStackTraces (for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetAllStackTraces) have inherited >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetMultipleStackTraces VM >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operation which provides the feature >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to generate jvmtiStackInfo. I >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> modified VM_GetMultipleStackTraces >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to a normal C++ class to share with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HandshakeClosure for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (GetSingleStackTraceClosure). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I added new testcases which >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test GetThreadListStackTraces() with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread_count == 1 and with all threads. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jvmti >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jdwp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jdi vmTestbase/nsk/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? JBS: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? webrev: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change replace following VM >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operations to direct handshake. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (GetFrameLocation()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (GetThreadListStackTrace()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTrace() uses >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> direct handshake if thread count == >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1. In other case (thread count > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1), it would be performed as VM >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Caller of VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (JvmtiEnvThreadState::reset_current_location()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> might be called at safepoint. So I >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> added safepoint check in its caller. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jvmti >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jdwp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jdi vmTestbase/ns >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I tested it on submit repo, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then it has execution error >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> due to dependency error. So I think >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it does not occur by this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>> > From suenaga at oss.nttdata.com Fri Jul 10 04:58:49 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Fri, 10 Jul 2020 13:58:49 +0900 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: <7ad907cb-24ef-aa55-972a-0a3f63576568@oracle.com> References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <4c43bc2c-551e-d042-8fa7-b63e755888d1@oracle.com> <8c387550-a06e-6320-f453-0e898486befb@oss.nttdata.com> <19cfa2a7-f441-425d-5623-1c11c6a870b6@oss.nttdata.com> <237e8d26-d185-1008-6e07-5b9f36314534@oracle.com> <2a9ba1f8-b1f7-cb8c-fe11-56dd174d9d58@oracle.com> <7ad907cb-24ef-aa55-972a-0a3f63576568@oracle.com> Message-ID: Hi Patricio, Thanks for your advice! I've fixed testcase as [1], but I still see an error in validate-headers-linux-x64-build-1 on submit repo. What does it mean? Can you share details? mach5-one-ysuenaga-JDK-8242428-20200710-0339-12529134 Of course this change can be built on my Linux box (Fedora 32, AMD64, GCC 10.1) Yasumasa [1] http://hg.openjdk.java.net/jdk/submit/file/45f52142db9d/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp On 2020/07/10 2:18, Patricio Chilano wrote: > > On 7/9/20 12:00 PM, Patricio Chilano wrote: >> Hi Yasumasa, >> >> On 7/9/20 9:30 AM, Yasumasa Suenaga wrote: >>> On 2020/07/09 17:58, David Holmes wrote: >>>> Hi Yasumasa, >>>> >>>> On 9/07/2020 10:25 am, Yasumasa Suenaga wrote: >>>>> Hi Dan, >>>>> >>>>> Thanks for your comment! >>>>> I uploaded new webrev: >>>>> >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.09/ >>>>> ?? Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/5d167adf8524 >>>>> >>>>> I saw similar build errors in libOneGetThreadListStackTraces.cpp on Windows. >>>>> This webrev fixes them. >>>> >>>> You shouldn't use %p as it may not be portable. In the VM we use INTPTR_FORMAT and convert the arg using p2i. I don't know what exists in the testing code. >>> >>> I replaced %p to %lx, and also cast values to unsigned long [1] [2], but the test on submit repo was failed. >>> Can anyone share details of mach5-one-ysuenaga-JDK-8242428-20200709-1030-12500928 ? >> These are the errors I see for the macOS build: >> >> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:53:14: error: format specifies type 'long long' but the argument has type 'jlocation' (aka 'long') [-Werror,-Wformat] >> ????????????? fi1->location, fi2->location); >> ????????????? ^~~~~~~~~~~~~ >> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:53:29: error: format specifies type 'long long' but the argument has type 'jlocation' (aka 'long') [-Werror,-Wformat] >> ????????????? fi1->location, fi2->location); >> ???????????????????????????? ^~~~~~~~~~~~~ >> >> These are the ones I see for the Windows build: >> >> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): warning C4311: 'type cast': pointer truncation from 'jmethodID' to 'unsigned long' >> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): warning C4302: 'type cast': truncation from 'jmethodID' to 'unsigned long' >> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): warning C4311: 'type cast': pointer truncation from 'jmethodID' to 'unsigned long' >> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): warning C4302: 'type cast': truncation from 'jmethodID' to 'unsigned long' >> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): warning C4311: 'type cast': pointer truncation from 'jthread' to 'unsigned long' >> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): warning C4302: 'type cast': truncation from 'jthread' to 'unsigned long' >> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): warning C4311: 'type cast': pointer truncation from 'jthread' to 'unsigned long' >> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): warning C4302: 'type cast': truncation from 'jthread' to 'unsigned long' >> >> >> You will probably want to use the macros defined in src/hotspot/share/utilities/globalDefinitions.hpp. Let me know if you need me to test something. > With these changes the build works okay on Linux, Windows and macOS: > > > --- a/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp > +++ b/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp > @@ -27,4 +27,5 @@ > ?#include > ?#include > +#include > > ?#define MAX_FRAMES 100 > @@ -45,11 +46,11 @@ > ?? if (fi1->method != fi2->method) { /* jvmtiFrameInfo::method */ > ???? snprintf(err_msg, sizeof(err_msg), > -???????????? "method is different: fi1 = %p, fi2 = %p", > -???????????? fi1->method, fi2->method); > +???????????? "method is different: fi1 = 0x%016" PRIxPTR " , fi2 = 0x%016" PRIxPTR, > +???????????? (intptr_t)fi1->method, (intptr_t)fi2->method); > ???? env->FatalError(err_msg); > ?? } else if (fi1->location != fi2->location) { /* jvmtiFrameInfo::location */ > ???? snprintf(err_msg, sizeof(err_msg), > -???????????? "location is different: fi1 = %lld, fi2 = %lld", > -???????????? fi1->location, fi2->location); > +???????????? "location is different: fi1 = %" PRId64 " , fi2 = %" PRId64, > +???????????? (int64_t)fi1->location, (int64_t)fi2->location); > ???? env->FatalError(err_msg); > ?? } > @@ -67,5 +68,5 @@ > ?? if (!is_same) { /* jvmtiStackInfo::thread */ > ???? snprintf(err_msg, sizeof(err_msg), > -???????????? "thread is different: si1 = %p, si2 = %p", si1->thread, si2->thread); > +???????????? "thread is different: si1 = 0x%016" PRIxPTR " , si2 = 0x%016" PRIxPTR, (intptr_t)si1->thread, (intptr_t)si2->thread); > ???? env->FatalError(err_msg); > ?? } else if (si1->state != si2->state) { /* jvmtiStackInfo::state */ > > > Maybe you can use something like that. > > > Thanks, > Patricio >> Thanks, >> Patricio >>> Thanks, >>> >>> Yasumasa >>> >>> >>> [1] http://hg.openjdk.java.net/jdk/submit/rev/dfca51958217 >>> [2] http://hg.openjdk.java.net/jdk/submit/rev/3665361fa91b >>> >>> >>>> David >>>> ----- >>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2020/07/09 1:42, Daniel D. Daugherty wrote: >>>>>> ?> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ >>>>>> >>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>> ???? No comments. >>>>>> >>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>> ???? L1159: ? Thread *current_thread = Thread::current(); >>>>>> ???????? Please add "#ifdef ASSERT" above and "#endif" below since >>>>>> ???????? current_thread is only used for the assert() in this function. >>>>>> >>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>> ???? L549: ???????????????????????????? jthread java_thread, jint max_frame_count) >>>>>> ???? L552: ????? _jthread(java_thread), >>>>>> ???????? Please: s/java_thread/thread/ on both lines. >>>>>> >>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>> ???? No comments. >>>>>> >>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>> ???? No comments. >>>>>> >>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>> ???? L27: #include >>>>>> ???????? This include is out of order; should be first in the list. >>>>>> >>>>>> ???? This file doesn't compile on my MBP13: >>>>>> >>>>>> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:49:14: error: format specifies type 'unsigned long' but the argument has type 'jmethodID' (aka '_jmethodID *') [-Werror,-Wformat] >>>>>> ????????????? fi1->method, fi2->method); >>>>>> ????????????? ^~~~~~~~~~~ >>>>>> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:49:27: error: format specifies type 'unsigned long' but the argument has type 'jmethodID' (aka '_jmethodID *') [-Werror,-Wformat] >>>>>> ????????????? fi1->method, fi2->method); >>>>>> ?????????????????????????? ^~~~~~~~~~~ >>>>>> ???? 2 errors generated. >>>>>> >>>>>> ???? This change made it compile on my MBP13, but that may break it on >>>>>> ???? other platforms: >>>>>> >>>>>> ???? $ hg diff >>>>>> ???? diff -r 560847c69fbe test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>> ???? --- a/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp Wed Jul 08 12:13:32 2020 -0400 >>>>>> ???? +++ b/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp Wed Jul 08 12:40:42 2020 -0400 >>>>>> ???? @@ -46,7 +46,7 @@ >>>>>> ?? ?? ? if (fi1->method != fi2->method) { /* jvmtiFrameInfo::method */ >>>>>> ?? ?? ??? snprintf(err_msg, sizeof(err_msg), >>>>>> ?? ? ? ??????????? "method is different: fi1 = %lx, fi2 = %lx", >>>>>> ???? -???????????? fi1->method, fi2->method); >>>>>> ???? +???????????? (unsigned long) fi1->method, (unsigned long) fi2->method); >>>>>> ?? ?? ??? env->FatalError(err_msg); >>>>>> ?? ?? ? } else if (fi1->location != fi2->location) { /* jvmtiFrameInfo::location */ >>>>>> ?? ?? ??? snprintf(err_msg, sizeof(err_msg), >>>>>> >>>>>> ???? I'm not sure of the right platform independent way to output >>>>>> ???? the 'method' field. >>>>>> >>>>>> Dan >>>>>> >>>>>> >>>>>> On 7/8/20 4:04 AM, Yasumasa Suenaga wrote: >>>>>>> Hi David, >>>>>>> >>>>>>> On 2020/07/08 15:27, David Holmes wrote: >>>>>>>> Hi Yasumasa, >>>>>>>> >>>>>>>> On 7/07/2020 6:54 pm, Yasumasa Suenaga wrote: >>>>>>>>> Hi David, Serguei, >>>>>>>>> >>>>>>>>> Serguei, thank you for replying even though you are on vacaiton! >>>>>>>>> >>>>>>>>> I uploaded new webrev: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.07/ >>>>>>>>> ?? Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/77243b1dcbfe >>>>>>>>> >>>>>>>>> c'tor of GetSingleStackTraceClosure has jthread argument in this webrev. >>>>>>>>> Also it does not contain testcase for GetThreadListStackTraces with all threads, and OneGetThreadListStackTraces would test main thread only. >>>>>>>> >>>>>>>> All those changes are fine in principle for me. One nit/suggestion: >>>>>>>> >>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>> >>>>>>>> ??544?? jthread _java_thread; >>>>>>>> >>>>>>>> elsewhere "java_thread" refers to a JavaThread, so to avoid confusion may I suggest this member be named _jthread. >>>>>>> >>>>>>> I uploaded new webrev: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ >>>>>>> ? Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/ca6263dbdc87 >>>>>>> >>>>>>> >>>>>>>> I'm going to be away for the next couple of days - sorry - but will try to check email on this if I can. >>>>>>> >>>>>>> Thanks! >>>>>>> >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> ----- >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2020/07/07 15:13, David Holmes wrote: >>>>>>>>>> On 7/07/2020 2:57 pm, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi David, >>>>>>>>>>> >>>>>>>>>>> On 2020/07/07 11:31, David Holmes wrote: >>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>> >>>>>>>>>>>> Hard to keep up with the changes - especially without incremental webrevs. >>>>>>>>>>> >>>>>>>>>>> Sorry, I will upload diff from previous webrev in the next. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> If GetSingleStackTraceClosure also took the jthread as a constructor arg, then you wouldn't need to recreate a JNI local handle when calling _collector.fill_frames. It's a small simplification and not essential at this stage. >>>>>>>>>>> >>>>>>>>>>> I think we should get jthread from an argument of do_thread() because do_thread() would pass the thread which are stopped certainly. >>>>>>>>>>> It might be simplification if we pass _calling_thread to MultipleStackTracesCollector. `jthread` is only needed to store jvmtiStackInfo.thread . What do you think? >>>>>>>>>> >>>>>>>>>> I'm not quite sure what you mean. >>>>>>>>>> >>>>>>>>>> I think there is a bit of a design wart with direct handshakes in that do_thread takes the target JavaThread as an argument. That's useful in a case where you want a HandshakeClosure that can be applied to multiple threads, but that's not typically what is needed with direct handshakes - there is only a single target. With a single-target HandshakeClosure you can capture all the "target" information for the operation in the closure instance. So if the actual do_thread operation wants the jthread corresponding to the target thread then we can store that in the closure rather than recomputing it (you could assert it is the same but that seems overkill to me). >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> For the test ... I don't see how Java_GetThreadListStackTraces_checkCallStacks is a valid test. It gets the stacks of all live threads, then uses that information to use GetThreadListStackTraces to get the stack for the same set of threads through a different API. It then compares the two sets of stacks for each thread expecting them to be the same, but that need only be the case for the main thread. Other threads could potentially have a different stack (e.g. if this test is run with JFR enabled there will be additional threads found.) Further I would have expected that there already exist tests that check that, for a given thread (which may be suspended or known to be blocked) the same stack is found through the two different APIs. >>>>>>>>>>> >>>>>>>>>>> vmTestbase/nsk/jvmti/unit/GetAllStackTraces/getallstktr001 would check all of threads via GetThreadListStackTraces() and GetAllStackTraces(), so we might be able to remove GetThreadListStackTraces.java from this webrev. >>>>>>>>>> >>>>>>>>>> Yes. The existing test only examines a set of test threads that are all blocked on a raw monitor. You do not need to duplicate that test. >>>>>>>>>> >>>>>>>>>>> OTOH we don't have testcase for GetThreadListStackTraces() with thread_count == 1, so we need to testcase for it (it is OneGetThreadListStackTraces.java) It would check whether the state of target thread is "waiting" before JNI call to call GetThreadListStackTraces(), >>>>>>>>>> >>>>>>>>>> Yes we need to test the special cases introduced by your changes - totally agree - and OneGetThreadListStackTraces.java is a good test for that. >>>>>>>>>> >>>>>>>>>>> and also I expect it would not be run with JFR. (it is not described @run) >>>>>>>>>> >>>>>>>>>> The arguments to run with JFR (or a bunch of other things) can be passed to the jtreg test harness to be applied to all tests. >>>>>>>>>> >>>>>>>>>>> Of course we can check GetThreadListStackTraces() with main thread, but it is not the test for direct handshake for other thread. >>>>>>>>>> >>>>>>>>>> Right - that test already exists as per the above. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> David >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> David >>>>>>>>>>>> ----- >>>>>>>>>>>> >>>>>>>>>>>> On 6/07/2020 11:29 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>> >>>>>>>>>>>>> I think C++ is more simple to implement the test agent as you said. >>>>>>>>>>>>> So I implement it in C++ in new webrev. Could you review again? >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.06/ >>>>>>>>>>>>> >>>>>>>>>>>>> Also I refactored libGetThreadListStackTraces.cpp, and I've kept exception check after IsSameObject(). >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 2020/07/06 16:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thank you for the update. >>>>>>>>>>>>>> I think, a pending exception after IsSameObject needs to be checked. >>>>>>>>>>>>>> >>>>>>>>>>>>>> The checkStackInfo() needs one more refactoring as I've already suggested. >>>>>>>>>>>>>> The body of the loop at L68-L78 should be converted to a function check_frame_info. >>>>>>>>>>>>>> The si1->frame_buffer[i] and si2->frame_buffer[i] will be passed as fi1 and fi2. >>>>>>>>>>>>>> The index can be passed as well. >>>>>>>>>>>>>> I'm still suggesting to simplify the local exception_msg to something shorter like err_msg or exc_msg. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I'm not sure using fatal is right here: >>>>>>>>>>>>>> >>>>>>>>>>>>>> This fragment looks strange: >>>>>>>>>>>>>> >>>>>>>>>>>>>> ? 152???? if ((*env)->IsSameObject(env, stack_info[i].thread, thread)) { >>>>>>>>>>>>>> ? 153?????? target_info = &stack_info[i]; >>>>>>>>>>>>>> ? 154?????? break; >>>>>>>>>>>>>> ? 155???? } else if ((*env)->ExceptionOccurred(env)) { >>>>>>>>>>>>>> ? 156 (*env)->ExceptionDescribe(env); >>>>>>>>>>>>>> ? 157?????? (*env)->FatalError(env, __FILE__); >>>>>>>>>>>>>> ? 158???? } >>>>>>>>>>>>>> >>>>>>>>>>>>>> I expected it to be: >>>>>>>>>>>>>> >>>>>>>>>>>>>> ??? jboolean same = (*env)->IsSameObject(env, stack_info[i].thread, thread); >>>>>>>>>>>>>> ??? if ((*env)->ExceptionOccurred(env)) { >>>>>>>>>>>>>> ????? (*env)->ExceptionDescribe(env); >>>>>>>>>>>>>> ????? (*env)->FatalError(env, __FILE__); >>>>>>>>>>>>>> ??? } >>>>>>>>>>>>>> ??? if (same) { >>>>>>>>>>>>>> ????? target_info = &stack_info[i]; >>>>>>>>>>>>>> ????? break; >>>>>>>>>>>>>> ??? } >>>>>>>>>>>>>> >>>>>>>>>>>>>> Would it better to port this agent to C++ to simplify this code nicer? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 7/5/20 06:13, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>> I refactored testcase. Could you review again? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.05/ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> It would check Java exception after IsSameObject() call. Does it need? >>>>>>>>>>>>>>> Any exceptions are not described in JNI document[1], and JNI implementation (jni_IsSameObject()) does not seem to throw it. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> [1] https://docs.oracle.com/en/java/javase/14/docs/specs/jni/functions.html#issameobject >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 2020/07/05 14:46, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Okay, thanks. >>>>>>>>>>>>>>>> Then I'm okay to keep the GetSingleStackTraceClosure. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c.html >>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c.html >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I'm not sure the function 'is_same_thread() is needed. >>>>>>>>>>>>>>>> Why do not use the JNI IsSameObject instead? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> It seems to be a typo at L132 and L137. >>>>>>>>>>>>>>>> You, probably. did not want to print the same information for stack_info_1[i].frame_buffer[j].XXX twice. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> The code at lines 112-142 is not readable. >>>>>>>>>>>>>>>> I'd suggest to make a couple of refactoring steps. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> First step to simplify this a little bit would be with some renaming and getting rid of indexes: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> ?? 71 char err_msg[EXCEPTION_MSG_LEN] = {0}; >>>>>>>>>>>>>>>> ??... >>>>>>>>>>>>>>>> ??112?? /* Iterate all jvmtiStackInfo to check */ >>>>>>>>>>>>>>>> ??113?? for (i = 0; i < num_threads, *exception_msg != '\0'; i++) { >>>>>>>>>>>>>>>> ????????? jvmtiStackInfo *si1 = stack_info_1[i]; >>>>>>>>>>>>>>>> ????????? jvmtiStackInfo *si2 = stack_info_2[i]; >>>>>>>>>>>>>>>> ??114???? if (!IsSameObject(env, si1.thread, si2.thread)) { /* jvmtiStackInfo::thread */ >>>>>>>>>>>>>>>> ??115?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>> ??116??????????????? "thread[%d] is different: stack_info_1 = %p, stack_info_2 = %p", >>>>>>>>>>>>>>>> ??117??????????????? i, sinfo1.thread, sinfo2.thread); >>>>>>>>>>>>>>>> ??118???? } else if (si1.state != si2.state) { /* jvmtiStackInfo::state */ >>>>>>>>>>>>>>>> ??119?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>> ??120??????????????? "state[%d] is different: stack_info_1 = %d, stack_info_2 = %d", >>>>>>>>>>>>>>>> ??121??????????????? i, si1.state, si2.state); >>>>>>>>>>>>>>>> ??122???? } else if (si1.frame_count != si2.frame_count) { /* jvmtiStackInfo::frame_count */ >>>>>>>>>>>>>>>> ??123?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>> ??124??????????????? "frame_count[%d] is different: stack_info_1 = %d, stack_info_2 = %d", >>>>>>>>>>>>>>>> ??125??????????????? i, si1.frame_count, si2.frame_count); >>>>>>>>>>>>>>>> ??126???? } else { >>>>>>>>>>>>>>>> ??127?????? /* Iterate all jvmtiFrameInfo to check */ >>>>>>>>>>>>>>>> ??128?????? for (j = 0; j < si1.frame_count; j++) { >>>>>>>>>>>>>>>> ??129???????? if (si1.frame_buffer[j].method != si1.frame_buffer[j].method) { /* jvmtiFrameInfo::method */ >>>>>>>>>>>>>>>> ??130?????????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>> ??131??????????????????? "thread [%d] frame_buffer[%d].method is different: stack_info_1 = %lx, stack_info_2 = %lx", >>>>>>>>>>>>>>>> ??132??????????????????? i, j, si1.frame_buffer[j].method, si2.frame_buffer[j].method); >>>>>>>>>>>>>>>> ??133?????????? break; >>>>>>>>>>>>>>>> ??134???????? } else if (si1.frame_buffer[j].location != si1.frame_buffer[j].location) { /* jvmtiFrameInfo::location */ >>>>>>>>>>>>>>>> ??135?????????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>> ??136??????????????????? "thread [%d] frame_buffer[%d].location is different: stack_info_1 = %ld, stack_info_2 = %ld", >>>>>>>>>>>>>>>> ??137??????????????????? i, j, si1.frame_buffer[j].location, si2.frame_buffer[j].location); >>>>>>>>>>>>>>>> ??138?????????? break; >>>>>>>>>>>>>>>> ??139???????? } >>>>>>>>>>>>>>>> ??140?????? } >>>>>>>>>>>>>>>> ??141???? } >>>>>>>>>>>>>>>> ??142?? } >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Another step would be to create functions that implement a body of each loop. >>>>>>>>>>>>>>>> You can use the same techniques to simplify similar place (L127-L138) in the libOneGetThreadListStackTraces.c. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 7/3/20 15:55, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I'm not an Oracle employee, so I cannot know real request(s) from your customers. >>>>>>>>>>>>>>>>> However JDK-8201641 says Dynatrace has requested this enhancement. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> BTW I haven't heared any request from my customers about this. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 2020/07/04 4:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> This difference is not that big to care about. >>>>>>>>>>>>>>>>>> I feel this is really rare case and so, does not worth these complications. >>>>>>>>>>>>>>>>>> Do we have a real request from customers to optimize it? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 7/3/20 01:16, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Generally I agree with you, but I have concern about the difference of the result of GetStackTrace() and GetThreadListStackTraces(). >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ? GetStackTrace: jvmtiFrameInfo >>>>>>>>>>>>>>>>>>> ? GetThreadListStackTraces: jvmtiStackInfo >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> jvmtiStackInfo contains thread state, and it is ensured it is the state of the call stack. >>>>>>>>>>>>>>>>>>> If we want to get both call stack and thread state, we need to suspend target thread, and call both GetStackTrace() and GetThreadState(). Is it ok? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I was wondering if JDK-8201641 (parent ticket of this change) needed them for profiling (dynatrace?) >>>>>>>>>>>>>>>>>>> If it is responsibility of JVMTI agent implementor, I remove this closure. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On 2020/07/03 16:45, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> After some thinking I've concluded that I do not like this optimization >>>>>>>>>>>>>>>>>>>> of the GetThreadListStackTraces with GetSingleStackTraceClosure. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> We may need more opinions on this but these are my points: >>>>>>>>>>>>>>>>>>>> ??- it adds some complexity and ugliness >>>>>>>>>>>>>>>>>>>> ??- a win is doubtful because it has to be a rare case, so that total overhead should not be high >>>>>>>>>>>>>>>>>>>> ??- if it is really high for some use cases then it is up to the user >>>>>>>>>>>>>>>>>>>> ??? to optimize it with using GetStackTrace instead >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> In such cases with doubtful overhead I usually prefer the simplicity. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Good examples where it makes sense to optimize are checks for target thread to be current thread. >>>>>>>>>>>>>>>>>>>> In such cases there is no need to suspend the target thread, or use a VMop/HandshakeClosure. >>>>>>>>>>>>>>>>>>>> For instance, please, see the Monitor functions with the check: (java_thread == calling_thread). >>>>>>>>>>>>>>>>>>>> Getting information for current thread is frequently used case, e.g. to get info at an event point. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On 7/2/20 23:29, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>> Hi Dan, David, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could you review again? >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/ >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> OneGetThreadListStackTraces.java in this webrev would wait until thread state is transited to "waiting" with spin wait. >>>>>>>>>>>>>>>>>>>>> CountDownLatch::await call as Dan pointed is fixed in it :) >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Diff from webrev.03: >>>>>>>>>>>>>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/c9aeb7001e50 >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On 2020/07/03 14:15, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 12:24, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>>>>>>>> On 7/2/20 10:50 PM, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>> Sorry I'm responding here without seeing latest webrev but there is enough context I think ... >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> Hi Dan, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I upload new webrev. Could you review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L1542: // Get stack trace with handshake >>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - please add a period at the end. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> L1591: *stack_info_ptr = op.stack_info(); >>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? The return parameter should not be touched unless the return >>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> ???? old L1582: if (err == JVMTI_ERROR_NONE) { >>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? Please restore this check. The return parameter should not >>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? be touched unless the return code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> But op.stack_info() will return NULL if the error is not JVMTI_ERROR_NONE. Are you (Dan) concerned about someone passing in a non-null/initialized out-pointer that will be reset to NULL if there was an error? >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Actually the way we used to test this in POSIX tests is to call >>>>>>>>>>>>>>>>>>>>>>>> an API with known bad parameters and the return parameter ptr >>>>>>>>>>>>>>>>>>>>>>>> set to NULL. If the return parameter ptr was touched when an >>>>>>>>>>>>>>>>>>>>>>>> error should have been detected on an earlier parameter, then >>>>>>>>>>>>>>>>>>>>>>>> the test failed. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> L1272: ? if (!jt->is_exiting() && (thread_oop != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>> ???? old L1532: _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? This deletion of the _result field threw me for a minute and then >>>>>>>>>>>>>>>>>>>>>>>>>>> ???? ? ? I figured out that the field is init to JVMTI_ERROR_THREAD_NOT_ALIVE >>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? in the constructor. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L1553: ? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L64: startSignal.countDown(); >>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? I was expecting this to be a call to await() instead of >>>>>>>>>>>>>>>>>>>>>>>>>>> countDown(). What am I missing here? >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? I think this test might be passing by accident right now, but... >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Main thread (which call JVMTI functions to test) should wait until test thread is ready. >>>>>>>>>>>>>>>>>>>>>>>>>> So main thread would wait startSignal, and test thread would count down. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> No! >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> The test thread that previously called obj.wait() now calls latch.await(). >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> The main thread that previously called obj.notify() now calls latch.countDown(). >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> The main thread continues to spin until it sees the target is WAITING before proceeding with the test. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> If I add spin wait to wait until transit target thread state is WAITING (as following), we don't need to call SuspendThread(). >>>>>>>>>>>>>>>>>>>>>>> Which is better? >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> The original spin-wait loop checking for? WAITING is better because it is the only guarantee that the target thread is blocked where you need it to be. suspending the thread is racy as you don't know exactly where the suspend will hit. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>> /* Wait until the thread state transits to "waiting" */ >>>>>>>>>>>>>>>>>>>>>>> while (th.getState() != Thread.State.WAITING) { >>>>>>>>>>>>>>>>>>>>>>> Thread.onSpinWait(); >>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> For simplify, spin wait is prefer to OneGetThreadListStackTraces.java in webrev.03. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Here's the flow as I see it: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>>>> ?? - start worker thread >>>>>>>>>>>>>>>>>>>>>>>> ?? - startSignal.await() >>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>>>>>>>> ?? - startSignal.countDown() >>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>>>>>>>> ?? - stopSignal.await() >>>>>>>>>>>>>>>>>>>>>>>> ???? - worker is now blocked >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>>>> ?? - checkCallStacks(th) >>>>>>>>>>>>>>>>>>>>>>>> ?? - stopSignal.countDown() >>>>>>>>>>>>>>>>>>>>>>>> ???? - worker is now unblocked >>>>>>>>>>>>>>>>>>>>>>>> ?? - th.join >>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>>>>>>>> ?? - runs off the end of run() >>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>>>> ?? - run off the end of main() >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L92: jthreads = (jthread *)malloc(sizeof(jthread) * num_threads); >>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? You don't check for malloc() failure. >>>>>>>>>>>>>>>>>>>>>>>>>>> 'jthreads' is allocated but never freed. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L91: result = (*jvmti)->SuspendThread(jvmti, thread); >>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? Why are you suspending the thread? GetAllStackTraces() and >>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() do not require the target thread(s) >>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? to be suspend. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? If you decide not to SuspendThread, then you don't need the >>>>>>>>>>>>>>>>>>>>>>>>>>> AddCapabilities or the ResumeThread calls. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Test thread might not be entered following code (stopSignal.await()). We might see deferent call stack between GetAllStackTraces() and GetThreadListStackTraces(). We cannot control to freeze call stack of test thread in Java code. >>>>>>>>>>>>>>>>>>>>>>>>>> (I didn't use SuspendThread() at first, but I saw some errors which causes in above.) >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> So we need to call SuspendThread() to ensure we can see same call stack. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> If you are checking that the thread is in state WAITING then it cannot escape from that state and you can sample the stack multiple times from any API and get the same result. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> I suspect the errors you saw were from the apparent incorrect use of the CountDownLatch. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> With the flow outlined above, the worker thread should be >>>>>>>>>>>>>>>>>>>>>>>> nicely blocked in stopSignal.await() when stuff is sampled. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Updates look fine - thanks. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> One minor nit: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1274 _collector.allocate_and_fill_stacks(1); >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1275 _collector.set_result(JVMTI_ERROR_NONE); >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the other places where you use _collector you rely on result being initialized to JVMTI_ERROR_NONE, rather than setting it directly after allocate_and_fill_stacks(). >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 823 "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This function (JvmtiEnvBase::get_stack_trace()) can be called to get own stack trace. For example, we can call GetStackTrace() for current thread at JVMTI event. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So I changed assert as below: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??820 assert(current_thread == java_thread || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??821 SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??823 "call by myself / at safepoint / at handshake"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yep good catch. I hope current tests caught that. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> They would be tested in vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ (own call stacks), and getstacktr003 (call stacks in other thread). >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Speaking of tests ... >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the native code I think you need to check the success of all JNI methods that can throw exceptions - otherwise I believe the tests may trigger warnings if -Xcheck:jni is used with them. See for example: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I updated testcases to check JNI and JVMTI function calls. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the Java code the target thread: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 45 public void run() { >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 46 try { >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 47 synchronized (lock) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 48 lock.wait(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 49 System.out.println("OK"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 50?????????? } >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> is potentially susceptible to spurious wakeups. Using a CountDownLatch would be robust. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If so, we can call make_local() in L1272 without JavaThread (or we can pass current thread to make_local()). Is it right? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry I got confused, _calling_thread may not be the current thread as we could be executing the handshake in the target thread itself. So the ResourceMark is correct as-is (implicitly for current thread). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The argument to fill_frames will be used in the jvmtiStackInfo and passed back to the _calling_thread, so it must be created via make_local(_calling_thread, ...) as you presently have. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you for reviewing! I will update new webrev tomorrow. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 500 JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??? : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I updated webrev for 8242428. Could you review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change migrate to use direct handshake for GetStackTrace() and GetThreadListStackTraces() (when thread_count == 1). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks really good now! I only have a few nits below. There is one thing I don't like about it but it requires a change to the main Handshake logic to address - in JvmtiEnv::GetThreadListStackTraces you have to create a ThreadsListHandle to convert the jthread to a JavaThread, but then the Handshake::execute_direct creates another ThreadsListHandle internally. That's a waste. I will discuss with Robbin and file a RFE to have an overload of execute_direct that takes an existing TLH. Actually it's worse than that because we have another TLH in use at the entry point for the JVMTI functions, so I think there may be some scope for simplifying the use of TLH instances - future RFE. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??451 GetStackTraceClosure(JvmtiEnv *env, jint start_depth, jint max_count, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??452 jvmtiFrameInfo* frame_buffer, jint* count_ptr) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??454 _env(env), _start_depth(start_depth), _max_count(max_count), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??455 _frame_buffer(frame_buffer), _count_ptr(count_ptr), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??456 _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Nit: can you do one initializer per line please. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??500 JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 481 MultipleStackTracesCollector(JvmtiEnv *env, jint max_frame_count) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??483 _max_frame_count = max_frame_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??484 _frame_count_total = 0; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??485 _head = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??486 _stack_info = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??487 _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As you are touching this can you change it to use an initializer list as you did for the HandshakeClosure, and please keep one item per line. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??823 "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1269 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is frustrating that this entire call chain started with a jthread reference, which we converted to a JavaThread, only to eventually need to convert it back to a jthread! I think there is some scope for simplification here but not as part of this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm still pondering the test logic but wanted to send this now. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetThreadListStackTrace (for GetThreadListStackTraces) and VM_GetAllStackTraces (for GetAllStackTraces) have inherited VM_GetMultipleStackTraces VM operation which provides the feature to generate jvmtiStackInfo. I modified VM_GetMultipleStackTraces to a normal C++ class to share with HandshakeClosure for GetThreadListStackTraces (GetSingleStackTraceClosure). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I added new testcases which test GetThreadListStackTraces() with thread_count == 1 and with all threads. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/nsk/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change replace following VM operations to direct handshake. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces (GetThreadListStackTrace()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTrace() uses direct handshake if thread count == 1. In other case (thread count > 1), it would be performed as VM operation (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Caller of VM_GetCurrentLocation (JvmtiEnvThreadState::reset_current_location()) might be called at safepoint. So I added safepoint check in its caller. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/ns >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I tested it on submit repo, then it has execution error (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) due to dependency error. So I think it does not occur by this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>> >> > From patricio.chilano.mateo at oracle.com Fri Jul 10 05:49:25 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Fri, 10 Jul 2020 02:49:25 -0300 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <4c43bc2c-551e-d042-8fa7-b63e755888d1@oracle.com> <8c387550-a06e-6320-f453-0e898486befb@oss.nttdata.com> <19cfa2a7-f441-425d-5623-1c11c6a870b6@oss.nttdata.com> <237e8d26-d185-1008-6e07-5b9f36314534@oracle.com> <2a9ba1f8-b1f7-cb8c-fe11-56dd174d9d58@oracle.com> <7ad907cb-24ef-aa55-972a-0a3f63576568@oracle.com> Message-ID: <5ab8c5d9-9a5e-74a6-4605-7e73732037bf@oracle.com> Hi Yasumasa, On 7/10/20 1:58 AM, Yasumasa Suenaga wrote: > Hi Patricio, > > Thanks for your advice! > I've fixed testcase as [1], but I still see an error in > validate-headers-linux-x64-build-1 on submit repo. > What does it mean? Can you share details? > > ? mach5-one-ysuenaga-JDK-8242428-20200710-0339-12529134 Not your bug, you are getting the header validation failure from 8248570. It was fixed two days ago so just update the repo and it should work now. Thanks, Patricio > Of course this change can be built on my Linux box (Fedora 32, AMD64, > GCC 10.1) > > > Yasumasa > > > [1] > http://hg.openjdk.java.net/jdk/submit/file/45f52142db9d/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp > > > On 2020/07/10 2:18, Patricio Chilano wrote: >> >> On 7/9/20 12:00 PM, Patricio Chilano wrote: >>> Hi Yasumasa, >>> >>> On 7/9/20 9:30 AM, Yasumasa Suenaga wrote: >>>> On 2020/07/09 17:58, David Holmes wrote: >>>>> Hi Yasumasa, >>>>> >>>>> On 9/07/2020 10:25 am, Yasumasa Suenaga wrote: >>>>>> Hi Dan, >>>>>> >>>>>> Thanks for your comment! >>>>>> I uploaded new webrev: >>>>>> >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.09/ >>>>>> ?? Diff from previous webrev: >>>>>> http://hg.openjdk.java.net/jdk/submit/rev/5d167adf8524 >>>>>> >>>>>> I saw similar build errors in libOneGetThreadListStackTraces.cpp >>>>>> on Windows. >>>>>> This webrev fixes them. >>>>> >>>>> You shouldn't use %p as it may not be portable. In the VM we use >>>>> INTPTR_FORMAT and convert the arg using p2i. I don't know what >>>>> exists in the testing code. >>>> >>>> I replaced %p to %lx, and also cast values to unsigned long [1] >>>> [2], but the test on submit repo was failed. >>>> Can anyone share details of >>>> mach5-one-ysuenaga-JDK-8242428-20200709-1030-12500928 ? >>> These are the errors I see for the macOS build: >>> >>> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:53:14: >>> error: format specifies type 'long long' but the argument has type >>> 'jlocation' (aka 'long') [-Werror,-Wformat] >>> ????????????? fi1->location, fi2->location); >>> ????????????? ^~~~~~~~~~~~~ >>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:53:29: >>> error: format specifies type 'long long' but the argument has type >>> 'jlocation' (aka 'long') [-Werror,-Wformat] >>> ????????????? fi1->location, fi2->location); >>> ???????????????????????????? ^~~~~~~~~~~~~ >>> >>> These are the ones I see for the Windows build: >>> >>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): >>> warning C4311: 'type cast': pointer truncation from 'jmethodID' to >>> 'unsigned long' >>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): >>> warning C4302: 'type cast': truncation from 'jmethodID' to 'unsigned >>> long' >>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): >>> warning C4311: 'type cast': pointer truncation from 'jmethodID' to >>> 'unsigned long' >>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): >>> warning C4302: 'type cast': truncation from 'jmethodID' to 'unsigned >>> long' >>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): >>> warning C4311: 'type cast': pointer truncation from 'jthread' to >>> 'unsigned long' >>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): >>> warning C4302: 'type cast': truncation from 'jthread' to 'unsigned >>> long' >>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): >>> warning C4311: 'type cast': pointer truncation from 'jthread' to >>> 'unsigned long' >>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): >>> warning C4302: 'type cast': truncation from 'jthread' to 'unsigned >>> long' >>> >>> >>> You will probably want to use the macros defined in >>> src/hotspot/share/utilities/globalDefinitions.hpp. Let me know if >>> you need me to test something. >> With these changes the build works okay on Linux, Windows and macOS: >> >> >> --- >> a/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >> +++ >> b/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >> @@ -27,4 +27,5 @@ >> ??#include >> ??#include >> +#include >> >> ??#define MAX_FRAMES 100 >> @@ -45,11 +46,11 @@ >> ??? if (fi1->method != fi2->method) { /* jvmtiFrameInfo::method */ >> ????? snprintf(err_msg, sizeof(err_msg), >> -???????????? "method is different: fi1 = %p, fi2 = %p", >> -???????????? fi1->method, fi2->method); >> +???????????? "method is different: fi1 = 0x%016" PRIxPTR " , fi2 = >> 0x%016" PRIxPTR, >> +???????????? (intptr_t)fi1->method, (intptr_t)fi2->method); >> ????? env->FatalError(err_msg); >> ??? } else if (fi1->location != fi2->location) { /* >> jvmtiFrameInfo::location */ >> ????? snprintf(err_msg, sizeof(err_msg), >> -???????????? "location is different: fi1 = %lld, fi2 = %lld", >> -???????????? fi1->location, fi2->location); >> +???????????? "location is different: fi1 = %" PRId64 " , fi2 = %" >> PRId64, >> +???????????? (int64_t)fi1->location, (int64_t)fi2->location); >> ????? env->FatalError(err_msg); >> ??? } >> @@ -67,5 +68,5 @@ >> ??? if (!is_same) { /* jvmtiStackInfo::thread */ >> ????? snprintf(err_msg, sizeof(err_msg), >> -???????????? "thread is different: si1 = %p, si2 = %p", si1->thread, >> si2->thread); >> +???????????? "thread is different: si1 = 0x%016" PRIxPTR " , si2 = >> 0x%016" PRIxPTR, (intptr_t)si1->thread, (intptr_t)si2->thread); >> ????? env->FatalError(err_msg); >> ??? } else if (si1->state != si2->state) { /* jvmtiStackInfo::state */ >> >> >> Maybe you can use something like that. >> >> >> Thanks, >> Patricio >>> Thanks, >>> Patricio >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> [1] http://hg.openjdk.java.net/jdk/submit/rev/dfca51958217 >>>> [2] http://hg.openjdk.java.net/jdk/submit/rev/3665361fa91b >>>> >>>> >>>>> David >>>>> ----- >>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2020/07/09 1:42, Daniel D. Daugherty wrote: >>>>>>> ?> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ >>>>>>> >>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>> ???? No comments. >>>>>>> >>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>> ???? L1159: ? Thread *current_thread = Thread::current(); >>>>>>> ???????? Please add "#ifdef ASSERT" above and "#endif" below since >>>>>>> ???????? current_thread is only used for the assert() in this >>>>>>> function. >>>>>>> >>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>> ???? L549: ???????????????????????????? jthread java_thread, >>>>>>> jint max_frame_count) >>>>>>> ???? L552: ????? _jthread(java_thread), >>>>>>> ???????? Please: s/java_thread/thread/ on both lines. >>>>>>> >>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>> ???? No comments. >>>>>>> >>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>> >>>>>>> ???? No comments. >>>>>>> >>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>>> >>>>>>> ???? L27: #include >>>>>>> ???????? This include is out of order; should be first in the list. >>>>>>> >>>>>>> ???? This file doesn't compile on my MBP13: >>>>>>> >>>>>>> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:49:14: >>>>>>> error: format specifies type 'unsigned long' but the argument >>>>>>> has type 'jmethodID' (aka '_jmethodID *') [-Werror,-Wformat] >>>>>>> ????????????? fi1->method, fi2->method); >>>>>>> ????????????? ^~~~~~~~~~~ >>>>>>> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:49:27: >>>>>>> error: format specifies type 'unsigned long' but the argument >>>>>>> has type 'jmethodID' (aka '_jmethodID *') [-Werror,-Wformat] >>>>>>> ????????????? fi1->method, fi2->method); >>>>>>> ?????????????????????????? ^~~~~~~~~~~ >>>>>>> ???? 2 errors generated. >>>>>>> >>>>>>> ???? This change made it compile on my MBP13, but that may break >>>>>>> it on >>>>>>> ???? other platforms: >>>>>>> >>>>>>> ???? $ hg diff >>>>>>> ???? diff -r 560847c69fbe >>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>>> ???? --- >>>>>>> a/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>>> Wed Jul 08 12:13:32 2020 -0400 >>>>>>> ???? +++ >>>>>>> b/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>>> Wed Jul 08 12:40:42 2020 -0400 >>>>>>> ???? @@ -46,7 +46,7 @@ >>>>>>> ?? ?? ? if (fi1->method != fi2->method) { /* >>>>>>> jvmtiFrameInfo::method */ >>>>>>> ?? ?? ??? snprintf(err_msg, sizeof(err_msg), >>>>>>> ?? ? ? ??????????? "method is different: fi1 = %lx, fi2 = %lx", >>>>>>> ???? -???????????? fi1->method, fi2->method); >>>>>>> ???? +???????????? (unsigned long) fi1->method, (unsigned long) >>>>>>> fi2->method); >>>>>>> ?? ?? ??? env->FatalError(err_msg); >>>>>>> ?? ?? ? } else if (fi1->location != fi2->location) { /* >>>>>>> jvmtiFrameInfo::location */ >>>>>>> ?? ?? ??? snprintf(err_msg, sizeof(err_msg), >>>>>>> >>>>>>> ???? I'm not sure of the right platform independent way to output >>>>>>> ???? the 'method' field. >>>>>>> >>>>>>> Dan >>>>>>> >>>>>>> >>>>>>> On 7/8/20 4:04 AM, Yasumasa Suenaga wrote: >>>>>>>> Hi David, >>>>>>>> >>>>>>>> On 2020/07/08 15:27, David Holmes wrote: >>>>>>>>> Hi Yasumasa, >>>>>>>>> >>>>>>>>> On 7/07/2020 6:54 pm, Yasumasa Suenaga wrote: >>>>>>>>>> Hi David, Serguei, >>>>>>>>>> >>>>>>>>>> Serguei, thank you for replying even though you are on vacaiton! >>>>>>>>>> >>>>>>>>>> I uploaded new webrev: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.07/ >>>>>>>>>> ?? Diff from previous webrev: >>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/77243b1dcbfe >>>>>>>>>> >>>>>>>>>> c'tor of GetSingleStackTraceClosure has jthread argument in >>>>>>>>>> this webrev. >>>>>>>>>> Also it does not contain testcase for >>>>>>>>>> GetThreadListStackTraces with all threads, and >>>>>>>>>> OneGetThreadListStackTraces would test main thread only. >>>>>>>>> >>>>>>>>> All those changes are fine in principle for me. One >>>>>>>>> nit/suggestion: >>>>>>>>> >>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>> >>>>>>>>> ??544?? jthread _java_thread; >>>>>>>>> >>>>>>>>> elsewhere "java_thread" refers to a JavaThread, so to avoid >>>>>>>>> confusion may I suggest this member be named _jthread. >>>>>>>> >>>>>>>> I uploaded new webrev: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ >>>>>>>> ? Diff from previous webrev: >>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/ca6263dbdc87 >>>>>>>> >>>>>>>> >>>>>>>>> I'm going to be away for the next couple of days - sorry - but >>>>>>>>> will try to check email on this if I can. >>>>>>>> >>>>>>>> Thanks! >>>>>>>> >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> David >>>>>>>>> ----- >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2020/07/07 15:13, David Holmes wrote: >>>>>>>>>>> On 7/07/2020 2:57 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>> Hi David, >>>>>>>>>>>> >>>>>>>>>>>> On 2020/07/07 11:31, David Holmes wrote: >>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>> >>>>>>>>>>>>> Hard to keep up with the changes - especially without >>>>>>>>>>>>> incremental webrevs. >>>>>>>>>>>> >>>>>>>>>>>> Sorry, I will upload diff from previous webrev in the next. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> If GetSingleStackTraceClosure also took the jthread as a >>>>>>>>>>>>> constructor arg, then you wouldn't need to recreate a JNI >>>>>>>>>>>>> local handle when calling _collector.fill_frames. It's a >>>>>>>>>>>>> small simplification and not essential at this stage. >>>>>>>>>>>> >>>>>>>>>>>> I think we should get jthread from an argument of >>>>>>>>>>>> do_thread() because do_thread() would pass the thread which >>>>>>>>>>>> are stopped certainly. >>>>>>>>>>>> It might be simplification if we pass _calling_thread to >>>>>>>>>>>> MultipleStackTracesCollector. `jthread` is only needed to >>>>>>>>>>>> store jvmtiStackInfo.thread . What do you think? >>>>>>>>>>> >>>>>>>>>>> I'm not quite sure what you mean. >>>>>>>>>>> >>>>>>>>>>> I think there is a bit of a design wart with direct >>>>>>>>>>> handshakes in that do_thread takes the target JavaThread as >>>>>>>>>>> an argument. That's useful in a case where you want a >>>>>>>>>>> HandshakeClosure that can be applied to multiple threads, >>>>>>>>>>> but that's not typically what is needed with direct >>>>>>>>>>> handshakes - there is only a single target. With a >>>>>>>>>>> single-target HandshakeClosure you can capture all the >>>>>>>>>>> "target" information for the operation in the closure >>>>>>>>>>> instance. So if the actual do_thread operation wants the >>>>>>>>>>> jthread corresponding to the target thread then we can store >>>>>>>>>>> that in the closure rather than recomputing it (you could >>>>>>>>>>> assert it is the same but that seems overkill to me). >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> For the test ... I don't see how >>>>>>>>>>>>> Java_GetThreadListStackTraces_checkCallStacks is a valid >>>>>>>>>>>>> test. It gets the stacks of all live threads, then uses >>>>>>>>>>>>> that information to use GetThreadListStackTraces to get >>>>>>>>>>>>> the stack for the same set of threads through a different >>>>>>>>>>>>> API. It then compares the two sets of stacks for each >>>>>>>>>>>>> thread expecting them to be the same, but that need only >>>>>>>>>>>>> be the case for the main thread. Other threads could >>>>>>>>>>>>> potentially have a different stack (e.g. if this test is >>>>>>>>>>>>> run with JFR enabled there will be additional threads >>>>>>>>>>>>> found.) Further I would have expected that there already >>>>>>>>>>>>> exist tests that check that, for a given thread (which may >>>>>>>>>>>>> be suspended or known to be blocked) the same stack is >>>>>>>>>>>>> found through the two different APIs. >>>>>>>>>>>> >>>>>>>>>>>> vmTestbase/nsk/jvmti/unit/GetAllStackTraces/getallstktr001 >>>>>>>>>>>> would check all of threads via GetThreadListStackTraces() >>>>>>>>>>>> and GetAllStackTraces(), so we might be able to remove >>>>>>>>>>>> GetThreadListStackTraces.java from this webrev. >>>>>>>>>>> >>>>>>>>>>> Yes. The existing test only examines a set of test threads >>>>>>>>>>> that are all blocked on a raw monitor. You do not need to >>>>>>>>>>> duplicate that test. >>>>>>>>>>> >>>>>>>>>>>> OTOH we don't have testcase for GetThreadListStackTraces() >>>>>>>>>>>> with thread_count == 1, so we need to testcase for it (it >>>>>>>>>>>> is OneGetThreadListStackTraces.java) It would check whether >>>>>>>>>>>> the state of target thread is "waiting" before JNI call to >>>>>>>>>>>> call GetThreadListStackTraces(), >>>>>>>>>>> >>>>>>>>>>> Yes we need to test the special cases introduced by your >>>>>>>>>>> changes - totally agree - and >>>>>>>>>>> OneGetThreadListStackTraces.java is a good test for that. >>>>>>>>>>> >>>>>>>>>>>> and also I expect it would not be run with JFR. (it is not >>>>>>>>>>>> described @run) >>>>>>>>>>> >>>>>>>>>>> The arguments to run with JFR (or a bunch of other things) >>>>>>>>>>> can be passed to the jtreg test harness to be applied to all >>>>>>>>>>> tests. >>>>>>>>>>> >>>>>>>>>>>> Of course we can check GetThreadListStackTraces() with main >>>>>>>>>>>> thread, but it is not the test for direct handshake for >>>>>>>>>>>> other thread. >>>>>>>>>>> >>>>>>>>>>> Right - that test already exists as per the above. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> David >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> David >>>>>>>>>>>>> ----- >>>>>>>>>>>>> >>>>>>>>>>>>> On 6/07/2020 11:29 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>> >>>>>>>>>>>>>> I think C++ is more simple to implement the test agent as >>>>>>>>>>>>>> you said. >>>>>>>>>>>>>> So I implement it in C++ in new webrev. Could you review >>>>>>>>>>>>>> again? >>>>>>>>>>>>>> >>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.06/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> Also I refactored libGetThreadListStackTraces.cpp, and >>>>>>>>>>>>>> I've kept exception check after IsSameObject(). >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 2020/07/06 16:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thank you for the update. >>>>>>>>>>>>>>> I think, a pending exception after IsSameObject needs to >>>>>>>>>>>>>>> be checked. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The checkStackInfo() needs one more refactoring as I've >>>>>>>>>>>>>>> already suggested. >>>>>>>>>>>>>>> The body of the loop at L68-L78 should be converted to a >>>>>>>>>>>>>>> function check_frame_info. >>>>>>>>>>>>>>> The si1->frame_buffer[i] and si2->frame_buffer[i] will >>>>>>>>>>>>>>> be passed as fi1 and fi2. >>>>>>>>>>>>>>> The index can be passed as well. >>>>>>>>>>>>>>> I'm still suggesting to simplify the local exception_msg >>>>>>>>>>>>>>> to something shorter like err_msg or exc_msg. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I'm not sure using fatal is right here: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> This fragment looks strange: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ? 152???? if ((*env)->IsSameObject(env, >>>>>>>>>>>>>>> stack_info[i].thread, thread)) { >>>>>>>>>>>>>>> ? 153?????? target_info = &stack_info[i]; >>>>>>>>>>>>>>> ? 154?????? break; >>>>>>>>>>>>>>> ? 155???? } else if ((*env)->ExceptionOccurred(env)) { >>>>>>>>>>>>>>> ? 156 (*env)->ExceptionDescribe(env); >>>>>>>>>>>>>>> ? 157?????? (*env)->FatalError(env, __FILE__); >>>>>>>>>>>>>>> ? 158???? } >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I expected it to be: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ??? jboolean same = (*env)->IsSameObject(env, >>>>>>>>>>>>>>> stack_info[i].thread, thread); >>>>>>>>>>>>>>> ??? if ((*env)->ExceptionOccurred(env)) { >>>>>>>>>>>>>>> (*env)->ExceptionDescribe(env); >>>>>>>>>>>>>>> ????? (*env)->FatalError(env, __FILE__); >>>>>>>>>>>>>>> ??? } >>>>>>>>>>>>>>> ??? if (same) { >>>>>>>>>>>>>>> ????? target_info = &stack_info[i]; >>>>>>>>>>>>>>> ????? break; >>>>>>>>>>>>>>> ??? } >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Would it better to port this agent to C++ to simplify >>>>>>>>>>>>>>> this code nicer? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 7/5/20 06:13, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>> I refactored testcase. Could you review again? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.05/ >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> It would check Java exception after IsSameObject() >>>>>>>>>>>>>>>> call. Does it need? >>>>>>>>>>>>>>>> Any exceptions are not described in JNI document[1], >>>>>>>>>>>>>>>> and JNI implementation (jni_IsSameObject()) does not >>>>>>>>>>>>>>>> seem to throw it. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>> https://docs.oracle.com/en/java/javase/14/docs/specs/jni/functions.html#issameobject >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 2020/07/05 14:46, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Okay, thanks. >>>>>>>>>>>>>>>>> Then I'm okay to keep the GetSingleStackTraceClosure. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c.html >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c.html >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I'm not sure the function 'is_same_thread() is needed. >>>>>>>>>>>>>>>>> Why do not use the JNI IsSameObject instead? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> It seems to be a typo at L132 and L137. >>>>>>>>>>>>>>>>> You, probably. did not want to print the same >>>>>>>>>>>>>>>>> information for stack_info_1[i].frame_buffer[j].XXX >>>>>>>>>>>>>>>>> twice. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> The code at lines 112-142 is not readable. >>>>>>>>>>>>>>>>> I'd suggest to make a couple of refactoring steps. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> First step to simplify this a little bit would be with >>>>>>>>>>>>>>>>> some renaming and getting rid of indexes: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> ?? 71 char err_msg[EXCEPTION_MSG_LEN] = {0}; >>>>>>>>>>>>>>>>> ??... >>>>>>>>>>>>>>>>> ??112?? /* Iterate all jvmtiStackInfo to check */ >>>>>>>>>>>>>>>>> ??113?? for (i = 0; i < num_threads, *exception_msg != >>>>>>>>>>>>>>>>> '\0'; i++) { >>>>>>>>>>>>>>>>> ????????? jvmtiStackInfo *si1 = stack_info_1[i]; >>>>>>>>>>>>>>>>> ????????? jvmtiStackInfo *si2 = stack_info_2[i]; >>>>>>>>>>>>>>>>> ??114???? if (!IsSameObject(env, si1.thread, >>>>>>>>>>>>>>>>> si2.thread)) { /* jvmtiStackInfo::thread */ >>>>>>>>>>>>>>>>> ??115?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>> ??116??????????????? "thread[%d] is different: >>>>>>>>>>>>>>>>> stack_info_1 = %p, stack_info_2 = %p", >>>>>>>>>>>>>>>>> ??117??????????????? i, sinfo1.thread, sinfo2.thread); >>>>>>>>>>>>>>>>> ??118???? } else if (si1.state != si2.state) { /* >>>>>>>>>>>>>>>>> jvmtiStackInfo::state */ >>>>>>>>>>>>>>>>> ??119?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>> ??120??????????????? "state[%d] is different: >>>>>>>>>>>>>>>>> stack_info_1 = %d, stack_info_2 = %d", >>>>>>>>>>>>>>>>> ??121??????????????? i, si1.state, si2.state); >>>>>>>>>>>>>>>>> ??122???? } else if (si1.frame_count != >>>>>>>>>>>>>>>>> si2.frame_count) { /* jvmtiStackInfo::frame_count */ >>>>>>>>>>>>>>>>> ??123?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>> ??124 "frame_count[%d] is different: stack_info_1 = >>>>>>>>>>>>>>>>> %d, stack_info_2 = %d", >>>>>>>>>>>>>>>>> ??125??????????????? i, si1.frame_count, >>>>>>>>>>>>>>>>> si2.frame_count); >>>>>>>>>>>>>>>>> ??126???? } else { >>>>>>>>>>>>>>>>> ??127?????? /* Iterate all jvmtiFrameInfo to check */ >>>>>>>>>>>>>>>>> ??128?????? for (j = 0; j < si1.frame_count; j++) { >>>>>>>>>>>>>>>>> ??129???????? if (si1.frame_buffer[j].method != >>>>>>>>>>>>>>>>> si1.frame_buffer[j].method) { /* >>>>>>>>>>>>>>>>> jvmtiFrameInfo::method */ >>>>>>>>>>>>>>>>> ??130?????????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>> ??131??????????????????? "thread [%d] >>>>>>>>>>>>>>>>> frame_buffer[%d].method is different: stack_info_1 = >>>>>>>>>>>>>>>>> %lx, stack_info_2 = %lx", >>>>>>>>>>>>>>>>> ??132??????????????????? i, j, >>>>>>>>>>>>>>>>> si1.frame_buffer[j].method, si2.frame_buffer[j].method); >>>>>>>>>>>>>>>>> ??133?????????? break; >>>>>>>>>>>>>>>>> ??134???????? } else if (si1.frame_buffer[j].location >>>>>>>>>>>>>>>>> != si1.frame_buffer[j].location) { /* >>>>>>>>>>>>>>>>> jvmtiFrameInfo::location */ >>>>>>>>>>>>>>>>> ??135?????????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>> ??136??????????????????? "thread [%d] >>>>>>>>>>>>>>>>> frame_buffer[%d].location is different: stack_info_1 = >>>>>>>>>>>>>>>>> %ld, stack_info_2 = %ld", >>>>>>>>>>>>>>>>> ??137??????????????????? i, j, >>>>>>>>>>>>>>>>> si1.frame_buffer[j].location, >>>>>>>>>>>>>>>>> si2.frame_buffer[j].location); >>>>>>>>>>>>>>>>> ??138?????????? break; >>>>>>>>>>>>>>>>> ??139???????? } >>>>>>>>>>>>>>>>> ??140?????? } >>>>>>>>>>>>>>>>> ??141???? } >>>>>>>>>>>>>>>>> ??142?? } >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Another step would be to create functions that >>>>>>>>>>>>>>>>> implement a body of each loop. >>>>>>>>>>>>>>>>> You can use the same techniques to simplify similar >>>>>>>>>>>>>>>>> place (L127-L138) in the >>>>>>>>>>>>>>>>> libOneGetThreadListStackTraces.c. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 7/3/20 15:55, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I'm not an Oracle employee, so I cannot know real >>>>>>>>>>>>>>>>>> request(s) from your customers. >>>>>>>>>>>>>>>>>> However JDK-8201641 says Dynatrace has requested this >>>>>>>>>>>>>>>>>> enhancement. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> BTW I haven't heared any request from my customers >>>>>>>>>>>>>>>>>> about this. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 2020/07/04 4:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> This difference is not that big to care about. >>>>>>>>>>>>>>>>>>> I feel this is really rare case and so, does not >>>>>>>>>>>>>>>>>>> worth these complications. >>>>>>>>>>>>>>>>>>> Do we have a real request from customers to optimize >>>>>>>>>>>>>>>>>>> it? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On 7/3/20 01:16, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Generally I agree with you, but I have concern >>>>>>>>>>>>>>>>>>>> about the difference of the result of >>>>>>>>>>>>>>>>>>>> GetStackTrace() and GetThreadListStackTraces(). >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> ? GetStackTrace: jvmtiFrameInfo >>>>>>>>>>>>>>>>>>>> ? GetThreadListStackTraces: jvmtiStackInfo >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> jvmtiStackInfo contains thread state, and it is >>>>>>>>>>>>>>>>>>>> ensured it is the state of the call stack. >>>>>>>>>>>>>>>>>>>> If we want to get both call stack and thread state, >>>>>>>>>>>>>>>>>>>> we need to suspend target thread, and call both >>>>>>>>>>>>>>>>>>>> GetStackTrace() and GetThreadState(). Is it ok? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I was wondering if JDK-8201641 (parent ticket of >>>>>>>>>>>>>>>>>>>> this change) needed them for profiling (dynatrace?) >>>>>>>>>>>>>>>>>>>> If it is responsibility of JVMTI agent implementor, >>>>>>>>>>>>>>>>>>>> I remove this closure. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On 2020/07/03 16:45, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> After some thinking I've concluded that I do not >>>>>>>>>>>>>>>>>>>>> like this optimization >>>>>>>>>>>>>>>>>>>>> of the GetThreadListStackTraces with >>>>>>>>>>>>>>>>>>>>> GetSingleStackTraceClosure. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> We may need more opinions on this but these are my >>>>>>>>>>>>>>>>>>>>> points: >>>>>>>>>>>>>>>>>>>>> ??- it adds some complexity and ugliness >>>>>>>>>>>>>>>>>>>>> ??- a win is doubtful because it has to be a rare >>>>>>>>>>>>>>>>>>>>> case, so that total overhead should not be high >>>>>>>>>>>>>>>>>>>>> ??- if it is really high for some use cases then >>>>>>>>>>>>>>>>>>>>> it is up to the user >>>>>>>>>>>>>>>>>>>>> ??? to optimize it with using GetStackTrace instead >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> In such cases with doubtful overhead I usually >>>>>>>>>>>>>>>>>>>>> prefer the simplicity. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Good examples where it makes sense to optimize are >>>>>>>>>>>>>>>>>>>>> checks for target thread to be current thread. >>>>>>>>>>>>>>>>>>>>> In such cases there is no need to suspend the >>>>>>>>>>>>>>>>>>>>> target thread, or use a VMop/HandshakeClosure. >>>>>>>>>>>>>>>>>>>>> For instance, please, see the Monitor functions >>>>>>>>>>>>>>>>>>>>> with the check: (java_thread == calling_thread). >>>>>>>>>>>>>>>>>>>>> Getting information for current thread is >>>>>>>>>>>>>>>>>>>>> frequently used case, e.g. to get info at an event >>>>>>>>>>>>>>>>>>>>> point. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On 7/2/20 23:29, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>> Hi Dan, David, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could you review again? >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/ >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> OneGetThreadListStackTraces.java in this webrev >>>>>>>>>>>>>>>>>>>>>> would wait until thread state is transited to >>>>>>>>>>>>>>>>>>>>>> "waiting" with spin wait. >>>>>>>>>>>>>>>>>>>>>> CountDownLatch::await call as Dan pointed is >>>>>>>>>>>>>>>>>>>>>> fixed in it :) >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Diff from webrev.03: >>>>>>>>>>>>>>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/c9aeb7001e50 >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 14:15, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 12:24, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>>>>>>>>> On 7/2/20 10:50 PM, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> Sorry I'm responding here without seeing >>>>>>>>>>>>>>>>>>>>>>>>>> latest webrev but there is enough context I >>>>>>>>>>>>>>>>>>>>>>>>>> think ... >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Dan, >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I upload new webrev. Could you review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L1542: // Get stack trace with handshake >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - please add a period at the end. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> L1591: *stack_info_ptr = op.stack_info(); >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? The return parameter should not be >>>>>>>>>>>>>>>>>>>>>>>>>>>> touched unless the return >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? old L1582: if (err == JVMTI_ERROR_NONE) { >>>>>>>>>>>>>>>>>>>>>>>>>>>> Please restore this check. The return >>>>>>>>>>>>>>>>>>>>>>>>>>>> parameter should not >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? be touched unless the return code >>>>>>>>>>>>>>>>>>>>>>>>>>>> in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> But op.stack_info() will return NULL if the >>>>>>>>>>>>>>>>>>>>>>>>>> error is not JVMTI_ERROR_NONE. Are you (Dan) >>>>>>>>>>>>>>>>>>>>>>>>>> concerned about someone passing in a >>>>>>>>>>>>>>>>>>>>>>>>>> non-null/initialized out-pointer that will be >>>>>>>>>>>>>>>>>>>>>>>>>> reset to NULL if there was an error? >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Actually the way we used to test this in POSIX >>>>>>>>>>>>>>>>>>>>>>>>> tests is to call >>>>>>>>>>>>>>>>>>>>>>>>> an API with known bad parameters and the >>>>>>>>>>>>>>>>>>>>>>>>> return parameter ptr >>>>>>>>>>>>>>>>>>>>>>>>> set to NULL. If the return parameter ptr was >>>>>>>>>>>>>>>>>>>>>>>>> touched when an >>>>>>>>>>>>>>>>>>>>>>>>> error should have been detected on an earlier >>>>>>>>>>>>>>>>>>>>>>>>> parameter, then >>>>>>>>>>>>>>>>>>>>>>>>> the test failed. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> L1272: ? if (!jt->is_exiting() && >>>>>>>>>>>>>>>>>>>>>>>>>>>> (thread_oop != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the >>>>>>>>>>>>>>>>>>>>>>>>>>>> second expression. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? old L1532: _result = >>>>>>>>>>>>>>>>>>>>>>>>>>>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? This deletion of the _result field >>>>>>>>>>>>>>>>>>>>>>>>>>>> threw me for a minute and then >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? ? ? I figured out that the field is >>>>>>>>>>>>>>>>>>>>>>>>>>>> init to JVMTI_ERROR_THREAD_NOT_ALIVE >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? in the constructor. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L1553: if (!jt->is_exiting() && >>>>>>>>>>>>>>>>>>>>>>>>>>>> (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the >>>>>>>>>>>>>>>>>>>>>>>>>>>> second expression. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L64: startSignal.countDown(); >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? I was expecting this to be a call >>>>>>>>>>>>>>>>>>>>>>>>>>>> to await() instead of >>>>>>>>>>>>>>>>>>>>>>>>>>>> countDown(). What am I missing here? >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? I think this test might be passing >>>>>>>>>>>>>>>>>>>>>>>>>>>> by accident right now, but... >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Main thread (which call JVMTI functions to >>>>>>>>>>>>>>>>>>>>>>>>>>> test) should wait until test thread is ready. >>>>>>>>>>>>>>>>>>>>>>>>>>> So main thread would wait startSignal, and >>>>>>>>>>>>>>>>>>>>>>>>>>> test thread would count down. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> No! >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> The test thread that previously called >>>>>>>>>>>>>>>>>>>>>>>>>> obj.wait() now calls latch.await(). >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> The main thread that previously called >>>>>>>>>>>>>>>>>>>>>>>>>> obj.notify() now calls latch.countDown(). >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> The main thread continues to spin until it >>>>>>>>>>>>>>>>>>>>>>>>>> sees the target is WAITING before proceeding >>>>>>>>>>>>>>>>>>>>>>>>>> with the test. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> If I add spin wait to wait until transit target >>>>>>>>>>>>>>>>>>>>>>>> thread state is WAITING (as following), we >>>>>>>>>>>>>>>>>>>>>>>> don't need to call SuspendThread(). >>>>>>>>>>>>>>>>>>>>>>>> Which is better? >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> The original spin-wait loop checking for WAITING >>>>>>>>>>>>>>>>>>>>>>> is better because it is the only guarantee that >>>>>>>>>>>>>>>>>>>>>>> the target thread is blocked where you need it >>>>>>>>>>>>>>>>>>>>>>> to be. suspending the thread is racy as you >>>>>>>>>>>>>>>>>>>>>>> don't know exactly where the suspend will hit. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>> /* Wait until the thread state transits to >>>>>>>>>>>>>>>>>>>>>>>> "waiting" */ >>>>>>>>>>>>>>>>>>>>>>>> while (th.getState() != Thread.State.WAITING) { >>>>>>>>>>>>>>>>>>>>>>>> Thread.onSpinWait(); >>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> For simplify, spin wait is prefer to >>>>>>>>>>>>>>>>>>>>>>>> OneGetThreadListStackTraces.java in webrev.03. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Here's the flow as I see it: >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>>>>> ?? - start worker thread >>>>>>>>>>>>>>>>>>>>>>>>> ?? - startSignal.await() >>>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>>>>>>>>> ?? - startSignal.countDown() >>>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>>>>>>>>> ?? - stopSignal.await() >>>>>>>>>>>>>>>>>>>>>>>>> ???? - worker is now blocked >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>>>>> ?? - checkCallStacks(th) >>>>>>>>>>>>>>>>>>>>>>>>> ?? - stopSignal.countDown() >>>>>>>>>>>>>>>>>>>>>>>>> ???? - worker is now unblocked >>>>>>>>>>>>>>>>>>>>>>>>> ?? - th.join >>>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>>>>>>>>> ?? - runs off the end of run() >>>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>>>>> ?? - run off the end of main() >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L92: jthreads = (jthread >>>>>>>>>>>>>>>>>>>>>>>>>>>> *)malloc(sizeof(jthread) * num_threads); >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? You don't check for malloc() failure. >>>>>>>>>>>>>>>>>>>>>>>>>>>> 'jthreads' is allocated but never freed. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L91: result = >>>>>>>>>>>>>>>>>>>>>>>>>>>> (*jvmti)->SuspendThread(jvmti, thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? Why are you suspending the thread? >>>>>>>>>>>>>>>>>>>>>>>>>>>> GetAllStackTraces() and >>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() do not require >>>>>>>>>>>>>>>>>>>>>>>>>>>> the target thread(s) >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? to be suspend. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? If you decide not to >>>>>>>>>>>>>>>>>>>>>>>>>>>> SuspendThread, then you don't need the >>>>>>>>>>>>>>>>>>>>>>>>>>>> AddCapabilities or the ResumeThread calls. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Test thread might not be entered following >>>>>>>>>>>>>>>>>>>>>>>>>>> code (stopSignal.await()). We might see >>>>>>>>>>>>>>>>>>>>>>>>>>> deferent call stack between >>>>>>>>>>>>>>>>>>>>>>>>>>> GetAllStackTraces() and >>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces(). We cannot >>>>>>>>>>>>>>>>>>>>>>>>>>> control to freeze call stack of test thread >>>>>>>>>>>>>>>>>>>>>>>>>>> in Java code. >>>>>>>>>>>>>>>>>>>>>>>>>>> (I didn't use SuspendThread() at first, but >>>>>>>>>>>>>>>>>>>>>>>>>>> I saw some errors which causes in above.) >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> So we need to call SuspendThread() to ensure >>>>>>>>>>>>>>>>>>>>>>>>>>> we can see same call stack. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> If you are checking that the thread is in >>>>>>>>>>>>>>>>>>>>>>>>>> state WAITING then it cannot escape from that >>>>>>>>>>>>>>>>>>>>>>>>>> state and you can sample the stack multiple >>>>>>>>>>>>>>>>>>>>>>>>>> times from any API and get the same result. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> I suspect the errors you saw were from the >>>>>>>>>>>>>>>>>>>>>>>>>> apparent incorrect use of the CountDownLatch. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> With the flow outlined above, the worker >>>>>>>>>>>>>>>>>>>>>>>>> thread should be >>>>>>>>>>>>>>>>>>>>>>>>> nicely blocked in stopSignal.await() when >>>>>>>>>>>>>>>>>>>>>>>>> stuff is sampled. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Updates look fine - thanks. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> One minor nit: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1274 _collector.allocate_and_fill_stacks(1); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1275 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector.set_result(JVMTI_ERROR_NONE); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the other places where you use >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector you rely on result being >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initialized to JVMTI_ERROR_NONE, rather >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than setting it directly after >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocate_and_fill_stacks(). >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 820 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> assert(SafepointSynchronize::is_at_safepoint() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 821 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 822 current_thread == >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 823 "at safepoint / handshake or >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is necessary, as even if the target >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is suspended we must still be at a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> safepoint or in a handshake with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it. Makes me wonder if we used to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allow a racy stacktrace operation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on a suspended thread, assuming it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This function >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (JvmtiEnvBase::get_stack_trace()) can be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> called to get own stack trace. For >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> example, we can call GetStackTrace() for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> current thread at JVMTI event. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So I changed assert as below: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??820 assert(current_thread == >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??821 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??823 "call by myself / at safepoint / >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at handshake"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yep good catch. I hope current tests >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> caught that. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> They would be tested in >>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>> (own call stacks), and getstacktr003 (call >>>>>>>>>>>>>>>>>>>>>>>>>>>>> stacks in other thread). >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Speaking of tests ... >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the native code I think you need to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> check the success of all JNI methods that >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can throw exceptions - otherwise I >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> believe the tests may trigger warnings if >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Xcheck:jni is used with them. See for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> example: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I updated testcases to check JNI and JVMTI >>>>>>>>>>>>>>>>>>>>>>>>>>>>> function calls. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the Java code the target thread: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 45 public void run() { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 46 try { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 47 synchronized (lock) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 48 lock.wait(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 49 System.out.println("OK"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 50?????????? } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is potentially susceptible to spurious >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wakeups. Using a CountDownLatch would be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> robust. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _calling_thread is the current >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If so, we can call make_local() in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> L1272 without JavaThread (or we can >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pass current thread to make_local()). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is it right? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry I got confused, _calling_thread >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> may not be the current thread as we >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could be executing the handshake in the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target thread itself. So the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark is correct as-is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (implicitly for current thread). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The argument to fill_frames will be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> used in the jvmtiStackInfo and passed >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back to the _calling_thread, so it must >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be created via >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make_local(_calling_thread, ...) as you >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> presently have. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you for reviewing! I will >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> update new webrev tomorrow. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 498 class VM_GetAllStackTraces : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 500 JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 502 MultipleStackTracesCollector >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> member of another class like that >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as it may not be on the stack. I >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think MultipleStackTracesCollector >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should not extend any allocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class, and should always be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??? : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class GetAllStackTraces : public >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/30 22:22, David Holmes >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I updated webrev for 8242428. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Could you review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change migrate to use direct >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> handshake for GetStackTrace() and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() (when >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread_count == 1). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks really good now! I only >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a few nits below. There is one >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thing I don't like about it but it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> requires a change to the main >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Handshake logic to address - in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JvmtiEnv::GetThreadListStackTraces >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you have to create a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ThreadsListHandle to convert the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jthread to a JavaThread, but then >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the Handshake::execute_direct >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates another ThreadsListHandle >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> internally. That's a waste. I will >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> discuss with Robbin and file a RFE >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to have an overload of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> execute_direct that takes an >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> existing TLH. Actually it's worse >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than that because we have another >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> TLH in use at the entry point for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the JVMTI functions, so I think >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there may be some scope for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> simplifying the use of TLH >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> instances - future RFE. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??451 GetStackTraceClosure(JvmtiEnv >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *env, jint start_depth, jint >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max_count, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??452 jvmtiFrameInfo* frame_buffer, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jint* count_ptr) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??453???? : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??454 _env(env), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _start_depth(start_depth), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _max_count(max_count), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??455 _frame_buffer(frame_buffer), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _count_ptr(count_ptr), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??456 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _result(JVMTI_ERROR_THREAD_NOT_ALIVE) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Nit: can you do one initializer per >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> line please. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??498 class VM_GetAllStackTraces : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??500 JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??502 MultipleStackTracesCollector >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> member of another class like that >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as it may not be on the stack. I >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think MultipleStackTracesCollector >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should not extend any allocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class, and should always be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 481 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector(JvmtiEnv >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *env, jint max_frame_count) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??483 _max_frame_count = >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max_frame_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??484 _frame_count_total = 0; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??485 _head = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??486 _stack_info = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??487 _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As you are touching this can you >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change it to use an initializer >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> list as you did for the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HandshakeClosure, and please keep >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one item per line. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??820 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> assert(SafepointSynchronize::is_at_safepoint() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??821 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??823 "at safepoint / handshake or >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is necessary, as even if the target >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is suspended we must still be at a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> safepoint or in a handshake with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it. Makes me wonder if we used to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allow a racy stacktrace operation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on a suspended thread, assuming it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1268?? oop thread_oop = >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jt->threadObj(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1269 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is frustrating that this entire >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call chain started with a jthread >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reference, which we converted to a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JavaThread, only to eventually need >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to convert it back to a jthread! I >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think there is some scope for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> simplification here but not as part >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _calling_thread is the current >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm still pondering the test logic >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but wanted to send this now. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetThreadListStackTrace (for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces) and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetAllStackTraces (for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetAllStackTraces) have inherited >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetMultipleStackTraces VM >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operation which provides the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature to generate >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jvmtiStackInfo. I modified >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetMultipleStackTraces to a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> normal C++ class to share with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HandshakeClosure for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (GetSingleStackTraceClosure). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I added new testcases which >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test GetThreadListStackTraces() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with thread_count == 1 and with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> all threads. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jvmti >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jdwp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jdi >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? JBS: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? webrev: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change replace following VM >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operations to direct handshake. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameCount >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (GetFrameCount()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (GetFrameLocation()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (GetThreadListStackTrace()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTrace() uses >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> direct handshake if thread count >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> == 1. In other case (thread count >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > 1), it would be performed as VM >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Caller of VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (JvmtiEnvThreadState::reset_current_location()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> might be called at safepoint. So >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I added safepoint check in its >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> caller. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jvmti >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jdwp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jdi vmTestbase/ns >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I tested it on submit repo, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then it has execution error >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> due to dependency error. So I >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it does not occur by this >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>> >>> >> From suenaga at oss.nttdata.com Fri Jul 10 07:29:02 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Fri, 10 Jul 2020 16:29:02 +0900 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: <5ab8c5d9-9a5e-74a6-4605-7e73732037bf@oracle.com> References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <8c387550-a06e-6320-f453-0e898486befb@oss.nttdata.com> <19cfa2a7-f441-425d-5623-1c11c6a870b6@oss.nttdata.com> <237e8d26-d185-1008-6e07-5b9f36314534@oracle.com> <2a9ba1f8-b1f7-cb8c-fe11-56dd174d9d58@oracle.com> <7ad907cb-24ef-aa55-972a-0a3f63576568@oracle.com> <5ab8c5d9-9a5e-74a6-4605-7e73732037bf@oracle.com> Message-ID: <4881e0da-f716-c62e-b744-17aa3d03e095@oss.nttdata.com> Thanks Patricio! I uploaded new webrev. Could you review again? It passed all tests on submit repo, and passed jtreg tests about JVMTI. http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.10/ Diff from webrev.09: http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.10/incremental/ Yasumasa On 2020/07/10 14:49, Patricio Chilano wrote: > Hi Yasumasa, > > On 7/10/20 1:58 AM, Yasumasa Suenaga wrote: >> Hi Patricio, >> >> Thanks for your advice! >> I've fixed testcase as [1], but I still see an error in validate-headers-linux-x64-build-1 on submit repo. >> What does it mean? Can you share details? >> >> ? mach5-one-ysuenaga-JDK-8242428-20200710-0339-12529134 > Not your bug, you are getting the header validation failure from 8248570. It was fixed two days ago so just update the repo and it should work now. > > Thanks, > Patricio >> Of course this change can be built on my Linux box (Fedora 32, AMD64, GCC 10.1) >> >> >> Yasumasa >> >> >> [1] http://hg.openjdk.java.net/jdk/submit/file/45f52142db9d/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >> >> >> On 2020/07/10 2:18, Patricio Chilano wrote: >>> >>> On 7/9/20 12:00 PM, Patricio Chilano wrote: >>>> Hi Yasumasa, >>>> >>>> On 7/9/20 9:30 AM, Yasumasa Suenaga wrote: >>>>> On 2020/07/09 17:58, David Holmes wrote: >>>>>> Hi Yasumasa, >>>>>> >>>>>> On 9/07/2020 10:25 am, Yasumasa Suenaga wrote: >>>>>>> Hi Dan, >>>>>>> >>>>>>> Thanks for your comment! >>>>>>> I uploaded new webrev: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.09/ >>>>>>> ?? Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/5d167adf8524 >>>>>>> >>>>>>> I saw similar build errors in libOneGetThreadListStackTraces.cpp on Windows. >>>>>>> This webrev fixes them. >>>>>> >>>>>> You shouldn't use %p as it may not be portable. In the VM we use INTPTR_FORMAT and convert the arg using p2i. I don't know what exists in the testing code. >>>>> >>>>> I replaced %p to %lx, and also cast values to unsigned long [1] [2], but the test on submit repo was failed. >>>>> Can anyone share details of mach5-one-ysuenaga-JDK-8242428-20200709-1030-12500928 ? >>>> These are the errors I see for the macOS build: >>>> >>>> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:53:14: error: format specifies type 'long long' but the argument has type 'jlocation' (aka 'long') [-Werror,-Wformat] >>>> ????????????? fi1->location, fi2->location); >>>> ????????????? ^~~~~~~~~~~~~ >>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:53:29: error: format specifies type 'long long' but the argument has type 'jlocation' (aka 'long') [-Werror,-Wformat] >>>> ????????????? fi1->location, fi2->location); >>>> ???????????????????????????? ^~~~~~~~~~~~~ >>>> >>>> These are the ones I see for the Windows build: >>>> >>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): warning C4311: 'type cast': pointer truncation from 'jmethodID' to 'unsigned long' >>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): warning C4302: 'type cast': truncation from 'jmethodID' to 'unsigned long' >>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): warning C4311: 'type cast': pointer truncation from 'jmethodID' to 'unsigned long' >>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): warning C4302: 'type cast': truncation from 'jmethodID' to 'unsigned long' >>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): warning C4311: 'type cast': pointer truncation from 'jthread' to 'unsigned long' >>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): warning C4302: 'type cast': truncation from 'jthread' to 'unsigned long' >>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): warning C4311: 'type cast': pointer truncation from 'jthread' to 'unsigned long' >>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): warning C4302: 'type cast': truncation from 'jthread' to 'unsigned long' >>>> >>>> >>>> You will probably want to use the macros defined in src/hotspot/share/utilities/globalDefinitions.hpp. Let me know if you need me to test something. >>> With these changes the build works okay on Linux, Windows and macOS: >>> >>> >>> --- a/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>> +++ b/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>> @@ -27,4 +27,5 @@ >>> ??#include >>> ??#include >>> +#include >>> >>> ??#define MAX_FRAMES 100 >>> @@ -45,11 +46,11 @@ >>> ??? if (fi1->method != fi2->method) { /* jvmtiFrameInfo::method */ >>> ????? snprintf(err_msg, sizeof(err_msg), >>> -???????????? "method is different: fi1 = %p, fi2 = %p", >>> -???????????? fi1->method, fi2->method); >>> +???????????? "method is different: fi1 = 0x%016" PRIxPTR " , fi2 = 0x%016" PRIxPTR, >>> +???????????? (intptr_t)fi1->method, (intptr_t)fi2->method); >>> ????? env->FatalError(err_msg); >>> ??? } else if (fi1->location != fi2->location) { /* jvmtiFrameInfo::location */ >>> ????? snprintf(err_msg, sizeof(err_msg), >>> -???????????? "location is different: fi1 = %lld, fi2 = %lld", >>> -???????????? fi1->location, fi2->location); >>> +???????????? "location is different: fi1 = %" PRId64 " , fi2 = %" PRId64, >>> +???????????? (int64_t)fi1->location, (int64_t)fi2->location); >>> ????? env->FatalError(err_msg); >>> ??? } >>> @@ -67,5 +68,5 @@ >>> ??? if (!is_same) { /* jvmtiStackInfo::thread */ >>> ????? snprintf(err_msg, sizeof(err_msg), >>> -???????????? "thread is different: si1 = %p, si2 = %p", si1->thread, si2->thread); >>> +???????????? "thread is different: si1 = 0x%016" PRIxPTR " , si2 = 0x%016" PRIxPTR, (intptr_t)si1->thread, (intptr_t)si2->thread); >>> ????? env->FatalError(err_msg); >>> ??? } else if (si1->state != si2->state) { /* jvmtiStackInfo::state */ >>> >>> >>> Maybe you can use something like that. >>> >>> >>> Thanks, >>> Patricio >>>> Thanks, >>>> Patricio >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> [1] http://hg.openjdk.java.net/jdk/submit/rev/dfca51958217 >>>>> [2] http://hg.openjdk.java.net/jdk/submit/rev/3665361fa91b >>>>> >>>>> >>>>>> David >>>>>> ----- >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2020/07/09 1:42, Daniel D. Daugherty wrote: >>>>>>>> ?> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ >>>>>>>> >>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>> ???? No comments. >>>>>>>> >>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>> ???? L1159: ? Thread *current_thread = Thread::current(); >>>>>>>> ???????? Please add "#ifdef ASSERT" above and "#endif" below since >>>>>>>> ???????? current_thread is only used for the assert() in this function. >>>>>>>> >>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>> ???? L549: ???????????????????????????? jthread java_thread, jint max_frame_count) >>>>>>>> ???? L552: ????? _jthread(java_thread), >>>>>>>> ???????? Please: s/java_thread/thread/ on both lines. >>>>>>>> >>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>> ???? No comments. >>>>>>>> >>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>> ???? No comments. >>>>>>>> >>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>>>> ???? L27: #include >>>>>>>> ???????? This include is out of order; should be first in the list. >>>>>>>> >>>>>>>> ???? This file doesn't compile on my MBP13: >>>>>>>> >>>>>>>> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:49:14: error: format specifies type 'unsigned long' but the argument has type 'jmethodID' (aka '_jmethodID *') [-Werror,-Wformat] >>>>>>>> ????????????? fi1->method, fi2->method); >>>>>>>> ????????????? ^~~~~~~~~~~ >>>>>>>> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:49:27: error: format specifies type 'unsigned long' but the argument has type 'jmethodID' (aka '_jmethodID *') [-Werror,-Wformat] >>>>>>>> ????????????? fi1->method, fi2->method); >>>>>>>> ?????????????????????????? ^~~~~~~~~~~ >>>>>>>> ???? 2 errors generated. >>>>>>>> >>>>>>>> ???? This change made it compile on my MBP13, but that may break it on >>>>>>>> ???? other platforms: >>>>>>>> >>>>>>>> ???? $ hg diff >>>>>>>> ???? diff -r 560847c69fbe test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>>>> ???? --- a/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp Wed Jul 08 12:13:32 2020 -0400 >>>>>>>> ???? +++ b/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp Wed Jul 08 12:40:42 2020 -0400 >>>>>>>> ???? @@ -46,7 +46,7 @@ >>>>>>>> ?? ?? ? if (fi1->method != fi2->method) { /* jvmtiFrameInfo::method */ >>>>>>>> ?? ?? ??? snprintf(err_msg, sizeof(err_msg), >>>>>>>> ?? ? ? ??????????? "method is different: fi1 = %lx, fi2 = %lx", >>>>>>>> ???? -???????????? fi1->method, fi2->method); >>>>>>>> ???? +???????????? (unsigned long) fi1->method, (unsigned long) fi2->method); >>>>>>>> ?? ?? ??? env->FatalError(err_msg); >>>>>>>> ?? ?? ? } else if (fi1->location != fi2->location) { /* jvmtiFrameInfo::location */ >>>>>>>> ?? ?? ??? snprintf(err_msg, sizeof(err_msg), >>>>>>>> >>>>>>>> ???? I'm not sure of the right platform independent way to output >>>>>>>> ???? the 'method' field. >>>>>>>> >>>>>>>> Dan >>>>>>>> >>>>>>>> >>>>>>>> On 7/8/20 4:04 AM, Yasumasa Suenaga wrote: >>>>>>>>> Hi David, >>>>>>>>> >>>>>>>>> On 2020/07/08 15:27, David Holmes wrote: >>>>>>>>>> Hi Yasumasa, >>>>>>>>>> >>>>>>>>>> On 7/07/2020 6:54 pm, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>> >>>>>>>>>>> Serguei, thank you for replying even though you are on vacaiton! >>>>>>>>>>> >>>>>>>>>>> I uploaded new webrev: >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.07/ >>>>>>>>>>> ?? Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/77243b1dcbfe >>>>>>>>>>> >>>>>>>>>>> c'tor of GetSingleStackTraceClosure has jthread argument in this webrev. >>>>>>>>>>> Also it does not contain testcase for GetThreadListStackTraces with all threads, and OneGetThreadListStackTraces would test main thread only. >>>>>>>>>> >>>>>>>>>> All those changes are fine in principle for me. One nit/suggestion: >>>>>>>>>> >>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>> >>>>>>>>>> ??544?? jthread _java_thread; >>>>>>>>>> >>>>>>>>>> elsewhere "java_thread" refers to a JavaThread, so to avoid confusion may I suggest this member be named _jthread. >>>>>>>>> >>>>>>>>> I uploaded new webrev: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ >>>>>>>>> ? Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/ca6263dbdc87 >>>>>>>>> >>>>>>>>> >>>>>>>>>> I'm going to be away for the next couple of days - sorry - but will try to check email on this if I can. >>>>>>>>> >>>>>>>>> Thanks! >>>>>>>>> >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> David >>>>>>>>>> ----- >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2020/07/07 15:13, David Holmes wrote: >>>>>>>>>>>> On 7/07/2020 2:57 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>> Hi David, >>>>>>>>>>>>> >>>>>>>>>>>>> On 2020/07/07 11:31, David Holmes wrote: >>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hard to keep up with the changes - especially without incremental webrevs. >>>>>>>>>>>>> >>>>>>>>>>>>> Sorry, I will upload diff from previous webrev in the next. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> If GetSingleStackTraceClosure also took the jthread as a constructor arg, then you wouldn't need to recreate a JNI local handle when calling _collector.fill_frames. It's a small simplification and not essential at this stage. >>>>>>>>>>>>> >>>>>>>>>>>>> I think we should get jthread from an argument of do_thread() because do_thread() would pass the thread which are stopped certainly. >>>>>>>>>>>>> It might be simplification if we pass _calling_thread to MultipleStackTracesCollector. `jthread` is only needed to store jvmtiStackInfo.thread . What do you think? >>>>>>>>>>>> >>>>>>>>>>>> I'm not quite sure what you mean. >>>>>>>>>>>> >>>>>>>>>>>> I think there is a bit of a design wart with direct handshakes in that do_thread takes the target JavaThread as an argument. That's useful in a case where you want a HandshakeClosure that can be applied to multiple threads, but that's not typically what is needed with direct handshakes - there is only a single target. With a single-target HandshakeClosure you can capture all the "target" information for the operation in the closure instance. So if the actual do_thread operation wants the jthread corresponding to the target thread then we can store that in the closure rather than recomputing it (you could assert it is the same but that seems overkill to me). >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> For the test ... I don't see how Java_GetThreadListStackTraces_checkCallStacks is a valid test. It gets the stacks of all live threads, then uses that information to use GetThreadListStackTraces to get the stack for the same set of threads through a different API. It then compares the two sets of stacks for each thread expecting them to be the same, but that need only be the case for the main thread. Other threads could potentially have a different stack (e.g. if this test is run with JFR enabled there will be additional threads found.) Further I would have expected that there already exist tests that check that, for a given thread (which may be suspended or known to be blocked) the same stack is found through the two different APIs. >>>>>>>>>>>>> >>>>>>>>>>>>> vmTestbase/nsk/jvmti/unit/GetAllStackTraces/getallstktr001 would check all of threads via GetThreadListStackTraces() and GetAllStackTraces(), so we might be able to remove GetThreadListStackTraces.java from this webrev. >>>>>>>>>>>> >>>>>>>>>>>> Yes. The existing test only examines a set of test threads that are all blocked on a raw monitor. You do not need to duplicate that test. >>>>>>>>>>>> >>>>>>>>>>>>> OTOH we don't have testcase for GetThreadListStackTraces() with thread_count == 1, so we need to testcase for it (it is OneGetThreadListStackTraces.java) It would check whether the state of target thread is "waiting" before JNI call to call GetThreadListStackTraces(), >>>>>>>>>>>> >>>>>>>>>>>> Yes we need to test the special cases introduced by your changes - totally agree - and OneGetThreadListStackTraces.java is a good test for that. >>>>>>>>>>>> >>>>>>>>>>>>> and also I expect it would not be run with JFR. (it is not described @run) >>>>>>>>>>>> >>>>>>>>>>>> The arguments to run with JFR (or a bunch of other things) can be passed to the jtreg test harness to be applied to all tests. >>>>>>>>>>>> >>>>>>>>>>>>> Of course we can check GetThreadListStackTraces() with main thread, but it is not the test for direct handshake for other thread. >>>>>>>>>>>> >>>>>>>>>>>> Right - that test already exists as per the above. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> David >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> David >>>>>>>>>>>>>> ----- >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 6/07/2020 11:29 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I think C++ is more simple to implement the test agent as you said. >>>>>>>>>>>>>>> So I implement it in C++ in new webrev. Could you review again? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.06/ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Also I refactored libGetThreadListStackTraces.cpp, and I've kept exception check after IsSameObject(). >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 2020/07/06 16:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thank you for the update. >>>>>>>>>>>>>>>> I think, a pending exception after IsSameObject needs to be checked. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> The checkStackInfo() needs one more refactoring as I've already suggested. >>>>>>>>>>>>>>>> The body of the loop at L68-L78 should be converted to a function check_frame_info. >>>>>>>>>>>>>>>> The si1->frame_buffer[i] and si2->frame_buffer[i] will be passed as fi1 and fi2. >>>>>>>>>>>>>>>> The index can be passed as well. >>>>>>>>>>>>>>>> I'm still suggesting to simplify the local exception_msg to something shorter like err_msg or exc_msg. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I'm not sure using fatal is right here: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> This fragment looks strange: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> ? 152???? if ((*env)->IsSameObject(env, stack_info[i].thread, thread)) { >>>>>>>>>>>>>>>> ? 153?????? target_info = &stack_info[i]; >>>>>>>>>>>>>>>> ? 154?????? break; >>>>>>>>>>>>>>>> ? 155???? } else if ((*env)->ExceptionOccurred(env)) { >>>>>>>>>>>>>>>> ? 156 (*env)->ExceptionDescribe(env); >>>>>>>>>>>>>>>> ? 157?????? (*env)->FatalError(env, __FILE__); >>>>>>>>>>>>>>>> ? 158???? } >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I expected it to be: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> ??? jboolean same = (*env)->IsSameObject(env, stack_info[i].thread, thread); >>>>>>>>>>>>>>>> ??? if ((*env)->ExceptionOccurred(env)) { >>>>>>>>>>>>>>>> (*env)->ExceptionDescribe(env); >>>>>>>>>>>>>>>> ????? (*env)->FatalError(env, __FILE__); >>>>>>>>>>>>>>>> ??? } >>>>>>>>>>>>>>>> ??? if (same) { >>>>>>>>>>>>>>>> ????? target_info = &stack_info[i]; >>>>>>>>>>>>>>>> ????? break; >>>>>>>>>>>>>>>> ??? } >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Would it better to port this agent to C++ to simplify this code nicer? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 7/5/20 06:13, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>> I refactored testcase. Could you review again? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.05/ >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> It would check Java exception after IsSameObject() call. Does it need? >>>>>>>>>>>>>>>>> Any exceptions are not described in JNI document[1], and JNI implementation (jni_IsSameObject()) does not seem to throw it. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> [1] https://docs.oracle.com/en/java/javase/14/docs/specs/jni/functions.html#issameobject >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 2020/07/05 14:46, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Okay, thanks. >>>>>>>>>>>>>>>>>> Then I'm okay to keep the GetSingleStackTraceClosure. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c.html >>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c.html >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I'm not sure the function 'is_same_thread() is needed. >>>>>>>>>>>>>>>>>> Why do not use the JNI IsSameObject instead? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> It seems to be a typo at L132 and L137. >>>>>>>>>>>>>>>>>> You, probably. did not want to print the same information for stack_info_1[i].frame_buffer[j].XXX twice. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> The code at lines 112-142 is not readable. >>>>>>>>>>>>>>>>>> I'd suggest to make a couple of refactoring steps. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> First step to simplify this a little bit would be with some renaming and getting rid of indexes: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> ?? 71 char err_msg[EXCEPTION_MSG_LEN] = {0}; >>>>>>>>>>>>>>>>>> ??... >>>>>>>>>>>>>>>>>> ??112?? /* Iterate all jvmtiStackInfo to check */ >>>>>>>>>>>>>>>>>> ??113?? for (i = 0; i < num_threads, *exception_msg != '\0'; i++) { >>>>>>>>>>>>>>>>>> ????????? jvmtiStackInfo *si1 = stack_info_1[i]; >>>>>>>>>>>>>>>>>> ????????? jvmtiStackInfo *si2 = stack_info_2[i]; >>>>>>>>>>>>>>>>>> ??114???? if (!IsSameObject(env, si1.thread, si2.thread)) { /* jvmtiStackInfo::thread */ >>>>>>>>>>>>>>>>>> ??115?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>> ??116??????????????? "thread[%d] is different: stack_info_1 = %p, stack_info_2 = %p", >>>>>>>>>>>>>>>>>> ??117??????????????? i, sinfo1.thread, sinfo2.thread); >>>>>>>>>>>>>>>>>> ??118???? } else if (si1.state != si2.state) { /* jvmtiStackInfo::state */ >>>>>>>>>>>>>>>>>> ??119?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>> ??120??????????????? "state[%d] is different: stack_info_1 = %d, stack_info_2 = %d", >>>>>>>>>>>>>>>>>> ??121??????????????? i, si1.state, si2.state); >>>>>>>>>>>>>>>>>> ??122???? } else if (si1.frame_count != si2.frame_count) { /* jvmtiStackInfo::frame_count */ >>>>>>>>>>>>>>>>>> ??123?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>> ??124 "frame_count[%d] is different: stack_info_1 = %d, stack_info_2 = %d", >>>>>>>>>>>>>>>>>> ??125??????????????? i, si1.frame_count, si2.frame_count); >>>>>>>>>>>>>>>>>> ??126???? } else { >>>>>>>>>>>>>>>>>> ??127?????? /* Iterate all jvmtiFrameInfo to check */ >>>>>>>>>>>>>>>>>> ??128?????? for (j = 0; j < si1.frame_count; j++) { >>>>>>>>>>>>>>>>>> ??129???????? if (si1.frame_buffer[j].method != si1.frame_buffer[j].method) { /* jvmtiFrameInfo::method */ >>>>>>>>>>>>>>>>>> ??130?????????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>> ??131??????????????????? "thread [%d] frame_buffer[%d].method is different: stack_info_1 = %lx, stack_info_2 = %lx", >>>>>>>>>>>>>>>>>> ??132??????????????????? i, j, si1.frame_buffer[j].method, si2.frame_buffer[j].method); >>>>>>>>>>>>>>>>>> ??133?????????? break; >>>>>>>>>>>>>>>>>> ??134???????? } else if (si1.frame_buffer[j].location != si1.frame_buffer[j].location) { /* jvmtiFrameInfo::location */ >>>>>>>>>>>>>>>>>> ??135?????????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>> ??136??????????????????? "thread [%d] frame_buffer[%d].location is different: stack_info_1 = %ld, stack_info_2 = %ld", >>>>>>>>>>>>>>>>>> ??137??????????????????? i, j, si1.frame_buffer[j].location, si2.frame_buffer[j].location); >>>>>>>>>>>>>>>>>> ??138?????????? break; >>>>>>>>>>>>>>>>>> ??139???????? } >>>>>>>>>>>>>>>>>> ??140?????? } >>>>>>>>>>>>>>>>>> ??141???? } >>>>>>>>>>>>>>>>>> ??142?? } >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Another step would be to create functions that implement a body of each loop. >>>>>>>>>>>>>>>>>> You can use the same techniques to simplify similar place (L127-L138) in the libOneGetThreadListStackTraces.c. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 7/3/20 15:55, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I'm not an Oracle employee, so I cannot know real request(s) from your customers. >>>>>>>>>>>>>>>>>>> However JDK-8201641 says Dynatrace has requested this enhancement. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> BTW I haven't heared any request from my customers about this. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On 2020/07/04 4:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> This difference is not that big to care about. >>>>>>>>>>>>>>>>>>>> I feel this is really rare case and so, does not worth these complications. >>>>>>>>>>>>>>>>>>>> Do we have a real request from customers to optimize it? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On 7/3/20 01:16, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Generally I agree with you, but I have concern about the difference of the result of GetStackTrace() and GetThreadListStackTraces(). >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ? GetStackTrace: jvmtiFrameInfo >>>>>>>>>>>>>>>>>>>>> ? GetThreadListStackTraces: jvmtiStackInfo >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> jvmtiStackInfo contains thread state, and it is ensured it is the state of the call stack. >>>>>>>>>>>>>>>>>>>>> If we want to get both call stack and thread state, we need to suspend target thread, and call both GetStackTrace() and GetThreadState(). Is it ok? >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> I was wondering if JDK-8201641 (parent ticket of this change) needed them for profiling (dynatrace?) >>>>>>>>>>>>>>>>>>>>> If it is responsibility of JVMTI agent implementor, I remove this closure. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On 2020/07/03 16:45, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> After some thinking I've concluded that I do not like this optimization >>>>>>>>>>>>>>>>>>>>>> of the GetThreadListStackTraces with GetSingleStackTraceClosure. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> We may need more opinions on this but these are my points: >>>>>>>>>>>>>>>>>>>>>> ??- it adds some complexity and ugliness >>>>>>>>>>>>>>>>>>>>>> ??- a win is doubtful because it has to be a rare case, so that total overhead should not be high >>>>>>>>>>>>>>>>>>>>>> ??- if it is really high for some use cases then it is up to the user >>>>>>>>>>>>>>>>>>>>>> ??? to optimize it with using GetStackTrace instead >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> In such cases with doubtful overhead I usually prefer the simplicity. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Good examples where it makes sense to optimize are checks for target thread to be current thread. >>>>>>>>>>>>>>>>>>>>>> In such cases there is no need to suspend the target thread, or use a VMop/HandshakeClosure. >>>>>>>>>>>>>>>>>>>>>> For instance, please, see the Monitor functions with the check: (java_thread == calling_thread). >>>>>>>>>>>>>>>>>>>>>> Getting information for current thread is frequently used case, e.g. to get info at an event point. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> On 7/2/20 23:29, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>> Hi Dan, David, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could you review again? >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/ >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> OneGetThreadListStackTraces.java in this webrev would wait until thread state is transited to "waiting" with spin wait. >>>>>>>>>>>>>>>>>>>>>>> CountDownLatch::await call as Dan pointed is fixed in it :) >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Diff from webrev.03: >>>>>>>>>>>>>>>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/c9aeb7001e50 >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 14:15, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 12:24, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> On 7/2/20 10:50 PM, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry I'm responding here without seeing latest webrev but there is enough context I think ... >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Dan, >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I upload new webrev. Could you review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L1542: // Get stack trace with handshake >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - please add a period at the end. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> L1591: *stack_info_ptr = op.stack_info(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? The return parameter should not be touched unless the return >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? old L1582: if (err == JVMTI_ERROR_NONE) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please restore this check. The return parameter should not >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? be touched unless the return code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> But op.stack_info() will return NULL if the error is not JVMTI_ERROR_NONE. Are you (Dan) concerned about someone passing in a non-null/initialized out-pointer that will be reset to NULL if there was an error? >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Actually the way we used to test this in POSIX tests is to call >>>>>>>>>>>>>>>>>>>>>>>>>> an API with known bad parameters and the return parameter ptr >>>>>>>>>>>>>>>>>>>>>>>>>> set to NULL. If the return parameter ptr was touched when an >>>>>>>>>>>>>>>>>>>>>>>>>> error should have been detected on an earlier parameter, then >>>>>>>>>>>>>>>>>>>>>>>>>> the test failed. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> L1272: ? if (!jt->is_exiting() && (thread_oop != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? old L1532: _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? This deletion of the _result field threw me for a minute and then >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? ? ? I figured out that the field is init to JVMTI_ERROR_THREAD_NOT_ALIVE >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? in the constructor. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L1553: if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L64: startSignal.countDown(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? I was expecting this to be a call to await() instead of >>>>>>>>>>>>>>>>>>>>>>>>>>>>> countDown(). What am I missing here? >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? I think this test might be passing by accident right now, but... >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Main thread (which call JVMTI functions to test) should wait until test thread is ready. >>>>>>>>>>>>>>>>>>>>>>>>>>>> So main thread would wait startSignal, and test thread would count down. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> No! >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> The test thread that previously called obj.wait() now calls latch.await(). >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> The main thread that previously called obj.notify() now calls latch.countDown(). >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> The main thread continues to spin until it sees the target is WAITING before proceeding with the test. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> If I add spin wait to wait until transit target thread state is WAITING (as following), we don't need to call SuspendThread(). >>>>>>>>>>>>>>>>>>>>>>>>> Which is better? >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> The original spin-wait loop checking for WAITING is better because it is the only guarantee that the target thread is blocked where you need it to be. suspending the thread is racy as you don't know exactly where the suspend will hit. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>> /* Wait until the thread state transits to "waiting" */ >>>>>>>>>>>>>>>>>>>>>>>>> while (th.getState() != Thread.State.WAITING) { >>>>>>>>>>>>>>>>>>>>>>>>> Thread.onSpinWait(); >>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> For simplify, spin wait is prefer to OneGetThreadListStackTraces.java in webrev.03. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Here's the flow as I see it: >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>>>>>> ?? - start worker thread >>>>>>>>>>>>>>>>>>>>>>>>>> ?? - startSignal.await() >>>>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>>>>>>>>>> ?? - startSignal.countDown() >>>>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>>>>>>>>>> ?? - stopSignal.await() >>>>>>>>>>>>>>>>>>>>>>>>>> ???? - worker is now blocked >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>>>>>> ?? - checkCallStacks(th) >>>>>>>>>>>>>>>>>>>>>>>>>> ?? - stopSignal.countDown() >>>>>>>>>>>>>>>>>>>>>>>>>> ???? - worker is now unblocked >>>>>>>>>>>>>>>>>>>>>>>>>> ?? - th.join >>>>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>>>>>>>>>> ?? - runs off the end of run() >>>>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>>>>>> ?? - run off the end of main() >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L92: jthreads = (jthread *)malloc(sizeof(jthread) * num_threads); >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? You don't check for malloc() failure. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'jthreads' is allocated but never freed. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L91: result = (*jvmti)->SuspendThread(jvmti, thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? Why are you suspending the thread? GetAllStackTraces() and >>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() do not require the target thread(s) >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? to be suspend. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? If you decide not to SuspendThread, then you don't need the >>>>>>>>>>>>>>>>>>>>>>>>>>>>> AddCapabilities or the ResumeThread calls. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Test thread might not be entered following code (stopSignal.await()). We might see deferent call stack between GetAllStackTraces() and GetThreadListStackTraces(). We cannot control to freeze call stack of test thread in Java code. >>>>>>>>>>>>>>>>>>>>>>>>>>>> (I didn't use SuspendThread() at first, but I saw some errors which causes in above.) >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> So we need to call SuspendThread() to ensure we can see same call stack. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> If you are checking that the thread is in state WAITING then it cannot escape from that state and you can sample the stack multiple times from any API and get the same result. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> I suspect the errors you saw were from the apparent incorrect use of the CountDownLatch. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> With the flow outlined above, the worker thread should be >>>>>>>>>>>>>>>>>>>>>>>>>> nicely blocked in stopSignal.await() when stuff is sampled. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Updates look fine - thanks. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> One minor nit: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1274 _collector.allocate_and_fill_stacks(1); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1275 _collector.set_result(JVMTI_ERROR_NONE); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the other places where you use _collector you rely on result being initialized to JVMTI_ERROR_NONE, rather than setting it directly after allocate_and_fill_stacks(). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 823 "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This function (JvmtiEnvBase::get_stack_trace()) can be called to get own stack trace. For example, we can call GetStackTrace() for current thread at JVMTI event. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So I changed assert as below: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??820 assert(current_thread == java_thread || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??821 SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??823 "call by myself / at safepoint / at handshake"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yep good catch. I hope current tests caught that. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> They would be tested in vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ (own call stacks), and getstacktr003 (call stacks in other thread). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Speaking of tests ... >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the native code I think you need to check the success of all JNI methods that can throw exceptions - otherwise I believe the tests may trigger warnings if -Xcheck:jni is used with them. See for example: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I updated testcases to check JNI and JVMTI function calls. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the Java code the target thread: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 45 public void run() { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 46 try { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 47 synchronized (lock) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 48 lock.wait(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 49 System.out.println("OK"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 50?????????? } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is potentially susceptible to spurious wakeups. Using a CountDownLatch would be robust. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If so, we can call make_local() in L1272 without JavaThread (or we can pass current thread to make_local()). Is it right? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry I got confused, _calling_thread may not be the current thread as we could be executing the handshake in the target thread itself. So the ResourceMark is correct as-is (implicitly for current thread). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The argument to fill_frames will be used in the jvmtiStackInfo and passed back to the _calling_thread, so it must be created via make_local(_calling_thread, ...) as you presently have. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you for reviewing! I will update new webrev tomorrow. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 500 JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??? : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I updated webrev for 8242428. Could you review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change migrate to use direct handshake for GetStackTrace() and GetThreadListStackTraces() (when thread_count == 1). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks really good now! I only have a few nits below. There is one thing I don't like about it but it requires a change to the main Handshake logic to address - in JvmtiEnv::GetThreadListStackTraces you have to create a ThreadsListHandle to convert the jthread to a JavaThread, but then the Handshake::execute_direct creates another ThreadsListHandle internally. That's a waste. I will discuss with Robbin and file a RFE to have an overload of execute_direct that takes an existing TLH. Actually it's worse than that because we have another TLH in use at the entry point for the JVMTI functions, so I think there may be some scope for simplifying the use of TLH instances - future RFE. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??451 GetStackTraceClosure(JvmtiEnv *env, jint start_depth, jint max_count, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??452 jvmtiFrameInfo* frame_buffer, jint* count_ptr) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??454 _env(env), _start_depth(start_depth), _max_count(max_count), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??455 _frame_buffer(frame_buffer), _count_ptr(count_ptr), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??456 _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Nit: can you do one initializer per line please. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??500 JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 481 MultipleStackTracesCollector(JvmtiEnv *env, jint max_frame_count) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??483 _max_frame_count = max_frame_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??484 _frame_count_total = 0; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??485 _head = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??486 _stack_info = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??487 _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As you are touching this can you change it to use an initializer list as you did for the HandshakeClosure, and please keep one item per line. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??823 "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1269 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is frustrating that this entire call chain started with a jthread reference, which we converted to a JavaThread, only to eventually need to convert it back to a jthread! I think there is some scope for simplification here but not as part of this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm still pondering the test logic but wanted to send this now. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetThreadListStackTrace (for GetThreadListStackTraces) and VM_GetAllStackTraces (for GetAllStackTraces) have inherited VM_GetMultipleStackTraces VM operation which provides the feature to generate jvmtiStackInfo. I modified VM_GetMultipleStackTraces to a normal C++ class to share with HandshakeClosure for GetThreadListStackTraces (GetSingleStackTraceClosure). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I added new testcases which test GetThreadListStackTraces() with thread_count == 1 and with all threads. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/nsk/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change replace following VM operations to direct handshake. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces (GetThreadListStackTrace()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTrace() uses direct handshake if thread count == 1. In other case (thread count > 1), it would be performed as VM operation (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Caller of VM_GetCurrentLocation (JvmtiEnvThreadState::reset_current_location()) might be called at safepoint. So I added safepoint check in its caller. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/ns >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I tested it on submit repo, then it has execution error (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) due to dependency error. So I think it does not occur by this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>> >>>> >>> > From shade at redhat.com Fri Jul 10 08:37:51 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 10 Jul 2020 10:37:51 +0200 Subject: RFR (S) 8249137: Remove CollectedHeap::obj_size In-Reply-To: <70984ebf-2717-c7b3-7076-12e2c8c7515c@oracle.com> References: <70984ebf-2717-c7b3-7076-12e2c8c7515c@oracle.com> Message-ID: Okay, thanks. I already have 2 reviewers (rkennke, tschatzl), do I need more specifically from serviceability-dev@? -Aleksey On 7/9/20 3:34 PM, Daniel D. Daugherty wrote: > Adding serviceability-dev at ... since a couple of JVM/TI files are changed > in this RFR. Also, I moved the bug from hotspot/runtime -> hotspot/gc. > > Dan > > > On 7/9/20 2:36 AM, Aleksey Shipilev wrote: >> RFE: >> https://bugs.openjdk.java.net/browse/JDK-8249137 >> >> It was added by JDK-8211270 to support old-style Shenandoah that needed a separate fwdptr slot. >> After JDK-8224584 it does not need this anymore. Additionally, CH::obj_size may disagree with other >> code that pokes at layout helper directly, for example GraphKit::new_instance. >> >> This also avoids a virtual call on some paths, although those paths are not very performance-sensitive. >> >> The patch is a simple series of few-liners: >> >> diff -r 9cc348ebdc82 src/hotspot/share/gc/shared/collectedHeap.cpp >> --- a/src/hotspot/share/gc/shared/collectedHeap.cpp Thu Jul 09 04:32:30 2020 +0200 >> +++ b/src/hotspot/share/gc/shared/collectedHeap.cpp Thu Jul 09 08:05:46 2020 +0200 >> @@ -578,6 +578,2 @@ >> >> -size_t CollectedHeap::obj_size(oop obj) const { >> - return obj->size(); >> -} >> - >> uint32_t CollectedHeap::hash_oop(oop obj) const { >> diff -r 9cc348ebdc82 src/hotspot/share/gc/shared/collectedHeap.hpp >> --- a/src/hotspot/share/gc/shared/collectedHeap.hpp Thu Jul 09 04:32:30 2020 +0200 >> +++ b/src/hotspot/share/gc/shared/collectedHeap.hpp Thu Jul 09 08:05:46 2020 +0200 >> @@ -495,4 +495,2 @@ >> >> - virtual size_t obj_size(oop obj) const; >> - >> // Non product verification and debugging. >> diff -r 9cc348ebdc82 src/hotspot/share/prims/jvmtiEnv.cpp >> --- a/src/hotspot/share/prims/jvmtiEnv.cpp Thu Jul 09 04:32:30 2020 +0200 >> +++ b/src/hotspot/share/prims/jvmtiEnv.cpp Thu Jul 09 08:05:46 2020 +0200 >> @@ -488,3 +488,3 @@ >> NULL_CHECK(mirror, JVMTI_ERROR_INVALID_OBJECT); >> - *size_ptr = (jlong)Universe::heap()->obj_size(mirror) * wordSize; >> + *size_ptr = (jlong)mirror->size() * wordSize; >> return JVMTI_ERROR_NONE; >> diff -r 9cc348ebdc82 src/hotspot/share/prims/jvmtiExport.cpp >> --- a/src/hotspot/share/prims/jvmtiExport.cpp Thu Jul 09 04:32:30 2020 +0200 >> +++ b/src/hotspot/share/prims/jvmtiExport.cpp Thu Jul 09 08:05:46 2020 +0200 >> @@ -1067,3 +1067,3 @@ >> _jobj = (jobject)to_jobject(obj); >> - _size = Universe::heap()->obj_size(obj) * wordSize; >> + _size = obj->size() * wordSize; >> }; >> diff -r 9cc348ebdc82 src/hotspot/share/prims/whitebox.cpp >> --- a/src/hotspot/share/prims/whitebox.cpp Thu Jul 09 04:32:30 2020 +0200 >> +++ b/src/hotspot/share/prims/whitebox.cpp Thu Jul 09 08:05:46 2020 +0200 >> @@ -389,3 +389,3 @@ >> oop p = JNIHandles::resolve(obj); >> - return Universe::heap()->obj_size(p) * HeapWordSize; >> + return p->size() * HeapWordSize; >> WB_END >> >> Testing: tier{1,2}; jdk-submit (running) >> > -- Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From daniel.daugherty at oracle.com Fri Jul 10 15:07:01 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 10 Jul 2020 11:07:01 -0400 Subject: RFR (S) 8249137: Remove CollectedHeap::obj_size In-Reply-To: References: <70984ebf-2717-c7b3-7076-12e2c8c7515c@oracle.com> Message-ID: <99b679ec-fcae-6f30-1186-1cf8de809c6f@oracle.com> On 7/10/20 4:37 AM, Aleksey Shipilev wrote: > Okay, thanks. > > I already have 2 reviewers (rkennke, tschatzl), do I need more specifically from serviceability-dev@? Since you're touching the Serviceability team's code, it would be polite to wait for a review... Dan > > -Aleksey > > On 7/9/20 3:34 PM, Daniel D. Daugherty wrote: >> Adding serviceability-dev at ... since a couple of JVM/TI files are changed >> in this RFR. Also, I moved the bug from hotspot/runtime -> hotspot/gc. >> >> Dan >> >> >> On 7/9/20 2:36 AM, Aleksey Shipilev wrote: >>> RFE: >>> https://bugs.openjdk.java.net/browse/JDK-8249137 >>> >>> It was added by JDK-8211270 to support old-style Shenandoah that needed a separate fwdptr slot. >>> After JDK-8224584 it does not need this anymore. Additionally, CH::obj_size may disagree with other >>> code that pokes at layout helper directly, for example GraphKit::new_instance. >>> >>> This also avoids a virtual call on some paths, although those paths are not very performance-sensitive. >>> >>> The patch is a simple series of few-liners: >>> >>> diff -r 9cc348ebdc82 src/hotspot/share/gc/shared/collectedHeap.cpp >>> --- a/src/hotspot/share/gc/shared/collectedHeap.cpp Thu Jul 09 04:32:30 2020 +0200 >>> +++ b/src/hotspot/share/gc/shared/collectedHeap.cpp Thu Jul 09 08:05:46 2020 +0200 >>> @@ -578,6 +578,2 @@ >>> >>> -size_t CollectedHeap::obj_size(oop obj) const { >>> - return obj->size(); >>> -} >>> - >>> uint32_t CollectedHeap::hash_oop(oop obj) const { >>> diff -r 9cc348ebdc82 src/hotspot/share/gc/shared/collectedHeap.hpp >>> --- a/src/hotspot/share/gc/shared/collectedHeap.hpp Thu Jul 09 04:32:30 2020 +0200 >>> +++ b/src/hotspot/share/gc/shared/collectedHeap.hpp Thu Jul 09 08:05:46 2020 +0200 >>> @@ -495,4 +495,2 @@ >>> >>> - virtual size_t obj_size(oop obj) const; >>> - >>> // Non product verification and debugging. >>> diff -r 9cc348ebdc82 src/hotspot/share/prims/jvmtiEnv.cpp >>> --- a/src/hotspot/share/prims/jvmtiEnv.cpp Thu Jul 09 04:32:30 2020 +0200 >>> +++ b/src/hotspot/share/prims/jvmtiEnv.cpp Thu Jul 09 08:05:46 2020 +0200 >>> @@ -488,3 +488,3 @@ >>> NULL_CHECK(mirror, JVMTI_ERROR_INVALID_OBJECT); >>> - *size_ptr = (jlong)Universe::heap()->obj_size(mirror) * wordSize; >>> + *size_ptr = (jlong)mirror->size() * wordSize; >>> return JVMTI_ERROR_NONE; >>> diff -r 9cc348ebdc82 src/hotspot/share/prims/jvmtiExport.cpp >>> --- a/src/hotspot/share/prims/jvmtiExport.cpp Thu Jul 09 04:32:30 2020 +0200 >>> +++ b/src/hotspot/share/prims/jvmtiExport.cpp Thu Jul 09 08:05:46 2020 +0200 >>> @@ -1067,3 +1067,3 @@ >>> _jobj = (jobject)to_jobject(obj); >>> - _size = Universe::heap()->obj_size(obj) * wordSize; >>> + _size = obj->size() * wordSize; >>> }; >>> diff -r 9cc348ebdc82 src/hotspot/share/prims/whitebox.cpp >>> --- a/src/hotspot/share/prims/whitebox.cpp Thu Jul 09 04:32:30 2020 +0200 >>> +++ b/src/hotspot/share/prims/whitebox.cpp Thu Jul 09 08:05:46 2020 +0200 >>> @@ -389,3 +389,3 @@ >>> oop p = JNIHandles::resolve(obj); >>> - return Universe::heap()->obj_size(p) * HeapWordSize; >>> + return p->size() * HeapWordSize; >>> WB_END >>> >>> Testing: tier{1,2}; jdk-submit (running) >>> > From daniel.daugherty at oracle.com Fri Jul 10 22:00:52 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 10 Jul 2020 18:00:52 -0400 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: <4881e0da-f716-c62e-b744-17aa3d03e095@oss.nttdata.com> References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <8c387550-a06e-6320-f453-0e898486befb@oss.nttdata.com> <19cfa2a7-f441-425d-5623-1c11c6a870b6@oss.nttdata.com> <237e8d26-d185-1008-6e07-5b9f36314534@oracle.com> <2a9ba1f8-b1f7-cb8c-fe11-56dd174d9d58@oracle.com> <7ad907cb-24ef-aa55-972a-0a3f63576568@oracle.com> <5ab8c5d9-9a5e-74a6-4605-7e73732037bf@oracle.com> <4881e0da-f716-c62e-b744-17aa3d03e095@oss.nttdata.com> Message-ID: On 7/10/20 3:29 AM, Yasumasa Suenaga wrote: > Thanks Patricio! > > I uploaded new webrev. Could you review again? > It passed all tests on submit repo, and passed jtreg tests about JVMTI. > > ? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.10/ > ? Diff from webrev.09: > http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.10/incremental/ test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp ??? No comments. Thumbs up on the incremental. Dan > > > Yasumasa > > > On 2020/07/10 14:49, Patricio Chilano wrote: >> Hi Yasumasa, >> >> On 7/10/20 1:58 AM, Yasumasa Suenaga wrote: >>> Hi Patricio, >>> >>> Thanks for your advice! >>> I've fixed testcase as [1], but I still see an error in >>> validate-headers-linux-x64-build-1 on submit repo. >>> What does it mean? Can you share details? >>> >>> ? mach5-one-ysuenaga-JDK-8242428-20200710-0339-12529134 >> Not your bug, you are getting the header validation failure from >> 8248570. It was fixed two days ago so just update the repo and it >> should work now. >> >> Thanks, >> Patricio >>> Of course this change can be built on my Linux box (Fedora 32, >>> AMD64, GCC 10.1) >>> >>> >>> Yasumasa >>> >>> >>> [1] >>> http://hg.openjdk.java.net/jdk/submit/file/45f52142db9d/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>> >>> >>> On 2020/07/10 2:18, Patricio Chilano wrote: >>>> >>>> On 7/9/20 12:00 PM, Patricio Chilano wrote: >>>>> Hi Yasumasa, >>>>> >>>>> On 7/9/20 9:30 AM, Yasumasa Suenaga wrote: >>>>>> On 2020/07/09 17:58, David Holmes wrote: >>>>>>> Hi Yasumasa, >>>>>>> >>>>>>> On 9/07/2020 10:25 am, Yasumasa Suenaga wrote: >>>>>>>> Hi Dan, >>>>>>>> >>>>>>>> Thanks for your comment! >>>>>>>> I uploaded new webrev: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.09/ >>>>>>>> ?? Diff from previous webrev: >>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/5d167adf8524 >>>>>>>> >>>>>>>> I saw similar build errors in >>>>>>>> libOneGetThreadListStackTraces.cpp on Windows. >>>>>>>> This webrev fixes them. >>>>>>> >>>>>>> You shouldn't use %p as it may not be portable. In the VM we use >>>>>>> INTPTR_FORMAT and convert the arg using p2i. I don't know what >>>>>>> exists in the testing code. >>>>>> >>>>>> I replaced %p to %lx, and also cast values to unsigned long [1] >>>>>> [2], but the test on submit repo was failed. >>>>>> Can anyone share details of >>>>>> mach5-one-ysuenaga-JDK-8242428-20200709-1030-12500928 ? >>>>> These are the errors I see for the macOS build: >>>>> >>>>> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:53:14: >>>>> error: format specifies type 'long long' but the argument has type >>>>> 'jlocation' (aka 'long') [-Werror,-Wformat] >>>>> ????????????? fi1->location, fi2->location); >>>>> ????????????? ^~~~~~~~~~~~~ >>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:53:29: >>>>> error: format specifies type 'long long' but the argument has type >>>>> 'jlocation' (aka 'long') [-Werror,-Wformat] >>>>> ????????????? fi1->location, fi2->location); >>>>> ???????????????????????????? ^~~~~~~~~~~~~ >>>>> >>>>> These are the ones I see for the Windows build: >>>>> >>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): >>>>> warning C4311: 'type cast': pointer truncation from 'jmethodID' to >>>>> 'unsigned long' >>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): >>>>> warning C4302: 'type cast': truncation from 'jmethodID' to >>>>> 'unsigned long' >>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): >>>>> warning C4311: 'type cast': pointer truncation from 'jmethodID' to >>>>> 'unsigned long' >>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): >>>>> warning C4302: 'type cast': truncation from 'jmethodID' to >>>>> 'unsigned long' >>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): >>>>> warning C4311: 'type cast': pointer truncation from 'jthread' to >>>>> 'unsigned long' >>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): >>>>> warning C4302: 'type cast': truncation from 'jthread' to 'unsigned >>>>> long' >>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): >>>>> warning C4311: 'type cast': pointer truncation from 'jthread' to >>>>> 'unsigned long' >>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): >>>>> warning C4302: 'type cast': truncation from 'jthread' to 'unsigned >>>>> long' >>>>> >>>>> >>>>> You will probably want to use the macros defined in >>>>> src/hotspot/share/utilities/globalDefinitions.hpp. Let me know if >>>>> you need me to test something. >>>> With these changes the build works okay on Linux, Windows and macOS: >>>> >>>> >>>> --- >>>> a/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>> +++ >>>> b/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>> @@ -27,4 +27,5 @@ >>>> ??#include >>>> ??#include >>>> +#include >>>> >>>> ??#define MAX_FRAMES 100 >>>> @@ -45,11 +46,11 @@ >>>> ??? if (fi1->method != fi2->method) { /* jvmtiFrameInfo::method */ >>>> ????? snprintf(err_msg, sizeof(err_msg), >>>> -???????????? "method is different: fi1 = %p, fi2 = %p", >>>> -???????????? fi1->method, fi2->method); >>>> +???????????? "method is different: fi1 = 0x%016" PRIxPTR " , fi2 = >>>> 0x%016" PRIxPTR, >>>> +???????????? (intptr_t)fi1->method, (intptr_t)fi2->method); >>>> ????? env->FatalError(err_msg); >>>> ??? } else if (fi1->location != fi2->location) { /* >>>> jvmtiFrameInfo::location */ >>>> ????? snprintf(err_msg, sizeof(err_msg), >>>> -???????????? "location is different: fi1 = %lld, fi2 = %lld", >>>> -???????????? fi1->location, fi2->location); >>>> +???????????? "location is different: fi1 = %" PRId64 " , fi2 = %" >>>> PRId64, >>>> +???????????? (int64_t)fi1->location, (int64_t)fi2->location); >>>> ????? env->FatalError(err_msg); >>>> ??? } >>>> @@ -67,5 +68,5 @@ >>>> ??? if (!is_same) { /* jvmtiStackInfo::thread */ >>>> ????? snprintf(err_msg, sizeof(err_msg), >>>> -???????????? "thread is different: si1 = %p, si2 = %p", >>>> si1->thread, si2->thread); >>>> +???????????? "thread is different: si1 = 0x%016" PRIxPTR " , si2 = >>>> 0x%016" PRIxPTR, (intptr_t)si1->thread, (intptr_t)si2->thread); >>>> ????? env->FatalError(err_msg); >>>> ??? } else if (si1->state != si2->state) { /* jvmtiStackInfo::state */ >>>> >>>> >>>> Maybe you can use something like that. >>>> >>>> >>>> Thanks, >>>> Patricio >>>>> Thanks, >>>>> Patricio >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> [1] http://hg.openjdk.java.net/jdk/submit/rev/dfca51958217 >>>>>> [2] http://hg.openjdk.java.net/jdk/submit/rev/3665361fa91b >>>>>> >>>>>> >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2020/07/09 1:42, Daniel D. Daugherty wrote: >>>>>>>>> ?> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ >>>>>>>>> >>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>> ???? No comments. >>>>>>>>> >>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>> ???? L1159: ? Thread *current_thread = Thread::current(); >>>>>>>>> ???????? Please add "#ifdef ASSERT" above and "#endif" below >>>>>>>>> since >>>>>>>>> ???????? current_thread is only used for the assert() in this >>>>>>>>> function. >>>>>>>>> >>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>> ???? L549: ???????????????????????????? jthread java_thread, >>>>>>>>> jint max_frame_count) >>>>>>>>> ???? L552: ????? _jthread(java_thread), >>>>>>>>> ???????? Please: s/java_thread/thread/ on both lines. >>>>>>>>> >>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>> ???? No comments. >>>>>>>>> >>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>> >>>>>>>>> ???? No comments. >>>>>>>>> >>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>>>>> >>>>>>>>> ???? L27: #include >>>>>>>>> ???????? This include is out of order; should be first in the >>>>>>>>> list. >>>>>>>>> >>>>>>>>> ???? This file doesn't compile on my MBP13: >>>>>>>>> >>>>>>>>> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:49:14: >>>>>>>>> error: format specifies type 'unsigned long' but the argument >>>>>>>>> has type 'jmethodID' (aka '_jmethodID *') [-Werror,-Wformat] >>>>>>>>> ????????????? fi1->method, fi2->method); >>>>>>>>> ????????????? ^~~~~~~~~~~ >>>>>>>>> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:49:27: >>>>>>>>> error: format specifies type 'unsigned long' but the argument >>>>>>>>> has type 'jmethodID' (aka '_jmethodID *') [-Werror,-Wformat] >>>>>>>>> ????????????? fi1->method, fi2->method); >>>>>>>>> ?????????????????????????? ^~~~~~~~~~~ >>>>>>>>> ???? 2 errors generated. >>>>>>>>> >>>>>>>>> ???? This change made it compile on my MBP13, but that may >>>>>>>>> break it on >>>>>>>>> ???? other platforms: >>>>>>>>> >>>>>>>>> ???? $ hg diff >>>>>>>>> ???? diff -r 560847c69fbe >>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>>>>> ???? --- >>>>>>>>> a/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>>>>> Wed Jul 08 12:13:32 2020 -0400 >>>>>>>>> ???? +++ >>>>>>>>> b/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>>>>> Wed Jul 08 12:40:42 2020 -0400 >>>>>>>>> ???? @@ -46,7 +46,7 @@ >>>>>>>>> ?? ?? ? if (fi1->method != fi2->method) { /* >>>>>>>>> jvmtiFrameInfo::method */ >>>>>>>>> ?? ?? ??? snprintf(err_msg, sizeof(err_msg), >>>>>>>>> ?? ? ? ??????????? "method is different: fi1 = %lx, fi2 = %lx", >>>>>>>>> ???? -???????????? fi1->method, fi2->method); >>>>>>>>> ???? +???????????? (unsigned long) fi1->method, (unsigned >>>>>>>>> long) fi2->method); >>>>>>>>> ?? ?? ??? env->FatalError(err_msg); >>>>>>>>> ?? ?? ? } else if (fi1->location != fi2->location) { /* >>>>>>>>> jvmtiFrameInfo::location */ >>>>>>>>> ?? ?? ??? snprintf(err_msg, sizeof(err_msg), >>>>>>>>> >>>>>>>>> ???? I'm not sure of the right platform independent way to output >>>>>>>>> ???? the 'method' field. >>>>>>>>> >>>>>>>>> Dan >>>>>>>>> >>>>>>>>> >>>>>>>>> On 7/8/20 4:04 AM, Yasumasa Suenaga wrote: >>>>>>>>>> Hi David, >>>>>>>>>> >>>>>>>>>> On 2020/07/08 15:27, David Holmes wrote: >>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>> >>>>>>>>>>> On 7/07/2020 6:54 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>> >>>>>>>>>>>> Serguei, thank you for replying even though you are on >>>>>>>>>>>> vacaiton! >>>>>>>>>>>> >>>>>>>>>>>> I uploaded new webrev: >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.07/ >>>>>>>>>>>> ?? Diff from previous webrev: >>>>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/77243b1dcbfe >>>>>>>>>>>> >>>>>>>>>>>> c'tor of GetSingleStackTraceClosure has jthread argument in >>>>>>>>>>>> this webrev. >>>>>>>>>>>> Also it does not contain testcase for >>>>>>>>>>>> GetThreadListStackTraces with all threads, and >>>>>>>>>>>> OneGetThreadListStackTraces would test main thread only. >>>>>>>>>>> >>>>>>>>>>> All those changes are fine in principle for me. One >>>>>>>>>>> nit/suggestion: >>>>>>>>>>> >>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>> >>>>>>>>>>> ??544?? jthread _java_thread; >>>>>>>>>>> >>>>>>>>>>> elsewhere "java_thread" refers to a JavaThread, so to avoid >>>>>>>>>>> confusion may I suggest this member be named _jthread. >>>>>>>>>> >>>>>>>>>> I uploaded new webrev: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ >>>>>>>>>> ? Diff from previous webrev: >>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/ca6263dbdc87 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> I'm going to be away for the next couple of days - sorry - >>>>>>>>>>> but will try to check email on this if I can. >>>>>>>>>> >>>>>>>>>> Thanks! >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> David >>>>>>>>>>> ----- >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 2020/07/07 15:13, David Holmes wrote: >>>>>>>>>>>>> On 7/07/2020 2:57 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 2020/07/07 11:31, David Holmes wrote: >>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hard to keep up with the changes - especially without >>>>>>>>>>>>>>> incremental webrevs. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Sorry, I will upload diff from previous webrev in the next. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> If GetSingleStackTraceClosure also took the jthread as a >>>>>>>>>>>>>>> constructor arg, then you wouldn't need to recreate a >>>>>>>>>>>>>>> JNI local handle when calling _collector.fill_frames. >>>>>>>>>>>>>>> It's a small simplification and not essential at this >>>>>>>>>>>>>>> stage. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I think we should get jthread from an argument of >>>>>>>>>>>>>> do_thread() because do_thread() would pass the thread >>>>>>>>>>>>>> which are stopped certainly. >>>>>>>>>>>>>> It might be simplification if we pass _calling_thread to >>>>>>>>>>>>>> MultipleStackTracesCollector. `jthread` is only needed to >>>>>>>>>>>>>> store jvmtiStackInfo.thread . What do you think? >>>>>>>>>>>>> >>>>>>>>>>>>> I'm not quite sure what you mean. >>>>>>>>>>>>> >>>>>>>>>>>>> I think there is a bit of a design wart with direct >>>>>>>>>>>>> handshakes in that do_thread takes the target JavaThread >>>>>>>>>>>>> as an argument. That's useful in a case where you want a >>>>>>>>>>>>> HandshakeClosure that can be applied to multiple threads, >>>>>>>>>>>>> but that's not typically what is needed with direct >>>>>>>>>>>>> handshakes - there is only a single target. With a >>>>>>>>>>>>> single-target HandshakeClosure you can capture all the >>>>>>>>>>>>> "target" information for the operation in the closure >>>>>>>>>>>>> instance. So if the actual do_thread operation wants the >>>>>>>>>>>>> jthread corresponding to the target thread then we can >>>>>>>>>>>>> store that in the closure rather than recomputing it (you >>>>>>>>>>>>> could assert it is the same but that seems overkill to me). >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> For the test ... I don't see how >>>>>>>>>>>>>>> Java_GetThreadListStackTraces_checkCallStacks is a valid >>>>>>>>>>>>>>> test. It gets the stacks of all live threads, then uses >>>>>>>>>>>>>>> that information to use GetThreadListStackTraces to get >>>>>>>>>>>>>>> the stack for the same set of threads through a >>>>>>>>>>>>>>> different API. It then compares the two sets of stacks >>>>>>>>>>>>>>> for each thread expecting them to be the same, but that >>>>>>>>>>>>>>> need only be the case for the main thread. Other threads >>>>>>>>>>>>>>> could potentially have a different stack (e.g. if this >>>>>>>>>>>>>>> test is run with JFR enabled there will be additional >>>>>>>>>>>>>>> threads found.) Further I would have expected that there >>>>>>>>>>>>>>> already exist tests that check that, for a given thread >>>>>>>>>>>>>>> (which may be suspended or known to be blocked) the same >>>>>>>>>>>>>>> stack is found through the two different APIs. >>>>>>>>>>>>>> >>>>>>>>>>>>>> vmTestbase/nsk/jvmti/unit/GetAllStackTraces/getallstktr001 >>>>>>>>>>>>>> would check all of threads via GetThreadListStackTraces() >>>>>>>>>>>>>> and GetAllStackTraces(), so we might be able to remove >>>>>>>>>>>>>> GetThreadListStackTraces.java from this webrev. >>>>>>>>>>>>> >>>>>>>>>>>>> Yes. The existing test only examines a set of test threads >>>>>>>>>>>>> that are all blocked on a raw monitor. You do not need to >>>>>>>>>>>>> duplicate that test. >>>>>>>>>>>>> >>>>>>>>>>>>>> OTOH we don't have testcase for >>>>>>>>>>>>>> GetThreadListStackTraces() with thread_count == 1, so we >>>>>>>>>>>>>> need to testcase for it (it is >>>>>>>>>>>>>> OneGetThreadListStackTraces.java) It would check whether >>>>>>>>>>>>>> the state of target thread is "waiting" before JNI call >>>>>>>>>>>>>> to call GetThreadListStackTraces(), >>>>>>>>>>>>> >>>>>>>>>>>>> Yes we need to test the special cases introduced by your >>>>>>>>>>>>> changes - totally agree - and >>>>>>>>>>>>> OneGetThreadListStackTraces.java is a good test for that. >>>>>>>>>>>>> >>>>>>>>>>>>>> and also I expect it would not be run with JFR. (it is >>>>>>>>>>>>>> not described @run) >>>>>>>>>>>>> >>>>>>>>>>>>> The arguments to run with JFR (or a bunch of other things) >>>>>>>>>>>>> can be passed to the jtreg test harness to be applied to >>>>>>>>>>>>> all tests. >>>>>>>>>>>>> >>>>>>>>>>>>>> Of course we can check GetThreadListStackTraces() with >>>>>>>>>>>>>> main thread, but it is not the test for direct handshake >>>>>>>>>>>>>> for other thread. >>>>>>>>>>>>> >>>>>>>>>>>>> Right - that test already exists as per the above. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> David >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> David >>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 6/07/2020 11:29 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I think C++ is more simple to implement the test agent >>>>>>>>>>>>>>>> as you said. >>>>>>>>>>>>>>>> So I implement it in C++ in new webrev. Could you >>>>>>>>>>>>>>>> review again? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.06/ >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Also I refactored libGetThreadListStackTraces.cpp, and >>>>>>>>>>>>>>>> I've kept exception check after IsSameObject(). >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 2020/07/06 16:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thank you for the update. >>>>>>>>>>>>>>>>> I think, a pending exception after IsSameObject needs >>>>>>>>>>>>>>>>> to be checked. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> The checkStackInfo() needs one more refactoring as >>>>>>>>>>>>>>>>> I've already suggested. >>>>>>>>>>>>>>>>> The body of the loop at L68-L78 should be converted to >>>>>>>>>>>>>>>>> a function check_frame_info. >>>>>>>>>>>>>>>>> The si1->frame_buffer[i] and si2->frame_buffer[i] will >>>>>>>>>>>>>>>>> be passed as fi1 and fi2. >>>>>>>>>>>>>>>>> The index can be passed as well. >>>>>>>>>>>>>>>>> I'm still suggesting to simplify the local >>>>>>>>>>>>>>>>> exception_msg to something shorter like err_msg or >>>>>>>>>>>>>>>>> exc_msg. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I'm not sure using fatal is right here: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> This fragment looks strange: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> ? 152???? if ((*env)->IsSameObject(env, >>>>>>>>>>>>>>>>> stack_info[i].thread, thread)) { >>>>>>>>>>>>>>>>> ? 153?????? target_info = &stack_info[i]; >>>>>>>>>>>>>>>>> ? 154?????? break; >>>>>>>>>>>>>>>>> ? 155???? } else if ((*env)->ExceptionOccurred(env)) { >>>>>>>>>>>>>>>>> ? 156 (*env)->ExceptionDescribe(env); >>>>>>>>>>>>>>>>> ? 157 (*env)->FatalError(env, __FILE__); >>>>>>>>>>>>>>>>> ? 158???? } >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I expected it to be: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> ??? jboolean same = (*env)->IsSameObject(env, >>>>>>>>>>>>>>>>> stack_info[i].thread, thread); >>>>>>>>>>>>>>>>> ??? if ((*env)->ExceptionOccurred(env)) { >>>>>>>>>>>>>>>>> (*env)->ExceptionDescribe(env); >>>>>>>>>>>>>>>>> ????? (*env)->FatalError(env, __FILE__); >>>>>>>>>>>>>>>>> ??? } >>>>>>>>>>>>>>>>> ??? if (same) { >>>>>>>>>>>>>>>>> ????? target_info = &stack_info[i]; >>>>>>>>>>>>>>>>> ????? break; >>>>>>>>>>>>>>>>> ??? } >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Would it better to port this agent to C++ to simplify >>>>>>>>>>>>>>>>> this code nicer? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 7/5/20 06:13, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>>> I refactored testcase. Could you review again? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.05/ >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> It would check Java exception after IsSameObject() >>>>>>>>>>>>>>>>>> call. Does it need? >>>>>>>>>>>>>>>>>> Any exceptions are not described in JNI document[1], >>>>>>>>>>>>>>>>>> and JNI implementation (jni_IsSameObject()) does not >>>>>>>>>>>>>>>>>> seem to throw it. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>>> https://docs.oracle.com/en/java/javase/14/docs/specs/jni/functions.html#issameobject >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 2020/07/05 14:46, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Okay, thanks. >>>>>>>>>>>>>>>>>>> Then I'm okay to keep the GetSingleStackTraceClosure. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c.html >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c.html >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I'm not sure the function 'is_same_thread() is needed. >>>>>>>>>>>>>>>>>>> Why do not use the JNI IsSameObject instead? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> It seems to be a typo at L132 and L137. >>>>>>>>>>>>>>>>>>> You, probably. did not want to print the same >>>>>>>>>>>>>>>>>>> information for stack_info_1[i].frame_buffer[j].XXX >>>>>>>>>>>>>>>>>>> twice. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> The code at lines 112-142 is not readable. >>>>>>>>>>>>>>>>>>> I'd suggest to make a couple of refactoring steps. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> First step to simplify this a little bit would be >>>>>>>>>>>>>>>>>>> with some renaming and getting rid of indexes: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ?? 71 char err_msg[EXCEPTION_MSG_LEN] = {0}; >>>>>>>>>>>>>>>>>>> ??... >>>>>>>>>>>>>>>>>>> ??112?? /* Iterate all jvmtiStackInfo to check */ >>>>>>>>>>>>>>>>>>> ??113?? for (i = 0; i < num_threads, *exception_msg >>>>>>>>>>>>>>>>>>> != '\0'; i++) { >>>>>>>>>>>>>>>>>>> ????????? jvmtiStackInfo *si1 = stack_info_1[i]; >>>>>>>>>>>>>>>>>>> ????????? jvmtiStackInfo *si2 = stack_info_2[i]; >>>>>>>>>>>>>>>>>>> ??114???? if (!IsSameObject(env, si1.thread, >>>>>>>>>>>>>>>>>>> si2.thread)) { /* jvmtiStackInfo::thread */ >>>>>>>>>>>>>>>>>>> ??115?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>>> ??116 "thread[%d] is different: stack_info_1 = %p, >>>>>>>>>>>>>>>>>>> stack_info_2 = %p", >>>>>>>>>>>>>>>>>>> ??117??????????????? i, sinfo1.thread, sinfo2.thread); >>>>>>>>>>>>>>>>>>> ??118???? } else if (si1.state != si2.state) { /* >>>>>>>>>>>>>>>>>>> jvmtiStackInfo::state */ >>>>>>>>>>>>>>>>>>> ??119?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>>> ??120 "state[%d] is different: stack_info_1 = %d, >>>>>>>>>>>>>>>>>>> stack_info_2 = %d", >>>>>>>>>>>>>>>>>>> ??121??????????????? i, si1.state, si2.state); >>>>>>>>>>>>>>>>>>> ??122???? } else if (si1.frame_count != >>>>>>>>>>>>>>>>>>> si2.frame_count) { /* jvmtiStackInfo::frame_count */ >>>>>>>>>>>>>>>>>>> ??123?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>>> ??124 "frame_count[%d] is different: stack_info_1 = >>>>>>>>>>>>>>>>>>> %d, stack_info_2 = %d", >>>>>>>>>>>>>>>>>>> ??125??????????????? i, si1.frame_count, >>>>>>>>>>>>>>>>>>> si2.frame_count); >>>>>>>>>>>>>>>>>>> ??126???? } else { >>>>>>>>>>>>>>>>>>> ??127?????? /* Iterate all jvmtiFrameInfo to check */ >>>>>>>>>>>>>>>>>>> ??128?????? for (j = 0; j < si1.frame_count; j++) { >>>>>>>>>>>>>>>>>>> ??129???????? if (si1.frame_buffer[j].method != >>>>>>>>>>>>>>>>>>> si1.frame_buffer[j].method) { /* >>>>>>>>>>>>>>>>>>> jvmtiFrameInfo::method */ >>>>>>>>>>>>>>>>>>> ??130 snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>>> ??131 "thread [%d] frame_buffer[%d].method is >>>>>>>>>>>>>>>>>>> different: stack_info_1 = %lx, stack_info_2 = %lx", >>>>>>>>>>>>>>>>>>> ??132??????????????????? i, j, >>>>>>>>>>>>>>>>>>> si1.frame_buffer[j].method, >>>>>>>>>>>>>>>>>>> si2.frame_buffer[j].method); >>>>>>>>>>>>>>>>>>> ??133?????????? break; >>>>>>>>>>>>>>>>>>> ??134???????? } else if >>>>>>>>>>>>>>>>>>> (si1.frame_buffer[j].location != >>>>>>>>>>>>>>>>>>> si1.frame_buffer[j].location) { /* >>>>>>>>>>>>>>>>>>> jvmtiFrameInfo::location */ >>>>>>>>>>>>>>>>>>> ??135 snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>>> ??136 "thread [%d] frame_buffer[%d].location is >>>>>>>>>>>>>>>>>>> different: stack_info_1 = %ld, stack_info_2 = %ld", >>>>>>>>>>>>>>>>>>> ??137??????????????????? i, j, >>>>>>>>>>>>>>>>>>> si1.frame_buffer[j].location, >>>>>>>>>>>>>>>>>>> si2.frame_buffer[j].location); >>>>>>>>>>>>>>>>>>> ??138?????????? break; >>>>>>>>>>>>>>>>>>> ??139???????? } >>>>>>>>>>>>>>>>>>> ??140?????? } >>>>>>>>>>>>>>>>>>> ??141???? } >>>>>>>>>>>>>>>>>>> ??142?? } >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Another step would be to create functions that >>>>>>>>>>>>>>>>>>> implement a body of each loop. >>>>>>>>>>>>>>>>>>> You can use the same techniques to simplify similar >>>>>>>>>>>>>>>>>>> place (L127-L138) in the >>>>>>>>>>>>>>>>>>> libOneGetThreadListStackTraces.c. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On 7/3/20 15:55, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I'm not an Oracle employee, so I cannot know real >>>>>>>>>>>>>>>>>>>> request(s) from your customers. >>>>>>>>>>>>>>>>>>>> However JDK-8201641 says Dynatrace has requested >>>>>>>>>>>>>>>>>>>> this enhancement. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> BTW I haven't heared any request from my customers >>>>>>>>>>>>>>>>>>>> about this. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On 2020/07/04 4:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> This difference is not that big to care about. >>>>>>>>>>>>>>>>>>>>> I feel this is really rare case and so, does not >>>>>>>>>>>>>>>>>>>>> worth these complications. >>>>>>>>>>>>>>>>>>>>> Do we have a real request from customers to >>>>>>>>>>>>>>>>>>>>> optimize it? >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On 7/3/20 01:16, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Generally I agree with you, but I have concern >>>>>>>>>>>>>>>>>>>>>> about the difference of the result of >>>>>>>>>>>>>>>>>>>>>> GetStackTrace() and GetThreadListStackTraces(). >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> ? GetStackTrace: jvmtiFrameInfo >>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces: jvmtiStackInfo >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> jvmtiStackInfo contains thread state, and it is >>>>>>>>>>>>>>>>>>>>>> ensured it is the state of the call stack. >>>>>>>>>>>>>>>>>>>>>> If we want to get both call stack and thread >>>>>>>>>>>>>>>>>>>>>> state, we need to suspend target thread, and call >>>>>>>>>>>>>>>>>>>>>> both GetStackTrace() and GetThreadState(). Is it ok? >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I was wondering if JDK-8201641 (parent ticket of >>>>>>>>>>>>>>>>>>>>>> this change) needed them for profiling (dynatrace?) >>>>>>>>>>>>>>>>>>>>>> If it is responsibility of JVMTI agent >>>>>>>>>>>>>>>>>>>>>> implementor, I remove this closure. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 16:45, serguei.spitsyn at oracle.com >>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> After some thinking I've concluded that I do not >>>>>>>>>>>>>>>>>>>>>>> like this optimization >>>>>>>>>>>>>>>>>>>>>>> of the GetThreadListStackTraces with >>>>>>>>>>>>>>>>>>>>>>> GetSingleStackTraceClosure. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> We may need more opinions on this but these are >>>>>>>>>>>>>>>>>>>>>>> my points: >>>>>>>>>>>>>>>>>>>>>>> ??- it adds some complexity and ugliness >>>>>>>>>>>>>>>>>>>>>>> ??- a win is doubtful because it has to be a >>>>>>>>>>>>>>>>>>>>>>> rare case, so that total overhead should not be >>>>>>>>>>>>>>>>>>>>>>> high >>>>>>>>>>>>>>>>>>>>>>> ??- if it is really high for some use cases then >>>>>>>>>>>>>>>>>>>>>>> it is up to the user >>>>>>>>>>>>>>>>>>>>>>> ??? to optimize it with using GetStackTrace instead >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> In such cases with doubtful overhead I usually >>>>>>>>>>>>>>>>>>>>>>> prefer the simplicity. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Good examples where it makes sense to optimize >>>>>>>>>>>>>>>>>>>>>>> are checks for target thread to be current thread. >>>>>>>>>>>>>>>>>>>>>>> In such cases there is no need to suspend the >>>>>>>>>>>>>>>>>>>>>>> target thread, or use a VMop/HandshakeClosure. >>>>>>>>>>>>>>>>>>>>>>> For instance, please, see the Monitor functions >>>>>>>>>>>>>>>>>>>>>>> with the check: (java_thread == calling_thread). >>>>>>>>>>>>>>>>>>>>>>> Getting information for current thread is >>>>>>>>>>>>>>>>>>>>>>> frequently used case, e.g. to get info at an >>>>>>>>>>>>>>>>>>>>>>> event point. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On 7/2/20 23:29, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>> Hi Dan, David, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could you review again? >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/ >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> OneGetThreadListStackTraces.java in this webrev >>>>>>>>>>>>>>>>>>>>>>>> would wait until thread state is transited to >>>>>>>>>>>>>>>>>>>>>>>> "waiting" with spin wait. >>>>>>>>>>>>>>>>>>>>>>>> CountDownLatch::await call as Dan pointed is >>>>>>>>>>>>>>>>>>>>>>>> fixed in it :) >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Diff from webrev.03: >>>>>>>>>>>>>>>>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/c9aeb7001e50 >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 14:15, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 12:24, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> On 7/2/20 10:50 PM, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry I'm responding here without seeing >>>>>>>>>>>>>>>>>>>>>>>>>>>> latest webrev but there is enough context I >>>>>>>>>>>>>>>>>>>>>>>>>>>> think ... >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Dan, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 7:16, Daniel D. Daugherty >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I upload new webrev. Could you review >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L1542: // Get stack trace with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> handshake >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - please add a period at the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> end. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> L1591: *stack_info_ptr = op.stack_info(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? The return parameter should not >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be touched unless the return >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? old L1582: if (err == >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JVMTI_ERROR_NONE) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please restore this check. The return >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> parameter should not >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? be touched unless the return >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> But op.stack_info() will return NULL if the >>>>>>>>>>>>>>>>>>>>>>>>>>>> error is not JVMTI_ERROR_NONE. Are you >>>>>>>>>>>>>>>>>>>>>>>>>>>> (Dan) concerned about someone passing in a >>>>>>>>>>>>>>>>>>>>>>>>>>>> non-null/initialized out-pointer that will >>>>>>>>>>>>>>>>>>>>>>>>>>>> be reset to NULL if there was an error? >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Actually the way we used to test this in >>>>>>>>>>>>>>>>>>>>>>>>>>> POSIX tests is to call >>>>>>>>>>>>>>>>>>>>>>>>>>> an API with known bad parameters and the >>>>>>>>>>>>>>>>>>>>>>>>>>> return parameter ptr >>>>>>>>>>>>>>>>>>>>>>>>>>> set to NULL. If the return parameter ptr was >>>>>>>>>>>>>>>>>>>>>>>>>>> touched when an >>>>>>>>>>>>>>>>>>>>>>>>>>> error should have been detected on an >>>>>>>>>>>>>>>>>>>>>>>>>>> earlier parameter, then >>>>>>>>>>>>>>>>>>>>>>>>>>> the test failed. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> L1272: ? if (!jt->is_exiting() && >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (thread_oop != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> second expression. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? old L1532: _result = >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? This deletion of the _result >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> field threw me for a minute and then >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? ? ? I figured out that the field is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> init to JVMTI_ERROR_THREAD_NOT_ALIVE >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? in the constructor. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L1553: if (!jt->is_exiting() && >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> second expression. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L64: startSignal.countDown(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? I was expecting this to be a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call to await() instead of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> countDown(). What am I missing here? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? I think this test might be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> passing by accident right now, but... >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Main thread (which call JVMTI functions to >>>>>>>>>>>>>>>>>>>>>>>>>>>>> test) should wait until test thread is ready. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> So main thread would wait startSignal, and >>>>>>>>>>>>>>>>>>>>>>>>>>>>> test thread would count down. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> No! >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> The test thread that previously called >>>>>>>>>>>>>>>>>>>>>>>>>>>> obj.wait() now calls latch.await(). >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> The main thread that previously called >>>>>>>>>>>>>>>>>>>>>>>>>>>> obj.notify() now calls latch.countDown(). >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> The main thread continues to spin until it >>>>>>>>>>>>>>>>>>>>>>>>>>>> sees the target is WAITING before >>>>>>>>>>>>>>>>>>>>>>>>>>>> proceeding with the test. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> If I add spin wait to wait until transit >>>>>>>>>>>>>>>>>>>>>>>>>> target thread state is WAITING (as >>>>>>>>>>>>>>>>>>>>>>>>>> following), we don't need to call >>>>>>>>>>>>>>>>>>>>>>>>>> SuspendThread(). >>>>>>>>>>>>>>>>>>>>>>>>>> Which is better? >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> The original spin-wait loop checking for >>>>>>>>>>>>>>>>>>>>>>>>> WAITING is better because it is the only >>>>>>>>>>>>>>>>>>>>>>>>> guarantee that the target thread is blocked >>>>>>>>>>>>>>>>>>>>>>>>> where you need it to be. suspending the thread >>>>>>>>>>>>>>>>>>>>>>>>> is racy as you don't know exactly where the >>>>>>>>>>>>>>>>>>>>>>>>> suspend will hit. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>> /* Wait until the thread state transits to >>>>>>>>>>>>>>>>>>>>>>>>>> "waiting" */ >>>>>>>>>>>>>>>>>>>>>>>>>> while (th.getState() != Thread.State.WAITING) { >>>>>>>>>>>>>>>>>>>>>>>>>> Thread.onSpinWait(); >>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> For simplify, spin wait is prefer to >>>>>>>>>>>>>>>>>>>>>>>>>> OneGetThreadListStackTraces.java in webrev.03. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Here's the flow as I see it: >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - start worker thread >>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - startSignal.await() >>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - startSignal.countDown() >>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - stopSignal.await() >>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - worker is now blocked >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - checkCallStacks(th) >>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - stopSignal.countDown() >>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - worker is now unblocked >>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - th.join >>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - runs off the end of run() >>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - run off the end of main() >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L92: jthreads = (jthread >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *)malloc(sizeof(jthread) * num_threads); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? You don't check for malloc() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> failure. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'jthreads' is allocated but never freed. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L91: result = >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (*jvmti)->SuspendThread(jvmti, thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? Why are you suspending the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread? GetAllStackTraces() and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() do not require >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the target thread(s) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? to be suspend. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? If you decide not to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SuspendThread, then you don't need the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AddCapabilities or the ResumeThread calls. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Test thread might not be entered following >>>>>>>>>>>>>>>>>>>>>>>>>>>>> code (stopSignal.await()). We might see >>>>>>>>>>>>>>>>>>>>>>>>>>>>> deferent call stack between >>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetAllStackTraces() and >>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces(). We cannot >>>>>>>>>>>>>>>>>>>>>>>>>>>>> control to freeze call stack of test >>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread in Java code. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> (I didn't use SuspendThread() at first, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> but I saw some errors which causes in above.) >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> So we need to call SuspendThread() to >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure we can see same call stack. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> If you are checking that the thread is in >>>>>>>>>>>>>>>>>>>>>>>>>>>> state WAITING then it cannot escape from >>>>>>>>>>>>>>>>>>>>>>>>>>>> that state and you can sample the stack >>>>>>>>>>>>>>>>>>>>>>>>>>>> multiple times from any API and get the >>>>>>>>>>>>>>>>>>>>>>>>>>>> same result. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I suspect the errors you saw were from the >>>>>>>>>>>>>>>>>>>>>>>>>>>> apparent incorrect use of the CountDownLatch. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> With the flow outlined above, the worker >>>>>>>>>>>>>>>>>>>>>>>>>>> thread should be >>>>>>>>>>>>>>>>>>>>>>>>>>> nicely blocked in stopSignal.await() when >>>>>>>>>>>>>>>>>>>>>>>>>>> stuff is sampled. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could review >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Updates look fine - thanks. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> One minor nit: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1274 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector.allocate_and_fill_stacks(1); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1275 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector.set_result(JVMTI_ERROR_NONE); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the other places where you use >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector you rely on result being >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initialized to JVMTI_ERROR_NONE, rather >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than setting it directly after >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocate_and_fill_stacks(). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 820 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> assert(SafepointSynchronize::is_at_safepoint() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 821 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 822 current_thread == >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 823 "at safepoint / handshake >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> check is necessary, as even if >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the target is suspended we must >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> still be at a safepoint or in a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> handshake with it. Makes me >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wonder if we used to allow a racy >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stacktrace operation on a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> suspended thread, assuming it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This function >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (JvmtiEnvBase::get_stack_trace()) can >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be called to get own stack trace. For >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> example, we can call GetStackTrace() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for current thread at JVMTI event. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So I changed assert as below: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??820 assert(current_thread == >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??821 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SafepointSynchronize::is_at_safepoint() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??823 "call by myself / at safepoint / >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at handshake"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yep good catch. I hope current tests >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> caught that. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> They would be tested in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (own call stacks), and getstacktr003 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (call stacks in other thread). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Speaking of tests ... >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the native code I think you need to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> check the success of all JNI methods >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that can throw exceptions - otherwise I >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> believe the tests may trigger warnings >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if -Xcheck:jni is used with them. See >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for example: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I updated testcases to check JNI and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JVMTI function calls. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the Java code the target thread: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 45 public void run() { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 46 try { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 47 synchronized (lock) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 48 lock.wait(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 49 System.out.println("OK"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 50?????????? } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is potentially susceptible to spurious >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wakeups. Using a CountDownLatch would >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be robust. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 9:05 am, Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _calling_thread is the current >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If so, we can call make_local() in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> L1272 without JavaThread (or we can >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pass current thread to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make_local()). Is it right? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry I got confused, _calling_thread >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> may not be the current thread as we >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could be executing the handshake in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the target thread itself. So the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark is correct as-is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (implicitly for current thread). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The argument to fill_frames will be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> used in the jvmtiStackInfo and passed >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back to the _calling_thread, so it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> must be created via >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make_local(_calling_thread, ...) as >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you presently have. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you for reviewing! I will >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> update new webrev tomorrow. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 498 class VM_GetAllStackTraces >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 500 JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 502 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> member of another class like that >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as it may not be on the stack. I >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should not extend any allocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class, and should always be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure what does mean >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "embedded". >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??? : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class GetAllStackTraces : public >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/30 22:22, David Holmes >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I updated webrev for 8242428. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Could you review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change migrate to use >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> direct handshake for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetStackTrace() and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() (when >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread_count == 1). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks really good now! I >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only have a few nits below. There >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is one thing I don't like about >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it but it requires a change to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the main Handshake logic to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> address - in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JvmtiEnv::GetThreadListStackTraces >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you have to create a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ThreadsListHandle to convert the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jthread to a JavaThread, but then >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the Handshake::execute_direct >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates another ThreadsListHandle >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> internally. That's a waste. I >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will discuss with Robbin and file >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a RFE to have an overload of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> execute_direct that takes an >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> existing TLH. Actually it's worse >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than that because we have another >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> TLH in use at the entry point for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the JVMTI functions, so I think >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there may be some scope for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> simplifying the use of TLH >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> instances - future RFE. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??451 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetStackTraceClosure(JvmtiEnv >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *env, jint start_depth, jint >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max_count, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??452 jvmtiFrameInfo* >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> frame_buffer, jint* count_ptr) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??453???? : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??454 _env(env), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _start_depth(start_depth), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _max_count(max_count), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??455 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _frame_buffer(frame_buffer), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _count_ptr(count_ptr), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??456 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _result(JVMTI_ERROR_THREAD_NOT_ALIVE) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Nit: can you do one initializer >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> per line please. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??498 class VM_GetAllStackTraces >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??500 JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??502 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> member of another class like that >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as it may not be on the stack. I >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should not extend any allocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class, and should always be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 481 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector(JvmtiEnv >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *env, jint max_frame_count) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??483 _max_frame_count = >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max_frame_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??484 _frame_count_total = 0; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??485 _head = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??486 _stack_info = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??487 _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As you are touching this can you >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change it to use an initializer >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> list as you did for the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HandshakeClosure, and please keep >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one item per line. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??820 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> assert(SafepointSynchronize::is_at_safepoint() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??821 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??823 "at safepoint / handshake >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> check is necessary, as even if >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the target is suspended we must >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> still be at a safepoint or in a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> handshake with it. Makes me >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wonder if we used to allow a racy >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stacktrace operation on a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> suspended thread, assuming it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1268?? oop thread_oop = >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jt->threadObj(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1269 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is frustrating that this >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entire call chain started with a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jthread reference, which we >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> converted to a JavaThread, only >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to eventually need to convert it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back to a jthread! I think there >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is some scope for simplification >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> here but not as part of this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _calling_thread is the current >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm still pondering the test >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic but wanted to send this now. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetThreadListStackTrace (for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces) and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetAllStackTraces (for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetAllStackTraces) have >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> inherited >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetMultipleStackTraces VM >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operation which provides the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature to generate >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jvmtiStackInfo. I modified >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetMultipleStackTraces to a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> normal C++ class to share with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HandshakeClosure for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (GetSingleStackTraceClosure). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I added new testcases which >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test GetThreadListStackTraces() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with thread_count == 1 and with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> all threads. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jvmti >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jdwp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jdi >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? JBS: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? webrev: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change replace following >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM operations to direct handshake. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameCount >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (GetFrameCount()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (GetFrameLocation()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (GetThreadListStackTrace()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTrace() uses >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> direct handshake if thread >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> count == 1. In other case >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (thread count > 1), it would be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> performed as VM operation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Caller of VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (JvmtiEnvThreadState::reset_current_location()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> might be called at safepoint. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So I added safepoint check in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> its caller. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jvmti >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jdwp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jdi vmTestbase/ns >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I tested it on submit >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repo, then it has execution >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> error >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> due to dependency error. So I >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it does not occur by this >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>> >>>>> >>>> >> From vladimir.kozlov at oracle.com Sat Jul 11 00:06:15 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 10 Jul 2020 17:06:15 -0700 Subject: RFR(s): 8236042: [TESTBUG] serviceability/sa/ClhsdbCDSCore.java fails with -Xcomp -XX:TieredStopAtLevel=1 In-Reply-To: <958fecdf-d7a1-4b22-835e-a75fadda0a84@default> References: <2abe9fba-e958-4b34-9f92-6bb8d8478f4e@default> <70057c31-e535-f03a-391d-d181b2ec150b@oracle.com> <958fecdf-d7a1-4b22-835e-a75fadda0a84@default> Message-ID: <37bb4585-21b5-a6e7-9ee1-88ccf9be0914@oracle.com> Fix is good. I think next are reasons you don't get MDO in this scenario. Tier1 (C1 compilation) does not generate profiling code and does not created MDO. C1 request MDO only with tiers 2 and 3 [1][2]. With -Xcomp flag a Java method is not executed in Interpreter but requests its compilation and waits when it is finished. As result MDO is not created in Interpreter too. May be late if a method is deoptimized it will be executed in Interpreter and MDO will be created. Thanks, Vladimir [1] http://hg.openjdk.java.net/jdk/jdk/file/796c9fa50850/src/hotspot/share/c1/c1_Compilation.hpp#l226 [2] http://hg.openjdk.java.net/jdk/jdk/file/796c9fa50850/src/hotspot/share/c1/c1_Compilation.cpp#l381 On 7/7/20 8:47 PM, Fairoz Matte wrote: > Thanks Chris, for the review comments. > > I have updated the suggested change. > > Thanks, > Fairoz > >> -----Original Message----- >> From: Chris Plummer >> Sent: Wednesday, July 8, 2020 3:38 AM >> To: Fairoz Matte ; hotspot-compiler- >> dev at openjdk.java.net; serviceability-dev at openjdk.java.net >> Subject: Re: RFR(s): 8236042: [TESTBUG] serviceability/sa/ClhsdbCDSCore.java >> fails with -Xcomp -XX:TieredStopAtLevel=1 >> >> Hi Fairoz, >> >> Looks good, except for the missing space in "if(testJavaOpts...". >> >> thanks, >> >> Chris >> >> On 7/7/20 7:49 AM, Fairoz Matte wrote: >>> Hi, >>> >>> Please review this small test change to consider the scenario when there is no >> "printmdo" output >>> >>> JBS - https://bugs.openjdk.java.net/browse/JDK-8236042 >>> Webrev - http://cr.openjdk.java.net/~fmatte/8236042/webrev.00/ >>> >>> Thanks, >>> Fairoz >> From suenaga at oss.nttdata.com Sat Jul 11 01:52:06 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Sat, 11 Jul 2020 10:52:06 +0900 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <8c387550-a06e-6320-f453-0e898486befb@oss.nttdata.com> <19cfa2a7-f441-425d-5623-1c11c6a870b6@oss.nttdata.com> <237e8d26-d185-1008-6e07-5b9f36314534@oracle.com> <2a9ba1f8-b1f7-cb8c-fe11-56dd174d9d58@oracle.com> <7ad907cb-24ef-aa55-972a-0a3f63576568@oracle.com> <5ab8c5d9-9a5e-74a6-4605-7e73732037bf@oracle.com> <4881e0da-f716-c62e-b744-17aa3d03e095@oss.nttdata.com> Message-ID: <8dfa047c-8f84-5775-d12d-3248137e4b45@oss.nttdata.com> Thanks Dan! David, Serguei, are you ok to this change? Yasumasa On 2020/07/11 7:00, Daniel D. Daugherty wrote: > On 7/10/20 3:29 AM, Yasumasa Suenaga wrote: >> Thanks Patricio! >> >> I uploaded new webrev. Could you review again? >> It passed all tests on submit repo, and passed jtreg tests about JVMTI. >> >> ? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.10/ >> ? Diff from webrev.09: http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.10/incremental/ > > test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp > ??? No comments. > > Thumbs up on the incremental. > > Dan > > >> >> >> Yasumasa >> >> >> On 2020/07/10 14:49, Patricio Chilano wrote: >>> Hi Yasumasa, >>> >>> On 7/10/20 1:58 AM, Yasumasa Suenaga wrote: >>>> Hi Patricio, >>>> >>>> Thanks for your advice! >>>> I've fixed testcase as [1], but I still see an error in validate-headers-linux-x64-build-1 on submit repo. >>>> What does it mean? Can you share details? >>>> >>>> ? mach5-one-ysuenaga-JDK-8242428-20200710-0339-12529134 >>> Not your bug, you are getting the header validation failure from 8248570. It was fixed two days ago so just update the repo and it should work now. >>> >>> Thanks, >>> Patricio >>>> Of course this change can be built on my Linux box (Fedora 32, AMD64, GCC 10.1) >>>> >>>> >>>> Yasumasa >>>> >>>> >>>> [1] http://hg.openjdk.java.net/jdk/submit/file/45f52142db9d/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>> >>>> >>>> On 2020/07/10 2:18, Patricio Chilano wrote: >>>>> >>>>> On 7/9/20 12:00 PM, Patricio Chilano wrote: >>>>>> Hi Yasumasa, >>>>>> >>>>>> On 7/9/20 9:30 AM, Yasumasa Suenaga wrote: >>>>>>> On 2020/07/09 17:58, David Holmes wrote: >>>>>>>> Hi Yasumasa, >>>>>>>> >>>>>>>> On 9/07/2020 10:25 am, Yasumasa Suenaga wrote: >>>>>>>>> Hi Dan, >>>>>>>>> >>>>>>>>> Thanks for your comment! >>>>>>>>> I uploaded new webrev: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.09/ >>>>>>>>> ?? Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/5d167adf8524 >>>>>>>>> >>>>>>>>> I saw similar build errors in libOneGetThreadListStackTraces.cpp on Windows. >>>>>>>>> This webrev fixes them. >>>>>>>> >>>>>>>> You shouldn't use %p as it may not be portable. In the VM we use INTPTR_FORMAT and convert the arg using p2i. I don't know what exists in the testing code. >>>>>>> >>>>>>> I replaced %p to %lx, and also cast values to unsigned long [1] [2], but the test on submit repo was failed. >>>>>>> Can anyone share details of mach5-one-ysuenaga-JDK-8242428-20200709-1030-12500928 ? >>>>>> These are the errors I see for the macOS build: >>>>>> >>>>>> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:53:14: error: format specifies type 'long long' but the argument has type 'jlocation' (aka 'long') [-Werror,-Wformat] >>>>>> ????????????? fi1->location, fi2->location); >>>>>> ????????????? ^~~~~~~~~~~~~ >>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:53:29: error: format specifies type 'long long' but the argument has type 'jlocation' (aka 'long') [-Werror,-Wformat] >>>>>> ????????????? fi1->location, fi2->location); >>>>>> ???????????????????????????? ^~~~~~~~~~~~~ >>>>>> >>>>>> These are the ones I see for the Windows build: >>>>>> >>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): warning C4311: 'type cast': pointer truncation from 'jmethodID' to 'unsigned long' >>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): warning C4302: 'type cast': truncation from 'jmethodID' to 'unsigned long' >>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): warning C4311: 'type cast': pointer truncation from 'jmethodID' to 'unsigned long' >>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): warning C4302: 'type cast': truncation from 'jmethodID' to 'unsigned long' >>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): warning C4311: 'type cast': pointer truncation from 'jthread' to 'unsigned long' >>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): warning C4302: 'type cast': truncation from 'jthread' to 'unsigned long' >>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): warning C4311: 'type cast': pointer truncation from 'jthread' to 'unsigned long' >>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): warning C4302: 'type cast': truncation from 'jthread' to 'unsigned long' >>>>>> >>>>>> >>>>>> You will probably want to use the macros defined in src/hotspot/share/utilities/globalDefinitions.hpp. Let me know if you need me to test something. >>>>> With these changes the build works okay on Linux, Windows and macOS: >>>>> >>>>> >>>>> --- a/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>> +++ b/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>> @@ -27,4 +27,5 @@ >>>>> ??#include >>>>> ??#include >>>>> +#include >>>>> >>>>> ??#define MAX_FRAMES 100 >>>>> @@ -45,11 +46,11 @@ >>>>> ??? if (fi1->method != fi2->method) { /* jvmtiFrameInfo::method */ >>>>> ????? snprintf(err_msg, sizeof(err_msg), >>>>> -???????????? "method is different: fi1 = %p, fi2 = %p", >>>>> -???????????? fi1->method, fi2->method); >>>>> +???????????? "method is different: fi1 = 0x%016" PRIxPTR " , fi2 = 0x%016" PRIxPTR, >>>>> +???????????? (intptr_t)fi1->method, (intptr_t)fi2->method); >>>>> ????? env->FatalError(err_msg); >>>>> ??? } else if (fi1->location != fi2->location) { /* jvmtiFrameInfo::location */ >>>>> ????? snprintf(err_msg, sizeof(err_msg), >>>>> -???????????? "location is different: fi1 = %lld, fi2 = %lld", >>>>> -???????????? fi1->location, fi2->location); >>>>> +???????????? "location is different: fi1 = %" PRId64 " , fi2 = %" PRId64, >>>>> +???????????? (int64_t)fi1->location, (int64_t)fi2->location); >>>>> ????? env->FatalError(err_msg); >>>>> ??? } >>>>> @@ -67,5 +68,5 @@ >>>>> ??? if (!is_same) { /* jvmtiStackInfo::thread */ >>>>> ????? snprintf(err_msg, sizeof(err_msg), >>>>> -???????????? "thread is different: si1 = %p, si2 = %p", si1->thread, si2->thread); >>>>> +???????????? "thread is different: si1 = 0x%016" PRIxPTR " , si2 = 0x%016" PRIxPTR, (intptr_t)si1->thread, (intptr_t)si2->thread); >>>>> ????? env->FatalError(err_msg); >>>>> ??? } else if (si1->state != si2->state) { /* jvmtiStackInfo::state */ >>>>> >>>>> >>>>> Maybe you can use something like that. >>>>> >>>>> >>>>> Thanks, >>>>> Patricio >>>>>> Thanks, >>>>>> Patricio >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> [1] http://hg.openjdk.java.net/jdk/submit/rev/dfca51958217 >>>>>>> [2] http://hg.openjdk.java.net/jdk/submit/rev/3665361fa91b >>>>>>> >>>>>>> >>>>>>>> David >>>>>>>> ----- >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2020/07/09 1:42, Daniel D. Daugherty wrote: >>>>>>>>>> ?> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ >>>>>>>>>> >>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>> ???? No comments. >>>>>>>>>> >>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>> ???? L1159: ? Thread *current_thread = Thread::current(); >>>>>>>>>> ???????? Please add "#ifdef ASSERT" above and "#endif" below since >>>>>>>>>> ???????? current_thread is only used for the assert() in this function. >>>>>>>>>> >>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>> ???? L549: ???????????????????????????? jthread java_thread, jint max_frame_count) >>>>>>>>>> ???? L552: ????? _jthread(java_thread), >>>>>>>>>> ???????? Please: s/java_thread/thread/ on both lines. >>>>>>>>>> >>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>> ???? No comments. >>>>>>>>>> >>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>> ???? No comments. >>>>>>>>>> >>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>>>>>> ???? L27: #include >>>>>>>>>> ???????? This include is out of order; should be first in the list. >>>>>>>>>> >>>>>>>>>> ???? This file doesn't compile on my MBP13: >>>>>>>>>> >>>>>>>>>> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:49:14: error: format specifies type 'unsigned long' but the argument has type 'jmethodID' (aka '_jmethodID *') [-Werror,-Wformat] >>>>>>>>>> ????????????? fi1->method, fi2->method); >>>>>>>>>> ????????????? ^~~~~~~~~~~ >>>>>>>>>> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:49:27: error: format specifies type 'unsigned long' but the argument has type 'jmethodID' (aka '_jmethodID *') [-Werror,-Wformat] >>>>>>>>>> ????????????? fi1->method, fi2->method); >>>>>>>>>> ?????????????????????????? ^~~~~~~~~~~ >>>>>>>>>> ???? 2 errors generated. >>>>>>>>>> >>>>>>>>>> ???? This change made it compile on my MBP13, but that may break it on >>>>>>>>>> ???? other platforms: >>>>>>>>>> >>>>>>>>>> ???? $ hg diff >>>>>>>>>> ???? diff -r 560847c69fbe test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>>>>>> ???? --- a/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp Wed Jul 08 12:13:32 2020 -0400 >>>>>>>>>> ???? +++ b/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp Wed Jul 08 12:40:42 2020 -0400 >>>>>>>>>> ???? @@ -46,7 +46,7 @@ >>>>>>>>>> ?? ?? ? if (fi1->method != fi2->method) { /* jvmtiFrameInfo::method */ >>>>>>>>>> ?? ?? ??? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>> ?? ? ? ??????????? "method is different: fi1 = %lx, fi2 = %lx", >>>>>>>>>> ???? -???????????? fi1->method, fi2->method); >>>>>>>>>> ???? +???????????? (unsigned long) fi1->method, (unsigned long) fi2->method); >>>>>>>>>> ?? ?? ??? env->FatalError(err_msg); >>>>>>>>>> ?? ?? ? } else if (fi1->location != fi2->location) { /* jvmtiFrameInfo::location */ >>>>>>>>>> ?? ?? ??? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>> >>>>>>>>>> ???? I'm not sure of the right platform independent way to output >>>>>>>>>> ???? the 'method' field. >>>>>>>>>> >>>>>>>>>> Dan >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 7/8/20 4:04 AM, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi David, >>>>>>>>>>> >>>>>>>>>>> On 2020/07/08 15:27, David Holmes wrote: >>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>> >>>>>>>>>>>> On 7/07/2020 6:54 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>> >>>>>>>>>>>>> Serguei, thank you for replying even though you are on vacaiton! >>>>>>>>>>>>> >>>>>>>>>>>>> I uploaded new webrev: >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.07/ >>>>>>>>>>>>> ?? Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/77243b1dcbfe >>>>>>>>>>>>> >>>>>>>>>>>>> c'tor of GetSingleStackTraceClosure has jthread argument in this webrev. >>>>>>>>>>>>> Also it does not contain testcase for GetThreadListStackTraces with all threads, and OneGetThreadListStackTraces would test main thread only. >>>>>>>>>>>> >>>>>>>>>>>> All those changes are fine in principle for me. One nit/suggestion: >>>>>>>>>>>> >>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>> >>>>>>>>>>>> ??544?? jthread _java_thread; >>>>>>>>>>>> >>>>>>>>>>>> elsewhere "java_thread" refers to a JavaThread, so to avoid confusion may I suggest this member be named _jthread. >>>>>>>>>>> >>>>>>>>>>> I uploaded new webrev: >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ >>>>>>>>>>> ? Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/ca6263dbdc87 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> I'm going to be away for the next couple of days - sorry - but will try to check email on this if I can. >>>>>>>>>>> >>>>>>>>>>> Thanks! >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> David >>>>>>>>>>>> ----- >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 2020/07/07 15:13, David Holmes wrote: >>>>>>>>>>>>>> On 7/07/2020 2:57 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 2020/07/07 11:31, David Holmes wrote: >>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hard to keep up with the changes - especially without incremental webrevs. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Sorry, I will upload diff from previous webrev in the next. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> If GetSingleStackTraceClosure also took the jthread as a constructor arg, then you wouldn't need to recreate a JNI local handle when calling _collector.fill_frames. It's a small simplification and not essential at this stage. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I think we should get jthread from an argument of do_thread() because do_thread() would pass the thread which are stopped certainly. >>>>>>>>>>>>>>> It might be simplification if we pass _calling_thread to MultipleStackTracesCollector. `jthread` is only needed to store jvmtiStackInfo.thread . What do you think? >>>>>>>>>>>>>> >>>>>>>>>>>>>> I'm not quite sure what you mean. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I think there is a bit of a design wart with direct handshakes in that do_thread takes the target JavaThread as an argument. That's useful in a case where you want a HandshakeClosure that can be applied to multiple threads, but that's not typically what is needed with direct handshakes - there is only a single target. With a single-target HandshakeClosure you can capture all the "target" information for the operation in the closure instance. So if the actual do_thread operation wants the jthread corresponding to the target thread then we can store that in the closure rather than recomputing it (you could assert it is the same but that seems overkill to me). >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> For the test ... I don't see how Java_GetThreadListStackTraces_checkCallStacks is a valid test. It gets the stacks of all live threads, then uses that information to use GetThreadListStackTraces to get the stack for the same set of threads through a different API. It then compares the two sets of stacks for each thread expecting them to be the same, but that need only be the case for the main thread. Other threads could potentially have a different stack (e.g. if this test is run with JFR enabled there will be additional threads found.) Further I would have expected that there already exist tests that check that, for a given thread (which may be suspended or known to be blocked) the same stack is found through the two different APIs. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> vmTestbase/nsk/jvmti/unit/GetAllStackTraces/getallstktr001 would check all of threads via GetThreadListStackTraces() and GetAllStackTraces(), so we might be able to remove GetThreadListStackTraces.java from this webrev. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yes. The existing test only examines a set of test threads that are all blocked on a raw monitor. You do not need to duplicate that test. >>>>>>>>>>>>>> >>>>>>>>>>>>>>> OTOH we don't have testcase for GetThreadListStackTraces() with thread_count == 1, so we need to testcase for it (it is OneGetThreadListStackTraces.java) It would check whether the state of target thread is "waiting" before JNI call to call GetThreadListStackTraces(), >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yes we need to test the special cases introduced by your changes - totally agree - and OneGetThreadListStackTraces.java is a good test for that. >>>>>>>>>>>>>> >>>>>>>>>>>>>>> and also I expect it would not be run with JFR. (it is not described @run) >>>>>>>>>>>>>> >>>>>>>>>>>>>> The arguments to run with JFR (or a bunch of other things) can be passed to the jtreg test harness to be applied to all tests. >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Of course we can check GetThreadListStackTraces() with main thread, but it is not the test for direct handshake for other thread. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Right - that test already exists as per the above. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> David >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 6/07/2020 11:29 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I think C++ is more simple to implement the test agent as you said. >>>>>>>>>>>>>>>>> So I implement it in C++ in new webrev. Could you review again? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.06/ >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Also I refactored libGetThreadListStackTraces.cpp, and I've kept exception check after IsSameObject(). >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 2020/07/06 16:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thank you for the update. >>>>>>>>>>>>>>>>>> I think, a pending exception after IsSameObject needs to be checked. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> The checkStackInfo() needs one more refactoring as I've already suggested. >>>>>>>>>>>>>>>>>> The body of the loop at L68-L78 should be converted to a function check_frame_info. >>>>>>>>>>>>>>>>>> The si1->frame_buffer[i] and si2->frame_buffer[i] will be passed as fi1 and fi2. >>>>>>>>>>>>>>>>>> The index can be passed as well. >>>>>>>>>>>>>>>>>> I'm still suggesting to simplify the local exception_msg to something shorter like err_msg or exc_msg. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I'm not sure using fatal is right here: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> This fragment looks strange: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> ? 152???? if ((*env)->IsSameObject(env, stack_info[i].thread, thread)) { >>>>>>>>>>>>>>>>>> ? 153?????? target_info = &stack_info[i]; >>>>>>>>>>>>>>>>>> ? 154?????? break; >>>>>>>>>>>>>>>>>> ? 155???? } else if ((*env)->ExceptionOccurred(env)) { >>>>>>>>>>>>>>>>>> ? 156 (*env)->ExceptionDescribe(env); >>>>>>>>>>>>>>>>>> ? 157 (*env)->FatalError(env, __FILE__); >>>>>>>>>>>>>>>>>> ? 158???? } >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I expected it to be: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> ??? jboolean same = (*env)->IsSameObject(env, stack_info[i].thread, thread); >>>>>>>>>>>>>>>>>> ??? if ((*env)->ExceptionOccurred(env)) { >>>>>>>>>>>>>>>>>> (*env)->ExceptionDescribe(env); >>>>>>>>>>>>>>>>>> ????? (*env)->FatalError(env, __FILE__); >>>>>>>>>>>>>>>>>> ??? } >>>>>>>>>>>>>>>>>> ??? if (same) { >>>>>>>>>>>>>>>>>> ????? target_info = &stack_info[i]; >>>>>>>>>>>>>>>>>> ????? break; >>>>>>>>>>>>>>>>>> ??? } >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Would it better to port this agent to C++ to simplify this code nicer? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 7/5/20 06:13, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>>>> I refactored testcase. Could you review again? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.05/ >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> It would check Java exception after IsSameObject() call. Does it need? >>>>>>>>>>>>>>>>>>> Any exceptions are not described in JNI document[1], and JNI implementation (jni_IsSameObject()) does not seem to throw it. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> [1] https://docs.oracle.com/en/java/javase/14/docs/specs/jni/functions.html#issameobject >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On 2020/07/05 14:46, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Okay, thanks. >>>>>>>>>>>>>>>>>>>> Then I'm okay to keep the GetSingleStackTraceClosure. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c.html >>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c.html >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I'm not sure the function 'is_same_thread() is needed. >>>>>>>>>>>>>>>>>>>> Why do not use the JNI IsSameObject instead? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> It seems to be a typo at L132 and L137. >>>>>>>>>>>>>>>>>>>> You, probably. did not want to print the same information for stack_info_1[i].frame_buffer[j].XXX twice. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> The code at lines 112-142 is not readable. >>>>>>>>>>>>>>>>>>>> I'd suggest to make a couple of refactoring steps. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> First step to simplify this a little bit would be with some renaming and getting rid of indexes: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> ?? 71 char err_msg[EXCEPTION_MSG_LEN] = {0}; >>>>>>>>>>>>>>>>>>>> ??... >>>>>>>>>>>>>>>>>>>> ??112?? /* Iterate all jvmtiStackInfo to check */ >>>>>>>>>>>>>>>>>>>> ??113?? for (i = 0; i < num_threads, *exception_msg != '\0'; i++) { >>>>>>>>>>>>>>>>>>>> ????????? jvmtiStackInfo *si1 = stack_info_1[i]; >>>>>>>>>>>>>>>>>>>> ????????? jvmtiStackInfo *si2 = stack_info_2[i]; >>>>>>>>>>>>>>>>>>>> ??114???? if (!IsSameObject(env, si1.thread, si2.thread)) { /* jvmtiStackInfo::thread */ >>>>>>>>>>>>>>>>>>>> ??115?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>>>> ??116 "thread[%d] is different: stack_info_1 = %p, stack_info_2 = %p", >>>>>>>>>>>>>>>>>>>> ??117??????????????? i, sinfo1.thread, sinfo2.thread); >>>>>>>>>>>>>>>>>>>> ??118???? } else if (si1.state != si2.state) { /* jvmtiStackInfo::state */ >>>>>>>>>>>>>>>>>>>> ??119?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>>>> ??120 "state[%d] is different: stack_info_1 = %d, stack_info_2 = %d", >>>>>>>>>>>>>>>>>>>> ??121??????????????? i, si1.state, si2.state); >>>>>>>>>>>>>>>>>>>> ??122???? } else if (si1.frame_count != si2.frame_count) { /* jvmtiStackInfo::frame_count */ >>>>>>>>>>>>>>>>>>>> ??123?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>>>> ??124 "frame_count[%d] is different: stack_info_1 = %d, stack_info_2 = %d", >>>>>>>>>>>>>>>>>>>> ??125??????????????? i, si1.frame_count, si2.frame_count); >>>>>>>>>>>>>>>>>>>> ??126???? } else { >>>>>>>>>>>>>>>>>>>> ??127?????? /* Iterate all jvmtiFrameInfo to check */ >>>>>>>>>>>>>>>>>>>> ??128?????? for (j = 0; j < si1.frame_count; j++) { >>>>>>>>>>>>>>>>>>>> ??129???????? if (si1.frame_buffer[j].method != si1.frame_buffer[j].method) { /* jvmtiFrameInfo::method */ >>>>>>>>>>>>>>>>>>>> ??130 snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>>>> ??131 "thread [%d] frame_buffer[%d].method is different: stack_info_1 = %lx, stack_info_2 = %lx", >>>>>>>>>>>>>>>>>>>> ??132??????????????????? i, j, si1.frame_buffer[j].method, si2.frame_buffer[j].method); >>>>>>>>>>>>>>>>>>>> ??133?????????? break; >>>>>>>>>>>>>>>>>>>> ??134???????? } else if (si1.frame_buffer[j].location != si1.frame_buffer[j].location) { /* jvmtiFrameInfo::location */ >>>>>>>>>>>>>>>>>>>> ??135 snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>>>> ??136 "thread [%d] frame_buffer[%d].location is different: stack_info_1 = %ld, stack_info_2 = %ld", >>>>>>>>>>>>>>>>>>>> ??137??????????????????? i, j, si1.frame_buffer[j].location, si2.frame_buffer[j].location); >>>>>>>>>>>>>>>>>>>> ??138?????????? break; >>>>>>>>>>>>>>>>>>>> ??139???????? } >>>>>>>>>>>>>>>>>>>> ??140?????? } >>>>>>>>>>>>>>>>>>>> ??141???? } >>>>>>>>>>>>>>>>>>>> ??142?? } >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Another step would be to create functions that implement a body of each loop. >>>>>>>>>>>>>>>>>>>> You can use the same techniques to simplify similar place (L127-L138) in the libOneGetThreadListStackTraces.c. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On 7/3/20 15:55, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> I'm not an Oracle employee, so I cannot know real request(s) from your customers. >>>>>>>>>>>>>>>>>>>>> However JDK-8201641 says Dynatrace has requested this enhancement. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> BTW I haven't heared any request from my customers about this. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On 2020/07/04 4:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> This difference is not that big to care about. >>>>>>>>>>>>>>>>>>>>>> I feel this is really rare case and so, does not worth these complications. >>>>>>>>>>>>>>>>>>>>>> Do we have a real request from customers to optimize it? >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> On 7/3/20 01:16, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Generally I agree with you, but I have concern about the difference of the result of GetStackTrace() and GetThreadListStackTraces(). >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ? GetStackTrace: jvmtiFrameInfo >>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces: jvmtiStackInfo >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> jvmtiStackInfo contains thread state, and it is ensured it is the state of the call stack. >>>>>>>>>>>>>>>>>>>>>>> If we want to get both call stack and thread state, we need to suspend target thread, and call both GetStackTrace() and GetThreadState(). Is it ok? >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I was wondering if JDK-8201641 (parent ticket of this change) needed them for profiling (dynatrace?) >>>>>>>>>>>>>>>>>>>>>>> If it is responsibility of JVMTI agent implementor, I remove this closure. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 16:45, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> After some thinking I've concluded that I do not like this optimization >>>>>>>>>>>>>>>>>>>>>>>> of the GetThreadListStackTraces with GetSingleStackTraceClosure. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> We may need more opinions on this but these are my points: >>>>>>>>>>>>>>>>>>>>>>>> ??- it adds some complexity and ugliness >>>>>>>>>>>>>>>>>>>>>>>> ??- a win is doubtful because it has to be a rare case, so that total overhead should not be high >>>>>>>>>>>>>>>>>>>>>>>> ??- if it is really high for some use cases then it is up to the user >>>>>>>>>>>>>>>>>>>>>>>> ??? to optimize it with using GetStackTrace instead >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> In such cases with doubtful overhead I usually prefer the simplicity. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Good examples where it makes sense to optimize are checks for target thread to be current thread. >>>>>>>>>>>>>>>>>>>>>>>> In such cases there is no need to suspend the target thread, or use a VMop/HandshakeClosure. >>>>>>>>>>>>>>>>>>>>>>>> For instance, please, see the Monitor functions with the check: (java_thread == calling_thread). >>>>>>>>>>>>>>>>>>>>>>>> Getting information for current thread is frequently used case, e.g. to get info at an event point. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On 7/2/20 23:29, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>> Hi Dan, David, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could you review again? >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/ >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> OneGetThreadListStackTraces.java in this webrev would wait until thread state is transited to "waiting" with spin wait. >>>>>>>>>>>>>>>>>>>>>>>>> CountDownLatch::await call as Dan pointed is fixed in it :) >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Diff from webrev.03: >>>>>>>>>>>>>>>>>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/c9aeb7001e50 >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 14:15, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 12:24, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>> On 7/2/20 10:50 PM, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry I'm responding here without seeing latest webrev but there is enough context I think ... >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Dan, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I upload new webrev. Could you review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L1542: // Get stack trace with handshake >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - please add a period at the end. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> L1591: *stack_info_ptr = op.stack_info(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? The return parameter should not be touched unless the return >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? old L1582: if (err == JVMTI_ERROR_NONE) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please restore this check. The return parameter should not >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? be touched unless the return code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> But op.stack_info() will return NULL if the error is not JVMTI_ERROR_NONE. Are you (Dan) concerned about someone passing in a non-null/initialized out-pointer that will be reset to NULL if there was an error? >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Actually the way we used to test this in POSIX tests is to call >>>>>>>>>>>>>>>>>>>>>>>>>>>> an API with known bad parameters and the return parameter ptr >>>>>>>>>>>>>>>>>>>>>>>>>>>> set to NULL. If the return parameter ptr was touched when an >>>>>>>>>>>>>>>>>>>>>>>>>>>> error should have been detected on an earlier parameter, then >>>>>>>>>>>>>>>>>>>>>>>>>>>> the test failed. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> L1272: ? if (!jt->is_exiting() && (thread_oop != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? old L1532: _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? This deletion of the _result field threw me for a minute and then >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? ? ? I figured out that the field is init to JVMTI_ERROR_THREAD_NOT_ALIVE >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? in the constructor. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L1553: if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L64: startSignal.countDown(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? I was expecting this to be a call to await() instead of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> countDown(). What am I missing here? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? I think this test might be passing by accident right now, but... >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Main thread (which call JVMTI functions to test) should wait until test thread is ready. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So main thread would wait startSignal, and test thread would count down. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> No! >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> The test thread that previously called obj.wait() now calls latch.await(). >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> The main thread that previously called obj.notify() now calls latch.countDown(). >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> The main thread continues to spin until it sees the target is WAITING before proceeding with the test. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> If I add spin wait to wait until transit target thread state is WAITING (as following), we don't need to call SuspendThread(). >>>>>>>>>>>>>>>>>>>>>>>>>>> Which is better? >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> The original spin-wait loop checking for WAITING is better because it is the only guarantee that the target thread is blocked where you need it to be. suspending the thread is racy as you don't know exactly where the suspend will hit. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>> /* Wait until the thread state transits to "waiting" */ >>>>>>>>>>>>>>>>>>>>>>>>>>> while (th.getState() != Thread.State.WAITING) { >>>>>>>>>>>>>>>>>>>>>>>>>>> Thread.onSpinWait(); >>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> For simplify, spin wait is prefer to OneGetThreadListStackTraces.java in webrev.03. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Here's the flow as I see it: >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - start worker thread >>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - startSignal.await() >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - startSignal.countDown() >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - stopSignal.await() >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - worker is now blocked >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - checkCallStacks(th) >>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - stopSignal.countDown() >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - worker is now unblocked >>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - th.join >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - runs off the end of run() >>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - run off the end of main() >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L92: jthreads = (jthread *)malloc(sizeof(jthread) * num_threads); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? You don't check for malloc() failure. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'jthreads' is allocated but never freed. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L91: result = (*jvmti)->SuspendThread(jvmti, thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? Why are you suspending the thread? GetAllStackTraces() and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() do not require the target thread(s) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? to be suspend. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? If you decide not to SuspendThread, then you don't need the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AddCapabilities or the ResumeThread calls. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Test thread might not be entered following code (stopSignal.await()). We might see deferent call stack between GetAllStackTraces() and GetThreadListStackTraces(). We cannot control to freeze call stack of test thread in Java code. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (I didn't use SuspendThread() at first, but I saw some errors which causes in above.) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So we need to call SuspendThread() to ensure we can see same call stack. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you are checking that the thread is in state WAITING then it cannot escape from that state and you can sample the stack multiple times from any API and get the same result. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I suspect the errors you saw were from the apparent incorrect use of the CountDownLatch. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> With the flow outlined above, the worker thread should be >>>>>>>>>>>>>>>>>>>>>>>>>>>> nicely blocked in stopSignal.await() when stuff is sampled. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Updates look fine - thanks. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> One minor nit: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1274 _collector.allocate_and_fill_stacks(1); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1275 _collector.set_result(JVMTI_ERROR_NONE); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the other places where you use _collector you rely on result being initialized to JVMTI_ERROR_NONE, rather than setting it directly after allocate_and_fill_stacks(). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 823 "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This function (JvmtiEnvBase::get_stack_trace()) can be called to get own stack trace. For example, we can call GetStackTrace() for current thread at JVMTI event. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So I changed assert as below: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??820 assert(current_thread == java_thread || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??821 SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??823 "call by myself / at safepoint / at handshake"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yep good catch. I hope current tests caught that. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> They would be tested in vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ (own call stacks), and getstacktr003 (call stacks in other thread). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Speaking of tests ... >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the native code I think you need to check the success of all JNI methods that can throw exceptions - otherwise I believe the tests may trigger warnings if -Xcheck:jni is used with them. See for example: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I updated testcases to check JNI and JVMTI function calls. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the Java code the target thread: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 45 public void run() { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 46 try { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 47 synchronized (lock) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 48 lock.wait(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 49 System.out.println("OK"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 50?????????? } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is potentially susceptible to spurious wakeups. Using a CountDownLatch would be robust. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If so, we can call make_local() in L1272 without JavaThread (or we can pass current thread to make_local()). Is it right? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry I got confused, _calling_thread may not be the current thread as we could be executing the handshake in the target thread itself. So the ResourceMark is correct as-is (implicitly for current thread). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The argument to fill_frames will be used in the jvmtiStackInfo and passed back to the _calling_thread, so it must be created via make_local(_calling_thread, ...) as you presently have. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you for reviewing! I will update new webrev tomorrow. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 500 JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??? : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I updated webrev for 8242428. Could you review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change migrate to use direct handshake for GetStackTrace() and GetThreadListStackTraces() (when thread_count == 1). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks really good now! I only have a few nits below. There is one thing I don't like about it but it requires a change to the main Handshake logic to address - in JvmtiEnv::GetThreadListStackTraces you have to create a ThreadsListHandle to convert the jthread to a JavaThread, but then the Handshake::execute_direct creates another ThreadsListHandle internally. That's a waste. I will discuss with Robbin and file a RFE to have an overload of execute_direct that takes an existing TLH. Actually it's worse than that because we have another TLH in use at the entry point for the JVMTI functions, so I think there may be some scope for simplifying the use of TLH instances - future RFE. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??451 GetStackTraceClosure(JvmtiEnv *env, jint start_depth, jint max_count, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??452 jvmtiFrameInfo* frame_buffer, jint* count_ptr) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??454 _env(env), _start_depth(start_depth), _max_count(max_count), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??455 _frame_buffer(frame_buffer), _count_ptr(count_ptr), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??456 _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Nit: can you do one initializer per line please. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??500 JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 481 MultipleStackTracesCollector(JvmtiEnv *env, jint max_frame_count) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??483 _max_frame_count = max_frame_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??484 _frame_count_total = 0; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??485 _head = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??486 _stack_info = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??487 _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As you are touching this can you change it to use an initializer list as you did for the HandshakeClosure, and please keep one item per line. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??823 "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1269 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is frustrating that this entire call chain started with a jthread reference, which we converted to a JavaThread, only to eventually need to convert it back to a jthread! I think there is some scope for simplification here but not as part of this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm still pondering the test logic but wanted to send this now. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetThreadListStackTrace (for GetThreadListStackTraces) and VM_GetAllStackTraces (for GetAllStackTraces) have inherited VM_GetMultipleStackTraces VM operation which provides the feature to generate jvmtiStackInfo. I modified VM_GetMultipleStackTraces to a normal C++ class to share with HandshakeClosure for GetThreadListStackTraces (GetSingleStackTraceClosure). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I added new testcases which test GetThreadListStackTraces() with thread_count == 1 and with all threads. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/nsk/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change replace following VM operations to direct handshake. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces (GetThreadListStackTrace()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTrace() uses direct handshake if thread count == 1. In other case (thread count > 1), it would be performed as VM operation (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Caller of VM_GetCurrentLocation (JvmtiEnvThreadState::reset_current_location()) might be called at safepoint. So I added safepoint check in its caller. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/ns >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I tested it on submit repo, then it has execution error (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) due to dependency error. So I think it does not occur by this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>> >>>>>> >>>>> >>> > From fairoz.matte at oracle.com Sat Jul 11 03:10:54 2020 From: fairoz.matte at oracle.com (Fairoz Matte) Date: Fri, 10 Jul 2020 20:10:54 -0700 (PDT) Subject: RFR(s): 8236042: [TESTBUG] serviceability/sa/ClhsdbCDSCore.java fails with -Xcomp -XX:TieredStopAtLevel=1 In-Reply-To: <37bb4585-21b5-a6e7-9ee1-88ccf9be0914@oracle.com> References: <2abe9fba-e958-4b34-9f92-6bb8d8478f4e@default> <70057c31-e535-f03a-391d-d181b2ec150b@oracle.com> <958fecdf-d7a1-4b22-835e-a75fadda0a84@default> <37bb4585-21b5-a6e7-9ee1-88ccf9be0914@oracle.com> Message-ID: Thanks Vladimir for the review. Thanks for mentioning the reasons for MDO's not being generated, I have added them as comment in bug for future reference. Thanks, Fairoz > -----Original Message----- > From: Vladimir Kozlov > Sent: Saturday, July 11, 2020 5:36 AM > To: Fairoz Matte ; Chris Plummer > ; hotspot-compiler-dev at openjdk.java.net; > serviceability-dev at openjdk.java.net > Subject: Re: RFR(s): 8236042: [TESTBUG] serviceability/sa/ClhsdbCDSCore.java > fails with -Xcomp -XX:TieredStopAtLevel=1 > > Fix is good. > > I think next are reasons you don't get MDO in this scenario. > > Tier1 (C1 compilation) does not generate profiling code and does not created > MDO. C1 request MDO only with tiers 2 and 3 [1][2]. > > With -Xcomp flag a Java method is not executed in Interpreter but requests its > compilation and waits when it is finished. As result MDO is not created in > Interpreter too. May be late if a method is deoptimized it will be executed in > Interpreter and MDO will be created. > > Thanks, > Vladimir > > [1] > http://hg.openjdk.java.net/jdk/jdk/file/796c9fa50850/src/hotspot/share/c1/c1_ > Compilation.hpp#l226 > [2] > http://hg.openjdk.java.net/jdk/jdk/file/796c9fa50850/src/hotspot/share/c1/c1_ > Compilation.cpp#l381 > > On 7/7/20 8:47 PM, Fairoz Matte wrote: > > Thanks Chris, for the review comments. > > > > I have updated the suggested change. > > > > Thanks, > > Fairoz > > > >> -----Original Message----- > >> From: Chris Plummer > >> Sent: Wednesday, July 8, 2020 3:38 AM > >> To: Fairoz Matte ; hotspot-compiler- > >> dev at openjdk.java.net; serviceability-dev at openjdk.java.net > >> Subject: Re: RFR(s): 8236042: [TESTBUG] > >> serviceability/sa/ClhsdbCDSCore.java > >> fails with -Xcomp -XX:TieredStopAtLevel=1 > >> > >> Hi Fairoz, > >> > >> Looks good, except for the missing space in "if(testJavaOpts...". > >> > >> thanks, > >> > >> Chris > >> > >> On 7/7/20 7:49 AM, Fairoz Matte wrote: > >>> Hi, > >>> > >>> Please review this small test change to consider the scenario when > >>> there is no > >> "printmdo" output > >>> > >>> JBS - https://bugs.openjdk.java.net/browse/JDK-8236042 > >>> Webrev - http://cr.openjdk.java.net/~fmatte/8236042/webrev.00/ > >>> > >>> Thanks, > >>> Fairoz > >> From serguei.spitsyn at oracle.com Sat Jul 11 05:51:25 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 10 Jul 2020 22:51:25 -0700 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: <8dfa047c-8f84-5775-d12d-3248137e4b45@oss.nttdata.com> References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <19cfa2a7-f441-425d-5623-1c11c6a870b6@oss.nttdata.com> <237e8d26-d185-1008-6e07-5b9f36314534@oracle.com> <2a9ba1f8-b1f7-cb8c-fe11-56dd174d9d58@oracle.com> <7ad907cb-24ef-aa55-972a-0a3f63576568@oracle.com> <5ab8c5d9-9a5e-74a6-4605-7e73732037bf@oracle.com> <4881e0da-f716-c62e-b744-17aa3d03e095@oss.nttdata.com> <8dfa047c-8f84-5775-d12d-3248137e4b45@oss.nttdata.com> Message-ID: <9a0eeb64-f05c-e266-db18-9e54fb7aa341@oracle.com> Hi Yasumasa, I'm okay with the update. Thanks, Serguei On 7/10/20 18:52, Yasumasa Suenaga wrote: > Thanks Dan! > > David, Serguei, are you ok to this change? > > > Yasumasa > > > On 2020/07/11 7:00, Daniel D. Daugherty wrote: >> On 7/10/20 3:29 AM, Yasumasa Suenaga wrote: >>> Thanks Patricio! >>> >>> I uploaded new webrev. Could you review again? >>> It passed all tests on submit repo, and passed jtreg tests about JVMTI. >>> >>> ? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.10/ >>> ? Diff from webrev.09: >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.10/incremental/ >> >> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >> >> ???? No comments. >> >> Thumbs up on the incremental. >> >> Dan >> >> >>> >>> >>> Yasumasa >>> >>> >>> On 2020/07/10 14:49, Patricio Chilano wrote: >>>> Hi Yasumasa, >>>> >>>> On 7/10/20 1:58 AM, Yasumasa Suenaga wrote: >>>>> Hi Patricio, >>>>> >>>>> Thanks for your advice! >>>>> I've fixed testcase as [1], but I still see an error in >>>>> validate-headers-linux-x64-build-1 on submit repo. >>>>> What does it mean? Can you share details? >>>>> >>>>> ? mach5-one-ysuenaga-JDK-8242428-20200710-0339-12529134 >>>> Not your bug, you are getting the header validation failure from >>>> 8248570. It was fixed two days ago so just update the repo and it >>>> should work now. >>>> >>>> Thanks, >>>> Patricio >>>>> Of course this change can be built on my Linux box (Fedora 32, >>>>> AMD64, GCC 10.1) >>>>> >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> [1] >>>>> http://hg.openjdk.java.net/jdk/submit/file/45f52142db9d/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>> >>>>> >>>>> On 2020/07/10 2:18, Patricio Chilano wrote: >>>>>> >>>>>> On 7/9/20 12:00 PM, Patricio Chilano wrote: >>>>>>> Hi Yasumasa, >>>>>>> >>>>>>> On 7/9/20 9:30 AM, Yasumasa Suenaga wrote: >>>>>>>> On 2020/07/09 17:58, David Holmes wrote: >>>>>>>>> Hi Yasumasa, >>>>>>>>> >>>>>>>>> On 9/07/2020 10:25 am, Yasumasa Suenaga wrote: >>>>>>>>>> Hi Dan, >>>>>>>>>> >>>>>>>>>> Thanks for your comment! >>>>>>>>>> I uploaded new webrev: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.09/ >>>>>>>>>> ?? Diff from previous webrev: >>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/5d167adf8524 >>>>>>>>>> >>>>>>>>>> I saw similar build errors in >>>>>>>>>> libOneGetThreadListStackTraces.cpp on Windows. >>>>>>>>>> This webrev fixes them. >>>>>>>>> >>>>>>>>> You shouldn't use %p as it may not be portable. In the VM we >>>>>>>>> use INTPTR_FORMAT and convert the arg using p2i. I don't know >>>>>>>>> what exists in the testing code. >>>>>>>> >>>>>>>> I replaced %p to %lx, and also cast values to unsigned long [1] >>>>>>>> [2], but the test on submit repo was failed. >>>>>>>> Can anyone share details of >>>>>>>> mach5-one-ysuenaga-JDK-8242428-20200709-1030-12500928 ? >>>>>>> These are the errors I see for the macOS build: >>>>>>> >>>>>>> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:53:14: >>>>>>> error: format specifies type 'long long' but the argument has >>>>>>> type 'jlocation' (aka 'long') [-Werror,-Wformat] >>>>>>> ????????????? fi1->location, fi2->location); >>>>>>> ????????????? ^~~~~~~~~~~~~ >>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:53:29: >>>>>>> error: format specifies type 'long long' but the argument has >>>>>>> type 'jlocation' (aka 'long') [-Werror,-Wformat] >>>>>>> ????????????? fi1->location, fi2->location); >>>>>>> ???????????????????????????? ^~~~~~~~~~~~~ >>>>>>> >>>>>>> These are the ones I see for the Windows build: >>>>>>> >>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): >>>>>>> warning C4311: 'type cast': pointer truncation from 'jmethodID' >>>>>>> to 'unsigned long' >>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): >>>>>>> warning C4302: 'type cast': truncation from 'jmethodID' to >>>>>>> 'unsigned long' >>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): >>>>>>> warning C4311: 'type cast': pointer truncation from 'jmethodID' >>>>>>> to 'unsigned long' >>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): >>>>>>> warning C4302: 'type cast': truncation from 'jmethodID' to >>>>>>> 'unsigned long' >>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): >>>>>>> warning C4311: 'type cast': pointer truncation from 'jthread' to >>>>>>> 'unsigned long' >>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): >>>>>>> warning C4302: 'type cast': truncation from 'jthread' to >>>>>>> 'unsigned long' >>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): >>>>>>> warning C4311: 'type cast': pointer truncation from 'jthread' to >>>>>>> 'unsigned long' >>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): >>>>>>> warning C4302: 'type cast': truncation from 'jthread' to >>>>>>> 'unsigned long' >>>>>>> >>>>>>> >>>>>>> You will probably want to use the macros defined in >>>>>>> src/hotspot/share/utilities/globalDefinitions.hpp. Let me know >>>>>>> if you need me to test something. >>>>>> With these changes the build works okay on Linux, Windows and macOS: >>>>>> >>>>>> >>>>>> --- >>>>>> a/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>> +++ >>>>>> b/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>> @@ -27,4 +27,5 @@ >>>>>> ??#include >>>>>> ??#include >>>>>> +#include >>>>>> >>>>>> ??#define MAX_FRAMES 100 >>>>>> @@ -45,11 +46,11 @@ >>>>>> ??? if (fi1->method != fi2->method) { /* jvmtiFrameInfo::method */ >>>>>> ????? snprintf(err_msg, sizeof(err_msg), >>>>>> -???????????? "method is different: fi1 = %p, fi2 = %p", >>>>>> -???????????? fi1->method, fi2->method); >>>>>> +???????????? "method is different: fi1 = 0x%016" PRIxPTR " , fi2 >>>>>> = 0x%016" PRIxPTR, >>>>>> +???????????? (intptr_t)fi1->method, (intptr_t)fi2->method); >>>>>> ????? env->FatalError(err_msg); >>>>>> ??? } else if (fi1->location != fi2->location) { /* >>>>>> jvmtiFrameInfo::location */ >>>>>> ????? snprintf(err_msg, sizeof(err_msg), >>>>>> -???????????? "location is different: fi1 = %lld, fi2 = %lld", >>>>>> -???????????? fi1->location, fi2->location); >>>>>> +???????????? "location is different: fi1 = %" PRId64 " , fi2 = >>>>>> %" PRId64, >>>>>> +???????????? (int64_t)fi1->location, (int64_t)fi2->location); >>>>>> ????? env->FatalError(err_msg); >>>>>> ??? } >>>>>> @@ -67,5 +68,5 @@ >>>>>> ??? if (!is_same) { /* jvmtiStackInfo::thread */ >>>>>> ????? snprintf(err_msg, sizeof(err_msg), >>>>>> -???????????? "thread is different: si1 = %p, si2 = %p", >>>>>> si1->thread, si2->thread); >>>>>> +???????????? "thread is different: si1 = 0x%016" PRIxPTR " , si2 >>>>>> = 0x%016" PRIxPTR, (intptr_t)si1->thread, (intptr_t)si2->thread); >>>>>> ????? env->FatalError(err_msg); >>>>>> ??? } else if (si1->state != si2->state) { /* >>>>>> jvmtiStackInfo::state */ >>>>>> >>>>>> >>>>>> Maybe you can use something like that. >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Patricio >>>>>>> Thanks, >>>>>>> Patricio >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> [1] http://hg.openjdk.java.net/jdk/submit/rev/dfca51958217 >>>>>>>> [2] http://hg.openjdk.java.net/jdk/submit/rev/3665361fa91b >>>>>>>> >>>>>>>> >>>>>>>>> David >>>>>>>>> ----- >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2020/07/09 1:42, Daniel D. Daugherty wrote: >>>>>>>>>>> ?> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ >>>>>>>>>>> >>>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>>> ???? No comments. >>>>>>>>>>> >>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>> ???? L1159: ? Thread *current_thread = Thread::current(); >>>>>>>>>>> ???????? Please add "#ifdef ASSERT" above and "#endif" below >>>>>>>>>>> since >>>>>>>>>>> ???????? current_thread is only used for the assert() in >>>>>>>>>>> this function. >>>>>>>>>>> >>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>> ???? L549: jthread java_thread, jint max_frame_count) >>>>>>>>>>> ???? L552: ????? _jthread(java_thread), >>>>>>>>>>> ???????? Please: s/java_thread/thread/ on both lines. >>>>>>>>>>> >>>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>>> ???? No comments. >>>>>>>>>>> >>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>>> >>>>>>>>>>> ???? No comments. >>>>>>>>>>> >>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>>>>>>> >>>>>>>>>>> ???? L27: #include >>>>>>>>>>> ???????? This include is out of order; should be first in >>>>>>>>>>> the list. >>>>>>>>>>> >>>>>>>>>>> ???? This file doesn't compile on my MBP13: >>>>>>>>>>> >>>>>>>>>>> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:49:14: >>>>>>>>>>> error: format specifies type 'unsigned long' but the >>>>>>>>>>> argument has type 'jmethodID' (aka '_jmethodID *') >>>>>>>>>>> [-Werror,-Wformat] >>>>>>>>>>> ????????????? fi1->method, fi2->method); >>>>>>>>>>> ????????????? ^~~~~~~~~~~ >>>>>>>>>>> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:49:27: >>>>>>>>>>> error: format specifies type 'unsigned long' but the >>>>>>>>>>> argument has type 'jmethodID' (aka '_jmethodID *') >>>>>>>>>>> [-Werror,-Wformat] >>>>>>>>>>> ????????????? fi1->method, fi2->method); >>>>>>>>>>> ?????????????????????????? ^~~~~~~~~~~ >>>>>>>>>>> ???? 2 errors generated. >>>>>>>>>>> >>>>>>>>>>> ???? This change made it compile on my MBP13, but that may >>>>>>>>>>> break it on >>>>>>>>>>> ???? other platforms: >>>>>>>>>>> >>>>>>>>>>> ???? $ hg diff >>>>>>>>>>> ???? diff -r 560847c69fbe >>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>>>>>>> ???? --- >>>>>>>>>>> a/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>>>>>>> Wed Jul 08 12:13:32 2020 -0400 >>>>>>>>>>> ???? +++ >>>>>>>>>>> b/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>>>>>>> Wed Jul 08 12:40:42 2020 -0400 >>>>>>>>>>> ???? @@ -46,7 +46,7 @@ >>>>>>>>>>> ?? ?? ? if (fi1->method != fi2->method) { /* >>>>>>>>>>> jvmtiFrameInfo::method */ >>>>>>>>>>> ?? ?? ??? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>> ?? ? ? ??????????? "method is different: fi1 = %lx, fi2 = %lx", >>>>>>>>>>> ???? -???????????? fi1->method, fi2->method); >>>>>>>>>>> ???? +???????????? (unsigned long) fi1->method, (unsigned >>>>>>>>>>> long) fi2->method); >>>>>>>>>>> ?? ?? ??? env->FatalError(err_msg); >>>>>>>>>>> ?? ?? ? } else if (fi1->location != fi2->location) { /* >>>>>>>>>>> jvmtiFrameInfo::location */ >>>>>>>>>>> ?? ?? ??? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>> >>>>>>>>>>> ???? I'm not sure of the right platform independent way to >>>>>>>>>>> output >>>>>>>>>>> ???? the 'method' field. >>>>>>>>>>> >>>>>>>>>>> Dan >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 7/8/20 4:04 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>> Hi David, >>>>>>>>>>>> >>>>>>>>>>>> On 2020/07/08 15:27, David Holmes wrote: >>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>> >>>>>>>>>>>>> On 7/07/2020 6:54 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Serguei, thank you for replying even though you are on >>>>>>>>>>>>>> vacaiton! >>>>>>>>>>>>>> >>>>>>>>>>>>>> I uploaded new webrev: >>>>>>>>>>>>>> >>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.07/ >>>>>>>>>>>>>> ?? Diff from previous webrev: >>>>>>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/77243b1dcbfe >>>>>>>>>>>>>> >>>>>>>>>>>>>> c'tor of GetSingleStackTraceClosure has jthread argument >>>>>>>>>>>>>> in this webrev. >>>>>>>>>>>>>> Also it does not contain testcase for >>>>>>>>>>>>>> GetThreadListStackTraces with all threads, and >>>>>>>>>>>>>> OneGetThreadListStackTraces would test main thread only. >>>>>>>>>>>>> >>>>>>>>>>>>> All those changes are fine in principle for me. One >>>>>>>>>>>>> nit/suggestion: >>>>>>>>>>>>> >>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>> >>>>>>>>>>>>> ??544?? jthread _java_thread; >>>>>>>>>>>>> >>>>>>>>>>>>> elsewhere "java_thread" refers to a JavaThread, so to >>>>>>>>>>>>> avoid confusion may I suggest this member be named _jthread. >>>>>>>>>>>> >>>>>>>>>>>> I uploaded new webrev: >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ >>>>>>>>>>>> ? Diff from previous webrev: >>>>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/ca6263dbdc87 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> I'm going to be away for the next couple of days - sorry - >>>>>>>>>>>>> but will try to check email on this if I can. >>>>>>>>>>>> >>>>>>>>>>>> Thanks! >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> David >>>>>>>>>>>>> ----- >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 2020/07/07 15:13, David Holmes wrote: >>>>>>>>>>>>>>> On 7/07/2020 2:57 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 2020/07/07 11:31, David Holmes wrote: >>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Hard to keep up with the changes - especially without >>>>>>>>>>>>>>>>> incremental webrevs. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Sorry, I will upload diff from previous webrev in the >>>>>>>>>>>>>>>> next. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> If GetSingleStackTraceClosure also took the jthread as >>>>>>>>>>>>>>>>> a constructor arg, then you wouldn't need to recreate >>>>>>>>>>>>>>>>> a JNI local handle when calling >>>>>>>>>>>>>>>>> _collector.fill_frames. It's a small simplification >>>>>>>>>>>>>>>>> and not essential at this stage. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I think we should get jthread from an argument of >>>>>>>>>>>>>>>> do_thread() because do_thread() would pass the thread >>>>>>>>>>>>>>>> which are stopped certainly. >>>>>>>>>>>>>>>> It might be simplification if we pass _calling_thread >>>>>>>>>>>>>>>> to MultipleStackTracesCollector. `jthread` is only >>>>>>>>>>>>>>>> needed to store jvmtiStackInfo.thread . What do you think? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I'm not quite sure what you mean. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I think there is a bit of a design wart with direct >>>>>>>>>>>>>>> handshakes in that do_thread takes the target JavaThread >>>>>>>>>>>>>>> as an argument. That's useful in a case where you want a >>>>>>>>>>>>>>> HandshakeClosure that can be applied to multiple >>>>>>>>>>>>>>> threads, but that's not typically what is needed with >>>>>>>>>>>>>>> direct handshakes - there is only a single target. With >>>>>>>>>>>>>>> a single-target HandshakeClosure you can capture all the >>>>>>>>>>>>>>> "target" information for the operation in the closure >>>>>>>>>>>>>>> instance. So if the actual do_thread operation wants the >>>>>>>>>>>>>>> jthread corresponding to the target thread then we can >>>>>>>>>>>>>>> store that in the closure rather than recomputing it >>>>>>>>>>>>>>> (you could assert it is the same but that seems overkill >>>>>>>>>>>>>>> to me). >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> For the test ... I don't see how >>>>>>>>>>>>>>>>> Java_GetThreadListStackTraces_checkCallStacks is a >>>>>>>>>>>>>>>>> valid test. It gets the stacks of all live threads, >>>>>>>>>>>>>>>>> then uses that information to use >>>>>>>>>>>>>>>>> GetThreadListStackTraces to get the stack for the same >>>>>>>>>>>>>>>>> set of threads through a different API. It then >>>>>>>>>>>>>>>>> compares the two sets of stacks for each thread >>>>>>>>>>>>>>>>> expecting them to be the same, but that need only be >>>>>>>>>>>>>>>>> the case for the main thread. Other threads could >>>>>>>>>>>>>>>>> potentially have a different stack (e.g. if this test >>>>>>>>>>>>>>>>> is run with JFR enabled there will be additional >>>>>>>>>>>>>>>>> threads found.) Further I would have expected that >>>>>>>>>>>>>>>>> there already exist tests that check that, for a given >>>>>>>>>>>>>>>>> thread (which may be suspended or known to be blocked) >>>>>>>>>>>>>>>>> the same stack is found through the two different APIs. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti/unit/GetAllStackTraces/getallstktr001 >>>>>>>>>>>>>>>> would check all of threads via >>>>>>>>>>>>>>>> GetThreadListStackTraces() and GetAllStackTraces(), so >>>>>>>>>>>>>>>> we might be able to remove >>>>>>>>>>>>>>>> GetThreadListStackTraces.java from this webrev. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yes. The existing test only examines a set of test >>>>>>>>>>>>>>> threads that are all blocked on a raw monitor. You do >>>>>>>>>>>>>>> not need to duplicate that test. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> OTOH we don't have testcase for >>>>>>>>>>>>>>>> GetThreadListStackTraces() with thread_count == 1, so >>>>>>>>>>>>>>>> we need to testcase for it (it is >>>>>>>>>>>>>>>> OneGetThreadListStackTraces.java) It would check >>>>>>>>>>>>>>>> whether the state of target thread is "waiting" before >>>>>>>>>>>>>>>> JNI call to call GetThreadListStackTraces(), >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yes we need to test the special cases introduced by your >>>>>>>>>>>>>>> changes - totally agree - and >>>>>>>>>>>>>>> OneGetThreadListStackTraces.java is a good test for that. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> and also I expect it would not be run with JFR. (it is >>>>>>>>>>>>>>>> not described @run) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The arguments to run with JFR (or a bunch of other >>>>>>>>>>>>>>> things) can be passed to the jtreg test harness to be >>>>>>>>>>>>>>> applied to all tests. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Of course we can check GetThreadListStackTraces() with >>>>>>>>>>>>>>>> main thread, but it is not the test for direct >>>>>>>>>>>>>>>> handshake for other thread. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Right - that test already exists as per the above. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> David >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 6/07/2020 11:29 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I think C++ is more simple to implement the test >>>>>>>>>>>>>>>>>> agent as you said. >>>>>>>>>>>>>>>>>> So I implement it in C++ in new webrev. Could you >>>>>>>>>>>>>>>>>> review again? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.06/ >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Also I refactored libGetThreadListStackTraces.cpp, >>>>>>>>>>>>>>>>>> and I've kept exception check after IsSameObject(). >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 2020/07/06 16:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thank you for the update. >>>>>>>>>>>>>>>>>>> I think, a pending exception after IsSameObject >>>>>>>>>>>>>>>>>>> needs to be checked. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> The checkStackInfo() needs one more refactoring as >>>>>>>>>>>>>>>>>>> I've already suggested. >>>>>>>>>>>>>>>>>>> The body of the loop at L68-L78 should be converted >>>>>>>>>>>>>>>>>>> to a function check_frame_info. >>>>>>>>>>>>>>>>>>> The si1->frame_buffer[i] and si2->frame_buffer[i] >>>>>>>>>>>>>>>>>>> will be passed as fi1 and fi2. >>>>>>>>>>>>>>>>>>> The index can be passed as well. >>>>>>>>>>>>>>>>>>> I'm still suggesting to simplify the local >>>>>>>>>>>>>>>>>>> exception_msg to something shorter like err_msg or >>>>>>>>>>>>>>>>>>> exc_msg. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I'm not sure using fatal is right here: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> This fragment looks strange: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ? 152???? if ((*env)->IsSameObject(env, >>>>>>>>>>>>>>>>>>> stack_info[i].thread, thread)) { >>>>>>>>>>>>>>>>>>> ? 153?????? target_info = &stack_info[i]; >>>>>>>>>>>>>>>>>>> ? 154?????? break; >>>>>>>>>>>>>>>>>>> ? 155???? } else if ((*env)->ExceptionOccurred(env)) { >>>>>>>>>>>>>>>>>>> ? 156 (*env)->ExceptionDescribe(env); >>>>>>>>>>>>>>>>>>> ? 157 (*env)->FatalError(env, __FILE__); >>>>>>>>>>>>>>>>>>> ? 158???? } >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I expected it to be: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ??? jboolean same = (*env)->IsSameObject(env, >>>>>>>>>>>>>>>>>>> stack_info[i].thread, thread); >>>>>>>>>>>>>>>>>>> ??? if ((*env)->ExceptionOccurred(env)) { >>>>>>>>>>>>>>>>>>> (*env)->ExceptionDescribe(env); >>>>>>>>>>>>>>>>>>> (*env)->FatalError(env, __FILE__); >>>>>>>>>>>>>>>>>>> ??? } >>>>>>>>>>>>>>>>>>> ??? if (same) { >>>>>>>>>>>>>>>>>>> ????? target_info = &stack_info[i]; >>>>>>>>>>>>>>>>>>> ????? break; >>>>>>>>>>>>>>>>>>> ??? } >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Would it better to port this agent to C++ to >>>>>>>>>>>>>>>>>>> simplify this code nicer? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On 7/5/20 06:13, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>>>>> I refactored testcase. Could you review again? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.05/ >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> It would check Java exception after IsSameObject() >>>>>>>>>>>>>>>>>>>> call. Does it need? >>>>>>>>>>>>>>>>>>>> Any exceptions are not described in JNI >>>>>>>>>>>>>>>>>>>> document[1], and JNI implementation >>>>>>>>>>>>>>>>>>>> (jni_IsSameObject()) does not seem to throw it. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>>>>> https://docs.oracle.com/en/java/javase/14/docs/specs/jni/functions.html#issameobject >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On 2020/07/05 14:46, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Okay, thanks. >>>>>>>>>>>>>>>>>>>>> Then I'm okay to keep the GetSingleStackTraceClosure. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c.html >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c.html >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> I'm not sure the function 'is_same_thread() is >>>>>>>>>>>>>>>>>>>>> needed. >>>>>>>>>>>>>>>>>>>>> Why do not use the JNI IsSameObject instead? >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> It seems to be a typo at L132 and L137. >>>>>>>>>>>>>>>>>>>>> You, probably. did not want to print the same >>>>>>>>>>>>>>>>>>>>> information for >>>>>>>>>>>>>>>>>>>>> stack_info_1[i].frame_buffer[j].XXX twice. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> The code at lines 112-142 is not readable. >>>>>>>>>>>>>>>>>>>>> I'd suggest to make a couple of refactoring steps. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> First step to simplify this a little bit would be >>>>>>>>>>>>>>>>>>>>> with some renaming and getting rid of indexes: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ?? 71 char err_msg[EXCEPTION_MSG_LEN] = {0}; >>>>>>>>>>>>>>>>>>>>> ??... >>>>>>>>>>>>>>>>>>>>> ??112?? /* Iterate all jvmtiStackInfo to check */ >>>>>>>>>>>>>>>>>>>>> ??113?? for (i = 0; i < num_threads, >>>>>>>>>>>>>>>>>>>>> *exception_msg != '\0'; i++) { >>>>>>>>>>>>>>>>>>>>> ????????? jvmtiStackInfo *si1 = stack_info_1[i]; >>>>>>>>>>>>>>>>>>>>> ????????? jvmtiStackInfo *si2 = stack_info_2[i]; >>>>>>>>>>>>>>>>>>>>> ??114???? if (!IsSameObject(env, si1.thread, >>>>>>>>>>>>>>>>>>>>> si2.thread)) { /* jvmtiStackInfo::thread */ >>>>>>>>>>>>>>>>>>>>> ??115 snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>>>>> ??116 "thread[%d] is different: stack_info_1 = %p, >>>>>>>>>>>>>>>>>>>>> stack_info_2 = %p", >>>>>>>>>>>>>>>>>>>>> ??117??????????????? i, sinfo1.thread, >>>>>>>>>>>>>>>>>>>>> sinfo2.thread); >>>>>>>>>>>>>>>>>>>>> ??118???? } else if (si1.state != si2.state) { /* >>>>>>>>>>>>>>>>>>>>> jvmtiStackInfo::state */ >>>>>>>>>>>>>>>>>>>>> ??119 snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>>>>> ??120 "state[%d] is different: stack_info_1 = %d, >>>>>>>>>>>>>>>>>>>>> stack_info_2 = %d", >>>>>>>>>>>>>>>>>>>>> ??121??????????????? i, si1.state, si2.state); >>>>>>>>>>>>>>>>>>>>> ??122???? } else if (si1.frame_count != >>>>>>>>>>>>>>>>>>>>> si2.frame_count) { /* jvmtiStackInfo::frame_count */ >>>>>>>>>>>>>>>>>>>>> ??123 snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>>>>> ??124 "frame_count[%d] is different: stack_info_1 >>>>>>>>>>>>>>>>>>>>> = %d, stack_info_2 = %d", >>>>>>>>>>>>>>>>>>>>> ??125??????????????? i, si1.frame_count, >>>>>>>>>>>>>>>>>>>>> si2.frame_count); >>>>>>>>>>>>>>>>>>>>> ??126???? } else { >>>>>>>>>>>>>>>>>>>>> ??127?????? /* Iterate all jvmtiFrameInfo to check */ >>>>>>>>>>>>>>>>>>>>> ??128?????? for (j = 0; j < si1.frame_count; j++) { >>>>>>>>>>>>>>>>>>>>> ??129???????? if (si1.frame_buffer[j].method != >>>>>>>>>>>>>>>>>>>>> si1.frame_buffer[j].method) { /* >>>>>>>>>>>>>>>>>>>>> jvmtiFrameInfo::method */ >>>>>>>>>>>>>>>>>>>>> ??130 snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>>>>> ??131 "thread [%d] frame_buffer[%d].method is >>>>>>>>>>>>>>>>>>>>> different: stack_info_1 = %lx, stack_info_2 = %lx", >>>>>>>>>>>>>>>>>>>>> ??132 i, j, si1.frame_buffer[j].method, >>>>>>>>>>>>>>>>>>>>> si2.frame_buffer[j].method); >>>>>>>>>>>>>>>>>>>>> ??133?????????? break; >>>>>>>>>>>>>>>>>>>>> ??134???????? } else if >>>>>>>>>>>>>>>>>>>>> (si1.frame_buffer[j].location != >>>>>>>>>>>>>>>>>>>>> si1.frame_buffer[j].location) { /* >>>>>>>>>>>>>>>>>>>>> jvmtiFrameInfo::location */ >>>>>>>>>>>>>>>>>>>>> ??135 snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>>>>> ??136 "thread [%d] frame_buffer[%d].location is >>>>>>>>>>>>>>>>>>>>> different: stack_info_1 = %ld, stack_info_2 = %ld", >>>>>>>>>>>>>>>>>>>>> ??137 i, j, si1.frame_buffer[j].location, >>>>>>>>>>>>>>>>>>>>> si2.frame_buffer[j].location); >>>>>>>>>>>>>>>>>>>>> ??138?????????? break; >>>>>>>>>>>>>>>>>>>>> ??139???????? } >>>>>>>>>>>>>>>>>>>>> ??140?????? } >>>>>>>>>>>>>>>>>>>>> ??141???? } >>>>>>>>>>>>>>>>>>>>> ??142?? } >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Another step would be to create functions that >>>>>>>>>>>>>>>>>>>>> implement a body of each loop. >>>>>>>>>>>>>>>>>>>>> You can use the same techniques to simplify >>>>>>>>>>>>>>>>>>>>> similar place (L127-L138) in the >>>>>>>>>>>>>>>>>>>>> libOneGetThreadListStackTraces.c. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On 7/3/20 15:55, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I'm not an Oracle employee, so I cannot know real >>>>>>>>>>>>>>>>>>>>>> request(s) from your customers. >>>>>>>>>>>>>>>>>>>>>> However JDK-8201641 says Dynatrace has requested >>>>>>>>>>>>>>>>>>>>>> this enhancement. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> BTW I haven't heared any request from my >>>>>>>>>>>>>>>>>>>>>> customers about this. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> On 2020/07/04 4:32, serguei.spitsyn at oracle.com >>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> This difference is not that big to care about. >>>>>>>>>>>>>>>>>>>>>>> I feel this is really rare case and so, does not >>>>>>>>>>>>>>>>>>>>>>> worth these complications. >>>>>>>>>>>>>>>>>>>>>>> Do we have a real request from customers to >>>>>>>>>>>>>>>>>>>>>>> optimize it? >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On 7/3/20 01:16, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Generally I agree with you, but I have concern >>>>>>>>>>>>>>>>>>>>>>>> about the difference of the result of >>>>>>>>>>>>>>>>>>>>>>>> GetStackTrace() and GetThreadListStackTraces(). >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> ? GetStackTrace: jvmtiFrameInfo >>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces: jvmtiStackInfo >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> jvmtiStackInfo contains thread state, and it is >>>>>>>>>>>>>>>>>>>>>>>> ensured it is the state of the call stack. >>>>>>>>>>>>>>>>>>>>>>>> If we want to get both call stack and thread >>>>>>>>>>>>>>>>>>>>>>>> state, we need to suspend target thread, and >>>>>>>>>>>>>>>>>>>>>>>> call both GetStackTrace() and GetThreadState(). >>>>>>>>>>>>>>>>>>>>>>>> Is it ok? >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> I was wondering if JDK-8201641 (parent ticket >>>>>>>>>>>>>>>>>>>>>>>> of this change) needed them for profiling >>>>>>>>>>>>>>>>>>>>>>>> (dynatrace?) >>>>>>>>>>>>>>>>>>>>>>>> If it is responsibility of JVMTI agent >>>>>>>>>>>>>>>>>>>>>>>> implementor, I remove this closure. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 16:45, serguei.spitsyn at oracle.com >>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> After some thinking I've concluded that I do >>>>>>>>>>>>>>>>>>>>>>>>> not like this optimization >>>>>>>>>>>>>>>>>>>>>>>>> of the GetThreadListStackTraces with >>>>>>>>>>>>>>>>>>>>>>>>> GetSingleStackTraceClosure. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> We may need more opinions on this but these >>>>>>>>>>>>>>>>>>>>>>>>> are my points: >>>>>>>>>>>>>>>>>>>>>>>>> ??- it adds some complexity and ugliness >>>>>>>>>>>>>>>>>>>>>>>>> ??- a win is doubtful because it has to be a >>>>>>>>>>>>>>>>>>>>>>>>> rare case, so that total overhead should not >>>>>>>>>>>>>>>>>>>>>>>>> be high >>>>>>>>>>>>>>>>>>>>>>>>> ??- if it is really high for some use cases >>>>>>>>>>>>>>>>>>>>>>>>> then it is up to the user >>>>>>>>>>>>>>>>>>>>>>>>> ??? to optimize it with using GetStackTrace >>>>>>>>>>>>>>>>>>>>>>>>> instead >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> In such cases with doubtful overhead I usually >>>>>>>>>>>>>>>>>>>>>>>>> prefer the simplicity. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Good examples where it makes sense to optimize >>>>>>>>>>>>>>>>>>>>>>>>> are checks for target thread to be current >>>>>>>>>>>>>>>>>>>>>>>>> thread. >>>>>>>>>>>>>>>>>>>>>>>>> In such cases there is no need to suspend the >>>>>>>>>>>>>>>>>>>>>>>>> target thread, or use a VMop/HandshakeClosure. >>>>>>>>>>>>>>>>>>>>>>>>> For instance, please, see the Monitor >>>>>>>>>>>>>>>>>>>>>>>>> functions with the check: (java_thread == >>>>>>>>>>>>>>>>>>>>>>>>> calling_thread). >>>>>>>>>>>>>>>>>>>>>>>>> Getting information for current thread is >>>>>>>>>>>>>>>>>>>>>>>>> frequently used case, e.g. to get info at an >>>>>>>>>>>>>>>>>>>>>>>>> event point. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On 7/2/20 23:29, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> Hi Dan, David, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could you review again? >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/ >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> OneGetThreadListStackTraces.java in this >>>>>>>>>>>>>>>>>>>>>>>>>> webrev would wait until thread state is >>>>>>>>>>>>>>>>>>>>>>>>>> transited to "waiting" with spin wait. >>>>>>>>>>>>>>>>>>>>>>>>>> CountDownLatch::await call as Dan pointed is >>>>>>>>>>>>>>>>>>>>>>>>>> fixed in it :) >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Diff from webrev.03: >>>>>>>>>>>>>>>>>>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/c9aeb7001e50 >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 14:15, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 12:24, Daniel D. Daugherty >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 7/2/20 10:50 PM, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry I'm responding here without seeing >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> latest webrev but there is enough context >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think ... >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 3/07/2020 9:14 am, Yasumasa Suenaga >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Dan, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 7:16, Daniel D. Daugherty >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I upload new webrev. Could you review >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L1542: // Get stack trace with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> handshake >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - please add a period at >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the end. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> L1591: *stack_info_ptr = op.stack_info(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? The return parameter should >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not be touched unless the return >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? code in 'err' == >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? old L1582: if (err == >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JVMTI_ERROR_NONE) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please restore this check. The return >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> parameter should not >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? be touched unless the return >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But op.stack_info() will return NULL if >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the error is not JVMTI_ERROR_NONE. Are >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you (Dan) concerned about someone passing >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in a non-null/initialized out-pointer >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that will be reset to NULL if there was >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an error? >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Actually the way we used to test this in >>>>>>>>>>>>>>>>>>>>>>>>>>>>> POSIX tests is to call >>>>>>>>>>>>>>>>>>>>>>>>>>>>> an API with known bad parameters and the >>>>>>>>>>>>>>>>>>>>>>>>>>>>> return parameter ptr >>>>>>>>>>>>>>>>>>>>>>>>>>>>> set to NULL. If the return parameter ptr >>>>>>>>>>>>>>>>>>>>>>>>>>>>> was touched when an >>>>>>>>>>>>>>>>>>>>>>>>>>>>> error should have been detected on an >>>>>>>>>>>>>>>>>>>>>>>>>>>>> earlier parameter, then >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the test failed. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> L1272: ? if (!jt->is_exiting() && >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (thread_oop != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> second expression. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? old L1532: _result = >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? This deletion of the _result >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> field threw me for a minute and then >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? ? ? I figured out that the field >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is init to JVMTI_ERROR_THREAD_NOT_ALIVE >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? in the constructor. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L1553: if (!jt->is_exiting() && >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> second expression. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L64: startSignal.countDown(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? I was expecting this to be a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call to await() instead of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> countDown(). What am I missing here? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? I think this test might be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> passing by accident right now, but... >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Main thread (which call JVMTI functions >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to test) should wait until test thread >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is ready. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So main thread would wait startSignal, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and test thread would count down. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No! >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The test thread that previously called >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> obj.wait() now calls latch.await(). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The main thread that previously called >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> obj.notify() now calls latch.countDown(). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The main thread continues to spin until >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it sees the target is WAITING before >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> proceeding with the test. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> If I add spin wait to wait until transit >>>>>>>>>>>>>>>>>>>>>>>>>>>> target thread state is WAITING (as >>>>>>>>>>>>>>>>>>>>>>>>>>>> following), we don't need to call >>>>>>>>>>>>>>>>>>>>>>>>>>>> SuspendThread(). >>>>>>>>>>>>>>>>>>>>>>>>>>>> Which is better? >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> The original spin-wait loop checking for >>>>>>>>>>>>>>>>>>>>>>>>>>> WAITING is better because it is the only >>>>>>>>>>>>>>>>>>>>>>>>>>> guarantee that the target thread is blocked >>>>>>>>>>>>>>>>>>>>>>>>>>> where you need it to be. suspending the >>>>>>>>>>>>>>>>>>>>>>>>>>> thread is racy as you don't know exactly >>>>>>>>>>>>>>>>>>>>>>>>>>> where the suspend will hit. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>> /* Wait until the thread state transits to >>>>>>>>>>>>>>>>>>>>>>>>>>>> "waiting" */ >>>>>>>>>>>>>>>>>>>>>>>>>>>> while (th.getState() != >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thread.State.WAITING) { >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thread.onSpinWait(); >>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> For simplify, spin wait is prefer to >>>>>>>>>>>>>>>>>>>>>>>>>>>> OneGetThreadListStackTraces.java in webrev.03. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here's the flow as I see it: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - start worker thread >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - startSignal.await() >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - startSignal.countDown() >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - stopSignal.await() >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - worker is now blocked >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - checkCallStacks(th) >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - stopSignal.countDown() >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - worker is now unblocked >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - th.join >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - runs off the end of run() >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - run off the end of main() >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L92: jthreads = (jthread >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *)malloc(sizeof(jthread) * num_threads); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? You don't check for malloc() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> failure. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'jthreads' is allocated but never freed. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L91: result = >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (*jvmti)->SuspendThread(jvmti, thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? Why are you suspending the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread? GetAllStackTraces() and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() do not >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> require the target thread(s) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? to be suspend. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ????????? If you decide not to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SuspendThread, then you don't need the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AddCapabilities or the ResumeThread calls. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Test thread might not be entered >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> following code (stopSignal.await()). We >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> might see deferent call stack between >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetAllStackTraces() and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces(). We cannot >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> control to freeze call stack of test >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread in Java code. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (I didn't use SuspendThread() at first, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but I saw some errors which causes in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> above.) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So we need to call SuspendThread() to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure we can see same call stack. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you are checking that the thread is in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state WAITING then it cannot escape from >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that state and you can sample the stack >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> multiple times from any API and get the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> same result. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I suspect the errors you saw were from >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the apparent incorrect use of the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CountDownLatch. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> With the flow outlined above, the worker >>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread should be >>>>>>>>>>>>>>>>>>>>>>>>>>>>> nicely blocked in stopSignal.await() when >>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuff is sampled. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 11:53 am, Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could review >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Updates look fine - thanks. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> One minor nit: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1274 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector.allocate_and_fill_stacks(1); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1275 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector.set_result(JVMTI_ERROR_NONE); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the other places where you use >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector you rely on result being >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initialized to JVMTI_ERROR_NONE, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather than setting it directly after >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocate_and_fill_stacks(). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 820 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> assert(SafepointSynchronize::is_at_safepoint() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 821 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 822 current_thread == >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 823 "at safepoint / handshake >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> check is necessary, as even if >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the target is suspended we must >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> still be at a safepoint or in a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> handshake with it. Makes me >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wonder if we used to allow a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> racy stacktrace operation on a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> suspended thread, assuming it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This function >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (JvmtiEnvBase::get_stack_trace()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can be called to get own stack >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> trace. For example, we can call >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetStackTrace() for current thread >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at JVMTI event. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So I changed assert as below: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??820 assert(current_thread == >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??821 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SafepointSynchronize::is_at_safepoint() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??823 "call by myself / at safepoint >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> / at handshake"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yep good catch. I hope current tests >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> caught that. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> They would be tested in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (own call stacks), and getstacktr003 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (call stacks in other thread). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Speaking of tests ... >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the native code I think you need >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to check the success of all JNI >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> methods that can throw exceptions - >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> otherwise I believe the tests may >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> trigger warnings if -Xcheck:jni is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> used with them. See for example: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I updated testcases to check JNI and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JVMTI function calls. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the Java code the target thread: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 45 public void run() { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 46 try { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 47 synchronized (lock) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 48 lock.wait(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 49 System.out.println("OK"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 50?????????? } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is potentially susceptible to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> spurious wakeups. Using a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CountDownLatch would be robust. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 9:05 am, Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _calling_thread is the current >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If so, we can call make_local() in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> L1272 without JavaThread (or we >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can pass current thread to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make_local()). Is it right? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry I got confused, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _calling_thread may not be the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> current thread as we could be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> executing the handshake in the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target thread itself. So the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark is correct as-is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (implicitly for current thread). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The argument to fill_frames will be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> used in the jvmtiStackInfo and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> passed back to the _calling_thread, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it must be created via >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make_local(_calling_thread, ...) as >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you presently have. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 7:05, David Holmes >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you for reviewing! I will >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> update new webrev tomorrow. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 498 class >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetAllStackTraces : public >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 500 JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 502 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> member of another class like >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that as it may not be on the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stack. I think >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should not extend any >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation class, and should >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> always be embedded directly in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure what does mean >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "embedded". >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??? : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class GetAllStackTraces : public >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/30 22:22, David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 30/06/2020 10:05 am, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I updated webrev for 8242428. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Could you review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change migrate to use >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> direct handshake for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetStackTrace() and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (when thread_count == 1). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks really good now! I >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only have a few nits below. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> There is one thing I don't like >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about it but it requires a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change to the main Handshake >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic to address - in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JvmtiEnv::GetThreadListStackTraces >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you have to create a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ThreadsListHandle to convert >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the jthread to a JavaThread, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but then the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Handshake::execute_direct >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates another >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ThreadsListHandle internally. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That's a waste. I will discuss >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with Robbin and file a RFE to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have an overload of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> execute_direct that takes an >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> existing TLH. Actually it's >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worse than that because we have >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> another TLH in use at the entry >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> point for the JVMTI functions, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so I think there may be some >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scope for simplifying the use >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of TLH instances - future RFE. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??451 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetStackTraceClosure(JvmtiEnv >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *env, jint start_depth, jint >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max_count, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??452 jvmtiFrameInfo* >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> frame_buffer, jint* count_ptr) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??453???? : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??454 _env(env), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _start_depth(start_depth), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _max_count(max_count), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??455 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _frame_buffer(frame_buffer), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _count_ptr(count_ptr), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??456 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _result(JVMTI_ERROR_THREAD_NOT_ALIVE) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Nit: can you do one initializer >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> per line please. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??498 class >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetAllStackTraces : public >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??500 JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??502 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> member of another class like >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that as it may not be on the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stack. I think >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should not extend any >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocation class, and should >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> always be embedded directly in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 481 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector(JvmtiEnv >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *env, jint max_frame_count) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??483 _max_frame_count = >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max_frame_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??484 _frame_count_total = 0; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??485 _head = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??486 _stack_info = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??487 _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As you are touching this can >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you change it to use an >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initializer list as you did for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the HandshakeClosure, and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> please keep one item per line. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??820 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> assert(SafepointSynchronize::is_at_safepoint() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??821 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??823 "at safepoint / handshake >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> check is necessary, as even if >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the target is suspended we must >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> still be at a safepoint or in a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> handshake with it. Makes me >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wonder if we used to allow a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> racy stacktrace operation on a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> suspended thread, assuming it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1268?? oop thread_oop = >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jt->threadObj(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1269 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can use thread_oop in line >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1270. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is frustrating that this >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entire call chain started with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a jthread reference, which we >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> converted to a JavaThread, only >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to eventually need to convert >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it back to a jthread! I think >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> there is some scope for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> simplification here but not as >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part of this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _calling_thread is the current >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please add @bug lines to the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tests. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm still pondering the test >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic but wanted to send this now. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetThreadListStackTrace >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (for GetThreadListStackTraces) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and VM_GetAllStackTraces (for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetAllStackTraces) have >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> inherited >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetMultipleStackTraces VM >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operation which provides the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature to generate >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jvmtiStackInfo. I modified >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetMultipleStackTraces to a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> normal C++ class to share with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HandshakeClosure for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (GetSingleStackTraceClosure). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I added new testcases >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which test >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with thread_count == 1 and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with all threads. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jvmti >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jdwp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jdi >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? JBS: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? webrev: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change replace following >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM operations to direct >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> handshake. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameCount >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (GetFrameCount()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (GetFrameLocation()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetThreadListStackTraces >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (GetThreadListStackTrace()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTrace() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> uses direct handshake if >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread count == 1. In other >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> case (thread count > 1), it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would be performed as VM >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Caller of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (JvmtiEnvThreadState::reset_current_location()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> might be called at safepoint. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So I added safepoint check in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> its caller. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in serviceability/jvmti >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jdwp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jdi vmTestbase/ns >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I tested it on submit >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repo, then it has execution >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> error >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> due to dependency error. So I >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it does not occur by >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>> >>>>>>> >>>>>> >>>> >> From suenaga at oss.nttdata.com Sat Jul 11 06:50:12 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Sat, 11 Jul 2020 15:50:12 +0900 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: <9a0eeb64-f05c-e266-db18-9e54fb7aa341@oracle.com> References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <19cfa2a7-f441-425d-5623-1c11c6a870b6@oss.nttdata.com> <237e8d26-d185-1008-6e07-5b9f36314534@oracle.com> <2a9ba1f8-b1f7-cb8c-fe11-56dd174d9d58@oracle.com> <7ad907cb-24ef-aa55-972a-0a3f63576568@oracle.com> <5ab8c5d9-9a5e-74a6-4605-7e73732037bf@oracle.com> <4881e0da-f716-c62e-b744-17aa3d03e095@oss.nttdata.com> <8dfa047c-8f84-5775-d12d-3248137e4b45@oss.nttdata.com> <9a0eeb64-f05c-e266-db18-9e54fb7aa341@oracle.com> Message-ID: Thanks Serguei! Yasumasa On 2020/07/11 14:51, serguei.spitsyn at oracle.com wrote: > Hi Yasumasa, > > I'm okay with the update. > > Thanks, > Serguei > > > On 7/10/20 18:52, Yasumasa Suenaga wrote: >> Thanks Dan! >> >> David, Serguei, are you ok to this change? >> >> >> Yasumasa >> >> >> On 2020/07/11 7:00, Daniel D. Daugherty wrote: >>> On 7/10/20 3:29 AM, Yasumasa Suenaga wrote: >>>> Thanks Patricio! >>>> >>>> I uploaded new webrev. Could you review again? >>>> It passed all tests on submit repo, and passed jtreg tests about JVMTI. >>>> >>>> ? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.10/ >>>> ? Diff from webrev.09: http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.10/incremental/ >>> >>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>> ???? No comments. >>> >>> Thumbs up on the incremental. >>> >>> Dan >>> >>> >>>> >>>> >>>> Yasumasa >>>> >>>> >>>> On 2020/07/10 14:49, Patricio Chilano wrote: >>>>> Hi Yasumasa, >>>>> >>>>> On 7/10/20 1:58 AM, Yasumasa Suenaga wrote: >>>>>> Hi Patricio, >>>>>> >>>>>> Thanks for your advice! >>>>>> I've fixed testcase as [1], but I still see an error in validate-headers-linux-x64-build-1 on submit repo. >>>>>> What does it mean? Can you share details? >>>>>> >>>>>> ? mach5-one-ysuenaga-JDK-8242428-20200710-0339-12529134 >>>>> Not your bug, you are getting the header validation failure from 8248570. It was fixed two days ago so just update the repo and it should work now. >>>>> >>>>> Thanks, >>>>> Patricio >>>>>> Of course this change can be built on my Linux box (Fedora 32, AMD64, GCC 10.1) >>>>>> >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> [1] http://hg.openjdk.java.net/jdk/submit/file/45f52142db9d/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>> >>>>>> >>>>>> On 2020/07/10 2:18, Patricio Chilano wrote: >>>>>>> >>>>>>> On 7/9/20 12:00 PM, Patricio Chilano wrote: >>>>>>>> Hi Yasumasa, >>>>>>>> >>>>>>>> On 7/9/20 9:30 AM, Yasumasa Suenaga wrote: >>>>>>>>> On 2020/07/09 17:58, David Holmes wrote: >>>>>>>>>> Hi Yasumasa, >>>>>>>>>> >>>>>>>>>> On 9/07/2020 10:25 am, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi Dan, >>>>>>>>>>> >>>>>>>>>>> Thanks for your comment! >>>>>>>>>>> I uploaded new webrev: >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.09/ >>>>>>>>>>> ?? Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/5d167adf8524 >>>>>>>>>>> >>>>>>>>>>> I saw similar build errors in libOneGetThreadListStackTraces.cpp on Windows. >>>>>>>>>>> This webrev fixes them. >>>>>>>>>> >>>>>>>>>> You shouldn't use %p as it may not be portable. In the VM we use INTPTR_FORMAT and convert the arg using p2i. I don't know what exists in the testing code. >>>>>>>>> >>>>>>>>> I replaced %p to %lx, and also cast values to unsigned long [1] [2], but the test on submit repo was failed. >>>>>>>>> Can anyone share details of mach5-one-ysuenaga-JDK-8242428-20200709-1030-12500928 ? >>>>>>>> These are the errors I see for the macOS build: >>>>>>>> >>>>>>>> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:53:14: error: format specifies type 'long long' but the argument has type 'jlocation' (aka 'long') [-Werror,-Wformat] >>>>>>>> ????????????? fi1->location, fi2->location); >>>>>>>> ????????????? ^~~~~~~~~~~~~ >>>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:53:29: error: format specifies type 'long long' but the argument has type 'jlocation' (aka 'long') [-Werror,-Wformat] >>>>>>>> ????????????? fi1->location, fi2->location); >>>>>>>> ???????????????????????????? ^~~~~~~~~~~~~ >>>>>>>> >>>>>>>> These are the ones I see for the Windows build: >>>>>>>> >>>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): warning C4311: 'type cast': pointer truncation from 'jmethodID' to 'unsigned long' >>>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): warning C4302: 'type cast': truncation from 'jmethodID' to 'unsigned long' >>>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): warning C4311: 'type cast': pointer truncation from 'jmethodID' to 'unsigned long' >>>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): warning C4302: 'type cast': truncation from 'jmethodID' to 'unsigned long' >>>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): warning C4311: 'type cast': pointer truncation from 'jthread' to 'unsigned long' >>>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): warning C4302: 'type cast': truncation from 'jthread' to 'unsigned long' >>>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): warning C4311: 'type cast': pointer truncation from 'jthread' to 'unsigned long' >>>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): warning C4302: 'type cast': truncation from 'jthread' to 'unsigned long' >>>>>>>> >>>>>>>> >>>>>>>> You will probably want to use the macros defined in src/hotspot/share/utilities/globalDefinitions.hpp. Let me know if you need me to test something. >>>>>>> With these changes the build works okay on Linux, Windows and macOS: >>>>>>> >>>>>>> >>>>>>> --- a/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>>> +++ b/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>>> @@ -27,4 +27,5 @@ >>>>>>> ??#include >>>>>>> ??#include >>>>>>> +#include >>>>>>> >>>>>>> ??#define MAX_FRAMES 100 >>>>>>> @@ -45,11 +46,11 @@ >>>>>>> ??? if (fi1->method != fi2->method) { /* jvmtiFrameInfo::method */ >>>>>>> ????? snprintf(err_msg, sizeof(err_msg), >>>>>>> -???????????? "method is different: fi1 = %p, fi2 = %p", >>>>>>> -???????????? fi1->method, fi2->method); >>>>>>> +???????????? "method is different: fi1 = 0x%016" PRIxPTR " , fi2 = 0x%016" PRIxPTR, >>>>>>> +???????????? (intptr_t)fi1->method, (intptr_t)fi2->method); >>>>>>> ????? env->FatalError(err_msg); >>>>>>> ??? } else if (fi1->location != fi2->location) { /* jvmtiFrameInfo::location */ >>>>>>> ????? snprintf(err_msg, sizeof(err_msg), >>>>>>> -???????????? "location is different: fi1 = %lld, fi2 = %lld", >>>>>>> -???????????? fi1->location, fi2->location); >>>>>>> +???????????? "location is different: fi1 = %" PRId64 " , fi2 = %" PRId64, >>>>>>> +???????????? (int64_t)fi1->location, (int64_t)fi2->location); >>>>>>> ????? env->FatalError(err_msg); >>>>>>> ??? } >>>>>>> @@ -67,5 +68,5 @@ >>>>>>> ??? if (!is_same) { /* jvmtiStackInfo::thread */ >>>>>>> ????? snprintf(err_msg, sizeof(err_msg), >>>>>>> -???????????? "thread is different: si1 = %p, si2 = %p", si1->thread, si2->thread); >>>>>>> +???????????? "thread is different: si1 = 0x%016" PRIxPTR " , si2 = 0x%016" PRIxPTR, (intptr_t)si1->thread, (intptr_t)si2->thread); >>>>>>> ????? env->FatalError(err_msg); >>>>>>> ??? } else if (si1->state != si2->state) { /* jvmtiStackInfo::state */ >>>>>>> >>>>>>> >>>>>>> Maybe you can use something like that. >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Patricio >>>>>>>> Thanks, >>>>>>>> Patricio >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> [1] http://hg.openjdk.java.net/jdk/submit/rev/dfca51958217 >>>>>>>>> [2] http://hg.openjdk.java.net/jdk/submit/rev/3665361fa91b >>>>>>>>> >>>>>>>>> >>>>>>>>>> David >>>>>>>>>> ----- >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2020/07/09 1:42, Daniel D. Daugherty wrote: >>>>>>>>>>>> ?> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ >>>>>>>>>>>> >>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>>>> ???? No comments. >>>>>>>>>>>> >>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>> ???? L1159: ? Thread *current_thread = Thread::current(); >>>>>>>>>>>> ???????? Please add "#ifdef ASSERT" above and "#endif" below since >>>>>>>>>>>> ???????? current_thread is only used for the assert() in this function. >>>>>>>>>>>> >>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>> ???? L549: jthread java_thread, jint max_frame_count) >>>>>>>>>>>> ???? L552: ????? _jthread(java_thread), >>>>>>>>>>>> ???????? Please: s/java_thread/thread/ on both lines. >>>>>>>>>>>> >>>>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>>>> ???? No comments. >>>>>>>>>>>> >>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>>>> ???? No comments. >>>>>>>>>>>> >>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>>>>>>>> ???? L27: #include >>>>>>>>>>>> ???????? This include is out of order; should be first in the list. >>>>>>>>>>>> >>>>>>>>>>>> ???? This file doesn't compile on my MBP13: >>>>>>>>>>>> >>>>>>>>>>>> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:49:14: error: format specifies type 'unsigned long' but the argument has type 'jmethodID' (aka '_jmethodID *') [-Werror,-Wformat] >>>>>>>>>>>> ????????????? fi1->method, fi2->method); >>>>>>>>>>>> ????????????? ^~~~~~~~~~~ >>>>>>>>>>>> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:49:27: error: format specifies type 'unsigned long' but the argument has type 'jmethodID' (aka '_jmethodID *') [-Werror,-Wformat] >>>>>>>>>>>> ????????????? fi1->method, fi2->method); >>>>>>>>>>>> ?????????????????????????? ^~~~~~~~~~~ >>>>>>>>>>>> ???? 2 errors generated. >>>>>>>>>>>> >>>>>>>>>>>> ???? This change made it compile on my MBP13, but that may break it on >>>>>>>>>>>> ???? other platforms: >>>>>>>>>>>> >>>>>>>>>>>> ???? $ hg diff >>>>>>>>>>>> ???? diff -r 560847c69fbe test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>>>>>>>> ???? --- a/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp Wed Jul 08 12:13:32 2020 -0400 >>>>>>>>>>>> ???? +++ b/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp Wed Jul 08 12:40:42 2020 -0400 >>>>>>>>>>>> ???? @@ -46,7 +46,7 @@ >>>>>>>>>>>> ?? ?? ? if (fi1->method != fi2->method) { /* jvmtiFrameInfo::method */ >>>>>>>>>>>> ?? ?? ??? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>> ?? ? ? ??????????? "method is different: fi1 = %lx, fi2 = %lx", >>>>>>>>>>>> ???? -???????????? fi1->method, fi2->method); >>>>>>>>>>>> ???? +???????????? (unsigned long) fi1->method, (unsigned long) fi2->method); >>>>>>>>>>>> ?? ?? ??? env->FatalError(err_msg); >>>>>>>>>>>> ?? ?? ? } else if (fi1->location != fi2->location) { /* jvmtiFrameInfo::location */ >>>>>>>>>>>> ?? ?? ??? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>> >>>>>>>>>>>> ???? I'm not sure of the right platform independent way to output >>>>>>>>>>>> ???? the 'method' field. >>>>>>>>>>>> >>>>>>>>>>>> Dan >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 7/8/20 4:04 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>> Hi David, >>>>>>>>>>>>> >>>>>>>>>>>>> On 2020/07/08 15:27, David Holmes wrote: >>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 7/07/2020 6:54 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Serguei, thank you for replying even though you are on vacaiton! >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I uploaded new webrev: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.07/ >>>>>>>>>>>>>>> ?? Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/77243b1dcbfe >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> c'tor of GetSingleStackTraceClosure has jthread argument in this webrev. >>>>>>>>>>>>>>> Also it does not contain testcase for GetThreadListStackTraces with all threads, and OneGetThreadListStackTraces would test main thread only. >>>>>>>>>>>>>> >>>>>>>>>>>>>> All those changes are fine in principle for me. One nit/suggestion: >>>>>>>>>>>>>> >>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>> >>>>>>>>>>>>>> ??544?? jthread _java_thread; >>>>>>>>>>>>>> >>>>>>>>>>>>>> elsewhere "java_thread" refers to a JavaThread, so to avoid confusion may I suggest this member be named _jthread. >>>>>>>>>>>>> >>>>>>>>>>>>> I uploaded new webrev: >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ >>>>>>>>>>>>> ? Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/ca6263dbdc87 >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> I'm going to be away for the next couple of days - sorry - but will try to check email on this if I can. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks! >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> David >>>>>>>>>>>>>> ----- >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 2020/07/07 15:13, David Holmes wrote: >>>>>>>>>>>>>>>> On 7/07/2020 2:57 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 2020/07/07 11:31, David Holmes wrote: >>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Hard to keep up with the changes - especially without incremental webrevs. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Sorry, I will upload diff from previous webrev in the next. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> If GetSingleStackTraceClosure also took the jthread as a constructor arg, then you wouldn't need to recreate a JNI local handle when calling _collector.fill_frames. It's a small simplification and not essential at this stage. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I think we should get jthread from an argument of do_thread() because do_thread() would pass the thread which are stopped certainly. >>>>>>>>>>>>>>>>> It might be simplification if we pass _calling_thread to MultipleStackTracesCollector. `jthread` is only needed to store jvmtiStackInfo.thread . What do you think? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I'm not quite sure what you mean. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I think there is a bit of a design wart with direct handshakes in that do_thread takes the target JavaThread as an argument. That's useful in a case where you want a HandshakeClosure that can be applied to multiple threads, but that's not typically what is needed with direct handshakes - there is only a single target. With a single-target HandshakeClosure you can capture all the "target" information for the operation in the closure instance. So if the actual do_thread operation wants the jthread corresponding to the target thread then we can store that in the closure rather than recomputing it (you could assert it is the same but that seems overkill to me). >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> For the test ... I don't see how Java_GetThreadListStackTraces_checkCallStacks is a valid test. It gets the stacks of all live threads, then uses that information to use GetThreadListStackTraces to get the stack for the same set of threads through a different API. It then compares the two sets of stacks for each thread expecting them to be the same, but that need only be the case for the main thread. Other threads could potentially have a different stack (e.g. if this test is run with JFR enabled there will be additional threads found.) Further I would have expected that there already exist tests that check that, for a given thread (which may be suspended or known to be blocked) the same stack is found through the two different APIs. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti/unit/GetAllStackTraces/getallstktr001 would check all of threads via GetThreadListStackTraces() and GetAllStackTraces(), so we might be able to remove GetThreadListStackTraces.java from this webrev. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Yes. The existing test only examines a set of test threads that are all blocked on a raw monitor. You do not need to duplicate that test. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> OTOH we don't have testcase for GetThreadListStackTraces() with thread_count == 1, so we need to testcase for it (it is OneGetThreadListStackTraces.java) It would check whether the state of target thread is "waiting" before JNI call to call GetThreadListStackTraces(), >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Yes we need to test the special cases introduced by your changes - totally agree - and OneGetThreadListStackTraces.java is a good test for that. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> and also I expect it would not be run with JFR. (it is not described @run) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> The arguments to run with JFR (or a bunch of other things) can be passed to the jtreg test harness to be applied to all tests. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Of course we can check GetThreadListStackTraces() with main thread, but it is not the test for direct handshake for other thread. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Right - that test already exists as per the above. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 6/07/2020 11:29 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I think C++ is more simple to implement the test agent as you said. >>>>>>>>>>>>>>>>>>> So I implement it in C++ in new webrev. Could you review again? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.06/ >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Also I refactored libGetThreadListStackTraces.cpp, and I've kept exception check after IsSameObject(). >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On 2020/07/06 16:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thank you for the update. >>>>>>>>>>>>>>>>>>>> I think, a pending exception after IsSameObject needs to be checked. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> The checkStackInfo() needs one more refactoring as I've already suggested. >>>>>>>>>>>>>>>>>>>> The body of the loop at L68-L78 should be converted to a function check_frame_info. >>>>>>>>>>>>>>>>>>>> The si1->frame_buffer[i] and si2->frame_buffer[i] will be passed as fi1 and fi2. >>>>>>>>>>>>>>>>>>>> The index can be passed as well. >>>>>>>>>>>>>>>>>>>> I'm still suggesting to simplify the local exception_msg to something shorter like err_msg or exc_msg. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I'm not sure using fatal is right here: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> This fragment looks strange: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> ? 152???? if ((*env)->IsSameObject(env, stack_info[i].thread, thread)) { >>>>>>>>>>>>>>>>>>>> ? 153?????? target_info = &stack_info[i]; >>>>>>>>>>>>>>>>>>>> ? 154?????? break; >>>>>>>>>>>>>>>>>>>> ? 155???? } else if ((*env)->ExceptionOccurred(env)) { >>>>>>>>>>>>>>>>>>>> ? 156 (*env)->ExceptionDescribe(env); >>>>>>>>>>>>>>>>>>>> ? 157 (*env)->FatalError(env, __FILE__); >>>>>>>>>>>>>>>>>>>> ? 158???? } >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I expected it to be: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> ??? jboolean same = (*env)->IsSameObject(env, stack_info[i].thread, thread); >>>>>>>>>>>>>>>>>>>> ??? if ((*env)->ExceptionOccurred(env)) { >>>>>>>>>>>>>>>>>>>> (*env)->ExceptionDescribe(env); >>>>>>>>>>>>>>>>>>>> (*env)->FatalError(env, __FILE__); >>>>>>>>>>>>>>>>>>>> ??? } >>>>>>>>>>>>>>>>>>>> ??? if (same) { >>>>>>>>>>>>>>>>>>>> ????? target_info = &stack_info[i]; >>>>>>>>>>>>>>>>>>>> ????? break; >>>>>>>>>>>>>>>>>>>> ??? } >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Would it better to port this agent to C++ to simplify this code nicer? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On 7/5/20 06:13, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>>>>>> I refactored testcase. Could you review again? >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.05/ >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> It would check Java exception after IsSameObject() call. Does it need? >>>>>>>>>>>>>>>>>>>>> Any exceptions are not described in JNI document[1], and JNI implementation (jni_IsSameObject()) does not seem to throw it. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> [1] https://docs.oracle.com/en/java/javase/14/docs/specs/jni/functions.html#issameobject >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On 2020/07/05 14:46, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Okay, thanks. >>>>>>>>>>>>>>>>>>>>>> Then I'm okay to keep the GetSingleStackTraceClosure. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c.html >>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c.html >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I'm not sure the function 'is_same_thread() is needed. >>>>>>>>>>>>>>>>>>>>>> Why do not use the JNI IsSameObject instead? >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> It seems to be a typo at L132 and L137. >>>>>>>>>>>>>>>>>>>>>> You, probably. did not want to print the same information for stack_info_1[i].frame_buffer[j].XXX twice. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> The code at lines 112-142 is not readable. >>>>>>>>>>>>>>>>>>>>>> I'd suggest to make a couple of refactoring steps. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> First step to simplify this a little bit would be with some renaming and getting rid of indexes: >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> ?? 71 char err_msg[EXCEPTION_MSG_LEN] = {0}; >>>>>>>>>>>>>>>>>>>>>> ??... >>>>>>>>>>>>>>>>>>>>>> ??112?? /* Iterate all jvmtiStackInfo to check */ >>>>>>>>>>>>>>>>>>>>>> ??113?? for (i = 0; i < num_threads, *exception_msg != '\0'; i++) { >>>>>>>>>>>>>>>>>>>>>> ????????? jvmtiStackInfo *si1 = stack_info_1[i]; >>>>>>>>>>>>>>>>>>>>>> ????????? jvmtiStackInfo *si2 = stack_info_2[i]; >>>>>>>>>>>>>>>>>>>>>> ??114???? if (!IsSameObject(env, si1.thread, si2.thread)) { /* jvmtiStackInfo::thread */ >>>>>>>>>>>>>>>>>>>>>> ??115 snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>>>>>> ??116 "thread[%d] is different: stack_info_1 = %p, stack_info_2 = %p", >>>>>>>>>>>>>>>>>>>>>> ??117??????????????? i, sinfo1.thread, sinfo2.thread); >>>>>>>>>>>>>>>>>>>>>> ??118???? } else if (si1.state != si2.state) { /* jvmtiStackInfo::state */ >>>>>>>>>>>>>>>>>>>>>> ??119 snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>>>>>> ??120 "state[%d] is different: stack_info_1 = %d, stack_info_2 = %d", >>>>>>>>>>>>>>>>>>>>>> ??121??????????????? i, si1.state, si2.state); >>>>>>>>>>>>>>>>>>>>>> ??122???? } else if (si1.frame_count != si2.frame_count) { /* jvmtiStackInfo::frame_count */ >>>>>>>>>>>>>>>>>>>>>> ??123 snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>>>>>> ??124 "frame_count[%d] is different: stack_info_1 = %d, stack_info_2 = %d", >>>>>>>>>>>>>>>>>>>>>> ??125??????????????? i, si1.frame_count, si2.frame_count); >>>>>>>>>>>>>>>>>>>>>> ??126???? } else { >>>>>>>>>>>>>>>>>>>>>> ??127?????? /* Iterate all jvmtiFrameInfo to check */ >>>>>>>>>>>>>>>>>>>>>> ??128?????? for (j = 0; j < si1.frame_count; j++) { >>>>>>>>>>>>>>>>>>>>>> ??129???????? if (si1.frame_buffer[j].method != si1.frame_buffer[j].method) { /* jvmtiFrameInfo::method */ >>>>>>>>>>>>>>>>>>>>>> ??130 snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>>>>>> ??131 "thread [%d] frame_buffer[%d].method is different: stack_info_1 = %lx, stack_info_2 = %lx", >>>>>>>>>>>>>>>>>>>>>> ??132 i, j, si1.frame_buffer[j].method, si2.frame_buffer[j].method); >>>>>>>>>>>>>>>>>>>>>> ??133?????????? break; >>>>>>>>>>>>>>>>>>>>>> ??134???????? } else if (si1.frame_buffer[j].location != si1.frame_buffer[j].location) { /* jvmtiFrameInfo::location */ >>>>>>>>>>>>>>>>>>>>>> ??135 snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>>>>>> ??136 "thread [%d] frame_buffer[%d].location is different: stack_info_1 = %ld, stack_info_2 = %ld", >>>>>>>>>>>>>>>>>>>>>> ??137 i, j, si1.frame_buffer[j].location, si2.frame_buffer[j].location); >>>>>>>>>>>>>>>>>>>>>> ??138?????????? break; >>>>>>>>>>>>>>>>>>>>>> ??139???????? } >>>>>>>>>>>>>>>>>>>>>> ??140?????? } >>>>>>>>>>>>>>>>>>>>>> ??141???? } >>>>>>>>>>>>>>>>>>>>>> ??142?? } >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Another step would be to create functions that implement a body of each loop. >>>>>>>>>>>>>>>>>>>>>> You can use the same techniques to simplify similar place (L127-L138) in the libOneGetThreadListStackTraces.c. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> On 7/3/20 15:55, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I'm not an Oracle employee, so I cannot know real request(s) from your customers. >>>>>>>>>>>>>>>>>>>>>>> However JDK-8201641 says Dynatrace has requested this enhancement. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> BTW I haven't heared any request from my customers about this. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On 2020/07/04 4:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> This difference is not that big to care about. >>>>>>>>>>>>>>>>>>>>>>>> I feel this is really rare case and so, does not worth these complications. >>>>>>>>>>>>>>>>>>>>>>>> Do we have a real request from customers to optimize it? >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On 7/3/20 01:16, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Generally I agree with you, but I have concern about the difference of the result of GetStackTrace() and GetThreadListStackTraces(). >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> ? GetStackTrace: jvmtiFrameInfo >>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces: jvmtiStackInfo >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> jvmtiStackInfo contains thread state, and it is ensured it is the state of the call stack. >>>>>>>>>>>>>>>>>>>>>>>>> If we want to get both call stack and thread state, we need to suspend target thread, and call both GetStackTrace() and GetThreadState(). Is it ok? >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> I was wondering if JDK-8201641 (parent ticket of this change) needed them for profiling (dynatrace?) >>>>>>>>>>>>>>>>>>>>>>>>> If it is responsibility of JVMTI agent implementor, I remove this closure. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 16:45, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> After some thinking I've concluded that I do not like this optimization >>>>>>>>>>>>>>>>>>>>>>>>>> of the GetThreadListStackTraces with GetSingleStackTraceClosure. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> We may need more opinions on this but these are my points: >>>>>>>>>>>>>>>>>>>>>>>>>> ??- it adds some complexity and ugliness >>>>>>>>>>>>>>>>>>>>>>>>>> ??- a win is doubtful because it has to be a rare case, so that total overhead should not be high >>>>>>>>>>>>>>>>>>>>>>>>>> ??- if it is really high for some use cases then it is up to the user >>>>>>>>>>>>>>>>>>>>>>>>>> ??? to optimize it with using GetStackTrace instead >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> In such cases with doubtful overhead I usually prefer the simplicity. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Good examples where it makes sense to optimize are checks for target thread to be current thread. >>>>>>>>>>>>>>>>>>>>>>>>>> In such cases there is no need to suspend the target thread, or use a VMop/HandshakeClosure. >>>>>>>>>>>>>>>>>>>>>>>>>> For instance, please, see the Monitor functions with the check: (java_thread == calling_thread). >>>>>>>>>>>>>>>>>>>>>>>>>> Getting information for current thread is frequently used case, e.g. to get info at an event point. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> On 7/2/20 23:29, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Dan, David, >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could you review again? >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/ >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> OneGetThreadListStackTraces.java in this webrev would wait until thread state is transited to "waiting" with spin wait. >>>>>>>>>>>>>>>>>>>>>>>>>>> CountDownLatch::await call as Dan pointed is fixed in it :) >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Diff from webrev.03: >>>>>>>>>>>>>>>>>>>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/c9aeb7001e50 >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 14:15, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 12:24, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 7/2/20 10:50 PM, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry I'm responding here without seeing latest webrev but there is enough context I think ... >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Dan, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I upload new webrev. Could you review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L1542: // Get stack trace with handshake >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - please add a period at the end. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> L1591: *stack_info_ptr = op.stack_info(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? The return parameter should not be touched unless the return >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? old L1582: if (err == JVMTI_ERROR_NONE) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please restore this check. The return parameter should not >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? be touched unless the return code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But op.stack_info() will return NULL if the error is not JVMTI_ERROR_NONE. Are you (Dan) concerned about someone passing in a non-null/initialized out-pointer that will be reset to NULL if there was an error? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Actually the way we used to test this in POSIX tests is to call >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an API with known bad parameters and the return parameter ptr >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> set to NULL. If the return parameter ptr was touched when an >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> error should have been detected on an earlier parameter, then >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the test failed. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> L1272: ? if (!jt->is_exiting() && (thread_oop != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? old L1532: _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? This deletion of the _result field threw me for a minute and then >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? ? ? I figured out that the field is init to JVMTI_ERROR_THREAD_NOT_ALIVE >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? in the constructor. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L1553: if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L64: startSignal.countDown(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? I was expecting this to be a call to await() instead of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> countDown(). What am I missing here? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? I think this test might be passing by accident right now, but... >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Main thread (which call JVMTI functions to test) should wait until test thread is ready. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So main thread would wait startSignal, and test thread would count down. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No! >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The test thread that previously called obj.wait() now calls latch.await(). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The main thread that previously called obj.notify() now calls latch.countDown(). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The main thread continues to spin until it sees the target is WAITING before proceeding with the test. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> If I add spin wait to wait until transit target thread state is WAITING (as following), we don't need to call SuspendThread(). >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Which is better? >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> The original spin-wait loop checking for WAITING is better because it is the only guarantee that the target thread is blocked where you need it to be. suspending the thread is racy as you don't know exactly where the suspend will hit. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>> /* Wait until the thread state transits to "waiting" */ >>>>>>>>>>>>>>>>>>>>>>>>>>>>> while (th.getState() != Thread.State.WAITING) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thread.onSpinWait(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> For simplify, spin wait is prefer to OneGetThreadListStackTraces.java in webrev.03. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here's the flow as I see it: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - start worker thread >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - startSignal.await() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - startSignal.countDown() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - stopSignal.await() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - worker is now blocked >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - checkCallStacks(th) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - stopSignal.countDown() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - worker is now unblocked >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - th.join >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - runs off the end of run() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - run off the end of main() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L92: jthreads = (jthread *)malloc(sizeof(jthread) * num_threads); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? You don't check for malloc() failure. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'jthreads' is allocated but never freed. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L91: result = (*jvmti)->SuspendThread(jvmti, thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? Why are you suspending the thread? GetAllStackTraces() and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() do not require the target thread(s) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? to be suspend. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ????????? If you decide not to SuspendThread, then you don't need the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AddCapabilities or the ResumeThread calls. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Test thread might not be entered following code (stopSignal.await()). We might see deferent call stack between GetAllStackTraces() and GetThreadListStackTraces(). We cannot control to freeze call stack of test thread in Java code. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (I didn't use SuspendThread() at first, but I saw some errors which causes in above.) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So we need to call SuspendThread() to ensure we can see same call stack. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you are checking that the thread is in state WAITING then it cannot escape from that state and you can sample the stack multiple times from any API and get the same result. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I suspect the errors you saw were from the apparent incorrect use of the CountDownLatch. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With the flow outlined above, the worker thread should be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nicely blocked in stopSignal.await() when stuff is sampled. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Updates look fine - thanks. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> One minor nit: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1274 _collector.allocate_and_fill_stacks(1); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1275 _collector.set_result(JVMTI_ERROR_NONE); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the other places where you use _collector you rely on result being initialized to JVMTI_ERROR_NONE, rather than setting it directly after allocate_and_fill_stacks(). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 823 "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This function (JvmtiEnvBase::get_stack_trace()) can be called to get own stack trace. For example, we can call GetStackTrace() for current thread at JVMTI event. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So I changed assert as below: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??820 assert(current_thread == java_thread || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??821 SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??823 "call by myself / at safepoint / at handshake"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yep good catch. I hope current tests caught that. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> They would be tested in vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ (own call stacks), and getstacktr003 (call stacks in other thread). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Speaking of tests ... >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the native code I think you need to check the success of all JNI methods that can throw exceptions - otherwise I believe the tests may trigger warnings if -Xcheck:jni is used with them. See for example: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I updated testcases to check JNI and JVMTI function calls. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the Java code the target thread: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 45 public void run() { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 46 try { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 47 synchronized (lock) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 48 lock.wait(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 49 System.out.println("OK"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 50?????????? } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is potentially susceptible to spurious wakeups. Using a CountDownLatch would be robust. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If so, we can call make_local() in L1272 without JavaThread (or we can pass current thread to make_local()). Is it right? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry I got confused, _calling_thread may not be the current thread as we could be executing the handshake in the target thread itself. So the ResourceMark is correct as-is (implicitly for current thread). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The argument to fill_frames will be used in the jvmtiStackInfo and passed back to the _calling_thread, so it must be created via make_local(_calling_thread, ...) as you presently have. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you for reviewing! I will update new webrev tomorrow. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 500 JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??? : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I updated webrev for 8242428. Could you review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change migrate to use direct handshake for GetStackTrace() and GetThreadListStackTraces() (when thread_count == 1). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks really good now! I only have a few nits below. There is one thing I don't like about it but it requires a change to the main Handshake logic to address - in JvmtiEnv::GetThreadListStackTraces you have to create a ThreadsListHandle to convert the jthread to a JavaThread, but then the Handshake::execute_direct creates another ThreadsListHandle internally. That's a waste. I will discuss with Robbin and file a RFE to have an overload of execute_direct that takes an existing TLH. Actually it's worse than that because we have another TLH in use at the entry point for the JVMTI functions, so I think there may be some scope for simplifying the use of TLH instances - future RFE. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??451 GetStackTraceClosure(JvmtiEnv *env, jint start_depth, jint max_count, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??452 jvmtiFrameInfo* frame_buffer, jint* count_ptr) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??454 _env(env), _start_depth(start_depth), _max_count(max_count), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??455 _frame_buffer(frame_buffer), _count_ptr(count_ptr), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??456 _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Nit: can you do one initializer per line please. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??500 JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 481 MultipleStackTracesCollector(JvmtiEnv *env, jint max_frame_count) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??483 _max_frame_count = max_frame_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??484 _frame_count_total = 0; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??485 _head = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??486 _stack_info = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??487 _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As you are touching this can you change it to use an initializer list as you did for the HandshakeClosure, and please keep one item per line. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??823 "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1269 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is frustrating that this entire call chain started with a jthread reference, which we converted to a JavaThread, only to eventually need to convert it back to a jthread! I think there is some scope for simplification here but not as part of this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm still pondering the test logic but wanted to send this now. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetThreadListStackTrace (for GetThreadListStackTraces) and VM_GetAllStackTraces (for GetAllStackTraces) have inherited VM_GetMultipleStackTraces VM operation which provides the feature to generate jvmtiStackInfo. I modified VM_GetMultipleStackTraces to a normal C++ class to share with HandshakeClosure for GetThreadListStackTraces (GetSingleStackTraceClosure). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I added new testcases which test GetThreadListStackTraces() with thread_count == 1 and with all threads. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/nsk/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change replace following VM operations to direct handshake. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces (GetThreadListStackTrace()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTrace() uses direct handshake if thread count == 1. In other case (thread count > 1), it would be performed as VM operation (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Caller of VM_GetCurrentLocation (JvmtiEnvThreadState::reset_current_location()) might be called at safepoint. So I added safepoint check in its caller. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/ns >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I tested it on submit repo, then it has execution error (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) due to dependency error. So I think it does not occur by this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>> > From david.holmes at oracle.com Sat Jul 11 13:50:12 2020 From: david.holmes at oracle.com (David Holmes) Date: Sat, 11 Jul 2020 23:50:12 +1000 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: <8dfa047c-8f84-5775-d12d-3248137e4b45@oss.nttdata.com> References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <19cfa2a7-f441-425d-5623-1c11c6a870b6@oss.nttdata.com> <237e8d26-d185-1008-6e07-5b9f36314534@oracle.com> <2a9ba1f8-b1f7-cb8c-fe11-56dd174d9d58@oracle.com> <7ad907cb-24ef-aa55-972a-0a3f63576568@oracle.com> <5ab8c5d9-9a5e-74a6-4605-7e73732037bf@oracle.com> <4881e0da-f716-c62e-b744-17aa3d03e095@oss.nttdata.com> <8dfa047c-8f84-5775-d12d-3248137e4b45@oss.nttdata.com> Message-ID: On 11/07/2020 11:52 am, Yasumasa Suenaga wrote: > Thanks Dan! > > David, Serguei, are you ok to this change? Yes it seems fine. In relation to an earlier comment: > I replaced %p to %lx, and also cast values to unsigned long Never use long or unsigned long as they can be different sizes on different platforms (specifically long is not 64-bit on Windows). Thanks, David ----- > > Yasumasa > > > On 2020/07/11 7:00, Daniel D. Daugherty wrote: >> On 7/10/20 3:29 AM, Yasumasa Suenaga wrote: >>> Thanks Patricio! >>> >>> I uploaded new webrev. Could you review again? >>> It passed all tests on submit repo, and passed jtreg tests about JVMTI. >>> >>> ? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.10/ >>> ? Diff from webrev.09: >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.10/incremental/ >> >> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >> >> ???? No comments. >> >> Thumbs up on the incremental. >> >> Dan >> >> >>> >>> >>> Yasumasa >>> >>> >>> On 2020/07/10 14:49, Patricio Chilano wrote: >>>> Hi Yasumasa, >>>> >>>> On 7/10/20 1:58 AM, Yasumasa Suenaga wrote: >>>>> Hi Patricio, >>>>> >>>>> Thanks for your advice! >>>>> I've fixed testcase as [1], but I still see an error in >>>>> validate-headers-linux-x64-build-1 on submit repo. >>>>> What does it mean? Can you share details? >>>>> >>>>> ? mach5-one-ysuenaga-JDK-8242428-20200710-0339-12529134 >>>> Not your bug, you are getting the header validation failure from >>>> 8248570. It was fixed two days ago so just update the repo and it >>>> should work now. >>>> >>>> Thanks, >>>> Patricio >>>>> Of course this change can be built on my Linux box (Fedora 32, >>>>> AMD64, GCC 10.1) >>>>> >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> [1] >>>>> http://hg.openjdk.java.net/jdk/submit/file/45f52142db9d/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>> >>>>> >>>>> >>>>> On 2020/07/10 2:18, Patricio Chilano wrote: >>>>>> >>>>>> On 7/9/20 12:00 PM, Patricio Chilano wrote: >>>>>>> Hi Yasumasa, >>>>>>> >>>>>>> On 7/9/20 9:30 AM, Yasumasa Suenaga wrote: >>>>>>>> On 2020/07/09 17:58, David Holmes wrote: >>>>>>>>> Hi Yasumasa, >>>>>>>>> >>>>>>>>> On 9/07/2020 10:25 am, Yasumasa Suenaga wrote: >>>>>>>>>> Hi Dan, >>>>>>>>>> >>>>>>>>>> Thanks for your comment! >>>>>>>>>> I uploaded new webrev: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.09/ >>>>>>>>>> ?? Diff from previous webrev: >>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/5d167adf8524 >>>>>>>>>> >>>>>>>>>> I saw similar build errors in >>>>>>>>>> libOneGetThreadListStackTraces.cpp on Windows. >>>>>>>>>> This webrev fixes them. >>>>>>>>> >>>>>>>>> You shouldn't use %p as it may not be portable. In the VM we >>>>>>>>> use INTPTR_FORMAT and convert the arg using p2i. I don't know >>>>>>>>> what exists in the testing code. >>>>>>>> >>>>>>>> I replaced %p to %lx, and also cast values to unsigned long [1] >>>>>>>> [2], but the test on submit repo was failed. >>>>>>>> Can anyone share details of >>>>>>>> mach5-one-ysuenaga-JDK-8242428-20200709-1030-12500928 ? >>>>>>> These are the errors I see for the macOS build: >>>>>>> >>>>>>> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:53:14: >>>>>>> error: format specifies type 'long long' but the argument has >>>>>>> type 'jlocation' (aka 'long') [-Werror,-Wformat] >>>>>>> ????????????? fi1->location, fi2->location); >>>>>>> ????????????? ^~~~~~~~~~~~~ >>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:53:29: >>>>>>> error: format specifies type 'long long' but the argument has >>>>>>> type 'jlocation' (aka 'long') [-Werror,-Wformat] >>>>>>> ????????????? fi1->location, fi2->location); >>>>>>> ???????????????????????????? ^~~~~~~~~~~~~ >>>>>>> >>>>>>> These are the ones I see for the Windows build: >>>>>>> >>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): >>>>>>> warning C4311: 'type cast': pointer truncation from 'jmethodID' >>>>>>> to 'unsigned long' >>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): >>>>>>> warning C4302: 'type cast': truncation from 'jmethodID' to >>>>>>> 'unsigned long' >>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): >>>>>>> warning C4311: 'type cast': pointer truncation from 'jmethodID' >>>>>>> to 'unsigned long' >>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): >>>>>>> warning C4302: 'type cast': truncation from 'jmethodID' to >>>>>>> 'unsigned long' >>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): >>>>>>> warning C4311: 'type cast': pointer truncation from 'jthread' to >>>>>>> 'unsigned long' >>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): >>>>>>> warning C4302: 'type cast': truncation from 'jthread' to >>>>>>> 'unsigned long' >>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): >>>>>>> warning C4311: 'type cast': pointer truncation from 'jthread' to >>>>>>> 'unsigned long' >>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): >>>>>>> warning C4302: 'type cast': truncation from 'jthread' to >>>>>>> 'unsigned long' >>>>>>> >>>>>>> >>>>>>> You will probably want to use the macros defined in >>>>>>> src/hotspot/share/utilities/globalDefinitions.hpp. Let me know if >>>>>>> you need me to test something. >>>>>> With these changes the build works okay on Linux, Windows and macOS: >>>>>> >>>>>> >>>>>> --- >>>>>> a/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>> >>>>>> +++ >>>>>> b/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>> >>>>>> @@ -27,4 +27,5 @@ >>>>>> ??#include >>>>>> ??#include >>>>>> +#include >>>>>> >>>>>> ??#define MAX_FRAMES 100 >>>>>> @@ -45,11 +46,11 @@ >>>>>> ??? if (fi1->method != fi2->method) { /* jvmtiFrameInfo::method */ >>>>>> ????? snprintf(err_msg, sizeof(err_msg), >>>>>> -???????????? "method is different: fi1 = %p, fi2 = %p", >>>>>> -???????????? fi1->method, fi2->method); >>>>>> +???????????? "method is different: fi1 = 0x%016" PRIxPTR " , fi2 >>>>>> = 0x%016" PRIxPTR, >>>>>> +???????????? (intptr_t)fi1->method, (intptr_t)fi2->method); >>>>>> ????? env->FatalError(err_msg); >>>>>> ??? } else if (fi1->location != fi2->location) { /* >>>>>> jvmtiFrameInfo::location */ >>>>>> ????? snprintf(err_msg, sizeof(err_msg), >>>>>> -???????????? "location is different: fi1 = %lld, fi2 = %lld", >>>>>> -???????????? fi1->location, fi2->location); >>>>>> +???????????? "location is different: fi1 = %" PRId64 " , fi2 = %" >>>>>> PRId64, >>>>>> +???????????? (int64_t)fi1->location, (int64_t)fi2->location); >>>>>> ????? env->FatalError(err_msg); >>>>>> ??? } >>>>>> @@ -67,5 +68,5 @@ >>>>>> ??? if (!is_same) { /* jvmtiStackInfo::thread */ >>>>>> ????? snprintf(err_msg, sizeof(err_msg), >>>>>> -???????????? "thread is different: si1 = %p, si2 = %p", >>>>>> si1->thread, si2->thread); >>>>>> +???????????? "thread is different: si1 = 0x%016" PRIxPTR " , si2 >>>>>> = 0x%016" PRIxPTR, (intptr_t)si1->thread, (intptr_t)si2->thread); >>>>>> ????? env->FatalError(err_msg); >>>>>> ??? } else if (si1->state != si2->state) { /* >>>>>> jvmtiStackInfo::state */ >>>>>> >>>>>> >>>>>> Maybe you can use something like that. >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Patricio >>>>>>> Thanks, >>>>>>> Patricio >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> [1] http://hg.openjdk.java.net/jdk/submit/rev/dfca51958217 >>>>>>>> [2] http://hg.openjdk.java.net/jdk/submit/rev/3665361fa91b >>>>>>>> >>>>>>>> >>>>>>>>> David >>>>>>>>> ----- >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2020/07/09 1:42, Daniel D. Daugherty wrote: >>>>>>>>>>> ?> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ >>>>>>>>>>> >>>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>>> ???? No comments. >>>>>>>>>>> >>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>> ???? L1159: ? Thread *current_thread = Thread::current(); >>>>>>>>>>> ???????? Please add "#ifdef ASSERT" above and "#endif" below >>>>>>>>>>> since >>>>>>>>>>> ???????? current_thread is only used for the assert() in this >>>>>>>>>>> function. >>>>>>>>>>> >>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>> ???? L549: ???????????????????????????? jthread java_thread, >>>>>>>>>>> jint max_frame_count) >>>>>>>>>>> ???? L552: ????? _jthread(java_thread), >>>>>>>>>>> ???????? Please: s/java_thread/thread/ on both lines. >>>>>>>>>>> >>>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>>> ???? No comments. >>>>>>>>>>> >>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>>> >>>>>>>>>>> ???? No comments. >>>>>>>>>>> >>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>>>>>>> >>>>>>>>>>> ???? L27: #include >>>>>>>>>>> ???????? This include is out of order; should be first in the >>>>>>>>>>> list. >>>>>>>>>>> >>>>>>>>>>> ???? This file doesn't compile on my MBP13: >>>>>>>>>>> >>>>>>>>>>> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:49:14: >>>>>>>>>>> error: format specifies type 'unsigned long' but the argument >>>>>>>>>>> has type 'jmethodID' (aka '_jmethodID *') [-Werror,-Wformat] >>>>>>>>>>> ????????????? fi1->method, fi2->method); >>>>>>>>>>> ????????????? ^~~~~~~~~~~ >>>>>>>>>>> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:49:27: >>>>>>>>>>> error: format specifies type 'unsigned long' but the argument >>>>>>>>>>> has type 'jmethodID' (aka '_jmethodID *') [-Werror,-Wformat] >>>>>>>>>>> ????????????? fi1->method, fi2->method); >>>>>>>>>>> ?????????????????????????? ^~~~~~~~~~~ >>>>>>>>>>> ???? 2 errors generated. >>>>>>>>>>> >>>>>>>>>>> ???? This change made it compile on my MBP13, but that may >>>>>>>>>>> break it on >>>>>>>>>>> ???? other platforms: >>>>>>>>>>> >>>>>>>>>>> ???? $ hg diff >>>>>>>>>>> ???? diff -r 560847c69fbe >>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>>>>>>> >>>>>>>>>>> ???? --- >>>>>>>>>>> a/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>>>>>>> Wed Jul 08 12:13:32 2020 -0400 >>>>>>>>>>> ???? +++ >>>>>>>>>>> b/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>>>>>>> Wed Jul 08 12:40:42 2020 -0400 >>>>>>>>>>> ???? @@ -46,7 +46,7 @@ >>>>>>>>>>> ?? ?? ? if (fi1->method != fi2->method) { /* >>>>>>>>>>> jvmtiFrameInfo::method */ >>>>>>>>>>> ?? ?? ??? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>> ?? ? ? ??????????? "method is different: fi1 = %lx, fi2 = %lx", >>>>>>>>>>> ???? -???????????? fi1->method, fi2->method); >>>>>>>>>>> ???? +???????????? (unsigned long) fi1->method, (unsigned >>>>>>>>>>> long) fi2->method); >>>>>>>>>>> ?? ?? ??? env->FatalError(err_msg); >>>>>>>>>>> ?? ?? ? } else if (fi1->location != fi2->location) { /* >>>>>>>>>>> jvmtiFrameInfo::location */ >>>>>>>>>>> ?? ?? ??? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>> >>>>>>>>>>> ???? I'm not sure of the right platform independent way to >>>>>>>>>>> output >>>>>>>>>>> ???? the 'method' field. >>>>>>>>>>> >>>>>>>>>>> Dan >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 7/8/20 4:04 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>> Hi David, >>>>>>>>>>>> >>>>>>>>>>>> On 2020/07/08 15:27, David Holmes wrote: >>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>> >>>>>>>>>>>>> On 7/07/2020 6:54 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Serguei, thank you for replying even though you are on >>>>>>>>>>>>>> vacaiton! >>>>>>>>>>>>>> >>>>>>>>>>>>>> I uploaded new webrev: >>>>>>>>>>>>>> >>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.07/ >>>>>>>>>>>>>> ?? Diff from previous webrev: >>>>>>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/77243b1dcbfe >>>>>>>>>>>>>> >>>>>>>>>>>>>> c'tor of GetSingleStackTraceClosure has jthread argument >>>>>>>>>>>>>> in this webrev. >>>>>>>>>>>>>> Also it does not contain testcase for >>>>>>>>>>>>>> GetThreadListStackTraces with all threads, and >>>>>>>>>>>>>> OneGetThreadListStackTraces would test main thread only. >>>>>>>>>>>>> >>>>>>>>>>>>> All those changes are fine in principle for me. One >>>>>>>>>>>>> nit/suggestion: >>>>>>>>>>>>> >>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>> >>>>>>>>>>>>> ??544?? jthread _java_thread; >>>>>>>>>>>>> >>>>>>>>>>>>> elsewhere "java_thread" refers to a JavaThread, so to avoid >>>>>>>>>>>>> confusion may I suggest this member be named _jthread. >>>>>>>>>>>> >>>>>>>>>>>> I uploaded new webrev: >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ >>>>>>>>>>>> ? Diff from previous webrev: >>>>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/ca6263dbdc87 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> I'm going to be away for the next couple of days - sorry - >>>>>>>>>>>>> but will try to check email on this if I can. >>>>>>>>>>>> >>>>>>>>>>>> Thanks! >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> David >>>>>>>>>>>>> ----- >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 2020/07/07 15:13, David Holmes wrote: >>>>>>>>>>>>>>> On 7/07/2020 2:57 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 2020/07/07 11:31, David Holmes wrote: >>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Hard to keep up with the changes - especially without >>>>>>>>>>>>>>>>> incremental webrevs. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Sorry, I will upload diff from previous webrev in the next. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> If GetSingleStackTraceClosure also took the jthread as >>>>>>>>>>>>>>>>> a constructor arg, then you wouldn't need to recreate a >>>>>>>>>>>>>>>>> JNI local handle when calling _collector.fill_frames. >>>>>>>>>>>>>>>>> It's a small simplification and not essential at this >>>>>>>>>>>>>>>>> stage. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I think we should get jthread from an argument of >>>>>>>>>>>>>>>> do_thread() because do_thread() would pass the thread >>>>>>>>>>>>>>>> which are stopped certainly. >>>>>>>>>>>>>>>> It might be simplification if we pass _calling_thread to >>>>>>>>>>>>>>>> MultipleStackTracesCollector. `jthread` is only needed >>>>>>>>>>>>>>>> to store jvmtiStackInfo.thread . What do you think? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I'm not quite sure what you mean. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I think there is a bit of a design wart with direct >>>>>>>>>>>>>>> handshakes in that do_thread takes the target JavaThread >>>>>>>>>>>>>>> as an argument. That's useful in a case where you want a >>>>>>>>>>>>>>> HandshakeClosure that can be applied to multiple threads, >>>>>>>>>>>>>>> but that's not typically what is needed with direct >>>>>>>>>>>>>>> handshakes - there is only a single target. With a >>>>>>>>>>>>>>> single-target HandshakeClosure you can capture all the >>>>>>>>>>>>>>> "target" information for the operation in the closure >>>>>>>>>>>>>>> instance. So if the actual do_thread operation wants the >>>>>>>>>>>>>>> jthread corresponding to the target thread then we can >>>>>>>>>>>>>>> store that in the closure rather than recomputing it (you >>>>>>>>>>>>>>> could assert it is the same but that seems overkill to me). >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> For the test ... I don't see how >>>>>>>>>>>>>>>>> Java_GetThreadListStackTraces_checkCallStacks is a >>>>>>>>>>>>>>>>> valid test. It gets the stacks of all live threads, >>>>>>>>>>>>>>>>> then uses that information to use >>>>>>>>>>>>>>>>> GetThreadListStackTraces to get the stack for the same >>>>>>>>>>>>>>>>> set of threads through a different API. It then >>>>>>>>>>>>>>>>> compares the two sets of stacks for each thread >>>>>>>>>>>>>>>>> expecting them to be the same, but that need only be >>>>>>>>>>>>>>>>> the case for the main thread. Other threads could >>>>>>>>>>>>>>>>> potentially have a different stack (e.g. if this test >>>>>>>>>>>>>>>>> is run with JFR enabled there will be additional >>>>>>>>>>>>>>>>> threads found.) Further I would have expected that >>>>>>>>>>>>>>>>> there already exist tests that check that, for a given >>>>>>>>>>>>>>>>> thread (which may be suspended or known to be blocked) >>>>>>>>>>>>>>>>> the same stack is found through the two different APIs. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti/unit/GetAllStackTraces/getallstktr001 >>>>>>>>>>>>>>>> would check all of threads via >>>>>>>>>>>>>>>> GetThreadListStackTraces() and GetAllStackTraces(), so >>>>>>>>>>>>>>>> we might be able to remove GetThreadListStackTraces.java >>>>>>>>>>>>>>>> from this webrev. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yes. The existing test only examines a set of test >>>>>>>>>>>>>>> threads that are all blocked on a raw monitor. You do not >>>>>>>>>>>>>>> need to duplicate that test. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> OTOH we don't have testcase for >>>>>>>>>>>>>>>> GetThreadListStackTraces() with thread_count == 1, so we >>>>>>>>>>>>>>>> need to testcase for it (it is >>>>>>>>>>>>>>>> OneGetThreadListStackTraces.java) It would check whether >>>>>>>>>>>>>>>> the state of target thread is "waiting" before JNI call >>>>>>>>>>>>>>>> to call GetThreadListStackTraces(), >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yes we need to test the special cases introduced by your >>>>>>>>>>>>>>> changes - totally agree - and >>>>>>>>>>>>>>> OneGetThreadListStackTraces.java is a good test for that. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> and also I expect it would not be run with JFR. (it is >>>>>>>>>>>>>>>> not described @run) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The arguments to run with JFR (or a bunch of other >>>>>>>>>>>>>>> things) can be passed to the jtreg test harness to be >>>>>>>>>>>>>>> applied to all tests. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Of course we can check GetThreadListStackTraces() with >>>>>>>>>>>>>>>> main thread, but it is not the test for direct handshake >>>>>>>>>>>>>>>> for other thread. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Right - that test already exists as per the above. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> David >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 6/07/2020 11:29 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I think C++ is more simple to implement the test agent >>>>>>>>>>>>>>>>>> as you said. >>>>>>>>>>>>>>>>>> So I implement it in C++ in new webrev. Could you >>>>>>>>>>>>>>>>>> review again? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.06/ >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Also I refactored libGetThreadListStackTraces.cpp, and >>>>>>>>>>>>>>>>>> I've kept exception check after IsSameObject(). >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 2020/07/06 16:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thank you for the update. >>>>>>>>>>>>>>>>>>> I think, a pending exception after IsSameObject needs >>>>>>>>>>>>>>>>>>> to be checked. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> The checkStackInfo() needs one more refactoring as >>>>>>>>>>>>>>>>>>> I've already suggested. >>>>>>>>>>>>>>>>>>> The body of the loop at L68-L78 should be converted >>>>>>>>>>>>>>>>>>> to a function check_frame_info. >>>>>>>>>>>>>>>>>>> The si1->frame_buffer[i] and si2->frame_buffer[i] >>>>>>>>>>>>>>>>>>> will be passed as fi1 and fi2. >>>>>>>>>>>>>>>>>>> The index can be passed as well. >>>>>>>>>>>>>>>>>>> I'm still suggesting to simplify the local >>>>>>>>>>>>>>>>>>> exception_msg to something shorter like err_msg or >>>>>>>>>>>>>>>>>>> exc_msg. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I'm not sure using fatal is right here: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> This fragment looks strange: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ? 152???? if ((*env)->IsSameObject(env, >>>>>>>>>>>>>>>>>>> stack_info[i].thread, thread)) { >>>>>>>>>>>>>>>>>>> ? 153?????? target_info = &stack_info[i]; >>>>>>>>>>>>>>>>>>> ? 154?????? break; >>>>>>>>>>>>>>>>>>> ? 155???? } else if ((*env)->ExceptionOccurred(env)) { >>>>>>>>>>>>>>>>>>> ? 156 (*env)->ExceptionDescribe(env); >>>>>>>>>>>>>>>>>>> ? 157 (*env)->FatalError(env, __FILE__); >>>>>>>>>>>>>>>>>>> ? 158???? } >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I expected it to be: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ??? jboolean same = (*env)->IsSameObject(env, >>>>>>>>>>>>>>>>>>> stack_info[i].thread, thread); >>>>>>>>>>>>>>>>>>> ??? if ((*env)->ExceptionOccurred(env)) { >>>>>>>>>>>>>>>>>>> (*env)->ExceptionDescribe(env); >>>>>>>>>>>>>>>>>>> ????? (*env)->FatalError(env, __FILE__); >>>>>>>>>>>>>>>>>>> ??? } >>>>>>>>>>>>>>>>>>> ??? if (same) { >>>>>>>>>>>>>>>>>>> ????? target_info = &stack_info[i]; >>>>>>>>>>>>>>>>>>> ????? break; >>>>>>>>>>>>>>>>>>> ??? } >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Would it better to port this agent to C++ to simplify >>>>>>>>>>>>>>>>>>> this code nicer? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On 7/5/20 06:13, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>>>>> I refactored testcase. Could you review again? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.05/ >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> It would check Java exception after IsSameObject() >>>>>>>>>>>>>>>>>>>> call. Does it need? >>>>>>>>>>>>>>>>>>>> Any exceptions are not described in JNI document[1], >>>>>>>>>>>>>>>>>>>> and JNI implementation (jni_IsSameObject()) does not >>>>>>>>>>>>>>>>>>>> seem to throw it. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>>>>> https://docs.oracle.com/en/java/javase/14/docs/specs/jni/functions.html#issameobject >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On 2020/07/05 14:46, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Okay, thanks. >>>>>>>>>>>>>>>>>>>>> Then I'm okay to keep the GetSingleStackTraceClosure. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c.html >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c.html >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> I'm not sure the function 'is_same_thread() is needed. >>>>>>>>>>>>>>>>>>>>> Why do not use the JNI IsSameObject instead? >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> It seems to be a typo at L132 and L137. >>>>>>>>>>>>>>>>>>>>> You, probably. did not want to print the same >>>>>>>>>>>>>>>>>>>>> information for stack_info_1[i].frame_buffer[j].XXX >>>>>>>>>>>>>>>>>>>>> twice. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> The code at lines 112-142 is not readable. >>>>>>>>>>>>>>>>>>>>> I'd suggest to make a couple of refactoring steps. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> First step to simplify this a little bit would be >>>>>>>>>>>>>>>>>>>>> with some renaming and getting rid of indexes: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ?? 71 char err_msg[EXCEPTION_MSG_LEN] = {0}; >>>>>>>>>>>>>>>>>>>>> ??... >>>>>>>>>>>>>>>>>>>>> ??112?? /* Iterate all jvmtiStackInfo to check */ >>>>>>>>>>>>>>>>>>>>> ??113?? for (i = 0; i < num_threads, *exception_msg >>>>>>>>>>>>>>>>>>>>> != '\0'; i++) { >>>>>>>>>>>>>>>>>>>>> ????????? jvmtiStackInfo *si1 = stack_info_1[i]; >>>>>>>>>>>>>>>>>>>>> ????????? jvmtiStackInfo *si2 = stack_info_2[i]; >>>>>>>>>>>>>>>>>>>>> ??114???? if (!IsSameObject(env, si1.thread, >>>>>>>>>>>>>>>>>>>>> si2.thread)) { /* jvmtiStackInfo::thread */ >>>>>>>>>>>>>>>>>>>>> ??115?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>>>>> ??116 "thread[%d] is different: stack_info_1 = %p, >>>>>>>>>>>>>>>>>>>>> stack_info_2 = %p", >>>>>>>>>>>>>>>>>>>>> ??117??????????????? i, sinfo1.thread, sinfo2.thread); >>>>>>>>>>>>>>>>>>>>> ??118???? } else if (si1.state != si2.state) { /* >>>>>>>>>>>>>>>>>>>>> jvmtiStackInfo::state */ >>>>>>>>>>>>>>>>>>>>> ??119?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>>>>> ??120 "state[%d] is different: stack_info_1 = %d, >>>>>>>>>>>>>>>>>>>>> stack_info_2 = %d", >>>>>>>>>>>>>>>>>>>>> ??121??????????????? i, si1.state, si2.state); >>>>>>>>>>>>>>>>>>>>> ??122???? } else if (si1.frame_count != >>>>>>>>>>>>>>>>>>>>> si2.frame_count) { /* jvmtiStackInfo::frame_count */ >>>>>>>>>>>>>>>>>>>>> ??123?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>>>>> ??124 "frame_count[%d] is different: stack_info_1 = >>>>>>>>>>>>>>>>>>>>> %d, stack_info_2 = %d", >>>>>>>>>>>>>>>>>>>>> ??125??????????????? i, si1.frame_count, >>>>>>>>>>>>>>>>>>>>> si2.frame_count); >>>>>>>>>>>>>>>>>>>>> ??126???? } else { >>>>>>>>>>>>>>>>>>>>> ??127?????? /* Iterate all jvmtiFrameInfo to check */ >>>>>>>>>>>>>>>>>>>>> ??128?????? for (j = 0; j < si1.frame_count; j++) { >>>>>>>>>>>>>>>>>>>>> ??129???????? if (si1.frame_buffer[j].method != >>>>>>>>>>>>>>>>>>>>> si1.frame_buffer[j].method) { /* >>>>>>>>>>>>>>>>>>>>> jvmtiFrameInfo::method */ >>>>>>>>>>>>>>>>>>>>> ??130 snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>>>>> ??131 "thread [%d] frame_buffer[%d].method is >>>>>>>>>>>>>>>>>>>>> different: stack_info_1 = %lx, stack_info_2 = %lx", >>>>>>>>>>>>>>>>>>>>> ??132??????????????????? i, j, >>>>>>>>>>>>>>>>>>>>> si1.frame_buffer[j].method, >>>>>>>>>>>>>>>>>>>>> si2.frame_buffer[j].method); >>>>>>>>>>>>>>>>>>>>> ??133?????????? break; >>>>>>>>>>>>>>>>>>>>> ??134???????? } else if >>>>>>>>>>>>>>>>>>>>> (si1.frame_buffer[j].location != >>>>>>>>>>>>>>>>>>>>> si1.frame_buffer[j].location) { /* >>>>>>>>>>>>>>>>>>>>> jvmtiFrameInfo::location */ >>>>>>>>>>>>>>>>>>>>> ??135 snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>>>>> ??136 "thread [%d] frame_buffer[%d].location is >>>>>>>>>>>>>>>>>>>>> different: stack_info_1 = %ld, stack_info_2 = %ld", >>>>>>>>>>>>>>>>>>>>> ??137??????????????????? i, j, >>>>>>>>>>>>>>>>>>>>> si1.frame_buffer[j].location, >>>>>>>>>>>>>>>>>>>>> si2.frame_buffer[j].location); >>>>>>>>>>>>>>>>>>>>> ??138?????????? break; >>>>>>>>>>>>>>>>>>>>> ??139???????? } >>>>>>>>>>>>>>>>>>>>> ??140?????? } >>>>>>>>>>>>>>>>>>>>> ??141???? } >>>>>>>>>>>>>>>>>>>>> ??142?? } >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Another step would be to create functions that >>>>>>>>>>>>>>>>>>>>> implement a body of each loop. >>>>>>>>>>>>>>>>>>>>> You can use the same techniques to simplify similar >>>>>>>>>>>>>>>>>>>>> place (L127-L138) in the >>>>>>>>>>>>>>>>>>>>> libOneGetThreadListStackTraces.c. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On 7/3/20 15:55, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I'm not an Oracle employee, so I cannot know real >>>>>>>>>>>>>>>>>>>>>> request(s) from your customers. >>>>>>>>>>>>>>>>>>>>>> However JDK-8201641 says Dynatrace has requested >>>>>>>>>>>>>>>>>>>>>> this enhancement. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> BTW I haven't heared any request from my customers >>>>>>>>>>>>>>>>>>>>>> about this. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> On 2020/07/04 4:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> This difference is not that big to care about. >>>>>>>>>>>>>>>>>>>>>>> I feel this is really rare case and so, does not >>>>>>>>>>>>>>>>>>>>>>> worth these complications. >>>>>>>>>>>>>>>>>>>>>>> Do we have a real request from customers to >>>>>>>>>>>>>>>>>>>>>>> optimize it? >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On 7/3/20 01:16, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Generally I agree with you, but I have concern >>>>>>>>>>>>>>>>>>>>>>>> about the difference of the result of >>>>>>>>>>>>>>>>>>>>>>>> GetStackTrace() and GetThreadListStackTraces(). >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> ? GetStackTrace: jvmtiFrameInfo >>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces: jvmtiStackInfo >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> jvmtiStackInfo contains thread state, and it is >>>>>>>>>>>>>>>>>>>>>>>> ensured it is the state of the call stack. >>>>>>>>>>>>>>>>>>>>>>>> If we want to get both call stack and thread >>>>>>>>>>>>>>>>>>>>>>>> state, we need to suspend target thread, and >>>>>>>>>>>>>>>>>>>>>>>> call both GetStackTrace() and GetThreadState(). >>>>>>>>>>>>>>>>>>>>>>>> Is it ok? >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> I was wondering if JDK-8201641 (parent ticket of >>>>>>>>>>>>>>>>>>>>>>>> this change) needed them for profiling (dynatrace?) >>>>>>>>>>>>>>>>>>>>>>>> If it is responsibility of JVMTI agent >>>>>>>>>>>>>>>>>>>>>>>> implementor, I remove this closure. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 16:45, serguei.spitsyn at oracle.com >>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> After some thinking I've concluded that I do >>>>>>>>>>>>>>>>>>>>>>>>> not like this optimization >>>>>>>>>>>>>>>>>>>>>>>>> of the GetThreadListStackTraces with >>>>>>>>>>>>>>>>>>>>>>>>> GetSingleStackTraceClosure. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> We may need more opinions on this but these are >>>>>>>>>>>>>>>>>>>>>>>>> my points: >>>>>>>>>>>>>>>>>>>>>>>>> ??- it adds some complexity and ugliness >>>>>>>>>>>>>>>>>>>>>>>>> ??- a win is doubtful because it has to be a >>>>>>>>>>>>>>>>>>>>>>>>> rare case, so that total overhead should not be >>>>>>>>>>>>>>>>>>>>>>>>> high >>>>>>>>>>>>>>>>>>>>>>>>> ??- if it is really high for some use cases >>>>>>>>>>>>>>>>>>>>>>>>> then it is up to the user >>>>>>>>>>>>>>>>>>>>>>>>> ??? to optimize it with using GetStackTrace >>>>>>>>>>>>>>>>>>>>>>>>> instead >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> In such cases with doubtful overhead I usually >>>>>>>>>>>>>>>>>>>>>>>>> prefer the simplicity. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Good examples where it makes sense to optimize >>>>>>>>>>>>>>>>>>>>>>>>> are checks for target thread to be current thread. >>>>>>>>>>>>>>>>>>>>>>>>> In such cases there is no need to suspend the >>>>>>>>>>>>>>>>>>>>>>>>> target thread, or use a VMop/HandshakeClosure. >>>>>>>>>>>>>>>>>>>>>>>>> For instance, please, see the Monitor functions >>>>>>>>>>>>>>>>>>>>>>>>> with the check: (java_thread == calling_thread). >>>>>>>>>>>>>>>>>>>>>>>>> Getting information for current thread is >>>>>>>>>>>>>>>>>>>>>>>>> frequently used case, e.g. to get info at an >>>>>>>>>>>>>>>>>>>>>>>>> event point. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On 7/2/20 23:29, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> Hi Dan, David, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could you review again? >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/ >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> OneGetThreadListStackTraces.java in this >>>>>>>>>>>>>>>>>>>>>>>>>> webrev would wait until thread state is >>>>>>>>>>>>>>>>>>>>>>>>>> transited to "waiting" with spin wait. >>>>>>>>>>>>>>>>>>>>>>>>>> CountDownLatch::await call as Dan pointed is >>>>>>>>>>>>>>>>>>>>>>>>>> fixed in it :) >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Diff from webrev.03: >>>>>>>>>>>>>>>>>>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/c9aeb7001e50 >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 14:15, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 12:24, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 7/2/20 10:50 PM, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry I'm responding here without seeing >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> latest webrev but there is enough context >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think ... >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Dan, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 7:16, Daniel D. Daugherty >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I upload new webrev. Could you review >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L1542: // Get stack trace with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> handshake >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - please add a period at >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the end. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> L1591: *stack_info_ptr = op.stack_info(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? The return parameter should not >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be touched unless the return >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? old L1582: if (err == >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JVMTI_ERROR_NONE) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please restore this check. The return >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> parameter should not >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? be touched unless the return >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But op.stack_info() will return NULL if >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the error is not JVMTI_ERROR_NONE. Are you >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (Dan) concerned about someone passing in a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> non-null/initialized out-pointer that will >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be reset to NULL if there was an error? >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Actually the way we used to test this in >>>>>>>>>>>>>>>>>>>>>>>>>>>>> POSIX tests is to call >>>>>>>>>>>>>>>>>>>>>>>>>>>>> an API with known bad parameters and the >>>>>>>>>>>>>>>>>>>>>>>>>>>>> return parameter ptr >>>>>>>>>>>>>>>>>>>>>>>>>>>>> set to NULL. If the return parameter ptr >>>>>>>>>>>>>>>>>>>>>>>>>>>>> was touched when an >>>>>>>>>>>>>>>>>>>>>>>>>>>>> error should have been detected on an >>>>>>>>>>>>>>>>>>>>>>>>>>>>> earlier parameter, then >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the test failed. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> L1272: ? if (!jt->is_exiting() && >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (thread_oop != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> second expression. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? old L1532: _result = >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? This deletion of the _result >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> field threw me for a minute and then >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? ? ? I figured out that the field is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> init to JVMTI_ERROR_THREAD_NOT_ALIVE >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? in the constructor. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L1553: if (!jt->is_exiting() && >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> second expression. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L64: startSignal.countDown(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? I was expecting this to be a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call to await() instead of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> countDown(). What am I missing here? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? I think this test might be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> passing by accident right now, but... >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Main thread (which call JVMTI functions >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to test) should wait until test thread is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ready. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So main thread would wait startSignal, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and test thread would count down. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No! >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The test thread that previously called >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> obj.wait() now calls latch.await(). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The main thread that previously called >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> obj.notify() now calls latch.countDown(). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The main thread continues to spin until it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sees the target is WAITING before >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> proceeding with the test. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> If I add spin wait to wait until transit >>>>>>>>>>>>>>>>>>>>>>>>>>>> target thread state is WAITING (as >>>>>>>>>>>>>>>>>>>>>>>>>>>> following), we don't need to call >>>>>>>>>>>>>>>>>>>>>>>>>>>> SuspendThread(). >>>>>>>>>>>>>>>>>>>>>>>>>>>> Which is better? >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> The original spin-wait loop checking for >>>>>>>>>>>>>>>>>>>>>>>>>>> WAITING is better because it is the only >>>>>>>>>>>>>>>>>>>>>>>>>>> guarantee that the target thread is blocked >>>>>>>>>>>>>>>>>>>>>>>>>>> where you need it to be. suspending the >>>>>>>>>>>>>>>>>>>>>>>>>>> thread is racy as you don't know exactly >>>>>>>>>>>>>>>>>>>>>>>>>>> where the suspend will hit. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>> /* Wait until the thread state transits to >>>>>>>>>>>>>>>>>>>>>>>>>>>> "waiting" */ >>>>>>>>>>>>>>>>>>>>>>>>>>>> while (th.getState() != Thread.State.WAITING) { >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thread.onSpinWait(); >>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> For simplify, spin wait is prefer to >>>>>>>>>>>>>>>>>>>>>>>>>>>> OneGetThreadListStackTraces.java in webrev.03. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here's the flow as I see it: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - start worker thread >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - startSignal.await() >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - startSignal.countDown() >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - stopSignal.await() >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - worker is now blocked >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - checkCallStacks(th) >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - stopSignal.countDown() >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - worker is now unblocked >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - th.join >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - runs off the end of run() >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - run off the end of main() >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L92: jthreads = (jthread >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *)malloc(sizeof(jthread) * num_threads); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? You don't check for malloc() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> failure. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'jthreads' is allocated but never freed. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L91: result = >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (*jvmti)->SuspendThread(jvmti, thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? Why are you suspending the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread? GetAllStackTraces() and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() do not >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> require the target thread(s) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? to be suspend. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ????????? If you decide not to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SuspendThread, then you don't need the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AddCapabilities or the ResumeThread calls. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Test thread might not be entered >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> following code (stopSignal.await()). We >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> might see deferent call stack between >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetAllStackTraces() and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces(). We cannot >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> control to freeze call stack of test >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread in Java code. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (I didn't use SuspendThread() at first, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but I saw some errors which causes in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> above.) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So we need to call SuspendThread() to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ensure we can see same call stack. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you are checking that the thread is in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state WAITING then it cannot escape from >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that state and you can sample the stack >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> multiple times from any API and get the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> same result. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I suspect the errors you saw were from the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> apparent incorrect use of the CountDownLatch. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> With the flow outlined above, the worker >>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread should be >>>>>>>>>>>>>>>>>>>>>>>>>>>>> nicely blocked in stopSignal.await() when >>>>>>>>>>>>>>>>>>>>>>>>>>>>> stuff is sampled. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 11:53 am, Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could review >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Updates look fine - thanks. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> One minor nit: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1274 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector.allocate_and_fill_stacks(1); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1275 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector.set_result(JVMTI_ERROR_NONE); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the other places where you use >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector you rely on result being >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> initialized to JVMTI_ERROR_NONE, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather than setting it directly after >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> allocate_and_fill_stacks(). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 820 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> assert(SafepointSynchronize::is_at_safepoint() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 821 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 822 current_thread == >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 823 "at safepoint / handshake >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> check is necessary, as even if >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the target is suspended we must >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> still be at a safepoint or in a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> handshake with it. Makes me >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wonder if we used to allow a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> racy stacktrace operation on a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> suspended thread, assuming it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This function >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (JvmtiEnvBase::get_stack_trace()) can >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be called to get own stack trace. For >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> example, we can call GetStackTrace() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for current thread at JVMTI event. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So I changed assert as below: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??820 assert(current_thread == >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??821 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SafepointSynchronize::is_at_safepoint() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??823 "call by myself / at safepoint >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> / at handshake"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yep good catch. I hope current tests >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> caught that. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> They would be tested in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (own call stacks), and getstacktr003 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (call stacks in other thread). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Speaking of tests ... >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the native code I think you need to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> check the success of all JNI methods >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that can throw exceptions - otherwise >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe the tests may trigger >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> warnings if -Xcheck:jni is used with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them. See for example: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I updated testcases to check JNI and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JVMTI function calls. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the Java code the target thread: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 45 public void run() { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 46 try { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 47 synchronized (lock) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 48 lock.wait(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 49 System.out.println("OK"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 50?????????? } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is potentially susceptible to spurious >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wakeups. Using a CountDownLatch would >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be robust. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 9:05 am, Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _calling_thread is the current >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If so, we can call make_local() in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> L1272 without JavaThread (or we can >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pass current thread to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make_local()). Is it right? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry I got confused, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _calling_thread may not be the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> current thread as we could be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> executing the handshake in the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> target thread itself. So the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark is correct as-is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (implicitly for current thread). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The argument to fill_frames will be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> used in the jvmtiStackInfo and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> passed back to the _calling_thread, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it must be created via >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make_local(_calling_thread, ...) as >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you presently have. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 7:05, David Holmes >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you for reviewing! I will >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> update new webrev tomorrow. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 498 class VM_GetAllStackTraces >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 500 JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 502 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> member of another class like >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that as it may not be on the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stack. I think >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should not extend any allocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class, and should always be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure what does mean >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "embedded". >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??? : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class GetAllStackTraces : public >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/30 22:22, David Holmes >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I updated webrev for 8242428. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Could you review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change migrate to use >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> direct handshake for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetStackTrace() and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (when thread_count == 1). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks really good now! I >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only have a few nits below. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> There is one thing I don't like >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about it but it requires a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change to the main Handshake >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic to address - in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JvmtiEnv::GetThreadListStackTraces >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you have to create a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ThreadsListHandle to convert the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jthread to a JavaThread, but >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Handshake::execute_direct >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> creates another >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ThreadsListHandle internally. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That's a waste. I will discuss >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with Robbin and file a RFE to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have an overload of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> execute_direct that takes an >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> existing TLH. Actually it's >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worse than that because we have >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> another TLH in use at the entry >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> point for the JVMTI functions, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so I think there may be some >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scope for simplifying the use of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> TLH instances - future RFE. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??451 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetStackTraceClosure(JvmtiEnv >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *env, jint start_depth, jint >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max_count, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??452 jvmtiFrameInfo* >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> frame_buffer, jint* count_ptr) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??453???? : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??454 _env(env), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _start_depth(start_depth), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _max_count(max_count), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??455 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _frame_buffer(frame_buffer), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _count_ptr(count_ptr), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??456 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _result(JVMTI_ERROR_THREAD_NOT_ALIVE) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Nit: can you do one initializer >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> per line please. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??498 class VM_GetAllStackTraces >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??500 JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??502 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> member of another class like >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that as it may not be on the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stack. I think >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should not extend any allocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class, and should always be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 481 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector(JvmtiEnv >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *env, jint max_frame_count) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??483 _max_frame_count = >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> max_frame_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??484 _frame_count_total = 0; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??485 _head = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??486 _stack_info = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??487 _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As you are touching this can you >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change it to use an initializer >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> list as you did for the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HandshakeClosure, and please >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> keep one item per line. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??820 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> assert(SafepointSynchronize::is_at_safepoint() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??821 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->is_thread_fully_suspended(false, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??823 "at safepoint / handshake >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> check is necessary, as even if >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the target is suspended we must >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> still be at a safepoint or in a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> handshake with it. Makes me >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wonder if we used to allow a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> racy stacktrace operation on a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> suspended thread, assuming it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1268?? oop thread_oop = >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jt->threadObj(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1269 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can use thread_oop in line >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1270. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is frustrating that this >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entire call chain started with a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jthread reference, which we >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> converted to a JavaThread, only >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to eventually need to convert it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> back to a jthread! I think there >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is some scope for simplification >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> here but not as part of this >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _calling_thread is the current >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm still pondering the test >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logic but wanted to send this now. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetThreadListStackTrace (for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces) and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetAllStackTraces (for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetAllStackTraces) have >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> inherited >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetMultipleStackTraces VM >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> operation which provides the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature to generate >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jvmtiStackInfo. I modified >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetMultipleStackTraces to a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> normal C++ class to share with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HandshakeClosure for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (GetSingleStackTraceClosure). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I added new testcases >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which test >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread_count == 1 and with all >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> threads. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jvmti >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jdwp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jdi >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? JBS: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? webrev: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change replace following >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM operations to direct >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> handshake. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameCount >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (GetFrameCount()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (GetFrameLocation()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetThreadListStackTraces >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (GetThreadListStackTrace()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTrace() uses >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> direct handshake if thread >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> count == 1. In other case >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (thread count > 1), it would >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be performed as VM operation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Caller of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (JvmtiEnvThreadState::reset_current_location()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> might be called at safepoint. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So I added safepoint check in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> its caller. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jvmti >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> serviceability/jdwp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vmTestbase/nsk/jdi vmTestbase/ns >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I tested it on submit >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repo, then it has execution >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> error >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> due to dependency error. So I >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> think it does not occur by >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>> >>>>>>> >>>>>> >>>> >> From suenaga at oss.nttdata.com Sat Jul 11 14:22:38 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Sat, 11 Jul 2020 23:22:38 +0900 Subject: RFR: 8242428: JVMTI thread operations should use Thread-Local Handshake In-Reply-To: References: <6c263e4c-0a83-be7a-0288-ecb2de0b7cea@oss.nttdata.com> <19cfa2a7-f441-425d-5623-1c11c6a870b6@oss.nttdata.com> <237e8d26-d185-1008-6e07-5b9f36314534@oracle.com> <2a9ba1f8-b1f7-cb8c-fe11-56dd174d9d58@oracle.com> <7ad907cb-24ef-aa55-972a-0a3f63576568@oracle.com> <5ab8c5d9-9a5e-74a6-4605-7e73732037bf@oracle.com> <4881e0da-f716-c62e-b744-17aa3d03e095@oss.nttdata.com> <8dfa047c-8f84-5775-d12d-3248137e4b45@oss.nttdata.com> Message-ID: <4fe63d0b-7515-6296-79d6-2f993d9201eb@oss.nttdata.com> On 2020/07/11 22:50, David Holmes wrote: > On 11/07/2020 11:52 am, Yasumasa Suenaga wrote: >> Thanks Dan! >> >> David, Serguei, are you ok to this change? > > Yes it seems fine. Thanks David! > In relation to an earlier comment: > > > I replaced %p to %lx, and also cast values to unsigned long > > Never use long or unsigned long as they can be different sizes on different platforms (specifically long is not 64-bit on Windows). I had forgotten that Windows x64 is LLP64 :) Yasumasa > Thanks, > David > ----- > >> >> Yasumasa >> >> >> On 2020/07/11 7:00, Daniel D. Daugherty wrote: >>> On 7/10/20 3:29 AM, Yasumasa Suenaga wrote: >>>> Thanks Patricio! >>>> >>>> I uploaded new webrev. Could you review again? >>>> It passed all tests on submit repo, and passed jtreg tests about JVMTI. >>>> >>>> ? http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.10/ >>>> ? Diff from webrev.09: http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.10/incremental/ >>> >>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>> ???? No comments. >>> >>> Thumbs up on the incremental. >>> >>> Dan >>> >>> >>>> >>>> >>>> Yasumasa >>>> >>>> >>>> On 2020/07/10 14:49, Patricio Chilano wrote: >>>>> Hi Yasumasa, >>>>> >>>>> On 7/10/20 1:58 AM, Yasumasa Suenaga wrote: >>>>>> Hi Patricio, >>>>>> >>>>>> Thanks for your advice! >>>>>> I've fixed testcase as [1], but I still see an error in validate-headers-linux-x64-build-1 on submit repo. >>>>>> What does it mean? Can you share details? >>>>>> >>>>>> ? mach5-one-ysuenaga-JDK-8242428-20200710-0339-12529134 >>>>> Not your bug, you are getting the header validation failure from 8248570. It was fixed two days ago so just update the repo and it should work now. >>>>> >>>>> Thanks, >>>>> Patricio >>>>>> Of course this change can be built on my Linux box (Fedora 32, AMD64, GCC 10.1) >>>>>> >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> [1] http://hg.openjdk.java.net/jdk/submit/file/45f52142db9d/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>> >>>>>> >>>>>> On 2020/07/10 2:18, Patricio Chilano wrote: >>>>>>> >>>>>>> On 7/9/20 12:00 PM, Patricio Chilano wrote: >>>>>>>> Hi Yasumasa, >>>>>>>> >>>>>>>> On 7/9/20 9:30 AM, Yasumasa Suenaga wrote: >>>>>>>>> On 2020/07/09 17:58, David Holmes wrote: >>>>>>>>>> Hi Yasumasa, >>>>>>>>>> >>>>>>>>>> On 9/07/2020 10:25 am, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi Dan, >>>>>>>>>>> >>>>>>>>>>> Thanks for your comment! >>>>>>>>>>> I uploaded new webrev: >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.09/ >>>>>>>>>>> ?? Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/5d167adf8524 >>>>>>>>>>> >>>>>>>>>>> I saw similar build errors in libOneGetThreadListStackTraces.cpp on Windows. >>>>>>>>>>> This webrev fixes them. >>>>>>>>>> >>>>>>>>>> You shouldn't use %p as it may not be portable. In the VM we use INTPTR_FORMAT and convert the arg using p2i. I don't know what exists in the testing code. >>>>>>>>> >>>>>>>>> I replaced %p to %lx, and also cast values to unsigned long [1] [2], but the test on submit repo was failed. >>>>>>>>> Can anyone share details of mach5-one-ysuenaga-JDK-8242428-20200709-1030-12500928 ? >>>>>>>> These are the errors I see for the macOS build: >>>>>>>> >>>>>>>> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:53:14: error: format specifies type 'long long' but the argument has type 'jlocation' (aka 'long') [-Werror,-Wformat] >>>>>>>> ????????????? fi1->location, fi2->location); >>>>>>>> ????????????? ^~~~~~~~~~~~~ >>>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:53:29: error: format specifies type 'long long' but the argument has type 'jlocation' (aka 'long') [-Werror,-Wformat] >>>>>>>> ????????????? fi1->location, fi2->location); >>>>>>>> ???????????????????????????? ^~~~~~~~~~~~~ >>>>>>>> >>>>>>>> These are the ones I see for the Windows build: >>>>>>>> >>>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): warning C4311: 'type cast': pointer truncation from 'jmethodID' to 'unsigned long' >>>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): warning C4302: 'type cast': truncation from 'jmethodID' to 'unsigned long' >>>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): warning C4311: 'type cast': pointer truncation from 'jmethodID' to 'unsigned long' >>>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(48): warning C4302: 'type cast': truncation from 'jmethodID' to 'unsigned long' >>>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): warning C4311: 'type cast': pointer truncation from 'jthread' to 'unsigned long' >>>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): warning C4302: 'type cast': truncation from 'jthread' to 'unsigned long' >>>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): warning C4311: 'type cast': pointer truncation from 'jthread' to 'unsigned long' >>>>>>>> ?./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp(70): warning C4302: 'type cast': truncation from 'jthread' to 'unsigned long' >>>>>>>> >>>>>>>> >>>>>>>> You will probably want to use the macros defined in src/hotspot/share/utilities/globalDefinitions.hpp. Let me know if you need me to test something. >>>>>>> With these changes the build works okay on Linux, Windows and macOS: >>>>>>> >>>>>>> >>>>>>> --- a/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>>> +++ b/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>>> @@ -27,4 +27,5 @@ >>>>>>> ??#include >>>>>>> ??#include >>>>>>> +#include >>>>>>> >>>>>>> ??#define MAX_FRAMES 100 >>>>>>> @@ -45,11 +46,11 @@ >>>>>>> ??? if (fi1->method != fi2->method) { /* jvmtiFrameInfo::method */ >>>>>>> ????? snprintf(err_msg, sizeof(err_msg), >>>>>>> -???????????? "method is different: fi1 = %p, fi2 = %p", >>>>>>> -???????????? fi1->method, fi2->method); >>>>>>> +???????????? "method is different: fi1 = 0x%016" PRIxPTR " , fi2 = 0x%016" PRIxPTR, >>>>>>> +???????????? (intptr_t)fi1->method, (intptr_t)fi2->method); >>>>>>> ????? env->FatalError(err_msg); >>>>>>> ??? } else if (fi1->location != fi2->location) { /* jvmtiFrameInfo::location */ >>>>>>> ????? snprintf(err_msg, sizeof(err_msg), >>>>>>> -???????????? "location is different: fi1 = %lld, fi2 = %lld", >>>>>>> -???????????? fi1->location, fi2->location); >>>>>>> +???????????? "location is different: fi1 = %" PRId64 " , fi2 = %" PRId64, >>>>>>> +???????????? (int64_t)fi1->location, (int64_t)fi2->location); >>>>>>> ????? env->FatalError(err_msg); >>>>>>> ??? } >>>>>>> @@ -67,5 +68,5 @@ >>>>>>> ??? if (!is_same) { /* jvmtiStackInfo::thread */ >>>>>>> ????? snprintf(err_msg, sizeof(err_msg), >>>>>>> -???????????? "thread is different: si1 = %p, si2 = %p", si1->thread, si2->thread); >>>>>>> +???????????? "thread is different: si1 = 0x%016" PRIxPTR " , si2 = 0x%016" PRIxPTR, (intptr_t)si1->thread, (intptr_t)si2->thread); >>>>>>> ????? env->FatalError(err_msg); >>>>>>> ??? } else if (si1->state != si2->state) { /* jvmtiStackInfo::state */ >>>>>>> >>>>>>> >>>>>>> Maybe you can use something like that. >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Patricio >>>>>>>> Thanks, >>>>>>>> Patricio >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> [1] http://hg.openjdk.java.net/jdk/submit/rev/dfca51958217 >>>>>>>>> [2] http://hg.openjdk.java.net/jdk/submit/rev/3665361fa91b >>>>>>>>> >>>>>>>>> >>>>>>>>>> David >>>>>>>>>> ----- >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2020/07/09 1:42, Daniel D. Daugherty wrote: >>>>>>>>>>>> ?> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ >>>>>>>>>>>> >>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>>>> ???? No comments. >>>>>>>>>>>> >>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>> ???? L1159: ? Thread *current_thread = Thread::current(); >>>>>>>>>>>> ???????? Please add "#ifdef ASSERT" above and "#endif" below since >>>>>>>>>>>> ???????? current_thread is only used for the assert() in this function. >>>>>>>>>>>> >>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>> ???? L549: ???????????????????????????? jthread java_thread, jint max_frame_count) >>>>>>>>>>>> ???? L552: ????? _jthread(java_thread), >>>>>>>>>>>> ???????? Please: s/java_thread/thread/ on both lines. >>>>>>>>>>>> >>>>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>>>> ???? No comments. >>>>>>>>>>>> >>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>>>> ???? No comments. >>>>>>>>>>>> >>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>>>>>>>> ???? L27: #include >>>>>>>>>>>> ???????? This include is out of order; should be first in the list. >>>>>>>>>>>> >>>>>>>>>>>> ???? This file doesn't compile on my MBP13: >>>>>>>>>>>> >>>>>>>>>>>> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:49:14: error: format specifies type 'unsigned long' but the argument has type 'jmethodID' (aka '_jmethodID *') [-Werror,-Wformat] >>>>>>>>>>>> ????????????? fi1->method, fi2->method); >>>>>>>>>>>> ????????????? ^~~~~~~~~~~ >>>>>>>>>>>> ./open/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp:49:27: error: format specifies type 'unsigned long' but the argument has type 'jmethodID' (aka '_jmethodID *') [-Werror,-Wformat] >>>>>>>>>>>> ????????????? fi1->method, fi2->method); >>>>>>>>>>>> ?????????????????????????? ^~~~~~~~~~~ >>>>>>>>>>>> ???? 2 errors generated. >>>>>>>>>>>> >>>>>>>>>>>> ???? This change made it compile on my MBP13, but that may break it on >>>>>>>>>>>> ???? other platforms: >>>>>>>>>>>> >>>>>>>>>>>> ???? $ hg diff >>>>>>>>>>>> ???? diff -r 560847c69fbe test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp >>>>>>>>>>>> ???? --- a/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp Wed Jul 08 12:13:32 2020 -0400 >>>>>>>>>>>> ???? +++ b/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.cpp Wed Jul 08 12:40:42 2020 -0400 >>>>>>>>>>>> ???? @@ -46,7 +46,7 @@ >>>>>>>>>>>> ?? ?? ? if (fi1->method != fi2->method) { /* jvmtiFrameInfo::method */ >>>>>>>>>>>> ?? ?? ??? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>> ?? ? ? ??????????? "method is different: fi1 = %lx, fi2 = %lx", >>>>>>>>>>>> ???? -???????????? fi1->method, fi2->method); >>>>>>>>>>>> ???? +???????????? (unsigned long) fi1->method, (unsigned long) fi2->method); >>>>>>>>>>>> ?? ?? ??? env->FatalError(err_msg); >>>>>>>>>>>> ?? ?? ? } else if (fi1->location != fi2->location) { /* jvmtiFrameInfo::location */ >>>>>>>>>>>> ?? ?? ??? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>> >>>>>>>>>>>> ???? I'm not sure of the right platform independent way to output >>>>>>>>>>>> ???? the 'method' field. >>>>>>>>>>>> >>>>>>>>>>>> Dan >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 7/8/20 4:04 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>> Hi David, >>>>>>>>>>>>> >>>>>>>>>>>>> On 2020/07/08 15:27, David Holmes wrote: >>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 7/07/2020 6:54 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Serguei, thank you for replying even though you are on vacaiton! >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I uploaded new webrev: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.07/ >>>>>>>>>>>>>>> ?? Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/77243b1dcbfe >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> c'tor of GetSingleStackTraceClosure has jthread argument in this webrev. >>>>>>>>>>>>>>> Also it does not contain testcase for GetThreadListStackTraces with all threads, and OneGetThreadListStackTraces would test main thread only. >>>>>>>>>>>>>> >>>>>>>>>>>>>> All those changes are fine in principle for me. One nit/suggestion: >>>>>>>>>>>>>> >>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>> >>>>>>>>>>>>>> ??544?? jthread _java_thread; >>>>>>>>>>>>>> >>>>>>>>>>>>>> elsewhere "java_thread" refers to a JavaThread, so to avoid confusion may I suggest this member be named _jthread. >>>>>>>>>>>>> >>>>>>>>>>>>> I uploaded new webrev: >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.08/ >>>>>>>>>>>>> ? Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/ca6263dbdc87 >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> I'm going to be away for the next couple of days - sorry - but will try to check email on this if I can. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks! >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> David >>>>>>>>>>>>>> ----- >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 2020/07/07 15:13, David Holmes wrote: >>>>>>>>>>>>>>>> On 7/07/2020 2:57 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 2020/07/07 11:31, David Holmes wrote: >>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Hard to keep up with the changes - especially without incremental webrevs. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Sorry, I will upload diff from previous webrev in the next. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> If GetSingleStackTraceClosure also took the jthread as a constructor arg, then you wouldn't need to recreate a JNI local handle when calling _collector.fill_frames. It's a small simplification and not essential at this stage. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I think we should get jthread from an argument of do_thread() because do_thread() would pass the thread which are stopped certainly. >>>>>>>>>>>>>>>>> It might be simplification if we pass _calling_thread to MultipleStackTracesCollector. `jthread` is only needed to store jvmtiStackInfo.thread . What do you think? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I'm not quite sure what you mean. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I think there is a bit of a design wart with direct handshakes in that do_thread takes the target JavaThread as an argument. That's useful in a case where you want a HandshakeClosure that can be applied to multiple threads, but that's not typically what is needed with direct handshakes - there is only a single target. With a single-target HandshakeClosure you can capture all the "target" information for the operation in the closure instance. So if the actual do_thread operation wants the jthread corresponding to the target thread then we can store that in the closure rather than recomputing it (you could assert it is the same but that seems overkill to me). >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> For the test ... I don't see how Java_GetThreadListStackTraces_checkCallStacks is a valid test. It gets the stacks of all live threads, then uses that information to use GetThreadListStackTraces to get the stack for the same set of threads through a different API. It then compares the two sets of stacks for each thread expecting them to be the same, but that need only be the case for the main thread. Other threads could potentially have a different stack (e.g. if this test is run with JFR enabled there will be additional threads found.) Further I would have expected that there already exist tests that check that, for a given thread (which may be suspended or known to be blocked) the same stack is found through the two different APIs. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> vmTestbase/nsk/jvmti/unit/GetAllStackTraces/getallstktr001 would check all of threads via GetThreadListStackTraces() and GetAllStackTraces(), so we might be able to remove GetThreadListStackTraces.java from this webrev. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Yes. The existing test only examines a set of test threads that are all blocked on a raw monitor. You do not need to duplicate that test. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> OTOH we don't have testcase for GetThreadListStackTraces() with thread_count == 1, so we need to testcase for it (it is OneGetThreadListStackTraces.java) It would check whether the state of target thread is "waiting" before JNI call to call GetThreadListStackTraces(), >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Yes we need to test the special cases introduced by your changes - totally agree - and OneGetThreadListStackTraces.java is a good test for that. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> and also I expect it would not be run with JFR. (it is not described @run) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> The arguments to run with JFR (or a bunch of other things) can be passed to the jtreg test harness to be applied to all tests. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Of course we can check GetThreadListStackTraces() with main thread, but it is not the test for direct handshake for other thread. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Right - that test already exists as per the above. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 6/07/2020 11:29 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I think C++ is more simple to implement the test agent as you said. >>>>>>>>>>>>>>>>>>> So I implement it in C++ in new webrev. Could you review again? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.06/ >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Also I refactored libGetThreadListStackTraces.cpp, and I've kept exception check after IsSameObject(). >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On 2020/07/06 16:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thank you for the update. >>>>>>>>>>>>>>>>>>>> I think, a pending exception after IsSameObject needs to be checked. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> The checkStackInfo() needs one more refactoring as I've already suggested. >>>>>>>>>>>>>>>>>>>> The body of the loop at L68-L78 should be converted to a function check_frame_info. >>>>>>>>>>>>>>>>>>>> The si1->frame_buffer[i] and si2->frame_buffer[i] will be passed as fi1 and fi2. >>>>>>>>>>>>>>>>>>>> The index can be passed as well. >>>>>>>>>>>>>>>>>>>> I'm still suggesting to simplify the local exception_msg to something shorter like err_msg or exc_msg. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I'm not sure using fatal is right here: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> This fragment looks strange: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> ? 152???? if ((*env)->IsSameObject(env, stack_info[i].thread, thread)) { >>>>>>>>>>>>>>>>>>>> ? 153?????? target_info = &stack_info[i]; >>>>>>>>>>>>>>>>>>>> ? 154?????? break; >>>>>>>>>>>>>>>>>>>> ? 155???? } else if ((*env)->ExceptionOccurred(env)) { >>>>>>>>>>>>>>>>>>>> ? 156 (*env)->ExceptionDescribe(env); >>>>>>>>>>>>>>>>>>>> ? 157 (*env)->FatalError(env, __FILE__); >>>>>>>>>>>>>>>>>>>> ? 158???? } >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I expected it to be: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> ??? jboolean same = (*env)->IsSameObject(env, stack_info[i].thread, thread); >>>>>>>>>>>>>>>>>>>> ??? if ((*env)->ExceptionOccurred(env)) { >>>>>>>>>>>>>>>>>>>> (*env)->ExceptionDescribe(env); >>>>>>>>>>>>>>>>>>>> ????? (*env)->FatalError(env, __FILE__); >>>>>>>>>>>>>>>>>>>> ??? } >>>>>>>>>>>>>>>>>>>> ??? if (same) { >>>>>>>>>>>>>>>>>>>> ????? target_info = &stack_info[i]; >>>>>>>>>>>>>>>>>>>> ????? break; >>>>>>>>>>>>>>>>>>>> ??? } >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Would it better to port this agent to C++ to simplify this code nicer? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On 7/5/20 06:13, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>>>>>> I refactored testcase. Could you review again? >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.05/ >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> It would check Java exception after IsSameObject() call. Does it need? >>>>>>>>>>>>>>>>>>>>> Any exceptions are not described in JNI document[1], and JNI implementation (jni_IsSameObject()) does not seem to throw it. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> [1] https://docs.oracle.com/en/java/javase/14/docs/specs/jni/functions.html#issameobject >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On 2020/07/05 14:46, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Okay, thanks. >>>>>>>>>>>>>>>>>>>>>> Then I'm okay to keep the GetSingleStackTraceClosure. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c.html >>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c.html >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I'm not sure the function 'is_same_thread() is needed. >>>>>>>>>>>>>>>>>>>>>> Why do not use the JNI IsSameObject instead? >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> It seems to be a typo at L132 and L137. >>>>>>>>>>>>>>>>>>>>>> You, probably. did not want to print the same information for stack_info_1[i].frame_buffer[j].XXX twice. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> The code at lines 112-142 is not readable. >>>>>>>>>>>>>>>>>>>>>> I'd suggest to make a couple of refactoring steps. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> First step to simplify this a little bit would be with some renaming and getting rid of indexes: >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> ?? 71 char err_msg[EXCEPTION_MSG_LEN] = {0}; >>>>>>>>>>>>>>>>>>>>>> ??... >>>>>>>>>>>>>>>>>>>>>> ??112?? /* Iterate all jvmtiStackInfo to check */ >>>>>>>>>>>>>>>>>>>>>> ??113?? for (i = 0; i < num_threads, *exception_msg != '\0'; i++) { >>>>>>>>>>>>>>>>>>>>>> ????????? jvmtiStackInfo *si1 = stack_info_1[i]; >>>>>>>>>>>>>>>>>>>>>> ????????? jvmtiStackInfo *si2 = stack_info_2[i]; >>>>>>>>>>>>>>>>>>>>>> ??114???? if (!IsSameObject(env, si1.thread, si2.thread)) { /* jvmtiStackInfo::thread */ >>>>>>>>>>>>>>>>>>>>>> ??115?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>>>>>> ??116 "thread[%d] is different: stack_info_1 = %p, stack_info_2 = %p", >>>>>>>>>>>>>>>>>>>>>> ??117??????????????? i, sinfo1.thread, sinfo2.thread); >>>>>>>>>>>>>>>>>>>>>> ??118???? } else if (si1.state != si2.state) { /* jvmtiStackInfo::state */ >>>>>>>>>>>>>>>>>>>>>> ??119?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>>>>>> ??120 "state[%d] is different: stack_info_1 = %d, stack_info_2 = %d", >>>>>>>>>>>>>>>>>>>>>> ??121??????????????? i, si1.state, si2.state); >>>>>>>>>>>>>>>>>>>>>> ??122???? } else if (si1.frame_count != si2.frame_count) { /* jvmtiStackInfo::frame_count */ >>>>>>>>>>>>>>>>>>>>>> ??123?????? snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>>>>>> ??124 "frame_count[%d] is different: stack_info_1 = %d, stack_info_2 = %d", >>>>>>>>>>>>>>>>>>>>>> ??125??????????????? i, si1.frame_count, si2.frame_count); >>>>>>>>>>>>>>>>>>>>>> ??126???? } else { >>>>>>>>>>>>>>>>>>>>>> ??127?????? /* Iterate all jvmtiFrameInfo to check */ >>>>>>>>>>>>>>>>>>>>>> ??128?????? for (j = 0; j < si1.frame_count; j++) { >>>>>>>>>>>>>>>>>>>>>> ??129???????? if (si1.frame_buffer[j].method != si1.frame_buffer[j].method) { /* jvmtiFrameInfo::method */ >>>>>>>>>>>>>>>>>>>>>> ??130 snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>>>>>> ??131 "thread [%d] frame_buffer[%d].method is different: stack_info_1 = %lx, stack_info_2 = %lx", >>>>>>>>>>>>>>>>>>>>>> ??132??????????????????? i, j, si1.frame_buffer[j].method, si2.frame_buffer[j].method); >>>>>>>>>>>>>>>>>>>>>> ??133?????????? break; >>>>>>>>>>>>>>>>>>>>>> ??134???????? } else if (si1.frame_buffer[j].location != si1.frame_buffer[j].location) { /* jvmtiFrameInfo::location */ >>>>>>>>>>>>>>>>>>>>>> ??135 snprintf(err_msg, sizeof(err_msg), >>>>>>>>>>>>>>>>>>>>>> ??136 "thread [%d] frame_buffer[%d].location is different: stack_info_1 = %ld, stack_info_2 = %ld", >>>>>>>>>>>>>>>>>>>>>> ??137??????????????????? i, j, si1.frame_buffer[j].location, si2.frame_buffer[j].location); >>>>>>>>>>>>>>>>>>>>>> ??138?????????? break; >>>>>>>>>>>>>>>>>>>>>> ??139???????? } >>>>>>>>>>>>>>>>>>>>>> ??140?????? } >>>>>>>>>>>>>>>>>>>>>> ??141???? } >>>>>>>>>>>>>>>>>>>>>> ??142?? } >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Another step would be to create functions that implement a body of each loop. >>>>>>>>>>>>>>>>>>>>>> You can use the same techniques to simplify similar place (L127-L138) in the libOneGetThreadListStackTraces.c. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> On 7/3/20 15:55, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I'm not an Oracle employee, so I cannot know real request(s) from your customers. >>>>>>>>>>>>>>>>>>>>>>> However JDK-8201641 says Dynatrace has requested this enhancement. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> BTW I haven't heared any request from my customers about this. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On 2020/07/04 4:32, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> This difference is not that big to care about. >>>>>>>>>>>>>>>>>>>>>>>> I feel this is really rare case and so, does not worth these complications. >>>>>>>>>>>>>>>>>>>>>>>> Do we have a real request from customers to optimize it? >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On 7/3/20 01:16, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>> Hi Serguei, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Generally I agree with you, but I have concern about the difference of the result of GetStackTrace() and GetThreadListStackTraces(). >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> ? GetStackTrace: jvmtiFrameInfo >>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces: jvmtiStackInfo >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> jvmtiStackInfo contains thread state, and it is ensured it is the state of the call stack. >>>>>>>>>>>>>>>>>>>>>>>>> If we want to get both call stack and thread state, we need to suspend target thread, and call both GetStackTrace() and GetThreadState(). Is it ok? >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> I was wondering if JDK-8201641 (parent ticket of this change) needed them for profiling (dynatrace?) >>>>>>>>>>>>>>>>>>>>>>>>> If it is responsibility of JVMTI agent implementor, I remove this closure. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 16:45, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> After some thinking I've concluded that I do not like this optimization >>>>>>>>>>>>>>>>>>>>>>>>>> of the GetThreadListStackTraces with GetSingleStackTraceClosure. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> We may need more opinions on this but these are my points: >>>>>>>>>>>>>>>>>>>>>>>>>> ??- it adds some complexity and ugliness >>>>>>>>>>>>>>>>>>>>>>>>>> ??- a win is doubtful because it has to be a rare case, so that total overhead should not be high >>>>>>>>>>>>>>>>>>>>>>>>>> ??- if it is really high for some use cases then it is up to the user >>>>>>>>>>>>>>>>>>>>>>>>>> ??? to optimize it with using GetStackTrace instead >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> In such cases with doubtful overhead I usually prefer the simplicity. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Good examples where it makes sense to optimize are checks for target thread to be current thread. >>>>>>>>>>>>>>>>>>>>>>>>>> In such cases there is no need to suspend the target thread, or use a VMop/HandshakeClosure. >>>>>>>>>>>>>>>>>>>>>>>>>> For instance, please, see the Monitor functions with the check: (java_thread == calling_thread). >>>>>>>>>>>>>>>>>>>>>>>>>> Getting information for current thread is frequently used case, e.g. to get info at an event point. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>> Serguei >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> On 7/2/20 23:29, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Dan, David, >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could you review again? >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.04/ >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> OneGetThreadListStackTraces.java in this webrev would wait until thread state is transited to "waiting" with spin wait. >>>>>>>>>>>>>>>>>>>>>>>>>>> CountDownLatch::await call as Dan pointed is fixed in it :) >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Diff from webrev.03: >>>>>>>>>>>>>>>>>>>>>>>>>>> http://hg.openjdk.java.net/jdk/submit/rev/c9aeb7001e50 >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 14:15, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On 3/07/2020 2:27 pm, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 12:24, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 7/2/20 10:50 PM, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry I'm responding here without seeing latest webrev but there is enough context I think ... >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 3/07/2020 9:14 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Dan, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for your comment! >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/03 7:16, Daniel D. Daugherty wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 7/2/20 5:19 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I upload new webrev. Could you review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.03/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnv.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L1542: // Get stack trace with handshake >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - please add a period at the end. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> L1591: *stack_info_ptr = op.stack_info(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? The return parameter should not be touched unless the return >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? old L1582: if (err == JVMTI_ERROR_NONE) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please restore this check. The return parameter should not >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? be touched unless the return code in 'err' == JVMTI_ERROR_NONE. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But op.stack_info() will return NULL if the error is not JVMTI_ERROR_NONE. Are you (Dan) concerned about someone passing in a non-null/initialized out-pointer that will be reset to NULL if there was an error? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Actually the way we used to test this in POSIX tests is to call >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an API with known bad parameters and the return parameter ptr >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> set to NULL. If the return parameter ptr was touched when an >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> error should have been detected on an earlier parameter, then >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the test failed. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> L1272: ? if (!jt->is_exiting() && (thread_oop != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? old L1532: _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? This deletion of the _result field threw me for a minute and then >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? ? ? I figured out that the field is init to JVMTI_ERROR_THREAD_NOT_ALIVE >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? in the constructor. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L1553: if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? nit - extra parens around the second expression. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/runtime/vmOperations.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/GetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? No comments. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/OneGetThreadListStackTraces.java >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L64: startSignal.countDown(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? I was expecting this to be a call to await() instead of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> countDown(). What am I missing here? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? I think this test might be passing by accident right now, but... >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Main thread (which call JVMTI functions to test) should wait until test thread is ready. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So main thread would wait startSignal, and test thread would count down. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No! >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The test thread that previously called obj.wait() now calls latch.await(). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The main thread that previously called obj.notify() now calls latch.countDown(). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The main thread continues to spin until it sees the target is WAITING before proceeding with the test. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> If I add spin wait to wait until transit target thread state is WAITING (as following), we don't need to call SuspendThread(). >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Which is better? >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> The original spin-wait loop checking for WAITING is better because it is the only guarantee that the target thread is blocked where you need it to be. suspending the thread is racy as you don't know exactly where the suspend will hit. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>> /* Wait until the thread state transits to "waiting" */ >>>>>>>>>>>>>>>>>>>>>>>>>>>>> while (th.getState() != Thread.State.WAITING) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thread.onSpinWait(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> For simplify, spin wait is prefer to OneGetThreadListStackTraces.java in webrev.03. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here's the flow as I see it: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - start worker thread >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - startSignal.await() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - startSignal.countDown() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - stopSignal.await() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - worker is now blocked >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - checkCallStacks(th) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - stopSignal.countDown() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - worker is now unblocked >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - th.join >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now blocked >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> worker thread >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - runs off the end of run() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? - main is now unblocked >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> main thread >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? - run off the end of main() >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L92: jthreads = (jthread *)malloc(sizeof(jthread) * num_threads); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? You don't check for malloc() failure. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 'jthreads' is allocated but never freed. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will fix it. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/GetThreadListStackTraces/libOneGetThreadListStackTraces.c >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???? L91: result = (*jvmti)->SuspendThread(jvmti, thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? Why are you suspending the thread? GetAllStackTraces() and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTraces() do not require the target thread(s) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ???????? to be suspend. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ????????? If you decide not to SuspendThread, then you don't need the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AddCapabilities or the ResumeThread calls. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Test thread might not be entered following code (stopSignal.await()). We might see deferent call stack between GetAllStackTraces() and GetThreadListStackTraces(). We cannot control to freeze call stack of test thread in Java code. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (I didn't use SuspendThread() at first, but I saw some errors which causes in above.) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So we need to call SuspendThread() to ensure we can see same call stack. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you are checking that the thread is in state WAITING then it cannot escape from that state and you can sample the stack multiple times from any API and get the same result. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I suspect the errors you saw were from the apparent incorrect use of the CountDownLatch. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With the flow outlined above, the worker thread should be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nicely blocked in stopSignal.await() when stuff is sampled. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dan >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/02 15:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 11:53 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I uploaded new webrev. Could review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.02/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Updates look fine - thanks. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> One minor nit: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1274 _collector.allocate_and_fill_stacks(1); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1275 _collector.set_result(JVMTI_ERROR_NONE); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the other places where you use _collector you rely on result being initialized to JVMTI_ERROR_NONE, rather than setting it directly after allocate_and_fill_stacks(). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 823 "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This function (JvmtiEnvBase::get_stack_trace()) can be called to get own stack trace. For example, we can call GetStackTrace() for current thread at JVMTI event. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So I changed assert as below: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??820 assert(current_thread == java_thread || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??821 SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??823 "call by myself / at safepoint / at handshake"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yep good catch. I hope current tests caught that. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> They would be tested in vmTestbase/nsk/jvmti/GetStackTrace/getstacktr001/ (own call stacks), and getstacktr003 (call stacks in other thread). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Speaking of tests ... >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the native code I think you need to check the success of all JNI methods that can throw exceptions - otherwise I believe the tests may trigger warnings if -Xcheck:jni is used with them. See for example: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I updated testcases to check JNI and JVMTI function calls. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the Java code the target thread: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 45 public void run() { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 46 try { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 47 synchronized (lock) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 48 lock.wait(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? 49 System.out.println("OK"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 50?????????? } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is potentially susceptible to spurious wakeups. Using a CountDownLatch would be robust. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Fixed. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 8:48, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 9:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If so, we can call make_local() in L1272 without JavaThread (or we can pass current thread to make_local()). Is it right? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry I got confused, _calling_thread may not be the current thread as we could be executing the handshake in the target thread itself. So the ResourceMark is correct as-is (implicitly for current thread). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The argument to fill_frames will be used in the jvmtiStackInfo and passed back to the _calling_thread, so it must be created via make_local(_calling_thread, ...) as you presently have. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/07/01 7:05, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 1/07/2020 12:17 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you for reviewing! I will update new webrev tomorrow. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 500 JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ? 502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure what does mean "embedded". >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is it ok as below? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class MultipleStackTracesCollector { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??? : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> class GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes that I what I meant. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/30 22:22, David Holmes wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 30/06/2020 10:05 am, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi David, Serguei, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I updated webrev for 8242428. Could you review again? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change migrate to use direct handshake for GetStackTrace() and GetThreadListStackTraces() (when thread_count == 1). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.01/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks really good now! I only have a few nits below. There is one thing I don't like about it but it requires a change to the main Handshake logic to address - in JvmtiEnv::GetThreadListStackTraces you have to create a ThreadsListHandle to convert the jthread to a JavaThread, but then the Handshake::execute_direct creates another ThreadsListHandle internally. That's a waste. I will discuss with Robbin and file a RFE to have an overload of execute_direct that takes an existing TLH. Actually it's worse than that because we have another TLH in use at the entry point for the JVMTI functions, so I think there may be some scope for simplifying the use of TLH instances - future RFE. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??451 GetStackTraceClosure(JvmtiEnv *env, jint start_depth, jint max_count, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??452 jvmtiFrameInfo* frame_buffer, jint* count_ptr) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??453???? : HandshakeClosure("GetStackTrace"), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??454 _env(env), _start_depth(start_depth), _max_count(max_count), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??455 _frame_buffer(frame_buffer), _count_ptr(count_ptr), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??456 _result(JVMTI_ERROR_THREAD_NOT_ALIVE) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Nit: can you do one initializer per line please. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This looks wrong: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 466 class MultipleStackTracesCollector : public StackObj { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??498 class VM_GetAllStackTraces : public VM_Operation { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??499 private: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??500 JavaThread *_calling_thread; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??501?? jint _final_thread_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??502 MultipleStackTracesCollector _collector; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can't have a StackObj as a member of another class like that as it may not be on the stack. I think MultipleStackTracesCollector should not extend any allocation class, and should always be embedded directly in another class. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 481 MultipleStackTracesCollector(JvmtiEnv *env, jint max_frame_count) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??482???? _env = env; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??483 _max_frame_count = max_frame_count; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??484 _frame_count_total = 0; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??485 _head = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??486 _stack_info = NULL; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??487 _result = JVMTI_ERROR_NONE; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??488?? } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As you are touching this can you change it to use an initializer list as you did for the HandshakeClosure, and please keep one item per line. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??820 assert(SafepointSynchronize::is_at_safepoint() || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??821 java_thread->is_thread_fully_suspended(false, &debug_bits) || >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??822 current_thread == java_thread->active_handshaker(), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??823 "at safepoint / handshake or target thread is suspended"); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think the suspension check is necessary, as even if the target is suspended we must still be at a safepoint or in a handshake with it. Makes me wonder if we used to allow a racy stacktrace operation on a suspended thread, assuming it would remain suspended? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1268?? oop thread_oop = jt->threadObj(); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1269 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1270?? if (!jt->is_exiting() && (jt->threadObj() != NULL)) { >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can use thread_oop in line 1270. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1272 _collector.fill_frames((jthread)JNIHandles::make_local(_calling_thread, thread_oop), >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1273 jt, thread_oop); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is frustrating that this entire call chain started with a jthread reference, which we converted to a JavaThread, only to eventually need to convert it back to a jthread! I think there is some scope for simplification here but not as part of this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1271 ResourceMark rm; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IIUC at this point the _calling_thread is the current thread, so we can use: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ResourceMark rm(_calling_thread); >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please add @bug lines to the tests. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm still pondering the test logic but wanted to send this now. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM_GetThreadListStackTrace (for GetThreadListStackTraces) and VM_GetAllStackTraces (for GetAllStackTraces) have inherited VM_GetMultipleStackTraces VM operation which provides the feature to generate jvmtiStackInfo. I modified VM_GetMultipleStackTraces to a normal C++ class to share with HandshakeClosure for GetThreadListStackTraces (GetSingleStackTraceClosure). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I added new testcases which test GetThreadListStackTraces() with thread_count == 1 and with all threads. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/nsk/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2020/06/24 15:50, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8242428 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8242428/webrev.00/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change replace following VM operations to direct handshake. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameCount (GetFrameCount()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetFrameLocation (GetFrameLocation()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetThreadListStackTraces (GetThreadListStackTrace()) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ??- VM_GetCurrentLocation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GetThreadListStackTrace() uses direct handshake if thread count == 1. In other case (thread count > 1), it would be performed as VM operation (VM_GetThreadListStackTraces). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Caller of VM_GetCurrentLocation (JvmtiEnvThreadState::reset_current_location()) might be called at safepoint. So I added safepoint check in its caller. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This change has been tested in serviceability/jvmti serviceability/jdwp vmTestbase/nsk/jvmti vmTestbase/nsk/jdi vmTestbase/ns >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> k/jdwp. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also I tested it on submit repo, then it has execution error (mach5-one-ysuenaga-JDK-8242428-20200624-0054-12034717) due to dependency error. So I think it does not occur by this change. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>> From chris.plummer at oracle.com Sun Jul 12 19:08:11 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Sun, 12 Jul 2020 12:08:11 -0700 Subject: RFR (S) 8249137: Remove CollectedHeap::obj_size In-Reply-To: <99b679ec-fcae-6f30-1186-1cf8de809c6f@oracle.com> References: <70984ebf-2717-c7b3-7076-12e2c8c7515c@oracle.com> <99b679ec-fcae-6f30-1186-1cf8de809c6f@oracle.com> Message-ID: Looks good to me. Chris On 7/10/20 8:07 AM, Daniel D. Daugherty wrote: > On 7/10/20 4:37 AM, Aleksey Shipilev wrote: >> Okay, thanks. >> >> I already have 2 reviewers (rkennke, tschatzl), do I need more >> specifically from serviceability-dev@? > > Since you're touching the Serviceability team's code, it would be > polite to wait for a review... > > Dan > > >> >> -Aleksey >> >> On 7/9/20 3:34 PM, Daniel D. Daugherty wrote: >>> Adding serviceability-dev at ... since a couple of JVM/TI files are >>> changed >>> in this RFR. Also, I moved the bug from hotspot/runtime -> hotspot/gc. >>> >>> Dan >>> >>> >>> On 7/9/20 2:36 AM, Aleksey Shipilev wrote: >>>> RFE: >>>> ??? https://bugs.openjdk.java.net/browse/JDK-8249137 >>>> >>>> It was added by JDK-8211270 to support old-style Shenandoah that >>>> needed a separate fwdptr slot. >>>> After JDK-8224584 it does not need this anymore. Additionally, >>>> CH::obj_size may disagree with other >>>> code that pokes at layout helper directly, for example >>>> GraphKit::new_instance. >>>> >>>> This also avoids a virtual call on some paths, although those paths >>>> are not very performance-sensitive. >>>> >>>> The patch is a simple series of few-liners: >>>> >>>> diff -r 9cc348ebdc82 src/hotspot/share/gc/shared/collectedHeap.cpp >>>> --- a/src/hotspot/share/gc/shared/collectedHeap.cpp???? Thu Jul 09 >>>> 04:32:30 2020 +0200 >>>> +++ b/src/hotspot/share/gc/shared/collectedHeap.cpp???? Thu Jul 09 >>>> 08:05:46 2020 +0200 >>>> @@ -578,6 +578,2 @@ >>>> >>>> -size_t CollectedHeap::obj_size(oop obj) const { >>>> -? return obj->size(); >>>> -} >>>> - >>>> ?? uint32_t CollectedHeap::hash_oop(oop obj) const { >>>> diff -r 9cc348ebdc82 src/hotspot/share/gc/shared/collectedHeap.hpp >>>> --- a/src/hotspot/share/gc/shared/collectedHeap.hpp???? Thu Jul 09 >>>> 04:32:30 2020 +0200 >>>> +++ b/src/hotspot/share/gc/shared/collectedHeap.hpp???? Thu Jul 09 >>>> 08:05:46 2020 +0200 >>>> @@ -495,4 +495,2 @@ >>>> >>>> -? virtual size_t obj_size(oop obj) const; >>>> - >>>> ???? // Non product verification and debugging. >>>> diff -r 9cc348ebdc82 src/hotspot/share/prims/jvmtiEnv.cpp >>>> --- a/src/hotspot/share/prims/jvmtiEnv.cpp????? Thu Jul 09 04:32:30 >>>> 2020 +0200 >>>> +++ b/src/hotspot/share/prims/jvmtiEnv.cpp????? Thu Jul 09 08:05:46 >>>> 2020 +0200 >>>> @@ -488,3 +488,3 @@ >>>> ???? NULL_CHECK(mirror, JVMTI_ERROR_INVALID_OBJECT); >>>> -? *size_ptr = (jlong)Universe::heap()->obj_size(mirror) * wordSize; >>>> +? *size_ptr = (jlong)mirror->size() * wordSize; >>>> ???? return JVMTI_ERROR_NONE; >>>> diff -r 9cc348ebdc82 src/hotspot/share/prims/jvmtiExport.cpp >>>> --- a/src/hotspot/share/prims/jvmtiExport.cpp?? Thu Jul 09 04:32:30 >>>> 2020 +0200 >>>> +++ b/src/hotspot/share/prims/jvmtiExport.cpp?? Thu Jul 09 08:05:46 >>>> 2020 +0200 >>>> @@ -1067,3 +1067,3 @@ >>>> ??????? _jobj = (jobject)to_jobject(obj); >>>> -???? _size = Universe::heap()->obj_size(obj) * wordSize; >>>> +???? _size = obj->size() * wordSize; >>>> ????? }; >>>> diff -r 9cc348ebdc82 src/hotspot/share/prims/whitebox.cpp >>>> --- a/src/hotspot/share/prims/whitebox.cpp????? Thu Jul 09 04:32:30 >>>> 2020 +0200 >>>> +++ b/src/hotspot/share/prims/whitebox.cpp????? Thu Jul 09 08:05:46 >>>> 2020 +0200 >>>> @@ -389,3 +389,3 @@ >>>> ???? oop p = JNIHandles::resolve(obj); >>>> -? return Universe::heap()->obj_size(p) * HeapWordSize; >>>> +? return p->size() * HeapWordSize; >>>> ?? WB_END >>>> >>>> Testing: tier{1,2}; jdk-submit (running) >>>> >> > From shade at redhat.com Mon Jul 13 05:40:48 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 13 Jul 2020 07:40:48 +0200 Subject: RFR (S) 8249137: Remove CollectedHeap::obj_size In-Reply-To: References: <70984ebf-2717-c7b3-7076-12e2c8c7515c@oracle.com> <99b679ec-fcae-6f30-1186-1cf8de809c6f@oracle.com> Message-ID: <8fea1a3f-8d62-4408-8f07-b92af35b6632@redhat.com> On 7/12/20 9:08 PM, Chris Plummer wrote: > Looks good to me. Thanks! Pushed. -- -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From richard.reingruber at sap.com Mon Jul 13 06:42:13 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Mon, 13 Jul 2020 06:42:13 +0000 Subject: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents In-Reply-To: References: <1f8a3c7a-fa0f-b5b2-4a8a-7d3d8dbbe1b5@oracle.com> <4b56a45c-a14c-6f74-2bfd-25deaabe8201@oracle.com> <5271429a-481d-ddb9-99dc-b3f6670fcc0b@oracle.com> Message-ID: Hi Goetz, thanks for looking at this! And my apologies for taking that long... So here is the new webrev.6 Webrev.6: http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.6/ Delta: http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.6.inc/ I spent most of the time running a microbenchmark [1] I wrote to answer questions from your review. At first I had trouble with variance in the results until I found out it was due to the NUMA architecture of the server I used. After that I noticed that there was a performance regression of about 5% even at low agent activity. I finally found out that it was due to the implementation of JavaThread::wait_for_object_deoptimization() which is called by the target of the JVMTI operation to self suspend for object deoptimization. I fixed this by adding limited spinning before calling wait() on the monitor. The delta includes many changes in comments, renaming of names, etc. So I'd like to summarize functional changes: * Collected all the code for the testing feature DeoptimizeObjectsALot in compileBroker.cpp and reworked it. With DeoptimizeObjectsALot enabled internal threads are started that deoptimize frames and objects. The number of threads started are given with DeoptimizeObjectsALotThreadCountAll and DeoptimizeObjectsALotThreadCountSingle. The former targets all existing threads whereas the latter operates on a single thread selected round robin. I removed the mode where deoptimizations were performed at every nth exit from the runtime. I never used it. * EscapeBarrier::sync_and_suspend_one(): use a direct handshake and execute it always independently of is_thread_fully_suspended(). * Bugfix in EscapeBarrier::thread_added(): must not clear deopt flag. Found this testing with DeoptimizeObjectsALot. * Added EscapeBarrier::thread_removed(). * EscapeBarrier constructors: barriers can now be entirely disabled by disabling DoEscapeAnalysis. This effectively disables the enhancement. * JavaThread::wait_for_object_deoptimization(): - Bugfix: the last check of is_obj_deopt_suspend() must be /after/ the safepoint check! This caused issues with not walkable stacks with DeoptimizeObjectsALot. - Added limited spinning inspired by HandshakeSpinYield to fix regression in microbenchmark [1] I refer to some more changes answering your questions and comments inline below. Thanks, Richard. [1] Microbenchmark: http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.6.microbenchmark/ > Hi Richard, > > I had a look at your change. It's complex, but not that big. > A lot of code is just passing info through layers of abstraction. Also it leverages preexisting functionality like materialization of virtual objects in non-top frames (see materializeVirtualObjects). > Also, one can tell this went through some iterations by now, > I think it's very well engineered. > I had a look at webrev.05 > > Unfortunately > "8242425: JVMTI monitor operations should use Thread-Local Handshakes" > breaks webrev.05. > I updated to before that change and took that as base of my review. > > I see four parts of the change that can be looked at > rather individually. > > * Refactoring the scopeDesc constructors. Trivial. > * Persisting information about the optimizations done by the compilers. > Large and mostly trivial. > * Deoptimizing. The most complicated part. Really well abstracted, though. > * DeoptimizeObjectsALot for testing and the tests. > > Review of compiler changes: > > I understand you annotate at safepoints where the escape analysis > finds out that an object is "better" than global escape. > This are the cases where the analysis identifies optimization > opportunities. These annotations are then used to deoptimize > frames and the objects referenced by them. > Doesn't this overestimate the optimized > objects? E.g., eliminate_alloc_node has many cases where it bails > out. Yes, the implementation is conservative, but it is comparatively simple and the additional debug info is just 2 flags per safepoint. On the other hand, those JVMTI operations that really trigger deoptimizations are expected to be comparatively infrequent such that switching to the interpreter for a few microseconds will hardly have an effect. I've done microbenchmarking to check this. http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.6.microbenchmark/ I found that in the worst case performance can be impacted by 10%. If the agent is extremely active and does relevant JVMTI calls like GetOwnedMonitorStackDepthInfo() every millisecond or more often, then the performance impact can be 30%. But I would think that this is not realistic. These calls are issued in interactive sessions to analyze deadlocks. We could get more precise deoptimizations by adding a third flag per safepoint for ea-local objects among the owned monitors. This would help improve the worst case in the benchmark. But I'm not convinced, if it is worth it. Refer to the README.txt of the microbenchmark for a more detailled discussion. > c1_IR.hpp > > OK, nothing to do for C1, just adapt to extended method signature. > > Break line once more so that it matches above line length. Done. > ciEnv.h|cpp > > Pass through another jvmti capability. Trivial & good. > > > debugInfoRec.hpp > > Pass through escape info that must be recorded. OK. > > pcDesc.hpp > > I would like to see some documentation of the methods. > > Maybe: > // There is an object in the scope that does not escape globally. > // It either does not escape at all or it escapes as arguemnt. > and > // One of the arguments is an object that is not globally visible > // but escapes to the callee. Done. I didn't take your text, though, because I only noticed it after writing my own. Let me know if you are not ok with it. > scopeDesc.cpp > > Besides refactoring copy escape info from pcDesc to scopeDesc > and add accessors. Trivial. > > In scopeDesc.hpp you talk about NoEscape and ArgEscape. > This are opto terms, but scopeDesc is a shared datastructure > that does not depend on a specific compiler. > Please explain what is going on without using these terms. Actually these are not too opto specific terms. They are used in the paper referenced in escape.hpp. Also you can easily google them. I'd rather keep the comments as they are. > jvmciCodeInstaller.cpp > > OK, nothing for JVMCI. Here support for Object Optimizations > for JVMCI compilers could be added. Leave this to graal people. > > callnode.hpp > > You add functionality to annotate callnodes with escape information > This is carried through code generation to final output where it is > added to the compiled methods meta information. > > At Safepoints in general jvmti can access > - Objects that were scalar replaced. They must be reallocated. > (Flag EliminateAllocations) > - Objects that should be locked but are not because they never > escape the thread. They need to be relocked. > > At calls, Objects where locks have been removed escape to callees. > We must persist this information so that if jvmti accesses the > object in a callee, we can determine by looking at the caller that > it needs to be relocked. Note that the ea-optimization must not be at the current location, it can also follow when control returns to the caller. Lock elimination isn't the only relevant optimization. Accesses to instance members or array elements can be optimized as well. > A side comment: > I think the flage handling in Opto is not very intuitive. > DoEscapeAnalysis depends on the jvmti capabilities. > This makes no sense. It is only an analysis. The optimizations > should depend on the jvmti capabilities. > The correct setup would be to handle this in > CompilerConfig::ergo_initialize(): > If the jvmti capabilities allow, enable the optimizations > EliminateAllocations or EliminateLocks/EliminateNestedLocks. > If one of these optimizations is on, enable EscapeAnalysis. > -- end side comment. > > So I would propose the following comments: > > // In the scope of this safepoints there are objects > // that do not globally escape. They are either NoEscape or > // ArgEscape. As such, they might be subject to optimizations. > // Persist this information here so that the frame an the > // Objects in scope can > // be deoptimized if jvmti accesses an object at this safepoint. > void set_not_global_escape_in_scope(bool b) { > > // This call passes objects that do not globally escape > // to its callee. The object might be subject to optimization, > // e.g. a lock might be omitted. Persist this information here > // so that on a jvmti access to the callee frame we can deoptimize > // the object and this frame. > void set_arg_escape(bool f) { _arg_escape = f; } I do not really like these comments. They are too verbose and do not match the comment style of the surrounding code. The names are descriptive enough IMO. Also the measures taken depending on the flags should be commented at the locations, where the flags are read. > Actuall I am not sure whether the name of these fields (and all > the others in the course of this change) should refer to > escape analysis. I think the term "Object deoptimization" > you also use is much better. You could call these properties > (througout the whole change) > set_optimized_objects_in_scope() > and > set_passes_optimized_objects(). > > I think this would make the whole matter much easier > to understand. I'd prefer the current names. They are closer to established terminology. And it is actually unknown, if optimizations based on their escape state exist. > Anyways, locks can already be removed without running > escape analysis at all. C2 recognizes some local patterns > that allow this. > > escape.h|cpp > > The code looks good. > > Line 325: The comment could be a bit more elaborate: > // Annotate at safepoints if they have <= ArgEscape objects in their > // scope. Additionally, if the safepoint is a java call, annotate > // whether it passes ArgEscape objects as parameters. > > And maybe add these comments?: > > // Returns true if an oop in the scope of sfn does not escape > // globally. > bool ConnectionGraph::has_not_global_escape_in_scope(SafePointNode* sfn) { > > // Returns true if at least one of the arguments to the call is an oop > // that does not escape globally. > bool ConnectionGraph::has_arg_escape(CallJavaNode* call) { IMHO the method names are descriptive and don't need the comments. But I give in :) (only replaced "oop" with "object") > General question: > You collect the information you want to annotate to the > method during escape analysis. > Don't you overestimate the optimized objects by this? > E.g. elimination of allocations does bail out for > various reasons. At the end, no optimization might > have happened, but then during runtime the frame is > deoptimized nevertheless. Please see statements and worst case microbenchmark above. > machnode.hpp: > > Extends MachSafePointNode similar to the ideal version. Good. > > matcher.cpp > > Copy info from ideal to mach node. good. > > output.cpp > > Now finally the information is written to the > debug info. Good. > > --------------------------------------------------------- > > So now let's have a look at the runtime part (including > relaxing constraints to escape analysis): > > rootResolver.cpp > > Adapt to changed interface. good. > > c2compiler.cpp / macro.cpp > > Make EscpaeAnlysis independent of jvmti capabilities. Good. > > jvmtiEnv.cpp/jvmtiEnvBase.cpp > > You add deoptimization of objects where they are > accessed. good. > > jvmtiImpl.cpp > > In deoptimize_objects, you check for DoEscapeAnalysis. > This is correct given the current design of the flag > handling in the compiler. > It's not really nice to have a dependency to C2 here, > though. I understand it's an optimization, the code > could be run anyways, it would check but not find > anything. But actually I would excpect dependencies > on EliminateLocks and EliminateAllocations (if they > were set according to jvmti capabilitiers as I elaborated > above.) > Would it make sense to protect the ArgEscape > loop by if (EliminateLocks)? You are right, it is not correct how flags are checked. Especially if only running with the JVMCI compiler. I changed Deoptimization::deoptimize_objects_internal() to make reallocation and relocking dependent on similar checks as in Deoptimization::fetch_unroll_info_helper(). Furthermore EscapeBarriers are conditionally activated depending on the following (see EscapeBarrier ctors): JVMCI_ONLY(UseJVMCICompiler) NOT_JVMCI(false) COMPILER2_PRESENT(|| DoEscapeAnalysis) So the enhancement can be practically completely disabled by disabling DoEscapeAnalysis, which is what C2 currently does if JVMTI capabilities that allow access to local references are taken. > jvmtiTagMap.cpp > > Deoptimize for jvmti operations. Good. > > deoptimization.cpp > > I guess this is the core of your work. > > > You add a new mode that just deoptimizes objects but not frames. > Good idea. You have to use reallocated objects in upper frames, > or by jvmti accesses to inner frames, which can not easily be > replaced by interpreter frames. > This way you can wait with replacing the frame until just before > execution returns. > > eliminate_allocations(): > (Strange method name, should at least be in past tense, even > better reallocate_eliminated_allocations() or > allocate_scalarized_objects(). Confused me until > I groked the code. Legacy though, not your business.) I still don't grok the name... ;) but it's preexisting as you noted > It's not that nice to return whether you only deoptimized > objects by the boolean reference argument. After all, > it again depends on the mode you pass in. > A different design would be to clone the method and > have an eliminate_allocations_no_unpack() variant, but that would > not be better as some code would be duplicated. > Maybe a comment for argument eliminate_allocations: > // deoptimized_objects is set to true if objects were deoptimized > // but not the frame. It is unchanged if there are no objects to > // be deoptimized, or if the frame was deoptim I agree: duplicating the code would be really bad, but I don't think that having reference parameters is not nice. I think it is a common pattern, if you return an error code and additional result data. The variable is a minor detail. With the meaningful name it is not necessary to document it. In my eyes it should be set independently of the exec_mode. I didn't do it to make the change smaller. > Similar for eliminate_locks(): > // deoptimized_objects is set to true if objects were relocked, > // else it is left unchanged. > > You reuse and extend the existing realloc/relock_objects, but extended it. > > deoptimize_objects_internal() > > Simple version of fetch_unroll_info_helper for EscapeBarrier. > Good. > I attributed the comment "Then relock objects if synchronization on them was eliminated." > to the if() just below. Add an empty line to make clear the comment > refers to the next 10 lines. > Alternatively, replace the whole comment by > // At first, reallocate the non-escaping objects and restore their fields > // so they are available for relocking. > And add > // Now relock objects with eliminated locks. > befor the if ((DoEscape... below. I went for the latter. > In fetch_unroll_info_helper, I don't understand why you need > && !EscapeBarrier::objs_are_deoptimized(thread, deoptee.id())) { > for eliminated locks, but not for skalar replaced objects? In short reallocation is idempotent, relocking is not. Without the enhancement Deoptimization::realloc_objects() can already be called more than once for a frame: First call in materializeVirtualObjects() (also iterateFrames()). Second (indirect) call in fetch_unroll_info_helper(). The objects from the first call are saved as jvmti deferred updates when realloc_objects() returns. Note that there is no relationship to jvmti. The thing in common is that updates cannot be directely installed into a compiled frame, it is necessary to deoptimize the frame and defer the updates until the compiled frame gets replaced. Every time the vframes corresponding to the owner frame are iterated, they get the deferred updates. So in fetch_unroll_info_helper() the GrowableArray* chunk reference them too. All references to the objects created by the second (indirect) call to realloc_objects() are never used, because compiledVFrame accessors to locals, expressions, and monitors override them with the deferred updates. The objects become unreachable and get gc'ed. materializeVirtualObjects() does not bother with relocking. deoptimize_objects_internal(), which is introduced by the enhancement, does relock objects, after all the lock elimination becomes illegal with the change in escape state. Relocking twice does not work, so the enhancement avoids it by checking EscapeBarrier::objs_are_deoptimized(thread, deoptee.id()). Note that materializeVirtualObjects() can be called more than once and will always return the very same objects, even though it calls realloc_objects() again. > I would guess it is because the eliminated locks can be applied to > argEscape, but scalar replacement only to noescape objects? > I.e. it might have been done before? > > But why isn't this the case for eliminate_allocations? > deoptimize_objects_internal does both unconditionally, > so both can happen to inner frames, right? Sorry, I don't quite understand. Hope the explanation above helps. > relock_objects() > > Ok, you need to undo biased locking. Also, you remember the > lock nesting for later relocking if waiting for lock. > > revoke_for_object_deoptimization() > I like if boolean operators are at the beginning of broken lines, > but I think hotspot convention is to have them at the end. Ok, fixed. > Code will get much more simple if BiasedLocking is removed. > > EscapeBarrier:: ... > > (This class maybe would qualify for a file of its own.) > > deoptimize_objects() > I would mention escape analysis only as side remark. Also, as I understand, > there is only one frame at given depth? > // Deoptimize frames with optimized objects. This can be omitted locks and > // objects not allocated but replaced by scalars. In C2, these optimizations > // are based on escape analysis. > // Up to depth, deoptimize frames with any optimized objects. > // From depth to entry_frame, deoptimize only frames that > // pass optimized objects to their callees. > (First part similar for the comment above EscapeBarrier::deoptimize_objects_internal().) I've reworked the comment. Let me know if you still think it needs to be improved. > > What is the check (cur_depth <= depth) good for? Can you > ever walk past entry_frame? Yes (assuming you mean the outer while-statement), there are java frames beyond the entry frame if a native method calls java methods again. So we visit all frames up to the given depth and from there we continue to the entry frame. It is not necessary to continue beyond that entry frame, because escape analysis assumes that arguments to native functions escape globally. Example: Let the java stack look like this: +---------+ | Frame A | +---------+ | Frame N | +---------+ | Frame B | +---------+ <- top of stack Where java method A calls native method N and N calls java method B. Very simplified the native stack will look like this +-------------------------+ | Frame of JIT Compiled A | +-------------------------+ | Frame N | +-------------------------+ | Entry Frame | +-------------------------+ | Frame B | +-------------------------+ <- top of stack The entry frame is an activation of the call stub, which is a small assembler routine that translates from the native calling convention to the java calling convention. There cannot be any ArgEscape that is passed to B (see above), therefore we can stop the stackwalk at the entry frame if depth is 1. If depth is 3 we have to continue to Frame A, as it is directely accessed. > Isn't vf->is_compiled_frame() prerequisite that "Move to next physical frame" > is needed? You could move it into the other check. > If so, similar for deoptimize_objects_all_threads(). Only compiledVFrame require moving to the /top/ frame. Fixed. > Syncronization: looks good. I think others had a look at this before. > > EscapeBarrier::deoptimize_objects_internal() > The method name is misleading, it is not used by > deoptimize_objects(). > Also, method with the same name is in Deopitmization. > Proposal: deoptimize_objects_thread() ? Sorry, but I don't see, why it would be misleading. What would be the meaning of 'deoptimize_objects_thread'? I don't understand that name. > C1 stubs: this really shows you tested all configurations, great! > > > mutexLocker: ok. > objectMonitor.cpp: ok > stackValue.hpp Is this missing clearing a bug? In short: that change is not needed anymore. I'll remove it again. Details: it is not a real bug, but the assertion in vframeArrayElement::fill_in() was triggered: assert(!value->obj_is_scalar_replaced() || realloc_failures) failed: object should be reallocated already. But only with the first version of the enhancement (webrev.0), were objects were only reallocated when replacing a compiled frame with equivalent interpreter frames iff virtual objects where not reallocated before. I changed this after prexisting code was refactored (JDK-8226705), because practically never already reallocated objects exist and if there should be any, it does not harm to reallocate again, because the unnecessarily allocated objects become immediately garbage and last but not least no tricky synchronization is required. Also that's what happens with the preexisting code if virtual objects are materialized with materializeVirtualObjects(). > > thread.hpp > > I would remove "_ea" from the flag and method names. Done. > > Renaming deferred_locals to deferred_updates is good, as well as > adding a datastructure for it. > (Adding this data structure might be a breakout, too.) > > good. > > thread.cpp > > good. > > vframe.cpp > > Is this a bug in existing code? > Makes sense. Depends on your definition of bug. There are no references to vframe::is_entry_frame() in the existing code. I would think it is a bug. > > vframe_hp.hpp > (What stands _hp for? helper? The file should be named compiledVFrame ...) > > not_global_escape_in_scope() ... > Again, you mention escape analysis here. Comments above hold, too. I think it is the right name, because it is meaningful and simple. > You introduce JvmtiDeferredUpdates. Good. > > vframe_hp.cpp > > Changes for JvmtiDeferredUpdates, escape state accessors, > > line 422: > Would an assertion assert(!info->owner_is_scalar_replaced(), ...) hold here? > > > macros.hpp > Good. > > > Test coding > ============ > > compileBroker.h|cpp > > You introduce a third class of threads handled here and > add a new flag to distinguish it. Before, the two kinds > of threads were distinguished implicitly by passing in > a compiler for compiler threads. > The new thread kind is only used for testing in debug. > > make_thread: > You could assert (comp != NULL...) to assure previous > conditions. If replaced the if-statements with a switch-statement, made sure all enum-elements are covered, and added the assertion you suggested. > line 989 indentation broken You are referring to this block I assume: (from http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.5/src/hotspot/share/compiler/compileBroker.cpp.frames.html) 976 if (MethodFlushing) { 977 // Initialize the sweeper thread 978 Handle thread_oop = create_thread_oop("Sweeper thread", CHECK); 979 jobject thread_handle = JNIHandles::make_local(THREAD, thread_oop()); 980 make_thread(sweeper_t, thread_handle, NULL, NULL, THREAD); 981 } 982 983 #if defined(ASSERT) && COMPILER2_OR_JVMCI 984 if (DeoptimizeObjectsALot == 2) { 985 // Initialize and start the object deoptimizer threads 986 for (int thread_count = 0; thread_count < DeoptimizeObjectsALotThreadCount; thread_count++) { 987 Handle thread_oop = create_thread_oop("Deoptimize objects a lot thread", CHECK); 988 jobject thread_handle = JNIHandles::make_local(THREAD, thread_oop()); 989 make_thread(deoptimizer_t, thread_handle, NULL, NULL, THREAD); 990 } 991 } 992 #endif // defined(ASSERT) && COMPILER2_OR_JVMCI I cannot really see broken indentation here. Am I looking at the wrong location? > escape.cpp > > You enable the optimization in case of testruns. good. > > whitebox.cpp ok. > > deoptimization.cpp > > deoptimize_objects_alot_loop() Good. > > globals.hpp > > Nice docu of flags, but pleas mention "for testing purposes" > or the like in DeoptimizeObjectsALot. > I would place the flags next to each other. > > interfaceSupport.cpp: good. Thanks! :) -----Original Message----- From: Lindenmaier, Goetz Sent: Mittwoch, 6. Mai 2020 12:28 To: Reingruber, Richard ; Doerr, Martin ; 'Robbin Ehn' ; David Holmes ; Vladimir Kozlov (vladimir.kozlov at oracle.com) ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents Hi Richard, I had a look at your change. It's complex, but not that big. A lot of code is just passing info through layers of abstraction. Also, one can tell this went through some iterations by now, I think it's very well engineered. I had a look at webrev.05 Unfortunately "8242425: JVMTI monitor operations should use Thread-Local Handshakes" breaks webrev.05. I updated to before that change and took that as base of my review. I see four parts of the change that can be looked at rather individually. * Refactoring the scopeDesc constructors. Trivial. * Persisting information about the optimizations done by the compilers. Large and mostly trivial. * Deoptimizing. The most complicated part. Really well abstracted, though. * DeoptimizeObjectsALot for testing and the tests. Review of compiler changes: I understand you annotate at safepoints where the escape analysis finds out that an object is "better" than global escape. This are the cases where the analysis identifies optimization opportunities. These annotations are then used to deoptimize frames and the objects referenced by them. Doesn't this overestimate the optimized objects? E.g., eliminate_alloc_node has many cases where it bails out. c1_IR.hpp OK, nothing to do for C1, just adapt to extended method signature. Break line once more so that it matches above line length. ciEnv.h|cpp Pass through another jvmti capability. Trivial & good. debugInfoRec.hpp Pass through escape info that must be recorded. OK. pcDesc.hpp I would like to see some documentation of the methods. Maybe: // There is an object in the scope that does not escape globally. // It either does not escape at all or it escapes as arguemnt. and // One of the arguments is an object that is not globally visible // but escapes to the callee. scopeDesc.cpp Besides refactoring copy escape info from pcDesc to scopeDesc and add accessors. Trivial. In scopeDesc.hpp you talk about NoEscape and ArgEscape. This are opto terms, but scopeDesc is a shared datastructure that does not depend on a specific compiler. Please explain what is going on without using these terms. jvmciCodeInstaller.cpp OK, nothing for JVMCI. Here support for Object Optimizations for JVMCI compilers could be added. Leave this to graal people. callnode.hpp You add functionality to annotate callnodes with escape information This is carried through code generation to final output where it is added to the compiled methods meta information. At Safepoints in general jvmti can access - Objects that were scalar replaced. They must be reallocated. (Flag EliminateAllocations) - Objects that should be locked but are not because they never escape the thread. They need to be relocked. At calls, Objects where locks have been removed escape to callees. We must persist this information so that if jvmti accesses the object in a callee, we can determine by looking at the caller that it needs to be relocked. A side comment: I think the flage handling in Opto is not very intuitive. DoEscapeAnalysis depends on the jvmti capabilities. This makes no sense. It is only an analysis. The optimizations should depend on the jvmti capabilities. The correct setup would be to handle this in CompilerConfig::ergo_initialize(): If the jvmti capabilities allow, enable the optimizations EliminateAllocations or EliminateLocks/EliminateNestedLocks. If one of these optimizations is on, enable EscapeAnalysis. -- end side comment. So I would propose the following comments: // In the scope of this safepoints there are objects // that do not globally escape. They are either NoEscape or // ArgEscape. As such, they might be subject to optimizations. // Persist this information here so that the frame an the // Objects in scope can // be deoptimized if jvmti accesses an object at this safepoint. void set_not_global_escape_in_scope(bool b) { // This call passes objects that do not globally escape // to its callee. The object might be subject to optimization, // e.g. a lock might be omitted. Persist this information here // so that on a jvmti access to the callee frame we can deoptimize // the object and this frame. void set_arg_escape(bool f) { _arg_escape = f; } Actuall I am not sure whether the name of these fields (and all the others in the course of this change) should refer to escape analysis. I think the term "Object deoptimization" you also use is much better. You could call these properties (througout the whole change) set_optimized_objects_in_scope() and set_passes_optimized_objects(). I think this would make the whole matter much easier to understand. Anyways, locks can already be removed without running escape analysis at all. C2 recognizes some local patterns that allow this. escape.h|cpp The code looks good. Line 325: The comment could be a bit more elaborate: // Annotate at safepoints if they have <= ArgEscape objects in their // scope. Additionally, if the safepoint is a java call, annotate // whether it passes ArgEscape objects as parameters. And maybe add these comments?: // Returns true if an oop in the scope of sfn does not escape // globally. bool ConnectionGraph::has_not_global_escape_in_scope(SafePointNode* sfn) { // Returns true if at least one of the arguments to the call is an oop // that does not escape globally. bool ConnectionGraph::has_arg_escape(CallJavaNode* call) { General question: You collect the information you want to annotate to the method during escape analysis. Don't you overestimate the optimized objects by this? E.g. elimination of allocations does bail out for various reasons. At the end, no optimization might have happened, but then during runtime the frame is deoptimized nevertheless. machnode.hpp: Extends MachSafePointNode similar to the ideal version. Good. matcher.cpp Copy info from ideal to mach node. good. output.cpp Now finally the information is written to the debug info. Good. --------------------------------------------------------- So now let's have a look at the runtime part (including relaxing constraints to escape analysis): rootResolver.cpp Adapt to changed interface. good. c2compiler.cpp / macro.cpp Make EscpaeAnlysis independent of jvmti capabilities. Good. jvmtiEnv.cpp/jvmtiEnvBase.cpp You add deoptimization of objects where they are accessed. good. jvmtiImpl.cpp In deoptimize_objects, you check for DoEscapeAnalysis. This is correct given the current design of the flag handling in the compiler. It's not really nice to have a dependency to C2 here, though. I understand it's an optimization, the code could be run anyways, it would check but not find anything. But actually I would excpect dependencies on EliminateLocks and EliminateAllocations (if they were set according to jvmti capabilitiers as I elaborated above.) Would it make sense to protect the ArgEscape loop by if (EliminateLocks)? jvmtiTagMap.cpp Deoptimize for jvmti operations. Good. deoptimization.cpp I guess this is the core of your work. You add a new mode that just deoptimizes objects but not frames. Good idea. You have to use reallocated objects in upper frames, or by jvmti accesses to inner frames, which can not easily be replaced by interpreter frames. This way you can wait with replacing the frame until just before execution returns. eliminate_allocations(): (Strange method name, should at least be in past tense, even better reallocate_eliminated_allocations() or allocate_scalarized_objects(). Confused me until I groked the code. Legacy though, not your business.) It's not that nice to return whether you only deoptimized objects by the boolean reference argument. After all, it again depends on the mode you pass in. A different design would be to clone the method and have an eliminate_allocations_no_unpack() variant, but that would not be better as some code would be duplicated. Maybe a comment for argument eliminate_allocations: // deoptimized_objects is set to true if objects were deoptimized // but not the frame. It is unchanged if there are no objects to // be deoptimized, or if the frame was deoptim Similar for eliminate_locks(): // deoptimized_objects is set to true if objects were relocked, // else it is left unchanged. You reuse and extend the existing realloc/relock_objects, but extended it. deoptimize_objects_internal() Simple version of fetch_unroll_info_helper for EscapeBarrier. Good. I attributed the comment "Then relock objects if synchronization on them was eliminated." to the if() just below. Add an empty line to make clear the comment refers to the next 10 lines. Alternatively, replace the whole comment by // At first, reallocate the non-escaping objects and restore their fields // so they are available for relocking. And add // Now relock objects with eliminated locks. befor the if ((DoEscape... below. In fetch_unroll_info_helper, I don't understand why you need && !EscapeBarrier::objs_are_deoptimized(thread, deoptee.id())) { for eliminated locks, but not for skalar replaced objects? I would guess it is because the eliminated locks can be applied to argEscape, but scalar replacement only to noescape objects? I.e. it might have been done before? But why isn't this the case for eliminate_allocations? deoptimize_objects_internal does both unconditionally, so both can happen to inner frames, right? relock_objects() Ok, you need to undo biased locking. Also, you remember the lock nesting for later relocking if waiting for lock. revoke_for_object_deoptimization() I like if boolean operators are at the beginning of broken lines, but I think hotspot convention is to have them at the end. Code will get much more simple if BiasedLocking is removed. EscapeBarrier:: ... (This class maybe would qualify for a file of its own.) deoptimize_objects() I would mention escape analysis only as side remark. Also, as I understand, there is only one frame at given depth? // Deoptimize frames with optimized objects. This can be omitted locks and // objects not allocated but replaced by scalars. In C2, these optimizations // are based on escape analysis. // Up to depth, deoptimize frames with any optimized objects. // From depth to entry_frame, deoptimize only frames that // pass optimized objects to their callees. (First part similar for the comment above EscapeBarrier::deoptimize_objects_internal().) What is the check (cur_depth <= depth) good for? Can you ever walk past entry_frame? Isn't vf->is_compiled_frame() prerequisite that "Move to next physical frame" is needed? You could move it into the other check. If so, similar for deoptimize_objects_all_threads(). Syncronization: looks good. I think others had a look at this before. EscapeBarrier::deoptimize_objects_internal() The method name is misleading, it is not used by deoptimize_objects(). Also, method with the same name is in Deopitmization. Proposal: deoptimize_objects_thread() ? C1 stubs: this really shows you tested all configurations, great! mutexLocker: ok. objectMonitor.cpp: ok stackValue.hpp Is this missing clearing a bug? thread.hpp I would remove "_ea" from the flag and method names. Renaming deferred_locals to deferred_updates is good, as well as adding a datastructure for it. (Adding this data structure might be a breakout, too.) good. thread.cpp good. vframe.cpp Is this a bug in existing code? Makes sense. vframe_hp.hpp (What stands _hp for? helper? The file should be named compiledVFrame ...) not_global_escape_in_scope() ... Again, you mention escape analysis here. Comments above hold, too. You introduce JvmtiDeferredUpdates. Good. vframe_hp.cpp Changes for JvmtiDeferredUpdates, escape state accessors, line 422: Would an assertion assert(!info->owner_is_scalar_replaced(), ...) hold here? macros.hpp Good. Test coding ============ compileBroker.h|cpp You introduce a third class of threads handled here and add a new flag to distinguish it. Before, the two kinds of threads were distinguished implicitly by passing in a compiler for compiler threads. The new thread kind is only used for testing in debug. make_thread: You could assert (comp != NULL...) to assure previous conditions. line 989 indentation broken escape.cpp You enable the optimization in case of testruns. good. whitebox.cpp ok. deoptimization.cpp deoptimize_objects_alot_loop() Good. globals.hpp Nice docu of flags, but pleas mention "for testing purposes" or the like in DeoptimizeObjectsALot. I would place the flags next to each other. interfaceSupport.cpp: good. I'll look at the test themselves in an extra mail (learning from Martin ??) Best regards, Goetz. > -----Original Message----- > From: Reingruber, Richard > Sent: Wednesday, April 1, 2020 8:15 AM > To: Doerr, Martin ; 'Robbin Ehn' > ; Lindenmaier, Goetz > ; David Holmes ; > Vladimir Kozlov (vladimir.kozlov at oracle.com) ; > serviceability-dev at openjdk.java.net; hotspot-compiler- > dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net > Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better Performance > in the Presence of JVMTI Agents > > Hi Martin, > > > thanks for addressing all my points. I've looked over webrev.5 and I'm > satisfied with your changes. > > Thanks! > > > I had also promised to review the tests. > > Thanks++ > I appreciate it very much, the tests are many lines of code. > > > test/jdk/com/sun/jdi/EATests.java > > This is a substantial amount of tests which is appropriate for a such a large > change. Skipping some subtests with UseJVMCICompiler makes sense > because it doesn't provide the necessary JVMTI functionality, yet. > > Nice work! > > I also like that you test with and without BiasedLocking. Your tests will still > be fine after BiasedLocking deprecation. > > Hope so :) > > > Very minor nits: > > - 2 typos in comment above EARelockingNestedInflatedTarget: "lockes are > ommitted" (sounds funny) > > - You sometimes write "graal" and sometimes "Graal". I guess the capital G > is better. (Also in EATestsJVMCI.java.) > > > test/jdk/com/sun/jdi/EATestsJVMCI.java > > EATests with Graal enabled. Nice that you support Graal to some extent. > Maybe Graal folks want to enhance them in the future. I think this is a good > starting point. > > Will change this in the next webrev. > > > Conclusion: Looks good and not trivial :-) > > Now, you have one full review. I'd be ok with covering 2nd review by partial > reviews. > > Compiler and JVMTI parts are not too complicated IMHO. > > Runtime part should get at least one additional careful review. > > Thanks a lot, > Richard. > > -----Original Message----- > From: Doerr, Martin > Sent: Dienstag, 31. M?rz 2020 16:01 > To: Reingruber, Richard ; 'Robbin Ehn' > ; Lindenmaier, Goetz > ; David Holmes ; > Vladimir Kozlov (vladimir.kozlov at oracle.com) ; > serviceability-dev at openjdk.java.net; hotspot-compiler- > dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net > Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better Performance > in the Presence of JVMTI Agents > > Hi Richard, > > thanks for addressing all my points. I've looked over webrev.5 and I'm > satisfied with your changes. > > > I had also promised to review the tests. > > test/hotspot/jtreg/serviceability/jvmti/Heap/IterateHeapWithEscapeAnalysis > Enabled.java > Thanks for updating the @summary comment. Looks good in webrev.5. > > test/hotspot/jtreg/serviceability/jvmti/Heap/libIterateHeapWithEscapeAnaly > sisEnabled.c > JVMTI agent for object tagging and heap iteration. Good. > > test/jdk/com/sun/jdi/EATests.java > This is a substantial amount of tests which is appropriate for a such a large > change. Skipping some subtests with UseJVMCICompiler makes sense > because it doesn't provide the necessary JVMTI functionality, yet. > Nice work! > I also like that you test with and without BiasedLocking. Your tests will still be > fine after BiasedLocking deprecation. > > Very minor nits: > - 2 typos in comment above EARelockingNestedInflatedTarget: "lockes are > ommitted" (sounds funny) > - You sometimes write "graal" and sometimes "Graal". I guess the capital G is > better. (Also in EATestsJVMCI.java.) > > test/jdk/com/sun/jdi/EATestsJVMCI.java > EATests with Graal enabled. Nice that you support Graal to some extent. > Maybe Graal folks want to enhance them in the future. I think this is a good > starting point. > > > Conclusion: Looks good and not trivial :-) > Now, you have one full review. I'd be ok with covering 2nd review by partial > reviews. > Compiler and JVMTI parts are not too complicated IMHO. > Runtime part should get at least one additional careful review. > > Best regards, > Martin > > > > -----Original Message----- > > From: Reingruber, Richard > > Sent: Montag, 30. M?rz 2020 10:32 > > To: Doerr, Martin ; 'Robbin Ehn' > > ; Lindenmaier, Goetz > > ; David Holmes > ; > > Vladimir Kozlov (vladimir.kozlov at oracle.com) > > ; serviceability-dev at openjdk.java.net; > > hotspot-compiler-dev at openjdk.java.net; hotspot-runtime- > > dev at openjdk.java.net > > Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better Performance > > in the Presence of JVMTI Agents > > > > Hi, > > > > this is webrev.5 based on Robbin's feedback and Martin's review - thanks! :) > > > > The change affects jvmti, hotspot and c2. Partial reviews are very welcome > > too. > > > > Full: http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.5/ > > Delta: > > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.5.inc/ > > > > Robbin, Martin, please let me know, if anything shouldn't be quite as you > > wanted it. Also find my > > comments on your feedback below. > > > > Robbin, can I count you as Reviewer for the runtime part? > > > > Thanks, Richard. > > > > -- > > > > > DeoptimizeObjectsALotThread is only used in compileBroker.cpp. > > > You can move both declaration and definition to that file, no need to > > clobber > > > thread.[c|h]pp. (and the static function deopt_objs_alot_thread_entry) > > > > Done. > > > > > Does JvmtiDeferredUpdates really need to be in thread.hpp, can't be in > it's > > own > > > hpp file? It doesn't seem right to add JVM TI classes into thread.hpp. > > > > I moved JvmtiDeferredUpdates to vframe_hp.hpp where preexisting > > jvmtiDeferredLocalVariableSet is > > declared. > > > > > src/hotspot/share/code/compiledMethod.cpp > > > Nice cleanup! > > > > Thanks :) > > > > > src/hotspot/share/code/debugInfoRec.cpp > > > src/hotspot/share/code/debugInfoRec.hpp > > > Additional parmeters. (Remark: I think "non_global_escape_in_scope" > > would read better than "not_global_escape_in_scope", but your version is > > consistent with existing code, so no change request from my side.) Ok. > > > > I've been thinking about this too and finally stayed with > > not_global_escape_in_scope. It's supposed > > to mean an object whose escape state is not GlobalEscape is in scope. > > > > > src/hotspot/share/compiler/compileBroker.cpp > > > src/hotspot/share/compiler/compileBroker.hpp > > > Extra thread for DeoptimizeObjectsALot. (Remark: I would have put it into > > a follow up change together with the test in order to make this webrev > > smaller, but since it is included, I'm reviewing everything at once. Not a big > > deal.) Ok. > > > > Yes the change would be a little smaller. And if it helps I'll split it off. In > > general I prefer > > patches that bring along a suitable amount of tests. > > > > > src/hotspot/share/opto/c2compiler.cpp > > > Make do_escape_analysis independent of JVMCI capabilities. Nice! > > > > It is the main goal of the enhancement. It is done for C2, but could be done > > for JVMCI compilers > > with just a small effort as well. > > > > > src/hotspot/share/opto/escape.cpp > > > Annotation for MachSafePointNodes. Your added functionality looks > > correct. > > > But I'd prefer to move the bulky code out of the large function. > > > I suggest to factor out something like has_not_global_escape and > > has_arg_escape. So the code could look like this: > > > SafePointNode* sfn = sfn_worklist.at(next); > > > sfn->set_not_global_escape_in_scope(has_not_global_escape(sfn)); > > > if (sfn->is_CallJava()) { > > > CallJavaNode* call = sfn->as_CallJava(); > > > call->set_arg_escape(has_arg_escape(call)); > > > } > > > This would also allow us to get rid of the found_..._escape_in_args > > variables making the loops better readable. > > > > Done. > > > > > It's kind of ugly to use strcmp to recognize uncommon trap, but that > seems > > to be the way to do it (there are more such places). So it's ok. > > > > Yeah. I copied the snippet. > > > > > src/hotspot/share/prims/jvmtiImpl.cpp > > > src/hotspot/share/prims/jvmtiImpl.hpp > > > The sequence is pretty complex: > > > VM_GetOrSetLocal element initialization executes EscapeBarrier code > > which suspends the target thread (extra VM Operation). > > > > Note that the target threads have to be suspended already for > > VM_GetOrSetLocal*. So it's mainly the > > synchronization effect of EscapeBarrier::sync_and_suspend_one() that is > > required here. Also no extra > > _handshake_ is executed, since sync_and_suspend_one() will find the > > target threads already > > suspended. > > > > > VM_GetOrSetLocal::doit_prologue performs object deoptimization (by > VM > > Thread to prepare VM Operation with frame deoptimization). > > > VM_GetOrSetLocal destructor implicitly calls EscapeBarrier destructor > > which resumes the target thread. > > > But I don't have any improvement proposal. Performance is probably not > a > > concern, here. So it's ok. > > > > > VM_GetOrSetLocal::deoptimize_objects deoptimizes the top frame if it > > has non-globally escaping objects and other frames if they have arg > escaping > > ones. Good. > > > > It's not specifically the top frame, but the frame that is accessed. > > > > > src/hotspot/share/runtime/deoptimization.cpp > > > Object deoptimization. I have more comments and proposals, here. > > > First of all, handling recursive and waiting locks in relock_objects is tricky, > > but looks correct. > > > Comments are sufficient to understand why things are done as they are > > implemented. > > > > > BiasedLocking related parts are complex, but we may get rid of them in > the > > future (with BiasedLocking removal). > > > Anyway, looks correct, too. > > > > > Typo in comment: "regularily" => "regularly" > > > > > Deoptimization::fetch_unroll_info_helper is the only place where > > _jvmti_deferred_updates get deallocated (except JavaThread destructor). > > But I think we always go through it, so I can't see a memory leak or such > kind > > of issues. > > > > That's correct. The compiled frame for which deferred updates are > allocated > > is always deoptimized > > before (see EscapeBarrier::deoptimize_objects()). This is also asserted in > > compiledVFrame::update_deferred_value(). I've added the same assertion > > to > > Deoptimization::relock_objects(). So we can be sure that > > _jvmti_deferred_updates are deallocated > > again in fetch_unroll_info_helper(). > > > > > EscapeBarrier::deoptimize_objects: ResourceMark should use > > calling_thread(). > > > > Sure, well spotted! > > > > > You can use MutexLocker and MonitorLocker with Thread* to save the > > Thread::current() call. > > > > Right, good hint. This was recently introduced with 8235678. I even had to > > resolve conflicts. Should > > have done this then. > > > > > I'd make set_objs_are_deoptimized static and remove it from the > > EscapeBarrier interface because I think it shouldn't be used outside of > > EscapeBarrier::deoptimize_objects. > > > > Done. > > > > > Typo in comment: "we must only deoptimize" => "we only have to > > deoptimize" > > > > Replaced with "[...] we deoptimize iff local objects are passed as args" > > > > > "bool EscapeBarrier::deoptimize_objects(intptr_t* fr_id)" is trivial and > > barrier_active() is redundant. Implementation can get moved to hpp file. > > > > Ok. Done. > > > > > I'll get back to suspend flags, later. > > > > > There are weird cases regarding _self_deoptimization_in_progress. > > > Assume we have 3 threads A, B and C. A deopts C, B deopts C, C deopts C. > > C can set _self_deoptimization_in_progress while A performs the > handshake > > for suspending C. I think this doesn't lead to errors, but it's probably not > > desired. > > > I think it would be better to use only one "wait" call in > > sync_and_suspend_one and sync_and_suspend_all. > > > > You're right. We've discussed that face-to-face, but couldn't find a real > issue. > > But now, thinking again, a reckon I found one: > > > > 2808 // Sync with other threads that might be doing deoptimizations > > 2809 { > > 2810 // Need to switch to _thread_blocked for the wait() call > > 2811 ThreadBlockInVM tbivm(_calling_thread); > > 2812 MonitorLocker ml(EscapeBarrier_lock, > > Mutex::_no_safepoint_check_flag); > > 2813 while (_self_deoptimization_in_progress) { > > 2814 ml.wait(); > > 2815 } > > 2816 > > 2817 if (self_deopt()) { > > 2818 _self_deoptimization_in_progress = true; > > 2819 } > > 2820 > > 2821 while (_deoptee_thread->is_ea_obj_deopt_suspend()) { > > 2822 ml.wait(); > > 2823 } > > 2824 > > 2825 if (self_deopt()) { > > 2826 return; > > 2827 } > > 2828 > > 2829 // set suspend flag for target thread > > 2830 _deoptee_thread->set_ea_obj_deopt_flag(); > > 2831 } > > > > - A waits in 2822 > > - C is suspended > > - B notifies all in resume_one() > > - A and C wake up > > - C wins over A and sets _self_deoptimization_in_progress = true in 2818 > > - C does the self deoptimization > > - A executes 2830 _deoptee_thread->set_ea_obj_deopt_flag() > > > > C will self suspend at some undefined point. The resulting state is illegal. > > > > > I first thought it'd be better to move ThreadBlockInVM before wait() to > > reduce thread state transitions, but that seems to be problematic because > > ThreadBlockInVM destructor contains a safepoint check which we > shouldn't > > do while holding EscapeBarrier_lock. So no change request. > > > > Yes, would be nice to have the state change only if needed, but for the > > reason you mentioned it is > > not quite as easy as it seems to be. I experimented as well with a second > > lock, but did not succeed. > > > > > Change in thred_added: > > > I think the sequence would be more comprehensive if we waited for > > deopt_all_threads in Thread::start and all other places where a new thread > > can run into Java code (e.g. JVMTI attach). > > > Your version makes new threads come up with suspend flag set. That > looks > > correct, too. Advantage is that you only have to change one place > > (thread_added). It'll be interesting to see how it will look like when we use > > async handshakes instead of suspend flags. > > > For now, I'm ok with your version. > > > > I had a version that did what you are suggesting. The current version also > has > > the advantage, that > > there are fewer places where a thread has to wait for ongoing object > > deoptimization. This means > > viewer places where you have to worry about correct thread state > > transitions, possible deadlocks, > > and if all oops are properly Handle'ed. > > > > > I'd only move MutexLocker ml(EscapeBarrier_lock...) after if (!jt- > > >is_hidden_from_external_view()). > > > > Done. > > > > > Having 4 different deoptimize_objects functions makes it a little hard to > > keep an overview of which one is used for what. > > > Maybe adding suffixes would help a little bit, but I can also live with what > > you have. > > > Implementation looks correct to me. > > > > 2 are internal. I added the suffix _internal to them. This leaves 2 to choose > > from. > > > > > src/hotspot/share/runtime/deoptimization.hpp > > > Escape barriers and object deoptimization functions. > > > Typo in comment: "helt" => "held" > > > > Done in place already. > > > > > src/hotspot/share/runtime/interfaceSupport.cpp > > > InterfaceSupport::deoptimizeAllObjects() is only used for > > DeoptimizeObjectsALot = 1. > > > I think DeoptimizeObjectsALot = 2 is more important, but I think it's not > bad > > to have DeoptimizeObjectsALot = 1 in addition. Ok. > > > > I never used DeoptimizeObjectsALot = 1 that much. It could be more > > deterministic in single threaded > > scenarios. I wouldn't object to get rid of it though. > > > > > src/hotspot/share/runtime/stackValue.hpp > > > Better reinitilization in StackValue. Good. > > > > StackValue::obj_is_scalar_replaced() should not return true after calling > > set_obj(). > > > > > src/hotspot/share/runtime/thread.cpp > > > src/hotspot/share/runtime/thread.hpp > > > src/hotspot/share/runtime/thread.inline.hpp > > > wait_for_object_deoptimization, suspend flag, deferred updates and test > > feature to deoptimize objects. > > > > > In the long term, we want to get rid of suspend flags, so it's not so nice to > > introduce a new one. But I agree with G?tz that it should be acceptable as > > temporary solution until async handshakes are available (which takes more > > time). So I'm ok with your change. > > > > I'm keen to build the feature on async handshakes when the arive. > > > > > You can use MutexLocker with Thread*. > > > > Done. > > > > > JVMTIDeferredUpdates: I agree with Robin. It'd be nice to move the class > > out of thread.hpp. > > > > Done. > > > > > src/hotspot/share/runtime/vframe.cpp > > > Added support for entry frame to new_vframe. Ok. > > > > > > > src/hotspot/share/runtime/vframe_hp.cpp > > > src/hotspot/share/runtime/vframe_hp.hpp > > > > > I think code()->as_nmethod() in not_global_escape_in_scope() and > > arg_escape() should better be under #ifdef ASSERT or inside the assert > > statement (no need for code cache walking in product build). > > > > Done. > > > > > jvmtiDeferredLocalVariableSet::update_monitors: > > > Please add a comment explaining that owner referenced by original info > > may be scalar replaced, but it is deoptimized in the vframe. > > > > Done. > > > > -----Original Message----- > > From: Doerr, Martin > > Sent: Donnerstag, 12. M?rz 2020 17:28 > > To: Reingruber, Richard ; 'Robbin Ehn' > > ; Lindenmaier, Goetz > > ; David Holmes > ; > > Vladimir Kozlov (vladimir.kozlov at oracle.com) > > ; serviceability-dev at openjdk.java.net; > > hotspot-compiler-dev at openjdk.java.net; hotspot-runtime- > > dev at openjdk.java.net > > Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better Performance > > in the Presence of JVMTI Agents > > > > Hi Richard, > > > > > > I managed to find time for a (almost) complete review of webrev.4. (I'll > > review the tests separately.) > > > > First of all, the change seems to be in pretty good quality for its significant > > complexity. I couldn't find any real bugs. But I'd like to propose minor > > improvements. > > I'm convinced that it's mature because we did substantial testing. > > > > I like the new functionality for object deoptimization. It can possibly be > > reused for future escape analysis based optimizations. So I appreciate > having > > it available in the code base. > > In addition to that, your change makes the JVMTI implementation better > > integrated into the VM. > > > > > > Now to the details: > > > > > > src/hotspot/share/c1/c1_IR.hpp > > describe_scope parameters. Ok. > > > > > > src/hotspot/share/ci/ciEnv.cpp > > src/hotspot/share/ci/ciEnv.hpp > > Fix for JvmtiExport::can_walk_any_space() capability. Ok. > > > > > > src/hotspot/share/code/compiledMethod.cpp > > Nice cleanup! > > > > > > src/hotspot/share/code/debugInfoRec.cpp > > src/hotspot/share/code/debugInfoRec.hpp > > Additional parmeters. (Remark: I think "non_global_escape_in_scope" > > would read better than "not_global_escape_in_scope", but your version is > > consistent with existing code, so no change request from my side.) Ok. > > > > > > src/hotspot/share/code/nmethod.cpp > > Nice cleanup! > > > > > > src/hotspot/share/code/pcDesc.hpp > > Additional parameters. Ok. > > > > > > src/hotspot/share/code/scopeDesc.cpp > > src/hotspot/share/code/scopeDesc.hpp > > Improved implementation + additional parameters. Ok. > > > > > > src/hotspot/share/compiler/compileBroker.cpp > > src/hotspot/share/compiler/compileBroker.hpp > > Extra thread for DeoptimizeObjectsALot. (Remark: I would have put it into a > > follow up change together with the test in order to make this webrev > > smaller, but since it is included, I'm reviewing everything at once. Not a big > > deal.) Ok. > > > > > > src/hotspot/share/jvmci/jvmciCodeInstaller.cpp > > Additional parameters. Ok. > > > > > > src/hotspot/share/opto/c2compiler.cpp > > Make do_escape_analysis independent of JVMCI capabilities. Nice! > > > > > > src/hotspot/share/opto/callnode.hpp > > Additional fields for MachSafePointNodes. Ok. > > > > > > src/hotspot/share/opto/escape.cpp > > Annotation for MachSafePointNodes. Your added functionality looks > correct. > > But I'd prefer to move the bulky code out of the large function. > > I suggest to factor out something like has_not_global_escape and > > has_arg_escape. So the code could look like this: > > SafePointNode* sfn = sfn_worklist.at(next); > > sfn->set_not_global_escape_in_scope(has_not_global_escape(sfn)); > > if (sfn->is_CallJava()) { > > CallJavaNode* call = sfn->as_CallJava(); > > call->set_arg_escape(has_arg_escape(call)); > > } > > This would also allow us to get rid of the found_..._escape_in_args > variables > > making the loops better readable. > > > > It's kind of ugly to use strcmp to recognize uncommon trap, but that seems > > to be the way to do it (there are more such places). So it's ok. > > > > > > src/hotspot/share/opto/machnode.hpp > > Additional fields for MachSafePointNodes. Ok. > > > > > > src/hotspot/share/opto/macro.cpp > > Allow elimination of non-escaping allocations. Ok. > > > > > > src/hotspot/share/opto/matcher.cpp > > src/hotspot/share/opto/output.cpp > > Copy attribute / pass parameters. Ok. > > > > > > src/hotspot/share/prims/jvmtiCodeBlobEvents.cpp > > Nice cleanup! > > > > > > src/hotspot/share/prims/jvmtiEnv.cpp > > src/hotspot/share/prims/jvmtiEnvBase.cpp > > Escape barriers + deoptimize objects for target thread. Good. > > > > > > src/hotspot/share/prims/jvmtiImpl.cpp > > src/hotspot/share/prims/jvmtiImpl.hpp > > The sequence is pretty complex: > > VM_GetOrSetLocal element initialization executes EscapeBarrier code > which > > suspends the target thread (extra VM Operation). > > VM_GetOrSetLocal::doit_prologue performs object deoptimization (by VM > > Thread to prepare VM Operation with frame deoptimization). > > VM_GetOrSetLocal destructor implicitly calls EscapeBarrier destructor > which > > resumes the target thread. > > But I don't have any improvement proposal. Performance is probably not a > > concern, here. So it's ok. > > > > VM_GetOrSetLocal::deoptimize_objects deoptimizes the top frame if it has > > non-globally escaping objects and other frames if they have arg escaping > > ones. Good. > > > > > > src/hotspot/share/prims/jvmtiTagMap.cpp > > Escape barriers + deoptimize objects for all threads. Ok. > > > > > > src/hotspot/share/prims/whitebox.cpp > > Added WB_IsFrameDeoptimized to API. Ok. > > > > > > src/hotspot/share/runtime/deoptimization.cpp > > Object deoptimization. I have more comments and proposals, here. > > First of all, handling recursive and waiting locks in relock_objects is tricky, > but > > looks correct. > > Comments are sufficient to understand why things are done as they are > > implemented. > > > > BiasedLocking related parts are complex, but we may get rid of them in the > > future (with BiasedLocking removal). > > Anyway, looks correct, too. > > > > Typo in comment: "regularily" => "regularly" > > > > Deoptimization::fetch_unroll_info_helper is the only place where > > _jvmti_deferred_updates get deallocated (except JavaThread destructor). > > But I think we always go through it, so I can't see a memory leak or such > kind > > of issues. > > > > EscapeBarrier::deoptimize_objects: ResourceMark should use > > calling_thread(). > > > > You can use MutexLocker and MonitorLocker with Thread* to save the > > Thread::current() call. > > > > I'd make set_objs_are_deoptimized static and remove it from the > > EscapeBarrier interface because I think it shouldn't be used outside of > > EscapeBarrier::deoptimize_objects. > > > > Typo in comment: "we must only deoptimize" => "we only have to > > deoptimize" > > > > "bool EscapeBarrier::deoptimize_objects(intptr_t* fr_id)" is trivial and > > barrier_active() is redundant. Implementation can get moved to hpp file. > > > > I'll get back to suspend flags, later. > > > > There are weird cases regarding _self_deoptimization_in_progress. > > Assume we have 3 threads A, B and C. A deopts C, B deopts C, C deopts C. > C > > can set _self_deoptimization_in_progress while A performs the handshake > > for suspending C. I think this doesn't lead to errors, but it's probably not > > desired. > > I think it would be better to use only one "wait" call in > > sync_and_suspend_one and sync_and_suspend_all. > > > > I first thought it'd be better to move ThreadBlockInVM before wait() to > > reduce thread state transitions, but that seems to be problematic because > > ThreadBlockInVM destructor contains a safepoint check which we > shouldn't > > do while holding EscapeBarrier_lock. So no change request. > > > > Change in thred_added: > > I think the sequence would be more comprehensive if we waited for > > deopt_all_threads in Thread::start and all other places where a new thread > > can run into Java code (e.g. JVMTI attach). > > Your version makes new threads come up with suspend flag set. That looks > > correct, too. Advantage is that you only have to change one place > > (thread_added). It'll be interesting to see how it will look like when we use > > async handshakes instead of suspend flags. > > For now, I'm ok with your version. > > > > I'd only move MutexLocker ml(EscapeBarrier_lock...) after if (!jt- > > >is_hidden_from_external_view()). > > > > Having 4 different deoptimize_objects functions makes it a little hard to > keep > > an overview of which one is used for what. > > Maybe adding suffixes would help a little bit, but I can also live with what > you > > have. > > Implementation looks correct to me. > > > > > > src/hotspot/share/runtime/deoptimization.hpp > > Escape barriers and object deoptimization functions. > > Typo in comment: "helt" => "held" > > > > > > src/hotspot/share/runtime/globals.hpp > > Addition of develop flag DeoptimizeObjectsALotInterval. Ok. > > > > > > src/hotspot/share/runtime/interfaceSupport.cpp > > InterfaceSupport::deoptimizeAllObjects() is only used for > > DeoptimizeObjectsALot = 1. > > I think DeoptimizeObjectsALot = 2 is more important, but I think it's not bad > > to have DeoptimizeObjectsALot = 1 in addition. Ok. > > > > > > src/hotspot/share/runtime/interfaceSupport.inline.hpp > > Addition of deoptimizeAllObjects. Ok. > > > > > > src/hotspot/share/runtime/mutexLocker.cpp > > src/hotspot/share/runtime/mutexLocker.hpp > > Addition of EscapeBarrier_lock. Ok. > > > > > > src/hotspot/share/runtime/objectMonitor.cpp > > Make recursion count relock aware. Ok. > > > > > > src/hotspot/share/runtime/stackValue.hpp > > Better reinitilization in StackValue. Good. > > > > > > src/hotspot/share/runtime/thread.cpp > > src/hotspot/share/runtime/thread.hpp > > src/hotspot/share/runtime/thread.inline.hpp > > wait_for_object_deoptimization, suspend flag, deferred updates and test > > feature to deoptimize objects. > > > > In the long term, we want to get rid of suspend flags, so it's not so nice to > > introduce a new one. But I agree with G?tz that it should be acceptable as > > temporary solution until async handshakes are available (which takes more > > time). So I'm ok with your change. > > > > You can use MutexLocker with Thread*. > > > > JVMTIDeferredUpdates: I agree with Robin. It'd be nice to move the class > out > > of thread.hpp. > > > > > > src/hotspot/share/runtime/vframe.cpp > > Added support for entry frame to new_vframe. Ok. > > > > > > src/hotspot/share/runtime/vframe_hp.cpp > > src/hotspot/share/runtime/vframe_hp.hpp > > > > I think code()->as_nmethod() in not_global_escape_in_scope() and > > arg_escape() should better be under #ifdef ASSERT or inside the assert > > statement (no need for code cache walking in product build). > > > > jvmtiDeferredLocalVariableSet::update_monitors: > > Please add a comment explaining that owner referenced by original info > may > > be scalar replaced, but it is deoptimized in the vframe. > > > > > > src/hotspot/share/utilities/macros.hpp > > Addition of NOT_COMPILER2_OR_JVMCI_RETURN macros. Ok. > > > > > > > test/hotspot/jtreg/serviceability/jvmti/Heap/IterateHeapWithEscapeAnalysi > > sEnabled.java > > > test/hotspot/jtreg/serviceability/jvmti/Heap/libIterateHeapWithEscapeAnal > > ysisEnabled.c > > New test. Will review separately. > > > > > > test/jdk/TEST.ROOT > > Addition of vm.jvmci as required property. Ok. > > > > > > test/jdk/com/sun/jdi/EATests.java > > test/jdk/com/sun/jdi/EATestsJVMCI.java > > New test. Will review separately. > > > > > > test/lib/sun/hotspot/WhiteBox.java > > Added isFrameDeoptimized to API. Ok. > > > > > > That was it. Best regards, > > Martin > > > > > > > -----Original Message----- > > > From: hotspot-compiler-dev > > bounces at openjdk.java.net> On Behalf Of Reingruber, Richard > > > Sent: Dienstag, 3. M?rz 2020 21:23 > > > To: 'Robbin Ehn' ; Lindenmaier, Goetz > > > ; David Holmes > > ; > > > Vladimir Kozlov (vladimir.kozlov at oracle.com) > > > ; serviceability-dev at openjdk.java.net; > > > hotspot-compiler-dev at openjdk.java.net; hotspot-runtime- > > > dev at openjdk.java.net > > > Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better > > > Performance in the Presence of JVMTI Agents > > > > > > Hi Robbin, > > > > > > > > I understand that Robbin proposed to replace the usage of > > > > > _suspend_flag with handshakes. Apparently, async handshakes > > > > > are needed to do so. We have been waiting a while for removal > > > > > of the _suspend_flag / introduction of async handshakes [2]. > > > > > What is the status here? > > > > > > > I have an old prototype which I would like to continue to work on. > > > > So do not assume asynch handshakes will make 15. > > > > Even if it would, I think there are a lot more investigate work to remove > > > > _suspend_flag. > > > > > > Let us know, if we can be of any help to you and be it only testing. > > > > > > > >> Full: > > > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.4/ > > > > > > > DeoptimizeObjectsALotThread is only used in compileBroker.cpp. > > > > You can move both declaration and definition to that file, no need to > > > clobber > > > > thread.[c|h]pp. (and the static function deopt_objs_alot_thread_entry) > > > > > > Will do. > > > > > > > Does JvmtiDeferredUpdates really need to be in thread.hpp, can't be in > > it's > > > own > > > > hpp file? It doesn't seem right to add JVM TI classes into thread.hpp. > > > > > > You are right. It shouldn't be declared in thread.hpp. I will look into that. > > > > > > > Note that we also think we may have a bug in deopt: > > > > https://bugs.openjdk.java.net/browse/JDK-8238237 > > > > > > > I think it would be best, if possible, to push after that is resolved. > > > > > > Sure. > > > > > > > Not even nearly a full review :) > > > > > > I know :) > > > > > > Anyways, thanks a lot, > > > Richard. > > > > > > > > > -----Original Message----- > > > From: Robbin Ehn > > > Sent: Monday, March 2, 2020 11:17 AM > > > To: Lindenmaier, Goetz ; Reingruber, > > Richard > > > ; David Holmes > > ; > > > Vladimir Kozlov (vladimir.kozlov at oracle.com) > > > ; serviceability-dev at openjdk.java.net; > > > hotspot-compiler-dev at openjdk.java.net; hotspot-runtime- > > > dev at openjdk.java.net > > > Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better > Performance > > > in the Presence of JVMTI Agents > > > > > > Hi, > > > > > > On 2/24/20 5:39 PM, Lindenmaier, Goetz wrote: > > > > Hi, > > > > > > > > I had a look at the progress of this change. Nothing > > > > happened since Richard posted his update using more > > > > handshakes [1]. > > > > But we (SAP) would appreciate a lot if this change could > > > > be successfully reviewed and pushed. > > > > > > > > I think there is basic understanding that this > > > > change is helpful. It fixes a number of issues with JVMTI, > > > > and will deliver the same performance benefits as EA > > > > does in current production mode for debugging scenarios. > > > > > > > > This is important for us as we run our VMs prepared > > > > for debugging in production mode. > > > > > > > > I understand that Robbin proposed to replace the usage of > > > > _suspend_flag with handshakes. Apparently, async handshakes > > > > are needed to do so. We have been waiting a while for removal > > > > of the _suspend_flag / introduction of async handshakes [2]. > > > > What is the status here? > > > > > > I have an old prototype which I would like to continue to work on. > > > So do not assume asynch handshakes will make 15. > > > Even if it would, I think there are a lot more investigate work to remove > > > _suspend_flag. > > > > > > > > > > > I think we should no longer wait, but proceed with > > > > this change. We will look into removing the usage of > > > > suspend_flag introduced here once it is possible to implement > > > > it with handshakes. > > > > > > Yes, sure. > > > > > > >> Full: > > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.4/ > > > > > > DeoptimizeObjectsALotThread is only used in compileBroker.cpp. > > > You can move both declaration and definition to that file, no need to > > clobber > > > thread.[c|h]pp. (and the static function deopt_objs_alot_thread_entry) > > > > > > Does JvmtiDeferredUpdates really need to be in thread.hpp, can't be in > it's > > > own > > > hpp file? It doesn't seem right to add JVM TI classes into thread.hpp. > > > > > > Note that we also think we may have a bug in deopt: > > > https://bugs.openjdk.java.net/browse/JDK-8238237 > > > > > > I think it would be best, if possible, to push after that is resolved. > > > > > > Not even nearly a full review :) > > > > > > Thanks, Robbin > > > > > > > > > >> Incremental: > > > >> > > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.4.inc/ > > > >> > > > >> I was not able to eliminate the additional suspend flag now. I'll take > care > > > of this > > > >> as soon as the > > > >> existing suspend-resume-mechanism is reworked. > > > >> > > > >> Testing: > > > >> > > > >> Nightly tests @SAP: > > > >> > > > >> JCK and JTREG, also in Xcomp mode, SPECjvm2008, SPECjbb2015, > > > Renaissance > > > >> Suite, SAP specific tests > > > >> with fastdebug and release builds on all platforms > > > >> > > > >> Stress testing with DeoptimizeObjectsALot running SPECjvm2008 40x > > > parallel > > > >> for 24h > > > >> > > > >> Thanks, Richard. > > > >> > > > >> > > > >> More details on the changes: > > > >> > > > >> * Hide DeoptimizeObjectsALotThread from external view. > > > >> > > > >> * Changed EscapeBarrier_lock to be a _safepoint_check_never lock. > > > >> It used to be _safepoint_check_sometimes, which will be eliminated > > > sooner or > > > >> later. > > > >> I added explicit thread state changes with ThreadBlockInVM to code > > > paths > > > >> where we can wait() > > > >> on EscapeBarrier_lock to become safepoint safe. > > > >> > > > >> * Use handshake EscapeBarrierSuspendHandshake to suspend target > > > threads > > > >> instead of vm operation > > > >> VM_ThreadSuspendAllForObjDeopt. > > > >> > > > >> * Removed uses of Threads_lock. When adding a new thread we > > suspend > > > it iff > > > >> EA optimizations are > > > >> being reverted. In the previous version we were waiting on > > > Threads_lock > > > >> while EA optimizations > > > >> were reverted. See EscapeBarrier::thread_added(). > > > >> > > > >> * Made tests require Xmixed compilation mode. > > > >> > > > >> * Made tests agnostic regarding tiered compilation. > > > >> I.e. tc isn't disabled anymore, and the tests can be run with tc > enabled > > or > > > >> disabled. > > > >> > > > >> * Exercising EATests.java as well with stress test options > > > >> DeoptimizeObjectsALot* > > > >> Due to the non-deterministic deoptimizations some tests need to be > > > skipped. > > > >> We do this to prevent bit-rot of the stress test code. > > > >> > > > >> * Executing EATests.java as well with graal if available. Driver for this is > > > >> EATestsJVMCI.java. Graal cannot pass all tests, because it does not > > > provide all > > > >> the new debug info > > > >> (namely not_global_escape_in_scope and arg_escape in > > > scopeDesc.hpp). > > > >> And graal does not yet support the JVMTI operations force early > > return > > > and > > > >> pop frame. > > > >> > > > >> * Removed tracing from new jdi tests in EATests.java. Too much trace > > > output > > > >> before the debugging > > > >> connection is established can cause deadlock because output buffers > > fill > > > up. > > > >> (See https://bugs.openjdk.java.net/browse/JDK-8173304) > > > >> > > > >> * Many copyright year changes and smaller clean-up changes of > testing > > > code > > > >> (trailing white-space and > > > >> the like). > > > >> > > > >> > > > >> -----Original Message----- > > > >> From: David Holmes > > > >> Sent: Donnerstag, 19. Dezember 2019 03:12 > > > >> To: Reingruber, Richard ; serviceability- > > > >> dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; > > > hotspot- > > > >> runtime-dev at openjdk.java.net; Vladimir Kozlov > > > (vladimir.kozlov at oracle.com) > > > >> > > > >> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better > > > Performance in > > > >> the Presence of JVMTI Agents > > > >> > > > >> Hi Richard, > > > >> > > > >> I think my issue is with the way EliminateNestedLocks works so I'm > going > > > >> to look into that more deeply. > > > >> > > > >> Thanks for the explanations. > > > >> > > > >> David > > > >> > > > >> On 18/12/2019 12:47 am, Reingruber, Richard wrote: > > > >>> Hi David, > > > >>> > > > >>> > > > Some further queries/concerns: > > > >>> > > > > > > >>> > > > src/hotspot/share/runtime/objectMonitor.cpp > > > >>> > > > > > > >>> > > > Can you please explain the changes to ObjectMonitor::wait: > > > >>> > > > > > > >>> > > > ! _recursions = save // restore the old recursion count > > > >>> > > > ! + jt->get_and_reset_relock_count_after_wait(); // > > > >>> > > > increased by the deferred relock count > > > >>> > > > > > > >>> > > > what is the "deferred relock count"? I gather it relates to > > > >>> > > > > > > >>> > > > "The code was extended to be able to deoptimize objects of > a > > > >>> > > frame that > > > >>> > > > is not the top frame and to let another thread than the > > owning > > > >>> > > thread do > > > >>> > > > it." > > > >>> > > > > > >>> > > Yes, these relate. Currently EA based optimizations are reverted, > > > when a > > > >> compiled frame is > > > >>> > > replaced with corresponding interpreter frames. Part of this is > > > relocking > > > >> objects with eliminated > > > >>> > > locking. New with the enhancement is that we do this also just > > > before > > > >> object references are > > > >>> > > acquired through JVMTI. In this case we deoptimize also the > > > owning > > > >> compiled frame C and we > > > >>> > > register deoptimized objects as deferred updates. When control > > > returns > > > >> to C it gets deoptimized, > > > >>> > > we notice that objects are already deoptimized (reallocated and > > > >> relocked), so we don't do it again > > > >>> > > (relocking twice would be incorrect of course). Deferred > updates > > > are > > > >> copied into the new > > > >>> > > interpreter frames. > > > >>> > > > > > >>> > > Problem: relocking is not possible if the target thread T is > waiting > > > on the > > > >> monitor that needs to > > > >>> > > be relocked. This happens only with non-local objects with > > > >> EliminateNestedLocks. Instead relocking > > > >>> > > is deferred until T owns the monitor again. This is what the > piece > > of > > > >> code above does. > > > >>> > > > > >>> > Sorry I need some more detail here. How can you wait() on an > > > object > > > >>> > monitor if the object allocation and/or locking was optimised > > away? > > > And > > > >>> > what is a "non-local object" in this context? Isn't EA restricted to > > > >>> > thread-confined objects? > > > >>> > > > >>> "Non-local object" is an object that escapes its thread. The issue I'm > > > >> addressing with the changes > > > >>> in ObjectMonitor::wait are almost unrelated to EA. They are caused > by > > > >> EliminateNestedLocks, where C2 > > > >>> eliminates recursive locking of an already owned lock. The lock > owning > > > object > > > >> exists on the heap, it > > > >>> is locked and you can call wait() on it. > > > >>> > > > >>> EliminateLocks is the C2 option that controls lock elimination based > on > > > EA. > > > >> Both optimizations have > > > >>> in common that objects with eliminated locking need to be relocked > > > when > > > >> deoptimizing a frame, > > > >>> i.e. when replacing a compiled frame with equivalent interpreter > > > >>> frames. Deoptimization::relock_objects does that job for /all/ > > eliminated > > > >> locks in scope. /All/ can > > > >>> be a mix of eliminated nested locks and locks of not-escaping objects. > > > >>> > > > >>> New with the enhancement: I call relock_objects earlier, just before > > > objects > > > >> pontentially > > > >>> escape. But then later when the owning compiled frame gets > > > deoptimized, I > > > >> must not do it again: > > > >>> > > > >>> See call to EscapeBarrier::objs_are_deoptimized in > > deoptimization.cpp: > > > >>> > > > >>> 373 if ((jvmci_enabled || ((DoEscapeAnalysis || > > > EliminateNestedLocks) && > > > >> EliminateLocks)) > > > >>> 374 && !EscapeBarrier::objs_are_deoptimized(thread, > > > deoptee.id())) { > > > >>> 375 bool unused; > > > >>> 376 eliminate_locks(thread, chunk, realloc_failures, deoptee, > > > exec_mode, > > > >> unused); > > > >>> 377 } > > > >>> > > > >>> Now when calling relock_objects early it is quiet possible that I have > to > > > relock > > > >> an object the > > > >>> target thread currently waits for. Obviously I cannot relock in this > case, > > > >> instead I chose to > > > >>> introduce relock_count_after_wait to JavaThread. > > > >>> > > > >>> > Is it just that some of the locking gets optimized away e.g. > > > >>> > > > > >>> > synchronised(obj) { > > > >>> > synchronised(obj) { > > > >>> > synchronised(obj) { > > > >>> > obj.wait(); > > > >>> > } > > > >>> > } > > > >>> > } > > > >>> > > > > >>> > If this is reduced to a form as-if it were a single lock of the > monitor > > > >>> > (due to EA) and the wait() triggers a JVM TI event which leads to > > the > > > >>> > escape of "obj" then we need to reconstruct the true lock state, > > and > > > so > > > >>> > when the wait() internally unblocks and reacquires the monitor it > > > has to > > > >>> > set the true recursion count to 3, not the 1 that it appeared to be > > > when > > > >>> > wait() was initially called. Is that the scenario? > > > >>> > > > >>> Kind of... except that the locking is not eliminated due to EA and > there > > is > > > no > > > >> JVM TI event > > > >>> triggered by wait. > > > >>> > > > >>> Add > > > >>> > > > >>> LocalObject l1 = new LocalObject(); > > > >>> > > > >>> in front of the synchrnized blocks and assume a JVM TI agent > acquires > > l1. > > > This > > > >> triggers the code in > > > >>> question. > > > >>> > > > >>> See that relocking/reallocating is transactional. If it is done then for > > /all/ > > > >> objects in scope and it is > > > >>> done at most once. It wouldn't be quite so easy to split this in > relocking > > > of > > > >> nested/EA-based > > > >>> eliminated locks. > > > >>> > > > >>> > If so I find this truly awful. Anyone using wait() in a realistic form > > > >>> > requires a notification and so the object cannot be thread > > confined. > > > In > > > >>> > > > >>> It is not thread confined. > > > >>> > > > >>> > which case I would strongly argue that upon hitting the wait() the > > > deopt > > > >>> > should occur unconditionally and so the lock state is correct > before > > > we > > > >>> > wait and so we don't need to mess with the recursion count > > > internally > > > >>> > when we reacquire the monitor. > > > >>> > > > > >>> > > > > > >>> > > > which I don't like the sound of at all when it comes to > > > ObjectMonitor > > > >>> > > > state. So I'd like to understand in detail exactly what is going > > on > > > here > > > >>> > > > and why. This is a very intrusive change that seems to badly > > > break > > > >>> > > > encapsulation and impacts future changes to ObjectMonitor > > > that are > > > >> under > > > >>> > > > investigation. > > > >>> > > > > > >>> > > I would not regard this as breaking encapsulation. Certainly not > > > badly. > > > >>> > > > > > >>> > > I've added a property relock_count_after_wait to JavaThread. > > The > > > >> property is well > > > >>> > > encapsulated. Future ObjectMonitor implementations have to > > deal > > > with > > > >> recursion too. They are free > > > >>> > > in choosing a way to do that as long as that property is taken > into > > > >> account. This is hardly a > > > >>> > > limitation. > > > >>> > > > > >>> > I do think this badly breaks encapsulation as you have to add a > > > callout > > > >>> > from the guts of the ObjectMonitor code to reach into the thread > > to > > > get > > > >>> > this lock count adjustment. I understand why you have had to do > > > this but > > > >>> > I would much rather see a change to the EA optimisation strategy > > so > > > that > > > >>> > this is not needed. > > > >>> > > > > >>> > > Note also that the property is a straight forward extension of > the > > > >> existing concept of deferred > > > >>> > > local updates. It is embedded into the structure holding them. > So > > > not > > > >> even the footprint of a > > > >>> > > JavaThread is enlarged if no deferred updates are generated. > > > >>> > > > > >>> > [...] > > > >>> > > > > >>> > > > > > >>> > > I'm actually duplicating the existing external suspend > mechanism, > > > >> because a thread can be > > > >>> > > suspended at most once. And hey, and don't like that either! > But > > it > > > >> seems not unlikely that the > > > >>> > > duplicate can be removed together with the original and the > new > > > type > > > >> of handshakes that will be > > > >>> > > used for thread suspend can be used for object deoptimization > > > too. See > > > >> today's discussion in > > > >>> > > JDK-8227745 [2]. > > > >>> > > > > >>> > I hope that discussion bears some fruit, at the moment it seems > > not > > > to > > > >>> > be possible to use handshakes here. :( > > > >>> > > > > >>> > The external suspend mechanism is a royal pain in the proverbial > > > that we > > > >>> > have to carefully live with. The idea that we're duplicating that > for > > > >>> > use in another fringe area of functionality does not thrill me at all. > > > >>> > > > > >>> > To be clear, I understand the problem that exists and that you > > wish > > > to > > > >>> > solve, but for the runtime parts I balk at the complexity cost of > > > >>> > solving it. > > > >>> > > > >>> I know it's complex, but by far no rocket science. > > > >>> > > > >>> Also I find it hard to imagine another fix for JDK-8233915 besides > > > changing > > > >> the JVM TI specification. > > > >>> > > > >>> Thanks, Richard. > > > >>> > > > >>> -----Original Message----- > > > >>> From: David Holmes > > > >>> Sent: Dienstag, 17. Dezember 2019 08:03 > > > >>> To: Reingruber, Richard ; > serviceability- > > > >> dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; > > > hotspot- > > > >> runtime-dev at openjdk.java.net; Vladimir Kozlov > > > (vladimir.kozlov at oracle.com) > > > >> > > > >>> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better > > > Performance > > > >> in the Presence of JVMTI Agents > > > >>> > > > >>> > > > >>> > > > >>> David > > > >>> > > > >>> On 17/12/2019 4:57 pm, David Holmes wrote: > > > >>>> Hi Richard, > > > >>>> > > > >>>> On 14/12/2019 5:01 am, Reingruber, Richard wrote: > > > >>>>> Hi David, > > > >>>>> > > > >>>>> ?? > Some further queries/concerns: > > > >>>>> ?? > > > > >>>>> ?? > src/hotspot/share/runtime/objectMonitor.cpp > > > >>>>> ?? > > > > >>>>> ?? > Can you please explain the changes to ObjectMonitor::wait: > > > >>>>> ?? > > > > >>>>> ?? > !?? _recursions = save????? // restore the old recursion count > > > >>>>> ?? > !???????????????? + jt->get_and_reset_relock_count_after_wait(); // > > > >>>>> ?? > increased by the deferred relock count > > > >>>>> ?? > > > > >>>>> ?? > what is the "deferred relock count"? I gather it relates to > > > >>>>> ?? > > > > >>>>> ?? > "The code was extended to be able to deoptimize objects of a > > > >>>>> frame that > > > >>>>> ?? > is not the top frame and to let another thread than the owning > > > >>>>> thread do > > > >>>>> ?? > it." > > > >>>>> > > > >>>>> Yes, these relate. Currently EA based optimizations are reverted, > > > when > > > >>>>> a compiled frame is replaced > > > >>>>> with corresponding interpreter frames. Part of this is relocking > > > >>>>> objects with eliminated > > > >>>>> locking. New with the enhancement is that we do this also just > > before > > > >>>>> object references are acquired > > > >>>>> through JVMTI. In this case we deoptimize also the owning > compiled > > > >>>>> frame C and we register > > > >>>>> deoptimized objects as deferred updates. When control returns to > > C > > > it > > > >>>>> gets deoptimized, we notice > > > >>>>> that objects are already deoptimized (reallocated and relocked), so > > > we > > > >>>>> don't do it again (relocking > > > >>>>> twice would be incorrect of course). Deferred updates are copied > > into > > > >>>>> the new interpreter frames. > > > >>>>> > > > >>>>> Problem: relocking is not possible if the target thread T is waiting > > > >>>>> on the monitor that needs to be > > > >>>>> relocked. This happens only with non-local objects with > > > >>>>> EliminateNestedLocks. Instead relocking is > > > >>>>> deferred until T owns the monitor again. This is what the piece of > > > >>>>> code above does. > > > >>>> > > > >>>> Sorry I need some more detail here. How can you wait() on an > object > > > >>>> monitor if the object allocation and/or locking was optimised away? > > > And > > > >>>> what is a "non-local object" in this context? Isn't EA restricted to > > > >>>> thread-confined objects? > > > >>>> > > > >>>> Is it just that some of the locking gets optimized away e.g. > > > >>>> > > > >>>> synchronised(obj) { > > > >>>> ? synchronised(obj) { > > > >>>> ??? synchronised(obj) { > > > >>>> ????? obj.wait(); > > > >>>> ??? } > > > >>>> ? } > > > >>>> } > > > >>>> > > > >>>> If this is reduced to a form as-if it were a single lock of the monitor > > > >>>> (due to EA) and the wait() triggers a JVM TI event which leads to the > > > >>>> escape of "obj" then we need to reconstruct the true lock state, and > > so > > > >>>> when the wait() internally unblocks and reacquires the monitor it > has > > to > > > >>>> set the true recursion count to 3, not the 1 that it appeared to be > > when > > > >>>> wait() was initially called. Is that the scenario? > > > >>>> > > > >>>> If so I find this truly awful. Anyone using wait() in a realistic form > > > >>>> requires a notification and so the object cannot be thread confined. > > In > > > >>>> which case I would strongly argue that upon hitting the wait() the > > > deopt > > > >>>> should occur unconditionally and so the lock state is correct before > > we > > > >>>> wait and so we don't need to mess with the recursion count > internally > > > >>>> when we reacquire the monitor. > > > >>>> > > > >>>>> > > > >>>>> ?? > which I don't like the sound of at all when it comes to > > > >>>>> ObjectMonitor > > > >>>>> ?? > state. So I'd like to understand in detail exactly what is going > > > >>>>> on here > > > >>>>> ?? > and why.? This is a very intrusive change that seems to badly > > > break > > > >>>>> ?? > encapsulation and impacts future changes to ObjectMonitor > > that > > > >>>>> are under > > > >>>>> ?? > investigation. > > > >>>>> > > > >>>>> I would not regard this as breaking encapsulation. Certainly not > > badly. > > > >>>>> > > > >>>>> I've added a property relock_count_after_wait to JavaThread. The > > > >>>>> property is well > > > >>>>> encapsulated. Future ObjectMonitor implementations have to deal > > > with > > > >>>>> recursion too. They are free in > > > >>>>> choosing a way to do that as long as that property is taken into > > > >>>>> account. This is hardly a > > > >>>>> limitation. > > > >>>> > > > >>>> I do think this badly breaks encapsulation as you have to add a > callout > > > >>>> from the guts of the ObjectMonitor code to reach into the thread to > > > get > > > >>>> this lock count adjustment. I understand why you have had to do > this > > > but > > > >>>> I would much rather see a change to the EA optimisation strategy so > > > that > > > >>>> this is not needed. > > > >>>> > > > >>>>> Note also that the property is a straight forward extension of the > > > >>>>> existing concept of deferred > > > >>>>> local updates. It is embedded into the structure holding them. So > > not > > > >>>>> even the footprint of a > > > >>>>> JavaThread is enlarged if no deferred updates are generated. > > > >>>>> > > > >>>>> ?? > --- > > > >>>>> ?? > > > > >>>>> ?? > src/hotspot/share/runtime/thread.cpp > > > >>>>> ?? > > > > >>>>> ?? > Can you please explain why > > > >>>>> JavaThread::wait_for_object_deoptimization > > > >>>>> ?? > has to be handcrafted in this way rather than using proper > > > >>>>> transitions. > > > >>>>> ?? > > > > >>>>> > > > >>>>> I wrote wait_for_object_deoptimization taking > > > >>>>> JavaThread::java_suspend_self_with_safepoint_check > > > >>>>> as template. So in short: for the same reasons :) > > > >>>>> > > > >>>>> Threads reach both methods as part of thread state transitions, > > > >>>>> therefore special handling is > > > >>>>> required to change thread state on top of ongoing transitions. > > > >>>>> > > > >>>>> ?? > We got rid of "deopt suspend" some time ago and it is > > disturbing > > > >>>>> to see > > > >>>>> ?? > it being added back (effectively). This seems like it may be > > > >>>>> something > > > >>>>> ?? > that handshakes could be used for. > > > >>>>> > > > >>>>> Deopt suspend used to be something rather different with a > similar > > > >>>>> name[1]. It is not being added back. > > > >>>> > > > >>>> I stand corrected. Despite comments in the code to the contrary > > > >>>> deopt_suspend didn't actually cause a self-suspend. I was doing a > lot > > of > > > >>>> cleanup in this area 13 years ago :) > > > >>>> > > > >>>>> > > > >>>>> I'm actually duplicating the existing external suspend mechanism, > > > >>>>> because a thread can be suspended > > > >>>>> at most once. And hey, and don't like that either! But it seems not > > > >>>>> unlikely that the duplicate can > > > >>>>> be removed together with the original and the new type of > > > handshakes > > > >>>>> that will be used for > > > >>>>> thread suspend can be used for object deoptimization too. See > > > today's > > > >>>>> discussion in JDK-8227745 [2]. > > > >>>> > > > >>>> I hope that discussion bears some fruit, at the moment it seems not > > to > > > >>>> be possible to use handshakes here. :( > > > >>>> > > > >>>> The external suspend mechanism is a royal pain in the proverbial > that > > > we > > > >>>> have to carefully live with. The idea that we're duplicating that for > > > >>>> use in another fringe area of functionality does not thrill me at all. > > > >>>> > > > >>>> To be clear, I understand the problem that exists and that you wish > to > > > >>>> solve, but for the runtime parts I balk at the complexity cost of > > > >>>> solving it. > > > >>>> > > > >>>> Thanks, > > > >>>> David > > > >>>> ----- > > > >>>> > > > >>>>> Thanks, Richard. > > > >>>>> > > > >>>>> [1] Deopt suspend was something like an async. handshake for > > > >>>>> architectures with register windows, > > > >>>>> ???? where patching the return pc for deoptimization of a compiled > > > >>>>> frame was racy if the owner thread > > > >>>>> ???? was in native code. Instead a "deopt" suspend flag was set on > > > >>>>> which the thread patched its own > > > >>>>> ???? frame upon return from native. So no thread was suspended. > It > > > got > > > >>>>> its name only from the name of > > > >>>>> ???? the flags. > > > >>>>> > > > >>>>> [2] Discussion about using handshakes to sync. with the target > > thread: > > > >>>>> > > > >>>>> https://bugs.openjdk.java.net/browse/JDK- > > > >> > > > > > > 8227745?focusedCommentId=14306727&page=com.atlassian.jira.plugin.syst > > > e > > > >> m.issuetabpanels:comment-tabpanel#comment-14306727 > > > >>>>> > > > >>>>> > > > >>>>> -----Original Message----- > > > >>>>> From: David Holmes > > > >>>>> Sent: Freitag, 13. Dezember 2019 00:56 > > > >>>>> To: Reingruber, Richard ; > > > >>>>> serviceability-dev at openjdk.java.net; > > > >>>>> hotspot-compiler-dev at openjdk.java.net; > > > >>>>> hotspot-runtime-dev at openjdk.java.net > > > >>>>> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better > > > >>>>> Performance in the Presence of JVMTI Agents > > > >>>>> > > > >>>>> Hi Richard, > > > >>>>> > > > >>>>> Some further queries/concerns: > > > >>>>> > > > >>>>> src/hotspot/share/runtime/objectMonitor.cpp > > > >>>>> > > > >>>>> Can you please explain the changes to ObjectMonitor::wait: > > > >>>>> > > > >>>>> !?? _recursions = save????? // restore the old recursion count > > > >>>>> !???????????????? + jt->get_and_reset_relock_count_after_wait(); // > > > >>>>> increased by the deferred relock count > > > >>>>> > > > >>>>> what is the "deferred relock count"? I gather it relates to > > > >>>>> > > > >>>>> "The code was extended to be able to deoptimize objects of a > > frame > > > that > > > >>>>> is not the top frame and to let another thread than the owning > > thread > > > do > > > >>>>> it." > > > >>>>> > > > >>>>> which I don't like the sound of at all when it comes to > ObjectMonitor > > > >>>>> state. So I'd like to understand in detail exactly what is going on > here > > > >>>>> and why.? This is a very intrusive change that seems to badly break > > > >>>>> encapsulation and impacts future changes to ObjectMonitor that > > are > > > under > > > >>>>> investigation. > > > >>>>> > > > >>>>> --- > > > >>>>> > > > >>>>> src/hotspot/share/runtime/thread.cpp > > > >>>>> > > > >>>>> Can you please explain why > > > JavaThread::wait_for_object_deoptimization > > > >>>>> has to be handcrafted in this way rather than using proper > > transitions. > > > >>>>> > > > >>>>> We got rid of "deopt suspend" some time ago and it is disturbing > to > > > see > > > >>>>> it being added back (effectively). This seems like it may be > > something > > > >>>>> that handshakes could be used for. > > > >>>>> > > > >>>>> Thanks, > > > >>>>> David > > > >>>>> ----- > > > >>>>> > > > >>>>> On 12/12/2019 7:02 am, David Holmes wrote: > > > >>>>>> On 12/12/2019 1:07 am, Reingruber, Richard wrote: > > > >>>>>>> Hi David, > > > >>>>>>> > > > >>>>>>> ??? > Most of the details here are in areas I can comment on in > > > detail, > > > >>>>>>> but I > > > >>>>>>> ??? > did take an initial general look at things. > > > >>>>>>> > > > >>>>>>> Thanks for taking the time! > > > >>>>>> > > > >>>>>> Apologies the above should read: > > > >>>>>> > > > >>>>>> "Most of the details here are in areas I *can't* comment on in > > detail > > > >>>>>> ..." > > > >>>>>> > > > >>>>>> David > > > >>>>>> > > > >>>>>>> ??? > The only thing that jumped out at me is that I think the > > > >>>>>>> ??? > DeoptimizeObjectsALotThread should be a hidden thread. > > > >>>>>>> ??? > > > > >>>>>>> ??? > +? bool is_hidden_from_external_view() const { return true; > > } > > > >>>>>>> > > > >>>>>>> Yes, it should. Will add the method like above. > > > >>>>>>> > > > >>>>>>> ??? > Also I don't see any testing of the > > > DeoptimizeObjectsALotThread. > > > >>>>>>> Without > > > >>>>>>> ??? > active testing this will just bit-rot. > > > >>>>>>> > > > >>>>>>> DeoptimizeObjectsALot is meant for stress testing with a larger > > > >>>>>>> workload. I will add a minimal test > > > >>>>>>> to keep it fresh. > > > >>>>>>> > > > >>>>>>> ??? > Also on the tests I don't understand your @requires clause: > > > >>>>>>> ??? > > > > >>>>>>> ??? >?? @requires ((vm.compMode != "Xcomp") & > > > vm.compiler2.enabled > > > >> & > > > >>>>>>> ??? > (vm.opt.TieredCompilation != true)) > > > >>>>>>> ??? > > > > >>>>>>> ??? > This seems to require that TieredCompilation is disabled, > but > > > >>>>>>> tiered is > > > >>>>>>> ??? > our normal mode of operation. ?? > > > >>>>>>> ??? > > > > >>>>>>> > > > >>>>>>> I removed the clause. I guess I wanted to target the tests > towards > > > the > > > >>>>>>> code they are supposed to > > > >>>>>>> test, and it's easier to analyze failures w/o tiered compilation > and > > > >>>>>>> with just one compiler thread. > > > >>>>>>> > > > >>>>>>> Additionally I will make use of > > > >>>>>>> compiler.whitebox.CompilerWhiteBoxTest.THRESHOLD in the > > tests. > > > >>>>>>> > > > >>>>>>> Thanks, > > > >>>>>>> Richard. > > > >>>>>>> > > > >>>>>>> -----Original Message----- > > > >>>>>>> From: David Holmes > > > >>>>>>> Sent: Mittwoch, 11. Dezember 2019 08:03 > > > >>>>>>> To: Reingruber, Richard ; > > > >>>>>>> serviceability-dev at openjdk.java.net; > > > >>>>>>> hotspot-compiler-dev at openjdk.java.net; > > > >>>>>>> hotspot-runtime-dev at openjdk.java.net > > > >>>>>>> Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better > > > >>>>>>> Performance in the Presence of JVMTI Agents > > > >>>>>>> > > > >>>>>>> Hi Richard, > > > >>>>>>> > > > >>>>>>> On 11/12/2019 7:45 am, Reingruber, Richard wrote: > > > >>>>>>>> Hi, > > > >>>>>>>> > > > >>>>>>>> I would like to get reviews please for > > > >>>>>>>> > > > >>>>>>>> > > > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.3/ > > > >>>>>>>> > > > >>>>>>>> Corresponding RFE: > > > >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8227745 > > > >>>>>>>> > > > >>>>>>>> Fixes also https://bugs.openjdk.java.net/browse/JDK-8233915 > > > >>>>>>>> And potentially https://bugs.openjdk.java.net/browse/JDK- > > > 8214584 [1] > > > >>>>>>>> > > > >>>>>>>> Vladimir Kozlov kindly put webrev.3 through tier1-8 testing > > > without > > > >>>>>>>> issues (thanks!). In addition the > > > >>>>>>>> change is being tested at SAP since I posted the first RFR some > > > >>>>>>>> months ago. > > > >>>>>>>> > > > >>>>>>>> The intention of this enhancement is to benefit performance > > wise > > > from > > > >>>>>>>> escape analysis even if JVMTI > > > >>>>>>>> agents request capabilities that allow them to access local > > variable > > > >>>>>>>> values. E.g. if you start-up > > > >>>>>>>> with -agentlib:jdwp=transport=dt_socket,server=y,suspend=n, > > > then > > > >>>>>>>> escape analysis is disabled right > > > >>>>>>>> from the beginning, well before a debugger attaches -- if ever > > one > > > >>>>>>>> should do so. With the > > > >>>>>>>> enhancement, escape analysis will remain enabled until and > > after > > > a > > > >>>>>>>> debugger attaches. EA based > > > >>>>>>>> optimizations are reverted just before an agent acquires the > > > >>>>>>>> reference to an object. In the JBS item > > > >>>>>>>> you'll find more details. > > > >>>>>>> > > > >>>>>>> Most of the details here are in areas I can comment on in detail, > > but > > > I > > > >>>>>>> did take an initial general look at things. > > > >>>>>>> > > > >>>>>>> The only thing that jumped out at me is that I think the > > > >>>>>>> DeoptimizeObjectsALotThread should be a hidden thread. > > > >>>>>>> > > > >>>>>>> +? bool is_hidden_from_external_view() const { return true; } > > > >>>>>>> > > > >>>>>>> Also I don't see any testing of the DeoptimizeObjectsALotThread. > > > >>>>>>> Without > > > >>>>>>> active testing this will just bit-rot. > > > >>>>>>> > > > >>>>>>> Also on the tests I don't understand your @requires clause: > > > >>>>>>> > > > >>>>>>> ??? @requires ((vm.compMode != "Xcomp") & > > > vm.compiler2.enabled & > > > >>>>>>> (vm.opt.TieredCompilation != true)) > > > >>>>>>> > > > >>>>>>> This seems to require that TieredCompilation is disabled, but > > tiered > > > is > > > >>>>>>> our normal mode of operation. ?? > > > >>>>>>> > > > >>>>>>> Thanks, > > > >>>>>>> David > > > >>>>>>> > > > >>>>>>>> Thanks, > > > >>>>>>>> Richard. > > > >>>>>>>> > > > >>>>>>>> [1] Experimental fix for JDK-8214584 based on JDK-8227745 > > > >>>>>>>> > > > >> > > > > > > http://cr.openjdk.java.net/~rrich/webrevs/2019/8214584/experiment_v1.pa > > > tc > > > >> h > > > >>>>>>>> > > > >>>>>>>> > > > >>>>>>>> From Alan.Bateman at oracle.com Mon Jul 13 13:05:38 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 13 Jul 2020 14:05:38 +0100 Subject: Some clarification about JVMTI_EVENT_THREAD_START and PRIMORDIAL phase In-Reply-To: <6287d398-5828-762d-3609-2767b684186d@oracle.com> References: <6287d398-5828-762d-3609-2767b684186d@oracle.com> Message-ID: <14e0fcf3-5dfe-9815-3eb8-cda7fa03bc8e@oracle.com> On 09/07/2020 17:25, Daniel D. Daugherty wrote: > : > > However, the above diffs prevent the JVMTI_EVENT_THREAD_START > and JVMTI_EVENT_THREAD_END events from being posted earlier than > the JVMTI_PHASE_START phase... Unfortunately, I can't find the > actual changeset pushed to the Jigsaw report for JDK-8072745. The intermediate history is in the jigsaw/jake repo [1] if we need it. I don't have time to dig into it now but just to say that the THREAD_START/THREAD_END events are specified to be notified in start + live phases. It would be a bug (as in spec/impl mismatch) if we are posting these events in the primordial phase. -Alan [1] http://hg.openjdk.java.net/jigsaw/jake/hotspot From daniel.daugherty at oracle.com Mon Jul 13 14:00:09 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 13 Jul 2020 10:00:09 -0400 Subject: Some clarification about JVMTI_EVENT_THREAD_START and PRIMORDIAL phase In-Reply-To: <529E99DA-4DFA-4C13-BCC1-A47C82521061@azul.com> References: <6287d398-5828-762d-3609-2767b684186d@oracle.com> <529E99DA-4DFA-4C13-BCC1-A47C82521061@azul.com> Message-ID: If there is going to be a spec change, then it would not affect JDK8. I don't think anyone has decided how this issue should be resolved yet. Dan On 7/13/20 7:10 AM, Sergey Nazarkin wrote: > Hi Daniel, > > Please correct me if I?m wrong but the spec change will touch post-JDK8 releases only? > > > > Sergey Nazarkin > > > > >> On Jul 9, 2020, at 19:25, Daniel D. Daugherty wrote: >> >> On 7/9/20 9:07 AM, Jaroslav Bachor?k wrote: >>> Hello, >>> >>> Recently, after backporting JDK-8233197 to JDK8u I received a report >>> from Sergey Nazarkin (cc) that the backport broke JVMTI spec basically >>> emitting JVMTI_EVENT_THREAD_START events in primordial phase while the >>> spec states that it should be emitted only in either start or live >>> phase. >>> >>> But if I am reading this correctly >>> (https://hg.openjdk.java.net/jdk8u/jdk8udev/hotspot/file/02b4fd2f9041/src/share/vm/prims/jvmtiEventController.cpp#l418) >>> the spec is also violated there. >>> >>> EARLY_EVENT_BITS are including THREAD_START_BIT and it is not filtered >>> out by THREAD_FILTERED_EVENT_BITS so it seems perfectly valid for >>> JVMTI_EVENT_THREAD_START to appear during PRIMORDIAL phase. >>> >>> So I would like to get clarification before going in and start >>> tinkering with the JVM initialization code order. >>> >>> Thanks! >>> >>> -JB- >> Hi Jaroslav! >> >> It has been a long time my friend... :-) >> >> It looks like that change was made by this Jigsaw change: >> >> JDK-8072745 Re-examine JVM TI execution phases and expectations >> on what is allowed in each phase >> https://bugs.openjdk.java.net/browse/JDK-8072745 >> >> These diffs in the bug reported jumped out at me: >> >> @@ -1022,6 +1032,10 @@ >> void JvmtiExport::post_thread_start(JavaThread *thread) { >> assert(thread->thread_state() == _thread_in_vm, "must be in vm state"); >> >> + if (JvmtiEnv::get_phase() <= JVMTI_PHASE_PRIMORDIAL) { >> + return; >> + } >> + >> EVT_TRIG_TRACE(JVMTI_EVENT_THREAD_START, ("JVMTI [%s] Trg Thread Start event triggered", >> JvmtiTrace::safe_get_thread_name(thread))); >> >> @@ -1050,6 +1064,10 @@ >> >> >> void JvmtiExport::post_thread_end(JavaThread *thread) { >> + if (JvmtiEnv::get_phase() <= JVMTI_PHASE_PRIMORDIAL) { >> + return; >> + } >> + >> EVT_TRIG_TRACE(JVMTI_EVENT_THREAD_END, ("JVMTI [%s] Trg Thread End event triggered", >> JvmtiTrace::safe_get_thread_name(thread))); >> >> >> Here's the order of phases from jvmti.h: >> >> /* Phases of execution */ >> >> typedef enum { >> JVMTI_PHASE_ONLOAD = 1, >> JVMTI_PHASE_PRIMORDIAL = 2, >> JVMTI_PHASE_START = 6, >> JVMTI_PHASE_LIVE = 4, >> JVMTI_PHASE_DEAD = 8 >> } jvmtiPhase; >> >> However, the above diffs prevent the JVMTI_EVENT_THREAD_START >> and JVMTI_EVENT_THREAD_END events from being posted earlier than >> the JVMTI_PHASE_START phase... Unfortunately, I can't find the >> actual changeset pushed to the Jigsaw report for JDK-8072745. >> >> >> When you look at the current JvmtiExport::post_thread_start() >> and JvmtiExport::post_thread_end() functions, you can see this: >> >> 1: void JvmtiExport::post_thread_start(JavaThread *thread) { >> 36508: if (JvmtiEnv::get_phase() < JVMTI_PHASE_PRIMORDIAL) { >> 36508: return; >> 36508: } >> >> 1: void JvmtiExport::post_thread_end(JavaThread *thread) { >> 36508: if (JvmtiEnv::get_phase() < JVMTI_PHASE_PRIMORDIAL) { >> 36508: return; >> 36508: } >> >> $ hg log -r 36508 >> changeset: 36508:5f9eee6b383b >> user: alanb >> date: Thu Mar 17 19:04:01 2016 +0000 >> summary: 8142968: Module System implementation >> >> >> So the Jigsaw integration pushed a slightly different change >> than is shown in JDK-8072745. I don't know whether the fix >> for JDK-8072745 evolved before being pushed to the Jigsaw >> repos or whether another JBS issue was used to modify the >> code after JDK-8072745 was pushed. >> >> Short version: The semantics of the JVMTI_EVENT_THREAD_START >> and JVMTI_EVENT_THREAD_END were changed by Jigsaw, but the >> JVM/TI spec was not updated to reflect those changes. >> >> I've filed a new bug to track this issue: >> >> JDK-8249158 THREAD_START and THREAD_END event posting changed >> by modules without updating the spec >> https://bugs.openjdk.java.net/browse/JDK-8249158 >> >> Dan From sa514093 at mail.ustc.edu.cn Mon Jul 13 08:40:31 2020 From: sa514093 at mail.ustc.edu.cn (sa514093 at mail.ustc.edu.cn) Date: Mon, 13 Jul 2020 16:40:31 +0800 (GMT+08:00) Subject: Java 14 jstat -class option man doc typo Message-ID: <4616bae2.607a.17347552ef7.Coremail.sa514093@mail.ustc.edu.cn> Hi all, I recently found a document typo in jstat -class option man doc. This typo maybe caused by automatically generated by Pandoc tool. https://github.com/openjdk/jdk/blob/d34f561de002510c818c73cd5871caf2222ca0a7/src/jdk.jcmd/share/man/jstat.1#L227 Bytes: Number of KB loaded. --> Bytes: Number of KB unloaded. I report issue to Github, and follow Martijn Verburg's advice to report to upstream here. https://github.com/AdoptOpenJDK/openjdk-jdk/issues/3 https://github.com/SAP/SapMachine/issues/687 Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From snazarkin at azul.com Mon Jul 13 11:10:35 2020 From: snazarkin at azul.com (Sergey Nazarkin) Date: Mon, 13 Jul 2020 11:10:35 +0000 Subject: Some clarification about JVMTI_EVENT_THREAD_START and PRIMORDIAL phase In-Reply-To: <6287d398-5828-762d-3609-2767b684186d@oracle.com> References: <6287d398-5828-762d-3609-2767b684186d@oracle.com> Message-ID: <529E99DA-4DFA-4C13-BCC1-A47C82521061@azul.com> Hi Daniel, Please correct me if I?m wrong but the spec change will touch post-JDK8 releases only? Sergey Nazarkin > On Jul 9, 2020, at 19:25, Daniel D. Daugherty wrote: > > On 7/9/20 9:07 AM, Jaroslav Bachor?k wrote: >> Hello, >> >> Recently, after backporting JDK-8233197 to JDK8u I received a report >> from Sergey Nazarkin (cc) that the backport broke JVMTI spec basically >> emitting JVMTI_EVENT_THREAD_START events in primordial phase while the >> spec states that it should be emitted only in either start or live >> phase. >> >> But if I am reading this correctly >> (https://hg.openjdk.java.net/jdk8u/jdk8udev/hotspot/file/02b4fd2f9041/src/share/vm/prims/jvmtiEventController.cpp#l418) >> the spec is also violated there. >> >> EARLY_EVENT_BITS are including THREAD_START_BIT and it is not filtered >> out by THREAD_FILTERED_EVENT_BITS so it seems perfectly valid for >> JVMTI_EVENT_THREAD_START to appear during PRIMORDIAL phase. >> >> So I would like to get clarification before going in and start >> tinkering with the JVM initialization code order. >> >> Thanks! >> >> -JB- > > Hi Jaroslav! > > It has been a long time my friend... :-) > > It looks like that change was made by this Jigsaw change: > > JDK-8072745 Re-examine JVM TI execution phases and expectations > on what is allowed in each phase > https://bugs.openjdk.java.net/browse/JDK-8072745 > > These diffs in the bug reported jumped out at me: > > @@ -1022,6 +1032,10 @@ > void JvmtiExport::post_thread_start(JavaThread *thread) { > assert(thread->thread_state() == _thread_in_vm, "must be in vm state"); > > + if (JvmtiEnv::get_phase() <= JVMTI_PHASE_PRIMORDIAL) { > + return; > + } > + > EVT_TRIG_TRACE(JVMTI_EVENT_THREAD_START, ("JVMTI [%s] Trg Thread Start event triggered", > JvmtiTrace::safe_get_thread_name(thread))); > > @@ -1050,6 +1064,10 @@ > > > void JvmtiExport::post_thread_end(JavaThread *thread) { > + if (JvmtiEnv::get_phase() <= JVMTI_PHASE_PRIMORDIAL) { > + return; > + } > + > EVT_TRIG_TRACE(JVMTI_EVENT_THREAD_END, ("JVMTI [%s] Trg Thread End event triggered", > JvmtiTrace::safe_get_thread_name(thread))); > > > Here's the order of phases from jvmti.h: > > /* Phases of execution */ > > typedef enum { > JVMTI_PHASE_ONLOAD = 1, > JVMTI_PHASE_PRIMORDIAL = 2, > JVMTI_PHASE_START = 6, > JVMTI_PHASE_LIVE = 4, > JVMTI_PHASE_DEAD = 8 > } jvmtiPhase; > > However, the above diffs prevent the JVMTI_EVENT_THREAD_START > and JVMTI_EVENT_THREAD_END events from being posted earlier than > the JVMTI_PHASE_START phase... Unfortunately, I can't find the > actual changeset pushed to the Jigsaw report for JDK-8072745. > > > When you look at the current JvmtiExport::post_thread_start() > and JvmtiExport::post_thread_end() functions, you can see this: > > 1: void JvmtiExport::post_thread_start(JavaThread *thread) { > 36508: if (JvmtiEnv::get_phase() < JVMTI_PHASE_PRIMORDIAL) { > 36508: return; > 36508: } > > 1: void JvmtiExport::post_thread_end(JavaThread *thread) { > 36508: if (JvmtiEnv::get_phase() < JVMTI_PHASE_PRIMORDIAL) { > 36508: return; > 36508: } > > $ hg log -r 36508 > changeset: 36508:5f9eee6b383b > user: alanb > date: Thu Mar 17 19:04:01 2016 +0000 > summary: 8142968: Module System implementation > > > So the Jigsaw integration pushed a slightly different change > than is shown in JDK-8072745. I don't know whether the fix > for JDK-8072745 evolved before being pushed to the Jigsaw > repos or whether another JBS issue was used to modify the > code after JDK-8072745 was pushed. > > Short version: The semantics of the JVMTI_EVENT_THREAD_START > and JVMTI_EVENT_THREAD_END were changed by Jigsaw, but the > JVM/TI spec was not updated to reflect those changes. > > I've filed a new bug to track this issue: > > JDK-8249158 THREAD_START and THREAD_END event posting changed > by modules without updating the spec > https://bugs.openjdk.java.net/browse/JDK-8249158 > > Dan From chris.plummer at oracle.com Mon Jul 13 18:12:50 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 13 Jul 2020 11:12:50 -0700 Subject: RFR(XS): 8248878: SA: Implement simple workaround for JDK-8248876 In-Reply-To: <6d2610d6-5ca1-8455-ddc6-d7f02b21c0d5@oracle.com> References: <39ac40d6-a738-3214-d86c-412152a631d4@oracle.com> <8e89d223-0f65-f26e-3bd7-2ca7421a940b@oracle.com> <6d2610d6-5ca1-8455-ddc6-d7f02b21c0d5@oracle.com> Message-ID: <046aced2-1d23-d096-748b-4ce0bfd01587@oracle.com> Hi Yasumasa, If you have no further suggestions on how to fix JDK-8248876, I'd like to proceed with this work around. Can I considered it reviewed by you? thanks, Chris On 7/7/20 7:29 PM, Chris Plummer wrote: > Hi Yasumasa, > > The executable is not opened with pathmap_open: > > ? if ((ph->core->exec_fd = open(exec_file, O_RDONLY)) < 0) { > > I think pathmap_open() is just used for libraries. > > thanks, > > Chris > > On 7/7/20 6:18 PM, Yasumasa Suenaga wrote: >> Hi Chris, >> >> SA would use `link_map` to decide to load address, but it does not >> seem to contain executable. >> I set breakpoint to pathmap_open() and I watched the argument of it, >> then I didn't see any executable (`java`) on it. >> Maybe current implementation is broken. >> >> I guess we can use note section in the core for deciding loading >> address. >> I can see valid address (includes executable) from `readelf -n`. >> Of course it might be big change for SA... >> >> >> Thanks, >> >> Yasumasa >> >> >> On 2020/07/07 15:38, Chris Plummer wrote: >>> Hi Yasumasa, >>> >>> Thanks for the review. I tried the following for line 188: >>> >>> ???? if ((phdr->p_type == PT_LOAD || phdr->p_type == PT_INTERP) && >>> phdr->p_vaddr < baseaddr) { >>> >>> However, "base" still ended up being 0. I added some printfs. For >>> the exec file there is both a PT_INTER with p_vaddr of 0x238 and a >>> PT_LOAD with p_vaddr 0. I'm not sure which to use, but in either >>> case that won't be the proper base when added to 0: >>> >>> ?? if (add_lib_info_fd(ph, exec_file, ph->core->exec_fd, >>> ?????????????????????? (uintptr_t)0 + >>> find_base_address(ph->core->exec_fd, &exec_ehdr)) == NULL) { >>> ???? goto err; >>> ?? } >>> >>> So maybe it's the (uintptr_t)0 that is the problem here. For shared >>> libs instead of 0 it computes the value to add: >>> >>> ??????????????? if (lib_base_diff == ZERO_LOAD_ADDRESS ) { >>> ????????????????? lib_base_diff = calc_prelinked_load_address(ph, >>> lib_fd, &elf_ehdr, link_map_addr); >>> ????????????????? if (lib_base_diff == INVALID_LOAD_ADDRESS) { >>> ??????????????????? close(lib_fd); >>> ??????????????????? return false; >>> ????????????????? } >>> ??????????????? } >>> >>> ??????????????? lib_base = lib_base_diff + find_base_address(lib_fd, >>> &elf_ehdr); >>> >>> So in this case we've actually computed lib_base_diff rather than >>> just assumed 0. >>> >>> Chris >>> >>> On 7/6/20 10:46 PM, Yasumasa Suenaga wrote: >>>> Hi Chris, >>>> >>>> Your change looks good. >>>> >>>> >>>> BTW I saw JDK-8248876. I'm not sure, but I guess we can fix this >>>> issue if we allow PT_INTERP in L118: >>>> >>>> ``` >>>> 105 uintptr_t find_base_address(int fd, ELF_EHDR* ehdr) { >>>> ????????????????? : >>>> 115?? // the base address of a shared object is the lowest vaddr of >>>> 116?? // its loadable segments (PT_LOAD) >>>> 117?? for (phdr = phbuf, cnt = 0; cnt < ehdr->e_phnum; cnt++, >>>> phdr++) { >>>> 118???? if (phdr->p_type == PT_LOAD && phdr->p_vaddr < baseaddr) { >>>> 119?????? baseaddr = phdr->p_vaddr; >>>> 120???? } >>>> 121?? } >>>> ``` >>>> >>>> /proc//maps shows top of `java` is 0x56543b9df000: >>>> >>>> 56543b9df000-56543b9e0000 r--p 00000000 08:10 55770 >>>> /usr/lib/jvm/java-11-openjdk-amd64/bin/java >>>> >>>> >>>> `i target` on GDB shows 0x56543b9df000 is .interp section: >>>> >>>> Local exec file: >>>> ??????? `/usr/lib/jvm/java-11-openjdk-amd64/bin/java', file type >>>> elf64-x86-64. >>>> ??????? Entry point: 0x56543b9e0330 >>>> ??????? 0x000056543b9df318 - 0x000056543b9df334 is .interp >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2020/07/07 13:18, Chris Plummer wrote: >>>>> Hello, >>>>> >>>>> Please help review the following: >>>>> >>>>> http://cr.openjdk.java.net/~cjplummer/8248878/webrev.00/index.html >>>>> https://bugs.openjdk.java.net/browse/JDK-8248878 >>>>> >>>>> The explanation of the fix is in the CR. The parent CR, >>>>> JDK-8248876 [1], explains the issue being addressed. >>>>> >>>>> There's no test for this fix yet. It requires the changes I'm >>>>> making for JDK-8247514 [2], which include changes to "findpc" >>>>> support and the ClhsdbFindPC.java test that trigger this issue. >>>>> >>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8248876 >>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8247514 >>>>> >>>>> thanks, >>>>> >>>>> Chris >>> > From igor.ignatyev at oracle.com Mon Jul 13 22:29:31 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 13 Jul 2020 15:29:31 -0700 Subject: RFR [15] : 8249039 : clean up FileInstaller $test.src $cwd in vmTestbase_nsk_aod tests Message-ID: <4A9D95BB-E8AD-42B7-BC92-71C16C0F2B40@oracle.com> http://cr.openjdk.java.net/~iignatyev/8249039/webrev.00/ > 38 lines changed: 8 ins; 13 del; 17 mod; Hi all, could you please review this clean-up which removes `FileInstaller` actions from :vmTestbase_nsk_aod tests? from the main issue(8204985): > all vmTestbase tests have '@run driver jdk.test.lib.FileInstaller . .' to mimic old test harness behavior and copy all files from a test source directory to a current work directory. some tests depend on this step, so we need 1st identify such tests and then either rewrite them not to have this dependency or leave FileInstaller only in these tests. VirtualMachine06 test assumed that manifest files are located in the current directory, the test has been updated to have ${test.src} in the path to manifests. all other vmTestbase_nsk_stress tests don't need FileInstaller, so the rest of the patch is produced by `ag -l '@run driver jdk.test.lib.FileInstaller . .' vmTestbase/nsk/aod xargs -I{} gsed -i '/@run driver jdk.test.lib.FileInstaller \. \./d' {}`. JBS: https://bugs.openjdk.java.net/browse/JDK-8249039 webrev: http://cr.openjdk.java.net/~iignatyev//8249039/webrev.00/ testing: :vmTestbase_nsk_aod on linux-x64 Thanks, -- Igor From igor.ignatyev at oracle.com Mon Jul 13 22:40:32 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 13 Jul 2020 15:40:32 -0700 Subject: RFR [15] : 8249035 : clean up FileInstaller $test.src $cwd in vmTestbase_nsk_jdwp tests Message-ID: http://cr.openjdk.java.net/~iignatyev/8249035/webrev.00/ > 251 lines changed: 0 ins; 128 del; 123 mod; Hi all, could you please review this clean-up which removes `FileInstaller` actions from : vmTestbase_nsk_jdwp tests? from the main issue(8204985): > all vmTestbase tests have '@run driver jdk.test.lib.FileInstaller . .' to mimic old test harness behavior and copy all files from a test source directory to a current work directory. some tests depend on this step, so we need 1st identify such tests and then either rewrite them not to have this dependency or leave FileInstaller only in these tests. the majority of the patch is created by `ag -l '@run driver jdk.test.lib.FileInstaller . .' vmTestbase/nsk/jdwp xargs -I{} gsed -i '/@run driver jdk.test.lib.FileInstaller \. \./d' {}`. only two tests needed FileInstaller: VirtualMachine/RedefineClasses/redefinecls001 and Method/IsObsolete/isobsolete002. both tests have been updated to use aux class nsk.share.ExtraClassesBuilder, which search for source files in test.src (as opposed to current directory) instead of calling javac by ExecDriver; b/c n.s.ExtraClassesBuilder puts compiled classes into ./bin/newclass, the test code had to be slightly updated to search for classfiles in bin/newclass instead of newclass. JBS: https://bugs.openjdk.java.net/browse/JDK-8249035 webrev: http://cr.openjdk.java.net/~iignatyev//8249035/webrev.00/ testing: :vmTestbase_nsk_jdwp on linux-x64 Thanks, -- Igor From leonid.mesnik at oracle.com Mon Jul 13 22:42:50 2020 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Mon, 13 Jul 2020 15:42:50 -0700 (PDT) Subject: RFR: 8242891: vmTestbase/nsk/jvmti/ test should be fixed to fail early if JVMTI function return error In-Reply-To: <611D0AF5-1295-4DD4-B6C9-F4D379AA069C@oracle.com> References: <11314027-4965-b38b-6bc7-5011515b94ab@oracle.com> <2cf4e45a-4d44-3c0a-a272-480f56a5e6e8@oracle.com> <3927ae7c-efa9-eb9f-ab98-18d778d5a966@oracle.com> <547f02e9-604c-09e5-5fe1-2afb1be54f2d@oracle.com> <611D0AF5-1295-4DD4-B6C9-F4D379AA069C@oracle.com> Message-ID: <01076F65-C037-4308-9782-F3348274C858@oracle.com> Could I have 2nd reviewer? Leonid > On Jun 25, 2020, at 10:58 AM, Leonid Mesnik wrote: > > Ping > >> On Jun 12, 2020, at 4:18 PM, Leonid Mesnik > wrote: >> >> Fixed all places, updated copyright. Still need second review >> >> http://cr.openjdk.java.net/~lmesnik/8242891/webrev.02/ >> Leonid >> >> On 6/11/20 8:41 PM, serguei.spitsyn at oracle.com wrote: >>> Hi Leonid, >>> >>> It is much better now. >>> >>> Several places still need the same fix. >>> >>> http://cr.openjdk.java.net/~lmesnik/8242891/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetAllThreads/allthr001/allthr001.cpp.frames.html >>> >>> 211 for (i = 0; i < thrInfo[ind].cnt; i++) { >>> 212 for (j = 0, found = 0; j < threadsCount && !found; j++) { >>> 213 err = jvmti->GetThreadInfo(threads[j], &inf); >>> 214 if (err != JVMTI_ERROR_NONE) { >>> 215 printf("Failed to get thread info: %s (%d)\n", >>> 216 TranslateError(err), err); >>> 217 result = STATUS_FAILED; >>> 218 } >>> 219 if (printdump == JNI_TRUE) { >>> 220 printf(" >>> %s", inf.name); >>> 221 } >>> 222 found = (inf.name != NULL && >>> 223 strstr(inf.name, thrInfo[ind].thrNames[i]) == inf.name && >>> 224 (ind == 4 || strlen(inf.name) == >>> 225 strlen(thrInfo[ind].thrNames[i]))); >>> 226 } >>> A return is needed after line 217, otherwise the the inf value is used at lines 222-224. >>> >>> http://cr.openjdk.java.net/~lmesnik/8242891/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetBytecodes/bytecodes003/bytecodes003.cpp.frames.html >>> >>> A return is needed for the errors: >>> 363 result = STATUS_FAILED; >>> 372 result = STATUS_FAILED; >>> 384 result = STATUS_FAILED; >>> >>> http://cr.openjdk.java.net/~lmesnik/8242891/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/MethodEntry/mentry001/mentry001.cpp.frames.html >>> >>> A return is needed for the errors: >>> 82 result = STATUS_FAILED; >>> 94 result = STATUS_FAILED; >>> 100 result = STATUS_FAILED; >>> >>> http://cr.openjdk.java.net/~lmesnik/8242891/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/MethodExit/mexit001/mexit001.cpp.frames.html >>> >>> A return is needed for the error: >>> 98 result = STATUS_FAILED; >>> >>> http://cr.openjdk.java.net/~lmesnik/8242891/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/MethodExit/mexit002/mexit002.cpp.frames.html >>> >>> A return is needed for the error: >>> 98 result = STATUS_FAILED; >>> >>> http://cr.openjdk.java.net/~lmesnik/8242891/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass019/redefclass019.cpp.frames.html >>> >>> A return is needed for the error: >>> 186 result = STATUS_FAILED; >>> >>> Also, I do not like many uninitialized locals in these tests. >>> But it is for another pass. >>> >>> Otherwise, it looks good. >>> No need for another webrev if you fix the above. >>> I hope, you will update copyright comments before push. >>> >>> Thanks, >>> Serguei >>> >>> >>> On 6/11/20 15:30, Leonid Mesnik wrote: >>>> Agree, it would be better to don't try to use data from functions with error code. The new webrev: >>>> >>>> http://cr.openjdk.java.net/~lmesnik/8242891/webrev.01/ >>>> I tried to prevent any usage of possibly corrupted data. Mostly strings or allocated data, sometimes method/class id which are used my other JVMTI functions. >>>> >>>> Leonid >>>> >>>> On 6/9/20 6:59 PM, serguei.spitsyn at oracle.com wrote: >>>>> On 6/9/20 12:58, Leonid Mesnik wrote: >>>>>> Hi >>>>>> >>>>>> >>>>>> On 6/9/20 12:34 PM, serguei.spitsyn at oracle.com wrote: >>>>>>> Hi Leonid, >>>>>>> >>>>>>> Thank you for taking care about this! >>>>>>> It looks good in general. >>>>>>> However, I think, a similar return is needed in more cases. >>>>>>> >>>>>>> One example: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~lmesnik/8242891/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/jvmti/Exception/exception001/exception001.cpp.frames.html >>>>>>> >>>>>>> 99 err = jvmti_env->GetMethodDeclaringClass(method, &cls); >>>>>>> 100 if (err != JVMTI_ERROR_NONE) { >>>>>>> 101 printf("(GetMethodDeclaringClass#t) unexpected error: %s (%d)\n", >>>>>>> 102 TranslateError(err), err); >>>>>>> 103 result = STATUS_FAILED; >>>>>>> <> 104 return; >>>>>>> 105 } >>>>>>> 106 err = jvmti_env->GetClassSignature(cls, &ex.t_cls, &generic); >>>>>>> 107 if (err != JVMTI_ERROR_NONE) { >>>>>>> 108 printf("(GetClassSignature#t) unexpected error: %s (%d)\n", >>>>>>> 109 TranslateError(err), err); >>>>>>> 110 result = STATUS_FAILED; >>>>>>> 111 } >>>>>>> 112 err = jvmti_env->GetMethodName(method, >>>>>>> 113 &ex.t_name, &ex.t_sig, &generic); >>>>>>> 114 if (err != JVMTI_ERROR_NONE) { >>>>>>> 115 printf("(GetMethodName#t) unexpected error: %s (%d)\n", >>>>>>> 116 TranslateError(err), err); >>>>>>> 117 result = STATUS_FAILED; >>>>>>> 118 } >>>>>>> 119 ex.t_loc = location; >>>>>>> 120 err = jvmti_env->GetMethodDeclaringClass(catch_method, &cls); >>>>>>> 121 if (err != JVMTI_ERROR_NONE) { >>>>>>> 122 printf("(GetMethodDeclaringClass#c) unexpected error: %s (%d)\n", >>>>>>> 123 TranslateError(err), err); >>>>>>> 124 result = STATUS_FAILED; >>>>>>> <> 125 return; >>>>>>> 126 } >>>>>>> 127 err = jvmti_env->GetClassSignature(cls, &ex.c_cls, &generic); >>>>>>> 128 if (err != JVMTI_ERROR_NONE) { >>>>>>> 129 printf("(GetClassSignature#c) unexpected error: %s (%d)\n", >>>>>>> 130 TranslateError(err), err); >>>>>>> 131 result = STATUS_FAILED; >>>>>>> 132 } >>>>>>> 133 err = jvmti_env->GetMethodName(catch_method, >>>>>>> 134 &ex.c_name, &ex.c_sig, &generic); >>>>>>> 135 if (err != JVMTI_ERROR_NONE) { >>>>>>> 136 printf("(GetMethodName#c) unexpected error: %s (%d)\n", >>>>>>> 137 TranslateError(err), err); >>>>>>> 138 result = STATUS_FAILED; >>>>>>> 139 } >>>>>>> >>>>>>> In the fragment above you added return for JVMTI GetMethodDeclaringClass error. >>>>>>> But GetMethodName and GetClassSignature can be also problematic as the returned names are printed below. >>>>>>> It seems to be more safe and even simpler to add returns for such cases as well. >>>>>>> Otherwise, the code reader is puzzled why there is a return in one failure case and there is no such return in another. >>>>>> It is a good question if we want to fix such places or even fails with first JVMTI failure. (I even started to fix it in the such way but find that existing tests usually don't fail always). >>>>>> >>>>> >>>>> I do not suggest to fix all the tests but those which you are already fixing. >>>>> >>>>> >>>>>> >>>>>> The difference is that test tries to reuse "cls" in other JVMTI function and going to generate very misleading crash. How it just tries to compare ex and exs values. So test might crash but clearly outside of JVMTI function and with some useful info. So I am not sure if fixing these lines improve test failure handling. >>>>>> >>>>> >>>>> If JVMTI functions fail with an error code the results with symbolic strings must be considered invalid. >>>>> However, they are used later (the values are printed). >>>>> It is better to bail out in such cases. >>>>> It should not be a problem to add similar returns in such cases. >>>>> Or do you think it is important to continue execution for some reason? >>>>> >>>>>> Assuming that most of existing tests fails early only if going to re-use possible corrupted data I propose to fix this separately. We need to figure out when to fail or to try to finish. >>>>>> >>>>> >>>>> Do you suggest it for the updated tests only or for all the tests with such problems? >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>>> >>>>>> Leonid >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Serguei >>>>>>> >>>>>>> >>>>>>> On 6/1/20 21:33, Leonid Mesnik wrote: >>>>>>>> Hi >>>>>>>> >>>>>>>> Could you please review following fix which stop test execution if JVMTI function returns error. The test fails anyway however using potentially bad data in JVMTI function might cause misleading crash failures. The hs_err will contains the stacktrace not with problem function but with function called with corrupted data. Most of tests already has such behavior but not all. Also I fixed a couple of tests to finish if they haven't managed to suspend thread. >>>>>>>> >>>>>>>> I've updated only tests which try to use corrupted data in JVMTI functions after errors. I haven't updated tests which just compare/print values from erroring JVMTI functions. The crash in strcmp/println is not so misleading and might be point to real issue. >>>>>>>> >>>>>>>> webrev: http://cr.openjdk.java.net/~lmesnik/8242891/webrev.00/ >>>>>>>> >>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8242891 >>>>>>>> >>>>>>>> Leonid >>>>>>>> >>>>>>> >>>>> >>> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From igor.ignatyev at oracle.com Mon Jul 13 23:22:57 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 13 Jul 2020 16:22:57 -0700 Subject: RFR [15] : 8249034 : clean up FileInstaller $test.src $cwd in vmTestbase_nsk_jvmti tests Message-ID: http://cr.openjdk.java.net/~iignatyev/8249034/webrev.00/ > 1289 lines changed: 2 ins; 652 del; 635 mod; Hi all, could you please review this clean-up which removes `FileInstaller` actions from :vmTestbase_nsk_jvmti tests? from the main issue(8204985): > all vmTestbase tests have '@run driver jdk.test.lib.FileInstaller . .' to mimic old test harness behavior and copy all files from a test source directory to a current work directory. some tests depend on this step, so we need 1st identify such tests and then either rewrite them not to have this dependency or leave FileInstaller only in these tests. `ag -l '@run driver jdk.test.lib.FileInstaller . .' vmTestbase/nsk/jvmti vmTestbase/nsk/share/ExceptionCheckingJniEnv xargs -I{} gsed -i '/@run driver jdk.test.lib.FileInstaller \. \./d' {}` was used to remove FileInstaller actions from all nsk_jvmti tests. 3 tests had dependency on that action: - Allocate/alloc001, where alloc001.sh was assuming that alloc001.bash is in the current directory. alloc001.sh has been updated to use ${TESTSRC} env variable - AttachOnDemand/attach002a and scenarios/bcinstr/BI04/bi04t002: both tests assumed that newclassXX directory is located in the current directory. the tests were fixed by prefixing 'newclassXX' w/ '${test.src}/' which gets expanded by PropertyResolvingWrapper JBS: https://bugs.openjdk.java.net/browse/JDK-8249034 webrev: http://cr.openjdk.java.net/~iignatyev//8249034/webrev.00/ testing: :vmTestbase_nsk_jvmti on linux-x64 Thanks, -- Igor From suenaga at oss.nttdata.com Mon Jul 13 23:48:42 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Tue, 14 Jul 2020 08:48:42 +0900 Subject: RFR(XS): 8248878: SA: Implement simple workaround for JDK-8248876 In-Reply-To: <046aced2-1d23-d096-748b-4ce0bfd01587@oracle.com> References: <39ac40d6-a738-3214-d86c-412152a631d4@oracle.com> <8e89d223-0f65-f26e-3bd7-2ca7421a940b@oracle.com> <6d2610d6-5ca1-8455-ddc6-d7f02b21c0d5@oracle.com> <046aced2-1d23-d096-748b-4ce0bfd01587@oracle.com> Message-ID: <1fde7056-f2d0-e256-ae6d-b8db94068c71@oss.nttdata.com> Sure, I reviewed for this workaround (JDK-8248878). I'm still think we can use note section in the core for JDK-8248876 as I said, but I'm not sure. Thanks, Yasumasa On 2020/07/14 3:12, Chris Plummer wrote: > Hi Yasumasa, > > If you have no further suggestions on how to fix JDK-8248876, I'd like to proceed with this work around. Can I considered it reviewed by you? > > thanks, > > Chris > > On 7/7/20 7:29 PM, Chris Plummer wrote: >> Hi Yasumasa, >> >> The executable is not opened with pathmap_open: >> >> ? if ((ph->core->exec_fd = open(exec_file, O_RDONLY)) < 0) { >> >> I think pathmap_open() is just used for libraries. >> >> thanks, >> >> Chris >> >> On 7/7/20 6:18 PM, Yasumasa Suenaga wrote: >>> Hi Chris, >>> >>> SA would use `link_map` to decide to load address, but it does not seem to contain executable. >>> I set breakpoint to pathmap_open() and I watched the argument of it, then I didn't see any executable (`java`) on it. >>> Maybe current implementation is broken. >>> >>> I guess we can use note section in the core for deciding loading address. >>> I can see valid address (includes executable) from `readelf -n`. >>> Of course it might be big change for SA... >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2020/07/07 15:38, Chris Plummer wrote: >>>> Hi Yasumasa, >>>> >>>> Thanks for the review. I tried the following for line 188: >>>> >>>> ???? if ((phdr->p_type == PT_LOAD || phdr->p_type == PT_INTERP) && phdr->p_vaddr < baseaddr) { >>>> >>>> However, "base" still ended up being 0. I added some printfs. For the exec file there is both a PT_INTER with p_vaddr of 0x238 and a PT_LOAD with p_vaddr 0. I'm not sure which to use, but in either case that won't be the proper base when added to 0: >>>> >>>> ?? if (add_lib_info_fd(ph, exec_file, ph->core->exec_fd, >>>> ?????????????????????? (uintptr_t)0 + find_base_address(ph->core->exec_fd, &exec_ehdr)) == NULL) { >>>> ???? goto err; >>>> ?? } >>>> >>>> So maybe it's the (uintptr_t)0 that is the problem here. For shared libs instead of 0 it computes the value to add: >>>> >>>> ??????????????? if (lib_base_diff == ZERO_LOAD_ADDRESS ) { >>>> ????????????????? lib_base_diff = calc_prelinked_load_address(ph, lib_fd, &elf_ehdr, link_map_addr); >>>> ????????????????? if (lib_base_diff == INVALID_LOAD_ADDRESS) { >>>> ??????????????????? close(lib_fd); >>>> ??????????????????? return false; >>>> ????????????????? } >>>> ??????????????? } >>>> >>>> ??????????????? lib_base = lib_base_diff + find_base_address(lib_fd, &elf_ehdr); >>>> >>>> So in this case we've actually computed lib_base_diff rather than just assumed 0. >>>> >>>> Chris >>>> >>>> On 7/6/20 10:46 PM, Yasumasa Suenaga wrote: >>>>> Hi Chris, >>>>> >>>>> Your change looks good. >>>>> >>>>> >>>>> BTW I saw JDK-8248876. I'm not sure, but I guess we can fix this issue if we allow PT_INTERP in L118: >>>>> >>>>> ``` >>>>> 105 uintptr_t find_base_address(int fd, ELF_EHDR* ehdr) { >>>>> ????????????????? : >>>>> 115?? // the base address of a shared object is the lowest vaddr of >>>>> 116?? // its loadable segments (PT_LOAD) >>>>> 117?? for (phdr = phbuf, cnt = 0; cnt < ehdr->e_phnum; cnt++, phdr++) { >>>>> 118???? if (phdr->p_type == PT_LOAD && phdr->p_vaddr < baseaddr) { >>>>> 119?????? baseaddr = phdr->p_vaddr; >>>>> 120???? } >>>>> 121?? } >>>>> ``` >>>>> >>>>> /proc//maps shows top of `java` is 0x56543b9df000: >>>>> >>>>> 56543b9df000-56543b9e0000 r--p 00000000 08:10 55770 /usr/lib/jvm/java-11-openjdk-amd64/bin/java >>>>> >>>>> >>>>> `i target` on GDB shows 0x56543b9df000 is .interp section: >>>>> >>>>> Local exec file: >>>>> ??????? `/usr/lib/jvm/java-11-openjdk-amd64/bin/java', file type elf64-x86-64. >>>>> ??????? Entry point: 0x56543b9e0330 >>>>> ??????? 0x000056543b9df318 - 0x000056543b9df334 is .interp >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2020/07/07 13:18, Chris Plummer wrote: >>>>>> Hello, >>>>>> >>>>>> Please help review the following: >>>>>> >>>>>> http://cr.openjdk.java.net/~cjplummer/8248878/webrev.00/index.html >>>>>> https://bugs.openjdk.java.net/browse/JDK-8248878 >>>>>> >>>>>> The explanation of the fix is in the CR. The parent CR, JDK-8248876 [1], explains the issue being addressed. >>>>>> >>>>>> There's no test for this fix yet. It requires the changes I'm making for JDK-8247514 [2], which include changes to "findpc" support and the ClhsdbFindPC.java test that trigger this issue. >>>>>> >>>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8248876 >>>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8247514 >>>>>> >>>>>> thanks, >>>>>> >>>>>> Chris >>>> >> > > From david.holmes at oracle.com Tue Jul 14 02:47:25 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 14 Jul 2020 12:47:25 +1000 Subject: RFR [15] : 8249028 : clean up FileInstaller $test.src $cwd in vmTestbase_nsk_monitoring tests In-Reply-To: References: Message-ID: <54a61c0c-1002-2dfd-d300-e4e11b1f0cf1@oracle.com> Looks good! Thanks, David On 9/07/2020 3:21 am, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8249028/webrev.00/ >> 547 lines changed: 0 ins; 361 del; 186 mod; > > Hi all, > > could you please review the patch which removes `FileInstaller . .` jtreg action from :vmTestbase_nsk_monitoring tests? > from the main issue(8204985): >> all vmTestbase tests have '@run driver jdk.test.lib.FileInstaller . .' to mimic old test harness behavior and copy all files from a test source directory to a current work directory. some tests depend on this step, so we need 1st identify such tests and then either rewrite them not to have this dependency or leave FileInstaller only in these tests. > > effectively, the patch is just `ag -l '@run driver jdk.test.lib.FileInstaller . .' vmTestbase/nsk/monitoring | xargs -I{} gsed -i '/@run driver jdk.test.lib.FileInstaller \. \./d' {}` > > testing: :vmTestbase_nsk_monitoring on linux-x64 > webrev: http://cr.openjdk.java.net/~iignatyev//8249018/webrev.00 > JBS: https://bugs.openjdk.java.net/browse/JDK-8249028 > > Thanks, > -- Igor > > From david.holmes at oracle.com Tue Jul 14 02:51:01 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 14 Jul 2020 12:51:01 +1000 Subject: RFR [15] : 8249035 : clean up FileInstaller $test.src $cwd in vmTestbase_nsk_jdwp tests In-Reply-To: References: Message-ID: Looks good! Thanks, David On 14/07/2020 8:40 am, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8249035/webrev.00/ >> 251 lines changed: 0 ins; 128 del; 123 mod; > > Hi all, > > could you please review this clean-up which removes `FileInstaller` actions from : vmTestbase_nsk_jdwp tests? > from the main issue(8204985): >> all vmTestbase tests have '@run driver jdk.test.lib.FileInstaller . .' to mimic old test harness behavior and copy all files from a test source directory to a current work directory. some tests depend on this step, so we need 1st identify such tests and then either rewrite them not to have this dependency or leave FileInstaller only in these tests. > > the majority of the patch is created by `ag -l '@run driver jdk.test.lib.FileInstaller . .' vmTestbase/nsk/jdwp xargs -I{} gsed -i '/@run driver jdk.test.lib.FileInstaller \. \./d' {}`. only two tests needed FileInstaller: VirtualMachine/RedefineClasses/redefinecls001 and Method/IsObsolete/isobsolete002. both tests have been updated to use aux class nsk.share.ExtraClassesBuilder, which search for source files in test.src (as opposed to current directory) instead of calling javac by ExecDriver; b/c n.s.ExtraClassesBuilder puts compiled classes into ./bin/newclass, the test code had to be slightly updated to search for classfiles in bin/newclass instead of newclass. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8249035 > webrev: http://cr.openjdk.java.net/~iignatyev//8249035/webrev.00/ > testing: :vmTestbase_nsk_jdwp on linux-x64 > > Thanks, > -- Igor > From jaroslav.bachorik at datadoghq.com Tue Jul 14 13:31:27 2020 From: jaroslav.bachorik at datadoghq.com (=?UTF-8?Q?Jaroslav_Bachor=C3=ADk?=) Date: Tue, 14 Jul 2020 15:31:27 +0200 Subject: Some clarification about JVMTI_EVENT_THREAD_START and PRIMORDIAL phase In-Reply-To: References: <6287d398-5828-762d-3609-2767b684186d@oracle.com> <529E99DA-4DFA-4C13-BCC1-A47C82521061@azul.com> Message-ID: Thank you all for chiming in! My main concern was that the JDK 8 JVMTI spec (https://docs.oracle.com/javase/8/docs/platform/jvmti/jvmti.html#ThreadStart) was contradicting what was actually put to the EARLY_EVENT_BITS variable and therefore which events were allowed in PRIMORDIAL and ONLOAD (https://hg.openjdk.java.net/jdk8u/jdk8u-dev/hotspot/file/02b4fd2f9041/src/share/vm/prims/jvmtiEventController.cpp#l100) So, if I went and remove `THREAD_START_BIT | THREAD_END_BIT` part from the EARLY_EVENT_BITS in order to fix the assumed regression due to JDK-8233197 I would be actually following the spec, right? Cheers, -JB- On Mon, Jul 13, 2020 at 4:02 PM Daniel D. Daugherty wrote: > > If there is going to be a spec change, then it would not affect JDK8. > I don't think anyone has decided how this issue should be resolved yet. > > Dan > > > On 7/13/20 7:10 AM, Sergey Nazarkin wrote: > > Hi Daniel, > > > > Please correct me if I?m wrong but the spec change will touch post-JDK8 releases only? > > > > > > > > Sergey Nazarkin > > > > > > > > > >> On Jul 9, 2020, at 19:25, Daniel D. Daugherty wrote: > >> > >> On 7/9/20 9:07 AM, Jaroslav Bachor?k wrote: > >>> Hello, > >>> > >>> Recently, after backporting JDK-8233197 to JDK8u I received a report > >>> from Sergey Nazarkin (cc) that the backport broke JVMTI spec basically > >>> emitting JVMTI_EVENT_THREAD_START events in primordial phase while the > >>> spec states that it should be emitted only in either start or live > >>> phase. > >>> > >>> But if I am reading this correctly > >>> (https://hg.openjdk.java.net/jdk8u/jdk8udev/hotspot/file/02b4fd2f9041/src/share/vm/prims/jvmtiEventController.cpp#l418) > >>> the spec is also violated there. > >>> > >>> EARLY_EVENT_BITS are including THREAD_START_BIT and it is not filtered > >>> out by THREAD_FILTERED_EVENT_BITS so it seems perfectly valid for > >>> JVMTI_EVENT_THREAD_START to appear during PRIMORDIAL phase. > >>> > >>> So I would like to get clarification before going in and start > >>> tinkering with the JVM initialization code order. > >>> > >>> Thanks! > >>> > >>> -JB- > >> Hi Jaroslav! > >> > >> It has been a long time my friend... :-) > >> > >> It looks like that change was made by this Jigsaw change: > >> > >> JDK-8072745 Re-examine JVM TI execution phases and expectations > >> on what is allowed in each phase > >> https://bugs.openjdk.java.net/browse/JDK-8072745 > >> > >> These diffs in the bug reported jumped out at me: > >> > >> @@ -1022,6 +1032,10 @@ > >> void JvmtiExport::post_thread_start(JavaThread *thread) { > >> assert(thread->thread_state() == _thread_in_vm, "must be in vm state"); > >> > >> + if (JvmtiEnv::get_phase() <= JVMTI_PHASE_PRIMORDIAL) { > >> + return; > >> + } > >> + > >> EVT_TRIG_TRACE(JVMTI_EVENT_THREAD_START, ("JVMTI [%s] Trg Thread Start event triggered", > >> JvmtiTrace::safe_get_thread_name(thread))); > >> > >> @@ -1050,6 +1064,10 @@ > >> > >> > >> void JvmtiExport::post_thread_end(JavaThread *thread) { > >> + if (JvmtiEnv::get_phase() <= JVMTI_PHASE_PRIMORDIAL) { > >> + return; > >> + } > >> + > >> EVT_TRIG_TRACE(JVMTI_EVENT_THREAD_END, ("JVMTI [%s] Trg Thread End event triggered", > >> JvmtiTrace::safe_get_thread_name(thread))); > >> > >> > >> Here's the order of phases from jvmti.h: > >> > >> /* Phases of execution */ > >> > >> typedef enum { > >> JVMTI_PHASE_ONLOAD = 1, > >> JVMTI_PHASE_PRIMORDIAL = 2, > >> JVMTI_PHASE_START = 6, > >> JVMTI_PHASE_LIVE = 4, > >> JVMTI_PHASE_DEAD = 8 > >> } jvmtiPhase; > >> > >> However, the above diffs prevent the JVMTI_EVENT_THREAD_START > >> and JVMTI_EVENT_THREAD_END events from being posted earlier than > >> the JVMTI_PHASE_START phase... Unfortunately, I can't find the > >> actual changeset pushed to the Jigsaw report for JDK-8072745. > >> > >> > >> When you look at the current JvmtiExport::post_thread_start() > >> and JvmtiExport::post_thread_end() functions, you can see this: > >> > >> 1: void JvmtiExport::post_thread_start(JavaThread *thread) { > >> 36508: if (JvmtiEnv::get_phase() < JVMTI_PHASE_PRIMORDIAL) { > >> 36508: return; > >> 36508: } > >> > >> 1: void JvmtiExport::post_thread_end(JavaThread *thread) { > >> 36508: if (JvmtiEnv::get_phase() < JVMTI_PHASE_PRIMORDIAL) { > >> 36508: return; > >> 36508: } > >> > >> $ hg log -r 36508 > >> changeset: 36508:5f9eee6b383b > >> user: alanb > >> date: Thu Mar 17 19:04:01 2016 +0000 > >> summary: 8142968: Module System implementation > >> > >> > >> So the Jigsaw integration pushed a slightly different change > >> than is shown in JDK-8072745. I don't know whether the fix > >> for JDK-8072745 evolved before being pushed to the Jigsaw > >> repos or whether another JBS issue was used to modify the > >> code after JDK-8072745 was pushed. > >> > >> Short version: The semantics of the JVMTI_EVENT_THREAD_START > >> and JVMTI_EVENT_THREAD_END were changed by Jigsaw, but the > >> JVM/TI spec was not updated to reflect those changes. > >> > >> I've filed a new bug to track this issue: > >> > >> JDK-8249158 THREAD_START and THREAD_END event posting changed > >> by modules without updating the spec > >> https://bugs.openjdk.java.net/browse/JDK-8249158 > >> > >> Dan > From daniel.daugherty at oracle.com Tue Jul 14 16:06:12 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 14 Jul 2020 12:06:12 -0400 Subject: Some clarification about JVMTI_EVENT_THREAD_START and PRIMORDIAL phase In-Reply-To: References: <6287d398-5828-762d-3609-2767b684186d@oracle.com> <529E99DA-4DFA-4C13-BCC1-A47C82521061@azul.com> Message-ID: Jarsolav, I'm copying this reply to the bug (JDK-8249158). I figure folks should continue the conversation there. Dan On 7/14/20 9:31 AM, Jaroslav Bachor?k wrote: > Thank you all for chiming in! > > My main concern was that the JDK 8 JVMTI spec > (https://docs.oracle.com/javase/8/docs/platform/jvmti/jvmti.html#ThreadStart) > was contradicting what was actually put to the EARLY_EVENT_BITS > variable and therefore which events were allowed in PRIMORDIAL and > ONLOAD (https://hg.openjdk.java.net/jdk8u/jdk8u-dev/hotspot/file/02b4fd2f9041/src/share/vm/prims/jvmtiEventController.cpp#l100) > > So, if I went and remove `THREAD_START_BIT | THREAD_END_BIT` part from > the EARLY_EVENT_BITS in order to fix the assumed regression due to > JDK-8233197 I would be actually following the spec, right? > > Cheers, > > -JB- > > On Mon, Jul 13, 2020 at 4:02 PM Daniel D. Daugherty > wrote: >> If there is going to be a spec change, then it would not affect JDK8. >> I don't think anyone has decided how this issue should be resolved yet. >> >> Dan >> >> >> On 7/13/20 7:10 AM, Sergey Nazarkin wrote: >>> Hi Daniel, >>> >>> Please correct me if I?m wrong but the spec change will touch post-JDK8 releases only? >>> >>> >>> >>> Sergey Nazarkin >>> >>> >>> >>> >>>> On Jul 9, 2020, at 19:25, Daniel D. Daugherty wrote: >>>> >>>> On 7/9/20 9:07 AM, Jaroslav Bachor?k wrote: >>>>> Hello, >>>>> >>>>> Recently, after backporting JDK-8233197 to JDK8u I received a report >>>>> from Sergey Nazarkin (cc) that the backport broke JVMTI spec basically >>>>> emitting JVMTI_EVENT_THREAD_START events in primordial phase while the >>>>> spec states that it should be emitted only in either start or live >>>>> phase. >>>>> >>>>> But if I am reading this correctly >>>>> (https://hg.openjdk.java.net/jdk8u/jdk8udev/hotspot/file/02b4fd2f9041/src/share/vm/prims/jvmtiEventController.cpp#l418) >>>>> the spec is also violated there. >>>>> >>>>> EARLY_EVENT_BITS are including THREAD_START_BIT and it is not filtered >>>>> out by THREAD_FILTERED_EVENT_BITS so it seems perfectly valid for >>>>> JVMTI_EVENT_THREAD_START to appear during PRIMORDIAL phase. >>>>> >>>>> So I would like to get clarification before going in and start >>>>> tinkering with the JVM initialization code order. >>>>> >>>>> Thanks! >>>>> >>>>> -JB- >>>> Hi Jaroslav! >>>> >>>> It has been a long time my friend... :-) >>>> >>>> It looks like that change was made by this Jigsaw change: >>>> >>>> JDK-8072745 Re-examine JVM TI execution phases and expectations >>>> on what is allowed in each phase >>>> https://bugs.openjdk.java.net/browse/JDK-8072745 >>>> >>>> These diffs in the bug reported jumped out at me: >>>> >>>> @@ -1022,6 +1032,10 @@ >>>> void JvmtiExport::post_thread_start(JavaThread *thread) { >>>> assert(thread->thread_state() == _thread_in_vm, "must be in vm state"); >>>> >>>> + if (JvmtiEnv::get_phase() <= JVMTI_PHASE_PRIMORDIAL) { >>>> + return; >>>> + } >>>> + >>>> EVT_TRIG_TRACE(JVMTI_EVENT_THREAD_START, ("JVMTI [%s] Trg Thread Start event triggered", >>>> JvmtiTrace::safe_get_thread_name(thread))); >>>> >>>> @@ -1050,6 +1064,10 @@ >>>> >>>> >>>> void JvmtiExport::post_thread_end(JavaThread *thread) { >>>> + if (JvmtiEnv::get_phase() <= JVMTI_PHASE_PRIMORDIAL) { >>>> + return; >>>> + } >>>> + >>>> EVT_TRIG_TRACE(JVMTI_EVENT_THREAD_END, ("JVMTI [%s] Trg Thread End event triggered", >>>> JvmtiTrace::safe_get_thread_name(thread))); >>>> >>>> >>>> Here's the order of phases from jvmti.h: >>>> >>>> /* Phases of execution */ >>>> >>>> typedef enum { >>>> JVMTI_PHASE_ONLOAD = 1, >>>> JVMTI_PHASE_PRIMORDIAL = 2, >>>> JVMTI_PHASE_START = 6, >>>> JVMTI_PHASE_LIVE = 4, >>>> JVMTI_PHASE_DEAD = 8 >>>> } jvmtiPhase; >>>> >>>> However, the above diffs prevent the JVMTI_EVENT_THREAD_START >>>> and JVMTI_EVENT_THREAD_END events from being posted earlier than >>>> the JVMTI_PHASE_START phase... Unfortunately, I can't find the >>>> actual changeset pushed to the Jigsaw report for JDK-8072745. >>>> >>>> >>>> When you look at the current JvmtiExport::post_thread_start() >>>> and JvmtiExport::post_thread_end() functions, you can see this: >>>> >>>> 1: void JvmtiExport::post_thread_start(JavaThread *thread) { >>>> 36508: if (JvmtiEnv::get_phase() < JVMTI_PHASE_PRIMORDIAL) { >>>> 36508: return; >>>> 36508: } >>>> >>>> 1: void JvmtiExport::post_thread_end(JavaThread *thread) { >>>> 36508: if (JvmtiEnv::get_phase() < JVMTI_PHASE_PRIMORDIAL) { >>>> 36508: return; >>>> 36508: } >>>> >>>> $ hg log -r 36508 >>>> changeset: 36508:5f9eee6b383b >>>> user: alanb >>>> date: Thu Mar 17 19:04:01 2016 +0000 >>>> summary: 8142968: Module System implementation >>>> >>>> >>>> So the Jigsaw integration pushed a slightly different change >>>> than is shown in JDK-8072745. I don't know whether the fix >>>> for JDK-8072745 evolved before being pushed to the Jigsaw >>>> repos or whether another JBS issue was used to modify the >>>> code after JDK-8072745 was pushed. >>>> >>>> Short version: The semantics of the JVMTI_EVENT_THREAD_START >>>> and JVMTI_EVENT_THREAD_END were changed by Jigsaw, but the >>>> JVM/TI spec was not updated to reflect those changes. >>>> >>>> I've filed a new bug to track this issue: >>>> >>>> JDK-8249158 THREAD_START and THREAD_END event posting changed >>>> by modules without updating the spec >>>> https://bugs.openjdk.java.net/browse/JDK-8249158 >>>> >>>> Dan From jaroslav.bachorik at datadoghq.com Tue Jul 14 16:13:41 2020 From: jaroslav.bachorik at datadoghq.com (=?UTF-8?Q?Jaroslav_Bachor=C3=ADk?=) Date: Tue, 14 Jul 2020 18:13:41 +0200 Subject: Some clarification about JVMTI_EVENT_THREAD_START and PRIMORDIAL phase In-Reply-To: References: <6287d398-5828-762d-3609-2767b684186d@oracle.com> <529E99DA-4DFA-4C13-BCC1-A47C82521061@azul.com> Message-ID: Thank you, Dan, for translating this to an issue! -JB- On Tue, Jul 14, 2020 at 6:08 PM Daniel D. Daugherty wrote: > > Jarsolav, > > I'm copying this reply to the bug (JDK-8249158). I figure folks should > continue the conversation there. > > Dan > > > On 7/14/20 9:31 AM, Jaroslav Bachor?k wrote: > > Thank you all for chiming in! > > > > My main concern was that the JDK 8 JVMTI spec > > (https://docs.oracle.com/javase/8/docs/platform/jvmti/jvmti.html#ThreadStart) > > was contradicting what was actually put to the EARLY_EVENT_BITS > > variable and therefore which events were allowed in PRIMORDIAL and > > ONLOAD (https://hg.openjdk.java.net/jdk8u/jdk8u-dev/hotspot/file/02b4fd2f9041/src/share/vm/prims/jvmtiEventController.cpp#l100) > > > > So, if I went and remove `THREAD_START_BIT | THREAD_END_BIT` part from > > the EARLY_EVENT_BITS in order to fix the assumed regression due to > > JDK-8233197 I would be actually following the spec, right? > > > > Cheers, > > > > -JB- > > > > On Mon, Jul 13, 2020 at 4:02 PM Daniel D. Daugherty > > wrote: > >> If there is going to be a spec change, then it would not affect JDK8. > >> I don't think anyone has decided how this issue should be resolved yet. > >> > >> Dan > >> > >> > >> On 7/13/20 7:10 AM, Sergey Nazarkin wrote: > >>> Hi Daniel, > >>> > >>> Please correct me if I?m wrong but the spec change will touch post-JDK8 releases only? > >>> > >>> > >>> > >>> Sergey Nazarkin > >>> > >>> > >>> > >>> > >>>> On Jul 9, 2020, at 19:25, Daniel D. Daugherty wrote: > >>>> > >>>> On 7/9/20 9:07 AM, Jaroslav Bachor?k wrote: > >>>>> Hello, > >>>>> > >>>>> Recently, after backporting JDK-8233197 to JDK8u I received a report > >>>>> from Sergey Nazarkin (cc) that the backport broke JVMTI spec basically > >>>>> emitting JVMTI_EVENT_THREAD_START events in primordial phase while the > >>>>> spec states that it should be emitted only in either start or live > >>>>> phase. > >>>>> > >>>>> But if I am reading this correctly > >>>>> (https://hg.openjdk.java.net/jdk8u/jdk8udev/hotspot/file/02b4fd2f9041/src/share/vm/prims/jvmtiEventController.cpp#l418) > >>>>> the spec is also violated there. > >>>>> > >>>>> EARLY_EVENT_BITS are including THREAD_START_BIT and it is not filtered > >>>>> out by THREAD_FILTERED_EVENT_BITS so it seems perfectly valid for > >>>>> JVMTI_EVENT_THREAD_START to appear during PRIMORDIAL phase. > >>>>> > >>>>> So I would like to get clarification before going in and start > >>>>> tinkering with the JVM initialization code order. > >>>>> > >>>>> Thanks! > >>>>> > >>>>> -JB- > >>>> Hi Jaroslav! > >>>> > >>>> It has been a long time my friend... :-) > >>>> > >>>> It looks like that change was made by this Jigsaw change: > >>>> > >>>> JDK-8072745 Re-examine JVM TI execution phases and expectations > >>>> on what is allowed in each phase > >>>> https://bugs.openjdk.java.net/browse/JDK-8072745 > >>>> > >>>> These diffs in the bug reported jumped out at me: > >>>> > >>>> @@ -1022,6 +1032,10 @@ > >>>> void JvmtiExport::post_thread_start(JavaThread *thread) { > >>>> assert(thread->thread_state() == _thread_in_vm, "must be in vm state"); > >>>> > >>>> + if (JvmtiEnv::get_phase() <= JVMTI_PHASE_PRIMORDIAL) { > >>>> + return; > >>>> + } > >>>> + > >>>> EVT_TRIG_TRACE(JVMTI_EVENT_THREAD_START, ("JVMTI [%s] Trg Thread Start event triggered", > >>>> JvmtiTrace::safe_get_thread_name(thread))); > >>>> > >>>> @@ -1050,6 +1064,10 @@ > >>>> > >>>> > >>>> void JvmtiExport::post_thread_end(JavaThread *thread) { > >>>> + if (JvmtiEnv::get_phase() <= JVMTI_PHASE_PRIMORDIAL) { > >>>> + return; > >>>> + } > >>>> + > >>>> EVT_TRIG_TRACE(JVMTI_EVENT_THREAD_END, ("JVMTI [%s] Trg Thread End event triggered", > >>>> JvmtiTrace::safe_get_thread_name(thread))); > >>>> > >>>> > >>>> Here's the order of phases from jvmti.h: > >>>> > >>>> /* Phases of execution */ > >>>> > >>>> typedef enum { > >>>> JVMTI_PHASE_ONLOAD = 1, > >>>> JVMTI_PHASE_PRIMORDIAL = 2, > >>>> JVMTI_PHASE_START = 6, > >>>> JVMTI_PHASE_LIVE = 4, > >>>> JVMTI_PHASE_DEAD = 8 > >>>> } jvmtiPhase; > >>>> > >>>> However, the above diffs prevent the JVMTI_EVENT_THREAD_START > >>>> and JVMTI_EVENT_THREAD_END events from being posted earlier than > >>>> the JVMTI_PHASE_START phase... Unfortunately, I can't find the > >>>> actual changeset pushed to the Jigsaw report for JDK-8072745. > >>>> > >>>> > >>>> When you look at the current JvmtiExport::post_thread_start() > >>>> and JvmtiExport::post_thread_end() functions, you can see this: > >>>> > >>>> 1: void JvmtiExport::post_thread_start(JavaThread *thread) { > >>>> 36508: if (JvmtiEnv::get_phase() < JVMTI_PHASE_PRIMORDIAL) { > >>>> 36508: return; > >>>> 36508: } > >>>> > >>>> 1: void JvmtiExport::post_thread_end(JavaThread *thread) { > >>>> 36508: if (JvmtiEnv::get_phase() < JVMTI_PHASE_PRIMORDIAL) { > >>>> 36508: return; > >>>> 36508: } > >>>> > >>>> $ hg log -r 36508 > >>>> changeset: 36508:5f9eee6b383b > >>>> user: alanb > >>>> date: Thu Mar 17 19:04:01 2016 +0000 > >>>> summary: 8142968: Module System implementation > >>>> > >>>> > >>>> So the Jigsaw integration pushed a slightly different change > >>>> than is shown in JDK-8072745. I don't know whether the fix > >>>> for JDK-8072745 evolved before being pushed to the Jigsaw > >>>> repos or whether another JBS issue was used to modify the > >>>> code after JDK-8072745 was pushed. > >>>> > >>>> Short version: The semantics of the JVMTI_EVENT_THREAD_START > >>>> and JVMTI_EVENT_THREAD_END were changed by Jigsaw, but the > >>>> JVM/TI spec was not updated to reflect those changes. > >>>> > >>>> I've filed a new bug to track this issue: > >>>> > >>>> JDK-8249158 THREAD_START and THREAD_END event posting changed > >>>> by modules without updating the spec > >>>> https://bugs.openjdk.java.net/browse/JDK-8249158 > >>>> > >>>> Dan > From jcbeyler at google.com Tue Jul 14 17:39:34 2020 From: jcbeyler at google.com (Jean Christophe Beyler) Date: Tue, 14 Jul 2020 13:39:34 -0400 Subject: RFR (S) 8247615: Initialize the bytes left for the heap sampler In-Reply-To: References: Message-ID: Ping on this to get a final review :) Webrev: http://cr.openjdk.java.net/~jcbeyler/8247615/webrev.01/ The bug is here: https://bugs.openjdk.java.net/browse/JDK-8247615 Thanks! Jc On Thu, Jul 9, 2020 at 12:03 AM Jean Christophe Beyler wrote: > Hello all, > > Any way to get two reviews of the new version: > http://cr.openjdk.java.net/~jcbeyler/8247615/webrev.01/ > > Thanks and have a great evening! > Jc > > On Tue, Jun 30, 2020 at 12:47 AM Martin Buchholz > wrote: > >> Looks good to me, but of course I'm not qualified to review. >> >> On Mon, Jun 29, 2020 at 3:26 PM Jean Christophe Beyler >> wrote: >> > >> > Agreed; missed a hg qrefresh; link is now updated: >> > http://cr.openjdk.java.net/~jcbeyler/8247615/webrev.01/ >> > >> > :) >> > Jc >> > >> > On Mon, Jun 29, 2020 at 2:28 PM Man Cao wrote: >> >> >> >> Looks good. >> >> >> >> > though adding the change that Man wants might make it more flaky so >> I added your numThreads / 2 in case >> >> I don't see the "numThreads / 2" in webrev.01 though. No need for a >> webrev for this fix. >> >> >> >> -Man >> >> >> >> >> >> On Mon, Jun 29, 2020 at 1:10 PM Jean Christophe Beyler < >> jcbeyler at google.com> wrote: >> >>> >> >>> Hi all, >> >>> >> >>> Sorry it took time to get back to this; could I get a new review from: >> >>> http://cr.openjdk.java.net/~jcbeyler/8247615/webrev.01/ >> >>> >> >>> The bug is here: >> >>> https://bugs.openjdk.java.net/browse/JDK-8247615 >> >>> >> >>> Note, this passed the submit repo testing. >> >>> >> >>> Thanks and have a great day! >> >>> Jc >> >>> >> >>> Ps: explicit inlined Acks/Done are below: >> >>> >> >>> Sorry it took time to get back to this: >> >>> @Martin: >> >>> - done the typo >> >>> - about the sampling test: No you won't get samples due to how the >> system is done, since we know we only will be allocating one object for the >> thread, it dies out before a sample is required... though adding the change >> that Man wants might make it more flaky so I added your numThreads / 2 in >> case >> >>> - done for the always in the description >> >>> >> >>> >> >>> On Thu, Jun 25, 2020 at 6:54 PM Derek Thomson >> wrote: >> >>>> >> >>>> > It could also avoid the problem where every thread >> deterministically allocates the same object at 512K, although this is >> unlikely. >> >>>> >> >>>> I've recently discovered that with certain server frameworks that >> this actually becomes quite likely! So I'd strongly recommend using >> pick_next_sample. >> >>> >> >>> >> >>> Ack, done :) >> >>> >> >>>> >> >>>> >> >>>> On Thu, Jun 25, 2020 at 4:56 PM Man Cao wrote: >> >>>>> >> >>>>> Thanks for fixing this! >> >>>>> >> >>>>> > 53 ThreadHeapSampler() : >> _bytes_until_sample(get_sampling_interval()) { >> >>>>> >> >>>>> Does this work better? (It has to be done after the initialization >> of _rnd.) >> >>>>> _bytes_until_sample = pick_next_sample(); >> >>>>> >> >>>>> It could avoid completely missing to sample the first 512K >> allocation. >> >>>>> It could also avoid the problem where every thread >> >>> >> >>> >> >>> Done. >> >>> >> >>> >> >>>>> >> >>>>> deterministically allocates the same object at 512K, although this >> is unlikely. >> >>>>> >> >>>>> -Man >> >>> >> >>> >> >>> >> >>> -- >> >>> >> >>> Thanks, >> >>> Jc >> > >> > >> > >> > -- >> > >> > Thanks, >> > Jc >> > > > -- > > Thanks, > Jc > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From igor.ignatyev at oracle.com Tue Jul 14 18:18:22 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 14 Jul 2020 11:18:22 -0700 Subject: RFR [15] : 8249028 : clean up FileInstaller $test.src $cwd in vmTestbase_nsk_monitoring tests In-Reply-To: <54a61c0c-1002-2dfd-d300-e4e11b1f0cf1@oracle.com> References: <54a61c0c-1002-2dfd-d300-e4e11b1f0cf1@oracle.com> Message-ID: <3086DC5B-5509-44A1-8467-A79D4521A04A@oracle.com> Thanks David, pushed to jdk15. -- Igor > On Jul 13, 2020, at 7:47 PM, David Holmes wrote: > > Looks good! > > Thanks, > David > > On 9/07/2020 3:21 am, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev/8249028/webrev.00/ >>> 547 lines changed: 0 ins; 361 del; 186 mod; >> Hi all, >> could you please review the patch which removes `FileInstaller . .` jtreg action from :vmTestbase_nsk_monitoring tests? >> from the main issue(8204985): >>> all vmTestbase tests have '@run driver jdk.test.lib.FileInstaller . .' to mimic old test harness behavior and copy all files from a test source directory to a current work directory. some tests depend on this step, so we need 1st identify such tests and then either rewrite them not to have this dependency or leave FileInstaller only in these tests. >> effectively, the patch is just `ag -l '@run driver jdk.test.lib.FileInstaller . .' vmTestbase/nsk/monitoring | xargs -I{} gsed -i '/@run driver jdk.test.lib.FileInstaller \. \./d' {}` >> testing: :vmTestbase_nsk_monitoring on linux-x64 >> webrev: http://cr.openjdk.java.net/~iignatyev//8249018/webrev.00 >> JBS: https://bugs.openjdk.java.net/browse/JDK-8249028 >> Thanks, >> -- Igor From igor.ignatyev at oracle.com Tue Jul 14 18:26:15 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 14 Jul 2020 11:26:15 -0700 Subject: RFR [15] : 8249035 : clean up FileInstaller $test.src $cwd in vmTestbase_nsk_jdwp tests In-Reply-To: References: Message-ID: <6BD2F991-82C4-4F1C-AAE2-056D8CC72E3B@oracle.com> Thanks David, pushed to jdk15. -- Igor > On Jul 13, 2020, at 7:51 PM, David Holmes wrote: > > Looks good! > > Thanks, > David > > On 14/07/2020 8:40 am, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev/8249035/webrev.00/ >>> 251 lines changed: 0 ins; 128 del; 123 mod; >> Hi all, >> could you please review this clean-up which removes `FileInstaller` actions from : vmTestbase_nsk_jdwp tests? >> from the main issue(8204985): >>> all vmTestbase tests have '@run driver jdk.test.lib.FileInstaller . .' to mimic old test harness behavior and copy all files from a test source directory to a current work directory. some tests depend on this step, so we need 1st identify such tests and then either rewrite them not to have this dependency or leave FileInstaller only in these tests. >> the majority of the patch is created by `ag -l '@run driver jdk.test.lib.FileInstaller . .' vmTestbase/nsk/jdwp xargs -I{} gsed -i '/@run driver jdk.test.lib.FileInstaller \. \./d' {}`. only two tests needed FileInstaller: VirtualMachine/RedefineClasses/redefinecls001 and Method/IsObsolete/isobsolete002. both tests have been updated to use aux class nsk.share.ExtraClassesBuilder, which search for source files in test.src (as opposed to current directory) instead of calling javac by ExecDriver; b/c n.s.ExtraClassesBuilder puts compiled classes into ./bin/newclass, the test code had to be slightly updated to search for classfiles in bin/newclass instead of newclass. >> JBS: https://bugs.openjdk.java.net/browse/JDK-8249035 >> webrev: http://cr.openjdk.java.net/~iignatyev//8249035/webrev.00/ >> testing: :vmTestbase_nsk_jdwp on linux-x64 >> Thanks, >> -- Igor From chris.plummer at oracle.com Tue Jul 14 18:56:47 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 14 Jul 2020 11:56:47 -0700 Subject: RFR: 8242891: vmTestbase/nsk/jvmti/ test should be fixed to fail early if JVMTI function return error In-Reply-To: <01076F65-C037-4308-9782-F3348274C858@oracle.com> References: <11314027-4965-b38b-6bc7-5011515b94ab@oracle.com> <2cf4e45a-4d44-3c0a-a272-480f56a5e6e8@oracle.com> <3927ae7c-efa9-eb9f-ab98-18d778d5a966@oracle.com> <547f02e9-604c-09e5-5fe1-2afb1be54f2d@oracle.com> <611D0AF5-1295-4DD4-B6C9-F4D379AA069C@oracle.com> <01076F65-C037-4308-9782-F3348274C858@oracle.com> Message-ID: <3a3a6988-e4e7-4677-2776-ec5f1eeafd4e@oracle.com> An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Tue Jul 14 19:01:24 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 14 Jul 2020 12:01:24 -0700 Subject: RFR(M): 8247272: SA ELF file support has never worked for 64-bit causing address to symbol name mapping to fail In-Reply-To: References: <338118cb-37ff-9d03-f22e-c96322bfd7a6@oracle.com> <54ca0720-61f2-4580-80e5-b2a0da966cc0@oracle.com> <9d743570-7e55-df53-abfc-77a62af4d136@oracle.com> Message-ID: <8279763d-65de-b3d6-beb5-e40ff94b2804@oracle.com> Hello, Can I get a second reviewer please. As noted below, it's easier to look at webrev.00 first to just see the coding changes. webrev.01 just adds some updated comments. thanks, Chris On 7/8/20 2:07 PM, Kevin Walls wrote: > Thanks Chris, it's a bit of clutter, but truthful clutter. 8-) > > > On 08/07/2020 20:26, Chris Plummer wrote: >> Webrev has been updated with the suggested comment changes. Note to >> new reviewers, look in webrev.00 first since it doesn't have the >> clutter of the comment changes, making it easier to see which lines >> actually have code changes. >> >> http://cr.openjdk.java.net/~cjplummer/8247272/webrev.01/index.html >> >> thanks, >> >> Chris >> >> On 7/8/20 11:04 AM, Chris Plummer wrote: >>> Hi Kevin, >>> >>> Thanks for the review. I'll add the additional Elf64_Addr and >>> Elf64_Off comments. Probably the others should be updated too. >>> Although they are the same size, they do have different names. For >>> example: >>> >>> /* Type for a 16-bit quantity.? */ >>> typedef uint16_t Elf32_Half; >>> typedef uint16_t Elf64_Half; >>> >>> thanks, >>> >>> Chris >>> >>> On 7/8/20 3:47 AM, Kevin Walls wrote: >>>> Hi Chris -- >>>> >>>> This is a great story/history lesson. >>>> >>>> You could if you like, edit those comments in ElfFileParser.java so >>>> "Elf32_Addr" as they will contain either "Elf64_Addr or >>>> Elf32_Addr", similarly Elf64_Off.? The other Elf64 fields are the >>>> same as the 32 bit ones. >>>> >>>> Yes, the symbol fields are ordered differently. >>>> >>>> So all looks good to me! >>>> >>>> Thanks >>>> Kevin >>>> >>>> >>>> >>>> On 08/07/2020 07:20, Chris Plummer wrote: >>>>> Hello, >>>>> >>>>> Please help review the following: >>>>> >>>>> http://cr.openjdk.java.net/~cjplummer/8247272/webrev.00/index.html >>>>> https://bugs.openjdk.java.net/browse/JDK-8247272 >>>>> >>>>> The short story is that SA address to native symbol name >>>>> mapping/lookup has never worked on 64-bit, and this is due to the >>>>> java level ELF file support only supporting 32-bit. This CR fixes >>>>> that, and I believe also maintains 32-bit compatibility, although >>>>> I have no way of testing that. >>>>> >>>>> There is more to the story however on how we got here. Before >>>>> going into the gory detail below, I just want to point out that >>>>> currently nothing is using this support, and therefore it is >>>>> technically not fixing anything, although I did verify that the >>>>> fixes work (see details below). Also, I intend to remove all the >>>>> java level ELF file support as part of JDK-8247516 [1]. The only >>>>> reason I want to push these changes first is because I already did >>>>> the work to get it working with 64-bit, and would like to get it >>>>> archived before removing it in case for some reason it is revived >>>>> in the future. >>>>> >>>>> Now for the ugly details on how we got here (and you really don't >>>>> need to read this unless you have any concerns with what I stated >>>>> above). It starts with the clhsdb "whatis" command, which was the >>>>> only (indirect) user of this java level ELF file support. It's >>>>> implementation is in javascript, so we have not had access to it >>>>> ever since JDK9 module support broke the SA javascript support >>>>> (and javascript support is now removed). I started the process of >>>>> converting "whatis" to java. It is basically the same as the >>>>> clhsdb "findpc" command, except it also checks for native symbols, >>>>> which it does with the following code: >>>>> >>>>> ? var dso = loadObjectContainingPC(addr); >>>>> ? var sym = dso.closestSymbolToPC(addr); >>>>> ? return sym.name + '+' + sym.offset; >>>>> >>>>> Converting this to java was trivial. I just stuck support for it >>>>> in the PointerFinder class, which is what findpc relies on. >>>>> However, it always failed to successfully lookup a symbol. I found >>>>> that DSO.closestSymbolToPC() called into the java level ELF >>>>> support, and that was failing badly. After some debugging I >>>>> noticed that the values read in for various ELF headers were >>>>> mostly garbage. It then occurred to me that it was reading in >>>>> 32-bit values that probably needed to be 64-bit. Sure enough, this >>>>> code was never converted to 64-bit support. I then went and tried >>>>> "whatis" on JDK8, the last version where it was available, and it >>>>> failed there also with 64-bit binaries. So this is why I initially >>>>> fixed it to work with 64-bit, and also how I tested it (using the >>>>> modified findpc on a native symbol). But the story continues... >>>>> >>>>> DSO.java, and as a consequence the java ELF file support, is used >>>>> by all our posix ports to do address to symbol lookups. So I >>>>> figured that after fixing the java level ELF file support for >>>>> 64-bit, my improved findpc would start working on OSX also. No >>>>> such luck, and for obvious reasons. OSX uses mach-o files. This >>>>> ELF code should never have been used for it, and of course has >>>>> never worked. >>>>> >>>>> So I was left trying to figure out how to do OSX address to native >>>>> symbol lookups. I then recalled that there was a >>>>> CFrame.closestSymbolToPC() API that did address to native symbol >>>>> lookups for native stack traces, and wondered how it was ever >>>>> working (even on linux with the broken ELF 64-bit support). It >>>>> turns out this takes a very different path to do the lookups, >>>>> ending up in native code in libsaproc, where we also have ELF file >>>>> support. I then converted DSO.closestSymbolToPC(addr) to use this >>>>> libsaproc code instead, and it worked fine. So now there was no >>>>> need for the java level ELF file support since its only user was >>>>> DSO.closestSymbolToPC(addr). I should also add that this is the >>>>> approach that has always been used on windows, with both >>>>> CFrame.closestSymbolToPC() and DSO.closestSymbolToPC(addr) using >>>>> the same libsaproc support. >>>>> >>>>> There is still a bit more to the story. After diverting >>>>> DSO.closestSymbolToPC(addr) to the libsaproc lookup code, it still >>>>> didn't work for OSX. I thought it would just work since the native >>>>> BsdDebuggerLocal.lookupByName0() is implemented, and it seems to >>>>> trickle down to the proper lower level APIs to find the symbol, >>>>> but there were two issues. The first is that for processes there >>>>> is no support for looking up all the libraries and populating the >>>>> list of ps_prochandle structures that are used to do the symbol >>>>> lookups. This was just never implemented (also is why PMap does >>>>> not work for OSX processes). For core files the ps_prochandle >>>>> structs are there, but the lookup code was badly broken. That has >>>>> now been fixed by JDK-8247515 [2], currently out for review. So >>>>> the end result is we'll have address to native symbol lookup for >>>>> everything but OSX processes. >>>>> >>>>> If? your still here, thanks for listening! >>>>> >>>>> Chris >>>>> >>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8247516 >>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8247515 >>>>> >>>>> >>>>> >>>>> >>> >>> >> >> From chris.plummer at oracle.com Tue Jul 14 19:01:41 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 14 Jul 2020 12:01:41 -0700 Subject: RFR(XS): 8248879: SA core file support on OSX has some bugs trying to locate the jvm libraries In-Reply-To: <6089f692-152b-8615-dfca-779f2b5028a0@oracle.com> References: <6089f692-152b-8615-dfca-779f2b5028a0@oracle.com> Message-ID: <219dd806-ca93-64ce-f5f9-d75dd7acb7f1@oracle.com> Ping! On 7/6/20 9:31 PM, Chris Plummer wrote: > Hello, > > Please help review the following: > > http://cr.openjdk.java.net/~cjplummer/8248879/webrev.00/index.html > https://bugs.openjdk.java.net/browse/JDK-8248879 > > The description of the problem and the fix are both in the CR. > > thanks, > > Chris From chris.plummer at oracle.com Tue Jul 14 19:04:28 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 14 Jul 2020 12:04:28 -0700 Subject: RFR(XS): 8248878: SA: Implement simple workaround for JDK-8248876 In-Reply-To: <1fde7056-f2d0-e256-ae6d-b8db94068c71@oss.nttdata.com> References: <39ac40d6-a738-3214-d86c-412152a631d4@oracle.com> <8e89d223-0f65-f26e-3bd7-2ca7421a940b@oracle.com> <6d2610d6-5ca1-8455-ddc6-d7f02b21c0d5@oracle.com> <046aced2-1d23-d096-748b-4ce0bfd01587@oracle.com> <1fde7056-f2d0-e256-ae6d-b8db94068c71@oss.nttdata.com> Message-ID: <099b6b7d-f3c2-57d4-832a-7144b51eecb7@oracle.com> Hello, Can I get a second reviewer please. Yasumasa has done the first review. The discussion below was all about how to possibly fix JDK-8248876, but for now we're sticking with this workaround. http://cr.openjdk.java.net/~cjplummer/8248878/webrev.00/index.html https://bugs.openjdk.java.net/browse/JDK-8248878 The explanation of the fix is in the CR. The parent CR, JDK-8248876 [1], explains the issue being addressed. There's no test for this fix yet. It requires the changes I'm making for JDK-8247514 [2], which include changes to "findpc" support and the ClhsdbFindPC.java test that trigger this issue. [1] https://bugs.openjdk.java.net/browse/JDK-8248876 [2] https://bugs.openjdk.java.net/browse/JDK-8247514 thanks, Chris On 7/13/20 4:48 PM, Yasumasa Suenaga wrote: > Sure, I reviewed for this workaround (JDK-8248878). > > I'm still think we can use note section in the core for JDK-8248876 as > I said, but I'm not sure. > > > Thanks, > > Yasumasa > > > On 2020/07/14 3:12, Chris Plummer wrote: >> Hi Yasumasa, >> >> If you have no further suggestions on how to fix JDK-8248876, I'd >> like to proceed with this work around. Can I considered it reviewed >> by you? >> >> thanks, >> >> Chris >> >> On 7/7/20 7:29 PM, Chris Plummer wrote: >>> Hi Yasumasa, >>> >>> The executable is not opened with pathmap_open: >>> >>> ? if ((ph->core->exec_fd = open(exec_file, O_RDONLY)) < 0) { >>> >>> I think pathmap_open() is just used for libraries. >>> >>> thanks, >>> >>> Chris >>> >>> On 7/7/20 6:18 PM, Yasumasa Suenaga wrote: >>>> Hi Chris, >>>> >>>> SA would use `link_map` to decide to load address, but it does not >>>> seem to contain executable. >>>> I set breakpoint to pathmap_open() and I watched the argument of >>>> it, then I didn't see any executable (`java`) on it. >>>> Maybe current implementation is broken. >>>> >>>> I guess we can use note section in the core for deciding loading >>>> address. >>>> I can see valid address (includes executable) from `readelf -n`. >>>> Of course it might be big change for SA... >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2020/07/07 15:38, Chris Plummer wrote: >>>>> Hi Yasumasa, >>>>> >>>>> Thanks for the review. I tried the following for line 188: >>>>> >>>>> ???? if ((phdr->p_type == PT_LOAD || phdr->p_type == PT_INTERP) && >>>>> phdr->p_vaddr < baseaddr) { >>>>> >>>>> However, "base" still ended up being 0. I added some printfs. For >>>>> the exec file there is both a PT_INTER with p_vaddr of 0x238 and a >>>>> PT_LOAD with p_vaddr 0. I'm not sure which to use, but in either >>>>> case that won't be the proper base when added to 0: >>>>> >>>>> ?? if (add_lib_info_fd(ph, exec_file, ph->core->exec_fd, >>>>> ?????????????????????? (uintptr_t)0 + >>>>> find_base_address(ph->core->exec_fd, &exec_ehdr)) == NULL) { >>>>> ???? goto err; >>>>> ?? } >>>>> >>>>> So maybe it's the (uintptr_t)0 that is the problem here. For >>>>> shared libs instead of 0 it computes the value to add: >>>>> >>>>> ??????????????? if (lib_base_diff == ZERO_LOAD_ADDRESS ) { >>>>> ????????????????? lib_base_diff = calc_prelinked_load_address(ph, >>>>> lib_fd, &elf_ehdr, link_map_addr); >>>>> ????????????????? if (lib_base_diff == INVALID_LOAD_ADDRESS) { >>>>> ??????????????????? close(lib_fd); >>>>> ??????????????????? return false; >>>>> ????????????????? } >>>>> ??????????????? } >>>>> >>>>> ??????????????? lib_base = lib_base_diff + >>>>> find_base_address(lib_fd, &elf_ehdr); >>>>> >>>>> So in this case we've actually computed lib_base_diff rather than >>>>> just assumed 0. >>>>> >>>>> Chris >>>>> >>>>> On 7/6/20 10:46 PM, Yasumasa Suenaga wrote: >>>>>> Hi Chris, >>>>>> >>>>>> Your change looks good. >>>>>> >>>>>> >>>>>> BTW I saw JDK-8248876. I'm not sure, but I guess we can fix this >>>>>> issue if we allow PT_INTERP in L118: >>>>>> >>>>>> ``` >>>>>> 105 uintptr_t find_base_address(int fd, ELF_EHDR* ehdr) { >>>>>> ????????????????? : >>>>>> 115?? // the base address of a shared object is the lowest vaddr of >>>>>> 116?? // its loadable segments (PT_LOAD) >>>>>> 117?? for (phdr = phbuf, cnt = 0; cnt < ehdr->e_phnum; cnt++, >>>>>> phdr++) { >>>>>> 118???? if (phdr->p_type == PT_LOAD && phdr->p_vaddr < baseaddr) { >>>>>> 119?????? baseaddr = phdr->p_vaddr; >>>>>> 120???? } >>>>>> 121?? } >>>>>> ``` >>>>>> >>>>>> /proc//maps shows top of `java` is 0x56543b9df000: >>>>>> >>>>>> 56543b9df000-56543b9e0000 r--p 00000000 08:10 55770 >>>>>> /usr/lib/jvm/java-11-openjdk-amd64/bin/java >>>>>> >>>>>> >>>>>> `i target` on GDB shows 0x56543b9df000 is .interp section: >>>>>> >>>>>> Local exec file: >>>>>> ??????? `/usr/lib/jvm/java-11-openjdk-amd64/bin/java', file type >>>>>> elf64-x86-64. >>>>>> ??????? Entry point: 0x56543b9e0330 >>>>>> ??????? 0x000056543b9df318 - 0x000056543b9df334 is .interp >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2020/07/07 13:18, Chris Plummer wrote: >>>>>>> Hello, >>>>>>> >>>>>>> Please help review the following: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~cjplummer/8248878/webrev.00/index.html >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8248878 >>>>>>> >>>>>>> The explanation of the fix is in the CR. The parent CR, >>>>>>> JDK-8248876 [1], explains the issue being addressed. >>>>>>> >>>>>>> There's no test for this fix yet. It requires the changes I'm >>>>>>> making for JDK-8247514 [2], which include changes to "findpc" >>>>>>> support and the ClhsdbFindPC.java test that trigger this issue. >>>>>>> >>>>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8248876 >>>>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8247514 >>>>>>> >>>>>>> thanks, >>>>>>> >>>>>>> Chris >>>>> >>> >> >> From alexey.menkov at oracle.com Tue Jul 14 20:11:43 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Tue, 14 Jul 2020 13:11:43 -0700 Subject: RFR(XS): 8248879: SA core file support on OSX has some bugs trying to locate the jvm libraries In-Reply-To: <219dd806-ca93-64ce-f5f9-d75dd7acb7f1@oracle.com> References: <6089f692-152b-8615-dfca-779f2b5028a0@oracle.com> <219dd806-ca93-64ce-f5f9-d75dd7acb7f1@oracle.com> Message-ID: <37eee0d3-6b00-eb11-6198-d8d87e3a7ad4@oracle.com> Hi Chris, I think it would be better to use strrstr to correctly handle paths like /something/bin/jdk/bin/jhsdb And I'd updated 358 char* posbin = strstr(execname, "/bin/java"); to use strrstr as well --alex On 07/14/2020 12:01, Chris Plummer wrote: > Ping! > > On 7/6/20 9:31 PM, Chris Plummer wrote: >> Hello, >> >> Please help review the following: >> >> http://cr.openjdk.java.net/~cjplummer/8248879/webrev.00/index.html >> https://bugs.openjdk.java.net/browse/JDK-8248879 >> >> The description of the problem and the fix are both in the CR. >> >> thanks, >> >> Chris > From chris.plummer at oracle.com Tue Jul 14 20:37:59 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 14 Jul 2020 13:37:59 -0700 Subject: RFR(XS): 8248879: SA core file support on OSX has some bugs trying to locate the jvm libraries In-Reply-To: <37eee0d3-6b00-eb11-6198-d8d87e3a7ad4@oracle.com> References: <6089f692-152b-8615-dfca-779f2b5028a0@oracle.com> <219dd806-ca93-64ce-f5f9-d75dd7acb7f1@oracle.com> <37eee0d3-6b00-eb11-6198-d8d87e3a7ad4@oracle.com> Message-ID: Ok. I'll change both references to use strrstr. thanks, Chris On 7/14/20 1:11 PM, Alex Menkov wrote: > Hi Chris, > > I think it would be better to use strrstr to correctly handle paths like > /something/bin/jdk/bin/jhsdb > > And I'd updated > 358?? char* posbin = strstr(execname, "/bin/java"); > to use strrstr as well > > --alex > > On 07/14/2020 12:01, Chris Plummer wrote: >> Ping! >> >> On 7/6/20 9:31 PM, Chris Plummer wrote: >>> Hello, >>> >>> Please help review the following: >>> >>> http://cr.openjdk.java.net/~cjplummer/8248879/webrev.00/index.html >>> https://bugs.openjdk.java.net/browse/JDK-8248879 >>> >>> The description of the problem and the fix are both in the CR. >>> >>> thanks, >>> >>> Chris >> From chris.plummer at oracle.com Tue Jul 14 20:43:26 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 14 Jul 2020 13:43:26 -0700 Subject: RFR(XS): 8248879: SA core file support on OSX has some bugs trying to locate the jvm libraries In-Reply-To: References: <6089f692-152b-8615-dfca-779f2b5028a0@oracle.com> <219dd806-ca93-64ce-f5f9-d75dd7acb7f1@oracle.com> <37eee0d3-6b00-eb11-6198-d8d87e3a7ad4@oracle.com> Message-ID: <99909975-074c-34a1-0f9b-69c4fb8f0eff@oracle.com> Actually it's not so easy. I don't see any other references to strrstr in our source. When I reference strstr, it gives a warning because it's not declared. The only man page I can find says to include sstring2.h, but this file does not exist. It also says to link with -lsstrings2. Chris On 7/14/20 1:37 PM, Chris Plummer wrote: > Ok. I'll change both references to use strrstr. > > thanks, > > Chris > > On 7/14/20 1:11 PM, Alex Menkov wrote: >> Hi Chris, >> >> I think it would be better to use strrstr to correctly handle paths like >> /something/bin/jdk/bin/jhsdb >> >> And I'd updated >> 358?? char* posbin = strstr(execname, "/bin/java"); >> to use strrstr as well >> >> --alex >> >> On 07/14/2020 12:01, Chris Plummer wrote: >>> Ping! >>> >>> On 7/6/20 9:31 PM, Chris Plummer wrote: >>>> Hello, >>>> >>>> Please help review the following: >>>> >>>> http://cr.openjdk.java.net/~cjplummer/8248879/webrev.00/index.html >>>> https://bugs.openjdk.java.net/browse/JDK-8248879 >>>> >>>> The description of the problem and the fix are both in the CR. >>>> >>>> thanks, >>>> >>>> Chris >>> > > From leonid.mesnik at oracle.com Tue Jul 14 21:15:19 2020 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Tue, 14 Jul 2020 14:15:19 -0700 Subject: RFR: 8242891: vmTestbase/nsk/jvmti/ test should be fixed to fail early if JVMTI function return error In-Reply-To: <3a3a6988-e4e7-4677-2776-ec5f1eeafd4e@oracle.com> References: <11314027-4965-b38b-6bc7-5011515b94ab@oracle.com> <2cf4e45a-4d44-3c0a-a272-480f56a5e6e8@oracle.com> <3927ae7c-efa9-eb9f-ab98-18d778d5a966@oracle.com> <547f02e9-604c-09e5-5fe1-2afb1be54f2d@oracle.com> <611D0AF5-1295-4DD4-B6C9-F4D379AA069C@oracle.com> <01076F65-C037-4308-9782-F3348274C858@oracle.com> <3a3a6988-e4e7-4677-2776-ec5f1eeafd4e@oracle.com> Message-ID: <4D92A17C-4D04-40B2-8B85-6F097B4EE2FA@oracle.com> Hi Thank you, I agree, it is safer to return. Leonid > On Jul 14, 2020, at 11:56 AM, Chris Plummer wrote: > > Hi Leonid, > > In allthr001.cpp I question if it is safe to proceed after the following error: > > 205 if (threadsCount - num_unexpected != thrInfo[ind].cnt + sys_cnt) { > 206 printf("Point %d: number of threads expected: %d, got: %d\n", > 207 ind, thrInfo[ind].cnt + sys_cnt, threadsCount - num_unexpected); > 208 result = STATUS_FAILED; > 209 } > > It seems it would be much safer to return on error, and even if not really necessary, there is not much point in continuing with the test code that follows. > > The rest of the changes look good. I don't need to see another webrev. > > thanks, > > Chris > > On 7/13/20 3:42 PM, Leonid Mesnik wrote: >> Could I have 2nd reviewer? >> >> Leonid >> >>> On Jun 25, 2020, at 10:58 AM, Leonid Mesnik > wrote: >>> >>> Ping >>> >>>> On Jun 12, 2020, at 4:18 PM, Leonid Mesnik > wrote: >>>> >>>> Fixed all places, updated copyright. Still need second review >>>> >>>> http://cr.openjdk.java.net/~lmesnik/8242891/webrev.02/ >>>> Leonid >>>> >>>> On 6/11/20 8:41 PM, serguei.spitsyn at oracle.com wrote: >>>>> Hi Leonid, >>>>> >>>>> It is much better now. >>>>> >>>>> Several places still need the same fix. >>>>> >>>>> http://cr.openjdk.java.net/~lmesnik/8242891/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetAllThreads/allthr001/allthr001.cpp.frames.html >>>>> >>>>> 211 for (i = 0; i < thrInfo[ind].cnt; i++) { >>>>> 212 for (j = 0, found = 0; j < threadsCount && !found; j++) { >>>>> 213 err = jvmti->GetThreadInfo(threads[j], &inf); >>>>> 214 if (err != JVMTI_ERROR_NONE) { >>>>> 215 printf("Failed to get thread info: %s (%d)\n", >>>>> 216 TranslateError(err), err); >>>>> 217 result = STATUS_FAILED; >>>>> 218 } >>>>> 219 if (printdump == JNI_TRUE) { >>>>> 220 printf(" >>> %s", inf.name); >>>>> 221 } >>>>> 222 found = (inf.name != NULL && >>>>> 223 strstr(inf.name, thrInfo[ind].thrNames[i]) == inf.name && >>>>> 224 (ind == 4 || strlen(inf.name) == >>>>> 225 strlen(thrInfo[ind].thrNames[i]))); >>>>> 226 } >>>>> A return is needed after line 217, otherwise the the inf value is used at lines 222-224. >>>>> >>>>> http://cr.openjdk.java.net/~lmesnik/8242891/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetBytecodes/bytecodes003/bytecodes003.cpp.frames.html >>>>> >>>>> A return is needed for the errors: >>>>> 363 result = STATUS_FAILED; >>>>> 372 result = STATUS_FAILED; >>>>> 384 result = STATUS_FAILED; >>>>> >>>>> http://cr.openjdk.java.net/~lmesnik/8242891/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/MethodEntry/mentry001/mentry001.cpp.frames.html >>>>> >>>>> A return is needed for the errors: >>>>> 82 result = STATUS_FAILED; >>>>> 94 result = STATUS_FAILED; >>>>> 100 result = STATUS_FAILED; >>>>> >>>>> http://cr.openjdk.java.net/~lmesnik/8242891/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/MethodExit/mexit001/mexit001.cpp.frames.html >>>>> >>>>> A return is needed for the error: >>>>> 98 result = STATUS_FAILED; >>>>> >>>>> http://cr.openjdk.java.net/~lmesnik/8242891/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/MethodExit/mexit002/mexit002.cpp.frames.html >>>>> >>>>> A return is needed for the error: >>>>> 98 result = STATUS_FAILED; >>>>> >>>>> http://cr.openjdk.java.net/~lmesnik/8242891/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/RedefineClasses/redefclass019/redefclass019.cpp.frames.html >>>>> >>>>> A return is needed for the error: >>>>> 186 result = STATUS_FAILED; >>>>> >>>>> Also, I do not like many uninitialized locals in these tests. >>>>> But it is for another pass. >>>>> >>>>> Otherwise, it looks good. >>>>> No need for another webrev if you fix the above. >>>>> I hope, you will update copyright comments before push. >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>> On 6/11/20 15:30, Leonid Mesnik wrote: >>>>>> Agree, it would be better to don't try to use data from functions with error code. The new webrev: >>>>>> >>>>>> http://cr.openjdk.java.net/~lmesnik/8242891/webrev.01/ >>>>>> I tried to prevent any usage of possibly corrupted data. Mostly strings or allocated data, sometimes method/class id which are used my other JVMTI functions. >>>>>> >>>>>> Leonid >>>>>> >>>>>> On 6/9/20 6:59 PM, serguei.spitsyn at oracle.com wrote: >>>>>>> On 6/9/20 12:58, Leonid Mesnik wrote: >>>>>>>> Hi >>>>>>>> >>>>>>>> >>>>>>>> On 6/9/20 12:34 PM, serguei.spitsyn at oracle.com wrote: >>>>>>>>> Hi Leonid, >>>>>>>>> >>>>>>>>> Thank you for taking care about this! >>>>>>>>> It looks good in general. >>>>>>>>> However, I think, a similar return is needed in more cases. >>>>>>>>> >>>>>>>>> One example: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~lmesnik/8242891/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/jvmti/Exception/exception001/exception001.cpp.frames.html >>>>>>>>> >>>>>>>>> 99 err = jvmti_env->GetMethodDeclaringClass(method, &cls); >>>>>>>>> 100 if (err != JVMTI_ERROR_NONE) { >>>>>>>>> 101 printf("(GetMethodDeclaringClass#t) unexpected error: %s (%d)\n", >>>>>>>>> 102 TranslateError(err), err); >>>>>>>>> 103 result = STATUS_FAILED; >>>>>>>>> <> 104 return; >>>>>>>>> 105 } >>>>>>>>> 106 err = jvmti_env->GetClassSignature(cls, &ex.t_cls, &generic); >>>>>>>>> 107 if (err != JVMTI_ERROR_NONE) { >>>>>>>>> 108 printf("(GetClassSignature#t) unexpected error: %s (%d)\n", >>>>>>>>> 109 TranslateError(err), err); >>>>>>>>> 110 result = STATUS_FAILED; >>>>>>>>> 111 } >>>>>>>>> 112 err = jvmti_env->GetMethodName(method, >>>>>>>>> 113 &ex.t_name, &ex.t_sig, &generic); >>>>>>>>> 114 if (err != JVMTI_ERROR_NONE) { >>>>>>>>> 115 printf("(GetMethodName#t) unexpected error: %s (%d)\n", >>>>>>>>> 116 TranslateError(err), err); >>>>>>>>> 117 result = STATUS_FAILED; >>>>>>>>> 118 } >>>>>>>>> 119 ex.t_loc = location; >>>>>>>>> 120 err = jvmti_env->GetMethodDeclaringClass(catch_method, &cls); >>>>>>>>> 121 if (err != JVMTI_ERROR_NONE) { >>>>>>>>> 122 printf("(GetMethodDeclaringClass#c) unexpected error: %s (%d)\n", >>>>>>>>> 123 TranslateError(err), err); >>>>>>>>> 124 result = STATUS_FAILED; >>>>>>>>> <> 125 return; >>>>>>>>> 126 } >>>>>>>>> 127 err = jvmti_env->GetClassSignature(cls, &ex.c_cls, &generic); >>>>>>>>> 128 if (err != JVMTI_ERROR_NONE) { >>>>>>>>> 129 printf("(GetClassSignature#c) unexpected error: %s (%d)\n", >>>>>>>>> 130 TranslateError(err), err); >>>>>>>>> 131 result = STATUS_FAILED; >>>>>>>>> 132 } >>>>>>>>> 133 err = jvmti_env->GetMethodName(catch_method, >>>>>>>>> 134 &ex.c_name, &ex.c_sig, &generic); >>>>>>>>> 135 if (err != JVMTI_ERROR_NONE) { >>>>>>>>> 136 printf("(GetMethodName#c) unexpected error: %s (%d)\n", >>>>>>>>> 137 TranslateError(err), err); >>>>>>>>> 138 result = STATUS_FAILED; >>>>>>>>> 139 } >>>>>>>>> >>>>>>>>> In the fragment above you added return for JVMTI GetMethodDeclaringClass error. >>>>>>>>> But GetMethodName and GetClassSignature can be also problematic as the returned names are printed below. >>>>>>>>> It seems to be more safe and even simpler to add returns for such cases as well. >>>>>>>>> Otherwise, the code reader is puzzled why there is a return in one failure case and there is no such return in another. >>>>>>>> It is a good question if we want to fix such places or even fails with first JVMTI failure. (I even started to fix it in the such way but find that existing tests usually don't fail always). >>>>>>>> >>>>>>> >>>>>>> I do not suggest to fix all the tests but those which you are already fixing. >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> The difference is that test tries to reuse "cls" in other JVMTI function and going to generate very misleading crash. How it just tries to compare ex and exs values. So test might crash but clearly outside of JVMTI function and with some useful info. So I am not sure if fixing these lines improve test failure handling. >>>>>>>> >>>>>>> >>>>>>> If JVMTI functions fail with an error code the results with symbolic strings must be considered invalid. >>>>>>> However, they are used later (the values are printed). >>>>>>> It is better to bail out in such cases. >>>>>>> It should not be a problem to add similar returns in such cases. >>>>>>> Or do you think it is important to continue execution for some reason? >>>>>>> >>>>>>>> Assuming that most of existing tests fails early only if going to re-use possible corrupted data I propose to fix this separately. We need to figure out when to fail or to try to finish. >>>>>>>> >>>>>>> >>>>>>> Do you suggest it for the updated tests only or for all the tests with such problems? >>>>>>> >>>>>>> Thanks, >>>>>>> Serguei >>>>>>> >>>>>>>> >>>>>>>> Leonid >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Serguei >>>>>>>>> >>>>>>>>> >>>>>>>>> On 6/1/20 21:33, Leonid Mesnik wrote: >>>>>>>>>> Hi >>>>>>>>>> >>>>>>>>>> Could you please review following fix which stop test execution if JVMTI function returns error. The test fails anyway however using potentially bad data in JVMTI function might cause misleading crash failures. The hs_err will contains the stacktrace not with problem function but with function called with corrupted data. Most of tests already has such behavior but not all. Also I fixed a couple of tests to finish if they haven't managed to suspend thread. >>>>>>>>>> >>>>>>>>>> I've updated only tests which try to use corrupted data in JVMTI functions after errors. I haven't updated tests which just compare/print values from erroring JVMTI functions. The crash in strcmp/println is not so misleading and might be point to real issue. >>>>>>>>>> >>>>>>>>>> webrev: http://cr.openjdk.java.net/~lmesnik/8242891/webrev.00/ >>>>>>>>>> >>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8242891 >>>>>>>>>> >>>>>>>>>> Leonid >>>>>>>>>> >>>>>>>>> >>>>>>> >>>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexey.menkov at oracle.com Tue Jul 14 22:14:03 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Tue, 14 Jul 2020 15:14:03 -0700 Subject: RFR(XS): 8248879: SA core file support on OSX has some bugs trying to locate the jvm libraries In-Reply-To: <99909975-074c-34a1-0f9b-69c4fb8f0eff@oracle.com> References: <6089f692-152b-8615-dfca-779f2b5028a0@oracle.com> <219dd806-ca93-64ce-f5f9-d75dd7acb7f1@oracle.com> <37eee0d3-6b00-eb11-6198-d8d87e3a7ad4@oracle.com> <99909975-074c-34a1-0f9b-69c4fb8f0eff@oracle.com> Message-ID: <8c7edf32-3037-b8d7-a18e-587f7ae8d294@oracle.com> Yes, you are right. This is not a function from strings.h Ok, you can leave strstr (and keep in mind that the path can't contain "/bin/" other than jdk's bin) or implement the functionality. It should be something simple like static const char* rstrstr(const char *str, const char *sub) { const char *result = NULL; for (const char *p = strstr(str, sub); p != NULL; p = strstr(p + 1, sub)) { result = p; } return result; } --alex On 07/14/2020 13:43, Chris Plummer wrote: > Actually it's not so easy. I don't see any other references to strrstr > in our source. When I reference strstr, it gives a warning because it's > not declared. The only man page I can find says to include sstring2.h, > but this file does not exist. It also says to link with -lsstrings2. > > Chris > > On 7/14/20 1:37 PM, Chris Plummer wrote: >> Ok. I'll change both references to use strrstr. >> >> thanks, >> >> Chris >> >> On 7/14/20 1:11 PM, Alex Menkov wrote: >>> Hi Chris, >>> >>> I think it would be better to use strrstr to correctly handle paths like >>> /something/bin/jdk/bin/jhsdb >>> >>> And I'd updated >>> 358?? char* posbin = strstr(execname, "/bin/java"); >>> to use strrstr as well >>> >>> --alex >>> >>> On 07/14/2020 12:01, Chris Plummer wrote: >>>> Ping! >>>> >>>> On 7/6/20 9:31 PM, Chris Plummer wrote: >>>>> Hello, >>>>> >>>>> Please help review the following: >>>>> >>>>> http://cr.openjdk.java.net/~cjplummer/8248879/webrev.00/index.html >>>>> https://bugs.openjdk.java.net/browse/JDK-8248879 >>>>> >>>>> The description of the problem and the fix are both in the CR. >>>>> >>>>> thanks, >>>>> >>>>> Chris >>>> >> >> > > From serguei.spitsyn at oracle.com Tue Jul 14 22:55:10 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 14 Jul 2020 15:55:10 -0700 Subject: RFR(XS): 8248879: SA core file support on OSX has some bugs trying to locate the jvm libraries In-Reply-To: <8c7edf32-3037-b8d7-a18e-587f7ae8d294@oracle.com> References: <6089f692-152b-8615-dfca-779f2b5028a0@oracle.com> <219dd806-ca93-64ce-f5f9-d75dd7acb7f1@oracle.com> <37eee0d3-6b00-eb11-6198-d8d87e3a7ad4@oracle.com> <99909975-074c-34a1-0f9b-69c4fb8f0eff@oracle.com> <8c7edf32-3037-b8d7-a18e-587f7ae8d294@oracle.com> Message-ID: <9bd21f67-982a-ef98-7c9a-3515c38b688c@oracle.com> Hi Chris and Alex, I agree the last occurrence of "/bin/" is better than the first. But I wonder if it makes sense to check all occurrences. Thanks, Serguei On 7/14/20 15:14, Alex Menkov wrote: > Yes, you are right. > This is not a function from strings.h > > Ok, you can leave strstr (and keep in mind that the path can't contain > "/bin/" other than jdk's bin) or implement the functionality. It > should be something simple like > > static const char* rstrstr(const char *str, const char *sub) { > ? const char *result = NULL; > ? for (const char *p = strstr(str, sub); p != NULL; p = strstr(p + 1, > sub)) { > ??? result = p; > ? } > ? return result; > } > > --alex > > On 07/14/2020 13:43, Chris Plummer wrote: >> Actually it's not so easy. I don't see any other references to >> strrstr in our source. When I reference strstr, it gives a warning >> because it's not declared. The only man page I can find says to >> include sstring2.h, but this file does not exist. It also says to >> link with -lsstrings2. >> >> Chris >> >> On 7/14/20 1:37 PM, Chris Plummer wrote: >>> Ok. I'll change both references to use strrstr. >>> >>> thanks, >>> >>> Chris >>> >>> On 7/14/20 1:11 PM, Alex Menkov wrote: >>>> Hi Chris, >>>> >>>> I think it would be better to use strrstr to correctly handle paths >>>> like >>>> /something/bin/jdk/bin/jhsdb >>>> >>>> And I'd updated >>>> 358?? char* posbin = strstr(execname, "/bin/java"); >>>> to use strrstr as well >>>> >>>> --alex >>>> >>>> On 07/14/2020 12:01, Chris Plummer wrote: >>>>> Ping! >>>>> >>>>> On 7/6/20 9:31 PM, Chris Plummer wrote: >>>>>> Hello, >>>>>> >>>>>> Please help review the following: >>>>>> >>>>>> http://cr.openjdk.java.net/~cjplummer/8248879/webrev.00/index.html >>>>>> https://bugs.openjdk.java.net/browse/JDK-8248879 >>>>>> >>>>>> The description of the problem and the fix are both in the CR. >>>>>> >>>>>> thanks, >>>>>> >>>>>> Chris >>>>> >>> >>> >> >> From serguei.spitsyn at oracle.com Tue Jul 14 23:15:12 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 14 Jul 2020 16:15:12 -0700 Subject: RFR(XS): 8248879: SA core file support on OSX has some bugs trying to locate the jvm libraries In-Reply-To: <9bd21f67-982a-ef98-7c9a-3515c38b688c@oracle.com> References: <6089f692-152b-8615-dfca-779f2b5028a0@oracle.com> <219dd806-ca93-64ce-f5f9-d75dd7acb7f1@oracle.com> <37eee0d3-6b00-eb11-6198-d8d87e3a7ad4@oracle.com> <99909975-074c-34a1-0f9b-69c4fb8f0eff@oracle.com> <8c7edf32-3037-b8d7-a18e-587f7ae8d294@oracle.com> <9bd21f67-982a-ef98-7c9a-3515c38b688c@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From igor.ignatyev at oracle.com Tue Jul 14 23:29:39 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 14 Jul 2020 16:29:39 -0700 Subject: [15] RFR : 8249040 : clean up FileInstaller $test.src $cwd in vmTestbase_nsk_jdb tests Message-ID: http://cr.openjdk.java.net/~iignatyev/8249040/webrev.00/ > 135 lines changed: 2 ins; 63 del; 70 mod; Hi all, could you please review the clean-up of nsk_jdb tests? from main issue(8204985) : > all vmTestbase tests have '@run driver jdk.test.lib.FileInstaller . .' to mimic old test harness behavior and copy all files from a test source directory to a current work directory. some tests depend on this step, so we need 1st identify such tests and then either rewrite them not to have this dependency or leave FileInstaller only in these tests. the patch removes FileInstaller actions in all the tests. as in all previous patches, the biggest (and tedious) part of the patch is just `ag -l '@run driver jdk.test.lib.FileInstaller . .' $DIR | xargs -I{} gsed -i '/@run driver jdk.test.lib.FileInstaller \. \./d' {}` with $DIR being test/hotspot/jtreg/vmTestbase/nsk/jdb/; two tests however required manual intervention: nsk/jdb/read/read001 and use001/use001 were misusing `-workdir` option. these tests expected this option to point to the directory w/ test's source code. 'workdir' option is processed by nsk/share/jdb/JdbArgumentHandler and is meant to store "full path to current test directory", which obviously doesn't have to be in the source tree. the tests were updated to use `test.src` property value instead of `workdir`. testing: :vmTestbase_nsk_jdb on linux-x64 JBS: https://bugs.openjdk.java.net/browse/JDK-8249040 webrev: http://cr.openjdk.java.net/~iignatyev/8249040/webrev.00/ Thanks, -- Igor From serguei.spitsyn at oracle.com Tue Jul 14 23:37:07 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 14 Jul 2020 16:37:07 -0700 Subject: RFR(M): 8247515: OSX pc_to_symbol() lookup does not work with core files In-Reply-To: <3c31622b-831e-d46f-daa6-d641cbf755b1@oracle.com> References: <092ef5a8-c973-e377-de91-b2002a1b8fe7@oracle.com> <38e1887e-a646-abc0-0b7e-53339af1d684@oracle.com> <3c31622b-831e-d46f-daa6-d641cbf755b1@oracle.com> Message-ID: <18f7a1d8-3802-c7da-c981-eac8cf0953eb@oracle.com> An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Tue Jul 14 23:41:08 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 15 Jul 2020 09:41:08 +1000 Subject: RFR [15] : 8249039 : clean up FileInstaller $test.src $cwd in vmTestbase_nsk_aod tests In-Reply-To: <4A9D95BB-E8AD-42B7-BC92-71C16C0F2B40@oracle.com> References: <4A9D95BB-E8AD-42B7-BC92-71C16C0F2B40@oracle.com> Message-ID: <4bedfc1c-8fd3-7486-4d67-32e76ac57409@oracle.com> Hi Igor, LGTM. (Sorry I skipped this one yesterday. :) ) Thanks, David On 14/07/2020 8:29 am, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8249039/webrev.00/ >> 38 lines changed: 8 ins; 13 del; 17 mod; > > Hi all, > > could you please review this clean-up which removes `FileInstaller` actions from :vmTestbase_nsk_aod tests? > from the main issue(8204985): >> all vmTestbase tests have '@run driver jdk.test.lib.FileInstaller . .' to mimic old test harness behavior and copy all files from a test source directory to a current work directory. some tests depend on this step, so we need 1st identify such tests and then either rewrite them not to have this dependency or leave FileInstaller only in these tests. > > VirtualMachine06 test assumed that manifest files are located in the current directory, the test has been updated to have ${test.src} in the path to manifests. all other vmTestbase_nsk_stress tests don't need FileInstaller, so the rest of the patch is produced by `ag -l '@run driver jdk.test.lib.FileInstaller . .' vmTestbase/nsk/aod xargs -I{} gsed -i '/@run driver jdk.test.lib.FileInstaller \. \./d' {}`. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8249039 > webrev: http://cr.openjdk.java.net/~iignatyev//8249039/webrev.00/ > testing: :vmTestbase_nsk_aod on linux-x64 > > Thanks, > -- Igor > From serguei.spitsyn at oracle.com Wed Jul 15 00:01:31 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 14 Jul 2020 17:01:31 -0700 Subject: RFR(M): 8247272: SA ELF file support has never worked for 64-bit causing address to symbol name mapping to fail In-Reply-To: <8279763d-65de-b3d6-beb5-e40ff94b2804@oracle.com> References: <338118cb-37ff-9d03-f22e-c96322bfd7a6@oracle.com> <54ca0720-61f2-4580-80e5-b2a0da966cc0@oracle.com> <9d743570-7e55-df53-abfc-77a62af4d136@oracle.com> <8279763d-65de-b3d6-beb5-e40ff94b2804@oracle.com> Message-ID: <19f7e6f5-14e0-4865-dc41-c7e0a906f6b0@oracle.com> An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Wed Jul 15 00:06:09 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 14 Jul 2020 17:06:09 -0700 Subject: RFR (S) 8247615: Initialize the bytes left for the heap sampler In-Reply-To: References: Message-ID: <0aeedd0d-3cf1-05fb-6ad9-a6207ea9e306@oracle.com> An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Wed Jul 15 00:25:19 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 14 Jul 2020 17:25:19 -0700 Subject: RFR [15] : 8249039 : clean up FileInstaller $test.src $cwd in vmTestbase_nsk_aod tests In-Reply-To: <4bedfc1c-8fd3-7486-4d67-32e76ac57409@oracle.com> References: <4A9D95BB-E8AD-42B7-BC92-71C16C0F2B40@oracle.com> <4bedfc1c-8fd3-7486-4d67-32e76ac57409@oracle.com> Message-ID: <2273f689-6229-f710-1d35-2b2e15f5cd34@oracle.com> Hi Igor, LGTM++ Thanks, Serguei On 7/14/20 16:41, David Holmes wrote: > Hi Igor, > > LGTM. > > (Sorry I skipped this one yesterday. :) ) > > Thanks, > David > > On 14/07/2020 8:29 am, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev/8249039/webrev.00/ >>> 38 lines changed: 8 ins; 13 del; 17 mod; >> >> Hi all, >> >> could you please review this clean-up which removes `FileInstaller` >> actions from :vmTestbase_nsk_aod tests? >> from the main issue(8204985): >>> all vmTestbase tests have '@run driver jdk.test.lib.FileInstaller . >>> .' to mimic old test harness behavior and copy all files from a test >>> source directory to a current work directory. some tests depend on >>> this step, so we need 1st identify such tests and then either >>> rewrite them not to have this dependency or leave FileInstaller only >>> in these tests. >> >> VirtualMachine06 test assumed that manifest files are located in the >> current directory, the test has been updated to have ${test.src} in >> the path to manifests. all other vmTestbase_nsk_stress tests don't >> need FileInstaller, so the rest of the patch is produced by `ag -l? >> '@run driver jdk.test.lib.FileInstaller . .' vmTestbase/nsk/aod xargs >> -I{} gsed -i '/@run driver jdk.test.lib.FileInstaller \. \./d' {}`. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8249039 >> webrev: http://cr.openjdk.java.net/~iignatyev//8249039/webrev.00/ >> testing: :vmTestbase_nsk_aod on linux-x64 >> >> Thanks, >> -- Igor >> From chris.plummer at oracle.com Wed Jul 15 00:32:46 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 14 Jul 2020 17:32:46 -0700 Subject: RFR(M): 8247272: SA ELF file support has never worked for 64-bit causing address to symbol name mapping to fail In-Reply-To: <19f7e6f5-14e0-4865-dc41-c7e0a906f6b0@oracle.com> References: <338118cb-37ff-9d03-f22e-c96322bfd7a6@oracle.com> <54ca0720-61f2-4580-80e5-b2a0da966cc0@oracle.com> <9d743570-7e55-df53-abfc-77a62af4d136@oracle.com> <8279763d-65de-b3d6-beb5-e40ff94b2804@oracle.com> <19f7e6f5-14e0-4865-dc41-c7e0a906f6b0@oracle.com> Message-ID: <18b3107a-54db-74cc-fd0e-a1dfe3e29942@oracle.com> An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Wed Jul 15 00:38:25 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 14 Jul 2020 17:38:25 -0700 Subject: RFR [15] : 8249034 : clean up FileInstaller $test.src $cwd in vmTestbase_nsk_jvmti tests In-Reply-To: References: Message-ID: <1ccb7b00-1330-5cd7-e266-11aa920034c4@oracle.com> Hi Igor, LGTM. Thanks, Serguei On 7/13/20 16:22, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8249034/webrev.00/ >> 1289 lines changed: 2 ins; 652 del; 635 mod; > Hi all, > > could you please review this clean-up which removes `FileInstaller` actions from :vmTestbase_nsk_jvmti tests? > from the main issue(8204985): >> all vmTestbase tests have '@run driver jdk.test.lib.FileInstaller . .' to mimic old test harness behavior and copy all files from a test source directory to a current work directory. some tests depend on this step, so we need 1st identify such tests and then either rewrite them not to have this dependency or leave FileInstaller only in these tests. > `ag -l '@run driver jdk.test.lib.FileInstaller . .' vmTestbase/nsk/jvmti vmTestbase/nsk/share/ExceptionCheckingJniEnv xargs -I{} gsed -i '/@run driver jdk.test.lib.FileInstaller \. \./d' {}` was used to remove FileInstaller actions from all nsk_jvmti tests. 3 tests had dependency on that action: > - Allocate/alloc001, where alloc001.sh was assuming that alloc001.bash is in the current directory. alloc001.sh has been updated to use ${TESTSRC} env variable > - AttachOnDemand/attach002a and scenarios/bcinstr/BI04/bi04t002: both tests assumed that newclassXX directory is located in the current directory. the tests were fixed by prefixing 'newclassXX' w/ '${test.src}/' which gets expanded by PropertyResolvingWrapper > > JBS: https://bugs.openjdk.java.net/browse/JDK-8249034 > webrev: http://cr.openjdk.java.net/~iignatyev//8249034/webrev.00/ > testing: :vmTestbase_nsk_jvmti on linux-x64 > > Thanks, > -- Igor From chris.plummer at oracle.com Wed Jul 15 00:40:32 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 14 Jul 2020 17:40:32 -0700 Subject: RFR(M): 8247515: OSX pc_to_symbol() lookup does not work with core files In-Reply-To: <18f7a1d8-3802-c7da-c981-eac8cf0953eb@oracle.com> References: <092ef5a8-c973-e377-de91-b2002a1b8fe7@oracle.com> <38e1887e-a646-abc0-0b7e-53339af1d684@oracle.com> <3c31622b-831e-d46f-daa6-d641cbf755b1@oracle.com> <18f7a1d8-3802-c7da-c981-eac8cf0953eb@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Wed Jul 15 00:43:17 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 14 Jul 2020 17:43:17 -0700 Subject: RFR(M): 8247272: SA ELF file support has never worked for 64-bit causing address to symbol name mapping to fail In-Reply-To: <18b3107a-54db-74cc-fd0e-a1dfe3e29942@oracle.com> References: <338118cb-37ff-9d03-f22e-c96322bfd7a6@oracle.com> <54ca0720-61f2-4580-80e5-b2a0da966cc0@oracle.com> <9d743570-7e55-df53-abfc-77a62af4d136@oracle.com> <8279763d-65de-b3d6-beb5-e40ff94b2804@oracle.com> <19f7e6f5-14e0-4865-dc41-c7e0a906f6b0@oracle.com> <18b3107a-54db-74cc-fd0e-a1dfe3e29942@oracle.com> Message-ID: <79f3a19c-6d9d-e5a3-be15-99cd02dd9235@oracle.com> An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Wed Jul 15 00:46:55 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 14 Jul 2020 17:46:55 -0700 Subject: RFR(M): 8247515: OSX pc_to_symbol() lookup does not work with core files In-Reply-To: References: <092ef5a8-c973-e377-de91-b2002a1b8fe7@oracle.com> <38e1887e-a646-abc0-0b7e-53339af1d684@oracle.com> <3c31622b-831e-d46f-daa6-d641cbf755b1@oracle.com> <18f7a1d8-3802-c7da-c981-eac8cf0953eb@oracle.com> Message-ID: <1e9e4786-79f8-c4e0-efe7-f1a8f369d2c6@oracle.com> An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Wed Jul 15 01:05:13 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 14 Jul 2020 18:05:13 -0700 Subject: RFR(XS): 8248878: SA: Implement simple workaround for JDK-8248876 In-Reply-To: <099b6b7d-f3c2-57d4-832a-7144b51eecb7@oracle.com> References: <39ac40d6-a738-3214-d86c-412152a631d4@oracle.com> <8e89d223-0f65-f26e-3bd7-2ca7421a940b@oracle.com> <6d2610d6-5ca1-8455-ddc6-d7f02b21c0d5@oracle.com> <046aced2-1d23-d096-748b-4ce0bfd01587@oracle.com> <1fde7056-f2d0-e256-ae6d-b8db94068c71@oss.nttdata.com> <099b6b7d-f3c2-57d4-832a-7144b51eecb7@oracle.com> Message-ID: Hi Chris, This workaround looks reasonable to me. Thanks, Serguei On 7/14/20 12:04, Chris Plummer wrote: > Hello, > > Can I get a second reviewer please. Yasumasa has done the first > review. The discussion below was all about how to possibly fix > JDK-8248876, but for now we're sticking with this workaround. > > http://cr.openjdk.java.net/~cjplummer/8248878/webrev.00/index.html > https://bugs.openjdk.java.net/browse/JDK-8248878 > > The explanation of the fix is in the CR. The parent CR, JDK-8248876 > [1], explains the issue being addressed. > > There's no test for this fix yet. It requires the changes I'm making > for JDK-8247514 [2], which include changes to "findpc" support and the > ClhsdbFindPC.java test that trigger this issue. > > [1] https://bugs.openjdk.java.net/browse/JDK-8248876 > [2] https://bugs.openjdk.java.net/browse/JDK-8247514 > > thanks, > > Chris > > On 7/13/20 4:48 PM, Yasumasa Suenaga wrote: >> Sure, I reviewed for this workaround (JDK-8248878). >> >> I'm still think we can use note section in the core for JDK-8248876 >> as I said, but I'm not sure. >> >> >> Thanks, >> >> Yasumasa >> >> >> On 2020/07/14 3:12, Chris Plummer wrote: >>> Hi Yasumasa, >>> >>> If you have no further suggestions on how to fix JDK-8248876, I'd >>> like to proceed with this work around. Can I considered it reviewed >>> by you? >>> >>> thanks, >>> >>> Chris >>> >>> On 7/7/20 7:29 PM, Chris Plummer wrote: >>>> Hi Yasumasa, >>>> >>>> The executable is not opened with pathmap_open: >>>> >>>> ? if ((ph->core->exec_fd = open(exec_file, O_RDONLY)) < 0) { >>>> >>>> I think pathmap_open() is just used for libraries. >>>> >>>> thanks, >>>> >>>> Chris >>>> >>>> On 7/7/20 6:18 PM, Yasumasa Suenaga wrote: >>>>> Hi Chris, >>>>> >>>>> SA would use `link_map` to decide to load address, but it does not >>>>> seem to contain executable. >>>>> I set breakpoint to pathmap_open() and I watched the argument of >>>>> it, then I didn't see any executable (`java`) on it. >>>>> Maybe current implementation is broken. >>>>> >>>>> I guess we can use note section in the core for deciding loading >>>>> address. >>>>> I can see valid address (includes executable) from `readelf -n`. >>>>> Of course it might be big change for SA... >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2020/07/07 15:38, Chris Plummer wrote: >>>>>> Hi Yasumasa, >>>>>> >>>>>> Thanks for the review. I tried the following for line 188: >>>>>> >>>>>> ???? if ((phdr->p_type == PT_LOAD || phdr->p_type == PT_INTERP) >>>>>> && phdr->p_vaddr < baseaddr) { >>>>>> >>>>>> However, "base" still ended up being 0. I added some printfs. For >>>>>> the exec file there is both a PT_INTER with p_vaddr of 0x238 and >>>>>> a PT_LOAD with p_vaddr 0. I'm not sure which to use, but in >>>>>> either case that won't be the proper base when added to 0: >>>>>> >>>>>> ?? if (add_lib_info_fd(ph, exec_file, ph->core->exec_fd, >>>>>> ?????????????????????? (uintptr_t)0 + >>>>>> find_base_address(ph->core->exec_fd, &exec_ehdr)) == NULL) { >>>>>> ???? goto err; >>>>>> ?? } >>>>>> >>>>>> So maybe it's the (uintptr_t)0 that is the problem here. For >>>>>> shared libs instead of 0 it computes the value to add: >>>>>> >>>>>> ??????????????? if (lib_base_diff == ZERO_LOAD_ADDRESS ) { >>>>>> ????????????????? lib_base_diff = calc_prelinked_load_address(ph, >>>>>> lib_fd, &elf_ehdr, link_map_addr); >>>>>> ????????????????? if (lib_base_diff == INVALID_LOAD_ADDRESS) { >>>>>> ??????????????????? close(lib_fd); >>>>>> ??????????????????? return false; >>>>>> ????????????????? } >>>>>> ??????????????? } >>>>>> >>>>>> ??????????????? lib_base = lib_base_diff + >>>>>> find_base_address(lib_fd, &elf_ehdr); >>>>>> >>>>>> So in this case we've actually computed lib_base_diff rather than >>>>>> just assumed 0. >>>>>> >>>>>> Chris >>>>>> >>>>>> On 7/6/20 10:46 PM, Yasumasa Suenaga wrote: >>>>>>> Hi Chris, >>>>>>> >>>>>>> Your change looks good. >>>>>>> >>>>>>> >>>>>>> BTW I saw JDK-8248876. I'm not sure, but I guess we can fix this >>>>>>> issue if we allow PT_INTERP in L118: >>>>>>> >>>>>>> ``` >>>>>>> 105 uintptr_t find_base_address(int fd, ELF_EHDR* ehdr) { >>>>>>> ????????????????? : >>>>>>> 115?? // the base address of a shared object is the lowest vaddr of >>>>>>> 116?? // its loadable segments (PT_LOAD) >>>>>>> 117?? for (phdr = phbuf, cnt = 0; cnt < ehdr->e_phnum; cnt++, >>>>>>> phdr++) { >>>>>>> 118???? if (phdr->p_type == PT_LOAD && phdr->p_vaddr < baseaddr) { >>>>>>> 119?????? baseaddr = phdr->p_vaddr; >>>>>>> 120???? } >>>>>>> 121?? } >>>>>>> ``` >>>>>>> >>>>>>> /proc//maps shows top of `java` is 0x56543b9df000: >>>>>>> >>>>>>> 56543b9df000-56543b9e0000 r--p 00000000 08:10 55770 >>>>>>> /usr/lib/jvm/java-11-openjdk-amd64/bin/java >>>>>>> >>>>>>> >>>>>>> `i target` on GDB shows 0x56543b9df000 is .interp section: >>>>>>> >>>>>>> Local exec file: >>>>>>> ??????? `/usr/lib/jvm/java-11-openjdk-amd64/bin/java', file type >>>>>>> elf64-x86-64. >>>>>>> ??????? Entry point: 0x56543b9e0330 >>>>>>> ??????? 0x000056543b9df318 - 0x000056543b9df334 is .interp >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2020/07/07 13:18, Chris Plummer wrote: >>>>>>>> Hello, >>>>>>>> >>>>>>>> Please help review the following: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~cjplummer/8248878/webrev.00/index.html >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8248878 >>>>>>>> >>>>>>>> The explanation of the fix is in the CR. The parent CR, >>>>>>>> JDK-8248876 [1], explains the issue being addressed. >>>>>>>> >>>>>>>> There's no test for this fix yet. It requires the changes I'm >>>>>>>> making for JDK-8247514 [2], which include changes to "findpc" >>>>>>>> support and the ClhsdbFindPC.java test that trigger this issue. >>>>>>>> >>>>>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8248876 >>>>>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8247514 >>>>>>>> >>>>>>>> thanks, >>>>>>>> >>>>>>>> Chris >>>>>> >>>> >>> >>> > > From alexey.menkov at oracle.com Wed Jul 15 01:19:10 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Tue, 14 Jul 2020 18:19:10 -0700 Subject: RFR(XS): 8248879: SA core file support on OSX has some bugs trying to locate the jvm libraries In-Reply-To: <9bd21f67-982a-ef98-7c9a-3515c38b688c@oracle.com> References: <6089f692-152b-8615-dfca-779f2b5028a0@oracle.com> <219dd806-ca93-64ce-f5f9-d75dd7acb7f1@oracle.com> <37eee0d3-6b00-eb11-6198-d8d87e3a7ad4@oracle.com> <99909975-074c-34a1-0f9b-69c4fb8f0eff@oracle.com> <8c7edf32-3037-b8d7-a18e-587f7ae8d294@oracle.com> <9bd21f67-982a-ef98-7c9a-3515c38b688c@oracle.com> Message-ID: <0906c510-8c2c-2c56-803f-0a0bf4340df6@oracle.com> Hi Serguei, On 07/14/2020 15:55, serguei.spitsyn at oracle.com wrote: > Hi Chris and Alex, > > I agree the last occurrence of "/bin/" is better than the first. > But I wonder if it makes sense to check all occurrences. The problem is strrstr (search for last occurrence) is not a part of std C lib. So to avoid dependency on new library I suggested this simple implementation using standard strstr. --alex > > Thanks, > Serguei > > > On 7/14/20 15:14, Alex Menkov wrote: >> Yes, you are right. >> This is not a function from strings.h >> >> Ok, you can leave strstr (and keep in mind that the path can't contain >> "/bin/" other than jdk's bin) or implement the functionality. It >> should be something simple like >> >> static const char* rstrstr(const char *str, const char *sub) { >> ? const char *result = NULL; >> ? for (const char *p = strstr(str, sub); p != NULL; p = strstr(p + 1, >> sub)) { >> ??? result = p; >> ? } >> ? return result; >> } >> >> --alex >> >> On 07/14/2020 13:43, Chris Plummer wrote: >>> Actually it's not so easy. I don't see any other references to >>> strrstr in our source. When I reference strstr, it gives a warning >>> because it's not declared. The only man page I can find says to >>> include sstring2.h, but this file does not exist. It also says to >>> link with -lsstrings2. >>> >>> Chris >>> >>> On 7/14/20 1:37 PM, Chris Plummer wrote: >>>> Ok. I'll change both references to use strrstr. >>>> >>>> thanks, >>>> >>>> Chris >>>> >>>> On 7/14/20 1:11 PM, Alex Menkov wrote: >>>>> Hi Chris, >>>>> >>>>> I think it would be better to use strrstr to correctly handle paths >>>>> like >>>>> /something/bin/jdk/bin/jhsdb >>>>> >>>>> And I'd updated >>>>> 358?? char* posbin = strstr(execname, "/bin/java"); >>>>> to use strrstr as well >>>>> >>>>> --alex >>>>> >>>>> On 07/14/2020 12:01, Chris Plummer wrote: >>>>>> Ping! >>>>>> >>>>>> On 7/6/20 9:31 PM, Chris Plummer wrote: >>>>>>> Hello, >>>>>>> >>>>>>> Please help review the following: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~cjplummer/8248879/webrev.00/index.html >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8248879 >>>>>>> >>>>>>> The description of the problem and the fix are both in the CR. >>>>>>> >>>>>>> thanks, >>>>>>> >>>>>>> Chris >>>>>> >>>> >>>> >>> >>> > From serguei.spitsyn at oracle.com Wed Jul 15 03:54:56 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 14 Jul 2020 20:54:56 -0700 Subject: RFR(XS): 8248879: SA core file support on OSX has some bugs trying to locate the jvm libraries In-Reply-To: <0906c510-8c2c-2c56-803f-0a0bf4340df6@oracle.com> References: <6089f692-152b-8615-dfca-779f2b5028a0@oracle.com> <219dd806-ca93-64ce-f5f9-d75dd7acb7f1@oracle.com> <37eee0d3-6b00-eb11-6198-d8d87e3a7ad4@oracle.com> <99909975-074c-34a1-0f9b-69c4fb8f0eff@oracle.com> <8c7edf32-3037-b8d7-a18e-587f7ae8d294@oracle.com> <9bd21f67-982a-ef98-7c9a-3515c38b688c@oracle.com> <0906c510-8c2c-2c56-803f-0a0bf4340df6@oracle.com> Message-ID: <6f8cebde-428e-9e70-ed15-b05b6f6446e8@oracle.com> Hi Alex, Yes, I understand this. After some thinking, I doubt my suggestion to check all occurrences or "/bin/" is good. :) Thanks, Serguei On 7/14/20 18:19, Alex Menkov wrote: > Hi Serguei, > > On 07/14/2020 15:55, serguei.spitsyn at oracle.com wrote: >> Hi Chris and Alex, >> >> I agree the last occurrence of "/bin/" is better than the first. >> But I wonder if it makes sense to check all occurrences. > > The problem is strrstr (search for last occurrence) is not a part of > std C lib. > So to avoid dependency on new library I suggested this simple > implementation using standard strstr. > > --alex > >> >> Thanks, >> Serguei >> >> >> On 7/14/20 15:14, Alex Menkov wrote: >>> Yes, you are right. >>> This is not a function from strings.h >>> >>> Ok, you can leave strstr (and keep in mind that the path can't >>> contain "/bin/" other than jdk's bin) or implement the >>> functionality. It should be something simple like >>> >>> static const char* rstrstr(const char *str, const char *sub) { >>> ? const char *result = NULL; >>> ? for (const char *p = strstr(str, sub); p != NULL; p = strstr(p + >>> 1, sub)) { >>> ??? result = p; >>> ? } >>> ? return result; >>> } >>> >>> --alex >>> >>> On 07/14/2020 13:43, Chris Plummer wrote: >>>> Actually it's not so easy. I don't see any other references to >>>> strrstr in our source. When I reference strstr, it gives a warning >>>> because it's not declared. The only man page I can find says to >>>> include sstring2.h, but this file does not exist. It also says to >>>> link with -lsstrings2. >>>> >>>> Chris >>>> >>>> On 7/14/20 1:37 PM, Chris Plummer wrote: >>>>> Ok. I'll change both references to use strrstr. >>>>> >>>>> thanks, >>>>> >>>>> Chris >>>>> >>>>> On 7/14/20 1:11 PM, Alex Menkov wrote: >>>>>> Hi Chris, >>>>>> >>>>>> I think it would be better to use strrstr to correctly handle >>>>>> paths like >>>>>> /something/bin/jdk/bin/jhsdb >>>>>> >>>>>> And I'd updated >>>>>> 358?? char* posbin = strstr(execname, "/bin/java"); >>>>>> to use strrstr as well >>>>>> >>>>>> --alex >>>>>> >>>>>> On 07/14/2020 12:01, Chris Plummer wrote: >>>>>>> Ping! >>>>>>> >>>>>>> On 7/6/20 9:31 PM, Chris Plummer wrote: >>>>>>>> Hello, >>>>>>>> >>>>>>>> Please help review the following: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~cjplummer/8248879/webrev.00/index.html >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8248879 >>>>>>>> >>>>>>>> The description of the problem and the fix are both in the CR. >>>>>>>> >>>>>>>> thanks, >>>>>>>> >>>>>>>> Chris >>>>>>> >>>>> >>>>> >>>> >>>> >> From david.holmes at oracle.com Wed Jul 15 04:01:54 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 15 Jul 2020 14:01:54 +1000 Subject: [15] RFR : 8249040 : clean up FileInstaller $test.src $cwd in vmTestbase_nsk_jdb tests In-Reply-To: References: Message-ID: Hi Igor, Looks good to me. On 15/07/2020 9:29 am, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8249040/webrev.00/ >> 135 lines changed: 2 ins; 63 del; 70 mod; > > Hi all, > > > could you please review the clean-up of nsk_jdb tests? > from main issue(8204985) : >> all vmTestbase tests have '@run driver jdk.test.lib.FileInstaller . .' to mimic old test harness behavior and copy all files from a test source directory to a current work directory. some tests depend on this step, so we need 1st identify such tests and then either rewrite them not to have this dependency or leave FileInstaller only in these tests. > > > the patch removes FileInstaller actions in all the tests. as in all previous patches, the biggest (and tedious) part of the patch is just `ag -l '@run driver jdk.test.lib.FileInstaller . .' $DIR | xargs -I{} gsed -i '/@run driver jdk.test.lib.FileInstaller \. \./d' {}` with $DIR being test/hotspot/jtreg/vmTestbase/nsk/jdb/; two tests however required manual intervention: nsk/jdb/read/read001 and use001/use001 were misusing `-workdir` option. these tests expected this option to point to the directory w/ test's source code. 'workdir' option is processed by nsk/share/jdb/JdbArgumentHandler and is meant to store "full path to current test directory", which obviously doesn't have to be in the source tree. the tests were updated to use `test.src` property value instead of `workdir`. All the commented out stuff in use001 is a little confusing, but not due to this change of course. Thanks, David > testing: :vmTestbase_nsk_jdb on linux-x64 > JBS: https://bugs.openjdk.java.net/browse/JDK-8249040 > webrev: http://cr.openjdk.java.net/~iignatyev/8249040/webrev.00/ > > Thanks, > -- Igor > From david.holmes at oracle.com Wed Jul 15 05:09:36 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 15 Jul 2020 15:09:36 +1000 Subject: Java 14 jstat -class option man doc typo In-Reply-To: <4616bae2.607a.17347552ef7.Coremail.sa514093@mail.ustc.edu.cn> References: <4616bae2.607a.17347552ef7.Coremail.sa514093@mail.ustc.edu.cn> Message-ID: <14d2adb5-537e-810a-591f-91b1a02b6ec2@oracle.com> Hi, I've filed: https://bugs.openjdk.java.net/browse/JDK-8249450 for this issue. As the sources for these tools are not maintained in the OpenJDK repository, the Oracle serviceability team will need to apply the fix and generate the open man page. Cheers, David On 13/07/2020 6:40 pm, sa514093 at mail.ustc.edu.cn wrote: > Hi all, > > I recently found a document typo in jstat -class option man doc. > > This typo maybe caused by automatically generated by Pandoc tool. > > https://github.com/openjdk/jdk/blob/d34f561de002510c818c73cd5871caf2222ca0a7/src/jdk.jcmd/share/man/jstat.1#L227 > > Bytes: Number of KB loaded. --> Bytes: Number of KB unloaded. > > I report issue to Github, and follow Martijn Verburg's advice to > report to upstream here. > > https://github.com/AdoptOpenJDK/openjdk-jdk/issues/3 > > https://github.com/SAP/SapMachine/issues/687 > > Thanks > From linzang at tencent.com Wed Jul 15 09:12:31 2020 From: linzang at tencent.com (=?utf-8?B?bGluemFuZyjoh6fnkLMp?=) Date: Wed, 15 Jul 2020 09:12:31 +0000 Subject: RFR(L): 8215624: add parallel heap inspection support for jmap histo(G1)(Internet mail) In-Reply-To: References: <02415944-2238-4430-A7C2-3625264A5505@amazon.com> Message-ID: Upload a new webrev at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_07/ It fix a potential issue that unexpected number of threads maybe calculated for "parallel" option of jmap -histo in container. As shown at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_07-delta/src/hotspot/share/services/attachListener.cpp.udiff.html ############### attachListener.cpp #################### @@ -252,11 +252,11 @@ static jint heap_inspection(AttachOperation* op, outputStream* out) { bool live_objects_only = true; // default is true to retain the behavior before this change is made outputStream* os = out; // if path not specified or path is NULL, use out fileStream* fs = NULL; const char* arg0 = op->arg(0); - uint parallel_thread_num = MAX(1, os::processor_count() * 3 / 8); // default is less than half of processors. + uint parallel_thread_num = MAX(1, os::initial_active_processor_count() * 3 / 8); // default is less than half of processors. if (arg0 != NULL && (strlen(arg0) > 0)) { if (strcmp(arg0, "-all") != 0 && strcmp(arg0, "-live") != 0) { out->print_cr("Invalid argument to inspectheap operation: %s", arg0); return JNI_ERR; } ################################################### Thanks. BRs, Lin ?On 2020/7/9, 3:22 PM, "linzang(??)" wrote: Hi Paul, Thanks for reviewing! >> >> I'd move all the argument parsing code to JMap.java and just pass the results to Hotspot. Both histo() in JMap.java and code in attachListener.* parse the command line arguments, though the code in histo() doesn't parse the argument to "parallel". I'd upgrade the code in histo() to do a complete parse and pass the option values to executeCommandForPid as before: there would just be more of them now. That would allow you to eliminate all the parsing code in attachListener.cpp as well as the change to arguments.hpp. >> The reason I made the change in Jmap.java that compose all arguments as 1 string , instead of passing 3 argments, is to avoid the compatibility issue, as we discussed in http://mail.openjdk.java.net/pipermail/serviceability-dev/2019-February/thread.html#27240. The root cause of the compatibility issue is because max argument count in HotspotVirtualMachineImpl.java and attachlistener.cpp need to be enlarged (changes like http://hg.openjdk.java.net/jdk/jdk/rev/e7cf035682e3#l2.1) when jmap has more than 3 arguments. But if user use an old jcmd/jmap tool, it may stuck at socket read(), because the "max argument count" don't match. I re-checked this change, the argument count of jmap histo is equal to 3 (live, file, parallel), so it can work normally even without the change of passing argument. But I think we have to face the problem if more arguments is added in jcmd alike tools later, not sure whether it should be sloved (or a workaround) in this changeset. And here are the lastest webrev and delta: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_06/ http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_06-delta/ Cheers, Lin On 2020/7/7, 5:57 AM, "Hohensee, Paul" wrote: I'd like to see this feature added. :) The CSR looks good, as does the basic parallel inspection algorithm. Stefan's done the GC part, so I'll stick to the non-GC part (fwiw, the GC part lgtm). I'd move all the argument parsing code to JMap.java and just pass the results to Hotspot. Both histo() in JMap.java and code in attachListener.* parse the command line arguments, though the code in histo() doesn't parse the argument to "parallel". I'd upgrade the code in histo() to do a complete parse and pass the option values to executeCommandForPid as before: there would just be more of them now. That would allow you to eliminate all the parsing code in attachListener.cpp as well as the change to arguments.hpp. heapInspection.hpp: _shared_miss_count (s/b _missed_count, see below) isn't a size, so it should be a uint instead of a size_t. Same with the new parallel_thread_num argument to heap_inspection() and populate_table(). Comment copy-edit: +// Parallel heap inspection task. Parallel inspection can fail due to +// a native OOM when allocating memory for TL-KlassInfoTable. +// _success will be set false on an OOM, and serial inspection tried. _shared_miss_count should be _missed_count to match the missed_count() getter, or rename missed_count() to be shared_miss_count(). Whichever way you go, the field type should match the getter result type: uint is reasonable. heapInspection.cpp: You might use ResourceMark twice in populate_table, separately for the parallel attempt and the serial code. If the parallel attempt fails and available memory is low, it would be good to clean up the memory used by the parallel attempt before doing the serial code. Style nit in KlassInfoTable::merge_entry(). I'd line up the definitions of k and elt, so "k" is even with "elt". And, because it's two lines shorter, I'd replace + } else { + return false; + } with + return false; KlassInfoTableMergeClosure.is_success() should be just success() (i.e., no "is_" prefix) because it's a getter. I'd reorganize the code in populate_table() to make it more clear, vis (I changed _shared_missed_count to _missed_count) + if (cit.allocation_failed()) { + // fail to allocate memory, stop parallel mode + Atomic::store(&_success, false); + return; + } + RecordInstanceClosure ric(&cit, _filter); + _poi->object_iterate(&ric, worker_id); + missed_count = ric.missed_count(); + { + MutexLocker x(&_mutex); + merge_success = _shared_cit->merge(&cit); + } + if (merge_success) { + Atomic::add(&_missed_count, missed_count); + else { + Atomic::store(&_success, false); + } Thanks, Paul On 6/29/20, 7:20 PM, "linzang(??)" wrote: Dear All, Sorry to bother again, I just want to make sure that is this change worth to be continue to work on? If decision is made to not. I think I can drop this work and stop asking for help reviewing... Thanks for all your help about reviewing this previously. BRs, Lin On 2020/5/9, 3:47 PM, "linzang(??)" wrote: Dear All, May I ask your help again for review the latest change? Thanks! BRs, Lin On 2020/4/28, 1:54 PM, "linzang(??)" wrote: Hi Stefan, >> - Adding Atomic::load/store. >> - Removing the time measurement in the run_task. I renamed G1's function >> to run_task_timed. If we need this outside of G1, we can rethink the API >> at that point. >> - ZGC style cleanups Thanks for revising the patch, they are all good to me, and I have made a tiny change based on it: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04/ http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04-delta/ it reduce the scope of mutex in ParHeapInspectTask, and delete unnecessary comments. BRs, Lin On 2020/4/27, 4:34 PM, "Stefan Karlsson" wrote: Hi Lin, On 2020-04-26 05:10, linzang(??) wrote: > Hi Stefan and Paul? > I have made a new patch based on your comments and Stefan's Poc code: > Webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03/ > Delta(based on Stefan's change:) : http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03-delta/webrev_03-delta/ Thanks for providing a delta patch. It makes it much easier to look at, and more likely for reviewers to continue reviewing. I'm going to continue focusing on the GC parts, and leave the rest to others to review. > > And Here are main changed I made and want to discuss with you: > 1. changed"parallelThreadNum=" to "parallel=" for jmap -histo options. > 2. Add logic to test where parallelHeapInspection is fail, in heapInspection.cpp > This is because the parHeapInspectTask create thread local KlassInfoTable in it's work() method, and this may fail because of native OOM, in this case, the parallel should fail and serial heap inspection can be tried. > One more thing I want discuss with you is about the member "_success" of parHeapInspectTask, when native OOM happenes, it is set to false. And since this "set" operation can be conducted in multiple threads, should it be atomic ops? IMO, this is not necessary because "_success" can only be set to false, and there is no way to change it from back to true after the ParHeapInspectTask instance is created, so it is save to be non-atomic, do you agree with that? In these situations you should be using the Atomic::load/store primitives. We're moving toward a later C++ standard were data races are considered undefined behavior. > 3. make CollectedHeap::run_task() be an abstract virtual func, so that every subclass of collectedHeap should support it, so later implementation of new collectedHeap will not miss the "parallel" features. > The problem I want to discuss with you is about epsilonHeap and SerialHeap, as they may not need parallel heap iteration, so I only make task->work(0), in case the run_task() is invoked someway in future. Another way is to left run_task() unimplemented, which one do you think is better? I don't have a strong opinion about this. And also please help take a look at the zHeap, as there is a class zTask that wrap the abstractGangTask, and the collectedHeap::run_task() only accept AbstraceGangTask* as argument, so I made a delegate class to adapt it , please see src/hotspot/share/gc/z/zHeap.cpp. > > There maybe other better ways to sovle the above problems, welcome for any comments, Thanks! I've created a few cleanups and changes on top of your latest patch: https://cr.openjdk.java.net/~stefank/8215624/webrev.02.delta https://cr.openjdk.java.net/~stefank/8215624/webrev.02 - Adding Atomic::load/store. - Removing the time measurement in the run_task. I renamed G1's function to run_task_timed. If we need this outside of G1, we can rethink the API at that point. - ZGC style cleanups Thanks, StefanK > > BRs, > Lin > > On 2020/4/23, 11:08 AM, "linzang(??)" wrote: > > Thanks Paul! I agree with using "parallel", will make the update in next patch, Thanks for help update the CSR. > > BRs, > Lin > > On 2020/4/23, 4:42 AM, "Hohensee, Paul" wrote: > > For the interface, I'd use "parallel" instead of "parallelThreadNum". All the other options are lower case, and it's a lot easier to type "parallel". I took the liberty of updating the CSR. If you're ok with it, you might want to change variable names and such, plus of course JMap.usage. > > Thanks, > Paul > > On 4/22/20, 2:29 AM, "serviceability-dev on behalf of linzang(??)" wrote: > > Dear Stefan, > > Thanks a lot! I agree with you to decouple the heap inspection code with GC's. > I will start from your POC code, may discuss with you later. > > > BRs, > Lin > > On 2020/4/22, 5:14 PM, "Stefan Karlsson" wrote: > > Hi Lin, > > I took a look at this earlier and saw that the heap inspection code is > strongly coupled with the CollectedHeap and G1CollectedHeap. I'd prefer > if we'd abstract this away, so that the GCs only provide a "parallel > object iteration" interface, and the heap inspection code is kept elsewhere. > > I started experimenting with doing that, but other higher-priority (to > me) tasks have had to take precedence. > > I've uploaded my work-in-progress / proof-of-concept: > https://cr.openjdk.java.net/~stefank/8215624/webrev.01.delta/ > https://cr.openjdk.java.net/~stefank/8215624/webrev.01/ > > The current code doesn't handle the lifecycle (deletion) of the > ParallelObjectIterators. There's also code left unimplemented in around > CollectedHeap::run_task. However, I think this could work as a basis to > pull out the heap inspection code out of the GCs. > > Thanks, > StefanK > > On 2020-04-22 02:21, linzang(??) wrote: > > Dear all, > > May I ask you help to review? This RFR has been there for quite a while. > > Thanks! > > > > BRs, > > Lin > > > > > On 2020/3/16, 5:18 PM, "linzang(??)" wrote:> > > > >> Just update a new path, my preliminary measure show about 3.5x speedup of jmap histo on a nearly full 4GB G1 heap (8-core platform with parallel thread number set to 4). > >> webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_02/ > >> bug: https://bugs.openjdk.java.net/browse/JDK-8215624 > >> CSR: https://bugs.openjdk.java.net/browse/JDK-8239290 > >> BRs, > >> Lin > >> > On 2020/3/2, 9:56 PM, "linzang(??)" wrote: > >> > > >> > Dear all, > >> > Let me try to ease the reviewing work by some explanation :P > >> > The patch's target is to speed up jmap -histo for heap iteration, from my experience it is necessary for large heap investigation. E.g in bigData scenario I have tried to conduct jmap -histo against 180GB heap, it does take quite a while. > >> > And if my understanding is corrent, even the jmap -histo without "live" option does heap inspection with heap lock acquired. so it is very likely to block mutator thread in allocation-sensitive scenario. I would say the faster the heap inspection does, the shorter the mutator be blocked. This is parallel iteration for jmap is necessary. > >> > I think the parallel heap inspection should be applied to all kind of heap. However, consider the heap layout are different for GCs, much time is required to understand all kinds of the heap layout to make the whole change. IMO, It is not wise to have a huge patch for the whole solution at once, and it is even harder to review it. So I plan to implement it incrementally, the first patch (this one) is going to confirm the implemention detail of how jmap accept the new option, passes it to attachListener of the jvm process and then how to make the parallel inspection closure be generic enough to make it easy to extend to different heap layout. And also how to implement the heap inspection in specific gc's heap. This patch use G1's heap as the begining. > >> > This patch actually do several things: > >> > 1. Add an option "parallelThreadNum=" to jmap -histo, the default behavior is to set N to 0, means let's JVM decide how many threads to use for heap inspection. Set this option to 1 will disable parallel heap inspection. (more details in CSR: https://bugs.openjdk.java.net/browse/JDK-8239290) > >> > 2. Make a change in how Jmap passing arguments, changes in http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java.udiff.html, originally it pass options as separate arguments to attachListener, this patch change to that all options be compose to a single string. So the arg_count_max in attachListener.hpp do not need to be changed, and hence avoid the compatibility issue, as disscussed at https://mail.openjdk.java.net/pipermail/serviceability-dev/2019-March/027334.html > >> > 3. Add an abstract class ParHeapInspectTask in heapInspection.hpp / heapInspection.cpp, It's work(uint worker_id) method prepares the data structure (KlassInfoTable) need for every parallel worker thread, and then call do_object_iterate_parallel() which is heap specific implementation. I also added some machenism in KlassInfoTable to support parallel iteration, such as merge(). > >> > 4. In specific heap (G1 in this patch), create a subclass of ParHeapInspectTask, implement the do_object_iterate_parallel() for parallel heap inspection. For G1, it simply invoke g1CollectedHeap's object_iterate_parallel(). > >> > 5. Add related test. > >> > 6. it may be easy to extend this patch for other kinds of heap by creating subclass of ParHeapInspectTask and implement the do_object_iterate_parallel(). > >> > > >> > Hope these info could help on code review and initate the discussion :-) > >> > Thanks! > >> > > >> > BRs, > >> > Lin > >> > >On 2020/2/19, 9:40 AM, "linzang(??)" wrote:. > >> > > > >> > > Re-post this RFR with correct enhancement number to make it trackable. > >> > > please ignore the previous wrong post. sorry for troubles. > >> > > > >> > > webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/ > >> > > Hi bug: https://bugs.openjdk.java.net/browse/JDK-8215624 > >> > > CSR: https://bugs.openjdk.java.net/browse/JDK-8239290 > >> > > -------------- > >> > > Lin > >> > > >Hi Lin, > > > > > > > >> > > >Could you, please, re-post your RFR with the right enhancement number in > >> > > >the message subject? > >> > > >It will be more trackable this way. > >> > > > > >> > > >Thanks, > >> > > >Serguei > >> > > > > >> > > > > >> > > >On 2/17/20 10:29 PM, linzang(??) wrote: > >> > > >> Dear David, > >> > > >> Thanks a lot! > >> > > >> I have updated the refined code to http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_01/. > >> > > >> IMHO the parallel heap inspection can be extended to all kinds of heap as long as the heap layout can support parallel iteration. > >> > > >> Maybe we can firstly use this webrev to discuss how to implement it, because I am not sure my current implementation is an appropriate way to communicate with collectedHeap, then we can extend the solution to other kinds of heap. > >> > > >> > >> > > >> Thanks, > >> > > >> -------------- > >> > > >> Lin > >> > > >>> Hi Lin, > >> > > >>> > >> > > >>> Adding in hotspot-gc-dev as they need to see how this interacts with GC > >> > > >>> worker threads, and whether it needs to be extended beyond G1. > >> > > >>> > >> > > >>> I happened to spot one nit when browsing: > >> > > >>> > >> > > >>> src/hotspot/share/gc/shared/collectedHeap.hpp > >> > > >>> > >> > > >>> + virtual bool run_par_heap_inspect_task(KlassInfoTable* cit, > >> > > >>> + BoolObjectClosure* filter, > >> > > >>> + size_t* missed_count, > >> > > >>> + size_t thread_num) { > >> > > >>> + return NULL; > >> > > >>> > >> > > >>> s/NULL/false/ > >> > > >>> > >> > > >>> Cheers, > >> > > >>> David > > > > > >>> > >> > > >>> On 18/02/2020 2:15 pm, linzang(??) wrote: > >> > > >>>> Dear All, > >> > > >>>> May I ask your help to review the follow changes: > >> > > >>>> webrev: > >> > > >>>> http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_00/ > >> > > >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8215624 > >> > > >>>> related CSR: https://bugs.openjdk.java.net/browse/JDK-8239290 > >> > > >>>> This patch enable parallel heap inspection of G1 for jmap histo. > >> > > >>>> my simple test shown it can speed up 2x of jmap -histo with > >> > > >>>> parallelThreadNum set to 2 for heap at ~500M on 4-core platform. > >> > > >>>> > >> > > >>>> ------------------------------------------------------------------------ > >> > > >>>> BRs, > >> > > >>>> Lin > >> > > >> > > >> > > > > > > > > > > > > > > > From sgehwolf at redhat.com Wed Jul 15 09:47:25 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Wed, 15 Jul 2020 11:47:25 +0200 Subject: [PING?] RFR(s): 8247863: Unreachable code in OperatingSystemImpl.getTotalSwapSpaceSize() In-Reply-To: References: Message-ID: <40bd7c49d36c5775d86b9d2ff5740b50cf4359e9.camel@redhat.com> Anyone? On Mon, 2020-06-29 at 17:53 +0200, Severin Gehwolf wrote: > Hi, > > Could I please get a review of this dead-code removal? During review of > JDK-8244500 it was discovered that with the new cgroups implementation > supporting v1 and v2 Metrics.getMemoryAndSwapLimit() will never return > 0 when relevant cgroup files are missing. E.g. on a system where the > kernel doesn't support swap limit capabilities. Therefore this code > introduced with JDK-8236617 can no longer be reached and should get > removed. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8247863 > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8247863/01/webrev/ > > Testing: Matthias tested this on the affected system and it did pass > for him. Docker tests on cgroup v1 and cgroup v2. > > Thanks, > Severin From matthias.baesken at sap.com Wed Jul 15 15:22:18 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 15 Jul 2020 15:22:18 +0000 Subject: [PING?] RFR(s): 8247863: Unreachable code in OperatingSystemImpl.getTotalSwapSpaceSize() In-Reply-To: <40bd7c49d36c5775d86b9d2ff5740b50cf4359e9.camel@redhat.com> References: <40bd7c49d36c5775d86b9d2ff5740b50cf4359e9.camel@redhat.com> Message-ID: Hello Severin , > the new cgroups implementation > supporting v1 and v2 Metrics.getMemoryAndSwapLimit() will never return 0 Wouldn?t it be possible that the coding of getMemoryAndSwapLimit returns a negative value (might not happen on a "healthy" system but you never know) : jdk/src/java.base/linux/classes/jdk/internal/platform/cgroupv1/CgroupV1Subsystem.java 444 public long getMemoryAndSwapLimit() { 445 long retval = getLongValue(memory, "memory.memsw.limit_in_bytes"); 446 if (retval > CgroupV1SubsystemController.UNLIMITED_MIN) { 447 if (memory.isHierarchical()) { 448 // memory.memsw.limit_in_bytes returned unlimited, attempt 449 // hierarchical memory limit 450 String match = "hierarchical_memsw_limit"; 451 retval = CgroupV1SubsystemController.getLongValueMatchingLine(memory, 452 "memory.stat", 453 match); 454 } 455 } 456 return CgroupV1SubsystemController.longValOrUnlimited(retval); 457 } jdk/src/java.base/linux/classes/jdk/internal/platform/cgroupv2/CgroupV2Subsystem.java 278 public long getMemoryAndSwapLimit() { 279 String strVal = CgroupSubsystemController.getStringValue(unified, "memory.swap.max"); 280 return limitFromString(strVal); 281 } So the check you want to clean up does no harm, and might handle "strange" cases - why not keep it? Best regards, Matthias -----Original Message----- From: Severin Gehwolf Sent: Mittwoch, 15. Juli 2020 11:47 To: core-libs-dev ; serviceability-dev Cc: Baesken, Matthias ; Bob Vandette Subject: Re: [PING?] RFR(s): 8247863: Unreachable code in OperatingSystemImpl.getTotalSwapSpaceSize() Anyone? On Mon, 2020-06-29 at 17:53 +0200, Severin Gehwolf wrote: > Hi, > > Could I please get a review of this dead-code removal? During review of > JDK-8244500 it was discovered that with the new cgroups implementation > supporting v1 and v2 Metrics.getMemoryAndSwapLimit() will never return > 0 when relevant cgroup files are missing. E.g. on a system where the > kernel doesn't support swap limit capabilities. Therefore this code > introduced with JDK-8236617 can no longer be reached and should get > removed. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8247863 > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8247863/01/webrev/ > > Testing: Matthias tested this on the affected system and it did pass > for him. Docker tests on cgroup v1 and cgroup v2. > > Thanks, > Severin From coleen.phillimore at oracle.com Wed Jul 15 15:38:30 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 15 Jul 2020 11:38:30 -0400 Subject: RFR 8247808: Move JVMTI strong oops to OopStorage In-Reply-To: <5051e29d-f519-f270-3a55-b8d2d0a02854@oracle.com> References: <890246ae-2d56-95ff-c360-dd66532579a8@oracle.com> <5051e29d-f519-f270-3a55-b8d2d0a02854@oracle.com> Message-ID: <14670943-33c3-952c-3091-40afc21ecda4@oracle.com> Hi, This patch has been reviewed and I was waiting for the ability to define different OopStorages, but I'd like to fix that in a further change after the GC changes have been agreed upon and reviewed.? Adding a new JVMTI OopStorage in the new mechanism is a smaller change. open webrev at http://cr.openjdk.java.net/~coleenp/2020/8247808.01/webrev Retested with tier1-3. Thanks, Coleen On 6/18/20 3:48 PM, coleen.phillimore at oracle.com wrote: > > > On 6/18/20 3:58 AM, Thomas Schatzl wrote: >> Hi, >> >> On 18.06.20 03:09, coleen.phillimore at oracle.com wrote: >>> >>> >>> On 6/17/20 7:49 PM, David Holmes wrote: >>>> Hi Coleen, >>>> >>>> On 18/06/2020 7:25 am, coleen.phillimore at oracle.com wrote: >>>>> Summary: Remove JVMTI oops_do calls from JVMTI and GCs >>>>> >>>>> Tested with tier1-3, also built shenandoah to verify shenandoah >>>>> changes. >>>>> >> [...] >>> >>> Kim noticed that G1 and ParallelGC should be processing these roots >>> in parallel (with many threads, since OopStorage has that support) >>> and he's going to or has filed a bug to fix it. As we add more >>> things to OopStorage (see upcoming RFRs), this will become important. >>> >> >> I do not know which exact roots you want to move into OopStorage, but >> I would like to mention this concern: with moving everything into a >> single OopStorage (i.e. vm_globals in this case), I am worried that >> every time important information about the source for these gets lost. >> >> Which makes it hard to understand from where these oops came from >> when there is a performance problem in the "VM Globals" bucket. > Hi Thomas, > > I understand this concern.? On the GC list there is a discussion about > having the ability to create different strong OopStorages, changing > the OopStorage code to process these roots and report statistics in > parallel (and/or concurrent), and not having to cascade the code > through all the GCs. > > I'm going to hold this change until this discussion is complete and > move the JVMTI and services/management oops_do oops into a different > OopStorage that can make use of this.? Then you'll have your > statistics and we won't have classes needing traversal with oops_do. > > Thanks, > Coleen > >> >> This may not apply to JVMTI oops, but others may occasionally have a >> significant amount of oops where it would be very interesting to know >> from where a particular slowdown comes from. >> >> So I would prefer keep some accounting here. >> >> Thanks, >> ? Thomas > From serguei.spitsyn at oracle.com Wed Jul 15 17:33:08 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 15 Jul 2020 10:33:08 -0700 Subject: RFR 8247808: Move JVMTI strong oops to OopStorage In-Reply-To: <14670943-33c3-952c-3091-40afc21ecda4@oracle.com> References: <890246ae-2d56-95ff-c360-dd66532579a8@oracle.com> <5051e29d-f519-f270-3a55-b8d2d0a02854@oracle.com> <14670943-33c3-952c-3091-40afc21ecda4@oracle.com> Message-ID: Hi Coleen, The update looks okay to me. Also, I wonder what should happen to the JvmtiExport::weak_oops_do(). Thanks, Serguei On 7/15/20 08:38, coleen.phillimore at oracle.com wrote: > > Hi, This patch has been reviewed and I was waiting for the ability to > define different OopStorages, but I'd like to fix that in a further > change after the GC changes have been agreed upon and reviewed.? > Adding a new JVMTI OopStorage in the new mechanism is a smaller change. > > open webrev at http://cr.openjdk.java.net/~coleenp/2020/8247808.01/webrev > > Retested with tier1-3. > > Thanks, > Coleen > > > > On 6/18/20 3:48 PM, coleen.phillimore at oracle.com wrote: >> >> >> On 6/18/20 3:58 AM, Thomas Schatzl wrote: >>> Hi, >>> >>> On 18.06.20 03:09, coleen.phillimore at oracle.com wrote: >>>> >>>> >>>> On 6/17/20 7:49 PM, David Holmes wrote: >>>>> Hi Coleen, >>>>> >>>>> On 18/06/2020 7:25 am, coleen.phillimore at oracle.com wrote: >>>>>> Summary: Remove JVMTI oops_do calls from JVMTI and GCs >>>>>> >>>>>> Tested with tier1-3, also built shenandoah to verify shenandoah >>>>>> changes. >>>>>> >>> [...] >>>> >>>> Kim noticed that G1 and ParallelGC should be processing these roots >>>> in parallel (with many threads, since OopStorage has that support) >>>> and he's going to or has filed a bug to fix it. As we add more >>>> things to OopStorage (see upcoming RFRs), this will become important. >>>> >>> >>> I do not know which exact roots you want to move into OopStorage, >>> but I would like to mention this concern: with moving everything >>> into a single OopStorage (i.e. vm_globals in this case), I am >>> worried that every time important information about the source for >>> these gets lost. >>> >>> Which makes it hard to understand from where these oops came from >>> when there is a performance problem in the "VM Globals" bucket. >> Hi Thomas, >> >> I understand this concern.? On the GC list there is a discussion >> about having the ability to create different strong OopStorages, >> changing the OopStorage code to process these roots and report >> statistics in parallel (and/or concurrent), and not having to cascade >> the code through all the GCs. >> >> I'm going to hold this change until this discussion is complete and >> move the JVMTI and services/management oops_do oops into a different >> OopStorage that can make use of this.? Then you'll have your >> statistics and we won't have classes needing traversal with oops_do. >> >> Thanks, >> Coleen >> >>> >>> This may not apply to JVMTI oops, but others may occasionally have a >>> significant amount of oops where it would be very interesting to >>> know from where a particular slowdown comes from. >>> >>> So I would prefer keep some accounting here. >>> >>> Thanks, >>> ? Thomas >> > From zgu at redhat.com Wed Jul 15 18:35:41 2020 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 15 Jul 2020 14:35:41 -0400 Subject: RFR 8247808: Move JVMTI strong oops to OopStorage In-Reply-To: <14670943-33c3-952c-3091-40afc21ecda4@oracle.com> References: <890246ae-2d56-95ff-c360-dd66532579a8@oracle.com> <5051e29d-f519-f270-3a55-b8d2d0a02854@oracle.com> <14670943-33c3-952c-3091-40afc21ecda4@oracle.com> Message-ID: <008cb366-a64f-a7ab-c70a-bbae09478b92@redhat.com> Hi Coleen, Shenandoah part looks good. Thanks, -Zhengyu On 7/15/20 11:38 AM, coleen.phillimore at oracle.com wrote: > > Hi, This patch has been reviewed and I was waiting for the ability to > define different OopStorages, but I'd like to fix that in a further > change after the GC changes have been agreed upon and reviewed.? Adding > a new JVMTI OopStorage in the new mechanism is a smaller change. > > open webrev at http://cr.openjdk.java.net/~coleenp/2020/8247808.01/webrev > > Retested with tier1-3. > > Thanks, > Coleen > > > > On 6/18/20 3:48 PM, coleen.phillimore at oracle.com wrote: >> >> >> On 6/18/20 3:58 AM, Thomas Schatzl wrote: >>> Hi, >>> >>> On 18.06.20 03:09, coleen.phillimore at oracle.com wrote: >>>> >>>> >>>> On 6/17/20 7:49 PM, David Holmes wrote: >>>>> Hi Coleen, >>>>> >>>>> On 18/06/2020 7:25 am, coleen.phillimore at oracle.com wrote: >>>>>> Summary: Remove JVMTI oops_do calls from JVMTI and GCs >>>>>> >>>>>> Tested with tier1-3, also built shenandoah to verify shenandoah >>>>>> changes. >>>>>> >>> [...] >>>> >>>> Kim noticed that G1 and ParallelGC should be processing these roots >>>> in parallel (with many threads, since OopStorage has that support) >>>> and he's going to or has filed a bug to fix it. As we add more >>>> things to OopStorage (see upcoming RFRs), this will become important. >>>> >>> >>> I do not know which exact roots you want to move into OopStorage, but >>> I would like to mention this concern: with moving everything into a >>> single OopStorage (i.e. vm_globals in this case), I am worried that >>> every time important information about the source for these gets lost. >>> >>> Which makes it hard to understand from where these oops came from >>> when there is a performance problem in the "VM Globals" bucket. >> Hi Thomas, >> >> I understand this concern.? On the GC list there is a discussion about >> having the ability to create different strong OopStorages, changing >> the OopStorage code to process these roots and report statistics in >> parallel (and/or concurrent), and not having to cascade the code >> through all the GCs. >> >> I'm going to hold this change until this discussion is complete and >> move the JVMTI and services/management oops_do oops into a different >> OopStorage that can make use of this.? Then you'll have your >> statistics and we won't have classes needing traversal with oops_do. >> >> Thanks, >> Coleen >> >>> >>> This may not apply to JVMTI oops, but others may occasionally have a >>> significant amount of oops where it would be very interesting to know >>> from where a particular slowdown comes from. >>> >>> So I would prefer keep some accounting here. >>> >>> Thanks, >>> ? Thomas >> > From igor.ignatyev at oracle.com Wed Jul 15 18:37:55 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 15 Jul 2020 11:37:55 -0700 Subject: [15] RFR : 8249040 : clean up FileInstaller $test.src $cwd in vmTestbase_nsk_jdb tests In-Reply-To: References: Message-ID: <45B4D401-D97B-4094-887A-FF71EF480593@oracle.com> Hi David, thanks for your review, pushed to jdk15. -- Igor > On Jul 14, 2020, at 9:01 PM, David Holmes wrote: > > Hi Igor, > > Looks good to me. > > On 15/07/2020 9:29 am, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev/8249040/webrev.00/ >>> 135 lines changed: 2 ins; 63 del; 70 mod; >> Hi all, >> could you please review the clean-up of nsk_jdb tests? >> from main issue(8204985) : >>> all vmTestbase tests have '@run driver jdk.test.lib.FileInstaller . .' to mimic old test harness behavior and copy all files from a test source directory to a current work directory. some tests depend on this step, so we need 1st identify such tests and then either rewrite them not to have this dependency or leave FileInstaller only in these tests. >> the patch removes FileInstaller actions in all the tests. as in all previous patches, the biggest (and tedious) part of the patch is just `ag -l '@run driver jdk.test.lib.FileInstaller . .' $DIR | xargs -I{} gsed -i '/@run driver jdk.test.lib.FileInstaller \. \./d' {}` with $DIR being test/hotspot/jtreg/vmTestbase/nsk/jdb/; two tests however required manual intervention: nsk/jdb/read/read001 and use001/use001 were misusing `-workdir` option. these tests expected this option to point to the directory w/ test's source code. 'workdir' option is processed by nsk/share/jdb/JdbArgumentHandler and is meant to store "full path to current test directory", which obviously doesn't have to be in the source tree. the tests were updated to use `test.src` property value instead of `workdir`. > > All the commented out stuff in use001 is a little confusing, but not due to this change of course. true, yet that's not the most awful code you can find in vmTestbase ;) I'll let it for svc team to decide if/when/how they want to clean this up. > > > Thanks, > David > >> testing: :vmTestbase_nsk_jdb on linux-x64 >> JBS: https://bugs.openjdk.java.net/browse/JDK-8249040 >> webrev: http://cr.openjdk.java.net/~iignatyev/8249040/webrev.00/ >> Thanks, >> -- Igor -------------- next part -------------- An HTML attachment was scrubbed... URL: From igor.ignatyev at oracle.com Wed Jul 15 18:38:08 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 15 Jul 2020 11:38:08 -0700 Subject: RFR [15] : 8249034 : clean up FileInstaller $test.src $cwd in vmTestbase_nsk_jvmti tests In-Reply-To: <1ccb7b00-1330-5cd7-e266-11aa920034c4@oracle.com> References: <1ccb7b00-1330-5cd7-e266-11aa920034c4@oracle.com> Message-ID: <4D3EC71E-8B87-4095-979D-482CD7F39D18@oracle.com> Thanks Serguei, pushed to jdk15. -- Igor > On Jul 14, 2020, at 5:38 PM, serguei.spitsyn at oracle.com wrote: > > Hi Igor, > > LGTM. > > Thanks, > Serguei > > > On 7/13/20 16:22, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev/8249034/webrev.00/ >>> 1289 lines changed: 2 ins; 652 del; 635 mod; >> Hi all, >> >> could you please review this clean-up which removes `FileInstaller` actions from :vmTestbase_nsk_jvmti tests? >> from the main issue(8204985): >>> all vmTestbase tests have '@run driver jdk.test.lib.FileInstaller . .' to mimic old test harness behavior and copy all files from a test source directory to a current work directory. some tests depend on this step, so we need 1st identify such tests and then either rewrite them not to have this dependency or leave FileInstaller only in these tests. >> `ag -l '@run driver jdk.test.lib.FileInstaller . .' vmTestbase/nsk/jvmti vmTestbase/nsk/share/ExceptionCheckingJniEnv xargs -I{} gsed -i '/@run driver jdk.test.lib.FileInstaller \. \./d' {}` was used to remove FileInstaller actions from all nsk_jvmti tests. 3 tests had dependency on that action: >> - Allocate/alloc001, where alloc001.sh was assuming that alloc001.bash is in the current directory. alloc001.sh has been updated to use ${TESTSRC} env variable >> - AttachOnDemand/attach002a and scenarios/bcinstr/BI04/bi04t002: both tests assumed that newclassXX directory is located in the current directory. the tests were fixed by prefixing 'newclassXX' w/ '${test.src}/' which gets expanded by PropertyResolvingWrapper >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8249034 >> webrev: http://cr.openjdk.java.net/~iignatyev//8249034/webrev.00/ >> testing: :vmTestbase_nsk_jvmti on linux-x64 >> >> Thanks, >> -- Igor > From igor.ignatyev at oracle.com Wed Jul 15 18:38:22 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 15 Jul 2020 11:38:22 -0700 Subject: RFR [15] : 8249039 : clean up FileInstaller $test.src $cwd in vmTestbase_nsk_aod tests In-Reply-To: <2273f689-6229-f710-1d35-2b2e15f5cd34@oracle.com> References: <4A9D95BB-E8AD-42B7-BC92-71C16C0F2B40@oracle.com> <4bedfc1c-8fd3-7486-4d67-32e76ac57409@oracle.com> <2273f689-6229-f710-1d35-2b2e15f5cd34@oracle.com> Message-ID: <6ACEB6F0-05AB-45EA-BFCF-A9C9B28C7E32@oracle.com> Serguei, David, thanks for your review! pushed to jdk15. -- Igor > On Jul 14, 2020, at 5:25 PM, serguei.spitsyn at oracle.com wrote: > > Hi Igor, > > LGTM++ > > Thanks, > Serguei > > > On 7/14/20 16:41, David Holmes wrote: >> Hi Igor, >> >> LGTM. >> >> (Sorry I skipped this one yesterday. :) ) (That's fine, BTW thanks a lot for reviewing all those clean ups) >> >> Thanks, >> David >> >> On 14/07/2020 8:29 am, Igor Ignatyev wrote: >>> http://cr.openjdk.java.net/~iignatyev/8249039/webrev.00/ >>>> 38 lines changed: 8 ins; 13 del; 17 mod; >>> >>> Hi all, >>> >>> could you please review this clean-up which removes `FileInstaller` actions from :vmTestbase_nsk_aod tests? >>> from the main issue(8204985): >>>> all vmTestbase tests have '@run driver jdk.test.lib.FileInstaller . .' to mimic old test harness behavior and copy all files from a test source directory to a current work directory. some tests depend on this step, so we need 1st identify such tests and then either rewrite them not to have this dependency or leave FileInstaller only in these tests. >>> >>> VirtualMachine06 test assumed that manifest files are located in the current directory, the test has been updated to have ${test.src} in the path to manifests. all other vmTestbase_nsk_stress tests don't need FileInstaller, so the rest of the patch is produced by `ag -l '@run driver jdk.test.lib.FileInstaller . .' vmTestbase/nsk/aod xargs -I{} gsed -i '/@run driver jdk.test.lib.FileInstaller \. \./d' {}`. >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8249039 >>> webrev: http://cr.openjdk.java.net/~iignatyev//8249039/webrev.00/ >>> testing: :vmTestbase_nsk_aod on linux-x64 >>> >>> Thanks, >>> -- Igor >>> > From coleen.phillimore at oracle.com Wed Jul 15 19:45:32 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 15 Jul 2020 15:45:32 -0400 Subject: RFR 8247808: Move JVMTI strong oops to OopStorage In-Reply-To: References: <890246ae-2d56-95ff-c360-dd66532579a8@oracle.com> <5051e29d-f519-f270-3a55-b8d2d0a02854@oracle.com> <14670943-33c3-952c-3091-40afc21ecda4@oracle.com> Message-ID: <730b30d5-db2f-6a15-82e7-326b19b6c1b8@oracle.com> Thank you for reviewing this, Serguei. On 7/15/20 1:33 PM, serguei.spitsyn at oracle.com wrote: > Hi Coleen, > > The update looks okay to me. > Also, I wonder what should happen to the JvmtiExport::weak_oops_do(). Unfortunately, JvmtiExport::weak_oops_do() calls JvmtiTagMap::weak_oops_do which ends up doing more than just doing GC on the weak oops in the hashtable that is stored for object tagging.? Since the hash code for the objects tagged is the address of the oop, GC also has to rehash the objects if they've been moved. I had a patch once to try go fix this to use weak OopStorage and use object->identity_hash() but hashing all the objects that JVMTI was trying to tag didn't turn out to be a good thing to do.? I ended up abandoning that change. Thanks, Coleen > > Thanks, > Serguei > > > On 7/15/20 08:38, coleen.phillimore at oracle.com wrote: >> >> Hi, This patch has been reviewed and I was waiting for the ability to >> define different OopStorages, but I'd like to fix that in a further >> change after the GC changes have been agreed upon and reviewed.? >> Adding a new JVMTI OopStorage in the new mechanism is a smaller change. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8247808.01/webrev >> >> Retested with tier1-3. >> >> Thanks, >> Coleen >> >> >> >> On 6/18/20 3:48 PM, coleen.phillimore at oracle.com wrote: >>> >>> >>> On 6/18/20 3:58 AM, Thomas Schatzl wrote: >>>> Hi, >>>> >>>> On 18.06.20 03:09, coleen.phillimore at oracle.com wrote: >>>>> >>>>> >>>>> On 6/17/20 7:49 PM, David Holmes wrote: >>>>>> Hi Coleen, >>>>>> >>>>>> On 18/06/2020 7:25 am, coleen.phillimore at oracle.com wrote: >>>>>>> Summary: Remove JVMTI oops_do calls from JVMTI and GCs >>>>>>> >>>>>>> Tested with tier1-3, also built shenandoah to verify shenandoah >>>>>>> changes. >>>>>>> >>>> [...] >>>>> >>>>> Kim noticed that G1 and ParallelGC should be processing these >>>>> roots in parallel (with many threads, since OopStorage has that >>>>> support) and he's going to or has filed a bug to fix it. As we add >>>>> more things to OopStorage (see upcoming RFRs), this will become >>>>> important. >>>>> >>>> >>>> I do not know which exact roots you want to move into OopStorage, >>>> but I would like to mention this concern: with moving everything >>>> into a single OopStorage (i.e. vm_globals in this case), I am >>>> worried that every time important information about the source for >>>> these gets lost. >>>> >>>> Which makes it hard to understand from where these oops came from >>>> when there is a performance problem in the "VM Globals" bucket. >>> Hi Thomas, >>> >>> I understand this concern.? On the GC list there is a discussion >>> about having the ability to create different strong OopStorages, >>> changing the OopStorage code to process these roots and report >>> statistics in parallel (and/or concurrent), and not having to >>> cascade the code through all the GCs. >>> >>> I'm going to hold this change until this discussion is complete and >>> move the JVMTI and services/management oops_do oops into a different >>> OopStorage that can make use of this. Then you'll have your >>> statistics and we won't have classes needing traversal with oops_do. >>> >>> Thanks, >>> Coleen >>> >>>> >>>> This may not apply to JVMTI oops, but others may occasionally have >>>> a significant amount of oops where it would be very interesting to >>>> know from where a particular slowdown comes from. >>>> >>>> So I would prefer keep some accounting here. >>>> >>>> Thanks, >>>> ? Thomas >>> >> > From coleen.phillimore at oracle.com Wed Jul 15 19:45:52 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 15 Jul 2020 15:45:52 -0400 Subject: RFR 8247808: Move JVMTI strong oops to OopStorage In-Reply-To: <008cb366-a64f-a7ab-c70a-bbae09478b92@redhat.com> References: <890246ae-2d56-95ff-c360-dd66532579a8@oracle.com> <5051e29d-f519-f270-3a55-b8d2d0a02854@oracle.com> <14670943-33c3-952c-3091-40afc21ecda4@oracle.com> <008cb366-a64f-a7ab-c70a-bbae09478b92@redhat.com> Message-ID: <07a90bcc-9ceb-182d-0a54-ca8c38e32bdc@oracle.com> Thank you Zhengyu. Coleen On 7/15/20 2:35 PM, Zhengyu Gu wrote: > Hi Coleen, > > Shenandoah part looks good. > > Thanks, > > -Zhengyu > > On 7/15/20 11:38 AM, coleen.phillimore at oracle.com wrote: >> >> Hi, This patch has been reviewed and I was waiting for the ability to >> define different OopStorages, but I'd like to fix that in a further >> change after the GC changes have been agreed upon and reviewed.? >> Adding a new JVMTI OopStorage in the new mechanism is a smaller change. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8247808.01/webrev >> >> Retested with tier1-3. >> >> Thanks, >> Coleen >> >> >> >> On 6/18/20 3:48 PM, coleen.phillimore at oracle.com wrote: >>> >>> >>> On 6/18/20 3:58 AM, Thomas Schatzl wrote: >>>> Hi, >>>> >>>> On 18.06.20 03:09, coleen.phillimore at oracle.com wrote: >>>>> >>>>> >>>>> On 6/17/20 7:49 PM, David Holmes wrote: >>>>>> Hi Coleen, >>>>>> >>>>>> On 18/06/2020 7:25 am, coleen.phillimore at oracle.com wrote: >>>>>>> Summary: Remove JVMTI oops_do calls from JVMTI and GCs >>>>>>> >>>>>>> Tested with tier1-3, also built shenandoah to verify shenandoah >>>>>>> changes. >>>>>>> >>>> [...] >>>>> >>>>> Kim noticed that G1 and ParallelGC should be processing these >>>>> roots in parallel (with many threads, since OopStorage has that >>>>> support) and he's going to or has filed a bug to fix it. As we add >>>>> more things to OopStorage (see upcoming RFRs), this will become >>>>> important. >>>>> >>>> >>>> I do not know which exact roots you want to move into OopStorage, >>>> but I would like to mention this concern: with moving everything >>>> into a single OopStorage (i.e. vm_globals in this case), I am >>>> worried that every time important information about the source for >>>> these gets lost. >>>> >>>> Which makes it hard to understand from where these oops came from >>>> when there is a performance problem in the "VM Globals" bucket. >>> Hi Thomas, >>> >>> I understand this concern.? On the GC list there is a discussion >>> about having the ability to create different strong OopStorages, >>> changing the OopStorage code to process these roots and report >>> statistics in parallel (and/or concurrent), and not having to >>> cascade the code through all the GCs. >>> >>> I'm going to hold this change until this discussion is complete and >>> move the JVMTI and services/management oops_do oops into a different >>> OopStorage that can make use of this. Then you'll have your >>> statistics and we won't have classes needing traversal with oops_do. >>> >>> Thanks, >>> Coleen >>> >>>> >>>> This may not apply to JVMTI oops, but others may occasionally have >>>> a significant amount of oops where it would be very interesting to >>>> know from where a particular slowdown comes from. >>>> >>>> So I would prefer keep some accounting here. >>>> >>>> Thanks, >>>> ? Thomas >>> >> > From serguei.spitsyn at oracle.com Wed Jul 15 22:26:29 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 15 Jul 2020 15:26:29 -0700 Subject: RFR 8247808: Move JVMTI strong oops to OopStorage In-Reply-To: <730b30d5-db2f-6a15-82e7-326b19b6c1b8@oracle.com> References: <890246ae-2d56-95ff-c360-dd66532579a8@oracle.com> <5051e29d-f519-f270-3a55-b8d2d0a02854@oracle.com> <14670943-33c3-952c-3091-40afc21ecda4@oracle.com> <730b30d5-db2f-6a15-82e7-326b19b6c1b8@oracle.com> Message-ID: <2fd78eda-4713-3cde-9f38-2b32b96751eb@oracle.com> Hi Coleen, Thank you for the explanation. Thanks, Serguei On 7/15/20 12:45, coleen.phillimore at oracle.com wrote: > > Thank you for reviewing this, Serguei. > > On 7/15/20 1:33 PM, serguei.spitsyn at oracle.com wrote: >> Hi Coleen, >> >> The update looks okay to me. >> Also, I wonder what should happen to the JvmtiExport::weak_oops_do(). > > Unfortunately, JvmtiExport::weak_oops_do() calls > JvmtiTagMap::weak_oops_do which ends up doing more than just doing GC > on the weak oops in the hashtable that is stored for object tagging.? > Since the hash code for the objects tagged is the address of the oop, > GC also has to rehash the objects if they've been moved. > > I had a patch once to try go fix this to use weak OopStorage and use > object->identity_hash() but hashing all the objects that JVMTI was > trying to tag didn't turn out to be a good thing to do. I ended up > abandoning that change. > > Thanks, > Coleen >> >> Thanks, >> Serguei >> >> >> On 7/15/20 08:38, coleen.phillimore at oracle.com wrote: >>> >>> Hi, This patch has been reviewed and I was waiting for the ability >>> to define different OopStorages, but I'd like to fix that in a >>> further change after the GC changes have been agreed upon and >>> reviewed.? Adding a new JVMTI OopStorage in the new mechanism is a >>> smaller change. >>> >>> open webrev at >>> http://cr.openjdk.java.net/~coleenp/2020/8247808.01/webrev >>> >>> Retested with tier1-3. >>> >>> Thanks, >>> Coleen >>> >>> >>> >>> On 6/18/20 3:48 PM, coleen.phillimore at oracle.com wrote: >>>> >>>> >>>> On 6/18/20 3:58 AM, Thomas Schatzl wrote: >>>>> Hi, >>>>> >>>>> On 18.06.20 03:09, coleen.phillimore at oracle.com wrote: >>>>>> >>>>>> >>>>>> On 6/17/20 7:49 PM, David Holmes wrote: >>>>>>> Hi Coleen, >>>>>>> >>>>>>> On 18/06/2020 7:25 am, coleen.phillimore at oracle.com wrote: >>>>>>>> Summary: Remove JVMTI oops_do calls from JVMTI and GCs >>>>>>>> >>>>>>>> Tested with tier1-3, also built shenandoah to verify shenandoah >>>>>>>> changes. >>>>>>>> >>>>> [...] >>>>>> >>>>>> Kim noticed that G1 and ParallelGC should be processing these >>>>>> roots in parallel (with many threads, since OopStorage has that >>>>>> support) and he's going to or has filed a bug to fix it. As we >>>>>> add more things to OopStorage (see upcoming RFRs), this will >>>>>> become important. >>>>>> >>>>> >>>>> I do not know which exact roots you want to move into OopStorage, >>>>> but I would like to mention this concern: with moving everything >>>>> into a single OopStorage (i.e. vm_globals in this case), I am >>>>> worried that every time important information about the source for >>>>> these gets lost. >>>>> >>>>> Which makes it hard to understand from where these oops came from >>>>> when there is a performance problem in the "VM Globals" bucket. >>>> Hi Thomas, >>>> >>>> I understand this concern.? On the GC list there is a discussion >>>> about having the ability to create different strong OopStorages, >>>> changing the OopStorage code to process these roots and report >>>> statistics in parallel (and/or concurrent), and not having to >>>> cascade the code through all the GCs. >>>> >>>> I'm going to hold this change until this discussion is complete and >>>> move the JVMTI and services/management oops_do oops into a >>>> different OopStorage that can make use of this. Then you'll have >>>> your statistics and we won't have classes needing traversal with >>>> oops_do. >>>> >>>> Thanks, >>>> Coleen >>>> >>>>> >>>>> This may not apply to JVMTI oops, but others may occasionally have >>>>> a significant amount of oops where it would be very interesting to >>>>> know from where a particular slowdown comes from. >>>>> >>>>> So I would prefer keep some accounting here. >>>>> >>>>> Thanks, >>>>> ? Thomas >>>> >>> >> > From david.holmes at oracle.com Wed Jul 15 23:50:35 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 16 Jul 2020 09:50:35 +1000 Subject: [15] RFR: 8246381: VM crashes with "Current BasicObjectLock* below than low_mark" In-Reply-To: <30bd811e-c890-5bb1-8c78-4cf944fd5a42@oracle.com> References: <7a802330-e836-1ff3-af0a-ede587e049ff@oracle.com> <30bd811e-c890-5bb1-8c78-4cf944fd5a42@oracle.com> Message-ID: <5d43f963-b931-3b69-4b5c-188c45b57de8@oracle.com> Hi Jamsheed, On 16/07/2020 8:16 am, Jamsheed C M wrote: > (Thank you Dean, adding serviceability team as this issue involves JVMTI > features PopFrame, EarlyReturn features) It is not at all obvious how your proposed fix impacts the JVM TI features. > JBS entry: https://bugs.openjdk.java.net/browse/JDK-8246381 > > (testing: mach5, tier1-5 links in JBS) > > Best regards, > > Jamsheed > > On 15/07/2020 21:25, Jamsheed C M wrote: >> >> Hi, >> >> Async handling at method entry requires it to be aware of >> synchronization(like whether it is doing async handling before lock >> acquire or after) >> >> This is required as exception handler rely on this info for >> unlocking.? Async handling code never had this special condition >> handled and it worked most of the time as we were using biased locking >> which got disabled by [1] >> >> There was one other issue reported in similar time[2]. This issue got >> triggered in test case by [3], back to back extra safepoint after >> suspend and TLH for ThreadDeath. So in this setup both PopFrame >> request and Thread.Stop request happened together for the test >> scenario and it reached java method entry with pending_exception set. >> >> I have done a partial fix for the issue, mainly to handle production >> mode crash failures(do not unlock flag related ones) >> >> Fix detail: >> >> 1) I save restore the "do not unlock" flag in async handling. Sorry but you completely changed the fix compared to what we discussed and what I pre-reviewed! What happened to changing from JRT_ENTRY to JRT_ENTRY_NOASYNC? It is going to take me a lot of time and effort to determine that this save/restore of the "do not unlock flag" is actually correct and valid! >> >> 2) Return for floating pending exception for some cases(PopFrame, >> Early return related). This is debug(JVMTI) feature and floating >> exception can get cleaned just like that in present compiler request >> and deopt code. What part of the change addresses this? Thanks, David ----- >> >> webrev :http://cr.openjdk.java.net/~jcm/8246381/webrev.02/ >> >> There are more problems in these code areas, like we clear all >> exceptions in compilation request path(interpreter,c1), as well as >> deoptimization path. >> >> All these un-handled cases will be separately handled by >> https://bugs.openjdk.java.net/browse/JDK-8249451 >> >> Request for review. >> >> Best regards, >> >> Jamsheed >> >> [1]https://bugs.openjdk.java.net/browse/JDK-8231264 >> >> >> [2] https://bugs.openjdk.java.net/browse/JDK-8246727 >> >> [3] https://bugs.openjdk.java.net/browse/JDK-8221207 >> From david.holmes at oracle.com Thu Jul 16 01:07:33 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 16 Jul 2020 11:07:33 +1000 Subject: [15] RFR: 8246381: VM crashes with "Current BasicObjectLock* below than low_mark" In-Reply-To: <5d43f963-b931-3b69-4b5c-188c45b57de8@oracle.com> References: <7a802330-e836-1ff3-af0a-ede587e049ff@oracle.com> <30bd811e-c890-5bb1-8c78-4cf944fd5a42@oracle.com> <5d43f963-b931-3b69-4b5c-188c45b57de8@oracle.com> Message-ID: <1af60254-a239-c21f-68df-be9b65534e7f@oracle.com> Hi Jamsheed, tl;dr version: fix looks good. Thanks for working through things with me on this one. Long version ... for the sake of other reviewers (and myself) I'm going to walk through the problem scenario and how the fix addresses it, because the bug report is long and confusing and touches on a number of different issues with async exception handling. We are dealing with the code generated for Java method entry, and in particular for a synchronized Java method. We do a lot of things in the entry code before we actually lock the monitor and jump to the Java method. Some of those things include method profiling and the counter overflow check for the JIT. If an exception is thrown at this point, the logic to remove the activation would unlock the monitor - which we haven't actually locked yet! So we have the do_not_unlock_if_synchronized flag which is stored in the current JavaThread. We set that flag true so that if any exceptions result in activation removal, the removal logic won't try to unlock the monitor. Once we're ready to lock the monitor we set the flag back to false (note there is an implicit assumption here that monitor locking can never raise an exception). The problem arises with async exceptions, or more specifically the async exception that is raised due to an "unsafe access error". This is where a memory-mapped ByteBuffer causes an access violation (SEGV) due to a bad pointer. The signal handler simply sets a flag to indicate we encountered an "unsafe access error", adjusts the BCI to the next instruction and allows execution to proceed at the next instruction. It is then expected that the runtime will "soon" notice this pending unsafe access error and create and throw the InternalError instance that indicates the ByteBuffer operation failed. This requires executing Java code. One of the places that checks for that pending unsafe access error is in the destructor of the JRT_ENTRY wrapper that is used for the method profiling and counter overflow checking. This occurs whilst the do_not_unlock_if_synchronized flag is true, so the resulting InternalError won't result in an attempt to unlock the not-locked monitor. The problem is that creating the InternalError executes Java code - it calls constructors, which call methods etc. And some of those methods are synchronized. So the method entry logic for such a call will set do_not_unlock_if_synchronized to true, perform all the preamble related to the call, then set do_not_unlock_if_synchronized to false, lock the monitor and make the call. When construction completes the InternalError is thrown and we remove the activation for the method we had originally started to call. But now the do_not_unlock_if_synchronized flag has been reset to false by the nested Java method call, so we do in fact try to unlock a monitor that was never locked, and things break. This nesting problem is well known and we have a mechanism for dealing with - the UnlockFlagSaver. The actual logic executed for profiling methods and doing the counter overflow check contains the requisite UnlockFlagSaver to avoid the problem just outlined. Unfortunately the async exception is processed in the JRT_ENTRY wrapper, which is outside the scope of those UnlockFlagSaver helpers and so they don't help in this case. So the fix is to "simply" move the UnlockFlagSaver deeper into the call stack to the code that actually does the async exception processing: void JavaThread::check_and_handle_async_exceptions(bool check_unsafe_error) { + // May be we are at method entry and requires to save do not unlock flag. + UnlockFlagSaver fs(this); so now after the InternalError has been created and thrown we will restore the original value of the do_not_unlock_if_synchronized flag (false) and so the InternalError will not cause activation removal to attempt to unlock the not-locked monitor. The scope of the UnlockFlagSaver could be narrowed to the actual logic for processing the unsafe access error, but it seems fine at method scope. A second fix is that the overflow counter check had an assertion that it was not executed with any pending exceptions. But that turned out to be false for reasons I can't fully explain, but it again appears to relate to a pending async exception being installed prior to the method call - and seems related to the two referenced JVM TI functions. The simple solution here is to delete the assertion and to check for pending exceptions on entry to the code and just return immediately. The JRT_ENTRY destructor will see the pending exception and propagate it. Cheers, David On 16/07/2020 9:50 am, David Holmes wrote: > Hi Jamsheed, > > On 16/07/2020 8:16 am, Jamsheed C M wrote: >> (Thank you Dean, adding serviceability team as this issue involves >> JVMTI features PopFrame, EarlyReturn features) > > It is not at all obvious how your proposed fix impacts the JVM TI features. > >> JBS entry: https://bugs.openjdk.java.net/browse/JDK-8246381 >> >> (testing: mach5, tier1-5 links in JBS) >> >> Best regards, >> >> Jamsheed >> >> On 15/07/2020 21:25, Jamsheed C M wrote: >>> >>> Hi, >>> >>> Async handling at method entry requires it to be aware of >>> synchronization(like whether it is doing async handling before lock >>> acquire or after) >>> >>> This is required as exception handler rely on this info for >>> unlocking.? Async handling code never had this special condition >>> handled and it worked most of the time as we were using biased >>> locking which got disabled by [1] >>> >>> There was one other issue reported in similar time[2]. This issue got >>> triggered in test case by [3], back to back extra safepoint after >>> suspend and TLH for ThreadDeath. So in this setup both PopFrame >>> request and Thread.Stop request happened together for the test >>> scenario and it reached java method entry with pending_exception set. >>> >>> I have done a partial fix for the issue, mainly to handle production >>> mode crash failures(do not unlock flag related ones) >>> >>> Fix detail: >>> >>> 1) I save restore the "do not unlock" flag in async handling. > > Sorry but you completely changed the fix compared to what we discussed > and what I pre-reviewed! What happened to changing from JRT_ENTRY to > JRT_ENTRY_NOASYNC? It is going to take me a lot of time and effort to > determine that this save/restore of the "do not unlock flag" is actually > correct and valid! > >>> >>> 2) Return for floating pending exception for some cases(PopFrame, >>> Early return related). This is debug(JVMTI) feature and floating >>> exception can get cleaned just like that in present compiler request >>> and deopt code. > > What part of the change addresses this? > > Thanks, > David > ----- > >>> >>> webrev :http://cr.openjdk.java.net/~jcm/8246381/webrev.02/ >>> >>> There are more problems in these code areas, like we clear all >>> exceptions in compilation request path(interpreter,c1), as well as >>> deoptimization path. >>> >>> All these un-handled cases will be separately handled by >>> https://bugs.openjdk.java.net/browse/JDK-8249451 >>> >>> Request for review. >>> >>> Best regards, >>> >>> Jamsheed >>> >>> [1]https://bugs.openjdk.java.net/browse/JDK-8231264 >>> >>> >>> [2] https://bugs.openjdk.java.net/browse/JDK-8246727 >>> >>> [3] https://bugs.openjdk.java.net/browse/JDK-8221207 >>> From jamsheed.c.m at oracle.com Wed Jul 15 22:16:11 2020 From: jamsheed.c.m at oracle.com (Jamsheed C M) Date: Thu, 16 Jul 2020 03:46:11 +0530 Subject: [15] RFR: 8246381: VM crashes with "Current BasicObjectLock* below than low_mark" In-Reply-To: <7a802330-e836-1ff3-af0a-ede587e049ff@oracle.com> References: <7a802330-e836-1ff3-af0a-ede587e049ff@oracle.com> Message-ID: <30bd811e-c890-5bb1-8c78-4cf944fd5a42@oracle.com> (Thank you Dean, adding serviceability team as this issue involves JVMTI features PopFrame, EarlyReturn features) JBS entry: https://bugs.openjdk.java.net/browse/JDK-8246381 (testing: mach5, tier1-5 links in JBS) Best regards, Jamsheed On 15/07/2020 21:25, Jamsheed C M wrote: > > Hi, > > Async handling at method entry requires it to be aware of > synchronization(like whether it is doing async handling before lock > acquire or after) > > This is required as exception handler rely on this info for > unlocking.? Async handling code never had this special condition > handled and it worked most of the time as we were using biased locking > which got disabled by [1] > > There was one other issue reported in similar time[2]. This issue got > triggered in test case by [3], back to back extra safepoint after > suspend and TLH for ThreadDeath. So in this setup both PopFrame > request and Thread.Stop request happened together for the test > scenario and it reached java method entry with pending_exception set. > > I have done a partial fix for the issue, mainly to handle production > mode crash failures(do not unlock flag related ones) > > Fix detail: > > 1) I save restore the "do not unlock" flag in async handling. > > 2) Return for floating pending exception for some cases(PopFrame, > Early return related). This is debug(JVMTI) feature and floating > exception can get cleaned just like that in present compiler request > and deopt code. > > webrev :http://cr.openjdk.java.net/~jcm/8246381/webrev.02/ > > There are more problems in these code areas, like we clear all > exceptions in compilation request path(interpreter,c1), as well as > deoptimization path. > > All these un-handled cases will be separately handled by > https://bugs.openjdk.java.net/browse/JDK-8249451 > > Request for review. > > Best regards, > > Jamsheed > > [1]https://bugs.openjdk.java.net/browse/JDK-8231264 > > > [2] https://bugs.openjdk.java.net/browse/JDK-8246727 > > [3] https://bugs.openjdk.java.net/browse/JDK-8221207 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jamsheed.c.m at oracle.com Thu Jul 16 00:01:21 2020 From: jamsheed.c.m at oracle.com (Jamsheed C M) Date: Thu, 16 Jul 2020 05:31:21 +0530 Subject: [15] RFR: 8246381: VM crashes with "Current BasicObjectLock* below than low_mark" In-Reply-To: <5d43f963-b931-3b69-4b5c-188c45b57de8@oracle.com> References: <7a802330-e836-1ff3-af0a-ede587e049ff@oracle.com> <30bd811e-c890-5bb1-8c78-4cf944fd5a42@oracle.com> <5d43f963-b931-3b69-4b5c-188c45b57de8@oracle.com> Message-ID: <973c4e4c-ed0e-7152-8387-28243a3ac275@oracle.com> Hi David, On 16/07/2020 05:20, David Holmes wrote: > Hi Jamsheed, > > On 16/07/2020 8:16 am, Jamsheed C M wrote: >> (Thank you Dean, adding serviceability team as this issue involves >> JVMTI features PopFrame, EarlyReturn features) > > It is not at all obvious how your proposed fix impacts the JVM TI > features. Yes, proposed fix doesn't. Fix doesn't plan to address JVMTI feature related issues. Added just to keep everyone in the loop. Best regards, Jamsheed > >> JBS entry: https://bugs.openjdk.java.net/browse/JDK-8246381 >> >> (testing: mach5, tier1-5 links in JBS) >> >> Best regards, >> >> Jamsheed >> >> On 15/07/2020 21:25, Jamsheed C M wrote: >>> >>> Hi, >>> >>> Async handling at method entry requires it to be aware of >>> synchronization(like whether it is doing async handling before lock >>> acquire or after) >>> >>> This is required as exception handler rely on this info for >>> unlocking.? Async handling code never had this special condition >>> handled and it worked most of the time as we were using biased >>> locking which got disabled by [1] >>> >>> There was one other issue reported in similar time[2]. This issue >>> got triggered in test case by [3], back to back extra safepoint >>> after suspend and TLH for ThreadDeath. So in this setup both >>> PopFrame request and Thread.Stop request happened together for the >>> test scenario and it reached java method entry with >>> pending_exception set. >>> >>> I have done a partial fix for the issue, mainly to handle production >>> mode crash failures(do not unlock flag related ones) >>> >>> Fix detail: >>> >>> 1) I save restore the "do not unlock" flag in async handling. > > Sorry but you completely changed the fix compared to what we discussed > and what I pre-reviewed! What happened to changing from JRT_ENTRY to > JRT_ENTRY_NOASYNC? It is going to take me a lot of time and effort to > determine that this save/restore of the "do not unlock flag" is > actually correct and valid! > >>> >>> 2) Return for floating pending exception for some cases(PopFrame, >>> Early return related). This is debug(JVMTI) feature and floating >>> exception can get cleaned just like that in present compiler request >>> and deopt code. > > What part of the change addresses this? > > Thanks, > David > ----- > >>> >>> webrev :http://cr.openjdk.java.net/~jcm/8246381/webrev.02/ >>> >>> There are more problems in these code areas, like we clear all >>> exceptions in compilation request path(interpreter,c1), as well as >>> deoptimization path. >>> >>> All these un-handled cases will be separately handled by >>> https://bugs.openjdk.java.net/browse/JDK-8249451 >>> >>> Request for review. >>> >>> Best regards, >>> >>> Jamsheed >>> >>> [1]https://bugs.openjdk.java.net/browse/JDK-8231264 >>> >>> >>> [2] https://bugs.openjdk.java.net/browse/JDK-8246727 >>> >>> [3] https://bugs.openjdk.java.net/browse/JDK-8221207 >>> From jamsheed.c.m at oracle.com Thu Jul 16 00:37:25 2020 From: jamsheed.c.m at oracle.com (Jamsheed C M) Date: Thu, 16 Jul 2020 06:07:25 +0530 Subject: [15] RFR: 8246381: VM crashes with "Current BasicObjectLock* below than low_mark" In-Reply-To: <5d43f963-b931-3b69-4b5c-188c45b57de8@oracle.com> References: <7a802330-e836-1ff3-af0a-ede587e049ff@oracle.com> <30bd811e-c890-5bb1-8c78-4cf944fd5a42@oracle.com> <5d43f963-b931-3b69-4b5c-188c45b57de8@oracle.com> Message-ID: <122f8079-958c-acdf-bb60-3934729a313a@oracle.com> Hi David, On 16/07/2020 05:20, David Holmes wrote: >>> >>> Hi, >>> >>> Async handling at method entry requires it to be aware of >>> synchronization(like whether it is doing async handling before lock >>> acquire or after) >>> >>> This is required as exception handler rely on this info for >>> unlocking.? Async handling code never had this special condition >>> handled and it worked most of the time as we were using biased >>> locking which got disabled by [1] >>> >>> There was one other issue reported in similar time[2]. This issue >>> got triggered in test case by [3], back to back extra safepoint >>> after suspend and TLH for ThreadDeath. So in this setup both >>> PopFrame request and Thread.Stop request happened together for the >>> test scenario and it reached java method entry with >>> pending_exception set. >>> >>> I have done a partial fix for the issue, mainly to handle production >>> mode crash failures(do not unlock flag related ones) >>> >>> Fix detail: >>> >>> 1) I save restore the "do not unlock" flag in async handling. > > Sorry but you completely changed the fix compared to what we discussed > and what I pre-reviewed! What happened to changing from JRT_ENTRY to > JRT_ENTRY_NOASYNC? It is going to take me a lot of time and effort to > determine that this save/restore of the "do not unlock flag" is > actually correct and valid! I tried JRT_ENTRY to JRT_ENTRY_NOASYNC. but unfortunately that made some tests to fail(logs in JBS), I didn't investigate it in detail, but what I presume is pending_async_exception is set for those failing scenarios but as we have? disabled async handling in some prominent code paths, the exception is never delivered. >>> >>> 2) Return for floating pending exception for some cases(PopFrame, >>> Early return related). This is debug(JVMTI) feature and floating >>> exception can get cleaned just like that in present compiler request >>> and deopt code. > > What part of the change addresses this? It doesn't address this issue completely. As it requires other changes in compilation request path(c1,interpreter) and deopt. Just made changes to interpreter part(compilation request part). that fixes interpreter part partially. JRT_ENTRY(nmethod*, InterpreterRuntime::frequency_counter_overflow_inner(JavaThread* thread, address branch_bcp)) + if (HAS_PENDING_EXCEPTION) { + return NULL; + } JRT_ENTRY(void, InterpreterRuntime::profile_method(JavaThread* thread)) + if (HAS_PENDING_EXCEPTION) { + return; + } Best regards Jamsheed > > Thanks, > David > ----- -------------- next part -------------- An HTML attachment was scrubbed... URL: From jamsheed.c.m at oracle.com Thu Jul 16 02:03:31 2020 From: jamsheed.c.m at oracle.com (Jamsheed C M) Date: Thu, 16 Jul 2020 07:33:31 +0530 Subject: [15] RFR: 8246381: VM crashes with "Current BasicObjectLock* below than low_mark" In-Reply-To: <1af60254-a239-c21f-68df-be9b65534e7f@oracle.com> References: <7a802330-e836-1ff3-af0a-ede587e049ff@oracle.com> <30bd811e-c890-5bb1-8c78-4cf944fd5a42@oracle.com> <5d43f963-b931-3b69-4b5c-188c45b57de8@oracle.com> <1af60254-a239-c21f-68df-be9b65534e7f@oracle.com> Message-ID: Hi David, On 16/07/2020 06:37, David Holmes wrote: > Hi Jamsheed, > > tl;dr version: fix looks good. Thanks for working through things with > me on this one. > > Long version ... for the sake of other reviewers (and myself) I'm > going to walk through the problem scenario and how the fix addresses > it, because the bug report is long and confusing and touches on a > number of different issues with async exception handling. > > We are dealing with the code generated for Java method entry, and in > particular for a synchronized Java method. We do a lot of things in > the entry code before we actually lock the monitor and jump to the > Java method. Some of those things include method profiling and the > counter overflow check for the JIT. If an exception is thrown at this > point, the logic to remove the activation would unlock the monitor - > which we haven't actually locked yet! So we have the > do_not_unlock_if_synchronized flag which is stored in the current > JavaThread. We set that flag true so that if any exceptions result in > activation removal, the removal logic won't try to unlock the monitor. > Once we're ready to lock the monitor we set the flag back to false > (note there is an implicit assumption here that monitor locking can > never raise an exception). > > The problem arises with async exceptions, or more specifically the > async exception that is raised due to an "unsafe access error". This > is where a memory-mapped ByteBuffer causes an access violation (SEGV) > due to a bad pointer. The signal handler simply sets a flag to > indicate we encountered an "unsafe access error", adjusts the BCI to > the next instruction and allows execution to proceed at the next > instruction. It is then expected that the runtime will "soon" notice > this pending unsafe access error and create and throw the > InternalError instance that indicates the ByteBuffer operation failed. > This requires executing Java code. > > One of the places that checks for that pending unsafe access error is > in the destructor of the JRT_ENTRY wrapper that is used for the method > profiling and counter overflow checking. This occurs whilst the > do_not_unlock_if_synchronized flag is true, so the resulting > InternalError won't result in an attempt to unlock the not-locked > monitor. > > The problem is that creating the InternalError executes Java code - it > calls constructors, which call methods etc. And some of those methods > are synchronized. So the method entry logic for such a call will set > do_not_unlock_if_synchronized to true, perform all the preamble > related to the call, then set do_not_unlock_if_synchronized to false, > lock the monitor and make the call. When construction completes the > InternalError is thrown and we remove the activation for the method we > had originally started to call. But now the > do_not_unlock_if_synchronized flag has been reset to false by the > nested Java method call, so we do in fact try to unlock a monitor that > was never locked, and things break. > > This nesting problem is well known and we have a mechanism for dealing > with - the UnlockFlagSaver. The actual logic executed for profiling > methods and doing the counter overflow check contains the requisite > UnlockFlagSaver to avoid the problem just outlined. Unfortunately the > async exception is processed in the JRT_ENTRY wrapper, which is > outside the scope of those UnlockFlagSaver helpers and so they don't > help in this case. > > So the fix is to "simply" move the UnlockFlagSaver deeper into the > call stack to the code that actually does the async exception processing: > > ?void JavaThread::check_and_handle_async_exceptions(bool > check_unsafe_error) { > +?? // May be we are at method entry and requires to save do not > unlock flag. > +?? UnlockFlagSaver fs(this); > > so now after the InternalError has been created and thrown we will > restore the original value of the do_not_unlock_if_synchronized flag > (false) and so the InternalError will not cause activation removal to > attempt to unlock the not-locked monitor. > > The scope of the UnlockFlagSaver could be narrowed to the actual logic > for processing the unsafe access error, but it seems fine at method > scope. > > A second fix is that the overflow counter check had an assertion that > it was not executed with any pending exceptions. But that turned out > to be false for reasons I can't fully explain, but it again appears to > relate to a pending async exception being installed prior to the > method call - and seems related to the two referenced JVM TI > functions. The simple solution here is to delete the assertion and to > check for pending exceptions on entry to the code and just return > immediately. The JRT_ENTRY destructor will see the pending exception > and propagate it. Thanks a lot for the opportunity, for all the help, and for putting detailed description of the problem here. Best regards, Jamsheed > > Cheers, > David > > On 16/07/2020 9:50 am, David Holmes wrote: >> Hi Jamsheed, >> >> On 16/07/2020 8:16 am, Jamsheed C M wrote: >>> (Thank you Dean, adding serviceability team as this issue involves >>> JVMTI features PopFrame, EarlyReturn features) >> >> It is not at all obvious how your proposed fix impacts the JVM TI >> features. >> >>> JBS entry: https://bugs.openjdk.java.net/browse/JDK-8246381 >>> >>> (testing: mach5, tier1-5 links in JBS) >>> >>> Best regards, >>> >>> Jamsheed >>> >>> On 15/07/2020 21:25, Jamsheed C M wrote: >>>> >>>> Hi, >>>> >>>> Async handling at method entry requires it to be aware of >>>> synchronization(like whether it is doing async handling before lock >>>> acquire or after) >>>> >>>> This is required as exception handler rely on this info for >>>> unlocking.? Async handling code never had this special condition >>>> handled and it worked most of the time as we were using biased >>>> locking which got disabled by [1] >>>> >>>> There was one other issue reported in similar time[2]. This issue >>>> got triggered in test case by [3], back to back extra safepoint >>>> after suspend and TLH for ThreadDeath. So in this setup both >>>> PopFrame request and Thread.Stop request happened together for the >>>> test scenario and it reached java method entry with >>>> pending_exception set. >>>> >>>> I have done a partial fix for the issue, mainly to handle >>>> production mode crash failures(do not unlock flag related ones) >>>> >>>> Fix detail: >>>> >>>> 1) I save restore the "do not unlock" flag in async handling. >> >> Sorry but you completely changed the fix compared to what we >> discussed and what I pre-reviewed! What happened to changing from >> JRT_ENTRY to JRT_ENTRY_NOASYNC? It is going to take me a lot of time >> and effort to determine that this save/restore of the "do not unlock >> flag" is actually correct and valid! >> >>>> >>>> 2) Return for floating pending exception for some cases(PopFrame, >>>> Early return related). This is debug(JVMTI) feature and floating >>>> exception can get cleaned just like that in present compiler >>>> request and deopt code. >> >> What part of the change addresses this? >> >> Thanks, >> David >> ----- >> >>>> >>>> webrev :http://cr.openjdk.java.net/~jcm/8246381/webrev.02/ >>>> >>>> There are more problems in these code areas, like we clear all >>>> exceptions in compilation request path(interpreter,c1), as well as >>>> deoptimization path. >>>> >>>> All these un-handled cases will be separately handled by >>>> https://bugs.openjdk.java.net/browse/JDK-8249451 >>>> >>>> Request for review. >>>> >>>> Best regards, >>>> >>>> Jamsheed >>>> >>>> [1]https://bugs.openjdk.java.net/browse/JDK-8231264 >>>> >>>> >>>> [2] https://bugs.openjdk.java.net/browse/JDK-8246727 >>>> >>>> [3] https://bugs.openjdk.java.net/browse/JDK-8221207 >>>> From david.holmes at oracle.com Thu Jul 16 06:39:27 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 16 Jul 2020 16:39:27 +1000 Subject: RFR 8247808: Move JVMTI strong oops to OopStorage In-Reply-To: <14670943-33c3-952c-3091-40afc21ecda4@oracle.com> References: <890246ae-2d56-95ff-c360-dd66532579a8@oracle.com> <5051e29d-f519-f270-3a55-b8d2d0a02854@oracle.com> <14670943-33c3-952c-3091-40afc21ecda4@oracle.com> Message-ID: <82b9d699-42b0-a0ad-f1b9-62241f8754f4@oracle.com> Hi Coleen, On 16/07/2020 1:38 am, coleen.phillimore at oracle.com wrote: > > Hi, This patch has been reviewed and I was waiting for the ability to > define different OopStorages, but I'd like to fix that in a further > change after the GC changes have been agreed upon and reviewed.? Adding > a new JVMTI OopStorage in the new mechanism is a smaller change. > > open webrev at http://cr.openjdk.java.net/~coleenp/2020/8247808.01/webrev This all looks okay to me. I think you missed updating share/gc/shared/genCollectedHeap.hpp to delete GCH_PS_jvmti_oops_do, Thanks, David ----- > Retested with tier1-3. > > Thanks, > Coleen > > > > On 6/18/20 3:48 PM, coleen.phillimore at oracle.com wrote: >> >> >> On 6/18/20 3:58 AM, Thomas Schatzl wrote: >>> Hi, >>> >>> On 18.06.20 03:09, coleen.phillimore at oracle.com wrote: >>>> >>>> >>>> On 6/17/20 7:49 PM, David Holmes wrote: >>>>> Hi Coleen, >>>>> >>>>> On 18/06/2020 7:25 am, coleen.phillimore at oracle.com wrote: >>>>>> Summary: Remove JVMTI oops_do calls from JVMTI and GCs >>>>>> >>>>>> Tested with tier1-3, also built shenandoah to verify shenandoah >>>>>> changes. >>>>>> >>> [...] >>>> >>>> Kim noticed that G1 and ParallelGC should be processing these roots >>>> in parallel (with many threads, since OopStorage has that support) >>>> and he's going to or has filed a bug to fix it. As we add more >>>> things to OopStorage (see upcoming RFRs), this will become important. >>>> >>> >>> I do not know which exact roots you want to move into OopStorage, but >>> I would like to mention this concern: with moving everything into a >>> single OopStorage (i.e. vm_globals in this case), I am worried that >>> every time important information about the source for these gets lost. >>> >>> Which makes it hard to understand from where these oops came from >>> when there is a performance problem in the "VM Globals" bucket. >> Hi Thomas, >> >> I understand this concern.? On the GC list there is a discussion about >> having the ability to create different strong OopStorages, changing >> the OopStorage code to process these roots and report statistics in >> parallel (and/or concurrent), and not having to cascade the code >> through all the GCs. >> >> I'm going to hold this change until this discussion is complete and >> move the JVMTI and services/management oops_do oops into a different >> OopStorage that can make use of this.? Then you'll have your >> statistics and we won't have classes needing traversal with oops_do. >> >> Thanks, >> Coleen >> >>> >>> This may not apply to JVMTI oops, but others may occasionally have a >>> significant amount of oops where it would be very interesting to know >>> from where a particular slowdown comes from. >>> >>> So I would prefer keep some accounting here. >>> >>> Thanks, >>> ? Thomas >> > From jamsheed.c.m at oracle.com Thu Jul 16 07:00:18 2020 From: jamsheed.c.m at oracle.com (Jamsheed C M) Date: Thu, 16 Jul 2020 12:30:18 +0530 Subject: [15] RFR: 8246381: VM crashes with "Current BasicObjectLock* below than low_mark" In-Reply-To: <1af60254-a239-c21f-68df-be9b65534e7f@oracle.com> References: <7a802330-e836-1ff3-af0a-ede587e049ff@oracle.com> <30bd811e-c890-5bb1-8c78-4cf944fd5a42@oracle.com> <5d43f963-b931-3b69-4b5c-188c45b57de8@oracle.com> <1af60254-a239-c21f-68df-be9b65534e7f@oracle.com> Message-ID: <55b4473d-8aa4-77e0-1145-2a94a0a5f62e@oracle.com> Hi all, could i get another review? Best regards, Jamsheed On 16/07/2020 06:37, David Holmes wrote: > Hi Jamsheed, > > tl;dr version: fix looks good. Thanks for working through things with > me on this one. > > Long version ... for the sake of other reviewers (and myself) I'm > going to walk through the problem scenario and how the fix addresses > it, because the bug report is long and confusing and touches on a > number of different issues with async exception handling. > > We are dealing with the code generated for Java method entry, and in > particular for a synchronized Java method. We do a lot of things in > the entry code before we actually lock the monitor and jump to the > Java method. Some of those things include method profiling and the > counter overflow check for the JIT. If an exception is thrown at this > point, the logic to remove the activation would unlock the monitor - > which we haven't actually locked yet! So we have the > do_not_unlock_if_synchronized flag which is stored in the current > JavaThread. We set that flag true so that if any exceptions result in > activation removal, the removal logic won't try to unlock the monitor. > Once we're ready to lock the monitor we set the flag back to false > (note there is an implicit assumption here that monitor locking can > never raise an exception). > > The problem arises with async exceptions, or more specifically the > async exception that is raised due to an "unsafe access error". This > is where a memory-mapped ByteBuffer causes an access violation (SEGV) > due to a bad pointer. The signal handler simply sets a flag to > indicate we encountered an "unsafe access error", adjusts the BCI to > the next instruction and allows execution to proceed at the next > instruction. It is then expected that the runtime will "soon" notice > this pending unsafe access error and create and throw the > InternalError instance that indicates the ByteBuffer operation failed. > This requires executing Java code. > > One of the places that checks for that pending unsafe access error is > in the destructor of the JRT_ENTRY wrapper that is used for the method > profiling and counter overflow checking. This occurs whilst the > do_not_unlock_if_synchronized flag is true, so the resulting > InternalError won't result in an attempt to unlock the not-locked > monitor. > > The problem is that creating the InternalError executes Java code - it > calls constructors, which call methods etc. And some of those methods > are synchronized. So the method entry logic for such a call will set > do_not_unlock_if_synchronized to true, perform all the preamble > related to the call, then set do_not_unlock_if_synchronized to false, > lock the monitor and make the call. When construction completes the > InternalError is thrown and we remove the activation for the method we > had originally started to call. But now the > do_not_unlock_if_synchronized flag has been reset to false by the > nested Java method call, so we do in fact try to unlock a monitor that > was never locked, and things break. > > This nesting problem is well known and we have a mechanism for dealing > with - the UnlockFlagSaver. The actual logic executed for profiling > methods and doing the counter overflow check contains the requisite > UnlockFlagSaver to avoid the problem just outlined. Unfortunately the > async exception is processed in the JRT_ENTRY wrapper, which is > outside the scope of those UnlockFlagSaver helpers and so they don't > help in this case. > > So the fix is to "simply" move the UnlockFlagSaver deeper into the > call stack to the code that actually does the async exception processing: > > ?void JavaThread::check_and_handle_async_exceptions(bool > check_unsafe_error) { > +?? // May be we are at method entry and requires to save do not > unlock flag. > +?? UnlockFlagSaver fs(this); > > so now after the InternalError has been created and thrown we will > restore the original value of the do_not_unlock_if_synchronized flag > (false) and so the InternalError will not cause activation removal to > attempt to unlock the not-locked monitor. > > The scope of the UnlockFlagSaver could be narrowed to the actual logic > for processing the unsafe access error, but it seems fine at method > scope. > > A second fix is that the overflow counter check had an assertion that > it was not executed with any pending exceptions. But that turned out > to be false for reasons I can't fully explain, but it again appears to > relate to a pending async exception being installed prior to the > method call - and seems related to the two referenced JVM TI > functions. The simple solution here is to delete the assertion and to > check for pending exceptions on entry to the code and just return > immediately. The JRT_ENTRY destructor will see the pending exception > and propagate it. > > Cheers, > David > > On 16/07/2020 9:50 am, David Holmes wrote: >> Hi Jamsheed, >> >> On 16/07/2020 8:16 am, Jamsheed C M wrote: >>> (Thank you Dean, adding serviceability team as this issue involves >>> JVMTI features PopFrame, EarlyReturn features) >> >> It is not at all obvious how your proposed fix impacts the JVM TI >> features. >> >>> JBS entry: https://bugs.openjdk.java.net/browse/JDK-8246381 >>> >>> (testing: mach5, tier1-5 links in JBS) >>> >>> Best regards, >>> >>> Jamsheed >>> >>> On 15/07/2020 21:25, Jamsheed C M wrote: >>>> >>>> Hi, >>>> >>>> Async handling at method entry requires it to be aware of >>>> synchronization(like whether it is doing async handling before lock >>>> acquire or after) >>>> >>>> This is required as exception handler rely on this info for >>>> unlocking.? Async handling code never had this special condition >>>> handled and it worked most of the time as we were using biased >>>> locking which got disabled by [1] >>>> >>>> There was one other issue reported in similar time[2]. This issue >>>> got triggered in test case by [3], back to back extra safepoint >>>> after suspend and TLH for ThreadDeath. So in this setup both >>>> PopFrame request and Thread.Stop request happened together for the >>>> test scenario and it reached java method entry with >>>> pending_exception set. >>>> >>>> I have done a partial fix for the issue, mainly to handle >>>> production mode crash failures(do not unlock flag related ones) >>>> >>>> Fix detail: >>>> >>>> 1) I save restore the "do not unlock" flag in async handling. >> >> Sorry but you completely changed the fix compared to what we >> discussed and what I pre-reviewed! What happened to changing from >> JRT_ENTRY to JRT_ENTRY_NOASYNC? It is going to take me a lot of time >> and effort to determine that this save/restore of the "do not unlock >> flag" is actually correct and valid! >> >>>> >>>> 2) Return for floating pending exception for some cases(PopFrame, >>>> Early return related). This is debug(JVMTI) feature and floating >>>> exception can get cleaned just like that in present compiler >>>> request and deopt code. >> >> What part of the change addresses this? >> >> Thanks, >> David >> ----- >> >>>> >>>> webrev :http://cr.openjdk.java.net/~jcm/8246381/webrev.02/ >>>> >>>> There are more problems in these code areas, like we clear all >>>> exceptions in compilation request path(interpreter,c1), as well as >>>> deoptimization path. >>>> >>>> All these un-handled cases will be separately handled by >>>> https://bugs.openjdk.java.net/browse/JDK-8249451 >>>> >>>> Request for review. >>>> >>>> Best regards, >>>> >>>> Jamsheed >>>> >>>> [1]https://bugs.openjdk.java.net/browse/JDK-8231264 >>>> >>>> >>>> [2] https://bugs.openjdk.java.net/browse/JDK-8246727 >>>> >>>> [3] https://bugs.openjdk.java.net/browse/JDK-8221207 >>>> From coleen.phillimore at oracle.com Thu Jul 16 13:25:46 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 16 Jul 2020 09:25:46 -0400 Subject: RFR 8247808: Move JVMTI strong oops to OopStorage In-Reply-To: <82b9d699-42b0-a0ad-f1b9-62241f8754f4@oracle.com> References: <890246ae-2d56-95ff-c360-dd66532579a8@oracle.com> <5051e29d-f519-f270-3a55-b8d2d0a02854@oracle.com> <14670943-33c3-952c-3091-40afc21ecda4@oracle.com> <82b9d699-42b0-a0ad-f1b9-62241f8754f4@oracle.com> Message-ID: <7e3f5365-b585-6a49-e9da-203e1bff88b5@oracle.com> Thank you for the review. On 7/16/20 2:39 AM, David Holmes wrote: > Hi Coleen, > > On 16/07/2020 1:38 am, coleen.phillimore at oracle.com wrote: >> >> Hi, This patch has been reviewed and I was waiting for the ability to >> define different OopStorages, but I'd like to fix that in a further >> change after the GC changes have been agreed upon and reviewed.? >> Adding a new JVMTI OopStorage in the new mechanism is a smaller change. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8247808.01/webrev > > This all looks okay to me. > > I think you missed updating > > share/gc/shared/genCollectedHeap.hpp > > to delete > > GCH_PS_jvmti_oops_do, I did miss that.? Looks like someone forgot to remove GCH_PS_FlatProfiler_oops_do also. Thanks, Coleen > > Thanks, > David > ----- > >> Retested with tier1-3. >> >> Thanks, >> Coleen >> >> >> >> On 6/18/20 3:48 PM, coleen.phillimore at oracle.com wrote: >>> >>> >>> On 6/18/20 3:58 AM, Thomas Schatzl wrote: >>>> Hi, >>>> >>>> On 18.06.20 03:09, coleen.phillimore at oracle.com wrote: >>>>> >>>>> >>>>> On 6/17/20 7:49 PM, David Holmes wrote: >>>>>> Hi Coleen, >>>>>> >>>>>> On 18/06/2020 7:25 am, coleen.phillimore at oracle.com wrote: >>>>>>> Summary: Remove JVMTI oops_do calls from JVMTI and GCs >>>>>>> >>>>>>> Tested with tier1-3, also built shenandoah to verify shenandoah >>>>>>> changes. >>>>>>> >>>> [...] >>>>> >>>>> Kim noticed that G1 and ParallelGC should be processing these >>>>> roots in parallel (with many threads, since OopStorage has that >>>>> support) and he's going to or has filed a bug to fix it. As we add >>>>> more things to OopStorage (see upcoming RFRs), this will become >>>>> important. >>>>> >>>> >>>> I do not know which exact roots you want to move into OopStorage, >>>> but I would like to mention this concern: with moving everything >>>> into a single OopStorage (i.e. vm_globals in this case), I am >>>> worried that every time important information about the source for >>>> these gets lost. >>>> >>>> Which makes it hard to understand from where these oops came from >>>> when there is a performance problem in the "VM Globals" bucket. >>> Hi Thomas, >>> >>> I understand this concern.? On the GC list there is a discussion >>> about having the ability to create different strong OopStorages, >>> changing the OopStorage code to process these roots and report >>> statistics in parallel (and/or concurrent), and not having to >>> cascade the code through all the GCs. >>> >>> I'm going to hold this change until this discussion is complete and >>> move the JVMTI and services/management oops_do oops into a different >>> OopStorage that can make use of this. Then you'll have your >>> statistics and we won't have classes needing traversal with oops_do. >>> >>> Thanks, >>> Coleen >>> >>>> >>>> This may not apply to JVMTI oops, but others may occasionally have >>>> a significant amount of oops where it would be very interesting to >>>> know from where a particular slowdown comes from. >>>> >>>> So I would prefer keep some accounting here. >>>> >>>> Thanks, >>>> ? Thomas >>> >> From coleen.phillimore at oracle.com Thu Jul 16 14:43:17 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 16 Jul 2020 10:43:17 -0400 Subject: [15] RFR: 8246381: VM crashes with "Current BasicObjectLock* below than low_mark" In-Reply-To: <55b4473d-8aa4-77e0-1145-2a94a0a5f62e@oracle.com> References: <7a802330-e836-1ff3-af0a-ede587e049ff@oracle.com> <30bd811e-c890-5bb1-8c78-4cf944fd5a42@oracle.com> <5d43f963-b931-3b69-4b5c-188c45b57de8@oracle.com> <1af60254-a239-c21f-68df-be9b65534e7f@oracle.com> <55b4473d-8aa4-77e0-1145-2a94a0a5f62e@oracle.com> Message-ID: <38336861-a8eb-fdb0-7860-9cbc8eb820b6@oracle.com> Thanks to David's description of the problem and the fix, this makes sense to me now. I don't like it and we should revisit async exceptions for all the other problems it causes, but this change looks safe and good. thanks, Coleen On 7/16/20 3:00 AM, Jamsheed C M wrote: > Hi all, > > could i get another review? > > Best regards, > > Jamsheed > > On 16/07/2020 06:37, David Holmes wrote: >> Hi Jamsheed, >> >> tl;dr version: fix looks good. Thanks for working through things with >> me on this one. >> >> Long version ... for the sake of other reviewers (and myself) I'm >> going to walk through the problem scenario and how the fix addresses >> it, because the bug report is long and confusing and touches on a >> number of different issues with async exception handling. >> >> We are dealing with the code generated for Java method entry, and in >> particular for a synchronized Java method. We do a lot of things in >> the entry code before we actually lock the monitor and jump to the >> Java method. Some of those things include method profiling and the >> counter overflow check for the JIT. If an exception is thrown at this >> point, the logic to remove the activation would unlock the monitor - >> which we haven't actually locked yet! So we have the >> do_not_unlock_if_synchronized flag which is stored in the current >> JavaThread. We set that flag true so that if any exceptions result in >> activation removal, the removal logic won't try to unlock the >> monitor. Once we're ready to lock the monitor we set the flag back to >> false (note there is an implicit assumption here that monitor locking >> can never raise an exception). >> >> The problem arises with async exceptions, or more specifically the >> async exception that is raised due to an "unsafe access error". This >> is where a memory-mapped ByteBuffer causes an access violation (SEGV) >> due to a bad pointer. The signal handler simply sets a flag to >> indicate we encountered an "unsafe access error", adjusts the BCI to >> the next instruction and allows execution to proceed at the next >> instruction. It is then expected that the runtime will "soon" notice >> this pending unsafe access error and create and throw the >> InternalError instance that indicates the ByteBuffer operation >> failed. This requires executing Java code. >> >> One of the places that checks for that pending unsafe access error is >> in the destructor of the JRT_ENTRY wrapper that is used for the >> method profiling and counter overflow checking. This occurs whilst >> the do_not_unlock_if_synchronized flag is true, so the resulting >> InternalError won't result in an attempt to unlock the not-locked >> monitor. >> >> The problem is that creating the InternalError executes Java code - >> it calls constructors, which call methods etc. And some of those >> methods are synchronized. So the method entry logic for such a call >> will set do_not_unlock_if_synchronized to true, perform all the >> preamble related to the call, then set do_not_unlock_if_synchronized >> to false, lock the monitor and make the call. When construction >> completes the InternalError is thrown and we remove the activation >> for the method we had originally started to call. But now the >> do_not_unlock_if_synchronized flag has been reset to false by the >> nested Java method call, so we do in fact try to unlock a monitor >> that was never locked, and things break. >> >> This nesting problem is well known and we have a mechanism for >> dealing with - the UnlockFlagSaver. The actual logic executed for >> profiling methods and doing the counter overflow check contains the >> requisite UnlockFlagSaver to avoid the problem just outlined. >> Unfortunately the async exception is processed in the JRT_ENTRY >> wrapper, which is outside the scope of those UnlockFlagSaver helpers >> and so they don't help in this case. >> >> So the fix is to "simply" move the UnlockFlagSaver deeper into the >> call stack to the code that actually does the async exception >> processing: >> >> ?void JavaThread::check_and_handle_async_exceptions(bool >> check_unsafe_error) { >> +?? // May be we are at method entry and requires to save do not >> unlock flag. >> +?? UnlockFlagSaver fs(this); >> >> so now after the InternalError has been created and thrown we will >> restore the original value of the do_not_unlock_if_synchronized flag >> (false) and so the InternalError will not cause activation removal to >> attempt to unlock the not-locked monitor. >> >> The scope of the UnlockFlagSaver could be narrowed to the actual >> logic for processing the unsafe access error, but it seems fine at >> method scope. >> >> A second fix is that the overflow counter check had an assertion that >> it was not executed with any pending exceptions. But that turned out >> to be false for reasons I can't fully explain, but it again appears >> to relate to a pending async exception being installed prior to the >> method call - and seems related to the two referenced JVM TI >> functions. The simple solution here is to delete the assertion and to >> check for pending exceptions on entry to the code and just return >> immediately. The JRT_ENTRY destructor will see the pending exception >> and propagate it. >> >> Cheers, >> David >> >> On 16/07/2020 9:50 am, David Holmes wrote: >>> Hi Jamsheed, >>> >>> On 16/07/2020 8:16 am, Jamsheed C M wrote: >>>> (Thank you Dean, adding serviceability team as this issue involves >>>> JVMTI features PopFrame, EarlyReturn features) >>> >>> It is not at all obvious how your proposed fix impacts the JVM TI >>> features. >>> >>>> JBS entry: https://bugs.openjdk.java.net/browse/JDK-8246381 >>>> >>>> (testing: mach5, tier1-5 links in JBS) >>>> >>>> Best regards, >>>> >>>> Jamsheed >>>> >>>> On 15/07/2020 21:25, Jamsheed C M wrote: >>>>> >>>>> Hi, >>>>> >>>>> Async handling at method entry requires it to be aware of >>>>> synchronization(like whether it is doing async handling before >>>>> lock acquire or after) >>>>> >>>>> This is required as exception handler rely on this info for >>>>> unlocking.? Async handling code never had this special condition >>>>> handled and it worked most of the time as we were using biased >>>>> locking which got disabled by [1] >>>>> >>>>> There was one other issue reported in similar time[2]. This issue >>>>> got triggered in test case by [3], back to back extra safepoint >>>>> after suspend and TLH for ThreadDeath. So in this setup both >>>>> PopFrame request and Thread.Stop request happened together for the >>>>> test scenario and it reached java method entry with >>>>> pending_exception set. >>>>> >>>>> I have done a partial fix for the issue, mainly to handle >>>>> production mode crash failures(do not unlock flag related ones) >>>>> >>>>> Fix detail: >>>>> >>>>> 1) I save restore the "do not unlock" flag in async handling. >>> >>> Sorry but you completely changed the fix compared to what we >>> discussed and what I pre-reviewed! What happened to changing from >>> JRT_ENTRY to JRT_ENTRY_NOASYNC? It is going to take me a lot of time >>> and effort to determine that this save/restore of the "do not unlock >>> flag" is actually correct and valid! >>> >>>>> >>>>> 2) Return for floating pending exception for some cases(PopFrame, >>>>> Early return related). This is debug(JVMTI) feature and floating >>>>> exception can get cleaned just like that in present compiler >>>>> request and deopt code. >>> >>> What part of the change addresses this? >>> >>> Thanks, >>> David >>> ----- >>> >>>>> >>>>> webrev :http://cr.openjdk.java.net/~jcm/8246381/webrev.02/ >>>>> >>>>> There are more problems in these code areas, like we clear all >>>>> exceptions in compilation request path(interpreter,c1), as well as >>>>> deoptimization path. >>>>> >>>>> All these un-handled cases will be separately handled by >>>>> https://bugs.openjdk.java.net/browse/JDK-8249451 >>>>> >>>>> Request for review. >>>>> >>>>> Best regards, >>>>> >>>>> Jamsheed >>>>> >>>>> [1]https://bugs.openjdk.java.net/browse/JDK-8231264 >>>>> >>>>> >>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8246727 >>>>> >>>>> [3] https://bugs.openjdk.java.net/browse/JDK-8221207 >>>>> From jamsheed.c.m at oracle.com Thu Jul 16 14:49:48 2020 From: jamsheed.c.m at oracle.com (Jamsheed C M) Date: Thu, 16 Jul 2020 20:19:48 +0530 Subject: [15] RFR: 8246381: VM crashes with "Current BasicObjectLock* below than low_mark" In-Reply-To: <38336861-a8eb-fdb0-7860-9cbc8eb820b6@oracle.com> References: <7a802330-e836-1ff3-af0a-ede587e049ff@oracle.com> <30bd811e-c890-5bb1-8c78-4cf944fd5a42@oracle.com> <5d43f963-b931-3b69-4b5c-188c45b57de8@oracle.com> <1af60254-a239-c21f-68df-be9b65534e7f@oracle.com> <55b4473d-8aa4-77e0-1145-2a94a0a5f62e@oracle.com> <38336861-a8eb-fdb0-7860-9cbc8eb820b6@oracle.com> Message-ID: <24043cec-b3f2-bfa0-fd66-f2fcedc4be27@oracle.com> Hi Coleen, Thank you for the review. Best regards, Jamsheed On 16/07/2020 20:13, coleen.phillimore at oracle.com wrote: > > Thanks to David's description of the problem and the fix, this makes > sense to me now. > I don't like it and we should revisit async exceptions for all the > other problems it causes, but this change looks safe and good. > > thanks, > Coleen > > On 7/16/20 3:00 AM, Jamsheed C M wrote: >> Hi all, >> >> could i get another review? >> >> Best regards, >> >> Jamsheed >> >> On 16/07/2020 06:37, David Holmes wrote: >>> Hi Jamsheed, >>> >>> tl;dr version: fix looks good. Thanks for working through things >>> with me on this one. >>> >>> Long version ... for the sake of other reviewers (and myself) I'm >>> going to walk through the problem scenario and how the fix addresses >>> it, because the bug report is long and confusing and touches on a >>> number of different issues with async exception handling. >>> >>> We are dealing with the code generated for Java method entry, and in >>> particular for a synchronized Java method. We do a lot of things in >>> the entry code before we actually lock the monitor and jump to the >>> Java method. Some of those things include method profiling and the >>> counter overflow check for the JIT. If an exception is thrown at >>> this point, the logic to remove the activation would unlock the >>> monitor - which we haven't actually locked yet! So we have the >>> do_not_unlock_if_synchronized flag which is stored in the current >>> JavaThread. We set that flag true so that if any exceptions result >>> in activation removal, the removal logic won't try to unlock the >>> monitor. Once we're ready to lock the monitor we set the flag back >>> to false (note there is an implicit assumption here that monitor >>> locking can never raise an exception). >>> >>> The problem arises with async exceptions, or more specifically the >>> async exception that is raised due to an "unsafe access error". This >>> is where a memory-mapped ByteBuffer causes an access violation >>> (SEGV) due to a bad pointer. The signal handler simply sets a flag >>> to indicate we encountered an "unsafe access error", adjusts the BCI >>> to the next instruction and allows execution to proceed at the next >>> instruction. It is then expected that the runtime will "soon" notice >>> this pending unsafe access error and create and throw the >>> InternalError instance that indicates the ByteBuffer operation >>> failed. This requires executing Java code. >>> >>> One of the places that checks for that pending unsafe access error >>> is in the destructor of the JRT_ENTRY wrapper that is used for the >>> method profiling and counter overflow checking. This occurs whilst >>> the do_not_unlock_if_synchronized flag is true, so the resulting >>> InternalError won't result in an attempt to unlock the not-locked >>> monitor. >>> >>> The problem is that creating the InternalError executes Java code - >>> it calls constructors, which call methods etc. And some of those >>> methods are synchronized. So the method entry logic for such a call >>> will set do_not_unlock_if_synchronized to true, perform all the >>> preamble related to the call, then set do_not_unlock_if_synchronized >>> to false, lock the monitor and make the call. When construction >>> completes the InternalError is thrown and we remove the activation >>> for the method we had originally started to call. But now the >>> do_not_unlock_if_synchronized flag has been reset to false by the >>> nested Java method call, so we do in fact try to unlock a monitor >>> that was never locked, and things break. >>> >>> This nesting problem is well known and we have a mechanism for >>> dealing with - the UnlockFlagSaver. The actual logic executed for >>> profiling methods and doing the counter overflow check contains the >>> requisite UnlockFlagSaver to avoid the problem just outlined. >>> Unfortunately the async exception is processed in the JRT_ENTRY >>> wrapper, which is outside the scope of those UnlockFlagSaver helpers >>> and so they don't help in this case. >>> >>> So the fix is to "simply" move the UnlockFlagSaver deeper into the >>> call stack to the code that actually does the async exception >>> processing: >>> >>> ?void JavaThread::check_and_handle_async_exceptions(bool >>> check_unsafe_error) { >>> +?? // May be we are at method entry and requires to save do not >>> unlock flag. >>> +?? UnlockFlagSaver fs(this); >>> >>> so now after the InternalError has been created and thrown we will >>> restore the original value of the do_not_unlock_if_synchronized flag >>> (false) and so the InternalError will not cause activation removal >>> to attempt to unlock the not-locked monitor. >>> >>> The scope of the UnlockFlagSaver could be narrowed to the actual >>> logic for processing the unsafe access error, but it seems fine at >>> method scope. >>> >>> A second fix is that the overflow counter check had an assertion >>> that it was not executed with any pending exceptions. But that >>> turned out to be false for reasons I can't fully explain, but it >>> again appears to relate to a pending async exception being installed >>> prior to the method call - and seems related to the two referenced >>> JVM TI functions. The simple solution here is to delete the >>> assertion and to check for pending exceptions on entry to the code >>> and just return immediately. The JRT_ENTRY destructor will see the >>> pending exception and propagate it. >>> >>> Cheers, >>> David >>> >>> On 16/07/2020 9:50 am, David Holmes wrote: >>>> Hi Jamsheed, >>>> >>>> On 16/07/2020 8:16 am, Jamsheed C M wrote: >>>>> (Thank you Dean, adding serviceability team as this issue involves >>>>> JVMTI features PopFrame, EarlyReturn features) >>>> >>>> It is not at all obvious how your proposed fix impacts the JVM TI >>>> features. >>>> >>>>> JBS entry: https://bugs.openjdk.java.net/browse/JDK-8246381 >>>>> >>>>> (testing: mach5, tier1-5 links in JBS) >>>>> >>>>> Best regards, >>>>> >>>>> Jamsheed >>>>> >>>>> On 15/07/2020 21:25, Jamsheed C M wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> Async handling at method entry requires it to be aware of >>>>>> synchronization(like whether it is doing async handling before >>>>>> lock acquire or after) >>>>>> >>>>>> This is required as exception handler rely on this info for >>>>>> unlocking.? Async handling code never had this special condition >>>>>> handled and it worked most of the time as we were using biased >>>>>> locking which got disabled by [1] >>>>>> >>>>>> There was one other issue reported in similar time[2]. This issue >>>>>> got triggered in test case by [3], back to back extra safepoint >>>>>> after suspend and TLH for ThreadDeath. So in this setup both >>>>>> PopFrame request and Thread.Stop request happened together for >>>>>> the test scenario and it reached java method entry with >>>>>> pending_exception set. >>>>>> >>>>>> I have done a partial fix for the issue, mainly to handle >>>>>> production mode crash failures(do not unlock flag related ones) >>>>>> >>>>>> Fix detail: >>>>>> >>>>>> 1) I save restore the "do not unlock" flag in async handling. >>>> >>>> Sorry but you completely changed the fix compared to what we >>>> discussed and what I pre-reviewed! What happened to changing from >>>> JRT_ENTRY to JRT_ENTRY_NOASYNC? It is going to take me a lot of >>>> time and effort to determine that this save/restore of the "do not >>>> unlock flag" is actually correct and valid! >>>> >>>>>> >>>>>> 2) Return for floating pending exception for some cases(PopFrame, >>>>>> Early return related). This is debug(JVMTI) feature and floating >>>>>> exception can get cleaned just like that in present compiler >>>>>> request and deopt code. >>>> >>>> What part of the change addresses this? >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>>>> >>>>>> webrev :http://cr.openjdk.java.net/~jcm/8246381/webrev.02/ >>>>>> >>>>>> There are more problems in these code areas, like we clear all >>>>>> exceptions in compilation request path(interpreter,c1), as well >>>>>> as deoptimization path. >>>>>> >>>>>> All these un-handled cases will be separately handled by >>>>>> https://bugs.openjdk.java.net/browse/JDK-8249451 >>>>>> >>>>>> Request for review. >>>>>> >>>>>> Best regards, >>>>>> >>>>>> Jamsheed >>>>>> >>>>>> [1]https://bugs.openjdk.java.net/browse/JDK-8231264 >>>>>> >>>>>> >>>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8246727 >>>>>> >>>>>> [3] https://bugs.openjdk.java.net/browse/JDK-8221207 >>>>>> > From coleen.phillimore at oracle.com Thu Jul 16 15:01:41 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 16 Jul 2020 11:01:41 -0400 Subject: RFR 8247878: Move Management strong oops to OopStorage Message-ID: <05ce335a-d836-cf15-65b8-e015a5a93c30@oracle.com> Summary: Use OopStorage for strong oops stored with memory and thread sampling and dumping, and remove oops_do and GC calls. These use OopStorageSet::vm_global()? OopStorage for now.? I'll change the thread sampling oops to use a new OopStorage once the GC code is changed to nicely allow additional oop storages.? The memory pool oops are never deleted once created, so they should stay in vm_global() oop storage. Tested with tiers 1-3 (tiers 4-6 with other changes) and javax/management tests.? I timed the tests to see if there was any noticeable performance difference, and there was not. open webrev at http://cr.openjdk.java.net/~coleenp/2020/8247878.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8247878 Thanks, Coleen From goetz.lindenmaier at sap.com Thu Jul 16 16:30:23 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 16 Jul 2020 16:30:23 +0000 Subject: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents In-Reply-To: References: <1f8a3c7a-fa0f-b5b2-4a8a-7d3d8dbbe1b5@oracle.com> <4b56a45c-a14c-6f74-2bfd-25deaabe8201@oracle.com> <5271429a-481d-ddb9-99dc-b3f6670fcc0b@oracle.com> Message-ID: Hi Richard, I'll answer to the obvious things in this mail now. I'll go through the code thoroughly again and write a review of my findings thereafter. > So here is the new webrev.6 > > Webrev.6: > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.6/ > Delta: > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.6.inc/ Thanks for the incremental webrev, it's helpful! > I spent most of the time running a microbenchmark [1] I wrote to answer > questions from your > review. At first I had trouble with variance in the results until I found out it > was due to the NUMA > architecture of the server I used. After that I noticed that there was a > performance regression of > about 5% even at low agent activity. I finally found out that it was due to the > implementation of > JavaThread::wait_for_object_deoptimization() which is called by the target > of the JVMTI operation to > self suspend for object deoptimization. I fixed this by adding limited spinning > before calling > wait() on the monitor. > > The delta includes many changes in comments, renaming of names, etc. So > I'd like to summarize > functional changes: > > * Collected all the code for the testing feature DeoptimizeObjectsALot in > compileBroker.cpp and reworked it. Thanks, this makes it much more compact. > With DeoptimizeObjectsALot enabled internal threads are started that > deoptimize frames and > objects. The number of threads started are given with > DeoptimizeObjectsALotThreadCountAll and > DeoptimizeObjectsALotThreadCountSingle. The former targets all existing > threads whereas the > latter operates on a single thread selected round robin. > > I removed the mode where deoptimizations were performed at every nth > exit from the runtime. I never used it. Do I get it right? You have a n:1 and a n:all test scenario. n:1: n threads deoptimize 1 Jana thread where n = DOALThreadCountSingle n:m: n threads deoptimize all Java threads where n = DOALThreadCountAll? > * EscapeBarrier::sync_and_suspend_one(): use a direct handshake and > execute it always independently > of is_thread_fully_suspended(). Is this also a performance optimization? > * Bugfix in EscapeBarrier::thread_added(): must not clear deopt flag. Found > this testing with DeoptimizeObjectsALot. Ok. > * Added EscapeBarrier::thread_removed(). Ok. > * EscapeBarrier constructors: barriers can now be entirely disabled by > disabling DoEscapeAnalysis. > This effectively disables the enhancement. Good! > * JavaThread::wait_for_object_deoptimization(): > - Bugfix: the last check of is_obj_deopt_suspend() must be /after/ the > safepoint check! This > caused issues with not walkable stacks with DeoptimizeObjectsALot. OK. As I understand, there was one safepoint check in the old version, now there is one in each iteration. I assume this is intended, right? > - Added limited spinning inspired by HandshakeSpinYield to fix regression in > microbenchmark [1] Ok. Nice improvement, nice catch! > > I refer to some more changes answering your questions and comments inline > below. > > Thanks, > Richard. > > [1] Microbenchmark: > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.6.microbenchmark/ > > > I understand you annotate at safepoints where the escape analysis > > finds out that an object is "better" than global escape. > > This are the cases where the analysis identifies optimization > > opportunities. These annotations are then used to deoptimize > > frames and the objects referenced by them. > > Doesn't this overestimate the optimized > > objects? E.g., eliminate_alloc_node has many cases where it bails > > out. > > Yes, the implementation is conservative, but it is comparatively simple and > the additional debug > info is just 2 flags per safepoint. Thanks. It also helped that you explained to me offline that there are more optimizations than only lock elimination and scalar replacement done based on the ea information. The ea refines the IR graph with allows follow up optimizations which can not easily be tracked back to the escaping objects or the call sites where they do not escape. Thus, if there are non-global escaping objects, you have to deoptimize the frame. Did I repeat that correctly? With this understanding, a row of my proposed renamings/comments are obsolete. > On the other hand, those JVMTI operations > that really trigger > deoptimizations are expected to be comparatively infrequent such that > switching to the interpreter > for a few microseconds will hardly have an effect. That sounds reasonable. > I've done microbenchmarking to check this. > > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.6.microbe > nchmark/ > > I found that in the worst case performance can be impacted by 10%. If the > agent is extremely active > and does relevant JVMTI calls like GetOwnedMonitorStackDepthInfo() every > millisecond or more often, > then the performance impact can be 30%. But I would think that this is not > realistic. These calls > are issued in interactive sessions to analyze deadlocks. Ok. > We could get more precise deoptimizations by adding a third flag per > safepoint for ea-local objects > among the owned monitors. This would help improve the worst case in the > benchmark. But I'm not > convinced, if it is worth it. > > Refer to the README.txt of the microbenchmark for a more detailled > discussion. > > pcDesc.hpp > > > > I would like to see some documentation of the methods. > Done. I didn't take your text, though, because I only noticed it after writing > my own. Let me know if you are not ok with it. That's fine. My texts were only proposals, you as author know better what goes on anyways. > > scopeDesc.cpp > > > > Besides refactoring copy escape info from pcDesc to scopeDesc > > and add accessors. Trivial. > > > > In scopeDesc.hpp you talk about NoEscape and ArgEscape. > > This are opto terms, but scopeDesc is a shared datastructure > > that does not depend on a specific compiler. > > Please explain what is going on without using these terms. > > Actually these are not too opto specific terms. They are used in the paper > referenced in > escape.hpp. Also you can easily google them. I'd rather keep the comments > as they are. Hmm, I'm not really happy with this, as also the papers are for the compiler community, and probably not familiar to others that work with HotSpot. But stay with your terms if you think it makes it clearer. Anyways, with now understanding why you use conservative Information (see above), the descriptions I had in mind are not precise. > > callnode.hpp > > > > You add functionality to annotate callnodes with escape information > > This is carried through code generation to final output where it is > > added to the compiled methods meta information. > > > > At Safepoints in general jvmti can access > > - Objects that were scalar replaced. They must be reallocated. > > (Flag EliminateAllocations) > > - Objects that should be locked but are not because they never > > escape the thread. They need to be relocked. > > > > At calls, Objects where locks have been removed escape to callees. > > We must persist this information so that if jvmti accesses the > > object in a callee, we can determine by looking at the caller that > > it needs to be relocked. > > Note that the ea-optimization must not be at the current location, it can also > follow when control > returns to the caller. Lock elimination isn't the only relevant optimization. Yes, I understood now, see above. Thanks for explaining. > Accesses to instance > members or array elements can be optimized as well. You mean the compiler can/will ignore volatile or memory ordering requirements for non-escaping objects? Sounds reasonable to do. > > // Returns true if at least one of the arguments to the call is an oop > > // that does not escape globally. > > bool ConnectionGraph::has_arg_escape(CallJavaNode* call) { > > IMHO the method names are descriptive and don't need the comments. But I > give in :) (only replaced > "oop" with "object") Thanks. Yes, object is better than oop. > You are right, it is not correct how flags are checked. Especially if only > running with the JVMCI compiler. > > I changed Deoptimization::deoptimize_objects_internal() to make > reallocation and relocking dependent > on similar checks as in Deoptimization::fetch_unroll_info_helper(). > Furthermore EscapeBarriers are > conditionally activated depending on the following (see EscapeBarrier ctors): > > JVMCI_ONLY(UseJVMCICompiler) NOT_JVMCI(false) > COMPILER2_PRESENT(|| DoEscapeAnalysis) > > So the enhancement can be practically completely disabled by disabling > DoEscapeAnalysis, which is > what C2 currently does if JVMTI capabilities that allow access to local > references are taken. Thanks for fixing. > I went for the latter. > > > In fetch_unroll_info_helper, I don't understand why you need > > && !EscapeBarrier::objs_are_deoptimized(thread, deoptee.id())) { > > for eliminated locks, but not for skalar replaced objects? > > In short reallocation is idempotent, relocking is not. > > Without the enhancement Deoptimization::realloc_objects() can already be > called more than once for a frame: > > First call in materializeVirtualObjects() (also iterateFrames()). > > Second (indirect) call in fetch_unroll_info_helper(). > > The objects from the first call are saved as jvmti deferred updates when > realloc_objects() > returns. Note that there is no relationship to jvmti. The thing in common is > that updates cannot be > directely installed into a compiled frame, it is necessary to deoptimize the > frame and defer the > updates until the compiled frame gets replaced. Every time the vframes > corresponding to the owner > frame are iterated, they get the deferred updates. So in > fetch_unroll_info_helper() the > GrowableArray* chunk reference them too. All > references to the objects created by > the second (indirect) call to realloc_objects() are never used, because > compiledVFrame accessors to > locals, expressions, and monitors override them with the deferred updates. > The objects become > unreachable and get gc'ed. OK, so repeatedly computed vFrames always have the first version of reallocated objects by construction, so it needs not be handled here. But also due to construction, objects might be allocated just to be discarded. > materializeVirtualObjects() does not bother with relocking. > deoptimize_objects_internal(), which is > introduced by the enhancement, does relock objects, after all the lock > elimination becomes illegal > with the change in escape state. Relocking twice does not work, so the > enhancement avoids it by > checking EscapeBarrier::objs_are_deoptimized(thread, deoptee.id()). > > Note that materializeVirtualObjects() can be called more than once and will > always return the very > same objects, even though it calls realloc_objects() again. Ok. > > I would guess it is because the eliminated locks can be applied to > > argEscape, but scalar replacement only to noescape objects? > > I.e. it might have been done before? > > > > But why isn't this the case for eliminate_allocations? > > deoptimize_objects_internal does both unconditionally, > > so both can happen to inner frames, right? > > Sorry, I don't quite understand. Hope the explanation above helps. Yes. I was guessing wrong :) > > I like if boolean operators are at the beginning of broken lines, > > but I think hotspot convention is to have them at the end. > Ok, fixed. Thanks. > > > Code will get much more simple if BiasedLocking is removed. > > > > EscapeBarrier:: ... > > > > (This class maybe would qualify for a file of its own.) > > > > deoptimize_objects() > > I would mention escape analysis only as side remark. Also, as I understand, > > there is only one frame at given depth? > > // Deoptimize frames with optimized objects. This can be omitted locks and > > // objects not allocated but replaced by scalars. In C2, these optimizations > > // are based on escape analysis. > > // Up to depth, deoptimize frames with any optimized objects. > > // From depth to entry_frame, deoptimize only frames that > > // pass optimized objects to their callees. > > (First part similar for the comment above > EscapeBarrier::deoptimize_objects_internal().) > > I've reworked the comment. Let me know if you still think it needs to be > improved. Good now, thanks (maybe break the long line ...) > > What is the check (cur_depth <= depth) good for? Can you > > ever walk past entry_frame? > > Yes (assuming you mean the outer while-statement), there are java frames > beyond the entry frame if a > native method calls java methods again. So we visit all frames up to the given > depth and from there > we continue to the entry frame. It is not necessary to continue beyond that > entry frame, because > escape analysis assumes that arguments to native functions escape globally. > > Example: Let the java stack look like this: > > +---------+ > | Frame A | > +---------+ > | Frame N | > +---------+ > | Frame B | > +---------+ <- top of stack > > Where java method A calls native method N and N calls java method B. > > Very simplified the native stack will look like this > > +-------------------------+ > | Frame of JIT Compiled A | > +-------------------------+ > | Frame N | > +-------------------------+ > | Entry Frame | > +-------------------------+ > | Frame B | > +-------------------------+ <- top of stack > > The entry frame is an activation of the call stub, which is a small assembler > routine that > translates from the native calling convention to the java calling convention. > > There cannot be any ArgEscape that is passed to B (see above), therefore we > can stop the stackwalk > at the entry frame if depth is 1. If depth is 3 we have to continue to Frame A, > as it is directely > accessed. Ok, thanks, nice explanation!! > > Isn't vf->is_compiled_frame() prerequisite that "Move to next physical > frame" > > is needed? You could move it into the other check. > > If so, similar for deoptimize_objects_all_threads(). > > Only compiledVFrame require moving to the /top/ frame. Fixed. Thanks, this looks better. > > Syncronization: looks good. I think others had a look at this before. > > > > EscapeBarrier::deoptimize_objects_internal() > > The method name is misleading, it is not used by > > deoptimize_objects(). > > Also, method with the same name is in Deopitmization. > > Proposal: deoptimize_objects_thread() ? > > Sorry, but I don't see, why it would be misleading. > What would be the meaning of 'deoptimize_objects_thread'? I don't > understand that name. 1. I have no idea why it's called "_internal". Because it is private? By the name, I would expect that EscapeBarrier::deoptimize_objects() calls it for some internal tasks. But it does not. 2. My proposal: deoptimize_objects_all_threads() iterates all threads and calls deoptimize_objects(_one)_thread(thread) for each of these. That's how I would have named it. But no bike shedding, if you don't see what I mean it's not obvious. > > C1 stubs: this really shows you tested all configurations, great! > > > > > > mutexLocker: ok. > > objectMonitor.cpp: ok > > stackValue.hpp Is this missing clearing a bug? > > In short: that change is not needed anymore. I'll remove it again. Good. Thanks for the details. > > Renaming deferred_locals to deferred_updates is good, as well as > > adding a datastructure for it. > > (Adding this data structure might be a breakout, too.) > > > > good. > > > > thread.cpp > > > > good. > > > > vframe.cpp > > > > Is this a bug in existing code? > > Makes sense. > > Depends on your definition of bug. There are no references to > vframe::is_entry_frame() in the > existing code. I would think it is a bug. So it is :) > > > > > vframe_hp.hpp > > (What stands _hp for? helper? The file should be named > compiledVFrame ...) > > > > not_global_escape_in_scope() ... > > Again, you mention escape analysis here. Comments above hold, too. > > I think it is the right name, because it is meaningful and simple. Ok, accepted ... given my understandings from above. > > > You introduce JvmtiDeferredUpdates. Good. > > > > vframe_hp.cpp > > > > Changes for JvmtiDeferredUpdates, escape state accessors, > > > > line 422: > > Would an assertion assert(!info->owner_is_scalar_replaced(), ...) hold here? > > > > > > macros.hpp > > Good. > > > > > > Test coding > > ============ > > > > compileBroker.h|cpp > > > > You introduce a third class of threads handled here and > > add a new flag to distinguish it. Before, the two kinds > > of threads were distinguished implicitly by passing in > > a compiler for compiler threads. > > The new thread kind is only used for testing in debug. > > > > make_thread: > > You could assert (comp != NULL...) to assure previous > > conditions. > > If replaced the if-statements with a switch-statement, made sure all enum- > elements are covered, and > added the assertion you suggested. > > > line 989 indentation broken > > You are referring to this block I assume: > (from > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.5/src/hots > pot/share/compiler/compileBroker.cpp.frames.html) > > 976 if (MethodFlushing) { > 977 // Initialize the sweeper thread > 978 Handle thread_oop = create_thread_oop("Sweeper thread", CHECK); > 979 jobject thread_handle = JNIHandles::make_local(THREAD, > thread_oop()); > 980 make_thread(sweeper_t, thread_handle, NULL, NULL, THREAD); > 981 } > 982 > 983 #if defined(ASSERT) && COMPILER2_OR_JVMCI > 984 if (DeoptimizeObjectsALot == 2) { > 985 // Initialize and start the object deoptimizer threads > 986 for (int thread_count = 0; thread_count < > DeoptimizeObjectsALotThreadCount; thread_count++) { > 987 Handle thread_oop = create_thread_oop("Deoptimize objects a lot > thread", CHECK); > 988 jobject thread_handle = JNIHandles::make_local(THREAD, > thread_oop()); > 989 make_thread(deoptimizer_t, thread_handle, NULL, NULL, THREAD); > 990 } > 991 } > 992 #endif // defined(ASSERT) && COMPILER2_OR_JVMCI > > I cannot really see broken indentation here. Am I looking at the wrong > location? I don't have the source version I reviewed last time any more, so I can't check. But maybe an artefact from patching ... if there were tabs jcheck would have told you, so that's not it. No problem. Best regards, Goetz. From david.holmes at oracle.com Fri Jul 17 04:13:42 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 17 Jul 2020 14:13:42 +1000 Subject: RFR 8247878: Move Management strong oops to OopStorage In-Reply-To: <05ce335a-d836-cf15-65b8-e015a5a93c30@oracle.com> References: <05ce335a-d836-cf15-65b8-e015a5a93c30@oracle.com> Message-ID: <3cc4ef8a-0b3d-3827-1385-a93aed40d786@oracle.com> Hi Coleen, On 17/07/2020 1:01 am, coleen.phillimore at oracle.com wrote: > Summary: Use OopStorage for strong oops stored with memory and thread > sampling and dumping, and remove oops_do and GC calls. > > These use OopStorageSet::vm_global()? OopStorage for now.? I'll change > the thread sampling oops to use a new OopStorage once the GC code is > changed to nicely allow additional oop storages.? The memory pool oops > are never deleted once created, so they should stay in vm_global() oop > storage. > > Tested with tiers 1-3 (tiers 4-6 with other changes) and > javax/management tests.? I timed the tests to see if there was any > noticeable performance difference, and there was not. > > open webrev at http://cr.openjdk.java.net/~coleenp/2020/8247878.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8247878 Overall the bulk of the changes look good - lots of red! :) I'm unsure about defining AtomicOopHandle. I think the memory ordering operations may be better kept in the code paths that use the OopHandle. The name doesn't convey what this actually does in terms of the release-set and resolve-acquire, and I can't think of a good name that would do so**. If you do want to keep it then I think there should be a non-acquire version of resolve as not all accesses require acquire semantics. Looking at memoryManager.cpp (but the same applies to memoryPool.cpp): 56 bool MemoryManager::is_manager(instanceHandle mh) const { 57 return mh() == _memory_mgr_obj.resolve_acquire(); 58 } This doesn't have acquire semantics currently and doesn't need it. 122 // The lock has done an acquire, so the load can't float above it, but 123 // we need to do a load_acquire as above. 124 mgr_obj = _memory_mgr_obj.resolve_acquire(); The original code and comment is wrong - this doesn't need acquire as the lock already handles that. If another thread has set the mgr object then it did so holding the lock that the current thread now owns, which means the other thread had to have released the lock first, hence when the current thread acquired the lock, all stores in relation to the mgr object are already guaranteed to be visible. ** Perhaps the OopHandle constructor, and resolve method should take an optional memory order Decorator parameter? --- Should we introduce a naming convention to help distinguish an oop variable from an oopHandle? You seem to have adopted the convention that _foo is the oopHandle and foo is the oop. But should we use foo_h as we do (sometimes at least) for other handles? (I'm not generally fond of encoding types in variable names but when handles and their contents can be accessed in the same code, it does make things clearer.) Thanks, David ----- > Thanks, > Coleen > > > From kim.barrett at oracle.com Fri Jul 17 09:02:34 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 17 Jul 2020 05:02:34 -0400 Subject: RFR 8247878: Move Management strong oops to OopStorage In-Reply-To: <05ce335a-d836-cf15-65b8-e015a5a93c30@oracle.com> References: <05ce335a-d836-cf15-65b8-e015a5a93c30@oracle.com> Message-ID: > On Jul 16, 2020, at 11:01 AM, coleen.phillimore at oracle.com wrote: > > Summary: Use OopStorage for strong oops stored with memory and thread sampling and dumping, and remove oops_do and GC calls. > > These use OopStorageSet::vm_global() OopStorage for now. I'll change the thread sampling oops to use a new OopStorage once the GC code is changed to nicely allow additional oop storages. The memory pool oops are never deleted once created, so they should stay in vm_global() oop storage. > > Tested with tiers 1-3 (tiers 4-6 with other changes) and javax/management tests. I timed the tests to see if there was any noticeable performance difference, and there was not. > > open webrev at http://cr.openjdk.java.net/~coleenp/2020/8247878.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8247878 > > Thanks, > Coleen ------------------------------------------------------------------------------ src/hotspot/share/oops/oopHandle.inline.hpp 50 if (peek() != NULL) { Adding that seems like an unrelated change, and it's not clear to me why this is being done. ------------------------------------------------------------------------------ src/hotspot/share/services/lowMemoryDetector.cpp 299 if (_sensor_obj.peek() != NULL) { 300 InstanceKlass* sensorKlass = Management::sun_management_Sensor_klass(CHECK); 301 Handle sensor_h(THREAD, _sensor_obj.resolve()); I see no reason for a peek operation here, and think it just makes the code harder to understand. Just unconditionally resolve the sensor. Similarly here: 364 if (_sensor_obj.peek() != NULL) { 365 InstanceKlass* sensorKlass = Management::sun_management_Sensor_klass(CHECK); 366 Handle sensor(THREAD, _sensor_obj.resolve()); ------------------------------------------------------------------------------ src/hotspot/share/services/memoryManager.cpp 136 _memory_mgr_obj = AtomicOopHandle(OopStorageSet::vm_global(), mgr_obj); There is a race here. The handle constructor allocates the oopstorage entry and then does a release_store of the value into it. After (in source order) the handle is constructed, it is copied into _memory_mgr_obj, which is just a copy-assign of the oop* from oopstorage. There's nothing to prevent that copy-assign from being reordered before the store of the value into the oopstorage, either by the compiler or the hardware. The _obj in _memory_mgr_obj is being read without locking and may be written by another thread, so should itself be appropriate atomic. It's not clear what the semantics of this new kind of handle are supposed to be, but I think as used for _memory_mgr_obj there are problems. The same is true for _memory_pool_obj. I think the atomicity here is in the wrong place. The pointee of the oop* doesn't need atomic access, the oop* itself does. At least I think so; both _memory_mgr_obj and _memory_pool_obj are lazily assigned on first use and never change after that, right? One way to do that would be that the type of _memory_mgr_obj is `oop* volatile`, and is initialized as oop* oop_ptr = ... allocate oopstorage entry ... NativeAccess<>::oop_store(oop_ptr, value); Atomic::release_store(&_memory_mgr_obj, oop_ptr); Alternatively, use volatile OopHandle _memory_mgr_obj; Atomic::release_store(&_memory_mgr_obj, OopHandle(...)); and teach Atomic how to deal with OopHandle by defining a PrimitiveConversions::Translator for it. The declaration would be volatile OopHandle _memory_mgr_obj; and accesses would be Atomic::load_acquire(&_memory_mgr_obj).resolve(); And AtomicOopHandle isn't useful and should be discarded. ------------------------------------------------------------------------------ From goetz.lindenmaier at sap.com Fri Jul 17 12:30:40 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 17 Jul 2020 12:30:40 +0000 Subject: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents In-Reply-To: References: <1f8a3c7a-fa0f-b5b2-4a8a-7d3d8dbbe1b5@oracle.com> <4b56a45c-a14c-6f74-2bfd-25deaabe8201@oracle.com> <5271429a-481d-ddb9-99dc-b3f6670fcc0b@oracle.com> Message-ID: Hi Richard, > I'll answer to the obvious things in this mail now. > I'll go through the code thoroughly again and write > a review of my findings thereafter. As promised a detailed walk-throug, but without any major findings: c1_IR.hpp: ok ci_Env.h|cpp: ok compiledMethod.cpp, nmethod.cpp: ok debugInfoRec.h|cpp: ok scopeDesc.h|cpp ok compileBroker.h|cpp: Maybe a bit of documentation how and why you start the threads? I had expected there are two test scenarios run after each other, but now I understand 'Single' and 'All' run simultaneously. Well, this really is a stress test! Also good the two variants of depotimization are stressed against each other. Besides that really nice it's all in one place. rootResolver.cpp: ok jvmciCodeInstaller.cpp: ok c2compiler.cpp: The essence of this change! Just one line :) Great! callnode.hpp ok escape.h|cpp ok macro.cpp I was not that happy with the names saying not_global_escape and similar. I now agreed you have to use the terms of the escape analysis (NoEscape ArgEscape= throughout the runtime code. I'm still not happy with the 'not' in the term, I always try to expand the name to some sentence with a negated verb, but it makes no sense. For example, "has_not_global_escape_in_scope" expands to "Hasn't a global escape in its scope." in my thinking, which makes no sense. You probably mean "Has not-global escape in its scope." or "Has {ArgEscape|NoEscape} in its scope." C2 is using the word "non" in this context, e.g., here alloc->is_non_escaping. non obviously negates the adjective 'global', non-global or nonglobal even is a English term I find in the net. So what about "has_non_global_escape_in_scope?" matcher.cpp ok output.cpp:1071 Please break the long line. jvmtiCodeBlobEvents.cpp ok jvmtiEnv.cpp MaxJavaStackTraceDepth is only documented to affect the exceptions stack trace depth, not to limit jvmti operations. Therefore I wondered why it is used here. Non of your business, but the flag should document this in globals.hpp, too. Does jvmti specify that the same limits are used ...? ok on your side. jvmtiEnvBase.cpp ok jvmtiImpl.h|cpp ok jvmtiTagMap.cpp ok whitebox.cpp ok deoptimization.cpp line 177: Please break line line 246, 281: Please break line 1578, 1583, 1589, 1632, 1649, 1651 Break line 1651: You use 'non'-terms, too: non-escaping :) 2805, 2929, 2946ff, break lines deoptimization.hpp 158, 174, 176 ... I would break lines too, but here you are in good company :) globals.hpp ok mutexLocker.h|cpp ok objectMonitor.cpp ok thread.cpp 2631 typo: sapfepont --> safepoint thread.hpp ok thread.inline.hpp ok vframe.cpp ok vframe_hp.cpp 458ff break lines vframe_hp.hpp ok macros.hpp ok TEST.ROOT ok WhiteBox.java ok IterateHeapWithEscapeAnalysisEnabled.java line 415: msg("wait until target thread has set testMethod_result"); while (testMethod_result == 0) { Thread.sleep(50); } Might the test run into timeouts at this place? The field is volatile, i.e. it will be reloaded in each iteration. But will dontinline_testMethod write it back to main memory in time? libIterateHeapWithEscapeAnalysisEnabled.c ok EATests.java This is a very elaborate test. I found a row of test cases illustrating issues we talked about before. Really helpful! 1311: TypeO materialize -> materialized 1640: setting local variable i triggers always deoptimization --> setting local variable i always triggers deoptimization 2176: dontinline_calee --> dontinline_callee 2510: poping --> popping ... but I'm not sure here. https://www.urbandictionary.com/define.php?term=poping poping Drinking large amounts of Dextromethorphan Hydrobromide (DXM)based cough syrup, and then embarking on an adventure while wandering around neighborhoods or parks all night. This is usually done while listening to Punk rock music from a portable jambox. ;) Don?t do it! ?? EATestsJVMTI.java I think you can just copy this test description into the other test. You can have two @test comments, they will be treated as separate tests. The @requires will be evaluated accordingly. For an example see test/hotspot/jtreg/runtime/exceptionMsgs/NullPointerException/NullPointerExceptionTest.java which has two different compile setups for the test class (-g). so, that's it for reading code ... Some general remarks, maybe a bit picky ...: I think you could use less commas ',' in comments. As I understand, you need a comma if the relative sentence is at the beginning, but not if it is at the end: If Corona is over, I go to the office. but I go to the office if Corona is over. I think the same holds for 'because', 'while' etc. E.g., jvmtiEnvBase.cpp:1313, jvmtiImpl.cpp:646ff, vframe_hp.hpp 104ff Also, I like full sentences in comments. Especially for me as foreign speaker, this makes things much more clear. I.e., I try to make it a real sentence with articles, capitalized and a dot at the end if there is a subject and a verb in first place. E.g., jvmtiEnvBase.cpp:1327 In many places, your comments read really well but some are quite abbreviated I think. E.g. thread.cpp:2601 is an example where a simple 'a' helps a lot. "Single deoptimization is typically very short." I would add 'A': "A single deoptimization is typically very short (fast?)." An other meaning of the comment I first considered is this: "Single deoptimization is typically very short, all_threads deoptimization takes longer" having in mind the functions EscapeBarries::deoptimize_objects_all_threads() and EscapeBarries::deoptimize_objects() doing a single thread. German with it's compound nouns is helpful here :) Einzeldeoptimierung <--> eine einzelne Deoptimierung Best regards, Goetz. From coleen.phillimore at oracle.com Fri Jul 17 14:37:30 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 17 Jul 2020 10:37:30 -0400 Subject: RFR 8247878: Move Management strong oops to OopStorage In-Reply-To: <3cc4ef8a-0b3d-3827-1385-a93aed40d786@oracle.com> References: <05ce335a-d836-cf15-65b8-e015a5a93c30@oracle.com> <3cc4ef8a-0b3d-3827-1385-a93aed40d786@oracle.com> Message-ID: <6d4718e7-a040-8d3b-07b7-4e5d4f98e63f@oracle.com> Hi David, thank you for reviewing this. On 7/17/20 12:13 AM, David Holmes wrote: > Hi Coleen, > > On 17/07/2020 1:01 am, coleen.phillimore at oracle.com wrote: >> Summary: Use OopStorage for strong oops stored with memory and thread >> sampling and dumping, and remove oops_do and GC calls. >> >> These use OopStorageSet::vm_global()? OopStorage for now.? I'll >> change the thread sampling oops to use a new OopStorage once the GC >> code is changed to nicely allow additional oop storages.? The memory >> pool oops are never deleted once created, so they should stay in >> vm_global() oop storage. >> >> Tested with tiers 1-3 (tiers 4-6 with other changes) and >> javax/management tests.? I timed the tests to see if there was any >> noticeable performance difference, and there was not. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8247878.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8247878 > > Overall the bulk of the changes look good - lots of red! :) Yup! > > I'm unsure about defining AtomicOopHandle. I think the memory ordering > operations may be better kept in the code paths that use the > OopHandle. The name doesn't convey what this actually does in terms of > the release-set and resolve-acquire, and I can't think of a good name > that would do so**. If you do want to keep it then I think there > should be a non-acquire version of resolve as not all accesses require > acquire semantics. Looking at memoryManager.cpp (but the same applies > to memoryPool.cpp): I do want to keep the separate type because it's sort of inconvient to declare the underlying oop in the OopHandle as volatile or whatever we're going to do to replace volatile.? Plus a separate type allows the compiler to give an error if the access rules are broken.? Scattering these varying accesses around the vm with reasoning makes the code too complicated.? Kim and I came up with the name AtomicOopHandle since it uses the same mechanism as the underlying MO_ACQUIRE/MO_RELEASE semantics. > > ? 56 bool MemoryManager::is_manager(instanceHandle mh) const { > ? 57?? return mh() == _memory_mgr_obj.resolve_acquire(); > ? 58 } > > This doesn't have acquire semantics currently and doesn't need it. > > ?122?????? // The lock has done an acquire, so the load can't float > above it, but > ?123?????? // we need to do a load_acquire as above. > ?124?????? mgr_obj = _memory_mgr_obj.resolve_acquire(); Yes, this used to just have Atomic::load() and not acquire.? I didn't really see any reason to optimize and break the consistency in order to do a plain resolve().? I could add it, it just seems not useful. > > The original code and comment is wrong - this doesn't need acquire as > the lock already handles that. If another thread has set the mgr > object then it did so holding the lock that the current thread now > owns, which means the other thread had to have released the lock > first, hence when the current thread acquired the lock, all stores in > relation to the mgr object are already guaranteed to be visible. Yes, true.? I can fix the comment or remove it since using OopHandle, there will be a load that cannot float. > > ** Perhaps the OopHandle constructor, and resolve method should take > an optional memory order Decorator parameter? We still need to redeclare the underlying oop* though. > > --- > > Should we introduce a naming convention to help distinguish an oop > variable from an oopHandle? You seem to have adopted the convention > that _foo is the oopHandle and foo is the oop. But should we use foo_h > as we do (sometimes at least) for other handles? (I'm not generally > fond of encoding types in variable names but when handles and their > contents can be accessed in the same code, it does make things clearer.) I want to vote no on that.? _foo is the OopHandle because that's how the field is declared.? The only place I did this was here: http://cr.openjdk.java.net/~coleenp/2020/8247878.01/webrev/src/hotspot/share/services/threadService.cpp.udiff.html I don't think it'll help to have _blocker_object_h distinguishible from blocker_object.? I think the code is somewhat clear that the field is assigned at the end. Thanks, Coleen > > Thanks, > David > ----- > >> Thanks, >> Coleen >> >> >> From coleen.phillimore at oracle.com Fri Jul 17 14:50:27 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 17 Jul 2020 10:50:27 -0400 Subject: RFR 8247878: Move Management strong oops to OopStorage In-Reply-To: References: <05ce335a-d836-cf15-65b8-e015a5a93c30@oracle.com> Message-ID: <35eaeb91-7b56-0074-b871-47f24998d31d@oracle.com> Hi Kim, Thank you for reviewing this. On 7/17/20 5:02 AM, Kim Barrett wrote: >> On Jul 16, 2020, at 11:01 AM, coleen.phillimore at oracle.com wrote: >> >> Summary: Use OopStorage for strong oops stored with memory and thread sampling and dumping, and remove oops_do and GC calls. >> >> These use OopStorageSet::vm_global() OopStorage for now. I'll change the thread sampling oops to use a new OopStorage once the GC code is changed to nicely allow additional oop storages. The memory pool oops are never deleted once created, so they should stay in vm_global() oop storage. >> >> Tested with tiers 1-3 (tiers 4-6 with other changes) and javax/management tests. I timed the tests to see if there was any noticeable performance difference, and there was not. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/2020/8247878.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8247878 >> >> Thanks, >> Coleen > ------------------------------------------------------------------------------ > src/hotspot/share/oops/oopHandle.inline.hpp > 50 if (peek() != NULL) { > > Adding that seems like an unrelated change, and it's not clear to me > why this is being done. This was to save null checks in all the callers, particularly here: ThreadSnapshot::~ThreadSnapshot() { + _blocker_object.release(OopStorageSet::vm_global()); + _blocker_object_owner.release(OopStorageSet::vm_global()); + _threadObj.release(OopStorageSet::vm_global()); + > > ------------------------------------------------------------------------------ > src/hotspot/share/services/lowMemoryDetector.cpp > 299 if (_sensor_obj.peek() != NULL) { > 300 InstanceKlass* sensorKlass = Management::sun_management_Sensor_klass(CHECK); > 301 Handle sensor_h(THREAD, _sensor_obj.resolve()); > > I see no reason for a peek operation here, and think it just makes the > code harder to understand. Just unconditionally resolve the sensor. > > Similarly here: > 364 if (_sensor_obj.peek() != NULL) { > 365 InstanceKlass* sensorKlass = Management::sun_management_Sensor_klass(CHECK); > 366 Handle sensor(THREAD, _sensor_obj.resolve()); I can move the NULL check down after the Handle.? I was mostly keeping the existing pattern. > > ------------------------------------------------------------------------------ > src/hotspot/share/services/memoryManager.cpp > 136 _memory_mgr_obj = AtomicOopHandle(OopStorageSet::vm_global(), mgr_obj); > > There is a race here. The handle constructor allocates the oopstorage > entry and then does a release_store of the value into it. After (in > source order) the handle is constructed, it is copied into > _memory_mgr_obj, which is just a copy-assign of the oop* from > oopstorage. There's nothing to prevent that copy-assign from being > reordered before the store of the value into the oopstorage, either by > the compiler or the hardware. Ok, I see that. > > The _obj in _memory_mgr_obj is being read without locking and may be > written by another thread, so should itself be appropriate atomic. > It's not clear what the semantics of this new kind of handle are > supposed to be, but I think as used for _memory_mgr_obj there are > problems. > > The same is true for _memory_pool_obj. > > I think the atomicity here is in the wrong place. The pointee of the > oop* doesn't need atomic access, the oop* itself does. At least I > think so; both _memory_mgr_obj and _memory_pool_obj are lazily > assigned on first use and never change after that, right? Yes.? volatile is in the wrong place. > > One way to do that would be that the type of _memory_mgr_obj is `oop* > volatile`, and is initialized as > > oop* oop_ptr = ... allocate oopstorage entry ... > NativeAccess<>::oop_store(oop_ptr, value); > Atomic::release_store(&_memory_mgr_obj, oop_ptr); > > Alternatively, use > > volatile OopHandle _memory_mgr_obj; > > Atomic::release_store(&_memory_mgr_obj, OopHandle(...)); > > and teach Atomic how to deal with OopHandle by defining a > PrimitiveConversions::Translator for it. > > The declaration would be > > volatile OopHandle _memory_mgr_obj; > > and accesses would be > > Atomic::load_acquire(&_memory_mgr_obj).resolve(); > > And AtomicOopHandle isn't useful and should be discarded. Ok, yes, this is exactly what I want.? And David will be happy because he didn't like AtomicOopHandle. Thanks for catching this and your help. Coleen > > ------------------------------------------------------------------------------ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From coleen.phillimore at oracle.com Fri Jul 17 21:29:09 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 17 Jul 2020 17:29:09 -0400 Subject: RFR 8247878: Move Management strong oops to OopStorage In-Reply-To: <35eaeb91-7b56-0074-b871-47f24998d31d@oracle.com> References: <05ce335a-d836-cf15-65b8-e015a5a93c30@oracle.com> <35eaeb91-7b56-0074-b871-47f24998d31d@oracle.com> Message-ID: I've corrected this change with Kim's and David's feedback, and retested with tier1-3.? This is much better. incremental webrev at http://cr.openjdk.java.net/~coleenp/2020/8247878.02.incr/webrev full webrev at http://cr.openjdk.java.net/~coleenp/2020/8247878.02/webrev Thanks, Coleen On 7/17/20 10:50 AM, coleen.phillimore at oracle.com wrote: > > Hi Kim, Thank you for reviewing this. > > On 7/17/20 5:02 AM, Kim Barrett wrote: >>> On Jul 16, 2020, at 11:01 AM, coleen.phillimore at oracle.com wrote: >>> >>> Summary: Use OopStorage for strong oops stored with memory and >>> thread sampling and dumping, and remove oops_do and GC calls. >>> >>> These use OopStorageSet::vm_global()? OopStorage for now. I'll >>> change the thread sampling oops to use a new OopStorage once the GC >>> code is changed to nicely allow additional oop storages.? The memory >>> pool oops are never deleted once created, so they should stay in >>> vm_global() oop storage. >>> >>> Tested with tiers 1-3 (tiers 4-6 with other changes) and >>> javax/management tests.? I timed the tests to see if there was any >>> noticeable performance difference, and there was not. >>> >>> open webrev at >>> http://cr.openjdk.java.net/~coleenp/2020/8247878.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8247878 >>> >>> Thanks, >>> Coleen >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/oops/oopHandle.inline.hpp >> ?? 50?? if (peek() != NULL) { >> >> Adding that seems like an unrelated change, and it's not clear to me >> why this is being done. > > This was to save null checks in all the callers, particularly here: > > ?ThreadSnapshot::~ThreadSnapshot() { > + _blocker_object.release(OopStorageSet::vm_global()); > + _blocker_object_owner.release(OopStorageSet::vm_global()); > + _threadObj.release(OopStorageSet::vm_global()); > + > >> >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/services/lowMemoryDetector.cpp >> ? 299?? if (_sensor_obj.peek() != NULL) { >> ? 300???? InstanceKlass* sensorKlass = >> Management::sun_management_Sensor_klass(CHECK); >> ? 301???? Handle sensor_h(THREAD, _sensor_obj.resolve()); >> >> I see no reason for a peek operation here, and think it just makes the >> code harder to understand.? Just unconditionally resolve the sensor. >> >> Similarly here: >> ? 364?? if (_sensor_obj.peek() != NULL) { >> ? 365???? InstanceKlass* sensorKlass = >> Management::sun_management_Sensor_klass(CHECK); >> ? 366???? Handle sensor(THREAD, _sensor_obj.resolve()); > > I can move the NULL check down after the Handle.? I was mostly keeping > the existing pattern. >> >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/services/memoryManager.cpp >> ? 136?????? _memory_mgr_obj = >> AtomicOopHandle(OopStorageSet::vm_global(), mgr_obj); >> >> There is a race here. The handle constructor allocates the oopstorage >> entry and then does a release_store of the value into it. After (in >> source order) the handle is constructed, it is copied into >> _memory_mgr_obj, which is just a copy-assign of the oop* from >> oopstorage. There's nothing to prevent that copy-assign from being >> reordered before the store of the value into the oopstorage, either by >> the compiler or the hardware. > > Ok, I see that. >> >> The _obj in _memory_mgr_obj is being read without locking and may be >> written by another thread, so should itself be appropriate atomic. >> It's not clear what the semantics of this new kind of handle are >> supposed to be, but I think as used for _memory_mgr_obj there are >> problems. >> >> The same is true for _memory_pool_obj. >> >> I think the atomicity here is in the wrong place. The pointee of the >> oop* doesn't need atomic access, the oop* itself does.? At least I >> think so; both _memory_mgr_obj and _memory_pool_obj are lazily >> assigned on first use and never change after that, right? > > Yes.? volatile is in the wrong place. >> >> One way to do that would be that the type of _memory_mgr_obj is `oop* >> volatile`, and is initialized as >> >> ?? oop* oop_ptr = ... allocate oopstorage entry ... >> ?? NativeAccess<>::oop_store(oop_ptr, value); >> ?? Atomic::release_store(&_memory_mgr_obj, oop_ptr); >> >> Alternatively, use >> >> ?? volatile OopHandle _memory_mgr_obj; >> >> ?? Atomic::release_store(&_memory_mgr_obj, OopHandle(...)); >> >> and teach Atomic how to deal with OopHandle by defining a >> PrimitiveConversions::Translator for it. >> >> The declaration would be >> >> ?? volatile OopHandle _memory_mgr_obj; >> >> and accesses would be >> >> ?? Atomic::load_acquire(&_memory_mgr_obj).resolve(); >> >> And AtomicOopHandle isn't useful and should be discarded. > > Ok, yes, this is exactly what I want.? And David will be happy because > he didn't like AtomicOopHandle. > > Thanks for catching this and your help. > Coleen > >> >> ------------------------------------------------------------------------------ >> >> > From serguei.spitsyn at oracle.com Fri Jul 17 23:44:10 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 17 Jul 2020 16:44:10 -0700 Subject: tt Message-ID: From david.holmes at oracle.com Mon Jul 20 01:06:44 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 20 Jul 2020 11:06:44 +1000 Subject: RFR (M) 8249650: Optimize JNIHandle::make_local thread variable Message-ID: <8410d4a2-bbad-090f-55bf-88940f786781@oracle.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8249650 webrev: http://cr.openjdk.java.net/~dholmes/8249650/webrev/ This is a simple cleanup that touches files across a number of VM areas - hence the cross-post. Whilst working on a different JNI fix I noticed that in most cases in jni.cpp we were using the following form of make_local: JNIHandles::make_local(env, obj); and what that form does is first extract the thread from the JNIEnv: JavaThread* thread = JavaThread::thread_from_jni_environment(env); return thread->active_handles()->allocate_handle(obj); but there is also another, faster, variant for when you already have the "thread": jobject JNIHandles::make_local(Thread* thread, oop obj) { return thread->active_handles()->allocate_handle(obj); } When you look at the JNI_ENTRY wrapper (and related JVM_ENTRY, WB_ENTRY, UNSAFE_ENTRY etc) it has already extracted the thread from the JNIEnv: JavaThread* thread=JavaThread::thread_from_jni_environment(env); and further defined: Thread* THREAD = thread; so we always already have direct access to the "thread" available (or indirect via TRAPS), and in fact we can end up removing the make_local(JNIEnv* env, oop obj) variant altogether. Along the way I spotted some related issues with unnecessary use of Thread::current() when it is already available from TRAPS, and some other cases where we extracted the JNIEnv from a thread only to later extract the thread from the JNIEnv. Testing: tiers 1 - 3 Thanks, David ----- From david.holmes at oracle.com Mon Jul 20 04:16:49 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 20 Jul 2020 14:16:49 +1000 Subject: RFR (M) 8249650: Optimize JNIHandle::make_local thread variable usage In-Reply-To: <8410d4a2-bbad-090f-55bf-88940f786781@oracle.com> References: <8410d4a2-bbad-090f-55bf-88940f786781@oracle.com> Message-ID: Subject line got truncated by accident ... On 20/07/2020 11:06 am, David Holmes wrote: > Bug: https://bugs.openjdk.java.net/browse/JDK-8249650 > webrev: http://cr.openjdk.java.net/~dholmes/8249650/webrev/ > > This is a simple cleanup that touches files across a number of VM areas > - hence the cross-post. > > Whilst working on a different JNI fix I noticed that in most cases in > jni.cpp we were using the following form of make_local: > > JNIHandles::make_local(env, obj); > > and what that form does is first extract the thread from the JNIEnv: > > JavaThread* thread = JavaThread::thread_from_jni_environment(env); > return thread->active_handles()->allocate_handle(obj); > > but there is also another, faster, variant for when you already have the > "thread": > > jobject JNIHandles::make_local(Thread* thread, oop obj) { > ? return thread->active_handles()->allocate_handle(obj); > } > > When you look at the JNI_ENTRY wrapper (and related JVM_ENTRY, WB_ENTRY, > UNSAFE_ENTRY etc) it has already extracted the thread from the JNIEnv: > > ??? JavaThread* thread=JavaThread::thread_from_jni_environment(env); > > and further defined: > > ??? Thread* THREAD = thread; > > so we always already have direct access to the "thread" available (or > indirect via TRAPS), and in fact we can end up removing the > make_local(JNIEnv* env, oop obj) variant altogether. > > Along the way I spotted some related issues with unnecessary use of > Thread::current() when it is already available from TRAPS, and some > other cases where we extracted the JNIEnv from a thread only to later > extract the thread from the JNIEnv. > > Testing: tiers 1 - 3 > > Thanks, > David > ----- From david.holmes at oracle.com Mon Jul 20 04:47:25 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 20 Jul 2020 14:47:25 +1000 Subject: RFR 8247878: Move Management strong oops to OopStorage In-Reply-To: References: <05ce335a-d836-cf15-65b8-e015a5a93c30@oracle.com> <35eaeb91-7b56-0074-b871-47f24998d31d@oracle.com> Message-ID: Hi Coleen, On 18/07/2020 7:29 am, coleen.phillimore at oracle.com wrote: > > I've corrected this change with Kim's and David's feedback, and retested > with tier1-3.? This is much better. Yes this is better. :) Thanks to Kim for clarifying the acquire/release issue. LGTM. Thanks, David ----- > incremental webrev at > http://cr.openjdk.java.net/~coleenp/2020/8247878.02.incr/webrev > full webrev at http://cr.openjdk.java.net/~coleenp/2020/8247878.02/webrev > > Thanks, > Coleen > > On 7/17/20 10:50 AM, coleen.phillimore at oracle.com wrote: >> >> Hi Kim, Thank you for reviewing this. >> >> On 7/17/20 5:02 AM, Kim Barrett wrote: >>>> On Jul 16, 2020, at 11:01 AM, coleen.phillimore at oracle.com wrote: >>>> >>>> Summary: Use OopStorage for strong oops stored with memory and >>>> thread sampling and dumping, and remove oops_do and GC calls. >>>> >>>> These use OopStorageSet::vm_global()? OopStorage for now. I'll >>>> change the thread sampling oops to use a new OopStorage once the GC >>>> code is changed to nicely allow additional oop storages.? The memory >>>> pool oops are never deleted once created, so they should stay in >>>> vm_global() oop storage. >>>> >>>> Tested with tiers 1-3 (tiers 4-6 with other changes) and >>>> javax/management tests.? I timed the tests to see if there was any >>>> noticeable performance difference, and there was not. >>>> >>>> open webrev at >>>> http://cr.openjdk.java.net/~coleenp/2020/8247878.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8247878 >>>> >>>> Thanks, >>>> Coleen >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/oops/oopHandle.inline.hpp >>> ?? 50?? if (peek() != NULL) { >>> >>> Adding that seems like an unrelated change, and it's not clear to me >>> why this is being done. >> >> This was to save null checks in all the callers, particularly here: >> >> ?ThreadSnapshot::~ThreadSnapshot() { >> + _blocker_object.release(OopStorageSet::vm_global()); >> + _blocker_object_owner.release(OopStorageSet::vm_global()); >> + _threadObj.release(OopStorageSet::vm_global()); >> + >> >>> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/services/lowMemoryDetector.cpp >>> ? 299?? if (_sensor_obj.peek() != NULL) { >>> ? 300???? InstanceKlass* sensorKlass = >>> Management::sun_management_Sensor_klass(CHECK); >>> ? 301???? Handle sensor_h(THREAD, _sensor_obj.resolve()); >>> >>> I see no reason for a peek operation here, and think it just makes the >>> code harder to understand.? Just unconditionally resolve the sensor. >>> >>> Similarly here: >>> ? 364?? if (_sensor_obj.peek() != NULL) { >>> ? 365???? InstanceKlass* sensorKlass = >>> Management::sun_management_Sensor_klass(CHECK); >>> ? 366???? Handle sensor(THREAD, _sensor_obj.resolve()); >> >> I can move the NULL check down after the Handle.? I was mostly keeping >> the existing pattern. >>> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/services/memoryManager.cpp >>> ? 136?????? _memory_mgr_obj = >>> AtomicOopHandle(OopStorageSet::vm_global(), mgr_obj); >>> >>> There is a race here. The handle constructor allocates the oopstorage >>> entry and then does a release_store of the value into it. After (in >>> source order) the handle is constructed, it is copied into >>> _memory_mgr_obj, which is just a copy-assign of the oop* from >>> oopstorage. There's nothing to prevent that copy-assign from being >>> reordered before the store of the value into the oopstorage, either by >>> the compiler or the hardware. >> >> Ok, I see that. >>> >>> The _obj in _memory_mgr_obj is being read without locking and may be >>> written by another thread, so should itself be appropriate atomic. >>> It's not clear what the semantics of this new kind of handle are >>> supposed to be, but I think as used for _memory_mgr_obj there are >>> problems. >>> >>> The same is true for _memory_pool_obj. >>> >>> I think the atomicity here is in the wrong place. The pointee of the >>> oop* doesn't need atomic access, the oop* itself does.? At least I >>> think so; both _memory_mgr_obj and _memory_pool_obj are lazily >>> assigned on first use and never change after that, right? >> >> Yes.? volatile is in the wrong place. >>> >>> One way to do that would be that the type of _memory_mgr_obj is `oop* >>> volatile`, and is initialized as >>> >>> ?? oop* oop_ptr = ... allocate oopstorage entry ... >>> ?? NativeAccess<>::oop_store(oop_ptr, value); >>> ?? Atomic::release_store(&_memory_mgr_obj, oop_ptr); >>> >>> Alternatively, use >>> >>> ?? volatile OopHandle _memory_mgr_obj; >>> >>> ?? Atomic::release_store(&_memory_mgr_obj, OopHandle(...)); >>> >>> and teach Atomic how to deal with OopHandle by defining a >>> PrimitiveConversions::Translator for it. >>> >>> The declaration would be >>> >>> ?? volatile OopHandle _memory_mgr_obj; >>> >>> and accesses would be >>> >>> ?? Atomic::load_acquire(&_memory_mgr_obj).resolve(); >>> >>> And AtomicOopHandle isn't useful and should be discarded. >> >> Ok, yes, this is exactly what I want.? And David will be happy because >> he didn't like AtomicOopHandle. >> >> Thanks for catching this and your help. >> Coleen >> >>> >>> ------------------------------------------------------------------------------ >>> >>> >> > From kim.barrett at oracle.com Mon Jul 20 05:22:49 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 20 Jul 2020 01:22:49 -0400 Subject: RFR (M) 8249650: Optimize JNIHandle::make_local thread variable usage In-Reply-To: References: <8410d4a2-bbad-090f-55bf-88940f786781@oracle.com> Message-ID: <0590E210-6F23-4498-A51A-C3DAEF54B5AB@oracle.com> > On Jul 20, 2020, at 12:16 AM, David Holmes wrote: > > Subject line got truncated by accident ... > > On 20/07/2020 11:06 am, David Holmes wrote: >> Bug: https://bugs.openjdk.java.net/browse/JDK-8249650 >> webrev: http://cr.openjdk.java.net/~dholmes/8249650/webrev/ >> This is a simple cleanup that touches files across a number of VM areas - hence the cross-post. >> Whilst working on a different JNI fix I noticed that in most cases in jni.cpp we were using the following form of make_local: >> JNIHandles::make_local(env, obj); >> and what that form does is first extract the thread from the JNIEnv: >> JavaThread* thread = JavaThread::thread_from_jni_environment(env); >> return thread->active_handles()->allocate_handle(obj); >> but there is also another, faster, variant for when you already have the "thread": >> jobject JNIHandles::make_local(Thread* thread, oop obj) { >> return thread->active_handles()->allocate_handle(obj); >> } >> When you look at the JNI_ENTRY wrapper (and related JVM_ENTRY, WB_ENTRY, UNSAFE_ENTRY etc) it has already extracted the thread from the JNIEnv: >> JavaThread* thread=JavaThread::thread_from_jni_environment(env); >> and further defined: >> Thread* THREAD = thread; >> so we always already have direct access to the "thread" available (or indirect via TRAPS), and in fact we can end up removing the make_local(JNIEnv* env, oop obj) variant altogether. >> Along the way I spotted some related issues with unnecessary use of Thread::current() when it is already available from TRAPS, and some other cases where we extracted the JNIEnv from a thread only to later extract the thread from the JNIEnv. >> Testing: tiers 1 - 3 >> Thanks, >> David >> ----- ------------------------------------------------------------------------------ src/hotspot/share/classfile/javaClasses.cpp 439 JNIEnv *env = thread->jni_environment(); Since env is no longer used on the next line, move this down to where it is used, at line 444. ------------------------------------------------------------------------------ src/hotspot/share/classfile/verifier.cpp 299 JNIEnv *env = thread->jni_environment(); env now seems to only be used at line 320. Move this closer. ------------------------------------------------------------------------------ src/hotspot/share/prims/jni.cpp 743 result = JNIHandles::make_local(THREAD, result_handle()); jni_PopLocalFrame is now using a mix of "thread" and "THREAD", where previously it just used "thread". Maybe this change shouldn't be made? Or can the other uses be changed to THREAD for consistency? ------------------------------------------------------------------------------ src/hotspot/share/prims/jvm.cpp The calls to JvmtiExport::post_vm_object_alloc have to use "thread" instead of "THREAD", even though other places nearby are using "THREAD". That inconsistency is kind of unfortunate, but doesn't seem easily avoidable. ------------------------------------------------------------------------------ From kim.barrett at oracle.com Mon Jul 20 05:25:13 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 20 Jul 2020 01:25:13 -0400 Subject: RFR 8247878: Move Management strong oops to OopStorage In-Reply-To: References: <05ce335a-d836-cf15-65b8-e015a5a93c30@oracle.com> <35eaeb91-7b56-0074-b871-47f24998d31d@oracle.com> Message-ID: <78CB6EA3-AAFA-498F-827A-D3EE82883FC9@oracle.com> > On Jul 20, 2020, at 12:47 AM, David Holmes wrote: > > Hi Coleen, > > On 18/07/2020 7:29 am, coleen.phillimore at oracle.com wrote: >> I've corrected this change with Kim's and David's feedback, and retested with tier1-3. This is much better. > > Yes this is better. :) Thanks to Kim for clarifying the acquire/release issue. > > LGTM. > > Thanks, > David Looks good. From david.holmes at oracle.com Mon Jul 20 05:53:37 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 20 Jul 2020 15:53:37 +1000 Subject: RFR (M) 8249650: Optimize JNIHandle::make_local thread variable usage In-Reply-To: <0590E210-6F23-4498-A51A-C3DAEF54B5AB@oracle.com> References: <8410d4a2-bbad-090f-55bf-88940f786781@oracle.com> <0590E210-6F23-4498-A51A-C3DAEF54B5AB@oracle.com> Message-ID: <6166e191-c954-70e5-5595-956a0c145d10@oracle.com> Hi Kim, Thanks for looking at this. Updated webrev at: http://cr.openjdk.java.net/~dholmes/8249650/webrev.v2/ On 20/07/2020 3:22 pm, Kim Barrett wrote: >> On Jul 20, 2020, at 12:16 AM, David Holmes wrote: >> >> Subject line got truncated by accident ... >> >> On 20/07/2020 11:06 am, David Holmes wrote: >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8249650 >>> webrev: http://cr.openjdk.java.net/~dholmes/8249650/webrev/ >>> This is a simple cleanup that touches files across a number of VM areas - hence the cross-post. >>> Whilst working on a different JNI fix I noticed that in most cases in jni.cpp we were using the following form of make_local: >>> JNIHandles::make_local(env, obj); >>> and what that form does is first extract the thread from the JNIEnv: >>> JavaThread* thread = JavaThread::thread_from_jni_environment(env); >>> return thread->active_handles()->allocate_handle(obj); >>> but there is also another, faster, variant for when you already have the "thread": >>> jobject JNIHandles::make_local(Thread* thread, oop obj) { >>> return thread->active_handles()->allocate_handle(obj); >>> } >>> When you look at the JNI_ENTRY wrapper (and related JVM_ENTRY, WB_ENTRY, UNSAFE_ENTRY etc) it has already extracted the thread from the JNIEnv: >>> JavaThread* thread=JavaThread::thread_from_jni_environment(env); >>> and further defined: >>> Thread* THREAD = thread; >>> so we always already have direct access to the "thread" available (or indirect via TRAPS), and in fact we can end up removing the make_local(JNIEnv* env, oop obj) variant altogether. >>> Along the way I spotted some related issues with unnecessary use of Thread::current() when it is already available from TRAPS, and some other cases where we extracted the JNIEnv from a thread only to later extract the thread from the JNIEnv. >>> Testing: tiers 1 - 3 >>> Thanks, >>> David >>> ----- > > ------------------------------------------------------------------------------ > src/hotspot/share/classfile/javaClasses.cpp > 439 JNIEnv *env = thread->jni_environment(); > > Since env is no longer used on the next line, move this down to where > it is used, at line 444. Fixed. > ------------------------------------------------------------------------------ > src/hotspot/share/classfile/verifier.cpp > 299 JNIEnv *env = thread->jni_environment(); > > env now seems to only be used at line 320. Move this closer. Fixed. > ------------------------------------------------------------------------------ > src/hotspot/share/prims/jni.cpp > 743 result = JNIHandles::make_local(THREAD, result_handle()); > > jni_PopLocalFrame is now using a mix of "thread" and "THREAD", where > previously it just used "thread". Maybe this change shouldn't be made? > Or can the other uses be changed to THREAD for consistency? "thread" and "THREAD" are interchangeable for anything expecting a "Thread*" (and somewhat surprisingly a number of API's that only work for JavaThreads actually take a Thread*. :( ). I had choice between trying to be file-wide consistent with the make_local calls, versus local-code consistent, and used THREAD as it is available in both JNI_ENTRY and via TRAPS. But I can certainly make a local change to "thread" for local consistency. > ------------------------------------------------------------------------------ > src/hotspot/share/prims/jvm.cpp > > The calls to JvmtiExport::post_vm_object_alloc have to use "thread" > instead of "THREAD", even though other places nearby are using > "THREAD". That inconsistency is kind of unfortunate, but doesn't seem > easily avoidable. Everything that uses THREAD in a JVM_ENTRY method can be changed to use "thread" instead. But I'm not sure it's a consistency worth pursuing at least as part of these changes (there are likely similar issues with most of the touched files). Thanks, David > ------------------------------------------------------------------------------ > From kim.barrett at oracle.com Mon Jul 20 06:15:13 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 20 Jul 2020 02:15:13 -0400 Subject: RFR (M) 8249650: Optimize JNIHandle::make_local thread variable usage In-Reply-To: <6166e191-c954-70e5-5595-956a0c145d10@oracle.com> References: <8410d4a2-bbad-090f-55bf-88940f786781@oracle.com> <0590E210-6F23-4498-A51A-C3DAEF54B5AB@oracle.com> <6166e191-c954-70e5-5595-956a0c145d10@oracle.com> Message-ID: > On Jul 20, 2020, at 1:53 AM, David Holmes wrote: > > Hi Kim, > > Thanks for looking at this. > > Updated webrev at: > > http://cr.openjdk.java.net/~dholmes/8249650/webrev.v2/ Looks good. > > On 20/07/2020 3:22 pm, Kim Barrett wrote: >>> On Jul 20, 2020, at 12:16 AM, David Holmes wrote: >> src/hotspot/share/prims/jni.cpp >> 743 result = JNIHandles::make_local(THREAD, result_handle()); >> jni_PopLocalFrame is now using a mix of "thread" and "THREAD", where >> previously it just used "thread". Maybe this change shouldn't be made? >> Or can the other uses be changed to THREAD for consistency? > > "thread" and "THREAD" are interchangeable for anything expecting a "Thread*" (and somewhat surprisingly a number of API's that only work for JavaThreads actually take a Thread*. :( ). I had choice between trying to be file-wide consistent with the make_local calls, versus local-code consistent, and used THREAD as it is available in both JNI_ENTRY and via TRAPS. But I can certainly make a local change to "thread" for local consistency. I don?t feel strongly either way. It just struck me as a little odd to have the mix in close proximity, especially since I think consistently using either one might work in this function. But being consistent about make_local usage has something to be said for it too. >> src/hotspot/share/prims/jvm.cpp >> The calls to JvmtiExport::post_vm_object_alloc have to use "thread" >> instead of "THREAD", even though other places nearby are using >> "THREAD". That inconsistency is kind of unfortunate, but doesn't seem >> easily avoidable. > > Everything that uses THREAD in a JVM_ENTRY method can be changed to use "thread" instead. But I'm not sure it's a consistency worth pursuing at least as part of these changes (there are likely similar issues with most of the touched files). Yeah, it?s not really obvious whether to use THREAD or thread in some cases. But I agree that addressing any inconsistencies there is mostly out of scope for this change. From david.holmes at oracle.com Mon Jul 20 07:53:48 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 20 Jul 2020 17:53:48 +1000 Subject: RFR (M) 8249650: Optimize JNIHandle::make_local thread variable usage In-Reply-To: References: <8410d4a2-bbad-090f-55bf-88940f786781@oracle.com> <0590E210-6F23-4498-A51A-C3DAEF54B5AB@oracle.com> <6166e191-c954-70e5-5595-956a0c145d10@oracle.com> Message-ID: <6e0d9af0-92f0-1eba-fc0a-22eebf008fe0@oracle.com> Thanks Kim! David On 20/07/2020 4:15 pm, Kim Barrett wrote: >> On Jul 20, 2020, at 1:53 AM, David Holmes wrote: >> >> Hi Kim, >> >> Thanks for looking at this. >> >> Updated webrev at: >> >> http://cr.openjdk.java.net/~dholmes/8249650/webrev.v2/ > > Looks good. > >> >> On 20/07/2020 3:22 pm, Kim Barrett wrote: >>>> On Jul 20, 2020, at 12:16 AM, David Holmes wrote: >>> src/hotspot/share/prims/jni.cpp >>> 743 result = JNIHandles::make_local(THREAD, result_handle()); >>> jni_PopLocalFrame is now using a mix of "thread" and "THREAD", where >>> previously it just used "thread". Maybe this change shouldn't be made? >>> Or can the other uses be changed to THREAD for consistency? >> >> "thread" and "THREAD" are interchangeable for anything expecting a "Thread*" (and somewhat surprisingly a number of API's that only work for JavaThreads actually take a Thread*. :( ). I had choice between trying to be file-wide consistent with the make_local calls, versus local-code consistent, and used THREAD as it is available in both JNI_ENTRY and via TRAPS. But I can certainly make a local change to "thread" for local consistency. > > I don?t feel strongly either way. It just struck me as a little odd to have the mix in close proximity, > especially since I think consistently using either one might work in this function. But being consistent > about make_local usage has something to be said for it too. > >>> src/hotspot/share/prims/jvm.cpp >>> The calls to JvmtiExport::post_vm_object_alloc have to use "thread" >>> instead of "THREAD", even though other places nearby are using >>> "THREAD". That inconsistency is kind of unfortunate, but doesn't seem >>> easily avoidable. >> >> Everything that uses THREAD in a JVM_ENTRY method can be changed to use "thread" instead. But I'm not sure it's a consistency worth pursuing at least as part of these changes (there are likely similar issues with most of the touched files). > > Yeah, it?s not really obvious whether to use THREAD or thread in some cases. > But I agree that addressing any inconsistencies there is mostly out of scope for > this change. > From richard.reingruber at sap.com Mon Jul 20 08:15:45 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Mon, 20 Jul 2020 08:15:45 +0000 Subject: RFR(S) 8249293: Unsafe stackwalk in VM_GetOrSetLocal::doit_prologue() Message-ID: Hi, please help review this fix for VM_GetOrSetLocal. It moves the unsafe stackwalk from the vm operation prologue before the safepoint into the doit() method executed at the safepoint. Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8249293/webrev.0/index.html Bug: https://bugs.openjdk.java.net/browse/JDK-8249293 According to the JVMTI spec on local variable access it is not required to suspend the target thread T [1]. The operation will simply fail with JVMTI_ERROR_NO_MORE_FRAMES if T is executing bytecodes. It will succeed though if T is blocked because of synchronization or executing some native code. The issue is that in the latter case the stack walk in VM_GetOrSetLocal::doit_prologue() to prepare the access to the local variable is unsafe, because it is done before the safepoint and it races with T returning to execute bytecodes making its stack not walkable. The included test shows that this can crash the VM if T wins the race. Manual testing: - new test test/hotspot/jtreg/serviceability/jvmti/GetLocalVariable/GetLocalWithoutSuspendTest.java - test/hotspot/jtreg/vmTestbase/nsk/jvmti - test/hotspot/jtreg/serviceability/jvmti Nightly regression tests @SAP: JCK and JTREG, also in Xcomp mode, SPECjvm2008, SPECjbb2015, Renaissance Suite, SAP specific tests with fastdebug and release builds on all platforms Thanks, Richard. [1] https://docs.oracle.com/en/java/javase/14/docs/specs/jvmti.html#local From coleen.phillimore at oracle.com Mon Jul 20 11:33:26 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 20 Jul 2020 07:33:26 -0400 Subject: RFR 8247878: Move Management strong oops to OopStorage In-Reply-To: References: <05ce335a-d836-cf15-65b8-e015a5a93c30@oracle.com> <35eaeb91-7b56-0074-b871-47f24998d31d@oracle.com> Message-ID: David, thank you for the review and help. Coleen On 7/20/20 12:47 AM, David Holmes wrote: > Hi Coleen, > > On 18/07/2020 7:29 am, coleen.phillimore at oracle.com wrote: >> >> I've corrected this change with Kim's and David's feedback, and >> retested with tier1-3.? This is much better. > > Yes this is better. :) Thanks to Kim for clarifying the > acquire/release issue. > > LGTM. > > Thanks, > David > ----- > >> incremental webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8247878.02.incr/webrev >> full webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8247878.02/webrev >> >> Thanks, >> Coleen >> >> On 7/17/20 10:50 AM, coleen.phillimore at oracle.com wrote: >>> >>> Hi Kim, Thank you for reviewing this. >>> >>> On 7/17/20 5:02 AM, Kim Barrett wrote: >>>>> On Jul 16, 2020, at 11:01 AM, coleen.phillimore at oracle.com wrote: >>>>> >>>>> Summary: Use OopStorage for strong oops stored with memory and >>>>> thread sampling and dumping, and remove oops_do and GC calls. >>>>> >>>>> These use OopStorageSet::vm_global()? OopStorage for now. I'll >>>>> change the thread sampling oops to use a new OopStorage once the >>>>> GC code is changed to nicely allow additional oop storages.? The >>>>> memory pool oops are never deleted once created, so they should >>>>> stay in vm_global() oop storage. >>>>> >>>>> Tested with tiers 1-3 (tiers 4-6 with other changes) and >>>>> javax/management tests.? I timed the tests to see if there was any >>>>> noticeable performance difference, and there was not. >>>>> >>>>> open webrev at >>>>> http://cr.openjdk.java.net/~coleenp/2020/8247878.01/webrev >>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8247878 >>>>> >>>>> Thanks, >>>>> Coleen >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/share/oops/oopHandle.inline.hpp >>>> ?? 50?? if (peek() != NULL) { >>>> >>>> Adding that seems like an unrelated change, and it's not clear to me >>>> why this is being done. >>> >>> This was to save null checks in all the callers, particularly here: >>> >>> ?ThreadSnapshot::~ThreadSnapshot() { >>> + _blocker_object.release(OopStorageSet::vm_global()); >>> + _blocker_object_owner.release(OopStorageSet::vm_global()); >>> + _threadObj.release(OopStorageSet::vm_global()); >>> + >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/share/services/lowMemoryDetector.cpp >>>> ? 299?? if (_sensor_obj.peek() != NULL) { >>>> ? 300???? InstanceKlass* sensorKlass = >>>> Management::sun_management_Sensor_klass(CHECK); >>>> ? 301???? Handle sensor_h(THREAD, _sensor_obj.resolve()); >>>> >>>> I see no reason for a peek operation here, and think it just makes the >>>> code harder to understand.? Just unconditionally resolve the sensor. >>>> >>>> Similarly here: >>>> ? 364?? if (_sensor_obj.peek() != NULL) { >>>> ? 365???? InstanceKlass* sensorKlass = >>>> Management::sun_management_Sensor_klass(CHECK); >>>> ? 366???? Handle sensor(THREAD, _sensor_obj.resolve()); >>> >>> I can move the NULL check down after the Handle.? I was mostly >>> keeping the existing pattern. >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/share/services/memoryManager.cpp >>>> ? 136?????? _memory_mgr_obj = >>>> AtomicOopHandle(OopStorageSet::vm_global(), mgr_obj); >>>> >>>> There is a race here. The handle constructor allocates the oopstorage >>>> entry and then does a release_store of the value into it. After (in >>>> source order) the handle is constructed, it is copied into >>>> _memory_mgr_obj, which is just a copy-assign of the oop* from >>>> oopstorage. There's nothing to prevent that copy-assign from being >>>> reordered before the store of the value into the oopstorage, either by >>>> the compiler or the hardware. >>> >>> Ok, I see that. >>>> >>>> The _obj in _memory_mgr_obj is being read without locking and may be >>>> written by another thread, so should itself be appropriate atomic. >>>> It's not clear what the semantics of this new kind of handle are >>>> supposed to be, but I think as used for _memory_mgr_obj there are >>>> problems. >>>> >>>> The same is true for _memory_pool_obj. >>>> >>>> I think the atomicity here is in the wrong place. The pointee of the >>>> oop* doesn't need atomic access, the oop* itself does.? At least I >>>> think so; both _memory_mgr_obj and _memory_pool_obj are lazily >>>> assigned on first use and never change after that, right? >>> >>> Yes.? volatile is in the wrong place. >>>> >>>> One way to do that would be that the type of _memory_mgr_obj is `oop* >>>> volatile`, and is initialized as >>>> >>>> ?? oop* oop_ptr = ... allocate oopstorage entry ... >>>> ?? NativeAccess<>::oop_store(oop_ptr, value); >>>> ?? Atomic::release_store(&_memory_mgr_obj, oop_ptr); >>>> >>>> Alternatively, use >>>> >>>> ?? volatile OopHandle _memory_mgr_obj; >>>> >>>> ?? Atomic::release_store(&_memory_mgr_obj, OopHandle(...)); >>>> >>>> and teach Atomic how to deal with OopHandle by defining a >>>> PrimitiveConversions::Translator for it. >>>> >>>> The declaration would be >>>> >>>> ?? volatile OopHandle _memory_mgr_obj; >>>> >>>> and accesses would be >>>> >>>> ?? Atomic::load_acquire(&_memory_mgr_obj).resolve(); >>>> >>>> And AtomicOopHandle isn't useful and should be discarded. >>> >>> Ok, yes, this is exactly what I want.? And David will be happy >>> because he didn't like AtomicOopHandle. >>> >>> Thanks for catching this and your help. >>> Coleen >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>> >> From coleen.phillimore at oracle.com Mon Jul 20 11:33:48 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 20 Jul 2020 07:33:48 -0400 Subject: RFR 8247878: Move Management strong oops to OopStorage In-Reply-To: <78CB6EA3-AAFA-498F-827A-D3EE82883FC9@oracle.com> References: <05ce335a-d836-cf15-65b8-e015a5a93c30@oracle.com> <35eaeb91-7b56-0074-b871-47f24998d31d@oracle.com> <78CB6EA3-AAFA-498F-827A-D3EE82883FC9@oracle.com> Message-ID: <5d8f7e96-674a-4e4f-b5cb-f4054263ba65@oracle.com> Kim, Thank you for your help. Coleen On 7/20/20 1:25 AM, Kim Barrett wrote: >> On Jul 20, 2020, at 12:47 AM, David Holmes wrote: >> >> Hi Coleen, >> >> On 18/07/2020 7:29 am, coleen.phillimore at oracle.com wrote: >>> I've corrected this change with Kim's and David's feedback, and retested with tier1-3. This is much better. >> Yes this is better. :) Thanks to Kim for clarifying the acquire/release issue. >> >> LGTM. >> >> Thanks, >> David > Looks good. > > From daniel.daugherty at oracle.com Mon Jul 20 17:07:10 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 20 Jul 2020 13:07:10 -0400 Subject: RFR (M) 8249650: Optimize JNIHandle::make_local thread variable usage In-Reply-To: <6166e191-c954-70e5-5595-956a0c145d10@oracle.com> References: <8410d4a2-bbad-090f-55bf-88940f786781@oracle.com> <0590E210-6F23-4498-A51A-C3DAEF54B5AB@oracle.com> <6166e191-c954-70e5-5595-956a0c145d10@oracle.com> Message-ID: <328fb322-5b14-968b-7b13-4b449a8d98fd@oracle.com> On 7/20/20 1:53 AM, David Holmes wrote: > Hi Kim, > > Thanks for looking at this. > > Updated webrev at: > > http://cr.openjdk.java.net/~dholmes/8249650/webrev.v2/ I like this cleanup very much! src/hotspot/share/classfile/javaClasses.cpp ??? No comments. src/hotspot/share/classfile/verifier.cpp ??? L298: ? JavaThread* thread = (JavaThread*)THREAD; ??? L307: ? ResourceMark rm(THREAD); ??????? Since we've gone to the trouble of creating the 'thread' variable, ??????? I would prefer it to be used instead of THREAD where possible. src/hotspot/share/jvmci/jvmciCompilerToVM.cpp ??? L1021: ? HandleMark hm; ??????? Can this be 'hm(THREAD)'? (Not your problem, but while you're ??????? in that file?) src/hotspot/share/prims/jni.cpp ??? No comments. src/hotspot/share/prims/jvm.cpp ??? L140: ? ResourceMark rm; ??????? Can this be 'rm(THREAD)'? (Not your problem, but while you're ??????? in that file?) ??? L611: ? Handle stackStream_h(THREAD, JNIHandles::resolve_non_null(stackStream)); ??? L617: ? objArrayHandle frames_array_h(THREAD, fa); ??? L626: ? return JNIHandles::make_local(THREAD, result); ??????? Since we've gone to the trouble of creating the 'jt' variable, ??????? I would prefer it to be used instead of THREAD where possible. ??? L767: ? vframeStream vfst(thread); ??? L788???????? return (jclass) JNIHandles::make_local(THREAD, m->method_holder()->java_mirror()); ??????? Can we use 'thread' on L788? (preferred) ??????? Can we use 'THREAD' on L767? (less preferred) ??? L949: ? ResourceMark rm(THREAD); ??? L951: ? Handle class_loader (THREAD, JNIHandles::resolve(loader)); ??? L955: ?????????????????????????? THREAD); ??? L957: ? Handle protection_domain (THREAD, JNIHandles::resolve(pd)); ??? L968: ? return (jclass) JNIHandles::make_local(THREAD, k->java_mirror()); ??????? Since we've gone to the trouble of creating the 'jt' variable, ??????? I would prefer it to be used instead of THREAD where possible. ??? L986: ? JavaThread* jt = (JavaThread*) THREAD; ??????? This 'jt' is unused and can be deleted (Not your problem, but while you're ??????? in that file?) ??? L1154: ? while (*p != '\0') { ??? L1155: ????? if (*p == '.') { ??? L1156: ????????? *p = '/'; ??? L1157: ????? } ??? L1158: ????? p++; ??????? Nit - the indents are wrong on L1155-58. (Not your problem, but while you're ??????? in that file?) ??? L1389: ? ResourceMark rm(THREAD); ??? L1446: ??? return JNIHandles::make_local(THREAD, result); ??? L1460: ? return JNIHandles::make_local(THREAD, result); ??????? Can we use 'thread' on L1389? (preferred) And then the line you ??????? touched could also be 'thread' and we'll be consistent in this ??????? function... ??? L3287: ? oop jthread = thread->threadObj(); ??? L3288: ? assert (thread != NULL, "no current thread!"); ??????? I think the assert is wrong. It should be: ??????????? assert(jthread != NULL, "no current thread!"); ??????? If 'thread == NULL', then we would have crashed at L3287. ??????? Also notice that I deleted the extra ' ' before '('. (Not ??????? your problem, but while you're in that file?) ??? L3289: ? return JNIHandles::make_local(THREAD, jthread); ??????? Can you use 'thread' instead of 'THREAD' here for consistency? ??? L3681: ??? method_handle = Handle(THREAD, JNIHandles::resolve(method)); ??? L3682: ??? Handle receiver(THREAD, JNIHandles::resolve(obj)); ??? L3683: ??? objArrayHandle args(THREAD, objArrayOop(JNIHandles::resolve(args0))); ??? L3685: ??? jobject res = JNIHandles::make_local(THREAD, result); ??????? Can you use 'thread' instead of 'THREAD' here for consistency? ??? L3705: ? objArrayHandle args(THREAD, objArrayOop(JNIHandles::resolve(args0))); ??? L3707?? jobject res = JNIHandles::make_local(THREAD, result); ??????? Can you use 'thread' instead of 'THREAD' here for consistency? src/hotspot/share/prims/methodHandles.cpp ??? No comments. src/hotspot/share/prims/methodHandles.hpp ??? No comments. src/hotspot/share/prims/unsafe.cpp ??? No comments. src/hotspot/share/prims/whitebox.cpp ??? No comments. src/hotspot/share/runtime/jniHandles.cpp ??? No comments. src/hotspot/share/runtime/jniHandles.hpp ??? No comments. src/hotspot/share/services/management.cpp ??? No comments. None of my comments above are "must do". If you choose to make the changes, a new webrev isn't required, but would be useful for a sanity check. Thumbs up. Dan > > On 20/07/2020 3:22 pm, Kim Barrett wrote: >>> On Jul 20, 2020, at 12:16 AM, David Holmes >>> wrote: >>> >>> Subject line got truncated by accident ... >>> >>> On 20/07/2020 11:06 am, David Holmes wrote: >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8249650 >>>> webrev: http://cr.openjdk.java.net/~dholmes/8249650/webrev/ >>>> This is a simple cleanup that touches files across a number of VM >>>> areas - hence the cross-post. >>>> Whilst working on a different JNI fix I noticed that in most cases >>>> in jni.cpp we were using the following form of make_local: >>>> JNIHandles::make_local(env, obj); >>>> and what that form does is first extract the thread from the JNIEnv: >>>> JavaThread* thread = JavaThread::thread_from_jni_environment(env); >>>> return thread->active_handles()->allocate_handle(obj); >>>> but there is also another, faster, variant for when you already >>>> have the "thread": >>>> jobject JNIHandles::make_local(Thread* thread, oop obj) { >>>> ?? return thread->active_handles()->allocate_handle(obj); >>>> } >>>> When you look at the JNI_ENTRY wrapper (and related JVM_ENTRY, >>>> WB_ENTRY, UNSAFE_ENTRY etc) it has already extracted the thread >>>> from the JNIEnv: >>>> ???? JavaThread* thread=JavaThread::thread_from_jni_environment(env); >>>> and further defined: >>>> ???? Thread* THREAD = thread; >>>> so we always already have direct access to the "thread" available >>>> (or indirect via TRAPS), and in fact we can end up removing the >>>> make_local(JNIEnv* env, oop obj) variant altogether. >>>> Along the way I spotted some related issues with unnecessary use of >>>> Thread::current() when it is already available from TRAPS, and some >>>> other cases where we extracted the JNIEnv from a thread only to >>>> later extract the thread from the JNIEnv. >>>> Testing: tiers 1 - 3 >>>> Thanks, >>>> David >>>> ----- >> >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/classfile/javaClasses.cpp >> ? 439???? JNIEnv *env = thread->jni_environment(); >> >> Since env is no longer used on the next line, move this down to where >> it is used, at line 444. > > Fixed. > >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/classfile/verifier.cpp >> ? 299?? JNIEnv *env = thread->jni_environment(); >> >> env now seems to only be used at line 320.? Move this closer. > > Fixed. > >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/prims/jni.cpp >> ? 743???? result = JNIHandles::make_local(THREAD, result_handle()); >> >> jni_PopLocalFrame is now using a mix of "thread" and "THREAD", where >> previously it just used "thread". Maybe this change shouldn't be made? >> Or can the other uses be changed to THREAD for consistency? > > "thread" and "THREAD" are interchangeable for anything expecting a > "Thread*" (and somewhat surprisingly a number of API's that only work > for JavaThreads actually take a Thread*. :( ). I had choice between > trying to be file-wide consistent with the make_local calls, versus > local-code consistent, and used THREAD as it is available in both > JNI_ENTRY and via TRAPS. But I can certainly make a local change to > "thread" for local consistency. > >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/prims/jvm.cpp >> >> The calls to JvmtiExport::post_vm_object_alloc have to use "thread" >> instead of "THREAD", even though other places nearby are using >> "THREAD".? That inconsistency is kind of unfortunate, but doesn't seem >> easily avoidable. > > Everything that uses THREAD in a JVM_ENTRY method can be changed to > use "thread" instead. But I'm not sure it's a consistency worth > pursuing at least as part of these changes (there are likely similar > issues with most of the touched files). > > Thanks, > David > >> ------------------------------------------------------------------------------ >> >> From daniil.x.titov at oracle.com Mon Jul 20 19:00:39 2020 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Mon, 20 Jul 2020 12:00:39 -0700 Subject: RFR: 8216324: GetClassMethods is confused by the presence of default methods in super interfaces Message-ID: Please review change [1] that fixes GetClassMethods behavior in cases if a default method is present in a super interface. Currently for such cases the information GetClassMethods returns for the sub-interface or implementing class incorrectly includes inherited not default methods. The fix ( thanks to Alex for providing this patch) ensures that overpass methods are filtered out in the returns. The fix also applies a change in the existing test as David suggested in the comments to the issue [2]. Testing: Mach5 tier1-tier3 tests successfully passed. [1] http://cr.openjdk.java.net/~dtitov/8216324/webrev.01/ [2] https://bugs.openjdk.java.net/browse/JDK-8216324 Thank you, Daniil From alexey.menkov at oracle.com Mon Jul 20 23:57:36 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Mon, 20 Jul 2020 16:57:36 -0700 Subject: RFR: 8216324: GetClassMethods is confused by the presence of default methods in super interfaces In-Reply-To: References: Message-ID: <46dafc62-bbab-9287-b398-59ca8d99d03a@oracle.com> Looks good to me :) Thanks for handling this. --alex On 07/20/2020 12:00, Daniil Titov wrote: > Please review change [1] that fixes GetClassMethods behavior in cases if a default method is present in a super interface. Currently for such cases the information GetClassMethods returns for the sub-interface or implementing class incorrectly includes inherited not default methods. > > The fix ( thanks to Alex for providing this patch) ensures that overpass methods are filtered out in the returns. The fix also applies a change in the existing test as David suggested in the comments to the issue [2]. > > Testing: Mach5 tier1-tier3 tests successfully passed. > > [1] http://cr.openjdk.java.net/~dtitov/8216324/webrev.01/ > [2] https://bugs.openjdk.java.net/browse/JDK-8216324 > > Thank you, > Daniil > > From david.holmes at oracle.com Tue Jul 21 01:29:37 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 21 Jul 2020 11:29:37 +1000 Subject: [15?] RFR (S): 8249192: MonitorInfo stores raw oops across safepoints In-Reply-To: <528ade6c-059a-d9af-b0c7-74d7ac31b29a@oracle.com> References: <3f838c8a-6e70-cd94-1753-8abe269b7cdf@oracle.com> <528ade6c-059a-d9af-b0c7-74d7ac31b29a@oracle.com> Message-ID: <14512a57-8d4d-5199-ed16-743b5da8a10d@oracle.com> Hi Thomas, On 21/07/2020 12:49 am, Thomas Schatzl wrote: > Forwarding to hotspot-dev where it belongs after wrongly sending to > hotspot-gc-dev. This touches serviceability code as well so cc'ing for good measure. Thanks for taking this one on as it wasn't actually a GC issue! > > -------- Forwarded Message -------- > Subject: [15?] RFR (S): 8249192: MonitorInfo stores raw oops across > safepoints > Date: Mon, 20 Jul 2020 12:07:38 +0200 > From: Thomas Schatzl > To: hotspot-dev at openjdk.java.net > > Hi all, > > ? can I get some reviews to handle'ize some raw oops in the MonitorInfo > class? > > (Afaiu only) in LiveFrameStream::monitors_to_object_array() we try to > allocate an objArray with raw oops held in the MonitorInfo class that > are passed in a GrowableArray. This allocation can lead to a garbage > collection, with the usual random crashes. Right - seems so obvious now. Took me a while to convince myself no such similar problem was lurking in the JVM TI code. > This change changes the raw oops in MonitorInfo to Handles, My main concern here was whether the MonitorInfo objects are thread confined. For the StackWalker API we are always dealing with the current thread so that is fine. For JVM TI, in mainline, we may be executing code in the calling thread or the target thread; and in older releases it will be the VMThread at a safepoint. But it seems that the MonitorInfo's are confined to whichever thread that is, and so Handle usage is safe. > and adds a few HandleMarks along the way to make these handles go away asap. That, and the ResourceMark changes, were a bit hard to follow. Basically a HandleMark is now present in the scope of, or just above, the call to monitors(). The need for the additional ResourceMarks is far from clear though. In particular I wonder if the RM introduced in Deoptimization::revoke_from_deopt_handler interacts with the special DeoptResourceMark in its caller Deoptimization::fetch_unroll_info_helper? (I have no idea what a DeoptResourceMark is.) > > This issue has been introduced in JDK-8140450: Implement Stack-Walking > API in jdk9. > > The CR has been triaged as P3, but I would like to ask whether it might > be good to increase its priority to P2 and apply for inclusion in 15. My > arguments are as follows: > > - the original issue why I started looking at this were lots of > seemingly random crashes (5 or 6 were reported and the change > temporarily backed out for this reason) in tier8 with a g1 change that > changed young gen sizing. These crashes including that young gen sizing > change are all gone now with this bugfix. > I.e. this suggests that so far we seem to have not encountered this > issue more frequently due to pure luck wrt to generation sizing. > > - it affects all collectors (naturally). > > - there are quite a few user reported random crashes with IntelliJ and > variants, which due to the nature of IDEs tending to retrieve stack > traces fairly frequently would be more affected than usual. So I suspect > at least some of them to be caused by this issue, these are the only raw > oops I am aware of. > > My understanding of the cause and fix is fairly good, but I am no expert > in this area, so I would like to defer to you about this suggestion. The > change is imo important enough to be backported to 11 and 15 anyway, but > the question is about the risk/reward tradeoff wrt to bringing it to 15 > and not 15.0.1. I'd classify this as a P2 without doubt. As Dan noted there is no workaround as such. > CR: > https://bugs.openjdk.java.net/browse/JDK-8249192 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8249192/webrev/ src/hotspot/share/runtime/deoptimization.cpp The code in collect_monitors takes the monitor owner oop and Handelises it to add to its own GrowableArray of Handles. Is it worth exposing the MonitorInfo owner() in Handle form to avoid this unwrapping and re-wrapping? src/hotspot/share/runtime/vframe.hpp I agree with Coleen that the MonitorInfo constructor should not take a Thread* but should itself materialize and use Thread::current(). Thanks, David ----- > Testing: > tier1-5,tier8 (with some unrelated changes), 1800+ runs of the reproducer > > Thanks, > ? Thomas > From serguei.spitsyn at oracle.com Tue Jul 21 01:48:24 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 20 Jul 2020 18:48:24 -0700 Subject: RFR: 8216324: GetClassMethods is confused by the presence of default methods in super interfaces In-Reply-To: <46dafc62-bbab-9287-b398-59ca8d99d03a@oracle.com> References: <46dafc62-bbab-9287-b398-59ca8d99d03a@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Tue Jul 21 02:47:24 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 21 Jul 2020 12:47:24 +1000 Subject: RFR (M) 8249768: Move static oops and NullPointerException oops from Universe into OopStorage In-Reply-To: <9fb1860f-06cf-9347-27d7-16205c86c326@oracle.com> References: <9fb1860f-06cf-9347-27d7-16205c86c326@oracle.com> Message-ID: Hi Coleen, cc'ing serviceability due to SA changes. On 21/07/2020 6:53 am, coleen.phillimore at oracle.com wrote: > Summary: Move static oops into OopStorage and make NPE oops an objArrayOop. > > I've broken up moving oops in Universe to OopStorage into several > parts.? This change moves the global static oops.? These OopHandles are > not released. Overall looks good. But two things ... 1. naming ! // preallocated error objects (no backtrace) ! static OopHandle _out_of_memory_error; // array of preallocated error objects with backtrace ! static OopHandle _preallocated_out_of_memory_error_array; Both of these are pre-allocated arrays of OopHandles, differing only in whether the underlying OOME has a backtrace or not. I find the newly introduced _out_of_memory_error unclear in that regard. At a minimum could _out_of_memory_error become _out_of_memory_errors ? But ideally can we name these two arrays in a more distinguishable way? 2. SA You've simply deleted all the SA/vmstructs code that referenced the oops that are no longer present. Does the SA not care about these things and need access to them? (I don't know hence cc to serviceability folk.) Thanks, David ----- > This has been tested with tier1-3 and on also remote-build -b > linux-arm32,linux-ppc64le-debug,linux-s390x-debug,linux-x64-zero. > > open webrev at http://cr.openjdk.java.net/~coleenp/2020/8249768.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8249768 > > Thanks, > Coleen From serguei.spitsyn at oracle.com Tue Jul 21 05:54:32 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 20 Jul 2020 22:54:32 -0700 Subject: RFR: 8216324: GetClassMethods is confused by the presence of default methods in super interfaces In-Reply-To: References: <46dafc62-bbab-9287-b398-59ca8d99d03a@oracle.com> Message-ID: <1a37116e-5122-e6bb-a375-4f666f767aa5@oracle.com> An HTML attachment was scrubbed... URL: From coleen.phillimore at oracle.com Tue Jul 21 14:19:18 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 21 Jul 2020 10:19:18 -0400 Subject: RFR (M) 8249768: Move static oops and NullPointerException oops from Universe into OopStorage In-Reply-To: References: <9fb1860f-06cf-9347-27d7-16205c86c326@oracle.com> Message-ID: On 7/20/20 10:47 PM, David Holmes wrote: > Hi Coleen, > > cc'ing serviceability due to SA changes. > > On 21/07/2020 6:53 am, coleen.phillimore at oracle.com wrote: >> Summary: Move static oops into OopStorage and make NPE oops an >> objArrayOop. >> >> I've broken up moving oops in Universe to OopStorage into several >> parts.? This change moves the global static oops.? These OopHandles >> are not released. > > Overall looks good. But two things ... > > 1. naming > > !?? // preallocated error objects (no backtrace) > !?? static OopHandle??? _out_of_memory_error; > > ??? // array of preallocated error objects with backtrace > !?? static OopHandle???? _preallocated_out_of_memory_error_array; > > Both of these are pre-allocated arrays of OopHandles, differing only > in whether the underlying OOME has a backtrace or not. I find the > newly introduced _out_of_memory_error unclear in that regard. At a > minimum could _out_of_memory_error become _out_of_memory_errors ? But > ideally can we name these two arrays in a more distinguishable way? I put this code in functions next to each other because it was confusing.? The _out_of_memory_error array is really preallocated throwables, that are used to fill in the message for preallocated_out_of_memory_errors if there are enough of the latter left. I could rename _out_of_memory_error => _out_of_memory_error_throwables? ? > > 2. SA > > You've simply deleted all the SA/vmstructs code that referenced the > oops that are no longer present. Does the SA not care about these > things and need access to them? (I don't know hence cc to > serviceability folk.) Yes, the SA code was never used, so I deleted it.? SA might need in oop inspection to add walking Universe::vm_global() OopStorage area to find where oops come from if there's an error but there doesn't seem to be any other reason for SA to use these oops. Thanks, Coleen > > Thanks, > David > ----- > >> This has been tested with tier1-3 and on also remote-build -b >> linux-arm32,linux-ppc64le-debug,linux-s390x-debug,linux-x64-zero. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8249768.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8249768 >> >> Thanks, >> Coleen From coleen.phillimore at oracle.com Tue Jul 21 17:57:36 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 21 Jul 2020 13:57:36 -0400 Subject: RFR (M) 8249650: Optimize JNIHandle::make_local thread variable usage In-Reply-To: <6166e191-c954-70e5-5595-956a0c145d10@oracle.com> References: <8410d4a2-bbad-090f-55bf-88940f786781@oracle.com> <0590E210-6F23-4498-A51A-C3DAEF54B5AB@oracle.com> <6166e191-c954-70e5-5595-956a0c145d10@oracle.com> Message-ID: <2b52127c-8637-ed24-2a63-0b1372d4bff0@oracle.com> One note below: On 7/20/20 1:53 AM, David Holmes wrote: > Hi Kim, > > Thanks for looking at this. > > Updated webrev at: > > http://cr.openjdk.java.net/~dholmes/8249650/webrev.v2/ > > On 20/07/2020 3:22 pm, Kim Barrett wrote: >>> On Jul 20, 2020, at 12:16 AM, David Holmes >>> wrote: >>> >>> Subject line got truncated by accident ... >>> >>> On 20/07/2020 11:06 am, David Holmes wrote: >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8249650 >>>> webrev: http://cr.openjdk.java.net/~dholmes/8249650/webrev/ >>>> This is a simple cleanup that touches files across a number of VM >>>> areas - hence the cross-post. >>>> Whilst working on a different JNI fix I noticed that in most cases >>>> in jni.cpp we were using the following form of make_local: >>>> JNIHandles::make_local(env, obj); >>>> and what that form does is first extract the thread from the JNIEnv: >>>> JavaThread* thread = JavaThread::thread_from_jni_environment(env); >>>> return thread->active_handles()->allocate_handle(obj); >>>> but there is also another, faster, variant for when you already >>>> have the "thread": >>>> jobject JNIHandles::make_local(Thread* thread, oop obj) { >>>> ?? return thread->active_handles()->allocate_handle(obj); >>>> } >>>> When you look at the JNI_ENTRY wrapper (and related JVM_ENTRY, >>>> WB_ENTRY, UNSAFE_ENTRY etc) it has already extracted the thread >>>> from the JNIEnv: >>>> ???? JavaThread* thread=JavaThread::thread_from_jni_environment(env); >>>> and further defined: >>>> ???? Thread* THREAD = thread; >>>> so we always already have direct access to the "thread" available >>>> (or indirect via TRAPS), and in fact we can end up removing the >>>> make_local(JNIEnv* env, oop obj) variant altogether. >>>> Along the way I spotted some related issues with unnecessary use of >>>> Thread::current() when it is already available from TRAPS, and some >>>> other cases where we extracted the JNIEnv from a thread only to >>>> later extract the thread from the JNIEnv. >>>> Testing: tiers 1 - 3 >>>> Thanks, >>>> David >>>> ----- >> >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/classfile/javaClasses.cpp >> ? 439???? JNIEnv *env = thread->jni_environment(); >> >> Since env is no longer used on the next line, move this down to where >> it is used, at line 444. > > Fixed. > >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/classfile/verifier.cpp >> ? 299?? JNIEnv *env = thread->jni_environment(); >> >> env now seems to only be used at line 320.? Move this closer. > > Fixed. > >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/prims/jni.cpp >> ? 743???? result = JNIHandles::make_local(THREAD, result_handle()); >> >> jni_PopLocalFrame is now using a mix of "thread" and "THREAD", where >> previously it just used "thread". Maybe this change shouldn't be made? >> Or can the other uses be changed to THREAD for consistency? > > "thread" and "THREAD" are interchangeable for anything expecting a > "Thread*" (and somewhat surprisingly a number of API's that only work > for JavaThreads actually take a Thread*. :( ). I had choice between > trying to be file-wide consistent with the make_local calls, versus > local-code consistent, and used THREAD as it is available in both > JNI_ENTRY and via TRAPS. But I can certainly make a local change to > "thread" for local consistency. > >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/prims/jvm.cpp >> >> The calls to JvmtiExport::post_vm_object_alloc have to use "thread" >> instead of "THREAD", even though other places nearby are using >> "THREAD".? That inconsistency is kind of unfortunate, but doesn't seem >> easily avoidable. > > Everything that uses THREAD in a JVM_ENTRY method can be changed to > use "thread" instead. But I'm not sure it's a consistency worth > pursuing at least as part of these changes (there are likely similar > issues with most of the touched files). The thing I like about THREAD if it's available is that it's assumed to be *always* the current thread, so I have to wonder no further. Also, "thread" is generally the current thread too, but if you have a choice, my preference would be to use THREAD. I wouldn't want to see this changed. Thanks, Coleen > > Thanks, > David > >> ------------------------------------------------------------------------------ >> >> From thomas.schatzl at oracle.com Tue Jul 21 18:00:11 2020 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 21 Jul 2020 20:00:11 +0200 Subject: [15?] RFR (S): 8249192: MonitorInfo stores raw oops across safepoints In-Reply-To: <14512a57-8d4d-5199-ed16-743b5da8a10d@oracle.com> References: <3f838c8a-6e70-cd94-1753-8abe269b7cdf@oracle.com> <528ade6c-059a-d9af-b0c7-74d7ac31b29a@oracle.com> <14512a57-8d4d-5199-ed16-743b5da8a10d@oracle.com> Message-ID: Hi Coleen and David, thanks for your reviews. On 21.07.20 03:29, David Holmes wrote: > Hi Thomas, > > On 21/07/2020 12:49 am, Thomas Schatzl wrote: >> Forwarding to hotspot-dev where it belongs after wrongly sending to >> hotspot-gc-dev. > > This touches serviceability code as well so cc'ing for good measure. > > Thanks for taking this one on as it wasn't actually a GC issue! I have been looking into strange G1 crashes with changed young gen sizing in tier8, which looked very similar to the crashes in that StackWalker/LocalsAndOperands.java test (thanks to Dean Long making us aware of these crashes). I guessed correctly in hindsight, that the main difference between C2 and Graal would be GC timing, so the problem could be related. Also we have some P2s for 15 with the same stack trace as in this reproducer and the mentioned tier8 (Kitchensink, 24h dacapo) failures that never reproduced... given that the issue reproduced much quicker in LocalsAndOperands (and given its source code gives a pretty narrow area where to look) it seemed easier to start with this one... >> >> -------- Forwarded Message -------- >> Subject: [15?] RFR (S): 8249192: MonitorInfo stores raw oops across >> safepoints >> Date: Mon, 20 Jul 2020 12:07:38 +0200 >> From: Thomas Schatzl >> To: hotspot-dev at openjdk.java.net >> >> Hi all, >> >> ?? can I get some reviews to handle'ize some raw oops in the >> MonitorInfo class? >> >> (Afaiu only) in LiveFrameStream::monitors_to_object_array() we try to >> allocate an objArray with raw oops held in the MonitorInfo class that >> are passed in a GrowableArray. This allocation can lead to a garbage >> collection, with the usual random crashes. > > Right - seems so obvious now. > > Took me a while to convince myself no such similar problem was lurking > in the JVM TI code. > >> This change changes the raw oops in MonitorInfo to Handles, > > My main concern here was whether the MonitorInfo objects are thread > confined. For the StackWalker API we are always dealing with the current > thread so that is fine. For JVM TI, in mainline, we may be executing > code in the calling thread or the target thread; and in older releases > it will be the VMThread at a safepoint. But it seems that the > MonitorInfo's are confined to whichever thread that is, and so Handle > usage is safe. > >> and adds a few HandleMarks along the way to make these handles go away >> asap. > > That, and the ResourceMark changes, were a bit hard to follow. Basically > a HandleMark is now present in the scope of, or just above, the call to > monitors(). The need for the additional ResourceMarks is far from clear > though. In particular I wonder if the RM introduced in > Deoptimization::revoke_from_deopt_handler interacts with the special > DeoptResourceMark in its caller > Deoptimization::fetch_unroll_info_helper? (I have no idea what a > DeoptResourceMark is.) The DeoptResourceMark in this case seems to act just like a regular ResourceMark, using the thread's resource area. Other than acting like a ResourceMark, the DeoptResourceMark only seems to be an indicator used in some asserts to verify that no further deoptimization is running. Looking at the called BiasedLocking::revoke_own_lock(), it adds its own (regular) ResourceMark quite early (further indicating that normal ResourceMarks and DeoptResourceMarks should be "compatible"), meaning that only any resource object allocated in the resource area between the added ResourceMark in Deoptimization::revoke_from_deopt_handler() and the existing one in BiasedLocking::revoke_own_lock() would have a different lifetime than before. There is no resource object allocation in there, actually the only thing that happens is unpacking the contents of the passed handle. The objects_to_revoke array in Deoptimization::revoke_from_deopt_handler does not escape the method too. I was much more worried about the caching of a GrowableArray in Thread::cached_monitor_info going on during biased locking... > >> >> This issue has been introduced in JDK-8140450: Implement Stack-Walking >> API in jdk9. >> >> The CR has been triaged as P3, but I would like to ask whether it >> might be good to increase its priority to P2 and apply for inclusion >> in 15. My arguments are as follows: >> >> - the original issue why I started looking at this were lots of >> seemingly random crashes (5 or 6 were reported and the change >> temporarily backed out for this reason) in tier8 with a g1 change that >> changed young gen sizing. These crashes including that young gen >> sizing change are all gone now with this bugfix. >> I.e. this suggests that so far we seem to have not encountered this >> issue more frequently due to pure luck wrt to generation sizing. >> >> - it affects all collectors (naturally). >> >> - there are quite a few user reported random crashes with IntelliJ and >> variants, which due to the nature of IDEs tending to retrieve stack >> traces fairly frequently would be more affected than usual. So I >> suspect at least some of them to be caused by this issue, these are >> the only raw oops I am aware of. >> >> My understanding of the cause and fix is fairly good, but I am no >> expert in this area, so I would like to defer to you about this >> suggestion. The change is imo important enough to be backported to 11 >> and 15 anyway, but the question is about the risk/reward tradeoff wrt >> to bringing it to 15 and not 15.0.1. > > I'd classify this as a P2 without doubt. As Dan noted there is no > workaround as such. > >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8249192 >> Webrev: >> http://cr.openjdk.java.net/~tschatzl/8249192/webrev/ > > src/hotspot/share/runtime/deoptimization.cpp > > The code in collect_monitors takes the monitor owner oop and Handelises > it to add to its own GrowableArray of Handles. Is it worth exposing the > MonitorInfo owner() in Handle form to avoid this unwrapping and > re-wrapping? We talked a bit about this internally and came to the conclusion to initially provide a small fix for 15 and 16, and do that suggested refactoring in 16 only. > > src/hotspot/share/runtime/vframe.hpp > > I agree with Coleen that the MonitorInfo constructor should not take a > Thread* but should itself materialize and use Thread::current(). Fixed. New webrevs (jdk16): http://cr.openjdk.java.net/~tschatzl/8249192/webrev.0_to_1/ (diff) http://cr.openjdk.java.net/~tschatzl/8249192/webrev.1/ (full) jdk15: http://cr.openjdk.java.net/~tschatzl/8249192/webrev.jdk15.1/ The only difference is that JDK-8247729 in 16 changed a ResourceMark rm; in jdk15 (jvmtiEnvBase.cpp:1029) from ResourceMark rm(current_thread); in jdk16 (jvmtiEnvBase.cpp:1008) which gives a merge error now. See also https://hg.openjdk.java.net/jdk/jdk/rev/f8a9be0f9e1a#l2.82 . Started another tier1-5 run for jdk15; both versions passed 1.2k iterations of the LocalsAndOperands.java test again. Thanks, Thomas From coleen.phillimore at oracle.com Tue Jul 21 18:01:36 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 21 Jul 2020 14:01:36 -0400 Subject: RFR (M) 8249650: Optimize JNIHandle::make_local thread variable usage In-Reply-To: <6166e191-c954-70e5-5595-956a0c145d10@oracle.com> References: <8410d4a2-bbad-090f-55bf-88940f786781@oracle.com> <0590E210-6F23-4498-A51A-C3DAEF54B5AB@oracle.com> <6166e191-c954-70e5-5595-956a0c145d10@oracle.com> Message-ID: <82ac807a-1492-9ac0-570a-d08b1dc93e09@oracle.com> This looks like a nice cleanup. http://cr.openjdk.java.net/~dholmes/8249650/webrev.v2/src/hotspot/share/runtime/jniHandles.cpp.udiff.html I'm wondering why you took out the NULL return for make_local() without a thread argument?? Here you may call Thread::current() unnecessarily. jobject JNIHandles::make_local(oop obj) { - if (obj == NULL) { - return NULL; // ignore null handles - } else { - Thread* thread = Thread::current(); - assert(oopDesc::is_oop(obj), "not an oop"); - assert(!current_thread_in_native(), "must not be in native"); - return thread->active_handles()->allocate_handle(obj); - } + return make_local(Thread::current(), obj); } Beyond the scope of this fix, but it'd be cool to not have a version that doesn't take thread, since there may be many more callers that already have Thread::current(). Coleen On 7/20/20 1:53 AM, David Holmes wrote: > Hi Kim, > > Thanks for looking at this. > > Updated webrev at: > > http://cr.openjdk.java.net/~dholmes/8249650/webrev.v2/ > > On 20/07/2020 3:22 pm, Kim Barrett wrote: >>> On Jul 20, 2020, at 12:16 AM, David Holmes >>> wrote: >>> >>> Subject line got truncated by accident ... >>> >>> On 20/07/2020 11:06 am, David Holmes wrote: >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8249650 >>>> webrev: http://cr.openjdk.java.net/~dholmes/8249650/webrev/ >>>> This is a simple cleanup that touches files across a number of VM >>>> areas - hence the cross-post. >>>> Whilst working on a different JNI fix I noticed that in most cases >>>> in jni.cpp we were using the following form of make_local: >>>> JNIHandles::make_local(env, obj); >>>> and what that form does is first extract the thread from the JNIEnv: >>>> JavaThread* thread = JavaThread::thread_from_jni_environment(env); >>>> return thread->active_handles()->allocate_handle(obj); >>>> but there is also another, faster, variant for when you already >>>> have the "thread": >>>> jobject JNIHandles::make_local(Thread* thread, oop obj) { >>>> ?? return thread->active_handles()->allocate_handle(obj); >>>> } >>>> When you look at the JNI_ENTRY wrapper (and related JVM_ENTRY, >>>> WB_ENTRY, UNSAFE_ENTRY etc) it has already extracted the thread >>>> from the JNIEnv: >>>> ???? JavaThread* thread=JavaThread::thread_from_jni_environment(env); >>>> and further defined: >>>> ???? Thread* THREAD = thread; >>>> so we always already have direct access to the "thread" available >>>> (or indirect via TRAPS), and in fact we can end up removing the >>>> make_local(JNIEnv* env, oop obj) variant altogether. >>>> Along the way I spotted some related issues with unnecessary use of >>>> Thread::current() when it is already available from TRAPS, and some >>>> other cases where we extracted the JNIEnv from a thread only to >>>> later extract the thread from the JNIEnv. >>>> Testing: tiers 1 - 3 >>>> Thanks, >>>> David >>>> ----- >> >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/classfile/javaClasses.cpp >> ? 439???? JNIEnv *env = thread->jni_environment(); >> >> Since env is no longer used on the next line, move this down to where >> it is used, at line 444. > > Fixed. > >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/classfile/verifier.cpp >> ? 299?? JNIEnv *env = thread->jni_environment(); >> >> env now seems to only be used at line 320.? Move this closer. > > Fixed. > >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/prims/jni.cpp >> ? 743???? result = JNIHandles::make_local(THREAD, result_handle()); >> >> jni_PopLocalFrame is now using a mix of "thread" and "THREAD", where >> previously it just used "thread". Maybe this change shouldn't be made? >> Or can the other uses be changed to THREAD for consistency? > > "thread" and "THREAD" are interchangeable for anything expecting a > "Thread*" (and somewhat surprisingly a number of API's that only work > for JavaThreads actually take a Thread*. :( ). I had choice between > trying to be file-wide consistent with the make_local calls, versus > local-code consistent, and used THREAD as it is available in both > JNI_ENTRY and via TRAPS. But I can certainly make a local change to > "thread" for local consistency. > >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/prims/jvm.cpp >> >> The calls to JvmtiExport::post_vm_object_alloc have to use "thread" >> instead of "THREAD", even though other places nearby are using >> "THREAD".? That inconsistency is kind of unfortunate, but doesn't seem >> easily avoidable. > > Everything that uses THREAD in a JVM_ENTRY method can be changed to > use "thread" instead. But I'm not sure it's a consistency worth > pursuing at least as part of these changes (there are likely similar > issues with most of the touched files). > > Thanks, > David > >> ------------------------------------------------------------------------------ >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Tue Jul 21 19:25:31 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 21 Jul 2020 12:25:31 -0700 Subject: RFR (M) 8249650: Optimize JNIHandle::make_local thread variable usage In-Reply-To: <6166e191-c954-70e5-5595-956a0c145d10@oracle.com> References: <8410d4a2-bbad-090f-55bf-88940f786781@oracle.com> <0590E210-6F23-4498-A51A-C3DAEF54B5AB@oracle.com> <6166e191-c954-70e5-5595-956a0c145d10@oracle.com> Message-ID: <1256c311-76cf-2d59-2e12-c79516728d34@oracle.com> Hi David, The fix looks good to me. Thanks, Serguei On 7/19/20 22:53, David Holmes wrote: > Hi Kim, > > Thanks for looking at this. > > Updated webrev at: > > http://cr.openjdk.java.net/~dholmes/8249650/webrev.v2/ > > On 20/07/2020 3:22 pm, Kim Barrett wrote: >>> On Jul 20, 2020, at 12:16 AM, David Holmes >>> wrote: >>> >>> Subject line got truncated by accident ... >>> >>> On 20/07/2020 11:06 am, David Holmes wrote: >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8249650 >>>> webrev: http://cr.openjdk.java.net/~dholmes/8249650/webrev/ >>>> This is a simple cleanup that touches files across a number of VM >>>> areas - hence the cross-post. >>>> Whilst working on a different JNI fix I noticed that in most cases >>>> in jni.cpp we were using the following form of make_local: >>>> JNIHandles::make_local(env, obj); >>>> and what that form does is first extract the thread from the JNIEnv: >>>> JavaThread* thread = JavaThread::thread_from_jni_environment(env); >>>> return thread->active_handles()->allocate_handle(obj); >>>> but there is also another, faster, variant for when you already >>>> have the "thread": >>>> jobject JNIHandles::make_local(Thread* thread, oop obj) { >>>> ?? return thread->active_handles()->allocate_handle(obj); >>>> } >>>> When you look at the JNI_ENTRY wrapper (and related JVM_ENTRY, >>>> WB_ENTRY, UNSAFE_ENTRY etc) it has already extracted the thread >>>> from the JNIEnv: >>>> ???? JavaThread* thread=JavaThread::thread_from_jni_environment(env); >>>> and further defined: >>>> ???? Thread* THREAD = thread; >>>> so we always already have direct access to the "thread" available >>>> (or indirect via TRAPS), and in fact we can end up removing the >>>> make_local(JNIEnv* env, oop obj) variant altogether. >>>> Along the way I spotted some related issues with unnecessary use of >>>> Thread::current() when it is already available from TRAPS, and some >>>> other cases where we extracted the JNIEnv from a thread only to >>>> later extract the thread from the JNIEnv. >>>> Testing: tiers 1 - 3 >>>> Thanks, >>>> David >>>> ----- >> >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/classfile/javaClasses.cpp >> ? 439???? JNIEnv *env = thread->jni_environment(); >> >> Since env is no longer used on the next line, move this down to where >> it is used, at line 444. > > Fixed. > >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/classfile/verifier.cpp >> ? 299?? JNIEnv *env = thread->jni_environment(); >> >> env now seems to only be used at line 320.? Move this closer. > > Fixed. > >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/prims/jni.cpp >> ? 743???? result = JNIHandles::make_local(THREAD, result_handle()); >> >> jni_PopLocalFrame is now using a mix of "thread" and "THREAD", where >> previously it just used "thread". Maybe this change shouldn't be made? >> Or can the other uses be changed to THREAD for consistency? > > "thread" and "THREAD" are interchangeable for anything expecting a > "Thread*" (and somewhat surprisingly a number of API's that only work > for JavaThreads actually take a Thread*. :( ). I had choice between > trying to be file-wide consistent with the make_local calls, versus > local-code consistent, and used THREAD as it is available in both > JNI_ENTRY and via TRAPS. But I can certainly make a local change to > "thread" for local consistency. > >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/prims/jvm.cpp >> >> The calls to JvmtiExport::post_vm_object_alloc have to use "thread" >> instead of "THREAD", even though other places nearby are using >> "THREAD".? That inconsistency is kind of unfortunate, but doesn't seem >> easily avoidable. > > Everything that uses THREAD in a JVM_ENTRY method can be changed to > use "thread" instead. But I'm not sure it's a consistency worth > pursuing at least as part of these changes (there are likely similar > issues with most of the touched files). > > Thanks, > David > >> ------------------------------------------------------------------------------ >> >> From chris.plummer at oracle.com Tue Jul 21 20:05:20 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 21 Jul 2020 13:05:20 -0700 Subject: RFR(M): 8247515: OSX pc_to_symbol() lookup does not work with core files In-Reply-To: <1e9e4786-79f8-c4e0-efe7-f1a8f369d2c6@oracle.com> References: <092ef5a8-c973-e377-de91-b2002a1b8fe7@oracle.com> <38e1887e-a646-abc0-0b7e-53339af1d684@oracle.com> <3c31622b-831e-d46f-daa6-d641cbf755b1@oracle.com> <18f7a1d8-3802-c7da-c981-eac8cf0953eb@oracle.com> <1e9e4786-79f8-c4e0-efe7-f1a8f369d2c6@oracle.com> Message-ID: <4c09d200-057b-e878-733a-524860453288@oracle.com> An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Tue Jul 21 20:13:40 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 21 Jul 2020 13:13:40 -0700 Subject: RFR(S): 8248882: SA PMap and PStack support on OSX works with core files. Enable them. Message-ID: <511f7770-2ebf-f3ec-fcf9-221f9e65ed87@oracle.com> Hello, Please help review the following: https://bugs.openjdk.java.net/browse/JDK-8248882 http://cr.openjdk.java.net/~cjplummer/8248882/webrev.00/index.html The changes enable PMap and PStack on OSX when debugging a core file. Unfortunately there appears to be no exposed way to determine if you are debugging a core file, so I had to rely on clues (certain lists being empty). As part of these changes I'm adding core file testing to the two relevant tests. Also note that for ClhsdbPMap.java I had to tinker with the list of expected libraries in the output. For core files on Windows, the nio and net libraries were missing, presumably because nothing had triggered their loading yet, so I removed them from the expected output list, and add jli, which seems to always be there. Also a minor change in CLHSB.java. It was sending a message to System.err. It makes the test output easier to read if it goes to System.out. thanks, Chris From alexey.menkov at oracle.com Tue Jul 21 22:04:51 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Tue, 21 Jul 2020 15:04:51 -0700 Subject: RFR: JDK-8249550: jdb should use loopback address when not using remote agent Message-ID: <541a2f57-4b86-b453-7739-f1de35b52212@oracle.com> Hi all, please review the fix for https://bugs.openjdk.java.net/browse/JDK-8249550 webrev: http://cr.openjdk.java.net/~amenkov/jdk16/jdb_loopback/webrev/ some background: https://bugs.openjdk.java.net/browse/JDK-8041435 made default listening on loopback address. Later https://bugs.openjdk.java.net/browse/JDK-8184770 added handling of "*" address to listen on all addresses, but it didn't fixed "default" startListening() method (used by jdb through SunCommandLineLauncher). The method called startListening(String localaddress, int port) with localaddress == null, but this method for null localladdress starts listening on all addresses (i.e. handle null value as "*"). The fix changes it to startListening(String address) which handles null address the same way as JDI socket connector does (i.e. listens on loopback address only) --alex From david.holmes at oracle.com Wed Jul 22 00:42:36 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 22 Jul 2020 10:42:36 +1000 Subject: [15?] RFR (S): 8249192: MonitorInfo stores raw oops across safepoints In-Reply-To: References: <3f838c8a-6e70-cd94-1753-8abe269b7cdf@oracle.com> <528ade6c-059a-d9af-b0c7-74d7ac31b29a@oracle.com> <14512a57-8d4d-5199-ed16-743b5da8a10d@oracle.com> Message-ID: Hi Thomas, I've looked at the incremental update and I am happy with that. I also, prompted by you mentioning it, took a deeper look at the biased-locking code to ensure it also keeps the MonitorInfo's thread-confined, and to see whether the handshake versions could themselves be susceptible to interference from safepoints (which they can't as far as I can determine). And that all seems fine. As per offline discussions I know that there has been an alternate proposal for a completely localized fix in the stackwalker code that simply retrieves the list of monitors, uses the length to create the array, then re-retrieves the list of monitors to populate the array (the length of which can't change as we are dealing with the current thread). My only concern with that approach is the performance impact if we have deep stacks with lots of monitors. There is a microbenchmark for StackWalker in the repo: open/test/micro/org/openjdk/bench/java/lang/StackWalkBench.java but it doesn't test anything to do with monitor usage. Thanks, David ----- On 22/07/2020 4:00 am, Thomas Schatzl wrote: > Hi Coleen and David, > > ? thanks for your reviews. > > On 21.07.20 03:29, David Holmes wrote: >> Hi Thomas, >> >> On 21/07/2020 12:49 am, Thomas Schatzl wrote: >>> Forwarding to hotspot-dev where it belongs after wrongly sending to >>> hotspot-gc-dev. >> >> This touches serviceability code as well so cc'ing for good measure. >> >> Thanks for taking this one on as it wasn't actually a GC issue! > > I have been looking into strange G1 crashes with changed young gen > sizing in tier8, which looked very similar to the crashes in that > StackWalker/LocalsAndOperands.java test (thanks to Dean Long making us > aware of these crashes). > I guessed correctly in hindsight, that the main difference between C2 > and Graal would be GC timing, so the problem could be related. Also we > have some P2s for 15 with the same stack trace as in this reproducer and > the mentioned tier8 (Kitchensink, 24h dacapo) failures that never > reproduced... given that the issue reproduced much quicker in > LocalsAndOperands (and given its source code gives a pretty narrow area > where to look) it seemed easier to start with this one... > >>> >>> -------- Forwarded Message -------- >>> Subject: [15?] RFR (S): 8249192: MonitorInfo stores raw oops across >>> safepoints >>> Date: Mon, 20 Jul 2020 12:07:38 +0200 >>> From: Thomas Schatzl >>> To: hotspot-dev at openjdk.java.net >>> >>> Hi all, >>> >>> ?? can I get some reviews to handle'ize some raw oops in the >>> MonitorInfo class? >>> >>> (Afaiu only) in LiveFrameStream::monitors_to_object_array() we try to >>> allocate an objArray with raw oops held in the MonitorInfo class that >>> are passed in a GrowableArray. This allocation can lead to a garbage >>> collection, with the usual random crashes. >> >> Right - seems so obvious now. >> >> Took me a while to convince myself no such similar problem was lurking >> in the JVM TI code. >> >>> This change changes the raw oops in MonitorInfo to Handles, >> >> My main concern here was whether the MonitorInfo objects are thread >> confined. For the StackWalker API we are always dealing with the >> current thread so that is fine. For JVM TI, in mainline, we may be >> executing code in the calling thread or the target thread; and in >> older releases it will be the VMThread at a safepoint. But it seems >> that the MonitorInfo's are confined to whichever thread that is, and >> so Handle usage is safe. > > >> >>> and adds a few HandleMarks along the way to make these handles go >>> away asap. >> >> That, and the ResourceMark changes, were a bit hard to follow. >> Basically a HandleMark is now present in the scope of, or just above, >> the call to monitors(). The need for the additional ResourceMarks is >> far from clear though. In particular I wonder if the RM introduced in >> Deoptimization::revoke_from_deopt_handler interacts with the special >> DeoptResourceMark in its caller >> Deoptimization::fetch_unroll_info_helper? (I have no idea what a >> DeoptResourceMark is.) > > The DeoptResourceMark in this case seems to act just like a regular > ResourceMark, using the thread's resource area. > > Other than acting like a ResourceMark, the DeoptResourceMark only seems > to be an indicator used in some asserts to verify that no further > deoptimization is running. > > Looking at the called BiasedLocking::revoke_own_lock(), it adds its own > (regular) ResourceMark quite early (further indicating that normal > ResourceMarks and DeoptResourceMarks should be "compatible"), meaning > that only any resource object allocated in the resource area between the > added ResourceMark in Deoptimization::revoke_from_deopt_handler() and > the existing one in BiasedLocking::revoke_own_lock() would have a > different lifetime than before. There is no resource object allocation > in there, actually the only thing that happens is unpacking the contents > of the passed handle. > > The objects_to_revoke array in Deoptimization::revoke_from_deopt_handler > does not escape the method too. > > I was much more worried about the caching of a > GrowableArray in Thread::cached_monitor_info going on > during biased locking... > >> >>> >>> This issue has been introduced in JDK-8140450: Implement >>> Stack-Walking API in jdk9. >>> >>> The CR has been triaged as P3, but I would like to ask whether it >>> might be good to increase its priority to P2 and apply for inclusion >>> in 15. My arguments are as follows: >>> >>> - the original issue why I started looking at this were lots of >>> seemingly random crashes (5 or 6 were reported and the change >>> temporarily backed out for this reason) in tier8 with a g1 change >>> that changed young gen sizing. These crashes including that young gen >>> sizing change are all gone now with this bugfix. >>> I.e. this suggests that so far we seem to have not encountered this >>> issue more frequently due to pure luck wrt to generation sizing. >>> >>> - it affects all collectors (naturally). >>> >>> - there are quite a few user reported random crashes with IntelliJ >>> and variants, which due to the nature of IDEs tending to retrieve >>> stack traces fairly frequently would be more affected than usual. So >>> I suspect at least some of them to be caused by this issue, these are >>> the only raw oops I am aware of. >>> >>> My understanding of the cause and fix is fairly good, but I am no >>> expert in this area, so I would like to defer to you about this >>> suggestion. The change is imo important enough to be backported to 11 >>> and 15 anyway, but the question is about the risk/reward tradeoff wrt >>> to bringing it to 15 and not 15.0.1. >> >> I'd classify this as a P2 without doubt. As Dan noted there is no >> workaround as such. >> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8249192 >>> Webrev: >>> http://cr.openjdk.java.net/~tschatzl/8249192/webrev/ >> >> src/hotspot/share/runtime/deoptimization.cpp >> >> The code in collect_monitors takes the monitor owner oop and >> Handelises it to add to its own GrowableArray of Handles. Is it worth >> exposing the MonitorInfo owner() in Handle form to avoid this >> unwrapping and re-wrapping? > > We talked a bit about this internally and came to the conclusion to > initially provide a small fix for 15 and 16, and do that suggested > refactoring in 16 only. > >> >> src/hotspot/share/runtime/vframe.hpp >> >> I agree with Coleen that the MonitorInfo constructor should not take a >> Thread* but should itself materialize and use Thread::current(). > > Fixed. > > New webrevs (jdk16): > > http://cr.openjdk.java.net/~tschatzl/8249192/webrev.0_to_1/ (diff) > http://cr.openjdk.java.net/~tschatzl/8249192/webrev.1/ (full) > > jdk15: > > http://cr.openjdk.java.net/~tschatzl/8249192/webrev.jdk15.1/ > > The only difference is that JDK-8247729 in 16 changed a > > ResourceMark rm; > > in jdk15 (jvmtiEnvBase.cpp:1029) from > > ResourceMark rm(current_thread); > > in jdk16 (jvmtiEnvBase.cpp:1008) > > which gives a merge error now. See also > https://hg.openjdk.java.net/jdk/jdk/rev/f8a9be0f9e1a#l2.82 . > > Started another tier1-5 run for jdk15; both versions passed 1.2k > iterations of the LocalsAndOperands.java test again. > > Thanks, > ? Thomas From serguei.spitsyn at oracle.com Wed Jul 22 02:13:51 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 21 Jul 2020 19:13:51 -0700 Subject: [15?] RFR (S): 8249192: MonitorInfo stores raw oops across safepoints In-Reply-To: References: <3f838c8a-6e70-cd94-1753-8abe269b7cdf@oracle.com> <528ade6c-059a-d9af-b0c7-74d7ac31b29a@oracle.com> <14512a57-8d4d-5199-ed16-743b5da8a10d@oracle.com> Message-ID: <893f862f-093a-5168-e554-1f4ffe129aea@oracle.com> Hi Thomas, The fix looks okay to me. The 15 fix is identical to 16. This file finally was not changed: src/hotspot/share/runtime/vframe_hp.cpp . Several files need a copyright comment update. What tests do you run? We need at least tier3 to make sure there are no regressions in the JVMTI and JDI tests. Thanks, Serguei On 7/21/20 11:00, Thomas Schatzl wrote: > Hi Coleen and David, > > ? thanks for your reviews. > > On 21.07.20 03:29, David Holmes wrote: >> Hi Thomas, >> >> On 21/07/2020 12:49 am, Thomas Schatzl wrote: >>> Forwarding to hotspot-dev where it belongs after wrongly sending to >>> hotspot-gc-dev. >> >> This touches serviceability code as well so cc'ing for good measure. >> >> Thanks for taking this one on as it wasn't actually a GC issue! > > I have been looking into strange G1 crashes with changed young gen > sizing in tier8, which looked very similar to the crashes in that > StackWalker/LocalsAndOperands.java test (thanks to Dean Long making us > aware of these crashes). > I guessed correctly in hindsight, that the main difference between C2 > and Graal would be GC timing, so the problem could be related. Also we > have some P2s for 15 with the same stack trace as in this reproducer > and the mentioned tier8 (Kitchensink, 24h dacapo) failures that never > reproduced... given that the issue reproduced much quicker in > LocalsAndOperands (and given its source code gives a pretty narrow > area where to look) it seemed easier to start with this one... > >>> >>> -------- Forwarded Message -------- >>> Subject: [15?] RFR (S): 8249192: MonitorInfo stores raw oops across >>> safepoints >>> Date: Mon, 20 Jul 2020 12:07:38 +0200 >>> From: Thomas Schatzl >>> To: hotspot-dev at openjdk.java.net >>> >>> Hi all, >>> >>> ?? can I get some reviews to handle'ize some raw oops in the >>> MonitorInfo class? >>> >>> (Afaiu only) in LiveFrameStream::monitors_to_object_array() we try >>> to allocate an objArray with raw oops held in the MonitorInfo class >>> that are passed in a GrowableArray. This allocation can lead to a >>> garbage collection, with the usual random crashes. >> >> Right - seems so obvious now. >> >> Took me a while to convince myself no such similar problem was >> lurking in the JVM TI code. >> >>> This change changes the raw oops in MonitorInfo to Handles, >> >> My main concern here was whether the MonitorInfo objects are thread >> confined. For the StackWalker API we are always dealing with the >> current thread so that is fine. For JVM TI, in mainline, we may be >> executing code in the calling thread or the target thread; and in >> older releases it will be the VMThread at a safepoint. But it seems >> that the MonitorInfo's are confined to whichever thread that is, and >> so Handle usage is safe. > > >> >>> and adds a few HandleMarks along the way to make these handles go >>> away asap. >> >> That, and the ResourceMark changes, were a bit hard to follow. >> Basically a HandleMark is now present in the scope of, or just above, >> the call to monitors(). The need for the additional ResourceMarks is >> far from clear though. In particular I wonder if the RM introduced in >> Deoptimization::revoke_from_deopt_handler interacts with the special >> DeoptResourceMark in its caller >> Deoptimization::fetch_unroll_info_helper? (I have no idea what a >> DeoptResourceMark is.) > > The DeoptResourceMark in this case seems to act just like a regular > ResourceMark, using the thread's resource area. > > Other than acting like a ResourceMark, the DeoptResourceMark only > seems to be an indicator used in some asserts to verify that no > further deoptimization is running. > > Looking at the called BiasedLocking::revoke_own_lock(), it adds its > own (regular) ResourceMark quite early (further indicating that normal > ResourceMarks and DeoptResourceMarks should be "compatible"), meaning > that only any resource object allocated in the resource area between > the added ResourceMark in Deoptimization::revoke_from_deopt_handler() > and the existing one in BiasedLocking::revoke_own_lock() would have a > different lifetime than before. There is no resource object allocation > in there, actually the only thing that happens is unpacking the > contents of the passed handle. > > The objects_to_revoke array in > Deoptimization::revoke_from_deopt_handler does not escape the method too. > > I was much more worried about the caching of a > GrowableArray in Thread::cached_monitor_info going on > during biased locking... > >> >>> >>> This issue has been introduced in JDK-8140450: Implement >>> Stack-Walking API in jdk9. >>> >>> The CR has been triaged as P3, but I would like to ask whether it >>> might be good to increase its priority to P2 and apply for inclusion >>> in 15. My arguments are as follows: >>> >>> - the original issue why I started looking at this were lots of >>> seemingly random crashes (5 or 6 were reported and the change >>> temporarily backed out for this reason) in tier8 with a g1 change >>> that changed young gen sizing. These crashes including that young >>> gen sizing change are all gone now with this bugfix. >>> I.e. this suggests that so far we seem to have not encountered this >>> issue more frequently due to pure luck wrt to generation sizing. >>> >>> - it affects all collectors (naturally). >>> >>> - there are quite a few user reported random crashes with IntelliJ >>> and variants, which due to the nature of IDEs tending to retrieve >>> stack traces fairly frequently would be more affected than usual. So >>> I suspect at least some of them to be caused by this issue, these >>> are the only raw oops I am aware of. >>> >>> My understanding of the cause and fix is fairly good, but I am no >>> expert in this area, so I would like to defer to you about this >>> suggestion. The change is imo important enough to be backported to >>> 11 and 15 anyway, but the question is about the risk/reward tradeoff >>> wrt to bringing it to 15 and not 15.0.1. >> >> I'd classify this as a P2 without doubt. As Dan noted there is no >> workaround as such. >> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8249192 >>> Webrev: >>> http://cr.openjdk.java.net/~tschatzl/8249192/webrev/ >> >> src/hotspot/share/runtime/deoptimization.cpp >> >> The code in collect_monitors takes the monitor owner oop and >> Handelises it to add to its own GrowableArray of Handles. Is it worth >> exposing the MonitorInfo owner() in Handle form to avoid this >> unwrapping and re-wrapping? > > We talked a bit about this internally and came to the conclusion to > initially provide a small fix for 15 and 16, and do that suggested > refactoring in 16 only. > >> >> src/hotspot/share/runtime/vframe.hpp >> >> I agree with Coleen that the MonitorInfo constructor should not take >> a Thread* but should itself materialize and use Thread::current(). > > Fixed. > > New webrevs (jdk16): > > http://cr.openjdk.java.net/~tschatzl/8249192/webrev.0_to_1/ (diff) > http://cr.openjdk.java.net/~tschatzl/8249192/webrev.1/ (full) > > jdk15: > > http://cr.openjdk.java.net/~tschatzl/8249192/webrev.jdk15.1/ > > The only difference is that JDK-8247729 in 16 changed a > > ResourceMark rm; > > in jdk15 (jvmtiEnvBase.cpp:1029) from > > ResourceMark rm(current_thread); > > in jdk16 (jvmtiEnvBase.cpp:1008) > > which gives a merge error now. See also > https://hg.openjdk.java.net/jdk/jdk/rev/f8a9be0f9e1a#l2.82 . > > Started another tier1-5 run for jdk15; both versions passed 1.2k > iterations of the LocalsAndOperands.java test again. > > Thanks, > ? Thomas From david.holmes at oracle.com Wed Jul 22 02:34:02 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 22 Jul 2020 12:34:02 +1000 Subject: RFR (M) 8249650: Optimize JNIHandle::make_local thread variable usage In-Reply-To: <1256c311-76cf-2d59-2e12-c79516728d34@oracle.com> References: <8410d4a2-bbad-090f-55bf-88940f786781@oracle.com> <0590E210-6F23-4498-A51A-C3DAEF54B5AB@oracle.com> <6166e191-c954-70e5-5595-956a0c145d10@oracle.com> <1256c311-76cf-2d59-2e12-c79516728d34@oracle.com> Message-ID: <63ff96e0-bcba-5041-0844-fb55b4fbfc1f@oracle.com> Thanks Serguei! David On 22/07/2020 5:25 am, serguei.spitsyn at oracle.com wrote: > Hi David, > > The fix looks good to me. > > Thanks, > Serguei > > > > On 7/19/20 22:53, David Holmes wrote: >> Hi Kim, >> >> Thanks for looking at this. >> >> Updated webrev at: >> >> http://cr.openjdk.java.net/~dholmes/8249650/webrev.v2/ >> >> On 20/07/2020 3:22 pm, Kim Barrett wrote: >>>> On Jul 20, 2020, at 12:16 AM, David Holmes >>>> wrote: >>>> >>>> Subject line got truncated by accident ... >>>> >>>> On 20/07/2020 11:06 am, David Holmes wrote: >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8249650 >>>>> webrev: http://cr.openjdk.java.net/~dholmes/8249650/webrev/ >>>>> This is a simple cleanup that touches files across a number of VM >>>>> areas - hence the cross-post. >>>>> Whilst working on a different JNI fix I noticed that in most cases >>>>> in jni.cpp we were using the following form of make_local: >>>>> JNIHandles::make_local(env, obj); >>>>> and what that form does is first extract the thread from the JNIEnv: >>>>> JavaThread* thread = JavaThread::thread_from_jni_environment(env); >>>>> return thread->active_handles()->allocate_handle(obj); >>>>> but there is also another, faster, variant for when you already >>>>> have the "thread": >>>>> jobject JNIHandles::make_local(Thread* thread, oop obj) { >>>>> ?? return thread->active_handles()->allocate_handle(obj); >>>>> } >>>>> When you look at the JNI_ENTRY wrapper (and related JVM_ENTRY, >>>>> WB_ENTRY, UNSAFE_ENTRY etc) it has already extracted the thread >>>>> from the JNIEnv: >>>>> ???? JavaThread* thread=JavaThread::thread_from_jni_environment(env); >>>>> and further defined: >>>>> ???? Thread* THREAD = thread; >>>>> so we always already have direct access to the "thread" available >>>>> (or indirect via TRAPS), and in fact we can end up removing the >>>>> make_local(JNIEnv* env, oop obj) variant altogether. >>>>> Along the way I spotted some related issues with unnecessary use of >>>>> Thread::current() when it is already available from TRAPS, and some >>>>> other cases where we extracted the JNIEnv from a thread only to >>>>> later extract the thread from the JNIEnv. >>>>> Testing: tiers 1 - 3 >>>>> Thanks, >>>>> David >>>>> ----- >>> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/classfile/javaClasses.cpp >>> ? 439???? JNIEnv *env = thread->jni_environment(); >>> >>> Since env is no longer used on the next line, move this down to where >>> it is used, at line 444. >> >> Fixed. >> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/classfile/verifier.cpp >>> ? 299?? JNIEnv *env = thread->jni_environment(); >>> >>> env now seems to only be used at line 320.? Move this closer. >> >> Fixed. >> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/prims/jni.cpp >>> ? 743???? result = JNIHandles::make_local(THREAD, result_handle()); >>> >>> jni_PopLocalFrame is now using a mix of "thread" and "THREAD", where >>> previously it just used "thread". Maybe this change shouldn't be made? >>> Or can the other uses be changed to THREAD for consistency? >> >> "thread" and "THREAD" are interchangeable for anything expecting a >> "Thread*" (and somewhat surprisingly a number of API's that only work >> for JavaThreads actually take a Thread*. :( ). I had choice between >> trying to be file-wide consistent with the make_local calls, versus >> local-code consistent, and used THREAD as it is available in both >> JNI_ENTRY and via TRAPS. But I can certainly make a local change to >> "thread" for local consistency. >> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/prims/jvm.cpp >>> >>> The calls to JvmtiExport::post_vm_object_alloc have to use "thread" >>> instead of "THREAD", even though other places nearby are using >>> "THREAD".? That inconsistency is kind of unfortunate, but doesn't seem >>> easily avoidable. >> >> Everything that uses THREAD in a JVM_ENTRY method can be changed to >> use "thread" instead. But I'm not sure it's a consistency worth >> pursuing at least as part of these changes (there are likely similar >> issues with most of the touched files). >> >> Thanks, >> David >> >>> ------------------------------------------------------------------------------ >>> >>> > From david.holmes at oracle.com Wed Jul 22 02:46:26 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 22 Jul 2020 12:46:26 +1000 Subject: RFR (M) 8249650: Optimize JNIHandle::make_local thread variable usage In-Reply-To: <328fb322-5b14-968b-7b13-4b449a8d98fd@oracle.com> References: <8410d4a2-bbad-090f-55bf-88940f786781@oracle.com> <0590E210-6F23-4498-A51A-C3DAEF54B5AB@oracle.com> <6166e191-c954-70e5-5595-956a0c145d10@oracle.com> <328fb322-5b14-968b-7b13-4b449a8d98fd@oracle.com> Message-ID: <4d763c6f-96e1-5c9b-8739-a441ee3b4b31@oracle.com> Hi Dan, On 21/07/2020 3:07 am, Daniel D. Daugherty wrote: > On 7/20/20 1:53 AM, David Holmes wrote: >> Hi Kim, >> >> Thanks for looking at this. >> >> Updated webrev at: >> >> http://cr.openjdk.java.net/~dholmes/8249650/webrev.v2/ > > I like this cleanup very much! Thanks for looking at it. > > src/hotspot/share/classfile/javaClasses.cpp > ??? No comments. > > src/hotspot/share/classfile/verifier.cpp > ??? L298: ? JavaThread* thread = (JavaThread*)THREAD; > ??? L307: ? ResourceMark rm(THREAD); > ??????? Since we've gone to the trouble of creating the 'thread' variable, > ??????? I would prefer it to be used instead of THREAD where possible. Okay I made this change as we already use "thread" throughout that method. > src/hotspot/share/jvmci/jvmciCompilerToVM.cpp > ??? L1021: ? HandleMark hm; > ??????? Can this be 'hm(THREAD)'? (Not your problem, but while you're > ??????? in that file?) It probably could but there are around 8 such uses and I don't want to expand this change any further than necessary for the current issue. I filed a general RFE for things that should take advantage of having a current thread reference already (that will encompass Coleen's make_local(obj) change as well). https://bugs.openjdk.java.net/browse/JDK-8249837 > src/hotspot/share/prims/jni.cpp > ??? No comments. > > src/hotspot/share/prims/jvm.cpp > ??? L140: ? ResourceMark rm; > ??????? Can this be 'rm(THREAD)'? (Not your problem, but while you're > ??????? in that file?) > > ??? L611: ? Handle stackStream_h(THREAD, > JNIHandles::resolve_non_null(stackStream)); > ??? L617: ? objArrayHandle frames_array_h(THREAD, fa); > ??? L626: ? return JNIHandles::make_local(THREAD, result); > ??????? Since we've gone to the trouble of creating the 'jt' variable, > ??????? I would prefer it to be used instead of THREAD where possible. > > ??? L767: ? vframeStream vfst(thread); > ??? L788???????? return (jclass) JNIHandles::make_local(THREAD, > m->method_holder()->java_mirror()); > ??????? Can we use 'thread' on L788? (preferred) > ??????? Can we use 'THREAD' on L767? (less preferred) > > ??? L949: ? ResourceMark rm(THREAD); > ??? L951: ? Handle class_loader (THREAD, JNIHandles::resolve(loader)); > ??? L955: ?????????????????????????? THREAD); > ??? L957: ? Handle protection_domain (THREAD, JNIHandles::resolve(pd)); > ??? L968: ? return (jclass) JNIHandles::make_local(THREAD, > k->java_mirror()); > ??????? Since we've gone to the trouble of creating the 'jt' variable, > ??????? I would prefer it to be used instead of THREAD where possible. As per our slack chat, and the fact you are okay with things as-is, I will forego a more general "consistency" pass as it is unclear what is best here. As Coleen notes THREAD is generally understood to always be the current thread, while thread/jthread/jt could be any old thread in general. Also THREAD usage can highlight a Thread* API, while "thread" has to be used for JavaThread* API - but obviously that needs to be carefully and consistently applied to be useful. :) > ??? L986: ? JavaThread* jt = (JavaThread*) THREAD; > ??????? This 'jt' is unused and can be deleted (Not your problem, but > while you're > ??????? in that file?) Fixed (and another case elsewhere). > ??? L1154: ? while (*p != '\0') { > ??? L1155: ????? if (*p == '.') { > ??? L1156: ????????? *p = '/'; > ??? L1157: ????? } > ??? L1158: ????? p++; > ??????? Nit - the indents are wrong on L1155-58. (Not your problem, but > while you're > ??????? in that file?) Fixed > ??? L1389: ? ResourceMark rm(THREAD); > ??? L1446: ??? return JNIHandles::make_local(THREAD, result); > ??? L1460: ? return JNIHandles::make_local(THREAD, result); > ??????? Can we use 'thread' on L1389? (preferred) And then the line you > ??????? touched could also be 'thread' and we'll be consistent in this > ??????? function... Left as-is. > ??? L3287: ? oop jthread = thread->threadObj(); > ??? L3288: ? assert (thread != NULL, "no current thread!"); > ??????? I think the assert is wrong. It should be: > > ??????????? assert(jthread != NULL, "no current thread!"); > > ??????? If 'thread == NULL', then we would have crashed at L3287. > ??????? Also notice that I deleted the extra ' ' before '('. (Not > ??????? your problem, but while you're in that file?) Fixed. I was initially concerned about bootstrapping but it is fine - we ensure we set threadObj() before executing any Java code. > ??? L3289: ? return JNIHandles::make_local(THREAD, jthread); > ??????? Can you use 'thread' instead of 'THREAD' here for consistency? > > ??? L3681: ??? method_handle = Handle(THREAD, > JNIHandles::resolve(method)); > ??? L3682: ??? Handle receiver(THREAD, JNIHandles::resolve(obj)); > ??? L3683: ??? objArrayHandle args(THREAD, > objArrayOop(JNIHandles::resolve(args0))); > ??? L3685: ??? jobject res = JNIHandles::make_local(THREAD, result); > ??????? Can you use 'thread' instead of 'THREAD' here for consistency? > > ??? L3705: ? objArrayHandle args(THREAD, > objArrayOop(JNIHandles::resolve(args0))); > ??? L3707?? jobject res = JNIHandles::make_local(THREAD, result); > ??????? Can you use 'thread' instead of 'THREAD' here for consistency? Left as-is. > src/hotspot/share/prims/methodHandles.cpp > ??? No comments. > > src/hotspot/share/prims/methodHandles.hpp > ??? No comments. > > src/hotspot/share/prims/unsafe.cpp > ??? No comments. > > src/hotspot/share/prims/whitebox.cpp > ??? No comments. > > src/hotspot/share/runtime/jniHandles.cpp > ??? No comments. > > src/hotspot/share/runtime/jniHandles.hpp > ??? No comments. > > src/hotspot/share/services/management.cpp > ??? No comments. > > > None of my comments above are "must do". If you choose to make the > changes, a new webrev isn't required, but would be useful for a > sanity check. In addition to the tweak above I found a bunch of make_locasl(obj) usages in jvm.cpp and jni.cpp thanks to Coleen, which I have also fixed. Updated webrev: http://cr.openjdk.java.net/~dholmes/8249650/webrev.v3/ If this passes tier 1-3 re-testing then I plan to push. Thanks, David ----- > Thumbs up. > > Dan > > >> >> On 20/07/2020 3:22 pm, Kim Barrett wrote: >>>> On Jul 20, 2020, at 12:16 AM, David Holmes >>>> wrote: >>>> >>>> Subject line got truncated by accident ... >>>> >>>> On 20/07/2020 11:06 am, David Holmes wrote: >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8249650 >>>>> webrev: http://cr.openjdk.java.net/~dholmes/8249650/webrev/ >>>>> This is a simple cleanup that touches files across a number of VM >>>>> areas - hence the cross-post. >>>>> Whilst working on a different JNI fix I noticed that in most cases >>>>> in jni.cpp we were using the following form of make_local: >>>>> JNIHandles::make_local(env, obj); >>>>> and what that form does is first extract the thread from the JNIEnv: >>>>> JavaThread* thread = JavaThread::thread_from_jni_environment(env); >>>>> return thread->active_handles()->allocate_handle(obj); >>>>> but there is also another, faster, variant for when you already >>>>> have the "thread": >>>>> jobject JNIHandles::make_local(Thread* thread, oop obj) { >>>>> ?? return thread->active_handles()->allocate_handle(obj); >>>>> } >>>>> When you look at the JNI_ENTRY wrapper (and related JVM_ENTRY, >>>>> WB_ENTRY, UNSAFE_ENTRY etc) it has already extracted the thread >>>>> from the JNIEnv: >>>>> ???? JavaThread* thread=JavaThread::thread_from_jni_environment(env); >>>>> and further defined: >>>>> ???? Thread* THREAD = thread; >>>>> so we always already have direct access to the "thread" available >>>>> (or indirect via TRAPS), and in fact we can end up removing the >>>>> make_local(JNIEnv* env, oop obj) variant altogether. >>>>> Along the way I spotted some related issues with unnecessary use of >>>>> Thread::current() when it is already available from TRAPS, and some >>>>> other cases where we extracted the JNIEnv from a thread only to >>>>> later extract the thread from the JNIEnv. >>>>> Testing: tiers 1 - 3 >>>>> Thanks, >>>>> David >>>>> ----- >>> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/classfile/javaClasses.cpp >>> ? 439???? JNIEnv *env = thread->jni_environment(); >>> >>> Since env is no longer used on the next line, move this down to where >>> it is used, at line 444. >> >> Fixed. >> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/classfile/verifier.cpp >>> ? 299?? JNIEnv *env = thread->jni_environment(); >>> >>> env now seems to only be used at line 320.? Move this closer. >> >> Fixed. >> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/prims/jni.cpp >>> ? 743???? result = JNIHandles::make_local(THREAD, result_handle()); >>> >>> jni_PopLocalFrame is now using a mix of "thread" and "THREAD", where >>> previously it just used "thread". Maybe this change shouldn't be made? >>> Or can the other uses be changed to THREAD for consistency? >> >> "thread" and "THREAD" are interchangeable for anything expecting a >> "Thread*" (and somewhat surprisingly a number of API's that only work >> for JavaThreads actually take a Thread*. :( ). I had choice between >> trying to be file-wide consistent with the make_local calls, versus >> local-code consistent, and used THREAD as it is available in both >> JNI_ENTRY and via TRAPS. But I can certainly make a local change to >> "thread" for local consistency. >> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/prims/jvm.cpp >>> >>> The calls to JvmtiExport::post_vm_object_alloc have to use "thread" >>> instead of "THREAD", even though other places nearby are using >>> "THREAD".? That inconsistency is kind of unfortunate, but doesn't seem >>> easily avoidable. >> >> Everything that uses THREAD in a JVM_ENTRY method can be changed to >> use "thread" instead. But I'm not sure it's a consistency worth >> pursuing at least as part of these changes (there are likely similar >> issues with most of the touched files). >> >> Thanks, >> David >> >>> ------------------------------------------------------------------------------ >>> >>> > From david.holmes at oracle.com Wed Jul 22 02:46:56 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 22 Jul 2020 12:46:56 +1000 Subject: RFR (M) 8249650: Optimize JNIHandle::make_local thread variable usage In-Reply-To: <82ac807a-1492-9ac0-570a-d08b1dc93e09@oracle.com> References: <8410d4a2-bbad-090f-55bf-88940f786781@oracle.com> <0590E210-6F23-4498-A51A-C3DAEF54B5AB@oracle.com> <6166e191-c954-70e5-5595-956a0c145d10@oracle.com> <82ac807a-1492-9ac0-570a-d08b1dc93e09@oracle.com> Message-ID: <4ca86ddb-8a73-783c-0b3f-e8003f7160a3@oracle.com> Hi Coleen, On 22/07/2020 4:01 am, coleen.phillimore at oracle.com wrote: > > This looks like a nice cleanup. Thanks for looking at this. > http://cr.openjdk.java.net/~dholmes/8249650/webrev.v2/src/hotspot/share/runtime/jniHandles.cpp.udiff.html > > I'm wondering why you took out the NULL return for make_local() without > a thread argument?? Here you may call Thread::current() unnecessarily. > > jobject JNIHandles::make_local(oop obj) { > - if (obj == NULL) { > - return NULL; // ignore null handles > - } else { > - Thread* thread = Thread::current(); > - assert(oopDesc::is_oop(obj), "not an oop"); > - assert(!current_thread_in_native(), "must not be in native"); > - return thread->active_handles()->allocate_handle(obj); > - } > + return make_local(Thread::current(), obj); > } I was simply using a standard call forwarding pattern to avoid code duplication. I suspect passing NULL is very rare so the unnecessary Thread::current() call is not an issue. Otherwise, if not NULL, the NULL check would happen twice (unless I keep the duplicated implementations). > Beyond the scope of this fix, but it'd be cool to not have a version > that doesn't take thread, since there may be many more callers that > already have Thread::current(). Indeed! And in fact I had missed a number of these in jvm.cpp and jni.cpp so I have fixed those. I've filed a RFE for other cases: https://bugs.openjdk.java.net/browse/JDK-8249837 Updated webrev: http://cr.openjdk.java.net/~dholmes/8249650/webrev.v3/ If this passes tier 1-3 re-testing then I plan to push. Thanks, David ----- > Coleen > > > On 7/20/20 1:53 AM, David Holmes wrote: >> Hi Kim, >> >> Thanks for looking at this. >> >> Updated webrev at: >> >> http://cr.openjdk.java.net/~dholmes/8249650/webrev.v2/ >> >> On 20/07/2020 3:22 pm, Kim Barrett wrote: >>>> On Jul 20, 2020, at 12:16 AM, David Holmes >>>> wrote: >>>> >>>> Subject line got truncated by accident ... >>>> >>>> On 20/07/2020 11:06 am, David Holmes wrote: >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8249650 >>>>> webrev: http://cr.openjdk.java.net/~dholmes/8249650/webrev/ >>>>> This is a simple cleanup that touches files across a number of VM >>>>> areas - hence the cross-post. >>>>> Whilst working on a different JNI fix I noticed that in most cases >>>>> in jni.cpp we were using the following form of make_local: >>>>> JNIHandles::make_local(env, obj); >>>>> and what that form does is first extract the thread from the JNIEnv: >>>>> JavaThread* thread = JavaThread::thread_from_jni_environment(env); >>>>> return thread->active_handles()->allocate_handle(obj); >>>>> but there is also another, faster, variant for when you already >>>>> have the "thread": >>>>> jobject JNIHandles::make_local(Thread* thread, oop obj) { >>>>> ?? return thread->active_handles()->allocate_handle(obj); >>>>> } >>>>> When you look at the JNI_ENTRY wrapper (and related JVM_ENTRY, >>>>> WB_ENTRY, UNSAFE_ENTRY etc) it has already extracted the thread >>>>> from the JNIEnv: >>>>> ???? JavaThread* thread=JavaThread::thread_from_jni_environment(env); >>>>> and further defined: >>>>> ???? Thread* THREAD = thread; >>>>> so we always already have direct access to the "thread" available >>>>> (or indirect via TRAPS), and in fact we can end up removing the >>>>> make_local(JNIEnv* env, oop obj) variant altogether. >>>>> Along the way I spotted some related issues with unnecessary use of >>>>> Thread::current() when it is already available from TRAPS, and some >>>>> other cases where we extracted the JNIEnv from a thread only to >>>>> later extract the thread from the JNIEnv. >>>>> Testing: tiers 1 - 3 >>>>> Thanks, >>>>> David >>>>> ----- >>> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/classfile/javaClasses.cpp >>> ? 439???? JNIEnv *env = thread->jni_environment(); >>> >>> Since env is no longer used on the next line, move this down to where >>> it is used, at line 444. >> >> Fixed. >> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/classfile/verifier.cpp >>> ? 299?? JNIEnv *env = thread->jni_environment(); >>> >>> env now seems to only be used at line 320.? Move this closer. >> >> Fixed. >> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/prims/jni.cpp >>> ? 743???? result = JNIHandles::make_local(THREAD, result_handle()); >>> >>> jni_PopLocalFrame is now using a mix of "thread" and "THREAD", where >>> previously it just used "thread". Maybe this change shouldn't be made? >>> Or can the other uses be changed to THREAD for consistency? >> >> "thread" and "THREAD" are interchangeable for anything expecting a >> "Thread*" (and somewhat surprisingly a number of API's that only work >> for JavaThreads actually take a Thread*. :( ). I had choice between >> trying to be file-wide consistent with the make_local calls, versus >> local-code consistent, and used THREAD as it is available in both >> JNI_ENTRY and via TRAPS. But I can certainly make a local change to >> "thread" for local consistency. >> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/prims/jvm.cpp >>> >>> The calls to JvmtiExport::post_vm_object_alloc have to use "thread" >>> instead of "THREAD", even though other places nearby are using >>> "THREAD".? That inconsistency is kind of unfortunate, but doesn't seem >>> easily avoidable. >> >> Everything that uses THREAD in a JVM_ENTRY method can be changed to >> use "thread" instead. But I'm not sure it's a consistency worth >> pursuing at least as part of these changes (there are likely similar >> issues with most of the touched files). >> >> Thanks, >> David >> >>> ------------------------------------------------------------------------------ >>> >>> > From daniil.x.titov at oracle.com Wed Jul 22 03:25:00 2020 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Tue, 21 Jul 2020 20:25:00 -0700 Subject: RFR: 8216324: GetClassMethods is confused by the presence of default methods in super interfaces In-Reply-To: References: <46dafc62-bbab-9287-b398-59ca8d99d03a@oracle.com> Message-ID: <6471F5FB-CB4E-49A7-BCC1-1C5718E5A67D@oracle.com> Hi Serguei and Alex, Please, review new version of the change [1] that includes changes as Serguei suggested. > 114 default void default_method() { } // should never be seen > The comment above is not clear. Should never be seen in what context? This method should not be included in the list of methods GetClassMethods returns for the sub-interface or implementing class. I don't think we want to comment each method in the test interfaces declared in this test when they should be seen in this test and when they should not. This information already exists in getclmthd007.cpp, thus I decided to omit this comment. Please see below the output from the new test. ----------messages:(4/215)---------- command: main -agentlib:DefaultMethods DefaultMethods reason: User specified action: run main/othervm/native -agentlib:DefaultMethods DefaultMethods Mode: othervm [/othervm specified] elapsed time (seconds): 0.241 ----------configuration:(0/0)---------- ----------System.out:(3/265)---------- Reflection getDeclaredMethods returned: [public abstract java.lang.String DefaultMethods$Child.method2()] JVMTI GetClassMethods returned: [public abstract java.lang.String DefaultMethods$Child.method2()] Test passed: Got expected output from JVMTI GetClassMethods! ----------System.err:(1/15)---------- STATUS:Passed. ----------messages:(4/276)---------- command: main -agentlib:DefaultMethods=maintain_original_method_order DefaultMethods reason: User specified action: run main/othervm/native -agentlib:DefaultMethods=maintain_original_method_order DefaultMethods Mode: othervm [/othervm specified] elapsed time (seconds): 0.25 ----------configuration:(0/0)---------- ----------System.out:(4/322)---------- Enabled capability: maintain_original_method_order: true Reflection getDeclaredMethods returned: [public abstract java.lang.String DefaultMethods$Child.method2()] JVMTI GetClassMethods returned: [public abstract java.lang.String DefaultMethods$Child.method2()] Test passed: Got expected output from JVMTI GetClassMethods! ----------System.err:(1/15)---------- STATUS:Passed. [1] http://cr.openjdk.java.net/~dtitov/8216324/webrev.02/ Thanks, Daniil From: "serguei.spitsyn at oracle.com" Date: Monday, July 20, 2020 at 6:48 PM To: Alex Menkov , Daniil Titov , serviceability-dev Subject: Re: RFR: 8216324: GetClassMethods is confused by the presence of default methods in super interfaces Hi Daniil, The fix looks pretty good to me. Just minor comments. http://cr.openjdk.java.net/~dtitov/8216324/webrev.01/src/hotspot/share/prims/jvmtiEnv.cpp.frames.html 2519 int skipped = 0; // skip default methods from superinterface (see JDK-8216324) You can say just:? // skip overpass methods There is probably no need to list the bug number. 2523 // Depending on can_maintain_original_method_order capability 2524 // use the original method ordering indices stored in the class, so we can emit 2525 // jmethodIDs in the order they appeared in the class file 2526 // or just copy in any order Could you, please re-balance the comment a little bit? Also, the comment has to be terminated with a dot. Replace: "or just copy in any order" => "or just copy in current order". http://cr.openjdk.java.net/~dtitov/8216324/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetClassMethods/getclmthd007.java.frames.html 114 default void default_method() { } // should never be seen The comment above is not clear. Should never be seen in what context? 117 interface OuterInterface1 extends DefaultInterface { An extra space before "extends". http://cr.openjdk.java.net/~dtitov/8216324/webrev.01/test/hotspot/jtreg/serviceability/jvmti/GetClassMethods/InterfaceDefMethods.java.html I like the test simplicity. Default methods are always in interfaces. I'd suggest to rename the test to something like: DefaultMethods.java or OverpassMethods.java. http://cr.openjdk.java.net/~dtitov/8216324/webrev.01/test/hotspot/jtreg/serviceability/jvmti/GetClassMethods/libInterfaceDefMethods.cpp.html 44 if (options != NULL && strcmp(options, "maintain_original_method_order") == 0) { 45 printf("maintain_original_method_order: true\n"); ... 54 } else { 55 printf("maintain_original_method_order: false\n"); I'd suggest to remove the lines 54 and 55 and adjust the line 45: ?? printf("Enabled capability: maintain_original_method_order: true\n"); 88 jobject m = env->ToReflectedMethod(klass, methods[i], (modifiers & 8) == 8); It is better to replace 8 with a symbolic constant. Thanks, Serguei On 7/20/20 16:57, Alex Menkov wrote: Looks good to me :) Thanks for handling this. --alex On 07/20/2020 12:00, Daniil Titov wrote: Please review change [1] that fixes GetClassMethods behavior in cases if a default method is present in a super interface. Currently for such cases the information GetClassMethods returns for the sub-interface or implementing class incorrectly includes? inherited not default? methods. The fix ( thanks to Alex for providing this patch) ensures that overpass methods are filtered out? in the returns. The fix also applies a change in the existing test as David suggested in the comments to the issue [2]. Testing: Mach5? tier1-tier3 tests successfully passed. [1] http://cr.openjdk.java.net/~dtitov/8216324/webrev.01/ [2] https://bugs.openjdk.java.net/browse/JDK-8216324 Thank you, Daniil From serguei.spitsyn at oracle.com Wed Jul 22 05:53:38 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 21 Jul 2020 22:53:38 -0700 Subject: RFR: 8216324: GetClassMethods is confused by the presence of default methods in super interfaces In-Reply-To: <6471F5FB-CB4E-49A7-BCC1-1C5718E5A67D@oracle.com> References: <46dafc62-bbab-9287-b398-59ca8d99d03a@oracle.com> <6471F5FB-CB4E-49A7-BCC1-1C5718E5A67D@oracle.com> Message-ID: <36efbae7-686d-ebb9-c280-489bf1920f2e@oracle.com> An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Wed Jul 22 06:32:13 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 22 Jul 2020 16:32:13 +1000 Subject: RFR (M) 8249768: Move static oops and NullPointerException oops from Universe into OopStorage In-Reply-To: References: <9fb1860f-06cf-9347-27d7-16205c86c326@oracle.com> Message-ID: Hi Coleen, On 22/07/2020 12:19 am, coleen.phillimore at oracle.com wrote: > > > On 7/20/20 10:47 PM, David Holmes wrote: >> Hi Coleen, >> >> cc'ing serviceability due to SA changes. >> >> On 21/07/2020 6:53 am, coleen.phillimore at oracle.com wrote: >>> Summary: Move static oops into OopStorage and make NPE oops an >>> objArrayOop. >>> >>> I've broken up moving oops in Universe to OopStorage into several >>> parts.? This change moves the global static oops.? These OopHandles >>> are not released. >> >> Overall looks good. But two things ... >> >> 1. naming >> >> !?? // preallocated error objects (no backtrace) >> !?? static OopHandle??? _out_of_memory_error; >> >> ??? // array of preallocated error objects with backtrace >> !?? static OopHandle???? _preallocated_out_of_memory_error_array; >> >> Both of these are pre-allocated arrays of OopHandles, differing only >> in whether the underlying OOME has a backtrace or not. I find the >> newly introduced _out_of_memory_error unclear in that regard. At a >> minimum could _out_of_memory_error become _out_of_memory_errors ? But >> ideally can we name these two arrays in a more distinguishable way? > > I put this code in functions next to each other because it was > confusing.? The _out_of_memory_error array is really preallocated > throwables, that are used to fill in the message for > preallocated_out_of_memory_errors if there are enough of the latter left. > I could rename _out_of_memory_error => _out_of_memory_error_throwables ? That doesn't really help. As I said both of these variables are arrays of pre-allocated OOME instances (which are throwables) - the only difference is one set is single-use (as it can have its backtrace set) while the other is reusable. The existing variable _preallocated_out_of_memory_error_array tells you clearly it is an array of preallocated OOME instances (but doesn't saying anything about the backtrace or being single-use). The problem is that that is exactly what _out_of_memory_error is as well, but the name _out_of_memory_error doesn't convey that it is an array, nor that anything is pre-allocated (and also nothing about backtraces or re-usability). So given we now have two arrays of extremely similar things, it would be clearer to give these clearer names. If we want to keep the existing _preallocated_out_of_memory_error_array name, then the new array should indicate how it differs e.g. _reusable_preallocated_out_of_memory_error_array What do you think? I also spotted a minor nit: 187 oop Universe::system_thread_group() { return _system_thread_group.resolve(); } 188 void Universe::set_system_thread_group(oop group) { _system_thread_group = OopHandle(vm_global(), group); } Extra spaces after oop on L187. Thanks, David ----- >> >> 2. SA >> >> You've simply deleted all the SA/vmstructs code that referenced the >> oops that are no longer present. Does the SA not care about these >> things and need access to them? (I don't know hence cc to >> serviceability folk.) > > Yes, the SA code was never used, so I deleted it.? SA might need in oop > inspection to add walking Universe::vm_global() OopStorage area to find > where oops come from if there's an error but there doesn't seem to be > any other reason for SA to use these oops. > > Thanks, > Coleen > >> >> Thanks, >> David >> ----- >> >>> This has been tested with tier1-3 and on also remote-build -b >>> linux-arm32,linux-ppc64le-debug,linux-s390x-debug,linux-x64-zero. >>> >>> open webrev at >>> http://cr.openjdk.java.net/~coleenp/2020/8249768.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8249768 >>> >>> Thanks, >>> Coleen > From suenaga at oss.nttdata.com Wed Jul 22 07:39:20 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Wed, 22 Jul 2020 16:39:20 +0900 Subject: 8248362: JVMTI frame operations should use Thread-Local Handshake Message-ID: Hi all, I'm working for fixing JDK-8248362, but I saw some errors on submit repo. Someone can share the details of mach5-one-ysuenaga-JDK-8248362-20200722-0550-12850261 ? I wonder why build task of linux-x64 was failed because I can do it on my Fedora 32 box. Thanks, Yasumasa From david.holmes at oracle.com Wed Jul 22 07:57:52 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 22 Jul 2020 17:57:52 +1000 Subject: 8248362: JVMTI frame operations should use Thread-Local Handshake In-Reply-To: References: Message-ID: <88260426-338e-03c2-0eb5-9d744e5d37b5@oracle.com> Hi Yasumasa, On 22/07/2020 5:39 pm, Yasumasa Suenaga wrote: > Hi all, > > I'm working for fixing JDK-8248362, but I saw some errors on submit repo. > Someone can share the details of > mach5-one-ysuenaga-JDK-8248362-20200722-0550-12850261 ? > > I wonder why build task of linux-x64 was failed because I can do it on > my Fedora 32 box. [2020-07-22T06:21:49,141Z] ./open/src/hotspot/share/prims/jvmtiThreadState.cpp:222:45: error: no member named 'active_handshaker' in 'JavaThread' [2020-07-22T06:21:49,142Z] current_thread == get_thread()->active_handshaker(), [2020-07-22T06:21:49,142Z] ~~~~~~~~~~~~ ^ David ----- > > Thanks, > > Yasumasa From suenaga at oss.nttdata.com Wed Jul 22 08:19:01 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Wed, 22 Jul 2020 17:19:01 +0900 Subject: 8248362: JVMTI frame operations should use Thread-Local Handshake In-Reply-To: <88260426-338e-03c2-0eb5-9d744e5d37b5@oracle.com> References: <88260426-338e-03c2-0eb5-9d744e5d37b5@oracle.com> Message-ID: On 2020/07/22 16:57, David Holmes wrote: > Hi Yasumasa, > > On 22/07/2020 5:39 pm, Yasumasa Suenaga wrote: >> Hi all, >> >> I'm working for fixing JDK-8248362, but I saw some errors on submit repo. >> Someone can share the details of mach5-one-ysuenaga-JDK-8248362-20200722-0550-12850261 ? >> >> I wonder why build task of linux-x64 was failed because I can do it on my Fedora 32 box. > > [2020-07-22T06:21:49,141Z] ./open/src/hotspot/share/prims/jvmtiThreadState.cpp:222:45: error: no member named 'active_handshaker' in 'JavaThread' > [2020-07-22T06:21:49,142Z]???????????? current_thread == get_thread()->active_handshaker(), > [2020-07-22T06:21:49,142Z]?????????????????????????????? ~~~~~~~~~~~~? ^ Thanks David! This statement is in guarantee(), so it seems to be failed to build for production VM. guarantee() call has been introduced in JDK-6471769, originally it was assert() call. Can we replace guarantee() to assert() at this point? or are there methods to detect the call is happened in direct handshake without active_handshaker()? Thanks, Yasumasa > David > ----- > >> >> Thanks, >> >> Yasumasa From thomas.schatzl at oracle.com Wed Jul 22 08:14:42 2020 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 22 Jul 2020 10:14:42 +0200 Subject: [15?] RFR (S): 8249192: MonitorInfo stores raw oops across safepoints In-Reply-To: <893f862f-093a-5168-e554-1f4ffe129aea@oracle.com> References: <3f838c8a-6e70-cd94-1753-8abe269b7cdf@oracle.com> <528ade6c-059a-d9af-b0c7-74d7ac31b29a@oracle.com> <14512a57-8d4d-5199-ed16-743b5da8a10d@oracle.com> <893f862f-093a-5168-e554-1f4ffe129aea@oracle.com> Message-ID: <07904b7e-8d40-8c9e-acc1-c713c1588b11@oracle.com> Hi Serguei, thanks for your review. On 22.07.20 04:13, serguei.spitsyn at oracle.com wrote: > Hi Thomas, > > The fix looks okay to me. > The 15 fix is identical to 16. no :) It is very subtle. As mentioned, in jvmtiEnvBase.cpp in the original code, in jdk15 we had: line 1029: ResourceMark rm; in jdk16: line 1008: ResourceMark rm(current_thread); Both lines were ultimately removed with this recent change, but still cause merge errors if you port the patch from one version to the other. Sorry if that has been confusing. > > This file finally was not changed: > src/hotspot/share/runtime/vframe_hp.cpp . This is an artifact of webrev in conjunction with mq because the original change had some modifications which were retracted in the -1 webrev there. There will not be any change in any push for that file. > Several files need a copyright comment update. Provided new webrevs with only comment updates below. Did not do new testing just for these comment updates though. > > What tests do you run? > We need at least tier3 to make sure there are no regressions in the > JVMTI and JDI tests. The jdk15 .1 patch has been tested with 1.2k of that LocalsAndOperands test with the options that originally reproduced it in 1/100 cases. hs-tier1-5, no failures at all. jdk16 has had testing with the .0 patch doing 1.8k of LocalsAndOperands.java, tier1-5, and tier8 with JDK-8249676 reinstated that earlier caused lots of issues there (and none without). Since there has been no substantial change in how the patch works, only some refactoring, so I think these are still valid. See the internal comments in the CR for links. New webrevs: jdk15: http://cr.openjdk.java.net/~tschatzl/8249192/webrev.jdk15.2/ (full) http://cr.openjdk.java.net/~tschatzl/8249192/webrev.jdk15.1_to_2/ (diff) jdk16: http://cr.openjdk.java.net/~tschatzl/8249192/webrev.2/ (full) http://cr.openjdk.java.net/~tschatzl/8249192/webrev.1_to_2/ (diff) Thanks, Thomas From thomas.schatzl at oracle.com Wed Jul 22 08:21:46 2020 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 22 Jul 2020 10:21:46 +0200 Subject: [15?] RFR (S): 8249192: MonitorInfo stores raw oops across safepoints In-Reply-To: References: <3f838c8a-6e70-cd94-1753-8abe269b7cdf@oracle.com> <528ade6c-059a-d9af-b0c7-74d7ac31b29a@oracle.com> <14512a57-8d4d-5199-ed16-743b5da8a10d@oracle.com> Message-ID: <5c33af91-446a-89ba-1a99-30c79cf2f1d9@oracle.com> Hi, On 22.07.20 02:42, David Holmes wrote: > Hi Thomas, > > I've looked at the incremental update and I am happy with that. In the response to Serguei's review there were some comment updates and new webrevs. > > I also, prompted by you mentioning it, took a deeper look at the > biased-locking code to ensure it also keeps the MonitorInfo's > thread-confined, and to see whether the handshake versions could > themselves be susceptible to interference from safepoints (which they > can't as far as I can determine). And that all seems fine. Thanks for looking at this again in more detail. I couldn't spot problems either, but this is not code I am proficient with. > As per offline discussions I know that there has been an alternate > proposal for a completely localized fix in the stackwalker code that > simply retrieves the list of monitors, uses the length to create the > array, then re-retrieves the list of monitors to populate the array (the > length of which can't change as we are dealing with the current thread). > My only concern with that approach is the performance impact if we have > deep stacks with lots of monitors. There is a microbenchmark for > StackWalker in the repo: > > open/test/micro/org/openjdk/bench/java/lang/StackWalkBench.java > > but it doesn't test anything to do with monitor usage. Ultimately I am good with either change, as long as it's being fixed. I initially thought about this solution too, but had the same concerns. Stacks can be really deep particularly with some frameworks (maybe not fully materialized) but idk the actual impact of doing the walk twice. Potentially you could have different fixes for different versions. Thanks, Thomas From richard.reingruber at sap.com Wed Jul 22 08:20:15 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Wed, 22 Jul 2020 08:20:15 +0000 Subject: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents In-Reply-To: References: <1f8a3c7a-fa0f-b5b2-4a8a-7d3d8dbbe1b5@oracle.com> <4b56a45c-a14c-6f74-2bfd-25deaabe8201@oracle.com> <5271429a-481d-ddb9-99dc-b3f6670fcc0b@oracle.com> Message-ID: Hi Goetz, > I'll answer to the obvious things in this mail now. > I'll go through the code thoroughly again and write > a review of my findings thereafter. Sure. If trimmed my citations to relevant parts. > > The delta includes many changes in comments, renaming of names, etc. So > > I'd like to summarize > > functional changes: > > > > * Collected all the code for the testing feature DeoptimizeObjectsALot in > > compileBroker.cpp and reworked it. > Thanks, this makes it much more compact. > > With DeoptimizeObjectsALot enabled internal threads are started that > > deoptimize frames and > > objects. The number of threads started are given with > > DeoptimizeObjectsALotThreadCountAll and > > DeoptimizeObjectsALotThreadCountSingle. The former targets all existing > > threads whereas the > > latter operates on a single thread selected round robin. > > > > I removed the mode where deoptimizations were performed at every nth > > exit from the runtime. I never used it. > Do I get it right? You have a n:1 and a n:all test scenario. > n:1: n threads deoptimize 1 Jana thread where n = DOALThreadCountSingle > n:m: n threads deoptimize all Java threads where n = DOALThreadCountAll? Not quite. -XX:+DeoptimizeObjectsALot // required -XX:DeoptimizeObjectsALotThreadCountAll=m -XX:DeoptimizeObjectsALotThreadCountSingle=n Will start m+n threads. Each operating on all existing JavaThreads using EscapeBarriers. The difference between the 2 thread types is that one distinct EscapeBarrier targets either just a single thread or all exisitng threads at onece. If just one single thread is targeted per EscapeBarrier, then it is not always the same thread, but threads are selected round robin. So there will be n threads selecting independently single threads round robin per EscapeBarrier and m threads that target all threads in every EscapeBarrier. > > * EscapeBarrier::sync_and_suspend_one(): use a direct handshake and > > execute it always independently > > of is_thread_fully_suspended(). > Is this also a performance optimization? Maybe a minor one. > > * JavaThread::wait_for_object_deoptimization(): > > - Bugfix: the last check of is_obj_deopt_suspend() must be /after/ the > > safepoint check! This > > caused issues with not walkable stacks with DeoptimizeObjectsALot. > OK. As I understand, there was one safepoint check in the old version, > now there is one in each iteration. I assume this is intended, right? Yes it is. The important thing here is (A) a safepoint check is needed /after/ leaving a safe state (_thread_in_native, _thread_blocked). (B) Shared variables that are modified at safepoints or with handshakes need to be reread /after/ the safepoint check. BTW: I only noticed now that since JDK-8240918 JavaThreads themselves must disarm their polling page. Originally (before handshakes) this was done by the VM thread. With handshakes it was done by the thread executing the handshake op. This was change for OrderAccess::cross_modify_fence() where the poll is left armed if the thread is in native and sice JDK-8240918 it is always left armed. So when a thread leaves a safe state (native, blocked) and there was a handshake/vm op, it will always call SafepointMechanism::block_if_requested_slow(), even if the handshake/vm operation have been processed already and everybody else is happyly executing bytecodes :) Still (A) and (B) hold. > > - Added limited spinning inspired by HandshakeSpinYield to fix regression in > > microbenchmark [1] > Ok. Nice improvement, nice catch! Yes. It certainly took some time to find out. > > > > I refer to some more changes answering your questions and comments inline > > below. > > > > Thanks, > > Richard. > > > > [1] Microbenchmark: > > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.6.microbenchmark/ > > > > > I understand you annotate at safepoints where the escape analysis > > > finds out that an object is "better" than global escape. > > > This are the cases where the analysis identifies optimization > > > opportunities. These annotations are then used to deoptimize > > > frames and the objects referenced by them. > > > Doesn't this overestimate the optimized > > > objects? E.g., eliminate_alloc_node has many cases where it bails > > > out. > > > > Yes, the implementation is conservative, but it is comparatively simple and > > the additional debug > > info is just 2 flags per safepoint. > Thanks. It also helped that you explained to me offline that > there are more optimizations than only lock elimination and scalar > replacement done based on the ea information. > The ea refines the IR graph with allows follow up optimizations > which can not easily be tracked back to the escaping objects or > the call sites where they do not escape. > Thus, if there are non-global escaping objects, you have to > deoptimize the frame. > Did I repeat that correctly? Mostly, but there are also cases, where deoptimization is required if and only if ea-local objects are passed as arguments. This is the case, when values are not read directely from a frame, but from a callee frame. > With this understanding, a row of my proposed renamings/comments > are obsolete. Ok. > > On the other hand, those JVMTI operations > > that really trigger > > deoptimizations are expected to be comparatively infrequent such that > > switching to the interpreter > > for a few microseconds will hardly have an effect. > That sounds reasonable. > > I've done microbenchmarking to check this. > > > > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.6.microbe > > nchmark/ > > > > I found that in the worst case performance can be impacted by 10%. If the > > agent is extremely active > > and does relevant JVMTI calls like GetOwnedMonitorStackDepthInfo() every > > millisecond or more often, > > then the performance impact can be 30%. But I would think that this is not > > realistic. These calls > > are issued in interactive sessions to analyze deadlocks. > Ok. > > We could get more precise deoptimizations by adding a third flag per > > safepoint for ea-local objects > > among the owned monitors. This would help improve the worst case in the > > benchmark. But I'm not > > convinced, if it is worth it. > > > > Refer to the README.txt of the microbenchmark for a more detailled > > discussion. > > > pcDesc.hpp > > > > > > I would like to see some documentation of the methods. > > Done. I didn't take your text, though, because I only noticed it after writing > > my own. Let me know if you are not ok with it. > That's fine. My texts were only proposals, you as author know better > what goes on anyways. Ok. > > > scopeDesc.cpp > > > > > > Besides refactoring copy escape info from pcDesc to scopeDesc > > > and add accessors. Trivial. > > > > > > In scopeDesc.hpp you talk about NoEscape and ArgEscape. > > > This are opto terms, but scopeDesc is a shared datastructure > > > that does not depend on a specific compiler. > > > Please explain what is going on without using these terms. > > > > Actually these are not too opto specific terms. They are used in the paper > > referenced in > > escape.hpp. Also you can easily google them. I'd rather keep the comments > > as they are. > Hmm, I'm not really happy with this, as also the papers > are for the compiler community, and probably not familiar to > others that work with HotSpot. > But stay with your terms if you think it makes it clearer. > Anyways, with now understanding why you use conservative > Information (see above), the descriptions I had in mind are not precise. Ok. > > > callnode.hpp > > > > > > You add functionality to annotate callnodes with escape information > > > This is carried through code generation to final output where it is > > > added to the compiled methods meta information. > > > > > > At Safepoints in general jvmti can access > > > - Objects that were scalar replaced. They must be reallocated. > > > (Flag EliminateAllocations) > > > - Objects that should be locked but are not because they never > > > escape the thread. They need to be relocked. > > > > > > At calls, Objects where locks have been removed escape to callees. > > > We must persist this information so that if jvmti accesses the > > > object in a callee, we can determine by looking at the caller that > > > it needs to be relocked. > > > > Note that the ea-optimization must not be at the current location, it can also > > follow when control > > returns to the caller. Lock elimination isn't the only relevant optimization. > Yes, I understood now, see above. Thanks for explaining. Ok. > > Accesses to instance > > members or array elements can be optimized as well. > You mean the compiler can/will ignore volatile or memory ordering > requirements for non-escaping objects? Sounds reasonable to do. Yes, for instance. Also without volatile modifiers it will eliminate accesses. Here is an example: Method A has a NoEscape allocation O that is not scalar replaced. A calls Method B, which is not inlined. When you use your debugger to break in B, then modify a field of O, then this modification would have no effect without deoptimization, because the jit assumes that B cannot modify O without a reference to it. > > You are right, it is not correct how flags are checked. Especially if only > > running with the JVMCI compiler. > > > > I changed Deoptimization::deoptimize_objects_internal() to make > > reallocation and relocking dependent > > on similar checks as in Deoptimization::fetch_unroll_info_helper(). > > Furthermore EscapeBarriers are > > conditionally activated depending on the following (see EscapeBarrier ctors): > > > > JVMCI_ONLY(UseJVMCICompiler) NOT_JVMCI(false) > > COMPILER2_PRESENT(|| DoEscapeAnalysis) > > > > So the enhancement can be practically completely disabled by disabling > > DoEscapeAnalysis, which is > > what C2 currently does if JVMTI capabilities that allow access to local > > references are taken. > Thanks for fixing. Thanks for finding :) > > I went for the latter. > > > > > In fetch_unroll_info_helper, I don't understand why you need > > > && !EscapeBarrier::objs_are_deoptimized(thread, deoptee.id())) { > > > for eliminated locks, but not for skalar replaced objects? > > > > In short reallocation is idempotent, relocking is not. > > > > Without the enhancement Deoptimization::realloc_objects() can already be > > called more than once for a frame: > > > > First call in materializeVirtualObjects() (also iterateFrames()). > > > > Second (indirect) call in fetch_unroll_info_helper(). > > > > The objects from the first call are saved as jvmti deferred updates when > > realloc_objects() > > returns. Note that there is no relationship to jvmti. The thing in common is > > that updates cannot be > > directely installed into a compiled frame, it is necessary to deoptimize the > > frame and defer the > > updates until the compiled frame gets replaced. Every time the vframes > > corresponding to the owner > > frame are iterated, they get the deferred updates. So in > > fetch_unroll_info_helper() the > > GrowableArray* chunk reference them too. All > > references to the objects created by > > the second (indirect) call to realloc_objects() are never used, because > > compiledVFrame accessors to > > locals, expressions, and monitors override them with the deferred updates. > > The objects become > > unreachable and get gc'ed. > OK, so repeatedly computed vFrames always have the first version of > reallocated objects by construction, so it needs not be handled here. > But also due to construction, objects might be allocated just to be > discarded. Yes. > > materializeVirtualObjects() does not bother with relocking. > > deoptimize_objects_internal(), which is > > introduced by the enhancement, does relock objects, after all the lock > > elimination becomes illegal > > with the change in escape state. Relocking twice does not work, so the > > enhancement avoids it by > > checking EscapeBarrier::objs_are_deoptimized(thread, deoptee.id()). > > > > Note that materializeVirtualObjects() can be called more than once and will > > always return the very > > same objects, even though it calls realloc_objects() again. > Ok. > > > I would guess it is because the eliminated locks can be applied to > > > argEscape, but scalar replacement only to noescape objects? > > > I.e. it might have been done before? > > > > > > But why isn't this the case for eliminate_allocations? > > > deoptimize_objects_internal does both unconditionally, > > > so both can happen to inner frames, right? > > > > Sorry, I don't quite understand. Hope the explanation above helps. > Yes. I was guessing wrong :) Ok, good :) > > > > > Code will get much more simple if BiasedLocking is removed. > > > > > > EscapeBarrier:: ... > > > > > > (This class maybe would qualify for a file of its own.) > > > > > > deoptimize_objects() > > > I would mention escape analysis only as side remark. Also, as I understand, > > > there is only one frame at given depth? > > > // Deoptimize frames with optimized objects. This can be omitted locks and > > > // objects not allocated but replaced by scalars. In C2, these optimizations > > > // are based on escape analysis. > > > // Up to depth, deoptimize frames with any optimized objects. > > > // From depth to entry_frame, deoptimize only frames that > > > // pass optimized objects to their callees. > > > (First part similar for the comment above > > EscapeBarrier::deoptimize_objects_internal().) > > > > I've reworked the comment. Let me know if you still think it needs to be > > improved. > Good now, thanks (maybe break the long line ...) Ok. Will do in next webrev.7 > > > Syncronization: looks good. I think others had a look at this before. > > > > > > EscapeBarrier::deoptimize_objects_internal() > > > The method name is misleading, it is not used by > > > deoptimize_objects(). > > > Also, method with the same name is in Deopitmization. > > > Proposal: deoptimize_objects_thread() ? > > > > Sorry, but I don't see, why it would be misleading. > > What would be the meaning of 'deoptimize_objects_thread'? I don't > > understand that name. > 1. I have no idea why it's called "_internal". Because it is private? > By the name, I would expect that EscapeBarrier::deoptimize_objects() > calls it for some internal tasks. But it does not. Well, I'd say it is pretty internal, what's happening in that method. So IMHO the suffix _internal is a match. > 2. My proposal: deoptimize_objects_all_threads() iterates all threads > and calls deoptimize_objects(_one)_thread(thread) for each of these. > That's how I would have named it. > But no bike shedding, if you don't see what I mean it's not obvious. Ok. We could have a quick call, too, if you like. > > > Renaming deferred_locals to deferred_updates is good, as well as > > > adding a datastructure for it. > > > (Adding this data structure might be a breakout, too.) > > > > > > good. > > > > > > thread.cpp > > > > > > good. > > > > > > vframe.cpp > > > > > > Is this a bug in existing code? > > > Makes sense. > > > > Depends on your definition of bug. There are no references to > > vframe::is_entry_frame() in the > > existing code. I would think it is a bug. > So it is :) I'm just afraid it could get fixed by removing the class entryVFrame. > > > > > > > > vframe_hp.hpp > > > (What stands _hp for? helper? The file should be named > > compiledVFrame ...) > > > > > > not_global_escape_in_scope() ... > > > Again, you mention escape analysis here. Comments above hold, too. > > > > I think it is the right name, because it is meaningful and simple. > Ok, accepted ... given my understandings from above. Ok. > > > > > You introduce JvmtiDeferredUpdates. Good. > > > > > > vframe_hp.cpp > > > > > > Changes for JvmtiDeferredUpdates, escape state accessors, > > > > > > line 422: > > > Would an assertion assert(!info->owner_is_scalar_replaced(), ...) hold here? > > > > > > > > > macros.hpp > > > Good. > > > > > > > > > Test coding > > > ============ > > > > > > compileBroker.h|cpp > > > > > > You introduce a third class of threads handled here and > > > add a new flag to distinguish it. Before, the two kinds > > > of threads were distinguished implicitly by passing in > > > a compiler for compiler threads. > > > The new thread kind is only used for testing in debug. > > > > > > make_thread: > > > You could assert (comp != NULL...) to assure previous > > > conditions. > > > > If replaced the if-statements with a switch-statement, made sure all enum- > > elements are covered, and > > added the assertion you suggested. > > > > > line 989 indentation broken > > > > You are referring to this block I assume: > > (from > > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.5/src/hots > > pot/share/compiler/compileBroker.cpp.frames.html) > > > > 976 if (MethodFlushing) { > > 977 // Initialize the sweeper thread > > 978 Handle thread_oop = create_thread_oop("Sweeper thread", CHECK); > > 979 jobject thread_handle = JNIHandles::make_local(THREAD, > > thread_oop()); > > 980 make_thread(sweeper_t, thread_handle, NULL, NULL, THREAD); > > 981 } > > 982 > > 983 #if defined(ASSERT) && COMPILER2_OR_JVMCI > > 984 if (DeoptimizeObjectsALot == 2) { > > 985 // Initialize and start the object deoptimizer threads > > 986 for (int thread_count = 0; thread_count < > > DeoptimizeObjectsALotThreadCount; thread_count++) { > > 987 Handle thread_oop = create_thread_oop("Deoptimize objects a lot > > thread", CHECK); > > 988 jobject thread_handle = JNIHandles::make_local(THREAD, > > thread_oop()); > > 989 make_thread(deoptimizer_t, thread_handle, NULL, NULL, THREAD); > > 990 } > > 991 } > > 992 #endif // defined(ASSERT) && COMPILER2_OR_JVMCI > > > > I cannot really see broken indentation here. Am I looking at the wrong > > location? > I don't have the source version I reviewed last time any more, so > I can't check. But maybe an artefact from patching ... if there were > tabs jcheck would have told you, so that's not it. No problem. Ok. Thanks again! Cheers, Richard. -----Original Message----- From: Lindenmaier, Goetz Sent: Donnerstag, 16. Juli 2020 18:30 To: Reingruber, Richard ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents Hi Richard, I'll answer to the obvious things in this mail now. I'll go through the code thoroughly again and write a review of my findings thereafter. > So here is the new webrev.6 > > Webrev.6: > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.6/ > Delta: > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.6.inc/ Thanks for the incremental webrev, it's helpful! > I spent most of the time running a microbenchmark [1] I wrote to answer > questions from your > review. At first I had trouble with variance in the results until I found out it > was due to the NUMA > architecture of the server I used. After that I noticed that there was a > performance regression of > about 5% even at low agent activity. I finally found out that it was due to the > implementation of > JavaThread::wait_for_object_deoptimization() which is called by the target > of the JVMTI operation to > self suspend for object deoptimization. I fixed this by adding limited spinning > before calling > wait() on the monitor. > > The delta includes many changes in comments, renaming of names, etc. So > I'd like to summarize > functional changes: > > * Collected all the code for the testing feature DeoptimizeObjectsALot in > compileBroker.cpp and reworked it. Thanks, this makes it much more compact. > With DeoptimizeObjectsALot enabled internal threads are started that > deoptimize frames and > objects. The number of threads started are given with > DeoptimizeObjectsALotThreadCountAll and > DeoptimizeObjectsALotThreadCountSingle. The former targets all existing > threads whereas the > latter operates on a single thread selected round robin. > > I removed the mode where deoptimizations were performed at every nth > exit from the runtime. I never used it. Do I get it right? You have a n:1 and a n:all test scenario. n:1: n threads deoptimize 1 Jana thread where n = DOALThreadCountSingle n:m: n threads deoptimize all Java threads where n = DOALThreadCountAll? > * EscapeBarrier::sync_and_suspend_one(): use a direct handshake and > execute it always independently > of is_thread_fully_suspended(). Is this also a performance optimization? > * Bugfix in EscapeBarrier::thread_added(): must not clear deopt flag. Found > this testing with DeoptimizeObjectsALot. Ok. > * Added EscapeBarrier::thread_removed(). Ok. > * EscapeBarrier constructors: barriers can now be entirely disabled by > disabling DoEscapeAnalysis. > This effectively disables the enhancement. Good! > * JavaThread::wait_for_object_deoptimization(): > - Bugfix: the last check of is_obj_deopt_suspend() must be /after/ the > safepoint check! This > caused issues with not walkable stacks with DeoptimizeObjectsALot. OK. As I understand, there was one safepoint check in the old version, now there is one in each iteration. I assume this is intended, right? > - Added limited spinning inspired by HandshakeSpinYield to fix regression in > microbenchmark [1] Ok. Nice improvement, nice catch! > > I refer to some more changes answering your questions and comments inline > below. > > Thanks, > Richard. > > [1] Microbenchmark: > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.6.microbenchmark/ > > > I understand you annotate at safepoints where the escape analysis > > finds out that an object is "better" than global escape. > > This are the cases where the analysis identifies optimization > > opportunities. These annotations are then used to deoptimize > > frames and the objects referenced by them. > > Doesn't this overestimate the optimized > > objects? E.g., eliminate_alloc_node has many cases where it bails > > out. > > Yes, the implementation is conservative, but it is comparatively simple and > the additional debug > info is just 2 flags per safepoint. Thanks. It also helped that you explained to me offline that there are more optimizations than only lock elimination and scalar replacement done based on the ea information. The ea refines the IR graph with allows follow up optimizations which can not easily be tracked back to the escaping objects or the call sites where they do not escape. Thus, if there are non-global escaping objects, you have to deoptimize the frame. Did I repeat that correctly? With this understanding, a row of my proposed renamings/comments are obsolete. > On the other hand, those JVMTI operations > that really trigger > deoptimizations are expected to be comparatively infrequent such that > switching to the interpreter > for a few microseconds will hardly have an effect. That sounds reasonable. > I've done microbenchmarking to check this. > > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.6.microbe > nchmark/ > > I found that in the worst case performance can be impacted by 10%. If the > agent is extremely active > and does relevant JVMTI calls like GetOwnedMonitorStackDepthInfo() every > millisecond or more often, > then the performance impact can be 30%. But I would think that this is not > realistic. These calls > are issued in interactive sessions to analyze deadlocks. Ok. > We could get more precise deoptimizations by adding a third flag per > safepoint for ea-local objects > among the owned monitors. This would help improve the worst case in the > benchmark. But I'm not > convinced, if it is worth it. > > Refer to the README.txt of the microbenchmark for a more detailled > discussion. > > pcDesc.hpp > > > > I would like to see some documentation of the methods. > Done. I didn't take your text, though, because I only noticed it after writing > my own. Let me know if you are not ok with it. That's fine. My texts were only proposals, you as author know better what goes on anyways. > > scopeDesc.cpp > > > > Besides refactoring copy escape info from pcDesc to scopeDesc > > and add accessors. Trivial. > > > > In scopeDesc.hpp you talk about NoEscape and ArgEscape. > > This are opto terms, but scopeDesc is a shared datastructure > > that does not depend on a specific compiler. > > Please explain what is going on without using these terms. > > Actually these are not too opto specific terms. They are used in the paper > referenced in > escape.hpp. Also you can easily google them. I'd rather keep the comments > as they are. Hmm, I'm not really happy with this, as also the papers are for the compiler community, and probably not familiar to others that work with HotSpot. But stay with your terms if you think it makes it clearer. Anyways, with now understanding why you use conservative Information (see above), the descriptions I had in mind are not precise. > > callnode.hpp > > > > You add functionality to annotate callnodes with escape information > > This is carried through code generation to final output where it is > > added to the compiled methods meta information. > > > > At Safepoints in general jvmti can access > > - Objects that were scalar replaced. They must be reallocated. > > (Flag EliminateAllocations) > > - Objects that should be locked but are not because they never > > escape the thread. They need to be relocked. > > > > At calls, Objects where locks have been removed escape to callees. > > We must persist this information so that if jvmti accesses the > > object in a callee, we can determine by looking at the caller that > > it needs to be relocked. > > Note that the ea-optimization must not be at the current location, it can also > follow when control > returns to the caller. Lock elimination isn't the only relevant optimization. Yes, I understood now, see above. Thanks for explaining. > Accesses to instance > members or array elements can be optimized as well. You mean the compiler can/will ignore volatile or memory ordering requirements for non-escaping objects? Sounds reasonable to do. > > // Returns true if at least one of the arguments to the call is an oop > > // that does not escape globally. > > bool ConnectionGraph::has_arg_escape(CallJavaNode* call) { > > IMHO the method names are descriptive and don't need the comments. But I > give in :) (only replaced > "oop" with "object") Thanks. Yes, object is better than oop. > You are right, it is not correct how flags are checked. Especially if only > running with the JVMCI compiler. > > I changed Deoptimization::deoptimize_objects_internal() to make > reallocation and relocking dependent > on similar checks as in Deoptimization::fetch_unroll_info_helper(). > Furthermore EscapeBarriers are > conditionally activated depending on the following (see EscapeBarrier ctors): > > JVMCI_ONLY(UseJVMCICompiler) NOT_JVMCI(false) > COMPILER2_PRESENT(|| DoEscapeAnalysis) > > So the enhancement can be practically completely disabled by disabling > DoEscapeAnalysis, which is > what C2 currently does if JVMTI capabilities that allow access to local > references are taken. Thanks for fixing. > I went for the latter. > > > In fetch_unroll_info_helper, I don't understand why you need > > && !EscapeBarrier::objs_are_deoptimized(thread, deoptee.id())) { > > for eliminated locks, but not for skalar replaced objects? > > In short reallocation is idempotent, relocking is not. > > Without the enhancement Deoptimization::realloc_objects() can already be > called more than once for a frame: > > First call in materializeVirtualObjects() (also iterateFrames()). > > Second (indirect) call in fetch_unroll_info_helper(). > > The objects from the first call are saved as jvmti deferred updates when > realloc_objects() > returns. Note that there is no relationship to jvmti. The thing in common is > that updates cannot be > directely installed into a compiled frame, it is necessary to deoptimize the > frame and defer the > updates until the compiled frame gets replaced. Every time the vframes > corresponding to the owner > frame are iterated, they get the deferred updates. So in > fetch_unroll_info_helper() the > GrowableArray* chunk reference them too. All > references to the objects created by > the second (indirect) call to realloc_objects() are never used, because > compiledVFrame accessors to > locals, expressions, and monitors override them with the deferred updates. > The objects become > unreachable and get gc'ed. OK, so repeatedly computed vFrames always have the first version of reallocated objects by construction, so it needs not be handled here. But also due to construction, objects might be allocated just to be discarded. > materializeVirtualObjects() does not bother with relocking. > deoptimize_objects_internal(), which is > introduced by the enhancement, does relock objects, after all the lock > elimination becomes illegal > with the change in escape state. Relocking twice does not work, so the > enhancement avoids it by > checking EscapeBarrier::objs_are_deoptimized(thread, deoptee.id()). > > Note that materializeVirtualObjects() can be called more than once and will > always return the very > same objects, even though it calls realloc_objects() again. Ok. > > I would guess it is because the eliminated locks can be applied to > > argEscape, but scalar replacement only to noescape objects? > > I.e. it might have been done before? > > > > But why isn't this the case for eliminate_allocations? > > deoptimize_objects_internal does both unconditionally, > > so both can happen to inner frames, right? > > Sorry, I don't quite understand. Hope the explanation above helps. Yes. I was guessing wrong :) > > I like if boolean operators are at the beginning of broken lines, > > but I think hotspot convention is to have them at the end. > Ok, fixed. Thanks. > > > Code will get much more simple if BiasedLocking is removed. > > > > EscapeBarrier:: ... > > > > (This class maybe would qualify for a file of its own.) > > > > deoptimize_objects() > > I would mention escape analysis only as side remark. Also, as I understand, > > there is only one frame at given depth? > > // Deoptimize frames with optimized objects. This can be omitted locks and > > // objects not allocated but replaced by scalars. In C2, these optimizations > > // are based on escape analysis. > > // Up to depth, deoptimize frames with any optimized objects. > > // From depth to entry_frame, deoptimize only frames that > > // pass optimized objects to their callees. > > (First part similar for the comment above > EscapeBarrier::deoptimize_objects_internal().) > > I've reworked the comment. Let me know if you still think it needs to be > improved. Good now, thanks (maybe break the long line ...) > > What is the check (cur_depth <= depth) good for? Can you > > ever walk past entry_frame? > > Yes (assuming you mean the outer while-statement), there are java frames > beyond the entry frame if a > native method calls java methods again. So we visit all frames up to the given > depth and from there > we continue to the entry frame. It is not necessary to continue beyond that > entry frame, because > escape analysis assumes that arguments to native functions escape globally. > > Example: Let the java stack look like this: > > +---------+ > | Frame A | > +---------+ > | Frame N | > +---------+ > | Frame B | > +---------+ <- top of stack > > Where java method A calls native method N and N calls java method B. > > Very simplified the native stack will look like this > > +-------------------------+ > | Frame of JIT Compiled A | > +-------------------------+ > | Frame N | > +-------------------------+ > | Entry Frame | > +-------------------------+ > | Frame B | > +-------------------------+ <- top of stack > > The entry frame is an activation of the call stub, which is a small assembler > routine that > translates from the native calling convention to the java calling convention. > > There cannot be any ArgEscape that is passed to B (see above), therefore we > can stop the stackwalk > at the entry frame if depth is 1. If depth is 3 we have to continue to Frame A, > as it is directely > accessed. Ok, thanks, nice explanation!! > > Isn't vf->is_compiled_frame() prerequisite that "Move to next physical > frame" > > is needed? You could move it into the other check. > > If so, similar for deoptimize_objects_all_threads(). > > Only compiledVFrame require moving to the /top/ frame. Fixed. Thanks, this looks better. > > Syncronization: looks good. I think others had a look at this before. > > > > EscapeBarrier::deoptimize_objects_internal() > > The method name is misleading, it is not used by > > deoptimize_objects(). > > Also, method with the same name is in Deopitmization. > > Proposal: deoptimize_objects_thread() ? > > Sorry, but I don't see, why it would be misleading. > What would be the meaning of 'deoptimize_objects_thread'? I don't > understand that name. 1. I have no idea why it's called "_internal". Because it is private? By the name, I would expect that EscapeBarrier::deoptimize_objects() calls it for some internal tasks. But it does not. 2. My proposal: deoptimize_objects_all_threads() iterates all threads and calls deoptimize_objects(_one)_thread(thread) for each of these. That's how I would have named it. But no bike shedding, if you don't see what I mean it's not obvious. > > C1 stubs: this really shows you tested all configurations, great! > > > > > > mutexLocker: ok. > > objectMonitor.cpp: ok > > stackValue.hpp Is this missing clearing a bug? > > In short: that change is not needed anymore. I'll remove it again. Good. Thanks for the details. > > Renaming deferred_locals to deferred_updates is good, as well as > > adding a datastructure for it. > > (Adding this data structure might be a breakout, too.) > > > > good. > > > > thread.cpp > > > > good. > > > > vframe.cpp > > > > Is this a bug in existing code? > > Makes sense. > > Depends on your definition of bug. There are no references to > vframe::is_entry_frame() in the > existing code. I would think it is a bug. So it is :) > > > > > vframe_hp.hpp > > (What stands _hp for? helper? The file should be named > compiledVFrame ...) > > > > not_global_escape_in_scope() ... > > Again, you mention escape analysis here. Comments above hold, too. > > I think it is the right name, because it is meaningful and simple. Ok, accepted ... given my understandings from above. > > > You introduce JvmtiDeferredUpdates. Good. > > > > vframe_hp.cpp > > > > Changes for JvmtiDeferredUpdates, escape state accessors, > > > > line 422: > > Would an assertion assert(!info->owner_is_scalar_replaced(), ...) hold here? > > > > > > macros.hpp > > Good. > > > > > > Test coding > > ============ > > > > compileBroker.h|cpp > > > > You introduce a third class of threads handled here and > > add a new flag to distinguish it. Before, the two kinds > > of threads were distinguished implicitly by passing in > > a compiler for compiler threads. > > The new thread kind is only used for testing in debug. > > > > make_thread: > > You could assert (comp != NULL...) to assure previous > > conditions. > > If replaced the if-statements with a switch-statement, made sure all enum- > elements are covered, and > added the assertion you suggested. > > > line 989 indentation broken > > You are referring to this block I assume: > (from > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.5/src/hots > pot/share/compiler/compileBroker.cpp.frames.html) > > 976 if (MethodFlushing) { > 977 // Initialize the sweeper thread > 978 Handle thread_oop = create_thread_oop("Sweeper thread", CHECK); > 979 jobject thread_handle = JNIHandles::make_local(THREAD, > thread_oop()); > 980 make_thread(sweeper_t, thread_handle, NULL, NULL, THREAD); > 981 } > 982 > 983 #if defined(ASSERT) && COMPILER2_OR_JVMCI > 984 if (DeoptimizeObjectsALot == 2) { > 985 // Initialize and start the object deoptimizer threads > 986 for (int thread_count = 0; thread_count < > DeoptimizeObjectsALotThreadCount; thread_count++) { > 987 Handle thread_oop = create_thread_oop("Deoptimize objects a lot > thread", CHECK); > 988 jobject thread_handle = JNIHandles::make_local(THREAD, > thread_oop()); > 989 make_thread(deoptimizer_t, thread_handle, NULL, NULL, THREAD); > 990 } > 991 } > 992 #endif // defined(ASSERT) && COMPILER2_OR_JVMCI > > I cannot really see broken indentation here. Am I looking at the wrong > location? I don't have the source version I reviewed last time any more, so I can't check. But maybe an artefact from patching ... if there were tabs jcheck would have told you, so that's not it. No problem. Best regards, Goetz. From kevin.walls at oracle.com Wed Jul 22 09:45:24 2020 From: kevin.walls at oracle.com (Kevin Walls) Date: Wed, 22 Jul 2020 10:45:24 +0100 Subject: RFR(M): 8247515: OSX pc_to_symbol() lookup does not work with core files In-Reply-To: <4c09d200-057b-e878-733a-524860453288@oracle.com> References: <092ef5a8-c973-e377-de91-b2002a1b8fe7@oracle.com> <38e1887e-a646-abc0-0b7e-53339af1d684@oracle.com> <3c31622b-831e-d46f-daa6-d641cbf755b1@oracle.com> <18f7a1d8-3802-c7da-c981-eac8cf0953eb@oracle.com> <1e9e4786-79f8-c4e0-efe7-f1a8f369d2c6@oracle.com> <4c09d200-057b-e878-733a-524860453288@oracle.com> Message-ID: <401dbc7a-dd2f-5612-74d3-3dbf6641dae9@oracle.com> Thanks Chris, yes looks good, I like that we check the library bounds before calling nearest_symbol. -- Kevin On 21/07/2020 21:05, Chris Plummer wrote: > Hi Serguei and Kevin, > > The webrev has been updated: > > http://cr.openjdk.java.net/~cjplummer/8247515/webrev.02/index.html > https://bugs.openjdk.java.net/browse/JDK-8247515 > > Two issues were addressed: > > (1) Code in symbol_for_pc() assumed the caller had first checked to > make sure that the symbol is in a library, where-as some callers > assume NULL will be returned if the symbol is not in a library. This > is the case for pstack for example, which first blindly does a pc to > symbol lookup, and only if that returns null does it then check if the > pc is in the codecache or interpreter. The logic in symbol_for_pc() > assumed that if the pc was greater than the start address of the last > library in the list, then it must be in that library. So in stack > traces the frames for compiled or interpreted pc's showed up as the > last symbol in the last library, plus some very large offset. The fix > is to now track the size of libraries so we can do a proper range check. > > (2) There are issues with finding system libraries. See [1] > JDK-8249779. Because of this I disabled support for trying to locate > them. This was done in ps_core.c, and there are "JDK-8249779" comment > references in the code where I did this. The end result of this is > that PMap for core files won't show system libraries, and PStack for > core files won't show symbols for addresses in system libraries. Note > that currently support for PMap and PStack is disabled for OSX, but I > will shortly send out a review to enable them for OSX core files as > part of the fix for [2] JDK-8248882. > > [1] https://bugs.openjdk.java.net/browse/JDK-8249779 > [2] https://bugs.openjdk.java.net/browse/JDK-8248882 > > thanks, > > Chris > > On 7/14/20 5:46 PM, serguei.spitsyn at oracle.com wrote: >> Okay, I'll wait for new webrev version to review. >> >> Thanks, >> Serguei >> >> >> On 7/14/20 17:40, Chris Plummer wrote: >>> Hi Serguie, >>> >>> Thanks for reviewing. This webrev is in limbo right now as I >>> discovered some issues that Kevin and I have been discussing off >>> line. One is that the code assumes the caller has first checked to >>> make sure that the symbol is in a library, where-as the actual >>> callers assume NULL will be returned if the symbol is not in a >>> library. The end result is that we end up returning a symbol, even >>> for address in the code cache or interpreter. So in stack traces >>> these frame show up as the last symbol in the last library, plus >>> some very large offset. I have a fix for that were now I track the >>> size of each library. But there is another issue with the code that >>> tries to discover all libraries in the core file. It misses a very >>> large number of system libraries. We understand why, but are not >>> sure of the solution. I might just change to code to only worry >>> about user libraries (JDK libs and other JNI libs). >>> >>> Some comments below also. >>> >>> On 7/14/20 4:37 PM, serguei.spitsyn at oracle.com wrote: >>>> Hi Chris, >>>> >>>> I like the suggestion from Kevin below. >>> I'm not sure which suggestion since I updated the webrev based on >>> his initial suggestion. >>>> >>>> I have a couple of minor comments so far. >>>> >>>> http://cr.openjdk.java.net/~cjplummer/8247515/webrev.01/src/jdk.hotspot.agent/macosx/native/libsaproc/libproc_impl.c.frames.html >>>> 313 if (!lib->next || lib->next->base >= addr) { >>>> I wonder if the check above has to be: >>>> 313 if (!lib->next || lib->next->base > addr) { >>> Yes, > would be better, although this goes away with my fix that >>> tracks the size of each library. >>>> >>>> >>>> http://cr.openjdk.java.net/~cjplummer/8247515/webrev.01/src/jdk.hotspot.agent/macosx/native/libsaproc/symtab.c.frames.html >>>> 417 if (offset_from_sym >= 0) { // ignore symbols that comes after >>>> "offset" >>>> Replace: comes => come >>> Ok. >>> >>> thanks, >>> >>> Chris >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 7/8/20 03:23, Kevin Walls wrote: >>>>> >>>>> >>>>> Sure -- I was thinking lowest_offset_from_sym initialising >>>>> starting at a high positive integer (that would now be PTRDIFF_MAX >>>>> I think) to save a comparison with e.g. -1, you can just check if >>>>> the new offset is less than lowest_offset_from_sym >>>>> >>>>> With the ptrdiff_t change you made, this all looks good to me >>>>> however you decide. 8-) >>>>> >>>>> >>>>> >>>>> On 07/07/2020 21:17, Chris Plummer wrote: >>>>>> Hi Kevin, >>>>>> >>>>>> Thanks for the review. Yes, that lack of initialization of >>>>>> lowest_offset_from_sym is a bug. I'm real surprised the compiler >>>>>> didn't catch it as it will be uninitialized garbage the first >>>>>> time it is referenced. Fortunately usually the eventual offset is >>>>>> very small if not 0, so probably this never prevented a proper >>>>>> match. I think there's also another bug: >>>>>> >>>>>> ?415?????? uintptr_t offset_from_sym = offset - sym->offset; >>>>>> >>>>>> "offset" is the passed in offset, essentially the address of the >>>>>> symbol we are interested in, but given as an offset from the >>>>>> start of the DSO. "sym->offset" is also an offset from the start >>>>>> of the DSO. It could be located before or after "offset". This >>>>>> means the math could result in a negative number, which when >>>>>> converted to unsigned would be a very large positive number. This >>>>>> happens whenever you check a symbol that is actually located >>>>>> after the address you are looking up. The end result is harmless, >>>>>> because it just means there's no way we will match that symbol, >>>>>> which is what you want, but it would be good to clean this up. >>>>>> >>>>>> I think what is best is to use ptrdiff_t and initialize >>>>>> lowest_offset_from_sym to -1. I've updated the webrev: >>>>>> >>>>>> http://cr.openjdk.java.net/~cjplummer/8247515/webrev.01/index.html >>>>>> >>>>>> thanks, >>>>>> >>>>>> Chris >>>>>> >>>>>> On 7/7/20 4:09 AM, Kevin Walls wrote: >>>>>>> Hi Chris, >>>>>>> >>>>>>> Yes I think this looks good. >>>>>>> >>>>>>> Question: In nearest_symbol, do we need to initialize >>>>>>> lowest_offset_from_sym to something impossibly high, as if it >>>>>>> defaults to zero we never find a better/nearer result? >>>>>>> >>>>>>> Thanks >>>>>>> Kevin >>>>>>> >>>>>>> >>>>>>> On 07/07/2020 06:10, Chris Plummer wrote: >>>>>>>> Hello, >>>>>>>> >>>>>>>> Please help review the following: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~cjplummer/8247515/webrev.00/index.html >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8247515 >>>>>>>> >>>>>>>> The CR contains a description of the issues being addressed. >>>>>>>> There is also no test for this symbol lookup support yet. It >>>>>>>> will be there after I push JDK-8247516 and? JDK-8247514, which >>>>>>>> are both blocked by the CR. >>>>>>>> >>>>>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8247516 >>>>>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8247514 >>>>>>>> >>>>>>>> thanks, >>>>>>>> >>>>>>>> Chris >>>>>>>> >>>>>> >>>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From coleen.phillimore at oracle.com Wed Jul 22 11:50:50 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 22 Jul 2020 07:50:50 -0400 Subject: RFR (M) 8249768: Move static oops and NullPointerException oops from Universe into OopStorage In-Reply-To: References: <9fb1860f-06cf-9347-27d7-16205c86c326@oracle.com> Message-ID: <16751a08-b686-9d4d-8a51-05db656b1249@oracle.com> On 7/22/20 2:32 AM, David Holmes wrote: > Hi Coleen, > > On 22/07/2020 12:19 am, coleen.phillimore at oracle.com wrote: >> >> >> On 7/20/20 10:47 PM, David Holmes wrote: >>> Hi Coleen, >>> >>> cc'ing serviceability due to SA changes. >>> >>> On 21/07/2020 6:53 am, coleen.phillimore at oracle.com wrote: >>>> Summary: Move static oops into OopStorage and make NPE oops an >>>> objArrayOop. >>>> >>>> I've broken up moving oops in Universe to OopStorage into several >>>> parts.? This change moves the global static oops. These OopHandles >>>> are not released. >>> >>> Overall looks good. But two things ... >>> >>> 1. naming >>> >>> !?? // preallocated error objects (no backtrace) >>> !?? static OopHandle??? _out_of_memory_error; >>> >>> ??? // array of preallocated error objects with backtrace >>> !?? static OopHandle _preallocated_out_of_memory_error_array; >>> >>> Both of these are pre-allocated arrays of OopHandles, differing only >>> in whether the underlying OOME has a backtrace or not. I find the >>> newly introduced _out_of_memory_error unclear in that regard. At a >>> minimum could _out_of_memory_error become _out_of_memory_errors ? >>> But ideally can we name these two arrays in a more distinguishable way? >> >> I put this code in functions next to each other because it was >> confusing.? The _out_of_memory_error array is really preallocated >> throwables, that are used to fill in the message for >> preallocated_out_of_memory_errors if there are enough of the latter >> left. >> I could rename _out_of_memory_error => >> _out_of_memory_error_throwables? ? > > That doesn't really help. As I said both of these variables are arrays > of pre-allocated OOME instances (which are throwables) - the only > difference is one set is single-use (as it can have its backtrace set) > while the other is reusable. The existing variable > > _preallocated_out_of_memory_error_array > > tells you clearly it is an array of preallocated OOME instances (but > doesn't saying anything about the backtrace or being single-use). The > problem is that that is exactly what _out_of_memory_error is as well, > but the name _out_of_memory_error doesn't convey that it is an array, > nor that anything is pre-allocated (and also nothing about backtraces > or re-usability). So given we now have two arrays of extremely similar > things, it would be clearer to give these clearer names. If we want to > keep the existing > > _preallocated_out_of_memory_error_array > > name, then the new array should indicate how it differs e.g. > > _reusable_preallocated_out_of_memory_error_array > What do you think? This confuses me more than the code does.? Which array is this? This is a terrible name for whichever one it is (I guess the original _out_of_memory_error).? I don't think it's interesting to have the name _array in it, but being plural does tell you what it is, like _out_of_memory_errors.? The implementation is a bit weird and some long name isn't going to help anyone.? The abstraction that this is _out_of_memory_errors is all anyone outside this implementation needs to know. > > I also spotted a minor nit: > > ?187 oop????? Universe::system_thread_group()????????? { return > _system_thread_group.resolve(); } > ?188 void Universe::set_system_thread_group(oop group) { > _system_thread_group = OopHandle(vm_global(), group); } > > Extra spaces after oop on L187. Ok I'll fix the spacing. Thanks, Coleen > > Thanks, > David > ----- > >>> >>> 2. SA >>> >>> You've simply deleted all the SA/vmstructs code that referenced the >>> oops that are no longer present. Does the SA not care about these >>> things and need access to them? (I don't know hence cc to >>> serviceability folk.) >> >> Yes, the SA code was never used, so I deleted it.? SA might need in >> oop inspection to add walking Universe::vm_global() OopStorage area >> to find where oops come from if there's an error but there doesn't >> seem to be any other reason for SA to use these oops. >> >> Thanks, >> Coleen >> >>> >>> Thanks, >>> David >>> ----- >>> >>>> This has been tested with tier1-3 and on also remote-build -b >>>> linux-arm32,linux-ppc64le-debug,linux-s390x-debug,linux-x64-zero. >>>> >>>> open webrev at >>>> http://cr.openjdk.java.net/~coleenp/2020/8249768.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8249768 >>>> >>>> Thanks, >>>> Coleen >> From coleen.phillimore at oracle.com Wed Jul 22 12:11:47 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 22 Jul 2020 08:11:47 -0400 Subject: [15?] RFR (S): 8249192: MonitorInfo stores raw oops across safepoints In-Reply-To: <5c33af91-446a-89ba-1a99-30c79cf2f1d9@oracle.com> References: <3f838c8a-6e70-cd94-1753-8abe269b7cdf@oracle.com> <528ade6c-059a-d9af-b0c7-74d7ac31b29a@oracle.com> <14512a57-8d4d-5199-ed16-743b5da8a10d@oracle.com> <5c33af91-446a-89ba-1a99-30c79cf2f1d9@oracle.com> Message-ID: <22541ed5-f9c1-7a71-0a9a-8e66a00b6c96@oracle.com> On 7/22/20 4:21 AM, Thomas Schatzl wrote: > Hi, > > On 22.07.20 02:42, David Holmes wrote: >> Hi Thomas, >> >> I've looked at the incremental update and I am happy with that. > > In the response to Serguei's review there were some comment updates > and new webrevs. > >> >> I also, prompted by you mentioning it, took a deeper look at the >> biased-locking code to ensure it also keeps the MonitorInfo's >> thread-confined, and to see whether the handshake versions could >> themselves be susceptible to interference from safepoints (which they >> can't as far as I can determine). And that all seems fine. > > Thanks for looking at this again in more detail. I couldn't spot > problems either, but this is not code I am proficient with. > >> As per offline discussions I know that there has been an alternate >> proposal for a completely localized fix in the stackwalker code that >> simply retrieves the list of monitors, uses the length to create the >> array, then re-retrieves the list of monitors to populate the array >> (the length of which can't change as we are dealing with the current >> thread). My only concern with that approach is the performance impact >> if we have deep stacks with lots of monitors. There is a >> microbenchmark for StackWalker in the repo: >> >> open/test/micro/org/openjdk/bench/java/lang/StackWalkBench.java >> >> but it doesn't test anything to do with monitor usage. > > Ultimately I am good with either change, as long as it's being fixed. > I initially thought about this solution too, but had the same > concerns. Stacks can be really deep particularly with some frameworks > (maybe not fully materialized) but idk the actual impact of doing the > walk twice. > > Potentially you could have different fixes for different versions. Yes.? These patches look good to me for JDK 15 and 16.? We'll open an RFE to consider the alternate patch after more performance testing for future versions, but this fixes the problem and will not be difficult to backport to JDK 11. Thank you! Coleen > > Thanks, > ? Thomas From coleen.phillimore at oracle.com Wed Jul 22 12:25:13 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 22 Jul 2020 08:25:13 -0400 Subject: RFR (M) 8249650: Optimize JNIHandle::make_local thread variable usage In-Reply-To: <4ca86ddb-8a73-783c-0b3f-e8003f7160a3@oracle.com> References: <8410d4a2-bbad-090f-55bf-88940f786781@oracle.com> <0590E210-6F23-4498-A51A-C3DAEF54B5AB@oracle.com> <6166e191-c954-70e5-5595-956a0c145d10@oracle.com> <82ac807a-1492-9ac0-570a-d08b1dc93e09@oracle.com> <4ca86ddb-8a73-783c-0b3f-e8003f7160a3@oracle.com> Message-ID: Ok, looks good to me. Colen On 7/21/20 10:46 PM, David Holmes wrote: > Hi Coleen, > > On 22/07/2020 4:01 am, coleen.phillimore at oracle.com wrote: >> >> This looks like a nice cleanup. > > Thanks for looking at this. > >> http://cr.openjdk.java.net/~dholmes/8249650/webrev.v2/src/hotspot/share/runtime/jniHandles.cpp.udiff.html >> >> >> I'm wondering why you took out the NULL return for make_local() >> without a thread argument?? Here you may call Thread::current() >> unnecessarily. >> >> ? jobject JNIHandles::make_local(oop obj) { >> - if (obj == NULL) { >> - return NULL; // ignore null handles >> - } else { >> - Thread* thread = Thread::current(); >> - assert(oopDesc::is_oop(obj), "not an oop"); >> - assert(!current_thread_in_native(), "must not be in native"); >> - return thread->active_handles()->allocate_handle(obj); >> - } >> + return make_local(Thread::current(), obj); >> ? } > > I was simply using a standard call forwarding pattern to avoid code > duplication. I suspect passing NULL is very rare so the unnecessary > Thread::current() call is not an issue. Otherwise, if not NULL, the > NULL check would happen twice (unless I keep the duplicated > implementations). > >> Beyond the scope of this fix, but it'd be cool to not have a version >> that doesn't take thread, since there may be many more callers that >> already have Thread::current(). > > Indeed! And in fact I had missed a number of these in jvm.cpp and > jni.cpp so I have fixed those. I've filed a RFE for other cases: > > https://bugs.openjdk.java.net/browse/JDK-8249837 > > Updated webrev: > > http://cr.openjdk.java.net/~dholmes/8249650/webrev.v3/ > > If this passes tier 1-3 re-testing then I plan to push. > > Thanks, > David > ----- > >> Coleen >> >> >> On 7/20/20 1:53 AM, David Holmes wrote: >>> Hi Kim, >>> >>> Thanks for looking at this. >>> >>> Updated webrev at: >>> >>> http://cr.openjdk.java.net/~dholmes/8249650/webrev.v2/ >>> >>> On 20/07/2020 3:22 pm, Kim Barrett wrote: >>>>> On Jul 20, 2020, at 12:16 AM, David Holmes >>>>> wrote: >>>>> >>>>> Subject line got truncated by accident ... >>>>> >>>>> On 20/07/2020 11:06 am, David Holmes wrote: >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8249650 >>>>>> webrev: http://cr.openjdk.java.net/~dholmes/8249650/webrev/ >>>>>> This is a simple cleanup that touches files across a number of VM >>>>>> areas - hence the cross-post. >>>>>> Whilst working on a different JNI fix I noticed that in most >>>>>> cases in jni.cpp we were using the following form of make_local: >>>>>> JNIHandles::make_local(env, obj); >>>>>> and what that form does is first extract the thread from the JNIEnv: >>>>>> JavaThread* thread = JavaThread::thread_from_jni_environment(env); >>>>>> return thread->active_handles()->allocate_handle(obj); >>>>>> but there is also another, faster, variant for when you already >>>>>> have the "thread": >>>>>> jobject JNIHandles::make_local(Thread* thread, oop obj) { >>>>>> ?? return thread->active_handles()->allocate_handle(obj); >>>>>> } >>>>>> When you look at the JNI_ENTRY wrapper (and related JVM_ENTRY, >>>>>> WB_ENTRY, UNSAFE_ENTRY etc) it has already extracted the thread >>>>>> from the JNIEnv: >>>>>> ???? JavaThread* >>>>>> thread=JavaThread::thread_from_jni_environment(env); >>>>>> and further defined: >>>>>> ???? Thread* THREAD = thread; >>>>>> so we always already have direct access to the "thread" available >>>>>> (or indirect via TRAPS), and in fact we can end up removing the >>>>>> make_local(JNIEnv* env, oop obj) variant altogether. >>>>>> Along the way I spotted some related issues with unnecessary use >>>>>> of Thread::current() when it is already available from TRAPS, and >>>>>> some other cases where we extracted the JNIEnv from a thread only >>>>>> to later extract the thread from the JNIEnv. >>>>>> Testing: tiers 1 - 3 >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/share/classfile/javaClasses.cpp >>>> ? 439???? JNIEnv *env = thread->jni_environment(); >>>> >>>> Since env is no longer used on the next line, move this down to where >>>> it is used, at line 444. >>> >>> Fixed. >>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/share/classfile/verifier.cpp >>>> ? 299?? JNIEnv *env = thread->jni_environment(); >>>> >>>> env now seems to only be used at line 320.? Move this closer. >>> >>> Fixed. >>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/share/prims/jni.cpp >>>> ? 743???? result = JNIHandles::make_local(THREAD, result_handle()); >>>> >>>> jni_PopLocalFrame is now using a mix of "thread" and "THREAD", where >>>> previously it just used "thread". Maybe this change shouldn't be made? >>>> Or can the other uses be changed to THREAD for consistency? >>> >>> "thread" and "THREAD" are interchangeable for anything expecting a >>> "Thread*" (and somewhat surprisingly a number of API's that only >>> work for JavaThreads actually take a Thread*. :( ). I had choice >>> between trying to be file-wide consistent with the make_local calls, >>> versus local-code consistent, and used THREAD as it is available in >>> both JNI_ENTRY and via TRAPS. But I can certainly make a local >>> change to "thread" for local consistency. >>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/share/prims/jvm.cpp >>>> >>>> The calls to JvmtiExport::post_vm_object_alloc have to use "thread" >>>> instead of "THREAD", even though other places nearby are using >>>> "THREAD".? That inconsistency is kind of unfortunate, but doesn't seem >>>> easily avoidable. >>> >>> Everything that uses THREAD in a JVM_ENTRY method can be changed to >>> use "thread" instead. But I'm not sure it's a consistency worth >>> pursuing at least as part of these changes (there are likely similar >>> issues with most of the touched files). >>> >>> Thanks, >>> David >>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >> From david.holmes at oracle.com Wed Jul 22 12:29:01 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 22 Jul 2020 22:29:01 +1000 Subject: 8248362: JVMTI frame operations should use Thread-Local Handshake In-Reply-To: References: <88260426-338e-03c2-0eb5-9d744e5d37b5@oracle.com> Message-ID: <64aad7e6-ecbd-c70d-15f3-42c1c0764987@oracle.com> On 22/07/2020 6:19 pm, Yasumasa Suenaga wrote: > On 2020/07/22 16:57, David Holmes wrote: >> Hi Yasumasa, >> >> On 22/07/2020 5:39 pm, Yasumasa Suenaga wrote: >>> Hi all, >>> >>> I'm working for fixing JDK-8248362, but I saw some errors on submit >>> repo. >>> Someone can share the details of >>> mach5-one-ysuenaga-JDK-8248362-20200722-0550-12850261 ? >>> >>> I wonder why build task of linux-x64 was failed because I can do it >>> on my Fedora 32 box. >> >> [2020-07-22T06:21:49,141Z] >> ./open/src/hotspot/share/prims/jvmtiThreadState.cpp:222:45: error: no >> member named 'active_handshaker' in 'JavaThread' >> [2020-07-22T06:21:49,142Z]???????????? current_thread == >> get_thread()->active_handshaker(), >> [2020-07-22T06:21:49,142Z]?????????????????????????????? ~~~~~~~~~~~~? ^ > > Thanks David! > This statement is in guarantee(), so it seems to be failed to build for > production VM. > > guarantee() call has been introduced in JDK-6471769, originally it was > assert() call. > Can we replace guarantee() to assert() at this point? or are there > methods to detect the call is happened in direct handshake without > active_handshaker()? I would replace with assert. There's no non-debug query for the handshaker. David ----- > > Thanks, > > Yasumasa > > >> David >> ----- >> >>> >>> Thanks, >>> >>> Yasumasa From david.holmes at oracle.com Wed Jul 22 12:42:29 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 22 Jul 2020 22:42:29 +1000 Subject: RFR (M) 8249768: Move static oops and NullPointerException oops from Universe into OopStorage In-Reply-To: <16751a08-b686-9d4d-8a51-05db656b1249@oracle.com> References: <9fb1860f-06cf-9347-27d7-16205c86c326@oracle.com> <16751a08-b686-9d4d-8a51-05db656b1249@oracle.com> Message-ID: On 22/07/2020 9:50 pm, coleen.phillimore at oracle.com wrote: > On 7/22/20 2:32 AM, David Holmes wrote: >> Hi Coleen, >> >> On 22/07/2020 12:19 am, coleen.phillimore at oracle.com wrote: >>> >>> >>> On 7/20/20 10:47 PM, David Holmes wrote: >>>> Hi Coleen, >>>> >>>> cc'ing serviceability due to SA changes. >>>> >>>> On 21/07/2020 6:53 am, coleen.phillimore at oracle.com wrote: >>>>> Summary: Move static oops into OopStorage and make NPE oops an >>>>> objArrayOop. >>>>> >>>>> I've broken up moving oops in Universe to OopStorage into several >>>>> parts.? This change moves the global static oops. These OopHandles >>>>> are not released. >>>> >>>> Overall looks good. But two things ... >>>> >>>> 1. naming >>>> >>>> !?? // preallocated error objects (no backtrace) >>>> !?? static OopHandle??? _out_of_memory_error; >>>> >>>> ??? // array of preallocated error objects with backtrace >>>> !?? static OopHandle _preallocated_out_of_memory_error_array; >>>> >>>> Both of these are pre-allocated arrays of OopHandles, differing only >>>> in whether the underlying OOME has a backtrace or not. I find the >>>> newly introduced _out_of_memory_error unclear in that regard. At a >>>> minimum could _out_of_memory_error become _out_of_memory_errors ? >>>> But ideally can we name these two arrays in a more distinguishable way? >>> >>> I put this code in functions next to each other because it was >>> confusing.? The _out_of_memory_error array is really preallocated >>> throwables, that are used to fill in the message for >>> preallocated_out_of_memory_errors if there are enough of the latter >>> left. >>> I could rename _out_of_memory_error => >>> _out_of_memory_error_throwables? ? >> >> That doesn't really help. As I said both of these variables are arrays >> of pre-allocated OOME instances (which are throwables) - the only >> difference is one set is single-use (as it can have its backtrace set) >> while the other is reusable. The existing variable >> >> _preallocated_out_of_memory_error_array >> >> tells you clearly it is an array of preallocated OOME instances (but >> doesn't saying anything about the backtrace or being single-use). The >> problem is that that is exactly what _out_of_memory_error is as well, >> but the name _out_of_memory_error doesn't convey that it is an array, >> nor that anything is pre-allocated (and also nothing about backtraces >> or re-usability). So given we now have two arrays of extremely similar >> things, it would be clearer to give these clearer names. If we want to >> keep the existing >> >> _preallocated_out_of_memory_error_array >> >> name, then the new array should indicate how it differs e.g. >> >> _reusable_preallocated_out_of_memory_error_array >> What do you think? > > This confuses me more than the code does.? Which array is this? This is > a terrible name for whichever one it is (I guess the original > _out_of_memory_error).? I don't think it's interesting to have the name > _array in it, but being plural does tell you what it is, like > _out_of_memory_errors. Yes at least being plural is essential to realize it is actually an array. >? The implementation is a bit weird and some long > name isn't going to help anyone.? The abstraction that this is > _out_of_memory_errors is all anyone outside this implementation needs to > know. My point, which is obviously not getting across, is that you now have two arrays of these out-of-memory-errors that are almost identical, except one is used for one purpose and one used for another, but the variable names don't give you any clue about this. But lets' just add the 's' and move on. :) Cheers, David ----- >> >> I also spotted a minor nit: >> >> ?187 oop????? Universe::system_thread_group()????????? { return >> _system_thread_group.resolve(); } >> ?188 void Universe::set_system_thread_group(oop group) { >> _system_thread_group = OopHandle(vm_global(), group); } >> >> Extra spaces after oop on L187. > > Ok I'll fix the spacing. > Thanks, > Coleen >> >> Thanks, >> David >> ----- >> >>>> >>>> 2. SA >>>> >>>> You've simply deleted all the SA/vmstructs code that referenced the >>>> oops that are no longer present. Does the SA not care about these >>>> things and need access to them? (I don't know hence cc to >>>> serviceability folk.) >>> >>> Yes, the SA code was never used, so I deleted it.? SA might need in >>> oop inspection to add walking Universe::vm_global() OopStorage area >>> to find where oops come from if there's an error but there doesn't >>> seem to be any other reason for SA to use these oops. >>> >>> Thanks, >>> Coleen >>> >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>>> This has been tested with tier1-3 and on also remote-build -b >>>>> linux-arm32,linux-ppc64le-debug,linux-s390x-debug,linux-x64-zero. >>>>> >>>>> open webrev at >>>>> http://cr.openjdk.java.net/~coleenp/2020/8249768.01/webrev >>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8249768 >>>>> >>>>> Thanks, >>>>> Coleen >>> > From coleen.phillimore at oracle.com Wed Jul 22 13:13:41 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 22 Jul 2020 09:13:41 -0400 Subject: RFR (M) 8249768: Move static oops and NullPointerException oops from Universe into OopStorage In-Reply-To: References: <9fb1860f-06cf-9347-27d7-16205c86c326@oracle.com> <16751a08-b686-9d4d-8a51-05db656b1249@oracle.com> Message-ID: <939f3762-758c-7af1-1edb-d33b89e2fa9e@oracle.com> On 7/22/20 8:42 AM, David Holmes wrote: > On 22/07/2020 9:50 pm, coleen.phillimore at oracle.com wrote: >> On 7/22/20 2:32 AM, David Holmes wrote: >>> Hi Coleen, >>> >>> On 22/07/2020 12:19 am, coleen.phillimore at oracle.com wrote: >>>> >>>> >>>> On 7/20/20 10:47 PM, David Holmes wrote: >>>>> Hi Coleen, >>>>> >>>>> cc'ing serviceability due to SA changes. >>>>> >>>>> On 21/07/2020 6:53 am, coleen.phillimore at oracle.com wrote: >>>>>> Summary: Move static oops into OopStorage and make NPE oops an >>>>>> objArrayOop. >>>>>> >>>>>> I've broken up moving oops in Universe to OopStorage into several >>>>>> parts.? This change moves the global static oops. These >>>>>> OopHandles are not released. >>>>> >>>>> Overall looks good. But two things ... >>>>> >>>>> 1. naming >>>>> >>>>> !?? // preallocated error objects (no backtrace) >>>>> !?? static OopHandle??? _out_of_memory_error; >>>>> >>>>> ??? // array of preallocated error objects with backtrace >>>>> !?? static OopHandle _preallocated_out_of_memory_error_array; >>>>> >>>>> Both of these are pre-allocated arrays of OopHandles, differing >>>>> only in whether the underlying OOME has a backtrace or not. I find >>>>> the newly introduced _out_of_memory_error unclear in that regard. >>>>> At a minimum could _out_of_memory_error become >>>>> _out_of_memory_errors ? But ideally can we name these two arrays >>>>> in a more distinguishable way? >>>> >>>> I put this code in functions next to each other because it was >>>> confusing.? The _out_of_memory_error array is really preallocated >>>> throwables, that are used to fill in the message for >>>> preallocated_out_of_memory_errors if there are enough of the latter >>>> left. >>>> I could rename _out_of_memory_error => >>>> _out_of_memory_error_throwables? ? >>> >>> That doesn't really help. As I said both of these variables are >>> arrays of pre-allocated OOME instances (which are throwables) - the >>> only difference is one set is single-use (as it can have its >>> backtrace set) while the other is reusable. The existing variable >>> >>> _preallocated_out_of_memory_error_array >>> >>> tells you clearly it is an array of preallocated OOME instances (but >>> doesn't saying anything about the backtrace or being single-use). >>> The problem is that that is exactly what _out_of_memory_error is as >>> well, but the name _out_of_memory_error doesn't convey that it is an >>> array, nor that anything is pre-allocated (and also nothing about >>> backtraces or re-usability). So given we now have two arrays of >>> extremely similar things, it would be clearer to give these clearer >>> names. If we want to keep the existing >>> >>> _preallocated_out_of_memory_error_array >>> >>> name, then the new array should indicate how it differs e.g. >>> >>> _reusable_preallocated_out_of_memory_error_array >>> What do you think? >> >> This confuses me more than the code does.? Which array is this? This >> is a terrible name for whichever one it is (I guess the original >> _out_of_memory_error).? I don't think it's interesting to have the >> name _array in it, but being plural does tell you what it is, like >> _out_of_memory_errors. > > Yes at least being plural is essential to realize it is actually an > array. > >> ? The implementation is a bit weird and some long name isn't going to >> help anyone.? The abstraction that this is _out_of_memory_errors is >> all anyone outside this implementation needs to know. > > My point, which is obviously not getting across, is that you now have > two arrays of these out-of-memory-errors that are almost identical, > except one is used for one purpose and one used for another, but the > variable names don't give you any clue about this. I actually understand this, but the suggested names don't help.? You really need to look at the code and the comments in universe.hpp to see the distinction. I don't think we can provide more illumination with long names.? Since I moved the functions next to each other, it makes more sense when one reads it. > > But lets' just add the 's' and move on. :) Thanks,? I added the 's' and fixed the formatting.? Thank you for reviewing this. Coleen > > Cheers, > David > ----- > >>> >>> I also spotted a minor nit: >>> >>> ?187 oop????? Universe::system_thread_group()????????? { return >>> _system_thread_group.resolve(); } >>> ?188 void Universe::set_system_thread_group(oop group) { >>> _system_thread_group = OopHandle(vm_global(), group); } >>> >>> Extra spaces after oop on L187. >> >> Ok I'll fix the spacing. >> Thanks, >> Coleen >>> >>> Thanks, >>> David >>> ----- >>> >>>>> >>>>> 2. SA >>>>> >>>>> You've simply deleted all the SA/vmstructs code that referenced >>>>> the oops that are no longer present. Does the SA not care about >>>>> these things and need access to them? (I don't know hence cc to >>>>> serviceability folk.) >>>> >>>> Yes, the SA code was never used, so I deleted it.? SA might need in >>>> oop inspection to add walking Universe::vm_global() OopStorage area >>>> to find where oops come from if there's an error but there doesn't >>>> seem to be any other reason for SA to use these oops. >>>> >>>> Thanks, >>>> Coleen >>>> >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>>> This has been tested with tier1-3 and on also remote-build -b >>>>>> linux-arm32,linux-ppc64le-debug,linux-s390x-debug,linux-x64-zero. >>>>>> >>>>>> open webrev at >>>>>> http://cr.openjdk.java.net/~coleenp/2020/8249768.01/webrev >>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8249768 >>>>>> >>>>>> Thanks, >>>>>> Coleen >>>> >> From coleen.phillimore at oracle.com Wed Jul 22 13:16:16 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 22 Jul 2020 09:16:16 -0400 Subject: RFR (M) 8249768: Move static oops and NullPointerException oops from Universe into OopStorage In-Reply-To: References: <9fb1860f-06cf-9347-27d7-16205c86c326@oracle.com> Message-ID: <4c59962c-1315-28c0-4763-43bd0b78ce97@oracle.com> On 7/22/20 8:44 AM, Erik ?sterlund wrote: > Hi Coleen, > > This looks good to me. It's still a bit shady to me that assignment of > OopHandles overwrites the oop*, potentially causing memory leaks (the > previous oop* is not released). > Therefore, it is implied that setters are only used once, to store a > new handle over an uninitialized handle. But we can fix the safety of > this in another patch. Thank you, Erik, and thank you for suggesting an assert in the assignment operator.? These assignments do not hit that assert but there were others that did, so I have a separate patch that I'm working on.? See https://bugs.openjdk.java.net/browse/JDK-8249822 Thank you for reviewing this. Coleen > > Thanks, > /Erik > > On 2020-07-20 22:53, coleen.phillimore at oracle.com wrote: >> Summary: Move static oops into OopStorage and make NPE oops an >> objArrayOop. >> >> I've broken up moving oops in Universe to OopStorage into several >> parts.? This change moves the global static oops.? These OopHandles >> are not released. >> >> This has been tested with tier1-3 and on also remote-build -b >> linux-arm32,linux-ppc64le-debug,linux-s390x-debug,linux-x64-zero. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8249768.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8249768 >> >> Thanks, >> Coleen > From daniel.daugherty at oracle.com Wed Jul 22 14:25:42 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 22 Jul 2020 10:25:42 -0400 Subject: [15?] RFR (S): 8249192: MonitorInfo stores raw oops across safepoints In-Reply-To: <22541ed5-f9c1-7a71-0a9a-8e66a00b6c96@oracle.com> References: <3f838c8a-6e70-cd94-1753-8abe269b7cdf@oracle.com> <528ade6c-059a-d9af-b0c7-74d7ac31b29a@oracle.com> <14512a57-8d4d-5199-ed16-743b5da8a10d@oracle.com> <5c33af91-446a-89ba-1a99-30c79cf2f1d9@oracle.com> <22541ed5-f9c1-7a71-0a9a-8e66a00b6c96@oracle.com> Message-ID: On 7/22/20 8:11 AM, coleen.phillimore at oracle.com wrote: > > > On 7/22/20 4:21 AM, Thomas Schatzl wrote: >> Hi, >> >> On 22.07.20 02:42, David Holmes wrote: >>> Hi Thomas, >>> >>> I've looked at the incremental update and I am happy with that. >> >> In the response to Serguei's review there were some comment updates >> and new webrevs. >> >>> >>> I also, prompted by you mentioning it, took a deeper look at the >>> biased-locking code to ensure it also keeps the MonitorInfo's >>> thread-confined, and to see whether the handshake versions could >>> themselves be susceptible to interference from safepoints (which >>> they can't as far as I can determine). And that all seems fine. >> >> Thanks for looking at this again in more detail. I couldn't spot >> problems either, but this is not code I am proficient with. >> >>> As per offline discussions I know that there has been an alternate >>> proposal for a completely localized fix in the stackwalker code that >>> simply retrieves the list of monitors, uses the length to create the >>> array, then re-retrieves the list of monitors to populate the array >>> (the length of which can't change as we are dealing with the current >>> thread). My only concern with that approach is the performance >>> impact if we have deep stacks with lots of monitors. There is a >>> microbenchmark for StackWalker in the repo: >>> >>> open/test/micro/org/openjdk/bench/java/lang/StackWalkBench.java >>> >>> but it doesn't test anything to do with monitor usage. >> >> Ultimately I am good with either change, as long as it's being fixed. >> I initially thought about this solution too, but had the same >> concerns. Stacks can be really deep particularly with some frameworks >> (maybe not fully materialized) but idk the actual impact of doing the >> walk twice. >> >> Potentially you could have different fixes for different versions. > > Yes.? These patches look good to me for JDK 15 and 16.? We'll open an > RFE to consider the alternate patch after more performance testing for > future versions, but this fixes the problem and will not be difficult > to backport to JDK 11. Coleen, So it looks like you, David and Serguei are comfortable with the larger patch for both JDK15 and JDK16. That's good news! It's also good news that you think this not be difficult to backport to JDK11. For a post RDP2 push to JDK15, the bug's "Fix in Release" value will have to be changed to 15 and a request for approval made. I've attached Mark R's email about the process. Follow the links and submit the approval requests. As for the alternate fix, I only came up with it as a lower risk alternative that could be pushed this late in the JDK15 cycle and would be easy to push to JDK11u-oracle. Right now it appears that we won't need it, but just in case we get push back on JDK15 approval, I'll finish the due diligence testing. Thomas, thanks for tackling this issue and for your patience in the review process. Also, thanks for the GC debugging patch! Dan > > Thank you! > Coleen >> >> Thanks, >> ? Thomas > -------------- next part -------------- An embedded message was scrubbed... From: mark.reinhold at oracle.com Subject: JDK 15 is now in Rampdown Phase Two Date: Thu, 16 Jul 2020 09:14:15 -0700 (PDT) Size: 4521 URL: From suenaga at oss.nttdata.com Wed Jul 22 14:38:32 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Wed, 22 Jul 2020 23:38:32 +0900 Subject: RFR: 8248362: JVMTI frame operations should use Thread-Local Handshake Message-ID: Hi all, Please review this change: JBS: https://bugs.openjdk.java.net/browse/JDK-8248362 webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8248362/webrev.00/ Migrate JVMTI frame operations to Thread-Local Handshakes from VM Operations. - VM_GetFrameCount - VM_GetFrameLocation They affects both GetFrameCount() and GetFrameLocation() JVMTI functions. This change has passed all tests on submit repo (mach5-one-ysuenaga-JDK-8248362-20200722-1249-12859056), and vmTestbase/nsk/{jdi,jdw,jvmti}, serviceability/{jdwp,jvmti} . Thanks, Yasumasa From suenaga at oss.nttdata.com Wed Jul 22 14:24:31 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Wed, 22 Jul 2020 23:24:31 +0900 Subject: 8248362: JVMTI frame operations should use Thread-Local Handshake In-Reply-To: <64aad7e6-ecbd-c70d-15f3-42c1c0764987@oracle.com> References: <88260426-338e-03c2-0eb5-9d744e5d37b5@oracle.com> <64aad7e6-ecbd-c70d-15f3-42c1c0764987@oracle.com> Message-ID: <14e245d4-bcee-f82e-1d5a-6c16cd8aa4a0@oss.nttdata.com> Thanks David! Your suggestion seems to work fine on submit repo. I will send review request. Yasumasa On 2020/07/22 21:29, David Holmes wrote: > On 22/07/2020 6:19 pm, Yasumasa Suenaga wrote: >> On 2020/07/22 16:57, David Holmes wrote: >>> Hi Yasumasa, >>> >>> On 22/07/2020 5:39 pm, Yasumasa Suenaga wrote: >>>> Hi all, >>>> >>>> I'm working for fixing JDK-8248362, but I saw some errors on submit repo. >>>> Someone can share the details of mach5-one-ysuenaga-JDK-8248362-20200722-0550-12850261 ? >>>> >>>> I wonder why build task of linux-x64 was failed because I can do it on my Fedora 32 box. >>> >>> [2020-07-22T06:21:49,141Z] ./open/src/hotspot/share/prims/jvmtiThreadState.cpp:222:45: error: no member named 'active_handshaker' in 'JavaThread' >>> [2020-07-22T06:21:49,142Z]???????????? current_thread == get_thread()->active_handshaker(), >>> [2020-07-22T06:21:49,142Z]?????????????????????????????? ~~~~~~~~~~~~? ^ >> >> Thanks David! >> This statement is in guarantee(), so it seems to be failed to build for production VM. >> >> guarantee() call has been introduced in JDK-6471769, originally it was assert() call. >> Can we replace guarantee() to assert() at this point? or are there methods to detect the call is happened in direct handshake without active_handshaker()? > > I would replace with assert. There's no non-debug query for the handshaker. > > David > ----- > >> >> Thanks, >> >> Yasumasa >> >> >>> David >>> ----- >>> >>>> >>>> Thanks, >>>> >>>> Yasumasa From goetz.lindenmaier at sap.com Wed Jul 22 16:21:38 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 22 Jul 2020 16:21:38 +0000 Subject: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents In-Reply-To: References: <1f8a3c7a-fa0f-b5b2-4a8a-7d3d8dbbe1b5@oracle.com> <4b56a45c-a14c-6f74-2bfd-25deaabe8201@oracle.com> <5271429a-481d-ddb9-99dc-b3f6670fcc0b@oracle.com> Message-ID: Hi Richard, Thanks for the quick reply. > > > With DeoptimizeObjectsALot enabled internal threads are started that > > > deoptimize frames and > > > objects. The number of threads started are given with > > > DeoptimizeObjectsALotThreadCountAll and > > > DeoptimizeObjectsALotThreadCountSingle. The former targets all > existing > > > threads whereas the > > > latter operates on a single thread selected round robin. > > > > > > I removed the mode where deoptimizations were performed at every nth > > > exit from the runtime. I never used it. > > > Do I get it right? You have a n:1 and a n:all test scenario. > > n:1: n threads deoptimize 1 Jana thread where n => DOALThreadCountSingle > > n:m: n threads deoptimize all Java threads where n = DOALThreadCountAll? > > Not quite. > > -XX:+DeoptimizeObjectsALot // required > -XX:DeoptimizeObjectsALotThreadCountAll=m > -XX:DeoptimizeObjectsALotThreadCountSingle=n > > Will start m+n threads. Each operating on all existing JavaThreads using > EscapeBarriers. The > difference between the 2 thread types is that one distinct EscapeBarrier > targets either just a > single thread or all exisitng threads at onece. If just one single thread is > targeted per > EscapeBarrier, then it is not always the same thread, but threads are selected > round robin. So there > will be n threads selecting independently single threads round robin per > EscapeBarrier and m threads > that target all threads in every EscapeBarrier. Ok, yes, that is how I understood it. > > > * EscapeBarrier::sync_and_suspend_one(): use a direct handshake and > > > execute it always independently > > > of is_thread_fully_suspended(). > > Is this also a performance optimization? > > Maybe a minor one. OK > > > * JavaThread::wait_for_object_deoptimization(): > > > - Bugfix: the last check of is_obj_deopt_suspend() must be /after/ the > > > safepoint check! This > > > caused issues with not walkable stacks with DeoptimizeObjectsALot. > > OK. As I understand, there was one safepoint check in the old version, > > now there is one in each iteration. I assume this is intended, right? > > Yes it is. The important thing here is (A) a safepoint check is needed /after/ > leaving a safe state > (_thread_in_native, _thread_blocked). (B) Shared variables that are modified > at safepoints or with handshakes need to be reread /after/ the safepoint check. > > BTW: I only noticed now that since JDK-8240918 JavaThreads themselves > must disarm their polling > page. Originally (before handshakes) this was done by the VM thread. With > handshakes it was done by > the thread executing the handshake op. This was changed for > OrderAccess::cross_modify_fence() where > the poll is left armed if the thread is in native and sice JDK-8240918 it is > always left armed. So > when a thread leaves a safe state (native, blocked) and there was a > handshake/vm op, it will always > call SafepointMechanism::block_if_requested_slow(), even if the > handshake/vm operation have been > processed already and everybody else is happyly executing bytecodes :) Ok. > Still (A) and (B) hold. > > > - Added limited spinning inspired by HandshakeSpinYield to fix regression in > > > microbenchmark [1] > > Ok. Nice improvement, nice catch! > > Yes. It certainly took some time to find out. > > > > > > > I refer to some more changes answering your questions and comments > inline > > > below. > > > > > > Thanks, > > > Richard. > > > > > > [1] Microbenchmark: > > > > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.6.microbe > nchmark/ > > > > > > > > > I understand you annotate at safepoints where the escape analysis > > > > finds out that an object is "better" than global escape. > > > > This are the cases where the analysis identifies optimization > > > > opportunities. These annotations are then used to deoptimize > > > > frames and the objects referenced by them. > > > > Doesn't this overestimate the optimized > > > > objects? E.g., eliminate_alloc_node has many cases where it bails > > > > out. > > > > > > Yes, the implementation is conservative, but it is comparatively simple > and > > > the additional debug > > > info is just 2 flags per safepoint. > > Thanks. It also helped that you explained to me offline that > > there are more optimizations than only lock elimination and scalar > > replacement done based on the ea information. > > The ea refines the IR graph with allows follow up optimizations > > which can not easily be tracked back to the escaping objects or > > the call sites where they do not escape. > > Thus, if there are non-global escaping objects, you have to > > deoptimize the frame. > > Did I repeat that correctly? > > Mostly, but there are also cases where deoptimization is required if and only > if ea-local objects > are passed as arguments. This is the case when values are not read directly > from a frame, but from a callee frame. Hmm, don't get this completely, but ok. > > > Accesses to instance > > > members or array elements can be optimized as well. > > You mean the compiler can/will ignore volatile or memory ordering > > requirements for non-escaping objects? Sounds reasonable to do. > > Yes, for instance. Also without volatile modifiers it will eliminate accesses. > Here is an example: > Method A has a NoEscape allocation O that is not scalar replaced. A calls > Method B, which is not > inlined. When you use your debugger to break in B, then modify a field of O, > then this modification > would have no effect without deoptimization, because the jit assumes that B > cannot modify O without > a reference to it. Yes, A can keep O in a register, while the JVMTI thread would write to the location in the stack where the local is held (if it was written back). > > > > Syncronization: looks good. I think others had a look at this before. > > > > > > > > EscapeBarrier::deoptimize_objects_internal() > > > > The method name is misleading, it is not used by > > > > deoptimize_objects(). > > > > Also, method with the same name is in Deopitmization. > > > > Proposal: deoptimize_objects_thread() ? > > > > > > Sorry, but I don't see, why it would be misleading. > > > What would be the meaning of 'deoptimize_objects_thread'? I don't > > > understand that name. > > 1. I have no idea why it's called "_internal". Because it is private? > > By the name, I would expect that EscapeBarrier::deoptimize_objects() > > calls it for some internal tasks. But it does not. > > Well, I'd say it is pretty internal, what's happening in that method. So IMHO > the suffix _internal > is a match. > > > 2. My proposal: deoptimize_objects_all_threads() iterates all threads > > and calls deoptimize_objects(_one)_thread(thread) for each of these. > > That's how I would have named it. > > But no bike shedding, if you don't see what I mean it's not obvious. > Ok. We could have a quick call, too, if you like. Ok, I think I have understood the remaining points. I'm fine with this so far. Thanks, Goetz. From daniil.x.titov at oracle.com Wed Jul 22 16:38:38 2020 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Wed, 22 Jul 2020 09:38:38 -0700 Subject: RFR: 8216324: GetClassMethods is confused by the presence of default methods in super interfaces In-Reply-To: <36efbae7-686d-ebb9-c280-489bf1920f2e@oracle.com> References: <46dafc62-bbab-9287-b398-59ca8d99d03a@oracle.com> <6471F5FB-CB4E-49A7-BCC1-1C5718E5A67D@oracle.com> <36efbae7-686d-ebb9-c280-489bf1920f2e@oracle.com> Message-ID: Thank you, Serguei and Alex, for reviewing this change. I will apply suggested corrections before pushing the fix. Best regards, Daniil From: "serguei.spitsyn at oracle.com" Date: Tuesday, July 21, 2020 at 10:53 PM To: Daniil Titov , Alex Menkov , serviceability-dev Subject: Re: RFR: 8216324: GetClassMethods is confused by the presence of default methods in super interfaces Hi Daniil, Thank you for the update! It looks good to me. Just two more minor comments and no need for another webrev you address them. 2519?? int skipped = 0;? // skip default methods Saying "overpass methods" would be more precise. ? 47???? printf("Enabled capability: maintain_original_method_order: true\n"); It is better to get rid of ": true" at the end (sorry, I missed this in my previous suggestion). Enabling capability means it is set to true. Thanks, Serguei On 7/21/20 20:25, Daniil Titov wrote: Hi Serguei and Alex, Please, review new version of the change [1]? that includes changes as Serguei suggested. 114???? default void default_method() { } // should never be seen The comment above is not clear. Should never be seen in what context? This method should not be included in the list of methods GetClassMethods returns for the sub-interface or implementing class.? I don't think we want to comment each method in the test interfaces declared in this test when they should be seen in this test and when they should not. This information already exists in getclmthd007.cpp, thus I decided to omit this comment. Please see below the output from the new test. ----------messages:(4/215)---------- command: main -agentlib:DefaultMethods DefaultMethods reason: User specified action: run main/othervm/native -agentlib:DefaultMethods DefaultMethods Mode: othervm [/othervm specified] elapsed time (seconds): 0.241 ----------configuration:(0/0)---------- ----------System.out:(3/265)---------- Reflection getDeclaredMethods returned: [public abstract java.lang.String DefaultMethods$Child.method2()] JVMTI GetClassMethods returned: [public abstract java.lang.String DefaultMethods$Child.method2()] Test passed: Got expected output from JVMTI GetClassMethods! ----------System.err:(1/15)---------- STATUS:Passed. ----------messages:(4/276)---------- command: main -agentlib:DefaultMethods=maintain_original_method_order DefaultMethods reason: User specified action: run main/othervm/native -agentlib:DefaultMethods=maintain_original_method_order DefaultMethods Mode: othervm [/othervm specified] elapsed time (seconds): 0.25 ----------configuration:(0/0)---------- ----------System.out:(4/322)---------- Enabled capability: maintain_original_method_order: true Reflection getDeclaredMethods returned: [public abstract java.lang.String DefaultMethods$Child.method2()] JVMTI GetClassMethods returned: [public abstract java.lang.String DefaultMethods$Child.method2()] Test passed: Got expected output from JVMTI GetClassMethods! ----------System.err:(1/15)---------- STATUS:Passed. [1] http://cr.openjdk.java.net/~dtitov/8216324/webrev.02/ Thanks, Daniil From: "serguei.spitsyn at oracle.com" Date: Monday, July 20, 2020 at 6:48 PM To: Alex Menkov , Daniil Titov , serviceability-dev Subject: Re: RFR: 8216324: GetClassMethods is confused by the presence of default methods in super interfaces Hi Daniil, The fix looks pretty good to me. Just minor comments. http://cr.openjdk.java.net/~dtitov/8216324/webrev.01/src/hotspot/share/prims/jvmtiEnv.cpp.frames.html 2519?? int skipped = 0;? // skip default methods from superinterface (see JDK-8216324) You can say just: // skip overpass methods There is probably no need to list the bug number. 2523???? // Depending on can_maintain_original_method_order capability 2524???? // use the original method ordering indices stored in the class, so we can emit 2525???? // jmethodIDs in the order they appeared in the class file 2526???? // or just copy in any order Could you, please re-balance the comment a little bit? Also, the comment has to be terminated with a dot. Replace: "or just copy in any order" => "or just copy in current order". http://cr.openjdk.java.net/~dtitov/8216324/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetClassMethods/getclmthd007.java.frames.html 114???? default void default_method() { } // should never be seen The comment above is not clear. Should never be seen in what context? 117 interface OuterInterface1? extends DefaultInterface { An extra space before "extends". http://cr.openjdk.java.net/~dtitov/8216324/webrev.01/test/hotspot/jtreg/serviceability/jvmti/GetClassMethods/InterfaceDefMethods.java.html I like the test simplicity. Default methods are always in interfaces. I'd suggest to rename the test to something like: DefaultMethods.java or OverpassMethods.java. http://cr.openjdk.java.net/~dtitov/8216324/webrev.01/test/hotspot/jtreg/serviceability/jvmti/GetClassMethods/libInterfaceDefMethods.cpp.html ? 44?? if (options != NULL && strcmp(options, "maintain_original_method_order") == 0) { ? 45???? printf("maintain_original_method_order: true\n"); ? ... ? 54?? } else { ? 55???? printf("maintain_original_method_order: false\n"); I'd suggest to remove the lines 54 and 55 and adjust the line 45: printf("Enabled capability: maintain_original_method_order: true\n"); ? 88???? jobject m = env->ToReflectedMethod(klass, methods[i], (modifiers & 8) == 8); It is better to replace 8 with a symbolic constant. Thanks, Serguei On 7/20/20 16:57, Alex Menkov wrote: Looks good to me :) Thanks for handling this. --alex On 07/20/2020 12:00, Daniil Titov wrote: Please review change [1] that fixes GetClassMethods behavior in cases if a default method is present in a super interface. Currently for such cases the information GetClassMethods returns for the sub-interface or implementing class incorrectly includes inherited not default methods. The fix ( thanks to Alex for providing this patch) ensures that overpass methods are filtered out in the returns. The fix also applies a change in the existing test as David suggested in the comments to the issue [2]. Testing: Mach5 tier1-tier3 tests successfully passed. [1] http://cr.openjdk.java.net/~dtitov/8216324/webrev.01/ [2] https://bugs.openjdk.java.net/browse/JDK-8216324 Thank you, Daniil -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.daugherty at oracle.com Wed Jul 22 17:04:30 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 22 Jul 2020 13:04:30 -0400 Subject: [15?] RFR (S): 8249192: MonitorInfo stores raw oops across safepoints In-Reply-To: <07904b7e-8d40-8c9e-acc1-c713c1588b11@oracle.com> References: <3f838c8a-6e70-cd94-1753-8abe269b7cdf@oracle.com> <528ade6c-059a-d9af-b0c7-74d7ac31b29a@oracle.com> <14512a57-8d4d-5199-ed16-743b5da8a10d@oracle.com> <893f862f-093a-5168-e554-1f4ffe129aea@oracle.com> <07904b7e-8d40-8c9e-acc1-c713c1588b11@oracle.com> Message-ID: > jdk15: > > http://cr.openjdk.java.net/~tschatzl/8249192/webrev.jdk15.2/ (full) src/hotspot/share/prims/jvmtiEnvBase.cpp ??? old L1029: ????? ResourceMark rm; ??????? It's not clear (to me anyway) why this ResourceMark is removed. ??????? Update: I saw the discussion of "ResourceMark rm" in JDK15 versus ??????? "ResourceMark rm(current_thread)" in JDK16, but that doesn't tell ??????? me why it was necessary to remove that ResourceMark. src/hotspot/share/prims/stackwalk.cpp ??? L291: ??? ResourceMark rm; ??? L292: ??? HandleMark hm; ??????? Since there's a TRAPS parameter, these should be 'rm(THREAD)' and ??????? 'hm(THREAD)'. src/hotspot/share/runtime/biasedLocking.cpp ??? No comments. src/hotspot/share/runtime/deoptimization.cpp ??? No comments. src/hotspot/share/runtime/vframe.cpp ??? L461: ? _lock? = lock; ??????? nit - extra space before '='. src/hotspot/share/runtime/vframe.hpp ??? L32: #include "runtime/handles.inline.hpp" ??????? nit - new include is out of order; should be after frame.hpp. src/hotspot/share/runtime/vframeArray.cpp ??? No comments. src/hotspot/share/runtime/vframe_hp.cpp ??? Skipped - no changes. src/hotspot/share/services/threadService.cpp ??? No comments. > jdk16: > > http://cr.openjdk.java.net/~tschatzl/8249192/webrev.2/ (full) Same comments as for the JDK15 version. I also compared the two patches using jfilemerge. Most of what I have above are nits so I'm good with both of these patches. I also looked for lifetime issues with the new ResourceMarks and HandleMarks, but I don't see any issues. Dan On 7/22/20 4:14 AM, Thomas Schatzl wrote: > Hi Serguei, > > ? thanks for your review. > > On 22.07.20 04:13, serguei.spitsyn at oracle.com wrote: >> Hi Thomas, >> >> The fix looks okay to me. >> The 15 fix is identical to 16. > > ? no :) It is very subtle. As mentioned, in jvmtiEnvBase.cpp in the > original code, in jdk15 we had: > > line 1029:?????? ResourceMark rm; > > in jdk16: > > line 1008:?????? ResourceMark rm(current_thread); > > Both lines were ultimately removed with this recent change, but still > cause merge errors if you port the patch from one version to the other. > Sorry if that has been confusing. > >> >> This file finally was not changed: >> src/hotspot/share/runtime/vframe_hp.cpp . > > This is an artifact of webrev in conjunction with mq because the > original change had some modifications which were retracted in the -1 > webrev there. There will not be any change in any push for that file. > >> Several files need a copyright comment update. > > Provided new webrevs with only comment updates below. Did not do new > testing just for these comment updates though. > >> >> What tests do you run? >> We need at least tier3 to make sure there are no regressions in the >> JVMTI and JDI tests. > > The jdk15 .1 patch has been tested with 1.2k of that LocalsAndOperands > test with the options that originally reproduced it in 1/100 cases. > > hs-tier1-5, no failures at all. > > jdk16 has had testing with the .0 patch doing 1.8k of > LocalsAndOperands.java, tier1-5, and tier8 with JDK-8249676 reinstated > that earlier caused lots of issues there (and none without). > Since there has been no substantial change in how the patch works, > only some refactoring, so I think these are still valid. > > See the internal comments in the CR for links. > > New webrevs: > > jdk15: > > http://cr.openjdk.java.net/~tschatzl/8249192/webrev.jdk15.2/ (full) > http://cr.openjdk.java.net/~tschatzl/8249192/webrev.jdk15.1_to_2/ (diff) > > jdk16: > > http://cr.openjdk.java.net/~tschatzl/8249192/webrev.2/ (full) > http://cr.openjdk.java.net/~tschatzl/8249192/webrev.1_to_2/ (diff) > > Thanks, > ? Thomas From richard.reingruber at sap.com Wed Jul 22 20:18:23 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Wed, 22 Jul 2020 20:18:23 +0000 Subject: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents In-Reply-To: References: <1f8a3c7a-fa0f-b5b2-4a8a-7d3d8dbbe1b5@oracle.com> <4b56a45c-a14c-6f74-2bfd-25deaabe8201@oracle.com> <5271429a-481d-ddb9-99dc-b3f6670fcc0b@oracle.com> Message-ID: Hi Goetz, > > I'll answer to the obvious things in this mail now. > > I'll go through the code thoroughly again and write > > a review of my findings thereafter. > As promised a detailed walk-throug, but without any major findings: > c1_IR.hpp: ok > ci_Env.h|cpp: ok > compiledMethod.cpp, nmethod.cpp: ok > debugInfoRec.h|cpp: ok > scopeDesc.h|cpp ok > compileBroker.h|cpp: > Maybe a bit of documentation how and why you start > the threads? I had expected there are two test > scenarios run after each other, but now I understand 'Single' > and 'All' run simultaneously. Well, this really is a stress test! > Also good the two variants of depotimization are > stressed against each other. > Besides that really nice it's all in one place. Done. > rootResolver.cpp: ok > jvmciCodeInstaller.cpp: ok > c2compiler.cpp: The essence of this change! Just one line :) > Great! :) > callnode.hpp ok > escape.h|cpp ok > macro.cpp > I was not that happy with the names saying not_global_escape > and similar. I now agreed you have to use the terms of the escape > analysis (NoEscape ArgEscape= throughout the runtime code. I'm still not happy with > the 'not' in the term, I always try to expand the name to some > sentence with a negated verb, but it makes no sense. > For example, "has_not_global_escape_in_scope" expands to > "Hasn't a global escape in its scope." in my thinking, which makes > no sense. You probably mean > "Has not-global escape in its scope." or "Has {ArgEscape|NoEscape} > in its scope." > C2 is using the word "non" in this context, e.g., here > alloc->is_non_escaping. There is also ConnectionGraph::not_global_escape() > non obviously negates the adjective 'global', > non-global or nonglobal even is a English term I find in the > net. > So what about "has_non_global_escape_in_scope?" And what about has_ea_local_in_scope? > matcher.cpp ok > output.cpp:1071 > Please break the long line. Done. > jvmtiCodeBlobEvents.cpp ok > jvmtiEnv.cpp > MaxJavaStackTraceDepth is only documented to affect > the exceptions stack trace depth, not to limit jvmti > operations. Therefore I wondered why it is used here. > Non of your business, but the flag should > document this in globals.hpp, too. > Does jvmti specify that the same limits are used ...? > ok on your side. I don't know and didn't find anything in a quick search. > jvmtiEnvBase.cpp ok > jvmtiImpl.h|cpp ok > jvmtiTagMap.cpp ok > whitebox.cpp ok > deoptimization.cpp > line 177: Please break line > line 246, 281: Please break line > 1578, 1583, 1589, 1632, 1649, 1651 Break line > 1651: You use 'non'-terms, too: non-escaping :) I know :) At least here it is wrong I'd say. "...has to be a not escaping obj..." sounds better (hopefully not only to my german ears). > 2805, 2929, 2946ff, break lines > deoptimization.hpp > 158, 174, 176 ... I would break lines too, but here you are in > good company :) Done. > globals.hpp ok > mutexLocker.h|cpp ok > objectMonitor.cpp ok > thread.cpp > 2631 typo: sapfepont --> safepoint Done. > thread.hpp ok > thread.inline.hpp ok > vframe.cpp ok > vframe_hp.cpp 458ff break lines > vframe_hp.hpp ok > macros.hpp ok > TEST.ROOT ok > WhiteBox.java ok > IterateHeapWithEscapeAnalysisEnabled.java > line 415: > msg("wait until target thread has set testMethod_result"); > while (testMethod_result == 0) { > Thread.sleep(50); > } > Might the test run into timeouts at this place? > The field is volatile, i.e. it will be reloaded > in each iteration. But will dontinline_testMethod > write it back to main memory in time? You mean, the test could hang in that loop for a couple of minutes? I don't think so. There are cache coherence protocols in place which will invalidate stale data very timely. > libIterateHeapWithEscapeAnalysisEnabled.c ok > EATests.java > This is a very elaborate test. > I found a row of test cases illustrating issues > we talked about before. Really helpful! > 1311: TypeO materialize -> materialized Found and fix typo at line 1369. (Probably the cursor was on 1311 and your eyes on 1369 ;)) > 1640: setting local variable i triggers always deoptimization > --> setting local variable i always triggers deoptimization Fixed. > 2176: dontinline_calee --> dontinline_callee > 2510: poping --> popping ... but I'm not sure here. Done. > https://www.urbandictionary.com/define.php?term=poping > poping > Drinking large amounts of Dextromethorphan Hydrobromide (DXM)based cough syrup, and then embarking on an adventure while wandering around neighborhoods or parks all night. This is usually done while listening to Punk rock music from a portable jambox. > ;) > Don?t do it! ?? OMG! How come you know?! ;) > EATestsJVMTI.java > I think you can just copy this test description into the other > test. You can have two @test comments, they will be treated > as separate tests. The @requires will be evaluated accordingly. > For an example see > test/hotspot/jtreg/runtime/exceptionMsgs/NullPointerException/NullPointerExceptionTest.java > which has two different compile setups for the test class (-g). Done. > so, that's it for reading code ... > Some general remarks, maybe a bit picky ...: > I think you could use less commas ',' in comments. > As I understand, you need a comma if the relative > sentence is at the beginning, but not if it is at > the end: > If Corona is over, I go to the office. > but > I go to the office if Corona is over. That seem's to be correct except "If Corona is over" isn't a relative sentence but a conditional sentence, isn't it? The general rule seems to be: the subordinate clause is separated with a comma from a following main clause. No comma separation is needed if the subordinate clause follows the main clause. Thanks, that's a lesson I learned! > I think the same holds for 'because', 'while' etc. > E.g., jvmtiEnvBase.cpp:1313, jvmtiImpl.cpp:646ff, > vframe_hp.hpp 104ff Ok. I've removed quite a lot of the occurrances. > Also, I like full sentences in comments. > Especially for me as foreign speaker, this makes > things much more clear. I.e., I try to make it > a real sentence with articles, capitalized and a > dot at the end if there is a subject and a verb > in first place. > E.g., jvmtiEnvBase.cpp:1327 Are you referring to the following? (from http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.6/src/hotspot/share/prims/jvmtiEnvBase.cpp.frames.html) 1326 1327 // If the frame is a compiled one, need to deoptimize it. 1328 if (vf->is_compiled_frame()) { This line 1327 is preexisting. > In many places, your comments read really > well but some are quite abbreviated I think. Yeah, but not only because I'm lazy... It is the style that I prefer and I think it matches the surrounding code quite well. > E.g. thread.cpp:2601 is an example where a simple > 'a' helps a lot. > "Single deoptimization is typically very short." > I would add 'A': "A single deoptimization is typically very short (fast?)." > An other meaning of the comment I first considered is this: > "Single deoptimization is typically very short, all_threads deoptimization takes longer" > having in mind the functions > EscapeBarries::deoptimize_objects_all_threads() > and > EscapeBarries::deoptimize_objects() doing a single thread. > German with it's compound nouns is helpful here :) > Einzeldeoptimierung <--> eine einzelne Deoptimierung I've added the 'A' and I'll try to use complete sentences in the future. The telegram style has advantages, too, though ;) Thanks! Cheers, Richard. -----Original Message----- From: Lindenmaier, Goetz Sent: Freitag, 17. Juli 2020 14:31 To: Lindenmaier, Goetz ; Reingruber, Richard ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents Hi Richard, > I'll answer to the obvious things in this mail now. > I'll go through the code thoroughly again and write > a review of my findings thereafter. As promised a detailed walk-throug, but without any major findings: c1_IR.hpp: ok ci_Env.h|cpp: ok compiledMethod.cpp, nmethod.cpp: ok debugInfoRec.h|cpp: ok scopeDesc.h|cpp ok compileBroker.h|cpp: Maybe a bit of documentation how and why you start the threads? I had expected there are two test scenarios run after each other, but now I understand 'Single' and 'All' run simultaneously. Well, this really is a stress test! Also good the two variants of depotimization are stressed against each other. Besides that really nice it's all in one place. rootResolver.cpp: ok jvmciCodeInstaller.cpp: ok c2compiler.cpp: The essence of this change! Just one line :) Great! callnode.hpp ok escape.h|cpp ok macro.cpp I was not that happy with the names saying not_global_escape and similar. I now agreed you have to use the terms of the escape analysis (NoEscape ArgEscape= throughout the runtime code. I'm still not happy with the 'not' in the term, I always try to expand the name to some sentence with a negated verb, but it makes no sense. For example, "has_not_global_escape_in_scope" expands to "Hasn't a global escape in its scope." in my thinking, which makes no sense. You probably mean "Has not-global escape in its scope." or "Has {ArgEscape|NoEscape} in its scope." C2 is using the word "non" in this context, e.g., here alloc->is_non_escaping. non obviously negates the adjective 'global', non-global or nonglobal even is a English term I find in the net. So what about "has_non_global_escape_in_scope?" matcher.cpp ok output.cpp:1071 Please break the long line. jvmtiCodeBlobEvents.cpp ok jvmtiEnv.cpp MaxJavaStackTraceDepth is only documented to affect the exceptions stack trace depth, not to limit jvmti operations. Therefore I wondered why it is used here. Non of your business, but the flag should document this in globals.hpp, too. Does jvmti specify that the same limits are used ...? ok on your side. jvmtiEnvBase.cpp ok jvmtiImpl.h|cpp ok jvmtiTagMap.cpp ok whitebox.cpp ok deoptimization.cpp line 177: Please break line line 246, 281: Please break line 1578, 1583, 1589, 1632, 1649, 1651 Break line 1651: You use 'non'-terms, too: non-escaping :) 2805, 2929, 2946ff, break lines deoptimization.hpp 158, 174, 176 ... I would break lines too, but here you are in good company :) globals.hpp ok mutexLocker.h|cpp ok objectMonitor.cpp ok thread.cpp 2631 typo: sapfepont --> safepoint thread.hpp ok thread.inline.hpp ok vframe.cpp ok vframe_hp.cpp 458ff break lines vframe_hp.hpp ok macros.hpp ok TEST.ROOT ok WhiteBox.java ok IterateHeapWithEscapeAnalysisEnabled.java line 415: msg("wait until target thread has set testMethod_result"); while (testMethod_result == 0) { Thread.sleep(50); } Might the test run into timeouts at this place? The field is volatile, i.e. it will be reloaded in each iteration. But will dontinline_testMethod write it back to main memory in time? libIterateHeapWithEscapeAnalysisEnabled.c ok EATests.java This is a very elaborate test. I found a row of test cases illustrating issues we talked about before. Really helpful! 1311: TypeO materialize -> materialized 1640: setting local variable i triggers always deoptimization --> setting local variable i always triggers deoptimization 2176: dontinline_calee --> dontinline_callee 2510: poping --> popping ... but I'm not sure here. https://www.urbandictionary.com/define.php?term=poping poping Drinking large amounts of Dextromethorphan Hydrobromide (DXM)based cough syrup, and then embarking on an adventure while wandering around neighborhoods or parks all night. This is usually done while listening to Punk rock music from a portable jambox. ;) Don?t do it! ?? EATestsJVMTI.java I think you can just copy this test description into the other test. You can have two @test comments, they will be treated as separate tests. The @requires will be evaluated accordingly. For an example see test/hotspot/jtreg/runtime/exceptionMsgs/NullPointerException/NullPointerExceptionTest.java which has two different compile setups for the test class (-g). so, that's it for reading code ... Some general remarks, maybe a bit picky ...: I think you could use less commas ',' in comments. As I understand, you need a comma if the relative sentence is at the beginning, but not if it is at the end: If Corona is over, I go to the office. but I go to the office if Corona is over. I think the same holds for 'because', 'while' etc. E.g., jvmtiEnvBase.cpp:1313, jvmtiImpl.cpp:646ff, vframe_hp.hpp 104ff Also, I like full sentences in comments. Especially for me as foreign speaker, this makes things much more clear. I.e., I try to make it a real sentence with articles, capitalized and a dot at the end if there is a subject and a verb in first place. E.g., jvmtiEnvBase.cpp:1327 In many places, your comments read really well but some are quite abbreviated I think. E.g. thread.cpp:2601 is an example where a simple 'a' helps a lot. "Single deoptimization is typically very short." I would add 'A': "A single deoptimization is typically very short (fast?)." An other meaning of the comment I first considered is this: "Single deoptimization is typically very short, all_threads deoptimization takes longer" having in mind the functions EscapeBarries::deoptimize_objects_all_threads() and EscapeBarries::deoptimize_objects() doing a single thread. German with it's compound nouns is helpful here :) Einzeldeoptimierung <--> eine einzelne Deoptimierung Best regards, Goetz. From richard.reingruber at sap.com Wed Jul 22 20:53:19 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Wed, 22 Jul 2020 20:53:19 +0000 Subject: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents In-Reply-To: References: <1f8a3c7a-fa0f-b5b2-4a8a-7d3d8dbbe1b5@oracle.com> <4b56a45c-a14c-6f74-2bfd-25deaabe8201@oracle.com> <5271429a-481d-ddb9-99dc-b3f6670fcc0b@oracle.com> Message-ID: Hi Goetz, > Thanks for the quick reply. Yes, this time it didn't take that long... [... snip ...] > > > > > I understand you annotate at safepoints where the escape analysis > > > > > finds out that an object is "better" than global escape. > > > > > This are the cases where the analysis identifies optimization > > > > > opportunities. These annotations are then used to deoptimize > > > > > frames and the objects referenced by them. > > > > > Doesn't this overestimate the optimized > > > > > objects? E.g., eliminate_alloc_node has many cases where it bails > > > > > out. > > > > > > > > Yes, the implementation is conservative, but it is comparatively simple > > and > > > > the additional debug > > > > info is just 2 flags per safepoint. > > > Thanks. It also helped that you explained to me offline that > > > there are more optimizations than only lock elimination and scalar > > > replacement done based on the ea information. > > > The ea refines the IR graph with allows follow up optimizations > > > which can not easily be tracked back to the escaping objects or > > > the call sites where they do not escape. > > > Thus, if there are non-global escaping objects, you have to > > > deoptimize the frame. > > > Did I repeat that correctly? > > > > Mostly, but there are also cases where deoptimization is required if and only > > if ea-local objects > > are passed as arguments. This is the case when values are not read directly > > from a frame, but from a callee frame. > Hmm, don't get this completely, but ok. Let C be a callee frame of B which is a callee of A. If you use JVMTI to read an object reference from a local variable of C then the implementation of JDK-8227745 deoptimizes A if it passes any ea-local as argument, because the reference could be ea-local in A and there might be optimizations that are invalid after the escape state change. > > > > Accesses to instance > > > > members or array elements can be optimized as well. > > > You mean the compiler can/will ignore volatile or memory ordering > > > requirements for non-escaping objects? Sounds reasonable to do. > > > > Yes, for instance. Also without volatile modifiers it will eliminate accesses. > > Here is an example: > > Method A has a NoEscape allocation O that is not scalar replaced. A calls > > Method B, which is not > > inlined. When you use your debugger to break in B, then modify a field of O, > > then this modification > > would have no effect without deoptimization, because the jit assumes that B > > cannot modify O without > > a reference to it. > Yes, A can keep O in a register, while the JVMTI thread would write to > the location in the stack where the local is held (if it was written back). Not quite. It is the value of the field of O that is in a register not the reference to O itself. The agent changes the field's value in the /java heap/ (remember: O is _not_ scalar replaced), but the fields value is not reloaded after return from B. > > > > > Syncronization: looks good. I think others had a look at this before. > > > > > > > > > > EscapeBarrier::deoptimize_objects_internal() > > > > > The method name is misleading, it is not used by > > > > > deoptimize_objects(). > > > > > Also, method with the same name is in Deopitmization. > > > > > Proposal: deoptimize_objects_thread() ? > > > > > > > > Sorry, but I don't see, why it would be misleading. > > > > What would be the meaning of 'deoptimize_objects_thread'? I don't > > > > understand that name. > > > 1. I have no idea why it's called "_internal". Because it is private? > > > By the name, I would expect that EscapeBarrier::deoptimize_objects() > > > calls it for some internal tasks. But it does not. > > > > Well, I'd say it is pretty internal, what's happening in that method. So IMHO > > the suffix _internal > > is a match. > > > > > 2. My proposal: deoptimize_objects_all_threads() iterates all threads > > > and calls deoptimize_objects(_one)_thread(thread) for each of these. > > > That's how I would have named it. > > > But no bike shedding, if you don't see what I mean it's not obvious. > > Ok. We could have a quick call, too, if you like. > Ok, I think I have understood the remaining points. I'm fine with this > so far. Thanks again and best regards, Richard. -----Original Message----- From: Lindenmaier, Goetz Sent: Mittwoch, 22. Juli 2020 18:22 To: Reingruber, Richard ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents Hi Richard, Thanks for the quick reply. > > > With DeoptimizeObjectsALot enabled internal threads are started that > > > deoptimize frames and > > > objects. The number of threads started are given with > > > DeoptimizeObjectsALotThreadCountAll and > > > DeoptimizeObjectsALotThreadCountSingle. The former targets all > existing > > > threads whereas the > > > latter operates on a single thread selected round robin. > > > > > > I removed the mode where deoptimizations were performed at every nth > > > exit from the runtime. I never used it. > > > Do I get it right? You have a n:1 and a n:all test scenario. > > n:1: n threads deoptimize 1 Jana thread where n => DOALThreadCountSingle > > n:m: n threads deoptimize all Java threads where n = DOALThreadCountAll? > > Not quite. > > -XX:+DeoptimizeObjectsALot // required > -XX:DeoptimizeObjectsALotThreadCountAll=m > -XX:DeoptimizeObjectsALotThreadCountSingle=n > > Will start m+n threads. Each operating on all existing JavaThreads using > EscapeBarriers. The > difference between the 2 thread types is that one distinct EscapeBarrier > targets either just a > single thread or all exisitng threads at onece. If just one single thread is > targeted per > EscapeBarrier, then it is not always the same thread, but threads are selected > round robin. So there > will be n threads selecting independently single threads round robin per > EscapeBarrier and m threads > that target all threads in every EscapeBarrier. Ok, yes, that is how I understood it. > > > * EscapeBarrier::sync_and_suspend_one(): use a direct handshake and > > > execute it always independently > > > of is_thread_fully_suspended(). > > Is this also a performance optimization? > > Maybe a minor one. OK > > > * JavaThread::wait_for_object_deoptimization(): > > > - Bugfix: the last check of is_obj_deopt_suspend() must be /after/ the > > > safepoint check! This > > > caused issues with not walkable stacks with DeoptimizeObjectsALot. > > OK. As I understand, there was one safepoint check in the old version, > > now there is one in each iteration. I assume this is intended, right? > > Yes it is. The important thing here is (A) a safepoint check is needed /after/ > leaving a safe state > (_thread_in_native, _thread_blocked). (B) Shared variables that are modified > at safepoints or with handshakes need to be reread /after/ the safepoint check. > > BTW: I only noticed now that since JDK-8240918 JavaThreads themselves > must disarm their polling > page. Originally (before handshakes) this was done by the VM thread. With > handshakes it was done by > the thread executing the handshake op. This was changed for > OrderAccess::cross_modify_fence() where > the poll is left armed if the thread is in native and sice JDK-8240918 it is > always left armed. So > when a thread leaves a safe state (native, blocked) and there was a > handshake/vm op, it will always > call SafepointMechanism::block_if_requested_slow(), even if the > handshake/vm operation have been > processed already and everybody else is happyly executing bytecodes :) Ok. > Still (A) and (B) hold. > > > - Added limited spinning inspired by HandshakeSpinYield to fix regression in > > > microbenchmark [1] > > Ok. Nice improvement, nice catch! > > Yes. It certainly took some time to find out. > > > > > > > I refer to some more changes answering your questions and comments > inline > > > below. > > > > > > Thanks, > > > Richard. > > > > > > [1] Microbenchmark: > > > > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.6.microbe > nchmark/ > > > > > > > > > I understand you annotate at safepoints where the escape analysis > > > > finds out that an object is "better" than global escape. > > > > This are the cases where the analysis identifies optimization > > > > opportunities. These annotations are then used to deoptimize > > > > frames and the objects referenced by them. > > > > Doesn't this overestimate the optimized > > > > objects? E.g., eliminate_alloc_node has many cases where it bails > > > > out. > > > > > > Yes, the implementation is conservative, but it is comparatively simple > and > > > the additional debug > > > info is just 2 flags per safepoint. > > Thanks. It also helped that you explained to me offline that > > there are more optimizations than only lock elimination and scalar > > replacement done based on the ea information. > > The ea refines the IR graph with allows follow up optimizations > > which can not easily be tracked back to the escaping objects or > > the call sites where they do not escape. > > Thus, if there are non-global escaping objects, you have to > > deoptimize the frame. > > Did I repeat that correctly? > > Mostly, but there are also cases where deoptimization is required if and only > if ea-local objects > are passed as arguments. This is the case when values are not read directly > from a frame, but from a callee frame. Hmm, don't get this completely, but ok. > > > Accesses to instance > > > members or array elements can be optimized as well. > > You mean the compiler can/will ignore volatile or memory ordering > > requirements for non-escaping objects? Sounds reasonable to do. > > Yes, for instance. Also without volatile modifiers it will eliminate accesses. > Here is an example: > Method A has a NoEscape allocation O that is not scalar replaced. A calls > Method B, which is not > inlined. When you use your debugger to break in B, then modify a field of O, > then this modification > would have no effect without deoptimization, because the jit assumes that B > cannot modify O without > a reference to it. Yes, A can keep O in a register, while the JVMTI thread would write to the location in the stack where the local is held (if it was written back). > > > > Syncronization: looks good. I think others had a look at this before. > > > > > > > > EscapeBarrier::deoptimize_objects_internal() > > > > The method name is misleading, it is not used by > > > > deoptimize_objects(). > > > > Also, method with the same name is in Deopitmization. > > > > Proposal: deoptimize_objects_thread() ? > > > > > > Sorry, but I don't see, why it would be misleading. > > > What would be the meaning of 'deoptimize_objects_thread'? I don't > > > understand that name. > > 1. I have no idea why it's called "_internal". Because it is private? > > By the name, I would expect that EscapeBarrier::deoptimize_objects() > > calls it for some internal tasks. But it does not. > > Well, I'd say it is pretty internal, what's happening in that method. So IMHO > the suffix _internal > is a match. > > > 2. My proposal: deoptimize_objects_all_threads() iterates all threads > > and calls deoptimize_objects(_one)_thread(thread) for each of these. > > That's how I would have named it. > > But no bike shedding, if you don't see what I mean it's not obvious. > Ok. We could have a quick call, too, if you like. Ok, I think I have understood the remaining points. I'm fine with this so far. Thanks, Goetz. From coleen.phillimore at oracle.com Wed Jul 22 20:55:30 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 22 Jul 2020 16:55:30 -0400 Subject: RFR (S) 8249822: SymbolPropertyTable creates an extra OopHandle per entry Message-ID: <84643f6c-0e05-061e-b6e3-a38d09ed973a@oracle.com> Summary: Add an assert to OopHandle assigment operator to catch leaking OopHandles, and fix code accordingly. There are some jvmtiRedefineClasses.cpp changes here - basically, I moved the RedefineVerifyMark and comments into jvmtiRedefineClasses.cpp because I needed a Handle. Ran tier1-6 tests and tier1 on all Oracle platforms. open webrev at http://cr.openjdk.java.net/~coleenp/2020/8249822.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8249822 Thanks, Coleen From hohensee at amazon.com Wed Jul 22 22:48:09 2020 From: hohensee at amazon.com (Hohensee, Paul) Date: Wed, 22 Jul 2020 22:48:09 +0000 Subject: RFR(L): 8215624: add parallel heap inspection support for jmap histo(G1)(Internet mail) Message-ID: Just small things. heapInspection.cpp: In ParHeapInspectTask::work, remove the final return statement and fix the following ?}? indent. I.e., replace + Atomic::store(&_success, false); + return; + } with + Atomic::store(&_success, false); + } In HeapInspection::heap_inspection, missed_count should be a uint to match other missed_count declarations, and should be initialized to the result of populate_table() rather than separately to 0. attachListener.cpp: In heap_inspection, initial_processor_count returns an int, so cast its result to a uint. Similarly, parse_uintx returns a uintx, so cast its result (num) to uint when assigning to parallel_thread_num. BasicJMapTest.java: I took the liberty of refactoring testHisto*/histoToFile/testDump*/dump to remove redundant interposition methods and make histoToFile and dump look as similar as possible. Webrev with the above changes in http://cr.openjdk.java.net/~phh/8214535/webrev.01/ Thanks, Paul ?On 7/15/20, 2:13 AM, "linzang(??)" wrote: Upload a new webrev at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_07/ It fix a potential issue that unexpected number of threads maybe calculated for "parallel" option of jmap -histo in container. As shown at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_07-delta/src/hotspot/share/services/attachListener.cpp.udiff.html ############### attachListener.cpp #################### @@ -252,11 +252,11 @@ static jint heap_inspection(AttachOperation* op, outputStream* out) { bool live_objects_only = true; // default is true to retain the behavior before this change is made outputStream* os = out; // if path not specified or path is NULL, use out fileStream* fs = NULL; const char* arg0 = op->arg(0); - uint parallel_thread_num = MAX(1, os::processor_count() * 3 / 8); // default is less than half of processors. + uint parallel_thread_num = MAX(1, os::initial_active_processor_count() * 3 / 8); // default is less than half of processors. if (arg0 != NULL && (strlen(arg0) > 0)) { if (strcmp(arg0, "-all") != 0 && strcmp(arg0, "-live") != 0) { out->print_cr("Invalid argument to inspectheap operation: %s", arg0); return JNI_ERR; } ################################################### Thanks. BRs, Lin On 2020/7/9, 3:22 PM, "linzang(??)" wrote: Hi Paul, Thanks for reviewing! >> >> I'd move all the argument parsing code to JMap.java and just pass the results to Hotspot. Both histo() in JMap.java and code in attachListener.* parse the command line arguments, though the code in histo() doesn't parse the argument to "parallel". I'd upgrade the code in histo() to do a complete parse and pass the option values to executeCommandForPid as before: there would just be more of them now. That would allow you to eliminate all the parsing code in attachListener.cpp as well as the change to arguments.hpp. >> The reason I made the change in Jmap.java that compose all arguments as 1 string , instead of passing 3 argments, is to avoid the compatibility issue, as we discussed in http://mail.openjdk.java.net/pipermail/serviceability-dev/2019-February/thread.html#27240. The root cause of the compatibility issue is because max argument count in HotspotVirtualMachineImpl.java and attachlistener.cpp need to be enlarged (changes like http://hg.openjdk.java.net/jdk/jdk/rev/e7cf035682e3#l2.1) when jmap has more than 3 arguments. But if user use an old jcmd/jmap tool, it may stuck at socket read(), because the "max argument count" don't match. I re-checked this change, the argument count of jmap histo is equal to 3 (live, file, parallel), so it can work normally even without the change of passing argument. But I think we have to face the problem if more arguments is added in jcmd alike tools later, not sure whether it should be sloved (or a workaround) in this changeset. And here are the lastest webrev and delta: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_06/ http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_06-delta/ Cheers, Lin On 2020/7/7, 5:57 AM, "Hohensee, Paul" wrote: I'd like to see this feature added. :) The CSR looks good, as does the basic parallel inspection algorithm. Stefan's done the GC part, so I'll stick to the non-GC part (fwiw, the GC part lgtm). I'd move all the argument parsing code to JMap.java and just pass the results to Hotspot. Both histo() in JMap.java and code in attachListener.* parse the command line arguments, though the code in histo() doesn't parse the argument to "parallel". I'd upgrade the code in histo() to do a complete parse and pass the option values to executeCommandForPid as before: there would just be more of them now. That would allow you to eliminate all the parsing code in attachListener.cpp as well as the change to arguments.hpp. heapInspection.hpp: _shared_miss_count (s/b _missed_count, see below) isn't a size, so it should be a uint instead of a size_t. Same with the new parallel_thread_num argument to heap_inspection() and populate_table(). Comment copy-edit: +// Parallel heap inspection task. Parallel inspection can fail due to +// a native OOM when allocating memory for TL-KlassInfoTable. +// _success will be set false on an OOM, and serial inspection tried. _shared_miss_count should be _missed_count to match the missed_count() getter, or rename missed_count() to be shared_miss_count(). Whichever way you go, the field type should match the getter result type: uint is reasonable. heapInspection.cpp: You might use ResourceMark twice in populate_table, separately for the parallel attempt and the serial code. If the parallel attempt fails and available memory is low, it would be good to clean up the memory used by the parallel attempt before doing the serial code. Style nit in KlassInfoTable::merge_entry(). I'd line up the definitions of k and elt, so "k" is even with "elt". And, because it's two lines shorter, I'd replace + } else { + return false; + } with + return false; KlassInfoTableMergeClosure.is_success() should be just success() (i.e., no "is_" prefix) because it's a getter. I'd reorganize the code in populate_table() to make it more clear, vis (I changed _shared_missed_count to _missed_count) + if (cit.allocation_failed()) { + // fail to allocate memory, stop parallel mode + Atomic::store(&_success, false); + return; + } + RecordInstanceClosure ric(&cit, _filter); + _poi->object_iterate(&ric, worker_id); + missed_count = ric.missed_count(); + { + MutexLocker x(&_mutex); + merge_success = _shared_cit->merge(&cit); + } + if (merge_success) { + Atomic::add(&_missed_count, missed_count); + else { + Atomic::store(&_success, false); + } Thanks, Paul On 6/29/20, 7:20 PM, "linzang(??)" wrote: Dear All, Sorry to bother again, I just want to make sure that is this change worth to be continue to work on? If decision is made to not. I think I can drop this work and stop asking for help reviewing... Thanks for all your help about reviewing this previously. BRs, Lin On 2020/5/9, 3:47 PM, "linzang(??)" wrote: Dear All, May I ask your help again for review the latest change? Thanks! BRs, Lin On 2020/4/28, 1:54 PM, "linzang(??)" wrote: Hi Stefan, >> - Adding Atomic::load/store. >> - Removing the time measurement in the run_task. I renamed G1's function >> to run_task_timed. If we need this outside of G1, we can rethink the API >> at that point. >> - ZGC style cleanups Thanks for revising the patch, they are all good to me, and I have made a tiny change based on it: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04/ http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04-delta/ it reduce the scope of mutex in ParHeapInspectTask, and delete unnecessary comments. BRs, Lin On 2020/4/27, 4:34 PM, "Stefan Karlsson" wrote: Hi Lin, On 2020-04-26 05:10, linzang(??) wrote: > Hi Stefan and Paul? > I have made a new patch based on your comments and Stefan's Poc code: > Webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03/ > Delta(based on Stefan's change:) : http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03-delta/webrev_03-delta/ Thanks for providing a delta patch. It makes it much easier to look at, and more likely for reviewers to continue reviewing. I'm going to continue focusing on the GC parts, and leave the rest to others to review. > > And Here are main changed I made and want to discuss with you: > 1. changed"parallelThreadNum=" to "parallel=" for jmap -histo options. > 2. Add logic to test where parallelHeapInspection is fail, in heapInspection.cpp > This is because the parHeapInspectTask create thread local KlassInfoTable in it's work() method, and this may fail because of native OOM, in this case, the parallel should fail and serial heap inspection can be tried. > One more thing I want discuss with you is about the member "_success" of parHeapInspectTask, when native OOM happenes, it is set to false. And since this "set" operation can be conducted in multiple threads, should it be atomic ops? IMO, this is not necessary because "_success" can only be set to false, and there is no way to change it from back to true after the ParHeapInspectTask instance is created, so it is save to be non-atomic, do you agree with that? In these situations you should be using the Atomic::load/store primitives. We're moving toward a later C++ standard were data races are considered undefined behavior. > 3. make CollectedHeap::run_task() be an abstract virtual func, so that every subclass of collectedHeap should support it, so later implementation of new collectedHeap will not miss the "parallel" features. > The problem I want to discuss with you is about epsilonHeap and SerialHeap, as they may not need parallel heap iteration, so I only make task->work(0), in case the run_task() is invoked someway in future. Another way is to left run_task() unimplemented, which one do you think is better? I don't have a strong opinion about this. And also please help take a look at the zHeap, as there is a class zTask that wrap the abstractGangTask, and the collectedHeap::run_task() only accept AbstraceGangTask* as argument, so I made a delegate class to adapt it , please see src/hotspot/share/gc/z/zHeap.cpp. > > There maybe other better ways to sovle the above problems, welcome for any comments, Thanks! I've created a few cleanups and changes on top of your latest patch: https://cr.openjdk.java.net/~stefank/8215624/webrev.02.delta https://cr.openjdk.java.net/~stefank/8215624/webrev.02 - Adding Atomic::load/store. - Removing the time measurement in the run_task. I renamed G1's function to run_task_timed. If we need this outside of G1, we can rethink the API at that point. - ZGC style cleanups Thanks, StefanK > > BRs, > Lin > > On 2020/4/23, 11:08 AM, "linzang(??)" wrote: > > Thanks Paul! I agree with using "parallel", will make the update in next patch, Thanks for help update the CSR. > > BRs, > Lin > > On 2020/4/23, 4:42 AM, "Hohensee, Paul" wrote: > > For the interface, I'd use "parallel" instead of "parallelThreadNum". All the other options are lower case, and it's a lot easier to type "parallel". I took the liberty of updating the CSR. If you're ok with it, you might want to change variable names and such, plus of course JMap.usage. > > Thanks, > Paul > > On 4/22/20, 2:29 AM, "serviceability-dev on behalf of linzang(??)" wrote: > > Dear Stefan, > > Thanks a lot! I agree with you to decouple the heap inspection code with GC's. > I will start from your POC code, may discuss with you later. > > > BRs, > Lin > > On 2020/4/22, 5:14 PM, "Stefan Karlsson" wrote: > > Hi Lin, > > I took a look at this earlier and saw that the heap inspection code is > strongly coupled with the CollectedHeap and G1CollectedHeap. I'd prefer > if we'd abstract this away, so that the GCs only provide a "parallel > object iteration" interface, and the heap inspection code is kept elsewhere. > > I started experimenting with doing that, but other higher-priority (to > me) tasks have had to take precedence. > > I've uploaded my work-in-progress / proof-of-concept: > https://cr.openjdk.java.net/~stefank/8215624/webrev.01.delta/ > https://cr.openjdk.java.net/~stefank/8215624/webrev.01/ > > The current code doesn't handle the lifecycle (deletion) of the > ParallelObjectIterators. There's also code left unimplemented in around > CollectedHeap::run_task. However, I think this could work as a basis to > pull out the heap inspection code out of the GCs. > > Thanks, > StefanK > > On 2020-04-22 02:21, linzang(??) wrote: > > Dear all, > > May I ask you help to review? This RFR has been there for quite a while. > > Thanks! > > > > BRs, > > Lin > > > > > On 2020/3/16, 5:18 PM, "linzang(??)" wrote:> > > > >> Just update a new path, my preliminary measure show about 3.5x speedup of jmap histo on a nearly full 4GB G1 heap (8-core platform with parallel thread number set to 4). > >> webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_02/ > >> bug: https://bugs.openjdk.java.net/browse/JDK-8215624 > >> CSR: https://bugs.openjdk.java.net/browse/JDK-8239290 > >> BRs, > >> Lin > >> > On 2020/3/2, 9:56 PM, "linzang(??)" wrote: > >> > > >> > Dear all, > >> > Let me try to ease the reviewing work by some explanation :P > >> > The patch's target is to speed up jmap -histo for heap iteration, from my experience it is necessary for large heap investigation. E.g in bigData scenario I have tried to conduct jmap -histo against 180GB heap, it does take quite a while. > >> > And if my understanding is corrent, even the jmap -histo without "live" option does heap inspection with heap lock acquired. so it is very likely to block mutator thread in allocation-sensitive scenario. I would say the faster the heap inspection does, the shorter the mutator be blocked. This is parallel iteration for jmap is necessary. > >> > I think the parallel heap inspection should be applied to all kind of heap. However, consider the heap layout are different for GCs, much time is required to understand all kinds of the heap layout to make the whole change. IMO, It is not wise to have a huge patch for the whole solution at once, and it is even harder to review it. So I plan to implement it incrementally, the first patch (this one) is going to confirm the implemention detail of how jmap accept the new option, passes it to attachListener of the jvm process and then how to make the parallel inspection closure be generic enough to make it easy to extend to different heap layout. And also how to implement the heap inspection in specific gc's heap. This patch use G1's heap as the begining. > >> > This patch actually do several things: > >> > 1. Add an option "parallelThreadNum=" to jmap -histo, the default behavior is to set N to 0, means let's JVM decide how many threads to use for heap inspection. Set this option to 1 will disable parallel heap inspection. (more details in CSR: https://bugs.openjdk.java.net/browse/JDK-8239290) > >> > 2. Make a change in how Jmap passing arguments, changes in http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java.udiff.html, originally it pass options as separate arguments to attachListener, this patch change to that all options be compose to a single string. So the arg_count_max in attachListener.hpp do not need to be changed, and hence avoid the compatibility issue, as disscussed at https://mail.openjdk.java.net/pipermail/serviceability-dev/2019-March/027334.html > >> > 3. Add an abstract class ParHeapInspectTask in heapInspection.hpp / heapInspection.cpp, It's work(uint worker_id) method prepares the data structure (KlassInfoTable) need for every parallel worker thread, and then call do_object_iterate_parallel() which is heap specific implementation. I also added some machenism in KlassInfoTable to support parallel iteration, such as merge(). > >> > 4. In specific heap (G1 in this patch), create a subclass of ParHeapInspectTask, implement the do_object_iterate_parallel() for parallel heap inspection. For G1, it simply invoke g1CollectedHeap's object_iterate_parallel(). > >> > 5. Add related test. > >> > 6. it may be easy to extend this patch for other kinds of heap by creating subclass of ParHeapInspectTask and implement the do_object_iterate_parallel(). > >> > > >> > Hope these info could help on code review and initate the discussion :-) > >> > Thanks! > >> > > >> > BRs, > >> > Lin > >> > >On 2020/2/19, 9:40 AM, "linzang(??)" wrote:. > >> > > > >> > > Re-post this RFR with correct enhancement number to make it trackable. > >> > > please ignore the previous wrong post. sorry for troubles. > >> > > > >> > > webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/ > >> > > Hi bug: https://bugs.openjdk.java.net/browse/JDK-8215624 > >> > > CSR: https://bugs.openjdk.java.net/browse/JDK-8239290 > >> > > -------------- > >> > > Lin > >> > > >Hi Lin, > > > > > > > >> > > >Could you, please, re-post your RFR with the right enhancement number in > >> > > >the message subject? > >> > > >It will be more trackable this way. > >> > > > > >> > > >Thanks, > >> > > >Serguei > >> > > > > >> > > > > >> > > >On 2/17/20 10:29 PM, linzang(??) wrote: > >> > > >> Dear David, > >> > > >> Thanks a lot! > >> > > >> I have updated the refined code to http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_01/. > >> > > >> IMHO the parallel heap inspection can be extended to all kinds of heap as long as the heap layout can support parallel iteration. > >> > > >> Maybe we can firstly use this webrev to discuss how to implement it, because I am not sure my current implementation is an appropriate way to communicate with collectedHeap, then we can extend the solution to other kinds of heap. > >> > > >> > >> > > >> Thanks, > >> > > >> -------------- > >> > > >> Lin > >> > > >>> Hi Lin, > >> > > >>> > >> > > >>> Adding in hotspot-gc-dev as they need to see how this interacts with GC > >> > > >>> worker threads, and whether it needs to be extended beyond G1. > >> > > >>> > >> > > >>> I happened to spot one nit when browsing: > >> > > >>> > >> > > >>> src/hotspot/share/gc/shared/collectedHeap.hpp > >> > > >>> > >> > > >>> + virtual bool run_par_heap_inspect_task(KlassInfoTable* cit, > >> > > >>> + BoolObjectClosure* filter, > >> > > >>> + size_t* missed_count, > >> > > >>> + size_t thread_num) { > >> > > >>> + return NULL; > >> > > >>> > >> > > >>> s/NULL/false/ > >> > > >>> > >> > > >>> Cheers, > >> > > >>> David > > > > > >>> > >> > > >>> On 18/02/2020 2:15 pm, linzang(??) wrote: > >> > > >>>> Dear All, > >> > > >>>> May I ask your help to review the follow changes: > >> > > >>>> webrev: > >> > > >>>> http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_00/ > >> > > >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8215624 > >> > > >>>> related CSR: https://bugs.openjdk.java.net/browse/JDK-8239290 > >> > > >>>> This patch enable parallel heap inspection of G1 for jmap histo. > >> > > >>>> my simple test shown it can speed up 2x of jmap -histo with > >> > > >>>> parallelThreadNum set to 2 for heap at ~500M on 4-core platform. > >> > > >>>> > >> > > >>>> ------------------------------------------------------------------------ > >> > > >>>> BRs, > >> > > >>>> Lin > >> > > >> > > >> > > > > > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Wed Jul 22 23:26:01 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 23 Jul 2020 09:26:01 +1000 Subject: Fwd: jdk/submit maintenance, July 23rd - July 27th In-Reply-To: <8754038A-93DA-40AC-9AAA-5AEBA5A59A1D@ORACLE.COM> References: <8754038A-93DA-40AC-9AAA-5AEBA5A59A1D@ORACLE.COM> Message-ID: <70d10304-2341-54fe-49f5-532811476e4c@oracle.com> Just in case people don't see the jdk-dev email. David ----- -------- Forwarded Message -------- Subject: jdk/submit maintenance, July 23rd - July 27th Date: Wed, 22 Jul 2020 15:26:30 -0400 From: Stanislav Smirnov To: jdk-dev Hello, A planned maintenance will happen this week, July 23rd - 27th. jdk/submit builds will not be available starting tomorrow (Thursday 23rd) at 10am PT / 17:00 GMT. The system should be back online by 10am PT / 17:00 GMT, July 27th. Best regards, Stanislav Smirnov From serguei.spitsyn at oracle.com Thu Jul 23 00:02:12 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 22 Jul 2020 17:02:12 -0700 Subject: [15?] RFR (S): 8249192: MonitorInfo stores raw oops across safepoints In-Reply-To: <07904b7e-8d40-8c9e-acc1-c713c1588b11@oracle.com> References: <3f838c8a-6e70-cd94-1753-8abe269b7cdf@oracle.com> <528ade6c-059a-d9af-b0c7-74d7ac31b29a@oracle.com> <14512a57-8d4d-5199-ed16-743b5da8a10d@oracle.com> <893f862f-093a-5168-e554-1f4ffe129aea@oracle.com> <07904b7e-8d40-8c9e-acc1-c713c1588b11@oracle.com> Message-ID: <1d400bf3-9940-0cf2-abaa-61a31e9b08d5@oracle.com> An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Thu Jul 23 00:31:02 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 22 Jul 2020 17:31:02 -0700 Subject: RFR: 8248362: JVMTI frame operations should use Thread-Local Handshake In-Reply-To: References: Message-ID: An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Thu Jul 23 00:53:49 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 22 Jul 2020 17:53:49 -0700 Subject: RFR (S) 8249822: SymbolPropertyTable creates an extra OopHandle per entry In-Reply-To: <84643f6c-0e05-061e-b6e3-a38d09ed973a@oracle.com> References: <84643f6c-0e05-061e-b6e3-a38d09ed973a@oracle.com> Message-ID: Hi Coleen, The fix looks good to me. On 7/22/20 13:55, coleen.phillimore at oracle.com wrote: > Summary: Add an assert to OopHandle assigment operator to catch > leaking OopHandles, and fix code accordingly. > > There are some jvmtiRedefineClasses.cpp changes here - basically, I > moved the RedefineVerifyMark and comments into > jvmtiRedefineClasses.cpp because I needed a Handle. I think, the jvmtiRedefineClasses.cpp is a better place for the RedefineVerifyMark. But could you, please, explain a little bit more why this move was necessary? Why Handle can not be used in the jvmtiThreadState.cpp? Thanks, Serguei > Ran tier1-6 tests and tier1 on all Oracle platforms. > > open webrev at http://cr.openjdk.java.net/~coleenp/2020/8249822.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8249822 > > Thanks, > Coleen From daniel.daugherty at oracle.com Thu Jul 23 01:04:41 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 22 Jul 2020 21:04:41 -0400 Subject: RFR: 8248362: JVMTI frame operations should use Thread-Local Handshake In-Reply-To: References: Message-ID: On 7/22/20 10:38 AM, Yasumasa Suenaga wrote: > Hi all, > > Please review this change: > > ? JBS: https://bugs.openjdk.java.net/browse/JDK-8248362 > ? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8248362/webrev.00/ src/hotspot/share/prims/jvmtiEnv.cpp ??? L1755???? // JVMTI get java stack frame location at safepoint. ??????? You missed updating this one. Perhaps: ????????????? // JVMTI get java stack frame location via direct handshake. src/hotspot/share/prims/jvmtiEnvBase.cpp ??? L1563: ? JavaThread* jt = _state->get_thread(); ??? L1564: ? assert(target == jt, "just checking"); ??????? This code is different than the other closures. It might be worth ??????? a comment to explain why. I don't remember why VM_GetFrameCount had ??????? to use the JvmtiThreadState to fetch the JavaThread. Serguei might ??????? remember. ??????? It could be that we don't need the _state field anymore because of ??????? the way that handshakes work (and provide the JavaThread* to the ??????? do_thread() function). Your choice on whether to deal with that as ??????? part of this fix or following with another RFE. ??????? Update: Uggg.... the get_frame_count() function takes JvmtiThreadState ??????? as a parameter. This is very much entangled... sigh... we should ??????? definitely look at untangling this in another RFE... ??? old L1565: ? ThreadsListHandle tlh; ??? new L1565: ? if (!jt->is_exiting() && jt->threadObj() != NULL) { ??????? Please consider add this comment above L1565: ???????????????? // ThreadsListHandle is not needed due to direct handshake. ??????? Yes, I see that previous closures were added without that comment, ??????? but when I see "if (!jt->is_exiting() && jt->threadObj() != NULL)" ??????? I immediately wonder where the ThreadsListHandle is... Please consider ??????? adding the comment to the others also. ??? old L1574: ? ThreadsListHandle tlh; ??? new L1574: ? if (!jt->is_exiting() && jt->threadObj() != NULL) { ??????? Please consider add this comment above L1574: ???????????????? // ThreadsListHandle is not needed due to direct handshake. src/hotspot/share/prims/jvmtiEnvBase.hpp ??? L580: // HandshakeClosure to frame location. ??????? typo - s/to frame/to get frame/ src/hotspot/share/prims/jvmtiThreadState.cpp ??? No comments on the changes. ??? For the above comments about VM_GetFrameCount, understanding why ??? JvmtiThreadState::count_frames() has to exist in JvmtiThreadState ??? will go along way towards untangling the mess. src/hotspot/share/runtime/vmOperations.hpp ??? No comments. Thumbs up. I only have nits above. If you choose to make the above changes, I do not need to see another webrev. Dan > > Migrate JVMTI frame operations to Thread-Local Handshakes from VM > Operations. > > ??? - VM_GetFrameCount > ??? - VM_GetFrameLocation > > They affects both GetFrameCount() and GetFrameLocation() JVMTI functions. > > This change has passed all tests on submit repo > (mach5-one-ysuenaga-JDK-8248362-20200722-1249-12859056), and > vmTestbase/nsk/{jdi,jdw,jvmti}, serviceability/{jdwp,jvmti} . > > > Thanks, > > Yasumasa From david.holmes at oracle.com Thu Jul 23 01:25:55 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 23 Jul 2020 11:25:55 +1000 Subject: RFR: 8248362: JVMTI frame operations should use Thread-Local Handshake In-Reply-To: References: Message-ID: Hi Dan, I just want to respond to one aspect of your response ... On 23/07/2020 11:04 am, Daniel D. Daugherty wrote: > On 7/22/20 10:38 AM, Yasumasa Suenaga wrote: >> Hi all, >> >> Please review this change: >> >> ? JBS: https://bugs.openjdk.java.net/browse/JDK-8248362 >> ? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8248362/webrev.00/ > > src/hotspot/share/prims/jvmtiEnv.cpp > ??? L1755???? // JVMTI get java stack frame location at safepoint. > ??????? You missed updating this one. Perhaps: > > ????????????? // JVMTI get java stack frame location via direct handshake. > > src/hotspot/share/prims/jvmtiEnvBase.cpp > ??? L1563: ? JavaThread* jt = _state->get_thread(); > ??? L1564: ? assert(target == jt, "just checking"); > ??????? This code is different than the other closures. It might be worth > ??????? a comment to explain why. I don't remember why VM_GetFrameCount > had > ??????? to use the JvmtiThreadState to fetch the JavaThread. Serguei might > ??????? remember. > > ??????? It could be that we don't need the _state field anymore because of > ??????? the way that handshakes work (and provide the JavaThread* to the > ??????? do_thread() function). Your choice on whether to deal with that as > ??????? part of this fix or following with another RFE. > > ??????? Update: Uggg.... the get_frame_count() function takes > JvmtiThreadState > ??????? as a parameter. This is very much entangled... sigh... we should > ??????? definitely look at untangling this in another RFE... > > ??? old L1565: ? ThreadsListHandle tlh; > ??? new L1565: ? if (!jt->is_exiting() && jt->threadObj() != NULL) { > > ??????? Please consider add this comment above L1565: > ???????????????? // ThreadsListHandle is not needed due to direct > handshake. > > ??????? Yes, I see that previous closures were added without that comment, > ??????? but when I see "if (!jt->is_exiting() && jt->threadObj() != NULL)" > ??????? I immediately wonder where the ThreadsListHandle is... Please > consider > ??????? adding the comment to the others also. I understand why, when looking at this change, you would like the comment, but outside of the webrev the reference to ThreadsListHandle doesn't really have any context. We need a TLH anywhere there is a risk the target thread might exit while we're interacting with it, but that can't happen in the context of a direct handshake operation with the target thread because the TLH exists in our caller. This is the new normal for code that executes as part of a direct handshake. David ----- > ??? old L1574: ? ThreadsListHandle tlh; > ??? new L1574: ? if (!jt->is_exiting() && jt->threadObj() != NULL) { > > ??????? Please consider add this comment above L1574: > ???????????????? // ThreadsListHandle is not needed due to direct > handshake. > > src/hotspot/share/prims/jvmtiEnvBase.hpp > ??? L580: // HandshakeClosure to frame location. > ??????? typo - s/to frame/to get frame/ > > src/hotspot/share/prims/jvmtiThreadState.cpp > ??? No comments on the changes. > > ??? For the above comments about VM_GetFrameCount, understanding why > ??? JvmtiThreadState::count_frames() has to exist in JvmtiThreadState > ??? will go along way towards untangling the mess. > > src/hotspot/share/runtime/vmOperations.hpp > ??? No comments. > > > Thumbs up. I only have nits above. If you choose to make the above > changes, I do not need to see another webrev. > > Dan > > > >> >> Migrate JVMTI frame operations to Thread-Local Handshakes from VM >> Operations. >> >> ??? - VM_GetFrameCount >> ??? - VM_GetFrameLocation >> >> They affects both GetFrameCount() and GetFrameLocation() JVMTI functions. >> >> This change has passed all tests on submit repo >> (mach5-one-ysuenaga-JDK-8248362-20200722-1249-12859056), and >> vmTestbase/nsk/{jdi,jdw,jvmti}, serviceability/{jdwp,jvmti} . >> >> >> Thanks, >> >> Yasumasa > From david.holmes at oracle.com Thu Jul 23 01:47:51 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 23 Jul 2020 11:47:51 +1000 Subject: Fwd: Re: jdk/submit maintenance, July 23rd - July 27th In-Reply-To: <335D0317-7810-46F1-82B7-BE9268887A83@ORACLE.COM> References: <335D0317-7810-46F1-82B7-BE9268887A83@ORACLE.COM> Message-ID: -------- Forwarded Message -------- Subject: Re: jdk/submit maintenance, July 23rd - July 27th Date: Wed, 22 Jul 2020 21:39:40 -0400 From: Stanislav Smirnov To: jdk-dev Hello, Due to circumstances beyond our control the planned maintenance is called off. This means, that jdk/submit will continue to work. Best regards, Stanislav Smirnov > On Jul 22, 2020, at 3:26 PM, Stanislav Smirnov wrote: > > Hello, > > A planned maintenance will happen this week, July 23rd - 27th. > jdk/submit builds will not be available starting tomorrow (Thursday 23rd) at 10am PT / 17:00 GMT. > > The system should be back online by 10am PT / 17:00 GMT, July 27th. > > Best regards, > Stanislav Smirnov From david.holmes at oracle.com Thu Jul 23 02:01:36 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 23 Jul 2020 12:01:36 +1000 Subject: RFR: 8248362: JVMTI frame operations should use Thread-Local Handshake In-Reply-To: References: Message-ID: <5ef612da-7682-154f-8c40-eaf4ed92b0f8@oracle.com> Hi Yasumasa, On 23/07/2020 12:38 am, Yasumasa Suenaga wrote: > Hi all, > > Please review this change: > > ? JBS: https://bugs.openjdk.java.net/browse/JDK-8248362 > ? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8248362/webrev.00/ > > Migrate JVMTI frame operations to Thread-Local Handshakes from VM > Operations. > > ??? - VM_GetFrameCount > ??? - VM_GetFrameLocation > > They affects both GetFrameCount() and GetFrameLocation() JVMTI functions. Your changes all seem fine. But I'm a bit confused about the existing code. In JvmtiEnv::GetFrameLocation we have: if (java_thread == JavaThread::current()) { err = get_frame_location(java_thread, depth, method_ptr, location_ptr); but then in get_frame_location we have: assert((SafepointSynchronize::is_at_safepoint() || java_thread->is_thread_fully_suspended(false, &debug_bits)), "at safepoint or target thread is suspended"); and that assert must surely fire if called by the current thread for itself! ??? Thanks, David > This change has passed all tests on submit repo > (mach5-one-ysuenaga-JDK-8248362-20200722-1249-12859056), and > vmTestbase/nsk/{jdi,jdw,jvmti}, serviceability/{jdwp,jvmti} . > > > Thanks, > > Yasumasa From suenaga at oss.nttdata.com Thu Jul 23 03:03:45 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Thu, 23 Jul 2020 12:03:45 +0900 Subject: RFR: 8248362: JVMTI frame operations should use Thread-Local Handshake In-Reply-To: References: Message-ID: <3c74c4d5-e1b6-439e-253f-a36a0d0b3cf3@oss.nttdata.com> Hi Serguei, Thanks for your comment! I fixed it in new webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8248362/webrev.01/ Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/df75038b5449 Yasumasa On 2020/07/23 9:31, serguei.spitsyn at oracle.com wrote: > Hi Yasumasa, > > Looks good. > Just one minor comment. > > http://cr.openjdk.java.net/~ysuenaga/JDK-8248362/webrev.00/src/hotspot/share/prims/jvmtiEnv.cpp.udiff.html > > // JVMTI get java stack frame location at safepoint. > > Replace: "at safepoint" => "with handshake". > > Thanks, > Serguei > > > On 7/22/20 07:38, Yasumasa Suenaga wrote: >> Hi all, >> >> Please review this change: >> >> ? JBS: https://bugs.openjdk.java.net/browse/JDK-8248362 >> ? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8248362/webrev.00/ >> >> Migrate JVMTI frame operations to Thread-Local Handshakes from VM Operations. >> >> ??? - VM_GetFrameCount >> ??? - VM_GetFrameLocation >> >> They affects both GetFrameCount() and GetFrameLocation() JVMTI functions. >> >> This change has passed all tests on submit repo (mach5-one-ysuenaga-JDK-8248362-20200722-1249-12859056), and vmTestbase/nsk/{jdi,jdw,jvmti}, serviceability/{jdwp,jvmti} . >> >> >> Thanks, >> >> Yasumasa > From suenaga at oss.nttdata.com Thu Jul 23 03:12:19 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Thu, 23 Jul 2020 12:12:19 +0900 Subject: RFR: 8248362: JVMTI frame operations should use Thread-Local Handshake In-Reply-To: References: Message-ID: <5c2bba7d-44aa-d5e7-6d5c-949c18ce9668@oss.nttdata.com> Hi Dan, Thanks for your comment! I uploaded new webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8248362/webrev.01/ Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/df75038b5449 On 2020/07/23 10:04, Daniel D. Daugherty wrote: > On 7/22/20 10:38 AM, Yasumasa Suenaga wrote: >> Hi all, >> >> Please review this change: >> >> ? JBS: https://bugs.openjdk.java.net/browse/JDK-8248362 >> ? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8248362/webrev.00/ > > src/hotspot/share/prims/jvmtiEnv.cpp > ??? L1755???? // JVMTI get java stack frame location at safepoint. > ??????? You missed updating this one. Perhaps: > > ????????????? // JVMTI get java stack frame location via direct handshake. Fixed. > src/hotspot/share/prims/jvmtiEnvBase.cpp > ??? L1563: ? JavaThread* jt = _state->get_thread(); > ??? L1564: ? assert(target == jt, "just checking"); > ??????? This code is different than the other closures. It might be worth > ??????? a comment to explain why. I don't remember why VM_GetFrameCount had > ??????? to use the JvmtiThreadState to fetch the JavaThread. Serguei might > ??????? remember. > > ??????? It could be that we don't need the _state field anymore because of > ??????? the way that handshakes work (and provide the JavaThread* to the > ??????? do_thread() function). Your choice on whether to deal with that as > ??????? part of this fix or following with another RFE. > > ??????? Update: Uggg.... the get_frame_count() function takes JvmtiThreadState > ??????? as a parameter. This is very much entangled... sigh... we should > ??????? definitely look at untangling this in another RFE... I want to fix it in another RFE as you said if it is needed. If we don't hear the reason from Serguei, I want to keep this change. > ??? old L1565: ? ThreadsListHandle tlh; > ??? new L1565: ? if (!jt->is_exiting() && jt->threadObj() != NULL) { > > ??????? Please consider add this comment above L1565: > ???????????????? // ThreadsListHandle is not needed due to direct handshake. > > ??????? Yes, I see that previous closures were added without that comment, > ??????? but when I see "if (!jt->is_exiting() && jt->threadObj() != NULL)" > ??????? I immediately wonder where the ThreadsListHandle is... Please consider > ??????? adding the comment to the others also. > > ??? old L1574: ? ThreadsListHandle tlh; > ??? new L1574: ? if (!jt->is_exiting() && jt->threadObj() != NULL) { > > ??????? Please consider add this comment above L1574: > ???????????????? // ThreadsListHandle is not needed due to direct handshake. I think it is new normal as David said in the reply, and also other closures don't say about it. So I did not change about it in new webrev. > src/hotspot/share/prims/jvmtiEnvBase.hpp > ??? L580: // HandshakeClosure to frame location. > ??????? typo - s/to frame/to get frame/ Fixed. > src/hotspot/share/prims/jvmtiThreadState.cpp > ??? No comments on the changes. > > ??? For the above comments about VM_GetFrameCount, understanding why > ??? JvmtiThreadState::count_frames() has to exist in JvmtiThreadState > ??? will go along way towards untangling the mess. > > src/hotspot/share/runtime/vmOperations.hpp > ??? No comments. > > > Thumbs up. I only have nits above. If you choose to make the above > changes, I do not need to see another webrev. Thanks, but I uploaded new webrev just in case :) Yasumasa > Dan > > > >> >> Migrate JVMTI frame operations to Thread-Local Handshakes from VM Operations. >> >> ??? - VM_GetFrameCount >> ??? - VM_GetFrameLocation >> >> They affects both GetFrameCount() and GetFrameLocation() JVMTI functions. >> >> This change has passed all tests on submit repo (mach5-one-ysuenaga-JDK-8248362-20200722-1249-12859056), and vmTestbase/nsk/{jdi,jdw,jvmti}, serviceability/{jdwp,jvmti} . >> >> >> Thanks, >> >> Yasumasa > From suenaga at oss.nttdata.com Thu Jul 23 03:13:35 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Thu, 23 Jul 2020 12:13:35 +0900 Subject: RFR: 8248362: JVMTI frame operations should use Thread-Local Handshake In-Reply-To: <5ef612da-7682-154f-8c40-eaf4ed92b0f8@oracle.com> References: <5ef612da-7682-154f-8c40-eaf4ed92b0f8@oracle.com> Message-ID: <803f35f4-faab-a195-d513-b43f24027b89@oss.nttdata.com> Hi David, Thanks for your comment! On 2020/07/23 11:01, David Holmes wrote: > Hi Yasumasa, > > On 23/07/2020 12:38 am, Yasumasa Suenaga wrote: >> Hi all, >> >> Please review this change: >> >> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8248362 >> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8248362/webrev.00/ >> >> Migrate JVMTI frame operations to Thread-Local Handshakes from VM Operations. >> >> ???? - VM_GetFrameCount >> ???? - VM_GetFrameLocation >> >> They affects both GetFrameCount() and GetFrameLocation() JVMTI functions. > > Your changes all seem fine. > > But I'm a bit confused about the existing code. In JvmtiEnv::GetFrameLocation we have: > > ?? if (java_thread == JavaThread::current()) { > ???? err = get_frame_location(java_thread, depth, method_ptr, location_ptr); > > but then in get_frame_location we have: > > ?? assert((SafepointSynchronize::is_at_safepoint() || > ?????????? java_thread->is_thread_fully_suspended(false, &debug_bits)), > ?????????? "at safepoint or target thread is suspended"); > > and that assert must surely fire if called by the current thread for itself! ??? is_thread_fully_suspended() returns true when it calles from current thread, so this assert would not fire. Yasumasa > Thanks, > David > >> This change has passed all tests on submit repo (mach5-one-ysuenaga-JDK-8248362-20200722-1249-12859056), and vmTestbase/nsk/{jdi,jdw,jvmti}, serviceability/{jdwp,jvmti} . >> >> >> Thanks, >> >> Yasumasa From linzang at tencent.com Thu Jul 23 03:56:43 2020 From: linzang at tencent.com (=?utf-8?B?bGluemFuZyjoh6fnkLMp?=) Date: Thu, 23 Jul 2020 03:56:43 +0000 Subject: RFR(L): 8215624: add parallel heap inspection support for jmap histo(G1)(Internet mail) In-Reply-To: References: Message-ID: Hi Paul, Thanks for your help, that all looks good to me. Just 2 minor changes: ? delete the final return in ParHeapInspectTask::work, you mentioned it but seems not include in the webrev :-) ? delete a unnecessary blank line in heapInspect.cpp at merge_entry() ######################################################################### --- old/src/hotspot/share/memory/heapInspection.cpp ? ? 2020-07-23 11:23:29.281666456 +0800 +++ new/src/hotspot/share/memory/heapInspection.cpp ? ? 2020-07-23 11:23:29.017666447 +0800 @@ -251,7 +251,6 @@ ?? ? _size_of_instances_in_words += cie->words(); ?? ? return true; ?? }? ? - ?? return false; ?} ? @@ -568,7 +567,6 @@ ?? ? Atomic::add(&_missed_count, missed_count); ?? } else { ?? ? Atomic::store(&_success, false); - ? return; ?? }? ? ?} ######################################################################### Here is the webrev http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_08/ BRs, Lin --------------------------------------------- From: "Hohensee, Paul" Date: Thursday, July 23, 2020 at 6:48 AM To: "linzang(??)" , Stefan Karlsson , "serguei.spitsyn at oracle.com" , David Holmes , serviceability-dev , "hotspot-gc-dev at openjdk.java.net" Subject: RE: RFR(L): 8215624: add parallel heap inspection support for jmap histo(G1)(Internet mail) Just small things. ? heapInspection.cpp: ? In ParHeapInspectTask::work, remove the final return statement and fix the following ?}? indent. I.e., replace ? +??? Atomic::store(&_success, false); +??? return; +?? } ? with ? +??? Atomic::store(&_success, false); +? } ? In HeapInspection::heap_inspection, missed_count should be a uint to match other missed_count declarations, and should be initialized to the result of populate_table() rather than separately to 0. ? attachListener.cpp: ? In heap_inspection, initial_processor_count returns an int, so cast its result to a uint. ? Similarly, parse_uintx returns a uintx, so cast its result (num) to uint when assigning to parallel_thread_num. ? BasicJMapTest.java: ? I took the liberty of refactoring testHisto*/histoToFile/testDump*/dump to remove redundant interposition methods and make histoToFile and dump look as similar as possible. ? Webrev with the above changes in ? http://cr.openjdk.java.net/~phh/8214535/webrev.01/ ? Thanks, Paul ? On 7/15/20, 2:13 AM, "linzang(??)" wrote: ? ???? Upload a new webrev at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_07/ ???? It fix a potential issue that unexpected number of threads maybe calculated for "parallel" option of jmap -histo in container. ??? As shown at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_07-delta/src/hotspot/share/services/attachListener.cpp.udiff.html ? ??? ############### attachListener.cpp #################### ??? @@ -252,11 +252,11 @@ ???? static jint heap_inspection(AttachOperation* op, outputStream* out) { ?????? bool live_objects_only = true;?? // default is true to retain the behavior before this change is made ?????? outputStream* os = out;?? // if path not specified or path is NULL, use out ?????? fileStream* fs = NULL; ?????? const char* arg0 = op->arg(0); ??? -? uint parallel_thread_num = MAX(1, os::processor_count() * 3 / 8); // default is less than half of processors. ??? +? uint parallel_thread_num = MAX(1, os::initial_active_processor_count() * 3 / 8); // default is less than half of processors. ?????? if (arg0 != NULL && (strlen(arg0) > 0)) { ???????? if (strcmp(arg0, "-all") != 0 && strcmp(arg0, "-live") != 0) { ?????????? out->print_cr("Invalid argument to inspectheap operation: %s", arg0); ?????????? return JNI_ERR; ???????? } ??? ################################################### ? ??? Thanks. ? ??? BRs, ???Lin ? ??? On 2020/7/9, 3:22 PM, "linzang(??)" wrote: ? ??????? Hi Paul, ??????????? Thanks for reviewing! ??????????? >> ??????????? >>???? I'd move all the argument parsing code to JMap.java and just pass the results to Hotspot. Both histo() in JMap.java and code in attachListener.* parse the command line arguments, though the code in histo() doesn't parse the argument to "parallel". I'd upgrade the code in histo() to do a complete parse and pass the option values to executeCommandForPid as before: there would just be more of them now. That would allow you to eliminate all the parsing code in attachListener.cpp as well as the change to arguments.hpp. ??????????? >> ??????????? The reason I made the change in Jmap.java that compose all arguments as 1 string , instead of passing 3 argments, is to avoid the compatibility issue, as we discussed in http://mail.openjdk.java.net/pipermail/serviceability-dev/2019-February/thread.html#27240.? The root cause of the compatibility issue is because max argument count in HotspotVirtualMachineImpl.java and attachlistener.cpp need to be enlarged (changes like http://hg.openjdk.java.net/jdk/jdk/rev/e7cf035682e3#l2.1) when jmap has more than 3 arguments. But if user use an old jcmd/jmap tool, it may stuck at socket read(), because the "max argument count" don't match. ???????????? I re-checked this change, the argument count of jmap histo is equal to 3 (live, file, parallel), so it can work normally even without the change of passing argument. But I think we have to face the problem if more arguments is added in jcmd alike tools later, not sure whether it should be sloved (or a workaround) in this changeset. ? ??????????? And here are the lastest webrev and delta: ??????????? http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_06/ ??????????? http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_06-delta/ ? ??????? Cheers, ??????? Lin ? ??????? On 2020/7/7, 5:57 AM, "Hohensee, Paul" wrote: ? ??????????? I'd like to see this feature added. :) ? ??????????? The CSR looks good, as does the basic parallel inspection algorithm. Stefan's done the GC part, so I'll stick to the non-GC part (fwiw, the GC part lgtm). ? ??????????? I'd move all the argument parsing code to JMap.java and just pass the results to Hotspot. Both histo() in JMap.java and code in attachListener.* parse the command line arguments, though the code in histo() doesn't parse the argument to "parallel". I'd upgrade the code in histo() to do a complete parse and pass the option values to executeCommandForPid as before: there would just be more of them now. That would allow you to eliminate all the parsing code in attachListener.cpp as well as the change to arguments.hpp. ? ??????????? heapInspection.hpp: ? ??????????? _shared_miss_count (s/b _missed_count, see below) isn't a size, so it should be a uint instead of a size_t. Same with the new parallel_thread_num argument to heap_inspection() and populate_table(). ? ??????????? Comment copy-edit: ??????????? +// Parallel heap inspection task. Parallel inspection can fail due to ??????????? +// a native OOM when allocating memory for TL-KlassInfoTable. ??????????? +// _success will be set false on an OOM, and serial inspection tried. ? ??????????? _shared_miss_count should be _missed_count to match the missed_count() getter, or rename missed_count() to be shared_miss_count(). Whichever way you go, the field type should match the getter result type: uint is reasonable. ? ??????????? heapInspection.cpp: ? ??????????? You might use ResourceMark twice in populate_table, separately for the parallel attempt and the serial code. If the parallel attempt fails and available memory is low, it would be good to clean up the memory used by the parallel attempt before doing the serial code. ? ??????????? Style nit in KlassInfoTable::merge_entry(). I'd line up the definitions of k and elt, so "k" is even with "elt". And, because it's two lines shorter, I'd replace ??????????? +? } else { ??????????? +??? return false; ??????????? +? } ??????????? with ??????????? +? return false; ? ??????????? KlassInfoTableMergeClosure.is_success() should be just success() (i.e., no "is_" prefix) because it's a getter. ? ??????????? I'd reorganize the code in populate_table() to make it more clear, vis (I changed _shared_missed_count to _missed_count) ??????????? +? if (cit.allocation_failed()) { ??????????? +??? // fail to allocate memory, stop parallel mode ??????????? +??? Atomic::store(&_success, false); ??????????? +??? return; ??????????? +? } ??????????? +? RecordInstanceClosure ric(&cit, _filter); ??????????? +? _poi->object_iterate(&ric, worker_id); ??????????? +? missed_count = ric.missed_count(); ??????????? +? { ??????????? +??? MutexLocker x(&_mutex); ??????????? +??? merge_success = _shared_cit->merge(&cit); ??????????? +? } ??????????? +? if (merge_success) { ??????????? +??? Atomic::add(&_missed_count, missed_count); ??????????? +? else { ??????????? +??? Atomic::store(&_success, false); ??????????? +? } ? ??????????? Thanks, ??????????? Paul ? ??????????? On 6/29/20, 7:20 PM, "linzang(??)" wrote: ? ??????????????? Dear All, ??????????????????????? Sorry to bother again, I just want to make sure that is this change worth to be continue to work on? If decision is made to not. I think I can drop this work and stop asking for help reviewing... ??????????????????????? Thanks for all your help about reviewing this previously. ? ??????????????? BRs, ??????????????? Lin ? ??????????????? On 2020/5/9, 3:47 PM, "linzang(??)" wrote: ? ??????????????????? Dear All, ?????????????????????????? May I ask your help again for review the latest change?? Thanks! ? ??????????????????? BRs, ??????????????????? Lin ? ??????????????????? On 2020/4/28, 1:54 PM, "linzang(??)" wrote: ? ??????????????????????? Hi Stefan, ????????????????????????? >>? - Adding Atomic::load/store. ????????????????????????? >>? - Removing the time measurement in the run_task. I renamed G1's function ????????????????????????? >>? to run_task_timed. If we need this outside of G1, we can rethink the API ????????????????????????? >>? at that point. ?????????????????????????? >>? - ZGC style cleanups ?? ????????????????????????Thanks for revising the patch,? they are all good to me, and I have made a tiny change based on it: ?????????????????????????????? http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04/ ?????????????????????????????? http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04-delta/ ????????????????????????? it reduce the scope of mutex in ParHeapInspectTask, and delete unnecessary comments. ? ??????????????????????? BRs, ??????????????????????? Lin ? ????????????????? ??????On 2020/4/27, 4:34 PM, "Stefan Karlsson" wrote: ? ??????????????????????????? Hi Lin, ? ??????????????????????????? On 2020-04-26 05:10, linzang(??) wrote: ??????????????????????????? > Hi Stefan and Paul? ???????????????? ???????????>????? I have made a new patch based on your comments and Stefan's Poc code: ??????????????????????????? >????? Webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03/ ??????????????????????????? >????? Delta(based on Stefan's change:) : http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03-delta/webrev_03-delta/ ? ??????????????????????????? Thanks for providing a delta patch. It makes it much easier to look at, ??????????????????????????? and more likely for reviewers to continue reviewing. ? ??????????????????????????? I'm going to continue focusing on the GC parts, and leave the rest to ??????????????????????????? others to review. ? ??????????????????????????? > ??????????????????????????? >????? And Here are main changed I made and want to discuss with you: ??????????????????????????? >????? 1.? changed"parallelThreadNum=" to "parallel=" for jmap -histo options. ??????????????????????????? >????? 2.? Add logic to test where parallelHeapInspection is fail, in heapInspection.cpp ??????????????????????????? >??????????? This is because the parHeapInspectTask create thread local KlassInfoTable in it's work() method, and this may fail because of native OOM, in this case, the parallel should fail and serial heap inspection can be tried. ??????????????????????????? >??????????? One more thing I want discuss with you is about the member "_success" of parHeapInspectTask, when native OOM happenes, it is set to false. And since this "set" operation can be conducted in multiple threads, should it be atomic ops?? IMO, this is not necessary because "_success" can only be set to false, and there is no way to change it from back to true after the ParHeapInspectTask instance is created, so it is save to be non-atomic, do you agree with that? ? ??????????????????????????? In these situations you should be using the Atomic::load/store ??????????????????????????? primitives. We're moving toward a later C++ standard were data races are ???????????????????????????considered undefined behavior. ? ??????????????????????????? >???? 3. make CollectedHeap::run_task() be an abstract virtual func, so that every subclass of collectedHeap should support it, so later implementation of new collectedHeap will not miss the "parallel" features. ??????????????????????????? >?????????? The problem I want to discuss with you is about epsilonHeap and SerialHeap, as they may not need parallel heap iteration, so I only make task->work(0), in case the run_task() is invoked someway in future. Another way is to left run_task()? unimplemented, which one do you think is better? ? ??????????????????????????? I don't have a strong opinion about this. ? ????????????????????????????? And also please help take a look at the zHeap, as there is a class ??????????????????????????? zTask that wrap the abstractGangTask, and the collectedHeap::run_task() ??????????????????????????? only accept? AbstraceGangTask* as argument, so I made a delegate class ??????????????????????????? to adapt it , please see src/hotspot/share/gc/z/zHeap.cpp. ??????????????????????????? > ??????????????????????????? >??????? There maybe other better ways to sovle the above problems, welcome for any comments, Thanks! ? ??????????????????????????? I've created a few cleanups and changes on top of your latest patch: ? ??????????????????????????? https://cr.openjdk.java.net/~stefank/8215624/webrev.02.delta ??????????????????????????? https://cr.openjdk.java.net/~stefank/8215624/webrev.02 ? ??????????????????????????? - Adding Atomic::load/store. ??????????????????????????? - Removing the time measurement in the run_task. I renamed G1's function ????????????? ??????????????to run_task_timed. If we need this outside of G1, we can rethink the API ??????????????????????????? at that point. ??????????????????????????? - ZGC style cleanups ? ??????????????????????????? Thanks, ??????????????????????????? StefanK ? ?? ?????????????????????????> ??????????????????????????? > BRs, ??????????????????????????? > Lin ??????????????????????????? > ??????????????????????????? > On 2020/4/23, 11:08 AM, "linzang(??)" wrote: ??????????????????????????? > ?? ?????????????????????????>????? Thanks Paul! I agree with using "parallel", will make the update in next patch, Thanks for help update the CSR. ??????????????????????????? > ??????????????????????????? >????? BRs, ??????????????????????????? >????? Lin ?? ?????????????????????????> ??????????????????????????? >????? On 2020/4/23, 4:42 AM, "Hohensee, Paul" wrote: ??????????????????????????? > ??????????????????????????? >????????? For the interface, I'd use "parallel" instead of "parallelThreadNum". All the other options are lower case, and it's a lot easier to type "parallel". I took the liberty of updating the CSR. If you're ok with it, you might want to change variable names and such, plus of course JMap.usage. ?????????????????????? ?????> ??????????????????????????? >????????? Thanks, ??????????????????????????? >????????? Paul ??????????????????????????? > ??????????????????????????? >????????? On 4/22/20, 2:29 AM, "serviceability-dev on behalf of linzang(??)" wrote: ??????????????????????????? > ??????????????????????????? >???? ?????????Dear Stefan, ??????????????????????????? > ??????????????????????????? >????????????????????? Thanks a lot! I agree with you to decouple the heap inspection code with GC's. ??????????????????????????? >????????????????????? I will start? from your POC code, may discuss with you later. ??????????????????????????? > ??????????????????????????? > ??????????????????????????? >????????????? BRs, ??????????????????????????? >????????????? Lin ??????????????????????????? > ??????????????????????????? >?? ???????????On 2020/4/22, 5:14 PM, "Stefan Karlsson" wrote: ??????????????????????????? > ??????????????????????????? >????????????????? Hi Lin, ??????????????????????????? > ??????????????????????????? >????????????????? I took a look at this earlier and saw that the heap inspection code is ??????????????????????????? >????????????????? strongly coupled with the CollectedHeap and G1CollectedHeap. I'd prefer ??????????????????????????? >????????????????? if we'd abstract this away, so that the GCs only provide a "parallel ??????????????????????????? >????????????????? object iteration" interface, and the heap inspection code is kept elsewhere. ??????????????????????????? > ??????????????????????????? >????????????????? I started experimenting with doing that, but other higher-priority (to ??????????????????????????? >????????????????? me) tasks have had to take precedence. ??????????????????????????? > ??????????????????????????? >????????????????? I've uploaded my work-in-progress / proof-of-concept: ??????????????????????????? >??????????????????? https://cr.openjdk.java.net/~stefank/8215624/webrev.01.delta/ ??????????????????????????? >??????????????????? https://cr.openjdk.java.net/~stefank/8215624/webrev.01/ ?????????????????? ?????????> ??????????????????????????? >????????????????? The current code doesn't handle the lifecycle (deletion) of the ??????????????????????????? >????????????????? ParallelObjectIterators. There's also code left unimplemented in around ??????????????????????????? >????????????????? CollectedHeap::run_task. However, I think this could work as a basis to ????????????????? ??????????>????????????????? pull out the heap inspection code out of the GCs. ??????????????????????????? > ??????????????????????????? >????????????????? Thanks, ??????????????????????????? >????????????????? StefanK ??????????????????????????? > ?????? ?????????????????????>????????????????? On 2020-04-22 02:21, linzang(??) wrote: ??????????????????????????? >????????????????? > Dear all, ??????????????????????????? >????????????????? >?????? May I ask you help to review? This RFR has been there for quite a while. ??????????????????????????? >????????????????? >?????? Thanks! ??????????????????????????? >????????????????? > ??????????????????????????? >????????????????? > BRs, ??????????????????????????? >????????????????? > Lin ????????????????????????? ??>????????????????? > ??????????????????????????? >????????????????? > > On 2020/3/16, 5:18 PM, "linzang(??)" wrote:> ??????????????????????????? >????????????????? > ??????????????????????????? >????????????????? >>??? Just update a new path, my preliminary measure show about 3.5x speedup of jmap histo on a nearly full 4GB G1 heap (8-core platform with parallel thread number set to 4). ??????????????????????????? >????????????????? >>???? webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_02/ ??????????????????????????? >????????????????? >>???? bug: https://bugs.openjdk.java.net/browse/JDK-8215624 ??????????????????????????? >????????????????? >>???? CSR: https://bugs.openjdk.java.net/browse/JDK-8239290 ?????????? ?????????????????>????????????????? >>???? BRs, ??????????????????????????? >????????????????? >>?????? Lin ??????????????????????????? >????????????????? >>?????? > On 2020/3/2, 9:56 PM, "linzang(??)" wrote: ???????????????????????? ???>????????????????? >>?????? > ??????????????????????????? >????????????????? >>?????? >??? Dear all, ??????????????????????????? >????????????????? >>?????? >????????? Let me try to ease the reviewing work by some explanation :P ??????????????????????? ????>????????????????? >>?????? >????????? The patch's target is to speed up jmap -histo for heap iteration, from my experience it is necessary for large heap investigation. E.g in bigData scenario I have tried to conduct jmap -histo against 180GB heap, it does take quite a while. ??????????????????????????? >????????????????? >>?????? >????????? And if my understanding is corrent, even the jmap -histo without "live" option does heap inspection with heap lock acquired. so it is very likely to block mutator thread in allocation-sensitive scenario. I would say the faster the heap inspection does, the shorter the mutator be blocked. This is parallel iteration for jmap is necessary. ??????????????????????????? >????????????????? >>?????? >????????? I think the parallel heap inspection should be applied to all kind of heap. However, consider the heap layout are different for? GCs, much time is required to understand all kinds of the heap layout to make the whole change. IMO, It is not wise to have a huge patch for the whole solution at once, and it is even harder to review it. So I plan to implement it incrementally, the first patch (this one) is going to confirm the implemention detail of how jmap accept the new option, passes it to attachListener of the jvm process and then how to make the parallel inspection closure be generic enough to make it easy to extend to different heap layout. And also how to implement the heap inspection in specific gc's heap. This patch use G1's heap as the begining. ??????????????????????????? >??????????????? ??>>?????? >????????? This patch actually do several things: ??????????????????????????? >????????????????? >>?????? >????????? 1. Add an option "parallelThreadNum=" to jmap -histo, the default behavior is to set N to 0, means let's JVM decide how many threads to use for heap inspection. Set this option to 1 will disable parallel heap inspection. (more details in CSR: https://bugs.openjdk.java.net/browse/JDK-8239290) ??????????????????????????? >????????????????? >>?????? >????????? 2. Make a change in how Jmap passing arguments, changes in http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java.udiff.html, originally it pass options as separate arguments to attachListener, this patch change to that all options be compose to a single string. So the arg_count_max in attachListener.hpp do not need to be changed, and hence avoid the compatibility issue, as disscussed at https://mail.openjdk.java.net/pipermail/serviceability-dev/2019-March/027334.html ??????????????????????????? >????????????????? >>?????? >???????? 3. Add an abstract class ParHeapInspectTask in heapInspection.hpp / heapInspection.cpp, It's work(uint worker_id) method prepares the data structure (KlassInfoTable) need for every parallel worker thread, and then call do_object_iterate_parallel() which is heap specific implementation. I also added some machenism in KlassInfoTable to support parallel iteration, such as merge(). ??????????????????????????? >????????????????? >>?????? >????? ??4. In specific heap (G1 in this patch), create a subclass of ParHeapInspectTask, implement the do_object_iterate_parallel() for parallel heap inspection. For G1, it simply invoke g1CollectedHeap's object_iterate_parallel(). ??????????????????????????? > ?????????????????>>?????? >??????? 5. Add related test. ??????????????????????????? >????????????????? >>?????? >??????? 6. it may be easy to extend this patch for other kinds of heap by creating subclass of ParHeapInspectTask and implement the do_object_iterate_parallel(). ??????????????????????????? >????????????????? >>?????? > ??????????????????????????? >????????????????? >>?????? >??? Hope these info could help on code review and initate the discussion :-) ??????????????????????????? >??????????????? ??>>?????? >??? Thanks! ??????????????????????????? >????????????????? >>?????? > ??????????????????????????? >????????????????? >>?????? >??? BRs, ??????????????????????????? >????????????????? >>?????? >??? Lin ??????????????????????????? >????????????? ????>>?????? >??? >On 2020/2/19, 9:40 AM, "linzang(??)" wrote:. ??????????????????????????? >????????????????? >>?????? >??? > ??????????????????????????? >????????????????? >>?????? >??? >? Re-post this RFR with correct enhancement number to make it trackable. ??????????????????????????? >????????????????? >>?????? >??? >? please ignore the previous wrong post. sorry for troubles. ??????????????????????????? >????????????????? >>?????? >??? > ??????????????????????????? >???????????? ?????>>?????? >??? >?? webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/ ??????????????????????????? >????????????????? >>?????? >??? >??? Hi bug: https://bugs.openjdk.java.net/browse/JDK-8215624 ??????????????????????????? >?????? ???????????>>?????? >??? >??? CSR: https://bugs.openjdk.java.net/browse/JDK-8239290 ??????????????????????????? >????????????????? >>?????? >??? >??? -------------- ??????????????????????????? >????????????????? >>?????? >??? >??? Lin ???????????????????? ???????>????????????????? >>?????? >??? >??? >Hi Lin, ??????????????????????????? >????????????????? >?? >???? >??? >??? > ??????????????????????????? >????????????????? >>?????? >??? >??? >Could you, please, re-post your RFR with the right enhancement number in ??????????????????????????? >????????????????? >>?????? >??? >??? >the message subject? ??????????????????????????? >????????????????? >>?????? >??? >??? >It will be more trackable this way. ??????????????????????????? >????????????????? >>?????? >??? >??? > ??????????????????????????? >????????????????? >>?????? >??? >??? >Thanks, ????????? ??????????????????>????????????????? >>?????? >??? >??? >Serguei ??????????????????????????? >????????????????? >>?????? >??? >??? > ??????????????????????????? >????????????????? >>?????? >??? >??? > ??????????????????????????? >????????????????? >>????? ?>??? >??? >On 2/17/20 10:29 PM, linzang(??) wrote: ??????????????????????????? >????????????????? >>?????? >??? >??? >> Dear David, ??????????????????????????? >????????????????? >>?????? >??? >??? >>??????? Thanks a lot! ??????????????????????????? >??? ??????????????>>?????? >??? >??? >>?????? I have updated the refined code to http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_01/. ??????????????????????????? >????????????????? >>?????? >??? >??? >>??????? IMHO the parallel heap inspection can be extended to all kinds of heap as long as the heap layout can support parallel iteration. ??????????????????????????? >????????????????? >>?????? >??? >??? >>??????? Maybe we can firstly use this webrev to discuss how to implement it, because I am not sure my current implementation is an appropriate way to communicate with collectedHeap, then we can extend the solution to other kinds of heap. ??????????????????????????? >????????????????? >>?????? >??? >??? >> ??????????????????????????? >????????????? ????>>?????? >??? >??? >> Thanks, ??????????????????????????? >????????????????? >>?????? >??? >??? >> -------------- ??????????????????????????? >????????????????? >>?????? >??? >??? >> Lin ??????????????????????????? >????????????????? >>?????? >??? >?? ?>>> Hi Lin, ??????????????????????????? >????????????????? >>?????? >??? >??? >>> ??????????????????????????? >????????????????? >>?????? >??? >??? >>> Adding in hotspot-gc-dev as they need to see how this interacts with GC ??????????????????????????? >? ????????????????>>?????? >??? >??? >>> worker threads, and whether it needs to be extended beyond G1. ??????????????????????????? >????????????????? >>?????? >??? >??? >>> ??????????????????????????? >????????????????? >>?????? >??? >?? >>> I happened to spot one nit when browsing: ??????????????????????????? >????????????????? >>?????? >??? >??? >>> ??????????????????????????? >????????????????? >>?????? >??? >??? >>> src/hotspot/share/gc/shared/collectedHeap.hpp ??????????????????????????? >????????????? ????>>?????? >??? >??? >>> ??????????????????????????? >????????????????? >>?????? >??? >??? >>> +?? virtual bool run_par_heap_inspect_task(KlassInfoTable* cit, ??????????????????????????? >????????????????? >>?????? >??? >??? >>> +??????????????????????? ??????????????????BoolObjectClosure* filter, ??????????????????????????? >????????????????? >>?????? >??? >??? >>> +????????????????????????????????????????? size_t* missed_count, ??????????????????????????? >????????????????? >>?????? >??? >??? >>> +???? ?????????????????????????????????????size_t thread_num) { ??????????????????????????? >????????????????? >>?????? >??? >??? >>> +???? return NULL; ??????????????????????????? >????????????????? >>?????? >??? >??? >>> ??????????????????????????? >????????????????? >>?????? >??? >??? >>> s/NULL/false/ ??????????????????????????? >????????????????? >>?????? >??? >??? >>> ??????????????????????????? >?? ???????????????>>?????? >??? >??? >>> Cheers, ??????????????????????????? >????????????????? >>?????? >??? >??? >>> David ??????????????????????????? >????????????????? >?? >???? >??? >??? >>> ??????????????????????????? >????????????????? >>?????? >??? > ???>>> On 18/02/2020 2:15 pm, linzang(??) wrote: ??????????????????????????? >????????????????? >>?????? >??? >??? >>>> Dear All, ??????????????????????????? >????????????????? >>?????? >??? >??? >>>>???????? May I ask your help to review the follow changes: ??????????????????????????? >????????????????? >>?????? >??? >??? >>>>???????? webrev: ??????????????????????????? >????????????????? >>?????? >??? >??? >>>> http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_00/ ??????????????????????????? >????????????????? >>?????? >??? >??? >>>>????? bug: https://bugs.openjdk.java.net/browse/JDK-8215624 ??????????????????????????? >????????????????? >>?????? >??? >??? >>>>????? related CSR: https://bugs.openjdk.java.net/browse/JDK-8239290 ???????????????? ???????????>????????????????? >>?????? >??? >??? >>>>???????? This patch enable parallel heap inspection of G1 for jmap histo. ??????????????????????????? >????????????????? >>?????? >??? >??? >>>>???????? my simple test shown it can speed up 2x of jmap -histo with ??????????????????????????? >????????????????? >>?????? >??? >??? >>>> parallelThreadNum set to 2 for heap at ~500M on 4-core platform. ??????????????????????????? >????????????????? >>?????? >??? >??? >>>> ??????????????????????????? >????????? ????????>>?????? >??? >??? >>>> ------------------------------------------------------------------------ ??????????????????????????? >????????????????? >>?????? >??? >??? >>>> BRs, ??????????????????????????? >????????????????? >>?????? >??? >??? >>>> Lin ??????????????????????????? >????????????????? >>?????? >??? >??? >> > ??????????????????????????? >????????????????? >>?????? >??? >??? > ??????????????????????????? >????????????????? > ??????????????????????????? >????????????????? > ?????????????????? ?????????>????????????????? > ??????????????????????????? > ??????????????????????????? > ??????????????????????????? > ??????????????????????????? > ??????????????????????????? > ??????????????????????????? > ? ? ? ? ? ? ? From david.holmes at oracle.com Thu Jul 23 04:41:13 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 23 Jul 2020 14:41:13 +1000 Subject: RFR: 8248362: JVMTI frame operations should use Thread-Local Handshake In-Reply-To: <803f35f4-faab-a195-d513-b43f24027b89@oss.nttdata.com> References: <5ef612da-7682-154f-8c40-eaf4ed92b0f8@oracle.com> <803f35f4-faab-a195-d513-b43f24027b89@oss.nttdata.com> Message-ID: <090507eb-80c3-ccb8-fb1a-44f2ddaa7cae@oracle.com> On 23/07/2020 1:13 pm, Yasumasa Suenaga wrote: > Hi David, > > Thanks for your comment! > > On 2020/07/23 11:01, David Holmes wrote: >> Hi Yasumasa, >> >> On 23/07/2020 12:38 am, Yasumasa Suenaga wrote: >>> Hi all, >>> >>> Please review this change: >>> >>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8248362 >>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8248362/webrev.00/ >>> >>> Migrate JVMTI frame operations to Thread-Local Handshakes from VM >>> Operations. >>> >>> ???? - VM_GetFrameCount >>> ???? - VM_GetFrameLocation >>> >>> They affects both GetFrameCount() and GetFrameLocation() JVMTI >>> functions. >> >> Your changes all seem fine. >> >> But I'm a bit confused about the existing code. In >> JvmtiEnv::GetFrameLocation we have: >> >> ??? if (java_thread == JavaThread::current()) { >> ????? err = get_frame_location(java_thread, depth, method_ptr, >> location_ptr); >> >> but then in get_frame_location we have: >> >> ??? assert((SafepointSynchronize::is_at_safepoint() || >> ??????????? java_thread->is_thread_fully_suspended(false, &debug_bits)), >> ??????????? "at safepoint or target thread is suspended"); >> >> and that assert must surely fire if called by the current thread for >> itself! ??? > > is_thread_fully_suspended() returns true when it calles from current > thread, so this assert would not fire. I had not realized it was defined that way. Seems like a bit of a kludge to avoid making the call conditional on a check of the current thread. :( Cheers, David > > Yasumasa > > >> Thanks, >> David >> >>> This change has passed all tests on submit repo >>> (mach5-one-ysuenaga-JDK-8248362-20200722-1249-12859056), and >>> vmTestbase/nsk/{jdi,jdw,jvmti}, serviceability/{jdwp,jvmti} . >>> >>> >>> Thanks, >>> >>> Yasumasa From chris.plummer at oracle.com Thu Jul 23 05:25:09 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 22 Jul 2020 22:25:09 -0700 Subject: RFR(M): 8247515: OSX pc_to_symbol() lookup does not work with core files In-Reply-To: <401dbc7a-dd2f-5612-74d3-3dbf6641dae9@oracle.com> References: <092ef5a8-c973-e377-de91-b2002a1b8fe7@oracle.com> <38e1887e-a646-abc0-0b7e-53339af1d684@oracle.com> <3c31622b-831e-d46f-daa6-d641cbf755b1@oracle.com> <18f7a1d8-3802-c7da-c981-eac8cf0953eb@oracle.com> <1e9e4786-79f8-c4e0-efe7-f1a8f369d2c6@oracle.com> <4c09d200-057b-e878-733a-524860453288@oracle.com> <401dbc7a-dd2f-5612-74d3-3dbf6641dae9@oracle.com> Message-ID: <8f1cec73-4397-0489-6031-a813d9bd23c6@oracle.com> An HTML attachment was scrubbed... URL: From thomas.schatzl at oracle.com Thu Jul 23 09:39:59 2020 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 23 Jul 2020 11:39:59 +0200 Subject: [15?] RFR (S): 8249192: MonitorInfo stores raw oops across safepoints In-Reply-To: References: <3f838c8a-6e70-cd94-1753-8abe269b7cdf@oracle.com> <528ade6c-059a-d9af-b0c7-74d7ac31b29a@oracle.com> <14512a57-8d4d-5199-ed16-743b5da8a10d@oracle.com> <893f862f-093a-5168-e554-1f4ffe129aea@oracle.com> <07904b7e-8d40-8c9e-acc1-c713c1588b11@oracle.com> Message-ID: <4ad72264-2269-fcc1-b2fc-0568a993ca7d@oracle.com> Hi Dan and Serguei, thanks for your reviews. On 22.07.20 19:04, Daniel D. Daugherty wrote: >> jdk15: >> >> http://cr.openjdk.java.net/~tschatzl/8249192/webrev.jdk15.2/ (full) > > src/hotspot/share/prims/jvmtiEnvBase.cpp > ??? old L1029: ????? ResourceMark rm; > ??????? It's not clear (to me anyway) why this ResourceMark is removed. > > ??????? Update: I saw the discussion of "ResourceMark rm" in JDK15 versus > ??????? "ResourceMark rm(current_thread)" in JDK16, but that doesn't tell > ??????? me why it was necessary to remove that ResourceMark. The method that is guarded by this ResourceMark contains the necessary ResourceMark itself, so I removed it. > > src/hotspot/share/prims/stackwalk.cpp > ??? L291: ??? ResourceMark rm; > ??? L292: ??? HandleMark hm; > ??????? Since there's a TRAPS parameter, these should be 'rm(THREAD)' and > ??????? 'hm(THREAD)'. > > src/hotspot/share/runtime/biasedLocking.cpp > ??? No comments. > > src/hotspot/share/runtime/deoptimization.cpp > ??? No comments. > > src/hotspot/share/runtime/vframe.cpp > ??? L461: ? _lock? = lock; > ??????? nit - extra space before '='. > > src/hotspot/share/runtime/vframe.hpp > ??? L32: #include "runtime/handles.inline.hpp" > ??????? nit - new include is out of order; should be after frame.hpp. > > src/hotspot/share/runtime/vframeArray.cpp > ??? No comments. > > src/hotspot/share/runtime/vframe_hp.cpp > ??? Skipped - no changes. > > src/hotspot/share/services/threadService.cpp > ??? No comments. > All fixed, and incorporating Serguei's changes in the other email as well. jdk16: http://cr.openjdk.java.net/~tschatzl/8249192/webrev.3/ (full) http://cr.openjdk.java.net/~tschatzl/8249192/webrev.2_to_3/ (diff) jdk15: http://cr.openjdk.java.net/~tschatzl/8249192/webrev.jdk15.3/ (full) http://cr.openjdk.java.net/~tschatzl/8249192/webrev.jdk15.2_to_3/ (diff) Note that the jdk15 change will only go into 15.0.2 as discussion with the release team showed that the change is too risky for earlier releases. See the relevant CR comment for details. Thanks, Thomas From coleen.phillimore at oracle.com Thu Jul 23 11:36:20 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 23 Jul 2020 07:36:20 -0400 Subject: RFR (S) 8249822: SymbolPropertyTable creates an extra OopHandle per entry In-Reply-To: References: <84643f6c-0e05-061e-b6e3-a38d09ed973a@oracle.com> Message-ID: <2e00b001-0749-8680-71e9-a7396f9bb48b@oracle.com> On 7/22/20 8:53 PM, serguei.spitsyn at oracle.com wrote: > Hi Coleen, > > The fix looks good to me. Thank you for looking at this Serguei. > > > On 7/22/20 13:55, coleen.phillimore at oracle.com wrote: >> Summary: Add an assert to OopHandle assigment operator to catch >> leaking OopHandles, and fix code accordingly. >> >> There are some jvmtiRedefineClasses.cpp changes here - basically, I >> moved the RedefineVerifyMark and comments into >> jvmtiRedefineClasses.cpp because I needed a Handle. > > I think, the jvmtiRedefineClasses.cpp is a better place for the > RedefineVerifyMark. > But could you, please, explain a little bit more why this move was > necessary? > Why Handle can not be used in the jvmtiThreadState.cpp? I did have a patch where I moved the constructor and destructor in jvmtiThreadState.cpp but thought it made more sense just to move the whole class since nothing else would ever use it.?? I couldn't put the code change in jvmtiThreadState.hpp in place because I'd need to include handles.inline.hpp there.? It seems like jvmtiRedefineClasses.cpp already included all the files it needed, so that's where I moved it. Thanks, Coleen > > Thanks, > Serguei > > >> Ran tier1-6 tests and tier1 on all Oracle platforms. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8249822.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8249822 >> >> Thanks, >> Coleen > From david.holmes at oracle.com Thu Jul 23 12:05:53 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 23 Jul 2020 22:05:53 +1000 Subject: [15?] RFR (S): 8249192: MonitorInfo stores raw oops across safepoints In-Reply-To: <4ad72264-2269-fcc1-b2fc-0568a993ca7d@oracle.com> References: <3f838c8a-6e70-cd94-1753-8abe269b7cdf@oracle.com> <528ade6c-059a-d9af-b0c7-74d7ac31b29a@oracle.com> <14512a57-8d4d-5199-ed16-743b5da8a10d@oracle.com> <893f862f-093a-5168-e554-1f4ffe129aea@oracle.com> <07904b7e-8d40-8c9e-acc1-c713c1588b11@oracle.com> <4ad72264-2269-fcc1-b2fc-0568a993ca7d@oracle.com> Message-ID: <1a148b15-ddf6-51b1-1b60-87528f101559@oracle.com> Hi Thomas, Incrementals looks good to me. Thanks, David ----- On 23/07/2020 7:39 pm, Thomas Schatzl wrote: > Hi Dan and Serguei, > > ? thanks for your reviews. > > On 22.07.20 19:04, Daniel D. Daugherty wrote: >>> jdk15: >>> >>> http://cr.openjdk.java.net/~tschatzl/8249192/webrev.jdk15.2/ (full) >> >> src/hotspot/share/prims/jvmtiEnvBase.cpp >> ???? old L1029: ????? ResourceMark rm; >> ???????? It's not clear (to me anyway) why this ResourceMark is removed. >> >> ???????? Update: I saw the discussion of "ResourceMark rm" in JDK15 >> versus >> ???????? "ResourceMark rm(current_thread)" in JDK16, but that doesn't >> tell >> ???????? me why it was necessary to remove that ResourceMark. > > The method that is guarded by this ResourceMark contains the necessary > ResourceMark itself, so I removed it. > >> >> src/hotspot/share/prims/stackwalk.cpp >> ???? L291: ??? ResourceMark rm; >> ???? L292: ??? HandleMark hm; >> ???????? Since there's a TRAPS parameter, these should be 'rm(THREAD)' >> and >> ???????? 'hm(THREAD)'. >> >> src/hotspot/share/runtime/biasedLocking.cpp >> ???? No comments. >> >> src/hotspot/share/runtime/deoptimization.cpp >> ???? No comments. >> >> src/hotspot/share/runtime/vframe.cpp >> ???? L461: ? _lock? = lock; >> ???????? nit - extra space before '='. >> >> src/hotspot/share/runtime/vframe.hpp >> ???? L32: #include "runtime/handles.inline.hpp" >> ???????? nit - new include is out of order; should be after frame.hpp. >> >> src/hotspot/share/runtime/vframeArray.cpp >> ???? No comments. >> >> src/hotspot/share/runtime/vframe_hp.cpp >> ???? Skipped - no changes. >> >> src/hotspot/share/services/threadService.cpp >> ???? No comments. >> > > All fixed, and incorporating Serguei's changes in the other email as well. > > jdk16: > http://cr.openjdk.java.net/~tschatzl/8249192/webrev.3/ (full) > http://cr.openjdk.java.net/~tschatzl/8249192/webrev.2_to_3/ (diff) > > jdk15: > http://cr.openjdk.java.net/~tschatzl/8249192/webrev.jdk15.3/ (full) > http://cr.openjdk.java.net/~tschatzl/8249192/webrev.jdk15.2_to_3/ (diff) > > Note that the jdk15 change will only go into 15.0.2 as discussion with > the release team showed that the change is too risky for earlier > releases. See the relevant CR comment for details. > > Thanks, > ? Thomas > From daniel.fuchs at oracle.com Thu Jul 23 13:36:06 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Thu, 23 Jul 2020 14:36:06 +0100 Subject: jmx-dev MBean attribute deprecation In-Reply-To: References: Message-ID: <3197874e-e2be-891a-74c3-efc2013ad04c@oracle.com> Hi Milan, JMX is being maintained by serviceability these days. I am forwarding your question there. This is an interesting observation. JMX makes it possible to define and use annotations on methods/getters/setters, to add key/value pairs to the Descriptor of the corresponding MBeanFeatureInfo. The mechanism should probably be extended so that the standard @Deprecated annotation is also reflected in the Descriptor. best regards, -- daniel On 23/07/2020 13:25, Milan Mimica wrote: > Hello list > > With deprecation of OperatingSystem#SystemCpuLoad in favor of > OperatingSystem#CpuLoad [1] an issue raised with generic tools that > query all MBean attributes. Querying CPU usage is not side-effect-free: > the immediate subsequent query for CPU load will often return a skewed > value (on Linux at least, haven't checked others)[2]. This results in > exported data where either new or deprecated MBean value is wrong and > misleading. > > There are many tools and libraries that generically export JMX metrics, > so I think this should be addressed somehow. Maybe we could add a > property in MBeanAttributeInfo which would mark the attribute as > deprecated? These tools could then skip such metrics. > > > [1] > https://docs.oracle.com/en/java/javase/14/docs/api/jdk.management/com/sun/management/OperatingSystemMXBean.html#getSystemCpuLoad() > [2] > https://medium.com/@milan.mimica/the-java-cpu-usage-observer-effect-18808b18323f > > -- > Milan Mimica > From goetz.lindenmaier at sap.com Thu Jul 23 14:19:57 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 23 Jul 2020 14:19:57 +0000 Subject: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents In-Reply-To: References: <1f8a3c7a-fa0f-b5b2-4a8a-7d3d8dbbe1b5@oracle.com> <4b56a45c-a14c-6f74-2bfd-25deaabe8201@oracle.com> <5271429a-481d-ddb9-99dc-b3f6670fcc0b@oracle.com> Message-ID: Hi Richard, Thanks for your two further explanations in the other thread. That made the points clear to me. > > I was not that happy with the names saying not_global_escape > > and similar. I now agreed you have to use the terms of the escape > > analysis (NoEscape ArgEscape= throughout the runtime code. I'm still not happy with > > the 'not' in the term, I always try to expand the name to some > > sentence with a negated verb, but it makes no sense. > > For example, "has_not_global_escape_in_scope" expands to > > "Hasn't a global escape in its scope." in my thinking, which makes > > no sense. You probably mean > > "Has not-global escape in its scope." or "Has {ArgEscape|NoEscape} > > in its scope." > > > C2 is using the word "non" in this context, e.g., here > > alloc->is_non_escaping. > > There is also ConnectionGraph::not_global_escape() That talks about a single node that represents a single Object. An object has a single state wrt. ea. You use the term for safepoint which tracks a set of objects. Here, has_not_global_excape can mean 1. None of the several objects does escape globaly. 2. There is at least one object that escapes globaly. > > non obviously negates the adjective 'global', > > non-global or nonglobal even is a English term I find in the > > net. > > So what about "has_non_global_escape_in_scope?" > > And what about has_ea_local_in_scope? That's good. Please document somewhere that Ea_local == ArgEscape | NoEscape. That's what it is, right? > > Does jvmti specify that the same limits are used ...? > > ok on your side. > > I don't know and didn't find anything in a quick search. Ok, not your business. > > > jvmtiEnvBase.cpp ok > > jvmtiImpl.h|cpp ok > > jvmtiTagMap.cpp ok > > whitebox.cpp ok > > > deoptimization.cpp > > > line 177: Please break line > > line 246, 281: Please break line > > 1578, 1583, 1589, 1632, 1649, 1651 Break line > > > 1651: You use 'non'-terms, too: non-escaping :) > > I know :) At least here it is wrong I'd say. "...has to be a not escaping obj..." > sounds better > (hopefully not only to my german ears). I thought the term non-escpaing makes it quite clear. I just wanted to point out that using non above would be similar to the wording here. > > IterateHeapWithEscapeAnalysisEnabled.java > > > line 415: > > msg("wait until target thread has set testMethod_result"); > > while (testMethod_result == 0) { > > Thread.sleep(50); > > } > > Might the test run into timeouts at this place? > > The field is volatile, i.e. it will be reloaded > > in each iteration. But will dontinline_testMethod > > write it back to main memory in time? > > You mean, the test could hang in that loop for a couple of minutes? I don't > think so. There are cache coherence protocols in place which will invalidate > stale data very timely. Ok, anyways, it would only be a hanging test. > > Ok. I've removed quite a lot of the occurrances. > > > Also, I like full sentences in comments. > > Especially for me as foreign speaker, this makes > > things much more clear. I.e., I try to make it > > a real sentence with articles, capitalized and a > > dot at the end if there is a subject and a verb > > in first place. > > E.g., jvmtiEnvBase.cpp:1327 > > Are you referring to the following? > (from > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.6/src/hots > pot/share/prims/jvmtiEnvBase.cpp.frames.html) > > 1326 > 1327 // If the frame is a compiled one, need to deoptimize it. > 1328 if (vf->is_compiled_frame()) { > > This line 1327 is preexisting. Sorry, wrong line number again. I think I meant 1333 // eagerly reallocate scalar replaced objects. But I must admit, the subject is missing. It's one of these imperative sentences where the subject is left out, which are used throughout documentation. Bad example, but still a correct sentence, so qualifies for punctuation? Best regards, Goetz. From erik.osterlund at oracle.com Thu Jul 23 14:40:55 2020 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 23 Jul 2020 16:40:55 +0200 Subject: RFR (S) 8249822: SymbolPropertyTable creates an extra OopHandle per entry In-Reply-To: <84643f6c-0e05-061e-b6e3-a38d09ed973a@oracle.com> References: <84643f6c-0e05-061e-b6e3-a38d09ed973a@oracle.com> Message-ID: Hi Coleen, Looks good. One thing though: in the replace() function, assert that the obj_ptr != NULL instead of silently doing nothing. Don't need another webrev. Thanks, /Erik On 2020-07-22 22:55, coleen.phillimore at oracle.com wrote: > Summary: Add an assert to OopHandle assigment operator to catch > leaking OopHandles, and fix code accordingly. > > There are some jvmtiRedefineClasses.cpp changes here - basically, I > moved the RedefineVerifyMark and comments into > jvmtiRedefineClasses.cpp because I needed a Handle. > > Ran tier1-6 tests and tier1 on all Oracle platforms. > > open webrev at http://cr.openjdk.java.net/~coleenp/2020/8249822.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8249822 > > Thanks, > Coleen From daniel.daugherty at oracle.com Thu Jul 23 14:43:32 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 23 Jul 2020 10:43:32 -0400 Subject: RFR: 8248362: JVMTI frame operations should use Thread-Local Handshake In-Reply-To: References: Message-ID: <690a1a26-92fe-866b-7402-fc275c423145@oracle.com> On 7/22/20 9:25 PM, David Holmes wrote: > Hi Dan, > > I just want to respond to one aspect of your response ... > > On 23/07/2020 11:04 am, Daniel D. Daugherty wrote: >> On 7/22/20 10:38 AM, Yasumasa Suenaga wrote: >>> Hi all, >>> >>> Please review this change: >>> >>> ? JBS: https://bugs.openjdk.java.net/browse/JDK-8248362 >>> ? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8248362/webrev.00/ >> >> src/hotspot/share/prims/jvmtiEnv.cpp >> ???? L1755???? // JVMTI get java stack frame location at safepoint. >> ???????? You missed updating this one. Perhaps: >> >> ?????????????? // JVMTI get java stack frame location via direct >> handshake. >> >> src/hotspot/share/prims/jvmtiEnvBase.cpp >> ???? L1563: ? JavaThread* jt = _state->get_thread(); >> ???? L1564: ? assert(target == jt, "just checking"); >> ???????? This code is different than the other closures. It might be >> worth >> ???????? a comment to explain why. I don't remember why >> VM_GetFrameCount had >> ???????? to use the JvmtiThreadState to fetch the JavaThread. Serguei >> might >> ???????? remember. >> >> ???????? It could be that we don't need the _state field anymore >> because of >> ???????? the way that handshakes work (and provide the JavaThread* to >> the >> ???????? do_thread() function). Your choice on whether to deal with >> that as >> ???????? part of this fix or following with another RFE. >> >> ???????? Update: Uggg.... the get_frame_count() function takes >> JvmtiThreadState >> ???????? as a parameter. This is very much entangled... sigh... we >> should >> ???????? definitely look at untangling this in another RFE... >> >> ???? old L1565: ? ThreadsListHandle tlh; >> ???? new L1565: ? if (!jt->is_exiting() && jt->threadObj() != NULL) { >> >> ???????? Please consider add this comment above L1565: >> ????????????????? // ThreadsListHandle is not needed due to direct >> handshake. >> >> ???????? Yes, I see that previous closures were added without that >> comment, >> ???????? but when I see "if (!jt->is_exiting() && jt->threadObj() != >> NULL)" >> ???????? I immediately wonder where the ThreadsListHandle is... >> Please consider >> ???????? adding the comment to the others also. > > I understand why, when looking at this change, you would like the > comment, but outside of the webrev the reference to ThreadsListHandle > doesn't really have any context. We need a TLH anywhere there is a > risk the target thread might exit while we're interacting with it, but > that can't happen in the context of a direct handshake operation with > the target thread because the TLH exists in our caller. This is the > new normal for code that executes as part of a direct handshake. Agreed. I have to adjust to the "new normal". Dan > > David > ----- > >> ???? old L1574: ? ThreadsListHandle tlh; >> ???? new L1574: ? if (!jt->is_exiting() && jt->threadObj() != NULL) { >> >> ???????? Please consider add this comment above L1574: >> ????????????????? // ThreadsListHandle is not needed due to direct >> handshake. >> >> src/hotspot/share/prims/jvmtiEnvBase.hpp >> ???? L580: // HandshakeClosure to frame location. >> ???????? typo - s/to frame/to get frame/ >> >> src/hotspot/share/prims/jvmtiThreadState.cpp >> ???? No comments on the changes. >> >> ???? For the above comments about VM_GetFrameCount, understanding why >> ???? JvmtiThreadState::count_frames() has to exist in JvmtiThreadState >> ???? will go along way towards untangling the mess. >> >> src/hotspot/share/runtime/vmOperations.hpp >> ???? No comments. >> >> >> Thumbs up. I only have nits above. If you choose to make the above >> changes, I do not need to see another webrev. >> >> Dan >> >> >> >>> >>> Migrate JVMTI frame operations to Thread-Local Handshakes from VM >>> Operations. >>> >>> ??? - VM_GetFrameCount >>> ??? - VM_GetFrameLocation >>> >>> They affects both GetFrameCount() and GetFrameLocation() JVMTI >>> functions. >>> >>> This change has passed all tests on submit repo >>> (mach5-one-ysuenaga-JDK-8248362-20200722-1249-12859056), and >>> vmTestbase/nsk/{jdi,jdw,jvmti}, serviceability/{jdwp,jvmti} . >>> >>> >>> Thanks, >>> >>> Yasumasa >> From coleen.phillimore at oracle.com Thu Jul 23 14:46:07 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 23 Jul 2020 10:46:07 -0400 Subject: RFR (S) 8249822: SymbolPropertyTable creates an extra OopHandle per entry In-Reply-To: References: <84643f6c-0e05-061e-b6e3-a38d09ed973a@oracle.com> Message-ID: Thanks Erik. I added the assert to replace() and reran tier1 and 3 tests. Coleen On 7/23/20 10:40 AM, Erik ?sterlund wrote: > Hi Coleen, > > Looks good. One thing though: in the replace() function, assert that > the obj_ptr != NULL instead of silently doing nothing. > > Don't need another webrev. > > Thanks, > /Erik > > On 2020-07-22 22:55, coleen.phillimore at oracle.com wrote: >> Summary: Add an assert to OopHandle assigment operator to catch >> leaking OopHandles, and fix code accordingly. >> >> There are some jvmtiRedefineClasses.cpp changes here - basically, I >> moved the RedefineVerifyMark and comments into >> jvmtiRedefineClasses.cpp because I needed a Handle. >> >> Ran tier1-6 tests and tier1 on all Oracle platforms. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8249822.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8249822 >> >> Thanks, >> Coleen > From daniel.daugherty at oracle.com Thu Jul 23 15:35:57 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 23 Jul 2020 11:35:57 -0400 Subject: [15?] RFR (S): 8249192: MonitorInfo stores raw oops across safepoints In-Reply-To: <4ad72264-2269-fcc1-b2fc-0568a993ca7d@oracle.com> References: <3f838c8a-6e70-cd94-1753-8abe269b7cdf@oracle.com> <528ade6c-059a-d9af-b0c7-74d7ac31b29a@oracle.com> <14512a57-8d4d-5199-ed16-743b5da8a10d@oracle.com> <893f862f-093a-5168-e554-1f4ffe129aea@oracle.com> <07904b7e-8d40-8c9e-acc1-c713c1588b11@oracle.com> <4ad72264-2269-fcc1-b2fc-0568a993ca7d@oracle.com> Message-ID: <87846f7d-c213-733a-9a6f-96e5d02df247@oracle.com> > jdk16: > http://cr.openjdk.java.net/~tschatzl/8249192/webrev.2_to_3/ (diff) > > jdk15: > http://cr.openjdk.java.net/~tschatzl/8249192/webrev.jdk15.2_to_3/ (diff) The delta webrevs look just fine. Dan On 7/23/20 5:39 AM, Thomas Schatzl wrote: > Hi Dan and Serguei, > > ? thanks for your reviews. > > On 22.07.20 19:04, Daniel D. Daugherty wrote: >>> jdk15: >>> >>> http://cr.openjdk.java.net/~tschatzl/8249192/webrev.jdk15.2/ (full) >> >> src/hotspot/share/prims/jvmtiEnvBase.cpp >> ???? old L1029: ????? ResourceMark rm; >> ???????? It's not clear (to me anyway) why this ResourceMark is removed. >> >> ???????? Update: I saw the discussion of "ResourceMark rm" in JDK15 >> versus >> ???????? "ResourceMark rm(current_thread)" in JDK16, but that doesn't >> tell >> ???????? me why it was necessary to remove that ResourceMark. > > The method that is guarded by this ResourceMark contains the necessary > ResourceMark itself, so I removed it. > >> >> src/hotspot/share/prims/stackwalk.cpp >> ???? L291: ??? ResourceMark rm; >> ???? L292: ??? HandleMark hm; >> ???????? Since there's a TRAPS parameter, these should be >> 'rm(THREAD)' and >> ???????? 'hm(THREAD)'. >> >> src/hotspot/share/runtime/biasedLocking.cpp >> ???? No comments. >> >> src/hotspot/share/runtime/deoptimization.cpp >> ???? No comments. >> >> src/hotspot/share/runtime/vframe.cpp >> ???? L461: ? _lock? = lock; >> ???????? nit - extra space before '='. >> >> src/hotspot/share/runtime/vframe.hpp >> ???? L32: #include "runtime/handles.inline.hpp" >> ???????? nit - new include is out of order; should be after frame.hpp. >> >> src/hotspot/share/runtime/vframeArray.cpp >> ???? No comments. >> >> src/hotspot/share/runtime/vframe_hp.cpp >> ???? Skipped - no changes. >> >> src/hotspot/share/services/threadService.cpp >> ???? No comments. >> > > All fixed, and incorporating Serguei's changes in the other email as > well. > > jdk16: > http://cr.openjdk.java.net/~tschatzl/8249192/webrev.3/ (full) > http://cr.openjdk.java.net/~tschatzl/8249192/webrev.2_to_3/ (diff) > > jdk15: > http://cr.openjdk.java.net/~tschatzl/8249192/webrev.jdk15.3/ (full) > http://cr.openjdk.java.net/~tschatzl/8249192/webrev.jdk15.2_to_3/ (diff) > > Note that the jdk15 change will only go into 15.0.2 as discussion with > the release team showed that the change is too risky for earlier > releases. See the relevant CR comment for details. > > Thanks, > ? Thomas > From coleen.phillimore at oracle.com Thu Jul 23 16:58:53 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 23 Jul 2020 12:58:53 -0400 Subject: RFR (T) 8250042: Clean up methodOop and method_oop names from the code Message-ID: <85efc3ab-abbf-c5f2-9b7b-47fa516d9a2d@oracle.com> See bug for more details.? I've been running into these names a lot lately.?? Many of these names are in JVMTI. Tested with tier1 on all Oracle platforms and built on non-Oracle platforms. open webrev at http://cr.openjdk.java.net/~coleenp/2020/8250042.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8250042 Thanks, Coleen From daniil.x.titov at oracle.com Thu Jul 23 18:32:25 2020 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Thu, 23 Jul 2020 11:32:25 -0700 Subject: RFR: 8216324: GetClassMethods is confused by the presence of default methods in super interfaces Message-ID: <511BC62A-9B69-4456-ABD9-6E47191D7829@oracle.com> Hi Serguei and Alex, Please review a new version of the webrev [1] that includes the changes Serguei suggested. This version also has new test renamed from DefaultMethods.java to OverpassMethods.java to avoid warnings during the build about conflict with native library for runtime/jni/8033445/DefaultMethods.java test. [1] http://cr.openjdk.java.net/~dtitov/8216324/webrev.04/ Thank you, Daniil From: "serguei.spitsyn at oracle.com" Date: Tuesday, July 21, 2020 at 10:53 PM To: Daniil Titov , Alex Menkov , serviceability-dev Subject: Re: RFR: 8216324: GetClassMethods is confused by the presence of default methods in super interfaces Hi Daniil, Thank you for the update! It looks good to me. Just two more minor comments and no need for another webrev you address them. 2519 int skipped = 0; // skip default methods Saying "overpass methods" would be more precise. 47 printf("Enabled capability: maintain_original_method_order: true\n"); It is better to get rid of ": true" at the end (sorry, I missed this in my previous suggestion). Enabling capability means it is set to true. Thanks, Serguei On 7/21/20 20:25, Daniil Titov wrote: Hi Serguei and Alex, Please, review new version of the change [1] that includes changes as Serguei suggested. 114 default void default_method() { } // should never be seen The comment above is not clear. Should never be seen in what context? This method should not be included in the list of methods GetClassMethods returns for the sub-interface or implementing class. I don't think we want to comment each method in the test interfaces declared in this test when they should be seen in this test and when they should not. This information already exists in getclmthd007.cpp, thus I decided to omit this comment. Please see below the output from the new test. ----------messages:(4/215)---------- command: main -agentlib:DefaultMethods DefaultMethods reason: User specified action: run main/othervm/native -agentlib:DefaultMethods DefaultMethods Mode: othervm [/othervm specified] elapsed time (seconds): 0.241 ----------configuration:(0/0)---------- ----------System.out:(3/265)---------- Reflection getDeclaredMethods returned: [public abstract java.lang.String DefaultMethods$Child.method2()] JVMTI GetClassMethods returned: [public abstract java.lang.String DefaultMethods$Child.method2()] Test passed: Got expected output from JVMTI GetClassMethods! ----------System.err:(1/15)---------- STATUS:Passed. ----------messages:(4/276)---------- command: main -agentlib:DefaultMethods=maintain_original_method_order DefaultMethods reason: User specified action: run main/othervm/native -agentlib:DefaultMethods=maintain_original_method_order DefaultMethods Mode: othervm [/othervm specified] elapsed time (seconds): 0.25 ----------configuration:(0/0)---------- ----------System.out:(4/322)---------- Enabled capability: maintain_original_method_order: true Reflection getDeclaredMethods returned: [public abstract java.lang.String DefaultMethods$Child.method2()] JVMTI GetClassMethods returned: [public abstract java.lang.String DefaultMethods$Child.method2()] Test passed: Got expected output from JVMTI GetClassMethods! ----------System.err:(1/15)---------- STATUS:Passed. [1] http://cr.openjdk.java.net/~dtitov/8216324/webrev.02/ Thanks, Daniil From: mailto:serguei.spitsyn at oracle.com mailto:serguei.spitsyn at oracle.com Date: Monday, July 20, 2020 at 6:48 PM To: Alex Menkov mailto:alexey.menkov at oracle.com, Daniil Titov mailto:daniil.x.titov at oracle.com, serviceability-dev mailto:serviceability-dev at openjdk.java.net Subject: Re: RFR: 8216324: GetClassMethods is confused by the presence of default methods in super interfaces Hi Daniil, The fix looks pretty good to me. Just minor comments. http://cr.openjdk.java.net/~dtitov/8216324/webrev.01/src/hotspot/share/prims/jvmtiEnv.cpp.frames.html 2519 int skipped = 0; // skip default methods from superinterface (see JDK-8216324) You can say just:? // skip overpass methods There is probably no need to list the bug number. 2523 // Depending on can_maintain_original_method_order capability 2524 // use the original method ordering indices stored in the class, so we can emit 2525 // jmethodIDs in the order they appeared in the class file 2526 // or just copy in any order Could you, please re-balance the comment a little bit? Also, the comment has to be terminated with a dot. Replace: "or just copy in any order" => "or just copy in current order". http://cr.openjdk.java.net/~dtitov/8216324/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetClassMethods/getclmthd007.java.frames.html 114 default void default_method() { } // should never be seen The comment above is not clear. Should never be seen in what context? 117 interface OuterInterface1 extends DefaultInterface { An extra space before "extends". http://cr.openjdk.java.net/~dtitov/8216324/webrev.01/test/hotspot/jtreg/serviceability/jvmti/GetClassMethods/InterfaceDefMethods.java.html I like the test simplicity. Default methods are always in interfaces. I'd suggest to rename the test to something like: DefaultMethods.java or OverpassMethods.java. http://cr.openjdk.java.net/~dtitov/8216324/webrev.01/test/hotspot/jtreg/serviceability/jvmti/GetClassMethods/libInterfaceDefMethods.cpp.html 44 if (options != NULL && strcmp(options, "maintain_original_method_order") == 0) { 45 printf("maintain_original_method_order: true\n"); ... 54 } else { 55 printf("maintain_original_method_order: false\n"); I'd suggest to remove the lines 54 and 55 and adjust the line 45: ?? printf("Enabled capability: maintain_original_method_order: true\n"); 88 jobject m = env->ToReflectedMethod(klass, methods[i], (modifiers & 8) == 8); It is better to replace 8 with a symbolic constant. Thanks, Serguei On 7/20/20 16:57, Alex Menkov wrote: Looks good to me :) Thanks for handling this. --alex On 07/20/2020 12:00, Daniil Titov wrote: Please review change [1] that fixes GetClassMethods behavior in cases if a default method is present in a super interface. Currently for such cases the information GetClassMethods returns for the sub-interface or implementing class incorrectly includes? inherited not default? methods. The fix ( thanks to Alex for providing this patch) ensures that overpass methods are filtered out? in the returns. The fix also applies a change in the existing test as David suggested in the comments to the issue [2]. Testing: Mach5? tier1-tier3 tests successfully passed. [1] http://cr.openjdk.java.net/~dtitov/8216324/webrev.01/ [2] https://bugs.openjdk.java.net/browse/JDK-8216324 Thank you, Daniil From serguei.spitsyn at oracle.com Thu Jul 23 19:18:57 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 23 Jul 2020 12:18:57 -0700 Subject: RFR (S) 8249822: SymbolPropertyTable creates an extra OopHandle per entry In-Reply-To: <2e00b001-0749-8680-71e9-a7396f9bb48b@oracle.com> References: <84643f6c-0e05-061e-b6e3-a38d09ed973a@oracle.com> <2e00b001-0749-8680-71e9-a7396f9bb48b@oracle.com> Message-ID: Thank you for answering my question, Coleen! Serguei On 7/23/20 04:36, coleen.phillimore at oracle.com wrote: > > > On 7/22/20 8:53 PM, serguei.spitsyn at oracle.com wrote: >> Hi Coleen, >> >> The fix looks good to me. > > Thank you for looking at this Serguei. >> >> >> On 7/22/20 13:55, coleen.phillimore at oracle.com wrote: >>> Summary: Add an assert to OopHandle assigment operator to catch >>> leaking OopHandles, and fix code accordingly. >>> >>> There are some jvmtiRedefineClasses.cpp changes here - basically, I >>> moved the RedefineVerifyMark and comments into >>> jvmtiRedefineClasses.cpp because I needed a Handle. >> >> I think, the jvmtiRedefineClasses.cpp is a better place for the >> RedefineVerifyMark. >> But could you, please, explain a little bit more why this move was >> necessary? >> Why Handle can not be used in the jvmtiThreadState.cpp? > > I did have a patch where I moved the constructor and destructor in > jvmtiThreadState.cpp but thought it made more sense just to move the > whole class since nothing else would ever use it.?? I couldn't put the > code change in jvmtiThreadState.hpp in place because I'd need to > include handles.inline.hpp there.? It seems like > jvmtiRedefineClasses.cpp already included all the files it needed, so > that's where I moved it. > > Thanks, > Coleen > > >> >> Thanks, >> Serguei >> >> >>> Ran tier1-6 tests and tier1 on all Oracle platforms. >>> >>> open webrev at >>> http://cr.openjdk.java.net/~coleenp/2020/8249822.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8249822 >>> >>> Thanks, >>> Coleen >> > From serguei.spitsyn at oracle.com Thu Jul 23 19:22:55 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 23 Jul 2020 12:22:55 -0700 Subject: [15?] RFR (S): 8249192: MonitorInfo stores raw oops across safepoints In-Reply-To: <87846f7d-c213-733a-9a6f-96e5d02df247@oracle.com> References: <3f838c8a-6e70-cd94-1753-8abe269b7cdf@oracle.com> <528ade6c-059a-d9af-b0c7-74d7ac31b29a@oracle.com> <14512a57-8d4d-5199-ed16-743b5da8a10d@oracle.com> <893f862f-093a-5168-e554-1f4ffe129aea@oracle.com> <07904b7e-8d40-8c9e-acc1-c713c1588b11@oracle.com> <4ad72264-2269-fcc1-b2fc-0568a993ca7d@oracle.com> <87846f7d-c213-733a-9a6f-96e5d02df247@oracle.com> Message-ID: <00d2c4b4-f3fa-97ee-92be-b311738daac0@oracle.com> LGTM++ Thanks, Serguei On 7/23/20 08:35, Daniel D. Daugherty wrote: >> jdk16: >> http://cr.openjdk.java.net/~tschatzl/8249192/webrev.2_to_3/ (diff) >> >> jdk15: >> http://cr.openjdk.java.net/~tschatzl/8249192/webrev.jdk15.2_to_3/ (diff) > > The delta webrevs look just fine. > > Dan > > > On 7/23/20 5:39 AM, Thomas Schatzl wrote: >> Hi Dan and Serguei, >> >> ? thanks for your reviews. >> >> On 22.07.20 19:04, Daniel D. Daugherty wrote: >>>> jdk15: >>>> >>>> http://cr.openjdk.java.net/~tschatzl/8249192/webrev.jdk15.2/ (full) >>> >>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>> ???? old L1029: ????? ResourceMark rm; >>> ???????? It's not clear (to me anyway) why this ResourceMark is >>> removed. >>> >>> ???????? Update: I saw the discussion of "ResourceMark rm" in JDK15 >>> versus >>> ???????? "ResourceMark rm(current_thread)" in JDK16, but that >>> doesn't tell >>> ???????? me why it was necessary to remove that ResourceMark. >> >> The method that is guarded by this ResourceMark contains the >> necessary ResourceMark itself, so I removed it. >> >>> >>> src/hotspot/share/prims/stackwalk.cpp >>> ???? L291: ??? ResourceMark rm; >>> ???? L292: ??? HandleMark hm; >>> ???????? Since there's a TRAPS parameter, these should be >>> 'rm(THREAD)' and >>> ???????? 'hm(THREAD)'. >>> >>> src/hotspot/share/runtime/biasedLocking.cpp >>> ???? No comments. >>> >>> src/hotspot/share/runtime/deoptimization.cpp >>> ???? No comments. >>> >>> src/hotspot/share/runtime/vframe.cpp >>> ???? L461: ? _lock? = lock; >>> ???????? nit - extra space before '='. >>> >>> src/hotspot/share/runtime/vframe.hpp >>> ???? L32: #include "runtime/handles.inline.hpp" >>> ???????? nit - new include is out of order; should be after frame.hpp. >>> >>> src/hotspot/share/runtime/vframeArray.cpp >>> ???? No comments. >>> >>> src/hotspot/share/runtime/vframe_hp.cpp >>> ???? Skipped - no changes. >>> >>> src/hotspot/share/services/threadService.cpp >>> ???? No comments. >>> >> >> All fixed, and incorporating Serguei's changes in the other email as >> well. >> >> jdk16: >> http://cr.openjdk.java.net/~tschatzl/8249192/webrev.3/ (full) >> http://cr.openjdk.java.net/~tschatzl/8249192/webrev.2_to_3/ (diff) >> >> jdk15: >> http://cr.openjdk.java.net/~tschatzl/8249192/webrev.jdk15.3/ (full) >> http://cr.openjdk.java.net/~tschatzl/8249192/webrev.jdk15.2_to_3/ (diff) >> >> Note that the jdk15 change will only go into 15.0.2 as discussion >> with the release team showed that the change is too risky for earlier >> releases. See the relevant CR comment for details. >> >> Thanks, >> ? Thomas >> > From coleen.phillimore at oracle.com Thu Jul 23 19:25:02 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 23 Jul 2020 15:25:02 -0400 Subject: RFR (S) 8249822: SymbolPropertyTable creates an extra OopHandle per entry In-Reply-To: References: <84643f6c-0e05-061e-b6e3-a38d09ed973a@oracle.com> <2e00b001-0749-8680-71e9-a7396f9bb48b@oracle.com> Message-ID: Thanks Serguei! Coleen On 7/23/20 3:18 PM, serguei.spitsyn at oracle.com wrote: > Thank you for answering my question, Coleen! > Serguei > > > On 7/23/20 04:36, coleen.phillimore at oracle.com wrote: >> >> >> On 7/22/20 8:53 PM, serguei.spitsyn at oracle.com wrote: >>> Hi Coleen, >>> >>> The fix looks good to me. >> >> Thank you for looking at this Serguei. >>> >>> >>> On 7/22/20 13:55, coleen.phillimore at oracle.com wrote: >>>> Summary: Add an assert to OopHandle assigment operator to catch >>>> leaking OopHandles, and fix code accordingly. >>>> >>>> There are some jvmtiRedefineClasses.cpp changes here - basically, I >>>> moved the RedefineVerifyMark and comments into >>>> jvmtiRedefineClasses.cpp because I needed a Handle. >>> >>> I think, the jvmtiRedefineClasses.cpp is a better place for the >>> RedefineVerifyMark. >>> But could you, please, explain a little bit more why this move was >>> necessary? >>> Why Handle can not be used in the jvmtiThreadState.cpp? >> >> I did have a patch where I moved the constructor and destructor in >> jvmtiThreadState.cpp but thought it made more sense just to move the >> whole class since nothing else would ever use it.?? I couldn't put >> the code change in jvmtiThreadState.hpp in place because I'd need to >> include handles.inline.hpp there.? It seems like >> jvmtiRedefineClasses.cpp already included all the files it needed, so >> that's where I moved it. >> >> Thanks, >> Coleen >> >> >>> >>> Thanks, >>> Serguei >>> >>> >>>> Ran tier1-6 tests and tier1 on all Oracle platforms. >>>> >>>> open webrev at >>>> http://cr.openjdk.java.net/~coleenp/2020/8249822.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8249822 >>>> >>>> Thanks, >>>> Coleen >>> >> > From serguei.spitsyn at oracle.com Thu Jul 23 19:28:14 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 23 Jul 2020 12:28:14 -0700 Subject: RFR: 8216324: GetClassMethods is confused by the presence of default methods in super interfaces In-Reply-To: <511BC62A-9B69-4456-ABD9-6E47191D7829@oracle.com> References: <511BC62A-9B69-4456-ABD9-6E47191D7829@oracle.com> Message-ID: <9253de99-c0b3-6b7f-f02e-6b45a0b78391@oracle.com> Hi Daniil, The update looks good to me. Thanks, Serguei On 7/23/20 11:32, Daniil Titov wrote: > Hi Serguei and Alex, > > Please review a new version of the webrev [1] that includes the changes Serguei suggested. This version also has new test renamed from DefaultMethods.java to OverpassMethods.java to avoid warnings during the build about conflict with native library for runtime/jni/8033445/DefaultMethods.java test. > > [1] http://cr.openjdk.java.net/~dtitov/8216324/webrev.04/ > > Thank you, > Daniil > > From: "serguei.spitsyn at oracle.com" > Date: Tuesday, July 21, 2020 at 10:53 PM > To: Daniil Titov , Alex Menkov , serviceability-dev > Subject: Re: RFR: 8216324: GetClassMethods is confused by the presence of default methods in super interfaces > > Hi Daniil, > > Thank you for the update! > It looks good to me. > > Just two more minor comments and no need for another webrev you address them. > 2519 int skipped = 0; // skip default methods > Saying "overpass methods" would be more precise. > 47 printf("Enabled capability: maintain_original_method_order: true\n"); > It is better to get rid of ": true" at the end (sorry, I missed this in my previous suggestion). > Enabling capability means it is set to true. > > > Thanks, > Serguei > > > On 7/21/20 20:25, Daniil Titov wrote: > Hi Serguei and Alex, > > Please, review new version of the change [1] that includes changes as Serguei suggested. > > 114 default void default_method() { } // should never be seen > The comment above is not clear. Should never be seen in what context? > This method should not be included in the list of methods GetClassMethods returns for the sub-interface or implementing class. I don't think we want to comment each method in the test interfaces declared in this test when they should be seen in this test and when they should not. This information already exists in getclmthd007.cpp, thus I decided to omit this comment. > > Please see below the output from the new test. > > > ----------messages:(4/215)---------- > command: main -agentlib:DefaultMethods DefaultMethods > reason: User specified action: run main/othervm/native -agentlib:DefaultMethods DefaultMethods > Mode: othervm [/othervm specified] > elapsed time (seconds): 0.241 > ----------configuration:(0/0)---------- > ----------System.out:(3/265)---------- > Reflection getDeclaredMethods returned: [public abstract java.lang.String DefaultMethods$Child.method2()] > JVMTI GetClassMethods returned: [public abstract java.lang.String DefaultMethods$Child.method2()] > Test passed: Got expected output from JVMTI GetClassMethods! > ----------System.err:(1/15)---------- > STATUS:Passed. > > > > ----------messages:(4/276)---------- > command: main -agentlib:DefaultMethods=maintain_original_method_order DefaultMethods > reason: User specified action: run main/othervm/native -agentlib:DefaultMethods=maintain_original_method_order DefaultMethods > Mode: othervm [/othervm specified] > elapsed time (seconds): 0.25 > ----------configuration:(0/0)---------- > ----------System.out:(4/322)---------- > Enabled capability: maintain_original_method_order: true > Reflection getDeclaredMethods returned: [public abstract java.lang.String DefaultMethods$Child.method2()] > JVMTI GetClassMethods returned: [public abstract java.lang.String DefaultMethods$Child.method2()] > Test passed: Got expected output from JVMTI GetClassMethods! > ----------System.err:(1/15)---------- > STATUS:Passed. > > > [1] http://cr.openjdk.java.net/~dtitov/8216324/webrev.02/ > > Thanks, > Daniil > > From: mailto:serguei.spitsyn at oracle.com mailto:serguei.spitsyn at oracle.com > Date: Monday, July 20, 2020 at 6:48 PM > To: Alex Menkov mailto:alexey.menkov at oracle.com, Daniil Titov mailto:daniil.x.titov at oracle.com, serviceability-dev mailto:serviceability-dev at openjdk.java.net > Subject: Re: RFR: 8216324: GetClassMethods is confused by the presence of default methods in super interfaces > > Hi Daniil, > > The fix looks pretty good to me. > > Just minor comments. > > > http://cr.openjdk.java.net/~dtitov/8216324/webrev.01/src/hotspot/share/prims/jvmtiEnv.cpp.frames.html > 2519 int skipped = 0; // skip default methods from superinterface (see JDK-8216324) > > You can say just:? // skip overpass methods > There is probably no need to list the bug number. > > 2523 // Depending on can_maintain_original_method_order capability > 2524 // use the original method ordering indices stored in the class, so we can emit > 2525 // jmethodIDs in the order they appeared in the class file > 2526 // or just copy in any order > Could you, please re-balance the comment a little bit? > Also, the comment has to be terminated with a dot. > Replace: "or just copy in any order" => "or just copy in current order". > > > http://cr.openjdk.java.net/~dtitov/8216324/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetClassMethods/getclmthd007.java.frames.html > 114 default void default_method() { } // should never be seen > The comment above is not clear. Should never be seen in what context? > 117 interface OuterInterface1 extends DefaultInterface { > An extra space before "extends". > > > http://cr.openjdk.java.net/~dtitov/8216324/webrev.01/test/hotspot/jtreg/serviceability/jvmti/GetClassMethods/InterfaceDefMethods.java.html > > I like the test simplicity. > Default methods are always in interfaces. > I'd suggest to rename the test to something like: DefaultMethods.java or OverpassMethods.java. > > > http://cr.openjdk.java.net/~dtitov/8216324/webrev.01/test/hotspot/jtreg/serviceability/jvmti/GetClassMethods/libInterfaceDefMethods.cpp.html > 44 if (options != NULL && strcmp(options, "maintain_original_method_order") == 0) { > 45 printf("maintain_original_method_order: true\n"); > ... > 54 } else { > 55 printf("maintain_original_method_order: false\n"); > I'd suggest to remove the lines 54 and 55 and adjust the line 45: > ?? printf("Enabled capability: maintain_original_method_order: true\n"); > 88 jobject m = env->ToReflectedMethod(klass, methods[i], (modifiers & 8) == 8); > It is better to replace 8 with a symbolic constant. > > > Thanks, > Serguei > > > On 7/20/20 16:57, Alex Menkov wrote: > Looks good to me :) > Thanks for handling this. > > --alex > > On 07/20/2020 12:00, Daniil Titov wrote: > > Please review change [1] that fixes GetClassMethods behavior in cases if a default method is present in a super interface. Currently for such cases the information GetClassMethods returns for the sub-interface or implementing class incorrectly includes? inherited not default? methods. > > The fix ( thanks to Alex for providing this patch) ensures that overpass methods are filtered out? in the returns. The fix also applies a change in the existing test as David suggested in the comments to the issue [2]. > > Testing: Mach5? tier1-tier3 tests successfully passed. > > [1] http://cr.openjdk.java.net/~dtitov/8216324/webrev.01/ > [2] https://bugs.openjdk.java.net/browse/JDK-8216324 > > Thank you, > Daniil > > > > > > > > > From serguei.spitsyn at oracle.com Thu Jul 23 19:31:01 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 23 Jul 2020 12:31:01 -0700 Subject: RFR: 8248362: JVMTI frame operations should use Thread-Local Handshake In-Reply-To: <3c74c4d5-e1b6-439e-253f-a36a0d0b3cf3@oss.nttdata.com> References: <3c74c4d5-e1b6-439e-253f-a36a0d0b3cf3@oss.nttdata.com> Message-ID: Hi Yasumasa, The update looks good. Thanks, Serguei On 7/22/20 20:03, Yasumasa Suenaga wrote: > Hi Serguei, > > Thanks for your comment! > I fixed it in new webrev: > > ? http://cr.openjdk.java.net/~ysuenaga/JDK-8248362/webrev.01/ > ? Diff from previous webrev: > http://hg.openjdk.java.net/jdk/submit/rev/df75038b5449 > > > Yasumasa > > > On 2020/07/23 9:31, serguei.spitsyn at oracle.com wrote: >> Hi Yasumasa, >> >> Looks good. >> Just one minor comment. >> >> http://cr.openjdk.java.net/~ysuenaga/JDK-8248362/webrev.00/src/hotspot/share/prims/jvmtiEnv.cpp.udiff.html >> >> >> ????? // JVMTI get java stack frame location at safepoint. >> >> Replace: "at safepoint" => "with handshake". >> >> Thanks, >> Serguei >> >> >> On 7/22/20 07:38, Yasumasa Suenaga wrote: >>> Hi all, >>> >>> Please review this change: >>> >>> ? JBS: https://bugs.openjdk.java.net/browse/JDK-8248362 >>> ? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8248362/webrev.00/ >>> >>> Migrate JVMTI frame operations to Thread-Local Handshakes from VM >>> Operations. >>> >>> ??? - VM_GetFrameCount >>> ??? - VM_GetFrameLocation >>> >>> They affects both GetFrameCount() and GetFrameLocation() JVMTI >>> functions. >>> >>> This change has passed all tests on submit repo >>> (mach5-one-ysuenaga-JDK-8248362-20200722-1249-12859056), and >>> vmTestbase/nsk/{jdi,jdw,jvmti}, serviceability/{jdwp,jvmti} . >>> >>> >>> Thanks, >>> >>> Yasumasa >> From alexey.menkov at oracle.com Thu Jul 23 19:50:53 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Thu, 23 Jul 2020 12:50:53 -0700 Subject: RFR: 8216324: GetClassMethods is confused by the presence of default methods in super interfaces In-Reply-To: <9253de99-c0b3-6b7f-f02e-6b45a0b78391@oracle.com> References: <511BC62A-9B69-4456-ABD9-6E47191D7829@oracle.com> <9253de99-c0b3-6b7f-f02e-6b45a0b78391@oracle.com> Message-ID: <40766da3-17ea-3226-ecd6-cee3fa5528dd@oracle.com> +1 --alex On 07/23/2020 12:28, serguei.spitsyn at oracle.com wrote: > Hi Daniil, > > The update looks good to me. > > Thanks, > Serguei > > > On 7/23/20 11:32, Daniil Titov wrote: >> Hi Serguei and Alex, >> >> Please review a new version of the webrev [1]? that includes the >> changes Serguei suggested.? This version also has new test renamed >> from DefaultMethods.java to OverpassMethods.java to avoid warnings >> during the build about conflict with native library for >> runtime/jni/8033445/DefaultMethods.java? test. >> >> [1] http://cr.openjdk.java.net/~dtitov/8216324/webrev.04/ >> >> Thank you, >> Daniil >> >> From: "serguei.spitsyn at oracle.com" >> Date: Tuesday, July 21, 2020 at 10:53 PM >> To: Daniil Titov , Alex Menkov >> , serviceability-dev >> >> Subject: Re: RFR: 8216324: GetClassMethods is confused by the presence >> of default methods in super interfaces >> >> Hi Daniil, >> >> Thank you for the update! >> It looks good to me. >> >> Just two more minor comments and no need for another webrev you >> address them. >> 2519?? int skipped = 0;? // skip default methods >> Saying "overpass methods" would be more precise. >> ?? 47???? printf("Enabled capability: maintain_original_method_order: >> true\n"); >> It is better to get rid of ": true" at the end (sorry, I missed this >> in my previous suggestion). >> Enabling capability means it is set to true. >> >> >> Thanks, >> Serguei >> >> >> On 7/21/20 20:25, Daniil Titov wrote: >> Hi Serguei and Alex, >> >> Please, review new version of the change [1]? that includes changes as >> Serguei suggested. >> >> 114???? default void default_method() { } // should never be seen >> The comment above is not clear. Should never be seen in what context? >> This method should not be included in the list of methods >> GetClassMethods returns for the sub-interface or implementing class. >> I don't think we want to comment each method in the test interfaces >> declared in this test when they should be seen in this test and when >> they should not. This information already exists in getclmthd007.cpp, >> thus I decided to omit this comment. >> >> Please see below the output from the new test. >> >> >> ----------messages:(4/215)---------- >> command: main -agentlib:DefaultMethods DefaultMethods >> reason: User specified action: run main/othervm/native >> -agentlib:DefaultMethods DefaultMethods >> Mode: othervm [/othervm specified] >> elapsed time (seconds): 0.241 >> ----------configuration:(0/0)---------- >> ----------System.out:(3/265)---------- >> Reflection getDeclaredMethods returned: [public abstract >> java.lang.String DefaultMethods$Child.method2()] >> JVMTI GetClassMethods returned: [public abstract java.lang.String >> DefaultMethods$Child.method2()] >> Test passed: Got expected output from JVMTI GetClassMethods! >> ----------System.err:(1/15)---------- >> STATUS:Passed. >> >> >> >> ----------messages:(4/276)---------- >> command: main -agentlib:DefaultMethods=maintain_original_method_order >> DefaultMethods >> reason: User specified action: run main/othervm/native >> -agentlib:DefaultMethods=maintain_original_method_order DefaultMethods >> Mode: othervm [/othervm specified] >> elapsed time (seconds): 0.25 >> ----------configuration:(0/0)---------- >> ----------System.out:(4/322)---------- >> Enabled capability: maintain_original_method_order: true >> Reflection getDeclaredMethods returned: [public abstract >> java.lang.String DefaultMethods$Child.method2()] >> JVMTI GetClassMethods returned: [public abstract java.lang.String >> DefaultMethods$Child.method2()] >> Test passed: Got expected output from JVMTI GetClassMethods! >> ----------System.err:(1/15)---------- >> STATUS:Passed. >> >> >> [1] http://cr.openjdk.java.net/~dtitov/8216324/webrev.02/ >> >> Thanks, >> Daniil >> >> From: mailto:serguei.spitsyn at oracle.com mailto:serguei.spitsyn at oracle.com >> Date: Monday, July 20, 2020 at 6:48 PM >> To: Alex Menkov mailto:alexey.menkov at oracle.com, Daniil Titov >> mailto:daniil.x.titov at oracle.com, serviceability-dev >> mailto:serviceability-dev at openjdk.java.net >> Subject: Re: RFR: 8216324: GetClassMethods is confused by the presence >> of default methods in super interfaces >> >> Hi Daniil, >> >> The fix looks pretty good to me. >> >> Just minor comments. >> >> >> http://cr.openjdk.java.net/~dtitov/8216324/webrev.01/src/hotspot/share/prims/jvmtiEnv.cpp.frames.html >> >> 2519?? int skipped = 0;? // skip default methods from superinterface >> (see JDK-8216324) >> >> You can say just:? // skip overpass methods >> There is probably no need to list the bug number. >> >> 2523???? // Depending on can_maintain_original_method_order capability >> 2524???? // use the original method ordering indices stored in the >> class, so we can emit >> 2525???? // jmethodIDs in the order they appeared in the class file >> 2526???? // or just copy in any order >> Could you, please re-balance the comment a little bit? >> Also, the comment has to be terminated with a dot. >> Replace: "or just copy in any order" => "or just copy in current order". >> >> >> http://cr.openjdk.java.net/~dtitov/8216324/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetClassMethods/getclmthd007.java.frames.html >> >> ? 114???? default void default_method() { } // should never be seen >> The comment above is not clear. Should never be seen in what context? >> ? 117 interface OuterInterface1? extends DefaultInterface { >> An extra space before "extends". >> >> >> http://cr.openjdk.java.net/~dtitov/8216324/webrev.01/test/hotspot/jtreg/serviceability/jvmti/GetClassMethods/InterfaceDefMethods.java.html >> >> >> I like the test simplicity. >> Default methods are always in interfaces. >> I'd suggest to rename the test to something like: DefaultMethods.java >> or OverpassMethods.java. >> >> >> http://cr.openjdk.java.net/~dtitov/8216324/webrev.01/test/hotspot/jtreg/serviceability/jvmti/GetClassMethods/libInterfaceDefMethods.cpp.html >> >> ?? 44?? if (options != NULL && strcmp(options, >> "maintain_original_method_order") == 0) { >> ?? 45???? printf("maintain_original_method_order: true\n"); >> ?? ... >> ?? 54?? } else { >> ?? 55???? printf("maintain_original_method_order: false\n"); >> I'd suggest to remove the lines 54 and 55 and adjust the line 45: >> ??? printf("Enabled capability: maintain_original_method_order: true\n"); >> ?? 88???? jobject m = env->ToReflectedMethod(klass, methods[i], >> (modifiers & 8) == 8); >> It is better to replace 8 with a symbolic constant. >> >> >> Thanks, >> Serguei >> >> >> On 7/20/20 16:57, Alex Menkov wrote: >> Looks good to me :) >> Thanks for handling this. >> >> --alex >> >> On 07/20/2020 12:00, Daniil Titov wrote: >> >> Please review change [1] that fixes GetClassMethods behavior in cases >> if a default method is present in a super interface. Currently for >> such cases the information GetClassMethods returns for the >> sub-interface or implementing class incorrectly includes? inherited >> not default? methods. >> >> The fix ( thanks to Alex for providing this patch) ensures that >> overpass methods are filtered out? in the returns. The fix also >> applies a change in the existing test as David suggested in the >> comments to the issue [2]. >> >> Testing: Mach5? tier1-tier3 tests successfully passed. >> >> [1] http://cr.openjdk.java.net/~dtitov/8216324/webrev.01/ >> [2] https://bugs.openjdk.java.net/browse/JDK-8216324 >> >> Thank you, >> Daniil >> >> >> >> >> >> >> >> >> > From chris.plummer at oracle.com Thu Jul 23 20:05:17 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 23 Jul 2020 13:05:17 -0700 Subject: RFR: JDK-8249550: jdb should use loopback address when not using remote agent In-Reply-To: <541a2f57-4b86-b453-7739-f1de35b52212@oracle.com> References: <541a2f57-4b86-b453-7739-f1de35b52212@oracle.com> Message-ID: <5b200434-6181-1478-6423-70cd9150ce56@oracle.com> Hi Alex, I'm no expert in this area, but the changes appear to do what you describe (use the loopback address), so thumbs up. thanks, Chris On 7/21/20 3:04 PM, Alex Menkov wrote: > Hi all, > > please review the fix for > https://bugs.openjdk.java.net/browse/JDK-8249550 > > webrev: > http://cr.openjdk.java.net/~amenkov/jdk16/jdb_loopback/webrev/ > > some background: > https://bugs.openjdk.java.net/browse/JDK-8041435 made default > listening on loopback address. > Later https://bugs.openjdk.java.net/browse/JDK-8184770 added handling > of "*" address to listen on all addresses, but it didn't fixed > "default" startListening() method (used by jdb through > SunCommandLineLauncher). > > The method called startListening(String localaddress, int port) with > localaddress == null, but this method for null localladdress starts > listening on all addresses (i.e. handle null value as "*"). > The fix changes it to startListening(String address) which handles > null address the same way as JDI socket connector does (i.e. listens > on loopback address only) > > --alex From chris.plummer at oracle.com Thu Jul 23 21:27:03 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 23 Jul 2020 14:27:03 -0700 Subject: RFR(M): 8247515: OSX pc_to_symbol() lookup does not work with core files In-Reply-To: <8f1cec73-4397-0489-6031-a813d9bd23c6@oracle.com> References: <092ef5a8-c973-e377-de91-b2002a1b8fe7@oracle.com> <38e1887e-a646-abc0-0b7e-53339af1d684@oracle.com> <3c31622b-831e-d46f-daa6-d641cbf755b1@oracle.com> <18f7a1d8-3802-c7da-c981-eac8cf0953eb@oracle.com> <1e9e4786-79f8-c4e0-efe7-f1a8f369d2c6@oracle.com> <4c09d200-057b-e878-733a-524860453288@oracle.com> <401dbc7a-dd2f-5612-74d3-3dbf6641dae9@oracle.com> <8f1cec73-4397-0489-6031-a813d9bd23c6@oracle.com> Message-ID: <526ac128-d29d-70f1-413b-b541c101835e@oracle.com> An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Thu Jul 23 21:59:49 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 23 Jul 2020 14:59:49 -0700 Subject: RFR(S) 8249293: Unsafe stackwalk in VM_GetOrSetLocal::doit_prologue() In-Reply-To: References: Message-ID: Hi Richard, Thank you for filing the CR and taking care about it! The fix itself looks good to me. I still need another look at new test. Could you, please, convert the agent of new test to C++? It will make it a little bit simpler. Thanks, Serguei On 7/20/20 01:15, Reingruber, Richard wrote: > Hi, > > please help review this fix for VM_GetOrSetLocal. It moves the unsafe stackwalk from the vm > operation prologue before the safepoint into the doit() method executed at the safepoint. > > Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8249293/webrev.0/index.html > Bug: https://bugs.openjdk.java.net/browse/JDK-8249293 > > According to the JVMTI spec on local variable access it is not required to suspend the target thread > T [1]. The operation will simply fail with JVMTI_ERROR_NO_MORE_FRAMES if T is executing > bytecodes. It will succeed though if T is blocked because of synchronization or executing some native > code. > > The issue is that in the latter case the stack walk in VM_GetOrSetLocal::doit_prologue() to prepare > the access to the local variable is unsafe, because it is done before the safepoint and it races > with T returning to execute bytecodes making its stack not walkable. The included test shows that > this can crash the VM if T wins the race. > > Manual testing: > > - new test test/hotspot/jtreg/serviceability/jvmti/GetLocalVariable/GetLocalWithoutSuspendTest.java > - test/hotspot/jtreg/vmTestbase/nsk/jvmti > - test/hotspot/jtreg/serviceability/jvmti > > Nightly regression tests @SAP: JCK and JTREG, also in Xcomp mode, SPECjvm2008, SPECjbb2015, > Renaissance Suite, SAP specific tests with fastdebug and release builds on all platforms > > Thanks, Richard. > > [1] https://docs.oracle.com/en/java/javase/14/docs/specs/jvmti.html#local From chris.plummer at oracle.com Thu Jul 23 22:29:33 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 23 Jul 2020 15:29:33 -0700 Subject: RFR (T) 8250042: Clean up methodOop and method_oop names from the code In-Reply-To: <85efc3ab-abbf-c5f2-9b7b-47fa516d9a2d@oracle.com> References: <85efc3ab-abbf-c5f2-9b7b-47fa516d9a2d@oracle.com> Message-ID: <93edcee0-5d38-81f2-fe04-4d9585eb3d30@oracle.com> Hi Coleen, The JVMTI changes look fine, although I wonder why in jvmtiEnter.xsl you did the rename to "checked_method" instead of just "method". Was there a conflict in some cases? thanks, Chris On 7/23/20 9:58 AM, coleen.phillimore at oracle.com wrote: > See bug for more details.? I've been running into these names a lot > lately.?? Many of these names are in JVMTI. > > Tested with tier1 on all Oracle platforms and built on non-Oracle > platforms. > > open webrev at http://cr.openjdk.java.net/~coleenp/2020/8250042.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8250042 > > Thanks, > Coleen From coleen.phillimore at oracle.com Thu Jul 23 23:17:09 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 23 Jul 2020 19:17:09 -0400 Subject: RFR (T) 8250042: Clean up methodOop and method_oop names from the code In-Reply-To: <93edcee0-5d38-81f2-fe04-4d9585eb3d30@oracle.com> References: <85efc3ab-abbf-c5f2-9b7b-47fa516d9a2d@oracle.com> <93edcee0-5d38-81f2-fe04-4d9585eb3d30@oracle.com> Message-ID: <4b3cc2f6-3729-77f3-51c6-48dee05de036@oracle.com> On 7/23/20 6:29 PM, Chris Plummer wrote: > Hi Coleen, > > The JVMTI changes look fine, although I wonder why in jvmtiEnter.xsl > you did the rename to "checked_method" instead of just "method". Was > there a conflict in some cases? Yes it was quite painful.? The jvmti.xml spec has jmethodID parameters to be named "method" and I didn't want to change that. ??????? ????????? ??????????? ????????????? The method in which to set the breakpoint ??????????? I picked checked_method because you get it by calling this below and the alternative "m" I think was used somewhere else (or not descriptive at all). +? ? Method* checked_method = Method::checked_resolve_jmethod_id( Thanks, Coleen > > thanks, > > Chris > > On 7/23/20 9:58 AM, coleen.phillimore at oracle.com wrote: >> See bug for more details.? I've been running into these names a lot >> lately.?? Many of these names are in JVMTI. >> >> Tested with tier1 on all Oracle platforms and built on non-Oracle >> platforms. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8250042.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8250042 >> >> Thanks, >> Coleen > > From chris.plummer at oracle.com Thu Jul 23 23:48:43 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 23 Jul 2020 16:48:43 -0700 Subject: RFR (T) 8250042: Clean up methodOop and method_oop names from the code In-Reply-To: <4b3cc2f6-3729-77f3-51c6-48dee05de036@oracle.com> References: <85efc3ab-abbf-c5f2-9b7b-47fa516d9a2d@oracle.com> <93edcee0-5d38-81f2-fe04-4d9585eb3d30@oracle.com> <4b3cc2f6-3729-77f3-51c6-48dee05de036@oracle.com> Message-ID: <8572e90b-3b95-3cef-ae4c-819703a015da@oracle.com> On 7/23/20 4:17 PM, coleen.phillimore at oracle.com wrote: > > > On 7/23/20 6:29 PM, Chris Plummer wrote: >> Hi Coleen, >> >> The JVMTI changes look fine, although I wonder why in jvmtiEnter.xsl >> you did the rename to "checked_method" instead of just "method". Was >> there a conflict in some cases? > > Yes it was quite painful.? The jvmti.xml spec has jmethodID parameters > to be named "method" and I didn't want to change that. > > ??????? > ????????? > ??????????? > ????????????? The method in which to set the breakpoint > ??????????? > > I picked checked_method because you get it by calling this below and > the alternative "m" I think was used somewhere else (or not > descriptive at all). > > +? ? Method* checked_method = > Method::checked_resolve_jmethod_id( > Ah, so this is the case of having to deal with a jmethodID and a Method* in the same scope. I think this is fine, although it would be nice if Serguei also gave his blessing. thanks, Chris > Thanks, > Coleen >> >> thanks, >> >> Chris >> >> On 7/23/20 9:58 AM, coleen.phillimore at oracle.com wrote: >>> See bug for more details.? I've been running into these names a lot >>> lately.?? Many of these names are in JVMTI. >>> >>> Tested with tier1 on all Oracle platforms and built on non-Oracle >>> platforms. >>> >>> open webrev at >>> http://cr.openjdk.java.net/~coleenp/2020/8250042.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8250042 >>> >>> Thanks, >>> Coleen >> >> > From suenaga at oss.nttdata.com Fri Jul 24 01:35:03 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Fri, 24 Jul 2020 10:35:03 +0900 Subject: RFR: 8248362: JVMTI frame operations should use Thread-Local Handshake In-Reply-To: References: <3c74c4d5-e1b6-439e-253f-a36a0d0b3cf3@oss.nttdata.com> Message-ID: <92f6dd21-0b5e-6f71-e2f3-6b75d9a89acc@oss.nttdata.com> Thanks Serguei! Yasumasa On 2020/07/24 4:31, serguei.spitsyn at oracle.com wrote: > Hi Yasumasa, > > The update looks good. > > Thanks, > Serguei > > > On 7/22/20 20:03, Yasumasa Suenaga wrote: >> Hi Serguei, >> >> Thanks for your comment! >> I fixed it in new webrev: >> >> ? http://cr.openjdk.java.net/~ysuenaga/JDK-8248362/webrev.01/ >> ? Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/df75038b5449 >> >> >> Yasumasa >> >> >> On 2020/07/23 9:31, serguei.spitsyn at oracle.com wrote: >>> Hi Yasumasa, >>> >>> Looks good. >>> Just one minor comment. >>> >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8248362/webrev.00/src/hotspot/share/prims/jvmtiEnv.cpp.udiff.html >>> >>> ????? // JVMTI get java stack frame location at safepoint. >>> >>> Replace: "at safepoint" => "with handshake". >>> >>> Thanks, >>> Serguei >>> >>> >>> On 7/22/20 07:38, Yasumasa Suenaga wrote: >>>> Hi all, >>>> >>>> Please review this change: >>>> >>>> ? JBS: https://bugs.openjdk.java.net/browse/JDK-8248362 >>>> ? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8248362/webrev.00/ >>>> >>>> Migrate JVMTI frame operations to Thread-Local Handshakes from VM Operations. >>>> >>>> ??? - VM_GetFrameCount >>>> ??? - VM_GetFrameLocation >>>> >>>> They affects both GetFrameCount() and GetFrameLocation() JVMTI functions. >>>> >>>> This change has passed all tests on submit repo (mach5-one-ysuenaga-JDK-8248362-20200722-1249-12859056), and vmTestbase/nsk/{jdi,jdw,jvmti}, serviceability/{jdwp,jvmti} . >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>> > From david.holmes at oracle.com Fri Jul 24 05:01:21 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 24 Jul 2020 15:01:21 +1000 Subject: RFR (T) 8250042: Clean up methodOop and method_oop names from the code In-Reply-To: <85efc3ab-abbf-c5f2-9b7b-47fa516d9a2d@oracle.com> References: <85efc3ab-abbf-c5f2-9b7b-47fa516d9a2d@oracle.com> Message-ID: <6f973a0a-cf55-e1ab-8de3-b57f68dbd2cf@oracle.com> Hi Coleen, On 24/07/2020 2:58 am, coleen.phillimore at oracle.com wrote: > See bug for more details.? I've been running into these names a lot > lately.?? Many of these names are in JVMTI. > > Tested with tier1 on all Oracle platforms and built on non-Oracle > platforms. > > open webrev at http://cr.openjdk.java.net/~coleenp/2020/8250042.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8250042 src/hotspot/cpu/*/*.ad These still refer to "method oop" and method_oop in a number of places. src/hotspot/share/adlc/adlparse.cpp + frame->_interpreter_method_oop_reg = parse_one_arg("method reg entry"); I guess I'm not understanding the scope of this renaming - why is _interpreter_method_oop_reg not renamed as well? Should this (and other uses) be parsed as method-(oop-reg) rather than (method-oop)-reg? Otherwise all okay. Thanks, David > Thanks, > Coleen From david.holmes at oracle.com Fri Jul 24 05:41:29 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 24 Jul 2020 15:41:29 +1000 Subject: RFR (S) 8249822: SymbolPropertyTable creates an extra OopHandle per entry In-Reply-To: <84643f6c-0e05-061e-b6e3-a38d09ed973a@oracle.com> References: <84643f6c-0e05-061e-b6e3-a38d09ed973a@oracle.com> Message-ID: On 23/07/2020 6:55 am, coleen.phillimore at oracle.com wrote: > Summary: Add an assert to OopHandle assigment operator to catch leaking > OopHandles, and fix code accordingly. > > There are some jvmtiRedefineClasses.cpp changes here - basically, I > moved the RedefineVerifyMark and comments into jvmtiRedefineClasses.cpp > because I needed a Handle. > > Ran tier1-6 tests and tier1 on all Oracle platforms. > > open webrev at http://cr.openjdk.java.net/~coleenp/2020/8249822.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8249822 This all seems okay. A couple of minor nits src/hotspot/share/oops/klass.cpp + void Klass::replace_java_mirror(oop mirror) { _java_mirror.replace(mirror); } Writing this all in one line is inconsistent with surrounding style. --- 1226 // a) A reference to the class being redefined (_the_class) and a 1230 // b) The _java_mirror field from _the_class is copied to the There's an extra space before b) on line 1230. Thanks, David ----- > Thanks, > Coleen From kevin.walls at oracle.com Fri Jul 24 09:17:19 2020 From: kevin.walls at oracle.com (Kevin Walls) Date: Fri, 24 Jul 2020 10:17:19 +0100 Subject: RFR(M): 8247515: OSX pc_to_symbol() lookup does not work with core files In-Reply-To: <526ac128-d29d-70f1-413b-b541c101835e@oracle.com> References: <092ef5a8-c973-e377-de91-b2002a1b8fe7@oracle.com> <38e1887e-a646-abc0-0b7e-53339af1d684@oracle.com> <3c31622b-831e-d46f-daa6-d641cbf755b1@oracle.com> <18f7a1d8-3802-c7da-c981-eac8cf0953eb@oracle.com> <1e9e4786-79f8-c4e0-efe7-f1a8f369d2c6@oracle.com> <4c09d200-057b-e878-733a-524860453288@oracle.com> <401dbc7a-dd2f-5612-74d3-3dbf6641dae9@oracle.com> <8f1cec73-4397-0489-6031-a813d9bd23c6@oracle.com> <526ac128-d29d-70f1-413b-b541c101835e@oracle.com> Message-ID: <3b354be8-d687-974a-7f5e-84d8bfb39545@oracle.com> Thanks Chris - all sounds good to me.? Thanks for all the MachO insights... On 23/07/2020 22:27, Chris Plummer wrote: > Just a minor update of some new findings (no new code change). The DB > hash table being used by default will overwrite an existing entry, not > duplicate it, and this is indeed what was happening. The second entry > added was the one with a 0 offset. When I enable the R_NOOVERWRITE > flag, it stops the overwrite and that also fixes the problem, but > that's only because the entry with offset 0 comes last. The fix I've > done is better since it avoids the offet 0 entry altogether, so even > if it came first it would not get used. > > thanks, > > Chris > > On 7/22/20 10:25 PM, Chris Plummer wrote: >> Hi Kevin, >> >> Thanks for the review. Unfortunately there was yet another bug which >> I have now addressed. Although testing with mach5 went fine, when I >> tried with a local build I had issues. SA couldn't even get past an >> early part of the core file handling which involves doing some >> adjustments related to CDS. It needs to look up a couple of hotspot >> symbols by name to do this, and get their values (such as >> _SharedBaseAddress). Although the symbol -> address lookup seemed to >> work, the values retrieved from the address were garbage. After some >> debugging I noticed the 3 symbols being looked up all had the same >> address. Then I noticed this address was at offset 0 of the libjvm >> segment. After a lot more debugging I found the problem. These >> symbols were actually in the symbol table twice, once with a proper >> offset and once with an offset of 0. I'm not sure why the ones with >> an offset of 0 were there (other than they originated in the mach-o >> symbol table). >> >> The reason this didn't always happen is because SA takes all the >> symbols it finds and adds them to a hash table for fast symbol -> >> address lookup. If a symbol is in there twice, it's a tossup which >> you'll get. It could change from build to build in fact. The trigger >> for my local build was probably how I ran configure, which likely is >> not the same as mach5, although I'm? unsure if this just gave me the >> unlucky hashing, or if in fact it resulted in the entries with offset >> 0. In any case, the fix is to ignore entries with offset 0. Here's >> the updated webrev: >> >> http://cr.openjdk.java.net/~cjplummer/8247515/webrev.03/index.html >> >> All the changes since webrev.02 are in build_symtab() in symtab.c. >> Besides ignoring entries with offset 0 to fix the bug, I also did >> some cleanup. There used to be two loops to iterate over the symbols. >> There wasn't really a good reason for this, so now there is just one. >> Also, it no longer adds entries with a file offset 0, an offset into >> the string section of 0, or an empty string. By doing this the size >> of the libjvm symbol table went from about 240k entries to 90k. Since >> it was originally allocated at it's full potential size, it's now >> reallocate to the smaller size after symbol table processing is over. >> >> thanks, >> >> Chris >> >> On 7/22/20 2:45 AM, Kevin Walls wrote: >>> >>> Thanks Chris, yes looks good, I like that we check the library >>> bounds before calling nearest_symbol. >>> >>> -- >>> Kevin >>> >>> >>> On 21/07/2020 21:05, Chris Plummer wrote: >>>> Hi Serguei and Kevin, >>>> >>>> The webrev has been updated: >>>> >>>> http://cr.openjdk.java.net/~cjplummer/8247515/webrev.02/index.html >>>> https://bugs.openjdk.java.net/browse/JDK-8247515 >>>> >>>> Two issues were addressed: >>>> >>>> (1) Code in symbol_for_pc() assumed the caller had first checked to >>>> make sure that the symbol is in a library, where-as some callers >>>> assume NULL will be returned if the symbol is not in a library. >>>> This is the case for pstack for example, which first blindly does a >>>> pc to symbol lookup, and only if that returns null does it then >>>> check if the pc is in the codecache or interpreter. The logic in >>>> symbol_for_pc() assumed that if the pc was greater than the start >>>> address of the last library in the list, then it must be in that >>>> library. So in stack traces the frames for compiled or interpreted >>>> pc's showed up as the last symbol in the last library, plus some >>>> very large offset. The fix is to now track the size of libraries so >>>> we can do a proper range check. >>>> >>>> (2) There are issues with finding system libraries. See [1] >>>> JDK-8249779. Because of this I disabled support for trying to >>>> locate them. This was done in ps_core.c, and there are >>>> "JDK-8249779" comment references in the code where I did this. The >>>> end result of this is that PMap for core files won't show system >>>> libraries, and PStack for core files won't show symbols for >>>> addresses in system libraries. Note that currently support for PMap >>>> and PStack is disabled for OSX, but I will shortly send out a >>>> review to enable them for OSX core files as part of the fix for [2] >>>> JDK-8248882. >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8249779 >>>> [2] https://bugs.openjdk.java.net/browse/JDK-8248882 >>>> >>>> thanks, >>>> >>>> Chris >>>> >>>> On 7/14/20 5:46 PM, serguei.spitsyn at oracle.com wrote: >>>>> Okay, I'll wait for new webrev version to review. >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>> On 7/14/20 17:40, Chris Plummer wrote: >>>>>> Hi Serguie, >>>>>> >>>>>> Thanks for reviewing. This webrev is in limbo right now as I >>>>>> discovered some issues that Kevin and I have been discussing off >>>>>> line. One is that the code assumes the caller has first checked >>>>>> to make sure that the symbol is in a library, where-as the actual >>>>>> callers assume NULL will be returned if the symbol is not in a >>>>>> library. The end result is that we end up returning a symbol, >>>>>> even for address in the code cache or interpreter. So in stack >>>>>> traces these frame show up as the last symbol in the last >>>>>> library, plus some very large offset. I have a fix for that were >>>>>> now I track the size of each library. But there is another issue >>>>>> with the code that tries to discover all libraries in the core >>>>>> file. It misses a very large number of system libraries. We >>>>>> understand why, but are not sure of the solution. I might just >>>>>> change to code to only worry about user libraries (JDK libs and >>>>>> other JNI libs). >>>>>> >>>>>> Some comments below also. >>>>>> >>>>>> On 7/14/20 4:37 PM, serguei.spitsyn at oracle.com wrote: >>>>>>> Hi Chris, >>>>>>> >>>>>>> I like the suggestion from Kevin below. >>>>>> I'm not sure which suggestion since I updated the webrev based on >>>>>> his initial suggestion. >>>>>>> >>>>>>> I have a couple of minor comments so far. >>>>>>> >>>>>>> http://cr.openjdk.java.net/~cjplummer/8247515/webrev.01/src/jdk.hotspot.agent/macosx/native/libsaproc/libproc_impl.c.frames.html >>>>>>> 313 if (!lib->next || lib->next->base >= addr) { >>>>>>> I wonder if the check above has to be: >>>>>>> 313 if (!lib->next || lib->next->base > addr) { >>>>>> Yes, > would be better, although this goes away with my fix that >>>>>> tracks the size of each library. >>>>>>> >>>>>>> >>>>>>> http://cr.openjdk.java.net/~cjplummer/8247515/webrev.01/src/jdk.hotspot.agent/macosx/native/libsaproc/symtab.c.frames.html >>>>>>> 417 if (offset_from_sym >= 0) { // ignore symbols that comes >>>>>>> after "offset" >>>>>>> Replace: comes => come >>>>>> Ok. >>>>>> >>>>>> thanks, >>>>>> >>>>>> Chris >>>>>>> >>>>>>> Thanks, >>>>>>> Serguei >>>>>>> >>>>>>> >>>>>>> On 7/8/20 03:23, Kevin Walls wrote: >>>>>>>> >>>>>>>> >>>>>>>> Sure -- I was thinking lowest_offset_from_sym initialising >>>>>>>> starting at a high positive integer (that would now be >>>>>>>> PTRDIFF_MAX I think) to save a comparison with e.g. -1, you can >>>>>>>> just check if the new offset is less than lowest_offset_from_sym >>>>>>>> >>>>>>>> With the ptrdiff_t change you made, this all looks good to me >>>>>>>> however you decide. 8-) >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 07/07/2020 21:17, Chris Plummer wrote: >>>>>>>>> Hi Kevin, >>>>>>>>> >>>>>>>>> Thanks for the review. Yes, that lack of initialization of >>>>>>>>> lowest_offset_from_sym is a bug. I'm real surprised the >>>>>>>>> compiler didn't catch it as it will be uninitialized garbage >>>>>>>>> the first time it is referenced. Fortunately usually the >>>>>>>>> eventual offset is very small if not 0, so probably this never >>>>>>>>> prevented a proper match. I think there's also another bug: >>>>>>>>> >>>>>>>>> ?415?????? uintptr_t offset_from_sym = offset - sym->offset; >>>>>>>>> >>>>>>>>> "offset" is the passed in offset, essentially the address of >>>>>>>>> the symbol we are interested in, but given as an offset from >>>>>>>>> the start of the DSO. "sym->offset" is also an offset from the >>>>>>>>> start of the DSO. It could be located before or after >>>>>>>>> "offset". This means the math could result in a negative >>>>>>>>> number, which when converted to unsigned would be a very large >>>>>>>>> positive number. This happens whenever you check a symbol that >>>>>>>>> is actually located after the address you are looking up. The >>>>>>>>> end result is harmless, because it just means there's no way >>>>>>>>> we will match that symbol, which is what you want, but it >>>>>>>>> would be good to clean this up. >>>>>>>>> >>>>>>>>> I think what is best is to use ptrdiff_t and initialize >>>>>>>>> lowest_offset_from_sym to -1. I've updated the webrev: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~cjplummer/8247515/webrev.01/index.html >>>>>>>>> >>>>>>>>> >>>>>>>>> thanks, >>>>>>>>> >>>>>>>>> Chris >>>>>>>>> >>>>>>>>> On 7/7/20 4:09 AM, Kevin Walls wrote: >>>>>>>>>> Hi Chris, >>>>>>>>>> >>>>>>>>>> Yes I think this looks good. >>>>>>>>>> >>>>>>>>>> Question: In nearest_symbol, do we need to initialize >>>>>>>>>> lowest_offset_from_sym to something impossibly high, as if it >>>>>>>>>> defaults to zero we never find a better/nearer result? >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> Kevin >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 07/07/2020 06:10, Chris Plummer wrote: >>>>>>>>>>> Hello, >>>>>>>>>>> >>>>>>>>>>> Please help review the following: >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~cjplummer/8247515/webrev.00/index.html >>>>>>>>>>> >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8247515 >>>>>>>>>>> >>>>>>>>>>> The CR contains a description of the issues being addressed. >>>>>>>>>>> There is also no test for this symbol lookup support yet. It >>>>>>>>>>> will be there after I push JDK-8247516 and JDK-8247514, >>>>>>>>>>> which are both blocked by the CR. >>>>>>>>>>> >>>>>>>>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8247516 >>>>>>>>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8247514 >>>>>>>>>>> >>>>>>>>>>> thanks, >>>>>>>>>>> >>>>>>>>>>> Chris >>>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From coleen.phillimore at oracle.com Fri Jul 24 11:44:17 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 24 Jul 2020 07:44:17 -0400 Subject: RFR (S) 8249822: SymbolPropertyTable creates an extra OopHandle per entry In-Reply-To: References: <84643f6c-0e05-061e-b6e3-a38d09ed973a@oracle.com> Message-ID: On 7/24/20 1:41 AM, David Holmes wrote: > On 23/07/2020 6:55 am, coleen.phillimore at oracle.com wrote: >> Summary: Add an assert to OopHandle assigment operator to catch >> leaking OopHandles, and fix code accordingly. >> >> There are some jvmtiRedefineClasses.cpp changes here - basically, I >> moved the RedefineVerifyMark and comments into >> jvmtiRedefineClasses.cpp because I needed a Handle. >> >> Ran tier1-6 tests and tier1 on all Oracle platforms. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8249822.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8249822 > > This all seems okay. A couple of minor nits > > src/hotspot/share/oops/klass.cpp > > + void Klass::replace_java_mirror(oop mirror) { > _java_mirror.replace(mirror); } > > Writing this all in one line is inconsistent with surrounding style. I was going to fix the java_mirror_no_keepalive() to match this style, but I've fixed this one instead. > > --- > > 1226 //? a) A reference to the class being redefined (_the_class) and a > 1230 //?? b) The _java_mirror field from _the_class is copied to the > > There's an extra space before b) on line 1230. Wow. fixed. Thanks for reviewing this! Coleen > > Thanks, > David > ----- > > >> Thanks, >> Coleen From coleen.phillimore at oracle.com Fri Jul 24 12:23:38 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 24 Jul 2020 08:23:38 -0400 Subject: RFR (T) 8250042: Clean up methodOop and method_oop names from the code In-Reply-To: <6f973a0a-cf55-e1ab-8de3-b57f68dbd2cf@oracle.com> References: <85efc3ab-abbf-c5f2-9b7b-47fa516d9a2d@oracle.com> <6f973a0a-cf55-e1ab-8de3-b57f68dbd2cf@oracle.com> Message-ID: Thanks for looking at this. On 7/24/20 1:01 AM, David Holmes wrote: > Hi Coleen, > > On 24/07/2020 2:58 am, coleen.phillimore at oracle.com wrote: >> See bug for more details.? I've been running into these names a lot >> lately.?? Many of these names are in JVMTI. >> >> Tested with tier1 on all Oracle platforms and built on non-Oracle >> platforms. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8250042.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8250042 > > src/hotspot/cpu/*/*.ad > > These still refer to "method oop" and method_oop in a number of places. Yes, I only replaced method_oop in the shared code and not in the AD code.? method_oop can be the name of a parameter and using "sed" to change it to "method" doesn't work.?? Somebody who understands this code and looks at it will have to make the rest of the changes. What I did was replace "method oop" with "method" and "methodOop" with "method" in all the sources.? I replaced "method_oop" with "method" or "checked_method" in the shared sources. > > src/hotspot/share/adlc/adlparse.cpp > > +? frame->_interpreter_method_oop_reg = parse_one_arg("method reg > entry"); > > I guess I'm not understanding the scope of this renaming - why is > _interpreter_method_oop_reg not renamed as well? Should this (and > other uses) be parsed as method-(oop-reg) rather than (method-oop)-reg? I don't know this code, so I'd rather not change any more of it. The comment makes sense changed, even though the variable name still refers to method_oop. Thanks, Coleen > > Otherwise all okay. > > Thanks, > David > >> Thanks, >> Coleen From richard.reingruber at sap.com Fri Jul 24 13:03:47 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Fri, 24 Jul 2020 13:03:47 +0000 Subject: RFR(S) 8249293: Unsafe stackwalk in VM_GetOrSetLocal::doit_prologue() In-Reply-To: References: Message-ID: Hi Serguei, > The fix itself looks good to me. thanks for looking at the fix. > I still need another look at new test. > Could you, please, convert the agent of new test to C++? > It will make it a little bit simpler. Sure, here is the new webrev.1 with a C++ version of the test agent: http://cr.openjdk.java.net/~rrich/webrevs/8249293/webrev.1/ I tested it on Linux and Windows but not yet on MacOS. Thanks, Richard. -----Original Message----- From: serguei.spitsyn at oracle.com Sent: Freitag, 24. Juli 2020 00:00 To: Reingruber, Richard ; serviceability-dev at openjdk.java.net Subject: Re: RFR(S) 8249293: Unsafe stackwalk in VM_GetOrSetLocal::doit_prologue() Hi Richard, Thank you for filing the CR and taking care about it! The fix itself looks good to me. I still need another look at new test. Could you, please, convert the agent of new test to C++? It will make it a little bit simpler. Thanks, Serguei On 7/20/20 01:15, Reingruber, Richard wrote: > Hi, > > please help review this fix for VM_GetOrSetLocal. It moves the unsafe stackwalk from the vm > operation prologue before the safepoint into the doit() method executed at the safepoint. > > Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8249293/webrev.0/index.html > Bug: https://bugs.openjdk.java.net/browse/JDK-8249293 > > According to the JVMTI spec on local variable access it is not required to suspend the target thread > T [1]. The operation will simply fail with JVMTI_ERROR_NO_MORE_FRAMES if T is executing > bytecodes. It will succeed though if T is blocked because of synchronization or executing some native > code. > > The issue is that in the latter case the stack walk in VM_GetOrSetLocal::doit_prologue() to prepare > the access to the local variable is unsafe, because it is done before the safepoint and it races > with T returning to execute bytecodes making its stack not walkable. The included test shows that > this can crash the VM if T wins the race. > > Manual testing: > > - new test test/hotspot/jtreg/serviceability/jvmti/GetLocalVariable/GetLocalWithoutSuspendTest.java > - test/hotspot/jtreg/vmTestbase/nsk/jvmti > - test/hotspot/jtreg/serviceability/jvmti > > Nightly regression tests @SAP: JCK and JTREG, also in Xcomp mode, SPECjvm2008, SPECjbb2015, > Renaissance Suite, SAP specific tests with fastdebug and release builds on all platforms > > Thanks, Richard. > > [1] https://docs.oracle.com/en/java/javase/14/docs/specs/jvmti.html#local From coleen.phillimore at oracle.com Fri Jul 24 13:10:24 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 24 Jul 2020 09:10:24 -0400 Subject: RFR (T) 8250042: Clean up methodOop and method_oop names from the code In-Reply-To: References: <85efc3ab-abbf-c5f2-9b7b-47fa516d9a2d@oracle.com> <6f973a0a-cf55-e1ab-8de3-b57f68dbd2cf@oracle.com> Message-ID: I can also replace method_oop with method_ptr in the CPU ad files, and this seems to build but now someone who knows the compiler area needs to comment; this was supposed to be trivial... :)? But it still is really trivial to look at. I left interpreter_method_oop_reg and compiler_method_oop_reg and friends in opto/matcher.cpp for someone else. incremental webrev at http://cr.openjdk.java.net/~coleenp/2020/8250042.02.incr/webrev full webrev at http://cr.openjdk.java.net/~coleenp/2020/8250042.02/webrev Thanks, Coleen On 7/24/20 8:23 AM, coleen.phillimore at oracle.com wrote: > > Thanks for looking at this. > > On 7/24/20 1:01 AM, David Holmes wrote: >> Hi Coleen, >> >> On 24/07/2020 2:58 am, coleen.phillimore at oracle.com wrote: >>> See bug for more details.? I've been running into these names a lot >>> lately.?? Many of these names are in JVMTI. >>> >>> Tested with tier1 on all Oracle platforms and built on non-Oracle >>> platforms. >>> >>> open webrev at >>> http://cr.openjdk.java.net/~coleenp/2020/8250042.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8250042 >> >> src/hotspot/cpu/*/*.ad >> >> These still refer to "method oop" and method_oop in a number of places. > > Yes, I only replaced method_oop in the shared code and not in the AD > code.? method_oop can be the name of a parameter and using "sed" to > change it to "method" doesn't work.?? Somebody who understands this > code and looks at it will have to make the rest of the changes. > > What I did was replace "method oop" with "method" and "methodOop" with > "method" in all the sources.? I replaced "method_oop" with "method" or > "checked_method" in the shared sources. > >> >> src/hotspot/share/adlc/adlparse.cpp >> >> +? frame->_interpreter_method_oop_reg = parse_one_arg("method reg >> entry"); >> >> I guess I'm not understanding the scope of this renaming - why is >> _interpreter_method_oop_reg not renamed as well? Should this (and >> other uses) be parsed as method-(oop-reg) rather than (method-oop)-reg? > > I don't know this code, so I'd rather not change any more of it. The > comment makes sense changed, even though the variable name still > refers to method_oop. > > Thanks, > Coleen >> >> Otherwise all okay. >> >> Thanks, >> David >> >>> Thanks, >>> Coleen > From sgehwolf at redhat.com Fri Jul 24 16:13:49 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Fri, 24 Jul 2020 18:13:49 +0200 Subject: No way to disable Java Container Metrics Message-ID: Hi, For hotspot one can disable container detection with a simple switch: $ java -XX:-UseContainerSupport -Xlog:os+container=trace -version [0.000s][trace][os,container] OSContainer::init: Initializing Container Support [0.000s][trace][os,container] Container Support not enabled openjdk version "15-internal" 2020-09-15 OpenJDK Runtime Environment (build 15-internal+0-adhoc.sgehwolf.openjdk-head-2) OpenJDK 64-Bit Server VM (build 15-internal+0-adhoc.sgehwolf.openjdk-head-2, mixed mode, sharing) The same cannot be achieved with the Java code, jdk.internal.platform.Metrics.java and friends in the JDK. At the time Metrics were added the only consumer of them was the Java Launcher via -XshowSettings:system. This has changed with JDK-8226575: OperatingSystemMXBean should be made container aware. It is known that in certain cases the container detection code will detect for a system to be supposedly in a container where it actually isn't: https://bugs.openjdk.java.net/browse/JDK-8227006?focusedCommentId=14275604&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14275604 For hotspot there is a flag, to turn auto-detection off for exactly the case when heuristics are wrong: -XX:-UseContainerSupport. However, this flag has no effect on the Java metrics code. There is a risk that on some systems the auto-detection will be wrong and might cause regressions. I propose to make the Java metrics code adhere to -XX:+/- UseContainerSupport flag. Do you think this would be useful? If yes, I'll file a bug and propose a patch for it. Thoughts? Opinions? Thanks, Severin From daniel.daugherty at oracle.com Fri Jul 24 18:24:03 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 24 Jul 2020 14:24:03 -0400 Subject: RFR: 8248362: JVMTI frame operations should use Thread-Local Handshake In-Reply-To: <5c2bba7d-44aa-d5e7-6d5c-949c18ce9668@oss.nttdata.com> References: <5c2bba7d-44aa-d5e7-6d5c-949c18ce9668@oss.nttdata.com> Message-ID: <164ccf3e-9054-4cf3-a970-d2562abd31dd@oracle.com> On 7/22/20 11:12 PM, Yasumasa Suenaga wrote: > Hi Dan, > > Thanks for your comment! > I uploaded new webrev: > > ? http://cr.openjdk.java.net/~ysuenaga/JDK-8248362/webrev.01/ > ? Diff from previous webrev: > http://hg.openjdk.java.net/jdk/submit/rev/df75038b5449 Thumbs up. Reviewed by comparing the previous webrev patch (webrev.00) with the latest (webrev.01). Dan > > On 2020/07/23 10:04, Daniel D. Daugherty wrote: >> On 7/22/20 10:38 AM, Yasumasa Suenaga wrote: >>> Hi all, >>> >>> Please review this change: >>> >>> ? JBS: https://bugs.openjdk.java.net/browse/JDK-8248362 >>> ? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8248362/webrev.00/ >> >> src/hotspot/share/prims/jvmtiEnv.cpp >> ???? L1755???? // JVMTI get java stack frame location at safepoint. >> ???????? You missed updating this one. Perhaps: >> >> ?????????????? // JVMTI get java stack frame location via direct >> handshake. > > Fixed. > > >> src/hotspot/share/prims/jvmtiEnvBase.cpp >> ???? L1563: ? JavaThread* jt = _state->get_thread(); >> ???? L1564: ? assert(target == jt, "just checking"); >> ???????? This code is different than the other closures. It might be >> worth >> ???????? a comment to explain why. I don't remember why >> VM_GetFrameCount had >> ???????? to use the JvmtiThreadState to fetch the JavaThread. Serguei >> might >> ???????? remember. >> >> ???????? It could be that we don't need the _state field anymore >> because of >> ???????? the way that handshakes work (and provide the JavaThread* to >> the >> ???????? do_thread() function). Your choice on whether to deal with >> that as >> ???????? part of this fix or following with another RFE. >> >> ???????? Update: Uggg.... the get_frame_count() function takes >> JvmtiThreadState >> ???????? as a parameter. This is very much entangled... sigh... we >> should >> ???????? definitely look at untangling this in another RFE... > > I want to fix it in another RFE as you said if it is needed. > If we don't hear the reason from Serguei, I want to keep this change. > > >> ???? old L1565: ? ThreadsListHandle tlh; >> ???? new L1565: ? if (!jt->is_exiting() && jt->threadObj() != NULL) { >> >> ???????? Please consider add this comment above L1565: >> ????????????????? // ThreadsListHandle is not needed due to direct >> handshake. >> >> ???????? Yes, I see that previous closures were added without that >> comment, >> ???????? but when I see "if (!jt->is_exiting() && jt->threadObj() != >> NULL)" >> ???????? I immediately wonder where the ThreadsListHandle is... >> Please consider >> ???????? adding the comment to the others also. >> >> ???? old L1574: ? ThreadsListHandle tlh; >> ???? new L1574: ? if (!jt->is_exiting() && jt->threadObj() != NULL) { >> >> ???????? Please consider add this comment above L1574: >> ????????????????? // ThreadsListHandle is not needed due to direct >> handshake. > > I think it is new normal as David said in the reply, and also other > closures don't say about it. > So I did not change about it in new webrev. > > >> src/hotspot/share/prims/jvmtiEnvBase.hpp >> ???? L580: // HandshakeClosure to frame location. >> ???????? typo - s/to frame/to get frame/ > > Fixed. > > >> src/hotspot/share/prims/jvmtiThreadState.cpp >> ???? No comments on the changes. >> >> ???? For the above comments about VM_GetFrameCount, understanding why >> ???? JvmtiThreadState::count_frames() has to exist in JvmtiThreadState >> ???? will go along way towards untangling the mess. >> >> src/hotspot/share/runtime/vmOperations.hpp >> ???? No comments. >> >> >> Thumbs up. I only have nits above. If you choose to make the above >> changes, I do not need to see another webrev. > > Thanks, but I uploaded new webrev just in case :) > > > Yasumasa > > >> Dan >> >> >> >>> >>> Migrate JVMTI frame operations to Thread-Local Handshakes from VM >>> Operations. >>> >>> ??? - VM_GetFrameCount >>> ??? - VM_GetFrameLocation >>> >>> They affects both GetFrameCount() and GetFrameLocation() JVMTI >>> functions. >>> >>> This change has passed all tests on submit repo >>> (mach5-one-ysuenaga-JDK-8248362-20200722-1249-12859056), and >>> vmTestbase/nsk/{jdi,jdw,jvmti}, serviceability/{jdwp,jvmti} . >>> >>> >>> Thanks, >>> >>> Yasumasa >> From bob.vandette at oracle.com Fri Jul 24 19:21:29 2020 From: bob.vandette at oracle.com (Bob Vandette) Date: Fri, 24 Jul 2020 15:21:29 -0400 Subject: No way to disable Java Container Metrics In-Reply-To: References: Message-ID: I?ve been monitoring the discussion on your jdk8u alias and I can?t help wondering if my original decision to provide two different implementations of this container detection logic was the best way to go. I didn?t want to have an all Java implementation since the VM needs to initialize it?s memory and cpu sizing very early on prior to its ability to run Java code. If we put all of the logic in the VM, then we?d end up with a pretty wide interface to the VM and more overhead extracting values (due to JNI) although the Java logic typically ends up in native code anyway. Having the functionality in the VM makes it easier to add JFR events. Having a pure Java implementation makes it easier to support the OS MBeans. The maintenance of supporting both implementations is a bit of a pain. Assuming no one has the time or desire to migrate the logic to the VM and provide Java wrapper logic, I?m ok with what you are proposing. It?s one step on the path. The VM support and the Java level support are really for two different consumers but I think it would be cleaner and less confusing to disable both on one flag rather than support two options. Bob. > On Jul 24, 2020, at 12:13 PM, Severin Gehwolf wrote: > > Hi, > > For hotspot one can disable container detection with a simple switch: > > $ java -XX:-UseContainerSupport -Xlog:os+container=trace -version > [0.000s][trace][os,container] OSContainer::init: Initializing Container Support > [0.000s][trace][os,container] Container Support not enabled > openjdk version "15-internal" 2020-09-15 > OpenJDK Runtime Environment (build 15-internal+0-adhoc.sgehwolf.openjdk-head-2) > OpenJDK 64-Bit Server VM (build 15-internal+0-adhoc.sgehwolf.openjdk-head-2, mixed mode, sharing) > > The same cannot be achieved with the Java code, > jdk.internal.platform.Metrics.java and friends in the JDK. At the time > Metrics were added the only consumer of them was the Java Launcher via > -XshowSettings:system. This has changed with JDK-8226575: > OperatingSystemMXBean should be made container aware. > > It is known that in certain cases the container detection code will > detect for a system to be supposedly in a container where it actually > isn't: > https://bugs.openjdk.java.net/browse/JDK-8227006?focusedCommentId=14275604&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14275604 > > For hotspot there is a flag, to turn auto-detection off for exactly the > case when heuristics are wrong: -XX:-UseContainerSupport. However, this > flag has no effect on the Java metrics code. There is a risk that on > some systems the auto-detection will be wrong and might cause > regressions. > > I propose to make the Java metrics code adhere to -XX:+/- > UseContainerSupport flag. Do you think this would be useful? If yes, > I'll file a bug and propose a patch for it. > > Thoughts? Opinions? > > Thanks, > Severin > From serguei.spitsyn at oracle.com Fri Jul 24 20:28:18 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 24 Jul 2020 13:28:18 -0700 Subject: RFR (T) 8250042: Clean up methodOop and method_oop names from the code In-Reply-To: References: <85efc3ab-abbf-c5f2-9b7b-47fa516d9a2d@oracle.com> <6f973a0a-cf55-e1ab-8de3-b57f68dbd2cf@oracle.com> Message-ID: <60098204-b23d-7da6-908f-80f3d40c2ebc@oracle.com> Hi Coleen, The fix looks good to me. I've more focused on the serviceability related update. Thank you for taking care about it! Thanks, Serguei On 7/24/20 06:10, coleen.phillimore at oracle.com wrote: > > I can also replace method_oop with method_ptr in the CPU ad files, and > this seems to build but now someone who knows the compiler area needs > to comment; this was supposed to be trivial... :)? But it still is > really trivial to look at. > > I left interpreter_method_oop_reg and compiler_method_oop_reg and > friends in opto/matcher.cpp for someone else. > > incremental webrev at > http://cr.openjdk.java.net/~coleenp/2020/8250042.02.incr/webrev > full webrev at http://cr.openjdk.java.net/~coleenp/2020/8250042.02/webrev > > Thanks, > Coleen > > > On 7/24/20 8:23 AM, coleen.phillimore at oracle.com wrote: >> >> Thanks for looking at this. >> >> On 7/24/20 1:01 AM, David Holmes wrote: >>> Hi Coleen, >>> >>> On 24/07/2020 2:58 am, coleen.phillimore at oracle.com wrote: >>>> See bug for more details.? I've been running into these names a lot >>>> lately.?? Many of these names are in JVMTI. >>>> >>>> Tested with tier1 on all Oracle platforms and built on non-Oracle >>>> platforms. >>>> >>>> open webrev at >>>> http://cr.openjdk.java.net/~coleenp/2020/8250042.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8250042 >>> >>> src/hotspot/cpu/*/*.ad >>> >>> These still refer to "method oop" and method_oop in a number of places. >> >> Yes, I only replaced method_oop in the shared code and not in the AD >> code.? method_oop can be the name of a parameter and using "sed" to >> change it to "method" doesn't work.?? Somebody who understands this >> code and looks at it will have to make the rest of the changes. >> >> What I did was replace "method oop" with "method" and "methodOop" >> with "method" in all the sources.? I replaced "method_oop" with >> "method" or "checked_method" in the shared sources. >> >>> >>> src/hotspot/share/adlc/adlparse.cpp >>> >>> +? frame->_interpreter_method_oop_reg = parse_one_arg("method reg >>> entry"); >>> >>> I guess I'm not understanding the scope of this renaming - why is >>> _interpreter_method_oop_reg not renamed as well? Should this (and >>> other uses) be parsed as method-(oop-reg) rather than (method-oop)-reg? >> >> I don't know this code, so I'd rather not change any more of it. The >> comment makes sense changed, even though the variable name still >> refers to method_oop. >> >> Thanks, >> Coleen >>> >>> Otherwise all okay. >>> >>> Thanks, >>> David >>> >>>> Thanks, >>>> Coleen >> > From coleen.phillimore at oracle.com Fri Jul 24 22:20:23 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 24 Jul 2020 18:20:23 -0400 Subject: RFR (T) 8250042: Clean up methodOop and method_oop names from the code In-Reply-To: <60098204-b23d-7da6-908f-80f3d40c2ebc@oracle.com> References: <85efc3ab-abbf-c5f2-9b7b-47fa516d9a2d@oracle.com> <6f973a0a-cf55-e1ab-8de3-b57f68dbd2cf@oracle.com> <60098204-b23d-7da6-908f-80f3d40c2ebc@oracle.com> Message-ID: <4232a089-3984-7feb-d2eb-46f1551ee0ab@oracle.com> On 7/24/20 4:28 PM, serguei.spitsyn at oracle.com wrote: > Hi Coleen, > > The fix looks good to me. > I've more focused on the serviceability related update. > Thank you for taking care about it! Thank you for reviewing it!? Most of the name changes were in jvmti.? Hope it's cleaner to work on now. Coleen > > Thanks, > Serguei > > > On 7/24/20 06:10, coleen.phillimore at oracle.com wrote: >> >> I can also replace method_oop with method_ptr in the CPU ad files, >> and this seems to build but now someone who knows the compiler area >> needs to comment; this was supposed to be trivial... :)? But it still >> is really trivial to look at. >> >> I left interpreter_method_oop_reg and compiler_method_oop_reg and >> friends in opto/matcher.cpp for someone else. >> >> incremental webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8250042.02.incr/webrev >> full webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8250042.02/webrev >> >> Thanks, >> Coleen >> >> >> On 7/24/20 8:23 AM, coleen.phillimore at oracle.com wrote: >>> >>> Thanks for looking at this. >>> >>> On 7/24/20 1:01 AM, David Holmes wrote: >>>> Hi Coleen, >>>> >>>> On 24/07/2020 2:58 am, coleen.phillimore at oracle.com wrote: >>>>> See bug for more details.? I've been running into these names a >>>>> lot lately.?? Many of these names are in JVMTI. >>>>> >>>>> Tested with tier1 on all Oracle platforms and built on non-Oracle >>>>> platforms. >>>>> >>>>> open webrev at >>>>> http://cr.openjdk.java.net/~coleenp/2020/8250042.01/webrev >>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8250042 >>>> >>>> src/hotspot/cpu/*/*.ad >>>> >>>> These still refer to "method oop" and method_oop in a number of >>>> places. >>> >>> Yes, I only replaced method_oop in the shared code and not in the AD >>> code.? method_oop can be the name of a parameter and using "sed" to >>> change it to "method" doesn't work.?? Somebody who understands this >>> code and looks at it will have to make the rest of the changes. >>> >>> What I did was replace "method oop" with "method" and "methodOop" >>> with "method" in all the sources.? I replaced "method_oop" with >>> "method" or "checked_method" in the shared sources. >>> >>>> >>>> src/hotspot/share/adlc/adlparse.cpp >>>> >>>> +? frame->_interpreter_method_oop_reg = parse_one_arg("method reg >>>> entry"); >>>> >>>> I guess I'm not understanding the scope of this renaming - why is >>>> _interpreter_method_oop_reg not renamed as well? Should this (and >>>> other uses) be parsed as method-(oop-reg) rather than >>>> (method-oop)-reg? >>> >>> I don't know this code, so I'd rather not change any more of it. The >>> comment makes sense changed, even though the variable name still >>> refers to method_oop. >>> >>> Thanks, >>> Coleen >>>> >>>> Otherwise all okay. >>>> >>>> Thanks, >>>> David >>>> >>>>> Thanks, >>>>> Coleen >>> >> > From larry.cable at oracle.com Fri Jul 24 22:59:54 2020 From: larry.cable at oracle.com (Laurence Cable) Date: Fri, 24 Jul 2020 15:59:54 -0700 Subject: No way to disable Java Container Metrics In-Reply-To: References: Message-ID: On 7/24/20 12:21 PM, Bob Vandette wrote: > I?ve been monitoring the discussion on your jdk8u alias and I can?t help wondering if > my original decision to provide two different implementations of this container detection > logic was the best way to go. > > I didn?t want to have an all Java implementation since the VM needs to initialize it?s > memory and cpu sizing very early on prior to its ability to run Java code. +1 makes sense to me. > > If we put all of the logic in the VM, then we?d end up with a pretty wide interface to > the VM and more overhead extracting values (due to JNI) although the Java logic > typically ends up in native code anyway. Do you think of JMX accesses as of sufficiently high frequency such that JNI overhead would become a serious issue? > Having the functionality in the VM > makes it easier to add JFR events. this is a plus > Having a pure Java implementation makes it > easier to support the OS MBeans. as of course is this!!! :) > The maintenance of supporting both implementations > is a bit of a pain. but probably not 2x right? > > Assuming no one has the time or desire to migrate the logic to the VM and provide > Java wrapper logic, Bob, is this something you think worth recording as an ER for future resolution? > I?m ok with what you are proposing. It?s one step on the path. > The VM support and the Java level support are really for two different consumers > but I think it would be cleaner and less confusing to disable both on one flag rather > than support two options. agreed > > Bob. > >> On Jul 24, 2020, at 12:13 PM, Severin Gehwolf wrote: >> >> Hi, >> >> For hotspot one can disable container detection with a simple switch: >> >> $ java -XX:-UseContainerSupport -Xlog:os+container=trace -version >> [0.000s][trace][os,container] OSContainer::init: Initializing Container Support >> [0.000s][trace][os,container] Container Support not enabled >> openjdk version "15-internal" 2020-09-15 >> OpenJDK Runtime Environment (build 15-internal+0-adhoc.sgehwolf.openjdk-head-2) >> OpenJDK 64-Bit Server VM (build 15-internal+0-adhoc.sgehwolf.openjdk-head-2, mixed mode, sharing) >> >> The same cannot be achieved with the Java code, >> jdk.internal.platform.Metrics.java and friends in the JDK. At the time >> Metrics were added the only consumer of them was the Java Launcher via >> -XshowSettings:system. This has changed with JDK-8226575: >> OperatingSystemMXBean should be made container aware. >> >> It is known that in certain cases the container detection code will >> detect for a system to be supposedly in a container where it actually >> isn't: >> https://bugs.openjdk.java.net/browse/JDK-8227006?focusedCommentId=14275604&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14275604 >> >> For hotspot there is a flag, to turn auto-detection off for exactly the >> case when heuristics are wrong: -XX:-UseContainerSupport. However, this >> flag has no effect on the Java metrics code. There is a risk that on >> some systems the auto-detection will be wrong and might cause >> regressions. >> >> I propose to make the Java metrics code adhere to -XX:+/- >> UseContainerSupport flag. Do you think this would be useful? If yes, >> I'll file a bug and propose a patch for it. >> >> Thoughts? Opinions? >> >> Thanks, >> Severin >> From suenaga at oss.nttdata.com Sat Jul 25 02:19:25 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Sat, 25 Jul 2020 11:19:25 +0900 Subject: RFR: 8248362: JVMTI frame operations should use Thread-Local Handshake In-Reply-To: <164ccf3e-9054-4cf3-a970-d2562abd31dd@oracle.com> References: <5c2bba7d-44aa-d5e7-6d5c-949c18ce9668@oss.nttdata.com> <164ccf3e-9054-4cf3-a970-d2562abd31dd@oracle.com> Message-ID: <3fb80424-9234-0a4f-1ec5-2c7f69854ae8@oss.nttdata.com> Thanks Dan! Yasumasa On 2020/07/25 3:24, Daniel D. Daugherty wrote: > On 7/22/20 11:12 PM, Yasumasa Suenaga wrote: >> Hi Dan, >> >> Thanks for your comment! >> I uploaded new webrev: >> >> ? http://cr.openjdk.java.net/~ysuenaga/JDK-8248362/webrev.01/ >> ? Diff from previous webrev: http://hg.openjdk.java.net/jdk/submit/rev/df75038b5449 > > Thumbs up. Reviewed by comparing the previous webrev patch (webrev.00) with > the latest (webrev.01). > > Dan > > >> >> On 2020/07/23 10:04, Daniel D. Daugherty wrote: >>> On 7/22/20 10:38 AM, Yasumasa Suenaga wrote: >>>> Hi all, >>>> >>>> Please review this change: >>>> >>>> ? JBS: https://bugs.openjdk.java.net/browse/JDK-8248362 >>>> ? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8248362/webrev.00/ >>> >>> src/hotspot/share/prims/jvmtiEnv.cpp >>> ???? L1755???? // JVMTI get java stack frame location at safepoint. >>> ???????? You missed updating this one. Perhaps: >>> >>> ?????????????? // JVMTI get java stack frame location via direct handshake. >> >> Fixed. >> >> >>> src/hotspot/share/prims/jvmtiEnvBase.cpp >>> ???? L1563: ? JavaThread* jt = _state->get_thread(); >>> ???? L1564: ? assert(target == jt, "just checking"); >>> ???????? This code is different than the other closures. It might be worth >>> ???????? a comment to explain why. I don't remember why VM_GetFrameCount had >>> ???????? to use the JvmtiThreadState to fetch the JavaThread. Serguei might >>> ???????? remember. >>> >>> ???????? It could be that we don't need the _state field anymore because of >>> ???????? the way that handshakes work (and provide the JavaThread* to the >>> ???????? do_thread() function). Your choice on whether to deal with that as >>> ???????? part of this fix or following with another RFE. >>> >>> ???????? Update: Uggg.... the get_frame_count() function takes JvmtiThreadState >>> ???????? as a parameter. This is very much entangled... sigh... we should >>> ???????? definitely look at untangling this in another RFE... >> >> I want to fix it in another RFE as you said if it is needed. >> If we don't hear the reason from Serguei, I want to keep this change. >> >> >>> ???? old L1565: ? ThreadsListHandle tlh; >>> ???? new L1565: ? if (!jt->is_exiting() && jt->threadObj() != NULL) { >>> >>> ???????? Please consider add this comment above L1565: >>> ????????????????? // ThreadsListHandle is not needed due to direct handshake. >>> >>> ???????? Yes, I see that previous closures were added without that comment, >>> ???????? but when I see "if (!jt->is_exiting() && jt->threadObj() != NULL)" >>> ???????? I immediately wonder where the ThreadsListHandle is... Please consider >>> ???????? adding the comment to the others also. >>> >>> ???? old L1574: ? ThreadsListHandle tlh; >>> ???? new L1574: ? if (!jt->is_exiting() && jt->threadObj() != NULL) { >>> >>> ???????? Please consider add this comment above L1574: >>> ????????????????? // ThreadsListHandle is not needed due to direct handshake. >> >> I think it is new normal as David said in the reply, and also other closures don't say about it. >> So I did not change about it in new webrev. >> >> >>> src/hotspot/share/prims/jvmtiEnvBase.hpp >>> ???? L580: // HandshakeClosure to frame location. >>> ???????? typo - s/to frame/to get frame/ >> >> Fixed. >> >> >>> src/hotspot/share/prims/jvmtiThreadState.cpp >>> ???? No comments on the changes. >>> >>> ???? For the above comments about VM_GetFrameCount, understanding why >>> ???? JvmtiThreadState::count_frames() has to exist in JvmtiThreadState >>> ???? will go along way towards untangling the mess. >>> >>> src/hotspot/share/runtime/vmOperations.hpp >>> ???? No comments. >>> >>> >>> Thumbs up. I only have nits above. If you choose to make the above >>> changes, I do not need to see another webrev. >> >> Thanks, but I uploaded new webrev just in case :) >> >> >> Yasumasa >> >> >>> Dan >>> >>> >>> >>>> >>>> Migrate JVMTI frame operations to Thread-Local Handshakes from VM Operations. >>>> >>>> ??? - VM_GetFrameCount >>>> ??? - VM_GetFrameLocation >>>> >>>> They affects both GetFrameCount() and GetFrameLocation() JVMTI functions. >>>> >>>> This change has passed all tests on submit repo (mach5-one-ysuenaga-JDK-8248362-20200722-1249-12859056), and vmTestbase/nsk/{jdi,jdw,jvmti}, serviceability/{jdwp,jvmti} . >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>> > From joe.darcy at oracle.com Sat Jul 25 17:52:51 2020 From: joe.darcy at oracle.com (Joe Darcy) Date: Sat, 25 Jul 2020 10:52:51 -0700 Subject: JDK 16 RFR of JDK-8250583: Address reliance on default constructors in java.management Message-ID: <5cb51cfc-da7a-32aa-eb22-4383d18cce1e@oracle.com> Hello, I'm not positive serviceability-dev is the best alias for this review; if not, please direct me elsewhere. In any case, please review the changes and CSR for: ??? JDK-8250583: Address reliance on default constructors in java.management ??? webrev: http://cr.openjdk.java.net/~darcy/8250583.0/ ??? CSR: https://bugs.openjdk.java.net/browse/JDK-8250584 (This is part of a larger library clean-up effort to remove reliance on default constructors in the JDK ahead of a new javac lint warning, JDK-8071961.) Generally the changes adds a public no-arg constructor equivalent to the implicit default constructor. In the case of RoleStatus, since it doesn't seem that the class should have a public constructor at all, I terminally deprecated the constructor. Patch below. Thanks, -Joe --- old/src/java.management/share/classes/javax/management/AttributeChangeNotificationFilter.java 2020-07-25 08:53:04.404999999 -0700 +++ new/src/java.management/share/classes/javax/management/AttributeChangeNotificationFilter.java 2020-07-25 08:53:03.720999999 -0700 @@ -1,5 +1,5 @@ ?/* - * Copyright (c) 1999, 2006, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1999, 2020, Oracle and/or its affiliates. All rights reserved. ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. ? * ? * This code is free software; you can redistribute it and/or modify it @@ -50,6 +50,10 @@ ????? */ ???? private Vector enabledAttributes = new Vector(); +??? /** +???? * Constructs an {@code AttributeChangeNotificationFilter}. +???? */ +??? public AttributeChangeNotificationFilter() {} ???? /** ????? * Invoked before sending the specified notification to the listener. --- old/src/java.management/share/classes/javax/management/DefaultLoaderRepository.java 2020-07-25 08:53:05.472999999 -0700 +++ new/src/java.management/share/classes/javax/management/DefaultLoaderRepository.java 2020-07-25 08:53:04.812999999 -0700 @@ -1,5 +1,5 @@ ?/* - * Copyright (c) 1999, 2008, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1999, 2020, Oracle and/or its affiliates. All rights reserved. ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. ? * ? * This code is free software; you can redistribute it and/or modify it @@ -51,6 +51,11 @@ ?@Deprecated ?public class DefaultLoaderRepository { ???? /** +???? * Constructs an {@code DefaultLoaderRepository}. +???? */ +??? public DefaultLoaderRepository() {} + +??? /** ????? * Go through the list of class loaders and try to load the requested class. ????? * The method will stop as soon as the class is found. If the class ????? * is not found the method will throw a ClassNotFoundException --- old/src/java.management/share/classes/javax/management/NotificationFilterSupport.java 2020-07-25 08:53:06.584999999 -0700 +++ new/src/java.management/share/classes/javax/management/NotificationFilterSupport.java 2020-07-25 08:53:05.876999999 -0700 @@ -1,5 +1,5 @@ ?/* - * Copyright (c) 1999, 2019, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1999, 2020, Oracle and/or its affiliates. All rights reserved. ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. ? * ? * This code is free software; you can redistribute it and/or modify it @@ -69,6 +69,11 @@ ???? /** +???? * Constructs a {@code NotificationFilterSupport}. +???? */ +??? public NotificationFilterSupport() {} + +??? /** ????? * Invoked before sending the specified notification to the listener. ????? *
This filter compares the type of the specified notification with each enabled type. ????? * If the notification type matches one of the enabled types, --- old/src/java.management/share/classes/javax/management/QueryEval.java 2020-07-25 08:53:07.704999999 -0700 +++ new/src/java.management/share/classes/javax/management/QueryEval.java 2020-07-25 08:53:07.004999999 -0700 @@ -1,5 +1,5 @@ ?/* - * Copyright (c) 1999, 2008, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1999, 2020, Oracle and/or its affiliates. All rights reserved. ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. ? * ? * This code is free software; you can redistribute it and/or modify it @@ -42,6 +42,11 @@ ???????? new InheritableThreadLocal(); ???? /** +???? * Constructor for subclasses to call. +???? */ +??? public QueryEval() {} + +??? /** ????? *

Sets the MBean server on which the query is to be performed. ????? * The setting is valid for the thread performing the set. ????? * It is copied to any threads created by that thread at the moment --- old/src/java.management/share/classes/javax/management/loading/DefaultLoaderRepository.java 2020-07-25 08:53:08.824999999 -0700 +++ new/src/java.management/share/classes/javax/management/loading/DefaultLoaderRepository.java 2020-07-25 08:53:08.124999999 -0700 @@ -1,5 +1,5 @@ ?/* - * Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved. ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. ? * ? * This code is free software; you can redistribute it and/or modify it @@ -56,6 +56,11 @@ ?public class DefaultLoaderRepository { ???? /** +???? * Constructs a {@code DefaultLoaderRepository}. +???? */ +??? public DefaultLoaderRepository() {} + +??? /** ????? * Go through the list of class loaders and try to load the requested ????? * class. ????? * The method will stop as soon as the class is found. If the class --- old/src/java.management/share/classes/javax/management/monitor/Monitor.java 2020-07-25 08:53:09.916999999 -0700 +++ new/src/java.management/share/classes/javax/management/monitor/Monitor.java 2020-07-25 08:53:09.244999999 -0700 @@ -1,5 +1,5 @@ ?/* - * Copyright (c) 1999, 2017, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1999, 2020, Oracle and/or its affiliates. All rights reserved. ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. ? * ? * This code is free software; you can redistribute it and/or modify it @@ -76,6 +76,11 @@ ???? extends NotificationBroadcasterSupport ???? implements MonitorMBean, MBeanRegistration { +??? /** +???? * Constructor for subclasses to call. +???? */ +??? public Monitor() {} + ???? /* ????? * ------------------------------------------ ????? *? PACKAGE CLASSES --- old/src/java.management/share/classes/javax/management/relation/RoleStatus.java 2020-07-25 08:53:11.060999999 -0700 +++ new/src/java.management/share/classes/javax/management/relation/RoleStatus.java 2020-07-25 08:53:10.384999999 -0700 @@ -1,5 +1,5 @@ ?/* - * Copyright (c) 2000, 2005, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved. ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. ? * ? * This code is free software; you can redistribute it and/or modify it @@ -33,6 +33,12 @@ ? */ ?public class RoleStatus { +??? /** +???? * Do not call. +???? */ +??? @Deprecated(since="16", forRemoval=true) +??? public RoleStatus() {} + ???? // ???? // Possible problems ???? // From Alan.Bateman at oracle.com Sat Jul 25 18:18:30 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 25 Jul 2020 19:18:30 +0100 Subject: JDK 16 RFR of JDK-8250583: Address reliance on default constructors in java.management In-Reply-To: <5cb51cfc-da7a-32aa-eb22-4383d18cce1e@oracle.com> References: <5cb51cfc-da7a-32aa-eb22-4383d18cce1e@oracle.com> Message-ID: <3b91a833-b041-2218-3114-0c0268f06d4c@oracle.com> On 25/07/2020 18:52, Joe Darcy wrote: > Hello, > > I'm not positive serviceability-dev is the best alias for this review; > if not, please direct me elsewhere. > > In any case, please review the changes and CSR for: > > ??? JDK-8250583: Address reliance on default constructors in > java.management > ??? webrev: http://cr.openjdk.java.net/~darcy/8250583.0/ > ??? CSR: https://bugs.openjdk.java.net/browse/JDK-8250584 > > (This is part of a larger library clean-up effort to remove reliance > on default constructors in the JDK ahead of a new javac lint warning, > JDK-8071961.) > > Generally the changes adds a public no-arg constructor equivalent to > the implicit default constructor. In the case of RoleStatus, since it > doesn't seem that the class should have a public constructor at all, I > terminally deprecated the constructor. serviceability-dev is fine as it has mostly subsumed the jmx-dev list. The changes look okay to me. -Alan From david.holmes at oracle.com Mon Jul 27 01:26:31 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 27 Jul 2020 11:26:31 +1000 Subject: RFR (T) 8250042: Clean up methodOop and method_oop names from the code In-Reply-To: References: <85efc3ab-abbf-c5f2-9b7b-47fa516d9a2d@oracle.com> <6f973a0a-cf55-e1ab-8de3-b57f68dbd2cf@oracle.com> Message-ID: <8eab1729-9a35-8c79-65cf-e67a098965d5@oracle.com> Hi Coleen, On 24/07/2020 11:10 pm, coleen.phillimore at oracle.com wrote: > > I can also replace method_oop with method_ptr in the CPU ad files, and > this seems to build but now someone who knows the compiler area needs to > comment; this was supposed to be trivial... :)? But it still is really > trivial to look at. method_ptr works for me. Changes seem fine. > I left interpreter_method_oop_reg and compiler_method_oop_reg and > friends in opto/matcher.cpp for someone else. Okay. Hopefully someone will pick it up. Thanks, David ----- > incremental webrev at > http://cr.openjdk.java.net/~coleenp/2020/8250042.02.incr/webrev > full webrev at http://cr.openjdk.java.net/~coleenp/2020/8250042.02/webrev > > Thanks, > Coleen > > > On 7/24/20 8:23 AM, coleen.phillimore at oracle.com wrote: >> >> Thanks for looking at this. >> >> On 7/24/20 1:01 AM, David Holmes wrote: >>> Hi Coleen, >>> >>> On 24/07/2020 2:58 am, coleen.phillimore at oracle.com wrote: >>>> See bug for more details.? I've been running into these names a lot >>>> lately.?? Many of these names are in JVMTI. >>>> >>>> Tested with tier1 on all Oracle platforms and built on non-Oracle >>>> platforms. >>>> >>>> open webrev at >>>> http://cr.openjdk.java.net/~coleenp/2020/8250042.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8250042 >>> >>> src/hotspot/cpu/*/*.ad >>> >>> These still refer to "method oop" and method_oop in a number of places. >> >> Yes, I only replaced method_oop in the shared code and not in the AD >> code.? method_oop can be the name of a parameter and using "sed" to >> change it to "method" doesn't work.?? Somebody who understands this >> code and looks at it will have to make the rest of the changes. >> >> What I did was replace "method oop" with "method" and "methodOop" with >> "method" in all the sources.? I replaced "method_oop" with "method" or >> "checked_method" in the shared sources. >> >>> >>> src/hotspot/share/adlc/adlparse.cpp >>> >>> +? frame->_interpreter_method_oop_reg = parse_one_arg("method reg >>> entry"); >>> >>> I guess I'm not understanding the scope of this renaming - why is >>> _interpreter_method_oop_reg not renamed as well? Should this (and >>> other uses) be parsed as method-(oop-reg) rather than (method-oop)-reg? >> >> I don't know this code, so I'd rather not change any more of it. The >> comment makes sense changed, even though the variable name still >> refers to method_oop. >> >> Thanks, >> Coleen >>> >>> Otherwise all okay. >>> >>> Thanks, >>> David >>> >>>> Thanks, >>>> Coleen >> > From linzang at tencent.com Mon Jul 27 03:26:16 2020 From: linzang at tencent.com (=?utf-8?B?bGluemFuZyjoh6fnkLMp?=) Date: Mon, 27 Jul 2020 03:26:16 +0000 Subject: RFR(L): 8215624: add parallel heap inspection support for jmap histo(G1)(Internet mail) In-Reply-To: References: Message-ID: <2C4A6B09-F281-433E-95CE-9587FC3C769F@tencent.com> I update a new change at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_09 It includes a tiny fix of build failure on windows: #################################### In attachListener.cpp: - uint parallel_thread_num = MAX(1, (uint)os::initial_active_processor_count() * 3 / 8); + uint parallel_thread_num = MAX2(1, (uint)os::initial_active_processor_count() * 3 / 8); #################################### BRs, Lin ?On 2020/7/23, 11:56 AM, "linzang(??)" wrote: Hi Paul, Thanks for your help, that all looks good to me. Just 2 minor changes: ? delete the final return in ParHeapInspectTask::work, you mentioned it but seems not include in the webrev :-) ? delete a unnecessary blank line in heapInspect.cpp at merge_entry() ######################################################################### --- old/src/hotspot/share/memory/heapInspection.cpp 2020-07-23 11:23:29.281666456 +0800 +++ new/src/hotspot/share/memory/heapInspection.cpp 2020-07-23 11:23:29.017666447 +0800 @@ -251,7 +251,6 @@ _size_of_instances_in_words += cie->words(); return true; } - return false; } @@ -568,7 +567,6 @@ Atomic::add(&_missed_count, missed_count); } else { Atomic::store(&_success, false); - return; } } ######################################################################### Here is the webrev http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_08/ BRs, Lin --------------------------------------------- From: "Hohensee, Paul" Date: Thursday, July 23, 2020 at 6:48 AM To: "linzang(??)" , Stefan Karlsson , "serguei.spitsyn at oracle.com" , David Holmes , serviceability-dev , "hotspot-gc-dev at openjdk.java.net" Subject: RE: RFR(L): 8215624: add parallel heap inspection support for jmap histo(G1)(Internet mail) Just small things. heapInspection.cpp: In ParHeapInspectTask::work, remove the final return statement and fix the following ?}? indent. I.e., replace + Atomic::store(&_success, false); + return; + } with + Atomic::store(&_success, false); + } In HeapInspection::heap_inspection, missed_count should be a uint to match other missed_count declarations, and should be initialized to the result of populate_table() rather than separately to 0. attachListener.cpp: In heap_inspection, initial_processor_count returns an int, so cast its result to a uint. Similarly, parse_uintx returns a uintx, so cast its result (num) to uint when assigning to parallel_thread_num. BasicJMapTest.java: I took the liberty of refactoring testHisto*/histoToFile/testDump*/dump to remove redundant interposition methods and make histoToFile and dump look as similar as possible. Webrev with the above changes in http://cr.openjdk.java.net/~phh/8214535/webrev.01/ Thanks, Paul On 7/15/20, 2:13 AM, "linzang(??)" wrote: Upload a new webrev at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_07/ It fix a potential issue that unexpected number of threads maybe calculated for "parallel" option of jmap -histo in container. As shown at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_07-delta/src/hotspot/share/services/attachListener.cpp.udiff.html ############### attachListener.cpp #################### @@ -252,11 +252,11 @@ static jint heap_inspection(AttachOperation* op, outputStream* out) { bool live_objects_only = true; // default is true to retain the behavior before this change is made outputStream* os = out; // if path not specified or path is NULL, use out fileStream* fs = NULL; const char* arg0 = op->arg(0); - uint parallel_thread_num = MAX(1, os::processor_count() * 3 / 8); // default is less than half of processors. + uint parallel_thread_num = MAX(1, os::initial_active_processor_count() * 3 / 8); // default is less than half of processors. if (arg0 != NULL && (strlen(arg0) > 0)) { if (strcmp(arg0, "-all") != 0 && strcmp(arg0, "-live") != 0) { out->print_cr("Invalid argument to inspectheap operation: %s", arg0); return JNI_ERR; } ################################################### Thanks. BRs, Lin On 2020/7/9, 3:22 PM, "linzang(??)" wrote: Hi Paul, Thanks for reviewing! >> >> I'd move all the argument parsing code to JMap.java and just pass the results to Hotspot. Both histo() in JMap.java and code in attachListener.* parse the command line arguments, though the code in histo() doesn't parse the argument to "parallel". I'd upgrade the code in histo() to do a complete parse and pass the option values to executeCommandForPid as before: there would just be more of them now. That would allow you to eliminate all the parsing code in attachListener.cpp as well as the change to arguments.hpp. >> The reason I made the change in Jmap.java that compose all arguments as 1 string , instead of passing 3 argments, is to avoid the compatibility issue, as we discussed in http://mail.openjdk.java.net/pipermail/serviceability-dev/2019-February/thread.html#27240. The root cause of the compatibility issue is because max argument count in HotspotVirtualMachineImpl.java and attachlistener.cpp need to be enlarged (changes like http://hg.openjdk.java.net/jdk/jdk/rev/e7cf035682e3#l2.1) when jmap has more than 3 arguments. But if user use an old jcmd/jmap tool, it may stuck at socket read(), because the "max argument count" don't match. I re-checked this change, the argument count of jmap histo is equal to 3 (live, file, parallel), so it can work normally even without the change of passing argument. But I think we have to face the problem if more arguments is added in jcmd alike tools later, not sure whether it should be sloved (or a workaround) in this changeset. And here are the lastest webrev and delta: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_06/ http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_06-delta/ Cheers, Lin On 2020/7/7, 5:57 AM, "Hohensee, Paul" wrote: I'd like to see this feature added. :) The CSR looks good, as does the basic parallel inspection algorithm. Stefan's done the GC part, so I'll stick to the non-GC part (fwiw, the GC part lgtm). I'd move all the argument parsing code to JMap.java and just pass the results to Hotspot. Both histo() in JMap.java and code in attachListener.* parse the command line arguments, though the code in histo() doesn't parse the argument to "parallel". I'd upgrade the code in histo() to do a complete parse and pass the option values to executeCommandForPid as before: there would just be more of them now. That would allow you to eliminate all the parsing code in attachListener.cpp as well as the change to arguments.hpp. heapInspection.hpp: _shared_miss_count (s/b _missed_count, see below) isn't a size, so it should be a uint instead of a size_t. Same with the new parallel_thread_num argument to heap_inspection() and populate_table(). Comment copy-edit: +// Parallel heap inspection task. Parallel inspection can fail due to +// a native OOM when allocating memory for TL-KlassInfoTable. +// _success will be set false on an OOM, and serial inspection tried. _shared_miss_count should be _missed_count to match the missed_count() getter, or rename missed_count() to be shared_miss_count(). Whichever way you go, the field type should match the getter result type: uint is reasonable. heapInspection.cpp: You might use ResourceMark twice in populate_table, separately for the parallel attempt and the serial code. If the parallel attempt fails and available memory is low, it would be good to clean up the memory used by the parallel attempt before doing the serial code. Style nit in KlassInfoTable::merge_entry(). I'd line up the definitions of k and elt, so "k" is even with "elt". And, because it's two lines shorter, I'd replace + } else { + return false; + } with + return false; KlassInfoTableMergeClosure.is_success() should be just success() (i.e., no "is_" prefix) because it's a getter. I'd reorganize the code in populate_table() to make it more clear, vis (I changed _shared_missed_count to _missed_count) + if (cit.allocation_failed()) { + // fail to allocate memory, stop parallel mode + Atomic::store(&_success, false); + return; + } + RecordInstanceClosure ric(&cit, _filter); + _poi->object_iterate(&ric, worker_id); + missed_count = ric.missed_count(); + { + MutexLocker x(&_mutex); + merge_success = _shared_cit->merge(&cit); + } + if (merge_success) { + Atomic::add(&_missed_count, missed_count); + else { + Atomic::store(&_success, false); + } Thanks, Paul On 6/29/20, 7:20 PM, "linzang(??)" wrote: Dear All, Sorry to bother again, I just want to make sure that is this change worth to be continue to work on? If decision is made to not. I think I can drop this work and stop asking for help reviewing... Thanks for all your help about reviewing this previously. BRs, Lin On 2020/5/9, 3:47 PM, "linzang(??)" wrote: Dear All, May I ask your help again for review the latest change? Thanks! BRs, Lin On 2020/4/28, 1:54 PM, "linzang(??)" wrote: Hi Stefan, >> - Adding Atomic::load/store. >> - Removing the time measurement in the run_task. I renamed G1's function >> to run_task_timed. If we need this outside of G1, we can rethink the API >> at that point. >> - ZGC style cleanups Thanks for revising the patch, they are all good to me, and I have made a tiny change based on it: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04/ http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04-delta/ it reduce the scope of mutex in ParHeapInspectTask, and delete unnecessary comments. BRs, Lin On 2020/4/27, 4:34 PM, "Stefan Karlsson" wrote: Hi Lin, On 2020-04-26 05:10, linzang(??) wrote: > Hi Stefan and Paul? > I have made a new patch based on your comments and Stefan's Poc code: > Webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03/ > Delta(based on Stefan's change:) : http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03-delta/webrev_03-delta/ Thanks for providing a delta patch. It makes it much easier to look at, and more likely for reviewers to continue reviewing. I'm going to continue focusing on the GC parts, and leave the rest to others to review. > > And Here are main changed I made and want to discuss with you: > 1. changed"parallelThreadNum=" to "parallel=" for jmap -histo options. > 2. Add logic to test where parallelHeapInspection is fail, in heapInspection.cpp > This is because the parHeapInspectTask create thread local KlassInfoTable in it's work() method, and this may fail because of native OOM, in this case, the parallel should fail and serial heap inspection can be tried. > One more thing I want discuss with you is about the member "_success" of parHeapInspectTask, when native OOM happenes, it is set to false. And since this "set" operation can be conducted in multiple threads, should it be atomic ops? IMO, this is not necessary because "_success" can only be set to false, and there is no way to change it from back to true after the ParHeapInspectTask instance is created, so it is save to be non-atomic, do you agree with that? In these situations you should be using the Atomic::load/store primitives. We're moving toward a later C++ standard were data races are considered undefined behavior. > 3. make CollectedHeap::run_task() be an abstract virtual func, so that every subclass of collectedHeap should support it, so later implementation of new collectedHeap will not miss the "parallel" features. > The problem I want to discuss with you is about epsilonHeap and SerialHeap, as they may not need parallel heap iteration, so I only make task->work(0), in case the run_task() is invoked someway in future. Another way is to left run_task() unimplemented, which one do you think is better? I don't have a strong opinion about this. And also please help take a look at the zHeap, as there is a class zTask that wrap the abstractGangTask, and the collectedHeap::run_task() only accept AbstraceGangTask* as argument, so I made a delegate class to adapt it , please see src/hotspot/share/gc/z/zHeap.cpp. > > There maybe other better ways to sovle the above problems, welcome for any comments, Thanks! I've created a few cleanups and changes on top of your latest patch: https://cr.openjdk.java.net/~stefank/8215624/webrev.02.delta https://cr.openjdk.java.net/~stefank/8215624/webrev.02 - Adding Atomic::load/store. - Removing the time measurement in the run_task. I renamed G1's function to run_task_timed. If we need this outside of G1, we can rethink the API at that point. - ZGC style cleanups Thanks, StefanK > > BRs, > Lin > > On 2020/4/23, 11:08 AM, "linzang(??)" wrote: > > Thanks Paul! I agree with using "parallel", will make the update in next patch, Thanks for help update the CSR. > > BRs, > Lin > > On 2020/4/23, 4:42 AM, "Hohensee, Paul" wrote: > > For the interface, I'd use "parallel" instead of "parallelThreadNum". All the other options are lower case, and it's a lot easier to type "parallel". I took the liberty of updating the CSR. If you're ok with it, you might want to change variable names and such, plus of course JMap.usage. > > Thanks, > Paul > > On 4/22/20, 2:29 AM, "serviceability-dev on behalf of linzang(??)" wrote: > > Dear Stefan, > > Thanks a lot! I agree with you to decouple the heap inspection code with GC's. > I will start from your POC code, may discuss with you later. > > > BRs, > Lin > > On 2020/4/22, 5:14 PM, "Stefan Karlsson" wrote: > > Hi Lin, > > I took a look at this earlier and saw that the heap inspection code is > strongly coupled with the CollectedHeap and G1CollectedHeap. I'd prefer > if we'd abstract this away, so that the GCs only provide a "parallel > object iteration" interface, and the heap inspection code is kept elsewhere. > > I started experimenting with doing that, but other higher-priority (to > me) tasks have had to take precedence. > > I've uploaded my work-in-progress / proof-of-concept: > https://cr.openjdk.java.net/~stefank/8215624/webrev.01.delta/ > https://cr.openjdk.java.net/~stefank/8215624/webrev.01/ > > The current code doesn't handle the lifecycle (deletion) of the > ParallelObjectIterators. There's also code left unimplemented in around > CollectedHeap::run_task. However, I think this could work as a basis to > pull out the heap inspection code out of the GCs. > > Thanks, > StefanK > > On 2020-04-22 02:21, linzang(??) wrote: > > Dear all, > > May I ask you help to review? This RFR has been there for quite a while. > > Thanks! > > > > BRs, > > Lin > > > > > On 2020/3/16, 5:18 PM, "linzang(??)" wrote:> > > > >> Just update a new path, my preliminary measure show about 3.5x speedup of jmap histo on a nearly full 4GB G1 heap (8-core platform with parallel thread number set to 4). > >> webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_02/ > >> bug: https://bugs.openjdk.java.net/browse/JDK-8215624 > >> CSR: https://bugs.openjdk.java.net/browse/JDK-8239290 > >> BRs, > >> Lin > >> > On 2020/3/2, 9:56 PM, "linzang(??)" wrote: > >> > > >> > Dear all, > >> > Let me try to ease the reviewing work by some explanation :P > >> > The patch's target is to speed up jmap -histo for heap iteration, from my experience it is necessary for large heap investigation. E.g in bigData scenario I have tried to conduct jmap -histo against 180GB heap, it does take quite a while. > >> > And if my understanding is corrent, even the jmap -histo without "live" option does heap inspection with heap lock acquired. so it is very likely to block mutator thread in allocation-sensitive scenario. I would say the faster the heap inspection does, the shorter the mutator be blocked. This is parallel iteration for jmap is necessary. > >> > I think the parallel heap inspection should be applied to all kind of heap. However, consider the heap layout are different for GCs, much time is required to understand all kinds of the heap layout to make the whole change. IMO, It is not wise to have a huge patch for the whole solution at once, and it is even harder to review it. So I plan to implement it incrementally, the first patch (this one) is going to confirm the implemention detail of how jmap accept the new option, passes it to attachListener of the jvm process and then how to make the parallel inspection closure be generic enough to make it easy to extend to different heap layout. And also how to implement the heap inspection in specific gc's heap. This patch use G1's heap as the begining. > >> > This patch actually do several things: > >> > 1. Add an option "parallelThreadNum=" to jmap -histo, the default behavior is to set N to 0, means let's JVM decide how many threads to use for heap inspection. Set this option to 1 will disable parallel heap inspection. (more details in CSR: https://bugs.openjdk.java.net/browse/JDK-8239290) > >> > 2. Make a change in how Jmap passing arguments, changes in http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java.udiff.html, originally it pass options as separate arguments to attachListener, this patch change to that all options be compose to a single string. So the arg_count_max in attachListener.hpp do not need to be changed, and hence avoid the compatibility issue, as disscussed at https://mail.openjdk.java.net/pipermail/serviceability-dev/2019-March/027334.html > >> > 3. Add an abstract class ParHeapInspectTask in heapInspection.hpp / heapInspection.cpp, It's work(uint worker_id) method prepares the data structure (KlassInfoTable) need for every parallel worker thread, and then call do_object_iterate_parallel() which is heap specific implementation. I also added some machenism in KlassInfoTable to support parallel iteration, such as merge(). > >> > 4. In specific heap (G1 in this patch), create a subclass of ParHeapInspectTask, implement the do_object_iterate_parallel() for parallel heap inspection. For G1, it simply invoke g1CollectedHeap's object_iterate_parallel(). > >> > 5. Add related test. > >> > 6. it may be easy to extend this patch for other kinds of heap by creating subclass of ParHeapInspectTask and implement the do_object_iterate_parallel(). > >> > > >> > Hope these info could help on code review and initate the discussion :-) > >> > Thanks! > >> > > >> > BRs, > >> > Lin > >> > >On 2020/2/19, 9:40 AM, "linzang(??)" wrote:. > >> > > > >> > > Re-post this RFR with correct enhancement number to make it trackable. > >> > > please ignore the previous wrong post. sorry for troubles. > >> > > > >> > > webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/ > >> > > Hi bug: https://bugs.openjdk.java.net/browse/JDK-8215624 > >> > > CSR: https://bugs.openjdk.java.net/browse/JDK-8239290 > >> > > -------------- > >> > > Lin > >> > > >Hi Lin, > > > > > > > >> > > >Could you, please, re-post your RFR with the right enhancement number in > >> > > >the message subject? > >> > > >It will be more trackable this way. > >> > > > > >> > > >Thanks, > >> > > >Serguei > >> > > > > >> > > > > >> > > >On 2/17/20 10:29 PM, linzang(??) wrote: > >> > > >> Dear David, > >> > > >> Thanks a lot! > >> > > >> I have updated the refined code to http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_01/. > >> > > >> IMHO the parallel heap inspection can be extended to all kinds of heap as long as the heap layout can support parallel iteration. > >> > > >> Maybe we can firstly use this webrev to discuss how to implement it, because I am not sure my current implementation is an appropriate way to communicate with collectedHeap, then we can extend the solution to other kinds of heap. > >> > > >> > >> > > >> Thanks, > >> > > >> -------------- > >> > > >> Lin > >> > > >>> Hi Lin, > >> > > >>> > >> > > >>> Adding in hotspot-gc-dev as they need to see how this interacts with GC > >> > > >>> worker threads, and whether it needs to be extended beyond G1. > >> > > >>> > >> > > >>> I happened to spot one nit when browsing: > >> > > >>> > >> > > >>> src/hotspot/share/gc/shared/collectedHeap.hpp > >> > > >>> > >> > > >>> + virtual bool run_par_heap_inspect_task(KlassInfoTable* cit, > >> > > >>> + BoolObjectClosure* filter, > >> > > >>> + size_t* missed_count, > >> > > >>> + size_t thread_num) { > >> > > >>> + return NULL; > >> > > >>> > >> > > >>> s/NULL/false/ > >> > > >>> > >> > > >>> Cheers, > >> > > >>> David > > > > > >>> > >> > > >>> On 18/02/2020 2:15 pm, linzang(??) wrote: > >> > > >>>> Dear All, > >> > > >>>> May I ask your help to review the follow changes: > >> > > >>>> webrev: > >> > > >>>> http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_00/ > >> > > >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8215624 > >> > > >>>> related CSR: https://bugs.openjdk.java.net/browse/JDK-8239290 > >> > > >>>> This patch enable parallel heap inspection of G1 for jmap histo. > >> > > >>>> my simple test shown it can speed up 2x of jmap -histo with > >> > > >>>> parallelThreadNum set to 2 for heap at ~500M on 4-core platform. > >> > > >>>> > >> > > >>>> ------------------------------------------------------------------------ > >> > > >>>> BRs, > >> > > >>>> Lin > >> > > >> > > >> > > > > > > > > > > > > > > > From richard.reingruber at sap.com Mon Jul 27 07:45:11 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Mon, 27 Jul 2020 07:45:11 +0000 Subject: RFR(S) 8249293: Unsafe stackwalk in VM_GetOrSetLocal::doit_prologue() In-Reply-To: References: Message-ID: Hi Serguei, > I tested it on Linux and Windows but not yet on MacOS. The test succeeded now on all platforms. Thanks, Richard. -----Original Message----- From: Reingruber, Richard Sent: Freitag, 24. Juli 2020 15:04 To: serguei.spitsyn at oracle.com; serviceability-dev at openjdk.java.net Subject: RE: RFR(S) 8249293: Unsafe stackwalk in VM_GetOrSetLocal::doit_prologue() Hi Serguei, > The fix itself looks good to me. thanks for looking at the fix. > I still need another look at new test. > Could you, please, convert the agent of new test to C++? > It will make it a little bit simpler. Sure, here is the new webrev.1 with a C++ version of the test agent: http://cr.openjdk.java.net/~rrich/webrevs/8249293/webrev.1/ I tested it on Linux and Windows but not yet on MacOS. Thanks, Richard. -----Original Message----- From: serguei.spitsyn at oracle.com Sent: Freitag, 24. Juli 2020 00:00 To: Reingruber, Richard ; serviceability-dev at openjdk.java.net Subject: Re: RFR(S) 8249293: Unsafe stackwalk in VM_GetOrSetLocal::doit_prologue() Hi Richard, Thank you for filing the CR and taking care about it! The fix itself looks good to me. I still need another look at new test. Could you, please, convert the agent of new test to C++? It will make it a little bit simpler. Thanks, Serguei On 7/20/20 01:15, Reingruber, Richard wrote: > Hi, > > please help review this fix for VM_GetOrSetLocal. It moves the unsafe stackwalk from the vm > operation prologue before the safepoint into the doit() method executed at the safepoint. > > Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8249293/webrev.0/index.html > Bug: https://bugs.openjdk.java.net/browse/JDK-8249293 > > According to the JVMTI spec on local variable access it is not required to suspend the target thread > T [1]. The operation will simply fail with JVMTI_ERROR_NO_MORE_FRAMES if T is executing > bytecodes. It will succeed though if T is blocked because of synchronization or executing some native > code. > > The issue is that in the latter case the stack walk in VM_GetOrSetLocal::doit_prologue() to prepare > the access to the local variable is unsafe, because it is done before the safepoint and it races > with T returning to execute bytecodes making its stack not walkable. The included test shows that > this can crash the VM if T wins the race. > > Manual testing: > > - new test test/hotspot/jtreg/serviceability/jvmti/GetLocalVariable/GetLocalWithoutSuspendTest.java > - test/hotspot/jtreg/vmTestbase/nsk/jvmti > - test/hotspot/jtreg/serviceability/jvmti > > Nightly regression tests @SAP: JCK and JTREG, also in Xcomp mode, SPECjvm2008, SPECjbb2015, > Renaissance Suite, SAP specific tests with fastdebug and release builds on all platforms > > Thanks, Richard. > > [1] https://docs.oracle.com/en/java/javase/14/docs/specs/jvmti.html#local From sgehwolf at redhat.com Mon Jul 27 15:32:54 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Mon, 27 Jul 2020 17:32:54 +0200 Subject: No way to disable Java Container Metrics In-Reply-To: References: Message-ID: Hi Bob, On Fri, 2020-07-24 at 15:21 -0400, Bob Vandette wrote: > I?ve been monitoring the discussion on your jdk8u alias and I can?t help wondering if > my original decision to provide two different implementations of this container detection > logic was the best way to go. > > I didn?t want to have an all Java implementation since the VM needs to initialize it?s > memory and cpu sizing very early on prior to its ability to run Java code. > > If we put all of the logic in the VM, then we?d end up with a pretty wide interface to > the VM and more overhead extracting values (due to JNI) although the Java logic > typically ends up in native code anyway. Having the functionality in the VM > makes it easier to add JFR events. Having a pure Java implementation makes it > easier to support the OS MBeans. The maintenance of supporting both implementations > is a bit of a pain. Add to that that Java metrics return non-null for any controller it finds. The JVM doesn't. Container support is turned on there only if all cpu and memory controllers are found. > Assuming no one has the time or desire to migrate the logic to the VM and provide > Java wrapper logic, I?m ok with what you are proposing. It?s one step on the path. > The VM support and the Java level support are really for two different consumers > but I think it would be cleaner and less confusing to disable both on one flag rather > than support two options. OK, agreed. I've filed: https://bugs.openjdk.java.net/browse/JDK-8250627 Thanks, Severin > Bob. > > > On Jul 24, 2020, at 12:13 PM, Severin Gehwolf > > wrote: > > > > Hi, > > > > For hotspot one can disable container detection with a simple > > switch: > > > > $ java -XX:-UseContainerSupport -Xlog:os+container=trace -version > > [0.000s][trace][os,container] OSContainer::init: Initializing > > Container Support > > [0.000s][trace][os,container] Container Support not enabled > > openjdk version "15-internal" 2020-09-15 > > OpenJDK Runtime Environment (build 15-internal+0- > > adhoc.sgehwolf.openjdk-head-2) > > OpenJDK 64-Bit Server VM (build 15-internal+0- > > adhoc.sgehwolf.openjdk-head-2, mixed mode, sharing) > > > > The same cannot be achieved with the Java code, > > jdk.internal.platform.Metrics.java and friends in the JDK. At the > > time > > Metrics were added the only consumer of them was the Java Launcher > > via > > -XshowSettings:system. This has changed with JDK-8226575: > > OperatingSystemMXBean should be made container aware. > > > > It is known that in certain cases the container detection code will > > detect for a system to be supposedly in a container where it > > actually > > isn't: > > https://bugs.openjdk.java.net/browse/JDK-8227006?focusedCommentId=14275604&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14275604 > > > > For hotspot there is a flag, to turn auto-detection off for exactly > > the > > case when heuristics are wrong: -XX:-UseContainerSupport. However, > > this > > flag has no effect on the Java metrics code. There is a risk that > > on > > some systems the auto-detection will be wrong and might cause > > regressions. > > > > I propose to make the Java metrics code adhere to -XX:+/- > > UseContainerSupport flag. Do you think this would be useful? If > > yes, > > I'll file a bug and propose a patch for it. > > > > Thoughts? Opinions? > > > > Thanks, > > Severin > > From bob.vandette at oracle.com Mon Jul 27 15:52:53 2020 From: bob.vandette at oracle.com (Bob Vandette) Date: Mon, 27 Jul 2020 11:52:53 -0400 Subject: No way to disable Java Container Metrics In-Reply-To: References: Message-ID: <811D4028-B350-4BF7-B963-5318DCE05820@oracle.com> > On Jul 27, 2020, at 11:32 AM, Severin Gehwolf wrote: > > Hi Bob, > > On Fri, 2020-07-24 at 15:21 -0400, Bob Vandette wrote: >> I?ve been monitoring the discussion on your jdk8u alias and I can?t help wondering if >> my original decision to provide two different implementations of this container detection >> logic was the best way to go. >> >> I didn?t want to have an all Java implementation since the VM needs to initialize it?s >> memory and cpu sizing very early on prior to its ability to run Java code. >> >> If we put all of the logic in the VM, then we?d end up with a pretty wide interface to >> the VM and more overhead extracting values (due to JNI) although the Java logic >> typically ends up in native code anyway. Having the functionality in the VM >> makes it easier to add JFR events. Having a pure Java implementation makes it >> easier to support the OS MBeans. The maintenance of supporting both implementations >> is a bit of a pain. > > Add to that that Java metrics return non-null for any controller it > finds. The JVM doesn't. Container support is turned on there only if > all cpu and memory controllers are found. That was intentional. I wanted the VM to configure itself consistently. Either all needed value are present or revert to non container mode. For Metrics, I felt it was ok to report whatever is available. Bob. > >> Assuming no one has the time or desire to migrate the logic to the VM and provide >> Java wrapper logic, I?m ok with what you are proposing. It?s one step on the path. >> The VM support and the Java level support are really for two different consumers >> but I think it would be cleaner and less confusing to disable both on one flag rather >> than support two options. > > OK, agreed. I've filed: > https://bugs.openjdk.java.net/browse/JDK-8250627 > > Thanks, > Severin > >> Bob. >> >>> On Jul 24, 2020, at 12:13 PM, Severin Gehwolf >>> wrote: >>> >>> Hi, >>> >>> For hotspot one can disable container detection with a simple >>> switch: >>> >>> $ java -XX:-UseContainerSupport -Xlog:os+container=trace -version >>> [0.000s][trace][os,container] OSContainer::init: Initializing >>> Container Support >>> [0.000s][trace][os,container] Container Support not enabled >>> openjdk version "15-internal" 2020-09-15 >>> OpenJDK Runtime Environment (build 15-internal+0- >>> adhoc.sgehwolf.openjdk-head-2) >>> OpenJDK 64-Bit Server VM (build 15-internal+0- >>> adhoc.sgehwolf.openjdk-head-2, mixed mode, sharing) >>> >>> The same cannot be achieved with the Java code, >>> jdk.internal.platform.Metrics.java and friends in the JDK. At the >>> time >>> Metrics were added the only consumer of them was the Java Launcher >>> via >>> -XshowSettings:system. This has changed with JDK-8226575: >>> OperatingSystemMXBean should be made container aware. >>> >>> It is known that in certain cases the container detection code will >>> detect for a system to be supposedly in a container where it >>> actually >>> isn't: >>> https://bugs.openjdk.java.net/browse/JDK-8227006?focusedCommentId=14275604&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14275604 >>> >>> For hotspot there is a flag, to turn auto-detection off for exactly >>> the >>> case when heuristics are wrong: -XX:-UseContainerSupport. However, >>> this >>> flag has no effect on the Java metrics code. There is a risk that >>> on >>> some systems the auto-detection will be wrong and might cause >>> regressions. >>> >>> I propose to make the Java metrics code adhere to -XX:+/- >>> UseContainerSupport flag. Do you think this would be useful? If >>> yes, >>> I'll file a bug and propose a patch for it. >>> >>> Thoughts? Opinions? >>> >>> Thanks, >>> Severin >>> > From sgehwolf at redhat.com Mon Jul 27 16:06:23 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Mon, 27 Jul 2020 18:06:23 +0200 Subject: No way to disable Java Container Metrics In-Reply-To: <811D4028-B350-4BF7-B963-5318DCE05820@oracle.com> References: <811D4028-B350-4BF7-B963-5318DCE05820@oracle.com> Message-ID: On Mon, 2020-07-27 at 11:52 -0400, Bob Vandette wrote: > > On Jul 27, 2020, at 11:32 AM, Severin Gehwolf wrote: > > > > Hi Bob, > > > > On Fri, 2020-07-24 at 15:21 -0400, Bob Vandette wrote: > > > I?ve been monitoring the discussion on your jdk8u alias and I can?t help wondering if > > > my original decision to provide two different implementations of this container detection > > > logic was the best way to go. > > > > > > I didn?t want to have an all Java implementation since the VM needs to initialize it?s > > > memory and cpu sizing very early on prior to its ability to run Java code. > > > > > > If we put all of the logic in the VM, then we?d end up with a pretty wide interface to > > > the VM and more overhead extracting values (due to JNI) although the Java logic > > > typically ends up in native code anyway. Having the functionality in the VM > > > makes it easier to add JFR events. Having a pure Java implementation makes it > > > easier to support the OS MBeans. The maintenance of supporting both implementations > > > is a bit of a pain. > > > > Add to that that Java metrics return non-null for any controller it > > finds. The JVM doesn't. Container support is turned on there only if > > all cpu and memory controllers are found. > > That was intentional. I wanted the VM to configure itself consistently. > Either all needed value are present or revert to non container mode. > For Metrics, I felt it was ok to report whatever is available. Yes, I know. Thanks. What I meant to say was that it's another consideration to take into account if at some point a JVM-only-impl + Java wrapper approach would be taken. Thanks, Severin > > > > > Assuming no one has the time or desire to migrate the logic to > > > the VM and provide > > > Java wrapper logic, I?m ok with what you are proposing. It?s one > > > step on the path. > > > The VM support and the Java level support are really for two > > > different consumers > > > but I think it would be cleaner and less confusing to disable > > > both on one flag rather > > > than support two options. > > > > OK, agreed. I've filed: > > https://bugs.openjdk.java.net/browse/JDK-8250627 > > > > Thanks, > > Severin > > > > > Bob. > > > > > > > On Jul 24, 2020, at 12:13 PM, Severin Gehwolf < > > > > sgehwolf at redhat.com> > > > > wrote: > > > > > > > > Hi, > > > > > > > > For hotspot one can disable container detection with a simple > > > > switch: > > > > > > > > $ java -XX:-UseContainerSupport -Xlog:os+container=trace > > > > -version > > > > [0.000s][trace][os,container] OSContainer::init: Initializing > > > > Container Support > > > > [0.000s][trace][os,container] Container Support not enabled > > > > openjdk version "15-internal" 2020-09-15 > > > > OpenJDK Runtime Environment (build 15-internal+0- > > > > adhoc.sgehwolf.openjdk-head-2) > > > > OpenJDK 64-Bit Server VM (build 15-internal+0- > > > > adhoc.sgehwolf.openjdk-head-2, mixed mode, sharing) > > > > > > > > The same cannot be achieved with the Java code, > > > > jdk.internal.platform.Metrics.java and friends in the JDK. At > > > > the > > > > time > > > > Metrics were added the only consumer of them was the Java > > > > Launcher > > > > via > > > > -XshowSettings:system. This has changed with JDK-8226575: > > > > OperatingSystemMXBean should be made container aware. > > > > > > > > It is known that in certain cases the container detection code > > > > will > > > > detect for a system to be supposedly in a container where it > > > > actually > > > > isn't: > > > > https://bugs.openjdk.java.net/browse/JDK-8227006?focusedCommentId=14275604&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14275604 > > > > > > > > For hotspot there is a flag, to turn auto-detection off for > > > > exactly > > > > the > > > > case when heuristics are wrong: -XX:-UseContainerSupport. > > > > However, > > > > this > > > > flag has no effect on the Java metrics code. There is a risk > > > > that > > > > on > > > > some systems the auto-detection will be wrong and might cause > > > > regressions. > > > > > > > > I propose to make the Java metrics code adhere to -XX:+/- > > > > UseContainerSupport flag. Do you think this would be useful? If > > > > yes, > > > > I'll file a bug and propose a patch for it. > > > > > > > > Thoughts? Opinions? > > > > > > > > Thanks, > > > > Severin > > > > From chris.plummer at oracle.com Mon Jul 27 19:09:16 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 27 Jul 2020 12:09:16 -0700 Subject: RFR(M): 8247515: OSX pc_to_symbol() lookup does not work with core files In-Reply-To: <3b354be8-d687-974a-7f5e-84d8bfb39545@oracle.com> References: <092ef5a8-c973-e377-de91-b2002a1b8fe7@oracle.com> <38e1887e-a646-abc0-0b7e-53339af1d684@oracle.com> <3c31622b-831e-d46f-daa6-d641cbf755b1@oracle.com> <18f7a1d8-3802-c7da-c981-eac8cf0953eb@oracle.com> <1e9e4786-79f8-c4e0-efe7-f1a8f369d2c6@oracle.com> <4c09d200-057b-e878-733a-524860453288@oracle.com> <401dbc7a-dd2f-5612-74d3-3dbf6641dae9@oracle.com> <8f1cec73-4397-0489-6031-a813d9bd23c6@oracle.com> <526ac128-d29d-70f1-413b-b541c101835e@oracle.com> <3b354be8-d687-974a-7f5e-84d8bfb39545@oracle.com> Message-ID: <59aa1a6d-ad73-ad15-4833-549b23fb9d59@oracle.com> An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Mon Jul 27 19:11:06 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 27 Jul 2020 12:11:06 -0700 Subject: RFR(S): 8248882: SA PMap and PStack support on OSX works with core files. Enable them. In-Reply-To: <511f7770-2ebf-f3ec-fcf9-221f9e65ed87@oracle.com> References: <511f7770-2ebf-f3ec-fcf9-221f9e65ed87@oracle.com> Message-ID: Ping! This one is pretty easy. I just changed the two tools to no longer refuse to run on OSX if we are debugging a core file, and updated the tests to also do a test run on a core file. thanks, Chris On 7/21/20 1:13 PM, Chris Plummer wrote: > Hello, > > Please help review the following: > > https://bugs.openjdk.java.net/browse/JDK-8248882 > http://cr.openjdk.java.net/~cjplummer/8248882/webrev.00/index.html > > The changes enable PMap and PStack on OSX when debugging a core file. > Unfortunately there appears to be no exposed way to determine if you > are debugging a core file, so I had to rely on clues (certain lists > being empty). > > As part of these changes I'm adding core file testing to the two > relevant tests. Also note that for ClhsdbPMap.java I had to tinker > with the list of expected libraries in the output. For core files on > Windows, the nio and net libraries were missing, presumably because > nothing had triggered their loading yet, so I removed them from the > expected output list, and add jli, which seems to always be there. > > Also a minor change in CLHSB.java. It was sending a message to > System.err. It makes the test output easier to read if it goes to > System.out. > > thanks, > > Chris From joe.darcy at oracle.com Mon Jul 27 20:42:58 2020 From: joe.darcy at oracle.com (Joe Darcy) Date: Mon, 27 Jul 2020 13:42:58 -0700 Subject: JDK 16 RFR of JDK-8250640: Address reliance on default constructors in jdk.jdi Message-ID: <901e55aa-7def-8341-2140-67993ce74713@oracle.com> Hello, Another module, another set of default constructors to replace with explicit ones. Please review the code changes and CSR to address: ??? JDK-8250640: Address reliance on default constructors in jdk.jdi ??? webrev: http://cr.openjdk.java.net/~darcy/8250640.0/ ??? CSR: https://bugs.openjdk.java.net/browse/JDK-8250642 Patch below; I'll update copyrights before pushing. In the Bootstrap class, since it doesn't define any non-static methods, it looks like it doesn't need a constructor in its API at all so I terminally deprecated the constructor. Thanks, -Joe --- old/src/jdk.jdi/share/classes/com/sun/jdi/Bootstrap.java 2020-07-27 13:24:06.454000000 -0700 +++ new/src/jdk.jdi/share/classes/com/sun/jdi/Bootstrap.java 2020-07-27 13:24:05.842000000 -0700 @@ -38,6 +38,11 @@ ? */ ?public class Bootstrap extends Object { +??? /** +???? * Do not call. +???? */ +??? @Deprecated(since="16", forRemoval=true) +??? public Bootstrap() {} ???? /** ????? * Returns the virtual machine manager. --- old/src/jdk.jdi/share/classes/com/sun/jdi/connect/spi/Connection.java 2020-07-27 13:24:07.506000000 -0700 +++ new/src/jdk.jdi/share/classes/com/sun/jdi/connect/spi/Connection.java 2020-07-27 13:24:06.878000000 -0700 @@ -55,6 +55,10 @@ ? * @since 1.5 ? */ ?public abstract class Connection { +??? /** +???? * Constructor for subclasses to call. +???? */ +??? public Connection() {} ???? /** ????? * Reads a packet from the target VM. --- old/src/jdk.jdi/share/classes/com/sun/jdi/connect/spi/TransportService.java 2020-07-27 13:24:08.570000000 -0700 +++ new/src/jdk.jdi/share/classes/com/sun/jdi/connect/spi/TransportService.java 2020-07-27 13:24:07.934000000 -0700 @@ -79,6 +79,10 @@ ? * @since 1.5 ? */ ?public abstract class TransportService { +??? /** +???? * Constructor for subclasses to call. +???? */ +??? public TransportService() {} ???? /** ????? * Returns a name to identify the transport service. @@ -98,6 +102,10 @@ ????? * The transport service capabilities. ????? */ ???? public static abstract class Capabilities { +??? /** +??? ?* Constructor for subclasses to call. +??? ?*/ +??? public Capabilities() {} ???????? /** ????????? * Tells whether or not this transport service can support @@ -229,6 +237,10 @@ ????? * service from listening on an address. ????? */ ???? public static abstract class ListenKey { +??? /** +??? ?* Constructor for subclasses to call. +??? ?*/ +??? public ListenKey() {} ???????? /** ????????? * Returns a string representation of the listen key. From alexey.menkov at oracle.com Mon Jul 27 21:45:42 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Mon, 27 Jul 2020 14:45:42 -0700 Subject: RFR(S): 8248882: SA PMap and PStack support on OSX works with core files. Enable them. In-Reply-To: References: <511f7770-2ebf-f3ec-fcf9-221f9e65ed87@oracle.com> Message-ID: <3d711af3-6e10-2663-9629-6ebd10fd1e3c@oracle.com> Hi Chris, Looks good to me. --alex On 07/27/2020 12:11, Chris Plummer wrote: > Ping! This one is pretty easy. I just changed the two tools to no longer > refuse to run on OSX if we are debugging a core file, and updated the > tests to also do a test run on a core file. > > thanks, > > Chris > > On 7/21/20 1:13 PM, Chris Plummer wrote: >> Hello, >> >> Please help review the following: >> >> https://bugs.openjdk.java.net/browse/JDK-8248882 >> http://cr.openjdk.java.net/~cjplummer/8248882/webrev.00/index.html >> >> The changes enable PMap and PStack on OSX when debugging a core file. >> Unfortunately there appears to be no exposed way to determine if you >> are debugging a core file, so I had to rely on clues (certain lists >> being empty). >> >> As part of these changes I'm adding core file testing to the two >> relevant tests. Also note that for ClhsdbPMap.java I had to tinker >> with the list of expected libraries in the output. For core files on >> Windows, the nio and net libraries were missing, presumably because >> nothing had triggered their loading yet, so I removed them from the >> expected output list, and add jli, which seems to always be there. >> >> Also a minor change in CLHSB.java. It was sending a message to >> System.err. It makes the test output easier to read if it goes to >> System.out. >> >> thanks, >> >> Chris > From serguei.spitsyn at oracle.com Mon Jul 27 22:23:09 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 27 Jul 2020 15:23:09 -0700 Subject: RFR(S): 8248882: SA PMap and PStack support on OSX works with core files. Enable them. In-Reply-To: <3d711af3-6e10-2663-9629-6ebd10fd1e3c@oracle.com> References: <511f7770-2ebf-f3ec-fcf9-221f9e65ed87@oracle.com> <3d711af3-6e10-2663-9629-6ebd10fd1e3c@oracle.com> Message-ID: <015b5627-d1ac-3aab-e494-0f306fe54453@oracle.com> Hi Chris, LGTM++ Thanks, Serguei On 7/27/20 14:45, Alex Menkov wrote: > Hi Chris, > > Looks good to me. > > --alex > > On 07/27/2020 12:11, Chris Plummer wrote: >> Ping! This one is pretty easy. I just changed the two tools to no >> longer refuse to run on OSX if we are debugging a core file, and >> updated the tests to also do a test run on a core file. >> >> thanks, >> >> Chris >> >> On 7/21/20 1:13 PM, Chris Plummer wrote: >>> Hello, >>> >>> Please help review the following: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8248882 >>> http://cr.openjdk.java.net/~cjplummer/8248882/webrev.00/index.html >>> >>> The changes enable PMap and PStack on OSX when debugging a core >>> file. Unfortunately there appears to be no exposed way to determine >>> if you are debugging a core file, so I had to rely on clues (certain >>> lists being empty). >>> >>> As part of these changes I'm adding core file testing to the two >>> relevant tests. Also note that for ClhsdbPMap.java I had to tinker >>> with the list of expected libraries in the output. For core files on >>> Windows, the nio and net libraries were missing, presumably because >>> nothing had triggered their loading yet, so I removed them from the >>> expected output list, and add jli, which seems to always be there. >>> >>> Also a minor change in CLHSB.java. It was sending a message to >>> System.err. It makes the test output easier to read if it goes to >>> System.out. >>> >>> thanks, >>> >>> Chris >> From chris.plummer at oracle.com Mon Jul 27 22:42:38 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 27 Jul 2020 15:42:38 -0700 Subject: RFR(S): 8248882: SA PMap and PStack support on OSX works with core files. Enable them. In-Reply-To: <015b5627-d1ac-3aab-e494-0f306fe54453@oracle.com> References: <511f7770-2ebf-f3ec-fcf9-221f9e65ed87@oracle.com> <3d711af3-6e10-2663-9629-6ebd10fd1e3c@oracle.com> <015b5627-d1ac-3aab-e494-0f306fe54453@oracle.com> Message-ID: <2b34438f-2e35-3d7c-1438-ff8060cf59fe@oracle.com> Thanks Alex and Serguei! I still need another reviewer for JDK-8247515 before I can push this change since it depends on the fixes done there. Chris On 7/27/20 3:23 PM, serguei.spitsyn at oracle.com wrote: > Hi Chris, > > LGTM++ > > Thanks, > Serguei > > > On 7/27/20 14:45, Alex Menkov wrote: >> Hi Chris, >> >> Looks good to me. >> >> --alex >> >> On 07/27/2020 12:11, Chris Plummer wrote: >>> Ping! This one is pretty easy. I just changed the two tools to no >>> longer refuse to run on OSX if we are debugging a core file, and >>> updated the tests to also do a test run on a core file. >>> >>> thanks, >>> >>> Chris >>> >>> On 7/21/20 1:13 PM, Chris Plummer wrote: >>>> Hello, >>>> >>>> Please help review the following: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8248882 >>>> http://cr.openjdk.java.net/~cjplummer/8248882/webrev.00/index.html >>>> >>>> The changes enable PMap and PStack on OSX when debugging a core >>>> file. Unfortunately there appears to be no exposed way to determine >>>> if you are debugging a core file, so I had to rely on clues >>>> (certain lists being empty). >>>> >>>> As part of these changes I'm adding core file testing to the two >>>> relevant tests. Also note that for ClhsdbPMap.java I had to tinker >>>> with the list of expected libraries in the output. For core files >>>> on Windows, the nio and net libraries were missing, presumably >>>> because nothing had triggered their loading yet, so I removed them >>>> from the expected output list, and add jli, which seems to always >>>> be there. >>>> >>>> Also a minor change in CLHSB.java. It was sending a message to >>>> System.err. It makes the test output easier to read if it goes to >>>> System.out. >>>> >>>> thanks, >>>> >>>> Chris >>> > From serguei.spitsyn at oracle.com Mon Jul 27 23:40:51 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 27 Jul 2020 16:40:51 -0700 Subject: RFR(M): 8247515: OSX pc_to_symbol() lookup does not work with core files In-Reply-To: <59aa1a6d-ad73-ad15-4833-549b23fb9d59@oracle.com> References: <092ef5a8-c973-e377-de91-b2002a1b8fe7@oracle.com> <38e1887e-a646-abc0-0b7e-53339af1d684@oracle.com> <3c31622b-831e-d46f-daa6-d641cbf755b1@oracle.com> <18f7a1d8-3802-c7da-c981-eac8cf0953eb@oracle.com> <1e9e4786-79f8-c4e0-efe7-f1a8f369d2c6@oracle.com> <4c09d200-057b-e878-733a-524860453288@oracle.com> <401dbc7a-dd2f-5612-74d3-3dbf6641dae9@oracle.com> <8f1cec73-4397-0489-6031-a813d9bd23c6@oracle.com> <526ac128-d29d-70f1-413b-b541c101835e@oracle.com> <3b354be8-d687-974a-7f5e-84d8bfb39545@oracle.com> <59aa1a6d-ad73-ad15-4833-549b23fb9d59@oracle.com> Message-ID: <2f1af266-5cfa-b728-13a3-4531a050b6ab@oracle.com> An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Mon Jul 27 23:46:12 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 27 Jul 2020 16:46:12 -0700 Subject: JDK 16 RFR of JDK-8250583: Address reliance on default constructors in java.management In-Reply-To: <3b91a833-b041-2218-3114-0c0268f06d4c@oracle.com> References: <5cb51cfc-da7a-32aa-eb22-4383d18cce1e@oracle.com> <3b91a833-b041-2218-3114-0c0268f06d4c@oracle.com> Message-ID: Hi Joe, The update looks okay to me too. Thanks, Serguei On 7/25/20 11:18, Alan Bateman wrote: > On 25/07/2020 18:52, Joe Darcy wrote: >> Hello, >> >> I'm not positive serviceability-dev is the best alias for this >> review; if not, please direct me elsewhere. >> >> In any case, please review the changes and CSR for: >> >> ??? JDK-8250583: Address reliance on default constructors in >> java.management >> ??? webrev: http://cr.openjdk.java.net/~darcy/8250583.0/ >> ??? CSR: https://bugs.openjdk.java.net/browse/JDK-8250584 >> >> (This is part of a larger library clean-up effort to remove reliance >> on default constructors in the JDK ahead of a new javac lint warning, >> JDK-8071961.) >> >> Generally the changes adds a public no-arg constructor equivalent to >> the implicit default constructor. In the case of RoleStatus, since it >> doesn't seem that the class should have a public constructor at all, >> I terminally deprecated the constructor. > serviceability-dev is fine as it has mostly subsumed the jmx-dev list. > > The changes look okay to me. > > -Alan From chris.plummer at oracle.com Mon Jul 27 23:53:32 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 27 Jul 2020 16:53:32 -0700 Subject: RFR(M): 8247515: OSX pc_to_symbol() lookup does not work with core files In-Reply-To: <2f1af266-5cfa-b728-13a3-4531a050b6ab@oracle.com> References: <092ef5a8-c973-e377-de91-b2002a1b8fe7@oracle.com> <38e1887e-a646-abc0-0b7e-53339af1d684@oracle.com> <3c31622b-831e-d46f-daa6-d641cbf755b1@oracle.com> <18f7a1d8-3802-c7da-c981-eac8cf0953eb@oracle.com> <1e9e4786-79f8-c4e0-efe7-f1a8f369d2c6@oracle.com> <4c09d200-057b-e878-733a-524860453288@oracle.com> <401dbc7a-dd2f-5612-74d3-3dbf6641dae9@oracle.com> <8f1cec73-4397-0489-6031-a813d9bd23c6@oracle.com> <526ac128-d29d-70f1-413b-b541c101835e@oracle.com> <3b354be8-d687-974a-7f5e-84d8bfb39545@oracle.com> <59aa1a6d-ad73-ad15-4833-549b23fb9d59@oracle.com> <2f1af266-5cfa-b728-13a3-4531a050b6ab@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Mon Jul 27 23:56:03 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 27 Jul 2020 16:56:03 -0700 Subject: RFR(M): 8247515: OSX pc_to_symbol() lookup does not work with core files In-Reply-To: References: <092ef5a8-c973-e377-de91-b2002a1b8fe7@oracle.com> <38e1887e-a646-abc0-0b7e-53339af1d684@oracle.com> <3c31622b-831e-d46f-daa6-d641cbf755b1@oracle.com> <18f7a1d8-3802-c7da-c981-eac8cf0953eb@oracle.com> <1e9e4786-79f8-c4e0-efe7-f1a8f369d2c6@oracle.com> <4c09d200-057b-e878-733a-524860453288@oracle.com> <401dbc7a-dd2f-5612-74d3-3dbf6641dae9@oracle.com> <8f1cec73-4397-0489-6031-a813d9bd23c6@oracle.com> <526ac128-d29d-70f1-413b-b541c101835e@oracle.com> <3b354be8-d687-974a-7f5e-84d8bfb39545@oracle.com> <59aa1a6d-ad73-ad15-4833-549b23fb9d59@oracle.com> <2f1af266-5cfa-b728-13a3-4531a050b6ab@oracle.com> Message-ID: <1858819e-cdb5-3dec-c3e9-a3df47cc25b9@oracle.com> An HTML attachment was scrubbed... URL: From leonid.mesnik at oracle.com Tue Jul 28 01:08:03 2020 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Mon, 27 Jul 2020 18:08:03 -0700 Subject: RFR: 8244537: JDI tests fail due to "ERROR: Exception : nsk.share.jdi.JDITestRuntimeException: JDITestRuntimeException : ** event IS NOT a breakpoint **" Message-ID: <16EB9126-382A-4092-BB61-14E9C3CF208C@oracle.com> Hi Could you please review following fix which suspends debugger VM while enabling/disabling events. All changed tests fail intermittently getting unexpected events instead of breakpoint used for communication between debugger/debuggee VM. The tests request different events and verify request's properties but don't process/verify events themselves. Test doesn't aware if events are generated or not. The vm suspension doesn't affect JDWP native agent and it still should get and verify JDWP commands. webrev: http://cr.openjdk.java.net/~lmesnik/8244537/webrev.00/ bug: https://bugs.openjdk.java.net/browse/JDK-8244537 Leonid -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Tue Jul 28 04:32:44 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 27 Jul 2020 21:32:44 -0700 Subject: RFR(S): 8247516: DSO.closestSymbolToPC() should use dbg.lookup() rather than rely on java ELF file support Message-ID: Hello, Please review the following: https://bugs.openjdk.java.net/browse/JDK-8247516 http://cr.openjdk.java.net/~cjplummer/8247516/webrev.00/index.html I put all the details in the description of the CR, including some background on how symbol lookups are done, including what LoadObjects are and their class hierarchy, and also info on JVMDebugger subclasses. One thing not covered in the bug description is the additional gutting of DSO.java that comes with these changes. Many APIs were not used so I removed them, such as setBase(), lookupSymbol(), and isDSO(). Doing so allowed completely severing any need for java ELF file support. Note I plan on removing the java ELF file support itself with another CR after pushing these changes. thanks, Chris From chris.plummer at oracle.com Tue Jul 28 05:04:17 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 27 Jul 2020 22:04:17 -0700 Subject: RFR(S): 8247516: DSO.closestSymbolToPC() should use dbg.lookup() rather than rely on java ELF file support In-Reply-To: References: Message-ID: <2873b4bd-09c6-7f29-04fc-3910f360def8@oracle.com> I should have mentioned that currently there is no testing of this code. There will with the changes for [1] JDK-8247514, which will add the lost clhsdb "whatis" functionality, which was lost when JavaScript support went away. "whatis" used DSO.closestSymbolToPC(), so as part of JDK-8247514 I'm adding this support to the PointerFinder class so the "findpc" will also be able to do address to native symbol lookups, and the ClhsdbFindPC will check that it is working. [1] https://bugs.openjdk.java.net/browse/JDK-8247514 thanks, Chris On 7/27/20 9:32 PM, Chris Plummer wrote: > Hello, > > Please review the following: > > https://bugs.openjdk.java.net/browse/JDK-8247516 > http://cr.openjdk.java.net/~cjplummer/8247516/webrev.00/index.html > > I put all the details in the description of the CR, including some > background on how symbol lookups are done, including what LoadObjects > are and their class hierarchy, and also info on JVMDebugger subclasses. > > One thing not covered in the bug description is the additional gutting > of DSO.java that comes with these changes. Many APIs were not used so > I removed them, such as setBase(), lookupSymbol(), and isDSO(). Doing > so allowed completely severing any need for java ELF file support. > Note I plan on removing the java ELF file support itself with another > CR after pushing these changes. > > thanks, > > Chris From serguei.spitsyn at oracle.com Tue Jul 28 05:28:32 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 27 Jul 2020 22:28:32 -0700 Subject: RFR: 8244537: JDI tests fail due to "ERROR: Exception : nsk.share.jdi.JDITestRuntimeException: JDITestRuntimeException : ** event IS NOT a breakpoint **" In-Reply-To: <16EB9126-382A-4092-BB61-14E9C3CF208C@oracle.com> References: <16EB9126-382A-4092-BB61-14E9C3CF208C@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Tue Jul 28 06:45:05 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 28 Jul 2020 07:45:05 +0100 Subject: JDK 16 RFR of JDK-8250640: Address reliance on default constructors in jdk.jdi In-Reply-To: <901e55aa-7def-8341-2140-67993ce74713@oracle.com> References: <901e55aa-7def-8341-2140-67993ce74713@oracle.com> Message-ID: <3b83628a-56d8-d755-1cd7-d920afd0e11b@oracle.com> On 27/07/2020 21:42, Joe Darcy wrote: > Hello, > > Another module, another set of default constructors to replace with > explicit ones. Please review the code changes and CSR to address: > > ??? JDK-8250640: Address reliance on default constructors in jdk.jdi > ??? webrev: http://cr.openjdk.java.net/~darcy/8250640.0/ > ??? CSR: https://bugs.openjdk.java.net/browse/JDK-8250642 > > Patch below; I'll update copyrights before pushing. In the Bootstrap > class, since it doesn't define any non-static methods, it looks like > it doesn't need a constructor in its API at all so I terminally > deprecated the constructor. Right, Bootstrap should have a public constructor. Ideally the abstract classes in the spi package would have a protected constructor. The change looks okay to me. -Alan From christian.hagedorn at oracle.com Tue Jul 28 07:31:41 2020 From: christian.hagedorn at oracle.com (Christian Hagedorn) Date: Tue, 28 Jul 2020 09:31:41 +0200 Subject: RFR (T) 8250042: Clean up methodOop and method_oop names from the code In-Reply-To: References: <85efc3ab-abbf-c5f2-9b7b-47fa516d9a2d@oracle.com> <6f973a0a-cf55-e1ab-8de3-b57f68dbd2cf@oracle.com> Message-ID: <8c737ff0-7a62-d18d-78b2-b415802ebbdb@oracle.com> Hi Coleen On 24.07.20 15:10, coleen.phillimore at oracle.com wrote: > incremental webrev at > http://cr.openjdk.java.net/~coleenp/2020/8250042.02.incr/webrev > full webrev at http://cr.openjdk.java.net/~coleenp/2020/8250042.02/webrev Thanks for cleaning this up! The compiler changes look good to me. Just a minor comment (no new webrev required): - arm.ad:8873 & x86_32.ad:13321: There is an extra whitespace before ")" Best regards, Christian > Thanks, > Coleen > > > On 7/24/20 8:23 AM, coleen.phillimore at oracle.com wrote: >> >> Thanks for looking at this. >> >> On 7/24/20 1:01 AM, David Holmes wrote: >>> Hi Coleen, >>> >>> On 24/07/2020 2:58 am, coleen.phillimore at oracle.com wrote: >>>> See bug for more details.? I've been running into these names a lot >>>> lately.?? Many of these names are in JVMTI. >>>> >>>> Tested with tier1 on all Oracle platforms and built on non-Oracle >>>> platforms. >>>> >>>> open webrev at >>>> http://cr.openjdk.java.net/~coleenp/2020/8250042.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8250042 >>> >>> src/hotspot/cpu/*/*.ad >>> >>> These still refer to "method oop" and method_oop in a number of places. >> >> Yes, I only replaced method_oop in the shared code and not in the AD >> code.? method_oop can be the name of a parameter and using "sed" to >> change it to "method" doesn't work.?? Somebody who understands this >> code and looks at it will have to make the rest of the changes. >> >> What I did was replace "method oop" with "method" and "methodOop" with >> "method" in all the sources.? I replaced "method_oop" with "method" or >> "checked_method" in the shared sources. >> >>> >>> src/hotspot/share/adlc/adlparse.cpp >>> >>> +? frame->_interpreter_method_oop_reg = parse_one_arg("method reg >>> entry"); >>> >>> I guess I'm not understanding the scope of this renaming - why is >>> _interpreter_method_oop_reg not renamed as well? Should this (and >>> other uses) be parsed as method-(oop-reg) rather than (method-oop)-reg? >> >> I don't know this code, so I'd rather not change any more of it. The >> comment makes sense changed, even though the variable name still >> refers to method_oop. >> >> Thanks, >> Coleen >>> >>> Otherwise all okay. >>> >>> Thanks, >>> David >>> >>>> Thanks, >>>> Coleen >> > From sgehwolf at redhat.com Tue Jul 28 08:27:05 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Tue, 28 Jul 2020 10:27:05 +0200 Subject: RFR(s): 8250627: Use -XX:+/-UseContainerSupport for enabling/disabling Java container metrics Message-ID: Hi, Please review this patch which makes the Java container metrics adhere to -XX:+/-UseContainerSupport so they can be disabled if heuristics turn out to be wrong. The approach taken is to use JNI and call into the JVM in order to determine the setting of UseContainerSupport before Metrics are being instantiated. The intention is for this patch to be backported to older JDKs so as to provide a means to disable container metrics should things go wrong with backports of the likes of JDK-8226575. Bug: https://bugs.openjdk.java.net/browse/JDK-8250627 webrev: https://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8250627/01/webrev/ Testing: New container test. Existing container tests. jdk/submit. tier1 on Linux x86_64. Thoughts? Thanks, Severin From aph at redhat.com Tue Jul 28 08:49:42 2020 From: aph at redhat.com (Andrew Haley) Date: Tue, 28 Jul 2020 09:49:42 +0100 Subject: RFR(s): 8250627: Use -XX:+/-UseContainerSupport for enabling/disabling Java container metrics In-Reply-To: References: Message-ID: On 7/28/20 9:27 AM, Severin Gehwolf wrote: > The intention is for this patch to be backported to older JDKs so as to > provide a means to disable container metrics should things go wrong > with backports of the likes of JDK-8226575. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8250627 > webrev: https://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8250627/01/webrev/ > > Testing: New container test. Existing container tests. jdk/submit. > tier1 on Linux x86_64. That looks right, and pleasingly simple. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From sgehwolf at redhat.com Tue Jul 28 09:01:59 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Tue, 28 Jul 2020 11:01:59 +0200 Subject: RFR(s): 8250627: Use -XX:+/-UseContainerSupport for enabling/disabling Java container metrics In-Reply-To: References: Message-ID: <42c27014e5276476cb24868b7db2814e96e3e2ea.camel@redhat.com> On Tue, 2020-07-28 at 09:49 +0100, Andrew Haley wrote: > On 7/28/20 9:27 AM, Severin Gehwolf wrote: > > The intention is for this patch to be backported to older JDKs so as to > > provide a means to disable container metrics should things go wrong > > with backports of the likes of JDK-8226575. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8250627 > > webrev: https://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8250627/01/webrev/ > > > > Testing: New container test. Existing container tests. jdk/submit. > > tier1 on Linux x86_64. > > That looks right, and pleasingly simple. Thanks for the review! Cheers, Severin From david.holmes at oracle.com Tue Jul 28 11:17:43 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 28 Jul 2020 21:17:43 +1000 Subject: RFR(s): 8250627: Use -XX:+/-UseContainerSupport for enabling/disabling Java container metrics In-Reply-To: References: Message-ID: <94cf1c36-0f55-7027-f2d2-f8b0dadd9366@oracle.com> Hi Severin, On 28/07/2020 6:27 pm, Severin Gehwolf wrote: > Hi, > > Please review this patch which makes the Java container metrics adhere > to -XX:+/-UseContainerSupport so they can be disabled if heuristics > turn out to be wrong. The approach taken is to use JNI and call into > the JVM in order to determine the setting of UseContainerSupport before > Metrics are being instantiated. > > The intention is for this patch to be backported to older JDKs so as to > provide a means to disable container metrics should things go wrong > with backports of the likes of JDK-8226575. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8250627 > webrev: https://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8250627/01/webrev/ Seems quite simple and clean. One query though, I'm not clear on who the expected caller of Metrics.getInstance() is? (Coming from the perspective of "might we want to cache the fact container support is not enabled?".) Also note that we no longer update JVM_INTERFACE_VERSION (last update was JDK 13) - it is meaningless now the JDK and hotspot are in sync version wise. Thanks, David > Testing: New container test. Existing container tests. jdk/submit. > tier1 on Linux x86_64. > > Thoughts? > > Thanks, > Severin > From coleen.phillimore at oracle.com Tue Jul 28 11:20:50 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 28 Jul 2020 07:20:50 -0400 Subject: RFR (T) 8250042: Clean up methodOop and method_oop names from the code In-Reply-To: <8c737ff0-7a62-d18d-78b2-b415802ebbdb@oracle.com> References: <85efc3ab-abbf-c5f2-9b7b-47fa516d9a2d@oracle.com> <6f973a0a-cf55-e1ab-8de3-b57f68dbd2cf@oracle.com> <8c737ff0-7a62-d18d-78b2-b415802ebbdb@oracle.com> Message-ID: <460241cc-7b49-b793-fa50-c12898d4b332@oracle.com> Hi, Thank you for reviewing the compiler changes. On 7/28/20 3:31 AM, Christian Hagedorn wrote: > Hi Coleen > > On 24.07.20 15:10, coleen.phillimore at oracle.com wrote: >> incremental webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8250042.02.incr/webrev >> full webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8250042.02/webrev > > Thanks for cleaning this up! The compiler changes look good to me. > > Just a minor comment (no new webrev required): > - arm.ad:8873 & x86_32.ad:13321: There is an extra whitespace before ")" Fixed! Thanks, Coleen > > Best regards, > Christian > >> Thanks, >> Coleen >> >> >> On 7/24/20 8:23 AM, coleen.phillimore at oracle.com wrote: >>> >>> Thanks for looking at this. >>> >>> On 7/24/20 1:01 AM, David Holmes wrote: >>>> Hi Coleen, >>>> >>>> On 24/07/2020 2:58 am, coleen.phillimore at oracle.com wrote: >>>>> See bug for more details.? I've been running into these names a >>>>> lot lately.?? Many of these names are in JVMTI. >>>>> >>>>> Tested with tier1 on all Oracle platforms and built on non-Oracle >>>>> platforms. >>>>> >>>>> open webrev at >>>>> http://cr.openjdk.java.net/~coleenp/2020/8250042.01/webrev >>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8250042 >>>> >>>> src/hotspot/cpu/*/*.ad >>>> >>>> These still refer to "method oop" and method_oop in a number of >>>> places. >>> >>> Yes, I only replaced method_oop in the shared code and not in the AD >>> code.? method_oop can be the name of a parameter and using "sed" to >>> change it to "method" doesn't work.?? Somebody who understands this >>> code and looks at it will have to make the rest of the changes. >>> >>> What I did was replace "method oop" with "method" and "methodOop" >>> with "method" in all the sources.? I replaced "method_oop" with >>> "method" or "checked_method" in the shared sources. >>> >>>> >>>> src/hotspot/share/adlc/adlparse.cpp >>>> >>>> +? frame->_interpreter_method_oop_reg = parse_one_arg("method reg >>>> entry"); >>>> >>>> I guess I'm not understanding the scope of this renaming - why is >>>> _interpreter_method_oop_reg not renamed as well? Should this (and >>>> other uses) be parsed as method-(oop-reg) rather than >>>> (method-oop)-reg? >>> >>> I don't know this code, so I'd rather not change any more of it. The >>> comment makes sense changed, even though the variable name still >>> refers to method_oop. >>> >>> Thanks, >>> Coleen >>>> >>>> Otherwise all okay. >>>> >>>> Thanks, >>>> David >>>> >>>>> Thanks, >>>>> Coleen >>> >> From sgehwolf at redhat.com Tue Jul 28 13:23:48 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Tue, 28 Jul 2020 15:23:48 +0200 Subject: RFR(s): 8250627: Use -XX:+/-UseContainerSupport for enabling/disabling Java container metrics In-Reply-To: <94cf1c36-0f55-7027-f2d2-f8b0dadd9366@oracle.com> References: <94cf1c36-0f55-7027-f2d2-f8b0dadd9366@oracle.com> Message-ID: <5c91581298ac70a98587a01edba46314cf53b0c2.camel@redhat.com> Hi David, On Tue, 2020-07-28 at 21:17 +1000, David Holmes wrote: > Hi Severin, > > On 28/07/2020 6:27 pm, Severin Gehwolf wrote: > > Hi, > > > > Please review this patch which makes the Java container metrics adhere > > to -XX:+/-UseContainerSupport so they can be disabled if heuristics > > turn out to be wrong. The approach taken is to use JNI and call into > > the JVM in order to determine the setting of UseContainerSupport before > > Metrics are being instantiated. > > > > The intention is for this patch to be backported to older JDKs so as to > > provide a means to disable container metrics should things go wrong > > with backports of the likes of JDK-8226575. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8250627 > > webrev: https://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8250627/01/webrev/ > > Seems quite simple and clean. > > One query though, I'm not clear on who the expected caller of > Metrics.getInstance() is? (Coming from the perspective of "might we want > to cache the fact container support is not enabled?".) I know of two uses so far: 1) Launcher (-XshowSettings:system): http://hg.openjdk.java.net/jdk/jdk/file/89fe9e02a522/src/java.base/share/classes/sun/launcher/LauncherHelper.java#l318 2) OperatingSystemMXBean: http://hg.openjdk.java.net/jdk/jdk/file/89fe9e02a522/src/jdk.management/unix/classes/com/sun/management/internal/OperatingSystemImpl.java#l48 Both uses seem OK as is. Is it worth caching something here? > Also note that we no longer update JVM_INTERFACE_VERSION (last update > was JDK 13) - it is meaningless now the JDK and hotspot are in sync > version wise. OK. Does that mean I should revert the version increment, then? Thanks, Severin > Thanks, > David > > > Testing: New container test. Existing container tests. jdk/submit. > > tier1 on Linux x86_64. > > > > Thoughts? > > > > Thanks, > > Severin > > From david.holmes at oracle.com Tue Jul 28 13:49:00 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 28 Jul 2020 23:49:00 +1000 Subject: RFR(s): 8250627: Use -XX:+/-UseContainerSupport for enabling/disabling Java container metrics In-Reply-To: <5c91581298ac70a98587a01edba46314cf53b0c2.camel@redhat.com> References: <94cf1c36-0f55-7027-f2d2-f8b0dadd9366@oracle.com> <5c91581298ac70a98587a01edba46314cf53b0c2.camel@redhat.com> Message-ID: <08475a75-aa37-c649-6585-404d42e10d6d@oracle.com> Hi Severin, On 28/07/2020 11:23 pm, Severin Gehwolf wrote: > Hi David, > > On Tue, 2020-07-28 at 21:17 +1000, David Holmes wrote: >> Hi Severin, >> >> On 28/07/2020 6:27 pm, Severin Gehwolf wrote: >>> Hi, >>> >>> Please review this patch which makes the Java container metrics adhere >>> to -XX:+/-UseContainerSupport so they can be disabled if heuristics >>> turn out to be wrong. The approach taken is to use JNI and call into >>> the JVM in order to determine the setting of UseContainerSupport before >>> Metrics are being instantiated. >>> >>> The intention is for this patch to be backported to older JDKs so as to >>> provide a means to disable container metrics should things go wrong >>> with backports of the likes of JDK-8226575. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8250627 >>> webrev: https://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8250627/01/webrev/ >> >> Seems quite simple and clean. >> >> One query though, I'm not clear on who the expected caller of >> Metrics.getInstance() is? (Coming from the perspective of "might we want >> to cache the fact container support is not enabled?".) > > I know of two uses so far: > > 1) Launcher (-XshowSettings:system): > http://hg.openjdk.java.net/jdk/jdk/file/89fe9e02a522/src/java.base/share/classes/sun/launcher/LauncherHelper.java#l318 > > 2) OperatingSystemMXBean: > http://hg.openjdk.java.net/jdk/jdk/file/89fe9e02a522/src/jdk.management/unix/classes/com/sun/management/internal/OperatingSystemImpl.java#l48 > > Both uses seem OK as is. Is it worth caching something here? No that looks fine. I didn't find them because of the reflective invocation in Metrics.systemMetrics(). >> Also note that we no longer update JVM_INTERFACE_VERSION (last update >> was JDK 13) - it is meaningless now the JDK and hotspot are in sync >> version wise. > > OK. Does that mean I should revert the version increment, then? I would leave it unchanged, yes. Thanks, David ----- > Thanks, > Severin > >> Thanks, >> David >> >>> Testing: New container test. Existing container tests. jdk/submit. >>> tier1 on Linux x86_64. >>> >>> Thoughts? >>> >>> Thanks, >>> Severin >>> > From sgehwolf at redhat.com Tue Jul 28 14:01:31 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Tue, 28 Jul 2020 16:01:31 +0200 Subject: RFR(s): 8250627: Use -XX:+/-UseContainerSupport for enabling/disabling Java container metrics In-Reply-To: <08475a75-aa37-c649-6585-404d42e10d6d@oracle.com> References: <94cf1c36-0f55-7027-f2d2-f8b0dadd9366@oracle.com> <5c91581298ac70a98587a01edba46314cf53b0c2.camel@redhat.com> <08475a75-aa37-c649-6585-404d42e10d6d@oracle.com> Message-ID: Hi David, Thanks for the review. On Tue, 2020-07-28 at 23:49 +1000, David Holmes wrote: > Hi Severin, > > On 28/07/2020 11:23 pm, Severin Gehwolf wrote: > > Hi David, > > > > On Tue, 2020-07-28 at 21:17 +1000, David Holmes wrote: > > > Hi Severin, > > > > > > On 28/07/2020 6:27 pm, Severin Gehwolf wrote: > > > > Hi, > > > > > > > > Please review this patch which makes the Java container metrics adhere > > > > to -XX:+/-UseContainerSupport so they can be disabled if heuristics > > > > turn out to be wrong. The approach taken is to use JNI and call into > > > > the JVM in order to determine the setting of UseContainerSupport before > > > > Metrics are being instantiated. > > > > > > > > The intention is for this patch to be backported to older JDKs so as to > > > > provide a means to disable container metrics should things go wrong > > > > with backports of the likes of JDK-8226575. > > > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8250627 > > > > webrev: https://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8250627/01/webrev/ > > > > > > Seems quite simple and clean. > > > > > > One query though, I'm not clear on who the expected caller of > > > Metrics.getInstance() is? (Coming from the perspective of "might we want > > > to cache the fact container support is not enabled?".) > > > > I know of two uses so far: > > > > 1) Launcher (-XshowSettings:system): > > http://hg.openjdk.java.net/jdk/jdk/file/89fe9e02a522/src/java.base/share/classes/sun/launcher/LauncherHelper.java#l318 > > > > 2) OperatingSystemMXBean: > > http://hg.openjdk.java.net/jdk/jdk/file/89fe9e02a522/src/jdk.management/unix/classes/com/sun/management/internal/OperatingSystemImpl.java#l48 > > > > Both uses seem OK as is. Is it worth caching something here? > > No that looks fine. I didn't find them because of the reflective > invocation in Metrics.systemMetrics(). > > > > Also note that we no longer update JVM_INTERFACE_VERSION (last update > > > was JDK 13) - it is meaningless now the JDK and hotspot are in sync > > > version wise. > > > > OK. Does that mean I should revert the version increment, then? > > I would leave it unchanged, yes. Here you go: https://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8250627/02/webrev/ OK? Thanks, Severin From bob.vandette at oracle.com Tue Jul 28 14:14:42 2020 From: bob.vandette at oracle.com (Bob Vandette) Date: Tue, 28 Jul 2020 10:14:42 -0400 Subject: RFR(s): 8250627: Use -XX:+/-UseContainerSupport for enabling/disabling Java container metrics In-Reply-To: References: <94cf1c36-0f55-7027-f2d2-f8b0dadd9366@oracle.com> <5c91581298ac70a98587a01edba46314cf53b0c2.camel@redhat.com> <08475a75-aa37-c649-6585-404d42e10d6d@oracle.com> Message-ID: Looks good to me. Bob. > On Jul 28, 2020, at 10:01 AM, Severin Gehwolf wrote: > > Hi David, > > Thanks for the review. > > On Tue, 2020-07-28 at 23:49 +1000, David Holmes wrote: >> Hi Severin, >> >> On 28/07/2020 11:23 pm, Severin Gehwolf wrote: >>> Hi David, >>> >>> On Tue, 2020-07-28 at 21:17 +1000, David Holmes wrote: >>>> Hi Severin, >>>> >>>> On 28/07/2020 6:27 pm, Severin Gehwolf wrote: >>>>> Hi, >>>>> >>>>> Please review this patch which makes the Java container metrics adhere >>>>> to -XX:+/-UseContainerSupport so they can be disabled if heuristics >>>>> turn out to be wrong. The approach taken is to use JNI and call into >>>>> the JVM in order to determine the setting of UseContainerSupport before >>>>> Metrics are being instantiated. >>>>> >>>>> The intention is for this patch to be backported to older JDKs so as to >>>>> provide a means to disable container metrics should things go wrong >>>>> with backports of the likes of JDK-8226575. >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8250627 >>>>> webrev: https://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8250627/01/webrev/ >>>> >>>> Seems quite simple and clean. >>>> >>>> One query though, I'm not clear on who the expected caller of >>>> Metrics.getInstance() is? (Coming from the perspective of "might we want >>>> to cache the fact container support is not enabled?".) >>> >>> I know of two uses so far: >>> >>> 1) Launcher (-XshowSettings:system): >>> http://hg.openjdk.java.net/jdk/jdk/file/89fe9e02a522/src/java.base/share/classes/sun/launcher/LauncherHelper.java#l318 >>> >>> 2) OperatingSystemMXBean: >>> http://hg.openjdk.java.net/jdk/jdk/file/89fe9e02a522/src/jdk.management/unix/classes/com/sun/management/internal/OperatingSystemImpl.java#l48 >>> >>> Both uses seem OK as is. Is it worth caching something here? >> >> No that looks fine. I didn't find them because of the reflective >> invocation in Metrics.systemMetrics(). >> >>>> Also note that we no longer update JVM_INTERFACE_VERSION (last update >>>> was JDK 13) - it is meaningless now the JDK and hotspot are in sync >>>> version wise. >>> >>> OK. Does that mean I should revert the version increment, then? >> >> I would leave it unchanged, yes. > > Here you go: > https://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8250627/02/webrev/ > > OK? > > Thanks, > Severin > From sgehwolf at redhat.com Tue Jul 28 14:17:12 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Tue, 28 Jul 2020 16:17:12 +0200 Subject: RFR(s): 8250627: Use -XX:+/-UseContainerSupport for enabling/disabling Java container metrics In-Reply-To: References: <94cf1c36-0f55-7027-f2d2-f8b0dadd9366@oracle.com> <5c91581298ac70a98587a01edba46314cf53b0c2.camel@redhat.com> <08475a75-aa37-c649-6585-404d42e10d6d@oracle.com> Message-ID: <79f88ebd6cf72be5f8fcc32360fa561c7cd2988c.camel@redhat.com> On Tue, 2020-07-28 at 10:14 -0400, Bob Vandette wrote: > Looks good to me. Thanks for the review, Bob! Cheers, Severin From leonid.mesnik at oracle.com Tue Jul 28 20:51:08 2020 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Tue, 28 Jul 2020 13:51:08 -0700 Subject: RFR: 8244537: JDI tests fail due to "ERROR: Exception : nsk.share.jdi.JDITestRuntimeException: JDITestRuntimeException : ** event IS NOT a breakpoint **" In-Reply-To: References: <16EB9126-382A-4092-BB61-14E9C3CF208C@oracle.com> Message-ID: <68FBDE47-11DA-45C3-AD07-8200DCC9BA29@oracle.com> Test should fail in cases 0 and 2 with IllegalThreadStateException if we can enable events. Such failures should be easily identified by reading logs. Leonid > On Jul 27, 2020, at 10:28 PM, serguei.spitsyn at oracle.com wrote: > > Hi Leonid, > > The fix looks good in general. > You missed to explain that the suspend/resume are added to avoid actual generation of event that cause this issue. > The reason is that these events are not actually required. > > > http://cr.openjdk.java.net/~lmesnik/8244537/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/jdi/EventRequest/setEnabled/setenabled003.java.frames.html > 316 case 1: > 317 vm.suspend(); > ... > 336 vm.resume(); > > Q: Why is only in case 1 suspend/resume used? > What about cases 0 and 2? > > Thanks, > Serguei > > > On 7/27/20 18:08, Leonid Mesnik wrote: >> Hi >> >> Could you please review following fix which suspends debugger VM while enabling/disabling events. >> >> All changed tests fail intermittently getting unexpected events instead of breakpoint used for communication between debugger/debuggee VM. The tests request different events and verify request's properties but don't process/verify events themselves. Test doesn't aware if events are generated or not. The vm suspension doesn't affect JDWP native agent and it still should get and verify JDWP commands. >> >> webrev: http://cr.openjdk.java.net/~lmesnik/8244537/webrev.00/ >> bug: https://bugs.openjdk.java.net/browse/JDK-8244537 >> >> >> Leonid > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Tue Jul 28 20:54:25 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 28 Jul 2020 13:54:25 -0700 Subject: RFR: 8244537: JDI tests fail due to "ERROR: Exception : nsk.share.jdi.JDITestRuntimeException: JDITestRuntimeException : ** event IS NOT a breakpoint **" In-Reply-To: <68FBDE47-11DA-45C3-AD07-8200DCC9BA29@oracle.com> References: <16EB9126-382A-4092-BB61-14E9C3CF208C@oracle.com> <68FBDE47-11DA-45C3-AD07-8200DCC9BA29@oracle.com> Message-ID: <5d78a07f-6374-7edb-c57c-8a34faf4d840@oracle.com> An HTML attachment was scrubbed... URL: From leonid.mesnik at oracle.com Tue Jul 28 20:59:06 2020 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Tue, 28 Jul 2020 13:59:06 -0700 Subject: RFR: 8244537: JDI tests fail due to "ERROR: Exception : nsk.share.jdi.JDITestRuntimeException: JDITestRuntimeException : ** event IS NOT a breakpoint **" In-Reply-To: <5d78a07f-6374-7edb-c57c-8a34faf4d840@oracle.com> References: <16EB9126-382A-4092-BB61-14E9C3CF208C@oracle.com> <68FBDE47-11DA-45C3-AD07-8200DCC9BA29@oracle.com> <5d78a07f-6374-7edb-c57c-8a34faf4d840@oracle.com> Message-ID: <4CF129DD-1733-47A0-91F9-B27891BCC63E@oracle.com> It should be failure anyway if we managed to enable events, so we don't expect to really enable anything in these cases. However I agree that adding suspend/resume shouldn't make it worse, just possible cleaner log (in very rare cases also). If you feel it is need I will just add suspension for all cases. Leonid > On Jul 28, 2020, at 1:54 PM, serguei.spitsyn at oracle.com wrote: > > Does it mean, you did not fix cases 0 and 2 because the related failures have never been observed? > > Thanks, > Serguei > > > On 7/28/20 13:51, Leonid Mesnik wrote: >> Test should fail in cases 0 and 2 with IllegalThreadStateException if we can enable events. Such failures should be easily identified by reading logs. >> >> Leonid >> >> >>> On Jul 27, 2020, at 10:28 PM, serguei.spitsyn at oracle.com wrote: >>> >>> Hi Leonid, >>> >>> The fix looks good in general. >>> You missed to explain that the suspend/resume are added to avoid actual generation of event that cause this issue. >>> The reason is that these events are not actually required. >>> >>> >>> http://cr.openjdk.java.net/~lmesnik/8244537/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/jdi/EventRequest/setEnabled/setenabled003.java.frames.html >>> 316 case 1: >>> 317 vm.suspend(); >>> ... >>> 336 vm.resume(); >>> >>> Q: Why is only in case 1 suspend/resume used? >>> What about cases 0 and 2? >>> >>> Thanks, >>> Serguei >>> >>> >>> On 7/27/20 18:08, Leonid Mesnik wrote: >>>> Hi >>>> >>>> Could you please review following fix which suspends debugger VM while enabling/disabling events. >>>> >>>> All changed tests fail intermittently getting unexpected events instead of breakpoint used for communication between debugger/debuggee VM. The tests request different events and verify request's properties but don't process/verify events themselves. Test doesn't aware if events are generated or not. The vm suspension doesn't affect JDWP native agent and it still should get and verify JDWP commands. >>>> >>>> webrev: http://cr.openjdk.java.net/~lmesnik/8244537/webrev.00/ >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8244537 >>>> >>>> >>>> Leonid >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Tue Jul 28 21:06:35 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 28 Jul 2020 14:06:35 -0700 Subject: RFR: 8244537: JDI tests fail due to "ERROR: Exception : nsk.share.jdi.JDITestRuntimeException: JDITestRuntimeException : ** event IS NOT a breakpoint **" In-Reply-To: <4CF129DD-1733-47A0-91F9-B27891BCC63E@oracle.com> References: <16EB9126-382A-4092-BB61-14E9C3CF208C@oracle.com> <68FBDE47-11DA-45C3-AD07-8200DCC9BA29@oracle.com> <5d78a07f-6374-7edb-c57c-8a34faf4d840@oracle.com> <4CF129DD-1733-47A0-91F9-B27891BCC63E@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Tue Jul 28 22:48:31 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 29 Jul 2020 08:48:31 +1000 Subject: RFR(s): 8250627: Use -XX:+/-UseContainerSupport for enabling/disabling Java container metrics In-Reply-To: References: <94cf1c36-0f55-7027-f2d2-f8b0dadd9366@oracle.com> <5c91581298ac70a98587a01edba46314cf53b0c2.camel@redhat.com> <08475a75-aa37-c649-6585-404d42e10d6d@oracle.com> Message-ID: Update looks good. Thanks, David On 29/07/2020 12:01 am, Severin Gehwolf wrote: > Hi David, > > Thanks for the review. > > On Tue, 2020-07-28 at 23:49 +1000, David Holmes wrote: >> Hi Severin, >> >> On 28/07/2020 11:23 pm, Severin Gehwolf wrote: >>> Hi David, >>> >>> On Tue, 2020-07-28 at 21:17 +1000, David Holmes wrote: >>>> Hi Severin, >>>> >>>> On 28/07/2020 6:27 pm, Severin Gehwolf wrote: >>>>> Hi, >>>>> >>>>> Please review this patch which makes the Java container metrics adhere >>>>> to -XX:+/-UseContainerSupport so they can be disabled if heuristics >>>>> turn out to be wrong. The approach taken is to use JNI and call into >>>>> the JVM in order to determine the setting of UseContainerSupport before >>>>> Metrics are being instantiated. >>>>> >>>>> The intention is for this patch to be backported to older JDKs so as to >>>>> provide a means to disable container metrics should things go wrong >>>>> with backports of the likes of JDK-8226575. >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8250627 >>>>> webrev: https://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8250627/01/webrev/ >>>> >>>> Seems quite simple and clean. >>>> >>>> One query though, I'm not clear on who the expected caller of >>>> Metrics.getInstance() is? (Coming from the perspective of "might we want >>>> to cache the fact container support is not enabled?".) >>> >>> I know of two uses so far: >>> >>> 1) Launcher (-XshowSettings:system): >>> http://hg.openjdk.java.net/jdk/jdk/file/89fe9e02a522/src/java.base/share/classes/sun/launcher/LauncherHelper.java#l318 >>> >>> 2) OperatingSystemMXBean: >>> http://hg.openjdk.java.net/jdk/jdk/file/89fe9e02a522/src/jdk.management/unix/classes/com/sun/management/internal/OperatingSystemImpl.java#l48 >>> >>> Both uses seem OK as is. Is it worth caching something here? >> >> No that looks fine. I didn't find them because of the reflective >> invocation in Metrics.systemMetrics(). >> >>>> Also note that we no longer update JVM_INTERFACE_VERSION (last update >>>> was JDK 13) - it is meaningless now the JDK and hotspot are in sync >>>> version wise. >>> >>> OK. Does that mean I should revert the version increment, then? >> >> I would leave it unchanged, yes. > > Here you go: > https://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8250627/02/webrev/ > > OK? > > Thanks, > Severin > From chris.plummer at oracle.com Tue Jul 28 23:02:09 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 28 Jul 2020 16:02:09 -0700 Subject: RFR(T): 8250742: ProblemList serviceability/sa/ClhsdbPstack.java #id0 and #id1 for ZGC Message-ID: <19635fd9-dfc6-ef73-ba15-4c763d5673ae@oracle.com> Hello, Please review the following. diff --git a/test/hotspot/jtreg/ProblemList-zgc.txt b/test/hotspot/jtreg/ProblemList-zgc.txt --- a/test/hotspot/jtreg/ProblemList-zgc.txt +++ b/test/hotspot/jtreg/ProblemList-zgc.txt @@ -44,7 +44,8 @@ ?serviceability/sa/ClhsdbJstackXcompStress.java 8220624?? generic-all ?serviceability/sa/ClhsdbPrintAs.java 8220624?? generic-all ?serviceability/sa/ClhsdbPrintStatics.java 8220624?? generic-all -serviceability/sa/ClhsdbPstack.java 8220624?? generic-all +serviceability/sa/ClhsdbPstack.java#id0 8220624?? generic-all +serviceability/sa/ClhsdbPstack.java#id1 8220624?? generic-all ?serviceability/sa/ClhsdbSource.java 8220624?? generic-all ?serviceability/sa/ClhsdbThread.java 8220624?? generic-all ?serviceability/sa/ClhsdbWhere.java 8220624?? generic-all ClhsdbPstack.java was already problem listed, but I added a new subtest, so now instead we have to problem list #id0 and #id1. Currently testing. I'll push once I get validation that the tests have been properly problem listed. thanks, Chris From serguei.spitsyn at oracle.com Tue Jul 28 23:17:21 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 28 Jul 2020 16:17:21 -0700 Subject: RFR(T): 8250742: ProblemList serviceability/sa/ClhsdbPstack.java #id0 and #id1 for ZGC In-Reply-To: <19635fd9-dfc6-ef73-ba15-4c763d5673ae@oracle.com> References: <19635fd9-dfc6-ef73-ba15-4c763d5673ae@oracle.com> Message-ID: <8e7a9fa0-88b3-a8ed-25a0-b2c16cea2520@oracle.com> Hi Chris, It looks good and trivial. Thanks, Serguei On 7/28/20 16:02, Chris Plummer wrote: > Hello, > > Please review the following. > > diff --git a/test/hotspot/jtreg/ProblemList-zgc.txt > b/test/hotspot/jtreg/ProblemList-zgc.txt > --- a/test/hotspot/jtreg/ProblemList-zgc.txt > +++ b/test/hotspot/jtreg/ProblemList-zgc.txt > @@ -44,7 +44,8 @@ > ?serviceability/sa/ClhsdbJstackXcompStress.java 8220624 generic-all > ?serviceability/sa/ClhsdbPrintAs.java 8220624?? generic-all > ?serviceability/sa/ClhsdbPrintStatics.java 8220624?? generic-all > -serviceability/sa/ClhsdbPstack.java 8220624?? generic-all > +serviceability/sa/ClhsdbPstack.java#id0 8220624?? generic-all > +serviceability/sa/ClhsdbPstack.java#id1 8220624?? generic-all > ?serviceability/sa/ClhsdbSource.java 8220624?? generic-all > ?serviceability/sa/ClhsdbThread.java 8220624?? generic-all > ?serviceability/sa/ClhsdbWhere.java 8220624?? generic-all > > ClhsdbPstack.java was already problem listed, but I added a new > subtest, so now instead we have to problem list #id0 and #id1. > Currently testing. I'll push once I get validation that the tests have > been properly problem listed. > > thanks, > > Chris > From leonid.mesnik at oracle.com Tue Jul 28 23:49:44 2020 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Tue, 28 Jul 2020 16:49:44 -0700 Subject: RFR: 8244537: JDI tests fail due to "ERROR: Exception : nsk.share.jdi.JDITestRuntimeException: JDITestRuntimeException : ** event IS NOT a breakpoint **" In-Reply-To: References: <16EB9126-382A-4092-BB61-14E9C3CF208C@oracle.com> <68FBDE47-11DA-45C3-AD07-8200DCC9BA29@oracle.com> <5d78a07f-6374-7edb-c57c-8a34faf4d840@oracle.com> <4CF129DD-1733-47A0-91F9-B27891BCC63E@oracle.com> Message-ID: I've update to suspend/resume in all cases. new webrev: http://cr.openjdk.java.net/~lmesnik/8244537/webrev.01/ Leonid > On Jul 28, 2020, at 2:06 PM, serguei.spitsyn at oracle.com wrote: > > I prefer to suspend/resume in all cases, so we avoid all these unexpected failures. > > Thanks, > Serguei > > > On 7/28/20 13:59, Leonid Mesnik wrote: >> It should be failure anyway if we managed to enable events, so we don't expect to really enable anything in these cases. >> However I agree that adding suspend/resume shouldn't make it worse, just possible cleaner log (in very rare cases also). If you feel it is need I will just add suspension for all cases. >> >> Leonid >> >>> On Jul 28, 2020, at 1:54 PM, serguei.spitsyn at oracle.com wrote: >>> >>> Does it mean, you did not fix cases 0 and 2 because the related failures have never been observed? >>> >>> Thanks, >>> Serguei >>> >>> >>> On 7/28/20 13:51, Leonid Mesnik wrote: >>>> Test should fail in cases 0 and 2 with IllegalThreadStateException if we can enable events. Such failures should be easily identified by reading logs. >>>> >>>> Leonid >>>> >>>> >>>>> On Jul 27, 2020, at 10:28 PM, serguei.spitsyn at oracle.com wrote: >>>>> >>>>> Hi Leonid, >>>>> >>>>> The fix looks good in general. >>>>> You missed to explain that the suspend/resume are added to avoid actual generation of event that cause this issue. >>>>> The reason is that these events are not actually required. >>>>> >>>>> >>>>> http://cr.openjdk.java.net/~lmesnik/8244537/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/jdi/EventRequest/setEnabled/setenabled003.java.frames.html >>>>> 316 case 1: >>>>> 317 vm.suspend(); >>>>> ... >>>>> 336 vm.resume(); >>>>> >>>>> Q: Why is only in case 1 suspend/resume used? >>>>> What about cases 0 and 2? >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>> On 7/27/20 18:08, Leonid Mesnik wrote: >>>>>> Hi >>>>>> >>>>>> Could you please review following fix which suspends debugger VM while enabling/disabling events. >>>>>> >>>>>> All changed tests fail intermittently getting unexpected events instead of breakpoint used for communication between debugger/debuggee VM. The tests request different events and verify request's properties but don't process/verify events themselves. Test doesn't aware if events are generated or not. The vm suspension doesn't affect JDWP native agent and it still should get and verify JDWP commands. >>>>>> >>>>>> webrev: http://cr.openjdk.java.net/~lmesnik/8244537/webrev.00/ >>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8244537 >>>>>> >>>>>> >>>>>> Leonid >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Wed Jul 29 00:45:20 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 28 Jul 2020 17:45:20 -0700 Subject: RFR: 8244537: JDI tests fail due to "ERROR: Exception : nsk.share.jdi.JDITestRuntimeException: JDITestRuntimeException : ** event IS NOT a breakpoint **" In-Reply-To: References: <16EB9126-382A-4092-BB61-14E9C3CF208C@oracle.com> <68FBDE47-11DA-45C3-AD07-8200DCC9BA29@oracle.com> <5d78a07f-6374-7edb-c57c-8a34faf4d840@oracle.com> <4CF129DD-1733-47A0-91F9-B27891BCC63E@oracle.com> Message-ID: <11e29ad2-8e06-ad30-df48-6dcfea0bd4dd@oracle.com> An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Wed Jul 29 01:44:27 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 28 Jul 2020 18:44:27 -0700 Subject: RFR: 8244537: JDI tests fail due to "ERROR: Exception : nsk.share.jdi.JDITestRuntimeException: JDITestRuntimeException : ** event IS NOT a breakpoint **" In-Reply-To: <11e29ad2-8e06-ad30-df48-6dcfea0bd4dd@oracle.com> References: <16EB9126-382A-4092-BB61-14E9C3CF208C@oracle.com> <68FBDE47-11DA-45C3-AD07-8200DCC9BA29@oracle.com> <5d78a07f-6374-7edb-c57c-8a34faf4d840@oracle.com> <4CF129DD-1733-47A0-91F9-B27891BCC63E@oracle.com> <11e29ad2-8e06-ad30-df48-6dcfea0bd4dd@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Wed Jul 29 03:40:31 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 28 Jul 2020 20:40:31 -0700 Subject: RFR(XS): 8248879: SA core file support on OSX has some bugs trying to locate the jvm libraries In-Reply-To: <6f8cebde-428e-9e70-ed15-b05b6f6446e8@oracle.com> References: <6089f692-152b-8615-dfca-779f2b5028a0@oracle.com> <219dd806-ca93-64ce-f5f9-d75dd7acb7f1@oracle.com> <37eee0d3-6b00-eb11-6198-d8d87e3a7ad4@oracle.com> <99909975-074c-34a1-0f9b-69c4fb8f0eff@oracle.com> <8c7edf32-3037-b8d7-a18e-587f7ae8d294@oracle.com> <9bd21f67-982a-ef98-7c9a-3515c38b688c@oracle.com> <0906c510-8c2c-2c56-803f-0a0bf4340df6@oracle.com> <6f8cebde-428e-9e70-ed15-b05b6f6446e8@oracle.com> Message-ID: <7fc9666b-ce1e-f83a-e7a7-e052ca451a09@oracle.com> Hi Serguei and Alex, Sorry about the delay getting back to this. I got sidetracked with other bugs and also realized the code needed more work than just Alex's suggestion for rstrstr(). As a bit of background first, get_real_path() is used to locate any library that is referenced from the core file using a relative path. So the core file will, for example, refer to @rpath/libjvm.dylib, and get_real_path() will convert that to a usable path to the file. Usually only JDK libraries and user libraries are specified with @rpath. System libraries all use full path names. get_real_path() had a couple of shortcomings. The way it worked is if the specified execname ended in bin/java or if $JAVA_HOME was set, then it only checked for libraries in subdirs of the first one of those 2 that it found to be valid. It would not look in both directories if both were valid, only in the first to be found valid. Only if neither of those were valid did it look in DYLD_LIBRARY_PATH. So, for example, as long as execname ended in bin/java, that's the only jdk directory that was checked for libraries. If it didn't end in bin/java, and $JAVA_HOME was set, then only it was checked. Then I added a 3rd option looking for the existence of any "bin/" in execname. Only if none of these 3 paths existed did the code defer to DYLD_LIBRARY_PATH. That made is hard to locate non JDK libraries, such as user JNI libraries, or to override the execname search for the JDK by setting $JAVA_HOME. I've fixed this by having it check all 3 of the potential JDK locations not only to see if the paths are valid, but also if the library is in any of the paths, and then check all the paths DYLD_LIBRARY_PATH if it failed to find the library in the JDK paths. So now all the potential locations are checked to see if they contain the library. By doing this I was able to make it find the JDK libraries by properly specifying the execname or JAVA_HOME, and still find a user JNI library in DYLD_LIBRARY_PATH. Since the code was kind of a mess and not well suited to just fix with some minor adjustments, I for the most part rewrote it. Although it still does a lot of the same things, it's much cleaner and easier to read now, and there's less replication of similar code. I also replaced strcat and strcpy calls with strncat and strncpy to prevent overflows. I would suggest for this review to just start by looking at get_real_path() and follow the code, and not compare the diffs, which aren't very readable. http://cr.openjdk.java.net/~cjplummer/8248879/webrev.02/index.html thanks, Chris On 7/14/20 8:54 PM, serguei.spitsyn at oracle.com wrote: > Hi Alex, > > Yes, I understand this. > After some thinking, I doubt my suggestion to check all occurrences or > "/bin/" is good. :) > > Thanks, > Serguei > > On 7/14/20 18:19, Alex Menkov wrote: >> Hi Serguei, >> >> On 07/14/2020 15:55, serguei.spitsyn at oracle.com wrote: >>> Hi Chris and Alex, >>> >>> I agree the last occurrence of "/bin/" is better than the first. >>> But I wonder if it makes sense to check all occurrences. >> >> The problem is strrstr (search for last occurrence) is not a part of >> std C lib. >> So to avoid dependency on new library I suggested this simple >> implementation using standard strstr. >> >> --alex >> >>> >>> Thanks, >>> Serguei >>> >>> >>> On 7/14/20 15:14, Alex Menkov wrote: >>>> Yes, you are right. >>>> This is not a function from strings.h >>>> >>>> Ok, you can leave strstr (and keep in mind that the path can't >>>> contain "/bin/" other than jdk's bin) or implement the >>>> functionality. It should be something simple like >>>> >>>> static const char* rstrstr(const char *str, const char *sub) { >>>> ? const char *result = NULL; >>>> ? for (const char *p = strstr(str, sub); p != NULL; p = strstr(p + >>>> 1, sub)) { >>>> ??? result = p; >>>> ? } >>>> ? return result; >>>> } >>>> >>>> --alex >>>> >>>> On 07/14/2020 13:43, Chris Plummer wrote: >>>>> Actually it's not so easy. I don't see any other references to >>>>> strrstr in our source. When I reference strstr, it gives a warning >>>>> because it's not declared. The only man page I can find says to >>>>> include sstring2.h, but this file does not exist. It also says to >>>>> link with -lsstrings2. >>>>> >>>>> Chris >>>>> >>>>> On 7/14/20 1:37 PM, Chris Plummer wrote: >>>>>> Ok. I'll change both references to use strrstr. >>>>>> >>>>>> thanks, >>>>>> >>>>>> Chris >>>>>> >>>>>> On 7/14/20 1:11 PM, Alex Menkov wrote: >>>>>>> Hi Chris, >>>>>>> >>>>>>> I think it would be better to use strrstr to correctly handle >>>>>>> paths like >>>>>>> /something/bin/jdk/bin/jhsdb >>>>>>> >>>>>>> And I'd updated >>>>>>> 358?? char* posbin = strstr(execname, "/bin/java"); >>>>>>> to use strrstr as well >>>>>>> >>>>>>> --alex >>>>>>> >>>>>>> On 07/14/2020 12:01, Chris Plummer wrote: >>>>>>>> Ping! >>>>>>>> >>>>>>>> On 7/6/20 9:31 PM, Chris Plummer wrote: >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> Please help review the following: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~cjplummer/8248879/webrev.00/index.html >>>>>>>>> >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8248879 >>>>>>>>> >>>>>>>>> The description of the problem and the fix are both in the CR. >>>>>>>>> >>>>>>>>> thanks, >>>>>>>>> >>>>>>>>> Chris >>>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>> > From leonid.mesnik at oracle.com Wed Jul 29 03:55:25 2020 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Tue, 28 Jul 2020 20:55:25 -0700 Subject: RFR: 8244537: JDI tests fail due to "ERROR: Exception : nsk.share.jdi.JDITestRuntimeException: JDITestRuntimeException : ** event IS NOT a breakpoint **" In-Reply-To: <11e29ad2-8e06-ad30-df48-6dcfea0bd4dd@oracle.com> References: <16EB9126-382A-4092-BB61-14E9C3CF208C@oracle.com> <68FBDE47-11DA-45C3-AD07-8200DCC9BA29@oracle.com> <5d78a07f-6374-7edb-c57c-8a34faf4d840@oracle.com> <4CF129DD-1733-47A0-91F9-B27891BCC63E@oracle.com> <11e29ad2-8e06-ad30-df48-6dcfea0bd4dd@oracle.com> Message-ID: Thanks for catching this! I placed it here by mistake. It should be before switch. http://cr.openjdk.java.net/~lmesnik/8244537/webrev.03/ Leonid > On Jul 28, 2020, at 5:45 PM, serguei.spitsyn at oracle.com wrote: > > http://cr.openjdk.java.net/~lmesnik/8244537/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jdi/EventRequest/setEnabled/setenabled003.java.frames.html > 288 case 0: > 289 thread1 = (ThreadReference) > 290 debuggeeClass.getValue(debuggeeClass.fieldByName(threadName1)); > 291 > 292 log2("......setting up StepRequest"); > 293 eventRequest1 = eventRManager.createStepRequest > 294 (thread1, StepRequest.STEP_MIN, StepRequest.STEP_INTO); > 295 > 296 vm.suspend(); > ... > 360 default: > 361 throw new JDITestRuntimeException("** default case 2 **"); > 362 } > 363 vm.resume(); > > Sorry, the fix is not going to work correctly. > The first vm.suspend() has to be before the switch statement to work for all 3 cases. > > Thanks, > Serguei > > > On 7/28/20 16:49, Leonid Mesnik wrote: >> I've update to suspend/resume in all cases. >> >> new webrev: http://cr.openjdk.java.net/~lmesnik/8244537/webrev.01/ >> >> Leonid >> >>> On Jul 28, 2020, at 2:06 PM, serguei.spitsyn at oracle.com wrote: >>> >>> I prefer to suspend/resume in all cases, so we avoid all these unexpected failures. >>> >>> Thanks, >>> Serguei >>> >>> >>> On 7/28/20 13:59, Leonid Mesnik wrote: >>>> It should be failure anyway if we managed to enable events, so we don't expect to really enable anything in these cases. >>>> However I agree that adding suspend/resume shouldn't make it worse, just possible cleaner log (in very rare cases also). If you feel it is need I will just add suspension for all cases. >>>> >>>> Leonid >>>> >>>>> On Jul 28, 2020, at 1:54 PM, serguei.spitsyn at oracle.com wrote: >>>>> >>>>> Does it mean, you did not fix cases 0 and 2 because the related failures have never been observed? >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>> On 7/28/20 13:51, Leonid Mesnik wrote: >>>>>> Test should fail in cases 0 and 2 with IllegalThreadStateException if we can enable events. Such failures should be easily identified by reading logs. >>>>>> >>>>>> Leonid >>>>>> >>>>>> >>>>>>> On Jul 27, 2020, at 10:28 PM, serguei.spitsyn at oracle.com wrote: >>>>>>> >>>>>>> Hi Leonid, >>>>>>> >>>>>>> The fix looks good in general. >>>>>>> You missed to explain that the suspend/resume are added to avoid actual generation of event that cause this issue. >>>>>>> The reason is that these events are not actually required. >>>>>>> >>>>>>> >>>>>>> http://cr.openjdk.java.net/~lmesnik/8244537/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/jdi/EventRequest/setEnabled/setenabled003.java.frames.html >>>>>>> 316 case 1: >>>>>>> 317 vm.suspend(); >>>>>>> ... >>>>>>> 336 vm.resume(); >>>>>>> >>>>>>> Q: Why is only in case 1 suspend/resume used? >>>>>>> What about cases 0 and 2? >>>>>>> >>>>>>> Thanks, >>>>>>> Serguei >>>>>>> >>>>>>> >>>>>>> On 7/27/20 18:08, Leonid Mesnik wrote: >>>>>>>> Hi >>>>>>>> >>>>>>>> Could you please review following fix which suspends debugger VM while enabling/disabling events. >>>>>>>> >>>>>>>> All changed tests fail intermittently getting unexpected events instead of breakpoint used for communication between debugger/debuggee VM. The tests request different events and verify request's properties but don't process/verify events themselves. Test doesn't aware if events are generated or not. The vm suspension doesn't affect JDWP native agent and it still should get and verify JDWP commands. >>>>>>>> >>>>>>>> webrev: http://cr.openjdk.java.net/~lmesnik/8244537/webrev.00/ >>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8244537 >>>>>>>> >>>>>>>> >>>>>>>> Leonid >>>>>>> >>>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From leonid.mesnik at oracle.com Wed Jul 29 04:15:46 2020 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Tue, 28 Jul 2020 21:15:46 -0700 Subject: RFR: 8244537: JDI tests fail due to "ERROR: Exception : nsk.share.jdi.JDITestRuntimeException: JDITestRuntimeException : ** event IS NOT a breakpoint **" In-Reply-To: References: <16EB9126-382A-4092-BB61-14E9C3CF208C@oracle.com> <68FBDE47-11DA-45C3-AD07-8200DCC9BA29@oracle.com> <5d78a07f-6374-7edb-c57c-8a34faf4d840@oracle.com> <4CF129DD-1733-47A0-91F9-B27891BCC63E@oracle.com> <11e29ad2-8e06-ad30-df48-6dcfea0bd4dd@oracle.com> Message-ID: Included in webrev.03 http://cr.openjdk.java.net/~lmesnik/8244537/webrev.03 Leonid > On Jul 28, 2020, at 6:44 PM, serguei.spitsyn at oracle.com wrote: > > http://cr.openjdk.java.net/~lmesnik/8244537/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jdi/EventRequest/setEnabled/setenabled002.java.frames.html > 371 if (eventRequest1 instanceof StepRequest) { > 372 try { > 373 log2("......eventRequest1.setEnabled(true); IllegalThreadStateException is expected"); > 374 eventRequest1.setEnabled(true); > 375 testExitCode = FAILED; > 376 log3("ERROR: NO IllegalThreadStateException for StepRequest"); > 377 } catch ( IllegalThreadStateException e ) { > 378 log2(" IllegalThreadStateException"); > 379 } > 380 try { > 381 log2("......eventRequest1.setEnabled(false); IllegalThreadStateException is not expected"); > 382 eventRequest1.setEnabled(false); > 383 log2(" no IllegalThreadStateException for StepRequest"); > 384 } catch ( IllegalThreadStateException e ) { > 385 testExitCode = FAILED; > 386 log3("ERROR: IllegalThreadStateException"); > 387 } > 388 } > Above is one more case where suspend/resume is needed, I guess. > > Thanks, > Serguei > > > On 7/28/20 17:45, serguei.spitsyn at oracle.com wrote: >> http://cr.openjdk.java.net/~lmesnik/8244537/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jdi/EventRequest/setEnabled/setenabled003.java.frames.html >> 288 case 0: >> 289 thread1 = (ThreadReference) >> 290 debuggeeClass.getValue(debuggeeClass.fieldByName(threadName1)); >> 291 >> 292 log2("......setting up StepRequest"); >> 293 eventRequest1 = eventRManager.createStepRequest >> 294 (thread1, StepRequest.STEP_MIN, StepRequest.STEP_INTO); >> 295 >> 296 vm.suspend(); >> ... >> 360 default: >> 361 throw new JDITestRuntimeException("** default case 2 **"); >> 362 } >> 363 vm.resume(); >> >> Sorry, the fix is not going to work correctly. >> The first vm.suspend() has to be before the switch statement to work for all 3 cases. >> >> Thanks, >> Serguei >> >> >> On 7/28/20 16:49, Leonid Mesnik wrote: >>> I've update to suspend/resume in all cases. >>> >>> new webrev: http://cr.openjdk.java.net/~lmesnik/8244537/webrev.01/ >>> >>> Leonid >>> >>>> On Jul 28, 2020, at 2:06 PM, serguei.spitsyn at oracle.com wrote: >>>> >>>> I prefer to suspend/resume in all cases, so we avoid all these unexpected failures. >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 7/28/20 13:59, Leonid Mesnik wrote: >>>>> It should be failure anyway if we managed to enable events, so we don't expect to really enable anything in these cases. >>>>> However I agree that adding suspend/resume shouldn't make it worse, just possible cleaner log (in very rare cases also). If you feel it is need I will just add suspension for all cases. >>>>> >>>>> Leonid >>>>> >>>>>> On Jul 28, 2020, at 1:54 PM, serguei.spitsyn at oracle.com wrote: >>>>>> >>>>>> Does it mean, you did not fix cases 0 and 2 because the related failures have never been observed? >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>>> >>>>>> On 7/28/20 13:51, Leonid Mesnik wrote: >>>>>>> Test should fail in cases 0 and 2 with IllegalThreadStateException if we can enable events. Such failures should be easily identified by reading logs. >>>>>>> >>>>>>> Leonid >>>>>>> >>>>>>> >>>>>>>> On Jul 27, 2020, at 10:28 PM, serguei.spitsyn at oracle.com wrote: >>>>>>>> >>>>>>>> Hi Leonid, >>>>>>>> >>>>>>>> The fix looks good in general. >>>>>>>> You missed to explain that the suspend/resume are added to avoid actual generation of event that cause this issue. >>>>>>>> The reason is that these events are not actually required. >>>>>>>> >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~lmesnik/8244537/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/jdi/EventRequest/setEnabled/setenabled003.java.frames.html >>>>>>>> 316 case 1: >>>>>>>> 317 vm.suspend(); >>>>>>>> ... >>>>>>>> 336 vm.resume(); >>>>>>>> >>>>>>>> Q: Why is only in case 1 suspend/resume used? >>>>>>>> What about cases 0 and 2? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Serguei >>>>>>>> >>>>>>>> >>>>>>>> On 7/27/20 18:08, Leonid Mesnik wrote: >>>>>>>>> Hi >>>>>>>>> >>>>>>>>> Could you please review following fix which suspends debugger VM while enabling/disabling events. >>>>>>>>> >>>>>>>>> All changed tests fail intermittently getting unexpected events instead of breakpoint used for communication between debugger/debuggee VM. The tests request different events and verify request's properties but don't process/verify events themselves. Test doesn't aware if events are generated or not. The vm suspension doesn't affect JDWP native agent and it still should get and verify JDWP commands. >>>>>>>>> >>>>>>>>> webrev: http://cr.openjdk.java.net/~lmesnik/8244537/webrev.00/ >>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8244537 >>>>>>>>> >>>>>>>>> >>>>>>>>> Leonid >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Wed Jul 29 06:06:35 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 28 Jul 2020 23:06:35 -0700 Subject: RFR: 8244537: JDI tests fail due to "ERROR: Exception : nsk.share.jdi.JDITestRuntimeException: JDITestRuntimeException : ** event IS NOT a breakpoint **" In-Reply-To: References: <16EB9126-382A-4092-BB61-14E9C3CF208C@oracle.com> <68FBDE47-11DA-45C3-AD07-8200DCC9BA29@oracle.com> <5d78a07f-6374-7edb-c57c-8a34faf4d840@oracle.com> <4CF129DD-1733-47A0-91F9-B27891BCC63E@oracle.com> <11e29ad2-8e06-ad30-df48-6dcfea0bd4dd@oracle.com> Message-ID: <764e3f2d-75a8-143d-d29f-01ea44d0b6e0@oracle.com> An HTML attachment was scrubbed... URL: From leonid.mesnik at oracle.com Wed Jul 29 06:26:46 2020 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Tue, 28 Jul 2020 23:26:46 -0700 Subject: RFR: 8244537: JDI tests fail due to "ERROR: Exception : nsk.share.jdi.JDITestRuntimeException: JDITestRuntimeException : ** event IS NOT a breakpoint **" In-Reply-To: <764e3f2d-75a8-143d-d29f-01ea44d0b6e0@oracle.com> References: <16EB9126-382A-4092-BB61-14E9C3CF208C@oracle.com> <68FBDE47-11DA-45C3-AD07-8200DCC9BA29@oracle.com> <5d78a07f-6374-7edb-c57c-8a34faf4d840@oracle.com> <4CF129DD-1733-47A0-91F9-B27891BCC63E@oracle.com> <11e29ad2-8e06-ad30-df48-6dcfea0bd4dd@oracle.com> <764e3f2d-75a8-143d-d29f-01ea44d0b6e0@oracle.com> Message-ID: <5CA977F6-2B49-4E47-B5BD-E4EA68F361FA@oracle.com> ok, let change it to following diff -r f489d5d13a51 test/hotspot/jtreg/vmTestbase/nsk/jdi/EventRequest/setEnabled/setenabled002.java --- a/test/hotspot/jtreg/vmTestbase/nsk/jdi/EventRequest/setEnabled/setenabled002.java Thu Jul 23 16:36:44 2020 -0400 +++ b/test/hotspot/jtreg/vmTestbase/nsk/jdi/EventRequest/setEnabled/setenabled002.java Tue Jul 28 23:17:22 2020 -0700 @@ -368,6 +368,7 @@ throw new JDITestRuntimeException("** default case 2 **"); } + vm.suspend(); if (eventRequest1 instanceof StepRequest) { try { log2("......eventRequest1.setEnabled(true); IllegalThreadStateException is expected"); @@ -405,6 +406,7 @@ } catch ( InvalidRequestStateException e ) { log2(" InvalidRequestStateException"); } + vm.resume(); //~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ } webrev: http://cr.openjdk.java.net/~lmesnik/8244537/webrev.04/ Leonid > On Jul 28, 2020, at 11:06 PM, serguei.spitsyn at oracle.com wrote: > > http://cr.openjdk.java.net/~lmesnik/8244537/webrev.03/test/hotspot/jtreg/vmTestbase/nsk/jdi/EventRequest/setEnabled/setenabled002.java.frames.html > > I'd suggest to simplify it: > > - insert suspend before the line: > 371 if (eventRequest1 instanceof StepRequest) { > - keep resume at the line: > 411 vm.resume(); > - these lines are not needed: > 389 vm.resume(); > 394 vm.suspend(); > > Thanks, > Serguei > > > On 7/28/20 21:15, Leonid Mesnik wrote: >> Included in webrev.03 >> http://cr.openjdk.java.net/~lmesnik/8244537/webrev.03 >> >> Leonid >> >>> On Jul 28, 2020, at 6:44 PM, serguei.spitsyn at oracle.com wrote: >>> >>> http://cr.openjdk.java.net/~lmesnik/8244537/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jdi/EventRequest/setEnabled/setenabled002.java.frames.html >>> 371 if (eventRequest1 instanceof StepRequest) { >>> 372 try { >>> 373 log2("......eventRequest1.setEnabled(true); IllegalThreadStateException is expected"); >>> 374 eventRequest1.setEnabled(true); >>> 375 testExitCode = FAILED; >>> 376 log3("ERROR: NO IllegalThreadStateException for StepRequest"); >>> 377 } catch ( IllegalThreadStateException e ) { >>> 378 log2(" IllegalThreadStateException"); >>> 379 } >>> 380 try { >>> 381 log2("......eventRequest1.setEnabled(false); IllegalThreadStateException is not expected"); >>> 382 eventRequest1.setEnabled(false); >>> 383 log2(" no IllegalThreadStateException for StepRequest"); >>> 384 } catch ( IllegalThreadStateException e ) { >>> 385 testExitCode = FAILED; >>> 386 log3("ERROR: IllegalThreadStateException"); >>> 387 } >>> 388 } >>> Above is one more case where suspend/resume is needed, I guess. >>> >>> Thanks, >>> Serguei >>> >>> >>> On 7/28/20 17:45, serguei.spitsyn at oracle.com wrote: >>>> http://cr.openjdk.java.net/~lmesnik/8244537/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jdi/EventRequest/setEnabled/setenabled003.java.frames.html >>>> 288 case 0: >>>> 289 thread1 = (ThreadReference) >>>> 290 debuggeeClass.getValue(debuggeeClass.fieldByName(threadName1)); >>>> 291 >>>> 292 log2("......setting up StepRequest"); >>>> 293 eventRequest1 = eventRManager.createStepRequest >>>> 294 (thread1, StepRequest.STEP_MIN, StepRequest.STEP_INTO); >>>> 295 >>>> 296 vm.suspend(); >>>> ... >>>> 360 default: >>>> 361 throw new JDITestRuntimeException("** default case 2 **"); >>>> 362 } >>>> 363 vm.resume(); >>>> >>>> Sorry, the fix is not going to work correctly. >>>> The first vm.suspend() has to be before the switch statement to work for all 3 cases. >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 7/28/20 16:49, Leonid Mesnik wrote: >>>>> I've update to suspend/resume in all cases. >>>>> >>>>> new webrev: http://cr.openjdk.java.net/~lmesnik/8244537/webrev.01/ >>>>> >>>>> Leonid >>>>> >>>>>> On Jul 28, 2020, at 2:06 PM, serguei.spitsyn at oracle.com wrote: >>>>>> >>>>>> I prefer to suspend/resume in all cases, so we avoid all these unexpected failures. >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>>> >>>>>> On 7/28/20 13:59, Leonid Mesnik wrote: >>>>>>> It should be failure anyway if we managed to enable events, so we don't expect to really enable anything in these cases. >>>>>>> However I agree that adding suspend/resume shouldn't make it worse, just possible cleaner log (in very rare cases also). If you feel it is need I will just add suspension for all cases. >>>>>>> >>>>>>> Leonid >>>>>>> >>>>>>>> On Jul 28, 2020, at 1:54 PM, serguei.spitsyn at oracle.com wrote: >>>>>>>> >>>>>>>> Does it mean, you did not fix cases 0 and 2 because the related failures have never been observed? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Serguei >>>>>>>> >>>>>>>> >>>>>>>> On 7/28/20 13:51, Leonid Mesnik wrote: >>>>>>>>> Test should fail in cases 0 and 2 with IllegalThreadStateException if we can enable events. Such failures should be easily identified by reading logs. >>>>>>>>> >>>>>>>>> Leonid >>>>>>>>> >>>>>>>>> >>>>>>>>>> On Jul 27, 2020, at 10:28 PM, serguei.spitsyn at oracle.com wrote: >>>>>>>>>> >>>>>>>>>> Hi Leonid, >>>>>>>>>> >>>>>>>>>> The fix looks good in general. >>>>>>>>>> You missed to explain that the suspend/resume are added to avoid actual generation of event that cause this issue. >>>>>>>>>> The reason is that these events are not actually required. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~lmesnik/8244537/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/jdi/EventRequest/setEnabled/setenabled003.java.frames.html >>>>>>>>>> 316 case 1: >>>>>>>>>> 317 vm.suspend(); >>>>>>>>>> ... >>>>>>>>>> 336 vm.resume(); >>>>>>>>>> >>>>>>>>>> Q: Why is only in case 1 suspend/resume used? >>>>>>>>>> What about cases 0 and 2? >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Serguei >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 7/27/20 18:08, Leonid Mesnik wrote: >>>>>>>>>>> Hi >>>>>>>>>>> >>>>>>>>>>> Could you please review following fix which suspends debugger VM while enabling/disabling events. >>>>>>>>>>> >>>>>>>>>>> All changed tests fail intermittently getting unexpected events instead of breakpoint used for communication between debugger/debuggee VM. The tests request different events and verify request's properties but don't process/verify events themselves. Test doesn't aware if events are generated or not. The vm suspension doesn't affect JDWP native agent and it still should get and verify JDWP commands. >>>>>>>>>>> >>>>>>>>>>> webrev: http://cr.openjdk.java.net/~lmesnik/8244537/webrev.00/ >>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8244537 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Leonid >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Wed Jul 29 07:08:39 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 29 Jul 2020 00:08:39 -0700 Subject: RFR: 8244537: JDI tests fail due to "ERROR: Exception : nsk.share.jdi.JDITestRuntimeException: JDITestRuntimeException : ** event IS NOT a breakpoint **" In-Reply-To: <5CA977F6-2B49-4E47-B5BD-E4EA68F361FA@oracle.com> References: <16EB9126-382A-4092-BB61-14E9C3CF208C@oracle.com> <68FBDE47-11DA-45C3-AD07-8200DCC9BA29@oracle.com> <5d78a07f-6374-7edb-c57c-8a34faf4d840@oracle.com> <4CF129DD-1733-47A0-91F9-B27891BCC63E@oracle.com> <11e29ad2-8e06-ad30-df48-6dcfea0bd4dd@oracle.com> <764e3f2d-75a8-143d-d29f-01ea44d0b6e0@oracle.com> <5CA977F6-2B49-4E47-B5BD-E4EA68F361FA@oracle.com> Message-ID: <1712b851-e835-cb4b-e1e4-ad91afbe8964@oracle.com> An HTML attachment was scrubbed... URL: From linzang at tencent.com Wed Jul 29 12:01:30 2020 From: linzang at tencent.com (=?utf-8?B?bGluemFuZyjoh6fnkLMp?=) Date: Wed, 29 Jul 2020 12:01:30 +0000 Subject: RFR(L): 8215624: add parallel heap inspection support for jmap histo(G1)(Internet mail) In-Reply-To: <2C4A6B09-F281-433E-95CE-9587FC3C769F@tencent.com> References: <2C4A6B09-F281-433E-95CE-9587FC3C769F@tencent.com> Message-ID: Upload a new change at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_10/ It fix an issue of windows fail : #################################### In heapInspect.cpp - size_t HeapInspection::populate_table(KlassInfoTable* cit, BoolObjectClosure *filter, uint parallel_thread_num) { + uint HeapInspection::populate_table(KlassInfoTable* cit, BoolObjectClosure *filter, uint parallel_thread_num) { #################################### In heapInspect.hpp - size_t populate_table(KlassInfoTable* cit, BoolObjectClosure* filter = NULL, uint parallel_thread_num = 1) NOT_SERVICES_RETURN_(0); + uint populate_table(KlassInfoTable* cit, BoolObjectClosure* filter = NULL, uint parallel_thread_num = 1) NOT_SERVICES_RETURN_(0); #################################### BRs, Lin ?On 2020/7/27, 11:26 AM, "linzang(??)" wrote: I update a new change at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_09 It includes a tiny fix of build failure on windows: #################################### In attachListener.cpp: - uint parallel_thread_num = MAX(1, (uint)os::initial_active_processor_count() * 3 / 8); + uint parallel_thread_num = MAX2(1, (uint)os::initial_active_processor_count() * 3 / 8); #################################### BRs, Lin On 2020/7/23, 11:56 AM, "linzang(??)" wrote: Hi Paul, Thanks for your help, that all looks good to me. Just 2 minor changes: ? delete the final return in ParHeapInspectTask::work, you mentioned it but seems not include in the webrev :-) ? delete a unnecessary blank line in heapInspect.cpp at merge_entry() ######################################################################### --- old/src/hotspot/share/memory/heapInspection.cpp 2020-07-23 11:23:29.281666456 +0800 +++ new/src/hotspot/share/memory/heapInspection.cpp 2020-07-23 11:23:29.017666447 +0800 @@ -251,7 +251,6 @@ _size_of_instances_in_words += cie->words(); return true; } - return false; } @@ -568,7 +567,6 @@ Atomic::add(&_missed_count, missed_count); } else { Atomic::store(&_success, false); - return; } } ######################################################################### Here is the webrev http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_08/ BRs, Lin --------------------------------------------- From: "Hohensee, Paul" Date: Thursday, July 23, 2020 at 6:48 AM To: "linzang(??)" , Stefan Karlsson , "serguei.spitsyn at oracle.com" , David Holmes , serviceability-dev , "hotspot-gc-dev at openjdk.java.net" Subject: RE: RFR(L): 8215624: add parallel heap inspection support for jmap histo(G1)(Internet mail) Just small things. heapInspection.cpp: In ParHeapInspectTask::work, remove the final return statement and fix the following ?}? indent. I.e., replace + Atomic::store(&_success, false); + return; + } with + Atomic::store(&_success, false); + } In HeapInspection::heap_inspection, missed_count should be a uint to match other missed_count declarations, and should be initialized to the result of populate_table() rather than separately to 0. attachListener.cpp: In heap_inspection, initial_processor_count returns an int, so cast its result to a uint. Similarly, parse_uintx returns a uintx, so cast its result (num) to uint when assigning to parallel_thread_num. BasicJMapTest.java: I took the liberty of refactoring testHisto*/histoToFile/testDump*/dump to remove redundant interposition methods and make histoToFile and dump look as similar as possible. Webrev with the above changes in http://cr.openjdk.java.net/~phh/8214535/webrev.01/ Thanks, Paul On 7/15/20, 2:13 AM, "linzang(??)" wrote: Upload a new webrev at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_07/ It fix a potential issue that unexpected number of threads maybe calculated for "parallel" option of jmap -histo in container. As shown at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_07-delta/src/hotspot/share/services/attachListener.cpp.udiff.html ############### attachListener.cpp #################### @@ -252,11 +252,11 @@ static jint heap_inspection(AttachOperation* op, outputStream* out) { bool live_objects_only = true; // default is true to retain the behavior before this change is made outputStream* os = out; // if path not specified or path is NULL, use out fileStream* fs = NULL; const char* arg0 = op->arg(0); - uint parallel_thread_num = MAX(1, os::processor_count() * 3 / 8); // default is less than half of processors. + uint parallel_thread_num = MAX(1, os::initial_active_processor_count() * 3 / 8); // default is less than half of processors. if (arg0 != NULL && (strlen(arg0) > 0)) { if (strcmp(arg0, "-all") != 0 && strcmp(arg0, "-live") != 0) { out->print_cr("Invalid argument to inspectheap operation: %s", arg0); return JNI_ERR; } ################################################### Thanks. BRs, Lin On 2020/7/9, 3:22 PM, "linzang(??)" wrote: Hi Paul, Thanks for reviewing! >> >> I'd move all the argument parsing code to JMap.java and just pass the results to Hotspot. Both histo() in JMap.java and code in attachListener.* parse the command line arguments, though the code in histo() doesn't parse the argument to "parallel". I'd upgrade the code in histo() to do a complete parse and pass the option values to executeCommandForPid as before: there would just be more of them now. That would allow you to eliminate all the parsing code in attachListener.cpp as well as the change to arguments.hpp. >> The reason I made the change in Jmap.java that compose all arguments as 1 string , instead of passing 3 argments, is to avoid the compatibility issue, as we discussed in http://mail.openjdk.java.net/pipermail/serviceability-dev/2019-February/thread.html#27240. The root cause of the compatibility issue is because max argument count in HotspotVirtualMachineImpl.java and attachlistener.cpp need to be enlarged (changes like http://hg.openjdk.java.net/jdk/jdk/rev/e7cf035682e3#l2.1) when jmap has more than 3 arguments. But if user use an old jcmd/jmap tool, it may stuck at socket read(), because the "max argument count" don't match. I re-checked this change, the argument count of jmap histo is equal to 3 (live, file, parallel), so it can work normally even without the change of passing argument. But I think we have to face the problem if more arguments is added in jcmd alike tools later, not sure whether it should be sloved (or a workaround) in this changeset. And here are the lastest webrev and delta: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_06/ http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_06-delta/ Cheers, Lin On 2020/7/7, 5:57 AM, "Hohensee, Paul" wrote: I'd like to see this feature added. :) The CSR looks good, as does the basic parallel inspection algorithm. Stefan's done the GC part, so I'll stick to the non-GC part (fwiw, the GC part lgtm). I'd move all the argument parsing code to JMap.java and just pass the results to Hotspot. Both histo() in JMap.java and code in attachListener.* parse the command line arguments, though the code in histo() doesn't parse the argument to "parallel". I'd upgrade the code in histo() to do a complete parse and pass the option values to executeCommandForPid as before: there would just be more of them now. That would allow you to eliminate all the parsing code in attachListener.cpp as well as the change to arguments.hpp. heapInspection.hpp: _shared_miss_count (s/b _missed_count, see below) isn't a size, so it should be a uint instead of a size_t. Same with the new parallel_thread_num argument to heap_inspection() and populate_table(). Comment copy-edit: +// Parallel heap inspection task. Parallel inspection can fail due to +// a native OOM when allocating memory for TL-KlassInfoTable. +// _success will be set false on an OOM, and serial inspection tried. _shared_miss_count should be _missed_count to match the missed_count() getter, or rename missed_count() to be shared_miss_count(). Whichever way you go, the field type should match the getter result type: uint is reasonable. heapInspection.cpp: You might use ResourceMark twice in populate_table, separately for the parallel attempt and the serial code. If the parallel attempt fails and available memory is low, it would be good to clean up the memory used by the parallel attempt before doing the serial code. Style nit in KlassInfoTable::merge_entry(). I'd line up the definitions of k and elt, so "k" is even with "elt". And, because it's two lines shorter, I'd replace + } else { + return false; + } with + return false; KlassInfoTableMergeClosure.is_success() should be just success() (i.e., no "is_" prefix) because it's a getter. I'd reorganize the code in populate_table() to make it more clear, vis (I changed _shared_missed_count to _missed_count) + if (cit.allocation_failed()) { + // fail to allocate memory, stop parallel mode + Atomic::store(&_success, false); + return; + } + RecordInstanceClosure ric(&cit, _filter); + _poi->object_iterate(&ric, worker_id); + missed_count = ric.missed_count(); + { + MutexLocker x(&_mutex); + merge_success = _shared_cit->merge(&cit); + } + if (merge_success) { + Atomic::add(&_missed_count, missed_count); + else { + Atomic::store(&_success, false); + } Thanks, Paul On 6/29/20, 7:20 PM, "linzang(??)" wrote: Dear All, Sorry to bother again, I just want to make sure that is this change worth to be continue to work on? If decision is made to not. I think I can drop this work and stop asking for help reviewing... Thanks for all your help about reviewing this previously. BRs, Lin On 2020/5/9, 3:47 PM, "linzang(??)" wrote: Dear All, May I ask your help again for review the latest change? Thanks! BRs, Lin On 2020/4/28, 1:54 PM, "linzang(??)" wrote: Hi Stefan, >> - Adding Atomic::load/store. >> - Removing the time measurement in the run_task. I renamed G1's function >> to run_task_timed. If we need this outside of G1, we can rethink the API >> at that point. >> - ZGC style cleanups Thanks for revising the patch, they are all good to me, and I have made a tiny change based on it: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04/ http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04-delta/ it reduce the scope of mutex in ParHeapInspectTask, and delete unnecessary comments. BRs, Lin On 2020/4/27, 4:34 PM, "Stefan Karlsson" wrote: Hi Lin, On 2020-04-26 05:10, linzang(??) wrote: > Hi Stefan and Paul? > I have made a new patch based on your comments and Stefan's Poc code: > Webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03/ > Delta(based on Stefan's change:) : http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03-delta/webrev_03-delta/ Thanks for providing a delta patch. It makes it much easier to look at, and more likely for reviewers to continue reviewing. I'm going to continue focusing on the GC parts, and leave the rest to others to review. > > And Here are main changed I made and want to discuss with you: > 1. changed"parallelThreadNum=" to "parallel=" for jmap -histo options. > 2. Add logic to test where parallelHeapInspection is fail, in heapInspection.cpp > This is because the parHeapInspectTask create thread local KlassInfoTable in it's work() method, and this may fail because of native OOM, in this case, the parallel should fail and serial heap inspection can be tried. > One more thing I want discuss with you is about the member "_success" of parHeapInspectTask, when native OOM happenes, it is set to false. And since this "set" operation can be conducted in multiple threads, should it be atomic ops? IMO, this is not necessary because "_success" can only be set to false, and there is no way to change it from back to true after the ParHeapInspectTask instance is created, so it is save to be non-atomic, do you agree with that? In these situations you should be using the Atomic::load/store primitives. We're moving toward a later C++ standard were data races are considered undefined behavior. > 3. make CollectedHeap::run_task() be an abstract virtual func, so that every subclass of collectedHeap should support it, so later implementation of new collectedHeap will not miss the "parallel" features. > The problem I want to discuss with you is about epsilonHeap and SerialHeap, as they may not need parallel heap iteration, so I only make task->work(0), in case the run_task() is invoked someway in future. Another way is to left run_task() unimplemented, which one do you think is better? I don't have a strong opinion about this. And also please help take a look at the zHeap, as there is a class zTask that wrap the abstractGangTask, and the collectedHeap::run_task() only accept AbstraceGangTask* as argument, so I made a delegate class to adapt it , please see src/hotspot/share/gc/z/zHeap.cpp. > > There maybe other better ways to sovle the above problems, welcome for any comments, Thanks! I've created a few cleanups and changes on top of your latest patch: https://cr.openjdk.java.net/~stefank/8215624/webrev.02.delta https://cr.openjdk.java.net/~stefank/8215624/webrev.02 - Adding Atomic::load/store. - Removing the time measurement in the run_task. I renamed G1's function to run_task_timed. If we need this outside of G1, we can rethink the API at that point. - ZGC style cleanups Thanks, StefanK > > BRs, > Lin > > On 2020/4/23, 11:08 AM, "linzang(??)" wrote: > > Thanks Paul! I agree with using "parallel", will make the update in next patch, Thanks for help update the CSR. > > BRs, > Lin > > On 2020/4/23, 4:42 AM, "Hohensee, Paul" wrote: > > For the interface, I'd use "parallel" instead of "parallelThreadNum". All the other options are lower case, and it's a lot easier to type "parallel". I took the liberty of updating the CSR. If you're ok with it, you might want to change variable names and such, plus of course JMap.usage. > > Thanks, > Paul > > On 4/22/20, 2:29 AM, "serviceability-dev on behalf of linzang(??)" wrote: > > Dear Stefan, > > Thanks a lot! I agree with you to decouple the heap inspection code with GC's. > I will start from your POC code, may discuss with you later. > > > BRs, > Lin > > On 2020/4/22, 5:14 PM, "Stefan Karlsson" wrote: > > Hi Lin, > > I took a look at this earlier and saw that the heap inspection code is > strongly coupled with the CollectedHeap and G1CollectedHeap. I'd prefer > if we'd abstract this away, so that the GCs only provide a "parallel > object iteration" interface, and the heap inspection code is kept elsewhere. > > I started experimenting with doing that, but other higher-priority (to > me) tasks have had to take precedence. > > I've uploaded my work-in-progress / proof-of-concept: > https://cr.openjdk.java.net/~stefank/8215624/webrev.01.delta/ > https://cr.openjdk.java.net/~stefank/8215624/webrev.01/ > > The current code doesn't handle the lifecycle (deletion) of the > ParallelObjectIterators. There's also code left unimplemented in around > CollectedHeap::run_task. However, I think this could work as a basis to > pull out the heap inspection code out of the GCs. > > Thanks, > StefanK > > On 2020-04-22 02:21, linzang(??) wrote: > > Dear all, > > May I ask you help to review? This RFR has been there for quite a while. > > Thanks! > > > > BRs, > > Lin > > > > > On 2020/3/16, 5:18 PM, "linzang(??)" wrote:> > > > >> Just update a new path, my preliminary measure show about 3.5x speedup of jmap histo on a nearly full 4GB G1 heap (8-core platform with parallel thread number set to 4). > >> webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_02/ > >> bug: https://bugs.openjdk.java.net/browse/JDK-8215624 > >> CSR: https://bugs.openjdk.java.net/browse/JDK-8239290 > >> BRs, > >> Lin > >> > On 2020/3/2, 9:56 PM, "linzang(??)" wrote: > >> > > >> > Dear all, > >> > Let me try to ease the reviewing work by some explanation :P > >> > The patch's target is to speed up jmap -histo for heap iteration, from my experience it is necessary for large heap investigation. E.g in bigData scenario I have tried to conduct jmap -histo against 180GB heap, it does take quite a while. > >> > And if my understanding is corrent, even the jmap -histo without "live" option does heap inspection with heap lock acquired. so it is very likely to block mutator thread in allocation-sensitive scenario. I would say the faster the heap inspection does, the shorter the mutator be blocked. This is parallel iteration for jmap is necessary. > >> > I think the parallel heap inspection should be applied to all kind of heap. However, consider the heap layout are different for GCs, much time is required to understand all kinds of the heap layout to make the whole change. IMO, It is not wise to have a huge patch for the whole solution at once, and it is even harder to review it. So I plan to implement it incrementally, the first patch (this one) is going to confirm the implemention detail of how jmap accept the new option, passes it to attachListener of the jvm process and then how to make the parallel inspection closure be generic enough to make it easy to extend to different heap layout. And also how to implement the heap inspection in specific gc's heap. This patch use G1's heap as the begining. > >> > This patch actually do several things: > >> > 1. Add an option "parallelThreadNum=" to jmap -histo, the default behavior is to set N to 0, means let's JVM decide how many threads to use for heap inspection. Set this option to 1 will disable parallel heap inspection. (more details in CSR: https://bugs.openjdk.java.net/browse/JDK-8239290) > >> > 2. Make a change in how Jmap passing arguments, changes in http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java.udiff.html, originally it pass options as separate arguments to attachListener, this patch change to that all options be compose to a single string. So the arg_count_max in attachListener.hpp do not need to be changed, and hence avoid the compatibility issue, as disscussed at https://mail.openjdk.java.net/pipermail/serviceability-dev/2019-March/027334.html > >> > 3. Add an abstract class ParHeapInspectTask in heapInspection.hpp / heapInspection.cpp, It's work(uint worker_id) method prepares the data structure (KlassInfoTable) need for every parallel worker thread, and then call do_object_iterate_parallel() which is heap specific implementation. I also added some machenism in KlassInfoTable to support parallel iteration, such as merge(). > >> > 4. In specific heap (G1 in this patch), create a subclass of ParHeapInspectTask, implement the do_object_iterate_parallel() for parallel heap inspection. For G1, it simply invoke g1CollectedHeap's object_iterate_parallel(). > >> > 5. Add related test. > >> > 6. it may be easy to extend this patch for other kinds of heap by creating subclass of ParHeapInspectTask and implement the do_object_iterate_parallel(). > >> > > >> > Hope these info could help on code review and initate the discussion :-) > >> > Thanks! > >> > > >> > BRs, > >> > Lin > >> > >On 2020/2/19, 9:40 AM, "linzang(??)" wrote:. > >> > > > >> > > Re-post this RFR with correct enhancement number to make it trackable. > >> > > please ignore the previous wrong post. sorry for troubles. > >> > > > >> > > webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/ > >> > > Hi bug: https://bugs.openjdk.java.net/browse/JDK-8215624 > >> > > CSR: https://bugs.openjdk.java.net/browse/JDK-8239290 > >> > > -------------- > >> > > Lin > >> > > >Hi Lin, > > > > > > > >> > > >Could you, please, re-post your RFR with the right enhancement number in > >> > > >the message subject? > >> > > >It will be more trackable this way. > >> > > > > >> > > >Thanks, > >> > > >Serguei > >> > > > > >> > > > > >> > > >On 2/17/20 10:29 PM, linzang(??) wrote: > >> > > >> Dear David, > >> > > >> Thanks a lot! > >> > > >> I have updated the refined code to http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_01/. > >> > > >> IMHO the parallel heap inspection can be extended to all kinds of heap as long as the heap layout can support parallel iteration. > >> > > >> Maybe we can firstly use this webrev to discuss how to implement it, because I am not sure my current implementation is an appropriate way to communicate with collectedHeap, then we can extend the solution to other kinds of heap. > >> > > >> > >> > > >> Thanks, > >> > > >> -------------- > >> > > >> Lin > >> > > >>> Hi Lin, > >> > > >>> > >> > > >>> Adding in hotspot-gc-dev as they need to see how this interacts with GC > >> > > >>> worker threads, and whether it needs to be extended beyond G1. > >> > > >>> > >> > > >>> I happened to spot one nit when browsing: > >> > > >>> > >> > > >>> src/hotspot/share/gc/shared/collectedHeap.hpp > >> > > >>> > >> > > >>> + virtual bool run_par_heap_inspect_task(KlassInfoTable* cit, > >> > > >>> + BoolObjectClosure* filter, > >> > > >>> + size_t* missed_count, > >> > > >>> + size_t thread_num) { > >> > > >>> + return NULL; > >> > > >>> > >> > > >>> s/NULL/false/ > >> > > >>> > >> > > >>> Cheers, > >> > > >>> David > > > > > >>> > >> > > >>> On 18/02/2020 2:15 pm, linzang(??) wrote: > >> > > >>>> Dear All, > >> > > >>>> May I ask your help to review the follow changes: > >> > > >>>> webrev: > >> > > >>>> http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_00/ > >> > > >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8215624 > >> > > >>>> related CSR: https://bugs.openjdk.java.net/browse/JDK-8239290 > >> > > >>>> This patch enable parallel heap inspection of G1 for jmap histo. > >> > > >>>> my simple test shown it can speed up 2x of jmap -histo with > >> > > >>>> parallelThreadNum set to 2 for heap at ~500M on 4-core platform. > >> > > >>>> > >> > > >>>> ------------------------------------------------------------------------ > >> > > >>>> BRs, > >> > > >>>> Lin > >> > > >> > > >> > > > > > > > > > > > > > > > From leonid.mesnik at oracle.com Wed Jul 29 17:41:26 2020 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Wed, 29 Jul 2020 10:41:26 -0700 Subject: RFR: 8244537: JDI tests fail due to "ERROR: Exception : nsk.share.jdi.JDITestRuntimeException: JDITestRuntimeException : ** event IS NOT a breakpoint **" In-Reply-To: <1712b851-e835-cb4b-e1e4-ad91afbe8964@oracle.com> References: <16EB9126-382A-4092-BB61-14E9C3CF208C@oracle.com> <68FBDE47-11DA-45C3-AD07-8200DCC9BA29@oracle.com> <5d78a07f-6374-7edb-c57c-8a34faf4d840@oracle.com> <4CF129DD-1733-47A0-91F9-B27891BCC63E@oracle.com> <11e29ad2-8e06-ad30-df48-6dcfea0bd4dd@oracle.com> <764e3f2d-75a8-143d-d29f-01ea44d0b6e0@oracle.com> <5CA977F6-2B49-4E47-B5BD-E4EA68F361FA@oracle.com> <1712b851-e835-cb4b-e1e4-ad91afbe8964@oracle.com> Message-ID: <0F59DEE6-1065-47CA-ACB2-003068D5516F@oracle.com> Thank you for review and noticing issues. Still waiting for a second review. Leonid > On Jul 29, 2020, at 12:08 AM, serguei.spitsyn at oracle.com wrote: > > It looks good. > Thank you for the update! > > Thanks, > Serguei > > > On 7/28/20 23:26, Leonid Mesnik wrote: >> ok, let change it to following >> >> diff -r f489d5d13a51 test/hotspot/jtreg/vmTestbase/nsk/jdi/EventRequest/setEnabled/setenabled002.java >> --- a/test/hotspot/jtreg/vmTestbase/nsk/jdi/EventRequest/setEnabled/setenabled002.java Thu Jul 23 16:36:44 2020 -0400 >> +++ b/test/hotspot/jtreg/vmTestbase/nsk/jdi/EventRequest/setEnabled/setenabled002.java Tue Jul 28 23:17:22 2020 -0700 >> @@ -368,6 +368,7 @@ >> throw new JDITestRuntimeException("** default case 2 **"); >> } >> >> + vm.suspend(); >> if (eventRequest1 instanceof StepRequest) { >> try { >> log2("......eventRequest1.setEnabled(true); IllegalThreadStateException is expected"); >> @@ -405,6 +406,7 @@ >> } catch ( InvalidRequestStateException e ) { >> log2(" InvalidRequestStateException"); >> } >> + vm.resume(); >> >> //~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> } >> >> webrev: http://cr.openjdk.java.net/~lmesnik/8244537/webrev.04/ >> Leonid >> >>> On Jul 28, 2020, at 11:06 PM, serguei.spitsyn at oracle.com wrote: >>> >>> http://cr.openjdk.java.net/~lmesnik/8244537/webrev.03/test/hotspot/jtreg/vmTestbase/nsk/jdi/EventRequest/setEnabled/setenabled002.java.frames.html >>> >>> I'd suggest to simplify it: >>> >>> - insert suspend before the line: >>> 371 if (eventRequest1 instanceof StepRequest) { >>> - keep resume at the line: >>> 411 vm.resume(); >>> - these lines are not needed: >>> 389 vm.resume(); >>> 394 vm.suspend(); >>> >>> Thanks, >>> Serguei >>> >>> >>> On 7/28/20 21:15, Leonid Mesnik wrote: >>>> Included in webrev.03 >>>> http://cr.openjdk.java.net/~lmesnik/8244537/webrev.03 >>>> >>>> Leonid >>>> >>>>> On Jul 28, 2020, at 6:44 PM, serguei.spitsyn at oracle.com wrote: >>>>> >>>>> http://cr.openjdk.java.net/~lmesnik/8244537/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jdi/EventRequest/setEnabled/setenabled002.java.frames.html >>>>> 371 if (eventRequest1 instanceof StepRequest) { >>>>> 372 try { >>>>> 373 log2("......eventRequest1.setEnabled(true); IllegalThreadStateException is expected"); >>>>> 374 eventRequest1.setEnabled(true); >>>>> 375 testExitCode = FAILED; >>>>> 376 log3("ERROR: NO IllegalThreadStateException for StepRequest"); >>>>> 377 } catch ( IllegalThreadStateException e ) { >>>>> 378 log2(" IllegalThreadStateException"); >>>>> 379 } >>>>> 380 try { >>>>> 381 log2("......eventRequest1.setEnabled(false); IllegalThreadStateException is not expected"); >>>>> 382 eventRequest1.setEnabled(false); >>>>> 383 log2(" no IllegalThreadStateException for StepRequest"); >>>>> 384 } catch ( IllegalThreadStateException e ) { >>>>> 385 testExitCode = FAILED; >>>>> 386 log3("ERROR: IllegalThreadStateException"); >>>>> 387 } >>>>> 388 } >>>>> Above is one more case where suspend/resume is needed, I guess. >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>> On 7/28/20 17:45, serguei.spitsyn at oracle.com wrote: >>>>>> http://cr.openjdk.java.net/~lmesnik/8244537/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jdi/EventRequest/setEnabled/setenabled003.java.frames.html >>>>>> 288 case 0: >>>>>> 289 thread1 = (ThreadReference) >>>>>> 290 debuggeeClass.getValue(debuggeeClass.fieldByName(threadName1)); >>>>>> 291 >>>>>> 292 log2("......setting up StepRequest"); >>>>>> 293 eventRequest1 = eventRManager.createStepRequest >>>>>> 294 (thread1, StepRequest.STEP_MIN, StepRequest.STEP_INTO); >>>>>> 295 >>>>>> 296 vm.suspend(); >>>>>> ... >>>>>> 360 default: >>>>>> 361 throw new JDITestRuntimeException("** default case 2 **"); >>>>>> 362 } >>>>>> 363 vm.resume(); >>>>>> >>>>>> Sorry, the fix is not going to work correctly. >>>>>> The first vm.suspend() has to be before the switch statement to work for all 3 cases. >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>>> >>>>>> On 7/28/20 16:49, Leonid Mesnik wrote: >>>>>>> I've update to suspend/resume in all cases. >>>>>>> >>>>>>> new webrev: http://cr.openjdk.java.net/~lmesnik/8244537/webrev.01/ >>>>>>> >>>>>>> Leonid >>>>>>> >>>>>>>> On Jul 28, 2020, at 2:06 PM, serguei.spitsyn at oracle.com wrote: >>>>>>>> >>>>>>>> I prefer to suspend/resume in all cases, so we avoid all these unexpected failures. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Serguei >>>>>>>> >>>>>>>> >>>>>>>> On 7/28/20 13:59, Leonid Mesnik wrote: >>>>>>>>> It should be failure anyway if we managed to enable events, so we don't expect to really enable anything in these cases. >>>>>>>>> However I agree that adding suspend/resume shouldn't make it worse, just possible cleaner log (in very rare cases also). If you feel it is need I will just add suspension for all cases. >>>>>>>>> >>>>>>>>> Leonid >>>>>>>>> >>>>>>>>>> On Jul 28, 2020, at 1:54 PM, serguei.spitsyn at oracle.com wrote: >>>>>>>>>> >>>>>>>>>> Does it mean, you did not fix cases 0 and 2 because the related failures have never been observed? >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Serguei >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 7/28/20 13:51, Leonid Mesnik wrote: >>>>>>>>>>> Test should fail in cases 0 and 2 with IllegalThreadStateException if we can enable events. Such failures should be easily identified by reading logs. >>>>>>>>>>> >>>>>>>>>>> Leonid >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> On Jul 27, 2020, at 10:28 PM, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi Leonid, >>>>>>>>>>>> >>>>>>>>>>>> The fix looks good in general. >>>>>>>>>>>> You missed to explain that the suspend/resume are added to avoid actual generation of event that cause this issue. >>>>>>>>>>>> The reason is that these events are not actually required. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~lmesnik/8244537/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/jdi/EventRequest/setEnabled/setenabled003.java.frames.html >>>>>>>>>>>> 316 case 1: >>>>>>>>>>>> 317 vm.suspend(); >>>>>>>>>>>> ... >>>>>>>>>>>> 336 vm.resume(); >>>>>>>>>>>> >>>>>>>>>>>> Q: Why is only in case 1 suspend/resume used? >>>>>>>>>>>> What about cases 0 and 2? >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Serguei >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 7/27/20 18:08, Leonid Mesnik wrote: >>>>>>>>>>>>> Hi >>>>>>>>>>>>> >>>>>>>>>>>>> Could you please review following fix which suspends debugger VM while enabling/disabling events. >>>>>>>>>>>>> >>>>>>>>>>>>> All changed tests fail intermittently getting unexpected events instead of breakpoint used for communication between debugger/debuggee VM. The tests request different events and verify request's properties but don't process/verify events themselves. Test doesn't aware if events are generated or not. The vm suspension doesn't affect JDWP native agent and it still should get and verify JDWP commands. >>>>>>>>>>>>> >>>>>>>>>>>>> webrev: http://cr.openjdk.java.net/~lmesnik/8244537/webrev.00/ >>>>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8244537 >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Leonid >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sgehwolf at redhat.com Wed Jul 29 19:21:37 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Wed, 29 Jul 2020 21:21:37 +0200 Subject: [PING?] RFR(s): 8247863: Unreachable code in OperatingSystemImpl.getTotalSwapSpaceSize() In-Reply-To: References: <40bd7c49d36c5775d86b9d2ff5740b50cf4359e9.camel@redhat.com> Message-ID: Hi Matthias, On Wed, 2020-07-15 at 15:22 +0000, Baesken, Matthias wrote: > Hello Severin , > > > the new cgroups implementation > > supporting v1 and v2 Metrics.getMemoryAndSwapLimit() will never return 0 > > Wouldn?t it be possible that the coding of getMemoryAndSwapLimit returns a negative value (might not happen on a "healthy" system but you never know) : In short, no, unless the value is actually unlimited. With the caveat that we cannot distinguish between "kernel not supporting swap" and "unlimited" swap with a potential cgroup memory.memsw.limit_in_bytes file missing. The premise is that negative values are not possible in cgroup interface files. So the only valid reason to return negative (-1, to be precise) is if the interface files aren't there or the values in those files are beyond a threshold indicating "unlimited". Let's look at this in detail: > jdk/src/java.base/linux/classes/jdk/internal/platform/cgroupv1/CgroupV1Subsystem.java > > 444 public long getMemoryAndSwapLimit() { > 445 long retval = getLongValue(memory, "memory.memsw.limit_in_bytes"); getLongValue() will not return a negative value unless an interface file is missing, a string in the cgroup files is empty, or a very large number is observed (which maps to unlimited, a.k.a non-container values win). > 446 if (retval > CgroupV1SubsystemController.UNLIMITED_MIN) { > 447 if (memory.isHierarchical()) { > 448 // memory.memsw.limit_in_bytes returned unlimited, attempt > 449 // hierarchical memory limit > 450 String match = "hierarchical_memsw_limit"; > 451 retval = CgroupV1SubsystemController.getLongValueMatchingLine(memory, > 452 "memory.stat", > 453 match); Same for this. > 454 } > 455 } > 456 return CgroupV1SubsystemController.longValOrUnlimited(retval); At this point we map any large value to -1, unlimited, or return 'retval'. > 457 } > > > jdk/src/java.base/linux/classes/jdk/internal/platform/cgroupv2/CgroupV2Subsystem.java > > 278 public long getMemoryAndSwapLimit() { > 279 String strVal = CgroupSubsystemController.getStringValue(unified, "memory.swap.max"); This will either return null, "MAX" or the actual value. > 280 return limitFromString(strVal); This maps MAX and null to -1 or the actual numberical value as string to its representation as long. > 281 } > > So the check you want to clean up does no harm, and might handle "strange" cases - why not keep it? The comment in this block is now misleading: if (limit >= 0 && memLimit >= 0) { // we see a limit == 0 on some machines where "kernel does not support swap limit capabilities" return (limit < memLimit) ? 0 : limit - memLimit; } The only reason why a 0 was observed, was because the cgroup interface files were missing and the old code mapped that to a 0. That's no longer the case and, thus, it seems it would make the code clearer if it wouldn't be there any more. I don't feel strongly about this, though, and can just drop this patch. The fix of JDK-8236617 has been superseded by JDK-8244500. Thanks, Severin > > > Best regards, Matthias > > > -----Original Message----- > From: Severin Gehwolf > Sent: Mittwoch, 15. Juli 2020 11:47 > To: core-libs-dev ; serviceability-dev > Cc: Baesken, Matthias ; Bob Vandette > Subject: Re: [PING?] RFR(s): 8247863: Unreachable code in OperatingSystemImpl.getTotalSwapSpaceSize() > > Anyone? > > On Mon, 2020-06-29 at 17:53 +0200, Severin Gehwolf wrote: > > Hi, > > > > Could I please get a review of this dead-code removal? During review of > > JDK-8244500 it was discovered that with the new cgroups implementation > > supporting v1 and v2 Metrics.getMemoryAndSwapLimit() will never return > > 0 when relevant cgroup files are missing. E.g. on a system where the > > kernel doesn't support swap limit capabilities. Therefore this code > > introduced with JDK-8236617 can no longer be reached and should get > > removed. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8247863 > > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8247863/01/webrev/ > > > > Testing: Matthias tested this on the affected system and it did pass > > for him. Docker tests on cgroup v1 and cgroup v2. > > > > Thanks, > > Severin From chris.plummer at oracle.com Wed Jul 29 20:08:25 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 29 Jul 2020 13:08:25 -0700 Subject: RFR(S): 8250750: JDK-8247515 fix for OSX pc_to_symbol() lookup fails with some symbols Message-ID: <33d6d75e-e9f0-65c2-fa56-b1dc2f2be223@oracle.com> Hello, Please help review the following: https://bugs.openjdk.java.net/browse/JDK-8250750 http://cr.openjdk.java.net/~cjplummer/8250750/webrev.00/index.html Details are in the CR description. thanks, Chris From hohensee at amazon.com Wed Jul 29 21:21:19 2020 From: hohensee at amazon.com (Hohensee, Paul) Date: Wed, 29 Jul 2020 21:21:19 +0000 Subject: RFR(L): 8215624: add parallel heap inspection support for jmap histo(G1)(Internet mail) Message-ID: <01181077-0DD7-4AEE-945B-5D6E21224A31@amazon.com> A submit repo run with this succeeded, so afaic you're good to go. Stefan, you reviewed the GC part before, it'd be great if you could ok the final version. Thanks, Paul ?On 7/29/20, 5:02 AM, "linzang(??)" wrote: Upload a new change at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_10/ It fix an issue of windows fail : #################################### In heapInspect.cpp - size_t HeapInspection::populate_table(KlassInfoTable* cit, BoolObjectClosure *filter, uint parallel_thread_num) { + uint HeapInspection::populate_table(KlassInfoTable* cit, BoolObjectClosure *filter, uint parallel_thread_num) { #################################### In heapInspect.hpp - size_t populate_table(KlassInfoTable* cit, BoolObjectClosure* filter = NULL, uint parallel_thread_num = 1) NOT_SERVICES_RETURN_(0); + uint populate_table(KlassInfoTable* cit, BoolObjectClosure* filter = NULL, uint parallel_thread_num = 1) NOT_SERVICES_RETURN_(0); #################################### BRs, Lin On 2020/7/27, 11:26 AM, "linzang(??)" wrote: I update a new change at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_09 It includes a tiny fix of build failure on windows: #################################### In attachListener.cpp: - uint parallel_thread_num = MAX(1, (uint)os::initial_active_processor_count() * 3 / 8); + uint parallel_thread_num = MAX2(1, (uint)os::initial_active_processor_count() * 3 / 8); #################################### BRs, Lin On 2020/7/23, 11:56 AM, "linzang(??)" wrote: Hi Paul, Thanks for your help, that all looks good to me. Just 2 minor changes: ? delete the final return in ParHeapInspectTask::work, you mentioned it but seems not include in the webrev :-) ? delete a unnecessary blank line in heapInspect.cpp at merge_entry() ######################################################################### --- old/src/hotspot/share/memory/heapInspection.cpp 2020-07-23 11:23:29.281666456 +0800 +++ new/src/hotspot/share/memory/heapInspection.cpp 2020-07-23 11:23:29.017666447 +0800 @@ -251,7 +251,6 @@ _size_of_instances_in_words += cie->words(); return true; } - return false; } @@ -568,7 +567,6 @@ Atomic::add(&_missed_count, missed_count); } else { Atomic::store(&_success, false); - return; } } ######################################################################### Here is the webrev http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_08/ BRs, Lin --------------------------------------------- From: "Hohensee, Paul" Date: Thursday, July 23, 2020 at 6:48 AM To: "linzang(??)" , Stefan Karlsson , "serguei.spitsyn at oracle.com" , David Holmes , serviceability-dev , "hotspot-gc-dev at openjdk.java.net" Subject: RE: RFR(L): 8215624: add parallel heap inspection support for jmap histo(G1)(Internet mail) Just small things. heapInspection.cpp: In ParHeapInspectTask::work, remove the final return statement and fix the following ?}? indent. I.e., replace + Atomic::store(&_success, false); + return; + } with + Atomic::store(&_success, false); + } In HeapInspection::heap_inspection, missed_count should be a uint to match other missed_count declarations, and should be initialized to the result of populate_table() rather than separately to 0. attachListener.cpp: In heap_inspection, initial_processor_count returns an int, so cast its result to a uint. Similarly, parse_uintx returns a uintx, so cast its result (num) to uint when assigning to parallel_thread_num. BasicJMapTest.java: I took the liberty of refactoring testHisto*/histoToFile/testDump*/dump to remove redundant interposition methods and make histoToFile and dump look as similar as possible. Webrev with the above changes in http://cr.openjdk.java.net/~phh/8214535/webrev.01/ Thanks, Paul On 7/15/20, 2:13 AM, "linzang(??)" wrote: Upload a new webrev at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_07/ It fix a potential issue that unexpected number of threads maybe calculated for "parallel" option of jmap -histo in container. As shown at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_07-delta/src/hotspot/share/services/attachListener.cpp.udiff.html ############### attachListener.cpp #################### @@ -252,11 +252,11 @@ static jint heap_inspection(AttachOperation* op, outputStream* out) { bool live_objects_only = true; // default is true to retain the behavior before this change is made outputStream* os = out; // if path not specified or path is NULL, use out fileStream* fs = NULL; const char* arg0 = op->arg(0); - uint parallel_thread_num = MAX(1, os::processor_count() * 3 / 8); // default is less than half of processors. + uint parallel_thread_num = MAX(1, os::initial_active_processor_count() * 3 / 8); // default is less than half of processors. if (arg0 != NULL && (strlen(arg0) > 0)) { if (strcmp(arg0, "-all") != 0 && strcmp(arg0, "-live") != 0) { out->print_cr("Invalid argument to inspectheap operation: %s", arg0); return JNI_ERR; } ################################################### Thanks. BRs, Lin On 2020/7/9, 3:22 PM, "linzang(??)" wrote: Hi Paul, Thanks for reviewing! >> >> I'd move all the argument parsing code to JMap.java and just pass the results to Hotspot. Both histo() in JMap.java and code in attachListener.* parse the command line arguments, though the code in histo() doesn't parse the argument to "parallel". I'd upgrade the code in histo() to do a complete parse and pass the option values to executeCommandForPid as before: there would just be more of them now. That would allow you to eliminate all the parsing code in attachListener.cpp as well as the change to arguments.hpp. >> The reason I made the change in Jmap.java that compose all arguments as 1 string , instead of passing 3 argments, is to avoid the compatibility issue, as we discussed in http://mail.openjdk.java.net/pipermail/serviceability-dev/2019-February/thread.html#27240. The root cause of the compatibility issue is because max argument count in HotspotVirtualMachineImpl.java and attachlistener.cpp need to be enlarged (changes like http://hg.openjdk.java.net/jdk/jdk/rev/e7cf035682e3#l2.1) when jmap has more than 3 arguments. But if user use an old jcmd/jmap tool, it may stuck at socket read(), because the "max argument count" don't match. I re-checked this change, the argument count of jmap histo is equal to 3 (live, file, parallel), so it can work normally even without the change of passing argument. But I think we have to face the problem if more arguments is added in jcmd alike tools later, not sure whether it should be sloved (or a workaround) in this changeset. And here are the lastest webrev and delta: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_06/ http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_06-delta/ Cheers, Lin On 2020/7/7, 5:57 AM, "Hohensee, Paul" wrote: I'd like to see this feature added. :) The CSR looks good, as does the basic parallel inspection algorithm. Stefan's done the GC part, so I'll stick to the non-GC part (fwiw, the GC part lgtm). I'd move all the argument parsing code to JMap.java and just pass the results to Hotspot. Both histo() in JMap.java and code in attachListener.* parse the command line arguments, though the code in histo() doesn't parse the argument to "parallel". I'd upgrade the code in histo() to do a complete parse and pass the option values to executeCommandForPid as before: there would just be more of them now. That would allow you to eliminate all the parsing code in attachListener.cpp as well as the change to arguments.hpp. heapInspection.hpp: _shared_miss_count (s/b _missed_count, see below) isn't a size, so it should be a uint instead of a size_t. Same with the new parallel_thread_num argument to heap_inspection() and populate_table(). Comment copy-edit: +// Parallel heap inspection task. Parallel inspection can fail due to +// a native OOM when allocating memory for TL-KlassInfoTable. +// _success will be set false on an OOM, and serial inspection tried. _shared_miss_count should be _missed_count to match the missed_count() getter, or rename missed_count() to be shared_miss_count(). Whichever way you go, the field type should match the getter result type: uint is reasonable. heapInspection.cpp: You might use ResourceMark twice in populate_table, separately for the parallel attempt and the serial code. If the parallel attempt fails and available memory is low, it would be good to clean up the memory used by the parallel attempt before doing the serial code. Style nit in KlassInfoTable::merge_entry(). I'd line up the definitions of k and elt, so "k" is even with "elt". And, because it's two lines shorter, I'd replace + } else { + return false; + } with + return false; KlassInfoTableMergeClosure.is_success() should be just success() (i.e., no "is_" prefix) because it's a getter. I'd reorganize the code in populate_table() to make it more clear, vis (I changed _shared_missed_count to _missed_count) + if (cit.allocation_failed()) { + // fail to allocate memory, stop parallel mode + Atomic::store(&_success, false); + return; + } + RecordInstanceClosure ric(&cit, _filter); + _poi->object_iterate(&ric, worker_id); + missed_count = ric.missed_count(); + { + MutexLocker x(&_mutex); + merge_success = _shared_cit->merge(&cit); + } + if (merge_success) { + Atomic::add(&_missed_count, missed_count); + else { + Atomic::store(&_success, false); + } Thanks, Paul On 6/29/20, 7:20 PM, "linzang(??)" wrote: Dear All, Sorry to bother again, I just want to make sure that is this change worth to be continue to work on? If decision is made to not. I think I can drop this work and stop asking for help reviewing... Thanks for all your help about reviewing this previously. BRs, Lin On 2020/5/9, 3:47 PM, "linzang(??)" wrote: Dear All, May I ask your help again for review the latest change? Thanks! BRs, Lin On 2020/4/28, 1:54 PM, "linzang(??)" wrote: Hi Stefan, >> - Adding Atomic::load/store. >> - Removing the time measurement in the run_task. I renamed G1's function >> to run_task_timed. If we need this outside of G1, we can rethink the API >> at that point. >> - ZGC style cleanups Thanks for revising the patch, they are all good to me, and I have made a tiny change based on it: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04/ http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04-delta/ it reduce the scope of mutex in ParHeapInspectTask, and delete unnecessary comments. BRs, Lin On 2020/4/27, 4:34 PM, "Stefan Karlsson" wrote: Hi Lin, On 2020-04-26 05:10, linzang(??) wrote: > Hi Stefan and Paul? > I have made a new patch based on your comments and Stefan's Poc code: > Webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03/ > Delta(based on Stefan's change:) : http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03-delta/webrev_03-delta/ Thanks for providing a delta patch. It makes it much easier to look at, and more likely for reviewers to continue reviewing. I'm going to continue focusing on the GC parts, and leave the rest to others to review. > > And Here are main changed I made and want to discuss with you: > 1. changed"parallelThreadNum=" to "parallel=" for jmap -histo options. > 2. Add logic to test where parallelHeapInspection is fail, in heapInspection.cpp > This is because the parHeapInspectTask create thread local KlassInfoTable in it's work() method, and this may fail because of native OOM, in this case, the parallel should fail and serial heap inspection can be tried. > One more thing I want discuss with you is about the member "_success" of parHeapInspectTask, when native OOM happenes, it is set to false. And since this "set" operation can be conducted in multiple threads, should it be atomic ops? IMO, this is not necessary because "_success" can only be set to false, and there is no way to change it from back to true after the ParHeapInspectTask instance is created, so it is save to be non-atomic, do you agree with that? In these situations you should be using the Atomic::load/store primitives. We're moving toward a later C++ standard were data races are considered undefined behavior. > 3. make CollectedHeap::run_task() be an abstract virtual func, so that every subclass of collectedHeap should support it, so later implementation of new collectedHeap will not miss the "parallel" features. > The problem I want to discuss with you is about epsilonHeap and SerialHeap, as they may not need parallel heap iteration, so I only make task->work(0), in case the run_task() is invoked someway in future. Another way is to left run_task() unimplemented, which one do you think is better? I don't have a strong opinion about this. And also please help take a look at the zHeap, as there is a class zTask that wrap the abstractGangTask, and the collectedHeap::run_task() only accept AbstraceGangTask* as argument, so I made a delegate class to adapt it , please see src/hotspot/share/gc/z/zHeap.cpp. > > There maybe other better ways to sovle the above problems, welcome for any comments, Thanks! I've created a few cleanups and changes on top of your latest patch: https://cr.openjdk.java.net/~stefank/8215624/webrev.02.delta https://cr.openjdk.java.net/~stefank/8215624/webrev.02 - Adding Atomic::load/store. - Removing the time measurement in the run_task. I renamed G1's function to run_task_timed. If we need this outside of G1, we can rethink the API at that point. - ZGC style cleanups Thanks, StefanK > > BRs, > Lin > > On 2020/4/23, 11:08 AM, "linzang(??)" wrote: > > Thanks Paul! I agree with using "parallel", will make the update in next patch, Thanks for help update the CSR. > > BRs, > Lin > > On 2020/4/23, 4:42 AM, "Hohensee, Paul" wrote: > > For the interface, I'd use "parallel" instead of "parallelThreadNum". All the other options are lower case, and it's a lot easier to type "parallel". I took the liberty of updating the CSR. If you're ok with it, you might want to change variable names and such, plus of course JMap.usage. > > Thanks, > Paul > > On 4/22/20, 2:29 AM, "serviceability-dev on behalf of linzang(??)" wrote: > > Dear Stefan, > > Thanks a lot! I agree with you to decouple the heap inspection code with GC's. > I will start from your POC code, may discuss with you later. > > > BRs, > Lin > > On 2020/4/22, 5:14 PM, "Stefan Karlsson" wrote: > > Hi Lin, > > I took a look at this earlier and saw that the heap inspection code is > strongly coupled with the CollectedHeap and G1CollectedHeap. I'd prefer > if we'd abstract this away, so that the GCs only provide a "parallel > object iteration" interface, and the heap inspection code is kept elsewhere. > > I started experimenting with doing that, but other higher-priority (to > me) tasks have had to take precedence. > > I've uploaded my work-in-progress / proof-of-concept: > https://cr.openjdk.java.net/~stefank/8215624/webrev.01.delta/ > https://cr.openjdk.java.net/~stefank/8215624/webrev.01/ > > The current code doesn't handle the lifecycle (deletion) of the > ParallelObjectIterators. There's also code left unimplemented in around > CollectedHeap::run_task. However, I think this could work as a basis to > pull out the heap inspection code out of the GCs. > > Thanks, > StefanK > > On 2020-04-22 02:21, linzang(??) wrote: > > Dear all, > > May I ask you help to review? This RFR has been there for quite a while. > > Thanks! > > > > BRs, > > Lin > > > > > On 2020/3/16, 5:18 PM, "linzang(??)" wrote:> > > > >> Just update a new path, my preliminary measure show about 3.5x speedup of jmap histo on a nearly full 4GB G1 heap (8-core platform with parallel thread number set to 4). > >> webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_02/ > >> bug: https://bugs.openjdk.java.net/browse/JDK-8215624 > >> CSR: https://bugs.openjdk.java.net/browse/JDK-8239290 > >> BRs, > >> Lin > >> > On 2020/3/2, 9:56 PM, "linzang(??)" wrote: > >> > > >> > Dear all, > >> > Let me try to ease the reviewing work by some explanation :P > >> > The patch's target is to speed up jmap -histo for heap iteration, from my experience it is necessary for large heap investigation. E.g in bigData scenario I have tried to conduct jmap -histo against 180GB heap, it does take quite a while. > >> > And if my understanding is corrent, even the jmap -histo without "live" option does heap inspection with heap lock acquired. so it is very likely to block mutator thread in allocation-sensitive scenario. I would say the faster the heap inspection does, the shorter the mutator be blocked. This is parallel iteration for jmap is necessary. > >> > I think the parallel heap inspection should be applied to all kind of heap. However, consider the heap layout are different for GCs, much time is required to understand all kinds of the heap layout to make the whole change. IMO, It is not wise to have a huge patch for the whole solution at once, and it is even harder to review it. So I plan to implement it incrementally, the first patch (this one) is going to confirm the implemention detail of how jmap accept the new option, passes it to attachListener of the jvm process and then how to make the parallel inspection closure be generic enough to make it easy to extend to different heap layout. And also how to implement the heap inspection in specific gc's heap. This patch use G1's heap as the begining. > >> > This patch actually do several things: > >> > 1. Add an option "parallelThreadNum=" to jmap -histo, the default behavior is to set N to 0, means let's JVM decide how many threads to use for heap inspection. Set this option to 1 will disable parallel heap inspection. (more details in CSR: https://bugs.openjdk.java.net/browse/JDK-8239290) > >> > 2. Make a change in how Jmap passing arguments, changes in http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java.udiff.html, originally it pass options as separate arguments to attachListener, this patch change to that all options be compose to a single string. So the arg_count_max in attachListener.hpp do not need to be changed, and hence avoid the compatibility issue, as disscussed at https://mail.openjdk.java.net/pipermail/serviceability-dev/2019-March/027334.html > >> > 3. Add an abstract class ParHeapInspectTask in heapInspection.hpp / heapInspection.cpp, It's work(uint worker_id) method prepares the data structure (KlassInfoTable) need for every parallel worker thread, and then call do_object_iterate_parallel() which is heap specific implementation. I also added some machenism in KlassInfoTable to support parallel iteration, such as merge(). > >> > 4. In specific heap (G1 in this patch), create a subclass of ParHeapInspectTask, implement the do_object_iterate_parallel() for parallel heap inspection. For G1, it simply invoke g1CollectedHeap's object_iterate_parallel(). > >> > 5. Add related test. > >> > 6. it may be easy to extend this patch for other kinds of heap by creating subclass of ParHeapInspectTask and implement the do_object_iterate_parallel(). > >> > > >> > Hope these info could help on code review and initate the discussion :-) > >> > Thanks! > >> > > >> > BRs, > >> > Lin > >> > >On 2020/2/19, 9:40 AM, "linzang(??)" wrote:. > >> > > > >> > > Re-post this RFR with correct enhancement number to make it trackable. > >> > > please ignore the previous wrong post. sorry for troubles. > >> > > > >> > > webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/ > >> > > Hi bug: https://bugs.openjdk.java.net/browse/JDK-8215624 > >> > > CSR: https://bugs.openjdk.java.net/browse/JDK-8239290 > >> > > -------------- > >> > > Lin > >> > > >Hi Lin, > > > > > > > >> > > >Could you, please, re-post your RFR with the right enhancement number in > >> > > >the message subject? > >> > > >It will be more trackable this way. > >> > > > > >> > > >Thanks, > >> > > >Serguei > >> > > > > >> > > > > >> > > >On 2/17/20 10:29 PM, linzang(??) wrote: > >> > > >> Dear David, > >> > > >> Thanks a lot! > >> > > >> I have updated the refined code to http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_01/. > >> > > >> IMHO the parallel heap inspection can be extended to all kinds of heap as long as the heap layout can support parallel iteration. > >> > > >> Maybe we can firstly use this webrev to discuss how to implement it, because I am not sure my current implementation is an appropriate way to communicate with collectedHeap, then we can extend the solution to other kinds of heap. > >> > > >> > >> > > >> Thanks, > >> > > >> -------------- > >> > > >> Lin > >> > > >>> Hi Lin, > >> > > >>> > >> > > >>> Adding in hotspot-gc-dev as they need to see how this interacts with GC > >> > > >>> worker threads, and whether it needs to be extended beyond G1. > >> > > >>> > >> > > >>> I happened to spot one nit when browsing: > >> > > >>> > >> > > >>> src/hotspot/share/gc/shared/collectedHeap.hpp > >> > > >>> > >> > > >>> + virtual bool run_par_heap_inspect_task(KlassInfoTable* cit, > >> > > >>> + BoolObjectClosure* filter, > >> > > >>> + size_t* missed_count, > >> > > >>> + size_t thread_num) { > >> > > >>> + return NULL; > >> > > >>> > >> > > >>> s/NULL/false/ > >> > > >>> > >> > > >>> Cheers, > >> > > >>> David > > > > > >>> > >> > > >>> On 18/02/2020 2:15 pm, linzang(??) wrote: > >> > > >>>> Dear All, > >> > > >>>> May I ask your help to review the follow changes: > >> > > >>>> webrev: > >> > > >>>> http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_00/ > >> > > >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8215624 > >> > > >>>> related CSR: https://bugs.openjdk.java.net/browse/JDK-8239290 > >> > > >>>> This patch enable parallel heap inspection of G1 for jmap histo. > >> > > >>>> my simple test shown it can speed up 2x of jmap -histo with > >> > > >>>> parallelThreadNum set to 2 for heap at ~500M on 4-core platform. > >> > > >>>> > >> > > >>>> ------------------------------------------------------------------------ > >> > > >>>> BRs, > >> > > >>>> Lin > >> > > >> > > >> > > > > > > > > > > > > > > > From matthias.baesken at sap.com Thu Jul 30 05:28:05 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 30 Jul 2020 05:28:05 +0000 Subject: [PING?] RFR(s): 8247863: Unreachable code in OperatingSystemImpl.getTotalSwapSpaceSize() In-Reply-To: References: <40bd7c49d36c5775d86b9d2ff5740b50cf4359e9.camel@redhat.com> Message-ID: >The only reason why a 0 was observed, was because the cgroup interface >files were missing and the old code mapped that to a 0. That's no >longer the case and, thus, it seems it would make the code clearer if >it wouldn't be there any more. > >I don't feel strongly about this, though, and can just drop this patch. Hi Severin, thanks for clarification, I'm fine with the patch ! Best regards, Matthias From kevin.walls at oracle.com Thu Jul 30 09:16:52 2020 From: kevin.walls at oracle.com (Kevin Walls) Date: Thu, 30 Jul 2020 10:16:52 +0100 Subject: RFR(S): 8250750: JDK-8247515 fix for OSX pc_to_symbol() lookup fails with some symbols In-Reply-To: <33d6d75e-e9f0-65c2-fa56-b1dc2f2be223@oracle.com> References: <33d6d75e-e9f0-65c2-fa56-b1dc2f2be223@oracle.com> Message-ID: <2e35e463-9940-02a7-ec35-e1eace2ce718@oracle.com> Hi Chris - Yes, that's a good discovery, looks good, Thanks Kevin On 29/07/2020 21:08, Chris Plummer wrote: > Hello, > > Please help review the following: > > https://bugs.openjdk.java.net/browse/JDK-8250750 > http://cr.openjdk.java.net/~cjplummer/8250750/webrev.00/index.html > > Details are in the CR description. > > thanks, > > Chris From sgehwolf at redhat.com Thu Jul 30 09:23:14 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Thu, 30 Jul 2020 11:23:14 +0200 Subject: [PING?] RFR(s): 8247863: Unreachable code in OperatingSystemImpl.getTotalSwapSpaceSize() In-Reply-To: References: <40bd7c49d36c5775d86b9d2ff5740b50cf4359e9.camel@redhat.com> Message-ID: On Thu, 2020-07-30 at 05:28 +0000, Baesken, Matthias wrote: > > The only reason why a 0 was observed, was because the cgroup interface > > files were missing and the old code mapped that to a 0. That's no > > longer the case and, thus, it seems it would make the code clearer if > > it wouldn't be there any more. > > > > I don't feel strongly about this, though, and can just drop this patch. > > Hi Severin, thanks for clarification, I'm fine with the patch ! Thanks for the review, Matthias. Cheers, Severin From suenaga at oss.nttdata.com Thu Jul 30 13:45:30 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Thu, 30 Jul 2020 22:45:30 +0900 Subject: RFR (trivial): 8250826: jhsdb does not work with coredump which comes from Substrate VM Message-ID: <06e53c65-3cc6-8c51-33d0-26e207f4d0a0@oss.nttdata.com> Hi all, Please review this trivial change: JBS: https://bugs.openjdk.java.net/browse/JDK-8250826 webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8250826/webrev.00/ I played Truffle NFI on GraalVM, but I cannot get Java stacks from coredump via jhsdb. I've reported this issue to GraalVM community [1], and I 've found out the cause of this issue is .svm_heap would be separated to RO and RW areas by mprotect() calls in run time in spite of .svm_heap is RO section in ELF (please see [1] for details). It is corner case, but we will see same problem on jhsdb when we attempt to analyze coredump which comes from some applications / libraries which would separate RO sections in ELF like Substrate VM. I sent PR to fix libsaproc.so in LabsJDK 11 for this issue [2], then community members suggested me to discuss in serviceability-dev. Thanks, Yasumasa [1] https://github.com/oracle/graal/issues/2579 [2] https://github.com/graalvm/labs-openjdk-11/pull/9 From chris.plummer at oracle.com Thu Jul 30 20:18:47 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 30 Jul 2020 13:18:47 -0700 Subject: RFR (trivial): 8250826: jhsdb does not work with coredump which comes from Substrate VM In-Reply-To: <06e53c65-3cc6-8c51-33d0-26e207f4d0a0@oss.nttdata.com> References: <06e53c65-3cc6-8c51-33d0-26e207f4d0a0@oss.nttdata.com> Message-ID: Hi Yasumasa, I'm reviewing this RFR, and I'd like to ask that it not be pushed as trivial. Although it is just a one line change, it takes an extensive knowledge to understand the impact. I'll read up on the filed graal issue and try to understand the ELF code a bit better. thanks, Chris On 7/30/20 6:45 AM, Yasumasa Suenaga wrote: > Hi all, > > Please review this trivial change: > > ? JBS: https://bugs.openjdk.java.net/browse/JDK-8250826 > ? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8250826/webrev.00/ > > I played Truffle NFI on GraalVM, but I cannot get Java stacks from > coredump via jhsdb. > > I've reported this issue to GraalVM community [1], and I 've found out > the cause of this issue is .svm_heap would be separated to RO and RW > areas by mprotect() calls in run time in spite of .svm_heap is RO > section in ELF (please see [1] for details). > > It is corner case, but we will see same problem on jhsdb when we > attempt to analyze coredump which comes from some applications / > libraries which would separate RO sections in ELF like Substrate VM. > > I sent PR to fix libsaproc.so in LabsJDK 11 for this issue [2], then > community members suggested me to discuss in serviceability-dev. > > > Thanks, > > Yasumasa > > > [1] https://github.com/oracle/graal/issues/2579 > [2] https://github.com/graalvm/labs-openjdk-11/pull/9 From chris.plummer at oracle.com Thu Jul 30 22:29:36 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 30 Jul 2020 15:29:36 -0700 Subject: RFR (trivial): 8250826: jhsdb does not work with coredump which comes from Substrate VM In-Reply-To: References: <06e53c65-3cc6-8c51-33d0-26e207f4d0a0@oss.nttdata.com> Message-ID: Hi Yasumasa, If I understand correctly we first call add_map_info() for all the PT_LOAD segments in the core file. We then process all the library segments, calling add_map_info() for them if the target_vaddr has not already been addded. If has already been added, which I assume is the case for any library segment that is already in the core file, then the core file version is replaced the the library version.? I'm a little unclear of the purpose of this replacing of the core PT_LOAD segments with those found in the libraries. If you could explain this that would help me understand your change. I'm also unsure why existing_map->fd would ever be something other than the core file. Why would another library map the same target_vaddr. thanks, Chris On 7/30/20 1:18 PM, Chris Plummer wrote: > Hi Yasumasa, > > I'm reviewing this RFR, and I'd like to ask that it not be pushed as > trivial. Although it is just a one line change, it takes an extensive > knowledge to understand the impact. I'll read up on the filed graal > issue and try to understand the ELF code a bit better. > > thanks, > > Chris > > On 7/30/20 6:45 AM, Yasumasa Suenaga wrote: >> Hi all, >> >> Please review this trivial change: >> >> ? JBS: https://bugs.openjdk.java.net/browse/JDK-8250826 >> ? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8250826/webrev.00/ >> >> I played Truffle NFI on GraalVM, but I cannot get Java stacks from >> coredump via jhsdb. >> >> I've reported this issue to GraalVM community [1], and I 've found >> out the cause of this issue is .svm_heap would be separated to RO and >> RW areas by mprotect() calls in run time in spite of .svm_heap is RO >> section in ELF (please see [1] for details). >> >> It is corner case, but we will see same problem on jhsdb when we >> attempt to analyze coredump which comes from some applications / >> libraries which would separate RO sections in ELF like Substrate VM. >> >> I sent PR to fix libsaproc.so in LabsJDK 11 for this issue [2], then >> community members suggested me to discuss in serviceability-dev. >> >> >> Thanks, >> >> Yasumasa >> >> >> [1] https://github.com/oracle/graal/issues/2579 >> [2] https://github.com/graalvm/labs-openjdk-11/pull/9 > > From suenaga at oss.nttdata.com Fri Jul 31 01:18:11 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Fri, 31 Jul 2020 10:18:11 +0900 Subject: RFR (trivial): 8250826: jhsdb does not work with coredump which comes from Substrate VM In-Reply-To: References: <06e53c65-3cc6-8c51-33d0-26e207f4d0a0@oss.nttdata.com> Message-ID: <3805349a-342f-a130-3b0b-7ee26f23a278@oss.nttdata.com> Hi Chris, On 2020/07/31 7:29, Chris Plummer wrote: > Hi Yasumasa, > > If I understand correctly we first call add_map_info() for all the PT_LOAD segments in the core file. We then process all the library segments, calling add_map_info() for them if the target_vaddr has not already been addded. If has already been added, which I assume is the case for any library segment that is already in the core file, then the core file version is replaced the the library version.? I'm a little unclear of the purpose of this replacing of the core PT_LOAD segments with those found in the libraries. If you could explain this that would help me understand your change. Read only segments in ELF should not be any different from PT_LOAD segments in the core. And head of ELF header might be included in coredump (See JDK-7133122). Thus we need to replace PT_LOAD segments the library version. > I'm also unsure why existing_map->fd would ever be something other than the core file. Why would another library map the same target_vaddr. When mmap() is called to read-only ELF segments / sections, Linux kernel seems to allocate other memory segments which has same top virtual memory address. I've not yet found out from the code of Linux kernel, but I confirmed this behavior on GDB. Thanks, Yasumasa > thanks, > > Chris > > On 7/30/20 1:18 PM, Chris Plummer wrote: >> Hi Yasumasa, >> >> I'm reviewing this RFR, and I'd like to ask that it not be pushed as trivial. Although it is just a one line change, it takes an extensive knowledge to understand the impact. I'll read up on the filed graal issue and try to understand the ELF code a bit better. >> >> thanks, >> >> Chris >> >> On 7/30/20 6:45 AM, Yasumasa Suenaga wrote: >>> Hi all, >>> >>> Please review this trivial change: >>> >>> ? JBS: https://bugs.openjdk.java.net/browse/JDK-8250826 >>> ? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8250826/webrev.00/ >>> >>> I played Truffle NFI on GraalVM, but I cannot get Java stacks from coredump via jhsdb. >>> >>> I've reported this issue to GraalVM community [1], and I 've found out the cause of this issue is .svm_heap would be separated to RO and RW areas by mprotect() calls in run time in spite of .svm_heap is RO section in ELF (please see [1] for details). >>> >>> It is corner case, but we will see same problem on jhsdb when we attempt to analyze coredump which comes from some applications / libraries which would separate RO sections in ELF like Substrate VM. >>> >>> I sent PR to fix libsaproc.so in LabsJDK 11 for this issue [2], then community members suggested me to discuss in serviceability-dev. >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> [1] https://github.com/oracle/graal/issues/2579 >>> [2] https://github.com/graalvm/labs-openjdk-11/pull/9 >> >> > > From richard.reingruber at sap.com Fri Jul 31 07:28:14 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Fri, 31 Jul 2020 07:28:14 +0000 Subject: RFR(S) 8249293: Unsafe stackwalk in VM_GetOrSetLocal::doit_prologue() In-Reply-To: References: Message-ID: Hi, I rebase the fix after JDK-8250042. New webrev: http://cr.openjdk.java.net/~rrich/webrevs/8249293/webrev.2/ Thanks, Richard. -----Original Message----- From: serviceability-dev On Behalf Of Reingruber, Richard Sent: Montag, 27. Juli 2020 09:45 To: serguei.spitsyn at oracle.com; serviceability-dev at openjdk.java.net Subject: [CAUTION] RE: RFR(S) 8249293: Unsafe stackwalk in VM_GetOrSetLocal::doit_prologue() Hi Serguei, > I tested it on Linux and Windows but not yet on MacOS. The test succeeded now on all platforms. Thanks, Richard. -----Original Message----- From: Reingruber, Richard Sent: Freitag, 24. Juli 2020 15:04 To: serguei.spitsyn at oracle.com; serviceability-dev at openjdk.java.net Subject: RE: RFR(S) 8249293: Unsafe stackwalk in VM_GetOrSetLocal::doit_prologue() Hi Serguei, > The fix itself looks good to me. thanks for looking at the fix. > I still need another look at new test. > Could you, please, convert the agent of new test to C++? > It will make it a little bit simpler. Sure, here is the new webrev.1 with a C++ version of the test agent: http://cr.openjdk.java.net/~rrich/webrevs/8249293/webrev.1/ I tested it on Linux and Windows but not yet on MacOS. Thanks, Richard. -----Original Message----- From: serguei.spitsyn at oracle.com Sent: Freitag, 24. Juli 2020 00:00 To: Reingruber, Richard ; serviceability-dev at openjdk.java.net Subject: Re: RFR(S) 8249293: Unsafe stackwalk in VM_GetOrSetLocal::doit_prologue() Hi Richard, Thank you for filing the CR and taking care about it! The fix itself looks good to me. I still need another look at new test. Could you, please, convert the agent of new test to C++? It will make it a little bit simpler. Thanks, Serguei On 7/20/20 01:15, Reingruber, Richard wrote: > Hi, > > please help review this fix for VM_GetOrSetLocal. It moves the unsafe stackwalk from the vm > operation prologue before the safepoint into the doit() method executed at the safepoint. > > Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8249293/webrev.0/index.html > Bug: https://bugs.openjdk.java.net/browse/JDK-8249293 > > According to the JVMTI spec on local variable access it is not required to suspend the target thread > T [1]. The operation will simply fail with JVMTI_ERROR_NO_MORE_FRAMES if T is executing > bytecodes. It will succeed though if T is blocked because of synchronization or executing some native > code. > > The issue is that in the latter case the stack walk in VM_GetOrSetLocal::doit_prologue() to prepare > the access to the local variable is unsafe, because it is done before the safepoint and it races > with T returning to execute bytecodes making its stack not walkable. The included test shows that > this can crash the VM if T wins the race. > > Manual testing: > > - new test test/hotspot/jtreg/serviceability/jvmti/GetLocalVariable/GetLocalWithoutSuspendTest.java > - test/hotspot/jtreg/vmTestbase/nsk/jvmti > - test/hotspot/jtreg/serviceability/jvmti > > Nightly regression tests @SAP: JCK and JTREG, also in Xcomp mode, SPECjvm2008, SPECjbb2015, > Renaissance Suite, SAP specific tests with fastdebug and release builds on all platforms > > Thanks, Richard. > > [1] https://docs.oracle.com/en/java/javase/14/docs/specs/jvmti.html#local From serguei.spitsyn at oracle.com Fri Jul 31 08:06:54 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 31 Jul 2020 01:06:54 -0700 Subject: JDK 16 RFR of JDK-8250640: Address reliance on default constructors in jdk.jdi In-Reply-To: <3b83628a-56d8-d755-1cd7-d920afd0e11b@oracle.com> References: <901e55aa-7def-8341-2140-67993ce74713@oracle.com> <3b83628a-56d8-d755-1cd7-d920afd0e11b@oracle.com> Message-ID: <791eba28-5113-3eef-b7ff-a4d5258d2d59@oracle.com> Hi Joe, It looks good to me. Thanks, Serguei On 7/27/20 23:45, Alan Bateman wrote: > > > On 27/07/2020 21:42, Joe Darcy wrote: >> Hello, >> >> Another module, another set of default constructors to replace with >> explicit ones. Please review the code changes and CSR to address: >> >> ??? JDK-8250640: Address reliance on default constructors in jdk.jdi >> ??? webrev: http://cr.openjdk.java.net/~darcy/8250640.0/ >> ??? CSR: https://bugs.openjdk.java.net/browse/JDK-8250642 >> >> Patch below; I'll update copyrights before pushing. In the Bootstrap >> class, since it doesn't define any non-static methods, it looks like >> it doesn't need a constructor in its API at all so I terminally >> deprecated the constructor. > Right, Bootstrap should have a public constructor. Ideally the > abstract classes in the spi package would have a protected constructor. > > The change looks okay to me. > > -Alan From chris.plummer at oracle.com Fri Jul 31 18:07:01 2020 From: chris.plummer at oracle.com (Chris Plummer) Date: Fri, 31 Jul 2020 11:07:01 -0700 Subject: RFR(XXS): 8249150: SA core file tests sometimes time out on OSX with "java.io.IOException: App waiting timeout" Message-ID: <124869c8-dc29-d453-1df1-fc4488f25acb@oracle.com> Hello, Please help review the following: https://bugs.openjdk.java.net/browse/JDK-8249150 http://cr.openjdk.java.net/~cjplummer/8249150/webrev.00/index.html The tests in question use recently added LingeredApp support for dumping a core file. On OSX a core file dump sometimes takes a very long time, exceeding the amount of time Lingeredapp.waitAppReady() is willing to wait. This wait time needs to be increased to allow the core file to finish dumping, and also a couple of the tests that use the LingeredApp core file support need longer test timeouts. Note ClhsdbFindPC already has a long timeout, so no timeout changes were needed for it. Tested by running all serviceability/sa tests once on all platforms, and 400 times on OSX (200 regular runs and 100 -Xcomp runs). thanks, Chris From igor.ignatyev at oracle.com Fri Jul 31 20:24:10 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 31 Jul 2020 13:24:10 -0700 Subject: RFR(L/S) : 8249030 : clean up FileInstaller $test.src $cwd in vmTestbase_nsk_jdi tests Message-ID: <637D6495-24E6-4874-9024-1B0082492085@oracle.com> http://cr.openjdk.java.net/~iignatyev//8249030/webrev.00 > 2258 lines changed: 0 ins; 1144 del; 1114 mod; Hi all, could you please review the clean-up of nsk_jdi tests? from main issue(8204985) : > all vmTestbase tests have '@run driver jdk.test.lib.FileInstaller . .' to mimic old test harness behavior and copy all files from a test source directory to a current work directory. some tests depend on this step, so we need 1st identify such tests and then either rewrite them not to have this dependency or leave FileInstaller only in these tests. the patch removes FileInstaller actions in the said tests, and as before, the biggest part of patch was done by `ag -l '@run driver jdk.test.lib.FileInstaller . .' $DIR | xargs -I{} gsed -i '/@run driver jdk.test.lib.FileInstaller \. \./d' {}` with $DIR being test/hotspot/jtreg/vmTestbase/nsk/jdi/. the 10 tests which had '-configFile ./<...>', and hence were looking for config file in the current directory, were updated to search for a config file in 'test.src' directory: http://cr.openjdk.java.net/~iignatyev//8249030/webrev.00-configFile testing: :vmTestbase_nsk_jdi on {linux,windows,macos}-x64 JBS: https://bugs.openjdk.java.net/browse/JDK-8249030 webrev: http://cr.openjdk.java.net/~iignatyev/8249030/webrev.00/ Thanks, -- Igor