From yasuenag at gmail.com Mon Apr 1 12:46:17 2019 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Mon, 1 Apr 2019 21:46:17 +0900 Subject: RFR: 8220657: JFR.dump does not work when filename is set In-Reply-To: References: <5C90E8C9.3010008@oracle.com> <14c04f3a-1fc0-af3b-714f-e35fa18d6a1b@gmail.com> <5C9103DE.4040304@oracle.com> Message-ID: <725f2ab6-71ee-4149-9d1d-ca6c19217408@gmail.com> Hi, Do you have concern(s) about this change? >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.01/ As in the email below, I think it does not affect scenarios except snapshot cloning. Please tell me if you have any opinions about this. Thanks, Yasumasa On 2019/03/20 10:37, Yasumasa Suenaga wrote: > Hi Erik, > > I think this change does not affect other scenarios. > > :jdk_jfr jtreg tests and submit repo have passed with this change. > newSnapshotClone() is called by PlatformRecording::dump and > DCmdDump::newSnapshot. > Both methods are related to user-requested dump. > > What scenarios do you think? I want to check them. > > > Thanks, > > Yasumasa > > > 2019?3?19?(?) 23:59 Erik Gahlin : >> >> Yes, but then you would lose the original destination in other scenarios >> >> Maybe I can look at this next week. >> >> Erik >> >>> Hi Erik, >>> >>> Thank you for your reply. >>> >>> How about this changeset? >>> >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.01/ >>> >>> I think PlatformRecording#newSnapshotClone() should not dump. >>> So I removed `setDestination()` from this. >>> >>> >>> Yasumasa >>> >>> >>> On 2019/03/19 22:04, Erik Gahlin wrote: >>>> This looks a bit hackish. >>>> >>>> The real issue seems to be that the method >>>> PlatformRecording#newSnapshotClone incorrectly sets the destination in >>>> the clone, which then triggers a dump prematurely (when stop is called). >>>> >>>> Purpose of that method is just to take a snapshot of existing chunks in >>>> the disk repository (or from memory). and then let dumpStopped(path) do >>>> the actual dump (copy out the data from the disk repository). >>>> >>>> To make it work, some other changes would be needed as well, but I don't >>>> have time to investigate. >>>> >>>> Erik >>>> >>>>> Hi all, >>>>> >>>>> Please review this change. >>>>> >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8220657 >>>>> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.00/ >>>>> >>>>> The user might not get flight record via JFR.dump if JFR is started >>>>> with filename option on target process. >>>>> >>>>> Please see JBS if you want to know how to reproduce. >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >> From yasuenag at gmail.com Mon Apr 1 12:49:43 2019 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Mon, 1 Apr 2019 21:49:43 +0900 Subject: PING: RFR: 8219566: JFR did not collect call stacks when MaxJavaStackTraceDepth is set to zero In-Reply-To: <7c203381-0091-f9a9-6e76-1b09d0e92c88@gmail.com> References: <6D0A14CC-3EBA-4A01-AFFC-F2987F3C4DFD@oracle.com> <59ed9471-45a5-5707-8356-367ae35a7829@gmail.com> <19551e1b-b9e8-62e1-6fab-7dd6e100f50a@gmail.com> <5C828C38.1090506@oracle.com> <47d27b56-0933-c173-55c1-7e225218ead8@gmail.com> <0b31693a-a8d4-1fdf-c830-8752f3ec379e@gmail.com> <5C95369B.8030404@oracle.com> <7c203381-0091-f9a9-6e76-1b09d0e92c88@gmail.com> Message-ID: <2ca33c8e-6565-fad3-294c-ae1804271f54@gmail.com> Hi, Do you have any concern(s) about this change? >>> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.02/ I think thread stack should be collected by JFR even if MaxJavaStackTraceDepth is set to zero. Thanks, Yasumasa On 2019/03/23 21:03, Yasumasa Suenaga wrote: > Hi Erik, > > MaxJavaStackTraceDepth do not only affect finding top stack but also stack walking. > > JfrGetCallTrace::find_top_frame() uses MaxJavaStackTraceDepth to find top frame as you say. > OTOH vframeStreamSamples::samples_next() uses it to find next valid frame. > > > Anyway thread stack should be collected even if MaxJavaStackTraceDepth is set to zero. > > > Thanks, > > Yasumasa > > > On 2019/03/23 4:25, Erik Gahlin wrote: >> To me, it seems that the purpose of the method is to find the first >> valid frame on the stack (the top frame). This is orthogonal to the >> number of frames that should be recorded by JFR. >> >> Erik >> >>> PING: Could you review it? >>> >>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8219566 >>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.02/ >>> >>> This change has passed :jdk_jfr tests and submit repo. >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2019/03/11 14:49, Yasumasa Suenaga wrote: >>>> 2019?3?9?(?) 20:47 Yasumasa Suenaga : >>>>> >>>>> Hi Erik, >>>>> >>>>> Reading your email, I think we should use JfrOptionSet::stackdepth() >>>>> rather than MaxJavaStackTraceDepth. >>>>> I created new webrev, and it passed jdk_jfr jtreg tests on my Linux >>>>> x64 box. >>>>> >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.01/ >>>>> >>>>> However it has not yet passed tests on submit repo >>>>> (mach5-one-ysuenaga-JDK-8219566-20190309-0909-1079793). >>>>> It seems to be failed at build phase which includes Linux. >>>>> (I can build with this change on Fedora 29 x64). >>>>> >>>>> Could you help if this webrev is ok? >>>> >>>> I found 1 warning, and I fixed it. >>>> This webrev works fine with :jdk_jfr jtreg tests on my Linux box, and >>>> submit repo test >>>> (mach5-one-ysuenaga-JDK-8219566-20190311-0150-1121570). >>>> Could you review it? >>>> >>>> ??? http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.02/ >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> >>>>> On 2019/03/09 0:37, Erik Gahlin wrote: >>>>>> JFR imposes a limit on how many frames that can be recorded, set to >>>>>> 64 by default but configurable up to 2048 using >>>>>> -XX:FlightRecorderOptions, so setting the value to 0 will still not >>>>>> mean all frames. >>>>>> >>>>>> That said, if you still want to fix it. It may make sense to set >>>>>> loop_max to INT_MAX, if MaxJavaStackTraceDepth is zero, i.e >>>>>> >>>>>> int loop_max = MaxJavaStackTraceDepth == 0 ? INT_MAX : >>>>>> MaxJavaStackTraceDepth * 2; >>>>>> >>>>>> and avoid evaluating it for every frame. Even if the compiler is >>>>>> able to hoist the comparison, it makes the code easier to read. >>>>>> >>>>>> Thanks >>>>>> Erik >>>>>> >>>>>> >>>>>>> PING: Could you review it? >>>>>>> >>>>>>>>>> ??? JBS: https://bugs.openjdk.java.net/browse/JDK-8219566 >>>>>>>>>> ??? webrev: >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.00/ >>>>>>> >>>>>>> >>>>>>> If you have any questions, please tell me. >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2019/02/27 22:46, Yasumasa Suenaga wrote: >>>>>>>> Hi Erik, >>>>>>>> >>>>>>>> On 2019/02/27 22:30, Erik Gahlin wrote: >>>>>>>>> Hi Yasumasa, >>>>>>>>> >>>>>>>>> Could you explain a little about the bug? >>>>>>>>> >>>>>>>>> Why should stack traces be collected if MaxJavaStackTraceDepth >>>>>>>>> is set to 0. >>>>>>>> >>>>>>>> "0" in MaxJavaStackTraceDepth means all exception stack traces >>>>>>>> will be shown. >>>>>>>> >>>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/72ce7dd54939/src/hotspot/share/runtime/globals.hpp#l1553 >>>>>>>> >>>>>>>> >>>>>>>> JDK-7179701 reports similar issue. >>>>>>>> I think we can take similar approach for JDK-8219566. >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>>> Thanks >>>>>>>>> Erik >>>>>>>>> >>>>>>>>>> On 22 Feb 2019, at 07:48, Yasumasa Suenaga >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> Hi all, >>>>>>>>>> >>>>>>>>>> Please review this change: >>>>>>>>>> >>>>>>>>>> ??? JBS: https://bugs.openjdk.java.net/browse/JDK-8219566 >>>>>>>>>> ??? webrev: >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.00/ >>>>>>>>>> >>>>>>>>>> JFR did not collect call stacks when MaxJavaStackTraceDepth is >>>>>>>>>> set to zero. >>>>>>>>>> This bug is similar to JDK-7179701. >>>>>>>>>> >>>>>>>>>> This change passed tests on submit repo and :jdk_jfr jtreg tests. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>> >>>>>> >> From mikhailo.seledtsov at oracle.com Tue Apr 9 03:28:02 2019 From: mikhailo.seledtsov at oracle.com (mikhailo.seledtsov at oracle.com) Date: Mon, 8 Apr 2019 20:28:02 -0700 Subject: RFR(M): 8221711: [TESTBUG] create more tests for JFR in container environment Message-ID: Please review this change that adds two more test cases for JFR events in the container environment: ? - making sure environment variables are reported correctly inside the container ? - host info is reported correctly inside the container ??? JBS: https://bugs.openjdk.java.net/browse/JDK-8221711 ??? Webrev: http://cr.openjdk.java.net/~mseledtsov/8221711.01/ ??? Testing: ????? Ran the affected tests on a single host - PASS ????? On multiple host variety - In PROGRESS Thank you, Misha From mikhailo.seledtsov at oracle.com Thu Apr 11 15:27:13 2019 From: mikhailo.seledtsov at oracle.com (mikhailo.seledtsov at oracle.com) Date: Thu, 11 Apr 2019 08:27:13 -0700 Subject: RFR(M): 8221711: [TESTBUG] create more tests for JFR in container environment In-Reply-To: References: Message-ID: Ping. On 4/8/19 8:28 PM, mikhailo.seledtsov at oracle.com wrote: > Please review this change that adds two more test cases for JFR events > in the container environment: > ? - making sure environment variables are reported correctly inside > the container > ? - host info is reported correctly inside the container > > ??? JBS: https://bugs.openjdk.java.net/browse/JDK-8221711 > ??? Webrev: http://cr.openjdk.java.net/~mseledtsov/8221711.01/ > ??? Testing: > ????? Ran the affected tests on a single host - PASS > ????? On multiple host variety - In PROGRESS > > > Thank you, > Misha > From erik.gahlin at oracle.com Fri Apr 12 13:48:21 2019 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Fri, 12 Apr 2019 15:48:21 +0200 Subject: RFR(M): 8221711: [TESTBUG] create more tests for JFR in container environment In-Reply-To: References: Message-ID: <5CB09725.5010108@oracle.com> Hi Misha, I'm a bit confused. I thought the purpose of JFrReporter was to write out the result of a recording in a container and let TestJFREvents do the actual verification by looking at the result from standard out. This doesn't seem to be the case with testEnvironmentVariables. It seems like it is doing both, or am I missing something? Would it be possible to simplify the JFrReporter into something like this: public class JfrReporter { public static void main(String[] args) throws Exception { String eventName = args[1]; try(Recording r = new Recording()) { r.enable(eventName); r.start(); r.stop(); Path p = Paths.get("tmp", eventName + ".jfr"); r.dump(p); for (RecordedEvent e : RecordingFile.readAllEvents(p)) { for (ValueDescriptor v : e.getEventType().getFields()) { System.out.println(v.getName() + " = " + e.getValue(v.getName())); } } } } } and move all the logic into TestJFREvents? Maybe there is an argument against this, but I'm curious. In JfrNetwork.java, you wait for 5 seconds for a join. It seems brittle to depend on a thread getting scheduled or not. Could we create something more deterministic, and not depend on something happening within a specific time? Thanks Erik > > > On 4/8/19 8:28 PM, mikhailo.seledtsov at oracle.com wrote: >> Please review this change that adds two more test cases for JFR >> events in the container environment: >> - making sure environment variables are reported correctly inside >> the container >> - host info is reported correctly inside the container >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8221711 >> Webrev: http://cr.openjdk.java.net/~mseledtsov/8221711.01/ >> Testing: >> Ran the affected tests on a single host - PASS >> On multiple host variety - In PROGRESS >> >> >> Thank you, >> Misha >> > From mikhailo.seledtsov at oracle.com Fri Apr 12 14:58:35 2019 From: mikhailo.seledtsov at oracle.com (mikhailo.seledtsov at oracle.com) Date: Fri, 12 Apr 2019 07:58:35 -0700 Subject: RFR(M): 8221711: [TESTBUG] create more tests for JFR in container environment In-Reply-To: <5CB09725.5010108@oracle.com> References: <5CB09725.5010108@oracle.com> Message-ID: Hi Erik, ? Thank you for taking a look at the changes. Please see my comments inline. On 4/12/19 6:48 AM, Erik Gahlin wrote: > Hi Misha, > > I'm a bit confused. I thought the purpose of JFrReporter was to write > out the result of a recording in a container and let TestJFREvents do > the actual verification by looking at the result from standard out. > > This doesn't seem to be the case with testEnvironmentVariables. It > seems like it is doing both, or am I missing something? Yes, you are right. I deviated from the original pattern, perhaps for no good reason. > > Would it be possible to simplify the JFrReporter into something like > this: > > public class JfrReporter { > > ? public static void main(String[] args) throws Exception { > ??? String eventName = args[1]; > ??? try(Recording r = new Recording()) { > ????? r.enable(eventName); > ????? r.start(); > ????? r.stop(); > ????? Path p = Paths.get("tmp", eventName + ".jfr"); > ????? r.dump(p); > ????? for (RecordedEvent e : RecordingFile.readAllEvents(p)) { > ??????? for (ValueDescriptor v : e.getEventType().getFields()) { > ????????? System.out.println(v.getName() + " = " + > e.getValue(v.getName())); > ??????? } > ????? } > ??? } > ? } > } > > and move all the logic into TestJFREvents? Maybe there is an argument > against this, but I'm curious. I can do that. Most likely it will work, with minor additions. For instance, I would like to validate some prerequisite conditions (or test assumptions), such as that environment variables are defined inside a container: assertTrue(System.getenv(testEnvVar) == null, "Testcase TEST_ENVIRONMENT_VARS failed:" + 71 testEnvVar + " should not be defined inside a container"); 72 assertTrue(System.getenv("JAVA_HOME") != null, 73 "Testcase ENVIRONMENT_VARS failed: JAVA_HOME should be defined inside a container"); ?If for some reason these assumptions are wrong, it is most likely due to test bug or bug in other parts of JDK, and helps us isolate the root cause. Other than that, your proposal for simplification makes sense. I will implement it. > In JfrNetwork.java, you wait for 5 seconds for a join. It seems > brittle to depend on a thread getting? scheduled or not. Could we > create something more deterministic, and not depend on something > happening within a specific time? OK, this makes sense. I will remove the wait time. The join should happen always; and if it does not, it indicates the error, which will manifest the test timeout, and be followed by investigation. The only reason IMO to use timed wait/join in such cases is to guard against external/environmental circumstances that we have no control over. E.g. we are waiting for another entity to accept a connection, I/O that may take longer time due to system or HW factors, etc. In this particular case looks like our code is in control, so it should be fine to remove the timed wait. I will make the changes, and post an updated webrev. Thank you, Misha > Thanks > Erik > >> >> >> On 4/8/19 8:28 PM, mikhailo.seledtsov at oracle.com wrote: >>> Please review this change that adds two more test cases for JFR >>> events in the container environment: >>> ? - making sure environment variables are reported correctly inside >>> the container >>> ? - host info is reported correctly inside the container >>> >>> ??? JBS: https://bugs.openjdk.java.net/browse/JDK-8221711 >>> ??? Webrev: http://cr.openjdk.java.net/~mseledtsov/8221711.01/ >>> ??? Testing: >>> ????? Ran the affected tests on a single host - PASS >>> ????? On multiple host variety - In PROGRESS >>> >>> >>> Thank you, >>> Misha >>> >> > From erik.gahlin at oracle.com Fri Apr 12 18:00:43 2019 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Fri, 12 Apr 2019 20:00:43 +0200 Subject: PING: RFR: 8219566: JFR did not collect call stacks when MaxJavaStackTraceDepth is set to zero In-Reply-To: <2ca33c8e-6565-fad3-294c-ae1804271f54@gmail.com> References: <6D0A14CC-3EBA-4A01-AFFC-F2987F3C4DFD@oracle.com> <59ed9471-45a5-5707-8356-367ae35a7829@gmail.com> <19551e1b-b9e8-62e1-6fab-7dd6e100f50a@gmail.com> <5C828C38.1090506@oracle.com> <47d27b56-0933-c173-55c1-7e225218ead8@gmail.com> <0b31693a-a8d4-1fdf-c830-8752f3ec379e@gmail.com> <5C95369B.8030404@oracle.com> <7c203381-0091-f9a9-6e76-1b09d0e92c88@gmail.com> <2ca33c8e-6565-fad3-294c-ae1804271f54@gmail.com> Message-ID: <5CB0D24B.305@oracle.com> Hi, Could we use the max stack depth used by JFR instead? It's not available as constant in a header file, but maybe you could move: static const u4 STACK_DEPTH_DEFAULT = 64; static const u4 MIN_STACK_DEPTH = 1; static const u4 MAX_STACK_DEPTH = 2048; from jfrOptionSet.cpp to jfrTypes.hpp where we have other constants. The benefit of using MAX_STACK_DEPTH is that it will work in scenarios where a user has set the number of frames to a low number, i.e -XX:FlightRecorderOptions=5, and we hit a stack with lots of frames that we can't record, i.e native frames. Thanks Erik > Hi, > > Do you have any concern(s) about this change? > >>>> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.02/ > > I think thread stack should be collected by JFR even if > MaxJavaStackTraceDepth is set to zero. > > > Thanks, > > Yasumasa > > > On 2019/03/23 21:03, Yasumasa Suenaga wrote: >> Hi Erik, >> >> MaxJavaStackTraceDepth do not only affect finding top stack but also >> stack walking. >> >> JfrGetCallTrace::find_top_frame() uses MaxJavaStackTraceDepth to find >> top frame as you say. >> OTOH vframeStreamSamples::samples_next() uses it to find next valid >> frame. >> >> >> Anyway thread stack should be collected even if >> MaxJavaStackTraceDepth is set to zero. >> >> >> Thanks, >> >> Yasumasa >> >> >> On 2019/03/23 4:25, Erik Gahlin wrote: >>> To me, it seems that the purpose of the method is to find the first >>> valid frame on the stack (the top frame). This is orthogonal to the >>> number of frames that should be recorded by JFR. >>> >>> Erik >>> >>>> PING: Could you review it? >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8219566 >>>> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.02/ >>>> >>>> This change has passed :jdk_jfr tests and submit repo. >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2019/03/11 14:49, Yasumasa Suenaga wrote: >>>>> 2019?3?9?(?) 20:47 Yasumasa Suenaga : >>>>>> >>>>>> Hi Erik, >>>>>> >>>>>> Reading your email, I think we should use JfrOptionSet::stackdepth() >>>>>> rather than MaxJavaStackTraceDepth. >>>>>> I created new webrev, and it passed jdk_jfr jtreg tests on my Linux >>>>>> x64 box. >>>>>> >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.01/ >>>>>> >>>>>> However it has not yet passed tests on submit repo >>>>>> (mach5-one-ysuenaga-JDK-8219566-20190309-0909-1079793). >>>>>> It seems to be failed at build phase which includes Linux. >>>>>> (I can build with this change on Fedora 29 x64). >>>>>> >>>>>> Could you help if this webrev is ok? >>>>> >>>>> I found 1 warning, and I fixed it. >>>>> This webrev works fine with :jdk_jfr jtreg tests on my Linux box, and >>>>> submit repo test >>>>> (mach5-one-ysuenaga-JDK-8219566-20190311-0150-1121570). >>>>> Could you review it? >>>>> >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.02/ >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> >>>>>> On 2019/03/09 0:37, Erik Gahlin wrote: >>>>>>> JFR imposes a limit on how many frames that can be recorded, set to >>>>>>> 64 by default but configurable up to 2048 using >>>>>>> -XX:FlightRecorderOptions, so setting the value to 0 will still not >>>>>>> mean all frames. >>>>>>> >>>>>>> That said, if you still want to fix it. It may make sense to set >>>>>>> loop_max to INT_MAX, if MaxJavaStackTraceDepth is zero, i.e >>>>>>> >>>>>>> int loop_max = MaxJavaStackTraceDepth == 0 ? INT_MAX : >>>>>>> MaxJavaStackTraceDepth * 2; >>>>>>> >>>>>>> and avoid evaluating it for every frame. Even if the compiler is >>>>>>> able to hoist the comparison, it makes the code easier to read. >>>>>>> >>>>>>> Thanks >>>>>>> Erik >>>>>>> >>>>>>> >>>>>>>> PING: Could you review it? >>>>>>>> >>>>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8219566 >>>>>>>>>>> webrev: >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.00/ >>>>>>>> >>>>>>>> >>>>>>>> If you have any questions, please tell me. >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2019/02/27 22:46, Yasumasa Suenaga wrote: >>>>>>>>> Hi Erik, >>>>>>>>> >>>>>>>>> On 2019/02/27 22:30, Erik Gahlin wrote: >>>>>>>>>> Hi Yasumasa, >>>>>>>>>> >>>>>>>>>> Could you explain a little about the bug? >>>>>>>>>> >>>>>>>>>> Why should stack traces be collected if MaxJavaStackTraceDepth >>>>>>>>>> is set to 0. >>>>>>>>> >>>>>>>>> "0" in MaxJavaStackTraceDepth means all exception stack traces >>>>>>>>> will be shown. >>>>>>>>> >>>>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/72ce7dd54939/src/hotspot/share/runtime/globals.hpp#l1553 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> JDK-7179701 reports similar issue. >>>>>>>>> I think we can take similar approach for JDK-8219566. >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> Erik >>>>>>>>>> >>>>>>>>>>> On 22 Feb 2019, at 07:48, Yasumasa Suenaga >>>>>>>>>> gmail.com> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> Hi all, >>>>>>>>>>> >>>>>>>>>>> Please review this change: >>>>>>>>>>> >>>>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8219566 >>>>>>>>>>> webrev: >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.00/ >>>>>>>>>>> >>>>>>>>>>> JFR did not collect call stacks when MaxJavaStackTraceDepth is >>>>>>>>>>> set to zero. >>>>>>>>>>> This bug is similar to JDK-7179701. >>>>>>>>>>> >>>>>>>>>>> This change passed tests on submit repo and :jdk_jfr jtreg >>>>>>>>>>> tests. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>> >>> From mikhailo.seledtsov at oracle.com Fri Apr 12 23:54:33 2019 From: mikhailo.seledtsov at oracle.com (mikhailo.seledtsov at oracle.com) Date: Fri, 12 Apr 2019 16:54:33 -0700 Subject: RFR(M): 8221711: [TESTBUG] create more tests for JFR in container environment In-Reply-To: References: <5CB09725.5010108@oracle.com> Message-ID: <4fc68d8b-6d44-d478-3d6e-3936e1c84470@oracle.com> Hi Erik, ? Here is the updated webrev based on your feedback: http://cr.openjdk.java.net/~mseledtsov/8221711.02/ Misha On 4/12/19 7:58 AM, mikhailo.seledtsov at oracle.com wrote: > Hi Erik, > > ? Thank you for taking a look at the changes. Please see my comments > inline. > > > On 4/12/19 6:48 AM, Erik Gahlin wrote: >> Hi Misha, >> >> I'm a bit confused. I thought the purpose of JFrReporter was to write >> out the result of a recording in a container and let TestJFREvents do >> the actual verification by looking at the result from standard out. >> >> This doesn't seem to be the case with testEnvironmentVariables. It >> seems like it is doing both, or am I missing something? > Yes, you are right. I deviated from the original pattern, perhaps for > no good reason. >> >> Would it be possible to simplify the JFrReporter into something like >> this: >> >> public class JfrReporter { >> >> ? public static void main(String[] args) throws Exception { >> ??? String eventName = args[1]; >> ??? try(Recording r = new Recording()) { >> ????? r.enable(eventName); >> ????? r.start(); >> ????? r.stop(); >> ????? Path p = Paths.get("tmp", eventName + ".jfr"); >> ????? r.dump(p); >> ????? for (RecordedEvent e : RecordingFile.readAllEvents(p)) { >> ??????? for (ValueDescriptor v : e.getEventType().getFields()) { >> ????????? System.out.println(v.getName() + " = " + >> e.getValue(v.getName())); >> ??????? } >> ????? } >> ??? } >> ? } >> } >> >> and move all the logic into TestJFREvents? Maybe there is an argument >> against this, but I'm curious. > I can do that. Most likely it will work, with minor additions. > For instance, I would like to validate some prerequisite conditions > (or test assumptions), such as that environment variables are defined > inside a container: > > assertTrue(System.getenv(testEnvVar) == null, "Testcase > TEST_ENVIRONMENT_VARS failed:" + > 71 testEnvVar + " should not be defined inside a container"); > 72 assertTrue(System.getenv("JAVA_HOME") != null, > 73 "Testcase ENVIRONMENT_VARS failed: JAVA_HOME should be defined > inside a container"); > > ?If for some reason these assumptions are wrong, it is most likely due > to test bug or bug in other parts of JDK, and helps us isolate the > root cause. > > Other than that, your proposal for simplification makes sense. I will > implement it. >> In JfrNetwork.java, you wait for 5 seconds for a join. It seems >> brittle to depend on a thread getting scheduled or not. Could we >> create something more deterministic, and not depend on something >> happening within a specific time? > OK, this makes sense. I will remove the wait time. The join should > happen always; and if it does not, it indicates the error, which will > manifest the test timeout, and be followed by investigation. The only > reason IMO to use timed wait/join in such cases is to guard against > external/environmental circumstances that we have no control over. > E.g. we are waiting for another entity to accept a connection, I/O > that may take longer time due to system or HW factors, etc. In this > particular case looks like our code is in control, so it should be > fine to remove the timed wait. > > > I will make the changes, and post an updated webrev. > > Thank you, > Misha >> Thanks >> Erik >> >>> >>> >>> On 4/8/19 8:28 PM, mikhailo.seledtsov at oracle.com wrote: >>>> Please review this change that adds two more test cases for JFR >>>> events in the container environment: >>>> ? - making sure environment variables are reported correctly inside >>>> the container >>>> ? - host info is reported correctly inside the container >>>> >>>> ??? JBS: https://bugs.openjdk.java.net/browse/JDK-8221711 >>>> ??? Webrev: http://cr.openjdk.java.net/~mseledtsov/8221711.01/ >>>> ??? Testing: >>>> ????? Ran the affected tests on a single host - PASS >>>> ????? On multiple host variety - In PROGRESS >>>> >>>> >>>> Thank you, >>>> Misha >>>> >>> >> > From erik.gahlin at oracle.com Sat Apr 13 02:42:21 2019 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Sat, 13 Apr 2019 04:42:21 +0200 Subject: RFR(M): 8221711: [TESTBUG] create more tests for JFR in container environment In-Reply-To: <4fc68d8b-6d44-d478-3d6e-3936e1c84470@oracle.com> References: <5CB09725.5010108@oracle.com> <4fc68d8b-6d44-d478-3d6e-3936e1c84470@oracle.com> Message-ID: <31C14840-3506-4458-AFC7-F57C4523B095@oracle.com> Looks good! Thanks for fixing this. Erik > On 13 Apr 2019, at 01:54, mikhailo.seledtsov at oracle.com wrote: > > Hi Erik, > > Here is the updated webrev based on your feedback: > > http://cr.openjdk.java.net/~mseledtsov/8221711.02/ > > > Misha > > > On 4/12/19 7:58 AM, mikhailo.seledtsov at oracle.com wrote: >> Hi Erik, >> >> Thank you for taking a look at the changes. Please see my comments inline. >> >> >> On 4/12/19 6:48 AM, Erik Gahlin wrote: >>> Hi Misha, >>> >>> I'm a bit confused. I thought the purpose of JFrReporter was to write out the result of a recording in a container and let TestJFREvents do the actual verification by looking at the result from standard out. >>> >>> This doesn't seem to be the case with testEnvironmentVariables. It seems like it is doing both, or am I missing something? >> Yes, you are right. I deviated from the original pattern, perhaps for no good reason. >>> >>> Would it be possible to simplify the JFrReporter into something like this: >>> >>> public class JfrReporter { >>> >>> public static void main(String[] args) throws Exception { >>> String eventName = args[1]; >>> try(Recording r = new Recording()) { >>> r.enable(eventName); >>> r.start(); >>> r.stop(); >>> Path p = Paths.get("tmp", eventName + ".jfr"); >>> r.dump(p); >>> for (RecordedEvent e : RecordingFile.readAllEvents(p)) { >>> for (ValueDescriptor v : e.getEventType().getFields()) { >>> System.out.println(v.getName() + " = " + e.getValue(v.getName())); >>> } >>> } >>> } >>> } >>> } >>> >>> and move all the logic into TestJFREvents? Maybe there is an argument against this, but I'm curious. >> I can do that. Most likely it will work, with minor additions. >> For instance, I would like to validate some prerequisite conditions (or test assumptions), such as that environment variables are defined inside a container: >> >> assertTrue(System.getenv(testEnvVar) == null, "Testcase TEST_ENVIRONMENT_VARS failed:" + >> 71 testEnvVar + " should not be defined inside a container"); >> 72 assertTrue(System.getenv("JAVA_HOME") != null, >> 73 "Testcase ENVIRONMENT_VARS failed: JAVA_HOME should be defined inside a container"); >> >> If for some reason these assumptions are wrong, it is most likely due to test bug or bug in other parts of JDK, and helps us isolate the root cause. >> >> Other than that, your proposal for simplification makes sense. I will implement it. >>> In JfrNetwork.java, you wait for 5 seconds for a join. It seems brittle to depend on a thread getting scheduled or not. Could we create something more deterministic, and not depend on something happening within a specific time? >> OK, this makes sense. I will remove the wait time. The join should happen always; and if it does not, it indicates the error, which will manifest the test timeout, and be followed by investigation. The only reason IMO to use timed wait/join in such cases is to guard against external/environmental circumstances that we have no control over. E.g. we are waiting for another entity to accept a connection, I/O that may take longer time due to system or HW factors, etc. In this particular case looks like our code is in control, so it should be fine to remove the timed wait. >> >> >> I will make the changes, and post an updated webrev. >> >> Thank you, >> Misha >>> Thanks >>> Erik >>> >>>> >>>> >>>> On 4/8/19 8:28 PM, mikhailo.seledtsov at oracle.com wrote: >>>>> Please review this change that adds two more test cases for JFR events in the container environment: >>>>> - making sure environment variables are reported correctly inside the container >>>>> - host info is reported correctly inside the container >>>>> >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8221711 >>>>> Webrev: http://cr.openjdk.java.net/~mseledtsov/8221711.01/ >>>>> Testing: >>>>> Ran the affected tests on a single host - PASS >>>>> On multiple host variety - In PROGRESS >>>>> >>>>> >>>>> Thank you, >>>>> Misha >>>>> >>>> >>> >> > From yasuenag at gmail.com Sun Apr 14 13:05:22 2019 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Sun, 14 Apr 2019 22:05:22 +0900 Subject: PING: RFR: 8219566: JFR did not collect call stacks when MaxJavaStackTraceDepth is set to zero In-Reply-To: <5CB0D24B.305@oracle.com> References: <6D0A14CC-3EBA-4A01-AFFC-F2987F3C4DFD@oracle.com> <59ed9471-45a5-5707-8356-367ae35a7829@gmail.com> <19551e1b-b9e8-62e1-6fab-7dd6e100f50a@gmail.com> <5C828C38.1090506@oracle.com> <47d27b56-0933-c173-55c1-7e225218ead8@gmail.com> <0b31693a-a8d4-1fdf-c830-8752f3ec379e@gmail.com> <5C95369B.8030404@oracle.com> <7c203381-0091-f9a9-6e76-1b09d0e92c88@gmail.com> <2ca33c8e-6565-fad3-294c-ae1804271f54@gmail.com> <5CB0D24B.305@oracle.com> Message-ID: Hi Erik, How about this change? http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.03/ IMHO I prefer webrev.00 if we need to think stackdepth is set to lower value. Fix in webrev.00 is introduced in JDK-7179701. As you said before, it does not make the code easier to read. So if webrev.03 is appropriate, I will push webrev.03. Thanks, Yasumasa On 2019/04/13 3:00, Erik Gahlin wrote: > Hi, > > Could we use the max stack depth used by JFR instead? > > It's not available as constant in a header file, but maybe you could move: > > static const u4 STACK_DEPTH_DEFAULT = 64; > static const u4 MIN_STACK_DEPTH = 1; > static const u4 MAX_STACK_DEPTH = 2048; > > from jfrOptionSet.cpp to jfrTypes.hpp where we have other constants. The benefit of using MAX_STACK_DEPTH is that it will work in scenarios where a user has set the number of frames to a low? number, i.e -XX:FlightRecorderOptions=5, and we hit a stack with lots of frames that we can't record, i.e native frames. > > Thanks > Erik > >> Hi, >> >> Do you have any concern(s) about this change? >> >>>>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.02/ >> >> I think thread stack should be collected by JFR even if MaxJavaStackTraceDepth is set to zero. >> >> >> Thanks, >> >> Yasumasa >> >> >> On 2019/03/23 21:03, Yasumasa Suenaga wrote: >>> Hi Erik, >>> >>> MaxJavaStackTraceDepth do not only affect finding top stack but also stack walking. >>> >>> JfrGetCallTrace::find_top_frame() uses MaxJavaStackTraceDepth to find top frame as you say. >>> OTOH vframeStreamSamples::samples_next() uses it to find next valid frame. >>> >>> >>> Anyway thread stack should be collected even if MaxJavaStackTraceDepth is set to zero. >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2019/03/23 4:25, Erik Gahlin wrote: >>>> To me, it seems that the purpose of the method is to find the first >>>> valid frame on the stack (the top frame). This is orthogonal to the >>>> number of frames that should be recorded by JFR. >>>> >>>> Erik >>>> >>>>> PING: Could you review it? >>>>> >>>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8219566 >>>>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.02/ >>>>> >>>>> This change has passed :jdk_jfr tests and submit repo. >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2019/03/11 14:49, Yasumasa Suenaga wrote: >>>>>> 2019?3?9?(?) 20:47 Yasumasa Suenaga : >>>>>>> >>>>>>> Hi Erik, >>>>>>> >>>>>>> Reading your email, I think we should use JfrOptionSet::stackdepth() >>>>>>> rather than MaxJavaStackTraceDepth. >>>>>>> I created new webrev, and it passed jdk_jfr jtreg tests on my Linux >>>>>>> x64 box. >>>>>>> >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.01/ >>>>>>> >>>>>>> However it has not yet passed tests on submit repo >>>>>>> (mach5-one-ysuenaga-JDK-8219566-20190309-0909-1079793). >>>>>>> It seems to be failed at build phase which includes Linux. >>>>>>> (I can build with this change on Fedora 29 x64). >>>>>>> >>>>>>> Could you help if this webrev is ok? >>>>>> >>>>>> I found 1 warning, and I fixed it. >>>>>> This webrev works fine with :jdk_jfr jtreg tests on my Linux box, and >>>>>> submit repo test >>>>>> (mach5-one-ysuenaga-JDK-8219566-20190311-0150-1121570). >>>>>> Could you review it? >>>>>> >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.02/ >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 2019/03/09 0:37, Erik Gahlin wrote: >>>>>>>> JFR imposes a limit on how many frames that can be recorded, set to >>>>>>>> 64 by default but configurable up to 2048 using >>>>>>>> -XX:FlightRecorderOptions, so setting the value to 0 will still not >>>>>>>> mean all frames. >>>>>>>> >>>>>>>> That said, if you still want to fix it. It may make sense to set >>>>>>>> loop_max to INT_MAX, if MaxJavaStackTraceDepth is zero, i.e >>>>>>>> >>>>>>>> int loop_max = MaxJavaStackTraceDepth == 0 ? INT_MAX : >>>>>>>> MaxJavaStackTraceDepth * 2; >>>>>>>> >>>>>>>> and avoid evaluating it for every frame. Even if the compiler is >>>>>>>> able to hoist the comparison, it makes the code easier to read. >>>>>>>> >>>>>>>> Thanks >>>>>>>> Erik >>>>>>>> >>>>>>>> >>>>>>>>> PING: Could you review it? >>>>>>>>> >>>>>>>>>>>> ??? JBS: https://bugs.openjdk.java.net/browse/JDK-8219566 >>>>>>>>>>>> ??? webrev: >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.00/ >>>>>>>>> >>>>>>>>> >>>>>>>>> If you have any questions, please tell me. >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2019/02/27 22:46, Yasumasa Suenaga wrote: >>>>>>>>>> Hi Erik, >>>>>>>>>> >>>>>>>>>> On 2019/02/27 22:30, Erik Gahlin wrote: >>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>> >>>>>>>>>>> Could you explain a little about the bug? >>>>>>>>>>> >>>>>>>>>>> Why should stack traces be collected if MaxJavaStackTraceDepth >>>>>>>>>>> is set to 0. >>>>>>>>>> >>>>>>>>>> "0" in MaxJavaStackTraceDepth means all exception stack traces >>>>>>>>>> will be shown. >>>>>>>>>> >>>>>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/72ce7dd54939/src/hotspot/share/runtime/globals.hpp#l1553 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> JDK-7179701 reports similar issue. >>>>>>>>>> I think we can take similar approach for JDK-8219566. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> Thanks >>>>>>>>>>> Erik >>>>>>>>>>> >>>>>>>>>>>> On 22 Feb 2019, at 07:48, Yasumasa Suenaga >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi all, >>>>>>>>>>>> >>>>>>>>>>>> Please review this change: >>>>>>>>>>>> >>>>>>>>>>>> ??? JBS: https://bugs.openjdk.java.net/browse/JDK-8219566 >>>>>>>>>>>> ??? webrev: >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.00/ >>>>>>>>>>>> >>>>>>>>>>>> JFR did not collect call stacks when MaxJavaStackTraceDepth is >>>>>>>>>>>> set to zero. >>>>>>>>>>>> This bug is similar to JDK-7179701. >>>>>>>>>>>> >>>>>>>>>>>> This change passed tests on submit repo and :jdk_jfr jtreg tests. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>> >>>> > From yasuenag at gmail.com Sun Apr 14 13:09:52 2019 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Sun, 14 Apr 2019 22:09:52 +0900 Subject: PING: RFR: 8220657: JFR.dump does not work when filename is set In-Reply-To: <725f2ab6-71ee-4149-9d1d-ca6c19217408@gmail.com> References: <5C90E8C9.3010008@oracle.com> <14c04f3a-1fc0-af3b-714f-e35fa18d6a1b@gmail.com> <5C9103DE.4040304@oracle.com> <725f2ab6-71ee-4149-9d1d-ca6c19217408@gmail.com> Message-ID: <2dc4e611-bd18-a187-f529-d243af51cf10@gmail.com> PING: Do you have any opinion(s) about this fix? Yasumasa On 2019/04/01 21:46, Yasumasa Suenaga wrote: > Hi, > > Do you have concern(s) about this change? > >>>> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.01/ > > As in the email below, I think it does not affect scenarios except snapshot cloning. > Please tell me if you have any opinions about this. > > > Thanks, > > Yasumasa > > > On 2019/03/20 10:37, Yasumasa Suenaga wrote: >> Hi Erik, >> >> I think this change does not affect other scenarios. >> >> :jdk_jfr jtreg tests and submit repo have passed with this change. >> newSnapshotClone() is called by PlatformRecording::dump and >> DCmdDump::newSnapshot. >> Both methods are related to user-requested dump. >> >> What scenarios do you think? I want to check them. >> >> >> Thanks, >> >> Yasumasa >> >> >> 2019?3?19?(?) 23:59 Erik Gahlin : >>> >>> Yes, but then you would lose the original destination in other scenarios >>> >>> Maybe I can look at this next week. >>> >>> Erik >>> >>>> Hi Erik, >>>> >>>> Thank you for your reply. >>>> >>>> How about this changeset? >>>> >>>> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.01/ >>>> >>>> I think PlatformRecording#newSnapshotClone() should not dump. >>>> So I removed `setDestination()` from this. >>>> >>>> >>>> Yasumasa >>>> >>>> >>>> On 2019/03/19 22:04, Erik Gahlin wrote: >>>>> This looks a bit hackish. >>>>> >>>>> The real issue seems to be that the method >>>>> PlatformRecording#newSnapshotClone incorrectly sets the destination in >>>>> the clone, which then triggers a dump prematurely (when stop is called). >>>>> >>>>> Purpose of that method is just to take a snapshot of existing chunks in >>>>> the disk repository (or from memory). and then let dumpStopped(path) do >>>>> the actual dump (copy out the data from the disk repository). >>>>> >>>>> To make it work, some other changes would be needed as well, but I don't >>>>> have time to investigate. >>>>> >>>>> Erik >>>>> >>>>>> Hi all, >>>>>> >>>>>> Please review this change. >>>>>> >>>>>> ???? JBS: https://bugs.openjdk.java.net/browse/JDK-8220657 >>>>>> ???? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.00/ >>>>>> >>>>>> The user might not get flight record via JFR.dump if JFR is started >>>>>> with filename option on target process. >>>>>> >>>>>> Please see JBS if you want to know how to reproduce. >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>> From kustos at gmx.net Sun Apr 14 18:48:33 2019 From: kustos at gmx.net (Philippe Marschall) Date: Sun, 14 Apr 2019 20:48:33 +0200 Subject: Supported data types for events Message-ID: <700a9182-62f2-2555-0b6a-12c1e0a84596@gmx.net> Hello I'm struggling a bit to find information about the supported data types for custom events. What seems to be supported are the following. - the primitive types and their wrappers - java.lang.String - java.lang.Class But I'm not sure what is additionally supported, eg. java.lang.reflect.Method doesn't seem to be supported. Are some of the following supported - JSR-310 (Java Date Time API) - java.sql Date Time types - java.util.Calendar - arrays or collections of the above types Cheers Philippe From erik.gahlin at oracle.com Sun Apr 14 22:42:59 2019 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Mon, 15 Apr 2019 00:42:59 +0200 Subject: PING: RFR: 8219566: JFR did not collect call stacks when MaxJavaStackTraceDepth is set to zero In-Reply-To: References: <6D0A14CC-3EBA-4A01-AFFC-F2987F3C4DFD@oracle.com> <59ed9471-45a5-5707-8356-367ae35a7829@gmail.com> <19551e1b-b9e8-62e1-6fab-7dd6e100f50a@gmail.com> <5C828C38.1090506@oracle.com> <47d27b56-0933-c173-55c1-7e225218ead8@gmail.com> <0b31693a-a8d4-1fdf-c830-8752f3ec379e@gmail.com> <5C95369B.8030404@oracle.com> <7c203381-0091-f9a9-6e76-1b09d0e92c88@gmail.com> <2ca33c8e-6565-fad3-294c-ae1804271f54@gmail.com> <5CB0D24B.305@oracle.com> Message-ID: <537B942B-1E3B-42B6-A121-05B5D7CC9CD9@oracle.com> Looks good (webrev.03) According to the option spec. MaxJavaStackTraceDepth controls "the maximum number of lines in the stack trace for Java exceptions?. This is not an exception stack trace, so I think it would be more appropriate to use MAX_STACK_DEPTH. Thanks Erik > On 14 Apr 2019, at 15:05, Yasumasa Suenaga wrote: > > Hi Erik, > > How about this change? > http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.03/ > > IMHO I prefer webrev.00 if we need to think stackdepth is set to lower value. > Fix in webrev.00 is introduced in JDK-7179701. > > As you said before, it does not make the code easier to read. > > So if webrev.03 is appropriate, I will push webrev.03. > > > Thanks, > > Yasumasa > > > On 2019/04/13 3:00, Erik Gahlin wrote: >> Hi, >> Could we use the max stack depth used by JFR instead? >> It's not available as constant in a header file, but maybe you could move: >> static const u4 STACK_DEPTH_DEFAULT = 64; >> static const u4 MIN_STACK_DEPTH = 1; >> static const u4 MAX_STACK_DEPTH = 2048; >> from jfrOptionSet.cpp to jfrTypes.hpp where we have other constants. The benefit of using MAX_STACK_DEPTH is that it will work in scenarios where a user has set the number of frames to a low number, i.e -XX:FlightRecorderOptions=5, and we hit a stack with lots of frames that we can't record, i.e native frames. >> Thanks >> Erik >>> Hi, >>> >>> Do you have any concern(s) about this change? >>> >>>>>> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.02/ >>> >>> I think thread stack should be collected by JFR even if MaxJavaStackTraceDepth is set to zero. >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2019/03/23 21:03, Yasumasa Suenaga wrote: >>>> Hi Erik, >>>> >>>> MaxJavaStackTraceDepth do not only affect finding top stack but also stack walking. >>>> >>>> JfrGetCallTrace::find_top_frame() uses MaxJavaStackTraceDepth to find top frame as you say. >>>> OTOH vframeStreamSamples::samples_next() uses it to find next valid frame. >>>> >>>> >>>> Anyway thread stack should be collected even if MaxJavaStackTraceDepth is set to zero. >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2019/03/23 4:25, Erik Gahlin wrote: >>>>> To me, it seems that the purpose of the method is to find the first >>>>> valid frame on the stack (the top frame). This is orthogonal to the >>>>> number of frames that should be recorded by JFR. >>>>> >>>>> Erik >>>>> >>>>>> PING: Could you review it? >>>>>> >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8219566 >>>>>> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.02/ >>>>>> >>>>>> This change has passed :jdk_jfr tests and submit repo. >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2019/03/11 14:49, Yasumasa Suenaga wrote: >>>>>>> 2019?3?9?(?) 20:47 Yasumasa Suenaga : >>>>>>>> >>>>>>>> Hi Erik, >>>>>>>> >>>>>>>> Reading your email, I think we should use JfrOptionSet::stackdepth() >>>>>>>> rather than MaxJavaStackTraceDepth. >>>>>>>> I created new webrev, and it passed jdk_jfr jtreg tests on my Linux >>>>>>>> x64 box. >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.01/ >>>>>>>> >>>>>>>> However it has not yet passed tests on submit repo >>>>>>>> (mach5-one-ysuenaga-JDK-8219566-20190309-0909-1079793). >>>>>>>> It seems to be failed at build phase which includes Linux. >>>>>>>> (I can build with this change on Fedora 29 x64). >>>>>>>> >>>>>>>> Could you help if this webrev is ok? >>>>>>> >>>>>>> I found 1 warning, and I fixed it. >>>>>>> This webrev works fine with :jdk_jfr jtreg tests on my Linux box, and >>>>>>> submit repo test >>>>>>> (mach5-one-ysuenaga-JDK-8219566-20190311-0150-1121570). >>>>>>> Could you review it? >>>>>>> >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.02/ >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 2019/03/09 0:37, Erik Gahlin wrote: >>>>>>>>> JFR imposes a limit on how many frames that can be recorded, set to >>>>>>>>> 64 by default but configurable up to 2048 using >>>>>>>>> -XX:FlightRecorderOptions, so setting the value to 0 will still not >>>>>>>>> mean all frames. >>>>>>>>> >>>>>>>>> That said, if you still want to fix it. It may make sense to set >>>>>>>>> loop_max to INT_MAX, if MaxJavaStackTraceDepth is zero, i.e >>>>>>>>> >>>>>>>>> int loop_max = MaxJavaStackTraceDepth == 0 ? INT_MAX : >>>>>>>>> MaxJavaStackTraceDepth * 2; >>>>>>>>> >>>>>>>>> and avoid evaluating it for every frame. Even if the compiler is >>>>>>>>> able to hoist the comparison, it makes the code easier to read. >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> Erik >>>>>>>>> >>>>>>>>> >>>>>>>>>> PING: Could you review it? >>>>>>>>>> >>>>>>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8219566 >>>>>>>>>>>>> webrev: >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.00/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> If you have any questions, please tell me. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2019/02/27 22:46, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi Erik, >>>>>>>>>>> >>>>>>>>>>> On 2019/02/27 22:30, Erik Gahlin wrote: >>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>> >>>>>>>>>>>> Could you explain a little about the bug? >>>>>>>>>>>> >>>>>>>>>>>> Why should stack traces be collected if MaxJavaStackTraceDepth >>>>>>>>>>>> is set to 0. >>>>>>>>>>> >>>>>>>>>>> "0" in MaxJavaStackTraceDepth means all exception stack traces >>>>>>>>>>> will be shown. >>>>>>>>>>> >>>>>>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/72ce7dd54939/src/hotspot/share/runtime/globals.hpp#l1553 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> JDK-7179701 reports similar issue. >>>>>>>>>>> I think we can take similar approach for JDK-8219566. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> Thanks >>>>>>>>>>>> Erik >>>>>>>>>>>> >>>>>>>>>>>>> On 22 Feb 2019, at 07:48, Yasumasa Suenaga >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Hi all, >>>>>>>>>>>>> >>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>> >>>>>>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8219566 >>>>>>>>>>>>> webrev: >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.00/ >>>>>>>>>>>>> >>>>>>>>>>>>> JFR did not collect call stacks when MaxJavaStackTraceDepth is >>>>>>>>>>>>> set to zero. >>>>>>>>>>>>> This bug is similar to JDK-7179701. >>>>>>>>>>>>> >>>>>>>>>>>>> This change passed tests on submit repo and :jdk_jfr jtreg tests. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>> >>>>>>>>> >>>>> From yasuenag at gmail.com Sun Apr 14 23:14:26 2019 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Mon, 15 Apr 2019 08:14:26 +0900 Subject: PING: RFR: 8219566: JFR did not collect call stacks when MaxJavaStackTraceDepth is set to zero In-Reply-To: <537B942B-1E3B-42B6-A121-05B5D7CC9CD9@oracle.com> References: <6D0A14CC-3EBA-4A01-AFFC-F2987F3C4DFD@oracle.com> <59ed9471-45a5-5707-8356-367ae35a7829@gmail.com> <19551e1b-b9e8-62e1-6fab-7dd6e100f50a@gmail.com> <5C828C38.1090506@oracle.com> <47d27b56-0933-c173-55c1-7e225218ead8@gmail.com> <0b31693a-a8d4-1fdf-c830-8752f3ec379e@gmail.com> <5C95369B.8030404@oracle.com> <7c203381-0091-f9a9-6e76-1b09d0e92c88@gmail.com> <2ca33c8e-6565-fad3-294c-ae1804271f54@gmail.com> <5CB0D24B.305@oracle.com> <537B942B-1E3B-42B6-A121-05B5D7CC9CD9@oracle.com> Message-ID: Thanks Erik! I'm waiting for another reviewer for webrev.03 . Yasumasa 2019?4?15?(?) 7:43 Erik Gahlin : > Looks good (webrev.03) > > According to the option spec. MaxJavaStackTraceDepth controls "the maximum > number of lines in the stack trace for Java exceptions?. This is not an > exception stack trace, so I think it would be more appropriate to use > MAX_STACK_DEPTH. > > Thanks > Erik > > > > On 14 Apr 2019, at 15:05, Yasumasa Suenaga wrote: > > > > Hi Erik, > > > > How about this change? > > http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.03/ > > > > IMHO I prefer webrev.00 if we need to think stackdepth is set to lower > value. > > Fix in webrev.00 is introduced in JDK-7179701. > > > > As you said before, it does not make the code easier to read. > > > > So if webrev.03 is appropriate, I will push webrev.03. > > > > > > Thanks, > > > > Yasumasa > > > > > > On 2019/04/13 3:00, Erik Gahlin wrote: > >> Hi, > >> Could we use the max stack depth used by JFR instead? > >> It's not available as constant in a header file, but maybe you could > move: > >> static const u4 STACK_DEPTH_DEFAULT = 64; > >> static const u4 MIN_STACK_DEPTH = 1; > >> static const u4 MAX_STACK_DEPTH = 2048; > >> from jfrOptionSet.cpp to jfrTypes.hpp where we have other constants. > The benefit of using MAX_STACK_DEPTH is that it will work in scenarios > where a user has set the number of frames to a low number, i.e > -XX:FlightRecorderOptions=5, and we hit a stack with lots of frames that we > can't record, i.e native frames. > >> Thanks > >> Erik > >>> Hi, > >>> > >>> Do you have any concern(s) about this change? > >>> > >>>>>> webrev: > http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.02/ > >>> > >>> I think thread stack should be collected by JFR even if > MaxJavaStackTraceDepth is set to zero. > >>> > >>> > >>> Thanks, > >>> > >>> Yasumasa > >>> > >>> > >>> On 2019/03/23 21:03, Yasumasa Suenaga wrote: > >>>> Hi Erik, > >>>> > >>>> MaxJavaStackTraceDepth do not only affect finding top stack but also > stack walking. > >>>> > >>>> JfrGetCallTrace::find_top_frame() uses MaxJavaStackTraceDepth to find > top frame as you say. > >>>> OTOH vframeStreamSamples::samples_next() uses it to find next valid > frame. > >>>> > >>>> > >>>> Anyway thread stack should be collected even if > MaxJavaStackTraceDepth is set to zero. > >>>> > >>>> > >>>> Thanks, > >>>> > >>>> Yasumasa > >>>> > >>>> > >>>> On 2019/03/23 4:25, Erik Gahlin wrote: > >>>>> To me, it seems that the purpose of the method is to find the first > >>>>> valid frame on the stack (the top frame). This is orthogonal to the > >>>>> number of frames that should be recorded by JFR. > >>>>> > >>>>> Erik > >>>>> > >>>>>> PING: Could you review it? > >>>>>> > >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8219566 > >>>>>> webrev: > http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.02/ > >>>>>> > >>>>>> This change has passed :jdk_jfr tests and submit repo. > >>>>>> > >>>>>> > >>>>>> Thanks, > >>>>>> > >>>>>> Yasumasa > >>>>>> > >>>>>> > >>>>>> On 2019/03/11 14:49, Yasumasa Suenaga wrote: > >>>>>>> 2019?3?9?(?) 20:47 Yasumasa Suenaga : > >>>>>>>> > >>>>>>>> Hi Erik, > >>>>>>>> > >>>>>>>> Reading your email, I think we should use > JfrOptionSet::stackdepth() > >>>>>>>> rather than MaxJavaStackTraceDepth. > >>>>>>>> I created new webrev, and it passed jdk_jfr jtreg tests on my > Linux > >>>>>>>> x64 box. > >>>>>>>> > >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.01/ > >>>>>>>> > >>>>>>>> However it has not yet passed tests on submit repo > >>>>>>>> (mach5-one-ysuenaga-JDK-8219566-20190309-0909-1079793). > >>>>>>>> It seems to be failed at build phase which includes Linux. > >>>>>>>> (I can build with this change on Fedora 29 x64). > >>>>>>>> > >>>>>>>> Could you help if this webrev is ok? > >>>>>>> > >>>>>>> I found 1 warning, and I fixed it. > >>>>>>> This webrev works fine with :jdk_jfr jtreg tests on my Linux box, > and > >>>>>>> submit repo test > >>>>>>> (mach5-one-ysuenaga-JDK-8219566-20190311-0150-1121570). > >>>>>>> Could you review it? > >>>>>>> > >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.02/ > >>>>>>> > >>>>>>> > >>>>>>> Thanks, > >>>>>>> > >>>>>>> Yasumasa > >>>>>>> > >>>>>>> > >>>>>>>> Thanks, > >>>>>>>> > >>>>>>>> Yasumasa > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> On 2019/03/09 0:37, Erik Gahlin wrote: > >>>>>>>>> JFR imposes a limit on how many frames that can be recorded, set > to > >>>>>>>>> 64 by default but configurable up to 2048 using > >>>>>>>>> -XX:FlightRecorderOptions, so setting the value to 0 will still > not > >>>>>>>>> mean all frames. > >>>>>>>>> > >>>>>>>>> That said, if you still want to fix it. It may make sense to set > >>>>>>>>> loop_max to INT_MAX, if MaxJavaStackTraceDepth is zero, i.e > >>>>>>>>> > >>>>>>>>> int loop_max = MaxJavaStackTraceDepth == 0 ? INT_MAX : > >>>>>>>>> MaxJavaStackTraceDepth * 2; > >>>>>>>>> > >>>>>>>>> and avoid evaluating it for every frame. Even if the compiler is > >>>>>>>>> able to hoist the comparison, it makes the code easier to read. > >>>>>>>>> > >>>>>>>>> Thanks > >>>>>>>>> Erik > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> PING: Could you review it? > >>>>>>>>>> > >>>>>>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8219566 > >>>>>>>>>>>>> webrev: > >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.00/ > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> If you have any questions, please tell me. > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> Thanks, > >>>>>>>>>> > >>>>>>>>>> Yasumasa > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> On 2019/02/27 22:46, Yasumasa Suenaga wrote: > >>>>>>>>>>> Hi Erik, > >>>>>>>>>>> > >>>>>>>>>>> On 2019/02/27 22:30, Erik Gahlin wrote: > >>>>>>>>>>>> Hi Yasumasa, > >>>>>>>>>>>> > >>>>>>>>>>>> Could you explain a little about the bug? > >>>>>>>>>>>> > >>>>>>>>>>>> Why should stack traces be collected if MaxJavaStackTraceDepth > >>>>>>>>>>>> is set to 0. > >>>>>>>>>>> > >>>>>>>>>>> "0" in MaxJavaStackTraceDepth means all exception stack traces > >>>>>>>>>>> will be shown. > >>>>>>>>>>> > >>>>>>>>>>> > http://hg.openjdk.java.net/jdk/jdk/file/72ce7dd54939/src/hotspot/share/runtime/globals.hpp#l1553 > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> JDK-7179701 reports similar issue. > >>>>>>>>>>> I think we can take similar approach for JDK-8219566. > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> Thanks, > >>>>>>>>>>> > >>>>>>>>>>> Yasumasa > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> Thanks > >>>>>>>>>>>> Erik > >>>>>>>>>>>> > >>>>>>>>>>>>> On 22 Feb 2019, at 07:48, Yasumasa Suenaga gmail.com> > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>> Hi all, > >>>>>>>>>>>>> > >>>>>>>>>>>>> Please review this change: > >>>>>>>>>>>>> > >>>>>>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8219566 > >>>>>>>>>>>>> webrev: > >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.00/ > >>>>>>>>>>>>> > >>>>>>>>>>>>> JFR did not collect call stacks when MaxJavaStackTraceDepth > is > >>>>>>>>>>>>> set to zero. > >>>>>>>>>>>>> This bug is similar to JDK-7179701. > >>>>>>>>>>>>> > >>>>>>>>>>>>> This change passed tests on submit repo and :jdk_jfr jtreg > tests. > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>> > >>>>>>>>>>>>> Yasumasa > >>>>>>>>>>>> > >>>>>>>>> > >>>>> > > From robbin.ehn at oracle.com Mon Apr 15 08:58:34 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Mon, 15 Apr 2019 10:58:34 +0200 Subject: RFR(s): 8218147: make_walkable asserts on multiple calls In-Reply-To: <24A2B80F-D3D4-486A-9E29-8D48D49DF030@oracle.com> References: <48b0d1d0-3b12-e08d-f606-bf96a8445bf8@oracle.com> <587d1afe-9f86-f801-92de-31711b711034@oracle.com> <24A2B80F-D3D4-486A-9E29-8D48D49DF030@oracle.com> Message-ID: Hi, please review. After reexamine this issue: Threads in native must always have their stack walkable. JFR sampler should never need to make a stack walkable (for native sample). I manage to locally reproduce reliable with changes to JFR sampler and having hundreds of threads running similar code as the in the bug. (Looping creating an array with negative size.) I found a place where we don't proper look at the suspend flags. The java thread can thus escape native and make it's stack unwalkable and later it tries to make it walkable at the same time as the JFR sampler. By removing some kind of fast check and instead always call the check_safepoint_and_suspend_for_native_trans I can no longer reproduce. (which have the JFR native trans suspend check) And it passes t1-5. Code: http://cr.openjdk.java.net/~rehn/8218147/v3/webrev/ Issue: https://bugs.openjdk.java.net/browse/JDK-8218147 Thanks, Robbin On 4/5/19 5:43 PM, Robbin Ehn wrote: > Hi Dean, > > Sorry, I missed this mail. > Yes we can do that. > Ignore my other mail, I'll update. > > Thanks, Robbin > > > dean.long at oracle.com skrev: (5 april 2019 09:22:24 CEST) >> On 4/4/19 5:16 PM, dean.long at oracle.com wrote: >>> >>>>> >>>>> If it's already set, should we check that _last_Java_pc matches the >> >>>>> new value? >>>> >>>> We manually set the pc in several places, so if it's set, it's not >>>> certain that >>>> it should be the same as in last sp. >>>> I can't distinguish between the cases. >>>> >>> >>> If we get pc from sp[-1] then it should match, but you're right, we >>> sometimes get pc from somewhere else. >> >> How about if we combine the !walkable check and the >> capture_last_Java_pc() logic into a single method? >> Then we can do something like: >> >> ??? if (!walkable()) { >> ??????? address pc = (address)_last_Java_sp[-1]; >> ??????? address a = Atomic::cmpxchg(pc, &_last_Java_pc, NULL); >> ??????? assert(a == NULL || a == pc, "unexpected PC %p", a); >> ??? } >> >> dl From erik.gahlin at oracle.com Mon Apr 15 09:19:08 2019 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Mon, 15 Apr 2019 11:19:08 +0200 Subject: Supported data types for events In-Reply-To: <700a9182-62f2-2555-0b6a-12c1e0a84596@gmx.net> References: <700a9182-62f2-2555-0b6a-12c1e0a84596@gmx.net> Message-ID: Hi Philippe, The supported data types in Java are primitives, java.lang.Class, java.lang.Thread and java.lang.String. You can see the list in the toType method here http://hg.openjdk.java.net/jdk/jdk/file/tip/src/jdk.jfr/share/classes/jdk/jfr/internal/ASMToolkit.java You can't use JSR-310 types, but it is possible to store durations and instants using the annotations @Timespan and @Timestamp on numeric fields. In native, arrays are supported and it is possible to define you own composite data types, for example VirtualSpace You can see the list here: http://hg.openjdk.java.net/jdk/jdk/file/tip/src/hotspot/share/jfr/metadata/metadata.xml The file format only support ?value types?, so it is not possible to store an array with different data types per element. Cheers Erik > On 14 Apr 2019, at 20:48, Philippe Marschall wrote: > > Hello > > I'm struggling a bit to find information about the supported data types > for custom events. What seems to be supported are the following. > > - the primitive types and their wrappers > - java.lang.String > - java.lang.Class > > But I'm not sure what is additionally supported, eg. > java.lang.reflect.Method doesn't seem to be supported. > > Are some of the following supported > - JSR-310 (Java Date Time API) > - java.sql Date Time types > - java.util.Calendar > - arrays or collections of the above types > > Cheers > Philippe From mikhailo.seledtsov at oracle.com Mon Apr 15 18:59:28 2019 From: mikhailo.seledtsov at oracle.com (mikhailo.seledtsov at oracle.com) Date: Mon, 15 Apr 2019 11:59:28 -0700 Subject: RFR(M): 8221711: [TESTBUG] create more tests for JFR in container environment In-Reply-To: <31C14840-3506-4458-AFC7-F57C4523B095@oracle.com> References: <5CB09725.5010108@oracle.com> <4fc68d8b-6d44-d478-3d6e-3936e1c84470@oracle.com> <31C14840-3506-4458-AFC7-F57C4523B095@oracle.com> Message-ID: Thank you Erik for review. On 4/12/19 7:42 PM, Erik Gahlin wrote: > Looks good! > > Thanks for fixing this. > > Erik > >> On 13 Apr 2019, at 01:54, mikhailo.seledtsov at oracle.com wrote: >> >> Hi Erik, >> >> Here is the updated webrev based on your feedback: >> >> http://cr.openjdk.java.net/~mseledtsov/8221711.02/ >> >> >> Misha >> >> >> On 4/12/19 7:58 AM, mikhailo.seledtsov at oracle.com wrote: >>> Hi Erik, >>> >>> Thank you for taking a look at the changes. Please see my comments inline. >>> >>> >>> On 4/12/19 6:48 AM, Erik Gahlin wrote: >>>> Hi Misha, >>>> >>>> I'm a bit confused. I thought the purpose of JFrReporter was to write out the result of a recording in a container and let TestJFREvents do the actual verification by looking at the result from standard out. >>>> >>>> This doesn't seem to be the case with testEnvironmentVariables. It seems like it is doing both, or am I missing something? >>> Yes, you are right. I deviated from the original pattern, perhaps for no good reason. >>>> Would it be possible to simplify the JFrReporter into something like this: >>>> >>>> public class JfrReporter { >>>> >>>> public static void main(String[] args) throws Exception { >>>> String eventName = args[1]; >>>> try(Recording r = new Recording()) { >>>> r.enable(eventName); >>>> r.start(); >>>> r.stop(); >>>> Path p = Paths.get("tmp", eventName + ".jfr"); >>>> r.dump(p); >>>> for (RecordedEvent e : RecordingFile.readAllEvents(p)) { >>>> for (ValueDescriptor v : e.getEventType().getFields()) { >>>> System.out.println(v.getName() + " = " + e.getValue(v.getName())); >>>> } >>>> } >>>> } >>>> } >>>> } >>>> >>>> and move all the logic into TestJFREvents? Maybe there is an argument against this, but I'm curious. >>> I can do that. Most likely it will work, with minor additions. >>> For instance, I would like to validate some prerequisite conditions (or test assumptions), such as that environment variables are defined inside a container: >>> >>> assertTrue(System.getenv(testEnvVar) == null, "Testcase TEST_ENVIRONMENT_VARS failed:" + >>> 71 testEnvVar + " should not be defined inside a container"); >>> 72 assertTrue(System.getenv("JAVA_HOME") != null, >>> 73 "Testcase ENVIRONMENT_VARS failed: JAVA_HOME should be defined inside a container"); >>> >>> If for some reason these assumptions are wrong, it is most likely due to test bug or bug in other parts of JDK, and helps us isolate the root cause. >>> >>> Other than that, your proposal for simplification makes sense. I will implement it. >>>> In JfrNetwork.java, you wait for 5 seconds for a join. It seems brittle to depend on a thread getting scheduled or not. Could we create something more deterministic, and not depend on something happening within a specific time? >>> OK, this makes sense. I will remove the wait time. The join should happen always; and if it does not, it indicates the error, which will manifest the test timeout, and be followed by investigation. The only reason IMO to use timed wait/join in such cases is to guard against external/environmental circumstances that we have no control over. E.g. we are waiting for another entity to accept a connection, I/O that may take longer time due to system or HW factors, etc. In this particular case looks like our code is in control, so it should be fine to remove the timed wait. >>> >>> >>> I will make the changes, and post an updated webrev. >>> >>> Thank you, >>> Misha >>>> Thanks >>>> Erik >>>> >>>>> >>>>> On 4/8/19 8:28 PM, mikhailo.seledtsov at oracle.com wrote: >>>>>> Please review this change that adds two more test cases for JFR events in the container environment: >>>>>> - making sure environment variables are reported correctly inside the container >>>>>> - host info is reported correctly inside the container >>>>>> >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8221711 >>>>>> Webrev: http://cr.openjdk.java.net/~mseledtsov/8221711.01/ >>>>>> Testing: >>>>>> Ran the affected tests on a single host - PASS >>>>>> On multiple host variety - In PROGRESS >>>>>> >>>>>> >>>>>> Thank you, >>>>>> Misha >>>>>> From david.holmes at oracle.com Tue Apr 16 02:00:58 2019 From: david.holmes at oracle.com (David Holmes) Date: Tue, 16 Apr 2019 12:00:58 +1000 Subject: RFR(s): 8218147: make_walkable asserts on multiple calls In-Reply-To: References: <48b0d1d0-3b12-e08d-f606-bf96a8445bf8@oracle.com> <587d1afe-9f86-f801-92de-31711b711034@oracle.com> <24A2B80F-D3D4-486A-9E29-8D48D49DF030@oracle.com> Message-ID: Hi Robbin, On 15/04/2019 6:58 pm, Robbin Ehn wrote: > Hi, please review. > > After reexamine this issue: > Threads in native must always have their stack walkable. > JFR sampler should never need to make a stack walkable (for native sample). > > I manage to locally reproduce reliable with changes to JFR sampler and > having > hundreds of threads running similar code as the in the bug. > (Looping creating an array with negative size.) > > I found a place where we don't proper look at the suspend flags. > The java thread can thus escape native and make it's stack unwalkable > and later > it tries to make it walkable at the same time as the JFR sampler. > > By removing some kind of fast check and instead always call the > check_safepoint_and_suspend_for_native_trans I can no longer reproduce. Sorry but I can't see how this can fix anything: - if (SafepointMechanism::should_block(thread) || thread->is_suspend_after_native()) { JavaThread::check_safepoint_and_suspend_for_native_trans(thread); - } All you are doing is changing the timing of the race between the thread re-entering the VM/Java and the request for a suspend or safepoint. If there is a race between the sampler logic acting on the thread, and the thread acting on itself then that race has to be precluded somehow. Thanks, David ----- > (which have the JFR native trans suspend check) > And it passes t1-5. > > Code: > http://cr.openjdk.java.net/~rehn/8218147/v3/webrev/ > Issue: > https://bugs.openjdk.java.net/browse/JDK-8218147 > > Thanks, Robbin > > On 4/5/19 5:43 PM, Robbin Ehn wrote: >> Hi Dean, >> >> Sorry, I missed this mail. >> Yes we can do that. >> Ignore my other mail, I'll update. >> >> Thanks, Robbin >> >> >> dean.long at oracle.com skrev: (5 april 2019 09:22:24 CEST) >>> On 4/4/19 5:16 PM, dean.long at oracle.com wrote: >>>> >>>>>> >>>>>> If it's already set, should we check that _last_Java_pc matches the >>> >>>>>> new value? >>>>> >>>>> We manually set the pc in several places, so if it's set, it's not >>>>> certain that >>>>> it should be the same as in last sp. >>>>> I can't distinguish between the cases. >>>>> >>>> >>>> If we get pc from sp[-1] then it should match, but you're right, we >>>> sometimes get pc from somewhere else. >>> >>> How about if we combine the !walkable check and the >>> capture_last_Java_pc() logic into a single method? >>> Then we can do something like: >>> >>> ???? if (!walkable()) { >>> ???????? address pc = (address)_last_Java_sp[-1]; >>> ???????? address a = Atomic::cmpxchg(pc, &_last_Java_pc, NULL); >>> ???????? assert(a == NULL || a == pc, "unexpected PC %p", a); >>> ???? } >>> >>> dl From robbin.ehn at oracle.com Tue Apr 16 06:51:17 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 16 Apr 2019 08:51:17 +0200 Subject: RFR(s): 8218147: make_walkable asserts on multiple calls In-Reply-To: References: <48b0d1d0-3b12-e08d-f606-bf96a8445bf8@oracle.com> <587d1afe-9f86-f801-92de-31711b711034@oracle.com> <24A2B80F-D3D4-486A-9E29-8D48D49DF030@oracle.com> Message-ID: Hi David, On 4/16/19 4:00 AM, David Holmes wrote: > Hi Robbin, > > On 15/04/2019 6:58 pm, Robbin Ehn wrote: >> Hi, please review. >> >> After reexamine this issue: >> Threads in native must always have their stack walkable. >> JFR sampler should never need to make a stack walkable (for native sample). >> >> I manage to locally reproduce reliable with changes to JFR sampler and having >> hundreds of threads running similar code as the in the bug. >> (Looping creating an array with negative size.) >> >> I found a place where we don't proper look at the suspend flags. >> The java thread can thus escape native and make it's stack unwalkable and later >> it tries to make it walkable at the same time as the JFR sampler. >> >> By removing some kind of fast check and instead always call the >> check_safepoint_and_suspend_for_native_trans I can no longer reproduce. > > Sorry but I can't see how this can fix anything: > > -???? if (SafepointMechanism::should_block(thread) || > thread->is_suspend_after_native()) { > ??????? JavaThread::check_safepoint_and_suspend_for_native_trans(thread); > -???? } > In check_safepoint_and_suspend_for_native_trans we check _trace_flag and stop with macro: JFR_ONLY(SUSPEND_THREAD_CONDITIONAL(thread);) This method does not do the right thing: bool is_suspend_after_native() const { return (_suspend_flags & (_external_suspend | _deopt_suspend)) != 0; } If we want to keep the out-of-line double checking, this is an alternative: diff -r 51211b2d6514 src/hotspot/share/runtime/thread.hpp --- a/src/hotspot/share/runtime/thread.hpp Tue Apr 16 08:38:32 2019 +0200 +++ b/src/hotspot/share/runtime/thread.hpp Tue Apr 16 08:44:21 2019 +0200 @@ -1417,3 +1417,3 @@ bool is_suspend_after_native() const { - return (_suspend_flags & (_external_suspend | _deopt_suspend)) != 0; + return (_suspend_flags & (_external_suspend | _deopt_suspend | _trace_flag)) != 0; } So we are missing checking that bit completely in this transition code. Thanks, Robbin > All you are doing is changing the timing of the race between the thread > re-entering the VM/Java and the request for a suspend or safepoint. > > If there is a race between the sampler logic acting on the thread, and the > thread acting on itself then that race has to be precluded somehow. > > Thanks, > David > ----- > >> (which have the JFR native trans suspend check) >> And it passes t1-5. >> >> Code: >> http://cr.openjdk.java.net/~rehn/8218147/v3/webrev/ >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8218147 >> >> Thanks, Robbin >> >> On 4/5/19 5:43 PM, Robbin Ehn wrote: >>> Hi Dean, >>> >>> Sorry, I missed this mail. >>> Yes we can do that. >>> Ignore my other mail, I'll update. >>> >>> Thanks, Robbin >>> >>> >>> dean.long at oracle.com skrev: (5 april 2019 09:22:24 CEST) >>>> On 4/4/19 5:16 PM, dean.long at oracle.com wrote: >>>>> >>>>>>> >>>>>>> If it's already set, should we check that _last_Java_pc matches the >>>> >>>>>>> new value? >>>>>> >>>>>> We manually set the pc in several places, so if it's set, it's not >>>>>> certain that >>>>>> it should be the same as in last sp. >>>>>> I can't distinguish between the cases. >>>>>> >>>>> >>>>> If we get pc from sp[-1] then it should match, but you're right, we >>>>> sometimes get pc from somewhere else. >>>> >>>> How about if we combine the !walkable check and the >>>> capture_last_Java_pc() logic into a single method? >>>> Then we can do something like: >>>> >>>> ???? if (!walkable()) { >>>> ???????? address pc = (address)_last_Java_sp[-1]; >>>> ???????? address a = Atomic::cmpxchg(pc, &_last_Java_pc, NULL); >>>> ???????? assert(a == NULL || a == pc, "unexpected PC %p", a); >>>> ???? } >>>> >>>> dl From david.holmes at oracle.com Tue Apr 16 07:27:19 2019 From: david.holmes at oracle.com (David Holmes) Date: Tue, 16 Apr 2019 17:27:19 +1000 Subject: RFR(s): 8218147: make_walkable asserts on multiple calls In-Reply-To: References: <48b0d1d0-3b12-e08d-f606-bf96a8445bf8@oracle.com> <587d1afe-9f86-f801-92de-31711b711034@oracle.com> <24A2B80F-D3D4-486A-9E29-8D48D49DF030@oracle.com> Message-ID: <3d0d841d-3bcf-ce3c-1d44-096c9aff2e72@oracle.com> On 16/04/2019 4:51 pm, Robbin Ehn wrote: > Hi David, > > On 4/16/19 4:00 AM, David Holmes wrote: >> Hi Robbin, >> >> On 15/04/2019 6:58 pm, Robbin Ehn wrote: >>> Hi, please review. >>> >>> After reexamine this issue: >>> Threads in native must always have their stack walkable. >>> JFR sampler should never need to make a stack walkable (for native >>> sample). >>> >>> I manage to locally reproduce reliable with changes to JFR sampler >>> and having >>> hundreds of threads running similar code as the in the bug. >>> (Looping creating an array with negative size.) >>> >>> I found a place where we don't proper look at the suspend flags. >>> The java thread can thus escape native and make it's stack unwalkable >>> and later >>> it tries to make it walkable at the same time as the JFR sampler. >>> >>> By removing some kind of fast check and instead always call the >>> check_safepoint_and_suspend_for_native_trans I can no longer reproduce. >> >> Sorry but I can't see how this can fix anything: >> >> -???? if (SafepointMechanism::should_block(thread) || >> thread->is_suspend_after_native()) { >> >> JavaThread::check_safepoint_and_suspend_for_native_trans(thread); >> -???? } >> > > In check_safepoint_and_suspend_for_native_trans we check _trace_flag and > stop with macro: > JFR_ONLY(SUSPEND_THREAD_CONDITIONAL(thread);) Ah I see - needed to expand that macro. > This method does not do the right thing: > bool is_suspend_after_native() const { > ? return (_suspend_flags & (_external_suspend | _deopt_suspend)) != 0; > } > > If we want to keep the out-of-line double checking, this is an alternative: > diff -r 51211b2d6514 src/hotspot/share/runtime/thread.hpp > --- a/src/hotspot/share/runtime/thread.hpp??? Tue Apr 16 08:38:32 2019 > +0200 > +++ b/src/hotspot/share/runtime/thread.hpp??? Tue Apr 16 08:44:21 2019 > +0200 > @@ -1417,3 +1417,3 @@ > ?? bool is_suspend_after_native() const { > -??? return (_suspend_flags & (_external_suspend | _deopt_suspend)) != 0; > +??? return (_suspend_flags & (_external_suspend | _deopt_suspend | > _trace_flag)) != 0; > ?? } Right. Should that use JFR_ONLY? I was a little concerned that thread->set_trace_flag() may not ensure visibility of the flag update, but then realized it should be covered by the fence: static inline void transition_from_native(JavaThread *thread, JavaThreadState to) { // Change to transition state and ensure it is seen by the VM thread. thread->set_thread_state_fence(_thread_in_native_trans); and the comment should say: // Change to transition state and ensure it is seen by other thread, // and we will see any _suspend_flag changes below. However it also seems to me that in JfrThreadSampleClosure::do_sample_thread we need a storeload() barrier after: thread->set_trace_flag(); to ensure its not reordered with the reads of thread->thread_state() ? Thanks, David ----- > So we are missing checking that bit completely in this transition code. > > Thanks, Robbin > > >> All you are doing is changing the timing of the race between the >> thread re-entering the VM/Java and the request for a suspend or >> safepoint. >> >> If there is a race between the sampler logic acting on the thread, and >> the thread acting on itself then that race has to be precluded somehow. >> >> Thanks, >> David >> ----- >> >>> (which have the JFR native trans suspend check) >>> And it passes t1-5. >>> >>> Code: >>> http://cr.openjdk.java.net/~rehn/8218147/v3/webrev/ >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8218147 >>> >>> Thanks, Robbin >>> >>> On 4/5/19 5:43 PM, Robbin Ehn wrote: >>>> Hi Dean, >>>> >>>> Sorry, I missed this mail. >>>> Yes we can do that. >>>> Ignore my other mail, I'll update. >>>> >>>> Thanks, Robbin >>>> >>>> >>>> dean.long at oracle.com skrev: (5 april 2019 09:22:24 CEST) >>>>> On 4/4/19 5:16 PM, dean.long at oracle.com wrote: >>>>>> >>>>>>>> >>>>>>>> If it's already set, should we check that _last_Java_pc matches the >>>>> >>>>>>>> new value? >>>>>>> >>>>>>> We manually set the pc in several places, so if it's set, it's not >>>>>>> certain that >>>>>>> it should be the same as in last sp. >>>>>>> I can't distinguish between the cases. >>>>>>> >>>>>> >>>>>> If we get pc from sp[-1] then it should match, but you're right, we >>>>>> sometimes get pc from somewhere else. >>>>> >>>>> How about if we combine the !walkable check and the >>>>> capture_last_Java_pc() logic into a single method? >>>>> Then we can do something like: >>>>> >>>>> ???? if (!walkable()) { >>>>> ???????? address pc = (address)_last_Java_sp[-1]; >>>>> ???????? address a = Atomic::cmpxchg(pc, &_last_Java_pc, NULL); >>>>> ???????? assert(a == NULL || a == pc, "unexpected PC %p", a); >>>>> ???? } >>>>> >>>>> dl From robbin.ehn at oracle.com Tue Apr 16 07:56:42 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 16 Apr 2019 09:56:42 +0200 Subject: RFR(s): 8218147: make_walkable asserts on multiple calls In-Reply-To: <3d0d841d-3bcf-ce3c-1d44-096c9aff2e72@oracle.com> References: <48b0d1d0-3b12-e08d-f606-bf96a8445bf8@oracle.com> <587d1afe-9f86-f801-92de-31711b711034@oracle.com> <24A2B80F-D3D4-486A-9E29-8D48D49DF030@oracle.com> <3d0d841d-3bcf-ce3c-1d44-096c9aff2e72@oracle.com> Message-ID: Hi David, *truncated* On 4/16/19 9:27 AM, David Holmes wrote: >> If we want to keep the out-of-line double checking, this is an alternative: >> diff -r 51211b2d6514 src/hotspot/share/runtime/thread.hpp >> --- a/src/hotspot/share/runtime/thread.hpp??? Tue Apr 16 08:38:32 2019 +0200 >> +++ b/src/hotspot/share/runtime/thread.hpp??? Tue Apr 16 08:44:21 2019 +0200 >> @@ -1417,3 +1417,3 @@ >> ??? bool is_suspend_after_native() const { >> -??? return (_suspend_flags & (_external_suspend | _deopt_suspend)) != 0; >> +??? return (_suspend_flags & (_external_suspend | _deopt_suspend | >> _trace_flag)) != 0; >> ??? } > So you prefer this patch? > Right. Should that use JFR_ONLY? _trace_flag is always present, so we don't need it. And I'm not sure how to get that macro into that method in a nice way? > > I was a little concerned that thread->set_trace_flag() may not ensure visibility > of the flag update, but then realized it should be covered by the fence: > > ?static inline void transition_from_native(JavaThread *thread, JavaThreadState > to) { > ??? // Change to transition state and ensure it is seen by the VM thread. > ??? thread->set_thread_state_fence(_thread_in_native_trans); > > and the comment should say: > > // Change to transition state and ensure it is seen by other thread, > // and we will see any _suspend_flag changes below. > > However it also seems to me that in JfrThreadSampleClosure::do_sample_thread we > need a storeload() barrier after: > > ? thread->set_trace_flag(); > > to ensure its not reordered with the reads of thread->thread_state() ? Setting/clearing suspend flags is always done with Atomic::cmpxchg, since there can be multiple threads manipulating the bit pattern. I can add a comment about it. Thanks, Robbin > > Thanks, > David > ----- > > >> So we are missing checking that bit completely in this transition code. >> >> Thanks, Robbin >> >> >>> All you are doing is changing the timing of the race between the thread >>> re-entering the VM/Java and the request for a suspend or safepoint. >>> >>> If there is a race between the sampler logic acting on the thread, and the >>> thread acting on itself then that race has to be precluded somehow. >>> >>> Thanks, >>> David >>> ----- >>> >>>> (which have the JFR native trans suspend check) >>>> And it passes t1-5. >>>> >>>> Code: >>>> http://cr.openjdk.java.net/~rehn/8218147/v3/webrev/ >>>> Issue: >>>> https://bugs.openjdk.java.net/browse/JDK-8218147 >>>> >>>> Thanks, Robbin >>>> >>>> On 4/5/19 5:43 PM, Robbin Ehn wrote: >>>>> Hi Dean, >>>>> >>>>> Sorry, I missed this mail. >>>>> Yes we can do that. >>>>> Ignore my other mail, I'll update. >>>>> >>>>> Thanks, Robbin >>>>> >>>>> >>>>> dean.long at oracle.com skrev: (5 april 2019 09:22:24 CEST) >>>>>> On 4/4/19 5:16 PM, dean.long at oracle.com wrote: >>>>>>> >>>>>>>>> >>>>>>>>> If it's already set, should we check that _last_Java_pc matches the >>>>>> >>>>>>>>> new value? >>>>>>>> >>>>>>>> We manually set the pc in several places, so if it's set, it's not >>>>>>>> certain that >>>>>>>> it should be the same as in last sp. >>>>>>>> I can't distinguish between the cases. >>>>>>>> >>>>>>> >>>>>>> If we get pc from sp[-1] then it should match, but you're right, we >>>>>>> sometimes get pc from somewhere else. >>>>>> >>>>>> How about if we combine the !walkable check and the >>>>>> capture_last_Java_pc() logic into a single method? >>>>>> Then we can do something like: >>>>>> >>>>>> ???? if (!walkable()) { >>>>>> ???????? address pc = (address)_last_Java_sp[-1]; >>>>>> ???????? address a = Atomic::cmpxchg(pc, &_last_Java_pc, NULL); >>>>>> ???????? assert(a == NULL || a == pc, "unexpected PC %p", a); >>>>>> ???? } >>>>>> >>>>>> dl From david.holmes at oracle.com Tue Apr 16 08:21:44 2019 From: david.holmes at oracle.com (David Holmes) Date: Tue, 16 Apr 2019 18:21:44 +1000 Subject: RFR(s): 8218147: make_walkable asserts on multiple calls In-Reply-To: References: <48b0d1d0-3b12-e08d-f606-bf96a8445bf8@oracle.com> <587d1afe-9f86-f801-92de-31711b711034@oracle.com> <24A2B80F-D3D4-486A-9E29-8D48D49DF030@oracle.com> <3d0d841d-3bcf-ce3c-1d44-096c9aff2e72@oracle.com> Message-ID: <40b529d0-aeca-24b8-636a-518b7445b593@oracle.com> On 16/04/2019 5:56 pm, Robbin Ehn wrote: > Hi David, > > *truncated* > > On 4/16/19 9:27 AM, David Holmes wrote: >>> If we want to keep the out-of-line double checking, this is an >>> alternative: >>> diff -r 51211b2d6514 src/hotspot/share/runtime/thread.hpp >>> --- a/src/hotspot/share/runtime/thread.hpp??? Tue Apr 16 08:38:32 >>> 2019 +0200 >>> +++ b/src/hotspot/share/runtime/thread.hpp??? Tue Apr 16 08:44:21 >>> 2019 +0200 >>> @@ -1417,3 +1417,3 @@ >>> ??? bool is_suspend_after_native() const { >>> -??? return (_suspend_flags & (_external_suspend | _deopt_suspend)) >>> != 0; >>> +??? return (_suspend_flags & (_external_suspend | _deopt_suspend | >>> _trace_flag)) != 0; >>> ??? } >> > > So you prefer this patch? Yes > >> Right. Should that use JFR_ONLY? > > _trace_flag is always present, so we don't need it. Okay ... not clear what will set it other than JFR ... > And I'm not sure how to get that macro into that method in a nice way? Define nice ;-) return (_suspend_flags & (_external_suspend | _deopt_suspend JFR_ONLY(| _trace_flag))) != 0; >> >> I was a little concerned that thread->set_trace_flag() may not ensure >> visibility of the flag update, but then realized it should be covered >> by the fence: >> >> ??static inline void transition_from_native(JavaThread *thread, >> JavaThreadState to) { >> ???? // Change to transition state and ensure it is seen by the VM >> thread. >> ???? thread->set_thread_state_fence(_thread_in_native_trans); >> >> and the comment should say: >> >> // Change to transition state and ensure it is seen by other thread, >> // and we will see any _suspend_flag changes below. >> >> However it also seems to me that in >> JfrThreadSampleClosure::do_sample_thread we need a storeload() barrier >> after: >> >> ?? thread->set_trace_flag(); >> >> to ensure its not reordered with the reads of thread->thread_state() ? > > Setting/clearing suspend flags is always done with Atomic::cmpxchg, > since there can be multiple threads manipulating the bit pattern. > I can add a comment about it. Missed that - thanks. David ----- > Thanks, Robbin > >> >> Thanks, >> David >> ----- >> >> >>> So we are missing checking that bit completely in this transition code. >>> >>> Thanks, Robbin >>> >>> >>>> All you are doing is changing the timing of the race between the >>>> thread re-entering the VM/Java and the request for a suspend or >>>> safepoint. >>>> >>>> If there is a race between the sampler logic acting on the thread, >>>> and the thread acting on itself then that race has to be precluded >>>> somehow. >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>>> (which have the JFR native trans suspend check) >>>>> And it passes t1-5. >>>>> >>>>> Code: >>>>> http://cr.openjdk.java.net/~rehn/8218147/v3/webrev/ >>>>> Issue: >>>>> https://bugs.openjdk.java.net/browse/JDK-8218147 >>>>> >>>>> Thanks, Robbin >>>>> >>>>> On 4/5/19 5:43 PM, Robbin Ehn wrote: >>>>>> Hi Dean, >>>>>> >>>>>> Sorry, I missed this mail. >>>>>> Yes we can do that. >>>>>> Ignore my other mail, I'll update. >>>>>> >>>>>> Thanks, Robbin >>>>>> >>>>>> >>>>>> dean.long at oracle.com skrev: (5 april 2019 09:22:24 CEST) >>>>>>> On 4/4/19 5:16 PM, dean.long at oracle.com wrote: >>>>>>>> >>>>>>>>>> >>>>>>>>>> If it's already set, should we check that _last_Java_pc >>>>>>>>>> matches the >>>>>>> >>>>>>>>>> new value? >>>>>>>>> >>>>>>>>> We manually set the pc in several places, so if it's set, it's not >>>>>>>>> certain that >>>>>>>>> it should be the same as in last sp. >>>>>>>>> I can't distinguish between the cases. >>>>>>>>> >>>>>>>> >>>>>>>> If we get pc from sp[-1] then it should match, but you're right, we >>>>>>>> sometimes get pc from somewhere else. >>>>>>> >>>>>>> How about if we combine the !walkable check and the >>>>>>> capture_last_Java_pc() logic into a single method? >>>>>>> Then we can do something like: >>>>>>> >>>>>>> ???? if (!walkable()) { >>>>>>> ???????? address pc = (address)_last_Java_sp[-1]; >>>>>>> ???????? address a = Atomic::cmpxchg(pc, &_last_Java_pc, NULL); >>>>>>> ???????? assert(a == NULL || a == pc, "unexpected PC %p", a); >>>>>>> ???? } >>>>>>> >>>>>>> dl From robbin.ehn at oracle.com Tue Apr 16 08:59:51 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 16 Apr 2019 10:59:51 +0200 Subject: RFR(s): 8218147: make_walkable asserts on multiple calls In-Reply-To: <40b529d0-aeca-24b8-636a-518b7445b593@oracle.com> References: <48b0d1d0-3b12-e08d-f606-bf96a8445bf8@oracle.com> <587d1afe-9f86-f801-92de-31711b711034@oracle.com> <24A2B80F-D3D4-486A-9E29-8D48D49DF030@oracle.com> <3d0d841d-3bcf-ce3c-1d44-096c9aff2e72@oracle.com> <40b529d0-aeca-24b8-636a-518b7445b593@oracle.com> Message-ID: <6f53161b-479b-6dff-5ead-18aaa4170bf3@oracle.com> Hi David, >> And I'm not sure how to get that macro into that method in a nice way? > > Define nice ;-) > > return (_suspend_flags & (_external_suspend | _deopt_suspend JFR_ONLY(| > _trace_flag))) != 0; Sure! I'll re-test and sent out a v4, thanks! /Robbin > >>> >>> I was a little concerned that thread->set_trace_flag() may not ensure >>> visibility of the flag update, but then realized it should be covered by the >>> fence: >>> >>> ??static inline void transition_from_native(JavaThread *thread, >>> JavaThreadState to) { >>> ???? // Change to transition state and ensure it is seen by the VM thread. >>> ???? thread->set_thread_state_fence(_thread_in_native_trans); >>> >>> and the comment should say: >>> >>> // Change to transition state and ensure it is seen by other thread, >>> // and we will see any _suspend_flag changes below. >>> >>> However it also seems to me that in JfrThreadSampleClosure::do_sample_thread >>> we need a storeload() barrier after: >>> >>> ?? thread->set_trace_flag(); >>> >>> to ensure its not reordered with the reads of thread->thread_state() ? >> >> Setting/clearing suspend flags is always done with Atomic::cmpxchg, since >> there can be multiple threads manipulating the bit pattern. >> I can add a comment about it. > > Missed that - thanks. > > David > ----- > >> Thanks, Robbin >> >>> >>> Thanks, >>> David >>> ----- >>> >>> >>>> So we are missing checking that bit completely in this transition code. >>>> >>>> Thanks, Robbin >>>> >>>> >>>>> All you are doing is changing the timing of the race between the thread >>>>> re-entering the VM/Java and the request for a suspend or safepoint. >>>>> >>>>> If there is a race between the sampler logic acting on the thread, and the >>>>> thread acting on itself then that race has to be precluded somehow. >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>>> (which have the JFR native trans suspend check) >>>>>> And it passes t1-5. >>>>>> >>>>>> Code: >>>>>> http://cr.openjdk.java.net/~rehn/8218147/v3/webrev/ >>>>>> Issue: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8218147 >>>>>> >>>>>> Thanks, Robbin >>>>>> >>>>>> On 4/5/19 5:43 PM, Robbin Ehn wrote: >>>>>>> Hi Dean, >>>>>>> >>>>>>> Sorry, I missed this mail. >>>>>>> Yes we can do that. >>>>>>> Ignore my other mail, I'll update. >>>>>>> >>>>>>> Thanks, Robbin >>>>>>> >>>>>>> >>>>>>> dean.long at oracle.com skrev: (5 april 2019 09:22:24 CEST) >>>>>>>> On 4/4/19 5:16 PM, dean.long at oracle.com wrote: >>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> If it's already set, should we check that _last_Java_pc matches the >>>>>>>> >>>>>>>>>>> new value? >>>>>>>>>> >>>>>>>>>> We manually set the pc in several places, so if it's set, it's not >>>>>>>>>> certain that >>>>>>>>>> it should be the same as in last sp. >>>>>>>>>> I can't distinguish between the cases. >>>>>>>>>> >>>>>>>>> >>>>>>>>> If we get pc from sp[-1] then it should match, but you're right, we >>>>>>>>> sometimes get pc from somewhere else. >>>>>>>> >>>>>>>> How about if we combine the !walkable check and the >>>>>>>> capture_last_Java_pc() logic into a single method? >>>>>>>> Then we can do something like: >>>>>>>> >>>>>>>> ???? if (!walkable()) { >>>>>>>> ???????? address pc = (address)_last_Java_sp[-1]; >>>>>>>> ???????? address a = Atomic::cmpxchg(pc, &_last_Java_pc, NULL); >>>>>>>> ???????? assert(a == NULL || a == pc, "unexpected PC %p", a); >>>>>>>> ???? } >>>>>>>> >>>>>>>> dl From robbin.ehn at oracle.com Tue Apr 16 12:06:41 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 16 Apr 2019 14:06:41 +0200 Subject: RFR(s): 8218147: make_walkable asserts on multiple calls In-Reply-To: References: <48b0d1d0-3b12-e08d-f606-bf96a8445bf8@oracle.com> <587d1afe-9f86-f801-92de-31711b711034@oracle.com> <24A2B80F-D3D4-486A-9E29-8D48D49DF030@oracle.com> Message-ID: <53a81521-1630-802f-ce9e-b597b62f6ac5@oracle.com> Hi, here is v4. http://cr.openjdk.java.net/~rehn/8218147/v4/webrev/index.html Re-prod test and t1-t2. Thanks, Robbin On 4/15/19 10:58 AM, Robbin Ehn wrote: > Hi, please review. > > After reexamine this issue: > Threads in native must always have their stack walkable. > JFR sampler should never need to make a stack walkable (for native sample). > > I manage to locally reproduce reliable with changes to JFR sampler and having > hundreds of threads running similar code as the in the bug. > (Looping creating an array with negative size.) > > I found a place where we don't proper look at the suspend flags. > The java thread can thus escape native and make it's stack unwalkable and later > it tries to make it walkable at the same time as the JFR sampler. > > By removing some kind of fast check and instead always call the > check_safepoint_and_suspend_for_native_trans I can no longer reproduce. > (which have the JFR native trans suspend check) > And it passes t1-5. > > Code: > http://cr.openjdk.java.net/~rehn/8218147/v3/webrev/ > Issue: > https://bugs.openjdk.java.net/browse/JDK-8218147 > > Thanks, Robbin > > On 4/5/19 5:43 PM, Robbin Ehn wrote: >> Hi Dean, >> >> Sorry, I missed this mail. >> Yes we can do that. >> Ignore my other mail, I'll update. >> >> Thanks, Robbin >> >> >> dean.long at oracle.com skrev: (5 april 2019 09:22:24 CEST) >>> On 4/4/19 5:16 PM, dean.long at oracle.com wrote: >>>> >>>>>> >>>>>> If it's already set, should we check that _last_Java_pc matches the >>> >>>>>> new value? >>>>> >>>>> We manually set the pc in several places, so if it's set, it's not >>>>> certain that >>>>> it should be the same as in last sp. >>>>> I can't distinguish between the cases. >>>>> >>>> >>>> If we get pc from sp[-1] then it should match, but you're right, we >>>> sometimes get pc from somewhere else. >>> >>> How about if we combine the !walkable check and the >>> capture_last_Java_pc() logic into a single method? >>> Then we can do something like: >>> >>> ???? if (!walkable()) { >>> ???????? address pc = (address)_last_Java_sp[-1]; >>> ???????? address a = Atomic::cmpxchg(pc, &_last_Java_pc, NULL); >>> ???????? assert(a == NULL || a == pc, "unexpected PC %p", a); >>> ???? } >>> >>> dl From david.holmes at oracle.com Tue Apr 16 13:00:04 2019 From: david.holmes at oracle.com (David Holmes) Date: Tue, 16 Apr 2019 23:00:04 +1000 Subject: RFR(s): 8218147: make_walkable asserts on multiple calls In-Reply-To: <53a81521-1630-802f-ce9e-b597b62f6ac5@oracle.com> References: <48b0d1d0-3b12-e08d-f606-bf96a8445bf8@oracle.com> <587d1afe-9f86-f801-92de-31711b711034@oracle.com> <24A2B80F-D3D4-486A-9E29-8D48D49DF030@oracle.com> <53a81521-1630-802f-ce9e-b597b62f6ac5@oracle.com> Message-ID: Looks good to me. Thanks, David On 16/04/2019 10:06 pm, Robbin Ehn wrote: > Hi, here is v4. > > http://cr.openjdk.java.net/~rehn/8218147/v4/webrev/index.html > > Re-prod test and t1-t2. > > Thanks, Robbin > > On 4/15/19 10:58 AM, Robbin Ehn wrote: >> Hi, please review. >> >> After reexamine this issue: >> Threads in native must always have their stack walkable. >> JFR sampler should never need to make a stack walkable (for native >> sample). >> >> I manage to locally reproduce reliable with changes to JFR sampler and >> having >> hundreds of threads running similar code as the in the bug. >> (Looping creating an array with negative size.) >> >> I found a place where we don't proper look at the suspend flags. >> The java thread can thus escape native and make it's stack unwalkable >> and later >> it tries to make it walkable at the same time as the JFR sampler. >> >> By removing some kind of fast check and instead always call the >> check_safepoint_and_suspend_for_native_trans I can no longer reproduce. >> (which have the JFR native trans suspend check) >> And it passes t1-5. >> >> Code: >> http://cr.openjdk.java.net/~rehn/8218147/v3/webrev/ >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8218147 >> >> Thanks, Robbin >> >> On 4/5/19 5:43 PM, Robbin Ehn wrote: >>> Hi Dean, >>> >>> Sorry, I missed this mail. >>> Yes we can do that. >>> Ignore my other mail, I'll update. >>> >>> Thanks, Robbin >>> >>> >>> dean.long at oracle.com skrev: (5 april 2019 09:22:24 CEST) >>>> On 4/4/19 5:16 PM, dean.long at oracle.com wrote: >>>>> >>>>>>> >>>>>>> If it's already set, should we check that _last_Java_pc matches the >>>> >>>>>>> new value? >>>>>> >>>>>> We manually set the pc in several places, so if it's set, it's not >>>>>> certain that >>>>>> it should be the same as in last sp. >>>>>> I can't distinguish between the cases. >>>>>> >>>>> >>>>> If we get pc from sp[-1] then it should match, but you're right, we >>>>> sometimes get pc from somewhere else. >>>> >>>> How about if we combine the !walkable check and the >>>> capture_last_Java_pc() logic into a single method? >>>> Then we can do something like: >>>> >>>> ???? if (!walkable()) { >>>> ???????? address pc = (address)_last_Java_sp[-1]; >>>> ???????? address a = Atomic::cmpxchg(pc, &_last_Java_pc, NULL); >>>> ???????? assert(a == NULL || a == pc, "unexpected PC %p", a); >>>> ???? } >>>> >>>> dl From daniel.daugherty at oracle.com Tue Apr 16 13:22:08 2019 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 16 Apr 2019 09:22:08 -0400 Subject: RFR(s): 8218147: make_walkable asserts on multiple calls In-Reply-To: <53a81521-1630-802f-ce9e-b597b62f6ac5@oracle.com> References: <48b0d1d0-3b12-e08d-f606-bf96a8445bf8@oracle.com> <587d1afe-9f86-f801-92de-31711b711034@oracle.com> <24A2B80F-D3D4-486A-9E29-8D48D49DF030@oracle.com> <53a81521-1630-802f-ce9e-b597b62f6ac5@oracle.com> Message-ID: <2799bfbb-ae0d-12f9-35f7-a0392a6b941e@oracle.com> On 4/16/19 8:06 AM, Robbin Ehn wrote: > Hi, here is v4. > > http://cr.openjdk.java.net/~rehn/8218147/v4/webrev/index.html src/hotspot/share/jfr/periodic/sampling/jfrThreadSampler.cpp ??? L362: ? thread->set_trace_flag();? // Provides StoreLoad, needed to keep read of thread state not floating up. ??????? Typo: s/not floating/from floating/ src/hotspot/share/runtime/thread.hpp ??? No comments. Thumbs up! Dan > > Re-prod test and t1-t2. > > Thanks, Robbin > > On 4/15/19 10:58 AM, Robbin Ehn wrote: >> Hi, please review. >> >> After reexamine this issue: >> Threads in native must always have their stack walkable. >> JFR sampler should never need to make a stack walkable (for native >> sample). >> >> I manage to locally reproduce reliable with changes to JFR sampler >> and having >> hundreds of threads running similar code as the in the bug. >> (Looping creating an array with negative size.) >> >> I found a place where we don't proper look at the suspend flags. >> The java thread can thus escape native and make it's stack unwalkable >> and later >> it tries to make it walkable at the same time as the JFR sampler. >> >> By removing some kind of fast check and instead always call the >> check_safepoint_and_suspend_for_native_trans I can no longer reproduce. >> (which have the JFR native trans suspend check) >> And it passes t1-5. >> >> Code: >> http://cr.openjdk.java.net/~rehn/8218147/v3/webrev/ >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8218147 >> >> Thanks, Robbin >> >> On 4/5/19 5:43 PM, Robbin Ehn wrote: >>> Hi Dean, >>> >>> Sorry, I missed this mail. >>> Yes we can do that. >>> Ignore my other mail, I'll update. >>> >>> Thanks, Robbin >>> >>> >>> dean.long at oracle.com skrev: (5 april 2019 09:22:24 CEST) >>>> On 4/4/19 5:16 PM, dean.long at oracle.com wrote: >>>>> >>>>>>> >>>>>>> If it's already set, should we check that _last_Java_pc matches the >>>> >>>>>>> new value? >>>>>> >>>>>> We manually set the pc in several places, so if it's set, it's not >>>>>> certain that >>>>>> it should be the same as in last sp. >>>>>> I can't distinguish between the cases. >>>>>> >>>>> >>>>> If we get pc from sp[-1] then it should match, but you're right, we >>>>> sometimes get pc from somewhere else. >>>> >>>> How about if we combine the !walkable check and the >>>> capture_last_Java_pc() logic into a single method? >>>> Then we can do something like: >>>> >>>> ???? if (!walkable()) { >>>> ???????? address pc = (address)_last_Java_sp[-1]; >>>> ???????? address a = Atomic::cmpxchg(pc, &_last_Java_pc, NULL); >>>> ???????? assert(a == NULL || a == pc, "unexpected PC %p", a); >>>> ???? } >>>> >>>> dl From robbin.ehn at oracle.com Tue Apr 16 13:30:30 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 16 Apr 2019 15:30:30 +0200 Subject: RFR(s): 8218147: make_walkable asserts on multiple calls In-Reply-To: References: <48b0d1d0-3b12-e08d-f606-bf96a8445bf8@oracle.com> <587d1afe-9f86-f801-92de-31711b711034@oracle.com> <24A2B80F-D3D4-486A-9E29-8D48D49DF030@oracle.com> <53a81521-1630-802f-ce9e-b597b62f6ac5@oracle.com> Message-ID: <5a8c7b88-bda7-ad73-3c2a-ef390a814de4@oracle.com> Thanks David, Robbin On 4/16/19 3:00 PM, David Holmes wrote: > Looks good to me. > > Thanks, > David > > On 16/04/2019 10:06 pm, Robbin Ehn wrote: >> Hi, here is v4. >> >> http://cr.openjdk.java.net/~rehn/8218147/v4/webrev/index.html >> >> Re-prod test and t1-t2. >> >> Thanks, Robbin >> >> On 4/15/19 10:58 AM, Robbin Ehn wrote: >>> Hi, please review. >>> >>> After reexamine this issue: >>> Threads in native must always have their stack walkable. >>> JFR sampler should never need to make a stack walkable (for native sample). >>> >>> I manage to locally reproduce reliable with changes to JFR sampler and having >>> hundreds of threads running similar code as the in the bug. >>> (Looping creating an array with negative size.) >>> >>> I found a place where we don't proper look at the suspend flags. >>> The java thread can thus escape native and make it's stack unwalkable and later >>> it tries to make it walkable at the same time as the JFR sampler. >>> >>> By removing some kind of fast check and instead always call the >>> check_safepoint_and_suspend_for_native_trans I can no longer reproduce. >>> (which have the JFR native trans suspend check) >>> And it passes t1-5. >>> >>> Code: >>> http://cr.openjdk.java.net/~rehn/8218147/v3/webrev/ >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8218147 >>> >>> Thanks, Robbin >>> >>> On 4/5/19 5:43 PM, Robbin Ehn wrote: >>>> Hi Dean, >>>> >>>> Sorry, I missed this mail. >>>> Yes we can do that. >>>> Ignore my other mail, I'll update. >>>> >>>> Thanks, Robbin >>>> >>>> >>>> dean.long at oracle.com skrev: (5 april 2019 09:22:24 CEST) >>>>> On 4/4/19 5:16 PM, dean.long at oracle.com wrote: >>>>>> >>>>>>>> >>>>>>>> If it's already set, should we check that _last_Java_pc matches the >>>>> >>>>>>>> new value? >>>>>>> >>>>>>> We manually set the pc in several places, so if it's set, it's not >>>>>>> certain that >>>>>>> it should be the same as in last sp. >>>>>>> I can't distinguish between the cases. >>>>>>> >>>>>> >>>>>> If we get pc from sp[-1] then it should match, but you're right, we >>>>>> sometimes get pc from somewhere else. >>>>> >>>>> How about if we combine the !walkable check and the >>>>> capture_last_Java_pc() logic into a single method? >>>>> Then we can do something like: >>>>> >>>>> ???? if (!walkable()) { >>>>> ???????? address pc = (address)_last_Java_sp[-1]; >>>>> ???????? address a = Atomic::cmpxchg(pc, &_last_Java_pc, NULL); >>>>> ???????? assert(a == NULL || a == pc, "unexpected PC %p", a); >>>>> ???? } >>>>> >>>>> dl From robbin.ehn at oracle.com Tue Apr 16 13:31:23 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 16 Apr 2019 15:31:23 +0200 Subject: RFR(s): 8218147: make_walkable asserts on multiple calls In-Reply-To: <2799bfbb-ae0d-12f9-35f7-a0392a6b941e@oracle.com> References: <48b0d1d0-3b12-e08d-f606-bf96a8445bf8@oracle.com> <587d1afe-9f86-f801-92de-31711b711034@oracle.com> <24A2B80F-D3D4-486A-9E29-8D48D49DF030@oracle.com> <53a81521-1630-802f-ce9e-b597b62f6ac5@oracle.com> <2799bfbb-ae0d-12f9-35f7-a0392a6b941e@oracle.com> Message-ID: Hi Dan, On 4/16/19 3:22 PM, Daniel D. Daugherty wrote: > On 4/16/19 8:06 AM, Robbin Ehn wrote: >> Hi, here is v4. >> >> http://cr.openjdk.java.net/~rehn/8218147/v4/webrev/index.html > > src/hotspot/share/jfr/periodic/sampling/jfrThreadSampler.cpp > ??? L362: ? thread->set_trace_flag();? // Provides StoreLoad, needed to keep > read of thread state not floating up. > ??????? Typo: s/not floating/from floating/ Fixed! > > src/hotspot/share/runtime/thread.hpp > ??? No comments. > > Thumbs up! > Thanks Dan! /Robbin > Dan > > >> >> Re-prod test and t1-t2. >> >> Thanks, Robbin >> >> On 4/15/19 10:58 AM, Robbin Ehn wrote: >>> Hi, please review. >>> >>> After reexamine this issue: >>> Threads in native must always have their stack walkable. >>> JFR sampler should never need to make a stack walkable (for native sample). >>> >>> I manage to locally reproduce reliable with changes to JFR sampler and having >>> hundreds of threads running similar code as the in the bug. >>> (Looping creating an array with negative size.) >>> >>> I found a place where we don't proper look at the suspend flags. >>> The java thread can thus escape native and make it's stack unwalkable and later >>> it tries to make it walkable at the same time as the JFR sampler. >>> >>> By removing some kind of fast check and instead always call the >>> check_safepoint_and_suspend_for_native_trans I can no longer reproduce. >>> (which have the JFR native trans suspend check) >>> And it passes t1-5. >>> >>> Code: >>> http://cr.openjdk.java.net/~rehn/8218147/v3/webrev/ >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8218147 >>> >>> Thanks, Robbin >>> >>> On 4/5/19 5:43 PM, Robbin Ehn wrote: >>>> Hi Dean, >>>> >>>> Sorry, I missed this mail. >>>> Yes we can do that. >>>> Ignore my other mail, I'll update. >>>> >>>> Thanks, Robbin >>>> >>>> >>>> dean.long at oracle.com skrev: (5 april 2019 09:22:24 CEST) >>>>> On 4/4/19 5:16 PM, dean.long at oracle.com wrote: >>>>>> >>>>>>>> >>>>>>>> If it's already set, should we check that _last_Java_pc matches the >>>>> >>>>>>>> new value? >>>>>>> >>>>>>> We manually set the pc in several places, so if it's set, it's not >>>>>>> certain that >>>>>>> it should be the same as in last sp. >>>>>>> I can't distinguish between the cases. >>>>>>> >>>>>> >>>>>> If we get pc from sp[-1] then it should match, but you're right, we >>>>>> sometimes get pc from somewhere else. >>>>> >>>>> How about if we combine the !walkable check and the >>>>> capture_last_Java_pc() logic into a single method? >>>>> Then we can do something like: >>>>> >>>>> ???? if (!walkable()) { >>>>> ???????? address pc = (address)_last_Java_sp[-1]; >>>>> ???????? address a = Atomic::cmpxchg(pc, &_last_Java_pc, NULL); >>>>> ???????? assert(a == NULL || a == pc, "unexpected PC %p", a); >>>>> ???? } >>>>> >>>>> dl > From kustos at gmx.net Wed Apr 17 06:12:44 2019 From: kustos at gmx.net (Philippe Marschall) Date: Wed, 17 Apr 2019 08:12:44 +0200 Subject: Supported data types for events In-Reply-To: References: <700a9182-62f2-2555-0b6a-12c1e0a84596@gmx.net> Message-ID: <0d4a7fea-3d53-bfba-3fd4-76713d97923a@gmx.net> On 15.04.19 11:19, Erik Gahlin wrote: > Hi Philippe, > > The supported data types in Java are primitives, java.lang.Class, java.lang.Thread and java.lang.String. You can see the list in the toType method here > > http://hg.openjdk.java.net/jdk/jdk/file/tip/src/jdk.jfr/share/classes/jdk/jfr/internal/ASMToolkit.java > > You can't use JSR-310 types, but it is possible to store durations and instants using the annotations @Timespan and @Timestamp on numeric fields. > In native, arrays are supported and it is possible to define you own composite data types, for example VirtualSpace You can see the list here: > http://hg.openjdk.java.net/jdk/jdk/file/tip/src/hotspot/share/jfr/metadata/metadata.xml > > The file format only support ?value types?, so it is not possible to store an array with different data types per element. Awesome thank you. Cheers Philippe From erik.gahlin at oracle.com Wed Apr 17 20:11:05 2019 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Wed, 17 Apr 2019 22:11:05 +0200 Subject: PING: RFR 8218631: jfr tools is effective only the last filter In-Reply-To: <152D2629-65F0-4A42-B2B3-25F3EB3321B3@oracle.com> References: <2e94f465-01f4-f40b-4726-d28d5041faa6@oracle.com> <152D2629-65F0-4A42-B2B3-25F3EB3321B3@oracle.com> Message-ID: <5CB78859.1010203@oracle.com> Hi Chihiro, Another issue surfaced that would have been fixed by your patch. See "8221569: JFR tool produces incorrect output when both --category and --event are specified" https://bugs.openjdk.java.net/browse/JDK-8221569 I modified your patch and added checks to prevent --categories and --events to be specified multiple times. Webrev: http://cr.openjdk.java.net/~egahlin/8218631/ If you like I can set you as contributor and sponsor it for you? (and close 8221569 as a duplicate) (An argument can be made that a user should be able to specify --events multiple times and get the union, but that is not according to spec. which is [--events ] without repeat. Having multiple ways to achieve the same goal may confuse users, i.e. did the event not get printed because of multiple --events parameters, or because it was not in the file?) Thanks Erik > Hi Chihiro, > > If you want to use multiple filters, you can specify them like this: > > ?events filter1,filter2,filter3 > ?categories filter1,filter2,filter2 > > We could add an error message, if a user specifies the ?events flag more than once, and describe how multiple filters can be specified, but I don?t think there should be multiple ways to specify filters. > > Thanks > Erik > > >> On 21 Feb 2019, at 08:38, Chihiro Ito wrote: >> >> Hi All, >> >> Could you review this change, please? >> >> webrev: >> http://cr.openjdk.java.net/~cito/JDK-8218631/webrev.00/ >> >> JBS: >> https://bugs.openjdk.java.net/browse/JDK-8218631 >> >> This change passed all tests in test-tier1. >> >> Regards, >> Chihiro >> >> >> On 2019/02/08 2:46, Chihiro Ito wrote: >>> Hi, >>> >>> Could you review this change, please? >>> >>> I have referred to the setting of Xlog, and I changed this should enable every filter as follows. >>> >>> jfr print --events jdk.JVMInformation --events jdk.OSInformation profiling-recording4399235951575204469.jfr >>> jdk.JVMInformation { >>> startTime = 17:40:16.536 >>> jvmName = "OpenJDK 64-Bit Server VM" >>> jvmVersion = "OpenJDK 64-Bit Server VM (fastdebug 13-internal+0-adhoc.chito.jdk) for linux-amd64 JRE (13-internal+0-adhoc.chito.jdk), built on Feb 5 2019 23:57:41 by "chito" with gcc 4.8.5 20150623 (Red Hat 4.8.5-28.0.1)" >>> jvmArguments = "-Dtest.class.path.prefix=/src/jdk/build/linux-x86_64-server-fastdebug/test-support/jtreg_test_jdk_jdk_jfr_tool_TestPrintDefault_java/classes/jdk/jfr/tool/TestPrintDefault.d:/src/jdk/test/jdk/jdk/jfr/tool:/src/jdk/build/linux-x86_64-server-fastdebug/test-support/jtreg_test_jdk_jdk_jfr_tool_TestPrintDefault_java/classes/test/lib:/src/jdk/build/linux-x86_64-server-fastdebug/test-support/jtreg_test_jdk_jdk_jfr_tool_TestPrintDefault_java/classes/test/jdk -Dtest.src=/src/jdk/test/jdk/jdk/jfr/tool -Dtest.src.path=/src/jdk/test/jdk/jdk/jfr/tool:/src/jdk/test/lib:/src/jdk/test/jdk -Dtest.classes=/src/jdk/build/linux-x86_64-server-fastdebug/test-support/jtreg_test_jdk_jdk_jfr_tool_TestPrintDefault_java/classes/jdk/jfr/tool/TestPrintDefault.d -Dtest.class.path=/src/jdk/build/linux-x86_64-server-fastdebug/test-support/jtreg_test_jdk_jdk_jfr_tool_TestPrintDefault_java/classes/jdk/jfr/tool/TestPrintDefault.d:/src/jdk/build/linux-x86_64-server-fastdebug/test-support/jtreg_test_jdk_jdk_jfr_tool_TestPrintDefault_java/classes/test/lib:/src/jdk/build/linux-x86_64-server-fastdebug/test-support/jtreg_test_jdk_jdk_jfr_tool_TestPrintDefault_java/classes/test/jdk -Dtest.vm.opts=-Xmx512m -XX:MaxRAMPercentage=25 -ea -esa -Dtest.tool.vm.opts=-J-Xmx512m -J-XX:MaxRAMPercentage=25 -J-ea -J-esa -Dtest.compiler.opts= -Dtest.java.opts= -Dtest.jdk=/src/jdk/build/linux-x86_64-server-fastdebug/images/jdk -Dcompile.jdk=/src/jdk/build/linux-x86_64-server-fastdebug/images/jdk -Dtest.timeout.factor=4.0 -Dtest.modules=java.scripting jdk.jfr -Dtest.nativepath=/src/jdk/build/linux-x86_64-server-fastdebug/images/test/jdk/jtreg/native --add-modules=java.scripting,jdk.jfr -Xmx512m -XX:MaxRAMPercentage=25 -ea -esa -Djava.library.path=/src/jdk/build/linux-x86_64-server-fastdebug/images/test/jdk/jtreg/native" >>> jvmFlags = N/A >>> javaArguments = "com.sun.javatest.regtest.agent.MainWrapper /src/jdk/build/linux-x86_64-server-fastdebug/test-support/jtreg_test_jdk_jdk_jfr_tool_TestPrintDefault_java/jdk/jfr/tool/TestPrintDefault.d/main.0.jta" >>> jvmStartTime = 17:39:51.394 >>> pid = 23337 >>> } >>> >>> jdk.OSInformation { >>> startTime = 17:40:16.542 >>> osVersion = "Oracle Linux Server release 7.5 >>> uname:Linux localhost.localdomain 4.1.12-112.16.4.el7uek.x86_64 #2 SMP Mon Mar 12 23:57:12 PDT 2018 x86_64 >>> libc:glibc 2.17 NPTL 2.17 >>> " >>> } >>> >>> webrev: >>> http://cr.openjdk.java.net/~cito/JDK-8218631/webrev.00/ >>> >>> JBS: >>> https://bugs.openjdk.java.net/browse/JDK-8218631 >>> >>> This change passed all tests in test-tier1. >>> >>> Regards, >>> Chihiro >>> From yasuenag at gmail.com Mon Apr 22 10:23:22 2019 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Mon, 22 Apr 2019 19:23:22 +0900 Subject: PING: RFR: 8219566: JFR did not collect call stacks when MaxJavaStackTraceDepth is set to zero In-Reply-To: References: <6D0A14CC-3EBA-4A01-AFFC-F2987F3C4DFD@oracle.com> <59ed9471-45a5-5707-8356-367ae35a7829@gmail.com> <19551e1b-b9e8-62e1-6fab-7dd6e100f50a@gmail.com> <5C828C38.1090506@oracle.com> <47d27b56-0933-c173-55c1-7e225218ead8@gmail.com> <0b31693a-a8d4-1fdf-c830-8752f3ec379e@gmail.com> <5C95369B.8030404@oracle.com> <7c203381-0091-f9a9-6e76-1b09d0e92c88@gmail.com> <2ca33c8e-6565-fad3-294c-ae1804271f54@gmail.com> <5CB0D24B.305@oracle.com> <537B942B-1E3B-42B6-A121-05B5D7CC9CD9@oracle.com> Message-ID: <07eaa3dc-fa15-d38f-2b5e-dbebc51a2a7a@gmail.com> PING: Could you review it? JBS: https://bugs.openjdk.java.net/browse/JDK-8219566 webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.03/ I need one more reviewer to push. Yasumasa On 2019/04/15 8:14, Yasumasa Suenaga wrote: > Thanks Erik! > > I'm waiting for another reviewer for webrev.03 . > > > Yasumasa > > > 2019?4?15?(?) 7:43 Erik Gahlin >: > > Looks good (webrev.03) > > According to the option spec. MaxJavaStackTraceDepth controls "the > maximum number of lines in the stack trace for Java exceptions?. > This is not an exception stack trace, so I think it would be more > appropriate to use MAX_STACK_DEPTH. > > Thanks > Erik > > > > On 14 Apr 2019, at 15:05, Yasumasa Suenaga > wrote: > > > > Hi Erik, > > > > How about this change? > > http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.03/ > > > > IMHO I prefer webrev.00 if we need to think stackdepth is set to > lower value. > > Fix in webrev.00 is introduced in JDK-7179701. > > > > As you said before, it does not make the code easier to read. > > > > So if webrev.03 is appropriate, I will push webrev.03. > > > > > > Thanks, > > > > Yasumasa > > > > > > On 2019/04/13 3:00, Erik Gahlin wrote: > >> Hi, > >> Could we use the max stack depth used by JFR instead? > >> It's not available as constant in a header file, but maybe you > could move: > >> static const u4 STACK_DEPTH_DEFAULT = 64; > >> static const u4 MIN_STACK_DEPTH = 1; > >> static const u4 MAX_STACK_DEPTH = 2048; > >> from jfrOptionSet.cpp to jfrTypes.hpp where we have other > constants. The benefit of using MAX_STACK_DEPTH is that it will work > in scenarios where a user has set the number of frames to a low > number, i.e -XX:FlightRecorderOptions=5, and we hit a stack with > lots of frames that we can't record, i.e native frames. > >> Thanks > >> Erik > >>> Hi, > >>> > >>> Do you have any concern(s) about this change? > >>> > >>>>>>? ? webrev: > http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.02/ > >>> > >>> I think thread stack should be collected by JFR even if > MaxJavaStackTraceDepth is set to zero. > >>> > >>> > >>> Thanks, > >>> > >>> Yasumasa > >>> > >>> > >>> On 2019/03/23 21:03, Yasumasa Suenaga wrote: > >>>> Hi Erik, > >>>> > >>>> MaxJavaStackTraceDepth do not only affect finding top stack > but also stack walking. > >>>> > >>>> JfrGetCallTrace::find_top_frame() uses MaxJavaStackTraceDepth > to find top frame as you say. > >>>> OTOH vframeStreamSamples::samples_next() uses it to find next > valid frame. > >>>> > >>>> > >>>> Anyway thread stack should be collected even if > MaxJavaStackTraceDepth is set to zero. > >>>> > >>>> > >>>> Thanks, > >>>> > >>>> Yasumasa > >>>> > >>>> > >>>> On 2019/03/23 4:25, Erik Gahlin wrote: > >>>>> To me, it seems that the purpose of the method is to find the > first > >>>>> valid frame on the stack (the top frame). This is orthogonal > to the > >>>>> number of frames that should be recorded by JFR. > >>>>> > >>>>> Erik > >>>>> > >>>>>> PING: Could you review it? > >>>>>> > >>>>>>? ? JBS: https://bugs.openjdk.java.net/browse/JDK-8219566 > >>>>>>? ? webrev: > http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.02/ > >>>>>> > >>>>>> This change has passed :jdk_jfr tests and submit repo. > >>>>>> > >>>>>> > >>>>>> Thanks, > >>>>>> > >>>>>> Yasumasa > >>>>>> > >>>>>> > >>>>>> On 2019/03/11 14:49, Yasumasa Suenaga wrote: > >>>>>>> 2019?3?9?(?) 20:47 Yasumasa Suenaga >: > >>>>>>>> > >>>>>>>> Hi Erik, > >>>>>>>> > >>>>>>>> Reading your email, I think we should use > JfrOptionSet::stackdepth() > >>>>>>>> rather than MaxJavaStackTraceDepth. > >>>>>>>> I created new webrev, and it passed jdk_jfr jtreg tests on > my Linux > >>>>>>>> x64 box. > >>>>>>>> > >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.01/ > >>>>>>>> > >>>>>>>> However it has not yet passed tests on submit repo > >>>>>>>> (mach5-one-ysuenaga-JDK-8219566-20190309-0909-1079793). > >>>>>>>> It seems to be failed at build phase which includes Linux. > >>>>>>>> (I can build with this change on Fedora 29 x64). > >>>>>>>> > >>>>>>>> Could you help if this webrev is ok? > >>>>>>> > >>>>>>> I found 1 warning, and I fixed it. > >>>>>>> This webrev works fine with :jdk_jfr jtreg tests on my > Linux box, and > >>>>>>> submit repo test > >>>>>>> (mach5-one-ysuenaga-JDK-8219566-20190311-0150-1121570). > >>>>>>> Could you review it? > >>>>>>> > >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.02/ > >>>>>>> > >>>>>>> > >>>>>>> Thanks, > >>>>>>> > >>>>>>> Yasumasa > >>>>>>> > >>>>>>> > >>>>>>>> Thanks, > >>>>>>>> > >>>>>>>> Yasumasa > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> On 2019/03/09 0:37, Erik Gahlin wrote: > >>>>>>>>> JFR imposes a limit on how many frames that can be > recorded, set to > >>>>>>>>> 64 by default but configurable up to 2048 using > >>>>>>>>> -XX:FlightRecorderOptions, so setting the value to 0 will > still not > >>>>>>>>> mean all frames. > >>>>>>>>> > >>>>>>>>> That said, if you still want to fix it. It may make sense > to set > >>>>>>>>> loop_max to INT_MAX, if MaxJavaStackTraceDepth is zero, i.e > >>>>>>>>> > >>>>>>>>> int loop_max = MaxJavaStackTraceDepth == 0 ? INT_MAX : > >>>>>>>>> MaxJavaStackTraceDepth * 2; > >>>>>>>>> > >>>>>>>>> and avoid evaluating it for every frame. Even if the > compiler is > >>>>>>>>> able to hoist the comparison, it makes the code easier to > read. > >>>>>>>>> > >>>>>>>>> Thanks > >>>>>>>>> Erik > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> PING: Could you review it? > >>>>>>>>>> > >>>>>>>>>>>>>? ? ?JBS: https://bugs.openjdk.java.net/browse/JDK-8219566 > >>>>>>>>>>>>>? ? ?webrev: > >>>>>>>>>>>>> > http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.00/ > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> If you have any questions, please tell me. > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> Thanks, > >>>>>>>>>> > >>>>>>>>>> Yasumasa > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> On 2019/02/27 22:46, Yasumasa Suenaga wrote: > >>>>>>>>>>> Hi Erik, > >>>>>>>>>>> > >>>>>>>>>>> On 2019/02/27 22:30, Erik Gahlin wrote: > >>>>>>>>>>>> Hi Yasumasa, > >>>>>>>>>>>> > >>>>>>>>>>>> Could you explain a little about the bug? > >>>>>>>>>>>> > >>>>>>>>>>>> Why should stack traces be collected if > MaxJavaStackTraceDepth > >>>>>>>>>>>> is set to 0. > >>>>>>>>>>> > >>>>>>>>>>> "0" in MaxJavaStackTraceDepth means all exception stack > traces > >>>>>>>>>>> will be shown. > >>>>>>>>>>> > >>>>>>>>>>> > http://hg.openjdk.java.net/jdk/jdk/file/72ce7dd54939/src/hotspot/share/runtime/globals.hpp#l1553 > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> JDK-7179701 reports similar issue. > >>>>>>>>>>> I think we can take similar approach for JDK-8219566. > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> Thanks, > >>>>>>>>>>> > >>>>>>>>>>> Yasumasa > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> Thanks > >>>>>>>>>>>> Erik > >>>>>>>>>>>> > >>>>>>>>>>>>> On 22 Feb 2019, at 07:48, Yasumasa Suenaga at gmail.com > > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>> Hi all, > >>>>>>>>>>>>> > >>>>>>>>>>>>> Please review this change: > >>>>>>>>>>>>> > >>>>>>>>>>>>>? ? ?JBS: https://bugs.openjdk.java.net/browse/JDK-8219566 > >>>>>>>>>>>>>? ? ?webrev: > >>>>>>>>>>>>> > http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.00/ > >>>>>>>>>>>>> > >>>>>>>>>>>>> JFR did not collect call stacks when > MaxJavaStackTraceDepth is > >>>>>>>>>>>>> set to zero. > >>>>>>>>>>>>> This bug is similar to JDK-7179701. > >>>>>>>>>>>>> > >>>>>>>>>>>>> This change passed tests on submit repo and :jdk_jfr > jtreg tests. > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>> > >>>>>>>>>>>>> Yasumasa > >>>>>>>>>>>> > >>>>>>>>> > >>>>> > From erik.gahlin at oracle.com Mon Apr 22 21:48:38 2019 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Mon, 22 Apr 2019 23:48:38 +0200 Subject: PING: RFR: 8220657: JFR.dump does not work when filename is set In-Reply-To: <2dc4e611-bd18-a187-f529-d243af51cf10@gmail.com> References: <5C90E8C9.3010008@oracle.com> <14c04f3a-1fc0-af3b-714f-e35fa18d6a1b@gmail.com> <5C9103DE.4040304@oracle.com> <725f2ab6-71ee-4149-9d1d-ca6c19217408@gmail.com> <2dc4e611-bd18-a187-f529-d243af51cf10@gmail.com> Message-ID: <5CBE36B6.6070603@oracle.com> I have looked further at the issue and while the message to the user was incorrect previously ( "Dump failed. No data found in the specified interval") it did result in a file with the specified filename, i.e "test.jfr" When "clone.setDestination(this.destination);" is removed, the filename passed on command line is ignored, but it does result in a message that is consistent with what is happening (a file with an autogenerated name is created). It does however change the behavior of 'JFR.dump name=1'. Logic needs to be added to DCmdDump so it uses an existing filename if it is set, but changes it if a user is providing an override. When comes to testing. I think it would be better to write a separate test that checks that things are dumped correctly when-XX:StartFlightRecording:filename= is used. Thanks Erik > PING: Do you have any opinion(s) about this fix? > > > Yasumasa > > > On 2019/04/01 21:46, Yasumasa Suenaga wrote: >> Hi, >> >> Do you have concern(s) about this change? >> >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.01/ >> >> As in the email below, I think it does not affect scenarios except >> snapshot cloning. >> Please tell me if you have any opinions about this. >> >> >> Thanks, >> >> Yasumasa >> >> >> On 2019/03/20 10:37, Yasumasa Suenaga wrote: >>> Hi Erik, >>> >>> I think this change does not affect other scenarios. >>> >>> :jdk_jfr jtreg tests and submit repo have passed with this change. >>> newSnapshotClone() is called by PlatformRecording::dump and >>> DCmdDump::newSnapshot. >>> Both methods are related to user-requested dump. >>> >>> What scenarios do you think? I want to check them. >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> 2019?3?19?(?) 23:59 Erik Gahlin : >>>> >>>> Yes, but then you would lose the original destination in other >>>> scenarios >>>> >>>> Maybe I can look at this next week. >>>> >>>> Erik >>>> >>>>> Hi Erik, >>>>> >>>>> Thank you for your reply. >>>>> >>>>> How about this changeset? >>>>> >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.01/ >>>>> >>>>> I think PlatformRecording#newSnapshotClone() should not dump. >>>>> So I removed `setDestination()` from this. >>>>> >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2019/03/19 22:04, Erik Gahlin wrote: >>>>>> This looks a bit hackish. >>>>>> >>>>>> The real issue seems to be that the method >>>>>> PlatformRecording#newSnapshotClone incorrectly sets the >>>>>> destination in >>>>>> the clone, which then triggers a dump prematurely (when stop is >>>>>> called). >>>>>> >>>>>> Purpose of that method is just to take a snapshot of existing >>>>>> chunks in >>>>>> the disk repository (or from memory). and then let >>>>>> dumpStopped(path) do >>>>>> the actual dump (copy out the data from the disk repository). >>>>>> >>>>>> To make it work, some other changes would be needed as well, but >>>>>> I don't >>>>>> have time to investigate. >>>>>> >>>>>> Erik >>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> Please review this change. >>>>>>> >>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8220657 >>>>>>> webrev: >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.00/ >>>>>>> >>>>>>> The user might not get flight record via JFR.dump if JFR is started >>>>>>> with filename option on target process. >>>>>>> >>>>>>> Please see JBS if you want to know how to reproduce. >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>> From markus.gronlund at oracle.com Tue Apr 23 12:37:38 2019 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Tue, 23 Apr 2019 05:37:38 -0700 (PDT) Subject: PING: RFR: 8219566: JFR did not collect call stacks when MaxJavaStackTraceDepth is set to zero In-Reply-To: <07eaa3dc-fa15-d38f-2b5e-dbebc51a2a7a@gmail.com> References: <6D0A14CC-3EBA-4A01-AFFC-F2987F3C4DFD@oracle.com> <59ed9471-45a5-5707-8356-367ae35a7829@gmail.com> <19551e1b-b9e8-62e1-6fab-7dd6e100f50a@gmail.com> <5C828C38.1090506@oracle.com> <47d27b56-0933-c173-55c1-7e225218ead8@gmail.com> <0b31693a-a8d4-1fdf-c830-8752f3ec379e@gmail.com> <5C95369B.8030404@oracle.com> <7c203381-0091-f9a9-6e76-1b09d0e92c88@gmail.com> <2ca33c8e-6565-fad3-294c-ae1804271f54@gmail.com> <5CB0D24B.305@oracle.com> <537B942B-1E3B-42B6-A121-05B5D7CC9CD9@oracle.com> <07eaa3dc-fa15-d38f-2b5e-dbebc51a2a7a@gmail.com> Message-ID: <915d2ca3-7923-4591-9b44-299e93817d7a@default> src/hotspot/share/jfr/periodic/sampling/jfrCallTrace.cpp Need #include "jfr/utilities/jfrTypes.hpp" src/hotspot/share/jfr/recorder/service/jfrOptionSet.cpp Need #include "jfr/utilities/jfrTypes.hpp" src/hotspot/share/jfr/recorder/stacktrace/jfrStackTraceRepository.cpp loop_count and loop_max might need to have type u4 instead of int to avoid unsigned / signed type warning / errors src/hotspot/share/jfr/utilities/jfrTypes.hpp No comment. Thanks Markus -----Original Message----- From: Yasumasa Suenaga Sent: den 22 april 2019 12:23 To: hotspot-jfr-dev at openjdk.java.net Subject: PING: RFR: 8219566: JFR did not collect call stacks when MaxJavaStackTraceDepth is set to zero PING: Could you review it? JBS: https://bugs.openjdk.java.net/browse/JDK-8219566 webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.03/ I need one more reviewer to push. Yasumasa On 2019/04/15 8:14, Yasumasa Suenaga wrote: > Thanks Erik! > > I'm waiting for another reviewer for webrev.03 . > > > Yasumasa > > > 2019?4?15?(?) 7:43 Erik Gahlin >: > > Looks good (webrev.03) > > According to the option spec. MaxJavaStackTraceDepth controls "the > maximum number of lines in the stack trace for Java exceptions?. > This is not an exception stack trace, so I think it would be more > appropriate to use MAX_STACK_DEPTH. > > Thanks > Erik > > > > On 14 Apr 2019, at 15:05, Yasumasa Suenaga > wrote: > > > > Hi Erik, > > > > How about this change? > > http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.03/ > > > > IMHO I prefer webrev.00 if we need to think stackdepth is set to > lower value. > > Fix in webrev.00 is introduced in JDK-7179701. > > > > As you said before, it does not make the code easier to read. > > > > So if webrev.03 is appropriate, I will push webrev.03. > > > > > > Thanks, > > > > Yasumasa > > > > > > On 2019/04/13 3:00, Erik Gahlin wrote: > >> Hi, > >> Could we use the max stack depth used by JFR instead? > >> It's not available as constant in a header file, but maybe you > could move: > >> static const u4 STACK_DEPTH_DEFAULT = 64; > >> static const u4 MIN_STACK_DEPTH = 1; > >> static const u4 MAX_STACK_DEPTH = 2048; > >> from jfrOptionSet.cpp to jfrTypes.hpp where we have other > constants. The benefit of using MAX_STACK_DEPTH is that it will work > in scenarios where a user has set the number of frames to a low > number, i.e -XX:FlightRecorderOptions=5, and we hit a stack with > lots of frames that we can't record, i.e native frames. > >> Thanks > >> Erik > >>> Hi, > >>> > >>> Do you have any concern(s) about this change? > >>> > >>>>>>? ? webrev: > http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.02/ > >>> > >>> I think thread stack should be collected by JFR even if > MaxJavaStackTraceDepth is set to zero. > >>> > >>> > >>> Thanks, > >>> > >>> Yasumasa > >>> > >>> > >>> On 2019/03/23 21:03, Yasumasa Suenaga wrote: > >>>> Hi Erik, > >>>> > >>>> MaxJavaStackTraceDepth do not only affect finding top stack > but also stack walking. > >>>> > >>>> JfrGetCallTrace::find_top_frame() uses MaxJavaStackTraceDepth > to find top frame as you say. > >>>> OTOH vframeStreamSamples::samples_next() uses it to find next > valid frame. > >>>> > >>>> > >>>> Anyway thread stack should be collected even if > MaxJavaStackTraceDepth is set to zero. > >>>> > >>>> > >>>> Thanks, > >>>> > >>>> Yasumasa > >>>> > >>>> > >>>> On 2019/03/23 4:25, Erik Gahlin wrote: > >>>>> To me, it seems that the purpose of the method is to find the > first > >>>>> valid frame on the stack (the top frame). This is orthogonal > to the > >>>>> number of frames that should be recorded by JFR. > >>>>> > >>>>> Erik > >>>>> > >>>>>> PING: Could you review it? > >>>>>> > >>>>>>? ? JBS: https://bugs.openjdk.java.net/browse/JDK-8219566 > >>>>>>? ? webrev: > http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.02/ > >>>>>> > >>>>>> This change has passed :jdk_jfr tests and submit repo. > >>>>>> > >>>>>> > >>>>>> Thanks, > >>>>>> > >>>>>> Yasumasa > >>>>>> > >>>>>> > >>>>>> On 2019/03/11 14:49, Yasumasa Suenaga wrote: > >>>>>>> 2019?3?9?(?) 20:47 Yasumasa Suenaga >: > >>>>>>>> > >>>>>>>> Hi Erik, > >>>>>>>> > >>>>>>>> Reading your email, I think we should use > JfrOptionSet::stackdepth() > >>>>>>>> rather than MaxJavaStackTraceDepth. > >>>>>>>> I created new webrev, and it passed jdk_jfr jtreg tests on > my Linux > >>>>>>>> x64 box. > >>>>>>>> > >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.01/ > >>>>>>>> > >>>>>>>> However it has not yet passed tests on submit repo > >>>>>>>> (mach5-one-ysuenaga-JDK-8219566-20190309-0909-1079793). > >>>>>>>> It seems to be failed at build phase which includes Linux. > >>>>>>>> (I can build with this change on Fedora 29 x64). > >>>>>>>> > >>>>>>>> Could you help if this webrev is ok? > >>>>>>> > >>>>>>> I found 1 warning, and I fixed it. > >>>>>>> This webrev works fine with :jdk_jfr jtreg tests on my > Linux box, and > >>>>>>> submit repo test > >>>>>>> (mach5-one-ysuenaga-JDK-8219566-20190311-0150-1121570). > >>>>>>> Could you review it? > >>>>>>> > >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.02/ > >>>>>>> > >>>>>>> > >>>>>>> Thanks, > >>>>>>> > >>>>>>> Yasumasa > >>>>>>> > >>>>>>> > >>>>>>>> Thanks, > >>>>>>>> > >>>>>>>> Yasumasa > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> On 2019/03/09 0:37, Erik Gahlin wrote: > >>>>>>>>> JFR imposes a limit on how many frames that can be > recorded, set to > >>>>>>>>> 64 by default but configurable up to 2048 using > >>>>>>>>> -XX:FlightRecorderOptions, so setting the value to 0 will > still not > >>>>>>>>> mean all frames. > >>>>>>>>> > >>>>>>>>> That said, if you still want to fix it. It may make sense > to set > >>>>>>>>> loop_max to INT_MAX, if MaxJavaStackTraceDepth is zero, i.e > >>>>>>>>> > >>>>>>>>> int loop_max = MaxJavaStackTraceDepth == 0 ? INT_MAX : > >>>>>>>>> MaxJavaStackTraceDepth * 2; > >>>>>>>>> > >>>>>>>>> and avoid evaluating it for every frame. Even if the > compiler is > >>>>>>>>> able to hoist the comparison, it makes the code easier to > read. > >>>>>>>>> > >>>>>>>>> Thanks > >>>>>>>>> Erik > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> PING: Could you review it? > >>>>>>>>>> > >>>>>>>>>>>>>? ? ?JBS: https://bugs.openjdk.java.net/browse/JDK-8219566 > >>>>>>>>>>>>>? ? ?webrev: > >>>>>>>>>>>>> > http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.00/ > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> If you have any questions, please tell me. > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> Thanks, > >>>>>>>>>> > >>>>>>>>>> Yasumasa > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> On 2019/02/27 22:46, Yasumasa Suenaga wrote: > >>>>>>>>>>> Hi Erik, > >>>>>>>>>>> > >>>>>>>>>>> On 2019/02/27 22:30, Erik Gahlin wrote: > >>>>>>>>>>>> Hi Yasumasa, > >>>>>>>>>>>> > >>>>>>>>>>>> Could you explain a little about the bug? > >>>>>>>>>>>> > >>>>>>>>>>>> Why should stack traces be collected if > MaxJavaStackTraceDepth > >>>>>>>>>>>> is set to 0. > >>>>>>>>>>> > >>>>>>>>>>> "0" in MaxJavaStackTraceDepth means all exception stack > traces > >>>>>>>>>>> will be shown. > >>>>>>>>>>> > >>>>>>>>>>> > http://hg.openjdk.java.net/jdk/jdk/file/72ce7dd54939/src/hotspot/share/runtime/globals.hpp#l1553 > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> JDK-7179701 reports similar issue. > >>>>>>>>>>> I think we can take similar approach for JDK-8219566. > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> Thanks, > >>>>>>>>>>> > >>>>>>>>>>> Yasumasa > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> Thanks > >>>>>>>>>>>> Erik > >>>>>>>>>>>> > >>>>>>>>>>>>> On 22 Feb 2019, at 07:48, Yasumasa Suenaga at gmail.com > > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>> Hi all, > >>>>>>>>>>>>> > >>>>>>>>>>>>> Please review this change: > >>>>>>>>>>>>> > >>>>>>>>>>>>>? ? ?JBS: https://bugs.openjdk.java.net/browse/JDK-8219566 > >>>>>>>>>>>>>? ? ?webrev: > >>>>>>>>>>>>> > http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.00/ > >>>>>>>>>>>>> > >>>>>>>>>>>>> JFR did not collect call stacks when > MaxJavaStackTraceDepth is > >>>>>>>>>>>>> set to zero. > >>>>>>>>>>>>> This bug is similar to JDK-7179701. > >>>>>>>>>>>>> > >>>>>>>>>>>>> This change passed tests on submit repo and :jdk_jfr > jtreg tests. > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>> > >>>>>>>>>>>>> Yasumasa > >>>>>>>>>>>> > >>>>>>>>> > >>>>> > From erik.gahlin at oracle.com Tue Apr 23 13:16:21 2019 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Tue, 23 Apr 2019 15:16:21 +0200 Subject: RFR: 8221569: JFR tool produces incorrect output when both --categories and --events are specified Message-ID: <5CBF1025.9060003@oracle.com> Hi, Could I have a review of a fix that makes sure the --events and -categories options can be used at the same time in the jfr tool. The fix also improves the error message if a user mistakenly types --event or --category. Webrev: http://cr.openjdk.java.net/~egahlin/8221569/ Bug: https://bugs.openjdk.java.net/browse/JDK-8221569 Testing: Tests in test/jdk/jdk/jfr Thanks Erik From markus.gronlund at oracle.com Tue Apr 23 14:13:14 2019 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Tue, 23 Apr 2019 07:13:14 -0700 (PDT) Subject: RFR: 8221569: JFR tool produces incorrect output when both --categories and --events are specified In-Reply-To: <5CBF1025.9060003@oracle.com> References: <5CBF1025.9060003@oracle.com> Message-ID: <3d5e1ef3-2b80-4793-9773-a4f0e13adc66@default> Looks good. Markus -----Original Message----- From: Erik Gahlin Sent: den 23 april 2019 15:16 To: hotspot-jfr-dev at openjdk.java.net Subject: RFR: 8221569: JFR tool produces incorrect output when both --categories and --events are specified Hi, Could I have a review of a fix that makes sure the --events and -categories options can be used at the same time in the jfr tool. The fix also improves the error message if a user mistakenly types --event or --category. Webrev: http://cr.openjdk.java.net/~egahlin/8221569/ Bug: https://bugs.openjdk.java.net/browse/JDK-8221569 Testing: Tests in test/jdk/jdk/jfr Thanks Erik From yasuenag at gmail.com Wed Apr 24 01:51:57 2019 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Wed, 24 Apr 2019 10:51:57 +0900 Subject: PING: RFR: 8219566: JFR did not collect call stacks when MaxJavaStackTraceDepth is set to zero In-Reply-To: <915d2ca3-7923-4591-9b44-299e93817d7a@default> References: <6D0A14CC-3EBA-4A01-AFFC-F2987F3C4DFD@oracle.com> <59ed9471-45a5-5707-8356-367ae35a7829@gmail.com> <19551e1b-b9e8-62e1-6fab-7dd6e100f50a@gmail.com> <5C828C38.1090506@oracle.com> <47d27b56-0933-c173-55c1-7e225218ead8@gmail.com> <0b31693a-a8d4-1fdf-c830-8752f3ec379e@gmail.com> <5C95369B.8030404@oracle.com> <7c203381-0091-f9a9-6e76-1b09d0e92c88@gmail.com> <2ca33c8e-6565-fad3-294c-ae1804271f54@gmail.com> <5CB0D24B.305@oracle.com> <537B942B-1E3B-42B6-A121-05B5D7CC9CD9@oracle.com> <07eaa3dc-fa15-d38f-2b5e-dbebc51a2a7a@gmail.com> <915d2ca3-7923-4591-9b44-299e93817d7a@default> Message-ID: <1d05cb79-e142-0a29-c0b1-b887f88317cf@gmail.com> Hi Markus, Thank you for your comment. I fixed them in webrev.04 . Could you review again? http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.04/ Yasumasa On 2019/04/23 21:37, Markus Gronlund wrote: > src/hotspot/share/jfr/periodic/sampling/jfrCallTrace.cpp > Need #include "jfr/utilities/jfrTypes.hpp" > > src/hotspot/share/jfr/recorder/service/jfrOptionSet.cpp > Need #include "jfr/utilities/jfrTypes.hpp" > > src/hotspot/share/jfr/recorder/stacktrace/jfrStackTraceRepository.cpp > loop_count and loop_max might need to have type u4 instead of int to avoid unsigned / signed type warning / errors > > src/hotspot/share/jfr/utilities/jfrTypes.hpp > No comment. > > Thanks > Markus > > -----Original Message----- > From: Yasumasa Suenaga > Sent: den 22 april 2019 12:23 > To: hotspot-jfr-dev at openjdk.java.net > Subject: PING: RFR: 8219566: JFR did not collect call stacks when MaxJavaStackTraceDepth is set to zero > > PING: Could you review it? > > JBS: https://bugs.openjdk.java.net/browse/JDK-8219566 > webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.03/ > > I need one more reviewer to push. > > > Yasumasa > > > On 2019/04/15 8:14, Yasumasa Suenaga wrote: >> Thanks Erik! >> >> I'm waiting for another reviewer for webrev.03 . >> >> >> Yasumasa >> >> >> 2019?4?15?(?) 7:43 Erik Gahlin > >: >> >> Looks good (webrev.03) >> >> According to the option spec. MaxJavaStackTraceDepth controls "the >> maximum number of lines in the stack trace for Java exceptions?. >> This is not an exception stack trace, so I think it would be more >> appropriate to use MAX_STACK_DEPTH. >> >> Thanks >> Erik >> >> >> > On 14 Apr 2019, at 15:05, Yasumasa Suenaga > > wrote: >> > >> > Hi Erik, >> > >> > How about this change? >> > http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.03/ >> > >> > IMHO I prefer webrev.00 if we need to think stackdepth is set to >> lower value. >> > Fix in webrev.00 is introduced in JDK-7179701. >> > >> > As you said before, it does not make the code easier to read. >> > >> > So if webrev.03 is appropriate, I will push webrev.03. >> > >> > >> > Thanks, >> > >> > Yasumasa >> > >> > >> > On 2019/04/13 3:00, Erik Gahlin wrote: >> >> Hi, >> >> Could we use the max stack depth used by JFR instead? >> >> It's not available as constant in a header file, but maybe you >> could move: >> >> static const u4 STACK_DEPTH_DEFAULT = 64; >> >> static const u4 MIN_STACK_DEPTH = 1; >> >> static const u4 MAX_STACK_DEPTH = 2048; >> >> from jfrOptionSet.cpp to jfrTypes.hpp where we have other >> constants. The benefit of using MAX_STACK_DEPTH is that it will work >> in scenarios where a user has set the number of frames to a low >> number, i.e -XX:FlightRecorderOptions=5, and we hit a stack with >> lots of frames that we can't record, i.e native frames. >> >> Thanks >> >> Erik >> >>> Hi, >> >>> >> >>> Do you have any concern(s) about this change? >> >>> >> >>>>>>? ? webrev: >> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.02/ >> >>> >> >>> I think thread stack should be collected by JFR even if >> MaxJavaStackTraceDepth is set to zero. >> >>> >> >>> >> >>> Thanks, >> >>> >> >>> Yasumasa >> >>> >> >>> >> >>> On 2019/03/23 21:03, Yasumasa Suenaga wrote: >> >>>> Hi Erik, >> >>>> >> >>>> MaxJavaStackTraceDepth do not only affect finding top stack >> but also stack walking. >> >>>> >> >>>> JfrGetCallTrace::find_top_frame() uses MaxJavaStackTraceDepth >> to find top frame as you say. >> >>>> OTOH vframeStreamSamples::samples_next() uses it to find next >> valid frame. >> >>>> >> >>>> >> >>>> Anyway thread stack should be collected even if >> MaxJavaStackTraceDepth is set to zero. >> >>>> >> >>>> >> >>>> Thanks, >> >>>> >> >>>> Yasumasa >> >>>> >> >>>> >> >>>> On 2019/03/23 4:25, Erik Gahlin wrote: >> >>>>> To me, it seems that the purpose of the method is to find the >> first >> >>>>> valid frame on the stack (the top frame). This is orthogonal >> to the >> >>>>> number of frames that should be recorded by JFR. >> >>>>> >> >>>>> Erik >> >>>>> >> >>>>>> PING: Could you review it? >> >>>>>> >> >>>>>>? ? JBS: https://bugs.openjdk.java.net/browse/JDK-8219566 >> >>>>>>? ? webrev: >> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.02/ >> >>>>>> >> >>>>>> This change has passed :jdk_jfr tests and submit repo. >> >>>>>> >> >>>>>> >> >>>>>> Thanks, >> >>>>>> >> >>>>>> Yasumasa >> >>>>>> >> >>>>>> >> >>>>>> On 2019/03/11 14:49, Yasumasa Suenaga wrote: >> >>>>>>> 2019?3?9?(?) 20:47 Yasumasa Suenaga > >: >> >>>>>>>> >> >>>>>>>> Hi Erik, >> >>>>>>>> >> >>>>>>>> Reading your email, I think we should use >> JfrOptionSet::stackdepth() >> >>>>>>>> rather than MaxJavaStackTraceDepth. >> >>>>>>>> I created new webrev, and it passed jdk_jfr jtreg tests on >> my Linux >> >>>>>>>> x64 box. >> >>>>>>>> >> >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.01/ >> >>>>>>>> >> >>>>>>>> However it has not yet passed tests on submit repo >> >>>>>>>> (mach5-one-ysuenaga-JDK-8219566-20190309-0909-1079793). >> >>>>>>>> It seems to be failed at build phase which includes Linux. >> >>>>>>>> (I can build with this change on Fedora 29 x64). >> >>>>>>>> >> >>>>>>>> Could you help if this webrev is ok? >> >>>>>>> >> >>>>>>> I found 1 warning, and I fixed it. >> >>>>>>> This webrev works fine with :jdk_jfr jtreg tests on my >> Linux box, and >> >>>>>>> submit repo test >> >>>>>>> (mach5-one-ysuenaga-JDK-8219566-20190311-0150-1121570). >> >>>>>>> Could you review it? >> >>>>>>> >> >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.02/ >> >>>>>>> >> >>>>>>> >> >>>>>>> Thanks, >> >>>>>>> >> >>>>>>> Yasumasa >> >>>>>>> >> >>>>>>> >> >>>>>>>> Thanks, >> >>>>>>>> >> >>>>>>>> Yasumasa >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> On 2019/03/09 0:37, Erik Gahlin wrote: >> >>>>>>>>> JFR imposes a limit on how many frames that can be >> recorded, set to >> >>>>>>>>> 64 by default but configurable up to 2048 using >> >>>>>>>>> -XX:FlightRecorderOptions, so setting the value to 0 will >> still not >> >>>>>>>>> mean all frames. >> >>>>>>>>> >> >>>>>>>>> That said, if you still want to fix it. It may make sense >> to set >> >>>>>>>>> loop_max to INT_MAX, if MaxJavaStackTraceDepth is zero, i.e >> >>>>>>>>> >> >>>>>>>>> int loop_max = MaxJavaStackTraceDepth == 0 ? INT_MAX : >> >>>>>>>>> MaxJavaStackTraceDepth * 2; >> >>>>>>>>> >> >>>>>>>>> and avoid evaluating it for every frame. Even if the >> compiler is >> >>>>>>>>> able to hoist the comparison, it makes the code easier to >> read. >> >>>>>>>>> >> >>>>>>>>> Thanks >> >>>>>>>>> Erik >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>>> PING: Could you review it? >> >>>>>>>>>> >> >>>>>>>>>>>>>? ? ?JBS: https://bugs.openjdk.java.net/browse/JDK-8219566 >> >>>>>>>>>>>>>? ? ?webrev: >> >>>>>>>>>>>>> >> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.00/ >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> If you have any questions, please tell me. >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> Thanks, >> >>>>>>>>>> >> >>>>>>>>>> Yasumasa >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> On 2019/02/27 22:46, Yasumasa Suenaga wrote: >> >>>>>>>>>>> Hi Erik, >> >>>>>>>>>>> >> >>>>>>>>>>> On 2019/02/27 22:30, Erik Gahlin wrote: >> >>>>>>>>>>>> Hi Yasumasa, >> >>>>>>>>>>>> >> >>>>>>>>>>>> Could you explain a little about the bug? >> >>>>>>>>>>>> >> >>>>>>>>>>>> Why should stack traces be collected if >> MaxJavaStackTraceDepth >> >>>>>>>>>>>> is set to 0. >> >>>>>>>>>>> >> >>>>>>>>>>> "0" in MaxJavaStackTraceDepth means all exception stack >> traces >> >>>>>>>>>>> will be shown. >> >>>>>>>>>>> >> >>>>>>>>>>> >> http://hg.openjdk.java.net/jdk/jdk/file/72ce7dd54939/src/hotspot/share/runtime/globals.hpp#l1553 >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> JDK-7179701 reports similar issue. >> >>>>>>>>>>> I think we can take similar approach for JDK-8219566. >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> Thanks, >> >>>>>>>>>>> >> >>>>>>>>>>> Yasumasa >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>>> Thanks >> >>>>>>>>>>>> Erik >> >>>>>>>>>>>> >> >>>>>>>>>>>>> On 22 Feb 2019, at 07:48, Yasumasa Suenaga > at gmail.com > >> >>>>>>>>>>>>> wrote: >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> Hi all, >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> Please review this change: >> >>>>>>>>>>>>> >> >>>>>>>>>>>>>? ? ?JBS: https://bugs.openjdk.java.net/browse/JDK-8219566 >> >>>>>>>>>>>>>? ? ?webrev: >> >>>>>>>>>>>>> >> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.00/ >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> JFR did not collect call stacks when >> MaxJavaStackTraceDepth is >> >>>>>>>>>>>>> set to zero. >> >>>>>>>>>>>>> This bug is similar to JDK-7179701. >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> This change passed tests on submit repo and :jdk_jfr >> jtreg tests. >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> Thanks, >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> Yasumasa >> >>>>>>>>>>>> >> >>>>>>>>> >> >>>>> >> From MARKUS.GRONLUND at ORACLE.COM Wed Apr 24 07:44:56 2019 From: MARKUS.GRONLUND at ORACLE.COM (Markus Gronlund) Date: Wed, 24 Apr 2019 09:44:56 +0200 Subject: PING: RFR: 8219566: JFR did not collect call stacks when MaxJavaStackTraceDepth is set to zero In-Reply-To: <1d05cb79-e142-0a29-c0b1-b887f88317cf@gmail.com> References: <6D0A14CC-3EBA-4A01-AFFC-F2987F3C4DFD@oracle.com> <59ed9471-45a5-5707-8356-367ae35a7829@gmail.com> <19551e1b-b9e8-62e1-6fab-7dd6e100f50a@gmail.com> <5C828C38.1090506@oracle.com> <47d27b56-0933-c173-55c1-7e225218ead8@gmail.com> <0b31693a-a8d4-1fdf-c830-8752f3ec379e@gmail.com> <5C95369B.8030404@oracle.com> <7c203381-0091-f9a9-6e76-1b09d0e92c88@gmail.com> <2ca33c8e-6565-fad3-294c-ae1804271f54@gmail.com> <5CB0D24B.305@oracle.com> <537B942B-1E3B-42B6-A121-05B5D7CC9CD9@oracle.com> <07eaa3dc-fa15-d38f-2b5e-dbebc51a2a7a@gmail.com> <915d2ca3-7923-4591-9b44-299e93817d7a@default> <1d05cb79-e142-0a29-c0b1-b887f88317cf@gmail.com> Message-ID: <5B97A480-BB7F-444B-8B66-75D524214FAF@ORACLE.COM> Ok, looks good. Markus Sent from my iPhone > On 24 Apr 2019, at 03:51, Yasumasa Suenaga wrote: > > Hi Markus, > > Thank you for your comment. > > I fixed them in webrev.04 . Could you review again? > > http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.04/ > > > Yasumasa > > >> On 2019/04/23 21:37, Markus Gronlund wrote: >> src/hotspot/share/jfr/periodic/sampling/jfrCallTrace.cpp >> Need #include "jfr/utilities/jfrTypes.hpp" >> src/hotspot/share/jfr/recorder/service/jfrOptionSet.cpp >> Need #include "jfr/utilities/jfrTypes.hpp" >> src/hotspot/share/jfr/recorder/stacktrace/jfrStackTraceRepository.cpp >> loop_count and loop_max might need to have type u4 instead of int to avoid unsigned / signed type warning / errors >> src/hotspot/share/jfr/utilities/jfrTypes.hpp >> No comment. >> Thanks >> Markus >> -----Original Message----- >> From: Yasumasa Suenaga >> Sent: den 22 april 2019 12:23 >> To: hotspot-jfr-dev at openjdk.java.net >> Subject: PING: RFR: 8219566: JFR did not collect call stacks when MaxJavaStackTraceDepth is set to zero >> PING: Could you review it? >> JBS: https://bugs.openjdk.java.net/browse/JDK-8219566 >> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.03/ >> I need one more reviewer to push. >> Yasumasa >>> On 2019/04/15 8:14, Yasumasa Suenaga wrote: >>> Thanks Erik! >>> >>> I'm waiting for another reviewer for webrev.03 . >>> >>> >>> Yasumasa >>> >>> >>> 2019?4?15?(?) 7:43 Erik Gahlin >> >: >>> >>> Looks good (webrev.03) >>> >>> According to the option spec. MaxJavaStackTraceDepth controls "the >>> maximum number of lines in the stack trace for Java exceptions?. >>> This is not an exception stack trace, so I think it would be more >>> appropriate to use MAX_STACK_DEPTH. >>> >>> Thanks >>> Erik >>> >>> >>> > On 14 Apr 2019, at 15:05, Yasumasa Suenaga >> > wrote: >>> > >>> > Hi Erik, >>> > >>> > How about this change? >>> > http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.03/ >>> > >>> > IMHO I prefer webrev.00 if we need to think stackdepth is set to >>> lower value. >>> > Fix in webrev.00 is introduced in JDK-7179701. >>> > >>> > As you said before, it does not make the code easier to read. >>> > >>> > So if webrev.03 is appropriate, I will push webrev.03. >>> > >>> > >>> > Thanks, >>> > >>> > Yasumasa >>> > >>> > >>> > On 2019/04/13 3:00, Erik Gahlin wrote: >>> >> Hi, >>> >> Could we use the max stack depth used by JFR instead? >>> >> It's not available as constant in a header file, but maybe you >>> could move: >>> >> static const u4 STACK_DEPTH_DEFAULT = 64; >>> >> static const u4 MIN_STACK_DEPTH = 1; >>> >> static const u4 MAX_STACK_DEPTH = 2048; >>> >> from jfrOptionSet.cpp to jfrTypes.hpp where we have other >>> constants. The benefit of using MAX_STACK_DEPTH is that it will work >>> in scenarios where a user has set the number of frames to a low >>> number, i.e -XX:FlightRecorderOptions=5, and we hit a stack with >>> lots of frames that we can't record, i.e native frames. >>> >> Thanks >>> >> Erik >>> >>> Hi, >>> >>> >>> >>> Do you have any concern(s) about this change? >>> >>> >>> >>>>>> webrev: >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.02/ >>> >>> >>> >>> I think thread stack should be collected by JFR even if >>> MaxJavaStackTraceDepth is set to zero. >>> >>> >>> >>> >>> >>> Thanks, >>> >>> >>> >>> Yasumasa >>> >>> >>> >>> >>> >>> On 2019/03/23 21:03, Yasumasa Suenaga wrote: >>> >>>> Hi Erik, >>> >>>> >>> >>>> MaxJavaStackTraceDepth do not only affect finding top stack >>> but also stack walking. >>> >>>> >>> >>>> JfrGetCallTrace::find_top_frame() uses MaxJavaStackTraceDepth >>> to find top frame as you say. >>> >>>> OTOH vframeStreamSamples::samples_next() uses it to find next >>> valid frame. >>> >>>> >>> >>>> >>> >>>> Anyway thread stack should be collected even if >>> MaxJavaStackTraceDepth is set to zero. >>> >>>> >>> >>>> >>> >>>> Thanks, >>> >>>> >>> >>>> Yasumasa >>> >>>> >>> >>>> >>> >>>> On 2019/03/23 4:25, Erik Gahlin wrote: >>> >>>>> To me, it seems that the purpose of the method is to find the >>> first >>> >>>>> valid frame on the stack (the top frame). This is orthogonal >>> to the >>> >>>>> number of frames that should be recorded by JFR. >>> >>>>> >>> >>>>> Erik >>> >>>>> >>> >>>>>> PING: Could you review it? >>> >>>>>> >>> >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8219566 >>> >>>>>> webrev: >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.02/ >>> >>>>>> >>> >>>>>> This change has passed :jdk_jfr tests and submit repo. >>> >>>>>> >>> >>>>>> >>> >>>>>> Thanks, >>> >>>>>> >>> >>>>>> Yasumasa >>> >>>>>> >>> >>>>>> >>> >>>>>> On 2019/03/11 14:49, Yasumasa Suenaga wrote: >>> >>>>>>> 2019?3?9?(?) 20:47 Yasumasa Suenaga >> >: >>> >>>>>>>> >>> >>>>>>>> Hi Erik, >>> >>>>>>>> >>> >>>>>>>> Reading your email, I think we should use >>> JfrOptionSet::stackdepth() >>> >>>>>>>> rather than MaxJavaStackTraceDepth. >>> >>>>>>>> I created new webrev, and it passed jdk_jfr jtreg tests on >>> my Linux >>> >>>>>>>> x64 box. >>> >>>>>>>> >>> >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.01/ >>> >>>>>>>> >>> >>>>>>>> However it has not yet passed tests on submit repo >>> >>>>>>>> (mach5-one-ysuenaga-JDK-8219566-20190309-0909-1079793). >>> >>>>>>>> It seems to be failed at build phase which includes Linux. >>> >>>>>>>> (I can build with this change on Fedora 29 x64). >>> >>>>>>>> >>> >>>>>>>> Could you help if this webrev is ok? >>> >>>>>>> >>> >>>>>>> I found 1 warning, and I fixed it. >>> >>>>>>> This webrev works fine with :jdk_jfr jtreg tests on my >>> Linux box, and >>> >>>>>>> submit repo test >>> >>>>>>> (mach5-one-ysuenaga-JDK-8219566-20190311-0150-1121570). >>> >>>>>>> Could you review it? >>> >>>>>>> >>> >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.02/ >>> >>>>>>> >>> >>>>>>> >>> >>>>>>> Thanks, >>> >>>>>>> >>> >>>>>>> Yasumasa >>> >>>>>>> >>> >>>>>>> >>> >>>>>>>> Thanks, >>> >>>>>>>> >>> >>>>>>>> Yasumasa >>> >>>>>>>> >>> >>>>>>>> >>> >>>>>>>> >>> >>>>>>>> On 2019/03/09 0:37, Erik Gahlin wrote: >>> >>>>>>>>> JFR imposes a limit on how many frames that can be >>> recorded, set to >>> >>>>>>>>> 64 by default but configurable up to 2048 using >>> >>>>>>>>> -XX:FlightRecorderOptions, so setting the value to 0 will >>> still not >>> >>>>>>>>> mean all frames. >>> >>>>>>>>> >>> >>>>>>>>> That said, if you still want to fix it. It may make sense >>> to set >>> >>>>>>>>> loop_max to INT_MAX, if MaxJavaStackTraceDepth is zero, i.e >>> >>>>>>>>> >>> >>>>>>>>> int loop_max = MaxJavaStackTraceDepth == 0 ? INT_MAX : >>> >>>>>>>>> MaxJavaStackTraceDepth * 2; >>> >>>>>>>>> >>> >>>>>>>>> and avoid evaluating it for every frame. Even if the >>> compiler is >>> >>>>>>>>> able to hoist the comparison, it makes the code easier to >>> read. >>> >>>>>>>>> >>> >>>>>>>>> Thanks >>> >>>>>>>>> Erik >>> >>>>>>>>> >>> >>>>>>>>> >>> >>>>>>>>>> PING: Could you review it? >>> >>>>>>>>>> >>> >>>>>>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8219566 >>> >>>>>>>>>>>>> webrev: >>> >>>>>>>>>>>>> >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.00/ >>> >>>>>>>>>> >>> >>>>>>>>>> >>> >>>>>>>>>> If you have any questions, please tell me. >>> >>>>>>>>>> >>> >>>>>>>>>> >>> >>>>>>>>>> Thanks, >>> >>>>>>>>>> >>> >>>>>>>>>> Yasumasa >>> >>>>>>>>>> >>> >>>>>>>>>> >>> >>>>>>>>>> On 2019/02/27 22:46, Yasumasa Suenaga wrote: >>> >>>>>>>>>>> Hi Erik, >>> >>>>>>>>>>> >>> >>>>>>>>>>> On 2019/02/27 22:30, Erik Gahlin wrote: >>> >>>>>>>>>>>> Hi Yasumasa, >>> >>>>>>>>>>>> >>> >>>>>>>>>>>> Could you explain a little about the bug? >>> >>>>>>>>>>>> >>> >>>>>>>>>>>> Why should stack traces be collected if >>> MaxJavaStackTraceDepth >>> >>>>>>>>>>>> is set to 0. >>> >>>>>>>>>>> >>> >>>>>>>>>>> "0" in MaxJavaStackTraceDepth means all exception stack >>> traces >>> >>>>>>>>>>> will be shown. >>> >>>>>>>>>>> >>> >>>>>>>>>>> >>> http://hg.openjdk.java.net/jdk/jdk/file/72ce7dd54939/src/hotspot/share/runtime/globals.hpp#l1553 >>> >>>>>>>>>>> >>> >>>>>>>>>>> >>> >>>>>>>>>>> JDK-7179701 reports similar issue. >>> >>>>>>>>>>> I think we can take similar approach for JDK-8219566. >>> >>>>>>>>>>> >>> >>>>>>>>>>> >>> >>>>>>>>>>> Thanks, >>> >>>>>>>>>>> >>> >>>>>>>>>>> Yasumasa >>> >>>>>>>>>>> >>> >>>>>>>>>>> >>> >>>>>>>>>>>> Thanks >>> >>>>>>>>>>>> Erik >>> >>>>>>>>>>>> >>> >>>>>>>>>>>>> On 22 Feb 2019, at 07:48, Yasumasa Suenaga >> at gmail.com > >>> >>>>>>>>>>>>> wrote: >>> >>>>>>>>>>>>> >>> >>>>>>>>>>>>> Hi all, >>> >>>>>>>>>>>>> >>> >>>>>>>>>>>>> Please review this change: >>> >>>>>>>>>>>>> >>> >>>>>>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8219566 >>> >>>>>>>>>>>>> webrev: >>> >>>>>>>>>>>>> >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8219566/webrev.00/ >>> >>>>>>>>>>>>> >>> >>>>>>>>>>>>> JFR did not collect call stacks when >>> MaxJavaStackTraceDepth is >>> >>>>>>>>>>>>> set to zero. >>> >>>>>>>>>>>>> This bug is similar to JDK-7179701. >>> >>>>>>>>>>>>> >>> >>>>>>>>>>>>> This change passed tests on submit repo and :jdk_jfr >>> jtreg tests. >>> >>>>>>>>>>>>> >>> >>>>>>>>>>>>> >>> >>>>>>>>>>>>> Thanks, >>> >>>>>>>>>>>>> >>> >>>>>>>>>>>>> Yasumasa >>> >>>>>>>>>>>> >>> >>>>>>>>> >>> >>>>> >>> From yasuenag at gmail.com Wed Apr 24 12:43:14 2019 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Wed, 24 Apr 2019 21:43:14 +0900 Subject: PING: RFR: 8220657: JFR.dump does not work when filename is set In-Reply-To: <5CBE36B6.6070603@oracle.com> References: <5C90E8C9.3010008@oracle.com> <14c04f3a-1fc0-af3b-714f-e35fa18d6a1b@gmail.com> <5C9103DE.4040304@oracle.com> <725f2ab6-71ee-4149-9d1d-ca6c19217408@gmail.com> <2dc4e611-bd18-a187-f529-d243af51cf10@gmail.com> <5CBE36B6.6070603@oracle.com> Message-ID: <6348d8ac-b713-1cb3-39db-859bc833e0ce@gmail.com> Hi Erik, I uploaded new webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.02/ I kept to remove setDestination() call in PlatformRecording. In this webrev, original filename value will be set to temporary snapshot if DCmdDump does not have recording name and filename. Also I added new testcase. It can test JFR.dump with filename. Thanks, Yasumasa On 2019/04/23 6:48, Erik Gahlin wrote: > I have looked further at the issue and while the message to the user was > incorrect previously ( "Dump failed. No data found in the specified > interval") it did result in a file with the specified filename, i.e > "test.jfr" > > When "clone.setDestination(this.destination);" is removed, the filename > passed on command line is ignored, but it does result in a message that > is consistent with what is happening (a file with an autogenerated name > is created). It does however change the behavior of 'JFR.dump > name=1'. Logic needs to be added to DCmdDump so it uses an existing > filename if it is set, but changes it if a user is providing an override. > > When comes to testing. I think it would be better to write a separate > test that checks that things are dumped correctly > when-XX:StartFlightRecording:filename= is used. > > Thanks > Erik > >> PING: Do you have any opinion(s) about this fix? >> >> >> Yasumasa >> >> >> On 2019/04/01 21:46, Yasumasa Suenaga wrote: >>> Hi, >>> >>> Do you have concern(s) about this change? >>> >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.01/ >>> >>> As in the email below, I think it does not affect scenarios except >>> snapshot cloning. >>> Please tell me if you have any opinions about this. >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2019/03/20 10:37, Yasumasa Suenaga wrote: >>>> Hi Erik, >>>> >>>> I think this change does not affect other scenarios. >>>> >>>> :jdk_jfr jtreg tests and submit repo have passed with this change. >>>> newSnapshotClone() is called by PlatformRecording::dump and >>>> DCmdDump::newSnapshot. >>>> Both methods are related to user-requested dump. >>>> >>>> What scenarios do you think? I want to check them. >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> 2019?3?19?(?) 23:59 Erik Gahlin : >>>>> >>>>> Yes, but then you would lose the original destination in other >>>>> scenarios >>>>> >>>>> Maybe I can look at this next week. >>>>> >>>>> Erik >>>>> >>>>>> Hi Erik, >>>>>> >>>>>> Thank you for your reply. >>>>>> >>>>>> How about this changeset? >>>>>> >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.01/ >>>>>> >>>>>> I think PlatformRecording#newSnapshotClone() should not dump. >>>>>> So I removed `setDestination()` from this. >>>>>> >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2019/03/19 22:04, Erik Gahlin wrote: >>>>>>> This looks a bit hackish. >>>>>>> >>>>>>> The real issue seems to be that the method >>>>>>> PlatformRecording#newSnapshotClone incorrectly sets the >>>>>>> destination in >>>>>>> the clone, which then triggers a dump prematurely (when stop is >>>>>>> called). >>>>>>> >>>>>>> Purpose of that method is just to take a snapshot of existing >>>>>>> chunks in >>>>>>> the disk repository (or from memory). and then let >>>>>>> dumpStopped(path) do >>>>>>> the actual dump (copy out the data from the disk repository). >>>>>>> >>>>>>> To make it work, some other changes would be needed as well, but >>>>>>> I don't >>>>>>> have time to investigate. >>>>>>> >>>>>>> Erik >>>>>>> >>>>>>>> Hi all, >>>>>>>> >>>>>>>> Please review this change. >>>>>>>> >>>>>>>> ???? JBS: https://bugs.openjdk.java.net/browse/JDK-8220657 >>>>>>>> ???? webrev: >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8220657/webrev.00/ >>>>>>>> >>>>>>>> The user might not get flight record via JFR.dump if JFR is started >>>>>>>> with filename option on target process. >>>>>>>> >>>>>>>> Please see JBS if you want to know how to reproduce. >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>> > From markus.gronlund at oracle.com Thu Apr 25 20:25:22 2019 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Thu, 25 Apr 2019 13:25:22 -0700 (PDT) Subject: RFR(XS): 8221121: applications/microbenchmarks are encountering crashes in tier5 Message-ID: <1515916c-6187-46d4-8815-5e824284e04b@default> Greetings, Please review this small patch to address the following: Bug: https://bugs.openjdk.java.net/browse/JDK-8221121 Webrev: http://cr.openjdk.java.net/~mgronlun/8221121/webrev01/ Description: The applications/microbenchmarks added to tier5 are failing in some instances (debug builds), with, as an example, the following trace: # # A fatal error has been detected by the Java Runtime Environment: # # Internal Error (/scratch/opt/mach5/mesos/work_dir/slaves/2dd962d0-8988-479b-a804-57ab764ada59-S77631/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/d8f4cb38-0dec-4477-a89a-f62853433c56/runs/cb3f55fa-9e13-481c-a7d6-9f33d2b8b457/workspace/open/src/hotspot/share/jfr/recorder/storage/jfrMemorySpace.inline.hpp:85), pid=5748, tid=5770 # assert(t->identity() == __null) failed: invariant This occurred when JFR was running with the in-memory configuration where buffers are reused FIFO-style. In the implementation, an "age node" will manage a full buffer for its reclamation, and age nodes provides for a linked "full" (fifo) list. Issue: The age node was not expected to retain an identity after being added to the full list, where the assertion fired during the subsequent discard-reuse processing step. This situation only manifests with running JFR in-memory configurations and using debug builds. Fix is to release the age node before insertion onto full list. Thanks Markus From erik.gahlin at oracle.com Fri Apr 26 18:06:35 2019 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Fri, 26 Apr 2019 20:06:35 +0200 Subject: RFR(XS): 8221121: applications/microbenchmarks are encountering crashes in tier5 In-Reply-To: <1515916c-6187-46d4-8815-5e824284e04b@default> References: <1515916c-6187-46d4-8815-5e824284e04b@default> Message-ID: <5CC348AB.7080704@oracle.com> Looks good. Erik > Greetings, > > Please review this small patch to address the following: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8221121 > Webrev: http://cr.openjdk.java.net/~mgronlun/8221121/webrev01/ > > Description: > > The applications/microbenchmarks added to tier5 are failing in some instances (debug builds), with, as an example, the following trace: > > # > # A fatal error has been detected by the Java Runtime Environment: > # > # Internal Error (/scratch/opt/mach5/mesos/work_dir/slaves/2dd962d0-8988-479b-a804-57ab764ada59-S77631/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/d8f4cb38-0dec-4477-a89a-f62853433c56/runs/cb3f55fa-9e13-481c-a7d6-9f33d2b8b457/workspace/open/src/hotspot/share/jfr/recorder/storage/jfrMemorySpace.inline.hpp:85), pid=5748, tid=5770 > # assert(t->identity() == __null) failed: invariant > > This occurred when JFR was running with the in-memory configuration where buffers are reused FIFO-style. > In the implementation, an "age node" will manage a full buffer for its reclamation, and age nodes provides for a linked "full" (fifo) list. > > Issue: > The age node was not expected to retain an identity after being added to the full list, where the assertion fired during the subsequent discard-reuse processing step. > This situation only manifests with running JFR in-memory configurations and using debug builds. > > Fix is to release the age node before insertion onto full list. > > Thanks > Markus From chihiro.ito at oracle.com Sun Apr 28 08:09:13 2019 From: chihiro.ito at oracle.com (Chihiro Ito) Date: Sun, 28 Apr 2019 17:09:13 +0900 Subject: PING: RFR 8218631: jfr tools is effective only the last filter In-Reply-To: <5CB78859.1010203@oracle.com> References: <2e94f465-01f4-f40b-4726-d28d5041faa6@oracle.com> <152D2629-65F0-4A42-B2B3-25F3EB3321B3@oracle.com> <5CB78859.1010203@oracle.com> Message-ID: Hi Erik, I'm grad to help you fix the issue. As you suggested, I would appreciate it if you could set me as a contributor. Regards, Chihiro On 2019/04/18 5:11, Erik Gahlin wrote: > Hi Chihiro, > > Another issue surfaced that would have been fixed by your patch. See > > "8221569: JFR tool produces incorrect output when both --category and > --event are specified" > https://bugs.openjdk.java.net/browse/JDK-8221569 > > I modified your patch and added checks to prevent --categories and > --events to be specified multiple times. > > Webrev: > http://cr.openjdk.java.net/~egahlin/8218631/ > > If you like I can set you as contributor and sponsor it for you? (and > close 8221569 as a duplicate) > > (An argument can be made that a user should be able to specify > --events multiple times and get the union, but that is not according > to spec. which is [--events ] without repeat. Having multiple > ways to achieve the same goal may confuse users, i.e. did the event > not get printed because of multiple --events parameters, or because it > was not in the file?) > > Thanks > Erik > >> Hi Chihiro, >> >> If you want to use multiple filters, you can specify them like this: >> >> ?events filter1,filter2,filter3 >> ?categories filter1,filter2,filter2 >> >> We could add an error message, if a user specifies the ?events flag >> more than once, and describe how multiple filters can be specified, >> but I don?t think there should be multiple ways to specify filters. >> >> Thanks >> Erik >> >> >>> On 21 Feb 2019, at 08:38, Chihiro Ito wrote: >>> >>> Hi All, >>> >>> Could you review this change, please? >>> >>> webrev: >>> http://cr.openjdk.java.net/~cito/JDK-8218631/webrev.00/ >>> >>> JBS: >>> https://bugs.openjdk.java.net/browse/JDK-8218631 >>> >>> This change passed all tests in test-tier1. >>> >>> Regards, >>> Chihiro >>> >>> >>> On 2019/02/08 2:46, Chihiro Ito wrote: >>>> Hi, >>>> >>>> Could you review this change, please? >>>> >>>> I have referred to the setting of Xlog, and I changed this should >>>> enable every filter as follows. >>>> >>>> jfr print --events jdk.JVMInformation --events jdk.OSInformation >>>> profiling-recording4399235951575204469.jfr >>>> jdk.JVMInformation { >>>> startTime = 17:40:16.536 >>>> jvmName = "OpenJDK 64-Bit Server VM" >>>> jvmVersion = "OpenJDK 64-Bit Server VM (fastdebug >>>> 13-internal+0-adhoc.chito.jdk) for linux-amd64 JRE >>>> (13-internal+0-adhoc.chito.jdk), built on Feb 5 2019 23:57:41 by >>>> "chito" with gcc 4.8.5 20150623 (Red Hat 4.8.5-28.0.1)" >>>> jvmArguments = >>>> "-Dtest.class.path.prefix=/src/jdk/build/linux-x86_64-server-fastdebug/test-support/jtreg_test_jdk_jdk_jfr_tool_TestPrintDefault_java/classes/jdk/jfr/tool/TestPrintDefault.d:/src/jdk/test/jdk/jdk/jfr/tool:/src/jdk/build/linux-x86_64-server-fastdebug/test-support/jtreg_test_jdk_jdk_jfr_tool_TestPrintDefault_java/classes/test/lib:/src/jdk/build/linux-x86_64-server-fastdebug/test-support/jtreg_test_jdk_jdk_jfr_tool_TestPrintDefault_java/classes/test/jdk >>>> -Dtest.src=/src/jdk/test/jdk/jdk/jfr/tool >>>> -Dtest.src.path=/src/jdk/test/jdk/jdk/jfr/tool:/src/jdk/test/lib:/src/jdk/test/jdk >>>> -Dtest.classes=/src/jdk/build/linux-x86_64-server-fastdebug/test-support/jtreg_test_jdk_jdk_jfr_tool_TestPrintDefault_java/classes/jdk/jfr/tool/TestPrintDefault.d >>>> -Dtest.class.path=/src/jdk/build/linux-x86_64-server-fastdebug/test-support/jtreg_test_jdk_jdk_jfr_tool_TestPrintDefault_java/classes/jdk/jfr/tool/TestPrintDefault.d:/src/jdk/build/linux-x86_64-server-fastdebug/test-support/jtreg_test_jdk_jdk_jfr_tool_TestPrintDefault_java/classes/test/lib:/src/jdk/build/linux-x86_64-server-fastdebug/test-support/jtreg_test_jdk_jdk_jfr_tool_TestPrintDefault_java/classes/test/jdk >>>> -Dtest.vm.opts=-Xmx512m -XX:MaxRAMPercentage=25 -ea -esa >>>> -Dtest.tool.vm.opts=-J-Xmx512m -J-XX:MaxRAMPercentage=25 -J-ea >>>> -J-esa -Dtest.compiler.opts= -Dtest.java.opts= >>>> -Dtest.jdk=/src/jdk/build/linux-x86_64-server-fastdebug/images/jdk >>>> -Dcompile.jdk=/src/jdk/build/linux-x86_64-server-fastdebug/images/jdk >>>> -Dtest.timeout.factor=4.0 -Dtest.modules=java.scripting jdk.jfr >>>> -Dtest.nativepath=/src/jdk/build/linux-x86_64-server-fastdebug/images/test/jdk/jtreg/native >>>> --add-modules=java.scripting,jdk.jfr -Xmx512m >>>> -XX:MaxRAMPercentage=25 -ea -esa >>>> -Djava.library.path=/src/jdk/build/linux-x86_64-server-fastdebug/images/test/jdk/jtreg/native" >>>> jvmFlags = N/A >>>> javaArguments = "com.sun.javatest.regtest.agent.MainWrapper >>>> /src/jdk/build/linux-x86_64-server-fastdebug/test-support/jtreg_test_jdk_jdk_jfr_tool_TestPrintDefault_java/jdk/jfr/tool/TestPrintDefault.d/main.0.jta" >>>> jvmStartTime = 17:39:51.394 >>>> pid = 23337 >>>> } >>>> >>>> jdk.OSInformation { >>>> startTime = 17:40:16.542 >>>> osVersion = "Oracle Linux Server release 7.5 >>>> uname:Linux localhost.localdomain 4.1.12-112.16.4.el7uek.x86_64 #2 >>>> SMP Mon Mar 12 23:57:12 PDT 2018 x86_64 >>>> libc:glibc 2.17 NPTL 2.17 >>>> " >>>> } >>>> >>>> webrev: >>>> http://cr.openjdk.java.net/~cito/JDK-8218631/webrev.00/ >>>> >>>> JBS: >>>> https://bugs.openjdk.java.net/browse/JDK-8218631 >>>> >>>> This change passed all tests in test-tier1. >>>> >>>> Regards, >>>> Chihiro >>>> > > > From chihiro.ito at oracle.com Mon Apr 29 17:34:58 2019 From: chihiro.ito at oracle.com (Chihiro Ito) Date: Tue, 30 Apr 2019 02:34:58 +0900 Subject: RFR: 8223066: "jfr metadata" output the @Name annotation twice. Message-ID: <398c166c-b681-6d32-4f4c-9d43350b622f@oracle.com> Hi, I fixed an issue that the @Name annotation was printed twice as followings in "jfr metadata" output. @Name("jdk.ActiveSetting") @Name("jdk.ActiveSetting") @Label("Recording Setting") @Category("Flight Recorder") class ActiveSetting extends jdk.jfr.Event { Could you reveiew this please? webrev: http://cr.openjdk.java.net/~cito/JDK-8223066/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8223066 I have passed all the tests in test-tier1. Best regards, Chihiro