From chris.plummer at oracle.com Fri Jun 1 00:23:54 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 31 May 2018 17:23:54 -0700 Subject: RFR: JDK-8187289: NotifyFramePop request is not cleared if JVMTI_EVENT_FRAME_POP is disabled In-Reply-To: References: <8d72b580-a98c-b459-74ef-df0fb729c7b2@oracle.com> <42222662-3c3e-3dc9-d01c-09d14c47ea1f@oracle.com> <89c3be67-7382-edf6-c2d5-721889866cc6@oracle.com> Message-ID: <9d39a219-e1b2-f584-fd32-849fecc9710e@oracle.com> Hi, I looked over the changes and also my comments from when I reviewed for Serguei, and everything looks fine. I assume testing has been done on solaris and there was no need for the -lc there. thanks, Chris On 5/29/18 1:59 PM, serguei.spitsyn at oracle.com wrote: > Hi Dan and Chris, > > You both already reviewed my version of the fix below. > For your convenience, my webrev was: > http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8187289-jvmti-framepop.2/ > > I did not push it into 10 because some number of regressions were > discovered. > Thanks a lot to Alex for isolating and finding how to fix those > regressions! > > The fix from Alex has a minor change since that time. > I think, the difference is in the jvmtiExport.cpp. > > I know you are/were busy or seek these days. > Just one more review is needed. > There is no big hurry, it can wait if necessary. > > Thanks, > Serguei > > > On 5/23/18 15:19, Alex Menkov wrote: >> Hi Serguei, >> >> Thank you for the review. >> >> The issue is described in CSR in details. >> Short summary: >> A NotifyFramePop request is only cleared if the JVMTI_EVENT_FRAME_POP >> is enabled. The problem is that if a NotifyFramePop request is not >> cleared when the JVMTI_EVENT_FRAME_POP events notification is >> disabled, then an unrelated FRAME_POP event is sent after the >> notification is re-enabled (test in the fix demonstrates the scenario). >> >> Changes in jvmtiExport.cpp updates updates methodExit handler to >> clear pending FramePop events even if JVMTI_EVENT_FRAME_POP is disabled. >> Changes in jvmtiEventController.cpp turns on interp_only_mode if >> NotifyFramePop is requested (this is also for the case when >> JVMTI_EVENT_FRAME_POP is disabled). >> >> --alex >> >> On 05/23/2018 13:58, serguei.spitsyn at oracle.com wrote: >>> Hi Alex, >>> >>> Looks good. >>> >>> Thank you a lot for taking care about this! >>> I know, it was not easy to sort out the regressions observed in >>> testing. >>> >>> Could you, please, explain the issue and fix to reviewers? >>> >>> Thanks, >>> Serguei >>> >>> >>> On 5/23/18 12:33, Alex Menkov wrote: >>>> Hi all, >>>> >>>> Please take a look at a fix for >>>> https://bugs.openjdk.java.net/browse/JDK-8187289 >>>> >>>> webrev: http://cr.openjdk.java.net/~amenkov/notifyFramePop/webrev/ >>>> >>>> CSR for the issue: https://bugs.openjdk.java.net/browse/JDK-8191467 >>>> >>>> --alex >>> > From david.holmes at oracle.com Fri Jun 1 01:16:09 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 1 Jun 2018 11:16:09 +1000 Subject: JDK-8069149: jdk.internal.pref.Perf is unaware of the VM option UsePerfData In-Reply-To: <5B10485C.2020606@oracle.com> References: <5B10485C.2020606@oracle.com> Message-ID: Hi Gary, Meta-question partially raised in the the recent bug report comments: do we care about this any more? Does anything need to disable UsePerfData? Can we not deprecate it in 11 then obsolete in 12? That would be a lot simpler. That aside ... It's hard to see how all the different paths in these related API's stitch together - especially when you have to consider the API being used on the current VM or a different one. I'm not at all clear what the control flow is, so it is hard to see exactly how the initialization issue is being handled. As I wrote in the initial report it is unclear exactly which parts of the "perf" subsystem are impacted by the value of UsePerfData. Can you summarize the role of the is_initialized check versus a direct UsePerfData check on the VM side i.e. which should be checked when? I immediately see an issue that Perf_Detach is a no-op when !UsePerfdata, yet Perf_Attach doesn't examine the flag at all! But perhaps we never reach those methods because of a higher-level call filtering it out ?? More below ... On 1/06/2018 5:09 AM, Gary Adams wrote: > A patch was done previously to prevent an error when -XX:-UsePerfData > is used, but this bug was filed to make the setting more visible in the > jdk.internal.perf package. > > ?? Webrev: http://cr.openjdk.java.net/~gadams/8069149/webrev.00/ > ?? Issue: https://bugs.openjdk.java.net/browse/JDK-8069149 > > I have tried a few initial tests using: > > ?? make run-tests \ > ???????? TEST_OPTS=VM_OPTIONS=-XX:-UsePerfData \ > ???????? TEST=open/test/jdk/sun/management/jmxremote > > While I'm tracking down one test timeout,? I'd like to get some > feedback on the approach used here : > > ?? - the basic mechanism propagates the "is initialized" state up to > Perf.java; Within PerfMemory UsePerfData and is_usable() are used for guarding execution. Perhaps that is what needs to be propagated up? > ????? does the state need to be exposed to users getPerf(), or is it > sufficient > ????? to provide exceptions at this level. Unclear what the control flow is here. I don't see that Perf triggers initialization of the perf subsystem, so what guarantees initialization should have happened by the time the Perf class is initialized ?? > ?? - an existing use of IOException was used in the case of attach > failures; > ????? added IOExceptions to the createXXX methods if the memory was not > initialized On the VM side can you at least add a message to the IOException so that the user can tell why it was thrown. > ?? - presuming a CSR would be needed to cover the IOExceptions It's not a public API so no CSR should be needed. > ?? - it appears that jdk.internal.perf has very limited usage, > ????? is it still needed/used (?) I can't answer that part, but I don't think UsePerfData is needed any more. Thanks, David > Any feedback appreciated. > From david.holmes at oracle.com Fri Jun 1 01:20:23 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 1 Jun 2018 11:20:23 +1000 Subject: JDK-8069149: jdk.internal.pref.Perf is unaware of the VM option UsePerfData In-Reply-To: <5B10485C.2020606@oracle.com> References: <5B10485C.2020606@oracle.com> Message-ID: <7b684864-e6eb-e154-9588-a87220074f60@oracle.com> PS. PerfCounter.java lb field should be left as final. You can work around definite-assignment problems by using a local variable and only assigning to the field after the catch block: private PerfCounter(String name, int type) { this.name = name; + LongBuffer temp = null; + try { ByteBuffer bb = perf.createLong(name, type, U_None, 0L); bb.order(ByteOrder.nativeOrder()); + temp = bb.asLongBuffer(); + } catch (IOException e) { + // may not be available + } + this.lb = temp; } David On 1/06/2018 5:09 AM, Gary Adams wrote: > A patch was done previously to prevent an error when -XX:-UsePerfData > is used, but this bug was filed to make the setting more visible in the > jdk.internal.perf package. > > ?? Webrev: http://cr.openjdk.java.net/~gadams/8069149/webrev.00/ > ?? Issue: https://bugs.openjdk.java.net/browse/JDK-8069149 > > I have tried a few initial tests using: > > ?? make run-tests \ > ???????? TEST_OPTS=VM_OPTIONS=-XX:-UsePerfData \ > ???????? TEST=open/test/jdk/sun/management/jmxremote > > While I'm tracking down one test timeout,? I'd like to get some > feedback on the approach used here : > > ?? - the basic mechanism propagates the "is initialized" state up to > Perf.java; > ????? does the state need to be exposed to users getPerf(), or is it > sufficient > ????? to provide exceptions at this level. > ?? - an existing use of IOException was used in the case of attach > failures; > ????? added IOExceptions to the createXXX methods if the memory was not > initialized > ?? - presuming a CSR would be needed to cover the IOExceptions > ?? - it appears that jdk.internal.perf has very limited usage, > ????? is it still needed/used (?) > > Any feedback appreciated. > From mandy.chung at oracle.com Fri Jun 1 03:36:35 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 31 May 2018 20:36:35 -0700 Subject: RFR: 8203357 Container Metrics In-Reply-To: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> Message-ID: <6b8f5228-03ea-9521-aa66-e74b5595722b@oracle.com> Hi Bob, On 5/30/18 12:45 PM, Bob Vandette wrote:> > RFE: Container Metrics > > https://bugs.openjdk.java.net/browse/JDK-8203357 > > WEBREV: > > http://cr.openjdk.java.net/~bobv/8203357/webrev.00 Looks fine in general. It's good to have this internal API ready for JFR and other library code to use. I skimmed through the new tests. It'd be good to add some comments to describe what it does (for example, set up a docker image etc). launcher.properties 154 \ -XshowSettings:system (Linux Only) show host system or container\n\ 155 \ configuration and continue\n\ A newline can be placed after -XshowSettings:system consistent with other suboptions. test/lib/jdk/test/lib/containers/docker/DockerTestUtils.java There are several long lines in the new test files such as: MetricsCpuTester.java MetricsMemoryTester.java MetricsTester.java It'd help future side-by-side review if they are wrapped. I think most of them are the construction of an exception. I see a pattern of a name after @test and that is not strictly needed. TestCgroupMetrics.java 25 * @test TestCgroupMetrics TestDockerCpuMetrics.java 34 * @test TestSystemMetrics TestDockerMemoryMetrics.java 30 * @test TestSystemMetrics TestSystemMetrics.java 25 * @test TestSystemMetrics This needs a CSR for the new -XshowSettings:system option. Mandy From david.holmes at oracle.com Fri Jun 1 03:39:52 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 1 Jun 2018 13:39:52 +1000 Subject: RFR (S): 8199882: compiler/uncommontrap/TestDeoptOOM.java failed w/ fatal error: ExceptionMark constructor expects no pending exceptions Message-ID: Bug: https://bugs.openjdk.java.net/browse/JDK-8199882 webrev: http://cr.openjdk.java.net/~dholmes/8199882/webrev/ The lazy loading (not initialization, despite the method names) of the AbstractOwnableSynchronizer class (needed for deadlock detection and stack dumps) has caused a number of problems due to the potential for loading exceptions (specifically OOME), which result in code that does not anticipate exceptions, encountering them. The bug discusses a number of possible solutions but the chosen one is to simply pre-load the AOS class along with the numerous other core classes that get pre-loaded. This does of course add a small increment to startup time, but it should be negligible in the overall scheme of things and not something to be unduly concerned about (Claes has commented in chat that he doesn't see it as an issue.) Testing: - mach5 tier1,2,3 - java/lang/management/* - java/util/concurrent/* - compiler/uncommontrap/TestDeoptOOM.java Thanks, David From sharath.ballal at oracle.com Fri Jun 1 06:01:35 2018 From: sharath.ballal at oracle.com (Sharath Ballal) Date: Thu, 31 May 2018 23:01:35 -0700 (PDT) Subject: RFR: JDK-8199729 - Usage Logger open sourcing In-Reply-To: <2ce31c7c-cb2d-4ed6-a86f-ff86087f87cb@default> References: <36b5fda6-85d8-450a-8344-0050642ee540@default> <8C274EA3A72EE413.2D584836-334B-4004-837C-E5CDF1911825@mail.outlook.com> <2ce31c7c-cb2d-4ed6-a86f-ff86087f87cb@default> Message-ID: Hello All, ? Based on internal discussions, we have decided to put this open sourcing activity on hold. ? Hence I would like to withdraw the review request at this time. Thanks for your time and feedback. Apologies for any inconvenience this might have caused. ? ? Thanks, Sharath ? ? From: Sharath Ballal Sent: Thursday, May 31, 2018 3:22 PM To: Bernd Eckenfels; serviceability-dev Subject: RE: RFR: JDK-8199729 - Usage Logger open sourcing ? Thanks for the comments Bernd.? I will work on them and send a webrev once done. ? Thanks, Sharath ? ? From: Bernd Eckenfels [mailto:ecki at zusammenkunft.net] Sent: Wednesday, May 30, 2018 10:54 PM To: serviceability-dev Subject: Re: RFR: JDK-8199729 - Usage Logger open sourcing ? Would it be possible to remove all the duplicate configuration code from the PostVMInitHook and just call run() { UsageLoggerClient.logJavaUsage(); }? This way all config code can be in one place. ? I also wonder if the ORCL_ prefix can and should be removed from all constants? ? Beside that I would return getAbsoluteFile() from the priveledged sections. ? Gru? Bernd ? -- https://Bernd.eckenfels.net ? On Wed, May 30, 2018 at 6:21 PM +0200, "Sharath Ballal" wrote: Hello, ? Requesting reviews for Usage Logger open sourcing (earlier known as Usage Tracker). ? ID: https://bugs.openjdk.java.net/browse/JDK-8199729 Webrev: http://cr.openjdk.java.net/~sballal/8199729/webrev.00/ ? Mach5 run has been successful for Usage Logger tests and hs-tier1, hs-tier2. ? Thanks, Sharath ? ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jini.george at oracle.com Fri Jun 1 08:13:39 2018 From: jini.george at oracle.com (Jini George) Date: Fri, 1 Jun 2018 13:43:39 +0530 Subject: RFR (round 4), JEP-318: Epsilon GC In-Reply-To: <094e1093-5a13-4853-aa34-d4e987a069b0@redhat.com> References: <094e1093-5a13-4853-aa34-d4e987a069b0@redhat.com> Message-ID: <0b5c827a-2763-98aa-06af-24df9028aed7@oracle.com> . Thank you very much, Aleksey, for making the SA changes. Some comments on those. ==> share/classes/sun/jvm/hotspot/oops/ObjectHeap.java 444 liveRegions.add(eh.space()); We would need to add an object of type 'Address' to the liveRegions list, instead of type VirtualSpace. Not doing so results in exceptions of the following form from the compare() method for various clhsdb commands like 'jhisto': Exception in thread "main" java.lang.ClassCastException: jdk.hotspot.agent/sun.jvm.hotspot.memory.VirtualSpace cannot be cast to jdk.hotspot.agent/sun.jvm.hotspot.debugger.Address ==> share/classes/sun/jvm/hotspot/oops/ObjectHeap.java 445 } else { 446 if (Assert.ASSERTS_ENABLED) { 447 Assert.that(false, "Expecting GenCollectedHeap, G1CollectedHeap, " + 448 "or ParallelScavengeHeap, but got " + 449 heap.getClass().getName()); 450 } 451 } * Please add EpsilonGC also to the assertion statement. ==> share/classes/sun/jvm/hotspot/tools/HeapSummary.java The run() method would need to handle Epsilon GC here to avoid the Unknown CollectedHeap type error with jhsdb jmap --heap. ==> share/classes/sun/jvm/hotspot/HSDB.java In showThreadStackMemory(), we have: 1101 } 1102 } else if (collHeap instanceof ParallelScavengeHeap) { 1103 ParallelScavengeHeap heap = (ParallelScavengeHeap) collHeap; 1104 if (heap.youngGen().isIn(handle)) { 1105 anno = "PSYoungGen "; 1106 bad = false; 1107 } else if (heap.oldGen().isIn(handle)) { 1108 anno = "PSOldGen "; 1109 bad = false; 1110 } 1111 } else { 1112 // Optimistically assume the oop isn't bad 1113 anno = "[Unknown generation] "; 1114 bad = false; 1115 } 1116 We would need to add the case of collHeap being an instanceof EpsilonHeap too. It would display "Unknown generation" while viewing the stack memory for the Java threads otherwise. ==> It would be great if test/hotspot/jtreg/serviceability/sa/TestUniverse.java is enhanced to add the minimalistic test for EpsilonGC. Thank you, Jini. On 5/31/2018 11:42 PM, Aleksey Shipilev wrote: > Hi, > > This is fourth (and hopefully final) round of code review for Epsilon GC changes. It includes the > fixes done as the result of third round of reviews, all of them inside gc/epsilon or epsilon tests. > > Webrev: > http://cr.openjdk.java.net/~shade/epsilon/webrev.08/ > > If we are good, I am going to push this with the following changeset metadata: > > 8204180: Implementation: JEP 318: Epsilon A No-Op Garbage Collector > Summary: Introduce Epsilon GC > Reviewed-by: rkennke, ihse, pliden, eosterlund, lmesnik > > Builds: > server X {x86_64, x86_32, aarch64, arm32, ppc64le, s390x} > minimal X {x86, x86_64} > zero X {x86_64} > > Testing: gc/epsilon on x86_64 > > Thanks, > -Aleksey > From christoph.langer at sap.com Fri Jun 1 10:12:40 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 1 Jun 2018 10:12:40 +0000 Subject: RFR(S): 8200720: Print additional information in thread dump (times, allocated bytes etc.) In-Reply-To: <082ca98f-9a68-15a0-95b5-c4d5c733ab36@oracle.com> References: <56817AB5-88FC-4FB9-A271-7206EA69ECA7@sap.com> <2728581E-0E32-4911-B1A9-79AD6CBA81B0@sap.com> <082ca98f-9a68-15a0-95b5-c4d5c733ab36@oracle.com> Message-ID: Hi Chris, I think I should chime in about your 3rd point regarding thread dump parameter parsing. For the SAP JVM we have extended the thread_dump operation a little bit and it has 3 params (3 is the limit for param numbers): arg(0) is support for the "-l" switch, as you currently find in OpenJDK arg(1) is a custom string that could be specified to print out a custom tag or headline in front of the thread dump arg(2) is a flag to toggle printing the thread dump to the target VM's tty (as opposed to the default behavior of printing the thread dump to the operation's output stream). So I can see that Gunter explicitly wanted to not break this behavior and added some extension just to arg(0). But I agree that it's probably not the nicest and straight forward thing to do. What if we parse each argument and check whether it contains "-l" or "-e" or even "-le". So we could add extensions like "-tty" for dump to tty and "-t" for tag or something like this? Then for instance arg(0) could look like "-l -e -tty"? Best regards Christoph > -----Original Message----- > From: serviceability-dev [mailto:serviceability-dev- > bounces at openjdk.java.net] On Behalf Of Chris Plummer > Sent: Freitag, 1. Juni 2018 00:04 > To: Haug, Gunter ; serviceability-dev > > Subject: Re: RFR(S): 8200720: Print additional information in thread dump > (times, allocated bytes etc.) > > Hi Gunter, > > globals.hpp: fix typo "informatiuon" > > I worry a little bit about the synchronizing (if that's the right word) > of PrintExtendedThreadInfo and the dcmd's -e flag. When using -e, you > are temporarily enabling PrintExtendedThreadInfo if it was false. This > temporarily changes the behavior of thread dumps, and could impact other > uses that happen in parallel. Also, could two simultaneous uses of -e > result in PrintExtendedThreadInfo not getting restored properly? > > thread_dump() doesn't look right. It looks like you are iterating char > by char over the argument, and expect something like "-el" to be > specified rather then "-e -l". The loop should be iterating over > op->arg(i), not op->arg(0)[i]. > > The rest of the changes look fine. > > thanks, > > Chris > > > On 5/30/18 8:12 AM, Haug, Gunter wrote: > > Hi all, > > > > As Chris proposed, I have made an the extended output switchable. There > is an VM flag (PrintExtendedThreadInfo), which is false by default. > Moreover, there is an Option (-e) which can be used with jcmd Thread.print > as well as with jstack. The -e option essentially sets PrintExtendedThreadInfo > true just for the respective thread dump. > > > > Here is the updated webrev: > > > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v2 > > > > (https://bugs.openjdk.java.net/browse/JDK-8200720) > > > > Thanks, > > Gunter > > > > > > ?On 02.05.18, 17:07, "serviceability-dev on behalf of Haug, Gunter" > gunter.haug at sap.com> wrote: > > > > Hi Chris, > > > > Thanks for looking into this. > > You're right, there is a little more we have. We have implemented an IO > tracing mechanism which - rather as a byproduct - keeps track of bytes read > and written per thread. However, this of course requires changes not only in > hotspot. We would be happy to contribute this as well, but this is a far bigger > change and will probably lead to a far bigger discussion. Anyway, with the > number of bytes read, the number of classes defined doesn't look that > arbitrary anymore, as one can correlate IO to class loading. > > > > Regarding the verbose option I think that's a good idea! > > > > Thanks again, > > Gunter > > > > On 01.05.18, 22:55, "Chris Plummer" > wrote: > > > > Hi Gunter, > > > > The output you are adding is all useful. I think the question is (and > > I'd like to see a few people chime in on this for this review) is > > whether or not all of it is the appropriate for a thread's stack dump. > > For example, defined_classes is on the fringe of what I would call > > generally useful info in a stack dump. Sure, there might be that rare > > case when you need it, but how often compared to other useful info > > maintained on a per thread basis. How many other bits of useful info > are > > not being printed in favor of defined_classes? It seems you have more > in > > the queue. How many? I'm worried about how cluttered the stack > dumps > > will get. Maybe we should add some sort of verbose thread dumping > > option. Just a thought. > > > > As for the implementation, overall it looks good, but I can't speak to > > whether or not you are doing proper accounting of defined_classes > and > > bytes allocated. You'll need input from someone with more knowledge > of > > those areas. We'll also need to do some testing to make sure tool tests > > are not impacted. > > > > thanks, > > > > Chris > > > > On 4/30/18 2:51 AM, Haug, Gunter wrote: > > > Hi, > > > > > > this is an update to an RFR I posted on hotspot-dev, but it is probably > more suitable to post it here. Can I please have a review and a sponsor for > the following enhancement: > > > > > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v1 > > > https://bugs.openjdk.java.net/browse/JDK-8200720 > > > > > > We at SAP have extended the thread dumps (obtained by jstack or > jcmd) by several fields providing thread specific information. These > extensions are quite popular with our support team. With some knowledge > of the architecture of the application, they often allow for quick and simple > diagnosis of a running system. Therefore we would like to contribute these > enhancements. > > > > > > I took a few simple examples here, namely cpu time, elapsed time > since thread creation, bytes allocated and classes defined by the thread and > the pthread-id or equivalent on platforms where it makes sense. Provided it > is known how the application should behave, a misbehaving thread can > identified easily. > > > > > > There is no measurable overhead for this enhancement. However, it > may be a problem that the format of the output is changed. Tools parsing the > output may have to be changed. > > > > > > Here is an example of the output generated: > > > > > > ------------------------------------------------------ > > > > > > "main" #1 prio=5 os_prio=31 cpu=6300.65ms elapsed=123.28s > allocated=242236760B defined_classes=1725 tid=0x00007fa13a806000 > nid=0x1c03 pthread-id=0x109708000 waiting on condition > [0x0000000109707000] > > > java.lang.Thread.State: TIMED_WAITING (sleeping) > > > JavaThread state: _thread_blocked > > > Thread: 0x00007fa13a806000 [0x1c03] State: _at_safepoint > _has_called_back 0 _at_poll_safepoint 0 > > > JavaThread state: _thread_blocked > > > at java.lang.Thread.sleep(java.base/Native Method) > > > ... > > > ------------------------------------------------------ > > > > > > As mentioned above, we have a whole bunch of other > enhancements to the thread dump similar to this one and would be willing to > contribute them if there is any interest. > > > > > > Thanks and best regards, > > > Gunter > > > > > > > > > > > > > > > > > From gunter.haug at sap.com Fri Jun 1 10:17:44 2018 From: gunter.haug at sap.com (Haug, Gunter) Date: Fri, 1 Jun 2018 10:17:44 +0000 Subject: RFR(S): 8200720: Print additional information in thread dump (times, allocated bytes etc.) In-Reply-To: <082ca98f-9a68-15a0-95b5-c4d5c733ab36@oracle.com> References: <56817AB5-88FC-4FB9-A271-7206EA69ECA7@sap.com> <2728581E-0E32-4911-B1A9-79AD6CBA81B0@sap.com> <082ca98f-9a68-15a0-95b5-c4d5c733ab36@oracle.com> Message-ID: Hi Chris, thanks for looking into this! Re the synchronization: The value is stored only in a VM operation at a safepoint by the VM thread. I was of the opinion, that this could not be interrupted by a second VM operation (of the same type). Or am I missing something else? I did think about passing an argument to the various print_on member functions of the thread classes, but this would require rather extensive code changes for a rather tiny extension. Therefore I feel doing it like this is the lesser evil. Re thread_dump(): I think it's correct or, well, at least it works ;-) In fact jstack will transfer the "-e" and "-l" in only one string, i.e. op->arg(0). Christoph has already explained my reasoning. But I agree, it's not nice and I would be happy to do it like Christoph suggested. And typo fixed, sorry. Thanks again, Gunter ?On 01.06.18, 00:03, "Chris Plummer" wrote: Hi Gunter, globals.hpp: fix typo "informatiuon" I worry a little bit about the synchronizing (if that's the right word) of PrintExtendedThreadInfo and the dcmd's -e flag. When using -e, you are temporarily enabling PrintExtendedThreadInfo if it was false. This temporarily changes the behavior of thread dumps, and could impact other uses that happen in parallel. Also, could two simultaneous uses of -e result in PrintExtendedThreadInfo not getting restored properly? thread_dump() doesn't look right. It looks like you are iterating char by char over the argument, and expect something like "-el" to be specified rather then "-e -l". The loop should be iterating over op->arg(i), not op->arg(0)[i]. The rest of the changes look fine. thanks, Chris On 5/30/18 8:12 AM, Haug, Gunter wrote: > Hi all, > > As Chris proposed, I have made an the extended output switchable. There is an VM flag (PrintExtendedThreadInfo), which is false by default. Moreover, there is an Option (-e) which can be used with jcmd Thread.print as well as with jstack. The -e option essentially sets PrintExtendedThreadInfo true just for the respective thread dump. > > Here is the updated webrev: > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v2 > > (https://bugs.openjdk.java.net/browse/JDK-8200720) > > Thanks, > Gunter > > > On 02.05.18, 17:07, "serviceability-dev on behalf of Haug, Gunter" wrote: > > Hi Chris, > > Thanks for looking into this. > You're right, there is a little more we have. We have implemented an IO tracing mechanism which - rather as a byproduct - keeps track of bytes read and written per thread. However, this of course requires changes not only in hotspot. We would be happy to contribute this as well, but this is a far bigger change and will probably lead to a far bigger discussion. Anyway, with the number of bytes read, the number of classes defined doesn't look that arbitrary anymore, as one can correlate IO to class loading. > > Regarding the verbose option I think that's a good idea! > > Thanks again, > Gunter > > On 01.05.18, 22:55, "Chris Plummer" wrote: > > Hi Gunter, > > The output you are adding is all useful. I think the question is (and > I'd like to see a few people chime in on this for this review) is > whether or not all of it is the appropriate for a thread's stack dump. > For example, defined_classes is on the fringe of what I would call > generally useful info in a stack dump. Sure, there might be that rare > case when you need it, but how often compared to other useful info > maintained on a per thread basis. How many other bits of useful info are > not being printed in favor of defined_classes? It seems you have more in > the queue. How many? I'm worried about how cluttered the stack dumps > will get. Maybe we should add some sort of verbose thread dumping > option. Just a thought. > > As for the implementation, overall it looks good, but I can't speak to > whether or not you are doing proper accounting of defined_classes and > bytes allocated. You'll need input from someone with more knowledge of > those areas. We'll also need to do some testing to make sure tool tests > are not impacted. > > thanks, > > Chris > > On 4/30/18 2:51 AM, Haug, Gunter wrote: > > Hi, > > > > this is an update to an RFR I posted on hotspot-dev, but it is probably more suitable to post it here. Can I please have a review and a sponsor for the following enhancement: > > > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v1 > > https://bugs.openjdk.java.net/browse/JDK-8200720 > > > > We at SAP have extended the thread dumps (obtained by jstack or jcmd) by several fields providing thread specific information. These extensions are quite popular with our support team. With some knowledge of the architecture of the application, they often allow for quick and simple diagnosis of a running system. Therefore we would like to contribute these enhancements. > > > > I took a few simple examples here, namely cpu time, elapsed time since thread creation, bytes allocated and classes defined by the thread and the pthread-id or equivalent on platforms where it makes sense. Provided it is known how the application should behave, a misbehaving thread can identified easily. > > > > There is no measurable overhead for this enhancement. However, it may be a problem that the format of the output is changed. Tools parsing the output may have to be changed. > > > > Here is an example of the output generated: > > > > ------------------------------------------------------ > > > > "main" #1 prio=5 os_prio=31 cpu=6300.65ms elapsed=123.28s allocated=242236760B defined_classes=1725 tid=0x00007fa13a806000 nid=0x1c03 pthread-id=0x109708000 waiting on condition [0x0000000109707000] > > java.lang.Thread.State: TIMED_WAITING (sleeping) > > JavaThread state: _thread_blocked > > Thread: 0x00007fa13a806000 [0x1c03] State: _at_safepoint _has_called_back 0 _at_poll_safepoint 0 > > JavaThread state: _thread_blocked > > at java.lang.Thread.sleep(java.base/Native Method) > > ... > > ------------------------------------------------------ > > > > As mentioned above, we have a whole bunch of other enhancements to the thread dump similar to this one and would be willing to contribute them if there is any interest. > > > > Thanks and best regards, > > Gunter > > > > > > > > > From gary.adams at oracle.com Fri Jun 1 13:06:13 2018 From: gary.adams at oracle.com (Gary Adams) Date: Fri, 01 Jun 2018 09:06:13 -0400 Subject: JDK-8069149: jdk.internal.pref.Perf is unaware of the VM option UsePerfData In-Reply-To: <5B1131E4.70407@oracle.com> References: <5B10485C.2020606@oracle.com> <5B1131E4.70407@oracle.com> Message-ID: <5B1144C5.4010706@oracle.com> Comments inline ... On 6/1/18, 7:45 AM, Gary Adams wrote: > PS. > > PerfCounter.java > > lb field should be left as final. You can work around > definite-assignment problems by using a local variable and only > assigning to the field after the catch block: > > private PerfCounter(String name, int type) { > this.name = name; > + LongBuffer temp = null; > + try { > ByteBuffer bb = perf.createLong(name, type, U_None, 0L); > bb.order(ByteOrder.nativeOrder()); > + temp = bb.asLongBuffer(); > + } catch (IOException e) { > + // may not be available > + } > + this.lb = temp; > } Thanks for the tip. The PerfCounter changes were the last thing I dropped in after adding the IOException and fixing the places where things did not compile. Fundamentally, what sort of things will break, when the previous interface did not expose the possibility that the perf memory was not initialized as assumed in the earlier code. Does the upper level code "have to" be aware, or is the contract to do a "best effort" if something lower down is not as expected. > Hi Gary, > > Meta-question partially raised in the the recent bug report comments: do > we care about this any more? Does anything need to disable UsePerfData? > Can we not deprecate it in 11 then obsolete in 12? That would be a lot > simpler. My own bias is that many of the motivations for faster startup and smaller footprint are still useful goals. If not for embedded systems, then for container environments. A process started with -XX:-UsePerfData does not support external attach and does not maintain the internal variables for the self attach. > > That aside ... > > It's hard to see how all the different paths in these related API's > stitch together - especially when you have to consider the API being > used on the current VM or a different one. I'm not at all clear what the > control flow is, so it is hard to see exactly how the initialization > issue is being handled. As I wrote in the initial report it is unclear > exactly which parts of the "perf" subsystem are impacted by the value of > UsePerfData. Starting with the Perf.java class initialization, I added the exposure of the PerfMemory::is_initialized(). Perf.java: 559 private native boolean isInitialized(); 560 561 private static native void registerNatives(); 562 563 static { 564 registerNatives(); 565 instance = new Perf(); 566 initialized = instance.isInitialized(); 567 } perf.cpp: 298 PERF_ENTRY(jboolean, Perf_IsInitialized(JNIEnv *env, jobject perf)) 299 300 PerfWrapper("Perf_IsInitialized"); 301 302 return PerfMemory::is_initialized(); 303 304 PERF_END From the VM perspective UsePerfData is both a command line argument and a flag to be used whenever that information is needed. There are scattered checks to deal with setup and shutdown, but also the runtime fulfillment of the PerfMemory data structures. Looking at perfMemory_init() and perfMemory_exit(), I concluded that the UsePerfData flag essentially noops all of the perfMemory allocation and updating. The only information required in the upper layers of the Perf.java code is the fact that the perf memory is not initialized. perfMemory.cpp: 58 void perfMemory_init() { 59 60 if (!UsePerfData) return; 61 62 PerfMemory::initialize(); 63 } 64 65 void perfMemory_exit() { 66 67 if (!UsePerfData) return; 68 if (!PerfMemory::is_usable()) return; 69 ... 90 void PerfMemory::initialize() { ... 159 OrderAccess::release_store(&_initialized, 1); 160 } ... 271 bool PerfMemory::is_initialized() { 272 return OrderAccess::load_acquire(&_initialized) != 0; 273 } > > Can you summarize the role of the is_initialized check versus a direct > UsePerfData check on the VM side i.e. which should be checked when? I > immediately see an issue that Perf_Detach is a no-op when !UsePerfdata, > yet Perf_Attach doesn't examine the flag at all! But perhaps we never > reach those methods because of a higher-level call filtering it out ?? There is split path when attaching to self versus attaching to other running VM processes. The extra detach processing is only needed for the remote VM case. If attach fails, there is no need to detach. Perf.java: ... 274 private ByteBuffer attachImpl(String user, int lvmid, int mode) 275 throws IllegalArgumentException, IOException 276 { 277 final ByteBuffer b = attach(user, lvmid, mode); 278 279 if (lvmid == 0) { 280 // The native instrumentation buffer for this Java virtual 281 // machine is never unmapped. 282 return b; 283 } 284 else { 285 // This is an instrumentation buffer for another Java virtual 286 // machine with native resources that need to be managed. We 287 // create a duplicate of the native ByteBuffer and manage it 288 // with a Cleaner. When the duplicate becomes phantom reachable, 289 // the native resources will be released. 290 291 final ByteBuffer dup = b.duplicate(); 292 293 CleanerFactory.cleaner() 294 .register(dup, new CleanerAction(instance, b)); 295 return dup; 296 } 297 } > > More below ... > > David > > On 1/06/2018 5:09 AM, Gary Adams wrote: > >/ A patch was done previously to prevent an error when -XX:-UsePerfData > />/ is used, but this bug was filed to make the setting more visible in the > />/ jdk.internal.perf package. > />/ > />/ Webrev:http://cr.openjdk.java.net/~gadams/8069149/webrev.00/ > />/ Issue:https://bugs.openjdk.java.net/browse/JDK-8069149 > />/ > />/ I have tried a few initial tests using: > />/ > />/ make run-tests \ > />/ TEST_OPTS=VM_OPTIONS=-XX:-UsePerfData \ > />/ TEST=open/test/jdk/sun/management/jmxremote > />/ > />/ While I'm tracking down one test timeout, I'd like to get some > />/ feedback on the approach used here : > />/ > />/ - the basic mechanism propagates the "is initialized" state up to > />/ Perf.java; > />/ does the state need to be exposed to users getPerf(), or is it > />/ sufficient/ > Within PerfMemory UsePerfData and is_usable() are used for guarding > execution. Perhaps that is what needs to be propagated up? /The is_usable() state is used to ensure the memory is initialized and not yet detached. We don't need that specific condition here. > /Unclear what the control flow is here. I don't see that Perf triggers > initialization of the perf subsystem, so what guarantees initialization > should have happened by the time the Perf class is initialized ?? The private constructor and the static initializer in Perf ensures the singleton behavior at class initialization. The perfMemory_init() is handled during VM initialization. init.cpp: 90 void vm_init_globals() { 91 check_ThreadShadow(); 92 basic_types_init(); 93 eventlog_init(); 94 mutex_init(); 95 chunkpool_init(); 96 perfMemory_init(); 97 SuspendibleThreadSet_init(); 98 } > >/ - an existing use of IOException was used in the case of attach /> >/ failures; /> >/ added IOExceptions to the createXXX methods if the memory was not /> >/ initialized / > On the VM side can you at least add a message to the IOException so that > the user can tell why it was thrown. Initially, I included a message with the IOException, and noticed the other exceptions being thrown in the surrounding code did not include a message, so opted for the empty message. There is no ambiguity here. > >/ - presuming a CSR would be needed to cover the IOExceptions / > It's not a public API so no CSR should be needed. Good to know. > >/ - it appears that jdk.internal.perf has very limited usage, /> >/ is it still needed/used (?) / > I can't answer that part, but I don't think UsePerfData is needed any more. If this turns into "removal of UsePerfData" support, we should move the bug to the runtime group. I grabbed the bug initially, because it looked like a loose end and a simple thing to address by fulfilling the original request to propagate the visibility up to the Perf.java code. Let's see if other reviewers have similar views - that the UsePerfData flag can be removed. This bug was originally filed against 8u60 and is currently targetted for 12. I only explored a potential fix at this time in case we need to get a deprecation process started at this time. ... -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob.vandette at oracle.com Fri Jun 1 15:52:54 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Fri, 1 Jun 2018 11:52:54 -0400 Subject: RFR: 8203357 Container Metrics In-Reply-To: <6b8f5228-03ea-9521-aa66-e74b5595722b@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <6b8f5228-03ea-9521-aa66-e74b5595722b@oracle.com> Message-ID: <57ADBBD0-381B-4890-9943-BF3222F19839@oracle.com> > On May 31, 2018, at 11:36 PM, mandy chung wrote: > > Hi Bob, > > On 5/30/18 12:45 PM, Bob Vandette wrote:> >> RFE: Container Metrics >> https://bugs.openjdk.java.net/browse/JDK-8203357 >> WEBREV: >> http://cr.openjdk.java.net/~bobv/8203357/webrev.00 Thanks for the review, here an updated webrev: http://cr.openjdk.java.net/~bobv/8203357/webrev.01/ > > Looks fine in general. It's good to have this internal API ready > for JFR and other library code to use. > > I skimmed through the new tests. It'd be good to add some comments > to describe what it does (for example, set up a docker image etc). DockerTestUtils.java does contain some comments describing what the utility functions do. I?ll add a brief comment in TestDockerCpuMetrics.java and TestDockerMemoryMetrics.java explaining the test process. > > launcher.properties > 154 \ -XshowSettings:system (Linux Only) show host system or container\n\ > 155 \ configuration and continue\n\ > > A newline can be placed after -XshowSettings:system consistent with > other suboptions. Done. I also added a newline after the vm sub-option. > > test/lib/jdk/test/lib/containers/docker/DockerTestUtils.java > > There are several long lines in the new test files such as: > MetricsCpuTester.java > MetricsMemoryTester.java > MetricsTester.java > > It'd help future side-by-side review if they are wrapped. I think > most of them are the construction of an exception. Fixed. > > I see a pattern of a name after @test and that is not strictly needed. > > TestCgroupMetrics.java > 25 * @test TestCgroupMetrics > > TestDockerCpuMetrics.java > 34 * @test TestSystemMetrics > > TestDockerMemoryMetrics.java > 30 * @test TestSystemMetrics > > TestSystemMetrics.java > 25 * @test TestSystemMetrics Remove the names after @test. > > This needs a CSR for the new -XshowSettings:system option. I filed a CSR for this a few days ago. https://bugs.openjdk.java.net/browse/JDK-8204107 Bob. > > Mandy From mandy.chung at oracle.com Fri Jun 1 20:00:22 2018 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 1 Jun 2018 13:00:22 -0700 Subject: RFR: 8203357 Container Metrics In-Reply-To: <57ADBBD0-381B-4890-9943-BF3222F19839@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <6b8f5228-03ea-9521-aa66-e74b5595722b@oracle.com> <57ADBBD0-381B-4890-9943-BF3222F19839@oracle.com> Message-ID: <548859ab-db3a-d028-0c92-d6598c722052@oracle.com> On 6/1/18 8:52 AM, Bob Vandette wrote: > I filed a CSR for this a few days ago. > > https://bugs.openjdk.java.net/browse/JDK-8204107 Typo: s/-XshowSetting/-XshowSettings In the specification section, you can include the new lines adding to java --extra-help output (that's the spec) and drop the diff. It may worth to state that the output is implementation detail. It may help to clarify the behavior when running on non-linux platform, i.e. the current launcher implementation accepts any suboption value. So java -XshowSettings:system on non-linux platform displays vm, properties, locale settings. (BTW, I file JDK-8204246 about invalid suboption). Mandy From chris.plummer at oracle.com Fri Jun 1 22:54:26 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Fri, 1 Jun 2018 15:54:26 -0700 Subject: RFR: JDK-8169718: nsk/jdb/locals/locals002: ERROR: Cannot find boolVar with expected value: false In-Reply-To: <5d58fa2b-7dda-db64-0280-1e19e791a86d@oracle.com> References: <5B082D2E.7000408@oracle.com> <5d58fa2b-7dda-db64-0280-1e19e791a86d@oracle.com> Message-ID: Hi Gary, I'm a bit unclear on one point. According to your description, should we be seeing two prompts for commands like "cont", and "step"? When you say we are expecting: ? "Breakpoint hit:" ?? Isn't it really: ?? ? "Breakpoint hit:" ?? I figure this must be the case, because your fix removes the printing of a prompt, and we still need one prompt, so there must have been two to start with. I think in the above the last is being produced sequentially right after the "Breakpoint hit:" output, so is never an issue. The first comes from the printPrompt() call after issuing the "cont" command, and it's possible to hit the breakpoint and start producing more output before printPrompt() has completed, thus causing the problem you are seeing. There seems to be some purpose to printing this first prompt and I'm not so sure eliminating it is the right thing to do. I think it might be there to indicate that the command has been initiated, because there could be some delay before the breakpoint is hit. So the issue then is how to make sure this prompt appears in the input stream before any of the "Step completed:" or "Breakpoint hit:" output. Possibly event processing should be blocked until executeCommand() completes. thanks, Chris On 5/26/18 3:50 AM, gary.adams at oracle.com wrote: > This is a review request for a previously closed test bug. > The test was recently moved to the open repos, and the > proposed fix is in the open code. > > ? Webrev: http://cr.openjdk.java.net/~gadams/8169718/webrev/ > > > -------- Forwarded Message -------- > Subject: RFR: JDK-8169718: nsk/jdb/locals/locals002: ERROR: Cannot > find boolVar with expected value: false > Date: Fri, 25 May 2018 11:35:10 -0400 > From: Gary Adams > Reply-To: gary.adams at oracle.com > > > > > > The jdb tests use stdin to send commands to a jdb process > and parses the stdout to determine if a command was > successful and when the process is prompting for new commands > to be sent. > > Some commands are synchronous, so when the command is completed > a new prompt is sent back immediately. > > Some commands are asynchronous, so there could be a delay > until a breakpoint is reached. The event handler then sends a prompt > when the application thread is stopped and new jdb commands can be sent. > > The problem causing the intermittent failures was a corruption in the > output stream when prompts were being sent at the wrong times. > > Instead of receiving > "Breakpoint hit:" > > > the log contained > "Breakpoint hit:" > > Once out of sync, jdb commands were being sent prematurely > and the wrong values were being compared against expected behavior. > The simple fix proposed here recognizes that commands like "cont", > "step" and "next" are asynchronous commands and should not send back > a prompt immediately. Instead. the event handler will deliver the next prompt > when the next "Breakpoint hit:" or "Step completed:" state change occurs. > > The bulk of the testing was done on windows-x64-debug builds where the > intermittent failures were observed in ~5 in 1000 testruns. The fix has > also been tested on linux-x64-debug, solaris-sparcv9-debug, > and macosx-x64-debug, even though the failures have never been reported > against those platforms. > > Failures have been observed in many of the nsk/jdb tests with similar corrupted > output streams, but never directly associated with this issue before. > > ?redefine001, caught_exception002, locals002, eval001, next001, > stop_at003, step002, print002, trace001, step_up001, read001, > clear004, kill001, set001 > > From chris.plummer at oracle.com Fri Jun 1 23:12:27 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Fri, 1 Jun 2018 16:12:27 -0700 Subject: RFR(S): 8200720: Print additional information in thread dump (times, allocated bytes etc.) In-Reply-To: References: <56817AB5-88FC-4FB9-A271-7206EA69ECA7@sap.com> <2728581E-0E32-4911-B1A9-79AD6CBA81B0@sap.com> <082ca98f-9a68-15a0-95b5-c4d5c733ab36@oracle.com> Message-ID: <933cdacc-9b6f-c8b5-e244-558f38e7888c@oracle.com> Hi Gunter, On 6/1/18 3:17 AM, Haug, Gunter wrote: > Hi Chris, > > thanks for looking into this! > > Re the synchronization: The value is stored only in a VM operation at a safepoint by the VM thread. I was of the opinion, that this could not be interrupted by a second VM operation (of the same type). Or am I missing something else? I was really thinking more about the temporary changing of PrintExtendedThreadInfo, not the value stored in the VMOp. You may be be correct that no more than one VMOp is executing, but while it is executing it is has changed the value of PrintExtendedThreadInfo, which might have an impact on anything else that triggers printing thread info while the VMOp is executing. > I did think about passing an argument to the various print_on member functions of the thread classes, but this would require rather extensive code changes for a rather tiny extension. Therefore I feel doing it like this is the lesser evil. > > Re thread_dump(): I think it's correct or, well, at least it works ;-) In fact jstack will transfer the "-e" and "-l" in only one string, i.e. op->arg(0). So you are saying that op->arg(0) is "-e -l" when using jstack? I think you really need to clean up the parsing. As it stands right now passing, I get the feeling that if a user erroneously asks for help by using "jcmd Thread.Print -help", it will end up executing with -e an -l enabled, and no failure for the invalid "-help" option. > Christoph has already explained my reasoning. But I agree, it's not nice and I would be happy to do it like Christoph suggested. I'll respond separately to his suggestion. thanks, Chris > > And typo fixed, sorry. > > Thanks again, > Gunter > > ?On 01.06.18, 00:03, "Chris Plummer" wrote: > > Hi Gunter, > > globals.hpp: fix typo "informatiuon" > > I worry a little bit about the synchronizing (if that's the right word) > of PrintExtendedThreadInfo and the dcmd's -e flag. When using -e, you > are temporarily enabling PrintExtendedThreadInfo if it was false. This > temporarily changes the behavior of thread dumps, and could impact other > uses that happen in parallel. Also, could two simultaneous uses of -e > result in PrintExtendedThreadInfo not getting restored properly? > > thread_dump() doesn't look right. It looks like you are iterating char > by char over the argument, and expect something like "-el" to be > specified rather then "-e -l". The loop should be iterating over > op->arg(i), not op->arg(0)[i]. > > The rest of the changes look fine. > > thanks, > > Chris > > > On 5/30/18 8:12 AM, Haug, Gunter wrote: > > Hi all, > > > > As Chris proposed, I have made an the extended output switchable. There is an VM flag (PrintExtendedThreadInfo), which is false by default. Moreover, there is an Option (-e) which can be used with jcmd Thread.print as well as with jstack. The -e option essentially sets PrintExtendedThreadInfo true just for the respective thread dump. > > > > Here is the updated webrev: > > > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v2 > > > > (https://bugs.openjdk.java.net/browse/JDK-8200720) > > > > Thanks, > > Gunter > > > > > > On 02.05.18, 17:07, "serviceability-dev on behalf of Haug, Gunter" wrote: > > > > Hi Chris, > > > > Thanks for looking into this. > > You're right, there is a little more we have. We have implemented an IO tracing mechanism which - rather as a byproduct - keeps track of bytes read and written per thread. However, this of course requires changes not only in hotspot. We would be happy to contribute this as well, but this is a far bigger change and will probably lead to a far bigger discussion. Anyway, with the number of bytes read, the number of classes defined doesn't look that arbitrary anymore, as one can correlate IO to class loading. > > > > Regarding the verbose option I think that's a good idea! > > > > Thanks again, > > Gunter > > > > On 01.05.18, 22:55, "Chris Plummer" wrote: > > > > Hi Gunter, > > > > The output you are adding is all useful. I think the question is (and > > I'd like to see a few people chime in on this for this review) is > > whether or not all of it is the appropriate for a thread's stack dump. > > For example, defined_classes is on the fringe of what I would call > > generally useful info in a stack dump. Sure, there might be that rare > > case when you need it, but how often compared to other useful info > > maintained on a per thread basis. How many other bits of useful info are > > not being printed in favor of defined_classes? It seems you have more in > > the queue. How many? I'm worried about how cluttered the stack dumps > > will get. Maybe we should add some sort of verbose thread dumping > > option. Just a thought. > > > > As for the implementation, overall it looks good, but I can't speak to > > whether or not you are doing proper accounting of defined_classes and > > bytes allocated. You'll need input from someone with more knowledge of > > those areas. We'll also need to do some testing to make sure tool tests > > are not impacted. > > > > thanks, > > > > Chris > > > > On 4/30/18 2:51 AM, Haug, Gunter wrote: > > > Hi, > > > > > > this is an update to an RFR I posted on hotspot-dev, but it is probably more suitable to post it here. Can I please have a review and a sponsor for the following enhancement: > > > > > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v1 > > > https://bugs.openjdk.java.net/browse/JDK-8200720 > > > > > > We at SAP have extended the thread dumps (obtained by jstack or jcmd) by several fields providing thread specific information. These extensions are quite popular with our support team. With some knowledge of the architecture of the application, they often allow for quick and simple diagnosis of a running system. Therefore we would like to contribute these enhancements. > > > > > > I took a few simple examples here, namely cpu time, elapsed time since thread creation, bytes allocated and classes defined by the thread and the pthread-id or equivalent on platforms where it makes sense. Provided it is known how the application should behave, a misbehaving thread can identified easily. > > > > > > There is no measurable overhead for this enhancement. However, it may be a problem that the format of the output is changed. Tools parsing the output may have to be changed. > > > > > > Here is an example of the output generated: > > > > > > ------------------------------------------------------ > > > > > > "main" #1 prio=5 os_prio=31 cpu=6300.65ms elapsed=123.28s allocated=242236760B defined_classes=1725 tid=0x00007fa13a806000 nid=0x1c03 pthread-id=0x109708000 waiting on condition [0x0000000109707000] > > > java.lang.Thread.State: TIMED_WAITING (sleeping) > > > JavaThread state: _thread_blocked > > > Thread: 0x00007fa13a806000 [0x1c03] State: _at_safepoint _has_called_back 0 _at_poll_safepoint 0 > > > JavaThread state: _thread_blocked > > > at java.lang.Thread.sleep(java.base/Native Method) > > > ... > > > ------------------------------------------------------ > > > > > > As mentioned above, we have a whole bunch of other enhancements to the thread dump similar to this one and would be willing to contribute them if there is any interest. > > > > > > Thanks and best regards, > > > Gunter > > > > > > > > > > > > > > > > > > > > From chris.plummer at oracle.com Fri Jun 1 23:26:20 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Fri, 1 Jun 2018 16:26:20 -0700 Subject: RFR(S): 8200720: Print additional information in thread dump (times, allocated bytes etc.) In-Reply-To: References: <56817AB5-88FC-4FB9-A271-7206EA69ECA7@sap.com> <2728581E-0E32-4911-B1A9-79AD6CBA81B0@sap.com> <082ca98f-9a68-15a0-95b5-c4d5c733ab36@oracle.com> Message-ID: <7a47b63d-280e-6ce1-0f1b-67163b33d75e@oracle.com> Hi Christoph, I'm just starting to realize something here. We have both the Thread.print dcmd and the threaddump attach command. They both use the same underlying VM_PrintThreads op, but both process arguments independently. I've been thinking I was looking at dcmd options parsing, but it is attach command options parsing that you are changing. So my previous comment about a possible errant "-help" option is not realistic since jstack would never send that, and when used from a jcmd the proper options parsing is done and the jcmd would properly fail (although I'm beginning to learn that it gives no useful info as to why the command failed). But that being said, it still should be cleaned up. I think basically you need to tokenize arg(0) and look for "-l" and "-e" and reject anything else that is not currently supported. thanks, Chris On 6/1/18 3:12 AM, Langer, Christoph wrote: > Hi Chris, > > I think I should chime in about your 3rd point regarding thread dump parameter parsing. > > For the SAP JVM we have extended the thread_dump operation a little bit and it has 3 params (3 is the limit for param numbers): > > arg(0) is support for the "-l" switch, as you currently find in OpenJDK > arg(1) is a custom string that could be specified to print out a custom tag or headline in front of the thread dump > arg(2) is a flag to toggle printing the thread dump to the target VM's tty (as opposed to the default behavior of printing the thread dump to the operation's output stream). > > So I can see that Gunter explicitly wanted to not break this behavior and added some extension just to arg(0). But I agree that it's probably not the nicest and straight forward thing to do. > > What if we parse each argument and check whether it contains "-l" or "-e" or even "-le". So we could add extensions like "-tty" for dump to tty and "-t" for tag or something like this? Then for instance arg(0) could look like "-l -e -tty"? > > Best regards > Christoph > >> -----Original Message----- >> From: serviceability-dev [mailto:serviceability-dev- >> bounces at openjdk.java.net] On Behalf Of Chris Plummer >> Sent: Freitag, 1. Juni 2018 00:04 >> To: Haug, Gunter ; serviceability-dev >> >> Subject: Re: RFR(S): 8200720: Print additional information in thread dump >> (times, allocated bytes etc.) >> >> Hi Gunter, >> >> globals.hpp: fix typo "informatiuon" >> >> I worry a little bit about the synchronizing (if that's the right word) >> of PrintExtendedThreadInfo and the dcmd's -e flag. When using -e, you >> are temporarily enabling PrintExtendedThreadInfo if it was false. This >> temporarily changes the behavior of thread dumps, and could impact other >> uses that happen in parallel. Also, could two simultaneous uses of -e >> result in PrintExtendedThreadInfo not getting restored properly? >> >> thread_dump() doesn't look right. It looks like you are iterating char >> by char over the argument, and expect something like "-el" to be >> specified rather then "-e -l". The loop should be iterating over >> op->arg(i), not op->arg(0)[i]. >> >> The rest of the changes look fine. >> >> thanks, >> >> Chris >> >> >> On 5/30/18 8:12 AM, Haug, Gunter wrote: >>> Hi all, >>> >>> As Chris proposed, I have made an the extended output switchable. There >> is an VM flag (PrintExtendedThreadInfo), which is false by default. >> Moreover, there is an Option (-e) which can be used with jcmd Thread.print >> as well as with jstack. The -e option essentially sets PrintExtendedThreadInfo >> true just for the respective thread dump. >>> Here is the updated webrev: >>> >>> http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v2 >>> >>> (https://bugs.openjdk.java.net/browse/JDK-8200720) >>> >>> Thanks, >>> Gunter >>> >>> >>> ?On 02.05.18, 17:07, "serviceability-dev on behalf of Haug, Gunter" >> > gunter.haug at sap.com> wrote: >>> Hi Chris, >>> >>> Thanks for looking into this. >>> You're right, there is a little more we have. We have implemented an IO >> tracing mechanism which - rather as a byproduct - keeps track of bytes read >> and written per thread. However, this of course requires changes not only in >> hotspot. We would be happy to contribute this as well, but this is a far bigger >> change and will probably lead to a far bigger discussion. Anyway, with the >> number of bytes read, the number of classes defined doesn't look that >> arbitrary anymore, as one can correlate IO to class loading. >>> Regarding the verbose option I think that's a good idea! >>> >>> Thanks again, >>> Gunter >>> >>> On 01.05.18, 22:55, "Chris Plummer" >> wrote: >>> Hi Gunter, >>> >>> The output you are adding is all useful. I think the question is (and >>> I'd like to see a few people chime in on this for this review) is >>> whether or not all of it is the appropriate for a thread's stack dump. >>> For example, defined_classes is on the fringe of what I would call >>> generally useful info in a stack dump. Sure, there might be that rare >>> case when you need it, but how often compared to other useful info >>> maintained on a per thread basis. How many other bits of useful info >> are >>> not being printed in favor of defined_classes? It seems you have more >> in >>> the queue. How many? I'm worried about how cluttered the stack >> dumps >>> will get. Maybe we should add some sort of verbose thread dumping >>> option. Just a thought. >>> >>> As for the implementation, overall it looks good, but I can't speak to >>> whether or not you are doing proper accounting of defined_classes >> and >>> bytes allocated. You'll need input from someone with more knowledge >> of >>> those areas. We'll also need to do some testing to make sure tool tests >>> are not impacted. >>> >>> thanks, >>> >>> Chris >>> >>> On 4/30/18 2:51 AM, Haug, Gunter wrote: >>> > Hi, >>> > >>> > this is an update to an RFR I posted on hotspot-dev, but it is probably >> more suitable to post it here. Can I please have a review and a sponsor for >> the following enhancement: >>> > >>> > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v1 >>> > https://bugs.openjdk.java.net/browse/JDK-8200720 >>> > >>> > We at SAP have extended the thread dumps (obtained by jstack or >> jcmd) by several fields providing thread specific information. These >> extensions are quite popular with our support team. With some knowledge >> of the architecture of the application, they often allow for quick and simple >> diagnosis of a running system. Therefore we would like to contribute these >> enhancements. >>> > >>> > I took a few simple examples here, namely cpu time, elapsed time >> since thread creation, bytes allocated and classes defined by the thread and >> the pthread-id or equivalent on platforms where it makes sense. Provided it >> is known how the application should behave, a misbehaving thread can >> identified easily. >>> > >>> > There is no measurable overhead for this enhancement. However, it >> may be a problem that the format of the output is changed. Tools parsing the >> output may have to be changed. >>> > >>> > Here is an example of the output generated: >>> > >>> > ------------------------------------------------------ >>> > >>> > "main" #1 prio=5 os_prio=31 cpu=6300.65ms elapsed=123.28s >> allocated=242236760B defined_classes=1725 tid=0x00007fa13a806000 >> nid=0x1c03 pthread-id=0x109708000 waiting on condition >> [0x0000000109707000] >>> > java.lang.Thread.State: TIMED_WAITING (sleeping) >>> > JavaThread state: _thread_blocked >>> > Thread: 0x00007fa13a806000 [0x1c03] State: _at_safepoint >> _has_called_back 0 _at_poll_safepoint 0 >>> > JavaThread state: _thread_blocked >>> > at java.lang.Thread.sleep(java.base/Native Method) >>> > ... >>> > ------------------------------------------------------ >>> > >>> > As mentioned above, we have a whole bunch of other >> enhancements to the thread dump similar to this one and would be willing to >> contribute them if there is any interest. >>> > >>> > Thanks and best regards, >>> > Gunter >>> > >>> > >>> >>> >>> >>> >>> From gary.adams at oracle.com Sat Jun 2 11:02:55 2018 From: gary.adams at oracle.com (gary.adams at oracle.com) Date: Sat, 2 Jun 2018 07:02:55 -0400 Subject: RFR: JDK-8169718: nsk/jdb/locals/locals002: ERROR: Cannot find boolVar with expected value: false In-Reply-To: References: <5B082D2E.7000408@oracle.com> <5d58fa2b-7dda-db64-0280-1e19e791a86d@oracle.com> Message-ID: On 6/1/18 6:54 PM, Chris Plummer wrote: > Hi Gary, > > I'm a bit unclear on one point. According to your description, should > we be seeing two prompts for commands like "cont", and "step"? When > you say we are expecting: > > ? "Breakpoint hit:" > ?? > > Isn't it really: > > ?? > ? "Breakpoint hit:" > ?? Let me stretch out the log a little bit The is main[1] or Thread-1[1], etc. and not distinguished for command or event completion. ?? stop at ?? stop at ?? cont ?? cont ?? "Breakpoint hit:" ? "Breakpoint hit:" ? The source of the command or event prompts was verified with a stacktrace in the printPrompt() method. > > I figure this must be the case, because your fix removes the printing > of a prompt, and we still need one prompt, so there must have been two > to start with. I think in the above the last is being > produced sequentially right after the "Breakpoint hit:" output, so is > never an issue. The first comes from the printPrompt() call > after issuing the "cont" command, and it's possible to hit the > breakpoint and start producing more output before printPrompt() has > completed, thus causing the problem you are seeing. printPrompt() is a very quick operation and is not being interrupted. But the output from the event handler is produced incrementally. ?? - event header, e.g. "Breakpont hit:", "Step completed:", etc. ?? - from vmInterrupted() ???? - printCurrentLocation() ???? - monitor command output (if any) ???? - printPrompt() > > There seems to be some purpose to printing this first prompt and I'm > not so sure eliminating it is the right thing to do. I think it might > be there to indicate that the command has been initiated, because > there could be some delay before the breakpoint is hit. So the issue > then is how to make sure this prompt appears in the input stream > before any of the "Step completed:" or "Breakpoint hit:" output. > Possibly event processing should be blocked until executeCommand() > completes. The primary purposes of prompts is for "user interactive" nudge to type another command. But the failures are from tests that are using the presence of a prompt as an ACK of the last command. This "prompt-protocol", does not include any sequence numbers , does not pair prompt to command or event, and does not block during any multipart message formation. To prevent the error we are observing we would have to buffer all of the event output ?? "Breakpoint hit:" ??? Another way to look at the current failure is to say the event handler output has been interrupted by a command sent prematurely. This would never be seen at user speeds typing in commands. I should also mention any other output from the user application can also interfere with the command and event output streams. I have seen stray prompts in the logs from System.out.println(). That was the reason earlier I was looking at changing the tests to look for specific prompts before continuing to the next command. \gra > thanks, > > Chris > > On 5/26/18 3:50 AM, gary.adams at oracle.com wrote: >> This is a review request for a previously closed test bug. >> The test was recently moved to the open repos, and the >> proposed fix is in the open code. >> >> ? Webrev: http://cr.openjdk.java.net/~gadams/8169718/webrev/ >> >> >> -------- Forwarded Message -------- >> Subject:???? RFR: JDK-8169718: nsk/jdb/locals/locals002: ERROR: >> Cannot find boolVar with expected value: false >> Date:???? Fri, 25 May 2018 11:35:10 -0400 >> From:???? Gary Adams >> Reply-To:???? gary.adams at oracle.com >> >> >> >> >> >> The jdb tests use stdin to send commands to a jdb process >> and parses the stdout to determine if a command was >> successful and when the process is prompting for new commands >> to be sent. >> >> Some commands are synchronous, so when the command is completed >> a new prompt is sent back immediately. >> >> Some commands are asynchronous, so there could be a delay >> until a breakpoint is reached. The event handler then sends a prompt >> when the application thread is stopped and new jdb commands can be sent. >> >> The problem causing the intermittent failures was a corruption in the >> output stream when prompts were being sent at the wrong times. >> >> Instead of receiving >> ?? "Breakpoint hit:" >> ??? >> >> the log contained >> ?? "Breakpoint hit:" >> >> Once out of sync, jdb commands were being sent prematurely >> and the wrong values were being compared against expected behavior. >> The simple fix proposed here recognizes that commands like "cont", >> "step" and "next" are asynchronous commands and should not send back >> a prompt immediately. Instead. the event handler will deliver the >> next prompt >> when the next "Breakpoint hit:" or "Step completed:" state change >> occurs. >> >> The bulk of the testing was done on windows-x64-debug builds where the >> intermittent failures were observed in ~5 in 1000 testruns. The fix has >> also been tested on linux-x64-debug, solaris-sparcv9-debug, >> and macosx-x64-debug, even though the failures have never been reported >> against those platforms. >> >> Failures have been observed in many of the nsk/jdb tests with similar >> corrupted >> output streams, but never directly associated with this issue before. >> >> ??redefine001, caught_exception002, locals002, eval001, next001, >> ? stop_at003, step002, print002, trace001, step_up001, read001, >> ? clear004, kill001, set001 >> >> > > From david.holmes at oracle.com Sun Jun 3 22:50:03 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 4 Jun 2018 08:50:03 +1000 Subject: JDK-8069149: jdk.internal.pref.Perf is unaware of the VM option UsePerfData In-Reply-To: <5B1144C5.4010706@oracle.com> References: <5B10485C.2020606@oracle.com> <5B1131E4.70407@oracle.com> <5B1144C5.4010706@oracle.com> Message-ID: <1315d59d-4d3c-94b3-0d2d-6808e438c098@oracle.com> Hi Gary, Thanks for the further explanations, I have a much better view of things now. To summarise: - The VM initializes the perfMemory subsystem during early VM initialization - If !UsePerfdata then perfMemory is not initialized. - The initialization state of the perfMemory system is exposed to the Java Perf class via a local initialization flag set at class init time - The Perf class API checks for initialization (either at the Java level, or in native code) and throws IOException if not initialized. - The users of the Perf class can check for IOExceptions. There was a question as to whether the Perf API should deal with the initialization issue and throw exceptions, or whether the obtaining of the Perf instance should throw the exception. I think the API level is fine as the time related parts of the API still function even if the rest doesn't. Looking at the changes again ... I see four clients of the Perf instance: 1. ./java.base/share/classes/jdk/internal/perf/PerfCounter.java This is the primary API used by the rest of the JDK libraries to interact with the "perf" systems. The counters must not throw any exceptions if the Perf system is not available due to !UsePerfData (which they don't). It's unclear how users of the counters may be impacted if the "counts" always return zero ?? Suggestion: it might be useful if PerfCounter.toString showed if the Perf system was disabled e.g. return name + (perf.isInitialized() ? "" : "[Disabled]" + " = " + get(); 2. ./java.management/share/classes/sun/management/VMManagementImpl.java This code is not modified by your webrev but seems to have its own expectations about the impact of !UsePerfdata: } catch (IllegalArgumentException e) { // If the shared memory doesn't exist e.g. if -XX:-UsePerfData // was set noPerfData = true; } catch (IOException e) { throw new AssertionError(e); } I can't see where/why this code thinks that -UsePerfData leads to IllegalArgumentException, but your throwing of IOException is going to break it badly! This needs further investigation. 3. ./jdk.internal.jvmstat/share/classes/sun/jvmstat/perfdata/monitor/protocol/local/PerfDataBuffer.java This class throws exceptions on any attach failure so in that sense the IOException doesn't change that, however it does appear that this class is intended to throw MonitorException if things go wrong. More generally we should look at how the jvmstat tool(s?) are used and whether they are reporting the lack of UsePerfData in a user-friendly way. 4. ./jdk.management.agent/share/classes/jdk/internal/agent/ConnectorAddressLink.java I'm not at all clear on the use of this class so it is hard to assess the change. The throwing of IOException from the "import" function is fine, but the silent inaction of the "export" functions is less clear. --- On the issue of IOException and message ... I think the native code that throws IOException when not-initialized should use the same message as the Java code. That's the only consistent thing to do IMHO. It would also be helpful to the user if the exception message could tie back to the -UsePerfData setting. I don't normally suggest things like this but these failures (as we found out in SE Embedded days when we did turn this off!) can be somewhat perplexing to the end user. To that end perhaps: throw new IOException("Perf memory is not initialized - check UsePerfData setting"); --- On the meta questions ... it seems the ability to disable UsePerfData can have a significant affect on startup so may be relied upon by some users. So any change to this would need to go through a rigorous process to determine impact. So I'd say that's future work beyond the scope of the current RFE/bug. Thanks, David On 1/06/2018 11:06 PM, Gary Adams wrote: > Comments inline ... > > On 6/1/18, 7:45 AM, Gary Adams wrote: >> PS. >> >> PerfCounter.java >> >> lb field should be left as final. You can work around >> definite-assignment problems by using a local variable and only >> assigning to the field after the catch block: >> >> private PerfCounter(String name, int type) { >> this.name = name; >> + LongBuffer temp = null; >> + try { >> ByteBuffer bb = perf.createLong(name, type, U_None, 0L); >> bb.order(ByteOrder.nativeOrder()); >> + temp = bb.asLongBuffer(); >> + } catch (IOException e) { >> + // may not be available >> + } >> + this.lb = temp; >> } > Thanks for the tip. The PerfCounter changes were the last thing I dropped in > after adding the IOException and fixing the places where things did not > compile. > > Fundamentally, what sort of things will break, when the previous interface > did not expose the possibility that the perf memory was not initialized > as assumed in the earlier code. Does the upper level code "have to" > be aware, or is the contract to do a "best effort" if something lower down > is not as expected. > >> Hi Gary, >> >> Meta-question partially raised in the the recent bug report comments: do >> we care about this any more? Does anything need to disable UsePerfData? >> Can we not deprecate it in 11 then obsolete in 12? That would be a lot >> simpler. > My own bias is that many of the motivations for faster startup and > smaller footprint > are still useful goals. If not for embedded systems, then for container > environments. > A process started with -XX:-UsePerfData does not support external attach > and does not maintain the internal variables for the self attach. > >> >> That aside ... >> >> It's hard to see how all the different paths in these related API's >> stitch together - especially when you have to consider the API being >> used on the current VM or a different one. I'm not at all clear what the >> control flow is, so it is hard to see exactly how the initialization >> issue is being handled. As I wrote in the initial report it is unclear >> exactly which parts of the "perf" subsystem are impacted by the value of >> UsePerfData. > Starting with the Perf.java class initialization, I added the exposure of > the PerfMemory::is_initialized(). > > Perf.java: > > 559 private native boolean isInitialized(); > 560 > 561 private static native void registerNatives(); > 562 > 563 static { > 564 registerNatives(); > 565 instance = new Perf(); > 566 initialized = instance.isInitialized(); > 567 } > > perf.cpp: > 298 PERF_ENTRY(jboolean, Perf_IsInitialized(JNIEnv *env, jobject perf)) > 299 > 300 PerfWrapper("Perf_IsInitialized"); > 301 > 302 return PerfMemory::is_initialized(); > 303 > 304 PERF_END > > > From the VM perspective UsePerfData is both a command line argument > and a flag to be used whenever that information is needed. There are > scattered checks to deal with setup and shutdown, but also the runtime > fulfillment of the PerfMemory data structures. > > Looking at perfMemory_init() and perfMemory_exit(), I concluded that > the UsePerfData flag essentially noops all of the perfMemory allocation and > updating. The only information required in the upper layers of the Perf.java > code is the fact that the perf memory is not initialized. > > perfMemory.cpp: > > 58 void perfMemory_init() { > 59 > 60 if (!UsePerfData) return; > 61 > 62 PerfMemory::initialize(); > 63 } > 64 > 65 void perfMemory_exit() { > 66 > 67 if (!UsePerfData) return; > 68 if (!PerfMemory::is_usable()) return; > 69 > ... > 90 void PerfMemory::initialize() { > ... > 159 OrderAccess::release_store(&_initialized, 1); > 160 } > ... > 271 bool PerfMemory::is_initialized() { > 272 return OrderAccess::load_acquire(&_initialized) != 0; > 273 } > >> >> Can you summarize the role of the is_initialized check versus a direct >> UsePerfData check on the VM side i.e. which should be checked when? I >> immediately see an issue that Perf_Detach is a no-op when !UsePerfdata, >> yet Perf_Attach doesn't examine the flag at all! But perhaps we never >> reach those methods because of a higher-level call filtering it out ?? > There is split path when attaching to self versus attaching to other > running VM processes. The extra detach processing is only needed > for the remote VM case. If attach fails, there is no need to detach. > > Perf.java: > ... > > 274 private ByteBuffer attachImpl(String user, int lvmid, int mode) > 275 throws IllegalArgumentException, IOException > 276 { > 277 final ByteBuffer b = attach(user, lvmid, mode); > 278 > 279 if (lvmid == 0) { > 280 // The native instrumentation buffer for this Java virtual > 281 // machine is never unmapped. > 282 return b; > 283 } > 284 else { > 285 // This is an instrumentation buffer for another Java virtual > 286 // machine with native resources that need to be managed. We > 287 // create a duplicate of the native ByteBuffer and manage it > 288 // with a Cleaner. When the duplicate becomes phantom reachable, > 289 // the native resources will be released. > 290 > 291 final ByteBuffer dup = b.duplicate(); > 292 > 293 CleanerFactory.cleaner() > 294 .register(dup, new CleanerAction(instance, b)); > 295 return dup; > 296 } > 297 } > > > >> >> More below ... > >> >> David >> >> On 1/06/2018 5:09 AM, Gary Adams wrote: >> >/A patch was done previously to prevent an error when -XX:-UsePerfData />/is used, but this bug was filed to make the setting more visible in the />/jdk.internal.perf package. />//>/?? Webrev: http://cr.openjdk.java.net/~gadams/8069149/webrev.00/ >> />/?? Issue: https://bugs.openjdk.java.net/browse/JDK-8069149 />//>/I have tried a few initial tests using: />//>/?? make run-tests \ />/???????? TEST_OPTS=VM_OPTIONS=-XX:-UsePerfData \ />/???????? TEST=open/test/jdk/sun/management/jmxremote />//>/While I'm tracking down one test timeout,? I'd like to get some />/feedback on the approach used here : />//>/?? - the basic mechanism propagates the "is initialized" state up to />/Perf.java; />/????? does the state need to be exposed to users getPerf(), or is it />/sufficient/ > >> Within PerfMemory UsePerfData and is_usable() are used for guarding >> execution. Perhaps that is what needs to be propagated up? > > /The is_usable() state is used to ensure the memory is initialized and > not yet detached. We don't need that specific condition here. > /Unclear what the control flow is here. I don't see that Perf triggers >> initialization of the perf subsystem, so what guarantees initialization >> should have happened by the time the Perf class is initialized ?? > > The private constructor and the static initializer in Perf ensures > the singleton behavior at class initialization. The perfMemory_init() > is handled during VM initialization. > > init.cpp: > > 90 void vm_init_globals() { > 91 check_ThreadShadow(); > 92 basic_types_init(); > 93 eventlog_init(); > 94 mutex_init(); > 95 chunkpool_init(); > 96 perfMemory_init(); > 97 SuspendibleThreadSet_init(); > 98 } > > >> >/ ?? - an existing use of IOException was used in the case of attach /> >/failures; /> >/ ????? added IOExceptions to the createXXX methods if the memory was not /> >/initialized / >> On the VM side can you at least add a message to the IOException so that >> the user can tell why it was thrown. > > Initially, I included a message with the IOException, and noticed the other > exceptions being thrown in the surrounding code did not include a message, > so opted for the empty message. There is no ambiguity here. > >> >/ ?? - presuming a CSR would be needed to cover the IOExceptions / >> It's not a public API so no CSR should be needed. > > Good to know. > >> >/ ?? - it appears that jdk.internal.perf has very limited usage, /> >/ ????? is it still needed/used (?) / >> I can't answer that part, but I don't think UsePerfData is needed any more. > > If this turns into "removal of UsePerfData" support, we should move the bug > to the runtime group. I grabbed the bug initially, because it looked like > a loose end and a simple thing to address by fulfilling the original > request to propagate the visibility up to the Perf.java code. > > Let's see if other reviewers have similar views - that the UsePerfData > flag can be removed. This bug was originally filed against 8u60 and is > currently targetted for 12. I only explored a potential fix at this time > in case we need to get a deprecation process started at this time. > > ... From david.holmes at oracle.com Sun Jun 3 23:12:53 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 4 Jun 2018 09:12:53 +1000 Subject: RFR(S): 8200720: Print additional information in thread dump (times, allocated bytes etc.) In-Reply-To: <933cdacc-9b6f-c8b5-e244-558f38e7888c@oracle.com> References: <56817AB5-88FC-4FB9-A271-7206EA69ECA7@sap.com> <2728581E-0E32-4911-B1A9-79AD6CBA81B0@sap.com> <082ca98f-9a68-15a0-95b5-c4d5c733ab36@oracle.com> <933cdacc-9b6f-c8b5-e244-558f38e7888c@oracle.com> Message-ID: <55806aa8-f09b-e9a8-b512-b9676029c494@oracle.com> Hi Gunter, Chris, Sorry just trying to catch up and this is only a partial look so far ... BTW these changes are not limited to serviceability code so adding in runtime team as well. On 2/06/2018 9:12 AM, Chris Plummer wrote: > Hi Gunter, > > On 6/1/18 3:17 AM, Haug, Gunter wrote: >> Hi Chris, >> >> thanks for looking into this! >> >> Re the synchronization: The value is stored only in a VM operation at >> a safepoint by the VM thread. I was of the opinion, that this could >> not be interrupted by a second VM operation (of the same type). Or am >> I missing something else? > I was really thinking more about the temporary changing of > PrintExtendedThreadInfo, not the value stored in the VMOp. You may be be > correct that no more than one VMOp is executing, but while it is > executing it is has changed the value of PrintExtendedThreadInfo, which > might have an impact on anything else that triggers printing thread info > while the VMOp is executing. Even if nothing else can possibly be running during the safepoint I find it extremely bad form to change a command-line flag in this way, particularly from a safepoint! If this flag is intended to be dynamically enabled as part of a dcmd then it should also be a Manageable flag IMHO. That said ... >> ? I did think about passing an argument to the various print_on member >> functions of the thread classes, but this would require rather >> extensive code changes for a rather tiny extension. Therefore I feel >> doing it like this is the lesser evil. ... it's obviously not truly a global setting, but one that is needed on a per-print-request basis. The flag is just the default, but if the default is off you still want to enable extended printing on a per-request basis. The current print_on mechanics is not set up for this kind of flexibility. I think this needs more thought. --- Re osThread.cpp shared code changes ... I really hate to see all the ifdefs in there. Please add a pd_print_on function to support this if you truly want platform specific information. --- threadStatisticInfo.hpp typo: getElepsedTime() -> getElapsedTime() Thanks, David >> >> Re thread_dump(): I think it's correct or, well, at least it works ;-) >> In fact jstack will transfer the "-e" and "-l" in only one string, >> i.e. op->arg(0). > So you are saying that op->arg(0) is "-e -l" when using jstack? I think > you really need to clean up the parsing. As it stands right now passing, > I get the feeling that if a user erroneously asks for help by using > "jcmd Thread.Print -help", it will end up executing with -e an -l > enabled, and no failure for the invalid "-help" option. >> ? Christoph has already explained my reasoning. But I agree, it's not >> nice and I would be happy to do it like Christoph suggested. > I'll respond separately to his suggestion. > > thanks, > > Chris >> >> And typo fixed, sorry. >> >> Thanks again, >> Gunter >> >> ?On 01.06.18, 00:03, "Chris Plummer" wrote: >> >> ???? Hi Gunter, >> ???? globals.hpp: fix typo "informatiuon" >> ???? I worry a little bit about the synchronizing (if that's the right >> word) >> ???? of PrintExtendedThreadInfo and the dcmd's -e flag. When using -e, >> you >> ???? are temporarily enabling PrintExtendedThreadInfo if it was false. >> This >> ???? temporarily changes the behavior of thread dumps, and could >> impact other >> ???? uses that happen in parallel. Also, could two simultaneous uses >> of -e >> ???? result in PrintExtendedThreadInfo not getting restored properly? >> ???? thread_dump() doesn't look right. It looks like you are iterating >> char >> ???? by char over the argument, and expect something like "-el" to be >> ???? specified rather then "-e -l". The loop should be iterating over >> ???? op->arg(i), not op->arg(0)[i]. >> ???? The rest of the changes look fine. >> ???? thanks, >> ???? Chris >> ???? On 5/30/18 8:12 AM, Haug, Gunter wrote: >> ???? > Hi all, >> ???? > >> ???? > As Chris proposed, I have made an the extended output >> switchable. There is an VM flag (PrintExtendedThreadInfo), which is >> false by default. Moreover, there is an Option (-e) which can be used >> with jcmd Thread.print as well as with jstack. The -e option >> essentially sets PrintExtendedThreadInfo true just for the respective >> thread dump. >> ???? > >> ???? > Here is the updated webrev: >> ???? > >> ???? > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v2 >> ???? > >> ???? > (https://bugs.openjdk.java.net/browse/JDK-8200720) >> ???? > >> ???? > Thanks, >> ???? > Gunter >> ???? > >> ???? > >> ???? > On 02.05.18, 17:07, "serviceability-dev on behalf of Haug, >> Gunter" > gunter.haug at sap.com> wrote: >> ???? > >> ???? >????? Hi Chris, >> ???? > >> ???? >????? Thanks for looking into this. >> ???? >????? You're right, there is a little more we have. We have >> implemented an IO tracing mechanism which - rather as a byproduct - >> keeps track of bytes read and written per thread. However, this of >> course requires changes not only in hotspot. We would be happy to >> contribute this as well, but this is a far bigger change and will >> probably lead to a far bigger discussion. Anyway, with the number of >> bytes read, the number of classes defined doesn't look that arbitrary >> anymore, as one can correlate IO to class loading. >> ???? > >> ???? >????? Regarding the verbose option I think that's a good idea! >> ???? > >> ???? >????? Thanks again, >> ???? >????? Gunter >> ???? > >> ???? >????? On 01.05.18, 22:55, "Chris Plummer" >> wrote: >> ???? > >> ???? >????????? Hi Gunter, >> ???? > >> ???? >????????? The output you are adding is all useful. I think the >> question is (and >> ???? >????????? I'd like to see a few people chime in on this for this >> review) is >> ???? >????????? whether or not all of it is the appropriate for a >> thread's stack dump. >> ???? >????????? For example, defined_classes is on the fringe of what >> I would call >> ???? >????????? generally useful info in a stack dump. Sure, there >> might be that rare >> ???? >????????? case when you need it, but how often compared to other >> useful info >> ???? >????????? maintained on a per thread basis. How many other bits >> of useful info are >> ???? >????????? not being printed in favor of defined_classes? It >> seems you have more in >> ???? >????????? the queue. How many? I'm worried about how cluttered >> the stack dumps >> ???? >????????? will get. Maybe we should add some sort of verbose >> thread dumping >> ???? >????????? option. Just a thought. >> ???? > >> ???? >????????? As for the implementation, overall it looks good, but >> I can't speak to >> ???? >????????? whether or not you are doing proper accounting of >> defined_classes and >> ???? >????????? bytes allocated. You'll need input from someone with >> more knowledge of >> ???? >????????? those areas. We'll also need to do some testing to >> make sure tool tests >> ???? >????????? are not impacted. >> ???? > >> ???? >????????? thanks, >> ???? > >> ???? >????????? Chris >> ???? > >> ???? >????????? On 4/30/18 2:51 AM, Haug, Gunter wrote: >> ???? >????????? > Hi, >> ???? >????????? > >> ???? >????????? > this is an update to an RFR I posted on hotspot-dev, >> but it is probably more suitable to post it here. Can I please have a >> review and a sponsor for the following enhancement: >> ???? >????????? > >> ???? >????????? > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v1 >> ???? >????????? > https://bugs.openjdk.java.net/browse/JDK-8200720 >> ???? >????????? > >> ???? >????????? > We at SAP have extended the thread dumps (obtained >> by jstack or jcmd) by several fields providing thread specific >> information. These extensions are quite popular with our support team. >> With some knowledge of the architecture of the application, they often >> allow for quick and simple diagnosis of a running system. Therefore we >> would like to contribute these enhancements. >> ???? >????????? > >> ???? >????????? > I took a few simple examples here, namely cpu time, >> elapsed time since thread creation, bytes allocated and classes >> defined by the thread and the pthread-id or equivalent on platforms >> where it makes sense. Provided it is known how the application should >> behave, a misbehaving thread can identified easily. >> ???? >????????? > >> ???? >????????? > There is no measurable overhead for this >> enhancement. However, it may be a problem that the format of the >> output is changed. Tools parsing the output may have to be changed. >> ???? >????????? > >> ???? >????????? > Here is an example of the output generated: >> ???? >????????? > >> ???? >????????? > ------------------------------------------------------ >> ???? >????????? > >> ???? >????????? > "main" #1 prio=5 os_prio=31 cpu=6300.65ms >> elapsed=123.28s allocated=242236760B defined_classes=1725 >> tid=0x00007fa13a806000 nid=0x1c03 pthread-id=0x109708000 waiting on >> condition [0x0000000109707000] >> ???? >????????? >???? java.lang.Thread.State: TIMED_WAITING (sleeping) >> ???? >????????? >???? JavaThread state: _thread_blocked >> ???? >????????? > Thread: 0x00007fa13a806000 [0x1c03] State: >> _at_safepoint _has_called_back 0 _at_poll_safepoint 0 >> ???? >????????? >???? JavaThread state: _thread_blocked >> ???? >????????? > at java.lang.Thread.sleep(java.base/Native Method) >> ???? >????????? >???????????? ... >> ???? >????????? > ------------------------------------------------------ >> ???? >????????? > >> ???? >????????? > As mentioned above, we have a whole bunch of other >> enhancements to the thread dump similar to this one and would be >> willing to contribute them if there is any interest. >> ???? >????????? > >> ???? >????????? > Thanks and best regards, >> ???? >????????? > Gunter >> ???? >????????? > >> ???? >????????? > >> ???? > >> ???? > >> ???? > >> ???? > >> ???? > >> > > From david.holmes at oracle.com Sun Jun 3 23:18:55 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 4 Jun 2018 09:18:55 +1000 Subject: RFR(S): 8200720: Print additional information in thread dump (times, allocated bytes etc.) In-Reply-To: <55806aa8-f09b-e9a8-b512-b9676029c494@oracle.com> References: <56817AB5-88FC-4FB9-A271-7206EA69ECA7@sap.com> <2728581E-0E32-4911-B1A9-79AD6CBA81B0@sap.com> <082ca98f-9a68-15a0-95b5-c4d5c733ab36@oracle.com> <933cdacc-9b6f-c8b5-e244-558f38e7888c@oracle.com> <55806aa8-f09b-e9a8-b512-b9676029c494@oracle.com> Message-ID: <816dda07-3523-50d1-6521-f76d45506889@oracle.com> PS. uint64_t getElepsedTime() const { return os::javaTimeMillis() - getStartTime(); } Millisecond resolution seems rather coarse. David ----- On 4/06/2018 9:12 AM, David Holmes wrote: > Hi Gunter, Chris, > > Sorry just trying to catch up and this is only a partial look so far ... > > BTW these changes are not limited to serviceability code so adding in > runtime team as well. > > On 2/06/2018 9:12 AM, Chris Plummer wrote: >> Hi Gunter, >> >> On 6/1/18 3:17 AM, Haug, Gunter wrote: >>> Hi Chris, >>> >>> thanks for looking into this! >>> >>> Re the synchronization: The value is stored only in a VM operation at >>> a safepoint by the VM thread. I was of the opinion, that this could >>> not be interrupted by a second VM operation (of the same type). Or am >>> I missing something else? >> I was really thinking more about the temporary changing of >> PrintExtendedThreadInfo, not the value stored in the VMOp. You may be >> be correct that no more than one VMOp is executing, but while it is >> executing it is has changed the value of PrintExtendedThreadInfo, >> which might have an impact on anything else that triggers printing >> thread info while the VMOp is executing. > > Even if nothing else can possibly be running during the safepoint I find > it extremely bad form to change a command-line flag in this way, > particularly from a safepoint! > > If this flag is intended to be dynamically enabled as part of a dcmd > then it should also be a Manageable flag IMHO. > > That said ... > >>> ? I did think about passing an argument to the various print_on >>> member functions of the thread classes, but this would require rather >>> extensive code changes for a rather tiny extension. Therefore I feel >>> doing it like this is the lesser evil. > > ... it's obviously not truly a global setting, but one that is needed on > a per-print-request basis. The flag is just the default, but if the > default is off you still want to enable extended printing on a > per-request basis. > > The current print_on mechanics is not set up for this kind of > flexibility. I think this needs more thought. > > --- > > Re osThread.cpp shared code changes ... I really hate to see all the > ifdefs in there. Please add a pd_print_on function to support this if > you truly want platform specific information. > > --- > > threadStatisticInfo.hpp > > typo: getElepsedTime() -> getElapsedTime() > > Thanks, > David > > > >>> >>> Re thread_dump(): I think it's correct or, well, at least it works >>> ;-) In fact jstack will transfer the "-e" and "-l" in only one >>> string, i.e. op->arg(0). >> So you are saying that op->arg(0) is "-e -l" when using jstack? I >> think you really need to clean up the parsing. As it stands right now >> passing, I get the feeling that if a user erroneously asks for help by >> using "jcmd Thread.Print -help", it will end up executing with >> -e an -l enabled, and no failure for the invalid "-help" option. >>> ? Christoph has already explained my reasoning. But I agree, it's not >>> nice and I would be happy to do it like Christoph suggested. >> I'll respond separately to his suggestion. >> >> thanks, >> >> Chris >>> >>> And typo fixed, sorry. >>> >>> Thanks again, >>> Gunter >>> >>> ?On 01.06.18, 00:03, "Chris Plummer" wrote: >>> >>> ???? Hi Gunter, >>> ???? globals.hpp: fix typo "informatiuon" >>> ???? I worry a little bit about the synchronizing (if that's the >>> right word) >>> ???? of PrintExtendedThreadInfo and the dcmd's -e flag. When using >>> -e, you >>> ???? are temporarily enabling PrintExtendedThreadInfo if it was >>> false. This >>> ???? temporarily changes the behavior of thread dumps, and could >>> impact other >>> ???? uses that happen in parallel. Also, could two simultaneous uses >>> of -e >>> ???? result in PrintExtendedThreadInfo not getting restored properly? >>> ???? thread_dump() doesn't look right. It looks like you are >>> iterating char >>> ???? by char over the argument, and expect something like "-el" to be >>> ???? specified rather then "-e -l". The loop should be iterating over >>> ???? op->arg(i), not op->arg(0)[i]. >>> ???? The rest of the changes look fine. >>> ???? thanks, >>> ???? Chris >>> ???? On 5/30/18 8:12 AM, Haug, Gunter wrote: >>> ???? > Hi all, >>> ???? > >>> ???? > As Chris proposed, I have made an the extended output >>> switchable. There is an VM flag (PrintExtendedThreadInfo), which is >>> false by default. Moreover, there is an Option (-e) which can be used >>> with jcmd Thread.print as well as with jstack. The -e option >>> essentially sets PrintExtendedThreadInfo true just for the respective >>> thread dump. >>> ???? > >>> ???? > Here is the updated webrev: >>> ???? > >>> ???? > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v2 >>> ???? > >>> ???? > (https://bugs.openjdk.java.net/browse/JDK-8200720) >>> ???? > >>> ???? > Thanks, >>> ???? > Gunter >>> ???? > >>> ???? > >>> ???? > On 02.05.18, 17:07, "serviceability-dev on behalf of Haug, >>> Gunter" >> gunter.haug at sap.com> wrote: >>> ???? > >>> ???? >????? Hi Chris, >>> ???? > >>> ???? >????? Thanks for looking into this. >>> ???? >????? You're right, there is a little more we have. We have >>> implemented an IO tracing mechanism which - rather as a byproduct - >>> keeps track of bytes read and written per thread. However, this of >>> course requires changes not only in hotspot. We would be happy to >>> contribute this as well, but this is a far bigger change and will >>> probably lead to a far bigger discussion. Anyway, with the number of >>> bytes read, the number of classes defined doesn't look that arbitrary >>> anymore, as one can correlate IO to class loading. >>> ???? > >>> ???? >????? Regarding the verbose option I think that's a good idea! >>> ???? > >>> ???? >????? Thanks again, >>> ???? >????? Gunter >>> ???? > >>> ???? >????? On 01.05.18, 22:55, "Chris Plummer" >>> wrote: >>> ???? > >>> ???? >????????? Hi Gunter, >>> ???? > >>> ???? >????????? The output you are adding is all useful. I think the >>> question is (and >>> ???? >????????? I'd like to see a few people chime in on this for >>> this review) is >>> ???? >????????? whether or not all of it is the appropriate for a >>> thread's stack dump. >>> ???? >????????? For example, defined_classes is on the fringe of what >>> I would call >>> ???? >????????? generally useful info in a stack dump. Sure, there >>> might be that rare >>> ???? >????????? case when you need it, but how often compared to >>> other useful info >>> ???? >????????? maintained on a per thread basis. How many other bits >>> of useful info are >>> ???? >????????? not being printed in favor of defined_classes? It >>> seems you have more in >>> ???? >????????? the queue. How many? I'm worried about how cluttered >>> the stack dumps >>> ???? >????????? will get. Maybe we should add some sort of verbose >>> thread dumping >>> ???? >????????? option. Just a thought. >>> ???? > >>> ???? >????????? As for the implementation, overall it looks good, but >>> I can't speak to >>> ???? >????????? whether or not you are doing proper accounting of >>> defined_classes and >>> ???? >????????? bytes allocated. You'll need input from someone with >>> more knowledge of >>> ???? >????????? those areas. We'll also need to do some testing to >>> make sure tool tests >>> ???? >????????? are not impacted. >>> ???? > >>> ???? >????????? thanks, >>> ???? > >>> ???? >????????? Chris >>> ???? > >>> ???? >????????? On 4/30/18 2:51 AM, Haug, Gunter wrote: >>> ???? >????????? > Hi, >>> ???? >????????? > >>> ???? >????????? > this is an update to an RFR I posted on >>> hotspot-dev, but it is probably more suitable to post it here. Can I >>> please have a review and a sponsor for the following enhancement: >>> ???? >????????? > >>> ???? >????????? > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v1 >>> ???? >????????? > https://bugs.openjdk.java.net/browse/JDK-8200720 >>> ???? >????????? > >>> ???? >????????? > We at SAP have extended the thread dumps (obtained >>> by jstack or jcmd) by several fields providing thread specific >>> information. These extensions are quite popular with our support >>> team. With some knowledge of the architecture of the application, >>> they often allow for quick and simple diagnosis of a running system. >>> Therefore we would like to contribute these enhancements. >>> ???? >????????? > >>> ???? >????????? > I took a few simple examples here, namely cpu time, >>> elapsed time since thread creation, bytes allocated and classes >>> defined by the thread and the pthread-id or equivalent on platforms >>> where it makes sense. Provided it is known how the application should >>> behave, a misbehaving thread can identified easily. >>> ???? >????????? > >>> ???? >????????? > There is no measurable overhead for this >>> enhancement. However, it may be a problem that the format of the >>> output is changed. Tools parsing the output may have to be changed. >>> ???? >????????? > >>> ???? >????????? > Here is an example of the output generated: >>> ???? >????????? > >>> ???? >????????? > ------------------------------------------------------ >>> ???? >????????? > >>> ???? >????????? > "main" #1 prio=5 os_prio=31 cpu=6300.65ms >>> elapsed=123.28s allocated=242236760B defined_classes=1725 >>> tid=0x00007fa13a806000 nid=0x1c03 pthread-id=0x109708000 waiting on >>> condition [0x0000000109707000] >>> ???? >????????? >???? java.lang.Thread.State: TIMED_WAITING (sleeping) >>> ???? >????????? >???? JavaThread state: _thread_blocked >>> ???? >????????? > Thread: 0x00007fa13a806000 [0x1c03] State: >>> _at_safepoint _has_called_back 0 _at_poll_safepoint 0 >>> ???? >????????? >???? JavaThread state: _thread_blocked >>> ???? >????????? > at java.lang.Thread.sleep(java.base/Native Method) >>> ???? >????????? >???????????? ... >>> ???? >????????? > ------------------------------------------------------ >>> ???? >????????? > >>> ???? >????????? > As mentioned above, we have a whole bunch of other >>> enhancements to the thread dump similar to this one and would be >>> willing to contribute them if there is any interest. >>> ???? >????????? > >>> ???? >????????? > Thanks and best regards, >>> ???? >????????? > Gunter >>> ???? >????????? > >>> ???? >????????? > >>> ???? > >>> ???? > >>> ???? > >>> ???? > >>> ???? > >>> >> >> From per.liden at oracle.com Mon Jun 4 06:51:40 2018 From: per.liden at oracle.com (Per Liden) Date: Mon, 4 Jun 2018 08:51:40 +0200 Subject: RFR (round 4), JEP-318: Epsilon GC In-Reply-To: <0b5c827a-2763-98aa-06af-24df9028aed7@oracle.com> References: <094e1093-5a13-4853-aa34-d4e987a069b0@redhat.com> <0b5c827a-2763-98aa-06af-24df9028aed7@oracle.com> Message-ID: Hi, On 06/01/2018 10:13 AM, Jini George wrote: [...] > ==>? share/classes/sun/jvm/hotspot/oops/ObjectHeap.java > > 445???? } else { > 446??????? if (Assert.ASSERTS_ENABLED) { > 447?????????? Assert.that(false, "Expecting GenCollectedHeap, > G1CollectedHeap, " + > 448?????????????????????? "or ParallelScavengeHeap, but got " + > 449?????????????????????? heap.getClass().getName()); > 450??????? } > 451???? } > > * Please add EpsilonGC also to the assertion statement. > May I suggest that we change this to something like this, to avoid having to update this message when a new collector is added? Assert.that(false, "Unexpected CollectedHeap type: " + heap.getClass().getName()); /Per From jini.george at oracle.com Mon Jun 4 06:58:16 2018 From: jini.george at oracle.com (Jini George) Date: Mon, 4 Jun 2018 12:28:16 +0530 Subject: RFR (round 4), JEP-318: Epsilon GC In-Reply-To: References: <094e1093-5a13-4853-aa34-d4e987a069b0@redhat.com> <0b5c827a-2763-98aa-06af-24df9028aed7@oracle.com> Message-ID: Sounds good to me. Thanks, Jini. On 6/4/2018 12:21 PM, Per Liden wrote: > Hi, > > On 06/01/2018 10:13 AM, Jini George wrote: > [...] >> ==>? share/classes/sun/jvm/hotspot/oops/ObjectHeap.java >> >> 445???? } else { >> 446??????? if (Assert.ASSERTS_ENABLED) { >> 447?????????? Assert.that(false, "Expecting GenCollectedHeap, >> G1CollectedHeap, " + >> 448?????????????????????? "or ParallelScavengeHeap, but got " + >> 449?????????????????????? heap.getClass().getName()); >> 450??????? } >> 451???? } >> >> * Please add EpsilonGC also to the assertion statement. >> > > May I suggest that we change this to something like this, to avoid > having to update this message when a new collector is added? > > Assert.that(false, "Unexpected CollectedHeap type: " + > heap.getClass().getName()); > > /Per From kevin.walls at oracle.com Mon Jun 4 13:36:37 2018 From: kevin.walls at oracle.com (Kevin Walls) Date: Mon, 4 Jun 2018 14:36:37 +0100 Subject: RFR: JDK-8199729 - Usage Logger open sourcing In-Reply-To: References: <36b5fda6-85d8-450a-8344-0050642ee540@default> Message-ID: <6f558c88-083c-2d24-dc2e-6e2f389494a3@oracle.com> Hi - I think the history of post_vm_init_hook_enabled was from the hotspot-express idea that the JVM and the runtime environment could be separate.? We could release a new JVM and slot it into an existing JDK/JRE...? In practice, the commented future optimisation never happened, but they've been in unison for years now and the VM has always tried to run PostVMInitHook, specifically ignoring any problems such as the class not existing in an open build. Duplication inside PostVMInitHook.java was to avoid loading additional classes in every JVM everywhere unless they were likely to have the logger configured. (I've seen Sharath's deferral message, but thought I'd mention this historical point...) Thanks! Kevin On 31/05/2018 03:44, David Holmes wrote: > Hi Sharath, > > I second Bernd's comments about removing all the ORCL_ prefixes. > > I'm unclear how this utility is managed. I expect it to be opt-in but > I see: > > ??? // Advertise presence of PostVMInitHook: > ??? // future optimization: detect if this is enabled. > ??? info->post_vm_init_hook_enabled = 1; > > in src/java.base/share/native/libjava/jdk_util.c, which suggests it is > in fact always going to be run (at least to the point where it looks > for the config file). > > Thanks, > David > > > On 31/05/2018 1:50 AM, Sharath Ballal wrote: >> Hello, >> >> Requesting reviews for Usage Logger open sourcing (earlier known as >> Usage Tracker). >> >> ID: https://bugs.openjdk.java.net/browse/JDK-8199729 >> >> Webrev: http://cr.openjdk.java.net/~sballal/8199729/webrev.00/ >> >> Mach5 run has been successful for Usage Logger tests and hs-tier1, >> hs-tier2. >> >> Thanks, >> >> Sharath >> From gary.adams at oracle.com Mon Jun 4 15:17:05 2018 From: gary.adams at oracle.com (gary.adams at oracle.com) Date: Mon, 4 Jun 2018 11:17:05 -0400 Subject: JDK-8069149: jdk.internal.pref.Perf is unaware of the VM option UsePerfData In-Reply-To: <1315d59d-4d3c-94b3-0d2d-6808e438c098@oracle.com> References: <5B10485C.2020606@oracle.com> <5B1131E4.70407@oracle.com> <5B1144C5.4010706@oracle.com> <1315d59d-4d3c-94b3-0d2d-6808e438c098@oracle.com> Message-ID: <2a350c2c-9853-44b4-98da-5969fdce9ee1@oracle.com> Comments inline below ... On 6/3/18 6:50 PM, David Holmes wrote: > Hi Gary, > > Thanks for the further explanations, I have a much better view of > things now. To summarise: > - The VM initializes the perfMemory subsystem during early VM > initialization > - If !UsePerfdata then perfMemory is not initialized. > - The initialization state of the perfMemory system is exposed to the > Java Perf class via a local initialization flag set at class init time > - The Perf class API checks for initialization (either at the Java > level, or in native code) and throws IOException if not initialized. > - The users of the Perf class can check for IOExceptions. > > There was a question as to whether the Perf API should deal with the > initialization issue and throw exceptions, or whether the obtaining of > the Perf instance should throw the exception. I think the API level is > fine as the time related parts of the API still function even if the > rest doesn't. > > Looking at the changes again ... > > I see four clients of the Perf instance: > > 1. ./java.base/share/classes/jdk/internal/perf/PerfCounter.java > > This is the primary API used by the rest of the JDK libraries to > interact with the "perf" systems. The counters must not throw any > exceptions if the Perf system is not available due to !UsePerfData > (which they don't). It's unclear how users of the counters may be > impacted if the "counts" always return zero ?? > > Suggestion: it might be useful if PerfCounter.toString showed if the > Perf system was disabled e.g. > > return name + (perf.isInitialized() ? "" : "[Disabled]" + " = " + get(); I'll include this in the next webrev. I'll also make sure no one has a dependency on the PerfCounter.toString formatting. > > > 2. ./java.management/share/classes/sun/management/VMManagementImpl.java > > This code is not modified by your webrev but seems to have its own > expectations about the impact of !UsePerfdata: > > ?????? } catch (IllegalArgumentException e) { > ??????????? // If the shared memory doesn't exist e.g. if > -XX:-UsePerfData > ??????????? // was set > ??????????? noPerfData = true; > ??????? } catch (IOException e) { > ??????????? throw new AssertionError(e); > ??????? } > > I can't see where/why this code thinks that -UsePerfData leads to > IllegalArgumentException, but your throwing of IOException is going to > break it badly! This needs further investigation. Yes, I saw that stray wrong comment earlier and was wondering what to do with it. In ConnectorAddressLink importFrom there is a sequence where a caught IllegalArgumentException is converted to an IOException from a failed attach(). The Perf.attach() only throws IllegalArgumentException when the rw or ro mode is not passed correctly. Or when the platform code detects a missing process or permissions error. See open/src/hotspot/os/perfMemory_.cpp The IOException is a catch all for any other error case, such as stat not being able to determine the size of the mapped shared memory file. The sequence in VMManagementImpl.java seems to imply the IllegalArgiumentException cases are fine to ignore, but the IOException is worth an exception. I need to follow that Assertion to see how it is used. > > 3. > ./jdk.internal.jvmstat/share/classes/sun/jvmstat/perfdata/monitor/protocol/local/PerfDataBuffer.java > > This class throws exceptions on any attach failure so in that sense > the IOException doesn't change that, however it does appear that this > class is intended to throw MonitorException if things go wrong. More > generally we should look at how the jvmstat tool(s?) are used and > whether they are reporting the lack of UsePerfData in a user-friendly > way. Agreed I need to follow up on the other higher level usages. > > 4. > ./jdk.management.agent/share/classes/jdk/internal/agent/ConnectorAddressLink.java > > I'm not at all clear on the use of this class so it is hard to assess > the change. The throwing of IOException from the "import" function is > fine, but the silent inaction of the "export" functions is less clear. > > --- > > On the issue of IOException and message ... I think the native code > that throws IOException when not-initialized should use the same > message as the Java code. That's the only consistent thing to do IMHO. > It would also be helpful to the user if the exception message could > tie back to the -UsePerfData setting. I don't normally suggest things > like this but these failures (as we found out in SE Embedded days when > we did turn this off!) can be somewhat perplexing to the end user. To > that end perhaps: > > throw new IOException("Perf memory is not initialized - check > UsePerfData setting"); I'm Ok with "Perf memory is not initialized", but uncomfortable with the "check UsePerfData setting". Obviously, there would be a disconnect between the disabling of the feature and the application attempting to use the capability. > > --- > > On the meta questions ... it seems the ability to disable UsePerfData > can have a significant affect on startup so may be relied upon by some > users. So any change to this would need to go through a rigorous > process to determine impact. So I'd say that's future work beyond the > scope of the current RFE/bug. > > Thanks, > David > > On 1/06/2018 11:06 PM, Gary Adams wrote: >> Comments inline ... >> >> On 6/1/18, 7:45 AM, Gary Adams wrote: >>> PS. >>> >>> PerfCounter.java >>> >>> lb field should be left as final. You can work around >>> definite-assignment problems by using a local variable and only >>> assigning to the field after the catch block: >>> >>> ?????? private PerfCounter(String name, int type) { >>> ??????????? this.name = name; >>> +???????? LongBuffer temp = null; >>> +???????? try { >>> ??????????????? ByteBuffer bb = perf.createLong(name, type, U_None, >>> 0L); >>> ??????????????? bb.order(ByteOrder.nativeOrder()); >>> +???????????? temp = bb.asLongBuffer(); >>> +???????? } catch (IOException e) { >>> +???????????? // may not be available >>> +???????? } >>> +???????? this.lb = temp; >>> ??????? } >> Thanks for the tip. The PerfCounter changes were the last thing I >> dropped in >> after adding the IOException and fixing the places where things did >> not compile. >> >> Fundamentally, what sort of things will break, when the previous >> interface >> did not expose the possibility that the perf memory was not initialized >> as assumed in the earlier code. Does the upper level code "have to" >> be aware, or is the contract to do a "best effort" if something lower >> down >> is not as expected. >> >>> Hi Gary, >>> >>> Meta-question partially raised in the the recent bug report >>> comments: do >>> we care about this any more? Does anything need to disable UsePerfData? >>> Can we not deprecate it in 11 then obsolete in 12? That would be a lot >>> simpler. >> My own bias is that many of the motivations for faster startup and >> smaller footprint >> are still useful goals. If not for embedded systems, then for >> container environments. >> A process started with -XX:-UsePerfData does not support external attach >> and does not maintain the internal variables for the self attach. >> >>> >>> That aside ... >>> >>> It's hard to see how all the different paths in these related API's >>> stitch together - especially when you have to consider the API being >>> used on the current VM or a different one. I'm not at all clear what >>> the >>> control flow is, so it is hard to see exactly how the initialization >>> issue is being handled. As I wrote in the initial report it is unclear >>> exactly which parts of the "perf" subsystem are impacted by the >>> value of >>> UsePerfData. >> Starting with the Perf.java class initialization, I added the >> exposure of >> the PerfMemory::is_initialized(). >> >> Perf.java: >> >> 559 private native boolean isInitialized(); >> 560 >> ? 561???? private static native void registerNatives(); >> ? 562 >> ? 563???? static { >> ? 564???????? registerNatives(); >> ? 565???????? instance = new Perf(); >> 566 initialized = instance.isInitialized(); >> ? 567???? } >> >> perf.cpp: >> 298 PERF_ENTRY(jboolean, Perf_IsInitialized(JNIEnv *env, jobject perf)) >> 299 >> 300 PerfWrapper("Perf_IsInitialized"); >> 301 >> 302 return PerfMemory::is_initialized(); >> 303 >> 304 PERF_END >> >> >> ?From the VM perspective UsePerfData is both a command line argument >> and a flag to be used whenever that information is needed. There are >> scattered checks to deal with setup and shutdown, but also the runtime >> fulfillment of the PerfMemory data structures. >> >> Looking at perfMemory_init() and perfMemory_exit(), I concluded that >> the UsePerfData flag essentially noops all of the perfMemory >> allocation and >> updating. The only information required in the upper layers of the >> Perf.java >> code is the fact that the perf memory is not initialized. >> >> perfMemory.cpp: >> >> ???? 58??? void perfMemory_init() { >> ???? 59 >> ???? 60????? if (!UsePerfData) return; >> ???? 61 >> ???? 62????? PerfMemory::initialize(); >> ???? 63??? } >> ???? 64 >> ???? 65??? void perfMemory_exit() { >> ???? 66 >> ???? 67????? if (!UsePerfData) return; >> ???? 68????? if (!PerfMemory::is_usable()) return; >> ???? 69 >> ... >> ???? 90??? void PerfMemory::initialize() { >> ... >> ??? 159????? OrderAccess::release_store(&_initialized, 1); >> ??? 160??? } >> ? ... >> ??? 271??? bool PerfMemory::is_initialized() { >> ??? 272????? return OrderAccess::load_acquire(&_initialized) != 0; >> ??? 273??? } >> >>> >>> Can you summarize the role of the is_initialized check versus a direct >>> UsePerfData check on the VM side i.e. which should be checked when? I >>> immediately see an issue that Perf_Detach is a no-op when !UsePerfdata, >>> yet Perf_Attach doesn't examine the flag at all! But perhaps we never >>> reach those methods because of a higher-level call filtering it out ?? >> There is split path when attaching to self versus attaching to other >> running VM processes. The extra detach processing is only needed >> for the remote VM case. If attach fails, there is no need to detach. >> >> Perf.java: >> ... >> >> ? 274???? private ByteBuffer attachImpl(String user, int lvmid, int >> mode) >> ? 275???????????? throws IllegalArgumentException, IOException >> ? 276???? { >> ? 277???????? final ByteBuffer b = attach(user, lvmid, mode); >> ? 278 >> ? 279???????? if (lvmid == 0) { >> ? 280???????????? // The native instrumentation buffer for this Java >> virtual >> ? 281???????????? // machine is never unmapped. >> ? 282???????????? return b; >> ? 283???????? } >> ? 284???????? else { >> ? 285???????????? // This is an instrumentation buffer for another >> Java virtual >> ? 286???????????? // machine with native resources that need to be >> managed. We >> ? 287???????????? // create a duplicate of the native ByteBuffer and >> manage it >> ? 288???????????? // with a Cleaner. When the duplicate becomes >> phantom reachable, >> ? 289???????????? // the native resources will be released. >> ? 290 >> ? 291???????????? final ByteBuffer dup = b.duplicate(); >> ? 292 >> ? 293???????????? CleanerFactory.cleaner() >> ? 294?????????????????????????? .register(dup, new >> CleanerAction(instance, b)); >> ? 295???????????? return dup; >> ? 296???????? } >> ? 297???? } >> >> >> >>> >>> More below ... >> >>> >>> David >>> >>> On 1/06/2018 5:09 AM, Gary Adams wrote: >>> >/A patch was done previously to prevent an error when >>> -XX:-UsePerfData />/is used, but this bug was filed to make the >>> setting more visible in the />/jdk.internal.perf package. />//>/?? >>> Webrev: http://cr.openjdk.java.net/~gadams/8069149/webrev.00/ >>> />/?? >>> Issue: https://bugs.openjdk.java.net/browse/JDK-8069149 />//>/I have >>> tried a few initial tests using: />//>/?? make run-tests \ />/ >>> TEST_OPTS=VM_OPTIONS=-XX:-UsePerfData \ />/ >>> TEST=open/test/jdk/sun/management/jmxremote />//>/While I'm tracking >>> down one test timeout,? I'd like to get some />/feedback on the >>> approach used here : />//>/?? - the basic mechanism propagates the >>> "is initialized" state up to />/Perf.java; />/????? does the state >>> need to be exposed to users getPerf(), or is it />/sufficient/ >> >>> Within PerfMemory UsePerfData and is_usable() are used for guarding >>> execution. Perhaps that is what needs to be propagated up? >> >> /The is_usable() state is used to ensure the memory is initialized >> and not yet detached. We don't need that specific condition here. > >> /Unclear what the control flow is here. I don't see that Perf triggers >>> initialization of the perf subsystem, so what guarantees >>> initialization should have happened by the time the Perf class is >>> initialized ?? >> >> The private constructor and the static initializer in Perf ensures >> the singleton behavior at class initialization. The perfMemory_init() >> is handled during VM initialization. >> >> init.cpp: >> >> ???? 90??? void vm_init_globals() { >> ???? 91????? check_ThreadShadow(); >> ???? 92????? basic_types_init(); >> ???? 93????? eventlog_init(); >> ???? 94????? mutex_init(); >> ???? 95????? chunkpool_init(); >> ???? 96????? perfMemory_init(); >> ???? 97????? SuspendibleThreadSet_init(); >> ???? 98??? } >> >> >>> >/ ?? - an existing use of IOException was used in the case of >>> attach /> >/failures; /> >/ ????? added IOExceptions to the >>> createXXX methods if the memory was not /> >/initialized / >>> On the VM side can you at least add a message to the IOException so >>> that the user can tell why it was thrown. >> >> Initially, I included a message with the IOException, and noticed the >> other >> exceptions being thrown in the surrounding code did not include a >> message, >> so opted for the empty message. There is no ambiguity here. >> >>> >/ ?? - presuming a CSR would be needed to cover the IOExceptions / >>> It's not a public API so no CSR should be needed. >> >> Good to know. >> >>> >/ ?? - it appears that jdk.internal.perf has very limited usage, /> >>> >/ ????? is it still needed/used (?) / >>> I can't answer that part, but I don't think UsePerfData is needed >>> any more. >> >> If this turns into "removal of UsePerfData" support, we should move >> the bug >> to the runtime group. I grabbed the bug initially, because it looked >> like >> a loose end and a simple thing to address by fulfilling the original >> request to propagate the visibility up to the Perf.java code. >> >> Let's see if other reviewers have similar views - that the UsePerfData >> flag can be removed. This bug was originally filed against 8u60 and is >> currently targetted for 12. I only explored a potential fix at this time >> in case we need to get a deprecation process started at this time. >> >> ... From gunter.haug at sap.com Mon Jun 4 15:27:33 2018 From: gunter.haug at sap.com (Haug, Gunter) Date: Mon, 4 Jun 2018 15:27:33 +0000 Subject: RFR(S): 8200720: Print additional information in thread dump (times, allocated bytes etc.) In-Reply-To: <55806aa8-f09b-e9a8-b512-b9676029c494@oracle.com> References: <56817AB5-88FC-4FB9-A271-7206EA69ECA7@sap.com> <2728581E-0E32-4911-B1A9-79AD6CBA81B0@sap.com> <082ca98f-9a68-15a0-95b5-c4d5c733ab36@oracle.com> <933cdacc-9b6f-c8b5-e244-558f38e7888c@oracle.com> <55806aa8-f09b-e9a8-b512-b9676029c494@oracle.com> Message-ID: <0F497BC6-2569-4EF6-9B59-E24A8E1B3A37@sap.com> Hi David, Chris, would it be an option to unconditionally print the additional information? Regardless which way this is implemented it will be rather complicated and it only switches on/off a few field in the thread dump. Thanks, Gunter ?On 04.06.18, 01:13, "David Holmes" wrote: Hi Gunter, Chris, Sorry just trying to catch up and this is only a partial look so far ... BTW these changes are not limited to serviceability code so adding in runtime team as well. On 2/06/2018 9:12 AM, Chris Plummer wrote: > Hi Gunter, > > On 6/1/18 3:17 AM, Haug, Gunter wrote: >> Hi Chris, >> >> thanks for looking into this! >> >> Re the synchronization: The value is stored only in a VM operation at >> a safepoint by the VM thread. I was of the opinion, that this could >> not be interrupted by a second VM operation (of the same type). Or am >> I missing something else? > I was really thinking more about the temporary changing of > PrintExtendedThreadInfo, not the value stored in the VMOp. You may be be > correct that no more than one VMOp is executing, but while it is > executing it is has changed the value of PrintExtendedThreadInfo, which > might have an impact on anything else that triggers printing thread info > while the VMOp is executing. Even if nothing else can possibly be running during the safepoint I find it extremely bad form to change a command-line flag in this way, particularly from a safepoint! If this flag is intended to be dynamically enabled as part of a dcmd then it should also be a Manageable flag IMHO. That said ... >> I did think about passing an argument to the various print_on member >> functions of the thread classes, but this would require rather >> extensive code changes for a rather tiny extension. Therefore I feel >> doing it like this is the lesser evil. ... it's obviously not truly a global setting, but one that is needed on a per-print-request basis. The flag is just the default, but if the default is off you still want to enable extended printing on a per-request basis. The current print_on mechanics is not set up for this kind of flexibility. I think this needs more thought. --- Re osThread.cpp shared code changes ... I really hate to see all the ifdefs in there. Please add a pd_print_on function to support this if you truly want platform specific information. --- threadStatisticInfo.hpp typo: getElepsedTime() -> getElapsedTime() Thanks, David >> >> Re thread_dump(): I think it's correct or, well, at least it works ;-) >> In fact jstack will transfer the "-e" and "-l" in only one string, >> i.e. op->arg(0). > So you are saying that op->arg(0) is "-e -l" when using jstack? I think > you really need to clean up the parsing. As it stands right now passing, > I get the feeling that if a user erroneously asks for help by using > "jcmd Thread.Print -help", it will end up executing with -e an -l > enabled, and no failure for the invalid "-help" option. >> Christoph has already explained my reasoning. But I agree, it's not >> nice and I would be happy to do it like Christoph suggested. > I'll respond separately to his suggestion. > > thanks, > > Chris >> >> And typo fixed, sorry. >> >> Thanks again, >> Gunter >> >> On 01.06.18, 00:03, "Chris Plummer" wrote: >> >> Hi Gunter, >> globals.hpp: fix typo "informatiuon" >> I worry a little bit about the synchronizing (if that's the right >> word) >> of PrintExtendedThreadInfo and the dcmd's -e flag. When using -e, >> you >> are temporarily enabling PrintExtendedThreadInfo if it was false. >> This >> temporarily changes the behavior of thread dumps, and could >> impact other >> uses that happen in parallel. Also, could two simultaneous uses >> of -e >> result in PrintExtendedThreadInfo not getting restored properly? >> thread_dump() doesn't look right. It looks like you are iterating >> char >> by char over the argument, and expect something like "-el" to be >> specified rather then "-e -l". The loop should be iterating over >> op->arg(i), not op->arg(0)[i]. >> The rest of the changes look fine. >> thanks, >> Chris >> On 5/30/18 8:12 AM, Haug, Gunter wrote: >> > Hi all, >> > >> > As Chris proposed, I have made an the extended output >> switchable. There is an VM flag (PrintExtendedThreadInfo), which is >> false by default. Moreover, there is an Option (-e) which can be used >> with jcmd Thread.print as well as with jstack. The -e option >> essentially sets PrintExtendedThreadInfo true just for the respective >> thread dump. >> > >> > Here is the updated webrev: >> > >> > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v2 >> > >> > (https://bugs.openjdk.java.net/browse/JDK-8200720) >> > >> > Thanks, >> > Gunter >> > >> > >> > On 02.05.18, 17:07, "serviceability-dev on behalf of Haug, >> Gunter" > gunter.haug at sap.com> wrote: >> > >> > Hi Chris, >> > >> > Thanks for looking into this. >> > You're right, there is a little more we have. We have >> implemented an IO tracing mechanism which - rather as a byproduct - >> keeps track of bytes read and written per thread. However, this of >> course requires changes not only in hotspot. We would be happy to >> contribute this as well, but this is a far bigger change and will >> probably lead to a far bigger discussion. Anyway, with the number of >> bytes read, the number of classes defined doesn't look that arbitrary >> anymore, as one can correlate IO to class loading. >> > >> > Regarding the verbose option I think that's a good idea! >> > >> > Thanks again, >> > Gunter >> > >> > On 01.05.18, 22:55, "Chris Plummer" >> wrote: >> > >> > Hi Gunter, >> > >> > The output you are adding is all useful. I think the >> question is (and >> > I'd like to see a few people chime in on this for this >> review) is >> > whether or not all of it is the appropriate for a >> thread's stack dump. >> > For example, defined_classes is on the fringe of what >> I would call >> > generally useful info in a stack dump. Sure, there >> might be that rare >> > case when you need it, but how often compared to other >> useful info >> > maintained on a per thread basis. How many other bits >> of useful info are >> > not being printed in favor of defined_classes? It >> seems you have more in >> > the queue. How many? I'm worried about how cluttered >> the stack dumps >> > will get. Maybe we should add some sort of verbose >> thread dumping >> > option. Just a thought. >> > >> > As for the implementation, overall it looks good, but >> I can't speak to >> > whether or not you are doing proper accounting of >> defined_classes and >> > bytes allocated. You'll need input from someone with >> more knowledge of >> > those areas. We'll also need to do some testing to >> make sure tool tests >> > are not impacted. >> > >> > thanks, >> > >> > Chris >> > >> > On 4/30/18 2:51 AM, Haug, Gunter wrote: >> > > Hi, >> > > >> > > this is an update to an RFR I posted on hotspot-dev, >> but it is probably more suitable to post it here. Can I please have a >> review and a sponsor for the following enhancement: >> > > >> > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v1 >> > > https://bugs.openjdk.java.net/browse/JDK-8200720 >> > > >> > > We at SAP have extended the thread dumps (obtained >> by jstack or jcmd) by several fields providing thread specific >> information. These extensions are quite popular with our support team. >> With some knowledge of the architecture of the application, they often >> allow for quick and simple diagnosis of a running system. Therefore we >> would like to contribute these enhancements. >> > > >> > > I took a few simple examples here, namely cpu time, >> elapsed time since thread creation, bytes allocated and classes >> defined by the thread and the pthread-id or equivalent on platforms >> where it makes sense. Provided it is known how the application should >> behave, a misbehaving thread can identified easily. >> > > >> > > There is no measurable overhead for this >> enhancement. However, it may be a problem that the format of the >> output is changed. Tools parsing the output may have to be changed. >> > > >> > > Here is an example of the output generated: >> > > >> > > ------------------------------------------------------ >> > > >> > > "main" #1 prio=5 os_prio=31 cpu=6300.65ms >> elapsed=123.28s allocated=242236760B defined_classes=1725 >> tid=0x00007fa13a806000 nid=0x1c03 pthread-id=0x109708000 waiting on >> condition [0x0000000109707000] >> > > java.lang.Thread.State: TIMED_WAITING (sleeping) >> > > JavaThread state: _thread_blocked >> > > Thread: 0x00007fa13a806000 [0x1c03] State: >> _at_safepoint _has_called_back 0 _at_poll_safepoint 0 >> > > JavaThread state: _thread_blocked >> > > at java.lang.Thread.sleep(java.base/Native Method) >> > > ... >> > > ------------------------------------------------------ >> > > >> > > As mentioned above, we have a whole bunch of other >> enhancements to the thread dump similar to this one and would be >> willing to contribute them if there is any interest. >> > > >> > > Thanks and best regards, >> > > Gunter >> > > >> > > >> > >> > >> > >> > >> > >> > > From chris.plummer at oracle.com Mon Jun 4 18:24:38 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 4 Jun 2018 11:24:38 -0700 Subject: RFR(S): 8200720: Print additional information in thread dump (times, allocated bytes etc.) In-Reply-To: <0F497BC6-2569-4EF6-9B59-E24A8E1B3A37@sap.com> References: <56817AB5-88FC-4FB9-A271-7206EA69ECA7@sap.com> <2728581E-0E32-4911-B1A9-79AD6CBA81B0@sap.com> <082ca98f-9a68-15a0-95b5-c4d5c733ab36@oracle.com> <933cdacc-9b6f-c8b5-e244-558f38e7888c@oracle.com> <55806aa8-f09b-e9a8-b512-b9676029c494@oracle.com> <0F497BC6-2569-4EF6-9B59-E24A8E1B3A37@sap.com> Message-ID: Hi Gunter, Just restating what I said with my initial review: > The output you are adding is all useful. I think the question is (and > I'd like to see a few people chime in on this for this review) is > whether or not all of it is the appropriate for a thread's stack dump. > For example, defined_classes is on the fringe of what I would call > generally useful info in a stack dump. Sure, there might be that rare > case when you need it, but how often compared to other useful info > maintained on a per thread basis. How many other bits of useful info > are not being printed in favor of defined_classes? It seems you have > more in the queue. How many? I'm worried about how cluttered the stack > dumps will get. Maybe we should add some sort of verbose thread > dumping option. Just a thought. Nothing has changed w.r.t. my opinions above. I would still like to know what others think about the added details. thanks, Chris On 6/4/18 8:27 AM, Haug, Gunter wrote: > Hi David, Chris, > > would it be an option to unconditionally print the additional information? Regardless which way this is implemented it will be rather complicated and it only switches on/off a few field in the thread dump. > > Thanks, > Gunter > > ?On 04.06.18, 01:13, "David Holmes" wrote: > > Hi Gunter, Chris, > > Sorry just trying to catch up and this is only a partial look so far ... > > BTW these changes are not limited to serviceability code so adding in > runtime team as well. > > On 2/06/2018 9:12 AM, Chris Plummer wrote: > > Hi Gunter, > > > > On 6/1/18 3:17 AM, Haug, Gunter wrote: > >> Hi Chris, > >> > >> thanks for looking into this! > >> > >> Re the synchronization: The value is stored only in a VM operation at > >> a safepoint by the VM thread. I was of the opinion, that this could > >> not be interrupted by a second VM operation (of the same type). Or am > >> I missing something else? > > I was really thinking more about the temporary changing of > > PrintExtendedThreadInfo, not the value stored in the VMOp. You may be be > > correct that no more than one VMOp is executing, but while it is > > executing it is has changed the value of PrintExtendedThreadInfo, which > > might have an impact on anything else that triggers printing thread info > > while the VMOp is executing. > > Even if nothing else can possibly be running during the safepoint I find > it extremely bad form to change a command-line flag in this way, > particularly from a safepoint! > > If this flag is intended to be dynamically enabled as part of a dcmd > then it should also be a Manageable flag IMHO. > > That said ... > > >> I did think about passing an argument to the various print_on member > >> functions of the thread classes, but this would require rather > >> extensive code changes for a rather tiny extension. Therefore I feel > >> doing it like this is the lesser evil. > > ... it's obviously not truly a global setting, but one that is needed on > a per-print-request basis. The flag is just the default, but if the > default is off you still want to enable extended printing on a > per-request basis. > > The current print_on mechanics is not set up for this kind of > flexibility. I think this needs more thought. > > --- > > Re osThread.cpp shared code changes ... I really hate to see all the > ifdefs in there. Please add a pd_print_on function to support this if > you truly want platform specific information. > > --- > > threadStatisticInfo.hpp > > typo: getElepsedTime() -> getElapsedTime() > > Thanks, > David > > > > >> > >> Re thread_dump(): I think it's correct or, well, at least it works ;-) > >> In fact jstack will transfer the "-e" and "-l" in only one string, > >> i.e. op->arg(0). > > So you are saying that op->arg(0) is "-e -l" when using jstack? I think > > you really need to clean up the parsing. As it stands right now passing, > > I get the feeling that if a user erroneously asks for help by using > > "jcmd Thread.Print -help", it will end up executing with -e an -l > > enabled, and no failure for the invalid "-help" option. > >> Christoph has already explained my reasoning. But I agree, it's not > >> nice and I would be happy to do it like Christoph suggested. > > I'll respond separately to his suggestion. > > > > thanks, > > > > Chris > >> > >> And typo fixed, sorry. > >> > >> Thanks again, > >> Gunter > >> > >> On 01.06.18, 00:03, "Chris Plummer" wrote: > >> > >> Hi Gunter, > >> globals.hpp: fix typo "informatiuon" > >> I worry a little bit about the synchronizing (if that's the right > >> word) > >> of PrintExtendedThreadInfo and the dcmd's -e flag. When using -e, > >> you > >> are temporarily enabling PrintExtendedThreadInfo if it was false. > >> This > >> temporarily changes the behavior of thread dumps, and could > >> impact other > >> uses that happen in parallel. Also, could two simultaneous uses > >> of -e > >> result in PrintExtendedThreadInfo not getting restored properly? > >> thread_dump() doesn't look right. It looks like you are iterating > >> char > >> by char over the argument, and expect something like "-el" to be > >> specified rather then "-e -l". The loop should be iterating over > >> op->arg(i), not op->arg(0)[i]. > >> The rest of the changes look fine. > >> thanks, > >> Chris > >> On 5/30/18 8:12 AM, Haug, Gunter wrote: > >> > Hi all, > >> > > >> > As Chris proposed, I have made an the extended output > >> switchable. There is an VM flag (PrintExtendedThreadInfo), which is > >> false by default. Moreover, there is an Option (-e) which can be used > >> with jcmd Thread.print as well as with jstack. The -e option > >> essentially sets PrintExtendedThreadInfo true just for the respective > >> thread dump. > >> > > >> > Here is the updated webrev: > >> > > >> > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v2 > >> > > >> > (https://bugs.openjdk.java.net/browse/JDK-8200720) > >> > > >> > Thanks, > >> > Gunter > >> > > >> > > >> > On 02.05.18, 17:07, "serviceability-dev on behalf of Haug, > >> Gunter" >> gunter.haug at sap.com> wrote: > >> > > >> > Hi Chris, > >> > > >> > Thanks for looking into this. > >> > You're right, there is a little more we have. We have > >> implemented an IO tracing mechanism which - rather as a byproduct - > >> keeps track of bytes read and written per thread. However, this of > >> course requires changes not only in hotspot. We would be happy to > >> contribute this as well, but this is a far bigger change and will > >> probably lead to a far bigger discussion. Anyway, with the number of > >> bytes read, the number of classes defined doesn't look that arbitrary > >> anymore, as one can correlate IO to class loading. > >> > > >> > Regarding the verbose option I think that's a good idea! > >> > > >> > Thanks again, > >> > Gunter > >> > > >> > On 01.05.18, 22:55, "Chris Plummer" > >> wrote: > >> > > >> > Hi Gunter, > >> > > >> > The output you are adding is all useful. I think the > >> question is (and > >> > I'd like to see a few people chime in on this for this > >> review) is > >> > whether or not all of it is the appropriate for a > >> thread's stack dump. > >> > For example, defined_classes is on the fringe of what > >> I would call > >> > generally useful info in a stack dump. Sure, there > >> might be that rare > >> > case when you need it, but how often compared to other > >> useful info > >> > maintained on a per thread basis. How many other bits > >> of useful info are > >> > not being printed in favor of defined_classes? It > >> seems you have more in > >> > the queue. How many? I'm worried about how cluttered > >> the stack dumps > >> > will get. Maybe we should add some sort of verbose > >> thread dumping > >> > option. Just a thought. > >> > > >> > As for the implementation, overall it looks good, but > >> I can't speak to > >> > whether or not you are doing proper accounting of > >> defined_classes and > >> > bytes allocated. You'll need input from someone with > >> more knowledge of > >> > those areas. We'll also need to do some testing to > >> make sure tool tests > >> > are not impacted. > >> > > >> > thanks, > >> > > >> > Chris > >> > > >> > On 4/30/18 2:51 AM, Haug, Gunter wrote: > >> > > Hi, > >> > > > >> > > this is an update to an RFR I posted on hotspot-dev, > >> but it is probably more suitable to post it here. Can I please have a > >> review and a sponsor for the following enhancement: > >> > > > >> > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v1 > >> > > https://bugs.openjdk.java.net/browse/JDK-8200720 > >> > > > >> > > We at SAP have extended the thread dumps (obtained > >> by jstack or jcmd) by several fields providing thread specific > >> information. These extensions are quite popular with our support team. > >> With some knowledge of the architecture of the application, they often > >> allow for quick and simple diagnosis of a running system. Therefore we > >> would like to contribute these enhancements. > >> > > > >> > > I took a few simple examples here, namely cpu time, > >> elapsed time since thread creation, bytes allocated and classes > >> defined by the thread and the pthread-id or equivalent on platforms > >> where it makes sense. Provided it is known how the application should > >> behave, a misbehaving thread can identified easily. > >> > > > >> > > There is no measurable overhead for this > >> enhancement. However, it may be a problem that the format of the > >> output is changed. Tools parsing the output may have to be changed. > >> > > > >> > > Here is an example of the output generated: > >> > > > >> > > ------------------------------------------------------ > >> > > > >> > > "main" #1 prio=5 os_prio=31 cpu=6300.65ms > >> elapsed=123.28s allocated=242236760B defined_classes=1725 > >> tid=0x00007fa13a806000 nid=0x1c03 pthread-id=0x109708000 waiting on > >> condition [0x0000000109707000] > >> > > java.lang.Thread.State: TIMED_WAITING (sleeping) > >> > > JavaThread state: _thread_blocked > >> > > Thread: 0x00007fa13a806000 [0x1c03] State: > >> _at_safepoint _has_called_back 0 _at_poll_safepoint 0 > >> > > JavaThread state: _thread_blocked > >> > > at java.lang.Thread.sleep(java.base/Native Method) > >> > > ... > >> > > ------------------------------------------------------ > >> > > > >> > > As mentioned above, we have a whole bunch of other > >> enhancements to the thread dump similar to this one and would be > >> willing to contribute them if there is any interest. > >> > > > >> > > Thanks and best regards, > >> > > Gunter > >> > > > >> > > > >> > > >> > > >> > > >> > > >> > > >> > > > > > > From kirk at kodewerk.com Mon Jun 4 18:46:09 2018 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Mon, 4 Jun 2018 20:46:09 +0200 Subject: RFR(S): 8200720: Print additional information in thread dump (times, allocated bytes etc.) In-Reply-To: References: <56817AB5-88FC-4FB9-A271-7206EA69ECA7@sap.com> <2728581E-0E32-4911-B1A9-79AD6CBA81B0@sap.com> <082ca98f-9a68-15a0-95b5-c4d5c733ab36@oracle.com> <933cdacc-9b6f-c8b5-e244-558f38e7888c@oracle.com> <55806aa8-f09b-e9a8-b512-b9676029c494@oracle.com> <0F497BC6-2569-4EF6-9B59-E24A8E1B3A37@sap.com> Message-ID: <3EF7809E-5E55-4A79-B2EB-CEC45F893639@kodewerk.com> > On Jun 4, 2018, at 8:24 PM, Chris Plummer wrote: > > Hi Gunter, > > Just restating what I said with my initial review: > >> The output you are adding is all useful. I think the question is (and I'd like to see a few people chime in on this for this review) is whether or not all of it is the appropriate for a thread's stack dump. For example, defined_classes is on the fringe of what I would call generally useful info in a stack dump. Sure, there might be that rare case when you need it, but how often compared to other useful info maintained on a per thread basis. How many other bits of useful info are not being printed in favor of defined_classes? It seems you have more in the queue. How many? I'm worried about how cluttered the stack dumps will get. Maybe we should add some sort of verbose thread dumping option. Just a thought. > Nothing has changed w.r.t. my opinions above. I would still like to know what others think about the added details. Thread dumps are a some what useful diagnostic tool.. what reduces their usefulness is the expense of acquiring them. I?d be hesitant to add to the cost. That said, this information can be useful that I fail to see how in the context of a thread dump. I?m wondering if a new diagnostic command would be in order to capture some of this information. Kind regards, Kirk From jesper.wilhelmsson at oracle.com Tue Jun 5 00:09:23 2018 From: jesper.wilhelmsson at oracle.com (jesper.wilhelmsson at oracle.com) Date: Tue, 5 Jun 2018 02:09:23 +0200 Subject: RFR(XXS): 8204103: Mark test serviceability/dcmd/compiler/CompilerQueueTest.java as intermittent and exclude it from tier1 In-Reply-To: <410FD58C-C8CB-4254-B7BB-FC712BC8D28E@oracle.com> References: <482434D6-4374-4E04-890F-D6DCB005FF20@oracle.com> <410FD58C-C8CB-4254-B7BB-FC712BC8D28E@oracle.com> Message-ID: Looks good. I'll sponsor this change. /Jesper > On 31 May 2018, at 19:59, Leonid Mesnik wrote: > > Thank you for review. > > It becomes a part of hotspot_tier2_runtime now and executed in tier2. (Also in tier3 and higher tiers with other options.) > > Leonid > >> On May 31, 2018, at 10:47 AM, Vladimir Kozlov wrote: >> >> Looks fine. In what tier the test will be run? >> >> Thanks, >> Vladimir >> >> On 5/31/18 10:11 AM, Leonid Mesnik wrote: >>> Hi >>> Please review following tiny fix which marks test serviceability/dcmd/compiler/CompilerQueueTest.java as intermittent and exclude it from tier1. >>> The test intermittently fails (1-2 times in 1000 runs) because of https://bugs.openjdk.java.net/browse/JDK-8158374. >>> Tier1 should be stable and don't contain any tests with known issues. It should always pass if there are no regression introduced by new fix. So this test should be excluded from tier1. >>> Also fix marks this test with corresponding key like it is done for jdk, langtoools and nashorn tests. >>> webrev: http://cr.openjdk.java.net/~lmesnik/8204103/ >>> bug: https://bugs.openjdk.java.net/browse/JDK-8204103 >>> I verified that tier1 runs and doesn't contain this test now. >>> See more information about intermittent keyword in https://bugs.openjdk.java.net/browse/JDK-8075565 >>> Leonid > From leonid.mesnik at oracle.com Tue Jun 5 01:02:21 2018 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Mon, 4 Jun 2018 18:02:21 -0700 Subject: RFR(XXS): 8204103: Mark test serviceability/dcmd/compiler/CompilerQueueTest.java as intermittent and exclude it from tier1 In-Reply-To: References: <482434D6-4374-4E04-890F-D6DCB005FF20@oracle.com> <410FD58C-C8CB-4254-B7BB-FC712BC8D28E@oracle.com> Message-ID: <975CE5EB-F26A-457A-824B-FA436293D103@oracle.com> Thank you. Leonid > On Jun 4, 2018, at 5:09 PM, jesper.wilhelmsson at oracle.com wrote: > > Looks good. > I'll sponsor this change. > /Jesper > >> On 31 May 2018, at 19:59, Leonid Mesnik wrote: >> >> Thank you for review. >> >> It becomes a part of hotspot_tier2_runtime now and executed in tier2. (Also in tier3 and higher tiers with other options.) >> >> Leonid >> >>> On May 31, 2018, at 10:47 AM, Vladimir Kozlov wrote: >>> >>> Looks fine. In what tier the test will be run? >>> >>> Thanks, >>> Vladimir >>> >>> On 5/31/18 10:11 AM, Leonid Mesnik wrote: >>>> Hi >>>> Please review following tiny fix which marks test serviceability/dcmd/compiler/CompilerQueueTest.java as intermittent and exclude it from tier1. >>>> The test intermittently fails (1-2 times in 1000 runs) because of https://bugs.openjdk.java.net/browse/JDK-8158374. >>>> Tier1 should be stable and don't contain any tests with known issues. It should always pass if there are no regression introduced by new fix. So this test should be excluded from tier1. >>>> Also fix marks this test with corresponding key like it is done for jdk, langtoools and nashorn tests. >>>> webrev: http://cr.openjdk.java.net/~lmesnik/8204103/ >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8204103 >>>> I verified that tier1 runs and doesn't contain this test now. >>>> See more information about intermittent keyword in https://bugs.openjdk.java.net/browse/JDK-8075565 >>>> Leonid >> > From david.holmes at oracle.com Tue Jun 5 02:53:05 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 5 Jun 2018 12:53:05 +1000 Subject: RFR(S): 8200720: Print additional information in thread dump (times, allocated bytes etc.) In-Reply-To: <0F497BC6-2569-4EF6-9B59-E24A8E1B3A37@sap.com> References: <56817AB5-88FC-4FB9-A271-7206EA69ECA7@sap.com> <2728581E-0E32-4911-B1A9-79AD6CBA81B0@sap.com> <082ca98f-9a68-15a0-95b5-c4d5c733ab36@oracle.com> <933cdacc-9b6f-c8b5-e244-558f38e7888c@oracle.com> <55806aa8-f09b-e9a8-b512-b9676029c494@oracle.com> <0F497BC6-2569-4EF6-9B59-E24A8E1B3A37@sap.com> Message-ID: Hi Gunter, On 5/06/2018 1:27 AM, Haug, Gunter wrote: > Hi David, Chris, > > would it be an option to unconditionally print the additional information? Regardless which way this is implemented it will be rather complicated and it only switches on/off a few field in the thread dump. I'm not convinced this is all suitable information for a thread stack dump. I can see the time information being useful if using the dump to try and diagnoze a hang or responsiveness issue. But the allocated-bytes and classes-defined just doesn't seem useful in the context of a thread dump to me. Anyone interested in allocation stats is going to be looking at GC logs etc which is where this belongs. Ditto the class-stats belong in some kind of classloading logging IMHO. I'm very reluctant to burden the implementation with capturing these kinds of stats, just to use for diagnostic purposes that may not even be asked for. I'm very much in the "you shouldn't pay for what you don't use" camp in that regard. (See my comments in JDK-8143176 referenced by Sean.) The ThreadStatisticInfo adds overhead to every thread object - and I'd have to suspect there could be overlap with whatever flight recorder sticks in there too. (Thread has become even more bloated in recent time!). Cheers, David > Thanks, > Gunter > > ?On 04.06.18, 01:13, "David Holmes" wrote: > > Hi Gunter, Chris, > > Sorry just trying to catch up and this is only a partial look so far ... > > BTW these changes are not limited to serviceability code so adding in > runtime team as well. > > On 2/06/2018 9:12 AM, Chris Plummer wrote: > > Hi Gunter, > > > > On 6/1/18 3:17 AM, Haug, Gunter wrote: > >> Hi Chris, > >> > >> thanks for looking into this! > >> > >> Re the synchronization: The value is stored only in a VM operation at > >> a safepoint by the VM thread. I was of the opinion, that this could > >> not be interrupted by a second VM operation (of the same type). Or am > >> I missing something else? > > I was really thinking more about the temporary changing of > > PrintExtendedThreadInfo, not the value stored in the VMOp. You may be be > > correct that no more than one VMOp is executing, but while it is > > executing it is has changed the value of PrintExtendedThreadInfo, which > > might have an impact on anything else that triggers printing thread info > > while the VMOp is executing. > > Even if nothing else can possibly be running during the safepoint I find > it extremely bad form to change a command-line flag in this way, > particularly from a safepoint! > > If this flag is intended to be dynamically enabled as part of a dcmd > then it should also be a Manageable flag IMHO. > > That said ... > > >> I did think about passing an argument to the various print_on member > >> functions of the thread classes, but this would require rather > >> extensive code changes for a rather tiny extension. Therefore I feel > >> doing it like this is the lesser evil. > > ... it's obviously not truly a global setting, but one that is needed on > a per-print-request basis. The flag is just the default, but if the > default is off you still want to enable extended printing on a > per-request basis. > > The current print_on mechanics is not set up for this kind of > flexibility. I think this needs more thought. > > --- > > Re osThread.cpp shared code changes ... I really hate to see all the > ifdefs in there. Please add a pd_print_on function to support this if > you truly want platform specific information. > > --- > > threadStatisticInfo.hpp > > typo: getElepsedTime() -> getElapsedTime() > > Thanks, > David > > > > >> > >> Re thread_dump(): I think it's correct or, well, at least it works ;-) > >> In fact jstack will transfer the "-e" and "-l" in only one string, > >> i.e. op->arg(0). > > So you are saying that op->arg(0) is "-e -l" when using jstack? I think > > you really need to clean up the parsing. As it stands right now passing, > > I get the feeling that if a user erroneously asks for help by using > > "jcmd Thread.Print -help", it will end up executing with -e an -l > > enabled, and no failure for the invalid "-help" option. > >> Christoph has already explained my reasoning. But I agree, it's not > >> nice and I would be happy to do it like Christoph suggested. > > I'll respond separately to his suggestion. > > > > thanks, > > > > Chris > >> > >> And typo fixed, sorry. > >> > >> Thanks again, > >> Gunter > >> > >> On 01.06.18, 00:03, "Chris Plummer" wrote: > >> > >> Hi Gunter, > >> globals.hpp: fix typo "informatiuon" > >> I worry a little bit about the synchronizing (if that's the right > >> word) > >> of PrintExtendedThreadInfo and the dcmd's -e flag. When using -e, > >> you > >> are temporarily enabling PrintExtendedThreadInfo if it was false. > >> This > >> temporarily changes the behavior of thread dumps, and could > >> impact other > >> uses that happen in parallel. Also, could two simultaneous uses > >> of -e > >> result in PrintExtendedThreadInfo not getting restored properly? > >> thread_dump() doesn't look right. It looks like you are iterating > >> char > >> by char over the argument, and expect something like "-el" to be > >> specified rather then "-e -l". The loop should be iterating over > >> op->arg(i), not op->arg(0)[i]. > >> The rest of the changes look fine. > >> thanks, > >> Chris > >> On 5/30/18 8:12 AM, Haug, Gunter wrote: > >> > Hi all, > >> > > >> > As Chris proposed, I have made an the extended output > >> switchable. There is an VM flag (PrintExtendedThreadInfo), which is > >> false by default. Moreover, there is an Option (-e) which can be used > >> with jcmd Thread.print as well as with jstack. The -e option > >> essentially sets PrintExtendedThreadInfo true just for the respective > >> thread dump. > >> > > >> > Here is the updated webrev: > >> > > >> > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v2 > >> > > >> > (https://bugs.openjdk.java.net/browse/JDK-8200720) > >> > > >> > Thanks, > >> > Gunter > >> > > >> > > >> > On 02.05.18, 17:07, "serviceability-dev on behalf of Haug, > >> Gunter" >> gunter.haug at sap.com> wrote: > >> > > >> > Hi Chris, > >> > > >> > Thanks for looking into this. > >> > You're right, there is a little more we have. We have > >> implemented an IO tracing mechanism which - rather as a byproduct - > >> keeps track of bytes read and written per thread. However, this of > >> course requires changes not only in hotspot. We would be happy to > >> contribute this as well, but this is a far bigger change and will > >> probably lead to a far bigger discussion. Anyway, with the number of > >> bytes read, the number of classes defined doesn't look that arbitrary > >> anymore, as one can correlate IO to class loading. > >> > > >> > Regarding the verbose option I think that's a good idea! > >> > > >> > Thanks again, > >> > Gunter > >> > > >> > On 01.05.18, 22:55, "Chris Plummer" > >> wrote: > >> > > >> > Hi Gunter, > >> > > >> > The output you are adding is all useful. I think the > >> question is (and > >> > I'd like to see a few people chime in on this for this > >> review) is > >> > whether or not all of it is the appropriate for a > >> thread's stack dump. > >> > For example, defined_classes is on the fringe of what > >> I would call > >> > generally useful info in a stack dump. Sure, there > >> might be that rare > >> > case when you need it, but how often compared to other > >> useful info > >> > maintained on a per thread basis. How many other bits > >> of useful info are > >> > not being printed in favor of defined_classes? It > >> seems you have more in > >> > the queue. How many? I'm worried about how cluttered > >> the stack dumps > >> > will get. Maybe we should add some sort of verbose > >> thread dumping > >> > option. Just a thought. > >> > > >> > As for the implementation, overall it looks good, but > >> I can't speak to > >> > whether or not you are doing proper accounting of > >> defined_classes and > >> > bytes allocated. You'll need input from someone with > >> more knowledge of > >> > those areas. We'll also need to do some testing to > >> make sure tool tests > >> > are not impacted. > >> > > >> > thanks, > >> > > >> > Chris > >> > > >> > On 4/30/18 2:51 AM, Haug, Gunter wrote: > >> > > Hi, > >> > > > >> > > this is an update to an RFR I posted on hotspot-dev, > >> but it is probably more suitable to post it here. Can I please have a > >> review and a sponsor for the following enhancement: > >> > > > >> > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v1 > >> > > https://bugs.openjdk.java.net/browse/JDK-8200720 > >> > > > >> > > We at SAP have extended the thread dumps (obtained > >> by jstack or jcmd) by several fields providing thread specific > >> information. These extensions are quite popular with our support team. > >> With some knowledge of the architecture of the application, they often > >> allow for quick and simple diagnosis of a running system. Therefore we > >> would like to contribute these enhancements. > >> > > > >> > > I took a few simple examples here, namely cpu time, > >> elapsed time since thread creation, bytes allocated and classes > >> defined by the thread and the pthread-id or equivalent on platforms > >> where it makes sense. Provided it is known how the application should > >> behave, a misbehaving thread can identified easily. > >> > > > >> > > There is no measurable overhead for this > >> enhancement. However, it may be a problem that the format of the > >> output is changed. Tools parsing the output may have to be changed. > >> > > > >> > > Here is an example of the output generated: > >> > > > >> > > ------------------------------------------------------ > >> > > > >> > > "main" #1 prio=5 os_prio=31 cpu=6300.65ms > >> elapsed=123.28s allocated=242236760B defined_classes=1725 > >> tid=0x00007fa13a806000 nid=0x1c03 pthread-id=0x109708000 waiting on > >> condition [0x0000000109707000] > >> > > java.lang.Thread.State: TIMED_WAITING (sleeping) > >> > > JavaThread state: _thread_blocked > >> > > Thread: 0x00007fa13a806000 [0x1c03] State: > >> _at_safepoint _has_called_back 0 _at_poll_safepoint 0 > >> > > JavaThread state: _thread_blocked > >> > > at java.lang.Thread.sleep(java.base/Native Method) > >> > > ... > >> > > ------------------------------------------------------ > >> > > > >> > > As mentioned above, we have a whole bunch of other > >> enhancements to the thread dump similar to this one and would be > >> willing to contribute them if there is any interest. > >> > > > >> > > Thanks and best regards, > >> > > Gunter > >> > > > >> > > > >> > > >> > > >> > > >> > > >> > > >> > > > > > > From david.holmes at oracle.com Tue Jun 5 05:57:53 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 5 Jun 2018 15:57:53 +1000 Subject: RFR (S): 8199882: compiler/uncommontrap/TestDeoptOOM.java failed w/ fatal error: ExceptionMark constructor expects no pending exceptions In-Reply-To: References: Message-ID: Ping! Please. Thanks, David On 1/06/2018 1:39 PM, David Holmes wrote: > Bug: https://bugs.openjdk.java.net/browse/JDK-8199882 > webrev: http://cr.openjdk.java.net/~dholmes/8199882/webrev/ > > The lazy loading (not initialization, despite the method names) of the > AbstractOwnableSynchronizer class (needed for deadlock detection and > stack dumps) has caused a number of problems due to the potential for > loading exceptions (specifically OOME), which result in code that does > not anticipate exceptions, encountering them. > > The bug discusses a number of possible solutions but the chosen one is > to simply pre-load the AOS class along with the numerous other core > classes that get pre-loaded. > > This does of course add a small increment to startup time, but it should > be negligible in the overall scheme of things and not something to be > unduly concerned about (Claes has commented in chat that he doesn't see > it as an issue.) > > Testing: > ? - mach5 tier1,2,3 > ? - java/lang/management/* > ? - java/util/concurrent/* > ? - compiler/uncommontrap/TestDeoptOOM.java > > Thanks, > David From serguei.spitsyn at oracle.com Tue Jun 5 08:09:00 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 5 Jun 2018 01:09:00 -0700 Subject: RFR (S): 8199882: compiler/uncommontrap/TestDeoptOOM.java failed w/ fatal error: ExceptionMark constructor expects no pending exceptions In-Reply-To: References: Message-ID: <1c5c3a3b-3fd2-6aa2-8cf7-9944967c9690@oracle.com> Hi David, The fix looks good to me. Also, it makes the initialization more simple. Thanks, Serguei On 6/4/18 22:57, David Holmes wrote: > Ping! Please. > > Thanks, > David > > On 1/06/2018 1:39 PM, David Holmes wrote: >> Bug: https://bugs.openjdk.java.net/browse/JDK-8199882 >> webrev: http://cr.openjdk.java.net/~dholmes/8199882/webrev/ >> >> The lazy loading (not initialization, despite the method names) of >> the AbstractOwnableSynchronizer class (needed for deadlock detection >> and stack dumps) has caused a number of problems due to the potential >> for loading exceptions (specifically OOME), which result in code that >> does not anticipate exceptions, encountering them. >> >> The bug discusses a number of possible solutions but the chosen one >> is to simply pre-load the AOS class along with the numerous other >> core classes that get pre-loaded. >> >> This does of course add a small increment to startup time, but it >> should be negligible in the overall scheme of things and not >> something to be unduly concerned about (Claes has commented in chat >> that he doesn't see it as an issue.) >> >> Testing: >> ?? - mach5 tier1,2,3 >> ?? - java/lang/management/* >> ?? - java/util/concurrent/* >> ?? - compiler/uncommontrap/TestDeoptOOM.java >> >> Thanks, >> David From david.holmes at oracle.com Tue Jun 5 08:38:23 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 5 Jun 2018 18:38:23 +1000 Subject: RFR (S): 8199882: compiler/uncommontrap/TestDeoptOOM.java failed w/ fatal error: ExceptionMark constructor expects no pending exceptions In-Reply-To: <1c5c3a3b-3fd2-6aa2-8cf7-9944967c9690@oracle.com> References: <1c5c3a3b-3fd2-6aa2-8cf7-9944967c9690@oracle.com> Message-ID: Thanks Serguei! David On 5/06/2018 6:09 PM, serguei.spitsyn at oracle.com wrote: > Hi David, > > The fix looks good to me. > Also, it makes the initialization more simple. > > Thanks, > Serguei > > > On 6/4/18 22:57, David Holmes wrote: >> Ping! Please. >> >> Thanks, >> David >> >> On 1/06/2018 1:39 PM, David Holmes wrote: >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8199882 >>> webrev: http://cr.openjdk.java.net/~dholmes/8199882/webrev/ >>> >>> The lazy loading (not initialization, despite the method names) of >>> the AbstractOwnableSynchronizer class (needed for deadlock detection >>> and stack dumps) has caused a number of problems due to the potential >>> for loading exceptions (specifically OOME), which result in code that >>> does not anticipate exceptions, encountering them. >>> >>> The bug discusses a number of possible solutions but the chosen one >>> is to simply pre-load the AOS class along with the numerous other >>> core classes that get pre-loaded. >>> >>> This does of course add a small increment to startup time, but it >>> should be negligible in the overall scheme of things and not >>> something to be unduly concerned about (Claes has commented in chat >>> that he doesn't see it as an issue.) >>> >>> Testing: >>> ?? - mach5 tier1,2,3 >>> ?? - java/lang/management/* >>> ?? - java/util/concurrent/* >>> ?? - compiler/uncommontrap/TestDeoptOOM.java >>> >>> Thanks, >>> David > From goetz.lindenmaier at sap.com Tue Jun 5 13:07:19 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 5 Jun 2018 13:07:19 +0000 Subject: RFR(S): 8200720: Print additional information in thread dump (times, allocated bytes etc.) In-Reply-To: References: <56817AB5-88FC-4FB9-A271-7206EA69ECA7@sap.com> <2728581E-0E32-4911-B1A9-79AD6CBA81B0@sap.com> <082ca98f-9a68-15a0-95b5-c4d5c733ab36@oracle.com> <933cdacc-9b6f-c8b5-e244-558f38e7888c@oracle.com> <55806aa8-f09b-e9a8-b512-b9676029c494@oracle.com> <0F497BC6-2569-4EF6-9B59-E24A8E1B3A37@sap.com> Message-ID: <5dc08f65e83f4374b8e1f73f218c6d6d@sap.com> Hi this discussion touched a lot of points so far, which seem to lead to different conclusions. I think we should look at the values printed: 1. cpu=6300.65ms elapsed=123.28s Overhead cpu time: * system calls at thread dump time elapsed time: * 1 system call at thread creation time * 1 64-bit field per thread for the thread start time * 1 system call at thread dump time As I understand, JDK-8143176 would have had to get the time at each locking, which is much more time critical than doing this during thread creation. While the time a lock was held would be much more useful, the ratio here, combined with knowledge about the application, could lead to the conclusion that the thread is wrongly blocked at much lower cost. 2. pthread-id=0x109708000 Overhead: * a field access at thread dump time. Negligible I would say. 3. allocated=242236760B defined_classes=1725 Overhead: * 1 64-bit field per thread. * counter increment on class creation Especially defined_classes seems to be controversial. As this only makes sense for Java threads, this could be printed in a line of it's own in Threads::print_on_error() (which already gets a flag to customize for jstack: print_concurrent_locks) by calling a dedicated function in Thread. No need for flag PrintExtendedThreadInfo thus. But I would propose to drop this information as it is too controversial. This leaves the times and the pthread-id to be decided whether they are worth the cost. If so, I think they should be printed unconditional. If a flag is required to avoid bloating the info in the default case, I would include more infos, as os_prio and nid under that flag. As jstack is deprecated, changes to jstack could be skipped always disabling the new printouts. Finally, I would propose to move _allocated_bytes into threadStatisticInfo.hpp. Best regards, Goetz. > -----Original Message----- > From: serviceability-dev [mailto:serviceability-dev- > bounces at openjdk.java.net] On Behalf Of David Holmes > Sent: Dienstag, 5. Juni 2018 04:53 > To: Haug, Gunter ; Chris Plummer > ; serviceability-dev dev at openjdk.java.net>; Langer, Christoph ; > hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR(S): 8200720: Print additional information in thread dump > (times, allocated bytes etc.) > > Hi Gunter, > > On 5/06/2018 1:27 AM, Haug, Gunter wrote: > > Hi David, Chris, > > > > would it be an option to unconditionally print the additional information? > Regardless which way this is implemented it will be rather complicated and it > only switches on/off a few field in the thread dump. > > I'm not convinced this is all suitable information for a thread stack > dump. I can see the time information being useful if using the dump to > try and diagnoze a hang or responsiveness issue. But the allocated-bytes > and classes-defined just doesn't seem useful in the context of a thread > dump to me. Anyone interested in allocation stats is going to be looking > at GC logs etc which is where this belongs. Ditto the class-stats belong > in some kind of classloading logging IMHO. > > I'm very reluctant to burden the implementation with capturing these > kinds of stats, just to use for diagnostic purposes that may not even be > asked for. I'm very much in the "you shouldn't pay for what you don't > use" camp in that regard. (See my comments in JDK-8143176 referenced by > Sean.) > > The ThreadStatisticInfo adds overhead to every thread object - and I'd > have to suspect there could be overlap with whatever flight recorder > sticks in there too. (Thread has become even more bloated in recent time!). > > Cheers, > David > > > > Thanks, > > Gunter > > > > ?On 04.06.18, 01:13, "David Holmes" wrote: > > > > Hi Gunter, Chris, > > > > Sorry just trying to catch up and this is only a partial look so far ... > > > > BTW these changes are not limited to serviceability code so adding in > > runtime team as well. > > > > On 2/06/2018 9:12 AM, Chris Plummer wrote: > > > Hi Gunter, > > > > > > On 6/1/18 3:17 AM, Haug, Gunter wrote: > > >> Hi Chris, > > >> > > >> thanks for looking into this! > > >> > > >> Re the synchronization: The value is stored only in a VM operation at > > >> a safepoint by the VM thread. I was of the opinion, that this could > > >> not be interrupted by a second VM operation (of the same type). Or > am > > >> I missing something else? > > > I was really thinking more about the temporary changing of > > > PrintExtendedThreadInfo, not the value stored in the VMOp. You may > be be > > > correct that no more than one VMOp is executing, but while it is > > > executing it is has changed the value of PrintExtendedThreadInfo, > which > > > might have an impact on anything else that triggers printing thread info > > > while the VMOp is executing. > > > > Even if nothing else can possibly be running during the safepoint I find > > it extremely bad form to change a command-line flag in this way, > > particularly from a safepoint! > > > > If this flag is intended to be dynamically enabled as part of a dcmd > > then it should also be a Manageable flag IMHO. > > > > That said ... > > > > >> I did think about passing an argument to the various print_on > member > > >> functions of the thread classes, but this would require rather > > >> extensive code changes for a rather tiny extension. Therefore I feel > > >> doing it like this is the lesser evil. > > > > ... it's obviously not truly a global setting, but one that is needed on > > a per-print-request basis. The flag is just the default, but if the > > default is off you still want to enable extended printing on a > > per-request basis. > > > > The current print_on mechanics is not set up for this kind of > > flexibility. I think this needs more thought. > > > > --- > > > > Re osThread.cpp shared code changes ... I really hate to see all the > > ifdefs in there. Please add a pd_print_on function to support this if > > you truly want platform specific information. > > > > --- > > > > threadStatisticInfo.hpp > > > > typo: getElepsedTime() -> getElapsedTime() > > > > Thanks, > > David > > > > > > > > >> > > >> Re thread_dump(): I think it's correct or, well, at least it works ;-) > > >> In fact jstack will transfer the "-e" and "-l" in only one string, > > >> i.e. op->arg(0). > > > So you are saying that op->arg(0) is "-e -l" when using jstack? I think > > > you really need to clean up the parsing. As it stands right now passing, > > > I get the feeling that if a user erroneously asks for help by using > > > "jcmd Thread.Print -help", it will end up executing with -e an -l > > > enabled, and no failure for the invalid "-help" option. > > >> Christoph has already explained my reasoning. But I agree, it's not > > >> nice and I would be happy to do it like Christoph suggested. > > > I'll respond separately to his suggestion. > > > > > > thanks, > > > > > > Chris > > >> > > >> And typo fixed, sorry. > > >> > > >> Thanks again, > > >> Gunter > > >> > > >> On 01.06.18, 00:03, "Chris Plummer" > wrote: > > >> > > >> Hi Gunter, > > >> globals.hpp: fix typo "informatiuon" > > >> I worry a little bit about the synchronizing (if that's the right > > >> word) > > >> of PrintExtendedThreadInfo and the dcmd's -e flag. When using -e, > > >> you > > >> are temporarily enabling PrintExtendedThreadInfo if it was false. > > >> This > > >> temporarily changes the behavior of thread dumps, and could > > >> impact other > > >> uses that happen in parallel. Also, could two simultaneous uses > > >> of -e > > >> result in PrintExtendedThreadInfo not getting restored properly? > > >> thread_dump() doesn't look right. It looks like you are iterating > > >> char > > >> by char over the argument, and expect something like "-el" to be > > >> specified rather then "-e -l". The loop should be iterating over > > >> op->arg(i), not op->arg(0)[i]. > > >> The rest of the changes look fine. > > >> thanks, > > >> Chris > > >> On 5/30/18 8:12 AM, Haug, Gunter wrote: > > >> > Hi all, > > >> > > > >> > As Chris proposed, I have made an the extended output > > >> switchable. There is an VM flag (PrintExtendedThreadInfo), which is > > >> false by default. Moreover, there is an Option (-e) which can be used > > >> with jcmd Thread.print as well as with jstack. The -e option > > >> essentially sets PrintExtendedThreadInfo true just for the respective > > >> thread dump. > > >> > > > >> > Here is the updated webrev: > > >> > > > >> > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v2 > > >> > > > >> > (https://bugs.openjdk.java.net/browse/JDK-8200720) > > >> > > > >> > Thanks, > > >> > Gunter > > >> > > > >> > > > >> > On 02.05.18, 17:07, "serviceability-dev on behalf of Haug, > > >> Gunter" > >> gunter.haug at sap.com> wrote: > > >> > > > >> > Hi Chris, > > >> > > > >> > Thanks for looking into this. > > >> > You're right, there is a little more we have. We have > > >> implemented an IO tracing mechanism which - rather as a byproduct - > > >> keeps track of bytes read and written per thread. However, this of > > >> course requires changes not only in hotspot. We would be happy to > > >> contribute this as well, but this is a far bigger change and will > > >> probably lead to a far bigger discussion. Anyway, with the number of > > >> bytes read, the number of classes defined doesn't look that arbitrary > > >> anymore, as one can correlate IO to class loading. > > >> > > > >> > Regarding the verbose option I think that's a good idea! > > >> > > > >> > Thanks again, > > >> > Gunter > > >> > > > >> > On 01.05.18, 22:55, "Chris Plummer" > > >> wrote: > > >> > > > >> > Hi Gunter, > > >> > > > >> > The output you are adding is all useful. I think the > > >> question is (and > > >> > I'd like to see a few people chime in on this for this > > >> review) is > > >> > whether or not all of it is the appropriate for a > > >> thread's stack dump. > > >> > For example, defined_classes is on the fringe of what > > >> I would call > > >> > generally useful info in a stack dump. Sure, there > > >> might be that rare > > >> > case when you need it, but how often compared to other > > >> useful info > > >> > maintained on a per thread basis. How many other bits > > >> of useful info are > > >> > not being printed in favor of defined_classes? It > > >> seems you have more in > > >> > the queue. How many? I'm worried about how cluttered > > >> the stack dumps > > >> > will get. Maybe we should add some sort of verbose > > >> thread dumping > > >> > option. Just a thought. > > >> > > > >> > As for the implementation, overall it looks good, but > > >> I can't speak to > > >> > whether or not you are doing proper accounting of > > >> defined_classes and > > >> > bytes allocated. You'll need input from someone with > > >> more knowledge of > > >> > those areas. We'll also need to do some testing to > > >> make sure tool tests > > >> > are not impacted. > > >> > > > >> > thanks, > > >> > > > >> > Chris > > >> > > > >> > On 4/30/18 2:51 AM, Haug, Gunter wrote: > > >> > > Hi, > > >> > > > > >> > > this is an update to an RFR I posted on hotspot-dev, > > >> but it is probably more suitable to post it here. Can I please have a > > >> review and a sponsor for the following enhancement: > > >> > > > > >> > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v1 > > >> > > https://bugs.openjdk.java.net/browse/JDK-8200720 > > >> > > > > >> > > We at SAP have extended the thread dumps (obtained > > >> by jstack or jcmd) by several fields providing thread specific > > >> information. These extensions are quite popular with our support > team. > > >> With some knowledge of the architecture of the application, they > often > > >> allow for quick and simple diagnosis of a running system. Therefore > we > > >> would like to contribute these enhancements. > > >> > > > > >> > > I took a few simple examples here, namely cpu time, > > >> elapsed time since thread creation, bytes allocated and classes > > >> defined by the thread and the pthread-id or equivalent on platforms > > >> where it makes sense. Provided it is known how the application > should > > >> behave, a misbehaving thread can identified easily. > > >> > > > > >> > > There is no measurable overhead for this > > >> enhancement. However, it may be a problem that the format of the > > >> output is changed. Tools parsing the output may have to be changed. > > >> > > > > >> > > Here is an example of the output generated: > > >> > > > > >> > > ------------------------------------------------------ > > >> > > > > >> > > "main" #1 prio=5 os_prio=31 cpu=6300.65ms > > >> elapsed=123.28s allocated=242236760B defined_classes=1725 > > >> tid=0x00007fa13a806000 nid=0x1c03 pthread-id=0x109708000 waiting > on > > >> condition [0x0000000109707000] > > >> > > java.lang.Thread.State: TIMED_WAITING (sleeping) > > >> > > JavaThread state: _thread_blocked > > >> > > Thread: 0x00007fa13a806000 [0x1c03] State: > > >> _at_safepoint _has_called_back 0 _at_poll_safepoint 0 > > >> > > JavaThread state: _thread_blocked > > >> > > at java.lang.Thread.sleep(java.base/Native Method) > > >> > > ... > > >> > > ------------------------------------------------------ > > >> > > > > >> > > As mentioned above, we have a whole bunch of other > > >> enhancements to the thread dump similar to this one and would be > > >> willing to contribute them if there is any interest. > > >> > > > > >> > > Thanks and best regards, > > >> > > Gunter > > >> > > > > >> > > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > > > > > > > > From shade at redhat.com Tue Jun 5 15:46:27 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 5 Jun 2018 17:46:27 +0200 Subject: RFR (round 4), JEP-318: Epsilon GC In-Reply-To: <0b5c827a-2763-98aa-06af-24df9028aed7@oracle.com> References: <094e1093-5a13-4853-aa34-d4e987a069b0@redhat.com> <0b5c827a-2763-98aa-06af-24df9028aed7@oracle.com> Message-ID: Hi Jini, Thanks for taking a look, see comments inline. On 06/01/2018 10:13 AM, Jini George wrote: > ==> share/classes/sun/jvm/hotspot/oops/ObjectHeap.java > > 444??????? liveRegions.add(eh.space()); > > We would need to add an object of type 'Address' to the liveRegions list, instead of type > VirtualSpace. Not doing so results in exceptions of the following form from the compare() method for > various clhsdb commands like 'jhisto': > > Exception in thread "main" java.lang.ClassCastException: > jdk.hotspot.agent/sun.jvm.hotspot.memory.VirtualSpace cannot be cast to > jdk.hotspot.agent/sun.jvm.hotspot.debugger.Address Oh, I see! Fixed: http://hg.openjdk.java.net/jdk/sandbox/rev/d999bdb8173c > ==>? share/classes/sun/jvm/hotspot/oops/ObjectHeap.java > > 445???? } else { > 446??????? if (Assert.ASSERTS_ENABLED) { > 447?????????? Assert.that(false, "Expecting GenCollectedHeap, G1CollectedHeap, " + > 448?????????????????????? "or ParallelScavengeHeap, but got " + > 449?????????????????????? heap.getClass().getName()); > 450??????? } > 451???? } > > * Please add EpsilonGC also to the assertion statement. I prefer to change it to non-GC-specific message as Per suggested: http://hg.openjdk.java.net/jdk/sandbox/rev/5cd17d3d3f83 > ==> share/classes/sun/jvm/hotspot/tools/HeapSummary.java > > The run() method would need to handle Epsilon GC here to avoid the Unknown CollectedHeap type error > with jhsdb jmap --heap. Right, implemented in both places, run() and detectGCAlgo(): http://hg.openjdk.java.net/jdk/sandbox/rev/b20a56352d78 > ==> share/classes/sun/jvm/hotspot/HSDB.java > > In? showThreadStackMemory(), we have: > > 1101?????????????????????????? } > 1102???????????????????????? } else if (collHeap instanceof ParallelScavengeHeap) { > 1103?????????????????????????? ParallelScavengeHeap heap = (ParallelScavengeHeap) collHeap; > 1104?????????????????????????? if (heap.youngGen().isIn(handle)) { > 1105???????????????????????????? anno = "PSYoungGen "; > 1106???????????????????????????? bad = false; > 1107?????????????????????????? } else if (heap.oldGen().isIn(handle)) { > 1108???????????????????????????? anno = "PSOldGen "; > 1109???????????????????????????? bad = false; > 1110?????????????????????????? } > 1111???????????????????????? } else { > 1112?????????????????????????? // Optimistically assume the oop isn't bad > 1113?????????????????????????? anno = "[Unknown generation] "; > 1114?????????????????????????? bad = false; > 1115???????????????????????? } > 1116 > > We would need to add the case of collHeap being an instanceof EpsilonHeap too. It would display > "Unknown generation" while viewing the stack memory for the Java threads otherwise. Right, fixed: http://hg.openjdk.java.net/jdk/sandbox/rev/f26c4a196a15 > ==> It would be great if test/hotspot/jtreg/serviceability/sa/TestUniverse.java is enhanced to add > the minimalistic test for EpsilonGC. Right: http://hg.openjdk.java.net/jdk/sandbox/rev/c559de946c7d This still passes gc/epsilon and serviceability/sa tests. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From daniil.x.titov at oracle.com Tue Jun 5 18:38:22 2018 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Tue, 05 Jun 2018 11:38:22 -0700 Subject: RFR 8203033: [Testbug] vmTestbase/nsk/jdi/TypeComponent/isSynthetic/issynthetic002/TestDescription.java fails with nestmates Message-ID: <9C78D8B6-F7C0-44EE-93B3-D8C22CAF2628@oracle.com> Please review the change that fixes vmTestbase/nsk/jdi/TypeComponent/isSynthetic/issynthetic002/TestDescription.java test. The fix uses the implementation of a parametrized interface in the test class to trigger the compiler to generate a synthetic bridge method. The test works fine in both nestmates and jdk/jdk repositories. Issue: https://bugs.openjdk.java.net/browse/JDK-8203033 Webrev: http://cr.openjdk.java.net/~dtitov/8203033/webrev.02 Thanks, Daniil From coleen.phillimore at oracle.com Tue Jun 5 19:01:22 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 5 Jun 2018 15:01:22 -0400 Subject: RFR: 8204168: Increase small heap sizes in tests to accommodate ZGC In-Reply-To: <102083fa-f2b0-dc2d-7d8b-89a748c2c5db@oracle.com> References: <102083fa-f2b0-dc2d-7d8b-89a748c2c5db@oracle.com> Message-ID: <53fc28f4-cb1e-2e4e-f7c6-7a6c1a6c5c87@oracle.com> I included the serviceability-dev list for the jdi tests.? I don't think this change increases the heap for these tests enough to cause any problems, so I think this change is fine. thanks, Coleen On 6/5/18 5:27 AM, Erik Helin wrote: > On 05/31/2018 02:32 PM, Stefan Karlsson wrote: >> Hi all, > > Hey Stefan, > >> Please review this patch to increase the heap size for tests that >> sets a small heap size. >> >> http://cr.openjdk.java.net/~stefank/8204168/webrev.01 >> https://bugs.openjdk.java.net/browse/JDK-8204168 > > I read through each test in compiler, gc and runtime carefully to > check that the increased heap size wouldn't render the test useless > (i.e. I ensured that the tests still test something). Good news, it > seems (to me at least) that increasing the heap size for all of the > above tests will work fine. Please consider the changes to those tests > Reviewed by me, nice work! > > Unfortunately I don't know the "nsk" tests well enough to review those > changes... :( > > Thanks, > Erik From chris.plummer at oracle.com Tue Jun 5 19:18:42 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 5 Jun 2018 12:18:42 -0700 Subject: RFR (S): 8199882: compiler/uncommontrap/TestDeoptOOM.java failed w/ fatal error: ExceptionMark constructor expects no pending exceptions In-Reply-To: <1c5c3a3b-3fd2-6aa2-8cf7-9944967c9690@oracle.com> References: <1c5c3a3b-3fd2-6aa2-8cf7-9944967c9690@oracle.com> Message-ID: +1 Chris On 6/5/18 1:09 AM, serguei.spitsyn at oracle.com wrote: > Hi David, > > The fix looks good to me. > Also, it makes the initialization more simple. > > Thanks, > Serguei > > > On 6/4/18 22:57, David Holmes wrote: >> Ping! Please. >> >> Thanks, >> David >> >> On 1/06/2018 1:39 PM, David Holmes wrote: >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8199882 >>> webrev: http://cr.openjdk.java.net/~dholmes/8199882/webrev/ >>> >>> The lazy loading (not initialization, despite the method names) of >>> the AbstractOwnableSynchronizer class (needed for deadlock detection >>> and stack dumps) has caused a number of problems due to the >>> potential for loading exceptions (specifically OOME), which result >>> in code that does not anticipate exceptions, encountering them. >>> >>> The bug discusses a number of possible solutions but the chosen one >>> is to simply pre-load the AOS class along with the numerous other >>> core classes that get pre-loaded. >>> >>> This does of course add a small increment to startup time, but it >>> should be negligible in the overall scheme of things and not >>> something to be unduly concerned about (Claes has commented in chat >>> that he doesn't see it as an issue.) >>> >>> Testing: >>> ?? - mach5 tier1,2,3 >>> ?? - java/lang/management/* >>> ?? - java/util/concurrent/* >>> ?? - compiler/uncommontrap/TestDeoptOOM.java >>> >>> Thanks, >>> David > From alexey.menkov at oracle.com Tue Jun 5 19:41:55 2018 From: alexey.menkov at oracle.com (Alex Menkov) Date: Tue, 5 Jun 2018 12:41:55 -0700 Subject: RFR(S): 8200387: [Graal] fromTonga/nsk/jdi tests hang with Graal as JIT In-Reply-To: References: Message-ID: <5584b238-8956-8eb3-094c-80361c2c6b75@oracle.com> Looks okay to me. --alex On 05/24/2018 22:43, serguei.spitsyn at oracle.com wrote: > Hi all, > > Please, review a fix for: > https://bugs.openjdk.java.net/browse/JDK-8200387 > > Webrev: > http://cr.openjdk.java.net/~sspitsyn/webrevs/2018/8200387-graal-queue.1/ > > Summary: > ? A number of tests are failing when executed with Graal compiler. > ? The issue is that the JDI client event queue becomes overloaded: > > 248 private synchronized void controlEventFlow(int maxQueueSize) { > 249 if (!eventsHeld && (maxQueueSize > OVERLOADED_QUEUE)) { > 250 eventController().hold(); > 251 eventsHeld = true; > 252 } else if (eventsHeld && (maxQueueSize < UNDERLOADED_QUEUE)) { > 253 eventController().release(); > 254 eventsHeld = false; > 255 } > 256 } > > ? with the events generated from the Graal compiler execution. > ? As a result, the eventsHeldmode is enabled. > ? The tests hang because the back-end does not get needed commands from > the JDI client. > ? Thanks a lot to Igor Veresov for the analysis and suggested fixes! > ? After some discussion we agreed on the work around fix in webrev > ? which is to increase the OVERLOADED_QUEUE value. > > ? In fact, the tests have to be fixed as well to filter out the Graal > threads posting the events. > ? At this point, fixing of all these tests requires resources but not > that critical. > Now, there is a mess with a relatively big number of the tests failing > with different failure modes. > ? The corresponding bugs have already been filed (not sure, all of them). > ? This fix is the first step and a good base for sorting out the rest > of the Graal related issues. > > ? I've just filed one more Graal bug that was masked other bugs: > https://bugs.openjdk.java.net/browse/JDK-8203809 > ? and suspect, more bugs might be needed to cover all different failure > modes. > ? Such bugs should include sub-tasks for including the impacted tests > into the ProblemList-graal. > > ? A part of the fix is a ProblemList-graal.txt update. > ? Almost all the tests listed in the following Graal related bug > stopped failing with the the fix: > https://bugs.openjdk.java.net/browse/JDK-8195600 > ? So, these tests are removed from the ProblemList-graal. > > Thanks, > Serguei > > > > From chris.plummer at oracle.com Tue Jun 5 20:34:42 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 5 Jun 2018 13:34:42 -0700 Subject: RFR(S): 8200387: [Graal] fromTonga/nsk/jdi tests hang with Graal as JIT In-Reply-To: <5ad21d24-43ae-f408-b1bc-40c676b087c4@oracle.com> References: <5ad21d24-43ae-f408-b1bc-40c676b087c4@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Tue Jun 5 21:49:32 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 5 Jun 2018 14:49:32 -0700 Subject: RFR(S): 8200387: [Graal] fromTonga/nsk/jdi tests hang with Graal as JIT In-Reply-To: <5584b238-8956-8eb3-094c-80361c2c6b75@oracle.com> References: <5584b238-8956-8eb3-094c-80361c2c6b75@oracle.com> Message-ID: Hi Alex, Thank you a lot for review! Thanks, Serguei On 6/5/18 12:41, Alex Menkov wrote: > Looks okay to me. > > --alex > > On 05/24/2018 22:43, serguei.spitsyn at oracle.com wrote: >> Hi all, >> >> Please, review a fix for: >> https://bugs.openjdk.java.net/browse/JDK-8200387 >> >> Webrev: >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2018/8200387-graal-queue.1/ >> >> Summary: >> ?? A number of tests are failing when executed with Graal compiler. >> ?? The issue is that the JDI client event queue becomes overloaded: >> >> ? 248???? private synchronized void controlEventFlow(int maxQueueSize) { >> ? 249???????? if (!eventsHeld && (maxQueueSize > OVERLOADED_QUEUE)) { >> ? 250???????????? eventController().hold(); >> ? 251???????????? eventsHeld = true; >> ? 252???????? } else if (eventsHeld && (maxQueueSize < >> UNDERLOADED_QUEUE)) { >> ? 253???????????? eventController().release(); >> ? 254???????????? eventsHeld = false; >> ? 255???????? } >> ? 256???? } >> >> ?? with the events generated from the Graal compiler execution. >> ?? As a result, the eventsHeldmode is enabled. >> ?? The tests hang because the back-end does not get needed commands >> from the JDI client. >> ?? Thanks a lot to Igor Veresov for the analysis and suggested fixes! >> ?? After some discussion we agreed on the work around fix in webrev >> ?? which is to increase the OVERLOADED_QUEUE value. >> >> ?? In fact, the tests have to be fixed as well to filter out the >> Graal threads posting the events. >> ?? At this point, fixing of all these tests requires resources but >> not that critical. >> Now, there is a mess with a relatively big number of the tests >> failing with different failure modes. >> ?? The corresponding bugs have already been filed (not sure, all of >> them). >> ?? This fix is the first step and a good base for sorting out the >> rest of the Graal related issues. >> >> ?? I've just filed one more Graal bug that was masked other bugs: >> https://bugs.openjdk.java.net/browse/JDK-8203809 >> ?? and suspect, more bugs might be needed to cover all different >> failure modes. >> ?? Such bugs should include sub-tasks for including the impacted >> tests into the ProblemList-graal. >> >> ?? A part of the fix is a ProblemList-graal.txt update. >> ?? Almost all the tests listed in the following Graal related bug >> stopped failing with the the fix: >> https://bugs.openjdk.java.net/browse/JDK-8195600 >> ?? So, these tests are removed from the ProblemList-graal. >> >> Thanks, >> Serguei >> >> >> >> From david.holmes at oracle.com Tue Jun 5 23:13:21 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 6 Jun 2018 09:13:21 +1000 Subject: RFR (S): 8199882: compiler/uncommontrap/TestDeoptOOM.java failed w/ fatal error: ExceptionMark constructor expects no pending exceptions In-Reply-To: References: <1c5c3a3b-3fd2-6aa2-8cf7-9944967c9690@oracle.com> Message-ID: Thanks Chris! David On 6/06/2018 5:18 AM, Chris Plummer wrote: > +1 > > Chris > > On 6/5/18 1:09 AM, serguei.spitsyn at oracle.com wrote: >> Hi David, >> >> The fix looks good to me. >> Also, it makes the initialization more simple. >> >> Thanks, >> Serguei >> >> >> On 6/4/18 22:57, David Holmes wrote: >>> Ping! Please. >>> >>> Thanks, >>> David >>> >>> On 1/06/2018 1:39 PM, David Holmes wrote: >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8199882 >>>> webrev: http://cr.openjdk.java.net/~dholmes/8199882/webrev/ >>>> >>>> The lazy loading (not initialization, despite the method names) of >>>> the AbstractOwnableSynchronizer class (needed for deadlock detection >>>> and stack dumps) has caused a number of problems due to the >>>> potential for loading exceptions (specifically OOME), which result >>>> in code that does not anticipate exceptions, encountering them. >>>> >>>> The bug discusses a number of possible solutions but the chosen one >>>> is to simply pre-load the AOS class along with the numerous other >>>> core classes that get pre-loaded. >>>> >>>> This does of course add a small increment to startup time, but it >>>> should be negligible in the overall scheme of things and not >>>> something to be unduly concerned about (Claes has commented in chat >>>> that he doesn't see it as an issue.) >>>> >>>> Testing: >>>> ?? - mach5 tier1,2,3 >>>> ?? - java/lang/management/* >>>> ?? - java/util/concurrent/* >>>> ?? - compiler/uncommontrap/TestDeoptOOM.java >>>> >>>> Thanks, >>>> David >> > > From stuart.marks at oracle.com Wed Jun 6 00:05:33 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Tue, 5 Jun 2018 17:05:33 -0700 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> Message-ID: [adding serviceability-dev] Hi serviceability folks, I'm in the process of removing Thread.destroy() and Thread.stop(Throwable) from the Java SE API. Alan and David have pointed out that there are some cross-references to Thread.stop(Throwable) in the JDWP and JVMTI specs, as well as in the JDI ThreadReference API. I've adjusted the relevant files. See please review this updated webrev: http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/ For context, please see the full review thread on core-libs-dev: http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053536.html On 6/4/18 11:11 PM, Alan Bateman wrote: > The source file that is used to generate the JDWP protocol code and the spec is > in make/data/jdwp/jdwp.spec. The JVM TI spec is at > src/hotspot/share/prims/jvmti.xml. It should be okay to skip those for this > change-set, assuming it is followed up quickly with another change-set to update > those specs. OK. I took a look at those other files and they seem simple enough to update as part of this changeset. If there aren't any objections from anyone, might as well get this all done at once. Thanks, s'marks From david.holmes at oracle.com Wed Jun 6 01:40:44 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 6 Jun 2018 11:40:44 +1000 Subject: RFR 8203033: [Testbug] vmTestbase/nsk/jdi/TypeComponent/isSynthetic/issynthetic002/TestDescription.java fails with nestmates In-Reply-To: <9C78D8B6-F7C0-44EE-93B3-D8C22CAF2628@oracle.com> References: <9C78D8B6-F7C0-44EE-93B3-D8C22CAF2628@oracle.com> Message-ID: <67c8c01a-2c82-b5d5-2143-fad5d3f1db07@oracle.com> Hi Daniil, Thanks for taking this on! On 6/06/2018 4:38 AM, Daniil Titov wrote: > Please review the change that fixes vmTestbase/nsk/jdi/TypeComponent/isSynthetic/issynthetic002/TestDescription.java test. > > The fix uses the implementation of a parametrized interface in the test class to trigger the compiler to generate a synthetic bridge method. The test works fine in both nestmates and jdk/jdk repositories. This looks fine to me and I like the simplification of not having to re-list all the methods in the debuggee class in the debugger class. The test may break again in the future if compilation strategies change, but any test for synthetic methods obviously has a dependency on the compilation strategy, so that's not a concern. Thanks, David > Issue: https://bugs.openjdk.java.net/browse/JDK-8203033 > Webrev: http://cr.openjdk.java.net/~dtitov/8203033/webrev.02 > > Thanks, > Daniil > > From david.holmes at oracle.com Wed Jun 6 02:55:53 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 6 Jun 2018 12:55:53 +1000 Subject: RFR(S): 8200720: Print additional information in thread dump (times, allocated bytes etc.) In-Reply-To: <5dc08f65e83f4374b8e1f73f218c6d6d@sap.com> References: <56817AB5-88FC-4FB9-A271-7206EA69ECA7@sap.com> <2728581E-0E32-4911-B1A9-79AD6CBA81B0@sap.com> <082ca98f-9a68-15a0-95b5-c4d5c733ab36@oracle.com> <933cdacc-9b6f-c8b5-e244-558f38e7888c@oracle.com> <55806aa8-f09b-e9a8-b512-b9676029c494@oracle.com> <0F497BC6-2569-4EF6-9B59-E24A8E1B3A37@sap.com> <5dc08f65e83f4374b8e1f73f218c6d6d@sap.com> Message-ID: <626701f8-b71b-58dc-f837-e5e1d81ef187@oracle.com> Hi Goetz, On 5/06/2018 11:07 PM, Lindenmaier, Goetz wrote: > Hi > > this discussion touched a lot of points so far, which seem > to lead to different conclusions. > > I think we should look at the values printed: > > 1. cpu=6300.65ms elapsed=123.28s > Overhead > cpu time: > * system calls at thread dump time > elapsed time: > * 1 system call at thread creation time > * 1 64-bit field per thread for the thread start time > * 1 system call at thread dump time > > As I understand, JDK-8143176 would have had to get the > time at each locking, which is much more time critical > than doing this during thread creation. While Correct. > the time a lock was held would be much more useful, > the ratio here, combined with knowledge about the application, > could lead to the conclusion that the thread is wrongly > blocked at much lower cost. Agreed. I see no issue with unconditionally adding this information as it should address some of the concerns of 8143176 as well. > > 2. pthread-id=0x109708000 > Overhead: > * a field access at thread dump time. Negligible I would say. I'd overlooked this addition, but it is of course the chunk of ifdef code in osThread.cpp that I objected to. The problem I have here is the convoluted mess of "thread identifiers" that we already have. We currently print: st->print("tid=" INTPTR_FORMAT " ", p2i(this)); which is just the address of the Thread/JavaThread object. Then: st->print("nid=0x%x ", thread_id()); where 'n' is supposed to represent "native thread id", which is: - linux: kernel thread id from syscall gettid - solaris: thread library identity from thr_create() or thr_self() - windows: thread id from _beginthreadex - OS X: kernel thread id from pthread_mach_thread_np(pthread_self()); - Other BSD: kernel (?) thread id from syscall thr_self or getthrid - AIX: thread library identity from pthread_create() or pthread_self() It's telling that the code Gunter added gets the thread library id on linux, but a "kernel thread id" on other platforms! Do we want to see a thread library id and a kernel thread id? IIRC nid is supposed be the id you need to see the thread in a debugger. In which case I'm unclear what role the thread id Gunter wants to add is playing? I don't think we need to see kernel ids in general, and pthread-id isn't useful for debugging is it? > > 3. allocated=242236760B defined_classes=1725 > Overhead: > * 1 64-bit field per thread. > * counter increment on class creation > > Especially defined_classes seems to be controversial. > As this only makes sense for Java threads, this could > be printed in a line of it's own in Threads::print_on_error() > (which already gets a flag to customize for jstack: > print_concurrent_locks) by calling a dedicated function in Thread. > No need for flag PrintExtendedThreadInfo thus. > > But I would propose to drop this information as it is > too controversial. Agreed. > This leaves the times and the pthread-id to be decided whether > they are worth the cost. If so, I think they should be printed > unconditional. Agreed. With some refinement of the "thread id" issue. > If a flag is required to avoid bloating the info in the default case, > I would include more infos, as os_prio and nid under that flag. Of course those things exist for good reason. The native id is supposed to be what aids in you matching what you see in our thread dumps with what you see in a debugger. Priority is less of an issue these days, but there was a time ... ;-) > As jstack is deprecated, changes to jstack could be skipped always > disabling the new printouts. > > Finally, I would propose to move _allocated_bytes into > threadStatisticInfo.hpp. Didn't that part get dropped? (You don't need it for the time info or thread id). That's a GC logging issue to me. Thanks, David > Best regards, > Goetz. > > > > >> -----Original Message----- >> From: serviceability-dev [mailto:serviceability-dev- >> bounces at openjdk.java.net] On Behalf Of David Holmes >> Sent: Dienstag, 5. Juni 2018 04:53 >> To: Haug, Gunter ; Chris Plummer >> ; serviceability-dev > dev at openjdk.java.net>; Langer, Christoph ; >> hotspot-runtime-dev at openjdk.java.net >> Subject: Re: RFR(S): 8200720: Print additional information in thread dump >> (times, allocated bytes etc.) >> >> Hi Gunter, >> >> On 5/06/2018 1:27 AM, Haug, Gunter wrote: >>> Hi David, Chris, >>> >>> would it be an option to unconditionally print the additional information? >> Regardless which way this is implemented it will be rather complicated and it >> only switches on/off a few field in the thread dump. >> >> I'm not convinced this is all suitable information for a thread stack >> dump. I can see the time information being useful if using the dump to >> try and diagnoze a hang or responsiveness issue. But the allocated-bytes >> and classes-defined just doesn't seem useful in the context of a thread >> dump to me. Anyone interested in allocation stats is going to be looking >> at GC logs etc which is where this belongs. Ditto the class-stats belong >> in some kind of classloading logging IMHO. >> >> I'm very reluctant to burden the implementation with capturing these >> kinds of stats, just to use for diagnostic purposes that may not even be >> asked for. I'm very much in the "you shouldn't pay for what you don't >> use" camp in that regard. (See my comments in JDK-8143176 referenced by >> Sean.) >> >> The ThreadStatisticInfo adds overhead to every thread object - and I'd >> have to suspect there could be overlap with whatever flight recorder >> sticks in there too. (Thread has become even more bloated in recent time!). >> >> Cheers, >> David >> >> >>> Thanks, >>> Gunter >>> >>> ?On 04.06.18, 01:13, "David Holmes" wrote: >>> >>> Hi Gunter, Chris, >>> >>> Sorry just trying to catch up and this is only a partial look so far ... >>> >>> BTW these changes are not limited to serviceability code so adding in >>> runtime team as well. >>> >>> On 2/06/2018 9:12 AM, Chris Plummer wrote: >>> > Hi Gunter, >>> > >>> > On 6/1/18 3:17 AM, Haug, Gunter wrote: >>> >> Hi Chris, >>> >> >>> >> thanks for looking into this! >>> >> >>> >> Re the synchronization: The value is stored only in a VM operation at >>> >> a safepoint by the VM thread. I was of the opinion, that this could >>> >> not be interrupted by a second VM operation (of the same type). Or >> am >>> >> I missing something else? >>> > I was really thinking more about the temporary changing of >>> > PrintExtendedThreadInfo, not the value stored in the VMOp. You may >> be be >>> > correct that no more than one VMOp is executing, but while it is >>> > executing it is has changed the value of PrintExtendedThreadInfo, >> which >>> > might have an impact on anything else that triggers printing thread info >>> > while the VMOp is executing. >>> >>> Even if nothing else can possibly be running during the safepoint I find >>> it extremely bad form to change a command-line flag in this way, >>> particularly from a safepoint! >>> >>> If this flag is intended to be dynamically enabled as part of a dcmd >>> then it should also be a Manageable flag IMHO. >>> >>> That said ... >>> >>> >> I did think about passing an argument to the various print_on >> member >>> >> functions of the thread classes, but this would require rather >>> >> extensive code changes for a rather tiny extension. Therefore I feel >>> >> doing it like this is the lesser evil. >>> >>> ... it's obviously not truly a global setting, but one that is needed on >>> a per-print-request basis. The flag is just the default, but if the >>> default is off you still want to enable extended printing on a >>> per-request basis. >>> >>> The current print_on mechanics is not set up for this kind of >>> flexibility. I think this needs more thought. >>> >>> --- >>> >>> Re osThread.cpp shared code changes ... I really hate to see all the >>> ifdefs in there. Please add a pd_print_on function to support this if >>> you truly want platform specific information. >>> >>> --- >>> >>> threadStatisticInfo.hpp >>> >>> typo: getElepsedTime() -> getElapsedTime() >>> >>> Thanks, >>> David >>> >>> >>> >>> >> >>> >> Re thread_dump(): I think it's correct or, well, at least it works ;-) >>> >> In fact jstack will transfer the "-e" and "-l" in only one string, >>> >> i.e. op->arg(0). >>> > So you are saying that op->arg(0) is "-e -l" when using jstack? I think >>> > you really need to clean up the parsing. As it stands right now passing, >>> > I get the feeling that if a user erroneously asks for help by using >>> > "jcmd Thread.Print -help", it will end up executing with -e an -l >>> > enabled, and no failure for the invalid "-help" option. >>> >> Christoph has already explained my reasoning. But I agree, it's not >>> >> nice and I would be happy to do it like Christoph suggested. >>> > I'll respond separately to his suggestion. >>> > >>> > thanks, >>> > >>> > Chris >>> >> >>> >> And typo fixed, sorry. >>> >> >>> >> Thanks again, >>> >> Gunter >>> >> >>> >> On 01.06.18, 00:03, "Chris Plummer" >> wrote: >>> >> >>> >> Hi Gunter, >>> >> globals.hpp: fix typo "informatiuon" >>> >> I worry a little bit about the synchronizing (if that's the right >>> >> word) >>> >> of PrintExtendedThreadInfo and the dcmd's -e flag. When using -e, >>> >> you >>> >> are temporarily enabling PrintExtendedThreadInfo if it was false. >>> >> This >>> >> temporarily changes the behavior of thread dumps, and could >>> >> impact other >>> >> uses that happen in parallel. Also, could two simultaneous uses >>> >> of -e >>> >> result in PrintExtendedThreadInfo not getting restored properly? >>> >> thread_dump() doesn't look right. It looks like you are iterating >>> >> char >>> >> by char over the argument, and expect something like "-el" to be >>> >> specified rather then "-e -l". The loop should be iterating over >>> >> op->arg(i), not op->arg(0)[i]. >>> >> The rest of the changes look fine. >>> >> thanks, >>> >> Chris >>> >> On 5/30/18 8:12 AM, Haug, Gunter wrote: >>> >> > Hi all, >>> >> > >>> >> > As Chris proposed, I have made an the extended output >>> >> switchable. There is an VM flag (PrintExtendedThreadInfo), which is >>> >> false by default. Moreover, there is an Option (-e) which can be used >>> >> with jcmd Thread.print as well as with jstack. The -e option >>> >> essentially sets PrintExtendedThreadInfo true just for the respective >>> >> thread dump. >>> >> > >>> >> > Here is the updated webrev: >>> >> > >>> >> > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v2 >>> >> > >>> >> > (https://bugs.openjdk.java.net/browse/JDK-8200720) >>> >> > >>> >> > Thanks, >>> >> > Gunter >>> >> > >>> >> > >>> >> > On 02.05.18, 17:07, "serviceability-dev on behalf of Haug, >>> >> Gunter" >> >> gunter.haug at sap.com> wrote: >>> >> > >>> >> > Hi Chris, >>> >> > >>> >> > Thanks for looking into this. >>> >> > You're right, there is a little more we have. We have >>> >> implemented an IO tracing mechanism which - rather as a byproduct - >>> >> keeps track of bytes read and written per thread. However, this of >>> >> course requires changes not only in hotspot. We would be happy to >>> >> contribute this as well, but this is a far bigger change and will >>> >> probably lead to a far bigger discussion. Anyway, with the number of >>> >> bytes read, the number of classes defined doesn't look that arbitrary >>> >> anymore, as one can correlate IO to class loading. >>> >> > >>> >> > Regarding the verbose option I think that's a good idea! >>> >> > >>> >> > Thanks again, >>> >> > Gunter >>> >> > >>> >> > On 01.05.18, 22:55, "Chris Plummer" >>> >> wrote: >>> >> > >>> >> > Hi Gunter, >>> >> > >>> >> > The output you are adding is all useful. I think the >>> >> question is (and >>> >> > I'd like to see a few people chime in on this for this >>> >> review) is >>> >> > whether or not all of it is the appropriate for a >>> >> thread's stack dump. >>> >> > For example, defined_classes is on the fringe of what >>> >> I would call >>> >> > generally useful info in a stack dump. Sure, there >>> >> might be that rare >>> >> > case when you need it, but how often compared to other >>> >> useful info >>> >> > maintained on a per thread basis. How many other bits >>> >> of useful info are >>> >> > not being printed in favor of defined_classes? It >>> >> seems you have more in >>> >> > the queue. How many? I'm worried about how cluttered >>> >> the stack dumps >>> >> > will get. Maybe we should add some sort of verbose >>> >> thread dumping >>> >> > option. Just a thought. >>> >> > >>> >> > As for the implementation, overall it looks good, but >>> >> I can't speak to >>> >> > whether or not you are doing proper accounting of >>> >> defined_classes and >>> >> > bytes allocated. You'll need input from someone with >>> >> more knowledge of >>> >> > those areas. We'll also need to do some testing to >>> >> make sure tool tests >>> >> > are not impacted. >>> >> > >>> >> > thanks, >>> >> > >>> >> > Chris >>> >> > >>> >> > On 4/30/18 2:51 AM, Haug, Gunter wrote: >>> >> > > Hi, >>> >> > > >>> >> > > this is an update to an RFR I posted on hotspot-dev, >>> >> but it is probably more suitable to post it here. Can I please have a >>> >> review and a sponsor for the following enhancement: >>> >> > > >>> >> > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v1 >>> >> > > https://bugs.openjdk.java.net/browse/JDK-8200720 >>> >> > > >>> >> > > We at SAP have extended the thread dumps (obtained >>> >> by jstack or jcmd) by several fields providing thread specific >>> >> information. These extensions are quite popular with our support >> team. >>> >> With some knowledge of the architecture of the application, they >> often >>> >> allow for quick and simple diagnosis of a running system. Therefore >> we >>> >> would like to contribute these enhancements. >>> >> > > >>> >> > > I took a few simple examples here, namely cpu time, >>> >> elapsed time since thread creation, bytes allocated and classes >>> >> defined by the thread and the pthread-id or equivalent on platforms >>> >> where it makes sense. Provided it is known how the application >> should >>> >> behave, a misbehaving thread can identified easily. >>> >> > > >>> >> > > There is no measurable overhead for this >>> >> enhancement. However, it may be a problem that the format of the >>> >> output is changed. Tools parsing the output may have to be changed. >>> >> > > >>> >> > > Here is an example of the output generated: >>> >> > > >>> >> > > ------------------------------------------------------ >>> >> > > >>> >> > > "main" #1 prio=5 os_prio=31 cpu=6300.65ms >>> >> elapsed=123.28s allocated=242236760B defined_classes=1725 >>> >> tid=0x00007fa13a806000 nid=0x1c03 pthread-id=0x109708000 waiting >> on >>> >> condition [0x0000000109707000] >>> >> > > java.lang.Thread.State: TIMED_WAITING (sleeping) >>> >> > > JavaThread state: _thread_blocked >>> >> > > Thread: 0x00007fa13a806000 [0x1c03] State: >>> >> _at_safepoint _has_called_back 0 _at_poll_safepoint 0 >>> >> > > JavaThread state: _thread_blocked >>> >> > > at java.lang.Thread.sleep(java.base/Native Method) >>> >> > > ... >>> >> > > ------------------------------------------------------ >>> >> > > >>> >> > > As mentioned above, we have a whole bunch of other >>> >> enhancements to the thread dump similar to this one and would be >>> >> willing to contribute them if there is any interest. >>> >> > > >>> >> > > Thanks and best regards, >>> >> > > Gunter >>> >> > > >>> >> > > >>> >> > >>> >> > >>> >> > >>> >> > >>> >> > >>> >> >>> > >>> > >>> >>> From david.holmes at oracle.com Wed Jun 6 03:48:30 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 6 Jun 2018 13:48:30 +1000 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> Message-ID: <33e2e693-1b5e-5e4f-cfd0-a8d5140b0eba@oracle.com> Hi Stuart, This all looks fine to me. One minor nit in threadPrimitiveDeprecation.html is whether the new statements you reference a particular Java version? It reads a little oddly to go through all the explanation and then say "Thread.x has been removed". In the spirit of @since you might say "has been removed as of JDK 11". Or arguably you could remove discussion of the removed methods altogether. Thanks, David On 6/06/2018 10:05 AM, Stuart Marks wrote: > [adding serviceability-dev] > > Hi serviceability folks, > > I'm in the process of removing Thread.destroy() and > Thread.stop(Throwable) from the Java SE API. Alan and David have pointed > out that there are some cross-references to Thread.stop(Throwable) in > the JDWP and JVMTI specs, as well as in the JDI ThreadReference API. > I've adjusted the relevant files. > > See please review this updated webrev: > > ??? http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/ > > For context, please see the full review thread on core-libs-dev: > > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053536.html > > > > On 6/4/18 11:11 PM, Alan Bateman wrote: >> The source file that is used to generate the JDWP protocol code and >> the spec is in make/data/jdwp/jdwp.spec. The JVM TI spec is at >> src/hotspot/share/prims/jvmti.xml. It should be okay to skip those for >> this change-set, assuming it is followed up quickly with another >> change-set to update those specs. > > OK. I took a look at those other files and they seem simple enough to > update as part of this changeset. If there aren't any objections from > anyone, might as well get this all done at once. > > Thanks, > > s'marks From thomas.stuefe at gmail.com Wed Jun 6 04:45:05 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 6 Jun 2018 06:45:05 +0200 Subject: RFR(S): 8200720: Print additional information in thread dump (times, allocated bytes etc.) In-Reply-To: <626701f8-b71b-58dc-f837-e5e1d81ef187@oracle.com> References: <56817AB5-88FC-4FB9-A271-7206EA69ECA7@sap.com> <2728581E-0E32-4911-B1A9-79AD6CBA81B0@sap.com> <082ca98f-9a68-15a0-95b5-c4d5c733ab36@oracle.com> <933cdacc-9b6f-c8b5-e244-558f38e7888c@oracle.com> <55806aa8-f09b-e9a8-b512-b9676029c494@oracle.com> <0F497BC6-2569-4EF6-9B59-E24A8E1B3A37@sap.com> <5dc08f65e83f4374b8e1f73f218c6d6d@sap.com> <626701f8-b71b-58dc-f837-e5e1d81ef187@oracle.com> Message-ID: On Wed, Jun 6, 2018 at 4:55 AM, David Holmes wrote: > Hi Goetz, > > On 5/06/2018 11:07 PM, Lindenmaier, Goetz wrote: >> >> Hi >> >> this discussion touched a lot of points so far, which seem >> to lead to different conclusions. >> >> I think we should look at the values printed: >> >> 1. cpu=6300.65ms elapsed=123.28s >> Overhead >> cpu time: >> * system calls at thread dump time >> elapsed time: >> * 1 system call at thread creation time >> * 1 64-bit field per thread for the thread start time >> * 1 system call at thread dump time >> >> As I understand, JDK-8143176 would have had to get the >> time at each locking, which is much more time critical >> than doing this during thread creation. While > > > Correct. > >> the time a lock was held would be much more useful, >> the ratio here, combined with knowledge about the application, >> could lead to the conclusion that the thread is wrongly >> blocked at much lower cost. > > > Agreed. I see no issue with unconditionally adding this information as it > should address some of the concerns of 8143176 as well. > >> >> 2. pthread-id=0x109708000 >> Overhead: >> * a field access at thread dump time. Negligible I would say. > > > I'd overlooked this addition, but it is of course the chunk of ifdef code in > osThread.cpp that I objected to. The problem I have here is the convoluted > mess of "thread identifiers" that we already have. We currently print: > > st->print("tid=" INTPTR_FORMAT " ", p2i(this)); > > which is just the address of the Thread/JavaThread object. Then: > > st->print("nid=0x%x ", thread_id()); > > where 'n' is supposed to represent "native thread id", which is: > - linux: kernel thread id from syscall gettid > - solaris: thread library identity from thr_create() or thr_self() > - windows: thread id from _beginthreadex > - OS X: kernel thread id from pthread_mach_thread_np(pthread_self()); > - Other BSD: kernel (?) thread id from syscall thr_self or getthrid > - AIX: thread library identity from pthread_create() or pthread_self() > > It's telling that the code Gunter added gets the thread library id on linux, > but a "kernel thread id" on other platforms! Do we want to see a thread > library id and a kernel thread id? IIRC nid is supposed be the id you need > to see the thread in a debugger. In which case I'm unclear what role the > thread id Gunter wants to add is playing? I don't think we need to see > kernel ids in general, and pthread-id isn't useful for debugging is it? > Yes, feels just wrong. Plus, not even sufficient, since you can have both pthread-id and kernel thread id, e.g. on AIX, and depending on your situation you may need both. But then, you may need a lot of things, and where do we draw the line? I admit I also feel uncomfortable with adding all this output to thread dump, which is noise in 99% of cases. (BTW what about tools parsing the thread dump in its current form? Wont we make a lot of scripts out there unhappy?) -- Code wise, at the very least, I would abstract this kind of coding into something like os::print_native_thread_id(outputStream* st) with the expectation of "only writes a few words, no newline". Platforms like AIX may then print both. (Note: not "get_native_thread_id()", since that would stipulates that there is only one and that is can be expressed as a common type over all architectures. Also, I would not want anyone to use it as a number.) -- As others before me (Kirk) noted, I find the information added by this patch useful. But after looking closely at the patch, I would strongly prefer a new separate jcmd for this. Lets call it "VM.thread_stats". You still would have to collect and carry the statistics by thread, but output would be separate both from the user perspective and also in coding - you would not have to force-shove it into Thread::print_on, for instance. Such a hypothetical command could be in the future extended with other statistics as we see fit. I do not see any advantage in overloading the thread dump vehicle in this way. Just my 5 cent. Best Regards, Thomas >> >> 3. allocated=242236760B defined_classes=1725 >> Overhead: >> * 1 64-bit field per thread. >> * counter increment on class creation >> >> Especially defined_classes seems to be controversial. >> As this only makes sense for Java threads, this could >> be printed in a line of it's own in Threads::print_on_error() >> (which already gets a flag to customize for jstack: >> print_concurrent_locks) by calling a dedicated function in Thread. >> No need for flag PrintExtendedThreadInfo thus. >> >> But I would propose to drop this information as it is >> too controversial. > > > Agreed. > >> This leaves the times and the pthread-id to be decided whether >> they are worth the cost. If so, I think they should be printed >> unconditional. > > > Agreed. With some refinement of the "thread id" issue. > >> If a flag is required to avoid bloating the info in the default case, >> I would include more infos, as os_prio and nid under that flag. > > > Of course those things exist for good reason. The native id is supposed to > be what aids in you matching what you see in our thread dumps with what you > see in a debugger. Priority is less of an issue these days, but there was a > time ... ;-) > >> As jstack is deprecated, changes to jstack could be skipped always >> disabling the new printouts. >> >> Finally, I would propose to move _allocated_bytes into >> threadStatisticInfo.hpp. > > > Didn't that part get dropped? (You don't need it for the time info or thread > id). That's a GC logging issue to me. > > Thanks, > David > > >> Best regards, >> Goetz. >> >> >> >> >>> -----Original Message----- >>> From: serviceability-dev [mailto:serviceability-dev- >>> bounces at openjdk.java.net] On Behalf Of David Holmes >>> Sent: Dienstag, 5. Juni 2018 04:53 >>> To: Haug, Gunter ; Chris Plummer >>> ; serviceability-dev >> dev at openjdk.java.net>; Langer, Christoph ; >>> hotspot-runtime-dev at openjdk.java.net >>> Subject: Re: RFR(S): 8200720: Print additional information in thread dump >>> (times, allocated bytes etc.) >>> >>> Hi Gunter, >>> >>> On 5/06/2018 1:27 AM, Haug, Gunter wrote: >>>> >>>> Hi David, Chris, >>>> >>>> would it be an option to unconditionally print the additional >>>> information? >>> >>> Regardless which way this is implemented it will be rather complicated >>> and it >>> only switches on/off a few field in the thread dump. >>> >>> I'm not convinced this is all suitable information for a thread stack >>> dump. I can see the time information being useful if using the dump to >>> try and diagnoze a hang or responsiveness issue. But the allocated-bytes >>> and classes-defined just doesn't seem useful in the context of a thread >>> dump to me. Anyone interested in allocation stats is going to be looking >>> at GC logs etc which is where this belongs. Ditto the class-stats belong >>> in some kind of classloading logging IMHO. >>> >>> I'm very reluctant to burden the implementation with capturing these >>> kinds of stats, just to use for diagnostic purposes that may not even be >>> asked for. I'm very much in the "you shouldn't pay for what you don't >>> use" camp in that regard. (See my comments in JDK-8143176 referenced by >>> Sean.) >>> >>> The ThreadStatisticInfo adds overhead to every thread object - and I'd >>> have to suspect there could be overlap with whatever flight recorder >>> sticks in there too. (Thread has become even more bloated in recent >>> time!). >>> >>> Cheers, >>> David >>> >>> >>>> Thanks, >>>> Gunter >>>> >>>> ?On 04.06.18, 01:13, "David Holmes" wrote: >>>> >>>> Hi Gunter, Chris, >>>> >>>> Sorry just trying to catch up and this is only a partial look so >>>> far ... >>>> >>>> BTW these changes are not limited to serviceability code so adding >>>> in >>>> runtime team as well. >>>> >>>> On 2/06/2018 9:12 AM, Chris Plummer wrote: >>>> > Hi Gunter, >>>> > >>>> > On 6/1/18 3:17 AM, Haug, Gunter wrote: >>>> >> Hi Chris, >>>> >> >>>> >> thanks for looking into this! >>>> >> >>>> >> Re the synchronization: The value is stored only in a VM >>>> operation at >>>> >> a safepoint by the VM thread. I was of the opinion, that this >>>> could >>>> >> not be interrupted by a second VM operation (of the same type). >>>> Or >>> >>> am >>>> >>>> >> I missing something else? >>>> > I was really thinking more about the temporary changing of >>>> > PrintExtendedThreadInfo, not the value stored in the VMOp. You >>>> may >>> >>> be be >>>> >>>> > correct that no more than one VMOp is executing, but while it is >>>> > executing it is has changed the value of >>>> PrintExtendedThreadInfo, >>> >>> which >>>> >>>> > might have an impact on anything else that triggers printing >>>> thread info >>>> > while the VMOp is executing. >>>> >>>> Even if nothing else can possibly be running during the safepoint >>>> I find >>>> it extremely bad form to change a command-line flag in this way, >>>> particularly from a safepoint! >>>> >>>> If this flag is intended to be dynamically enabled as part of a >>>> dcmd >>>> then it should also be a Manageable flag IMHO. >>>> >>>> That said ... >>>> >>>> >> I did think about passing an argument to the various print_on >>> >>> member >>>> >>>> >> functions of the thread classes, but this would require rather >>>> >> extensive code changes for a rather tiny extension. Therefore I >>>> feel >>>> >> doing it like this is the lesser evil. >>>> >>>> ... it's obviously not truly a global setting, but one that is >>>> needed on >>>> a per-print-request basis. The flag is just the default, but if >>>> the >>>> default is off you still want to enable extended printing on a >>>> per-request basis. >>>> >>>> The current print_on mechanics is not set up for this kind of >>>> flexibility. I think this needs more thought. >>>> >>>> --- >>>> >>>> Re osThread.cpp shared code changes ... I really hate to see all >>>> the >>>> ifdefs in there. Please add a pd_print_on function to support this >>>> if >>>> you truly want platform specific information. >>>> >>>> --- >>>> >>>> threadStatisticInfo.hpp >>>> >>>> typo: getElepsedTime() -> getElapsedTime() >>>> >>>> Thanks, >>>> David >>>> >>>> >>>> >>>> >> >>>> >> Re thread_dump(): I think it's correct or, well, at least it >>>> works ;-) >>>> >> In fact jstack will transfer the "-e" and "-l" in only one >>>> string, >>>> >> i.e. op->arg(0). >>>> > So you are saying that op->arg(0) is "-e -l" when using jstack? >>>> I think >>>> > you really need to clean up the parsing. As it stands right now >>>> passing, >>>> > I get the feeling that if a user erroneously asks for help by >>>> using >>>> > "jcmd Thread.Print -help", it will end up executing with >>>> -e an -l >>>> > enabled, and no failure for the invalid "-help" option. >>>> >> Christoph has already explained my reasoning. But I agree, >>>> it's not >>>> >> nice and I would be happy to do it like Christoph suggested. >>>> > I'll respond separately to his suggestion. >>>> > >>>> > thanks, >>>> > >>>> > Chris >>>> >> >>>> >> And typo fixed, sorry. >>>> >> >>>> >> Thanks again, >>>> >> Gunter >>>> >> >>>> >> On 01.06.18, 00:03, "Chris Plummer" >>> >>> wrote: >>>> >>>> >> >>>> >> Hi Gunter, >>>> >> globals.hpp: fix typo "informatiuon" >>>> >> I worry a little bit about the synchronizing (if that's >>>> the right >>>> >> word) >>>> >> of PrintExtendedThreadInfo and the dcmd's -e flag. When >>>> using -e, >>>> >> you >>>> >> are temporarily enabling PrintExtendedThreadInfo if it was >>>> false. >>>> >> This >>>> >> temporarily changes the behavior of thread dumps, and >>>> could >>>> >> impact other >>>> >> uses that happen in parallel. Also, could two simultaneous >>>> uses >>>> >> of -e >>>> >> result in PrintExtendedThreadInfo not getting restored >>>> properly? >>>> >> thread_dump() doesn't look right. It looks like you are >>>> iterating >>>> >> char >>>> >> by char over the argument, and expect something like "-el" >>>> to be >>>> >> specified rather then "-e -l". The loop should be >>>> iterating over >>>> >> op->arg(i), not op->arg(0)[i]. >>>> >> The rest of the changes look fine. >>>> >> thanks, >>>> >> Chris >>>> >> On 5/30/18 8:12 AM, Haug, Gunter wrote: >>>> >> > Hi all, >>>> >> > >>>> >> > As Chris proposed, I have made an the extended output >>>> >> switchable. There is an VM flag (PrintExtendedThreadInfo), >>>> which is >>>> >> false by default. Moreover, there is an Option (-e) which can >>>> be used >>>> >> with jcmd Thread.print as well as with jstack. The -e option >>>> >> essentially sets PrintExtendedThreadInfo true just for the >>>> respective >>>> >> thread dump. >>>> >> > >>>> >> > Here is the updated webrev: >>>> >> > >>>> >> > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v2 >>>> >> > >>>> >> > (https://bugs.openjdk.java.net/browse/JDK-8200720) >>>> >> > >>>> >> > Thanks, >>>> >> > Gunter >>>> >> > >>>> >> > >>>> >> > On 02.05.18, 17:07, "serviceability-dev on behalf of >>>> Haug, >>>> >> Gunter" >>> of >>>> >> gunter.haug at sap.com> wrote: >>>> >> > >>>> >> > Hi Chris, >>>> >> > >>>> >> > Thanks for looking into this. >>>> >> > You're right, there is a little more we have. We >>>> have >>>> >> implemented an IO tracing mechanism which - rather as a >>>> byproduct - >>>> >> keeps track of bytes read and written per thread. However, this >>>> of >>>> >> course requires changes not only in hotspot. We would be happy >>>> to >>>> >> contribute this as well, but this is a far bigger change and >>>> will >>>> >> probably lead to a far bigger discussion. Anyway, with the >>>> number of >>>> >> bytes read, the number of classes defined doesn't look that >>>> arbitrary >>>> >> anymore, as one can correlate IO to class loading. >>>> >> > >>>> >> > Regarding the verbose option I think that's a good >>>> idea! >>>> >> > >>>> >> > Thanks again, >>>> >> > Gunter >>>> >> > >>>> >> > On 01.05.18, 22:55, "Chris Plummer" >>>> >> wrote: >>>> >> > >>>> >> > Hi Gunter, >>>> >> > >>>> >> > The output you are adding is all useful. I >>>> think the >>>> >> question is (and >>>> >> > I'd like to see a few people chime in on this >>>> for this >>>> >> review) is >>>> >> > whether or not all of it is the appropriate for >>>> a >>>> >> thread's stack dump. >>>> >> > For example, defined_classes is on the fringe >>>> of what >>>> >> I would call >>>> >> > generally useful info in a stack dump. Sure, >>>> there >>>> >> might be that rare >>>> >> > case when you need it, but how often compared >>>> to other >>>> >> useful info >>>> >> > maintained on a per thread basis. How many >>>> other bits >>>> >> of useful info are >>>> >> > not being printed in favor of defined_classes? >>>> It >>>> >> seems you have more in >>>> >> > the queue. How many? I'm worried about how >>>> cluttered >>>> >> the stack dumps >>>> >> > will get. Maybe we should add some sort of >>>> verbose >>>> >> thread dumping >>>> >> > option. Just a thought. >>>> >> > >>>> >> > As for the implementation, overall it looks >>>> good, but >>>> >> I can't speak to >>>> >> > whether or not you are doing proper accounting >>>> of >>>> >> defined_classes and >>>> >> > bytes allocated. You'll need input from someone >>>> with >>>> >> more knowledge of >>>> >> > those areas. We'll also need to do some testing >>>> to >>>> >> make sure tool tests >>>> >> > are not impacted. >>>> >> > >>>> >> > thanks, >>>> >> > >>>> >> > Chris >>>> >> > >>>> >> > On 4/30/18 2:51 AM, Haug, Gunter wrote: >>>> >> > > Hi, >>>> >> > > >>>> >> > > this is an update to an RFR I posted on >>>> hotspot-dev, >>>> >> but it is probably more suitable to post it here. Can I please >>>> have a >>>> >> review and a sponsor for the following enhancement: >>>> >> > > >>>> >> > > >>>> http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v1 >>>> >> > > >>>> https://bugs.openjdk.java.net/browse/JDK-8200720 >>>> >> > > >>>> >> > > We at SAP have extended the thread dumps >>>> (obtained >>>> >> by jstack or jcmd) by several fields providing thread specific >>>> >> information. These extensions are quite popular with our >>>> support >>> >>> team. >>>> >>>> >> With some knowledge of the architecture of the application, >>>> they >>> >>> often >>>> >>>> >> allow for quick and simple diagnosis of a running system. >>>> Therefore >>> >>> we >>>> >>>> >> would like to contribute these enhancements. >>>> >> > > >>>> >> > > I took a few simple examples here, namely cpu >>>> time, >>>> >> elapsed time since thread creation, bytes allocated and classes >>>> >> defined by the thread and the pthread-id or equivalent on >>>> platforms >>>> >> where it makes sense. Provided it is known how the application >>> >>> should >>>> >>>> >> behave, a misbehaving thread can identified easily. >>>> >> > > >>>> >> > > There is no measurable overhead for this >>>> >> enhancement. However, it may be a problem that the format of >>>> the >>>> >> output is changed. Tools parsing the output may have to be >>>> changed. >>>> >> > > >>>> >> > > Here is an example of the output generated: >>>> >> > > >>>> >> > > >>>> ------------------------------------------------------ >>>> >> > > >>>> >> > > "main" #1 prio=5 os_prio=31 cpu=6300.65ms >>>> >> elapsed=123.28s allocated=242236760B defined_classes=1725 >>>> >> tid=0x00007fa13a806000 nid=0x1c03 pthread-id=0x109708000 >>>> waiting >>> >>> on >>>> >>>> >> condition [0x0000000109707000] >>>> >> > > java.lang.Thread.State: TIMED_WAITING >>>> (sleeping) >>>> >> > > JavaThread state: _thread_blocked >>>> >> > > Thread: 0x00007fa13a806000 [0x1c03] State: >>>> >> _at_safepoint _has_called_back 0 _at_poll_safepoint 0 >>>> >> > > JavaThread state: _thread_blocked >>>> >> > > at java.lang.Thread.sleep(java.base/Native >>>> Method) >>>> >> > > ... >>>> >> > > >>>> ------------------------------------------------------ >>>> >> > > >>>> >> > > As mentioned above, we have a whole bunch of >>>> other >>>> >> enhancements to the thread dump similar to this one and would >>>> be >>>> >> willing to contribute them if there is any interest. >>>> >> > > >>>> >> > > Thanks and best regards, >>>> >> > > Gunter >>>> >> > > >>>> >> > > >>>> >> > >>>> >> > >>>> >> > >>>> >> > >>>> >> > >>>> >> >>>> > >>>> > >>>> >>>> > From Alan.Bateman at oracle.com Wed Jun 6 10:40:24 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 6 Jun 2018 11:40:24 +0100 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> Message-ID: <7f96f3ae-246e-8960-93fc-0e93aad5dd04@oracle.com> On 06/06/2018 01:05, Stuart Marks wrote: > [adding serviceability-dev] > > Hi serviceability folks, > > I'm in the process of removing Thread.destroy() and > Thread.stop(Throwable) from the Java SE API. Alan and David have > pointed out that there are some cross-references to > Thread.stop(Throwable) in the JDWP and JVMTI specs, as well as in the > JDI ThreadReference API. I've adjusted the relevant files. > > See please review this updated webrev: > > ??? http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/ Have you considered removing the "What about Thread.stop(Throwable)" section completely from the threadPrimitiveDeprecation doc? A new developer reading the javadoc in 2019 shouldn't need to read about a removed method. Otherwise I think this looks good, including the updates to the JDWP and JVM TI specs. -Alan From daniel.daugherty at oracle.com Wed Jun 6 15:51:29 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 6 Jun 2018 11:51:29 -0400 Subject: RFR: JDK-8169718: nsk/jdb/locals/locals002: ERROR: Cannot find boolVar with expected value: false In-Reply-To: References: <5B082D2E.7000408@oracle.com> <5d58fa2b-7dda-db64-0280-1e19e791a86d@oracle.com> Message-ID: The two prompts are for two different things so I don't think that removing one of the prompts is the right solution either. In the case of an asynchronous command like 'cont', you get the '' right away because the command processor is ready for another command. You will also get the '' prompt when the breakpoint is hit because that identifies the thread that posted the breakpoint event. If it takes a long time to hit the breakpoint, then it will be a long time before the breakpoint prompt is seen. Likewise, if the breakpoint is hit quickly, then the breakpoint prompt will be seen more quickly and will sometimes interfere with the command processor prompt. That's normal for an async command. One possible way to fix this kind of issue is to structure the test flow to slightly delay the target thread so it takes longer to get to the breakpoint. '' will have a better chance to be printed before (and not colliding with) the '' prompt. So if the test program has: ??? ;? // breakpoint here change it to something like: ??? try { Thread.sleep(100); } catch (InterruptedException ie) {}; ??? ;? // breakpoint here Of course, I'm assuming here that the test driver code is looking for the breakpoint to be hit (and not for ''). Jerry and I did find a few jdb based tests looking for the wrong prompt... Dan On 6/2/18 7:02 AM, gary.adams at oracle.com wrote: > On 6/1/18 6:54 PM, Chris Plummer wrote: >> Hi Gary, >> >> I'm a bit unclear on one point. According to your description, should >> we be seeing two prompts for commands like "cont", and "step"? When >> you say we are expecting: >> >> ? "Breakpoint hit:" >> ?? >> >> Isn't it really: >> >> ?? >> ? "Breakpoint hit:" >> ?? > Let me stretch out the log a little bit > The is main[1] or Thread-1[1], etc. > and not distinguished for command or event completion. > > > ?? stop at > > ?? stop at > > ?? cont > > ?? cont > ?? "Breakpoint hit:" > > ? "Breakpoint hit:" > > ? > > > The source of the command or event prompts > was verified with a stacktrace in the printPrompt() > method. >> >> I figure this must be the case, because your fix removes the printing >> of a prompt, and we still need one prompt, so there must have been >> two to start with. I think in the above the last is being >> produced sequentially right after the "Breakpoint hit:" output, so is >> never an issue. The first comes from the printPrompt() call >> after issuing the "cont" command, and it's possible to hit the >> breakpoint and start producing more output before printPrompt() has >> completed, thus causing the problem you are seeing. > printPrompt() is a very quick operation and is not being interrupted. > > But the output from the event handler is produced incrementally. > ?? - event header, e.g. "Breakpont hit:", "Step completed:", etc. > ?? - from vmInterrupted() > ???? - printCurrentLocation() > ???? - monitor command output (if any) > ???? - printPrompt() >> >> There seems to be some purpose to printing this first prompt and I'm >> not so sure eliminating it is the right thing to do. I think it might >> be there to indicate that the command has been initiated, because >> there could be some delay before the breakpoint is hit. So the issue >> then is how to make sure this prompt appears in the input stream >> before any of the "Step completed:" or "Breakpoint hit:" output. >> Possibly event processing should be blocked until executeCommand() >> completes. > The primary purposes of prompts is for "user interactive" nudge to > type another command. > But the failures are from tests that are using the presence of a > prompt as an ACK of the > last command. This "prompt-protocol", does not include any sequence > numbers , > does not pair prompt to command or event, and does not block during > any multipart message formation. > > To prevent the error we are observing we would have to buffer all of > the event output > > ?? "Breakpoint hit:" > ??? > > Another way to look at the current failure is to say the event handler > output > has been interrupted by a command sent prematurely. This would never > be seen at user speeds typing in commands. > > I should also mention any other output from the user application can also > interfere with the command and event output streams. I have seen > stray prompts in the logs from System.out.println(). That was the > reason earlier I was looking at changing the tests to look for > specific prompts before continuing to the next command. > > \gra >> thanks, >> >> Chris >> >> On 5/26/18 3:50 AM, gary.adams at oracle.com wrote: >>> This is a review request for a previously closed test bug. >>> The test was recently moved to the open repos, and the >>> proposed fix is in the open code. >>> >>> ? Webrev: http://cr.openjdk.java.net/~gadams/8169718/webrev/ >>> >>> >>> -------- Forwarded Message -------- >>> Subject:???? RFR: JDK-8169718: nsk/jdb/locals/locals002: ERROR: >>> Cannot find boolVar with expected value: false >>> Date:???? Fri, 25 May 2018 11:35:10 -0400 >>> From:???? Gary Adams >>> Reply-To:???? gary.adams at oracle.com >>> >>> >>> >>> >>> >>> The jdb tests use stdin to send commands to a jdb process >>> and parses the stdout to determine if a command was >>> successful and when the process is prompting for new commands >>> to be sent. >>> >>> Some commands are synchronous, so when the command is completed >>> a new prompt is sent back immediately. >>> >>> Some commands are asynchronous, so there could be a delay >>> until a breakpoint is reached. The event handler then sends a prompt >>> when the application thread is stopped and new jdb commands can be >>> sent. >>> >>> The problem causing the intermittent failures was a corruption in the >>> output stream when prompts were being sent at the wrong times. >>> >>> Instead of receiving >>> ?? "Breakpoint hit:" >>> ??? >>> >>> the log contained >>> ?? "Breakpoint hit:" >>> >>> Once out of sync, jdb commands were being sent prematurely >>> and the wrong values were being compared against expected behavior. >>> The simple fix proposed here recognizes that commands like "cont", >>> "step" and "next" are asynchronous commands and should not send back >>> a prompt immediately. Instead. the event handler will deliver the >>> next prompt >>> when the next "Breakpoint hit:" or "Step completed:" state change >>> occurs. >>> >>> The bulk of the testing was done on windows-x64-debug builds where the >>> intermittent failures were observed in ~5 in 1000 testruns. The fix has >>> also been tested on linux-x64-debug, solaris-sparcv9-debug, >>> and macosx-x64-debug, even though the failures have never been reported >>> against those platforms. >>> >>> Failures have been observed in many of the nsk/jdb tests with >>> similar corrupted >>> output streams, but never directly associated with this issue before. >>> >>> ??redefine001, caught_exception002, locals002, eval001, next001, >>> ? stop_at003, step002, print002, trace001, step_up001, read001, >>> ? clear004, kill001, set001 >>> >>> >> >> > From thomas.stuefe at gmail.com Wed Jun 6 16:05:55 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 6 Jun 2018 18:05:55 +0200 Subject: RFR(s): 8203343: VM.{metaspace|classloaders|classhierarchy...} jcmd should show invocation targets for Generated{Method|Constructor}AccessorImpl classes Message-ID: Dear all, may I please have feedback and if possible reviews for this small addition: CR: https://bugs.openjdk.java.net/browse/JDK-8203343 Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/webrev.00/webrev/ (Note: this patch goes atop of https://bugs.openjdk.java.net/browse/JDK-8203682, which is currently up for RFR). --- When analyzing situations involving a lot of reflection, one often stares at walls of "GeneratedXXXAccessorXXX" classes. These names are generated and not at all helpful in analyzing the problem (e.g. which component in a server node does this much reflection and hence uses so much metaspace). This patch adds the ability to print out invocation targets additionally to class names if the class is a generated accessor - for now this ability has been added to VM.metaspace, VM.classloaders and VM.class_hierarchy. Example output from "VM.class_hierarchy": |--jdk.internal.reflect.MagicAccessorImpl/null | |--jdk.internal.reflect.ConstructorAccessorImpl/null | | |--jdk.internal.reflect.GeneratedConstructorAccessor18/0x00007f9ee8350c10 (invokes: org/springframework/boot/context/properties/ConfigurationPropertiesBindingPostProcessorRegistrar:: ()V) | | |--jdk.internal.reflect.GeneratedConstructorAccessor17/0x00007f9ee8349c00 (invokes: org/springframework/boot/context/properties/EnableConfigurationPropertiesImportSelector$ConfigurationPropertiesBeanRegistrar:: ()V) | |--jdk.internal.reflect.MethodAccessorImpl/null | | |--jdk.internal.reflect.GeneratedMethodAccessor23/0x00007f9ec8329b60 (invokes: org/apache/tomcat/util/modeler/AttributeInfo::setIs (Z)V) | | |--jdk.internal.reflect.GeneratedMethodAccessor22/0x00007f9ee831bc70 (invokes: org/springframework/boot/autoconfigure/SpringBootApplication::exclude ()[Ljava/lang/Class;) See here more examples: "VM.class_hierarchy" http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/example-VM.class_hierarchy.txt "VM.classloaders show-classes" http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/example-VM.classloaders.txt ---- Note: I am not sure if this is a fit for hotspot-runtime or serviceability. Sorry for crossposting. Thank you, Thomas From kirk.pepperdine at gmail.com Wed Jun 6 16:18:09 2018 From: kirk.pepperdine at gmail.com (Kirk Pepperdine) Date: Wed, 6 Jun 2018 18:18:09 +0200 Subject: RFR(s): 8203343: VM.{metaspace|classloaders|classhierarchy...} jcmd should show invocation targets for Generated{Method|Constructor}AccessorImpl classes In-Reply-To: References: Message-ID: > On Jun 6, 2018, at 6:05 PM, Thomas St?fe wrote: > > Dear all, > > may I please have feedback and if possible reviews for this small addition: I can see the need to visualize this but the output looks easily parsable so it all looks good from my perspective. Not an official review ? Kirk From iris.clark at oracle.com Wed Jun 6 16:28:15 2018 From: iris.clark at oracle.com (Iris Clark) Date: Wed, 6 Jun 2018 09:28:15 -0700 (PDT) Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: <7f96f3ae-246e-8960-93fc-0e93aad5dd04@oracle.com> References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> <7f96f3ae-246e-8960-93fc-0e93aad5dd04@oracle.com> Message-ID: <9d09f2b9-775d-4c95-b54b-37c426c62d3d@default> Hi, Stuart. I think you need to make changes to this file too: http://hg.openjdk.java.net/jdk/jdk/file/tip/src/java.base/share/classes/java/lang/doc-files/threadPrimitiveDeprecation.html Thanks, iris -----Original Message----- From: Alan Bateman Sent: Wednesday, June 6, 2018 3:40 AM To: Stuart Marks ; serviceability-dev Cc: core-libs-dev Subject: Re: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) On 06/06/2018 01:05, Stuart Marks wrote: > [adding serviceability-dev] > > Hi serviceability folks, > > I'm in the process of removing Thread.destroy() and > Thread.stop(Throwable) from the Java SE API. Alan and David have > pointed out that there are some cross-references to > Thread.stop(Throwable) in the JDWP and JVMTI specs, as well as in the > JDI ThreadReference API. I've adjusted the relevant files. > > See please review this updated webrev: > > ??? http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/ Have you considered removing the "What about Thread.stop(Throwable)" section completely from the threadPrimitiveDeprecation doc? A new developer reading the javadoc in 2019 shouldn't need to read about a removed method. Otherwise I think this looks good, including the updates to the JDWP and JVM TI specs. -Alan From shade at redhat.com Wed Jun 6 16:30:54 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 6 Jun 2018 18:30:54 +0200 Subject: RFR: 8204180: Implementation: JEP 318: Epsilon GC (round 5) Message-ID: <2ee03550-2b1e-b873-5ef3-26312e2a760e@redhat.com> Hi, This is fifth (and hopefully final) round of code review for Epsilon GC changes. It includes the fixes done as the result of fourth round of reviews, mostly in Serviceability. The build parts are the same since last few reviews, so this is not posted to build-dev at . Webrev: http://cr.openjdk.java.net/~shade/epsilon/webrev.09/ If we are good -- should somebody, e.g. Project Lead ack this? -- I am going to push this with the following changeset metadata: 8204180: Implementation: JEP 318: Epsilon, A No-Op Garbage Collector Summary: Introduce Epsilon GC Reviewed-by: rkennke, ihse, pliden, eosterlund, lmesnik, jgeorge Builds: server X {x86_64, x86_32, aarch64, arm32, ppc64le, s390x} minimal X {x86, x86_64} zero X {x86_64} Testing: gc/epsilon on x86_64 Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From gary.adams at oracle.com Wed Jun 6 17:07:21 2018 From: gary.adams at oracle.com (Gary Adams) Date: Wed, 06 Jun 2018 13:07:21 -0400 Subject: RFR: JDK-8169718: nsk/jdb/locals/locals002: ERROR: Cannot find boolVar with expected value: false In-Reply-To: References: <5B082D2E.7000408@oracle.com> <5d58fa2b-7dda-db64-0280-1e19e791a86d@oracle.com> Message-ID: <5B1814C9.6030807@oracle.com> It may be sufficient when the command prompt is delivered to introduce a brief pause to allow an asynchronous event to be processed before another command can be sent. e.g. throttling the stream of commands being sent diff --git a/test/hotspot/jtreg/vmTestbase/nsk/share/jdb/Jdb.java b/test/hotspot/jtreg/vmTestbase/nsk/share/jdb/Jdb.java --- a/test/hotspot/jtreg/vmTestbase/nsk/share/jdb/Jdb.java +++ b/test/hotspot/jtreg/vmTestbase/nsk/share/jdb/Jdb.java @@ -348,7 +348,17 @@ int startPos = stdoutBuffer.length(); sendCommand(command); - return receiveReply(startPos, compoundPromptOnly, count); + String[] reply = receiveReply(startPos, compoundPromptOnly, count); + + if (asyncCommand(command)) { + // delay to allow debuggee time to process async command + try { + Thread.sleep(100); + } catch (InterruptedException e) { + // ignored + } + } + return reply; } On 6/6/18, 11:51 AM, Daniel D. Daugherty wrote: > The two prompts are for two different things so I don't think > that removing one of the prompts is the right solution either. > > In the case of an asynchronous command like 'cont', you get > the '' right away because the command > processor is ready for another command. You will also get the > '' prompt when the > breakpoint is hit because that identifies the thread that posted > the breakpoint event. If it takes a long time to hit the breakpoint, > then it will be a long time before the breakpoint prompt is seen. > Likewise, if the breakpoint is hit quickly, then the breakpoint > prompt will be seen more quickly and will sometimes interfere with > the command processor prompt. That's normal for an async command. > > One possible way to fix this kind of issue is to structure the > test flow to slightly delay the target thread so it takes longer > to get to the breakpoint. '' will have > a better chance to be printed before (and not colliding with) the > '' prompt. > > So if the test program has: > > ; // breakpoint here > > change it to something like: > > try { Thread.sleep(100); } catch (InterruptedException ie) {}; > ; // breakpoint here > > Of course, I'm assuming here that the test driver code is looking > for the breakpoint to be hit (and not for ''). > Jerry and I did find a few jdb based tests looking for the wrong > prompt... > > Dan > > > On 6/2/18 7:02 AM, gary.adams at oracle.com wrote: >> On 6/1/18 6:54 PM, Chris Plummer wrote: >>> Hi Gary, >>> >>> I'm a bit unclear on one point. According to your description, >>> should we be seeing two prompts for commands like "cont", and >>> "step"? When you say we are expecting: >>> >>> "Breakpoint hit:" >>> >>> >>> Isn't it really: >>> >>> >>> "Breakpoint hit:" >>> >> Let me stretch out the log a little bit >> The is main[1] or Thread-1[1], etc. >> and not distinguished for command or event completion. >> >> >> stop at >> >> stop at >> >> cont >> >> cont >> "Breakpoint hit:" >> >> "Breakpoint hit:" >> >> >> >> >> The source of the command or event prompts >> was verified with a stacktrace in the printPrompt() >> method. >>> >>> I figure this must be the case, because your fix removes the >>> printing of a prompt, and we still need one prompt, so there must >>> have been two to start with. I think in the above the last >>> is being produced sequentially right after the "Breakpoint hit:" >>> output, so is never an issue. The first comes from the >>> printPrompt() call after issuing the "cont" command, and it's >>> possible to hit the breakpoint and start producing more output >>> before printPrompt() has completed, thus causing the problem you are >>> seeing. >> printPrompt() is a very quick operation and is not being interrupted. >> >> But the output from the event handler is produced incrementally. >> - event header, e.g. "Breakpont hit:", "Step completed:", etc. >> - from vmInterrupted() >> - printCurrentLocation() >> - monitor command output (if any) >> - printPrompt() >>> >>> There seems to be some purpose to printing this first prompt and I'm >>> not so sure eliminating it is the right thing to do. I think it >>> might be there to indicate that the command has been initiated, >>> because there could be some delay before the breakpoint is hit. So >>> the issue then is how to make sure this prompt appears in the input >>> stream before any of the "Step completed:" or "Breakpoint hit:" >>> output. Possibly event processing should be blocked until >>> executeCommand() completes. >> The primary purposes of prompts is for "user interactive" nudge to >> type another command. >> But the failures are from tests that are using the presence of a >> prompt as an ACK of the >> last command. This "prompt-protocol", does not include any sequence >> numbers , >> does not pair prompt to command or event, and does not block during >> any multipart message formation. >> >> To prevent the error we are observing we would have to buffer all of >> the event output >> >> "Breakpoint hit:" >> >> >> Another way to look at the current failure is to say the event >> handler output >> has been interrupted by a command sent prematurely. This would never >> be seen at user speeds typing in commands. >> >> I should also mention any other output from the user application can >> also >> interfere with the command and event output streams. I have seen >> stray prompts in the logs from System.out.println(). That was the >> reason earlier I was looking at changing the tests to look for >> specific prompts before continuing to the next command. >> >> \gra >>> thanks, >>> >>> Chris >>> >>> On 5/26/18 3:50 AM, gary.adams at oracle.com wrote: >>>> This is a review request for a previously closed test bug. >>>> The test was recently moved to the open repos, and the >>>> proposed fix is in the open code. >>>> >>>> Webrev: http://cr.openjdk.java.net/~gadams/8169718/webrev/ >>>> >>>> >>>> -------- Forwarded Message -------- >>>> Subject: RFR: JDK-8169718: nsk/jdb/locals/locals002: ERROR: >>>> Cannot find boolVar with expected value: false >>>> Date: Fri, 25 May 2018 11:35:10 -0400 >>>> From: Gary Adams >>>> Reply-To: gary.adams at oracle.com >>>> >>>> >>>> >>>> >>>> >>>> The jdb tests use stdin to send commands to a jdb process >>>> and parses the stdout to determine if a command was >>>> successful and when the process is prompting for new commands >>>> to be sent. >>>> >>>> Some commands are synchronous, so when the command is completed >>>> a new prompt is sent back immediately. >>>> >>>> Some commands are asynchronous, so there could be a delay >>>> until a breakpoint is reached. The event handler then sends a prompt >>>> when the application thread is stopped and new jdb commands can be >>>> sent. >>>> >>>> The problem causing the intermittent failures was a corruption in the >>>> output stream when prompts were being sent at the wrong times. >>>> >>>> Instead of receiving >>>> "Breakpoint hit:" >>>> >>>> >>>> the log contained >>>> "Breakpoint hit:" >>>> >>>> Once out of sync, jdb commands were being sent prematurely >>>> and the wrong values were being compared against expected behavior. >>>> The simple fix proposed here recognizes that commands like "cont", >>>> "step" and "next" are asynchronous commands and should not send back >>>> a prompt immediately. Instead. the event handler will deliver the >>>> next prompt >>>> when the next "Breakpoint hit:" or "Step completed:" state change >>>> occurs. >>>> >>>> The bulk of the testing was done on windows-x64-debug builds where the >>>> intermittent failures were observed in ~5 in 1000 testruns. The fix >>>> has >>>> also been tested on linux-x64-debug, solaris-sparcv9-debug, >>>> and macosx-x64-debug, even though the failures have never been >>>> reported >>>> against those platforms. >>>> >>>> Failures have been observed in many of the nsk/jdb tests with >>>> similar corrupted >>>> output streams, but never directly associated with this issue before. >>>> >>>> redefine001, caught_exception002, locals002, eval001, next001, >>>> stop_at003, step002, print002, trace001, step_up001, read001, >>>> clear004, kill001, set001 >>>> >>>> >>> >>> >> > From chris.plummer at oracle.com Wed Jun 6 17:06:35 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 6 Jun 2018 10:06:35 -0700 Subject: RFR: JDK-8169718: nsk/jdb/locals/locals002: ERROR: Cannot find boolVar with expected value: false In-Reply-To: References: <5B082D2E.7000408@oracle.com> <5d58fa2b-7dda-db64-0280-1e19e791a86d@oracle.com> Message-ID: On 6/6/18 8:51 AM, Daniel D. Daugherty wrote: > The two prompts are for two different things so I don't think > that removing one of the prompts is the right solution either. > > In the case of an asynchronous command like 'cont', you get > the '' right away because the command > processor is ready for another command. You will also get the > '' prompt when the > breakpoint is hit because that identifies the thread that posted > the breakpoint event. If it takes a long time to hit the breakpoint, > then it will be a long time before the breakpoint prompt is seen. > Likewise, if the breakpoint is hit quickly, then the breakpoint > prompt will be seen more quickly and will sometimes interfere with > the command processor prompt. That's normal for an async command. > That was the point I was trying to make. I also felt that printing both prompts was appropriate. > One possible way to fix this kind of issue is to structure the > test flow to slightly delay the target thread so it takes longer > to get to the breakpoint. '' will have > a better chance to be printed before (and not colliding with) the > '' prompt. > > So if the test program has: > > ??? ;? // breakpoint here > > change it to something like: > > ??? try { Thread.sleep(100); } catch (InterruptedException ie) {}; > ??? ;? // breakpoint here > > Of course, I'm assuming here that the test driver code is looking > for the breakpoint to be hit (and not for ''). > Jerry and I did find a few jdb based tests looking for the wrong > prompt... The issue is really with the delivery of the first prompt. It can be intermingled with the output from the breakpoint. Yes, your solution might work most of the time (still not 100% guaranteed) when dealing with breakpoints, but is not a solution when dealing with this same problem occurring while single stepping. My suggestion was to have processing of breakpoints and single stepping events block until executeCommand() completed and had printed the first prompt. I'm not sure if this is feasible or practical. thanks, Chris > > Dan > > > On 6/2/18 7:02 AM, gary.adams at oracle.com wrote: >> On 6/1/18 6:54 PM, Chris Plummer wrote: >>> Hi Gary, >>> >>> I'm a bit unclear on one point. According to your description, >>> should we be seeing two prompts for commands like "cont", and >>> "step"? When you say we are expecting: >>> >>> ? "Breakpoint hit:" >>> ?? >>> >>> Isn't it really: >>> >>> ?? >>> ? "Breakpoint hit:" >>> ?? >> Let me stretch out the log a little bit >> The is main[1] or Thread-1[1], etc. >> and not distinguished for command or event completion. >> >> >> ?? stop at >> >> ?? stop at >> >> ?? cont >> >> ?? cont >> ?? "Breakpoint hit:" >> >> ? "Breakpoint hit:" >> >> ? >> >> >> The source of the command or event prompts >> was verified with a stacktrace in the printPrompt() >> method. >>> >>> I figure this must be the case, because your fix removes the >>> printing of a prompt, and we still need one prompt, so there must >>> have been two to start with. I think in the above the last >>> is being produced sequentially right after the "Breakpoint hit:" >>> output, so is never an issue. The first comes from the >>> printPrompt() call after issuing the "cont" command, and it's >>> possible to hit the breakpoint and start producing more output >>> before printPrompt() has completed, thus causing the problem you are >>> seeing. >> printPrompt() is a very quick operation and is not being interrupted. >> >> But the output from the event handler is produced incrementally. >> ?? - event header, e.g. "Breakpont hit:", "Step completed:", etc. >> ?? - from vmInterrupted() >> ???? - printCurrentLocation() >> ???? - monitor command output (if any) >> ???? - printPrompt() >>> >>> There seems to be some purpose to printing this first prompt and I'm >>> not so sure eliminating it is the right thing to do. I think it >>> might be there to indicate that the command has been initiated, >>> because there could be some delay before the breakpoint is hit. So >>> the issue then is how to make sure this prompt appears in the input >>> stream before any of the "Step completed:" or "Breakpoint hit:" >>> output. Possibly event processing should be blocked until >>> executeCommand() completes. >> The primary purposes of prompts is for "user interactive" nudge to >> type another command. >> But the failures are from tests that are using the presence of a >> prompt as an ACK of the >> last command. This "prompt-protocol", does not include any sequence >> numbers , >> does not pair prompt to command or event, and does not block during >> any multipart message formation. >> >> To prevent the error we are observing we would have to buffer all of >> the event output >> >> ?? "Breakpoint hit:" >> ??? >> >> Another way to look at the current failure is to say the event >> handler output >> has been interrupted by a command sent prematurely. This would never >> be seen at user speeds typing in commands. >> >> I should also mention any other output from the user application can >> also >> interfere with the command and event output streams. I have seen >> stray prompts in the logs from System.out.println(). That was the >> reason earlier I was looking at changing the tests to look for >> specific prompts before continuing to the next command. >> >> \gra >>> thanks, >>> >>> Chris >>> >>> On 5/26/18 3:50 AM, gary.adams at oracle.com wrote: >>>> This is a review request for a previously closed test bug. >>>> The test was recently moved to the open repos, and the >>>> proposed fix is in the open code. >>>> >>>> ? Webrev: http://cr.openjdk.java.net/~gadams/8169718/webrev/ >>>> >>>> >>>> -------- Forwarded Message -------- >>>> Subject:???? RFR: JDK-8169718: nsk/jdb/locals/locals002: ERROR: >>>> Cannot find boolVar with expected value: false >>>> Date:???? Fri, 25 May 2018 11:35:10 -0400 >>>> From:???? Gary Adams >>>> Reply-To:???? gary.adams at oracle.com >>>> >>>> >>>> >>>> >>>> >>>> The jdb tests use stdin to send commands to a jdb process >>>> and parses the stdout to determine if a command was >>>> successful and when the process is prompting for new commands >>>> to be sent. >>>> >>>> Some commands are synchronous, so when the command is completed >>>> a new prompt is sent back immediately. >>>> >>>> Some commands are asynchronous, so there could be a delay >>>> until a breakpoint is reached. The event handler then sends a prompt >>>> when the application thread is stopped and new jdb commands can be >>>> sent. >>>> >>>> The problem causing the intermittent failures was a corruption in the >>>> output stream when prompts were being sent at the wrong times. >>>> >>>> Instead of receiving >>>> ?? "Breakpoint hit:" >>>> ??? >>>> >>>> the log contained >>>> ?? "Breakpoint hit:" >>>> >>>> Once out of sync, jdb commands were being sent prematurely >>>> and the wrong values were being compared against expected behavior. >>>> The simple fix proposed here recognizes that commands like "cont", >>>> "step" and "next" are asynchronous commands and should not send back >>>> a prompt immediately. Instead. the event handler will deliver the >>>> next prompt >>>> when the next "Breakpoint hit:" or "Step completed:" state change >>>> occurs. >>>> >>>> The bulk of the testing was done on windows-x64-debug builds where the >>>> intermittent failures were observed in ~5 in 1000 testruns. The fix >>>> has >>>> also been tested on linux-x64-debug, solaris-sparcv9-debug, >>>> and macosx-x64-debug, even though the failures have never been >>>> reported >>>> against those platforms. >>>> >>>> Failures have been observed in many of the nsk/jdb tests with >>>> similar corrupted >>>> output streams, but never directly associated with this issue before. >>>> >>>> ??redefine001, caught_exception002, locals002, eval001, next001, >>>> ? stop_at003, step002, print002, trace001, step_up001, read001, >>>> ? clear004, kill001, set001 >>>> >>>> >>> >>> >> > From chris.plummer at oracle.com Wed Jun 6 17:26:27 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 6 Jun 2018 10:26:27 -0700 Subject: RFR: JDK-8169718: nsk/jdb/locals/locals002: ERROR: Cannot find boolVar with expected value: false In-Reply-To: <5B1814C9.6030807@oracle.com> References: <5B082D2E.7000408@oracle.com> <5d58fa2b-7dda-db64-0280-1e19e791a86d@oracle.com> <5B1814C9.6030807@oracle.com> Message-ID: <1d47721c-3498-92ee-997e-1d9fa216ea6f@oracle.com> Hi Gary, I thought the issue was having 1 command like "cont" or "step" producing two prompts, and the first prompt ending up in the middle of the output for the breakpoint. So instead of: main[1] cont main[1] "Breakpoint hit:" main[1] We sometimes get: main[1] cont "Breakpoint hit:" main[1] main[1] I don't think a delay on the receiving end like you have added can effect this. You need to make sure the main[1] produced for the "cont" command is in the output stream before handling the breakpoint event. Dan's solution was a bit different in that it actually delayed reaching the breakpoint. I don't think your solution does the same. Chris On 6/6/18 10:07 AM, Gary Adams wrote: > It may be sufficient when the command prompt is delivered > to introduce a brief pause to allow an asynchronous event to > be processed before another command can be sent. e.g. > throttling the stream of commands being sent > > diff --git a/test/hotspot/jtreg/vmTestbase/nsk/share/jdb/Jdb.java > b/test/hotspot/jtreg/vmTestbase/nsk/share/jdb/Jdb.java > --- a/test/hotspot/jtreg/vmTestbase/nsk/share/jdb/Jdb.java > +++ b/test/hotspot/jtreg/vmTestbase/nsk/share/jdb/Jdb.java > @@ -348,7 +348,17 @@ > > ???????? int startPos = stdoutBuffer.length(); > ???????? sendCommand(command); > -??????? return receiveReply(startPos, compoundPromptOnly, count); > +??????? String[] reply = receiveReply(startPos, compoundPromptOnly, > count); > + > +??????? if (asyncCommand(command)) { > +??????????? // delay to allow debuggee time to process async command > +??????????? try { > +??????????????? Thread.sleep(100); > +??????????? } catch (InterruptedException e) { > +??????????????? // ignored > +??????????? } > +??????? } > +??????? return reply; > ???? } > > > On 6/6/18, 11:51 AM, Daniel D. Daugherty wrote: >> The two prompts are for two different things so I don't think >> that removing one of the prompts is the right solution either. >> >> In the case of an asynchronous command like 'cont', you get >> the '' right away because the command >> processor is ready for another command. You will also get the >> '' prompt when the >> breakpoint is hit because that identifies the thread that posted >> the breakpoint event. If it takes a long time to hit the breakpoint, >> then it will be a long time before the breakpoint prompt is seen. >> Likewise, if the breakpoint is hit quickly, then the breakpoint >> prompt will be seen more quickly and will sometimes interfere with >> the command processor prompt. That's normal for an async command. >> >> One possible way to fix this kind of issue is to structure the >> test flow to slightly delay the target thread so it takes longer >> to get to the breakpoint. '' will have >> a better chance to be printed before (and not colliding with) the >> '' prompt. >> >> So if the test program has: >> >> ;? // breakpoint here >> >> change it to something like: >> >> ??? try { Thread.sleep(100); } catch (InterruptedException ie) {}; >> ;? // breakpoint here >> >> Of course, I'm assuming here that the test driver code is looking >> for the breakpoint to be hit (and not for >> ''). >> Jerry and I did find a few jdb based tests looking for the wrong >> prompt... >> >> Dan >> >> >> On 6/2/18 7:02 AM, gary.adams at oracle.com wrote: >>> On 6/1/18 6:54 PM, Chris Plummer wrote: >>>> Hi Gary, >>>> >>>> I'm a bit unclear on one point. According to your description, >>>> should we be seeing two prompts for commands like "cont", and >>>> "step"? When you say we are expecting: >>>> >>>> ? "Breakpoint hit:" >>>> >>>> >>>> Isn't it really: >>>> >>>> >>>> ? "Breakpoint hit:" >>>> >>> Let me stretch out the log a little bit >>> The is main[1] or Thread-1[1], etc. >>> and not distinguished for command or event completion. >>> >>> >>> ?? stop at >>> >>> ?? stop at >>> >>> ?? cont >>> >>> ?? cont >>> ?? "Breakpoint hit:" >>> >>> ? "Breakpoint hit:" >>> >>> >>> >>> >>> The source of the command or event prompts >>> was verified with a stacktrace in the printPrompt() >>> method. >>>> >>>> I figure this must be the case, because your fix removes the >>>> printing of a prompt, and we still need one prompt, so there must >>>> have been two to start with. I think in the above the last >>>> is being produced sequentially right after the "Breakpoint hit:" >>>> output, so is never an issue. The first comes from the >>>> printPrompt() call after issuing the "cont" command, and it's >>>> possible to hit the breakpoint and start producing more output >>>> before printPrompt() has completed, thus causing the problem you >>>> are seeing. >>> printPrompt() is a very quick operation and is not being interrupted. >>> >>> But the output from the event handler is produced incrementally. >>> ?? - event header, e.g. "Breakpont hit:", "Step completed:", etc. >>> ?? - from vmInterrupted() >>> ???? - printCurrentLocation() >>> ???? - monitor command output (if any) >>> ???? - printPrompt() >>>> >>>> There seems to be some purpose to printing this first prompt and >>>> I'm not so sure eliminating it is the right thing to do. I think it >>>> might be there to indicate that the command has been initiated, >>>> because there could be some delay before the breakpoint is hit. So >>>> the issue then is how to make sure this prompt appears in the input >>>> stream before any of the "Step completed:" or "Breakpoint hit:" >>>> output. Possibly event processing should be blocked until >>>> executeCommand() completes. >>> The primary purposes of prompts is for "user interactive" nudge to >>> type another command. >>> But the failures are from tests that are using the presence of a >>> prompt as an ACK of the >>> last command. This "prompt-protocol", does not include any sequence >>> numbers , >>> does not pair prompt to command or event, and does not block during >>> any multipart message formation. >>> >>> To prevent the error we are observing we would have to buffer all of >>> the event output >>> >>> ?? "Breakpoint hit:" >>> >>> >>> Another way to look at the current failure is to say the event >>> handler output >>> has been interrupted by a command sent prematurely. This would never >>> be seen at user speeds typing in commands. >>> >>> I should also mention any other output from the user application can >>> also >>> interfere with the command and event output streams. I have seen >>> stray prompts in the logs from System.out.println(). That was the >>> reason earlier I was looking at changing the tests to look for >>> specific prompts before continuing to the next command. >>> >>> \gra >>>> thanks, >>>> >>>> Chris >>>> >>>> On 5/26/18 3:50 AM, gary.adams at oracle.com wrote: >>>>> This is a review request for a previously closed test bug. >>>>> The test was recently moved to the open repos, and the >>>>> proposed fix is in the open code. >>>>> >>>>> ? Webrev: http://cr.openjdk.java.net/~gadams/8169718/webrev/ >>>>> >>>>> >>>>> -------- Forwarded Message -------- >>>>> Subject:???? RFR: JDK-8169718: nsk/jdb/locals/locals002: ERROR: >>>>> Cannot find boolVar with expected value: false >>>>> Date:???? Fri, 25 May 2018 11:35:10 -0400 >>>>> From:???? Gary Adams >>>>> Reply-To:???? gary.adams at oracle.com >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> The jdb tests use stdin to send commands to a jdb process >>>>> and parses the stdout to determine if a command was >>>>> successful and when the process is prompting for new commands >>>>> to be sent. >>>>> >>>>> Some commands are synchronous, so when the command is completed >>>>> a new prompt is sent back immediately. >>>>> >>>>> Some commands are asynchronous, so there could be a delay >>>>> until a breakpoint is reached. The event handler then sends a prompt >>>>> when the application thread is stopped and new jdb commands can be >>>>> sent. >>>>> >>>>> The problem causing the intermittent failures was a corruption in the >>>>> output stream when prompts were being sent at the wrong times. >>>>> >>>>> Instead of receiving >>>>> ?? "Breakpoint hit:" >>>>> >>>>> >>>>> the log contained >>>>> ?? "Breakpoint hit:" >>>>> >>>>> Once out of sync, jdb commands were being sent prematurely >>>>> and the wrong values were being compared against expected behavior. >>>>> The simple fix proposed here recognizes that commands like "cont", >>>>> "step" and "next" are asynchronous commands and should not send back >>>>> a prompt immediately. Instead. the event handler will deliver the >>>>> next prompt >>>>> when the next "Breakpoint hit:" or "Step completed:" state change >>>>> occurs. >>>>> >>>>> The bulk of the testing was done on windows-x64-debug builds where >>>>> the >>>>> intermittent failures were observed in ~5 in 1000 testruns. The >>>>> fix has >>>>> also been tested on linux-x64-debug, solaris-sparcv9-debug, >>>>> and macosx-x64-debug, even though the failures have never been >>>>> reported >>>>> against those platforms. >>>>> >>>>> Failures have been observed in many of the nsk/jdb tests with >>>>> similar corrupted >>>>> output streams, but never directly associated with this issue before. >>>>> >>>>> ? redefine001, caught_exception002, locals002, eval001, next001, >>>>> ? stop_at003, step002, print002, trace001, step_up001, read001, >>>>> ? clear004, kill001, set001 >>>>> >>>>> >>>> >>>> >>> >> > From gary.adams at oracle.com Wed Jun 6 17:44:31 2018 From: gary.adams at oracle.com (Gary Adams) Date: Wed, 06 Jun 2018 13:44:31 -0400 Subject: RFR: JDK-8169718: nsk/jdb/locals/locals002: ERROR: Cannot find boolVar with expected value: false In-Reply-To: <1d47721c-3498-92ee-997e-1d9fa216ea6f@oracle.com> References: <5B082D2E.7000408@oracle.com> <5d58fa2b-7dda-db64-0280-1e19e791a86d@oracle.com> <5B1814C9.6030807@oracle.com> <1d47721c-3498-92ee-997e-1d9fa216ea6f@oracle.com> Message-ID: <5B181D7F.1040300@oracle.com> On 6/6/18, 1:26 PM, Chris Plummer wrote: > Hi Gary, > > I thought the issue was having 1 command like "cont" or "step" > producing two prompts, and the first prompt ending up in the middle of > the output for the breakpoint. So instead of: > > main[1] cont > main[1] > "Breakpoint hit:" > main[1] > > We sometimes get: > > main[1] cont > "Breakpoint hit:" main[1] > main[1] If you look at the commands sent : stop at location1 stop at location2 cont cont the expectation is that breakpoint will be encountered and then the test will continue til another breakpoint is encountered. The problem only occurs because the commands are sent so quickly a race condition is introduced between the command processing and the event handling in the debuggee process. Since prompts are written both from the command thread and the event handler, the output can end up interspersed. > > I don't think a delay on the receiving end like you have added can > effect this. You need to make sure the main[1] produced for the "cont" > command is in the output stream before handling the breakpoint event. > Dan's solution was a bit different in that it actually delayed > reaching the breakpoint. I don't think your solution does the same. My goal is to delay the sending of the next command, because the debuggee is not actually ready for the command. > > Chris > > On 6/6/18 10:07 AM, Gary Adams wrote: >> It may be sufficient when the command prompt is delivered >> to introduce a brief pause to allow an asynchronous event to >> be processed before another command can be sent. e.g. >> throttling the stream of commands being sent >> >> diff --git a/test/hotspot/jtreg/vmTestbase/nsk/share/jdb/Jdb.java >> b/test/hotspot/jtreg/vmTestbase/nsk/share/jdb/Jdb.java >> --- a/test/hotspot/jtreg/vmTestbase/nsk/share/jdb/Jdb.java >> +++ b/test/hotspot/jtreg/vmTestbase/nsk/share/jdb/Jdb.java >> @@ -348,7 +348,17 @@ >> >> int startPos = stdoutBuffer.length(); >> sendCommand(command); >> - return receiveReply(startPos, compoundPromptOnly, count); >> + String[] reply = receiveReply(startPos, compoundPromptOnly, >> count); >> + >> + if (asyncCommand(command)) { >> + // delay to allow debuggee time to process async command >> + try { >> + Thread.sleep(100); >> + } catch (InterruptedException e) { >> + // ignored >> + } >> + } >> + return reply; >> } >> >> >> On 6/6/18, 11:51 AM, Daniel D. Daugherty wrote: >>> The two prompts are for two different things so I don't think >>> that removing one of the prompts is the right solution either. >>> >>> In the case of an asynchronous command like 'cont', you get >>> the '' right away because the command >>> processor is ready for another command. You will also get the >>> '' prompt when the >>> breakpoint is hit because that identifies the thread that posted >>> the breakpoint event. If it takes a long time to hit the breakpoint, >>> then it will be a long time before the breakpoint prompt is seen. >>> Likewise, if the breakpoint is hit quickly, then the breakpoint >>> prompt will be seen more quickly and will sometimes interfere with >>> the command processor prompt. That's normal for an async command. >>> >>> One possible way to fix this kind of issue is to structure the >>> test flow to slightly delay the target thread so it takes longer >>> to get to the breakpoint. '' will have >>> a better chance to be printed before (and not colliding with) the >>> '' prompt. >>> >>> So if the test program has: >>> >>> ; // breakpoint here >>> >>> change it to something like: >>> >>> try { Thread.sleep(100); } catch (InterruptedException ie) {}; >>> ; // breakpoint here >>> >>> Of course, I'm assuming here that the test driver code is looking >>> for the breakpoint to be hit (and not for >>> ''). >>> Jerry and I did find a few jdb based tests looking for the wrong >>> prompt... >>> >>> Dan >>> >>> >>> On 6/2/18 7:02 AM, gary.adams at oracle.com wrote: >>>> On 6/1/18 6:54 PM, Chris Plummer wrote: >>>>> Hi Gary, >>>>> >>>>> I'm a bit unclear on one point. According to your description, >>>>> should we be seeing two prompts for commands like "cont", and >>>>> "step"? When you say we are expecting: >>>>> >>>>> "Breakpoint hit:" >>>>> >>>>> >>>>> Isn't it really: >>>>> >>>>> >>>>> "Breakpoint hit:" >>>>> >>>> Let me stretch out the log a little bit >>>> The is main[1] or Thread-1[1], etc. >>>> and not distinguished for command or event completion. >>>> >>>> >>>> stop at >>>> >>>> stop at >>>> >>>> cont >>>> >>>> cont >>>> "Breakpoint hit:" >>>> >>>> "Breakpoint hit:" >>>> >>>> >>>> >>>> >>>> The source of the command or event prompts >>>> was verified with a stacktrace in the printPrompt() >>>> method. >>>>> >>>>> I figure this must be the case, because your fix removes the >>>>> printing of a prompt, and we still need one prompt, so there must >>>>> have been two to start with. I think in the above the last >>>>> is being produced sequentially right after the >>>>> "Breakpoint hit:" output, so is never an issue. The first >>>>> comes from the printPrompt() call after issuing the "cont" >>>>> command, and it's possible to hit the breakpoint and start >>>>> producing more output before printPrompt() has completed, thus >>>>> causing the problem you are seeing. >>>> printPrompt() is a very quick operation and is not being interrupted. >>>> >>>> But the output from the event handler is produced incrementally. >>>> - event header, e.g. "Breakpont hit:", "Step completed:", etc. >>>> - from vmInterrupted() >>>> - printCurrentLocation() >>>> - monitor command output (if any) >>>> - printPrompt() >>>>> >>>>> There seems to be some purpose to printing this first prompt and >>>>> I'm not so sure eliminating it is the right thing to do. I think >>>>> it might be there to indicate that the command has been initiated, >>>>> because there could be some delay before the breakpoint is hit. So >>>>> the issue then is how to make sure this prompt appears in the >>>>> input stream before any of the "Step completed:" or "Breakpoint >>>>> hit:" output. Possibly event processing should be blocked until >>>>> executeCommand() completes. >>>> The primary purposes of prompts is for "user interactive" nudge to >>>> type another command. >>>> But the failures are from tests that are using the presence of a >>>> prompt as an ACK of the >>>> last command. This "prompt-protocol", does not include any sequence >>>> numbers , >>>> does not pair prompt to command or event, and does not block during >>>> any multipart message formation. >>>> >>>> To prevent the error we are observing we would have to buffer all >>>> of the event output >>>> >>>> "Breakpoint hit:" >>>> >>>> >>>> Another way to look at the current failure is to say the event >>>> handler output >>>> has been interrupted by a command sent prematurely. This would never >>>> be seen at user speeds typing in commands. >>>> >>>> I should also mention any other output from the user application >>>> can also >>>> interfere with the command and event output streams. I have seen >>>> stray prompts in the logs from System.out.println(). That was the >>>> reason earlier I was looking at changing the tests to look for >>>> specific prompts before continuing to the next command. >>>> >>>> \gra >>>>> thanks, >>>>> >>>>> Chris >>>>> >>>>> On 5/26/18 3:50 AM, gary.adams at oracle.com wrote: >>>>>> This is a review request for a previously closed test bug. >>>>>> The test was recently moved to the open repos, and the >>>>>> proposed fix is in the open code. >>>>>> >>>>>> Webrev: http://cr.openjdk.java.net/~gadams/8169718/webrev/ >>>>>> >>>>>> >>>>>> -------- Forwarded Message -------- >>>>>> Subject: RFR: JDK-8169718: nsk/jdb/locals/locals002: ERROR: >>>>>> Cannot find boolVar with expected value: false >>>>>> Date: Fri, 25 May 2018 11:35:10 -0400 >>>>>> From: Gary Adams >>>>>> Reply-To: gary.adams at oracle.com >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> The jdb tests use stdin to send commands to a jdb process >>>>>> and parses the stdout to determine if a command was >>>>>> successful and when the process is prompting for new commands >>>>>> to be sent. >>>>>> >>>>>> Some commands are synchronous, so when the command is completed >>>>>> a new prompt is sent back immediately. >>>>>> >>>>>> Some commands are asynchronous, so there could be a delay >>>>>> until a breakpoint is reached. The event handler then sends a prompt >>>>>> when the application thread is stopped and new jdb commands can >>>>>> be sent. >>>>>> >>>>>> The problem causing the intermittent failures was a corruption in >>>>>> the >>>>>> output stream when prompts were being sent at the wrong times. >>>>>> >>>>>> Instead of receiving >>>>>> "Breakpoint hit:" >>>>>> >>>>>> >>>>>> the log contained >>>>>> "Breakpoint hit:" >>>>>> >>>>>> Once out of sync, jdb commands were being sent prematurely >>>>>> and the wrong values were being compared against expected behavior. >>>>>> The simple fix proposed here recognizes that commands like "cont", >>>>>> "step" and "next" are asynchronous commands and should not send back >>>>>> a prompt immediately. Instead. the event handler will deliver the >>>>>> next prompt >>>>>> when the next "Breakpoint hit:" or "Step completed:" state change >>>>>> occurs. >>>>>> >>>>>> The bulk of the testing was done on windows-x64-debug builds >>>>>> where the >>>>>> intermittent failures were observed in ~5 in 1000 testruns. The >>>>>> fix has >>>>>> also been tested on linux-x64-debug, solaris-sparcv9-debug, >>>>>> and macosx-x64-debug, even though the failures have never been >>>>>> reported >>>>>> against those platforms. >>>>>> >>>>>> Failures have been observed in many of the nsk/jdb tests with >>>>>> similar corrupted >>>>>> output streams, but never directly associated with this issue >>>>>> before. >>>>>> >>>>>> redefine001, caught_exception002, locals002, eval001, next001, >>>>>> stop_at003, step002, print002, trace001, step_up001, read001, >>>>>> clear004, kill001, set001 >>>>>> >>>>>> >>>>> >>>>> >>>> >>> >> > > From thomas.stuefe at gmail.com Wed Jun 6 17:55:00 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 6 Jun 2018 19:55:00 +0200 Subject: RFR(s): 8203343: VM.{metaspace|classloaders|classhierarchy...} jcmd should show invocation targets for Generated{Method|Constructor}AccessorImpl classes In-Reply-To: References: Message-ID: On Wed, Jun 6, 2018 at 6:18 PM, Kirk Pepperdine wrote: > >> On Jun 6, 2018, at 6:05 PM, Thomas St?fe wrote: >> >> Dear all, >> >> may I please have feedback and if possible reviews for this small addition: > > > I can see the need to visualize this but the output looks easily parsable so it all looks good from my perspective. > > Not an official review > ? Kirk > Thank you Kirk. From serguei.spitsyn at oracle.com Wed Jun 6 17:58:39 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 6 Jun 2018 10:58:39 -0700 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Wed Jun 6 18:20:38 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 6 Jun 2018 11:20:38 -0700 Subject: RFR: JDK-8169718: nsk/jdb/locals/locals002: ERROR: Cannot find boolVar with expected value: false In-Reply-To: <5B181D7F.1040300@oracle.com> References: <5B082D2E.7000408@oracle.com> <5d58fa2b-7dda-db64-0280-1e19e791a86d@oracle.com> <5B1814C9.6030807@oracle.com> <1d47721c-3498-92ee-997e-1d9fa216ea6f@oracle.com> <5B181D7F.1040300@oracle.com> Message-ID: <9794304d-d166-554a-399d-0f8b53fe8786@oracle.com> An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Wed Jun 6 18:23:39 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 6 Jun 2018 11:23:39 -0700 Subject: RFR 8203033: [Testbug] vmTestbase/nsk/jdi/TypeComponent/isSynthetic/issynthetic002/TestDescription.java fails with nestmates In-Reply-To: <9C78D8B6-F7C0-44EE-93B3-D8C22CAF2628@oracle.com> References: <9C78D8B6-F7C0-44EE-93B3-D8C22CAF2628@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From gary.adams at oracle.com Wed Jun 6 18:52:27 2018 From: gary.adams at oracle.com (Gary Adams) Date: Wed, 06 Jun 2018 14:52:27 -0400 Subject: RFR: JDK-8169718: nsk/jdb/locals/locals002: ERROR: Cannot find boolVar with expected value: false In-Reply-To: <9794304d-d166-554a-399d-0f8b53fe8786@oracle.com> References: <5B082D2E.7000408@oracle.com> <5d58fa2b-7dda-db64-0280-1e19e791a86d@oracle.com> <5B1814C9.6030807@oracle.com> <1d47721c-3498-92ee-997e-1d9fa216ea6f@oracle.com> <5B181D7F.1040300@oracle.com> <9794304d-d166-554a-399d-0f8b53fe8786@oracle.com> Message-ID: <5B182D6B.8060206@oracle.com> On 6/6/18, 2:20 PM, Chris Plummer wrote: > If that were the case, shouldn't the delay be added to sendCommand() > instead? > > I'm still a bit unsure about how this sequence is suppose to work in > the first place: > > sendCommand(command); > return receiveReply(startPos, compoundPromptOnly, count); Send command "cont" Receive the reply "main[1]" At this point the delay allows the event handler to run in the debuggee process. > > I guess the assumption is that the first prompt will always be a > simple one? Is that the case? If so, then the expectation is that we > will see: > > main[1] cont > > > "Breakpoint hit:" > main[1] > > But then we can still see, even after your fix: > > main[1] cont > "Breakpoint hit:" > > main[1] I'm not clear on what you are saying in this example. > > However, I think maybe the issue is different for step commands, which > seem to result in two compound prompts. Remember your example: > >> Here's another recent failure with eval001 >> 1. cont and Break hit >> 2. step and Step completed (partial reply) >> 3. eval and error with second half of Step completed >> >> *====================================* >> Sending command: cont >> >> reply[0]: > >> reply[1]: >> Breakpoint hit: "thread=main", >> nsk.jdb.eval.eval001.eval001a.lastBreak(), line=23 bci=0 >> reply[2]: >> 23 static void lastBreak () {} >> reply[3]: >> >> reply[4]: * >> ==================================== >> >> *main[1] >> Sending command: step >> >> reply[0]: >> reply[1]: >> Step completed: main[1] >> *====================================* >> Sending command: eval nsk.jdb.eval.eval001.eval001a.myStaticField >> *reply[0]: "thread=main", nsk.jdb.eval.eval001.eval001a.runIt(), >> line=36 bci=53* >> reply[1]: >> 36 } >> reply[2]: >> >> reply[3]: >> >> *====================================* >> main[1] >> # ERROR: jdb failed to report value of expression: >> nsk.jdb.eval.eval001.eval001a.myStaticField >> >> # ERROR: expected : -2147483648 ; >> *# ERROR: reported: "thread=main", >> nsk.jdb.eval.eval001.eval001a.runIt(), line=36 bci=53* >> >> >> Finally the sequence from the jdb.session file. Notice the main[1] prompt >> embedded in the Step completed reply. >> >> *main[1]* cont >> > >> Breakpoint hit: "thread=main", >> nsk.jdb.eval.eval001.eval001a.lastBreak(), line=23 bci=0 >> 23 static void lastBreak () {} >> >> *main[1] *step >> >> Step completed: *main[1]* "thread=main", >> nsk.jdb.eval.eval001.eval001a.runIt(), line=36 bci=53 >> 36 } >> >> *main[1]* eval nsk.jdb.eval.eval001.eval001a.myStaticField > The part that uses "cont" is fine. The problems started when the step > command was issued. Rather than receiving: > > Step completed: "thread=main", nsk.jdb.eval.eval001.eval001a.runIt(), > line=36 bci=53 > > We got: > > Step completed: main[1] > *====================================* > Sending command: eval nsk.jdb.eval.eval001.eval001a.myStaticField > reply[0]: "thread=main", nsk.jdb.eval.eval001.eval001a.runIt(), > line=36 bci=53 > > I don't see your change addressing this. The step command was > considered to be done as soon as the main[] prompt arrived, even > though it came right in the middle of the full "Step completed: ..." > message. Your delay won't change this. Yes, it will make sure the rest > of the reply has arrived, but the part after main[1] will not be > considered part of the step reply because it will terminate at the > main[1] prompt. With the proposed change to delay after receiving a reply after command, we would see *====================================* Sending command: cont reply[0]: > *==================================== delay after command* Breakpoint hit: "thread=main", nsk.jdb.eval.eval001.eval001a.lastBreak(), line=23 bci=0 23 static void lastBreak () {}* *main[1] *====================================* Sending command: step main[1] *==================================== delay after command* Step completed: *"thread=main", nsk.jdb.eval.eval001.eval001a.runIt(), line=36 bci=53* 36 } *====================================* Sending command: eval nsk.jdb.eval.eval001.eval001a.myStaticField main[1] *==================================== delay after command* ... I'll try some testing over night. > > Chris > > On 6/6/18 10:44 AM, Gary Adams wrote: >> On 6/6/18, 1:26 PM, Chris Plummer wrote: >>> Hi Gary, >>> >>> I thought the issue was having 1 command like "cont" or "step" >>> producing two prompts, and the first prompt ending up in the middle >>> of the output for the breakpoint. So instead of: >>> >>> main[1] cont >>> main[1] >>> "Breakpoint hit:" >>> main[1] >>> >>> We sometimes get: >>> >>> main[1] cont >>> "Breakpoint hit:" main[1] >>> main[1] >> If you look at the commands sent : >> >> stop at location1 >> stop at location2 >> cont >> cont >> >> the expectation is that breakpoint will be encountered and then >> the test will continue til another breakpoint is encountered. >> >> The problem only occurs because the commands are sent >> so quickly a race condition is introduced between the >> command processing and the event handling in the debuggee >> process. Since prompts are written both from the command >> thread and the event handler, the output can end up interspersed. >> >>> >>> I don't think a delay on the receiving end like you have added can >>> effect this. You need to make sure the main[1] produced for the >>> "cont" command is in the output stream before handling the >>> breakpoint event. Dan's solution was a bit different in that it >>> actually delayed reaching the breakpoint. I don't think your >>> solution does the same. >> My goal is to delay the sending of the next command, >> because the debuggee is not actually ready for the command. >>> >>> Chris >>> >>> On 6/6/18 10:07 AM, Gary Adams wrote: >>>> It may be sufficient when the command prompt is delivered >>>> to introduce a brief pause to allow an asynchronous event to >>>> be processed before another command can be sent. e.g. >>>> throttling the stream of commands being sent >>>> >>>> diff --git a/test/hotspot/jtreg/vmTestbase/nsk/share/jdb/Jdb.java >>>> b/test/hotspot/jtreg/vmTestbase/nsk/share/jdb/Jdb.java >>>> --- a/test/hotspot/jtreg/vmTestbase/nsk/share/jdb/Jdb.java >>>> +++ b/test/hotspot/jtreg/vmTestbase/nsk/share/jdb/Jdb.java >>>> @@ -348,7 +348,17 @@ >>>> >>>> int startPos = stdoutBuffer.length(); >>>> sendCommand(command); >>>> - return receiveReply(startPos, compoundPromptOnly, count); >>>> + String[] reply = receiveReply(startPos, >>>> compoundPromptOnly, count); >>>> + >>>> + if (asyncCommand(command)) { >>>> + // delay to allow debuggee time to process async command >>>> + try { >>>> + Thread.sleep(100); >>>> + } catch (InterruptedException e) { >>>> + // ignored >>>> + } >>>> + } >>>> + return reply; >>>> } >>>> >>>> >>>> On 6/6/18, 11:51 AM, Daniel D. Daugherty wrote: >>>>> The two prompts are for two different things so I don't think >>>>> that removing one of the prompts is the right solution either. >>>>> >>>>> In the case of an asynchronous command like 'cont', you get >>>>> the '' right away because the command >>>>> processor is ready for another command. You will also get the >>>>> '' prompt when the >>>>> breakpoint is hit because that identifies the thread that posted >>>>> the breakpoint event. If it takes a long time to hit the breakpoint, >>>>> then it will be a long time before the breakpoint prompt is seen. >>>>> Likewise, if the breakpoint is hit quickly, then the breakpoint >>>>> prompt will be seen more quickly and will sometimes interfere with >>>>> the command processor prompt. That's normal for an async command. >>>>> >>>>> One possible way to fix this kind of issue is to structure the >>>>> test flow to slightly delay the target thread so it takes longer >>>>> to get to the breakpoint. '' will have >>>>> a better chance to be printed before (and not colliding with) the >>>>> '' prompt. >>>>> >>>>> So if the test program has: >>>>> >>>>> ; // breakpoint here >>>>> >>>>> change it to something like: >>>>> >>>>> try { Thread.sleep(100); } catch (InterruptedException ie) {}; >>>>> ; // breakpoint here >>>>> >>>>> Of course, I'm assuming here that the test driver code is looking >>>>> for the breakpoint to be hit (and not for >>>>> ''). >>>>> Jerry and I did find a few jdb based tests looking for the wrong >>>>> prompt... >>>>> >>>>> Dan >>>>> >>>>> >>>>> On 6/2/18 7:02 AM, gary.adams at oracle.com wrote: >>>>>> On 6/1/18 6:54 PM, Chris Plummer wrote: >>>>>>> Hi Gary, >>>>>>> >>>>>>> I'm a bit unclear on one point. According to your description, >>>>>>> should we be seeing two prompts for commands like "cont", and >>>>>>> "step"? When you say we are expecting: >>>>>>> >>>>>>> "Breakpoint hit:" >>>>>>> >>>>>>> >>>>>>> Isn't it really: >>>>>>> >>>>>>> >>>>>>> "Breakpoint hit:" >>>>>>> >>>>>> Let me stretch out the log a little bit >>>>>> The is main[1] or Thread-1[1], etc. >>>>>> and not distinguished for command or event completion. >>>>>> >>>>>> >>>>>> stop at >>>>>> >>>>>> stop at >>>>>> >>>>>> cont >>>>>> >>>>>> cont >>>>>> "Breakpoint hit:" >>>>>> >>>>>> "Breakpoint hit:" >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> The source of the command or event prompts >>>>>> was verified with a stacktrace in the printPrompt() >>>>>> method. >>>>>>> >>>>>>> I figure this must be the case, because your fix removes the >>>>>>> printing of a prompt, and we still need one prompt, so there >>>>>>> must have been two to start with. I think in the above the last >>>>>>> is being produced sequentially right after the >>>>>>> "Breakpoint hit:" output, so is never an issue. The first >>>>>>> comes from the printPrompt() call after issuing the >>>>>>> "cont" command, and it's possible to hit the breakpoint and >>>>>>> start producing more output before printPrompt() has completed, >>>>>>> thus causing the problem you are seeing. >>>>>> printPrompt() is a very quick operation and is not being >>>>>> interrupted. >>>>>> >>>>>> But the output from the event handler is produced incrementally. >>>>>> - event header, e.g. "Breakpont hit:", "Step completed:", etc. >>>>>> - from vmInterrupted() >>>>>> - printCurrentLocation() >>>>>> - monitor command output (if any) >>>>>> - printPrompt() >>>>>>> >>>>>>> There seems to be some purpose to printing this first prompt and >>>>>>> I'm not so sure eliminating it is the right thing to do. I think >>>>>>> it might be there to indicate that the command has been >>>>>>> initiated, because there could be some delay before the >>>>>>> breakpoint is hit. So the issue then is how to make sure this >>>>>>> prompt appears in the input stream before any of the "Step >>>>>>> completed:" or "Breakpoint hit:" output. Possibly event >>>>>>> processing should be blocked until executeCommand() completes. >>>>>> The primary purposes of prompts is for "user interactive" nudge >>>>>> to type another command. >>>>>> But the failures are from tests that are using the presence of a >>>>>> prompt as an ACK of the >>>>>> last command. This "prompt-protocol", does not include any >>>>>> sequence numbers , >>>>>> does not pair prompt to command or event, and does not block during >>>>>> any multipart message formation. >>>>>> >>>>>> To prevent the error we are observing we would have to buffer all >>>>>> of the event output >>>>>> >>>>>> "Breakpoint hit:" >>>>>> >>>>>> >>>>>> Another way to look at the current failure is to say the event >>>>>> handler output >>>>>> has been interrupted by a command sent prematurely. This would never >>>>>> be seen at user speeds typing in commands. >>>>>> >>>>>> I should also mention any other output from the user application >>>>>> can also >>>>>> interfere with the command and event output streams. I have seen >>>>>> stray prompts in the logs from System.out.println(). That was the >>>>>> reason earlier I was looking at changing the tests to look for >>>>>> specific prompts before continuing to the next command. >>>>>> >>>>>> \gra >>>>>>> thanks, >>>>>>> >>>>>>> Chris >>>>>>> >>>>>>> On 5/26/18 3:50 AM, gary.adams at oracle.com wrote: >>>>>>>> This is a review request for a previously closed test bug. >>>>>>>> The test was recently moved to the open repos, and the >>>>>>>> proposed fix is in the open code. >>>>>>>> >>>>>>>> Webrev: http://cr.openjdk.java.net/~gadams/8169718/webrev/ >>>>>>>> >>>>>>>> >>>>>>>> -------- Forwarded Message -------- >>>>>>>> Subject: RFR: JDK-8169718: nsk/jdb/locals/locals002: ERROR: >>>>>>>> Cannot find boolVar with expected value: false >>>>>>>> Date: Fri, 25 May 2018 11:35:10 -0400 >>>>>>>> From: Gary Adams >>>>>>>> Reply-To: gary.adams at oracle.com >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> The jdb tests use stdin to send commands to a jdb process >>>>>>>> and parses the stdout to determine if a command was >>>>>>>> successful and when the process is prompting for new commands >>>>>>>> to be sent. >>>>>>>> >>>>>>>> Some commands are synchronous, so when the command is completed >>>>>>>> a new prompt is sent back immediately. >>>>>>>> >>>>>>>> Some commands are asynchronous, so there could be a delay >>>>>>>> until a breakpoint is reached. The event handler then sends a >>>>>>>> prompt >>>>>>>> when the application thread is stopped and new jdb commands can >>>>>>>> be sent. >>>>>>>> >>>>>>>> The problem causing the intermittent failures was a corruption >>>>>>>> in the >>>>>>>> output stream when prompts were being sent at the wrong times. >>>>>>>> >>>>>>>> Instead of receiving >>>>>>>> "Breakpoint hit:" >>>>>>>> >>>>>>>> >>>>>>>> the log contained >>>>>>>> "Breakpoint hit:" >>>>>>>> >>>>>>>> Once out of sync, jdb commands were being sent prematurely >>>>>>>> and the wrong values were being compared against expected >>>>>>>> behavior. >>>>>>>> The simple fix proposed here recognizes that commands like "cont", >>>>>>>> "step" and "next" are asynchronous commands and should not send >>>>>>>> back >>>>>>>> a prompt immediately. Instead. the event handler will deliver >>>>>>>> the next prompt >>>>>>>> when the next "Breakpoint hit:" or "Step completed:" state >>>>>>>> change occurs. >>>>>>>> >>>>>>>> The bulk of the testing was done on windows-x64-debug builds >>>>>>>> where the >>>>>>>> intermittent failures were observed in ~5 in 1000 testruns. The >>>>>>>> fix has >>>>>>>> also been tested on linux-x64-debug, solaris-sparcv9-debug, >>>>>>>> and macosx-x64-debug, even though the failures have never been >>>>>>>> reported >>>>>>>> against those platforms. >>>>>>>> >>>>>>>> Failures have been observed in many of the nsk/jdb tests with >>>>>>>> similar corrupted >>>>>>>> output streams, but never directly associated with this issue >>>>>>>> before. >>>>>>>> >>>>>>>> redefine001, caught_exception002, locals002, eval001, next001, >>>>>>>> stop_at003, step002, print002, trace001, step_up001, read001, >>>>>>>> clear004, kill001, set001 >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniil.x.titov at oracle.com Wed Jun 6 19:09:20 2018 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Wed, 06 Jun 2018 12:09:20 -0700 Subject: RFR 8203033: [Testbug] vmTestbase/nsk/jdi/TypeComponent/isSynthetic/issynthetic002/TestDescription.java fails with nestmates In-Reply-To: References: <9C78D8B6-F7C0-44EE-93B3-D8C22CAF2628@oracle.com> Message-ID: <5FA679F2-6A90-4A5A-8D79-42CC7C05BD04@oracle.com> Thank you, Serguei and David, for reviewing this change. Please review a new version ?of the change that corrects the diagnostic message as Serguei suggested ?and fixes typos. Webrev: http://cr.openjdk.java.net/~dtitov/8203033/webrev.03 Issue: https://bugs.openjdk.java.net/browse/JDK-8203033 Best regards, Daniil From: "serguei.spitsyn at oracle.com" Date: Wednesday, June 6, 2018 at 11:23 AM To: Daniil Titov , Subject: Re: RFR 8203033: [Testbug] vmTestbase/nsk/jdi/TypeComponent/isSynthetic/issynthetic002/TestDescription.java fails with nestmates Hi Daniil, It looks good. One minor comment. http://cr.openjdk.java.net/~dtitov/8203033/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jdi/TypeComponent/isSynthetic/issynthetic002.java.frames.html 153????????????? ???????log.complain("debuger FAILURE 3> Method's " + name + signature 154???????????????????????????? + " synthetic is true, but expected " 155???????????????????????????? + "is false"); The sub-string at lines 154 and 155 can be combined into one: + " synthetic is true, but expected is false"); But I'd suggest to simplify it to something like this: log.complain("debugger FAILURE 3> found unexpected synthetic method " + name + signature); Also, there are many instances of the typo: debuger => debugger. Could you, please, fix it? Thank you a lot for taking care about this! Thanks, Serguei On 6/5/18 11:38, Daniil Titov wrote: Please review the change that fixes vmTestbase/nsk/jdi/TypeComponent/isSynthetic/issynthetic002/TestDescription.java test. The fix uses the implementation of a parametrized interface in the test class to trigger the compiler to generate a synthetic bridge method. The test works fine in both nestmates and jdk/jdk repositories. Issue: https://bugs.openjdk.java.net/browse/JDK-8203033 Webrev: http://cr.openjdk.java.net/~dtitov/8203033/webrev.02 Thanks, Daniil -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Wed Jun 6 19:18:17 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 6 Jun 2018 12:18:17 -0700 Subject: RFR 8203033: [Testbug] vmTestbase/nsk/jdi/TypeComponent/isSynthetic/issynthetic002/TestDescription.java fails with nestmates In-Reply-To: <5FA679F2-6A90-4A5A-8D79-42CC7C05BD04@oracle.com> References: <9C78D8B6-F7C0-44EE-93B3-D8C22CAF2628@oracle.com> <5FA679F2-6A90-4A5A-8D79-42CC7C05BD04@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Wed Jun 6 19:27:40 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 6 Jun 2018 12:27:40 -0700 Subject: RFR: JDK-8169718: nsk/jdb/locals/locals002: ERROR: Cannot find boolVar with expected value: false In-Reply-To: <5B182D6B.8060206@oracle.com> References: <5B082D2E.7000408@oracle.com> <5d58fa2b-7dda-db64-0280-1e19e791a86d@oracle.com> <5B1814C9.6030807@oracle.com> <1d47721c-3498-92ee-997e-1d9fa216ea6f@oracle.com> <5B181D7F.1040300@oracle.com> <9794304d-d166-554a-399d-0f8b53fe8786@oracle.com> <5B182D6B.8060206@oracle.com> Message-ID: <40d44ef3-97dc-ead0-158e-a5ca93d49b9e@oracle.com> An HTML attachment was scrubbed... URL: From daniil.x.titov at oracle.com Wed Jun 6 20:04:24 2018 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Wed, 06 Jun 2018 13:04:24 -0700 Subject: RFR 8203033: [Testbug] vmTestbase/nsk/jdi/TypeComponent/isSynthetic/issynthetic002/TestDescription.java fails with nestmates In-Reply-To: References: <9C78D8B6-F7C0-44EE-93B3-D8C22CAF2628@oracle.com> <5FA679F2-6A90-4A5A-8D79-42CC7C05BD04@oracle.com> Message-ID: <02827A43-670E-4B72-903B-058510D169A3@oracle.com> Hi Serguei, I updated the webrev to correct typo for debuggee instances as well. ?However, I didn?t rename the shared class nsk.share.jdi.Debugee and nsk.share.jdi.Binder.bindToDebugee method ?since I am not sure it should be done as a part of this fix. I would suggest filing a separate issue for that as that changes will affect multiple tests. Webrev: http://cr.openjdk.java.net/~dtitov/8203033/webrev.04/ Issue: https://bugs.openjdk.java.net/browse/JDK-8203033 Thanks, Daniil From: "serguei.spitsyn at oracle.com" Date: Wednesday, June 6, 2018 at 12:18 PM To: Daniil Titov , Subject: Re: RFR 8203033: [Testbug] vmTestbase/nsk/jdi/TypeComponent/isSynthetic/issynthetic002/TestDescription.java fails with nestmates It looks good. There are also instances of "debugee" but they are because of the supporting class named "Debugee". Thanks, Serguei On 6/6/18 12:09, Daniil Titov wrote: Thank you, Serguei and David, for reviewing this change. Please review a new version of the change that corrects the diagnostic message as Serguei suggested and fixes typos. Webrev: http://cr.openjdk.java.net/~dtitov/8203033/webrev.03 Issue: https://bugs.openjdk.java.net/browse/JDK-8203033 Best regards, Daniil From: "serguei.spitsyn at oracle.com" Date: Wednesday, June 6, 2018 at 11:23 AM To: Daniil Titov , Subject: Re: RFR 8203033: [Testbug] vmTestbase/nsk/jdi/TypeComponent/isSynthetic/issynthetic002/TestDescription.java fails with nestmates Hi Daniil, It looks good. One minor comment. http://cr.openjdk.java.net/~dtitov/8203033/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jdi/TypeComponent/isSynthetic/issynthetic002.java.frames.html 153 log.complain("debuger FAILURE 3> Method's " + name + signature 154 + " synthetic is true, but expected " 155 + "is false"); The sub-string at lines 154 and 155 can be combined into one: + " synthetic is true, but expected is false"); But I'd suggest to simplify it to something like this: log.complain("debugger FAILURE 3> found unexpected synthetic method " + name + signature); Also, there are many instances of the typo: debuger => debugger. Could you, please, fix it? Thank you a lot for taking care about this! Thanks, Serguei On 6/5/18 11:38, Daniil Titov wrote: Please review the change that fixes vmTestbase/nsk/jdi/TypeComponent/isSynthetic/issynthetic002/TestDescription.java test. The fix uses the implementation of a parametrized interface in the test class to trigger the compiler to generate a synthetic bridge method. The test works fine in both nestmates and jdk/jdk repositories. Issue: https://bugs.openjdk.java.net/browse/JDK-8203033 Webrev: http://cr.openjdk.java.net/~dtitov/8203033/webrev.02 Thanks, Daniil -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Wed Jun 6 20:08:27 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 6 Jun 2018 13:08:27 -0700 Subject: RFR 8203033: [Testbug] vmTestbase/nsk/jdi/TypeComponent/isSynthetic/issynthetic002/TestDescription.java fails with nestmates In-Reply-To: <02827A43-670E-4B72-903B-058510D169A3@oracle.com> References: <9C78D8B6-F7C0-44EE-93B3-D8C22CAF2628@oracle.com> <5FA679F2-6A90-4A5A-8D79-42CC7C05BD04@oracle.com> <02827A43-670E-4B72-903B-058510D169A3@oracle.com> Message-ID: <399d51c6-3818-1e40-16a2-15673d7187e9@oracle.com> An HTML attachment was scrubbed... URL: From daniel.daugherty at oracle.com Wed Jun 6 20:59:52 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 6 Jun 2018 16:59:52 -0400 Subject: RFR: JDK-8187289: NotifyFramePop request is not cleared if JVMTI_EVENT_FRAME_POP is disabled In-Reply-To: <8d72b580-a98c-b459-74ef-df0fb729c7b2@oracle.com> References: <8d72b580-a98c-b459-74ef-df0fb729c7b2@oracle.com> Message-ID: <3398646f-c3fe-c28c-1c10-b53039636ea9@oracle.com> On 5/23/18 3:33 PM, Alex Menkov wrote: > Hi all, > > Please take a look at a fix for > https://bugs.openjdk.java.net/browse/JDK-8187289 > > webrev: http://cr.openjdk.java.net/~amenkov/notifyFramePop/webrev/ src/hotspot/share/prims/jvmtiEventController.cpp ??? No comments. src/hotspot/share/prims/jvmtiExport.cpp ??? No comments. test/hotspot/jtreg/serviceability/jvmti/NotifyFramePop/NotifyFramePopTest.java ??? No comments. test/hotspot/jtreg/serviceability/jvmti/NotifyFramePop/libNotifyFramePopTest.c ??? No comments. I compared this patch with Serguei's last patch and I _think_ I understand the differences between the two version... but it has been a while... Thumbs up. Dan > > CSR for the issue: https://bugs.openjdk.java.net/browse/JDK-8191467 > > --alex > From stuart.marks at oracle.com Wed Jun 6 21:02:26 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Wed, 6 Jun 2018 14:02:26 -0700 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> Message-ID: <34074732-06a4-cc54-d935-835edc69a558@oracle.com> On 6/6/18 10:58 AM, serguei.spitsyn at oracle.com wrote: > But the fix below is not clear to me: > > http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/src/hotspot/share/prims/jvmti.xml.udiff.html > > Stop Thread > > - Send the specified asynchronous exception to the specified thread > - (similar to java.lang.Thread.stop). > + Send the specified asynchronous exception to the specified thread. > Normally, this function is used to kill the specified thread with an > - instance of the exception ThreadDeath. > + instance of the exception ThreadDeath, similar to > + java.lang.Thread.stop. > > A reference to the java.lang.Thread.stop has been removed in one place and > added in another. > I thought, we wanted to get rid of these references in the spec. > Do I miss anything? Could you, explain this a little bit? Sure. The current state (through JDK 10) is that there two APIs: ??? 1. Thread.stop(Throwable) ??? 2. Thread.stop() // no-arg They are both deprecated, but only (1) is deprecated for removal, and it's the one being removed by this changeset. Method (2) will remain in the platform for the forseeable future. Method (1) causes the target thread asynchronously to throw the argument, which can be an instance of any subtype of Throwable. Method (2) causes the target thread to throw ThreadDeath (a subtype of Error) asynchronously. The wording in the JVMTI doc isn't terribly explicit. The original wording actually means "similar to Thread.stop(Throwable)" that is method (1). My rewording places the mention of Thread.stop into the context of throwing ThreadDeath, implying method (2). Since that will be the only Thread.stop method remaining, there's no ambiguity. So yes, the wording change looks odd, but there is a subtle shift in the meaning, and I think the meaning is clear after (1) has been removed. But I can remove the "similar to Thread.stop" bit if you prefer. s'marks > > Thanks, > Serguei > > On 6/5/18 17:05, Stuart Marks wrote: >> [adding serviceability-dev] >> >> Hi serviceability folks, >> >> I'm in the process of removing Thread.destroy() and Thread.stop(Throwable) >> from the Java SE API. Alan and David have pointed out that there are some >> cross-references to Thread.stop(Throwable) in the JDWP and JVMTI specs, as >> well as in the JDI ThreadReference API. I've adjusted the relevant files. >> >> See please review this updated webrev: >> >> http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/ >> >> For context, please see the full review thread on core-libs-dev: >> >> http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053536.html >> >> >> >> On 6/4/18 11:11 PM, Alan Bateman wrote: >>> The source file that is used to generate the JDWP protocol code and the spec >>> is in make/data/jdwp/jdwp.spec. The JVM TI spec is at >>> src/hotspot/share/prims/jvmti.xml. It should be okay to skip those for this >>> change-set, assuming it is followed up quickly with another change-set to >>> update those specs. >> >> OK. I took a look at those other files and they seem simple enough to update >> as part of this changeset. If there aren't any objections from anyone, might >> as well get this all done at once. >> >> Thanks, >> >> s'marks > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stuart.marks at oracle.com Wed Jun 6 21:29:58 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Wed, 6 Jun 2018 14:29:58 -0700 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: <9d09f2b9-775d-4c95-b54b-37c426c62d3d@default> References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> <7f96f3ae-246e-8960-93fc-0e93aad5dd04@oracle.com> <9d09f2b9-775d-4c95-b54b-37c426c62d3d@default> Message-ID: <8e780019-dc81-9f6b-5bbc-06d2ffe234d1@oracle.com> Yeah, maybe it's better simply to remove the mentions of the methods that are being removed. I'll do so. I note that there are many other updates that could be done (the examples use applets!) but I think that's a task for another time. s'marks On 6/6/18 9:28 AM, Iris Clark wrote: > Hi, Stuart. > > I think you need to make changes to this file too: > > http://hg.openjdk.java.net/jdk/jdk/file/tip/src/java.base/share/classes/java/lang/doc-files/threadPrimitiveDeprecation.html > > Thanks, > iris > > -----Original Message----- > From: Alan Bateman > Sent: Wednesday, June 6, 2018 3:40 AM > To: Stuart Marks ; serviceability-dev > Cc: core-libs-dev > Subject: Re: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) > > On 06/06/2018 01:05, Stuart Marks wrote: >> [adding serviceability-dev] >> >> Hi serviceability folks, >> >> I'm in the process of removing Thread.destroy() and >> Thread.stop(Throwable) from the Java SE API. Alan and David have >> pointed out that there are some cross-references to >> Thread.stop(Throwable) in the JDWP and JVMTI specs, as well as in the >> JDI ThreadReference API. I've adjusted the relevant files. >> >> See please review this updated webrev: >> >> ??? http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/ > Have you considered removing the "What about Thread.stop(Throwable)" > section completely from the threadPrimitiveDeprecation doc? A new developer reading the javadoc in 2019 shouldn't need to read about a removed method. > > Otherwise I think this looks good, including the updates to the JDWP and JVM TI specs. > > -Alan > From serguei.spitsyn at oracle.com Wed Jun 6 21:37:51 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 6 Jun 2018 14:37:51 -0700 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: <34074732-06a4-cc54-d935-835edc69a558@oracle.com> References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> <34074732-06a4-cc54-d935-835edc69a558@oracle.com> Message-ID: <3c10d04f-180f-a945-cf5e-f6e9089bb58e@oracle.com> An HTML attachment was scrubbed... URL: From stuart.marks at oracle.com Wed Jun 6 21:58:13 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Wed, 6 Jun 2018 14:58:13 -0700 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: <3c10d04f-180f-a945-cf5e-f6e9089bb58e@oracle.com> References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> <34074732-06a4-cc54-d935-835edc69a558@oracle.com> <3c10d04f-180f-a945-cf5e-f6e9089bb58e@oracle.com> Message-ID: Serguei, great! Thanks. All, I've updated the webrev to with changes to threadPrimitiveDeprecation.html to remove the sections that mention the removed methods. http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.2/ s'marks On 6/6/18 2:37 PM, serguei.spitsyn at oracle.com wrote: > Stuart, > > Thank you for explanation! > I'm Okay with the fix it is now. > > Thanks, > Serguei > > > On 6/6/18 14:02, Stuart Marks wrote: >> On 6/6/18 10:58 AM, serguei.spitsyn at oracle.com wrote: >>> But the fix below is not clear to me: >>> >>> http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/src/hotspot/share/prims/jvmti.xml.udiff.html >>> >>> Stop Thread >>> >>> - Send the specified asynchronous exception to the specified thread >>> - (similar to java.lang.Thread.stop). >>> + Send the specified asynchronous exception to the specified thread. >>> Normally, this function is used to kill the specified thread with an >>> - instance of the exception ThreadDeath. >>> + instance of the exception ThreadDeath, similar to >>> + java.lang.Thread.stop. >>> >>> A reference to the java.lang.Thread.stop has been removed in one place and >>> added in another. >>> I thought, we wanted to get rid of these references in the spec. >>> Do I miss anything? Could you, explain this a little bit? >> Sure. The current state (through JDK 10) is that there two APIs: >> >> ??? 1. Thread.stop(Throwable) >> ??? 2. Thread.stop() // no-arg >> >> They are both deprecated, but only (1) is deprecated for removal, and it's >> the one being removed by this changeset. Method (2) will remain in the >> platform for the forseeable future. >> >> Method (1) causes the target thread asynchronously to throw the argument, >> which can be an instance of any subtype of Throwable. Method (2) causes the >> target thread to throw ThreadDeath (a subtype of Error) asynchronously. >> >> The wording in the JVMTI doc isn't terribly explicit. The original wording >> actually means "similar to Thread.stop(Throwable)" that is method (1). My >> rewording places the mention of Thread.stop into the context of throwing >> ThreadDeath, implying method (2). Since that will be the only Thread.stop >> method remaining, there's no ambiguity. So yes, the wording change looks odd, >> but there is a subtle shift in the meaning, and I think the meaning is clear >> after (1) has been removed. >> >> But I can remove the "similar to Thread.stop" bit if you prefer. >> >> s'marks >>> >>> Thanks, >>> Serguei >>> >>> On 6/5/18 17:05, Stuart Marks wrote: >>>> [adding serviceability-dev] >>>> >>>> Hi serviceability folks, >>>> >>>> I'm in the process of removing Thread.destroy() and Thread.stop(Throwable) >>>> from the Java SE API. Alan and David have pointed out that there are some >>>> cross-references to Thread.stop(Throwable) in the JDWP and JVMTI specs, as >>>> well as in the JDI ThreadReference API. I've adjusted the relevant files. >>>> >>>> See please review this updated webrev: >>>> >>>> http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/ >>>> >>>> For context, please see the full review thread on core-libs-dev: >>>> >>>> http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053536.html >>>> >>>> >>>> >>>> On 6/4/18 11:11 PM, Alan Bateman wrote: >>>>> The source file that is used to generate the JDWP protocol code and the >>>>> spec is in make/data/jdwp/jdwp.spec. The JVM TI spec is at >>>>> src/hotspot/share/prims/jvmti.xml. It should be okay to skip those for >>>>> this change-set, assuming it is followed up quickly with another >>>>> change-set to update those specs. >>>> >>>> OK. I took a look at those other files and they seem simple enough to >>>> update as part of this changeset. If there aren't any objections from >>>> anyone, might as well get this all done at once. >>>> >>>> Thanks, >>>> >>>> s'marks >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Wed Jun 6 22:12:53 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 6 Jun 2018 15:12:53 -0700 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> <34074732-06a4-cc54-d935-835edc69a558@oracle.com> <3c10d04f-180f-a945-cf5e-f6e9089bb58e@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From iris.clark at oracle.com Wed Jun 6 22:18:02 2018 From: iris.clark at oracle.com (Iris Clark) Date: Wed, 6 Jun 2018 15:18:02 -0700 (PDT) Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> <34074732-06a4-cc54-d935-835edc69a558@oracle.com> <3c10d04f-180f-a945-cf5e-f6e9089bb58e@oracle.com> Message-ID: <4a93f51f-01b9-489f-9e53-cb5bba47506c@default> Hi, Stuart. ? http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.2/ ? The simple update to remove references to the removed methods looks good. ? Thanks, iris ? From: Stuart Marks Sent: Wednesday, June 6, 2018 2:58 PM To: Serguei Spitsyn ; David Holmes ; Alan Bateman ; Iris Clark Cc: serviceability-dev ; core-libs-dev Subject: Re: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) ? Serguei, great! Thanks. All, I've updated the webrev to with changes to threadPrimitiveDeprecation.html to remove the sections that mention the removed methods. http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.2/ s'marks ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Wed Jun 6 22:24:15 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 6 Jun 2018 15:24:15 -0700 Subject: RFR(S): 8203329: JDWP/JDI VM information string is incorrect Message-ID: <3aa60339-daa9-12b9-435c-2d94909aa279@oracle.com> Hello, Please review the following: https://bugs.openjdk.java.net/browse/JDK-8203329 http://cr.openjdk.java.net/~cjplummer/8203329/webrev.00/ The native version of the java.vm.info property was getting out of sync with the java version. Details can be found here: https://bugs.openjdk.java.net/browse/JDK-8203329?focusedCommentId=14185679&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14185679 thanks, Chris From serguei.spitsyn at oracle.com Wed Jun 6 22:44:12 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 6 Jun 2018 15:44:12 -0700 Subject: RFR: JDK-8187289: NotifyFramePop request is not cleared if JVMTI_EVENT_FRAME_POP is disabled In-Reply-To: <3398646f-c3fe-c28c-1c10-b53039636ea9@oracle.com> References: <8d72b580-a98c-b459-74ef-df0fb729c7b2@oracle.com> <3398646f-c3fe-c28c-1c10-b53039636ea9@oracle.com> Message-ID: <0676f3d8-14bc-0313-e8bf-d2632bb8b9df@oracle.com> Hi Dan, Thank you for the review! The fix has been already pushed. I hope, it is Okay with you. Are you Okay with the release note sub-task? : ? https://bugs.openjdk.java.net/browse/JDK-8192891 It tells: ? A NotifyFramePop request was only cleared if the JVMTI_EVENT_FRAME_POP is enabled. ? Now it is always cleared when the corresponding frame is popped, ? regardless of whether the JVMTI_EVENT_FRAME_POP is enabled or not. Thanks, Serguei On 6/6/18 13:59, Daniel D. Daugherty wrote: > On 5/23/18 3:33 PM, Alex Menkov wrote: >> Hi all, >> >> Please take a look at a fix for >> https://bugs.openjdk.java.net/browse/JDK-8187289 >> >> webrev: http://cr.openjdk.java.net/~amenkov/notifyFramePop/webrev/ > > src/hotspot/share/prims/jvmtiEventController.cpp > ??? No comments. > > src/hotspot/share/prims/jvmtiExport.cpp > ??? No comments. > > test/hotspot/jtreg/serviceability/jvmti/NotifyFramePop/NotifyFramePopTest.java > > ??? No comments. > > test/hotspot/jtreg/serviceability/jvmti/NotifyFramePop/libNotifyFramePopTest.c > > ??? No comments. > > I compared this patch with Serguei's last patch and I _think_ I > understand > the differences between the two version... but it has been a while... > > Thumbs up. > > Dan > > >> >> CSR for the issue: https://bugs.openjdk.java.net/browse/JDK-8191467 >> >> --alex >> > From daniel.daugherty at oracle.com Wed Jun 6 23:52:43 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 6 Jun 2018 19:52:43 -0400 Subject: RFR: JDK-8187289: NotifyFramePop request is not cleared if JVMTI_EVENT_FRAME_POP is disabled In-Reply-To: <0676f3d8-14bc-0313-e8bf-d2632bb8b9df@oracle.com> References: <8d72b580-a98c-b459-74ef-df0fb729c7b2@oracle.com> <3398646f-c3fe-c28c-1c10-b53039636ea9@oracle.com> <0676f3d8-14bc-0313-e8bf-d2632bb8b9df@oracle.com> Message-ID: <32fef5fc-d9a0-ee1f-fffa-b055b4f47df9@oracle.com> On 6/6/18 6:44 PM, serguei.spitsyn at oracle.com wrote: > Hi Dan, > > Thank you for the review! As always, you're welcome. > The fix has been already pushed. Yup. I noticed that after I finished the review. > I hope, it is Okay with you. Of course. I'm playing catch up after being off-the-air for two weeks so I don't/didn't expect you guys to wait for me. > > Are you Okay with the release note sub-task? : > ? https://bugs.openjdk.java.net/browse/JDK-8192891 > > It tells: > ? A NotifyFramePop request was only cleared if the > JVMTI_EVENT_FRAME_POP is enabled. > ? Now it is always cleared when the corresponding frame is popped, > ? regardless of whether the JVMTI_EVENT_FRAME_POP is enabled or not. Looks like a good note to me. Dan > > Thanks, > Serguei > > > On 6/6/18 13:59, Daniel D. Daugherty wrote: >> On 5/23/18 3:33 PM, Alex Menkov wrote: >>> Hi all, >>> >>> Please take a look at a fix for >>> https://bugs.openjdk.java.net/browse/JDK-8187289 >>> >>> webrev: http://cr.openjdk.java.net/~amenkov/notifyFramePop/webrev/ >> >> src/hotspot/share/prims/jvmtiEventController.cpp >> ??? No comments. >> >> src/hotspot/share/prims/jvmtiExport.cpp >> ??? No comments. >> >> test/hotspot/jtreg/serviceability/jvmti/NotifyFramePop/NotifyFramePopTest.java >> >> ??? No comments. >> >> test/hotspot/jtreg/serviceability/jvmti/NotifyFramePop/libNotifyFramePopTest.c >> >> ??? No comments. >> >> I compared this patch with Serguei's last patch and I _think_ I >> understand >> the differences between the two version... but it has been a while... >> >> Thumbs up. >> >> Dan >> >> >>> >>> CSR for the issue: https://bugs.openjdk.java.net/browse/JDK-8191467 >>> >>> --alex >>> >> > From serguei.spitsyn at oracle.com Wed Jun 6 23:55:36 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 6 Jun 2018 16:55:36 -0700 Subject: RFR: JDK-8187289: NotifyFramePop request is not cleared if JVMTI_EVENT_FRAME_POP is disabled In-Reply-To: <32fef5fc-d9a0-ee1f-fffa-b055b4f47df9@oracle.com> References: <8d72b580-a98c-b459-74ef-df0fb729c7b2@oracle.com> <3398646f-c3fe-c28c-1c10-b53039636ea9@oracle.com> <0676f3d8-14bc-0313-e8bf-d2632bb8b9df@oracle.com> <32fef5fc-d9a0-ee1f-fffa-b055b4f47df9@oracle.com> Message-ID: Thank you, Dan! Serguei On 6/6/18 16:52, Daniel D. Daugherty wrote: > On 6/6/18 6:44 PM, serguei.spitsyn at oracle.com wrote: >> Hi Dan, >> >> Thank you for the review! > > As always, you're welcome. > > >> The fix has been already pushed. > > Yup. I noticed that after I finished the review. > > >> I hope, it is Okay with you. > > Of course. I'm playing catch up after being off-the-air for > two weeks so I don't/didn't expect you guys to wait for me. > > >> >> Are you Okay with the release note sub-task? : >> ? https://bugs.openjdk.java.net/browse/JDK-8192891 >> >> It tells: >> ? A NotifyFramePop request was only cleared if the >> JVMTI_EVENT_FRAME_POP is enabled. >> ? Now it is always cleared when the corresponding frame is popped, >> ? regardless of whether the JVMTI_EVENT_FRAME_POP is enabled or not. > > Looks like a good note to me. > > Dan > > >> >> Thanks, >> Serguei >> >> >> On 6/6/18 13:59, Daniel D. Daugherty wrote: >>> On 5/23/18 3:33 PM, Alex Menkov wrote: >>>> Hi all, >>>> >>>> Please take a look at a fix for >>>> https://bugs.openjdk.java.net/browse/JDK-8187289 >>>> >>>> webrev: http://cr.openjdk.java.net/~amenkov/notifyFramePop/webrev/ >>> >>> src/hotspot/share/prims/jvmtiEventController.cpp >>> ??? No comments. >>> >>> src/hotspot/share/prims/jvmtiExport.cpp >>> ??? No comments. >>> >>> test/hotspot/jtreg/serviceability/jvmti/NotifyFramePop/NotifyFramePopTest.java >>> >>> ??? No comments. >>> >>> test/hotspot/jtreg/serviceability/jvmti/NotifyFramePop/libNotifyFramePopTest.c >>> >>> ??? No comments. >>> >>> I compared this patch with Serguei's last patch and I _think_ I >>> understand >>> the differences between the two version... but it has been a while... >>> >>> Thumbs up. >>> >>> Dan >>> >>> >>>> >>>> CSR for the issue: https://bugs.openjdk.java.net/browse/JDK-8191467 >>>> >>>> --alex >>>> >>> >> > From serguei.spitsyn at oracle.com Thu Jun 7 00:12:38 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 6 Jun 2018 17:12:38 -0700 Subject: RFR(S): 8203329: JDWP/JDI VM information string is incorrect In-Reply-To: <3aa60339-daa9-12b9-435c-2d94909aa279@oracle.com> References: <3aa60339-daa9-12b9-435c-2d94909aa279@oracle.com> Message-ID: <624dead6-ac0b-8f67-5c7f-7e8a7e970cf8@oracle.com> Hi Chris, It looks good. Thank you for taking care about this! Thanks, Serguei On 6/6/18 15:24, Chris Plummer wrote: > Hello, > > Please review the following: > > https://bugs.openjdk.java.net/browse/JDK-8203329 > http://cr.openjdk.java.net/~cjplummer/8203329/webrev.00/ > > The native version of the java.vm.info property was getting out of > sync with the java version. Details can be found here: > > https://bugs.openjdk.java.net/browse/JDK-8203329?focusedCommentId=14185679&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14185679 > > > thanks, > > Chris From david.holmes at oracle.com Thu Jun 7 00:37:40 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 7 Jun 2018 10:37:40 +1000 Subject: RFR 8203033: [Testbug] vmTestbase/nsk/jdi/TypeComponent/isSynthetic/issynthetic002/TestDescription.java fails with nestmates In-Reply-To: <02827A43-670E-4B72-903B-058510D169A3@oracle.com> References: <9C78D8B6-F7C0-44EE-93B3-D8C22CAF2628@oracle.com> <5FA679F2-6A90-4A5A-8D79-42CC7C05BD04@oracle.com> <02827A43-670E-4B72-903B-058510D169A3@oracle.com> Message-ID: <2b5fe167-dbaf-bb70-4488-c52881e29c48@oracle.com> Looks okay. I didn't mention the spelling issues because they were everywhere and would just have obscured the fix initially. Once you start cleaning up these old tests you might never stop. ;-) Thanks, David On 7/06/2018 6:04 AM, Daniil Titov wrote: > Hi Serguei, > > I updated the webrev to correct typo for debuggee instances as well. > ?However, I didn?t rename the shared class nsk.share.jdi.Debugee and > nsk.share.jdi.Binder.bindToDebugee method ?since I am not sure it should > be done as a part of this fix. I would suggest filing a separate issue > for that as that changes will affect multiple tests. > > Webrev: http://cr.openjdk.java.net/~dtitov/8203033/webrev.04/ > > Issue: https://bugs.openjdk.java.net/browse/JDK-8203033 > > Thanks, > > Daniil > > *From: *"serguei.spitsyn at oracle.com" > *Date: *Wednesday, June 6, 2018 at 12:18 PM > *To: *Daniil Titov , > > *Subject: *Re: RFR 8203033: [Testbug] > vmTestbase/nsk/jdi/TypeComponent/isSynthetic/issynthetic002/TestDescription.java > fails with nestmates > > It looks good. > There are also instances of "debugee" but they are > because of the supporting class named "Debugee". > > Thanks, > Serguei > > > On 6/6/18 12:09, Daniil Titov wrote: > > Thank you, Serguei and David, for reviewing this change. > > Please review a new version ?of the change that corrects the > diagnostic message as Serguei suggested ?and fixes typos. > > Webrev: http://cr.openjdk.java.net/~dtitov/8203033/webrev.03 > > > Issue: https://bugs.openjdk.java.net/browse/JDK-8203033 > > Best regards, > > Daniil > > *From: *"serguei.spitsyn at oracle.com" > > > *Date: *Wednesday, June 6, 2018 at 11:23 AM > *To: *Daniil Titov > , > > > *Subject: *Re: RFR 8203033: [Testbug] > vmTestbase/nsk/jdi/TypeComponent/isSynthetic/issynthetic002/TestDescription.java > fails with nestmates > > Hi Daniil, > > It looks good. > One minor comment. > > http://cr.openjdk.java.net/~dtitov/8203033/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jdi/TypeComponent/isSynthetic/issynthetic002.java.frames.html > > > 153????????????? ???????log.complain("debuger FAILURE 3> Method's " > + name + signature > > 154???????????????????????????? + " synthetic is true, but expected " > > 155???????????????????????????? + "is false"); > > The sub-string at lines 154 and 155 can be combined into one: > + " synthetic is true, but expected is false"); > > But I'd suggest to simplify it to something like this: > ?? log.complain("debugger FAILURE 3> found unexpected synthetic > method " + name + signature); > > Also, there are many instances of the typo: debuger => debugger. > Could you, please, fix it? > > Thank you a lot for taking care about this! > > Thanks, > Serguei > > > On 6/5/18 11:38, Daniil Titov wrote: > > Please review the change that fixes vmTestbase/nsk/jdi/TypeComponent/isSynthetic/issynthetic002/TestDescription.java test. > > > > The fix uses the implementation of a parametrized interface in the test class to trigger the compiler to generate a synthetic bridge method. The test works fine in both nestmates and jdk/jdk repositories. > > > > Issue:https://bugs.openjdk.java.net/browse/JDK-8203033 > > Webrev:http://cr.openjdk.java.net/~dtitov/8203033/webrev.02 > > > > > Thanks, > > Daniil > > > > > > > From serguei.spitsyn at oracle.com Thu Jun 7 00:45:59 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 6 Jun 2018 17:45:59 -0700 Subject: RFR 8203033: [Testbug] vmTestbase/nsk/jdi/TypeComponent/isSynthetic/issynthetic002/TestDescription.java fails with nestmates In-Reply-To: <2b5fe167-dbaf-bb70-4488-c52881e29c48@oracle.com> References: <9C78D8B6-F7C0-44EE-93B3-D8C22CAF2628@oracle.com> <5FA679F2-6A90-4A5A-8D79-42CC7C05BD04@oracle.com> <02827A43-670E-4B72-903B-058510D169A3@oracle.com> <2b5fe167-dbaf-bb70-4488-c52881e29c48@oracle.com> Message-ID: <1b3dd9f7-3f5d-dcfd-5864-a85c4515048b@oracle.com> On 6/6/18 17:37, David Holmes wrote: > Looks okay. > > I didn't mention the spelling issues because they were everywhere and > would just have obscured the fix initially. Once you start cleaning up > these old tests you might never stop. ;-) It is why I'm against filing a bug on the Debugee class. :) I'm afraid it is too much of cleanup. Thanks, Serguei > > Thanks, > David > > On 7/06/2018 6:04 AM, Daniil Titov wrote: >> Hi Serguei, >> >> I updated the webrev to correct typo for debuggee instances as well. >> ??However, I didn?t rename the shared class nsk.share.jdi.Debugee and >> nsk.share.jdi.Binder.bindToDebugee method ?since I am not sure it >> should be done as a part of this fix. I would suggest filing a >> separate issue for that as that changes will affect multiple tests. >> >> Webrev: http://cr.openjdk.java.net/~dtitov/8203033/webrev.04/ >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-8203033 >> >> Thanks, >> >> Daniil >> >> *From: *"serguei.spitsyn at oracle.com" >> *Date: *Wednesday, June 6, 2018 at 12:18 PM >> *To: *Daniil Titov , >> >> *Subject: *Re: RFR 8203033: [Testbug] >> vmTestbase/nsk/jdi/TypeComponent/isSynthetic/issynthetic002/TestDescription.java >> fails with nestmates >> >> It looks good. >> There are also instances of "debugee" but they are >> because of the supporting class named "Debugee". >> >> Thanks, >> Serguei >> >> >> On 6/6/18 12:09, Daniil Titov wrote: >> >> ??? Thank you, Serguei and David, for reviewing this change. >> >> ??? Please review a new version ?of the change that corrects the >> ??? diagnostic message as Serguei suggested ?and fixes typos. >> >> ??? Webrev: http://cr.openjdk.java.net/~dtitov/8203033/webrev.03 >> >> >> ??? Issue: https://bugs.openjdk.java.net/browse/JDK-8203033 >> >> ??? Best regards, >> >> ??? Daniil >> >> ??? *From: *"serguei.spitsyn at oracle.com" >> ??? >> ??? >> ??? *Date: *Wednesday, June 6, 2018 at 11:23 AM >> ??? *To: *Daniil Titov >> ??? , >> ??? >> ??? >> ??? *Subject: *Re: RFR 8203033: [Testbug] >> vmTestbase/nsk/jdi/TypeComponent/isSynthetic/issynthetic002/TestDescription.java >> ??? fails with nestmates >> >> ??? Hi Daniil, >> >> ??? It looks good. >> ??? One minor comment. >> >> http://cr.openjdk.java.net/~dtitov/8203033/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jdi/TypeComponent/isSynthetic/issynthetic002.java.frames.html >> >> >> ??? 153????????????? ???????log.complain("debuger FAILURE 3> Method's " >> ??? + name + signature >> >> ??? 154???????????????????????????? + " synthetic is true, but >> expected " >> >> ??? 155???????????????????????????? + "is false"); >> >> ??? The sub-string at lines 154 and 155 can be combined into one: >> ??? + " synthetic is true, but expected is false"); >> >> ??? But I'd suggest to simplify it to something like this: >> ???? ?? log.complain("debugger FAILURE 3> found unexpected synthetic >> ??? method " + name + signature); >> >> ??? Also, there are many instances of the typo: debuger => debugger. >> ??? Could you, please, fix it? >> >> ??? Thank you a lot for taking care about this! >> >> ??? Thanks, >> ??? Serguei >> >> >> ??? On 6/5/18 11:38, Daniil Titov wrote: >> >> ??????? Please review the change that fixes >> vmTestbase/nsk/jdi/TypeComponent/isSynthetic/issynthetic002/TestDescription.java >> test. >> >> >> ??????? The fix uses the implementation of a parametrized interface >> in the test class to trigger the compiler to generate a synthetic >> bridge method. The test works fine in both nestmates and jdk/jdk >> repositories. >> >> >> ??????? Issue:https://bugs.openjdk.java.net/browse/JDK-8203033 >> Webrev:http://cr.openjdk.java.net/~dtitov/8203033/webrev.02 >> >> >> >> ??????? Thanks, >> >> ??????? Daniil >> >> >> >> >> From david.holmes at oracle.com Thu Jun 7 02:01:37 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 7 Jun 2018 12:01:37 +1000 Subject: RFR(S): 8203329: JDWP/JDI VM information string is incorrect In-Reply-To: <3aa60339-daa9-12b9-435c-2d94909aa279@oracle.com> References: <3aa60339-daa9-12b9-435c-2d94909aa279@oracle.com> Message-ID: <3aed8e46-9780-dde6-c0ea-67f2c84a96ff@oracle.com> Hi Chris, Great sleuthing on this one and thanks for explaining things for me in the bug report! The fix you have addresses the problem observed with the "vm info" string. One small request, can you update this comment in thread.cpp please: // We need this for ClassDataSharing - the initial vm.info property is set // with the default value of CDS "sharing" which may be reset through // command line options. reset_vm_info_property(CHECK_JNI_ERR); As we actually need that for any piece of information that might be used in the vm info string initially, that can subsequently change due to option processing e.g. UseSharedSpaces and UseAOT. Suggestion: // We need this to update the vm.info property in case any flags used // to initially define it have been changed with command-line options. reset_vm_info_property(CHECK_JNI_ERR); Thanks! --- That said I think we can simplify things somewhat now you discovered the bug in reset_vm_info_property. If we simply update the native vm_info property as soon as we've completed argument parsing, then we will have the right value in place for copying to the Java level during java.lang.System initialization. We don't need any of the existing code in reset_vm_info because we copy the correct vm_info in the first place. In short you should be able to fix this with simply: jint parse_result = Arguments::parse(args); if (parse_result != JNI_OK) return parse_result; os::init_before_ergo(); jint ergo_result = Arguments::apply_ergo(); if (ergo_result != JNI_OK) return ergo_result; + const char *vm_info = VM_Version::vm_info_string(); + Arguments::PropertyList_update_value(Arguments::system_properties(), "java.vm.info", vm_info); ... - // We need this for ClassDataSharing - the initial vm.info property is set - // with the default value of CDS "sharing" which may be reset through - // command line options. - reset_vm_info_property(CHECK_JNI_ERR); and delete reset_vm_info_property. Just a thought - feel free to defer - it's really a runtime issue anyway. :) Thanks, David On 7/06/2018 8:24 AM, Chris Plummer wrote: > Hello, > > Please review the following: > > https://bugs.openjdk.java.net/browse/JDK-8203329 > http://cr.openjdk.java.net/~cjplummer/8203329/webrev.00/ > > The native version of the java.vm.info property was getting out of sync > with the java version. Details can be found here: > > https://bugs.openjdk.java.net/browse/JDK-8203329?focusedCommentId=14185679&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14185679 > > > thanks, > > Chris From chris.plummer at oracle.com Thu Jun 7 02:10:26 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 6 Jun 2018 19:10:26 -0700 Subject: RFR(S): 8203329: JDWP/JDI VM information string is incorrect In-Reply-To: <3aed8e46-9780-dde6-c0ea-67f2c84a96ff@oracle.com> References: <3aa60339-daa9-12b9-435c-2d94909aa279@oracle.com> <3aed8e46-9780-dde6-c0ea-67f2c84a96ff@oracle.com> Message-ID: Ok, I'll update the comment. I think I'll still mention and CDS and maybe add AOT as examples. thanks, Chris On 6/6/18 7:01 PM, David Holmes wrote: > Hi Chris, > > Great sleuthing on this one and thanks for explaining things for me in > the bug report! > > The fix you have addresses the problem observed with the "vm info" > string. One small request, can you update this comment in thread.cpp > please: > > ?// We need this for ClassDataSharing - the initial vm.info property > is set > ? // with the default value of CDS "sharing" which may be reset through > ? // command line options. > ? reset_vm_info_property(CHECK_JNI_ERR); > > As we actually need that for any piece of information that might be > used in the vm info string initially, that can subsequently change due > to option processing e.g. UseSharedSpaces and UseAOT. Suggestion: > > // We need this to update the vm.info property in case any flags used > // to initially define it have been changed with command-line options. > reset_vm_info_property(CHECK_JNI_ERR); > > Thanks! > > --- > > That said I think we can simplify things somewhat now you discovered > the bug in reset_vm_info_property. If we simply update the native > vm_info property as soon as we've completed argument parsing, then we > will have the right value in place for copying to the Java level > during java.lang.System initialization. We don't need any of the > existing code in reset_vm_info because we copy the correct vm_info in > the first place. In short you should be able to fix this with simply: > > ? jint parse_result = Arguments::parse(args); > ? if (parse_result != JNI_OK) return parse_result; > > ? os::init_before_ergo(); > > ? jint ergo_result = Arguments::apply_ergo(); > ? if (ergo_result != JNI_OK) return ergo_result; > > + const char *vm_info = VM_Version::vm_info_string(); > + Arguments::PropertyList_update_value(Arguments::system_properties(), > "java.vm.info", vm_info); > ... > -? // We need this for ClassDataSharing - the initial vm.info property > is set > -? // with the default value of CDS "sharing" which may be reset through > -? // command line options. > -? reset_vm_info_property(CHECK_JNI_ERR); > > and delete reset_vm_info_property. > > Just a thought - feel free to defer - it's really a runtime issue > anyway. :) > > Thanks, > David > > On 7/06/2018 8:24 AM, Chris Plummer wrote: >> Hello, >> >> Please review the following: >> >> https://bugs.openjdk.java.net/browse/JDK-8203329 >> http://cr.openjdk.java.net/~cjplummer/8203329/webrev.00/ >> >> The native version of the java.vm.info property was getting out of >> sync with the java version. Details can be found here: >> >> https://bugs.openjdk.java.net/browse/JDK-8203329?focusedCommentId=14185679&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14185679 >> >> >> thanks, >> >> Chris From david.holmes at oracle.com Thu Jun 7 02:13:32 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 7 Jun 2018 12:13:32 +1000 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> <34074732-06a4-cc54-d935-835edc69a558@oracle.com> <3c10d04f-180f-a945-cf5e-f6e9089bb58e@oracle.com> Message-ID: On 7/06/2018 7:58 AM, Stuart Marks wrote: > Serguei, great! Thanks. > > All, I've updated the webrev to with changes to > threadPrimitiveDeprecation.html to remove the sections that mention the > removed methods. That reads much better! Thanks, David > http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.2/ > > s'marks > > > On 6/6/18 2:37 PM, serguei.spitsyn at oracle.com wrote: >> Stuart, >> >> Thank you for explanation! >> I'm Okay with the fix it is now. >> >> Thanks, >> Serguei >> >> >> On 6/6/18 14:02, Stuart Marks wrote: >>> On 6/6/18 10:58 AM, serguei.spitsyn at oracle.com wrote: >>>> But the fix below is not clear to me: >>>> >>>> http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/src/hotspot/share/prims/jvmti.xml.udiff.html >>>> >>>> Stop Thread >>>> >>>> - Send the specified asynchronous exception to the specified thread >>>> - (similar to java.lang.Thread.stop). >>>> + Send the specified asynchronous exception to the specified thread. >>>> Normally, this function is used to kill the specified thread with an >>>> - instance of the exception ThreadDeath. >>>> + instance of the exception ThreadDeath, similar to >>>> + java.lang.Thread.stop. >>>> >>>> A reference to the java.lang.Thread.stop has been removed in one >>>> place and added in another. >>>> I thought, we wanted to get rid of these references in the spec. >>>> Do I miss anything? Could you, explain this a little bit? >>> Sure. The current state (through JDK 10) is that there two APIs: >>> >>> ??? 1. Thread.stop(Throwable) >>> ??? 2. Thread.stop() // no-arg >>> >>> They are both deprecated, but only (1) is deprecated for removal, and >>> it's the one being removed by this changeset. Method (2) will remain >>> in the platform for the forseeable future. >>> >>> Method (1) causes the target thread asynchronously to throw the >>> argument, which can be an instance of any subtype of Throwable. >>> Method (2) causes the target thread to throw ThreadDeath (a subtype >>> of Error) asynchronously. >>> >>> The wording in the JVMTI doc isn't terribly explicit. The original >>> wording actually means "similar to Thread.stop(Throwable)" that is >>> method (1). My rewording places the mention of Thread.stop into the >>> context of throwing ThreadDeath, implying method (2). Since that will >>> be the only Thread.stop method remaining, there's no ambiguity. So >>> yes, the wording change looks odd, but there is a subtle shift in the >>> meaning, and I think the meaning is clear after (1) has been removed. >>> >>> But I can remove the "similar to Thread.stop" bit if you prefer. >>> >>> s'marks >>>> >>>> Thanks, >>>> Serguei >>>> >>>> On 6/5/18 17:05, Stuart Marks wrote: >>>>> [adding serviceability-dev] >>>>> >>>>> Hi serviceability folks, >>>>> >>>>> I'm in the process of removing Thread.destroy() and >>>>> Thread.stop(Throwable) from the Java SE API. Alan and David have >>>>> pointed out that there are some cross-references to >>>>> Thread.stop(Throwable) in the JDWP and JVMTI specs, as well as in >>>>> the JDI ThreadReference API. I've adjusted the relevant files. >>>>> >>>>> See please review this updated webrev: >>>>> >>>>> http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/ >>>>> >>>>> For context, please see the full review thread on core-libs-dev: >>>>> >>>>> http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053536.html >>>>> >>>>> >>>>> >>>>> On 6/4/18 11:11 PM, Alan Bateman wrote: >>>>>> The source file that is used to generate the JDWP protocol code >>>>>> and the spec is in make/data/jdwp/jdwp.spec. The JVM TI spec is at >>>>>> src/hotspot/share/prims/jvmti.xml. It should be okay to skip those >>>>>> for this change-set, assuming it is followed up quickly with >>>>>> another change-set to update those specs. >>>>> >>>>> OK. I took a look at those other files and they seem simple enough >>>>> to update as part of this changeset. If there aren't any objections >>>>> from anyone, might as well get this all done at once. >>>>> >>>>> Thanks, >>>>> >>>>> s'marks >>>> >>> >> > From thomas.stuefe at gmail.com Thu Jun 7 04:36:39 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 7 Jun 2018 06:36:39 +0200 Subject: RFR(s): 8203682: Add jcmd "VM.classloaders" command to print out class loader hierarchy, details In-Reply-To: References: Message-ID: Ping... may I please have a second review? For your convenience, the latest webrev with Coleen's requests worked in is: http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/webrev.01/webrev/ JBS issue: https://bugs.openjdk.java.net/browse/JDK-8203682 Thank you very much. Best Regards, Thomas On Wed, May 23, 2018 at 2:46 PM, Thomas St?fe wrote: > Dear all, > > (not sure if this would be a serviceability or runtime rfe, so sorry > for crossposting) > > may I please have feedback/reviews for this small enhancement. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8203682 > Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/webrev.00/webrev/ > > This adds a new command to jcmd, "VM.classloaders". It complements the > existing command "VM.classloader_stats". > > This command, in its simplest form, prints the class loader tree. In > addition to that, it optionally prints out loaded classes (both > non-anonymous and anonymous) and various classloader specific > information. > > Examples: > > http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example.txt > http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example-with-classes.txt > http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example-with-reflection-and-noinflation.txt > > > Thanks and Best Regards, > > Thomas From david.holmes at oracle.com Thu Jun 7 05:14:06 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 7 Jun 2018 15:14:06 +1000 Subject: RFR(s): 8203682: Add jcmd "VM.classloaders" command to print out class loader hierarchy, details In-Reply-To: References: Message-ID: Hi Thomas, On 7/06/2018 2:36 PM, Thomas St?fe wrote: > Ping... > > may I please have a second review? Seems fine in terms of overall code structure etc. I can't comment on the output produced as such. :) Some minor code-style nits (just to prove I read it): - inconsistent placement of opening { in method definitions: should always be after the end of the parameter list ie ....) { - inconsistent variable naming wrt. use of underscores: maxdepth -> max_depth?, twiglen -> twig_len, branchtracker -> branch_tracker? No need to see any update. Thanks, David > > For your convenience, the latest webrev with Coleen's requests worked in is: > > http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/webrev.01/webrev/ > > JBS issue: https://bugs.openjdk.java.net/browse/JDK-8203682 > > Thank you very much. > > Best Regards, Thomas > > > > > On Wed, May 23, 2018 at 2:46 PM, Thomas St?fe wrote: >> Dear all, >> >> (not sure if this would be a serviceability or runtime rfe, so sorry >> for crossposting) >> >> may I please have feedback/reviews for this small enhancement. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8203682 >> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/webrev.00/webrev/ >> >> This adds a new command to jcmd, "VM.classloaders". It complements the >> existing command "VM.classloader_stats". >> >> This command, in its simplest form, prints the class loader tree. In >> addition to that, it optionally prints out loaded classes (both >> non-anonymous and anonymous) and various classloader specific >> information. >> >> Examples: >> >> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example.txt >> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example-with-classes.txt >> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example-with-reflection-and-noinflation.txt >> >> >> Thanks and Best Regards, >> >> Thomas From thomas.stuefe at gmail.com Thu Jun 7 05:16:37 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 7 Jun 2018 07:16:37 +0200 Subject: RFR(s): 8203682: Add jcmd "VM.classloaders" command to print out class loader hierarchy, details In-Reply-To: References: Message-ID: On Thu, Jun 7, 2018 at 7:14 AM, David Holmes wrote: > Hi Thomas, > > On 7/06/2018 2:36 PM, Thomas St?fe wrote: >> >> Ping... >> >> may I please have a second review? > > > Seems fine in terms of overall code structure etc. I can't comment on the > output produced as such. :) > > Some minor code-style nits (just to prove I read it): > - inconsistent placement of opening { in method definitions: should always > be after the end of the parameter list ie ....) { > - inconsistent variable naming wrt. use of underscores: maxdepth -> > max_depth?, twiglen -> twig_len, branchtracker -> branch_tracker? > > No need to see any update. Great, thank you David! ..Thomas > > Thanks, > David > > >> >> For your convenience, the latest webrev with Coleen's requests worked in >> is: >> >> >> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/webrev.01/webrev/ >> >> JBS issue: https://bugs.openjdk.java.net/browse/JDK-8203682 >> >> Thank you very much. >> >> Best Regards, Thomas >> >> >> >> >> On Wed, May 23, 2018 at 2:46 PM, Thomas St?fe >> wrote: >>> >>> Dear all, >>> >>> (not sure if this would be a serviceability or runtime rfe, so sorry >>> for crossposting) >>> >>> may I please have feedback/reviews for this small enhancement. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203682 >>> Webrev: >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/webrev.00/webrev/ >>> >>> This adds a new command to jcmd, "VM.classloaders". It complements the >>> existing command "VM.classloader_stats". >>> >>> This command, in its simplest form, prints the class loader tree. In >>> addition to that, it optionally prints out loaded classes (both >>> non-anonymous and anonymous) and various classloader specific >>> information. >>> >>> Examples: >>> >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example.txt >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example-with-classes.txt >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example-with-reflection-and-noinflation.txt >>> >>> >>> Thanks and Best Regards, >>> >>> Thomas From jini.george at oracle.com Thu Jun 7 05:19:28 2018 From: jini.george at oracle.com (Jini George) Date: Thu, 7 Jun 2018 10:49:28 +0530 Subject: RFR (round 4), JEP-318: Epsilon GC In-Reply-To: References: <094e1093-5a13-4853-aa34-d4e987a069b0@redhat.com> <0b5c827a-2763-98aa-06af-24df9028aed7@oracle.com> Message-ID: <5375c956-089b-2bba-a39c-3f051b895ba1@oracle.com> Thanks for making the changes, Aleksey. The changes look good. Thanks, Jini. On 6/5/2018 9:16 PM, Aleksey Shipilev wrote: > Hi Jini, > > Thanks for taking a look, see comments inline. > > On 06/01/2018 10:13 AM, Jini George wrote: >> ==> share/classes/sun/jvm/hotspot/oops/ObjectHeap.java >> >> 444??????? liveRegions.add(eh.space()); >> >> We would need to add an object of type 'Address' to the liveRegions list, instead of type >> VirtualSpace. Not doing so results in exceptions of the following form from the compare() method for >> various clhsdb commands like 'jhisto': >> >> Exception in thread "main" java.lang.ClassCastException: >> jdk.hotspot.agent/sun.jvm.hotspot.memory.VirtualSpace cannot be cast to >> jdk.hotspot.agent/sun.jvm.hotspot.debugger.Address > > Oh, I see! Fixed: > http://hg.openjdk.java.net/jdk/sandbox/rev/d999bdb8173c > > >> ==>? share/classes/sun/jvm/hotspot/oops/ObjectHeap.java >> >> 445???? } else { >> 446??????? if (Assert.ASSERTS_ENABLED) { >> 447?????????? Assert.that(false, "Expecting GenCollectedHeap, G1CollectedHeap, " + >> 448?????????????????????? "or ParallelScavengeHeap, but got " + >> 449?????????????????????? heap.getClass().getName()); >> 450??????? } >> 451???? } >> >> * Please add EpsilonGC also to the assertion statement. > > I prefer to change it to non-GC-specific message as Per suggested: > http://hg.openjdk.java.net/jdk/sandbox/rev/5cd17d3d3f83 > > >> ==> share/classes/sun/jvm/hotspot/tools/HeapSummary.java >> >> The run() method would need to handle Epsilon GC here to avoid the Unknown CollectedHeap type error >> with jhsdb jmap --heap. > > Right, implemented in both places, run() and detectGCAlgo(): > http://hg.openjdk.java.net/jdk/sandbox/rev/b20a56352d78 > > >> ==> share/classes/sun/jvm/hotspot/HSDB.java >> >> In? showThreadStackMemory(), we have: >> >> 1101?????????????????????????? } >> 1102???????????????????????? } else if (collHeap instanceof ParallelScavengeHeap) { >> 1103?????????????????????????? ParallelScavengeHeap heap = (ParallelScavengeHeap) collHeap; >> 1104?????????????????????????? if (heap.youngGen().isIn(handle)) { >> 1105???????????????????????????? anno = "PSYoungGen "; >> 1106???????????????????????????? bad = false; >> 1107?????????????????????????? } else if (heap.oldGen().isIn(handle)) { >> 1108???????????????????????????? anno = "PSOldGen "; >> 1109???????????????????????????? bad = false; >> 1110?????????????????????????? } >> 1111???????????????????????? } else { >> 1112?????????????????????????? // Optimistically assume the oop isn't bad >> 1113?????????????????????????? anno = "[Unknown generation] "; >> 1114?????????????????????????? bad = false; >> 1115???????????????????????? } >> 1116 >> >> We would need to add the case of collHeap being an instanceof EpsilonHeap too. It would display >> "Unknown generation" while viewing the stack memory for the Java threads otherwise. > > Right, fixed: > http://hg.openjdk.java.net/jdk/sandbox/rev/f26c4a196a15 > > >> ==> It would be great if test/hotspot/jtreg/serviceability/sa/TestUniverse.java is enhanced to add >> the minimalistic test for EpsilonGC. > > Right: > http://hg.openjdk.java.net/jdk/sandbox/rev/c559de946c7d > > This still passes gc/epsilon and serviceability/sa tests. > > -Aleksey > From per.liden at oracle.com Thu Jun 7 10:13:56 2018 From: per.liden at oracle.com (Per Liden) Date: Thu, 7 Jun 2018 12:13:56 +0200 Subject: RFR: 8204180: Implementation: JEP 318: Epsilon GC (round 5) In-Reply-To: <2ee03550-2b1e-b873-5ef3-26312e2a760e@redhat.com> References: <2ee03550-2b1e-b873-5ef3-26312e2a760e@redhat.com> Message-ID: <7211f820-4999-2988-e06c-1067b1fb138f@oracle.com> On 06/06/2018 06:30 PM, Aleksey Shipilev wrote: > Hi, > > This is fifth (and hopefully final) round of code review for Epsilon GC changes. It includes the > fixes done as the result of fourth round of reviews, mostly in Serviceability. The build parts are > the same since last few reviews, so this is not posted to build-dev at . > > Webrev: > http://cr.openjdk.java.net/~shade/epsilon/webrev.09/ Still looks good to me! > > If we are good -- should somebody, e.g. Project Lead ack this? -- I am going to push this with the > following changeset metadata: You should be good to go, if the JEP is targeted and reviewers are happy. cheers, Per > > 8204180: Implementation: JEP 318: Epsilon, A No-Op Garbage Collector > Summary: Introduce Epsilon GC > Reviewed-by: rkennke, ihse, pliden, eosterlund, lmesnik, jgeorge > > Builds: > server X {x86_64, x86_32, aarch64, arm32, ppc64le, s390x} > minimal X {x86, x86_64} > zero X {x86_64} > > Testing: gc/epsilon on x86_64 > > Thanks, > -Aleksey > From shade at redhat.com Thu Jun 7 12:38:08 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 7 Jun 2018 14:38:08 +0200 Subject: RFR: 8204180: Implementation: JEP 318: Epsilon GC (round 5) In-Reply-To: <7211f820-4999-2988-e06c-1067b1fb138f@oracle.com> References: <2ee03550-2b1e-b873-5ef3-26312e2a760e@redhat.com> <7211f820-4999-2988-e06c-1067b1fb138f@oracle.com> Message-ID: On 06/07/2018 12:13 PM, Per Liden wrote: > On 06/06/2018 06:30 PM, Aleksey Shipilev wrote: >> Hi, >> >> This is fifth (and hopefully final) round of code review for Epsilon GC changes. It includes the >> fixes done as the result of fourth round of reviews, mostly in Serviceability. The build parts are >> the same since last few reviews, so this is not posted to build-dev at . >> >> Webrev: >> ?? http://cr.openjdk.java.net/~shade/epsilon/webrev.09/ > > Still looks good to me! > >> >> If we are good -- should somebody, e.g. Project Lead ack this? -- I am going to push this with the >> following changeset metadata: > > You should be good to go, if the JEP is targeted and reviewers are happy. Thanks Per, pending no other reviews, I am going to push this on Monday, June 11, after I get back from vacation. ZGC webrev is quite probably clashing with Epsilon in shared parts (e.g. CollectedHeap enums), so trivial rebases would be needed. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From per.liden at oracle.com Thu Jun 7 12:42:45 2018 From: per.liden at oracle.com (Per Liden) Date: Thu, 7 Jun 2018 14:42:45 +0200 Subject: RFR: 8204180: Implementation: JEP 318: Epsilon GC (round 5) In-Reply-To: References: <2ee03550-2b1e-b873-5ef3-26312e2a760e@redhat.com> <7211f820-4999-2988-e06c-1067b1fb138f@oracle.com> Message-ID: <0e63a401-d28d-2bdb-d024-3b48f3fba0cb@oracle.com> On 06/07/2018 02:38 PM, Aleksey Shipilev wrote: > On 06/07/2018 12:13 PM, Per Liden wrote: >> On 06/06/2018 06:30 PM, Aleksey Shipilev wrote: >>> Hi, >>> >>> This is fifth (and hopefully final) round of code review for Epsilon GC changes. It includes the >>> fixes done as the result of fourth round of reviews, mostly in Serviceability. The build parts are >>> the same since last few reviews, so this is not posted to build-dev at . >>> >>> Webrev: >>> ?? http://cr.openjdk.java.net/~shade/epsilon/webrev.09/ >> >> Still looks good to me! >> >>> >>> If we are good -- should somebody, e.g. Project Lead ack this? -- I am going to push this with the >>> following changeset metadata: >> >> You should be good to go, if the JEP is targeted and reviewers are happy. > > Thanks Per, pending no other reviews, I am going to push this on Monday, June 11, after I get back > from vacation. ZGC webrev is quite probably clashing with Epsilon in shared parts (e.g. > CollectedHeap enums), so trivial rebases would be needed. Yep, any conflicts there should be trivial to resolve. /Per From yasuenag at gmail.com Thu Jun 7 12:43:22 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Thu, 7 Jun 2018 21:43:22 +0900 Subject: RFR: 8204531: Remove unused chars following '\0' Message-ID: <6d19c245-eef8-96ce-3ca8-c3786d923c6a@gmail.com> Hi all, Please review this change: webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.00/ JBS: https://bugs.openjdk.java.net/browse/JDK-8204531 We can use `jhsdb jsnap` to check all PerfData. String values in PerfData are defined as jbyte array, but JSnap cannot handle it well as following: ``` $ jhsdb jsnap --pid 28542 --all | less sun.gc.cause=No GC^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ ``` You can see this value via `less` and `vim` on Linux. `^@` shows it is non-ascii character. PerfDataEntry has null-terminated C string. So we should restore as it in Java layer. Thanks, Yasumasa From gary.adams at oracle.com Thu Jun 7 12:48:52 2018 From: gary.adams at oracle.com (Gary Adams) Date: Thu, 07 Jun 2018 08:48:52 -0400 Subject: RFR: JDK-8169718: nsk/jdb/locals/locals002: ERROR: Cannot find boolVar with expected value: false In-Reply-To: <40d44ef3-97dc-ead0-158e-a5ca93d49b9e@oracle.com> References: <5B082D2E.7000408@oracle.com> <5d58fa2b-7dda-db64-0280-1e19e791a86d@oracle.com> <5B1814C9.6030807@oracle.com> <1d47721c-3498-92ee-997e-1d9fa216ea6f@oracle.com> <5B181D7F.1040300@oracle.com> <9794304d-d166-554a-399d-0f8b53fe8786@oracle.com> <5B182D6B.8060206@oracle.com> <40d44ef3-97dc-ead0-158e-a5ca93d49b9e@oracle.com> Message-ID: <5B1929B4.6020203@oracle.com> It's theoretically possible that the corrupted output could be produced with a single step command because of the separate event handler and command processing threads, although the failures that have been recorded were all observed later in a test scenario. So let's look at a third possible way to fix the issue. - we need a mechanism that will protect executeCommand and handleEvent - to ensure that neither produces a partial output A few key snippets from the code below. The method executeCommand() is called from both the command processing thread and the event handler thread. It is also called recursively, so a lock may not be appropriate The eventHandler output is produce incrementally - handleEvent produces the header, e.g. "Set completed:" - vmInterrupted() produces printCurrentLocation() monitor commands output (if any) by calling executeCommand() printPrompt() I don't have concrete fix to propose, yet, but I'm leaning towards having executeCommand buffer it's output and return it as a single string. It would involve a getPrompt() instead of printPrompt, so the text could be composed. It would also require aggregating any output that is currently sent directly to MessageOutput. It might also be addressed by enhancing MessageOutput to include some sort of buffering for the current event or current command. It's unlikely this would be ready before the jdk11 repos are cloned, so it may be best to target these bugs for the next release. src/jdk.jdi/share/classes/com/sun/tools/example/debug/tty/TTY.java: ... 209 public void vmInterrupted() { 210 Thread.yield(); // fetch output 211 printCurrentLocation(); 212 for (String cmd : monitorCommands) { 213 StringTokenizer t = new StringTokenizer(cmd); 214 t.nextToken(); // get rid of monitor number * 215 executeCommand(t);* 216 } 217 MessageOutput.printPrompt(); 218 } ... * 391 void executeCommand(StringTokenizer t) {* 392 String cmd = t.nextToken().toLowerCase(); 393 // Normally, prompt for the next command after this one is done 394 boolean showPrompt = true; 395 ... 589 if (showPrompt) { 590 MessageOutput.printPrompt(); 591 } 592 } src/jdk.jdi/share/classes/com/sun/tools/example/debug/tty/EventHandler.java: ... 66 public void run() { 67 EventQueue queue = Env.vm().eventQueue(); 68 while (connected) { 69 try { 70 EventSet eventSet = queue.remove(); 71 boolean resumeStoppedApp = false; 72 EventIterator it = eventSet.eventIterator(); 73 while (it.hasNext()) { * 74 resumeStoppedApp |= !handleEvent(it.nextEvent());* 75 } 76 77 if (resumeStoppedApp) { 78 eventSet.resume(); 79 } else if (eventSet.suspendPolicy() == EventRequest.SUSPEND_ALL) { 80 setCurrentThread(eventSet); * 81 notifier.vmInterrupted();* 82 } 83 } catch (InterruptedException exc) { 84 // Do nothing. Any changes will be seen at top of loop. 85 } catch (VMDisconnectedException discExc) { 86 handleDisconnectedException(); 87 break; 88 } 89 } 90 synchronized (this) { 91 completed = true; 92 notifyAll(); 93 } 94 } 96 private boolean handleEvent(Event event) { 97 notifier.receivedEvent(event); 98 99 if (event instanceof ExceptionEvent) { 100 return exceptionEvent(event); 101 } else if (event instanceof BreakpointEvent) { 102 return breakpointEvent(event); 103 } else if (event instanceof WatchpointEvent) { 104 return fieldWatchEvent(event); 105 } else if (event instanceof StepEvent) { 106 return stepEvent(event); 107 } else if (event instanceof MethodEntryEvent) { 108 return methodEntryEvent(event); 109 } else if (event instanceof MethodExitEvent) { 110 return methodExitEvent(event); 111 } else if (event instanceof ClassPrepareEvent) { 112 return classPrepareEvent(event); 113 } else if (event instanceof ClassUnloadEvent) { 114 return classUnloadEvent(event); 115 } else if (event instanceof ThreadStartEvent) { 116 return threadStartEvent(event); 117 } else if (event instanceof ThreadDeathEvent) { 118 return threadDeathEvent(event); 119 } else if (event instanceof VMStartEvent) { 120 return vmStartEvent(event); 121 } else { 122 return handleExitEvent(event); 123 } 124 } On 6/6/18, 3:27 PM, Chris Plummer wrote: ... > > The point I'm trying to get across is I don't think the issue is with > the rate of commands being sent. I think the issue can be isolated to > a single step command that has the first prompt appearing in the > middle of the "Step completed: ..." output. > > command thread: receive and execute "step" command > event thread: receive "step" breakpoint. print "Step completed: " > command thread: print "main[1]" > event thread: print text that comes after "Step completed: " > event thread: print "main[1]" > > This has to be fixed in the sender of the output, not the receiver of > the output (or the sender of the command). > > Chris ... -------------- next part -------------- An HTML attachment was scrubbed... URL: From calvin.cheung at oracle.com Thu Jun 7 16:54:59 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Thu, 07 Jun 2018 09:54:59 -0700 Subject: RFR(S): 8204110: serviceability/sa/ClhsdbSymbol.java and ClhsdbInspect.java failed when running in CDS mode Message-ID: <5B196363.5000709@oracle.com> bug: https://bugs.openjdk.java.net/browse/JDK-8204110 webrev: http://cr.openjdk.java.net/~ccheung/8204110/webrev.00/ For fixing the ClhsdbSymbol.java test, I've added dumping the symbols from the sharedTable if it isn't null. The change touches all 3 files in the webrev. SymbolTable.java - a helper function to return the sharedTable. CompactHashTable.java - added the SymbolVisitor interface and a symbolsDo() method; similar interface and method are also in SymbolTable.java CommandProcessor.java - see lines 649 - 659 for printing the symbols of the sharedTable. For fixing the ClhsdbInspect.java test, see lines 1063 - 1071 of CommandProcessor.java. I've adapted the block of code from BasicTypeDataBase.findDynamicTypeForAddress() method which can't be called directly from CommandProcessor because the Type isn't known at that point. Ran the tests with and without CDS on linux-x64. Will run tests on other platforms. thanks, Calvin From bob.vandette at oracle.com Thu Jun 7 17:43:23 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Thu, 7 Jun 2018 13:43:23 -0400 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> Message-ID: Can I get one more reviewer for this RFE so I can integrate it? > http://cr.openjdk.java.net/~bobv/8203357/webrev.01 Mandy Chung has reviewed this change. I?ve run Mach5 hotspot and core lib tests. I?ve reviewed the tests which were written by Harsha Wardhana I filed a CSR for the command line change and it?s now approved and closed. Thanks, Bob. > On May 30, 2018, at 3:45 PM, Bob Vandette wrote: > > Please review the following RFE which adds an internal API, along with jtreg tests that provide > access to Docker container configuration data and metrics. In addition to the API which we hope to > take advantage of in the future with Java Flight Recorder and a JMX Mbean, I?ve added an additional > option to -XshowSettings:system than dumps out the container or host cgroup confguration > information. See the sample output below: > > RFE: Container Metrics > > https://bugs.openjdk.java.net/browse/JDK-8203357 > > WEBREV: > > http://cr.openjdk.java.net/~bobv/8203357/webrev.01 > > > This commit will also include a fix for the following bug. > > BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails > > https://bugs.openjdk.java.net/browse/JDK-8203691 > > WEBREV: > > http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html > > SAMPLE USAGE and OUTPUT: > > docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash > ./java -XshowSettings:system > Operating System Metrics: > Provider: cgroupv1 > Effective CPU Count: 4 > CPU Period: 100000 > CPU Quota: -1 > CPU Shares: -1 > List of Processors, 4 total: > 4 5 6 7 > List of Effective Processors, 4 total: > 4 5 6 7 > List of Memory Nodes, 2 total: > 0 1 > List of Available Memory Nodes, 2 total: > 0 1 > CPUSet Memory Pressure Enabled: false > Memory Limit: 256.00M > Memory Soft Limit: Unlimited > Memory & Swap Limit: 512.00M > Kernel Memory Limit: Unlimited > TCP Memory Limit: Unlimited > Out Of Memory Killer Enabled: true > > TEST RESULTS: > > testing runtime container APIs > Directory "JTwork" not found: creating > Passed: runtime/containers/cgroup/PlainRead.java > Passed: runtime/containers/docker/DockerBasicTest.java > Passed: runtime/containers/docker/TestCPUAwareness.java > Passed: runtime/containers/docker/TestCPUSets.java > Passed: runtime/containers/docker/TestMemoryAwareness.java > Passed: runtime/containers/docker/TestMisc.java > Test results: passed: 6 > Results written to /export/users/bobv/jdk11/build/jtreg/JTwork > > testing jdk.internal.platform APIs > Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java > Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java > Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java > Passed: jdk/internal/platform/docker/TestSystemMetrics.java > Test results: passed: 4 > Results written to /export/users/bobv/jdk11/build/jtreg/JTwork > > testing -XshowSettings:system launcher option > Passed: tools/launcher/Settings.java > Test results: passed: 1 > > > Bob. > > From mikhailo.seledtsov at oracle.com Thu Jun 7 21:30:24 2018 From: mikhailo.seledtsov at oracle.com (Mikhailo Seledtsov) Date: Thu, 07 Jun 2018 14:30:24 -0700 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> Message-ID: <5B19A3F0.1010804@oracle.com> Hi Bob, I looked at the tests. In general they look good. I am a bit concerned about the use of ERROR_MARGIN in one of the tests. We need to make sure that the tests are stable, and do not produce intermittent failures. Thank you, Misha On 6/7/18, 10:43 AM, Bob Vandette wrote: > Can I get one more reviewer for this RFE so I can integrate it? > >> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 > Mandy Chung has reviewed this change. > > I?ve run Mach5 hotspot and core lib tests. > > I?ve reviewed the tests which were written by Harsha Wardhana > > I filed a CSR for the command line change and it?s now approved and closed. > > Thanks, > Bob. > > >> On May 30, 2018, at 3:45 PM, Bob Vandette wrote: >> >> Please review the following RFE which adds an internal API, along with jtreg tests that provide >> access to Docker container configuration data and metrics. In addition to the API which we hope to >> take advantage of in the future with Java Flight Recorder and a JMX Mbean, I?ve added an additional >> option to -XshowSettings:system than dumps out the container or host cgroup confguration >> information. See the sample output below: >> >> RFE: Container Metrics >> >> https://bugs.openjdk.java.net/browse/JDK-8203357 >> >> WEBREV: >> >> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >> >> >> This commit will also include a fix for the following bug. >> >> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >> >> https://bugs.openjdk.java.net/browse/JDK-8203691 >> >> WEBREV: >> >> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >> >> SAMPLE USAGE and OUTPUT: >> >> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >> ./java -XshowSettings:system >> Operating System Metrics: >> Provider: cgroupv1 >> Effective CPU Count: 4 >> CPU Period: 100000 >> CPU Quota: -1 >> CPU Shares: -1 >> List of Processors, 4 total: >> 4 5 6 7 >> List of Effective Processors, 4 total: >> 4 5 6 7 >> List of Memory Nodes, 2 total: >> 0 1 >> List of Available Memory Nodes, 2 total: >> 0 1 >> CPUSet Memory Pressure Enabled: false >> Memory Limit: 256.00M >> Memory Soft Limit: Unlimited >> Memory& Swap Limit: 512.00M >> Kernel Memory Limit: Unlimited >> TCP Memory Limit: Unlimited >> Out Of Memory Killer Enabled: true >> >> TEST RESULTS: >> >> testing runtime container APIs >> Directory "JTwork" not found: creating >> Passed: runtime/containers/cgroup/PlainRead.java >> Passed: runtime/containers/docker/DockerBasicTest.java >> Passed: runtime/containers/docker/TestCPUAwareness.java >> Passed: runtime/containers/docker/TestCPUSets.java >> Passed: runtime/containers/docker/TestMemoryAwareness.java >> Passed: runtime/containers/docker/TestMisc.java >> Test results: passed: 6 >> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >> >> testing jdk.internal.platform APIs >> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >> Test results: passed: 4 >> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >> >> testing -XshowSettings:system launcher option >> Passed: tools/launcher/Settings.java >> Test results: passed: 1 >> >> >> Bob. >> >> From chris.plummer at oracle.com Thu Jun 7 23:21:19 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 7 Jun 2018 16:21:19 -0700 Subject: RFR: JDK-8169718: nsk/jdb/locals/locals002: ERROR: Cannot find boolVar with expected value: false In-Reply-To: <5B1929B4.6020203@oracle.com> References: <5B082D2E.7000408@oracle.com> <5d58fa2b-7dda-db64-0280-1e19e791a86d@oracle.com> <5B1814C9.6030807@oracle.com> <1d47721c-3498-92ee-997e-1d9fa216ea6f@oracle.com> <5B181D7F.1040300@oracle.com> <9794304d-d166-554a-399d-0f8b53fe8786@oracle.com> <5B182D6B.8060206@oracle.com> <40d44ef3-97dc-ead0-158e-a5ca93d49b9e@oracle.com> <5B1929B4.6020203@oracle.com> Message-ID: <7d2f6bd3-1411-0d20-b5bb-d91fdb4b1ad6@oracle.com> An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Fri Jun 8 01:47:10 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 8 Jun 2018 11:47:10 +1000 Subject: RFR: 8204531: Remove unused chars following '\0' In-Reply-To: <6d19c245-eef8-96ce-3ca8-c3786d923c6a@gmail.com> References: <6d19c245-eef8-96ce-3ca8-c3786d923c6a@gmail.com> Message-ID: <17c32d23-c2dc-884c-c9ae-15c7afd72bd9@oracle.com> Hi Yasumasa, On 7/06/2018 10:43 PM, Yasumasa Suenaga wrote: > Hi all, > > Please review this change: > > ? webrev: > ??? http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.00/ > > ? JBS: > ??? https://bugs.openjdk.java.net/browse/JDK-8204531 > > > We can use `jhsdb jsnap` to check all PerfData. > String values in PerfData are defined as jbyte array, but JSnap cannot > handle it well as following: > > ``` > $ jhsdb jsnap --pid 28542 --all | less > > sun.gc.cause=No > GC^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ > > ``` > > You can see this value via `less` and `vim` on Linux. `^@` shows it is > non-ascii character. So your synopsis says "remove unused chars following \0". Is that really what is happening? I would have expected "new String(bytes, encoding)" to stop processing bytes when it encounters a NUL! > PerfDataEntry has null-terminated C string. So we should restore as it > in Java layer. If the issue is really "junk" beyond the \0 nul-terminator, how did we get such values? Shouldn't the byte array already be terminated at the NUL? Or is this just a raw representation of an array in the VM using the exact length of the array regardless of content? I see CStringUtilities.getString() both stops when it encounters a 0 value, and uses the default file encoding (else ASCII). The fix seems perfectly reasonable, I'm just unclear where exactly this "bad" data should have been stopped. Thanks, David > > Thanks, > > Yasumasa > From harsha.wardhana.b at oracle.com Fri Jun 8 04:27:40 2018 From: harsha.wardhana.b at oracle.com (Harsha Wardhana B) Date: Fri, 8 Jun 2018 09:57:40 +0530 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: <5B19A3F0.1010804@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <5B19A3F0.1010804@oracle.com> Message-ID: Hi Misha, The ERROR_MARGIN in tests was introduced to make the tests stable. There are times where metric values (specifically CPU usage) can change drastically in between two reads. The metrics value got from the API and the cgroup file can be different and 0.1 ERROR_MARGIN should take care of that, though at times even that may not be enough. Hence the CPU usages related the tests only print a warning if ERROR_MARGIN is exceeded. Thanks Harsha On Friday 08 June 2018 03:00 AM, Mikhailo Seledtsov wrote: > Hi Bob, > > ? I looked at the tests. In general they look good. I am a bit > concerned about the use of ERROR_MARGIN in one of the tests. We need > to make sure that the tests are stable, and do not produce > intermittent failures. > > > Thank you, > Misha > > On 6/7/18, 10:43 AM, Bob Vandette wrote: >> Can I get one more reviewer for this RFE so I can integrate it? >> >>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >> Mandy Chung has reviewed this change. >> >> I?ve run Mach5 hotspot and core lib tests. >> >> I?ve reviewed the tests which were written by Harsha Wardhana >> >> I filed a CSR for the command line change and it?s now approved and >> closed. >> >> Thanks, >> Bob. >> >> >>> On May 30, 2018, at 3:45 PM, Bob Vandette? >>> wrote: >>> >>> Please review the following RFE which adds an internal API, along >>> with jtreg tests that provide >>> access to Docker container configuration data and metrics.? In >>> addition to the API which we hope to >>> take advantage of in the future with Java Flight Recorder and a JMX >>> Mbean, I?ve added an additional >>> option to -XshowSettings:system than dumps out the container or host >>> cgroup confguration >>> information.? See the sample output below: >>> >>> RFE: Container Metrics >>> >>> https://bugs.openjdk.java.net/browse/JDK-8203357 >>> >>> WEBREV: >>> >>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>> >>> >>> This commit will also include a fix for the following bug. >>> >>> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >>> >>> https://bugs.openjdk.java.net/browse/JDK-8203691 >>> >>> WEBREV: >>> >>> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >>> >>> >>> SAMPLE USAGE and OUTPUT: >>> >>> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >>> ./java -XshowSettings:system >>> Operating System Metrics: >>> ??? Provider: cgroupv1 >>> ??? Effective CPU Count: 4 >>> ??? CPU Period: 100000 >>> ??? CPU Quota: -1 >>> ??? CPU Shares: -1 >>> ??? List of Processors, 4 total: >>> ??? 4 5 6 7 >>> ??? List of Effective Processors, 4 total: >>> ??? 4 5 6 7 >>> ??? List of Memory Nodes, 2 total: >>> ??? 0 1 >>> ??? List of Available Memory Nodes, 2 total: >>> ??? 0 1 >>> ??? CPUSet Memory Pressure Enabled: false >>> ??? Memory Limit: 256.00M >>> ??? Memory Soft Limit: Unlimited >>> ??? Memory&? Swap Limit: 512.00M >>> ??? Kernel Memory Limit: Unlimited >>> ??? TCP Memory Limit: Unlimited >>> ??? Out Of Memory Killer Enabled: true >>> >>> TEST RESULTS: >>> >>> testing runtime container APIs >>> Directory "JTwork" not found: creating >>> Passed: runtime/containers/cgroup/PlainRead.java >>> Passed: runtime/containers/docker/DockerBasicTest.java >>> Passed: runtime/containers/docker/TestCPUAwareness.java >>> Passed: runtime/containers/docker/TestCPUSets.java >>> Passed: runtime/containers/docker/TestMemoryAwareness.java >>> Passed: runtime/containers/docker/TestMisc.java >>> Test results: passed: 6 >>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>> >>> testing jdk.internal.platform APIs >>> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >>> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >>> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >>> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >>> Test results: passed: 4 >>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>> >>> testing -XshowSettings:system launcher option >>> Passed: tools/launcher/Settings.java >>> Test results: passed: 1 >>> >>> >>> Bob. >>> >>> From harsha.wardhana.b at oracle.com Fri Jun 8 04:30:39 2018 From: harsha.wardhana.b at oracle.com (Harsha Wardhana B) Date: Fri, 8 Jun 2018 10:00:39 +0530 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: <5B19A3F0.1010804@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <5B19A3F0.1010804@oracle.com> Message-ID: <78ad68bd-0020-836f-7511-2b7a49dc6d73@oracle.com> [Replying to all mailing-lists] Hi Misha, The ERROR_MARGIN in tests was introduced to make the tests stable. There are times where metric values (specifically CPU usage) can change drastically in between two reads. The metrics value got from the API and the cgroup file can be different and 0.1 ERROR_MARGIN should take care of that, though at times even that may not be enough. Hence the CPU usage related tests only print a warning if ERROR_MARGIN is exceeded. Thanks Harsha On Friday 08 June 2018 03:00 AM, Mikhailo Seledtsov wrote: > Hi Bob, > > ? I looked at the tests. In general they look good. I am a bit > concerned about the use of ERROR_MARGIN in one of the tests. We need > to make sure that the tests are stable, and do not produce > intermittent failures. > > > Thank you, > Misha > > On 6/7/18, 10:43 AM, Bob Vandette wrote: >> Can I get one more reviewer for this RFE so I can integrate it? >> >>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >> Mandy Chung has reviewed this change. >> >> I?ve run Mach5 hotspot and core lib tests. >> >> I?ve reviewed the tests which were written by Harsha Wardhana >> >> I filed a CSR for the command line change and it?s now approved and >> closed. >> >> Thanks, >> Bob. >> >> >>> On May 30, 2018, at 3:45 PM, Bob Vandette? >>> wrote: >>> >>> Please review the following RFE which adds an internal API, along >>> with jtreg tests that provide >>> access to Docker container configuration data and metrics.? In >>> addition to the API which we hope to >>> take advantage of in the future with Java Flight Recorder and a JMX >>> Mbean, I?ve added an additional >>> option to -XshowSettings:system than dumps out the container or host >>> cgroup confguration >>> information.? See the sample output below: >>> >>> RFE: Container Metrics >>> >>> https://bugs.openjdk.java.net/browse/JDK-8203357 >>> >>> WEBREV: >>> >>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>> >>> >>> This commit will also include a fix for the following bug. >>> >>> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >>> >>> https://bugs.openjdk.java.net/browse/JDK-8203691 >>> >>> WEBREV: >>> >>> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >>> >>> >>> SAMPLE USAGE and OUTPUT: >>> >>> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >>> ./java -XshowSettings:system >>> Operating System Metrics: >>> ??? Provider: cgroupv1 >>> ??? Effective CPU Count: 4 >>> ??? CPU Period: 100000 >>> ??? CPU Quota: -1 >>> ??? CPU Shares: -1 >>> ??? List of Processors, 4 total: >>> ??? 4 5 6 7 >>> ??? List of Effective Processors, 4 total: >>> ??? 4 5 6 7 >>> ??? List of Memory Nodes, 2 total: >>> ??? 0 1 >>> ??? List of Available Memory Nodes, 2 total: >>> ??? 0 1 >>> ??? CPUSet Memory Pressure Enabled: false >>> ??? Memory Limit: 256.00M >>> ??? Memory Soft Limit: Unlimited >>> ??? Memory&? Swap Limit: 512.00M >>> ??? Kernel Memory Limit: Unlimited >>> ??? TCP Memory Limit: Unlimited >>> ??? Out Of Memory Killer Enabled: true >>> >>> TEST RESULTS: >>> >>> testing runtime container APIs >>> Directory "JTwork" not found: creating >>> Passed: runtime/containers/cgroup/PlainRead.java >>> Passed: runtime/containers/docker/DockerBasicTest.java >>> Passed: runtime/containers/docker/TestCPUAwareness.java >>> Passed: runtime/containers/docker/TestCPUSets.java >>> Passed: runtime/containers/docker/TestMemoryAwareness.java >>> Passed: runtime/containers/docker/TestMisc.java >>> Test results: passed: 6 >>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>> >>> testing jdk.internal.platform APIs >>> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >>> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >>> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >>> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >>> Test results: passed: 4 >>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>> >>> testing -XshowSettings:system launcher option >>> Passed: tools/launcher/Settings.java >>> Test results: passed: 1 >>> >>> >>> Bob. >>> >>> From jini.george at oracle.com Fri Jun 8 05:04:08 2018 From: jini.george at oracle.com (Jini George) Date: Fri, 8 Jun 2018 10:34:08 +0530 Subject: RFR(S): 8204110: serviceability/sa/ClhsdbSymbol.java and ClhsdbInspect.java failed when running in CDS mode In-Reply-To: <5B196363.5000709@oracle.com> References: <5B196363.5000709@oracle.com> Message-ID: Your changes look good to me, Calvin. Thanks, Jini. On 6/7/2018 10:24 PM, Calvin Cheung wrote: > bug: https://bugs.openjdk.java.net/browse/JDK-8204110 > webrev: http://cr.openjdk.java.net/~ccheung/8204110/webrev.00/ > > For fixing the ClhsdbSymbol.java test, I've added dumping the symbols > from the sharedTable if it isn't null. > > The change touches all 3 files in the webrev. > > SymbolTable.java - a helper function to return the sharedTable. > > CompactHashTable.java - added the SymbolVisitor interface and a > symbolsDo() method; similar interface and method are also in > SymbolTable.java > > CommandProcessor.java - see lines 649 - 659 for printing the symbols of > the sharedTable. > > For fixing the ClhsdbInspect.java test, see lines 1063 - 1071 of > CommandProcessor.java. > I've adapted the block of code from > BasicTypeDataBase.findDynamicTypeForAddress() method which can't be > called directly from CommandProcessor because the Type isn't known at > that point. > > Ran the tests with and without CDS on linux-x64. > Will run tests on other platforms. > > thanks, > Calvin From yasuenag at gmail.com Fri Jun 8 06:38:45 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Fri, 8 Jun 2018 15:38:45 +0900 Subject: RFR: 8204531: Remove unused chars following '\0' In-Reply-To: <17c32d23-c2dc-884c-c9ae-15c7afd72bd9@oracle.com> References: <6d19c245-eef8-96ce-3ca8-c3786d923c6a@gmail.com> <17c32d23-c2dc-884c-c9ae-15c7afd72bd9@oracle.com> Message-ID: Hi David, 2018-06-08 10:47 GMT+09:00 David Holmes : > Hi Yasumasa, > > On 7/06/2018 10:43 PM, Yasumasa Suenaga wrote: >> >> Hi all, >> >> Please review this change: >> >> webrev: >> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.00/ >> >> JBS: >> https://bugs.openjdk.java.net/browse/JDK-8204531 >> >> >> We can use `jhsdb jsnap` to check all PerfData. >> String values in PerfData are defined as jbyte array, but JSnap cannot >> handle it well as following: >> >> ``` >> $ jhsdb jsnap --pid 28542 --all | less >> >> sun.gc.cause=No >> GC^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ >> ``` >> >> You can see this value via `less` and `vim` on Linux. `^@` shows it is >> non-ascii character. > > > So your synopsis says "remove unused chars following \0". Is that really > what is happening? I would have expected "new String(bytes, encoding)" to > stop processing bytes when it encounters a NUL! String c'tor will continue to parse byte array even if it finds '\0'. You can see it on JShell as below: ------------- jshell> var bytes = new byte[]{(byte)'a', (byte)'b', (byte)'c', (byte)'\0', (byte)'d', (byte)'e', (byte)'f'} bytes ==> byte[7] { 97, 98, 99, 0, 100, 101, 102 } jshell> new String(bytes, "US-ASCII") $2 ==> "abc\000def" ------------- >> PerfDataEntry has null-terminated C string. So we should restore as it in >> Java layer. > > > If the issue is really "junk" beyond the \0 nul-terminator, how did we get > such values? Shouldn't the byte array already be terminated at the NUL? Or > is this just a raw representation of an array in the VM using the exact > length of the array regardless of content? I think JSnap shows the array regardless of content. > I see CStringUtilities.getString() both stops when it encounters a 0 value, > and uses the default file encoding (else ASCII). > > The fix seems perfectly reasonable, I'm just unclear where exactly this > "bad" data should have been stopped. Thanks! Yasumasa > Thanks, > David > > >> >> Thanks, >> >> Yasumasa >> > From thomas.stuefe at gmail.com Fri Jun 8 06:41:15 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 8 Jun 2018 08:41:15 +0200 Subject: RFC (csr): 8204595: Extend jcmd vm selection syntax to allow wildcards. Message-ID: Hi all, may I have thoughts and potentially reviews for the following CSR: https://bugs.openjdk.java.net/browse/JDK-8204595 This is a proposal to add simple wildcard matching to the jcmd VM selection process. Thanks & Best Regards, ..Thomas From gunter.haug at sap.com Fri Jun 8 14:05:17 2018 From: gunter.haug at sap.com (Haug, Gunter) Date: Fri, 8 Jun 2018 14:05:17 +0000 Subject: RFR(S): 8200720: Print additional information in thread dump (times, allocated bytes etc.) In-Reply-To: <626701f8-b71b-58dc-f837-e5e1d81ef187@oracle.com> References: <56817AB5-88FC-4FB9-A271-7206EA69ECA7@sap.com> <2728581E-0E32-4911-B1A9-79AD6CBA81B0@sap.com> <082ca98f-9a68-15a0-95b5-c4d5c733ab36@oracle.com> <933cdacc-9b6f-c8b5-e244-558f38e7888c@oracle.com> <55806aa8-f09b-e9a8-b512-b9676029c494@oracle.com> <0F497BC6-2569-4EF6-9B59-E24A8E1B3A37@sap.com> <5dc08f65e83f4374b8e1f73f218c6d6d@sap.com> <626701f8-b71b-58dc-f837-e5e1d81ef187@oracle.com> Message-ID: Hi all, thanks a lot for all the input! I have prepared a new version of the webrev incorporating the suggestions you made (at least I tried): http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v3/ This version outputs the thread times unconditionally while the other information is guarded by a flags. I think, most participants found the thread times the most valuable information and had no (strong) objections to adding them unconditionally. @David Implementation is much simpler with conditional output only for JavaThreads. I could get rid of the terrible hack (changing the flag) without having to change too many source files. The information on allocated bytes is present in the Thread class already before this proposed change, I just print it. It might be sensible to move _allocated_bytes and the respective methods to the ThreadStatisticInfo class as G?tz suggested. I haven?t done that in the current version, though. @David and Thomas I've removed the pthread-id output. I'm unsure myself what it could be good for. Anyway, we could add it with a separate change (with a better implementation) if there is a need to. @Chris As you have written, the user has no direct contact to the attach listener of the VM and jstack won't misinterpret e.g. -help. I agree that the parsing in attachListener.cpp could be more robust. However, it hasn't been so far either. All that is done in the current implementation is a strcmp to -l everything else will be silently ignored. I can try to make this more robust or we could say that the new output is only available via jcmd. OTH nothing bad can happen with the current version of the proposed change, so we could also leave it as it is. What would you prefer? @Kirk and Thomas Implementing a new diagnostic command is certainly well worth a thought. However, it appears to me that it's not necessary in this case. There are already flags to jstack and jcmd Thread.print, so it's not that uncommon. The amount of information that would be available by a new diagnostic command and that is now added to the thread dump is quite small and I think that is tolerable. Moreover, and most importantly, our support team is use to find the information in the thread dump and it would make things easier to us if I could leave it there. Thanks again and have a nice weekend, Gunter ?On 06.06.18, 04:56, "David Holmes" wrote: Hi Goetz, On 5/06/2018 11:07 PM, Lindenmaier, Goetz wrote: > Hi > > this discussion touched a lot of points so far, which seem > to lead to different conclusions. > > I think we should look at the values printed: > > 1. cpu=6300.65ms elapsed=123.28s > Overhead > cpu time: > * system calls at thread dump time > elapsed time: > * 1 system call at thread creation time > * 1 64-bit field per thread for the thread start time > * 1 system call at thread dump time > > As I understand, JDK-8143176 would have had to get the > time at each locking, which is much more time critical > than doing this during thread creation. While Correct. > the time a lock was held would be much more useful, > the ratio here, combined with knowledge about the application, > could lead to the conclusion that the thread is wrongly > blocked at much lower cost. Agreed. I see no issue with unconditionally adding this information as it should address some of the concerns of 8143176 as well. > > 2. pthread-id=0x109708000 > Overhead: > * a field access at thread dump time. Negligible I would say. I'd overlooked this addition, but it is of course the chunk of ifdef code in osThread.cpp that I objected to. The problem I have here is the convoluted mess of "thread identifiers" that we already have. We currently print: st->print("tid=" INTPTR_FORMAT " ", p2i(this)); which is just the address of the Thread/JavaThread object. Then: st->print("nid=0x%x ", thread_id()); where 'n' is supposed to represent "native thread id", which is: - linux: kernel thread id from syscall gettid - solaris: thread library identity from thr_create() or thr_self() - windows: thread id from _beginthreadex - OS X: kernel thread id from pthread_mach_thread_np(pthread_self()); - Other BSD: kernel (?) thread id from syscall thr_self or getthrid - AIX: thread library identity from pthread_create() or pthread_self() It's telling that the code Gunter added gets the thread library id on linux, but a "kernel thread id" on other platforms! Do we want to see a thread library id and a kernel thread id? IIRC nid is supposed be the id you need to see the thread in a debugger. In which case I'm unclear what role the thread id Gunter wants to add is playing? I don't think we need to see kernel ids in general, and pthread-id isn't useful for debugging is it? > > 3. allocated=242236760B defined_classes=1725 > Overhead: > * 1 64-bit field per thread. > * counter increment on class creation > > Especially defined_classes seems to be controversial. > As this only makes sense for Java threads, this could > be printed in a line of it's own in Threads::print_on_error() > (which already gets a flag to customize for jstack: > print_concurrent_locks) by calling a dedicated function in Thread. > No need for flag PrintExtendedThreadInfo thus. > > But I would propose to drop this information as it is > too controversial. Agreed. > This leaves the times and the pthread-id to be decided whether > they are worth the cost. If so, I think they should be printed > unconditional. Agreed. With some refinement of the "thread id" issue. > If a flag is required to avoid bloating the info in the default case, > I would include more infos, as os_prio and nid under that flag. Of course those things exist for good reason. The native id is supposed to be what aids in you matching what you see in our thread dumps with what you see in a debugger. Priority is less of an issue these days, but there was a time ... ;-) > As jstack is deprecated, changes to jstack could be skipped always > disabling the new printouts. > > Finally, I would propose to move _allocated_bytes into > threadStatisticInfo.hpp. Didn't that part get dropped? (You don't need it for the time info or thread id). That's a GC logging issue to me. Thanks, David > Best regards, > Goetz. > > > > >> -----Original Message----- >> From: serviceability-dev [mailto:serviceability-dev- >> bounces at openjdk.java.net] On Behalf Of David Holmes >> Sent: Dienstag, 5. Juni 2018 04:53 >> To: Haug, Gunter ; Chris Plummer >> ; serviceability-dev > dev at openjdk.java.net>; Langer, Christoph ; >> hotspot-runtime-dev at openjdk.java.net >> Subject: Re: RFR(S): 8200720: Print additional information in thread dump >> (times, allocated bytes etc.) >> >> Hi Gunter, >> >> On 5/06/2018 1:27 AM, Haug, Gunter wrote: >>> Hi David, Chris, >>> >>> would it be an option to unconditionally print the additional information? >> Regardless which way this is implemented it will be rather complicated and it >> only switches on/off a few field in the thread dump. >> >> I'm not convinced this is all suitable information for a thread stack >> dump. I can see the time information being useful if using the dump to >> try and diagnoze a hang or responsiveness issue. But the allocated-bytes >> and classes-defined just doesn't seem useful in the context of a thread >> dump to me. Anyone interested in allocation stats is going to be looking >> at GC logs etc which is where this belongs. Ditto the class-stats belong >> in some kind of classloading logging IMHO. >> >> I'm very reluctant to burden the implementation with capturing these >> kinds of stats, just to use for diagnostic purposes that may not even be >> asked for. I'm very much in the "you shouldn't pay for what you don't >> use" camp in that regard. (See my comments in JDK-8143176 referenced by >> Sean.) >> >> The ThreadStatisticInfo adds overhead to every thread object - and I'd >> have to suspect there could be overlap with whatever flight recorder >> sticks in there too. (Thread has become even more bloated in recent time!). >> >> Cheers, >> David >> >> >>> Thanks, >>> Gunter >>> >>> On 04.06.18, 01:13, "David Holmes" wrote: >>> >>> Hi Gunter, Chris, >>> >>> Sorry just trying to catch up and this is only a partial look so far ... >>> >>> BTW these changes are not limited to serviceability code so adding in >>> runtime team as well. >>> >>> On 2/06/2018 9:12 AM, Chris Plummer wrote: >>> > Hi Gunter, >>> > >>> > On 6/1/18 3:17 AM, Haug, Gunter wrote: >>> >> Hi Chris, >>> >> >>> >> thanks for looking into this! >>> >> >>> >> Re the synchronization: The value is stored only in a VM operation at >>> >> a safepoint by the VM thread. I was of the opinion, that this could >>> >> not be interrupted by a second VM operation (of the same type). Or >> am >>> >> I missing something else? >>> > I was really thinking more about the temporary changing of >>> > PrintExtendedThreadInfo, not the value stored in the VMOp. You may >> be be >>> > correct that no more than one VMOp is executing, but while it is >>> > executing it is has changed the value of PrintExtendedThreadInfo, >> which >>> > might have an impact on anything else that triggers printing thread info >>> > while the VMOp is executing. >>> >>> Even if nothing else can possibly be running during the safepoint I find >>> it extremely bad form to change a command-line flag in this way, >>> particularly from a safepoint! >>> >>> If this flag is intended to be dynamically enabled as part of a dcmd >>> then it should also be a Manageable flag IMHO. >>> >>> That said ... >>> >>> >> I did think about passing an argument to the various print_on >> member >>> >> functions of the thread classes, but this would require rather >>> >> extensive code changes for a rather tiny extension. Therefore I feel >>> >> doing it like this is the lesser evil. >>> >>> ... it's obviously not truly a global setting, but one that is needed on >>> a per-print-request basis. The flag is just the default, but if the >>> default is off you still want to enable extended printing on a >>> per-request basis. >>> >>> The current print_on mechanics is not set up for this kind of >>> flexibility. I think this needs more thought. >>> >>> --- >>> >>> Re osThread.cpp shared code changes ... I really hate to see all the >>> ifdefs in there. Please add a pd_print_on function to support this if >>> you truly want platform specific information. >>> >>> --- >>> >>> threadStatisticInfo.hpp >>> >>> typo: getElepsedTime() -> getElapsedTime() >>> >>> Thanks, >>> David >>> >>> >>> >>> >> >>> >> Re thread_dump(): I think it's correct or, well, at least it works ;-) >>> >> In fact jstack will transfer the "-e" and "-l" in only one string, >>> >> i.e. op->arg(0). >>> > So you are saying that op->arg(0) is "-e -l" when using jstack? I think >>> > you really need to clean up the parsing. As it stands right now passing, >>> > I get the feeling that if a user erroneously asks for help by using >>> > "jcmd Thread.Print -help", it will end up executing with -e an -l >>> > enabled, and no failure for the invalid "-help" option. >>> >> Christoph has already explained my reasoning. But I agree, it's not >>> >> nice and I would be happy to do it like Christoph suggested. >>> > I'll respond separately to his suggestion. >>> > >>> > thanks, >>> > >>> > Chris >>> >> >>> >> And typo fixed, sorry. >>> >> >>> >> Thanks again, >>> >> Gunter >>> >> >>> >> On 01.06.18, 00:03, "Chris Plummer" >> wrote: >>> >> >>> >> Hi Gunter, >>> >> globals.hpp: fix typo "informatiuon" >>> >> I worry a little bit about the synchronizing (if that's the right >>> >> word) >>> >> of PrintExtendedThreadInfo and the dcmd's -e flag. When using -e, >>> >> you >>> >> are temporarily enabling PrintExtendedThreadInfo if it was false. >>> >> This >>> >> temporarily changes the behavior of thread dumps, and could >>> >> impact other >>> >> uses that happen in parallel. Also, could two simultaneous uses >>> >> of -e >>> >> result in PrintExtendedThreadInfo not getting restored properly? >>> >> thread_dump() doesn't look right. It looks like you are iterating >>> >> char >>> >> by char over the argument, and expect something like "-el" to be >>> >> specified rather then "-e -l". The loop should be iterating over >>> >> op->arg(i), not op->arg(0)[i]. >>> >> The rest of the changes look fine. >>> >> thanks, >>> >> Chris >>> >> On 5/30/18 8:12 AM, Haug, Gunter wrote: >>> >> > Hi all, >>> >> > >>> >> > As Chris proposed, I have made an the extended output >>> >> switchable. There is an VM flag (PrintExtendedThreadInfo), which is >>> >> false by default. Moreover, there is an Option (-e) which can be used >>> >> with jcmd Thread.print as well as with jstack. The -e option >>> >> essentially sets PrintExtendedThreadInfo true just for the respective >>> >> thread dump. >>> >> > >>> >> > Here is the updated webrev: >>> >> > >>> >> > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v2 >>> >> > >>> >> > (https://bugs.openjdk.java.net/browse/JDK-8200720) >>> >> > >>> >> > Thanks, >>> >> > Gunter >>> >> > >>> >> > >>> >> > On 02.05.18, 17:07, "serviceability-dev on behalf of Haug, >>> >> Gunter" >> >> gunter.haug at sap.com> wrote: >>> >> > >>> >> > Hi Chris, >>> >> > >>> >> > Thanks for looking into this. >>> >> > You're right, there is a little more we have. We have >>> >> implemented an IO tracing mechanism which - rather as a byproduct - >>> >> keeps track of bytes read and written per thread. However, this of >>> >> course requires changes not only in hotspot. We would be happy to >>> >> contribute this as well, but this is a far bigger change and will >>> >> probably lead to a far bigger discussion. Anyway, with the number of >>> >> bytes read, the number of classes defined doesn't look that arbitrary >>> >> anymore, as one can correlate IO to class loading. >>> >> > >>> >> > Regarding the verbose option I think that's a good idea! >>> >> > >>> >> > Thanks again, >>> >> > Gunter >>> >> > >>> >> > On 01.05.18, 22:55, "Chris Plummer" >>> >> wrote: >>> >> > >>> >> > Hi Gunter, >>> >> > >>> >> > The output you are adding is all useful. I think the >>> >> question is (and >>> >> > I'd like to see a few people chime in on this for this >>> >> review) is >>> >> > whether or not all of it is the appropriate for a >>> >> thread's stack dump. >>> >> > For example, defined_classes is on the fringe of what >>> >> I would call >>> >> > generally useful info in a stack dump. Sure, there >>> >> might be that rare >>> >> > case when you need it, but how often compared to other >>> >> useful info >>> >> > maintained on a per thread basis. How many other bits >>> >> of useful info are >>> >> > not being printed in favor of defined_classes? It >>> >> seems you have more in >>> >> > the queue. How many? I'm worried about how cluttered >>> >> the stack dumps >>> >> > will get. Maybe we should add some sort of verbose >>> >> thread dumping >>> >> > option. Just a thought. >>> >> > >>> >> > As for the implementation, overall it looks good, but >>> >> I can't speak to >>> >> > whether or not you are doing proper accounting of >>> >> defined_classes and >>> >> > bytes allocated. You'll need input from someone with >>> >> more knowledge of >>> >> > those areas. We'll also need to do some testing to >>> >> make sure tool tests >>> >> > are not impacted. >>> >> > >>> >> > thanks, >>> >> > >>> >> > Chris >>> >> > >>> >> > On 4/30/18 2:51 AM, Haug, Gunter wrote: >>> >> > > Hi, >>> >> > > >>> >> > > this is an update to an RFR I posted on hotspot-dev, >>> >> but it is probably more suitable to post it here. Can I please have a >>> >> review and a sponsor for the following enhancement: >>> >> > > >>> >> > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v1 >>> >> > > https://bugs.openjdk.java.net/browse/JDK-8200720 >>> >> > > >>> >> > > We at SAP have extended the thread dumps (obtained >>> >> by jstack or jcmd) by several fields providing thread specific >>> >> information. These extensions are quite popular with our support >> team. >>> >> With some knowledge of the architecture of the application, they >> often >>> >> allow for quick and simple diagnosis of a running system. Therefore >> we >>> >> would like to contribute these enhancements. >>> >> > > >>> >> > > I took a few simple examples here, namely cpu time, >>> >> elapsed time since thread creation, bytes allocated and classes >>> >> defined by the thread and the pthread-id or equivalent on platforms >>> >> where it makes sense. Provided it is known how the application >> should >>> >> behave, a misbehaving thread can identified easily. >>> >> > > >>> >> > > There is no measurable overhead for this >>> >> enhancement. However, it may be a problem that the format of the >>> >> output is changed. Tools parsing the output may have to be changed. >>> >> > > >>> >> > > Here is an example of the output generated: >>> >> > > >>> >> > > ------------------------------------------------------ >>> >> > > >>> >> > > "main" #1 prio=5 os_prio=31 cpu=6300.65ms >>> >> elapsed=123.28s allocated=242236760B defined_classes=1725 >>> >> tid=0x00007fa13a806000 nid=0x1c03 pthread-id=0x109708000 waiting >> on >>> >> condition [0x0000000109707000] >>> >> > > java.lang.Thread.State: TIMED_WAITING (sleeping) >>> >> > > JavaThread state: _thread_blocked >>> >> > > Thread: 0x00007fa13a806000 [0x1c03] State: >>> >> _at_safepoint _has_called_back 0 _at_poll_safepoint 0 >>> >> > > JavaThread state: _thread_blocked >>> >> > > at java.lang.Thread.sleep(java.base/Native Method) >>> >> > > ... >>> >> > > ------------------------------------------------------ >>> >> > > >>> >> > > As mentioned above, we have a whole bunch of other >>> >> enhancements to the thread dump similar to this one and would be >>> >> willing to contribute them if there is any interest. >>> >> > > >>> >> > > Thanks and best regards, >>> >> > > Gunter >>> >> > > >>> >> > > >>> >> > >>> >> > >>> >> > >>> >> > >>> >> > >>> >> >>> > >>> > >>> >>> From bob.vandette at oracle.com Fri Jun 8 15:03:23 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Fri, 8 Jun 2018 11:03:23 -0400 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: <78ad68bd-0020-836f-7511-2b7a49dc6d73@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <5B19A3F0.1010804@oracle.com> <78ad68bd-0020-836f-7511-2b7a49dc6d73@oracle.com> Message-ID: <4B4CBD97-1081-4DB6-871F-5BF292BF4DD0@oracle.com> I didn?t actually have any ERROR_MARGIN problems during testing. I had issues with the testCpuConsumption test in http://cr.openjdk.java.net/~bobv/8203357/webrev.01/test/lib/jdk/test/lib/containers/cgroup/MetricsTester.java.html I had to initialize the cpu usage values during setup rather than inside the test to ensure that sufficient cpu usage had occurred by the time the test was run. The original code executed and received the same values after attempting to exec a linux utility. My change uses the time taken to run several tests instead. This seems to have eliminated any intermittent failures. Bob. > On Jun 8, 2018, at 12:30 AM, Harsha Wardhana B wrote: > > [Replying to all mailing-lists] > Hi Misha, > > The ERROR_MARGIN in tests was introduced to make the tests stable. There are times where metric values (specifically CPU usage) can change drastically in between two reads. The metrics value got from the API and the cgroup file can be different and 0.1 ERROR_MARGIN should take care of that, though at times even that may not be enough. Hence the CPU usage related tests only print a warning if ERROR_MARGIN is exceeded. > > Thanks > Harsha > > On Friday 08 June 2018 03:00 AM, Mikhailo Seledtsov wrote: >> Hi Bob, >> >> I looked at the tests. In general they look good. I am a bit concerned about the use of ERROR_MARGIN in one of the tests. We need to make sure that the tests are stable, and do not produce intermittent failures. >> >> >> Thank you, >> Misha >> >> On 6/7/18, 10:43 AM, Bob Vandette wrote: >>> Can I get one more reviewer for this RFE so I can integrate it? >>> >>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>> Mandy Chung has reviewed this change. >>> >>> I?ve run Mach5 hotspot and core lib tests. >>> >>> I?ve reviewed the tests which were written by Harsha Wardhana >>> >>> I filed a CSR for the command line change and it?s now approved and closed. >>> >>> Thanks, >>> Bob. >>> >>> >>>> On May 30, 2018, at 3:45 PM, Bob Vandette wrote: >>>> >>>> Please review the following RFE which adds an internal API, along with jtreg tests that provide >>>> access to Docker container configuration data and metrics. In addition to the API which we hope to >>>> take advantage of in the future with Java Flight Recorder and a JMX Mbean, I?ve added an additional >>>> option to -XshowSettings:system than dumps out the container or host cgroup confguration >>>> information. See the sample output below: >>>> >>>> RFE: Container Metrics >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8203357 >>>> >>>> WEBREV: >>>> >>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>>> >>>> >>>> This commit will also include a fix for the following bug. >>>> >>>> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8203691 >>>> >>>> WEBREV: >>>> >>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >>>> >>>> SAMPLE USAGE and OUTPUT: >>>> >>>> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >>>> ./java -XshowSettings:system >>>> Operating System Metrics: >>>> Provider: cgroupv1 >>>> Effective CPU Count: 4 >>>> CPU Period: 100000 >>>> CPU Quota: -1 >>>> CPU Shares: -1 >>>> List of Processors, 4 total: >>>> 4 5 6 7 >>>> List of Effective Processors, 4 total: >>>> 4 5 6 7 >>>> List of Memory Nodes, 2 total: >>>> 0 1 >>>> List of Available Memory Nodes, 2 total: >>>> 0 1 >>>> CPUSet Memory Pressure Enabled: false >>>> Memory Limit: 256.00M >>>> Memory Soft Limit: Unlimited >>>> Memory& Swap Limit: 512.00M >>>> Kernel Memory Limit: Unlimited >>>> TCP Memory Limit: Unlimited >>>> Out Of Memory Killer Enabled: true >>>> >>>> TEST RESULTS: >>>> >>>> testing runtime container APIs >>>> Directory "JTwork" not found: creating >>>> Passed: runtime/containers/cgroup/PlainRead.java >>>> Passed: runtime/containers/docker/DockerBasicTest.java >>>> Passed: runtime/containers/docker/TestCPUAwareness.java >>>> Passed: runtime/containers/docker/TestCPUSets.java >>>> Passed: runtime/containers/docker/TestMemoryAwareness.java >>>> Passed: runtime/containers/docker/TestMisc.java >>>> Test results: passed: 6 >>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>> >>>> testing jdk.internal.platform APIs >>>> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >>>> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >>>> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >>>> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >>>> Test results: passed: 4 >>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>> >>>> testing -XshowSettings:system launcher option >>>> Passed: tools/launcher/Settings.java >>>> Test results: passed: 1 >>>> >>>> >>>> Bob. >>>> >>>> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From calvin.cheung at oracle.com Fri Jun 8 15:25:14 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Fri, 08 Jun 2018 08:25:14 -0700 Subject: RFR(S): 8204110: serviceability/sa/ClhsdbSymbol.java and ClhsdbInspect.java failed when running in CDS mode In-Reply-To: References: <5B196363.5000709@oracle.com> Message-ID: <5B1A9FDA.4060202@oracle.com> Thanks for your review, Jini. Calvin On 6/7/18, 10:04 PM, Jini George wrote: > Your changes look good to me, Calvin. > > Thanks, > Jini. > > On 6/7/2018 10:24 PM, Calvin Cheung wrote: >> bug: https://bugs.openjdk.java.net/browse/JDK-8204110 >> webrev: http://cr.openjdk.java.net/~ccheung/8204110/webrev.00/ >> >> For fixing the ClhsdbSymbol.java test, I've added dumping the symbols >> from the sharedTable if it isn't null. >> >> The change touches all 3 files in the webrev. >> >> SymbolTable.java - a helper function to return the sharedTable. >> >> CompactHashTable.java - added the SymbolVisitor interface and a >> symbolsDo() method; similar interface and method are also in >> SymbolTable.java >> >> CommandProcessor.java - see lines 649 - 659 for printing the symbols >> of the sharedTable. >> >> For fixing the ClhsdbInspect.java test, see lines 1063 - 1071 of >> CommandProcessor.java. >> I've adapted the block of code from >> BasicTypeDataBase.findDynamicTypeForAddress() method which can't be >> called directly from CommandProcessor because the Type isn't known at >> that point. >> >> Ran the tests with and without CDS on linux-x64. >> Will run tests on other platforms. >> >> thanks, >> Calvin From mikhailo.seledtsov at oracle.com Fri Jun 8 15:31:52 2018 From: mikhailo.seledtsov at oracle.com (mikhailo) Date: Fri, 8 Jun 2018 08:31:52 -0700 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: <78ad68bd-0020-836f-7511-2b7a49dc6d73@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <5B19A3F0.1010804@oracle.com> <78ad68bd-0020-836f-7511-2b7a49dc6d73@oracle.com> Message-ID: <4f9ac7ba-5a81-c9e4-feaa-42023d9f518f@oracle.com> Hi Harsha, ? Thank you for the explanation, makes sense to me. Please be aware, if a specific test turns out to be unstable in CI testing, it should be problem listed until solution is found to make it more stable. If the test is highly intermittent (fails intermittently but rarely) then it should be tagged with intermittent keyword. Overall tests look good to me, Thank you, Misha On 06/07/2018 09:30 PM, Harsha Wardhana B wrote: > [Replying to all mailing-lists] > Hi Misha, > > The ERROR_MARGIN in tests was introduced to make the tests stable. > There are times where metric values (specifically CPU usage) can > change drastically in between two reads. The metrics value got from > the API and the cgroup file can be different and 0.1 ERROR_MARGIN > should take care of that, though at times even that may not be enough. > Hence the CPU usage related tests only print a warning if ERROR_MARGIN > is exceeded. > > Thanks > Harsha > > On Friday 08 June 2018 03:00 AM, Mikhailo Seledtsov wrote: >> Hi Bob, >> >> ? I looked at the tests. In general they look good. I am a bit >> concerned about the use of ERROR_MARGIN in one of the tests. We need >> to make sure that the tests are stable, and do not produce >> intermittent failures. >> >> >> Thank you, >> Misha >> >> On 6/7/18, 10:43 AM, Bob Vandette wrote: >>> Can I get one more reviewer for this RFE so I can integrate it? >>> >>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>> Mandy Chung has reviewed this change. >>> >>> I?ve run Mach5 hotspot and core lib tests. >>> >>> I?ve reviewed the tests which were written by Harsha Wardhana >>> >>> I filed a CSR for the command line change and it?s now approved and >>> closed. >>> >>> Thanks, >>> Bob. >>> >>> >>>> On May 30, 2018, at 3:45 PM, Bob Vandette? >>>> wrote: >>>> >>>> Please review the following RFE which adds an internal API, along >>>> with jtreg tests that provide >>>> access to Docker container configuration data and metrics. In >>>> addition to the API which we hope to >>>> take advantage of in the future with Java Flight Recorder and a JMX >>>> Mbean, I?ve added an additional >>>> option to -XshowSettings:system than dumps out the container or >>>> host cgroup confguration >>>> information.? See the sample output below: >>>> >>>> RFE: Container Metrics >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8203357 >>>> >>>> WEBREV: >>>> >>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>>> >>>> >>>> This commit will also include a fix for the following bug. >>>> >>>> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8203691 >>>> >>>> WEBREV: >>>> >>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >>>> >>>> >>>> SAMPLE USAGE and OUTPUT: >>>> >>>> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >>>> ./java -XshowSettings:system >>>> Operating System Metrics: >>>> ??? Provider: cgroupv1 >>>> ??? Effective CPU Count: 4 >>>> ??? CPU Period: 100000 >>>> ??? CPU Quota: -1 >>>> ??? CPU Shares: -1 >>>> ??? List of Processors, 4 total: >>>> ??? 4 5 6 7 >>>> ??? List of Effective Processors, 4 total: >>>> ??? 4 5 6 7 >>>> ??? List of Memory Nodes, 2 total: >>>> ??? 0 1 >>>> ??? List of Available Memory Nodes, 2 total: >>>> ??? 0 1 >>>> ??? CPUSet Memory Pressure Enabled: false >>>> ??? Memory Limit: 256.00M >>>> ??? Memory Soft Limit: Unlimited >>>> ??? Memory&? Swap Limit: 512.00M >>>> ??? Kernel Memory Limit: Unlimited >>>> ??? TCP Memory Limit: Unlimited >>>> ??? Out Of Memory Killer Enabled: true >>>> >>>> TEST RESULTS: >>>> >>>> testing runtime container APIs >>>> Directory "JTwork" not found: creating >>>> Passed: runtime/containers/cgroup/PlainRead.java >>>> Passed: runtime/containers/docker/DockerBasicTest.java >>>> Passed: runtime/containers/docker/TestCPUAwareness.java >>>> Passed: runtime/containers/docker/TestCPUSets.java >>>> Passed: runtime/containers/docker/TestMemoryAwareness.java >>>> Passed: runtime/containers/docker/TestMisc.java >>>> Test results: passed: 6 >>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>> >>>> testing jdk.internal.platform APIs >>>> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >>>> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >>>> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >>>> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >>>> Test results: passed: 4 >>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>> >>>> testing -XshowSettings:system launcher option >>>> Passed: tools/launcher/Settings.java >>>> Test results: passed: 1 >>>> >>>> >>>> Bob. >>>> >>>> > From serguei.spitsyn at oracle.com Fri Jun 8 16:14:06 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 8 Jun 2018 09:14:06 -0700 Subject: RFR(S): 8204110: serviceability/sa/ClhsdbSymbol.java and ClhsdbInspect.java failed when running in CDS mode In-Reply-To: References: <5B196363.5000709@oracle.com> Message-ID: +1 Thanks, Serguei On 6/7/18 22:04, Jini George wrote: > Your changes look good to me, Calvin. > > Thanks, > Jini. > > On 6/7/2018 10:24 PM, Calvin Cheung wrote: >> bug: https://bugs.openjdk.java.net/browse/JDK-8204110 >> webrev: http://cr.openjdk.java.net/~ccheung/8204110/webrev.00/ >> >> For fixing the ClhsdbSymbol.java test, I've added dumping the symbols >> from the sharedTable if it isn't null. >> >> The change touches all 3 files in the webrev. >> >> SymbolTable.java - a helper function to return the sharedTable. >> >> CompactHashTable.java - added the SymbolVisitor interface and a >> symbolsDo() method; similar interface and method are also in >> SymbolTable.java >> >> CommandProcessor.java - see lines 649 - 659 for printing the symbols >> of the sharedTable. >> >> For fixing the ClhsdbInspect.java test, see lines 1063 - 1071 of >> CommandProcessor.java. >> I've adapted the block of code from >> BasicTypeDataBase.findDynamicTypeForAddress() method which can't be >> called directly from CommandProcessor because the Type isn't known at >> that point. >> >> Ran the tests with and without CDS on linux-x64. >> Will run tests on other platforms. >> >> thanks, >> Calvin From ioi.lam at oracle.com Fri Jun 8 16:26:34 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 8 Jun 2018 09:26:34 -0700 Subject: RFR(S): 8204110: serviceability/sa/ClhsdbSymbol.java and ClhsdbInspect.java failed when running in CDS mode In-Reply-To: <5B196363.5000709@oracle.com> References: <5B196363.5000709@oracle.com> Message-ID: <8a3a8e48-f7ab-0c20-df08-b94d53ad1198@oracle.com> Hi Calvin, The changes look good to me. Thanks! - Ioi On 6/7/18 9:54 AM, Calvin Cheung wrote: > bug: https://bugs.openjdk.java.net/browse/JDK-8204110 > webrev: http://cr.openjdk.java.net/~ccheung/8204110/webrev.00/ > > For fixing the ClhsdbSymbol.java test, I've added dumping the symbols > from the sharedTable if it isn't null. > > The change touches all 3 files in the webrev. > > SymbolTable.java - a helper function to return the sharedTable. > > CompactHashTable.java - added the SymbolVisitor interface and a > symbolsDo() method; similar interface and method are also in > SymbolTable.java > > CommandProcessor.java - see lines 649 - 659 for printing the symbols > of the sharedTable. > > For fixing the ClhsdbInspect.java test, see lines 1063 - 1071 of > CommandProcessor.java. > I've adapted the block of code from > BasicTypeDataBase.findDynamicTypeForAddress() method which can't be > called directly from CommandProcessor because the Type isn't known at > that point. > > Ran the tests with and without CDS on linux-x64. > Will run tests on other platforms. > > thanks, > Calvin From calvin.cheung at oracle.com Fri Jun 8 16:26:57 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Fri, 08 Jun 2018 09:26:57 -0700 Subject: RFR(S): 8204110: serviceability/sa/ClhsdbSymbol.java and ClhsdbInspect.java failed when running in CDS mode In-Reply-To: References: <5B196363.5000709@oracle.com> Message-ID: <5B1AAE51.9080501@oracle.com> Thanks for your review, Serguei. Calvin On 6/8/18, 9:14 AM, serguei.spitsyn at oracle.com wrote: > +1 > > Thanks, > Serguei > > On 6/7/18 22:04, Jini George wrote: >> Your changes look good to me, Calvin. >> >> Thanks, >> Jini. >> >> On 6/7/2018 10:24 PM, Calvin Cheung wrote: >>> bug: https://bugs.openjdk.java.net/browse/JDK-8204110 >>> webrev: http://cr.openjdk.java.net/~ccheung/8204110/webrev.00/ >>> >>> For fixing the ClhsdbSymbol.java test, I've added dumping the >>> symbols from the sharedTable if it isn't null. >>> >>> The change touches all 3 files in the webrev. >>> >>> SymbolTable.java - a helper function to return the sharedTable. >>> >>> CompactHashTable.java - added the SymbolVisitor interface and a >>> symbolsDo() method; similar interface and method are also in >>> SymbolTable.java >>> >>> CommandProcessor.java - see lines 649 - 659 for printing the symbols >>> of the sharedTable. >>> >>> For fixing the ClhsdbInspect.java test, see lines 1063 - 1071 of >>> CommandProcessor.java. >>> I've adapted the block of code from >>> BasicTypeDataBase.findDynamicTypeForAddress() method which can't be >>> called directly from CommandProcessor because the Type isn't known >>> at that point. >>> >>> Ran the tests with and without CDS on linux-x64. >>> Will run tests on other platforms. >>> >>> thanks, >>> Calvin > From calvin.cheung at oracle.com Fri Jun 8 16:29:20 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Fri, 08 Jun 2018 09:29:20 -0700 Subject: RFR(S): 8204110: serviceability/sa/ClhsdbSymbol.java and ClhsdbInspect.java failed when running in CDS mode In-Reply-To: <8a3a8e48-f7ab-0c20-df08-b94d53ad1198@oracle.com> References: <5B196363.5000709@oracle.com> <8a3a8e48-f7ab-0c20-df08-b94d53ad1198@oracle.com> Message-ID: <5B1AAEE0.9040502@oracle.com> Thanks for your review, Ioi. Calvin On 6/8/18, 9:26 AM, Ioi Lam wrote: > Hi Calvin, > > The changes look good to me. Thanks! > > - Ioi > > > On 6/7/18 9:54 AM, Calvin Cheung wrote: >> bug: https://bugs.openjdk.java.net/browse/JDK-8204110 >> webrev: http://cr.openjdk.java.net/~ccheung/8204110/webrev.00/ >> >> For fixing the ClhsdbSymbol.java test, I've added dumping the symbols >> from the sharedTable if it isn't null. >> >> The change touches all 3 files in the webrev. >> >> SymbolTable.java - a helper function to return the sharedTable. >> >> CompactHashTable.java - added the SymbolVisitor interface and a >> symbolsDo() method; similar interface and method are also in >> SymbolTable.java >> >> CommandProcessor.java - see lines 649 - 659 for printing the symbols >> of the sharedTable. >> >> For fixing the ClhsdbInspect.java test, see lines 1063 - 1071 of >> CommandProcessor.java. >> I've adapted the block of code from >> BasicTypeDataBase.findDynamicTypeForAddress() method which can't be >> called directly from CommandProcessor because the Type isn't known at >> that point. >> >> Ran the tests with and without CDS on linux-x64. >> Will run tests on other platforms. >> >> thanks, >> Calvin > From hohensee at amazon.com Fri Jun 8 17:13:35 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Fri, 8 Jun 2018 17:13:35 +0000 Subject: RFR (S): 8195115: G1 Old Gen MemoryPool CollectionUsage.used values don't reflect mixed GC results In-Reply-To: <9f972398-115f-06ad-1e0c-513abceb097a@oracle.com> References: <3A67BF1B-CE35-4759-B5B4-959C24020A45@amazon.com> <878D6678-84CB-471E-B126-AF14B1BC8EB9@amazon.com> <727f9bcb-9404-df32-917f-70aca6e31cb6@oracle.com> <4BA32239-D645-4B1A-A58E-502258C8FFB7@amazon.com> <997a766f-9f4b-f192-ae87-03f4c8d2666d@oracle.com> <64793ff3-4805-79f9-4e72-6e469cd511dc@oracle.com> <15754d34-f3c7-6431-7abf-05214bd6b9d2@oracle.com> <93417506-8e12-6ad3-690a-439e36719652@oracle.com> <99C937B6-F67B-4F4A-ACC8-444B5F83932B@amazon.com> <3922628B-81DE-4904-AEFE-F4163EB1E655@amazon.com> <8444d766-8adb-013f-f75b-b5d4420635df@oracle.com> <5768E4BF-5556-455B-899B-94A67BF940A0@amazon.com> <3E15AAD8-5017-4CCB-B927-6B31FD0D7809@amazon.com> <22216AD4-9A78-4427-9AB8-629EC685C296@amazon.com> <9f972398-115f-06ad-1e0c-513abceb097a@oracle.com> Message-ID: <48344C2E-2423-4CB1-A5C3-09223CA1ED78@amazon.com> Back after a long hiatus... Thanks, Eric, for your review. Here's a new webrev incorporating your recommendations. Bug: https://bugs.openjdk.java.net/browse/JDK-8195115 Webrev: http://cr.openjdk.java.net/~phh/8195115/webrev.02/ TIA for your re-review. Plus, may I have another reviewer look at it please? Paul ?On 2/26/18, 8:47 AM, "Erik Helin" wrote: Hi Paul, a couple of comments on the patch: - memoryService.hpp: + 150 bool countCollection, + 151 bool allMemoryPoolsAffected = true); There is no need to use a default value for the parameter allMemoryPoolsAffected here. Skipping the default value also allows you to put allMemoryPoolsAffected to TraceMemoryManager::initialize in the same relative position as for the constructor parameter (this will make the code more uniform and easier to follow). - memoryManager.cpp Instead of adding a default parameter, maybe add a new method? Something like GCMemoryManager::add_not_always_affected_pool() (I couldn't come up with a shorter name at the moment). - TestMixedOldGenCollectionUsage.java The test is too strict about how and when collections should occur. Tests written this way often become very brittle, they might e.g. fail to finish a concurrent mark on time on a very slow, single core, machine. It is better to either force collections by using the WhiteBox API or make the test more lenient. Thanks, Erik On 02/22/2018 09:54 PM, Hohensee, Paul wrote: > Ping for a review please. > > Thanks, > > Paul > > On 2/16/18, 12:26 PM, "serviceability-dev on behalf of Hohensee, Paul" wrote: > > The CSR https://bugs.openjdk.java.net/browse/JDK-8196719 for the original fix has been approved, so I?m back to requesting a code review, please. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8195115 > Webrev: http://cr.openjdk.java.net/~phh/8195115/webrev.hs.01/ > > Passed a submit repo run, passes its jtreg test, and a JDK8 version is in production use at Amazon. > > From the original RR: > > > The bug is that from the JMX point of view, G1?s incremental collector > > (misnamed as the ?G1 Young Generation? collector) only affects G1?s > > survivor and eden spaces. In fact, mixed collections run by this > > collector also affect the G1 old generation. > > > > This proposed fix is to record, for each of a JMX garbage collector's > > memory pools, whether that memory pool is affected by all collections > > using that collector. And, for each collection, record whether or not > > all the collector's memory pools are affected. After each collection, > > for each memory pool, if either all the collector's memory pools were > > affected or the memory pool is affected for all collections, record > > CollectionUsage for that pool. > > > > For collectors other than G1 Young Generation, all pools are recorded as > > affected by all collections and every collection is recorded as > > affecting all the collector?s memory pools. For the G1 Young Generation > > collector, the G1 Old Gen pool is recorded as not being affected by all > > collections, and non-mixed collections are recorded as not affecting all > > memory pools. The result is that for non-mixed collections, > > CollectionUsage is recorded after a collection only the G1 Eden Space > > and G1 Survivor Space pools, while for mixed collections CollectionUsage > > is recorded for G1 Old Gen as well. > > > > Other than the effect of the fix on G1 Old Gen MemoryPool. > > CollectionUsage, the only external behavior change is that > > GarbageCollectorMXBean.getMemoryPoolNames will now return 3 pool names > > rather than 2. > > > > With this fix, a collector?s memory pools can be divided into two > > disjoint subsets, one that participates in all collections and one that > > doesn?t. This is a bit more general than the minimum necessary to fix > > G1, but not by much. Because I expect it to apply to other incremental > > region-based collectors, I went with the more general solution. I > > minimized the amount of code I had to touch by using default parameters > > for GCMemoryManager::add_pool and the TraceMemoryManagerStats constructors. > > > From christoph.langer at sap.com Mon Jun 11 07:30:30 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Mon, 11 Jun 2018 07:30:30 +0000 Subject: RFR(S): 8200720: Print additional information in thread dump (times, allocated bytes etc.) In-Reply-To: References: <56817AB5-88FC-4FB9-A271-7206EA69ECA7@sap.com> <2728581E-0E32-4911-B1A9-79AD6CBA81B0@sap.com> <082ca98f-9a68-15a0-95b5-c4d5c733ab36@oracle.com> <933cdacc-9b6f-c8b5-e244-558f38e7888c@oracle.com> <55806aa8-f09b-e9a8-b512-b9676029c494@oracle.com> <0F497BC6-2569-4EF6-9B59-E24A8E1B3A37@sap.com> <5dc08f65e83f4374b8e1f73f218c6d6d@sap.com> <626701f8-b71b-58dc-f837-e5e1d81ef187@oracle.com> Message-ID: <2ca6b0ac27754fe5bbc34eb49ce60a67@sap.com> Hi Gunter, the change looks good to me now. I would think it is most pragmatic if parsing in the thread_dump function of attachListener.cpp could be done as you proposed. Let's wait if Chris is fine with it. Small nit: In src/jdk.jcmd/share/classes/sun/tools/jstack/JStack.java, around line 70, I'd prefer if you could write the if statements like this: if (arg.equals("-l")) { locks = true; } else if (arg.equals("-e")) { extended = true; } else { usage(1); } That'll keep the indentation a bit more flat... Best regards Christoph > -----Original Message----- > From: Haug, Gunter > Sent: Freitag, 8. Juni 2018 16:05 > To: David Holmes ; Lindenmaier, Goetz > ; Chris Plummer > ; serviceability-dev dev at openjdk.java.net>; Langer, Christoph ; > hotspot-runtime-dev at openjdk.java.net; Kirk Pepperdine > ; Thomas St?fe > Subject: Re: RFR(S): 8200720: Print additional information in thread dump > (times, allocated bytes etc.) > > Hi all, > > thanks a lot for all the input! I have prepared a new version of the webrev > incorporating the suggestions you made (at least I tried): > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v3/ > > This version outputs the thread times unconditionally while the other > information is guarded by a flags. I think, most participants found the thread > times the most valuable information and had no (strong) objections to > adding them unconditionally. > > @David > Implementation is much simpler with conditional output only for > JavaThreads. I could get rid of the terrible hack (changing the flag) without > having to change too many source files. > > The information on allocated bytes is present in the Thread class already > before this proposed change, I just print it. It might be sensible to move > _allocated_bytes and the respective methods to the ThreadStatisticInfo class > as G?tz suggested. I haven?t done that in the current version, though. > > @David and Thomas > I've removed the pthread-id output. I'm unsure myself what it could be good > for. Anyway, we could add it with a separate change (with a better > implementation) if there is a need to. > > @Chris > As you have written, the user has no direct contact to the attach listener of > the VM and jstack won't misinterpret e.g. -help. I agree that the parsing in > attachListener.cpp could be more robust. However, it hasn't been so far > either. All that is done in the current implementation is a strcmp to -l > everything else will be silently ignored. I can try to make this more robust or > we could say that the new output is only available via jcmd. OTH nothing bad > can happen with the current version of the proposed change, so we could > also leave it as it is. What would you prefer? > > @Kirk and Thomas > Implementing a new diagnostic command is certainly well worth a thought. > However, it appears to me that it's not necessary in this case. There are > already flags to jstack and jcmd Thread.print, so it's not that uncommon. The > amount of information that would be available by a new diagnostic command > and that is now added to the thread dump is quite small and I think that is > tolerable. Moreover, and most importantly, our support team is use to find > the information in the thread dump and it would make things easier to us if I > could leave it there. > > Thanks again and have a nice weekend, > Gunter > > > ?On 06.06.18, 04:56, "David Holmes" wrote: > > Hi Goetz, > > On 5/06/2018 11:07 PM, Lindenmaier, Goetz wrote: > > Hi > > > > this discussion touched a lot of points so far, which seem > > to lead to different conclusions. > > > > I think we should look at the values printed: > > > > 1. cpu=6300.65ms elapsed=123.28s > > Overhead > > cpu time: > > * system calls at thread dump time > > elapsed time: > > * 1 system call at thread creation time > > * 1 64-bit field per thread for the thread start time > > * 1 system call at thread dump time > > > > As I understand, JDK-8143176 would have had to get the > > time at each locking, which is much more time critical > > than doing this during thread creation. While > > Correct. > > > the time a lock was held would be much more useful, > > the ratio here, combined with knowledge about the application, > > could lead to the conclusion that the thread is wrongly > > blocked at much lower cost. > > Agreed. I see no issue with unconditionally adding this information as > it should address some of the concerns of 8143176 as well. > > > > > 2. pthread-id=0x109708000 > > Overhead: > > * a field access at thread dump time. Negligible I would say. > > I'd overlooked this addition, but it is of course the chunk of ifdef > code in osThread.cpp that I objected to. The problem I have here is the > convoluted mess of "thread identifiers" that we already have. We > currently print: > > st->print("tid=" INTPTR_FORMAT " ", p2i(this)); > > which is just the address of the Thread/JavaThread object. Then: > > st->print("nid=0x%x ", thread_id()); > > where 'n' is supposed to represent "native thread id", which is: > - linux: kernel thread id from syscall gettid > - solaris: thread library identity from thr_create() or thr_self() > - windows: thread id from _beginthreadex > - OS X: kernel thread id from pthread_mach_thread_np(pthread_self()); > - Other BSD: kernel (?) thread id from syscall thr_self or getthrid > - AIX: thread library identity from pthread_create() or pthread_self() > > It's telling that the code Gunter added gets the thread library id on > linux, but a "kernel thread id" on other platforms! Do we want to see a > thread library id and a kernel thread id? IIRC nid is supposed be the id > you need to see the thread in a debugger. In which case I'm unclear what > role the thread id Gunter wants to add is playing? I don't think we need > to see kernel ids in general, and pthread-id isn't useful for debugging > is it? > > > > > 3. allocated=242236760B defined_classes=1725 > > Overhead: > > * 1 64-bit field per thread. > > * counter increment on class creation > > > > Especially defined_classes seems to be controversial. > > As this only makes sense for Java threads, this could > > be printed in a line of it's own in Threads::print_on_error() > > (which already gets a flag to customize for jstack: > > print_concurrent_locks) by calling a dedicated function in Thread. > > No need for flag PrintExtendedThreadInfo thus. > > > > But I would propose to drop this information as it is > > too controversial. > > Agreed. > > > This leaves the times and the pthread-id to be decided whether > > they are worth the cost. If so, I think they should be printed > > unconditional. > > Agreed. With some refinement of the "thread id" issue. > > > If a flag is required to avoid bloating the info in the default case, > > I would include more infos, as os_prio and nid under that flag. > > Of course those things exist for good reason. The native id is supposed > to be what aids in you matching what you see in our thread dumps with > what you see in a debugger. Priority is less of an issue these days, but > there was a time ... ;-) > > > As jstack is deprecated, changes to jstack could be skipped always > > disabling the new printouts. > > > > Finally, I would propose to move _allocated_bytes into > > threadStatisticInfo.hpp. > > Didn't that part get dropped? (You don't need it for the time info or > thread id). That's a GC logging issue to me. > > Thanks, > David > > > Best regards, > > Goetz. > > > > > > > > > >> -----Original Message----- > >> From: serviceability-dev [mailto:serviceability-dev- > >> bounces at openjdk.java.net] On Behalf Of David Holmes > >> Sent: Dienstag, 5. Juni 2018 04:53 > >> To: Haug, Gunter ; Chris Plummer > >> ; serviceability-dev >> dev at openjdk.java.net>; Langer, Christoph > ; > >> hotspot-runtime-dev at openjdk.java.net > >> Subject: Re: RFR(S): 8200720: Print additional information in thread > dump > >> (times, allocated bytes etc.) > >> > >> Hi Gunter, > >> > >> On 5/06/2018 1:27 AM, Haug, Gunter wrote: > >>> Hi David, Chris, > >>> > >>> would it be an option to unconditionally print the additional > information? > >> Regardless which way this is implemented it will be rather complicated > and it > >> only switches on/off a few field in the thread dump. > >> > >> I'm not convinced this is all suitable information for a thread stack > >> dump. I can see the time information being useful if using the dump to > >> try and diagnoze a hang or responsiveness issue. But the allocated- > bytes > >> and classes-defined just doesn't seem useful in the context of a thread > >> dump to me. Anyone interested in allocation stats is going to be looking > >> at GC logs etc which is where this belongs. Ditto the class-stats belong > >> in some kind of classloading logging IMHO. > >> > >> I'm very reluctant to burden the implementation with capturing these > >> kinds of stats, just to use for diagnostic purposes that may not even be > >> asked for. I'm very much in the "you shouldn't pay for what you don't > >> use" camp in that regard. (See my comments in JDK-8143176 referenced > by > >> Sean.) > >> > >> The ThreadStatisticInfo adds overhead to every thread object - and I'd > >> have to suspect there could be overlap with whatever flight recorder > >> sticks in there too. (Thread has become even more bloated in recent > time!). > >> > >> Cheers, > >> David > >> > >> > >>> Thanks, > >>> Gunter > >>> > >>> On 04.06.18, 01:13, "David Holmes" > wrote: > >>> > >>> Hi Gunter, Chris, > >>> > >>> Sorry just trying to catch up and this is only a partial look so far ... > >>> > >>> BTW these changes are not limited to serviceability code so adding > in > >>> runtime team as well. > >>> > >>> On 2/06/2018 9:12 AM, Chris Plummer wrote: > >>> > Hi Gunter, > >>> > > >>> > On 6/1/18 3:17 AM, Haug, Gunter wrote: > >>> >> Hi Chris, > >>> >> > >>> >> thanks for looking into this! > >>> >> > >>> >> Re the synchronization: The value is stored only in a VM > operation at > >>> >> a safepoint by the VM thread. I was of the opinion, that this > could > >>> >> not be interrupted by a second VM operation (of the same > type). Or > >> am > >>> >> I missing something else? > >>> > I was really thinking more about the temporary changing of > >>> > PrintExtendedThreadInfo, not the value stored in the VMOp. You > may > >> be be > >>> > correct that no more than one VMOp is executing, but while it is > >>> > executing it is has changed the value of PrintExtendedThreadInfo, > >> which > >>> > might have an impact on anything else that triggers printing > thread info > >>> > while the VMOp is executing. > >>> > >>> Even if nothing else can possibly be running during the safepoint I > find > >>> it extremely bad form to change a command-line flag in this way, > >>> particularly from a safepoint! > >>> > >>> If this flag is intended to be dynamically enabled as part of a dcmd > >>> then it should also be a Manageable flag IMHO. > >>> > >>> That said ... > >>> > >>> >> I did think about passing an argument to the various print_on > >> member > >>> >> functions of the thread classes, but this would require rather > >>> >> extensive code changes for a rather tiny extension. Therefore I > feel > >>> >> doing it like this is the lesser evil. > >>> > >>> ... it's obviously not truly a global setting, but one that is needed on > >>> a per-print-request basis. The flag is just the default, but if the > >>> default is off you still want to enable extended printing on a > >>> per-request basis. > >>> > >>> The current print_on mechanics is not set up for this kind of > >>> flexibility. I think this needs more thought. > >>> > >>> --- > >>> > >>> Re osThread.cpp shared code changes ... I really hate to see all the > >>> ifdefs in there. Please add a pd_print_on function to support this if > >>> you truly want platform specific information. > >>> > >>> --- > >>> > >>> threadStatisticInfo.hpp > >>> > >>> typo: getElepsedTime() -> getElapsedTime() > >>> > >>> Thanks, > >>> David > >>> > >>> > >>> > >>> >> > >>> >> Re thread_dump(): I think it's correct or, well, at least it works ;-) > >>> >> In fact jstack will transfer the "-e" and "-l" in only one string, > >>> >> i.e. op->arg(0). > >>> > So you are saying that op->arg(0) is "-e -l" when using jstack? I > think > >>> > you really need to clean up the parsing. As it stands right now > passing, > >>> > I get the feeling that if a user erroneously asks for help by using > >>> > "jcmd Thread.Print -help", it will end up executing with -e > an -l > >>> > enabled, and no failure for the invalid "-help" option. > >>> >> Christoph has already explained my reasoning. But I agree, it's > not > >>> >> nice and I would be happy to do it like Christoph suggested. > >>> > I'll respond separately to his suggestion. > >>> > > >>> > thanks, > >>> > > >>> > Chris > >>> >> > >>> >> And typo fixed, sorry. > >>> >> > >>> >> Thanks again, > >>> >> Gunter > >>> >> > >>> >> On 01.06.18, 00:03, "Chris Plummer" > > >> wrote: > >>> >> > >>> >> Hi Gunter, > >>> >> globals.hpp: fix typo "informatiuon" > >>> >> I worry a little bit about the synchronizing (if that's the right > >>> >> word) > >>> >> of PrintExtendedThreadInfo and the dcmd's -e flag. When > using -e, > >>> >> you > >>> >> are temporarily enabling PrintExtendedThreadInfo if it was > false. > >>> >> This > >>> >> temporarily changes the behavior of thread dumps, and could > >>> >> impact other > >>> >> uses that happen in parallel. Also, could two simultaneous > uses > >>> >> of -e > >>> >> result in PrintExtendedThreadInfo not getting restored > properly? > >>> >> thread_dump() doesn't look right. It looks like you are > iterating > >>> >> char > >>> >> by char over the argument, and expect something like "-el" to > be > >>> >> specified rather then "-e -l". The loop should be iterating over > >>> >> op->arg(i), not op->arg(0)[i]. > >>> >> The rest of the changes look fine. > >>> >> thanks, > >>> >> Chris > >>> >> On 5/30/18 8:12 AM, Haug, Gunter wrote: > >>> >> > Hi all, > >>> >> > > >>> >> > As Chris proposed, I have made an the extended output > >>> >> switchable. There is an VM flag (PrintExtendedThreadInfo), > which is > >>> >> false by default. Moreover, there is an Option (-e) which can be > used > >>> >> with jcmd Thread.print as well as with jstack. The -e option > >>> >> essentially sets PrintExtendedThreadInfo true just for the > respective > >>> >> thread dump. > >>> >> > > >>> >> > Here is the updated webrev: > >>> >> > > >>> >> > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v2 > >>> >> > > >>> >> > (https://bugs.openjdk.java.net/browse/JDK-8200720) > >>> >> > > >>> >> > Thanks, > >>> >> > Gunter > >>> >> > > >>> >> > > >>> >> > On 02.05.18, 17:07, "serviceability-dev on behalf of Haug, > >>> >> Gunter" behalf of > >>> >> gunter.haug at sap.com> wrote: > >>> >> > > >>> >> > Hi Chris, > >>> >> > > >>> >> > Thanks for looking into this. > >>> >> > You're right, there is a little more we have. We have > >>> >> implemented an IO tracing mechanism which - rather as a > byproduct - > >>> >> keeps track of bytes read and written per thread. However, this > of > >>> >> course requires changes not only in hotspot. We would be > happy to > >>> >> contribute this as well, but this is a far bigger change and will > >>> >> probably lead to a far bigger discussion. Anyway, with the > number of > >>> >> bytes read, the number of classes defined doesn't look that > arbitrary > >>> >> anymore, as one can correlate IO to class loading. > >>> >> > > >>> >> > Regarding the verbose option I think that's a good idea! > >>> >> > > >>> >> > Thanks again, > >>> >> > Gunter > >>> >> > > >>> >> > On 01.05.18, 22:55, "Chris Plummer" > >>> >> wrote: > >>> >> > > >>> >> > Hi Gunter, > >>> >> > > >>> >> > The output you are adding is all useful. I think the > >>> >> question is (and > >>> >> > I'd like to see a few people chime in on this for this > >>> >> review) is > >>> >> > whether or not all of it is the appropriate for a > >>> >> thread's stack dump. > >>> >> > For example, defined_classes is on the fringe of what > >>> >> I would call > >>> >> > generally useful info in a stack dump. Sure, there > >>> >> might be that rare > >>> >> > case when you need it, but how often compared to > other > >>> >> useful info > >>> >> > maintained on a per thread basis. How many other bits > >>> >> of useful info are > >>> >> > not being printed in favor of defined_classes? It > >>> >> seems you have more in > >>> >> > the queue. How many? I'm worried about how > cluttered > >>> >> the stack dumps > >>> >> > will get. Maybe we should add some sort of verbose > >>> >> thread dumping > >>> >> > option. Just a thought. > >>> >> > > >>> >> > As for the implementation, overall it looks good, but > >>> >> I can't speak to > >>> >> > whether or not you are doing proper accounting of > >>> >> defined_classes and > >>> >> > bytes allocated. You'll need input from someone with > >>> >> more knowledge of > >>> >> > those areas. We'll also need to do some testing to > >>> >> make sure tool tests > >>> >> > are not impacted. > >>> >> > > >>> >> > thanks, > >>> >> > > >>> >> > Chris > >>> >> > > >>> >> > On 4/30/18 2:51 AM, Haug, Gunter wrote: > >>> >> > > Hi, > >>> >> > > > >>> >> > > this is an update to an RFR I posted on hotspot-dev, > >>> >> but it is probably more suitable to post it here. Can I please have > a > >>> >> review and a sponsor for the following enhancement: > >>> >> > > > >>> >> > > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v1 > >>> >> > > https://bugs.openjdk.java.net/browse/JDK-8200720 > >>> >> > > > >>> >> > > We at SAP have extended the thread dumps > (obtained > >>> >> by jstack or jcmd) by several fields providing thread specific > >>> >> information. These extensions are quite popular with our > support > >> team. > >>> >> With some knowledge of the architecture of the application, > they > >> often > >>> >> allow for quick and simple diagnosis of a running system. > Therefore > >> we > >>> >> would like to contribute these enhancements. > >>> >> > > > >>> >> > > I took a few simple examples here, namely cpu time, > >>> >> elapsed time since thread creation, bytes allocated and classes > >>> >> defined by the thread and the pthread-id or equivalent on > platforms > >>> >> where it makes sense. Provided it is known how the application > >> should > >>> >> behave, a misbehaving thread can identified easily. > >>> >> > > > >>> >> > > There is no measurable overhead for this > >>> >> enhancement. However, it may be a problem that the format of > the > >>> >> output is changed. Tools parsing the output may have to be > changed. > >>> >> > > > >>> >> > > Here is an example of the output generated: > >>> >> > > > >>> >> > > ------------------------------------------------------ > >>> >> > > > >>> >> > > "main" #1 prio=5 os_prio=31 cpu=6300.65ms > >>> >> elapsed=123.28s allocated=242236760B defined_classes=1725 > >>> >> tid=0x00007fa13a806000 nid=0x1c03 pthread-id=0x109708000 > waiting > >> on > >>> >> condition [0x0000000109707000] > >>> >> > > java.lang.Thread.State: TIMED_WAITING (sleeping) > >>> >> > > JavaThread state: _thread_blocked > >>> >> > > Thread: 0x00007fa13a806000 [0x1c03] State: > >>> >> _at_safepoint _has_called_back 0 _at_poll_safepoint 0 > >>> >> > > JavaThread state: _thread_blocked > >>> >> > > at java.lang.Thread.sleep(java.base/Native Method) > >>> >> > > ... > >>> >> > > ------------------------------------------------------ > >>> >> > > > >>> >> > > As mentioned above, we have a whole bunch of > other > >>> >> enhancements to the thread dump similar to this one and would > be > >>> >> willing to contribute them if there is any interest. > >>> >> > > > >>> >> > > Thanks and best regards, > >>> >> > > Gunter > >>> >> > > > >>> >> > > > >>> >> > > >>> >> > > >>> >> > > >>> >> > > >>> >> > > >>> >> > >>> > > >>> > > >>> > >>> > From robbin.ehn at oracle.com Mon Jun 11 08:07:07 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Mon, 11 Jun 2018 10:07:07 +0200 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> Message-ID: Hi Bob, On 06/07/2018 07:43 PM, Bob Vandette wrote: > Can I get one more reviewer for this RFE so I can integrate it? > >> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 Seems okay. Metrics.java "Returns the length of the operating system time slice" Note that is is only true if you are using a batch scheduler. Otherwise this period may be split on multiple 'time slices'. In printSystemMetrics there is no units, maybe intentional? Do we have support now in mach5 for docker jtreg, or do we still run these separate? You can ship it. Thanks for fixing, and super thanks for fixing the bug in PlainRead also! /Robbin > > Mandy Chung has reviewed this change. > > I?ve run Mach5 hotspot and core lib tests. > > I?ve reviewed the tests which were written by Harsha Wardhana > > I filed a CSR for the command line change and it?s now approved and closed. > > Thanks, > Bob. > > >> On May 30, 2018, at 3:45 PM, Bob Vandette wrote: >> >> Please review the following RFE which adds an internal API, along with jtreg tests that provide >> access to Docker container configuration data and metrics. In addition to the API which we hope to >> take advantage of in the future with Java Flight Recorder and a JMX Mbean, I?ve added an additional >> option to -XshowSettings:system than dumps out the container or host cgroup confguration >> information. See the sample output below: >> >> RFE: Container Metrics >> >> https://bugs.openjdk.java.net/browse/JDK-8203357 >> >> WEBREV: >> >> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >> >> >> This commit will also include a fix for the following bug. >> >> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >> >> https://bugs.openjdk.java.net/browse/JDK-8203691 >> >> WEBREV: >> >> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >> >> SAMPLE USAGE and OUTPUT: >> >> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >> ./java -XshowSettings:system >> Operating System Metrics: >> Provider: cgroupv1 >> Effective CPU Count: 4 >> CPU Period: 100000 >> CPU Quota: -1 >> CPU Shares: -1 >> List of Processors, 4 total: >> 4 5 6 7 >> List of Effective Processors, 4 total: >> 4 5 6 7 >> List of Memory Nodes, 2 total: >> 0 1 >> List of Available Memory Nodes, 2 total: >> 0 1 >> CPUSet Memory Pressure Enabled: false >> Memory Limit: 256.00M >> Memory Soft Limit: Unlimited >> Memory & Swap Limit: 512.00M >> Kernel Memory Limit: Unlimited >> TCP Memory Limit: Unlimited >> Out Of Memory Killer Enabled: true >> >> TEST RESULTS: >> >> testing runtime container APIs >> Directory "JTwork" not found: creating >> Passed: runtime/containers/cgroup/PlainRead.java >> Passed: runtime/containers/docker/DockerBasicTest.java >> Passed: runtime/containers/docker/TestCPUAwareness.java >> Passed: runtime/containers/docker/TestCPUSets.java >> Passed: runtime/containers/docker/TestMemoryAwareness.java >> Passed: runtime/containers/docker/TestMisc.java >> Test results: passed: 6 >> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >> >> testing jdk.internal.platform APIs >> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >> Test results: passed: 4 >> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >> >> testing -XshowSettings:system launcher option >> Passed: tools/launcher/Settings.java >> Test results: passed: 1 >> >> >> Bob. >> >> > From david.holmes at oracle.com Mon Jun 11 08:32:00 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 11 Jun 2018 18:32:00 +1000 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> Message-ID: <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> Sorry Bob I haven't had a chance to look at this detail. For the Java code ... methods that return arrays should return zero-length arrays when something is not available rather than null. For getCpuPeriod() the term "operating system time slice" can be misconstrued as being related to the scheduler timeslice that may, or may not, exist, depending on the scheduler and scheduling policy etc. This "timeslice" is something specific to cgroups - no? David On 8/06/2018 3:43 AM, Bob Vandette wrote: > Can I get one more reviewer for this RFE so I can integrate it? > >> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 > > Mandy Chung has reviewed this change. > > I?ve run Mach5 hotspot and core lib tests. > > I?ve reviewed the tests which were written by Harsha Wardhana > > I filed a CSR for the command line change and it?s now approved and closed. > > Thanks, > Bob. > > >> On May 30, 2018, at 3:45 PM, Bob Vandette wrote: >> >> Please review the following RFE which adds an internal API, along with jtreg tests that provide >> access to Docker container configuration data and metrics. In addition to the API which we hope to >> take advantage of in the future with Java Flight Recorder and a JMX Mbean, I?ve added an additional >> option to -XshowSettings:system than dumps out the container or host cgroup confguration >> information. See the sample output below: >> >> RFE: Container Metrics >> >> https://bugs.openjdk.java.net/browse/JDK-8203357 >> >> WEBREV: >> >> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >> >> >> This commit will also include a fix for the following bug. >> >> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >> >> https://bugs.openjdk.java.net/browse/JDK-8203691 >> >> WEBREV: >> >> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >> >> SAMPLE USAGE and OUTPUT: >> >> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >> ./java -XshowSettings:system >> Operating System Metrics: >> Provider: cgroupv1 >> Effective CPU Count: 4 >> CPU Period: 100000 >> CPU Quota: -1 >> CPU Shares: -1 >> List of Processors, 4 total: >> 4 5 6 7 >> List of Effective Processors, 4 total: >> 4 5 6 7 >> List of Memory Nodes, 2 total: >> 0 1 >> List of Available Memory Nodes, 2 total: >> 0 1 >> CPUSet Memory Pressure Enabled: false >> Memory Limit: 256.00M >> Memory Soft Limit: Unlimited >> Memory & Swap Limit: 512.00M >> Kernel Memory Limit: Unlimited >> TCP Memory Limit: Unlimited >> Out Of Memory Killer Enabled: true >> >> TEST RESULTS: >> >> testing runtime container APIs >> Directory "JTwork" not found: creating >> Passed: runtime/containers/cgroup/PlainRead.java >> Passed: runtime/containers/docker/DockerBasicTest.java >> Passed: runtime/containers/docker/TestCPUAwareness.java >> Passed: runtime/containers/docker/TestCPUSets.java >> Passed: runtime/containers/docker/TestMemoryAwareness.java >> Passed: runtime/containers/docker/TestMisc.java >> Test results: passed: 6 >> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >> >> testing jdk.internal.platform APIs >> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >> Test results: passed: 4 >> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >> >> testing -XshowSettings:system launcher option >> Passed: tools/launcher/Settings.java >> Test results: passed: 1 >> >> >> Bob. >> >> > From goetz.lindenmaier at sap.com Mon Jun 11 13:12:03 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 11 Jun 2018 13:12:03 +0000 Subject: RFR(S): Message-ID: <3a081a228401448eb4d862773740d5bd@sap.com> Hi, please review this test fix: http://cr.openjdk.java.net/~goetz/wr18/8204654-fixJstatTest/01 gcCauseOutput1.awk: The pattern scans 11 numbers, while the output contains 13. Also, more '-' are possible then checked. The other awk scripts need to check more patterns where '-' can appear. We have a machine with user.country=de where jstat prints ',' instead of '.' in numbers. Explicitly start with user.country=en as already done for user.language=en. I also refactored the common flags to a variable in utils.sh ... so there won't be the need to edit all these files once more :) Best regards, Goetz. From peter.gergely.horvath at gmail.com Mon Jun 11 13:14:04 2018 From: peter.gergely.horvath at gmail.com (=?UTF-8?Q?P=C3=A9ter_Gergely_Horv=C3=A1th?=) Date: Mon, 11 Jun 2018 15:14:04 +0200 Subject: Kerberos authentication for JMX? Message-ID: Hi All, I have been working with Big Data for a while and I have seen that a number of the components have started to have their own custom baked solutions (minimalistic Web UIs) for basic management operations, like showing metrics, debugging etc instead of using JMX. I have the feeling that getting JMX working for dozens of different Java services within a large cluster is an overly tough task, especially if you do not want to make compromises around security. For me it seems, that at the moment there is a gap between what the JDK offers regarding JMX monitoring/management and what people would need in a real world setting to use it effectively in an easy and secure way. I am wondering if it would be possible to implement a Kerberos-based authentication mechanism for JMX, allowing all services of a cluster to authenticate JMX clients against a centrally managed Kerberos service, that would also be officially supported by VisualVM so as to give an easy-to-use user interface. Based on my understanding, this could either be a new protocol implementation or assuming JDK-8171311: REST APIs for JMX gets done, an additional feature around there to support GSS Negotiate/SPNEGO based authentication. Could you please share your thoughts on this? Would anyone be interested to sponsor this topic? Thanks, Peter -------------- next part -------------- An HTML attachment was scrubbed... URL: From goetz.lindenmaier at sap.com Mon Jun 11 13:14:37 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 11 Jun 2018 13:14:37 +0000 Subject: RFR(S): 8204654: [testbug] Fix pattern matching in jstat tests. Message-ID: <18662fc429004f34bae0343b03dcd28e@sap.com> Hi, please review this test fix: http://cr.openjdk.java.net/~goetz/wr18/8204654-fixJstatTest/01 gcCauseOutput1.awk: The pattern scans 11 numbers, while the output contains 13. Also, more '-' are possible then checked. The other awk scripts need to check more patterns where '-' can appear. We have a machine with user.country=de where jstat prints ',' instead of '.' in numbers. Explicitly start with user.country=en as already done for user.language=en. I also refactored the common flags to a variable in utils.sh ... so there won't be the need to edit all these files once more :) Best regards, Goetz (Sorry for the second mail, first missed the bug text ...) From bob.vandette at oracle.com Mon Jun 11 14:12:29 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Mon, 11 Jun 2018 10:12:29 -0400 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> Message-ID: > On Jun 11, 2018, at 4:32 AM, David Holmes wrote: > > Sorry Bob I haven't had a chance to look at this detail. > > For the Java code ... methods that return arrays should return zero-length arrays when something is not available rather than null. All methods do return zero length arrays except I missed the getPerCpuUsage. I?ll fix that one and correct the javadoc. > > For getCpuPeriod() the term "operating system time slice" can be misconstrued as being related to the scheduler timeslice that may, or may not, exist, depending on the scheduler and scheduling policy etc. This "timeslice" is something specific to cgroups - no? The comments reads: * Returns the length of the operating system time slice, in * milliseconds, for processes within the Isolation Group. The comment does infer that it?s process and cgroup (Isolation group) specific and not the generic os timeslice. Isn?t this sufficient? Thanks, Bob. > > David > > On 8/06/2018 3:43 AM, Bob Vandette wrote: >> Can I get one more reviewer for this RFE so I can integrate it? >>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >> Mandy Chung has reviewed this change. >> I?ve run Mach5 hotspot and core lib tests. >> I?ve reviewed the tests which were written by Harsha Wardhana >> I filed a CSR for the command line change and it?s now approved and closed. >> Thanks, >> Bob. >>> On May 30, 2018, at 3:45 PM, Bob Vandette wrote: >>> >>> Please review the following RFE which adds an internal API, along with jtreg tests that provide >>> access to Docker container configuration data and metrics. In addition to the API which we hope to >>> take advantage of in the future with Java Flight Recorder and a JMX Mbean, I?ve added an additional >>> option to -XshowSettings:system than dumps out the container or host cgroup confguration >>> information. See the sample output below: >>> >>> RFE: Container Metrics >>> >>> https://bugs.openjdk.java.net/browse/JDK-8203357 >>> >>> WEBREV: >>> >>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>> >>> >>> This commit will also include a fix for the following bug. >>> >>> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >>> >>> https://bugs.openjdk.java.net/browse/JDK-8203691 >>> >>> WEBREV: >>> >>> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >>> >>> SAMPLE USAGE and OUTPUT: >>> >>> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >>> ./java -XshowSettings:system >>> Operating System Metrics: >>> Provider: cgroupv1 >>> Effective CPU Count: 4 >>> CPU Period: 100000 >>> CPU Quota: -1 >>> CPU Shares: -1 >>> List of Processors, 4 total: >>> 4 5 6 7 >>> List of Effective Processors, 4 total: >>> 4 5 6 7 >>> List of Memory Nodes, 2 total: >>> 0 1 >>> List of Available Memory Nodes, 2 total: >>> 0 1 >>> CPUSet Memory Pressure Enabled: false >>> Memory Limit: 256.00M >>> Memory Soft Limit: Unlimited >>> Memory & Swap Limit: 512.00M >>> Kernel Memory Limit: Unlimited >>> TCP Memory Limit: Unlimited >>> Out Of Memory Killer Enabled: true >>> >>> TEST RESULTS: >>> >>> testing runtime container APIs >>> Directory "JTwork" not found: creating >>> Passed: runtime/containers/cgroup/PlainRead.java >>> Passed: runtime/containers/docker/DockerBasicTest.java >>> Passed: runtime/containers/docker/TestCPUAwareness.java >>> Passed: runtime/containers/docker/TestCPUSets.java >>> Passed: runtime/containers/docker/TestMemoryAwareness.java >>> Passed: runtime/containers/docker/TestMisc.java >>> Test results: passed: 6 >>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>> >>> testing jdk.internal.platform APIs >>> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >>> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >>> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >>> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >>> Test results: passed: 4 >>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>> >>> testing -XshowSettings:system launcher option >>> Passed: tools/launcher/Settings.java >>> Test results: passed: 1 >>> >>> >>> Bob. >>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger.riggs at oracle.com Mon Jun 11 15:00:46 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Mon, 11 Jun 2018 11:00:46 -0400 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> Message-ID: Hi, Just a thought on the timeslice units,? would nanoseconds be more future proof? In a variety of cases, milliseconds are starting to seem too coarse. $.02, Roger On 6/11/18 10:12 AM, Bob Vandette wrote: > >> On Jun 11, 2018, at 4:32 AM, David Holmes > > wrote: >> >> Sorry Bob I haven't had a chance to look at this detail. >> >> For the Java code ... methods that return arrays should return >> zero-length arrays when something is not available rather than null. > > All methods do return zero length arrays except I missed the > getPerCpuUsage. ?I?ll fix that one and > correct the javadoc. > >> >> For getCpuPeriod() the term "operating system time slice" can be >> misconstrued as being related to the scheduler timeslice that may, or >> may not, exist, depending on the scheduler and scheduling policy etc. >> This "timeslice" is something specific to cgroups - no? > > The comments reads: > > * Returns the length of the operating system time slice, in > * milliseconds, for processes within the Isolation Group. > The comment does infer that it?s process and cgroup (Isolation group) > specific and not the generic os timeslice. > Isn?t this sufficient? > > Thanks, > Bob. >> >> David >> >> On 8/06/2018 3:43 AM, Bob Vandette wrote: >>> Can I get one more reviewer for this RFE so I can integrate it? >>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>>> >>> Mandy Chung has reviewed this change. >>> I?ve run Mach5 hotspot and core lib tests. >>> I?ve reviewed the tests which were written by Harsha Wardhana >>> I filed a CSR for the command line change and it?s now approved and >>> closed. >>> Thanks, >>> Bob. >>>> On May 30, 2018, at 3:45 PM, Bob Vandette >>> > wrote: >>>> >>>> Please review the following RFE which adds an internal API, along >>>> with jtreg tests that provide >>>> access to Docker container configuration data and metrics. ?In >>>> addition to the API which we hope to >>>> take advantage of in the future with Java Flight Recorder and a JMX >>>> Mbean, I?ve added an additional >>>> option to -XshowSettings:system than dumps out the container or >>>> host cgroup confguration >>>> information. ?See the sample output below: >>>> >>>> RFE: Container Metrics >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8203357 >>>> >>>> WEBREV: >>>> >>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>>> >>>> >>>> This commit will also include a fix for the following bug. >>>> >>>> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8203691 >>>> >>>> WEBREV: >>>> >>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >>>> >>>> SAMPLE USAGE and OUTPUT: >>>> >>>> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >>>> ./java -XshowSettings:system >>>> Operating System Metrics: >>>> ???Provider: cgroupv1 >>>> ???Effective CPU Count: 4 >>>> ???CPU Period: 100000 >>>> ???CPU Quota: -1 >>>> ???CPU Shares: -1 >>>> ???List of Processors, 4 total: >>>> ???4 5 6 7 >>>> ???List of Effective Processors, 4 total: >>>> ???4 5 6 7 >>>> ???List of Memory Nodes, 2 total: >>>> ???0 1 >>>> ???List of Available Memory Nodes, 2 total: >>>> ???0 1 >>>> ???CPUSet Memory Pressure Enabled: false >>>> ???Memory Limit: 256.00M >>>> ???Memory Soft Limit: Unlimited >>>> ???Memory & Swap Limit: 512.00M >>>> ???Kernel Memory Limit: Unlimited >>>> ???TCP Memory Limit: Unlimited >>>> ???Out Of Memory Killer Enabled: true >>>> >>>> TEST RESULTS: >>>> >>>> testing runtime container APIs >>>> Directory "JTwork" not found: creating >>>> Passed: runtime/containers/cgroup/PlainRead.java >>>> Passed: runtime/containers/docker/DockerBasicTest.java >>>> Passed: runtime/containers/docker/TestCPUAwareness.java >>>> Passed: runtime/containers/docker/TestCPUSets.java >>>> Passed: runtime/containers/docker/TestMemoryAwareness.java >>>> Passed: runtime/containers/docker/TestMisc.java >>>> Test results: passed: 6 >>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>> >>>> testing jdk.internal.platform APIs >>>> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >>>> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >>>> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >>>> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >>>> Test results: passed: 4 >>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>> >>>> testing -XshowSettings:system launcher option >>>> Passed: tools/launcher/Settings.java >>>> Test results: passed: 1 >>>> >>>> >>>> Bob. >>>> >>>> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob.vandette at oracle.com Mon Jun 11 18:28:05 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Mon, 11 Jun 2018 14:28:05 -0400 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> Message-ID: <54435D5C-5E6B-4174-8344-984CDDF5D46A@oracle.com> > On Jun 11, 2018, at 4:07 AM, Robbin Ehn wrote: > > Hi Bob, > > On 06/07/2018 07:43 PM, Bob Vandette wrote: >> Can I get one more reviewer for this RFE so I can integrate it? >>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 > > Seems okay. > > Metrics.java > "Returns the length of the operating system time slice" > > Note that is is only true if you are using a batch scheduler. > Otherwise this period may be split on multiple 'time slices?. This is a cgroup metric which uses CFS not the OS time slice. 136 /** 137 * Returns the length of the operating system time slice, in 138 * milliseconds, for processes within the Isolation Group. > > In printSystemMetrics there is no units, maybe intentional? I?ll add ms for the quote/period output. The memory metrics do have units. > > Do we have support now in mach5 for docker jtreg, or do we still run these separate? > > You can ship it. Thanks! Bob. > > Thanks for fixing, and super thanks for fixing the bug in PlainRead also! > > /Robbin > >> Mandy Chung has reviewed this change. >> I?ve run Mach5 hotspot and core lib tests. >> I?ve reviewed the tests which were written by Harsha Wardhana >> I filed a CSR for the command line change and it?s now approved and closed. >> Thanks, >> Bob. >>> On May 30, 2018, at 3:45 PM, Bob Vandette wrote: >>> >>> Please review the following RFE which adds an internal API, along with jtreg tests that provide >>> access to Docker container configuration data and metrics. In addition to the API which we hope to >>> take advantage of in the future with Java Flight Recorder and a JMX Mbean, I?ve added an additional >>> option to -XshowSettings:system than dumps out the container or host cgroup confguration >>> information. See the sample output below: >>> >>> RFE: Container Metrics >>> >>> https://bugs.openjdk.java.net/browse/JDK-8203357 >>> >>> WEBREV: >>> >>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>> >>> >>> This commit will also include a fix for the following bug. >>> >>> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >>> >>> https://bugs.openjdk.java.net/browse/JDK-8203691 >>> >>> WEBREV: >>> >>> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >>> >>> SAMPLE USAGE and OUTPUT: >>> >>> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >>> ./java -XshowSettings:system >>> Operating System Metrics: >>> Provider: cgroupv1 >>> Effective CPU Count: 4 >>> CPU Period: 100000 >>> CPU Quota: -1 >>> CPU Shares: -1 >>> List of Processors, 4 total: >>> 4 5 6 7 >>> List of Effective Processors, 4 total: >>> 4 5 6 7 >>> List of Memory Nodes, 2 total: >>> 0 1 >>> List of Available Memory Nodes, 2 total: >>> 0 1 >>> CPUSet Memory Pressure Enabled: false >>> Memory Limit: 256.00M >>> Memory Soft Limit: Unlimited >>> Memory & Swap Limit: 512.00M >>> Kernel Memory Limit: Unlimited >>> TCP Memory Limit: Unlimited >>> Out Of Memory Killer Enabled: true >>> >>> TEST RESULTS: >>> >>> testing runtime container APIs >>> Directory "JTwork" not found: creating >>> Passed: runtime/containers/cgroup/PlainRead.java >>> Passed: runtime/containers/docker/DockerBasicTest.java >>> Passed: runtime/containers/docker/TestCPUAwareness.java >>> Passed: runtime/containers/docker/TestCPUSets.java >>> Passed: runtime/containers/docker/TestMemoryAwareness.java >>> Passed: runtime/containers/docker/TestMisc.java >>> Test results: passed: 6 >>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>> >>> testing jdk.internal.platform APIs >>> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >>> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >>> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >>> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >>> Test results: passed: 4 >>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>> >>> testing -XshowSettings:system launcher option >>> Passed: tools/launcher/Settings.java >>> Test results: passed: 1 >>> >>> >>> Bob. >>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Mon Jun 11 19:14:33 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 11 Jun 2018 12:14:33 -0700 Subject: RFR(S): 8200720: Print additional information in thread dump (times, allocated bytes etc.) In-Reply-To: References: <56817AB5-88FC-4FB9-A271-7206EA69ECA7@sap.com> <2728581E-0E32-4911-B1A9-79AD6CBA81B0@sap.com> <082ca98f-9a68-15a0-95b5-c4d5c733ab36@oracle.com> <933cdacc-9b6f-c8b5-e244-558f38e7888c@oracle.com> <55806aa8-f09b-e9a8-b512-b9676029c494@oracle.com> <0F497BC6-2569-4EF6-9B59-E24A8E1B3A37@sap.com> <5dc08f65e83f4374b8e1f73f218c6d6d@sap.com> <626701f8-b71b-58dc-f837-e5e1d81ef187@oracle.com> Message-ID: Hi Gunter, The changes look fine. I can live with the options parsing in attachListener.cpp. As you point out, unrecognized options were already silently ignored. thanks, Chris On 6/8/18 7:05 AM, Haug, Gunter wrote: > Hi all, > > thanks a lot for all the input! I have prepared a new version of the webrev incorporating the suggestions you made (at least I tried): > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v3/ > > This version outputs the thread times unconditionally while the other information is guarded by a flags. I think, most participants found the thread times the most valuable information and had no (strong) objections to adding them unconditionally. > > @David > Implementation is much simpler with conditional output only for JavaThreads. I could get rid of the terrible hack (changing the flag) without having to change too many source files. > > The information on allocated bytes is present in the Thread class already before this proposed change, I just print it. It might be sensible to move _allocated_bytes and the respective methods to the ThreadStatisticInfo class as G?tz suggested. I haven?t done that in the current version, though. > > @David and Thomas > I've removed the pthread-id output. I'm unsure myself what it could be good for. Anyway, we could add it with a separate change (with a better implementation) if there is a need to. > > @Chris > As you have written, the user has no direct contact to the attach listener of the VM and jstack won't misinterpret e.g. -help. I agree that the parsing in attachListener.cpp could be more robust. However, it hasn't been so far either. All that is done in the current implementation is a strcmp to -l everything else will be silently ignored. I can try to make this more robust or we could say that the new output is only available via jcmd. OTH nothing bad can happen with the current version of the proposed change, so we could also leave it as it is. What would you prefer? > > @Kirk and Thomas > Implementing a new diagnostic command is certainly well worth a thought. However, it appears to me that it's not necessary in this case. There are already flags to jstack and jcmd Thread.print, so it's not that uncommon. The amount of information that would be available by a new diagnostic command and that is now added to the thread dump is quite small and I think that is tolerable. Moreover, and most importantly, our support team is use to find the information in the thread dump and it would make things easier to us if I could leave it there. > > Thanks again and have a nice weekend, > Gunter > > > ?On 06.06.18, 04:56, "David Holmes" wrote: > > Hi Goetz, > > On 5/06/2018 11:07 PM, Lindenmaier, Goetz wrote: > > Hi > > > > this discussion touched a lot of points so far, which seem > > to lead to different conclusions. > > > > I think we should look at the values printed: > > > > 1. cpu=6300.65ms elapsed=123.28s > > Overhead > > cpu time: > > * system calls at thread dump time > > elapsed time: > > * 1 system call at thread creation time > > * 1 64-bit field per thread for the thread start time > > * 1 system call at thread dump time > > > > As I understand, JDK-8143176 would have had to get the > > time at each locking, which is much more time critical > > than doing this during thread creation. While > > Correct. > > > the time a lock was held would be much more useful, > > the ratio here, combined with knowledge about the application, > > could lead to the conclusion that the thread is wrongly > > blocked at much lower cost. > > Agreed. I see no issue with unconditionally adding this information as > it should address some of the concerns of 8143176 as well. > > > > > 2. pthread-id=0x109708000 > > Overhead: > > * a field access at thread dump time. Negligible I would say. > > I'd overlooked this addition, but it is of course the chunk of ifdef > code in osThread.cpp that I objected to. The problem I have here is the > convoluted mess of "thread identifiers" that we already have. We > currently print: > > st->print("tid=" INTPTR_FORMAT " ", p2i(this)); > > which is just the address of the Thread/JavaThread object. Then: > > st->print("nid=0x%x ", thread_id()); > > where 'n' is supposed to represent "native thread id", which is: > - linux: kernel thread id from syscall gettid > - solaris: thread library identity from thr_create() or thr_self() > - windows: thread id from _beginthreadex > - OS X: kernel thread id from pthread_mach_thread_np(pthread_self()); > - Other BSD: kernel (?) thread id from syscall thr_self or getthrid > - AIX: thread library identity from pthread_create() or pthread_self() > > It's telling that the code Gunter added gets the thread library id on > linux, but a "kernel thread id" on other platforms! Do we want to see a > thread library id and a kernel thread id? IIRC nid is supposed be the id > you need to see the thread in a debugger. In which case I'm unclear what > role the thread id Gunter wants to add is playing? I don't think we need > to see kernel ids in general, and pthread-id isn't useful for debugging > is it? > > > > > 3. allocated=242236760B defined_classes=1725 > > Overhead: > > * 1 64-bit field per thread. > > * counter increment on class creation > > > > Especially defined_classes seems to be controversial. > > As this only makes sense for Java threads, this could > > be printed in a line of it's own in Threads::print_on_error() > > (which already gets a flag to customize for jstack: > > print_concurrent_locks) by calling a dedicated function in Thread. > > No need for flag PrintExtendedThreadInfo thus. > > > > But I would propose to drop this information as it is > > too controversial. > > Agreed. > > > This leaves the times and the pthread-id to be decided whether > > they are worth the cost. If so, I think they should be printed > > unconditional. > > Agreed. With some refinement of the "thread id" issue. > > > If a flag is required to avoid bloating the info in the default case, > > I would include more infos, as os_prio and nid under that flag. > > Of course those things exist for good reason. The native id is supposed > to be what aids in you matching what you see in our thread dumps with > what you see in a debugger. Priority is less of an issue these days, but > there was a time ... ;-) > > > As jstack is deprecated, changes to jstack could be skipped always > > disabling the new printouts. > > > > Finally, I would propose to move _allocated_bytes into > > threadStatisticInfo.hpp. > > Didn't that part get dropped? (You don't need it for the time info or > thread id). That's a GC logging issue to me. > > Thanks, > David > > > Best regards, > > Goetz. > > > > > > > > > >> -----Original Message----- > >> From: serviceability-dev [mailto:serviceability-dev- > >> bounces at openjdk.java.net] On Behalf Of David Holmes > >> Sent: Dienstag, 5. Juni 2018 04:53 > >> To: Haug, Gunter ; Chris Plummer > >> ; serviceability-dev >> dev at openjdk.java.net>; Langer, Christoph ; > >> hotspot-runtime-dev at openjdk.java.net > >> Subject: Re: RFR(S): 8200720: Print additional information in thread dump > >> (times, allocated bytes etc.) > >> > >> Hi Gunter, > >> > >> On 5/06/2018 1:27 AM, Haug, Gunter wrote: > >>> Hi David, Chris, > >>> > >>> would it be an option to unconditionally print the additional information? > >> Regardless which way this is implemented it will be rather complicated and it > >> only switches on/off a few field in the thread dump. > >> > >> I'm not convinced this is all suitable information for a thread stack > >> dump. I can see the time information being useful if using the dump to > >> try and diagnoze a hang or responsiveness issue. But the allocated-bytes > >> and classes-defined just doesn't seem useful in the context of a thread > >> dump to me. Anyone interested in allocation stats is going to be looking > >> at GC logs etc which is where this belongs. Ditto the class-stats belong > >> in some kind of classloading logging IMHO. > >> > >> I'm very reluctant to burden the implementation with capturing these > >> kinds of stats, just to use for diagnostic purposes that may not even be > >> asked for. I'm very much in the "you shouldn't pay for what you don't > >> use" camp in that regard. (See my comments in JDK-8143176 referenced by > >> Sean.) > >> > >> The ThreadStatisticInfo adds overhead to every thread object - and I'd > >> have to suspect there could be overlap with whatever flight recorder > >> sticks in there too. (Thread has become even more bloated in recent time!). > >> > >> Cheers, > >> David > >> > >> > >>> Thanks, > >>> Gunter > >>> > >>> On 04.06.18, 01:13, "David Holmes" wrote: > >>> > >>> Hi Gunter, Chris, > >>> > >>> Sorry just trying to catch up and this is only a partial look so far ... > >>> > >>> BTW these changes are not limited to serviceability code so adding in > >>> runtime team as well. > >>> > >>> On 2/06/2018 9:12 AM, Chris Plummer wrote: > >>> > Hi Gunter, > >>> > > >>> > On 6/1/18 3:17 AM, Haug, Gunter wrote: > >>> >> Hi Chris, > >>> >> > >>> >> thanks for looking into this! > >>> >> > >>> >> Re the synchronization: The value is stored only in a VM operation at > >>> >> a safepoint by the VM thread. I was of the opinion, that this could > >>> >> not be interrupted by a second VM operation (of the same type). Or > >> am > >>> >> I missing something else? > >>> > I was really thinking more about the temporary changing of > >>> > PrintExtendedThreadInfo, not the value stored in the VMOp. You may > >> be be > >>> > correct that no more than one VMOp is executing, but while it is > >>> > executing it is has changed the value of PrintExtendedThreadInfo, > >> which > >>> > might have an impact on anything else that triggers printing thread info > >>> > while the VMOp is executing. > >>> > >>> Even if nothing else can possibly be running during the safepoint I find > >>> it extremely bad form to change a command-line flag in this way, > >>> particularly from a safepoint! > >>> > >>> If this flag is intended to be dynamically enabled as part of a dcmd > >>> then it should also be a Manageable flag IMHO. > >>> > >>> That said ... > >>> > >>> >> I did think about passing an argument to the various print_on > >> member > >>> >> functions of the thread classes, but this would require rather > >>> >> extensive code changes for a rather tiny extension. Therefore I feel > >>> >> doing it like this is the lesser evil. > >>> > >>> ... it's obviously not truly a global setting, but one that is needed on > >>> a per-print-request basis. The flag is just the default, but if the > >>> default is off you still want to enable extended printing on a > >>> per-request basis. > >>> > >>> The current print_on mechanics is not set up for this kind of > >>> flexibility. I think this needs more thought. > >>> > >>> --- > >>> > >>> Re osThread.cpp shared code changes ... I really hate to see all the > >>> ifdefs in there. Please add a pd_print_on function to support this if > >>> you truly want platform specific information. > >>> > >>> --- > >>> > >>> threadStatisticInfo.hpp > >>> > >>> typo: getElepsedTime() -> getElapsedTime() > >>> > >>> Thanks, > >>> David > >>> > >>> > >>> > >>> >> > >>> >> Re thread_dump(): I think it's correct or, well, at least it works ;-) > >>> >> In fact jstack will transfer the "-e" and "-l" in only one string, > >>> >> i.e. op->arg(0). > >>> > So you are saying that op->arg(0) is "-e -l" when using jstack? I think > >>> > you really need to clean up the parsing. As it stands right now passing, > >>> > I get the feeling that if a user erroneously asks for help by using > >>> > "jcmd Thread.Print -help", it will end up executing with -e an -l > >>> > enabled, and no failure for the invalid "-help" option. > >>> >> Christoph has already explained my reasoning. But I agree, it's not > >>> >> nice and I would be happy to do it like Christoph suggested. > >>> > I'll respond separately to his suggestion. > >>> > > >>> > thanks, > >>> > > >>> > Chris > >>> >> > >>> >> And typo fixed, sorry. > >>> >> > >>> >> Thanks again, > >>> >> Gunter > >>> >> > >>> >> On 01.06.18, 00:03, "Chris Plummer" > >> wrote: > >>> >> > >>> >> Hi Gunter, > >>> >> globals.hpp: fix typo "informatiuon" > >>> >> I worry a little bit about the synchronizing (if that's the right > >>> >> word) > >>> >> of PrintExtendedThreadInfo and the dcmd's -e flag. When using -e, > >>> >> you > >>> >> are temporarily enabling PrintExtendedThreadInfo if it was false. > >>> >> This > >>> >> temporarily changes the behavior of thread dumps, and could > >>> >> impact other > >>> >> uses that happen in parallel. Also, could two simultaneous uses > >>> >> of -e > >>> >> result in PrintExtendedThreadInfo not getting restored properly? > >>> >> thread_dump() doesn't look right. It looks like you are iterating > >>> >> char > >>> >> by char over the argument, and expect something like "-el" to be > >>> >> specified rather then "-e -l". The loop should be iterating over > >>> >> op->arg(i), not op->arg(0)[i]. > >>> >> The rest of the changes look fine. > >>> >> thanks, > >>> >> Chris > >>> >> On 5/30/18 8:12 AM, Haug, Gunter wrote: > >>> >> > Hi all, > >>> >> > > >>> >> > As Chris proposed, I have made an the extended output > >>> >> switchable. There is an VM flag (PrintExtendedThreadInfo), which is > >>> >> false by default. Moreover, there is an Option (-e) which can be used > >>> >> with jcmd Thread.print as well as with jstack. The -e option > >>> >> essentially sets PrintExtendedThreadInfo true just for the respective > >>> >> thread dump. > >>> >> > > >>> >> > Here is the updated webrev: > >>> >> > > >>> >> > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v2 > >>> >> > > >>> >> > (https://bugs.openjdk.java.net/browse/JDK-8200720) > >>> >> > > >>> >> > Thanks, > >>> >> > Gunter > >>> >> > > >>> >> > > >>> >> > On 02.05.18, 17:07, "serviceability-dev on behalf of Haug, > >>> >> Gunter" >>> >> gunter.haug at sap.com> wrote: > >>> >> > > >>> >> > Hi Chris, > >>> >> > > >>> >> > Thanks for looking into this. > >>> >> > You're right, there is a little more we have. We have > >>> >> implemented an IO tracing mechanism which - rather as a byproduct - > >>> >> keeps track of bytes read and written per thread. However, this of > >>> >> course requires changes not only in hotspot. We would be happy to > >>> >> contribute this as well, but this is a far bigger change and will > >>> >> probably lead to a far bigger discussion. Anyway, with the number of > >>> >> bytes read, the number of classes defined doesn't look that arbitrary > >>> >> anymore, as one can correlate IO to class loading. > >>> >> > > >>> >> > Regarding the verbose option I think that's a good idea! > >>> >> > > >>> >> > Thanks again, > >>> >> > Gunter > >>> >> > > >>> >> > On 01.05.18, 22:55, "Chris Plummer" > >>> >> wrote: > >>> >> > > >>> >> > Hi Gunter, > >>> >> > > >>> >> > The output you are adding is all useful. I think the > >>> >> question is (and > >>> >> > I'd like to see a few people chime in on this for this > >>> >> review) is > >>> >> > whether or not all of it is the appropriate for a > >>> >> thread's stack dump. > >>> >> > For example, defined_classes is on the fringe of what > >>> >> I would call > >>> >> > generally useful info in a stack dump. Sure, there > >>> >> might be that rare > >>> >> > case when you need it, but how often compared to other > >>> >> useful info > >>> >> > maintained on a per thread basis. How many other bits > >>> >> of useful info are > >>> >> > not being printed in favor of defined_classes? It > >>> >> seems you have more in > >>> >> > the queue. How many? I'm worried about how cluttered > >>> >> the stack dumps > >>> >> > will get. Maybe we should add some sort of verbose > >>> >> thread dumping > >>> >> > option. Just a thought. > >>> >> > > >>> >> > As for the implementation, overall it looks good, but > >>> >> I can't speak to > >>> >> > whether or not you are doing proper accounting of > >>> >> defined_classes and > >>> >> > bytes allocated. You'll need input from someone with > >>> >> more knowledge of > >>> >> > those areas. We'll also need to do some testing to > >>> >> make sure tool tests > >>> >> > are not impacted. > >>> >> > > >>> >> > thanks, > >>> >> > > >>> >> > Chris > >>> >> > > >>> >> > On 4/30/18 2:51 AM, Haug, Gunter wrote: > >>> >> > > Hi, > >>> >> > > > >>> >> > > this is an update to an RFR I posted on hotspot-dev, > >>> >> but it is probably more suitable to post it here. Can I please have a > >>> >> review and a sponsor for the following enhancement: > >>> >> > > > >>> >> > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v1 > >>> >> > > https://bugs.openjdk.java.net/browse/JDK-8200720 > >>> >> > > > >>> >> > > We at SAP have extended the thread dumps (obtained > >>> >> by jstack or jcmd) by several fields providing thread specific > >>> >> information. These extensions are quite popular with our support > >> team. > >>> >> With some knowledge of the architecture of the application, they > >> often > >>> >> allow for quick and simple diagnosis of a running system. Therefore > >> we > >>> >> would like to contribute these enhancements. > >>> >> > > > >>> >> > > I took a few simple examples here, namely cpu time, > >>> >> elapsed time since thread creation, bytes allocated and classes > >>> >> defined by the thread and the pthread-id or equivalent on platforms > >>> >> where it makes sense. Provided it is known how the application > >> should > >>> >> behave, a misbehaving thread can identified easily. > >>> >> > > > >>> >> > > There is no measurable overhead for this > >>> >> enhancement. However, it may be a problem that the format of the > >>> >> output is changed. Tools parsing the output may have to be changed. > >>> >> > > > >>> >> > > Here is an example of the output generated: > >>> >> > > > >>> >> > > ------------------------------------------------------ > >>> >> > > > >>> >> > > "main" #1 prio=5 os_prio=31 cpu=6300.65ms > >>> >> elapsed=123.28s allocated=242236760B defined_classes=1725 > >>> >> tid=0x00007fa13a806000 nid=0x1c03 pthread-id=0x109708000 waiting > >> on > >>> >> condition [0x0000000109707000] > >>> >> > > java.lang.Thread.State: TIMED_WAITING (sleeping) > >>> >> > > JavaThread state: _thread_blocked > >>> >> > > Thread: 0x00007fa13a806000 [0x1c03] State: > >>> >> _at_safepoint _has_called_back 0 _at_poll_safepoint 0 > >>> >> > > JavaThread state: _thread_blocked > >>> >> > > at java.lang.Thread.sleep(java.base/Native Method) > >>> >> > > ... > >>> >> > > ------------------------------------------------------ > >>> >> > > > >>> >> > > As mentioned above, we have a whole bunch of other > >>> >> enhancements to the thread dump similar to this one and would be > >>> >> willing to contribute them if there is any interest. > >>> >> > > > >>> >> > > Thanks and best regards, > >>> >> > > Gunter > >>> >> > > > >>> >> > > > >>> >> > > >>> >> > > >>> >> > > >>> >> > > >>> >> > > >>> >> > >>> > > >>> > > >>> > >>> > > From david.holmes at oracle.com Mon Jun 11 21:21:23 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 12 Jun 2018 07:21:23 +1000 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> Message-ID: <469a46b5-aa9c-c6fd-b270-6a4230b4e08a@oracle.com> On 12/06/2018 12:12 AM, Bob Vandette wrote: > >> On Jun 11, 2018, at 4:32 AM, David Holmes > > wrote: >> >> Sorry Bob I haven't had a chance to look at this detail. >> >> For the Java code ... methods that return arrays should return >> zero-length arrays when something is not available rather than null. > > All methods do return zero length arrays except I missed the > getPerCpuUsage. ?I?ll fix that one and correct the javadoc. There are a few more too: 231 * @return An array of available CPUs or null if metric is not available. 232 * 233 */ 234 public int[] getCpuSetCpus(); 242 * @return An array of available and online CPUs or null if the metric 243 * is not available. 244 * 245 */ 246 public int[] getEffectiveCpuSetCpus(); 256 * @return An array of available memory nodes or null if metric is not available. 257 * 258 */ 259 public int[] getCpuSetMems(); 267 * @return An array of available and online nodes or null if the metric 268 * is not available. 269 * 270 */ 271 public int[] getEffectiveCpuSetMems(); > >> >> For getCpuPeriod() the term "operating system time slice" can be >> misconstrued as being related to the scheduler timeslice that may, or >> may not, exist, depending on the scheduler and scheduling policy etc. >> This "timeslice" is something specific to cgroups - no? > > The comments reads: > > * Returns the length of the operating system time slice, in > * milliseconds, for processes within the Isolation Group. > > The comment does infer that it?s process and cgroup (Isolation group) > specific and not the generic os timeslice. > Isn?t this sufficient? The phrase "operating system" makes this sound like some kind of global timeslice notion - which it isn't. And I don't like to think of cpu periods/shares/quotas in terms of "time slice" anyway. I don't see the Docker or Cgroup documentation using "time slice" either. It suffices IMHO to just say for period: * Returns the length of the scheduling period, in * milliseconds, for processes within the Isolation Group. then for quota: * Returns the total available run-time allowed, in milliseconds, * during each scheduling period for all tasks in the Isolation Group. Thanks, David > > Thanks, > Bob. >> >> David >> >> On 8/06/2018 3:43 AM, Bob Vandette wrote: >>> Can I get one more reviewer for this RFE so I can integrate it? >>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>> Mandy Chung has reviewed this change. >>> I?ve run Mach5 hotspot and core lib tests. >>> I?ve reviewed the tests which were written by Harsha Wardhana >>> I filed a CSR for the command line change and it?s now approved and >>> closed. >>> Thanks, >>> Bob. >>>> On May 30, 2018, at 3:45 PM, Bob Vandette >>> > wrote: >>>> >>>> Please review the following RFE which adds an internal API, along >>>> with jtreg tests that provide >>>> access to Docker container configuration data and metrics. ?In >>>> addition to the API which we hope to >>>> take advantage of in the future with Java Flight Recorder and a JMX >>>> Mbean, I?ve added an additional >>>> option to -XshowSettings:system than dumps out the container or host >>>> cgroup confguration >>>> information. ?See the sample output below: >>>> >>>> RFE: Container Metrics >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8203357 >>>> >>>> WEBREV: >>>> >>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>>> >>>> >>>> This commit will also include a fix for the following bug. >>>> >>>> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8203691 >>>> >>>> WEBREV: >>>> >>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >>>> >>>> SAMPLE USAGE and OUTPUT: >>>> >>>> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >>>> ./java -XshowSettings:system >>>> Operating System Metrics: >>>> ???Provider: cgroupv1 >>>> ???Effective CPU Count: 4 >>>> ???CPU Period: 100000 >>>> ???CPU Quota: -1 >>>> ???CPU Shares: -1 >>>> ???List of Processors, 4 total: >>>> ???4 5 6 7 >>>> ???List of Effective Processors, 4 total: >>>> ???4 5 6 7 >>>> ???List of Memory Nodes, 2 total: >>>> ???0 1 >>>> ???List of Available Memory Nodes, 2 total: >>>> ???0 1 >>>> ???CPUSet Memory Pressure Enabled: false >>>> ???Memory Limit: 256.00M >>>> ???Memory Soft Limit: Unlimited >>>> ???Memory & Swap Limit: 512.00M >>>> ???Kernel Memory Limit: Unlimited >>>> ???TCP Memory Limit: Unlimited >>>> ???Out Of Memory Killer Enabled: true >>>> >>>> TEST RESULTS: >>>> >>>> testing runtime container APIs >>>> Directory "JTwork" not found: creating >>>> Passed: runtime/containers/cgroup/PlainRead.java >>>> Passed: runtime/containers/docker/DockerBasicTest.java >>>> Passed: runtime/containers/docker/TestCPUAwareness.java >>>> Passed: runtime/containers/docker/TestCPUSets.java >>>> Passed: runtime/containers/docker/TestMemoryAwareness.java >>>> Passed: runtime/containers/docker/TestMisc.java >>>> Test results: passed: 6 >>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>> >>>> testing jdk.internal.platform APIs >>>> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >>>> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >>>> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >>>> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >>>> Test results: passed: 4 >>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>> >>>> testing -XshowSettings:system launcher option >>>> Passed: tools/launcher/Settings.java >>>> Test results: passed: 1 >>>> >>>> >>>> Bob. >>>> >>>> > From bob.vandette at oracle.com Mon Jun 11 23:30:04 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Mon, 11 Jun 2018 19:30:04 -0400 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: <469a46b5-aa9c-c6fd-b270-6a4230b4e08a@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> <469a46b5-aa9c-c6fd-b270-6a4230b4e08a@oracle.com> Message-ID: > On Jun 11, 2018, at 5:21 PM, David Holmes wrote: > > On 12/06/2018 12:12 AM, Bob Vandette wrote: >>> On Jun 11, 2018, at 4:32 AM, David Holmes > wrote: >>> >>> Sorry Bob I haven't had a chance to look at this detail. >>> >>> For the Java code ... methods that return arrays should return zero-length arrays when something is not available rather than null. >> All methods do return zero length arrays except I missed the getPerCpuUsage. I?ll fix that one and correct the javadoc. > > There are a few more too: > Those are covered by the function that converts the string range. > 231 * @return An array of available CPUs or null if metric is not available. > 232 * > 233 */ > 234 public int[] getCpuSetCpus(); > > 242 * @return An array of available and online CPUs or null if the metric > 243 * is not available. > 244 * > 245 */ > 246 public int[] getEffectiveCpuSetCpus(); > > 256 * @return An array of available memory nodes or null if metric is not available. > 257 * > 258 */ > 259 public int[] getCpuSetMems(); > > 267 * @return An array of available and online nodes or null if the metric > 268 * is not available. > 269 * > 270 */ > 271 public int[] getEffectiveCpuSetMems(); >>> >>> For getCpuPeriod() the term "operating system time slice" can be misconstrued as being related to the scheduler timeslice that may, or may not, exist, depending on the scheduler and scheduling policy etc. This "timeslice" is something specific to cgroups - no? >> The comments reads: >> * Returns the length of the operating system time slice, in >> * milliseconds, for processes within the Isolation Group. >> The comment does infer that it?s process and cgroup (Isolation group) specific and not the generic os timeslice. >> Isn?t this sufficient? > > The phrase "operating system" makes this sound like some kind of global timeslice notion - which it isn't. And I don't like to think of cpu periods/shares/quotas in terms of "time slice" anyway. I don't see the Docker or Cgroup documentation using "time slice" either. It suffices IMHO to just say for period: > > * Returns the length of the scheduling period, in > * milliseconds, for processes within the Isolation Group. > > then for quota: > > * Returns the total available run-time allowed, in milliseconds, > * during each scheduling period for all tasks in the Isolation Group. > Ok. I?ll update the docs. Bob > Thanks, > David > >> Thanks, >> Bob. >>> >>> David >>> >>>> On 8/06/2018 3:43 AM, Bob Vandette wrote: >>>> Can I get one more reviewer for this RFE so I can integrate it? >>>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>>> Mandy Chung has reviewed this change. >>>> I?ve run Mach5 hotspot and core lib tests. >>>> I?ve reviewed the tests which were written by Harsha Wardhana >>>> I filed a CSR for the command line change and it?s now approved and closed. >>>> Thanks, >>>> Bob. >>>>> On May 30, 2018, at 3:45 PM, Bob Vandette > wrote: >>>>> >>>>> Please review the following RFE which adds an internal API, along with jtreg tests that provide >>>>> access to Docker container configuration data and metrics. In addition to the API which we hope to >>>>> take advantage of in the future with Java Flight Recorder and a JMX Mbean, I?ve added an additional >>>>> option to -XshowSettings:system than dumps out the container or host cgroup confguration >>>>> information. See the sample output below: >>>>> >>>>> RFE: Container Metrics >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8203357 >>>>> >>>>> WEBREV: >>>>> >>>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>>>> >>>>> >>>>> This commit will also include a fix for the following bug. >>>>> >>>>> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8203691 >>>>> >>>>> WEBREV: >>>>> >>>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >>>>> >>>>> SAMPLE USAGE and OUTPUT: >>>>> >>>>> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >>>>> ./java -XshowSettings:system >>>>> Operating System Metrics: >>>>> Provider: cgroupv1 >>>>> Effective CPU Count: 4 >>>>> CPU Period: 100000 >>>>> CPU Quota: -1 >>>>> CPU Shares: -1 >>>>> List of Processors, 4 total: >>>>> 4 5 6 7 >>>>> List of Effective Processors, 4 total: >>>>> 4 5 6 7 >>>>> List of Memory Nodes, 2 total: >>>>> 0 1 >>>>> List of Available Memory Nodes, 2 total: >>>>> 0 1 >>>>> CPUSet Memory Pressure Enabled: false >>>>> Memory Limit: 256.00M >>>>> Memory Soft Limit: Unlimited >>>>> Memory & Swap Limit: 512.00M >>>>> Kernel Memory Limit: Unlimited >>>>> TCP Memory Limit: Unlimited >>>>> Out Of Memory Killer Enabled: true >>>>> >>>>> TEST RESULTS: >>>>> >>>>> testing runtime container APIs >>>>> Directory "JTwork" not found: creating >>>>> Passed: runtime/containers/cgroup/PlainRead.java >>>>> Passed: runtime/containers/docker/DockerBasicTest.java >>>>> Passed: runtime/containers/docker/TestCPUAwareness.java >>>>> Passed: runtime/containers/docker/TestCPUSets.java >>>>> Passed: runtime/containers/docker/TestMemoryAwareness.java >>>>> Passed: runtime/containers/docker/TestMisc.java >>>>> Test results: passed: 6 >>>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>>> >>>>> testing jdk.internal.platform APIs >>>>> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >>>>> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >>>>> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >>>>> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >>>>> Test results: passed: 4 >>>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>>> >>>>> testing -XshowSettings:system launcher option >>>>> Passed: tools/launcher/Settings.java >>>>> Test results: passed: 1 >>>>> >>>>> >>>>> Bob. >>>>> >>>>> From daniil.x.titov at oracle.com Tue Jun 12 00:52:19 2018 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Mon, 11 Jun 2018 17:52:19 -0700 Subject: RFR 8203809: [Graal] JDI tests fail with: JDITestRuntimeException : ** event IS NOT a breakpoint ** Message-ID: Please review the changes that fix failure of 6 JDI tests run with Graal. The problem here is that currently the tests filter out only 2 specific JFR events (nsk.share.jdi.EventsFilter.filtered()) before analyzing them and fail if the received event doesn't match the expected one. But with Graal turned on new events generated by the compiler thread are occasionally found in the event queue and cause the tests failure. The fix makes the tests to use an additional check to filter out events not related to the debuggee class. Bug: https://bugs.openjdk.java.net/browse/JDK-8203809 Webrev: http://cr.openjdk.java.net/~dtitov/8203809/webrev.01 Thanks, Daniil From serguei.spitsyn at oracle.com Tue Jun 12 01:18:19 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 11 Jun 2018 18:18:19 -0700 Subject: RFR 8203809: [Graal] JDI tests fail with: JDITestRuntimeException : ** event IS NOT a breakpoint ** In-Reply-To: References: Message-ID: <36f11b76-ce82-bdff-e763-4d3d73208119@oracle.com> An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Tue Jun 12 05:12:54 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 12 Jun 2018 15:12:54 +1000 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> <469a46b5-aa9c-c6fd-b270-6a4230b4e08a@oracle.com> Message-ID: <2b705a94-afe0-c9bb-4377-accabf73696e@oracle.com> On 12/06/2018 9:30 AM, Bob Vandette wrote: > > >> On Jun 11, 2018, at 5:21 PM, David Holmes wrote: >> >> On 12/06/2018 12:12 AM, Bob Vandette wrote: >>>> On Jun 11, 2018, at 4:32 AM, David Holmes > wrote: >>>> >>>> Sorry Bob I haven't had a chance to look at this detail. >>>> >>>> For the Java code ... methods that return arrays should return zero-length arrays when something is not available rather than null. >>> All methods do return zero length arrays except I missed the getPerCpuUsage. I?ll fix that one and correct the javadoc. >> >> There are a few more too: >> > > Those are covered by the function that converts the string range. ??? I have no idea what you mean. Java API design style is to return zero-length arrays rather than null. [Ref: Effective Java First Edition, Item 27]. Cheers, David ----- >> 231 * @return An array of available CPUs or null if metric is not available. >> 232 * >> 233 */ >> 234 public int[] getCpuSetCpus(); >> >> 242 * @return An array of available and online CPUs or null if the metric >> 243 * is not available. >> 244 * >> 245 */ >> 246 public int[] getEffectiveCpuSetCpus(); >> >> 256 * @return An array of available memory nodes or null if metric is not available. >> 257 * >> 258 */ >> 259 public int[] getCpuSetMems(); >> >> 267 * @return An array of available and online nodes or null if the metric >> 268 * is not available. >> 269 * >> 270 */ >> 271 public int[] getEffectiveCpuSetMems(); >>>> >>>> For getCpuPeriod() the term "operating system time slice" can be misconstrued as being related to the scheduler timeslice that may, or may not, exist, depending on the scheduler and scheduling policy etc. This "timeslice" is something specific to cgroups - no? >>> The comments reads: >>> * Returns the length of the operating system time slice, in >>> * milliseconds, for processes within the Isolation Group. >>> The comment does infer that it?s process and cgroup (Isolation group) specific and not the generic os timeslice. >>> Isn?t this sufficient? >> >> The phrase "operating system" makes this sound like some kind of global timeslice notion - which it isn't. And I don't like to think of cpu periods/shares/quotas in terms of "time slice" anyway. I don't see the Docker or Cgroup documentation using "time slice" either. It suffices IMHO to just say for period: >> >> * Returns the length of the scheduling period, in >> * milliseconds, for processes within the Isolation Group. >> >> then for quota: >> >> * Returns the total available run-time allowed, in milliseconds, >> * during each scheduling period for all tasks in the Isolation Group. >> > > Ok. I?ll update the docs. > Bob > >> Thanks, >> David >> >>> Thanks, >>> Bob. >>>> >>>> David >>>> >>>>> On 8/06/2018 3:43 AM, Bob Vandette wrote: >>>>> Can I get one more reviewer for this RFE so I can integrate it? >>>>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>>>> Mandy Chung has reviewed this change. >>>>> I?ve run Mach5 hotspot and core lib tests. >>>>> I?ve reviewed the tests which were written by Harsha Wardhana >>>>> I filed a CSR for the command line change and it?s now approved and closed. >>>>> Thanks, >>>>> Bob. >>>>>> On May 30, 2018, at 3:45 PM, Bob Vandette > wrote: >>>>>> >>>>>> Please review the following RFE which adds an internal API, along with jtreg tests that provide >>>>>> access to Docker container configuration data and metrics. In addition to the API which we hope to >>>>>> take advantage of in the future with Java Flight Recorder and a JMX Mbean, I?ve added an additional >>>>>> option to -XshowSettings:system than dumps out the container or host cgroup confguration >>>>>> information. See the sample output below: >>>>>> >>>>>> RFE: Container Metrics >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8203357 >>>>>> >>>>>> WEBREV: >>>>>> >>>>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>>>>> >>>>>> >>>>>> This commit will also include a fix for the following bug. >>>>>> >>>>>> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8203691 >>>>>> >>>>>> WEBREV: >>>>>> >>>>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >>>>>> >>>>>> SAMPLE USAGE and OUTPUT: >>>>>> >>>>>> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >>>>>> ./java -XshowSettings:system >>>>>> Operating System Metrics: >>>>>> Provider: cgroupv1 >>>>>> Effective CPU Count: 4 >>>>>> CPU Period: 100000 >>>>>> CPU Quota: -1 >>>>>> CPU Shares: -1 >>>>>> List of Processors, 4 total: >>>>>> 4 5 6 7 >>>>>> List of Effective Processors, 4 total: >>>>>> 4 5 6 7 >>>>>> List of Memory Nodes, 2 total: >>>>>> 0 1 >>>>>> List of Available Memory Nodes, 2 total: >>>>>> 0 1 >>>>>> CPUSet Memory Pressure Enabled: false >>>>>> Memory Limit: 256.00M >>>>>> Memory Soft Limit: Unlimited >>>>>> Memory & Swap Limit: 512.00M >>>>>> Kernel Memory Limit: Unlimited >>>>>> TCP Memory Limit: Unlimited >>>>>> Out Of Memory Killer Enabled: true >>>>>> >>>>>> TEST RESULTS: >>>>>> >>>>>> testing runtime container APIs >>>>>> Directory "JTwork" not found: creating >>>>>> Passed: runtime/containers/cgroup/PlainRead.java >>>>>> Passed: runtime/containers/docker/DockerBasicTest.java >>>>>> Passed: runtime/containers/docker/TestCPUAwareness.java >>>>>> Passed: runtime/containers/docker/TestCPUSets.java >>>>>> Passed: runtime/containers/docker/TestMemoryAwareness.java >>>>>> Passed: runtime/containers/docker/TestMisc.java >>>>>> Test results: passed: 6 >>>>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>>>> >>>>>> testing jdk.internal.platform APIs >>>>>> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >>>>>> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >>>>>> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >>>>>> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >>>>>> Test results: passed: 4 >>>>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>>>> >>>>>> testing -XshowSettings:system launcher option >>>>>> Passed: tools/launcher/Settings.java >>>>>> Test results: passed: 1 >>>>>> >>>>>> >>>>>> Bob. >>>>>> >>>>>> > From mandy.chung at oracle.com Tue Jun 12 05:31:05 2018 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 11 Jun 2018 22:31:05 -0700 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: <2b705a94-afe0-c9bb-4377-accabf73696e@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> <469a46b5-aa9c-c6fd-b270-6a4230b4e08a@oracle.com> <2b705a94-afe0-c9bb-4377-accabf73696e@oracle.com> Message-ID: On 6/11/18 10:12 PM, David Holmes wrote: >>>>> >>>>> For the Java code ... methods that return arrays should return >>>>> zero-length arrays when something is not available rather than null. >>>> All methods do return zero length arrays except I missed the >>>> getPerCpuUsage.? I?ll fix that one and correct the javadoc. >>> >>> There are a few more too: >>> >> >> Those are covered by the function that converts the string range. > > ??? I have no idea what you mean. I think the methods returning an array calls Subsystem::StringRangeToIntArray which returns an empty array. 171 public static int[] StringRangeToIntArray(String range) { 172 int[] ints = new int[0]; 173 174 if (range == null) return ints; Mandy From thomas.stuefe at gmail.com Tue Jun 12 05:36:12 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 12 Jun 2018 07:36:12 +0200 Subject: RFR(s): 8203343: VM.{metaspace|classloaders|classhierarchy...} jcmd should show invocation targets for Generated{Method|Constructor}AccessorImpl classes In-Reply-To: References: Message-ID: Ping. jdk-submit tests came back clean. Thanks, Thomas On Wed, Jun 6, 2018 at 6:05 PM, Thomas St?fe wrote: > Dear all, > > may I please have feedback and if possible reviews for this small addition: > > CR: https://bugs.openjdk.java.net/browse/JDK-8203343 > Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/webrev.00/webrev/ > > (Note: this patch goes atop of > https://bugs.openjdk.java.net/browse/JDK-8203682, which is currently > up for RFR). > > --- > > When analyzing situations involving a lot of reflection, one often > stares at walls of "GeneratedXXXAccessorXXX" classes. These names are > generated and not at all helpful in analyzing the problem (e.g. which > component in a server node does this much reflection and hence uses so > much metaspace). > > This patch adds the ability to print out invocation targets > additionally to class names if the class is a generated accessor - for > now this ability has been added to VM.metaspace, VM.classloaders and > VM.class_hierarchy. > > Example output from "VM.class_hierarchy": > > > |--jdk.internal.reflect.MagicAccessorImpl/null > > | |--jdk.internal.reflect.ConstructorAccessorImpl/null > | | |--jdk.internal.reflect.GeneratedConstructorAccessor18/0x00007f9ee8350c10 > (invokes: org/springframework/boot/context/properties/ConfigurationPropertiesBindingPostProcessorRegistrar:: > ()V) > | | |--jdk.internal.reflect.GeneratedConstructorAccessor17/0x00007f9ee8349c00 > (invokes: org/springframework/boot/context/properties/EnableConfigurationPropertiesImportSelector$ConfigurationPropertiesBeanRegistrar:: > ()V) > > | |--jdk.internal.reflect.MethodAccessorImpl/null > | | |--jdk.internal.reflect.GeneratedMethodAccessor23/0x00007f9ec8329b60 > (invokes: org/apache/tomcat/util/modeler/AttributeInfo::setIs (Z)V) > | | |--jdk.internal.reflect.GeneratedMethodAccessor22/0x00007f9ee831bc70 > (invokes: org/springframework/boot/autoconfigure/SpringBootApplication::exclude > ()[Ljava/lang/Class;) > > > See here more examples: > > "VM.class_hierarchy" > http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/example-VM.class_hierarchy.txt > > "VM.classloaders show-classes" > http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/example-VM.classloaders.txt > > > ---- > > Note: I am not sure if this is a fit for hotspot-runtime or > serviceability. Sorry for crossposting. > > Thank you, > > Thomas From david.holmes at oracle.com Tue Jun 12 05:43:13 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 12 Jun 2018 15:43:13 +1000 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> <469a46b5-aa9c-c6fd-b270-6a4230b4e08a@oracle.com> <2b705a94-afe0-c9bb-4377-accabf73696e@oracle.com> Message-ID: <73dd06ec-544f-3cc9-00d5-c5c8c8ffdacc@oracle.com> On 12/06/2018 3:31 PM, mandy chung wrote: > On 6/11/18 10:12 PM, David Holmes wrote: >>>>>> >>>>>> For the Java code ... methods that return arrays should return >>>>>> zero-length arrays when something is not available rather than null. >>>>> All methods do return zero length arrays except I missed the >>>>> getPerCpuUsage.? I?ll fix that one and correct the javadoc. >>>> >>>> There are a few more too: >>>> >>> >>> Those are covered by the function that converts the string range. >> >> ??? I have no idea what you mean. > > > I think the methods returning an array calls > Subsystem::StringRangeToIntArray which returns an empty array. > > ?171???? public static int[] StringRangeToIntArray(String range) { > ?172???????? int[] ints = new int[0]; > ?173 > ?174???????? if (range == null) return ints; I'm commenting on the specification of the Metrics interface: http://cr.openjdk.java.net/~bobv/8203357/webrev.01/src/java.base/share/classes/jdk/internal/platform/Metrics.java.html not any implementation. Cheers, David > > Mandy From mandy.chung at oracle.com Tue Jun 12 06:00:57 2018 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 11 Jun 2018 23:00:57 -0700 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: <73dd06ec-544f-3cc9-00d5-c5c8c8ffdacc@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> <469a46b5-aa9c-c6fd-b270-6a4230b4e08a@oracle.com> <2b705a94-afe0-c9bb-4377-accabf73696e@oracle.com> <73dd06ec-544f-3cc9-00d5-c5c8c8ffdacc@oracle.com> Message-ID: <413aa29c-1a7c-8cd2-ca27-201dc6b53432@oracle.com> On 6/11/18 10:43 PM, David Holmes wrote: > On 12/06/2018 3:31 PM, mandy chung wrote: >> On 6/11/18 10:12 PM, David Holmes wrote: >>>>>>> >>>>>>> For the Java code ... methods that return arrays should return >>>>>>> zero-length arrays when something is not available rather than null. >>>>>> All methods do return zero length arrays except I missed the >>>>>> getPerCpuUsage.? I?ll fix that one and correct the javadoc. >>>>> >>>>> There are a few more too: >>>>> >>>> >>>> Those are covered by the function that converts the string range. >>> >>> ??? I have no idea what you mean. >> >> >> I think the methods returning an array calls >> Subsystem::StringRangeToIntArray which returns an empty array. >> >> ??171???? public static int[] StringRangeToIntArray(String range) { >> ??172???????? int[] ints = new int[0]; >> ??173 >> ??174???????? if (range == null) return ints; > > I'm commenting on the specification of the Metrics interface: > > http://cr.openjdk.java.net/~bobv/8203357/webrev.01/src/java.base/share/classes/jdk/internal/platform/Metrics.java.html > not any implementation. The implementation returns empty array, which is good. Yes the javadoc should be updated. Mandy From kirk.pepperdine at gmail.com Tue Jun 12 08:00:47 2018 From: kirk.pepperdine at gmail.com (Kirk Pepperdine) Date: Tue, 12 Jun 2018 11:00:47 +0300 Subject: Kerberos authentication for JMX? In-Reply-To: References: Message-ID: <2BA66DF8-9833-4D8A-A436-78AD36108065@gmail.com> Hi Peter, This is an issue for prod environments that is becoming bigger as clusters become bigger and bigger. I believe the answer to your issues and others related to the reliance of RMI has been proven by a project call Jolokia (https://jolokia.org ) which uses REST. At issue is that Jolokia is *not* a drop in JMXConnector replacement meaning you can?t use standard client tooling and this unfortunately compromises Jolokia?s usefulness. There is a JEP (http://openjdk.java.net/jeps/8171311 ) for providing a REST adaptor that unfortunately also misses the mark in that it?s not a JMXConnector. I?m not sure *why* these efforts have seemingly avoided the obvious solution which would be an REST based implementation of the JMXConnector interface as I believe that would be about the same about of work and would allow everyone to continue to use already available tooling. I have the task to prototype my own implementation running 2rd on my todo list right after I get my heap dump analysis tooling functional. So, yes, this is a real issue and I hope a discussion will lead to a more scalable solution. Kind regards, Kirk > On Jun 11, 2018, at 4:14 PM, P?ter Gergely Horv?th wrote: > > Hi All, > > I have been working with Big Data for a while and I have seen that a number of the components have started to have their own custom baked solutions (minimalistic Web UIs) for basic management operations, like showing metrics, debugging etc instead of using JMX. > > I have the feeling that getting JMX working for dozens of different Java services within a large cluster is an overly tough task, especially if you do not want to make compromises around security. For me it seems, that at the moment there is a gap between what the JDK offers regarding JMX monitoring/management and what people would need in a real world setting to use it effectively in an easy and secure way. > > I am wondering if it would be possible to implement a Kerberos-based authentication mechanism for JMX, allowing all services of a cluster to authenticate JMX clients against a centrally managed Kerberos service, that would also be officially supported by VisualVM so as to give an easy-to-use user interface. > > > Based on my understanding, this could either be a new protocol implementation or assuming JDK-8171311: REST APIs for JMX gets done, an additional feature around there to support GSS Negotiate/SPNEGO based authentication. > > Could you please share your thoughts on this? Would anyone be interested to sponsor this topic? > > Thanks, > Peter > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From shade at redhat.com Tue Jun 12 08:48:24 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 12 Jun 2018 10:48:24 +0200 Subject: RFR: 8204180: Implementation: JEP 318: Epsilon GC (round 5) In-Reply-To: <0e63a401-d28d-2bdb-d024-3b48f3fba0cb@oracle.com> References: <2ee03550-2b1e-b873-5ef3-26312e2a760e@redhat.com> <7211f820-4999-2988-e06c-1067b1fb138f@oracle.com> <0e63a401-d28d-2bdb-d024-3b48f3fba0cb@oracle.com> Message-ID: <2eab1a10-121f-884d-0155-5a422f2a3268@redhat.com> On 06/07/2018 02:42 PM, Per Liden wrote: > On 06/07/2018 02:38 PM, Aleksey Shipilev wrote: >> On 06/07/2018 12:13 PM, Per Liden wrote: >>> On 06/06/2018 06:30 PM, Aleksey Shipilev wrote: >>>> Hi, >>>> >>>> This is fifth (and hopefully final) round of code review for Epsilon GC changes. It includes the >>>> fixes done as the result of fourth round of reviews, mostly in Serviceability. The build parts are >>>> the same since last few reviews, so this is not posted to build-dev at . >>>> >>>> Webrev: >>>> ??? http://cr.openjdk.java.net/~shade/epsilon/webrev.09/ >>> >>> Still looks good to me! >>> I have two follow ups after jdk-submit Windows and Solaris build/test failures: http://hg.openjdk.java.net/jdk/sandbox/rev/24339b23a56b http://hg.openjdk.java.net/jdk/sandbox/rev/f21420b61fd2 Full webrev that passes jdk-submit: http://cr.openjdk.java.net/~shade/epsilon/webrev.11/ Thanks, -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From per.liden at oracle.com Tue Jun 12 09:17:33 2018 From: per.liden at oracle.com (Per Liden) Date: Tue, 12 Jun 2018 11:17:33 +0200 Subject: RFR: 8204180: Implementation: JEP 318: Epsilon GC (round 5) In-Reply-To: <2eab1a10-121f-884d-0155-5a422f2a3268@redhat.com> References: <2ee03550-2b1e-b873-5ef3-26312e2a760e@redhat.com> <7211f820-4999-2988-e06c-1067b1fb138f@oracle.com> <0e63a401-d28d-2bdb-d024-3b48f3fba0cb@oracle.com> <2eab1a10-121f-884d-0155-5a422f2a3268@redhat.com> Message-ID: <5e4c692e-c638-1ec2-e9d3-faf613bcbf18@oracle.com> On 06/12/2018 10:48 AM, Aleksey Shipilev wrote: > On 06/07/2018 02:42 PM, Per Liden wrote: >> On 06/07/2018 02:38 PM, Aleksey Shipilev wrote: >>> On 06/07/2018 12:13 PM, Per Liden wrote: >>>> On 06/06/2018 06:30 PM, Aleksey Shipilev wrote: >>>>> Hi, >>>>> >>>>> This is fifth (and hopefully final) round of code review for Epsilon GC changes. It includes the >>>>> fixes done as the result of fourth round of reviews, mostly in Serviceability. The build parts are >>>>> the same since last few reviews, so this is not posted to build-dev at . >>>>> >>>>> Webrev: >>>>> ??? http://cr.openjdk.java.net/~shade/epsilon/webrev.09/ >>>> >>>> Still looks good to me! >>>> > > I have two follow ups after jdk-submit Windows and Solaris build/test failures: > http://hg.openjdk.java.net/jdk/sandbox/rev/24339b23a56b > http://hg.openjdk.java.net/jdk/sandbox/rev/f21420b61fd2 Looks good to me! /Per > > Full webrev that passes jdk-submit: > http://cr.openjdk.java.net/~shade/epsilon/webrev.11/ > > Thanks, > -Aleksey > From harsha.wardhana.b at oracle.com Tue Jun 12 09:41:52 2018 From: harsha.wardhana.b at oracle.com (Harsha Wardhana B) Date: Tue, 12 Jun 2018 15:11:52 +0530 Subject: Kerberos authentication for JMX? In-Reply-To: References: Message-ID: <51f09658-41ff-f83e-617a-18abf25413c8@oracle.com> Hi Peter, JMX agents support JAAS based authentication. The login module - Krb5LoginModule along with JAAS can be used to setup Kerberos-based authentication for JMX. A JAAS config file for Kerberos clients could look like, kerberosClient { ?????? com.sun.security.auth.module.Krb5LoginModule required ?????? debug=true ?????? useKeyTab=false ????? ........... }; where the configuration options for above login module can be found at, https://docs.oracle.com/javase/10/docs/api/com/sun/security/auth/module/Krb5LoginModule.html The default JMX agent can be started by setting below two system properties. 1. com.sun.management.jmxremote.login.config - The JAAS entry in config file above 2. java.security.auth.login.config - The path to the above file Example: java -Dcom.sun.management.jmxremote.port=5000 ???? -Dcom.sun.management.jmxremote.login.config=kerberosClient ???? -Djava.security.auth.login.config=kerberos.config ???? -jar MyApplication.jar I have not tried the steps above myself but should work in theory. Give it a try and let me know if it works. Thanks Harsha On Monday 11 June 2018 06:44 PM, P?ter Gergely Horv?th wrote: > Hi All, > > I have been working with Big Data for a while and I have seen that a > number of the components have started to have their own custom baked > solutions (minimalistic Web UIs) for basic management operations, like > showing metrics, debugging etc instead of using JMX. > > I have the feeling that getting JMX working for dozens of different > Java services within a large cluster is an overly tough task, > especially if you do not want to make compromises around security. For > me it seems, that at the moment there is a gap between what the JDK > offers regarding JMX monitoring/management and what people would need > in a real world setting to use iteffectively in an easy and secure way. > > I am wondering if it would be possible to implement a Kerberos-based > authentication mechanism for JMX, allowing all services of a cluster > to authenticate JMX clients against a centrally managed Kerberos > service, that would also be officially supported by VisualVM so as to > give an easy-to-use user interface. > > > Based on my understanding, this could either be a new protocol > implementation or assuming JDK-8171311: REST APIs for JMX gets done, > an additional feature around there to support GSS > Negotiate/SPNEGO?based authentication. > > Could you please share your thoughts on this? Would anyone be > interested to sponsor this topic? > > Thanks, > Peter > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rkennke at redhat.com Tue Jun 12 09:58:40 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 12 Jun 2018 11:58:40 +0200 Subject: RFR: 8204180: Implementation: JEP 318: Epsilon GC (round 5) In-Reply-To: <2eab1a10-121f-884d-0155-5a422f2a3268@redhat.com> References: <2ee03550-2b1e-b873-5ef3-26312e2a760e@redhat.com> <7211f820-4999-2988-e06c-1067b1fb138f@oracle.com> <0e63a401-d28d-2bdb-d024-3b48f3fba0cb@oracle.com> <2eab1a10-121f-884d-0155-5a422f2a3268@redhat.com> Message-ID: > On 06/07/2018 02:42 PM, Per Liden wrote: >> On 06/07/2018 02:38 PM, Aleksey Shipilev wrote: >>> On 06/07/2018 12:13 PM, Per Liden wrote: >>>> On 06/06/2018 06:30 PM, Aleksey Shipilev wrote: >>>>> Hi, >>>>> >>>>> This is fifth (and hopefully final) round of code review for Epsilon GC changes. It includes the >>>>> fixes done as the result of fourth round of reviews, mostly in Serviceability. The build parts are >>>>> the same since last few reviews, so this is not posted to build-dev at . >>>>> >>>>> Webrev: >>>>> ??? http://cr.openjdk.java.net/~shade/epsilon/webrev.09/ >>>> >>>> Still looks good to me! >>>> > > I have two follow ups after jdk-submit Windows and Solaris build/test failures: > http://hg.openjdk.java.net/jdk/sandbox/rev/24339b23a56b > http://hg.openjdk.java.net/jdk/sandbox/rev/f21420b61fd2 > > Full webrev that passes jdk-submit: > http://cr.openjdk.java.net/~shade/epsilon/webrev.11/ > > Thanks, > -Aleksey > Looks good to me! Thanks, Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From bob.vandette at oracle.com Tue Jun 12 10:56:33 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Tue, 12 Jun 2018 06:56:33 -0400 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: <2b705a94-afe0-c9bb-4377-accabf73696e@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> <469a46b5-aa9c-c6fd-b270-6a4230b4e08a@oracle.com> <2b705a94-afe0-c9bb-4377-accabf73696e@oracle.com> Message-ID: > On Jun 12, 2018, at 1:12 AM, David Holmes wrote: > > On 12/06/2018 9:30 AM, Bob Vandette wrote: >>> On Jun 11, 2018, at 5:21 PM, David Holmes wrote: >>> >>> On 12/06/2018 12:12 AM, Bob Vandette wrote: >>>>> On Jun 11, 2018, at 4:32 AM, David Holmes > wrote: >>>>> >>>>> Sorry Bob I haven't had a chance to look at this detail. >>>>> >>>>> For the Java code ... methods that return arrays should return zero-length arrays when something is not available rather than null. >>>> All methods do return zero length arrays except I missed the getPerCpuUsage. I?ll fix that one and correct the javadoc. >>> >>> There are a few more too: >>> >> Those are covered by the function that converts the string range. > > ??? I have no idea what you mean. > > Java API design style is to return zero-length arrays rather than null. [Ref: Effective Java First Edition, Item 27]. Look at line 174 in this file. http://cr.openjdk.java.net/~bobv/8203357/webrev.01/src/java.base/linux/classes/jdk/internal/platform/cgroupv1/SubSystem.java.html Bob > > Cheers, > David > ----- > >>> 231 * @return An array of available CPUs or null if metric is not available. >>> 232 * >>> 233 */ >>> 234 public int[] getCpuSetCpus(); >>> >>> 242 * @return An array of available and online CPUs or null if the metric >>> 243 * is not available. >>> 244 * >>> 245 */ >>> 246 public int[] getEffectiveCpuSetCpus(); >>> >>> 256 * @return An array of available memory nodes or null if metric is not available. >>> 257 * >>> 258 */ >>> 259 public int[] getCpuSetMems(); >>> >>> 267 * @return An array of available and online nodes or null if the metric >>> 268 * is not available. >>> 269 * >>> 270 */ >>> 271 public int[] getEffectiveCpuSetMems(); >>>>> >>>>> For getCpuPeriod() the term "operating system time slice" can be misconstrued as being related to the scheduler timeslice that may, or may not, exist, depending on the scheduler and scheduling policy etc. This "timeslice" is something specific to cgroups - no? >>>> The comments reads: >>>> * Returns the length of the operating system time slice, in >>>> * milliseconds, for processes within the Isolation Group. >>>> The comment does infer that it?s process and cgroup (Isolation group) specific and not the generic os timeslice. >>>> Isn?t this sufficient? >>> >>> The phrase "operating system" makes this sound like some kind of global timeslice notion - which it isn't. And I don't like to think of cpu periods/shares/quotas in terms of "time slice" anyway. I don't see the Docker or Cgroup documentation using "time slice" either. It suffices IMHO to just say for period: >>> >>> * Returns the length of the scheduling period, in >>> * milliseconds, for processes within the Isolation Group. >>> >>> then for quota: >>> >>> * Returns the total available run-time allowed, in milliseconds, >>> * during each scheduling period for all tasks in the Isolation Group. >>> >> Ok. I?ll update the docs. >> Bob >>> Thanks, >>> David >>> >>>> Thanks, >>>> Bob. >>>>> >>>>> David >>>>> >>>>>>> On 8/06/2018 3:43 AM, Bob Vandette wrote: >>>>>>> Can I get one more reviewer for this RFE so I can integrate it? >>>>>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>>>>> Mandy Chung has reviewed this change. >>>>>> I?ve run Mach5 hotspot and core lib tests. >>>>>> I?ve reviewed the tests which were written by Harsha Wardhana >>>>>> I filed a CSR for the command line change and it?s now approved and closed. >>>>>> Thanks, >>>>>> Bob. >>>>>>> On May 30, 2018, at 3:45 PM, Bob Vandette > wrote: >>>>>>> >>>>>>> Please review the following RFE which adds an internal API, along with jtreg tests that provide >>>>>>> access to Docker container configuration data and metrics. In addition to the API which we hope to >>>>>>> take advantage of in the future with Java Flight Recorder and a JMX Mbean, I?ve added an additional >>>>>>> option to -XshowSettings:system than dumps out the container or host cgroup confguration >>>>>>> information. See the sample output below: >>>>>>> >>>>>>> RFE: Container Metrics >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8203357 >>>>>>> >>>>>>> WEBREV: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>>>>>> >>>>>>> >>>>>>> This commit will also include a fix for the following bug. >>>>>>> >>>>>>> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8203691 >>>>>>> >>>>>>> WEBREV: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >>>>>>> >>>>>>> SAMPLE USAGE and OUTPUT: >>>>>>> >>>>>>> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >>>>>>> ./java -XshowSettings:system >>>>>>> Operating System Metrics: >>>>>>> Provider: cgroupv1 >>>>>>> Effective CPU Count: 4 >>>>>>> CPU Period: 100000 >>>>>>> CPU Quota: -1 >>>>>>> CPU Shares: -1 >>>>>>> List of Processors, 4 total: >>>>>>> 4 5 6 7 >>>>>>> List of Effective Processors, 4 total: >>>>>>> 4 5 6 7 >>>>>>> List of Memory Nodes, 2 total: >>>>>>> 0 1 >>>>>>> List of Available Memory Nodes, 2 total: >>>>>>> 0 1 >>>>>>> CPUSet Memory Pressure Enabled: false >>>>>>> Memory Limit: 256.00M >>>>>>> Memory Soft Limit: Unlimited >>>>>>> Memory & Swap Limit: 512.00M >>>>>>> Kernel Memory Limit: Unlimited >>>>>>> TCP Memory Limit: Unlimited >>>>>>> Out Of Memory Killer Enabled: true >>>>>>> >>>>>>> TEST RESULTS: >>>>>>> >>>>>>> testing runtime container APIs >>>>>>> Directory "JTwork" not found: creating >>>>>>> Passed: runtime/containers/cgroup/PlainRead.java >>>>>>> Passed: runtime/containers/docker/DockerBasicTest.java >>>>>>> Passed: runtime/containers/docker/TestCPUAwareness.java >>>>>>> Passed: runtime/containers/docker/TestCPUSets.java >>>>>>> Passed: runtime/containers/docker/TestMemoryAwareness.java >>>>>>> Passed: runtime/containers/docker/TestMisc.java >>>>>>> Test results: passed: 6 >>>>>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>>>>> >>>>>>> testing jdk.internal.platform APIs >>>>>>> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >>>>>>> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >>>>>>> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >>>>>>> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >>>>>>> Test results: passed: 4 >>>>>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>>>>> >>>>>>> testing -XshowSettings:system launcher option >>>>>>> Passed: tools/launcher/Settings.java >>>>>>> Test results: passed: 1 >>>>>>> >>>>>>> >>>>>>> Bob. >>>>>>> >>>>>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob.vandette at oracle.com Tue Jun 12 10:59:23 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Tue, 12 Jun 2018 06:59:23 -0400 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: <73dd06ec-544f-3cc9-00d5-c5c8c8ffdacc@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> <469a46b5-aa9c-c6fd-b270-6a4230b4e08a@oracle.com> <2b705a94-afe0-c9bb-4377-accabf73696e@oracle.com> <73dd06ec-544f-3cc9-00d5-c5c8c8ffdacc@oracle.com> Message-ID: > On Jun 12, 2018, at 1:43 AM, David Holmes wrote: > >> On 12/06/2018 3:31 PM, mandy chung wrote: >> On 6/11/18 10:12 PM, David Holmes wrote: >>>>>>> >>>>>>> For the Java code ... methods that return arrays should return zero-length arrays when something is not available rather than null. >>>>>> All methods do return zero length arrays except I missed the getPerCpuUsage. I?ll fix that one and correct the javadoc. >>>>> >>>>> There are a few more too: >>>>> >>>> >>>> Those are covered by the function that converts the string range. >>> >>> ??? I have no idea what you mean. >> I think the methods returning an array calls Subsystem::StringRangeToIntArray which returns an empty array. >> 171 public static int[] StringRangeToIntArray(String range) { >> 172 int[] ints = new int[0]; >> 173 >> 174 if (range == null) return ints; > > I'm commenting on the specification of the Metrics interface: > > http://cr.openjdk.java.net/~bobv/8203357/webrev.01/src/java.base/share/classes/jdk/internal/platform/Metrics.java.html > > not any implementation. Oh. I previously mentioned that I needed to correct the javadoc comments. I had corrected the implementation but hadn?t fixed the spec. Bob. > > Cheers, > David > >> Mandy From shade at redhat.com Tue Jun 12 13:20:11 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 12 Jun 2018 15:20:11 +0200 Subject: RFR: 8204180: Implementation: JEP 318: Epsilon GC (round 5) In-Reply-To: <5e4c692e-c638-1ec2-e9d3-faf613bcbf18@oracle.com> References: <2ee03550-2b1e-b873-5ef3-26312e2a760e@redhat.com> <7211f820-4999-2988-e06c-1067b1fb138f@oracle.com> <0e63a401-d28d-2bdb-d024-3b48f3fba0cb@oracle.com> <2eab1a10-121f-884d-0155-5a422f2a3268@redhat.com> <5e4c692e-c638-1ec2-e9d3-faf613bcbf18@oracle.com> Message-ID: <9ebadbff-2461-a77c-3649-de8426e21adf@redhat.com> On 06/12/2018 11:17 AM, Per Liden wrote: > On 06/12/2018 10:48 AM, Aleksey Shipilev wrote: >> On 06/07/2018 02:42 PM, Per Liden wrote: >>> On 06/07/2018 02:38 PM, Aleksey Shipilev wrote: >>>> On 06/07/2018 12:13 PM, Per Liden wrote: >>>>> On 06/06/2018 06:30 PM, Aleksey Shipilev wrote: >>>>>> Hi, >>>>>> >>>>>> This is fifth (and hopefully final) round of code review for Epsilon GC changes. It includes the >>>>>> fixes done as the result of fourth round of reviews, mostly in Serviceability. The build parts >>>>>> are >>>>>> the same since last few reviews, so this is not posted to build-dev at . >>>>>> >>>>>> Webrev: >>>>>> ???? http://cr.openjdk.java.net/~shade/epsilon/webrev.09/ >>>>> >>>>> Still looks good to me! >>>>> >> >> I have two follow ups after jdk-submit Windows and Solaris build/test failures: >> ?? http://hg.openjdk.java.net/jdk/sandbox/rev/24339b23a56b >> ?? http://hg.openjdk.java.net/jdk/sandbox/rev/f21420b61fd2 > > Looks good to me! Integrated: https://bugs.openjdk.java.net/browse/JDK-8204180 http://hg.openjdk.java.net/jdk/jdk/rev/7b7c75d87f9b Let's see if it breaks anything not captured by our pre-integration testing. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From erik.helin at oracle.com Tue Jun 12 13:48:44 2018 From: erik.helin at oracle.com (Erik Helin) Date: Tue, 12 Jun 2018 15:48:44 +0200 Subject: FW: RFR (S): 8195115: G1 Old Gen MemoryPool CollectionUsage.used values don't reflect mixed GC results In-Reply-To: <815C8166-AE5B-40EC-B7B6-1DF9946C715D@amazon.com> References: <3A67BF1B-CE35-4759-B5B4-959C24020A45@amazon.com> <997a766f-9f4b-f192-ae87-03f4c8d2666d@oracle.com> <64793ff3-4805-79f9-4e72-6e469cd511dc@oracle.com> <15754d34-f3c7-6431-7abf-05214bd6b9d2@oracle.com> <93417506-8e12-6ad3-690a-439e36719652@oracle.com> <99C937B6-F67B-4F4A-ACC8-444B5F83932B@amazon.com> <3922628B-81DE-4904-AEFE-F4163EB1E655@amazon.com> <8444d766-8adb-013f-f75b-b5d4420635df@oracle.com> <5768E4BF-5556-455B-899B-94A67BF940A0@amazon.com> <3E15AAD8-5017-4CCB-B927-6B31FD0D7809@amazon.com> <22216AD4-9A78-4427-9AB8-629EC685C296@amazon.com> <9f972398-115f-06ad-1e0c-513abceb097a@oracle.com> <48344C2E-2423-4CB1-A5C3-09223CA1ED78@amazon.com> <815C8166-AE5B-40EC-B7B6-1DF9946C715D@amazon.com> Message-ID: (adding back serviceability-dev, please keep both hotspot-gc-dev and serviceability-dev) Hi Paul, before I start re-reviewing, did you test the new version of the patch via the jdk/submit repository [0]? Thanks, Erik [0]: http://hg.openjdk.java.net/jdk/submit On 06/09/2018 03:29 PM, Hohensee, Paul wrote: > Didn't seem to make it to hotspot-gc-dev... > > ?On 6/8/18, 10:14 AM, "serviceability-dev on behalf of Hohensee, Paul" wrote: > > Back after a long hiatus... > > Thanks, Eric, for your review. Here's a new webrev incorporating your recommendations. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8195115 > Webrev: http://cr.openjdk.java.net/~phh/8195115/webrev.02/ > > TIA for your re-review. Plus, may I have another reviewer look at it please? > > Paul > > On 2/26/18, 8:47 AM, "Erik Helin" wrote: > > Hi Paul, > > a couple of comments on the patch: > > - memoryService.hpp: > + 150 bool countCollection, > + 151 bool allMemoryPoolsAffected = true); > > There is no need to use a default value for the parameter > allMemoryPoolsAffected here. Skipping the default value also allows > you to put allMemoryPoolsAffected to TraceMemoryManager::initialize > in the same relative position as for the constructor parameter (this > will make the code more uniform and easier to follow). > > - memoryManager.cpp > > Instead of adding a default parameter, maybe add a new method? > Something like GCMemoryManager::add_not_always_affected_pool() > (I couldn't come up with a shorter name at the moment). > > - TestMixedOldGenCollectionUsage.java > > The test is too strict about how and when collections should > occur. Tests written this way often become very brittle, they might > e.g. fail to finish a concurrent mark on time on a very slow, single > core, machine. It is better to either force collections by using the > WhiteBox API or make the test more lenient. > > Thanks, > Erik > > On 02/22/2018 09:54 PM, Hohensee, Paul wrote: > > Ping for a review please. > > > > Thanks, > > > > Paul > > > > On 2/16/18, 12:26 PM, "serviceability-dev on behalf of Hohensee, Paul" wrote: > > > > The CSR https://bugs.openjdk.java.net/browse/JDK-8196719 for the original fix has been approved, so I?m back to requesting a code review, please. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8195115 > > Webrev: http://cr.openjdk.java.net/~phh/8195115/webrev.hs.01/ > > > > Passed a submit repo run, passes its jtreg test, and a JDK8 version is in production use at Amazon. > > > > From the original RR: > > > > > The bug is that from the JMX point of view, G1?s incremental collector > > > (misnamed as the ?G1 Young Generation? collector) only affects G1?s > > > survivor and eden spaces. In fact, mixed collections run by this > > > collector also affect the G1 old generation. > > > > > > This proposed fix is to record, for each of a JMX garbage collector's > > > memory pools, whether that memory pool is affected by all collections > > > using that collector. And, for each collection, record whether or not > > > all the collector's memory pools are affected. After each collection, > > > for each memory pool, if either all the collector's memory pools were > > > affected or the memory pool is affected for all collections, record > > > CollectionUsage for that pool. > > > > > > For collectors other than G1 Young Generation, all pools are recorded as > > > affected by all collections and every collection is recorded as > > > affecting all the collector?s memory pools. For the G1 Young Generation > > > collector, the G1 Old Gen pool is recorded as not being affected by all > > > collections, and non-mixed collections are recorded as not affecting all > > > memory pools. The result is that for non-mixed collections, > > > CollectionUsage is recorded after a collection only the G1 Eden Space > > > and G1 Survivor Space pools, while for mixed collections CollectionUsage > > > is recorded for G1 Old Gen as well. > > > > > > Other than the effect of the fix on G1 Old Gen MemoryPool. > > > CollectionUsage, the only external behavior change is that > > > GarbageCollectorMXBean.getMemoryPoolNames will now return 3 pool names > > > rather than 2. > > > > > > With this fix, a collector?s memory pools can be divided into two > > > disjoint subsets, one that participates in all collections and one that > > > doesn?t. This is a bit more general than the minimum necessary to fix > > > G1, but not by much. Because I expect it to apply to other incremental > > > region-based collectors, I went with the more general solution. I > > > minimized the amount of code I had to touch by using default parameters > > > for GCMemoryManager::add_pool and the TraceMemoryManagerStats constructors. > > > > > > > > > > From thomas.stuefe at gmail.com Tue Jun 12 15:39:30 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 12 Jun 2018 17:39:30 +0200 Subject: RFR(s): 8203343: VM.{metaspace|classloaders|classhierarchy...} jcmd should show invocation targets for Generated{Method|Constructor}AccessorImpl classes In-Reply-To: References: Message-ID: Could I have a second review, please? Thanks a lot, Thomas On Tue, Jun 12, 2018 at 7:36 AM, Thomas St?fe wrote: > Ping. > > jdk-submit tests came back clean. > > Thanks, Thomas > > On Wed, Jun 6, 2018 at 6:05 PM, Thomas St?fe wrote: >> Dear all, >> >> may I please have feedback and if possible reviews for this small addition: >> >> CR: https://bugs.openjdk.java.net/browse/JDK-8203343 >> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/webrev.00/webrev/ >> >> (Note: this patch goes atop of >> https://bugs.openjdk.java.net/browse/JDK-8203682, which is currently >> up for RFR). >> >> --- >> >> When analyzing situations involving a lot of reflection, one often >> stares at walls of "GeneratedXXXAccessorXXX" classes. These names are >> generated and not at all helpful in analyzing the problem (e.g. which >> component in a server node does this much reflection and hence uses so >> much metaspace). >> >> This patch adds the ability to print out invocation targets >> additionally to class names if the class is a generated accessor - for >> now this ability has been added to VM.metaspace, VM.classloaders and >> VM.class_hierarchy. >> >> Example output from "VM.class_hierarchy": >> >> >> |--jdk.internal.reflect.MagicAccessorImpl/null >> >> | |--jdk.internal.reflect.ConstructorAccessorImpl/null >> | | |--jdk.internal.reflect.GeneratedConstructorAccessor18/0x00007f9ee8350c10 >> (invokes: org/springframework/boot/context/properties/ConfigurationPropertiesBindingPostProcessorRegistrar:: >> ()V) >> | | |--jdk.internal.reflect.GeneratedConstructorAccessor17/0x00007f9ee8349c00 >> (invokes: org/springframework/boot/context/properties/EnableConfigurationPropertiesImportSelector$ConfigurationPropertiesBeanRegistrar:: >> ()V) >> >> | |--jdk.internal.reflect.MethodAccessorImpl/null >> | | |--jdk.internal.reflect.GeneratedMethodAccessor23/0x00007f9ec8329b60 >> (invokes: org/apache/tomcat/util/modeler/AttributeInfo::setIs (Z)V) >> | | |--jdk.internal.reflect.GeneratedMethodAccessor22/0x00007f9ee831bc70 >> (invokes: org/springframework/boot/autoconfigure/SpringBootApplication::exclude >> ()[Ljava/lang/Class;) >> >> >> See here more examples: >> >> "VM.class_hierarchy" >> http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/example-VM.class_hierarchy.txt >> >> "VM.classloaders show-classes" >> http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/example-VM.classloaders.txt >> >> >> ---- >> >> Note: I am not sure if this is a fit for hotspot-runtime or >> serviceability. Sorry for crossposting. >> >> Thank you, >> >> Thomas From daniil.x.titov at oracle.com Tue Jun 12 17:57:50 2018 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Tue, 12 Jun 2018 10:57:50 -0700 Subject: RFR 8203809: [Graal] JDI tests fail with: JDITestRuntimeException : ** event IS NOT a breakpoint ** In-Reply-To: <36f11b76-ce82-bdff-e763-4d3d73208119@oracle.com> References: <36f11b76-ce82-bdff-e763-4d3d73208119@oracle.com> Message-ID: <2EDA1CDD-6872-4C5B-9262-B60E59FA0A61@oracle.com> Hi Serguei, There are 4 more tests that use EventFilters.filter() method and not all of them need to be changed to call EventFilters .filtered(Event event, String typeName). Specifically, vmTestbase/nsk/jdi/ExceptionEvent/exception/exception001/TestDescription.java and vmTestbase/nsk/jdi/ExceptionEvent/catchLocation/location001/TestDescription.java tests expect to receive an ExceptionEvent for NumberFormatException exception and the location of this event happens to be not the debuggee class ( java.lang.Integer:652). The tests pass now and changing them to use EventFilters .filtered(event, debugeeName) makes them fail. There is also an alternative approach I would like to suggest. All these new Graal events are generated by the compiler threads (?JVMCI CompilerThread0, JVMCI CompilerThread1, etc.?) and could be filtered out inside EventFilters .filtered(Event event) method by checking the thread reference for the given event object. Please review the updated fix the uses this approach. Webrev: http://cr.openjdk.java.net/~dtitov/8203809/webrev.02/ Bug: https://bugs.openjdk.java.net/browse/JDK-8203809 Thanks! Best regards, Daniil From: "serguei.spitsyn at oracle.com" Date: Monday, June 11, 2018 at 6:18 PM To: Daniil Titov , "serviceability-dev at openjdk.java.net serviceability-dev at openjdk.java.net" Subject: Re: RFR 8203809: [Graal] JDI tests fail with: JDITestRuntimeException : ** event IS NOT a breakpoint ** Hi Daniil, It looks good in general. Some minor comments though. http://cr.openjdk.java.net/~dtitov/8203809/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/share/jdi/EventFilters.java.udiff.html + +??? // Filters out events with location not matching the given type. +??? public static boolean filteredByLocation(Event event, String typeName) { +??????? if (event instanceof Locatable) { +??????????? Location location = ((Locatable) event).location(); +??????????? if (location != null) { +??????????????? ReferenceType declaringType = location.declaringType(); +??????????????? if (declaringType != null && typeName.equals(declaringType.name())) { +??????????????????? return false; +??????????????? } +??????????? } +??????? } +??????? return true; +??? } + +??? // Filters out internal JFR events and events with location not matching +??? // the given type. +??? public static boolean filtered(Event event, String typeName) { + +??????? return filtered(event) || filteredByLocation(event,typeName); + +??? } + + There are a couple of extra empty lines. I'm suggesting to get rid of the second method and rename filteredByLocation() to filtered(): +??? // Filters out events with location not matching the given type. +??? public static boolean filtered(Event event, String typeName) { +??????? if (event instanceof Locatable) { +??????????? Location location = ((Locatable) event).location(); +??????????? if (location != null) { +??????????????? ReferenceType declaringType = location.declaringType(); +??????????????? if (declaringType != null && typeName.equals(declaringType.name())) { +??????????????????? return false; +??????????????? } +??????????? } +??????? } +??????? return true; +??? } As I understand, it should filter out the events post on the JFR threads. Please, correct me if I'm wrong. My view is that we may want to get rid of the filtered(Event event) method and convert all the tests to use the filtered(Event event, String typeName) instead. Thank you a lot for taking care about these issues! Thanks, Serguei On 6/11/18 17:52, Daniil Titov wrote: Please review the changes that fix failure of 6 JDI tests run with Graal. The problem here is that currently the tests filter out only 2 specific JFR events (nsk.share.jdi.EventsFilter.filtered()) before analyzing them and fail if the received event doesn't match the expected one.? But with Graal turned on new events generated by the compiler thread are occasionally found in the event queue and cause the tests failure. The fix makes the tests to use an additional check to filter out events not related to the debuggee class. Bug: https://bugs.openjdk.java.net/browse/JDK-8203809 Webrev: http://cr.openjdk.java.net/~dtitov/8203809/webrev.01 Thanks, Daniil -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Tue Jun 12 18:26:10 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 12 Jun 2018 11:26:10 -0700 Subject: RFR 8203809: [Graal] JDI tests fail with: JDITestRuntimeException : ** event IS NOT a breakpoint ** In-Reply-To: <36f11b76-ce82-bdff-e763-4d3d73208119@oracle.com> References: <36f11b76-ce82-bdff-e763-4d3d73208119@oracle.com> Message-ID: <7f4201dc-12c3-5885-0aed-617e3dfacd6e@oracle.com> An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Tue Jun 12 18:43:33 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 12 Jun 2018 11:43:33 -0700 Subject: RFR 8203809: [Graal] JDI tests fail with: JDITestRuntimeException : ** event IS NOT a breakpoint ** In-Reply-To: <2EDA1CDD-6872-4C5B-9262-B60E59FA0A61@oracle.com> References: <36f11b76-ce82-bdff-e763-4d3d73208119@oracle.com> <2EDA1CDD-6872-4C5B-9262-B60E59FA0A61@oracle.com> Message-ID: <2ff56571-55e5-8a22-6bcd-56662e806c04@oracle.com> An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Tue Jun 12 18:53:38 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 12 Jun 2018 11:53:38 -0700 Subject: RFR 8203809: [Graal] JDI tests fail with: JDITestRuntimeException : ** event IS NOT a breakpoint ** In-Reply-To: <2EDA1CDD-6872-4C5B-9262-B60E59FA0A61@oracle.com> References: <36f11b76-ce82-bdff-e763-4d3d73208119@oracle.com> <2EDA1CDD-6872-4C5B-9262-B60E59FA0A61@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Tue Jun 12 19:00:24 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 12 Jun 2018 12:00:24 -0700 Subject: RFR 8203809: [Graal] JDI tests fail with: JDITestRuntimeException : ** event IS NOT a breakpoint ** In-Reply-To: References: <36f11b76-ce82-bdff-e763-4d3d73208119@oracle.com> <2EDA1CDD-6872-4C5B-9262-B60E59FA0A61@oracle.com> Message-ID: <8ce917b5-c918-a43d-8adc-e574d2eea97b@oracle.com> An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Tue Jun 12 19:02:52 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 12 Jun 2018 12:02:52 -0700 Subject: RFR 8203809: [Graal] JDI tests fail with: JDITestRuntimeException : ** event IS NOT a breakpoint ** In-Reply-To: References: <36f11b76-ce82-bdff-e763-4d3d73208119@oracle.com> <2EDA1CDD-6872-4C5B-9262-B60E59FA0A61@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Tue Jun 12 19:21:26 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 12 Jun 2018 12:21:26 -0700 Subject: RFR 8203809: [Graal] JDI tests fail with: JDITestRuntimeException : ** event IS NOT a breakpoint ** In-Reply-To: References: <36f11b76-ce82-bdff-e763-4d3d73208119@oracle.com> <2EDA1CDD-6872-4C5B-9262-B60E59FA0A61@oracle.com> Message-ID: <098f5218-b1c5-651e-b35c-29a34b669596@oracle.com> An HTML attachment was scrubbed... URL: From daniil.x.titov at oracle.com Tue Jun 12 19:30:07 2018 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Tue, 12 Jun 2018 12:30:07 -0700 Subject: RFR 8203809: [Graal] JDI tests fail with: JDITestRuntimeException : ** event IS NOT a breakpoint ** In-Reply-To: <098f5218-b1c5-651e-b35c-29a34b669596@oracle.com> References: <36f11b76-ce82-bdff-e763-4d3d73208119@oracle.com> <2EDA1CDD-6872-4C5B-9262-B60E59FA0A61@oracle.com> <098f5218-b1c5-651e-b35c-29a34b669596@oracle.com> Message-ID: <9209AA31-1A35-4818-B2B6-571D3C9268CF@oracle.com> Hi Serguei and Chris, As I mentioned in the previous emails there are only 4 other tests that calls EventFiliters.filtered(event) and only 2 of them need to be changed to use? EventFiliters.filtered(event, debuggeName). So if the approach in the first webrev http://cr.openjdk.java.net/~dtitov/8203809/webrev.02/ is preferable we could stick with it and I will just update this patch to add 2 more tests and combine EventFiliters.filtered(event, typeName) and EventFiliters.filteredByLocation(event, typeName) in the single method as Serguei suggested. ? Thanks! Best regards, Daniil From: "serguei.spitsyn at oracle.com" Date: Tuesday, June 12, 2018 at 12:21 PM To: Chris Plummer , Daniil Titov , "serviceability-dev at openjdk.java.net serviceability-dev at openjdk.java.net" Subject: Re: RFR 8203809: [Graal] JDI tests fail with: JDITestRuntimeException : ** event IS NOT a breakpoint ** On 6/12/18 12:02, Chris Plummer wrote: On 6/12/18 11:53 AM, serguei.spitsyn at oracle.com wrote: Hi Daniil, I don't think, the approach for the tests to filter out the JFR and Graal threads is not good despite the fact it is relatively easy to implement. It does not cover any threads that may come in the future. So that I consider the webrev below as just a work around of the real problem. True, but his changes are just an extension of what is already being done - filtering out threads we don't care about. I understand it. The tests should just filter out everything that is not a part of their testing. It can be done on a base of the Debuggee classes or threads, or any other approaches. One problem is that events don't always come in on debuggee thread, so you can't filter out all threads other than the debuggee thread. I think probably filtering out all classes not related to the debuggee is probably the way to go. However, do we want to fix all the tests to do this, or just go with Daniil's quick fix? In general, I'm Okay with this quick workaround fix. But wanted to underline that we still have a base for problems in the future. Thanks, Serguei Chris Thanks, Serguei On 6/12/18 10:57, Daniil Titov wrote: Hi Serguei, There are 4 more tests that use EventFilters.filter() method and not all of them need to be changed to call EventFilters .filtered(Event event, String typeName). Specifically, vmTestbase/nsk/jdi/ExceptionEvent/exception/exception001/TestDescription.java and vmTestbase/nsk/jdi/ExceptionEvent/catchLocation/location001/TestDescription.java tests expect to receive an ExceptionEvent for NumberFormatException exception and the location of this event happens to be not the debuggee class ( java.lang.Integer:652). The tests pass now and changing them to use EventFilters .filtered(event, debugeeName) makes them fail. There is also an alternative approach I would like to suggest. All these new Graal events are generated by the compiler threads (?JVMCI CompilerThread0, JVMCI CompilerThread1, etc.?) and could be filtered out inside EventFilters .filtered(Event event) method by checking the thread reference for the given event object. Please review the updated fix the uses this approach. Webrev: http://cr.openjdk.java.net/~dtitov/8203809/webrev.02/ Bug: https://bugs.openjdk.java.net/browse/JDK-8203809 Thanks! Best regards, Daniil From: "serguei.spitsyn at oracle.com" Date: Monday, June 11, 2018 at 6:18 PM To: Daniil Titov , "serviceability-dev at openjdk.java.net serviceability-dev at openjdk.java.net" Subject: Re: RFR 8203809: [Graal] JDI tests fail with: JDITestRuntimeException : ** event IS NOT a breakpoint ** Hi Daniil, It looks good in general. Some minor comments though. http://cr.openjdk.java.net/~dtitov/8203809/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/share/jdi/EventFilters.java.udiff.html + + // Filters out events with location not matching the given type. + public static boolean filteredByLocation(Event event, String typeName) { + if (event instanceof Locatable) { + Location location = ((Locatable) event).location(); + if (location != null) { + ReferenceType declaringType = location.declaringType(); + if (declaringType != null && typeName.equals(declaringType.name())) { + return false; + } + } + } + return true; + } + + // Filters out internal JFR events and events with location not matching + // the given type. + public static boolean filtered(Event event, String typeName) { + + return filtered(event) || filteredByLocation(event,typeName); + + } + + There are a couple of extra empty lines. I'm suggesting to get rid of the second method and rename filteredByLocation() to filtered(): + // Filters out events with location not matching the given type. + public static boolean filtered(Event event, String typeName) { + if (event instanceof Locatable) { + Location location = ((Locatable) event).location(); + if (location != null) { + ReferenceType declaringType = location.declaringType(); + if (declaringType != null && typeName.equals(declaringType.name())) { + return false; + } + } + } + return true; + } As I understand, it should filter out the events post on the JFR threads. Please, correct me if I'm wrong. My view is that we may want to get rid of the filtered(Event event) method and convert all the tests to use the filtered(Event event, String typeName) instead. Thank you a lot for taking care about these issues! Thanks, Serguei On 6/11/18 17:52, Daniil Titov wrote: Please review the changes that fix failure of 6 JDI tests run with Graal. The problem here is that currently the tests filter out only 2 specific JFR events (nsk.share.jdi.EventsFilter.filtered()) before analyzing them and fail if the received event doesn't match the expected one. But with Graal turned on new events generated by the compiler thread are occasionally found in the event queue and cause the tests failure. The fix makes the tests to use an additional check to filter out events not related to the debuggee class. Bug: https://bugs.openjdk.java.net/browse/JDK-8203809 Webrev: http://cr.openjdk.java.net/~dtitov/8203809/webrev.01 Thanks, Daniil -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Tue Jun 12 20:02:07 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 12 Jun 2018 13:02:07 -0700 Subject: RFR 8203809: [Graal] JDI tests fail with: JDITestRuntimeException : ** event IS NOT a breakpoint ** In-Reply-To: <2EDA1CDD-6872-4C5B-9262-B60E59FA0A61@oracle.com> References: <36f11b76-ce82-bdff-e763-4d3d73208119@oracle.com> <2EDA1CDD-6872-4C5B-9262-B60E59FA0A61@oracle.com> Message-ID: <9574502d-ca3b-45f6-3e82-63e7e85dae48@oracle.com> An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Tue Jun 12 20:08:06 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 12 Jun 2018 13:08:06 -0700 Subject: RFR 8203809: [Graal] JDI tests fail with: JDITestRuntimeException : ** event IS NOT a breakpoint ** In-Reply-To: <9209AA31-1A35-4818-B2B6-571D3C9268CF@oracle.com> References: <36f11b76-ce82-bdff-e763-4d3d73208119@oracle.com> <2EDA1CDD-6872-4C5B-9262-B60E59FA0A61@oracle.com> <098f5218-b1c5-651e-b35c-29a34b669596@oracle.com> <9209AA31-1A35-4818-B2B6-571D3C9268CF@oracle.com> Message-ID: <05726b58-3522-28af-a1e8-27a088380566@oracle.com> An HTML attachment was scrubbed... URL: From daniil.x.titov at oracle.com Tue Jun 12 20:14:24 2018 From: daniil.x.titov at oracle.com (daniil.x.titov at oracle.com) Date: Tue, 12 Jun 2018 13:14:24 -0700 Subject: RFR 8203809: [Graal] JDI tests fail with: JDITestRuntimeException : ** event IS NOT a breakpoint ** In-Reply-To: <05726b58-3522-28af-a1e8-27a088380566@oracle.com> References: <36f11b76-ce82-bdff-e763-4d3d73208119@oracle.com> <2EDA1CDD-6872-4C5B-9262-B60E59FA0A61@oracle.com> <098f5218-b1c5-651e-b35c-29a34b669596@oracle.com> <9209AA31-1A35-4818-B2B6-571D3C9268CF@oracle.com> <05726b58-3522-28af-a1e8-27a088380566@oracle.com> Message-ID: <84af977a-b88c-2c39-7a5b-4194ecebee07@oracle.com> Hi Serguei, On 6/12/18 1:08 PM, serguei.spitsyn at oracle.com wrote: > Hi Daniil, > > > On 6/12/18 12:30, Daniil Titov wrote: >> >> Hi Serguei and Chris, >> >> As I mentioned in the previous emails there are only 4 other tests >> that calls EventFiliters.filtered(event) and only 2 of them need to >> be changed to use? EventFiliters.filtered(event, debuggeName). >> >> So if the approach in the first webrev >> http://cr.openjdk.java.net/~dtitov/8203809/webrev.02/ >> is preferable >> > > I guess, it is just a typo above. > It is not the first approach but the second. > It is a typo. The proper link is http://cr.openjdk.java.net/~dtitov/8203809/webrev.01/ > >> we could stick with it and I will just update this patch to add 2 >> more tests and combine EventFiliters.filtered(event, typeName) and >> EventFiliters.filteredByLocation(event, typeName) in the single >> method as Serguei suggested. >> > > Yes, my preference is this approach. > It adds some ground for future improvements as well. > It is also possible to use this method for outer classes being > debugged like this: > filtered(event, "java.lang.Integer") > > Thanks, > Serguei > Agree. Will prepare an updated webrev soon. Thanks! >> Thanks! >> >> Best regards, >> >> Daniil >> >> *From: *"serguei.spitsyn at oracle.com" >> *Date: *Tuesday, June 12, 2018 at 12:21 PM >> *To: *Chris Plummer , Daniil Titov >> , "serviceability-dev at openjdk.java.net >> serviceability-dev at openjdk.java.net" >> >> *Subject: *Re: RFR 8203809: [Graal] JDI tests fail with: >> JDITestRuntimeException : ** event IS NOT a breakpoint ** >> >> >> >> On 6/12/18 12:02, Chris Plummer wrote: >> >> On 6/12/18 11:53 AM, serguei.spitsyn at oracle.com >> wrote: >> >> Hi Daniil, >> >> I don't think, the approach for the tests to filter out the >> JFR and Graal threads >> is not good despite the fact it is relatively easy to implement. >> It does not cover any threads that may come in the future. >> So that I consider the webrev below as just a work around of >> the real problem. >> >> True, but his changes are just an extension of what is already >> being done - filtering out threads we don't care about. >> >> >> I understand it. >> >> >> The tests should just filter out everything that is not a >> part of their testing. >> It can be done on a base of the Debuggee classes or threads, >> or any other approaches. >> >> One problem is that events don't always come in on debuggee >> thread, so you can't filter out all threads other than the >> debuggee thread. I think probably filtering out all classes not >> related to the debuggee is probably the way to go. However, do we >> want to fix all the tests to do this, or just go with Daniil's >> quick fix? >> >> >> In general, I'm Okay with this quick workaround fix. >> But wanted to underline that we still have a base for problems in the >> future. >> >> Thanks, >> Serguei >> >> >> >> Chris >> >> >> Thanks, >> Serguei >> >> >> On 6/12/18 10:57, Daniil Titov wrote: >> >> Hi Serguei, >> >> There are 4 more tests that use EventFilters.filter() >> method and not all of them need to be changed to call >> EventFilters.filtered(Event event, String typeName). >> Specifically, >> vmTestbase/nsk/jdi/ExceptionEvent/exception/exception001/TestDescription.java >> and >> vmTestbase/nsk/jdi/ExceptionEvent/catchLocation/location001/TestDescription.java >> tests expect to receive an ExceptionEvent for >> NumberFormatException exception and the location of this >> event happens to be not the debuggee class ( >> java.lang.Integer:652). The tests pass now and changing >> them to use EventFilters.filtered(event, debugeeName) >> makes them fail. >> >> There is also an alternative approach I would like to >> suggest. All these new Graal events are generated by the >> compiler threads (?JVMCI CompilerThread0, JVMCI >> CompilerThread1, etc.?) and could be filtered out inside >> EventFilters.filtered(Event event) method by checking the >> thread reference for the given event object. >> >> Please review the updated fix the uses this approach. >> >> Webrev: >> http://cr.openjdk.java.net/~dtitov/8203809/webrev.02/ >> >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8203809 >> >> Thanks! >> >> Best regards, >> >> Daniil >> >> *From: *"serguei.spitsyn at oracle.com" >> >> >> >> *Date: *Monday, June 11, 2018 at 6:18 PM >> *To: *Daniil Titov >> , >> "serviceability-dev at openjdk.java.net >> serviceability-dev at openjdk.java.net" >> >> >> >> *Subject: *Re: RFR 8203809: [Graal] JDI tests fail with: >> JDITestRuntimeException : ** event IS NOT a breakpoint ** >> >> Hi Daniil, >> >> It looks good in general. >> Some minor comments though. >> >> http://cr.openjdk.java.net/~dtitov/8203809/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/share/jdi/EventFilters.java.udiff.html >> >> >> + >> >> +??? // Filters out events with location not matching the >> given type. >> >> +??? public static boolean filteredByLocation(Event >> event, String typeName) { >> >> +??????? if (event instanceof Locatable) { >> >> +??????????? Location location = ((Locatable) >> event).location(); >> >> +??????????? if (location != null) { >> >> +??????????????? ReferenceType declaringType = >> location.declaringType(); >> >> +??????????????? if (declaringType != null && >> typeName.equals(declaringType.name())) { >> >> +??????????????????? return false; >> >> +??????????????? } >> >> +??????????? } >> >> +??????? } >> >> +??????? return true; >> >> +??? } >> >> + >> >> +??? // Filters out internal JFR events and events with >> location not matching >> >> +??? // the given type. >> >> +??? public static boolean filtered(Event event, String >> typeName) { >> >> + >> >> +??????? return filtered(event) || >> filteredByLocation(event,typeName); >> >> + >> >> +??? } >> >> + >> >> + >> >> There are a couple of extra empty lines. >> >> I'm suggesting to get rid of the second method and rename >> filteredByLocation() to filtered(): >> >> +??? // Filters out events with location not matching the >> given type. >> >> +??? public static boolean filtered(Event event, String >> typeName) { >> >> +??????? if (event instanceof Locatable) { >> >> +??????????? Location location = ((Locatable) >> event).location(); >> >> +??????????? if (location != null) { >> >> +??????????????? ReferenceType declaringType = >> location.declaringType(); >> >> +??????????????? if (declaringType != null && >> typeName.equals(declaringType.name())) { >> >> +??????????????????? return false; >> >> +??????????????? } >> >> +??????????? } >> >> +??????? } >> >> +??????? return true; >> >> +??? } >> >> >> As I understand, it should filter out the events post on >> the JFR threads. >> Please, correct me if I'm wrong. >> My view is that we may want to get rid of the >> filtered(Event event) method and >> convert all the tests to use the filtered(Event event, >> String typeName) instead. >> >> Thank you a lot for taking care about these issues! >> >> Thanks, >> Serguei >> >> >> On 6/11/18 17:52, Daniil Titov wrote: >> >> Please review the changes that fix failure of 6 JDI tests run with Graal. >> >> >> >> The problem here is that currently the tests filter out only 2 specific JFR events (nsk.share.jdi.EventsFilter.filtered()) before analyzing them and fail if the received event doesn't match the expected one.? But with Graal turned on new events generated by the compiler thread are occasionally found in the event queue and cause the tests failure. The fix makes the tests to use an additional check to filter out events not related to the debuggee class. >> >> >> >> Bug:https://bugs.openjdk.java.net/browse/JDK-8203809 >> >> Webrev:http://cr.openjdk.java.net/~dtitov/8203809/webrev.01 >> >> >> >> >> Thanks, >> >> Daniil >> >> >> >> >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Tue Jun 12 21:01:02 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 12 Jun 2018 14:01:02 -0700 Subject: RFR(s): 8203343: VM.{metaspace|classloaders|classhierarchy...} jcmd should show invocation targets for Generated{Method|Constructor}AccessorImpl classes In-Reply-To: References: Message-ID: <5a79df03-9001-bff8-7a45-edfb0284c9f1@oracle.com> Hi Thomas, It looks good to me. But I'm not an expert in the area of Generated Accessors. How was this tested? Does it make sense to add a unit test for this? Thanks, Serguei On 6/6/18 09:05, Thomas St?fe wrote: > Dear all, > > may I please have feedback and if possible reviews for this small addition: > > CR: https://bugs.openjdk.java.net/browse/JDK-8203343 > Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/webrev.00/webrev/ > > (Note: this patch goes atop of > https://bugs.openjdk.java.net/browse/JDK-8203682, which is currently > up for RFR). > > --- > > When analyzing situations involving a lot of reflection, one often > stares at walls of "GeneratedXXXAccessorXXX" classes. These names are > generated and not at all helpful in analyzing the problem (e.g. which > component in a server node does this much reflection and hence uses so > much metaspace). > > This patch adds the ability to print out invocation targets > additionally to class names if the class is a generated accessor - for > now this ability has been added to VM.metaspace, VM.classloaders and > VM.class_hierarchy. > > Example output from "VM.class_hierarchy": > > > |--jdk.internal.reflect.MagicAccessorImpl/null > > | |--jdk.internal.reflect.ConstructorAccessorImpl/null > | | |--jdk.internal.reflect.GeneratedConstructorAccessor18/0x00007f9ee8350c10 > (invokes: org/springframework/boot/context/properties/ConfigurationPropertiesBindingPostProcessorRegistrar:: > ()V) > | | |--jdk.internal.reflect.GeneratedConstructorAccessor17/0x00007f9ee8349c00 > (invokes: org/springframework/boot/context/properties/EnableConfigurationPropertiesImportSelector$ConfigurationPropertiesBeanRegistrar:: > ()V) > > | |--jdk.internal.reflect.MethodAccessorImpl/null > | | |--jdk.internal.reflect.GeneratedMethodAccessor23/0x00007f9ec8329b60 > (invokes: org/apache/tomcat/util/modeler/AttributeInfo::setIs (Z)V) > | | |--jdk.internal.reflect.GeneratedMethodAccessor22/0x00007f9ee831bc70 > (invokes: org/springframework/boot/autoconfigure/SpringBootApplication::exclude > ()[Ljava/lang/Class;) > > > See here more examples: > > "VM.class_hierarchy" > http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/example-VM.class_hierarchy.txt > > "VM.classloaders show-classes" > http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/example-VM.classloaders.txt > > > ---- > > Note: I am not sure if this is a fit for hotspot-runtime or > serviceability. Sorry for crossposting. > > Thank you, > > Thomas From daniil.x.titov at oracle.com Tue Jun 12 21:57:01 2018 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Tue, 12 Jun 2018 14:57:01 -0700 Subject: RFR 8203809: [Graal] JDI tests fail with: JDITestRuntimeException : ** event IS NOT a breakpoint ** In-Reply-To: <05726b58-3522-28af-a1e8-27a088380566@oracle.com> References: <36f11b76-ce82-bdff-e763-4d3d73208119@oracle.com> <2EDA1CDD-6872-4C5B-9262-B60E59FA0A61@oracle.com> <098f5218-b1c5-651e-b35c-29a34b669596@oracle.com> <9209AA31-1A35-4818-B2B6-571D3C9268CF@oracle.com> <05726b58-3522-28af-a1e8-27a088380566@oracle.com> Message-ID: Thank you, Serguei and Chris, Please review a new version of the fix. Webrev: http://cr.openjdk.java.net/~dtitov/8203809/webrev.03/ Bug: https://bugs.openjdk.java.net/browse/JDK-8203809 Thanks! Best regards, Daniil From: "serguei.spitsyn at oracle.com" Date: Tuesday, June 12, 2018 at 1:08 PM To: Daniil Titov , Chris Plummer , "serviceability-dev at openjdk.java.net serviceability-dev at openjdk.java.net" Subject: Re: RFR 8203809: [Graal] JDI tests fail with: JDITestRuntimeException : ** event IS NOT a breakpoint ** Hi Daniil, On 6/12/18 12:30, Daniil Titov wrote: Hi Serguei and Chris, As I mentioned in the previous emails there are only 4 other tests that calls EventFiliters.filtered(event) and only 2 of them need to be changed to use EventFiliters.filtered(event, debuggeName). So if the approach in the first webrev http://cr.openjdk.java.net/~dtitov/8203809/webrev.02/ is preferable I guess, it is just a typo above. It is not the first approach but the second. we could stick with it and I will just update this patch to add 2 more tests and combine EventFiliters.filtered(event, typeName) and EventFiliters.filteredByLocation(event, typeName) in the single method as Serguei suggested. Yes, my preference is this approach. It adds some ground for future improvements as well. It is also possible to use this method for outer classes being debugged like this: filtered(event, "java.lang.Integer") Thanks, Serguei Thanks! Best regards, Daniil From: "serguei.spitsyn at oracle.com" Date: Tuesday, June 12, 2018 at 12:21 PM To: Chris Plummer , Daniil Titov , "serviceability-dev at openjdk.java.net serviceability-dev at openjdk.java.net" Subject: Re: RFR 8203809: [Graal] JDI tests fail with: JDITestRuntimeException : ** event IS NOT a breakpoint ** On 6/12/18 12:02, Chris Plummer wrote: On 6/12/18 11:53 AM, serguei.spitsyn at oracle.com wrote: Hi Daniil, I don't think, the approach for the tests to filter out the JFR and Graal threads is not good despite the fact it is relatively easy to implement. It does not cover any threads that may come in the future. So that I consider the webrev below as just a work around of the real problem. True, but his changes are just an extension of what is already being done - filtering out threads we don't care about. I understand it. The tests should just filter out everything that is not a part of their testing. It can be done on a base of the Debuggee classes or threads, or any other approaches. One problem is that events don't always come in on debuggee thread, so you can't filter out all threads other than the debuggee thread. I think probably filtering out all classes not related to the debuggee is probably the way to go. However, do we want to fix all the tests to do this, or just go with Daniil's quick fix? In general, I'm Okay with this quick workaround fix. But wanted to underline that we still have a base for problems in the future. Thanks, Serguei Chris Thanks, Serguei On 6/12/18 10:57, Daniil Titov wrote: Hi Serguei, There are 4 more tests that use EventFilters.filter() method and not all of them need to be changed to call EventFilters .filtered(Event event, String typeName). Specifically, vmTestbase/nsk/jdi/ExceptionEvent/exception/exception001/TestDescription.java and vmTestbase/nsk/jdi/ExceptionEvent/catchLocation/location001/TestDescription.java tests expect to receive an ExceptionEvent for NumberFormatException exception and the location of this event happens to be not the debuggee class ( java.lang.Integer:652). The tests pass now and changing them to use EventFilters .filtered(event, debugeeName) makes them fail. There is also an alternative approach I would like to suggest. All these new Graal events are generated by the compiler threads (?JVMCI CompilerThread0, JVMCI CompilerThread1, etc.?) and could be filtered out inside EventFilters .filtered(Event event) method by checking the thread reference for the given event object. Please review the updated fix the uses this approach. Webrev: http://cr.openjdk.java.net/~dtitov/8203809/webrev.02/ Bug: https://bugs.openjdk.java.net/browse/JDK-8203809 Thanks! Best regards, Daniil From: "serguei.spitsyn at oracle.com" Date: Monday, June 11, 2018 at 6:18 PM To: Daniil Titov , "serviceability-dev at openjdk.java.net serviceability-dev at openjdk.java.net" Subject: Re: RFR 8203809: [Graal] JDI tests fail with: JDITestRuntimeException : ** event IS NOT a breakpoint ** Hi Daniil, It looks good in general. Some minor comments though. http://cr.openjdk.java.net/~dtitov/8203809/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/share/jdi/EventFilters.java.udiff.html + + // Filters out events with location not matching the given type. + public static boolean filteredByLocation(Event event, String typeName) { + if (event instanceof Locatable) { + Location location = ((Locatable) event).location(); + if (location != null) { + ReferenceType declaringType = location.declaringType(); + if (declaringType != null && typeName.equals(declaringType.name())) { + return false; + } + } + } + return true; + } + + // Filters out internal JFR events and events with location not matching + // the given type. + public static boolean filtered(Event event, String typeName) { + + return filtered(event) || filteredByLocation(event,typeName); + + } + + There are a couple of extra empty lines. I'm suggesting to get rid of the second method and rename filteredByLocation() to filtered(): + // Filters out events with location not matching the given type. + public static boolean filtered(Event event, String typeName) { + if (event instanceof Locatable) { + Location location = ((Locatable) event).location(); + if (location != null) { + ReferenceType declaringType = location.declaringType(); + if (declaringType != null && typeName.equals(declaringType.name())) { + return false; + } + } + } + return true; + } As I understand, it should filter out the events post on the JFR threads. Please, correct me if I'm wrong. My view is that we may want to get rid of the filtered(Event event) method and convert all the tests to use the filtered(Event event, String typeName) instead. Thank you a lot for taking care about these issues! Thanks, Serguei On 6/11/18 17:52, Daniil Titov wrote: Please review the changes that fix failure of 6 JDI tests run with Graal. The problem here is that currently the tests filter out only 2 specific JFR events (nsk.share.jdi.EventsFilter.filtered()) before analyzing them and fail if the received event doesn't match the expected one. But with Graal turned on new events generated by the compiler thread are occasionally found in the event queue and cause the tests failure. The fix makes the tests to use an additional check to filter out events not related to the debuggee class. Bug: https://bugs.openjdk.java.net/browse/JDK-8203809 Webrev: http://cr.openjdk.java.net/~dtitov/8203809/webrev.01 Thanks, Daniil -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Tue Jun 12 22:02:02 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 12 Jun 2018 15:02:02 -0700 Subject: RFR 8203809: [Graal] JDI tests fail with: JDITestRuntimeException : ** event IS NOT a breakpoint ** In-Reply-To: References: <36f11b76-ce82-bdff-e763-4d3d73208119@oracle.com> <2EDA1CDD-6872-4C5B-9262-B60E59FA0A61@oracle.com> <098f5218-b1c5-651e-b35c-29a34b669596@oracle.com> <9209AA31-1A35-4818-B2B6-571D3C9268CF@oracle.com> <05726b58-3522-28af-a1e8-27a088380566@oracle.com> Message-ID: <283b889a-3f2d-7537-24c2-14bf1d3b1274@oracle.com> An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Tue Jun 12 22:30:59 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 12 Jun 2018 15:30:59 -0700 Subject: RFR 8203809: [Graal] JDI tests fail with: JDITestRuntimeException : ** event IS NOT a breakpoint ** In-Reply-To: <283b889a-3f2d-7537-24c2-14bf1d3b1274@oracle.com> References: <36f11b76-ce82-bdff-e763-4d3d73208119@oracle.com> <2EDA1CDD-6872-4C5B-9262-B60E59FA0A61@oracle.com> <098f5218-b1c5-651e-b35c-29a34b669596@oracle.com> <9209AA31-1A35-4818-B2B6-571D3C9268CF@oracle.com> <05726b58-3522-28af-a1e8-27a088380566@oracle.com> <283b889a-3f2d-7537-24c2-14bf1d3b1274@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From christoph.langer at sap.com Wed Jun 13 08:32:55 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Wed, 13 Jun 2018 08:32:55 +0000 Subject: RFR(S): 8204654: [testbug] Fix pattern matching in jstat tests. In-Reply-To: <18662fc429004f34bae0343b03dcd28e@sap.com> References: <18662fc429004f34bae0343b03dcd28e@sap.com> Message-ID: <2e951d3f9e6d43679354a4714ca6840a@sap.com> Hi Goetz, this looks good to me. Thanks for making these tests more robust. Best regards Christoph > -----Original Message----- > From: serviceability-dev [mailto:serviceability-dev- > bounces at openjdk.java.net] On Behalf Of Lindenmaier, Goetz > Sent: Montag, 11. Juni 2018 15:15 > To: serviceability-dev (serviceability-dev at openjdk.java.net) dev at openjdk.java.net> > Subject: [CAUTION] RFR(S): 8204654: [testbug] Fix pattern matching in jstat > tests. > > Hi, > > please review this test fix: > http://cr.openjdk.java.net/~goetz/wr18/8204654-fixJstatTest/01 > > gcCauseOutput1.awk: > The pattern scans 11 numbers, while the output contains 13. Also, more '-' > are possible then checked. > > The other awk scripts need to check more patterns where '-' can appear. > > We have a machine with user.country=de where jstat prints ',' instead of '.' in > numbers. Explicitly > start with user.country=en as already done for user.language=en. > > I also refactored the common flags to a variable in utils.sh ... so there won't > be the need > to edit all these files once more :) > > Best regards, > Goetz > > (Sorry for the second mail, first missed the bug text ...) From thomas.stuefe at gmail.com Wed Jun 13 10:32:09 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 13 Jun 2018 12:32:09 +0200 Subject: RFR(s): 8203343: VM.{metaspace|classloaders|classhierarchy...} jcmd should show invocation targets for Generated{Method|Constructor}AccessorImpl classes In-Reply-To: <5a79df03-9001-bff8-7a45-edfb0284c9f1@oracle.com> References: <5a79df03-9001-bff8-7a45-edfb0284c9f1@oracle.com> Message-ID: Hi Serguei, thank you for your review! You are right, a regression test makes sense here. I wrote one. See updated webrev (only added the test, nothing else did change): http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/webrev.01/webrev/ I am re-running the jdk-submit tests. Thanks & Best Regards, Thomas On Tue, Jun 12, 2018 at 11:01 PM, serguei.spitsyn at oracle.com wrote: > Hi Thomas, > > It looks good to me. > But I'm not an expert in the area of Generated Accessors. > > How was this tested? > Does it make sense to add a unit test for this? > > Thanks, > Serguei > > > > On 6/6/18 09:05, Thomas St?fe wrote: >> >> Dear all, >> >> may I please have feedback and if possible reviews for this small >> addition: >> >> CR: https://bugs.openjdk.java.net/browse/JDK-8203343 >> Webrev: >> http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/webrev.00/webrev/ >> >> (Note: this patch goes atop of >> https://bugs.openjdk.java.net/browse/JDK-8203682, which is currently >> up for RFR). >> >> --- >> >> When analyzing situations involving a lot of reflection, one often >> stares at walls of "GeneratedXXXAccessorXXX" classes. These names are >> generated and not at all helpful in analyzing the problem (e.g. which >> component in a server node does this much reflection and hence uses so >> much metaspace). >> >> This patch adds the ability to print out invocation targets >> additionally to class names if the class is a generated accessor - for >> now this ability has been added to VM.metaspace, VM.classloaders and >> VM.class_hierarchy. >> >> Example output from "VM.class_hierarchy": >> >> >> |--jdk.internal.reflect.MagicAccessorImpl/null >> >> | |--jdk.internal.reflect.ConstructorAccessorImpl/null >> | | >> |--jdk.internal.reflect.GeneratedConstructorAccessor18/0x00007f9ee8350c10 >> (invokes: >> org/springframework/boot/context/properties/ConfigurationPropertiesBindingPostProcessorRegistrar:: >> ()V) >> | | >> |--jdk.internal.reflect.GeneratedConstructorAccessor17/0x00007f9ee8349c00 >> (invokes: >> org/springframework/boot/context/properties/EnableConfigurationPropertiesImportSelector$ConfigurationPropertiesBeanRegistrar:: >> ()V) >> >> | |--jdk.internal.reflect.MethodAccessorImpl/null >> | | |--jdk.internal.reflect.GeneratedMethodAccessor23/0x00007f9ec8329b60 >> (invokes: org/apache/tomcat/util/modeler/AttributeInfo::setIs (Z)V) >> | | |--jdk.internal.reflect.GeneratedMethodAccessor22/0x00007f9ee831bc70 >> (invokes: >> org/springframework/boot/autoconfigure/SpringBootApplication::exclude >> ()[Ljava/lang/Class;) >> >> >> See here more examples: >> >> "VM.class_hierarchy" >> >> http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/example-VM.class_hierarchy.txt >> >> "VM.classloaders show-classes" >> >> http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/example-VM.classloaders.txt >> >> >> ---- >> >> Note: I am not sure if this is a fit for hotspot-runtime or >> serviceability. Sorry for crossposting. >> >> Thank you, >> >> Thomas > > From yasuenag at gmail.com Wed Jun 13 12:09:45 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Wed, 13 Jun 2018 21:09:45 +0900 Subject: PING: RFR: 8204531: Remove unused chars following '\0' In-Reply-To: References: <6d19c245-eef8-96ce-3ca8-c3786d923c6a@gmail.com> <17c32d23-c2dc-884c-c9ae-15c7afd72bd9@oracle.com> Message-ID: PING: Could you review it? I want to merge this change before JDK 11 RDP 1. >>> webrev: >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.00/ >>> >>> JBS: >>> https://bugs.openjdk.java.net/browse/JDK-8204531 Yasumasa On 2018/06/08 15:38, Yasumasa Suenaga wrote: > Hi David, > > 2018-06-08 10:47 GMT+09:00 David Holmes : >> Hi Yasumasa, >> >> On 7/06/2018 10:43 PM, Yasumasa Suenaga wrote: >>> >>> Hi all, >>> >>> Please review this change: >>> >>> webrev: >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.00/ >>> >>> JBS: >>> https://bugs.openjdk.java.net/browse/JDK-8204531 >>> >>> >>> We can use `jhsdb jsnap` to check all PerfData. >>> String values in PerfData are defined as jbyte array, but JSnap cannot >>> handle it well as following: >>> >>> ``` >>> $ jhsdb jsnap --pid 28542 --all | less >>> >>> sun.gc.cause=No >>> GC^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ >>> ``` >>> >>> You can see this value via `less` and `vim` on Linux. `^@` shows it is >>> non-ascii character. >> >> >> So your synopsis says "remove unused chars following \0". Is that really >> what is happening? I would have expected "new String(bytes, encoding)" to >> stop processing bytes when it encounters a NUL! > > String c'tor will continue to parse byte array even if it finds '\0'. > You can see it on JShell as below: > > ------------- > jshell> var bytes = new byte[]{(byte)'a', (byte)'b', (byte)'c', > (byte)'\0', (byte)'d', (byte)'e', (byte)'f'} > bytes ==> byte[7] { 97, 98, 99, 0, 100, 101, 102 } > > jshell> new String(bytes, "US-ASCII") > $2 ==> "abc\000def" > ------------- > > >>> PerfDataEntry has null-terminated C string. So we should restore as it in >>> Java layer. >> >> >> If the issue is really "junk" beyond the \0 nul-terminator, how did we get >> such values? Shouldn't the byte array already be terminated at the NUL? Or >> is this just a raw representation of an array in the VM using the exact >> length of the array regardless of content? > > I think JSnap shows the array regardless of content. > > >> I see CStringUtilities.getString() both stops when it encounters a 0 value, >> and uses the default file encoding (else ASCII). >> >> The fix seems perfectly reasonable, I'm just unclear where exactly this >> "bad" data should have been stopped. > > Thanks! > > > Yasumasa > > >> Thanks, >> David >> >> >>> >>> Thanks, >>> >>> Yasumasa >>> >> From thomas.stuefe at gmail.com Wed Jun 13 12:18:20 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 13 Jun 2018 14:18:20 +0200 Subject: PING: RFR: 8204531: Remove unused chars following '\0' In-Reply-To: References: <6d19c245-eef8-96ce-3ca8-c3786d923c6a@gmail.com> <17c32d23-c2dc-884c-c9ae-15c7afd72bd9@oracle.com> Message-ID: Hi Yasumasa, your patch will change behavior of jsnap if file.encoding is set to something different than US-ASCII. Old version uses, hard wired, US-ASCII. You now use jvm/hotspot/utilities/CStringUtilities.java, which uses private static String encoding = System.getProperty("file.encoding", "US-ASCII"); and thus may use a different encoding. Best Regards, Thomas On Wed, Jun 13, 2018 at 2:09 PM, Yasumasa Suenaga wrote: > PING: Could you review it? > I want to merge this change before JDK 11 RDP 1. > >>>> webrev: >>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.00/ >>>> >>>> JBS: >>>> https://bugs.openjdk.java.net/browse/JDK-8204531 > > > > Yasumasa > > > On 2018/06/08 15:38, Yasumasa Suenaga wrote: >> >> Hi David, >> >> 2018-06-08 10:47 GMT+09:00 David Holmes : >>> >>> Hi Yasumasa, >>> >>> On 7/06/2018 10:43 PM, Yasumasa Suenaga wrote: >>>> >>>> >>>> Hi all, >>>> >>>> Please review this change: >>>> >>>> webrev: >>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.00/ >>>> >>>> JBS: >>>> https://bugs.openjdk.java.net/browse/JDK-8204531 >>>> >>>> >>>> We can use `jhsdb jsnap` to check all PerfData. >>>> String values in PerfData are defined as jbyte array, but JSnap cannot >>>> handle it well as following: >>>> >>>> ``` >>>> $ jhsdb jsnap --pid 28542 --all | less >>>> >>>> sun.gc.cause=No >>>> >>>> GC^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ >>>> ``` >>>> >>>> You can see this value via `less` and `vim` on Linux. `^@` shows it is >>>> non-ascii character. >>> >>> >>> >>> So your synopsis says "remove unused chars following \0". Is that really >>> what is happening? I would have expected "new String(bytes, encoding)" to >>> stop processing bytes when it encounters a NUL! >> >> >> String c'tor will continue to parse byte array even if it finds '\0'. >> You can see it on JShell as below: >> >> ------------- >> jshell> var bytes = new byte[]{(byte)'a', (byte)'b', (byte)'c', >> (byte)'\0', (byte)'d', (byte)'e', (byte)'f'} >> bytes ==> byte[7] { 97, 98, 99, 0, 100, 101, 102 } >> >> jshell> new String(bytes, "US-ASCII") >> $2 ==> "abc\000def" >> ------------- >> >> >>>> PerfDataEntry has null-terminated C string. So we should restore as it >>>> in >>>> Java layer. >>> >>> >>> >>> If the issue is really "junk" beyond the \0 nul-terminator, how did we >>> get >>> such values? Shouldn't the byte array already be terminated at the NUL? >>> Or >>> is this just a raw representation of an array in the VM using the exact >>> length of the array regardless of content? >> >> >> I think JSnap shows the array regardless of content. >> >> >>> I see CStringUtilities.getString() both stops when it encounters a 0 >>> value, >>> and uses the default file encoding (else ASCII). >>> >>> The fix seems perfectly reasonable, I'm just unclear where exactly this >>> "bad" data should have been stopped. >> >> >> Thanks! >> >> >> Yasumasa >> >> >>> Thanks, >>> David >>> >>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>> > From yasuenag at gmail.com Wed Jun 13 12:37:01 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Wed, 13 Jun 2018 21:37:01 +0900 Subject: PING: RFR: 8204531: Remove unused chars following '\0' In-Reply-To: References: <6d19c245-eef8-96ce-3ca8-c3786d923c6a@gmail.com> <17c32d23-c2dc-884c-c9ae-15c7afd72bd9@oracle.com> Message-ID: Hi Thomas, Thank you for your comment. CStringUtilities.java says the String which is constructed from byte array should use Charset.defaultCharset(). [1] I'm not convinced how should we fix - US-ASCII or default charset. IMHO we should use default charset. However I concern I need to get CSR approvement. (In addition, I want to refactor CStringUtilities. I guess we can make more simple code :-) What do you think about it? Yasumasa [1] http://hg.openjdk.java.net/jdk/jdk/file/7bf4f1b5e438/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/utilities/CStringUtilities.java#l73 On 2018/06/13 21:18, Thomas St?fe wrote: > Hi Yasumasa, > > your patch will change behavior of jsnap if file.encoding is set to > something different than US-ASCII. > > Old version uses, hard wired, US-ASCII. > > You now use jvm/hotspot/utilities/CStringUtilities.java, which uses > > private static String encoding = System.getProperty("file.encoding", > "US-ASCII"); > > and thus may use a different encoding. > > Best Regards, Thomas > > > > On Wed, Jun 13, 2018 at 2:09 PM, Yasumasa Suenaga wrote: >> PING: Could you review it? >> I want to merge this change before JDK 11 RDP 1. >> >>>>> webrev: >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.00/ >>>>> >>>>> JBS: >>>>> https://bugs.openjdk.java.net/browse/JDK-8204531 >> >> >> >> Yasumasa >> >> >> On 2018/06/08 15:38, Yasumasa Suenaga wrote: >>> >>> Hi David, >>> >>> 2018-06-08 10:47 GMT+09:00 David Holmes : >>>> >>>> Hi Yasumasa, >>>> >>>> On 7/06/2018 10:43 PM, Yasumasa Suenaga wrote: >>>>> >>>>> >>>>> Hi all, >>>>> >>>>> Please review this change: >>>>> >>>>> webrev: >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.00/ >>>>> >>>>> JBS: >>>>> https://bugs.openjdk.java.net/browse/JDK-8204531 >>>>> >>>>> >>>>> We can use `jhsdb jsnap` to check all PerfData. >>>>> String values in PerfData are defined as jbyte array, but JSnap cannot >>>>> handle it well as following: >>>>> >>>>> ``` >>>>> $ jhsdb jsnap --pid 28542 --all | less >>>>> >>>>> sun.gc.cause=No >>>>> >>>>> GC^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ >>>>> ``` >>>>> >>>>> You can see this value via `less` and `vim` on Linux. `^@` shows it is >>>>> non-ascii character. >>>> >>>> >>>> >>>> So your synopsis says "remove unused chars following \0". Is that really >>>> what is happening? I would have expected "new String(bytes, encoding)" to >>>> stop processing bytes when it encounters a NUL! >>> >>> >>> String c'tor will continue to parse byte array even if it finds '\0'. >>> You can see it on JShell as below: >>> >>> ------------- >>> jshell> var bytes = new byte[]{(byte)'a', (byte)'b', (byte)'c', >>> (byte)'\0', (byte)'d', (byte)'e', (byte)'f'} >>> bytes ==> byte[7] { 97, 98, 99, 0, 100, 101, 102 } >>> >>> jshell> new String(bytes, "US-ASCII") >>> $2 ==> "abc\000def" >>> ------------- >>> >>> >>>>> PerfDataEntry has null-terminated C string. So we should restore as it >>>>> in >>>>> Java layer. >>>> >>>> >>>> >>>> If the issue is really "junk" beyond the \0 nul-terminator, how did we >>>> get >>>> such values? Shouldn't the byte array already be terminated at the NUL? >>>> Or >>>> is this just a raw representation of an array in the VM using the exact >>>> length of the array regardless of content? >>> >>> >>> I think JSnap shows the array regardless of content. >>> >>> >>>> I see CStringUtilities.getString() both stops when it encounters a 0 >>>> value, >>>> and uses the default file encoding (else ASCII). >>>> >>>> The fix seems perfectly reasonable, I'm just unclear where exactly this >>>> "bad" data should have been stopped. >>> >>> >>> Thanks! >>> >>> >>> Yasumasa >>> >>> >>>> Thanks, >>>> David >>>> >>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>> >> From thomas.stuefe at gmail.com Wed Jun 13 12:46:03 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 13 Jun 2018 14:46:03 +0200 Subject: PING: RFR: 8204531: Remove unused chars following '\0' In-Reply-To: References: <6d19c245-eef8-96ce-3ca8-c3786d923c6a@gmail.com> <17c32d23-c2dc-884c-c9ae-15c7afd72bd9@oracle.com> Message-ID: Disclaimer: I am Reviewer but not versed in this corner of the jdk, nor have knowledge of the history, so take what I suggest with a grain of salt. That said, I'd probably just add a variant to CStringUtilities::getString which takes encoding as an argument, and then pass "US-ASCII" to remain compatible with the current version of jsnap. ..Thomas On Wed, Jun 13, 2018 at 2:37 PM, Yasumasa Suenaga wrote: > Hi Thomas, > > Thank you for your comment. > > CStringUtilities.java says the String which is constructed from byte array > should use Charset.defaultCharset(). [1] > I'm not convinced how should we fix - US-ASCII or default charset. > > IMHO we should use default charset. However I concern I need to get CSR > approvement. > (In addition, I want to refactor CStringUtilities. I guess we can make more > simple code :-) > > > What do you think about it? > > Yasumasa > > > [1] > http://hg.openjdk.java.net/jdk/jdk/file/7bf4f1b5e438/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/utilities/CStringUtilities.java#l73 > > > > On 2018/06/13 21:18, Thomas St?fe wrote: >> >> Hi Yasumasa, >> >> your patch will change behavior of jsnap if file.encoding is set to >> something different than US-ASCII. >> >> Old version uses, hard wired, US-ASCII. >> >> You now use jvm/hotspot/utilities/CStringUtilities.java, which uses >> >> private static String encoding = System.getProperty("file.encoding", >> "US-ASCII"); >> >> and thus may use a different encoding. >> >> Best Regards, Thomas >> >> >> >> On Wed, Jun 13, 2018 at 2:09 PM, Yasumasa Suenaga >> wrote: >>> >>> PING: Could you review it? >>> I want to merge this change before JDK 11 RDP 1. >>> >>>>>> webrev: >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.00/ >>>>>> >>>>>> JBS: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8204531 >>> >>> >>> >>> >>> Yasumasa >>> >>> >>> On 2018/06/08 15:38, Yasumasa Suenaga wrote: >>>> >>>> >>>> Hi David, >>>> >>>> 2018-06-08 10:47 GMT+09:00 David Holmes : >>>>> >>>>> >>>>> Hi Yasumasa, >>>>> >>>>> On 7/06/2018 10:43 PM, Yasumasa Suenaga wrote: >>>>>> >>>>>> >>>>>> >>>>>> Hi all, >>>>>> >>>>>> Please review this change: >>>>>> >>>>>> webrev: >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.00/ >>>>>> >>>>>> JBS: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8204531 >>>>>> >>>>>> >>>>>> We can use `jhsdb jsnap` to check all PerfData. >>>>>> String values in PerfData are defined as jbyte array, but JSnap cannot >>>>>> handle it well as following: >>>>>> >>>>>> ``` >>>>>> $ jhsdb jsnap --pid 28542 --all | less >>>>>> >>>>>> sun.gc.cause=No >>>>>> >>>>>> >>>>>> GC^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ >>>>>> ``` >>>>>> >>>>>> You can see this value via `less` and `vim` on Linux. `^@` shows it is >>>>>> non-ascii character. >>>>> >>>>> >>>>> >>>>> >>>>> So your synopsis says "remove unused chars following \0". Is that >>>>> really >>>>> what is happening? I would have expected "new String(bytes, encoding)" >>>>> to >>>>> stop processing bytes when it encounters a NUL! >>>> >>>> >>>> >>>> String c'tor will continue to parse byte array even if it finds '\0'. >>>> You can see it on JShell as below: >>>> >>>> ------------- >>>> jshell> var bytes = new byte[]{(byte)'a', (byte)'b', (byte)'c', >>>> (byte)'\0', (byte)'d', (byte)'e', (byte)'f'} >>>> bytes ==> byte[7] { 97, 98, 99, 0, 100, 101, 102 } >>>> >>>> jshell> new String(bytes, "US-ASCII") >>>> $2 ==> "abc\000def" >>>> ------------- >>>> >>>> >>>>>> PerfDataEntry has null-terminated C string. So we should restore as it >>>>>> in >>>>>> Java layer. >>>>> >>>>> >>>>> >>>>> >>>>> If the issue is really "junk" beyond the \0 nul-terminator, how did we >>>>> get >>>>> such values? Shouldn't the byte array already be terminated at the NUL? >>>>> Or >>>>> is this just a raw representation of an array in the VM using the exact >>>>> length of the array regardless of content? >>>> >>>> >>>> >>>> I think JSnap shows the array regardless of content. >>>> >>>> >>>>> I see CStringUtilities.getString() both stops when it encounters a 0 >>>>> value, >>>>> and uses the default file encoding (else ASCII). >>>>> >>>>> The fix seems perfectly reasonable, I'm just unclear where exactly this >>>>> "bad" data should have been stopped. >>>> >>>> >>>> >>>> Thanks! >>>> >>>> >>>> Yasumasa >>>> >>>> >>>>> Thanks, >>>>> David >>>>> >>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>> >>> > From yasuenag at gmail.com Wed Jun 13 13:33:04 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Wed, 13 Jun 2018 22:33:04 +0900 Subject: PING: RFR: 8204531: Remove unused chars following '\0' In-Reply-To: References: <6d19c245-eef8-96ce-3ca8-c3786d923c6a@gmail.com> <17c32d23-c2dc-884c-c9ae-15c7afd72bd9@oracle.com> Message-ID: <6d9dab04-6abe-71c0-bbe8-26e6d19c4e9c@gmail.com> Thanks Thomas, > That said, I'd probably just add a variant to > CStringUtilities::getString which takes encoding as an argument, and > then pass "US-ASCII" to remain compatible with the current version of > jsnap. I agree with you. My opinion is one of the ideals. Anyway, I uploaded new webrev which is added charset support in CStringUtils: http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.01/ Thanks, Yasumasa On 2018/06/13 21:46, Thomas St?fe wrote: > Disclaimer: I am Reviewer but not versed in this corner of the jdk, > nor have knowledge of the history, so take what I suggest with a grain > of salt. > > That said, I'd probably just add a variant to > CStringUtilities::getString which takes encoding as an argument, and > then pass "US-ASCII" to remain compatible with the current version of > jsnap. > > ..Thomas > > On Wed, Jun 13, 2018 at 2:37 PM, Yasumasa Suenaga wrote: >> Hi Thomas, >> >> Thank you for your comment. >> >> CStringUtilities.java says the String which is constructed from byte array >> should use Charset.defaultCharset(). [1] >> I'm not convinced how should we fix - US-ASCII or default charset. >> >> IMHO we should use default charset. However I concern I need to get CSR >> approvement. >> (In addition, I want to refactor CStringUtilities. I guess we can make more >> simple code :-) >> >> >> What do you think about it? >> >> Yasumasa >> >> >> [1] >> http://hg.openjdk.java.net/jdk/jdk/file/7bf4f1b5e438/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/utilities/CStringUtilities.java#l73 >> >> >> >> On 2018/06/13 21:18, Thomas St?fe wrote: >>> >>> Hi Yasumasa, >>> >>> your patch will change behavior of jsnap if file.encoding is set to >>> something different than US-ASCII. >>> >>> Old version uses, hard wired, US-ASCII. >>> >>> You now use jvm/hotspot/utilities/CStringUtilities.java, which uses >>> >>> private static String encoding = System.getProperty("file.encoding", >>> "US-ASCII"); >>> >>> and thus may use a different encoding. >>> >>> Best Regards, Thomas >>> >>> >>> >>> On Wed, Jun 13, 2018 at 2:09 PM, Yasumasa Suenaga >>> wrote: >>>> >>>> PING: Could you review it? >>>> I want to merge this change before JDK 11 RDP 1. >>>> >>>>>>> webrev: >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.00/ >>>>>>> >>>>>>> JBS: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8204531 >>>> >>>> >>>> >>>> >>>> Yasumasa >>>> >>>> >>>> On 2018/06/08 15:38, Yasumasa Suenaga wrote: >>>>> >>>>> >>>>> Hi David, >>>>> >>>>> 2018-06-08 10:47 GMT+09:00 David Holmes : >>>>>> >>>>>> >>>>>> Hi Yasumasa, >>>>>> >>>>>> On 7/06/2018 10:43 PM, Yasumasa Suenaga wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> Please review this change: >>>>>>> >>>>>>> webrev: >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.00/ >>>>>>> >>>>>>> JBS: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8204531 >>>>>>> >>>>>>> >>>>>>> We can use `jhsdb jsnap` to check all PerfData. >>>>>>> String values in PerfData are defined as jbyte array, but JSnap cannot >>>>>>> handle it well as following: >>>>>>> >>>>>>> ``` >>>>>>> $ jhsdb jsnap --pid 28542 --all | less >>>>>>> >>>>>>> sun.gc.cause=No >>>>>>> >>>>>>> >>>>>>> GC^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ >>>>>>> ``` >>>>>>> >>>>>>> You can see this value via `less` and `vim` on Linux. `^@` shows it is >>>>>>> non-ascii character. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> So your synopsis says "remove unused chars following \0". Is that >>>>>> really >>>>>> what is happening? I would have expected "new String(bytes, encoding)" >>>>>> to >>>>>> stop processing bytes when it encounters a NUL! >>>>> >>>>> >>>>> >>>>> String c'tor will continue to parse byte array even if it finds '\0'. >>>>> You can see it on JShell as below: >>>>> >>>>> ------------- >>>>> jshell> var bytes = new byte[]{(byte)'a', (byte)'b', (byte)'c', >>>>> (byte)'\0', (byte)'d', (byte)'e', (byte)'f'} >>>>> bytes ==> byte[7] { 97, 98, 99, 0, 100, 101, 102 } >>>>> >>>>> jshell> new String(bytes, "US-ASCII") >>>>> $2 ==> "abc\000def" >>>>> ------------- >>>>> >>>>> >>>>>>> PerfDataEntry has null-terminated C string. So we should restore as it >>>>>>> in >>>>>>> Java layer. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> If the issue is really "junk" beyond the \0 nul-terminator, how did we >>>>>> get >>>>>> such values? Shouldn't the byte array already be terminated at the NUL? >>>>>> Or >>>>>> is this just a raw representation of an array in the VM using the exact >>>>>> length of the array regardless of content? >>>>> >>>>> >>>>> >>>>> I think JSnap shows the array regardless of content. >>>>> >>>>> >>>>>> I see CStringUtilities.getString() both stops when it encounters a 0 >>>>>> value, >>>>>> and uses the default file encoding (else ASCII). >>>>>> >>>>>> The fix seems perfectly reasonable, I'm just unclear where exactly this >>>>>> "bad" data should have been stopped. >>>>> >>>>> >>>>> >>>>> Thanks! >>>>> >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>> >>>> >> From thomas.stuefe at gmail.com Wed Jun 13 13:41:51 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 13 Jun 2018 15:41:51 +0200 Subject: RFR(s): 8204958: Minor cleanups for the diagnostic framework Message-ID: Hi all, while working on jcmd I saw some minor cleanup possibilities. Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.00/webrev/ Bug: https://bugs.openjdk.java.net/browse/JDK-8204958 Most of them are const correctness and using initializer lists instead of java-style member initialization in these C++ classes. Other changes: - Added a ResourceMark into the DCmd::parse_and_execute() loop - removed DCmdFactory::create_global_DCmd() (and following that, DCmdFactory::create_Cheap_instance()) since I did not find anyone using that function. - Also in DCmdFactory, I removed a number of setters which were never called and for attributes which should be immutable: hidden, flags, enabled. This is really a rather light cleanup. It did not touch any larger issues. Thank you, Thomas From thomas.stuefe at gmail.com Wed Jun 13 18:15:04 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 13 Jun 2018 20:15:04 +0200 Subject: RFR(s): 8204958: Minor cleanups for the diagnostic framework In-Reply-To: References: Message-ID: (jdk-submit tests came back clean) On Wed, Jun 13, 2018 at 3:41 PM, Thomas St?fe wrote: > Hi all, > > while working on jcmd I saw some minor cleanup possibilities. > > Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.00/webrev/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8204958 > > Most of them are const correctness and using initializer lists instead > of java-style member initialization in these C++ classes. > > Other changes: > > - Added a ResourceMark into the DCmd::parse_and_execute() loop > - removed DCmdFactory::create_global_DCmd() (and following that, > DCmdFactory::create_Cheap_instance()) since I did not find anyone > using that function. > - Also in DCmdFactory, I removed a number of setters which were never > called and for attributes which should be immutable: hidden, flags, > enabled. > > This is really a rather light cleanup. It did not touch any larger issues. > > Thank you, > > Thomas From thomas.stuefe at gmail.com Wed Jun 13 18:34:26 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 13 Jun 2018 20:34:26 +0200 Subject: PING: RFR: 8204531: Remove unused chars following '\0' In-Reply-To: <6d9dab04-6abe-71c0-bbe8-26e6d19c4e9c@gmail.com> References: <6d19c245-eef8-96ce-3ca8-c3786d923c6a@gmail.com> <17c32d23-c2dc-884c-c9ae-15c7afd72bd9@oracle.com> <6d9dab04-6abe-71c0-bbe8-26e6d19c4e9c@gmail.com> Message-ID: Hi Yasumasa, looks good to me. small nit: import java.nio.charset.* -> import java.nio.charset.Charset; We loose the assert for type now from byteArrayValue(). Can you re-add it to the location where you call CStringUtils? We do not seem to need byteArrayValue(), can this be removed? Best Regards, Thomas On Wed, Jun 13, 2018 at 3:33 PM, Yasumasa Suenaga wrote: > Thanks Thomas, > >> That said, I'd probably just add a variant to >> CStringUtilities::getString which takes encoding as an argument, and >> then pass "US-ASCII" to remain compatible with the current version of >> jsnap. > > > I agree with you. My opinion is one of the ideals. > Anyway, I uploaded new webrev which is added charset support in > CStringUtils: > > http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.01/ > > > Thanks, > > Yasumasa > > > > On 2018/06/13 21:46, Thomas St?fe wrote: >> >> Disclaimer: I am Reviewer but not versed in this corner of the jdk, >> nor have knowledge of the history, so take what I suggest with a grain >> of salt. >> >> That said, I'd probably just add a variant to >> CStringUtilities::getString which takes encoding as an argument, and >> then pass "US-ASCII" to remain compatible with the current version of >> jsnap. >> >> ..Thomas >> >> On Wed, Jun 13, 2018 at 2:37 PM, Yasumasa Suenaga >> wrote: >>> >>> Hi Thomas, >>> >>> Thank you for your comment. >>> >>> CStringUtilities.java says the String which is constructed from byte >>> array >>> should use Charset.defaultCharset(). [1] >>> I'm not convinced how should we fix - US-ASCII or default charset. >>> >>> IMHO we should use default charset. However I concern I need to get CSR >>> approvement. >>> (In addition, I want to refactor CStringUtilities. I guess we can make >>> more >>> simple code :-) >>> >>> >>> What do you think about it? >>> >>> Yasumasa >>> >>> >>> [1] >>> >>> http://hg.openjdk.java.net/jdk/jdk/file/7bf4f1b5e438/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/utilities/CStringUtilities.java#l73 >>> >>> >>> >>> On 2018/06/13 21:18, Thomas St?fe wrote: >>>> >>>> >>>> Hi Yasumasa, >>>> >>>> your patch will change behavior of jsnap if file.encoding is set to >>>> something different than US-ASCII. >>>> >>>> Old version uses, hard wired, US-ASCII. >>>> >>>> You now use jvm/hotspot/utilities/CStringUtilities.java, which uses >>>> >>>> private static String encoding = System.getProperty("file.encoding", >>>> "US-ASCII"); >>>> >>>> and thus may use a different encoding. >>>> >>>> Best Regards, Thomas >>>> >>>> >>>> >>>> On Wed, Jun 13, 2018 at 2:09 PM, Yasumasa Suenaga >>>> wrote: >>>>> >>>>> >>>>> PING: Could you review it? >>>>> I want to merge this change before JDK 11 RDP 1. >>>>> >>>>>>>> webrev: >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.00/ >>>>>>>> >>>>>>>> JBS: >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8204531 >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2018/06/08 15:38, Yasumasa Suenaga wrote: >>>>>> >>>>>> >>>>>> >>>>>> Hi David, >>>>>> >>>>>> 2018-06-08 10:47 GMT+09:00 David Holmes : >>>>>>> >>>>>>> >>>>>>> >>>>>>> Hi Yasumasa, >>>>>>> >>>>>>> On 7/06/2018 10:43 PM, Yasumasa Suenaga wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Hi all, >>>>>>>> >>>>>>>> Please review this change: >>>>>>>> >>>>>>>> webrev: >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.00/ >>>>>>>> >>>>>>>> JBS: >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8204531 >>>>>>>> >>>>>>>> >>>>>>>> We can use `jhsdb jsnap` to check all PerfData. >>>>>>>> String values in PerfData are defined as jbyte array, but JSnap >>>>>>>> cannot >>>>>>>> handle it well as following: >>>>>>>> >>>>>>>> ``` >>>>>>>> $ jhsdb jsnap --pid 28542 --all | less >>>>>>>> >>>>>>>> sun.gc.cause=No >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> GC^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ >>>>>>>> ``` >>>>>>>> >>>>>>>> You can see this value via `less` and `vim` on Linux. `^@` shows it >>>>>>>> is >>>>>>>> non-ascii character. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> So your synopsis says "remove unused chars following \0". Is that >>>>>>> really >>>>>>> what is happening? I would have expected "new String(bytes, >>>>>>> encoding)" >>>>>>> to >>>>>>> stop processing bytes when it encounters a NUL! >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> String c'tor will continue to parse byte array even if it finds '\0'. >>>>>> You can see it on JShell as below: >>>>>> >>>>>> ------------- >>>>>> jshell> var bytes = new byte[]{(byte)'a', (byte)'b', (byte)'c', >>>>>> (byte)'\0', (byte)'d', (byte)'e', (byte)'f'} >>>>>> bytes ==> byte[7] { 97, 98, 99, 0, 100, 101, 102 } >>>>>> >>>>>> jshell> new String(bytes, "US-ASCII") >>>>>> $2 ==> "abc\000def" >>>>>> ------------- >>>>>> >>>>>> >>>>>>>> PerfDataEntry has null-terminated C string. So we should restore as >>>>>>>> it >>>>>>>> in >>>>>>>> Java layer. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> If the issue is really "junk" beyond the \0 nul-terminator, how did >>>>>>> we >>>>>>> get >>>>>>> such values? Shouldn't the byte array already be terminated at the >>>>>>> NUL? >>>>>>> Or >>>>>>> is this just a raw representation of an array in the VM using the >>>>>>> exact >>>>>>> length of the array regardless of content? >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> I think JSnap shows the array regardless of content. >>>>>> >>>>>> >>>>>>> I see CStringUtilities.getString() both stops when it encounters a 0 >>>>>>> value, >>>>>>> and uses the default file encoding (else ASCII). >>>>>>> >>>>>>> The fix seems perfectly reasonable, I'm just unclear where exactly >>>>>>> this >>>>>>> "bad" data should have been stopped. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Thanks! >>>>>> >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>> >>>>> >>> > From serguei.spitsyn at oracle.com Wed Jun 13 23:11:50 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 13 Jun 2018 16:11:50 -0700 Subject: RFR(s): 8204958: Minor cleanups for the diagnostic framework In-Reply-To: References: Message-ID: An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Wed Jun 13 23:44:22 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 13 Jun 2018 16:44:22 -0700 Subject: RFR(s): 8204958: Minor cleanups for the diagnostic framework In-Reply-To: References: Message-ID: <1a78f599-bf6a-ff72-c687-4af4f474a58d@oracle.com> An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Thu Jun 14 01:11:21 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 13 Jun 2018 18:11:21 -0700 Subject: RFR(s): 8204958: Minor cleanups for the diagnostic framework In-Reply-To: <1a78f599-bf6a-ff72-c687-4af4f474a58d@oracle.com> References: <1a78f599-bf6a-ff72-c687-4af4f474a58d@oracle.com> Message-ID: <5fff0bb9-81e0-498d-53ff-a480d51efff9@oracle.com> An HTML attachment was scrubbed... URL: From yasuenag at gmail.com Thu Jun 14 04:18:53 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Thu, 14 Jun 2018 13:18:53 +0900 Subject: PING: RFR: 8204531: Remove unused chars following '\0' In-Reply-To: References: <6d19c245-eef8-96ce-3ca8-c3786d923c6a@gmail.com> <17c32d23-c2dc-884c-c9ae-15c7afd72bd9@oracle.com> <6d9dab04-6abe-71c0-bbe8-26e6d19c4e9c@gmail.com> Message-ID: Hi Thomas, Thank you for your comment. I uploaded new webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.02/ 2018-06-14 3:34 GMT+09:00 Thomas St?fe : > Hi Yasumasa, > > looks good to me. > > small nit: > import java.nio.charset.* -> import java.nio.charset.Charset; Fixed. > We loose the assert for type now from byteArrayValue(). Can you re-add > it to the location where you call CStringUtils? I added the assertion to check vector length. I did not add type assertion because if-statement branches by data type. > We do not seem to need byteArrayValue(), can this be removed? I cannot agree that. byteArrayValue() is public method (but it is not used AFAICS). However, similar getter methods are privided from PerfDataEntry (e.g. charArrayValue()). So I think we should not remove byteArrayValue(). Regards, Yasumasa > Best Regards, Thomas > > > > On Wed, Jun 13, 2018 at 3:33 PM, Yasumasa Suenaga wrote: >> Thanks Thomas, >> >>> That said, I'd probably just add a variant to >>> CStringUtilities::getString which takes encoding as an argument, and >>> then pass "US-ASCII" to remain compatible with the current version of >>> jsnap. >> >> >> I agree with you. My opinion is one of the ideals. >> Anyway, I uploaded new webrev which is added charset support in >> CStringUtils: >> >> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.01/ >> >> >> Thanks, >> >> Yasumasa >> >> >> >> On 2018/06/13 21:46, Thomas St?fe wrote: >>> >>> Disclaimer: I am Reviewer but not versed in this corner of the jdk, >>> nor have knowledge of the history, so take what I suggest with a grain >>> of salt. >>> >>> That said, I'd probably just add a variant to >>> CStringUtilities::getString which takes encoding as an argument, and >>> then pass "US-ASCII" to remain compatible with the current version of >>> jsnap. >>> >>> ..Thomas >>> >>> On Wed, Jun 13, 2018 at 2:37 PM, Yasumasa Suenaga >>> wrote: >>>> >>>> Hi Thomas, >>>> >>>> Thank you for your comment. >>>> >>>> CStringUtilities.java says the String which is constructed from byte >>>> array >>>> should use Charset.defaultCharset(). [1] >>>> I'm not convinced how should we fix - US-ASCII or default charset. >>>> >>>> IMHO we should use default charset. However I concern I need to get CSR >>>> approvement. >>>> (In addition, I want to refactor CStringUtilities. I guess we can make >>>> more >>>> simple code :-) >>>> >>>> >>>> What do you think about it? >>>> >>>> Yasumasa >>>> >>>> >>>> [1] >>>> >>>> http://hg.openjdk.java.net/jdk/jdk/file/7bf4f1b5e438/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/utilities/CStringUtilities.java#l73 >>>> >>>> >>>> >>>> On 2018/06/13 21:18, Thomas St?fe wrote: >>>>> >>>>> >>>>> Hi Yasumasa, >>>>> >>>>> your patch will change behavior of jsnap if file.encoding is set to >>>>> something different than US-ASCII. >>>>> >>>>> Old version uses, hard wired, US-ASCII. >>>>> >>>>> You now use jvm/hotspot/utilities/CStringUtilities.java, which uses >>>>> >>>>> private static String encoding = System.getProperty("file.encoding", >>>>> "US-ASCII"); >>>>> >>>>> and thus may use a different encoding. >>>>> >>>>> Best Regards, Thomas >>>>> >>>>> >>>>> >>>>> On Wed, Jun 13, 2018 at 2:09 PM, Yasumasa Suenaga >>>>> wrote: >>>>>> >>>>>> >>>>>> PING: Could you review it? >>>>>> I want to merge this change before JDK 11 RDP 1. >>>>>> >>>>>>>>> webrev: >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.00/ >>>>>>>>> >>>>>>>>> JBS: >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8204531 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2018/06/08 15:38, Yasumasa Suenaga wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> Hi David, >>>>>>> >>>>>>> 2018-06-08 10:47 GMT+09:00 David Holmes : >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Hi Yasumasa, >>>>>>>> >>>>>>>> On 7/06/2018 10:43 PM, Yasumasa Suenaga wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Hi all, >>>>>>>>> >>>>>>>>> Please review this change: >>>>>>>>> >>>>>>>>> webrev: >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.00/ >>>>>>>>> >>>>>>>>> JBS: >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8204531 >>>>>>>>> >>>>>>>>> >>>>>>>>> We can use `jhsdb jsnap` to check all PerfData. >>>>>>>>> String values in PerfData are defined as jbyte array, but JSnap >>>>>>>>> cannot >>>>>>>>> handle it well as following: >>>>>>>>> >>>>>>>>> ``` >>>>>>>>> $ jhsdb jsnap --pid 28542 --all | less >>>>>>>>> >>>>>>>>> sun.gc.cause=No >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> GC^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ >>>>>>>>> ``` >>>>>>>>> >>>>>>>>> You can see this value via `less` and `vim` on Linux. `^@` shows it >>>>>>>>> is >>>>>>>>> non-ascii character. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> So your synopsis says "remove unused chars following \0". Is that >>>>>>>> really >>>>>>>> what is happening? I would have expected "new String(bytes, >>>>>>>> encoding)" >>>>>>>> to >>>>>>>> stop processing bytes when it encounters a NUL! >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> String c'tor will continue to parse byte array even if it finds '\0'. >>>>>>> You can see it on JShell as below: >>>>>>> >>>>>>> ------------- >>>>>>> jshell> var bytes = new byte[]{(byte)'a', (byte)'b', (byte)'c', >>>>>>> (byte)'\0', (byte)'d', (byte)'e', (byte)'f'} >>>>>>> bytes ==> byte[7] { 97, 98, 99, 0, 100, 101, 102 } >>>>>>> >>>>>>> jshell> new String(bytes, "US-ASCII") >>>>>>> $2 ==> "abc\000def" >>>>>>> ------------- >>>>>>> >>>>>>> >>>>>>>>> PerfDataEntry has null-terminated C string. So we should restore as >>>>>>>>> it >>>>>>>>> in >>>>>>>>> Java layer. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> If the issue is really "junk" beyond the \0 nul-terminator, how did >>>>>>>> we >>>>>>>> get >>>>>>>> such values? Shouldn't the byte array already be terminated at the >>>>>>>> NUL? >>>>>>>> Or >>>>>>>> is this just a raw representation of an array in the VM using the >>>>>>>> exact >>>>>>>> length of the array regardless of content? >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> I think JSnap shows the array regardless of content. >>>>>>> >>>>>>> >>>>>>>> I see CStringUtilities.getString() both stops when it encounters a 0 >>>>>>>> value, >>>>>>>> and uses the default file encoding (else ASCII). >>>>>>>> >>>>>>>> The fix seems perfectly reasonable, I'm just unclear where exactly >>>>>>>> this >>>>>>>> "bad" data should have been stopped. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Thanks! >>>>>>> >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>> >>>>>> >>>> >> From chris.plummer at oracle.com Thu Jun 14 04:19:18 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 13 Jun 2018 21:19:18 -0700 Subject: RFR(s): 8204958: Minor cleanups for the diagnostic framework In-Reply-To: <5fff0bb9-81e0-498d-53ff-a480d51efff9@oracle.com> References: <1a78f599-bf6a-ff72-c687-4af4f474a58d@oracle.com> <5fff0bb9-81e0-498d-53ff-a480d51efff9@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Thu Jun 14 04:41:37 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 14 Jun 2018 14:41:37 +1000 Subject: PING: RFR: 8204531: Remove unused chars following '\0' In-Reply-To: References: <6d19c245-eef8-96ce-3ca8-c3786d923c6a@gmail.com> <17c32d23-c2dc-884c-c9ae-15c7afd72bd9@oracle.com> <6d9dab04-6abe-71c0-bbe8-26e6d19c4e9c@gmail.com> Message-ID: <5d3d3768-cf61-6cfd-f631-2a9056d6f172@oracle.com> Hi Yasumasa, This seems mostly okay - though like Thomas this is not an area I'm that familiar with. But it seems this addresses the problem without impacting other things. This assert is redundant: 366 if (Assert.ASSERTS_ENABLED) { 367 Assert.that(vectorLength() > 0, "not a byte vector"); You can't reach it unless len != 0 and I assume you're not really checking that len is not-negative - if you are then the assert needs to go further up after: 328 int len = vectorLength(); Thanks, David On 14/06/2018 2:18 PM, Yasumasa Suenaga wrote: > Hi Thomas, > > Thank you for your comment. > I uploaded new webrev: > > http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.02/ > > > 2018-06-14 3:34 GMT+09:00 Thomas St?fe : >> Hi Yasumasa, >> >> looks good to me. >> >> small nit: >> import java.nio.charset.* -> import java.nio.charset.Charset; > > Fixed. > > >> We loose the assert for type now from byteArrayValue(). Can you re-add >> it to the location where you call CStringUtils? > > I added the assertion to check vector length. > I did not add type assertion because if-statement branches by data type. > > >> We do not seem to need byteArrayValue(), can this be removed? > > I cannot agree that. > byteArrayValue() is public method (but it is not used AFAICS). > However, similar getter methods are privided from PerfDataEntry (e.g. > charArrayValue()). So I think we should not remove byteArrayValue(). > > > Regards, > > Yasumasa > > >> Best Regards, Thomas >> >> >> >> On Wed, Jun 13, 2018 at 3:33 PM, Yasumasa Suenaga wrote: >>> Thanks Thomas, >>> >>>> That said, I'd probably just add a variant to >>>> CStringUtilities::getString which takes encoding as an argument, and >>>> then pass "US-ASCII" to remain compatible with the current version of >>>> jsnap. >>> >>> >>> I agree with you. My opinion is one of the ideals. >>> Anyway, I uploaded new webrev which is added charset support in >>> CStringUtils: >>> >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.01/ >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> >>> On 2018/06/13 21:46, Thomas St?fe wrote: >>>> >>>> Disclaimer: I am Reviewer but not versed in this corner of the jdk, >>>> nor have knowledge of the history, so take what I suggest with a grain >>>> of salt. >>>> >>>> That said, I'd probably just add a variant to >>>> CStringUtilities::getString which takes encoding as an argument, and >>>> then pass "US-ASCII" to remain compatible with the current version of >>>> jsnap. >>>> >>>> ..Thomas >>>> >>>> On Wed, Jun 13, 2018 at 2:37 PM, Yasumasa Suenaga >>>> wrote: >>>>> >>>>> Hi Thomas, >>>>> >>>>> Thank you for your comment. >>>>> >>>>> CStringUtilities.java says the String which is constructed from byte >>>>> array >>>>> should use Charset.defaultCharset(). [1] >>>>> I'm not convinced how should we fix - US-ASCII or default charset. >>>>> >>>>> IMHO we should use default charset. However I concern I need to get CSR >>>>> approvement. >>>>> (In addition, I want to refactor CStringUtilities. I guess we can make >>>>> more >>>>> simple code :-) >>>>> >>>>> >>>>> What do you think about it? >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> [1] >>>>> >>>>> http://hg.openjdk.java.net/jdk/jdk/file/7bf4f1b5e438/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/utilities/CStringUtilities.java#l73 >>>>> >>>>> >>>>> >>>>> On 2018/06/13 21:18, Thomas St?fe wrote: >>>>>> >>>>>> >>>>>> Hi Yasumasa, >>>>>> >>>>>> your patch will change behavior of jsnap if file.encoding is set to >>>>>> something different than US-ASCII. >>>>>> >>>>>> Old version uses, hard wired, US-ASCII. >>>>>> >>>>>> You now use jvm/hotspot/utilities/CStringUtilities.java, which uses >>>>>> >>>>>> private static String encoding = System.getProperty("file.encoding", >>>>>> "US-ASCII"); >>>>>> >>>>>> and thus may use a different encoding. >>>>>> >>>>>> Best Regards, Thomas >>>>>> >>>>>> >>>>>> >>>>>> On Wed, Jun 13, 2018 at 2:09 PM, Yasumasa Suenaga >>>>>> wrote: >>>>>>> >>>>>>> >>>>>>> PING: Could you review it? >>>>>>> I want to merge this change before JDK 11 RDP 1. >>>>>>> >>>>>>>>>> webrev: >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.00/ >>>>>>>>>> >>>>>>>>>> JBS: >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8204531 >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2018/06/08 15:38, Yasumasa Suenaga wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Hi David, >>>>>>>> >>>>>>>> 2018-06-08 10:47 GMT+09:00 David Holmes : >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Hi Yasumasa, >>>>>>>>> >>>>>>>>> On 7/06/2018 10:43 PM, Yasumasa Suenaga wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hi all, >>>>>>>>>> >>>>>>>>>> Please review this change: >>>>>>>>>> >>>>>>>>>> webrev: >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.00/ >>>>>>>>>> >>>>>>>>>> JBS: >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8204531 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> We can use `jhsdb jsnap` to check all PerfData. >>>>>>>>>> String values in PerfData are defined as jbyte array, but JSnap >>>>>>>>>> cannot >>>>>>>>>> handle it well as following: >>>>>>>>>> >>>>>>>>>> ``` >>>>>>>>>> $ jhsdb jsnap --pid 28542 --all | less >>>>>>>>>> >>>>>>>>>> sun.gc.cause=No >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> GC^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ >>>>>>>>>> ``` >>>>>>>>>> >>>>>>>>>> You can see this value via `less` and `vim` on Linux. `^@` shows it >>>>>>>>>> is >>>>>>>>>> non-ascii character. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> So your synopsis says "remove unused chars following \0". Is that >>>>>>>>> really >>>>>>>>> what is happening? I would have expected "new String(bytes, >>>>>>>>> encoding)" >>>>>>>>> to >>>>>>>>> stop processing bytes when it encounters a NUL! >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> String c'tor will continue to parse byte array even if it finds '\0'. >>>>>>>> You can see it on JShell as below: >>>>>>>> >>>>>>>> ------------- >>>>>>>> jshell> var bytes = new byte[]{(byte)'a', (byte)'b', (byte)'c', >>>>>>>> (byte)'\0', (byte)'d', (byte)'e', (byte)'f'} >>>>>>>> bytes ==> byte[7] { 97, 98, 99, 0, 100, 101, 102 } >>>>>>>> >>>>>>>> jshell> new String(bytes, "US-ASCII") >>>>>>>> $2 ==> "abc\000def" >>>>>>>> ------------- >>>>>>>> >>>>>>>> >>>>>>>>>> PerfDataEntry has null-terminated C string. So we should restore as >>>>>>>>>> it >>>>>>>>>> in >>>>>>>>>> Java layer. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> If the issue is really "junk" beyond the \0 nul-terminator, how did >>>>>>>>> we >>>>>>>>> get >>>>>>>>> such values? Shouldn't the byte array already be terminated at the >>>>>>>>> NUL? >>>>>>>>> Or >>>>>>>>> is this just a raw representation of an array in the VM using the >>>>>>>>> exact >>>>>>>>> length of the array regardless of content? >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> I think JSnap shows the array regardless of content. >>>>>>>> >>>>>>>> >>>>>>>>> I see CStringUtilities.getString() both stops when it encounters a 0 >>>>>>>>> value, >>>>>>>>> and uses the default file encoding (else ASCII). >>>>>>>>> >>>>>>>>> The fix seems perfectly reasonable, I'm just unclear where exactly >>>>>>>>> this >>>>>>>>> "bad" data should have been stopped. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Thanks! >>>>>>>> >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> David >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>> >>>>>>> >>>>> >>> From serguei.spitsyn at oracle.com Thu Jun 14 04:53:44 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 13 Jun 2018 21:53:44 -0700 Subject: RFR(s): 8204958: Minor cleanups for the diagnostic framework In-Reply-To: References: <1a78f599-bf6a-ff72-c687-4af4f474a58d@oracle.com> <5fff0bb9-81e0-498d-53ff-a480d51efff9@oracle.com> Message-ID: <948d89d5-63c3-24e0-d33c-01e8c8d75547@oracle.com> An HTML attachment was scrubbed... URL: From thomas.stuefe at gmail.com Thu Jun 14 04:55:26 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 14 Jun 2018 06:55:26 +0200 Subject: RFR(s): 8204958: Minor cleanups for the diagnostic framework In-Reply-To: References: <1a78f599-bf6a-ff72-c687-4af4f474a58d@oracle.com> <5fff0bb9-81e0-498d-53ff-a480d51efff9@oracle.com> Message-ID: On Thu, Jun 14, 2018 at 6:19 AM, Chris Plummer wrote: > On 6/13/18 6:11 PM, serguei.spitsyn at oracle.com wrote: > > On 6/13/18 16:44, Chris Plummer wrote: > > On 6/13/18 4:11 PM, serguei.spitsyn at oracle.com wrote: > > Hi Thomas, > > I looks good to me. > > A couple of minor comments. > > http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.00/webrev/src/hotspot/share/services/diagnosticArgument.hpp.udiff.html > > - void value_as_str(char *buf, size_t len) { return to_string(_value, buf, > len);} > + void value_as_str(char *buf, size_t len) const { to_string(_value, buf, > len);} > > I'm puzzled a little bit. > How did the value_as_str() returned something before if the function has > void return type? > > I guess because to_string() returns void. I just did a little experiment and > the following compiles just fine: > > void return_void() { > return; > } > void return_void2() { > return return_void(); > } > > > Thanks, Chris. > I thought about the same. > > http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.00/webrev/src/hotspot/share/services/diagnosticFramework.hpp.frames.html > > 170 DCmdArgumentInfo(const char* name, const char* description, const > char* type, > 171 const char* default_string, bool mandatory, bool > option, > 172 bool multiple, int position = -1) > 173 : _name(name), _description(description), _type(type) > 174 , _default_string(default_string), _mandatory(mandatory) > 175 , _option(option), _multiple(multiple), _position(-1) {} > > > Would it be more safe for the _position initialization to be: > _position(position)? > > I think it's actually a bug and should be _position(position). > > > Agreed. > > Are you Okay with this fix? > > I think it should be _position(position). If the argument is not specified, > it will get the default of -1 as specified in the prototype, which is what > you want. > Oh, good catch. Thank you Chris, I'll fix this. I'll prepare a new webrev. ..Thomas > thanks, > > Chris > > > Thanks, > Serguei > > > Chris > > > Thanks, > Serguei > > > > On 6/13/18 06:41, Thomas St?fe wrote: > > Hi all, > > while working on jcmd I saw some minor cleanup possibilities. > > Webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.00/webrev/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8204958 > > Most of them are const correctness and using initializer lists instead > of java-style member initialization in these C++ classes. > > Other changes: > > - Added a ResourceMark into the DCmd::parse_and_execute() loop > - removed DCmdFactory::create_global_DCmd() (and following that, > DCmdFactory::create_Cheap_instance()) since I did not find anyone > using that function. > - Also in DCmdFactory, I removed a number of setters which were never > called and for attributes which should be immutable: hidden, flags, > enabled. > > This is really a rather light cleanup. It did not touch any larger issues. > > Thank you, > > Thomas > > > > > From yasuenag at gmail.com Thu Jun 14 05:01:06 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Thu, 14 Jun 2018 14:01:06 +0900 Subject: PING: RFR: 8204531: Remove unused chars following '\0' In-Reply-To: <5d3d3768-cf61-6cfd-f631-2a9056d6f172@oracle.com> References: <6d19c245-eef8-96ce-3ca8-c3786d923c6a@gmail.com> <17c32d23-c2dc-884c-c9ae-15c7afd72bd9@oracle.com> <6d9dab04-6abe-71c0-bbe8-26e6d19c4e9c@gmail.com> <5d3d3768-cf61-6cfd-f631-2a9056d6f172@oracle.com> Message-ID: Hi David, Thank you for your comment. I removed the assertion from PerfDataEntry.java . Could you review again? http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.03/ Thanks, Yasuamsa 2018-06-14 13:41 GMT+09:00 David Holmes : > Hi Yasumasa, > > This seems mostly okay - though like Thomas this is not an area I'm that > familiar with. But it seems this addresses the problem without impacting > other things. > > This assert is redundant: > > 366 if (Assert.ASSERTS_ENABLED) { > 367 Assert.that(vectorLength() > 0, "not a byte > vector"); > > You can't reach it unless len != 0 and I assume you're not really checking > that len is not-negative - if you are then the assert needs to go further up > after: > > 328 int len = vectorLength(); > > Thanks, > David > > > On 14/06/2018 2:18 PM, Yasumasa Suenaga wrote: >> >> Hi Thomas, >> >> Thank you for your comment. >> I uploaded new webrev: >> >> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.02/ >> >> >> 2018-06-14 3:34 GMT+09:00 Thomas St?fe : >>> >>> Hi Yasumasa, >>> >>> looks good to me. >>> >>> small nit: >>> import java.nio.charset.* -> import java.nio.charset.Charset; >> >> >> Fixed. >> >> >>> We loose the assert for type now from byteArrayValue(). Can you re-add >>> it to the location where you call CStringUtils? >> >> >> I added the assertion to check vector length. >> I did not add type assertion because if-statement branches by data type. >> >> >>> We do not seem to need byteArrayValue(), can this be removed? >> >> >> I cannot agree that. >> byteArrayValue() is public method (but it is not used AFAICS). >> However, similar getter methods are privided from PerfDataEntry (e.g. >> charArrayValue()). So I think we should not remove byteArrayValue(). >> >> >> Regards, >> >> Yasumasa >> >> >>> Best Regards, Thomas >>> >>> >>> >>> On Wed, Jun 13, 2018 at 3:33 PM, Yasumasa Suenaga >>> wrote: >>>> >>>> Thanks Thomas, >>>> >>>>> That said, I'd probably just add a variant to >>>>> CStringUtilities::getString which takes encoding as an argument, and >>>>> then pass "US-ASCII" to remain compatible with the current version of >>>>> jsnap. >>>> >>>> >>>> >>>> I agree with you. My opinion is one of the ideals. >>>> Anyway, I uploaded new webrev which is added charset support in >>>> CStringUtils: >>>> >>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.01/ >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> >>>> On 2018/06/13 21:46, Thomas St?fe wrote: >>>>> >>>>> >>>>> Disclaimer: I am Reviewer but not versed in this corner of the jdk, >>>>> nor have knowledge of the history, so take what I suggest with a grain >>>>> of salt. >>>>> >>>>> That said, I'd probably just add a variant to >>>>> CStringUtilities::getString which takes encoding as an argument, and >>>>> then pass "US-ASCII" to remain compatible with the current version of >>>>> jsnap. >>>>> >>>>> ..Thomas >>>>> >>>>> On Wed, Jun 13, 2018 at 2:37 PM, Yasumasa Suenaga >>>>> wrote: >>>>>> >>>>>> >>>>>> Hi Thomas, >>>>>> >>>>>> Thank you for your comment. >>>>>> >>>>>> CStringUtilities.java says the String which is constructed from byte >>>>>> array >>>>>> should use Charset.defaultCharset(). [1] >>>>>> I'm not convinced how should we fix - US-ASCII or default charset. >>>>>> >>>>>> IMHO we should use default charset. However I concern I need to get >>>>>> CSR >>>>>> approvement. >>>>>> (In addition, I want to refactor CStringUtilities. I guess we can make >>>>>> more >>>>>> simple code :-) >>>>>> >>>>>> >>>>>> What do you think about it? >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> [1] >>>>>> >>>>>> >>>>>> http://hg.openjdk.java.net/jdk/jdk/file/7bf4f1b5e438/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/utilities/CStringUtilities.java#l73 >>>>>> >>>>>> >>>>>> >>>>>> On 2018/06/13 21:18, Thomas St?fe wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> Hi Yasumasa, >>>>>>> >>>>>>> your patch will change behavior of jsnap if file.encoding is set to >>>>>>> something different than US-ASCII. >>>>>>> >>>>>>> Old version uses, hard wired, US-ASCII. >>>>>>> >>>>>>> You now use jvm/hotspot/utilities/CStringUtilities.java, which uses >>>>>>> >>>>>>> private static String encoding = System.getProperty("file.encoding", >>>>>>> "US-ASCII"); >>>>>>> >>>>>>> and thus may use a different encoding. >>>>>>> >>>>>>> Best Regards, Thomas >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Wed, Jun 13, 2018 at 2:09 PM, Yasumasa Suenaga >>>>>>> >>>>>>> wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> PING: Could you review it? >>>>>>>> I want to merge this change before JDK 11 RDP 1. >>>>>>>> >>>>>>>>>>> webrev: >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.00/ >>>>>>>>>>> >>>>>>>>>>> JBS: >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8204531 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2018/06/08 15:38, Yasumasa Suenaga wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Hi David, >>>>>>>>> >>>>>>>>> 2018-06-08 10:47 GMT+09:00 David Holmes : >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hi Yasumasa, >>>>>>>>>> >>>>>>>>>> On 7/06/2018 10:43 PM, Yasumasa Suenaga wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hi all, >>>>>>>>>>> >>>>>>>>>>> Please review this change: >>>>>>>>>>> >>>>>>>>>>> webrev: >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.00/ >>>>>>>>>>> >>>>>>>>>>> JBS: >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8204531 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> We can use `jhsdb jsnap` to check all PerfData. >>>>>>>>>>> String values in PerfData are defined as jbyte array, but JSnap >>>>>>>>>>> cannot >>>>>>>>>>> handle it well as following: >>>>>>>>>>> >>>>>>>>>>> ``` >>>>>>>>>>> $ jhsdb jsnap --pid 28542 --all | less >>>>>>>>>>> >>>>>>>>>>> sun.gc.cause=No >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> GC^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ >>>>>>>>>>> ``` >>>>>>>>>>> >>>>>>>>>>> You can see this value via `less` and `vim` on Linux. `^@` shows >>>>>>>>>>> it >>>>>>>>>>> is >>>>>>>>>>> non-ascii character. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> So your synopsis says "remove unused chars following \0". Is that >>>>>>>>>> really >>>>>>>>>> what is happening? I would have expected "new String(bytes, >>>>>>>>>> encoding)" >>>>>>>>>> to >>>>>>>>>> stop processing bytes when it encounters a NUL! >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> String c'tor will continue to parse byte array even if it finds >>>>>>>>> '\0'. >>>>>>>>> You can see it on JShell as below: >>>>>>>>> >>>>>>>>> ------------- >>>>>>>>> jshell> var bytes = new byte[]{(byte)'a', (byte)'b', (byte)'c', >>>>>>>>> (byte)'\0', (byte)'d', (byte)'e', (byte)'f'} >>>>>>>>> bytes ==> byte[7] { 97, 98, 99, 0, 100, 101, 102 } >>>>>>>>> >>>>>>>>> jshell> new String(bytes, "US-ASCII") >>>>>>>>> $2 ==> "abc\000def" >>>>>>>>> ------------- >>>>>>>>> >>>>>>>>> >>>>>>>>>>> PerfDataEntry has null-terminated C string. So we should restore >>>>>>>>>>> as >>>>>>>>>>> it >>>>>>>>>>> in >>>>>>>>>>> Java layer. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> If the issue is really "junk" beyond the \0 nul-terminator, how >>>>>>>>>> did >>>>>>>>>> we >>>>>>>>>> get >>>>>>>>>> such values? Shouldn't the byte array already be terminated at the >>>>>>>>>> NUL? >>>>>>>>>> Or >>>>>>>>>> is this just a raw representation of an array in the VM using the >>>>>>>>>> exact >>>>>>>>>> length of the array regardless of content? >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> I think JSnap shows the array regardless of content. >>>>>>>>> >>>>>>>>> >>>>>>>>>> I see CStringUtilities.getString() both stops when it encounters a >>>>>>>>>> 0 >>>>>>>>>> value, >>>>>>>>>> and uses the default file encoding (else ASCII). >>>>>>>>>> >>>>>>>>>> The fix seems perfectly reasonable, I'm just unclear where exactly >>>>>>>>>> this >>>>>>>>>> "bad" data should have been stopped. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks! >>>>>>>>> >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> David >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>> >>>> > From chris.plummer at oracle.com Thu Jun 14 05:01:19 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 13 Jun 2018 22:01:19 -0700 Subject: RFR(s): 8204958: Minor cleanups for the diagnostic framework In-Reply-To: <948d89d5-63c3-24e0-d33c-01e8c8d75547@oracle.com> References: <1a78f599-bf6a-ff72-c687-4af4f474a58d@oracle.com> <5fff0bb9-81e0-498d-53ff-a480d51efff9@oracle.com> <948d89d5-63c3-24e0-d33c-01e8c8d75547@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From thomas.stuefe at gmail.com Thu Jun 14 05:09:23 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 14 Jun 2018 07:09:23 +0200 Subject: RFR(s): 8204958: Minor cleanups for the diagnostic framework In-Reply-To: References: Message-ID: Hi Serguei, On Thu, Jun 14, 2018 at 1:11 AM, serguei.spitsyn at oracle.com wrote: > Hi Thomas, > > I looks good to me. > > A couple of minor comments. > > http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.00/webrev/src/hotspot/share/services/diagnosticArgument.hpp.udiff.html > > - void value_as_str(char *buf, size_t len) { return to_string(_value, buf, > len);} > + void value_as_str(char *buf, size_t len) const { to_string(_value, buf, > len);} > > I'm puzzled a little bit. > How did the value_as_str() returned something before if the function has > void return type? > Yes, I was surprised that this even compiled, but it did. I only did catch it because CDT complained. > > > http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.00/webrev/src/hotspot/share/services/diagnosticFramework.hpp.frames.html > > 170 DCmdArgumentInfo(const char* name, const char* description, const > char* type, > 171 const char* default_string, bool mandatory, bool > option, > 172 bool multiple, int position = -1) > 173 : _name(name), _description(description), _type(type) > 174 , _default_string(default_string), _mandatory(mandatory) > 175 , _option(option), _multiple(multiple), _position(-1) {} > > > Would it be more safe for the _position initialization to be: > _position(position)? > Yes, this is a real bug. Sorry, I'll fix it and comb the rest of the change again. Thanks, Thomas > Thanks, > Serguei > > > > > On 6/13/18 06:41, Thomas St?fe wrote: > > Hi all, > > while working on jcmd I saw some minor cleanup possibilities. > > Webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.00/webrev/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8204958 > > Most of them are const correctness and using initializer lists instead > of java-style member initialization in these C++ classes. > > Other changes: > > - Added a ResourceMark into the DCmd::parse_and_execute() loop > - removed DCmdFactory::create_global_DCmd() (and following that, > DCmdFactory::create_Cheap_instance()) since I did not find anyone > using that function. > - Also in DCmdFactory, I removed a number of setters which were never > called and for attributes which should be immutable: hidden, flags, > enabled. > > This is really a rather light cleanup. It did not touch any larger issues. > > Thank you, > > Thomas > > From david.holmes at oracle.com Thu Jun 14 05:11:22 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 14 Jun 2018 15:11:22 +1000 Subject: RFR(s): 8203343: VM.{metaspace|classloaders|classhierarchy...} jcmd should show invocation targets for Generated{Method|Constructor}AccessorImpl classes In-Reply-To: References: <5a79df03-9001-bff8-7a45-edfb0284c9f1@oracle.com> Message-ID: Hi Thomas, Just a nit in the test: 54 // Do some reflection, enough times to hit the sun.reflect.inflationThreshold and force 55 // generation of reflection accessor classes. ... 59 for (int i = 0; i < 100; i ++) { The default threshold is only 15. The test is already assuming it knows what the threshold is so it may as well only run 15 (16?) times; or else explicitly set the threshold and use a loop count to match. Or even better disable inflation and you don't need a loop at all. I didn't look at anything else. Cheers, David On 13/06/2018 8:32 PM, Thomas St?fe wrote: > Hi Serguei, > > thank you for your review! > > You are right, a regression test makes sense here. I wrote one. See > updated webrev (only added the test, nothing else did change): > > http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/webrev.01/webrev/ > > I am re-running the jdk-submit tests. > > Thanks & Best Regards, Thomas > > On Tue, Jun 12, 2018 at 11:01 PM, serguei.spitsyn at oracle.com > wrote: >> Hi Thomas, >> >> It looks good to me. >> But I'm not an expert in the area of Generated Accessors. >> >> How was this tested? >> Does it make sense to add a unit test for this? >> >> Thanks, >> Serguei >> >> >> >> On 6/6/18 09:05, Thomas St?fe wrote: >>> >>> Dear all, >>> >>> may I please have feedback and if possible reviews for this small >>> addition: >>> >>> CR: https://bugs.openjdk.java.net/browse/JDK-8203343 >>> Webrev: >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/webrev.00/webrev/ >>> >>> (Note: this patch goes atop of >>> https://bugs.openjdk.java.net/browse/JDK-8203682, which is currently >>> up for RFR). >>> >>> --- >>> >>> When analyzing situations involving a lot of reflection, one often >>> stares at walls of "GeneratedXXXAccessorXXX" classes. These names are >>> generated and not at all helpful in analyzing the problem (e.g. which >>> component in a server node does this much reflection and hence uses so >>> much metaspace). >>> >>> This patch adds the ability to print out invocation targets >>> additionally to class names if the class is a generated accessor - for >>> now this ability has been added to VM.metaspace, VM.classloaders and >>> VM.class_hierarchy. >>> >>> Example output from "VM.class_hierarchy": >>> >>> >>> |--jdk.internal.reflect.MagicAccessorImpl/null >>> >>> | |--jdk.internal.reflect.ConstructorAccessorImpl/null >>> | | >>> |--jdk.internal.reflect.GeneratedConstructorAccessor18/0x00007f9ee8350c10 >>> (invokes: >>> org/springframework/boot/context/properties/ConfigurationPropertiesBindingPostProcessorRegistrar:: >>> ()V) >>> | | >>> |--jdk.internal.reflect.GeneratedConstructorAccessor17/0x00007f9ee8349c00 >>> (invokes: >>> org/springframework/boot/context/properties/EnableConfigurationPropertiesImportSelector$ConfigurationPropertiesBeanRegistrar:: >>> ()V) >>> >>> | |--jdk.internal.reflect.MethodAccessorImpl/null >>> | | |--jdk.internal.reflect.GeneratedMethodAccessor23/0x00007f9ec8329b60 >>> (invokes: org/apache/tomcat/util/modeler/AttributeInfo::setIs (Z)V) >>> | | |--jdk.internal.reflect.GeneratedMethodAccessor22/0x00007f9ee831bc70 >>> (invokes: >>> org/springframework/boot/autoconfigure/SpringBootApplication::exclude >>> ()[Ljava/lang/Class;) >>> >>> >>> See here more examples: >>> >>> "VM.class_hierarchy" >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/example-VM.class_hierarchy.txt >>> >>> "VM.classloaders show-classes" >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/example-VM.classloaders.txt >>> >>> >>> ---- >>> >>> Note: I am not sure if this is a fit for hotspot-runtime or >>> serviceability. Sorry for crossposting. >>> >>> Thank you, >>> >>> Thomas >> >> From thomas.stuefe at gmail.com Thu Jun 14 05:13:49 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 14 Jun 2018 07:13:49 +0200 Subject: RFR(s): 8204958: Minor cleanups for the diagnostic framework In-Reply-To: References: <1a78f599-bf6a-ff72-c687-4af4f474a58d@oracle.com> <5fff0bb9-81e0-498d-53ff-a480d51efff9@oracle.com> <948d89d5-63c3-24e0-d33c-01e8c8d75547@oracle.com> Message-ID: Hi Chris, On Thu, Jun 14, 2018 at 7:01 AM, Chris Plummer wrote: > On 6/13/18 9:53 PM, serguei.spitsyn at oracle.com wrote: > > On 6/13/18 21:19, Chris Plummer wrote: > > On 6/13/18 6:11 PM, serguei.spitsyn at oracle.com wrote: > > On 6/13/18 16:44, Chris Plummer wrote: > > On 6/13/18 4:11 PM, serguei.spitsyn at oracle.com wrote: > > Hi Thomas, > > I looks good to me. > > A couple of minor comments. > > http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.00/webrev/src/hotspot/share/services/diagnosticArgument.hpp.udiff.html > > - void value_as_str(char *buf, size_t len) { return to_string(_value, buf, > len);} > + void value_as_str(char *buf, size_t len) const { to_string(_value, buf, > len);} > > I'm puzzled a little bit. > How did the value_as_str() returned something before if the function has > void return type? > > I guess because to_string() returns void. I just did a little experiment and > the following compiles just fine: > > void return_void() { > return; > } > void return_void2() { > return return_void(); > } > > > Thanks, Chris. > I thought about the same. > > http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.00/webrev/src/hotspot/share/services/diagnosticFramework.hpp.frames.html > > 170 DCmdArgumentInfo(const char* name, const char* description, const > char* type, > 171 const char* default_string, bool mandatory, bool > option, > 172 bool multiple, int position = -1) > 173 : _name(name), _description(description), _type(type) > 174 , _default_string(default_string), _mandatory(mandatory) > 175 , _option(option), _multiple(multiple), _position(-1) {} > > > Would it be more safe for the _position initialization to be: > _position(position)? > > I think it's actually a bug and should be _position(position). > > > Agreed. > > Are you Okay with this fix? > > I think it should be _position(position). If the argument is not specified, > it will get the default of -1 as specified in the prototype, which is what > you want. > > > I asked not about this part but about the whole webrev. > Sorry, it was not clear. > > Yes, but I'm always a little hesitant about constifying (ok, so that's > probably not really a word) everything that seems to logically be const. > I've been bit too much in the past with having to cast away the const > because it was not well thought out, but I think it looks pretty safe in the > work that Thomas has done. > Constifying makes the code really a lot more readable, especially to outsiders like me. Since it conveys purpose: which variables are supposed to change, which are immutable attributes? I usually constify in sections, where the interfaces to the outside are slim and one has to cast away the const as seldom as possible, if at all. You are right, when done wrong it can get annoying. ..Thomas > Chris > > > Thanks, > Serguei > > > thanks, > > Chris > > > Thanks, > Serguei > > > Chris > > > Thanks, > Serguei > > > > On 6/13/18 06:41, Thomas St?fe wrote: > > Hi all, > > while working on jcmd I saw some minor cleanup possibilities. > > Webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.00/webrev/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8204958 > > Most of them are const correctness and using initializer lists instead > of java-style member initialization in these C++ classes. > > Other changes: > > - Added a ResourceMark into the DCmd::parse_and_execute() loop > - removed DCmdFactory::create_global_DCmd() (and following that, > DCmdFactory::create_Cheap_instance()) since I did not find anyone > using that function. > - Also in DCmdFactory, I removed a number of setters which were never > called and for attributes which should be immutable: hidden, flags, > enabled. > > This is really a rather light cleanup. It did not touch any larger issues. > > Thank you, > > Thomas > > > > > > > From thomas.stuefe at gmail.com Thu Jun 14 06:01:41 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 14 Jun 2018 08:01:41 +0200 Subject: RFR(s): 8203343: VM.{metaspace|classloaders|classhierarchy...} jcmd should show invocation targets for Generated{Method|Constructor}AccessorImpl classes In-Reply-To: References: <5a79df03-9001-bff8-7a45-edfb0284c9f1@oracle.com> Message-ID: On Thu, Jun 14, 2018 at 7:11 AM, David Holmes wrote: > Hi Thomas, > > Just a nit in the test: > > 54 // Do some reflection, enough times to hit the > sun.reflect.inflationThreshold and force > 55 // generation of reflection accessor classes. > ... > 59 for (int i = 0; i < 100; i ++) { > > The default threshold is only 15. The test is already assuming it knows what > the threshold is so it may as well only run 15 (16?) times; or else > explicitly set the threshold and use a loop count to match. Or even better > disable inflation and you don't need a loop at all. > Yes, you are right, that would be cleaner. I was not sure how to set -Ds.r.noInflation, but I will check if this can be done. Thanks, Thomas > I didn't look at anything else. > > Cheers, > David > > > On 13/06/2018 8:32 PM, Thomas St?fe wrote: >> >> Hi Serguei, >> >> thank you for your review! >> >> You are right, a regression test makes sense here. I wrote one. See >> updated webrev (only added the test, nothing else did change): >> >> >> http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/webrev.01/webrev/ >> >> I am re-running the jdk-submit tests. >> >> Thanks & Best Regards, Thomas >> >> On Tue, Jun 12, 2018 at 11:01 PM, serguei.spitsyn at oracle.com >> wrote: >>> >>> Hi Thomas, >>> >>> It looks good to me. >>> But I'm not an expert in the area of Generated Accessors. >>> >>> How was this tested? >>> Does it make sense to add a unit test for this? >>> >>> Thanks, >>> Serguei >>> >>> >>> >>> On 6/6/18 09:05, Thomas St?fe wrote: >>>> >>>> >>>> Dear all, >>>> >>>> may I please have feedback and if possible reviews for this small >>>> addition: >>>> >>>> CR: https://bugs.openjdk.java.net/browse/JDK-8203343 >>>> Webrev: >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/webrev.00/webrev/ >>>> >>>> (Note: this patch goes atop of >>>> https://bugs.openjdk.java.net/browse/JDK-8203682, which is currently >>>> up for RFR). >>>> >>>> --- >>>> >>>> When analyzing situations involving a lot of reflection, one often >>>> stares at walls of "GeneratedXXXAccessorXXX" classes. These names are >>>> generated and not at all helpful in analyzing the problem (e.g. which >>>> component in a server node does this much reflection and hence uses so >>>> much metaspace). >>>> >>>> This patch adds the ability to print out invocation targets >>>> additionally to class names if the class is a generated accessor - for >>>> now this ability has been added to VM.metaspace, VM.classloaders and >>>> VM.class_hierarchy. >>>> >>>> Example output from "VM.class_hierarchy": >>>> >>>> >>>> |--jdk.internal.reflect.MagicAccessorImpl/null >>>> >>>> | |--jdk.internal.reflect.ConstructorAccessorImpl/null >>>> | | >>>> >>>> |--jdk.internal.reflect.GeneratedConstructorAccessor18/0x00007f9ee8350c10 >>>> (invokes: >>>> >>>> org/springframework/boot/context/properties/ConfigurationPropertiesBindingPostProcessorRegistrar:: >>>> ()V) >>>> | | >>>> >>>> |--jdk.internal.reflect.GeneratedConstructorAccessor17/0x00007f9ee8349c00 >>>> (invokes: >>>> >>>> org/springframework/boot/context/properties/EnableConfigurationPropertiesImportSelector$ConfigurationPropertiesBeanRegistrar:: >>>> ()V) >>>> >>>> | |--jdk.internal.reflect.MethodAccessorImpl/null >>>> | | >>>> |--jdk.internal.reflect.GeneratedMethodAccessor23/0x00007f9ec8329b60 >>>> (invokes: org/apache/tomcat/util/modeler/AttributeInfo::setIs (Z)V) >>>> | | >>>> |--jdk.internal.reflect.GeneratedMethodAccessor22/0x00007f9ee831bc70 >>>> (invokes: >>>> org/springframework/boot/autoconfigure/SpringBootApplication::exclude >>>> ()[Ljava/lang/Class;) >>>> >>>> >>>> See here more examples: >>>> >>>> "VM.class_hierarchy" >>>> >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/example-VM.class_hierarchy.txt >>>> >>>> "VM.classloaders show-classes" >>>> >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/example-VM.classloaders.txt >>>> >>>> >>>> ---- >>>> >>>> Note: I am not sure if this is a fit for hotspot-runtime or >>>> serviceability. Sorry for crossposting. >>>> >>>> Thank you, >>>> >>>> Thomas >>> >>> >>> > From david.holmes at oracle.com Thu Jun 14 06:52:24 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 14 Jun 2018 16:52:24 +1000 Subject: PING: RFR: 8204531: Remove unused chars following '\0' In-Reply-To: References: <6d19c245-eef8-96ce-3ca8-c3786d923c6a@gmail.com> <17c32d23-c2dc-884c-c9ae-15c7afd72bd9@oracle.com> <6d9dab04-6abe-71c0-bbe8-26e6d19c4e9c@gmail.com> <5d3d3768-cf61-6cfd-f631-2a9056d6f172@oracle.com> Message-ID: Seems fine. Thanks, David On 14/06/2018 3:01 PM, Yasumasa Suenaga wrote: > Hi David, > > Thank you for your comment. > > I removed the assertion from PerfDataEntry.java . > Could you review again? > > http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.03/ > > > Thanks, > > Yasuamsa > > > > 2018-06-14 13:41 GMT+09:00 David Holmes : >> Hi Yasumasa, >> >> This seems mostly okay - though like Thomas this is not an area I'm that >> familiar with. But it seems this addresses the problem without impacting >> other things. >> >> This assert is redundant: >> >> 366 if (Assert.ASSERTS_ENABLED) { >> 367 Assert.that(vectorLength() > 0, "not a byte >> vector"); >> >> You can't reach it unless len != 0 and I assume you're not really checking >> that len is not-negative - if you are then the assert needs to go further up >> after: >> >> 328 int len = vectorLength(); >> >> Thanks, >> David >> >> >> On 14/06/2018 2:18 PM, Yasumasa Suenaga wrote: >>> >>> Hi Thomas, >>> >>> Thank you for your comment. >>> I uploaded new webrev: >>> >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.02/ >>> >>> >>> 2018-06-14 3:34 GMT+09:00 Thomas St?fe : >>>> >>>> Hi Yasumasa, >>>> >>>> looks good to me. >>>> >>>> small nit: >>>> import java.nio.charset.* -> import java.nio.charset.Charset; >>> >>> >>> Fixed. >>> >>> >>>> We loose the assert for type now from byteArrayValue(). Can you re-add >>>> it to the location where you call CStringUtils? >>> >>> >>> I added the assertion to check vector length. >>> I did not add type assertion because if-statement branches by data type. >>> >>> >>>> We do not seem to need byteArrayValue(), can this be removed? >>> >>> >>> I cannot agree that. >>> byteArrayValue() is public method (but it is not used AFAICS). >>> However, similar getter methods are privided from PerfDataEntry (e.g. >>> charArrayValue()). So I think we should not remove byteArrayValue(). >>> >>> >>> Regards, >>> >>> Yasumasa >>> >>> >>>> Best Regards, Thomas >>>> >>>> >>>> >>>> On Wed, Jun 13, 2018 at 3:33 PM, Yasumasa Suenaga >>>> wrote: >>>>> >>>>> Thanks Thomas, >>>>> >>>>>> That said, I'd probably just add a variant to >>>>>> CStringUtilities::getString which takes encoding as an argument, and >>>>>> then pass "US-ASCII" to remain compatible with the current version of >>>>>> jsnap. >>>>> >>>>> >>>>> >>>>> I agree with you. My opinion is one of the ideals. >>>>> Anyway, I uploaded new webrev which is added charset support in >>>>> CStringUtils: >>>>> >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.01/ >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> >>>>> On 2018/06/13 21:46, Thomas St?fe wrote: >>>>>> >>>>>> >>>>>> Disclaimer: I am Reviewer but not versed in this corner of the jdk, >>>>>> nor have knowledge of the history, so take what I suggest with a grain >>>>>> of salt. >>>>>> >>>>>> That said, I'd probably just add a variant to >>>>>> CStringUtilities::getString which takes encoding as an argument, and >>>>>> then pass "US-ASCII" to remain compatible with the current version of >>>>>> jsnap. >>>>>> >>>>>> ..Thomas >>>>>> >>>>>> On Wed, Jun 13, 2018 at 2:37 PM, Yasumasa Suenaga >>>>>> wrote: >>>>>>> >>>>>>> >>>>>>> Hi Thomas, >>>>>>> >>>>>>> Thank you for your comment. >>>>>>> >>>>>>> CStringUtilities.java says the String which is constructed from byte >>>>>>> array >>>>>>> should use Charset.defaultCharset(). [1] >>>>>>> I'm not convinced how should we fix - US-ASCII or default charset. >>>>>>> >>>>>>> IMHO we should use default charset. However I concern I need to get >>>>>>> CSR >>>>>>> approvement. >>>>>>> (In addition, I want to refactor CStringUtilities. I guess we can make >>>>>>> more >>>>>>> simple code :-) >>>>>>> >>>>>>> >>>>>>> What do you think about it? >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> [1] >>>>>>> >>>>>>> >>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/7bf4f1b5e438/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/utilities/CStringUtilities.java#l73 >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 2018/06/13 21:18, Thomas St?fe wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Hi Yasumasa, >>>>>>>> >>>>>>>> your patch will change behavior of jsnap if file.encoding is set to >>>>>>>> something different than US-ASCII. >>>>>>>> >>>>>>>> Old version uses, hard wired, US-ASCII. >>>>>>>> >>>>>>>> You now use jvm/hotspot/utilities/CStringUtilities.java, which uses >>>>>>>> >>>>>>>> private static String encoding = System.getProperty("file.encoding", >>>>>>>> "US-ASCII"); >>>>>>>> >>>>>>>> and thus may use a different encoding. >>>>>>>> >>>>>>>> Best Regards, Thomas >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Jun 13, 2018 at 2:09 PM, Yasumasa Suenaga >>>>>>>> >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> PING: Could you review it? >>>>>>>>> I want to merge this change before JDK 11 RDP 1. >>>>>>>>> >>>>>>>>>>>> webrev: >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.00/ >>>>>>>>>>>> >>>>>>>>>>>> JBS: >>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8204531 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2018/06/08 15:38, Yasumasa Suenaga wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hi David, >>>>>>>>>> >>>>>>>>>> 2018-06-08 10:47 GMT+09:00 David Holmes : >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>> >>>>>>>>>>> On 7/06/2018 10:43 PM, Yasumasa Suenaga wrote: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Hi all, >>>>>>>>>>>> >>>>>>>>>>>> Please review this change: >>>>>>>>>>>> >>>>>>>>>>>> webrev: >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.00/ >>>>>>>>>>>> >>>>>>>>>>>> JBS: >>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8204531 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> We can use `jhsdb jsnap` to check all PerfData. >>>>>>>>>>>> String values in PerfData are defined as jbyte array, but JSnap >>>>>>>>>>>> cannot >>>>>>>>>>>> handle it well as following: >>>>>>>>>>>> >>>>>>>>>>>> ``` >>>>>>>>>>>> $ jhsdb jsnap --pid 28542 --all | less >>>>>>>>>>>> >>>>>>>>>>>> sun.gc.cause=No >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> GC^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ >>>>>>>>>>>> ``` >>>>>>>>>>>> >>>>>>>>>>>> You can see this value via `less` and `vim` on Linux. `^@` shows >>>>>>>>>>>> it >>>>>>>>>>>> is >>>>>>>>>>>> non-ascii character. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> So your synopsis says "remove unused chars following \0". Is that >>>>>>>>>>> really >>>>>>>>>>> what is happening? I would have expected "new String(bytes, >>>>>>>>>>> encoding)" >>>>>>>>>>> to >>>>>>>>>>> stop processing bytes when it encounters a NUL! >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> String c'tor will continue to parse byte array even if it finds >>>>>>>>>> '\0'. >>>>>>>>>> You can see it on JShell as below: >>>>>>>>>> >>>>>>>>>> ------------- >>>>>>>>>> jshell> var bytes = new byte[]{(byte)'a', (byte)'b', (byte)'c', >>>>>>>>>> (byte)'\0', (byte)'d', (byte)'e', (byte)'f'} >>>>>>>>>> bytes ==> byte[7] { 97, 98, 99, 0, 100, 101, 102 } >>>>>>>>>> >>>>>>>>>> jshell> new String(bytes, "US-ASCII") >>>>>>>>>> $2 ==> "abc\000def" >>>>>>>>>> ------------- >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>>> PerfDataEntry has null-terminated C string. So we should restore >>>>>>>>>>>> as >>>>>>>>>>>> it >>>>>>>>>>>> in >>>>>>>>>>>> Java layer. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> If the issue is really "junk" beyond the \0 nul-terminator, how >>>>>>>>>>> did >>>>>>>>>>> we >>>>>>>>>>> get >>>>>>>>>>> such values? Shouldn't the byte array already be terminated at the >>>>>>>>>>> NUL? >>>>>>>>>>> Or >>>>>>>>>>> is this just a raw representation of an array in the VM using the >>>>>>>>>>> exact >>>>>>>>>>> length of the array regardless of content? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> I think JSnap shows the array regardless of content. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> I see CStringUtilities.getString() both stops when it encounters a >>>>>>>>>>> 0 >>>>>>>>>>> value, >>>>>>>>>>> and uses the default file encoding (else ASCII). >>>>>>>>>>> >>>>>>>>>>> The fix seems perfectly reasonable, I'm just unclear where exactly >>>>>>>>>>> this >>>>>>>>>>> "bad" data should have been stopped. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks! >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> David >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>> >>>>> >> From yasuenag at gmail.com Thu Jun 14 07:37:07 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Thu, 14 Jun 2018 16:37:07 +0900 Subject: PING: RFR: 8204531: Remove unused chars following '\0' In-Reply-To: References: <6d19c245-eef8-96ce-3ca8-c3786d923c6a@gmail.com> <17c32d23-c2dc-884c-c9ae-15c7afd72bd9@oracle.com> <6d9dab04-6abe-71c0-bbe8-26e6d19c4e9c@gmail.com> <5d3d3768-cf61-6cfd-f631-2a9056d6f172@oracle.com> Message-ID: Thanks David! I'm waiting for second reviewer. Yasumasa 2018-06-14 15:52 GMT+09:00 David Holmes : > Seems fine. > > Thanks, > David > > > On 14/06/2018 3:01 PM, Yasumasa Suenaga wrote: >> >> Hi David, >> >> Thank you for your comment. >> >> I removed the assertion from PerfDataEntry.java . >> Could you review again? >> >> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.03/ >> >> >> Thanks, >> >> Yasuamsa >> >> >> >> 2018-06-14 13:41 GMT+09:00 David Holmes : >>> >>> Hi Yasumasa, >>> >>> This seems mostly okay - though like Thomas this is not an area I'm that >>> familiar with. But it seems this addresses the problem without impacting >>> other things. >>> >>> This assert is redundant: >>> >>> 366 if (Assert.ASSERTS_ENABLED) { >>> 367 Assert.that(vectorLength() > 0, "not a byte >>> vector"); >>> >>> You can't reach it unless len != 0 and I assume you're not really >>> checking >>> that len is not-negative - if you are then the assert needs to go further >>> up >>> after: >>> >>> 328 int len = vectorLength(); >>> >>> Thanks, >>> David >>> >>> >>> On 14/06/2018 2:18 PM, Yasumasa Suenaga wrote: >>>> >>>> >>>> Hi Thomas, >>>> >>>> Thank you for your comment. >>>> I uploaded new webrev: >>>> >>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.02/ >>>> >>>> >>>> 2018-06-14 3:34 GMT+09:00 Thomas St?fe : >>>>> >>>>> >>>>> Hi Yasumasa, >>>>> >>>>> looks good to me. >>>>> >>>>> small nit: >>>>> import java.nio.charset.* -> import java.nio.charset.Charset; >>>> >>>> >>>> >>>> Fixed. >>>> >>>> >>>>> We loose the assert for type now from byteArrayValue(). Can you re-add >>>>> it to the location where you call CStringUtils? >>>> >>>> >>>> >>>> I added the assertion to check vector length. >>>> I did not add type assertion because if-statement branches by data type. >>>> >>>> >>>>> We do not seem to need byteArrayValue(), can this be removed? >>>> >>>> >>>> >>>> I cannot agree that. >>>> byteArrayValue() is public method (but it is not used AFAICS). >>>> However, similar getter methods are privided from PerfDataEntry (e.g. >>>> charArrayValue()). So I think we should not remove byteArrayValue(). >>>> >>>> >>>> Regards, >>>> >>>> Yasumasa >>>> >>>> >>>>> Best Regards, Thomas >>>>> >>>>> >>>>> >>>>> On Wed, Jun 13, 2018 at 3:33 PM, Yasumasa Suenaga >>>>> wrote: >>>>>> >>>>>> >>>>>> Thanks Thomas, >>>>>> >>>>>>> That said, I'd probably just add a variant to >>>>>>> CStringUtilities::getString which takes encoding as an argument, and >>>>>>> then pass "US-ASCII" to remain compatible with the current version of >>>>>>> jsnap. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> I agree with you. My opinion is one of the ideals. >>>>>> Anyway, I uploaded new webrev which is added charset support in >>>>>> CStringUtils: >>>>>> >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.01/ >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> >>>>>> On 2018/06/13 21:46, Thomas St?fe wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> Disclaimer: I am Reviewer but not versed in this corner of the jdk, >>>>>>> nor have knowledge of the history, so take what I suggest with a >>>>>>> grain >>>>>>> of salt. >>>>>>> >>>>>>> That said, I'd probably just add a variant to >>>>>>> CStringUtilities::getString which takes encoding as an argument, and >>>>>>> then pass "US-ASCII" to remain compatible with the current version of >>>>>>> jsnap. >>>>>>> >>>>>>> ..Thomas >>>>>>> >>>>>>> On Wed, Jun 13, 2018 at 2:37 PM, Yasumasa Suenaga >>>>>>> >>>>>>> wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Hi Thomas, >>>>>>>> >>>>>>>> Thank you for your comment. >>>>>>>> >>>>>>>> CStringUtilities.java says the String which is constructed from byte >>>>>>>> array >>>>>>>> should use Charset.defaultCharset(). [1] >>>>>>>> I'm not convinced how should we fix - US-ASCII or default charset. >>>>>>>> >>>>>>>> IMHO we should use default charset. However I concern I need to get >>>>>>>> CSR >>>>>>>> approvement. >>>>>>>> (In addition, I want to refactor CStringUtilities. I guess we can >>>>>>>> make >>>>>>>> more >>>>>>>> simple code :-) >>>>>>>> >>>>>>>> >>>>>>>> What do you think about it? >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> [1] >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/7bf4f1b5e438/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/utilities/CStringUtilities.java#l73 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 2018/06/13 21:18, Thomas St?fe wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Hi Yasumasa, >>>>>>>>> >>>>>>>>> your patch will change behavior of jsnap if file.encoding is set to >>>>>>>>> something different than US-ASCII. >>>>>>>>> >>>>>>>>> Old version uses, hard wired, US-ASCII. >>>>>>>>> >>>>>>>>> You now use jvm/hotspot/utilities/CStringUtilities.java, which uses >>>>>>>>> >>>>>>>>> private static String encoding = >>>>>>>>> System.getProperty("file.encoding", >>>>>>>>> "US-ASCII"); >>>>>>>>> >>>>>>>>> and thus may use a different encoding. >>>>>>>>> >>>>>>>>> Best Regards, Thomas >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Wed, Jun 13, 2018 at 2:09 PM, Yasumasa Suenaga >>>>>>>>> >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> PING: Could you review it? >>>>>>>>>> I want to merge this change before JDK 11 RDP 1. >>>>>>>>>> >>>>>>>>>>>>> webrev: >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.00/ >>>>>>>>>>>>> >>>>>>>>>>>>> JBS: >>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8204531 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2018/06/08 15:38, Yasumasa Suenaga wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hi David, >>>>>>>>>>> >>>>>>>>>>> 2018-06-08 10:47 GMT+09:00 David Holmes >>>>>>>>>>> : >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>> >>>>>>>>>>>> On 7/06/2018 10:43 PM, Yasumasa Suenaga wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Hi all, >>>>>>>>>>>>> >>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>> >>>>>>>>>>>>> webrev: >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.00/ >>>>>>>>>>>>> >>>>>>>>>>>>> JBS: >>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8204531 >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> We can use `jhsdb jsnap` to check all PerfData. >>>>>>>>>>>>> String values in PerfData are defined as jbyte array, but JSnap >>>>>>>>>>>>> cannot >>>>>>>>>>>>> handle it well as following: >>>>>>>>>>>>> >>>>>>>>>>>>> ``` >>>>>>>>>>>>> $ jhsdb jsnap --pid 28542 --all | less >>>>>>>>>>>>> >>>>>>>>>>>>> sun.gc.cause=No >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> GC^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ >>>>>>>>>>>>> ``` >>>>>>>>>>>>> >>>>>>>>>>>>> You can see this value via `less` and `vim` on Linux. `^@` >>>>>>>>>>>>> shows >>>>>>>>>>>>> it >>>>>>>>>>>>> is >>>>>>>>>>>>> non-ascii character. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> So your synopsis says "remove unused chars following \0". Is >>>>>>>>>>>> that >>>>>>>>>>>> really >>>>>>>>>>>> what is happening? I would have expected "new String(bytes, >>>>>>>>>>>> encoding)" >>>>>>>>>>>> to >>>>>>>>>>>> stop processing bytes when it encounters a NUL! >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> String c'tor will continue to parse byte array even if it finds >>>>>>>>>>> '\0'. >>>>>>>>>>> You can see it on JShell as below: >>>>>>>>>>> >>>>>>>>>>> ------------- >>>>>>>>>>> jshell> var bytes = new byte[]{(byte)'a', (byte)'b', (byte)'c', >>>>>>>>>>> (byte)'\0', (byte)'d', (byte)'e', (byte)'f'} >>>>>>>>>>> bytes ==> byte[7] { 97, 98, 99, 0, 100, 101, 102 } >>>>>>>>>>> >>>>>>>>>>> jshell> new String(bytes, "US-ASCII") >>>>>>>>>>> $2 ==> "abc\000def" >>>>>>>>>>> ------------- >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>>> PerfDataEntry has null-terminated C string. So we should >>>>>>>>>>>>> restore >>>>>>>>>>>>> as >>>>>>>>>>>>> it >>>>>>>>>>>>> in >>>>>>>>>>>>> Java layer. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> If the issue is really "junk" beyond the \0 nul-terminator, how >>>>>>>>>>>> did >>>>>>>>>>>> we >>>>>>>>>>>> get >>>>>>>>>>>> such values? Shouldn't the byte array already be terminated at >>>>>>>>>>>> the >>>>>>>>>>>> NUL? >>>>>>>>>>>> Or >>>>>>>>>>>> is this just a raw representation of an array in the VM using >>>>>>>>>>>> the >>>>>>>>>>>> exact >>>>>>>>>>>> length of the array regardless of content? >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I think JSnap shows the array regardless of content. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> I see CStringUtilities.getString() both stops when it encounters >>>>>>>>>>>> a >>>>>>>>>>>> 0 >>>>>>>>>>>> value, >>>>>>>>>>>> and uses the default file encoding (else ASCII). >>>>>>>>>>>> >>>>>>>>>>>> The fix seems perfectly reasonable, I'm just unclear where >>>>>>>>>>>> exactly >>>>>>>>>>>> this >>>>>>>>>>>> "bad" data should have been stopped. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks! >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> David >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>> >>> > From thomas.stuefe at gmail.com Thu Jun 14 07:45:37 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 14 Jun 2018 09:45:37 +0200 Subject: PING: RFR: 8204531: Remove unused chars following '\0' In-Reply-To: References: <6d19c245-eef8-96ce-3ca8-c3786d923c6a@gmail.com> <17c32d23-c2dc-884c-c9ae-15c7afd72bd9@oracle.com> <6d9dab04-6abe-71c0-bbe8-26e6d19c4e9c@gmail.com> <5d3d3768-cf61-6cfd-f631-2a9056d6f172@oracle.com> Message-ID: Hi Yasumasa, this is fine for me. Thanks for fixing! ..Thomas On Thu, Jun 14, 2018 at 9:37 AM, Yasumasa Suenaga wrote: > Thanks David! > I'm waiting for second reviewer. > > > Yasumasa > > > 2018-06-14 15:52 GMT+09:00 David Holmes : >> Seems fine. >> >> Thanks, >> David >> >> >> On 14/06/2018 3:01 PM, Yasumasa Suenaga wrote: >>> >>> Hi David, >>> >>> Thank you for your comment. >>> >>> I removed the assertion from PerfDataEntry.java . >>> Could you review again? >>> >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.03/ >>> >>> >>> Thanks, >>> >>> Yasuamsa >>> >>> >>> >>> 2018-06-14 13:41 GMT+09:00 David Holmes : >>>> >>>> Hi Yasumasa, >>>> >>>> This seems mostly okay - though like Thomas this is not an area I'm that >>>> familiar with. But it seems this addresses the problem without impacting >>>> other things. >>>> >>>> This assert is redundant: >>>> >>>> 366 if (Assert.ASSERTS_ENABLED) { >>>> 367 Assert.that(vectorLength() > 0, "not a byte >>>> vector"); >>>> >>>> You can't reach it unless len != 0 and I assume you're not really >>>> checking >>>> that len is not-negative - if you are then the assert needs to go further >>>> up >>>> after: >>>> >>>> 328 int len = vectorLength(); >>>> >>>> Thanks, >>>> David >>>> >>>> >>>> On 14/06/2018 2:18 PM, Yasumasa Suenaga wrote: >>>>> >>>>> >>>>> Hi Thomas, >>>>> >>>>> Thank you for your comment. >>>>> I uploaded new webrev: >>>>> >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.02/ >>>>> >>>>> >>>>> 2018-06-14 3:34 GMT+09:00 Thomas St?fe : >>>>>> >>>>>> >>>>>> Hi Yasumasa, >>>>>> >>>>>> looks good to me. >>>>>> >>>>>> small nit: >>>>>> import java.nio.charset.* -> import java.nio.charset.Charset; >>>>> >>>>> >>>>> >>>>> Fixed. >>>>> >>>>> >>>>>> We loose the assert for type now from byteArrayValue(). Can you re-add >>>>>> it to the location where you call CStringUtils? >>>>> >>>>> >>>>> >>>>> I added the assertion to check vector length. >>>>> I did not add type assertion because if-statement branches by data type. >>>>> >>>>> >>>>>> We do not seem to need byteArrayValue(), can this be removed? >>>>> >>>>> >>>>> >>>>> I cannot agree that. >>>>> byteArrayValue() is public method (but it is not used AFAICS). >>>>> However, similar getter methods are privided from PerfDataEntry (e.g. >>>>> charArrayValue()). So I think we should not remove byteArrayValue(). >>>>> >>>>> >>>>> Regards, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>>> Best Regards, Thomas >>>>>> >>>>>> >>>>>> >>>>>> On Wed, Jun 13, 2018 at 3:33 PM, Yasumasa Suenaga >>>>>> wrote: >>>>>>> >>>>>>> >>>>>>> Thanks Thomas, >>>>>>> >>>>>>>> That said, I'd probably just add a variant to >>>>>>>> CStringUtilities::getString which takes encoding as an argument, and >>>>>>>> then pass "US-ASCII" to remain compatible with the current version of >>>>>>>> jsnap. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> I agree with you. My opinion is one of the ideals. >>>>>>> Anyway, I uploaded new webrev which is added charset support in >>>>>>> CStringUtils: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.01/ >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 2018/06/13 21:46, Thomas St?fe wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Disclaimer: I am Reviewer but not versed in this corner of the jdk, >>>>>>>> nor have knowledge of the history, so take what I suggest with a >>>>>>>> grain >>>>>>>> of salt. >>>>>>>> >>>>>>>> That said, I'd probably just add a variant to >>>>>>>> CStringUtilities::getString which takes encoding as an argument, and >>>>>>>> then pass "US-ASCII" to remain compatible with the current version of >>>>>>>> jsnap. >>>>>>>> >>>>>>>> ..Thomas >>>>>>>> >>>>>>>> On Wed, Jun 13, 2018 at 2:37 PM, Yasumasa Suenaga >>>>>>>> >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Hi Thomas, >>>>>>>>> >>>>>>>>> Thank you for your comment. >>>>>>>>> >>>>>>>>> CStringUtilities.java says the String which is constructed from byte >>>>>>>>> array >>>>>>>>> should use Charset.defaultCharset(). [1] >>>>>>>>> I'm not convinced how should we fix - US-ASCII or default charset. >>>>>>>>> >>>>>>>>> IMHO we should use default charset. However I concern I need to get >>>>>>>>> CSR >>>>>>>>> approvement. >>>>>>>>> (In addition, I want to refactor CStringUtilities. I guess we can >>>>>>>>> make >>>>>>>>> more >>>>>>>>> simple code :-) >>>>>>>>> >>>>>>>>> >>>>>>>>> What do you think about it? >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> [1] >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/7bf4f1b5e438/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/utilities/CStringUtilities.java#l73 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2018/06/13 21:18, Thomas St?fe wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hi Yasumasa, >>>>>>>>>> >>>>>>>>>> your patch will change behavior of jsnap if file.encoding is set to >>>>>>>>>> something different than US-ASCII. >>>>>>>>>> >>>>>>>>>> Old version uses, hard wired, US-ASCII. >>>>>>>>>> >>>>>>>>>> You now use jvm/hotspot/utilities/CStringUtilities.java, which uses >>>>>>>>>> >>>>>>>>>> private static String encoding = >>>>>>>>>> System.getProperty("file.encoding", >>>>>>>>>> "US-ASCII"); >>>>>>>>>> >>>>>>>>>> and thus may use a different encoding. >>>>>>>>>> >>>>>>>>>> Best Regards, Thomas >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Wed, Jun 13, 2018 at 2:09 PM, Yasumasa Suenaga >>>>>>>>>> >>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> PING: Could you review it? >>>>>>>>>>> I want to merge this change before JDK 11 RDP 1. >>>>>>>>>>> >>>>>>>>>>>>>> webrev: >>>>>>>>>>>>>> >>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.00/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> JBS: >>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8204531 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2018/06/08 15:38, Yasumasa Suenaga wrote: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Hi David, >>>>>>>>>>>> >>>>>>>>>>>> 2018-06-08 10:47 GMT+09:00 David Holmes >>>>>>>>>>>> : >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>> >>>>>>>>>>>>> On 7/06/2018 10:43 PM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>> >>>>>>>>>>>>>> webrev: >>>>>>>>>>>>>> >>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.00/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> JBS: >>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8204531 >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> We can use `jhsdb jsnap` to check all PerfData. >>>>>>>>>>>>>> String values in PerfData are defined as jbyte array, but JSnap >>>>>>>>>>>>>> cannot >>>>>>>>>>>>>> handle it well as following: >>>>>>>>>>>>>> >>>>>>>>>>>>>> ``` >>>>>>>>>>>>>> $ jhsdb jsnap --pid 28542 --all | less >>>>>>>>>>>>>> >>>>>>>>>>>>>> sun.gc.cause=No >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> GC^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ >>>>>>>>>>>>>> ``` >>>>>>>>>>>>>> >>>>>>>>>>>>>> You can see this value via `less` and `vim` on Linux. `^@` >>>>>>>>>>>>>> shows >>>>>>>>>>>>>> it >>>>>>>>>>>>>> is >>>>>>>>>>>>>> non-ascii character. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> So your synopsis says "remove unused chars following \0". Is >>>>>>>>>>>>> that >>>>>>>>>>>>> really >>>>>>>>>>>>> what is happening? I would have expected "new String(bytes, >>>>>>>>>>>>> encoding)" >>>>>>>>>>>>> to >>>>>>>>>>>>> stop processing bytes when it encounters a NUL! >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> String c'tor will continue to parse byte array even if it finds >>>>>>>>>>>> '\0'. >>>>>>>>>>>> You can see it on JShell as below: >>>>>>>>>>>> >>>>>>>>>>>> ------------- >>>>>>>>>>>> jshell> var bytes = new byte[]{(byte)'a', (byte)'b', (byte)'c', >>>>>>>>>>>> (byte)'\0', (byte)'d', (byte)'e', (byte)'f'} >>>>>>>>>>>> bytes ==> byte[7] { 97, 98, 99, 0, 100, 101, 102 } >>>>>>>>>>>> >>>>>>>>>>>> jshell> new String(bytes, "US-ASCII") >>>>>>>>>>>> $2 ==> "abc\000def" >>>>>>>>>>>> ------------- >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>>> PerfDataEntry has null-terminated C string. So we should >>>>>>>>>>>>>> restore >>>>>>>>>>>>>> as >>>>>>>>>>>>>> it >>>>>>>>>>>>>> in >>>>>>>>>>>>>> Java layer. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> If the issue is really "junk" beyond the \0 nul-terminator, how >>>>>>>>>>>>> did >>>>>>>>>>>>> we >>>>>>>>>>>>> get >>>>>>>>>>>>> such values? Shouldn't the byte array already be terminated at >>>>>>>>>>>>> the >>>>>>>>>>>>> NUL? >>>>>>>>>>>>> Or >>>>>>>>>>>>> is this just a raw representation of an array in the VM using >>>>>>>>>>>>> the >>>>>>>>>>>>> exact >>>>>>>>>>>>> length of the array regardless of content? >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I think JSnap shows the array regardless of content. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> I see CStringUtilities.getString() both stops when it encounters >>>>>>>>>>>>> a >>>>>>>>>>>>> 0 >>>>>>>>>>>>> value, >>>>>>>>>>>>> and uses the default file encoding (else ASCII). >>>>>>>>>>>>> >>>>>>>>>>>>> The fix seems perfectly reasonable, I'm just unclear where >>>>>>>>>>>>> exactly >>>>>>>>>>>>> this >>>>>>>>>>>>> "bad" data should have been stopped. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks! >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> David >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>> >>>> >> From yasuenag at gmail.com Thu Jun 14 07:54:13 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Thu, 14 Jun 2018 16:54:13 +0900 Subject: PING: RFR: 8204531: Remove unused chars following '\0' In-Reply-To: References: <6d19c245-eef8-96ce-3ca8-c3786d923c6a@gmail.com> <17c32d23-c2dc-884c-c9ae-15c7afd72bd9@oracle.com> <6d9dab04-6abe-71c0-bbe8-26e6d19c4e9c@gmail.com> <5d3d3768-cf61-6cfd-f631-2a9056d6f172@oracle.com> Message-ID: Thanks Thomas! Yasumasa 2018-06-14 16:45 GMT+09:00 Thomas St?fe : > Hi Yasumasa, > > this is fine for me. Thanks for fixing! > > ..Thomas > > On Thu, Jun 14, 2018 at 9:37 AM, Yasumasa Suenaga wrote: >> Thanks David! >> I'm waiting for second reviewer. >> >> >> Yasumasa >> >> >> 2018-06-14 15:52 GMT+09:00 David Holmes : >>> Seems fine. >>> >>> Thanks, >>> David >>> >>> >>> On 14/06/2018 3:01 PM, Yasumasa Suenaga wrote: >>>> >>>> Hi David, >>>> >>>> Thank you for your comment. >>>> >>>> I removed the assertion from PerfDataEntry.java . >>>> Could you review again? >>>> >>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.03/ >>>> >>>> >>>> Thanks, >>>> >>>> Yasuamsa >>>> >>>> >>>> >>>> 2018-06-14 13:41 GMT+09:00 David Holmes : >>>>> >>>>> Hi Yasumasa, >>>>> >>>>> This seems mostly okay - though like Thomas this is not an area I'm that >>>>> familiar with. But it seems this addresses the problem without impacting >>>>> other things. >>>>> >>>>> This assert is redundant: >>>>> >>>>> 366 if (Assert.ASSERTS_ENABLED) { >>>>> 367 Assert.that(vectorLength() > 0, "not a byte >>>>> vector"); >>>>> >>>>> You can't reach it unless len != 0 and I assume you're not really >>>>> checking >>>>> that len is not-negative - if you are then the assert needs to go further >>>>> up >>>>> after: >>>>> >>>>> 328 int len = vectorLength(); >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>> >>>>> On 14/06/2018 2:18 PM, Yasumasa Suenaga wrote: >>>>>> >>>>>> >>>>>> Hi Thomas, >>>>>> >>>>>> Thank you for your comment. >>>>>> I uploaded new webrev: >>>>>> >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.02/ >>>>>> >>>>>> >>>>>> 2018-06-14 3:34 GMT+09:00 Thomas St?fe : >>>>>>> >>>>>>> >>>>>>> Hi Yasumasa, >>>>>>> >>>>>>> looks good to me. >>>>>>> >>>>>>> small nit: >>>>>>> import java.nio.charset.* -> import java.nio.charset.Charset; >>>>>> >>>>>> >>>>>> >>>>>> Fixed. >>>>>> >>>>>> >>>>>>> We loose the assert for type now from byteArrayValue(). Can you re-add >>>>>>> it to the location where you call CStringUtils? >>>>>> >>>>>> >>>>>> >>>>>> I added the assertion to check vector length. >>>>>> I did not add type assertion because if-statement branches by data type. >>>>>> >>>>>> >>>>>>> We do not seem to need byteArrayValue(), can this be removed? >>>>>> >>>>>> >>>>>> >>>>>> I cannot agree that. >>>>>> byteArrayValue() is public method (but it is not used AFAICS). >>>>>> However, similar getter methods are privided from PerfDataEntry (e.g. >>>>>> charArrayValue()). So I think we should not remove byteArrayValue(). >>>>>> >>>>>> >>>>>> Regards, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>>> Best Regards, Thomas >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Wed, Jun 13, 2018 at 3:33 PM, Yasumasa Suenaga >>>>>>> wrote: >>>>>>>> >>>>>>>> >>>>>>>> Thanks Thomas, >>>>>>>> >>>>>>>>> That said, I'd probably just add a variant to >>>>>>>>> CStringUtilities::getString which takes encoding as an argument, and >>>>>>>>> then pass "US-ASCII" to remain compatible with the current version of >>>>>>>>> jsnap. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> I agree with you. My opinion is one of the ideals. >>>>>>>> Anyway, I uploaded new webrev which is added charset support in >>>>>>>> CStringUtils: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.01/ >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 2018/06/13 21:46, Thomas St?fe wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Disclaimer: I am Reviewer but not versed in this corner of the jdk, >>>>>>>>> nor have knowledge of the history, so take what I suggest with a >>>>>>>>> grain >>>>>>>>> of salt. >>>>>>>>> >>>>>>>>> That said, I'd probably just add a variant to >>>>>>>>> CStringUtilities::getString which takes encoding as an argument, and >>>>>>>>> then pass "US-ASCII" to remain compatible with the current version of >>>>>>>>> jsnap. >>>>>>>>> >>>>>>>>> ..Thomas >>>>>>>>> >>>>>>>>> On Wed, Jun 13, 2018 at 2:37 PM, Yasumasa Suenaga >>>>>>>>> >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hi Thomas, >>>>>>>>>> >>>>>>>>>> Thank you for your comment. >>>>>>>>>> >>>>>>>>>> CStringUtilities.java says the String which is constructed from byte >>>>>>>>>> array >>>>>>>>>> should use Charset.defaultCharset(). [1] >>>>>>>>>> I'm not convinced how should we fix - US-ASCII or default charset. >>>>>>>>>> >>>>>>>>>> IMHO we should use default charset. However I concern I need to get >>>>>>>>>> CSR >>>>>>>>>> approvement. >>>>>>>>>> (In addition, I want to refactor CStringUtilities. I guess we can >>>>>>>>>> make >>>>>>>>>> more >>>>>>>>>> simple code :-) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> What do you think about it? >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> [1] >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/7bf4f1b5e438/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/utilities/CStringUtilities.java#l73 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2018/06/13 21:18, Thomas St?fe wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>> >>>>>>>>>>> your patch will change behavior of jsnap if file.encoding is set to >>>>>>>>>>> something different than US-ASCII. >>>>>>>>>>> >>>>>>>>>>> Old version uses, hard wired, US-ASCII. >>>>>>>>>>> >>>>>>>>>>> You now use jvm/hotspot/utilities/CStringUtilities.java, which uses >>>>>>>>>>> >>>>>>>>>>> private static String encoding = >>>>>>>>>>> System.getProperty("file.encoding", >>>>>>>>>>> "US-ASCII"); >>>>>>>>>>> >>>>>>>>>>> and thus may use a different encoding. >>>>>>>>>>> >>>>>>>>>>> Best Regards, Thomas >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Wed, Jun 13, 2018 at 2:09 PM, Yasumasa Suenaga >>>>>>>>>>> >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> PING: Could you review it? >>>>>>>>>>>> I want to merge this change before JDK 11 RDP 1. >>>>>>>>>>>> >>>>>>>>>>>>>>> webrev: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.00/ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> JBS: >>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8204531 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 2018/06/08 15:38, Yasumasa Suenaga wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Hi David, >>>>>>>>>>>>> >>>>>>>>>>>>> 2018-06-08 10:47 GMT+09:00 David Holmes >>>>>>>>>>>>> : >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 7/06/2018 10:43 PM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Please review this change: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> webrev: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8204531/webrev.00/ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> JBS: >>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8204531 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> We can use `jhsdb jsnap` to check all PerfData. >>>>>>>>>>>>>>> String values in PerfData are defined as jbyte array, but JSnap >>>>>>>>>>>>>>> cannot >>>>>>>>>>>>>>> handle it well as following: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>> $ jhsdb jsnap --pid 28542 --all | less >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> sun.gc.cause=No >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> GC^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ >>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> You can see this value via `less` and `vim` on Linux. `^@` >>>>>>>>>>>>>>> shows >>>>>>>>>>>>>>> it >>>>>>>>>>>>>>> is >>>>>>>>>>>>>>> non-ascii character. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> So your synopsis says "remove unused chars following \0". Is >>>>>>>>>>>>>> that >>>>>>>>>>>>>> really >>>>>>>>>>>>>> what is happening? I would have expected "new String(bytes, >>>>>>>>>>>>>> encoding)" >>>>>>>>>>>>>> to >>>>>>>>>>>>>> stop processing bytes when it encounters a NUL! >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> String c'tor will continue to parse byte array even if it finds >>>>>>>>>>>>> '\0'. >>>>>>>>>>>>> You can see it on JShell as below: >>>>>>>>>>>>> >>>>>>>>>>>>> ------------- >>>>>>>>>>>>> jshell> var bytes = new byte[]{(byte)'a', (byte)'b', (byte)'c', >>>>>>>>>>>>> (byte)'\0', (byte)'d', (byte)'e', (byte)'f'} >>>>>>>>>>>>> bytes ==> byte[7] { 97, 98, 99, 0, 100, 101, 102 } >>>>>>>>>>>>> >>>>>>>>>>>>> jshell> new String(bytes, "US-ASCII") >>>>>>>>>>>>> $2 ==> "abc\000def" >>>>>>>>>>>>> ------------- >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>>> PerfDataEntry has null-terminated C string. So we should >>>>>>>>>>>>>>> restore >>>>>>>>>>>>>>> as >>>>>>>>>>>>>>> it >>>>>>>>>>>>>>> in >>>>>>>>>>>>>>> Java layer. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> If the issue is really "junk" beyond the \0 nul-terminator, how >>>>>>>>>>>>>> did >>>>>>>>>>>>>> we >>>>>>>>>>>>>> get >>>>>>>>>>>>>> such values? Shouldn't the byte array already be terminated at >>>>>>>>>>>>>> the >>>>>>>>>>>>>> NUL? >>>>>>>>>>>>>> Or >>>>>>>>>>>>>> is this just a raw representation of an array in the VM using >>>>>>>>>>>>>> the >>>>>>>>>>>>>> exact >>>>>>>>>>>>>> length of the array regardless of content? >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I think JSnap shows the array regardless of content. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> I see CStringUtilities.getString() both stops when it encounters >>>>>>>>>>>>>> a >>>>>>>>>>>>>> 0 >>>>>>>>>>>>>> value, >>>>>>>>>>>>>> and uses the default file encoding (else ASCII). >>>>>>>>>>>>>> >>>>>>>>>>>>>> The fix seems perfectly reasonable, I'm just unclear where >>>>>>>>>>>>>> exactly >>>>>>>>>>>>>> this >>>>>>>>>>>>>> "bad" data should have been stopped. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks! >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> David >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>> >>> From thomas.stuefe at gmail.com Thu Jun 14 08:38:03 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 14 Jun 2018 10:38:03 +0200 Subject: RFR(s): 8204958: Minor cleanups for the diagnostic framework In-Reply-To: References: Message-ID: Hi all, (I moved the issue to target release 12 - if we get this through the review quick, fine, but no reason to add pressure) some more background: The reason for these cleanups is that I later plan to add a new sub command which allows us to run multiple commands in one safepoint (see [1] for more details and discussions). Since this will require tweaking the parse process a bit, I try to loosen up the coding before adding complexity again. Please find here the new webrev: Delta: http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.00-to-01/webrev/ Full: http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.01/webrev/ What changed: Aesthetics: - I reordered member init lists to member order, as Coleen did ask. I also reform them as lists, just to increase readability. - For a number of inline methods, I aligned their implementations so that they too would be better readable and it would be clearer which methods are const - Moved DCmdFactory::_DCmdFactoryList up to the other static members. Functional: - Added initializer list for CmdLine - Added missing initializations for DCmdArgIter members _key_addr, _key_len, _value_addr, _value_len, as Coleen suggested - constified DCmdInfo too - since this is an immutable value holder, all members can be const. - also in DCmdInfo, changed JavaPermission permission() to const JavaPermission& permission() to save a bit unnecessary copying. - Fixed initialization of DCmdArgumentInfo::_position as Serguei and Chris requested - Removed default implementation of DCmd::name and DCmd::description. There is no good reason to provide one - all child classes should (and do) provide an implementation. I rather have linkage errors if someone forgets to add one. - Removed default implementations for DCmdWithParser::name/description - same reasoning as above - Removed DCmdWithParser::permission(), it is already provided by the base class. - Removed DCmdWithParser::num_arguments() since childs of DCmdWithParser will always have arguments and hence should implement this function. - Made all functions in DCmdFactoryImpl non-virtual. There is no point in allowing to override them. -- Notes: I see the following commands do not implement "permission" and hence rely on the default impl of DCmd::permission, which returns a permission structure { NULL, NULL, NULL } - is that okay and intended? JMXStopRemoteDCmd JMXStartLocalDCmd JMXStartRemoteDCmd TouchedMethodsDCmd ClassStatsDCmd RunFinalizationDCmd SystemGCDCmd VMUptimeDCmd HelpDCmd I am not sure where and how these permissions are tested. Thinking through this further: Of the 47 child classes of DCmd, 37 seem to return "java.lang.management.ManagementPermission", name "monitor". So, we have this function 37 times: static const JavaPermission permission() { JavaPermission p = {"java.lang.management.ManagementPermission", "monitor", NULL}; return p; } Would it then not make sense to make this the default implementation for DCmd::permission() - instead of the NULL permission - and only require child DCmds to provide a permission if it differs from this default? We could then loose all 37 methods. --- Similar thoughts for "static DCmd impact()" which is not provided by: JMXStatusDCmd JMXStopRemoteDCmd JMXStartLocalDCmd which thus return the default impact - DCmd::impact() - which returns "Low". Here I think we should either remove the default implementation and force all commands to provide an implementation or stay with the default "Low" implementation and remove all 20+ functions in child classes which also return "Low". -- Finally, some thoughts for possible future cleanups: GenDCmdArgument internally uses C-heap to hold copies of strings (DCmdArgument and DCmdArgument). I wonder whether we could use resource area instead. Then we could get rid of all cleanup code (GenDCmdArgument::cleanup(), GenDCmdArgument::destroy_value()) etc. And if one agrees that parsing is only done once and the commands are not reused (which is the case currently) we also may get rid of DCmdArgument::reset() and friends. Coding would become a lot easier. -- IMHO the way object- und template based polymorphy is mixed is a bit of a mismatch here and it complicates things a bit. For instance, Dcmd has above mentioned static functions which return immutable attributes for the commands. It is "overwritten" template-style by redefining the same static functions in derived classes and called via DCmdFactoryImpl. Maybe it is just me, but I find this a bit roundabout for the problem it tries to solve. Also, it is really difficult to parse for modern IDEs - e.g. my CDT is unable to tell me who calls "DCmd::permission()" since the relation between DCmd::permission() and XXXCmd::permission() is just by name. Also, why pay for the text size of 47 just slightly different template definitions of DCmdFactory where one class would be enough? The mis-fit between templates and object polymorphism also shows up as weird artifacts. E.g. we have 27 variants of this function: int XXXXDCmd::num_arguments() { ResourceMark rm; XXXDCmd* dcmd = new SetVMFlagDCmd(NULL, false); if (dcmd != NULL) { DCmdMark mark(dcmd); return dcmd->_dcmdparser.num_arguments(); } else { return 0; } } - so, we want to implement "static XXXDCmd::num_arguments", which - being static - has to create an object of the same type to ask it its number of arguments. Maybe we could un-templify the command classes at some point to simplify all this? I think we need two things, a class to hold meta information about the command - name, description, argument types etc - which needs to be available at startup. And then, we need a second thing to hold runtime context: the concrete argument values the user gave us, the outputStream* pointer, the execute() function, maybe parser state... I think this may be expressed simpler and more efficiently with simple C++ objects and inheritance, without templates. We could also shrink the code quite a bit. And my CDT would find who calls "permissions()" :) Best Regards, Thomas ------- [1] http://mail.openjdk.java.net/pipermail/serviceability-dev/2018-May/023673.html On Wed, Jun 13, 2018 at 3:41 PM, Thomas St?fe wrote: > Hi all, > > while working on jcmd I saw some minor cleanup possibilities. > > Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.00/webrev/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8204958 > > Most of them are const correctness and using initializer lists instead > of java-style member initialization in these C++ classes. > > Other changes: > > - Added a ResourceMark into the DCmd::parse_and_execute() loop > - removed DCmdFactory::create_global_DCmd() (and following that, > DCmdFactory::create_Cheap_instance()) since I did not find anyone > using that function. > - Also in DCmdFactory, I removed a number of setters which were never > called and for attributes which should be immutable: hidden, flags, > enabled. > > This is really a rather light cleanup. It did not touch any larger issues. > > Thank you, > > Thomas From thomas.stuefe at gmail.com Thu Jun 14 10:39:23 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 14 Jun 2018 12:39:23 +0200 Subject: RFR(S): 8204654: [testbug] Fix pattern matching in jstat tests. In-Reply-To: <18662fc429004f34bae0343b03dcd28e@sap.com> References: <18662fc429004f34bae0343b03dcd28e@sap.com> Message-ID: Hi Goetz, test/jdk/sun/jvmstat/testlibrary/utils.sh consider better comment: -# The awk scripts parsing the output do not respect any locale dependent setting. +# The awk scripts parsing the jstat output expect it to be in en-us locale. -------------- http://cr.openjdk.java.net/~goetz/wr18/8204654-fixJstatTest/01/test/jdk/sun/tools/jstat/classloadOutput1.awk.udiff.html I assume ([0-9])|-+ should match a single whole number for column "FindClass", or "-"? do not think you need grouping here, and the '+' is misplaced. Better: [0-9]+|- --- similar here: + misplaced: - ([0-9]+\.[0-9])|-+ + ([0-9]+\.[0-9]+)|- Also, it would be helpful if you could print column name or at least number atop the matching sequences, in a comment, that makes the reading easer. But this is hard to read! Any chance of splitting this expression into sub expressions? Rest seems fine. ..Thomas On Mon, Jun 11, 2018 at 3:14 PM, Lindenmaier, Goetz wrote: > Hi, > > please review this test fix: > http://cr.openjdk.java.net/~goetz/wr18/8204654-fixJstatTest/01 > > gcCauseOutput1.awk: > The pattern scans 11 numbers, while the output contains 13. Also, more '-' are possible then checked. > > The other awk scripts need to check more patterns where '-' can appear. > > We have a machine with user.country=de where jstat prints ',' instead of '.' in numbers. Explicitly > start with user.country=en as already done for user.language=en. > > I also refactored the common flags to a variable in utils.sh ... so there won't be the need > to edit all these files once more :) > > Best regards, > Goetz > > (Sorry for the second mail, first missed the bug text ...) From thomas.stuefe at gmail.com Thu Jun 14 11:30:21 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 14 Jun 2018 13:30:21 +0200 Subject: RFR(s): 8203343: VM.{metaspace|classloaders|classhierarchy...} jcmd should show invocation targets for Generated{Method|Constructor}AccessorImpl classes In-Reply-To: References: Message-ID: Hi all, hopefully last changes, with feedback added from Coleen and David. Only changes in the provided regression test: I run it now with -Dsun.reflect.noInflation to make the test independent from the reflection inflation threshold. I also corrected the copyright dates. Delta: http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/webrev.01-to-02/webrev/ Full: http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/webrev.02/webrev/ Thank you, Thomas On Wed, Jun 6, 2018 at 6:05 PM, Thomas St?fe wrote: > Dear all, > > may I please have feedback and if possible reviews for this small addition: > > CR: https://bugs.openjdk.java.net/browse/JDK-8203343 > Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/webrev.00/webrev/ > > (Note: this patch goes atop of > https://bugs.openjdk.java.net/browse/JDK-8203682, which is currently > up for RFR). > > --- > > When analyzing situations involving a lot of reflection, one often > stares at walls of "GeneratedXXXAccessorXXX" classes. These names are > generated and not at all helpful in analyzing the problem (e.g. which > component in a server node does this much reflection and hence uses so > much metaspace). > > This patch adds the ability to print out invocation targets > additionally to class names if the class is a generated accessor - for > now this ability has been added to VM.metaspace, VM.classloaders and > VM.class_hierarchy. > > Example output from "VM.class_hierarchy": > > > |--jdk.internal.reflect.MagicAccessorImpl/null > > | |--jdk.internal.reflect.ConstructorAccessorImpl/null > | | |--jdk.internal.reflect.GeneratedConstructorAccessor18/0x00007f9ee8350c10 > (invokes: org/springframework/boot/context/properties/ConfigurationPropertiesBindingPostProcessorRegistrar:: > ()V) > | | |--jdk.internal.reflect.GeneratedConstructorAccessor17/0x00007f9ee8349c00 > (invokes: org/springframework/boot/context/properties/EnableConfigurationPropertiesImportSelector$ConfigurationPropertiesBeanRegistrar:: > ()V) > > | |--jdk.internal.reflect.MethodAccessorImpl/null > | | |--jdk.internal.reflect.GeneratedMethodAccessor23/0x00007f9ec8329b60 > (invokes: org/apache/tomcat/util/modeler/AttributeInfo::setIs (Z)V) > | | |--jdk.internal.reflect.GeneratedMethodAccessor22/0x00007f9ee831bc70 > (invokes: org/springframework/boot/autoconfigure/SpringBootApplication::exclude > ()[Ljava/lang/Class;) > > > See here more examples: > > "VM.class_hierarchy" > http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/example-VM.class_hierarchy.txt > > "VM.classloaders show-classes" > http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/example-VM.classloaders.txt > > > ---- > > Note: I am not sure if this is a fit for hotspot-runtime or > serviceability. Sorry for crossposting. > > Thank you, > > Thomas From goetz.lindenmaier at sap.com Thu Jun 14 12:48:55 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 14 Jun 2018 12:48:55 +0000 Subject: RFR(S): 8204654: [testbug] Fix pattern matching in jstat tests. In-Reply-To: References: <18662fc429004f34bae0343b03dcd28e@sap.com> Message-ID: <8f9fd18d1aa040a7a9f5c88508740cb3@sap.com> Hi Thomas, thanks for your review! New webrev: http://cr.openjdk.java.net/~goetz/wr18/8204654-fixJstatTest/02/ I'll run this through our testing again and push tomorrow if all is green. > consider better comment: > -# The awk scripts parsing the output do not respect any locale > dependent setting. > +# The awk scripts parsing the jstat output expect it to be in en-us locale. Fixed. > I assume > ([0-9])|-+ > should match a single whole number for column "FindClass", or "-"? Yes, fixed. > similar here: + misplaced: > - ([0-9]+\.[0-9])|-+ > + ([0-9]+\.[0-9]+)|- Also fixed. Great catch! > Also, it would be helpful if you could print column name or at least > number atop the matching sequences, in a comment, that makes the > reading easer. Yes, but there are a row of similar files I don't touch. Don't want to do this for all of them. > But this is hard to read! Any chance of splitting this expression into > sub expressions? Yes, this is ugly. But I think the whole tests should be rewritten to do the parsing in Java, as the jstatd tests do. I think this is out of scope here. Best regards, Goetz. > > > Rest seems fine. > > ..Thomas > > On Mon, Jun 11, 2018 at 3:14 PM, Lindenmaier, Goetz > wrote: > > Hi, > > > > please review this test fix: > > http://cr.openjdk.java.net/~goetz/wr18/8204654-fixJstatTest/01 > > > > gcCauseOutput1.awk: > > The pattern scans 11 numbers, while the output contains 13. Also, more '-' > are possible then checked. > > > > The other awk scripts need to check more patterns where '-' can appear. > > > > We have a machine with user.country=de where jstat prints ',' instead of '.' > in numbers. Explicitly > > start with user.country=en as already done for user.language=en. > > > > I also refactored the common flags to a variable in utils.sh ... so there won't > be the need > > to edit all these files once more :) > > > > Best regards, > > Goetz > > > > (Sorry for the second mail, first missed the bug text ...) From thomas.stuefe at gmail.com Thu Jun 14 12:54:30 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 14 Jun 2018 14:54:30 +0200 Subject: RFR(S): 8204654: [testbug] Fix pattern matching in jstat tests. In-Reply-To: <8f9fd18d1aa040a7a9f5c88508740cb3@sap.com> References: <18662fc429004f34bae0343b03dcd28e@sap.com> <8f9fd18d1aa040a7a9f5c88508740cb3@sap.com> Message-ID: On Thu, Jun 14, 2018 at 2:48 PM, Lindenmaier, Goetz wrote: > Hi Thomas, > > thanks for your review! > New webrev: > http://cr.openjdk.java.net/~goetz/wr18/8204654-fixJstatTest/02/ > I'll run this through our testing again and push tomorrow if all > is green. > >> consider better comment: >> -# The awk scripts parsing the output do not respect any locale >> dependent setting. >> +# The awk scripts parsing the jstat output expect it to be in en-us locale. > Fixed. > >> I assume >> ([0-9])|-+ >> should match a single whole number for column "FindClass", or "-"? > Yes, fixed. > >> similar here: + misplaced: >> - ([0-9]+\.[0-9])|-+ >> + ([0-9]+\.[0-9]+)|- > Also fixed. Great catch! > >> Also, it would be helpful if you could print column name or at least >> number atop the matching sequences, in a comment, that makes the >> reading easer. > Yes, but there are a row of similar files I don't touch. Don't want to do > this for all of them. > I totally understand :) >> But this is hard to read! Any chance of splitting this expression into >> sub expressions? > Yes, this is ugly. > But I think the whole tests should be rewritten to do the parsing in Java, > as the jstatd tests do. I think this is out of scope here. Sure. > > Best regards, > Goetz. I do not need to see a new webrev. ..Thomas > > > >> >> >> Rest seems fine. >> >> ..Thomas >> >> On Mon, Jun 11, 2018 at 3:14 PM, Lindenmaier, Goetz >> wrote: >> > Hi, >> > >> > please review this test fix: >> > http://cr.openjdk.java.net/~goetz/wr18/8204654-fixJstatTest/01 >> > >> > gcCauseOutput1.awk: >> > The pattern scans 11 numbers, while the output contains 13. Also, more '-' >> are possible then checked. >> > >> > The other awk scripts need to check more patterns where '-' can appear. >> > >> > We have a machine with user.country=de where jstat prints ',' instead of '.' >> in numbers. Explicitly >> > start with user.country=en as already done for user.language=en. >> > >> > I also refactored the common flags to a variable in utils.sh ... so there won't >> be the need >> > to edit all these files once more :) >> > >> > Best regards, >> > Goetz >> > >> > (Sorry for the second mail, first missed the bug text ...) From david.holmes at oracle.com Thu Jun 14 12:55:15 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 14 Jun 2018 22:55:15 +1000 Subject: RFR (XS) 8204961: JVMTI jtreg tests build warnings on 32-bit platforms In-Reply-To: References: Message-ID: <80d0c562-c684-111e-ba70-a1860247aa6a@oracle.com> Hi Boris, I added serviceability-dev as JVM TI and its tests are technically serviceability concerns. On 14/06/2018 10:39 PM, Boris Ulasevich wrote: > Hi all, > > Please review the following patch: > ? https://bugs.openjdk.java.net/browse/JDK-8204961 > ? http://cr.openjdk.java.net/~bulasevich/8204961/webrev.01 > > Recently opensourced JVMTI tests gives build warnings for ARM32 build. I'm guessing the compiler version must have changed since we last ran these tests on 32-bit ARM. :) > GCC complains about conversion between 4-byte pointer to 8-byte jlong > type which is Ok in this case. I propose to hide warning using > conversion to intptr_t. I was concerned about what the warnings might imply but now I see that a JVM TI "tag" is simply a jlong used to funnel real pointers around to use for the tagging. So on 32-bit the upper 32-bits of the tag will always be zero and there is no data loss in any of the conversions. So assuming none of the other compilers complain about this, this seems fine to me. Thanks, David > thanks, > Boris From coleen.phillimore at oracle.com Thu Jun 14 16:04:57 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 14 Jun 2018 12:04:57 -0400 Subject: RFR(s): 8203343: VM.{metaspace|classloaders|classhierarchy...} jcmd should show invocation targets for Generated{Method|Constructor}AccessorImpl classes In-Reply-To: References: Message-ID: <17c0776f-2287-caa4-baa7-44122a03e099@oracle.com> This was a good find of David's.? Thank you for fixing the test. Coleen On 6/14/18 7:30 AM, Thomas St?fe wrote: > Hi all, > > hopefully last changes, with feedback added from Coleen and David. > > Only changes in the provided regression test: I run it now with > -Dsun.reflect.noInflation to make the test independent from the > reflection inflation threshold. I also corrected the copyright dates. > > Delta: http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/webrev.01-to-02/webrev/ > Full: http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/webrev.02/webrev/ > > Thank you, > > Thomas > > > On Wed, Jun 6, 2018 at 6:05 PM, Thomas St?fe wrote: >> Dear all, >> >> may I please have feedback and if possible reviews for this small addition: >> >> CR: https://bugs.openjdk.java.net/browse/JDK-8203343 >> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/webrev.00/webrev/ >> >> (Note: this patch goes atop of >> https://bugs.openjdk.java.net/browse/JDK-8203682, which is currently >> up for RFR). >> >> --- >> >> When analyzing situations involving a lot of reflection, one often >> stares at walls of "GeneratedXXXAccessorXXX" classes. These names are >> generated and not at all helpful in analyzing the problem (e.g. which >> component in a server node does this much reflection and hence uses so >> much metaspace). >> >> This patch adds the ability to print out invocation targets >> additionally to class names if the class is a generated accessor - for >> now this ability has been added to VM.metaspace, VM.classloaders and >> VM.class_hierarchy. >> >> Example output from "VM.class_hierarchy": >> >> >> |--jdk.internal.reflect.MagicAccessorImpl/null >> >> | |--jdk.internal.reflect.ConstructorAccessorImpl/null >> | | |--jdk.internal.reflect.GeneratedConstructorAccessor18/0x00007f9ee8350c10 >> (invokes: org/springframework/boot/context/properties/ConfigurationPropertiesBindingPostProcessorRegistrar:: >> ()V) >> | | |--jdk.internal.reflect.GeneratedConstructorAccessor17/0x00007f9ee8349c00 >> (invokes: org/springframework/boot/context/properties/EnableConfigurationPropertiesImportSelector$ConfigurationPropertiesBeanRegistrar:: >> ()V) >> >> | |--jdk.internal.reflect.MethodAccessorImpl/null >> | | |--jdk.internal.reflect.GeneratedMethodAccessor23/0x00007f9ec8329b60 >> (invokes: org/apache/tomcat/util/modeler/AttributeInfo::setIs (Z)V) >> | | |--jdk.internal.reflect.GeneratedMethodAccessor22/0x00007f9ee831bc70 >> (invokes: org/springframework/boot/autoconfigure/SpringBootApplication::exclude >> ()[Ljava/lang/Class;) >> >> >> See here more examples: >> >> "VM.class_hierarchy" >> http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/example-VM.class_hierarchy.txt >> >> "VM.classloaders show-classes" >> http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/example-VM.classloaders.txt >> >> >> ---- >> >> Note: I am not sure if this is a fit for hotspot-runtime or >> serviceability. Sorry for crossposting. >> >> Thank you, >> >> Thomas From thomas.stuefe at gmail.com Thu Jun 14 16:12:58 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 14 Jun 2018 18:12:58 +0200 Subject: RFR(s): 8203343: VM.{metaspace|classloaders|classhierarchy...} jcmd should show invocation targets for Generated{Method|Constructor}AccessorImpl classes In-Reply-To: <17c0776f-2287-caa4-baa7-44122a03e099@oracle.com> References: <17c0776f-2287-caa4-baa7-44122a03e099@oracle.com> Message-ID: Thanks Coleen! On Thu, Jun 14, 2018, 18:04 wrote: > > This was a good find of David's. Thank you for fixing the test. > Coleen > > On 6/14/18 7:30 AM, Thomas St?fe wrote: > > Hi all, > > > > hopefully last changes, with feedback added from Coleen and David. > > > > Only changes in the provided regression test: I run it now with > > -Dsun.reflect.noInflation to make the test independent from the > > reflection inflation threshold. I also corrected the copyright dates. > > > > Delta: > http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/webrev.01-to-02/webrev/ > > Full: > http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/webrev.02/webrev/ > > > > Thank you, > > > > Thomas > > > > > > On Wed, Jun 6, 2018 at 6:05 PM, Thomas St?fe > wrote: > >> Dear all, > >> > >> may I please have feedback and if possible reviews for this small > addition: > >> > >> CR: https://bugs.openjdk.java.net/browse/JDK-8203343 > >> Webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/webrev.00/webrev/ > >> > >> (Note: this patch goes atop of > >> https://bugs.openjdk.java.net/browse/JDK-8203682, which is currently > >> up for RFR). > >> > >> --- > >> > >> When analyzing situations involving a lot of reflection, one often > >> stares at walls of "GeneratedXXXAccessorXXX" classes. These names are > >> generated and not at all helpful in analyzing the problem (e.g. which > >> component in a server node does this much reflection and hence uses so > >> much metaspace). > >> > >> This patch adds the ability to print out invocation targets > >> additionally to class names if the class is a generated accessor - for > >> now this ability has been added to VM.metaspace, VM.classloaders and > >> VM.class_hierarchy. > >> > >> Example output from "VM.class_hierarchy": > >> > >> > >> |--jdk.internal.reflect.MagicAccessorImpl/null > >> > >> | |--jdk.internal.reflect.ConstructorAccessorImpl/null > >> | | > |--jdk.internal.reflect.GeneratedConstructorAccessor18/0x00007f9ee8350c10 > >> (invokes: > org/springframework/boot/context/properties/ConfigurationPropertiesBindingPostProcessorRegistrar:: > >> ()V) > >> | | > |--jdk.internal.reflect.GeneratedConstructorAccessor17/0x00007f9ee8349c00 > >> (invokes: > org/springframework/boot/context/properties/EnableConfigurationPropertiesImportSelector$ConfigurationPropertiesBeanRegistrar:: > >> ()V) > >> > >> | |--jdk.internal.reflect.MethodAccessorImpl/null > >> | | > |--jdk.internal.reflect.GeneratedMethodAccessor23/0x00007f9ec8329b60 > >> (invokes: org/apache/tomcat/util/modeler/AttributeInfo::setIs (Z)V) > >> | | > |--jdk.internal.reflect.GeneratedMethodAccessor22/0x00007f9ee831bc70 > >> (invokes: > org/springframework/boot/autoconfigure/SpringBootApplication::exclude > >> ()[Ljava/lang/Class;) > >> > >> > >> See here more examples: > >> > >> "VM.class_hierarchy" > >> > http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/example-VM.class_hierarchy.txt > >> > >> "VM.classloaders show-classes" > >> > http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/example-VM.classloaders.txt > >> > >> > >> ---- > >> > >> Note: I am not sure if this is a fit for hotspot-runtime or > >> serviceability. Sorry for crossposting. > >> > >> Thank you, > >> > >> Thomas > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Thu Jun 14 19:09:00 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 14 Jun 2018 12:09:00 -0700 Subject: RFR (XS) 8204961: JVMTI jtreg tests build warnings on 32-bit platforms In-Reply-To: <80d0c562-c684-111e-ba70-a1860247aa6a@oracle.com> References: <80d0c562-c684-111e-ba70-a1860247aa6a@oracle.com> Message-ID: Hi Boris, It looks good to me. Thank you for taking care about these warnings! Thanks, Serguei On 6/14/18 05:55, David Holmes wrote: > Hi Boris, > > I added serviceability-dev as JVM TI and its tests are technically > serviceability concerns. > > On 14/06/2018 10:39 PM, Boris Ulasevich wrote: >> Hi all, >> >> Please review the following patch: >> ?? https://bugs.openjdk.java.net/browse/JDK-8204961 >> ?? http://cr.openjdk.java.net/~bulasevich/8204961/webrev.01 >> >> Recently opensourced JVMTI tests gives build warnings for ARM32 build. > > I'm guessing the compiler version must have changed since we last ran > these tests on 32-bit ARM. :) > >> GCC complains about conversion between 4-byte pointer to 8-byte jlong >> type which is Ok in this case. I propose to hide warning using >> conversion to intptr_t. > > I was concerned about what the warnings might imply but now I see that > a JVM TI "tag" is simply a jlong used to funnel real pointers around > to use for the tagging. So on 32-bit the upper 32-bits of the tag will > always be zero and there is no data loss in any of the conversions. > > So assuming none of the other compilers complain about this, this > seems fine to me. > > Thanks, > David > >> thanks, >> Boris From serguei.spitsyn at oracle.com Thu Jun 14 19:14:49 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 14 Jun 2018 12:14:49 -0700 Subject: RFR(s): 8203343: VM.{metaspace|classloaders|classhierarchy...} jcmd should show invocation targets for Generated{Method|Constructor}AccessorImpl classes In-Reply-To: References: Message-ID: <4acbd94c-cb2f-da59-4541-1985d6afa8bf@oracle.com> Hi Thomas, The update looks good to me. Thanks, Serguei On 6/14/18 04:30, Thomas St?fe wrote: > Hi all, > > hopefully last changes, with feedback added from Coleen and David. > > Only changes in the provided regression test: I run it now with > -Dsun.reflect.noInflation to make the test independent from the > reflection inflation threshold. I also corrected the copyright dates. > > Delta: http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/webrev.01-to-02/webrev/ > Full: http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/webrev.02/webrev/ > > Thank you, > > Thomas > > > On Wed, Jun 6, 2018 at 6:05 PM, Thomas St?fe wrote: >> Dear all, >> >> may I please have feedback and if possible reviews for this small addition: >> >> CR: https://bugs.openjdk.java.net/browse/JDK-8203343 >> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/webrev.00/webrev/ >> >> (Note: this patch goes atop of >> https://bugs.openjdk.java.net/browse/JDK-8203682, which is currently >> up for RFR). >> >> --- >> >> When analyzing situations involving a lot of reflection, one often >> stares at walls of "GeneratedXXXAccessorXXX" classes. These names are >> generated and not at all helpful in analyzing the problem (e.g. which >> component in a server node does this much reflection and hence uses so >> much metaspace). >> >> This patch adds the ability to print out invocation targets >> additionally to class names if the class is a generated accessor - for >> now this ability has been added to VM.metaspace, VM.classloaders and >> VM.class_hierarchy. >> >> Example output from "VM.class_hierarchy": >> >> >> |--jdk.internal.reflect.MagicAccessorImpl/null >> >> | |--jdk.internal.reflect.ConstructorAccessorImpl/null >> | | |--jdk.internal.reflect.GeneratedConstructorAccessor18/0x00007f9ee8350c10 >> (invokes: org/springframework/boot/context/properties/ConfigurationPropertiesBindingPostProcessorRegistrar:: >> ()V) >> | | |--jdk.internal.reflect.GeneratedConstructorAccessor17/0x00007f9ee8349c00 >> (invokes: org/springframework/boot/context/properties/EnableConfigurationPropertiesImportSelector$ConfigurationPropertiesBeanRegistrar:: >> ()V) >> >> | |--jdk.internal.reflect.MethodAccessorImpl/null >> | | |--jdk.internal.reflect.GeneratedMethodAccessor23/0x00007f9ec8329b60 >> (invokes: org/apache/tomcat/util/modeler/AttributeInfo::setIs (Z)V) >> | | |--jdk.internal.reflect.GeneratedMethodAccessor22/0x00007f9ee831bc70 >> (invokes: org/springframework/boot/autoconfigure/SpringBootApplication::exclude >> ()[Ljava/lang/Class;) >> >> >> See here more examples: >> >> "VM.class_hierarchy" >> http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/example-VM.class_hierarchy.txt >> >> "VM.classloaders show-classes" >> http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/example-VM.classloaders.txt >> >> >> ---- >> >> Note: I am not sure if this is a fit for hotspot-runtime or >> serviceability. Sorry for crossposting. >> >> Thank you, >> >> Thomas From thomas.stuefe at gmail.com Thu Jun 14 19:51:01 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 14 Jun 2018 21:51:01 +0200 Subject: RFR(s): 8203343: VM.{metaspace|classloaders|classhierarchy...} jcmd should show invocation targets for Generated{Method|Constructor}AccessorImpl classes In-Reply-To: <4acbd94c-cb2f-da59-4541-1985d6afa8bf@oracle.com> References: <4acbd94c-cb2f-da59-4541-1985d6afa8bf@oracle.com> Message-ID: Thank you Serguei! On Thu, Jun 14, 2018 at 9:14 PM, serguei.spitsyn at oracle.com wrote: > Hi Thomas, > > The update looks good to me. > > Thanks, > Serguei > > > > On 6/14/18 04:30, Thomas St?fe wrote: >> >> Hi all, >> >> hopefully last changes, with feedback added from Coleen and David. >> >> Only changes in the provided regression test: I run it now with >> -Dsun.reflect.noInflation to make the test independent from the >> reflection inflation threshold. I also corrected the copyright dates. >> >> Delta: >> http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/webrev.01-to-02/webrev/ >> Full: >> http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/webrev.02/webrev/ >> >> Thank you, >> >> Thomas >> >> >> On Wed, Jun 6, 2018 at 6:05 PM, Thomas St?fe >> wrote: >>> >>> Dear all, >>> >>> may I please have feedback and if possible reviews for this small >>> addition: >>> >>> CR: https://bugs.openjdk.java.net/browse/JDK-8203343 >>> Webrev: >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/webrev.00/webrev/ >>> >>> (Note: this patch goes atop of >>> https://bugs.openjdk.java.net/browse/JDK-8203682, which is currently >>> up for RFR). >>> >>> --- >>> >>> When analyzing situations involving a lot of reflection, one often >>> stares at walls of "GeneratedXXXAccessorXXX" classes. These names are >>> generated and not at all helpful in analyzing the problem (e.g. which >>> component in a server node does this much reflection and hence uses so >>> much metaspace). >>> >>> This patch adds the ability to print out invocation targets >>> additionally to class names if the class is a generated accessor - for >>> now this ability has been added to VM.metaspace, VM.classloaders and >>> VM.class_hierarchy. >>> >>> Example output from "VM.class_hierarchy": >>> >>> >>> |--jdk.internal.reflect.MagicAccessorImpl/null >>> >>> | |--jdk.internal.reflect.ConstructorAccessorImpl/null >>> | | >>> |--jdk.internal.reflect.GeneratedConstructorAccessor18/0x00007f9ee8350c10 >>> (invokes: >>> org/springframework/boot/context/properties/ConfigurationPropertiesBindingPostProcessorRegistrar:: >>> ()V) >>> | | >>> |--jdk.internal.reflect.GeneratedConstructorAccessor17/0x00007f9ee8349c00 >>> (invokes: >>> org/springframework/boot/context/properties/EnableConfigurationPropertiesImportSelector$ConfigurationPropertiesBeanRegistrar:: >>> ()V) >>> >>> | |--jdk.internal.reflect.MethodAccessorImpl/null >>> | | >>> |--jdk.internal.reflect.GeneratedMethodAccessor23/0x00007f9ec8329b60 >>> (invokes: org/apache/tomcat/util/modeler/AttributeInfo::setIs (Z)V) >>> | | >>> |--jdk.internal.reflect.GeneratedMethodAccessor22/0x00007f9ee831bc70 >>> (invokes: >>> org/springframework/boot/autoconfigure/SpringBootApplication::exclude >>> ()[Ljava/lang/Class;) >>> >>> >>> See here more examples: >>> >>> "VM.class_hierarchy" >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/example-VM.class_hierarchy.txt >>> >>> "VM.classloaders show-classes" >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/example-VM.classloaders.txt >>> >>> >>> ---- >>> >>> Note: I am not sure if this is a fit for hotspot-runtime or >>> serviceability. Sorry for crossposting. >>> >>> Thank you, >>> >>> Thomas > > From chris.plummer at oracle.com Fri Jun 15 05:04:13 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 14 Jun 2018 22:04:13 -0700 Subject: RFR(s): 8204958: Minor cleanups for the diagnostic framework In-Reply-To: References: Message-ID: Hi Thomas, I'm not certain why you added default field initializers for CmdLine::CmdLine when each field is explicitly initialized within the method body. A number of copyrights need updating. The rest of the changes look good. thanks, Chris On 6/14/18 1:38 AM, Thomas St?fe wrote: > Hi all, > > (I moved the issue to target release 12 - if we get this through the > review quick, fine, but no reason to add pressure) > > some more background: The reason for these cleanups is that I later > plan to add a new sub command which allows us to run multiple commands > in one safepoint (see [1] for more details and discussions). Since > this will require tweaking the parse process a bit, I try to loosen up > the coding before adding complexity again. > > Please find here the new webrev: > > Delta: http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.00-to-01/webrev/ > Full: http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.01/webrev/ > > What changed: > > Aesthetics: > - I reordered member init lists to member order, as Coleen did ask. I > also reform them as lists, just to increase readability. > - For a number of inline methods, I aligned their implementations so > that they too would be better readable and it would be clearer which > methods are const > - Moved DCmdFactory::_DCmdFactoryList up to the other static members. > > Functional: > - Added initializer list for CmdLine > - Added missing initializations for DCmdArgIter members _key_addr, > _key_len, _value_addr, _value_len, as Coleen suggested > - constified DCmdInfo too - since this is an immutable value holder, > all members can be const. > - also in DCmdInfo, changed JavaPermission permission() to const > JavaPermission& permission() to save a bit unnecessary copying. > - Fixed initialization of DCmdArgumentInfo::_position as Serguei and > Chris requested > - Removed default implementation of DCmd::name and DCmd::description. > There is no good reason to provide one - all child classes should (and > do) provide an implementation. I rather have linkage errors if someone > forgets to add one. > - Removed default implementations for DCmdWithParser::name/description > - same reasoning as above > - Removed DCmdWithParser::permission(), it is already provided by the > base class. > - Removed DCmdWithParser::num_arguments() since childs of > DCmdWithParser will always have arguments and hence should implement > this function. > - Made all functions in DCmdFactoryImpl non-virtual. There is no point > in allowing to override them. > > -- > > Notes: > > I see the following commands do not implement "permission" and hence > rely on the default impl of DCmd::permission, which returns a > permission structure { NULL, NULL, NULL } - is that okay and intended? > > JMXStopRemoteDCmd > JMXStartLocalDCmd > JMXStartRemoteDCmd > TouchedMethodsDCmd > ClassStatsDCmd > RunFinalizationDCmd > SystemGCDCmd > VMUptimeDCmd > HelpDCmd > > I am not sure where and how these permissions are tested. > > Thinking through this further: > > Of the 47 child classes of DCmd, 37 seem to return > "java.lang.management.ManagementPermission", name "monitor". So, we > have this function 37 times: > > static const JavaPermission permission() { > JavaPermission p = {"java.lang.management.ManagementPermission", > "monitor", NULL}; > return p; > } > > Would it then not make sense to make this the default implementation > for DCmd::permission() - instead of the NULL permission - and only > require child DCmds to provide a permission if it differs from this > default? We could then loose all 37 methods. > > --- > > Similar thoughts for "static DCmd impact()" > > which is not provided by: > > JMXStatusDCmd > JMXStopRemoteDCmd > JMXStartLocalDCmd > > which thus return the default impact - DCmd::impact() - which returns "Low". > > Here I think we should either remove the default implementation and > force all commands to provide an implementation or stay with the > default "Low" implementation and remove all 20+ functions in child > classes which also return "Low". > > -- > > Finally, some thoughts for possible future cleanups: > > GenDCmdArgument internally uses C-heap to hold copies of strings > (DCmdArgument and DCmdArgument). I wonder > whether we could use resource area instead. Then we could get rid of > all cleanup code (GenDCmdArgument::cleanup(), > GenDCmdArgument::destroy_value()) etc. And if one agrees that parsing > is only done once and the commands are not reused (which is the case > currently) we also may get rid of DCmdArgument::reset() and friends. > Coding would become a lot easier. > > -- > > IMHO the way object- und template based polymorphy is mixed is a bit > of a mismatch here and it complicates things a bit. > > For instance, Dcmd has above mentioned static functions which return > immutable attributes for the commands. It is "overwritten" > template-style by redefining the same static functions in derived > classes and called via DCmdFactoryImpl. > > Maybe it is just me, but I find this a bit roundabout for the problem > it tries to solve. Also, it is really difficult to parse for modern > IDEs - e.g. my CDT is unable to tell me who calls "DCmd::permission()" > since the relation between DCmd::permission() and XXXCmd::permission() > is just by name. Also, why pay for the text size of 47 just slightly > different template definitions of DCmdFactory where one class > would be enough? > > The mis-fit between templates and object polymorphism also shows up as > weird artifacts. E.g. we have 27 variants of this function: > > int XXXXDCmd::num_arguments() { > ResourceMark rm; > XXXDCmd* dcmd = new SetVMFlagDCmd(NULL, false); > if (dcmd != NULL) { > DCmdMark mark(dcmd); > return dcmd->_dcmdparser.num_arguments(); > } else { > return 0; > } > } > > - so, we want to implement "static XXXDCmd::num_arguments", which - > being static - has to create an object of the same type to ask it its > number of arguments. > > Maybe we could un-templify the command classes at some point to > simplify all this? > > I think we need two things, a class to hold meta information about the > command - name, description, argument types etc - which needs to be > available at startup. And then, we need a second thing to hold runtime > context: the concrete argument values the user gave us, the > outputStream* pointer, the execute() function, maybe parser state... > > I think this may be expressed simpler and more efficiently with simple > C++ objects and inheritance, without templates. We could also shrink > the code quite a bit. And my CDT would find who calls "permissions()" > :) > > > Best Regards, Thomas > > ------- > > [1] http://mail.openjdk.java.net/pipermail/serviceability-dev/2018-May/023673.html > > > On Wed, Jun 13, 2018 at 3:41 PM, Thomas St?fe wrote: >> Hi all, >> >> while working on jcmd I saw some minor cleanup possibilities. >> >> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.00/webrev/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8204958 >> >> Most of them are const correctness and using initializer lists instead >> of java-style member initialization in these C++ classes. >> >> Other changes: >> >> - Added a ResourceMark into the DCmd::parse_and_execute() loop >> - removed DCmdFactory::create_global_DCmd() (and following that, >> DCmdFactory::create_Cheap_instance()) since I did not find anyone >> using that function. >> - Also in DCmdFactory, I removed a number of setters which were never >> called and for attributes which should be immutable: hidden, flags, >> enabled. >> >> This is really a rather light cleanup. It did not touch any larger issues. >> >> Thank you, >> >> Thomas From serguei.spitsyn at oracle.com Fri Jun 15 09:16:27 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 15 Jun 2018 02:16:27 -0700 Subject: [URGENT] RFR: 8205096: Add missing files for 8203394 Message-ID: An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Fri Jun 15 09:23:02 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 15 Jun 2018 19:23:02 +1000 Subject: [URGENT] RFR: 8205096: Add missing files for 8203394 In-Reply-To: References: Message-ID: <79e8f0e2-4bb6-7792-9c9d-93c985bbca23@oracle.com> Ship it! Thanks, David On 15/06/2018 7:16 PM, serguei.spitsyn at oracle.com wrote: > Please, review a fix for: > https://bugs.openjdk.java.net/browse/JDK-8205096 > > Webrev: > http://cr.openjdk.java.net/~sspitsyn/webrevs/2018/8205096-missed-files-for-8203394/ > > > Summary: > ? I forgot to "hg add" all new files when committed fixes for the > JDK-8203394 > > Thanks a lot! > Serguei From serguei.spitsyn at oracle.com Fri Jun 15 09:24:00 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 15 Jun 2018 02:24:00 -0700 Subject: [URGENT] RFR: 8205096: Add missing files for 8203394 In-Reply-To: <79e8f0e2-4bb6-7792-9c9d-93c985bbca23@oracle.com> References: <79e8f0e2-4bb6-7792-9c9d-93c985bbca23@oracle.com> Message-ID: <75baca12-0573-d9a7-56e6-3378a522c359@oracle.com> Thanks, David! Serguei On 6/15/18 02:23, David Holmes wrote: > Ship it! > > Thanks, > David > > On 15/06/2018 7:16 PM, serguei.spitsyn at oracle.com wrote: >> Please, review a fix for: >> https://bugs.openjdk.java.net/browse/JDK-8205096 >> >> Webrev: >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2018/8205096-missed-files-for-8203394/ >> >> >> >> Summary: >> ?? I forgot to "hg add" all new files when committed fixes for the >> JDK-8203394 >> >> Thanks a lot! >> Serguei From thomas.schatzl at oracle.com Fri Jun 15 09:26:47 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 15 Jun 2018 11:26:47 +0200 Subject: [URGENT] RFR: 8205096: Add missing files for 8203394 In-Reply-To: <79e8f0e2-4bb6-7792-9c9d-93c985bbca23@oracle.com> References: <79e8f0e2-4bb6-7792-9c9d-93c985bbca23@oracle.com> Message-ID: <9bcd116bbb10db217e5d28085ec5ff7b31c3bac2.camel@oracle.com> Hi, On Fri, 2018-06-15 at 19:23 +1000, David Holmes wrote: > Ship it! +1 Thomas From serguei.spitsyn at oracle.com Fri Jun 15 09:27:56 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 15 Jun 2018 02:27:56 -0700 Subject: [URGENT] RFR: 8205096: Add missing files for 8203394 In-Reply-To: <9bcd116bbb10db217e5d28085ec5ff7b31c3bac2.camel@oracle.com> References: <79e8f0e2-4bb6-7792-9c9d-93c985bbca23@oracle.com> <9bcd116bbb10db217e5d28085ec5ff7b31c3bac2.camel@oracle.com> Message-ID: <62ed54e3-69a5-6f0b-f288-a8e4b2c7285e@oracle.com> Thanks a lot, Thomas! Serguei On 6/15/18 02:26, Thomas Schatzl wrote: > Hi, > > On Fri, 2018-06-15 at 19:23 +1000, David Holmes wrote: >> Ship it! > +1 > > Thomas > From robbin.ehn at oracle.com Fri Jun 15 09:27:50 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Fri, 15 Jun 2018 11:27:50 +0200 Subject: [URGENT] RFR: 8205096: Add missing files for 8203394 In-Reply-To: <79e8f0e2-4bb6-7792-9c9d-93c985bbca23@oracle.com> References: <79e8f0e2-4bb6-7792-9c9d-93c985bbca23@oracle.com> Message-ID: On 06/15/2018 11:23 AM, David Holmes wrote: > Ship it! +1 /Robbin > > Thanks, > David > > On 15/06/2018 7:16 PM, serguei.spitsyn at oracle.com wrote: >> Please, review a fix for: >> https://bugs.openjdk.java.net/browse/JDK-8205096 >> >> Webrev: >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2018/8205096-missed-files-for-8203394/ >> >> >> >> Summary: >> ?? I forgot to "hg add" all new files when committed fixes for the JDK-8203394 >> >> Thanks a lot! >> Serguei From serguei.spitsyn at oracle.com Fri Jun 15 09:28:41 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 15 Jun 2018 02:28:41 -0700 Subject: [URGENT] RFR: 8205096: Add missing files for 8203394 In-Reply-To: References: <79e8f0e2-4bb6-7792-9c9d-93c985bbca23@oracle.com> Message-ID: <2535b2aa-21c2-697f-a9a5-7914f43d7c68@oracle.com> Thank you, Robbin! Serguei On 6/15/18 02:27, Robbin Ehn wrote: > On 06/15/2018 11:23 AM, David Holmes wrote: >> Ship it! > > +1 > > /Robbin > >> >> Thanks, >> David >> >> On 15/06/2018 7:16 PM, serguei.spitsyn at oracle.com wrote: >>> Please, review a fix for: >>> https://bugs.openjdk.java.net/browse/JDK-8205096 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2018/8205096-missed-files-for-8203394/ >>> >>> >>> >>> Summary: >>> ?? I forgot to "hg add" all new files when committed fixes for the >>> JDK-8203394 >>> >>> Thanks a lot! >>> Serguei From volker.simonis at gmail.com Fri Jun 15 09:28:42 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 15 Jun 2018 11:28:42 +0200 Subject: [URGENT] RFR: 8205096: Add missing files for 8203394 In-Reply-To: References: Message-ID: Can't comment on the content, but at least it fixes the build so thumbs up from me! Regards, Volker On Fri, Jun 15, 2018 at 11:16 AM, serguei.spitsyn at oracle.com wrote: > Please, review a fix for: > https://bugs.openjdk.java.net/browse/JDK-8205096 > > Webrev: > > http://cr.openjdk.java.net/~sspitsyn/webrevs/2018/8205096-missed-files-for-8203394/ > > > Summary: > I forgot to "hg add" all new files when committed fixes for the > JDK-8203394 > > Thanks a lot! > Serguei From erik.helin at oracle.com Fri Jun 15 09:30:22 2018 From: erik.helin at oracle.com (Erik Helin) Date: Fri, 15 Jun 2018 11:30:22 +0200 Subject: [URGENT] RFR: 8205096: Add missing files for 8203394 In-Reply-To: References: Message-ID: <96266260-40c8-9777-a9d7-33c3a4c35b35@oracle.com> On 06/15/2018 11:16 AM, serguei.spitsyn at oracle.com wrote: > Please, review a fix for: > https://bugs.openjdk.java.net/browse/JDK-8205096 > > Webrev: > http://cr.openjdk.java.net/~sspitsyn/webrevs/2018/8205096-missed-files-for-8203394/ Looks good, Reviewed. I can confirm that jdk/jdk builds with patch applied, so please push this as soon as possible :) Thanks, Erik > Summary: > ? I forgot to "hg add" all new files when committed fixes for the > JDK-8203394 > > Thanks a lot! > Serguei From serguei.spitsyn at oracle.com Fri Jun 15 09:34:56 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 15 Jun 2018 02:34:56 -0700 Subject: [URGENT] RFR: 8205096: Add missing files for 8203394 In-Reply-To: <96266260-40c8-9777-a9d7-33c3a4c35b35@oracle.com> References: <96266260-40c8-9777-a9d7-33c3a4c35b35@oracle.com> Message-ID: Thank you, Erik H., Erik D. and Volker for the review! I've pushed the patch. Thanks, Serguei On 6/15/18 02:30, Erik Helin wrote: > On 06/15/2018 11:16 AM, serguei.spitsyn at oracle.com wrote: >> Please, review a fix for: >> https://bugs.openjdk.java.net/browse/JDK-8205096 >> >> Webrev: >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2018/8205096-missed-files-for-8203394/ >> > > Looks good, Reviewed. I can confirm that jdk/jdk builds with patch > applied, so please push this as soon as possible :) > > Thanks, > Erik > >> Summary: >> ?? I forgot to "hg add" all new files when committed fixes for the >> JDK-8203394 >> >> Thanks a lot! >> Serguei From coleen.phillimore at oracle.com Fri Jun 15 11:53:15 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 15 Jun 2018 07:53:15 -0400 Subject: RFR(s): 8204958: Minor cleanups for the diagnostic framework In-Reply-To: References: Message-ID: On 6/15/18 1:04 AM, Chris Plummer wrote: > Hi Thomas, > > I'm not certain why you added default field initializers for > CmdLine::CmdLine when each field is explicitly initialized within the > method body. I think the C++ optimizer would elide any unnecessary initializations and it seems safe in case new fields are added, although 38 : _cmd(NULL) could be initialized as _cmd(line). Can I bikeshed and say I don't like the style where the , are at the beginning of the lines??? We don't use this style in other places, that I've noticed, and I really don't like reading the punctuation at the beginning of the line. ? If I could find our open source wiki for the coding style among the broken openjdk wiki pages, I would see if the punctuation rule is there.? One initializer per line is ok. Thanks, Coleen > > A number of copyrights need updating. > > The rest of the changes look good. > > thanks, > > Chris > > On 6/14/18 1:38 AM, Thomas St?fe wrote: >> Hi all, >> >> (I moved the issue to target release 12 - if we get this through the >> review quick, fine, but no reason to add pressure) >> >> some more background: The reason for these cleanups is that I later >> plan to add a new sub command which allows us to run multiple commands >> in one safepoint (see [1] for more details and discussions). Since >> this will require tweaking the parse process a bit, I try to loosen up >> the coding before adding complexity again. >> >> Please find here the new webrev: >> >> Delta: >> http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.00-to-01/webrev/ >> Full: >> http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.01/webrev/ >> >> What changed: >> >> Aesthetics: >> - I reordered member init lists to member order, as Coleen did ask. I >> also reform them as lists, just to increase readability. >> - For a number of inline methods, I aligned their implementations so >> that they too would be better readable and it would be clearer which >> methods are const >> - Moved DCmdFactory::_DCmdFactoryList up to the other static members. >> >> Functional: >> - Added initializer list for CmdLine >> - Added missing initializations for DCmdArgIter members _key_addr, >> _key_len, _value_addr, _value_len, as Coleen suggested >> - constified DCmdInfo too - since this is an immutable value holder, >> all members can be const. >> - also in DCmdInfo, changed JavaPermission permission() to const >> JavaPermission& permission() to save a bit unnecessary copying. >> - Fixed initialization of DCmdArgumentInfo::_position as Serguei and >> Chris requested >> - Removed default implementation of DCmd::name and DCmd::description. >> There is no good reason to provide one - all child classes should (and >> do) provide an implementation. I rather have linkage errors if someone >> forgets to add one. >> - Removed default implementations for DCmdWithParser::name/description >> - same reasoning as above >> - Removed DCmdWithParser::permission(), it is already provided by the >> base class. >> - Removed DCmdWithParser::num_arguments() since childs of >> DCmdWithParser will always have arguments and hence should implement >> this function. >> - Made all functions in DCmdFactoryImpl non-virtual. There is no point >> in allowing to override them. >> >> -- >> >> Notes: >> >> I see the following commands do not implement "permission" and hence >> rely on the default impl of DCmd::permission, which returns a >> permission structure { NULL, NULL, NULL } - is that okay and intended? >> >> JMXStopRemoteDCmd >> JMXStartLocalDCmd >> JMXStartRemoteDCmd >> TouchedMethodsDCmd >> ClassStatsDCmd >> RunFinalizationDCmd >> SystemGCDCmd >> VMUptimeDCmd >> HelpDCmd >> >> I am not sure where and how these permissions are tested. >> >> Thinking through this further: >> >> Of the 47 child classes of DCmd, 37 seem to return >> "java.lang.management.ManagementPermission", name "monitor". So, we >> have this function 37 times: >> >> ?? static const JavaPermission permission() { >> ???? JavaPermission p = {"java.lang.management.ManagementPermission", >> ???????????????????????? "monitor", NULL}; >> ???? return p; >> ?? } >> >> Would it then not make sense to make this the default implementation >> for DCmd::permission() - instead of the NULL permission - and only >> require child DCmds to provide a permission if it differs from this >> default? We could then loose all 37 methods. >> >> --- >> >> Similar thoughts for "static DCmd impact()" >> >> which is not provided by: >> >> JMXStatusDCmd >> JMXStopRemoteDCmd >> JMXStartLocalDCmd >> >> which thus return the default impact - DCmd::impact() - which returns >> "Low". >> >> Here I think we should either remove the default implementation and >> force all commands to provide an implementation or stay with the >> default "Low" implementation and remove all 20+ functions in child >> classes which also return "Low". >> >> -- >> >> Finally, some thoughts for possible future cleanups: >> >> GenDCmdArgument internally uses C-heap to hold copies of strings >> (DCmdArgument and DCmdArgument). I wonder >> whether we could use resource area instead. Then we could get rid of >> all cleanup code (GenDCmdArgument::cleanup(), >> GenDCmdArgument::destroy_value()) etc. And if one agrees that parsing >> is only done once and the commands are not reused (which is the case >> currently) we also may get rid of DCmdArgument::reset() and friends. >> Coding would become a lot easier. >> >> -- >> >> IMHO the way object- und template based polymorphy is mixed is a bit >> of a mismatch here and it complicates things a bit. >> >> For instance, Dcmd has above mentioned static functions which return >> immutable attributes for the commands. It is "overwritten" >> template-style by redefining the same static functions in derived >> classes and called via DCmdFactoryImpl. >> >> Maybe it is just me, but I find this a bit roundabout for the problem >> it tries to solve. Also, it is really difficult to parse for modern >> IDEs - e.g. my CDT is unable to tell me who calls "DCmd::permission()" >> since the relation between DCmd::permission() and XXXCmd::permission() >> is just by name. Also, why pay for the text size of 47 just slightly >> different template definitions of DCmdFactory where one class >> would be enough? >> >> The mis-fit between templates and object polymorphism also shows up as >> weird artifacts. E.g. we have 27 variants of this function: >> >> int XXXXDCmd::num_arguments() { >> ?? ResourceMark rm; >> ?? XXXDCmd* dcmd = new SetVMFlagDCmd(NULL, false); >> ?? if (dcmd != NULL) { >> ???? DCmdMark mark(dcmd); >> ???? return dcmd->_dcmdparser.num_arguments(); >> ?? } else { >> ???? return 0; >> ?? } >> } >> >> - so, we want to implement "static XXXDCmd::num_arguments", which - >> being static - has to create an object of the same type to ask it its >> number of arguments. >> >> Maybe we could un-templify the command classes at some point to >> simplify all this? >> >> I think we need two things, a class to hold meta information about the >> command - name, description, argument types etc - which needs to be >> available at startup. And then, we need a second thing to hold runtime >> context: the concrete argument values the user gave us, the >> outputStream* pointer, the execute() function, maybe parser state... >> >> I think this may be expressed simpler and more efficiently with simple >> C++ objects and inheritance, without templates. We could also shrink >> the code quite a bit. And my CDT would find who calls "permissions()" >> :) >> >> >> Best Regards, Thomas >> >> ------- >> >> [1] >> http://mail.openjdk.java.net/pipermail/serviceability-dev/2018-May/023673.html >> >> >> On Wed, Jun 13, 2018 at 3:41 PM, Thomas St?fe >> wrote: >>> Hi all, >>> >>> while working on jcmd I saw some minor cleanup possibilities. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.00/webrev/ >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8204958 >>> >>> Most of them are const correctness and using initializer lists instead >>> of java-style member initialization in these C++ classes. >>> >>> Other changes: >>> >>> - Added a ResourceMark into the DCmd::parse_and_execute() loop >>> - removed DCmdFactory::create_global_DCmd() (and following that, >>> DCmdFactory::create_Cheap_instance()) since I did not find anyone >>> using that function. >>> - Also in DCmdFactory, I removed a number of setters which were never >>> called and for attributes which should be immutable: hidden, flags, >>> enabled. >>> >>> This is really a rather light cleanup. It did not touch any larger >>> issues. >>> >>> Thank you, >>> >>> Thomas > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From frederic.parain at oracle.com Fri Jun 15 15:32:26 2018 From: frederic.parain at oracle.com (Frederic Parain) Date: Fri, 15 Jun 2018 11:32:26 -0400 Subject: jcmd - executing multiple commands at the same safepoint? In-Reply-To: <3D31B965-68BE-4CBD-8952-907EBDA71BCE@oracle.com> References: <3D40CD1F-1DD0-44A4-A729-3781B6E3EA7D@gmail.com> <69360DA5-9287-4C84-B5FE-89B240D9C413@gmail.com> <90E966D9-476B-46D9-B08A-7968F6740851@gmail.com> <3606226E-E365-4F9E-B6F3-B573DE1A5579@gmail.com> <5B0486CC.8050801@oracle.com> <937f5c14-327b-5cc5-897a-09aaef349610@oracle.com> <3D31B965-68BE-4CBD-8952-907EBDA71BCE@oracle.com> Message-ID: <09FA5DC6-5839-4E13-9F8A-CBB772AEA2A3@oracle.com> Hi, Here?s a first prototype to explore running multiple commands in a single safepoint: http://cr.openjdk.java.net/~fparain/DCmdRunAtSafepoint/webrev.00/ The syntax is very simple, with the character ?+? used as the delimiter between commands. $ ./build/macosx-x64-debug/images/jdk/bin/jcmd 21164 VM.run_at_safepoint VM.version + VM.uptime 21164: Java HotSpot(TM) 64-Bit Server VM version 11-internal+0-2018-05-25-2202478.fparain.DCmd JDK 11.0.0 44.143 s Playing a bit with it, it quickly shows what are the issues when trying to run diagnostic commands at a safepoint. Issue 1: One important point to remember is that only the VMThread can run at a safepoint, all JavaThreads are strictly forbidden to run during the safepoint. So executing commands at a safepoint means they will be executed by the VMThread. Unfortunately, the VMThread is subject to restrictions and cannot execute all code that a JavaThread can execute. This issue is visible with the VM.info command, which takes the Heap_lock. The VMThread is not allowed to grab this lock because it could cause the VMThread to be blocked by a JavaThread. So far, VM.info cannot be run at a safepoint. Issue 2: Executing a code at a safepoint requires a VMOperation. VMOperations are made of a prologue, a doit() method witch contains the code executed at the safepoint, and an epilogue. The prologue and the epilogue are executed by the requester of the VMOperation, usually a JavaThread, and they are not executed at a safepoint. Prologues and epilogues are designed to execute code the VMThread cannot execute, like taking locks or loading classes. When the VMThread itself wants to execute a VMOperation, what is called a nested VMOperation, it only executes the doit() method, skipping the prologue and the epilogue. This is an issue for diagnostic commands using a VMOp in their implementation: when these commands are called while the JVM is already at a safepoint, their prologue and epilogue cannot be executed. This issue impacts Thread.print and GC.run. There are probably other issues still to be found, so I encourage people to test this patch to see if the combinations of commands they would like to run at a safepoint can indeed be run in this mode. Note that the support for invoking the new command from the DiagnosticCommand MBean has not be implemented. This is a not negligible work in order to correctly check Java Permissions before executing the commands. Let me know if you consider this code useful or if the limitations described above make this feature not as pretty as we wanted it to be. Regards, Fred > On May 29, 2018, at 15:54, Frederic Parain wrote: > > Hi Thomas, > > >> On May 26, 2018, at 04:07, Thomas St?fe wrote: >> > >>> >>> I think that it is a fundamental piece of the design. Adding a new diagnostic >>> command to group several commands under a unique safepoint: >>> 1 - requires explicit action from the user (and acknowledgment of the risks) >>> 2 - doesn?t touch the current syntax/semantic at all >>> >>> Another reason I?m pushing for a specific diagnostic command rather than >>> changing the syntax is remote execution. jcmd is not the only way to invoke >>> a diagnostic command, they can also be invoke remotely using JMX and the >>> com.sun.management.DiagnosticCommand dynamic MBean. Through the >>> platform MBean, users do not have access to the jcmd syntax, they see >>> each diagnostic command as a MBean operation (providing that they have >>> been registered with the DCmd_Source_MBean flag). The initial support >>> for remote execution was taking a single String and processed it just like >>> a string passed by jcmd, but people developing consoles for remote >>> management complained that this was not convenient for them, and asked >>> for something they could use more programmatically. >>> If the safepoint bundling is implemented in the syntax, JMX clients won?t >>> be able to use the feature. If we implement it with a new diagnostic command, >>> it will be visible from the MBean and they will be able to use it. >> >> That is interesting. Do you have examples for such programs? I was >> looking for ways of how to use jcmd (or equivalent) remotely. > > If you?re looking for a console application to invoke diagnostic command > on a remote JVM, Mission Control has a dedicated support for that in the > ?Diagnostic Commands? tab. > If you?re looking for exemples of source code doing remote invocations, > there?re several unit tests for the DiagnosticCommand MBean in the directory > test/jdk/com/sun/management/DiagnosticCommandMBean. > >> >>> >>> At last, creating a specific diagnostic command for safepoint bundling would >>> show the way on how to group commands for consistency. Safepoints is not >>> the only way to get consistency. For some JVM internals reasons, you might >>> want to group commands under a single lock, like the Threads_lock. Adding >>> a diagnostic command for each type of bundling seems more extensible. >>> >> >> You make good points. >> >> Adding a new command has also the advantage that it does not affect >> downward compatibility and hence would not require a CSR. CSR seems to >> slow down things a bit (I have a very simple CSR for jcmd open >> undecided since 12 days: >> https://bugs.openjdk.java.net/browse/JDK-8203043). So, while I think >> CSR are a necessary process, I am happy to avoid them if possible. >> >> Lets spin this out a bit more: >> >> -> However the user specifies "please execute these commands at a >> single safepoint", if one of the enclosed commands cannot be run >> inside a safepoint the whole bundle should fail, agreed? So, >> specifying e.g. JFR.start as part of such a command bundle would cause >> the whole bundle to not being executed. > > Yes. The new command N would ask for the execution of a list of > commands { A, B, C, ? }, and would fail if: > - any of the specified commands { A, B, C, ? } is not a registered > command > - any of the specified commands { A, B, C, ? } doesn?t support > execution at safepoint > - any of the specified commands { A, B, C, ? } is not exported to > the DCmdSource used to invoke N (*) > > (*) Not all commands are exported to all interfaces (jcmd,JMX), for > instance, some commands related to the JMX agent are cannot > be invoked through the JMX. Even if the new command can be > invoked from all interfaces, it must not allow to circumvent this > restriction. This should not be an issue today, because all > JMX agent commands are calling back Java code, and are not > suitable for execution at a safepoint. But this would ensure the > consistency of the model in the long term (and the test is cheap). > >> >> -> If we introduce a command like you propose (lets call it "VM.run" >> for the sake of the argument) we still need to decide how to delimit >> the commands at the command line. That brings us back to the syntax >> question: separation by a explicit delimiter, if possible one that >> does not clash with common unix shells? >> >> Lets try colon ':' : >> >> jcmd VM.run VM.uptime : VM.metaspace show-loaders show-classes >> by-chunktype : VM.classloaders show-classes : VM.info >> >> Of course, we can wrap all arguments to jcmd in quotes, then the >> delimiter sign does not affect things as much, unless we use quotes :) >> We could e.g. use semicolon: >> >> jcmd ' VM.run VM.uptime ; VM.metaspace show-loaders show-classes >> by-chunktype ; VM.classloaders show-classes ; VM.info ? >> > > I thought about colon and semi-colon too, because they are the most > obvious delimiters. However, they are already as delimiters in classpaths > (semi-colon on Unix, colon on Windows), so parsing to find the right > delimiters would be tricky. > > I?d start with ?!? or ?+? in a first time, to make some progress on the other > aspects of the command, and buy some time to discuss the delimiter > issue in parallel. > > The last resort would be to force each command in the bundle to > be specified with quotes (arguments included). But it would make the > syntax very heavy, I?d prefer a simpler delimiter. > > >> Or, we go back to my original idea of letting the command keyword be >> the indicator for a new command start. Honestly, at this point I have >> no preference. Both approaches seem reasonable. >> >> The only niggle I have with an explicit command is the syntax, which I >> find a bit awkward. My thinking before reading your response was that >> I'd prefer a jcmd flag to specify the same behavior: that all commands >> should be executed at a single safepoint. E.g. [-s/--stacked] >> >> jcmd -s ' VM.uptime ; VM.metaspace show-loaders show-classes >> by-chunktype ; VM.classloaders show-classes ; VM.info ' >> >> But I see the disadvantage: depending on how it is implemented this >> would require consistent changes in both jcmd and hotspot, and not >> work with other clients using MBeans. Whereas the hypothetical >> "VM.run" would work out of the box for any old jcmd version and any >> other client. >> >> Of course one could keep the "VM.run" idea and still give jcmd a >> "-s/--stacked" option, as a shallow convenience feature, which just >> translates to "VM.run" and runs that. >> >> -> A variation of your "have special commands to run a command bundle >> in one particular context" theme could be having only one command, >> e.g. "VM.run", specifying the conditions under which to run the bundle >> as sub options. E.g. "VM.run -s " for "run commands at >> safepoint", "VM.run -t " for "run commands under >> Threads_lock". That would even allow to combine these contexts. I do >> not know if that makes sense. >> >> - Another advantage of having an explicit command just occurring to me >> is that it enables more evolved command files, where one can >> intersperse command bundles and single commands. For instance, I could >> give this to a customer: >> >> >> 1 VM.uptime >> 2 VM.run_commands VM.metaspace show-loaders show-classes by-chunktype >> VM.classloaders details VM.classloader_stats >> 3 GC.run >> 4 VM.run_commands VM.metaspace show-loaders show-classes by-chunktype >> VM.classloaders details VM.classloader_stats >> >> to execute a command bundle before and after triggering a GC. > > This would work fine . The parser of the jcmd string will split the > string based on ?\n? characters, then each line of command will be executed > sequentially, and lines containing the VM.run_command will then create the > safepoint and trigger the execution of the different commands specified in > its argument list. > >> >>> We have a three days week-end in the US, but I should be able to code >>> something next week. >>> >> >> Oh, you want to take over the implementation, sure! Fine by me. > > I?m not promising an implementation, I haven?t worked on this code > since I left the serviceability team. I need to code at least a skeleton > of the command to refresh my memory about all aspects of the > diagnostic command framework. > >> >>> Thank you for your ideas and your interest in diagnostic commands. >> >> Sure. Why I am interested in this: We strongly rely on tools similar >> to jcmd for our support. We have quite capable tools in our (licensed, >> non-open) VM port, which are similar to jcmd in many aspects. So we >> try to bring some of the capabilities upstream, in order to ease the >> merge costs for us and to help the OpenJDK community. > > The diagnostic command framework was designed to support an > extensible set of commands. Feel free to extend, and thank you > for contributing to the OpenJDK community. > > Fred > >>> >>> >>>> >>>>> >>>>> A new separator, more convenient than newline, would be require to have >>>>> a single line syntax. >>>>> >>>>> My 2 cents, >>>>> >>>> >>>> Certainly worth more than 2 cents :) Thanks for your thoughts! >>>> >>>> ..Thomas >>>> >>>>> Fred >>>>> >>>>> >>>>>> >>>>>>> >>>>>>> Erik >>>>>>> >>>>>>> Hi Thomas, >>>>>>> >>>>>>> very positive suggestion. >>>>>>> >>>>>>> One observation: >>>>>>> There already seems to be some different interpretation of the command >>>>>>> line >>>>>>> in different Java versions: >>>>>>> For instance when I try to dump a flight recording in 1.8.0_152-ea-b05 : >>>>>>> I >>>>>>> can use: >>>>>>> jcmd 33014 JFR.dump filename="recording1.jfr" recording=1 >>>>>>> but in build 9+181 , the same command must be: >>>>>>> jcmd 33014 JFR.dump filename="recording1.jfr",recording=1 >>>>>>> (the comma to separate sub-options you mentioned earlier) >>>>>>> >>>>>>> Therefore the suggestion without curly brackets, giving several commands >>>>>>> after one another seems good with regard to backwards compatibility. >>>>>>> >>>>>>> Motivation: hawt.io uses the MBean >>>>>>> com.sun.management:type=DiagnosticCommand >>>>>>> to access the same functionality as jcmd. Variations in option syntax >>>>>>> across >>>>>>> Java versions is already an issue (only affecting sub-options though, as >>>>>>> each command is a separate JMX operation). So syntax compatibility is >>>>>>> highly >>>>>>> appreciated :-) >>>>>>> >>>>>>> Martin >>>>>>> >>>>>>> l?r. 12. mai 2018 kl. 20:11 skrev Kirk Pepperdine >>>>>>> : >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> On May 10, 2018, at 11:26 AM, Thomas St?fe >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> On Thu, May 10, 2018 at 9:13 AM, Kirk Pepperdine >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> The stacking at the safe point would be a huge win. Right now thread >>>>>>>>>> dump consistency can really confuse people as the tooling will show >>>>>>>>>> two >>>>>>>>>> threads owning the same lock at seemingly the same time. Reality, it?s >>>>>>>>>> just >>>>>>>>>> that people didn?t realize you get a safe point for each thread >>>>>>>>>> therefor >>>>>>>>>> there is motion in the system as you?re collecting the data. >>>>>>>>> >>>>>>>>> >>>>>>>>> I am a bit confused. What tooling are you talking about? >>>>>>>> >>>>>>>> >>>>>>>> jstack. I?ve not seen it with jcmd. But I often see 2 threads holding >>>>>>>> the >>>>>>>> same lock at the ?same time? which is of course non-sense. I can dig one >>>>>>>> up >>>>>>>> for you if you?re still confused. >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> As an aside, it?s amazing how many dev?s aren?t aware of jcmd. Just >>>>>>>>>> yesterday after my session at Devoxx I had someone ask me about using >>>>>>>>>> jfr >>>>>>>>>> from the command line, many in that session had not seen jcmd before. >>>>>>>>>> The >>>>>>>>>> feedback was, wow, this is very useful and I wished I had of known >>>>>>>>>> about it >>>>>>>>>> earlier. >>>>>>>>> >>>>>>>>> >>>>>>>>> Yes, jcmd is quite useful. I also really like the simple design, which >>>>>>>>> is by default up- and downward compatible (so you can talk to any VM, >>>>>>>>> new, old, it should not matter). That is just good design. We - Sap - >>>>>>>>> work to extend jcmd, to help our support folks. E.g. the whole new >>>>>>>>> VM.metaspace command chain. >>>>>>>> >>>>>>>> >>>>>>>> And simple it is?.well done!!! >>>>>>>> >>>>>>>> ? Kirk >>>>>>>> >>>>>>> >>>>> >>> > From jeremymanson at google.com Fri Jun 15 16:33:21 2018 From: jeremymanson at google.com (Jeremy Manson) Date: Fri, 15 Jun 2018 09:33:21 -0700 Subject: RFR 8205113: Update JVMTI doc references to object allocation tracking Message-ID: Hi all, There are a number of references in the JVMTI doc to its not doing object allocation tracking. Now that JEP 331 has landed, these references are obsolete. This patch changes those references accordingly. While I was there, I took the liberty of fixing some spelling errors. As far as I know, these are non-normative changes and modify no API, so they should not require a CSR. Bug: https://bugs.openjdk.java.net/browse/JDK-8205113 Webrev: http://cr.openjdk.java.net/~jmanson/8205113/webrev.00/ Thanks! Jeremy -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.stuefe at gmail.com Fri Jun 15 17:31:53 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 15 Jun 2018 19:31:53 +0200 Subject: RFR(s): 8204958: Minor cleanups for the diagnostic framework In-Reply-To: References: Message-ID: Coleen, Chris, thanks for the review. Please find the latest webrev: Delta: http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.01-to-02/webrev/index.html Full: http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.02/webrev/ I changed the copyright dates, the format of the initializer lists (I looked around and used what the most common denominator seemed to be - still, Coleen, should you ever dig up that style guide of yours, that would be valuable), and initialized CmdLine::_cmd to line as Coleen sugggested. @Chris > I'm not certain why you added default field initializers for CmdLine::CmdLine when each field is explicitly initialized within the method body. See Coleens response. I usually prefer it that way if the constructor is not extremely obvious, to prevent missing initializations should the constructor body evolve. C++ compiler should optimize that away anyway. Thanks for the reviews, Best Regards, Thomas On Fri, Jun 15, 2018 at 1:53 PM, wrote: > > > On 6/15/18 1:04 AM, Chris Plummer wrote: > > Hi Thomas, > > I'm not certain why you added default field initializers for > CmdLine::CmdLine when each field is explicitly initialized within the method > body. > > > I think the C++ optimizer would elide any unnecessary initializations and it > seems safe in case new fields are added, although > > 38 : _cmd(NULL) > > > could be initialized as _cmd(line). > > Can I bikeshed and say I don't like the style where the , are at the > beginning of the lines? We don't use this style in other places, that I've > noticed, and I really don't like reading the punctuation at the beginning of > the line. If I could find our open source wiki for the coding style among > the broken openjdk wiki pages, I would see if the punctuation rule is there. > One initializer per line is ok. > > Thanks, > Coleen > > > A number of copyrights need updating. > > The rest of the changes look good. > > thanks, > > Chris > > On 6/14/18 1:38 AM, Thomas St?fe wrote: > > Hi all, > > (I moved the issue to target release 12 - if we get this through the > review quick, fine, but no reason to add pressure) > > some more background: The reason for these cleanups is that I later > plan to add a new sub command which allows us to run multiple commands > in one safepoint (see [1] for more details and discussions). Since > this will require tweaking the parse process a bit, I try to loosen up > the coding before adding complexity again. > > Please find here the new webrev: > > Delta: > http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.00-to-01/webrev/ > Full: > http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.01/webrev/ > > What changed: > > Aesthetics: > - I reordered member init lists to member order, as Coleen did ask. I > also reform them as lists, just to increase readability. > - For a number of inline methods, I aligned their implementations so > that they too would be better readable and it would be clearer which > methods are const > - Moved DCmdFactory::_DCmdFactoryList up to the other static members. > > Functional: > - Added initializer list for CmdLine > - Added missing initializations for DCmdArgIter members _key_addr, > _key_len, _value_addr, _value_len, as Coleen suggested > - constified DCmdInfo too - since this is an immutable value holder, > all members can be const. > - also in DCmdInfo, changed JavaPermission permission() to const > JavaPermission& permission() to save a bit unnecessary copying. > - Fixed initialization of DCmdArgumentInfo::_position as Serguei and > Chris requested > - Removed default implementation of DCmd::name and DCmd::description. > There is no good reason to provide one - all child classes should (and > do) provide an implementation. I rather have linkage errors if someone > forgets to add one. > - Removed default implementations for DCmdWithParser::name/description > - same reasoning as above > - Removed DCmdWithParser::permission(), it is already provided by the > base class. > - Removed DCmdWithParser::num_arguments() since childs of > DCmdWithParser will always have arguments and hence should implement > this function. > - Made all functions in DCmdFactoryImpl non-virtual. There is no point > in allowing to override them. > > -- > > Notes: > > I see the following commands do not implement "permission" and hence > rely on the default impl of DCmd::permission, which returns a > permission structure { NULL, NULL, NULL } - is that okay and intended? > > JMXStopRemoteDCmd > JMXStartLocalDCmd > JMXStartRemoteDCmd > TouchedMethodsDCmd > ClassStatsDCmd > RunFinalizationDCmd > SystemGCDCmd > VMUptimeDCmd > HelpDCmd > > I am not sure where and how these permissions are tested. > > Thinking through this further: > > Of the 47 child classes of DCmd, 37 seem to return > "java.lang.management.ManagementPermission", name "monitor". So, we > have this function 37 times: > > static const JavaPermission permission() { > JavaPermission p = {"java.lang.management.ManagementPermission", > "monitor", NULL}; > return p; > } > > Would it then not make sense to make this the default implementation > for DCmd::permission() - instead of the NULL permission - and only > require child DCmds to provide a permission if it differs from this > default? We could then loose all 37 methods. > > --- > > Similar thoughts for "static DCmd impact()" > > which is not provided by: > > JMXStatusDCmd > JMXStopRemoteDCmd > JMXStartLocalDCmd > > which thus return the default impact - DCmd::impact() - which returns "Low". > > Here I think we should either remove the default implementation and > force all commands to provide an implementation or stay with the > default "Low" implementation and remove all 20+ functions in child > classes which also return "Low". > > -- > > Finally, some thoughts for possible future cleanups: > > GenDCmdArgument internally uses C-heap to hold copies of strings > (DCmdArgument and DCmdArgument). I wonder > whether we could use resource area instead. Then we could get rid of > all cleanup code (GenDCmdArgument::cleanup(), > GenDCmdArgument::destroy_value()) etc. And if one agrees that parsing > is only done once and the commands are not reused (which is the case > currently) we also may get rid of DCmdArgument::reset() and friends. > Coding would become a lot easier. > > -- > > IMHO the way object- und template based polymorphy is mixed is a bit > of a mismatch here and it complicates things a bit. > > For instance, Dcmd has above mentioned static functions which return > immutable attributes for the commands. It is "overwritten" > template-style by redefining the same static functions in derived > classes and called via DCmdFactoryImpl. > > Maybe it is just me, but I find this a bit roundabout for the problem > it tries to solve. Also, it is really difficult to parse for modern > IDEs - e.g. my CDT is unable to tell me who calls "DCmd::permission()" > since the relation between DCmd::permission() and XXXCmd::permission() > is just by name. Also, why pay for the text size of 47 just slightly > different template definitions of DCmdFactory where one class > would be enough? > > The mis-fit between templates and object polymorphism also shows up as > weird artifacts. E.g. we have 27 variants of this function: > > int XXXXDCmd::num_arguments() { > ResourceMark rm; > XXXDCmd* dcmd = new SetVMFlagDCmd(NULL, false); > if (dcmd != NULL) { > DCmdMark mark(dcmd); > return dcmd->_dcmdparser.num_arguments(); > } else { > return 0; > } > } > > - so, we want to implement "static XXXDCmd::num_arguments", which - > being static - has to create an object of the same type to ask it its > number of arguments. > > Maybe we could un-templify the command classes at some point to > simplify all this? > > I think we need two things, a class to hold meta information about the > command - name, description, argument types etc - which needs to be > available at startup. And then, we need a second thing to hold runtime > context: the concrete argument values the user gave us, the > outputStream* pointer, the execute() function, maybe parser state... > > I think this may be expressed simpler and more efficiently with simple > C++ objects and inheritance, without templates. We could also shrink > the code quite a bit. And my CDT would find who calls "permissions()" > :) > > > Best Regards, Thomas > > ------- > > [1] > http://mail.openjdk.java.net/pipermail/serviceability-dev/2018-May/023673.html > > > On Wed, Jun 13, 2018 at 3:41 PM, Thomas St?fe > wrote: > > Hi all, > > while working on jcmd I saw some minor cleanup possibilities. > > Webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.00/webrev/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8204958 > > Most of them are const correctness and using initializer lists instead > of java-style member initialization in these C++ classes. > > Other changes: > > - Added a ResourceMark into the DCmd::parse_and_execute() loop > - removed DCmdFactory::create_global_DCmd() (and following that, > DCmdFactory::create_Cheap_instance()) since I did not find anyone > using that function. > - Also in DCmdFactory, I removed a number of setters which were never > called and for attributes which should be immutable: hidden, flags, > enabled. > > This is really a rather light cleanup. It did not touch any larger issues. > > Thank you, > > Thomas > > > > > From thomas.stuefe at gmail.com Fri Jun 15 18:09:20 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 15 Jun 2018 20:09:20 +0200 Subject: jcmd - executing multiple commands at the same safepoint? In-Reply-To: <09FA5DC6-5839-4E13-9F8A-CBB772AEA2A3@oracle.com> References: <3D40CD1F-1DD0-44A4-A729-3781B6E3EA7D@gmail.com> <69360DA5-9287-4C84-B5FE-89B240D9C413@gmail.com> <90E966D9-476B-46D9-B08A-7968F6740851@gmail.com> <3606226E-E365-4F9E-B6F3-B573DE1A5579@gmail.com> <5B0486CC.8050801@oracle.com> <937f5c14-327b-5cc5-897a-09aaef349610@oracle.com> <3D31B965-68BE-4CBD-8952-907EBDA71BCE@oracle.com> <09FA5DC6-5839-4E13-9F8A-CBB772AEA2A3@oracle.com> Message-ID: Hi Frederic, this is nice and now I am surprised how small the change is :) I played around with it a bit, combining VM.metaspace, VM.classloaders and VM.systemdictionary, including options. All worked well. VM.stringtable did not produce output and VM.info, as you predicted, crashes the VM because it needs the heap lock. (Side note: I think VM.info is too useful to be held back by not getting the heap lock. It would be nice if it would try to lock and if failing, would just omit the steps needing a heap lock.) So, yes, on first glance I find this useful. I'll play around with this a bit more. Lets wait for others to chime in. Best Regards, Thomas On Fri, Jun 15, 2018 at 5:32 PM, Frederic Parain wrote: > Hi, > > Here?s a first prototype to explore running multiple commands in a single safepoint: > > http://cr.openjdk.java.net/~fparain/DCmdRunAtSafepoint/webrev.00/ > > The syntax is very simple, with the character ?+? used as the delimiter between commands. > > $ ./build/macosx-x64-debug/images/jdk/bin/jcmd 21164 VM.run_at_safepoint VM.version + VM.uptime > 21164: > Java HotSpot(TM) 64-Bit Server VM version 11-internal+0-2018-05-25-2202478.fparain.DCmd > JDK 11.0.0 > 44.143 s > > Playing a bit with it, it quickly shows what are the issues when trying to run > diagnostic commands at a safepoint. > > Issue 1: One important point to remember is that only the VMThread can run at a safepoint, > all JavaThreads are strictly forbidden to run during the safepoint. So executing commands at a > safepoint means they will be executed by the VMThread. Unfortunately, the VMThread is > subject to restrictions and cannot execute all code that a JavaThread can execute. This > issue is visible with the VM.info command, which takes the Heap_lock. The VMThread > is not allowed to grab this lock because it could cause the VMThread to be blocked by > a JavaThread. So far, VM.info cannot be run at a safepoint. > > Issue 2: Executing a code at a safepoint requires a VMOperation. VMOperations are > made of a prologue, a doit() method witch contains the code executed at the safepoint, > and an epilogue. The prologue and the epilogue are executed by the requester of the > VMOperation, usually a JavaThread, and they are not executed at a safepoint. Prologues > and epilogues are designed to execute code the VMThread cannot execute, like taking > locks or loading classes. When the VMThread itself wants to execute a VMOperation, > what is called a nested VMOperation, it only executes the doit() method, skipping the > prologue and the epilogue. This is an issue for diagnostic commands using a VMOp > in their implementation: when these commands are called while the JVM is already > at a safepoint, their prologue and epilogue cannot be executed. This issue impacts > Thread.print and GC.run. > > There are probably other issues still to be found, so I encourage people to test > this patch to see if the combinations of commands they would like to run at a > safepoint can indeed be run in this mode. > > Note that the support for invoking the new command from the DiagnosticCommand > MBean has not be implemented. This is a not negligible work in order to correctly > check Java Permissions before executing the commands. > > Let me know if you consider this code useful or if the limitations described above > make this feature not as pretty as we wanted it to be. > > Regards, > > Fred > > >> On May 29, 2018, at 15:54, Frederic Parain wrote: >> >> Hi Thomas, >> >> >>> On May 26, 2018, at 04:07, Thomas St?fe wrote: >>> >> >>>> >>>> I think that it is a fundamental piece of the design. Adding a new diagnostic >>>> command to group several commands under a unique safepoint: >>>> 1 - requires explicit action from the user (and acknowledgment of the risks) >>>> 2 - doesn?t touch the current syntax/semantic at all >>>> >>>> Another reason I?m pushing for a specific diagnostic command rather than >>>> changing the syntax is remote execution. jcmd is not the only way to invoke >>>> a diagnostic command, they can also be invoke remotely using JMX and the >>>> com.sun.management.DiagnosticCommand dynamic MBean. Through the >>>> platform MBean, users do not have access to the jcmd syntax, they see >>>> each diagnostic command as a MBean operation (providing that they have >>>> been registered with the DCmd_Source_MBean flag). The initial support >>>> for remote execution was taking a single String and processed it just like >>>> a string passed by jcmd, but people developing consoles for remote >>>> management complained that this was not convenient for them, and asked >>>> for something they could use more programmatically. >>>> If the safepoint bundling is implemented in the syntax, JMX clients won?t >>>> be able to use the feature. If we implement it with a new diagnostic command, >>>> it will be visible from the MBean and they will be able to use it. >>> >>> That is interesting. Do you have examples for such programs? I was >>> looking for ways of how to use jcmd (or equivalent) remotely. >> >> If you?re looking for a console application to invoke diagnostic command >> on a remote JVM, Mission Control has a dedicated support for that in the >> ?Diagnostic Commands? tab. >> If you?re looking for exemples of source code doing remote invocations, >> there?re several unit tests for the DiagnosticCommand MBean in the directory >> test/jdk/com/sun/management/DiagnosticCommandMBean. >> >>> >>>> >>>> At last, creating a specific diagnostic command for safepoint bundling would >>>> show the way on how to group commands for consistency. Safepoints is not >>>> the only way to get consistency. For some JVM internals reasons, you might >>>> want to group commands under a single lock, like the Threads_lock. Adding >>>> a diagnostic command for each type of bundling seems more extensible. >>>> >>> >>> You make good points. >>> >>> Adding a new command has also the advantage that it does not affect >>> downward compatibility and hence would not require a CSR. CSR seems to >>> slow down things a bit (I have a very simple CSR for jcmd open >>> undecided since 12 days: >>> https://bugs.openjdk.java.net/browse/JDK-8203043). So, while I think >>> CSR are a necessary process, I am happy to avoid them if possible. >>> >>> Lets spin this out a bit more: >>> >>> -> However the user specifies "please execute these commands at a >>> single safepoint", if one of the enclosed commands cannot be run >>> inside a safepoint the whole bundle should fail, agreed? So, >>> specifying e.g. JFR.start as part of such a command bundle would cause >>> the whole bundle to not being executed. >> >> Yes. The new command N would ask for the execution of a list of >> commands { A, B, C, ? }, and would fail if: >> - any of the specified commands { A, B, C, ? } is not a registered >> command >> - any of the specified commands { A, B, C, ? } doesn?t support >> execution at safepoint >> - any of the specified commands { A, B, C, ? } is not exported to >> the DCmdSource used to invoke N (*) >> >> (*) Not all commands are exported to all interfaces (jcmd,JMX), for >> instance, some commands related to the JMX agent are cannot >> be invoked through the JMX. Even if the new command can be >> invoked from all interfaces, it must not allow to circumvent this >> restriction. This should not be an issue today, because all >> JMX agent commands are calling back Java code, and are not >> suitable for execution at a safepoint. But this would ensure the >> consistency of the model in the long term (and the test is cheap). >> >>> >>> -> If we introduce a command like you propose (lets call it "VM.run" >>> for the sake of the argument) we still need to decide how to delimit >>> the commands at the command line. That brings us back to the syntax >>> question: separation by a explicit delimiter, if possible one that >>> does not clash with common unix shells? >>> >>> Lets try colon ':' : >>> >>> jcmd VM.run VM.uptime : VM.metaspace show-loaders show-classes >>> by-chunktype : VM.classloaders show-classes : VM.info >>> >>> Of course, we can wrap all arguments to jcmd in quotes, then the >>> delimiter sign does not affect things as much, unless we use quotes :) >>> We could e.g. use semicolon: >>> >>> jcmd ' VM.run VM.uptime ; VM.metaspace show-loaders show-classes >>> by-chunktype ; VM.classloaders show-classes ; VM.info ? >>> >> >> I thought about colon and semi-colon too, because they are the most >> obvious delimiters. However, they are already as delimiters in classpaths >> (semi-colon on Unix, colon on Windows), so parsing to find the right >> delimiters would be tricky. >> >> I?d start with ?!? or ?+? in a first time, to make some progress on the other >> aspects of the command, and buy some time to discuss the delimiter >> issue in parallel. >> >> The last resort would be to force each command in the bundle to >> be specified with quotes (arguments included). But it would make the >> syntax very heavy, I?d prefer a simpler delimiter. >> >> >>> Or, we go back to my original idea of letting the command keyword be >>> the indicator for a new command start. Honestly, at this point I have >>> no preference. Both approaches seem reasonable. >>> >>> The only niggle I have with an explicit command is the syntax, which I >>> find a bit awkward. My thinking before reading your response was that >>> I'd prefer a jcmd flag to specify the same behavior: that all commands >>> should be executed at a single safepoint. E.g. [-s/--stacked] >>> >>> jcmd -s ' VM.uptime ; VM.metaspace show-loaders show-classes >>> by-chunktype ; VM.classloaders show-classes ; VM.info ' >>> >>> But I see the disadvantage: depending on how it is implemented this >>> would require consistent changes in both jcmd and hotspot, and not >>> work with other clients using MBeans. Whereas the hypothetical >>> "VM.run" would work out of the box for any old jcmd version and any >>> other client. >>> >>> Of course one could keep the "VM.run" idea and still give jcmd a >>> "-s/--stacked" option, as a shallow convenience feature, which just >>> translates to "VM.run" and runs that. >>> >>> -> A variation of your "have special commands to run a command bundle >>> in one particular context" theme could be having only one command, >>> e.g. "VM.run", specifying the conditions under which to run the bundle >>> as sub options. E.g. "VM.run -s " for "run commands at >>> safepoint", "VM.run -t " for "run commands under >>> Threads_lock". That would even allow to combine these contexts. I do >>> not know if that makes sense. >>> >>> - Another advantage of having an explicit command just occurring to me >>> is that it enables more evolved command files, where one can >>> intersperse command bundles and single commands. For instance, I could >>> give this to a customer: >>> >>> >>> 1 VM.uptime >>> 2 VM.run_commands VM.metaspace show-loaders show-classes by-chunktype >>> VM.classloaders details VM.classloader_stats >>> 3 GC.run >>> 4 VM.run_commands VM.metaspace show-loaders show-classes by-chunktype >>> VM.classloaders details VM.classloader_stats >>> >>> to execute a command bundle before and after triggering a GC. >> >> This would work fine . The parser of the jcmd string will split the >> string based on ?\n? characters, then each line of command will be executed >> sequentially, and lines containing the VM.run_command will then create the >> safepoint and trigger the execution of the different commands specified in >> its argument list. >> >>> >>>> We have a three days week-end in the US, but I should be able to code >>>> something next week. >>>> >>> >>> Oh, you want to take over the implementation, sure! Fine by me. >> >> I?m not promising an implementation, I haven?t worked on this code >> since I left the serviceability team. I need to code at least a skeleton >> of the command to refresh my memory about all aspects of the >> diagnostic command framework. >> >>> >>>> Thank you for your ideas and your interest in diagnostic commands. >>> >>> Sure. Why I am interested in this: We strongly rely on tools similar >>> to jcmd for our support. We have quite capable tools in our (licensed, >>> non-open) VM port, which are similar to jcmd in many aspects. So we >>> try to bring some of the capabilities upstream, in order to ease the >>> merge costs for us and to help the OpenJDK community. >> >> The diagnostic command framework was designed to support an >> extensible set of commands. Feel free to extend, and thank you >> for contributing to the OpenJDK community. >> >> Fred >> >>>> >>>> >>>>> >>>>>> >>>>>> A new separator, more convenient than newline, would be require to have >>>>>> a single line syntax. >>>>>> >>>>>> My 2 cents, >>>>>> >>>>> >>>>> Certainly worth more than 2 cents :) Thanks for your thoughts! >>>>> >>>>> ..Thomas >>>>> >>>>>> Fred >>>>>> >>>>>> >>>>>>> >>>>>>>> >>>>>>>> Erik >>>>>>>> >>>>>>>> Hi Thomas, >>>>>>>> >>>>>>>> very positive suggestion. >>>>>>>> >>>>>>>> One observation: >>>>>>>> There already seems to be some different interpretation of the command >>>>>>>> line >>>>>>>> in different Java versions: >>>>>>>> For instance when I try to dump a flight recording in 1.8.0_152-ea-b05 : >>>>>>>> I >>>>>>>> can use: >>>>>>>> jcmd 33014 JFR.dump filename="recording1.jfr" recording=1 >>>>>>>> but in build 9+181 , the same command must be: >>>>>>>> jcmd 33014 JFR.dump filename="recording1.jfr",recording=1 >>>>>>>> (the comma to separate sub-options you mentioned earlier) >>>>>>>> >>>>>>>> Therefore the suggestion without curly brackets, giving several commands >>>>>>>> after one another seems good with regard to backwards compatibility. >>>>>>>> >>>>>>>> Motivation: hawt.io uses the MBean >>>>>>>> com.sun.management:type=DiagnosticCommand >>>>>>>> to access the same functionality as jcmd. Variations in option syntax >>>>>>>> across >>>>>>>> Java versions is already an issue (only affecting sub-options though, as >>>>>>>> each command is a separate JMX operation). So syntax compatibility is >>>>>>>> highly >>>>>>>> appreciated :-) >>>>>>>> >>>>>>>> Martin >>>>>>>> >>>>>>>> l?r. 12. mai 2018 kl. 20:11 skrev Kirk Pepperdine >>>>>>>> : >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> On May 10, 2018, at 11:26 AM, Thomas St?fe >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> On Thu, May 10, 2018 at 9:13 AM, Kirk Pepperdine >>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> The stacking at the safe point would be a huge win. Right now thread >>>>>>>>>>> dump consistency can really confuse people as the tooling will show >>>>>>>>>>> two >>>>>>>>>>> threads owning the same lock at seemingly the same time. Reality, it?s >>>>>>>>>>> just >>>>>>>>>>> that people didn?t realize you get a safe point for each thread >>>>>>>>>>> therefor >>>>>>>>>>> there is motion in the system as you?re collecting the data. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> I am a bit confused. What tooling are you talking about? >>>>>>>>> >>>>>>>>> >>>>>>>>> jstack. I?ve not seen it with jcmd. But I often see 2 threads holding >>>>>>>>> the >>>>>>>>> same lock at the ?same time? which is of course non-sense. I can dig one >>>>>>>>> up >>>>>>>>> for you if you?re still confused. >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> As an aside, it?s amazing how many dev?s aren?t aware of jcmd. Just >>>>>>>>>>> yesterday after my session at Devoxx I had someone ask me about using >>>>>>>>>>> jfr >>>>>>>>>>> from the command line, many in that session had not seen jcmd before. >>>>>>>>>>> The >>>>>>>>>>> feedback was, wow, this is very useful and I wished I had of known >>>>>>>>>>> about it >>>>>>>>>>> earlier. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Yes, jcmd is quite useful. I also really like the simple design, which >>>>>>>>>> is by default up- and downward compatible (so you can talk to any VM, >>>>>>>>>> new, old, it should not matter). That is just good design. We - Sap - >>>>>>>>>> work to extend jcmd, to help our support folks. E.g. the whole new >>>>>>>>>> VM.metaspace command chain. >>>>>>>>> >>>>>>>>> >>>>>>>>> And simple it is?.well done!!! >>>>>>>>> >>>>>>>>> ? Kirk >>>>>>>>> >>>>>>>> >>>>>> >>>> >> > From chris.plummer at oracle.com Fri Jun 15 20:36:19 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Fri, 15 Jun 2018 13:36:19 -0700 Subject: RFR(s): 8204958: Minor cleanups for the diagnostic framework In-Reply-To: References: Message-ID: <58051f76-31b9-658c-cfb3-7f10ac7d00c7@oracle.com> On 6/15/18 10:31 AM, Thomas St?fe wrote: > Coleen, Chris, > > thanks for the review. Please find the latest webrev: > > Delta: http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.01-to-02/webrev/index.html > Full: http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.02/webrev/ > > I changed the copyright dates, the format of the initializer lists (I > looked around and used what the most common denominator seemed to be - > still, Coleen, should you ever dig up that style guide of yours, that > would be valuable), and initialized CmdLine::_cmd to line as Coleen > sugggested. > > @Chris >> I'm not certain why you added default field initializers for CmdLine::CmdLine when each field is explicitly initialized within the method body. > See Coleens response. I usually prefer it that way if the constructor > is not extremely obvious, to prevent missing initializations should > the constructor body evolve. C++ compiler should optimize that away > anyway. I tend to think the opposite. Using the initializer list in this way would lead me to think that the field was not initialized in the constructor body, or at the very least there was a path where it might not be initialized. I don't see how this helps when the constructor is not obvious. Is the point of the initializer (in this case) solely to have the code document which fields get initialized in the constructor? thanks, Chris > Thanks for the reviews, > > Best Regards, Thomas > > > On Fri, Jun 15, 2018 at 1:53 PM, wrote: >> >> On 6/15/18 1:04 AM, Chris Plummer wrote: >> >> Hi Thomas, >> >> I'm not certain why you added default field initializers for >> CmdLine::CmdLine when each field is explicitly initialized within the method >> body. >> >> >> I think the C++ optimizer would elide any unnecessary initializations and it >> seems safe in case new fields are added, although >> >> 38 : _cmd(NULL) >> >> >> could be initialized as _cmd(line). >> >> Can I bikeshed and say I don't like the style where the , are at the >> beginning of the lines? We don't use this style in other places, that I've >> noticed, and I really don't like reading the punctuation at the beginning of >> the line. If I could find our open source wiki for the coding style among >> the broken openjdk wiki pages, I would see if the punctuation rule is there. >> One initializer per line is ok. >> >> Thanks, >> Coleen >> >> >> A number of copyrights need updating. >> >> The rest of the changes look good. >> >> thanks, >> >> Chris >> >> On 6/14/18 1:38 AM, Thomas St?fe wrote: >> >> Hi all, >> >> (I moved the issue to target release 12 - if we get this through the >> review quick, fine, but no reason to add pressure) >> >> some more background: The reason for these cleanups is that I later >> plan to add a new sub command which allows us to run multiple commands >> in one safepoint (see [1] for more details and discussions). Since >> this will require tweaking the parse process a bit, I try to loosen up >> the coding before adding complexity again. >> >> Please find here the new webrev: >> >> Delta: >> http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.00-to-01/webrev/ >> Full: >> http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.01/webrev/ >> >> What changed: >> >> Aesthetics: >> - I reordered member init lists to member order, as Coleen did ask. I >> also reform them as lists, just to increase readability. >> - For a number of inline methods, I aligned their implementations so >> that they too would be better readable and it would be clearer which >> methods are const >> - Moved DCmdFactory::_DCmdFactoryList up to the other static members. >> >> Functional: >> - Added initializer list for CmdLine >> - Added missing initializations for DCmdArgIter members _key_addr, >> _key_len, _value_addr, _value_len, as Coleen suggested >> - constified DCmdInfo too - since this is an immutable value holder, >> all members can be const. >> - also in DCmdInfo, changed JavaPermission permission() to const >> JavaPermission& permission() to save a bit unnecessary copying. >> - Fixed initialization of DCmdArgumentInfo::_position as Serguei and >> Chris requested >> - Removed default implementation of DCmd::name and DCmd::description. >> There is no good reason to provide one - all child classes should (and >> do) provide an implementation. I rather have linkage errors if someone >> forgets to add one. >> - Removed default implementations for DCmdWithParser::name/description >> - same reasoning as above >> - Removed DCmdWithParser::permission(), it is already provided by the >> base class. >> - Removed DCmdWithParser::num_arguments() since childs of >> DCmdWithParser will always have arguments and hence should implement >> this function. >> - Made all functions in DCmdFactoryImpl non-virtual. There is no point >> in allowing to override them. >> >> -- >> >> Notes: >> >> I see the following commands do not implement "permission" and hence >> rely on the default impl of DCmd::permission, which returns a >> permission structure { NULL, NULL, NULL } - is that okay and intended? >> >> JMXStopRemoteDCmd >> JMXStartLocalDCmd >> JMXStartRemoteDCmd >> TouchedMethodsDCmd >> ClassStatsDCmd >> RunFinalizationDCmd >> SystemGCDCmd >> VMUptimeDCmd >> HelpDCmd >> >> I am not sure where and how these permissions are tested. >> >> Thinking through this further: >> >> Of the 47 child classes of DCmd, 37 seem to return >> "java.lang.management.ManagementPermission", name "monitor". So, we >> have this function 37 times: >> >> static const JavaPermission permission() { >> JavaPermission p = {"java.lang.management.ManagementPermission", >> "monitor", NULL}; >> return p; >> } >> >> Would it then not make sense to make this the default implementation >> for DCmd::permission() - instead of the NULL permission - and only >> require child DCmds to provide a permission if it differs from this >> default? We could then loose all 37 methods. >> >> --- >> >> Similar thoughts for "static DCmd impact()" >> >> which is not provided by: >> >> JMXStatusDCmd >> JMXStopRemoteDCmd >> JMXStartLocalDCmd >> >> which thus return the default impact - DCmd::impact() - which returns "Low". >> >> Here I think we should either remove the default implementation and >> force all commands to provide an implementation or stay with the >> default "Low" implementation and remove all 20+ functions in child >> classes which also return "Low". >> >> -- >> >> Finally, some thoughts for possible future cleanups: >> >> GenDCmdArgument internally uses C-heap to hold copies of strings >> (DCmdArgument and DCmdArgument). I wonder >> whether we could use resource area instead. Then we could get rid of >> all cleanup code (GenDCmdArgument::cleanup(), >> GenDCmdArgument::destroy_value()) etc. And if one agrees that parsing >> is only done once and the commands are not reused (which is the case >> currently) we also may get rid of DCmdArgument::reset() and friends. >> Coding would become a lot easier. >> >> -- >> >> IMHO the way object- und template based polymorphy is mixed is a bit >> of a mismatch here and it complicates things a bit. >> >> For instance, Dcmd has above mentioned static functions which return >> immutable attributes for the commands. It is "overwritten" >> template-style by redefining the same static functions in derived >> classes and called via DCmdFactoryImpl. >> >> Maybe it is just me, but I find this a bit roundabout for the problem >> it tries to solve. Also, it is really difficult to parse for modern >> IDEs - e.g. my CDT is unable to tell me who calls "DCmd::permission()" >> since the relation between DCmd::permission() and XXXCmd::permission() >> is just by name. Also, why pay for the text size of 47 just slightly >> different template definitions of DCmdFactory where one class >> would be enough? >> >> The mis-fit between templates and object polymorphism also shows up as >> weird artifacts. E.g. we have 27 variants of this function: >> >> int XXXXDCmd::num_arguments() { >> ResourceMark rm; >> XXXDCmd* dcmd = new SetVMFlagDCmd(NULL, false); >> if (dcmd != NULL) { >> DCmdMark mark(dcmd); >> return dcmd->_dcmdparser.num_arguments(); >> } else { >> return 0; >> } >> } >> >> - so, we want to implement "static XXXDCmd::num_arguments", which - >> being static - has to create an object of the same type to ask it its >> number of arguments. >> >> Maybe we could un-templify the command classes at some point to >> simplify all this? >> >> I think we need two things, a class to hold meta information about the >> command - name, description, argument types etc - which needs to be >> available at startup. And then, we need a second thing to hold runtime >> context: the concrete argument values the user gave us, the >> outputStream* pointer, the execute() function, maybe parser state... >> >> I think this may be expressed simpler and more efficiently with simple >> C++ objects and inheritance, without templates. We could also shrink >> the code quite a bit. And my CDT would find who calls "permissions()" >> :) >> >> >> Best Regards, Thomas >> >> ------- >> >> [1] >> http://mail.openjdk.java.net/pipermail/serviceability-dev/2018-May/023673.html >> >> >> On Wed, Jun 13, 2018 at 3:41 PM, Thomas St?fe >> wrote: >> >> Hi all, >> >> while working on jcmd I saw some minor cleanup possibilities. >> >> Webrev: >> http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.00/webrev/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8204958 >> >> Most of them are const correctness and using initializer lists instead >> of java-style member initialization in these C++ classes. >> >> Other changes: >> >> - Added a ResourceMark into the DCmd::parse_and_execute() loop >> - removed DCmdFactory::create_global_DCmd() (and following that, >> DCmdFactory::create_Cheap_instance()) since I did not find anyone >> using that function. >> - Also in DCmdFactory, I removed a number of setters which were never >> called and for attributes which should be immutable: hidden, flags, >> enabled. >> >> This is really a rather light cleanup. It did not touch any larger issues. >> >> Thank you, >> >> Thomas >> >> >> >> >> From hohensee at amazon.com Fri Jun 15 20:21:59 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Fri, 15 Jun 2018 20:21:59 +0000 Subject: RFR (S): 8195115: G1 Old Gen MemoryPool CollectionUsage.used values don't reflect mixed GC results Message-ID: After some difficulty with the submit cluster, with which Erik helped me out, the patch passes. It also passed fastdebug hotspot tier 1 testing on my Mac laptop, which former includes the new test. I had to increase -Xmx and -Xms to 12m in order to get TestOldGenCollectionUsage to pass on the submit cluster, though the old 10m works fine on my Mac. New webrev: http://cr.openjdk.java.net/~phh/8195115/webrev.03/ Thanks, Paul ?On 6/12/18, 6:52 AM, "Erik Helin" wrote: (adding back serviceability-dev, please keep both hotspot-gc-dev and serviceability-dev) Hi Paul, before I start re-reviewing, did you test the new version of the patch via the jdk/submit repository [0]? Thanks, Erik [0]: http://hg.openjdk.java.net/jdk/submit On 06/09/2018 03:29 PM, Hohensee, Paul wrote: > Didn't seem to make it to hotspot-gc-dev... > > On 6/8/18, 10:14 AM, "serviceability-dev on behalf of Hohensee, Paul" wrote: > > Back after a long hiatus... > > Thanks, Eric, for your review. Here's a new webrev incorporating your recommendations. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8195115 > Webrev: http://cr.openjdk.java.net/~phh/8195115/webrev.02/ > > TIA for your re-review. Plus, may I have another reviewer look at it please? > > Paul > > On 2/26/18, 8:47 AM, "Erik Helin" wrote: > > Hi Paul, > > a couple of comments on the patch: > > - memoryService.hpp: > + 150 bool countCollection, > + 151 bool allMemoryPoolsAffected = true); > > There is no need to use a default value for the parameter > allMemoryPoolsAffected here. Skipping the default value also allows > you to put allMemoryPoolsAffected to TraceMemoryManager::initialize > in the same relative position as for the constructor parameter (this > will make the code more uniform and easier to follow). > > - memoryManager.cpp > > Instead of adding a default parameter, maybe add a new method? > Something like GCMemoryManager::add_not_always_affected_pool() > (I couldn't come up with a shorter name at the moment). > > - TestMixedOldGenCollectionUsage.java > > The test is too strict about how and when collections should > occur. Tests written this way often become very brittle, they might > e.g. fail to finish a concurrent mark on time on a very slow, single > core, machine. It is better to either force collections by using the > WhiteBox API or make the test more lenient. > > Thanks, > Erik > > On 02/22/2018 09:54 PM, Hohensee, Paul wrote: > > Ping for a review please. > > > > Thanks, > > > > Paul > > > > On 2/16/18, 12:26 PM, "serviceability-dev on behalf of Hohensee, Paul" wrote: > > > > The CSR https://bugs.openjdk.java.net/browse/JDK-8196719 for the original fix has been approved, so I?m back to requesting a code review, please. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8195115 > > Webrev: http://cr.openjdk.java.net/~phh/8195115/webrev.hs.01/ > > > > Passed a submit repo run, passes its jtreg test, and a JDK8 version is in production use at Amazon. > > > > From the original RR: > > > > > The bug is that from the JMX point of view, G1?s incremental collector > > > (misnamed as the ?G1 Young Generation? collector) only affects G1?s > > > survivor and eden spaces. In fact, mixed collections run by this > > > collector also affect the G1 old generation. > > > > > > This proposed fix is to record, for each of a JMX garbage collector's > > > memory pools, whether that memory pool is affected by all collections > > > using that collector. And, for each collection, record whether or not > > > all the collector's memory pools are affected. After each collection, > > > for each memory pool, if either all the collector's memory pools were > > > affected or the memory pool is affected for all collections, record > > > CollectionUsage for that pool. > > > > > > For collectors other than G1 Young Generation, all pools are recorded as > > > affected by all collections and every collection is recorded as > > > affecting all the collector?s memory pools. For the G1 Young Generation > > > collector, the G1 Old Gen pool is recorded as not being affected by all > > > collections, and non-mixed collections are recorded as not affecting all > > > memory pools. The result is that for non-mixed collections, > > > CollectionUsage is recorded after a collection only the G1 Eden Space > > > and G1 Survivor Space pools, while for mixed collections CollectionUsage > > > is recorded for G1 Old Gen as well. > > > > > > Other than the effect of the fix on G1 Old Gen MemoryPool. > > > CollectionUsage, the only external behavior change is that > > > GarbageCollectorMXBean.getMemoryPoolNames will now return 3 pool names > > > rather than 2. > > > > > > With this fix, a collector?s memory pools can be divided into two > > > disjoint subsets, one that participates in all collections and one that > > > doesn?t. This is a bit more general than the minimum necessary to fix > > > G1, but not by much. Because I expect it to apply to other incremental > > > region-based collectors, I went with the more general solution. I > > > minimized the amount of code I had to touch by using default parameters > > > for GCMemoryManager::add_pool and the TraceMemoryManagerStats constructors. > > > > > > > > > > From thomas.stuefe at gmail.com Fri Jun 15 20:52:48 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 15 Jun 2018 22:52:48 +0200 Subject: RFR(s): 8204958: Minor cleanups for the diagnostic framework In-Reply-To: <58051f76-31b9-658c-cfb3-7f10ac7d00c7@oracle.com> References: <58051f76-31b9-658c-cfb3-7f10ac7d00c7@oracle.com> Message-ID: On Fri, Jun 15, 2018 at 10:36 PM, Chris Plummer wrote: > On 6/15/18 10:31 AM, Thomas St?fe wrote: >> >> Coleen, Chris, >> >> thanks for the review. Please find the latest webrev: >> >> Delta: >> http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.01-to-02/webrev/index.html >> Full: >> http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.02/webrev/ >> >> I changed the copyright dates, the format of the initializer lists (I >> looked around and used what the most common denominator seemed to be - >> still, Coleen, should you ever dig up that style guide of yours, that >> would be valuable), and initialized CmdLine::_cmd to line as Coleen >> sugggested. >> >> @Chris >>> >>> I'm not certain why you added default field initializers for >>> CmdLine::CmdLine when each field is explicitly initialized within the method >>> body. >> >> See Coleens response. I usually prefer it that way if the constructor >> is not extremely obvious, to prevent missing initializations should >> the constructor body evolve. C++ compiler should optimize that away >> anyway. > > I tend to think the opposite. Using the initializer list in this way would > lead me to think that the field was not initialized in the constructor body, > or at the very least there was a path where it might not be initialized. I > don't see how this helps when the constructor is not obvious. Is the point > of the initializer (in this case) solely to have the code document which > fields get initialized in the constructor? > For me, it is a safety valve against missing initialization. Consider this: class Parser { const char* p; .. }; Parser(some input) { // complicated stuff happening, p gets initialized in all paths. Ok. } evolves to this: Parser(some input) { // more complicated stuff, on one path p may not get initialized. } if p does not get initialized, its value is essentially random, at least in relase builds. Especially if Parser is placed on the stack, since p then is mapped to whatever happened to be at this stack slot before. So, by always initializing it to NULL: Parser(some input) : p(NULL) { // complicated stuff happening, p gets initialized in all paths } from the beginning, I get reproducable errors. Using p == NULL will crash, using p == random-and-maybe-readable-pointer may do random things or even work out of accident. Thanks, Thomas > thanks, > > Chris > >> Thanks for the reviews, >> >> Best Regards, Thomas >> >> >> On Fri, Jun 15, 2018 at 1:53 PM, wrote: >>> >>> >>> On 6/15/18 1:04 AM, Chris Plummer wrote: >>> >>> Hi Thomas, >>> >>> I'm not certain why you added default field initializers for >>> CmdLine::CmdLine when each field is explicitly initialized within the >>> method >>> body. >>> >>> >>> I think the C++ optimizer would elide any unnecessary initializations and >>> it >>> seems safe in case new fields are added, although >>> >>> 38 : _cmd(NULL) >>> >>> >>> could be initialized as _cmd(line). >>> >>> Can I bikeshed and say I don't like the style where the , are at the >>> beginning of the lines? We don't use this style in other places, that >>> I've >>> noticed, and I really don't like reading the punctuation at the beginning >>> of >>> the line. If I could find our open source wiki for the coding style >>> among >>> the broken openjdk wiki pages, I would see if the punctuation rule is >>> there. >>> One initializer per line is ok. >>> >>> Thanks, >>> Coleen >>> >>> >>> A number of copyrights need updating. >>> >>> The rest of the changes look good. >>> >>> thanks, >>> >>> Chris >>> >>> On 6/14/18 1:38 AM, Thomas St?fe wrote: >>> >>> Hi all, >>> >>> (I moved the issue to target release 12 - if we get this through the >>> review quick, fine, but no reason to add pressure) >>> >>> some more background: The reason for these cleanups is that I later >>> plan to add a new sub command which allows us to run multiple commands >>> in one safepoint (see [1] for more details and discussions). Since >>> this will require tweaking the parse process a bit, I try to loosen up >>> the coding before adding complexity again. >>> >>> Please find here the new webrev: >>> >>> Delta: >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.00-to-01/webrev/ >>> Full: >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.01/webrev/ >>> >>> What changed: >>> >>> Aesthetics: >>> - I reordered member init lists to member order, as Coleen did ask. I >>> also reform them as lists, just to increase readability. >>> - For a number of inline methods, I aligned their implementations so >>> that they too would be better readable and it would be clearer which >>> methods are const >>> - Moved DCmdFactory::_DCmdFactoryList up to the other static members. >>> >>> Functional: >>> - Added initializer list for CmdLine >>> - Added missing initializations for DCmdArgIter members _key_addr, >>> _key_len, _value_addr, _value_len, as Coleen suggested >>> - constified DCmdInfo too - since this is an immutable value holder, >>> all members can be const. >>> - also in DCmdInfo, changed JavaPermission permission() to const >>> JavaPermission& permission() to save a bit unnecessary copying. >>> - Fixed initialization of DCmdArgumentInfo::_position as Serguei and >>> Chris requested >>> - Removed default implementation of DCmd::name and DCmd::description. >>> There is no good reason to provide one - all child classes should (and >>> do) provide an implementation. I rather have linkage errors if someone >>> forgets to add one. >>> - Removed default implementations for DCmdWithParser::name/description >>> - same reasoning as above >>> - Removed DCmdWithParser::permission(), it is already provided by the >>> base class. >>> - Removed DCmdWithParser::num_arguments() since childs of >>> DCmdWithParser will always have arguments and hence should implement >>> this function. >>> - Made all functions in DCmdFactoryImpl non-virtual. There is no point >>> in allowing to override them. >>> >>> -- >>> >>> Notes: >>> >>> I see the following commands do not implement "permission" and hence >>> rely on the default impl of DCmd::permission, which returns a >>> permission structure { NULL, NULL, NULL } - is that okay and intended? >>> >>> JMXStopRemoteDCmd >>> JMXStartLocalDCmd >>> JMXStartRemoteDCmd >>> TouchedMethodsDCmd >>> ClassStatsDCmd >>> RunFinalizationDCmd >>> SystemGCDCmd >>> VMUptimeDCmd >>> HelpDCmd >>> >>> I am not sure where and how these permissions are tested. >>> >>> Thinking through this further: >>> >>> Of the 47 child classes of DCmd, 37 seem to return >>> "java.lang.management.ManagementPermission", name "monitor". So, we >>> have this function 37 times: >>> >>> static const JavaPermission permission() { >>> JavaPermission p = {"java.lang.management.ManagementPermission", >>> "monitor", NULL}; >>> return p; >>> } >>> >>> Would it then not make sense to make this the default implementation >>> for DCmd::permission() - instead of the NULL permission - and only >>> require child DCmds to provide a permission if it differs from this >>> default? We could then loose all 37 methods. >>> >>> --- >>> >>> Similar thoughts for "static DCmd impact()" >>> >>> which is not provided by: >>> >>> JMXStatusDCmd >>> JMXStopRemoteDCmd >>> JMXStartLocalDCmd >>> >>> which thus return the default impact - DCmd::impact() - which returns >>> "Low". >>> >>> Here I think we should either remove the default implementation and >>> force all commands to provide an implementation or stay with the >>> default "Low" implementation and remove all 20+ functions in child >>> classes which also return "Low". >>> >>> -- >>> >>> Finally, some thoughts for possible future cleanups: >>> >>> GenDCmdArgument internally uses C-heap to hold copies of strings >>> (DCmdArgument and DCmdArgument). I wonder >>> whether we could use resource area instead. Then we could get rid of >>> all cleanup code (GenDCmdArgument::cleanup(), >>> GenDCmdArgument::destroy_value()) etc. And if one agrees that parsing >>> is only done once and the commands are not reused (which is the case >>> currently) we also may get rid of DCmdArgument::reset() and friends. >>> Coding would become a lot easier. >>> >>> -- >>> >>> IMHO the way object- und template based polymorphy is mixed is a bit >>> of a mismatch here and it complicates things a bit. >>> >>> For instance, Dcmd has above mentioned static functions which return >>> immutable attributes for the commands. It is "overwritten" >>> template-style by redefining the same static functions in derived >>> classes and called via DCmdFactoryImpl. >>> >>> Maybe it is just me, but I find this a bit roundabout for the problem >>> it tries to solve. Also, it is really difficult to parse for modern >>> IDEs - e.g. my CDT is unable to tell me who calls "DCmd::permission()" >>> since the relation between DCmd::permission() and XXXCmd::permission() >>> is just by name. Also, why pay for the text size of 47 just slightly >>> different template definitions of DCmdFactory where one class >>> would be enough? >>> >>> The mis-fit between templates and object polymorphism also shows up as >>> weird artifacts. E.g. we have 27 variants of this function: >>> >>> int XXXXDCmd::num_arguments() { >>> ResourceMark rm; >>> XXXDCmd* dcmd = new SetVMFlagDCmd(NULL, false); >>> if (dcmd != NULL) { >>> DCmdMark mark(dcmd); >>> return dcmd->_dcmdparser.num_arguments(); >>> } else { >>> return 0; >>> } >>> } >>> >>> - so, we want to implement "static XXXDCmd::num_arguments", which - >>> being static - has to create an object of the same type to ask it its >>> number of arguments. >>> >>> Maybe we could un-templify the command classes at some point to >>> simplify all this? >>> >>> I think we need two things, a class to hold meta information about the >>> command - name, description, argument types etc - which needs to be >>> available at startup. And then, we need a second thing to hold runtime >>> context: the concrete argument values the user gave us, the >>> outputStream* pointer, the execute() function, maybe parser state... >>> >>> I think this may be expressed simpler and more efficiently with simple >>> C++ objects and inheritance, without templates. We could also shrink >>> the code quite a bit. And my CDT would find who calls "permissions()" >>> :) >>> >>> >>> Best Regards, Thomas >>> >>> ------- >>> >>> [1] >>> >>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2018-May/023673.html >>> >>> >>> On Wed, Jun 13, 2018 at 3:41 PM, Thomas St?fe >>> wrote: >>> >>> Hi all, >>> >>> while working on jcmd I saw some minor cleanup possibilities. >>> >>> Webrev: >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.00/webrev/ >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8204958 >>> >>> Most of them are const correctness and using initializer lists instead >>> of java-style member initialization in these C++ classes. >>> >>> Other changes: >>> >>> - Added a ResourceMark into the DCmd::parse_and_execute() loop >>> - removed DCmdFactory::create_global_DCmd() (and following that, >>> DCmdFactory::create_Cheap_instance()) since I did not find anyone >>> using that function. >>> - Also in DCmdFactory, I removed a number of setters which were never >>> called and for attributes which should be immutable: hidden, flags, >>> enabled. >>> >>> This is really a rather light cleanup. It did not touch any larger >>> issues. >>> >>> Thank you, >>> >>> Thomas >>> >>> >>> >>> >>> > > From coleen.phillimore at oracle.com Fri Jun 15 22:07:17 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 15 Jun 2018 18:07:17 -0400 Subject: RFR(s): 8204958: Minor cleanups for the diagnostic framework In-Reply-To: References: <58051f76-31b9-658c-cfb3-7f10ac7d00c7@oracle.com> Message-ID: <6b514a32-c071-e6b8-6c87-f02bc172d05a@oracle.com> On 6/15/18 4:52 PM, Thomas St?fe wrote: > On Fri, Jun 15, 2018 at 10:36 PM, Chris Plummer > wrote: >> On 6/15/18 10:31 AM, Thomas St?fe wrote: >>> Coleen, Chris, >>> >>> thanks for the review. Please find the latest webrev: >>> >>> Delta: >>> http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.01-to-02/webrev/index.html >>> Full: >>> http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.02/webrev/ >>> >>> I changed the copyright dates, the format of the initializer lists (I >>> looked around and used what the most common denominator seemed to be - >>> still, Coleen, should you ever dig up that style guide of yours, that >>> would be valuable), and initialized CmdLine::_cmd to line as Coleen >>> sugggested. >>> >>> @Chris >>>> I'm not certain why you added default field initializers for >>>> CmdLine::CmdLine when each field is explicitly initialized within the method >>>> body. >>> See Coleens response. I usually prefer it that way if the constructor >>> is not extremely obvious, to prevent missing initializations should >>> the constructor body evolve. C++ compiler should optimize that away >>> anyway. >> I tend to think the opposite. Using the initializer list in this way would >> lead me to think that the field was not initialized in the constructor body, >> or at the very least there was a path where it might not be initialized. I >> don't see how this helps when the constructor is not obvious. Is the point >> of the initializer (in this case) solely to have the code document which >> fields get initialized in the constructor? >> > For me, it is a safety valve against missing initialization. Consider this: > > class Parser { const char* p; .. }; > > Parser(some input) { > // complicated stuff happening, p gets initialized in all paths. Ok. > } > > evolves to this: > > Parser(some input) { > // more complicated stuff, on one path p may not get initialized. > } > > if p does not get initialized, its value is essentially random, at > least in relase builds. Especially if Parser is placed on the stack, > since p then is mapped to whatever happened to be at this stack slot > before. > > So, by always initializing it to NULL: > > Parser(some input) : p(NULL) { > // complicated stuff happening, p gets initialized in all paths > } > > from the beginning, I get reproducable errors. Using p == NULL will > crash, using p == random-and-maybe-readable-pointer may do random > things or even work out of accident. > > Thanks, Thomas > I agree with Thomas and I think the change looks good.? Thank you for humoring me with the punctuation.? It doesn't seem to be here. https://wiki.openjdk.java.net/display/HotSpot/StyleGuide Thanks, Coleen > > > > > > >> thanks, >> >> Chris >> >>> Thanks for the reviews, >>> >>> Best Regards, Thomas >>> >>> >>> On Fri, Jun 15, 2018 at 1:53 PM, wrote: >>>> >>>> On 6/15/18 1:04 AM, Chris Plummer wrote: >>>> >>>> Hi Thomas, >>>> >>>> I'm not certain why you added default field initializers for >>>> CmdLine::CmdLine when each field is explicitly initialized within the >>>> method >>>> body. >>>> >>>> >>>> I think the C++ optimizer would elide any unnecessary initializations and >>>> it >>>> seems safe in case new fields are added, although >>>> >>>> 38 : _cmd(NULL) >>>> >>>> >>>> could be initialized as _cmd(line). >>>> >>>> Can I bikeshed and say I don't like the style where the , are at the >>>> beginning of the lines? We don't use this style in other places, that >>>> I've >>>> noticed, and I really don't like reading the punctuation at the beginning >>>> of >>>> the line. If I could find our open source wiki for the coding style >>>> among >>>> the broken openjdk wiki pages, I would see if the punctuation rule is >>>> there. >>>> One initializer per line is ok. >>>> >>>> Thanks, >>>> Coleen >>>> >>>> >>>> A number of copyrights need updating. >>>> >>>> The rest of the changes look good. >>>> >>>> thanks, >>>> >>>> Chris >>>> >>>> On 6/14/18 1:38 AM, Thomas St?fe wrote: >>>> >>>> Hi all, >>>> >>>> (I moved the issue to target release 12 - if we get this through the >>>> review quick, fine, but no reason to add pressure) >>>> >>>> some more background: The reason for these cleanups is that I later >>>> plan to add a new sub command which allows us to run multiple commands >>>> in one safepoint (see [1] for more details and discussions). Since >>>> this will require tweaking the parse process a bit, I try to loosen up >>>> the coding before adding complexity again. >>>> >>>> Please find here the new webrev: >>>> >>>> Delta: >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.00-to-01/webrev/ >>>> Full: >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.01/webrev/ >>>> >>>> What changed: >>>> >>>> Aesthetics: >>>> - I reordered member init lists to member order, as Coleen did ask. I >>>> also reform them as lists, just to increase readability. >>>> - For a number of inline methods, I aligned their implementations so >>>> that they too would be better readable and it would be clearer which >>>> methods are const >>>> - Moved DCmdFactory::_DCmdFactoryList up to the other static members. >>>> >>>> Functional: >>>> - Added initializer list for CmdLine >>>> - Added missing initializations for DCmdArgIter members _key_addr, >>>> _key_len, _value_addr, _value_len, as Coleen suggested >>>> - constified DCmdInfo too - since this is an immutable value holder, >>>> all members can be const. >>>> - also in DCmdInfo, changed JavaPermission permission() to const >>>> JavaPermission& permission() to save a bit unnecessary copying. >>>> - Fixed initialization of DCmdArgumentInfo::_position as Serguei and >>>> Chris requested >>>> - Removed default implementation of DCmd::name and DCmd::description. >>>> There is no good reason to provide one - all child classes should (and >>>> do) provide an implementation. I rather have linkage errors if someone >>>> forgets to add one. >>>> - Removed default implementations for DCmdWithParser::name/description >>>> - same reasoning as above >>>> - Removed DCmdWithParser::permission(), it is already provided by the >>>> base class. >>>> - Removed DCmdWithParser::num_arguments() since childs of >>>> DCmdWithParser will always have arguments and hence should implement >>>> this function. >>>> - Made all functions in DCmdFactoryImpl non-virtual. There is no point >>>> in allowing to override them. >>>> >>>> -- >>>> >>>> Notes: >>>> >>>> I see the following commands do not implement "permission" and hence >>>> rely on the default impl of DCmd::permission, which returns a >>>> permission structure { NULL, NULL, NULL } - is that okay and intended? >>>> >>>> JMXStopRemoteDCmd >>>> JMXStartLocalDCmd >>>> JMXStartRemoteDCmd >>>> TouchedMethodsDCmd >>>> ClassStatsDCmd >>>> RunFinalizationDCmd >>>> SystemGCDCmd >>>> VMUptimeDCmd >>>> HelpDCmd >>>> >>>> I am not sure where and how these permissions are tested. >>>> >>>> Thinking through this further: >>>> >>>> Of the 47 child classes of DCmd, 37 seem to return >>>> "java.lang.management.ManagementPermission", name "monitor". So, we >>>> have this function 37 times: >>>> >>>> static const JavaPermission permission() { >>>> JavaPermission p = {"java.lang.management.ManagementPermission", >>>> "monitor", NULL}; >>>> return p; >>>> } >>>> >>>> Would it then not make sense to make this the default implementation >>>> for DCmd::permission() - instead of the NULL permission - and only >>>> require child DCmds to provide a permission if it differs from this >>>> default? We could then loose all 37 methods. >>>> >>>> --- >>>> >>>> Similar thoughts for "static DCmd impact()" >>>> >>>> which is not provided by: >>>> >>>> JMXStatusDCmd >>>> JMXStopRemoteDCmd >>>> JMXStartLocalDCmd >>>> >>>> which thus return the default impact - DCmd::impact() - which returns >>>> "Low". >>>> >>>> Here I think we should either remove the default implementation and >>>> force all commands to provide an implementation or stay with the >>>> default "Low" implementation and remove all 20+ functions in child >>>> classes which also return "Low". >>>> >>>> -- >>>> >>>> Finally, some thoughts for possible future cleanups: >>>> >>>> GenDCmdArgument internally uses C-heap to hold copies of strings >>>> (DCmdArgument and DCmdArgument). I wonder >>>> whether we could use resource area instead. Then we could get rid of >>>> all cleanup code (GenDCmdArgument::cleanup(), >>>> GenDCmdArgument::destroy_value()) etc. And if one agrees that parsing >>>> is only done once and the commands are not reused (which is the case >>>> currently) we also may get rid of DCmdArgument::reset() and friends. >>>> Coding would become a lot easier. >>>> >>>> -- >>>> >>>> IMHO the way object- und template based polymorphy is mixed is a bit >>>> of a mismatch here and it complicates things a bit. >>>> >>>> For instance, Dcmd has above mentioned static functions which return >>>> immutable attributes for the commands. It is "overwritten" >>>> template-style by redefining the same static functions in derived >>>> classes and called via DCmdFactoryImpl. >>>> >>>> Maybe it is just me, but I find this a bit roundabout for the problem >>>> it tries to solve. Also, it is really difficult to parse for modern >>>> IDEs - e.g. my CDT is unable to tell me who calls "DCmd::permission()" >>>> since the relation between DCmd::permission() and XXXCmd::permission() >>>> is just by name. Also, why pay for the text size of 47 just slightly >>>> different template definitions of DCmdFactory where one class >>>> would be enough? >>>> >>>> The mis-fit between templates and object polymorphism also shows up as >>>> weird artifacts. E.g. we have 27 variants of this function: >>>> >>>> int XXXXDCmd::num_arguments() { >>>> ResourceMark rm; >>>> XXXDCmd* dcmd = new SetVMFlagDCmd(NULL, false); >>>> if (dcmd != NULL) { >>>> DCmdMark mark(dcmd); >>>> return dcmd->_dcmdparser.num_arguments(); >>>> } else { >>>> return 0; >>>> } >>>> } >>>> >>>> - so, we want to implement "static XXXDCmd::num_arguments", which - >>>> being static - has to create an object of the same type to ask it its >>>> number of arguments. >>>> >>>> Maybe we could un-templify the command classes at some point to >>>> simplify all this? >>>> >>>> I think we need two things, a class to hold meta information about the >>>> command - name, description, argument types etc - which needs to be >>>> available at startup. And then, we need a second thing to hold runtime >>>> context: the concrete argument values the user gave us, the >>>> outputStream* pointer, the execute() function, maybe parser state... >>>> >>>> I think this may be expressed simpler and more efficiently with simple >>>> C++ objects and inheritance, without templates. We could also shrink >>>> the code quite a bit. And my CDT would find who calls "permissions()" >>>> :) >>>> >>>> >>>> Best Regards, Thomas >>>> >>>> ------- >>>> >>>> [1] >>>> >>>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2018-May/023673.html >>>> >>>> >>>> On Wed, Jun 13, 2018 at 3:41 PM, Thomas St?fe >>>> wrote: >>>> >>>> Hi all, >>>> >>>> while working on jcmd I saw some minor cleanup possibilities. >>>> >>>> Webrev: >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.00/webrev/ >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8204958 >>>> >>>> Most of them are const correctness and using initializer lists instead >>>> of java-style member initialization in these C++ classes. >>>> >>>> Other changes: >>>> >>>> - Added a ResourceMark into the DCmd::parse_and_execute() loop >>>> - removed DCmdFactory::create_global_DCmd() (and following that, >>>> DCmdFactory::create_Cheap_instance()) since I did not find anyone >>>> using that function. >>>> - Also in DCmdFactory, I removed a number of setters which were never >>>> called and for attributes which should be immutable: hidden, flags, >>>> enabled. >>>> >>>> This is really a rather light cleanup. It did not touch any larger >>>> issues. >>>> >>>> Thank you, >>>> >>>> Thomas >>>> >>>> >>>> >>>> >>>> >> From goetz.lindenmaier at sap.com Sat Jun 16 07:24:33 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Sat, 16 Jun 2018 07:24:33 +0000 Subject: RFR(XS): 8205108: [testbug] Fix pattern matching in jstatd tests. Message-ID: <4d7ec5ef1e224177af8409e9845e71bb@sap.com> Hi, Please review this test fix: http://cr.openjdk.java.net/~goetz/wr18/8205108-fixJstatDTestPattern/01/ The values for 'M' must be parsed as PERCENTAGE_OR_DASH. The parsing of numbers should not depend on the locale. I changed the parsing to use java.text.NumberFormat Best regards, Goetz. From erik.helin at oracle.com Sat Jun 16 08:23:59 2018 From: erik.helin at oracle.com (Erik Helin) Date: Sat, 16 Jun 2018 10:23:59 +0200 Subject: RFR (S): 8195115: G1 Old Gen MemoryPool CollectionUsage.used values don't reflect mixed GC results In-Reply-To: References: Message-ID: <958dd448-efde-2414-9c88-a0f1c869dd56@oracle.com> On 06/15/2018 10:21 PM, Hohensee, Paul wrote: > After some difficulty with the submit cluster, with which Erik helped me out, the patch passes. It also passed fastdebug hotspot tier 1 testing on my Mac laptop, which former includes the new test. > > I had to increase -Xmx and -Xms to 12m in order to get TestOldGenCollectionUsage to pass on the submit cluster, though the old 10m works fine on my Mac. New webrev: Thanks, the change of -Xmx and -Xms to 12m now also makes the test pass on my workstation. > http://cr.openjdk.java.net/~phh/8195115/webrev.03/ There seems to be some trailing whitespace in the patch, have you run jcheck (or `hg diff` which highlights trailing whitespace in red)? Please see + TraceMemoryManagerStats tms(&_memory_manager, gc_cause(), + collector_state()->yc_type() == Mixed /* allMemoryPoolsAffected */); + ^---- whitespace and +int MemoryManager::add_pool(MemoryPool* pool) { + int index = _num_pools; ^---- whitespace Another small comment, I would have written +void GCMemoryManager::add_pool(MemoryPool* pool) { + int index = MemoryManager::add_pool(pool); + _pool_always_affected_by_gc[index] = true; +} + +void GCMemoryManager::add_pool(MemoryPool* pool, bool always_affected_by_gc) { + int index = MemoryManager::add_pool(pool); + _pool_always_affected_by_gc[index] = always_affected_by_gc; +} + as +void GCMemoryManager::add_pool(MemoryPool* pool) { + add_pool(pool, true); +} + +void GCMemoryManager::add_pool(MemoryPool* pool, bool always_affected_by_gc) { + int index = MemoryManager::add_pool(pool); + _pool_always_affected_by_gc[index] = always_affected_by_gc; +} + to not have to two duplicate implementations of GCMemoryManager::add_pool. Would you mind updating the patch with this change (and remove the trailing whitespace)? Thanks, Erik > Thanks, > > Paul > > ?On 6/12/18, 6:52 AM, "Erik Helin" wrote: > > (adding back serviceability-dev, please keep both hotspot-gc-dev and > serviceability-dev) > > Hi Paul, > > before I start re-reviewing, did you test the new version of the patch > via the jdk/submit repository [0]? > > Thanks, > Erik > > [0]: http://hg.openjdk.java.net/jdk/submit > > On 06/09/2018 03:29 PM, Hohensee, Paul wrote: > > Didn't seem to make it to hotspot-gc-dev... > > > > On 6/8/18, 10:14 AM, "serviceability-dev on behalf of Hohensee, Paul" wrote: > > > > Back after a long hiatus... > > > > Thanks, Eric, for your review. Here's a new webrev incorporating your recommendations. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8195115 > > Webrev: http://cr.openjdk.java.net/~phh/8195115/webrev.02/ > > > > TIA for your re-review. Plus, may I have another reviewer look at it please? > > > > Paul > > > > On 2/26/18, 8:47 AM, "Erik Helin" wrote: > > > > Hi Paul, > > > > a couple of comments on the patch: > > > > - memoryService.hpp: > > + 150 bool countCollection, > > + 151 bool allMemoryPoolsAffected = true); > > > > There is no need to use a default value for the parameter > > allMemoryPoolsAffected here. Skipping the default value also allows > > you to put allMemoryPoolsAffected to TraceMemoryManager::initialize > > in the same relative position as for the constructor parameter (this > > will make the code more uniform and easier to follow). > > > > - memoryManager.cpp > > > > Instead of adding a default parameter, maybe add a new method? > > Something like GCMemoryManager::add_not_always_affected_pool() > > (I couldn't come up with a shorter name at the moment). > > > > - TestMixedOldGenCollectionUsage.java > > > > The test is too strict about how and when collections should > > occur. Tests written this way often become very brittle, they might > > e.g. fail to finish a concurrent mark on time on a very slow, single > > core, machine. It is better to either force collections by using the > > WhiteBox API or make the test more lenient. > > > > Thanks, > > Erik > > > > On 02/22/2018 09:54 PM, Hohensee, Paul wrote: > > > Ping for a review please. > > > > > > Thanks, > > > > > > Paul > > > > > > On 2/16/18, 12:26 PM, "serviceability-dev on behalf of Hohensee, Paul" wrote: > > > > > > The CSR https://bugs.openjdk.java.net/browse/JDK-8196719 for the original fix has been approved, so I?m back to requesting a code review, please. > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8195115 > > > Webrev: http://cr.openjdk.java.net/~phh/8195115/webrev.hs.01/ > > > > > > Passed a submit repo run, passes its jtreg test, and a JDK8 version is in production use at Amazon. > > > > > > From the original RR: > > > > > > > The bug is that from the JMX point of view, G1?s incremental collector > > > > (misnamed as the ?G1 Young Generation? collector) only affects G1?s > > > > survivor and eden spaces. In fact, mixed collections run by this > > > > collector also affect the G1 old generation. > > > > > > > > This proposed fix is to record, for each of a JMX garbage collector's > > > > memory pools, whether that memory pool is affected by all collections > > > > using that collector. And, for each collection, record whether or not > > > > all the collector's memory pools are affected. After each collection, > > > > for each memory pool, if either all the collector's memory pools were > > > > affected or the memory pool is affected for all collections, record > > > > CollectionUsage for that pool. > > > > > > > > For collectors other than G1 Young Generation, all pools are recorded as > > > > affected by all collections and every collection is recorded as > > > > affecting all the collector?s memory pools. For the G1 Young Generation > > > > collector, the G1 Old Gen pool is recorded as not being affected by all > > > > collections, and non-mixed collections are recorded as not affecting all > > > > memory pools. The result is that for non-mixed collections, > > > > CollectionUsage is recorded after a collection only the G1 Eden Space > > > > and G1 Survivor Space pools, while for mixed collections CollectionUsage > > > > is recorded for G1 Old Gen as well. > > > > > > > > Other than the effect of the fix on G1 Old Gen MemoryPool. > > > > CollectionUsage, the only external behavior change is that > > > > GarbageCollectorMXBean.getMemoryPoolNames will now return 3 pool names > > > > rather than 2. > > > > > > > > With this fix, a collector?s memory pools can be divided into two > > > > disjoint subsets, one that participates in all collections and one that > > > > doesn?t. This is a bit more general than the minimum necessary to fix > > > > G1, but not by much. Because I expect it to apply to other incremental > > > > region-based collectors, I went with the more general solution. I > > > > minimized the amount of code I had to touch by using default parameters > > > > for GCMemoryManager::add_pool and the TraceMemoryManagerStats constructors. > > > > > > > > > > > > > > > > > > > From thomas.stuefe at gmail.com Sat Jun 16 17:23:44 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 16 Jun 2018 19:23:44 +0200 Subject: jstat does not accept program names as vmid? Message-ID: Hi, just found that jstat does not accept program names as VM ids, it just accepts Pids: ./images/jdk/bin/jcmd hello.Application 5073: The following commands are available: Compiler.CodeHeap_Analytics Compiler.codecache ... ./images/jdk/bin/jstat -gcutil hello.Application Malformed VM Identifier: hello.Application ./images/jdk/bin/jstat -gcutil 5073 S0 S1 E O M CCS YGC YGCT FGC FGCT CGC CGCT GCT 0.00 100.00 66.67 2.64 96.55 88.91 5 0.089 0 0.000 4 0.068 0.156 Is that by design? Thanks, Thomas From thomas.stuefe at gmail.com Sat Jun 16 17:25:30 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 16 Jun 2018 19:25:30 +0200 Subject: RFR(XS): 8205108: [testbug] Fix pattern matching in jstatd tests. In-Reply-To: <4d7ec5ef1e224177af8409e9845e71bb@sap.com> References: <4d7ec5ef1e224177af8409e9845e71bb@sap.com> Message-ID: Hi Goetz, Looks reasonable. Could you please fix up the example output above the alternative you added, it misses the columns "CGC" and "CGCT". Thanks, Thomas On Sat, Jun 16, 2018 at 9:24 AM, Lindenmaier, Goetz wrote: > Hi, > > Please review this test fix: > http://cr.openjdk.java.net/~goetz/wr18/8205108-fixJstatDTestPattern/01/ > > The values for 'M' must be parsed as PERCENTAGE_OR_DASH. > The parsing of numbers should not depend on the locale. > I changed the parsing to use java.text.NumberFormat > > Best regards, > Goetz. > From hohensee at amazon.com Sat Jun 16 19:00:15 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Sat, 16 Jun 2018 19:00:15 +0000 Subject: RFR (S): 8195115: G1 Old Gen MemoryPool CollectionUsage.used values don't reflect mixed GC results In-Reply-To: <958dd448-efde-2414-9c88-a0f1c869dd56@oracle.com> References: <958dd448-efde-2414-9c88-a0f1c869dd56@oracle.com> Message-ID: <9F239517-8707-4786-B7E9-7F88CC0D4C6E@amazon.com> Thanks for the re-review, Erik. New webrev with your fixes: http://cr.openjdk.java.net/~phh/8195115/webrev.04/ Need another reviewer, please. Thanks, Paul ?On 6/16/18, 1:25 AM, "Erik Helin" wrote: On 06/15/2018 10:21 PM, Hohensee, Paul wrote: > After some difficulty with the submit cluster, with which Erik helped me out, the patch passes. It also passed fastdebug hotspot tier 1 testing on my Mac laptop, which former includes the new test. > > I had to increase -Xmx and -Xms to 12m in order to get TestOldGenCollectionUsage to pass on the submit cluster, though the old 10m works fine on my Mac. New webrev: Thanks, the change of -Xmx and -Xms to 12m now also makes the test pass on my workstation. > http://cr.openjdk.java.net/~phh/8195115/webrev.03/ There seems to be some trailing whitespace in the patch, have you run jcheck (or `hg diff` which highlights trailing whitespace in red)? Please see + TraceMemoryManagerStats tms(&_memory_manager, gc_cause(), + collector_state()->yc_type() == Mixed /* allMemoryPoolsAffected */); + ^---- whitespace and +int MemoryManager::add_pool(MemoryPool* pool) { + int index = _num_pools; ^---- whitespace Another small comment, I would have written +void GCMemoryManager::add_pool(MemoryPool* pool) { + int index = MemoryManager::add_pool(pool); + _pool_always_affected_by_gc[index] = true; +} + +void GCMemoryManager::add_pool(MemoryPool* pool, bool always_affected_by_gc) { + int index = MemoryManager::add_pool(pool); + _pool_always_affected_by_gc[index] = always_affected_by_gc; +} + as +void GCMemoryManager::add_pool(MemoryPool* pool) { + add_pool(pool, true); +} + +void GCMemoryManager::add_pool(MemoryPool* pool, bool always_affected_by_gc) { + int index = MemoryManager::add_pool(pool); + _pool_always_affected_by_gc[index] = always_affected_by_gc; +} + to not have to two duplicate implementations of GCMemoryManager::add_pool. Would you mind updating the patch with this change (and remove the trailing whitespace)? Thanks, Erik > Thanks, > > Paul > > On 6/12/18, 6:52 AM, "Erik Helin" wrote: > > (adding back serviceability-dev, please keep both hotspot-gc-dev and > serviceability-dev) > > Hi Paul, > > before I start re-reviewing, did you test the new version of the patch > via the jdk/submit repository [0]? > > Thanks, > Erik > > [0]: http://hg.openjdk.java.net/jdk/submit > > On 06/09/2018 03:29 PM, Hohensee, Paul wrote: > > Didn't seem to make it to hotspot-gc-dev... > > > > On 6/8/18, 10:14 AM, "serviceability-dev on behalf of Hohensee, Paul" wrote: > > > > Back after a long hiatus... > > > > Thanks, Eric, for your review. Here's a new webrev incorporating your recommendations. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8195115 > > Webrev: http://cr.openjdk.java.net/~phh/8195115/webrev.02/ > > > > TIA for your re-review. Plus, may I have another reviewer look at it please? > > > > Paul > > > > On 2/26/18, 8:47 AM, "Erik Helin" wrote: > > > > Hi Paul, > > > > a couple of comments on the patch: > > > > - memoryService.hpp: > > + 150 bool countCollection, > > + 151 bool allMemoryPoolsAffected = true); > > > > There is no need to use a default value for the parameter > > allMemoryPoolsAffected here. Skipping the default value also allows > > you to put allMemoryPoolsAffected to TraceMemoryManager::initialize > > in the same relative position as for the constructor parameter (this > > will make the code more uniform and easier to follow). > > > > - memoryManager.cpp > > > > Instead of adding a default parameter, maybe add a new method? > > Something like GCMemoryManager::add_not_always_affected_pool() > > (I couldn't come up with a shorter name at the moment). > > > > - TestMixedOldGenCollectionUsage.java > > > > The test is too strict about how and when collections should > > occur. Tests written this way often become very brittle, they might > > e.g. fail to finish a concurrent mark on time on a very slow, single > > core, machine. It is better to either force collections by using the > > WhiteBox API or make the test more lenient. > > > > Thanks, > > Erik > > > > On 02/22/2018 09:54 PM, Hohensee, Paul wrote: > > > Ping for a review please. > > > > > > Thanks, > > > > > > Paul > > > > > > On 2/16/18, 12:26 PM, "serviceability-dev on behalf of Hohensee, Paul" wrote: > > > > > > The CSR https://bugs.openjdk.java.net/browse/JDK-8196719 for the original fix has been approved, so I?m back to requesting a code review, please. > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8195115 > > > Webrev: http://cr.openjdk.java.net/~phh/8195115/webrev.hs.01/ > > > > > > Passed a submit repo run, passes its jtreg test, and a JDK8 version is in production use at Amazon. > > > > > > From the original RR: > > > > > > > The bug is that from the JMX point of view, G1?s incremental collector > > > > (misnamed as the ?G1 Young Generation? collector) only affects G1?s > > > > survivor and eden spaces. In fact, mixed collections run by this > > > > collector also affect the G1 old generation. > > > > > > > > This proposed fix is to record, for each of a JMX garbage collector's > > > > memory pools, whether that memory pool is affected by all collections > > > > using that collector. And, for each collection, record whether or not > > > > all the collector's memory pools are affected. After each collection, > > > > for each memory pool, if either all the collector's memory pools were > > > > affected or the memory pool is affected for all collections, record > > > > CollectionUsage for that pool. > > > > > > > > For collectors other than G1 Young Generation, all pools are recorded as > > > > affected by all collections and every collection is recorded as > > > > affecting all the collector?s memory pools. For the G1 Young Generation > > > > collector, the G1 Old Gen pool is recorded as not being affected by all > > > > collections, and non-mixed collections are recorded as not affecting all > > > > memory pools. The result is that for non-mixed collections, > > > > CollectionUsage is recorded after a collection only the G1 Eden Space > > > > and G1 Survivor Space pools, while for mixed collections CollectionUsage > > > > is recorded for G1 Old Gen as well. > > > > > > > > Other than the effect of the fix on G1 Old Gen MemoryPool. > > > > CollectionUsage, the only external behavior change is that > > > > GarbageCollectorMXBean.getMemoryPoolNames will now return 3 pool names > > > > rather than 2. > > > > > > > > With this fix, a collector?s memory pools can be divided into two > > > > disjoint subsets, one that participates in all collections and one that > > > > doesn?t. This is a bit more general than the minimum necessary to fix > > > > G1, but not by much. Because I expect it to apply to other incremental > > > > region-based collectors, I went with the more general solution. I > > > > minimized the amount of code I had to touch by using default parameters > > > > for GCMemoryManager::add_pool and the TraceMemoryManagerStats constructors. > > > > > > > > > > > > > > > > > > > From thomas.stuefe at gmail.com Sun Jun 17 05:48:08 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sun, 17 Jun 2018 07:48:08 +0200 Subject: RFR(s): 8204958: Minor cleanups for the diagnostic framework In-Reply-To: <6b514a32-c071-e6b8-6c87-f02bc172d05a@oracle.com> References: <58051f76-31b9-658c-cfb3-7f10ac7d00c7@oracle.com> <6b514a32-c071-e6b8-6c87-f02bc172d05a@oracle.com> Message-ID: Thank you Coleen. I pushed. The document reads a bit outdated in places: Be sparing with templates. I think that ship has sailed :P Best Regards, Thomas On Sat, Jun 16, 2018 at 12:07 AM, wrote: > > > On 6/15/18 4:52 PM, Thomas St?fe wrote: >> >> On Fri, Jun 15, 2018 at 10:36 PM, Chris Plummer >> wrote: >>> >>> On 6/15/18 10:31 AM, Thomas St?fe wrote: >>>> >>>> Coleen, Chris, >>>> >>>> thanks for the review. Please find the latest webrev: >>>> >>>> Delta: >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.01-to-02/webrev/index.html >>>> Full: >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.02/webrev/ >>>> >>>> I changed the copyright dates, the format of the initializer lists (I >>>> looked around and used what the most common denominator seemed to be - >>>> still, Coleen, should you ever dig up that style guide of yours, that >>>> would be valuable), and initialized CmdLine::_cmd to line as Coleen >>>> sugggested. >>>> >>>> @Chris >>>>> >>>>> I'm not certain why you added default field initializers for >>>>> CmdLine::CmdLine when each field is explicitly initialized within the >>>>> method >>>>> body. >>>> >>>> See Coleens response. I usually prefer it that way if the constructor >>>> is not extremely obvious, to prevent missing initializations should >>>> the constructor body evolve. C++ compiler should optimize that away >>>> anyway. >>> >>> I tend to think the opposite. Using the initializer list in this way >>> would >>> lead me to think that the field was not initialized in the constructor >>> body, >>> or at the very least there was a path where it might not be initialized. >>> I >>> don't see how this helps when the constructor is not obvious. Is the >>> point >>> of the initializer (in this case) solely to have the code document which >>> fields get initialized in the constructor? >>> >> For me, it is a safety valve against missing initialization. Consider >> this: >> >> class Parser { const char* p; .. }; >> >> Parser(some input) { >> // complicated stuff happening, p gets initialized in all paths. Ok. >> } >> >> evolves to this: >> >> Parser(some input) { >> // more complicated stuff, on one path p may not get initialized. >> } >> >> if p does not get initialized, its value is essentially random, at >> least in relase builds. Especially if Parser is placed on the stack, >> since p then is mapped to whatever happened to be at this stack slot >> before. >> >> So, by always initializing it to NULL: >> >> Parser(some input) : p(NULL) { >> // complicated stuff happening, p gets initialized in all paths >> } >> >> from the beginning, I get reproducable errors. Using p == NULL will >> crash, using p == random-and-maybe-readable-pointer may do random >> things or even work out of accident. >> >> Thanks, Thomas >> > > I agree with Thomas and I think the change looks good. Thank you for > humoring me with the punctuation. It doesn't seem to be here. > > https://wiki.openjdk.java.net/display/HotSpot/StyleGuide > > Thanks, > Coleen > > >> >> >> >> >> >> >>> thanks, >>> >>> Chris >>> >>>> Thanks for the reviews, >>>> >>>> Best Regards, Thomas >>>> >>>> >>>> On Fri, Jun 15, 2018 at 1:53 PM, wrote: >>>>> >>>>> >>>>> On 6/15/18 1:04 AM, Chris Plummer wrote: >>>>> >>>>> Hi Thomas, >>>>> >>>>> I'm not certain why you added default field initializers for >>>>> CmdLine::CmdLine when each field is explicitly initialized within the >>>>> method >>>>> body. >>>>> >>>>> >>>>> I think the C++ optimizer would elide any unnecessary initializations >>>>> and >>>>> it >>>>> seems safe in case new fields are added, although >>>>> >>>>> 38 : _cmd(NULL) >>>>> >>>>> >>>>> could be initialized as _cmd(line). >>>>> >>>>> Can I bikeshed and say I don't like the style where the , are at the >>>>> beginning of the lines? We don't use this style in other places, that >>>>> I've >>>>> noticed, and I really don't like reading the punctuation at the >>>>> beginning >>>>> of >>>>> the line. If I could find our open source wiki for the coding style >>>>> among >>>>> the broken openjdk wiki pages, I would see if the punctuation rule is >>>>> there. >>>>> One initializer per line is ok. >>>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>> >>>>> A number of copyrights need updating. >>>>> >>>>> The rest of the changes look good. >>>>> >>>>> thanks, >>>>> >>>>> Chris >>>>> >>>>> On 6/14/18 1:38 AM, Thomas St?fe wrote: >>>>> >>>>> Hi all, >>>>> >>>>> (I moved the issue to target release 12 - if we get this through the >>>>> review quick, fine, but no reason to add pressure) >>>>> >>>>> some more background: The reason for these cleanups is that I later >>>>> plan to add a new sub command which allows us to run multiple commands >>>>> in one safepoint (see [1] for more details and discussions). Since >>>>> this will require tweaking the parse process a bit, I try to loosen up >>>>> the coding before adding complexity again. >>>>> >>>>> Please find here the new webrev: >>>>> >>>>> Delta: >>>>> >>>>> >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.00-to-01/webrev/ >>>>> Full: >>>>> >>>>> >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.01/webrev/ >>>>> >>>>> What changed: >>>>> >>>>> Aesthetics: >>>>> - I reordered member init lists to member order, as Coleen did ask. I >>>>> also reform them as lists, just to increase readability. >>>>> - For a number of inline methods, I aligned their implementations so >>>>> that they too would be better readable and it would be clearer which >>>>> methods are const >>>>> - Moved DCmdFactory::_DCmdFactoryList up to the other static members. >>>>> >>>>> Functional: >>>>> - Added initializer list for CmdLine >>>>> - Added missing initializations for DCmdArgIter members _key_addr, >>>>> _key_len, _value_addr, _value_len, as Coleen suggested >>>>> - constified DCmdInfo too - since this is an immutable value holder, >>>>> all members can be const. >>>>> - also in DCmdInfo, changed JavaPermission permission() to const >>>>> JavaPermission& permission() to save a bit unnecessary copying. >>>>> - Fixed initialization of DCmdArgumentInfo::_position as Serguei and >>>>> Chris requested >>>>> - Removed default implementation of DCmd::name and DCmd::description. >>>>> There is no good reason to provide one - all child classes should (and >>>>> do) provide an implementation. I rather have linkage errors if someone >>>>> forgets to add one. >>>>> - Removed default implementations for DCmdWithParser::name/description >>>>> - same reasoning as above >>>>> - Removed DCmdWithParser::permission(), it is already provided by the >>>>> base class. >>>>> - Removed DCmdWithParser::num_arguments() since childs of >>>>> DCmdWithParser will always have arguments and hence should implement >>>>> this function. >>>>> - Made all functions in DCmdFactoryImpl non-virtual. There is no point >>>>> in allowing to override them. >>>>> >>>>> -- >>>>> >>>>> Notes: >>>>> >>>>> I see the following commands do not implement "permission" and hence >>>>> rely on the default impl of DCmd::permission, which returns a >>>>> permission structure { NULL, NULL, NULL } - is that okay and intended? >>>>> >>>>> JMXStopRemoteDCmd >>>>> JMXStartLocalDCmd >>>>> JMXStartRemoteDCmd >>>>> TouchedMethodsDCmd >>>>> ClassStatsDCmd >>>>> RunFinalizationDCmd >>>>> SystemGCDCmd >>>>> VMUptimeDCmd >>>>> HelpDCmd >>>>> >>>>> I am not sure where and how these permissions are tested. >>>>> >>>>> Thinking through this further: >>>>> >>>>> Of the 47 child classes of DCmd, 37 seem to return >>>>> "java.lang.management.ManagementPermission", name "monitor". So, we >>>>> have this function 37 times: >>>>> >>>>> static const JavaPermission permission() { >>>>> JavaPermission p = {"java.lang.management.ManagementPermission", >>>>> "monitor", NULL}; >>>>> return p; >>>>> } >>>>> >>>>> Would it then not make sense to make this the default implementation >>>>> for DCmd::permission() - instead of the NULL permission - and only >>>>> require child DCmds to provide a permission if it differs from this >>>>> default? We could then loose all 37 methods. >>>>> >>>>> --- >>>>> >>>>> Similar thoughts for "static DCmd impact()" >>>>> >>>>> which is not provided by: >>>>> >>>>> JMXStatusDCmd >>>>> JMXStopRemoteDCmd >>>>> JMXStartLocalDCmd >>>>> >>>>> which thus return the default impact - DCmd::impact() - which returns >>>>> "Low". >>>>> >>>>> Here I think we should either remove the default implementation and >>>>> force all commands to provide an implementation or stay with the >>>>> default "Low" implementation and remove all 20+ functions in child >>>>> classes which also return "Low". >>>>> >>>>> -- >>>>> >>>>> Finally, some thoughts for possible future cleanups: >>>>> >>>>> GenDCmdArgument internally uses C-heap to hold copies of strings >>>>> (DCmdArgument and DCmdArgument). I wonder >>>>> whether we could use resource area instead. Then we could get rid of >>>>> all cleanup code (GenDCmdArgument::cleanup(), >>>>> GenDCmdArgument::destroy_value()) etc. And if one agrees that parsing >>>>> is only done once and the commands are not reused (which is the case >>>>> currently) we also may get rid of DCmdArgument::reset() and friends. >>>>> Coding would become a lot easier. >>>>> >>>>> -- >>>>> >>>>> IMHO the way object- und template based polymorphy is mixed is a bit >>>>> of a mismatch here and it complicates things a bit. >>>>> >>>>> For instance, Dcmd has above mentioned static functions which return >>>>> immutable attributes for the commands. It is "overwritten" >>>>> template-style by redefining the same static functions in derived >>>>> classes and called via DCmdFactoryImpl. >>>>> >>>>> Maybe it is just me, but I find this a bit roundabout for the problem >>>>> it tries to solve. Also, it is really difficult to parse for modern >>>>> IDEs - e.g. my CDT is unable to tell me who calls "DCmd::permission()" >>>>> since the relation between DCmd::permission() and XXXCmd::permission() >>>>> is just by name. Also, why pay for the text size of 47 just slightly >>>>> different template definitions of DCmdFactory where one class >>>>> would be enough? >>>>> >>>>> The mis-fit between templates and object polymorphism also shows up as >>>>> weird artifacts. E.g. we have 27 variants of this function: >>>>> >>>>> int XXXXDCmd::num_arguments() { >>>>> ResourceMark rm; >>>>> XXXDCmd* dcmd = new SetVMFlagDCmd(NULL, false); >>>>> if (dcmd != NULL) { >>>>> DCmdMark mark(dcmd); >>>>> return dcmd->_dcmdparser.num_arguments(); >>>>> } else { >>>>> return 0; >>>>> } >>>>> } >>>>> >>>>> - so, we want to implement "static XXXDCmd::num_arguments", which - >>>>> being static - has to create an object of the same type to ask it its >>>>> number of arguments. >>>>> >>>>> Maybe we could un-templify the command classes at some point to >>>>> simplify all this? >>>>> >>>>> I think we need two things, a class to hold meta information about the >>>>> command - name, description, argument types etc - which needs to be >>>>> available at startup. And then, we need a second thing to hold runtime >>>>> context: the concrete argument values the user gave us, the >>>>> outputStream* pointer, the execute() function, maybe parser state... >>>>> >>>>> I think this may be expressed simpler and more efficiently with simple >>>>> C++ objects and inheritance, without templates. We could also shrink >>>>> the code quite a bit. And my CDT would find who calls "permissions()" >>>>> :) >>>>> >>>>> >>>>> Best Regards, Thomas >>>>> >>>>> ------- >>>>> >>>>> [1] >>>>> >>>>> >>>>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2018-May/023673.html >>>>> >>>>> >>>>> On Wed, Jun 13, 2018 at 3:41 PM, Thomas St?fe >>>>> wrote: >>>>> >>>>> Hi all, >>>>> >>>>> while working on jcmd I saw some minor cleanup possibilities. >>>>> >>>>> Webrev: >>>>> >>>>> >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8204958-jcmd-cleanups/webrev.00/webrev/ >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8204958 >>>>> >>>>> Most of them are const correctness and using initializer lists instead >>>>> of java-style member initialization in these C++ classes. >>>>> >>>>> Other changes: >>>>> >>>>> - Added a ResourceMark into the DCmd::parse_and_execute() loop >>>>> - removed DCmdFactory::create_global_DCmd() (and following that, >>>>> DCmdFactory::create_Cheap_instance()) since I did not find anyone >>>>> using that function. >>>>> - Also in DCmdFactory, I removed a number of setters which were never >>>>> called and for attributes which should be immutable: hidden, flags, >>>>> enabled. >>>>> >>>>> This is really a rather light cleanup. It did not touch any larger >>>>> issues. >>>>> >>>>> Thank you, >>>>> >>>>> Thomas >>>>> >>>>> >>>>> >>>>> >>>>> >>> > From david.holmes at oracle.com Mon Jun 18 02:08:11 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 18 Jun 2018 12:08:11 +1000 Subject: jcmd - executing multiple commands at the same safepoint? In-Reply-To: <09FA5DC6-5839-4E13-9F8A-CBB772AEA2A3@oracle.com> References: <3D40CD1F-1DD0-44A4-A729-3781B6E3EA7D@gmail.com> <69360DA5-9287-4C84-B5FE-89B240D9C413@gmail.com> <90E966D9-476B-46D9-B08A-7968F6740851@gmail.com> <3606226E-E365-4F9E-B6F3-B573DE1A5579@gmail.com> <5B0486CC.8050801@oracle.com> <937f5c14-327b-5cc5-897a-09aaef349610@oracle.com> <3D31B965-68BE-4CBD-8952-907EBDA71BCE@oracle.com> <09FA5DC6-5839-4E13-9F8A-CBB772AEA2A3@oracle.com> Message-ID: <739cb0e6-357c-75af-e752-a08bc9c90162@oracle.com> Hi Fred, Thanks for this and the analysis below. My 2c is that it is far too risky to provide a general purpose facility like this where arbitrary command combinations can result in crashing the target VM! Certainly no command that requires a JavaThread should be allowed to be requested - only commands actually implemented via safepointing VMOps should be considered. And then we should perhaps constrain things further so that no commands that require prolog or epilog logic are accepted. Turning things around, perhaps we should flag which Dcmds, that operate at a safepoint, are considered "combinable" and only allow those commands to be specified? If we want a general purpose way of expressing a list of Dcmds to execute then perhaps it should be specified to process each command in turn, with a sequence of combinable safepoint ops processed at a single safepoint? Thanks, David On 16/06/2018 1:32 AM, Frederic Parain wrote: > Hi, > > Here?s a first prototype to explore running multiple commands in a single safepoint: > > http://cr.openjdk.java.net/~fparain/DCmdRunAtSafepoint/webrev.00/ > > The syntax is very simple, with the character ?+? used as the delimiter between commands. > > $ ./build/macosx-x64-debug/images/jdk/bin/jcmd 21164 VM.run_at_safepoint VM.version + VM.uptime > 21164: > Java HotSpot(TM) 64-Bit Server VM version 11-internal+0-2018-05-25-2202478.fparain.DCmd > JDK 11.0.0 > 44.143 s > > Playing a bit with it, it quickly shows what are the issues when trying to run > diagnostic commands at a safepoint. > > Issue 1: One important point to remember is that only the VMThread can run at a safepoint, > all JavaThreads are strictly forbidden to run during the safepoint. So executing commands at a > safepoint means they will be executed by the VMThread. Unfortunately, the VMThread is > subject to restrictions and cannot execute all code that a JavaThread can execute. This > issue is visible with the VM.info command, which takes the Heap_lock. The VMThread > is not allowed to grab this lock because it could cause the VMThread to be blocked by > a JavaThread. So far, VM.info cannot be run at a safepoint. > > Issue 2: Executing a code at a safepoint requires a VMOperation. VMOperations are > made of a prologue, a doit() method witch contains the code executed at the safepoint, > and an epilogue. The prologue and the epilogue are executed by the requester of the > VMOperation, usually a JavaThread, and they are not executed at a safepoint. Prologues > and epilogues are designed to execute code the VMThread cannot execute, like taking > locks or loading classes. When the VMThread itself wants to execute a VMOperation, > what is called a nested VMOperation, it only executes the doit() method, skipping the > prologue and the epilogue. This is an issue for diagnostic commands using a VMOp > in their implementation: when these commands are called while the JVM is already > at a safepoint, their prologue and epilogue cannot be executed. This issue impacts > Thread.print and GC.run. > > There are probably other issues still to be found, so I encourage people to test > this patch to see if the combinations of commands they would like to run at a > safepoint can indeed be run in this mode. > > Note that the support for invoking the new command from the DiagnosticCommand > MBean has not be implemented. This is a not negligible work in order to correctly > check Java Permissions before executing the commands. > > Let me know if you consider this code useful or if the limitations described above > make this feature not as pretty as we wanted it to be. > > Regards, > > Fred > > >> On May 29, 2018, at 15:54, Frederic Parain wrote: >> >> Hi Thomas, >> >> >>> On May 26, 2018, at 04:07, Thomas St?fe wrote: >>> >> >>>> >>>> I think that it is a fundamental piece of the design. Adding a new diagnostic >>>> command to group several commands under a unique safepoint: >>>> 1 - requires explicit action from the user (and acknowledgment of the risks) >>>> 2 - doesn?t touch the current syntax/semantic at all >>>> >>>> Another reason I?m pushing for a specific diagnostic command rather than >>>> changing the syntax is remote execution. jcmd is not the only way to invoke >>>> a diagnostic command, they can also be invoke remotely using JMX and the >>>> com.sun.management.DiagnosticCommand dynamic MBean. Through the >>>> platform MBean, users do not have access to the jcmd syntax, they see >>>> each diagnostic command as a MBean operation (providing that they have >>>> been registered with the DCmd_Source_MBean flag). The initial support >>>> for remote execution was taking a single String and processed it just like >>>> a string passed by jcmd, but people developing consoles for remote >>>> management complained that this was not convenient for them, and asked >>>> for something they could use more programmatically. >>>> If the safepoint bundling is implemented in the syntax, JMX clients won?t >>>> be able to use the feature. If we implement it with a new diagnostic command, >>>> it will be visible from the MBean and they will be able to use it. >>> >>> That is interesting. Do you have examples for such programs? I was >>> looking for ways of how to use jcmd (or equivalent) remotely. >> >> If you?re looking for a console application to invoke diagnostic command >> on a remote JVM, Mission Control has a dedicated support for that in the >> ?Diagnostic Commands? tab. >> If you?re looking for exemples of source code doing remote invocations, >> there?re several unit tests for the DiagnosticCommand MBean in the directory >> test/jdk/com/sun/management/DiagnosticCommandMBean. >> >>> >>>> >>>> At last, creating a specific diagnostic command for safepoint bundling would >>>> show the way on how to group commands for consistency. Safepoints is not >>>> the only way to get consistency. For some JVM internals reasons, you might >>>> want to group commands under a single lock, like the Threads_lock. Adding >>>> a diagnostic command for each type of bundling seems more extensible. >>>> >>> >>> You make good points. >>> >>> Adding a new command has also the advantage that it does not affect >>> downward compatibility and hence would not require a CSR. CSR seems to >>> slow down things a bit (I have a very simple CSR for jcmd open >>> undecided since 12 days: >>> https://bugs.openjdk.java.net/browse/JDK-8203043). So, while I think >>> CSR are a necessary process, I am happy to avoid them if possible. >>> >>> Lets spin this out a bit more: >>> >>> -> However the user specifies "please execute these commands at a >>> single safepoint", if one of the enclosed commands cannot be run >>> inside a safepoint the whole bundle should fail, agreed? So, >>> specifying e.g. JFR.start as part of such a command bundle would cause >>> the whole bundle to not being executed. >> >> Yes. The new command N would ask for the execution of a list of >> commands { A, B, C, ? }, and would fail if: >> - any of the specified commands { A, B, C, ? } is not a registered >> command >> - any of the specified commands { A, B, C, ? } doesn?t support >> execution at safepoint >> - any of the specified commands { A, B, C, ? } is not exported to >> the DCmdSource used to invoke N (*) >> >> (*) Not all commands are exported to all interfaces (jcmd,JMX), for >> instance, some commands related to the JMX agent are cannot >> be invoked through the JMX. Even if the new command can be >> invoked from all interfaces, it must not allow to circumvent this >> restriction. This should not be an issue today, because all >> JMX agent commands are calling back Java code, and are not >> suitable for execution at a safepoint. But this would ensure the >> consistency of the model in the long term (and the test is cheap). >> >>> >>> -> If we introduce a command like you propose (lets call it "VM.run" >>> for the sake of the argument) we still need to decide how to delimit >>> the commands at the command line. That brings us back to the syntax >>> question: separation by a explicit delimiter, if possible one that >>> does not clash with common unix shells? >>> >>> Lets try colon ':' : >>> >>> jcmd VM.run VM.uptime : VM.metaspace show-loaders show-classes >>> by-chunktype : VM.classloaders show-classes : VM.info >>> >>> Of course, we can wrap all arguments to jcmd in quotes, then the >>> delimiter sign does not affect things as much, unless we use quotes :) >>> We could e.g. use semicolon: >>> >>> jcmd ' VM.run VM.uptime ; VM.metaspace show-loaders show-classes >>> by-chunktype ; VM.classloaders show-classes ; VM.info ? >>> >> >> I thought about colon and semi-colon too, because they are the most >> obvious delimiters. However, they are already as delimiters in classpaths >> (semi-colon on Unix, colon on Windows), so parsing to find the right >> delimiters would be tricky. >> >> I?d start with ?!? or ?+? in a first time, to make some progress on the other >> aspects of the command, and buy some time to discuss the delimiter >> issue in parallel. >> >> The last resort would be to force each command in the bundle to >> be specified with quotes (arguments included). But it would make the >> syntax very heavy, I?d prefer a simpler delimiter. >> >> >>> Or, we go back to my original idea of letting the command keyword be >>> the indicator for a new command start. Honestly, at this point I have >>> no preference. Both approaches seem reasonable. >>> >>> The only niggle I have with an explicit command is the syntax, which I >>> find a bit awkward. My thinking before reading your response was that >>> I'd prefer a jcmd flag to specify the same behavior: that all commands >>> should be executed at a single safepoint. E.g. [-s/--stacked] >>> >>> jcmd -s ' VM.uptime ; VM.metaspace show-loaders show-classes >>> by-chunktype ; VM.classloaders show-classes ; VM.info ' >>> >>> But I see the disadvantage: depending on how it is implemented this >>> would require consistent changes in both jcmd and hotspot, and not >>> work with other clients using MBeans. Whereas the hypothetical >>> "VM.run" would work out of the box for any old jcmd version and any >>> other client. >>> >>> Of course one could keep the "VM.run" idea and still give jcmd a >>> "-s/--stacked" option, as a shallow convenience feature, which just >>> translates to "VM.run" and runs that. >>> >>> -> A variation of your "have special commands to run a command bundle >>> in one particular context" theme could be having only one command, >>> e.g. "VM.run", specifying the conditions under which to run the bundle >>> as sub options. E.g. "VM.run -s " for "run commands at >>> safepoint", "VM.run -t " for "run commands under >>> Threads_lock". That would even allow to combine these contexts. I do >>> not know if that makes sense. >>> >>> - Another advantage of having an explicit command just occurring to me >>> is that it enables more evolved command files, where one can >>> intersperse command bundles and single commands. For instance, I could >>> give this to a customer: >>> >>> >>> 1 VM.uptime >>> 2 VM.run_commands VM.metaspace show-loaders show-classes by-chunktype >>> VM.classloaders details VM.classloader_stats >>> 3 GC.run >>> 4 VM.run_commands VM.metaspace show-loaders show-classes by-chunktype >>> VM.classloaders details VM.classloader_stats >>> >>> to execute a command bundle before and after triggering a GC. >> >> This would work fine . The parser of the jcmd string will split the >> string based on ?\n? characters, then each line of command will be executed >> sequentially, and lines containing the VM.run_command will then create the >> safepoint and trigger the execution of the different commands specified in >> its argument list. >> >>> >>>> We have a three days week-end in the US, but I should be able to code >>>> something next week. >>>> >>> >>> Oh, you want to take over the implementation, sure! Fine by me. >> >> I?m not promising an implementation, I haven?t worked on this code >> since I left the serviceability team. I need to code at least a skeleton >> of the command to refresh my memory about all aspects of the >> diagnostic command framework. >> >>> >>>> Thank you for your ideas and your interest in diagnostic commands. >>> >>> Sure. Why I am interested in this: We strongly rely on tools similar >>> to jcmd for our support. We have quite capable tools in our (licensed, >>> non-open) VM port, which are similar to jcmd in many aspects. So we >>> try to bring some of the capabilities upstream, in order to ease the >>> merge costs for us and to help the OpenJDK community. >> >> The diagnostic command framework was designed to support an >> extensible set of commands. Feel free to extend, and thank you >> for contributing to the OpenJDK community. >> >> Fred >> >>>> >>>> >>>>> >>>>>> >>>>>> A new separator, more convenient than newline, would be require to have >>>>>> a single line syntax. >>>>>> >>>>>> My 2 cents, >>>>>> >>>>> >>>>> Certainly worth more than 2 cents :) Thanks for your thoughts! >>>>> >>>>> ..Thomas >>>>> >>>>>> Fred >>>>>> >>>>>> >>>>>>> >>>>>>>> >>>>>>>> Erik >>>>>>>> >>>>>>>> Hi Thomas, >>>>>>>> >>>>>>>> very positive suggestion. >>>>>>>> >>>>>>>> One observation: >>>>>>>> There already seems to be some different interpretation of the command >>>>>>>> line >>>>>>>> in different Java versions: >>>>>>>> For instance when I try to dump a flight recording in 1.8.0_152-ea-b05 : >>>>>>>> I >>>>>>>> can use: >>>>>>>> jcmd 33014 JFR.dump filename="recording1.jfr" recording=1 >>>>>>>> but in build 9+181 , the same command must be: >>>>>>>> jcmd 33014 JFR.dump filename="recording1.jfr",recording=1 >>>>>>>> (the comma to separate sub-options you mentioned earlier) >>>>>>>> >>>>>>>> Therefore the suggestion without curly brackets, giving several commands >>>>>>>> after one another seems good with regard to backwards compatibility. >>>>>>>> >>>>>>>> Motivation: hawt.io uses the MBean >>>>>>>> com.sun.management:type=DiagnosticCommand >>>>>>>> to access the same functionality as jcmd. Variations in option syntax >>>>>>>> across >>>>>>>> Java versions is already an issue (only affecting sub-options though, as >>>>>>>> each command is a separate JMX operation). So syntax compatibility is >>>>>>>> highly >>>>>>>> appreciated :-) >>>>>>>> >>>>>>>> Martin >>>>>>>> >>>>>>>> l?r. 12. mai 2018 kl. 20:11 skrev Kirk Pepperdine >>>>>>>> : >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> On May 10, 2018, at 11:26 AM, Thomas St?fe >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> On Thu, May 10, 2018 at 9:13 AM, Kirk Pepperdine >>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> The stacking at the safe point would be a huge win. Right now thread >>>>>>>>>>> dump consistency can really confuse people as the tooling will show >>>>>>>>>>> two >>>>>>>>>>> threads owning the same lock at seemingly the same time. Reality, it?s >>>>>>>>>>> just >>>>>>>>>>> that people didn?t realize you get a safe point for each thread >>>>>>>>>>> therefor >>>>>>>>>>> there is motion in the system as you?re collecting the data. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> I am a bit confused. What tooling are you talking about? >>>>>>>>> >>>>>>>>> >>>>>>>>> jstack. I?ve not seen it with jcmd. But I often see 2 threads holding >>>>>>>>> the >>>>>>>>> same lock at the ?same time? which is of course non-sense. I can dig one >>>>>>>>> up >>>>>>>>> for you if you?re still confused. >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> As an aside, it?s amazing how many dev?s aren?t aware of jcmd. Just >>>>>>>>>>> yesterday after my session at Devoxx I had someone ask me about using >>>>>>>>>>> jfr >>>>>>>>>>> from the command line, many in that session had not seen jcmd before. >>>>>>>>>>> The >>>>>>>>>>> feedback was, wow, this is very useful and I wished I had of known >>>>>>>>>>> about it >>>>>>>>>>> earlier. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Yes, jcmd is quite useful. I also really like the simple design, which >>>>>>>>>> is by default up- and downward compatible (so you can talk to any VM, >>>>>>>>>> new, old, it should not matter). That is just good design. We - Sap - >>>>>>>>>> work to extend jcmd, to help our support folks. E.g. the whole new >>>>>>>>>> VM.metaspace command chain. >>>>>>>>> >>>>>>>>> >>>>>>>>> And simple it is?.well done!!! >>>>>>>>> >>>>>>>>> ? Kirk >>>>>>>>> >>>>>>>> >>>>>> >>>> >> > From mandy.chung at oracle.com Mon Jun 18 03:41:33 2018 From: mandy.chung at oracle.com (mandy chung) Date: Sun, 17 Jun 2018 20:41:33 -0700 Subject: RFR (S): 8195115: G1 Old Gen MemoryPool CollectionUsage.used values don't reflect mixed GC results In-Reply-To: <9F239517-8707-4786-B7E9-7F88CC0D4C6E@amazon.com> References: <958dd448-efde-2414-9c88-a0f1c869dd56@oracle.com> <9F239517-8707-4786-B7E9-7F88CC0D4C6E@amazon.com> Message-ID: <49f2eabc-3d3e-46cd-00b7-d52e11d74dab@oracle.com> Looks fine to me. Mandy On 6/16/18 12:00 PM, Hohensee, Paul wrote: > Thanks for the re-review, Erik. New webrev with your fixes: > > http://cr.openjdk.java.net/~phh/8195115/webrev.04/ > > Need another reviewer, please. > > Thanks, > > Paul From david.holmes at oracle.com Mon Jun 18 04:11:45 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 18 Jun 2018 14:11:45 +1000 Subject: RFR 8205113: Update JVMTI doc references to object allocation tracking In-Reply-To: References: Message-ID: <384783b9-9ec2-093a-2662-a7a817849720@oracle.com> Hi Jeremy, On 16/06/2018 2:33 AM, Jeremy Manson wrote: > Hi all, > > There are a number of references in the JVMTI doc to its not doing > object allocation tracking.? Now that JEP 331 has landed, these > references are obsolete.? This patch changes those references accordingly. > > While I was there, I took the liberty of fixing some spelling?errors. > > As far as I know, these are non-normative changes and modify no API, so > they should not require a CSR. I'm unclear on the nature of the change to "VM Object Allocation". Does the existence of SampledObjectAlloc change when a VM should issue "VM object allocation" events? Thanks, David > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8205113 > > Webrev: > http://cr.openjdk.java.net/~jmanson/8205113/webrev.00/ > > Thanks! > > Jeremy From david.holmes at oracle.com Mon Jun 18 04:54:05 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 18 Jun 2018 14:54:05 +1000 Subject: RFR (XS) 8204961: JVMTI jtreg tests build warnings on 32-bit platforms In-Reply-To: <80d0c562-c684-111e-ba70-a1860247aa6a@oracle.com> References: <80d0c562-c684-111e-ba70-a1860247aa6a@oracle.com> Message-ID: <5283f9f6-87a1-f945-40c3-bb56d1b84971@oracle.com> I ran this through our testing and it was fine. I can sponsor this for you if you like Boris. Thanks, David On 14/06/2018 10:55 PM, David Holmes wrote: > Hi Boris, > > I added serviceability-dev as JVM TI and its tests are technically > serviceability concerns. > > On 14/06/2018 10:39 PM, Boris Ulasevich wrote: >> Hi all, >> >> Please review the following patch: >> ?? https://bugs.openjdk.java.net/browse/JDK-8204961 >> ?? http://cr.openjdk.java.net/~bulasevich/8204961/webrev.01 >> >> Recently opensourced JVMTI tests gives build warnings for ARM32 build. > > I'm guessing the compiler version must have changed since we last ran > these tests on 32-bit ARM. :) > >> GCC complains about conversion between 4-byte pointer to 8-byte jlong >> type which is Ok in this case. I propose to hide warning using >> conversion to intptr_t. > > I was concerned about what the warnings might imply but now I see that a > JVM TI "tag" is simply a jlong used to funnel real pointers around to > use for the tagging. So on 32-bit the upper 32-bits of the tag will > always be zero and there is no data loss in any of the conversions. > > So assuming none of the other compilers complain about this, this seems > fine to me. > > Thanks, > David > >> thanks, >> Boris From david.holmes at oracle.com Mon Jun 18 06:21:28 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 18 Jun 2018 16:21:28 +1000 Subject: RFR(s): 8203343: VM.{metaspace|classloaders|classhierarchy...} jcmd should show invocation targets for Generated{Method|Constructor}AccessorImpl classes In-Reply-To: References: Message-ID: <9b5834fc-a094-a423-2944-174596293c46@oracle.com> Test update looks good! Thanks, David On 14/06/2018 9:30 PM, Thomas St?fe wrote: > Hi all, > > hopefully last changes, with feedback added from Coleen and David. > > Only changes in the provided regression test: I run it now with > -Dsun.reflect.noInflation to make the test independent from the > reflection inflation threshold. I also corrected the copyright dates. > > Delta: http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/webrev.01-to-02/webrev/ > Full: http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/webrev.02/webrev/ > > Thank you, > > Thomas > > > On Wed, Jun 6, 2018 at 6:05 PM, Thomas St?fe wrote: >> Dear all, >> >> may I please have feedback and if possible reviews for this small addition: >> >> CR: https://bugs.openjdk.java.net/browse/JDK-8203343 >> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/webrev.00/webrev/ >> >> (Note: this patch goes atop of >> https://bugs.openjdk.java.net/browse/JDK-8203682, which is currently >> up for RFR). >> >> --- >> >> When analyzing situations involving a lot of reflection, one often >> stares at walls of "GeneratedXXXAccessorXXX" classes. These names are >> generated and not at all helpful in analyzing the problem (e.g. which >> component in a server node does this much reflection and hence uses so >> much metaspace). >> >> This patch adds the ability to print out invocation targets >> additionally to class names if the class is a generated accessor - for >> now this ability has been added to VM.metaspace, VM.classloaders and >> VM.class_hierarchy. >> >> Example output from "VM.class_hierarchy": >> >> >> |--jdk.internal.reflect.MagicAccessorImpl/null >> >> | |--jdk.internal.reflect.ConstructorAccessorImpl/null >> | | |--jdk.internal.reflect.GeneratedConstructorAccessor18/0x00007f9ee8350c10 >> (invokes: org/springframework/boot/context/properties/ConfigurationPropertiesBindingPostProcessorRegistrar:: >> ()V) >> | | |--jdk.internal.reflect.GeneratedConstructorAccessor17/0x00007f9ee8349c00 >> (invokes: org/springframework/boot/context/properties/EnableConfigurationPropertiesImportSelector$ConfigurationPropertiesBeanRegistrar:: >> ()V) >> >> | |--jdk.internal.reflect.MethodAccessorImpl/null >> | | |--jdk.internal.reflect.GeneratedMethodAccessor23/0x00007f9ec8329b60 >> (invokes: org/apache/tomcat/util/modeler/AttributeInfo::setIs (Z)V) >> | | |--jdk.internal.reflect.GeneratedMethodAccessor22/0x00007f9ee831bc70 >> (invokes: org/springframework/boot/autoconfigure/SpringBootApplication::exclude >> ()[Ljava/lang/Class;) >> >> >> See here more examples: >> >> "VM.class_hierarchy" >> http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/example-VM.class_hierarchy.txt >> >> "VM.classloaders show-classes" >> http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/example-VM.classloaders.txt >> >> >> ---- >> >> Note: I am not sure if this is a fit for hotspot-runtime or >> serviceability. Sorry for crossposting. >> >> Thank you, >> >> Thomas From jeremymanson at google.com Mon Jun 18 07:01:18 2018 From: jeremymanson at google.com (Jeremy Manson) Date: Mon, 18 Jun 2018 00:01:18 -0700 Subject: RFR 8205113: Update JVMTI doc references to object allocation tracking In-Reply-To: <384783b9-9ec2-093a-2662-a7a817849720@oracle.com> References: <384783b9-9ec2-093a-2662-a7a817849720@oracle.com> Message-ID: We haven't changed when a VM issues "VM object allocation" events. There were references in the docs to a requirement to use bytecode rewriting and JNI interception to track allocations. With SampledObjectAlloc, this is no longer the case - SampledObjectAlloc can track them. This change is supposed to remove the references to those requirements, and provide suitable replacement text. VM object alloc has specific language about being able to use it to track allocations that cannot be tracked with bytecode instrumentation and JNI interception. My goal in rephrasing was to make it clear that, while you can still use it for this, you can also just use SampledObjectAlloc for everything. Jeremy On Sun, Jun 17, 2018 at 9:11 PM David Holmes wrote: > Hi Jeremy, > > On 16/06/2018 2:33 AM, Jeremy Manson wrote: > > Hi all, > > > > There are a number of references in the JVMTI doc to its not doing > > object allocation tracking. Now that JEP 331 has landed, these > > references are obsolete. This patch changes those references > accordingly. > > > > While I was there, I took the liberty of fixing some spelling errors. > > > > As far as I know, these are non-normative changes and modify no API, so > > they should not require a CSR. > > I'm unclear on the nature of the change to "VM Object Allocation". Does > the existence of SampledObjectAlloc change when a VM should issue "VM > object allocation" events? > > Thanks, > David > > > > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8205113 > > > > Webrev: > > http://cr.openjdk.java.net/~jmanson/8205113/webrev.00/ > > > > Thanks! > > > > Jeremy > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Mon Jun 18 07:57:08 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 18 Jun 2018 17:57:08 +1000 Subject: RFR 8205113: Update JVMTI doc references to object allocation tracking In-Reply-To: References: <384783b9-9ec2-093a-2662-a7a817849720@oracle.com> Message-ID: On 18/06/2018 5:01 PM, Jeremy Manson wrote: > We haven't changed when a VM issues "VM object allocation" events. > > There were references in the docs to a requirement to use bytecode > rewriting and JNI interception to track allocations.? With > SampledObjectAlloc, this is no longer the case - SampledObjectAlloc can > track them.? This change is supposed to remove the references to those > requirements, and provide suitable replacement text. > > VM object alloc has specific language about being able to use it to > track allocations that cannot be tracked with bytecode instrumentation > and JNI interception.? My goal in rephrasing was to make it clear that, > while you can still use it for this, you can also just use > SampledObjectAlloc for everything. Okay. That doesn't come across clearly to me - sorry. So you will now get both kinds of events for allocations done in the VM? Thanks, David > Jeremy > > On Sun, Jun 17, 2018 at 9:11 PM David Holmes > wrote: > > Hi Jeremy, > > On 16/06/2018 2:33 AM, Jeremy Manson wrote: > > Hi all, > > > > There are a number of references in the JVMTI doc to its not doing > > object allocation tracking.? Now that JEP 331 has landed, these > > references are obsolete.? This patch changes those references > accordingly. > > > > While I was there, I took the liberty of fixing some spelling?errors. > > > > As far as I know, these are non-normative changes and modify no > API, so > > they should not require a CSR. > > I'm unclear on the nature of the change to "VM Object Allocation". Does > the existence of SampledObjectAlloc change when a VM should issue "VM > object allocation" events? > > Thanks, > David > > > > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8205113 > > > > Webrev: > > http://cr.openjdk.java.net/~jmanson/8205113/webrev.00/ > > > > Thanks! > > > > Jeremy > From boris.ulasevich at bell-sw.com Mon Jun 18 09:24:12 2018 From: boris.ulasevich at bell-sw.com (Boris Ulasevich) Date: Mon, 18 Jun 2018 12:24:12 +0300 Subject: RFR (XS) 8204961: JVMTI jtreg tests build warnings on 32-bit platforms In-Reply-To: <5283f9f6-87a1-f945-40c3-bb56d1b84971@oracle.com> References: <80d0c562-c684-111e-ba70-a1860247aa6a@oracle.com> <5283f9f6-87a1-f945-40c3-bb56d1b84971@oracle.com> Message-ID: Hi David, Thank you very much! Yes, sponsor this please! best regards, Boris On 18.06.2018 07:54, David Holmes wrote: > I ran this through our testing and it was fine. > > I can sponsor this for you if you like Boris. > > Thanks, > David > > On 14/06/2018 10:55 PM, David Holmes wrote: >> Hi Boris, >> >> I added serviceability-dev as JVM TI and its tests are technically >> serviceability concerns. >> >> On 14/06/2018 10:39 PM, Boris Ulasevich wrote: >>> Hi all, >>> >>> Please review the following patch: >>> ?? https://bugs.openjdk.java.net/browse/JDK-8204961 >>> ?? http://cr.openjdk.java.net/~bulasevich/8204961/webrev.01 >>> >>> Recently opensourced JVMTI tests gives build warnings for ARM32 build. >> >> I'm guessing the compiler version must have changed since we last ran >> these tests on 32-bit ARM. :) >> >>> GCC complains about conversion between 4-byte pointer to 8-byte jlong >>> type which is Ok in this case. I propose to hide warning using >>> conversion to intptr_t. >> >> I was concerned about what the warnings might imply but now I see that >> a JVM TI "tag" is simply a jlong used to funnel real pointers around >> to use for the tagging. So on 32-bit the upper 32-bits of the tag will >> always be zero and there is no data loss in any of the conversions. >> >> So assuming none of the other compilers complain about this, this >> seems fine to me. >> >> Thanks, >> David >> >>> thanks, >>> Boris From thomas.stuefe at gmail.com Mon Jun 18 12:07:26 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 18 Jun 2018 14:07:26 +0200 Subject: RFR(s): 8203343: VM.{metaspace|classloaders|classhierarchy...} jcmd should show invocation targets for Generated{Method|Constructor}AccessorImpl classes In-Reply-To: <9b5834fc-a094-a423-2944-174596293c46@oracle.com> References: <9b5834fc-a094-a423-2944-174596293c46@oracle.com> Message-ID: Thanks, David! On Mon, Jun 18, 2018 at 8:21 AM, David Holmes wrote: > Test update looks good! > > Thanks, > David > > > On 14/06/2018 9:30 PM, Thomas St?fe wrote: >> >> Hi all, >> >> hopefully last changes, with feedback added from Coleen and David. >> >> Only changes in the provided regression test: I run it now with >> -Dsun.reflect.noInflation to make the test independent from the >> reflection inflation threshold. I also corrected the copyright dates. >> >> Delta: >> http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/webrev.01-to-02/webrev/ >> Full: >> http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/webrev.02/webrev/ >> >> Thank you, >> >> Thomas >> >> >> On Wed, Jun 6, 2018 at 6:05 PM, Thomas St?fe >> wrote: >>> >>> Dear all, >>> >>> may I please have feedback and if possible reviews for this small >>> addition: >>> >>> CR: https://bugs.openjdk.java.net/browse/JDK-8203343 >>> Webrev: >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/webrev.00/webrev/ >>> >>> (Note: this patch goes atop of >>> https://bugs.openjdk.java.net/browse/JDK-8203682, which is currently >>> up for RFR). >>> >>> --- >>> >>> When analyzing situations involving a lot of reflection, one often >>> stares at walls of "GeneratedXXXAccessorXXX" classes. These names are >>> generated and not at all helpful in analyzing the problem (e.g. which >>> component in a server node does this much reflection and hence uses so >>> much metaspace). >>> >>> This patch adds the ability to print out invocation targets >>> additionally to class names if the class is a generated accessor - for >>> now this ability has been added to VM.metaspace, VM.classloaders and >>> VM.class_hierarchy. >>> >>> Example output from "VM.class_hierarchy": >>> >>> >>> |--jdk.internal.reflect.MagicAccessorImpl/null >>> >>> | |--jdk.internal.reflect.ConstructorAccessorImpl/null >>> | | >>> |--jdk.internal.reflect.GeneratedConstructorAccessor18/0x00007f9ee8350c10 >>> (invokes: >>> org/springframework/boot/context/properties/ConfigurationPropertiesBindingPostProcessorRegistrar:: >>> ()V) >>> | | >>> |--jdk.internal.reflect.GeneratedConstructorAccessor17/0x00007f9ee8349c00 >>> (invokes: >>> org/springframework/boot/context/properties/EnableConfigurationPropertiesImportSelector$ConfigurationPropertiesBeanRegistrar:: >>> ()V) >>> >>> | |--jdk.internal.reflect.MethodAccessorImpl/null >>> | | >>> |--jdk.internal.reflect.GeneratedMethodAccessor23/0x00007f9ec8329b60 >>> (invokes: org/apache/tomcat/util/modeler/AttributeInfo::setIs (Z)V) >>> | | >>> |--jdk.internal.reflect.GeneratedMethodAccessor22/0x00007f9ee831bc70 >>> (invokes: >>> org/springframework/boot/autoconfigure/SpringBootApplication::exclude >>> ()[Ljava/lang/Class;) >>> >>> >>> See here more examples: >>> >>> "VM.class_hierarchy" >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/example-VM.class_hierarchy.txt >>> >>> "VM.classloaders show-classes" >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203343-VM.metaspace-show-reflection-invocation-targets/example-VM.classloaders.txt >>> >>> >>> ---- >>> >>> Note: I am not sure if this is a fit for hotspot-runtime or >>> serviceability. Sorry for crossposting. >>> >>> Thank you, >>> >>> Thomas From erik.helin at oracle.com Mon Jun 18 14:05:13 2018 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 18 Jun 2018 16:05:13 +0200 Subject: RFR (S): 8195115: G1 Old Gen MemoryPool CollectionUsage.used values don't reflect mixed GC results In-Reply-To: <9F239517-8707-4786-B7E9-7F88CC0D4C6E@amazon.com> References: <958dd448-efde-2414-9c88-a0f1c869dd56@oracle.com> <9F239517-8707-4786-B7E9-7F88CC0D4C6E@amazon.com> Message-ID: On 06/16/2018 09:00 PM, Hohensee, Paul wrote: > Thanks for the re-review, Erik. New webrev with your fixes: > > http://cr.openjdk.java.net/~phh/8195115/webrev.04/ The patch is good to go now, Reviewed. Thanks, Erik > Need another reviewer, please. > > Thanks, > > Paul > > ?On 6/16/18, 1:25 AM, "Erik Helin" wrote: > > On 06/15/2018 10:21 PM, Hohensee, Paul wrote: > > After some difficulty with the submit cluster, with which Erik helped me out, the patch passes. It also passed fastdebug hotspot tier 1 testing on my Mac laptop, which former includes the new test. > > > > I had to increase -Xmx and -Xms to 12m in order to get TestOldGenCollectionUsage to pass on the submit cluster, though the old 10m works fine on my Mac. New webrev: > > Thanks, the change of -Xmx and -Xms to 12m now also makes the test pass > on my workstation. > > > http://cr.openjdk.java.net/~phh/8195115/webrev.03/ > > There seems to be some trailing whitespace in the patch, have you run > jcheck (or `hg diff` which highlights trailing whitespace in red)? > Please see > > + TraceMemoryManagerStats tms(&_memory_manager, gc_cause(), > + collector_state()->yc_type() == Mixed > /* allMemoryPoolsAffected */); > + > ^---- whitespace > > and > > +int MemoryManager::add_pool(MemoryPool* pool) { > + int index = _num_pools; > ^---- whitespace > > Another small comment, I would have written > > +void GCMemoryManager::add_pool(MemoryPool* pool) { > + int index = MemoryManager::add_pool(pool); > + _pool_always_affected_by_gc[index] = true; > +} > + > +void GCMemoryManager::add_pool(MemoryPool* pool, bool > always_affected_by_gc) { > + int index = MemoryManager::add_pool(pool); > + _pool_always_affected_by_gc[index] = always_affected_by_gc; > +} > + > > as > > +void GCMemoryManager::add_pool(MemoryPool* pool) { > + add_pool(pool, true); > +} > + > +void GCMemoryManager::add_pool(MemoryPool* pool, bool > always_affected_by_gc) { > + int index = MemoryManager::add_pool(pool); > + _pool_always_affected_by_gc[index] = always_affected_by_gc; > +} > + > > to not have to two duplicate implementations of > GCMemoryManager::add_pool. Would you mind updating the patch with this > change (and remove the trailing whitespace)? > > Thanks, > Erik > > > Thanks, > > > > Paul > > > > On 6/12/18, 6:52 AM, "Erik Helin" wrote: > > > > (adding back serviceability-dev, please keep both hotspot-gc-dev and > > serviceability-dev) > > > > Hi Paul, > > > > before I start re-reviewing, did you test the new version of the patch > > via the jdk/submit repository [0]? > > > > Thanks, > > Erik > > > > [0]: http://hg.openjdk.java.net/jdk/submit > > > > On 06/09/2018 03:29 PM, Hohensee, Paul wrote: > > > Didn't seem to make it to hotspot-gc-dev... > > > > > > On 6/8/18, 10:14 AM, "serviceability-dev on behalf of Hohensee, Paul" wrote: > > > > > > Back after a long hiatus... > > > > > > Thanks, Eric, for your review. Here's a new webrev incorporating your recommendations. > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8195115 > > > Webrev: http://cr.openjdk.java.net/~phh/8195115/webrev.02/ > > > > > > TIA for your re-review. Plus, may I have another reviewer look at it please? > > > > > > Paul > > > > > > On 2/26/18, 8:47 AM, "Erik Helin" wrote: > > > > > > Hi Paul, > > > > > > a couple of comments on the patch: > > > > > > - memoryService.hpp: > > > + 150 bool countCollection, > > > + 151 bool allMemoryPoolsAffected = true); > > > > > > There is no need to use a default value for the parameter > > > allMemoryPoolsAffected here. Skipping the default value also allows > > > you to put allMemoryPoolsAffected to TraceMemoryManager::initialize > > > in the same relative position as for the constructor parameter (this > > > will make the code more uniform and easier to follow). > > > > > > - memoryManager.cpp > > > > > > Instead of adding a default parameter, maybe add a new method? > > > Something like GCMemoryManager::add_not_always_affected_pool() > > > (I couldn't come up with a shorter name at the moment). > > > > > > - TestMixedOldGenCollectionUsage.java > > > > > > The test is too strict about how and when collections should > > > occur. Tests written this way often become very brittle, they might > > > e.g. fail to finish a concurrent mark on time on a very slow, single > > > core, machine. It is better to either force collections by using the > > > WhiteBox API or make the test more lenient. > > > > > > Thanks, > > > Erik > > > > > > On 02/22/2018 09:54 PM, Hohensee, Paul wrote: > > > > Ping for a review please. > > > > > > > > Thanks, > > > > > > > > Paul > > > > > > > > On 2/16/18, 12:26 PM, "serviceability-dev on behalf of Hohensee, Paul" wrote: > > > > > > > > The CSR https://bugs.openjdk.java.net/browse/JDK-8196719 for the original fix has been approved, so I?m back to requesting a code review, please. > > > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8195115 > > > > Webrev: http://cr.openjdk.java.net/~phh/8195115/webrev.hs.01/ > > > > > > > > Passed a submit repo run, passes its jtreg test, and a JDK8 version is in production use at Amazon. > > > > > > > > From the original RR: > > > > > > > > > The bug is that from the JMX point of view, G1?s incremental collector > > > > > (misnamed as the ?G1 Young Generation? collector) only affects G1?s > > > > > survivor and eden spaces. In fact, mixed collections run by this > > > > > collector also affect the G1 old generation. > > > > > > > > > > This proposed fix is to record, for each of a JMX garbage collector's > > > > > memory pools, whether that memory pool is affected by all collections > > > > > using that collector. And, for each collection, record whether or not > > > > > all the collector's memory pools are affected. After each collection, > > > > > for each memory pool, if either all the collector's memory pools were > > > > > affected or the memory pool is affected for all collections, record > > > > > CollectionUsage for that pool. > > > > > > > > > > For collectors other than G1 Young Generation, all pools are recorded as > > > > > affected by all collections and every collection is recorded as > > > > > affecting all the collector?s memory pools. For the G1 Young Generation > > > > > collector, the G1 Old Gen pool is recorded as not being affected by all > > > > > collections, and non-mixed collections are recorded as not affecting all > > > > > memory pools. The result is that for non-mixed collections, > > > > > CollectionUsage is recorded after a collection only the G1 Eden Space > > > > > and G1 Survivor Space pools, while for mixed collections CollectionUsage > > > > > is recorded for G1 Old Gen as well. > > > > > > > > > > Other than the effect of the fix on G1 Old Gen MemoryPool. > > > > > CollectionUsage, the only external behavior change is that > > > > > GarbageCollectorMXBean.getMemoryPoolNames will now return 3 pool names > > > > > rather than 2. > > > > > > > > > > With this fix, a collector?s memory pools can be divided into two > > > > > disjoint subsets, one that participates in all collections and one that > > > > > doesn?t. This is a bit more general than the minimum necessary to fix > > > > > G1, but not by much. Because I expect it to apply to other incremental > > > > > region-based collectors, I went with the more general solution. I > > > > > minimized the amount of code I had to touch by using default parameters > > > > > for GCMemoryManager::add_pool and the TraceMemoryManagerStats constructors. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From hohensee at amazon.com Mon Jun 18 15:47:10 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Mon, 18 Jun 2018 15:47:10 +0000 Subject: RFR (S): 8195115: G1 Old Gen MemoryPool CollectionUsage.used values don't reflect mixed GC results In-Reply-To: <49f2eabc-3d3e-46cd-00b7-d52e11d74dab@oracle.com> References: <958dd448-efde-2414-9c88-a0f1c869dd56@oracle.com> <9F239517-8707-4786-B7E9-7F88CC0D4C6E@amazon.com> <49f2eabc-3d3e-46cd-00b7-d52e11d74dab@oracle.com> Message-ID: <83241C02-BDBB-4B81-994C-108B6559F986@amazon.com> Thanks for the review, Mandy. :) ?On 6/17/18, 8:43 PM, "mandy chung" wrote: Looks fine to me. Mandy On 6/16/18 12:00 PM, Hohensee, Paul wrote: > Thanks for the re-review, Erik. New webrev with your fixes: > > http://cr.openjdk.java.net/~phh/8195115/webrev.04/ > > Need another reviewer, please. > > Thanks, > > Paul From Alan.Bateman at oracle.com Mon Jun 18 15:56:32 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 18 Jun 2018 16:56:32 +0100 Subject: jstat does not accept program names as vmid? In-Reply-To: References: Message-ID: <2cc814d1-a5a8-db18-0468-de2588296555@oracle.com> On 16/06/2018 18:23, Thomas St?fe wrote: > : > > > Is that by design? > It's pluggable. The two built-in vmid formats are the pid and pid at host for local and remote respectively. I don't recall any discussion here about identifying an application by name. I assume there would at least be issues with uniqueness. -Alan From hohensee at amazon.com Mon Jun 18 16:14:31 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Mon, 18 Jun 2018 16:14:31 +0000 Subject: RFR (S): 8195115: G1 Old Gen MemoryPool CollectionUsage.used values don't reflect mixed GC results In-Reply-To: References: <958dd448-efde-2414-9c88-a0f1c869dd56@oracle.com> <9F239517-8707-4786-B7E9-7F88CC0D4C6E@amazon.com> Message-ID: <63187C09-2278-4346-8E98-90D04F731D59@amazon.com> Thanks, Eric! I'd push, but it seems I don't seem to have permission at the moment. Who should I contact to get that fixed? Thanks, Paul ?On 6/18/18, 7:09 AM, "Erik Helin" wrote: On 06/16/2018 09:00 PM, Hohensee, Paul wrote: > Thanks for the re-review, Erik. New webrev with your fixes: > > http://cr.openjdk.java.net/~phh/8195115/webrev.04/ The patch is good to go now, Reviewed. Thanks, Erik > Need another reviewer, please. > > Thanks, > > Paul > > On 6/16/18, 1:25 AM, "Erik Helin" wrote: > > On 06/15/2018 10:21 PM, Hohensee, Paul wrote: > > After some difficulty with the submit cluster, with which Erik helped me out, the patch passes. It also passed fastdebug hotspot tier 1 testing on my Mac laptop, which former includes the new test. > > > > I had to increase -Xmx and -Xms to 12m in order to get TestOldGenCollectionUsage to pass on the submit cluster, though the old 10m works fine on my Mac. New webrev: > > Thanks, the change of -Xmx and -Xms to 12m now also makes the test pass > on my workstation. > > > http://cr.openjdk.java.net/~phh/8195115/webrev.03/ > > There seems to be some trailing whitespace in the patch, have you run > jcheck (or `hg diff` which highlights trailing whitespace in red)? > Please see > > + TraceMemoryManagerStats tms(&_memory_manager, gc_cause(), > + collector_state()->yc_type() == Mixed > /* allMemoryPoolsAffected */); > + > ^---- whitespace > > and > > +int MemoryManager::add_pool(MemoryPool* pool) { > + int index = _num_pools; > ^---- whitespace > > Another small comment, I would have written > > +void GCMemoryManager::add_pool(MemoryPool* pool) { > + int index = MemoryManager::add_pool(pool); > + _pool_always_affected_by_gc[index] = true; > +} > + > +void GCMemoryManager::add_pool(MemoryPool* pool, bool > always_affected_by_gc) { > + int index = MemoryManager::add_pool(pool); > + _pool_always_affected_by_gc[index] = always_affected_by_gc; > +} > + > > as > > +void GCMemoryManager::add_pool(MemoryPool* pool) { > + add_pool(pool, true); > +} > + > +void GCMemoryManager::add_pool(MemoryPool* pool, bool > always_affected_by_gc) { > + int index = MemoryManager::add_pool(pool); > + _pool_always_affected_by_gc[index] = always_affected_by_gc; > +} > + > > to not have to two duplicate implementations of > GCMemoryManager::add_pool. Would you mind updating the patch with this > change (and remove the trailing whitespace)? > > Thanks, > Erik > > > Thanks, > > > > Paul > > > > On 6/12/18, 6:52 AM, "Erik Helin" wrote: > > > > (adding back serviceability-dev, please keep both hotspot-gc-dev and > > serviceability-dev) > > > > Hi Paul, > > > > before I start re-reviewing, did you test the new version of the patch > > via the jdk/submit repository [0]? > > > > Thanks, > > Erik > > > > [0]: http://hg.openjdk.java.net/jdk/submit > > > > On 06/09/2018 03:29 PM, Hohensee, Paul wrote: > > > Didn't seem to make it to hotspot-gc-dev... > > > > > > On 6/8/18, 10:14 AM, "serviceability-dev on behalf of Hohensee, Paul" wrote: > > > > > > Back after a long hiatus... > > > > > > Thanks, Eric, for your review. Here's a new webrev incorporating your recommendations. > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8195115 > > > Webrev: http://cr.openjdk.java.net/~phh/8195115/webrev.02/ > > > > > > TIA for your re-review. Plus, may I have another reviewer look at it please? > > > > > > Paul > > > > > > On 2/26/18, 8:47 AM, "Erik Helin" wrote: > > > > > > Hi Paul, > > > > > > a couple of comments on the patch: > > > > > > - memoryService.hpp: > > > + 150 bool countCollection, > > > + 151 bool allMemoryPoolsAffected = true); > > > > > > There is no need to use a default value for the parameter > > > allMemoryPoolsAffected here. Skipping the default value also allows > > > you to put allMemoryPoolsAffected to TraceMemoryManager::initialize > > > in the same relative position as for the constructor parameter (this > > > will make the code more uniform and easier to follow). > > > > > > - memoryManager.cpp > > > > > > Instead of adding a default parameter, maybe add a new method? > > > Something like GCMemoryManager::add_not_always_affected_pool() > > > (I couldn't come up with a shorter name at the moment). > > > > > > - TestMixedOldGenCollectionUsage.java > > > > > > The test is too strict about how and when collections should > > > occur. Tests written this way often become very brittle, they might > > > e.g. fail to finish a concurrent mark on time on a very slow, single > > > core, machine. It is better to either force collections by using the > > > WhiteBox API or make the test more lenient. > > > > > > Thanks, > > > Erik > > > > > > On 02/22/2018 09:54 PM, Hohensee, Paul wrote: > > > > Ping for a review please. > > > > > > > > Thanks, > > > > > > > > Paul > > > > > > > > On 2/16/18, 12:26 PM, "serviceability-dev on behalf of Hohensee, Paul" wrote: > > > > > > > > The CSR https://bugs.openjdk.java.net/browse/JDK-8196719 for the original fix has been approved, so I?m back to requesting a code review, please. > > > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8195115 > > > > Webrev: http://cr.openjdk.java.net/~phh/8195115/webrev.hs.01/ > > > > > > > > Passed a submit repo run, passes its jtreg test, and a JDK8 version is in production use at Amazon. > > > > > > > > From the original RR: > > > > > > > > > The bug is that from the JMX point of view, G1?s incremental collector > > > > > (misnamed as the ?G1 Young Generation? collector) only affects G1?s > > > > > survivor and eden spaces. In fact, mixed collections run by this > > > > > collector also affect the G1 old generation. > > > > > > > > > > This proposed fix is to record, for each of a JMX garbage collector's > > > > > memory pools, whether that memory pool is affected by all collections > > > > > using that collector. And, for each collection, record whether or not > > > > > all the collector's memory pools are affected. After each collection, > > > > > for each memory pool, if either all the collector's memory pools were > > > > > affected or the memory pool is affected for all collections, record > > > > > CollectionUsage for that pool. > > > > > > > > > > For collectors other than G1 Young Generation, all pools are recorded as > > > > > affected by all collections and every collection is recorded as > > > > > affecting all the collector?s memory pools. For the G1 Young Generation > > > > > collector, the G1 Old Gen pool is recorded as not being affected by all > > > > > collections, and non-mixed collections are recorded as not affecting all > > > > > memory pools. The result is that for non-mixed collections, > > > > > CollectionUsage is recorded after a collection only the G1 Eden Space > > > > > and G1 Survivor Space pools, while for mixed collections CollectionUsage > > > > > is recorded for G1 Old Gen as well. > > > > > > > > > > Other than the effect of the fix on G1 Old Gen MemoryPool. > > > > > CollectionUsage, the only external behavior change is that > > > > > GarbageCollectorMXBean.getMemoryPoolNames will now return 3 pool names > > > > > rather than 2. > > > > > > > > > > With this fix, a collector?s memory pools can be divided into two > > > > > disjoint subsets, one that participates in all collections and one that > > > > > doesn?t. This is a bit more general than the minimum necessary to fix > > > > > G1, but not by much. Because I expect it to apply to other incremental > > > > > region-based collectors, I went with the more general solution. I > > > > > minimized the amount of code I had to touch by using default parameters > > > > > for GCMemoryManager::add_pool and the TraceMemoryManagerStats constructors. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From erik.helin at oracle.com Mon Jun 18 17:25:07 2018 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 18 Jun 2018 19:25:07 +0200 Subject: RFR (S): 8195115: G1 Old Gen MemoryPool CollectionUsage.used values don't reflect mixed GC results In-Reply-To: <63187C09-2278-4346-8E98-90D04F731D59@amazon.com> References: <958dd448-efde-2414-9c88-a0f1c869dd56@oracle.com> <9F239517-8707-4786-B7E9-7F88CC0D4C6E@amazon.com> <63187C09-2278-4346-8E98-90D04F731D59@amazon.com> Message-ID: <43d0ba3d-3006-254f-2927-5ffef51e3e4d@oracle.com> On 06/18/2018 06:14 PM, Hohensee, Paul wrote: > Thanks, Eric! > > I'd push, but it seems I don't seem to have permission at the moment. Who should I contact to get that fixed? That would be ops at openjdk.java.net. Thanks, Erik > Thanks, > > Paul > > ?On 6/18/18, 7:09 AM, "Erik Helin" wrote: > > On 06/16/2018 09:00 PM, Hohensee, Paul wrote: > > Thanks for the re-review, Erik. New webrev with your fixes: > > > > http://cr.openjdk.java.net/~phh/8195115/webrev.04/ > > The patch is good to go now, Reviewed. > > Thanks, > Erik > > > Need another reviewer, please. > > > > Thanks, > > > > Paul > > > > On 6/16/18, 1:25 AM, "Erik Helin" wrote: > > > > On 06/15/2018 10:21 PM, Hohensee, Paul wrote: > > > After some difficulty with the submit cluster, with which Erik helped me out, the patch passes. It also passed fastdebug hotspot tier 1 testing on my Mac laptop, which former includes the new test. > > > > > > I had to increase -Xmx and -Xms to 12m in order to get TestOldGenCollectionUsage to pass on the submit cluster, though the old 10m works fine on my Mac. New webrev: > > > > Thanks, the change of -Xmx and -Xms to 12m now also makes the test pass > > on my workstation. > > > > > http://cr.openjdk.java.net/~phh/8195115/webrev.03/ > > > > There seems to be some trailing whitespace in the patch, have you run > > jcheck (or `hg diff` which highlights trailing whitespace in red)? > > Please see > > > > + TraceMemoryManagerStats tms(&_memory_manager, gc_cause(), > > + collector_state()->yc_type() == Mixed > > /* allMemoryPoolsAffected */); > > + > > ^---- whitespace > > > > and > > > > +int MemoryManager::add_pool(MemoryPool* pool) { > > + int index = _num_pools; > > ^---- whitespace > > > > Another small comment, I would have written > > > > +void GCMemoryManager::add_pool(MemoryPool* pool) { > > + int index = MemoryManager::add_pool(pool); > > + _pool_always_affected_by_gc[index] = true; > > +} > > + > > +void GCMemoryManager::add_pool(MemoryPool* pool, bool > > always_affected_by_gc) { > > + int index = MemoryManager::add_pool(pool); > > + _pool_always_affected_by_gc[index] = always_affected_by_gc; > > +} > > + > > > > as > > > > +void GCMemoryManager::add_pool(MemoryPool* pool) { > > + add_pool(pool, true); > > +} > > + > > +void GCMemoryManager::add_pool(MemoryPool* pool, bool > > always_affected_by_gc) { > > + int index = MemoryManager::add_pool(pool); > > + _pool_always_affected_by_gc[index] = always_affected_by_gc; > > +} > > + > > > > to not have to two duplicate implementations of > > GCMemoryManager::add_pool. Would you mind updating the patch with this > > change (and remove the trailing whitespace)? > > > > Thanks, > > Erik > > > > > Thanks, > > > > > > Paul > > > > > > On 6/12/18, 6:52 AM, "Erik Helin" wrote: > > > > > > (adding back serviceability-dev, please keep both hotspot-gc-dev and > > > serviceability-dev) > > > > > > Hi Paul, > > > > > > before I start re-reviewing, did you test the new version of the patch > > > via the jdk/submit repository [0]? > > > > > > Thanks, > > > Erik > > > > > > [0]: http://hg.openjdk.java.net/jdk/submit > > > > > > On 06/09/2018 03:29 PM, Hohensee, Paul wrote: > > > > Didn't seem to make it to hotspot-gc-dev... > > > > > > > > On 6/8/18, 10:14 AM, "serviceability-dev on behalf of Hohensee, Paul" wrote: > > > > > > > > Back after a long hiatus... > > > > > > > > Thanks, Eric, for your review. Here's a new webrev incorporating your recommendations. > > > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8195115 > > > > Webrev: http://cr.openjdk.java.net/~phh/8195115/webrev.02/ > > > > > > > > TIA for your re-review. Plus, may I have another reviewer look at it please? > > > > > > > > Paul > > > > > > > > On 2/26/18, 8:47 AM, "Erik Helin" wrote: > > > > > > > > Hi Paul, > > > > > > > > a couple of comments on the patch: > > > > > > > > - memoryService.hpp: > > > > + 150 bool countCollection, > > > > + 151 bool allMemoryPoolsAffected = true); > > > > > > > > There is no need to use a default value for the parameter > > > > allMemoryPoolsAffected here. Skipping the default value also allows > > > > you to put allMemoryPoolsAffected to TraceMemoryManager::initialize > > > > in the same relative position as for the constructor parameter (this > > > > will make the code more uniform and easier to follow). > > > > > > > > - memoryManager.cpp > > > > > > > > Instead of adding a default parameter, maybe add a new method? > > > > Something like GCMemoryManager::add_not_always_affected_pool() > > > > (I couldn't come up with a shorter name at the moment). > > > > > > > > - TestMixedOldGenCollectionUsage.java > > > > > > > > The test is too strict about how and when collections should > > > > occur. Tests written this way often become very brittle, they might > > > > e.g. fail to finish a concurrent mark on time on a very slow, single > > > > core, machine. It is better to either force collections by using the > > > > WhiteBox API or make the test more lenient. > > > > > > > > Thanks, > > > > Erik > > > > > > > > On 02/22/2018 09:54 PM, Hohensee, Paul wrote: > > > > > Ping for a review please. > > > > > > > > > > Thanks, > > > > > > > > > > Paul > > > > > > > > > > On 2/16/18, 12:26 PM, "serviceability-dev on behalf of Hohensee, Paul" wrote: > > > > > > > > > > The CSR https://bugs.openjdk.java.net/browse/JDK-8196719 for the original fix has been approved, so I?m back to requesting a code review, please. > > > > > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8195115 > > > > > Webrev: http://cr.openjdk.java.net/~phh/8195115/webrev.hs.01/ > > > > > > > > > > Passed a submit repo run, passes its jtreg test, and a JDK8 version is in production use at Amazon. > > > > > > > > > > From the original RR: > > > > > > > > > > > The bug is that from the JMX point of view, G1?s incremental collector > > > > > > (misnamed as the ?G1 Young Generation? collector) only affects G1?s > > > > > > survivor and eden spaces. In fact, mixed collections run by this > > > > > > collector also affect the G1 old generation. > > > > > > > > > > > > This proposed fix is to record, for each of a JMX garbage collector's > > > > > > memory pools, whether that memory pool is affected by all collections > > > > > > using that collector. And, for each collection, record whether or not > > > > > > all the collector's memory pools are affected. After each collection, > > > > > > for each memory pool, if either all the collector's memory pools were > > > > > > affected or the memory pool is affected for all collections, record > > > > > > CollectionUsage for that pool. > > > > > > > > > > > > For collectors other than G1 Young Generation, all pools are recorded as > > > > > > affected by all collections and every collection is recorded as > > > > > > affecting all the collector?s memory pools. For the G1 Young Generation > > > > > > collector, the G1 Old Gen pool is recorded as not being affected by all > > > > > > collections, and non-mixed collections are recorded as not affecting all > > > > > > memory pools. The result is that for non-mixed collections, > > > > > > CollectionUsage is recorded after a collection only the G1 Eden Space > > > > > > and G1 Survivor Space pools, while for mixed collections CollectionUsage > > > > > > is recorded for G1 Old Gen as well. > > > > > > > > > > > > Other than the effect of the fix on G1 Old Gen MemoryPool. > > > > > > CollectionUsage, the only external behavior change is that > > > > > > GarbageCollectorMXBean.getMemoryPoolNames will now return 3 pool names > > > > > > rather than 2. > > > > > > > > > > > > With this fix, a collector?s memory pools can be divided into two > > > > > > disjoint subsets, one that participates in all collections and one that > > > > > > doesn?t. This is a bit more general than the minimum necessary to fix > > > > > > G1, but not by much. Because I expect it to apply to other incremental > > > > > > region-based collectors, I went with the more general solution. I > > > > > > minimized the amount of code I had to touch by using default parameters > > > > > > for GCMemoryManager::add_pool and the TraceMemoryManagerStats constructors. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From hohensee at amazon.com Mon Jun 18 18:17:43 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Mon, 18 Jun 2018 18:17:43 +0000 Subject: RFR (S): 8195115: G1 Old Gen MemoryPool CollectionUsage.used values don't reflect mixed GC results In-Reply-To: <43d0ba3d-3006-254f-2927-5ffef51e3e4d@oracle.com> References: <958dd448-efde-2414-9c88-a0f1c869dd56@oracle.com> <9F239517-8707-4786-B7E9-7F88CC0D4C6E@amazon.com> <63187C09-2278-4346-8E98-90D04F731D59@amazon.com> <43d0ba3d-3006-254f-2927-5ffef51e3e4d@oracle.com> Message-ID: Thanks, Erik. ?On 6/18/18, 10:26 AM, "Erik Helin" wrote: On 06/18/2018 06:14 PM, Hohensee, Paul wrote: > Thanks, Eric! > > I'd push, but it seems I don't seem to have permission at the moment. Who should I contact to get that fixed? That would be ops at openjdk.java.net. Thanks, Erik > Thanks, > > Paul > > On 6/18/18, 7:09 AM, "Erik Helin" wrote: > > On 06/16/2018 09:00 PM, Hohensee, Paul wrote: > > Thanks for the re-review, Erik. New webrev with your fixes: > > > > http://cr.openjdk.java.net/~phh/8195115/webrev.04/ > > The patch is good to go now, Reviewed. > > Thanks, > Erik > > > Need another reviewer, please. > > > > Thanks, > > > > Paul > > > > On 6/16/18, 1:25 AM, "Erik Helin" wrote: > > > > On 06/15/2018 10:21 PM, Hohensee, Paul wrote: > > > After some difficulty with the submit cluster, with which Erik helped me out, the patch passes. It also passed fastdebug hotspot tier 1 testing on my Mac laptop, which former includes the new test. > > > > > > I had to increase -Xmx and -Xms to 12m in order to get TestOldGenCollectionUsage to pass on the submit cluster, though the old 10m works fine on my Mac. New webrev: > > > > Thanks, the change of -Xmx and -Xms to 12m now also makes the test pass > > on my workstation. > > > > > http://cr.openjdk.java.net/~phh/8195115/webrev.03/ > > > > There seems to be some trailing whitespace in the patch, have you run > > jcheck (or `hg diff` which highlights trailing whitespace in red)? > > Please see > > > > + TraceMemoryManagerStats tms(&_memory_manager, gc_cause(), > > + collector_state()->yc_type() == Mixed > > /* allMemoryPoolsAffected */); > > + > > ^---- whitespace > > > > and > > > > +int MemoryManager::add_pool(MemoryPool* pool) { > > + int index = _num_pools; > > ^---- whitespace > > > > Another small comment, I would have written > > > > +void GCMemoryManager::add_pool(MemoryPool* pool) { > > + int index = MemoryManager::add_pool(pool); > > + _pool_always_affected_by_gc[index] = true; > > +} > > + > > +void GCMemoryManager::add_pool(MemoryPool* pool, bool > > always_affected_by_gc) { > > + int index = MemoryManager::add_pool(pool); > > + _pool_always_affected_by_gc[index] = always_affected_by_gc; > > +} > > + > > > > as > > > > +void GCMemoryManager::add_pool(MemoryPool* pool) { > > + add_pool(pool, true); > > +} > > + > > +void GCMemoryManager::add_pool(MemoryPool* pool, bool > > always_affected_by_gc) { > > + int index = MemoryManager::add_pool(pool); > > + _pool_always_affected_by_gc[index] = always_affected_by_gc; > > +} > > + > > > > to not have to two duplicate implementations of > > GCMemoryManager::add_pool. Would you mind updating the patch with this > > change (and remove the trailing whitespace)? > > > > Thanks, > > Erik > > > > > Thanks, > > > > > > Paul > > > > > > On 6/12/18, 6:52 AM, "Erik Helin" wrote: > > > > > > (adding back serviceability-dev, please keep both hotspot-gc-dev and > > > serviceability-dev) > > > > > > Hi Paul, > > > > > > before I start re-reviewing, did you test the new version of the patch > > > via the jdk/submit repository [0]? > > > > > > Thanks, > > > Erik > > > > > > [0]: http://hg.openjdk.java.net/jdk/submit > > > > > > On 06/09/2018 03:29 PM, Hohensee, Paul wrote: > > > > Didn't seem to make it to hotspot-gc-dev... > > > > > > > > On 6/8/18, 10:14 AM, "serviceability-dev on behalf of Hohensee, Paul" wrote: > > > > > > > > Back after a long hiatus... > > > > > > > > Thanks, Eric, for your review. Here's a new webrev incorporating your recommendations. > > > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8195115 > > > > Webrev: http://cr.openjdk.java.net/~phh/8195115/webrev.02/ > > > > > > > > TIA for your re-review. Plus, may I have another reviewer look at it please? > > > > > > > > Paul > > > > > > > > On 2/26/18, 8:47 AM, "Erik Helin" wrote: > > > > > > > > Hi Paul, > > > > > > > > a couple of comments on the patch: > > > > > > > > - memoryService.hpp: > > > > + 150 bool countCollection, > > > > + 151 bool allMemoryPoolsAffected = true); > > > > > > > > There is no need to use a default value for the parameter > > > > allMemoryPoolsAffected here. Skipping the default value also allows > > > > you to put allMemoryPoolsAffected to TraceMemoryManager::initialize > > > > in the same relative position as for the constructor parameter (this > > > > will make the code more uniform and easier to follow). > > > > > > > > - memoryManager.cpp > > > > > > > > Instead of adding a default parameter, maybe add a new method? > > > > Something like GCMemoryManager::add_not_always_affected_pool() > > > > (I couldn't come up with a shorter name at the moment). > > > > > > > > - TestMixedOldGenCollectionUsage.java > > > > > > > > The test is too strict about how and when collections should > > > > occur. Tests written this way often become very brittle, they might > > > > e.g. fail to finish a concurrent mark on time on a very slow, single > > > > core, machine. It is better to either force collections by using the > > > > WhiteBox API or make the test more lenient. > > > > > > > > Thanks, > > > > Erik > > > > > > > > On 02/22/2018 09:54 PM, Hohensee, Paul wrote: > > > > > Ping for a review please. > > > > > > > > > > Thanks, > > > > > > > > > > Paul > > > > > > > > > > On 2/16/18, 12:26 PM, "serviceability-dev on behalf of Hohensee, Paul" wrote: > > > > > > > > > > The CSR https://bugs.openjdk.java.net/browse/JDK-8196719 for the original fix has been approved, so I?m back to requesting a code review, please. > > > > > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8195115 > > > > > Webrev: http://cr.openjdk.java.net/~phh/8195115/webrev.hs.01/ > > > > > > > > > > Passed a submit repo run, passes its jtreg test, and a JDK8 version is in production use at Amazon. > > > > > > > > > > From the original RR: > > > > > > > > > > > The bug is that from the JMX point of view, G1?s incremental collector > > > > > > (misnamed as the ?G1 Young Generation? collector) only affects G1?s > > > > > > survivor and eden spaces. In fact, mixed collections run by this > > > > > > collector also affect the G1 old generation. > > > > > > > > > > > > This proposed fix is to record, for each of a JMX garbage collector's > > > > > > memory pools, whether that memory pool is affected by all collections > > > > > > using that collector. And, for each collection, record whether or not > > > > > > all the collector's memory pools are affected. After each collection, > > > > > > for each memory pool, if either all the collector's memory pools were > > > > > > affected or the memory pool is affected for all collections, record > > > > > > CollectionUsage for that pool. > > > > > > > > > > > > For collectors other than G1 Young Generation, all pools are recorded as > > > > > > affected by all collections and every collection is recorded as > > > > > > affecting all the collector?s memory pools. For the G1 Young Generation > > > > > > collector, the G1 Old Gen pool is recorded as not being affected by all > > > > > > collections, and non-mixed collections are recorded as not affecting all > > > > > > memory pools. The result is that for non-mixed collections, > > > > > > CollectionUsage is recorded after a collection only the G1 Eden Space > > > > > > and G1 Survivor Space pools, while for mixed collections CollectionUsage > > > > > > is recorded for G1 Old Gen as well. > > > > > > > > > > > > Other than the effect of the fix on G1 Old Gen MemoryPool. > > > > > > CollectionUsage, the only external behavior change is that > > > > > > GarbageCollectorMXBean.getMemoryPoolNames will now return 3 pool names > > > > > > rather than 2. > > > > > > > > > > > > With this fix, a collector?s memory pools can be divided into two > > > > > > disjoint subsets, one that participates in all collections and one that > > > > > > doesn?t. This is a bit more general than the minimum necessary to fix > > > > > > G1, but not by much. Because I expect it to apply to other incremental > > > > > > region-based collectors, I went with the more general solution. I > > > > > > minimized the amount of code I had to touch by using default parameters > > > > > > for GCMemoryManager::add_pool and the TraceMemoryManagerStats constructors. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From jeremymanson at google.com Mon Jun 18 18:50:36 2018 From: jeremymanson at google.com (Jeremy Manson) Date: Mon, 18 Jun 2018 11:50:36 -0700 Subject: RFR 8205113: Update JVMTI doc references to object allocation tracking In-Reply-To: References: <384783b9-9ec2-093a-2662-a7a817849720@oracle.com> Message-ID: Yup! The paragraph meanders a bit. How about something like: Sent when the virtual machine allocates an Object visible to Java programming language code without using a new bytecode variant or a JNI method. Many approaches to tracking object allocation use a combination of bytecode-based instrumentation and JNI function interception to intercept allocations. However, this approach can leave a number of allocations undetected. Allocations that are neither triggered by bytecode nor JNI are executed directly by the VM. When those allocations occur, the VM should send this event. Virtual machines that are incapable of bytecode instrumentation for some or all of their methods may also send this event.

Note that the SampledObjectAlloc event is triggered on all Java object allocations, including those triggered by bytecode, JNI methods, and VM events. On Mon, Jun 18, 2018 at 12:57 AM David Holmes wrote: > On 18/06/2018 5:01 PM, Jeremy Manson wrote: > > We haven't changed when a VM issues "VM object allocation" events. > > > > There were references in the docs to a requirement to use bytecode > > rewriting and JNI interception to track allocations. With > > SampledObjectAlloc, this is no longer the case - SampledObjectAlloc can > > track them. This change is supposed to remove the references to those > > requirements, and provide suitable replacement text. > > > > VM object alloc has specific language about being able to use it to > > track allocations that cannot be tracked with bytecode instrumentation > > and JNI interception. My goal in rephrasing was to make it clear that, > > while you can still use it for this, you can also just use > > SampledObjectAlloc for everything. > > Okay. That doesn't come across clearly to me - sorry. So you will now > get both kinds of events for allocations done in the VM? > > Thanks, > David > > > > Jeremy > > > > On Sun, Jun 17, 2018 at 9:11 PM David Holmes > > wrote: > > > > Hi Jeremy, > > > > On 16/06/2018 2:33 AM, Jeremy Manson wrote: > > > Hi all, > > > > > > There are a number of references in the JVMTI doc to its not doing > > > object allocation tracking. Now that JEP 331 has landed, these > > > references are obsolete. This patch changes those references > > accordingly. > > > > > > While I was there, I took the liberty of fixing some > spelling errors. > > > > > > As far as I know, these are non-normative changes and modify no > > API, so > > > they should not require a CSR. > > > > I'm unclear on the nature of the change to "VM Object Allocation". > Does > > the existence of SampledObjectAlloc change when a VM should issue "VM > > object allocation" events? > > > > Thanks, > > David > > > > > > > > Bug: > > > https://bugs.openjdk.java.net/browse/JDK-8205113 > > > > > > Webrev: > > > http://cr.openjdk.java.net/~jmanson/8205113/webrev.00/ > > > > > > Thanks! > > > > > > Jeremy > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From frederic.parain at oracle.com Mon Jun 18 20:40:41 2018 From: frederic.parain at oracle.com (Frederic Parain) Date: Mon, 18 Jun 2018 16:40:41 -0400 Subject: jcmd - executing multiple commands at the same safepoint? In-Reply-To: References: <3D40CD1F-1DD0-44A4-A729-3781B6E3EA7D@gmail.com> <69360DA5-9287-4C84-B5FE-89B240D9C413@gmail.com> <90E966D9-476B-46D9-B08A-7968F6740851@gmail.com> <3606226E-E365-4F9E-B6F3-B573DE1A5579@gmail.com> <5B0486CC.8050801@oracle.com> <937f5c14-327b-5cc5-897a-09aaef349610@oracle.com> <3D31B965-68BE-4CBD-8952-907EBDA71BCE@oracle.com> <09FA5DC6-5839-4E13-9F8A-CBB772AEA2A3@oracle.com> Message-ID: <81073715-B865-439A-8E5A-26D951A78E2A@oracle.com> Hi Thomas, Thank you for experimenting with the patch and providing feedback. I?ve looked to VM.info, and the only issue seems to be this section of vmError.cpp: 1064 if (Universe::is_fully_initialized()) { 1065 MutexLocker hl(Heap_lock); 1066 Universe::heap()->print_on_error(st); 1067 st->cr(); 1068 st->print_cr("Polling page: " INTPTR_FORMAT, p2i(os::get_polling_page())); 1069 st->cr(); 1070 } It would be interesting to have more information from the GC team about while taking the Heap_lock is required. If it is only to prevent mutator threads to update the counters dumped by Universe::heap()->print_on_error(st); then it would be safe to not take the lock when running at a safepoint: 1064 if (Universe::is_fully_initialized()) { 1065 if (SafepointSynchronize::is_at_safepoint()) { 1066 Universe::heap()->print_on_error(st); 1067 st->cr(); 1068 st->print_cr("Polling page: " INTPTR_FORMAT, p2i(os::get_polling_page())); 1069 st->cr(); 1070 } else { 1071 MutexLocker hl(Heap_lock); 1072 Universe::heap()->print_on_error(st); 1073 st->cr(); 1074 st->print_cr("Polling page: " INTPTR_FORMAT, p2i(os::get_polling_page())); 1075 st->cr(); 1076 } 1077 } If the problem is different, we would need to know what concurrency issue is addressed here, in order to determine what can be done when run at a safepoint. Fred > On Jun 15, 2018, at 14:09, Thomas St?fe wrote: > > Hi Frederic, > > this is nice and now I am surprised how small the change is :) > > I played around with it a bit, combining VM.metaspace, VM.classloaders > and VM.systemdictionary, including options. All worked well. > VM.stringtable did not produce output and VM.info, as you predicted, > crashes the VM because it needs the heap lock. > > (Side note: I think VM.info is too useful to be held back by not > getting the heap lock. It would be nice if it would try to lock and if > failing, would just omit the steps needing a heap lock.) > > So, yes, on first glance I find this useful. I'll play around with > this a bit more. Lets wait for others to chime in. > > Best Regards, Thomas > > > On Fri, Jun 15, 2018 at 5:32 PM, Frederic Parain > wrote: >> Hi, >> >> Here?s a first prototype to explore running multiple commands in a single safepoint: >> >> http://cr.openjdk.java.net/~fparain/DCmdRunAtSafepoint/webrev.00/ >> >> The syntax is very simple, with the character ?+? used as the delimiter between commands. >> >> $ ./build/macosx-x64-debug/images/jdk/bin/jcmd 21164 VM.run_at_safepoint VM.version + VM.uptime >> 21164: >> Java HotSpot(TM) 64-Bit Server VM version 11-internal+0-2018-05-25-2202478.fparain.DCmd >> JDK 11.0.0 >> 44.143 s >> >> Playing a bit with it, it quickly shows what are the issues when trying to run >> diagnostic commands at a safepoint. >> >> Issue 1: One important point to remember is that only the VMThread can run at a safepoint, >> all JavaThreads are strictly forbidden to run during the safepoint. So executing commands at a >> safepoint means they will be executed by the VMThread. Unfortunately, the VMThread is >> subject to restrictions and cannot execute all code that a JavaThread can execute. This >> issue is visible with the VM.info command, which takes the Heap_lock. The VMThread >> is not allowed to grab this lock because it could cause the VMThread to be blocked by >> a JavaThread. So far, VM.info cannot be run at a safepoint. >> >> Issue 2: Executing a code at a safepoint requires a VMOperation. VMOperations are >> made of a prologue, a doit() method witch contains the code executed at the safepoint, >> and an epilogue. The prologue and the epilogue are executed by the requester of the >> VMOperation, usually a JavaThread, and they are not executed at a safepoint. Prologues >> and epilogues are designed to execute code the VMThread cannot execute, like taking >> locks or loading classes. When the VMThread itself wants to execute a VMOperation, >> what is called a nested VMOperation, it only executes the doit() method, skipping the >> prologue and the epilogue. This is an issue for diagnostic commands using a VMOp >> in their implementation: when these commands are called while the JVM is already >> at a safepoint, their prologue and epilogue cannot be executed. This issue impacts >> Thread.print and GC.run. >> >> There are probably other issues still to be found, so I encourage people to test >> this patch to see if the combinations of commands they would like to run at a >> safepoint can indeed be run in this mode. >> >> Note that the support for invoking the new command from the DiagnosticCommand >> MBean has not be implemented. This is a not negligible work in order to correctly >> check Java Permissions before executing the commands. >> >> Let me know if you consider this code useful or if the limitations described above >> make this feature not as pretty as we wanted it to be. >> >> Regards, >> >> Fred >> >> >>> On May 29, 2018, at 15:54, Frederic Parain wrote: >>> >>> Hi Thomas, >>> >>> >>>> On May 26, 2018, at 04:07, Thomas St?fe wrote: >>>> >>> >>>>> >>>>> I think that it is a fundamental piece of the design. Adding a new diagnostic >>>>> command to group several commands under a unique safepoint: >>>>> 1 - requires explicit action from the user (and acknowledgment of the risks) >>>>> 2 - doesn?t touch the current syntax/semantic at all >>>>> >>>>> Another reason I?m pushing for a specific diagnostic command rather than >>>>> changing the syntax is remote execution. jcmd is not the only way to invoke >>>>> a diagnostic command, they can also be invoke remotely using JMX and the >>>>> com.sun.management.DiagnosticCommand dynamic MBean. Through the >>>>> platform MBean, users do not have access to the jcmd syntax, they see >>>>> each diagnostic command as a MBean operation (providing that they have >>>>> been registered with the DCmd_Source_MBean flag). The initial support >>>>> for remote execution was taking a single String and processed it just like >>>>> a string passed by jcmd, but people developing consoles for remote >>>>> management complained that this was not convenient for them, and asked >>>>> for something they could use more programmatically. >>>>> If the safepoint bundling is implemented in the syntax, JMX clients won?t >>>>> be able to use the feature. If we implement it with a new diagnostic command, >>>>> it will be visible from the MBean and they will be able to use it. >>>> >>>> That is interesting. Do you have examples for such programs? I was >>>> looking for ways of how to use jcmd (or equivalent) remotely. >>> >>> If you?re looking for a console application to invoke diagnostic command >>> on a remote JVM, Mission Control has a dedicated support for that in the >>> ?Diagnostic Commands? tab. >>> If you?re looking for exemples of source code doing remote invocations, >>> there?re several unit tests for the DiagnosticCommand MBean in the directory >>> test/jdk/com/sun/management/DiagnosticCommandMBean. >>> >>>> >>>>> >>>>> At last, creating a specific diagnostic command for safepoint bundling would >>>>> show the way on how to group commands for consistency. Safepoints is not >>>>> the only way to get consistency. For some JVM internals reasons, you might >>>>> want to group commands under a single lock, like the Threads_lock. Adding >>>>> a diagnostic command for each type of bundling seems more extensible. >>>>> >>>> >>>> You make good points. >>>> >>>> Adding a new command has also the advantage that it does not affect >>>> downward compatibility and hence would not require a CSR. CSR seems to >>>> slow down things a bit (I have a very simple CSR for jcmd open >>>> undecided since 12 days: >>>> https://bugs.openjdk.java.net/browse/JDK-8203043). So, while I think >>>> CSR are a necessary process, I am happy to avoid them if possible. >>>> >>>> Lets spin this out a bit more: >>>> >>>> -> However the user specifies "please execute these commands at a >>>> single safepoint", if one of the enclosed commands cannot be run >>>> inside a safepoint the whole bundle should fail, agreed? So, >>>> specifying e.g. JFR.start as part of such a command bundle would cause >>>> the whole bundle to not being executed. >>> >>> Yes. The new command N would ask for the execution of a list of >>> commands { A, B, C, ? }, and would fail if: >>> - any of the specified commands { A, B, C, ? } is not a registered >>> command >>> - any of the specified commands { A, B, C, ? } doesn?t support >>> execution at safepoint >>> - any of the specified commands { A, B, C, ? } is not exported to >>> the DCmdSource used to invoke N (*) >>> >>> (*) Not all commands are exported to all interfaces (jcmd,JMX), for >>> instance, some commands related to the JMX agent are cannot >>> be invoked through the JMX. Even if the new command can be >>> invoked from all interfaces, it must not allow to circumvent this >>> restriction. This should not be an issue today, because all >>> JMX agent commands are calling back Java code, and are not >>> suitable for execution at a safepoint. But this would ensure the >>> consistency of the model in the long term (and the test is cheap). >>> >>>> >>>> -> If we introduce a command like you propose (lets call it "VM.run" >>>> for the sake of the argument) we still need to decide how to delimit >>>> the commands at the command line. That brings us back to the syntax >>>> question: separation by a explicit delimiter, if possible one that >>>> does not clash with common unix shells? >>>> >>>> Lets try colon ':' : >>>> >>>> jcmd VM.run VM.uptime : VM.metaspace show-loaders show-classes >>>> by-chunktype : VM.classloaders show-classes : VM.info >>>> >>>> Of course, we can wrap all arguments to jcmd in quotes, then the >>>> delimiter sign does not affect things as much, unless we use quotes :) >>>> We could e.g. use semicolon: >>>> >>>> jcmd ' VM.run VM.uptime ; VM.metaspace show-loaders show-classes >>>> by-chunktype ; VM.classloaders show-classes ; VM.info ? >>>> >>> >>> I thought about colon and semi-colon too, because they are the most >>> obvious delimiters. However, they are already as delimiters in classpaths >>> (semi-colon on Unix, colon on Windows), so parsing to find the right >>> delimiters would be tricky. >>> >>> I?d start with ?!? or ?+? in a first time, to make some progress on the other >>> aspects of the command, and buy some time to discuss the delimiter >>> issue in parallel. >>> >>> The last resort would be to force each command in the bundle to >>> be specified with quotes (arguments included). But it would make the >>> syntax very heavy, I?d prefer a simpler delimiter. >>> >>> >>>> Or, we go back to my original idea of letting the command keyword be >>>> the indicator for a new command start. Honestly, at this point I have >>>> no preference. Both approaches seem reasonable. >>>> >>>> The only niggle I have with an explicit command is the syntax, which I >>>> find a bit awkward. My thinking before reading your response was that >>>> I'd prefer a jcmd flag to specify the same behavior: that all commands >>>> should be executed at a single safepoint. E.g. [-s/--stacked] >>>> >>>> jcmd -s ' VM.uptime ; VM.metaspace show-loaders show-classes >>>> by-chunktype ; VM.classloaders show-classes ; VM.info ' >>>> >>>> But I see the disadvantage: depending on how it is implemented this >>>> would require consistent changes in both jcmd and hotspot, and not >>>> work with other clients using MBeans. Whereas the hypothetical >>>> "VM.run" would work out of the box for any old jcmd version and any >>>> other client. >>>> >>>> Of course one could keep the "VM.run" idea and still give jcmd a >>>> "-s/--stacked" option, as a shallow convenience feature, which just >>>> translates to "VM.run" and runs that. >>>> >>>> -> A variation of your "have special commands to run a command bundle >>>> in one particular context" theme could be having only one command, >>>> e.g. "VM.run", specifying the conditions under which to run the bundle >>>> as sub options. E.g. "VM.run -s " for "run commands at >>>> safepoint", "VM.run -t " for "run commands under >>>> Threads_lock". That would even allow to combine these contexts. I do >>>> not know if that makes sense. >>>> >>>> - Another advantage of having an explicit command just occurring to me >>>> is that it enables more evolved command files, where one can >>>> intersperse command bundles and single commands. For instance, I could >>>> give this to a customer: >>>> >>>> >>>> 1 VM.uptime >>>> 2 VM.run_commands VM.metaspace show-loaders show-classes by-chunktype >>>> VM.classloaders details VM.classloader_stats >>>> 3 GC.run >>>> 4 VM.run_commands VM.metaspace show-loaders show-classes by-chunktype >>>> VM.classloaders details VM.classloader_stats >>>> >>>> to execute a command bundle before and after triggering a GC. >>> >>> This would work fine . The parser of the jcmd string will split the >>> string based on ?\n? characters, then each line of command will be executed >>> sequentially, and lines containing the VM.run_command will then create the >>> safepoint and trigger the execution of the different commands specified in >>> its argument list. >>> >>>> >>>>> We have a three days week-end in the US, but I should be able to code >>>>> something next week. >>>>> >>>> >>>> Oh, you want to take over the implementation, sure! Fine by me. >>> >>> I?m not promising an implementation, I haven?t worked on this code >>> since I left the serviceability team. I need to code at least a skeleton >>> of the command to refresh my memory about all aspects of the >>> diagnostic command framework. >>> >>>> >>>>> Thank you for your ideas and your interest in diagnostic commands. >>>> >>>> Sure. Why I am interested in this: We strongly rely on tools similar >>>> to jcmd for our support. We have quite capable tools in our (licensed, >>>> non-open) VM port, which are similar to jcmd in many aspects. So we >>>> try to bring some of the capabilities upstream, in order to ease the >>>> merge costs for us and to help the OpenJDK community. >>> >>> The diagnostic command framework was designed to support an >>> extensible set of commands. Feel free to extend, and thank you >>> for contributing to the OpenJDK community. >>> >>> Fred >>> >>>>> >>>>> >>>>>> >>>>>>> >>>>>>> A new separator, more convenient than newline, would be require to have >>>>>>> a single line syntax. >>>>>>> >>>>>>> My 2 cents, >>>>>>> >>>>>> >>>>>> Certainly worth more than 2 cents :) Thanks for your thoughts! >>>>>> >>>>>> ..Thomas >>>>>> >>>>>>> Fred >>>>>>> >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> Erik >>>>>>>>> >>>>>>>>> Hi Thomas, >>>>>>>>> >>>>>>>>> very positive suggestion. >>>>>>>>> >>>>>>>>> One observation: >>>>>>>>> There already seems to be some different interpretation of the command >>>>>>>>> line >>>>>>>>> in different Java versions: >>>>>>>>> For instance when I try to dump a flight recording in 1.8.0_152-ea-b05 : >>>>>>>>> I >>>>>>>>> can use: >>>>>>>>> jcmd 33014 JFR.dump filename="recording1.jfr" recording=1 >>>>>>>>> but in build 9+181 , the same command must be: >>>>>>>>> jcmd 33014 JFR.dump filename="recording1.jfr",recording=1 >>>>>>>>> (the comma to separate sub-options you mentioned earlier) >>>>>>>>> >>>>>>>>> Therefore the suggestion without curly brackets, giving several commands >>>>>>>>> after one another seems good with regard to backwards compatibility. >>>>>>>>> >>>>>>>>> Motivation: hawt.io uses the MBean >>>>>>>>> com.sun.management:type=DiagnosticCommand >>>>>>>>> to access the same functionality as jcmd. Variations in option syntax >>>>>>>>> across >>>>>>>>> Java versions is already an issue (only affecting sub-options though, as >>>>>>>>> each command is a separate JMX operation). So syntax compatibility is >>>>>>>>> highly >>>>>>>>> appreciated :-) >>>>>>>>> >>>>>>>>> Martin >>>>>>>>> >>>>>>>>> l?r. 12. mai 2018 kl. 20:11 skrev Kirk Pepperdine >>>>>>>>> : >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On May 10, 2018, at 11:26 AM, Thomas St?fe >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> On Thu, May 10, 2018 at 9:13 AM, Kirk Pepperdine >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> The stacking at the safe point would be a huge win. Right now thread >>>>>>>>>>>> dump consistency can really confuse people as the tooling will show >>>>>>>>>>>> two >>>>>>>>>>>> threads owning the same lock at seemingly the same time. Reality, it?s >>>>>>>>>>>> just >>>>>>>>>>>> that people didn?t realize you get a safe point for each thread >>>>>>>>>>>> therefor >>>>>>>>>>>> there is motion in the system as you?re collecting the data. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I am a bit confused. What tooling are you talking about? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> jstack. I?ve not seen it with jcmd. But I often see 2 threads holding >>>>>>>>>> the >>>>>>>>>> same lock at the ?same time? which is of course non-sense. I can dig one >>>>>>>>>> up >>>>>>>>>> for you if you?re still confused. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> As an aside, it?s amazing how many dev?s aren?t aware of jcmd. Just >>>>>>>>>>>> yesterday after my session at Devoxx I had someone ask me about using >>>>>>>>>>>> jfr >>>>>>>>>>>> from the command line, many in that session had not seen jcmd before. >>>>>>>>>>>> The >>>>>>>>>>>> feedback was, wow, this is very useful and I wished I had of known >>>>>>>>>>>> about it >>>>>>>>>>>> earlier. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Yes, jcmd is quite useful. I also really like the simple design, which >>>>>>>>>>> is by default up- and downward compatible (so you can talk to any VM, >>>>>>>>>>> new, old, it should not matter). That is just good design. We - Sap - >>>>>>>>>>> work to extend jcmd, to help our support folks. E.g. the whole new >>>>>>>>>>> VM.metaspace command chain. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> And simple it is?.well done!!! >>>>>>>>>> >>>>>>>>>> ? Kirk >>>>>>>>>> >>>>>>>>> >>>>>>> >>>>> >>> >> From frederic.parain at oracle.com Mon Jun 18 20:52:49 2018 From: frederic.parain at oracle.com (Frederic Parain) Date: Mon, 18 Jun 2018 16:52:49 -0400 Subject: jcmd - executing multiple commands at the same safepoint? In-Reply-To: <739cb0e6-357c-75af-e752-a08bc9c90162@oracle.com> References: <3D40CD1F-1DD0-44A4-A729-3781B6E3EA7D@gmail.com> <69360DA5-9287-4C84-B5FE-89B240D9C413@gmail.com> <90E966D9-476B-46D9-B08A-7968F6740851@gmail.com> <3606226E-E365-4F9E-B6F3-B573DE1A5579@gmail.com> <5B0486CC.8050801@oracle.com> <937f5c14-327b-5cc5-897a-09aaef349610@oracle.com> <3D31B965-68BE-4CBD-8952-907EBDA71BCE@oracle.com> <09FA5DC6-5839-4E13-9F8A-CBB772AEA2A3@oracle.com> <739cb0e6-357c-75af-e752-a08bc9c90162@oracle.com> Message-ID: Hi David, Thank you for looking at this issue. The intend has never been to provide a command to easily crash the JVM :-) The patch was provided without any safety measures in purpose: to enable explorations in order to see how badly things could go south. The final design must include a per-command flag or method specifying if the command can be executed inside a safepoint. This information has to be provided by JVM developers, as there?s no automatic way to determine if a command is safepoint safe or not. So far, we already have the following rules to decide if a command can be executed at a safepoint or not: - it must not perform Java callbacks - it must not take locks - it must not invoke VMOp relying on the execution of their prologue and epilogue Let?s see: 1 - if there are more restrictions to consider 2 - what can be achieved with respect of these restrictions (which commands can run at a safepoint, which commands can easily be modified to comply with the restrictions above) Regards, Fred > On Jun 17, 2018, at 22:08, David Holmes wrote: > > Hi Fred, > > Thanks for this and the analysis below. > > My 2c is that it is far too risky to provide a general purpose facility like this where arbitrary command combinations can result in crashing the target VM! > > Certainly no command that requires a JavaThread should be allowed to be requested - only commands actually implemented via safepointing VMOps should be considered. And then we should perhaps constrain things further so that no commands that require prolog or epilog logic are accepted. > > Turning things around, perhaps we should flag which Dcmds, that operate at a safepoint, are considered "combinable" and only allow those commands to be specified? > > If we want a general purpose way of expressing a list of Dcmds to execute then perhaps it should be specified to process each command in turn, with a sequence of combinable safepoint ops processed at a single safepoint? > > Thanks, > David > > On 16/06/2018 1:32 AM, Frederic Parain wrote: >> Hi, >> Here?s a first prototype to explore running multiple commands in a single safepoint: >> http://cr.openjdk.java.net/~fparain/DCmdRunAtSafepoint/webrev.00/ >> The syntax is very simple, with the character ?+? used as the delimiter between commands. >> $ ./build/macosx-x64-debug/images/jdk/bin/jcmd 21164 VM.run_at_safepoint VM.version + VM.uptime >> 21164: >> Java HotSpot(TM) 64-Bit Server VM version 11-internal+0-2018-05-25-2202478.fparain.DCmd >> JDK 11.0.0 >> 44.143 s >> Playing a bit with it, it quickly shows what are the issues when trying to run >> diagnostic commands at a safepoint. >> Issue 1: One important point to remember is that only the VMThread can run at a safepoint, >> all JavaThreads are strictly forbidden to run during the safepoint. So executing commands at a >> safepoint means they will be executed by the VMThread. Unfortunately, the VMThread is >> subject to restrictions and cannot execute all code that a JavaThread can execute. This >> issue is visible with the VM.info command, which takes the Heap_lock. The VMThread >> is not allowed to grab this lock because it could cause the VMThread to be blocked by >> a JavaThread. So far, VM.info cannot be run at a safepoint. >> Issue 2: Executing a code at a safepoint requires a VMOperation. VMOperations are >> made of a prologue, a doit() method witch contains the code executed at the safepoint, >> and an epilogue. The prologue and the epilogue are executed by the requester of the >> VMOperation, usually a JavaThread, and they are not executed at a safepoint. Prologues >> and epilogues are designed to execute code the VMThread cannot execute, like taking >> locks or loading classes. When the VMThread itself wants to execute a VMOperation, >> what is called a nested VMOperation, it only executes the doit() method, skipping the >> prologue and the epilogue. This is an issue for diagnostic commands using a VMOp >> in their implementation: when these commands are called while the JVM is already >> at a safepoint, their prologue and epilogue cannot be executed. This issue impacts >> Thread.print and GC.run. >> There are probably other issues still to be found, so I encourage people to test >> this patch to see if the combinations of commands they would like to run at a >> safepoint can indeed be run in this mode. >> Note that the support for invoking the new command from the DiagnosticCommand >> MBean has not be implemented. This is a not negligible work in order to correctly >> check Java Permissions before executing the commands. >> Let me know if you consider this code useful or if the limitations described above >> make this feature not as pretty as we wanted it to be. >> Regards, >> Fred >>> On May 29, 2018, at 15:54, Frederic Parain wrote: >>> >>> Hi Thomas, >>> >>> >>>> On May 26, 2018, at 04:07, Thomas St?fe wrote: >>>> >>> >>>>> >>>>> I think that it is a fundamental piece of the design. Adding a new diagnostic >>>>> command to group several commands under a unique safepoint: >>>>> 1 - requires explicit action from the user (and acknowledgment of the risks) >>>>> 2 - doesn?t touch the current syntax/semantic at all >>>>> >>>>> Another reason I?m pushing for a specific diagnostic command rather than >>>>> changing the syntax is remote execution. jcmd is not the only way to invoke >>>>> a diagnostic command, they can also be invoke remotely using JMX and the >>>>> com.sun.management.DiagnosticCommand dynamic MBean. Through the >>>>> platform MBean, users do not have access to the jcmd syntax, they see >>>>> each diagnostic command as a MBean operation (providing that they have >>>>> been registered with the DCmd_Source_MBean flag). The initial support >>>>> for remote execution was taking a single String and processed it just like >>>>> a string passed by jcmd, but people developing consoles for remote >>>>> management complained that this was not convenient for them, and asked >>>>> for something they could use more programmatically. >>>>> If the safepoint bundling is implemented in the syntax, JMX clients won?t >>>>> be able to use the feature. If we implement it with a new diagnostic command, >>>>> it will be visible from the MBean and they will be able to use it. >>>> >>>> That is interesting. Do you have examples for such programs? I was >>>> looking for ways of how to use jcmd (or equivalent) remotely. >>> >>> If you?re looking for a console application to invoke diagnostic command >>> on a remote JVM, Mission Control has a dedicated support for that in the >>> ?Diagnostic Commands? tab. >>> If you?re looking for exemples of source code doing remote invocations, >>> there?re several unit tests for the DiagnosticCommand MBean in the directory >>> test/jdk/com/sun/management/DiagnosticCommandMBean. >>> >>>> >>>>> >>>>> At last, creating a specific diagnostic command for safepoint bundling would >>>>> show the way on how to group commands for consistency. Safepoints is not >>>>> the only way to get consistency. For some JVM internals reasons, you might >>>>> want to group commands under a single lock, like the Threads_lock. Adding >>>>> a diagnostic command for each type of bundling seems more extensible. >>>>> >>>> >>>> You make good points. >>>> >>>> Adding a new command has also the advantage that it does not affect >>>> downward compatibility and hence would not require a CSR. CSR seems to >>>> slow down things a bit (I have a very simple CSR for jcmd open >>>> undecided since 12 days: >>>> https://bugs.openjdk.java.net/browse/JDK-8203043). So, while I think >>>> CSR are a necessary process, I am happy to avoid them if possible. >>>> >>>> Lets spin this out a bit more: >>>> >>>> -> However the user specifies "please execute these commands at a >>>> single safepoint", if one of the enclosed commands cannot be run >>>> inside a safepoint the whole bundle should fail, agreed? So, >>>> specifying e.g. JFR.start as part of such a command bundle would cause >>>> the whole bundle to not being executed. >>> >>> Yes. The new command N would ask for the execution of a list of >>> commands { A, B, C, ? }, and would fail if: >>> - any of the specified commands { A, B, C, ? } is not a registered >>> command >>> - any of the specified commands { A, B, C, ? } doesn?t support >>> execution at safepoint >>> - any of the specified commands { A, B, C, ? } is not exported to >>> the DCmdSource used to invoke N (*) >>> >>> (*) Not all commands are exported to all interfaces (jcmd,JMX), for >>> instance, some commands related to the JMX agent are cannot >>> be invoked through the JMX. Even if the new command can be >>> invoked from all interfaces, it must not allow to circumvent this >>> restriction. This should not be an issue today, because all >>> JMX agent commands are calling back Java code, and are not >>> suitable for execution at a safepoint. But this would ensure the >>> consistency of the model in the long term (and the test is cheap). >>> >>>> >>>> -> If we introduce a command like you propose (lets call it "VM.run" >>>> for the sake of the argument) we still need to decide how to delimit >>>> the commands at the command line. That brings us back to the syntax >>>> question: separation by a explicit delimiter, if possible one that >>>> does not clash with common unix shells? >>>> >>>> Lets try colon ':' : >>>> >>>> jcmd VM.run VM.uptime : VM.metaspace show-loaders show-classes >>>> by-chunktype : VM.classloaders show-classes : VM.info >>>> >>>> Of course, we can wrap all arguments to jcmd in quotes, then the >>>> delimiter sign does not affect things as much, unless we use quotes :) >>>> We could e.g. use semicolon: >>>> >>>> jcmd ' VM.run VM.uptime ; VM.metaspace show-loaders show-classes >>>> by-chunktype ; VM.classloaders show-classes ; VM.info ? >>>> >>> >>> I thought about colon and semi-colon too, because they are the most >>> obvious delimiters. However, they are already as delimiters in classpaths >>> (semi-colon on Unix, colon on Windows), so parsing to find the right >>> delimiters would be tricky. >>> >>> I?d start with ?!? or ?+? in a first time, to make some progress on the other >>> aspects of the command, and buy some time to discuss the delimiter >>> issue in parallel. >>> >>> The last resort would be to force each command in the bundle to >>> be specified with quotes (arguments included). But it would make the >>> syntax very heavy, I?d prefer a simpler delimiter. >>> >>> >>>> Or, we go back to my original idea of letting the command keyword be >>>> the indicator for a new command start. Honestly, at this point I have >>>> no preference. Both approaches seem reasonable. >>>> >>>> The only niggle I have with an explicit command is the syntax, which I >>>> find a bit awkward. My thinking before reading your response was that >>>> I'd prefer a jcmd flag to specify the same behavior: that all commands >>>> should be executed at a single safepoint. E.g. [-s/--stacked] >>>> >>>> jcmd -s ' VM.uptime ; VM.metaspace show-loaders show-classes >>>> by-chunktype ; VM.classloaders show-classes ; VM.info ' >>>> >>>> But I see the disadvantage: depending on how it is implemented this >>>> would require consistent changes in both jcmd and hotspot, and not >>>> work with other clients using MBeans. Whereas the hypothetical >>>> "VM.run" would work out of the box for any old jcmd version and any >>>> other client. >>>> >>>> Of course one could keep the "VM.run" idea and still give jcmd a >>>> "-s/--stacked" option, as a shallow convenience feature, which just >>>> translates to "VM.run" and runs that. >>>> >>>> -> A variation of your "have special commands to run a command bundle >>>> in one particular context" theme could be having only one command, >>>> e.g. "VM.run", specifying the conditions under which to run the bundle >>>> as sub options. E.g. "VM.run -s " for "run commands at >>>> safepoint", "VM.run -t " for "run commands under >>>> Threads_lock". That would even allow to combine these contexts. I do >>>> not know if that makes sense. >>>> >>>> - Another advantage of having an explicit command just occurring to me >>>> is that it enables more evolved command files, where one can >>>> intersperse command bundles and single commands. For instance, I could >>>> give this to a customer: >>>> >>>> >>>> 1 VM.uptime >>>> 2 VM.run_commands VM.metaspace show-loaders show-classes by-chunktype >>>> VM.classloaders details VM.classloader_stats >>>> 3 GC.run >>>> 4 VM.run_commands VM.metaspace show-loaders show-classes by-chunktype >>>> VM.classloaders details VM.classloader_stats >>>> >>>> to execute a command bundle before and after triggering a GC. >>> >>> This would work fine . The parser of the jcmd string will split the >>> string based on ?\n? characters, then each line of command will be executed >>> sequentially, and lines containing the VM.run_command will then create the >>> safepoint and trigger the execution of the different commands specified in >>> its argument list. >>> >>>> >>>>> We have a three days week-end in the US, but I should be able to code >>>>> something next week. >>>>> >>>> >>>> Oh, you want to take over the implementation, sure! Fine by me. >>> >>> I?m not promising an implementation, I haven?t worked on this code >>> since I left the serviceability team. I need to code at least a skeleton >>> of the command to refresh my memory about all aspects of the >>> diagnostic command framework. >>> >>>> >>>>> Thank you for your ideas and your interest in diagnostic commands. >>>> >>>> Sure. Why I am interested in this: We strongly rely on tools similar >>>> to jcmd for our support. We have quite capable tools in our (licensed, >>>> non-open) VM port, which are similar to jcmd in many aspects. So we >>>> try to bring some of the capabilities upstream, in order to ease the >>>> merge costs for us and to help the OpenJDK community. >>> >>> The diagnostic command framework was designed to support an >>> extensible set of commands. Feel free to extend, and thank you >>> for contributing to the OpenJDK community. >>> >>> Fred >>> >>>>> >>>>> >>>>>> >>>>>>> >>>>>>> A new separator, more convenient than newline, would be require to have >>>>>>> a single line syntax. >>>>>>> >>>>>>> My 2 cents, >>>>>>> >>>>>> >>>>>> Certainly worth more than 2 cents :) Thanks for your thoughts! >>>>>> >>>>>> ..Thomas >>>>>> >>>>>>> Fred >>>>>>> >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> Erik >>>>>>>>> >>>>>>>>> Hi Thomas, >>>>>>>>> >>>>>>>>> very positive suggestion. >>>>>>>>> >>>>>>>>> One observation: >>>>>>>>> There already seems to be some different interpretation of the command >>>>>>>>> line >>>>>>>>> in different Java versions: >>>>>>>>> For instance when I try to dump a flight recording in 1.8.0_152-ea-b05 : >>>>>>>>> I >>>>>>>>> can use: >>>>>>>>> jcmd 33014 JFR.dump filename="recording1.jfr" recording=1 >>>>>>>>> but in build 9+181 , the same command must be: >>>>>>>>> jcmd 33014 JFR.dump filename="recording1.jfr",recording=1 >>>>>>>>> (the comma to separate sub-options you mentioned earlier) >>>>>>>>> >>>>>>>>> Therefore the suggestion without curly brackets, giving several commands >>>>>>>>> after one another seems good with regard to backwards compatibility. >>>>>>>>> >>>>>>>>> Motivation: hawt.io uses the MBean >>>>>>>>> com.sun.management:type=DiagnosticCommand >>>>>>>>> to access the same functionality as jcmd. Variations in option syntax >>>>>>>>> across >>>>>>>>> Java versions is already an issue (only affecting sub-options though, as >>>>>>>>> each command is a separate JMX operation). So syntax compatibility is >>>>>>>>> highly >>>>>>>>> appreciated :-) >>>>>>>>> >>>>>>>>> Martin >>>>>>>>> >>>>>>>>> l?r. 12. mai 2018 kl. 20:11 skrev Kirk Pepperdine >>>>>>>>> : >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On May 10, 2018, at 11:26 AM, Thomas St?fe >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> On Thu, May 10, 2018 at 9:13 AM, Kirk Pepperdine >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> The stacking at the safe point would be a huge win. Right now thread >>>>>>>>>>>> dump consistency can really confuse people as the tooling will show >>>>>>>>>>>> two >>>>>>>>>>>> threads owning the same lock at seemingly the same time. Reality, it?s >>>>>>>>>>>> just >>>>>>>>>>>> that people didn?t realize you get a safe point for each thread >>>>>>>>>>>> therefor >>>>>>>>>>>> there is motion in the system as you?re collecting the data. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I am a bit confused. What tooling are you talking about? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> jstack. I?ve not seen it with jcmd. But I often see 2 threads holding >>>>>>>>>> the >>>>>>>>>> same lock at the ?same time? which is of course non-sense. I can dig one >>>>>>>>>> up >>>>>>>>>> for you if you?re still confused. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> As an aside, it?s amazing how many dev?s aren?t aware of jcmd. Just >>>>>>>>>>>> yesterday after my session at Devoxx I had someone ask me about using >>>>>>>>>>>> jfr >>>>>>>>>>>> from the command line, many in that session had not seen jcmd before. >>>>>>>>>>>> The >>>>>>>>>>>> feedback was, wow, this is very useful and I wished I had of known >>>>>>>>>>>> about it >>>>>>>>>>>> earlier. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Yes, jcmd is quite useful. I also really like the simple design, which >>>>>>>>>>> is by default up- and downward compatible (so you can talk to any VM, >>>>>>>>>>> new, old, it should not matter). That is just good design. We - Sap - >>>>>>>>>>> work to extend jcmd, to help our support folks. E.g. the whole new >>>>>>>>>>> VM.metaspace command chain. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> And simple it is?.well done!!! >>>>>>>>>> >>>>>>>>>> ? Kirk >>>>>>>>>> >>>>>>>>> >>>>>>> >>>>> >>> From david.holmes at oracle.com Tue Jun 19 02:23:52 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 19 Jun 2018 12:23:52 +1000 Subject: jcmd - executing multiple commands at the same safepoint? In-Reply-To: References: <69360DA5-9287-4C84-B5FE-89B240D9C413@gmail.com> <90E966D9-476B-46D9-B08A-7968F6740851@gmail.com> <3606226E-E365-4F9E-B6F3-B573DE1A5579@gmail.com> <5B0486CC.8050801@oracle.com> <937f5c14-327b-5cc5-897a-09aaef349610@oracle.com> <3D31B965-68BE-4CBD-8952-907EBDA71BCE@oracle.com> <09FA5DC6-5839-4E13-9F8A-CBB772AEA2A3@oracle.com> <739cb0e6-357c-75af-e752-a08bc9c90162@oracle.com> Message-ID: Hi Fred, On 19/06/2018 6:52 AM, Frederic Parain wrote: > Hi David, > > Thank you for looking at this issue. > > The intend has never been to provide a command to easily crash the JVM :-) > The patch was provided without any safety measures in purpose: to enable > explorations in order to see how badly things could go south. > > The final design must include a per-command flag or method specifying if > the command can be executed inside a safepoint. This information has to > be provided by JVM developers, as there?s no automatic way to determine if > a command is safepoint safe or not. > > So far, we already have the following rules to decide if a command can be > executed at a safepoint or not: > - it must not perform Java callbacks > - it must not take locks > - it must not invoke VMOp relying on the execution of their prologue and > epilogue > > Let?s see: > 1 - if there are more restrictions to consider > 2 - what can be achieved with respect of these restrictions (which commands > can run at a safepoint, which commands can easily be modified to comply > with the restrictions above) Doesn't each Dcmd already explicitly get implemented as either a VMOp or not? I would not be concerned with trying to run non-VMop Dcmds at a safepoint - what does that buy us? The issue as I understood it was the problem of requiring a series of safepoints for a series of safepoint-needing commands. Or are there useful sequences of interleaving VMop and non-VMop actions that would benefit from this? Cheers, David > Regards, > > Fred > > >> On Jun 17, 2018, at 22:08, David Holmes wrote: >> >> Hi Fred, >> >> Thanks for this and the analysis below. >> >> My 2c is that it is far too risky to provide a general purpose facility like this where arbitrary command combinations can result in crashing the target VM! >> >> Certainly no command that requires a JavaThread should be allowed to be requested - only commands actually implemented via safepointing VMOps should be considered. And then we should perhaps constrain things further so that no commands that require prolog or epilog logic are accepted. >> >> Turning things around, perhaps we should flag which Dcmds, that operate at a safepoint, are considered "combinable" and only allow those commands to be specified? >> >> If we want a general purpose way of expressing a list of Dcmds to execute then perhaps it should be specified to process each command in turn, with a sequence of combinable safepoint ops processed at a single safepoint? >> >> Thanks, >> David >> >> On 16/06/2018 1:32 AM, Frederic Parain wrote: >>> Hi, >>> Here?s a first prototype to explore running multiple commands in a single safepoint: >>> http://cr.openjdk.java.net/~fparain/DCmdRunAtSafepoint/webrev.00/ >>> The syntax is very simple, with the character ?+? used as the delimiter between commands. >>> $ ./build/macosx-x64-debug/images/jdk/bin/jcmd 21164 VM.run_at_safepoint VM.version + VM.uptime >>> 21164: >>> Java HotSpot(TM) 64-Bit Server VM version 11-internal+0-2018-05-25-2202478.fparain.DCmd >>> JDK 11.0.0 >>> 44.143 s >>> Playing a bit with it, it quickly shows what are the issues when trying to run >>> diagnostic commands at a safepoint. >>> Issue 1: One important point to remember is that only the VMThread can run at a safepoint, >>> all JavaThreads are strictly forbidden to run during the safepoint. So executing commands at a >>> safepoint means they will be executed by the VMThread. Unfortunately, the VMThread is >>> subject to restrictions and cannot execute all code that a JavaThread can execute. This >>> issue is visible with the VM.info command, which takes the Heap_lock. The VMThread >>> is not allowed to grab this lock because it could cause the VMThread to be blocked by >>> a JavaThread. So far, VM.info cannot be run at a safepoint. >>> Issue 2: Executing a code at a safepoint requires a VMOperation. VMOperations are >>> made of a prologue, a doit() method witch contains the code executed at the safepoint, >>> and an epilogue. The prologue and the epilogue are executed by the requester of the >>> VMOperation, usually a JavaThread, and they are not executed at a safepoint. Prologues >>> and epilogues are designed to execute code the VMThread cannot execute, like taking >>> locks or loading classes. When the VMThread itself wants to execute a VMOperation, >>> what is called a nested VMOperation, it only executes the doit() method, skipping the >>> prologue and the epilogue. This is an issue for diagnostic commands using a VMOp >>> in their implementation: when these commands are called while the JVM is already >>> at a safepoint, their prologue and epilogue cannot be executed. This issue impacts >>> Thread.print and GC.run. >>> There are probably other issues still to be found, so I encourage people to test >>> this patch to see if the combinations of commands they would like to run at a >>> safepoint can indeed be run in this mode. >>> Note that the support for invoking the new command from the DiagnosticCommand >>> MBean has not be implemented. This is a not negligible work in order to correctly >>> check Java Permissions before executing the commands. >>> Let me know if you consider this code useful or if the limitations described above >>> make this feature not as pretty as we wanted it to be. >>> Regards, >>> Fred >>>> On May 29, 2018, at 15:54, Frederic Parain wrote: >>>> >>>> Hi Thomas, >>>> >>>> >>>>> On May 26, 2018, at 04:07, Thomas St?fe wrote: >>>>> >>>> >>>>>> >>>>>> I think that it is a fundamental piece of the design. Adding a new diagnostic >>>>>> command to group several commands under a unique safepoint: >>>>>> 1 - requires explicit action from the user (and acknowledgment of the risks) >>>>>> 2 - doesn?t touch the current syntax/semantic at all >>>>>> >>>>>> Another reason I?m pushing for a specific diagnostic command rather than >>>>>> changing the syntax is remote execution. jcmd is not the only way to invoke >>>>>> a diagnostic command, they can also be invoke remotely using JMX and the >>>>>> com.sun.management.DiagnosticCommand dynamic MBean. Through the >>>>>> platform MBean, users do not have access to the jcmd syntax, they see >>>>>> each diagnostic command as a MBean operation (providing that they have >>>>>> been registered with the DCmd_Source_MBean flag). The initial support >>>>>> for remote execution was taking a single String and processed it just like >>>>>> a string passed by jcmd, but people developing consoles for remote >>>>>> management complained that this was not convenient for them, and asked >>>>>> for something they could use more programmatically. >>>>>> If the safepoint bundling is implemented in the syntax, JMX clients won?t >>>>>> be able to use the feature. If we implement it with a new diagnostic command, >>>>>> it will be visible from the MBean and they will be able to use it. >>>>> >>>>> That is interesting. Do you have examples for such programs? I was >>>>> looking for ways of how to use jcmd (or equivalent) remotely. >>>> >>>> If you?re looking for a console application to invoke diagnostic command >>>> on a remote JVM, Mission Control has a dedicated support for that in the >>>> ?Diagnostic Commands? tab. >>>> If you?re looking for exemples of source code doing remote invocations, >>>> there?re several unit tests for the DiagnosticCommand MBean in the directory >>>> test/jdk/com/sun/management/DiagnosticCommandMBean. >>>> >>>>> >>>>>> >>>>>> At last, creating a specific diagnostic command for safepoint bundling would >>>>>> show the way on how to group commands for consistency. Safepoints is not >>>>>> the only way to get consistency. For some JVM internals reasons, you might >>>>>> want to group commands under a single lock, like the Threads_lock. Adding >>>>>> a diagnostic command for each type of bundling seems more extensible. >>>>>> >>>>> >>>>> You make good points. >>>>> >>>>> Adding a new command has also the advantage that it does not affect >>>>> downward compatibility and hence would not require a CSR. CSR seems to >>>>> slow down things a bit (I have a very simple CSR for jcmd open >>>>> undecided since 12 days: >>>>> https://bugs.openjdk.java.net/browse/JDK-8203043). So, while I think >>>>> CSR are a necessary process, I am happy to avoid them if possible. >>>>> >>>>> Lets spin this out a bit more: >>>>> >>>>> -> However the user specifies "please execute these commands at a >>>>> single safepoint", if one of the enclosed commands cannot be run >>>>> inside a safepoint the whole bundle should fail, agreed? So, >>>>> specifying e.g. JFR.start as part of such a command bundle would cause >>>>> the whole bundle to not being executed. >>>> >>>> Yes. The new command N would ask for the execution of a list of >>>> commands { A, B, C, ? }, and would fail if: >>>> - any of the specified commands { A, B, C, ? } is not a registered >>>> command >>>> - any of the specified commands { A, B, C, ? } doesn?t support >>>> execution at safepoint >>>> - any of the specified commands { A, B, C, ? } is not exported to >>>> the DCmdSource used to invoke N (*) >>>> >>>> (*) Not all commands are exported to all interfaces (jcmd,JMX), for >>>> instance, some commands related to the JMX agent are cannot >>>> be invoked through the JMX. Even if the new command can be >>>> invoked from all interfaces, it must not allow to circumvent this >>>> restriction. This should not be an issue today, because all >>>> JMX agent commands are calling back Java code, and are not >>>> suitable for execution at a safepoint. But this would ensure the >>>> consistency of the model in the long term (and the test is cheap). >>>> >>>>> >>>>> -> If we introduce a command like you propose (lets call it "VM.run" >>>>> for the sake of the argument) we still need to decide how to delimit >>>>> the commands at the command line. That brings us back to the syntax >>>>> question: separation by a explicit delimiter, if possible one that >>>>> does not clash with common unix shells? >>>>> >>>>> Lets try colon ':' : >>>>> >>>>> jcmd VM.run VM.uptime : VM.metaspace show-loaders show-classes >>>>> by-chunktype : VM.classloaders show-classes : VM.info >>>>> >>>>> Of course, we can wrap all arguments to jcmd in quotes, then the >>>>> delimiter sign does not affect things as much, unless we use quotes :) >>>>> We could e.g. use semicolon: >>>>> >>>>> jcmd ' VM.run VM.uptime ; VM.metaspace show-loaders show-classes >>>>> by-chunktype ; VM.classloaders show-classes ; VM.info ? >>>>> >>>> >>>> I thought about colon and semi-colon too, because they are the most >>>> obvious delimiters. However, they are already as delimiters in classpaths >>>> (semi-colon on Unix, colon on Windows), so parsing to find the right >>>> delimiters would be tricky. >>>> >>>> I?d start with ?!? or ?+? in a first time, to make some progress on the other >>>> aspects of the command, and buy some time to discuss the delimiter >>>> issue in parallel. >>>> >>>> The last resort would be to force each command in the bundle to >>>> be specified with quotes (arguments included). But it would make the >>>> syntax very heavy, I?d prefer a simpler delimiter. >>>> >>>> >>>>> Or, we go back to my original idea of letting the command keyword be >>>>> the indicator for a new command start. Honestly, at this point I have >>>>> no preference. Both approaches seem reasonable. >>>>> >>>>> The only niggle I have with an explicit command is the syntax, which I >>>>> find a bit awkward. My thinking before reading your response was that >>>>> I'd prefer a jcmd flag to specify the same behavior: that all commands >>>>> should be executed at a single safepoint. E.g. [-s/--stacked] >>>>> >>>>> jcmd -s ' VM.uptime ; VM.metaspace show-loaders show-classes >>>>> by-chunktype ; VM.classloaders show-classes ; VM.info ' >>>>> >>>>> But I see the disadvantage: depending on how it is implemented this >>>>> would require consistent changes in both jcmd and hotspot, and not >>>>> work with other clients using MBeans. Whereas the hypothetical >>>>> "VM.run" would work out of the box for any old jcmd version and any >>>>> other client. >>>>> >>>>> Of course one could keep the "VM.run" idea and still give jcmd a >>>>> "-s/--stacked" option, as a shallow convenience feature, which just >>>>> translates to "VM.run" and runs that. >>>>> >>>>> -> A variation of your "have special commands to run a command bundle >>>>> in one particular context" theme could be having only one command, >>>>> e.g. "VM.run", specifying the conditions under which to run the bundle >>>>> as sub options. E.g. "VM.run -s " for "run commands at >>>>> safepoint", "VM.run -t " for "run commands under >>>>> Threads_lock". That would even allow to combine these contexts. I do >>>>> not know if that makes sense. >>>>> >>>>> - Another advantage of having an explicit command just occurring to me >>>>> is that it enables more evolved command files, where one can >>>>> intersperse command bundles and single commands. For instance, I could >>>>> give this to a customer: >>>>> >>>>> >>>>> 1 VM.uptime >>>>> 2 VM.run_commands VM.metaspace show-loaders show-classes by-chunktype >>>>> VM.classloaders details VM.classloader_stats >>>>> 3 GC.run >>>>> 4 VM.run_commands VM.metaspace show-loaders show-classes by-chunktype >>>>> VM.classloaders details VM.classloader_stats >>>>> >>>>> to execute a command bundle before and after triggering a GC. >>>> >>>> This would work fine . The parser of the jcmd string will split the >>>> string based on ?\n? characters, then each line of command will be executed >>>> sequentially, and lines containing the VM.run_command will then create the >>>> safepoint and trigger the execution of the different commands specified in >>>> its argument list. >>>> >>>>> >>>>>> We have a three days week-end in the US, but I should be able to code >>>>>> something next week. >>>>>> >>>>> >>>>> Oh, you want to take over the implementation, sure! Fine by me. >>>> >>>> I?m not promising an implementation, I haven?t worked on this code >>>> since I left the serviceability team. I need to code at least a skeleton >>>> of the command to refresh my memory about all aspects of the >>>> diagnostic command framework. >>>> >>>>> >>>>>> Thank you for your ideas and your interest in diagnostic commands. >>>>> >>>>> Sure. Why I am interested in this: We strongly rely on tools similar >>>>> to jcmd for our support. We have quite capable tools in our (licensed, >>>>> non-open) VM port, which are similar to jcmd in many aspects. So we >>>>> try to bring some of the capabilities upstream, in order to ease the >>>>> merge costs for us and to help the OpenJDK community. >>>> >>>> The diagnostic command framework was designed to support an >>>> extensible set of commands. Feel free to extend, and thank you >>>> for contributing to the OpenJDK community. >>>> >>>> Fred >>>> >>>>>> >>>>>> >>>>>>> >>>>>>>> >>>>>>>> A new separator, more convenient than newline, would be require to have >>>>>>>> a single line syntax. >>>>>>>> >>>>>>>> My 2 cents, >>>>>>>> >>>>>>> >>>>>>> Certainly worth more than 2 cents :) Thanks for your thoughts! >>>>>>> >>>>>>> ..Thomas >>>>>>> >>>>>>>> Fred >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Erik >>>>>>>>>> >>>>>>>>>> Hi Thomas, >>>>>>>>>> >>>>>>>>>> very positive suggestion. >>>>>>>>>> >>>>>>>>>> One observation: >>>>>>>>>> There already seems to be some different interpretation of the command >>>>>>>>>> line >>>>>>>>>> in different Java versions: >>>>>>>>>> For instance when I try to dump a flight recording in 1.8.0_152-ea-b05 : >>>>>>>>>> I >>>>>>>>>> can use: >>>>>>>>>> jcmd 33014 JFR.dump filename="recording1.jfr" recording=1 >>>>>>>>>> but in build 9+181 , the same command must be: >>>>>>>>>> jcmd 33014 JFR.dump filename="recording1.jfr",recording=1 >>>>>>>>>> (the comma to separate sub-options you mentioned earlier) >>>>>>>>>> >>>>>>>>>> Therefore the suggestion without curly brackets, giving several commands >>>>>>>>>> after one another seems good with regard to backwards compatibility. >>>>>>>>>> >>>>>>>>>> Motivation: hawt.io uses the MBean >>>>>>>>>> com.sun.management:type=DiagnosticCommand >>>>>>>>>> to access the same functionality as jcmd. Variations in option syntax >>>>>>>>>> across >>>>>>>>>> Java versions is already an issue (only affecting sub-options though, as >>>>>>>>>> each command is a separate JMX operation). So syntax compatibility is >>>>>>>>>> highly >>>>>>>>>> appreciated :-) >>>>>>>>>> >>>>>>>>>> Martin >>>>>>>>>> >>>>>>>>>> l?r. 12. mai 2018 kl. 20:11 skrev Kirk Pepperdine >>>>>>>>>> : >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> On May 10, 2018, at 11:26 AM, Thomas St?fe >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> On Thu, May 10, 2018 at 9:13 AM, Kirk Pepperdine >>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> The stacking at the safe point would be a huge win. Right now thread >>>>>>>>>>>>> dump consistency can really confuse people as the tooling will show >>>>>>>>>>>>> two >>>>>>>>>>>>> threads owning the same lock at seemingly the same time. Reality, it?s >>>>>>>>>>>>> just >>>>>>>>>>>>> that people didn?t realize you get a safe point for each thread >>>>>>>>>>>>> therefor >>>>>>>>>>>>> there is motion in the system as you?re collecting the data. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I am a bit confused. What tooling are you talking about? >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> jstack. I?ve not seen it with jcmd. But I often see 2 threads holding >>>>>>>>>>> the >>>>>>>>>>> same lock at the ?same time? which is of course non-sense. I can dig one >>>>>>>>>>> up >>>>>>>>>>> for you if you?re still confused. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> As an aside, it?s amazing how many dev?s aren?t aware of jcmd. Just >>>>>>>>>>>>> yesterday after my session at Devoxx I had someone ask me about using >>>>>>>>>>>>> jfr >>>>>>>>>>>>> from the command line, many in that session had not seen jcmd before. >>>>>>>>>>>>> The >>>>>>>>>>>>> feedback was, wow, this is very useful and I wished I had of known >>>>>>>>>>>>> about it >>>>>>>>>>>>> earlier. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Yes, jcmd is quite useful. I also really like the simple design, which >>>>>>>>>>>> is by default up- and downward compatible (so you can talk to any VM, >>>>>>>>>>>> new, old, it should not matter). That is just good design. We - Sap - >>>>>>>>>>>> work to extend jcmd, to help our support folks. E.g. the whole new >>>>>>>>>>>> VM.metaspace command chain. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> And simple it is?.well done!!! >>>>>>>>>>> >>>>>>>>>>> ? Kirk >>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>> >>>> > From david.holmes at oracle.com Tue Jun 19 04:36:00 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 19 Jun 2018 14:36:00 +1000 Subject: RFR 8205113: Update JVMTI doc references to object allocation tracking In-Reply-To: References: <384783b9-9ec2-093a-2662-a7a817849720@oracle.com> Message-ID: <3d01255e-8712-46aa-7659-4ccd7325f81c@oracle.com> On 19/06/2018 4:50 AM, Jeremy Manson wrote: > Yup!? The paragraph meanders a bit.? How about something like: I'm not sure some of the change quite works. The original text considers there to be three kinds of methods that can cause allocation when executed: - Java (bytecode) methods - JNI methods - VM methods but you've turned this into three kinds of allocation: via bytecode, via JNI, and via the VM. You then refer to "triggering" an allocation when we tend to use triggering for events. You also refer to an allocation being "executed directly by the VM" (a phrase previously applied when the subject was a 'method') - but you don't really execute allocations. IIUC the problem with the existing text is just that it considers VM allocation events as being undetectable other than by this "VM object allocation event" - but that's no longer true. So how about something minimally changed like this: --- Sent when a method causes the virtual machine to directly allocate an Object visible to Java programming language code. Generally object allocation can be detected by instrumenting the bytecodes of allocating methods. Object allocation generated in native code by JNI function calls can be detected using JNI function interception. Some methods might not have associated bytecodes and are not native methods, they instead are executed directly by the VM. These methods should send this event. Virtual machines which are incapable of bytecode instrumentation for some or all of their methods can send this event. Note that the SampledObjectAlloc event is triggered on all Java object allocations, including those caused by bytecode method execution, JNI method execution, and directly by VM methods. --- Thanks, David > Sent when the virtual machine allocates an > Object visible to Java programming language code without using a > new bytecode variant or a JNI method. > Many approaches to tracking object allocation use a combination of > bytecode-based instrumentation and JNI > function > interception to intercept allocations.? However, this > approach can leave a number of allocations undetected.? Allocations that > are neither > triggered by bytecode nor JNI are executed directly by the VM. > When those allocations occur, the VM should send this event. > Virtual machines that are incapable of bytecode instrumentation > for some or all of their methods may also send this event. >

> Note that the id="SampledObjectAlloc">SampledObjectAlloc > event is triggered on all Java object allocations, including those > triggered by bytecode, > JNI methods, and VM events. > > > > On Mon, Jun 18, 2018 at 12:57 AM David Holmes > wrote: > > On 18/06/2018 5:01 PM, Jeremy Manson wrote: > > We haven't changed when a VM issues "VM object allocation" events. > > > > There were references in the docs to a requirement to use bytecode > > rewriting and JNI interception to track allocations.? With > > SampledObjectAlloc, this is no longer the case - > SampledObjectAlloc can > > track them.? This change is supposed to remove the references to > those > > requirements, and provide suitable replacement text. > > > > VM object alloc has specific language about being able to use it to > > track allocations that cannot be tracked with bytecode > instrumentation > > and JNI interception.? My goal in rephrasing was to make it clear > that, > > while you can still use it for this, you can also just use > > SampledObjectAlloc for everything. > > Okay. That doesn't come across clearly to me - sorry. So you will now > get both kinds of events for allocations done in the VM? > > Thanks, > David > > > > Jeremy > > > > On Sun, Jun 17, 2018 at 9:11 PM David Holmes > > > >> wrote: > > > >? ? ?Hi Jeremy, > > > >? ? ?On 16/06/2018 2:33 AM, Jeremy Manson wrote: > >? ? ? > Hi all, > >? ? ? > > >? ? ? > There are a number of references in the JVMTI doc to its > not doing > >? ? ? > object allocation tracking.? Now that JEP 331 has landed, > these > >? ? ? > references are obsolete.? This patch changes those references > >? ? ?accordingly. > >? ? ? > > >? ? ? > While I was there, I took the liberty of fixing some > spelling?errors. > >? ? ? > > >? ? ? > As far as I know, these are non-normative changes and > modify no > >? ? ?API, so > >? ? ? > they should not require a CSR. > > > >? ? ?I'm unclear on the nature of the change to "VM Object > Allocation". Does > >? ? ?the existence of SampledObjectAlloc change when a VM should > issue "VM > >? ? ?object allocation" events? > > > >? ? ?Thanks, > >? ? ?David > > > >? ? ? > > >? ? ? > Bug: > >? ? ? > https://bugs.openjdk.java.net/browse/JDK-8205113 > >? ? ? > > >? ? ? > Webrev: > >? ? ? > http://cr.openjdk.java.net/~jmanson/8205113/webrev.00/ > >? ? ? > > >? ? ? > Thanks! > >? ? ? > > >? ? ? > Jeremy > > > From thomas.stuefe at gmail.com Tue Jun 19 04:55:31 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 19 Jun 2018 06:55:31 +0200 Subject: jcmd - executing multiple commands at the same safepoint? In-Reply-To: References: <69360DA5-9287-4C84-B5FE-89B240D9C413@gmail.com> <90E966D9-476B-46D9-B08A-7968F6740851@gmail.com> <3606226E-E365-4F9E-B6F3-B573DE1A5579@gmail.com> <5B0486CC.8050801@oracle.com> <937f5c14-327b-5cc5-897a-09aaef349610@oracle.com> <3D31B965-68BE-4CBD-8952-907EBDA71BCE@oracle.com> <09FA5DC6-5839-4E13-9F8A-CBB772AEA2A3@oracle.com> <739cb0e6-357c-75af-e752-a08bc9c90162@oracle.com> Message-ID: On Tue, Jun 19, 2018 at 4:23 AM, David Holmes wrote: > Hi Fred, > > On 19/06/2018 6:52 AM, Frederic Parain wrote: >> >> Hi David, >> >> Thank you for looking at this issue. >> >> The intend has never been to provide a command to easily crash the JVM :-) XD >> The patch was provided without any safety measures in purpose: to enable >> explorations in order to see how badly things could go south. >> >> The final design must include a per-command flag or method specifying if >> the command can be executed inside a safepoint. This information has to >> be provided by JVM developers, as there?s no automatic way to determine if >> a command is safepoint safe or not. >> >> So far, we already have the following rules to decide if a command can be >> executed at a safepoint or not: >> - it must not perform Java callbacks >> - it must not take locks >> - it must not invoke VMOp relying on the execution of their prologue >> and >> epilogue >> >> Let?s see: >> 1 - if there are more restrictions to consider >> 2 - what can be achieved with respect of these restrictions (which >> commands >> can run at a safepoint, which commands can easily be modified to >> comply >> with the restrictions above) > > > Doesn't each Dcmd already explicitly get implemented as either a VMOp or > not? I would not be concerned with trying to run non-VMop Dcmds at a > safepoint - what does that buy us? The issue as I understood it was the > problem of requiring a series of safepoints for a series of > safepoint-needing commands. Or are there useful sequences of interleaving > VMop and non-VMop actions that would benefit from this? It does not hurt either, no? Just from the usability standpoint I would allow bundling non-VMop commands with VMop commands. I only would explicitly forbid those commands which must not run at a safepoint. Note that one hope I have with such a command is that it will encourage small, clean command design unix style a la "do one thing and do it right" - right now, if you want to have a consistent picture involving multiple information containing oop, you'd have to write a single command doing it all together and have to run it at a safepoint. With Freds command, we can write small singular commands which output oops and if we need oop consistency, we bundle them in one safe point. Best Regards, Thomas > > Cheers, > David > > >> Regards, >> >> Fred >> >>> >>> On Jun 17, 2018, at 22:08, David Holmes wrote: >>> >>> Hi Fred, >>> >>> Thanks for this and the analysis below. >>> >>> My 2c is that it is far too risky to provide a general purpose facility >>> like this where arbitrary command combinations can result in crashing the >>> target VM! >>> >>> Certainly no command that requires a JavaThread should be allowed to be >>> requested - only commands actually implemented via safepointing VMOps should >>> be considered. And then we should perhaps constrain things further so that >>> no commands that require prolog or epilog logic are accepted. >>> >>> Turning things around, perhaps we should flag which Dcmds, that operate >>> at a safepoint, are considered "combinable" and only allow those commands to >>> be specified? >>> >>> If we want a general purpose way of expressing a list of Dcmds to execute >>> then perhaps it should be specified to process each command in turn, with a >>> sequence of combinable safepoint ops processed at a single safepoint? >>> >>> Thanks, >>> David >>> >>> On 16/06/2018 1:32 AM, Frederic Parain wrote: >>>> >>>> Hi, >>>> Here?s a first prototype to explore running multiple commands in a >>>> single safepoint: >>>> http://cr.openjdk.java.net/~fparain/DCmdRunAtSafepoint/webrev.00/ >>>> The syntax is very simple, with the character ?+? used as the delimiter >>>> between commands. >>>> $ ./build/macosx-x64-debug/images/jdk/bin/jcmd 21164 VM.run_at_safepoint >>>> VM.version + VM.uptime >>>> 21164: >>>> Java HotSpot(TM) 64-Bit Server VM version >>>> 11-internal+0-2018-05-25-2202478.fparain.DCmd >>>> JDK 11.0.0 >>>> 44.143 s >>>> Playing a bit with it, it quickly shows what are the issues when trying >>>> to run >>>> diagnostic commands at a safepoint. >>>> Issue 1: One important point to remember is that only the VMThread can >>>> run at a safepoint, >>>> all JavaThreads are strictly forbidden to run during the safepoint. So >>>> executing commands at a >>>> safepoint means they will be executed by the VMThread. Unfortunately, >>>> the VMThread is >>>> subject to restrictions and cannot execute all code that a JavaThread >>>> can execute. This >>>> issue is visible with the VM.info command, which takes the Heap_lock. >>>> The VMThread >>>> is not allowed to grab this lock because it could cause the VMThread to >>>> be blocked by >>>> a JavaThread. So far, VM.info cannot be run at a safepoint. >>>> Issue 2: Executing a code at a safepoint requires a VMOperation. >>>> VMOperations are >>>> made of a prologue, a doit() method witch contains the code executed at >>>> the safepoint, >>>> and an epilogue. The prologue and the epilogue are executed by the >>>> requester of the >>>> VMOperation, usually a JavaThread, and they are not executed at a >>>> safepoint. Prologues >>>> and epilogues are designed to execute code the VMThread cannot execute, >>>> like taking >>>> locks or loading classes. When the VMThread itself wants to execute a >>>> VMOperation, >>>> what is called a nested VMOperation, it only executes the doit() method, >>>> skipping the >>>> prologue and the epilogue. This is an issue for diagnostic commands >>>> using a VMOp >>>> in their implementation: when these commands are called while the JVM is >>>> already >>>> at a safepoint, their prologue and epilogue cannot be executed. This >>>> issue impacts >>>> Thread.print and GC.run. >>>> There are probably other issues still to be found, so I encourage people >>>> to test >>>> this patch to see if the combinations of commands they would like to run >>>> at a >>>> safepoint can indeed be run in this mode. >>>> Note that the support for invoking the new command from the >>>> DiagnosticCommand >>>> MBean has not be implemented. This is a not negligible work in order to >>>> correctly >>>> check Java Permissions before executing the commands. >>>> Let me know if you consider this code useful or if the limitations >>>> described above >>>> make this feature not as pretty as we wanted it to be. >>>> Regards, >>>> Fred >>>>> >>>>> On May 29, 2018, at 15:54, Frederic Parain >>>>> wrote: >>>>> >>>>> Hi Thomas, >>>>> >>>>> >>>>>> On May 26, 2018, at 04:07, Thomas St?fe >>>>>> wrote: >>>>>> >>>>> >>>>>>> >>>>>>> >>>>>>> I think that it is a fundamental piece of the design. Adding a new >>>>>>> diagnostic >>>>>>> command to group several commands under a unique safepoint: >>>>>>> 1 - requires explicit action from the user (and acknowledgment of the >>>>>>> risks) >>>>>>> 2 - doesn?t touch the current syntax/semantic at all >>>>>>> >>>>>>> Another reason I?m pushing for a specific diagnostic command rather >>>>>>> than >>>>>>> changing the syntax is remote execution. jcmd is not the only way to >>>>>>> invoke >>>>>>> a diagnostic command, they can also be invoke remotely using JMX and >>>>>>> the >>>>>>> com.sun.management.DiagnosticCommand dynamic MBean. Through the >>>>>>> platform MBean, users do not have access to the jcmd syntax, they see >>>>>>> each diagnostic command as a MBean operation (providing that they >>>>>>> have >>>>>>> been registered with the DCmd_Source_MBean flag). The initial support >>>>>>> for remote execution was taking a single String and processed it just >>>>>>> like >>>>>>> a string passed by jcmd, but people developing consoles for remote >>>>>>> management complained that this was not convenient for them, and >>>>>>> asked >>>>>>> for something they could use more programmatically. >>>>>>> If the safepoint bundling is implemented in the syntax, JMX clients >>>>>>> won?t >>>>>>> be able to use the feature. If we implement it with a new diagnostic >>>>>>> command, >>>>>>> it will be visible from the MBean and they will be able to use it. >>>>>> >>>>>> >>>>>> That is interesting. Do you have examples for such programs? I was >>>>>> looking for ways of how to use jcmd (or equivalent) remotely. >>>>> >>>>> >>>>> If you?re looking for a console application to invoke diagnostic >>>>> command >>>>> on a remote JVM, Mission Control has a dedicated support for that in >>>>> the >>>>> ?Diagnostic Commands? tab. >>>>> If you?re looking for exemples of source code doing remote invocations, >>>>> there?re several unit tests for the DiagnosticCommand MBean in the >>>>> directory >>>>> test/jdk/com/sun/management/DiagnosticCommandMBean. >>>>> >>>>>> >>>>>>> >>>>>>> At last, creating a specific diagnostic command for safepoint >>>>>>> bundling would >>>>>>> show the way on how to group commands for consistency. Safepoints is >>>>>>> not >>>>>>> the only way to get consistency. For some JVM internals reasons, you >>>>>>> might >>>>>>> want to group commands under a single lock, like the Threads_lock. >>>>>>> Adding >>>>>>> a diagnostic command for each type of bundling seems more extensible. >>>>>>> >>>>>> >>>>>> You make good points. >>>>>> >>>>>> Adding a new command has also the advantage that it does not affect >>>>>> downward compatibility and hence would not require a CSR. CSR seems to >>>>>> slow down things a bit (I have a very simple CSR for jcmd open >>>>>> undecided since 12 days: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8203043). So, while I think >>>>>> CSR are a necessary process, I am happy to avoid them if possible. >>>>>> >>>>>> Lets spin this out a bit more: >>>>>> >>>>>> -> However the user specifies "please execute these commands at a >>>>>> single safepoint", if one of the enclosed commands cannot be run >>>>>> inside a safepoint the whole bundle should fail, agreed? So, >>>>>> specifying e.g. JFR.start as part of such a command bundle would cause >>>>>> the whole bundle to not being executed. >>>>> >>>>> >>>>> Yes. The new command N would ask for the execution of a list of >>>>> commands { A, B, C, ? }, and would fail if: >>>>> - any of the specified commands { A, B, C, ? } is not a registered >>>>> command >>>>> - any of the specified commands { A, B, C, ? } doesn?t support >>>>> execution at safepoint >>>>> - any of the specified commands { A, B, C, ? } is not exported to >>>>> the DCmdSource used to invoke N (*) >>>>> >>>>> (*) Not all commands are exported to all interfaces (jcmd,JMX), for >>>>> instance, some commands related to the JMX agent are cannot >>>>> be invoked through the JMX. Even if the new command can be >>>>> invoked from all interfaces, it must not allow to circumvent this >>>>> restriction. This should not be an issue today, because all >>>>> JMX agent commands are calling back Java code, and are not >>>>> suitable for execution at a safepoint. But this would ensure the >>>>> consistency of the model in the long term (and the test is cheap). >>>>> >>>>>> >>>>>> -> If we introduce a command like you propose (lets call it "VM.run" >>>>>> for the sake of the argument) we still need to decide how to delimit >>>>>> the commands at the command line. That brings us back to the syntax >>>>>> question: separation by a explicit delimiter, if possible one that >>>>>> does not clash with common unix shells? >>>>>> >>>>>> Lets try colon ':' : >>>>>> >>>>>> jcmd VM.run VM.uptime : VM.metaspace show-loaders show-classes >>>>>> by-chunktype : VM.classloaders show-classes : VM.info >>>>>> >>>>>> Of course, we can wrap all arguments to jcmd in quotes, then the >>>>>> delimiter sign does not affect things as much, unless we use quotes :) >>>>>> We could e.g. use semicolon: >>>>>> >>>>>> jcmd ' VM.run VM.uptime ; VM.metaspace show-loaders show-classes >>>>>> by-chunktype ; VM.classloaders show-classes ; VM.info ? >>>>>> >>>>> >>>>> I thought about colon and semi-colon too, because they are the most >>>>> obvious delimiters. However, they are already as delimiters in >>>>> classpaths >>>>> (semi-colon on Unix, colon on Windows), so parsing to find the right >>>>> delimiters would be tricky. >>>>> >>>>> I?d start with ?!? or ?+? in a first time, to make some progress on the >>>>> other >>>>> aspects of the command, and buy some time to discuss the delimiter >>>>> issue in parallel. >>>>> >>>>> The last resort would be to force each command in the bundle to >>>>> be specified with quotes (arguments included). But it would make the >>>>> syntax very heavy, I?d prefer a simpler delimiter. >>>>> >>>>> >>>>>> Or, we go back to my original idea of letting the command keyword be >>>>>> the indicator for a new command start. Honestly, at this point I have >>>>>> no preference. Both approaches seem reasonable. >>>>>> >>>>>> The only niggle I have with an explicit command is the syntax, which I >>>>>> find a bit awkward. My thinking before reading your response was that >>>>>> I'd prefer a jcmd flag to specify the same behavior: that all commands >>>>>> should be executed at a single safepoint. E.g. [-s/--stacked] >>>>>> >>>>>> jcmd -s ' VM.uptime ; VM.metaspace show-loaders show-classes >>>>>> by-chunktype ; VM.classloaders show-classes ; VM.info ' >>>>>> >>>>>> But I see the disadvantage: depending on how it is implemented this >>>>>> would require consistent changes in both jcmd and hotspot, and not >>>>>> work with other clients using MBeans. Whereas the hypothetical >>>>>> "VM.run" would work out of the box for any old jcmd version and any >>>>>> other client. >>>>>> >>>>>> Of course one could keep the "VM.run" idea and still give jcmd a >>>>>> "-s/--stacked" option, as a shallow convenience feature, which just >>>>>> translates to "VM.run" and runs that. >>>>>> >>>>>> -> A variation of your "have special commands to run a command bundle >>>>>> in one particular context" theme could be having only one command, >>>>>> e.g. "VM.run", specifying the conditions under which to run the bundle >>>>>> as sub options. E.g. "VM.run -s " for "run commands at >>>>>> safepoint", "VM.run -t " for "run commands under >>>>>> Threads_lock". That would even allow to combine these contexts. I do >>>>>> not know if that makes sense. >>>>>> >>>>>> - Another advantage of having an explicit command just occurring to me >>>>>> is that it enables more evolved command files, where one can >>>>>> intersperse command bundles and single commands. For instance, I could >>>>>> give this to a customer: >>>>>> >>>>>> >>>>>> 1 VM.uptime >>>>>> 2 VM.run_commands VM.metaspace show-loaders show-classes by-chunktype >>>>>> VM.classloaders details VM.classloader_stats >>>>>> 3 GC.run >>>>>> 4 VM.run_commands VM.metaspace show-loaders show-classes by-chunktype >>>>>> VM.classloaders details VM.classloader_stats >>>>>> >>>>>> to execute a command bundle before and after triggering a GC. >>>>> >>>>> >>>>> This would work fine . The parser of the jcmd string will split the >>>>> string based on ?\n? characters, then each line of command will be >>>>> executed >>>>> sequentially, and lines containing the VM.run_command will then create >>>>> the >>>>> safepoint and trigger the execution of the different commands specified >>>>> in >>>>> its argument list. >>>>> >>>>>> >>>>>>> We have a three days week-end in the US, but I should be able to code >>>>>>> something next week. >>>>>>> >>>>>> >>>>>> Oh, you want to take over the implementation, sure! Fine by me. >>>>> >>>>> >>>>> I?m not promising an implementation, I haven?t worked on this code >>>>> since I left the serviceability team. I need to code at least a >>>>> skeleton >>>>> of the command to refresh my memory about all aspects of the >>>>> diagnostic command framework. >>>>> >>>>>> >>>>>>> Thank you for your ideas and your interest in diagnostic commands. >>>>>> >>>>>> >>>>>> Sure. Why I am interested in this: We strongly rely on tools similar >>>>>> to jcmd for our support. We have quite capable tools in our (licensed, >>>>>> non-open) VM port, which are similar to jcmd in many aspects. So we >>>>>> try to bring some of the capabilities upstream, in order to ease the >>>>>> merge costs for us and to help the OpenJDK community. >>>>> >>>>> >>>>> The diagnostic command framework was designed to support an >>>>> extensible set of commands. Feel free to extend, and thank you >>>>> for contributing to the OpenJDK community. >>>>> >>>>> Fred >>>>> >>>>>>> >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> A new separator, more convenient than newline, would be require to >>>>>>>>> have >>>>>>>>> a single line syntax. >>>>>>>>> >>>>>>>>> My 2 cents, >>>>>>>>> >>>>>>>> >>>>>>>> Certainly worth more than 2 cents :) Thanks for your thoughts! >>>>>>>> >>>>>>>> ..Thomas >>>>>>>> >>>>>>>>> Fred >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Erik >>>>>>>>>>> >>>>>>>>>>> Hi Thomas, >>>>>>>>>>> >>>>>>>>>>> very positive suggestion. >>>>>>>>>>> >>>>>>>>>>> One observation: >>>>>>>>>>> There already seems to be some different interpretation of the >>>>>>>>>>> command >>>>>>>>>>> line >>>>>>>>>>> in different Java versions: >>>>>>>>>>> For instance when I try to dump a flight recording in >>>>>>>>>>> 1.8.0_152-ea-b05 : >>>>>>>>>>> I >>>>>>>>>>> can use: >>>>>>>>>>> jcmd 33014 JFR.dump filename="recording1.jfr" recording=1 >>>>>>>>>>> but in build 9+181 , the same command must be: >>>>>>>>>>> jcmd 33014 JFR.dump filename="recording1.jfr",recording=1 >>>>>>>>>>> (the comma to separate sub-options you mentioned earlier) >>>>>>>>>>> >>>>>>>>>>> Therefore the suggestion without curly brackets, giving several >>>>>>>>>>> commands >>>>>>>>>>> after one another seems good with regard to backwards >>>>>>>>>>> compatibility. >>>>>>>>>>> >>>>>>>>>>> Motivation: hawt.io uses the MBean >>>>>>>>>>> com.sun.management:type=DiagnosticCommand >>>>>>>>>>> to access the same functionality as jcmd. Variations in option >>>>>>>>>>> syntax >>>>>>>>>>> across >>>>>>>>>>> Java versions is already an issue (only affecting sub-options >>>>>>>>>>> though, as >>>>>>>>>>> each command is a separate JMX operation). So syntax >>>>>>>>>>> compatibility is >>>>>>>>>>> highly >>>>>>>>>>> appreciated :-) >>>>>>>>>>> >>>>>>>>>>> Martin >>>>>>>>>>> >>>>>>>>>>> l?r. 12. mai 2018 kl. 20:11 skrev Kirk Pepperdine >>>>>>>>>>> : >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> On May 10, 2018, at 11:26 AM, Thomas St?fe >>>>>>>>>>>>> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> On Thu, May 10, 2018 at 9:13 AM, Kirk Pepperdine >>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> The stacking at the safe point would be a huge win. Right now >>>>>>>>>>>>>> thread >>>>>>>>>>>>>> dump consistency can really confuse people as the tooling will >>>>>>>>>>>>>> show >>>>>>>>>>>>>> two >>>>>>>>>>>>>> threads owning the same lock at seemingly the same time. >>>>>>>>>>>>>> Reality, it?s >>>>>>>>>>>>>> just >>>>>>>>>>>>>> that people didn?t realize you get a safe point for each >>>>>>>>>>>>>> thread >>>>>>>>>>>>>> therefor >>>>>>>>>>>>>> there is motion in the system as you?re collecting the data. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I am a bit confused. What tooling are you talking about? >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> jstack. I?ve not seen it with jcmd. But I often see 2 threads >>>>>>>>>>>> holding >>>>>>>>>>>> the >>>>>>>>>>>> same lock at the ?same time? which is of course non-sense. I can >>>>>>>>>>>> dig one >>>>>>>>>>>> up >>>>>>>>>>>> for you if you?re still confused. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> As an aside, it?s amazing how many dev?s aren?t aware of jcmd. >>>>>>>>>>>>>> Just >>>>>>>>>>>>>> yesterday after my session at Devoxx I had someone ask me >>>>>>>>>>>>>> about using >>>>>>>>>>>>>> jfr >>>>>>>>>>>>>> from the command line, many in that session had not seen jcmd >>>>>>>>>>>>>> before. >>>>>>>>>>>>>> The >>>>>>>>>>>>>> feedback was, wow, this is very useful and I wished I had of >>>>>>>>>>>>>> known >>>>>>>>>>>>>> about it >>>>>>>>>>>>>> earlier. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Yes, jcmd is quite useful. I also really like the simple >>>>>>>>>>>>> design, which >>>>>>>>>>>>> is by default up- and downward compatible (so you can talk to >>>>>>>>>>>>> any VM, >>>>>>>>>>>>> new, old, it should not matter). That is just good design. We - >>>>>>>>>>>>> Sap - >>>>>>>>>>>>> work to extend jcmd, to help our support folks. E.g. the whole >>>>>>>>>>>>> new >>>>>>>>>>>>> VM.metaspace command chain. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> And simple it is?.well done!!! >>>>>>>>>>>> >>>>>>>>>>>> ? Kirk >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>> >>>>> >> > From serguei.spitsyn at oracle.com Tue Jun 19 06:01:07 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 18 Jun 2018 23:01:07 -0700 Subject: RFR 8205113: Update JVMTI doc references to object allocation tracking In-Reply-To: <3d01255e-8712-46aa-7659-4ccd7325f81c@oracle.com> References: <384783b9-9ec2-093a-2662-a7a817849720@oracle.com> <3d01255e-8712-46aa-7659-4ccd7325f81c@oracle.com> Message-ID: <6927237d-ce0b-17c0-68f8-06d05b84e516@oracle.com> An HTML attachment was scrubbed... URL: From goetz.lindenmaier at sap.com Tue Jun 19 10:46:10 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 19 Jun 2018 10:46:10 +0000 Subject: RFR(XS): 8205108: [testbug] Fix pattern matching in jstatd tests. In-Reply-To: References: <4d7ec5ef1e224177af8409e9845e71bb@sap.com> Message-ID: Hi Thomas, thanks for your review! I think the upper example output stems from some older jstatd. Probably the output got two new values added. I don't feel like making up values, so I'd rather remove the old output. Is that fine with you? Best regards, Goetz. > -----Original Message----- > From: Thomas St?fe [mailto:thomas.stuefe at gmail.com] > Sent: Samstag, 16. Juni 2018 19:26 > To: Lindenmaier, Goetz > Cc: serviceability-dev (serviceability-dev at openjdk.java.net) dev at openjdk.java.net> > Subject: Re: RFR(XS): 8205108: [testbug] Fix pattern matching in jstatd tests. > > Hi Goetz, > > Looks reasonable. Could you please fix up the example output above the > alternative you added, it misses the columns "CGC" and "CGCT". > > Thanks, Thomas > > On Sat, Jun 16, 2018 at 9:24 AM, Lindenmaier, Goetz > wrote: > > Hi, > > > > Please review this test fix: > > http://cr.openjdk.java.net/~goetz/wr18/8205108-fixJstatDTestPattern/01/ > > > > The values for 'M' must be parsed as PERCENTAGE_OR_DASH. > > The parsing of numbers should not depend on the locale. > > I changed the parsing to use java.text.NumberFormat > > > > Best regards, > > Goetz. > > From gunter.haug at sap.com Tue Jun 19 10:51:52 2018 From: gunter.haug at sap.com (Haug, Gunter) Date: Tue, 19 Jun 2018 10:51:52 +0000 Subject: RFR(S): 8200720: Print additional information in thread dump (times, allocated bytes etc.) In-Reply-To: References: <56817AB5-88FC-4FB9-A271-7206EA69ECA7@sap.com> <2728581E-0E32-4911-B1A9-79AD6CBA81B0@sap.com> <082ca98f-9a68-15a0-95b5-c4d5c733ab36@oracle.com> <933cdacc-9b6f-c8b5-e244-558f38e7888c@oracle.com> <55806aa8-f09b-e9a8-b512-b9676029c494@oracle.com> <0F497BC6-2569-4EF6-9B59-E24A8E1B3A37@sap.com> <5dc08f65e83f4374b8e1f73f218c6d6d@sap.com> <626701f8-b71b-58dc-f837-e5e1d81ef187@oracle.com> Message-ID: <19BE9680-3E6E-4620-B72D-D379F39C1C92@sap.com> Hi all, Thanks Chris and Christoph for the reviews! Christoph, I'll incorporate the little improvement you suggested. David, are you OK with the change as well? Thanks again, Gunter ?On 12.06.18, 01:13, "Chris Plummer" wrote: Hi Gunter, The changes look fine. I can live with the options parsing in attachListener.cpp. As you point out, unrecognized options were already silently ignored. thanks, Chris On 6/8/18 7:05 AM, Haug, Gunter wrote: > Hi all, > > thanks a lot for all the input! I have prepared a new version of the webrev incorporating the suggestions you made (at least I tried): > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v3/ > > This version outputs the thread times unconditionally while the other information is guarded by a flags. I think, most participants found the thread times the most valuable information and had no (strong) objections to adding them unconditionally. > > @David > Implementation is much simpler with conditional output only for JavaThreads. I could get rid of the terrible hack (changing the flag) without having to change too many source files. > > The information on allocated bytes is present in the Thread class already before this proposed change, I just print it. It might be sensible to move _allocated_bytes and the respective methods to the ThreadStatisticInfo class as G?tz suggested. I haven?t done that in the current version, though. > > @David and Thomas > I've removed the pthread-id output. I'm unsure myself what it could be good for. Anyway, we could add it with a separate change (with a better implementation) if there is a need to. > > @Chris > As you have written, the user has no direct contact to the attach listener of the VM and jstack won't misinterpret e.g. -help. I agree that the parsing in attachListener.cpp could be more robust. However, it hasn't been so far either. All that is done in the current implementation is a strcmp to -l everything else will be silently ignored. I can try to make this more robust or we could say that the new output is only available via jcmd. OTH nothing bad can happen with the current version of the proposed change, so we could also leave it as it is. What would you prefer? > > @Kirk and Thomas > Implementing a new diagnostic command is certainly well worth a thought. However, it appears to me that it's not necessary in this case. There are already flags to jstack and jcmd Thread.print, so it's not that uncommon. The amount of information that would be available by a new diagnostic command and that is now added to the thread dump is quite small and I think that is tolerable. Moreover, and most importantly, our support team is use to find the information in the thread dump and it would make things easier to us if I could leave it there. > > Thanks again and have a nice weekend, > Gunter > > > On 06.06.18, 04:56, "David Holmes" wrote: > > Hi Goetz, > > On 5/06/2018 11:07 PM, Lindenmaier, Goetz wrote: > > Hi > > > > this discussion touched a lot of points so far, which seem > > to lead to different conclusions. > > > > I think we should look at the values printed: > > > > 1. cpu=6300.65ms elapsed=123.28s > > Overhead > > cpu time: > > * system calls at thread dump time > > elapsed time: > > * 1 system call at thread creation time > > * 1 64-bit field per thread for the thread start time > > * 1 system call at thread dump time > > > > As I understand, JDK-8143176 would have had to get the > > time at each locking, which is much more time critical > > than doing this during thread creation. While > > Correct. > > > the time a lock was held would be much more useful, > > the ratio here, combined with knowledge about the application, > > could lead to the conclusion that the thread is wrongly > > blocked at much lower cost. > > Agreed. I see no issue with unconditionally adding this information as > it should address some of the concerns of 8143176 as well. > > > > > 2. pthread-id=0x109708000 > > Overhead: > > * a field access at thread dump time. Negligible I would say. > > I'd overlooked this addition, but it is of course the chunk of ifdef > code in osThread.cpp that I objected to. The problem I have here is the > convoluted mess of "thread identifiers" that we already have. We > currently print: > > st->print("tid=" INTPTR_FORMAT " ", p2i(this)); > > which is just the address of the Thread/JavaThread object. Then: > > st->print("nid=0x%x ", thread_id()); > > where 'n' is supposed to represent "native thread id", which is: > - linux: kernel thread id from syscall gettid > - solaris: thread library identity from thr_create() or thr_self() > - windows: thread id from _beginthreadex > - OS X: kernel thread id from pthread_mach_thread_np(pthread_self()); > - Other BSD: kernel (?) thread id from syscall thr_self or getthrid > - AIX: thread library identity from pthread_create() or pthread_self() > > It's telling that the code Gunter added gets the thread library id on > linux, but a "kernel thread id" on other platforms! Do we want to see a > thread library id and a kernel thread id? IIRC nid is supposed be the id > you need to see the thread in a debugger. In which case I'm unclear what > role the thread id Gunter wants to add is playing? I don't think we need > to see kernel ids in general, and pthread-id isn't useful for debugging > is it? > > > > > 3. allocated=242236760B defined_classes=1725 > > Overhead: > > * 1 64-bit field per thread. > > * counter increment on class creation > > > > Especially defined_classes seems to be controversial. > > As this only makes sense for Java threads, this could > > be printed in a line of it's own in Threads::print_on_error() > > (which already gets a flag to customize for jstack: > > print_concurrent_locks) by calling a dedicated function in Thread. > > No need for flag PrintExtendedThreadInfo thus. > > > > But I would propose to drop this information as it is > > too controversial. > > Agreed. > > > This leaves the times and the pthread-id to be decided whether > > they are worth the cost. If so, I think they should be printed > > unconditional. > > Agreed. With some refinement of the "thread id" issue. > > > If a flag is required to avoid bloating the info in the default case, > > I would include more infos, as os_prio and nid under that flag. > > Of course those things exist for good reason. The native id is supposed > to be what aids in you matching what you see in our thread dumps with > what you see in a debugger. Priority is less of an issue these days, but > there was a time ... ;-) > > > As jstack is deprecated, changes to jstack could be skipped always > > disabling the new printouts. > > > > Finally, I would propose to move _allocated_bytes into > > threadStatisticInfo.hpp. > > Didn't that part get dropped? (You don't need it for the time info or > thread id). That's a GC logging issue to me. > > Thanks, > David > > > Best regards, > > Goetz. > > > > > > > > > >> -----Original Message----- > >> From: serviceability-dev [mailto:serviceability-dev- > >> bounces at openjdk.java.net] On Behalf Of David Holmes > >> Sent: Dienstag, 5. Juni 2018 04:53 > >> To: Haug, Gunter ; Chris Plummer > >> ; serviceability-dev >> dev at openjdk.java.net>; Langer, Christoph ; > >> hotspot-runtime-dev at openjdk.java.net > >> Subject: Re: RFR(S): 8200720: Print additional information in thread dump > >> (times, allocated bytes etc.) > >> > >> Hi Gunter, > >> > >> On 5/06/2018 1:27 AM, Haug, Gunter wrote: > >>> Hi David, Chris, > >>> > >>> would it be an option to unconditionally print the additional information? > >> Regardless which way this is implemented it will be rather complicated and it > >> only switches on/off a few field in the thread dump. > >> > >> I'm not convinced this is all suitable information for a thread stack > >> dump. I can see the time information being useful if using the dump to > >> try and diagnoze a hang or responsiveness issue. But the allocated-bytes > >> and classes-defined just doesn't seem useful in the context of a thread > >> dump to me. Anyone interested in allocation stats is going to be looking > >> at GC logs etc which is where this belongs. Ditto the class-stats belong > >> in some kind of classloading logging IMHO. > >> > >> I'm very reluctant to burden the implementation with capturing these > >> kinds of stats, just to use for diagnostic purposes that may not even be > >> asked for. I'm very much in the "you shouldn't pay for what you don't > >> use" camp in that regard. (See my comments in JDK-8143176 referenced by > >> Sean.) > >> > >> The ThreadStatisticInfo adds overhead to every thread object - and I'd > >> have to suspect there could be overlap with whatever flight recorder > >> sticks in there too. (Thread has become even more bloated in recent time!). > >> > >> Cheers, > >> David > >> > >> > >>> Thanks, > >>> Gunter > >>> > >>> On 04.06.18, 01:13, "David Holmes" wrote: > >>> > >>> Hi Gunter, Chris, > >>> > >>> Sorry just trying to catch up and this is only a partial look so far ... > >>> > >>> BTW these changes are not limited to serviceability code so adding in > >>> runtime team as well. > >>> > >>> On 2/06/2018 9:12 AM, Chris Plummer wrote: > >>> > Hi Gunter, > >>> > > >>> > On 6/1/18 3:17 AM, Haug, Gunter wrote: > >>> >> Hi Chris, > >>> >> > >>> >> thanks for looking into this! > >>> >> > >>> >> Re the synchronization: The value is stored only in a VM operation at > >>> >> a safepoint by the VM thread. I was of the opinion, that this could > >>> >> not be interrupted by a second VM operation (of the same type). Or > >> am > >>> >> I missing something else? > >>> > I was really thinking more about the temporary changing of > >>> > PrintExtendedThreadInfo, not the value stored in the VMOp. You may > >> be be > >>> > correct that no more than one VMOp is executing, but while it is > >>> > executing it is has changed the value of PrintExtendedThreadInfo, > >> which > >>> > might have an impact on anything else that triggers printing thread info > >>> > while the VMOp is executing. > >>> > >>> Even if nothing else can possibly be running during the safepoint I find > >>> it extremely bad form to change a command-line flag in this way, > >>> particularly from a safepoint! > >>> > >>> If this flag is intended to be dynamically enabled as part of a dcmd > >>> then it should also be a Manageable flag IMHO. > >>> > >>> That said ... > >>> > >>> >> I did think about passing an argument to the various print_on > >> member > >>> >> functions of the thread classes, but this would require rather > >>> >> extensive code changes for a rather tiny extension. Therefore I feel > >>> >> doing it like this is the lesser evil. > >>> > >>> ... it's obviously not truly a global setting, but one that is needed on > >>> a per-print-request basis. The flag is just the default, but if the > >>> default is off you still want to enable extended printing on a > >>> per-request basis. > >>> > >>> The current print_on mechanics is not set up for this kind of > >>> flexibility. I think this needs more thought. > >>> > >>> --- > >>> > >>> Re osThread.cpp shared code changes ... I really hate to see all the > >>> ifdefs in there. Please add a pd_print_on function to support this if > >>> you truly want platform specific information. > >>> > >>> --- > >>> > >>> threadStatisticInfo.hpp > >>> > >>> typo: getElepsedTime() -> getElapsedTime() > >>> > >>> Thanks, > >>> David > >>> > >>> > >>> > >>> >> > >>> >> Re thread_dump(): I think it's correct or, well, at least it works ;-) > >>> >> In fact jstack will transfer the "-e" and "-l" in only one string, > >>> >> i.e. op->arg(0). > >>> > So you are saying that op->arg(0) is "-e -l" when using jstack? I think > >>> > you really need to clean up the parsing. As it stands right now passing, > >>> > I get the feeling that if a user erroneously asks for help by using > >>> > "jcmd Thread.Print -help", it will end up executing with -e an -l > >>> > enabled, and no failure for the invalid "-help" option. > >>> >> Christoph has already explained my reasoning. But I agree, it's not > >>> >> nice and I would be happy to do it like Christoph suggested. > >>> > I'll respond separately to his suggestion. > >>> > > >>> > thanks, > >>> > > >>> > Chris > >>> >> > >>> >> And typo fixed, sorry. > >>> >> > >>> >> Thanks again, > >>> >> Gunter > >>> >> > >>> >> On 01.06.18, 00:03, "Chris Plummer" > >> wrote: > >>> >> > >>> >> Hi Gunter, > >>> >> globals.hpp: fix typo "informatiuon" > >>> >> I worry a little bit about the synchronizing (if that's the right > >>> >> word) > >>> >> of PrintExtendedThreadInfo and the dcmd's -e flag. When using -e, > >>> >> you > >>> >> are temporarily enabling PrintExtendedThreadInfo if it was false. > >>> >> This > >>> >> temporarily changes the behavior of thread dumps, and could > >>> >> impact other > >>> >> uses that happen in parallel. Also, could two simultaneous uses > >>> >> of -e > >>> >> result in PrintExtendedThreadInfo not getting restored properly? > >>> >> thread_dump() doesn't look right. It looks like you are iterating > >>> >> char > >>> >> by char over the argument, and expect something like "-el" to be > >>> >> specified rather then "-e -l". The loop should be iterating over > >>> >> op->arg(i), not op->arg(0)[i]. > >>> >> The rest of the changes look fine. > >>> >> thanks, > >>> >> Chris > >>> >> On 5/30/18 8:12 AM, Haug, Gunter wrote: > >>> >> > Hi all, > >>> >> > > >>> >> > As Chris proposed, I have made an the extended output > >>> >> switchable. There is an VM flag (PrintExtendedThreadInfo), which is > >>> >> false by default. Moreover, there is an Option (-e) which can be used > >>> >> with jcmd Thread.print as well as with jstack. The -e option > >>> >> essentially sets PrintExtendedThreadInfo true just for the respective > >>> >> thread dump. > >>> >> > > >>> >> > Here is the updated webrev: > >>> >> > > >>> >> > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v2 > >>> >> > > >>> >> > (https://bugs.openjdk.java.net/browse/JDK-8200720) > >>> >> > > >>> >> > Thanks, > >>> >> > Gunter > >>> >> > > >>> >> > > >>> >> > On 02.05.18, 17:07, "serviceability-dev on behalf of Haug, > >>> >> Gunter" >>> >> gunter.haug at sap.com> wrote: > >>> >> > > >>> >> > Hi Chris, > >>> >> > > >>> >> > Thanks for looking into this. > >>> >> > You're right, there is a little more we have. We have > >>> >> implemented an IO tracing mechanism which - rather as a byproduct - > >>> >> keeps track of bytes read and written per thread. However, this of > >>> >> course requires changes not only in hotspot. We would be happy to > >>> >> contribute this as well, but this is a far bigger change and will > >>> >> probably lead to a far bigger discussion. Anyway, with the number of > >>> >> bytes read, the number of classes defined doesn't look that arbitrary > >>> >> anymore, as one can correlate IO to class loading. > >>> >> > > >>> >> > Regarding the verbose option I think that's a good idea! > >>> >> > > >>> >> > Thanks again, > >>> >> > Gunter > >>> >> > > >>> >> > On 01.05.18, 22:55, "Chris Plummer" > >>> >> wrote: > >>> >> > > >>> >> > Hi Gunter, > >>> >> > > >>> >> > The output you are adding is all useful. I think the > >>> >> question is (and > >>> >> > I'd like to see a few people chime in on this for this > >>> >> review) is > >>> >> > whether or not all of it is the appropriate for a > >>> >> thread's stack dump. > >>> >> > For example, defined_classes is on the fringe of what > >>> >> I would call > >>> >> > generally useful info in a stack dump. Sure, there > >>> >> might be that rare > >>> >> > case when you need it, but how often compared to other > >>> >> useful info > >>> >> > maintained on a per thread basis. How many other bits > >>> >> of useful info are > >>> >> > not being printed in favor of defined_classes? It > >>> >> seems you have more in > >>> >> > the queue. How many? I'm worried about how cluttered > >>> >> the stack dumps > >>> >> > will get. Maybe we should add some sort of verbose > >>> >> thread dumping > >>> >> > option. Just a thought. > >>> >> > > >>> >> > As for the implementation, overall it looks good, but > >>> >> I can't speak to > >>> >> > whether or not you are doing proper accounting of > >>> >> defined_classes and > >>> >> > bytes allocated. You'll need input from someone with > >>> >> more knowledge of > >>> >> > those areas. We'll also need to do some testing to > >>> >> make sure tool tests > >>> >> > are not impacted. > >>> >> > > >>> >> > thanks, > >>> >> > > >>> >> > Chris > >>> >> > > >>> >> > On 4/30/18 2:51 AM, Haug, Gunter wrote: > >>> >> > > Hi, > >>> >> > > > >>> >> > > this is an update to an RFR I posted on hotspot-dev, > >>> >> but it is probably more suitable to post it here. Can I please have a > >>> >> review and a sponsor for the following enhancement: > >>> >> > > > >>> >> > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v1 > >>> >> > > https://bugs.openjdk.java.net/browse/JDK-8200720 > >>> >> > > > >>> >> > > We at SAP have extended the thread dumps (obtained > >>> >> by jstack or jcmd) by several fields providing thread specific > >>> >> information. These extensions are quite popular with our support > >> team. > >>> >> With some knowledge of the architecture of the application, they > >> often > >>> >> allow for quick and simple diagnosis of a running system. Therefore > >> we > >>> >> would like to contribute these enhancements. > >>> >> > > > >>> >> > > I took a few simple examples here, namely cpu time, > >>> >> elapsed time since thread creation, bytes allocated and classes > >>> >> defined by the thread and the pthread-id or equivalent on platforms > >>> >> where it makes sense. Provided it is known how the application > >> should > >>> >> behave, a misbehaving thread can identified easily. > >>> >> > > > >>> >> > > There is no measurable overhead for this > >>> >> enhancement. However, it may be a problem that the format of the > >>> >> output is changed. Tools parsing the output may have to be changed. > >>> >> > > > >>> >> > > Here is an example of the output generated: > >>> >> > > > >>> >> > > ------------------------------------------------------ > >>> >> > > > >>> >> > > "main" #1 prio=5 os_prio=31 cpu=6300.65ms > >>> >> elapsed=123.28s allocated=242236760B defined_classes=1725 > >>> >> tid=0x00007fa13a806000 nid=0x1c03 pthread-id=0x109708000 waiting > >> on > >>> >> condition [0x0000000109707000] > >>> >> > > java.lang.Thread.State: TIMED_WAITING (sleeping) > >>> >> > > JavaThread state: _thread_blocked > >>> >> > > Thread: 0x00007fa13a806000 [0x1c03] State: > >>> >> _at_safepoint _has_called_back 0 _at_poll_safepoint 0 > >>> >> > > JavaThread state: _thread_blocked > >>> >> > > at java.lang.Thread.sleep(java.base/Native Method) > >>> >> > > ... > >>> >> > > ------------------------------------------------------ > >>> >> > > > >>> >> > > As mentioned above, we have a whole bunch of other > >>> >> enhancements to the thread dump similar to this one and would be > >>> >> willing to contribute them if there is any interest. > >>> >> > > > >>> >> > > Thanks and best regards, > >>> >> > > Gunter > >>> >> > > > >>> >> > > > >>> >> > > >>> >> > > >>> >> > > >>> >> > > >>> >> > > >>> >> > >>> > > >>> > > >>> > >>> > > From robbin.ehn at oracle.com Tue Jun 19 11:53:57 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 19 Jun 2018 13:53:57 +0200 Subject: jstat does not accept program names as vmid? In-Reply-To: <2cc814d1-a5a8-db18-0468-de2588296555@oracle.com> References: <2cc814d1-a5a8-db18-0468-de2588296555@oracle.com> Message-ID: <3ee0f33d-1eb8-ee53-0495-c04c5b8ca92d@oracle.com> Hi, On 06/18/2018 05:56 PM, Alan Bateman wrote: > On 16/06/2018 18:23, Thomas St?fe wrote: >> : >> >> >> Is that by design? >> > It's pluggable. The two built-in vmid formats are the pid and pid at host for local > and remote respectively. I don't recall any discussion here about identifying an > application by name. I assume there would at least be issues with uniqueness. I believe this question comes from the fact that jcmd can use a main class instead of a pid. There are several discrepancies between various j executables. /Robbin > > -Alan From thomas.stuefe at gmail.com Tue Jun 19 12:00:28 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 19 Jun 2018 14:00:28 +0200 Subject: jstat does not accept program names as vmid? In-Reply-To: <3ee0f33d-1eb8-ee53-0495-c04c5b8ca92d@oracle.com> References: <2cc814d1-a5a8-db18-0468-de2588296555@oracle.com> <3ee0f33d-1eb8-ee53-0495-c04c5b8ca92d@oracle.com> Message-ID: Hi Alan, Robbin, On Tue, Jun 19, 2018 at 1:53 PM, Robbin Ehn wrote: > Hi, > > On 06/18/2018 05:56 PM, Alan Bateman wrote: >> >> On 16/06/2018 18:23, Thomas St?fe wrote: >>> >>> : >>> >>> >>> Is that by design? >>> >> It's pluggable. The two built-in vmid formats are the pid and pid at host for >> local and remote respectively. I don't recall any discussion here about >> identifying an application by name. I assume there would at least be issues >> with uniqueness. > > > I believe this question comes from the fact that jcmd can use a main class > instead of a pid. There are several discrepancies between various j > executables. > Yes, exactly. Since I am more at home with jcmd, I was surprised by the different semantics, especially since both parameters are termed "vm id". I would have not been surprised at all (or, only pleasantly so :) had jstack allowed to connect to multiple VMs too. ..Thomas > /Robbin > >> >> -Alan From gary.adams at oracle.com Tue Jun 19 18:42:49 2018 From: gary.adams at oracle.com (Gary Adams) Date: Tue, 19 Jun 2018 14:42:49 -0400 Subject: RFR: JDK-6545967: sp05t003 failed ResumeThread() due to THREAD_NOT_SUSPENDED Message-ID: <5B294EA9.5030003@oracle.com> There are some rare race conditions that impact some jvmti tests that suspend and resume threads. These tests were recently moved into the open repos. Webrev: http://cr.openjdk.java.net/~gadams/6545967/webrev.00/ The fix in hs203t003 replaces a blind 10 second sleep, with a specific check to wait for a redefineClass to be performed, which takes place as part of callbackFieldAccess. This let's the rest of the operations be performed at a known point in the test sequence. The fix in sp05t003 moves the incrementing of the counters to after the suspend thread calls. The agentProc function is watching the counters to know when to continue processing and eventually resuming of the suspended threads. From serguei.spitsyn at oracle.com Tue Jun 19 21:26:32 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 19 Jun 2018 14:26:32 -0700 Subject: RFR: JDK-6545967: sp05t003 failed ResumeThread() due to THREAD_NOT_SUSPENDED In-Reply-To: <5B294EA9.5030003@oracle.com> References: <5B294EA9.5030003@oracle.com> Message-ID: <4f541d71-1a90-cebe-67bb-dd11f85c072b@oracle.com> An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Tue Jun 19 22:23:37 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 19 Jun 2018 15:23:37 -0700 Subject: RFR(XS): 8205149: hs201t002 should be put on the problem list Message-ID: Hi, Please review the following change to problem list a test. I plan to push under the trivial change rules. I tested by running the :vmTestbase_nsk_jvmti test group and verifying that the test was not run. https://bugs.openjdk.java.net/browse/JDK-8205149 diff --git a/test/hotspot/jtreg/ProblemList.txt b/test/hotspot/jtreg/ProblemList.txt --- a/test/hotspot/jtreg/ProblemList.txt +++ b/test/hotspot/jtreg/ProblemList.txt @@ -136,6 +136,7 @@ ?vmTestbase/nsk/jvmti/ThreadStart/threadstart001/TestDescription.java 8016181 generic-all ?vmTestbase/nsk/jvmti/ThreadStart/threadstart003/TestDescription.java 8034084 generic-all ?vmTestbase/nsk/jvmti/scenarios/extension/EX03/ex03t001/TestDescription.java 8173658 generic-all +vmTestbase/nsk/jvmti/scenarios/hotswap/HS102/hs102t002/TestDescription.java 8204506,8203350 generic-all ?vmTestbase/nsk/jvmti/scenarios/hotswap/HS204/hs204t001/hs204t001.java 6813266 generic-all ?vmTestbase/nsk/jvmti/scenarios/sampling/SP06/sp06t003/TestDescription.java 8051349 generic-all ?vmTestbase/nsk/jvmti/AttachOnDemand/attach034/TestDescription.java 8042145 generic-all thanks, Chris From chris.plummer at oracle.com Tue Jun 19 23:53:47 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 19 Jun 2018 16:53:47 -0700 Subject: RFR: JDK-6545967: sp05t003 failed ResumeThread() due to THREAD_NOT_SUSPENDED In-Reply-To: <4f541d71-1a90-cebe-67bb-dd11f85c072b@oracle.com> References: <5B294EA9.5030003@oracle.com> <4f541d71-1a90-cebe-67bb-dd11f85c072b@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Wed Jun 20 00:10:40 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 19 Jun 2018 17:10:40 -0700 Subject: RFR: JDK-6545967: sp05t003 failed ResumeThread() due to THREAD_NOT_SUSPENDED In-Reply-To: References: <5B294EA9.5030003@oracle.com> <4f541d71-1a90-cebe-67bb-dd11f85c072b@oracle.com> Message-ID: <43178eac-99bb-6b2c-36a4-e4ba1ec51794@oracle.com> An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Wed Jun 20 00:38:36 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 19 Jun 2018 17:38:36 -0700 Subject: RFR(XS): 8205149: hs201t002 should be put on the problem list In-Reply-To: References: Message-ID: Looks fine. Thanks, Serguei On 6/19/18 15:23, Chris Plummer wrote: > Hi, > > Please review the following change to problem list a test. I plan to > push under the trivial change rules. I tested by running the > :vmTestbase_nsk_jvmti test group and verifying that the test was not run. > > https://bugs.openjdk.java.net/browse/JDK-8205149 > > diff --git a/test/hotspot/jtreg/ProblemList.txt > b/test/hotspot/jtreg/ProblemList.txt > --- a/test/hotspot/jtreg/ProblemList.txt > +++ b/test/hotspot/jtreg/ProblemList.txt > @@ -136,6 +136,7 @@ > ?vmTestbase/nsk/jvmti/ThreadStart/threadstart001/TestDescription.java > 8016181 generic-all > ?vmTestbase/nsk/jvmti/ThreadStart/threadstart003/TestDescription.java > 8034084 generic-all > ?vmTestbase/nsk/jvmti/scenarios/extension/EX03/ex03t001/TestDescription.java > 8173658 generic-all > +vmTestbase/nsk/jvmti/scenarios/hotswap/HS102/hs102t002/TestDescription.java > 8204506,8203350 generic-all > ?vmTestbase/nsk/jvmti/scenarios/hotswap/HS204/hs204t001/hs204t001.java > 6813266 generic-all > ?vmTestbase/nsk/jvmti/scenarios/sampling/SP06/sp06t003/TestDescription.java > 8051349 generic-all > ?vmTestbase/nsk/jvmti/AttachOnDemand/attach034/TestDescription.java > 8042145 generic-all > > thanks, > > Chris > > From chris.plummer at oracle.com Wed Jun 20 00:47:08 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 19 Jun 2018 17:47:08 -0700 Subject: RFR(XS): 8205149: hs201t002 should be put on the problem list In-Reply-To: References: Message-ID: Thanks! On 6/19/18 5:38 PM, serguei.spitsyn at oracle.com wrote: > Looks fine. > > Thanks, > Serguei > > > On 6/19/18 15:23, Chris Plummer wrote: >> Hi, >> >> Please review the following change to problem list a test. I plan to >> push under the trivial change rules. I tested by running the >> :vmTestbase_nsk_jvmti test group and verifying that the test was not >> run. >> >> https://bugs.openjdk.java.net/browse/JDK-8205149 >> >> diff --git a/test/hotspot/jtreg/ProblemList.txt >> b/test/hotspot/jtreg/ProblemList.txt >> --- a/test/hotspot/jtreg/ProblemList.txt >> +++ b/test/hotspot/jtreg/ProblemList.txt >> @@ -136,6 +136,7 @@ >> ?vmTestbase/nsk/jvmti/ThreadStart/threadstart001/TestDescription.java >> 8016181 generic-all >> ?vmTestbase/nsk/jvmti/ThreadStart/threadstart003/TestDescription.java >> 8034084 generic-all >> ?vmTestbase/nsk/jvmti/scenarios/extension/EX03/ex03t001/TestDescription.java >> 8173658 generic-all >> +vmTestbase/nsk/jvmti/scenarios/hotswap/HS102/hs102t002/TestDescription.java >> 8204506,8203350 generic-all >> ?vmTestbase/nsk/jvmti/scenarios/hotswap/HS204/hs204t001/hs204t001.java >> 6813266 generic-all >> ?vmTestbase/nsk/jvmti/scenarios/sampling/SP06/sp06t003/TestDescription.java >> 8051349 generic-all >> ?vmTestbase/nsk/jvmti/AttachOnDemand/attach034/TestDescription.java >> 8042145 generic-all >> >> thanks, >> >> Chris >> >> > From jeremymanson at google.com Wed Jun 20 04:11:45 2018 From: jeremymanson at google.com (Jeremy Manson) Date: Tue, 19 Jun 2018 21:11:45 -0700 Subject: RFR 8205113: Update JVMTI doc references to object allocation tracking In-Reply-To: <6927237d-ce0b-17c0-68f8-06d05b84e516@oracle.com> References: <384783b9-9ec2-093a-2662-a7a817849720@oracle.com> <3d01255e-8712-46aa-7659-4ccd7325f81c@oracle.com> <6927237d-ce0b-17c0-68f8-06d05b84e516@oracle.com> Message-ID: That would be okay with me, assuming that my other corrections are made. I'd also like to fix the spelling of instrumentation in the first sentence. Jeremy On Mon, Jun 18, 2018 at 11:01 PM serguei.spitsyn at oracle.com < serguei.spitsyn at oracle.com> wrote: > Hi Jeremy and David, > > Sorry for being late to the party. > > I'm also concerned about the Jeremy's spec update is more intrusive than > necessary. > One specifics of the new SampledObjectAlloc event is that it is posted > conditionally. > So, it is not fully comparable with the VMObjectAlloc event and can not > replace it in any way. > I'm even not yet convinced that any spec update is necessary. > > However, something like below would look minimal and reasonable: > > == Sent when a method causes the virtual machine to allocate an > == Object visible to Java programming language code and the allocation > += is not detectable by other *unconditional* intrumentation mechanisms. > > == Generally object allocation should be detected by instrumenting > == the bytecodes of allocating methods. > > == Object allocation generated in native code by JNI function > == calls should be detected using > == JNI function > interception. > > == Some methods might not have associated bytecodes and are not > == native methods, they instead are executed directly by the > == VM. These methods should send this event. > > == Virtual machines which are incapable of bytecode instrumentation > == for some or all of their methods can send this event. > > *++ Note that the id="SampledObjectAlloc">SampledObjectAlloc* > *++ event is conditionally triggered on all Java object allocations, > including those* > *++ caused by bytecode method execution, JNI method execution, and > directly by VM methods.* > > > Maybe, just adding the last statement would be good enough. > > Thanks, > Serguei > > > On 6/18/18 21:36, David Holmes wrote: > > On 19/06/2018 4:50 AM, Jeremy Manson wrote: > > Yup! The paragraph meanders a bit. How about something like: > > > I'm not sure some of the change quite works. The original text considers > there to be three kinds of methods that can cause allocation when executed: > - Java (bytecode) methods > - JNI methods > - VM methods > > but you've turned this into three kinds of allocation: via bytecode, via > JNI, and via the VM. You then refer to "triggering" an allocation when we > tend to use triggering for events. You also refer to an allocation being > "executed directly by the VM" (a phrase previously applied when the subject > was a 'method') - but you don't really execute allocations. > > IIUC the problem with the existing text is just that it considers VM > allocation events as being undetectable other than by this "VM object > allocation event" - but that's no longer true. So how about something > minimally changed like this: > > --- > Sent when a method causes the virtual machine to directly allocate an > Object visible to Java programming language code. > Generally object allocation can be detected by instrumenting > the bytecodes of allocating methods. > Object allocation generated in native code by JNI function > calls can be detected using > JNI function > interception. > Some methods might not have associated bytecodes and are not > native methods, they instead are executed directly by the > VM. These methods should send this event. > Virtual machines which are incapable of bytecode instrumentation > for some or all of their methods can send this event. > > Note that the id="SampledObjectAlloc">SampledObjectAlloc > event is triggered on all Java object allocations, including those > caused by bytecode method execution, JNI method execution, and > directly by VM methods. > --- > > Thanks, > David > > Sent when the virtual machine allocates an > Object visible to Java programming language code without using a > new bytecode variant or a JNI method. > Many approaches to tracking object allocation use a combination of > bytecode-based instrumentation and JNI > function > interception to intercept allocations. However, this > approach can leave a number of allocations undetected. Allocations that > are neither > triggered by bytecode nor JNI are executed directly by the VM. > When those allocations occur, the VM should send this event. > Virtual machines that are incapable of bytecode instrumentation > for some or all of their methods may also send this event. >

> Note that the id="SampledObjectAlloc">SampledObjectAlloc > event is triggered on all Java object allocations, including those > triggered by bytecode, > JNI methods, and VM events. > > > > On Mon, Jun 18, 2018 at 12:57 AM David Holmes > wrote: > > On 18/06/2018 5:01 PM, Jeremy Manson wrote: > > We haven't changed when a VM issues "VM object allocation" events. > > > > There were references in the docs to a requirement to use bytecode > > rewriting and JNI interception to track allocations. With > > SampledObjectAlloc, this is no longer the case - > SampledObjectAlloc can > > track them. This change is supposed to remove the references to > those > > requirements, and provide suitable replacement text. > > > > VM object alloc has specific language about being able to use it to > > track allocations that cannot be tracked with bytecode > instrumentation > > and JNI interception. My goal in rephrasing was to make it clear > that, > > while you can still use it for this, you can also just use > > SampledObjectAlloc for everything. > > Okay. That doesn't come across clearly to me - sorry. So you will now > get both kinds of events for allocations done in the VM? > > Thanks, > David > > > > Jeremy > > > > On Sun, Jun 17, 2018 at 9:11 PM David Holmes > > > > > >> wrote: > > > > Hi Jeremy, > > > > On 16/06/2018 2:33 AM, Jeremy Manson wrote: > > > Hi all, > > > > > > There are a number of references in the JVMTI doc to its > not doing > > > object allocation tracking. Now that JEP 331 has landed, > these > > > references are obsolete. This patch changes those > references > > accordingly. > > > > > > While I was there, I took the liberty of fixing some > spelling errors. > > > > > > As far as I know, these are non-normative changes and > modify no > > API, so > > > they should not require a CSR. > > > > I'm unclear on the nature of the change to "VM Object > Allocation". Does > > the existence of SampledObjectAlloc change when a VM should > issue "VM > > object allocation" events? > > > > Thanks, > > David > > > > > > > > Bug: > > > https://bugs.openjdk.java.net/browse/JDK-8205113 > > > > > > Webrev: > > > http://cr.openjdk.java.net/~jmanson/8205113/webrev.00/ > > > > > > Thanks! > > > > > > Jeremy > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Wed Jun 20 04:41:46 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 19 Jun 2018 21:41:46 -0700 Subject: RFR 8205113: Update JVMTI doc references to object allocation tracking In-Reply-To: References: <384783b9-9ec2-093a-2662-a7a817849720@oracle.com> <3d01255e-8712-46aa-7659-4ccd7325f81c@oracle.com> <6927237d-ce0b-17c0-68f8-06d05b84e516@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Wed Jun 20 04:54:52 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 20 Jun 2018 14:54:52 +1000 Subject: RFR 8205113: Update JVMTI doc references to object allocation tracking In-Reply-To: References: <384783b9-9ec2-093a-2662-a7a817849720@oracle.com> <3d01255e-8712-46aa-7659-4ccd7325f81c@oracle.com> <6927237d-ce0b-17c0-68f8-06d05b84e516@oracle.com> Message-ID: <53f576e2-0adb-a6c5-8a3c-026dcb1aa7ad@oracle.com> On 20/06/2018 2:41 PM, serguei.spitsyn at oracle.com wrote: > On 6/19/18 21:11, Jeremy Manson wrote: >> That would be okay with me, assuming that my other corrections are made. > > Another option would be to say "non-sampling" instead of "unconditional": > > == Sent when a method causes the virtual machine to allocate an > == Object visible to Java programming language code and the allocation > += is not detectable by other *non-sampling* instrumentation mechanisms. > > >> I'd also like to fix the spelling of instrumentation in the first >> sentence. > > Yes, of course. > > Let's wait for David's opinion. I'm okay with Serguei's suggestion (combined with Jeremy's other changes). I'm not that familiar with conditional versus unconditional instrumentation. Thanks, David > > Thanks, > Serguei > > >> >> Jeremy >> >> On Mon, Jun 18, 2018 at 11:01 PM serguei.spitsyn at oracle.com >> > > wrote: >> >> Hi Jeremy and David, >> >> Sorry for being late to the party. >> >> I'm also concerned about the Jeremy's spec update is more >> intrusive than necessary. >> One specifics of the new SampledObjectAlloc event is that it is >> posted conditionally. >> So, it is not fully comparable with the VMObjectAlloc event and >> can not replace it in any way. >> I'm even not yet convinced that any spec update is necessary. >> >> However, something like below would look minimal and reasonable: >> >> == Sent when a method causes the virtual machine to allocate an >> == Object visible to Java programming language code and the allocation >> += is not detectable by other *unconditional* intrumentation >> mechanisms. >> >> == Generally object allocation should be detected by instrumenting >> == the bytecodes of allocating methods. >> >> == Object allocation generated in native code by JNI function >> == calls should be detected using >> == JNI function >> interception. >> >> == Some methods might not have associated bytecodes and are not >> == native methods, they instead are executed directly by the >> == VM. These methods should send this event. >> >> == Virtual machines which are incapable of bytecode instrumentation >> == for some or all of their methods can send this event. >> >> *++ Note that the > id="SampledObjectAlloc">SampledObjectAlloc** >> **++ event is conditionally triggered on all Java object >> allocations, including those** >> **++ caused by bytecode method execution, JNI method execution, >> and directly by VM methods.** >> * >> >> Maybe, just adding the last statement would be good enough. >> >> Thanks, >> Serguei >> >> >> On 6/18/18 21:36, David Holmes wrote: >>> On 19/06/2018 4:50 AM, Jeremy Manson wrote: >>>> Yup!? The paragraph meanders a bit.? How about something like: >>> >>> I'm not sure some of the change quite works. The original text >>> considers there to be three kinds of methods that can cause >>> allocation when executed: >>> - Java (bytecode) methods >>> - JNI methods >>> - VM methods >>> >>> but you've turned this into three kinds of allocation: via >>> bytecode, via JNI, and via the VM. You then refer to "triggering" >>> an allocation when we tend to use triggering for events. You also >>> refer to an allocation being "executed directly by the VM" (a >>> phrase previously applied when the subject was a 'method') - but >>> you don't really execute allocations. >>> >>> IIUC the problem with the existing text is just that it considers >>> VM allocation events as being undetectable other than by this "VM >>> object allocation event" - but that's no longer true. So how >>> about something minimally changed like this: >>> >>> --- >>> ? Sent when a method causes the virtual machine to directly >>> allocate an >>> ? Object visible to Java programming language code. >>> ? Generally object allocation can be detected by instrumenting >>> ? the bytecodes of allocating methods. >>> ? Object allocation generated in native code by JNI function >>> ? calls can be detected using >>> ? JNI function >>> interception. >>> ?? Some methods might not have associated bytecodes and are not >>> ?? native methods, they instead are executed directly by the >>> ?? VM. These methods should send this event. >>> ?? Virtual machines which are incapable of bytecode instrumentation >>> ?? for some or all of their methods can send this event. >>> >>> ?? Note that the >> id="SampledObjectAlloc">SampledObjectAlloc >>> ?? event is triggered on all Java object allocations, including >>> those >>> ?? caused by bytecode method execution, JNI method execution, and >>> ?? directly by VM methods. >>> --- >>> >>> Thanks, >>> David >>> >>>> Sent when the virtual machine allocates an >>>> Object visible to Java programming language code without using a >>>> new bytecode variant or a JNI method. >>>> Many approaches to tracking object allocation use a combination of >>>> bytecode-based instrumentation and >>> id="jniIntercept">JNI function >>>> interception to intercept allocations.? However, >>>> this >>>> approach can leave a number of allocations undetected. >>>> Allocations that are neither >>>> triggered by bytecode nor JNI are executed directly by the VM. >>>> When those allocations occur, the VM should send this event. >>>> Virtual machines that are incapable of bytecode instrumentation >>>> for some or all of their methods may also send this event. >>>>

>>>> Note that the >>> id="SampledObjectAlloc">SampledObjectAlloc >>>> event is triggered on all Java object allocations, including >>>> those triggered by bytecode, >>>> JNI methods, and VM events. >>>> >>>> >>>> >>>> On Mon, Jun 18, 2018 at 12:57 AM David Holmes >>>> >>>> >>>> > wrote: >>>> >>>> ??? On 18/06/2018 5:01 PM, Jeremy Manson wrote: >>>> ???? > We haven't changed when a VM issues "VM object >>>> allocation" events. >>>> ???? > >>>> ???? > There were references in the docs to a requirement to use >>>> bytecode >>>> ???? > rewriting and JNI interception to track allocations.? With >>>> ???? > SampledObjectAlloc, this is no longer the case - >>>> ??? SampledObjectAlloc can >>>> ???? > track them.? This change is supposed to remove the >>>> references to >>>> ??? those >>>> ???? > requirements, and provide suitable replacement text. >>>> ???? > >>>> ???? > VM object alloc has specific language about being able to >>>> use it to >>>> ???? > track allocations that cannot be tracked with bytecode >>>> ??? instrumentation >>>> ???? > and JNI interception.? My goal in rephrasing was to make >>>> it clear >>>> ??? that, >>>> ???? > while you can still use it for this, you can also just use >>>> ???? > SampledObjectAlloc for everything. >>>> >>>> ??? Okay. That doesn't come across clearly to me - sorry. So you >>>> will now >>>> ??? get both kinds of events for allocations done in the VM? >>>> >>>> ??? Thanks, >>>> ??? David >>>> >>>> >>>> ???? > Jeremy >>>> ???? > >>>> ???? > On Sun, Jun 17, 2018 at 9:11 PM David Holmes >>>> ??? >>>> >>>> ???? > >>> >>>> >> wrote: >>>> ???? > >>>> ???? >? ? ?Hi Jeremy, >>>> ???? > >>>> ???? >? ? ?On 16/06/2018 2:33 AM, Jeremy Manson wrote: >>>> ???? >? ? ? > Hi all, >>>> ???? >? ? ? > >>>> ???? >? ? ? > There are a number of references in the JVMTI doc >>>> to its >>>> ??? not doing >>>> ???? >? ? ? > object allocation tracking.? Now that JEP 331 has >>>> landed, >>>> ??? these >>>> ???? >? ? ? > references are obsolete.? This patch changes those >>>> references >>>> ???? >? ? ?accordingly. >>>> ???? >? ? ? > >>>> ???? >? ? ? > While I was there, I took the liberty of fixing some >>>> ??? spelling?errors. >>>> ???? >? ? ? > >>>> ???? >? ? ? > As far as I know, these are non-normative changes and >>>> ??? modify no >>>> ???? >? ? ?API, so >>>> ???? >? ? ? > they should not require a CSR. >>>> ???? > >>>> ???? >? ? ?I'm unclear on the nature of the change to "VM Object >>>> ??? Allocation". Does >>>> ???? >? ? ?the existence of SampledObjectAlloc change when a VM >>>> should >>>> ??? issue "VM >>>> ???? >? ? ?object allocation" events? >>>> ???? > >>>> ???? >? ? ?Thanks, >>>> ???? >? ? ?David >>>> ???? > >>>> ???? >? ? ? > >>>> ???? >? ? ? > Bug: >>>> ???? >? ? ? > https://bugs.openjdk.java.net/browse/JDK-8205113 >>>> ???? >? ? ? > >>>> ???? >? ? ? > Webrev: >>>> ???? >? ? ? > >>>> http://cr.openjdk.java.net/~jmanson/8205113/webrev.00/ >>>> >>>> ???? >? ? ? > >>>> ???? >? ? ? > Thanks! >>>> ???? >? ? ? > >>>> ???? >? ? ? > Jeremy >>>> ???? > >>>> >> > From serguei.spitsyn at oracle.com Wed Jun 20 05:41:18 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 19 Jun 2018 22:41:18 -0700 Subject: RFR 8205113: Update JVMTI doc references to object allocation tracking In-Reply-To: <53f576e2-0adb-a6c5-8a3c-026dcb1aa7ad@oracle.com> References: <384783b9-9ec2-093a-2662-a7a817849720@oracle.com> <3d01255e-8712-46aa-7659-4ccd7325f81c@oracle.com> <6927237d-ce0b-17c0-68f8-06d05b84e516@oracle.com> <53f576e2-0adb-a6c5-8a3c-026dcb1aa7ad@oracle.com> Message-ID: <8224afa2-a6cb-fee2-8fa3-184828ac7c50@oracle.com> On 6/19/18 21:54, David Holmes wrote: > On 20/06/2018 2:41 PM, serguei.spitsyn at oracle.com wrote: >> On 6/19/18 21:11, Jeremy Manson wrote: >>> That would be okay with me, assuming that my other corrections are >>> made. >> >> Another option would be to say "non-sampling" instead of >> "unconditional": >> >> == Sent when a method causes the virtual machine to allocate an >> == Object visible to Java programming language code and the allocation >> += is not detectable by other *non-sampling* instrumentation mechanisms. >> >> >>> I'd also like to fix the spelling of instrumentation in the first >>> sentence. >> >> Yes, of course. >> >> Let's wait for David's opinion. > > I'm okay with Serguei's suggestion (combined with Jeremy's other > changes). > > I'm not that familiar with conditional versus unconditional > instrumentation. The whole point of the SampledObjectAlloc events is to post them conditionally depending on the SamplingRate so that just some amount of allocations is sampled. It is why the overhead can be minimal (less than 5 percents). It is still possible to sample every allocation with the SamplingRate == 1. The VMObjectAlloc events are posted unconditionally, so every VM allocation is posted. Thanks, Serguei > > Thanks, > David > >> >> Thanks, >> Serguei >> >> >>> >>> Jeremy >>> >>> On Mon, Jun 18, 2018 at 11:01 PM serguei.spitsyn at oracle.com >>> >> > wrote: >>> >>> ??? Hi Jeremy and David, >>> >>> ??? Sorry for being late to the party. >>> >>> ??? I'm also concerned about the Jeremy's spec update is more >>> ??? intrusive than necessary. >>> ??? One specifics of the new SampledObjectAlloc event is that it is >>> ??? posted conditionally. >>> ??? So, it is not fully comparable with the VMObjectAlloc event and >>> ??? can not replace it in any way. >>> ??? I'm even not yet convinced that any spec update is necessary. >>> >>> ??? However, something like below would look minimal and reasonable: >>> >>> ??? == Sent when a method causes the virtual machine to allocate an >>> ??? == Object visible to Java programming language code and the >>> allocation >>> ??? += is not detectable by other *unconditional* intrumentation >>> ??? mechanisms. >>> >>> ??? == Generally object allocation should be detected by instrumenting >>> ??? == the bytecodes of allocating methods. >>> >>> ??? == Object allocation generated in native code by JNI function >>> ??? == calls should be detected using >>> ??? == JNI function >>> ??? interception. >>> >>> ??? == Some methods might not have associated bytecodes and are not >>> ??? == native methods, they instead are executed directly by the >>> ??? == VM. These methods should send this event. >>> >>> ??? == Virtual machines which are incapable of bytecode instrumentation >>> ??? == for some or all of their methods can send this event. >>> >>> ??? *++ Note that the >> id="SampledObjectAlloc">SampledObjectAlloc** >>> ??? **++ event is conditionally triggered on all Java object >>> ??? allocations, including those** >>> ??? **++ caused by bytecode method execution, JNI method execution, >>> ??? and directly by VM methods.** >>> ??? * >>> >>> ??? Maybe, just adding the last statement would be good enough. >>> >>> ??? Thanks, >>> ??? Serguei >>> >>> >>> ??? On 6/18/18 21:36, David Holmes wrote: >>>> ??? On 19/06/2018 4:50 AM, Jeremy Manson wrote: >>>>> ??? Yup!? The paragraph meanders a bit.? How about something like: >>>> >>>> ??? I'm not sure some of the change quite works. The original text >>>> ??? considers there to be three kinds of methods that can cause >>>> ??? allocation when executed: >>>> ??? - Java (bytecode) methods >>>> ??? - JNI methods >>>> ??? - VM methods >>>> >>>> ??? but you've turned this into three kinds of allocation: via >>>> ??? bytecode, via JNI, and via the VM. You then refer to "triggering" >>>> ??? an allocation when we tend to use triggering for events. You also >>>> ??? refer to an allocation being "executed directly by the VM" (a >>>> ??? phrase previously applied when the subject was a 'method') - but >>>> ??? you don't really execute allocations. >>>> >>>> ??? IIUC the problem with the existing text is just that it considers >>>> ??? VM allocation events as being undetectable other than by this "VM >>>> ??? object allocation event" - but that's no longer true. So how >>>> ??? about something minimally changed like this: >>>> >>>> ??? --- >>>> ??? ? Sent when a method causes the virtual machine to directly >>>> ??? allocate an >>>> ??? ? Object visible to Java programming language code. >>>> ??? ? Generally object allocation can be detected by instrumenting >>>> ??? ? the bytecodes of allocating methods. >>>> ??? ? Object allocation generated in native code by JNI function >>>> ??? ? calls can be detected using >>>> ??? ? JNI function >>>> ??? interception. >>>> ??? ?? Some methods might not have associated bytecodes and are not >>>> ??? ?? native methods, they instead are executed directly by the >>>> ??? ?? VM. These methods should send this event. >>>> ??? ?? Virtual machines which are incapable of bytecode >>>> instrumentation >>>> ??? ?? for some or all of their methods can send this event. >>>> >>>> ??? ?? Note that the >>> id="SampledObjectAlloc">SampledObjectAlloc >>>> ??? ?? event is triggered on all Java object allocations, including >>>> ??? those >>>> ??? ?? caused by bytecode method execution, JNI method execution, and >>>> ??? ?? directly by VM methods. >>>> ??? --- >>>> >>>> ??? Thanks, >>>> ??? David >>>> >>>>> ??? Sent when the virtual machine allocates an >>>>> ??? Object visible to Java programming language code without using a >>>>> ??? new bytecode variant or a JNI method. >>>>> ??? Many approaches to tracking object allocation use a >>>>> combination of >>>>> ??? bytecode-based instrumentation and >>>> ??? id="jniIntercept">JNI function >>>>> ??? interception to intercept allocations.? However, >>>>> ??? this >>>>> ??? approach can leave a number of allocations undetected. >>>>> ??? Allocations that are neither >>>>> ??? triggered by bytecode nor JNI are executed directly by the VM. >>>>> ??? When those allocations occur, the VM should send this event. >>>>> ??? Virtual machines that are incapable of bytecode instrumentation >>>>> ??? for some or all of their methods may also send this event. >>>>> ???

>>>>> ??? Note that the >>>> id="SampledObjectAlloc">SampledObjectAlloc >>>>> ??? event is triggered on all Java object allocations, including >>>>> ??? those triggered by bytecode, >>>>> ??? JNI methods, and VM events. >>>>> >>>>> >>>>> >>>>> ??? On Mon, Jun 18, 2018 at 12:57 AM David Holmes >>>>> ??? >>>>> ??? >>>>> ??? > wrote: >>>>> >>>>> ??? ??? On 18/06/2018 5:01 PM, Jeremy Manson wrote: >>>>> ??? ???? > We haven't changed when a VM issues "VM object >>>>> ??? allocation" events. >>>>> ??? ???? > >>>>> ??? ???? > There were references in the docs to a requirement to use >>>>> ??? bytecode >>>>> ??? ???? > rewriting and JNI interception to track allocations.? With >>>>> ??? ???? > SampledObjectAlloc, this is no longer the case - >>>>> ??? ??? SampledObjectAlloc can >>>>> ??? ???? > track them.? This change is supposed to remove the >>>>> ??? references to >>>>> ??? ??? those >>>>> ??? ???? > requirements, and provide suitable replacement text. >>>>> ??? ???? > >>>>> ??? ???? > VM object alloc has specific language about being able to >>>>> ??? use it to >>>>> ??? ???? > track allocations that cannot be tracked with bytecode >>>>> ??? ??? instrumentation >>>>> ??? ???? > and JNI interception.? My goal in rephrasing was to make >>>>> ??? it clear >>>>> ??? ??? that, >>>>> ??? ???? > while you can still use it for this, you can also just use >>>>> ??? ???? > SampledObjectAlloc for everything. >>>>> >>>>> ??? ??? Okay. That doesn't come across clearly to me - sorry. So you >>>>> ??? will now >>>>> ??? ??? get both kinds of events for allocations done in the VM? >>>>> >>>>> ??? ??? Thanks, >>>>> ??? ??? David >>>>> >>>>> >>>>> ??? ???? > Jeremy >>>>> ??? ???? > >>>>> ??? ???? > On Sun, Jun 17, 2018 at 9:11 PM David Holmes >>>>> ??? ??? >>>>> ??? >>>>> ??? ???? > >>>> ??? >>>>> ??? >> wrote: >>>>> ??? ???? > >>>>> ??? ???? >? ? ?Hi Jeremy, >>>>> ??? ???? > >>>>> ??? ???? >? ? ?On 16/06/2018 2:33 AM, Jeremy Manson wrote: >>>>> ??? ???? >? ? ? > Hi all, >>>>> ??? ???? >? ? ? > >>>>> ??? ???? >? ? ? > There are a number of references in the JVMTI doc >>>>> ??? to its >>>>> ??? ??? not doing >>>>> ??? ???? >? ? ? > object allocation tracking.? Now that JEP 331 has >>>>> ??? landed, >>>>> ??? ??? these >>>>> ??? ???? >? ? ? > references are obsolete.? This patch changes those >>>>> ??? references >>>>> ??? ???? >? ? ?accordingly. >>>>> ??? ???? >? ? ? > >>>>> ??? ???? >? ? ? > While I was there, I took the liberty of fixing >>>>> some >>>>> ??? ??? spelling?errors. >>>>> ??? ???? >? ? ? > >>>>> ??? ???? >? ? ? > As far as I know, these are non-normative >>>>> changes and >>>>> ??? ??? modify no >>>>> ??? ???? >? ? ?API, so >>>>> ??? ???? >? ? ? > they should not require a CSR. >>>>> ??? ???? > >>>>> ??? ???? >? ? ?I'm unclear on the nature of the change to "VM Object >>>>> ??? ??? Allocation". Does >>>>> ??? ???? >? ? ?the existence of SampledObjectAlloc change when a VM >>>>> ??? should >>>>> ??? ??? issue "VM >>>>> ??? ???? >? ? ?object allocation" events? >>>>> ??? ???? > >>>>> ??? ???? >? ? ?Thanks, >>>>> ??? ???? >? ? ?David >>>>> ??? ???? > >>>>> ??? ???? >? ? ? > >>>>> ??? ???? >? ? ? > Bug: >>>>> ??? ???? >? ? ? > https://bugs.openjdk.java.net/browse/JDK-8205113 >>>>> ??? ???? >? ? ? > >>>>> ??? ???? >? ? ? > Webrev: >>>>> ??? ???? >? ? ? > >>>>> ??? http://cr.openjdk.java.net/~jmanson/8205113/webrev.00/ >>>>> >>>>> ??? ???? >? ? ? > >>>>> ??? ???? >? ? ? > Thanks! >>>>> ??? ???? >? ? ? > >>>>> ??? ???? >? ? ? > Jeremy >>>>> ??? ???? > >>>>> >>> >> From jeremymanson at google.com Wed Jun 20 06:29:38 2018 From: jeremymanson at google.com (Jeremy Manson) Date: Tue, 19 Jun 2018 23:29:38 -0700 Subject: RFR 8205113: Update JVMTI doc references to object allocation tracking In-Reply-To: <8224afa2-a6cb-fee2-8fa3-184828ac7c50@oracle.com> References: <384783b9-9ec2-093a-2662-a7a817849720@oracle.com> <3d01255e-8712-46aa-7659-4ccd7325f81c@oracle.com> <6927237d-ce0b-17c0-68f8-06d05b84e516@oracle.com> <53f576e2-0adb-a6c5-8a3c-026dcb1aa7ad@oracle.com> <8224afa2-a6cb-fee2-8fa3-184828ac7c50@oracle.com> Message-ID: Maybe we should make that clarification. Also, the reason I danced around that in my revision is that if you set the sampling rate to 0, you get unconditional sampling. Jeremy On Tue, Jun 19, 2018 at 10:41 PM serguei.spitsyn at oracle.com < serguei.spitsyn at oracle.com> wrote: > On 6/19/18 21:54, David Holmes wrote: > > On 20/06/2018 2:41 PM, serguei.spitsyn at oracle.com wrote: > >> On 6/19/18 21:11, Jeremy Manson wrote: > >>> That would be okay with me, assuming that my other corrections are > >>> made. > >> > >> Another option would be to say "non-sampling" instead of > >> "unconditional": > >> > >> == Sent when a method causes the virtual machine to allocate an > >> == Object visible to Java programming language code and the allocation > >> += is not detectable by other *non-sampling* instrumentation mechanisms. > >> > >> > >>> I'd also like to fix the spelling of instrumentation in the first > >>> sentence. > >> > >> Yes, of course. > >> > >> Let's wait for David's opinion. > > > > I'm okay with Serguei's suggestion (combined with Jeremy's other > > changes). > > > > I'm not that familiar with conditional versus unconditional > > instrumentation. > > The whole point of the SampledObjectAlloc events is to post them > conditionally > depending on the SamplingRate so that just some amount of allocations is > sampled. > It is why the overhead can be minimal (less than 5 percents). > It is still possible to sample every allocation with the SamplingRate == 1. > > The VMObjectAlloc events are posted unconditionally, so every VM > allocation is posted. > > Thanks, > Serguei > > > > > > Thanks, > > David > > > >> > >> Thanks, > >> Serguei > >> > >> > >>> > >>> Jeremy > >>> > >>> On Mon, Jun 18, 2018 at 11:01 PM serguei.spitsyn at oracle.com > >>> >>> > wrote: > >>> > >>> Hi Jeremy and David, > >>> > >>> Sorry for being late to the party. > >>> > >>> I'm also concerned about the Jeremy's spec update is more > >>> intrusive than necessary. > >>> One specifics of the new SampledObjectAlloc event is that it is > >>> posted conditionally. > >>> So, it is not fully comparable with the VMObjectAlloc event and > >>> can not replace it in any way. > >>> I'm even not yet convinced that any spec update is necessary. > >>> > >>> However, something like below would look minimal and reasonable: > >>> > >>> == Sent when a method causes the virtual machine to allocate an > >>> == Object visible to Java programming language code and the > >>> allocation > >>> += is not detectable by other *unconditional* intrumentation > >>> mechanisms. > >>> > >>> == Generally object allocation should be detected by instrumenting > >>> == the bytecodes of allocating methods. > >>> > >>> == Object allocation generated in native code by JNI function > >>> == calls should be detected using > >>> == JNI function > >>> interception. > >>> > >>> == Some methods might not have associated bytecodes and are not > >>> == native methods, they instead are executed directly by the > >>> == VM. These methods should send this event. > >>> > >>> == Virtual machines which are incapable of bytecode instrumentation > >>> == for some or all of their methods can send this event. > >>> > >>> *++ Note that the >>> id="SampledObjectAlloc">SampledObjectAlloc** > >>> **++ event is conditionally triggered on all Java object > >>> allocations, including those** > >>> **++ caused by bytecode method execution, JNI method execution, > >>> and directly by VM methods.** > >>> * > >>> > >>> Maybe, just adding the last statement would be good enough. > >>> > >>> Thanks, > >>> Serguei > >>> > >>> > >>> On 6/18/18 21:36, David Holmes wrote: > >>>> On 19/06/2018 4:50 AM, Jeremy Manson wrote: > >>>>> Yup! The paragraph meanders a bit. How about something like: > >>>> > >>>> I'm not sure some of the change quite works. The original text > >>>> considers there to be three kinds of methods that can cause > >>>> allocation when executed: > >>>> - Java (bytecode) methods > >>>> - JNI methods > >>>> - VM methods > >>>> > >>>> but you've turned this into three kinds of allocation: via > >>>> bytecode, via JNI, and via the VM. You then refer to "triggering" > >>>> an allocation when we tend to use triggering for events. You also > >>>> refer to an allocation being "executed directly by the VM" (a > >>>> phrase previously applied when the subject was a 'method') - but > >>>> you don't really execute allocations. > >>>> > >>>> IIUC the problem with the existing text is just that it considers > >>>> VM allocation events as being undetectable other than by this "VM > >>>> object allocation event" - but that's no longer true. So how > >>>> about something minimally changed like this: > >>>> > >>>> --- > >>>> Sent when a method causes the virtual machine to directly > >>>> allocate an > >>>> Object visible to Java programming language code. > >>>> Generally object allocation can be detected by instrumenting > >>>> the bytecodes of allocating methods. > >>>> Object allocation generated in native code by JNI function > >>>> calls can be detected using > >>>> JNI function > >>>> interception. > >>>> Some methods might not have associated bytecodes and are not > >>>> native methods, they instead are executed directly by the > >>>> VM. These methods should send this event. > >>>> Virtual machines which are incapable of bytecode > >>>> instrumentation > >>>> for some or all of their methods can send this event. > >>>> > >>>> Note that the >>>> id="SampledObjectAlloc">SampledObjectAlloc > >>>> event is triggered on all Java object allocations, including > >>>> those > >>>> caused by bytecode method execution, JNI method execution, and > >>>> directly by VM methods. > >>>> --- > >>>> > >>>> Thanks, > >>>> David > >>>> > >>>>> Sent when the virtual machine allocates an > >>>>> Object visible to Java programming language code without using a > >>>>> new bytecode variant or a JNI method. > >>>>> Many approaches to tracking object allocation use a > >>>>> combination of > >>>>> bytecode-based instrumentation and >>>>> id="jniIntercept">JNI function > >>>>> interception to intercept allocations. However, > >>>>> this > >>>>> approach can leave a number of allocations undetected. > >>>>> Allocations that are neither > >>>>> triggered by bytecode nor JNI are executed directly by the VM. > >>>>> When those allocations occur, the VM should send this event. > >>>>> Virtual machines that are incapable of bytecode instrumentation > >>>>> for some or all of their methods may also send this event. > >>>>>

> >>>>> Note that the >>>>> id="SampledObjectAlloc">SampledObjectAlloc > >>>>> event is triggered on all Java object allocations, including > >>>>> those triggered by bytecode, > >>>>> JNI methods, and VM events. > >>>>> > >>>>> > >>>>> > >>>>> On Mon, Jun 18, 2018 at 12:57 AM David Holmes > >>>>> > >>>>> > >>>>> > wrote: > >>>>> > >>>>> On 18/06/2018 5:01 PM, Jeremy Manson wrote: > >>>>> > We haven't changed when a VM issues "VM object > >>>>> allocation" events. > >>>>> > > >>>>> > There were references in the docs to a requirement to use > >>>>> bytecode > >>>>> > rewriting and JNI interception to track allocations. With > >>>>> > SampledObjectAlloc, this is no longer the case - > >>>>> SampledObjectAlloc can > >>>>> > track them. This change is supposed to remove the > >>>>> references to > >>>>> those > >>>>> > requirements, and provide suitable replacement text. > >>>>> > > >>>>> > VM object alloc has specific language about being able to > >>>>> use it to > >>>>> > track allocations that cannot be tracked with bytecode > >>>>> instrumentation > >>>>> > and JNI interception. My goal in rephrasing was to make > >>>>> it clear > >>>>> that, > >>>>> > while you can still use it for this, you can also just use > >>>>> > SampledObjectAlloc for everything. > >>>>> > >>>>> Okay. That doesn't come across clearly to me - sorry. So you > >>>>> will now > >>>>> get both kinds of events for allocations done in the VM? > >>>>> > >>>>> Thanks, > >>>>> David > >>>>> > >>>>> > >>>>> > Jeremy > >>>>> > > >>>>> > On Sun, Jun 17, 2018 at 9:11 PM David Holmes > >>>>> > >>>>> > > >>>>> > >>>>> > >>>>> >> wrote: > >>>>> > > >>>>> > Hi Jeremy, > >>>>> > > >>>>> > On 16/06/2018 2:33 AM, Jeremy Manson wrote: > >>>>> > > Hi all, > >>>>> > > > >>>>> > > There are a number of references in the JVMTI doc > >>>>> to its > >>>>> not doing > >>>>> > > object allocation tracking. Now that JEP 331 has > >>>>> landed, > >>>>> these > >>>>> > > references are obsolete. This patch changes those > >>>>> references > >>>>> > accordingly. > >>>>> > > > >>>>> > > While I was there, I took the liberty of fixing > >>>>> some > >>>>> spelling errors. > >>>>> > > > >>>>> > > As far as I know, these are non-normative > >>>>> changes and > >>>>> modify no > >>>>> > API, so > >>>>> > > they should not require a CSR. > >>>>> > > >>>>> > I'm unclear on the nature of the change to "VM Object > >>>>> Allocation". Does > >>>>> > the existence of SampledObjectAlloc change when a VM > >>>>> should > >>>>> issue "VM > >>>>> > object allocation" events? > >>>>> > > >>>>> > Thanks, > >>>>> > David > >>>>> > > >>>>> > > > >>>>> > > Bug: > >>>>> > > https://bugs.openjdk.java.net/browse/JDK-8205113 > >>>>> > > > >>>>> > > Webrev: > >>>>> > > > >>>>> http://cr.openjdk.java.net/~jmanson/8205113/webrev.00/ > >>>>> > >>>>> > > > >>>>> > > Thanks! > >>>>> > > > >>>>> > > Jeremy > >>>>> > > >>>>> > >>> > >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Wed Jun 20 06:48:40 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 19 Jun 2018 23:48:40 -0700 Subject: RFR 8205113: Update JVMTI doc references to object allocation tracking In-Reply-To: References: <384783b9-9ec2-093a-2662-a7a817849720@oracle.com> <3d01255e-8712-46aa-7659-4ccd7325f81c@oracle.com> <6927237d-ce0b-17c0-68f8-06d05b84e516@oracle.com> <53f576e2-0adb-a6c5-8a3c-026dcb1aa7ad@oracle.com> <8224afa2-a6cb-fee2-8fa3-184828ac7c50@oracle.com> Message-ID: <60899f09-12df-1e6e-7e16-406a60939b7c@oracle.com> An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Wed Jun 20 07:14:45 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 20 Jun 2018 17:14:45 +1000 Subject: RFR(S): 8200720: Print additional information in thread dump (times, allocated bytes etc.) In-Reply-To: <19BE9680-3E6E-4620-B72D-D379F39C1C92@sap.com> References: <56817AB5-88FC-4FB9-A271-7206EA69ECA7@sap.com> <2728581E-0E32-4911-B1A9-79AD6CBA81B0@sap.com> <082ca98f-9a68-15a0-95b5-c4d5c733ab36@oracle.com> <933cdacc-9b6f-c8b5-e244-558f38e7888c@oracle.com> <55806aa8-f09b-e9a8-b512-b9676029c494@oracle.com> <0F497BC6-2569-4EF6-9B59-E24A8E1B3A37@sap.com> <5dc08f65e83f4374b8e1f73f218c6d6d@sap.com> <626701f8-b71b-58dc-f837-e5e1d81ef187@oracle.com> <19BE9680-3E6E-4620-B72D-D379F39C1C92@sap.com> Message-ID: <6ce47417-311b-e6cd-b8cd-d46eb7e04c2f@oracle.com> Hi Gunter, On 19/06/2018 8:51 PM, Haug, Gunter wrote: > Hi all, > > Thanks Chris and Christoph for the reviews! Christoph, I'll incorporate the little improvement you suggested. > David, are you OK with the change as well? Sorry I'd lost track of this one a bit ... The overall approach now seems okay. Some naming/terminology issues: "statistic info" is not very good grammatically. These things are either statistics, or statistical information. So e.g. class ThreadStatistics ThreadStatistics& statistics() { return _statistics; } etc. "extended_thread_info" should really be print_extended_info (similar to print_concurrent_locks). You don't need "thread" in there when this is always part of a Thread related API. src/hotspot/share/runtime/thread.cpp const_cast(this)-> Why do we need a const cast to invoke a method on ourselves ?? --- src/hotspot/share/runtime/thread.hpp Please put the new #include in alphabetical order. 641 // Printing 642 void print_on(outputStream* st, bool extended_thread_info) const; 643 virtual void print_on(outputStream* st) const { print_on(st, false); } I was expecting to see a default parameter used here rather than adding an overload: virtual void print_on(outputStream* st, bool print_extended_info = false) const { print_on(st, print_extended_info); } 2179 static void print_on(outputStream* st, bool print_stacks, bool internal_format, bool print_concurrent_locks, bool extended_thread_info); Again I expected to see a default parameter here - but I didn't check if all callers themselves take the new parameter. ?? --- src/hotspot/share/runtime/vm_operations.hpp 376 VM_PrintThreads() 377 { _out = tty; _print_concurrent_locks = PrintConcurrentLocks; ; _extended_thread_info = false; } 378 VM_PrintThreads(outputStream* out, bool print_concurrent_locks, bool extended_thread_info) 379 { _out = out; _print_concurrent_locks = print_concurrent_locks; _extended_thread_info = extended_thread_info; } 380 VMOp_Type type() const 381 { return VMOp_PrintThreads; } Style nits: either keep everything on one line as before (though I agree the lines are now too long) or else the style should be: 380 VMOp_Type type() const { 381 return VMOp_PrintThreads; 382 } Also unclear why (existing) VM_PrintThreads constructor doesn't use initializer list (like VM_PrintMetadata below it) or default parameters? --- src/hotspot/share/runtime/threadStatisticInfo.hpp There are no includes in this file. It should include the appropriate std header for type definitions, and os.hpp. I'm not sure if class ThreadStatisticInfo needs an allocation type as a super type. Might be worth adding a constructor to give a default value so that you can tell if the statistics have been initialized when they appear in the printout. ? --- Thanks, David > Thanks again, > Gunter > > ?On 12.06.18, 01:13, "Chris Plummer" wrote: > > Hi Gunter, > > The changes look fine. I can live with the options parsing in > attachListener.cpp. As you point out, unrecognized options were already > silently ignored. > > thanks, > > Chris > > On 6/8/18 7:05 AM, Haug, Gunter wrote: > > Hi all, > > > > thanks a lot for all the input! I have prepared a new version of the webrev incorporating the suggestions you made (at least I tried): > > > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v3/ > > > > This version outputs the thread times unconditionally while the other information is guarded by a flags. I think, most participants found the thread times the most valuable information and had no (strong) objections to adding them unconditionally. > > > > @David > > Implementation is much simpler with conditional output only for JavaThreads. I could get rid of the terrible hack (changing the flag) without having to change too many source files. > > > > The information on allocated bytes is present in the Thread class already before this proposed change, I just print it. It might be sensible to move _allocated_bytes and the respective methods to the ThreadStatisticInfo class as G?tz suggested. I haven?t done that in the current version, though. > > > > @David and Thomas > > I've removed the pthread-id output. I'm unsure myself what it could be good for. Anyway, we could add it with a separate change (with a better implementation) if there is a need to. > > > > @Chris > > As you have written, the user has no direct contact to the attach listener of the VM and jstack won't misinterpret e.g. -help. I agree that the parsing in attachListener.cpp could be more robust. However, it hasn't been so far either. All that is done in the current implementation is a strcmp to -l everything else will be silently ignored. I can try to make this more robust or we could say that the new output is only available via jcmd. OTH nothing bad can happen with the current version of the proposed change, so we could also leave it as it is. What would you prefer? > > > > @Kirk and Thomas > > Implementing a new diagnostic command is certainly well worth a thought. However, it appears to me that it's not necessary in this case. There are already flags to jstack and jcmd Thread.print, so it's not that uncommon. The amount of information that would be available by a new diagnostic command and that is now added to the thread dump is quite small and I think that is tolerable. Moreover, and most importantly, our support team is use to find the information in the thread dump and it would make things easier to us if I could leave it there. > > > > Thanks again and have a nice weekend, > > Gunter > > > > > > On 06.06.18, 04:56, "David Holmes" wrote: > > > > Hi Goetz, > > > > On 5/06/2018 11:07 PM, Lindenmaier, Goetz wrote: > > > Hi > > > > > > this discussion touched a lot of points so far, which seem > > > to lead to different conclusions. > > > > > > I think we should look at the values printed: > > > > > > 1. cpu=6300.65ms elapsed=123.28s > > > Overhead > > > cpu time: > > > * system calls at thread dump time > > > elapsed time: > > > * 1 system call at thread creation time > > > * 1 64-bit field per thread for the thread start time > > > * 1 system call at thread dump time > > > > > > As I understand, JDK-8143176 would have had to get the > > > time at each locking, which is much more time critical > > > than doing this during thread creation. While > > > > Correct. > > > > > the time a lock was held would be much more useful, > > > the ratio here, combined with knowledge about the application, > > > could lead to the conclusion that the thread is wrongly > > > blocked at much lower cost. > > > > Agreed. I see no issue with unconditionally adding this information as > > it should address some of the concerns of 8143176 as well. > > > > > > > > 2. pthread-id=0x109708000 > > > Overhead: > > > * a field access at thread dump time. Negligible I would say. > > > > I'd overlooked this addition, but it is of course the chunk of ifdef > > code in osThread.cpp that I objected to. The problem I have here is the > > convoluted mess of "thread identifiers" that we already have. We > > currently print: > > > > st->print("tid=" INTPTR_FORMAT " ", p2i(this)); > > > > which is just the address of the Thread/JavaThread object. Then: > > > > st->print("nid=0x%x ", thread_id()); > > > > where 'n' is supposed to represent "native thread id", which is: > > - linux: kernel thread id from syscall gettid > > - solaris: thread library identity from thr_create() or thr_self() > > - windows: thread id from _beginthreadex > > - OS X: kernel thread id from pthread_mach_thread_np(pthread_self()); > > - Other BSD: kernel (?) thread id from syscall thr_self or getthrid > > - AIX: thread library identity from pthread_create() or pthread_self() > > > > It's telling that the code Gunter added gets the thread library id on > > linux, but a "kernel thread id" on other platforms! Do we want to see a > > thread library id and a kernel thread id? IIRC nid is supposed be the id > > you need to see the thread in a debugger. In which case I'm unclear what > > role the thread id Gunter wants to add is playing? I don't think we need > > to see kernel ids in general, and pthread-id isn't useful for debugging > > is it? > > > > > > > > 3. allocated=242236760B defined_classes=1725 > > > Overhead: > > > * 1 64-bit field per thread. > > > * counter increment on class creation > > > > > > Especially defined_classes seems to be controversial. > > > As this only makes sense for Java threads, this could > > > be printed in a line of it's own in Threads::print_on_error() > > > (which already gets a flag to customize for jstack: > > > print_concurrent_locks) by calling a dedicated function in Thread. > > > No need for flag PrintExtendedThreadInfo thus. > > > > > > But I would propose to drop this information as it is > > > too controversial. > > > > Agreed. > > > > > This leaves the times and the pthread-id to be decided whether > > > they are worth the cost. If so, I think they should be printed > > > unconditional. > > > > Agreed. With some refinement of the "thread id" issue. > > > > > If a flag is required to avoid bloating the info in the default case, > > > I would include more infos, as os_prio and nid under that flag. > > > > Of course those things exist for good reason. The native id is supposed > > to be what aids in you matching what you see in our thread dumps with > > what you see in a debugger. Priority is less of an issue these days, but > > there was a time ... ;-) > > > > > As jstack is deprecated, changes to jstack could be skipped always > > > disabling the new printouts. > > > > > > Finally, I would propose to move _allocated_bytes into > > > threadStatisticInfo.hpp. > > > > Didn't that part get dropped? (You don't need it for the time info or > > thread id). That's a GC logging issue to me. > > > > Thanks, > > David > > > > > Best regards, > > > Goetz. > > > > > > > > > > > > > > >> -----Original Message----- > > >> From: serviceability-dev [mailto:serviceability-dev- > > >> bounces at openjdk.java.net] On Behalf Of David Holmes > > >> Sent: Dienstag, 5. Juni 2018 04:53 > > >> To: Haug, Gunter ; Chris Plummer > > >> ; serviceability-dev > >> dev at openjdk.java.net>; Langer, Christoph ; > > >> hotspot-runtime-dev at openjdk.java.net > > >> Subject: Re: RFR(S): 8200720: Print additional information in thread dump > > >> (times, allocated bytes etc.) > > >> > > >> Hi Gunter, > > >> > > >> On 5/06/2018 1:27 AM, Haug, Gunter wrote: > > >>> Hi David, Chris, > > >>> > > >>> would it be an option to unconditionally print the additional information? > > >> Regardless which way this is implemented it will be rather complicated and it > > >> only switches on/off a few field in the thread dump. > > >> > > >> I'm not convinced this is all suitable information for a thread stack > > >> dump. I can see the time information being useful if using the dump to > > >> try and diagnoze a hang or responsiveness issue. But the allocated-bytes > > >> and classes-defined just doesn't seem useful in the context of a thread > > >> dump to me. Anyone interested in allocation stats is going to be looking > > >> at GC logs etc which is where this belongs. Ditto the class-stats belong > > >> in some kind of classloading logging IMHO. > > >> > > >> I'm very reluctant to burden the implementation with capturing these > > >> kinds of stats, just to use for diagnostic purposes that may not even be > > >> asked for. I'm very much in the "you shouldn't pay for what you don't > > >> use" camp in that regard. (See my comments in JDK-8143176 referenced by > > >> Sean.) > > >> > > >> The ThreadStatisticInfo adds overhead to every thread object - and I'd > > >> have to suspect there could be overlap with whatever flight recorder > > >> sticks in there too. (Thread has become even more bloated in recent time!). > > >> > > >> Cheers, > > >> David > > >> > > >> > > >>> Thanks, > > >>> Gunter > > >>> > > >>> On 04.06.18, 01:13, "David Holmes" wrote: > > >>> > > >>> Hi Gunter, Chris, > > >>> > > >>> Sorry just trying to catch up and this is only a partial look so far ... > > >>> > > >>> BTW these changes are not limited to serviceability code so adding in > > >>> runtime team as well. > > >>> > > >>> On 2/06/2018 9:12 AM, Chris Plummer wrote: > > >>> > Hi Gunter, > > >>> > > > >>> > On 6/1/18 3:17 AM, Haug, Gunter wrote: > > >>> >> Hi Chris, > > >>> >> > > >>> >> thanks for looking into this! > > >>> >> > > >>> >> Re the synchronization: The value is stored only in a VM operation at > > >>> >> a safepoint by the VM thread. I was of the opinion, that this could > > >>> >> not be interrupted by a second VM operation (of the same type). Or > > >> am > > >>> >> I missing something else? > > >>> > I was really thinking more about the temporary changing of > > >>> > PrintExtendedThreadInfo, not the value stored in the VMOp. You may > > >> be be > > >>> > correct that no more than one VMOp is executing, but while it is > > >>> > executing it is has changed the value of PrintExtendedThreadInfo, > > >> which > > >>> > might have an impact on anything else that triggers printing thread info > > >>> > while the VMOp is executing. > > >>> > > >>> Even if nothing else can possibly be running during the safepoint I find > > >>> it extremely bad form to change a command-line flag in this way, > > >>> particularly from a safepoint! > > >>> > > >>> If this flag is intended to be dynamically enabled as part of a dcmd > > >>> then it should also be a Manageable flag IMHO. > > >>> > > >>> That said ... > > >>> > > >>> >> I did think about passing an argument to the various print_on > > >> member > > >>> >> functions of the thread classes, but this would require rather > > >>> >> extensive code changes for a rather tiny extension. Therefore I feel > > >>> >> doing it like this is the lesser evil. > > >>> > > >>> ... it's obviously not truly a global setting, but one that is needed on > > >>> a per-print-request basis. The flag is just the default, but if the > > >>> default is off you still want to enable extended printing on a > > >>> per-request basis. > > >>> > > >>> The current print_on mechanics is not set up for this kind of > > >>> flexibility. I think this needs more thought. > > >>> > > >>> --- > > >>> > > >>> Re osThread.cpp shared code changes ... I really hate to see all the > > >>> ifdefs in there. Please add a pd_print_on function to support this if > > >>> you truly want platform specific information. > > >>> > > >>> --- > > >>> > > >>> threadStatisticInfo.hpp > > >>> > > >>> typo: getElepsedTime() -> getElapsedTime() > > >>> > > >>> Thanks, > > >>> David > > >>> > > >>> > > >>> > > >>> >> > > >>> >> Re thread_dump(): I think it's correct or, well, at least it works ;-) > > >>> >> In fact jstack will transfer the "-e" and "-l" in only one string, > > >>> >> i.e. op->arg(0). > > >>> > So you are saying that op->arg(0) is "-e -l" when using jstack? I think > > >>> > you really need to clean up the parsing. As it stands right now passing, > > >>> > I get the feeling that if a user erroneously asks for help by using > > >>> > "jcmd Thread.Print -help", it will end up executing with -e an -l > > >>> > enabled, and no failure for the invalid "-help" option. > > >>> >> Christoph has already explained my reasoning. But I agree, it's not > > >>> >> nice and I would be happy to do it like Christoph suggested. > > >>> > I'll respond separately to his suggestion. > > >>> > > > >>> > thanks, > > >>> > > > >>> > Chris > > >>> >> > > >>> >> And typo fixed, sorry. > > >>> >> > > >>> >> Thanks again, > > >>> >> Gunter > > >>> >> > > >>> >> On 01.06.18, 00:03, "Chris Plummer" > > >> wrote: > > >>> >> > > >>> >> Hi Gunter, > > >>> >> globals.hpp: fix typo "informatiuon" > > >>> >> I worry a little bit about the synchronizing (if that's the right > > >>> >> word) > > >>> >> of PrintExtendedThreadInfo and the dcmd's -e flag. When using -e, > > >>> >> you > > >>> >> are temporarily enabling PrintExtendedThreadInfo if it was false. > > >>> >> This > > >>> >> temporarily changes the behavior of thread dumps, and could > > >>> >> impact other > > >>> >> uses that happen in parallel. Also, could two simultaneous uses > > >>> >> of -e > > >>> >> result in PrintExtendedThreadInfo not getting restored properly? > > >>> >> thread_dump() doesn't look right. It looks like you are iterating > > >>> >> char > > >>> >> by char over the argument, and expect something like "-el" to be > > >>> >> specified rather then "-e -l". The loop should be iterating over > > >>> >> op->arg(i), not op->arg(0)[i]. > > >>> >> The rest of the changes look fine. > > >>> >> thanks, > > >>> >> Chris > > >>> >> On 5/30/18 8:12 AM, Haug, Gunter wrote: > > >>> >> > Hi all, > > >>> >> > > > >>> >> > As Chris proposed, I have made an the extended output > > >>> >> switchable. There is an VM flag (PrintExtendedThreadInfo), which is > > >>> >> false by default. Moreover, there is an Option (-e) which can be used > > >>> >> with jcmd Thread.print as well as with jstack. The -e option > > >>> >> essentially sets PrintExtendedThreadInfo true just for the respective > > >>> >> thread dump. > > >>> >> > > > >>> >> > Here is the updated webrev: > > >>> >> > > > >>> >> > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v2 > > >>> >> > > > >>> >> > (https://bugs.openjdk.java.net/browse/JDK-8200720) > > >>> >> > > > >>> >> > Thanks, > > >>> >> > Gunter > > >>> >> > > > >>> >> > > > >>> >> > On 02.05.18, 17:07, "serviceability-dev on behalf of Haug, > > >>> >> Gunter" > >>> >> gunter.haug at sap.com> wrote: > > >>> >> > > > >>> >> > Hi Chris, > > >>> >> > > > >>> >> > Thanks for looking into this. > > >>> >> > You're right, there is a little more we have. We have > > >>> >> implemented an IO tracing mechanism which - rather as a byproduct - > > >>> >> keeps track of bytes read and written per thread. However, this of > > >>> >> course requires changes not only in hotspot. We would be happy to > > >>> >> contribute this as well, but this is a far bigger change and will > > >>> >> probably lead to a far bigger discussion. Anyway, with the number of > > >>> >> bytes read, the number of classes defined doesn't look that arbitrary > > >>> >> anymore, as one can correlate IO to class loading. > > >>> >> > > > >>> >> > Regarding the verbose option I think that's a good idea! > > >>> >> > > > >>> >> > Thanks again, > > >>> >> > Gunter > > >>> >> > > > >>> >> > On 01.05.18, 22:55, "Chris Plummer" > > >>> >> wrote: > > >>> >> > > > >>> >> > Hi Gunter, > > >>> >> > > > >>> >> > The output you are adding is all useful. I think the > > >>> >> question is (and > > >>> >> > I'd like to see a few people chime in on this for this > > >>> >> review) is > > >>> >> > whether or not all of it is the appropriate for a > > >>> >> thread's stack dump. > > >>> >> > For example, defined_classes is on the fringe of what > > >>> >> I would call > > >>> >> > generally useful info in a stack dump. Sure, there > > >>> >> might be that rare > > >>> >> > case when you need it, but how often compared to other > > >>> >> useful info > > >>> >> > maintained on a per thread basis. How many other bits > > >>> >> of useful info are > > >>> >> > not being printed in favor of defined_classes? It > > >>> >> seems you have more in > > >>> >> > the queue. How many? I'm worried about how cluttered > > >>> >> the stack dumps > > >>> >> > will get. Maybe we should add some sort of verbose > > >>> >> thread dumping > > >>> >> > option. Just a thought. > > >>> >> > > > >>> >> > As for the implementation, overall it looks good, but > > >>> >> I can't speak to > > >>> >> > whether or not you are doing proper accounting of > > >>> >> defined_classes and > > >>> >> > bytes allocated. You'll need input from someone with > > >>> >> more knowledge of > > >>> >> > those areas. We'll also need to do some testing to > > >>> >> make sure tool tests > > >>> >> > are not impacted. > > >>> >> > > > >>> >> > thanks, > > >>> >> > > > >>> >> > Chris > > >>> >> > > > >>> >> > On 4/30/18 2:51 AM, Haug, Gunter wrote: > > >>> >> > > Hi, > > >>> >> > > > > >>> >> > > this is an update to an RFR I posted on hotspot-dev, > > >>> >> but it is probably more suitable to post it here. Can I please have a > > >>> >> review and a sponsor for the following enhancement: > > >>> >> > > > > >>> >> > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v1 > > >>> >> > > https://bugs.openjdk.java.net/browse/JDK-8200720 > > >>> >> > > > > >>> >> > > We at SAP have extended the thread dumps (obtained > > >>> >> by jstack or jcmd) by several fields providing thread specific > > >>> >> information. These extensions are quite popular with our support > > >> team. > > >>> >> With some knowledge of the architecture of the application, they > > >> often > > >>> >> allow for quick and simple diagnosis of a running system. Therefore > > >> we > > >>> >> would like to contribute these enhancements. > > >>> >> > > > > >>> >> > > I took a few simple examples here, namely cpu time, > > >>> >> elapsed time since thread creation, bytes allocated and classes > > >>> >> defined by the thread and the pthread-id or equivalent on platforms > > >>> >> where it makes sense. Provided it is known how the application > > >> should > > >>> >> behave, a misbehaving thread can identified easily. > > >>> >> > > > > >>> >> > > There is no measurable overhead for this > > >>> >> enhancement. However, it may be a problem that the format of the > > >>> >> output is changed. Tools parsing the output may have to be changed. > > >>> >> > > > > >>> >> > > Here is an example of the output generated: > > >>> >> > > > > >>> >> > > ------------------------------------------------------ > > >>> >> > > > > >>> >> > > "main" #1 prio=5 os_prio=31 cpu=6300.65ms > > >>> >> elapsed=123.28s allocated=242236760B defined_classes=1725 > > >>> >> tid=0x00007fa13a806000 nid=0x1c03 pthread-id=0x109708000 waiting > > >> on > > >>> >> condition [0x0000000109707000] > > >>> >> > > java.lang.Thread.State: TIMED_WAITING (sleeping) > > >>> >> > > JavaThread state: _thread_blocked > > >>> >> > > Thread: 0x00007fa13a806000 [0x1c03] State: > > >>> >> _at_safepoint _has_called_back 0 _at_poll_safepoint 0 > > >>> >> > > JavaThread state: _thread_blocked > > >>> >> > > at java.lang.Thread.sleep(java.base/Native Method) > > >>> >> > > ... > > >>> >> > > ------------------------------------------------------ > > >>> >> > > > > >>> >> > > As mentioned above, we have a whole bunch of other > > >>> >> enhancements to the thread dump similar to this one and would be > > >>> >> willing to contribute them if there is any interest. > > >>> >> > > > > >>> >> > > Thanks and best regards, > > >>> >> > > Gunter > > >>> >> > > > > >>> >> > > > > >>> >> > > > >>> >> > > > >>> >> > > > >>> >> > > > >>> >> > > > >>> >> > > >>> > > > >>> > > > >>> > > >>> > > > > > > > > From arno.zeller at sap.com Wed Jun 20 08:11:39 2018 From: arno.zeller at sap.com (Zeller, Arno) Date: Wed, 20 Jun 2018 08:11:39 +0000 Subject: RFR(XS): 8205108: [testbug] Fix pattern matching in jstatd tests. In-Reply-To: References: <4d7ec5ef1e224177af8409e9845e71bb@sap.com> Message-ID: <0e8b07dd83b0449588eb1c318e83855d@sap.com> Hi Goetz, the change looks good to me and I think the newer example output should be enough. Nevertheless, I have to point out that I am no reviewer. Best regards, Arno >-----Original Message----- >From: serviceability-dev [mailto:serviceability-dev- >bounces at openjdk.java.net] On Behalf Of Lindenmaier, Goetz >Sent: Dienstag, 19. Juni 2018 12:46 >To: Thomas St?fe >Cc: serviceability-dev (serviceability-dev at openjdk.java.net) dev at openjdk.java.net> >Subject: [CAUTION] RE: RFR(XS): 8205108: [testbug] Fix pattern matching in >jstatd tests. > >Hi Thomas, > >thanks for your review! > >I think the upper example output stems from some older >jstatd. Probably the output got two new values added. >I don't feel like making up values, so I'd rather remove the >old output. Is that fine with you? > >Best regards, > Goetz. > >> -----Original Message----- >> From: Thomas St?fe [mailto:thomas.stuefe at gmail.com] >> Sent: Samstag, 16. Juni 2018 19:26 >> To: Lindenmaier, Goetz >> Cc: serviceability-dev (serviceability-dev at openjdk.java.net) > dev at openjdk.java.net> >> Subject: Re: RFR(XS): 8205108: [testbug] Fix pattern matching in jstatd tests. >> >> Hi Goetz, >> >> Looks reasonable. Could you please fix up the example output above the >> alternative you added, it misses the columns "CGC" and "CGCT". >> >> Thanks, Thomas >> >> On Sat, Jun 16, 2018 at 9:24 AM, Lindenmaier, Goetz >> wrote: >> > Hi, >> > >> > Please review this test fix: >> > http://cr.openjdk.java.net/~goetz/wr18/8205108- >fixJstatDTestPattern/01/ >> > >> > The values for 'M' must be parsed as PERCENTAGE_OR_DASH. >> > The parsing of numbers should not depend on the locale. >> > I changed the parsing to use java.text.NumberFormat >> > >> > Best regards, >> > Goetz. >> > From harsha.wardhana.b at oracle.com Wed Jun 20 08:14:22 2018 From: harsha.wardhana.b at oracle.com (Harsha Wardhana B) Date: Wed, 20 Jun 2018 13:44:22 +0530 Subject: RFR : JDK-8204661 - Show error 'Port already in use' in HashedPasswordFileTest.java Message-ID: <9fa7bd72-6476-b702-ba34-ec8fc572cfcb@oracle.com> Hi, Please find the fix below for the bug JDK-8204661 : Show error 'Port already in use' in HashedPasswordFileTest.java having webrev at, http://cr.openjdk.java.net/~hb/8204661/webrev.00/ The problem root-cause is discussed in the comments section of the bug. The fix above lets the default agent pick a free port by passing 'port=0' value and then reads the JMX Connector URL from Perf Counters. Please review the fix above and provide comments. Thanks Harsha From goetz.lindenmaier at sap.com Wed Jun 20 08:31:39 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 20 Jun 2018 08:31:39 +0000 Subject: RFR(XS): 8205108: [testbug] Fix pattern matching in jstatd tests. In-Reply-To: <0e8b07dd83b0449588eb1c318e83855d@sap.com> References: <4d7ec5ef1e224177af8409e9845e71bb@sap.com> <0e8b07dd83b0449588eb1c318e83855d@sap.com> Message-ID: <3ac814df479b446392413855d5f757c7@sap.com> Hi Arno, thanks for the 'r'eview! Best regards, Goetz. > -----Original Message----- > From: Zeller, Arno > Sent: Mittwoch, 20. Juni 2018 10:12 > To: Lindenmaier, Goetz > Cc: serviceability-dev (serviceability-dev at openjdk.java.net) dev at openjdk.java.net>; Thomas St?fe > Subject: RE: RFR(XS): 8205108: [testbug] Fix pattern matching in jstatd tests. > > Hi Goetz, > > the change looks good to me and I think the newer example output should > be enough. > Nevertheless, I have to point out that I am no reviewer. > > Best regards, > Arno > > >-----Original Message----- > >From: serviceability-dev [mailto:serviceability-dev- > >bounces at openjdk.java.net] On Behalf Of Lindenmaier, Goetz > >Sent: Dienstag, 19. Juni 2018 12:46 > >To: Thomas St?fe > >Cc: serviceability-dev (serviceability-dev at openjdk.java.net) >dev at openjdk.java.net> > >Subject: [CAUTION] RE: RFR(XS): 8205108: [testbug] Fix pattern matching in > >jstatd tests. > > > >Hi Thomas, > > > >thanks for your review! > > > >I think the upper example output stems from some older > >jstatd. Probably the output got two new values added. > >I don't feel like making up values, so I'd rather remove the > >old output. Is that fine with you? > > > >Best regards, > > Goetz. > > > >> -----Original Message----- > >> From: Thomas St?fe [mailto:thomas.stuefe at gmail.com] > >> Sent: Samstag, 16. Juni 2018 19:26 > >> To: Lindenmaier, Goetz > >> Cc: serviceability-dev (serviceability-dev at openjdk.java.net) > >> dev at openjdk.java.net> > >> Subject: Re: RFR(XS): 8205108: [testbug] Fix pattern matching in jstatd > tests. > >> > >> Hi Goetz, > >> > >> Looks reasonable. Could you please fix up the example output above the > >> alternative you added, it misses the columns "CGC" and "CGCT". > >> > >> Thanks, Thomas > >> > >> On Sat, Jun 16, 2018 at 9:24 AM, Lindenmaier, Goetz > >> wrote: > >> > Hi, > >> > > >> > Please review this test fix: > >> > http://cr.openjdk.java.net/~goetz/wr18/8205108- > >fixJstatDTestPattern/01/ > >> > > >> > The values for 'M' must be parsed as PERCENTAGE_OR_DASH. > >> > The parsing of numbers should not depend on the locale. > >> > I changed the parsing to use java.text.NumberFormat > >> > > >> > Best regards, > >> > Goetz. > >> > From david.holmes at oracle.com Wed Jun 20 11:11:33 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 20 Jun 2018 21:11:33 +1000 Subject: RFR 8205113: Update JVMTI doc references to object allocation tracking In-Reply-To: <60899f09-12df-1e6e-7e16-406a60939b7c@oracle.com> References: <384783b9-9ec2-093a-2662-a7a817849720@oracle.com> <3d01255e-8712-46aa-7659-4ccd7325f81c@oracle.com> <6927237d-ce0b-17c0-68f8-06d05b84e516@oracle.com> <53f576e2-0adb-a6c5-8a3c-026dcb1aa7ad@oracle.com> <8224afa2-a6cb-fee2-8fa3-184828ac7c50@oracle.com> <60899f09-12df-1e6e-7e16-406a60939b7c@oracle.com> Message-ID: On 20/06/2018 4:48 PM, serguei.spitsyn at oracle.com wrote: > On 6/19/18 23:29, Jeremy Manson wrote: >> Maybe we should make that clarification. >> >> Also, the reason I danced around that in my revision is that > > Understand that. > But it is not a good style to clarify about SampledObjectAlloc in the > spec of VMObjectAlloc. > Would be nice to find a way to keep it simple. > > We could keep the original spec as it is but add that all this does not > apply to the SampledObjectAlloc events. I don't think that really presents a coherent spec. I still like my suggestion :) David > > Something like below: > > ? Note, the above does not apply to the id="SampledObjectAlloc">SampledObjectAlloc > ? event as it is triggered on all Java object allocations, including > those caused by bytecode method execution, > ? JNI method execution, and directly by VM methods.* > * > >> if you set the sampling rate to 0, you get unconditional sampling. > A correction. > > I wrote: > > It is still possible to sample every allocation with the SamplingRate > == 1. > > but had write: SamplingRate == 0 > > Thanks, > Serguei > > >> >> Jeremy >> >> On Tue, Jun 19, 2018 at 10:41 PM serguei.spitsyn at oracle.com >> > > wrote: >> >> On 6/19/18 21:54, David Holmes wrote: >> > On 20/06/2018 2:41 PM, serguei.spitsyn at oracle.com >> wrote: >> >> On 6/19/18 21:11, Jeremy Manson wrote: >> >>> That would be okay with me, assuming that my other corrections >> are >> >>> made. >> >> >> >> Another option would be to say "non-sampling" instead of >> >> "unconditional": >> >> >> >> == Sent when a method causes the virtual machine to allocate an >> >> == Object visible to Java programming language code and the >> allocation >> >> += is not detectable by other *non-sampling* instrumentation >> mechanisms. >> >> >> >> >> >>> I'd also like to fix the spelling of instrumentation in the first >> >>> sentence. >> >> >> >> Yes, of course. >> >> >> >> Let's wait for David's opinion. >> > >> > I'm okay with Serguei's suggestion (combined with Jeremy's other >> > changes). >> > >> > I'm not that familiar with conditional versus unconditional >> > instrumentation. >> >> The whole point of the SampledObjectAlloc events is to post them >> conditionally >> depending on the SamplingRate so that just some amount of >> allocations is >> sampled. >> It is why the overhead can be minimal (less than 5 percents). >> It is still possible to sample every allocation with the >> SamplingRate == 1. >> >> The VMObjectAlloc events are posted unconditionally, so every VM >> allocation is posted. >> >> Thanks, >> Serguei >> >> >> > >> > Thanks, >> > David >> > >> >> >> >> Thanks, >> >> Serguei >> >> >> >> >> >>> >> >>> Jeremy >> >>> >> >>> On Mon, Jun 18, 2018 at 11:01 PM serguei.spitsyn at oracle.com >> >> >>> > > > >> >>> > >> wrote: >> >>> >> >>> ??? Hi Jeremy and David, >> >>> >> >>> ??? Sorry for being late to the party. >> >>> >> >>> ??? I'm also concerned about the Jeremy's spec update is more >> >>> ??? intrusive than necessary. >> >>> ??? One specifics of the new SampledObjectAlloc event is that >> it is >> >>> ??? posted conditionally. >> >>> ??? So, it is not fully comparable with the VMObjectAlloc >> event and >> >>> ??? can not replace it in any way. >> >>> ??? I'm even not yet convinced that any spec update is necessary. >> >>> >> >>> ??? However, something like below would look minimal and >> reasonable: >> >>> >> >>> ??? == Sent when a method causes the virtual machine to >> allocate an >> >>> ??? == Object visible to Java programming language code and the >> >>> allocation >> >>> ??? += is not detectable by other *unconditional* intrumentation >> >>> ??? mechanisms. >> >>> >> >>> ??? == Generally object allocation should be detected by >> instrumenting >> >>> ??? == the bytecodes of allocating methods. >> >>> >> >>> ??? == Object allocation generated in native code by JNI function >> >>> ??? == calls should be detected using >> >>> ??? == JNI function >> >>> ??? interception. >> >>> >> >>> ??? == Some methods might not have associated bytecodes and >> are not >> >>> ??? == native methods, they instead are executed directly by the >> >>> ??? == VM. These methods should send this event. >> >>> >> >>> ??? == Virtual machines which are incapable of bytecode >> instrumentation >> >>> ??? == for some or all of their methods can send this event. >> >>> >> >>> ??? *++ Note that the > >>> id="SampledObjectAlloc">SampledObjectAlloc** >> >>> ??? **++ event is conditionally triggered on all Java object >> >>> ??? allocations, including those** >> >>> ??? **++ caused by bytecode method execution, JNI method >> execution, >> >>> ??? and directly by VM methods.** >> >>> ??? * >> >>> >> >>> ??? Maybe, just adding the last statement would be good enough. >> >>> >> >>> ??? Thanks, >> >>> ??? Serguei >> >>> >> >>> >> >>> ??? On 6/18/18 21:36, David Holmes wrote: >> >>>> ??? On 19/06/2018 4:50 AM, Jeremy Manson wrote: >> >>>>> ??? Yup!? The paragraph meanders a bit. How about something >> like: >> >>>> >> >>>> ??? I'm not sure some of the change quite works. The original >> text >> >>>> ??? considers there to be three kinds of methods that can cause >> >>>> ??? allocation when executed: >> >>>> ??? - Java (bytecode) methods >> >>>> ??? - JNI methods >> >>>> ??? - VM methods >> >>>> >> >>>> ??? but you've turned this into three kinds of allocation: via >> >>>> ??? bytecode, via JNI, and via the VM. You then refer to >> "triggering" >> >>>> ??? an allocation when we tend to use triggering for events. >> You also >> >>>> ??? refer to an allocation being "executed directly by the VM" (a >> >>>> ??? phrase previously applied when the subject was a >> 'method') - but >> >>>> ??? you don't really execute allocations. >> >>>> >> >>>> ??? IIUC the problem with the existing text is just that it >> considers >> >>>> ??? VM allocation events as being undetectable other than by >> this "VM >> >>>> ??? object allocation event" - but that's no longer true. So how >> >>>> ??? about something minimally changed like this: >> >>>> >> >>>> ??? --- >> >>>> ??? ? Sent when a method causes the virtual machine to directly >> >>>> ??? allocate an >> >>>> ??? ? Object visible to Java programming language code. >> >>>> ??? ? Generally object allocation can be detected by >> instrumenting >> >>>> ??? ? the bytecodes of allocating methods. >> >>>> ??? ? Object allocation generated in native code by JNI function >> >>>> ??? ? calls can be detected using >> >>>> ??? ? JNI function >> >>>> ??? interception. >> >>>> ??? ?? Some methods might not have associated bytecodes and >> are not >> >>>> ??? ?? native methods, they instead are executed directly by the >> >>>> ??? ?? VM. These methods should send this event. >> >>>> ??? ?? Virtual machines which are incapable of bytecode >> >>>> instrumentation >> >>>> ??? ?? for some or all of their methods can send this event. >> >>>> >> >>>> ??? ?? Note that the > >>>> id="SampledObjectAlloc">SampledObjectAlloc >> >>>> ??? ?? event is triggered on all Java object allocations, >> including >> >>>> ??? those >> >>>> ??? ?? caused by bytecode method execution, JNI method >> execution, and >> >>>> ??? ?? directly by VM methods. >> >>>> ??? --- >> >>>> >> >>>> ??? Thanks, >> >>>> ??? David >> >>>> >> >>>>> ??? Sent when the virtual machine allocates an >> >>>>> ??? Object visible to Java programming language code without >> using a >> >>>>> ??? new bytecode variant or a JNI method. >> >>>>> ??? Many approaches to tracking object allocation use a >> >>>>> combination of >> >>>>> ??? bytecode-based instrumentation and > >>>>> ??? id="jniIntercept">JNI function >> >>>>> ??? interception to intercept allocations. >> However, >> >>>>> ??? this >> >>>>> ??? approach can leave a number of allocations undetected. >> >>>>> ??? Allocations that are neither >> >>>>> ??? triggered by bytecode nor JNI are executed directly by >> the VM. >> >>>>> ??? When those allocations occur, the VM should send this event. >> >>>>> ??? Virtual machines that are incapable of bytecode >> instrumentation >> >>>>> ??? for some or all of their methods may also send this event. >> >>>>> ???

>> >>>>> ??? Note that the > >>>>> id="SampledObjectAlloc">SampledObjectAlloc >> >>>>> ??? event is triggered on all Java object allocations, including >> >>>>> ??? those triggered by bytecode, >> >>>>> ??? JNI methods, and VM events. >> >>>>> >> >>>>> >> >>>>> >> >>>>> ??? On Mon, Jun 18, 2018 at 12:57 AM David Holmes >> >>>>> ??? > > > >> >>>>> ??? > > >> >>>>> ??? > >> wrote: >> >>>>> >> >>>>> ??? ??? On 18/06/2018 5:01 PM, Jeremy Manson wrote: >> >>>>> ??? ???? > We haven't changed when a VM issues "VM object >> >>>>> ??? allocation" events. >> >>>>> ??? ???? > >> >>>>> ??? ???? > There were references in the docs to a >> requirement to use >> >>>>> ??? bytecode >> >>>>> ??? ???? > rewriting and JNI interception to track >> allocations.? With >> >>>>> ??? ???? > SampledObjectAlloc, this is no longer the case - >> >>>>> ??? ??? SampledObjectAlloc can >> >>>>> ??? ???? > track them.? This change is supposed to remove the >> >>>>> ??? references to >> >>>>> ??? ??? those >> >>>>> ??? ???? > requirements, and provide suitable replacement text. >> >>>>> ??? ???? > >> >>>>> ??? ???? > VM object alloc has specific language about being >> able to >> >>>>> ??? use it to >> >>>>> ??? ???? > track allocations that cannot be tracked with >> bytecode >> >>>>> ??? ??? instrumentation >> >>>>> ??? ???? > and JNI interception.? My goal in rephrasing was >> to make >> >>>>> ??? it clear >> >>>>> ??? ??? that, >> >>>>> ??? ???? > while you can still use it for this, you can also >> just use >> >>>>> ??? ???? > SampledObjectAlloc for everything. >> >>>>> >> >>>>> ??? ??? Okay. That doesn't come across clearly to me - >> sorry. So you >> >>>>> ??? will now >> >>>>> ??? ??? get both kinds of events for allocations done in the VM? >> >>>>> >> >>>>> ??? ??? Thanks, >> >>>>> ??? ??? David >> >>>>> >> >>>>> >> >>>>> ??? ???? > Jeremy >> >>>>> ??? ???? > >> >>>>> ??? ???? > On Sun, Jun 17, 2018 at 9:11 PM David Holmes >> >>>>> ??? ??? > > > >> >>>>> ??? > > > > >> >>>>> ??? ???? > > >> >>>>> ??? > > >> >>>>> ??? > >>> wrote: >> >>>>> ??? ???? > >> >>>>> ??? ???? >? ? ?Hi Jeremy, >> >>>>> ??? ???? > >> >>>>> ??? ???? >? ? ?On 16/06/2018 2:33 AM, Jeremy Manson wrote: >> >>>>> ??? ???? >? ? ? > Hi all, >> >>>>> ??? ???? >? ? ? > >> >>>>> ??? ???? >? ? ? > There are a number of references in the >> JVMTI doc >> >>>>> ??? to its >> >>>>> ??? ??? not doing >> >>>>> ??? ???? >? ? ? > object allocation tracking.? Now that JEP >> 331 has >> >>>>> ??? landed, >> >>>>> ??? ??? these >> >>>>> ??? ???? >? ? ? > references are obsolete.? This patch >> changes those >> >>>>> ??? references >> >>>>> ??? ???? >? ? ?accordingly. >> >>>>> ??? ???? >? ? ? > >> >>>>> ??? ???? >? ? ? > While I was there, I took the liberty of >> fixing >> >>>>> some >> >>>>> ??? ??? spelling?errors. >> >>>>> ??? ???? >? ? ? > >> >>>>> ??? ???? >? ? ? > As far as I know, these are non-normative >> >>>>> changes and >> >>>>> ??? ??? modify no >> >>>>> ??? ???? >? ? ?API, so >> >>>>> ??? ???? >? ? ? > they should not require a CSR. >> >>>>> ??? ???? > >> >>>>> ??? ???? >? ? ?I'm unclear on the nature of the change to >> "VM Object >> >>>>> ??? ??? Allocation". Does >> >>>>> ??? ???? >? ? ?the existence of SampledObjectAlloc change >> when a VM >> >>>>> ??? should >> >>>>> ??? ??? issue "VM >> >>>>> ??? ???? >? ? ?object allocation" events? >> >>>>> ??? ???? > >> >>>>> ??? ???? >? ? ?Thanks, >> >>>>> ??? ???? >? ? ?David >> >>>>> ??? ???? > >> >>>>> ??? ???? >? ? ? > >> >>>>> ??? ???? >? ? ? > Bug: >> >>>>> ??? ???? >? ? ? > >> https://bugs.openjdk.java.net/browse/JDK-8205113 >> >>>>> ??? ???? >? ? ? > >> >>>>> ??? ???? >? ? ? > Webrev: >> >>>>> ??? ???? >? ? ? > >> >>>>> http://cr.openjdk.java.net/~jmanson/8205113/webrev.00/ >> >> >>>>> >> >>>>> ??? ???? >? ? ? > >> >>>>> ??? ???? >? ? ? > Thanks! >> >>>>> ??? ???? >? ? ? > >> >>>>> ??? ???? >? ? ? > Jeremy >> >>>>> ??? ???? > >> >>>>> >> >>> >> >> >> > From david.holmes at oracle.com Wed Jun 20 11:42:51 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 20 Jun 2018 21:42:51 +1000 Subject: RFR : JDK-8204661 - Show error 'Port already in use' in HashedPasswordFileTest.java In-Reply-To: <9fa7bd72-6476-b702-ba34-ec8fc572cfcb@oracle.com> References: <9fa7bd72-6476-b702-ba34-ec8fc572cfcb@oracle.com> Message-ID: Hi Harsha, On 20/06/2018 6:14 PM, Harsha Wardhana B wrote: > Hi, > > Please find the fix below for the bug > > JDK-8204661 : Show error 'Port already in use' in > HashedPasswordFileTest.java > > having webrev at, > > http://cr.openjdk.java.net/~hb/8204661/webrev.00/ > > The problem root-cause is discussed in the comments section of the bug. > > The fix above lets the default agent pick a free port by passing > 'port=0' value and then reads the JMX Connector URL from Perf Counters. Can you update the bug report with the actual solution chosen. I find it interesting that I can't see any other test using this technique. If port=0 means the port bind can't fail then this may be the technique all tests should be using for reliability - though it's a pity there's no public API supporting reading back the URL. Have you run the test through mach5 repeatedly to check all (Posix) platforms and a range of systems? You can use "--test-repeat 10" for example to run it 10 times on each platform. > Please review the fix above and provide comments. The copyright needs to changed to the two year format: * Copyright (c) 2017, Oracle and/or its affiliates. All rights reserved. becomes * Copyright (c) 2017, 2018, Oracle and/or its affiliates. All rights reserved. Otherwise this seems okay. Thanks, David > Thanks > Harsha > > From gary.adams at oracle.com Wed Jun 20 12:31:33 2018 From: gary.adams at oracle.com (Gary Adams) Date: Wed, 20 Jun 2018 08:31:33 -0400 Subject: RFR: JDK-6545967: sp05t003 failed ResumeThread() due to THREAD_NOT_SUSPENDED In-Reply-To: References: <5B294EA9.5030003@oracle.com> <4f541d71-1a90-cebe-67bb-dd11f85c072b@oracle.com> Message-ID: <5B2A4925.4000704@oracle.com> We do need to consider an early return for the case where the redefine class or suspend thread operation have detected an error. Webrev: http://bussund0416.us.oracle.com/export/users/gradams/work/webrevs/6545967/webrev.01/ 80 public boolean agentMethod() { 81 boolean passed = false; 82 MyThread mt = new MyThread(); 83 try { 84 mt.start(); 85 // check if we can can pop the thread. 86 // we can not do redefine/pop frame on run method. 87 while (!MyThread.resume.get()); 88 // sleep for some few secs to get redefined. 89 while (!isRedefined()) { 90 if (!agentStatus()) { 91 System.out.println("Failed to redefine class"); 92 return passed; 93 } 94 Thread.sleep(100); 95 } 96 popThreadFrame(mt); // pop the frame. 97 resumeThread(mt); // resume the thread. 98 mt.join(); 99 // wait till the other thread completes its execution. 100 System.out.println("Thread state after popping/redefining = " 101 + mt.threadState); 102 } catch(Exception ie) { 103 ie.printStackTrace(); 104 } 105 if ((mt.threadState< 1000)&& 106 agentStatus()) { 107 passed = true; 108 } 109 return passed; 110 } On 6/19/18, 7:53 PM, Chris Plummer wrote: > Hi Gary, > > The sp05t003 changes look fine. Serquei's question about hs203t003 > leads me to ask what happens if the redefine never happens, possibly > due to some error. Seems we are destined for a timeout then (not sure > how long that will take), whereas the current implementation would > report the failure after about 10 seconds. > > thanks, > > Chris > > On 6/19/18 2:26 PM, serguei.spitsyn at oracle.com wrote: >> Hi Gary, >> >> The fix looks good in general. >> >> One comment though: >> >> http://cr.openjdk.java.net/~gadams/6545967/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS203/hs203t003/hs203t003.java.frames.html >> >> 80 public boolean agentMethod() { >> 81 boolean passed = false; >> 82 MyThread mt = new MyThread(); >> 83 try { >> 84 mt.start(); >> 85 // check if we can can pop the thread. >> 86 // we can not do redefine / pop frame on run method. >> 87 while(!MyThread.resume.get()); >> 88 // sleep for some few secs to get redefined. >> 89 while(!isRedefined()) { >> 90 Thread.sleep(100); >> 91 } >> 92 popThreadFrame(mt); // pop the frame. >> 93 resumeThread(mt); // resume the thread. >> 94 mt.join(); >> 95 // wait till the other thread completes its execution. >> 96 System.out.println(" Thread state after poping / redefining = "+mt.threadState); >> 97 } catch(Exception ie) { >> 98 ie.printStackTrace(); >> 99 } >> 100 if ( ( mt.threadState< 1000 )&& >> 101 ( redefineAttempted()&& isRedefined())&& >> 102 agentStatus() ) { >> 103 passed = true; >> 104 } >> 105 return passed; >> 106 } >> >> It seems, the two checks ( redefineAttempted() && isRedefined()) at >> L101 are not needed any more. >> Now, there is no way out of the while loop at the line 89. >> >> Thanks, >> Serguei >> >> >> >> On 6/19/18 11:42, Gary Adams wrote: >>> There are some rare race conditions that impact some jvmti tests that >>> suspend and resume threads. These tests were recently moved into >>> the open repos. >>> >>> Webrev: http://cr.openjdk.java.net/~gadams/6545967/webrev.00/ >>> >>> The fix in hs203t003 replaces a blind 10 second sleep, with a specific >>> check to wait for a redefineClass to be performed, which takes place as >>> part of callbackFieldAccess. This let's the rest of the operations >>> be performed >>> at a known point in the test sequence. >>> >>> The fix in sp05t003 moves the incrementing of the counters to after the >>> suspend thread calls. The agentProc function is watching the >>> counters to know >>> when to continue processing and eventually resuming of the suspended >>> threads. >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From milan.mimica at gmail.com Wed Jun 20 08:58:17 2018 From: milan.mimica at gmail.com (Milan Mimica) Date: Wed, 20 Jun 2018 10:58:17 +0200 Subject: RFR(S): 8200720: Print additional information in thread dump (times, allocated bytes etc.) In-Reply-To: References: <56817AB5-88FC-4FB9-A271-7206EA69ECA7@sap.com> <2728581E-0E32-4911-B1A9-79AD6CBA81B0@sap.com> <082ca98f-9a68-15a0-95b5-c4d5c733ab36@oracle.com> <933cdacc-9b6f-c8b5-e244-558f38e7888c@oracle.com> <55806aa8-f09b-e9a8-b512-b9676029c494@oracle.com> <0F497BC6-2569-4EF6-9B59-E24A8E1B3A37@sap.com> <5dc08f65e83f4374b8e1f73f218c6d6d@sap.com> <626701f8-b71b-58dc-f837-e5e1d81ef187@oracle.com> Message-ID: pet, 8. lip 2018. u 17:36 Haug, Gunter napisao je: > Hi all, > > thanks a lot for all the input! I have prepared a new version of the > webrev incorporating the suggestions you made (at least I tried): > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v3/ + st->print("allocated=" JLONG_FORMAT "B ", allocated_bytes); Can you please make this easier for humans, ie use byte_size_in_proper_unit() and proper_unit_for_byte_size()? -------------- next part -------------- An HTML attachment was scrubbed... URL: From goetz.lindenmaier at sap.com Wed Jun 20 13:49:46 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 20 Jun 2018 13:49:46 +0000 Subject: RFR(S/M): 8205419: [testbug] TestJmapCore failing without SA: introduce @requires vm.hasSA Message-ID: <112db96d59084bdfafba2174e121c409@sap.com> Hi, TestJmapCore is failing on aix because there jhsdb is not implemented. So far, such tests check at runtime Platform.shouldSAAttach() which is missing in TestJmapCore. Instead, I implement @requires vm.hasSAandCanAttach. This makes jtreg skip the tests without compiling and starting them. I added the new property to all tests that check shouldSAAttach in jdk and hostpot test directories. This is the implementation of the new property, the rest is just repetitive checking it: http://cr.openjdk.java.net/~goetz/wr18/8205419-requiresHasSA/01/test/jtreg-ext/requires/VMProps.java.udiff.html webrev: http://cr.openjdk.java.net/~goetz/wr18/8205419-requiresHasSA/01/ Is it correct to assume the VMProps is evaluated on the same machine and with the same user as used for executing the test? canPtraceAttachLinux directly accesses the system. (Should I remove the checks for shouldSAAttach() from the changed tests?) Best regards, Goetz. From gunter.haug at sap.com Wed Jun 20 15:28:54 2018 From: gunter.haug at sap.com (Haug, Gunter) Date: Wed, 20 Jun 2018 15:28:54 +0000 Subject: RFR(S): 8200720: Print additional information in thread dump (times, allocated bytes etc.) In-Reply-To: <6ce47417-311b-e6cd-b8cd-d46eb7e04c2f@oracle.com> References: <56817AB5-88FC-4FB9-A271-7206EA69ECA7@sap.com> <2728581E-0E32-4911-B1A9-79AD6CBA81B0@sap.com> <082ca98f-9a68-15a0-95b5-c4d5c733ab36@oracle.com> <933cdacc-9b6f-c8b5-e244-558f38e7888c@oracle.com> <55806aa8-f09b-e9a8-b512-b9676029c494@oracle.com> <0F497BC6-2569-4EF6-9B59-E24A8E1B3A37@sap.com> <5dc08f65e83f4374b8e1f73f218c6d6d@sap.com> <626701f8-b71b-58dc-f837-e5e1d81ef187@oracle.com> <19BE9680-3E6E-4620-B72D-D379F39C1C92@sap.com> <6ce47417-311b-e6cd-b8cd-d46eb7e04c2f@oracle.com> Message-ID: <3795D1AE-CBF4-4BC3-B707-ED77D4DE00D0@sap.com> Hi David, Thanks for taking the time to look into this! > "statistic info" is not very good grammatically. These things are either > statistics, or statistical information. So e.g. > > class ThreadStatistics >... OK, I'll change that! > const_cast(this)->cooked_allocated_bytes(); > > Why do we need a const cast to invoke a method on ourselves ?? cooked_allocated_bytes() is not declared const and we can't make it, as it uses OrderAccess::load_acquire(&_allocated_bytes). Therefor the const_cast. > src/hotspot/share/runtime/thread.hpp > > Please put the new #include in alphabetical order. Wilco! > I was expecting to see a default parameter used here rather than adding > an overload: > virtual void print_on(outputStream* st, bool print_extended_info = > false) const { print_on(st, print_extended_info); } Thread inherits from AllocatedObj (in dbg, at least) where virtual void AllocatedObj::print_on(outputStream*) const is declared. Some compilers, e.g. gcc, will complain that this is hidden by Thread::print_on(outputStream*, bool). Others, e.g. clang, are happy with that. We could implement a method with a different name but would that be nicer? > 2179 static void print_on(outputStream* st, bool print_stacks, bool > internal_format, bool print_concurrent_locks, bool extended_thread_info); > > Again I expected to see a default parameter here - but I didn't check if > all callers themselves take the new parameter. ?? This one has just one single caller, so I think it's OK. > src/hotspot/share/runtime/vm_operations.hpp >... I'll change it to conform to the guidelines. > src/hotspot/share/runtime/threadStatisticInfo.hpp > > There are no includes in this file. It should include the appropriate > std header for type definitions, and os.hpp. Agreed, I'll add the includes. > I'm not sure if class ThreadStatisticInfo needs an allocation type as a > super type. > > Might be worth adding a constructor to give a default value so that you > can tell if the statistics have been initialized when they appear in the > printout. ? ThreadStatisticInfo is a data member of Thread. Currently it is initialized in Thread::Thread, but I can add a constructor, if you prefer. Thanks again, Gunter ?On 20.06.18, 09:15, "David Holmes" wrote: Hi Gunter, On 19/06/2018 8:51 PM, Haug, Gunter wrote: > Hi all, > > Thanks Chris and Christoph for the reviews! Christoph, I'll incorporate the little improvement you suggested. > David, are you OK with the change as well? Sorry I'd lost track of this one a bit ... The overall approach now seems okay. Some naming/terminology issues: "statistic info" is not very good grammatically. These things are either statistics, or statistical information. So e.g. class ThreadStatistics ThreadStatistics& statistics() { return _statistics; } etc. "extended_thread_info" should really be print_extended_info (similar to print_concurrent_locks). You don't need "thread" in there when this is always part of a Thread related API. src/hotspot/share/runtime/thread.cpp const_cast(this)-> Why do we need a const cast to invoke a method on ourselves ?? --- src/hotspot/share/runtime/thread.hpp Please put the new #include in alphabetical order. 641 // Printing 642 void print_on(outputStream* st, bool extended_thread_info) const; 643 virtual void print_on(outputStream* st) const { print_on(st, false); } I was expecting to see a default parameter used here rather than adding an overload: virtual void print_on(outputStream* st, bool print_extended_info = false) const { print_on(st, print_extended_info); } 2179 static void print_on(outputStream* st, bool print_stacks, bool internal_format, bool print_concurrent_locks, bool extended_thread_info); Again I expected to see a default parameter here - but I didn't check if all callers themselves take the new parameter. ?? --- src/hotspot/share/runtime/vm_operations.hpp 376 VM_PrintThreads() 377 { _out = tty; _print_concurrent_locks = PrintConcurrentLocks; ; _extended_thread_info = false; } 378 VM_PrintThreads(outputStream* out, bool print_concurrent_locks, bool extended_thread_info) 379 { _out = out; _print_concurrent_locks = print_concurrent_locks; _extended_thread_info = extended_thread_info; } 380 VMOp_Type type() const 381 { return VMOp_PrintThreads; } Style nits: either keep everything on one line as before (though I agree the lines are now too long) or else the style should be: 380 VMOp_Type type() const { 381 return VMOp_PrintThreads; 382 } Also unclear why (existing) VM_PrintThreads constructor doesn't use initializer list (like VM_PrintMetadata below it) or default parameters? --- src/hotspot/share/runtime/threadStatisticInfo.hpp There are no includes in this file. It should include the appropriate std header for type definitions, and os.hpp. I'm not sure if class ThreadStatisticInfo needs an allocation type as a super type. Might be worth adding a constructor to give a default value so that you can tell if the statistics have been initialized when they appear in the printout. ? --- Thanks, David > Thanks again, > Gunter > > On 12.06.18, 01:13, "Chris Plummer" wrote: > > Hi Gunter, > > The changes look fine. I can live with the options parsing in > attachListener.cpp. As you point out, unrecognized options were already > silently ignored. > > thanks, > > Chris > > On 6/8/18 7:05 AM, Haug, Gunter wrote: > > Hi all, > > > > thanks a lot for all the input! I have prepared a new version of the webrev incorporating the suggestions you made (at least I tried): > > > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v3/ > > > > This version outputs the thread times unconditionally while the other information is guarded by a flags. I think, most participants found the thread times the most valuable information and had no (strong) objections to adding them unconditionally. > > > > @David > > Implementation is much simpler with conditional output only for JavaThreads. I could get rid of the terrible hack (changing the flag) without having to change too many source files. > > > > The information on allocated bytes is present in the Thread class already before this proposed change, I just print it. It might be sensible to move _allocated_bytes and the respective methods to the ThreadStatisticInfo class as G?tz suggested. I haven?t done that in the current version, though. > > > > @David and Thomas > > I've removed the pthread-id output. I'm unsure myself what it could be good for. Anyway, we could add it with a separate change (with a better implementation) if there is a need to. > > > > @Chris > > As you have written, the user has no direct contact to the attach listener of the VM and jstack won't misinterpret e.g. -help. I agree that the parsing in attachListener.cpp could be more robust. However, it hasn't been so far either. All that is done in the current implementation is a strcmp to -l everything else will be silently ignored. I can try to make this more robust or we could say that the new output is only available via jcmd. OTH nothing bad can happen with the current version of the proposed change, so we could also leave it as it is. What would you prefer? > > > > @Kirk and Thomas > > Implementing a new diagnostic command is certainly well worth a thought. However, it appears to me that it's not necessary in this case. There are already flags to jstack and jcmd Thread.print, so it's not that uncommon. The amount of information that would be available by a new diagnostic command and that is now added to the thread dump is quite small and I think that is tolerable. Moreover, and most importantly, our support team is use to find the information in the thread dump and it would make things easier to us if I could leave it there. > > > > Thanks again and have a nice weekend, > > Gunter > > > > > > On 06.06.18, 04:56, "David Holmes" wrote: > > > > Hi Goetz, > > > > On 5/06/2018 11:07 PM, Lindenmaier, Goetz wrote: > > > Hi > > > > > > this discussion touched a lot of points so far, which seem > > > to lead to different conclusions. > > > > > > I think we should look at the values printed: > > > > > > 1. cpu=6300.65ms elapsed=123.28s > > > Overhead > > > cpu time: > > > * system calls at thread dump time > > > elapsed time: > > > * 1 system call at thread creation time > > > * 1 64-bit field per thread for the thread start time > > > * 1 system call at thread dump time > > > > > > As I understand, JDK-8143176 would have had to get the > > > time at each locking, which is much more time critical > > > than doing this during thread creation. While > > > > Correct. > > > > > the time a lock was held would be much more useful, > > > the ratio here, combined with knowledge about the application, > > > could lead to the conclusion that the thread is wrongly > > > blocked at much lower cost. > > > > Agreed. I see no issue with unconditionally adding this information as > > it should address some of the concerns of 8143176 as well. > > > > > > > > 2. pthread-id=0x109708000 > > > Overhead: > > > * a field access at thread dump time. Negligible I would say. > > > > I'd overlooked this addition, but it is of course the chunk of ifdef > > code in osThread.cpp that I objected to. The problem I have here is the > > convoluted mess of "thread identifiers" that we already have. We > > currently print: > > > > st->print("tid=" INTPTR_FORMAT " ", p2i(this)); > > > > which is just the address of the Thread/JavaThread object. Then: > > > > st->print("nid=0x%x ", thread_id()); > > > > where 'n' is supposed to represent "native thread id", which is: > > - linux: kernel thread id from syscall gettid > > - solaris: thread library identity from thr_create() or thr_self() > > - windows: thread id from _beginthreadex > > - OS X: kernel thread id from pthread_mach_thread_np(pthread_self()); > > - Other BSD: kernel (?) thread id from syscall thr_self or getthrid > > - AIX: thread library identity from pthread_create() or pthread_self() > > > > It's telling that the code Gunter added gets the thread library id on > > linux, but a "kernel thread id" on other platforms! Do we want to see a > > thread library id and a kernel thread id? IIRC nid is supposed be the id > > you need to see the thread in a debugger. In which case I'm unclear what > > role the thread id Gunter wants to add is playing? I don't think we need > > to see kernel ids in general, and pthread-id isn't useful for debugging > > is it? > > > > > > > > 3. allocated=242236760B defined_classes=1725 > > > Overhead: > > > * 1 64-bit field per thread. > > > * counter increment on class creation > > > > > > Especially defined_classes seems to be controversial. > > > As this only makes sense for Java threads, this could > > > be printed in a line of it's own in Threads::print_on_error() > > > (which already gets a flag to customize for jstack: > > > print_concurrent_locks) by calling a dedicated function in Thread. > > > No need for flag PrintExtendedThreadInfo thus. > > > > > > But I would propose to drop this information as it is > > > too controversial. > > > > Agreed. > > > > > This leaves the times and the pthread-id to be decided whether > > > they are worth the cost. If so, I think they should be printed > > > unconditional. > > > > Agreed. With some refinement of the "thread id" issue. > > > > > If a flag is required to avoid bloating the info in the default case, > > > I would include more infos, as os_prio and nid under that flag. > > > > Of course those things exist for good reason. The native id is supposed > > to be what aids in you matching what you see in our thread dumps with > > what you see in a debugger. Priority is less of an issue these days, but > > there was a time ... ;-) > > > > > As jstack is deprecated, changes to jstack could be skipped always > > > disabling the new printouts. > > > > > > Finally, I would propose to move _allocated_bytes into > > > threadStatisticInfo.hpp. > > > > Didn't that part get dropped? (You don't need it for the time info or > > thread id). That's a GC logging issue to me. > > > > Thanks, > > David > > > > > Best regards, > > > Goetz. > > > > > > > > > > > > > > >> -----Original Message----- > > >> From: serviceability-dev [mailto:serviceability-dev- > > >> bounces at openjdk.java.net] On Behalf Of David Holmes > > >> Sent: Dienstag, 5. Juni 2018 04:53 > > >> To: Haug, Gunter ; Chris Plummer > > >> ; serviceability-dev > >> dev at openjdk.java.net>; Langer, Christoph ; > > >> hotspot-runtime-dev at openjdk.java.net > > >> Subject: Re: RFR(S): 8200720: Print additional information in thread dump > > >> (times, allocated bytes etc.) > > >> > > >> Hi Gunter, > > >> > > >> On 5/06/2018 1:27 AM, Haug, Gunter wrote: > > >>> Hi David, Chris, > > >>> > > >>> would it be an option to unconditionally print the additional information? > > >> Regardless which way this is implemented it will be rather complicated and it > > >> only switches on/off a few field in the thread dump. > > >> > > >> I'm not convinced this is all suitable information for a thread stack > > >> dump. I can see the time information being useful if using the dump to > > >> try and diagnoze a hang or responsiveness issue. But the allocated-bytes > > >> and classes-defined just doesn't seem useful in the context of a thread > > >> dump to me. Anyone interested in allocation stats is going to be looking > > >> at GC logs etc which is where this belongs. Ditto the class-stats belong > > >> in some kind of classloading logging IMHO. > > >> > > >> I'm very reluctant to burden the implementation with capturing these > > >> kinds of stats, just to use for diagnostic purposes that may not even be > > >> asked for. I'm very much in the "you shouldn't pay for what you don't > > >> use" camp in that regard. (See my comments in JDK-8143176 referenced by > > >> Sean.) > > >> > > >> The ThreadStatisticInfo adds overhead to every thread object - and I'd > > >> have to suspect there could be overlap with whatever flight recorder > > >> sticks in there too. (Thread has become even more bloated in recent time!). > > >> > > >> Cheers, > > >> David > > >> > > >> > > >>> Thanks, > > >>> Gunter > > >>> > > >>> On 04.06.18, 01:13, "David Holmes" wrote: > > >>> > > >>> Hi Gunter, Chris, > > >>> > > >>> Sorry just trying to catch up and this is only a partial look so far ... > > >>> > > >>> BTW these changes are not limited to serviceability code so adding in > > >>> runtime team as well. > > >>> > > >>> On 2/06/2018 9:12 AM, Chris Plummer wrote: > > >>> > Hi Gunter, > > >>> > > > >>> > On 6/1/18 3:17 AM, Haug, Gunter wrote: > > >>> >> Hi Chris, > > >>> >> > > >>> >> thanks for looking into this! > > >>> >> > > >>> >> Re the synchronization: The value is stored only in a VM operation at > > >>> >> a safepoint by the VM thread. I was of the opinion, that this could > > >>> >> not be interrupted by a second VM operation (of the same type). Or > > >> am > > >>> >> I missing something else? > > >>> > I was really thinking more about the temporary changing of > > >>> > PrintExtendedThreadInfo, not the value stored in the VMOp. You may > > >> be be > > >>> > correct that no more than one VMOp is executing, but while it is > > >>> > executing it is has changed the value of PrintExtendedThreadInfo, > > >> which > > >>> > might have an impact on anything else that triggers printing thread info > > >>> > while the VMOp is executing. > > >>> > > >>> Even if nothing else can possibly be running during the safepoint I find > > >>> it extremely bad form to change a command-line flag in this way, > > >>> particularly from a safepoint! > > >>> > > >>> If this flag is intended to be dynamically enabled as part of a dcmd > > >>> then it should also be a Manageable flag IMHO. > > >>> > > >>> That said ... > > >>> > > >>> >> I did think about passing an argument to the various print_on > > >> member > > >>> >> functions of the thread classes, but this would require rather > > >>> >> extensive code changes for a rather tiny extension. Therefore I feel > > >>> >> doing it like this is the lesser evil. > > >>> > > >>> ... it's obviously not truly a global setting, but one that is needed on > > >>> a per-print-request basis. The flag is just the default, but if the > > >>> default is off you still want to enable extended printing on a > > >>> per-request basis. > > >>> > > >>> The current print_on mechanics is not set up for this kind of > > >>> flexibility. I think this needs more thought. > > >>> > > >>> --- > > >>> > > >>> Re osThread.cpp shared code changes ... I really hate to see all the > > >>> ifdefs in there. Please add a pd_print_on function to support this if > > >>> you truly want platform specific information. > > >>> > > >>> --- > > >>> > > >>> threadStatisticInfo.hpp > > >>> > > >>> typo: getElepsedTime() -> getElapsedTime() > > >>> > > >>> Thanks, > > >>> David > > >>> > > >>> > > >>> > > >>> >> > > >>> >> Re thread_dump(): I think it's correct or, well, at least it works ;-) > > >>> >> In fact jstack will transfer the "-e" and "-l" in only one string, > > >>> >> i.e. op->arg(0). > > >>> > So you are saying that op->arg(0) is "-e -l" when using jstack? I think > > >>> > you really need to clean up the parsing. As it stands right now passing, > > >>> > I get the feeling that if a user erroneously asks for help by using > > >>> > "jcmd Thread.Print -help", it will end up executing with -e an -l > > >>> > enabled, and no failure for the invalid "-help" option. > > >>> >> Christoph has already explained my reasoning. But I agree, it's not > > >>> >> nice and I would be happy to do it like Christoph suggested. > > >>> > I'll respond separately to his suggestion. > > >>> > > > >>> > thanks, > > >>> > > > >>> > Chris > > >>> >> > > >>> >> And typo fixed, sorry. > > >>> >> > > >>> >> Thanks again, > > >>> >> Gunter > > >>> >> > > >>> >> On 01.06.18, 00:03, "Chris Plummer" > > >> wrote: > > >>> >> > > >>> >> Hi Gunter, > > >>> >> globals.hpp: fix typo "informatiuon" > > >>> >> I worry a little bit about the synchronizing (if that's the right > > >>> >> word) > > >>> >> of PrintExtendedThreadInfo and the dcmd's -e flag. When using -e, > > >>> >> you > > >>> >> are temporarily enabling PrintExtendedThreadInfo if it was false. > > >>> >> This > > >>> >> temporarily changes the behavior of thread dumps, and could > > >>> >> impact other > > >>> >> uses that happen in parallel. Also, could two simultaneous uses > > >>> >> of -e > > >>> >> result in PrintExtendedThreadInfo not getting restored properly? > > >>> >> thread_dump() doesn't look right. It looks like you are iterating > > >>> >> char > > >>> >> by char over the argument, and expect something like "-el" to be > > >>> >> specified rather then "-e -l". The loop should be iterating over > > >>> >> op->arg(i), not op->arg(0)[i]. > > >>> >> The rest of the changes look fine. > > >>> >> thanks, > > >>> >> Chris > > >>> >> On 5/30/18 8:12 AM, Haug, Gunter wrote: > > >>> >> > Hi all, > > >>> >> > > > >>> >> > As Chris proposed, I have made an the extended output > > >>> >> switchable. There is an VM flag (PrintExtendedThreadInfo), which is > > >>> >> false by default. Moreover, there is an Option (-e) which can be used > > >>> >> with jcmd Thread.print as well as with jstack. The -e option > > >>> >> essentially sets PrintExtendedThreadInfo true just for the respective > > >>> >> thread dump. > > >>> >> > > > >>> >> > Here is the updated webrev: > > >>> >> > > > >>> >> > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v2 > > >>> >> > > > >>> >> > (https://bugs.openjdk.java.net/browse/JDK-8200720) > > >>> >> > > > >>> >> > Thanks, > > >>> >> > Gunter > > >>> >> > > > >>> >> > > > >>> >> > On 02.05.18, 17:07, "serviceability-dev on behalf of Haug, > > >>> >> Gunter" > >>> >> gunter.haug at sap.com> wrote: > > >>> >> > > > >>> >> > Hi Chris, > > >>> >> > > > >>> >> > Thanks for looking into this. > > >>> >> > You're right, there is a little more we have. We have > > >>> >> implemented an IO tracing mechanism which - rather as a byproduct - > > >>> >> keeps track of bytes read and written per thread. However, this of > > >>> >> course requires changes not only in hotspot. We would be happy to > > >>> >> contribute this as well, but this is a far bigger change and will > > >>> >> probably lead to a far bigger discussion. Anyway, with the number of > > >>> >> bytes read, the number of classes defined doesn't look that arbitrary > > >>> >> anymore, as one can correlate IO to class loading. > > >>> >> > > > >>> >> > Regarding the verbose option I think that's a good idea! > > >>> >> > > > >>> >> > Thanks again, > > >>> >> > Gunter > > >>> >> > > > >>> >> > On 01.05.18, 22:55, "Chris Plummer" > > >>> >> wrote: > > >>> >> > > > >>> >> > Hi Gunter, > > >>> >> > > > >>> >> > The output you are adding is all useful. I think the > > >>> >> question is (and > > >>> >> > I'd like to see a few people chime in on this for this > > >>> >> review) is > > >>> >> > whether or not all of it is the appropriate for a > > >>> >> thread's stack dump. > > >>> >> > For example, defined_classes is on the fringe of what > > >>> >> I would call > > >>> >> > generally useful info in a stack dump. Sure, there > > >>> >> might be that rare > > >>> >> > case when you need it, but how often compared to other > > >>> >> useful info > > >>> >> > maintained on a per thread basis. How many other bits > > >>> >> of useful info are > > >>> >> > not being printed in favor of defined_classes? It > > >>> >> seems you have more in > > >>> >> > the queue. How many? I'm worried about how cluttered > > >>> >> the stack dumps > > >>> >> > will get. Maybe we should add some sort of verbose > > >>> >> thread dumping > > >>> >> > option. Just a thought. > > >>> >> > > > >>> >> > As for the implementation, overall it looks good, but > > >>> >> I can't speak to > > >>> >> > whether or not you are doing proper accounting of > > >>> >> defined_classes and > > >>> >> > bytes allocated. You'll need input from someone with > > >>> >> more knowledge of > > >>> >> > those areas. We'll also need to do some testing to > > >>> >> make sure tool tests > > >>> >> > are not impacted. > > >>> >> > > > >>> >> > thanks, > > >>> >> > > > >>> >> > Chris > > >>> >> > > > >>> >> > On 4/30/18 2:51 AM, Haug, Gunter wrote: > > >>> >> > > Hi, > > >>> >> > > > > >>> >> > > this is an update to an RFR I posted on hotspot-dev, > > >>> >> but it is probably more suitable to post it here. Can I please have a > > >>> >> review and a sponsor for the following enhancement: > > >>> >> > > > > >>> >> > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v1 > > >>> >> > > https://bugs.openjdk.java.net/browse/JDK-8200720 > > >>> >> > > > > >>> >> > > We at SAP have extended the thread dumps (obtained > > >>> >> by jstack or jcmd) by several fields providing thread specific > > >>> >> information. These extensions are quite popular with our support > > >> team. > > >>> >> With some knowledge of the architecture of the application, they > > >> often > > >>> >> allow for quick and simple diagnosis of a running system. Therefore > > >> we > > >>> >> would like to contribute these enhancements. > > >>> >> > > > > >>> >> > > I took a few simple examples here, namely cpu time, > > >>> >> elapsed time since thread creation, bytes allocated and classes > > >>> >> defined by the thread and the pthread-id or equivalent on platforms > > >>> >> where it makes sense. Provided it is known how the application > > >> should > > >>> >> behave, a misbehaving thread can identified easily. > > >>> >> > > > > >>> >> > > There is no measurable overhead for this > > >>> >> enhancement. However, it may be a problem that the format of the > > >>> >> output is changed. Tools parsing the output may have to be changed. > > >>> >> > > > > >>> >> > > Here is an example of the output generated: > > >>> >> > > > > >>> >> > > ------------------------------------------------------ > > >>> >> > > > > >>> >> > > "main" #1 prio=5 os_prio=31 cpu=6300.65ms > > >>> >> elapsed=123.28s allocated=242236760B defined_classes=1725 > > >>> >> tid=0x00007fa13a806000 nid=0x1c03 pthread-id=0x109708000 waiting > > >> on > > >>> >> condition [0x0000000109707000] > > >>> >> > > java.lang.Thread.State: TIMED_WAITING (sleeping) > > >>> >> > > JavaThread state: _thread_blocked > > >>> >> > > Thread: 0x00007fa13a806000 [0x1c03] State: > > >>> >> _at_safepoint _has_called_back 0 _at_poll_safepoint 0 > > >>> >> > > JavaThread state: _thread_blocked > > >>> >> > > at java.lang.Thread.sleep(java.base/Native Method) > > >>> >> > > ... > > >>> >> > > ------------------------------------------------------ > > >>> >> > > > > >>> >> > > As mentioned above, we have a whole bunch of other > > >>> >> enhancements to the thread dump similar to this one and would be > > >>> >> willing to contribute them if there is any interest. > > >>> >> > > > > >>> >> > > Thanks and best regards, > > >>> >> > > Gunter > > >>> >> > > > > >>> >> > > > > >>> >> > > > >>> >> > > > >>> >> > > > >>> >> > > > >>> >> > > > >>> >> > > >>> > > > >>> > > > >>> > > >>> > > > > > > > > From gunter.haug at sap.com Wed Jun 20 15:31:10 2018 From: gunter.haug at sap.com (Haug, Gunter) Date: Wed, 20 Jun 2018 15:31:10 +0000 Subject: RFR(S): 8200720: Print additional information in thread dump (times, allocated bytes etc.) In-Reply-To: References: <56817AB5-88FC-4FB9-A271-7206EA69ECA7@sap.com> <2728581E-0E32-4911-B1A9-79AD6CBA81B0@sap.com> <082ca98f-9a68-15a0-95b5-c4d5c733ab36@oracle.com> <933cdacc-9b6f-c8b5-e244-558f38e7888c@oracle.com> <55806aa8-f09b-e9a8-b512-b9676029c494@oracle.com> <0F497BC6-2569-4EF6-9B59-E24A8E1B3A37@sap.com> <5dc08f65e83f4374b8e1f73f218c6d6d@sap.com> <626701f8-b71b-58dc-f837-e5e1d81ef187@oracle.com> Message-ID: Yes, good idea, I'll do that. From: Milan Mimica Date: Wednesday, 20. June 2018 at 10:58 To: "Haug, Gunter" Cc: David Holmes , "Lindenmaier, Goetz" , Chris Plummer , serviceability-dev , "Langer, Christoph" , "hotspot-runtime-dev at openjdk.java.net" , Kirk Pepperdine , Thomas St?fe Subject: Re: RFR(S): 8200720: Print additional information in thread dump (times, allocated bytes etc.) + st->print("allocated=" JLONG_FORMAT "B ", allocated_bytes); Can you please make this easier for humans, ie use byte_size_in_proper_unit() and proper_unit_for_byte_size()? From jeremymanson at google.com Wed Jun 20 16:41:15 2018 From: jeremymanson at google.com (Jeremy Manson) Date: Wed, 20 Jun 2018 09:41:15 -0700 Subject: RFR 8205113: Update JVMTI doc references to object allocation tracking In-Reply-To: References: <384783b9-9ec2-093a-2662-a7a817849720@oracle.com> <3d01255e-8712-46aa-7659-4ccd7325f81c@oracle.com> <6927237d-ce0b-17c0-68f8-06d05b84e516@oracle.com> <53f576e2-0adb-a6c5-8a3c-026dcb1aa7ad@oracle.com> <8224afa2-a6cb-fee2-8fa3-184828ac7c50@oracle.com> <60899f09-12df-1e6e-7e16-406a60939b7c@oracle.com> Message-ID: FWIW, I'm also okay with David's version, with the caveat that "These methods should send this event" should probably read "Invocations of these methods should trigger this event". Jeremy On Wed, Jun 20, 2018 at 4:11 AM David Holmes wrote: > On 20/06/2018 4:48 PM, serguei.spitsyn at oracle.com wrote: > > On 6/19/18 23:29, Jeremy Manson wrote: > >> Maybe we should make that clarification. > >> > >> Also, the reason I danced around that in my revision is that > > > > Understand that. > > But it is not a good style to clarify about SampledObjectAlloc in the > > spec of VMObjectAlloc. > > Would be nice to find a way to keep it simple. > > > > We could keep the original spec as it is but add that all this does not > > apply to the SampledObjectAlloc events. > > I don't think that really presents a coherent spec. I still like my > suggestion :) > > David > > > > > Something like below: > > > > Note, the above does not apply to the > id="SampledObjectAlloc">SampledObjectAlloc > > event as it is triggered on all Java object allocations, including > > those caused by bytecode method execution, > > JNI method execution, and directly by VM methods.* > > * > > > >> if you set the sampling rate to 0, you get unconditional sampling. > > A correction. > > > > I wrote: > > > It is still possible to sample every allocation with the SamplingRate > > == 1. > > > > but had write: SamplingRate == 0 > > > > Thanks, > > Serguei > > > > > >> > >> Jeremy > >> > >> On Tue, Jun 19, 2018 at 10:41 PM serguei.spitsyn at oracle.com > >> >> > wrote: > >> > >> On 6/19/18 21:54, David Holmes wrote: > >> > On 20/06/2018 2:41 PM, serguei.spitsyn at oracle.com > >> wrote: > >> >> On 6/19/18 21:11, Jeremy Manson wrote: > >> >>> That would be okay with me, assuming that my other corrections > >> are > >> >>> made. > >> >> > >> >> Another option would be to say "non-sampling" instead of > >> >> "unconditional": > >> >> > >> >> == Sent when a method causes the virtual machine to allocate an > >> >> == Object visible to Java programming language code and the > >> allocation > >> >> += is not detectable by other *non-sampling* instrumentation > >> mechanisms. > >> >> > >> >> > >> >>> I'd also like to fix the spelling of instrumentation in the > first > >> >>> sentence. > >> >> > >> >> Yes, of course. > >> >> > >> >> Let's wait for David's opinion. > >> > > >> > I'm okay with Serguei's suggestion (combined with Jeremy's other > >> > changes). > >> > > >> > I'm not that familiar with conditional versus unconditional > >> > instrumentation. > >> > >> The whole point of the SampledObjectAlloc events is to post them > >> conditionally > >> depending on the SamplingRate so that just some amount of > >> allocations is > >> sampled. > >> It is why the overhead can be minimal (less than 5 percents). > >> It is still possible to sample every allocation with the > >> SamplingRate == 1. > >> > >> The VMObjectAlloc events are posted unconditionally, so every VM > >> allocation is posted. > >> > >> Thanks, > >> Serguei > >> > >> > >> > > >> > Thanks, > >> > David > >> > > >> >> > >> >> Thanks, > >> >> Serguei > >> >> > >> >> > >> >>> > >> >>> Jeremy > >> >>> > >> >>> On Mon, Jun 18, 2018 at 11:01 PM serguei.spitsyn at oracle.com > >> > >> >>> >> > >> > >> >>> >> >> wrote: > >> >>> > >> >>> Hi Jeremy and David, > >> >>> > >> >>> Sorry for being late to the party. > >> >>> > >> >>> I'm also concerned about the Jeremy's spec update is more > >> >>> intrusive than necessary. > >> >>> One specifics of the new SampledObjectAlloc event is that > >> it is > >> >>> posted conditionally. > >> >>> So, it is not fully comparable with the VMObjectAlloc > >> event and > >> >>> can not replace it in any way. > >> >>> I'm even not yet convinced that any spec update is > necessary. > >> >>> > >> >>> However, something like below would look minimal and > >> reasonable: > >> >>> > >> >>> == Sent when a method causes the virtual machine to > >> allocate an > >> >>> == Object visible to Java programming language code and the > >> >>> allocation > >> >>> += is not detectable by other *unconditional* intrumentation > >> >>> mechanisms. > >> >>> > >> >>> == Generally object allocation should be detected by > >> instrumenting > >> >>> == the bytecodes of allocating methods. > >> >>> > >> >>> == Object allocation generated in native code by JNI > function > >> >>> == calls should be detected using > >> >>> == JNI function > >> >>> interception. > >> >>> > >> >>> == Some methods might not have associated bytecodes and > >> are not > >> >>> == native methods, they instead are executed directly by the > >> >>> == VM. These methods should send this event. > >> >>> > >> >>> == Virtual machines which are incapable of bytecode > >> instrumentation > >> >>> == for some or all of their methods can send this event. > >> >>> > >> >>> *++ Note that the >> >>> id="SampledObjectAlloc">SampledObjectAlloc** > >> >>> **++ event is conditionally triggered on all Java object > >> >>> allocations, including those** > >> >>> **++ caused by bytecode method execution, JNI method > >> execution, > >> >>> and directly by VM methods.** > >> >>> * > >> >>> > >> >>> Maybe, just adding the last statement would be good enough. > >> >>> > >> >>> Thanks, > >> >>> Serguei > >> >>> > >> >>> > >> >>> On 6/18/18 21:36, David Holmes wrote: > >> >>>> On 19/06/2018 4:50 AM, Jeremy Manson wrote: > >> >>>>> Yup! The paragraph meanders a bit. How about something > >> like: > >> >>>> > >> >>>> I'm not sure some of the change quite works. The original > >> text > >> >>>> considers there to be three kinds of methods that can cause > >> >>>> allocation when executed: > >> >>>> - Java (bytecode) methods > >> >>>> - JNI methods > >> >>>> - VM methods > >> >>>> > >> >>>> but you've turned this into three kinds of allocation: via > >> >>>> bytecode, via JNI, and via the VM. You then refer to > >> "triggering" > >> >>>> an allocation when we tend to use triggering for events. > >> You also > >> >>>> refer to an allocation being "executed directly by the VM" > (a > >> >>>> phrase previously applied when the subject was a > >> 'method') - but > >> >>>> you don't really execute allocations. > >> >>>> > >> >>>> IIUC the problem with the existing text is just that it > >> considers > >> >>>> VM allocation events as being undetectable other than by > >> this "VM > >> >>>> object allocation event" - but that's no longer true. So > how > >> >>>> about something minimally changed like this: > >> >>>> > >> >>>> --- > >> >>>> Sent when a method causes the virtual machine to directly > >> >>>> allocate an > >> >>>> Object visible to Java programming language code. > >> >>>> Generally object allocation can be detected by > >> instrumenting > >> >>>> the bytecodes of allocating methods. > >> >>>> Object allocation generated in native code by JNI > function > >> >>>> calls can be detected using > >> >>>> JNI function > >> >>>> interception. > >> >>>> Some methods might not have associated bytecodes and > >> are not > >> >>>> native methods, they instead are executed directly by > the > >> >>>> VM. These methods should send this event. > >> >>>> Virtual machines which are incapable of bytecode > >> >>>> instrumentation > >> >>>> for some or all of their methods can send this event. > >> >>>> > >> >>>> Note that the >> >>>> id="SampledObjectAlloc">SampledObjectAlloc > >> >>>> event is triggered on all Java object allocations, > >> including > >> >>>> those > >> >>>> caused by bytecode method execution, JNI method > >> execution, and > >> >>>> directly by VM methods. > >> >>>> --- > >> >>>> > >> >>>> Thanks, > >> >>>> David > >> >>>> > >> >>>>> Sent when the virtual machine allocates an > >> >>>>> Object visible to Java programming language code without > >> using a > >> >>>>> new bytecode variant or a JNI method. > >> >>>>> Many approaches to tracking object allocation use a > >> >>>>> combination of > >> >>>>> bytecode-based instrumentation and >> >>>>> id="jniIntercept">JNI function > >> >>>>> interception to intercept allocations. > >> However, > >> >>>>> this > >> >>>>> approach can leave a number of allocations undetected. > >> >>>>> Allocations that are neither > >> >>>>> triggered by bytecode nor JNI are executed directly by > >> the VM. > >> >>>>> When those allocations occur, the VM should send this > event. > >> >>>>> Virtual machines that are incapable of bytecode > >> instrumentation > >> >>>>> for some or all of their methods may also send this event. > >> >>>>>

> >> >>>>> Note that the >> >>>>> id="SampledObjectAlloc">SampledObjectAlloc > >> >>>>> event is triggered on all Java object allocations, > including > >> >>>>> those triggered by bytecode, > >> >>>>> JNI methods, and VM events. > >> >>>>> > >> >>>>> > >> >>>>> > >> >>>>> On Mon, Jun 18, 2018 at 12:57 AM David Holmes > >> >>>>> >> >> > > >> >>>>> >> > > >> >>>>> >> >> wrote: > >> >>>>> > >> >>>>> On 18/06/2018 5:01 PM, Jeremy Manson wrote: > >> >>>>> > We haven't changed when a VM issues "VM object > >> >>>>> allocation" events. > >> >>>>> > > >> >>>>> > There were references in the docs to a > >> requirement to use > >> >>>>> bytecode > >> >>>>> > rewriting and JNI interception to track > >> allocations. With > >> >>>>> > SampledObjectAlloc, this is no longer the case - > >> >>>>> SampledObjectAlloc can > >> >>>>> > track them. This change is supposed to remove the > >> >>>>> references to > >> >>>>> those > >> >>>>> > requirements, and provide suitable replacement > text. > >> >>>>> > > >> >>>>> > VM object alloc has specific language about being > >> able to > >> >>>>> use it to > >> >>>>> > track allocations that cannot be tracked with > >> bytecode > >> >>>>> instrumentation > >> >>>>> > and JNI interception. My goal in rephrasing was > >> to make > >> >>>>> it clear > >> >>>>> that, > >> >>>>> > while you can still use it for this, you can also > >> just use > >> >>>>> > SampledObjectAlloc for everything. > >> >>>>> > >> >>>>> Okay. That doesn't come across clearly to me - > >> sorry. So you > >> >>>>> will now > >> >>>>> get both kinds of events for allocations done in the > VM? > >> >>>>> > >> >>>>> Thanks, > >> >>>>> David > >> >>>>> > >> >>>>> > >> >>>>> > Jeremy > >> >>>>> > > >> >>>>> > On Sun, Jun 17, 2018 at 9:11 PM David Holmes > >> >>>>> >> >> > > >> >>>>> >> > >> > > >> >>>>> > >> > >> >>>>> >> > > >> >>>>> >> >>> wrote: > >> >>>>> > > >> >>>>> > Hi Jeremy, > >> >>>>> > > >> >>>>> > On 16/06/2018 2:33 AM, Jeremy Manson wrote: > >> >>>>> > > Hi all, > >> >>>>> > > > >> >>>>> > > There are a number of references in the > >> JVMTI doc > >> >>>>> to its > >> >>>>> not doing > >> >>>>> > > object allocation tracking. Now that JEP > >> 331 has > >> >>>>> landed, > >> >>>>> these > >> >>>>> > > references are obsolete. This patch > >> changes those > >> >>>>> references > >> >>>>> > accordingly. > >> >>>>> > > > >> >>>>> > > While I was there, I took the liberty of > >> fixing > >> >>>>> some > >> >>>>> spelling errors. > >> >>>>> > > > >> >>>>> > > As far as I know, these are non-normative > >> >>>>> changes and > >> >>>>> modify no > >> >>>>> > API, so > >> >>>>> > > they should not require a CSR. > >> >>>>> > > >> >>>>> > I'm unclear on the nature of the change to > >> "VM Object > >> >>>>> Allocation". Does > >> >>>>> > the existence of SampledObjectAlloc change > >> when a VM > >> >>>>> should > >> >>>>> issue "VM > >> >>>>> > object allocation" events? > >> >>>>> > > >> >>>>> > Thanks, > >> >>>>> > David > >> >>>>> > > >> >>>>> > > > >> >>>>> > > Bug: > >> >>>>> > > > >> https://bugs.openjdk.java.net/browse/JDK-8205113 > >> >>>>> > > > >> >>>>> > > Webrev: > >> >>>>> > > > >> >>>>> http://cr.openjdk.java.net/~jmanson/8205113/webrev.00/ > >> > >> >>>>> > >> >>>>> > > > >> >>>>> > > Thanks! > >> >>>>> > > > >> >>>>> > > Jeremy > >> >>>>> > > >> >>>>> > >> >>> > >> >> > >> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Wed Jun 20 17:45:40 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 20 Jun 2018 10:45:40 -0700 Subject: RFR: JDK-6545967: sp05t003 failed ResumeThread() due to THREAD_NOT_SUSPENDED In-Reply-To: <5B2A4925.4000704@oracle.com> References: <5B294EA9.5030003@oracle.com> <4f541d71-1a90-cebe-67bb-dd11f85c072b@oracle.com> <5B2A4925.4000704@oracle.com> Message-ID: <17a8bd6e-0df1-8ec0-b8bd-4e45651e6fc6@oracle.com> Hi Gary, Overall looks good. One minor formatting comment: 105 if ((mt.threadState < 1000) && 106 agentStatus()) { 107 passed = true; 108 } Lines 105 and 106 should be combined. thanks Chris On 6/20/18 5:31 AM, Gary Adams wrote: > We do need to consider an early return for the case where > the redefine class or suspend thread operation have detected > an error. > > > Webrev:http://bussund0416.us.oracle.com/export/users/gradams/work/webrevs/6545967/webrev.01/ > > > 80 public boolean agentMethod() { > 81 boolean passed = false; > 82 MyThread mt = new MyThread(); > 83 try { > 84 mt.start(); > 85 // check if we can can pop the thread. > 86 // we can not do redefine/pop frame on run method. > 87 while (!MyThread.resume.get()); > > 88 // sleep for some few secs to get redefined. > 89 while (!isRedefined()) { > 90 if (!agentStatus()) { > 91 System.out.println("Failed to redefine class"); > 92 return passed; > 93 } > 94 Thread.sleep(100); > 95 } > 96 popThreadFrame(mt); // pop the frame. > 97 resumeThread(mt); // resume the thread. > 98 mt.join(); > 99 // wait till the other thread completes its execution. > 100 System.out.println("Thread state after popping/redefining = " > 101 + mt.threadState); > 102 } catch(Exception ie) { > 103 ie.printStackTrace(); > 104 } > 105 if ((mt.threadState < 1000) && > 106 agentStatus()) { > > 107 passed = true; > 108 } > 109 return passed; > 110 } > > > > On 6/19/18, 7:53 PM, Chris Plummer wrote: >> Hi Gary, >> >> The sp05t003 changes look fine. Serquei's question about hs203t003 >> leads me to ask what happens if the redefine never happens, possibly >> due to some error. Seems we are destined for a timeout then (not sure >> how long that will take), whereas the current implementation would >> report the failure after about 10 seconds. >> >> thanks, >> >> Chris >> >> On 6/19/18 2:26 PM, serguei.spitsyn at oracle.com wrote: >>> Hi Gary, >>> >>> The fix looks good in general. >>> >>> One comment though: >>> >>> http://cr.openjdk.java.net/~gadams/6545967/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS203/hs203t003/hs203t003.java.frames.html >>> >>> 80 public boolean agentMethod() { >>> 81 boolean passed = false; >>> 82 MyThread mt = new MyThread(); >>> 83 try { >>> 84 mt.start(); >>> 85 // check if we can can pop the thread. >>> 86 // we can not do redefine / pop frame on run method. >>> 87 while(!MyThread.resume.get()); >>> 88 // sleep for some few secs to get redefined. >>> 89 while(!isRedefined()) { >>> 90 Thread.sleep(100); >>> 91 } >>> 92 popThreadFrame(mt); // pop the frame. >>> 93 resumeThread(mt); // resume the thread. >>> 94 mt.join(); >>> 95 // wait till the other thread completes its execution. >>> 96 System.out.println(" Thread state after poping / redefining = "+mt.threadState); >>> 97 } catch(Exception ie) { >>> 98 ie.printStackTrace(); >>> 99 } >>> 100 if ( ( mt.threadState < 1000 ) && >>> 101 ( redefineAttempted() && isRedefined()) && >>> 102 agentStatus() ) { >>> 103 passed = true; >>> 104 } >>> 105 return passed; >>> 106 } >>> >>> It seems, the two checks ( redefineAttempted() && isRedefined()) at >>> L101 are not needed any more. >>> Now, there is no way out of the while loop at the line 89. >>> >>> Thanks, >>> Serguei >>> >>> >>> >>> On 6/19/18 11:42, Gary Adams wrote: >>>> There are some rare race conditions that impact some jvmti tests that >>>> suspend and resume threads. These tests were recently moved into >>>> the open repos. >>>> >>>> ? Webrev: http://cr.openjdk.java.net/~gadams/6545967/webrev.00/ >>>> >>>> The fix in hs203t003 replaces a blind 10 second sleep, with a specific >>>> check to wait for a redefineClass to be performed, which takes >>>> place as >>>> part of callbackFieldAccess. This let's the rest of the operations >>>> be performed >>>> at a known point in the test sequence. >>>> >>>> The fix in sp05t003 moves the incrementing of the counters to after >>>> the >>>> suspend thread calls. The agentProc function is watching the >>>> counters to know >>>> when to continue processing and eventually resuming of the >>>> suspended threads. >>>> >>> >> > From serguei.spitsyn at oracle.com Wed Jun 20 17:48:23 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 20 Jun 2018 10:48:23 -0700 Subject: RFR: JDK-6545967: sp05t003 failed ResumeThread() due to THREAD_NOT_SUSPENDED In-Reply-To: <17a8bd6e-0df1-8ec0-b8bd-4e45651e6fc6@oracle.com> References: <5B294EA9.5030003@oracle.com> <4f541d71-1a90-cebe-67bb-dd11f85c072b@oracle.com> <5B2A4925.4000704@oracle.com> <17a8bd6e-0df1-8ec0-b8bd-4e45651e6fc6@oracle.com> Message-ID: Hi Gary, Looks great modulo the suggestion from Chris. Thanks, Serguei On 6/20/18 10:45, Chris Plummer wrote: > Hi Gary, > > Overall looks good. One minor formatting comment: > > 105 if ((mt.threadState < 1000) && > 106 agentStatus()) { > ?107???????????? passed = true; > ?108???????? } > > > Lines 105 and 106 should be combined. > > thanks > > Chris > > On 6/20/18 5:31 AM, Gary Adams wrote: >> We do need to consider an early return for the case where >> the redefine class or suspend thread operation have detected >> an error. >> >> >> Webrev:http://bussund0416.us.oracle.com/export/users/gradams/work/webrevs/6545967/webrev.01/ >> >> >> ?? 80???? public boolean agentMethod() { >> ?? 81???????? boolean passed = false; >> ?? 82???????? MyThread mt = new MyThread(); >> ?? 83???????? try { >> ?? 84???????????? mt.start(); >> ?? 85???????????? // check if we can can pop the thread. >> 86 // we can not do redefine/pop frame on run method. >> 87 while (!MyThread.resume.get()); >> >> ?? 88???????????? // sleep for some few secs to get redefined. >> 89 while (!isRedefined()) { >> 90 if (!agentStatus()) { >> 91 System.out.println("Failed to redefine class"); >> 92 return passed; >> 93 } >> 94 Thread.sleep(100); >> 95 } >> 96 popThreadFrame(mt); // pop the frame. >> 97 resumeThread(mt); // resume the thread. >> ?? 98???????????? mt.join(); >> ?? 99???????????? // wait till the other thread completes its execution. >> 100 System.out.println("Thread state after popping/redefining = " >> 101 + mt.threadState); >> ? 102???????? } catch(Exception ie) { >> ? 103???????????? ie.printStackTrace(); >> ? 104???????? } >> 105 if ((mt.threadState < 1000) && >> 106 agentStatus()) { >> >> ? 107???????????? passed = true; >> ? 108???????? } >> ? 109???????? return passed; >> ? 110???? } >> >> >> >> On 6/19/18, 7:53 PM, Chris Plummer wrote: >>> Hi Gary, >>> >>> The sp05t003 changes look fine. Serquei's question about hs203t003 >>> leads me to ask what happens if the redefine never happens, possibly >>> due to some error. Seems we are destined for a timeout then (not >>> sure how long that will take), whereas the current implementation >>> would report the failure after about 10 seconds. >>> >>> thanks, >>> >>> Chris >>> >>> On 6/19/18 2:26 PM, serguei.spitsyn at oracle.com wrote: >>>> Hi Gary, >>>> >>>> The fix looks good in general. >>>> >>>> One comment though: >>>> >>>> http://cr.openjdk.java.net/~gadams/6545967/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS203/hs203t003/hs203t003.java.frames.html >>>> >>>> >>>> ?? 80???? public boolean? agentMethod() { >>>> ?? 81???????? boolean passed = false; >>>> ?? 82???????? MyThread mt = new MyThread(); >>>> ?? 83???????????????? try { >>>> ?? 84???????????? mt.start(); >>>> ?? 85???????????? // check if we can can pop the thread. >>>> ?? 86???????????? // we can not do redefine / pop frame on run method. >>>> ?? 87???????????? while(!MyThread.resume.get()); >>>> ?? 88???????????? // sleep for some few secs to get redefined. >>>> 89 while(!isRedefined()) { >>>> 90 Thread.sleep(100); >>>> 91 } >>>> 92 popThreadFrame(mt); // pop the frame. >>>> 93 resumeThread(mt); // resume the thread. >>>> ?? 94???????????? mt.join(); >>>> ?? 95???????????? // wait till the other thread completes its >>>> execution. >>>> ?? 96???????????? System.out.println(" Thread state after poping / >>>> redefining = "+mt.threadState); >>>> ?? 97???????? } catch(Exception ie) { >>>> ?? 98???????????? ie.printStackTrace(); >>>> ?? 99???????? } >>>> ? 100???????? if ( ( mt.threadState < 1000 ) && >>>> ? 101???????????????? ( redefineAttempted() && isRedefined())? && >>>> ? 102???????????????? agentStatus() ) { >>>> ? 103???????????? passed = true; >>>> ? 104???????? } >>>> ? 105???????? return passed; >>>> ? 106???? } >>>> ? It seems, the two checks ( redefineAttempted() && isRedefined()) >>>> at L101 are not needed any more. >>>> Now, there is no way out of the while loop at the line 89. >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> >>>> On 6/19/18 11:42, Gary Adams wrote: >>>>> There are some rare race conditions that impact some jvmti tests that >>>>> suspend and resume threads. These tests were recently moved into >>>>> the open repos. >>>>> >>>>> ? Webrev: http://cr.openjdk.java.net/~gadams/6545967/webrev.00/ >>>>> >>>>> The fix in hs203t003 replaces a blind 10 second sleep, with a >>>>> specific >>>>> check to wait for a redefineClass to be performed, which takes >>>>> place as >>>>> part of callbackFieldAccess. This let's the rest of the operations >>>>> be performed >>>>> at a known point in the test sequence. >>>>> >>>>> The fix in sp05t003 moves the incrementing of the counters to >>>>> after the >>>>> suspend thread calls. The agentProc function is watching the >>>>> counters to know >>>>> when to continue processing and eventually resuming of the >>>>> suspended threads. >>>>> >>>> >>> >> > > From gary.adams at oracle.com Wed Jun 20 18:03:04 2018 From: gary.adams at oracle.com (Gary Adams) Date: Wed, 20 Jun 2018 14:03:04 -0400 Subject: RFR: JDK-6545967: sp05t003 failed ResumeThread() due to THREAD_NOT_SUSPENDED In-Reply-To: References: <5B294EA9.5030003@oracle.com> <4f541d71-1a90-cebe-67bb-dd11f85c072b@oracle.com> <5B2A4925.4000704@oracle.com> <17a8bd6e-0df1-8ec0-b8bd-4e45651e6fc6@oracle.com> Message-ID: <5B2A96D8.4090805@oracle.com> I'll need a sponsor for the push. Patch attached. On 6/20/18, 1:48 PM, serguei.spitsyn at oracle.com wrote: > Hi Gary, > > Looks great modulo the suggestion from Chris. > > Thanks, > Serguei > > > On 6/20/18 10:45, Chris Plummer wrote: >> Hi Gary, >> >> Overall looks good. One minor formatting comment: >> >> 105 if ((mt.threadState < 1000) && >> 106 agentStatus()) { >> 107 passed = true; >> 108 } >> >> >> Lines 105 and 106 should be combined. >> >> thanks >> >> Chris >> >> On 6/20/18 5:31 AM, Gary Adams wrote: >>> We do need to consider an early return for the case where >>> the redefine class or suspend thread operation have detected >>> an error. >>> >>> >>> Webrev:http://bussund0416.us.oracle.com/export/users/gradams/work/webrevs/6545967/webrev.01/ >>> >>> >>> >>> 80 public boolean agentMethod() { >>> 81 boolean passed = false; >>> 82 MyThread mt = new MyThread(); >>> 83 try { >>> 84 mt.start(); >>> 85 // check if we can can pop the thread. >>> 86 // we can not do redefine/pop frame on run method. >>> 87 while (!MyThread.resume.get()); >>> >>> 88 // sleep for some few secs to get redefined. >>> 89 while (!isRedefined()) { >>> 90 if (!agentStatus()) { >>> 91 System.out.println("Failed to redefine class"); >>> 92 return passed; >>> 93 } >>> 94 Thread.sleep(100); >>> 95 } >>> 96 popThreadFrame(mt); // pop the frame. >>> 97 resumeThread(mt); // resume the thread. >>> 98 mt.join(); >>> 99 // wait till the other thread completes its >>> execution. >>> 100 System.out.println("Thread state after popping/redefining = " >>> 101 + mt.threadState); >>> 102 } catch(Exception ie) { >>> 103 ie.printStackTrace(); >>> 104 } >>> 105 if ((mt.threadState < 1000) && >>> 106 agentStatus()) { >>> >>> 107 passed = true; >>> 108 } >>> 109 return passed; >>> 110 } >>> >>> >>> >>> On 6/19/18, 7:53 PM, Chris Plummer wrote: >>>> Hi Gary, >>>> >>>> The sp05t003 changes look fine. Serquei's question about hs203t003 >>>> leads me to ask what happens if the redefine never happens, >>>> possibly due to some error. Seems we are destined for a timeout >>>> then (not sure how long that will take), whereas the current >>>> implementation would report the failure after about 10 seconds. >>>> >>>> thanks, >>>> >>>> Chris >>>> >>>> On 6/19/18 2:26 PM, serguei.spitsyn at oracle.com wrote: >>>>> Hi Gary, >>>>> >>>>> The fix looks good in general. >>>>> >>>>> One comment though: >>>>> >>>>> http://cr.openjdk.java.net/~gadams/6545967/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS203/hs203t003/hs203t003.java.frames.html >>>>> >>>>> >>>>> 80 public boolean agentMethod() { >>>>> 81 boolean passed = false; >>>>> 82 MyThread mt = new MyThread(); >>>>> 83 try { >>>>> 84 mt.start(); >>>>> 85 // check if we can can pop the thread. >>>>> 86 // we can not do redefine / pop frame on run >>>>> method. >>>>> 87 while(!MyThread.resume.get()); >>>>> 88 // sleep for some few secs to get redefined. >>>>> 89 while(!isRedefined()) { >>>>> 90 Thread.sleep(100); >>>>> 91 } >>>>> 92 popThreadFrame(mt); // pop the frame. >>>>> 93 resumeThread(mt); // resume the thread. >>>>> 94 mt.join(); >>>>> 95 // wait till the other thread completes its >>>>> execution. >>>>> 96 System.out.println(" Thread state after poping / >>>>> redefining = "+mt.threadState); >>>>> 97 } catch(Exception ie) { >>>>> 98 ie.printStackTrace(); >>>>> 99 } >>>>> 100 if ( ( mt.threadState < 1000 ) && >>>>> 101 ( redefineAttempted() && isRedefined()) && >>>>> 102 agentStatus() ) { >>>>> 103 passed = true; >>>>> 104 } >>>>> 105 return passed; >>>>> 106 } >>>>> It seems, the two checks ( redefineAttempted() && isRedefined()) >>>>> at L101 are not needed any more. >>>>> Now, there is no way out of the while loop at the line 89. >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>> >>>>> On 6/19/18 11:42, Gary Adams wrote: >>>>>> There are some rare race conditions that impact some jvmti tests >>>>>> that >>>>>> suspend and resume threads. These tests were recently moved into >>>>>> the open repos. >>>>>> >>>>>> Webrev: http://cr.openjdk.java.net/~gadams/6545967/webrev.00/ >>>>>> >>>>>> The fix in hs203t003 replaces a blind 10 second sleep, with a >>>>>> specific >>>>>> check to wait for a redefineClass to be performed, which takes >>>>>> place as >>>>>> part of callbackFieldAccess. This let's the rest of the >>>>>> operations be performed >>>>>> at a known point in the test sequence. >>>>>> >>>>>> The fix in sp05t003 moves the incrementing of the counters to >>>>>> after the >>>>>> suspend thread calls. The agentProc function is watching the >>>>>> counters to know >>>>>> when to continue processing and eventually resuming of the >>>>>> suspended threads. >>>>>> >>>>> >>>> >>> >> >> > -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: 6545967.patch URL: From chris.plummer at oracle.com Wed Jun 20 18:07:15 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 20 Jun 2018 11:07:15 -0700 Subject: RFR(S/M): 8205419: [testbug] TestJmapCore failing without SA: introduce @requires vm.hasSA In-Reply-To: <112db96d59084bdfafba2174e121c409@sap.com> References: <112db96d59084bdfafba2174e121c409@sap.com> Message-ID: <431269fc-7174-3051-0a86-baa00032754a@oracle.com> Hi Goetz, Can't VMProps.vmHasSA() just return Platform.shouldSAAttach()? It seems you are unnecessarily replicating Platform.shouldSAAttach() logic. If you are adding "@requires vm.hasSAandCanAttach" to a test, shouldn't you remove the test's Platform.shouldSAAttach() check? Is there a reason not to take the more direct and established approach of just adding the Platform.shouldSAAttach() check to TestJmapCore? It would be a lot less disruptive.? I know the vm.hasSAandCanAttach approach has advantages in not even attempting to compile and run the test, but I'm not so sure those advantages outweigh the simplicity of just adding a Platform.shouldSAAttach() check to one test. I can go either way here. Just wondering if there are additional reasons I might not be seeing. Sorry, I don't have an answer to your VMProps evaluation question. You might need to ask a wider audience than just svc. thanks, Chris On 6/20/18 6:49 AM, Lindenmaier, Goetz wrote: > Hi, > > TestJmapCore is failing on aix because there jhsdb is not implemented. > So far, such tests check at runtime Platform.shouldSAAttach() which is missing > in TestJmapCore. > > Instead, I implement @requires vm.hasSAandCanAttach. > This makes jtreg skip the tests without compiling and starting them. > > I added the new property to all tests that check shouldSAAttach in jdk and hostpot test > directories. > This is the implementation of the new property, the rest is just repetitive checking it: > http://cr.openjdk.java.net/~goetz/wr18/8205419-requiresHasSA/01/test/jtreg-ext/requires/VMProps.java.udiff.html > webrev: > http://cr.openjdk.java.net/~goetz/wr18/8205419-requiresHasSA/01/ > > Is it correct to assume the VMProps is evaluated on the same machine and with the same > user as used for executing the test? canPtraceAttachLinux directly accesses the system. > > (Should I remove the checks for shouldSAAttach() from the changed tests?) > > Best regards, > Goetz. From chris.plummer at oracle.com Wed Jun 20 18:07:42 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 20 Jun 2018 11:07:42 -0700 Subject: RFR: JDK-6545967: sp05t003 failed ResumeThread() due to THREAD_NOT_SUSPENDED In-Reply-To: <5B2A96D8.4090805@oracle.com> References: <5B294EA9.5030003@oracle.com> <4f541d71-1a90-cebe-67bb-dd11f85c072b@oracle.com> <5B2A4925.4000704@oracle.com> <17a8bd6e-0df1-8ec0-b8bd-4e45651e6fc6@oracle.com> <5B2A96D8.4090805@oracle.com> Message-ID: <14d2a44a-24ef-ea8d-0b54-7c00439ecc3c@oracle.com> I got it. Chris On 6/20/18 11:03 AM, Gary Adams wrote: > I'll need a sponsor for the push. > Patch attached. > > On 6/20/18, 1:48 PM, serguei.spitsyn at oracle.com wrote: >> Hi Gary, >> >> Looks great modulo the suggestion from Chris. >> >> Thanks, >> Serguei >> >> >> On 6/20/18 10:45, Chris Plummer wrote: >>> Hi Gary, >>> >>> Overall looks good. One minor formatting comment: >>> >>> 105 if ((mt.threadState < 1000) && >>> 106 agentStatus()) { >>> ?107???????????? passed = true; >>> ?108???????? } >>> >>> >>> Lines 105 and 106 should be combined. >>> >>> thanks >>> >>> Chris >>> >>> On 6/20/18 5:31 AM, Gary Adams wrote: >>>> We do need to consider an early return for the case where >>>> the redefine class or suspend thread operation have detected >>>> an error. >>>> >>>> >>>> Webrev:http://bussund0416.us.oracle.com/export/users/gradams/work/webrevs/6545967/webrev.01/ >>>> >>>> >>>> >>>> ?? 80???? public boolean agentMethod() { >>>> ?? 81???????? boolean passed = false; >>>> ?? 82???????? MyThread mt = new MyThread(); >>>> ?? 83???????? try { >>>> ?? 84???????????? mt.start(); >>>> ?? 85???????????? // check if we can can pop the thread. >>>> 86 // we can not do redefine/pop frame on run method. >>>> 87 while (!MyThread.resume.get()); >>>> >>>> ?? 88???????????? // sleep for some few secs to get redefined. >>>> 89 while (!isRedefined()) { >>>> 90 if (!agentStatus()) { >>>> 91 System.out.println("Failed to redefine class"); >>>> 92 return passed; >>>> 93 } >>>> 94 Thread.sleep(100); >>>> 95 } >>>> 96 popThreadFrame(mt); // pop the frame. >>>> 97 resumeThread(mt); // resume the thread. >>>> ?? 98???????????? mt.join(); >>>> ?? 99???????????? // wait till the other thread completes its >>>> execution. >>>> 100 System.out.println("Thread state after popping/redefining = " >>>> 101 + mt.threadState); >>>> ? 102???????? } catch(Exception ie) { >>>> ? 103???????????? ie.printStackTrace(); >>>> ? 104???????? } >>>> 105 if ((mt.threadState < 1000) && >>>> 106 agentStatus()) { >>>> >>>> ? 107???????????? passed = true; >>>> ? 108???????? } >>>> ? 109???????? return passed; >>>> ? 110???? } >>>> >>>> >>>> >>>> On 6/19/18, 7:53 PM, Chris Plummer wrote: >>>>> Hi Gary, >>>>> >>>>> The sp05t003 changes look fine. Serquei's question about hs203t003 >>>>> leads me to ask what happens if the redefine never happens, >>>>> possibly due to some error. Seems we are destined for a timeout >>>>> then (not sure how long that will take), whereas the current >>>>> implementation would report the failure after about 10 seconds. >>>>> >>>>> thanks, >>>>> >>>>> Chris >>>>> >>>>> On 6/19/18 2:26 PM, serguei.spitsyn at oracle.com wrote: >>>>>> Hi Gary, >>>>>> >>>>>> The fix looks good in general. >>>>>> >>>>>> One comment though: >>>>>> >>>>>> http://cr.openjdk.java.net/~gadams/6545967/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/hotswap/HS203/hs203t003/hs203t003.java.frames.html >>>>>> >>>>>> >>>>>> ?? 80???? public boolean? agentMethod() { >>>>>> ?? 81???????? boolean passed = false; >>>>>> ?? 82???????? MyThread mt = new MyThread(); >>>>>> ?? 83???????????????? try { >>>>>> ?? 84???????????? mt.start(); >>>>>> ?? 85???????????? // check if we can can pop the thread. >>>>>> ?? 86???????????? // we can not do redefine / pop frame on run >>>>>> method. >>>>>> ?? 87???????????? while(!MyThread.resume.get()); >>>>>> ?? 88???????????? // sleep for some few secs to get redefined. >>>>>> 89 while(!isRedefined()) { >>>>>> 90 Thread.sleep(100); >>>>>> 91 } >>>>>> 92 popThreadFrame(mt); // pop the frame. >>>>>> 93 resumeThread(mt); // resume the thread. >>>>>> ?? 94???????????? mt.join(); >>>>>> ?? 95???????????? // wait till the other thread completes its >>>>>> execution. >>>>>> ?? 96???????????? System.out.println(" Thread state after poping >>>>>> / redefining = "+mt.threadState); >>>>>> ?? 97???????? } catch(Exception ie) { >>>>>> ?? 98???????????? ie.printStackTrace(); >>>>>> ?? 99???????? } >>>>>> ? 100???????? if ( ( mt.threadState < 1000 ) && >>>>>> ? 101???????????????? ( redefineAttempted() && isRedefined()) && >>>>>> ? 102???????????????? agentStatus() ) { >>>>>> ? 103???????????? passed = true; >>>>>> ? 104???????? } >>>>>> ? 105???????? return passed; >>>>>> ? 106???? } >>>>>> ? It seems, the two checks ( redefineAttempted() && >>>>>> isRedefined()) at L101 are not needed any more. >>>>>> Now, there is no way out of the while loop at the line 89. >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>>> >>>>>> >>>>>> On 6/19/18 11:42, Gary Adams wrote: >>>>>>> There are some rare race conditions that impact some jvmti tests >>>>>>> that >>>>>>> suspend and resume threads. These tests were recently moved into >>>>>>> the open repos. >>>>>>> >>>>>>> ? Webrev: http://cr.openjdk.java.net/~gadams/6545967/webrev.00/ >>>>>>> >>>>>>> The fix in hs203t003 replaces a blind 10 second sleep, with a >>>>>>> specific >>>>>>> check to wait for a redefineClass to be performed, which takes >>>>>>> place as >>>>>>> part of callbackFieldAccess. This let's the rest of the >>>>>>> operations be performed >>>>>>> at a known point in the test sequence. >>>>>>> >>>>>>> The fix in sp05t003 moves the incrementing of the counters to >>>>>>> after the >>>>>>> suspend thread calls. The agentProc function is watching the >>>>>>> counters to know >>>>>>> when to continue processing and eventually resuming of the >>>>>>> suspended threads. >>>>>>> >>>>>> >>>>> >>>> >>> >>> >> > From chris.plummer at oracle.com Wed Jun 20 20:04:47 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 20 Jun 2018 13:04:47 -0700 Subject: RFR(XS): 8205385: Problem list vmTestbase/nsk/jvmti/RedefineClasses/StressRedefineWithoutBytecodeCorruption/TestDescription.java Message-ID: <69c7b811-597d-3250-7308-d81aad48fa01@oracle.com> Hello, Please review the following. I tested by running :vmTestbase_nsk_jvmti on all platforms and confirmed that the test is not run on linux-x64 or linux-x64-debug, but is run on all other platforms. I also ran the test 25 times on all non-linux supported platforms with no failures. I plan to push under the trivial change rules. https://bugs.openjdk.java.net/browse/JDK-8205385 diff --git a/test/hotspot/jtreg/ProblemList.txt b/test/hotspot/jtreg/ProblemList.txt --- a/test/hotspot/jtreg/ProblemList.txt +++ b/test/hotspot/jtreg/ProblemList.txt @@ -130,6 +130,7 @@ ?vmTestbase/nsk/jvmti/ClearBreakpoint/clrbrk001/TestDescription.java 8016181 generic-all ?vmTestbase/nsk/jvmti/FieldModification/fieldmod001/TestDescription.java 8016181 generic-all +vmTestbase/nsk/jvmti/RedefineClasses/StressRedefineWithoutBytecodeCorruption/TestDescription.java 8202896 linux-x64 ?vmTestbase/nsk/jvmti/ResourceExhausted/resexhausted001/TestDescription.java 7013634 generic-all ?vmTestbase/nsk/jvmti/ResourceExhausted/resexhausted003/TestDescription.java 6606767 generic-all ?vmTestbase/nsk/jvmti/ResourceExhausted/resexhausted004/TestDescription.java 7013634,6606767 generic-all thanks, Chris From goetz.lindenmaier at sap.com Wed Jun 20 20:22:41 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 20 Jun 2018 20:22:41 +0000 Subject: RFR(S/M): 8205419: [testbug] TestJmapCore failing without SA: introduce @requires vm.hasSA In-Reply-To: <431269fc-7174-3051-0a86-baa00032754a@oracle.com> References: <112db96d59084bdfafba2174e121c409@sap.com> <431269fc-7174-3051-0a86-baa00032754a@oracle.com> Message-ID: <3712cc995426467fa67b65493d556298@sap.com> Hi Chris, Thanks for looking at this change. > Can't VMProps.vmHasSA() just return Platform.shouldSAAttach()? It seems > you are unnecessarily replicating Platform.shouldSAAttach() logic. I think there are two things to distinguish: - platforms that don't implement SA - systems/setups where SA does not work. If both can be recognized when VMProps is evaluated, shouldSAAttach() is the one that is superfluous in the long run?! If not, shouldSAAttach should only return those where it is necessary to check in the test. But this is probably too much brains in this case ?? > If you are adding "@requires vm.hasSAandCanAttach" to a test, shouldn't > you remove the test's Platform.shouldSAAttach() check? Well I asked in my mail whether I should do that. So I take this as a yes. But it depends on whether it must be evaluated in the test. > Is there a reason not to take the more direct and established approach > of just adding the Platform.shouldSAAttach() check to TestJmapCore? It > would be a lot less disruptive.? I know the vm.hasSAandCanAttach > approach has advantages in not even attempting to compile and run the > test, but I'm not so sure those advantages outweigh the simplicity of > just adding a Platform.shouldSAAttach() check to one test. I can go > either way here. Just wondering if there are additional reasons I might > not be seeing. I like the @requires much more. It tells right where the test is described that it does not run with SA. The other is quite hidden in the test, e.g., in helper class ClhsdbLauncher.java. Also, it's more easy to remember (The implementor of TestJmapCore forgot it...). And, last but not least, it seems best practice nowadays. There are lots of excludes for mac, they are also done by @requires and not as a check using Platform at runtime. > Sorry, I don't have an answer to your VMProps evaluation question. You > might need to ask a wider audience than just svc. Whom should I ask? Hotspot-dev? Best regards, Goetz. > > thanks, > > Chris > > On 6/20/18 6:49 AM, Lindenmaier, Goetz wrote: > > Hi, > > > > TestJmapCore is failing on aix because there jhsdb is not implemented. > > So far, such tests check at runtime Platform.shouldSAAttach() which is > missing > > in TestJmapCore. > > > > Instead, I implement @requires vm.hasSAandCanAttach. > > This makes jtreg skip the tests without compiling and starting them. > > > > I added the new property to all tests that check shouldSAAttach in jdk and > hostpot test > > directories. > > This is the implementation of the new property, the rest is just repetitive > checking it: > > http://cr.openjdk.java.net/~goetz/wr18/8205419- > requiresHasSA/01/test/jtreg-ext/requires/VMProps.java.udiff.html > > webrev: > > http://cr.openjdk.java.net/~goetz/wr18/8205419-requiresHasSA/01/ > > > > Is it correct to assume the VMProps is evaluated on the same machine and > with the same > > user as used for executing the test? canPtraceAttachLinux directly > accesses the system. > > > > (Should I remove the checks for shouldSAAttach() from the changed tests?) > > > > Best regards, > > Goetz. > > From serguei.spitsyn at oracle.com Wed Jun 20 22:14:20 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 20 Jun 2018 15:14:20 -0700 Subject: RFR(XS): 8205385: Problem list vmTestbase/nsk/jvmti/RedefineClasses/StressRedefineWithoutBytecodeCorruption/TestDescription.java In-Reply-To: <69c7b811-597d-3250-7308-d81aad48fa01@oracle.com> References: <69c7b811-597d-3250-7308-d81aad48fa01@oracle.com> Message-ID: Looks good. Thanks, Serguei On 6/20/18 13:04, Chris Plummer wrote: > Hello, > > Please review the following. I tested by running :vmTestbase_nsk_jvmti > on all platforms and confirmed that the test is not run on linux-x64 > or linux-x64-debug, but is run on all other platforms. I also ran the > test 25 times on all non-linux supported platforms with no failures. I > plan to push under the trivial change rules. > > https://bugs.openjdk.java.net/browse/JDK-8205385 > > diff --git a/test/hotspot/jtreg/ProblemList.txt > b/test/hotspot/jtreg/ProblemList.txt > --- a/test/hotspot/jtreg/ProblemList.txt > +++ b/test/hotspot/jtreg/ProblemList.txt > @@ -130,6 +130,7 @@ > > ?vmTestbase/nsk/jvmti/ClearBreakpoint/clrbrk001/TestDescription.java > 8016181 generic-all > ?vmTestbase/nsk/jvmti/FieldModification/fieldmod001/TestDescription.java > 8016181 generic-all > +vmTestbase/nsk/jvmti/RedefineClasses/StressRedefineWithoutBytecodeCorruption/TestDescription.java > 8202896 linux-x64 > ?vmTestbase/nsk/jvmti/ResourceExhausted/resexhausted001/TestDescription.java > 7013634 generic-all > ?vmTestbase/nsk/jvmti/ResourceExhausted/resexhausted003/TestDescription.java > 6606767 generic-all > ?vmTestbase/nsk/jvmti/ResourceExhausted/resexhausted004/TestDescription.java > 7013634,6606767 generic-all > > thanks, > > Chris > > From chris.plummer at oracle.com Wed Jun 20 23:16:19 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 20 Jun 2018 16:16:19 -0700 Subject: RFR(XS): 8205385: Problem list vmTestbase/nsk/jvmti/RedefineClasses/StressRedefineWithoutBytecodeCorruption/TestDescription.java In-Reply-To: References: <69c7b811-597d-3250-7308-d81aad48fa01@oracle.com> Message-ID: Thanks! On 6/20/18 3:14 PM, serguei.spitsyn at oracle.com wrote: > Looks good. > > Thanks, > Serguei > > > On 6/20/18 13:04, Chris Plummer wrote: >> Hello, >> >> Please review the following. I tested by running >> :vmTestbase_nsk_jvmti on all platforms and confirmed that the test is >> not run on linux-x64 or linux-x64-debug, but is run on all other >> platforms. I also ran the test 25 times on all non-linux supported >> platforms with no failures. I plan to push under the trivial change >> rules. >> >> https://bugs.openjdk.java.net/browse/JDK-8205385 >> >> diff --git a/test/hotspot/jtreg/ProblemList.txt >> b/test/hotspot/jtreg/ProblemList.txt >> --- a/test/hotspot/jtreg/ProblemList.txt >> +++ b/test/hotspot/jtreg/ProblemList.txt >> @@ -130,6 +130,7 @@ >> >> ?vmTestbase/nsk/jvmti/ClearBreakpoint/clrbrk001/TestDescription.java >> 8016181 generic-all >> ?vmTestbase/nsk/jvmti/FieldModification/fieldmod001/TestDescription.java >> 8016181 generic-all >> +vmTestbase/nsk/jvmti/RedefineClasses/StressRedefineWithoutBytecodeCorruption/TestDescription.java >> 8202896 linux-x64 >> ?vmTestbase/nsk/jvmti/ResourceExhausted/resexhausted001/TestDescription.java >> 7013634 generic-all >> ?vmTestbase/nsk/jvmti/ResourceExhausted/resexhausted003/TestDescription.java >> 6606767 generic-all >> ?vmTestbase/nsk/jvmti/ResourceExhausted/resexhausted004/TestDescription.java >> 7013634,6606767 generic-all >> >> thanks, >> >> Chris >> >> > From chris.plummer at oracle.com Wed Jun 20 23:22:06 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 20 Jun 2018 16:22:06 -0700 Subject: RFR(S/M): 8205419: [testbug] TestJmapCore failing without SA: introduce @requires vm.hasSA In-Reply-To: <3712cc995426467fa67b65493d556298@sap.com> References: <112db96d59084bdfafba2174e121c409@sap.com> <431269fc-7174-3051-0a86-baa00032754a@oracle.com> <3712cc995426467fa67b65493d556298@sap.com> Message-ID: Hi Goetz, On 6/20/18 1:22 PM, Lindenmaier, Goetz wrote: > Hi Chris, > > Thanks for looking at this change. > >> Can't VMProps.vmHasSA() just return Platform.shouldSAAttach()? It seems >> you are unnecessarily replicating Platform.shouldSAAttach() logic. > I think there are two things to distinguish: > - platforms that don't implement SA > - systems/setups where SA does not work. > If both can be recognized when VMProps is evaluated, shouldSAAttach() > is the one that is superfluous in the long run?! If not, shouldSAAttach > should only return those where it is necessary to check in the test. > But this is probably too much brains in this case ?? I'm not really sure what you are pointing our here. My point it that VMProps.vmHasSA() can just return Platform.shouldSAAttach() and not include all the extra logic you have. I wasn't saying one or the other is not needed. > >> If you are adding "@requires vm.hasSAandCanAttach" to a test, shouldn't >> you remove the test's Platform.shouldSAAttach() check? > Well I asked in my mail whether I should do that. So I take this as a yes. > But it depends on whether it must be evaluated in the test. I think yes for consistency. > >> Is there a reason not to take the more direct and established approach >> of just adding the Platform.shouldSAAttach() check to TestJmapCore? It >> would be a lot less disruptive.? I know the vm.hasSAandCanAttach >> approach has advantages in not even attempting to compile and run the >> test, but I'm not so sure those advantages outweigh the simplicity of >> just adding a Platform.shouldSAAttach() check to one test. I can go >> either way here. Just wondering if there are additional reasons I might >> not be seeing. > I like the @requires much more. It tells right where the test is described > that it does not run with SA. The other is quite hidden in the test, e.g., in > helper class ClhsdbLauncher.java. > Also, it's more easy to remember (The implementor of > TestJmapCore forgot it...). And, last but not least, it seems best practice > nowadays. There are lots of excludes for mac, they are also done by @requires > and not as a check using Platform at runtime. Sometimes however you just can't avoid the platform or functionality check in the code because only part of the test is reliant on it. >> Sorry, I don't have an answer to your VMProps evaluation question. You >> might need to ask a wider audience than just svc. > Whom should I ask? Hotspot-dev? Well, that would cover the widest possible audience for this question. :) It's up to you. I was just suggesting that you'll be more likely to get an answer if you expand beyond svc. You could got with runtime-dev, hotspot-dev, or jdk-dev. cheers, Chris > > Best regards, > Goetz. > > >> thanks, >> >> Chris >> >> On 6/20/18 6:49 AM, Lindenmaier, Goetz wrote: >>> Hi, >>> >>> TestJmapCore is failing on aix because there jhsdb is not implemented. >>> So far, such tests check at runtime Platform.shouldSAAttach() which is >> missing >>> in TestJmapCore. >>> >>> Instead, I implement @requires vm.hasSAandCanAttach. >>> This makes jtreg skip the tests without compiling and starting them. >>> >>> I added the new property to all tests that check shouldSAAttach in jdk and >> hostpot test >>> directories. >>> This is the implementation of the new property, the rest is just repetitive >> checking it: >>> http://cr.openjdk.java.net/~goetz/wr18/8205419- >> requiresHasSA/01/test/jtreg-ext/requires/VMProps.java.udiff.html >>> webrev: >>> http://cr.openjdk.java.net/~goetz/wr18/8205419-requiresHasSA/01/ >>> >>> Is it correct to assume the VMProps is evaluated on the same machine and >> with the same >>> user as used for executing the test? canPtraceAttachLinux directly >> accesses the system. >>> (Should I remove the checks for shouldSAAttach() from the changed tests?) >>> >>> Best regards, >>> Goetz. >> From daniel.daugherty at oracle.com Thu Jun 21 00:18:17 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 20 Jun 2018 20:18:17 -0400 Subject: RFR(XS): 8205195 NestedThreadsListHandleInErrorHandlingTest fails because hs_err doesn't contain _nested_thread_list_max Message-ID: Greetings, I have a fix for a recent (very rare) Thread-SMR related test failure. Since the fix is related to the ErrorHandling tests and affects hs_err_pid file generation, this code review is being sent to both the Runtime and the Serviceability teams. Please make sure you reply-all to any responses so we have complete review threads on both aliases. Bug URL: https://bugs.openjdk.java.net/browse/JDK-8205195 Webrev URL: http://cr.openjdk.java.net/~dcubed/8205195-webrev/0-for-jdk-jdk/ The bug itself contains analysis about the root cause of the bug and the comment updates to the code describes the no win scenario that the hs_err_pid file generation code is in. Of course, I also have a comment where I was able to harden the ErrorHandling tests. I did manage to resist the urge to mention the "Kobiyashi Maru" [1] in the new comments. Testing: Mach5 builds-tier1,jdk-tier1,jdk-tier2,hs-tier1,hs-tier2,hs-tier3 ?? ? ? ? on the usual Oracle platforms. Thanks, in advance, for any comments, questions or suggestions. Dan [1] https://www.urbandictionary.com/define.php?term=Kobayashi%20Maru From harsha.wardhana.b at oracle.com Thu Jun 21 04:13:18 2018 From: harsha.wardhana.b at oracle.com (Harsha Wardhana B) Date: Thu, 21 Jun 2018 09:43:18 +0530 Subject: RFR : JDK-8204661 - Show error 'Port already in use' in HashedPasswordFileTest.java In-Reply-To: References: <9fa7bd72-6476-b702-ba34-ec8fc572cfcb@oracle.com> Message-ID: Hi David, Thanks for the review. I have ran the tests in Mach5 10 times on each platform and I haven't seen any failure. I will update the bug with the chosen approach. Would you require a separate webrev for copyright update? Regards Harsha On Wednesday 20 June 2018 05:12 PM, David Holmes wrote: > Hi Harsha, > > On 20/06/2018 6:14 PM, Harsha Wardhana B wrote: >> Hi, >> >> Please find the fix below for the bug >> >> JDK-8204661 : Show error 'Port already in use' in >> HashedPasswordFileTest.java >> >> having webrev at, >> >> http://cr.openjdk.java.net/~hb/8204661/webrev.00/ >> >> The problem root-cause is discussed in the comments section of the bug. >> >> The fix above lets the default agent pick a free port by passing >> 'port=0' value and then reads the JMX Connector URL from Perf Counters. > > Can you update the bug report with the actual solution chosen. I find > it interesting that I can't see any other test using this technique. > If port=0 means the port bind can't fail then this may be the > technique all tests should be using for reliability - though it's a > pity there's no public API supporting reading back the URL. > > Have you run the test through mach5 repeatedly to check all (Posix) > platforms and a range of systems? You can use "--test-repeat 10" for > example to run it 10 times on each platform. > >> Please review the fix above and provide comments. > > The copyright needs to changed to the two year format: > > * Copyright (c) 2017, Oracle and/or its affiliates. All rights reserved. > > becomes > > * Copyright (c) 2017, 2018, Oracle and/or its affiliates. All rights > reserved. > > Otherwise this seems okay. > > Thanks, > David > >> Thanks >> Harsha >> >> From mandy.chung at oracle.com Thu Jun 21 05:00:13 2018 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 20 Jun 2018 22:00:13 -0700 Subject: RFR : JDK-8204661 - Show error 'Port already in use' in HashedPasswordFileTest.java In-Reply-To: <9fa7bd72-6476-b702-ba34-ec8fc572cfcb@oracle.com> References: <9fa7bd72-6476-b702-ba34-ec8fc572cfcb@oracle.com> Message-ID: <674a1a9a-0528-a2d1-f09f-b9d8724d62ec@oracle.com> On 6/20/18 1:14 AM, Harsha Wardhana B wrote: > Hi, > > Please find the fix below for the bug > > JDK-8204661 : Show error 'Port already in use' in > HashedPasswordFileTest.java > > having webrev at, > > http://cr.openjdk.java.net/~hb/8204661/webrev.00/ > > The problem root-cause is discussed in the comments section of the bug. > > The fix above lets the default agent pick a free port by passing > 'port=0' value and then reads the JMX Connector URL from Perf Counters. This looks fine. Please add this issue number to @bug. This is interesting. I was not aware of setting port=0 will auto-assign a free port. Do you know if it was added for testing purpose (which I assume so)? For example test/jdk/sun/management/jmxremote/bootstrap/RmiBootstrapTest.java uses jdk.testlibrary.Utils.getFreePort() to get a free port number. Mandy From harsha.wardhana.b at oracle.com Thu Jun 21 06:47:23 2018 From: harsha.wardhana.b at oracle.com (Harsha Wardhana B) Date: Thu, 21 Jun 2018 12:17:23 +0530 Subject: RFR : JDK-8204661 - Show error 'Port already in use' in HashedPasswordFileTest.java In-Reply-To: <674a1a9a-0528-a2d1-f09f-b9d8724d62ec@oracle.com> References: <9fa7bd72-6476-b702-ba34-ec8fc572cfcb@oracle.com> <674a1a9a-0528-a2d1-f09f-b9d8724d62ec@oracle.com> Message-ID: <4d87b834-3b78-7b03-e90b-13385050fb19@oracle.com> On Thursday 21 June 2018 10:30 AM, mandy chung wrote: > > > On 6/20/18 1:14 AM, Harsha Wardhana B wrote: >> Hi, >> >> Please find the fix below for the bug >> >> JDK-8204661 : Show error 'Port already in use' in >> HashedPasswordFileTest.java >> >> having webrev at, >> >> http://cr.openjdk.java.net/~hb/8204661/webrev.00/ >> >> The problem root-cause is discussed in the comments section of the bug. >> >> The fix above lets the default agent pick a free port by passing >> 'port=0' value and then reads the JMX Connector URL from Perf Counters. > > This looks fine.? Please add this issue number to @bug. > > This is interesting.? I was not aware of setting port=0 will > auto-assign a free port.? Do you know if it was added for testing > purpose (which I assume so)? Port number of 0 is not handled at JMX or RMI layer. The given port number is passed onto the ServerSocket. https://docs.oracle.com/javase/10/docs/api/java/net/ServerSocket.html#%3Cinit%3E(int) > > For example > test/jdk/sun/management/jmxremote/bootstrap/RmiBootstrapTest.java uses > jdk.testlibrary.Utils.getFreePort() to get a free port number. > > Mandy Harsha From thomas.stuefe at gmail.com Thu Jun 21 06:53:54 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 21 Jun 2018 08:53:54 +0200 Subject: RFR(XS): 8205195 NestedThreadsListHandleInErrorHandlingTest fails because hs_err doesn't contain _nested_thread_list_max In-Reply-To: References: Message-ID: Hi Daniel, yes, that is annoying. I am okay with your fix, if you want to push it in this form. But preparing the test crash in this way feels weird since the whole point of this exercise is to test error handling in close-to-real scenarios... but it sure does not hurt in this case. Also, note that at different places we decide differently, see e.g. the "printing heap information" STEP - we omit locking Heap_lock in VMError::report() and only lock it in VMError::print_vm_info() (where we have no secondary signal handling and must not crash). So, in that case we are okay with risking a secondary crash in error handling. Probably there are just no regression tests for the heap information printout whose intermittent fails could annoy us :) My feeling is that I would like to see a solution at the test framework side. Maybe, if a test is marked as "may fail rarely and thats okay", the test framework could retry the test and only fail if the error happens again. Thanks, Thomas On Thu, Jun 21, 2018 at 2:18 AM, Daniel D. Daugherty wrote: > Greetings, > > I have a fix for a recent (very rare) Thread-SMR related test failure. > > Since the fix is related to the ErrorHandling tests and affects hs_err_pid > file generation, this code review is being sent to both the Runtime and > the Serviceability teams. Please make sure you reply-all to any responses > so we have complete review threads on both aliases. > > Bug URL: https://bugs.openjdk.java.net/browse/JDK-8205195 > > Webrev URL: http://cr.openjdk.java.net/~dcubed/8205195-webrev/0-for-jdk-jdk/ > > The bug itself contains analysis about the root cause of the bug and > the comment updates to the code describes the no win scenario that the > hs_err_pid file generation code is in. Of course, I also have a comment > where I was able to harden the ErrorHandling tests. I did manage to > resist the urge to mention the "Kobiyashi Maru" [1] in the new comments. > > Testing: Mach5 builds-tier1,jdk-tier1,jdk-tier2,hs-tier1,hs-tier2,hs-tier3 > on the usual Oracle platforms. > > Thanks, in advance, for any comments, questions or suggestions. > > Dan > > [1] https://www.urbandictionary.com/define.php?term=Kobayashi%20Maru > From jini.george at oracle.com Thu Jun 21 10:18:06 2018 From: jini.george at oracle.com (Jini George) Date: Thu, 21 Jun 2018 15:48:06 +0530 Subject: RFR(S/M): 8205419: [testbug] TestJmapCore failing without SA: introduce @requires vm.hasSA In-Reply-To: <3712cc995426467fa67b65493d556298@sap.com> References: <112db96d59084bdfafba2174e121c409@sap.com> <431269fc-7174-3051-0a86-baa00032754a@oracle.com> <3712cc995426467fa67b65493d556298@sap.com> Message-ID: Hi Goetz, Tests like TestJmapCore which involve only corefile debugging by SA (and not attaching to a live process) would only need to check for the presence of SA, and would not need to check for ptrace related attach permissions like what is done for Linux and OSX -- since super user privileges are not required to debug corefiles. So a single @requires catering to both these might not be desirable. Having said that, I realized that runOnCore() of ClhsdbLauncher incorrectly checks for the ptrace related attach permissions. (Will file an issue to correct this). An @requires instead of a shouldSAAttach() might help in avoiding fake passes like those on OSX when not run as "root". (https://bugs.openjdk.java.net/browse/JDK-8199700) Thanks, Jini. On 6/21/2018 1:52 AM, Lindenmaier, Goetz wrote: > Hi Chris, > > Thanks for looking at this change. > >> Can't VMProps.vmHasSA() just return Platform.shouldSAAttach()? It seems >> you are unnecessarily replicating Platform.shouldSAAttach() logic. > I think there are two things to distinguish: > - platforms that don't implement SA > - systems/setups where SA does not work. > If both can be recognized when VMProps is evaluated, shouldSAAttach() > is the one that is superfluous in the long run?! If not, shouldSAAttach > should only return those where it is necessary to check in the test. > But this is probably too much brains in this case ?? > >> If you are adding "@requires vm.hasSAandCanAttach" to a test, shouldn't >> you remove the test's Platform.shouldSAAttach() check? > Well I asked in my mail whether I should do that. So I take this as a yes. > But it depends on whether it must be evaluated in the test. > >> Is there a reason not to take the more direct and established approach >> of just adding the Platform.shouldSAAttach() check to TestJmapCore? It >> would be a lot less disruptive.? I know the vm.hasSAandCanAttach >> approach has advantages in not even attempting to compile and run the >> test, but I'm not so sure those advantages outweigh the simplicity of >> just adding a Platform.shouldSAAttach() check to one test. I can go >> either way here. Just wondering if there are additional reasons I might >> not be seeing. > I like the @requires much more. It tells right where the test is described > that it does not run with SA. The other is quite hidden in the test, e.g., in > helper class ClhsdbLauncher.java. > Also, it's more easy to remember (The implementor of > TestJmapCore forgot it...). And, last but not least, it seems best practice > nowadays. There are lots of excludes for mac, they are also done by @requires > and not as a check using Platform at runtime. > >> Sorry, I don't have an answer to your VMProps evaluation question. You >> might need to ask a wider audience than just svc. > Whom should I ask? Hotspot-dev? > > Best regards, > Goetz. > > >> >> thanks, >> >> Chris >> >> On 6/20/18 6:49 AM, Lindenmaier, Goetz wrote: >>> Hi, >>> >>> TestJmapCore is failing on aix because there jhsdb is not implemented. >>> So far, such tests check at runtime Platform.shouldSAAttach() which is >> missing >>> in TestJmapCore. >>> >>> Instead, I implement @requires vm.hasSAandCanAttach. >>> This makes jtreg skip the tests without compiling and starting them. >>> >>> I added the new property to all tests that check shouldSAAttach in jdk and >> hostpot test >>> directories. >>> This is the implementation of the new property, the rest is just repetitive >> checking it: >>> http://cr.openjdk.java.net/~goetz/wr18/8205419- >> requiresHasSA/01/test/jtreg-ext/requires/VMProps.java.udiff.html >>> webrev: >>> http://cr.openjdk.java.net/~goetz/wr18/8205419-requiresHasSA/01/ >>> >>> Is it correct to assume the VMProps is evaluated on the same machine and >> with the same >>> user as used for executing the test? canPtraceAttachLinux directly >> accesses the system. >>> >>> (Should I remove the checks for shouldSAAttach() from the changed tests?) >>> >>> Best regards, >>> Goetz. >> >> > From david.holmes at oracle.com Thu Jun 21 10:59:34 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 21 Jun 2018 20:59:34 +1000 Subject: RFR : JDK-8204661 - Show error 'Port already in use' in HashedPasswordFileTest.java In-Reply-To: References: <9fa7bd72-6476-b702-ba34-ec8fc572cfcb@oracle.com> Message-ID: Hi Harsha, On 21/06/2018 2:13 PM, Harsha Wardhana B wrote: > Hi David, > > Thanks for the review. I have ran the tests in Mach5 10 times on each > platform and I haven't seen any failure. I will update the bug with the Thanks for the additional testing. > chosen approach. Would you require a separate webrev for copyright update? No. :) Thanks, David > Regards > Harsha > > On Wednesday 20 June 2018 05:12 PM, David Holmes wrote: >> Hi Harsha, >> >> On 20/06/2018 6:14 PM, Harsha Wardhana B wrote: >>> Hi, >>> >>> Please find the fix below for the bug >>> >>> JDK-8204661 : Show error 'Port already in use' in >>> HashedPasswordFileTest.java >>> >>> having webrev at, >>> >>> http://cr.openjdk.java.net/~hb/8204661/webrev.00/ >>> >>> The problem root-cause is discussed in the comments section of the bug. >>> >>> The fix above lets the default agent pick a free port by passing >>> 'port=0' value and then reads the JMX Connector URL from Perf Counters. >> >> Can you update the bug report with the actual solution chosen. I find >> it interesting that I can't see any other test using this technique. >> If port=0 means the port bind can't fail then this may be the >> technique all tests should be using for reliability - though it's a >> pity there's no public API supporting reading back the URL. >> >> Have you run the test through mach5 repeatedly to check all (Posix) >> platforms and a range of systems? You can use "--test-repeat 10" for >> example to run it 10 times on each platform. >> >>> Please review the fix above and provide comments. >> >> The copyright needs to changed to the two year format: >> >> * Copyright (c) 2017, Oracle and/or its affiliates. All rights reserved. >> >> becomes >> >> * Copyright (c) 2017, 2018, Oracle and/or its affiliates. All rights >> reserved. >> >> Otherwise this seems okay. >> >> Thanks, >> David >> >>> Thanks >>> Harsha >>> >>> > From david.holmes at oracle.com Thu Jun 21 11:14:14 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 21 Jun 2018 21:14:14 +1000 Subject: RFR(S/M): 8205419: [testbug] TestJmapCore failing without SA: introduce @requires vm.hasSA In-Reply-To: <112db96d59084bdfafba2174e121c409@sap.com> References: <112db96d59084bdfafba2174e121c409@sap.com> Message-ID: Answering VMProps question ... On 20/06/2018 11:49 PM, Lindenmaier, Goetz wrote: > Hi, > > TestJmapCore is failing on aix because there jhsdb is not implemented. > So far, such tests check at runtime Platform.shouldSAAttach() which is missing > in TestJmapCore. > > Instead, I implement @requires vm.hasSAandCanAttach. > This makes jtreg skip the tests without compiling and starting them. > > I added the new property to all tests that check shouldSAAttach in jdk and hostpot test > directories. > This is the implementation of the new property, the rest is just repetitive checking it: > http://cr.openjdk.java.net/~goetz/wr18/8205419-requiresHasSA/01/test/jtreg-ext/requires/VMProps.java.udiff.html > webrev: > http://cr.openjdk.java.net/~goetz/wr18/8205419-requiresHasSA/01/ > > Is it correct to assume the VMProps is evaluated on the same machine and with the same > user as used for executing the test? canPtraceAttachLinux directly accesses the system. jtreg will run the VM under test with all the VM flags specified to jtreg and execute VMProps to gather information about the VM under test before running the tests themselves. Obviously using "@run othervm " you can invalidate what may have been checked in a @require. HTH David > (Should I remove the checks for shouldSAAttach() from the changed tests?) > > Best regards, > Goetz. > From daniel.daugherty at oracle.com Thu Jun 21 12:57:54 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 21 Jun 2018 08:57:54 -0400 Subject: RFR(XS): 8205195 NestedThreadsListHandleInErrorHandlingTest fails because hs_err doesn't contain _nested_thread_list_max In-Reply-To: References: Message-ID: Thomas, Thanks for the quick review. On 6/21/18 2:53 AM, Thomas St?fe wrote: > Hi Daniel, > > yes, that is annoying. > > I am okay with your fix, if you want to push it in this form. Thanks. > But preparing the test crash in this way feels weird since the whole > point of this exercise is to test error handling in close-to-real > scenarios... but it sure does not hurt in this case. Yup. This is definitely weird, but my goal is to reduce testing noise. I do acknowledge that the real world use of error reporting may run into this failure mode which will suppress a section of hs_err_pid output. > Also, note that at different places we decide differently, see e.g. > the "printing heap information" STEP - we omit locking Heap_lock in > VMError::report() and only lock it in VMError::print_vm_info() (where > we have no secondary signal handling and must not crash). So, in that > case we are okay with risking a secondary crash in error handling. > Probably there are just no regression tests for the heap information > printout whose intermittent fails could annoy us :) Yup. I recognized that when I wrote the Thread-SMR tests I was making them picky enough to possible run into failure modes we never would have detected before. > My feeling is that I would like to see a solution at the test > framework side. Maybe, if a test is marked as "may fail rarely and > thats okay", the test framework could retry the test and only fail if > the error happens again. We currently don't have a way of tagging a test like that and I'm not convinced that I would really want us to do that. However, this particular bug truly falls into a no win scenario and that's a different situation than I've encountered before. Again, thanks for the review. Dan > > Thanks, Thomas > > > > On Thu, Jun 21, 2018 at 2:18 AM, Daniel D. Daugherty > wrote: >> Greetings, >> >> I have a fix for a recent (very rare) Thread-SMR related test failure. >> >> Since the fix is related to the ErrorHandling tests and affects hs_err_pid >> file generation, this code review is being sent to both the Runtime and >> the Serviceability teams. Please make sure you reply-all to any responses >> so we have complete review threads on both aliases. >> >> Bug URL: https://bugs.openjdk.java.net/browse/JDK-8205195 >> >> Webrev URL: http://cr.openjdk.java.net/~dcubed/8205195-webrev/0-for-jdk-jdk/ >> >> The bug itself contains analysis about the root cause of the bug and >> the comment updates to the code describes the no win scenario that the >> hs_err_pid file generation code is in. Of course, I also have a comment >> where I was able to harden the ErrorHandling tests. I did manage to >> resist the urge to mention the "Kobiyashi Maru" [1] in the new comments. >> >> Testing: Mach5 builds-tier1,jdk-tier1,jdk-tier2,hs-tier1,hs-tier2,hs-tier3 >> on the usual Oracle platforms. >> >> Thanks, in advance, for any comments, questions or suggestions. >> >> Dan >> >> [1] https://www.urbandictionary.com/define.php?term=Kobayashi%20Maru >> From daniel.daugherty at oracle.com Thu Jun 21 13:58:08 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 21 Jun 2018 09:58:08 -0400 Subject: RFR : JDK-8204661 - Show error 'Port already in use' in HashedPasswordFileTest.java In-Reply-To: <4d87b834-3b78-7b03-e90b-13385050fb19@oracle.com> References: <9fa7bd72-6476-b702-ba34-ec8fc572cfcb@oracle.com> <674a1a9a-0528-a2d1-f09f-b9d8724d62ec@oracle.com> <4d87b834-3b78-7b03-e90b-13385050fb19@oracle.com> Message-ID: > Port number of 0 is not handled at JMX or RMI layer. The given port number is passed onto the ServerSocket. > > https://docs.oracle.com/javase/10/docs/api/java/net/ServerSocket.html#%3Cinit%3E(int) Using a port number of 0 to get a free port has to be done very carefully to avoid running into other, more difficult to find issues. Jerry T. fix a few of these before he left us: ??? JDK-8182757 JDWP: Socket Transport handshake hangs on Solaris ??? https://bugs.openjdk.java.net/browse/JDK-8182757 ??? JDK-8178676 nsk/jvmti/AttachOnDemand/attach045 fails with Exception ??? https://bugs.openjdk.java.net/browse/JDK-8178676 ??? JDK-8188867 nsk/jdi/VirtualMachineManager/createVirtualMachine/createVM004 ??????????????? (and other tests) timeout do to socket problem ??? https://bugs.openjdk.java.net/browse/JDK-8188867 As an example of the difficulty, I tried to fix this bug: ??? JDK-8182307 Error during JRMP connection establishment ??? https://bugs.openjdk.java.net/browse/JDK-8182307 and had to back out my fix because it caused a couple of tests to fail. I'm tracking the redo here: ??? JDK-8193227 [REDO] 8182307 Error during JRMP connection establishment ??? https://bugs.openjdk.java.net/browse/JDK-8193227 and I haven't had an epiphany for how to fix it without breaking other tests (yet) so I just decommitted it from JDK11. I'm not saying that your current fix will be susceptible to very intermittent hangs on Solaris. I would have to take a very close look at it and I haven't done that yet. Dan On 6/21/18 2:47 AM, Harsha Wardhana B wrote: > > > On Thursday 21 June 2018 10:30 AM, mandy chung wrote: >> >> >> On 6/20/18 1:14 AM, Harsha Wardhana B wrote: >>> Hi, >>> >>> Please find the fix below for the bug >>> >>> JDK-8204661 : Show error 'Port already in use' in >>> HashedPasswordFileTest.java >>> >>> having webrev at, >>> >>> http://cr.openjdk.java.net/~hb/8204661/webrev.00/ >>> >>> The problem root-cause is discussed in the comments section of the bug. >>> >>> The fix above lets the default agent pick a free port by passing >>> 'port=0' value and then reads the JMX Connector URL from Perf Counters. >> >> This looks fine.? Please add this issue number to @bug. >> >> This is interesting.? I was not aware of setting port=0 will >> auto-assign a free port.? Do you know if it was added for testing >> purpose (which I assume so)? > Port number of 0 is not handled at JMX or RMI layer. The given port > number is passed onto the ServerSocket. > > https://docs.oracle.com/javase/10/docs/api/java/net/ServerSocket.html#%3Cinit%3E(int) > > >> >> For example >> test/jdk/sun/management/jmxremote/bootstrap/RmiBootstrapTest.java >> uses jdk.testlibrary.Utils.getFreePort() to get a free port number. >> >> Mandy > Harsha > From daniel.daugherty at oracle.com Fri Jun 22 00:06:56 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 21 Jun 2018 20:06:56 -0400 Subject: RFR(XS): 8205195 NestedThreadsListHandleInErrorHandlingTest fails because hs_err doesn't contain _nested_thread_list_max In-Reply-To: References: Message-ID: May I please have a second reviewer on this one... Dan On 6/21/18 8:57 AM, Daniel D. Daugherty wrote: > Thomas, > > Thanks for the quick review. > > > On 6/21/18 2:53 AM, Thomas St?fe wrote: >> Hi Daniel, >> >> yes, that is annoying. >> >> I am okay with your fix, if you want to push it in this form. > > Thanks. > > >> But preparing the test crash in this way feels weird since the whole >> point of this exercise is to test error handling in close-to-real >> scenarios... but it sure does not hurt in this case. > > Yup. This is definitely weird, but my goal is to reduce testing noise. > I do acknowledge that the real world use of error reporting may run > into this failure mode which will suppress a section of hs_err_pid > output. > > >> Also, note that at different places we decide differently, see e.g. >> the "printing heap information" STEP - we omit locking Heap_lock in >> VMError::report() and only lock it in VMError::print_vm_info() (where >> we have no secondary signal handling and must not crash). So, in that >> case we are okay with risking a secondary crash in error handling. >> Probably there are just no regression tests for the heap information >> printout whose intermittent fails could annoy us :) > > Yup. I recognized that when I wrote the Thread-SMR tests I was making > them picky enough to possible run into failure modes we never would > have detected before. > > >> My feeling is that I would like to see a solution at the test >> framework side. Maybe, if a test is marked as "may fail rarely and >> thats okay", the test framework could retry the test and only fail if >> the error happens again. > > We currently don't have a way of tagging a test like that and I'm > not convinced that I would really want us to do that. However, this > particular bug truly falls into a no win scenario and that's a > different situation than I've encountered before. > > Again, thanks for the review. > > Dan > > >> >> Thanks, Thomas >> >> >> >> On Thu, Jun 21, 2018 at 2:18 AM, Daniel D. Daugherty >> wrote: >>> Greetings, >>> >>> I have a fix for a recent (very rare) Thread-SMR related test failure. >>> >>> Since the fix is related to the ErrorHandling tests and affects >>> hs_err_pid >>> file generation, this code review is being sent to both the Runtime and >>> the Serviceability teams. Please make sure you reply-all to any >>> responses >>> so we have complete review threads on both aliases. >>> >>> Bug URL: https://bugs.openjdk.java.net/browse/JDK-8205195 >>> >>> Webrev URL: >>> http://cr.openjdk.java.net/~dcubed/8205195-webrev/0-for-jdk-jdk/ >>> >>> The bug itself contains analysis about the root cause of the bug and >>> the comment updates to the code describes the no win scenario that the >>> hs_err_pid file generation code is in. Of course, I also have a comment >>> where I was able to harden the ErrorHandling tests. I did manage to >>> resist the urge to mention the "Kobiyashi Maru" [1] in the new >>> comments. >>> >>> Testing: Mach5 >>> builds-tier1,jdk-tier1,jdk-tier2,hs-tier1,hs-tier2,hs-tier3 >>> ????????? on the usual Oracle platforms. >>> >>> Thanks, in advance, for any comments, questions or suggestions. >>> >>> Dan >>> >>> [1] https://www.urbandictionary.com/define.php?term=Kobayashi%20Maru >>> > > From serguei.spitsyn at oracle.com Fri Jun 22 00:21:01 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 21 Jun 2018 17:21:01 -0700 Subject: RFR(XS): 8205195 NestedThreadsListHandleInErrorHandlingTest fails because hs_err doesn't contain _nested_thread_list_max In-Reply-To: References: Message-ID: <6d92d5b6-e6fe-de47-59d8-146a42940dd7@oracle.com> An HTML attachment was scrubbed... URL: From daniel.daugherty at oracle.com Fri Jun 22 01:07:22 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 21 Jun 2018 21:07:22 -0400 Subject: RFR(XS): 8205195 NestedThreadsListHandleInErrorHandlingTest fails because hs_err doesn't contain _nested_thread_list_max In-Reply-To: <6d92d5b6-e6fe-de47-59d8-146a42940dd7@oracle.com> References: <6d92d5b6-e6fe-de47-59d8-146a42940dd7@oracle.com> Message-ID: <70312635-87b3-f998-ad80-25a9d3cefa38@oracle.com> On 6/21/18 8:21 PM, serguei.spitsyn at oracle.com wrote: > Hi Dan, > > The fix looks Okay to me. Thanks for the review! > http://cr.openjdk.java.net/~dcubed/8205195-webrev/0-for-jdk-jdk/src/hotspot/share/runtime/threadSMR.cpp.udiff.html > > A typo in comment - "during" occurred twice: > ? + // Note: Not grabbing the Threads_lock during during error reporting Nice catch. Will fix that before I push (and reformat the comment). I read that comment at least three times before I sent out the review... sigh... Dan > > > Thanks, > Serguei > > > On 6/20/18 17:18, Daniel D. Daugherty wrote: >> Greetings, >> >> I have a fix for a recent (very rare) Thread-SMR related test failure. >> >> Since the fix is related to the ErrorHandling tests and affects >> hs_err_pid >> file generation, this code review is being sent to both the Runtime and >> the Serviceability teams. Please make sure you reply-all to any >> responses >> so we have complete review threads on both aliases. >> >> Bug URL: https://bugs.openjdk.java.net/browse/JDK-8205195 >> >> Webrev URL: >> http://cr.openjdk.java.net/~dcubed/8205195-webrev/0-for-jdk-jdk/ >> >> The bug itself contains analysis about the root cause of the bug and >> the comment updates to the code describes the no win scenario that the >> hs_err_pid file generation code is in. Of course, I also have a comment >> where I was able to harden the ErrorHandling tests. I did manage to >> resist the urge to mention the "Kobiyashi Maru" [1] in the new comments. >> >> Testing: Mach5 >> builds-tier1,jdk-tier1,jdk-tier2,hs-tier1,hs-tier2,hs-tier3 >> ?? ? ? ? on the usual Oracle platforms. >> >> Thanks, in advance, for any comments, questions or suggestions. >> >> Dan >> >> [1] https://www.urbandictionary.com/define.php?term=Kobayashi%20Maru >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From harsha.wardhana.b at oracle.com Fri Jun 22 04:37:51 2018 From: harsha.wardhana.b at oracle.com (Harsha Wardhana B) Date: Fri, 22 Jun 2018 10:07:51 +0530 Subject: RFR : JDK-8204661 - Show error 'Port already in use' in HashedPasswordFileTest.java In-Reply-To: References: <9fa7bd72-6476-b702-ba34-ec8fc572cfcb@oracle.com> <674a1a9a-0528-a2d1-f09f-b9d8724d62ec@oracle.com> <4d87b834-3b78-7b03-e90b-13385050fb19@oracle.com> Message-ID: Hi Dan, The utility function Utils.getFreePort gets free port by creating server socket with port 0. Instead of getting a free port before hand (via Utils.getFreePort), I am letting the JMX agent choose the free port. The risks associated with choosing free port has moved from test lib to the product. Do you think this is something that should be handled at test-level? Maybe refactor the test library (Utils.getFreePort)? Thanks Harsha On Thursday 21 June 2018 07:28 PM, Daniel D. Daugherty wrote: > > Port number of 0 is not handled at JMX or RMI layer. The given port > number is passed onto the ServerSocket. > > > > > https://docs.oracle.com/javase/10/docs/api/java/net/ServerSocket.html#%3Cinit%3E(int) > > > Using a port number of 0 to get a free port has to be done very carefully > to avoid running into other, more difficult to find issues. > > Jerry T. fix a few of these before he left us: > > ??? JDK-8182757 JDWP: Socket Transport handshake hangs on Solaris > ??? https://bugs.openjdk.java.net/browse/JDK-8182757 > > ??? JDK-8178676 nsk/jvmti/AttachOnDemand/attach045 fails with Exception > ??? https://bugs.openjdk.java.net/browse/JDK-8178676 > > ??? JDK-8188867 > nsk/jdi/VirtualMachineManager/createVirtualMachine/createVM004 > ??????????????? (and other tests) timeout do to socket problem > ??? https://bugs.openjdk.java.net/browse/JDK-8188867 > > > As an example of the difficulty, I tried to fix this bug: > > ??? JDK-8182307 Error during JRMP connection establishment > ??? https://bugs.openjdk.java.net/browse/JDK-8182307 > > and had to back out my fix because it caused a couple of tests to fail. > I'm tracking the redo here: > > ??? JDK-8193227 [REDO] 8182307 Error during JRMP connection establishment > ??? https://bugs.openjdk.java.net/browse/JDK-8193227 > > and I haven't had an epiphany for how to fix it without breaking > other tests (yet) so I just decommitted it from JDK11. > > > I'm not saying that your current fix will be susceptible to very > intermittent hangs on Solaris. I would have to take a very close > look at it and I haven't done that yet. > > Dan > > > On 6/21/18 2:47 AM, Harsha Wardhana B wrote: >> >> >> On Thursday 21 June 2018 10:30 AM, mandy chung wrote: >>> >>> >>> On 6/20/18 1:14 AM, Harsha Wardhana B wrote: >>>> Hi, >>>> >>>> Please find the fix below for the bug >>>> >>>> JDK-8204661 : Show error 'Port already in use' in >>>> HashedPasswordFileTest.java >>>> >>>> having webrev at, >>>> >>>> http://cr.openjdk.java.net/~hb/8204661/webrev.00/ >>>> >>>> The problem root-cause is discussed in the comments section of the >>>> bug. >>>> >>>> The fix above lets the default agent pick a free port by passing >>>> 'port=0' value and then reads the JMX Connector URL from Perf >>>> Counters. >>> >>> This looks fine.? Please add this issue number to @bug. >>> >>> This is interesting.? I was not aware of setting port=0 will >>> auto-assign a free port.? Do you know if it was added for testing >>> purpose (which I assume so)? >> Port number of 0 is not handled at JMX or RMI layer. The given port >> number is passed onto the ServerSocket. >> >> https://docs.oracle.com/javase/10/docs/api/java/net/ServerSocket.html#%3Cinit%3E(int) >> >> >>> >>> For example >>> test/jdk/sun/management/jmxremote/bootstrap/RmiBootstrapTest.java >>> uses jdk.testlibrary.Utils.getFreePort() to get a free port number. >>> >>> Mandy >> Harsha >> > From goetz.lindenmaier at sap.com Fri Jun 22 06:08:39 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 22 Jun 2018 06:08:39 +0000 Subject: RFR(S/M): 8205419: [testbug] TestJmapCore failing without SA: introduce @requires vm.hasSA In-Reply-To: References: <112db96d59084bdfafba2174e121c409@sap.com> Message-ID: Hi David, > Obviously using "@run othervm " you can invalidate what may have > been checked in a @require. As I understand with othervm you can not switch the user, or even start the other vm on a different system. So in this case, where no flags are checked, it should be fine. Even if you Sudo the jtreg runner as proposed in JDK-8199700. Thanks and best regards, Goetz > -----Original Message----- > From: David Holmes > Sent: Thursday, June 21, 2018 1:14 PM > To: Lindenmaier, Goetz ; serviceability-dev > (serviceability-dev at openjdk.java.net) dev at openjdk.java.net> > Subject: Re: RFR(S/M): 8205419: [testbug] TestJmapCore failing without SA: > introduce @requires vm.hasSA > > Answering VMProps question ... > > On 20/06/2018 11:49 PM, Lindenmaier, Goetz wrote: > > Hi, > > > > TestJmapCore is failing on aix because there jhsdb is not implemented. > > So far, such tests check at runtime Platform.shouldSAAttach() which is > missing > > in TestJmapCore. > > > > Instead, I implement @requires vm.hasSAandCanAttach. > > This makes jtreg skip the tests without compiling and starting them. > > > > I added the new property to all tests that check shouldSAAttach in jdk and > hostpot test > > directories. > > This is the implementation of the new property, the rest is just repetitive > checking it: > > http://cr.openjdk.java.net/~goetz/wr18/8205419- > requiresHasSA/01/test/jtreg-ext/requires/VMProps.java.udiff.html > > webrev: > > http://cr.openjdk.java.net/~goetz/wr18/8205419-requiresHasSA/01/ > > > > Is it correct to assume the VMProps is evaluated on the same machine and > with the same > > user as used for executing the test? canPtraceAttachLinux directly > accesses the system. > > jtreg will run the VM under test with all the VM flags specified to > jtreg and execute VMProps to gather information about the VM under test > before running the tests themselves. > > Obviously using "@run othervm " you can invalidate what may have > been checked in a @require. > > HTH > > David > > > > (Should I remove the checks for shouldSAAttach() from the changed tests?) > > > > Best regards, > > Goetz. > > From goetz.lindenmaier at sap.com Fri Jun 22 08:20:54 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 22 Jun 2018 08:20:54 +0000 Subject: RFR(S/M): 8205419: [testbug] TestJmapCore failing without SA: introduce @requires vm.hasSA In-Reply-To: References: <112db96d59084bdfafba2174e121c409@sap.com> <431269fc-7174-3051-0a86-baa00032754a@oracle.com> <3712cc995426467fa67b65493d556298@sap.com> Message-ID: <486ee0f785144f55bd739cc41bcd7e2d@sap.com> Hi, Ok, so I now added vm.hasSA and vm.hasSAandCanAttach I use the first in the JMapCore tests, and in the tests that don't check shouldSAAttach() right at the beginning. Wherever the test is immediately ended after checking shouldSAAttach(), I removed shouldSAAttach(), and added vm.hasSAandCanAttach. The others depend on ClhsdbLauncher. Just checking vm.hasSA is on the safe side if runOnCore() is changed. http://cr.openjdk.java.net/~goetz/wr18/8205419-requiresHasSA/02/ This also makes the implementation of VMProps and VMPlatform more clear as requested by Chris. Best regards, Goetz. > -----Original Message----- > From: Jini George > Sent: Thursday, June 21, 2018 12:18 PM > To: Lindenmaier, Goetz ; 'Chris Plummer' > ; serviceability-dev (serviceability- > dev at openjdk.java.net) > Subject: Re: RFR(S/M): 8205419: [testbug] TestJmapCore failing without SA: > introduce @requires vm.hasSA > > Hi Goetz, > > Tests like TestJmapCore which involve only corefile debugging by SA (and > not attaching to a live process) would only need to check for the > presence of SA, and would not need to check for ptrace related attach > permissions like what is done for Linux and OSX -- since super user > privileges are not required to debug corefiles. So a single @requires > catering to both these might not be desirable. > > Having said that, I realized that runOnCore() of ClhsdbLauncher > incorrectly checks for the ptrace related attach permissions. (Will file > an issue to correct this). > > An @requires instead of a shouldSAAttach() might help in avoiding fake > passes like those on OSX when not run as "root". > (https://bugs.openjdk.java.net/browse/JDK-8199700) > > Thanks, > Jini. > > > > > On 6/21/2018 1:52 AM, Lindenmaier, Goetz wrote: > > Hi Chris, > > > > Thanks for looking at this change. > > > >> Can't VMProps.vmHasSA() just return Platform.shouldSAAttach()? It > seems > >> you are unnecessarily replicating Platform.shouldSAAttach() logic. > > I think there are two things to distinguish: > > - platforms that don't implement SA > > - systems/setups where SA does not work. > > If both can be recognized when VMProps is evaluated, shouldSAAttach() > > is the one that is superfluous in the long run?! If not, shouldSAAttach > > should only return those where it is necessary to check in the test. > > But this is probably too much brains in this case ?? > > > >> If you are adding "@requires vm.hasSAandCanAttach" to a test, shouldn't > >> you remove the test's Platform.shouldSAAttach() check? > > Well I asked in my mail whether I should do that. So I take this as a yes. > > But it depends on whether it must be evaluated in the test. > > > >> Is there a reason not to take the more direct and established approach > >> of just adding the Platform.shouldSAAttach() check to TestJmapCore? It > >> would be a lot less disruptive.? I know the vm.hasSAandCanAttach > >> approach has advantages in not even attempting to compile and run the > >> test, but I'm not so sure those advantages outweigh the simplicity of > >> just adding a Platform.shouldSAAttach() check to one test. I can go > >> either way here. Just wondering if there are additional reasons I might > >> not be seeing. > > I like the @requires much more. It tells right where the test is described > > that it does not run with SA. The other is quite hidden in the test, e.g., in > > helper class ClhsdbLauncher.java. > > Also, it's more easy to remember (The implementor of > > TestJmapCore forgot it...). And, last but not least, it seems best practice > > nowadays. There are lots of excludes for mac, they are also done by > @requires > > and not as a check using Platform at runtime. > > > >> Sorry, I don't have an answer to your VMProps evaluation question. You > >> might need to ask a wider audience than just svc. > > Whom should I ask? Hotspot-dev? > > > > Best regards, > > Goetz. > > > > > >> > >> thanks, > >> > >> Chris > >> > >> On 6/20/18 6:49 AM, Lindenmaier, Goetz wrote: > >>> Hi, > >>> > >>> TestJmapCore is failing on aix because there jhsdb is not implemented. > >>> So far, such tests check at runtime Platform.shouldSAAttach() which is > >> missing > >>> in TestJmapCore. > >>> > >>> Instead, I implement @requires vm.hasSAandCanAttach. > >>> This makes jtreg skip the tests without compiling and starting them. > >>> > >>> I added the new property to all tests that check shouldSAAttach in jdk > and > >> hostpot test > >>> directories. > >>> This is the implementation of the new property, the rest is just > repetitive > >> checking it: > >>> http://cr.openjdk.java.net/~goetz/wr18/8205419- > >> requiresHasSA/01/test/jtreg-ext/requires/VMProps.java.udiff.html > >>> webrev: > >>> http://cr.openjdk.java.net/~goetz/wr18/8205419-requiresHasSA/01/ > >>> > >>> Is it correct to assume the VMProps is evaluated on the same machine > and > >> with the same > >>> user as used for executing the test? canPtraceAttachLinux directly > >> accesses the system. > >>> > >>> (Should I remove the checks for shouldSAAttach() from the changed > tests?) > >>> > >>> Best regards, > >>> Goetz. > >> > >> > > From david.holmes at oracle.com Fri Jun 22 12:02:34 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 22 Jun 2018 22:02:34 +1000 Subject: RFR(S/M): 8205419: [testbug] TestJmapCore failing without SA: introduce @requires vm.hasSA In-Reply-To: References: <112db96d59084bdfafba2174e121c409@sap.com> Message-ID: On 22/06/2018 4:08 PM, Lindenmaier, Goetz wrote: > Hi David, > >> Obviously using "@run othervm " you can invalidate what may have >> been checked in a @require. > As I understand with othervm you can not switch the user, or even start the > other vm on a different system. No of course not, this all happens on the system under the same user within the context of the same jtreg execution. I was simply pointing out that you can do silly things like: @require vm.gc = G1GC (or whatever the syntax is) and then have: @run main/othervm -XX:-UseG1GC -XX:+UseSerialGC Test Cheers, David > So in this case, where no flags are checked, it should be fine. Even if you > Sudo the jtreg runner as proposed in JDK-8199700. > > Thanks and best regards, > Goetz > > >> -----Original Message----- >> From: David Holmes >> Sent: Thursday, June 21, 2018 1:14 PM >> To: Lindenmaier, Goetz ; serviceability-dev >> (serviceability-dev at openjdk.java.net) > dev at openjdk.java.net> >> Subject: Re: RFR(S/M): 8205419: [testbug] TestJmapCore failing without SA: >> introduce @requires vm.hasSA >> >> Answering VMProps question ... >> >> On 20/06/2018 11:49 PM, Lindenmaier, Goetz wrote: >>> Hi, >>> >>> TestJmapCore is failing on aix because there jhsdb is not implemented. >>> So far, such tests check at runtime Platform.shouldSAAttach() which is >> missing >>> in TestJmapCore. >>> >>> Instead, I implement @requires vm.hasSAandCanAttach. >>> This makes jtreg skip the tests without compiling and starting them. >>> >>> I added the new property to all tests that check shouldSAAttach in jdk and >> hostpot test >>> directories. >>> This is the implementation of the new property, the rest is just repetitive >> checking it: >>> http://cr.openjdk.java.net/~goetz/wr18/8205419- >> requiresHasSA/01/test/jtreg-ext/requires/VMProps.java.udiff.html >>> webrev: >>> http://cr.openjdk.java.net/~goetz/wr18/8205419-requiresHasSA/01/ >>> >>> Is it correct to assume the VMProps is evaluated on the same machine and >> with the same >>> user as used for executing the test? canPtraceAttachLinux directly >> accesses the system. >> >> jtreg will run the VM under test with all the VM flags specified to >> jtreg and execute VMProps to gather information about the VM under test >> before running the tests themselves. >> >> Obviously using "@run othervm " you can invalidate what may have >> been checked in a @require. >> >> HTH >> >> David >> >> >>> (Should I remove the checks for shouldSAAttach() from the changed tests?) >>> >>> Best regards, >>> Goetz. >>> From jeremymanson at google.com Fri Jun 22 15:22:57 2018 From: jeremymanson at google.com (Jeremy Manson) Date: Fri, 22 Jun 2018 08:22:57 -0700 Subject: RFR 8205113: Update JVMTI doc references to object allocation tracking In-Reply-To: References: <384783b9-9ec2-093a-2662-a7a817849720@oracle.com> <3d01255e-8712-46aa-7659-4ccd7325f81c@oracle.com> <6927237d-ce0b-17c0-68f8-06d05b84e516@oracle.com> <53f576e2-0adb-a6c5-8a3c-026dcb1aa7ad@oracle.com> <8224afa2-a6cb-fee2-8fa3-184828ac7c50@oracle.com> <60899f09-12df-1e6e-7e16-406a60939b7c@oracle.com> Message-ID: Hey folks - With the window closing soon for 11, I feel that we should get this revision in (just so that the spec is clear). That said, I don't feel strongly about the wording. Who gets to make the decision? Jeremy On Wed, Jun 20, 2018 at 9:41 AM Jeremy Manson wrote: > FWIW, I'm also okay with David's version, with the caveat that "These > methods should send this event" should probably read "Invocations of these > methods should trigger this event". > > Jeremy > > On Wed, Jun 20, 2018 at 4:11 AM David Holmes > wrote: > >> On 20/06/2018 4:48 PM, serguei.spitsyn at oracle.com wrote: >> > On 6/19/18 23:29, Jeremy Manson wrote: >> >> Maybe we should make that clarification. >> >> >> >> Also, the reason I danced around that in my revision is that >> > >> > Understand that. >> > But it is not a good style to clarify about SampledObjectAlloc in the >> > spec of VMObjectAlloc. >> > Would be nice to find a way to keep it simple. >> > >> > We could keep the original spec as it is but add that all this does not >> > apply to the SampledObjectAlloc events. >> >> I don't think that really presents a coherent spec. I still like my >> suggestion :) >> >> David >> >> > >> > Something like below: >> > >> > Note, the above does not apply to the > > id="SampledObjectAlloc">SampledObjectAlloc >> > event as it is triggered on all Java object allocations, including >> > those caused by bytecode method execution, >> > JNI method execution, and directly by VM methods.* >> > * >> > >> >> if you set the sampling rate to 0, you get unconditional sampling. >> > A correction. >> > >> > I wrote: >> > > It is still possible to sample every allocation with the >> SamplingRate >> > == 1. >> > >> > but had write: SamplingRate == 0 >> > >> > Thanks, >> > Serguei >> > >> > >> >> >> >> Jeremy >> >> >> >> On Tue, Jun 19, 2018 at 10:41 PM serguei.spitsyn at oracle.com >> >> > >> > wrote: >> >> >> >> On 6/19/18 21:54, David Holmes wrote: >> >> > On 20/06/2018 2:41 PM, serguei.spitsyn at oracle.com >> >> wrote: >> >> >> On 6/19/18 21:11, Jeremy Manson wrote: >> >> >>> That would be okay with me, assuming that my other corrections >> >> are >> >> >>> made. >> >> >> >> >> >> Another option would be to say "non-sampling" instead of >> >> >> "unconditional": >> >> >> >> >> >> == Sent when a method causes the virtual machine to allocate an >> >> >> == Object visible to Java programming language code and the >> >> allocation >> >> >> += is not detectable by other *non-sampling* instrumentation >> >> mechanisms. >> >> >> >> >> >> >> >> >>> I'd also like to fix the spelling of instrumentation in the >> first >> >> >>> sentence. >> >> >> >> >> >> Yes, of course. >> >> >> >> >> >> Let's wait for David's opinion. >> >> > >> >> > I'm okay with Serguei's suggestion (combined with Jeremy's other >> >> > changes). >> >> > >> >> > I'm not that familiar with conditional versus unconditional >> >> > instrumentation. >> >> >> >> The whole point of the SampledObjectAlloc events is to post them >> >> conditionally >> >> depending on the SamplingRate so that just some amount of >> >> allocations is >> >> sampled. >> >> It is why the overhead can be minimal (less than 5 percents). >> >> It is still possible to sample every allocation with the >> >> SamplingRate == 1. >> >> >> >> The VMObjectAlloc events are posted unconditionally, so every VM >> >> allocation is posted. >> >> >> >> Thanks, >> >> Serguei >> >> >> >> >> >> > >> >> > Thanks, >> >> > David >> >> > >> >> >> >> >> >> Thanks, >> >> >> Serguei >> >> >> >> >> >> >> >> >>> >> >> >>> Jeremy >> >> >>> >> >> >>> On Mon, Jun 18, 2018 at 11:01 PM serguei.spitsyn at oracle.com >> >> >> >> >>> > >> > > >> >> >> >>> > >> >> wrote: >> >> >>> >> >> >>> Hi Jeremy and David, >> >> >>> >> >> >>> Sorry for being late to the party. >> >> >>> >> >> >>> I'm also concerned about the Jeremy's spec update is more >> >> >>> intrusive than necessary. >> >> >>> One specifics of the new SampledObjectAlloc event is that >> >> it is >> >> >>> posted conditionally. >> >> >>> So, it is not fully comparable with the VMObjectAlloc >> >> event and >> >> >>> can not replace it in any way. >> >> >>> I'm even not yet convinced that any spec update is >> necessary. >> >> >>> >> >> >>> However, something like below would look minimal and >> >> reasonable: >> >> >>> >> >> >>> == Sent when a method causes the virtual machine to >> >> allocate an >> >> >>> == Object visible to Java programming language code and the >> >> >>> allocation >> >> >>> += is not detectable by other *unconditional* >> intrumentation >> >> >>> mechanisms. >> >> >>> >> >> >>> == Generally object allocation should be detected by >> >> instrumenting >> >> >>> == the bytecodes of allocating methods. >> >> >>> >> >> >>> == Object allocation generated in native code by JNI >> function >> >> >>> == calls should be detected using >> >> >>> == JNI function >> >> >>> interception. >> >> >>> >> >> >>> == Some methods might not have associated bytecodes and >> >> are not >> >> >>> == native methods, they instead are executed directly by >> the >> >> >>> == VM. These methods should send this event. >> >> >>> >> >> >>> == Virtual machines which are incapable of bytecode >> >> instrumentation >> >> >>> == for some or all of their methods can send this event. >> >> >>> >> >> >>> *++ Note that the > >> >>> id="SampledObjectAlloc">SampledObjectAlloc** >> >> >>> **++ event is conditionally triggered on all Java object >> >> >>> allocations, including those** >> >> >>> **++ caused by bytecode method execution, JNI method >> >> execution, >> >> >>> and directly by VM methods.** >> >> >>> * >> >> >>> >> >> >>> Maybe, just adding the last statement would be good enough. >> >> >>> >> >> >>> Thanks, >> >> >>> Serguei >> >> >>> >> >> >>> >> >> >>> On 6/18/18 21:36, David Holmes wrote: >> >> >>>> On 19/06/2018 4:50 AM, Jeremy Manson wrote: >> >> >>>>> Yup! The paragraph meanders a bit. How about something >> >> like: >> >> >>>> >> >> >>>> I'm not sure some of the change quite works. The original >> >> text >> >> >>>> considers there to be three kinds of methods that can >> cause >> >> >>>> allocation when executed: >> >> >>>> - Java (bytecode) methods >> >> >>>> - JNI methods >> >> >>>> - VM methods >> >> >>>> >> >> >>>> but you've turned this into three kinds of allocation: via >> >> >>>> bytecode, via JNI, and via the VM. You then refer to >> >> "triggering" >> >> >>>> an allocation when we tend to use triggering for events. >> >> You also >> >> >>>> refer to an allocation being "executed directly by the >> VM" (a >> >> >>>> phrase previously applied when the subject was a >> >> 'method') - but >> >> >>>> you don't really execute allocations. >> >> >>>> >> >> >>>> IIUC the problem with the existing text is just that it >> >> considers >> >> >>>> VM allocation events as being undetectable other than by >> >> this "VM >> >> >>>> object allocation event" - but that's no longer true. So >> how >> >> >>>> about something minimally changed like this: >> >> >>>> >> >> >>>> --- >> >> >>>> Sent when a method causes the virtual machine to >> directly >> >> >>>> allocate an >> >> >>>> Object visible to Java programming language code. >> >> >>>> Generally object allocation can be detected by >> >> instrumenting >> >> >>>> the bytecodes of allocating methods. >> >> >>>> Object allocation generated in native code by JNI >> function >> >> >>>> calls can be detected using >> >> >>>> JNI function >> >> >>>> interception. >> >> >>>> Some methods might not have associated bytecodes and >> >> are not >> >> >>>> native methods, they instead are executed directly by >> the >> >> >>>> VM. These methods should send this event. >> >> >>>> Virtual machines which are incapable of bytecode >> >> >>>> instrumentation >> >> >>>> for some or all of their methods can send this event. >> >> >>>> >> >> >>>> Note that the > >> >>>> id="SampledObjectAlloc">SampledObjectAlloc >> >> >>>> event is triggered on all Java object allocations, >> >> including >> >> >>>> those >> >> >>>> caused by bytecode method execution, JNI method >> >> execution, and >> >> >>>> directly by VM methods. >> >> >>>> --- >> >> >>>> >> >> >>>> Thanks, >> >> >>>> David >> >> >>>> >> >> >>>>> Sent when the virtual machine allocates an >> >> >>>>> Object visible to Java programming language code without >> >> using a >> >> >>>>> new bytecode variant or a JNI method. >> >> >>>>> Many approaches to tracking object allocation use a >> >> >>>>> combination of >> >> >>>>> bytecode-based instrumentation and > >> >>>>> id="jniIntercept">JNI function >> >> >>>>> interception to intercept allocations. >> >> However, >> >> >>>>> this >> >> >>>>> approach can leave a number of allocations undetected. >> >> >>>>> Allocations that are neither >> >> >>>>> triggered by bytecode nor JNI are executed directly by >> >> the VM. >> >> >>>>> When those allocations occur, the VM should send this >> event. >> >> >>>>> Virtual machines that are incapable of bytecode >> >> instrumentation >> >> >>>>> for some or all of their methods may also send this >> event. >> >> >>>>>

>> >> >>>>> Note that the > >> >>>>> id="SampledObjectAlloc">SampledObjectAlloc >> >> >>>>> event is triggered on all Java object allocations, >> including >> >> >>>>> those triggered by bytecode, >> >> >>>>> JNI methods, and VM events. >> >> >>>>> >> >> >>>>> >> >> >>>>> >> >> >>>>> On Mon, Jun 18, 2018 at 12:57 AM David Holmes >> >> >>>>> > >> > >> > >> >> >>>>> > >> > >> >> >>>>> > >> >> wrote: >> >> >>>>> >> >> >>>>> On 18/06/2018 5:01 PM, Jeremy Manson wrote: >> >> >>>>> > We haven't changed when a VM issues "VM object >> >> >>>>> allocation" events. >> >> >>>>> > >> >> >>>>> > There were references in the docs to a >> >> requirement to use >> >> >>>>> bytecode >> >> >>>>> > rewriting and JNI interception to track >> >> allocations. With >> >> >>>>> > SampledObjectAlloc, this is no longer the case - >> >> >>>>> SampledObjectAlloc can >> >> >>>>> > track them. This change is supposed to remove the >> >> >>>>> references to >> >> >>>>> those >> >> >>>>> > requirements, and provide suitable replacement >> text. >> >> >>>>> > >> >> >>>>> > VM object alloc has specific language about being >> >> able to >> >> >>>>> use it to >> >> >>>>> > track allocations that cannot be tracked with >> >> bytecode >> >> >>>>> instrumentation >> >> >>>>> > and JNI interception. My goal in rephrasing was >> >> to make >> >> >>>>> it clear >> >> >>>>> that, >> >> >>>>> > while you can still use it for this, you can also >> >> just use >> >> >>>>> > SampledObjectAlloc for everything. >> >> >>>>> >> >> >>>>> Okay. That doesn't come across clearly to me - >> >> sorry. So you >> >> >>>>> will now >> >> >>>>> get both kinds of events for allocations done in the >> VM? >> >> >>>>> >> >> >>>>> Thanks, >> >> >>>>> David >> >> >>>>> >> >> >>>>> >> >> >>>>> > Jeremy >> >> >>>>> > >> >> >>>>> > On Sun, Jun 17, 2018 at 9:11 PM David Holmes >> >> >>>>> > >> > >> > >> >> >>>>> > >> > > >> > >> >> >>>>> > > >> >> >> >>>>> > >> > >> >> >>>>> > >> >>> wrote: >> >> >>>>> > >> >> >>>>> > Hi Jeremy, >> >> >>>>> > >> >> >>>>> > On 16/06/2018 2:33 AM, Jeremy Manson wrote: >> >> >>>>> > > Hi all, >> >> >>>>> > > >> >> >>>>> > > There are a number of references in the >> >> JVMTI doc >> >> >>>>> to its >> >> >>>>> not doing >> >> >>>>> > > object allocation tracking. Now that JEP >> >> 331 has >> >> >>>>> landed, >> >> >>>>> these >> >> >>>>> > > references are obsolete. This patch >> >> changes those >> >> >>>>> references >> >> >>>>> > accordingly. >> >> >>>>> > > >> >> >>>>> > > While I was there, I took the liberty of >> >> fixing >> >> >>>>> some >> >> >>>>> spelling errors. >> >> >>>>> > > >> >> >>>>> > > As far as I know, these are non-normative >> >> >>>>> changes and >> >> >>>>> modify no >> >> >>>>> > API, so >> >> >>>>> > > they should not require a CSR. >> >> >>>>> > >> >> >>>>> > I'm unclear on the nature of the change to >> >> "VM Object >> >> >>>>> Allocation". Does >> >> >>>>> > the existence of SampledObjectAlloc change >> >> when a VM >> >> >>>>> should >> >> >>>>> issue "VM >> >> >>>>> > object allocation" events? >> >> >>>>> > >> >> >>>>> > Thanks, >> >> >>>>> > David >> >> >>>>> > >> >> >>>>> > > >> >> >>>>> > > Bug: >> >> >>>>> > > >> >> https://bugs.openjdk.java.net/browse/JDK-8205113 >> >> >>>>> > > >> >> >>>>> > > Webrev: >> >> >>>>> > > >> >> >>>>> http://cr.openjdk.java.net/~jmanson/8205113/webrev.00/ >> >> >> >> >>>>> >> >> >>>>> > > >> >> >>>>> > > Thanks! >> >> >>>>> > > >> >> >>>>> > > Jeremy >> >> >>>>> > >> >> >>>>> >> >> >>> >> >> >> >> >> >> > >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Fri Jun 22 16:00:15 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 22 Jun 2018 09:00:15 -0700 Subject: RFR 8205113: Update JVMTI doc references to object allocation tracking In-Reply-To: References: <384783b9-9ec2-093a-2662-a7a817849720@oracle.com> <3d01255e-8712-46aa-7659-4ccd7325f81c@oracle.com> <6927237d-ce0b-17c0-68f8-06d05b84e516@oracle.com> <53f576e2-0adb-a6c5-8a3c-026dcb1aa7ad@oracle.com> <8224afa2-a6cb-fee2-8fa3-184828ac7c50@oracle.com> <60899f09-12df-1e6e-7e16-406a60939b7c@oracle.com> Message-ID: <96713bc3-9350-fd4a-75e5-2c36257066b2@oracle.com> An HTML attachment was scrubbed... URL: From jini.george at oracle.com Fri Jun 22 18:11:52 2018 From: jini.george at oracle.com (Jini George) Date: Fri, 22 Jun 2018 23:41:52 +0530 Subject: RFR: JDK-8189429: SA: MacOSX: Replace the deprecated PT_ATTACH with PT_ATTACHEXC In-Reply-To: References: Message-ID: <7abea33e-7fb0-fbe0-d463-8999ccdf54da@oracle.com> Hi all, [Including build-dev also since this includes build related changes]. Requesting reviews for: https://bugs.openjdk.java.net/browse/JDK-8189429 (SA: MacOSX: Replace the deprecated PT_ATTACH with PT_ATTACHEXC) Webrev: http://cr.openjdk.java.net/~jgeorge/8189429/webrev.04/ This is the follow-up solution for the temporary restoration of PT_ATTACH to fix https://bugs.openjdk.java.net/browse/JDK-8184042 (several serviceability/sa tests timed out on MacOS X), which was done in Oct 2017. The mails related to this are at: http://mail.openjdk.java.net/pipermail/serviceability-dev/2017-August/021702.html Changes as compared to the patch sent last year (http://cr.openjdk.java.net/~jgeorge/8184042/webrev.00/): * Addressed the review comments which were provided by Poonam, Dan, Dmitry. * A major change as compared to what was done last year is that the MIG generated files have been included as a part of the JDK build process. * The test case which was provided in the patch last year is no longer required since we have ClhsdbAttach.java testing the same functionality as a part of the SA testsuite now. * Other than that, some minor improvements have been done wrt error handling. The steps for the proposed solution have been provided in JBS. Testing: ALL the SA tests pass on MacOSX and the other Mach5 platforms. Thanks to Sharath, Robin, Gerard and Dan for looking into the changes and providing valuable comments. Thanks, Jini. From thomas.stuefe at gmail.com Fri Jun 22 19:08:33 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 22 Jun 2018 21:08:33 +0200 Subject: RFR(s): https://bugs.openjdk.java.net/browse/JDK-8205531 Message-ID: Hi all, may I have reviews for this small enhancement to the jcmd VM.classloader diagnostic command: https://bugs.openjdk.java.net/browse/JDK-8205531 http://cr.openjdk.java.net/~stuefe/webrevs/8205531-vm.classloader-tree-folding/webrev.00/webrev/ VM.classloaders prints a tree of class loaders. This tree can grow a lot and become unwieldy, especially with a lot of similar loaders. One prime example is the DelegatingClassLoader. It would be helpful were all these loaders: 13114: +-- | +-- "platform", jdk.internal.loader.ClassLoaders$PlatformClassLoader | +-- "app", jdk.internal.loader.ClassLoaders$AppClassLoader | +-- test3.internals.InMemoryClassLoader | +-- jdk.internal.reflect.DelegatingClassLoader | +-- jdk.internal.reflect.DelegatingClassLoader | +-- jdk.internal.reflect.DelegatingClassLoader | +-- jdk.internal.reflect.DelegatingClassLoader | +-- jdk.internal.reflect.DelegatingClassLoader | ...... repeat 1495 times folded into one: 13114: +-- | +-- "platform", jdk.internal.loader.ClassLoaders$PlatformClassLoader | +-- "app", jdk.internal.loader.ClassLoaders$AppClassLoader | +-- test3.internals.InMemoryClassLoader | +-- jdk.internal.reflect.DelegatingClassLoader (+ 1499 more) Original idea by Bernd Eckenfels, see http://mail.openjdk.java.net/pipermail/serviceability-dev/2018-May/023824.html Thank you, Thomas From thomas.stuefe at gmail.com Fri Jun 22 19:10:30 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 22 Jun 2018 21:10:30 +0200 Subject: RFR(s): jcmd VM.classloaders should fold similar loaders Message-ID: Resent with the correct subject, sorry. ..Thomas On Fri, Jun 22, 2018 at 9:08 PM, Thomas St?fe wrote: > Hi all, > > may I have reviews for this small enhancement to the jcmd > VM.classloader diagnostic command: > > https://bugs.openjdk.java.net/browse/JDK-8205531 > http://cr.openjdk.java.net/~stuefe/webrevs/8205531-vm.classloader-tree-folding/webrev.00/webrev/ > > > VM.classloaders prints a tree of class loaders. This tree can grow a > lot and become unwieldy, especially with a lot of similar loaders. One > prime example is the DelegatingClassLoader. It would be helpful were > all these loaders: > > 13114: > +-- > | > +-- "platform", jdk.internal.loader.ClassLoaders$PlatformClassLoader > | > +-- "app", jdk.internal.loader.ClassLoaders$AppClassLoader > | > +-- test3.internals.InMemoryClassLoader > | > +-- jdk.internal.reflect.DelegatingClassLoader > | > +-- jdk.internal.reflect.DelegatingClassLoader > | > +-- jdk.internal.reflect.DelegatingClassLoader > | > +-- jdk.internal.reflect.DelegatingClassLoader > | > +-- jdk.internal.reflect.DelegatingClassLoader > | > ...... repeat 1495 times > > folded into one: > > 13114: > +-- > | > +-- "platform", jdk.internal.loader.ClassLoaders$PlatformClassLoader > | > +-- "app", jdk.internal.loader.ClassLoaders$AppClassLoader > | > +-- test3.internals.InMemoryClassLoader > | > +-- jdk.internal.reflect.DelegatingClassLoader > (+ 1499 more) > > > Original idea by Bernd Eckenfels, see > http://mail.openjdk.java.net/pipermail/serviceability-dev/2018-May/023824.html > > Thank you, Thomas From coleen.phillimore at oracle.com Fri Jun 22 21:40:44 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 22 Jun 2018 17:40:44 -0400 Subject: [12] RFR (S) 8205534: Remove SymbolTable dependency from serviceability agent Message-ID: <88e391a8-78a2-8dbc-a489-fce9c6b922b5@oracle.com> Summary: Modify SA code to not use SymbolTable and remove it. This is to support the concurrent hashtable for SymbolTable. open webrev at http://cr.openjdk.java.net/~coleenp/8205534.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8205534 Tested with hs-tier1-5. Thanks, Coleen From serguei.spitsyn at oracle.com Fri Jun 22 21:55:41 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 22 Jun 2018 14:55:41 -0700 Subject: RFR(s): jcmd VM.classloaders should fold similar loaders In-Reply-To: References: Message-ID: An HTML attachment was scrubbed... URL: From erik.joelsson at oracle.com Fri Jun 22 22:01:13 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Fri, 22 Jun 2018 15:01:13 -0700 Subject: RFR: JDK-8189429: SA: MacOSX: Replace the deprecated PT_ATTACH with PT_ATTACHEXC In-Reply-To: <7abea33e-7fb0-fbe0-d463-8999ccdf54da@oracle.com> References: <7abea33e-7fb0-fbe0-d463-8999ccdf54da@oracle.com> Message-ID: Hello Jini, In general this looks pretty good, but it's also breaking some new ground as it's adding generation of native source in the java gensrc step. Mixing native code with the java code that the genrcs targets and gensrc output directories are meant for seems ok for now, but may cause trouble in the future. I'm going to accept it for now though. In Gensrc-jdk.hotspot.agent.gmk: Please put the new macosx stuff in its own section (as delimited by the 80x # lines) and put that whole thing inside a conditional for macosx. Also please break line 47 (for recipe lines, indent with tab and 4 additional spaces for continuation [1]). In Lib-jdk.hotspot.agent.gmk: I believe adding extra src should be enough as that will implicitly add the same directories as header dirs by default. At least that's the intention. /Erik [1] http://openjdk.java.net/groups/build/doc/code-conventions.html On 2018-06-22 11:11, Jini George wrote: > Hi all, > > [Including build-dev also since this includes build related changes]. > > Requesting reviews for: > > https://bugs.openjdk.java.net/browse/JDK-8189429 (SA: MacOSX: Replace > the deprecated PT_ATTACH with PT_ATTACHEXC) > > Webrev: http://cr.openjdk.java.net/~jgeorge/8189429/webrev.04/ > > This is the follow-up solution for the temporary restoration of > PT_ATTACH to fix https://bugs.openjdk.java.net/browse/JDK-8184042 > (several serviceability/sa tests timed out on MacOS X), which was done > in Oct 2017. The mails related to this are at: > > http://mail.openjdk.java.net/pipermail/serviceability-dev/2017-August/021702.html > > > Changes as compared to the patch sent last year > (http://cr.openjdk.java.net/~jgeorge/8184042/webrev.00/): > > * Addressed the review comments which were provided by Poonam, Dan, > Dmitry. > * A major change as compared to what was done last year is that the > MIG generated files have been included as a part of the JDK build > process. > * The test case which was provided in the patch last year is no longer > required since we have ClhsdbAttach.java testing the same > functionality as a part of the SA testsuite now. > * Other than that, some minor improvements have been done wrt error > handling. > > The steps for the proposed solution have been provided in JBS. > > Testing: ALL the SA tests pass on MacOSX and the other Mach5 platforms. > > Thanks to Sharath, Robin, Gerard and Dan for looking into the changes > and providing valuable comments. > > Thanks, > Jini. From daniil.x.titov at oracle.com Sat Jun 23 02:13:46 2018 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Fri, 22 Jun 2018 19:13:46 -0700 Subject: RFR 8203174: [Graal] JDI tests fail with Unexpected exception: com.sun.jdi.ObjectCollectedException Message-ID: Please review the changes that fix 15 tests failures when running with Graal. The main problem here is that the tests assume that no garbage collection happens before the tests complete their checks. The other problem is that the tests could not strictly distinguish the objects created by the debuggee on the request from the test from the objects of the same class created by the debuggee VM on its own. In case when java.lang.String or arrays of primitives are used in the tests the number of objects the test operates could significantly exceeds the number of the objects the test actually created and iterating over all of them results in the timeout. The fix ensures that the object references created in the tests are protected from garbage collection or modifies the test to expect an ObjectCollectedException where necessary. In some cases, the fix ensures that only the main thread of the debuggee is running to prevent the garbage collector thread to create objects of the same class the test is working with and avoid timeouts. Webrev: http://cr.openjdk.java.net/~dtitov/8203174/webrev.01 Bug: https://bugs.openjdk.java.net/browse/JDK-8203174 Thanks, Daniil From serguei.spitsyn at oracle.com Sat Jun 23 04:28:26 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 22 Jun 2018 21:28:26 -0700 Subject: RFR 8203174: [Graal] JDI tests fail with Unexpected exception: com.sun.jdi.ObjectCollectedException In-Reply-To: References: Message-ID: <41ae13c5-2513-3de5-b941-a79d0e332e96@oracle.com> Hi Daniil, It looks good - great analysis. Thank you for taking care about these failures! I guess, the tests could fail not only with the Graal. Hopefully, they became more stable now. Thanks, Serguei On 6/22/18 19:13, Daniil Titov wrote: > Please review the changes that fix 15 tests failures when running with Graal. > > The main problem here is that the tests assume that no garbage collection happens before the tests complete their checks. The other problem is that the tests could not strictly distinguish the objects created by the debuggee on the request from the test from the objects of the same class created by the debuggee VM on its own. In case when java.lang.String or arrays of primitives are used in the tests the number of objects the test operates could significantly exceeds the number of the objects the test actually created and iterating over all of them results in the timeout. > > The fix ensures that the object references created in the tests are protected from garbage collection or modifies the test to expect an ObjectCollectedException where necessary. In some cases, the fix ensures that only the main thread of the debuggee is running to prevent the garbage collector thread to create objects of the same class the test is working with and avoid timeouts. > > Webrev: http://cr.openjdk.java.net/~dtitov/8203174/webrev.01 > Bug: https://bugs.openjdk.java.net/browse/JDK-8203174 > > Thanks, > Daniil > > From chris.plummer at oracle.com Sat Jun 23 05:22:11 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Fri, 22 Jun 2018 22:22:11 -0700 Subject: RFR 8203174: [Graal] JDI tests fail with Unexpected exception: com.sun.jdi.ObjectCollectedException In-Reply-To: References: Message-ID: <365c7478-0795-cfd1-82c9-8dfbde0a2022@oracle.com> Hi Daniil, Thank you for stabilizing these tests. I don't think your comment changes for the following are correct: ?120???????????? // instances created in this way aren't reachable for the purposes of garbage collection, ?121???????????? // to make it reachable call enableCollection() for this objects ?122???????????? ArrayReference arrayReference = debuggee.createArray(arrayType, arraySize); They actually are reachable now that you are calling createArray(). Maybe you can just get rid of the comment now since its purpose was to explain why disableCollection() was being called, and this is no longer being done (at least no inline here). In referringObjects001.java you removed the following comment: ?155???????? // Note! This test is broken, in the sense that it incorrectly assumes ?156???????? // that no GC can happen before it walks the heap. In practice, it seems ?157???????? // to only affect this test when using ZGC. However, this test will also ?158???????? // fail when using other GCs if an explicit GC is done here. I guess the ZGC team must have run into failures with this test. I'm not so sure your changes fix this for ZGC. Don't the ZGC threads continue to run even after you call debugee.suspendOtherThreads()? Also, I don't see how the suspendOtherThreads helps here. createInstance() will always resume all threads before calling the constructor and then suspend all threads afterwards. So suspending beforehand doesn't seem like it would help at all. ?657???? public? ArrayReference createArray(ArrayType array, int arrayLength) { ?658???????? vm.suspend(); ?659???????? try { ?660???????????? ArrayReference arrayRef = array.newInstance(arrayLength); ?661???????????? arrayRef.disableCollection(); ?662???????????? return arrayRef; ?663???????? } finally { ?664???????????? vm.resume(); ?665???????? } ?666 ?667???? } Line 666 can be removed. thanks, Chris On 6/22/18 7:13 PM, Daniil Titov wrote: > Please review the changes that fix 15 tests failures when running with Graal. > > The main problem here is that the tests assume that no garbage collection happens before the tests complete their checks. The other problem is that the tests could not strictly distinguish the objects created by the debuggee on the request from the test from the objects of the same class created by the debuggee VM on its own. In case when java.lang.String or arrays of primitives are used in the tests the number of objects the test operates could significantly exceeds the number of the objects the test actually created and iterating over all of them results in the timeout. > > The fix ensures that the object references created in the tests are protected from garbage collection or modifies the test to expect an ObjectCollectedException where necessary. In some cases, the fix ensures that only the main thread of the debuggee is running to prevent the garbage collector thread to create objects of the same class the test is working with and avoid timeouts. > > Webrev: http://cr.openjdk.java.net/~dtitov/8203174/webrev.01 > Bug: https://bugs.openjdk.java.net/browse/JDK-8203174 > > Thanks, > Daniil > > From chris.plummer at oracle.com Sat Jun 23 05:42:09 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Fri, 22 Jun 2018 22:42:09 -0700 Subject: RFR(XS): 8205561: problem list vmTestbase/nsk/jdb/trace/trace001/trace001.java Message-ID: <12bed6fe-c1e8-1ec3-95a8-7f80aae887dc@oracle.com> Hello, Please review the following: https://bugs.openjdk.java.net/browse/JDK-8205561 diff --git a/test/hotspot/jtreg/ProblemList.txt b/test/hotspot/jtreg/ProblemList.txt --- a/test/hotspot/jtreg/ProblemList.txt +++ b/test/hotspot/jtreg/ProblemList.txt @@ -166,7 +166,7 @@ ?vmTestbase/vm/mlvm/indy/func/jvmti/mergeCP_indy2manySame_b/TestDescription.java 8013267 generic-all ?vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java 8197938 windows-all - +vmTestbase/nsk/jdb/trace/trace001/trace001.java 8205540 generic-all ?vmTestbase/nsk/jdwp/ThreadReference/ForceEarlyReturn/forceEarlyReturn001/forceEarlyReturn001.java 7199837 generic-all ?############################################################################# thanks, Chris From thomas.stuefe at gmail.com Sat Jun 23 07:13:26 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 23 Jun 2018 09:13:26 +0200 Subject: RFR(s): jcmd VM.classloaders should fold similar loaders In-Reply-To: References: Message-ID: Hi Serguei, On Fri, Jun 22, 2018 at 11:55 PM, serguei.spitsyn at oracle.com wrote: > Hi Thomas, > > > It looks good to me. > A couple of minor comments below. > > http://cr.openjdk.java.net/~stuefe/webrevs/8205531-vm.classloader-tree-folding/webrev.00/webrev/src/hotspot/share/classfile/classLoaderHierarchyDCmd.cpp.frames.html > > 135 // we we add a preliminary empty LoaderTreeNode for it. This > preliminary node > > A typo: "we we" > > 151 // In default view, similar tree nodes (same loader class, same or no > name) > > I'd suggest a minor tweak: "same or no name" => "same name or no name" as > at the line: > > 291 // and they have the same name or no name (note: leaf check is > done by caller). > > Otherwise, it is a little bit confusing because of the preceding "same > loader class". > > Makes all sense. Update: Delta: http://cr.openjdk.java.net/~stuefe/webrevs/8205531-vm.classloader-tree-folding/webrev.00-to-01/webrev/ Full: http://cr.openjdk.java.net/~stuefe/webrevs/8205531-vm.classloader-tree-folding/webrev.01/webrev/ I only changed the comments as suggested. Tests (jdk-submit) came back fine. > > On 6/22/18 12:10, Thomas St?fe wrote: > > Resent with the correct subject, sorry. > > > Also, it is better to list a bug number in the email subject. :) > :) I really must have been tired yesterday. But I will not change the subject again, to not tear up the thread. Thanks, Thomas > Thanks, > Serguei > > ..Thomas > > On Fri, Jun 22, 2018 at 9:08 PM, Thomas St?fe > wrote: > > Hi all, > > may I have reviews for this small enhancement to the jcmd > VM.classloader diagnostic command: > > https://bugs.openjdk.java.net/browse/JDK-8205531 > http://cr.openjdk.java.net/~stuefe/webrevs/8205531-vm.classloader-tree-folding/webrev.00/webrev/ > > > VM.classloaders prints a tree of class loaders. This tree can grow a > lot and become unwieldy, especially with a lot of similar loaders. One > prime example is the DelegatingClassLoader. It would be helpful were > all these loaders: > > 13114: > +-- > | > +-- "platform", jdk.internal.loader.ClassLoaders$PlatformClassLoader > | > +-- "app", jdk.internal.loader.ClassLoaders$AppClassLoader > | > +-- test3.internals.InMemoryClassLoader > | > +-- jdk.internal.reflect.DelegatingClassLoader > | > +-- jdk.internal.reflect.DelegatingClassLoader > | > +-- jdk.internal.reflect.DelegatingClassLoader > | > +-- jdk.internal.reflect.DelegatingClassLoader > | > +-- jdk.internal.reflect.DelegatingClassLoader > | > ...... repeat 1495 times > > folded into one: > > 13114: > +-- > | > +-- "platform", jdk.internal.loader.ClassLoaders$PlatformClassLoader > | > +-- "app", jdk.internal.loader.ClassLoaders$AppClassLoader > | > +-- test3.internals.InMemoryClassLoader > | > +-- jdk.internal.reflect.DelegatingClassLoader > (+ 1499 more) > > > Original idea by Bernd Eckenfels, see > http://mail.openjdk.java.net/pipermail/serviceability-dev/2018-May/023824.html > > Thank you, Thomas > > From thomas.stuefe at gmail.com Sat Jun 23 07:50:34 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 23 Jun 2018 09:50:34 +0200 Subject: Unify jcmd option syntax ? Message-ID: We have a cheerful multitude of syntaxes for sub options: some use words without leading dashes: Compiler.CodeHeap_Analytics VM.metaspace VM.classloaders ManagementAgent.start VM.native_memory e.g. VM.metaspace show-classes, VM.classloaders verbose some use words with dashes (-all, -verbose) GC.class_stats -csv some use single letters with dashes (-i, -s) VM.class_hierarchy -i -- Long term, should we clearly define a single syntax and stick with it? Preferably gnu-tools style, so single-dash-with-letters, double-dash-with-words? -- Short term, for the not yet released commands SAP added: Compiler.CodeHeap_Analytics VM.metaspace VM.classloaders we still can change the format before releasing. Should we? E.g. move to the "words with dashes" format the majority seems to use: VM.metaspace -show-classes, VM.classloaders -verbose ? Thanks, Thomas From serguei.spitsyn at oracle.com Sat Jun 23 07:52:29 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Sat, 23 Jun 2018 00:52:29 -0700 Subject: RFR(s): jcmd VM.classloaders should fold similar loaders In-Reply-To: References: Message-ID: <4284ab56-a62a-72ea-ed01-eb585a703e67@oracle.com> Hi Thomas, I forgot to tell that I did not want another webrev as it is a minor update. Please, ship it. Thanks, Serguei On 6/23/18 00:13, Thomas St?fe wrote: > Hi Serguei, > > On Fri, Jun 22, 2018 at 11:55 PM, serguei.spitsyn at oracle.com > wrote: >> Hi Thomas, >> >> >> It looks good to me. >> A couple of minor comments below. >> >> http://cr.openjdk.java.net/~stuefe/webrevs/8205531-vm.classloader-tree-folding/webrev.00/webrev/src/hotspot/share/classfile/classLoaderHierarchyDCmd.cpp.frames.html >> >> 135 // we we add a preliminary empty LoaderTreeNode for it. This >> preliminary node >> >> A typo: "we we" >> >> 151 // In default view, similar tree nodes (same loader class, same or no >> name) >> >> I'd suggest a minor tweak: "same or no name" => "same name or no name" as >> at the line: >> >> 291 // and they have the same name or no name (note: leaf check is >> done by caller). >> >> Otherwise, it is a little bit confusing because of the preceding "same >> loader class". >> >> > Makes all sense. Update: > > Delta: http://cr.openjdk.java.net/~stuefe/webrevs/8205531-vm.classloader-tree-folding/webrev.00-to-01/webrev/ > Full: http://cr.openjdk.java.net/~stuefe/webrevs/8205531-vm.classloader-tree-folding/webrev.01/webrev/ > > I only changed the comments as suggested. > > Tests (jdk-submit) came back fine. > >> On 6/22/18 12:10, Thomas St?fe wrote: >> >> Resent with the correct subject, sorry. >> >> >> Also, it is better to list a bug number in the email subject. :) >> > :) I really must have been tired yesterday. But I will not change the > subject again, to not tear up the thread. > > Thanks, Thomas > >> Thanks, >> Serguei >> >> ..Thomas >> >> On Fri, Jun 22, 2018 at 9:08 PM, Thomas St?fe >> wrote: >> >> Hi all, >> >> may I have reviews for this small enhancement to the jcmd >> VM.classloader diagnostic command: >> >> https://bugs.openjdk.java.net/browse/JDK-8205531 >> http://cr.openjdk.java.net/~stuefe/webrevs/8205531-vm.classloader-tree-folding/webrev.00/webrev/ >> >> >> VM.classloaders prints a tree of class loaders. This tree can grow a >> lot and become unwieldy, especially with a lot of similar loaders. One >> prime example is the DelegatingClassLoader. It would be helpful were >> all these loaders: >> >> 13114: >> +-- >> | >> +-- "platform", jdk.internal.loader.ClassLoaders$PlatformClassLoader >> | >> +-- "app", jdk.internal.loader.ClassLoaders$AppClassLoader >> | >> +-- test3.internals.InMemoryClassLoader >> | >> +-- jdk.internal.reflect.DelegatingClassLoader >> | >> +-- jdk.internal.reflect.DelegatingClassLoader >> | >> +-- jdk.internal.reflect.DelegatingClassLoader >> | >> +-- jdk.internal.reflect.DelegatingClassLoader >> | >> +-- jdk.internal.reflect.DelegatingClassLoader >> | >> ...... repeat 1495 times >> >> folded into one: >> >> 13114: >> +-- >> | >> +-- "platform", jdk.internal.loader.ClassLoaders$PlatformClassLoader >> | >> +-- "app", jdk.internal.loader.ClassLoaders$AppClassLoader >> | >> +-- test3.internals.InMemoryClassLoader >> | >> +-- jdk.internal.reflect.DelegatingClassLoader >> (+ 1499 more) >> >> >> Original idea by Bernd Eckenfels, see >> http://mail.openjdk.java.net/pipermail/serviceability-dev/2018-May/023824.html >> >> Thank you, Thomas >> >> From serguei.spitsyn at oracle.com Sat Jun 23 07:59:14 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Sat, 23 Jun 2018 00:59:14 -0700 Subject: RFR(XS): 8205561: problem list vmTestbase/nsk/jdb/trace/trace001/trace001.java In-Reply-To: <12bed6fe-c1e8-1ec3-95a8-7f80aae887dc@oracle.com> References: <12bed6fe-c1e8-1ec3-95a8-7f80aae887dc@oracle.com> Message-ID: Hi Chris, It looks good. Thanks, Serguei On 6/22/18 22:42, Chris Plummer wrote: > Hello, > > Please review the following: > > https://bugs.openjdk.java.net/browse/JDK-8205561 > > diff --git a/test/hotspot/jtreg/ProblemList.txt > b/test/hotspot/jtreg/ProblemList.txt > --- a/test/hotspot/jtreg/ProblemList.txt > +++ b/test/hotspot/jtreg/ProblemList.txt > @@ -166,7 +166,7 @@ > ?vmTestbase/vm/mlvm/indy/func/jvmti/mergeCP_indy2manySame_b/TestDescription.java > 8013267 generic-all > > ?vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java 8197938 > windows-all > - > +vmTestbase/nsk/jdb/trace/trace001/trace001.java 8205540 generic-all > ?vmTestbase/nsk/jdwp/ThreadReference/ForceEarlyReturn/forceEarlyReturn001/forceEarlyReturn001.java > 7199837 generic-all > > ?############################################################################# > > > thanks, > > Chris > > From serguei.spitsyn at oracle.com Sat Jun 23 08:25:18 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Sat, 23 Jun 2018 01:25:18 -0700 Subject: RFR 8205113: Update JVMTI doc references to object allocation tracking In-Reply-To: <96713bc3-9350-fd4a-75e5-2c36257066b2@oracle.com> References: <384783b9-9ec2-093a-2662-a7a817849720@oracle.com> <3d01255e-8712-46aa-7659-4ccd7325f81c@oracle.com> <6927237d-ce0b-17c0-68f8-06d05b84e516@oracle.com> <53f576e2-0adb-a6c5-8a3c-026dcb1aa7ad@oracle.com> <8224afa2-a6cb-fee2-8fa3-184828ac7c50@oracle.com> <60899f09-12df-1e6e-7e16-406a60939b7c@oracle.com> <96713bc3-9350-fd4a-75e5-2c36257066b2@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Sat Jun 23 11:49:14 2018 From: david.holmes at oracle.com (David Holmes) Date: Sat, 23 Jun 2018 21:49:14 +1000 Subject: RFR 8205113: Update JVMTI doc references to object allocation tracking In-Reply-To: References: <3d01255e-8712-46aa-7659-4ccd7325f81c@oracle.com> <6927237d-ce0b-17c0-68f8-06d05b84e516@oracle.com> <53f576e2-0adb-a6c5-8a3c-026dcb1aa7ad@oracle.com> <8224afa2-a6cb-fee2-8fa3-184828ac7c50@oracle.com> <60899f09-12df-1e6e-7e16-406a60939b7c@oracle.com> <96713bc3-9350-fd4a-75e5-2c36257066b2@oracle.com> Message-ID: <3018f21d-52b0-9ec4-151e-e4fb01da10ff@oracle.com> On 23/06/2018 6:25 PM, serguei.spitsyn at oracle.com wrote: > I've pushed the version suggested by David. But you left out all of Jeremy's other fixups! David > Thanks, > serguei > > > On 6/22/18 09:00, serguei.spitsyn at oracle.com wrote: >> Hi Jeremy, >> >> Okay, let me look at it once more before making final decision. >> We have all suggestions and preferences listed. >> >> Thanks, >> Serguei >> >> >> On 6/22/18 08:22, Jeremy Manson wrote: >>> Hey folks - >>> >>> With the window closing soon for 11, I feel that we should get this >>> revision in (just so that the spec is clear).? That said, I don't >>> feel strongly about the wording.? Who gets to make the decision? >>> >>> Jeremy >>> >>> On Wed, Jun 20, 2018 at 9:41 AM Jeremy Manson >>> > wrote: >>> >>> FWIW, I'm also okay with David's version, with the caveat that >>> "These methods should send this event" should probably read >>> "Invocations of these methods should trigger this event". >>> >>> Jeremy >>> >>> On Wed, Jun 20, 2018 at 4:11 AM David Holmes >>> > wrote: >>> >>> On 20/06/2018 4:48 PM, serguei.spitsyn at oracle.com >>> wrote: >>> > On 6/19/18 23:29, Jeremy Manson wrote: >>> >> Maybe we should make that clarification. >>> >> >>> >> Also, the reason I danced around that in my revision is that >>> > >>> > Understand that. >>> > But it is not a good style to clarify about >>> SampledObjectAlloc in the >>> > spec of VMObjectAlloc. >>> > Would be nice to find a way to keep it simple. >>> > >>> > We could keep the original spec as it is but add that all >>> this does not >>> > apply to the SampledObjectAlloc events. >>> >>> I don't think that really presents a coherent spec. I still >>> like my >>> suggestion :) >>> >>> David >>> >>> > >>> > Something like below: >>> > >>> >? ? Note, the above does not apply to the >> > id="SampledObjectAlloc">SampledObjectAlloc >>> >? ? event as it is triggered on all Java object allocations, >>> including >>> > those caused by bytecode method execution, >>> >? ? JNI method execution, and directly by VM methods.* >>> > * >>> > >>> >> if you set the sampling rate to 0, you get unconditional >>> sampling. >>> > A correction. >>> > >>> > I wrote: >>> >? > It is still possible to sample every allocation with the >>> SamplingRate >>> > == 1. >>> > >>> > but had write: SamplingRate == 0 >>> > >>> > Thanks, >>> > Serguei >>> > >>> > >>> >> >>> >> Jeremy >>> >> >>> >> On Tue, Jun 19, 2018 at 10:41 PM >>> serguei.spitsyn at oracle.com >>> >> >> > >>> >>> >> >> >> wrote: >>> >> >>> >>? ? ?On 6/19/18 21:54, David Holmes wrote: >>> >>? ? ?> On 20/06/2018 2:41 PM, serguei.spitsyn at oracle.com >>> >>> >>? ? ?>> > wrote: >>> >>? ? ?>> On 6/19/18 21:11, Jeremy Manson wrote: >>> >>? ? ?>>> That would be okay with me, assuming that my other >>> corrections >>> >>? ? ?are >>> >>? ? ?>>> made. >>> >>? ? ?>> >>> >>? ? ?>> Another option would be to say "non-sampling" >>> instead of >>> >>? ? ?>> "unconditional": >>> >>? ? ?>> >>> >>? ? ?>> == Sent when a method causes the virtual machine to >>> allocate an >>> >>? ? ?>> == Object visible to Java programming language code >>> and the >>> >>? ? ?allocation >>> >>? ? ?>> += is not detectable by other *non-sampling* >>> instrumentation >>> >>? ? ?mechanisms. >>> >>? ? ?>> >>> >>? ? ?>> >>> >>? ? ?>>> I'd also like to fix the spelling of >>> instrumentation in the first >>> >>? ? ?>>> sentence. >>> >>? ? ?>> >>> >>? ? ?>> Yes, of course. >>> >>? ? ?>> >>> >>? ? ?>> Let's wait for David's opinion. >>> >>? ? ?> >>> >>? ? ?> I'm okay with Serguei's suggestion (combined with >>> Jeremy's other >>> >>? ? ?> changes). >>> >>? ? ?> >>> >>? ? ?> I'm not that familiar with conditional versus >>> unconditional >>> >>? ? ?> instrumentation. >>> >> >>> >>? ? ?The whole point of the SampledObjectAlloc events is to >>> post them >>> >>? ? ?conditionally >>> >>? ? ?depending on the SamplingRate so that just some amount of >>> >>? ? ?allocations is >>> >>? ? ?sampled. >>> >>? ? ?It is why the overhead can be minimal (less than 5 >>> percents). >>> >>? ? ?It is still possible to sample every allocation with the >>> >>? ? ?SamplingRate == 1. >>> >> >>> >>? ? ?The VMObjectAlloc events are posted unconditionally, >>> so every VM >>> >>? ? ?allocation is posted. >>> >> >>> >>? ? ?Thanks, >>> >>? ? ?Serguei >>> >> >>> >> >>> >>? ? ?> >>> >>? ? ?> Thanks, >>> >>? ? ?> David >>> >>? ? ?> >>> >>? ? ?>> >>> >>? ? ?>> Thanks, >>> >>? ? ?>> Serguei >>> >>? ? ?>> >>> >>? ? ?>> >>> >>? ? ?>>> >>> >>? ? ?>>> Jeremy >>> >>? ? ?>>> >>> >>? ? ?>>> On Mon, Jun 18, 2018 at 11:01 PM >>> serguei.spitsyn at oracle.com >>> >>? ? ?>> > >>> >>? ? ?>>> >> >>> >>? ? ?>> >> >>> >>> >>? ? ?>> > >>> >>? ? ?>>> >> >>> >>? ? ?>> >>> wrote: >>> >>? ? ?>>> >>> >>? ? ?>>> ??? Hi Jeremy and David, >>> >>? ? ?>>> >>> >>? ? ?>>> ??? Sorry for being late to the party. >>> >>? ? ?>>> >>> >>? ? ?>>> ??? I'm also concerned about the Jeremy's spec >>> update is more >>> >>? ? ?>>> ??? intrusive than necessary. >>> >>? ? ?>>> ??? One specifics of the new SampledObjectAlloc >>> event is that >>> >>? ? ?it is >>> >>? ? ?>>> ??? posted conditionally. >>> >>? ? ?>>> ??? So, it is not fully comparable with the >>> VMObjectAlloc >>> >>? ? ?event and >>> >>? ? ?>>> ??? can not replace it in any way. >>> >>? ? ?>>> ??? I'm even not yet convinced that any spec >>> update is necessary. >>> >>? ? ?>>> >>> >>? ? ?>>> ??? However, something like below would look >>> minimal and >>> >>? ? ?reasonable: >>> >>? ? ?>>> >>> >>? ? ?>>> ??? == Sent when a method causes the virtual >>> machine to >>> >>? ? ?allocate an >>> >>? ? ?>>> ??? == Object visible to Java programming language >>> code and the >>> >>? ? ?>>> allocation >>> >>? ? ?>>> ??? += is not detectable by other *unconditional* >>> intrumentation >>> >>? ? ?>>> ??? mechanisms. >>> >>? ? ?>>> >>> >>? ? ?>>> ??? == Generally object allocation should be >>> detected by >>> >>? ? ?instrumenting >>> >>? ? ?>>> ??? == the bytecodes of allocating methods. >>> >>? ? ?>>> >>> >>? ? ?>>> ??? == Object allocation generated in native code >>> by JNI function >>> >>? ? ?>>> ??? == calls should be detected using >>> >>? ? ?>>> ??? == JNI function >>> >>? ? ?>>> interception. >>> >>? ? ?>>> >>> >>? ? ?>>> ??? == Some methods might not have associated >>> bytecodes and >>> >>? ? ?are not >>> >>? ? ?>>> ??? == native methods, they instead are executed >>> directly by the >>> >>? ? ?>>> ??? == VM. These methods should send this event. >>> >>? ? ?>>> >>> >>? ? ?>>> ??? == Virtual machines which are incapable of >>> bytecode >>> >>? ? ?instrumentation >>> >>? ? ?>>> ??? == for some or all of their methods can send >>> this event. >>> >>? ? ?>>> >>> >>? ? ?>>> ??? *++ Note that the >> >>? ? ?>>> >>> id="SampledObjectAlloc">SampledObjectAlloc** >>> >>? ? ?>>> ??? **++ event is conditionally triggered on all >>> Java object >>> >>? ? ?>>> ??? allocations, including those** >>> >>? ? ?>>> ??? **++ caused by bytecode method execution, JNI >>> method >>> >>? ? ?execution, >>> >>? ? ?>>> ??? and directly by VM methods.** >>> >>? ? ?>>> ??? * >>> >>? ? ?>>> >>> >>? ? ?>>> ??? Maybe, just adding the last statement would be >>> good enough. >>> >>? ? ?>>> >>> >>? ? ?>>> ??? Thanks, >>> >>? ? ?>>> ??? Serguei >>> >>? ? ?>>> >>> >>? ? ?>>> >>> >>? ? ?>>> ??? On 6/18/18 21:36, David Holmes wrote: >>> >>? ? ?>>>> ??? On 19/06/2018 4:50 AM, Jeremy Manson wrote: >>> >>? ? ?>>>>> ??? Yup!? The paragraph meanders a bit. How >>> about something >>> >>? ? ?like: >>> >>? ? ?>>>> >>> >>? ? ?>>>> ??? I'm not sure some of the change quite works. >>> The original >>> >>? ? ?text >>> >>? ? ?>>>> ??? considers there to be three kinds of methods >>> that can cause >>> >>? ? ?>>>> ??? allocation when executed: >>> >>? ? ?>>>> ??? - Java (bytecode) methods >>> >>? ? ?>>>> ??? - JNI methods >>> >>? ? ?>>>> ??? - VM methods >>> >>? ? ?>>>> >>> >>? ? ?>>>> ??? but you've turned this into three kinds of >>> allocation: via >>> >>? ? ?>>>> ??? bytecode, via JNI, and via the VM. You then >>> refer to >>> >>? ? ?"triggering" >>> >>? ? ?>>>> ??? an allocation when we tend to use triggering >>> for events. >>> >>? ? ?You also >>> >>? ? ?>>>> ??? refer to an allocation being "executed >>> directly by the VM" (a >>> >>? ? ?>>>> ??? phrase previously applied when the subject was a >>> >>? ? ?'method') - but >>> >>? ? ?>>>> ??? you don't really execute allocations. >>> >>? ? ?>>>> >>> >>? ? ?>>>> ??? IIUC the problem with the existing text is >>> just that it >>> >>? ? ?considers >>> >>? ? ?>>>> ??? VM allocation events as being undetectable >>> other than by >>> >>? ? ?this "VM >>> >>? ? ?>>>> ??? object allocation event" - but that's no >>> longer true. So how >>> >>? ? ?>>>> ??? about something minimally changed like this: >>> >>? ? ?>>>> >>> >>? ? ?>>>> ??? --- >>> >>? ? ?>>>> ??? ? Sent when a method causes the virtual >>> machine to directly >>> >>? ? ?>>>> ??? allocate an >>> >>? ? ?>>>> ??? ? Object visible to Java programming language >>> code. >>> >>? ? ?>>>> ??? ? Generally object allocation can be detected by >>> >>? ? ?instrumenting >>> >>? ? ?>>>> ??? ? the bytecodes of allocating methods. >>> >>? ? ?>>>> ??? ? Object allocation generated in native code >>> by JNI function >>> >>? ? ?>>>> ??? ? calls can be detected using >>> >>? ? ?>>>> ??? ? JNI function >>> >>? ? ?>>>> interception. >>> >>? ? ?>>>> ??? ?? Some methods might not have associated >>> bytecodes and >>> >>? ? ?are not >>> >>? ? ?>>>> ??? ?? native methods, they instead are executed >>> directly by the >>> >>? ? ?>>>> ??? ?? VM. These methods should send this event. >>> >>? ? ?>>>> ??? ?? Virtual machines which are incapable of >>> bytecode >>> >>? ? ?>>>> instrumentation >>> >>? ? ?>>>> ??? ?? for some or all of their methods can send >>> this event. >>> >>? ? ?>>>> >>> >>? ? ?>>>> ??? ?? Note that the >> >>? ? ?>>>> >>> id="SampledObjectAlloc">SampledObjectAlloc >>> >>? ? ?>>>> ??? ?? event is triggered on all Java object >>> allocations, >>> >>? ? ?including >>> >>? ? ?>>>> ??? those >>> >>? ? ?>>>> ??? ?? caused by bytecode method execution, JNI >>> method >>> >>? ? ?execution, and >>> >>? ? ?>>>> ??? ?? directly by VM methods. >>> >>? ? ?>>>> ??? --- >>> >>? ? ?>>>> >>> >>? ? ?>>>> ??? Thanks, >>> >>? ? ?>>>> ??? David >>> >>? ? ?>>>> >>> >>? ? ?>>>>> ??? Sent when the virtual machine allocates an >>> >>? ? ?>>>>> ??? Object visible to Java programming language >>> code without >>> >>? ? ?using a >>> >>? ? ?>>>>> new bytecode variant or a JNI method. >>> >>? ? ?>>>>> ??? Many approaches to tracking object >>> allocation use a >>> >>? ? ?>>>>> combination of >>> >>? ? ?>>>>> ??? bytecode-based instrumentation and >> >>? ? ?>>>>> id="jniIntercept">JNI function >>> >>? ? ?>>>>> interception to intercept >>> allocations. >>> >>? ? ?However, >>> >>? ? ?>>>>> ??? this >>> >>? ? ?>>>>> ??? approach can leave a number of allocations >>> undetected. >>> >>? ? ?>>>>> ??? Allocations that are neither >>> >>? ? ?>>>>> ??? triggered by bytecode nor JNI are executed >>> directly by >>> >>? ? ?the VM. >>> >>? ? ?>>>>> ??? When those allocations occur, the VM should >>> send this event. >>> >>? ? ?>>>>> ??? Virtual machines that are incapable of bytecode >>> >>? ? ?instrumentation >>> >>? ? ?>>>>> ??? for some or all of their methods may also >>> send this event. >>> >>? ? ?>>>>> ???

>>> >>? ? ?>>>>> ??? Note that the >> >>? ? ?>>>>> >>> id="SampledObjectAlloc">SampledObjectAlloc >>> >>? ? ?>>>>> ??? event is triggered on all Java object >>> allocations, including >>> >>? ? ?>>>>> ??? those triggered by bytecode, >>> >>? ? ?>>>>> ??? JNI methods, and VM events. >>> >>? ? ?>>>>> >>> >>? ? ?>>>>> >>> >>? ? ?>>>>> >>> >>? ? ?>>>>> ??? On Mon, Jun 18, 2018 at 12:57 AM David Holmes >>> >>? ? ?>>>>> ??? >> >>> >>? ? ?>> > >>> >>> >>? ? ?>> >> >>> >>? ? ?>>>>> ??? >> >>> >>? ? ?>> >> >>> >>? ? ?>>>>> ??? >> >>> >>? ? ?>> >>> wrote: >>> >>? ? ?>>>>> >>> >>? ? ?>>>>> ??? ??? On 18/06/2018 5:01 PM, Jeremy Manson wrote: >>> >>? ? ?>>>>> ??? ???? > We haven't changed when a VM issues >>> "VM object >>> >>? ? ?>>>>> ??? allocation" events. >>> >>? ? ?>>>>> ??? ???? > >>> >>? ? ?>>>>> ??? ???? > There were references in the docs to a >>> >>? ? ?requirement to use >>> >>? ? ?>>>>> ??? bytecode >>> >>? ? ?>>>>> ??? ???? > rewriting and JNI interception to track >>> >>? ? ?allocations.? With >>> >>? ? ?>>>>> ??? ???? > SampledObjectAlloc, this is no longer >>> the case - >>> >>? ? ?>>>>> SampledObjectAlloc can >>> >>? ? ?>>>>> ??? ???? > track them.? This change is supposed >>> to remove the >>> >>? ? ?>>>>> ??? references to >>> >>? ? ?>>>>> ??? ??? those >>> >>? ? ?>>>>> ??? ???? > requirements, and provide suitable >>> replacement text. >>> >>? ? ?>>>>> ??? ???? > >>> >>? ? ?>>>>> ??? ???? > VM object alloc has specific language >>> about being >>> >>? ? ?able to >>> >>? ? ?>>>>> ??? use it to >>> >>? ? ?>>>>> ??? ???? > track allocations that cannot be >>> tracked with >>> >>? ? ?bytecode >>> >>? ? ?>>>>> instrumentation >>> >>? ? ?>>>>> ??? ???? > and JNI interception.? My goal in >>> rephrasing was >>> >>? ? ?to make >>> >>? ? ?>>>>> ??? it clear >>> >>? ? ?>>>>> ??? ??? that, >>> >>? ? ?>>>>> ??? ???? > while you can still use it for this, >>> you can also >>> >>? ? ?just use >>> >>? ? ?>>>>> ??? ???? > SampledObjectAlloc for everything. >>> >>? ? ?>>>>> >>> >>? ? ?>>>>> ??? ??? Okay. That doesn't come across clearly >>> to me - >>> >>? ? ?sorry. So you >>> >>? ? ?>>>>> ??? will now >>> >>? ? ?>>>>> ??? ??? get both kinds of events for allocations >>> done in the VM? >>> >>? ? ?>>>>> >>> >>? ? ?>>>>> ??? ??? Thanks, >>> >>? ? ?>>>>> ??? ??? David >>> >>? ? ?>>>>> >>> >>? ? ?>>>>> >>> >>? ? ?>>>>> ??? ???? > Jeremy >>> >>? ? ?>>>>> ??? ???? > >>> >>? ? ?>>>>> ??? ???? > On Sun, Jun 17, 2018 at 9:11 PM David >>> Holmes >>> >>? ? ?>>>>> ??? ??? >> >>> >>? ? ?>> > >>> >>> >>? ? ?>> >> >>> >>? ? ?>>>>> ??? >> >>> >>? ? ?>> >> >>> >>> >>? ? ?>> >> >>> >>? ? ?>>>>> ??? ???? > >> >>> >>? ? ?>> > >>> >>? ? ?>>>>> ??? >> >>> >>? ? ?>> >> >>> >>? ? ?>>>>> ??? >> >>> >>? ? ?>> >>>> wrote: >>> >>? ? ?>>>>> ??? ???? > >>> >>? ? ?>>>>> ??? ???? >? ? ?Hi Jeremy, >>> >>? ? ?>>>>> ??? ???? > >>> >>? ? ?>>>>> ??? ???? >? ? ?On 16/06/2018 2:33 AM, Jeremy >>> Manson wrote: >>> >>? ? ?>>>>> ??? ???? > > Hi all, >>> >>? ? ?>>>>> ??? ???? > > >>> >>? ? ?>>>>> ??? ???? > > There are a number of references in the >>> >>? ? ?JVMTI doc >>> >>? ? ?>>>>> ??? to its >>> >>? ? ?>>>>> ??? ??? not doing >>> >>? ? ?>>>>> ??? ???? > > object allocation tracking.? Now >>> that JEP >>> >>? ? ?331 has >>> >>? ? ?>>>>> ??? landed, >>> >>? ? ?>>>>> ??? ??? these >>> >>? ? ?>>>>> ??? ???? > > references are obsolete.? This patch >>> >>? ? ?changes those >>> >>? ? ?>>>>> ??? references >>> >>? ? ?>>>>> ??? ???? > ?accordingly. >>> >>? ? ?>>>>> ??? ???? > > >>> >>? ? ?>>>>> ??? ???? > > While I was there, I took the >>> liberty of >>> >>? ? ?fixing >>> >>? ? ?>>>>> some >>> >>? ? ?>>>>> spelling?errors. >>> >>? ? ?>>>>> ??? ???? > > >>> >>? ? ?>>>>> ??? ???? > > As far as I know, these are >>> non-normative >>> >>? ? ?>>>>> changes and >>> >>? ? ?>>>>> ??? ??? modify no >>> >>? ? ?>>>>> ??? ???? >? ? ?API, so >>> >>? ? ?>>>>> ??? ???? > > they should not require a CSR. >>> >>? ? ?>>>>> ??? ???? > >>> >>? ? ?>>>>> ??? ???? >? ? ?I'm unclear on the nature of the >>> change to >>> >>? ? ?"VM Object >>> >>? ? ?>>>>> ??? ??? Allocation". Does >>> >>? ? ?>>>>> ??? ???? >? ? ?the existence of >>> SampledObjectAlloc change >>> >>? ? ?when a VM >>> >>? ? ?>>>>> ??? should >>> >>? ? ?>>>>> ??? ??? issue "VM >>> >>? ? ?>>>>> ??? ???? > ?object allocation" events? >>> >>? ? ?>>>>> ??? ???? > >>> >>? ? ?>>>>> ??? ???? > ?Thanks, >>> >>? ? ?>>>>> ??? ???? > ?David >>> >>? ? ?>>>>> ??? ???? > >>> >>? ? ?>>>>> ??? ???? > > >>> >>? ? ?>>>>> ??? ???? > > Bug: >>> >>? ? ?>>>>> ??? ???? > > >>> >> https://bugs.openjdk.java.net/browse/JDK-8205113 >>> >>? ? ?>>>>> ??? ???? > > >>> >>? ? ?>>>>> ??? ???? > > Webrev: >>> >>? ? ?>>>>> ??? ???? > > >>> >>? ? ?>>>>> >>> http://cr.openjdk.java.net/~jmanson/8205113/webrev.00/ >>> >>> >>? ? ? >>> >>? ? ?>>>>> >>> >>> >>? ? ?>>>>> ??? ???? > > >>> >>? ? ?>>>>> ??? ???? > > Thanks! >>> >>? ? ?>>>>> ??? ???? > > >>> >>? ? ?>>>>> ??? ???? > > Jeremy >>> >>? ? ?>>>>> ??? ???? > >>> >>? ? ?>>>>> >>> >>? ? ?>>> >>> >>? ? ?>> >>> >> >>> > >>> >> > From serguei.spitsyn at oracle.com Sat Jun 23 21:58:37 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Sat, 23 Jun 2018 14:58:37 -0700 Subject: RFR 8205113: Update JVMTI doc references to object allocation tracking In-Reply-To: <3018f21d-52b0-9ec4-151e-e4fb01da10ff@oracle.com> References: <3d01255e-8712-46aa-7659-4ccd7325f81c@oracle.com> <6927237d-ce0b-17c0-68f8-06d05b84e516@oracle.com> <53f576e2-0adb-a6c5-8a3c-026dcb1aa7ad@oracle.com> <8224afa2-a6cb-fee2-8fa3-184828ac7c50@oracle.com> <60899f09-12df-1e6e-7e16-406a60939b7c@oracle.com> <96713bc3-9350-fd4a-75e5-2c36257066b2@oracle.com> <3018f21d-52b0-9ec4-151e-e4fb01da10ff@oracle.com> Message-ID: <3ffcc780-2568-ec21-b503-228c91d18969@oracle.com> Hi David, This was your suggestion: --- ? Sent when a method causes the virtual machine to directly allocate an ? Object visible to Java programming language code. ? Generally object allocation can be detected by instrumenting ? the bytecodes of allocating methods. ? Object allocation generated in native code by JNI function ? calls can be detected using ? JNI function interception. ?? Some methods might not have associated bytecodes and are not ?? native methods, they instead are executed directly by the ?? VM. These methods should send this event. ?? Virtual machines which are incapable of bytecode instrumentation ?? for some or all of their methods can send this event. ?? Note that the SampledObjectAlloc ?? event is triggered on all Java object allocations, including those ?? caused by bytecode method execution, JNI method execution, and ?? directly by VM methods. --- I've pushed the patch: diff -r f703d45c5687 src/hotspot/share/prims/jvmti.xml --- a/src/hotspot/share/prims/jvmti.xml??? Tue Jun 05 11:55:39 2018 +0200 +++ b/src/hotspot/share/prims/jvmti.xml??? Sat Jun 23 01:18:57 2018 -0700 @@ -13507,9 +13507,8 @@ ?? ???? -????? Sent when a method causes the virtual machine to allocate an -????? Object visible to Java programming language code and the -????? allocation is not detectable by other intrumentation mechanisms. +????? Sent when a method causes the virtual machine to directly allocate an +????? Object visible to Java programming language code. ?????? Generally object allocation should be detected by instrumenting ?????? the bytecodes of allocating methods. ?????? Object allocation generated in native code by JNI function @@ -13520,6 +13519,12 @@ ?????? VM. These methods should send this event. ?????? Virtual machines which are incapable of bytecode instrumentation ?????? for some or all of their methods can send this event. + +????? Note that the SampledObjectAlloc +????? event is triggered on all Java object allocations, including those +????? caused by bytecode method execution, JNI method execution, and +????? directly by VM methods. ??????

?????? Typical examples where this event might be sent: ??????

    Thanks, Serguei On 6/23/18 04:49, David Holmes wrote: > On 23/06/2018 6:25 PM, serguei.spitsyn at oracle.com wrote: >> I've pushed the version suggested by David. > > But you left out all of Jeremy's other fixups! > > David > >> Thanks, >> serguei >> >> >> On 6/22/18 09:00, serguei.spitsyn at oracle.com wrote: >>> Hi Jeremy, >>> >>> Okay, let me look at it once more before making final decision. >>> We have all suggestions and preferences listed. >>> >>> Thanks, >>> Serguei >>> >>> >>> On 6/22/18 08:22, Jeremy Manson wrote: >>>> Hey folks - >>>> >>>> With the window closing soon for 11, I feel that we should get this >>>> revision in (just so that the spec is clear).? That said, I don't >>>> feel strongly about the wording.? Who gets to make the decision? >>>> >>>> Jeremy >>>> >>>> On Wed, Jun 20, 2018 at 9:41 AM Jeremy Manson >>>> > wrote: >>>> >>>> ??? FWIW, I'm also okay with David's version, with the caveat that >>>> ??? "These methods should send this event" should probably read >>>> ??? "Invocations of these methods should trigger this event". >>>> >>>> ??? Jeremy >>>> >>>> ??? On Wed, Jun 20, 2018 at 4:11 AM David Holmes >>>> ??? > wrote: >>>> >>>> ??????? On 20/06/2018 4:48 PM, serguei.spitsyn at oracle.com >>>> ??????? wrote: >>>> ??????? > On 6/19/18 23:29, Jeremy Manson wrote: >>>> ??????? >> Maybe we should make that clarification. >>>> ??????? >> >>>> ??????? >> Also, the reason I danced around that in my revision is >>>> that >>>> ??????? > >>>> ??????? > Understand that. >>>> ??????? > But it is not a good style to clarify about >>>> ??????? SampledObjectAlloc in the >>>> ??????? > spec of VMObjectAlloc. >>>> ??????? > Would be nice to find a way to keep it simple. >>>> ??????? > >>>> ??????? > We could keep the original spec as it is but add that all >>>> ??????? this does not >>>> ??????? > apply to the SampledObjectAlloc events. >>>> >>>> ??????? I don't think that really presents a coherent spec. I still >>>> ??????? like my >>>> ??????? suggestion :) >>>> >>>> ??????? David >>>> >>>> ??????? > >>>> ??????? > Something like below: >>>> ??????? > >>>> ??????? >? ? Note, the above does not apply to the >>> ??????? > id="SampledObjectAlloc">SampledObjectAlloc >>>> ??????? >? ? event as it is triggered on all Java object allocations, >>>> ??????? including >>>> ??????? > those caused by bytecode method execution, >>>> ??????? >? ? JNI method execution, and directly by VM methods.* >>>> ??????? > * >>>> ??????? > >>>> ??????? >> if you set the sampling rate to 0, you get unconditional >>>> ??????? sampling. >>>> ??????? > A correction. >>>> ??????? > >>>> ??????? > I wrote: >>>> ??????? >? > It is still possible to sample every allocation with the >>>> ??????? SamplingRate >>>> ??????? > == 1. >>>> ??????? > >>>> ??????? > but had write: SamplingRate == 0 >>>> ??????? > >>>> ??????? > Thanks, >>>> ??????? > Serguei >>>> ??????? > >>>> ??????? > >>>> ??????? >> >>>> ??????? >> Jeremy >>>> ??????? >> >>>> ??????? >> On Tue, Jun 19, 2018 at 10:41 PM >>>> ??????? serguei.spitsyn at oracle.com >>>> ??????? >> >>> ??????? > >>>> ??????? >>> >>>> ??????? >> >>> ??????? >> wrote: >>>> ??????? >> >>>> ??????? >>? ? ?On 6/19/18 21:54, David Holmes wrote: >>>> ??????? >>? ? ?> On 20/06/2018 2:41 PM, serguei.spitsyn at oracle.com >>>> ??????? >>>> ??????? >>? ? ?>>> ??????? > wrote: >>>> ??????? >>? ? ?>> On 6/19/18 21:11, Jeremy Manson wrote: >>>> ??????? >>? ? ?>>> That would be okay with me, assuming that my other >>>> ??????? corrections >>>> ??????? >>? ? ?are >>>> ??????? >>? ? ?>>> made. >>>> ??????? >>? ? ?>> >>>> ??????? >>? ? ?>> Another option would be to say "non-sampling" >>>> ??????? instead of >>>> ??????? >>? ? ?>> "unconditional": >>>> ??????? >>? ? ?>> >>>> ??????? >>? ? ?>> == Sent when a method causes the virtual machine to >>>> ??????? allocate an >>>> ??????? >>? ? ?>> == Object visible to Java programming language code >>>> ??????? and the >>>> ??????? >>? ? ?allocation >>>> ??????? >>? ? ?>> += is not detectable by other *non-sampling* >>>> ??????? instrumentation >>>> ??????? >>? ? ?mechanisms. >>>> ??????? >>? ? ?>> >>>> ??????? >>? ? ?>> >>>> ??????? >>? ? ?>>> I'd also like to fix the spelling of >>>> ??????? instrumentation in the first >>>> ??????? >>? ? ?>>> sentence. >>>> ??????? >>? ? ?>> >>>> ??????? >>? ? ?>> Yes, of course. >>>> ??????? >>? ? ?>> >>>> ??????? >>? ? ?>> Let's wait for David's opinion. >>>> ??????? >>? ? ?> >>>> ??????? >>? ? ?> I'm okay with Serguei's suggestion (combined with >>>> ??????? Jeremy's other >>>> ??????? >>? ? ?> changes). >>>> ??????? >>? ? ?> >>>> ??????? >>? ? ?> I'm not that familiar with conditional versus >>>> ??????? unconditional >>>> ??????? >>? ? ?> instrumentation. >>>> ??????? >> >>>> ??????? >>? ? ?The whole point of the SampledObjectAlloc events is to >>>> ??????? post them >>>> ??????? >>? ? ?conditionally >>>> ??????? >>? ? ?depending on the SamplingRate so that just some >>>> amount of >>>> ??????? >>? ? ?allocations is >>>> ??????? >>? ? ?sampled. >>>> ??????? >>? ? ?It is why the overhead can be minimal (less than 5 >>>> ??????? percents). >>>> ??????? >>? ? ?It is still possible to sample every allocation with >>>> the >>>> ??????? >>? ? ?SamplingRate == 1. >>>> ??????? >> >>>> ??????? >>? ? ?The VMObjectAlloc events are posted unconditionally, >>>> ??????? so every VM >>>> ??????? >>? ? ?allocation is posted. >>>> ??????? >> >>>> ??????? >>? ? ?Thanks, >>>> ??????? >>? ? ?Serguei >>>> ??????? >> >>>> ??????? >> >>>> ??????? >>? ? ?> >>>> ??????? >>? ? ?> Thanks, >>>> ??????? >>? ? ?> David >>>> ??????? >>? ? ?> >>>> ??????? >>? ? ?>> >>>> ??????? >>? ? ?>> Thanks, >>>> ??????? >>? ? ?>> Serguei >>>> ??????? >>? ? ?>> >>>> ??????? >>? ? ?>> >>>> ??????? >>? ? ?>>> >>>> ??????? >>? ? ?>>> Jeremy >>>> ??????? >>? ? ?>>> >>>> ??????? >>? ? ?>>> On Mon, Jun 18, 2018 at 11:01 PM >>>> ??????? serguei.spitsyn at oracle.com >>>> ??????? >>? ? ?>>> ??????? > >>>> ??????? >>? ? ?>>> >>> ??????? >>>> ??????? >>? ? ?>>> ??????? >> >>>> ??????? >>> >>>> ??????? >>? ? ?>>> ??????? > >>>> ??????? >>? ? ?>>> >>> ??????? >>>> ??????? >>? ? ?>>> >>> wrote: >>>> ??????? >>? ? ?>>> >>>> ??????? >>? ? ?>>> ??? Hi Jeremy and David, >>>> ??????? >>? ? ?>>> >>>> ??????? >>? ? ?>>> ??? Sorry for being late to the party. >>>> ??????? >>? ? ?>>> >>>> ??????? >>? ? ?>>> ??? I'm also concerned about the Jeremy's spec >>>> ??????? update is more >>>> ??????? >>? ? ?>>> ??? intrusive than necessary. >>>> ??????? >>? ? ?>>> ??? One specifics of the new SampledObjectAlloc >>>> ??????? event is that >>>> ??????? >>? ? ?it is >>>> ??????? >>? ? ?>>> ??? posted conditionally. >>>> ??????? >>? ? ?>>> ??? So, it is not fully comparable with the >>>> ??????? VMObjectAlloc >>>> ??????? >>? ? ?event and >>>> ??????? >>? ? ?>>> ??? can not replace it in any way. >>>> ??????? >>? ? ?>>> ??? I'm even not yet convinced that any spec >>>> ??????? update is necessary. >>>> ??????? >>? ? ?>>> >>>> ??????? >>? ? ?>>> ??? However, something like below would look >>>> ??????? minimal and >>>> ??????? >>? ? ?reasonable: >>>> ??????? >>? ? ?>>> >>>> ??????? >>? ? ?>>> ??? == Sent when a method causes the virtual >>>> ??????? machine to >>>> ??????? >>? ? ?allocate an >>>> ??????? >>? ? ?>>> ??? == Object visible to Java programming language >>>> ??????? code and the >>>> ??????? >>? ? ?>>> allocation >>>> ??????? >>? ? ?>>> ??? += is not detectable by other *unconditional* >>>> ??????? intrumentation >>>> ??????? >>? ? ?>>> ??? mechanisms. >>>> ??????? >>? ? ?>>> >>>> ??????? >>? ? ?>>> ??? == Generally object allocation should be >>>> ??????? detected by >>>> ??????? >>? ? ?instrumenting >>>> ??????? >>? ? ?>>> ??? == the bytecodes of allocating methods. >>>> ??????? >>? ? ?>>> >>>> ??????? >>? ? ?>>> ??? == Object allocation generated in native code >>>> ??????? by JNI function >>>> ??????? >>? ? ?>>> ??? == calls should be detected using >>>> ??????? >>? ? ?>>> ??? == JNI function >>>> ??????? >>? ? ?>>> interception. >>>> ??????? >>? ? ?>>> >>>> ??????? >>? ? ?>>> ??? == Some methods might not have associated >>>> ??????? bytecodes and >>>> ??????? >>? ? ?are not >>>> ??????? >>? ? ?>>> ??? == native methods, they instead are executed >>>> ??????? directly by the >>>> ??????? >>? ? ?>>> ??? == VM. These methods should send this event. >>>> ??????? >>? ? ?>>> >>>> ??????? >>? ? ?>>> ??? == Virtual machines which are incapable of >>>> ??????? bytecode >>>> ??????? >>? ? ?instrumentation >>>> ??????? >>? ? ?>>> ??? == for some or all of their methods can send >>>> ??????? this event. >>>> ??????? >>? ? ?>>> >>>> ??????? >>? ? ?>>> ??? *++ Note that the >>> ??????? >>? ? ?>>> >>>> id="SampledObjectAlloc">SampledObjectAlloc** >>>> ??????? >>? ? ?>>> ??? **++ event is conditionally triggered on all >>>> ??????? Java object >>>> ??????? >>? ? ?>>> ??? allocations, including those** >>>> ??????? >>? ? ?>>> ??? **++ caused by bytecode method execution, JNI >>>> ??????? method >>>> ??????? >>? ? ?execution, >>>> ??????? >>? ? ?>>> ??? and directly by VM methods.** >>>> ??????? >>? ? ?>>> ??? * >>>> ??????? >>? ? ?>>> >>>> ??????? >>? ? ?>>> ??? Maybe, just adding the last statement would be >>>> ??????? good enough. >>>> ??????? >>? ? ?>>> >>>> ??????? >>? ? ?>>> ??? Thanks, >>>> ??????? >>? ? ?>>> ??? Serguei >>>> ??????? >>? ? ?>>> >>>> ??????? >>? ? ?>>> >>>> ??????? >>? ? ?>>> ??? On 6/18/18 21:36, David Holmes wrote: >>>> ??????? >>? ? ?>>>> ??? On 19/06/2018 4:50 AM, Jeremy Manson wrote: >>>> ??????? >>? ? ?>>>>> ??? Yup!? The paragraph meanders a bit. How >>>> ??????? about something >>>> ??????? >>? ? ?like: >>>> ??????? >>? ? ?>>>> >>>> ??????? >>? ? ?>>>> ??? I'm not sure some of the change quite works. >>>> ??????? The original >>>> ??????? >>? ? ?text >>>> ??????? >>? ? ?>>>> ??? considers there to be three kinds of methods >>>> ??????? that can cause >>>> ??????? >>? ? ?>>>> ??? allocation when executed: >>>> ??????? >>? ? ?>>>> ??? - Java (bytecode) methods >>>> ??????? >>? ? ?>>>> ??? - JNI methods >>>> ??????? >>? ? ?>>>> ??? - VM methods >>>> ??????? >>? ? ?>>>> >>>> ??????? >>? ? ?>>>> ??? but you've turned this into three kinds of >>>> ??????? allocation: via >>>> ??????? >>? ? ?>>>> ??? bytecode, via JNI, and via the VM. You then >>>> ??????? refer to >>>> ??????? >>? ? ?"triggering" >>>> ??????? >>? ? ?>>>> ??? an allocation when we tend to use triggering >>>> ??????? for events. >>>> ??????? >>? ? ?You also >>>> ??????? >>? ? ?>>>> ??? refer to an allocation being "executed >>>> ??????? directly by the VM" (a >>>> ??????? >>? ? ?>>>> ??? phrase previously applied when the subject >>>> was a >>>> ??????? >>? ? ?'method') - but >>>> ??????? >>? ? ?>>>> ??? you don't really execute allocations. >>>> ??????? >>? ? ?>>>> >>>> ??????? >>? ? ?>>>> ??? IIUC the problem with the existing text is >>>> ??????? just that it >>>> ??????? >>? ? ?considers >>>> ??????? >>? ? ?>>>> ??? VM allocation events as being undetectable >>>> ??????? other than by >>>> ??????? >>? ? ?this "VM >>>> ??????? >>? ? ?>>>> ??? object allocation event" - but that's no >>>> ??????? longer true. So how >>>> ??????? >>? ? ?>>>> ??? about something minimally changed like this: >>>> ??????? >>? ? ?>>>> >>>> ??????? >>? ? ?>>>> ??? --- >>>> ??????? >>? ? ?>>>> ??? ? Sent when a method causes the virtual >>>> ??????? machine to directly >>>> ??????? >>? ? ?>>>> ??? allocate an >>>> ??????? >>? ? ?>>>> ??? ? Object visible to Java programming language >>>> ??????? code. >>>> ??????? >>? ? ?>>>> ??? ? Generally object allocation can be >>>> detected by >>>> ??????? >>? ? ?instrumenting >>>> ??????? >>? ? ?>>>> ??? ? the bytecodes of allocating methods. >>>> ??????? >>? ? ?>>>> ??? ? Object allocation generated in native code >>>> ??????? by JNI function >>>> ??????? >>? ? ?>>>> ??? ? calls can be detected using >>>> ??????? >>? ? ?>>>> ??? ? JNI function >>>> ??????? >>? ? ?>>>> interception. >>>> ??????? >>? ? ?>>>> ??? ?? Some methods might not have associated >>>> ??????? bytecodes and >>>> ??????? >>? ? ?are not >>>> ??????? >>? ? ?>>>> ??? ?? native methods, they instead are executed >>>> ??????? directly by the >>>> ??????? >>? ? ?>>>> ??? ?? VM. These methods should send this event. >>>> ??????? >>? ? ?>>>> ??? ?? Virtual machines which are incapable of >>>> ??????? bytecode >>>> ??????? >>? ? ?>>>> instrumentation >>>> ??????? >>? ? ?>>>> ??? ?? for some or all of their methods can send >>>> ??????? this event. >>>> ??????? >>? ? ?>>>> >>>> ??????? >>? ? ?>>>> ??? ?? Note that the >>> ??????? >>? ? ?>>>> >>>> id="SampledObjectAlloc">SampledObjectAlloc >>>> ??????? >>? ? ?>>>> ??? ?? event is triggered on all Java object >>>> ??????? allocations, >>>> ??????? >>? ? ?including >>>> ??????? >>? ? ?>>>> ??? those >>>> ??????? >>? ? ?>>>> ??? ?? caused by bytecode method execution, JNI >>>> ??????? method >>>> ??????? >>? ? ?execution, and >>>> ??????? >>? ? ?>>>> ??? ?? directly by VM methods. >>>> ??????? >>? ? ?>>>> ??? --- >>>> ??????? >>? ? ?>>>> >>>> ??????? >>? ? ?>>>> ??? Thanks, >>>> ??????? >>? ? ?>>>> ??? David >>>> ??????? >>? ? ?>>>> >>>> ??????? >>? ? ?>>>>> ??? Sent when the virtual machine allocates an >>>> ??????? >>? ? ?>>>>> ??? Object visible to Java programming language >>>> ??????? code without >>>> ??????? >>? ? ?using a >>>> ??????? >>? ? ?>>>>> new bytecode variant or a JNI >>>> method. >>>> ??????? >>? ? ?>>>>> ??? Many approaches to tracking object >>>> ??????? allocation use a >>>> ??????? >>? ? ?>>>>> combination of >>>> ??????? >>? ? ?>>>>> ??? bytecode-based instrumentation and >>>> >>> ??????? >>? ? ?>>>>> id="jniIntercept">JNI function >>>> ??????? >>? ? ?>>>>> interception to intercept >>>> ??????? allocations. >>>> ??????? >>? ? ?However, >>>> ??????? >>? ? ?>>>>> ??? this >>>> ??????? >>? ? ?>>>>> ??? approach can leave a number of allocations >>>> ??????? undetected. >>>> ??????? >>? ? ?>>>>> ??? Allocations that are neither >>>> ??????? >>? ? ?>>>>> ??? triggered by bytecode nor JNI are executed >>>> ??????? directly by >>>> ??????? >>? ? ?the VM. >>>> ??????? >>? ? ?>>>>> ??? When those allocations occur, the VM should >>>> ??????? send this event. >>>> ??????? >>? ? ?>>>>> ??? Virtual machines that are incapable of >>>> bytecode >>>> ??????? >>? ? ?instrumentation >>>> ??????? >>? ? ?>>>>> ??? for some or all of their methods may also >>>> ??????? send this event. >>>> ??????? >>? ? ?>>>>> ???

    >>>> ??????? >>? ? ?>>>>> ??? Note that the >>> ??????? >>? ? ?>>>>> >>>> id="SampledObjectAlloc">SampledObjectAlloc >>>> ??????? >>? ? ?>>>>> ??? event is triggered on all Java object >>>> ??????? allocations, including >>>> ??????? >>? ? ?>>>>> ??? those triggered by bytecode, >>>> ??????? >>? ? ?>>>>> ??? JNI methods, and VM events. >>>> ??????? >>? ? ?>>>>> >>>> ??????? >>? ? ?>>>>> >>>> ??????? >>? ? ?>>>>> >>>> ??????? >>? ? ?>>>>> ??? On Mon, Jun 18, 2018 at 12:57 AM David Holmes >>>> ??????? >>? ? ?>>>>> >>> ??????? >>>> ??????? >>? ? ?>>> ??????? > >>>> ??????? >>> >>>> ??????? >>? ? ?>>> ??????? >> >>>> ??????? >>? ? ?>>>>> >>> ??????? >>>> ??????? >>? ? ?>>> ??????? >> >>>> ??????? >>? ? ?>>>>> >>> ??????? >>>> ??????? >>? ? ?>>> ??????? >>> wrote: >>>> ??????? >>? ? ?>>>>> >>>> ??????? >>? ? ?>>>>> ??? ??? On 18/06/2018 5:01 PM, Jeremy Manson >>>> wrote: >>>> ??????? >>? ? ?>>>>> ??? ???? > We haven't changed when a VM issues >>>> ??????? "VM object >>>> ??????? >>? ? ?>>>>> ??? allocation" events. >>>> ??????? >>? ? ?>>>>> ??? ???? > >>>> ??????? >>? ? ?>>>>> ??? ???? > There were references in the docs to a >>>> ??????? >>? ? ?requirement to use >>>> ??????? >>? ? ?>>>>> ??? bytecode >>>> ??????? >>? ? ?>>>>> ??? ???? > rewriting and JNI interception to >>>> track >>>> ??????? >>? ? ?allocations.? With >>>> ??????? >>? ? ?>>>>> ??? ???? > SampledObjectAlloc, this is no longer >>>> ??????? the case - >>>> ??????? >>? ? ?>>>>> SampledObjectAlloc can >>>> ??????? >>? ? ?>>>>> ??? ???? > track them.? This change is supposed >>>> ??????? to remove the >>>> ??????? >>? ? ?>>>>> ??? references to >>>> ??????? >>? ? ?>>>>> ??? ??? those >>>> ??????? >>? ? ?>>>>> ??? ???? > requirements, and provide suitable >>>> ??????? replacement text. >>>> ??????? >>? ? ?>>>>> ??? ???? > >>>> ??????? >>? ? ?>>>>> ??? ???? > VM object alloc has specific language >>>> ??????? about being >>>> ??????? >>? ? ?able to >>>> ??????? >>? ? ?>>>>> ??? use it to >>>> ??????? >>? ? ?>>>>> ??? ???? > track allocations that cannot be >>>> ??????? tracked with >>>> ??????? >>? ? ?bytecode >>>> ??????? >>? ? ?>>>>> instrumentation >>>> ??????? >>? ? ?>>>>> ??? ???? > and JNI interception.? My goal in >>>> ??????? rephrasing was >>>> ??????? >>? ? ?to make >>>> ??????? >>? ? ?>>>>> ??? it clear >>>> ??????? >>? ? ?>>>>> ??? ??? that, >>>> ??????? >>? ? ?>>>>> ??? ???? > while you can still use it for this, >>>> ??????? you can also >>>> ??????? >>? ? ?just use >>>> ??????? >>? ? ?>>>>> ??? ???? > SampledObjectAlloc for everything. >>>> ??????? >>? ? ?>>>>> >>>> ??????? >>? ? ?>>>>> ??? ??? Okay. That doesn't come across clearly >>>> ??????? to me - >>>> ??????? >>? ? ?sorry. So you >>>> ??????? >>? ? ?>>>>> ??? will now >>>> ??????? >>? ? ?>>>>> ??? ??? get both kinds of events for allocations >>>> ??????? done in the VM? >>>> ??????? >>? ? ?>>>>> >>>> ??????? >>? ? ?>>>>> ??? ??? Thanks, >>>> ??????? >>? ? ?>>>>> ??? ??? David >>>> ??????? >>? ? ?>>>>> >>>> ??????? >>? ? ?>>>>> >>>> ??????? >>? ? ?>>>>> ??? ???? > Jeremy >>>> ??????? >>? ? ?>>>>> ??? ???? > >>>> ??????? >>? ? ?>>>>> ??? ???? > On Sun, Jun 17, 2018 at 9:11 PM David >>>> ??????? Holmes >>>> ??????? >>? ? ?>>>>> >>> ??????? >>>> ??????? >>? ? ?>>> ??????? > >>>> ??????? >>> >>>> ??????? >>? ? ?>>> ??????? >> >>>> ??????? >>? ? ?>>>>> >>> ??????? >>>> ??????? >>? ? ?>>> ??????? >> >>>> ??????? >>> >>>> ??????? >>? ? ?>>> ??????? >> >>>> ??????? >>? ? ?>>>>> ??? ???? > >>> ??????? >>>> ??????? >>? ? ?>>> ??????? > >>>> ??????? >>? ? ?>>>>> >>> ??????? >>>> ??????? >>? ? ?>>> ??????? >> >>>> ??????? >>? ? ?>>>>> >>> ??????? >>>> ??????? >>? ? ?>>> >>>> wrote: >>>> ??????? >>? ? ?>>>>> ??? ???? > >>>> ??????? >>? ? ?>>>>> ??? ???? > ?Hi Jeremy, >>>> ??????? >>? ? ?>>>>> ??? ???? > >>>> ??????? >>? ? ?>>>>> ??? ???? > ?On 16/06/2018 2:33 AM, Jeremy >>>> ??????? Manson wrote: >>>> ??????? >>? ? ?>>>>> ??? ???? > > Hi all, >>>> ??????? >>? ? ?>>>>> ??? ???? > > >>>> ??????? >>? ? ?>>>>> ??? ???? > > There are a number of references >>>> in the >>>> ??????? >>? ? ?JVMTI doc >>>> ??????? >>? ? ?>>>>> ??? to its >>>> ??????? >>? ? ?>>>>> ??? ??? not doing >>>> ??????? >>? ? ?>>>>> ??? ???? > > object allocation tracking.? Now >>>> ??????? that JEP >>>> ??????? >>? ? ?331 has >>>> ??????? >>? ? ?>>>>> ??? landed, >>>> ??????? >>? ? ?>>>>> ??? ??? these >>>> ??????? >>? ? ?>>>>> ??? ???? > > references are obsolete.? This patch >>>> ??????? >>? ? ?changes those >>>> ??????? >>? ? ?>>>>> ??? references >>>> ??????? >>? ? ?>>>>> ??? ???? > ?accordingly. >>>> ??????? >>? ? ?>>>>> ??? ???? > > >>>> ??????? >>? ? ?>>>>> ??? ???? > > While I was there, I took the >>>> ??????? liberty of >>>> ??????? >>? ? ?fixing >>>> ??????? >>? ? ?>>>>> some >>>> ??????? >>? ? ?>>>>> spelling?errors. >>>> ??????? >>? ? ?>>>>> ??? ???? > > >>>> ??????? >>? ? ?>>>>> ??? ???? > > As far as I know, these are >>>> ??????? non-normative >>>> ??????? >>? ? ?>>>>> changes and >>>> ??????? >>? ? ?>>>>> ??? ??? modify no >>>> ??????? >>? ? ?>>>>> ??? ???? > ?API, so >>>> ??????? >>? ? ?>>>>> ??? ???? > > they should not require a CSR. >>>> ??????? >>? ? ?>>>>> ??? ???? > >>>> ??????? >>? ? ?>>>>> ??? ???? > ?I'm unclear on the nature of the >>>> ??????? change to >>>> ??????? >>? ? ?"VM Object >>>> ??????? >>? ? ?>>>>> Allocation". Does >>>> ??????? >>? ? ?>>>>> ??? ???? > ?the existence of >>>> ??????? SampledObjectAlloc change >>>> ??????? >>? ? ?when a VM >>>> ??????? >>? ? ?>>>>> ??? should >>>> ??????? >>? ? ?>>>>> ??? ??? issue "VM >>>> ??????? >>? ? ?>>>>> ??? ???? > ?object allocation" events? >>>> ??????? >>? ? ?>>>>> ??? ???? > >>>> ??????? >>? ? ?>>>>> ??? ???? > ?Thanks, >>>> ??????? >>? ? ?>>>>> ??? ???? > ?David >>>> ??????? >>? ? ?>>>>> ??? ???? > >>>> ??????? >>? ? ?>>>>> ??? ???? > > >>>> ??????? >>? ? ?>>>>> ??? ???? > > Bug: >>>> ??????? >>? ? ?>>>>> ??? ???? > > >>>> ??????? >> https://bugs.openjdk.java.net/browse/JDK-8205113 >>>> ??????? >>? ? ?>>>>> ??? ???? > > >>>> ??????? >>? ? ?>>>>> ??? ???? > > Webrev: >>>> ??????? >>? ? ?>>>>> ??? ???? > > >>>> ??????? >>? ? ?>>>>> >>>> http://cr.openjdk.java.net/~jmanson/8205113/webrev.00/ >>>> >>>> ??????? >> ? >>>> ??????? >>? ? ?>>>>> >>>> >>>> ??????? >>? ? ?>>>>> ??? ???? > > >>>> ??????? >>? ? ?>>>>> ??? ???? > > Thanks! >>>> ??????? >>? ? ?>>>>> ??? ???? > > >>>> ??????? >>? ? ?>>>>> ??? ???? > > Jeremy >>>> ??????? >>? ? ?>>>>> ??? ???? > >>>> ??????? >>? ? ?>>>>> >>>> ??????? >>? ? ?>>> >>>> ??????? >>? ? ?>> >>>> ??????? >> >>>> ??????? > >>>> >>> >> From david.holmes at oracle.com Sat Jun 23 22:53:54 2018 From: david.holmes at oracle.com (David Holmes) Date: Sun, 24 Jun 2018 08:53:54 +1000 Subject: RFR 8205113: Update JVMTI doc references to object allocation tracking In-Reply-To: <3ffcc780-2568-ec21-b503-228c91d18969@oracle.com> References: <3d01255e-8712-46aa-7659-4ccd7325f81c@oracle.com> <6927237d-ce0b-17c0-68f8-06d05b84e516@oracle.com> <53f576e2-0adb-a6c5-8a3c-026dcb1aa7ad@oracle.com> <8224afa2-a6cb-fee2-8fa3-184828ac7c50@oracle.com> <60899f09-12df-1e6e-7e16-406a60939b7c@oracle.com> <96713bc3-9350-fd4a-75e5-2c36257066b2@oracle.com> <3018f21d-52b0-9ec4-151e-e4fb01da10ff@oracle.com> <3ffcc780-2568-ec21-b503-228c91d18969@oracle.com> Message-ID: <845054d9-ec16-e2f8-afec-c885127bbd04@oracle.com> On 24/06/2018 7:58 AM, serguei.spitsyn at oracle.com wrote: > Hi David, > > This was your suggestion: There was only one part of the patch being debated! The rest of the patch was still applicable. Jeremy specifically stated "That would be okay with me, assuming that my other corrections are made. I'd also like to fix the spelling of instrumentation in the first sentence." David > --- > ? Sent when a method causes the virtual machine to directly allocate an > ? Object visible to Java programming language code. > ? Generally object allocation can be detected by instrumenting > ? the bytecodes of allocating methods. > ? Object allocation generated in native code by JNI function > ? calls can be detected using > ? JNI function > interception. > ?? Some methods might not have associated bytecodes and are not > ?? native methods, they instead are executed directly by the > ?? VM. These methods should send this event. > ?? Virtual machines which are incapable of bytecode instrumentation > ?? for some or all of their methods can send this event. > > ?? Note that the id="SampledObjectAlloc">SampledObjectAlloc > ?? event is triggered on all Java object allocations, including those > ?? caused by bytecode method execution, JNI method execution, and > ?? directly by VM methods. > --- > > I've pushed the patch: > > diff -r f703d45c5687 src/hotspot/share/prims/jvmti.xml > --- a/src/hotspot/share/prims/jvmti.xml??? Tue Jun 05 11:55:39 2018 +0200 > +++ b/src/hotspot/share/prims/jvmti.xml??? Sat Jun 23 01:18:57 2018 -0700 > @@ -13507,9 +13507,8 @@ > ?? ???? ?id="VMObjectAlloc" const="JVMTI_EVENT_VM_OBJECT_ALLOC" num="84"> > ???? > -????? Sent when a method causes the virtual machine to allocate an > -????? Object visible to Java programming language code and the > -????? allocation is not detectable by other intrumentation mechanisms. > +????? Sent when a method causes the virtual machine to directly > allocate an > +????? Object visible to Java programming language code. > ?????? Generally object allocation should be detected by instrumenting > ?????? the bytecodes of allocating methods. > ?????? Object allocation generated in native code by JNI function > @@ -13520,6 +13519,12 @@ > ?????? VM. These methods should send this event. > ?????? Virtual machines which are incapable of bytecode instrumentation > ?????? for some or all of their methods can send this event. > + > +????? Note that the + id="SampledObjectAlloc">SampledObjectAlloc > +????? event is triggered on all Java object allocations, including those > +????? caused by bytecode method execution, JNI method execution, and > +????? directly by VM methods. > ??????

    > ?????? Typical examples where this event might be sent: > ??????

      > > > Thanks, > Serguei > > > On 6/23/18 04:49, David Holmes wrote: >> On 23/06/2018 6:25 PM, serguei.spitsyn at oracle.com wrote: >>> I've pushed the version suggested by David. >> >> But you left out all of Jeremy's other fixups! >> >> David >> >>> Thanks, >>> serguei >>> >>> >>> On 6/22/18 09:00, serguei.spitsyn at oracle.com wrote: >>>> Hi Jeremy, >>>> >>>> Okay, let me look at it once more before making final decision. >>>> We have all suggestions and preferences listed. >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 6/22/18 08:22, Jeremy Manson wrote: >>>>> Hey folks - >>>>> >>>>> With the window closing soon for 11, I feel that we should get this >>>>> revision in (just so that the spec is clear).? That said, I don't >>>>> feel strongly about the wording.? Who gets to make the decision? >>>>> >>>>> Jeremy >>>>> >>>>> On Wed, Jun 20, 2018 at 9:41 AM Jeremy Manson >>>>> > wrote: >>>>> >>>>> ??? FWIW, I'm also okay with David's version, with the caveat that >>>>> ??? "These methods should send this event" should probably read >>>>> ??? "Invocations of these methods should trigger this event". >>>>> >>>>> ??? Jeremy >>>>> >>>>> ??? On Wed, Jun 20, 2018 at 4:11 AM David Holmes >>>>> ??? > wrote: >>>>> >>>>> ??????? On 20/06/2018 4:48 PM, serguei.spitsyn at oracle.com >>>>> ??????? wrote: >>>>> ??????? > On 6/19/18 23:29, Jeremy Manson wrote: >>>>> ??????? >> Maybe we should make that clarification. >>>>> ??????? >> >>>>> ??????? >> Also, the reason I danced around that in my revision is >>>>> that >>>>> ??????? > >>>>> ??????? > Understand that. >>>>> ??????? > But it is not a good style to clarify about >>>>> ??????? SampledObjectAlloc in the >>>>> ??????? > spec of VMObjectAlloc. >>>>> ??????? > Would be nice to find a way to keep it simple. >>>>> ??????? > >>>>> ??????? > We could keep the original spec as it is but add that all >>>>> ??????? this does not >>>>> ??????? > apply to the SampledObjectAlloc events. >>>>> >>>>> ??????? I don't think that really presents a coherent spec. I still >>>>> ??????? like my >>>>> ??????? suggestion :) >>>>> >>>>> ??????? David >>>>> >>>>> ??????? > >>>>> ??????? > Something like below: >>>>> ??????? > >>>>> ??????? >? ? Note, the above does not apply to the >>>> ??????? > id="SampledObjectAlloc">SampledObjectAlloc >>>>> ??????? >? ? event as it is triggered on all Java object allocations, >>>>> ??????? including >>>>> ??????? > those caused by bytecode method execution, >>>>> ??????? >? ? JNI method execution, and directly by VM methods.* >>>>> ??????? > * >>>>> ??????? > >>>>> ??????? >> if you set the sampling rate to 0, you get unconditional >>>>> ??????? sampling. >>>>> ??????? > A correction. >>>>> ??????? > >>>>> ??????? > I wrote: >>>>> ??????? >? > It is still possible to sample every allocation with the >>>>> ??????? SamplingRate >>>>> ??????? > == 1. >>>>> ??????? > >>>>> ??????? > but had write: SamplingRate == 0 >>>>> ??????? > >>>>> ??????? > Thanks, >>>>> ??????? > Serguei >>>>> ??????? > >>>>> ??????? > >>>>> ??????? >> >>>>> ??????? >> Jeremy >>>>> ??????? >> >>>>> ??????? >> On Tue, Jun 19, 2018 at 10:41 PM >>>>> ??????? serguei.spitsyn at oracle.com >>>>> ??????? >> >>>> ??????? > >>>>> ??????? >>>> >>>>> ??????? >> >>>> ??????? >> wrote: >>>>> ??????? >> >>>>> ??????? >>? ? ?On 6/19/18 21:54, David Holmes wrote: >>>>> ??????? >>? ? ?> On 20/06/2018 2:41 PM, serguei.spitsyn at oracle.com >>>>> ??????? >>>>> ??????? >>? ? ?>>>> ??????? > wrote: >>>>> ??????? >>? ? ?>> On 6/19/18 21:11, Jeremy Manson wrote: >>>>> ??????? >>? ? ?>>> That would be okay with me, assuming that my other >>>>> ??????? corrections >>>>> ??????? >>? ? ?are >>>>> ??????? >>? ? ?>>> made. >>>>> ??????? >>? ? ?>> >>>>> ??????? >>? ? ?>> Another option would be to say "non-sampling" >>>>> ??????? instead of >>>>> ??????? >>? ? ?>> "unconditional": >>>>> ??????? >>? ? ?>> >>>>> ??????? >>? ? ?>> == Sent when a method causes the virtual machine to >>>>> ??????? allocate an >>>>> ??????? >>? ? ?>> == Object visible to Java programming language code >>>>> ??????? and the >>>>> ??????? >>? ? ?allocation >>>>> ??????? >>? ? ?>> += is not detectable by other *non-sampling* >>>>> ??????? instrumentation >>>>> ??????? >>? ? ?mechanisms. >>>>> ??????? >>? ? ?>> >>>>> ??????? >>? ? ?>> >>>>> ??????? >>? ? ?>>> I'd also like to fix the spelling of >>>>> ??????? instrumentation in the first >>>>> ??????? >>? ? ?>>> sentence. >>>>> ??????? >>? ? ?>> >>>>> ??????? >>? ? ?>> Yes, of course. >>>>> ??????? >>? ? ?>> >>>>> ??????? >>? ? ?>> Let's wait for David's opinion. >>>>> ??????? >>? ? ?> >>>>> ??????? >>? ? ?> I'm okay with Serguei's suggestion (combined with >>>>> ??????? Jeremy's other >>>>> ??????? >>? ? ?> changes). >>>>> ??????? >>? ? ?> >>>>> ??????? >>? ? ?> I'm not that familiar with conditional versus >>>>> ??????? unconditional >>>>> ??????? >>? ? ?> instrumentation. >>>>> ??????? >> >>>>> ??????? >>? ? ?The whole point of the SampledObjectAlloc events is to >>>>> ??????? post them >>>>> ??????? >>? ? ?conditionally >>>>> ??????? >>? ? ?depending on the SamplingRate so that just some >>>>> amount of >>>>> ??????? >>? ? ?allocations is >>>>> ??????? >>? ? ?sampled. >>>>> ??????? >>? ? ?It is why the overhead can be minimal (less than 5 >>>>> ??????? percents). >>>>> ??????? >>? ? ?It is still possible to sample every allocation with >>>>> the >>>>> ??????? >>? ? ?SamplingRate == 1. >>>>> ??????? >> >>>>> ??????? >>? ? ?The VMObjectAlloc events are posted unconditionally, >>>>> ??????? so every VM >>>>> ??????? >>? ? ?allocation is posted. >>>>> ??????? >> >>>>> ??????? >>? ? ?Thanks, >>>>> ??????? >>? ? ?Serguei >>>>> ??????? >> >>>>> ??????? >> >>>>> ??????? >>? ? ?> >>>>> ??????? >>? ? ?> Thanks, >>>>> ??????? >>? ? ?> David >>>>> ??????? >>? ? ?> >>>>> ??????? >>? ? ?>> >>>>> ??????? >>? ? ?>> Thanks, >>>>> ??????? >>? ? ?>> Serguei >>>>> ??????? >>? ? ?>> >>>>> ??????? >>? ? ?>> >>>>> ??????? >>? ? ?>>> >>>>> ??????? >>? ? ?>>> Jeremy >>>>> ??????? >>? ? ?>>> >>>>> ??????? >>? ? ?>>> On Mon, Jun 18, 2018 at 11:01 PM >>>>> ??????? serguei.spitsyn at oracle.com >>>>> ??????? >>? ? ?>>>> ??????? > >>>>> ??????? >>? ? ?>>> >>>> ??????? >>>>> ??????? >>? ? ?>>>> ??????? >> >>>>> ??????? >>>> >>>>> ??????? >>? ? ?>>>> ??????? > >>>>> ??????? >>? ? ?>>> >>>> ??????? >>>>> ??????? >>? ? ?>>>> >>> wrote: >>>>> ??????? >>? ? ?>>> >>>>> ??????? >>? ? ?>>> ??? Hi Jeremy and David, >>>>> ??????? >>? ? ?>>> >>>>> ??????? >>? ? ?>>> ??? Sorry for being late to the party. >>>>> ??????? >>? ? ?>>> >>>>> ??????? >>? ? ?>>> ??? I'm also concerned about the Jeremy's spec >>>>> ??????? update is more >>>>> ??????? >>? ? ?>>> ??? intrusive than necessary. >>>>> ??????? >>? ? ?>>> ??? One specifics of the new SampledObjectAlloc >>>>> ??????? event is that >>>>> ??????? >>? ? ?it is >>>>> ??????? >>? ? ?>>> ??? posted conditionally. >>>>> ??????? >>? ? ?>>> ??? So, it is not fully comparable with the >>>>> ??????? VMObjectAlloc >>>>> ??????? >>? ? ?event and >>>>> ??????? >>? ? ?>>> ??? can not replace it in any way. >>>>> ??????? >>? ? ?>>> ??? I'm even not yet convinced that any spec >>>>> ??????? update is necessary. >>>>> ??????? >>? ? ?>>> >>>>> ??????? >>? ? ?>>> ??? However, something like below would look >>>>> ??????? minimal and >>>>> ??????? >>? ? ?reasonable: >>>>> ??????? >>? ? ?>>> >>>>> ??????? >>? ? ?>>> ??? == Sent when a method causes the virtual >>>>> ??????? machine to >>>>> ??????? >>? ? ?allocate an >>>>> ??????? >>? ? ?>>> ??? == Object visible to Java programming language >>>>> ??????? code and the >>>>> ??????? >>? ? ?>>> allocation >>>>> ??????? >>? ? ?>>> ??? += is not detectable by other *unconditional* >>>>> ??????? intrumentation >>>>> ??????? >>? ? ?>>> ??? mechanisms. >>>>> ??????? >>? ? ?>>> >>>>> ??????? >>? ? ?>>> ??? == Generally object allocation should be >>>>> ??????? detected by >>>>> ??????? >>? ? ?instrumenting >>>>> ??????? >>? ? ?>>> ??? == the bytecodes of allocating methods. >>>>> ??????? >>? ? ?>>> >>>>> ??????? >>? ? ?>>> ??? == Object allocation generated in native code >>>>> ??????? by JNI function >>>>> ??????? >>? ? ?>>> ??? == calls should be detected using >>>>> ??????? >>? ? ?>>> ??? == JNI function >>>>> ??????? >>? ? ?>>> interception. >>>>> ??????? >>? ? ?>>> >>>>> ??????? >>? ? ?>>> ??? == Some methods might not have associated >>>>> ??????? bytecodes and >>>>> ??????? >>? ? ?are not >>>>> ??????? >>? ? ?>>> ??? == native methods, they instead are executed >>>>> ??????? directly by the >>>>> ??????? >>? ? ?>>> ??? == VM. These methods should send this event. >>>>> ??????? >>? ? ?>>> >>>>> ??????? >>? ? ?>>> ??? == Virtual machines which are incapable of >>>>> ??????? bytecode >>>>> ??????? >>? ? ?instrumentation >>>>> ??????? >>? ? ?>>> ??? == for some or all of their methods can send >>>>> ??????? this event. >>>>> ??????? >>? ? ?>>> >>>>> ??????? >>? ? ?>>> ??? *++ Note that the >>>> ??????? >>? ? ?>>> >>>>> id="SampledObjectAlloc">SampledObjectAlloc** >>>>> ??????? >>? ? ?>>> ??? **++ event is conditionally triggered on all >>>>> ??????? Java object >>>>> ??????? >>? ? ?>>> ??? allocations, including those** >>>>> ??????? >>? ? ?>>> ??? **++ caused by bytecode method execution, JNI >>>>> ??????? method >>>>> ??????? >>? ? ?execution, >>>>> ??????? >>? ? ?>>> ??? and directly by VM methods.** >>>>> ??????? >>? ? ?>>> ??? * >>>>> ??????? >>? ? ?>>> >>>>> ??????? >>? ? ?>>> ??? Maybe, just adding the last statement would be >>>>> ??????? good enough. >>>>> ??????? >>? ? ?>>> >>>>> ??????? >>? ? ?>>> ??? Thanks, >>>>> ??????? >>? ? ?>>> ??? Serguei >>>>> ??????? >>? ? ?>>> >>>>> ??????? >>? ? ?>>> >>>>> ??????? >>? ? ?>>> ??? On 6/18/18 21:36, David Holmes wrote: >>>>> ??????? >>? ? ?>>>> ??? On 19/06/2018 4:50 AM, Jeremy Manson wrote: >>>>> ??????? >>? ? ?>>>>> ??? Yup!? The paragraph meanders a bit. How >>>>> ??????? about something >>>>> ??????? >>? ? ?like: >>>>> ??????? >>? ? ?>>>> >>>>> ??????? >>? ? ?>>>> ??? I'm not sure some of the change quite works. >>>>> ??????? The original >>>>> ??????? >>? ? ?text >>>>> ??????? >>? ? ?>>>> ??? considers there to be three kinds of methods >>>>> ??????? that can cause >>>>> ??????? >>? ? ?>>>> ??? allocation when executed: >>>>> ??????? >>? ? ?>>>> ??? - Java (bytecode) methods >>>>> ??????? >>? ? ?>>>> ??? - JNI methods >>>>> ??????? >>? ? ?>>>> ??? - VM methods >>>>> ??????? >>? ? ?>>>> >>>>> ??????? >>? ? ?>>>> ??? but you've turned this into three kinds of >>>>> ??????? allocation: via >>>>> ??????? >>? ? ?>>>> ??? bytecode, via JNI, and via the VM. You then >>>>> ??????? refer to >>>>> ??????? >>? ? ?"triggering" >>>>> ??????? >>? ? ?>>>> ??? an allocation when we tend to use triggering >>>>> ??????? for events. >>>>> ??????? >>? ? ?You also >>>>> ??????? >>? ? ?>>>> ??? refer to an allocation being "executed >>>>> ??????? directly by the VM" (a >>>>> ??????? >>? ? ?>>>> ??? phrase previously applied when the subject >>>>> was a >>>>> ??????? >>? ? ?'method') - but >>>>> ??????? >>? ? ?>>>> ??? you don't really execute allocations. >>>>> ??????? >>? ? ?>>>> >>>>> ??????? >>? ? ?>>>> ??? IIUC the problem with the existing text is >>>>> ??????? just that it >>>>> ??????? >>? ? ?considers >>>>> ??????? >>? ? ?>>>> ??? VM allocation events as being undetectable >>>>> ??????? other than by >>>>> ??????? >>? ? ?this "VM >>>>> ??????? >>? ? ?>>>> ??? object allocation event" - but that's no >>>>> ??????? longer true. So how >>>>> ??????? >>? ? ?>>>> ??? about something minimally changed like this: >>>>> ??????? >>? ? ?>>>> >>>>> ??????? >>? ? ?>>>> ??? --- >>>>> ??????? >>? ? ?>>>> ??? ? Sent when a method causes the virtual >>>>> ??????? machine to directly >>>>> ??????? >>? ? ?>>>> ??? allocate an >>>>> ??????? >>? ? ?>>>> ??? ? Object visible to Java programming language >>>>> ??????? code. >>>>> ??????? >>? ? ?>>>> ??? ? Generally object allocation can be >>>>> detected by >>>>> ??????? >>? ? ?instrumenting >>>>> ??????? >>? ? ?>>>> ??? ? the bytecodes of allocating methods. >>>>> ??????? >>? ? ?>>>> ??? ? Object allocation generated in native code >>>>> ??????? by JNI function >>>>> ??????? >>? ? ?>>>> ??? ? calls can be detected using >>>>> ??????? >>? ? ?>>>> ??? ? JNI function >>>>> ??????? >>? ? ?>>>> interception. >>>>> ??????? >>? ? ?>>>> ??? ?? Some methods might not have associated >>>>> ??????? bytecodes and >>>>> ??????? >>? ? ?are not >>>>> ??????? >>? ? ?>>>> ??? ?? native methods, they instead are executed >>>>> ??????? directly by the >>>>> ??????? >>? ? ?>>>> ??? ?? VM. These methods should send this event. >>>>> ??????? >>? ? ?>>>> ??? ?? Virtual machines which are incapable of >>>>> ??????? bytecode >>>>> ??????? >>? ? ?>>>> instrumentation >>>>> ??????? >>? ? ?>>>> ??? ?? for some or all of their methods can send >>>>> ??????? this event. >>>>> ??????? >>? ? ?>>>> >>>>> ??????? >>? ? ?>>>> ??? ?? Note that the >>>> ??????? >>? ? ?>>>> >>>>> id="SampledObjectAlloc">SampledObjectAlloc >>>>> ??????? >>? ? ?>>>> ??? ?? event is triggered on all Java object >>>>> ??????? allocations, >>>>> ??????? >>? ? ?including >>>>> ??????? >>? ? ?>>>> ??? those >>>>> ??????? >>? ? ?>>>> ??? ?? caused by bytecode method execution, JNI >>>>> ??????? method >>>>> ??????? >>? ? ?execution, and >>>>> ??????? >>? ? ?>>>> ??? ?? directly by VM methods. >>>>> ??????? >>? ? ?>>>> ??? --- >>>>> ??????? >>? ? ?>>>> >>>>> ??????? >>? ? ?>>>> ??? Thanks, >>>>> ??????? >>? ? ?>>>> ??? David >>>>> ??????? >>? ? ?>>>> >>>>> ??????? >>? ? ?>>>>> ??? Sent when the virtual machine allocates an >>>>> ??????? >>? ? ?>>>>> ??? Object visible to Java programming language >>>>> ??????? code without >>>>> ??????? >>? ? ?using a >>>>> ??????? >>? ? ?>>>>> new bytecode variant or a JNI >>>>> method. >>>>> ??????? >>? ? ?>>>>> ??? Many approaches to tracking object >>>>> ??????? allocation use a >>>>> ??????? >>? ? ?>>>>> combination of >>>>> ??????? >>? ? ?>>>>> ??? bytecode-based instrumentation and >>>>> >>>> ??????? >>? ? ?>>>>> id="jniIntercept">JNI function >>>>> ??????? >>? ? ?>>>>> interception to intercept >>>>> ??????? allocations. >>>>> ??????? >>? ? ?However, >>>>> ??????? >>? ? ?>>>>> ??? this >>>>> ??????? >>? ? ?>>>>> ??? approach can leave a number of allocations >>>>> ??????? undetected. >>>>> ??????? >>? ? ?>>>>> ??? Allocations that are neither >>>>> ??????? >>? ? ?>>>>> ??? triggered by bytecode nor JNI are executed >>>>> ??????? directly by >>>>> ??????? >>? ? ?the VM. >>>>> ??????? >>? ? ?>>>>> ??? When those allocations occur, the VM should >>>>> ??????? send this event. >>>>> ??????? >>? ? ?>>>>> ??? Virtual machines that are incapable of >>>>> bytecode >>>>> ??????? >>? ? ?instrumentation >>>>> ??????? >>? ? ?>>>>> ??? for some or all of their methods may also >>>>> ??????? send this event. >>>>> ??????? >>? ? ?>>>>> ???

      >>>>> ??????? >>? ? ?>>>>> ??? Note that the >>>> ??????? >>? ? ?>>>>> >>>>> id="SampledObjectAlloc">SampledObjectAlloc >>>>> ??????? >>? ? ?>>>>> ??? event is triggered on all Java object >>>>> ??????? allocations, including >>>>> ??????? >>? ? ?>>>>> ??? those triggered by bytecode, >>>>> ??????? >>? ? ?>>>>> ??? JNI methods, and VM events. >>>>> ??????? >>? ? ?>>>>> >>>>> ??????? >>? ? ?>>>>> >>>>> ??????? >>? ? ?>>>>> >>>>> ??????? >>? ? ?>>>>> ??? On Mon, Jun 18, 2018 at 12:57 AM David Holmes >>>>> ??????? >>? ? ?>>>>> >>>> ??????? >>>>> ??????? >>? ? ?>>>> ??????? > >>>>> ??????? >>>> >>>>> ??????? >>? ? ?>>>> ??????? >> >>>>> ??????? >>? ? ?>>>>> >>>> ??????? >>>>> ??????? >>? ? ?>>>> ??????? >> >>>>> ??????? >>? ? ?>>>>> >>>> ??????? >>>>> ??????? >>? ? ?>>>> ??????? >>> wrote: >>>>> ??????? >>? ? ?>>>>> >>>>> ??????? >>? ? ?>>>>> ??? ??? On 18/06/2018 5:01 PM, Jeremy Manson >>>>> wrote: >>>>> ??????? >>? ? ?>>>>> ??? ???? > We haven't changed when a VM issues >>>>> ??????? "VM object >>>>> ??????? >>? ? ?>>>>> ??? allocation" events. >>>>> ??????? >>? ? ?>>>>> ??? ???? > >>>>> ??????? >>? ? ?>>>>> ??? ???? > There were references in the docs to a >>>>> ??????? >>? ? ?requirement to use >>>>> ??????? >>? ? ?>>>>> ??? bytecode >>>>> ??????? >>? ? ?>>>>> ??? ???? > rewriting and JNI interception to >>>>> track >>>>> ??????? >>? ? ?allocations.? With >>>>> ??????? >>? ? ?>>>>> ??? ???? > SampledObjectAlloc, this is no longer >>>>> ??????? the case - >>>>> ??????? >>? ? ?>>>>> SampledObjectAlloc can >>>>> ??????? >>? ? ?>>>>> ??? ???? > track them.? This change is supposed >>>>> ??????? to remove the >>>>> ??????? >>? ? ?>>>>> ??? references to >>>>> ??????? >>? ? ?>>>>> ??? ??? those >>>>> ??????? >>? ? ?>>>>> ??? ???? > requirements, and provide suitable >>>>> ??????? replacement text. >>>>> ??????? >>? ? ?>>>>> ??? ???? > >>>>> ??????? >>? ? ?>>>>> ??? ???? > VM object alloc has specific language >>>>> ??????? about being >>>>> ??????? >>? ? ?able to >>>>> ??????? >>? ? ?>>>>> ??? use it to >>>>> ??????? >>? ? ?>>>>> ??? ???? > track allocations that cannot be >>>>> ??????? tracked with >>>>> ??????? >>? ? ?bytecode >>>>> ??????? >>? ? ?>>>>> instrumentation >>>>> ??????? >>? ? ?>>>>> ??? ???? > and JNI interception.? My goal in >>>>> ??????? rephrasing was >>>>> ??????? >>? ? ?to make >>>>> ??????? >>? ? ?>>>>> ??? it clear >>>>> ??????? >>? ? ?>>>>> ??? ??? that, >>>>> ??????? >>? ? ?>>>>> ??? ???? > while you can still use it for this, >>>>> ??????? you can also >>>>> ??????? >>? ? ?just use >>>>> ??????? >>? ? ?>>>>> ??? ???? > SampledObjectAlloc for everything. >>>>> ??????? >>? ? ?>>>>> >>>>> ??????? >>? ? ?>>>>> ??? ??? Okay. That doesn't come across clearly >>>>> ??????? to me - >>>>> ??????? >>? ? ?sorry. So you >>>>> ??????? >>? ? ?>>>>> ??? will now >>>>> ??????? >>? ? ?>>>>> ??? ??? get both kinds of events for allocations >>>>> ??????? done in the VM? >>>>> ??????? >>? ? ?>>>>> >>>>> ??????? >>? ? ?>>>>> ??? ??? Thanks, >>>>> ??????? >>? ? ?>>>>> ??? ??? David >>>>> ??????? >>? ? ?>>>>> >>>>> ??????? >>? ? ?>>>>> >>>>> ??????? >>? ? ?>>>>> ??? ???? > Jeremy >>>>> ??????? >>? ? ?>>>>> ??? ???? > >>>>> ??????? >>? ? ?>>>>> ??? ???? > On Sun, Jun 17, 2018 at 9:11 PM David >>>>> ??????? Holmes >>>>> ??????? >>? ? ?>>>>> >>>> ??????? >>>>> ??????? >>? ? ?>>>> ??????? > >>>>> ??????? >>>> >>>>> ??????? >>? ? ?>>>> ??????? >> >>>>> ??????? >>? ? ?>>>>> >>>> ??????? >>>>> ??????? >>? ? ?>>>> ??????? >> >>>>> ??????? >>>> >>>>> ??????? >>? ? ?>>>> ??????? >> >>>>> ??????? >>? ? ?>>>>> ??? ???? > >>>> ??????? >>>>> ??????? >>? ? ?>>>> ??????? > >>>>> ??????? >>? ? ?>>>>> >>>> ??????? >>>>> ??????? >>? ? ?>>>> ??????? >> >>>>> ??????? >>? ? ?>>>>> >>>> ??????? >>>>> ??????? >>? ? ?>>>> >>>> wrote: >>>>> ??????? >>? ? ?>>>>> ??? ???? > >>>>> ??????? >>? ? ?>>>>> ??? ???? > ?Hi Jeremy, >>>>> ??????? >>? ? ?>>>>> ??? ???? > >>>>> ??????? >>? ? ?>>>>> ??? ???? > ?On 16/06/2018 2:33 AM, Jeremy >>>>> ??????? Manson wrote: >>>>> ??????? >>? ? ?>>>>> ??? ???? > > Hi all, >>>>> ??????? >>? ? ?>>>>> ??? ???? > > >>>>> ??????? >>? ? ?>>>>> ??? ???? > > There are a number of references >>>>> in the >>>>> ??????? >>? ? ?JVMTI doc >>>>> ??????? >>? ? ?>>>>> ??? to its >>>>> ??????? >>? ? ?>>>>> ??? ??? not doing >>>>> ??????? >>? ? ?>>>>> ??? ???? > > object allocation tracking.? Now >>>>> ??????? that JEP >>>>> ??????? >>? ? ?331 has >>>>> ??????? >>? ? ?>>>>> ??? landed, >>>>> ??????? >>? ? ?>>>>> ??? ??? these >>>>> ??????? >>? ? ?>>>>> ??? ???? > > references are obsolete.? This patch >>>>> ??????? >>? ? ?changes those >>>>> ??????? >>? ? ?>>>>> ??? references >>>>> ??????? >>? ? ?>>>>> ??? ???? > ?accordingly. >>>>> ??????? >>? ? ?>>>>> ??? ???? > > >>>>> ??????? >>? ? ?>>>>> ??? ???? > > While I was there, I took the >>>>> ??????? liberty of >>>>> ??????? >>? ? ?fixing >>>>> ??????? >>? ? ?>>>>> some >>>>> ??????? >>? ? ?>>>>> spelling?errors. >>>>> ??????? >>? ? ?>>>>> ??? ???? > > >>>>> ??????? >>? ? ?>>>>> ??? ???? > > As far as I know, these are >>>>> ??????? non-normative >>>>> ??????? >>? ? ?>>>>> changes and >>>>> ??????? >>? ? ?>>>>> ??? ??? modify no >>>>> ??????? >>? ? ?>>>>> ??? ???? > ?API, so >>>>> ??????? >>? ? ?>>>>> ??? ???? > > they should not require a CSR. >>>>> ??????? >>? ? ?>>>>> ??? ???? > >>>>> ??????? >>? ? ?>>>>> ??? ???? > ?I'm unclear on the nature of the >>>>> ??????? change to >>>>> ??????? >>? ? ?"VM Object >>>>> ??????? >>? ? ?>>>>> Allocation". Does >>>>> ??????? >>? ? ?>>>>> ??? ???? > ?the existence of >>>>> ??????? SampledObjectAlloc change >>>>> ??????? >>? ? ?when a VM >>>>> ??????? >>? ? ?>>>>> ??? should >>>>> ??????? >>? ? ?>>>>> ??? ??? issue "VM >>>>> ??????? >>? ? ?>>>>> ??? ???? > ?object allocation" events? >>>>> ??????? >>? ? ?>>>>> ??? ???? > >>>>> ??????? >>? ? ?>>>>> ??? ???? > ?Thanks, >>>>> ??????? >>? ? ?>>>>> ??? ???? > ?David >>>>> ??????? >>? ? ?>>>>> ??? ???? > >>>>> ??????? >>? ? ?>>>>> ??? ???? > > >>>>> ??????? >>? ? ?>>>>> ??? ???? > > Bug: >>>>> ??????? >>? ? ?>>>>> ??? ???? > > >>>>> ??????? >> https://bugs.openjdk.java.net/browse/JDK-8205113 >>>>> ??????? >>? ? ?>>>>> ??? ???? > > >>>>> ??????? >>? ? ?>>>>> ??? ???? > > Webrev: >>>>> ??????? >>? ? ?>>>>> ??? ???? > > >>>>> ??????? >>? ? ?>>>>> >>>>> http://cr.openjdk.java.net/~jmanson/8205113/webrev.00/ >>>>> >>>>> ??????? >> ? >>>>> ??????? >>? ? ?>>>>> >>>>> >>>>> ??????? >>? ? ?>>>>> ??? ???? > > >>>>> ??????? >>? ? ?>>>>> ??? ???? > > Thanks! >>>>> ??????? >>? ? ?>>>>> ??? ???? > > >>>>> ??????? >>? ? ?>>>>> ??? ???? > > Jeremy >>>>> ??????? >>? ? ?>>>>> ??? ???? > >>>>> ??????? >>? ? ?>>>>> >>>>> ??????? >>? ? ?>>> >>>>> ??????? >>? ? ?>> >>>>> ??????? >> >>>>> ??????? > >>>>> >>>> >>> > From david.holmes at oracle.com Sat Jun 23 23:10:56 2018 From: david.holmes at oracle.com (David Holmes) Date: Sun, 24 Jun 2018 09:10:56 +1000 Subject: RFR(XS): 8205561: problem list vmTestbase/nsk/jdb/trace/trace001/trace001.java In-Reply-To: <12bed6fe-c1e8-1ec3-95a8-7f80aae887dc@oracle.com> References: <12bed6fe-c1e8-1ec3-95a8-7f80aae887dc@oracle.com> Message-ID: <9ab73ed0-1715-65b6-9e3e-5caefa1bedb2@oracle.com> Hi Chris, Peter (cc'd) has an RFR out to fix 8205540 now - which I just reviewed. Thanks, David On 23/06/2018 3:42 PM, Chris Plummer wrote: > Hello, > > Please review the following: > > https://bugs.openjdk.java.net/browse/JDK-8205561 > > diff --git a/test/hotspot/jtreg/ProblemList.txt > b/test/hotspot/jtreg/ProblemList.txt > --- a/test/hotspot/jtreg/ProblemList.txt > +++ b/test/hotspot/jtreg/ProblemList.txt > @@ -166,7 +166,7 @@ > ?vmTestbase/vm/mlvm/indy/func/jvmti/mergeCP_indy2manySame_b/TestDescription.java 8013267 generic-all > > ?vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java 8197938 windows-all > - > +vmTestbase/nsk/jdb/trace/trace001/trace001.java 8205540 generic-all > ?vmTestbase/nsk/jdwp/ThreadReference/ForceEarlyReturn/forceEarlyReturn001/forceEarlyReturn001.java 7199837 generic-all > > ?############################################################################# > > thanks, > > Chris > > From serguei.spitsyn at oracle.com Sat Jun 23 23:19:54 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Sat, 23 Jun 2018 16:19:54 -0700 Subject: RFR 8205113: Update JVMTI doc references to object allocation tracking In-Reply-To: <845054d9-ec16-e2f8-afec-c885127bbd04@oracle.com> References: <3d01255e-8712-46aa-7659-4ccd7325f81c@oracle.com> <6927237d-ce0b-17c0-68f8-06d05b84e516@oracle.com> <53f576e2-0adb-a6c5-8a3c-026dcb1aa7ad@oracle.com> <8224afa2-a6cb-fee2-8fa3-184828ac7c50@oracle.com> <60899f09-12df-1e6e-7e16-406a60939b7c@oracle.com> <96713bc3-9350-fd4a-75e5-2c36257066b2@oracle.com> <3018f21d-52b0-9ec4-151e-e4fb01da10ff@oracle.com> <3ffcc780-2568-ec21-b503-228c91d18969@oracle.com> <845054d9-ec16-e2f8-afec-c885127bbd04@oracle.com> Message-ID: <51b86cf0-0287-4246-fe1d-f7f536ca3140@oracle.com> On 6/23/18 15:53, David Holmes wrote: > On 24/06/2018 7:58 AM, serguei.spitsyn at oracle.com wrote: >> Hi David, >> >> This was your suggestion: > > There was only one part of the patch being debated! The rest of the > patch was still applicable. Jeremy specifically stated > > "That would be okay with me, assuming that my other corrections are > made.? I'd also like to fix the spelling of instrumentation in the > first sentence." Yes, I missed the other part - sorry. :( Will push it under a separate bug id. Not sure if another review is needed. Thanks, Serguei > David > >> --- >> ?? Sent when a method causes the virtual machine to directly allocate an >> ?? Object visible to Java programming language code. >> ?? Generally object allocation can be detected by instrumenting >> ?? the bytecodes of allocating methods. >> ?? Object allocation generated in native code by JNI function >> ?? calls can be detected using >> ?? JNI function >> interception. >> ??? Some methods might not have associated bytecodes and are not >> ??? native methods, they instead are executed directly by the >> ??? VM. These methods should send this event. >> ??? Virtual machines which are incapable of bytecode instrumentation >> ??? for some or all of their methods can send this event. >> >> ??? Note that the > id="SampledObjectAlloc">SampledObjectAlloc >> ??? event is triggered on all Java object allocations, including those >> ??? caused by bytecode method execution, JNI method execution, and >> ??? directly by VM methods. >> --- >> >> I've pushed the patch: >> >> diff -r f703d45c5687 src/hotspot/share/prims/jvmti.xml >> --- a/src/hotspot/share/prims/jvmti.xml??? Tue Jun 05 11:55:39 2018 >> +0200 >> +++ b/src/hotspot/share/prims/jvmti.xml??? Sat Jun 23 01:18:57 2018 >> -0700 >> @@ -13507,9 +13507,8 @@ >> ??? > ????? ?id="VMObjectAlloc" const="JVMTI_EVENT_VM_OBJECT_ALLOC" num="84"> >> ????? >> -????? Sent when a method causes the virtual machine to allocate an >> -????? Object visible to Java programming language code and the >> -????? allocation is not detectable by other intrumentation mechanisms. >> +????? Sent when a method causes the virtual machine to directly >> allocate an >> +????? Object visible to Java programming language code. >> ??????? Generally object allocation should be detected by instrumenting >> ??????? the bytecodes of allocating methods. >> ??????? Object allocation generated in native code by JNI function >> @@ -13520,6 +13519,12 @@ >> ??????? VM. These methods should send this event. >> ??????? Virtual machines which are incapable of bytecode instrumentation >> ??????? for some or all of their methods can send this event. >> + >> +????? Note that the > + id="SampledObjectAlloc">SampledObjectAlloc >> +????? event is triggered on all Java object allocations, including >> those >> +????? caused by bytecode method execution, JNI method execution, and >> +????? directly by VM methods. >> ???????

      >> ??????? Typical examples where this event might be sent: >> ???????

        >> >> >> Thanks, >> Serguei >> >> >> On 6/23/18 04:49, David Holmes wrote: >>> On 23/06/2018 6:25 PM, serguei.spitsyn at oracle.com wrote: >>>> I've pushed the version suggested by David. >>> >>> But you left out all of Jeremy's other fixups! >>> >>> David >>> >>>> Thanks, >>>> serguei >>>> >>>> >>>> On 6/22/18 09:00, serguei.spitsyn at oracle.com wrote: >>>>> Hi Jeremy, >>>>> >>>>> Okay, let me look at it once more before making final decision. >>>>> We have all suggestions and preferences listed. >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>> On 6/22/18 08:22, Jeremy Manson wrote: >>>>>> Hey folks - >>>>>> >>>>>> With the window closing soon for 11, I feel that we should get >>>>>> this revision in (just so that the spec is clear).? That said, I >>>>>> don't feel strongly about the wording.? Who gets to make the >>>>>> decision? >>>>>> >>>>>> Jeremy >>>>>> >>>>>> On Wed, Jun 20, 2018 at 9:41 AM Jeremy Manson >>>>>> > wrote: >>>>>> >>>>>> ??? FWIW, I'm also okay with David's version, with the caveat that >>>>>> ??? "These methods should send this event" should probably read >>>>>> ??? "Invocations of these methods should trigger this event". >>>>>> >>>>>> ??? Jeremy >>>>>> >>>>>> ??? On Wed, Jun 20, 2018 at 4:11 AM David Holmes >>>>>> ??? > >>>>>> wrote: >>>>>> >>>>>> ??????? On 20/06/2018 4:48 PM, serguei.spitsyn at oracle.com >>>>>> ??????? wrote: >>>>>> ??????? > On 6/19/18 23:29, Jeremy Manson wrote: >>>>>> ??????? >> Maybe we should make that clarification. >>>>>> ??????? >> >>>>>> ??????? >> Also, the reason I danced around that in my revision >>>>>> is that >>>>>> ??????? > >>>>>> ??????? > Understand that. >>>>>> ??????? > But it is not a good style to clarify about >>>>>> ??????? SampledObjectAlloc in the >>>>>> ??????? > spec of VMObjectAlloc. >>>>>> ??????? > Would be nice to find a way to keep it simple. >>>>>> ??????? > >>>>>> ??????? > We could keep the original spec as it is but add that all >>>>>> ??????? this does not >>>>>> ??????? > apply to the SampledObjectAlloc events. >>>>>> >>>>>> ??????? I don't think that really presents a coherent spec. I still >>>>>> ??????? like my >>>>>> ??????? suggestion :) >>>>>> >>>>>> ??????? David >>>>>> >>>>>> ??????? > >>>>>> ??????? > Something like below: >>>>>> ??????? > >>>>>> ??????? >? ? Note, the above does not apply to the >>>>> ??????? > id="SampledObjectAlloc">SampledObjectAlloc >>>>>> ??????? >? ? event as it is triggered on all Java object >>>>>> allocations, >>>>>> ??????? including >>>>>> ??????? > those caused by bytecode method execution, >>>>>> ??????? >? ? JNI method execution, and directly by VM methods.* >>>>>> ??????? > * >>>>>> ??????? > >>>>>> ??????? >> if you set the sampling rate to 0, you get unconditional >>>>>> ??????? sampling. >>>>>> ??????? > A correction. >>>>>> ??????? > >>>>>> ??????? > I wrote: >>>>>> ??????? >? > It is still possible to sample every allocation with >>>>>> the >>>>>> ??????? SamplingRate >>>>>> ??????? > == 1. >>>>>> ??????? > >>>>>> ??????? > but had write: SamplingRate == 0 >>>>>> ??????? > >>>>>> ??????? > Thanks, >>>>>> ??????? > Serguei >>>>>> ??????? > >>>>>> ??????? > >>>>>> ??????? >> >>>>>> ??????? >> Jeremy >>>>>> ??????? >> >>>>>> ??????? >> On Tue, Jun 19, 2018 at 10:41 PM >>>>>> ??????? serguei.spitsyn at oracle.com >>>>>> >>>>>> ??????? >> >>>>> ??????? > >>>>>> ??????? >>>>> >>>>>> ??????? >> >>>>> >> wrote: >>>>>> ??????? >> >>>>>> ??????? >>? ? ?On 6/19/18 21:54, David Holmes wrote: >>>>>> ??????? >>? ? ?> On 20/06/2018 2:41 PM, serguei.spitsyn at oracle.com >>>>>> ??????? >>>>>> ??????? >> ?>>>>> ??????? > wrote: >>>>>> ??????? >>? ? ?>> On 6/19/18 21:11, Jeremy Manson wrote: >>>>>> ??????? >>? ? ?>>> That would be okay with me, assuming that my >>>>>> other >>>>>> ??????? corrections >>>>>> ??????? >>? ? ?are >>>>>> ??????? >>? ? ?>>> made. >>>>>> ??????? >>? ? ?>> >>>>>> ??????? >>? ? ?>> Another option would be to say "non-sampling" >>>>>> ??????? instead of >>>>>> ??????? >>? ? ?>> "unconditional": >>>>>> ??????? >>? ? ?>> >>>>>> ??????? >>? ? ?>> == Sent when a method causes the virtual >>>>>> machine to >>>>>> ??????? allocate an >>>>>> ??????? >>? ? ?>> == Object visible to Java programming language >>>>>> code >>>>>> ??????? and the >>>>>> ??????? >>? ? ?allocation >>>>>> ??????? >>? ? ?>> += is not detectable by other *non-sampling* >>>>>> ??????? instrumentation >>>>>> ??????? >>? ? ?mechanisms. >>>>>> ??????? >>? ? ?>> >>>>>> ??????? >>? ? ?>> >>>>>> ??????? >>? ? ?>>> I'd also like to fix the spelling of >>>>>> ??????? instrumentation in the first >>>>>> ??????? >>? ? ?>>> sentence. >>>>>> ??????? >>? ? ?>> >>>>>> ??????? >>? ? ?>> Yes, of course. >>>>>> ??????? >>? ? ?>> >>>>>> ??????? >>? ? ?>> Let's wait for David's opinion. >>>>>> ??????? >>? ? ?> >>>>>> ??????? >>? ? ?> I'm okay with Serguei's suggestion (combined with >>>>>> ??????? Jeremy's other >>>>>> ??????? >>? ? ?> changes). >>>>>> ??????? >>? ? ?> >>>>>> ??????? >>? ? ?> I'm not that familiar with conditional versus >>>>>> ??????? unconditional >>>>>> ??????? >>? ? ?> instrumentation. >>>>>> ??????? >> >>>>>> ??????? >>? ? ?The whole point of the SampledObjectAlloc events >>>>>> is to >>>>>> ??????? post them >>>>>> ??????? >>? ? ?conditionally >>>>>> ??????? >>? ? ?depending on the SamplingRate so that just some >>>>>> amount of >>>>>> ??????? >>? ? ?allocations is >>>>>> ??????? >>? ? ?sampled. >>>>>> ??????? >>? ? ?It is why the overhead can be minimal (less than 5 >>>>>> ??????? percents). >>>>>> ??????? >>? ? ?It is still possible to sample every allocation >>>>>> with the >>>>>> ??????? >>? ? ?SamplingRate == 1. >>>>>> ??????? >> >>>>>> ??????? >>? ? ?The VMObjectAlloc events are posted unconditionally, >>>>>> ??????? so every VM >>>>>> ??????? >>? ? ?allocation is posted. >>>>>> ??????? >> >>>>>> ??????? >>? ? ?Thanks, >>>>>> ??????? >>? ? ?Serguei >>>>>> ??????? >> >>>>>> ??????? >> >>>>>> ??????? >>? ? ?> >>>>>> ??????? >>? ? ?> Thanks, >>>>>> ??????? >>? ? ?> David >>>>>> ??????? >>? ? ?> >>>>>> ??????? >>? ? ?>> >>>>>> ??????? >>? ? ?>> Thanks, >>>>>> ??????? >>? ? ?>> Serguei >>>>>> ??????? >>? ? ?>> >>>>>> ??????? >>? ? ?>> >>>>>> ??????? >>? ? ?>>> >>>>>> ??????? >>? ? ?>>> Jeremy >>>>>> ??????? >>? ? ?>>> >>>>>> ??????? >>? ? ?>>> On Mon, Jun 18, 2018 at 11:01 PM >>>>>> ??????? serguei.spitsyn at oracle.com >>>>>> >>>>>> ??????? >> ?>>>>> ??????? > >>>>>> ??????? >>? ? ?>>> >>>>> ??????? >>>>>> ??????? >> ?>>>>> >> >>>>>> ??????? >>>>> >>>>>> ??????? >> ?>>>>> ??????? > >>>>>> ??????? >>? ? ?>>> >>>>> ??????? >>>>>> ??????? >> ?>>>>> >>> wrote: >>>>>> ??????? >>? ? ?>>> >>>>>> ??????? >>? ? ?>>> ??? Hi Jeremy and David, >>>>>> ??????? >>? ? ?>>> >>>>>> ??????? >>? ? ?>>> ??? Sorry for being late to the party. >>>>>> ??????? >>? ? ?>>> >>>>>> ??????? >>? ? ?>>> ??? I'm also concerned about the Jeremy's spec >>>>>> ??????? update is more >>>>>> ??????? >>? ? ?>>> ??? intrusive than necessary. >>>>>> ??????? >>? ? ?>>> ??? One specifics of the new SampledObjectAlloc >>>>>> ??????? event is that >>>>>> ??????? >>? ? ?it is >>>>>> ??????? >>? ? ?>>> ??? posted conditionally. >>>>>> ??????? >>? ? ?>>> ??? So, it is not fully comparable with the >>>>>> ??????? VMObjectAlloc >>>>>> ??????? >>? ? ?event and >>>>>> ??????? >>? ? ?>>> ??? can not replace it in any way. >>>>>> ??????? >>? ? ?>>> ??? I'm even not yet convinced that any spec >>>>>> ??????? update is necessary. >>>>>> ??????? >>? ? ?>>> >>>>>> ??????? >>? ? ?>>> ??? However, something like below would look >>>>>> ??????? minimal and >>>>>> ??????? >>? ? ?reasonable: >>>>>> ??????? >>? ? ?>>> >>>>>> ??????? >>? ? ?>>> ??? == Sent when a method causes the virtual >>>>>> ??????? machine to >>>>>> ??????? >>? ? ?allocate an >>>>>> ??????? >>? ? ?>>> ??? == Object visible to Java programming >>>>>> language >>>>>> ??????? code and the >>>>>> ??????? >>? ? ?>>> allocation >>>>>> ??????? >>? ? ?>>> ??? += is not detectable by other *unconditional* >>>>>> ??????? intrumentation >>>>>> ??????? >>? ? ?>>> ??? mechanisms. >>>>>> ??????? >>? ? ?>>> >>>>>> ??????? >>? ? ?>>> ??? == Generally object allocation should be >>>>>> ??????? detected by >>>>>> ??????? >>? ? ?instrumenting >>>>>> ??????? >>? ? ?>>> ??? == the bytecodes of allocating methods. >>>>>> ??????? >>? ? ?>>> >>>>>> ??????? >>? ? ?>>> ??? == Object allocation generated in native code >>>>>> ??????? by JNI function >>>>>> ??????? >>? ? ?>>> ??? == calls should be detected using >>>>>> ??????? >>? ? ?>>> ??? == JNI >>>>>> function >>>>>> ??????? >>? ? ?>>> interception. >>>>>> ??????? >>? ? ?>>> >>>>>> ??????? >>? ? ?>>> ??? == Some methods might not have associated >>>>>> ??????? bytecodes and >>>>>> ??????? >>? ? ?are not >>>>>> ??????? >>? ? ?>>> ??? == native methods, they instead are executed >>>>>> ??????? directly by the >>>>>> ??????? >>? ? ?>>> ??? == VM. These methods should send this event. >>>>>> ??????? >>? ? ?>>> >>>>>> ??????? >>? ? ?>>> ??? == Virtual machines which are incapable of >>>>>> ??????? bytecode >>>>>> ??????? >>? ? ?instrumentation >>>>>> ??????? >>? ? ?>>> ??? == for some or all of their methods can send >>>>>> ??????? this event. >>>>>> ??????? >>? ? ?>>> >>>>>> ??????? >>? ? ?>>> ??? *++ Note that the >>>>> ??????? >>? ? ?>>> >>>>>> id="SampledObjectAlloc">SampledObjectAlloc** >>>>>> ??????? >>? ? ?>>> ??? **++ event is conditionally triggered on all >>>>>> ??????? Java object >>>>>> ??????? >>? ? ?>>> ??? allocations, including those** >>>>>> ??????? >>? ? ?>>> ??? **++ caused by bytecode method execution, JNI >>>>>> ??????? method >>>>>> ??????? >>? ? ?execution, >>>>>> ??????? >>? ? ?>>> ??? and directly by VM methods.** >>>>>> ??????? >>? ? ?>>> ??? * >>>>>> ??????? >>? ? ?>>> >>>>>> ??????? >>? ? ?>>> ??? Maybe, just adding the last statement >>>>>> would be >>>>>> ??????? good enough. >>>>>> ??????? >>? ? ?>>> >>>>>> ??????? >>? ? ?>>> ??? Thanks, >>>>>> ??????? >>? ? ?>>> ??? Serguei >>>>>> ??????? >>? ? ?>>> >>>>>> ??????? >>? ? ?>>> >>>>>> ??????? >>? ? ?>>> ??? On 6/18/18 21:36, David Holmes wrote: >>>>>> ??????? >>? ? ?>>>> ??? On 19/06/2018 4:50 AM, Jeremy Manson wrote: >>>>>> ??????? >>? ? ?>>>>> ??? Yup!? The paragraph meanders a bit. How >>>>>> ??????? about something >>>>>> ??????? >>? ? ?like: >>>>>> ??????? >>? ? ?>>>> >>>>>> ??????? >>? ? ?>>>> ??? I'm not sure some of the change quite works. >>>>>> ??????? The original >>>>>> ??????? >>? ? ?text >>>>>> ??????? >>? ? ?>>>> ??? considers there to be three kinds of methods >>>>>> ??????? that can cause >>>>>> ??????? >>? ? ?>>>> ??? allocation when executed: >>>>>> ??????? >>? ? ?>>>> ??? - Java (bytecode) methods >>>>>> ??????? >>? ? ?>>>> ??? - JNI methods >>>>>> ??????? >>? ? ?>>>> ??? - VM methods >>>>>> ??????? >>? ? ?>>>> >>>>>> ??????? >>? ? ?>>>> ??? but you've turned this into three kinds of >>>>>> ??????? allocation: via >>>>>> ??????? >>? ? ?>>>> ??? bytecode, via JNI, and via the VM. You then >>>>>> ??????? refer to >>>>>> ??????? >>? ? ?"triggering" >>>>>> ??????? >>? ? ?>>>> ??? an allocation when we tend to use triggering >>>>>> ??????? for events. >>>>>> ??????? >>? ? ?You also >>>>>> ??????? >>? ? ?>>>> ??? refer to an allocation being "executed >>>>>> ??????? directly by the VM" (a >>>>>> ??????? >>? ? ?>>>> ??? phrase previously applied when the >>>>>> subject was a >>>>>> ??????? >>? ? ?'method') - but >>>>>> ??????? >>? ? ?>>>> ??? you don't really execute allocations. >>>>>> ??????? >>? ? ?>>>> >>>>>> ??????? >>? ? ?>>>> ??? IIUC the problem with the existing text is >>>>>> ??????? just that it >>>>>> ??????? >>? ? ?considers >>>>>> ??????? >>? ? ?>>>> ??? VM allocation events as being undetectable >>>>>> ??????? other than by >>>>>> ??????? >>? ? ?this "VM >>>>>> ??????? >>? ? ?>>>> ??? object allocation event" - but that's no >>>>>> ??????? longer true. So how >>>>>> ??????? >>? ? ?>>>> ??? about something minimally changed like this: >>>>>> ??????? >>? ? ?>>>> >>>>>> ??????? >>? ? ?>>>> ??? --- >>>>>> ??????? >>? ? ?>>>> ??? ? Sent when a method causes the virtual >>>>>> ??????? machine to directly >>>>>> ??????? >>? ? ?>>>> ??? allocate an >>>>>> ??????? >>? ? ?>>>> ??? ? Object visible to Java programming >>>>>> language >>>>>> ??????? code. >>>>>> ??????? >>? ? ?>>>> ??? ? Generally object allocation can be >>>>>> detected by >>>>>> ??????? >>? ? ?instrumenting >>>>>> ??????? >>? ? ?>>>> ??? ? the bytecodes of allocating methods. >>>>>> ??????? >>? ? ?>>>> ??? ? Object allocation generated in native code >>>>>> ??????? by JNI function >>>>>> ??????? >>? ? ?>>>> ??? ? calls can be detected using >>>>>> ??????? >>? ? ?>>>> JNI function >>>>>> ??????? >>? ? ?>>>> interception. >>>>>> ??????? >>? ? ?>>>> ??? ?? Some methods might not have associated >>>>>> ??????? bytecodes and >>>>>> ??????? >>? ? ?are not >>>>>> ??????? >>? ? ?>>>> ??? ?? native methods, they instead are executed >>>>>> ??????? directly by the >>>>>> ??????? >>? ? ?>>>> ??? ?? VM. These methods should send this event. >>>>>> ??????? >>? ? ?>>>> ??? ?? Virtual machines which are incapable of >>>>>> ??????? bytecode >>>>>> ??????? >>? ? ?>>>> instrumentation >>>>>> ??????? >>? ? ?>>>> ??? ?? for some or all of their methods can send >>>>>> ??????? this event. >>>>>> ??????? >>? ? ?>>>> >>>>>> ??????? >>? ? ?>>>> ??? ?? Note that the >>>>> ??????? >>? ? ?>>>> >>>>>> id="SampledObjectAlloc">SampledObjectAlloc >>>>>> ??????? >>? ? ?>>>> ??? ?? event is triggered on all Java object >>>>>> ??????? allocations, >>>>>> ??????? >>? ? ?including >>>>>> ??????? >>? ? ?>>>> ??? those >>>>>> ??????? >>? ? ?>>>> ??? ?? caused by bytecode method execution, JNI >>>>>> ??????? method >>>>>> ??????? >>? ? ?execution, and >>>>>> ??????? >>? ? ?>>>> ??? ?? directly by VM methods. >>>>>> ??????? >>? ? ?>>>> ??? --- >>>>>> ??????? >>? ? ?>>>> >>>>>> ??????? >>? ? ?>>>> ??? Thanks, >>>>>> ??????? >>? ? ?>>>> ??? David >>>>>> ??????? >>? ? ?>>>> >>>>>> ??????? >>? ? ?>>>>> ??? Sent when the virtual machine allocates an >>>>>> ??????? >>? ? ?>>>>> ??? Object visible to Java programming language >>>>>> ??????? code without >>>>>> ??????? >>? ? ?using a >>>>>> ??????? >>? ? ?>>>>> new bytecode variant or a JNI >>>>>> method. >>>>>> ??????? >>? ? ?>>>>> ??? Many approaches to tracking object >>>>>> ??????? allocation use a >>>>>> ??????? >>? ? ?>>>>> combination of >>>>>> ??????? >>? ? ?>>>>> bytecode-based instrumentation and >>>>>> >>>>> ??????? >>? ? ?>>>>> id="jniIntercept">JNI function >>>>>> ??????? >>? ? ?>>>>> interception to intercept >>>>>> ??????? allocations. >>>>>> ??????? >>? ? ?However, >>>>>> ??????? >>? ? ?>>>>> ??? this >>>>>> ??????? >>? ? ?>>>>> ??? approach can leave a number of allocations >>>>>> ??????? undetected. >>>>>> ??????? >>? ? ?>>>>> Allocations that are neither >>>>>> ??????? >>? ? ?>>>>> ??? triggered by bytecode nor JNI are executed >>>>>> ??????? directly by >>>>>> ??????? >>? ? ?the VM. >>>>>> ??????? >>? ? ?>>>>> ??? When those allocations occur, the VM should >>>>>> ??????? send this event. >>>>>> ??????? >>? ? ?>>>>> ??? Virtual machines that are incapable of >>>>>> bytecode >>>>>> ??????? >>? ? ?instrumentation >>>>>> ??????? >>? ? ?>>>>> ??? for some or all of their methods may also >>>>>> ??????? send this event. >>>>>> ??????? >>? ? ?>>>>> ???

        >>>>>> ??????? >>? ? ?>>>>> ??? Note that the >>>>> ??????? >>? ? ?>>>>> >>>>>> id="SampledObjectAlloc">SampledObjectAlloc >>>>>> ??????? >>? ? ?>>>>> ??? event is triggered on all Java object >>>>>> ??????? allocations, including >>>>>> ??????? >>? ? ?>>>>> ??? those triggered by bytecode, >>>>>> ??????? >>? ? ?>>>>> ??? JNI methods, and VM events. >>>>>> ??????? >>? ? ?>>>>> >>>>>> ??????? >>? ? ?>>>>> >>>>>> ??????? >>? ? ?>>>>> >>>>>> ??????? >>? ? ?>>>>> ??? On Mon, Jun 18, 2018 at 12:57 AM David >>>>>> Holmes >>>>>> ??????? >>? ? ?>>>>> >>>>> ??????? >>>>>> ??????? >>? ? ?>>>>> ??????? > >>>>>> ??????? >>>>> >>>>>> ??????? >>? ? ?>>>>> ??????? >> >>>>>> ??????? >>? ? ?>>>>> >>>>> ??????? >>>>>> ??????? >>? ? ?>>>>> ??????? >> >>>>>> ??????? >>? ? ?>>>>> >>>>> ??????? >>>>>> ??????? >>? ? ?>>>>> >>> wrote: >>>>>> ??????? >>? ? ?>>>>> >>>>>> ??????? >>? ? ?>>>>> ??? ??? On 18/06/2018 5:01 PM, Jeremy Manson >>>>>> wrote: >>>>>> ??????? >>? ? ?>>>>> ??? ???? > We haven't changed when a VM issues >>>>>> ??????? "VM object >>>>>> ??????? >>? ? ?>>>>> allocation" events. >>>>>> ??????? >>? ? ?>>>>> ??? ???? > >>>>>> ??????? >>? ? ?>>>>> ??? ???? > There were references in the docs >>>>>> to a >>>>>> ??????? >>? ? ?requirement to use >>>>>> ??????? >>? ? ?>>>>> ??? bytecode >>>>>> ??????? >>? ? ?>>>>> ??? ???? > rewriting and JNI interception to >>>>>> track >>>>>> ??????? >>? ? ?allocations.? With >>>>>> ??????? >>? ? ?>>>>> ??? ???? > SampledObjectAlloc, this is no >>>>>> longer >>>>>> ??????? the case - >>>>>> ??????? >>? ? ?>>>>> SampledObjectAlloc can >>>>>> ??????? >>? ? ?>>>>> ??? ???? > track them.? This change is supposed >>>>>> ??????? to remove the >>>>>> ??????? >>? ? ?>>>>> ??? references to >>>>>> ??????? >>? ? ?>>>>> ??? ??? those >>>>>> ??????? >>? ? ?>>>>> ??? ???? > requirements, and provide suitable >>>>>> ??????? replacement text. >>>>>> ??????? >>? ? ?>>>>> ??? ???? > >>>>>> ??????? >>? ? ?>>>>> ??? ???? > VM object alloc has specific >>>>>> language >>>>>> ??????? about being >>>>>> ??????? >>? ? ?able to >>>>>> ??????? >>? ? ?>>>>> ??? use it to >>>>>> ??????? >>? ? ?>>>>> ??? ???? > track allocations that cannot be >>>>>> ??????? tracked with >>>>>> ??????? >>? ? ?bytecode >>>>>> ??????? >>? ? ?>>>>> instrumentation >>>>>> ??????? >>? ? ?>>>>> ??? ???? > and JNI interception.? My goal in >>>>>> ??????? rephrasing was >>>>>> ??????? >>? ? ?to make >>>>>> ??????? >>? ? ?>>>>> ??? it clear >>>>>> ??????? >>? ? ?>>>>> ??? ??? that, >>>>>> ??????? >>? ? ?>>>>> ??? ???? > while you can still use it for this, >>>>>> ??????? you can also >>>>>> ??????? >>? ? ?just use >>>>>> ??????? >>? ? ?>>>>> ??? ???? > SampledObjectAlloc for everything. >>>>>> ??????? >>? ? ?>>>>> >>>>>> ??????? >>? ? ?>>>>> ??? ??? Okay. That doesn't come across clearly >>>>>> ??????? to me - >>>>>> ??????? >>? ? ?sorry. So you >>>>>> ??????? >>? ? ?>>>>> ??? will now >>>>>> ??????? >>? ? ?>>>>> ??? ??? get both kinds of events for >>>>>> allocations >>>>>> ??????? done in the VM? >>>>>> ??????? >>? ? ?>>>>> >>>>>> ??????? >>? ? ?>>>>> Thanks, >>>>>> ??????? >>? ? ?>>>>> ??? ??? David >>>>>> ??????? >>? ? ?>>>>> >>>>>> ??????? >>? ? ?>>>>> >>>>>> ??????? >>? ? ?>>>>> ??? ???? > Jeremy >>>>>> ??????? >>? ? ?>>>>> ??? ???? > >>>>>> ??????? >>? ? ?>>>>> ??? ???? > On Sun, Jun 17, 2018 at 9:11 PM >>>>>> David >>>>>> ??????? Holmes >>>>>> ??????? >>? ? ?>>>>> >>>>> ??????? >>>>>> ??????? >>? ? ?>>>>> ??????? > >>>>>> ??????? >>>>> >>>>>> ??????? >>? ? ?>>>>> ??????? >> >>>>>> ??????? >>? ? ?>>>>> >>>>> ??????? >>>>>> ??????? >>? ? ?>>>>> ??????? >> >>>>>> ??????? >>>>> >>>>>> ??????? >>? ? ?>>>>> ??????? >> >>>>>> ??????? >>? ? ?>>>>> ??? ???? > >>>>> ??????? >>>>>> ??????? >>? ? ?>>>>> ??????? > >>>>>> ??????? >>? ? ?>>>>> >>>>> ??????? >>>>>> ??????? >>? ? ?>>>>> ??????? >> >>>>>> ??????? >>? ? ?>>>>> >>>>> ??????? >>>>>> ??????? >>? ? ?>>>>> >>>> wrote: >>>>>> ??????? >>? ? ?>>>>> ??? ???? > >>>>>> ??????? >>? ? ?>>>>> ??? ???? > ?Hi Jeremy, >>>>>> ??????? >>? ? ?>>>>> ??? ???? > >>>>>> ??????? >>? ? ?>>>>> ??? ???? > ?On 16/06/2018 2:33 AM, Jeremy >>>>>> ??????? Manson wrote: >>>>>> ??????? >>? ? ?>>>>> ??? ???? > > Hi all, >>>>>> ??????? >>? ? ?>>>>> ??? ???? > > >>>>>> ??????? >>? ? ?>>>>> ??? ???? > > There are a number of >>>>>> references in the >>>>>> ??????? >>? ? ?JVMTI doc >>>>>> ??????? >>? ? ?>>>>> ??? to its >>>>>> ??????? >>? ? ?>>>>> ??? ??? not doing >>>>>> ??????? >>? ? ?>>>>> ??? ???? > > object allocation tracking.? Now >>>>>> ??????? that JEP >>>>>> ??????? >>? ? ?331 has >>>>>> ??????? >>? ? ?>>>>> ??? landed, >>>>>> ??????? >>? ? ?>>>>> ??? ??? these >>>>>> ??????? >>? ? ?>>>>> ??? ???? > > references are obsolete.? This >>>>>> patch >>>>>> ??????? >>? ? ?changes those >>>>>> ??????? >>? ? ?>>>>> ??? references >>>>>> ??????? >>? ? ?>>>>> ??? ???? > ?accordingly. >>>>>> ??????? >>? ? ?>>>>> ??? ???? > > >>>>>> ??????? >>? ? ?>>>>> ??? ???? > > While I was there, I took the >>>>>> ??????? liberty of >>>>>> ??????? >>? ? ?fixing >>>>>> ??????? >>? ? ?>>>>> some >>>>>> ??????? >>? ? ?>>>>> spelling?errors. >>>>>> ??????? >>? ? ?>>>>> ??? ???? > > >>>>>> ??????? >>? ? ?>>>>> ??? ???? > > As far as I know, these are >>>>>> ??????? non-normative >>>>>> ??????? >>? ? ?>>>>> changes and >>>>>> ??????? >>? ? ?>>>>> ??? ??? modify no >>>>>> ??????? >>? ? ?>>>>> ??? ???? > ?API, so >>>>>> ??????? >>? ? ?>>>>> ??? ???? > > they should not require a CSR. >>>>>> ??????? >>? ? ?>>>>> ??? ???? > >>>>>> ??????? >>? ? ?>>>>> ??? ???? > ?I'm unclear on the nature of the >>>>>> ??????? change to >>>>>> ??????? >>? ? ?"VM Object >>>>>> ??????? >>? ? ?>>>>> Allocation". Does >>>>>> ??????? >>? ? ?>>>>> ??? ???? > ?the existence of >>>>>> ??????? SampledObjectAlloc change >>>>>> ??????? >>? ? ?when a VM >>>>>> ??????? >>? ? ?>>>>> ??? should >>>>>> ??????? >>? ? ?>>>>> ??? ??? issue "VM >>>>>> ??????? >>? ? ?>>>>> ??? ???? > ?object allocation" events? >>>>>> ??????? >>? ? ?>>>>> ??? ???? > >>>>>> ??????? >>? ? ?>>>>> ??? ???? > ?Thanks, >>>>>> ??????? >>? ? ?>>>>> ??? ???? > ?David >>>>>> ??????? >>? ? ?>>>>> ??? ???? > >>>>>> ??????? >>? ? ?>>>>> ??? ???? > > >>>>>> ??????? >>? ? ?>>>>> ??? ???? > > Bug: >>>>>> ??????? >>? ? ?>>>>> ??? ???? > > >>>>>> ??????? >> https://bugs.openjdk.java.net/browse/JDK-8205113 >>>>>> ??????? >>? ? ?>>>>> ??? ???? > > >>>>>> ??????? >>? ? ?>>>>> ??? ???? > > Webrev: >>>>>> ??????? >>? ? ?>>>>> ??? ???? > > >>>>>> ??????? >>? ? ?>>>>> >>>>>> http://cr.openjdk.java.net/~jmanson/8205113/webrev.00/ >>>>>> >>>>>> ??????? >> >>>>>> ? >>>>>> ??????? >>? ? ?>>>>> >>>>>> >>>>>> ??????? >>? ? ?>>>>> ??? ???? > > >>>>>> ??????? >>? ? ?>>>>> ??? ???? > > Thanks! >>>>>> ??????? >>? ? ?>>>>> ??? ???? > > >>>>>> ??????? >>? ? ?>>>>> ??? ???? > > Jeremy >>>>>> ??????? >>? ? ?>>>>> ??? ???? > >>>>>> ??????? >>? ? ?>>>>> >>>>>> ??????? >>? ? ?>>> >>>>>> ??????? >>? ? ?>> >>>>>> ??????? >> >>>>>> ??????? > >>>>>> >>>>> >>>> >> From chris.plummer at oracle.com Sun Jun 24 00:06:33 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Sat, 23 Jun 2018 17:06:33 -0700 Subject: RFR(XS): 8205561: problem list vmTestbase/nsk/jdb/trace/trace001/trace001.java In-Reply-To: <9ab73ed0-1715-65b6-9e3e-5caefa1bedb2@oracle.com> References: <12bed6fe-c1e8-1ec3-95a8-7f80aae887dc@oracle.com> <9ab73ed0-1715-65b6-9e3e-5caefa1bedb2@oracle.com> Message-ID: <32972335-5d19-52cf-8a29-ecb0cd9146af@oracle.com> Yes, I wasn't planning on pushing this change since Peter offered up a fix. thanks, Chris On 6/23/18 4:10 PM, David Holmes wrote: > Hi Chris, > > Peter (cc'd) has an RFR out to fix 8205540 now - which I just reviewed. > > Thanks, > David > > On 23/06/2018 3:42 PM, Chris Plummer wrote: >> Hello, >> >> Please review the following: >> >> https://bugs.openjdk.java.net/browse/JDK-8205561 >> >> diff --git a/test/hotspot/jtreg/ProblemList.txt >> b/test/hotspot/jtreg/ProblemList.txt >> --- a/test/hotspot/jtreg/ProblemList.txt >> +++ b/test/hotspot/jtreg/ProblemList.txt >> @@ -166,7 +166,7 @@ >> ??vmTestbase/vm/mlvm/indy/func/jvmti/mergeCP_indy2manySame_b/TestDescription.java >> 8013267 generic-all >> >> ??vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java 8197938 >> windows-all >> - >> +vmTestbase/nsk/jdb/trace/trace001/trace001.java 8205540 generic-all >> ??vmTestbase/nsk/jdwp/ThreadReference/ForceEarlyReturn/forceEarlyReturn001/forceEarlyReturn001.java >> 7199837 generic-all >> >> ??############################################################################# >> >> >> thanks, >> >> Chris >> >> From serguei.spitsyn at oracle.com Sun Jun 24 05:55:40 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Sat, 23 Jun 2018 22:55:40 -0700 Subject: RFR 8205113: Update JVMTI doc references to object allocation tracking In-Reply-To: <51b86cf0-0287-4246-fe1d-f7f536ca3140@oracle.com> References: <6927237d-ce0b-17c0-68f8-06d05b84e516@oracle.com> <53f576e2-0adb-a6c5-8a3c-026dcb1aa7ad@oracle.com> <8224afa2-a6cb-fee2-8fa3-184828ac7c50@oracle.com> <60899f09-12df-1e6e-7e16-406a60939b7c@oracle.com> <96713bc3-9350-fd4a-75e5-2c36257066b2@oracle.com> <3018f21d-52b0-9ec4-151e-e4fb01da10ff@oracle.com> <3ffcc780-2568-ec21-b503-228c91d18969@oracle.com> <845054d9-ec16-e2f8-afec-c885127bbd04@oracle.com> <51b86cf0-0287-4246-fe1d-f7f536ca3140@oracle.com> Message-ID: On 6/23/18 16:19, serguei.spitsyn at oracle.com wrote: > > On 6/23/18 15:53, David Holmes wrote: >> On 24/06/2018 7:58 AM, serguei.spitsyn at oracle.com wrote: >>> Hi David, >>> >>> This was your suggestion: >> >> There was only one part of the patch being debated! The rest of the >> patch was still applicable. Jeremy specifically stated >> >> "That would be okay with me, assuming that my other corrections are >> made.? I'd also like to fix the spelling of instrumentation in the >> first sentence." > > Yes, I missed the other part - sorry. :( > Will push it under a separate bug id. Pushed the fix for typos under: ? 8205570 fix a number of typos in the JVMTI spec8 Thanks, Serguei > Not sure if another review is needed. > > Thanks, > Serguei > >> David >> >>> --- >>> ?? Sent when a method causes the virtual machine to directly >>> allocate an >>> ?? Object visible to Java programming language code. >>> ?? Generally object allocation can be detected by instrumenting >>> ?? the bytecodes of allocating methods. >>> ?? Object allocation generated in native code by JNI function >>> ?? calls can be detected using >>> ?? JNI function >>> interception. >>> ??? Some methods might not have associated bytecodes and are not >>> ??? native methods, they instead are executed directly by the >>> ??? VM. These methods should send this event. >>> ??? Virtual machines which are incapable of bytecode instrumentation >>> ??? for some or all of their methods can send this event. >>> >>> ??? Note that the >> id="SampledObjectAlloc">SampledObjectAlloc >>> ??? event is triggered on all Java object allocations, including those >>> ??? caused by bytecode method execution, JNI method execution, and >>> ??? directly by VM methods. >>> --- >>> >>> I've pushed the patch: >>> >>> diff -r f703d45c5687 src/hotspot/share/prims/jvmti.xml >>> --- a/src/hotspot/share/prims/jvmti.xml??? Tue Jun 05 11:55:39 2018 >>> +0200 >>> +++ b/src/hotspot/share/prims/jvmti.xml??? Sat Jun 23 01:18:57 2018 >>> -0700 >>> @@ -13507,9 +13507,8 @@ >>> ??? >> ????? ?id="VMObjectAlloc" const="JVMTI_EVENT_VM_OBJECT_ALLOC" num="84"> >>> ????? >>> -????? Sent when a method causes the virtual machine to allocate an >>> -????? Object visible to Java programming language code and the >>> -????? allocation is not detectable by other intrumentation mechanisms. >>> +????? Sent when a method causes the virtual machine to directly >>> allocate an >>> +????? Object visible to Java programming language code. >>> ??????? Generally object allocation should be detected by instrumenting >>> ??????? the bytecodes of allocating methods. >>> ??????? Object allocation generated in native code by JNI function >>> @@ -13520,6 +13519,12 @@ >>> ??????? VM. These methods should send this event. >>> ??????? Virtual machines which are incapable of bytecode >>> instrumentation >>> ??????? for some or all of their methods can send this event. >>> + >>> +????? Note that the >> + id="SampledObjectAlloc">SampledObjectAlloc >>> +????? event is triggered on all Java object allocations, including >>> those >>> +????? caused by bytecode method execution, JNI method execution, and >>> +????? directly by VM methods. >>> ???????

        >>> ??????? Typical examples where this event might be sent: >>> ???????

          >>> >>> >>> Thanks, >>> Serguei >>> >>> >>> On 6/23/18 04:49, David Holmes wrote: >>>> On 23/06/2018 6:25 PM, serguei.spitsyn at oracle.com wrote: >>>>> I've pushed the version suggested by David. >>>> >>>> But you left out all of Jeremy's other fixups! >>>> >>>> David >>>> >>>>> Thanks, >>>>> serguei >>>>> >>>>> >>>>> On 6/22/18 09:00, serguei.spitsyn at oracle.com wrote: >>>>>> Hi Jeremy, >>>>>> >>>>>> Okay, let me look at it once more before making final decision. >>>>>> We have all suggestions and preferences listed. >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>>> >>>>>> On 6/22/18 08:22, Jeremy Manson wrote: >>>>>>> Hey folks - >>>>>>> >>>>>>> With the window closing soon for 11, I feel that we should get >>>>>>> this revision in (just so that the spec is clear).? That said, I >>>>>>> don't feel strongly about the wording.? Who gets to make the >>>>>>> decision? >>>>>>> >>>>>>> Jeremy >>>>>>> >>>>>>> On Wed, Jun 20, 2018 at 9:41 AM Jeremy Manson >>>>>>> > wrote: >>>>>>> >>>>>>> ??? FWIW, I'm also okay with David's version, with the caveat that >>>>>>> ??? "These methods should send this event" should probably read >>>>>>> ??? "Invocations of these methods should trigger this event". >>>>>>> >>>>>>> ??? Jeremy >>>>>>> >>>>>>> ??? On Wed, Jun 20, 2018 at 4:11 AM David Holmes >>>>>>> ??? > >>>>>>> wrote: >>>>>>> >>>>>>> ??????? On 20/06/2018 4:48 PM, serguei.spitsyn at oracle.com >>>>>>> ??????? wrote: >>>>>>> ??????? > On 6/19/18 23:29, Jeremy Manson wrote: >>>>>>> ??????? >> Maybe we should make that clarification. >>>>>>> ??????? >> >>>>>>> ??????? >> Also, the reason I danced around that in my revision >>>>>>> is that >>>>>>> ??????? > >>>>>>> ??????? > Understand that. >>>>>>> ??????? > But it is not a good style to clarify about >>>>>>> ??????? SampledObjectAlloc in the >>>>>>> ??????? > spec of VMObjectAlloc. >>>>>>> ??????? > Would be nice to find a way to keep it simple. >>>>>>> ??????? > >>>>>>> ??????? > We could keep the original spec as it is but add that all >>>>>>> ??????? this does not >>>>>>> ??????? > apply to the SampledObjectAlloc events. >>>>>>> >>>>>>> ??????? I don't think that really presents a coherent spec. I still >>>>>>> ??????? like my >>>>>>> ??????? suggestion :) >>>>>>> >>>>>>> ??????? David >>>>>>> >>>>>>> ??????? > >>>>>>> ??????? > Something like below: >>>>>>> ??????? > >>>>>>> ??????? >? ? Note, the above does not apply to the >>>>>> ??????? > id="SampledObjectAlloc">SampledObjectAlloc >>>>>>> ??????? >? ? event as it is triggered on all Java object >>>>>>> allocations, >>>>>>> ??????? including >>>>>>> ??????? > those caused by bytecode method execution, >>>>>>> ??????? >? ? JNI method execution, and directly by VM methods.* >>>>>>> ??????? > * >>>>>>> ??????? > >>>>>>> ??????? >> if you set the sampling rate to 0, you get unconditional >>>>>>> ??????? sampling. >>>>>>> ??????? > A correction. >>>>>>> ??????? > >>>>>>> ??????? > I wrote: >>>>>>> ??????? >? > It is still possible to sample every allocation >>>>>>> with the >>>>>>> ??????? SamplingRate >>>>>>> ??????? > == 1. >>>>>>> ??????? > >>>>>>> ??????? > but had write: SamplingRate == 0 >>>>>>> ??????? > >>>>>>> ??????? > Thanks, >>>>>>> ??????? > Serguei >>>>>>> ??????? > >>>>>>> ??????? > >>>>>>> ??????? >> >>>>>>> ??????? >> Jeremy >>>>>>> ??????? >> >>>>>>> ??????? >> On Tue, Jun 19, 2018 at 10:41 PM >>>>>>> ??????? serguei.spitsyn at oracle.com >>>>>>> >>>>>>> ??????? >> >>>>>> ??????? > >>>>>>> ??????? >>>>>> >>>>>>> ??????? >> >>>>>> >> wrote: >>>>>>> ??????? >> >>>>>>> ??????? >>? ? ?On 6/19/18 21:54, David Holmes wrote: >>>>>>> ??????? >>? ? ?> On 20/06/2018 2:41 PM, serguei.spitsyn at oracle.com >>>>>>> ??????? >>>>>>> ??????? >> ?>>>>>> ??????? > wrote: >>>>>>> ??????? >>? ? ?>> On 6/19/18 21:11, Jeremy Manson wrote: >>>>>>> ??????? >>? ? ?>>> That would be okay with me, assuming that my >>>>>>> other >>>>>>> ??????? corrections >>>>>>> ??????? >>? ? ?are >>>>>>> ??????? >>? ? ?>>> made. >>>>>>> ??????? >>? ? ?>> >>>>>>> ??????? >>? ? ?>> Another option would be to say "non-sampling" >>>>>>> ??????? instead of >>>>>>> ??????? >>? ? ?>> "unconditional": >>>>>>> ??????? >>? ? ?>> >>>>>>> ??????? >>? ? ?>> == Sent when a method causes the virtual >>>>>>> machine to >>>>>>> ??????? allocate an >>>>>>> ??????? >>? ? ?>> == Object visible to Java programming language >>>>>>> code >>>>>>> ??????? and the >>>>>>> ??????? >>? ? ?allocation >>>>>>> ??????? >>? ? ?>> += is not detectable by other *non-sampling* >>>>>>> ??????? instrumentation >>>>>>> ??????? >>? ? ?mechanisms. >>>>>>> ??????? >>? ? ?>> >>>>>>> ??????? >>? ? ?>> >>>>>>> ??????? >>? ? ?>>> I'd also like to fix the spelling of >>>>>>> ??????? instrumentation in the first >>>>>>> ??????? >>? ? ?>>> sentence. >>>>>>> ??????? >>? ? ?>> >>>>>>> ??????? >>? ? ?>> Yes, of course. >>>>>>> ??????? >>? ? ?>> >>>>>>> ??????? >>? ? ?>> Let's wait for David's opinion. >>>>>>> ??????? >>? ? ?> >>>>>>> ??????? >>? ? ?> I'm okay with Serguei's suggestion (combined with >>>>>>> ??????? Jeremy's other >>>>>>> ??????? >>? ? ?> changes). >>>>>>> ??????? >>? ? ?> >>>>>>> ??????? >>? ? ?> I'm not that familiar with conditional versus >>>>>>> ??????? unconditional >>>>>>> ??????? >>? ? ?> instrumentation. >>>>>>> ??????? >> >>>>>>> ??????? >>? ? ?The whole point of the SampledObjectAlloc events >>>>>>> is to >>>>>>> ??????? post them >>>>>>> ??????? >>? ? ?conditionally >>>>>>> ??????? >>? ? ?depending on the SamplingRate so that just some >>>>>>> amount of >>>>>>> ??????? >>? ? ?allocations is >>>>>>> ??????? >>? ? ?sampled. >>>>>>> ??????? >>? ? ?It is why the overhead can be minimal (less than 5 >>>>>>> ??????? percents). >>>>>>> ??????? >>? ? ?It is still possible to sample every allocation >>>>>>> with the >>>>>>> ??????? >>? ? ?SamplingRate == 1. >>>>>>> ??????? >> >>>>>>> ??????? >>? ? ?The VMObjectAlloc events are posted unconditionally, >>>>>>> ??????? so every VM >>>>>>> ??????? >>? ? ?allocation is posted. >>>>>>> ??????? >> >>>>>>> ??????? >>? ? ?Thanks, >>>>>>> ??????? >>? ? ?Serguei >>>>>>> ??????? >> >>>>>>> ??????? >> >>>>>>> ??????? >>? ? ?> >>>>>>> ??????? >>? ? ?> Thanks, >>>>>>> ??????? >>? ? ?> David >>>>>>> ??????? >>? ? ?> >>>>>>> ??????? >>? ? ?>> >>>>>>> ??????? >>? ? ?>> Thanks, >>>>>>> ??????? >>? ? ?>> Serguei >>>>>>> ??????? >>? ? ?>> >>>>>>> ??????? >>? ? ?>> >>>>>>> ??????? >>? ? ?>>> >>>>>>> ??????? >>? ? ?>>> Jeremy >>>>>>> ??????? >>? ? ?>>> >>>>>>> ??????? >>? ? ?>>> On Mon, Jun 18, 2018 at 11:01 PM >>>>>>> ??????? serguei.spitsyn at oracle.com >>>>>>> >>>>>>> ??????? >> ?>>>>>> ??????? > >>>>>>> ??????? >>? ? ?>>> >>>>>> ??????? >>>>>>> ??????? >> ?>>>>>> >> >>>>>>> ??????? >>>>>> >>>>>>> ??????? >> ?>>>>>> ??????? > >>>>>>> ??????? >>? ? ?>>> >>>>>> ??????? >>>>>>> ??????? >> ?>>>>>> >>> wrote: >>>>>>> ??????? >>? ? ?>>> >>>>>>> ??????? >>? ? ?>>> ??? Hi Jeremy and David, >>>>>>> ??????? >>? ? ?>>> >>>>>>> ??????? >>? ? ?>>> ??? Sorry for being late to the party. >>>>>>> ??????? >>? ? ?>>> >>>>>>> ??????? >>? ? ?>>> ??? I'm also concerned about the Jeremy's spec >>>>>>> ??????? update is more >>>>>>> ??????? >>? ? ?>>> ??? intrusive than necessary. >>>>>>> ??????? >>? ? ?>>> ??? One specifics of the new SampledObjectAlloc >>>>>>> ??????? event is that >>>>>>> ??????? >>? ? ?it is >>>>>>> ??????? >>? ? ?>>> ??? posted conditionally. >>>>>>> ??????? >>? ? ?>>> ??? So, it is not fully comparable with the >>>>>>> ??????? VMObjectAlloc >>>>>>> ??????? >>? ? ?event and >>>>>>> ??????? >>? ? ?>>> ??? can not replace it in any way. >>>>>>> ??????? >>? ? ?>>> ??? I'm even not yet convinced that any spec >>>>>>> ??????? update is necessary. >>>>>>> ??????? >>? ? ?>>> >>>>>>> ??????? >>? ? ?>>> ??? However, something like below would look >>>>>>> ??????? minimal and >>>>>>> ??????? >>? ? ?reasonable: >>>>>>> ??????? >>? ? ?>>> >>>>>>> ??????? >>? ? ?>>> ??? == Sent when a method causes the virtual >>>>>>> ??????? machine to >>>>>>> ??????? >>? ? ?allocate an >>>>>>> ??????? >>? ? ?>>> ??? == Object visible to Java programming >>>>>>> language >>>>>>> ??????? code and the >>>>>>> ??????? >>? ? ?>>> allocation >>>>>>> ??????? >>? ? ?>>> ??? += is not detectable by other >>>>>>> *unconditional* >>>>>>> ??????? intrumentation >>>>>>> ??????? >>? ? ?>>> ??? mechanisms. >>>>>>> ??????? >>? ? ?>>> >>>>>>> ??????? >>? ? ?>>> ??? == Generally object allocation should be >>>>>>> ??????? detected by >>>>>>> ??????? >>? ? ?instrumenting >>>>>>> ??????? >>? ? ?>>> ??? == the bytecodes of allocating methods. >>>>>>> ??????? >>? ? ?>>> >>>>>>> ??????? >>? ? ?>>> ??? == Object allocation generated in native >>>>>>> code >>>>>>> ??????? by JNI function >>>>>>> ??????? >>? ? ?>>> ??? == calls should be detected using >>>>>>> ??????? >>? ? ?>>> ??? == JNI >>>>>>> function >>>>>>> ??????? >>? ? ?>>> interception. >>>>>>> ??????? >>? ? ?>>> >>>>>>> ??????? >>? ? ?>>> ??? == Some methods might not have associated >>>>>>> ??????? bytecodes and >>>>>>> ??????? >>? ? ?are not >>>>>>> ??????? >>? ? ?>>> ??? == native methods, they instead are executed >>>>>>> ??????? directly by the >>>>>>> ??????? >>? ? ?>>> ??? == VM. These methods should send this event. >>>>>>> ??????? >>? ? ?>>> >>>>>>> ??????? >>? ? ?>>> ??? == Virtual machines which are incapable of >>>>>>> ??????? bytecode >>>>>>> ??????? >>? ? ?instrumentation >>>>>>> ??????? >>? ? ?>>> ??? == for some or all of their methods can send >>>>>>> ??????? this event. >>>>>>> ??????? >>? ? ?>>> >>>>>>> ??????? >>? ? ?>>> ??? *++ Note that the >>>>>> ??????? >>? ? ?>>> >>>>>>> id="SampledObjectAlloc">SampledObjectAlloc** >>>>>>> ??????? >>? ? ?>>> ??? **++ event is conditionally triggered on all >>>>>>> ??????? Java object >>>>>>> ??????? >>? ? ?>>> ??? allocations, including those** >>>>>>> ??????? >>? ? ?>>> ??? **++ caused by bytecode method execution, >>>>>>> JNI >>>>>>> ??????? method >>>>>>> ??????? >>? ? ?execution, >>>>>>> ??????? >>? ? ?>>> ??? and directly by VM methods.** >>>>>>> ??????? >>? ? ?>>> ??? * >>>>>>> ??????? >>? ? ?>>> >>>>>>> ??????? >>? ? ?>>> ??? Maybe, just adding the last statement >>>>>>> would be >>>>>>> ??????? good enough. >>>>>>> ??????? >>? ? ?>>> >>>>>>> ??????? >>? ? ?>>> ??? Thanks, >>>>>>> ??????? >>? ? ?>>> ??? Serguei >>>>>>> ??????? >>? ? ?>>> >>>>>>> ??????? >>? ? ?>>> >>>>>>> ??????? >>? ? ?>>> ??? On 6/18/18 21:36, David Holmes wrote: >>>>>>> ??????? >>? ? ?>>>> ??? On 19/06/2018 4:50 AM, Jeremy Manson wrote: >>>>>>> ??????? >>? ? ?>>>>> ??? Yup! The paragraph meanders a bit. How >>>>>>> ??????? about something >>>>>>> ??????? >>? ? ?like: >>>>>>> ??????? >>? ? ?>>>> >>>>>>> ??????? >>? ? ?>>>> ??? I'm not sure some of the change quite >>>>>>> works. >>>>>>> ??????? The original >>>>>>> ??????? >>? ? ?text >>>>>>> ??????? >>? ? ?>>>> ??? considers there to be three kinds of >>>>>>> methods >>>>>>> ??????? that can cause >>>>>>> ??????? >>? ? ?>>>> ??? allocation when executed: >>>>>>> ??????? >>? ? ?>>>> ??? - Java (bytecode) methods >>>>>>> ??????? >>? ? ?>>>> ??? - JNI methods >>>>>>> ??????? >>? ? ?>>>> ??? - VM methods >>>>>>> ??????? >>? ? ?>>>> >>>>>>> ??????? >>? ? ?>>>> ??? but you've turned this into three kinds of >>>>>>> ??????? allocation: via >>>>>>> ??????? >>? ? ?>>>> ??? bytecode, via JNI, and via the VM. You then >>>>>>> ??????? refer to >>>>>>> ??????? >>? ? ?"triggering" >>>>>>> ??????? >>? ? ?>>>> ??? an allocation when we tend to use >>>>>>> triggering >>>>>>> ??????? for events. >>>>>>> ??????? >>? ? ?You also >>>>>>> ??????? >>? ? ?>>>> ??? refer to an allocation being "executed >>>>>>> ??????? directly by the VM" (a >>>>>>> ??????? >>? ? ?>>>> ??? phrase previously applied when the >>>>>>> subject was a >>>>>>> ??????? >>? ? ?'method') - but >>>>>>> ??????? >>? ? ?>>>> ??? you don't really execute allocations. >>>>>>> ??????? >>? ? ?>>>> >>>>>>> ??????? >>? ? ?>>>> ??? IIUC the problem with the existing text is >>>>>>> ??????? just that it >>>>>>> ??????? >>? ? ?considers >>>>>>> ??????? >>? ? ?>>>> ??? VM allocation events as being undetectable >>>>>>> ??????? other than by >>>>>>> ??????? >>? ? ?this "VM >>>>>>> ??????? >>? ? ?>>>> ??? object allocation event" - but that's no >>>>>>> ??????? longer true. So how >>>>>>> ??????? >>? ? ?>>>> ??? about something minimally changed like >>>>>>> this: >>>>>>> ??????? >>? ? ?>>>> >>>>>>> ??????? >>? ? ?>>>> ??? --- >>>>>>> ??????? >>? ? ?>>>> ??? ? Sent when a method causes the virtual >>>>>>> ??????? machine to directly >>>>>>> ??????? >>? ? ?>>>> ??? allocate an >>>>>>> ??????? >>? ? ?>>>> ??? ? Object visible to Java programming >>>>>>> language >>>>>>> ??????? code. >>>>>>> ??????? >>? ? ?>>>> ??? ? Generally object allocation can be >>>>>>> detected by >>>>>>> ??????? >>? ? ?instrumenting >>>>>>> ??????? >>? ? ?>>>> ??? ? the bytecodes of allocating methods. >>>>>>> ??????? >>? ? ?>>>> ??? ? Object allocation generated in native >>>>>>> code >>>>>>> ??????? by JNI function >>>>>>> ??????? >>? ? ?>>>> ??? ? calls can be detected using >>>>>>> ??????? >>? ? ?>>>> JNI function >>>>>>> ??????? >>? ? ?>>>> interception. >>>>>>> ??????? >>? ? ?>>>> ??? ?? Some methods might not have associated >>>>>>> ??????? bytecodes and >>>>>>> ??????? >>? ? ?are not >>>>>>> ??????? >>? ? ?>>>> ??? ?? native methods, they instead are >>>>>>> executed >>>>>>> ??????? directly by the >>>>>>> ??????? >>? ? ?>>>> ??? ?? VM. These methods should send this >>>>>>> event. >>>>>>> ??????? >>? ? ?>>>> ??? ?? Virtual machines which are incapable of >>>>>>> ??????? bytecode >>>>>>> ??????? >>? ? ?>>>> instrumentation >>>>>>> ??????? >>? ? ?>>>> ??? ?? for some or all of their methods can >>>>>>> send >>>>>>> ??????? this event. >>>>>>> ??????? >>? ? ?>>>> >>>>>>> ??????? >>? ? ?>>>> ??? ?? Note that the >>>>>> ??????? >>? ? ?>>>> >>>>>>> id="SampledObjectAlloc">SampledObjectAlloc >>>>>>> ??????? >>? ? ?>>>> ??? ?? event is triggered on all Java object >>>>>>> ??????? allocations, >>>>>>> ??????? >>? ? ?including >>>>>>> ??????? >>? ? ?>>>> ??? those >>>>>>> ??????? >>? ? ?>>>> ??? ?? caused by bytecode method execution, JNI >>>>>>> ??????? method >>>>>>> ??????? >>? ? ?execution, and >>>>>>> ??????? >>? ? ?>>>> ??? ?? directly by VM methods. >>>>>>> ??????? >>? ? ?>>>> ??? --- >>>>>>> ??????? >>? ? ?>>>> >>>>>>> ??????? >>? ? ?>>>> ??? Thanks, >>>>>>> ??????? >>? ? ?>>>> ??? David >>>>>>> ??????? >>? ? ?>>>> >>>>>>> ??????? >>? ? ?>>>>> ??? Sent when the virtual machine allocates an >>>>>>> ??????? >>? ? ?>>>>> ??? Object visible to Java programming >>>>>>> language >>>>>>> ??????? code without >>>>>>> ??????? >>? ? ?using a >>>>>>> ??????? >>? ? ?>>>>> new bytecode variant or a JNI >>>>>>> method. >>>>>>> ??????? >>? ? ?>>>>> ??? Many approaches to tracking object >>>>>>> ??????? allocation use a >>>>>>> ??????? >>? ? ?>>>>> combination of >>>>>>> ??????? >>? ? ?>>>>> bytecode-based instrumentation and >>>>>>> >>>>>> ??????? >>? ? ?>>>>> id="jniIntercept">JNI function >>>>>>> ??????? >>? ? ?>>>>> interception to intercept >>>>>>> ??????? allocations. >>>>>>> ??????? >>? ? ?However, >>>>>>> ??????? >>? ? ?>>>>> ??? this >>>>>>> ??????? >>? ? ?>>>>> ??? approach can leave a number of allocations >>>>>>> ??????? undetected. >>>>>>> ??????? >>? ? ?>>>>> Allocations that are neither >>>>>>> ??????? >>? ? ?>>>>> triggered by bytecode nor JNI are executed >>>>>>> ??????? directly by >>>>>>> ??????? >>? ? ?the VM. >>>>>>> ??????? >>? ? ?>>>>> ??? When those allocations occur, the VM >>>>>>> should >>>>>>> ??????? send this event. >>>>>>> ??????? >>? ? ?>>>>> ??? Virtual machines that are incapable of >>>>>>> bytecode >>>>>>> ??????? >>? ? ?instrumentation >>>>>>> ??????? >>? ? ?>>>>> ??? for some or all of their methods may also >>>>>>> ??????? send this event. >>>>>>> ??????? >>? ? ?>>>>>

          >>>>>>> ??????? >>? ? ?>>>>> ??? Note that the >>>>>> ??????? >>? ? ?>>>>> >>>>>>> id="SampledObjectAlloc">SampledObjectAlloc >>>>>>> ??????? >>? ? ?>>>>> ??? event is triggered on all Java object >>>>>>> ??????? allocations, including >>>>>>> ??????? >>? ? ?>>>>> ??? those triggered by bytecode, >>>>>>> ??????? >>? ? ?>>>>> ??? JNI methods, and VM events. >>>>>>> ??????? >>? ? ?>>>>> >>>>>>> ??????? >>? ? ?>>>>> >>>>>>> ??????? >>? ? ?>>>>> >>>>>>> ??????? >>? ? ?>>>>> ??? On Mon, Jun 18, 2018 at 12:57 AM David >>>>>>> Holmes >>>>>>> ??????? >>? ? ?>>>>> >>>>>> ??????? >>>>>>> ??????? >> ?>>>>>> ??????? > >>>>>>> ??????? >>>>>> >>>>>>> ??????? >> ?>>>>>> ??????? >> >>>>>>> ??????? >>? ? ?>>>>> >>>>>> ??????? >>>>>>> ??????? >> ?>>>>>> ??????? >> >>>>>>> ??????? >>? ? ?>>>>> >>>>>> ??????? >>>>>>> ??????? >> ?>>>>>> >>> wrote: >>>>>>> ??????? >>? ? ?>>>>> >>>>>>> ??????? >>? ? ?>>>>> ??? ??? On 18/06/2018 5:01 PM, Jeremy >>>>>>> Manson wrote: >>>>>>> ??????? >>? ? ?>>>>> > We haven't changed when a VM issues >>>>>>> ??????? "VM object >>>>>>> ??????? >>? ? ?>>>>> allocation" events. >>>>>>> ??????? >>? ? ?>>>>> > >>>>>>> ??????? >>? ? ?>>>>> > There were references in the docs to a >>>>>>> ??????? >>? ? ?requirement to use >>>>>>> ??????? >>? ? ?>>>>> ??? bytecode >>>>>>> ??????? >>? ? ?>>>>> > rewriting and JNI interception to track >>>>>>> ??????? >>? ? ?allocations.? With >>>>>>> ??????? >>? ? ?>>>>> > SampledObjectAlloc, this is no longer >>>>>>> ??????? the case - >>>>>>> ??????? >>? ? ?>>>>> SampledObjectAlloc can >>>>>>> ??????? >>? ? ?>>>>> > track them.? This change is supposed >>>>>>> ??????? to remove the >>>>>>> ??????? >>? ? ?>>>>> references to >>>>>>> ??????? >>? ? ?>>>>> those >>>>>>> ??????? >>? ? ?>>>>> > requirements, and provide suitable >>>>>>> ??????? replacement text. >>>>>>> ??????? >>? ? ?>>>>> > >>>>>>> ??????? >>? ? ?>>>>> > VM object alloc has specific language >>>>>>> ??????? about being >>>>>>> ??????? >>? ? ?able to >>>>>>> ??????? >>? ? ?>>>>> ??? use it to >>>>>>> ??????? >>? ? ?>>>>> > track allocations that cannot be >>>>>>> ??????? tracked with >>>>>>> ??????? >>? ? ?bytecode >>>>>>> ??????? >>? ? ?>>>>> instrumentation >>>>>>> ??????? >>? ? ?>>>>> > and JNI interception.? My goal in >>>>>>> ??????? rephrasing was >>>>>>> ??????? >>? ? ?to make >>>>>>> ??????? >>? ? ?>>>>> ??? it clear >>>>>>> ??????? >>? ? ?>>>>> that, >>>>>>> ??????? >>? ? ?>>>>> > while you can still use it for this, >>>>>>> ??????? you can also >>>>>>> ??????? >>? ? ?just use >>>>>>> ??????? >>? ? ?>>>>> > SampledObjectAlloc for everything. >>>>>>> ??????? >>? ? ?>>>>> >>>>>>> ??????? >>? ? ?>>>>> Okay. That doesn't come across clearly >>>>>>> ??????? to me - >>>>>>> ??????? >>? ? ?sorry. So you >>>>>>> ??????? >>? ? ?>>>>> ??? will now >>>>>>> ??????? >>? ? ?>>>>> ??? ??? get both kinds of events for >>>>>>> allocations >>>>>>> ??????? done in the VM? >>>>>>> ??????? >>? ? ?>>>>> >>>>>>> ??????? >>? ? ?>>>>> Thanks, >>>>>>> ??????? >>? ? ?>>>>> David >>>>>>> ??????? >>? ? ?>>>>> >>>>>>> ??????? >>? ? ?>>>>> >>>>>>> ??????? >>? ? ?>>>>> > Jeremy >>>>>>> ??????? >>? ? ?>>>>> > >>>>>>> ??????? >>? ? ?>>>>> > On Sun, Jun 17, 2018 at 9:11 PM David >>>>>>> ??????? Holmes >>>>>>> ??????? >>? ? ?>>>>> >>>>>> ??????? >>>>>>> ??????? >> ?>>>>>> ??????? > >>>>>>> ??????? >>>>>> >>>>>>> ??????? >> ?>>>>>> ??????? >> >>>>>>> ??????? >>? ? ?>>>>> >>>>>> ??????? >>>>>>> ??????? >> ?>>>>>> ??????? >> >>>>>>> ??????? >>>>>> >>>>>>> ??????? >> ?>>>>>> ??????? >> >>>>>>> ??????? >>? ? ?>>>>> > >>>>>> ??????? >>>>>>> ??????? >> ?>>>>>> ??????? > >>>>>>> ??????? >>? ? ?>>>>> >>>>>> ??????? >>>>>>> ??????? >> ?>>>>>> ??????? >> >>>>>>> ??????? >>? ? ?>>>>> >>>>>> ??????? >>>>>>> ??????? >> ?>>>>>> >>>> wrote: >>>>>>> ??????? >>? ? ?>>>>> > >>>>>>> ??????? >>? ? ?>>>>> > ?Hi Jeremy, >>>>>>> ??????? >>? ? ?>>>>> > >>>>>>> ??????? >>? ? ?>>>>> > ?On 16/06/2018 2:33 AM, Jeremy >>>>>>> ??????? Manson wrote: >>>>>>> ??????? >>? ? ?>>>>> > > Hi all, >>>>>>> ??????? >>? ? ?>>>>> > > >>>>>>> ??????? >>? ? ?>>>>> > > There are a number of references in the >>>>>>> ??????? >>? ? ?JVMTI doc >>>>>>> ??????? >>? ? ?>>>>> ??? to its >>>>>>> ??????? >>? ? ?>>>>> ??? ??? not doing >>>>>>> ??????? >>? ? ?>>>>> > > object allocation tracking.? Now >>>>>>> ??????? that JEP >>>>>>> ??????? >>? ? ?331 has >>>>>>> ??????? >>? ? ?>>>>> ??? landed, >>>>>>> ??????? >>? ? ?>>>>> these >>>>>>> ??????? >>? ? ?>>>>> > > references are obsolete.? This patch >>>>>>> ??????? >>? ? ?changes those >>>>>>> ??????? >>? ? ?>>>>> references >>>>>>> ??????? >>? ? ?>>>>> > ?accordingly. >>>>>>> ??????? >>? ? ?>>>>> > > >>>>>>> ??????? >>? ? ?>>>>> > > While I was there, I took the >>>>>>> ??????? liberty of >>>>>>> ??????? >>? ? ?fixing >>>>>>> ??????? >>? ? ?>>>>> some >>>>>>> ??????? >>? ? ?>>>>> spelling?errors. >>>>>>> ??????? >>? ? ?>>>>> > > >>>>>>> ??????? >>? ? ?>>>>> > > As far as I know, these are >>>>>>> ??????? non-normative >>>>>>> ??????? >>? ? ?>>>>> changes and >>>>>>> ??????? >>? ? ?>>>>> modify no >>>>>>> ??????? >>? ? ?>>>>> > ?API, so >>>>>>> ??????? >>? ? ?>>>>> > > they should not require a CSR. >>>>>>> ??????? >>? ? ?>>>>> > >>>>>>> ??????? >>? ? ?>>>>> > ?I'm unclear on the nature of the >>>>>>> ??????? change to >>>>>>> ??????? >>? ? ?"VM Object >>>>>>> ??????? >>? ? ?>>>>> Allocation". Does >>>>>>> ??????? >>? ? ?>>>>> > ?the existence of >>>>>>> ??????? SampledObjectAlloc change >>>>>>> ??????? >>? ? ?when a VM >>>>>>> ??????? >>? ? ?>>>>> ??? should >>>>>>> ??????? >>? ? ?>>>>> issue "VM >>>>>>> ??????? >>? ? ?>>>>> > ?object allocation" events? >>>>>>> ??????? >>? ? ?>>>>> > >>>>>>> ??????? >>? ? ?>>>>> > ?Thanks, >>>>>>> ??????? >>? ? ?>>>>> > ?David >>>>>>> ??????? >>? ? ?>>>>> > >>>>>>> ??????? >>? ? ?>>>>> > > >>>>>>> ??????? >>? ? ?>>>>> > > Bug: >>>>>>> ??????? >>? ? ?>>>>> > > >>>>>>> ??????? >> https://bugs.openjdk.java.net/browse/JDK-8205113 >>>>>>> ??????? >>? ? ?>>>>> > > >>>>>>> ??????? >>? ? ?>>>>> > > Webrev: >>>>>>> ??????? >>? ? ?>>>>> > > >>>>>>> ??????? >>? ? ?>>>>> >>>>>>> http://cr.openjdk.java.net/~jmanson/8205113/webrev.00/ >>>>>>> >>>>>>> ??????? >> >>>>>>> ? >>>>>>> ??????? >>? ? ?>>>>> >>>>>>> >>>>>>> ??????? >>? ? ?>>>>> > > >>>>>>> ??????? >>? ? ?>>>>> > > Thanks! >>>>>>> ??????? >>? ? ?>>>>> > > >>>>>>> ??????? >>? ? ?>>>>> > > Jeremy >>>>>>> ??????? >>? ? ?>>>>> > >>>>>>> ??????? >>? ? ?>>>>> >>>>>>> ??????? >>? ? ?>>> >>>>>>> ??????? >>? ? ?>> >>>>>>> ??????? >> >>>>>>> ??????? > >>>>>>> >>>>>> >>>>> >>> > From jini.george at oracle.com Sun Jun 24 17:37:26 2018 From: jini.george at oracle.com (Jini George) Date: Sun, 24 Jun 2018 23:07:26 +0530 Subject: RFR: JDK-8189429: SA: MacOSX: Replace the deprecated PT_ATTACH with PT_ATTACHEXC In-Reply-To: References: <7abea33e-7fb0-fbe0-d463-8999ccdf54da@oracle.com> Message-ID: <0c93044e-7aaf-179b-c80c-45ff4d7e2712@oracle.com> Hi Erik, Thank you very much for looking into this. I have addressed your comments. The latest webrev is at: http://cr.openjdk.java.net/~jgeorge/8189429/webrev.06/ Thank you, Jini. On 6/23/2018 3:31 AM, Erik Joelsson wrote: > Hello Jini, > > In general this looks pretty good, but it's also breaking some new > ground as it's adding generation of native source in the java gensrc > step. Mixing native code with the java code that the genrcs targets and > gensrc output directories are meant for seems ok for now, but may cause > trouble in the future. I'm going to accept it for now though. > > In Gensrc-jdk.hotspot.agent.gmk: Please put the new macosx stuff in its > own section (as delimited by the 80x # lines) and put that whole thing > inside a conditional for macosx. Also please break line 47 (for recipe > lines, indent with tab and 4 additional spaces for continuation [1]). > > In Lib-jdk.hotspot.agent.gmk: I believe adding extra src should be > enough as that will implicitly add the same directories as header dirs > by default. At least that's the intention. > > /Erik > > [1] http://openjdk.java.net/groups/build/doc/code-conventions.html > > On 2018-06-22 11:11, Jini George wrote: >> Hi all, >> >> [Including build-dev also since this includes build related changes]. >> >> Requesting reviews for: >> >> https://bugs.openjdk.java.net/browse/JDK-8189429 (SA: MacOSX: Replace >> the deprecated PT_ATTACH with PT_ATTACHEXC) >> >> Webrev: http://cr.openjdk.java.net/~jgeorge/8189429/webrev.04/ >> >> This is the follow-up solution for the temporary restoration of >> PT_ATTACH to fix https://bugs.openjdk.java.net/browse/JDK-8184042 >> (several serviceability/sa tests timed out on MacOS X), which was done >> in Oct 2017. The mails related to this are at: >> >> http://mail.openjdk.java.net/pipermail/serviceability-dev/2017-August/021702.html >> >> >> Changes as compared to the patch sent last year >> (http://cr.openjdk.java.net/~jgeorge/8184042/webrev.00/): >> >> * Addressed the review comments which were provided by Poonam, Dan, >> Dmitry. >> * A major change as compared to what was done last year is that the >> MIG generated files have been included as a part of the JDK build >> process. >> * The test case which was provided in the patch last year is no longer >> required since we have ClhsdbAttach.java testing the same >> functionality as a part of the SA testsuite now. >> * Other than that, some minor improvements have been done wrt error >> handling. >> >> The steps for the proposed solution have been provided in JBS. >> >> Testing: ALL the SA tests pass on MacOSX and the other Mach5 platforms. >> >> Thanks to Sharath, Robin, Gerard and Dan for looking into the changes >> and providing valuable comments. >> >> Thanks, >> Jini. > From david.holmes at oracle.com Mon Jun 25 05:12:17 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 25 Jun 2018 15:12:17 +1000 Subject: RFR 8205113: Update JVMTI doc references to object allocation tracking In-Reply-To: References: <53f576e2-0adb-a6c5-8a3c-026dcb1aa7ad@oracle.com> <8224afa2-a6cb-fee2-8fa3-184828ac7c50@oracle.com> <60899f09-12df-1e6e-7e16-406a60939b7c@oracle.com> <96713bc3-9350-fd4a-75e5-2c36257066b2@oracle.com> <3018f21d-52b0-9ec4-151e-e4fb01da10ff@oracle.com> <3ffcc780-2568-ec21-b503-228c91d18969@oracle.com> <845054d9-ec16-e2f8-afec-c885127bbd04@oracle.com> <51b86cf0-0287-4246-fe1d-f7f536ca3140@oracle.com> Message-ID: <00f5491d-e5b4-9266-2725-04c96ccca5d0@oracle.com> Thanks Serguei! David On 24/06/2018 3:55 PM, serguei.spitsyn at oracle.com wrote: > On 6/23/18 16:19, serguei.spitsyn at oracle.com wrote: >> >> On 6/23/18 15:53, David Holmes wrote: >>> On 24/06/2018 7:58 AM, serguei.spitsyn at oracle.com wrote: >>>> Hi David, >>>> >>>> This was your suggestion: >>> >>> There was only one part of the patch being debated! The rest of the >>> patch was still applicable. Jeremy specifically stated >>> >>> "That would be okay with me, assuming that my other corrections are >>> made.? I'd also like to fix the spelling of instrumentation in the >>> first sentence." >> >> Yes, I missed the other part - sorry. :( >> Will push it under a separate bug id. > > Pushed the fix for typos under: > ? 8205570 fix a number of typos in the JVMTI spec8 > > Thanks, > Serguei > >> Not sure if another review is needed. >> >> Thanks, >> Serguei >> >>> David >>> >>>> --- >>>> ?? Sent when a method causes the virtual machine to directly >>>> allocate an >>>> ?? Object visible to Java programming language code. >>>> ?? Generally object allocation can be detected by instrumenting >>>> ?? the bytecodes of allocating methods. >>>> ?? Object allocation generated in native code by JNI function >>>> ?? calls can be detected using >>>> ?? JNI function >>>> interception. >>>> ??? Some methods might not have associated bytecodes and are not >>>> ??? native methods, they instead are executed directly by the >>>> ??? VM. These methods should send this event. >>>> ??? Virtual machines which are incapable of bytecode instrumentation >>>> ??? for some or all of their methods can send this event. >>>> >>>> ??? Note that the >>> id="SampledObjectAlloc">SampledObjectAlloc >>>> ??? event is triggered on all Java object allocations, including those >>>> ??? caused by bytecode method execution, JNI method execution, and >>>> ??? directly by VM methods. >>>> --- >>>> >>>> I've pushed the patch: >>>> >>>> diff -r f703d45c5687 src/hotspot/share/prims/jvmti.xml >>>> --- a/src/hotspot/share/prims/jvmti.xml??? Tue Jun 05 11:55:39 2018 >>>> +0200 >>>> +++ b/src/hotspot/share/prims/jvmti.xml??? Sat Jun 23 01:18:57 2018 >>>> -0700 >>>> @@ -13507,9 +13507,8 @@ >>>> ??? >>> ????? ?id="VMObjectAlloc" const="JVMTI_EVENT_VM_OBJECT_ALLOC" num="84"> >>>> ????? >>>> -????? Sent when a method causes the virtual machine to allocate an >>>> -????? Object visible to Java programming language code and the >>>> -????? allocation is not detectable by other intrumentation mechanisms. >>>> +????? Sent when a method causes the virtual machine to directly >>>> allocate an >>>> +????? Object visible to Java programming language code. >>>> ??????? Generally object allocation should be detected by instrumenting >>>> ??????? the bytecodes of allocating methods. >>>> ??????? Object allocation generated in native code by JNI function >>>> @@ -13520,6 +13519,12 @@ >>>> ??????? VM. These methods should send this event. >>>> ??????? Virtual machines which are incapable of bytecode >>>> instrumentation >>>> ??????? for some or all of their methods can send this event. >>>> + >>>> +????? Note that the >>> + id="SampledObjectAlloc">SampledObjectAlloc >>>> +????? event is triggered on all Java object allocations, including >>>> those >>>> +????? caused by bytecode method execution, JNI method execution, and >>>> +????? directly by VM methods. >>>> ???????

          >>>> ??????? Typical examples where this event might be sent: >>>> ???????

            >>>> >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 6/23/18 04:49, David Holmes wrote: >>>>> On 23/06/2018 6:25 PM, serguei.spitsyn at oracle.com wrote: >>>>>> I've pushed the version suggested by David. >>>>> >>>>> But you left out all of Jeremy's other fixups! >>>>> >>>>> David >>>>> >>>>>> Thanks, >>>>>> serguei >>>>>> >>>>>> >>>>>> On 6/22/18 09:00, serguei.spitsyn at oracle.com wrote: >>>>>>> Hi Jeremy, >>>>>>> >>>>>>> Okay, let me look at it once more before making final decision. >>>>>>> We have all suggestions and preferences listed. >>>>>>> >>>>>>> Thanks, >>>>>>> Serguei >>>>>>> >>>>>>> >>>>>>> On 6/22/18 08:22, Jeremy Manson wrote: >>>>>>>> Hey folks - >>>>>>>> >>>>>>>> With the window closing soon for 11, I feel that we should get >>>>>>>> this revision in (just so that the spec is clear).? That said, I >>>>>>>> don't feel strongly about the wording.? Who gets to make the >>>>>>>> decision? >>>>>>>> >>>>>>>> Jeremy >>>>>>>> >>>>>>>> On Wed, Jun 20, 2018 at 9:41 AM Jeremy Manson >>>>>>>> > wrote: >>>>>>>> >>>>>>>> ??? FWIW, I'm also okay with David's version, with the caveat that >>>>>>>> ??? "These methods should send this event" should probably read >>>>>>>> ??? "Invocations of these methods should trigger this event". >>>>>>>> >>>>>>>> ??? Jeremy >>>>>>>> >>>>>>>> ??? On Wed, Jun 20, 2018 at 4:11 AM David Holmes >>>>>>>> ??? > >>>>>>>> wrote: >>>>>>>> >>>>>>>> ??????? On 20/06/2018 4:48 PM, serguei.spitsyn at oracle.com >>>>>>>> ??????? wrote: >>>>>>>> ??????? > On 6/19/18 23:29, Jeremy Manson wrote: >>>>>>>> ??????? >> Maybe we should make that clarification. >>>>>>>> ??????? >> >>>>>>>> ??????? >> Also, the reason I danced around that in my revision >>>>>>>> is that >>>>>>>> ??????? > >>>>>>>> ??????? > Understand that. >>>>>>>> ??????? > But it is not a good style to clarify about >>>>>>>> ??????? SampledObjectAlloc in the >>>>>>>> ??????? > spec of VMObjectAlloc. >>>>>>>> ??????? > Would be nice to find a way to keep it simple. >>>>>>>> ??????? > >>>>>>>> ??????? > We could keep the original spec as it is but add that all >>>>>>>> ??????? this does not >>>>>>>> ??????? > apply to the SampledObjectAlloc events. >>>>>>>> >>>>>>>> ??????? I don't think that really presents a coherent spec. I still >>>>>>>> ??????? like my >>>>>>>> ??????? suggestion :) >>>>>>>> >>>>>>>> ??????? David >>>>>>>> >>>>>>>> ??????? > >>>>>>>> ??????? > Something like below: >>>>>>>> ??????? > >>>>>>>> ??????? >? ? Note, the above does not apply to the >>>>>>> ??????? > id="SampledObjectAlloc">SampledObjectAlloc >>>>>>>> ??????? >? ? event as it is triggered on all Java object >>>>>>>> allocations, >>>>>>>> ??????? including >>>>>>>> ??????? > those caused by bytecode method execution, >>>>>>>> ??????? >? ? JNI method execution, and directly by VM methods.* >>>>>>>> ??????? > * >>>>>>>> ??????? > >>>>>>>> ??????? >> if you set the sampling rate to 0, you get unconditional >>>>>>>> ??????? sampling. >>>>>>>> ??????? > A correction. >>>>>>>> ??????? > >>>>>>>> ??????? > I wrote: >>>>>>>> ??????? >? > It is still possible to sample every allocation >>>>>>>> with the >>>>>>>> ??????? SamplingRate >>>>>>>> ??????? > == 1. >>>>>>>> ??????? > >>>>>>>> ??????? > but had write: SamplingRate == 0 >>>>>>>> ??????? > >>>>>>>> ??????? > Thanks, >>>>>>>> ??????? > Serguei >>>>>>>> ??????? > >>>>>>>> ??????? > >>>>>>>> ??????? >> >>>>>>>> ??????? >> Jeremy >>>>>>>> ??????? >> >>>>>>>> ??????? >> On Tue, Jun 19, 2018 at 10:41 PM >>>>>>>> ??????? serguei.spitsyn at oracle.com >>>>>>>> >>>>>>>> ??????? >> >>>>>>> ??????? > >>>>>>>> ??????? >>>>>>> >>>>>>>> ??????? >> >>>>>>> >> wrote: >>>>>>>> ??????? >> >>>>>>>> ??????? >>? ? ?On 6/19/18 21:54, David Holmes wrote: >>>>>>>> ??????? >>? ? ?> On 20/06/2018 2:41 PM, serguei.spitsyn at oracle.com >>>>>>>> ??????? >>>>>>>> ??????? >> ?>>>>>>> ??????? > wrote: >>>>>>>> ??????? >>? ? ?>> On 6/19/18 21:11, Jeremy Manson wrote: >>>>>>>> ??????? >>? ? ?>>> That would be okay with me, assuming that my >>>>>>>> other >>>>>>>> ??????? corrections >>>>>>>> ??????? >>? ? ?are >>>>>>>> ??????? >>? ? ?>>> made. >>>>>>>> ??????? >>? ? ?>> >>>>>>>> ??????? >>? ? ?>> Another option would be to say "non-sampling" >>>>>>>> ??????? instead of >>>>>>>> ??????? >>? ? ?>> "unconditional": >>>>>>>> ??????? >>? ? ?>> >>>>>>>> ??????? >>? ? ?>> == Sent when a method causes the virtual >>>>>>>> machine to >>>>>>>> ??????? allocate an >>>>>>>> ??????? >>? ? ?>> == Object visible to Java programming language >>>>>>>> code >>>>>>>> ??????? and the >>>>>>>> ??????? >>? ? ?allocation >>>>>>>> ??????? >>? ? ?>> += is not detectable by other *non-sampling* >>>>>>>> ??????? instrumentation >>>>>>>> ??????? >>? ? ?mechanisms. >>>>>>>> ??????? >>? ? ?>> >>>>>>>> ??????? >>? ? ?>> >>>>>>>> ??????? >>? ? ?>>> I'd also like to fix the spelling of >>>>>>>> ??????? instrumentation in the first >>>>>>>> ??????? >>? ? ?>>> sentence. >>>>>>>> ??????? >>? ? ?>> >>>>>>>> ??????? >>? ? ?>> Yes, of course. >>>>>>>> ??????? >>? ? ?>> >>>>>>>> ??????? >>? ? ?>> Let's wait for David's opinion. >>>>>>>> ??????? >>? ? ?> >>>>>>>> ??????? >>? ? ?> I'm okay with Serguei's suggestion (combined with >>>>>>>> ??????? Jeremy's other >>>>>>>> ??????? >>? ? ?> changes). >>>>>>>> ??????? >>? ? ?> >>>>>>>> ??????? >>? ? ?> I'm not that familiar with conditional versus >>>>>>>> ??????? unconditional >>>>>>>> ??????? >>? ? ?> instrumentation. >>>>>>>> ??????? >> >>>>>>>> ??????? >>? ? ?The whole point of the SampledObjectAlloc events >>>>>>>> is to >>>>>>>> ??????? post them >>>>>>>> ??????? >>? ? ?conditionally >>>>>>>> ??????? >>? ? ?depending on the SamplingRate so that just some >>>>>>>> amount of >>>>>>>> ??????? >>? ? ?allocations is >>>>>>>> ??????? >>? ? ?sampled. >>>>>>>> ??????? >>? ? ?It is why the overhead can be minimal (less than 5 >>>>>>>> ??????? percents). >>>>>>>> ??????? >>? ? ?It is still possible to sample every allocation >>>>>>>> with the >>>>>>>> ??????? >>? ? ?SamplingRate == 1. >>>>>>>> ??????? >> >>>>>>>> ??????? >>? ? ?The VMObjectAlloc events are posted unconditionally, >>>>>>>> ??????? so every VM >>>>>>>> ??????? >>? ? ?allocation is posted. >>>>>>>> ??????? >> >>>>>>>> ??????? >>? ? ?Thanks, >>>>>>>> ??????? >>? ? ?Serguei >>>>>>>> ??????? >> >>>>>>>> ??????? >> >>>>>>>> ??????? >>? ? ?> >>>>>>>> ??????? >>? ? ?> Thanks, >>>>>>>> ??????? >>? ? ?> David >>>>>>>> ??????? >>? ? ?> >>>>>>>> ??????? >>? ? ?>> >>>>>>>> ??????? >>? ? ?>> Thanks, >>>>>>>> ??????? >>? ? ?>> Serguei >>>>>>>> ??????? >>? ? ?>> >>>>>>>> ??????? >>? ? ?>> >>>>>>>> ??????? >>? ? ?>>> >>>>>>>> ??????? >>? ? ?>>> Jeremy >>>>>>>> ??????? >>? ? ?>>> >>>>>>>> ??????? >>? ? ?>>> On Mon, Jun 18, 2018 at 11:01 PM >>>>>>>> ??????? serguei.spitsyn at oracle.com >>>>>>>> >>>>>>>> ??????? >> ?>>>>>>> ??????? > >>>>>>>> ??????? >>? ? ?>>> >>>>>>> ??????? >>>>>>>> ??????? >> ?>>>>>>> >> >>>>>>>> ??????? >>>>>>> >>>>>>>> ??????? >> ?>>>>>>> ??????? > >>>>>>>> ??????? >>? ? ?>>> >>>>>>> ??????? >>>>>>>> ??????? >> ?>>>>>>> >>> wrote: >>>>>>>> ??????? >>? ? ?>>> >>>>>>>> ??????? >>? ? ?>>> ??? Hi Jeremy and David, >>>>>>>> ??????? >>? ? ?>>> >>>>>>>> ??????? >>? ? ?>>> ??? Sorry for being late to the party. >>>>>>>> ??????? >>? ? ?>>> >>>>>>>> ??????? >>? ? ?>>> ??? I'm also concerned about the Jeremy's spec >>>>>>>> ??????? update is more >>>>>>>> ??????? >>? ? ?>>> ??? intrusive than necessary. >>>>>>>> ??????? >>? ? ?>>> ??? One specifics of the new SampledObjectAlloc >>>>>>>> ??????? event is that >>>>>>>> ??????? >>? ? ?it is >>>>>>>> ??????? >>? ? ?>>> ??? posted conditionally. >>>>>>>> ??????? >>? ? ?>>> ??? So, it is not fully comparable with the >>>>>>>> ??????? VMObjectAlloc >>>>>>>> ??????? >>? ? ?event and >>>>>>>> ??????? >>? ? ?>>> ??? can not replace it in any way. >>>>>>>> ??????? >>? ? ?>>> ??? I'm even not yet convinced that any spec >>>>>>>> ??????? update is necessary. >>>>>>>> ??????? >>? ? ?>>> >>>>>>>> ??????? >>? ? ?>>> ??? However, something like below would look >>>>>>>> ??????? minimal and >>>>>>>> ??????? >>? ? ?reasonable: >>>>>>>> ??????? >>? ? ?>>> >>>>>>>> ??????? >>? ? ?>>> ??? == Sent when a method causes the virtual >>>>>>>> ??????? machine to >>>>>>>> ??????? >>? ? ?allocate an >>>>>>>> ??????? >>? ? ?>>> ??? == Object visible to Java programming >>>>>>>> language >>>>>>>> ??????? code and the >>>>>>>> ??????? >>? ? ?>>> allocation >>>>>>>> ??????? >>? ? ?>>> ??? += is not detectable by other >>>>>>>> *unconditional* >>>>>>>> ??????? intrumentation >>>>>>>> ??????? >>? ? ?>>> ??? mechanisms. >>>>>>>> ??????? >>? ? ?>>> >>>>>>>> ??????? >>? ? ?>>> ??? == Generally object allocation should be >>>>>>>> ??????? detected by >>>>>>>> ??????? >>? ? ?instrumenting >>>>>>>> ??????? >>? ? ?>>> ??? == the bytecodes of allocating methods. >>>>>>>> ??????? >>? ? ?>>> >>>>>>>> ??????? >>? ? ?>>> ??? == Object allocation generated in native >>>>>>>> code >>>>>>>> ??????? by JNI function >>>>>>>> ??????? >>? ? ?>>> ??? == calls should be detected using >>>>>>>> ??????? >>? ? ?>>> ??? == JNI >>>>>>>> function >>>>>>>> ??????? >>? ? ?>>> interception. >>>>>>>> ??????? >>? ? ?>>> >>>>>>>> ??????? >>? ? ?>>> ??? == Some methods might not have associated >>>>>>>> ??????? bytecodes and >>>>>>>> ??????? >>? ? ?are not >>>>>>>> ??????? >>? ? ?>>> ??? == native methods, they instead are executed >>>>>>>> ??????? directly by the >>>>>>>> ??????? >>? ? ?>>> ??? == VM. These methods should send this event. >>>>>>>> ??????? >>? ? ?>>> >>>>>>>> ??????? >>? ? ?>>> ??? == Virtual machines which are incapable of >>>>>>>> ??????? bytecode >>>>>>>> ??????? >>? ? ?instrumentation >>>>>>>> ??????? >>? ? ?>>> ??? == for some or all of their methods can send >>>>>>>> ??????? this event. >>>>>>>> ??????? >>? ? ?>>> >>>>>>>> ??????? >>? ? ?>>> ??? *++ Note that the >>>>>>> ??????? >>? ? ?>>> >>>>>>>> id="SampledObjectAlloc">SampledObjectAlloc** >>>>>>>> ??????? >>? ? ?>>> ??? **++ event is conditionally triggered on all >>>>>>>> ??????? Java object >>>>>>>> ??????? >>? ? ?>>> ??? allocations, including those** >>>>>>>> ??????? >>? ? ?>>> ??? **++ caused by bytecode method execution, >>>>>>>> JNI >>>>>>>> ??????? method >>>>>>>> ??????? >>? ? ?execution, >>>>>>>> ??????? >>? ? ?>>> ??? and directly by VM methods.** >>>>>>>> ??????? >>? ? ?>>> ??? * >>>>>>>> ??????? >>? ? ?>>> >>>>>>>> ??????? >>? ? ?>>> ??? Maybe, just adding the last statement >>>>>>>> would be >>>>>>>> ??????? good enough. >>>>>>>> ??????? >>? ? ?>>> >>>>>>>> ??????? >>? ? ?>>> ??? Thanks, >>>>>>>> ??????? >>? ? ?>>> ??? Serguei >>>>>>>> ??????? >>? ? ?>>> >>>>>>>> ??????? >>? ? ?>>> >>>>>>>> ??????? >>? ? ?>>> ??? On 6/18/18 21:36, David Holmes wrote: >>>>>>>> ??????? >>? ? ?>>>> ??? On 19/06/2018 4:50 AM, Jeremy Manson wrote: >>>>>>>> ??????? >>? ? ?>>>>> ??? Yup! The paragraph meanders a bit. How >>>>>>>> ??????? about something >>>>>>>> ??????? >>? ? ?like: >>>>>>>> ??????? >>? ? ?>>>> >>>>>>>> ??????? >>? ? ?>>>> ??? I'm not sure some of the change quite >>>>>>>> works. >>>>>>>> ??????? The original >>>>>>>> ??????? >>? ? ?text >>>>>>>> ??????? >>? ? ?>>>> ??? considers there to be three kinds of >>>>>>>> methods >>>>>>>> ??????? that can cause >>>>>>>> ??????? >>? ? ?>>>> ??? allocation when executed: >>>>>>>> ??????? >>? ? ?>>>> ??? - Java (bytecode) methods >>>>>>>> ??????? >>? ? ?>>>> ??? - JNI methods >>>>>>>> ??????? >>? ? ?>>>> ??? - VM methods >>>>>>>> ??????? >>? ? ?>>>> >>>>>>>> ??????? >>? ? ?>>>> ??? but you've turned this into three kinds of >>>>>>>> ??????? allocation: via >>>>>>>> ??????? >>? ? ?>>>> ??? bytecode, via JNI, and via the VM. You then >>>>>>>> ??????? refer to >>>>>>>> ??????? >>? ? ?"triggering" >>>>>>>> ??????? >>? ? ?>>>> ??? an allocation when we tend to use >>>>>>>> triggering >>>>>>>> ??????? for events. >>>>>>>> ??????? >>? ? ?You also >>>>>>>> ??????? >>? ? ?>>>> ??? refer to an allocation being "executed >>>>>>>> ??????? directly by the VM" (a >>>>>>>> ??????? >>? ? ?>>>> ??? phrase previously applied when the >>>>>>>> subject was a >>>>>>>> ??????? >>? ? ?'method') - but >>>>>>>> ??????? >>? ? ?>>>> ??? you don't really execute allocations. >>>>>>>> ??????? >>? ? ?>>>> >>>>>>>> ??????? >>? ? ?>>>> ??? IIUC the problem with the existing text is >>>>>>>> ??????? just that it >>>>>>>> ??????? >>? ? ?considers >>>>>>>> ??????? >>? ? ?>>>> ??? VM allocation events as being undetectable >>>>>>>> ??????? other than by >>>>>>>> ??????? >>? ? ?this "VM >>>>>>>> ??????? >>? ? ?>>>> ??? object allocation event" - but that's no >>>>>>>> ??????? longer true. So how >>>>>>>> ??????? >>? ? ?>>>> ??? about something minimally changed like >>>>>>>> this: >>>>>>>> ??????? >>? ? ?>>>> >>>>>>>> ??????? >>? ? ?>>>> ??? --- >>>>>>>> ??????? >>? ? ?>>>> ??? ? Sent when a method causes the virtual >>>>>>>> ??????? machine to directly >>>>>>>> ??????? >>? ? ?>>>> ??? allocate an >>>>>>>> ??????? >>? ? ?>>>> ??? ? Object visible to Java programming >>>>>>>> language >>>>>>>> ??????? code. >>>>>>>> ??????? >>? ? ?>>>> ??? ? Generally object allocation can be >>>>>>>> detected by >>>>>>>> ??????? >>? ? ?instrumenting >>>>>>>> ??????? >>? ? ?>>>> ??? ? the bytecodes of allocating methods. >>>>>>>> ??????? >>? ? ?>>>> ??? ? Object allocation generated in native >>>>>>>> code >>>>>>>> ??????? by JNI function >>>>>>>> ??????? >>? ? ?>>>> ??? ? calls can be detected using >>>>>>>> ??????? >>? ? ?>>>> JNI function >>>>>>>> ??????? >>? ? ?>>>> interception. >>>>>>>> ??????? >>? ? ?>>>> ??? ?? Some methods might not have associated >>>>>>>> ??????? bytecodes and >>>>>>>> ??????? >>? ? ?are not >>>>>>>> ??????? >>? ? ?>>>> ??? ?? native methods, they instead are >>>>>>>> executed >>>>>>>> ??????? directly by the >>>>>>>> ??????? >>? ? ?>>>> ??? ?? VM. These methods should send this >>>>>>>> event. >>>>>>>> ??????? >>? ? ?>>>> ??? ?? Virtual machines which are incapable of >>>>>>>> ??????? bytecode >>>>>>>> ??????? >>? ? ?>>>> instrumentation >>>>>>>> ??????? >>? ? ?>>>> ??? ?? for some or all of their methods can >>>>>>>> send >>>>>>>> ??????? this event. >>>>>>>> ??????? >>? ? ?>>>> >>>>>>>> ??????? >>? ? ?>>>> ??? ?? Note that the >>>>>>> ??????? >>? ? ?>>>> >>>>>>>> id="SampledObjectAlloc">SampledObjectAlloc >>>>>>>> ??????? >>? ? ?>>>> ??? ?? event is triggered on all Java object >>>>>>>> ??????? allocations, >>>>>>>> ??????? >>? ? ?including >>>>>>>> ??????? >>? ? ?>>>> ??? those >>>>>>>> ??????? >>? ? ?>>>> ??? ?? caused by bytecode method execution, JNI >>>>>>>> ??????? method >>>>>>>> ??????? >>? ? ?execution, and >>>>>>>> ??????? >>? ? ?>>>> ??? ?? directly by VM methods. >>>>>>>> ??????? >>? ? ?>>>> ??? --- >>>>>>>> ??????? >>? ? ?>>>> >>>>>>>> ??????? >>? ? ?>>>> ??? Thanks, >>>>>>>> ??????? >>? ? ?>>>> ??? David >>>>>>>> ??????? >>? ? ?>>>> >>>>>>>> ??????? >>? ? ?>>>>> ??? Sent when the virtual machine allocates an >>>>>>>> ??????? >>? ? ?>>>>> ??? Object visible to Java programming >>>>>>>> language >>>>>>>> ??????? code without >>>>>>>> ??????? >>? ? ?using a >>>>>>>> ??????? >>? ? ?>>>>> new bytecode variant or a JNI >>>>>>>> method. >>>>>>>> ??????? >>? ? ?>>>>> ??? Many approaches to tracking object >>>>>>>> ??????? allocation use a >>>>>>>> ??????? >>? ? ?>>>>> combination of >>>>>>>> ??????? >>? ? ?>>>>> bytecode-based instrumentation and >>>>>>>> >>>>>>> ??????? >>? ? ?>>>>> id="jniIntercept">JNI function >>>>>>>> ??????? >>? ? ?>>>>> interception to intercept >>>>>>>> ??????? allocations. >>>>>>>> ??????? >>? ? ?However, >>>>>>>> ??????? >>? ? ?>>>>> ??? this >>>>>>>> ??????? >>? ? ?>>>>> ??? approach can leave a number of allocations >>>>>>>> ??????? undetected. >>>>>>>> ??????? >>? ? ?>>>>> Allocations that are neither >>>>>>>> ??????? >>? ? ?>>>>> triggered by bytecode nor JNI are executed >>>>>>>> ??????? directly by >>>>>>>> ??????? >>? ? ?the VM. >>>>>>>> ??????? >>? ? ?>>>>> ??? When those allocations occur, the VM >>>>>>>> should >>>>>>>> ??????? send this event. >>>>>>>> ??????? >>? ? ?>>>>> ??? Virtual machines that are incapable of >>>>>>>> bytecode >>>>>>>> ??????? >>? ? ?instrumentation >>>>>>>> ??????? >>? ? ?>>>>> ??? for some or all of their methods may also >>>>>>>> ??????? send this event. >>>>>>>> ??????? >>? ? ?>>>>>

            >>>>>>>> ??????? >>? ? ?>>>>> ??? Note that the >>>>>>> ??????? >>? ? ?>>>>> >>>>>>>> id="SampledObjectAlloc">SampledObjectAlloc >>>>>>>> ??????? >>? ? ?>>>>> ??? event is triggered on all Java object >>>>>>>> ??????? allocations, including >>>>>>>> ??????? >>? ? ?>>>>> ??? those triggered by bytecode, >>>>>>>> ??????? >>? ? ?>>>>> ??? JNI methods, and VM events. >>>>>>>> ??????? >>? ? ?>>>>> >>>>>>>> ??????? >>? ? ?>>>>> >>>>>>>> ??????? >>? ? ?>>>>> >>>>>>>> ??????? >>? ? ?>>>>> ??? On Mon, Jun 18, 2018 at 12:57 AM David >>>>>>>> Holmes >>>>>>>> ??????? >>? ? ?>>>>> >>>>>>> ??????? >>>>>>>> ??????? >> ?>>>>>>> ??????? > >>>>>>>> ??????? >>>>>>> >>>>>>>> ??????? >> ?>>>>>>> ??????? >> >>>>>>>> ??????? >>? ? ?>>>>> >>>>>>> ??????? >>>>>>>> ??????? >> ?>>>>>>> ??????? >> >>>>>>>> ??????? >>? ? ?>>>>> >>>>>>> ??????? >>>>>>>> ??????? >> ?>>>>>>> >>> wrote: >>>>>>>> ??????? >>? ? ?>>>>> >>>>>>>> ??????? >>? ? ?>>>>> ??? ??? On 18/06/2018 5:01 PM, Jeremy >>>>>>>> Manson wrote: >>>>>>>> ??????? >>? ? ?>>>>> > We haven't changed when a VM issues >>>>>>>> ??????? "VM object >>>>>>>> ??????? >>? ? ?>>>>> allocation" events. >>>>>>>> ??????? >>? ? ?>>>>> > >>>>>>>> ??????? >>? ? ?>>>>> > There were references in the docs to a >>>>>>>> ??????? >>? ? ?requirement to use >>>>>>>> ??????? >>? ? ?>>>>> ??? bytecode >>>>>>>> ??????? >>? ? ?>>>>> > rewriting and JNI interception to track >>>>>>>> ??????? >>? ? ?allocations.? With >>>>>>>> ??????? >>? ? ?>>>>> > SampledObjectAlloc, this is no longer >>>>>>>> ??????? the case - >>>>>>>> ??????? >>? ? ?>>>>> SampledObjectAlloc can >>>>>>>> ??????? >>? ? ?>>>>> > track them.? This change is supposed >>>>>>>> ??????? to remove the >>>>>>>> ??????? >>? ? ?>>>>> references to >>>>>>>> ??????? >>? ? ?>>>>> those >>>>>>>> ??????? >>? ? ?>>>>> > requirements, and provide suitable >>>>>>>> ??????? replacement text. >>>>>>>> ??????? >>? ? ?>>>>> > >>>>>>>> ??????? >>? ? ?>>>>> > VM object alloc has specific language >>>>>>>> ??????? about being >>>>>>>> ??????? >>? ? ?able to >>>>>>>> ??????? >>? ? ?>>>>> ??? use it to >>>>>>>> ??????? >>? ? ?>>>>> > track allocations that cannot be >>>>>>>> ??????? tracked with >>>>>>>> ??????? >>? ? ?bytecode >>>>>>>> ??????? >>? ? ?>>>>> instrumentation >>>>>>>> ??????? >>? ? ?>>>>> > and JNI interception.? My goal in >>>>>>>> ??????? rephrasing was >>>>>>>> ??????? >>? ? ?to make >>>>>>>> ??????? >>? ? ?>>>>> ??? it clear >>>>>>>> ??????? >>? ? ?>>>>> that, >>>>>>>> ??????? >>? ? ?>>>>> > while you can still use it for this, >>>>>>>> ??????? you can also >>>>>>>> ??????? >>? ? ?just use >>>>>>>> ??????? >>? ? ?>>>>> > SampledObjectAlloc for everything. >>>>>>>> ??????? >>? ? ?>>>>> >>>>>>>> ??????? >>? ? ?>>>>> Okay. That doesn't come across clearly >>>>>>>> ??????? to me - >>>>>>>> ??????? >>? ? ?sorry. So you >>>>>>>> ??????? >>? ? ?>>>>> ??? will now >>>>>>>> ??????? >>? ? ?>>>>> ??? ??? get both kinds of events for >>>>>>>> allocations >>>>>>>> ??????? done in the VM? >>>>>>>> ??????? >>? ? ?>>>>> >>>>>>>> ??????? >>? ? ?>>>>> Thanks, >>>>>>>> ??????? >>? ? ?>>>>> David >>>>>>>> ??????? >>? ? ?>>>>> >>>>>>>> ??????? >>? ? ?>>>>> >>>>>>>> ??????? >>? ? ?>>>>> > Jeremy >>>>>>>> ??????? >>? ? ?>>>>> > >>>>>>>> ??????? >>? ? ?>>>>> > On Sun, Jun 17, 2018 at 9:11 PM David >>>>>>>> ??????? Holmes >>>>>>>> ??????? >>? ? ?>>>>> >>>>>>> ??????? >>>>>>>> ??????? >> ?>>>>>>> ??????? > >>>>>>>> ??????? >>>>>>> >>>>>>>> ??????? >> ?>>>>>>> ??????? >> >>>>>>>> ??????? >>? ? ?>>>>> >>>>>>> ??????? >>>>>>>> ??????? >> ?>>>>>>> ??????? >> >>>>>>>> ??????? >>>>>>> >>>>>>>> ??????? >> ?>>>>>>> ??????? >> >>>>>>>> ??????? >>? ? ?>>>>> > >>>>>>> ??????? >>>>>>>> ??????? >> ?>>>>>>> ??????? > >>>>>>>> ??????? >>? ? ?>>>>> >>>>>>> ??????? >>>>>>>> ??????? >> ?>>>>>>> ??????? >> >>>>>>>> ??????? >>? ? ?>>>>> >>>>>>> ??????? >>>>>>>> ??????? >> ?>>>>>>> >>>> wrote: >>>>>>>> ??????? >>? ? ?>>>>> > >>>>>>>> ??????? >>? ? ?>>>>> > ?Hi Jeremy, >>>>>>>> ??????? >>? ? ?>>>>> > >>>>>>>> ??????? >>? ? ?>>>>> > ?On 16/06/2018 2:33 AM, Jeremy >>>>>>>> ??????? Manson wrote: >>>>>>>> ??????? >>? ? ?>>>>> > > Hi all, >>>>>>>> ??????? >>? ? ?>>>>> > > >>>>>>>> ??????? >>? ? ?>>>>> > > There are a number of references in the >>>>>>>> ??????? >>? ? ?JVMTI doc >>>>>>>> ??????? >>? ? ?>>>>> ??? to its >>>>>>>> ??????? >>? ? ?>>>>> ??? ??? not doing >>>>>>>> ??????? >>? ? ?>>>>> > > object allocation tracking.? Now >>>>>>>> ??????? that JEP >>>>>>>> ??????? >>? ? ?331 has >>>>>>>> ??????? >>? ? ?>>>>> ??? landed, >>>>>>>> ??????? >>? ? ?>>>>> these >>>>>>>> ??????? >>? ? ?>>>>> > > references are obsolete.? This patch >>>>>>>> ??????? >>? ? ?changes those >>>>>>>> ??????? >>? ? ?>>>>> references >>>>>>>> ??????? >>? ? ?>>>>> > ?accordingly. >>>>>>>> ??????? >>? ? ?>>>>> > > >>>>>>>> ??????? >>? ? ?>>>>> > > While I was there, I took the >>>>>>>> ??????? liberty of >>>>>>>> ??????? >>? ? ?fixing >>>>>>>> ??????? >>? ? ?>>>>> some >>>>>>>> ??????? >>? ? ?>>>>> spelling?errors. >>>>>>>> ??????? >>? ? ?>>>>> > > >>>>>>>> ??????? >>? ? ?>>>>> > > As far as I know, these are >>>>>>>> ??????? non-normative >>>>>>>> ??????? >>? ? ?>>>>> changes and >>>>>>>> ??????? >>? ? ?>>>>> modify no >>>>>>>> ??????? >>? ? ?>>>>> > ?API, so >>>>>>>> ??????? >>? ? ?>>>>> > > they should not require a CSR. >>>>>>>> ??????? >>? ? ?>>>>> > >>>>>>>> ??????? >>? ? ?>>>>> > ?I'm unclear on the nature of the >>>>>>>> ??????? change to >>>>>>>> ??????? >>? ? ?"VM Object >>>>>>>> ??????? >>? ? ?>>>>> Allocation". Does >>>>>>>> ??????? >>? ? ?>>>>> > ?the existence of >>>>>>>> ??????? SampledObjectAlloc change >>>>>>>> ??????? >>? ? ?when a VM >>>>>>>> ??????? >>? ? ?>>>>> ??? should >>>>>>>> ??????? >>? ? ?>>>>> issue "VM >>>>>>>> ??????? >>? ? ?>>>>> > ?object allocation" events? >>>>>>>> ??????? >>? ? ?>>>>> > >>>>>>>> ??????? >>? ? ?>>>>> > ?Thanks, >>>>>>>> ??????? >>? ? ?>>>>> > ?David >>>>>>>> ??????? >>? ? ?>>>>> > >>>>>>>> ??????? >>? ? ?>>>>> > > >>>>>>>> ??????? >>? ? ?>>>>> > > Bug: >>>>>>>> ??????? >>? ? ?>>>>> > > >>>>>>>> ??????? >> https://bugs.openjdk.java.net/browse/JDK-8205113 >>>>>>>> ??????? >>? ? ?>>>>> > > >>>>>>>> ??????? >>? ? ?>>>>> > > Webrev: >>>>>>>> ??????? >>? ? ?>>>>> > > >>>>>>>> ??????? >>? ? ?>>>>> >>>>>>>> http://cr.openjdk.java.net/~jmanson/8205113/webrev.00/ >>>>>>>> >>>>>>>> ??????? >> >>>>>>>> ? >>>>>>>> ??????? >>? ? ?>>>>> >>>>>>>> >>>>>>>> ??????? >>? ? ?>>>>> > > >>>>>>>> ??????? >>? ? ?>>>>> > > Thanks! >>>>>>>> ??????? >>? ? ?>>>>> > > >>>>>>>> ??????? >>? ? ?>>>>> > > Jeremy >>>>>>>> ??????? >>? ? ?>>>>> > >>>>>>>> ??????? >>? ? ?>>>>> >>>>>>>> ??????? >>? ? ?>>> >>>>>>>> ??????? >>? ? ?>> >>>>>>>> ??????? >> >>>>>>>> ??????? > >>>>>>>> >>>>>>> >>>>>> >>>> >> > From david.holmes at oracle.com Mon Jun 25 05:48:34 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 25 Jun 2018 15:48:34 +1000 Subject: RFR(s): jcmd VM.classloaders should fold similar loaders In-Reply-To: References: Message-ID: <9fee006f-0de6-08ee-2e53-5cf683066005@oracle.com> Hi Thomas, On 23/06/2018 5:10 AM, Thomas St?fe wrote: > Resent with the correct subject, sorry. > > ..Thomas > > On Fri, Jun 22, 2018 at 9:08 PM, Thomas St?fe wrote: >> Hi all, >> >> may I have reviews for this small enhancement to the jcmd >> VM.classloader diagnostic command: >> >> https://bugs.openjdk.java.net/browse/JDK-8205531 >> http://cr.openjdk.java.net/~stuefe/webrevs/8205531-vm.classloader-tree-folding/webrev.00/webrev/ >> >> >> VM.classloaders prints a tree of class loaders. This tree can grow a >> lot and become unwieldy, especially with a lot of similar loaders. One >> prime example is the DelegatingClassLoader. It would be helpful were >> all these loaders: >> >> 13114: >> +-- >> | >> +-- "platform", jdk.internal.loader.ClassLoaders$PlatformClassLoader >> | >> +-- "app", jdk.internal.loader.ClassLoaders$AppClassLoader >> | >> +-- test3.internals.InMemoryClassLoader >> | >> +-- jdk.internal.reflect.DelegatingClassLoader >> | >> +-- jdk.internal.reflect.DelegatingClassLoader >> | >> +-- jdk.internal.reflect.DelegatingClassLoader >> | >> +-- jdk.internal.reflect.DelegatingClassLoader >> | >> +-- jdk.internal.reflect.DelegatingClassLoader >> | >> ...... repeat 1495 times >> >> folded into one: >> >> 13114: >> +-- >> | >> +-- "platform", jdk.internal.loader.ClassLoaders$PlatformClassLoader >> | >> +-- "app", jdk.internal.loader.ClassLoaders$AppClassLoader >> | >> +-- test3.internals.InMemoryClassLoader >> | >> +-- jdk.internal.reflect.DelegatingClassLoader >> (+ 1499 more) I don't quite understand that. These are different instances aren't they - potentially uniquely named? So if we gave all loaders a default name (like we do threads) they would all show up expanded again - right? typo: ! // we we add a Not a review as I don't know any of the logic being modified. David >> >> >> Original idea by Bernd Eckenfels, see >> http://mail.openjdk.java.net/pipermail/serviceability-dev/2018-May/023824.html >> >> Thank you, Thomas From david.holmes at oracle.com Mon Jun 25 05:57:30 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 25 Jun 2018 15:57:30 +1000 Subject: RFR(XS): 8205195 NestedThreadsListHandleInErrorHandlingTest fails because hs_err doesn't contain _nested_thread_list_max In-Reply-To: References: Message-ID: <126cc7a5-a92b-a22c-6d28-e35779ee69ef@oracle.com> Hi Dan, On 21/06/2018 10:18 AM, Daniel D. Daugherty wrote: > Greetings, > > I have a fix for a recent (very rare) Thread-SMR related test failure. > > Since the fix is related to the ErrorHandling tests and affects hs_err_pid > file generation, this code review is being sent to both the Runtime and > the Serviceability teams. Please make sure you reply-all to any responses > so we have complete review threads on both aliases. > > Bug URL: https://bugs.openjdk.java.net/browse/JDK-8205195 > > Webrev URL: > http://cr.openjdk.java.net/~dcubed/8205195-webrev/0-for-jdk-jdk/ > > The bug itself contains analysis about the root cause of the bug and > the comment updates to the code describes the no win scenario that the > hs_err_pid file generation code is in. Of course, I also have a comment > where I was able to harden the ErrorHandling tests. I did manage to > resist the urge to mention the "Kobiyashi Maru" [1] in the new comments. > > Testing: Mach5 builds-tier1,jdk-tier1,jdk-tier2,hs-tier1,hs-tier2,hs-tier3 > ?? ? ? ? on the usual Oracle platforms. > > Thanks, in advance, for any comments, questions or suggestions. I don't quite follow the fix. Won't you self-deadlock on acquiring the Threads_lock in the secondary error handler test, due to the recursive call to controlled_crash ? David > Dan > > [1] https://www.urbandictionary.com/define.php?term=Kobayashi%20Maru > From david.holmes at oracle.com Mon Jun 25 06:52:09 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 25 Jun 2018 16:52:09 +1000 Subject: RFR(S): 8200720: Print additional information in thread dump (times, allocated bytes etc.) In-Reply-To: <3795D1AE-CBF4-4BC3-B707-ED77D4DE00D0@sap.com> References: <56817AB5-88FC-4FB9-A271-7206EA69ECA7@sap.com> <2728581E-0E32-4911-B1A9-79AD6CBA81B0@sap.com> <082ca98f-9a68-15a0-95b5-c4d5c733ab36@oracle.com> <933cdacc-9b6f-c8b5-e244-558f38e7888c@oracle.com> <55806aa8-f09b-e9a8-b512-b9676029c494@oracle.com> <0F497BC6-2569-4EF6-9B59-E24A8E1B3A37@sap.com> <5dc08f65e83f4374b8e1f73f218c6d6d@sap.com> <626701f8-b71b-58dc-f837-e5e1d81ef187@oracle.com> <19BE9680-3E6E-4620-B72D-D379F39C1C92@sap.com> <6ce47417-311b-e6cd-b8cd-d46eb7e04c2f@oracle.com> <3795D1AE-CBF4-4BC3-B707-ED77D4DE00D0@sap.com> Message-ID: <9f5a404a-24e6-9131-3c0d-80c5c01ccfca@oracle.com> Hi Gunter, I see what you mean about the initialization point of the statistics. Personally I'd prefer to see the startTime as read-only and only set by the constructor rather than having a general-purpose setter function for it. Thanks, David On 21/06/2018 1:28 AM, Haug, Gunter wrote: > Hi David, > > Thanks for taking the time to look into this! > >> "statistic info" is not very good grammatically. These things are either >> statistics, or statistical information. So e.g. >> >> class ThreadStatistics >> ... > > OK, I'll change that! > >> const_cast(this)->cooked_allocated_bytes(); >> >> Why do we need a const cast to invoke a method on ourselves ?? > > cooked_allocated_bytes() is not declared const and we can't make it, as it uses OrderAccess::load_acquire(&_allocated_bytes). Therefor the const_cast. > >> src/hotspot/share/runtime/thread.hpp >> >> Please put the new #include in alphabetical order. > > Wilco! > >> I was expecting to see a default parameter used here rather than adding >> an overload: >> virtual void print_on(outputStream* st, bool print_extended_info = >> false) const { print_on(st, print_extended_info); } > > Thread inherits from AllocatedObj (in dbg, at least) where virtual void AllocatedObj::print_on(outputStream*) const is declared. Some compilers, e.g. gcc, will complain that this is hidden by Thread::print_on(outputStream*, bool). Others, e.g. clang, are happy with that. We could implement a method with a different name but would that be nicer? > >> 2179 static void print_on(outputStream* st, bool print_stacks, bool >> internal_format, bool print_concurrent_locks, bool extended_thread_info); >> >> Again I expected to see a default parameter here - but I didn't check if >> all callers themselves take the new parameter. ?? > > This one has just one single caller, so I think it's OK. > >> src/hotspot/share/runtime/vm_operations.hpp >> ... > > I'll change it to conform to the guidelines. > >> src/hotspot/share/runtime/threadStatisticInfo.hpp >> >> There are no includes in this file. It should include the appropriate >> std header for type definitions, and os.hpp. > > Agreed, I'll add the includes. > > >> I'm not sure if class ThreadStatisticInfo needs an allocation type as a >> super type. >> >> Might be worth adding a constructor to give a default value so that you >> can tell if the statistics have been initialized when they appear in the >> printout. ? > > ThreadStatisticInfo is a data member of Thread. Currently it is initialized in Thread::Thread, but I can add a constructor, if you prefer. > > Thanks again, > Gunter > > > > ?On 20.06.18, 09:15, "David Holmes" wrote: > > Hi Gunter, > > On 19/06/2018 8:51 PM, Haug, Gunter wrote: > > Hi all, > > > > Thanks Chris and Christoph for the reviews! Christoph, I'll incorporate the little improvement you suggested. > > David, are you OK with the change as well? > > Sorry I'd lost track of this one a bit ... > > The overall approach now seems okay. > > Some naming/terminology issues: > > "statistic info" is not very good grammatically. These things are either > statistics, or statistical information. So e.g. > > class ThreadStatistics > > ThreadStatistics& statistics() { return _statistics; } > > etc. > > "extended_thread_info" should really be print_extended_info (similar to > print_concurrent_locks). You don't need "thread" in there when this is > always part of a Thread related API. > > > src/hotspot/share/runtime/thread.cpp > > const_cast(this)-> > > Why do we need a const cast to invoke a method on ourselves ?? > > --- > > src/hotspot/share/runtime/thread.hpp > > Please put the new #include in alphabetical order. > > 641 // Printing > 642 void print_on(outputStream* st, bool extended_thread_info) const; > 643 virtual void print_on(outputStream* st) const { print_on(st, > false); } > > I was expecting to see a default parameter used here rather than adding > an overload: > > virtual void print_on(outputStream* st, bool print_extended_info = > false) const { print_on(st, print_extended_info); } > > > 2179 static void print_on(outputStream* st, bool print_stacks, bool > internal_format, bool print_concurrent_locks, bool extended_thread_info); > > Again I expected to see a default parameter here - but I didn't check if > all callers themselves take the new parameter. ?? > > --- > > src/hotspot/share/runtime/vm_operations.hpp > > 376 VM_PrintThreads() > 377 { _out = tty; _print_concurrent_locks = PrintConcurrentLocks; ; > _extended_thread_info = false; } > 378 VM_PrintThreads(outputStream* out, bool print_concurrent_locks, > bool extended_thread_info) > 379 { _out = out; _print_concurrent_locks = print_concurrent_locks; > _extended_thread_info = extended_thread_info; } > 380 VMOp_Type type() const > 381 { return VMOp_PrintThreads; } > > Style nits: either keep everything on one line as before (though I agree > the lines are now too long) or else the style should be: > > 380 VMOp_Type type() const { > 381 return VMOp_PrintThreads; > 382 } > > Also unclear why (existing) VM_PrintThreads constructor doesn't use > initializer list (like VM_PrintMetadata below it) or default parameters? > > --- > > src/hotspot/share/runtime/threadStatisticInfo.hpp > > There are no includes in this file. It should include the appropriate > std header for type definitions, and os.hpp. > > I'm not sure if class ThreadStatisticInfo needs an allocation type as a > super type. > > Might be worth adding a constructor to give a default value so that you > can tell if the statistics have been initialized when they appear in the > printout. ? > > --- > > Thanks, > David > > > > Thanks again, > > Gunter > > > > On 12.06.18, 01:13, "Chris Plummer" wrote: > > > > Hi Gunter, > > > > The changes look fine. I can live with the options parsing in > > attachListener.cpp. As you point out, unrecognized options were already > > silently ignored. > > > > thanks, > > > > Chris > > > > On 6/8/18 7:05 AM, Haug, Gunter wrote: > > > Hi all, > > > > > > thanks a lot for all the input! I have prepared a new version of the webrev incorporating the suggestions you made (at least I tried): > > > > > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v3/ > > > > > > This version outputs the thread times unconditionally while the other information is guarded by a flags. I think, most participants found the thread times the most valuable information and had no (strong) objections to adding them unconditionally. > > > > > > @David > > > Implementation is much simpler with conditional output only for JavaThreads. I could get rid of the terrible hack (changing the flag) without having to change too many source files. > > > > > > The information on allocated bytes is present in the Thread class already before this proposed change, I just print it. It might be sensible to move _allocated_bytes and the respective methods to the ThreadStatisticInfo class as G?tz suggested. I haven?t done that in the current version, though. > > > > > > @David and Thomas > > > I've removed the pthread-id output. I'm unsure myself what it could be good for. Anyway, we could add it with a separate change (with a better implementation) if there is a need to. > > > > > > @Chris > > > As you have written, the user has no direct contact to the attach listener of the VM and jstack won't misinterpret e.g. -help. I agree that the parsing in attachListener.cpp could be more robust. However, it hasn't been so far either. All that is done in the current implementation is a strcmp to -l everything else will be silently ignored. I can try to make this more robust or we could say that the new output is only available via jcmd. OTH nothing bad can happen with the current version of the proposed change, so we could also leave it as it is. What would you prefer? > > > > > > @Kirk and Thomas > > > Implementing a new diagnostic command is certainly well worth a thought. However, it appears to me that it's not necessary in this case. There are already flags to jstack and jcmd Thread.print, so it's not that uncommon. The amount of information that would be available by a new diagnostic command and that is now added to the thread dump is quite small and I think that is tolerable. Moreover, and most importantly, our support team is use to find the information in the thread dump and it would make things easier to us if I could leave it there. > > > > > > Thanks again and have a nice weekend, > > > Gunter > > > > > > > > > On 06.06.18, 04:56, "David Holmes" wrote: > > > > > > Hi Goetz, > > > > > > On 5/06/2018 11:07 PM, Lindenmaier, Goetz wrote: > > > > Hi > > > > > > > > this discussion touched a lot of points so far, which seem > > > > to lead to different conclusions. > > > > > > > > I think we should look at the values printed: > > > > > > > > 1. cpu=6300.65ms elapsed=123.28s > > > > Overhead > > > > cpu time: > > > > * system calls at thread dump time > > > > elapsed time: > > > > * 1 system call at thread creation time > > > > * 1 64-bit field per thread for the thread start time > > > > * 1 system call at thread dump time > > > > > > > > As I understand, JDK-8143176 would have had to get the > > > > time at each locking, which is much more time critical > > > > than doing this during thread creation. While > > > > > > Correct. > > > > > > > the time a lock was held would be much more useful, > > > > the ratio here, combined with knowledge about the application, > > > > could lead to the conclusion that the thread is wrongly > > > > blocked at much lower cost. > > > > > > Agreed. I see no issue with unconditionally adding this information as > > > it should address some of the concerns of 8143176 as well. > > > > > > > > > > > 2. pthread-id=0x109708000 > > > > Overhead: > > > > * a field access at thread dump time. Negligible I would say. > > > > > > I'd overlooked this addition, but it is of course the chunk of ifdef > > > code in osThread.cpp that I objected to. The problem I have here is the > > > convoluted mess of "thread identifiers" that we already have. We > > > currently print: > > > > > > st->print("tid=" INTPTR_FORMAT " ", p2i(this)); > > > > > > which is just the address of the Thread/JavaThread object. Then: > > > > > > st->print("nid=0x%x ", thread_id()); > > > > > > where 'n' is supposed to represent "native thread id", which is: > > > - linux: kernel thread id from syscall gettid > > > - solaris: thread library identity from thr_create() or thr_self() > > > - windows: thread id from _beginthreadex > > > - OS X: kernel thread id from pthread_mach_thread_np(pthread_self()); > > > - Other BSD: kernel (?) thread id from syscall thr_self or getthrid > > > - AIX: thread library identity from pthread_create() or pthread_self() > > > > > > It's telling that the code Gunter added gets the thread library id on > > > linux, but a "kernel thread id" on other platforms! Do we want to see a > > > thread library id and a kernel thread id? IIRC nid is supposed be the id > > > you need to see the thread in a debugger. In which case I'm unclear what > > > role the thread id Gunter wants to add is playing? I don't think we need > > > to see kernel ids in general, and pthread-id isn't useful for debugging > > > is it? > > > > > > > > > > > 3. allocated=242236760B defined_classes=1725 > > > > Overhead: > > > > * 1 64-bit field per thread. > > > > * counter increment on class creation > > > > > > > > Especially defined_classes seems to be controversial. > > > > As this only makes sense for Java threads, this could > > > > be printed in a line of it's own in Threads::print_on_error() > > > > (which already gets a flag to customize for jstack: > > > > print_concurrent_locks) by calling a dedicated function in Thread. > > > > No need for flag PrintExtendedThreadInfo thus. > > > > > > > > But I would propose to drop this information as it is > > > > too controversial. > > > > > > Agreed. > > > > > > > This leaves the times and the pthread-id to be decided whether > > > > they are worth the cost. If so, I think they should be printed > > > > unconditional. > > > > > > Agreed. With some refinement of the "thread id" issue. > > > > > > > If a flag is required to avoid bloating the info in the default case, > > > > I would include more infos, as os_prio and nid under that flag. > > > > > > Of course those things exist for good reason. The native id is supposed > > > to be what aids in you matching what you see in our thread dumps with > > > what you see in a debugger. Priority is less of an issue these days, but > > > there was a time ... ;-) > > > > > > > As jstack is deprecated, changes to jstack could be skipped always > > > > disabling the new printouts. > > > > > > > > Finally, I would propose to move _allocated_bytes into > > > > threadStatisticInfo.hpp. > > > > > > Didn't that part get dropped? (You don't need it for the time info or > > > thread id). That's a GC logging issue to me. > > > > > > Thanks, > > > David > > > > > > > Best regards, > > > > Goetz. > > > > > > > > > > > > > > > > > > > >> -----Original Message----- > > > >> From: serviceability-dev [mailto:serviceability-dev- > > > >> bounces at openjdk.java.net] On Behalf Of David Holmes > > > >> Sent: Dienstag, 5. Juni 2018 04:53 > > > >> To: Haug, Gunter ; Chris Plummer > > > >> ; serviceability-dev > > >> dev at openjdk.java.net>; Langer, Christoph ; > > > >> hotspot-runtime-dev at openjdk.java.net > > > >> Subject: Re: RFR(S): 8200720: Print additional information in thread dump > > > >> (times, allocated bytes etc.) > > > >> > > > >> Hi Gunter, > > > >> > > > >> On 5/06/2018 1:27 AM, Haug, Gunter wrote: > > > >>> Hi David, Chris, > > > >>> > > > >>> would it be an option to unconditionally print the additional information? > > > >> Regardless which way this is implemented it will be rather complicated and it > > > >> only switches on/off a few field in the thread dump. > > > >> > > > >> I'm not convinced this is all suitable information for a thread stack > > > >> dump. I can see the time information being useful if using the dump to > > > >> try and diagnoze a hang or responsiveness issue. But the allocated-bytes > > > >> and classes-defined just doesn't seem useful in the context of a thread > > > >> dump to me. Anyone interested in allocation stats is going to be looking > > > >> at GC logs etc which is where this belongs. Ditto the class-stats belong > > > >> in some kind of classloading logging IMHO. > > > >> > > > >> I'm very reluctant to burden the implementation with capturing these > > > >> kinds of stats, just to use for diagnostic purposes that may not even be > > > >> asked for. I'm very much in the "you shouldn't pay for what you don't > > > >> use" camp in that regard. (See my comments in JDK-8143176 referenced by > > > >> Sean.) > > > >> > > > >> The ThreadStatisticInfo adds overhead to every thread object - and I'd > > > >> have to suspect there could be overlap with whatever flight recorder > > > >> sticks in there too. (Thread has become even more bloated in recent time!). > > > >> > > > >> Cheers, > > > >> David > > > >> > > > >> > > > >>> Thanks, > > > >>> Gunter > > > >>> > > > >>> On 04.06.18, 01:13, "David Holmes" wrote: > > > >>> > > > >>> Hi Gunter, Chris, > > > >>> > > > >>> Sorry just trying to catch up and this is only a partial look so far ... > > > >>> > > > >>> BTW these changes are not limited to serviceability code so adding in > > > >>> runtime team as well. > > > >>> > > > >>> On 2/06/2018 9:12 AM, Chris Plummer wrote: > > > >>> > Hi Gunter, > > > >>> > > > > >>> > On 6/1/18 3:17 AM, Haug, Gunter wrote: > > > >>> >> Hi Chris, > > > >>> >> > > > >>> >> thanks for looking into this! > > > >>> >> > > > >>> >> Re the synchronization: The value is stored only in a VM operation at > > > >>> >> a safepoint by the VM thread. I was of the opinion, that this could > > > >>> >> not be interrupted by a second VM operation (of the same type). Or > > > >> am > > > >>> >> I missing something else? > > > >>> > I was really thinking more about the temporary changing of > > > >>> > PrintExtendedThreadInfo, not the value stored in the VMOp. You may > > > >> be be > > > >>> > correct that no more than one VMOp is executing, but while it is > > > >>> > executing it is has changed the value of PrintExtendedThreadInfo, > > > >> which > > > >>> > might have an impact on anything else that triggers printing thread info > > > >>> > while the VMOp is executing. > > > >>> > > > >>> Even if nothing else can possibly be running during the safepoint I find > > > >>> it extremely bad form to change a command-line flag in this way, > > > >>> particularly from a safepoint! > > > >>> > > > >>> If this flag is intended to be dynamically enabled as part of a dcmd > > > >>> then it should also be a Manageable flag IMHO. > > > >>> > > > >>> That said ... > > > >>> > > > >>> >> I did think about passing an argument to the various print_on > > > >> member > > > >>> >> functions of the thread classes, but this would require rather > > > >>> >> extensive code changes for a rather tiny extension. Therefore I feel > > > >>> >> doing it like this is the lesser evil. > > > >>> > > > >>> ... it's obviously not truly a global setting, but one that is needed on > > > >>> a per-print-request basis. The flag is just the default, but if the > > > >>> default is off you still want to enable extended printing on a > > > >>> per-request basis. > > > >>> > > > >>> The current print_on mechanics is not set up for this kind of > > > >>> flexibility. I think this needs more thought. > > > >>> > > > >>> --- > > > >>> > > > >>> Re osThread.cpp shared code changes ... I really hate to see all the > > > >>> ifdefs in there. Please add a pd_print_on function to support this if > > > >>> you truly want platform specific information. > > > >>> > > > >>> --- > > > >>> > > > >>> threadStatisticInfo.hpp > > > >>> > > > >>> typo: getElepsedTime() -> getElapsedTime() > > > >>> > > > >>> Thanks, > > > >>> David > > > >>> > > > >>> > > > >>> > > > >>> >> > > > >>> >> Re thread_dump(): I think it's correct or, well, at least it works ;-) > > > >>> >> In fact jstack will transfer the "-e" and "-l" in only one string, > > > >>> >> i.e. op->arg(0). > > > >>> > So you are saying that op->arg(0) is "-e -l" when using jstack? I think > > > >>> > you really need to clean up the parsing. As it stands right now passing, > > > >>> > I get the feeling that if a user erroneously asks for help by using > > > >>> > "jcmd Thread.Print -help", it will end up executing with -e an -l > > > >>> > enabled, and no failure for the invalid "-help" option. > > > >>> >> Christoph has already explained my reasoning. But I agree, it's not > > > >>> >> nice and I would be happy to do it like Christoph suggested. > > > >>> > I'll respond separately to his suggestion. > > > >>> > > > > >>> > thanks, > > > >>> > > > > >>> > Chris > > > >>> >> > > > >>> >> And typo fixed, sorry. > > > >>> >> > > > >>> >> Thanks again, > > > >>> >> Gunter > > > >>> >> > > > >>> >> On 01.06.18, 00:03, "Chris Plummer" > > > >> wrote: > > > >>> >> > > > >>> >> Hi Gunter, > > > >>> >> globals.hpp: fix typo "informatiuon" > > > >>> >> I worry a little bit about the synchronizing (if that's the right > > > >>> >> word) > > > >>> >> of PrintExtendedThreadInfo and the dcmd's -e flag. When using -e, > > > >>> >> you > > > >>> >> are temporarily enabling PrintExtendedThreadInfo if it was false. > > > >>> >> This > > > >>> >> temporarily changes the behavior of thread dumps, and could > > > >>> >> impact other > > > >>> >> uses that happen in parallel. Also, could two simultaneous uses > > > >>> >> of -e > > > >>> >> result in PrintExtendedThreadInfo not getting restored properly? > > > >>> >> thread_dump() doesn't look right. It looks like you are iterating > > > >>> >> char > > > >>> >> by char over the argument, and expect something like "-el" to be > > > >>> >> specified rather then "-e -l". The loop should be iterating over > > > >>> >> op->arg(i), not op->arg(0)[i]. > > > >>> >> The rest of the changes look fine. > > > >>> >> thanks, > > > >>> >> Chris > > > >>> >> On 5/30/18 8:12 AM, Haug, Gunter wrote: > > > >>> >> > Hi all, > > > >>> >> > > > > >>> >> > As Chris proposed, I have made an the extended output > > > >>> >> switchable. There is an VM flag (PrintExtendedThreadInfo), which is > > > >>> >> false by default. Moreover, there is an Option (-e) which can be used > > > >>> >> with jcmd Thread.print as well as with jstack. The -e option > > > >>> >> essentially sets PrintExtendedThreadInfo true just for the respective > > > >>> >> thread dump. > > > >>> >> > > > > >>> >> > Here is the updated webrev: > > > >>> >> > > > > >>> >> > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v2 > > > >>> >> > > > > >>> >> > (https://bugs.openjdk.java.net/browse/JDK-8200720) > > > >>> >> > > > > >>> >> > Thanks, > > > >>> >> > Gunter > > > >>> >> > > > > >>> >> > > > > >>> >> > On 02.05.18, 17:07, "serviceability-dev on behalf of Haug, > > > >>> >> Gunter" > > >>> >> gunter.haug at sap.com> wrote: > > > >>> >> > > > > >>> >> > Hi Chris, > > > >>> >> > > > > >>> >> > Thanks for looking into this. > > > >>> >> > You're right, there is a little more we have. We have > > > >>> >> implemented an IO tracing mechanism which - rather as a byproduct - > > > >>> >> keeps track of bytes read and written per thread. However, this of > > > >>> >> course requires changes not only in hotspot. We would be happy to > > > >>> >> contribute this as well, but this is a far bigger change and will > > > >>> >> probably lead to a far bigger discussion. Anyway, with the number of > > > >>> >> bytes read, the number of classes defined doesn't look that arbitrary > > > >>> >> anymore, as one can correlate IO to class loading. > > > >>> >> > > > > >>> >> > Regarding the verbose option I think that's a good idea! > > > >>> >> > > > > >>> >> > Thanks again, > > > >>> >> > Gunter > > > >>> >> > > > > >>> >> > On 01.05.18, 22:55, "Chris Plummer" > > > >>> >> wrote: > > > >>> >> > > > > >>> >> > Hi Gunter, > > > >>> >> > > > > >>> >> > The output you are adding is all useful. I think the > > > >>> >> question is (and > > > >>> >> > I'd like to see a few people chime in on this for this > > > >>> >> review) is > > > >>> >> > whether or not all of it is the appropriate for a > > > >>> >> thread's stack dump. > > > >>> >> > For example, defined_classes is on the fringe of what > > > >>> >> I would call > > > >>> >> > generally useful info in a stack dump. Sure, there > > > >>> >> might be that rare > > > >>> >> > case when you need it, but how often compared to other > > > >>> >> useful info > > > >>> >> > maintained on a per thread basis. How many other bits > > > >>> >> of useful info are > > > >>> >> > not being printed in favor of defined_classes? It > > > >>> >> seems you have more in > > > >>> >> > the queue. How many? I'm worried about how cluttered > > > >>> >> the stack dumps > > > >>> >> > will get. Maybe we should add some sort of verbose > > > >>> >> thread dumping > > > >>> >> > option. Just a thought. > > > >>> >> > > > > >>> >> > As for the implementation, overall it looks good, but > > > >>> >> I can't speak to > > > >>> >> > whether or not you are doing proper accounting of > > > >>> >> defined_classes and > > > >>> >> > bytes allocated. You'll need input from someone with > > > >>> >> more knowledge of > > > >>> >> > those areas. We'll also need to do some testing to > > > >>> >> make sure tool tests > > > >>> >> > are not impacted. > > > >>> >> > > > > >>> >> > thanks, > > > >>> >> > > > > >>> >> > Chris > > > >>> >> > > > > >>> >> > On 4/30/18 2:51 AM, Haug, Gunter wrote: > > > >>> >> > > Hi, > > > >>> >> > > > > > >>> >> > > this is an update to an RFR I posted on hotspot-dev, > > > >>> >> but it is probably more suitable to post it here. Can I please have a > > > >>> >> review and a sponsor for the following enhancement: > > > >>> >> > > > > > >>> >> > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v1 > > > >>> >> > > https://bugs.openjdk.java.net/browse/JDK-8200720 > > > >>> >> > > > > > >>> >> > > We at SAP have extended the thread dumps (obtained > > > >>> >> by jstack or jcmd) by several fields providing thread specific > > > >>> >> information. These extensions are quite popular with our support > > > >> team. > > > >>> >> With some knowledge of the architecture of the application, they > > > >> often > > > >>> >> allow for quick and simple diagnosis of a running system. Therefore > > > >> we > > > >>> >> would like to contribute these enhancements. > > > >>> >> > > > > > >>> >> > > I took a few simple examples here, namely cpu time, > > > >>> >> elapsed time since thread creation, bytes allocated and classes > > > >>> >> defined by the thread and the pthread-id or equivalent on platforms > > > >>> >> where it makes sense. Provided it is known how the application > > > >> should > > > >>> >> behave, a misbehaving thread can identified easily. > > > >>> >> > > > > > >>> >> > > There is no measurable overhead for this > > > >>> >> enhancement. However, it may be a problem that the format of the > > > >>> >> output is changed. Tools parsing the output may have to be changed. > > > >>> >> > > > > > >>> >> > > Here is an example of the output generated: > > > >>> >> > > > > > >>> >> > > ------------------------------------------------------ > > > >>> >> > > > > > >>> >> > > "main" #1 prio=5 os_prio=31 cpu=6300.65ms > > > >>> >> elapsed=123.28s allocated=242236760B defined_classes=1725 > > > >>> >> tid=0x00007fa13a806000 nid=0x1c03 pthread-id=0x109708000 waiting > > > >> on > > > >>> >> condition [0x0000000109707000] > > > >>> >> > > java.lang.Thread.State: TIMED_WAITING (sleeping) > > > >>> >> > > JavaThread state: _thread_blocked > > > >>> >> > > Thread: 0x00007fa13a806000 [0x1c03] State: > > > >>> >> _at_safepoint _has_called_back 0 _at_poll_safepoint 0 > > > >>> >> > > JavaThread state: _thread_blocked > > > >>> >> > > at java.lang.Thread.sleep(java.base/Native Method) > > > >>> >> > > ... > > > >>> >> > > ------------------------------------------------------ > > > >>> >> > > > > > >>> >> > > As mentioned above, we have a whole bunch of other > > > >>> >> enhancements to the thread dump similar to this one and would be > > > >>> >> willing to contribute them if there is any interest. > > > >>> >> > > > > > >>> >> > > Thanks and best regards, > > > >>> >> > > Gunter > > > >>> >> > > > > > >>> >> > > > > > >>> >> > > > > >>> >> > > > > >>> >> > > > > >>> >> > > > > >>> >> > > > > >>> >> > > > >>> > > > > >>> > > > > >>> > > > >>> > > > > > > > > > > > > > > > > From harsha.wardhana.b at oracle.com Mon Jun 25 07:04:29 2018 From: harsha.wardhana.b at oracle.com (Harsha Wardhana B) Date: Mon, 25 Jun 2018 12:34:29 +0530 Subject: RFR : JDK-8204661 - Show error 'Port already in use' in HashedPasswordFileTest.java In-Reply-To: References: <9fa7bd72-6476-b702-ba34-ec8fc572cfcb@oracle.com> <674a1a9a-0528-a2d1-f09f-b9d8724d62ec@oracle.com> <4d87b834-3b78-7b03-e90b-13385050fb19@oracle.com> Message-ID: <5e6d785d-d02a-5dca-7e1d-1cd3b2ffcff3@oracle.com> After internal discussions, we have decided to proceed with the current fix. Thank you Mandy, Daniel, Dan and David for the review. Harsha On Friday 22 June 2018 10:07 AM, Harsha Wardhana B wrote: > Hi Dan, > > The utility function Utils.getFreePort gets free port by creating > server socket with port 0. Instead of getting a free port before hand > (via Utils.getFreePort), I am letting the JMX agent choose the free > port. The risks associated with choosing free port has moved from test > lib to the product. > > Do you think this is something that should be handled at test-level? > Maybe refactor the test library (Utils.getFreePort)? > > Thanks > Harsha > > On Thursday 21 June 2018 07:28 PM, Daniel D. Daugherty wrote: >> > Port number of 0 is not handled at JMX or RMI layer. The given port >> number is passed onto the ServerSocket. >> > >> > >> https://docs.oracle.com/javase/10/docs/api/java/net/ServerSocket.html#%3Cinit%3E(int) >> >> >> Using a port number of 0 to get a free port has to be done very >> carefully >> to avoid running into other, more difficult to find issues. >> >> Jerry T. fix a few of these before he left us: >> >> ??? JDK-8182757 JDWP: Socket Transport handshake hangs on Solaris >> ??? https://bugs.openjdk.java.net/browse/JDK-8182757 >> >> ??? JDK-8178676 nsk/jvmti/AttachOnDemand/attach045 fails with Exception >> ??? https://bugs.openjdk.java.net/browse/JDK-8178676 >> >> ??? JDK-8188867 >> nsk/jdi/VirtualMachineManager/createVirtualMachine/createVM004 >> ??????????????? (and other tests) timeout do to socket problem >> ??? https://bugs.openjdk.java.net/browse/JDK-8188867 >> >> >> As an example of the difficulty, I tried to fix this bug: >> >> ??? JDK-8182307 Error during JRMP connection establishment >> ??? https://bugs.openjdk.java.net/browse/JDK-8182307 >> >> and had to back out my fix because it caused a couple of tests to fail. >> I'm tracking the redo here: >> >> ??? JDK-8193227 [REDO] 8182307 Error during JRMP connection >> establishment >> ??? https://bugs.openjdk.java.net/browse/JDK-8193227 >> >> and I haven't had an epiphany for how to fix it without breaking >> other tests (yet) so I just decommitted it from JDK11. >> >> >> I'm not saying that your current fix will be susceptible to very >> intermittent hangs on Solaris. I would have to take a very close >> look at it and I haven't done that yet. >> >> Dan >> >> >> On 6/21/18 2:47 AM, Harsha Wardhana B wrote: >>> >>> >>> On Thursday 21 June 2018 10:30 AM, mandy chung wrote: >>>> >>>> >>>> On 6/20/18 1:14 AM, Harsha Wardhana B wrote: >>>>> Hi, >>>>> >>>>> Please find the fix below for the bug >>>>> >>>>> JDK-8204661 : Show error 'Port already in use' in >>>>> HashedPasswordFileTest.java >>>>> >>>>> having webrev at, >>>>> >>>>> http://cr.openjdk.java.net/~hb/8204661/webrev.00/ >>>>> >>>>> The problem root-cause is discussed in the comments section of the >>>>> bug. >>>>> >>>>> The fix above lets the default agent pick a free port by passing >>>>> 'port=0' value and then reads the JMX Connector URL from Perf >>>>> Counters. >>>> >>>> This looks fine.? Please add this issue number to @bug. >>>> >>>> This is interesting.? I was not aware of setting port=0 will >>>> auto-assign a free port.? Do you know if it was added for testing >>>> purpose (which I assume so)? >>> Port number of 0 is not handled at JMX or RMI layer. The given port >>> number is passed onto the ServerSocket. >>> >>> https://docs.oracle.com/javase/10/docs/api/java/net/ServerSocket.html#%3Cinit%3E(int) >>> >>> >>>> >>>> For example >>>> test/jdk/sun/management/jmxremote/bootstrap/RmiBootstrapTest.java >>>> uses jdk.testlibrary.Utils.getFreePort() to get a free port number. >>>> >>>> Mandy >>> Harsha >>> >> > From sharath.ballal at oracle.com Mon Jun 25 08:22:10 2018 From: sharath.ballal at oracle.com (Sharath Ballal) Date: Mon, 25 Jun 2018 01:22:10 -0700 (PDT) Subject: RFR: JDK-8189429: SA: MacOSX: Replace the deprecated PT_ATTACH with PT_ATTACHEXC In-Reply-To: <0c93044e-7aaf-179b-c80c-45ff4d7e2712@oracle.com> References: <7abea33e-7fb0-fbe0-d463-8999ccdf54da@oracle.com> <0c93044e-7aaf-179b-c80c-45ff4d7e2712@oracle.com> Message-ID: Hi Jini, Changes in MacosxDebuggerLocal.m looks good. Thanks, Sharath -----Original Message----- From: Jini George Sent: Sunday, June 24, 2018 11:07 PM To: Erik Joelsson; serviceability-dev at openjdk.java.net; build-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net Subject: Re: RFR: JDK-8189429: SA: MacOSX: Replace the deprecated PT_ATTACH with PT_ATTACHEXC Hi Erik, Thank you very much for looking into this. I have addressed your comments. The latest webrev is at: http://cr.openjdk.java.net/~jgeorge/8189429/webrev.06/ Thank you, Jini. On 6/23/2018 3:31 AM, Erik Joelsson wrote: > Hello Jini, > > In general this looks pretty good, but it's also breaking some new > ground as it's adding generation of native source in the java gensrc > step. Mixing native code with the java code that the genrcs targets > and gensrc output directories are meant for seems ok for now, but may > cause trouble in the future. I'm going to accept it for now though. > > In Gensrc-jdk.hotspot.agent.gmk: Please put the new macosx stuff in > its own section (as delimited by the 80x # lines) and put that whole > thing inside a conditional for macosx. Also please break line 47 (for > recipe lines, indent with tab and 4 additional spaces for continuation [1]). > > In Lib-jdk.hotspot.agent.gmk: I believe adding extra src should be > enough as that will implicitly add the same directories as header dirs > by default. At least that's the intention. > > /Erik > > [1] http://openjdk.java.net/groups/build/doc/code-conventions.html > > On 2018-06-22 11:11, Jini George wrote: >> Hi all, >> >> [Including build-dev also since this includes build related changes]. >> >> Requesting reviews for: >> >> https://bugs.openjdk.java.net/browse/JDK-8189429 (SA: MacOSX: Replace >> the deprecated PT_ATTACH with PT_ATTACHEXC) >> >> Webrev: http://cr.openjdk.java.net/~jgeorge/8189429/webrev.04/ >> >> This is the follow-up solution for the temporary restoration of >> PT_ATTACH to fix https://bugs.openjdk.java.net/browse/JDK-8184042 >> (several serviceability/sa tests timed out on MacOS X), which was >> done in Oct 2017. The mails related to this are at: >> >> http://mail.openjdk.java.net/pipermail/serviceability-dev/2017-August >> /021702.html >> >> >> Changes as compared to the patch sent last year >> (http://cr.openjdk.java.net/~jgeorge/8184042/webrev.00/): >> >> * Addressed the review comments which were provided by Poonam, Dan, >> Dmitry. >> * A major change as compared to what was done last year is that the >> MIG generated files have been included as a part of the JDK build >> process. >> * The test case which was provided in the patch last year is no >> longer required since we have ClhsdbAttach.java testing the same >> functionality as a part of the SA testsuite now. >> * Other than that, some minor improvements have been done wrt error >> handling. >> >> The steps for the proposed solution have been provided in JBS. >> >> Testing: ALL the SA tests pass on MacOSX and the other Mach5 platforms. >> >> Thanks to Sharath, Robin, Gerard and Dan for looking into the changes >> and providing valuable comments. >> >> Thanks, >> Jini. > From gunter.haug at sap.com Mon Jun 25 08:26:41 2018 From: gunter.haug at sap.com (Haug, Gunter) Date: Mon, 25 Jun 2018 08:26:41 +0000 Subject: RFR(S): 8200720: Print additional information in thread dump (times, allocated bytes etc.) In-Reply-To: <3795D1AE-CBF4-4BC3-B707-ED77D4DE00D0@sap.com> References: <56817AB5-88FC-4FB9-A271-7206EA69ECA7@sap.com> <2728581E-0E32-4911-B1A9-79AD6CBA81B0@sap.com> <082ca98f-9a68-15a0-95b5-c4d5c733ab36@oracle.com> <933cdacc-9b6f-c8b5-e244-558f38e7888c@oracle.com> <55806aa8-f09b-e9a8-b512-b9676029c494@oracle.com> <0F497BC6-2569-4EF6-9B59-E24A8E1B3A37@sap.com> <5dc08f65e83f4374b8e1f73f218c6d6d@sap.com> <626701f8-b71b-58dc-f837-e5e1d81ef187@oracle.com> <19BE9680-3E6E-4620-B72D-D379F39C1C92@sap.com> <6ce47417-311b-e6cd-b8cd-d46eb7e04c2f@oracle.com> <3795D1AE-CBF4-4BC3-B707-ED77D4DE00D0@sap.com> Message-ID: <02AFA85A-36A7-49D0-A308-E509FFC39C97@sap.com> Hi David, Here is the updated webrev. I've included all the changes you suggested, except for default parameter (see original mail). http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v4/ Are you OK with it now? Thanks again, Gunter ?On 20.06.18, 17:40, "serviceability-dev on behalf of Haug, Gunter" wrote: Hi David, Thanks for taking the time to look into this! > "statistic info" is not very good grammatically. These things are either > statistics, or statistical information. So e.g. > > class ThreadStatistics >... OK, I'll change that! > const_cast(this)->cooked_allocated_bytes(); > > Why do we need a const cast to invoke a method on ourselves ?? cooked_allocated_bytes() is not declared const and we can't make it, as it uses OrderAccess::load_acquire(&_allocated_bytes). Therefor the const_cast. > src/hotspot/share/runtime/thread.hpp > > Please put the new #include in alphabetical order. Wilco! > I was expecting to see a default parameter used here rather than adding > an overload: > virtual void print_on(outputStream* st, bool print_extended_info = > false) const { print_on(st, print_extended_info); } Thread inherits from AllocatedObj (in dbg, at least) where virtual void AllocatedObj::print_on(outputStream*) const is declared. Some compilers, e.g. gcc, will complain that this is hidden by Thread::print_on(outputStream*, bool). Others, e.g. clang, are happy with that. We could implement a method with a different name but would that be nicer? > 2179 static void print_on(outputStream* st, bool print_stacks, bool > internal_format, bool print_concurrent_locks, bool extended_thread_info); > > Again I expected to see a default parameter here - but I didn't check if > all callers themselves take the new parameter. ?? This one has just one single caller, so I think it's OK. > src/hotspot/share/runtime/vm_operations.hpp >... I'll change it to conform to the guidelines. > src/hotspot/share/runtime/threadStatisticInfo.hpp > > There are no includes in this file. It should include the appropriate > std header for type definitions, and os.hpp. Agreed, I'll add the includes. > I'm not sure if class ThreadStatisticInfo needs an allocation type as a > super type. > > Might be worth adding a constructor to give a default value so that you > can tell if the statistics have been initialized when they appear in the > printout. ? ThreadStatisticInfo is a data member of Thread. Currently it is initialized in Thread::Thread, but I can add a constructor, if you prefer. Thanks again, Gunter On 20.06.18, 09:15, "David Holmes" wrote: Hi Gunter, On 19/06/2018 8:51 PM, Haug, Gunter wrote: > Hi all, > > Thanks Chris and Christoph for the reviews! Christoph, I'll incorporate the little improvement you suggested. > David, are you OK with the change as well? Sorry I'd lost track of this one a bit ... The overall approach now seems okay. Some naming/terminology issues: "statistic info" is not very good grammatically. These things are either statistics, or statistical information. So e.g. class ThreadStatistics ThreadStatistics& statistics() { return _statistics; } etc. "extended_thread_info" should really be print_extended_info (similar to print_concurrent_locks). You don't need "thread" in there when this is always part of a Thread related API. src/hotspot/share/runtime/thread.cpp const_cast(this)-> Why do we need a const cast to invoke a method on ourselves ?? --- src/hotspot/share/runtime/thread.hpp Please put the new #include in alphabetical order. 641 // Printing 642 void print_on(outputStream* st, bool extended_thread_info) const; 643 virtual void print_on(outputStream* st) const { print_on(st, false); } I was expecting to see a default parameter used here rather than adding an overload: virtual void print_on(outputStream* st, bool print_extended_info = false) const { print_on(st, print_extended_info); } 2179 static void print_on(outputStream* st, bool print_stacks, bool internal_format, bool print_concurrent_locks, bool extended_thread_info); Again I expected to see a default parameter here - but I didn't check if all callers themselves take the new parameter. ?? --- src/hotspot/share/runtime/vm_operations.hpp 376 VM_PrintThreads() 377 { _out = tty; _print_concurrent_locks = PrintConcurrentLocks; ; _extended_thread_info = false; } 378 VM_PrintThreads(outputStream* out, bool print_concurrent_locks, bool extended_thread_info) 379 { _out = out; _print_concurrent_locks = print_concurrent_locks; _extended_thread_info = extended_thread_info; } 380 VMOp_Type type() const 381 { return VMOp_PrintThreads; } Style nits: either keep everything on one line as before (though I agree the lines are now too long) or else the style should be: 380 VMOp_Type type() const { 381 return VMOp_PrintThreads; 382 } Also unclear why (existing) VM_PrintThreads constructor doesn't use initializer list (like VM_PrintMetadata below it) or default parameters? --- src/hotspot/share/runtime/threadStatisticInfo.hpp There are no includes in this file. It should include the appropriate std header for type definitions, and os.hpp. I'm not sure if class ThreadStatisticInfo needs an allocation type as a super type. Might be worth adding a constructor to give a default value so that you can tell if the statistics have been initialized when they appear in the printout. ? --- Thanks, David > Thanks again, > Gunter > > On 12.06.18, 01:13, "Chris Plummer" wrote: > > Hi Gunter, > > The changes look fine. I can live with the options parsing in > attachListener.cpp. As you point out, unrecognized options were already > silently ignored. > > thanks, > > Chris > > On 6/8/18 7:05 AM, Haug, Gunter wrote: > > Hi all, > > > > thanks a lot for all the input! I have prepared a new version of the webrev incorporating the suggestions you made (at least I tried): > > > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v3/ > > > > This version outputs the thread times unconditionally while the other information is guarded by a flags. I think, most participants found the thread times the most valuable information and had no (strong) objections to adding them unconditionally. > > > > @David > > Implementation is much simpler with conditional output only for JavaThreads. I could get rid of the terrible hack (changing the flag) without having to change too many source files. > > > > The information on allocated bytes is present in the Thread class already before this proposed change, I just print it. It might be sensible to move _allocated_bytes and the respective methods to the ThreadStatisticInfo class as G?tz suggested. I haven?t done that in the current version, though. > > > > @David and Thomas > > I've removed the pthread-id output. I'm unsure myself what it could be good for. Anyway, we could add it with a separate change (with a better implementation) if there is a need to. > > > > @Chris > > As you have written, the user has no direct contact to the attach listener of the VM and jstack won't misinterpret e.g. -help. I agree that the parsing in attachListener.cpp could be more robust. However, it hasn't been so far either. All that is done in the current implementation is a strcmp to -l everything else will be silently ignored. I can try to make this more robust or we could say that the new output is only available via jcmd. OTH nothing bad can happen with the current version of the proposed change, so we could also leave it as it is. What would you prefer? > > > > @Kirk and Thomas > > Implementing a new diagnostic command is certainly well worth a thought. However, it appears to me that it's not necessary in this case. There are already flags to jstack and jcmd Thread.print, so it's not that uncommon. The amount of information that would be available by a new diagnostic command and that is now added to the thread dump is quite small and I think that is tolerable. Moreover, and most importantly, our support team is use to find the information in the thread dump and it would make things easier to us if I could leave it there. > > > > Thanks again and have a nice weekend, > > Gunter > > > > > > On 06.06.18, 04:56, "David Holmes" wrote: > > > > Hi Goetz, > > > > On 5/06/2018 11:07 PM, Lindenmaier, Goetz wrote: > > > Hi > > > > > > this discussion touched a lot of points so far, which seem > > > to lead to different conclusions. > > > > > > I think we should look at the values printed: > > > > > > 1. cpu=6300.65ms elapsed=123.28s > > > Overhead > > > cpu time: > > > * system calls at thread dump time > > > elapsed time: > > > * 1 system call at thread creation time > > > * 1 64-bit field per thread for the thread start time > > > * 1 system call at thread dump time > > > > > > As I understand, JDK-8143176 would have had to get the > > > time at each locking, which is much more time critical > > > than doing this during thread creation. While > > > > Correct. > > > > > the time a lock was held would be much more useful, > > > the ratio here, combined with knowledge about the application, > > > could lead to the conclusion that the thread is wrongly > > > blocked at much lower cost. > > > > Agreed. I see no issue with unconditionally adding this information as > > it should address some of the concerns of 8143176 as well. > > > > > > > > 2. pthread-id=0x109708000 > > > Overhead: > > > * a field access at thread dump time. Negligible I would say. > > > > I'd overlooked this addition, but it is of course the chunk of ifdef > > code in osThread.cpp that I objected to. The problem I have here is the > > convoluted mess of "thread identifiers" that we already have. We > > currently print: > > > > st->print("tid=" INTPTR_FORMAT " ", p2i(this)); > > > > which is just the address of the Thread/JavaThread object. Then: > > > > st->print("nid=0x%x ", thread_id()); > > > > where 'n' is supposed to represent "native thread id", which is: > > - linux: kernel thread id from syscall gettid > > - solaris: thread library identity from thr_create() or thr_self() > > - windows: thread id from _beginthreadex > > - OS X: kernel thread id from pthread_mach_thread_np(pthread_self()); > > - Other BSD: kernel (?) thread id from syscall thr_self or getthrid > > - AIX: thread library identity from pthread_create() or pthread_self() > > > > It's telling that the code Gunter added gets the thread library id on > > linux, but a "kernel thread id" on other platforms! Do we want to see a > > thread library id and a kernel thread id? IIRC nid is supposed be the id > > you need to see the thread in a debugger. In which case I'm unclear what > > role the thread id Gunter wants to add is playing? I don't think we need > > to see kernel ids in general, and pthread-id isn't useful for debugging > > is it? > > > > > > > > 3. allocated=242236760B defined_classes=1725 > > > Overhead: > > > * 1 64-bit field per thread. > > > * counter increment on class creation > > > > > > Especially defined_classes seems to be controversial. > > > As this only makes sense for Java threads, this could > > > be printed in a line of it's own in Threads::print_on_error() > > > (which already gets a flag to customize for jstack: > > > print_concurrent_locks) by calling a dedicated function in Thread. > > > No need for flag PrintExtendedThreadInfo thus. > > > > > > But I would propose to drop this information as it is > > > too controversial. > > > > Agreed. > > > > > This leaves the times and the pthread-id to be decided whether > > > they are worth the cost. If so, I think they should be printed > > > unconditional. > > > > Agreed. With some refinement of the "thread id" issue. > > > > > If a flag is required to avoid bloating the info in the default case, > > > I would include more infos, as os_prio and nid under that flag. > > > > Of course those things exist for good reason. The native id is supposed > > to be what aids in you matching what you see in our thread dumps with > > what you see in a debugger. Priority is less of an issue these days, but > > there was a time ... ;-) > > > > > As jstack is deprecated, changes to jstack could be skipped always > > > disabling the new printouts. > > > > > > Finally, I would propose to move _allocated_bytes into > > > threadStatisticInfo.hpp. > > > > Didn't that part get dropped? (You don't need it for the time info or > > thread id). That's a GC logging issue to me. > > > > Thanks, > > David > > > > > Best regards, > > > Goetz. > > > > > > > > > > > > > > >> -----Original Message----- > > >> From: serviceability-dev [mailto:serviceability-dev- > > >> bounces at openjdk.java.net] On Behalf Of David Holmes > > >> Sent: Dienstag, 5. Juni 2018 04:53 > > >> To: Haug, Gunter ; Chris Plummer > > >> ; serviceability-dev > >> dev at openjdk.java.net>; Langer, Christoph ; > > >> hotspot-runtime-dev at openjdk.java.net > > >> Subject: Re: RFR(S): 8200720: Print additional information in thread dump > > >> (times, allocated bytes etc.) > > >> > > >> Hi Gunter, > > >> > > >> On 5/06/2018 1:27 AM, Haug, Gunter wrote: > > >>> Hi David, Chris, > > >>> > > >>> would it be an option to unconditionally print the additional information? > > >> Regardless which way this is implemented it will be rather complicated and it > > >> only switches on/off a few field in the thread dump. > > >> > > >> I'm not convinced this is all suitable information for a thread stack > > >> dump. I can see the time information being useful if using the dump to > > >> try and diagnoze a hang or responsiveness issue. But the allocated-bytes > > >> and classes-defined just doesn't seem useful in the context of a thread > > >> dump to me. Anyone interested in allocation stats is going to be looking > > >> at GC logs etc which is where this belongs. Ditto the class-stats belong > > >> in some kind of classloading logging IMHO. > > >> > > >> I'm very reluctant to burden the implementation with capturing these > > >> kinds of stats, just to use for diagnostic purposes that may not even be > > >> asked for. I'm very much in the "you shouldn't pay for what you don't > > >> use" camp in that regard. (See my comments in JDK-8143176 referenced by > > >> Sean.) > > >> > > >> The ThreadStatisticInfo adds overhead to every thread object - and I'd > > >> have to suspect there could be overlap with whatever flight recorder > > >> sticks in there too. (Thread has become even more bloated in recent time!). > > >> > > >> Cheers, > > >> David > > >> > > >> > > >>> Thanks, > > >>> Gunter > > >>> > > >>> On 04.06.18, 01:13, "David Holmes" wrote: > > >>> > > >>> Hi Gunter, Chris, > > >>> > > >>> Sorry just trying to catch up and this is only a partial look so far ... > > >>> > > >>> BTW these changes are not limited to serviceability code so adding in > > >>> runtime team as well. > > >>> > > >>> On 2/06/2018 9:12 AM, Chris Plummer wrote: > > >>> > Hi Gunter, > > >>> > > > >>> > On 6/1/18 3:17 AM, Haug, Gunter wrote: > > >>> >> Hi Chris, > > >>> >> > > >>> >> thanks for looking into this! > > >>> >> > > >>> >> Re the synchronization: The value is stored only in a VM operation at > > >>> >> a safepoint by the VM thread. I was of the opinion, that this could > > >>> >> not be interrupted by a second VM operation (of the same type). Or > > >> am > > >>> >> I missing something else? > > >>> > I was really thinking more about the temporary changing of > > >>> > PrintExtendedThreadInfo, not the value stored in the VMOp. You may > > >> be be > > >>> > correct that no more than one VMOp is executing, but while it is > > >>> > executing it is has changed the value of PrintExtendedThreadInfo, > > >> which > > >>> > might have an impact on anything else that triggers printing thread info > > >>> > while the VMOp is executing. > > >>> > > >>> Even if nothing else can possibly be running during the safepoint I find > > >>> it extremely bad form to change a command-line flag in this way, > > >>> particularly from a safepoint! > > >>> > > >>> If this flag is intended to be dynamically enabled as part of a dcmd > > >>> then it should also be a Manageable flag IMHO. > > >>> > > >>> That said ... > > >>> > > >>> >> I did think about passing an argument to the various print_on > > >> member > > >>> >> functions of the thread classes, but this would require rather > > >>> >> extensive code changes for a rather tiny extension. Therefore I feel > > >>> >> doing it like this is the lesser evil. > > >>> > > >>> ... it's obviously not truly a global setting, but one that is needed on > > >>> a per-print-request basis. The flag is just the default, but if the > > >>> default is off you still want to enable extended printing on a > > >>> per-request basis. > > >>> > > >>> The current print_on mechanics is not set up for this kind of > > >>> flexibility. I think this needs more thought. > > >>> > > >>> --- > > >>> > > >>> Re osThread.cpp shared code changes ... I really hate to see all the > > >>> ifdefs in there. Please add a pd_print_on function to support this if > > >>> you truly want platform specific information. > > >>> > > >>> --- > > >>> > > >>> threadStatisticInfo.hpp > > >>> > > >>> typo: getElepsedTime() -> getElapsedTime() > > >>> > > >>> Thanks, > > >>> David > > >>> > > >>> > > >>> > > >>> >> > > >>> >> Re thread_dump(): I think it's correct or, well, at least it works ;-) > > >>> >> In fact jstack will transfer the "-e" and "-l" in only one string, > > >>> >> i.e. op->arg(0). > > >>> > So you are saying that op->arg(0) is "-e -l" when using jstack? I think > > >>> > you really need to clean up the parsing. As it stands right now passing, > > >>> > I get the feeling that if a user erroneously asks for help by using > > >>> > "jcmd Thread.Print -help", it will end up executing with -e an -l > > >>> > enabled, and no failure for the invalid "-help" option. > > >>> >> Christoph has already explained my reasoning. But I agree, it's not > > >>> >> nice and I would be happy to do it like Christoph suggested. > > >>> > I'll respond separately to his suggestion. > > >>> > > > >>> > thanks, > > >>> > > > >>> > Chris > > >>> >> > > >>> >> And typo fixed, sorry. > > >>> >> > > >>> >> Thanks again, > > >>> >> Gunter > > >>> >> > > >>> >> On 01.06.18, 00:03, "Chris Plummer" > > >> wrote: > > >>> >> > > >>> >> Hi Gunter, > > >>> >> globals.hpp: fix typo "informatiuon" > > >>> >> I worry a little bit about the synchronizing (if that's the right > > >>> >> word) > > >>> >> of PrintExtendedThreadInfo and the dcmd's -e flag. When using -e, > > >>> >> you > > >>> >> are temporarily enabling PrintExtendedThreadInfo if it was false. > > >>> >> This > > >>> >> temporarily changes the behavior of thread dumps, and could > > >>> >> impact other > > >>> >> uses that happen in parallel. Also, could two simultaneous uses > > >>> >> of -e > > >>> >> result in PrintExtendedThreadInfo not getting restored properly? > > >>> >> thread_dump() doesn't look right. It looks like you are iterating > > >>> >> char > > >>> >> by char over the argument, and expect something like "-el" to be > > >>> >> specified rather then "-e -l". The loop should be iterating over > > >>> >> op->arg(i), not op->arg(0)[i]. > > >>> >> The rest of the changes look fine. > > >>> >> thanks, > > >>> >> Chris > > >>> >> On 5/30/18 8:12 AM, Haug, Gunter wrote: > > >>> >> > Hi all, > > >>> >> > > > >>> >> > As Chris proposed, I have made an the extended output > > >>> >> switchable. There is an VM flag (PrintExtendedThreadInfo), which is > > >>> >> false by default. Moreover, there is an Option (-e) which can be used > > >>> >> with jcmd Thread.print as well as with jstack. The -e option > > >>> >> essentially sets PrintExtendedThreadInfo true just for the respective > > >>> >> thread dump. > > >>> >> > > > >>> >> > Here is the updated webrev: > > >>> >> > > > >>> >> > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v2 > > >>> >> > > > >>> >> > (https://bugs.openjdk.java.net/browse/JDK-8200720) > > >>> >> > > > >>> >> > Thanks, > > >>> >> > Gunter > > >>> >> > > > >>> >> > > > >>> >> > On 02.05.18, 17:07, "serviceability-dev on behalf of Haug, > > >>> >> Gunter" > >>> >> gunter.haug at sap.com> wrote: > > >>> >> > > > >>> >> > Hi Chris, > > >>> >> > > > >>> >> > Thanks for looking into this. > > >>> >> > You're right, there is a little more we have. We have > > >>> >> implemented an IO tracing mechanism which - rather as a byproduct - > > >>> >> keeps track of bytes read and written per thread. However, this of > > >>> >> course requires changes not only in hotspot. We would be happy to > > >>> >> contribute this as well, but this is a far bigger change and will > > >>> >> probably lead to a far bigger discussion. Anyway, with the number of > > >>> >> bytes read, the number of classes defined doesn't look that arbitrary > > >>> >> anymore, as one can correlate IO to class loading. > > >>> >> > > > >>> >> > Regarding the verbose option I think that's a good idea! > > >>> >> > > > >>> >> > Thanks again, > > >>> >> > Gunter > > >>> >> > > > >>> >> > On 01.05.18, 22:55, "Chris Plummer" > > >>> >> wrote: > > >>> >> > > > >>> >> > Hi Gunter, > > >>> >> > > > >>> >> > The output you are adding is all useful. I think the > > >>> >> question is (and > > >>> >> > I'd like to see a few people chime in on this for this > > >>> >> review) is > > >>> >> > whether or not all of it is the appropriate for a > > >>> >> thread's stack dump. > > >>> >> > For example, defined_classes is on the fringe of what > > >>> >> I would call > > >>> >> > generally useful info in a stack dump. Sure, there > > >>> >> might be that rare > > >>> >> > case when you need it, but how often compared to other > > >>> >> useful info > > >>> >> > maintained on a per thread basis. How many other bits > > >>> >> of useful info are > > >>> >> > not being printed in favor of defined_classes? It > > >>> >> seems you have more in > > >>> >> > the queue. How many? I'm worried about how cluttered > > >>> >> the stack dumps > > >>> >> > will get. Maybe we should add some sort of verbose > > >>> >> thread dumping > > >>> >> > option. Just a thought. > > >>> >> > > > >>> >> > As for the implementation, overall it looks good, but > > >>> >> I can't speak to > > >>> >> > whether or not you are doing proper accounting of > > >>> >> defined_classes and > > >>> >> > bytes allocated. You'll need input from someone with > > >>> >> more knowledge of > > >>> >> > those areas. We'll also need to do some testing to > > >>> >> make sure tool tests > > >>> >> > are not impacted. > > >>> >> > > > >>> >> > thanks, > > >>> >> > > > >>> >> > Chris > > >>> >> > > > >>> >> > On 4/30/18 2:51 AM, Haug, Gunter wrote: > > >>> >> > > Hi, > > >>> >> > > > > >>> >> > > this is an update to an RFR I posted on hotspot-dev, > > >>> >> but it is probably more suitable to post it here. Can I please have a > > >>> >> review and a sponsor for the following enhancement: > > >>> >> > > > > >>> >> > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v1 > > >>> >> > > https://bugs.openjdk.java.net/browse/JDK-8200720 > > >>> >> > > > > >>> >> > > We at SAP have extended the thread dumps (obtained > > >>> >> by jstack or jcmd) by several fields providing thread specific > > >>> >> information. These extensions are quite popular with our support > > >> team. > > >>> >> With some knowledge of the architecture of the application, they > > >> often > > >>> >> allow for quick and simple diagnosis of a running system. Therefore > > >> we > > >>> >> would like to contribute these enhancements. > > >>> >> > > > > >>> >> > > I took a few simple examples here, namely cpu time, > > >>> >> elapsed time since thread creation, bytes allocated and classes > > >>> >> defined by the thread and the pthread-id or equivalent on platforms > > >>> >> where it makes sense. Provided it is known how the application > > >> should > > >>> >> behave, a misbehaving thread can identified easily. > > >>> >> > > > > >>> >> > > There is no measurable overhead for this > > >>> >> enhancement. However, it may be a problem that the format of the > > >>> >> output is changed. Tools parsing the output may have to be changed. > > >>> >> > > > > >>> >> > > Here is an example of the output generated: > > >>> >> > > > > >>> >> > > ------------------------------------------------------ > > >>> >> > > > > >>> >> > > "main" #1 prio=5 os_prio=31 cpu=6300.65ms > > >>> >> elapsed=123.28s allocated=242236760B defined_classes=1725 > > >>> >> tid=0x00007fa13a806000 nid=0x1c03 pthread-id=0x109708000 waiting > > >> on > > >>> >> condition [0x0000000109707000] > > >>> >> > > java.lang.Thread.State: TIMED_WAITING (sleeping) > > >>> >> > > JavaThread state: _thread_blocked > > >>> >> > > Thread: 0x00007fa13a806000 [0x1c03] State: > > >>> >> _at_safepoint _has_called_back 0 _at_poll_safepoint 0 > > >>> >> > > JavaThread state: _thread_blocked > > >>> >> > > at java.lang.Thread.sleep(java.base/Native Method) > > >>> >> > > ... > > >>> >> > > ------------------------------------------------------ > > >>> >> > > > > >>> >> > > As mentioned above, we have a whole bunch of other > > >>> >> enhancements to the thread dump similar to this one and would be > > >>> >> willing to contribute them if there is any interest. > > >>> >> > > > > >>> >> > > Thanks and best regards, > > >>> >> > > Gunter > > >>> >> > > > > >>> >> > > > > >>> >> > > > >>> >> > > > >>> >> > > > >>> >> > > > >>> >> > > > >>> >> > > >>> > > > >>> > > > >>> > > >>> > > > > > > > > From david.holmes at oracle.com Mon Jun 25 08:38:30 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 25 Jun 2018 18:38:30 +1000 Subject: RFR(S): 8200720: Print additional information in thread dump (times, allocated bytes etc.) In-Reply-To: <02AFA85A-36A7-49D0-A308-E509FFC39C97@sap.com> References: <56817AB5-88FC-4FB9-A271-7206EA69ECA7@sap.com> <2728581E-0E32-4911-B1A9-79AD6CBA81B0@sap.com> <082ca98f-9a68-15a0-95b5-c4d5c733ab36@oracle.com> <933cdacc-9b6f-c8b5-e244-558f38e7888c@oracle.com> <55806aa8-f09b-e9a8-b512-b9676029c494@oracle.com> <0F497BC6-2569-4EF6-9B59-E24A8E1B3A37@sap.com> <5dc08f65e83f4374b8e1f73f218c6d6d@sap.com> <626701f8-b71b-58dc-f837-e5e1d81ef187@oracle.com> <19BE9680-3E6E-4620-B72D-D379F39C1C92@sap.com> <6ce47417-311b-e6cd-b8cd-d46eb7e04c2f@oracle.com> <3795D1AE-CBF4-4BC3-B707-ED77D4DE00D0@sap.com> <02AFA85A-36A7-49D0-A308-E509FFC39C97@sap.com> Message-ID: <1e7e234a-91ea-2c14-9634-08f78b90b225@oracle.com> Hi Gunter, On 25/06/2018 6:26 PM, Haug, Gunter wrote: > Hi David, > > Here is the updated webrev. I've included all the changes you suggested, except for default parameter (see original mail). > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v4/ > > Are you OK with it now? You don't need the setStartTime setter as by definition a "start time" is only set once. (Though "start" may not be the right term here.) Otherwise changes seem okay. Note however that you are adding a product flag so that means you need to file a CSR request. Thanks, David > Thanks again, > Gunter > > ?On 20.06.18, 17:40, "serviceability-dev on behalf of Haug, Gunter" wrote: > > Hi David, > > Thanks for taking the time to look into this! > > > "statistic info" is not very good grammatically. These things are either > > statistics, or statistical information. So e.g. > > > > class ThreadStatistics > >... > > OK, I'll change that! > > > const_cast(this)->cooked_allocated_bytes(); > > > > Why do we need a const cast to invoke a method on ourselves ?? > > cooked_allocated_bytes() is not declared const and we can't make it, as it uses OrderAccess::load_acquire(&_allocated_bytes). Therefor the const_cast. > > > src/hotspot/share/runtime/thread.hpp > > > > Please put the new #include in alphabetical order. > > Wilco! > > > I was expecting to see a default parameter used here rather than adding > > an overload: > > virtual void print_on(outputStream* st, bool print_extended_info = > > false) const { print_on(st, print_extended_info); } > > Thread inherits from AllocatedObj (in dbg, at least) where virtual void AllocatedObj::print_on(outputStream*) const is declared. Some compilers, e.g. gcc, will complain that this is hidden by Thread::print_on(outputStream*, bool). Others, e.g. clang, are happy with that. We could implement a method with a different name but would that be nicer? > > > 2179 static void print_on(outputStream* st, bool print_stacks, bool > > internal_format, bool print_concurrent_locks, bool extended_thread_info); > > > > Again I expected to see a default parameter here - but I didn't check if > > all callers themselves take the new parameter. ?? > > This one has just one single caller, so I think it's OK. > > > src/hotspot/share/runtime/vm_operations.hpp > >... > > I'll change it to conform to the guidelines. > > > src/hotspot/share/runtime/threadStatisticInfo.hpp > > > > There are no includes in this file. It should include the appropriate > > std header for type definitions, and os.hpp. > > Agreed, I'll add the includes. > > > > I'm not sure if class ThreadStatisticInfo needs an allocation type as a > > super type. > > > > Might be worth adding a constructor to give a default value so that you > > can tell if the statistics have been initialized when they appear in the > > printout. ? > > ThreadStatisticInfo is a data member of Thread. Currently it is initialized in Thread::Thread, but I can add a constructor, if you prefer. > > Thanks again, > Gunter > > > > On 20.06.18, 09:15, "David Holmes" wrote: > > Hi Gunter, > > On 19/06/2018 8:51 PM, Haug, Gunter wrote: > > Hi all, > > > > Thanks Chris and Christoph for the reviews! Christoph, I'll incorporate the little improvement you suggested. > > David, are you OK with the change as well? > > Sorry I'd lost track of this one a bit ... > > The overall approach now seems okay. > > Some naming/terminology issues: > > "statistic info" is not very good grammatically. These things are either > statistics, or statistical information. So e.g. > > class ThreadStatistics > > ThreadStatistics& statistics() { return _statistics; } > > etc. > > "extended_thread_info" should really be print_extended_info (similar to > print_concurrent_locks). You don't need "thread" in there when this is > always part of a Thread related API. > > > src/hotspot/share/runtime/thread.cpp > > const_cast(this)-> > > Why do we need a const cast to invoke a method on ourselves ?? > > --- > > src/hotspot/share/runtime/thread.hpp > > Please put the new #include in alphabetical order. > > 641 // Printing > 642 void print_on(outputStream* st, bool extended_thread_info) const; > 643 virtual void print_on(outputStream* st) const { print_on(st, > false); } > > I was expecting to see a default parameter used here rather than adding > an overload: > > virtual void print_on(outputStream* st, bool print_extended_info = > false) const { print_on(st, print_extended_info); } > > > 2179 static void print_on(outputStream* st, bool print_stacks, bool > internal_format, bool print_concurrent_locks, bool extended_thread_info); > > Again I expected to see a default parameter here - but I didn't check if > all callers themselves take the new parameter. ?? > > --- > > src/hotspot/share/runtime/vm_operations.hpp > > 376 VM_PrintThreads() > 377 { _out = tty; _print_concurrent_locks = PrintConcurrentLocks; ; > _extended_thread_info = false; } > 378 VM_PrintThreads(outputStream* out, bool print_concurrent_locks, > bool extended_thread_info) > 379 { _out = out; _print_concurrent_locks = print_concurrent_locks; > _extended_thread_info = extended_thread_info; } > 380 VMOp_Type type() const > 381 { return VMOp_PrintThreads; } > > Style nits: either keep everything on one line as before (though I agree > the lines are now too long) or else the style should be: > > 380 VMOp_Type type() const { > 381 return VMOp_PrintThreads; > 382 } > > Also unclear why (existing) VM_PrintThreads constructor doesn't use > initializer list (like VM_PrintMetadata below it) or default parameters? > > --- > > src/hotspot/share/runtime/threadStatisticInfo.hpp > > There are no includes in this file. It should include the appropriate > std header for type definitions, and os.hpp. > > I'm not sure if class ThreadStatisticInfo needs an allocation type as a > super type. > > Might be worth adding a constructor to give a default value so that you > can tell if the statistics have been initialized when they appear in the > printout. ? > > --- > > Thanks, > David > > > > Thanks again, > > Gunter > > > > On 12.06.18, 01:13, "Chris Plummer" wrote: > > > > Hi Gunter, > > > > The changes look fine. I can live with the options parsing in > > attachListener.cpp. As you point out, unrecognized options were already > > silently ignored. > > > > thanks, > > > > Chris > > > > On 6/8/18 7:05 AM, Haug, Gunter wrote: > > > Hi all, > > > > > > thanks a lot for all the input! I have prepared a new version of the webrev incorporating the suggestions you made (at least I tried): > > > > > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v3/ > > > > > > This version outputs the thread times unconditionally while the other information is guarded by a flags. I think, most participants found the thread times the most valuable information and had no (strong) objections to adding them unconditionally. > > > > > > @David > > > Implementation is much simpler with conditional output only for JavaThreads. I could get rid of the terrible hack (changing the flag) without having to change too many source files. > > > > > > The information on allocated bytes is present in the Thread class already before this proposed change, I just print it. It might be sensible to move _allocated_bytes and the respective methods to the ThreadStatisticInfo class as G?tz suggested. I haven?t done that in the current version, though. > > > > > > @David and Thomas > > > I've removed the pthread-id output. I'm unsure myself what it could be good for. Anyway, we could add it with a separate change (with a better implementation) if there is a need to. > > > > > > @Chris > > > As you have written, the user has no direct contact to the attach listener of the VM and jstack won't misinterpret e.g. -help. I agree that the parsing in attachListener.cpp could be more robust. However, it hasn't been so far either. All that is done in the current implementation is a strcmp to -l everything else will be silently ignored. I can try to make this more robust or we could say that the new output is only available via jcmd. OTH nothing bad can happen with the current version of the proposed change, so we could also leave it as it is. What would you prefer? > > > > > > @Kirk and Thomas > > > Implementing a new diagnostic command is certainly well worth a thought. However, it appears to me that it's not necessary in this case. There are already flags to jstack and jcmd Thread.print, so it's not that uncommon. The amount of information that would be available by a new diagnostic command and that is now added to the thread dump is quite small and I think that is tolerable. Moreover, and most importantly, our support team is use to find the information in the thread dump and it would make things easier to us if I could leave it there. > > > > > > Thanks again and have a nice weekend, > > > Gunter > > > > > > > > > On 06.06.18, 04:56, "David Holmes" wrote: > > > > > > Hi Goetz, > > > > > > On 5/06/2018 11:07 PM, Lindenmaier, Goetz wrote: > > > > Hi > > > > > > > > this discussion touched a lot of points so far, which seem > > > > to lead to different conclusions. > > > > > > > > I think we should look at the values printed: > > > > > > > > 1. cpu=6300.65ms elapsed=123.28s > > > > Overhead > > > > cpu time: > > > > * system calls at thread dump time > > > > elapsed time: > > > > * 1 system call at thread creation time > > > > * 1 64-bit field per thread for the thread start time > > > > * 1 system call at thread dump time > > > > > > > > As I understand, JDK-8143176 would have had to get the > > > > time at each locking, which is much more time critical > > > > than doing this during thread creation. While > > > > > > Correct. > > > > > > > the time a lock was held would be much more useful, > > > > the ratio here, combined with knowledge about the application, > > > > could lead to the conclusion that the thread is wrongly > > > > blocked at much lower cost. > > > > > > Agreed. I see no issue with unconditionally adding this information as > > > it should address some of the concerns of 8143176 as well. > > > > > > > > > > > 2. pthread-id=0x109708000 > > > > Overhead: > > > > * a field access at thread dump time. Negligible I would say. > > > > > > I'd overlooked this addition, but it is of course the chunk of ifdef > > > code in osThread.cpp that I objected to. The problem I have here is the > > > convoluted mess of "thread identifiers" that we already have. We > > > currently print: > > > > > > st->print("tid=" INTPTR_FORMAT " ", p2i(this)); > > > > > > which is just the address of the Thread/JavaThread object. Then: > > > > > > st->print("nid=0x%x ", thread_id()); > > > > > > where 'n' is supposed to represent "native thread id", which is: > > > - linux: kernel thread id from syscall gettid > > > - solaris: thread library identity from thr_create() or thr_self() > > > - windows: thread id from _beginthreadex > > > - OS X: kernel thread id from pthread_mach_thread_np(pthread_self()); > > > - Other BSD: kernel (?) thread id from syscall thr_self or getthrid > > > - AIX: thread library identity from pthread_create() or pthread_self() > > > > > > It's telling that the code Gunter added gets the thread library id on > > > linux, but a "kernel thread id" on other platforms! Do we want to see a > > > thread library id and a kernel thread id? IIRC nid is supposed be the id > > > you need to see the thread in a debugger. In which case I'm unclear what > > > role the thread id Gunter wants to add is playing? I don't think we need > > > to see kernel ids in general, and pthread-id isn't useful for debugging > > > is it? > > > > > > > > > > > 3. allocated=242236760B defined_classes=1725 > > > > Overhead: > > > > * 1 64-bit field per thread. > > > > * counter increment on class creation > > > > > > > > Especially defined_classes seems to be controversial. > > > > As this only makes sense for Java threads, this could > > > > be printed in a line of it's own in Threads::print_on_error() > > > > (which already gets a flag to customize for jstack: > > > > print_concurrent_locks) by calling a dedicated function in Thread. > > > > No need for flag PrintExtendedThreadInfo thus. > > > > > > > > But I would propose to drop this information as it is > > > > too controversial. > > > > > > Agreed. > > > > > > > This leaves the times and the pthread-id to be decided whether > > > > they are worth the cost. If so, I think they should be printed > > > > unconditional. > > > > > > Agreed. With some refinement of the "thread id" issue. > > > > > > > If a flag is required to avoid bloating the info in the default case, > > > > I would include more infos, as os_prio and nid under that flag. > > > > > > Of course those things exist for good reason. The native id is supposed > > > to be what aids in you matching what you see in our thread dumps with > > > what you see in a debugger. Priority is less of an issue these days, but > > > there was a time ... ;-) > > > > > > > As jstack is deprecated, changes to jstack could be skipped always > > > > disabling the new printouts. > > > > > > > > Finally, I would propose to move _allocated_bytes into > > > > threadStatisticInfo.hpp. > > > > > > Didn't that part get dropped? (You don't need it for the time info or > > > thread id). That's a GC logging issue to me. > > > > > > Thanks, > > > David > > > > > > > Best regards, > > > > Goetz. > > > > > > > > > > > > > > > > > > > >> -----Original Message----- > > > >> From: serviceability-dev [mailto:serviceability-dev- > > > >> bounces at openjdk.java.net] On Behalf Of David Holmes > > > >> Sent: Dienstag, 5. Juni 2018 04:53 > > > >> To: Haug, Gunter ; Chris Plummer > > > >> ; serviceability-dev > > >> dev at openjdk.java.net>; Langer, Christoph ; > > > >> hotspot-runtime-dev at openjdk.java.net > > > >> Subject: Re: RFR(S): 8200720: Print additional information in thread dump > > > >> (times, allocated bytes etc.) > > > >> > > > >> Hi Gunter, > > > >> > > > >> On 5/06/2018 1:27 AM, Haug, Gunter wrote: > > > >>> Hi David, Chris, > > > >>> > > > >>> would it be an option to unconditionally print the additional information? > > > >> Regardless which way this is implemented it will be rather complicated and it > > > >> only switches on/off a few field in the thread dump. > > > >> > > > >> I'm not convinced this is all suitable information for a thread stack > > > >> dump. I can see the time information being useful if using the dump to > > > >> try and diagnoze a hang or responsiveness issue. But the allocated-bytes > > > >> and classes-defined just doesn't seem useful in the context of a thread > > > >> dump to me. Anyone interested in allocation stats is going to be looking > > > >> at GC logs etc which is where this belongs. Ditto the class-stats belong > > > >> in some kind of classloading logging IMHO. > > > >> > > > >> I'm very reluctant to burden the implementation with capturing these > > > >> kinds of stats, just to use for diagnostic purposes that may not even be > > > >> asked for. I'm very much in the "you shouldn't pay for what you don't > > > >> use" camp in that regard. (See my comments in JDK-8143176 referenced by > > > >> Sean.) > > > >> > > > >> The ThreadStatisticInfo adds overhead to every thread object - and I'd > > > >> have to suspect there could be overlap with whatever flight recorder > > > >> sticks in there too. (Thread has become even more bloated in recent time!). > > > >> > > > >> Cheers, > > > >> David > > > >> > > > >> > > > >>> Thanks, > > > >>> Gunter > > > >>> > > > >>> On 04.06.18, 01:13, "David Holmes" wrote: > > > >>> > > > >>> Hi Gunter, Chris, > > > >>> > > > >>> Sorry just trying to catch up and this is only a partial look so far ... > > > >>> > > > >>> BTW these changes are not limited to serviceability code so adding in > > > >>> runtime team as well. > > > >>> > > > >>> On 2/06/2018 9:12 AM, Chris Plummer wrote: > > > >>> > Hi Gunter, > > > >>> > > > > >>> > On 6/1/18 3:17 AM, Haug, Gunter wrote: > > > >>> >> Hi Chris, > > > >>> >> > > > >>> >> thanks for looking into this! > > > >>> >> > > > >>> >> Re the synchronization: The value is stored only in a VM operation at > > > >>> >> a safepoint by the VM thread. I was of the opinion, that this could > > > >>> >> not be interrupted by a second VM operation (of the same type). Or > > > >> am > > > >>> >> I missing something else? > > > >>> > I was really thinking more about the temporary changing of > > > >>> > PrintExtendedThreadInfo, not the value stored in the VMOp. You may > > > >> be be > > > >>> > correct that no more than one VMOp is executing, but while it is > > > >>> > executing it is has changed the value of PrintExtendedThreadInfo, > > > >> which > > > >>> > might have an impact on anything else that triggers printing thread info > > > >>> > while the VMOp is executing. > > > >>> > > > >>> Even if nothing else can possibly be running during the safepoint I find > > > >>> it extremely bad form to change a command-line flag in this way, > > > >>> particularly from a safepoint! > > > >>> > > > >>> If this flag is intended to be dynamically enabled as part of a dcmd > > > >>> then it should also be a Manageable flag IMHO. > > > >>> > > > >>> That said ... > > > >>> > > > >>> >> I did think about passing an argument to the various print_on > > > >> member > > > >>> >> functions of the thread classes, but this would require rather > > > >>> >> extensive code changes for a rather tiny extension. Therefore I feel > > > >>> >> doing it like this is the lesser evil. > > > >>> > > > >>> ... it's obviously not truly a global setting, but one that is needed on > > > >>> a per-print-request basis. The flag is just the default, but if the > > > >>> default is off you still want to enable extended printing on a > > > >>> per-request basis. > > > >>> > > > >>> The current print_on mechanics is not set up for this kind of > > > >>> flexibility. I think this needs more thought. > > > >>> > > > >>> --- > > > >>> > > > >>> Re osThread.cpp shared code changes ... I really hate to see all the > > > >>> ifdefs in there. Please add a pd_print_on function to support this if > > > >>> you truly want platform specific information. > > > >>> > > > >>> --- > > > >>> > > > >>> threadStatisticInfo.hpp > > > >>> > > > >>> typo: getElepsedTime() -> getElapsedTime() > > > >>> > > > >>> Thanks, > > > >>> David > > > >>> > > > >>> > > > >>> > > > >>> >> > > > >>> >> Re thread_dump(): I think it's correct or, well, at least it works ;-) > > > >>> >> In fact jstack will transfer the "-e" and "-l" in only one string, > > > >>> >> i.e. op->arg(0). > > > >>> > So you are saying that op->arg(0) is "-e -l" when using jstack? I think > > > >>> > you really need to clean up the parsing. As it stands right now passing, > > > >>> > I get the feeling that if a user erroneously asks for help by using > > > >>> > "jcmd Thread.Print -help", it will end up executing with -e an -l > > > >>> > enabled, and no failure for the invalid "-help" option. > > > >>> >> Christoph has already explained my reasoning. But I agree, it's not > > > >>> >> nice and I would be happy to do it like Christoph suggested. > > > >>> > I'll respond separately to his suggestion. > > > >>> > > > > >>> > thanks, > > > >>> > > > > >>> > Chris > > > >>> >> > > > >>> >> And typo fixed, sorry. > > > >>> >> > > > >>> >> Thanks again, > > > >>> >> Gunter > > > >>> >> > > > >>> >> On 01.06.18, 00:03, "Chris Plummer" > > > >> wrote: > > > >>> >> > > > >>> >> Hi Gunter, > > > >>> >> globals.hpp: fix typo "informatiuon" > > > >>> >> I worry a little bit about the synchronizing (if that's the right > > > >>> >> word) > > > >>> >> of PrintExtendedThreadInfo and the dcmd's -e flag. When using -e, > > > >>> >> you > > > >>> >> are temporarily enabling PrintExtendedThreadInfo if it was false. > > > >>> >> This > > > >>> >> temporarily changes the behavior of thread dumps, and could > > > >>> >> impact other > > > >>> >> uses that happen in parallel. Also, could two simultaneous uses > > > >>> >> of -e > > > >>> >> result in PrintExtendedThreadInfo not getting restored properly? > > > >>> >> thread_dump() doesn't look right. It looks like you are iterating > > > >>> >> char > > > >>> >> by char over the argument, and expect something like "-el" to be > > > >>> >> specified rather then "-e -l". The loop should be iterating over > > > >>> >> op->arg(i), not op->arg(0)[i]. > > > >>> >> The rest of the changes look fine. > > > >>> >> thanks, > > > >>> >> Chris > > > >>> >> On 5/30/18 8:12 AM, Haug, Gunter wrote: > > > >>> >> > Hi all, > > > >>> >> > > > > >>> >> > As Chris proposed, I have made an the extended output > > > >>> >> switchable. There is an VM flag (PrintExtendedThreadInfo), which is > > > >>> >> false by default. Moreover, there is an Option (-e) which can be used > > > >>> >> with jcmd Thread.print as well as with jstack. The -e option > > > >>> >> essentially sets PrintExtendedThreadInfo true just for the respective > > > >>> >> thread dump. > > > >>> >> > > > > >>> >> > Here is the updated webrev: > > > >>> >> > > > > >>> >> > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v2 > > > >>> >> > > > > >>> >> > (https://bugs.openjdk.java.net/browse/JDK-8200720) > > > >>> >> > > > > >>> >> > Thanks, > > > >>> >> > Gunter > > > >>> >> > > > > >>> >> > > > > >>> >> > On 02.05.18, 17:07, "serviceability-dev on behalf of Haug, > > > >>> >> Gunter" > > >>> >> gunter.haug at sap.com> wrote: > > > >>> >> > > > > >>> >> > Hi Chris, > > > >>> >> > > > > >>> >> > Thanks for looking into this. > > > >>> >> > You're right, there is a little more we have. We have > > > >>> >> implemented an IO tracing mechanism which - rather as a byproduct - > > > >>> >> keeps track of bytes read and written per thread. However, this of > > > >>> >> course requires changes not only in hotspot. We would be happy to > > > >>> >> contribute this as well, but this is a far bigger change and will > > > >>> >> probably lead to a far bigger discussion. Anyway, with the number of > > > >>> >> bytes read, the number of classes defined doesn't look that arbitrary > > > >>> >> anymore, as one can correlate IO to class loading. > > > >>> >> > > > > >>> >> > Regarding the verbose option I think that's a good idea! > > > >>> >> > > > > >>> >> > Thanks again, > > > >>> >> > Gunter > > > >>> >> > > > > >>> >> > On 01.05.18, 22:55, "Chris Plummer" > > > >>> >> wrote: > > > >>> >> > > > > >>> >> > Hi Gunter, > > > >>> >> > > > > >>> >> > The output you are adding is all useful. I think the > > > >>> >> question is (and > > > >>> >> > I'd like to see a few people chime in on this for this > > > >>> >> review) is > > > >>> >> > whether or not all of it is the appropriate for a > > > >>> >> thread's stack dump. > > > >>> >> > For example, defined_classes is on the fringe of what > > > >>> >> I would call > > > >>> >> > generally useful info in a stack dump. Sure, there > > > >>> >> might be that rare > > > >>> >> > case when you need it, but how often compared to other > > > >>> >> useful info > > > >>> >> > maintained on a per thread basis. How many other bits > > > >>> >> of useful info are > > > >>> >> > not being printed in favor of defined_classes? It > > > >>> >> seems you have more in > > > >>> >> > the queue. How many? I'm worried about how cluttered > > > >>> >> the stack dumps > > > >>> >> > will get. Maybe we should add some sort of verbose > > > >>> >> thread dumping > > > >>> >> > option. Just a thought. > > > >>> >> > > > > >>> >> > As for the implementation, overall it looks good, but > > > >>> >> I can't speak to > > > >>> >> > whether or not you are doing proper accounting of > > > >>> >> defined_classes and > > > >>> >> > bytes allocated. You'll need input from someone with > > > >>> >> more knowledge of > > > >>> >> > those areas. We'll also need to do some testing to > > > >>> >> make sure tool tests > > > >>> >> > are not impacted. > > > >>> >> > > > > >>> >> > thanks, > > > >>> >> > > > > >>> >> > Chris > > > >>> >> > > > > >>> >> > On 4/30/18 2:51 AM, Haug, Gunter wrote: > > > >>> >> > > Hi, > > > >>> >> > > > > > >>> >> > > this is an update to an RFR I posted on hotspot-dev, > > > >>> >> but it is probably more suitable to post it here. Can I please have a > > > >>> >> review and a sponsor for the following enhancement: > > > >>> >> > > > > > >>> >> > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v1 > > > >>> >> > > https://bugs.openjdk.java.net/browse/JDK-8200720 > > > >>> >> > > > > > >>> >> > > We at SAP have extended the thread dumps (obtained > > > >>> >> by jstack or jcmd) by several fields providing thread specific > > > >>> >> information. These extensions are quite popular with our support > > > >> team. > > > >>> >> With some knowledge of the architecture of the application, they > > > >> often > > > >>> >> allow for quick and simple diagnosis of a running system. Therefore > > > >> we > > > >>> >> would like to contribute these enhancements. > > > >>> >> > > > > > >>> >> > > I took a few simple examples here, namely cpu time, > > > >>> >> elapsed time since thread creation, bytes allocated and classes > > > >>> >> defined by the thread and the pthread-id or equivalent on platforms > > > >>> >> where it makes sense. Provided it is known how the application > > > >> should > > > >>> >> behave, a misbehaving thread can identified easily. > > > >>> >> > > > > > >>> >> > > There is no measurable overhead for this > > > >>> >> enhancement. However, it may be a problem that the format of the > > > >>> >> output is changed. Tools parsing the output may have to be changed. > > > >>> >> > > > > > >>> >> > > Here is an example of the output generated: > > > >>> >> > > > > > >>> >> > > ------------------------------------------------------ > > > >>> >> > > > > > >>> >> > > "main" #1 prio=5 os_prio=31 cpu=6300.65ms > > > >>> >> elapsed=123.28s allocated=242236760B defined_classes=1725 > > > >>> >> tid=0x00007fa13a806000 nid=0x1c03 pthread-id=0x109708000 waiting > > > >> on > > > >>> >> condition [0x0000000109707000] > > > >>> >> > > java.lang.Thread.State: TIMED_WAITING (sleeping) > > > >>> >> > > JavaThread state: _thread_blocked > > > >>> >> > > Thread: 0x00007fa13a806000 [0x1c03] State: > > > >>> >> _at_safepoint _has_called_back 0 _at_poll_safepoint 0 > > > >>> >> > > JavaThread state: _thread_blocked > > > >>> >> > > at java.lang.Thread.sleep(java.base/Native Method) > > > >>> >> > > ... > > > >>> >> > > ------------------------------------------------------ > > > >>> >> > > > > > >>> >> > > As mentioned above, we have a whole bunch of other > > > >>> >> enhancements to the thread dump similar to this one and would be > > > >>> >> willing to contribute them if there is any interest. > > > >>> >> > > > > > >>> >> > > Thanks and best regards, > > > >>> >> > > Gunter > > > >>> >> > > > > > >>> >> > > > > > >>> >> > > > > >>> >> > > > > >>> >> > > > > >>> >> > > > > >>> >> > > > > >>> >> > > > >>> > > > > >>> > > > > >>> > > > >>> > > > > > > > > > > > > > > > > > > From gunter.haug at sap.com Mon Jun 25 10:04:00 2018 From: gunter.haug at sap.com (Haug, Gunter) Date: Mon, 25 Jun 2018 10:04:00 +0000 Subject: RFR(S): 8200720: Print additional information in thread dump (times, allocated bytes etc.) In-Reply-To: <1e7e234a-91ea-2c14-9634-08f78b90b225@oracle.com> References: <56817AB5-88FC-4FB9-A271-7206EA69ECA7@sap.com> <2728581E-0E32-4911-B1A9-79AD6CBA81B0@sap.com> <082ca98f-9a68-15a0-95b5-c4d5c733ab36@oracle.com> <933cdacc-9b6f-c8b5-e244-558f38e7888c@oracle.com> <55806aa8-f09b-e9a8-b512-b9676029c494@oracle.com> <0F497BC6-2569-4EF6-9B59-E24A8E1B3A37@sap.com> <5dc08f65e83f4374b8e1f73f218c6d6d@sap.com> <626701f8-b71b-58dc-f837-e5e1d81ef187@oracle.com> <19BE9680-3E6E-4620-B72D-D379F39C1C92@sap.com> <6ce47417-311b-e6cd-b8cd-d46eb7e04c2f@oracle.com> <3795D1AE-CBF4-4BC3-B707-ED77D4DE00D0@sap.com> <02AFA85A-36A7-49D0-A308-E509FFC39C97@sap.com> <1e7e234a-91ea-2c14-9634-08f78b90b225@oracle.com> Message-ID: <6AA6AE61-C852-4C24-A83D-D3570B528D26@sap.com> Hi David, Thank you. You're right, the setter is not required, I'll remove it. I've filed a CSR request: https://bugs.openjdk.java.net/browse/JDK-8205604 Is this what is expected? It's my first one. Thank you, Gunter ?On 25.06.18, 10:38, "David Holmes" wrote: Hi Gunter, On 25/06/2018 6:26 PM, Haug, Gunter wrote: > Hi David, > > Here is the updated webrev. I've included all the changes you suggested, except for default parameter (see original mail). > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v4/ > > Are you OK with it now? You don't need the setStartTime setter as by definition a "start time" is only set once. (Though "start" may not be the right term here.) Otherwise changes seem okay. Note however that you are adding a product flag so that means you need to file a CSR request. Thanks, David > Thanks again, > Gunter > > On 20.06.18, 17:40, "serviceability-dev on behalf of Haug, Gunter" wrote: > > Hi David, > > Thanks for taking the time to look into this! > > > "statistic info" is not very good grammatically. These things are either > > statistics, or statistical information. So e.g. > > > > class ThreadStatistics > >... > > OK, I'll change that! > > > const_cast(this)->cooked_allocated_bytes(); > > > > Why do we need a const cast to invoke a method on ourselves ?? > > cooked_allocated_bytes() is not declared const and we can't make it, as it uses OrderAccess::load_acquire(&_allocated_bytes). Therefor the const_cast. > > > src/hotspot/share/runtime/thread.hpp > > > > Please put the new #include in alphabetical order. > > Wilco! > > > I was expecting to see a default parameter used here rather than adding > > an overload: > > virtual void print_on(outputStream* st, bool print_extended_info = > > false) const { print_on(st, print_extended_info); } > > Thread inherits from AllocatedObj (in dbg, at least) where virtual void AllocatedObj::print_on(outputStream*) const is declared. Some compilers, e.g. gcc, will complain that this is hidden by Thread::print_on(outputStream*, bool). Others, e.g. clang, are happy with that. We could implement a method with a different name but would that be nicer? > > > 2179 static void print_on(outputStream* st, bool print_stacks, bool > > internal_format, bool print_concurrent_locks, bool extended_thread_info); > > > > Again I expected to see a default parameter here - but I didn't check if > > all callers themselves take the new parameter. ?? > > This one has just one single caller, so I think it's OK. > > > src/hotspot/share/runtime/vm_operations.hpp > >... > > I'll change it to conform to the guidelines. > > > src/hotspot/share/runtime/threadStatisticInfo.hpp > > > > There are no includes in this file. It should include the appropriate > > std header for type definitions, and os.hpp. > > Agreed, I'll add the includes. > > > > I'm not sure if class ThreadStatisticInfo needs an allocation type as a > > super type. > > > > Might be worth adding a constructor to give a default value so that you > > can tell if the statistics have been initialized when they appear in the > > printout. ? > > ThreadStatisticInfo is a data member of Thread. Currently it is initialized in Thread::Thread, but I can add a constructor, if you prefer. > > Thanks again, > Gunter > > > > On 20.06.18, 09:15, "David Holmes" wrote: > > Hi Gunter, > > On 19/06/2018 8:51 PM, Haug, Gunter wrote: > > Hi all, > > > > Thanks Chris and Christoph for the reviews! Christoph, I'll incorporate the little improvement you suggested. > > David, are you OK with the change as well? > > Sorry I'd lost track of this one a bit ... > > The overall approach now seems okay. > > Some naming/terminology issues: > > "statistic info" is not very good grammatically. These things are either > statistics, or statistical information. So e.g. > > class ThreadStatistics > > ThreadStatistics& statistics() { return _statistics; } > > etc. > > "extended_thread_info" should really be print_extended_info (similar to > print_concurrent_locks). You don't need "thread" in there when this is > always part of a Thread related API. > > > src/hotspot/share/runtime/thread.cpp > > const_cast(this)-> > > Why do we need a const cast to invoke a method on ourselves ?? > > --- > > src/hotspot/share/runtime/thread.hpp > > Please put the new #include in alphabetical order. > > 641 // Printing > 642 void print_on(outputStream* st, bool extended_thread_info) const; > 643 virtual void print_on(outputStream* st) const { print_on(st, > false); } > > I was expecting to see a default parameter used here rather than adding > an overload: > > virtual void print_on(outputStream* st, bool print_extended_info = > false) const { print_on(st, print_extended_info); } > > > 2179 static void print_on(outputStream* st, bool print_stacks, bool > internal_format, bool print_concurrent_locks, bool extended_thread_info); > > Again I expected to see a default parameter here - but I didn't check if > all callers themselves take the new parameter. ?? > > --- > > src/hotspot/share/runtime/vm_operations.hpp > > 376 VM_PrintThreads() > 377 { _out = tty; _print_concurrent_locks = PrintConcurrentLocks; ; > _extended_thread_info = false; } > 378 VM_PrintThreads(outputStream* out, bool print_concurrent_locks, > bool extended_thread_info) > 379 { _out = out; _print_concurrent_locks = print_concurrent_locks; > _extended_thread_info = extended_thread_info; } > 380 VMOp_Type type() const > 381 { return VMOp_PrintThreads; } > > Style nits: either keep everything on one line as before (though I agree > the lines are now too long) or else the style should be: > > 380 VMOp_Type type() const { > 381 return VMOp_PrintThreads; > 382 } > > Also unclear why (existing) VM_PrintThreads constructor doesn't use > initializer list (like VM_PrintMetadata below it) or default parameters? > > --- > > src/hotspot/share/runtime/threadStatisticInfo.hpp > > There are no includes in this file. It should include the appropriate > std header for type definitions, and os.hpp. > > I'm not sure if class ThreadStatisticInfo needs an allocation type as a > super type. > > Might be worth adding a constructor to give a default value so that you > can tell if the statistics have been initialized when they appear in the > printout. ? > > --- > > Thanks, > David > > > > Thanks again, > > Gunter > > > > On 12.06.18, 01:13, "Chris Plummer" wrote: > > > > Hi Gunter, > > > > The changes look fine. I can live with the options parsing in > > attachListener.cpp. As you point out, unrecognized options were already > > silently ignored. > > > > thanks, > > > > Chris > > > > On 6/8/18 7:05 AM, Haug, Gunter wrote: > > > Hi all, > > > > > > thanks a lot for all the input! I have prepared a new version of the webrev incorporating the suggestions you made (at least I tried): > > > > > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v3/ > > > > > > This version outputs the thread times unconditionally while the other information is guarded by a flags. I think, most participants found the thread times the most valuable information and had no (strong) objections to adding them unconditionally. > > > > > > @David > > > Implementation is much simpler with conditional output only for JavaThreads. I could get rid of the terrible hack (changing the flag) without having to change too many source files. > > > > > > The information on allocated bytes is present in the Thread class already before this proposed change, I just print it. It might be sensible to move _allocated_bytes and the respective methods to the ThreadStatisticInfo class as G?tz suggested. I haven?t done that in the current version, though. > > > > > > @David and Thomas > > > I've removed the pthread-id output. I'm unsure myself what it could be good for. Anyway, we could add it with a separate change (with a better implementation) if there is a need to. > > > > > > @Chris > > > As you have written, the user has no direct contact to the attach listener of the VM and jstack won't misinterpret e.g. -help. I agree that the parsing in attachListener.cpp could be more robust. However, it hasn't been so far either. All that is done in the current implementation is a strcmp to -l everything else will be silently ignored. I can try to make this more robust or we could say that the new output is only available via jcmd. OTH nothing bad can happen with the current version of the proposed change, so we could also leave it as it is. What would you prefer? > > > > > > @Kirk and Thomas > > > Implementing a new diagnostic command is certainly well worth a thought. However, it appears to me that it's not necessary in this case. There are already flags to jstack and jcmd Thread.print, so it's not that uncommon. The amount of information that would be available by a new diagnostic command and that is now added to the thread dump is quite small and I think that is tolerable. Moreover, and most importantly, our support team is use to find the information in the thread dump and it would make things easier to us if I could leave it there. > > > > > > Thanks again and have a nice weekend, > > > Gunter > > > > > > > > > On 06.06.18, 04:56, "David Holmes" wrote: > > > > > > Hi Goetz, > > > > > > On 5/06/2018 11:07 PM, Lindenmaier, Goetz wrote: > > > > Hi > > > > > > > > this discussion touched a lot of points so far, which seem > > > > to lead to different conclusions. > > > > > > > > I think we should look at the values printed: > > > > > > > > 1. cpu=6300.65ms elapsed=123.28s > > > > Overhead > > > > cpu time: > > > > * system calls at thread dump time > > > > elapsed time: > > > > * 1 system call at thread creation time > > > > * 1 64-bit field per thread for the thread start time > > > > * 1 system call at thread dump time > > > > > > > > As I understand, JDK-8143176 would have had to get the > > > > time at each locking, which is much more time critical > > > > than doing this during thread creation. While > > > > > > Correct. > > > > > > > the time a lock was held would be much more useful, > > > > the ratio here, combined with knowledge about the application, > > > > could lead to the conclusion that the thread is wrongly > > > > blocked at much lower cost. > > > > > > Agreed. I see no issue with unconditionally adding this information as > > > it should address some of the concerns of 8143176 as well. > > > > > > > > > > > 2. pthread-id=0x109708000 > > > > Overhead: > > > > * a field access at thread dump time. Negligible I would say. > > > > > > I'd overlooked this addition, but it is of course the chunk of ifdef > > > code in osThread.cpp that I objected to. The problem I have here is the > > > convoluted mess of "thread identifiers" that we already have. We > > > currently print: > > > > > > st->print("tid=" INTPTR_FORMAT " ", p2i(this)); > > > > > > which is just the address of the Thread/JavaThread object. Then: > > > > > > st->print("nid=0x%x ", thread_id()); > > > > > > where 'n' is supposed to represent "native thread id", which is: > > > - linux: kernel thread id from syscall gettid > > > - solaris: thread library identity from thr_create() or thr_self() > > > - windows: thread id from _beginthreadex > > > - OS X: kernel thread id from pthread_mach_thread_np(pthread_self()); > > > - Other BSD: kernel (?) thread id from syscall thr_self or getthrid > > > - AIX: thread library identity from pthread_create() or pthread_self() > > > > > > It's telling that the code Gunter added gets the thread library id on > > > linux, but a "kernel thread id" on other platforms! Do we want to see a > > > thread library id and a kernel thread id? IIRC nid is supposed be the id > > > you need to see the thread in a debugger. In which case I'm unclear what > > > role the thread id Gunter wants to add is playing? I don't think we need > > > to see kernel ids in general, and pthread-id isn't useful for debugging > > > is it? > > > > > > > > > > > 3. allocated=242236760B defined_classes=1725 > > > > Overhead: > > > > * 1 64-bit field per thread. > > > > * counter increment on class creation > > > > > > > > Especially defined_classes seems to be controversial. > > > > As this only makes sense for Java threads, this could > > > > be printed in a line of it's own in Threads::print_on_error() > > > > (which already gets a flag to customize for jstack: > > > > print_concurrent_locks) by calling a dedicated function in Thread. > > > > No need for flag PrintExtendedThreadInfo thus. > > > > > > > > But I would propose to drop this information as it is > > > > too controversial. > > > > > > Agreed. > > > > > > > This leaves the times and the pthread-id to be decided whether > > > > they are worth the cost. If so, I think they should be printed > > > > unconditional. > > > > > > Agreed. With some refinement of the "thread id" issue. > > > > > > > If a flag is required to avoid bloating the info in the default case, > > > > I would include more infos, as os_prio and nid under that flag. > > > > > > Of course those things exist for good reason. The native id is supposed > > > to be what aids in you matching what you see in our thread dumps with > > > what you see in a debugger. Priority is less of an issue these days, but > > > there was a time ... ;-) > > > > > > > As jstack is deprecated, changes to jstack could be skipped always > > > > disabling the new printouts. > > > > > > > > Finally, I would propose to move _allocated_bytes into > > > > threadStatisticInfo.hpp. > > > > > > Didn't that part get dropped? (You don't need it for the time info or > > > thread id). That's a GC logging issue to me. > > > > > > Thanks, > > > David > > > > > > > Best regards, > > > > Goetz. > > > > > > > > > > > > > > > > > > > >> -----Original Message----- > > > >> From: serviceability-dev [mailto:serviceability-dev- > > > >> bounces at openjdk.java.net] On Behalf Of David Holmes > > > >> Sent: Dienstag, 5. Juni 2018 04:53 > > > >> To: Haug, Gunter ; Chris Plummer > > > >> ; serviceability-dev > > >> dev at openjdk.java.net>; Langer, Christoph ; > > > >> hotspot-runtime-dev at openjdk.java.net > > > >> Subject: Re: RFR(S): 8200720: Print additional information in thread dump > > > >> (times, allocated bytes etc.) > > > >> > > > >> Hi Gunter, > > > >> > > > >> On 5/06/2018 1:27 AM, Haug, Gunter wrote: > > > >>> Hi David, Chris, > > > >>> > > > >>> would it be an option to unconditionally print the additional information? > > > >> Regardless which way this is implemented it will be rather complicated and it > > > >> only switches on/off a few field in the thread dump. > > > >> > > > >> I'm not convinced this is all suitable information for a thread stack > > > >> dump. I can see the time information being useful if using the dump to > > > >> try and diagnoze a hang or responsiveness issue. But the allocated-bytes > > > >> and classes-defined just doesn't seem useful in the context of a thread > > > >> dump to me. Anyone interested in allocation stats is going to be looking > > > >> at GC logs etc which is where this belongs. Ditto the class-stats belong > > > >> in some kind of classloading logging IMHO. > > > >> > > > >> I'm very reluctant to burden the implementation with capturing these > > > >> kinds of stats, just to use for diagnostic purposes that may not even be > > > >> asked for. I'm very much in the "you shouldn't pay for what you don't > > > >> use" camp in that regard. (See my comments in JDK-8143176 referenced by > > > >> Sean.) > > > >> > > > >> The ThreadStatisticInfo adds overhead to every thread object - and I'd > > > >> have to suspect there could be overlap with whatever flight recorder > > > >> sticks in there too. (Thread has become even more bloated in recent time!). > > > >> > > > >> Cheers, > > > >> David > > > >> > > > >> > > > >>> Thanks, > > > >>> Gunter > > > >>> > > > >>> On 04.06.18, 01:13, "David Holmes" wrote: > > > >>> > > > >>> Hi Gunter, Chris, > > > >>> > > > >>> Sorry just trying to catch up and this is only a partial look so far ... > > > >>> > > > >>> BTW these changes are not limited to serviceability code so adding in > > > >>> runtime team as well. > > > >>> > > > >>> On 2/06/2018 9:12 AM, Chris Plummer wrote: > > > >>> > Hi Gunter, > > > >>> > > > > >>> > On 6/1/18 3:17 AM, Haug, Gunter wrote: > > > >>> >> Hi Chris, > > > >>> >> > > > >>> >> thanks for looking into this! > > > >>> >> > > > >>> >> Re the synchronization: The value is stored only in a VM operation at > > > >>> >> a safepoint by the VM thread. I was of the opinion, that this could > > > >>> >> not be interrupted by a second VM operation (of the same type). Or > > > >> am > > > >>> >> I missing something else? > > > >>> > I was really thinking more about the temporary changing of > > > >>> > PrintExtendedThreadInfo, not the value stored in the VMOp. You may > > > >> be be > > > >>> > correct that no more than one VMOp is executing, but while it is > > > >>> > executing it is has changed the value of PrintExtendedThreadInfo, > > > >> which > > > >>> > might have an impact on anything else that triggers printing thread info > > > >>> > while the VMOp is executing. > > > >>> > > > >>> Even if nothing else can possibly be running during the safepoint I find > > > >>> it extremely bad form to change a command-line flag in this way, > > > >>> particularly from a safepoint! > > > >>> > > > >>> If this flag is intended to be dynamically enabled as part of a dcmd > > > >>> then it should also be a Manageable flag IMHO. > > > >>> > > > >>> That said ... > > > >>> > > > >>> >> I did think about passing an argument to the various print_on > > > >> member > > > >>> >> functions of the thread classes, but this would require rather > > > >>> >> extensive code changes for a rather tiny extension. Therefore I feel > > > >>> >> doing it like this is the lesser evil. > > > >>> > > > >>> ... it's obviously not truly a global setting, but one that is needed on > > > >>> a per-print-request basis. The flag is just the default, but if the > > > >>> default is off you still want to enable extended printing on a > > > >>> per-request basis. > > > >>> > > > >>> The current print_on mechanics is not set up for this kind of > > > >>> flexibility. I think this needs more thought. > > > >>> > > > >>> --- > > > >>> > > > >>> Re osThread.cpp shared code changes ... I really hate to see all the > > > >>> ifdefs in there. Please add a pd_print_on function to support this if > > > >>> you truly want platform specific information. > > > >>> > > > >>> --- > > > >>> > > > >>> threadStatisticInfo.hpp > > > >>> > > > >>> typo: getElepsedTime() -> getElapsedTime() > > > >>> > > > >>> Thanks, > > > >>> David > > > >>> > > > >>> > > > >>> > > > >>> >> > > > >>> >> Re thread_dump(): I think it's correct or, well, at least it works ;-) > > > >>> >> In fact jstack will transfer the "-e" and "-l" in only one string, > > > >>> >> i.e. op->arg(0). > > > >>> > So you are saying that op->arg(0) is "-e -l" when using jstack? I think > > > >>> > you really need to clean up the parsing. As it stands right now passing, > > > >>> > I get the feeling that if a user erroneously asks for help by using > > > >>> > "jcmd Thread.Print -help", it will end up executing with -e an -l > > > >>> > enabled, and no failure for the invalid "-help" option. > > > >>> >> Christoph has already explained my reasoning. But I agree, it's not > > > >>> >> nice and I would be happy to do it like Christoph suggested. > > > >>> > I'll respond separately to his suggestion. > > > >>> > > > > >>> > thanks, > > > >>> > > > > >>> > Chris > > > >>> >> > > > >>> >> And typo fixed, sorry. > > > >>> >> > > > >>> >> Thanks again, > > > >>> >> Gunter > > > >>> >> > > > >>> >> On 01.06.18, 00:03, "Chris Plummer" > > > >> wrote: > > > >>> >> > > > >>> >> Hi Gunter, > > > >>> >> globals.hpp: fix typo "informatiuon" > > > >>> >> I worry a little bit about the synchronizing (if that's the right > > > >>> >> word) > > > >>> >> of PrintExtendedThreadInfo and the dcmd's -e flag. When using -e, > > > >>> >> you > > > >>> >> are temporarily enabling PrintExtendedThreadInfo if it was false. > > > >>> >> This > > > >>> >> temporarily changes the behavior of thread dumps, and could > > > >>> >> impact other > > > >>> >> uses that happen in parallel. Also, could two simultaneous uses > > > >>> >> of -e > > > >>> >> result in PrintExtendedThreadInfo not getting restored properly? > > > >>> >> thread_dump() doesn't look right. It looks like you are iterating > > > >>> >> char > > > >>> >> by char over the argument, and expect something like "-el" to be > > > >>> >> specified rather then "-e -l". The loop should be iterating over > > > >>> >> op->arg(i), not op->arg(0)[i]. > > > >>> >> The rest of the changes look fine. > > > >>> >> thanks, > > > >>> >> Chris > > > >>> >> On 5/30/18 8:12 AM, Haug, Gunter wrote: > > > >>> >> > Hi all, > > > >>> >> > > > > >>> >> > As Chris proposed, I have made an the extended output > > > >>> >> switchable. There is an VM flag (PrintExtendedThreadInfo), which is > > > >>> >> false by default. Moreover, there is an Option (-e) which can be used > > > >>> >> with jcmd Thread.print as well as with jstack. The -e option > > > >>> >> essentially sets PrintExtendedThreadInfo true just for the respective > > > >>> >> thread dump. > > > >>> >> > > > > >>> >> > Here is the updated webrev: > > > >>> >> > > > > >>> >> > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v2 > > > >>> >> > > > > >>> >> > (https://bugs.openjdk.java.net/browse/JDK-8200720) > > > >>> >> > > > > >>> >> > Thanks, > > > >>> >> > Gunter > > > >>> >> > > > > >>> >> > > > > >>> >> > On 02.05.18, 17:07, "serviceability-dev on behalf of Haug, > > > >>> >> Gunter" > > >>> >> gunter.haug at sap.com> wrote: > > > >>> >> > > > > >>> >> > Hi Chris, > > > >>> >> > > > > >>> >> > Thanks for looking into this. > > > >>> >> > You're right, there is a little more we have. We have > > > >>> >> implemented an IO tracing mechanism which - rather as a byproduct - > > > >>> >> keeps track of bytes read and written per thread. However, this of > > > >>> >> course requires changes not only in hotspot. We would be happy to > > > >>> >> contribute this as well, but this is a far bigger change and will > > > >>> >> probably lead to a far bigger discussion. Anyway, with the number of > > > >>> >> bytes read, the number of classes defined doesn't look that arbitrary > > > >>> >> anymore, as one can correlate IO to class loading. > > > >>> >> > > > > >>> >> > Regarding the verbose option I think that's a good idea! > > > >>> >> > > > > >>> >> > Thanks again, > > > >>> >> > Gunter > > > >>> >> > > > > >>> >> > On 01.05.18, 22:55, "Chris Plummer" > > > >>> >> wrote: > > > >>> >> > > > > >>> >> > Hi Gunter, > > > >>> >> > > > > >>> >> > The output you are adding is all useful. I think the > > > >>> >> question is (and > > > >>> >> > I'd like to see a few people chime in on this for this > > > >>> >> review) is > > > >>> >> > whether or not all of it is the appropriate for a > > > >>> >> thread's stack dump. > > > >>> >> > For example, defined_classes is on the fringe of what > > > >>> >> I would call > > > >>> >> > generally useful info in a stack dump. Sure, there > > > >>> >> might be that rare > > > >>> >> > case when you need it, but how often compared to other > > > >>> >> useful info > > > >>> >> > maintained on a per thread basis. How many other bits > > > >>> >> of useful info are > > > >>> >> > not being printed in favor of defined_classes? It > > > >>> >> seems you have more in > > > >>> >> > the queue. How many? I'm worried about how cluttered > > > >>> >> the stack dumps > > > >>> >> > will get. Maybe we should add some sort of verbose > > > >>> >> thread dumping > > > >>> >> > option. Just a thought. > > > >>> >> > > > > >>> >> > As for the implementation, overall it looks good, but > > > >>> >> I can't speak to > > > >>> >> > whether or not you are doing proper accounting of > > > >>> >> defined_classes and > > > >>> >> > bytes allocated. You'll need input from someone with > > > >>> >> more knowledge of > > > >>> >> > those areas. We'll also need to do some testing to > > > >>> >> make sure tool tests > > > >>> >> > are not impacted. > > > >>> >> > > > > >>> >> > thanks, > > > >>> >> > > > > >>> >> > Chris > > > >>> >> > > > > >>> >> > On 4/30/18 2:51 AM, Haug, Gunter wrote: > > > >>> >> > > Hi, > > > >>> >> > > > > > >>> >> > > this is an update to an RFR I posted on hotspot-dev, > > > >>> >> but it is probably more suitable to post it here. Can I please have a > > > >>> >> review and a sponsor for the following enhancement: > > > >>> >> > > > > > >>> >> > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v1 > > > >>> >> > > https://bugs.openjdk.java.net/browse/JDK-8200720 > > > >>> >> > > > > > >>> >> > > We at SAP have extended the thread dumps (obtained > > > >>> >> by jstack or jcmd) by several fields providing thread specific > > > >>> >> information. These extensions are quite popular with our support > > > >> team. > > > >>> >> With some knowledge of the architecture of the application, they > > > >> often > > > >>> >> allow for quick and simple diagnosis of a running system. Therefore > > > >> we > > > >>> >> would like to contribute these enhancements. > > > >>> >> > > > > > >>> >> > > I took a few simple examples here, namely cpu time, > > > >>> >> elapsed time since thread creation, bytes allocated and classes > > > >>> >> defined by the thread and the pthread-id or equivalent on platforms > > > >>> >> where it makes sense. Provided it is known how the application > > > >> should > > > >>> >> behave, a misbehaving thread can identified easily. > > > >>> >> > > > > > >>> >> > > There is no measurable overhead for this > > > >>> >> enhancement. However, it may be a problem that the format of the > > > >>> >> output is changed. Tools parsing the output may have to be changed. > > > >>> >> > > > > > >>> >> > > Here is an example of the output generated: > > > >>> >> > > > > > >>> >> > > ------------------------------------------------------ > > > >>> >> > > > > > >>> >> > > "main" #1 prio=5 os_prio=31 cpu=6300.65ms > > > >>> >> elapsed=123.28s allocated=242236760B defined_classes=1725 > > > >>> >> tid=0x00007fa13a806000 nid=0x1c03 pthread-id=0x109708000 waiting > > > >> on > > > >>> >> condition [0x0000000109707000] > > > >>> >> > > java.lang.Thread.State: TIMED_WAITING (sleeping) > > > >>> >> > > JavaThread state: _thread_blocked > > > >>> >> > > Thread: 0x00007fa13a806000 [0x1c03] State: > > > >>> >> _at_safepoint _has_called_back 0 _at_poll_safepoint 0 > > > >>> >> > > JavaThread state: _thread_blocked > > > >>> >> > > at java.lang.Thread.sleep(java.base/Native Method) > > > >>> >> > > ... > > > >>> >> > > ------------------------------------------------------ > > > >>> >> > > > > > >>> >> > > As mentioned above, we have a whole bunch of other > > > >>> >> enhancements to the thread dump similar to this one and would be > > > >>> >> willing to contribute them if there is any interest. > > > >>> >> > > > > > >>> >> > > Thanks and best regards, > > > >>> >> > > Gunter > > > >>> >> > > > > > >>> >> > > > > > >>> >> > > > > >>> >> > > > > >>> >> > > > > >>> >> > > > > >>> >> > > > > >>> >> > > > >>> > > > > >>> > > > > >>> > > > >>> > > > > > > > > > > > > > > > > > > From jini.george at oracle.com Mon Jun 25 10:11:10 2018 From: jini.george at oracle.com (Jini George) Date: Mon, 25 Jun 2018 15:41:10 +0530 Subject: RFR: JDK-8189429: SA: MacOSX: Replace the deprecated PT_ATTACH with PT_ATTACHEXC In-Reply-To: References: <7abea33e-7fb0-fbe0-d463-8999ccdf54da@oracle.com> <0c93044e-7aaf-179b-c80c-45ff4d7e2712@oracle.com> Message-ID: <5ba6916b-35bf-617e-fff8-93f096162047@oracle.com> Thank you, Sharath. May I have a Reviewer to take a look at the MacosxDebuggerLocal code? Thanks, Jini. On 6/25/2018 1:52 PM, Sharath Ballal wrote: > Hi Jini, > > Changes in MacosxDebuggerLocal.m looks good. > > Thanks, > Sharath > > > -----Original Message----- > From: Jini George > Sent: Sunday, June 24, 2018 11:07 PM > To: Erik Joelsson; serviceability-dev at openjdk.java.net; build-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR: JDK-8189429: SA: MacOSX: Replace the deprecated PT_ATTACH with PT_ATTACHEXC > > Hi Erik, > > Thank you very much for looking into this. I have addressed your comments. The latest webrev is at: > > http://cr.openjdk.java.net/~jgeorge/8189429/webrev.06/ > > Thank you, > Jini. > > > On 6/23/2018 3:31 AM, Erik Joelsson wrote: >> Hello Jini, >> >> In general this looks pretty good, but it's also breaking some new >> ground as it's adding generation of native source in the java gensrc >> step. Mixing native code with the java code that the genrcs targets >> and gensrc output directories are meant for seems ok for now, but may >> cause trouble in the future. I'm going to accept it for now though. >> >> In Gensrc-jdk.hotspot.agent.gmk: Please put the new macosx stuff in >> its own section (as delimited by the 80x # lines) and put that whole >> thing inside a conditional for macosx. Also please break line 47 (for >> recipe lines, indent with tab and 4 additional spaces for continuation [1]). >> >> In Lib-jdk.hotspot.agent.gmk: I believe adding extra src should be >> enough as that will implicitly add the same directories as header dirs >> by default. At least that's the intention. >> >> /Erik >> >> [1] http://openjdk.java.net/groups/build/doc/code-conventions.html >> >> On 2018-06-22 11:11, Jini George wrote: >>> Hi all, >>> >>> [Including build-dev also since this includes build related changes]. >>> >>> Requesting reviews for: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8189429 (SA: MacOSX: Replace >>> the deprecated PT_ATTACH with PT_ATTACHEXC) >>> >>> Webrev: http://cr.openjdk.java.net/~jgeorge/8189429/webrev.04/ >>> >>> This is the follow-up solution for the temporary restoration of >>> PT_ATTACH to fix https://bugs.openjdk.java.net/browse/JDK-8184042 >>> (several serviceability/sa tests timed out on MacOS X), which was >>> done in Oct 2017. The mails related to this are at: >>> >>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2017-August >>> /021702.html >>> >>> >>> Changes as compared to the patch sent last year >>> (http://cr.openjdk.java.net/~jgeorge/8184042/webrev.00/): >>> >>> * Addressed the review comments which were provided by Poonam, Dan, >>> Dmitry. >>> * A major change as compared to what was done last year is that the >>> MIG generated files have been included as a part of the JDK build >>> process. >>> * The test case which was provided in the patch last year is no >>> longer required since we have ClhsdbAttach.java testing the same >>> functionality as a part of the SA testsuite now. >>> * Other than that, some minor improvements have been done wrt error >>> handling. >>> >>> The steps for the proposed solution have been provided in JBS. >>> >>> Testing: ALL the SA tests pass on MacOSX and the other Mach5 platforms. >>> >>> Thanks to Sharath, Robin, Gerard and Dan for looking into the changes >>> and providing valuable comments. >>> >>> Thanks, >>> Jini. >> From david.holmes at oracle.com Mon Jun 25 11:15:38 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 25 Jun 2018 21:15:38 +1000 Subject: RFR(S): 8200720: Print additional information in thread dump (times, allocated bytes etc.) In-Reply-To: <6AA6AE61-C852-4C24-A83D-D3570B528D26@sap.com> References: <56817AB5-88FC-4FB9-A271-7206EA69ECA7@sap.com> <2728581E-0E32-4911-B1A9-79AD6CBA81B0@sap.com> <082ca98f-9a68-15a0-95b5-c4d5c733ab36@oracle.com> <933cdacc-9b6f-c8b5-e244-558f38e7888c@oracle.com> <55806aa8-f09b-e9a8-b512-b9676029c494@oracle.com> <0F497BC6-2569-4EF6-9B59-E24A8E1B3A37@sap.com> <5dc08f65e83f4374b8e1f73f218c6d6d@sap.com> <626701f8-b71b-58dc-f837-e5e1d81ef187@oracle.com> <19BE9680-3E6E-4620-B72D-D379F39C1C92@sap.com> <6ce47417-311b-e6cd-b8cd-d46eb7e04c2f@oracle.com> <3795D1AE-CBF4-4BC3-B707-ED77D4DE00D0@sap.com> <02AFA85A-36A7-49D0-A308-E509FFC39C97@sap.com> <1e7e234a-91ea-2c14-9634-08f78b90b225@oracle.com> <6AA6AE61-C852-4C24-A83D-D3570B528D26@sap.com> Message-ID: On 25/06/2018 8:04 PM, Haug, Gunter wrote: > Hi David, > > Thank you. You're right, the setter is not required, I'll remove it. > > I've filed a CSR request: https://bugs.openjdk.java.net/browse/JDK-8205604 > Is this what is expected? It's my first one. Yes that's pretty much it. I added myself as a Reviewer of the CSR and filled in the "scope" information ("implementation"). You can move it to "Finalized" now and then just have to wait for it to be approved before integrating the change. Probably best for your other reviewers to chime back in on this latest version as well. Cheers, David > Thank you, > Gunter > > ?On 25.06.18, 10:38, "David Holmes" wrote: > > Hi Gunter, > > On 25/06/2018 6:26 PM, Haug, Gunter wrote: > > Hi David, > > > > Here is the updated webrev. I've included all the changes you suggested, except for default parameter (see original mail). > > > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v4/ > > > > Are you OK with it now? > > You don't need the setStartTime setter as by definition a "start time" > is only set once. (Though "start" may not be the right term here.) > > Otherwise changes seem okay. > > Note however that you are adding a product flag so that means you need > to file a CSR request. > > Thanks, > David > > > Thanks again, > > Gunter > > > > On 20.06.18, 17:40, "serviceability-dev on behalf of Haug, Gunter" wrote: > > > > Hi David, > > > > Thanks for taking the time to look into this! > > > > > "statistic info" is not very good grammatically. These things are either > > > statistics, or statistical information. So e.g. > > > > > > class ThreadStatistics > > >... > > > > OK, I'll change that! > > > > > const_cast(this)->cooked_allocated_bytes(); > > > > > > Why do we need a const cast to invoke a method on ourselves ?? > > > > cooked_allocated_bytes() is not declared const and we can't make it, as it uses OrderAccess::load_acquire(&_allocated_bytes). Therefor the const_cast. > > > > > src/hotspot/share/runtime/thread.hpp > > > > > > Please put the new #include in alphabetical order. > > > > Wilco! > > > > > I was expecting to see a default parameter used here rather than adding > > > an overload: > > > virtual void print_on(outputStream* st, bool print_extended_info = > > > false) const { print_on(st, print_extended_info); } > > > > Thread inherits from AllocatedObj (in dbg, at least) where virtual void AllocatedObj::print_on(outputStream*) const is declared. Some compilers, e.g. gcc, will complain that this is hidden by Thread::print_on(outputStream*, bool). Others, e.g. clang, are happy with that. We could implement a method with a different name but would that be nicer? > > > > > 2179 static void print_on(outputStream* st, bool print_stacks, bool > > > internal_format, bool print_concurrent_locks, bool extended_thread_info); > > > > > > Again I expected to see a default parameter here - but I didn't check if > > > all callers themselves take the new parameter. ?? > > > > This one has just one single caller, so I think it's OK. > > > > > src/hotspot/share/runtime/vm_operations.hpp > > >... > > > > I'll change it to conform to the guidelines. > > > > > src/hotspot/share/runtime/threadStatisticInfo.hpp > > > > > > There are no includes in this file. It should include the appropriate > > > std header for type definitions, and os.hpp. > > > > Agreed, I'll add the includes. > > > > > > > I'm not sure if class ThreadStatisticInfo needs an allocation type as a > > > super type. > > > > > > Might be worth adding a constructor to give a default value so that you > > > can tell if the statistics have been initialized when they appear in the > > > printout. ? > > > > ThreadStatisticInfo is a data member of Thread. Currently it is initialized in Thread::Thread, but I can add a constructor, if you prefer. > > > > Thanks again, > > Gunter > > > > > > > > On 20.06.18, 09:15, "David Holmes" wrote: > > > > Hi Gunter, > > > > On 19/06/2018 8:51 PM, Haug, Gunter wrote: > > > Hi all, > > > > > > Thanks Chris and Christoph for the reviews! Christoph, I'll incorporate the little improvement you suggested. > > > David, are you OK with the change as well? > > > > Sorry I'd lost track of this one a bit ... > > > > The overall approach now seems okay. > > > > Some naming/terminology issues: > > > > "statistic info" is not very good grammatically. These things are either > > statistics, or statistical information. So e.g. > > > > class ThreadStatistics > > > > ThreadStatistics& statistics() { return _statistics; } > > > > etc. > > > > "extended_thread_info" should really be print_extended_info (similar to > > print_concurrent_locks). You don't need "thread" in there when this is > > always part of a Thread related API. > > > > > > src/hotspot/share/runtime/thread.cpp > > > > const_cast(this)-> > > > > Why do we need a const cast to invoke a method on ourselves ?? > > > > --- > > > > src/hotspot/share/runtime/thread.hpp > > > > Please put the new #include in alphabetical order. > > > > 641 // Printing > > 642 void print_on(outputStream* st, bool extended_thread_info) const; > > 643 virtual void print_on(outputStream* st) const { print_on(st, > > false); } > > > > I was expecting to see a default parameter used here rather than adding > > an overload: > > > > virtual void print_on(outputStream* st, bool print_extended_info = > > false) const { print_on(st, print_extended_info); } > > > > > > 2179 static void print_on(outputStream* st, bool print_stacks, bool > > internal_format, bool print_concurrent_locks, bool extended_thread_info); > > > > Again I expected to see a default parameter here - but I didn't check if > > all callers themselves take the new parameter. ?? > > > > --- > > > > src/hotspot/share/runtime/vm_operations.hpp > > > > 376 VM_PrintThreads() > > 377 { _out = tty; _print_concurrent_locks = PrintConcurrentLocks; ; > > _extended_thread_info = false; } > > 378 VM_PrintThreads(outputStream* out, bool print_concurrent_locks, > > bool extended_thread_info) > > 379 { _out = out; _print_concurrent_locks = print_concurrent_locks; > > _extended_thread_info = extended_thread_info; } > > 380 VMOp_Type type() const > > 381 { return VMOp_PrintThreads; } > > > > Style nits: either keep everything on one line as before (though I agree > > the lines are now too long) or else the style should be: > > > > 380 VMOp_Type type() const { > > 381 return VMOp_PrintThreads; > > 382 } > > > > Also unclear why (existing) VM_PrintThreads constructor doesn't use > > initializer list (like VM_PrintMetadata below it) or default parameters? > > > > --- > > > > src/hotspot/share/runtime/threadStatisticInfo.hpp > > > > There are no includes in this file. It should include the appropriate > > std header for type definitions, and os.hpp. > > > > I'm not sure if class ThreadStatisticInfo needs an allocation type as a > > super type. > > > > Might be worth adding a constructor to give a default value so that you > > can tell if the statistics have been initialized when they appear in the > > printout. ? > > > > --- > > > > Thanks, > > David > > > > > > > Thanks again, > > > Gunter > > > > > > On 12.06.18, 01:13, "Chris Plummer" wrote: > > > > > > Hi Gunter, > > > > > > The changes look fine. I can live with the options parsing in > > > attachListener.cpp. As you point out, unrecognized options were already > > > silently ignored. > > > > > > thanks, > > > > > > Chris > > > > > > On 6/8/18 7:05 AM, Haug, Gunter wrote: > > > > Hi all, > > > > > > > > thanks a lot for all the input! I have prepared a new version of the webrev incorporating the suggestions you made (at least I tried): > > > > > > > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v3/ > > > > > > > > This version outputs the thread times unconditionally while the other information is guarded by a flags. I think, most participants found the thread times the most valuable information and had no (strong) objections to adding them unconditionally. > > > > > > > > @David > > > > Implementation is much simpler with conditional output only for JavaThreads. I could get rid of the terrible hack (changing the flag) without having to change too many source files. > > > > > > > > The information on allocated bytes is present in the Thread class already before this proposed change, I just print it. It might be sensible to move _allocated_bytes and the respective methods to the ThreadStatisticInfo class as G?tz suggested. I haven?t done that in the current version, though. > > > > > > > > @David and Thomas > > > > I've removed the pthread-id output. I'm unsure myself what it could be good for. Anyway, we could add it with a separate change (with a better implementation) if there is a need to. > > > > > > > > @Chris > > > > As you have written, the user has no direct contact to the attach listener of the VM and jstack won't misinterpret e.g. -help. I agree that the parsing in attachListener.cpp could be more robust. However, it hasn't been so far either. All that is done in the current implementation is a strcmp to -l everything else will be silently ignored. I can try to make this more robust or we could say that the new output is only available via jcmd. OTH nothing bad can happen with the current version of the proposed change, so we could also leave it as it is. What would you prefer? > > > > > > > > @Kirk and Thomas > > > > Implementing a new diagnostic command is certainly well worth a thought. However, it appears to me that it's not necessary in this case. There are already flags to jstack and jcmd Thread.print, so it's not that uncommon. The amount of information that would be available by a new diagnostic command and that is now added to the thread dump is quite small and I think that is tolerable. Moreover, and most importantly, our support team is use to find the information in the thread dump and it would make things easier to us if I could leave it there. > > > > > > > > Thanks again and have a nice weekend, > > > > Gunter > > > > > > > > > > > > On 06.06.18, 04:56, "David Holmes" wrote: > > > > > > > > Hi Goetz, > > > > > > > > On 5/06/2018 11:07 PM, Lindenmaier, Goetz wrote: > > > > > Hi > > > > > > > > > > this discussion touched a lot of points so far, which seem > > > > > to lead to different conclusions. > > > > > > > > > > I think we should look at the values printed: > > > > > > > > > > 1. cpu=6300.65ms elapsed=123.28s > > > > > Overhead > > > > > cpu time: > > > > > * system calls at thread dump time > > > > > elapsed time: > > > > > * 1 system call at thread creation time > > > > > * 1 64-bit field per thread for the thread start time > > > > > * 1 system call at thread dump time > > > > > > > > > > As I understand, JDK-8143176 would have had to get the > > > > > time at each locking, which is much more time critical > > > > > than doing this during thread creation. While > > > > > > > > Correct. > > > > > > > > > the time a lock was held would be much more useful, > > > > > the ratio here, combined with knowledge about the application, > > > > > could lead to the conclusion that the thread is wrongly > > > > > blocked at much lower cost. > > > > > > > > Agreed. I see no issue with unconditionally adding this information as > > > > it should address some of the concerns of 8143176 as well. > > > > > > > > > > > > > > 2. pthread-id=0x109708000 > > > > > Overhead: > > > > > * a field access at thread dump time. Negligible I would say. > > > > > > > > I'd overlooked this addition, but it is of course the chunk of ifdef > > > > code in osThread.cpp that I objected to. The problem I have here is the > > > > convoluted mess of "thread identifiers" that we already have. We > > > > currently print: > > > > > > > > st->print("tid=" INTPTR_FORMAT " ", p2i(this)); > > > > > > > > which is just the address of the Thread/JavaThread object. Then: > > > > > > > > st->print("nid=0x%x ", thread_id()); > > > > > > > > where 'n' is supposed to represent "native thread id", which is: > > > > - linux: kernel thread id from syscall gettid > > > > - solaris: thread library identity from thr_create() or thr_self() > > > > - windows: thread id from _beginthreadex > > > > - OS X: kernel thread id from pthread_mach_thread_np(pthread_self()); > > > > - Other BSD: kernel (?) thread id from syscall thr_self or getthrid > > > > - AIX: thread library identity from pthread_create() or pthread_self() > > > > > > > > It's telling that the code Gunter added gets the thread library id on > > > > linux, but a "kernel thread id" on other platforms! Do we want to see a > > > > thread library id and a kernel thread id? IIRC nid is supposed be the id > > > > you need to see the thread in a debugger. In which case I'm unclear what > > > > role the thread id Gunter wants to add is playing? I don't think we need > > > > to see kernel ids in general, and pthread-id isn't useful for debugging > > > > is it? > > > > > > > > > > > > > > 3. allocated=242236760B defined_classes=1725 > > > > > Overhead: > > > > > * 1 64-bit field per thread. > > > > > * counter increment on class creation > > > > > > > > > > Especially defined_classes seems to be controversial. > > > > > As this only makes sense for Java threads, this could > > > > > be printed in a line of it's own in Threads::print_on_error() > > > > > (which already gets a flag to customize for jstack: > > > > > print_concurrent_locks) by calling a dedicated function in Thread. > > > > > No need for flag PrintExtendedThreadInfo thus. > > > > > > > > > > But I would propose to drop this information as it is > > > > > too controversial. > > > > > > > > Agreed. > > > > > > > > > This leaves the times and the pthread-id to be decided whether > > > > > they are worth the cost. If so, I think they should be printed > > > > > unconditional. > > > > > > > > Agreed. With some refinement of the "thread id" issue. > > > > > > > > > If a flag is required to avoid bloating the info in the default case, > > > > > I would include more infos, as os_prio and nid under that flag. > > > > > > > > Of course those things exist for good reason. The native id is supposed > > > > to be what aids in you matching what you see in our thread dumps with > > > > what you see in a debugger. Priority is less of an issue these days, but > > > > there was a time ... ;-) > > > > > > > > > As jstack is deprecated, changes to jstack could be skipped always > > > > > disabling the new printouts. > > > > > > > > > > Finally, I would propose to move _allocated_bytes into > > > > > threadStatisticInfo.hpp. > > > > > > > > Didn't that part get dropped? (You don't need it for the time info or > > > > thread id). That's a GC logging issue to me. > > > > > > > > Thanks, > > > > David > > > > > > > > > Best regards, > > > > > Goetz. > > > > > > > > > > > > > > > > > > > > > > > > >> -----Original Message----- > > > > >> From: serviceability-dev [mailto:serviceability-dev- > > > > >> bounces at openjdk.java.net] On Behalf Of David Holmes > > > > >> Sent: Dienstag, 5. Juni 2018 04:53 > > > > >> To: Haug, Gunter ; Chris Plummer > > > > >> ; serviceability-dev > > > >> dev at openjdk.java.net>; Langer, Christoph ; > > > > >> hotspot-runtime-dev at openjdk.java.net > > > > >> Subject: Re: RFR(S): 8200720: Print additional information in thread dump > > > > >> (times, allocated bytes etc.) > > > > >> > > > > >> Hi Gunter, > > > > >> > > > > >> On 5/06/2018 1:27 AM, Haug, Gunter wrote: > > > > >>> Hi David, Chris, > > > > >>> > > > > >>> would it be an option to unconditionally print the additional information? > > > > >> Regardless which way this is implemented it will be rather complicated and it > > > > >> only switches on/off a few field in the thread dump. > > > > >> > > > > >> I'm not convinced this is all suitable information for a thread stack > > > > >> dump. I can see the time information being useful if using the dump to > > > > >> try and diagnoze a hang or responsiveness issue. But the allocated-bytes > > > > >> and classes-defined just doesn't seem useful in the context of a thread > > > > >> dump to me. Anyone interested in allocation stats is going to be looking > > > > >> at GC logs etc which is where this belongs. Ditto the class-stats belong > > > > >> in some kind of classloading logging IMHO. > > > > >> > > > > >> I'm very reluctant to burden the implementation with capturing these > > > > >> kinds of stats, just to use for diagnostic purposes that may not even be > > > > >> asked for. I'm very much in the "you shouldn't pay for what you don't > > > > >> use" camp in that regard. (See my comments in JDK-8143176 referenced by > > > > >> Sean.) > > > > >> > > > > >> The ThreadStatisticInfo adds overhead to every thread object - and I'd > > > > >> have to suspect there could be overlap with whatever flight recorder > > > > >> sticks in there too. (Thread has become even more bloated in recent time!). > > > > >> > > > > >> Cheers, > > > > >> David > > > > >> > > > > >> > > > > >>> Thanks, > > > > >>> Gunter > > > > >>> > > > > >>> On 04.06.18, 01:13, "David Holmes" wrote: > > > > >>> > > > > >>> Hi Gunter, Chris, > > > > >>> > > > > >>> Sorry just trying to catch up and this is only a partial look so far ... > > > > >>> > > > > >>> BTW these changes are not limited to serviceability code so adding in > > > > >>> runtime team as well. > > > > >>> > > > > >>> On 2/06/2018 9:12 AM, Chris Plummer wrote: > > > > >>> > Hi Gunter, > > > > >>> > > > > > >>> > On 6/1/18 3:17 AM, Haug, Gunter wrote: > > > > >>> >> Hi Chris, > > > > >>> >> > > > > >>> >> thanks for looking into this! > > > > >>> >> > > > > >>> >> Re the synchronization: The value is stored only in a VM operation at > > > > >>> >> a safepoint by the VM thread. I was of the opinion, that this could > > > > >>> >> not be interrupted by a second VM operation (of the same type). Or > > > > >> am > > > > >>> >> I missing something else? > > > > >>> > I was really thinking more about the temporary changing of > > > > >>> > PrintExtendedThreadInfo, not the value stored in the VMOp. You may > > > > >> be be > > > > >>> > correct that no more than one VMOp is executing, but while it is > > > > >>> > executing it is has changed the value of PrintExtendedThreadInfo, > > > > >> which > > > > >>> > might have an impact on anything else that triggers printing thread info > > > > >>> > while the VMOp is executing. > > > > >>> > > > > >>> Even if nothing else can possibly be running during the safepoint I find > > > > >>> it extremely bad form to change a command-line flag in this way, > > > > >>> particularly from a safepoint! > > > > >>> > > > > >>> If this flag is intended to be dynamically enabled as part of a dcmd > > > > >>> then it should also be a Manageable flag IMHO. > > > > >>> > > > > >>> That said ... > > > > >>> > > > > >>> >> I did think about passing an argument to the various print_on > > > > >> member > > > > >>> >> functions of the thread classes, but this would require rather > > > > >>> >> extensive code changes for a rather tiny extension. Therefore I feel > > > > >>> >> doing it like this is the lesser evil. > > > > >>> > > > > >>> ... it's obviously not truly a global setting, but one that is needed on > > > > >>> a per-print-request basis. The flag is just the default, but if the > > > > >>> default is off you still want to enable extended printing on a > > > > >>> per-request basis. > > > > >>> > > > > >>> The current print_on mechanics is not set up for this kind of > > > > >>> flexibility. I think this needs more thought. > > > > >>> > > > > >>> --- > > > > >>> > > > > >>> Re osThread.cpp shared code changes ... I really hate to see all the > > > > >>> ifdefs in there. Please add a pd_print_on function to support this if > > > > >>> you truly want platform specific information. > > > > >>> > > > > >>> --- > > > > >>> > > > > >>> threadStatisticInfo.hpp > > > > >>> > > > > >>> typo: getElepsedTime() -> getElapsedTime() > > > > >>> > > > > >>> Thanks, > > > > >>> David > > > > >>> > > > > >>> > > > > >>> > > > > >>> >> > > > > >>> >> Re thread_dump(): I think it's correct or, well, at least it works ;-) > > > > >>> >> In fact jstack will transfer the "-e" and "-l" in only one string, > > > > >>> >> i.e. op->arg(0). > > > > >>> > So you are saying that op->arg(0) is "-e -l" when using jstack? I think > > > > >>> > you really need to clean up the parsing. As it stands right now passing, > > > > >>> > I get the feeling that if a user erroneously asks for help by using > > > > >>> > "jcmd Thread.Print -help", it will end up executing with -e an -l > > > > >>> > enabled, and no failure for the invalid "-help" option. > > > > >>> >> Christoph has already explained my reasoning. But I agree, it's not > > > > >>> >> nice and I would be happy to do it like Christoph suggested. > > > > >>> > I'll respond separately to his suggestion. > > > > >>> > > > > > >>> > thanks, > > > > >>> > > > > > >>> > Chris > > > > >>> >> > > > > >>> >> And typo fixed, sorry. > > > > >>> >> > > > > >>> >> Thanks again, > > > > >>> >> Gunter > > > > >>> >> > > > > >>> >> On 01.06.18, 00:03, "Chris Plummer" > > > > >> wrote: > > > > >>> >> > > > > >>> >> Hi Gunter, > > > > >>> >> globals.hpp: fix typo "informatiuon" > > > > >>> >> I worry a little bit about the synchronizing (if that's the right > > > > >>> >> word) > > > > >>> >> of PrintExtendedThreadInfo and the dcmd's -e flag. When using -e, > > > > >>> >> you > > > > >>> >> are temporarily enabling PrintExtendedThreadInfo if it was false. > > > > >>> >> This > > > > >>> >> temporarily changes the behavior of thread dumps, and could > > > > >>> >> impact other > > > > >>> >> uses that happen in parallel. Also, could two simultaneous uses > > > > >>> >> of -e > > > > >>> >> result in PrintExtendedThreadInfo not getting restored properly? > > > > >>> >> thread_dump() doesn't look right. It looks like you are iterating > > > > >>> >> char > > > > >>> >> by char over the argument, and expect something like "-el" to be > > > > >>> >> specified rather then "-e -l". The loop should be iterating over > > > > >>> >> op->arg(i), not op->arg(0)[i]. > > > > >>> >> The rest of the changes look fine. > > > > >>> >> thanks, > > > > >>> >> Chris > > > > >>> >> On 5/30/18 8:12 AM, Haug, Gunter wrote: > > > > >>> >> > Hi all, > > > > >>> >> > > > > > >>> >> > As Chris proposed, I have made an the extended output > > > > >>> >> switchable. There is an VM flag (PrintExtendedThreadInfo), which is > > > > >>> >> false by default. Moreover, there is an Option (-e) which can be used > > > > >>> >> with jcmd Thread.print as well as with jstack. The -e option > > > > >>> >> essentially sets PrintExtendedThreadInfo true just for the respective > > > > >>> >> thread dump. > > > > >>> >> > > > > > >>> >> > Here is the updated webrev: > > > > >>> >> > > > > > >>> >> > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v2 > > > > >>> >> > > > > > >>> >> > (https://bugs.openjdk.java.net/browse/JDK-8200720) > > > > >>> >> > > > > > >>> >> > Thanks, > > > > >>> >> > Gunter > > > > >>> >> > > > > > >>> >> > > > > > >>> >> > On 02.05.18, 17:07, "serviceability-dev on behalf of Haug, > > > > >>> >> Gunter" > > > >>> >> gunter.haug at sap.com> wrote: > > > > >>> >> > > > > > >>> >> > Hi Chris, > > > > >>> >> > > > > > >>> >> > Thanks for looking into this. > > > > >>> >> > You're right, there is a little more we have. We have > > > > >>> >> implemented an IO tracing mechanism which - rather as a byproduct - > > > > >>> >> keeps track of bytes read and written per thread. However, this of > > > > >>> >> course requires changes not only in hotspot. We would be happy to > > > > >>> >> contribute this as well, but this is a far bigger change and will > > > > >>> >> probably lead to a far bigger discussion. Anyway, with the number of > > > > >>> >> bytes read, the number of classes defined doesn't look that arbitrary > > > > >>> >> anymore, as one can correlate IO to class loading. > > > > >>> >> > > > > > >>> >> > Regarding the verbose option I think that's a good idea! > > > > >>> >> > > > > > >>> >> > Thanks again, > > > > >>> >> > Gunter > > > > >>> >> > > > > > >>> >> > On 01.05.18, 22:55, "Chris Plummer" > > > > >>> >> wrote: > > > > >>> >> > > > > > >>> >> > Hi Gunter, > > > > >>> >> > > > > > >>> >> > The output you are adding is all useful. I think the > > > > >>> >> question is (and > > > > >>> >> > I'd like to see a few people chime in on this for this > > > > >>> >> review) is > > > > >>> >> > whether or not all of it is the appropriate for a > > > > >>> >> thread's stack dump. > > > > >>> >> > For example, defined_classes is on the fringe of what > > > > >>> >> I would call > > > > >>> >> > generally useful info in a stack dump. Sure, there > > > > >>> >> might be that rare > > > > >>> >> > case when you need it, but how often compared to other > > > > >>> >> useful info > > > > >>> >> > maintained on a per thread basis. How many other bits > > > > >>> >> of useful info are > > > > >>> >> > not being printed in favor of defined_classes? It > > > > >>> >> seems you have more in > > > > >>> >> > the queue. How many? I'm worried about how cluttered > > > > >>> >> the stack dumps > > > > >>> >> > will get. Maybe we should add some sort of verbose > > > > >>> >> thread dumping > > > > >>> >> > option. Just a thought. > > > > >>> >> > > > > > >>> >> > As for the implementation, overall it looks good, but > > > > >>> >> I can't speak to > > > > >>> >> > whether or not you are doing proper accounting of > > > > >>> >> defined_classes and > > > > >>> >> > bytes allocated. You'll need input from someone with > > > > >>> >> more knowledge of > > > > >>> >> > those areas. We'll also need to do some testing to > > > > >>> >> make sure tool tests > > > > >>> >> > are not impacted. > > > > >>> >> > > > > > >>> >> > thanks, > > > > >>> >> > > > > > >>> >> > Chris > > > > >>> >> > > > > > >>> >> > On 4/30/18 2:51 AM, Haug, Gunter wrote: > > > > >>> >> > > Hi, > > > > >>> >> > > > > > > >>> >> > > this is an update to an RFR I posted on hotspot-dev, > > > > >>> >> but it is probably more suitable to post it here. Can I please have a > > > > >>> >> review and a sponsor for the following enhancement: > > > > >>> >> > > > > > > >>> >> > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v1 > > > > >>> >> > > https://bugs.openjdk.java.net/browse/JDK-8200720 > > > > >>> >> > > > > > > >>> >> > > We at SAP have extended the thread dumps (obtained > > > > >>> >> by jstack or jcmd) by several fields providing thread specific > > > > >>> >> information. These extensions are quite popular with our support > > > > >> team. > > > > >>> >> With some knowledge of the architecture of the application, they > > > > >> often > > > > >>> >> allow for quick and simple diagnosis of a running system. Therefore > > > > >> we > > > > >>> >> would like to contribute these enhancements. > > > > >>> >> > > > > > > >>> >> > > I took a few simple examples here, namely cpu time, > > > > >>> >> elapsed time since thread creation, bytes allocated and classes > > > > >>> >> defined by the thread and the pthread-id or equivalent on platforms > > > > >>> >> where it makes sense. Provided it is known how the application > > > > >> should > > > > >>> >> behave, a misbehaving thread can identified easily. > > > > >>> >> > > > > > > >>> >> > > There is no measurable overhead for this > > > > >>> >> enhancement. However, it may be a problem that the format of the > > > > >>> >> output is changed. Tools parsing the output may have to be changed. > > > > >>> >> > > > > > > >>> >> > > Here is an example of the output generated: > > > > >>> >> > > > > > > >>> >> > > ------------------------------------------------------ > > > > >>> >> > > > > > > >>> >> > > "main" #1 prio=5 os_prio=31 cpu=6300.65ms > > > > >>> >> elapsed=123.28s allocated=242236760B defined_classes=1725 > > > > >>> >> tid=0x00007fa13a806000 nid=0x1c03 pthread-id=0x109708000 waiting > > > > >> on > > > > >>> >> condition [0x0000000109707000] > > > > >>> >> > > java.lang.Thread.State: TIMED_WAITING (sleeping) > > > > >>> >> > > JavaThread state: _thread_blocked > > > > >>> >> > > Thread: 0x00007fa13a806000 [0x1c03] State: > > > > >>> >> _at_safepoint _has_called_back 0 _at_poll_safepoint 0 > > > > >>> >> > > JavaThread state: _thread_blocked > > > > >>> >> > > at java.lang.Thread.sleep(java.base/Native Method) > > > > >>> >> > > ... > > > > >>> >> > > ------------------------------------------------------ > > > > >>> >> > > > > > > >>> >> > > As mentioned above, we have a whole bunch of other > > > > >>> >> enhancements to the thread dump similar to this one and would be > > > > >>> >> willing to contribute them if there is any interest. > > > > >>> >> > > > > > > >>> >> > > Thanks and best regards, > > > > >>> >> > > Gunter > > > > >>> >> > > > > > > >>> >> > > > > > > >>> >> > > > > > >>> >> > > > > > >>> >> > > > > > >>> >> > > > > > >>> >> > > > > > >>> >> > > > > >>> > > > > > >>> > > > > > >>> > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From daniel.daugherty at oracle.com Mon Jun 25 13:27:37 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 25 Jun 2018 09:27:37 -0400 Subject: RFR(XS): 8205195 NestedThreadsListHandleInErrorHandlingTest fails because hs_err doesn't contain _nested_thread_list_max In-Reply-To: <126cc7a5-a92b-a22c-6d28-e35779ee69ef@oracle.com> References: <126cc7a5-a92b-a22c-6d28-e35779ee69ef@oracle.com> Message-ID: <476d805c-2bfc-fa31-3f4e-e1e44129d9b2@oracle.com> On 6/25/18 1:57 AM, David Holmes wrote: > Hi Dan, > > On 21/06/2018 10:18 AM, Daniel D. Daugherty wrote: >> Greetings, >> >> I have a fix for a recent (very rare) Thread-SMR related test failure. >> >> Since the fix is related to the ErrorHandling tests and affects >> hs_err_pid >> file generation, this code review is being sent to both the Runtime and >> the Serviceability teams. Please make sure you reply-all to any >> responses >> so we have complete review threads on both aliases. >> >> Bug URL: https://bugs.openjdk.java.net/browse/JDK-8205195 >> >> Webrev URL: >> http://cr.openjdk.java.net/~dcubed/8205195-webrev/0-for-jdk-jdk/ >> >> The bug itself contains analysis about the root cause of the bug and >> the comment updates to the code describes the no win scenario that the >> hs_err_pid file generation code is in. Of course, I also have a comment >> where I was able to harden the ErrorHandling tests. I did manage to >> resist the urge to mention the "Kobiyashi Maru" [1] in the new comments. >> >> Testing: Mach5 >> builds-tier1,jdk-tier1,jdk-tier2,hs-tier1,hs-tier2,hs-tier3 >> ??? ? ? ? on the usual Oracle platforms. >> >> Thanks, in advance, for any comments, questions or suggestions. > > I don't quite follow the fix. I think this comment explains what the fix is _trying_ to do: 1702?? // We grab Threads_lock to keep ThreadsSMRSupport::print_info_on() 1703?? // from racing with Threads::add() or Threads::remove() as we 1704?? // generate the hs_err_pid file. This makes our ErrorHandling tests 1705?? // more stable. > Won't you self-deadlock on acquiring the Threads_lock in the secondary > error handler test, due to the recursive call to controlled_crash ? I missed that possibility, but now I'm puzzled why my testing didn't reveal this situation. We have a test for secondary error handling and it should have self-deadlocked (and failed). I'll investigate. Dan > > David > >> Dan >> >> [1] https://www.urbandictionary.com/define.php?term=Kobayashi%20Maru >> From gunter.haug at sap.com Mon Jun 25 14:22:22 2018 From: gunter.haug at sap.com (Haug, Gunter) Date: Mon, 25 Jun 2018 14:22:22 +0000 Subject: RFR(S): 8200720: Print additional information in thread dump (times, allocated bytes etc.) In-Reply-To: <9f5a404a-24e6-9131-3c0d-80c5c01ccfca@oracle.com> References: <56817AB5-88FC-4FB9-A271-7206EA69ECA7@sap.com> <2728581E-0E32-4911-B1A9-79AD6CBA81B0@sap.com> <082ca98f-9a68-15a0-95b5-c4d5c733ab36@oracle.com> <933cdacc-9b6f-c8b5-e244-558f38e7888c@oracle.com> <55806aa8-f09b-e9a8-b512-b9676029c494@oracle.com> <0F497BC6-2569-4EF6-9B59-E24A8E1B3A37@sap.com> <5dc08f65e83f4374b8e1f73f218c6d6d@sap.com> <626701f8-b71b-58dc-f837-e5e1d81ef187@oracle.com> <19BE9680-3E6E-4620-B72D-D379F39C1C92@sap.com> <6ce47417-311b-e6cd-b8cd-d46eb7e04c2f@oracle.com> <3795D1AE-CBF4-4BC3-B707-ED77D4DE00D0@sap.com> <9f5a404a-24e6-9131-3c0d-80c5c01ccfca@oracle.com> Message-ID: <70FECB3F-4E90-4B2C-AB22-09B42069323A@sap.com> Hi all, Thank you, David for taking the time to do the review. Here is hopefully the last version of the change: http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v5/ Following David's suggestion, _start_time_stamp is now constant and set by the constructor and the setter has been removed. This is the only difference to the previous version. Chris and Christoph, are you still OK with this? Thanks, Gunter ?On 25.06.18, 12:29, "David Holmes" wrote: Hi Gunter, I see what you mean about the initialization point of the statistics. Personally I'd prefer to see the startTime as read-only and only set by the constructor rather than having a general-purpose setter function for it. Thanks, David On 21/06/2018 1:28 AM, Haug, Gunter wrote: > Hi David, > > Thanks for taking the time to look into this! > >> "statistic info" is not very good grammatically. These things are either >> statistics, or statistical information. So e.g. >> >> class ThreadStatistics >> ... > > OK, I'll change that! > >> const_cast(this)->cooked_allocated_bytes(); >> >> Why do we need a const cast to invoke a method on ourselves ?? > > cooked_allocated_bytes() is not declared const and we can't make it, as it uses OrderAccess::load_acquire(&_allocated_bytes). Therefor the const_cast. > >> src/hotspot/share/runtime/thread.hpp >> >> Please put the new #include in alphabetical order. > > Wilco! > >> I was expecting to see a default parameter used here rather than adding >> an overload: >> virtual void print_on(outputStream* st, bool print_extended_info = >> false) const { print_on(st, print_extended_info); } > > Thread inherits from AllocatedObj (in dbg, at least) where virtual void AllocatedObj::print_on(outputStream*) const is declared. Some compilers, e.g. gcc, will complain that this is hidden by Thread::print_on(outputStream*, bool). Others, e.g. clang, are happy with that. We could implement a method with a different name but would that be nicer? > >> 2179 static void print_on(outputStream* st, bool print_stacks, bool >> internal_format, bool print_concurrent_locks, bool extended_thread_info); >> >> Again I expected to see a default parameter here - but I didn't check if >> all callers themselves take the new parameter. ?? > > This one has just one single caller, so I think it's OK. > >> src/hotspot/share/runtime/vm_operations.hpp >> ... > > I'll change it to conform to the guidelines. > >> src/hotspot/share/runtime/threadStatisticInfo.hpp >> >> There are no includes in this file. It should include the appropriate >> std header for type definitions, and os.hpp. > > Agreed, I'll add the includes. > > >> I'm not sure if class ThreadStatisticInfo needs an allocation type as a >> super type. >> >> Might be worth adding a constructor to give a default value so that you >> can tell if the statistics have been initialized when they appear in the >> printout. ? > > ThreadStatisticInfo is a data member of Thread. Currently it is initialized in Thread::Thread, but I can add a constructor, if you prefer. > > Thanks again, > Gunter > > > > On 20.06.18, 09:15, "David Holmes" wrote: > > Hi Gunter, > > On 19/06/2018 8:51 PM, Haug, Gunter wrote: > > Hi all, > > > > Thanks Chris and Christoph for the reviews! Christoph, I'll incorporate the little improvement you suggested. > > David, are you OK with the change as well? > > Sorry I'd lost track of this one a bit ... > > The overall approach now seems okay. > > Some naming/terminology issues: > > "statistic info" is not very good grammatically. These things are either > statistics, or statistical information. So e.g. > > class ThreadStatistics > > ThreadStatistics& statistics() { return _statistics; } > > etc. > > "extended_thread_info" should really be print_extended_info (similar to > print_concurrent_locks). You don't need "thread" in there when this is > always part of a Thread related API. > > > src/hotspot/share/runtime/thread.cpp > > const_cast(this)-> > > Why do we need a const cast to invoke a method on ourselves ?? > > --- > > src/hotspot/share/runtime/thread.hpp > > Please put the new #include in alphabetical order. > > 641 // Printing > 642 void print_on(outputStream* st, bool extended_thread_info) const; > 643 virtual void print_on(outputStream* st) const { print_on(st, > false); } > > I was expecting to see a default parameter used here rather than adding > an overload: > > virtual void print_on(outputStream* st, bool print_extended_info = > false) const { print_on(st, print_extended_info); } > > > 2179 static void print_on(outputStream* st, bool print_stacks, bool > internal_format, bool print_concurrent_locks, bool extended_thread_info); > > Again I expected to see a default parameter here - but I didn't check if > all callers themselves take the new parameter. ?? > > --- > > src/hotspot/share/runtime/vm_operations.hpp > > 376 VM_PrintThreads() > 377 { _out = tty; _print_concurrent_locks = PrintConcurrentLocks; ; > _extended_thread_info = false; } > 378 VM_PrintThreads(outputStream* out, bool print_concurrent_locks, > bool extended_thread_info) > 379 { _out = out; _print_concurrent_locks = print_concurrent_locks; > _extended_thread_info = extended_thread_info; } > 380 VMOp_Type type() const > 381 { return VMOp_PrintThreads; } > > Style nits: either keep everything on one line as before (though I agree > the lines are now too long) or else the style should be: > > 380 VMOp_Type type() const { > 381 return VMOp_PrintThreads; > 382 } > > Also unclear why (existing) VM_PrintThreads constructor doesn't use > initializer list (like VM_PrintMetadata below it) or default parameters? > > --- > > src/hotspot/share/runtime/threadStatisticInfo.hpp > > There are no includes in this file. It should include the appropriate > std header for type definitions, and os.hpp. > > I'm not sure if class ThreadStatisticInfo needs an allocation type as a > super type. > > Might be worth adding a constructor to give a default value so that you > can tell if the statistics have been initialized when they appear in the > printout. ? > > --- > > Thanks, > David > > > > Thanks again, > > Gunter > > > > On 12.06.18, 01:13, "Chris Plummer" wrote: > > > > Hi Gunter, > > > > The changes look fine. I can live with the options parsing in > > attachListener.cpp. As you point out, unrecognized options were already > > silently ignored. > > > > thanks, > > > > Chris > > > > On 6/8/18 7:05 AM, Haug, Gunter wrote: > > > Hi all, > > > > > > thanks a lot for all the input! I have prepared a new version of the webrev incorporating the suggestions you made (at least I tried): > > > > > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v3/ > > > > > > This version outputs the thread times unconditionally while the other information is guarded by a flags. I think, most participants found the thread times the most valuable information and had no (strong) objections to adding them unconditionally. > > > > > > @David > > > Implementation is much simpler with conditional output only for JavaThreads. I could get rid of the terrible hack (changing the flag) without having to change too many source files. > > > > > > The information on allocated bytes is present in the Thread class already before this proposed change, I just print it. It might be sensible to move _allocated_bytes and the respective methods to the ThreadStatisticInfo class as G?tz suggested. I haven?t done that in the current version, though. > > > > > > @David and Thomas > > > I've removed the pthread-id output. I'm unsure myself what it could be good for. Anyway, we could add it with a separate change (with a better implementation) if there is a need to. > > > > > > @Chris > > > As you have written, the user has no direct contact to the attach listener of the VM and jstack won't misinterpret e.g. -help. I agree that the parsing in attachListener.cpp could be more robust. However, it hasn't been so far either. All that is done in the current implementation is a strcmp to -l everything else will be silently ignored. I can try to make this more robust or we could say that the new output is only available via jcmd. OTH nothing bad can happen with the current version of the proposed change, so we could also leave it as it is. What would you prefer? > > > > > > @Kirk and Thomas > > > Implementing a new diagnostic command is certainly well worth a thought. However, it appears to me that it's not necessary in this case. There are already flags to jstack and jcmd Thread.print, so it's not that uncommon. The amount of information that would be available by a new diagnostic command and that is now added to the thread dump is quite small and I think that is tolerable. Moreover, and most importantly, our support team is use to find the information in the thread dump and it would make things easier to us if I could leave it there. > > > > > > Thanks again and have a nice weekend, > > > Gunter > > > > > > > > > On 06.06.18, 04:56, "David Holmes" wrote: > > > > > > Hi Goetz, > > > > > > On 5/06/2018 11:07 PM, Lindenmaier, Goetz wrote: > > > > Hi > > > > > > > > this discussion touched a lot of points so far, which seem > > > > to lead to different conclusions. > > > > > > > > I think we should look at the values printed: > > > > > > > > 1. cpu=6300.65ms elapsed=123.28s > > > > Overhead > > > > cpu time: > > > > * system calls at thread dump time > > > > elapsed time: > > > > * 1 system call at thread creation time > > > > * 1 64-bit field per thread for the thread start time > > > > * 1 system call at thread dump time > > > > > > > > As I understand, JDK-8143176 would have had to get the > > > > time at each locking, which is much more time critical > > > > than doing this during thread creation. While > > > > > > Correct. > > > > > > > the time a lock was held would be much more useful, > > > > the ratio here, combined with knowledge about the application, > > > > could lead to the conclusion that the thread is wrongly > > > > blocked at much lower cost. > > > > > > Agreed. I see no issue with unconditionally adding this information as > > > it should address some of the concerns of 8143176 as well. > > > > > > > > > > > 2. pthread-id=0x109708000 > > > > Overhead: > > > > * a field access at thread dump time. Negligible I would say. > > > > > > I'd overlooked this addition, but it is of course the chunk of ifdef > > > code in osThread.cpp that I objected to. The problem I have here is the > > > convoluted mess of "thread identifiers" that we already have. We > > > currently print: > > > > > > st->print("tid=" INTPTR_FORMAT " ", p2i(this)); > > > > > > which is just the address of the Thread/JavaThread object. Then: > > > > > > st->print("nid=0x%x ", thread_id()); > > > > > > where 'n' is supposed to represent "native thread id", which is: > > > - linux: kernel thread id from syscall gettid > > > - solaris: thread library identity from thr_create() or thr_self() > > > - windows: thread id from _beginthreadex > > > - OS X: kernel thread id from pthread_mach_thread_np(pthread_self()); > > > - Other BSD: kernel (?) thread id from syscall thr_self or getthrid > > > - AIX: thread library identity from pthread_create() or pthread_self() > > > > > > It's telling that the code Gunter added gets the thread library id on > > > linux, but a "kernel thread id" on other platforms! Do we want to see a > > > thread library id and a kernel thread id? IIRC nid is supposed be the id > > > you need to see the thread in a debugger. In which case I'm unclear what > > > role the thread id Gunter wants to add is playing? I don't think we need > > > to see kernel ids in general, and pthread-id isn't useful for debugging > > > is it? > > > > > > > > > > > 3. allocated=242236760B defined_classes=1725 > > > > Overhead: > > > > * 1 64-bit field per thread. > > > > * counter increment on class creation > > > > > > > > Especially defined_classes seems to be controversial. > > > > As this only makes sense for Java threads, this could > > > > be printed in a line of it's own in Threads::print_on_error() > > > > (which already gets a flag to customize for jstack: > > > > print_concurrent_locks) by calling a dedicated function in Thread. > > > > No need for flag PrintExtendedThreadInfo thus. > > > > > > > > But I would propose to drop this information as it is > > > > too controversial. > > > > > > Agreed. > > > > > > > This leaves the times and the pthread-id to be decided whether > > > > they are worth the cost. If so, I think they should be printed > > > > unconditional. > > > > > > Agreed. With some refinement of the "thread id" issue. > > > > > > > If a flag is required to avoid bloating the info in the default case, > > > > I would include more infos, as os_prio and nid under that flag. > > > > > > Of course those things exist for good reason. The native id is supposed > > > to be what aids in you matching what you see in our thread dumps with > > > what you see in a debugger. Priority is less of an issue these days, but > > > there was a time ... ;-) > > > > > > > As jstack is deprecated, changes to jstack could be skipped always > > > > disabling the new printouts. > > > > > > > > Finally, I would propose to move _allocated_bytes into > > > > threadStatisticInfo.hpp. > > > > > > Didn't that part get dropped? (You don't need it for the time info or > > > thread id). That's a GC logging issue to me. > > > > > > Thanks, > > > David > > > > > > > Best regards, > > > > Goetz. > > > > > > > > > > > > > > > > > > > >> -----Original Message----- > > > >> From: serviceability-dev [mailto:serviceability-dev- > > > >> bounces at openjdk.java.net] On Behalf Of David Holmes > > > >> Sent: Dienstag, 5. Juni 2018 04:53 > > > >> To: Haug, Gunter ; Chris Plummer > > > >> ; serviceability-dev > > >> dev at openjdk.java.net>; Langer, Christoph ; > > > >> hotspot-runtime-dev at openjdk.java.net > > > >> Subject: Re: RFR(S): 8200720: Print additional information in thread dump > > > >> (times, allocated bytes etc.) > > > >> > > > >> Hi Gunter, > > > >> > > > >> On 5/06/2018 1:27 AM, Haug, Gunter wrote: > > > >>> Hi David, Chris, > > > >>> > > > >>> would it be an option to unconditionally print the additional information? > > > >> Regardless which way this is implemented it will be rather complicated and it > > > >> only switches on/off a few field in the thread dump. > > > >> > > > >> I'm not convinced this is all suitable information for a thread stack > > > >> dump. I can see the time information being useful if using the dump to > > > >> try and diagnoze a hang or responsiveness issue. But the allocated-bytes > > > >> and classes-defined just doesn't seem useful in the context of a thread > > > >> dump to me. Anyone interested in allocation stats is going to be looking > > > >> at GC logs etc which is where this belongs. Ditto the class-stats belong > > > >> in some kind of classloading logging IMHO. > > > >> > > > >> I'm very reluctant to burden the implementation with capturing these > > > >> kinds of stats, just to use for diagnostic purposes that may not even be > > > >> asked for. I'm very much in the "you shouldn't pay for what you don't > > > >> use" camp in that regard. (See my comments in JDK-8143176 referenced by > > > >> Sean.) > > > >> > > > >> The ThreadStatisticInfo adds overhead to every thread object - and I'd > > > >> have to suspect there could be overlap with whatever flight recorder > > > >> sticks in there too. (Thread has become even more bloated in recent time!). > > > >> > > > >> Cheers, > > > >> David > > > >> > > > >> > > > >>> Thanks, > > > >>> Gunter > > > >>> > > > >>> On 04.06.18, 01:13, "David Holmes" wrote: > > > >>> > > > >>> Hi Gunter, Chris, > > > >>> > > > >>> Sorry just trying to catch up and this is only a partial look so far ... > > > >>> > > > >>> BTW these changes are not limited to serviceability code so adding in > > > >>> runtime team as well. > > > >>> > > > >>> On 2/06/2018 9:12 AM, Chris Plummer wrote: > > > >>> > Hi Gunter, > > > >>> > > > > >>> > On 6/1/18 3:17 AM, Haug, Gunter wrote: > > > >>> >> Hi Chris, > > > >>> >> > > > >>> >> thanks for looking into this! > > > >>> >> > > > >>> >> Re the synchronization: The value is stored only in a VM operation at > > > >>> >> a safepoint by the VM thread. I was of the opinion, that this could > > > >>> >> not be interrupted by a second VM operation (of the same type). Or > > > >> am > > > >>> >> I missing something else? > > > >>> > I was really thinking more about the temporary changing of > > > >>> > PrintExtendedThreadInfo, not the value stored in the VMOp. You may > > > >> be be > > > >>> > correct that no more than one VMOp is executing, but while it is > > > >>> > executing it is has changed the value of PrintExtendedThreadInfo, > > > >> which > > > >>> > might have an impact on anything else that triggers printing thread info > > > >>> > while the VMOp is executing. > > > >>> > > > >>> Even if nothing else can possibly be running during the safepoint I find > > > >>> it extremely bad form to change a command-line flag in this way, > > > >>> particularly from a safepoint! > > > >>> > > > >>> If this flag is intended to be dynamically enabled as part of a dcmd > > > >>> then it should also be a Manageable flag IMHO. > > > >>> > > > >>> That said ... > > > >>> > > > >>> >> I did think about passing an argument to the various print_on > > > >> member > > > >>> >> functions of the thread classes, but this would require rather > > > >>> >> extensive code changes for a rather tiny extension. Therefore I feel > > > >>> >> doing it like this is the lesser evil. > > > >>> > > > >>> ... it's obviously not truly a global setting, but one that is needed on > > > >>> a per-print-request basis. The flag is just the default, but if the > > > >>> default is off you still want to enable extended printing on a > > > >>> per-request basis. > > > >>> > > > >>> The current print_on mechanics is not set up for this kind of > > > >>> flexibility. I think this needs more thought. > > > >>> > > > >>> --- > > > >>> > > > >>> Re osThread.cpp shared code changes ... I really hate to see all the > > > >>> ifdefs in there. Please add a pd_print_on function to support this if > > > >>> you truly want platform specific information. > > > >>> > > > >>> --- > > > >>> > > > >>> threadStatisticInfo.hpp > > > >>> > > > >>> typo: getElepsedTime() -> getElapsedTime() > > > >>> > > > >>> Thanks, > > > >>> David > > > >>> > > > >>> > > > >>> > > > >>> >> > > > >>> >> Re thread_dump(): I think it's correct or, well, at least it works ;-) > > > >>> >> In fact jstack will transfer the "-e" and "-l" in only one string, > > > >>> >> i.e. op->arg(0). > > > >>> > So you are saying that op->arg(0) is "-e -l" when using jstack? I think > > > >>> > you really need to clean up the parsing. As it stands right now passing, > > > >>> > I get the feeling that if a user erroneously asks for help by using > > > >>> > "jcmd Thread.Print -help", it will end up executing with -e an -l > > > >>> > enabled, and no failure for the invalid "-help" option. > > > >>> >> Christoph has already explained my reasoning. But I agree, it's not > > > >>> >> nice and I would be happy to do it like Christoph suggested. > > > >>> > I'll respond separately to his suggestion. > > > >>> > > > > >>> > thanks, > > > >>> > > > > >>> > Chris > > > >>> >> > > > >>> >> And typo fixed, sorry. > > > >>> >> > > > >>> >> Thanks again, > > > >>> >> Gunter > > > >>> >> > > > >>> >> On 01.06.18, 00:03, "Chris Plummer" > > > >> wrote: > > > >>> >> > > > >>> >> Hi Gunter, > > > >>> >> globals.hpp: fix typo "informatiuon" > > > >>> >> I worry a little bit about the synchronizing (if that's the right > > > >>> >> word) > > > >>> >> of PrintExtendedThreadInfo and the dcmd's -e flag. When using -e, > > > >>> >> you > > > >>> >> are temporarily enabling PrintExtendedThreadInfo if it was false. > > > >>> >> This > > > >>> >> temporarily changes the behavior of thread dumps, and could > > > >>> >> impact other > > > >>> >> uses that happen in parallel. Also, could two simultaneous uses > > > >>> >> of -e > > > >>> >> result in PrintExtendedThreadInfo not getting restored properly? > > > >>> >> thread_dump() doesn't look right. It looks like you are iterating > > > >>> >> char > > > >>> >> by char over the argument, and expect something like "-el" to be > > > >>> >> specified rather then "-e -l". The loop should be iterating over > > > >>> >> op->arg(i), not op->arg(0)[i]. > > > >>> >> The rest of the changes look fine. > > > >>> >> thanks, > > > >>> >> Chris > > > >>> >> On 5/30/18 8:12 AM, Haug, Gunter wrote: > > > >>> >> > Hi all, > > > >>> >> > > > > >>> >> > As Chris proposed, I have made an the extended output > > > >>> >> switchable. There is an VM flag (PrintExtendedThreadInfo), which is > > > >>> >> false by default. Moreover, there is an Option (-e) which can be used > > > >>> >> with jcmd Thread.print as well as with jstack. The -e option > > > >>> >> essentially sets PrintExtendedThreadInfo true just for the respective > > > >>> >> thread dump. > > > >>> >> > > > > >>> >> > Here is the updated webrev: > > > >>> >> > > > > >>> >> > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v2 > > > >>> >> > > > > >>> >> > (https://bugs.openjdk.java.net/browse/JDK-8200720) > > > >>> >> > > > > >>> >> > Thanks, > > > >>> >> > Gunter > > > >>> >> > > > > >>> >> > > > > >>> >> > On 02.05.18, 17:07, "serviceability-dev on behalf of Haug, > > > >>> >> Gunter" > > >>> >> gunter.haug at sap.com> wrote: > > > >>> >> > > > > >>> >> > Hi Chris, > > > >>> >> > > > > >>> >> > Thanks for looking into this. > > > >>> >> > You're right, there is a little more we have. We have > > > >>> >> implemented an IO tracing mechanism which - rather as a byproduct - > > > >>> >> keeps track of bytes read and written per thread. However, this of > > > >>> >> course requires changes not only in hotspot. We would be happy to > > > >>> >> contribute this as well, but this is a far bigger change and will > > > >>> >> probably lead to a far bigger discussion. Anyway, with the number of > > > >>> >> bytes read, the number of classes defined doesn't look that arbitrary > > > >>> >> anymore, as one can correlate IO to class loading. > > > >>> >> > > > > >>> >> > Regarding the verbose option I think that's a good idea! > > > >>> >> > > > > >>> >> > Thanks again, > > > >>> >> > Gunter > > > >>> >> > > > > >>> >> > On 01.05.18, 22:55, "Chris Plummer" > > > >>> >> wrote: > > > >>> >> > > > > >>> >> > Hi Gunter, > > > >>> >> > > > > >>> >> > The output you are adding is all useful. I think the > > > >>> >> question is (and > > > >>> >> > I'd like to see a few people chime in on this for this > > > >>> >> review) is > > > >>> >> > whether or not all of it is the appropriate for a > > > >>> >> thread's stack dump. > > > >>> >> > For example, defined_classes is on the fringe of what > > > >>> >> I would call > > > >>> >> > generally useful info in a stack dump. Sure, there > > > >>> >> might be that rare > > > >>> >> > case when you need it, but how often compared to other > > > >>> >> useful info > > > >>> >> > maintained on a per thread basis. How many other bits > > > >>> >> of useful info are > > > >>> >> > not being printed in favor of defined_classes? It > > > >>> >> seems you have more in > > > >>> >> > the queue. How many? I'm worried about how cluttered > > > >>> >> the stack dumps > > > >>> >> > will get. Maybe we should add some sort of verbose > > > >>> >> thread dumping > > > >>> >> > option. Just a thought. > > > >>> >> > > > > >>> >> > As for the implementation, overall it looks good, but > > > >>> >> I can't speak to > > > >>> >> > whether or not you are doing proper accounting of > > > >>> >> defined_classes and > > > >>> >> > bytes allocated. You'll need input from someone with > > > >>> >> more knowledge of > > > >>> >> > those areas. We'll also need to do some testing to > > > >>> >> make sure tool tests > > > >>> >> > are not impacted. > > > >>> >> > > > > >>> >> > thanks, > > > >>> >> > > > > >>> >> > Chris > > > >>> >> > > > > >>> >> > On 4/30/18 2:51 AM, Haug, Gunter wrote: > > > >>> >> > > Hi, > > > >>> >> > > > > > >>> >> > > this is an update to an RFR I posted on hotspot-dev, > > > >>> >> but it is probably more suitable to post it here. Can I please have a > > > >>> >> review and a sponsor for the following enhancement: > > > >>> >> > > > > > >>> >> > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v1 > > > >>> >> > > https://bugs.openjdk.java.net/browse/JDK-8200720 > > > >>> >> > > > > > >>> >> > > We at SAP have extended the thread dumps (obtained > > > >>> >> by jstack or jcmd) by several fields providing thread specific > > > >>> >> information. These extensions are quite popular with our support > > > >> team. > > > >>> >> With some knowledge of the architecture of the application, they > > > >> often > > > >>> >> allow for quick and simple diagnosis of a running system. Therefore > > > >> we > > > >>> >> would like to contribute these enhancements. > > > >>> >> > > > > > >>> >> > > I took a few simple examples here, namely cpu time, > > > >>> >> elapsed time since thread creation, bytes allocated and classes > > > >>> >> defined by the thread and the pthread-id or equivalent on platforms > > > >>> >> where it makes sense. Provided it is known how the application > > > >> should > > > >>> >> behave, a misbehaving thread can identified easily. > > > >>> >> > > > > > >>> >> > > There is no measurable overhead for this > > > >>> >> enhancement. However, it may be a problem that the format of the > > > >>> >> output is changed. Tools parsing the output may have to be changed. > > > >>> >> > > > > > >>> >> > > Here is an example of the output generated: > > > >>> >> > > > > > >>> >> > > ------------------------------------------------------ > > > >>> >> > > > > > >>> >> > > "main" #1 prio=5 os_prio=31 cpu=6300.65ms > > > >>> >> elapsed=123.28s allocated=242236760B defined_classes=1725 > > > >>> >> tid=0x00007fa13a806000 nid=0x1c03 pthread-id=0x109708000 waiting > > > >> on > > > >>> >> condition [0x0000000109707000] > > > >>> >> > > java.lang.Thread.State: TIMED_WAITING (sleeping) > > > >>> >> > > JavaThread state: _thread_blocked > > > >>> >> > > Thread: 0x00007fa13a806000 [0x1c03] State: > > > >>> >> _at_safepoint _has_called_back 0 _at_poll_safepoint 0 > > > >>> >> > > JavaThread state: _thread_blocked > > > >>> >> > > at java.lang.Thread.sleep(java.base/Native Method) > > > >>> >> > > ... > > > >>> >> > > ------------------------------------------------------ > > > >>> >> > > > > > >>> >> > > As mentioned above, we have a whole bunch of other > > > >>> >> enhancements to the thread dump similar to this one and would be > > > >>> >> willing to contribute them if there is any interest. > > > >>> >> > > > > > >>> >> > > Thanks and best regards, > > > >>> >> > > Gunter > > > >>> >> > > > > > >>> >> > > > > > >>> >> > > > > >>> >> > > > > >>> >> > > > > >>> >> > > > > >>> >> > > > > >>> >> > > > >>> > > > > >>> > > > > >>> > > > >>> > > > > > > > > > > > > > > > > From erik.joelsson at oracle.com Mon Jun 25 15:53:39 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Mon, 25 Jun 2018 08:53:39 -0700 Subject: RFR: JDK-8189429: SA: MacOSX: Replace the deprecated PT_ATTACH with PT_ATTACHEXC In-Reply-To: <0c93044e-7aaf-179b-c80c-45ff4d7e2712@oracle.com> References: <7abea33e-7fb0-fbe0-d463-8999ccdf54da@oracle.com> <0c93044e-7aaf-179b-c80c-45ff4d7e2712@oracle.com> Message-ID: <30d34a04-1611-f6f5-6d9b-548cb6bc9b45@oracle.com> Build changes look good. /Erik On 2018-06-24 10:37, Jini George wrote: > Hi Erik, > > Thank you very much for looking into this. I have addressed your > comments. The latest webrev is at: > > http://cr.openjdk.java.net/~jgeorge/8189429/webrev.06/ > > Thank you, > Jini. > > > On 6/23/2018 3:31 AM, Erik Joelsson wrote: >> Hello Jini, >> >> In general this looks pretty good, but it's also breaking some new >> ground as it's adding generation of native source in the java gensrc >> step. Mixing native code with the java code that the genrcs targets >> and gensrc output directories are meant for seems ok for now, but may >> cause trouble in the future. I'm going to accept it for now though. >> >> In Gensrc-jdk.hotspot.agent.gmk: Please put the new macosx stuff in >> its own section (as delimited by the 80x # lines) and put that whole >> thing inside a conditional for macosx. Also please break line 47 (for >> recipe lines, indent with tab and 4 additional spaces for >> continuation [1]). >> >> In Lib-jdk.hotspot.agent.gmk: I believe adding extra src should be >> enough as that will implicitly add the same directories as header >> dirs by default. At least that's the intention. >> >> /Erik >> >> [1] http://openjdk.java.net/groups/build/doc/code-conventions.html >> >> On 2018-06-22 11:11, Jini George wrote: >>> Hi all, >>> >>> [Including build-dev also since this includes build related changes]. >>> >>> Requesting reviews for: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8189429 (SA: MacOSX: >>> Replace the deprecated PT_ATTACH with PT_ATTACHEXC) >>> >>> Webrev: http://cr.openjdk.java.net/~jgeorge/8189429/webrev.04/ >>> >>> This is the follow-up solution for the temporary restoration of >>> PT_ATTACH to fix https://bugs.openjdk.java.net/browse/JDK-8184042 >>> (several serviceability/sa tests timed out on MacOS X), which was >>> done in Oct 2017. The mails related to this are at: >>> >>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2017-August/021702.html >>> >>> >>> Changes as compared to the patch sent last year >>> (http://cr.openjdk.java.net/~jgeorge/8184042/webrev.00/): >>> >>> * Addressed the review comments which were provided by Poonam, Dan, >>> Dmitry. >>> * A major change as compared to what was done last year is that the >>> MIG generated files have been included as a part of the JDK build >>> process. >>> * The test case which was provided in the patch last year is no >>> longer required since we have ClhsdbAttach.java testing the same >>> functionality as a part of the SA testsuite now. >>> * Other than that, some minor improvements have been done wrt error >>> handling. >>> >>> The steps for the proposed solution have been provided in JBS. >>> >>> Testing: ALL the SA tests pass on MacOSX and the other Mach5 platforms. >>> >>> Thanks to Sharath, Robin, Gerard and Dan for looking into the >>> changes and providing valuable comments. >>> >>> Thanks, >>> Jini. >> From jini.george at oracle.com Mon Jun 25 16:19:49 2018 From: jini.george at oracle.com (Jini George) Date: Mon, 25 Jun 2018 21:49:49 +0530 Subject: RFR(S/M): 8205419: [testbug] TestJmapCore failing without SA: introduce @requires vm.hasSA In-Reply-To: <486ee0f785144f55bd739cc41bcd7e2d@sap.com> References: <112db96d59084bdfafba2174e121c409@sap.com> <431269fc-7174-3051-0a86-baa00032754a@oracle.com> <3712cc995426467fa67b65493d556298@sap.com> <486ee0f785144f55bd739cc41bcd7e2d@sap.com> Message-ID: Hi Goetz, Your changes look good, though I feel that even for the tests that use ClhsdbLauncher and which need an attach, an "@requires vm.hasSAandCanAttach" could be used, and the Platform.shouldSAAttach() calls can be removed from ClhsdbLauncher. Some nits: 1. In VMProps.java, servicability ==> serviceability (an extra 'e') 2. In Platform.java reqires ==> requires Thanks, Jini (Not a (R)eviewer). On 6/22/2018 1:50 PM, Lindenmaier, Goetz wrote: > Hi, > > Ok, so I now added > vm.hasSA and > vm.hasSAandCanAttach > I use the first in the JMapCore tests, and in the tests that don't check shouldSAAttach() > right at the beginning. > Wherever the test is immediately ended after checking shouldSAAttach(), I > removed shouldSAAttach(), and added vm.hasSAandCanAttach. The others > depend on ClhsdbLauncher. Just checking vm.hasSA is on the safe side > if runOnCore() is changed. > http://cr.openjdk.java.net/~goetz/wr18/8205419-requiresHasSA/02/ > > This also makes the implementation of VMProps and VMPlatform more clear > as requested by Chris. > > Best regards, > Goetz. > > >> -----Original Message----- >> From: Jini George >> Sent: Thursday, June 21, 2018 12:18 PM >> To: Lindenmaier, Goetz ; 'Chris Plummer' >> ; serviceability-dev (serviceability- >> dev at openjdk.java.net) >> Subject: Re: RFR(S/M): 8205419: [testbug] TestJmapCore failing without SA: >> introduce @requires vm.hasSA >> >> Hi Goetz, >> >> Tests like TestJmapCore which involve only corefile debugging by SA (and >> not attaching to a live process) would only need to check for the >> presence of SA, and would not need to check for ptrace related attach >> permissions like what is done for Linux and OSX -- since super user >> privileges are not required to debug corefiles. So a single @requires >> catering to both these might not be desirable. >> >> Having said that, I realized that runOnCore() of ClhsdbLauncher >> incorrectly checks for the ptrace related attach permissions. (Will file >> an issue to correct this). >> >> An @requires instead of a shouldSAAttach() might help in avoiding fake >> passes like those on OSX when not run as "root". >> (https://bugs.openjdk.java.net/browse/JDK-8199700) >> >> Thanks, >> Jini. >> >> >> >> >> On 6/21/2018 1:52 AM, Lindenmaier, Goetz wrote: >>> Hi Chris, >>> >>> Thanks for looking at this change. >>> >>>> Can't VMProps.vmHasSA() just return Platform.shouldSAAttach()? It >> seems >>>> you are unnecessarily replicating Platform.shouldSAAttach() logic. >>> I think there are two things to distinguish: >>> - platforms that don't implement SA >>> - systems/setups where SA does not work. >>> If both can be recognized when VMProps is evaluated, shouldSAAttach() >>> is the one that is superfluous in the long run?! If not, shouldSAAttach >>> should only return those where it is necessary to check in the test. >>> But this is probably too much brains in this case ?? >>> >>>> If you are adding "@requires vm.hasSAandCanAttach" to a test, shouldn't >>>> you remove the test's Platform.shouldSAAttach() check? >>> Well I asked in my mail whether I should do that. So I take this as a yes. >>> But it depends on whether it must be evaluated in the test. >>> >>>> Is there a reason not to take the more direct and established approach >>>> of just adding the Platform.shouldSAAttach() check to TestJmapCore? It >>>> would be a lot less disruptive.? I know the vm.hasSAandCanAttach >>>> approach has advantages in not even attempting to compile and run the >>>> test, but I'm not so sure those advantages outweigh the simplicity of >>>> just adding a Platform.shouldSAAttach() check to one test. I can go >>>> either way here. Just wondering if there are additional reasons I might >>>> not be seeing. >>> I like the @requires much more. It tells right where the test is described >>> that it does not run with SA. The other is quite hidden in the test, e.g., in >>> helper class ClhsdbLauncher.java. >>> Also, it's more easy to remember (The implementor of >>> TestJmapCore forgot it...). And, last but not least, it seems best practice >>> nowadays. There are lots of excludes for mac, they are also done by >> @requires >>> and not as a check using Platform at runtime. >>> >>>> Sorry, I don't have an answer to your VMProps evaluation question. You >>>> might need to ask a wider audience than just svc. >>> Whom should I ask? Hotspot-dev? >>> >>> Best regards, >>> Goetz. >>> >>> >>>> >>>> thanks, >>>> >>>> Chris >>>> >>>> On 6/20/18 6:49 AM, Lindenmaier, Goetz wrote: >>>>> Hi, >>>>> >>>>> TestJmapCore is failing on aix because there jhsdb is not implemented. >>>>> So far, such tests check at runtime Platform.shouldSAAttach() which is >>>> missing >>>>> in TestJmapCore. >>>>> >>>>> Instead, I implement @requires vm.hasSAandCanAttach. >>>>> This makes jtreg skip the tests without compiling and starting them. >>>>> >>>>> I added the new property to all tests that check shouldSAAttach in jdk >> and >>>> hostpot test >>>>> directories. >>>>> This is the implementation of the new property, the rest is just >> repetitive >>>> checking it: >>>>> http://cr.openjdk.java.net/~goetz/wr18/8205419- >>>> requiresHasSA/01/test/jtreg-ext/requires/VMProps.java.udiff.html >>>>> webrev: >>>>> http://cr.openjdk.java.net/~goetz/wr18/8205419-requiresHasSA/01/ >>>>> >>>>> Is it correct to assume the VMProps is evaluated on the same machine >> and >>>> with the same >>>>> user as used for executing the test? canPtraceAttachLinux directly >>>> accesses the system. >>>>> >>>>> (Should I remove the checks for shouldSAAttach() from the changed >> tests?) >>>>> >>>>> Best regards, >>>>> Goetz. >>>> >>>> >>> From jini.george at oracle.com Mon Jun 25 16:21:47 2018 From: jini.george at oracle.com (Jini George) Date: Mon, 25 Jun 2018 21:51:47 +0530 Subject: RFR: JDK-8189429: SA: MacOSX: Replace the deprecated PT_ATTACH with PT_ATTACHEXC In-Reply-To: <30d34a04-1611-f6f5-6d9b-548cb6bc9b45@oracle.com> References: <7abea33e-7fb0-fbe0-d463-8999ccdf54da@oracle.com> <0c93044e-7aaf-179b-c80c-45ff4d7e2712@oracle.com> <30d34a04-1611-f6f5-6d9b-548cb6bc9b45@oracle.com> Message-ID: Thank you, Erik ! - Jini. On 6/25/2018 9:23 PM, Erik Joelsson wrote: > Build changes look good. > > /Erik > > > On 2018-06-24 10:37, Jini George wrote: >> Hi Erik, >> >> Thank you very much for looking into this. I have addressed your >> comments. The latest webrev is at: >> >> http://cr.openjdk.java.net/~jgeorge/8189429/webrev.06/ >> >> Thank you, >> Jini. >> >> >> On 6/23/2018 3:31 AM, Erik Joelsson wrote: >>> Hello Jini, >>> >>> In general this looks pretty good, but it's also breaking some new >>> ground as it's adding generation of native source in the java gensrc >>> step. Mixing native code with the java code that the genrcs targets >>> and gensrc output directories are meant for seems ok for now, but may >>> cause trouble in the future. I'm going to accept it for now though. >>> >>> In Gensrc-jdk.hotspot.agent.gmk: Please put the new macosx stuff in >>> its own section (as delimited by the 80x # lines) and put that whole >>> thing inside a conditional for macosx. Also please break line 47 (for >>> recipe lines, indent with tab and 4 additional spaces for >>> continuation [1]). >>> >>> In Lib-jdk.hotspot.agent.gmk: I believe adding extra src should be >>> enough as that will implicitly add the same directories as header >>> dirs by default. At least that's the intention. >>> >>> /Erik >>> >>> [1] http://openjdk.java.net/groups/build/doc/code-conventions.html >>> >>> On 2018-06-22 11:11, Jini George wrote: >>>> Hi all, >>>> >>>> [Including build-dev also since this includes build related changes]. >>>> >>>> Requesting reviews for: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8189429 (SA: MacOSX: >>>> Replace the deprecated PT_ATTACH with PT_ATTACHEXC) >>>> >>>> Webrev: http://cr.openjdk.java.net/~jgeorge/8189429/webrev.04/ >>>> >>>> This is the follow-up solution for the temporary restoration of >>>> PT_ATTACH to fix https://bugs.openjdk.java.net/browse/JDK-8184042 >>>> (several serviceability/sa tests timed out on MacOS X), which was >>>> done in Oct 2017. The mails related to this are at: >>>> >>>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2017-August/021702.html >>>> >>>> >>>> Changes as compared to the patch sent last year >>>> (http://cr.openjdk.java.net/~jgeorge/8184042/webrev.00/): >>>> >>>> * Addressed the review comments which were provided by Poonam, Dan, >>>> Dmitry. >>>> * A major change as compared to what was done last year is that the >>>> MIG generated files have been included as a part of the JDK build >>>> process. >>>> * The test case which was provided in the patch last year is no >>>> longer required since we have ClhsdbAttach.java testing the same >>>> functionality as a part of the SA testsuite now. >>>> * Other than that, some minor improvements have been done wrt error >>>> handling. >>>> >>>> The steps for the proposed solution have been provided in JBS. >>>> >>>> Testing: ALL the SA tests pass on MacOSX and the other Mach5 platforms. >>>> >>>> Thanks to Sharath, Robin, Gerard and Dan for looking into the >>>> changes and providing valuable comments. >>>> >>>> Thanks, >>>> Jini. >>> > From chris.plummer at oracle.com Mon Jun 25 17:01:46 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 25 Jun 2018 10:01:46 -0700 Subject: RFR(S/M): 8205419: [testbug] TestJmapCore failing without SA: introduce @requires vm.hasSA In-Reply-To: <486ee0f785144f55bd739cc41bcd7e2d@sap.com> References: <112db96d59084bdfafba2174e121c409@sap.com> <431269fc-7174-3051-0a86-baa00032754a@oracle.com> <3712cc995426467fa67b65493d556298@sap.com> <486ee0f785144f55bd739cc41bcd7e2d@sap.com> Message-ID: Hi Goetz, The changes look good. Just one question though. In places where you have: ? 42? * @requires vm.hasSAandCanAttach & os.family != "mac" Do you know why these tests don't run on osx? I just wonder if it related to attach not working unless root, and that is now covered by hasSAandCanAttach(). If that is the case, the "mac" check can be removed. thanks, Chris On 6/22/18 1:20 AM, Lindenmaier, Goetz wrote: > Hi, > > Ok, so I now added > vm.hasSA and > vm.hasSAandCanAttach > I use the first in the JMapCore tests, and in the tests that don't check shouldSAAttach() > right at the beginning. > Wherever the test is immediately ended after checking shouldSAAttach(), I > removed shouldSAAttach(), and added vm.hasSAandCanAttach. The others > depend on ClhsdbLauncher. Just checking vm.hasSA is on the safe side > if runOnCore() is changed. > http://cr.openjdk.java.net/~goetz/wr18/8205419-requiresHasSA/02/ > > This also makes the implementation of VMProps and VMPlatform more clear > as requested by Chris. > > Best regards, > Goetz. > > >> -----Original Message----- >> From: Jini George >> Sent: Thursday, June 21, 2018 12:18 PM >> To: Lindenmaier, Goetz ; 'Chris Plummer' >> ; serviceability-dev (serviceability- >> dev at openjdk.java.net) >> Subject: Re: RFR(S/M): 8205419: [testbug] TestJmapCore failing without SA: >> introduce @requires vm.hasSA >> >> Hi Goetz, >> >> Tests like TestJmapCore which involve only corefile debugging by SA (and >> not attaching to a live process) would only need to check for the >> presence of SA, and would not need to check for ptrace related attach >> permissions like what is done for Linux and OSX -- since super user >> privileges are not required to debug corefiles. So a single @requires >> catering to both these might not be desirable. >> >> Having said that, I realized that runOnCore() of ClhsdbLauncher >> incorrectly checks for the ptrace related attach permissions. (Will file >> an issue to correct this). >> >> An @requires instead of a shouldSAAttach() might help in avoiding fake >> passes like those on OSX when not run as "root". >> (https://bugs.openjdk.java.net/browse/JDK-8199700) >> >> Thanks, >> Jini. >> >> >> >> >> On 6/21/2018 1:52 AM, Lindenmaier, Goetz wrote: >>> Hi Chris, >>> >>> Thanks for looking at this change. >>> >>>> Can't VMProps.vmHasSA() just return Platform.shouldSAAttach()? It >> seems >>>> you are unnecessarily replicating Platform.shouldSAAttach() logic. >>> I think there are two things to distinguish: >>> - platforms that don't implement SA >>> - systems/setups where SA does not work. >>> If both can be recognized when VMProps is evaluated, shouldSAAttach() >>> is the one that is superfluous in the long run?! If not, shouldSAAttach >>> should only return those where it is necessary to check in the test. >>> But this is probably too much brains in this case ?? >>> >>>> If you are adding "@requires vm.hasSAandCanAttach" to a test, shouldn't >>>> you remove the test's Platform.shouldSAAttach() check? >>> Well I asked in my mail whether I should do that. So I take this as a yes. >>> But it depends on whether it must be evaluated in the test. >>> >>>> Is there a reason not to take the more direct and established approach >>>> of just adding the Platform.shouldSAAttach() check to TestJmapCore? It >>>> would be a lot less disruptive.? I know the vm.hasSAandCanAttach >>>> approach has advantages in not even attempting to compile and run the >>>> test, but I'm not so sure those advantages outweigh the simplicity of >>>> just adding a Platform.shouldSAAttach() check to one test. I can go >>>> either way here. Just wondering if there are additional reasons I might >>>> not be seeing. >>> I like the @requires much more. It tells right where the test is described >>> that it does not run with SA. The other is quite hidden in the test, e.g., in >>> helper class ClhsdbLauncher.java. >>> Also, it's more easy to remember (The implementor of >>> TestJmapCore forgot it...). And, last but not least, it seems best practice >>> nowadays. There are lots of excludes for mac, they are also done by >> @requires >>> and not as a check using Platform at runtime. >>> >>>> Sorry, I don't have an answer to your VMProps evaluation question. You >>>> might need to ask a wider audience than just svc. >>> Whom should I ask? Hotspot-dev? >>> >>> Best regards, >>> Goetz. >>> >>> >>>> thanks, >>>> >>>> Chris >>>> >>>> On 6/20/18 6:49 AM, Lindenmaier, Goetz wrote: >>>>> Hi, >>>>> >>>>> TestJmapCore is failing on aix because there jhsdb is not implemented. >>>>> So far, such tests check at runtime Platform.shouldSAAttach() which is >>>> missing >>>>> in TestJmapCore. >>>>> >>>>> Instead, I implement @requires vm.hasSAandCanAttach. >>>>> This makes jtreg skip the tests without compiling and starting them. >>>>> >>>>> I added the new property to all tests that check shouldSAAttach in jdk >> and >>>> hostpot test >>>>> directories. >>>>> This is the implementation of the new property, the rest is just >> repetitive >>>> checking it: >>>>> http://cr.openjdk.java.net/~goetz/wr18/8205419- >>>> requiresHasSA/01/test/jtreg-ext/requires/VMProps.java.udiff.html >>>>> webrev: >>>>> http://cr.openjdk.java.net/~goetz/wr18/8205419-requiresHasSA/01/ >>>>> >>>>> Is it correct to assume the VMProps is evaluated on the same machine >> and >>>> with the same >>>>> user as used for executing the test? canPtraceAttachLinux directly >>>> accesses the system. >>>>> (Should I remove the checks for shouldSAAttach() from the changed >> tests?) >>>>> Best regards, >>>>> Goetz. >>>> From chris.plummer at oracle.com Mon Jun 25 17:13:28 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 25 Jun 2018 10:13:28 -0700 Subject: RFR(S): 8200720: Print additional information in thread dump (times, allocated bytes etc.) In-Reply-To: <70FECB3F-4E90-4B2C-AB22-09B42069323A@sap.com> References: <56817AB5-88FC-4FB9-A271-7206EA69ECA7@sap.com> <2728581E-0E32-4911-B1A9-79AD6CBA81B0@sap.com> <082ca98f-9a68-15a0-95b5-c4d5c733ab36@oracle.com> <933cdacc-9b6f-c8b5-e244-558f38e7888c@oracle.com> <55806aa8-f09b-e9a8-b512-b9676029c494@oracle.com> <0F497BC6-2569-4EF6-9B59-E24A8E1B3A37@sap.com> <5dc08f65e83f4374b8e1f73f218c6d6d@sap.com> <626701f8-b71b-58dc-f837-e5e1d81ef187@oracle.com> <19BE9680-3E6E-4620-B72D-D379F39C1C92@sap.com> <6ce47417-311b-e6cd-b8cd-d46eb7e04c2f@oracle.com> <3795D1AE-CBF4-4BC3-B707-ED77D4DE00D0@sap.com> <9f5a404a-24e6-9131-3c0d-80c5c01ccfca@oracle.com> <70FECB3F-4E90-4B2C-AB22-09B42069323A@sap.com> Message-ID: <00c57078-ad4a-b54b-e8c6-eb1a2502c88a@oracle.com> Hi Gunter, Changes look good to me. thanks, Chris On 6/25/18 7:22 AM, Haug, Gunter wrote: > Hi all, > > Thank you, David for taking the time to do the review. Here is hopefully the last version of the change: > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v5/ > > Following David's suggestion, _start_time_stamp is now constant and set by the constructor and the setter has been removed. This is the only difference to the previous version. > > Chris and Christoph, are you still OK with this? > > Thanks, > Gunter > > ?On 25.06.18, 12:29, "David Holmes" wrote: > > Hi Gunter, > > I see what you mean about the initialization point of the statistics. > Personally I'd prefer to see the startTime as read-only and only set by > the constructor rather than having a general-purpose setter function for it. > > Thanks, > David > > On 21/06/2018 1:28 AM, Haug, Gunter wrote: > > Hi David, > > > > Thanks for taking the time to look into this! > > > >> "statistic info" is not very good grammatically. These things are either > >> statistics, or statistical information. So e.g. > >> > >> class ThreadStatistics > >> ... > > > > OK, I'll change that! > > > >> const_cast(this)->cooked_allocated_bytes(); > >> > >> Why do we need a const cast to invoke a method on ourselves ?? > > > > cooked_allocated_bytes() is not declared const and we can't make it, as it uses OrderAccess::load_acquire(&_allocated_bytes). Therefor the const_cast. > > > >> src/hotspot/share/runtime/thread.hpp > >> > >> Please put the new #include in alphabetical order. > > > > Wilco! > > > >> I was expecting to see a default parameter used here rather than adding > >> an overload: > >> virtual void print_on(outputStream* st, bool print_extended_info = > >> false) const { print_on(st, print_extended_info); } > > > > Thread inherits from AllocatedObj (in dbg, at least) where virtual void AllocatedObj::print_on(outputStream*) const is declared. Some compilers, e.g. gcc, will complain that this is hidden by Thread::print_on(outputStream*, bool). Others, e.g. clang, are happy with that. We could implement a method with a different name but would that be nicer? > > > >> 2179 static void print_on(outputStream* st, bool print_stacks, bool > >> internal_format, bool print_concurrent_locks, bool extended_thread_info); > >> > >> Again I expected to see a default parameter here - but I didn't check if > >> all callers themselves take the new parameter. ?? > > > > This one has just one single caller, so I think it's OK. > > > >> src/hotspot/share/runtime/vm_operations.hpp > >> ... > > > > I'll change it to conform to the guidelines. > > > >> src/hotspot/share/runtime/threadStatisticInfo.hpp > >> > >> There are no includes in this file. It should include the appropriate > >> std header for type definitions, and os.hpp. > > > > Agreed, I'll add the includes. > > > > > >> I'm not sure if class ThreadStatisticInfo needs an allocation type as a > >> super type. > >> > >> Might be worth adding a constructor to give a default value so that you > >> can tell if the statistics have been initialized when they appear in the > >> printout. ? > > > > ThreadStatisticInfo is a data member of Thread. Currently it is initialized in Thread::Thread, but I can add a constructor, if you prefer. > > > > Thanks again, > > Gunter > > > > > > > > On 20.06.18, 09:15, "David Holmes" wrote: > > > > Hi Gunter, > > > > On 19/06/2018 8:51 PM, Haug, Gunter wrote: > > > Hi all, > > > > > > Thanks Chris and Christoph for the reviews! Christoph, I'll incorporate the little improvement you suggested. > > > David, are you OK with the change as well? > > > > Sorry I'd lost track of this one a bit ... > > > > The overall approach now seems okay. > > > > Some naming/terminology issues: > > > > "statistic info" is not very good grammatically. These things are either > > statistics, or statistical information. So e.g. > > > > class ThreadStatistics > > > > ThreadStatistics& statistics() { return _statistics; } > > > > etc. > > > > "extended_thread_info" should really be print_extended_info (similar to > > print_concurrent_locks). You don't need "thread" in there when this is > > always part of a Thread related API. > > > > > > src/hotspot/share/runtime/thread.cpp > > > > const_cast(this)-> > > > > Why do we need a const cast to invoke a method on ourselves ?? > > > > --- > > > > src/hotspot/share/runtime/thread.hpp > > > > Please put the new #include in alphabetical order. > > > > 641 // Printing > > 642 void print_on(outputStream* st, bool extended_thread_info) const; > > 643 virtual void print_on(outputStream* st) const { print_on(st, > > false); } > > > > I was expecting to see a default parameter used here rather than adding > > an overload: > > > > virtual void print_on(outputStream* st, bool print_extended_info = > > false) const { print_on(st, print_extended_info); } > > > > > > 2179 static void print_on(outputStream* st, bool print_stacks, bool > > internal_format, bool print_concurrent_locks, bool extended_thread_info); > > > > Again I expected to see a default parameter here - but I didn't check if > > all callers themselves take the new parameter. ?? > > > > --- > > > > src/hotspot/share/runtime/vm_operations.hpp > > > > 376 VM_PrintThreads() > > 377 { _out = tty; _print_concurrent_locks = PrintConcurrentLocks; ; > > _extended_thread_info = false; } > > 378 VM_PrintThreads(outputStream* out, bool print_concurrent_locks, > > bool extended_thread_info) > > 379 { _out = out; _print_concurrent_locks = print_concurrent_locks; > > _extended_thread_info = extended_thread_info; } > > 380 VMOp_Type type() const > > 381 { return VMOp_PrintThreads; } > > > > Style nits: either keep everything on one line as before (though I agree > > the lines are now too long) or else the style should be: > > > > 380 VMOp_Type type() const { > > 381 return VMOp_PrintThreads; > > 382 } > > > > Also unclear why (existing) VM_PrintThreads constructor doesn't use > > initializer list (like VM_PrintMetadata below it) or default parameters? > > > > --- > > > > src/hotspot/share/runtime/threadStatisticInfo.hpp > > > > There are no includes in this file. It should include the appropriate > > std header for type definitions, and os.hpp. > > > > I'm not sure if class ThreadStatisticInfo needs an allocation type as a > > super type. > > > > Might be worth adding a constructor to give a default value so that you > > can tell if the statistics have been initialized when they appear in the > > printout. ? > > > > --- > > > > Thanks, > > David > > > > > > > Thanks again, > > > Gunter > > > > > > On 12.06.18, 01:13, "Chris Plummer" wrote: > > > > > > Hi Gunter, > > > > > > The changes look fine. I can live with the options parsing in > > > attachListener.cpp. As you point out, unrecognized options were already > > > silently ignored. > > > > > > thanks, > > > > > > Chris > > > > > > On 6/8/18 7:05 AM, Haug, Gunter wrote: > > > > Hi all, > > > > > > > > thanks a lot for all the input! I have prepared a new version of the webrev incorporating the suggestions you made (at least I tried): > > > > > > > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v3/ > > > > > > > > This version outputs the thread times unconditionally while the other information is guarded by a flags. I think, most participants found the thread times the most valuable information and had no (strong) objections to adding them unconditionally. > > > > > > > > @David > > > > Implementation is much simpler with conditional output only for JavaThreads. I could get rid of the terrible hack (changing the flag) without having to change too many source files. > > > > > > > > The information on allocated bytes is present in the Thread class already before this proposed change, I just print it. It might be sensible to move _allocated_bytes and the respective methods to the ThreadStatisticInfo class as G?tz suggested. I haven?t done that in the current version, though. > > > > > > > > @David and Thomas > > > > I've removed the pthread-id output. I'm unsure myself what it could be good for. Anyway, we could add it with a separate change (with a better implementation) if there is a need to. > > > > > > > > @Chris > > > > As you have written, the user has no direct contact to the attach listener of the VM and jstack won't misinterpret e.g. -help. I agree that the parsing in attachListener.cpp could be more robust. However, it hasn't been so far either. All that is done in the current implementation is a strcmp to -l everything else will be silently ignored. I can try to make this more robust or we could say that the new output is only available via jcmd. OTH nothing bad can happen with the current version of the proposed change, so we could also leave it as it is. What would you prefer? > > > > > > > > @Kirk and Thomas > > > > Implementing a new diagnostic command is certainly well worth a thought. However, it appears to me that it's not necessary in this case. There are already flags to jstack and jcmd Thread.print, so it's not that uncommon. The amount of information that would be available by a new diagnostic command and that is now added to the thread dump is quite small and I think that is tolerable. Moreover, and most importantly, our support team is use to find the information in the thread dump and it would make things easier to us if I could leave it there. > > > > > > > > Thanks again and have a nice weekend, > > > > Gunter > > > > > > > > > > > > On 06.06.18, 04:56, "David Holmes" wrote: > > > > > > > > Hi Goetz, > > > > > > > > On 5/06/2018 11:07 PM, Lindenmaier, Goetz wrote: > > > > > Hi > > > > > > > > > > this discussion touched a lot of points so far, which seem > > > > > to lead to different conclusions. > > > > > > > > > > I think we should look at the values printed: > > > > > > > > > > 1. cpu=6300.65ms elapsed=123.28s > > > > > Overhead > > > > > cpu time: > > > > > * system calls at thread dump time > > > > > elapsed time: > > > > > * 1 system call at thread creation time > > > > > * 1 64-bit field per thread for the thread start time > > > > > * 1 system call at thread dump time > > > > > > > > > > As I understand, JDK-8143176 would have had to get the > > > > > time at each locking, which is much more time critical > > > > > than doing this during thread creation. While > > > > > > > > Correct. > > > > > > > > > the time a lock was held would be much more useful, > > > > > the ratio here, combined with knowledge about the application, > > > > > could lead to the conclusion that the thread is wrongly > > > > > blocked at much lower cost. > > > > > > > > Agreed. I see no issue with unconditionally adding this information as > > > > it should address some of the concerns of 8143176 as well. > > > > > > > > > > > > > > 2. pthread-id=0x109708000 > > > > > Overhead: > > > > > * a field access at thread dump time. Negligible I would say. > > > > > > > > I'd overlooked this addition, but it is of course the chunk of ifdef > > > > code in osThread.cpp that I objected to. The problem I have here is the > > > > convoluted mess of "thread identifiers" that we already have. We > > > > currently print: > > > > > > > > st->print("tid=" INTPTR_FORMAT " ", p2i(this)); > > > > > > > > which is just the address of the Thread/JavaThread object. Then: > > > > > > > > st->print("nid=0x%x ", thread_id()); > > > > > > > > where 'n' is supposed to represent "native thread id", which is: > > > > - linux: kernel thread id from syscall gettid > > > > - solaris: thread library identity from thr_create() or thr_self() > > > > - windows: thread id from _beginthreadex > > > > - OS X: kernel thread id from pthread_mach_thread_np(pthread_self()); > > > > - Other BSD: kernel (?) thread id from syscall thr_self or getthrid > > > > - AIX: thread library identity from pthread_create() or pthread_self() > > > > > > > > It's telling that the code Gunter added gets the thread library id on > > > > linux, but a "kernel thread id" on other platforms! Do we want to see a > > > > thread library id and a kernel thread id? IIRC nid is supposed be the id > > > > you need to see the thread in a debugger. In which case I'm unclear what > > > > role the thread id Gunter wants to add is playing? I don't think we need > > > > to see kernel ids in general, and pthread-id isn't useful for debugging > > > > is it? > > > > > > > > > > > > > > 3. allocated=242236760B defined_classes=1725 > > > > > Overhead: > > > > > * 1 64-bit field per thread. > > > > > * counter increment on class creation > > > > > > > > > > Especially defined_classes seems to be controversial. > > > > > As this only makes sense for Java threads, this could > > > > > be printed in a line of it's own in Threads::print_on_error() > > > > > (which already gets a flag to customize for jstack: > > > > > print_concurrent_locks) by calling a dedicated function in Thread. > > > > > No need for flag PrintExtendedThreadInfo thus. > > > > > > > > > > But I would propose to drop this information as it is > > > > > too controversial. > > > > > > > > Agreed. > > > > > > > > > This leaves the times and the pthread-id to be decided whether > > > > > they are worth the cost. If so, I think they should be printed > > > > > unconditional. > > > > > > > > Agreed. With some refinement of the "thread id" issue. > > > > > > > > > If a flag is required to avoid bloating the info in the default case, > > > > > I would include more infos, as os_prio and nid under that flag. > > > > > > > > Of course those things exist for good reason. The native id is supposed > > > > to be what aids in you matching what you see in our thread dumps with > > > > what you see in a debugger. Priority is less of an issue these days, but > > > > there was a time ... ;-) > > > > > > > > > As jstack is deprecated, changes to jstack could be skipped always > > > > > disabling the new printouts. > > > > > > > > > > Finally, I would propose to move _allocated_bytes into > > > > > threadStatisticInfo.hpp. > > > > > > > > Didn't that part get dropped? (You don't need it for the time info or > > > > thread id). That's a GC logging issue to me. > > > > > > > > Thanks, > > > > David > > > > > > > > > Best regards, > > > > > Goetz. > > > > > > > > > > > > > > > > > > > > > > > > >> -----Original Message----- > > > > >> From: serviceability-dev [mailto:serviceability-dev- > > > > >> bounces at openjdk.java.net] On Behalf Of David Holmes > > > > >> Sent: Dienstag, 5. Juni 2018 04:53 > > > > >> To: Haug, Gunter ; Chris Plummer > > > > >> ; serviceability-dev > > > >> dev at openjdk.java.net>; Langer, Christoph ; > > > > >> hotspot-runtime-dev at openjdk.java.net > > > > >> Subject: Re: RFR(S): 8200720: Print additional information in thread dump > > > > >> (times, allocated bytes etc.) > > > > >> > > > > >> Hi Gunter, > > > > >> > > > > >> On 5/06/2018 1:27 AM, Haug, Gunter wrote: > > > > >>> Hi David, Chris, > > > > >>> > > > > >>> would it be an option to unconditionally print the additional information? > > > > >> Regardless which way this is implemented it will be rather complicated and it > > > > >> only switches on/off a few field in the thread dump. > > > > >> > > > > >> I'm not convinced this is all suitable information for a thread stack > > > > >> dump. I can see the time information being useful if using the dump to > > > > >> try and diagnoze a hang or responsiveness issue. But the allocated-bytes > > > > >> and classes-defined just doesn't seem useful in the context of a thread > > > > >> dump to me. Anyone interested in allocation stats is going to be looking > > > > >> at GC logs etc which is where this belongs. Ditto the class-stats belong > > > > >> in some kind of classloading logging IMHO. > > > > >> > > > > >> I'm very reluctant to burden the implementation with capturing these > > > > >> kinds of stats, just to use for diagnostic purposes that may not even be > > > > >> asked for. I'm very much in the "you shouldn't pay for what you don't > > > > >> use" camp in that regard. (See my comments in JDK-8143176 referenced by > > > > >> Sean.) > > > > >> > > > > >> The ThreadStatisticInfo adds overhead to every thread object - and I'd > > > > >> have to suspect there could be overlap with whatever flight recorder > > > > >> sticks in there too. (Thread has become even more bloated in recent time!). > > > > >> > > > > >> Cheers, > > > > >> David > > > > >> > > > > >> > > > > >>> Thanks, > > > > >>> Gunter > > > > >>> > > > > >>> On 04.06.18, 01:13, "David Holmes" wrote: > > > > >>> > > > > >>> Hi Gunter, Chris, > > > > >>> > > > > >>> Sorry just trying to catch up and this is only a partial look so far ... > > > > >>> > > > > >>> BTW these changes are not limited to serviceability code so adding in > > > > >>> runtime team as well. > > > > >>> > > > > >>> On 2/06/2018 9:12 AM, Chris Plummer wrote: > > > > >>> > Hi Gunter, > > > > >>> > > > > > >>> > On 6/1/18 3:17 AM, Haug, Gunter wrote: > > > > >>> >> Hi Chris, > > > > >>> >> > > > > >>> >> thanks for looking into this! > > > > >>> >> > > > > >>> >> Re the synchronization: The value is stored only in a VM operation at > > > > >>> >> a safepoint by the VM thread. I was of the opinion, that this could > > > > >>> >> not be interrupted by a second VM operation (of the same type). Or > > > > >> am > > > > >>> >> I missing something else? > > > > >>> > I was really thinking more about the temporary changing of > > > > >>> > PrintExtendedThreadInfo, not the value stored in the VMOp. You may > > > > >> be be > > > > >>> > correct that no more than one VMOp is executing, but while it is > > > > >>> > executing it is has changed the value of PrintExtendedThreadInfo, > > > > >> which > > > > >>> > might have an impact on anything else that triggers printing thread info > > > > >>> > while the VMOp is executing. > > > > >>> > > > > >>> Even if nothing else can possibly be running during the safepoint I find > > > > >>> it extremely bad form to change a command-line flag in this way, > > > > >>> particularly from a safepoint! > > > > >>> > > > > >>> If this flag is intended to be dynamically enabled as part of a dcmd > > > > >>> then it should also be a Manageable flag IMHO. > > > > >>> > > > > >>> That said ... > > > > >>> > > > > >>> >> I did think about passing an argument to the various print_on > > > > >> member > > > > >>> >> functions of the thread classes, but this would require rather > > > > >>> >> extensive code changes for a rather tiny extension. Therefore I feel > > > > >>> >> doing it like this is the lesser evil. > > > > >>> > > > > >>> ... it's obviously not truly a global setting, but one that is needed on > > > > >>> a per-print-request basis. The flag is just the default, but if the > > > > >>> default is off you still want to enable extended printing on a > > > > >>> per-request basis. > > > > >>> > > > > >>> The current print_on mechanics is not set up for this kind of > > > > >>> flexibility. I think this needs more thought. > > > > >>> > > > > >>> --- > > > > >>> > > > > >>> Re osThread.cpp shared code changes ... I really hate to see all the > > > > >>> ifdefs in there. Please add a pd_print_on function to support this if > > > > >>> you truly want platform specific information. > > > > >>> > > > > >>> --- > > > > >>> > > > > >>> threadStatisticInfo.hpp > > > > >>> > > > > >>> typo: getElepsedTime() -> getElapsedTime() > > > > >>> > > > > >>> Thanks, > > > > >>> David > > > > >>> > > > > >>> > > > > >>> > > > > >>> >> > > > > >>> >> Re thread_dump(): I think it's correct or, well, at least it works ;-) > > > > >>> >> In fact jstack will transfer the "-e" and "-l" in only one string, > > > > >>> >> i.e. op->arg(0). > > > > >>> > So you are saying that op->arg(0) is "-e -l" when using jstack? I think > > > > >>> > you really need to clean up the parsing. As it stands right now passing, > > > > >>> > I get the feeling that if a user erroneously asks for help by using > > > > >>> > "jcmd Thread.Print -help", it will end up executing with -e an -l > > > > >>> > enabled, and no failure for the invalid "-help" option. > > > > >>> >> Christoph has already explained my reasoning. But I agree, it's not > > > > >>> >> nice and I would be happy to do it like Christoph suggested. > > > > >>> > I'll respond separately to his suggestion. > > > > >>> > > > > > >>> > thanks, > > > > >>> > > > > > >>> > Chris > > > > >>> >> > > > > >>> >> And typo fixed, sorry. > > > > >>> >> > > > > >>> >> Thanks again, > > > > >>> >> Gunter > > > > >>> >> > > > > >>> >> On 01.06.18, 00:03, "Chris Plummer" > > > > >> wrote: > > > > >>> >> > > > > >>> >> Hi Gunter, > > > > >>> >> globals.hpp: fix typo "informatiuon" > > > > >>> >> I worry a little bit about the synchronizing (if that's the right > > > > >>> >> word) > > > > >>> >> of PrintExtendedThreadInfo and the dcmd's -e flag. When using -e, > > > > >>> >> you > > > > >>> >> are temporarily enabling PrintExtendedThreadInfo if it was false. > > > > >>> >> This > > > > >>> >> temporarily changes the behavior of thread dumps, and could > > > > >>> >> impact other > > > > >>> >> uses that happen in parallel. Also, could two simultaneous uses > > > > >>> >> of -e > > > > >>> >> result in PrintExtendedThreadInfo not getting restored properly? > > > > >>> >> thread_dump() doesn't look right. It looks like you are iterating > > > > >>> >> char > > > > >>> >> by char over the argument, and expect something like "-el" to be > > > > >>> >> specified rather then "-e -l". The loop should be iterating over > > > > >>> >> op->arg(i), not op->arg(0)[i]. > > > > >>> >> The rest of the changes look fine. > > > > >>> >> thanks, > > > > >>> >> Chris > > > > >>> >> On 5/30/18 8:12 AM, Haug, Gunter wrote: > > > > >>> >> > Hi all, > > > > >>> >> > > > > > >>> >> > As Chris proposed, I have made an the extended output > > > > >>> >> switchable. There is an VM flag (PrintExtendedThreadInfo), which is > > > > >>> >> false by default. Moreover, there is an Option (-e) which can be used > > > > >>> >> with jcmd Thread.print as well as with jstack. The -e option > > > > >>> >> essentially sets PrintExtendedThreadInfo true just for the respective > > > > >>> >> thread dump. > > > > >>> >> > > > > > >>> >> > Here is the updated webrev: > > > > >>> >> > > > > > >>> >> > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v2 > > > > >>> >> > > > > > >>> >> > (https://bugs.openjdk.java.net/browse/JDK-8200720) > > > > >>> >> > > > > > >>> >> > Thanks, > > > > >>> >> > Gunter > > > > >>> >> > > > > > >>> >> > > > > > >>> >> > On 02.05.18, 17:07, "serviceability-dev on behalf of Haug, > > > > >>> >> Gunter" > > > >>> >> gunter.haug at sap.com> wrote: > > > > >>> >> > > > > > >>> >> > Hi Chris, > > > > >>> >> > > > > > >>> >> > Thanks for looking into this. > > > > >>> >> > You're right, there is a little more we have. We have > > > > >>> >> implemented an IO tracing mechanism which - rather as a byproduct - > > > > >>> >> keeps track of bytes read and written per thread. However, this of > > > > >>> >> course requires changes not only in hotspot. We would be happy to > > > > >>> >> contribute this as well, but this is a far bigger change and will > > > > >>> >> probably lead to a far bigger discussion. Anyway, with the number of > > > > >>> >> bytes read, the number of classes defined doesn't look that arbitrary > > > > >>> >> anymore, as one can correlate IO to class loading. > > > > >>> >> > > > > > >>> >> > Regarding the verbose option I think that's a good idea! > > > > >>> >> > > > > > >>> >> > Thanks again, > > > > >>> >> > Gunter > > > > >>> >> > > > > > >>> >> > On 01.05.18, 22:55, "Chris Plummer" > > > > >>> >> wrote: > > > > >>> >> > > > > > >>> >> > Hi Gunter, > > > > >>> >> > > > > > >>> >> > The output you are adding is all useful. I think the > > > > >>> >> question is (and > > > > >>> >> > I'd like to see a few people chime in on this for this > > > > >>> >> review) is > > > > >>> >> > whether or not all of it is the appropriate for a > > > > >>> >> thread's stack dump. > > > > >>> >> > For example, defined_classes is on the fringe of what > > > > >>> >> I would call > > > > >>> >> > generally useful info in a stack dump. Sure, there > > > > >>> >> might be that rare > > > > >>> >> > case when you need it, but how often compared to other > > > > >>> >> useful info > > > > >>> >> > maintained on a per thread basis. How many other bits > > > > >>> >> of useful info are > > > > >>> >> > not being printed in favor of defined_classes? It > > > > >>> >> seems you have more in > > > > >>> >> > the queue. How many? I'm worried about how cluttered > > > > >>> >> the stack dumps > > > > >>> >> > will get. Maybe we should add some sort of verbose > > > > >>> >> thread dumping > > > > >>> >> > option. Just a thought. > > > > >>> >> > > > > > >>> >> > As for the implementation, overall it looks good, but > > > > >>> >> I can't speak to > > > > >>> >> > whether or not you are doing proper accounting of > > > > >>> >> defined_classes and > > > > >>> >> > bytes allocated. You'll need input from someone with > > > > >>> >> more knowledge of > > > > >>> >> > those areas. We'll also need to do some testing to > > > > >>> >> make sure tool tests > > > > >>> >> > are not impacted. > > > > >>> >> > > > > > >>> >> > thanks, > > > > >>> >> > > > > > >>> >> > Chris > > > > >>> >> > > > > > >>> >> > On 4/30/18 2:51 AM, Haug, Gunter wrote: > > > > >>> >> > > Hi, > > > > >>> >> > > > > > > >>> >> > > this is an update to an RFR I posted on hotspot-dev, > > > > >>> >> but it is probably more suitable to post it here. Can I please have a > > > > >>> >> review and a sponsor for the following enhancement: > > > > >>> >> > > > > > > >>> >> > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v1 > > > > >>> >> > > https://bugs.openjdk.java.net/browse/JDK-8200720 > > > > >>> >> > > > > > > >>> >> > > We at SAP have extended the thread dumps (obtained > > > > >>> >> by jstack or jcmd) by several fields providing thread specific > > > > >>> >> information. These extensions are quite popular with our support > > > > >> team. > > > > >>> >> With some knowledge of the architecture of the application, they > > > > >> often > > > > >>> >> allow for quick and simple diagnosis of a running system. Therefore > > > > >> we > > > > >>> >> would like to contribute these enhancements. > > > > >>> >> > > > > > > >>> >> > > I took a few simple examples here, namely cpu time, > > > > >>> >> elapsed time since thread creation, bytes allocated and classes > > > > >>> >> defined by the thread and the pthread-id or equivalent on platforms > > > > >>> >> where it makes sense. Provided it is known how the application > > > > >> should > > > > >>> >> behave, a misbehaving thread can identified easily. > > > > >>> >> > > > > > > >>> >> > > There is no measurable overhead for this > > > > >>> >> enhancement. However, it may be a problem that the format of the > > > > >>> >> output is changed. Tools parsing the output may have to be changed. > > > > >>> >> > > > > > > >>> >> > > Here is an example of the output generated: > > > > >>> >> > > > > > > >>> >> > > ------------------------------------------------------ > > > > >>> >> > > > > > > >>> >> > > "main" #1 prio=5 os_prio=31 cpu=6300.65ms > > > > >>> >> elapsed=123.28s allocated=242236760B defined_classes=1725 > > > > >>> >> tid=0x00007fa13a806000 nid=0x1c03 pthread-id=0x109708000 waiting > > > > >> on > > > > >>> >> condition [0x0000000109707000] > > > > >>> >> > > java.lang.Thread.State: TIMED_WAITING (sleeping) > > > > >>> >> > > JavaThread state: _thread_blocked > > > > >>> >> > > Thread: 0x00007fa13a806000 [0x1c03] State: > > > > >>> >> _at_safepoint _has_called_back 0 _at_poll_safepoint 0 > > > > >>> >> > > JavaThread state: _thread_blocked > > > > >>> >> > > at java.lang.Thread.sleep(java.base/Native Method) > > > > >>> >> > > ... > > > > >>> >> > > ------------------------------------------------------ > > > > >>> >> > > > > > > >>> >> > > As mentioned above, we have a whole bunch of other > > > > >>> >> enhancements to the thread dump similar to this one and would be > > > > >>> >> willing to contribute them if there is any interest. > > > > >>> >> > > > > > > >>> >> > > Thanks and best regards, > > > > >>> >> > > Gunter > > > > >>> >> > > > > > > >>> >> > > > > > > >>> >> > > > > > >>> >> > > > > > >>> >> > > > > > >>> >> > > > > > >>> >> > > > > > >>> >> > > > > >>> > > > > > >>> > > > > > >>> > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > From gary.adams at oracle.com Mon Jun 25 17:20:41 2018 From: gary.adams at oracle.com (Gary Adams) Date: Mon, 25 Jun 2018 13:20:41 -0400 Subject: RFR: JDK-8205508: hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java fails with Prompt is not received during 300200 milliseconds. Message-ID: <5B312469.7060206@oracle.com> The exclude001 test times out on solaris sparc debug builds. Basically, this test is all about tracing method calls and introduces exclude filters to reduce the callbacks to a select set of packages. The time spent tracing/filtering method callbacks is purely a function of the number of methods that are processed. On this particularly slow target platform, more time is needed before issuing a timeout. Issue: https://bugs.openjdk.java.net/browse/JDK-8205508 Proposed fix: diff --git a/test/hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java b/test/hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java --- a/test/hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java +++ b/test/hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java @@ -63,7 +63,7 @@ * nsk.jdb.exclude.exclude001.exclude001a * @run main/othervm PropertyResolvingWrapper nsk.jdb.exclude.exclude001.exclude001 * -arch=${os.family}-${os.simpleArch} - * -waittime=5 + * -waittime=7 * -debugee.vmkind=java * -transport.address=dynamic * -jdb=${test.jdk}/bin/jdb -------------- next part -------------- An HTML attachment was scrubbed... URL: From jini.george at oracle.com Mon Jun 25 17:24:23 2018 From: jini.george at oracle.com (Jini George) Date: Mon, 25 Jun 2018 22:54:23 +0530 Subject: RFR(S/M): 8205419: [testbug] TestJmapCore failing without SA: introduce @requires vm.hasSA In-Reply-To: References: <112db96d59084bdfafba2174e121c409@sap.com> <431269fc-7174-3051-0a86-baa00032754a@oracle.com> <3712cc995426467fa67b65493d556298@sap.com> <486ee0f785144f55bd739cc41bcd7e2d@sap.com> Message-ID: <48e96da6-8987-e581-da24-4364c291d096@oracle.com> Some of the tests were skipped on OSX due to stackwalking issues on OSX. (Some were pstack related, and some for jstack on corefiles). Thanks, Jini. On 6/25/2018 10:31 PM, Chris Plummer wrote: > Hi Goetz, > > The changes look good. Just one question though. In places where you have: > > ? 42? * @requires vm.hasSAandCanAttach & os.family != "mac" > > Do you know why these tests don't run on osx? I just wonder if it > related to attach not working unless root, and that is now covered by > hasSAandCanAttach(). If that is the case, the "mac" check can be removed. > > thanks, > > Chris > > On 6/22/18 1:20 AM, Lindenmaier, Goetz wrote: >> Hi, >> >> Ok, so I now added >> ?? vm.hasSA and >> ?? vm.hasSAandCanAttach >> I use the first in the JMapCore tests, and in the tests that don't >> check shouldSAAttach() >> right at the beginning. >> Wherever the test is immediately ended after checking shouldSAAttach(), I >> removed shouldSAAttach(), and added vm.hasSAandCanAttach.? The others >> depend on? ClhsdbLauncher. Just checking vm.hasSA is on the safe side >> if runOnCore() is changed. >> http://cr.openjdk.java.net/~goetz/wr18/8205419-requiresHasSA/02/ >> >> This also makes the implementation of VMProps and VMPlatform more clear >> as requested by Chris. >> >> Best regards, >> ? Goetz. >> >> >>> -----Original Message----- >>> From: Jini George >>> Sent: Thursday, June 21, 2018 12:18 PM >>> To: Lindenmaier, Goetz ; 'Chris Plummer' >>> ; serviceability-dev (serviceability- >>> dev at openjdk.java.net) >>> Subject: Re: RFR(S/M): 8205419: [testbug] TestJmapCore failing >>> without SA: >>> introduce @requires vm.hasSA >>> >>> Hi Goetz, >>> >>> Tests like TestJmapCore which involve only corefile debugging by SA (and >>> not attaching to a live process) would only need to check for the >>> presence of SA, and would not need to check for ptrace related attach >>> permissions like what is done for Linux and OSX -- since super user >>> privileges are not required to debug corefiles. So a single @requires >>> catering to both these might not be desirable. >>> >>> Having said that, I realized that runOnCore() of ClhsdbLauncher >>> incorrectly checks for the ptrace related attach permissions. (Will file >>> an issue to correct this). >>> >>> An @requires instead of a shouldSAAttach() might help in avoiding fake >>> passes like those on OSX when not run as "root". >>> (https://bugs.openjdk.java.net/browse/JDK-8199700) >>> >>> Thanks, >>> Jini. >>> >>> >>> >>> >>> On 6/21/2018 1:52 AM, Lindenmaier, Goetz wrote: >>>> Hi Chris, >>>> >>>> Thanks for looking at this change. >>>> >>>>> Can't VMProps.vmHasSA() just return Platform.shouldSAAttach()? It >>> seems >>>>> you are unnecessarily replicating Platform.shouldSAAttach() logic. >>>> I think there are two things to distinguish: >>>> ?? - platforms that don't implement SA >>>> ?? - systems/setups where SA does not work. >>>> If both can be recognized when VMProps is evaluated, shouldSAAttach() >>>> is the one that is superfluous in the long run?!? If not, >>>> shouldSAAttach >>>> should only return those where it is necessary to check in the test. >>>> But this is probably too much brains in this case ?? >>>> >>>>> If you are adding "@requires vm.hasSAandCanAttach" to a test, >>>>> shouldn't >>>>> you remove the test's Platform.shouldSAAttach() check? >>>> Well I asked in my mail whether I should do that. So I take this as >>>> a yes. >>>> But it depends on whether it must be evaluated in the test. >>>> >>>>> Is there a reason not to take the more direct and established approach >>>>> of just adding the Platform.shouldSAAttach() check to TestJmapCore? It >>>>> would be a lot less disruptive.? I know the vm.hasSAandCanAttach >>>>> approach has advantages in not even attempting to compile and run the >>>>> test, but I'm not so sure those advantages outweigh the simplicity of >>>>> just adding a Platform.shouldSAAttach() check to one test. I can go >>>>> either way here. Just wondering if there are additional reasons I >>>>> might >>>>> not be seeing. >>>> I like the @requires much more. It tells right where the test is >>>> described >>>> that it does not run with SA. The other is quite hidden in the test, >>>> e.g., in >>>> helper class ClhsdbLauncher.java. >>>> Also, it's more easy to remember (The implementor of >>>> TestJmapCore forgot it...).? And, last but not least, it seems best >>>> practice >>>> nowadays. There are lots of excludes for mac, they are also done by >>> @requires >>>> and not as a check using Platform at runtime. >>>> >>>>> Sorry, I don't have an answer to your VMProps evaluation question. You >>>>> might need to ask a wider audience than just svc. >>>> Whom should I ask? Hotspot-dev? >>>> >>>> Best regards, >>>> ??? Goetz. >>>> >>>> >>>>> thanks, >>>>> >>>>> Chris >>>>> >>>>> On 6/20/18 6:49 AM, Lindenmaier, Goetz wrote: >>>>>> Hi, >>>>>> >>>>>> TestJmapCore is failing on aix because there jhsdb is not >>>>>> implemented. >>>>>> So far, such tests check at runtime Platform.shouldSAAttach() >>>>>> which is >>>>> missing >>>>>> in TestJmapCore. >>>>>> >>>>>> Instead, I implement @requires vm.hasSAandCanAttach. >>>>>> This makes jtreg skip the tests without compiling and starting them. >>>>>> >>>>>> I added the new property to all tests that check shouldSAAttach in >>>>>> jdk >>> and >>>>> hostpot test >>>>>> directories. >>>>>> This is the implementation of the new property, the rest is just >>> repetitive >>>>> checking it: >>>>>> http://cr.openjdk.java.net/~goetz/wr18/8205419- >>>>> requiresHasSA/01/test/jtreg-ext/requires/VMProps.java.udiff.html >>>>>> webrev: >>>>>> http://cr.openjdk.java.net/~goetz/wr18/8205419-requiresHasSA/01/ >>>>>> >>>>>> Is it correct to assume the VMProps is evaluated on the same machine >>> and >>>>> with the same >>>>>> user as used for executing the test?? canPtraceAttachLinux directly >>>>> accesses the system. >>>>>> (Should I remove the checks for shouldSAAttach() from the changed >>> tests?) >>>>>> Best regards, >>>>>> ???? Goetz. >>>>> > > From serguei.spitsyn at oracle.com Mon Jun 25 17:55:06 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 25 Jun 2018 10:55:06 -0700 Subject: RFR(S): 8200720: Print additional information in thread dump (times, allocated bytes etc.) In-Reply-To: <70FECB3F-4E90-4B2C-AB22-09B42069323A@sap.com> References: <56817AB5-88FC-4FB9-A271-7206EA69ECA7@sap.com> <2728581E-0E32-4911-B1A9-79AD6CBA81B0@sap.com> <082ca98f-9a68-15a0-95b5-c4d5c733ab36@oracle.com> <933cdacc-9b6f-c8b5-e244-558f38e7888c@oracle.com> <55806aa8-f09b-e9a8-b512-b9676029c494@oracle.com> <0F497BC6-2569-4EF6-9B59-E24A8E1B3A37@sap.com> <5dc08f65e83f4374b8e1f73f218c6d6d@sap.com> <626701f8-b71b-58dc-f837-e5e1d81ef187@oracle.com> <19BE9680-3E6E-4620-B72D-D379F39C1C92@sap.com> <6ce47417-311b-e6cd-b8cd-d46eb7e04c2f@oracle.com> <3795D1AE-CBF4-4BC3-B707-ED77D4DE00D0@sap.com> <9f5a404a-24e6-9131-3c0d-80c5c01ccfca@oracle.com> <70FECB3F-4E90-4B2C-AB22-09B42069323A@sap.com> Message-ID: <0466f047-14e2-87b8-5d70-e1e9218c9bde@oracle.com> Hi Gunter, The fix looks good. Thanks, Serguei On 6/25/18 07:22, Haug, Gunter wrote: > Hi all, > > Thank you, David for taking the time to do the review. Here is hopefully the last version of the change: > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v5/ > > Following David's suggestion, _start_time_stamp is now constant and set by the constructor and the setter has been removed. This is the only difference to the previous version. > > Chris and Christoph, are you still OK with this? > > Thanks, > Gunter > > ?On 25.06.18, 12:29, "David Holmes" wrote: > > Hi Gunter, > > I see what you mean about the initialization point of the statistics. > Personally I'd prefer to see the startTime as read-only and only set by > the constructor rather than having a general-purpose setter function for it. > > Thanks, > David > > On 21/06/2018 1:28 AM, Haug, Gunter wrote: > > Hi David, > > > > Thanks for taking the time to look into this! > > > >> "statistic info" is not very good grammatically. These things are either > >> statistics, or statistical information. So e.g. > >> > >> class ThreadStatistics > >> ... > > > > OK, I'll change that! > > > >> const_cast(this)->cooked_allocated_bytes(); > >> > >> Why do we need a const cast to invoke a method on ourselves ?? > > > > cooked_allocated_bytes() is not declared const and we can't make it, as it uses OrderAccess::load_acquire(&_allocated_bytes). Therefor the const_cast. > > > >> src/hotspot/share/runtime/thread.hpp > >> > >> Please put the new #include in alphabetical order. > > > > Wilco! > > > >> I was expecting to see a default parameter used here rather than adding > >> an overload: > >> virtual void print_on(outputStream* st, bool print_extended_info = > >> false) const { print_on(st, print_extended_info); } > > > > Thread inherits from AllocatedObj (in dbg, at least) where virtual void AllocatedObj::print_on(outputStream*) const is declared. Some compilers, e.g. gcc, will complain that this is hidden by Thread::print_on(outputStream*, bool). Others, e.g. clang, are happy with that. We could implement a method with a different name but would that be nicer? > > > >> 2179 static void print_on(outputStream* st, bool print_stacks, bool > >> internal_format, bool print_concurrent_locks, bool extended_thread_info); > >> > >> Again I expected to see a default parameter here - but I didn't check if > >> all callers themselves take the new parameter. ?? > > > > This one has just one single caller, so I think it's OK. > > > >> src/hotspot/share/runtime/vm_operations.hpp > >> ... > > > > I'll change it to conform to the guidelines. > > > >> src/hotspot/share/runtime/threadStatisticInfo.hpp > >> > >> There are no includes in this file. It should include the appropriate > >> std header for type definitions, and os.hpp. > > > > Agreed, I'll add the includes. > > > > > >> I'm not sure if class ThreadStatisticInfo needs an allocation type as a > >> super type. > >> > >> Might be worth adding a constructor to give a default value so that you > >> can tell if the statistics have been initialized when they appear in the > >> printout. ? > > > > ThreadStatisticInfo is a data member of Thread. Currently it is initialized in Thread::Thread, but I can add a constructor, if you prefer. > > > > Thanks again, > > Gunter > > > > > > > > On 20.06.18, 09:15, "David Holmes" wrote: > > > > Hi Gunter, > > > > On 19/06/2018 8:51 PM, Haug, Gunter wrote: > > > Hi all, > > > > > > Thanks Chris and Christoph for the reviews! Christoph, I'll incorporate the little improvement you suggested. > > > David, are you OK with the change as well? > > > > Sorry I'd lost track of this one a bit ... > > > > The overall approach now seems okay. > > > > Some naming/terminology issues: > > > > "statistic info" is not very good grammatically. These things are either > > statistics, or statistical information. So e.g. > > > > class ThreadStatistics > > > > ThreadStatistics& statistics() { return _statistics; } > > > > etc. > > > > "extended_thread_info" should really be print_extended_info (similar to > > print_concurrent_locks). You don't need "thread" in there when this is > > always part of a Thread related API. > > > > > > src/hotspot/share/runtime/thread.cpp > > > > const_cast(this)-> > > > > Why do we need a const cast to invoke a method on ourselves ?? > > > > --- > > > > src/hotspot/share/runtime/thread.hpp > > > > Please put the new #include in alphabetical order. > > > > 641 // Printing > > 642 void print_on(outputStream* st, bool extended_thread_info) const; > > 643 virtual void print_on(outputStream* st) const { print_on(st, > > false); } > > > > I was expecting to see a default parameter used here rather than adding > > an overload: > > > > virtual void print_on(outputStream* st, bool print_extended_info = > > false) const { print_on(st, print_extended_info); } > > > > > > 2179 static void print_on(outputStream* st, bool print_stacks, bool > > internal_format, bool print_concurrent_locks, bool extended_thread_info); > > > > Again I expected to see a default parameter here - but I didn't check if > > all callers themselves take the new parameter. ?? > > > > --- > > > > src/hotspot/share/runtime/vm_operations.hpp > > > > 376 VM_PrintThreads() > > 377 { _out = tty; _print_concurrent_locks = PrintConcurrentLocks; ; > > _extended_thread_info = false; } > > 378 VM_PrintThreads(outputStream* out, bool print_concurrent_locks, > > bool extended_thread_info) > > 379 { _out = out; _print_concurrent_locks = print_concurrent_locks; > > _extended_thread_info = extended_thread_info; } > > 380 VMOp_Type type() const > > 381 { return VMOp_PrintThreads; } > > > > Style nits: either keep everything on one line as before (though I agree > > the lines are now too long) or else the style should be: > > > > 380 VMOp_Type type() const { > > 381 return VMOp_PrintThreads; > > 382 } > > > > Also unclear why (existing) VM_PrintThreads constructor doesn't use > > initializer list (like VM_PrintMetadata below it) or default parameters? > > > > --- > > > > src/hotspot/share/runtime/threadStatisticInfo.hpp > > > > There are no includes in this file. It should include the appropriate > > std header for type definitions, and os.hpp. > > > > I'm not sure if class ThreadStatisticInfo needs an allocation type as a > > super type. > > > > Might be worth adding a constructor to give a default value so that you > > can tell if the statistics have been initialized when they appear in the > > printout. ? > > > > --- > > > > Thanks, > > David > > > > > > > Thanks again, > > > Gunter > > > > > > On 12.06.18, 01:13, "Chris Plummer" wrote: > > > > > > Hi Gunter, > > > > > > The changes look fine. I can live with the options parsing in > > > attachListener.cpp. As you point out, unrecognized options were already > > > silently ignored. > > > > > > thanks, > > > > > > Chris > > > > > > On 6/8/18 7:05 AM, Haug, Gunter wrote: > > > > Hi all, > > > > > > > > thanks a lot for all the input! I have prepared a new version of the webrev incorporating the suggestions you made (at least I tried): > > > > > > > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v3/ > > > > > > > > This version outputs the thread times unconditionally while the other information is guarded by a flags. I think, most participants found the thread times the most valuable information and had no (strong) objections to adding them unconditionally. > > > > > > > > @David > > > > Implementation is much simpler with conditional output only for JavaThreads. I could get rid of the terrible hack (changing the flag) without having to change too many source files. > > > > > > > > The information on allocated bytes is present in the Thread class already before this proposed change, I just print it. It might be sensible to move _allocated_bytes and the respective methods to the ThreadStatisticInfo class as G?tz suggested. I haven?t done that in the current version, though. > > > > > > > > @David and Thomas > > > > I've removed the pthread-id output. I'm unsure myself what it could be good for. Anyway, we could add it with a separate change (with a better implementation) if there is a need to. > > > > > > > > @Chris > > > > As you have written, the user has no direct contact to the attach listener of the VM and jstack won't misinterpret e.g. -help. I agree that the parsing in attachListener.cpp could be more robust. However, it hasn't been so far either. All that is done in the current implementation is a strcmp to -l everything else will be silently ignored. I can try to make this more robust or we could say that the new output is only available via jcmd. OTH nothing bad can happen with the current version of the proposed change, so we could also leave it as it is. What would you prefer? > > > > > > > > @Kirk and Thomas > > > > Implementing a new diagnostic command is certainly well worth a thought. However, it appears to me that it's not necessary in this case. There are already flags to jstack and jcmd Thread.print, so it's not that uncommon. The amount of information that would be available by a new diagnostic command and that is now added to the thread dump is quite small and I think that is tolerable. Moreover, and most importantly, our support team is use to find the information in the thread dump and it would make things easier to us if I could leave it there. > > > > > > > > Thanks again and have a nice weekend, > > > > Gunter > > > > > > > > > > > > On 06.06.18, 04:56, "David Holmes" wrote: > > > > > > > > Hi Goetz, > > > > > > > > On 5/06/2018 11:07 PM, Lindenmaier, Goetz wrote: > > > > > Hi > > > > > > > > > > this discussion touched a lot of points so far, which seem > > > > > to lead to different conclusions. > > > > > > > > > > I think we should look at the values printed: > > > > > > > > > > 1. cpu=6300.65ms elapsed=123.28s > > > > > Overhead > > > > > cpu time: > > > > > * system calls at thread dump time > > > > > elapsed time: > > > > > * 1 system call at thread creation time > > > > > * 1 64-bit field per thread for the thread start time > > > > > * 1 system call at thread dump time > > > > > > > > > > As I understand, JDK-8143176 would have had to get the > > > > > time at each locking, which is much more time critical > > > > > than doing this during thread creation. While > > > > > > > > Correct. > > > > > > > > > the time a lock was held would be much more useful, > > > > > the ratio here, combined with knowledge about the application, > > > > > could lead to the conclusion that the thread is wrongly > > > > > blocked at much lower cost. > > > > > > > > Agreed. I see no issue with unconditionally adding this information as > > > > it should address some of the concerns of 8143176 as well. > > > > > > > > > > > > > > 2. pthread-id=0x109708000 > > > > > Overhead: > > > > > * a field access at thread dump time. Negligible I would say. > > > > > > > > I'd overlooked this addition, but it is of course the chunk of ifdef > > > > code in osThread.cpp that I objected to. The problem I have here is the > > > > convoluted mess of "thread identifiers" that we already have. We > > > > currently print: > > > > > > > > st->print("tid=" INTPTR_FORMAT " ", p2i(this)); > > > > > > > > which is just the address of the Thread/JavaThread object. Then: > > > > > > > > st->print("nid=0x%x ", thread_id()); > > > > > > > > where 'n' is supposed to represent "native thread id", which is: > > > > - linux: kernel thread id from syscall gettid > > > > - solaris: thread library identity from thr_create() or thr_self() > > > > - windows: thread id from _beginthreadex > > > > - OS X: kernel thread id from pthread_mach_thread_np(pthread_self()); > > > > - Other BSD: kernel (?) thread id from syscall thr_self or getthrid > > > > - AIX: thread library identity from pthread_create() or pthread_self() > > > > > > > > It's telling that the code Gunter added gets the thread library id on > > > > linux, but a "kernel thread id" on other platforms! Do we want to see a > > > > thread library id and a kernel thread id? IIRC nid is supposed be the id > > > > you need to see the thread in a debugger. In which case I'm unclear what > > > > role the thread id Gunter wants to add is playing? I don't think we need > > > > to see kernel ids in general, and pthread-id isn't useful for debugging > > > > is it? > > > > > > > > > > > > > > 3. allocated=242236760B defined_classes=1725 > > > > > Overhead: > > > > > * 1 64-bit field per thread. > > > > > * counter increment on class creation > > > > > > > > > > Especially defined_classes seems to be controversial. > > > > > As this only makes sense for Java threads, this could > > > > > be printed in a line of it's own in Threads::print_on_error() > > > > > (which already gets a flag to customize for jstack: > > > > > print_concurrent_locks) by calling a dedicated function in Thread. > > > > > No need for flag PrintExtendedThreadInfo thus. > > > > > > > > > > But I would propose to drop this information as it is > > > > > too controversial. > > > > > > > > Agreed. > > > > > > > > > This leaves the times and the pthread-id to be decided whether > > > > > they are worth the cost. If so, I think they should be printed > > > > > unconditional. > > > > > > > > Agreed. With some refinement of the "thread id" issue. > > > > > > > > > If a flag is required to avoid bloating the info in the default case, > > > > > I would include more infos, as os_prio and nid under that flag. > > > > > > > > Of course those things exist for good reason. The native id is supposed > > > > to be what aids in you matching what you see in our thread dumps with > > > > what you see in a debugger. Priority is less of an issue these days, but > > > > there was a time ... ;-) > > > > > > > > > As jstack is deprecated, changes to jstack could be skipped always > > > > > disabling the new printouts. > > > > > > > > > > Finally, I would propose to move _allocated_bytes into > > > > > threadStatisticInfo.hpp. > > > > > > > > Didn't that part get dropped? (You don't need it for the time info or > > > > thread id). That's a GC logging issue to me. > > > > > > > > Thanks, > > > > David > > > > > > > > > Best regards, > > > > > Goetz. > > > > > > > > > > > > > > > > > > > > > > > > >> -----Original Message----- > > > > >> From: serviceability-dev [mailto:serviceability-dev- > > > > >> bounces at openjdk.java.net] On Behalf Of David Holmes > > > > >> Sent: Dienstag, 5. Juni 2018 04:53 > > > > >> To: Haug, Gunter ; Chris Plummer > > > > >> ; serviceability-dev > > > >> dev at openjdk.java.net>; Langer, Christoph ; > > > > >> hotspot-runtime-dev at openjdk.java.net > > > > >> Subject: Re: RFR(S): 8200720: Print additional information in thread dump > > > > >> (times, allocated bytes etc.) > > > > >> > > > > >> Hi Gunter, > > > > >> > > > > >> On 5/06/2018 1:27 AM, Haug, Gunter wrote: > > > > >>> Hi David, Chris, > > > > >>> > > > > >>> would it be an option to unconditionally print the additional information? > > > > >> Regardless which way this is implemented it will be rather complicated and it > > > > >> only switches on/off a few field in the thread dump. > > > > >> > > > > >> I'm not convinced this is all suitable information for a thread stack > > > > >> dump. I can see the time information being useful if using the dump to > > > > >> try and diagnoze a hang or responsiveness issue. But the allocated-bytes > > > > >> and classes-defined just doesn't seem useful in the context of a thread > > > > >> dump to me. Anyone interested in allocation stats is going to be looking > > > > >> at GC logs etc which is where this belongs. Ditto the class-stats belong > > > > >> in some kind of classloading logging IMHO. > > > > >> > > > > >> I'm very reluctant to burden the implementation with capturing these > > > > >> kinds of stats, just to use for diagnostic purposes that may not even be > > > > >> asked for. I'm very much in the "you shouldn't pay for what you don't > > > > >> use" camp in that regard. (See my comments in JDK-8143176 referenced by > > > > >> Sean.) > > > > >> > > > > >> The ThreadStatisticInfo adds overhead to every thread object - and I'd > > > > >> have to suspect there could be overlap with whatever flight recorder > > > > >> sticks in there too. (Thread has become even more bloated in recent time!). > > > > >> > > > > >> Cheers, > > > > >> David > > > > >> > > > > >> > > > > >>> Thanks, > > > > >>> Gunter > > > > >>> > > > > >>> On 04.06.18, 01:13, "David Holmes" wrote: > > > > >>> > > > > >>> Hi Gunter, Chris, > > > > >>> > > > > >>> Sorry just trying to catch up and this is only a partial look so far ... > > > > >>> > > > > >>> BTW these changes are not limited to serviceability code so adding in > > > > >>> runtime team as well. > > > > >>> > > > > >>> On 2/06/2018 9:12 AM, Chris Plummer wrote: > > > > >>> > Hi Gunter, > > > > >>> > > > > > >>> > On 6/1/18 3:17 AM, Haug, Gunter wrote: > > > > >>> >> Hi Chris, > > > > >>> >> > > > > >>> >> thanks for looking into this! > > > > >>> >> > > > > >>> >> Re the synchronization: The value is stored only in a VM operation at > > > > >>> >> a safepoint by the VM thread. I was of the opinion, that this could > > > > >>> >> not be interrupted by a second VM operation (of the same type). Or > > > > >> am > > > > >>> >> I missing something else? > > > > >>> > I was really thinking more about the temporary changing of > > > > >>> > PrintExtendedThreadInfo, not the value stored in the VMOp. You may > > > > >> be be > > > > >>> > correct that no more than one VMOp is executing, but while it is > > > > >>> > executing it is has changed the value of PrintExtendedThreadInfo, > > > > >> which > > > > >>> > might have an impact on anything else that triggers printing thread info > > > > >>> > while the VMOp is executing. > > > > >>> > > > > >>> Even if nothing else can possibly be running during the safepoint I find > > > > >>> it extremely bad form to change a command-line flag in this way, > > > > >>> particularly from a safepoint! > > > > >>> > > > > >>> If this flag is intended to be dynamically enabled as part of a dcmd > > > > >>> then it should also be a Manageable flag IMHO. > > > > >>> > > > > >>> That said ... > > > > >>> > > > > >>> >> I did think about passing an argument to the various print_on > > > > >> member > > > > >>> >> functions of the thread classes, but this would require rather > > > > >>> >> extensive code changes for a rather tiny extension. Therefore I feel > > > > >>> >> doing it like this is the lesser evil. > > > > >>> > > > > >>> ... it's obviously not truly a global setting, but one that is needed on > > > > >>> a per-print-request basis. The flag is just the default, but if the > > > > >>> default is off you still want to enable extended printing on a > > > > >>> per-request basis. > > > > >>> > > > > >>> The current print_on mechanics is not set up for this kind of > > > > >>> flexibility. I think this needs more thought. > > > > >>> > > > > >>> --- > > > > >>> > > > > >>> Re osThread.cpp shared code changes ... I really hate to see all the > > > > >>> ifdefs in there. Please add a pd_print_on function to support this if > > > > >>> you truly want platform specific information. > > > > >>> > > > > >>> --- > > > > >>> > > > > >>> threadStatisticInfo.hpp > > > > >>> > > > > >>> typo: getElepsedTime() -> getElapsedTime() > > > > >>> > > > > >>> Thanks, > > > > >>> David > > > > >>> > > > > >>> > > > > >>> > > > > >>> >> > > > > >>> >> Re thread_dump(): I think it's correct or, well, at least it works ;-) > > > > >>> >> In fact jstack will transfer the "-e" and "-l" in only one string, > > > > >>> >> i.e. op->arg(0). > > > > >>> > So you are saying that op->arg(0) is "-e -l" when using jstack? I think > > > > >>> > you really need to clean up the parsing. As it stands right now passing, > > > > >>> > I get the feeling that if a user erroneously asks for help by using > > > > >>> > "jcmd Thread.Print -help", it will end up executing with -e an -l > > > > >>> > enabled, and no failure for the invalid "-help" option. > > > > >>> >> Christoph has already explained my reasoning. But I agree, it's not > > > > >>> >> nice and I would be happy to do it like Christoph suggested. > > > > >>> > I'll respond separately to his suggestion. > > > > >>> > > > > > >>> > thanks, > > > > >>> > > > > > >>> > Chris > > > > >>> >> > > > > >>> >> And typo fixed, sorry. > > > > >>> >> > > > > >>> >> Thanks again, > > > > >>> >> Gunter > > > > >>> >> > > > > >>> >> On 01.06.18, 00:03, "Chris Plummer" > > > > >> wrote: > > > > >>> >> > > > > >>> >> Hi Gunter, > > > > >>> >> globals.hpp: fix typo "informatiuon" > > > > >>> >> I worry a little bit about the synchronizing (if that's the right > > > > >>> >> word) > > > > >>> >> of PrintExtendedThreadInfo and the dcmd's -e flag. When using -e, > > > > >>> >> you > > > > >>> >> are temporarily enabling PrintExtendedThreadInfo if it was false. > > > > >>> >> This > > > > >>> >> temporarily changes the behavior of thread dumps, and could > > > > >>> >> impact other > > > > >>> >> uses that happen in parallel. Also, could two simultaneous uses > > > > >>> >> of -e > > > > >>> >> result in PrintExtendedThreadInfo not getting restored properly? > > > > >>> >> thread_dump() doesn't look right. It looks like you are iterating > > > > >>> >> char > > > > >>> >> by char over the argument, and expect something like "-el" to be > > > > >>> >> specified rather then "-e -l". The loop should be iterating over > > > > >>> >> op->arg(i), not op->arg(0)[i]. > > > > >>> >> The rest of the changes look fine. > > > > >>> >> thanks, > > > > >>> >> Chris > > > > >>> >> On 5/30/18 8:12 AM, Haug, Gunter wrote: > > > > >>> >> > Hi all, > > > > >>> >> > > > > > >>> >> > As Chris proposed, I have made an the extended output > > > > >>> >> switchable. There is an VM flag (PrintExtendedThreadInfo), which is > > > > >>> >> false by default. Moreover, there is an Option (-e) which can be used > > > > >>> >> with jcmd Thread.print as well as with jstack. The -e option > > > > >>> >> essentially sets PrintExtendedThreadInfo true just for the respective > > > > >>> >> thread dump. > > > > >>> >> > > > > > >>> >> > Here is the updated webrev: > > > > >>> >> > > > > > >>> >> > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v2 > > > > >>> >> > > > > > >>> >> > (https://bugs.openjdk.java.net/browse/JDK-8200720) > > > > >>> >> > > > > > >>> >> > Thanks, > > > > >>> >> > Gunter > > > > >>> >> > > > > > >>> >> > > > > > >>> >> > On 02.05.18, 17:07, "serviceability-dev on behalf of Haug, > > > > >>> >> Gunter" > > > >>> >> gunter.haug at sap.com> wrote: > > > > >>> >> > > > > > >>> >> > Hi Chris, > > > > >>> >> > > > > > >>> >> > Thanks for looking into this. > > > > >>> >> > You're right, there is a little more we have. We have > > > > >>> >> implemented an IO tracing mechanism which - rather as a byproduct - > > > > >>> >> keeps track of bytes read and written per thread. However, this of > > > > >>> >> course requires changes not only in hotspot. We would be happy to > > > > >>> >> contribute this as well, but this is a far bigger change and will > > > > >>> >> probably lead to a far bigger discussion. Anyway, with the number of > > > > >>> >> bytes read, the number of classes defined doesn't look that arbitrary > > > > >>> >> anymore, as one can correlate IO to class loading. > > > > >>> >> > > > > > >>> >> > Regarding the verbose option I think that's a good idea! > > > > >>> >> > > > > > >>> >> > Thanks again, > > > > >>> >> > Gunter > > > > >>> >> > > > > > >>> >> > On 01.05.18, 22:55, "Chris Plummer" > > > > >>> >> wrote: > > > > >>> >> > > > > > >>> >> > Hi Gunter, > > > > >>> >> > > > > > >>> >> > The output you are adding is all useful. I think the > > > > >>> >> question is (and > > > > >>> >> > I'd like to see a few people chime in on this for this > > > > >>> >> review) is > > > > >>> >> > whether or not all of it is the appropriate for a > > > > >>> >> thread's stack dump. > > > > >>> >> > For example, defined_classes is on the fringe of what > > > > >>> >> I would call > > > > >>> >> > generally useful info in a stack dump. Sure, there > > > > >>> >> might be that rare > > > > >>> >> > case when you need it, but how often compared to other > > > > >>> >> useful info > > > > >>> >> > maintained on a per thread basis. How many other bits > > > > >>> >> of useful info are > > > > >>> >> > not being printed in favor of defined_classes? It > > > > >>> >> seems you have more in > > > > >>> >> > the queue. How many? I'm worried about how cluttered > > > > >>> >> the stack dumps > > > > >>> >> > will get. Maybe we should add some sort of verbose > > > > >>> >> thread dumping > > > > >>> >> > option. Just a thought. > > > > >>> >> > > > > > >>> >> > As for the implementation, overall it looks good, but > > > > >>> >> I can't speak to > > > > >>> >> > whether or not you are doing proper accounting of > > > > >>> >> defined_classes and > > > > >>> >> > bytes allocated. You'll need input from someone with > > > > >>> >> more knowledge of > > > > >>> >> > those areas. We'll also need to do some testing to > > > > >>> >> make sure tool tests > > > > >>> >> > are not impacted. > > > > >>> >> > > > > > >>> >> > thanks, > > > > >>> >> > > > > > >>> >> > Chris > > > > >>> >> > > > > > >>> >> > On 4/30/18 2:51 AM, Haug, Gunter wrote: > > > > >>> >> > > Hi, > > > > >>> >> > > > > > > >>> >> > > this is an update to an RFR I posted on hotspot-dev, > > > > >>> >> but it is probably more suitable to post it here. Can I please have a > > > > >>> >> review and a sponsor for the following enhancement: > > > > >>> >> > > > > > > >>> >> > > http://cr.openjdk.java.net/~ghaug/webrevs/8200720.v1 > > > > >>> >> > > https://bugs.openjdk.java.net/browse/JDK-8200720 > > > > >>> >> > > > > > > >>> >> > > We at SAP have extended the thread dumps (obtained > > > > >>> >> by jstack or jcmd) by several fields providing thread specific > > > > >>> >> information. These extensions are quite popular with our support > > > > >> team. > > > > >>> >> With some knowledge of the architecture of the application, they > > > > >> often > > > > >>> >> allow for quick and simple diagnosis of a running system. Therefore > > > > >> we > > > > >>> >> would like to contribute these enhancements. > > > > >>> >> > > > > > > >>> >> > > I took a few simple examples here, namely cpu time, > > > > >>> >> elapsed time since thread creation, bytes allocated and classes > > > > >>> >> defined by the thread and the pthread-id or equivalent on platforms > > > > >>> >> where it makes sense. Provided it is known how the application > > > > >> should > > > > >>> >> behave, a misbehaving thread can identified easily. > > > > >>> >> > > > > > > >>> >> > > There is no measurable overhead for this > > > > >>> >> enhancement. However, it may be a problem that the format of the > > > > >>> >> output is changed. Tools parsing the output may have to be changed. > > > > >>> >> > > > > > > >>> >> > > Here is an example of the output generated: > > > > >>> >> > > > > > > >>> >> > > ------------------------------------------------------ > > > > >>> >> > > > > > > >>> >> > > "main" #1 prio=5 os_prio=31 cpu=6300.65ms > > > > >>> >> elapsed=123.28s allocated=242236760B defined_classes=1725 > > > > >>> >> tid=0x00007fa13a806000 nid=0x1c03 pthread-id=0x109708000 waiting > > > > >> on > > > > >>> >> condition [0x0000000109707000] > > > > >>> >> > > java.lang.Thread.State: TIMED_WAITING (sleeping) > > > > >>> >> > > JavaThread state: _thread_blocked > > > > >>> >> > > Thread: 0x00007fa13a806000 [0x1c03] State: > > > > >>> >> _at_safepoint _has_called_back 0 _at_poll_safepoint 0 > > > > >>> >> > > JavaThread state: _thread_blocked > > > > >>> >> > > at java.lang.Thread.sleep(java.base/Native Method) > > > > >>> >> > > ... > > > > >>> >> > > ------------------------------------------------------ > > > > >>> >> > > > > > > >>> >> > > As mentioned above, we have a whole bunch of other > > > > >>> >> enhancements to the thread dump similar to this one and would be > > > > >>> >> willing to contribute them if there is any interest. > > > > >>> >> > > > > > > >>> >> > > Thanks and best regards, > > > > >>> >> > > Gunter > > > > >>> >> > > > > > > >>> >> > > > > > > >>> >> > > > > > >>> >> > > > > > >>> >> > > > > > >>> >> > > > > > >>> >> > > > > > >>> >> > > > > >>> > > > > > >>> > > > > > >>> > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > From serguei.spitsyn at oracle.com Mon Jun 25 18:00:17 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 25 Jun 2018 11:00:17 -0700 Subject: RFR: JDK-8205508: hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java fails with Prompt is not received during 300200 milliseconds. In-Reply-To: <5B312469.7060206@oracle.com> References: <5B312469.7060206@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Mon Jun 25 18:11:30 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 25 Jun 2018 11:11:30 -0700 Subject: RFR: JDK-8205508: hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java fails with Prompt is not received during 300200 milliseconds. In-Reply-To: References: <5B312469.7060206@oracle.com> Message-ID: <890d5683-63a9-18c7-d108-00b5b374ca0b@oracle.com> An HTML attachment was scrubbed... URL: From alexey.menkov at oracle.com Mon Jun 25 18:15:41 2018 From: alexey.menkov at oracle.com (Alex Menkov) Date: Mon, 25 Jun 2018 11:15:41 -0700 Subject: RFR 8203174: [Graal] JDI tests fail with Unexpected exception: com.sun.jdi.ObjectCollectedException In-Reply-To: References: Message-ID: <211a436c-c1af-2dca-11e2-f75185888b57@oracle.com> Hi Daniil, Couple minor notes: - Debugee.DEFAULT_MAIN_THREAD_NAME makes sense only for HeapwalkingDebuggee-inherited tests, so it would be better to move the constant to the class (and make it protected); - HeapwalkingDebuggee.useStrictCheck can be simplified to something like switch (className) { case "java.lang.String": case "char[]": case "byte[]": case "boolean[]": case "float[]": case "long[]": case "int[]": case "double[]": return false; case "java.lang.Thread": if (otherThreadPresent) { return false; } break; } return true; --alex On 06/22/2018 19:13, Daniil Titov wrote: > Please review the changes that fix 15 tests failures when running with Graal. > > The main problem here is that the tests assume that no garbage collection happens before the tests complete their checks. The other problem is that the tests could not strictly distinguish the objects created by the debuggee on the request from the test from the objects of the same class created by the debuggee VM on its own. In case when java.lang.String or arrays of primitives are used in the tests the number of objects the test operates could significantly exceeds the number of the objects the test actually created and iterating over all of them results in the timeout. > > The fix ensures that the object references created in the tests are protected from garbage collection or modifies the test to expect an ObjectCollectedException where necessary. In some cases, the fix ensures that only the main thread of the debuggee is running to prevent the garbage collector thread to create objects of the same class the test is working with and avoid timeouts. > > Webrev: http://cr.openjdk.java.net/~dtitov/8203174/webrev.01 > Bug: https://bugs.openjdk.java.net/browse/JDK-8203174 > > Thanks, > Daniil > > From gary.adams at oracle.com Mon Jun 25 18:27:56 2018 From: gary.adams at oracle.com (Gary Adams) Date: Mon, 25 Jun 2018 14:27:56 -0400 Subject: RFR: JDK-8205508: hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java fails with Prompt is not received during 300200 milliseconds. In-Reply-To: <890d5683-63a9-18c7-d108-00b5b374ca0b@oracle.com> References: <5B312469.7060206@oracle.com> <890d5683-63a9-18c7-d108-00b5b374ca0b@oracle.com> Message-ID: <5B31342C.4080104@oracle.com> The first time I looked into problems with exclude001 test, we discovered a large number of new packages in the jdk.internal classes that were introduced in jdk9. The test needed to add excludes for any of the jdk.* methods or it could not finish in time. As a follow up I'll try a test run with unlimited time and no methods excluded to get a specific count of methods that are being processed. Over time new features have been added, e.g. string concatenation optimizations, lambda functions, etc., etc., etc. For a test that does method tracing, each new method adds to the collective time. If you can not reduce the number of methods called, then the time for the test needs to be increased. ... On 6/25/18, 2:11 PM, Chris Plummer wrote: > I'm also wondering how fast this test runs on other platforms and when > passing on solaris-sparc. 5 minutes already seems like a long time for > this test. There could be an underlying issue that needs to be addressed. > > Chris > > On 6/25/18 11:00 AM, serguei.spitsyn at oracle.com wrote: >> Hi Gary, >> >> It looks Okay. >> But I'm curious when this started failing and what triggered it to fail? >> >> Thanks, >> Serguei >> >> >> On 6/25/18 10:20, Gary Adams wrote: >>> The exclude001 test times out on solaris sparc debug builds. >>> >>> Basically, this test is all about tracing method calls and introduces >>> exclude filters to reduce the callbacks to a select set of packages. >>> The time spent tracing/filtering method callbacks is purely a function >>> of the number of methods that are processed. On this particularly slow >>> target platform, more time is needed before issuing a timeout. >>> >>> Issue: https://bugs.openjdk.java.net/browse/JDK-8205508 >>> >>> Proposed fix: >>> diff --git >>> a/test/hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.javab/test/hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java >>> >>> --- >>> a/test/hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java >>> +++ >>> b/test/hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java >>> @@ -63,7 +63,7 @@ >>> * nsk.jdb.exclude.exclude001.exclude001a >>> * @run main/othervm PropertyResolvingWrapper >>> nsk.jdb.exclude.exclude001.exclude001 >>> * -arch=${os.family}-${os.simpleArch} >>> - * -waittime=5 >>> + * -waittime=7 >>> * -debugee.vmkind=java >>> * -transport.address=dynamic >>> * -jdb=${test.jdk}/bin/jdb >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From goetz.lindenmaier at sap.com Mon Jun 25 21:00:19 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 25 Jun 2018 21:00:19 +0000 Subject: RFR(S/M): 8205419: [testbug] TestJmapCore failing without SA: introduce @requires vm.hasSA In-Reply-To: References: <112db96d59084bdfafba2174e121c409@sap.com> <431269fc-7174-3051-0a86-baa00032754a@oracle.com> <3712cc995426467fa67b65493d556298@sap.com> <486ee0f785144f55bd739cc41bcd7e2d@sap.com> Message-ID: <9324ebf53f9242d9b79c7dd71cceca79@sap.com> Hi Jini Thanks for reviewing again, I fixed the typo's! Best regards, Goetz. > -----Original Message----- > From: Jini George > Sent: Monday, June 25, 2018 6:20 PM > To: Lindenmaier, Goetz ; 'Chris Plummer' > ; serviceability-dev (serviceability- > dev at openjdk.java.net) > Subject: Re: RFR(S/M): 8205419: [testbug] TestJmapCore failing without SA: > introduce @requires vm.hasSA > > Hi Goetz, > > Your changes look good, though I feel that even for the tests that use > ClhsdbLauncher and which need an attach, an "@requires > vm.hasSAandCanAttach" could be used, and the Platform.shouldSAAttach() > calls can be removed from ClhsdbLauncher. > > Some nits: > > 1. In VMProps.java, > > servicability ==> serviceability (an extra 'e') > > 2. In Platform.java > > reqires ==> requires > > Thanks, > Jini (Not a (R)eviewer). > > On 6/22/2018 1:50 PM, Lindenmaier, Goetz wrote: > > Hi, > > > > Ok, so I now added > > vm.hasSA and > > vm.hasSAandCanAttach > > I use the first in the JMapCore tests, and in the tests that don't check > shouldSAAttach() > > right at the beginning. > > Wherever the test is immediately ended after checking shouldSAAttach(), I > > removed shouldSAAttach(), and added vm.hasSAandCanAttach. The > others > > depend on ClhsdbLauncher. Just checking vm.hasSA is on the safe side > > if runOnCore() is changed. > > http://cr.openjdk.java.net/~goetz/wr18/8205419-requiresHasSA/02/ > > > > This also makes the implementation of VMProps and VMPlatform more > clear > > as requested by Chris. > > > > Best regards, > > Goetz. > > > > > >> -----Original Message----- > >> From: Jini George > >> Sent: Thursday, June 21, 2018 12:18 PM > >> To: Lindenmaier, Goetz ; 'Chris Plummer' > >> ; serviceability-dev (serviceability- > >> dev at openjdk.java.net) > >> Subject: Re: RFR(S/M): 8205419: [testbug] TestJmapCore failing without > SA: > >> introduce @requires vm.hasSA > >> > >> Hi Goetz, > >> > >> Tests like TestJmapCore which involve only corefile debugging by SA (and > >> not attaching to a live process) would only need to check for the > >> presence of SA, and would not need to check for ptrace related attach > >> permissions like what is done for Linux and OSX -- since super user > >> privileges are not required to debug corefiles. So a single @requires > >> catering to both these might not be desirable. > >> > >> Having said that, I realized that runOnCore() of ClhsdbLauncher > >> incorrectly checks for the ptrace related attach permissions. (Will file > >> an issue to correct this). > >> > >> An @requires instead of a shouldSAAttach() might help in avoiding fake > >> passes like those on OSX when not run as "root". > >> (https://bugs.openjdk.java.net/browse/JDK-8199700) > >> > >> Thanks, > >> Jini. > >> > >> > >> > >> > >> On 6/21/2018 1:52 AM, Lindenmaier, Goetz wrote: > >>> Hi Chris, > >>> > >>> Thanks for looking at this change. > >>> > >>>> Can't VMProps.vmHasSA() just return Platform.shouldSAAttach()? It > >> seems > >>>> you are unnecessarily replicating Platform.shouldSAAttach() logic. > >>> I think there are two things to distinguish: > >>> - platforms that don't implement SA > >>> - systems/setups where SA does not work. > >>> If both can be recognized when VMProps is evaluated, shouldSAAttach() > >>> is the one that is superfluous in the long run?! If not, shouldSAAttach > >>> should only return those where it is necessary to check in the test. > >>> But this is probably too much brains in this case ?? > >>> > >>>> If you are adding "@requires vm.hasSAandCanAttach" to a test, > shouldn't > >>>> you remove the test's Platform.shouldSAAttach() check? > >>> Well I asked in my mail whether I should do that. So I take this as a yes. > >>> But it depends on whether it must be evaluated in the test. > >>> > >>>> Is there a reason not to take the more direct and established approach > >>>> of just adding the Platform.shouldSAAttach() check to TestJmapCore? > It > >>>> would be a lot less disruptive.? I know the vm.hasSAandCanAttach > >>>> approach has advantages in not even attempting to compile and run > the > >>>> test, but I'm not so sure those advantages outweigh the simplicity of > >>>> just adding a Platform.shouldSAAttach() check to one test. I can go > >>>> either way here. Just wondering if there are additional reasons I might > >>>> not be seeing. > >>> I like the @requires much more. It tells right where the test is described > >>> that it does not run with SA. The other is quite hidden in the test, e.g., in > >>> helper class ClhsdbLauncher.java. > >>> Also, it's more easy to remember (The implementor of > >>> TestJmapCore forgot it...). And, last but not least, it seems best practice > >>> nowadays. There are lots of excludes for mac, they are also done by > >> @requires > >>> and not as a check using Platform at runtime. > >>> > >>>> Sorry, I don't have an answer to your VMProps evaluation question. > You > >>>> might need to ask a wider audience than just svc. > >>> Whom should I ask? Hotspot-dev? > >>> > >>> Best regards, > >>> Goetz. > >>> > >>> > >>>> > >>>> thanks, > >>>> > >>>> Chris > >>>> > >>>> On 6/20/18 6:49 AM, Lindenmaier, Goetz wrote: > >>>>> Hi, > >>>>> > >>>>> TestJmapCore is failing on aix because there jhsdb is not implemented. > >>>>> So far, such tests check at runtime Platform.shouldSAAttach() which is > >>>> missing > >>>>> in TestJmapCore. > >>>>> > >>>>> Instead, I implement @requires vm.hasSAandCanAttach. > >>>>> This makes jtreg skip the tests without compiling and starting them. > >>>>> > >>>>> I added the new property to all tests that check shouldSAAttach in jdk > >> and > >>>> hostpot test > >>>>> directories. > >>>>> This is the implementation of the new property, the rest is just > >> repetitive > >>>> checking it: > >>>>> http://cr.openjdk.java.net/~goetz/wr18/8205419- > >>>> requiresHasSA/01/test/jtreg-ext/requires/VMProps.java.udiff.html > >>>>> webrev: > >>>>> http://cr.openjdk.java.net/~goetz/wr18/8205419-requiresHasSA/01/ > >>>>> > >>>>> Is it correct to assume the VMProps is evaluated on the same machine > >> and > >>>> with the same > >>>>> user as used for executing the test? canPtraceAttachLinux directly > >>>> accesses the system. > >>>>> > >>>>> (Should I remove the checks for shouldSAAttach() from the changed > >> tests?) > >>>>> > >>>>> Best regards, > >>>>> Goetz. > >>>> > >>>> > >>> From goetz.lindenmaier at sap.com Mon Jun 25 21:04:14 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 25 Jun 2018 21:04:14 +0000 Subject: RFR(S/M): 8205419: [testbug] TestJmapCore failing without SA: introduce @requires vm.hasSA In-Reply-To: References: <112db96d59084bdfafba2174e121c409@sap.com> <431269fc-7174-3051-0a86-baa00032754a@oracle.com> <3712cc995426467fa67b65493d556298@sap.com> <486ee0f785144f55bd739cc41bcd7e2d@sap.com> Message-ID: <6af2ac5b2a9d4d179934eb2c10ef7cbf@sap.com> Hi, Thanks for reviewing! Yes, what Jini is saying is also what I found in the comments. Best regards, Goetz. > -----Original Message----- > From: Chris Plummer > Sent: Monday, June 25, 2018 7:02 PM > To: Lindenmaier, Goetz ; 'Jini George' > ; serviceability-dev (serviceability- > dev at openjdk.java.net) > Subject: Re: RFR(S/M): 8205419: [testbug] TestJmapCore failing without SA: > introduce @requires vm.hasSA > > Hi Goetz, > > The changes look good. Just one question though. In places where you have: > > ? 42? * @requires vm.hasSAandCanAttach & os.family != "mac" > > Do you know why these tests don't run on osx? I just wonder if it > related to attach not working unless root, and that is now covered by > hasSAandCanAttach(). If that is the case, the "mac" check can be removed. > > thanks, > > Chris > > On 6/22/18 1:20 AM, Lindenmaier, Goetz wrote: > > Hi, > > > > Ok, so I now added > > vm.hasSA and > > vm.hasSAandCanAttach > > I use the first in the JMapCore tests, and in the tests that don't check > shouldSAAttach() > > right at the beginning. > > Wherever the test is immediately ended after checking shouldSAAttach(), I > > removed shouldSAAttach(), and added vm.hasSAandCanAttach. The > others > > depend on ClhsdbLauncher. Just checking vm.hasSA is on the safe side > > if runOnCore() is changed. > > http://cr.openjdk.java.net/~goetz/wr18/8205419-requiresHasSA/02/ > > > > This also makes the implementation of VMProps and VMPlatform more > clear > > as requested by Chris. > > > > Best regards, > > Goetz. > > > > > >> -----Original Message----- > >> From: Jini George > >> Sent: Thursday, June 21, 2018 12:18 PM > >> To: Lindenmaier, Goetz ; 'Chris Plummer' > >> ; serviceability-dev (serviceability- > >> dev at openjdk.java.net) > >> Subject: Re: RFR(S/M): 8205419: [testbug] TestJmapCore failing without > SA: > >> introduce @requires vm.hasSA > >> > >> Hi Goetz, > >> > >> Tests like TestJmapCore which involve only corefile debugging by SA (and > >> not attaching to a live process) would only need to check for the > >> presence of SA, and would not need to check for ptrace related attach > >> permissions like what is done for Linux and OSX -- since super user > >> privileges are not required to debug corefiles. So a single @requires > >> catering to both these might not be desirable. > >> > >> Having said that, I realized that runOnCore() of ClhsdbLauncher > >> incorrectly checks for the ptrace related attach permissions. (Will file > >> an issue to correct this). > >> > >> An @requires instead of a shouldSAAttach() might help in avoiding fake > >> passes like those on OSX when not run as "root". > >> (https://bugs.openjdk.java.net/browse/JDK-8199700) > >> > >> Thanks, > >> Jini. > >> > >> > >> > >> > >> On 6/21/2018 1:52 AM, Lindenmaier, Goetz wrote: > >>> Hi Chris, > >>> > >>> Thanks for looking at this change. > >>> > >>>> Can't VMProps.vmHasSA() just return Platform.shouldSAAttach()? It > >> seems > >>>> you are unnecessarily replicating Platform.shouldSAAttach() logic. > >>> I think there are two things to distinguish: > >>> - platforms that don't implement SA > >>> - systems/setups where SA does not work. > >>> If both can be recognized when VMProps is evaluated, shouldSAAttach() > >>> is the one that is superfluous in the long run?! If not, shouldSAAttach > >>> should only return those where it is necessary to check in the test. > >>> But this is probably too much brains in this case ?? > >>> > >>>> If you are adding "@requires vm.hasSAandCanAttach" to a test, > shouldn't > >>>> you remove the test's Platform.shouldSAAttach() check? > >>> Well I asked in my mail whether I should do that. So I take this as a yes. > >>> But it depends on whether it must be evaluated in the test. > >>> > >>>> Is there a reason not to take the more direct and established approach > >>>> of just adding the Platform.shouldSAAttach() check to TestJmapCore? > It > >>>> would be a lot less disruptive.? I know the vm.hasSAandCanAttach > >>>> approach has advantages in not even attempting to compile and run > the > >>>> test, but I'm not so sure those advantages outweigh the simplicity of > >>>> just adding a Platform.shouldSAAttach() check to one test. I can go > >>>> either way here. Just wondering if there are additional reasons I might > >>>> not be seeing. > >>> I like the @requires much more. It tells right where the test is described > >>> that it does not run with SA. The other is quite hidden in the test, e.g., in > >>> helper class ClhsdbLauncher.java. > >>> Also, it's more easy to remember (The implementor of > >>> TestJmapCore forgot it...). And, last but not least, it seems best practice > >>> nowadays. There are lots of excludes for mac, they are also done by > >> @requires > >>> and not as a check using Platform at runtime. > >>> > >>>> Sorry, I don't have an answer to your VMProps evaluation question. > You > >>>> might need to ask a wider audience than just svc. > >>> Whom should I ask? Hotspot-dev? > >>> > >>> Best regards, > >>> Goetz. > >>> > >>> > >>>> thanks, > >>>> > >>>> Chris > >>>> > >>>> On 6/20/18 6:49 AM, Lindenmaier, Goetz wrote: > >>>>> Hi, > >>>>> > >>>>> TestJmapCore is failing on aix because there jhsdb is not implemented. > >>>>> So far, such tests check at runtime Platform.shouldSAAttach() which is > >>>> missing > >>>>> in TestJmapCore. > >>>>> > >>>>> Instead, I implement @requires vm.hasSAandCanAttach. > >>>>> This makes jtreg skip the tests without compiling and starting them. > >>>>> > >>>>> I added the new property to all tests that check shouldSAAttach in jdk > >> and > >>>> hostpot test > >>>>> directories. > >>>>> This is the implementation of the new property, the rest is just > >> repetitive > >>>> checking it: > >>>>> http://cr.openjdk.java.net/~goetz/wr18/8205419- > >>>> requiresHasSA/01/test/jtreg-ext/requires/VMProps.java.udiff.html > >>>>> webrev: > >>>>> http://cr.openjdk.java.net/~goetz/wr18/8205419-requiresHasSA/01/ > >>>>> > >>>>> Is it correct to assume the VMProps is evaluated on the same machine > >> and > >>>> with the same > >>>>> user as used for executing the test? canPtraceAttachLinux directly > >>>> accesses the system. > >>>>> (Should I remove the checks for shouldSAAttach() from the changed > >> tests?) > >>>>> Best regards, > >>>>> Goetz. > >>>> > From daniel.daugherty at oracle.com Tue Jun 26 01:32:30 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 25 Jun 2018 21:32:30 -0400 Subject: RFR(XS): 8205195 NestedThreadsListHandleInErrorHandlingTest fails because hs_err doesn't contain _nested_thread_list_max In-Reply-To: <476d805c-2bfc-fa31-3f4e-e1e44129d9b2@oracle.com> References: <126cc7a5-a92b-a22c-6d28-e35779ee69ef@oracle.com> <476d805c-2bfc-fa31-3f4e-e1e44129d9b2@oracle.com> Message-ID: <6dbfbca4-e809-25c8-4600-d376d853812b@oracle.com> David, Thanks for your eagle eyes! More below... On 6/25/18 9:27 AM, Daniel D. Daugherty wrote: > On 6/25/18 1:57 AM, David Holmes wrote: >> Hi Dan, >> >> On 21/06/2018 10:18 AM, Daniel D. Daugherty wrote: >>> Greetings, >>> >>> I have a fix for a recent (very rare) Thread-SMR related test failure. >>> >>> Since the fix is related to the ErrorHandling tests and affects >>> hs_err_pid >>> file generation, this code review is being sent to both the Runtime and >>> the Serviceability teams. Please make sure you reply-all to any >>> responses >>> so we have complete review threads on both aliases. >>> >>> Bug URL: https://bugs.openjdk.java.net/browse/JDK-8205195 >>> >>> Webrev URL: >>> http://cr.openjdk.java.net/~dcubed/8205195-webrev/0-for-jdk-jdk/ >>> >>> The bug itself contains analysis about the root cause of the bug and >>> the comment updates to the code describes the no win scenario that the >>> hs_err_pid file generation code is in. Of course, I also have a comment >>> where I was able to harden the ErrorHandling tests. I did manage to >>> resist the urge to mention the "Kobiyashi Maru" [1] in the new >>> comments. >>> >>> Testing: Mach5 >>> builds-tier1,jdk-tier1,jdk-tier2,hs-tier1,hs-tier2,hs-tier3 >>> ??? ? ? ? on the usual Oracle platforms. >>> >>> Thanks, in advance, for any comments, questions or suggestions. >> >> I don't quite follow the fix. > > I think this comment explains what the fix is _trying_ to do: > > 1702?? // We grab Threads_lock to keep ThreadsSMRSupport::print_info_on() > 1703?? // from racing with Threads::add() or Threads::remove() as we > 1704?? // generate the hs_err_pid file. This makes our ErrorHandling > tests > 1705?? // more stable. > > >> Won't you self-deadlock on acquiring the Threads_lock in the >> secondary error handler test, due to the recursive call to >> controlled_crash ? > > I missed that possibility, but now I'm puzzled why my testing didn't > reveal this situation. We have a test for secondary error handling > and it should have self-deadlocked (and failed). I'll investigate. The ErrorHandling tests don't run with 'release' bits so no self-deadlock. The 'fastdebug' and 'slowdebug' versions will fail an assertion that gets hidden by the fact that it is a secondary failure mode. I've filed the following bug: JDK-8205648 fix for 8205195 breaks secondary error handling https://bugs.openjdk.java.net/browse/JDK-8205648 Dan > > Dan > > >> >> David >> >>> Dan >>> >>> [1] https://www.urbandictionary.com/define.php?term=Kobayashi%20Maru >>> > > From david.holmes at oracle.com Tue Jun 26 05:41:32 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 26 Jun 2018 15:41:32 +1000 Subject: RFR(XS): 8205195 NestedThreadsListHandleInErrorHandlingTest fails because hs_err doesn't contain _nested_thread_list_max In-Reply-To: <6dbfbca4-e809-25c8-4600-d376d853812b@oracle.com> References: <126cc7a5-a92b-a22c-6d28-e35779ee69ef@oracle.com> <476d805c-2bfc-fa31-3f4e-e1e44129d9b2@oracle.com> <6dbfbca4-e809-25c8-4600-d376d853812b@oracle.com> Message-ID: <1e37f0e5-a5cd-9c3c-168d-c617b3b1ff82@oracle.com> On 26/06/2018 11:32 AM, Daniel D. Daugherty wrote: > David, > > Thanks for your eagle eyes! More below... > > > On 6/25/18 9:27 AM, Daniel D. Daugherty wrote: >> On 6/25/18 1:57 AM, David Holmes wrote: >>> Hi Dan, >>> >>> On 21/06/2018 10:18 AM, Daniel D. Daugherty wrote: >>>> Greetings, >>>> >>>> I have a fix for a recent (very rare) Thread-SMR related test failure. >>>> >>>> Since the fix is related to the ErrorHandling tests and affects >>>> hs_err_pid >>>> file generation, this code review is being sent to both the Runtime and >>>> the Serviceability teams. Please make sure you reply-all to any >>>> responses >>>> so we have complete review threads on both aliases. >>>> >>>> Bug URL: https://bugs.openjdk.java.net/browse/JDK-8205195 >>>> >>>> Webrev URL: >>>> http://cr.openjdk.java.net/~dcubed/8205195-webrev/0-for-jdk-jdk/ >>>> >>>> The bug itself contains analysis about the root cause of the bug and >>>> the comment updates to the code describes the no win scenario that the >>>> hs_err_pid file generation code is in. Of course, I also have a comment >>>> where I was able to harden the ErrorHandling tests. I did manage to >>>> resist the urge to mention the "Kobiyashi Maru" [1] in the new >>>> comments. >>>> >>>> Testing: Mach5 >>>> builds-tier1,jdk-tier1,jdk-tier2,hs-tier1,hs-tier2,hs-tier3 >>>> ??? ? ? ? on the usual Oracle platforms. >>>> >>>> Thanks, in advance, for any comments, questions or suggestions. >>> >>> I don't quite follow the fix. >> >> I think this comment explains what the fix is _trying_ to do: >> >> 1702?? // We grab Threads_lock to keep ThreadsSMRSupport::print_info_on() >> 1703?? // from racing with Threads::add() or Threads::remove() as we >> 1704?? // generate the hs_err_pid file. This makes our ErrorHandling >> tests >> 1705?? // more stable. >> >> >>> Won't you self-deadlock on acquiring the Threads_lock in the >>> secondary error handler test, due to the recursive call to >>> controlled_crash ? >> >> I missed that possibility, but now I'm puzzled why my testing didn't >> reveal this situation. We have a test for secondary error handling >> and it should have self-deadlocked (and failed). I'll investigate. > > The ErrorHandling tests don't run with 'release' bits so no self-deadlock. > The 'fastdebug' and 'slowdebug' versions will fail an assertion that gets > hidden by the fact that it is a secondary failure mode. Ah - of course. > I've filed the following bug: > > JDK-8205648 fix for 8205195 breaks secondary error handling > https://bugs.openjdk.java.net/browse/JDK-8205648 Thanks, David > Dan > > >> >> Dan >> >> >>> >>> David >>> >>>> Dan >>>> >>>> [1] https://www.urbandictionary.com/define.php?term=Kobayashi%20Maru >>>> >> >> > From david.holmes at oracle.com Tue Jun 26 05:45:23 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 26 Jun 2018 15:45:23 +1000 Subject: RFR: JDK-8205508: hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java fails with Prompt is not received during 300200 milliseconds. In-Reply-To: <5B31342C.4080104@oracle.com> References: <5B312469.7060206@oracle.com> <890d5683-63a9-18c7-d108-00b5b374ca0b@oracle.com> <5B31342C.4080104@oracle.com> Message-ID: <3c44a61e-1ee7-b549-72d9-253ed53d9db1@oracle.com> Hi Gary, On 26/06/2018 4:27 AM, Gary Adams wrote: > The first time I looked into problems with exclude001 test, > we discovered a large number of new packages in the jdk.internal > classes that were introduced in jdk9. The test needed to add excludes for > any of the jdk.* methods or it could not finish in time. > > As a follow up I'll try a test run with unlimited time and no methods > excluded to get > a specific count of methods that are being processed. Over time new > features > have been added, e.g. string concatenation optimizations, lambda functions, > etc., etc., etc. For a test that does method tracing, each new method > adds to the > collective time. If you can? not reduce the number of methods called, > then the time > for the test needs to be increased. That sounds quite reasonable. I'm just wondering how the "-waittime=7" interacts with the jtreg timeout handling? Thanks, David > ... > > On 6/25/18, 2:11 PM, Chris Plummer wrote: >> I'm also wondering how fast this test runs on other platforms and when >> passing on solaris-sparc. 5 minutes already seems like a long time for >> this test. There could be an underlying issue that needs to be addressed. >> >> Chris >> >> On 6/25/18 11:00 AM, serguei.spitsyn at oracle.com wrote: >>> Hi Gary, >>> >>> It looks Okay. >>> But I'm curious when this started failing and what triggered it to fail? >>> >>> Thanks, >>> Serguei >>> >>> >>> On 6/25/18 10:20, Gary Adams wrote: >>>> The exclude001 test times out on solaris sparc debug builds. >>>> >>>> Basically, this test is all about tracing method calls and introduces >>>> exclude filters to reduce the callbacks to a select set of packages. >>>> The time spent tracing/filtering method callbacks is purely a function >>>> of the number of methods that are processed. On this particularly slow >>>> target platform, more time is needed before issuing a timeout. >>>> >>>> ? Issue: https://bugs.openjdk.java.net/browse/JDK-8205508 >>>> >>>> Proposed fix: >>>> diff --git >>>> a/test/hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.javab/test/hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java >>>> >>>> --- >>>> a/test/hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java >>>> +++ >>>> b/test/hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java >>>> @@ -63,7 +63,7 @@ >>>> ??* nsk.jdb.exclude.exclude001.exclude001a >>>> ??* @run main/othervm PropertyResolvingWrapper >>>> nsk.jdb.exclude.exclude001.exclude001 >>>> ??* -arch=${os.family}-${os.simpleArch} >>>> - * -waittime=5 >>>> + * -waittime=7 >>>> ??* -debugee.vmkind=java >>>> ??* -transport.address=dynamic >>>> ??* -jdb=${test.jdk}/bin/jdb >>>> >>>> >>> >> > From jini.george at oracle.com Tue Jun 26 07:25:22 2018 From: jini.george at oracle.com (Jini George) Date: Tue, 26 Jun 2018 12:55:22 +0530 Subject: RFR: JDK-8189429: SA: MacOSX: Replace the deprecated PT_ATTACH with PT_ATTACHEXC In-Reply-To: <5ba6916b-35bf-617e-fff8-93f096162047@oracle.com> References: <7abea33e-7fb0-fbe0-d463-8999ccdf54da@oracle.com> <0c93044e-7aaf-179b-c80c-45ff4d7e2712@oracle.com> <5ba6916b-35bf-617e-fff8-93f096162047@oracle.com> Message-ID: <8e5f1747-3d1f-7381-549c-965ee3d5ef08@oracle.com> Ping - Gentle reminder ! Thanks, Jini. On 6/25/2018 3:41 PM, Jini George wrote: > Thank you, Sharath. May I have a Reviewer to take a look at the > MacosxDebuggerLocal code? > > Thanks, > Jini. > > On 6/25/2018 1:52 PM, Sharath Ballal wrote: >> Hi Jini, >> >> Changes in MacosxDebuggerLocal.m looks good. >> >> Thanks, >> Sharath >> >> >> -----Original Message----- >> From: Jini George >> Sent: Sunday, June 24, 2018 11:07 PM >> To: Erik Joelsson; serviceability-dev at openjdk.java.net; >> build-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net >> Subject: Re: RFR: JDK-8189429: SA: MacOSX: Replace the deprecated >> PT_ATTACH with PT_ATTACHEXC >> >> Hi Erik, >> >> Thank you very much for looking into this. I have addressed your >> comments. The latest webrev is at: >> >> http://cr.openjdk.java.net/~jgeorge/8189429/webrev.06/ >> >> Thank you, >> Jini. >> >> >> On 6/23/2018 3:31 AM, Erik Joelsson wrote: >>> Hello Jini, >>> >>> In general this looks pretty good, but it's also breaking some new >>> ground as it's adding generation of native source in the java gensrc >>> step. Mixing native code with the java code that the genrcs targets >>> and gensrc output directories are meant for seems ok for now, but may >>> cause trouble in the future. I'm going to accept it for now though. >>> >>> In Gensrc-jdk.hotspot.agent.gmk: Please put the new macosx stuff in >>> its own section (as delimited by the 80x # lines) and put that whole >>> thing inside a conditional for macosx. Also please break line 47 (for >>> recipe lines, indent with tab and 4 additional spaces for >>> continuation [1]). >>> >>> In Lib-jdk.hotspot.agent.gmk: I believe adding extra src should be >>> enough as that will implicitly add the same directories as header dirs >>> by default. At least that's the intention. >>> >>> /Erik >>> >>> [1] http://openjdk.java.net/groups/build/doc/code-conventions.html >>> >>> On 2018-06-22 11:11, Jini George wrote: >>>> Hi all, >>>> >>>> [Including build-dev also since this includes build related changes]. >>>> >>>> Requesting reviews for: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8189429 (SA: MacOSX: Replace >>>> the deprecated PT_ATTACH with PT_ATTACHEXC) >>>> >>>> Webrev: http://cr.openjdk.java.net/~jgeorge/8189429/webrev.04/ >>>> >>>> This is the follow-up solution for the temporary restoration of >>>> PT_ATTACH to fix https://bugs.openjdk.java.net/browse/JDK-8184042 >>>> (several serviceability/sa tests timed out on MacOS X), which was >>>> done in Oct 2017. The mails related to this are at: >>>> >>>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2017-August >>>> /021702.html >>>> >>>> >>>> Changes as compared to the patch sent last year >>>> (http://cr.openjdk.java.net/~jgeorge/8184042/webrev.00/): >>>> >>>> * Addressed the review comments which were provided by Poonam, Dan, >>>> Dmitry. >>>> * A major change as compared to what was done last year is that the >>>> MIG generated files have been included as a part of the JDK build >>>> process. >>>> * The test case which was provided in the patch last year is no >>>> longer required since we have ClhsdbAttach.java testing the same >>>> functionality as a part of the SA testsuite now. >>>> * Other than that, some minor improvements have been done wrt error >>>> handling. >>>> >>>> The steps for the proposed solution have been provided in JBS. >>>> >>>> Testing: ALL the SA tests pass on MacOSX and the other Mach5 platforms. >>>> >>>> Thanks to Sharath, Robin, Gerard and Dan for looking into the changes >>>> and providing valuable comments. >>>> >>>> Thanks, >>>> Jini. >>> From amy.lu at oracle.com Tue Jun 26 09:20:18 2018 From: amy.lu at oracle.com (Amy Lu) Date: Tue, 26 Jun 2018 17:20:18 +0800 Subject: [11] RFR 8205673: Problem list RmiRegistrySslTest.java and RmiSslBootstrapTest.sh Message-ID: <6bdad50f-4ac0-3595-d483-681b98fe4911@oracle.com> sun/management/jmxremote/bootstrap/RmiSslBootstrapTest.sh sun/management/jmxremote/bootstrap/RmiRegistrySslTest.java Above two tests have been failing on all platforms (JDK-8205653) since JDK-8196584. Please review the patch to add these tests to ProblemList.txt until a full fix is developed. Thanks, Amy --- old/test/jdk/ProblemList.txt??? 2018-06-26 14:44:21.000000000 +0800 +++ new/test/jdk/ProblemList.txt??? 2018-06-26 14:44:21.000000000 +0800 @@ -542,6 +542,9 @@ ?com/sun/management/OperatingSystemMXBean/GetProcessCpuLoad.java 8030957 aix-all ?com/sun/management/OperatingSystemMXBean/GetSystemCpuLoad.java 8030957 aix-all +sun/management/jmxremote/bootstrap/RmiSslBootstrapTest.sh 8205653 generic-all +sun/management/jmxremote/bootstrap/RmiRegistrySslTest.java 8205653 generic-all + ?############################################################################ ?# jdk_jmx From gary.adams at oracle.com Tue Jun 26 11:15:06 2018 From: gary.adams at oracle.com (Gary Adams) Date: Tue, 26 Jun 2018 07:15:06 -0400 Subject: RFR: JDK-8205508: hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java fails with Prompt is not received during 300200 milliseconds. In-Reply-To: <3c44a61e-1ee7-b549-72d9-253ed53d9db1@oracle.com> References: <5B312469.7060206@oracle.com> <890d5683-63a9-18c7-d108-00b5b374ca0b@oracle.com> <5B31342C.4080104@oracle.com> <3c44a61e-1ee7-b549-72d9-253ed53d9db1@oracle.com> Message-ID: <5B32203A.1090603@oracle.com> For the vmTestbase tests recently moved to the open repos, see test/hotspot/jtreg/vmTestbase/nsk/share/TimeoutHandler.java. It uses a simple wrapper around a test to ensure a single test completes within a specific time window. The vmTestbase tests were only minimally changed so they could be run with the jtreg test harness, but were not fully ported to rely on features in the jtreg harness itself. /** * Perform test execution in separate thread and wait for * thread finishes or timeout exceeds. */ public void runTest(Thread testThread) { long millisec = waitTime * 60 * 1000; testThread.start(); try { testThread.join(millisec); } catch (InterruptedException ex) { throw new Failure(ex); } } For the jtreg TimeoutHandlers, see /build/images/jtreg/doc/jtreg/usage.txt. ... Timeout Options These options control the behavior when tests run longer than their specified timeout value. -th: | -timeoutHandler: Specifies the class to handle timeouts. The class must extend com.sun.javatest.regtest.TimeoutHandler. E.g. -th:MyHandler -thd: | -timeoutHandlerDir: Specifies the pathname of a directory or .jar file in which the timeout handler class is located. The given pathname is simply appended to the CLASSPATH used for the tests, thus care should be taken when naming an timeout handler not to collide with the names of classes internal to the JavaTest harness or the JRE, e.g., put the timeout handler class in its own named package. -thtimeout:<#seconds> | -timeoutHandlerTimeout:<#seconds> Specifies execution time limitation for the timeout handler. If the timeout handler does not finish its actions within the specified period of time, it will be interrupted. Non-positive values mean no limitation. The default value is 5 minutes (300 seconds). -timeout: | -timeoutFactor: A scaling factor to extend the default timeout of all tests. Typically used when running tests on slow systems or systems with slow file systems. -tl:<#seconds> | -timelimit:<#seconds> Do not run tests which specify a timeout longer than a given value. The comparison is done against any values specified in the test, before any timeout factor is applied. Which would you prefer at this point in time : - increase the timeout so it can run on the slower platforms - problem list the test so it is bypassed completely On 6/26/18, 1:45 AM, David Holmes wrote: > Hi Gary, > > On 26/06/2018 4:27 AM, Gary Adams wrote: >> The first time I looked into problems with exclude001 test, >> we discovered a large number of new packages in the jdk.internal >> classes that were introduced in jdk9. The test needed to add excludes >> for >> any of the jdk.* methods or it could not finish in time. >> >> As a follow up I'll try a test run with unlimited time and no methods >> excluded to get >> a specific count of methods that are being processed. Over time new >> features >> have been added, e.g. string concatenation optimizations, lambda >> functions, >> etc., etc., etc. For a test that does method tracing, each new method >> adds to the >> collective time. If you can not reduce the number of methods called, >> then the time >> for the test needs to be increased. > > That sounds quite reasonable. I'm just wondering how the "-waittime=7" > interacts with the jtreg timeout handling? > > Thanks, > David > >> ... >> >> On 6/25/18, 2:11 PM, Chris Plummer wrote: >>> I'm also wondering how fast this test runs on other platforms and >>> when passing on solaris-sparc. 5 minutes already seems like a long >>> time for this test. There could be an underlying issue that needs to >>> be addressed. >>> >>> Chris >>> >>> On 6/25/18 11:00 AM, serguei.spitsyn at oracle.com wrote: >>>> Hi Gary, >>>> >>>> It looks Okay. >>>> But I'm curious when this started failing and what triggered it to >>>> fail? >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 6/25/18 10:20, Gary Adams wrote: >>>>> The exclude001 test times out on solaris sparc debug builds. >>>>> >>>>> Basically, this test is all about tracing method calls and introduces >>>>> exclude filters to reduce the callbacks to a select set of packages. >>>>> The time spent tracing/filtering method callbacks is purely a >>>>> function >>>>> of the number of methods that are processed. On this particularly >>>>> slow >>>>> target platform, more time is needed before issuing a timeout. >>>>> >>>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8205508 >>>>> >>>>> Proposed fix: >>>>> diff --git >>>>> a/test/hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.javab/test/hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java >>>>> >>>>> --- >>>>> a/test/hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java >>>>> +++ >>>>> b/test/hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java >>>>> @@ -63,7 +63,7 @@ >>>>> * nsk.jdb.exclude.exclude001.exclude001a >>>>> * @run main/othervm PropertyResolvingWrapper >>>>> nsk.jdb.exclude.exclude001.exclude001 >>>>> * -arch=${os.family}-${os.simpleArch} >>>>> - * -waittime=5 >>>>> + * -waittime=7 >>>>> * -debugee.vmkind=java >>>>> * -transport.address=dynamic >>>>> * -jdb=${test.jdk}/bin/jdb >>>>> >>>>> >>>> >>> >> From david.holmes at oracle.com Tue Jun 26 12:58:19 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 26 Jun 2018 22:58:19 +1000 Subject: RFR: JDK-8205508: hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java fails with Prompt is not received during 300200 milliseconds. In-Reply-To: <5B32203A.1090603@oracle.com> References: <5B312469.7060206@oracle.com> <890d5683-63a9-18c7-d108-00b5b374ca0b@oracle.com> <5B31342C.4080104@oracle.com> <3c44a61e-1ee7-b549-72d9-253ed53d9db1@oracle.com> <5B32203A.1090603@oracle.com> Message-ID: On 26/06/2018 9:15 PM, Gary Adams wrote: > For the vmTestbase tests recently moved to the open repos, > see test/hotspot/jtreg/vmTestbase/nsk/share/TimeoutHandler.java. > It uses a simple wrapper around a test to ensure a single test completes > within a specific time window. The vmTestbase tests were only minimally > changed so they could be run with the jtreg test harness, but were not > fully ported to rely on features in the jtreg harness itself. > > ??? /** > ???? * Perform test execution in separate thread and wait for > ???? * thread finishes or timeout exceeds. > ???? */ > ??? public void runTest(Thread testThread) { > ??????? long millisec = waitTime * 60 * 1000; > ??????? testThread.start(); > ??????? try { > ??????????? testThread.join(millisec); > ??????? } catch (InterruptedException ex) { > ??????????? throw new Failure(ex); > ??????? } > ??? } > > For the jtreg TimeoutHandlers, > see /build/images/jtreg/doc/jtreg/usage.txt. > > ... > Timeout Options > ??????????????? These options control the behavior when tests run > longer than > ??????????????? their specified timeout value. > ??? -th: | -timeoutHandler: > ??????????????????? Specifies the class to handle timeouts. The class must > ??????????????????? extend com.sun.javatest.regtest.TimeoutHandler. E.g. > ??????????????????? -th:MyHandler > ??? -thd: | -timeoutHandlerDir: > ??????????????????? Specifies the pathname of a directory or .jar file > in which > ??????????????????? the timeout handler class is located. The given > pathname is > ??????????????????? simply appended to the CLASSPATH used for the > tests, thus > ??????????????????? care should be taken when naming an timeout handler > not to > ??????????????????? collide with the names of classes internal to the > JavaTest > ??????????????????? harness or the JRE, e.g., put the timeout handler > class in > ??????????????????? its own named package. > ??? -thtimeout:<#seconds> | -timeoutHandlerTimeout:<#seconds> > ??????????????????? Specifies execution time limitation for the timeout > handler. > ??????????????????? If the timeout handler does not finish its actions > within > ??????????????????? the specified period of time, it will be interrupted. > ??????????????????? Non-positive values mean no limitation. The default > value is > ??????????????????? 5 minutes (300 seconds). > ??? -timeout: | -timeoutFactor: > ??????????????????? A scaling factor to extend the default timeout of > all tests. > ??????????????????? Typically used when running tests on slow systems > or systems > ??????????????????? with slow file systems. > ??? -tl:<#seconds> | -timelimit:<#seconds> > ??????????????????? Do not run tests which specify a timeout longer > than a given > ??????????????????? value. The comparison is done against any values > specified > ??????????????????? in the test, before any timeout factor is applied. > > Which would you prefer at this point in time : > ? - increase the timeout so it can run on the slower platforms > ? - problem list the test so it is bypassed completely I simply wanted to understand how the waitTime related to the jtreg timeout mechanism. There's no point, afterall, in adding an extra minute or two internally to the test if jtreg would time it out before then. David ----- > On 6/26/18, 1:45 AM, David Holmes wrote: >> Hi Gary, >> >> On 26/06/2018 4:27 AM, Gary Adams wrote: >>> The first time I looked into problems with exclude001 test, >>> we discovered a large number of new packages in the jdk.internal >>> classes that were introduced in jdk9. The test needed to add excludes >>> for >>> any of the jdk.* methods or it could not finish in time. >>> >>> As a follow up I'll try a test run with unlimited time and no methods >>> excluded to get >>> a specific count of methods that are being processed. Over time new >>> features >>> have been added, e.g. string concatenation optimizations, lambda >>> functions, >>> etc., etc., etc. For a test that does method tracing, each new method >>> adds to the >>> collective time. If you can? not reduce the number of methods called, >>> then the time >>> for the test needs to be increased. >> >> That sounds quite reasonable. I'm just wondering how the "-waittime=7" >> interacts with the jtreg timeout handling? >> >> Thanks, >> David >> >>> ... >>> >>> On 6/25/18, 2:11 PM, Chris Plummer wrote: >>>> I'm also wondering how fast this test runs on other platforms and >>>> when passing on solaris-sparc. 5 minutes already seems like a long >>>> time for this test. There could be an underlying issue that needs to >>>> be addressed. >>>> >>>> Chris >>>> >>>> On 6/25/18 11:00 AM, serguei.spitsyn at oracle.com wrote: >>>>> Hi Gary, >>>>> >>>>> It looks Okay. >>>>> But I'm curious when this started failing and what triggered it to >>>>> fail? >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>> On 6/25/18 10:20, Gary Adams wrote: >>>>>> The exclude001 test times out on solaris sparc debug builds. >>>>>> >>>>>> Basically, this test is all about tracing method calls and introduces >>>>>> exclude filters to reduce the callbacks to a select set of packages. >>>>>> The time spent tracing/filtering method callbacks is purely a >>>>>> function >>>>>> of the number of methods that are processed. On this particularly >>>>>> slow >>>>>> target platform, more time is needed before issuing a timeout. >>>>>> >>>>>> ? Issue: https://bugs.openjdk.java.net/browse/JDK-8205508 >>>>>> >>>>>> Proposed fix: >>>>>> diff --git >>>>>> a/test/hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.javab/test/hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java >>>>>> >>>>>> --- >>>>>> a/test/hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java >>>>>> >>>>>> +++ >>>>>> b/test/hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java >>>>>> >>>>>> @@ -63,7 +63,7 @@ >>>>>> ? * nsk.jdb.exclude.exclude001.exclude001a >>>>>> ? * @run main/othervm PropertyResolvingWrapper >>>>>> nsk.jdb.exclude.exclude001.exclude001 >>>>>> ? * -arch=${os.family}-${os.simpleArch} >>>>>> - * -waittime=5 >>>>>> + * -waittime=7 >>>>>> ? * -debugee.vmkind=java >>>>>> ? * -transport.address=dynamic >>>>>> ? * -jdb=${test.jdk}/bin/jdb >>>>>> >>>>>> >>>>> >>>> >>> > From daniel.daugherty at oracle.com Tue Jun 26 13:06:27 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 26 Jun 2018 09:06:27 -0400 Subject: RFR(XXXS): 8205648 fix for 8205195 breaks secondary error handling Message-ID: <4f5a5d7d-f4e4-657e-dbc2-d17b06c1de91@oracle.com> Greetings, For this one liner review, I'd love to hear from David H., Thomas St?fe, and Serguei Spitsyn. David Holmes spotted a locking problem with my fix for the following bug: ??? JDK-8205195 NestedThreadsListHandleInErrorHandlingTest fails because ??????????????? hs_err doesn't contain _nested_thread_list_max ??? https://bugs.openjdk.java.net/browse/JDK-8205195 ??? Webrev URL: http://cr.openjdk.java.net/~dcubed/8205195-webrev/0-for-jdk-jdk/ The fix for the issue is a one liner that only grabs the Threads_lock when the caller does not already own it: $ hg diff diff -r 5698cf4e50f1 src/hotspot/share/utilities/vmError.cpp --- a/src/hotspot/share/utilities/vmError.cpp Fri Jun 22 12:15:16 2018 -0400 +++ b/src/hotspot/share/utilities/vmError.cpp Mon Jun 25 21:20:52 2018 -0400 @@ -1703,7 +1703,7 @@ ?? // from racing with Threads::add() or Threads::remove() as we ?? // generate the hs_err_pid file. This makes our ErrorHandling tests ?? // more stable. - MutexLockerEx ml(Threads_lock, Mutex::_no_safepoint_check_flag); + MutexLockerEx ml(Threads_lock->owned_by_self() ? NULL : Threads_lock, Mutex::_no_safepoint_check_flag); ?? switch (how) { ???? case 1: vmassert(str == NULL, "expected null"); break; Figuring out a way to detect the failure mode in order to test the fix was much more involved than the fix itself. Gory details are in the bug: ??? JDK-8205648 fix for 8205195 breaks secondary error handling ??? https://bugs.openjdk.java.net/browse/JDK-8205648 No webrev since the one liner diff is shown above. This fix was tested with the test/hotspot/jtreg/runtime/ErrorHandling tests on Linux-X64 and with a special version of ErrorHandling/SecondaryErrorTest.java that is documented in JDK-8205648. Thanks, in advance, for any comments, questions or suggestions. Dan From Xuelei.Fan at Oracle.Com Tue Jun 26 13:15:44 2018 From: Xuelei.Fan at Oracle.Com (Xuelei Fan) Date: Tue, 26 Jun 2018 06:15:44 -0700 Subject: [11] RFR 8205673: Problem list RmiRegistrySslTest.java and RmiSslBootstrapTest.sh In-Reply-To: <6bdad50f-4ac0-3595-d483-681b98fe4911@oracle.com> References: <6bdad50f-4ac0-3595-d483-681b98fe4911@oracle.com> Message-ID: Looks fine to me. Thanks, Xuelei > On Jun 26, 2018, at 2:20 AM, Amy Lu wrote: > > sun/management/jmxremote/bootstrap/RmiSslBootstrapTest.sh > sun/management/jmxremote/bootstrap/RmiRegistrySslTest.java > > Above two tests have been failing on all platforms (JDK-8205653) since JDK-8196584. > > Please review the patch to add these tests to ProblemList.txt until a full fix is developed. > > Thanks, > Amy > > --- old/test/jdk/ProblemList.txt 2018-06-26 14:44:21.000000000 +0800 > +++ new/test/jdk/ProblemList.txt 2018-06-26 14:44:21.000000000 +0800 > @@ -542,6 +542,9 @@ > com/sun/management/OperatingSystemMXBean/GetProcessCpuLoad.java 8030957 aix-all > com/sun/management/OperatingSystemMXBean/GetSystemCpuLoad.java 8030957 aix-all > > +sun/management/jmxremote/bootstrap/RmiSslBootstrapTest.sh 8205653 generic-all > +sun/management/jmxremote/bootstrap/RmiRegistrySslTest.java 8205653 generic-all > + > ############################################################################ > > # jdk_jmx > From gary.adams at oracle.com Tue Jun 26 13:59:57 2018 From: gary.adams at oracle.com (Gary Adams) Date: Tue, 26 Jun 2018 09:59:57 -0400 Subject: RFR: JDK-8205508: hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java fails with Prompt is not received during 300200 milliseconds. In-Reply-To: References: <5B312469.7060206@oracle.com> <890d5683-63a9-18c7-d108-00b5b374ca0b@oracle.com> <5B31342C.4080104@oracle.com> <3c44a61e-1ee7-b549-72d9-253ed53d9db1@oracle.com> <5B32203A.1090603@oracle.com> Message-ID: <5B3246DD.9030407@oracle.com> As far as I can tell, the default jtreg harness timeout is 5 minutes and the RunTests.gmk or TestCommon.gmk includes a default time factor 4. So the jtreg harness/invoker is set up to timeout after 20 minutes, and the vmTestbase wrapper is set to 5 minutes before it times out internally. The failure we see in JDK-8205508 is hitting the internal timeout of the vmTestbase wrapper, not the external jtreg harness. On 6/26/18, 8:58 AM, David Holmes wrote: > On 26/06/2018 9:15 PM, Gary Adams wrote: >> For the vmTestbase tests recently moved to the open repos, >> see test/hotspot/jtreg/vmTestbase/nsk/share/TimeoutHandler.java. >> It uses a simple wrapper around a test to ensure a single test completes >> within a specific time window. The vmTestbase tests were only minimally >> changed so they could be run with the jtreg test harness, but were not >> fully ported to rely on features in the jtreg harness itself. >> >> /** >> * Perform test execution in separate thread and wait for >> * thread finishes or timeout exceeds. >> */ >> public void runTest(Thread testThread) { >> long millisec = waitTime * 60 * 1000; >> testThread.start(); >> try { >> testThread.join(millisec); >> } catch (InterruptedException ex) { >> throw new Failure(ex); >> } >> } >> >> For the jtreg TimeoutHandlers, >> see /build/images/jtreg/doc/jtreg/usage.txt. >> >> ... >> Timeout Options >> These options control the behavior when tests run >> longer than >> their specified timeout value. >> -th: | -timeoutHandler: >> Specifies the class to handle timeouts. The >> class must >> extend com.sun.javatest.regtest.TimeoutHandler. >> E.g. >> -th:MyHandler >> -thd: | -timeoutHandlerDir: >> Specifies the pathname of a directory or .jar >> file in which >> the timeout handler class is located. The given >> pathname is >> simply appended to the CLASSPATH used for the >> tests, thus >> care should be taken when naming an timeout >> handler not to >> collide with the names of classes internal to >> the JavaTest >> harness or the JRE, e.g., put the timeout >> handler class in >> its own named package. >> -thtimeout:<#seconds> | -timeoutHandlerTimeout:<#seconds> >> Specifies execution time limitation for the >> timeout handler. >> If the timeout handler does not finish its >> actions within >> the specified period of time, it will be >> interrupted. >> Non-positive values mean no limitation. The >> default value is >> 5 minutes (300 seconds). >> -timeout: | -timeoutFactor: >> A scaling factor to extend the default timeout >> of all tests. >> Typically used when running tests on slow >> systems or systems >> with slow file systems. >> -tl:<#seconds> | -timelimit:<#seconds> >> Do not run tests which specify a timeout longer >> than a given >> value. The comparison is done against any values >> specified >> in the test, before any timeout factor is applied. >> >> Which would you prefer at this point in time : >> - increase the timeout so it can run on the slower platforms >> - problem list the test so it is bypassed completely > > I simply wanted to understand how the waitTime related to the jtreg > timeout mechanism. There's no point, afterall, in adding an extra > minute or two internally to the test if jtreg would time it out before > then. > > David > ----- > > >> On 6/26/18, 1:45 AM, David Holmes wrote: >>> Hi Gary, >>> >>> On 26/06/2018 4:27 AM, Gary Adams wrote: >>>> The first time I looked into problems with exclude001 test, >>>> we discovered a large number of new packages in the jdk.internal >>>> classes that were introduced in jdk9. The test needed to add >>>> excludes for >>>> any of the jdk.* methods or it could not finish in time. >>>> >>>> As a follow up I'll try a test run with unlimited time and no >>>> methods excluded to get >>>> a specific count of methods that are being processed. Over time new >>>> features >>>> have been added, e.g. string concatenation optimizations, lambda >>>> functions, >>>> etc., etc., etc. For a test that does method tracing, each new >>>> method adds to the >>>> collective time. If you can not reduce the number of methods >>>> called, then the time >>>> for the test needs to be increased. >>> >>> That sounds quite reasonable. I'm just wondering how the >>> "-waittime=7" interacts with the jtreg timeout handling? >>> >>> Thanks, >>> David >>> >>>> ... >>>> >>>> On 6/25/18, 2:11 PM, Chris Plummer wrote: >>>>> I'm also wondering how fast this test runs on other platforms and >>>>> when passing on solaris-sparc. 5 minutes already seems like a long >>>>> time for this test. There could be an underlying issue that needs >>>>> to be addressed. >>>>> >>>>> Chris >>>>> >>>>> On 6/25/18 11:00 AM, serguei.spitsyn at oracle.com wrote: >>>>>> Hi Gary, >>>>>> >>>>>> It looks Okay. >>>>>> But I'm curious when this started failing and what triggered it >>>>>> to fail? >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>>> >>>>>> On 6/25/18 10:20, Gary Adams wrote: >>>>>>> The exclude001 test times out on solaris sparc debug builds. >>>>>>> >>>>>>> Basically, this test is all about tracing method calls and >>>>>>> introduces >>>>>>> exclude filters to reduce the callbacks to a select set of >>>>>>> packages. >>>>>>> The time spent tracing/filtering method callbacks is purely a >>>>>>> function >>>>>>> of the number of methods that are processed. On this >>>>>>> particularly slow >>>>>>> target platform, more time is needed before issuing a timeout. >>>>>>> >>>>>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8205508 >>>>>>> >>>>>>> Proposed fix: >>>>>>> diff --git >>>>>>> a/test/hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.javab/test/hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java >>>>>>> >>>>>>> --- >>>>>>> a/test/hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java >>>>>>> >>>>>>> +++ >>>>>>> b/test/hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java >>>>>>> >>>>>>> @@ -63,7 +63,7 @@ >>>>>>> * nsk.jdb.exclude.exclude001.exclude001a >>>>>>> * @run main/othervm PropertyResolvingWrapper >>>>>>> nsk.jdb.exclude.exclude001.exclude001 >>>>>>> * -arch=${os.family}-${os.simpleArch} >>>>>>> - * -waittime=5 >>>>>>> + * -waittime=7 >>>>>>> * -debugee.vmkind=java >>>>>>> * -transport.address=dynamic >>>>>>> * -jdb=${test.jdk}/bin/jdb >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >> From harsha.wardhana.b at oracle.com Tue Jun 26 14:09:07 2018 From: harsha.wardhana.b at oracle.com (Harsha Wardhana B) Date: Tue, 26 Jun 2018 19:39:07 +0530 Subject: RFR : JDK-8192953 - sun/management/jmxremote/bootstrap/*.sh tests fail with error : revokeall.exe: Permission denied Message-ID: Hi All, Please find the fix for the bug, https://bugs.openjdk.java.net/browse/JDK-8192953 having webrev at, http://cr.openjdk.java.net/~hb/8192953/webrev.00/ The fix grants execute permission for revokeall.exe. The paths in the shell sciprt had to be converted to cygwin paths (/cygwin/c/... ) from windows path (C:/...). Using windows path was causing strange behavior in cygwin. revokeall.exe should be removed and the above tests need to be refactored to use java.nio.Acl* APIs. That plan is in the near future, and the current fix needs to go in to stop consistent failures in Mach5. Please review the above patch and provide feedback if any. Thanks Harsha From serguei.spitsyn at oracle.com Tue Jun 26 16:12:55 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 26 Jun 2018 09:12:55 -0700 Subject: RFR(XXXS): 8205648 fix for 8205195 breaks secondary error handling In-Reply-To: <4f5a5d7d-f4e4-657e-dbc2-d17b06c1de91@oracle.com> References: <4f5a5d7d-f4e4-657e-dbc2-d17b06c1de91@oracle.com> Message-ID: <183b18d9-252c-b736-74e3-d481c572f5da@oracle.com> Hi Dan, The fix looks reasonable to me. Nice catch by David! Thanks, Serguei On 6/26/18 06:06, Daniel D. Daugherty wrote: > Greetings, > > For this one liner review, I'd love to hear from David H., Thomas St?fe, > and Serguei Spitsyn. > > David Holmes spotted a locking problem with my fix for the following bug: > > ??? JDK-8205195 NestedThreadsListHandleInErrorHandlingTest fails because > ??????????????? hs_err doesn't contain _nested_thread_list_max > ??? https://bugs.openjdk.java.net/browse/JDK-8205195 > > ??? Webrev URL: > http://cr.openjdk.java.net/~dcubed/8205195-webrev/0-for-jdk-jdk/ > > The fix for the issue is a one liner that only grabs the Threads_lock > when the caller does not already own it: > > $ hg diff > diff -r 5698cf4e50f1 src/hotspot/share/utilities/vmError.cpp > --- a/src/hotspot/share/utilities/vmError.cpp Fri Jun 22 12:15:16 2018 > -0400 > +++ b/src/hotspot/share/utilities/vmError.cpp Mon Jun 25 21:20:52 2018 > -0400 > @@ -1703,7 +1703,7 @@ > ?? // from racing with Threads::add() or Threads::remove() as we > ?? // generate the hs_err_pid file. This makes our ErrorHandling tests > ?? // more stable. > - MutexLockerEx ml(Threads_lock, Mutex::_no_safepoint_check_flag); > + MutexLockerEx ml(Threads_lock->owned_by_self() ? NULL : > Threads_lock, Mutex::_no_safepoint_check_flag); > > ?? switch (how) { > ???? case 1: vmassert(str == NULL, "expected null"); break; > > > Figuring out a way to detect the failure mode in order to test the > fix was much more involved than the fix itself. Gory details are > in the bug: > > ??? JDK-8205648 fix for 8205195 breaks secondary error handling > ??? https://bugs.openjdk.java.net/browse/JDK-8205648 > > No webrev since the one liner diff is shown above. This fix was tested > with the test/hotspot/jtreg/runtime/ErrorHandling tests on Linux-X64 > and with a special version of ErrorHandling/SecondaryErrorTest.java > that is documented in JDK-8205648. > > Thanks, in advance, for any comments, questions or suggestions. > > Dan From daniel.daugherty at oracle.com Tue Jun 26 16:17:42 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 26 Jun 2018 12:17:42 -0400 Subject: RFR(XXXS): 8205648 fix for 8205195 breaks secondary error handling In-Reply-To: <183b18d9-252c-b736-74e3-d481c572f5da@oracle.com> References: <4f5a5d7d-f4e4-657e-dbc2-d17b06c1de91@oracle.com> <183b18d9-252c-b736-74e3-d481c572f5da@oracle.com> Message-ID: Thanks for the quick review Serguei! One more review would be nice... David H. or Thomas St?fe? On 6/26/18 12:12 PM, serguei.spitsyn at oracle.com wrote: > Hi Dan, > > The fix looks reasonable to me. Thanks. I like simple one liners... :-) > Nice catch by David! Definitely. Dan > > Thanks, > Serguei > > > On 6/26/18 06:06, Daniel D. Daugherty wrote: >> Greetings, >> >> For this one liner review, I'd love to hear from David H., Thomas St?fe, >> and Serguei Spitsyn. >> >> David Holmes spotted a locking problem with my fix for the following >> bug: >> >> ??? JDK-8205195 NestedThreadsListHandleInErrorHandlingTest fails because >> ??????????????? hs_err doesn't contain _nested_thread_list_max >> ??? https://bugs.openjdk.java.net/browse/JDK-8205195 >> >> ??? Webrev URL: >> http://cr.openjdk.java.net/~dcubed/8205195-webrev/0-for-jdk-jdk/ >> >> The fix for the issue is a one liner that only grabs the Threads_lock >> when the caller does not already own it: >> >> $ hg diff >> diff -r 5698cf4e50f1 src/hotspot/share/utilities/vmError.cpp >> --- a/src/hotspot/share/utilities/vmError.cpp Fri Jun 22 12:15:16 >> 2018 -0400 >> +++ b/src/hotspot/share/utilities/vmError.cpp Mon Jun 25 21:20:52 >> 2018 -0400 >> @@ -1703,7 +1703,7 @@ >> ?? // from racing with Threads::add() or Threads::remove() as we >> ?? // generate the hs_err_pid file. This makes our ErrorHandling tests >> ?? // more stable. >> - MutexLockerEx ml(Threads_lock, Mutex::_no_safepoint_check_flag); >> + MutexLockerEx ml(Threads_lock->owned_by_self() ? NULL : >> Threads_lock, Mutex::_no_safepoint_check_flag); >> >> ?? switch (how) { >> ???? case 1: vmassert(str == NULL, "expected null"); break; >> >> >> Figuring out a way to detect the failure mode in order to test the >> fix was much more involved than the fix itself. Gory details are >> in the bug: >> >> ??? JDK-8205648 fix for 8205195 breaks secondary error handling >> ??? https://bugs.openjdk.java.net/browse/JDK-8205648 >> >> No webrev since the one liner diff is shown above. This fix was tested >> with the test/hotspot/jtreg/runtime/ErrorHandling tests on Linux-X64 >> and with a special version of ErrorHandling/SecondaryErrorTest.java >> that is documented in JDK-8205648. >> >> Thanks, in advance, for any comments, questions or suggestions. >> >> Dan > From thomas.stuefe at gmail.com Tue Jun 26 17:06:12 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 26 Jun 2018 19:06:12 +0200 Subject: RFR(XXXS): 8205648 fix for 8205195 breaks secondary error handling In-Reply-To: References: <4f5a5d7d-f4e4-657e-dbc2-d17b06c1de91@oracle.com> <183b18d9-252c-b736-74e3-d481c572f5da@oracle.com> Message-ID: Hi Daniel, test if I understand this correctly: The original bug: we execute NestedThreadsListHandleInErrorHandlingTest, which is supposed to controlled-crash the VM and find SMR related info in hs-err file. But during error reporting, ThreadsSMRSupport::print_info_on() crashes, since internal SMR data structures are modified concurrently and ThreadsSMRSupport::print_info_on() does not lock. The solution was to unconditionally, in VMError::controlled_crash(), grab the Threads_lock. David spotted one path where this may be called recursively: when we test secondary signal handling, we call VMError::controlled_crash() from within the error handler. In that case, the second call to VMError::controlled_crash() will now, instead of crashing, trigger an assert, since we already own Threads_lock. This did not trip up SecondaryErrorTest.java since it was supposed to crash. Now it crashes with an assert, not a secondary signal. Which quietly castrates the test, btw - the reason we contributed this as a regression for running into blocked signals during signal handling, which is a bad thing. Converting this second signal to assert makes the test ineffective. So far correct? --- Yes, the secondary signal test is too coarse. In our port, we have embellished the "Error occurred during error reporting" message and print out the signal number and pc. But we have not yet brought that improvement upstream. I think your patch is okay. But now I wonder if it would be cleaner to try-lock-if-not-owned inside ThreadsSMRSupport::print_info_on() ? Or its caller, Thread::print_on_error()? Or above that, around that call in that one STEP inside VMError::report() ? Thanks, Thomas On Tue, Jun 26, 2018 at 6:17 PM, Daniel D. Daugherty wrote: > Thanks for the quick review Serguei! > > One more review would be nice... David H. or Thomas St?fe? > > > On 6/26/18 12:12 PM, serguei.spitsyn at oracle.com wrote: >> >> Hi Dan, >> >> The fix looks reasonable to me. > > > Thanks. I like simple one liners... :-) > > >> Nice catch by David! > > > Definitely. > > Dan > > > >> >> Thanks, >> Serguei >> >> >> On 6/26/18 06:06, Daniel D. Daugherty wrote: >>> >>> Greetings, >>> >>> For this one liner review, I'd love to hear from David H., Thomas St?fe, >>> and Serguei Spitsyn. >>> >>> David Holmes spotted a locking problem with my fix for the following bug: >>> >>> JDK-8205195 NestedThreadsListHandleInErrorHandlingTest fails because >>> hs_err doesn't contain _nested_thread_list_max >>> https://bugs.openjdk.java.net/browse/JDK-8205195 >>> >>> Webrev URL: >>> http://cr.openjdk.java.net/~dcubed/8205195-webrev/0-for-jdk-jdk/ >>> >>> The fix for the issue is a one liner that only grabs the Threads_lock >>> when the caller does not already own it: >>> >>> $ hg diff >>> diff -r 5698cf4e50f1 src/hotspot/share/utilities/vmError.cpp >>> --- a/src/hotspot/share/utilities/vmError.cpp Fri Jun 22 12:15:16 2018 >>> -0400 >>> +++ b/src/hotspot/share/utilities/vmError.cpp Mon Jun 25 21:20:52 2018 >>> -0400 >>> @@ -1703,7 +1703,7 @@ >>> // from racing with Threads::add() or Threads::remove() as we >>> // generate the hs_err_pid file. This makes our ErrorHandling tests >>> // more stable. >>> - MutexLockerEx ml(Threads_lock, Mutex::_no_safepoint_check_flag); >>> + MutexLockerEx ml(Threads_lock->owned_by_self() ? NULL : Threads_lock, >>> Mutex::_no_safepoint_check_flag); >>> >>> switch (how) { >>> case 1: vmassert(str == NULL, "expected null"); break; >>> >>> >>> Figuring out a way to detect the failure mode in order to test the >>> fix was much more involved than the fix itself. Gory details are >>> in the bug: >>> >>> JDK-8205648 fix for 8205195 breaks secondary error handling >>> https://bugs.openjdk.java.net/browse/JDK-8205648 >>> >>> No webrev since the one liner diff is shown above. This fix was tested >>> with the test/hotspot/jtreg/runtime/ErrorHandling tests on Linux-X64 >>> and with a special version of ErrorHandling/SecondaryErrorTest.java >>> that is documented in JDK-8205648. >>> >>> Thanks, in advance, for any comments, questions or suggestions. >>> >>> Dan >> >> > From thomas.stuefe at gmail.com Tue Jun 26 17:09:36 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 26 Jun 2018 19:09:36 +0200 Subject: RFR(XXXS): 8205648 fix for 8205195 breaks secondary error handling In-Reply-To: References: <4f5a5d7d-f4e4-657e-dbc2-d17b06c1de91@oracle.com> <183b18d9-252c-b736-74e3-d481c572f5da@oracle.com> Message-ID: On Tue, Jun 26, 2018 at 7:06 PM, Thomas St?fe wrote: > Hi Daniel, > > test if I understand this correctly: > > The original bug: we execute > NestedThreadsListHandleInErrorHandlingTest, which is supposed to > controlled-crash the VM and find SMR related info in hs-err file. But > during error reporting, ThreadsSMRSupport::print_info_on() crashes, > since internal SMR data structures are modified concurrently and > ThreadsSMRSupport::print_info_on() does not lock. The solution was to > unconditionally, in VMError::controlled_crash(), grab the > Threads_lock. > > David spotted one path where this may be called recursively: when we > test secondary signal handling, we call VMError::controlled_crash() > from within the error handler. In that case, the second call to > VMError::controlled_crash() will now, instead of crashing, trigger an > assert, since we already own Threads_lock. > > This did not trip up SecondaryErrorTest.java since it was supposed to > crash. Now it crashes with an assert, not a secondary signal. Which > quietly castrates the test, btw - the reason we contributed this as a > regression for running into blocked signals during signal handling, > which is a bad thing. Converting this second signal to assert makes > the test ineffective. > > So far correct? > > --- > > Yes, the secondary signal test is too coarse. In our port, we have > embellished the "Error occurred during error reporting" message and > print out the signal number and pc. But we have not yet brought that > improvement upstream. > > I think your patch is okay. > > But now I wonder if it would be cleaner to try-lock-if-not-owned > inside ThreadsSMRSupport::print_info_on() ? Or its caller, > Thread::print_on_error()? Or above that, around that call in that one > STEP inside VMError::report() ? > just to be clear, I am fine with the patch in its current form, if you want to ship that. ..Thomas > Thanks, Thomas > > > On Tue, Jun 26, 2018 at 6:17 PM, Daniel D. Daugherty > wrote: >> Thanks for the quick review Serguei! >> >> One more review would be nice... David H. or Thomas St?fe? >> >> >> On 6/26/18 12:12 PM, serguei.spitsyn at oracle.com wrote: >>> >>> Hi Dan, >>> >>> The fix looks reasonable to me. >> >> >> Thanks. I like simple one liners... :-) >> >> >>> Nice catch by David! >> >> >> Definitely. >> >> Dan >> >> >> >>> >>> Thanks, >>> Serguei >>> >>> >>> On 6/26/18 06:06, Daniel D. Daugherty wrote: >>>> >>>> Greetings, >>>> >>>> For this one liner review, I'd love to hear from David H., Thomas St?fe, >>>> and Serguei Spitsyn. >>>> >>>> David Holmes spotted a locking problem with my fix for the following bug: >>>> >>>> JDK-8205195 NestedThreadsListHandleInErrorHandlingTest fails because >>>> hs_err doesn't contain _nested_thread_list_max >>>> https://bugs.openjdk.java.net/browse/JDK-8205195 >>>> >>>> Webrev URL: >>>> http://cr.openjdk.java.net/~dcubed/8205195-webrev/0-for-jdk-jdk/ >>>> >>>> The fix for the issue is a one liner that only grabs the Threads_lock >>>> when the caller does not already own it: >>>> >>>> $ hg diff >>>> diff -r 5698cf4e50f1 src/hotspot/share/utilities/vmError.cpp >>>> --- a/src/hotspot/share/utilities/vmError.cpp Fri Jun 22 12:15:16 2018 >>>> -0400 >>>> +++ b/src/hotspot/share/utilities/vmError.cpp Mon Jun 25 21:20:52 2018 >>>> -0400 >>>> @@ -1703,7 +1703,7 @@ >>>> // from racing with Threads::add() or Threads::remove() as we >>>> // generate the hs_err_pid file. This makes our ErrorHandling tests >>>> // more stable. >>>> - MutexLockerEx ml(Threads_lock, Mutex::_no_safepoint_check_flag); >>>> + MutexLockerEx ml(Threads_lock->owned_by_self() ? NULL : Threads_lock, >>>> Mutex::_no_safepoint_check_flag); >>>> >>>> switch (how) { >>>> case 1: vmassert(str == NULL, "expected null"); break; >>>> >>>> >>>> Figuring out a way to detect the failure mode in order to test the >>>> fix was much more involved than the fix itself. Gory details are >>>> in the bug: >>>> >>>> JDK-8205648 fix for 8205195 breaks secondary error handling >>>> https://bugs.openjdk.java.net/browse/JDK-8205648 >>>> >>>> No webrev since the one liner diff is shown above. This fix was tested >>>> with the test/hotspot/jtreg/runtime/ErrorHandling tests on Linux-X64 >>>> and with a special version of ErrorHandling/SecondaryErrorTest.java >>>> that is documented in JDK-8205648. >>>> >>>> Thanks, in advance, for any comments, questions or suggestions. >>>> >>>> Dan >>> >>> >> From daniel.daugherty at oracle.com Tue Jun 26 18:01:52 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 26 Jun 2018 14:01:52 -0400 Subject: RFR(XXXS): 8205648 fix for 8205195 breaks secondary error handling In-Reply-To: References: <4f5a5d7d-f4e4-657e-dbc2-d17b06c1de91@oracle.com> <183b18d9-252c-b736-74e3-d481c572f5da@oracle.com> Message-ID: <9593519e-7dde-5152-b80f-f8ee64443703@oracle.com> Thomas, Thanks for the review. Replies embedded below... On 6/26/18 1:06 PM, Thomas St?fe wrote: > Hi Daniel, > > test if I understand this correctly: > > The original bug: we execute > NestedThreadsListHandleInErrorHandlingTest, which is supposed to > controlled-crash the VM and find SMR related info in hs-err file. But > during error reporting, ThreadsSMRSupport::print_info_on() crashes, > since internal SMR data structures are modified concurrently and > ThreadsSMRSupport::print_info_on() does not lock. The solution was to > unconditionally, in VMError::controlled_crash(), grab the > Threads_lock. > > David spotted one path where this may be called recursively: when we > test secondary signal handling, we call VMError::controlled_crash() > from within the error handler. In that case, the second call to > VMError::controlled_crash() will now, instead of crashing, trigger an > assert, since we already own Threads_lock. > > This did not trip up SecondaryErrorTest.java since it was supposed to > crash. Now it crashes with an assert, not a secondary signal. Which > quietly castrates the test, btw - the reason we contributed this as a > regression for running into blocked signals during signal handling, > which is a bad thing. Converting this second signal to assert makes > the test ineffective. > > So far correct? Exactly right. :-) > --- > > Yes, the secondary signal test is too coarse. In our port, we have > embellished the "Error occurred during error reporting" message and > print out the signal number and pc. But we have not yet brought that > improvement upstream. I thought about making the "reason" for the secondary crash available via the hs_err_pid file, but that is too big of a change for this. > I think your patch is okay. Thanks! > But now I wonder if it would be cleaner to try-lock-if-not-owned > inside ThreadsSMRSupport::print_info_on() ? Or its caller, > Thread::print_on_error()? Or above that, around that call in that one > STEP inside VMError::report() ? I considered that during the work on JDK-8205195, but that just seems too risky. Hence the big comment: 1070?? // Only grab the Threads_lock if we don't already own it and if we 1071?? // are not reporting an error. 1072?? // Note: Not grabbing the Threads_lock during during error reporting 1073?? // is dangerous because the data structures we want to print can be 1074?? // freed concurrently. However, grabbing the Threads_lock during 1075?? // error reporting can be equally dangerous since this thread might 1076?? // block during error reporting or a nested error could leave the 1077?? // Threads_lock held. The classic no win scenario. So I could do a try-lock-if-not-owned, but then I run into the risk of leaving the Threads_lock held if something else fails in that code path and we end up throwing another signal... More Kobiyashi Maru [1] Again, thanks for the review! Dan [1] https://www.urbandictionary.com/define.php?term=Kobayashi%20Maru > > Thanks, Thomas > > > On Tue, Jun 26, 2018 at 6:17 PM, Daniel D. Daugherty > wrote: >> Thanks for the quick review Serguei! >> >> One more review would be nice... David H. or Thomas St?fe? >> >> >> On 6/26/18 12:12 PM, serguei.spitsyn at oracle.com wrote: >>> Hi Dan, >>> >>> The fix looks reasonable to me. >> >> Thanks. I like simple one liners... :-) >> >> >>> Nice catch by David! >> >> Definitely. >> >> Dan >> >> >> >>> Thanks, >>> Serguei >>> >>> >>> On 6/26/18 06:06, Daniel D. Daugherty wrote: >>>> Greetings, >>>> >>>> For this one liner review, I'd love to hear from David H., Thomas St?fe, >>>> and Serguei Spitsyn. >>>> >>>> David Holmes spotted a locking problem with my fix for the following bug: >>>> >>>> JDK-8205195 NestedThreadsListHandleInErrorHandlingTest fails because >>>> hs_err doesn't contain _nested_thread_list_max >>>> https://bugs.openjdk.java.net/browse/JDK-8205195 >>>> >>>> Webrev URL: >>>> http://cr.openjdk.java.net/~dcubed/8205195-webrev/0-for-jdk-jdk/ >>>> >>>> The fix for the issue is a one liner that only grabs the Threads_lock >>>> when the caller does not already own it: >>>> >>>> $ hg diff >>>> diff -r 5698cf4e50f1 src/hotspot/share/utilities/vmError.cpp >>>> --- a/src/hotspot/share/utilities/vmError.cpp Fri Jun 22 12:15:16 2018 >>>> -0400 >>>> +++ b/src/hotspot/share/utilities/vmError.cpp Mon Jun 25 21:20:52 2018 >>>> -0400 >>>> @@ -1703,7 +1703,7 @@ >>>> // from racing with Threads::add() or Threads::remove() as we >>>> // generate the hs_err_pid file. This makes our ErrorHandling tests >>>> // more stable. >>>> - MutexLockerEx ml(Threads_lock, Mutex::_no_safepoint_check_flag); >>>> + MutexLockerEx ml(Threads_lock->owned_by_self() ? NULL : Threads_lock, >>>> Mutex::_no_safepoint_check_flag); >>>> >>>> switch (how) { >>>> case 1: vmassert(str == NULL, "expected null"); break; >>>> >>>> >>>> Figuring out a way to detect the failure mode in order to test the >>>> fix was much more involved than the fix itself. Gory details are >>>> in the bug: >>>> >>>> JDK-8205648 fix for 8205195 breaks secondary error handling >>>> https://bugs.openjdk.java.net/browse/JDK-8205648 >>>> >>>> No webrev since the one liner diff is shown above. This fix was tested >>>> with the test/hotspot/jtreg/runtime/ErrorHandling tests on Linux-X64 >>>> and with a special version of ErrorHandling/SecondaryErrorTest.java >>>> that is documented in JDK-8205648. >>>> >>>> Thanks, in advance, for any comments, questions or suggestions. >>>> >>>> Dan >>> From daniel.daugherty at oracle.com Tue Jun 26 18:02:52 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 26 Jun 2018 14:02:52 -0400 Subject: RFR(XXXS): 8205648 fix for 8205195 breaks secondary error handling In-Reply-To: References: <4f5a5d7d-f4e4-657e-dbc2-d17b06c1de91@oracle.com> <183b18d9-252c-b736-74e3-d481c572f5da@oracle.com> Message-ID: <7bb043ab-79c4-df9e-88da-20c8260d5428@oracle.com> On 6/26/18 1:09 PM, Thomas St?fe wrote: > On Tue, Jun 26, 2018 at 7:06 PM, Thomas St?fe wrote: >> Hi Daniel, >> >> test if I understand this correctly: >> >> The original bug: we execute >> NestedThreadsListHandleInErrorHandlingTest, which is supposed to >> controlled-crash the VM and find SMR related info in hs-err file. But >> during error reporting, ThreadsSMRSupport::print_info_on() crashes, >> since internal SMR data structures are modified concurrently and >> ThreadsSMRSupport::print_info_on() does not lock. The solution was to >> unconditionally, in VMError::controlled_crash(), grab the >> Threads_lock. >> >> David spotted one path where this may be called recursively: when we >> test secondary signal handling, we call VMError::controlled_crash() >> from within the error handler. In that case, the second call to >> VMError::controlled_crash() will now, instead of crashing, trigger an >> assert, since we already own Threads_lock. >> >> This did not trip up SecondaryErrorTest.java since it was supposed to >> crash. Now it crashes with an assert, not a secondary signal. Which >> quietly castrates the test, btw - the reason we contributed this as a >> regression for running into blocked signals during signal handling, >> which is a bad thing. Converting this second signal to assert makes >> the test ineffective. >> >> So far correct? >> >> --- >> >> Yes, the secondary signal test is too coarse. In our port, we have >> embellished the "Error occurred during error reporting" message and >> print out the signal number and pc. But we have not yet brought that >> improvement upstream. >> >> I think your patch is okay. >> >> But now I wonder if it would be cleaner to try-lock-if-not-owned >> inside ThreadsSMRSupport::print_info_on() ? Or its caller, >> Thread::print_on_error()? Or above that, around that call in that one >> STEP inside VMError::report() ? >> > just to be clear, I am fine with the patch in its current form, if you > want to ship that. Yup. I got that based on the "I think your patch is okay." above... Dan > > ..Thomas > >> Thanks, Thomas >> >> >> On Tue, Jun 26, 2018 at 6:17 PM, Daniel D. Daugherty >> wrote: >>> Thanks for the quick review Serguei! >>> >>> One more review would be nice... David H. or Thomas St?fe? >>> >>> >>> On 6/26/18 12:12 PM, serguei.spitsyn at oracle.com wrote: >>>> Hi Dan, >>>> >>>> The fix looks reasonable to me. >>> >>> Thanks. I like simple one liners... :-) >>> >>> >>>> Nice catch by David! >>> >>> Definitely. >>> >>> Dan >>> >>> >>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 6/26/18 06:06, Daniel D. Daugherty wrote: >>>>> Greetings, >>>>> >>>>> For this one liner review, I'd love to hear from David H., Thomas St?fe, >>>>> and Serguei Spitsyn. >>>>> >>>>> David Holmes spotted a locking problem with my fix for the following bug: >>>>> >>>>> JDK-8205195 NestedThreadsListHandleInErrorHandlingTest fails because >>>>> hs_err doesn't contain _nested_thread_list_max >>>>> https://bugs.openjdk.java.net/browse/JDK-8205195 >>>>> >>>>> Webrev URL: >>>>> http://cr.openjdk.java.net/~dcubed/8205195-webrev/0-for-jdk-jdk/ >>>>> >>>>> The fix for the issue is a one liner that only grabs the Threads_lock >>>>> when the caller does not already own it: >>>>> >>>>> $ hg diff >>>>> diff -r 5698cf4e50f1 src/hotspot/share/utilities/vmError.cpp >>>>> --- a/src/hotspot/share/utilities/vmError.cpp Fri Jun 22 12:15:16 2018 >>>>> -0400 >>>>> +++ b/src/hotspot/share/utilities/vmError.cpp Mon Jun 25 21:20:52 2018 >>>>> -0400 >>>>> @@ -1703,7 +1703,7 @@ >>>>> // from racing with Threads::add() or Threads::remove() as we >>>>> // generate the hs_err_pid file. This makes our ErrorHandling tests >>>>> // more stable. >>>>> - MutexLockerEx ml(Threads_lock, Mutex::_no_safepoint_check_flag); >>>>> + MutexLockerEx ml(Threads_lock->owned_by_self() ? NULL : Threads_lock, >>>>> Mutex::_no_safepoint_check_flag); >>>>> >>>>> switch (how) { >>>>> case 1: vmassert(str == NULL, "expected null"); break; >>>>> >>>>> >>>>> Figuring out a way to detect the failure mode in order to test the >>>>> fix was much more involved than the fix itself. Gory details are >>>>> in the bug: >>>>> >>>>> JDK-8205648 fix for 8205195 breaks secondary error handling >>>>> https://bugs.openjdk.java.net/browse/JDK-8205648 >>>>> >>>>> No webrev since the one liner diff is shown above. This fix was tested >>>>> with the test/hotspot/jtreg/runtime/ErrorHandling tests on Linux-X64 >>>>> and with a special version of ErrorHandling/SecondaryErrorTest.java >>>>> that is documented in JDK-8205648. >>>>> >>>>> Thanks, in advance, for any comments, questions or suggestions. >>>>> >>>>> Dan >>>> From serguei.spitsyn at oracle.com Tue Jun 26 19:14:46 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 26 Jun 2018 12:14:46 -0700 Subject: RFR(XS): 8205701: Problem list serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorGCCMSTest.java Message-ID: <876e51cb-5731-91ed-a733-628554d950eb@oracle.com> An HTML attachment was scrubbed... URL: From daniel.daugherty at oracle.com Tue Jun 26 19:21:17 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 26 Jun 2018 15:21:17 -0400 Subject: RFR(XS): 8205701: Problem list serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorGCCMSTest.java In-Reply-To: <876e51cb-5731-91ed-a733-628554d950eb@oracle.com> References: <876e51cb-5731-91ed-a733-628554d950eb@oracle.com> Message-ID: <71917bfb-49ed-a938-e90a-f3b323f0015b@oracle.com> Thumbs up and the Trivial Change rule applies. Dan On 6/26/18 3:14 PM, serguei.spitsyn at oracle.com wrote: > Please, review the fix for sub-task: > https://bugs.openjdk.java.net/browse/JDK-8205701 > > The test HeapMonitorGCCMSTest.java needs to be problem listed until > main bug is fixed: > https://bugs.openjdk.java.net/browse/JDK-8205643 > > The test failed with the Graal compiler but has a potential to fail > without it as well. > > The patch is: > > diff -r bba1deda9216 test/hotspot/jtreg/ProblemList.txt > --- a/test/hotspot/jtreg/ProblemList.txt??? Tue Jun 26 10:43:50 2018 +0800 > +++ b/test/hotspot/jtreg/ProblemList.txt??? Tue Jun 26 12:06:17 2018 -0700 > @@ -81,6 +81,7 @@ > > ?serviceability/sa/TestRevPtrsForInvokeDynamic.java?? 8191270 generic-all > ?serviceability/sa/sadebugd/SADebugDTest.java???????? 8163805 generic-all > +serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorGCCMSTest.java > 8205643 generic-all > > ?############################################################################# > > > Thanks, > Serguei -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Tue Jun 26 19:24:22 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 26 Jun 2018 12:24:22 -0700 Subject: RFR(XS): 8205701: Problem list serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorGCCMSTest.java In-Reply-To: <71917bfb-49ed-a938-e90a-f3b323f0015b@oracle.com> References: <876e51cb-5731-91ed-a733-628554d950eb@oracle.com> <71917bfb-49ed-a938-e90a-f3b323f0015b@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Tue Jun 26 19:29:53 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 26 Jun 2018 12:29:53 -0700 Subject: RFR(XS): 8205719: Windows Java_sun_tools_attach_VirtualMachineImpl_enqueue() method should include exitCode in exception message Message-ID: <30f1ccc6-46ae-bdd6-7460-e69c88d90e7a@oracle.com> Hello, Please review the following: https://bugs.openjdk.java.net/browse/JDK-8205719 http://cr.openjdk.java.net/~cjplummer/8205719/webrev.00/ I am adding some debugging code to help track down the root cause of JDK-8199811. thanks, Chris From gerard.ziemski at oracle.com Tue Jun 26 19:31:25 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Tue, 26 Jun 2018 14:31:25 -0500 Subject: [12] RFR (S) 8205534: Remove SymbolTable dependency from serviceability agent In-Reply-To: <88e391a8-78a2-8dbc-a489-fce9c6b922b5@oracle.com> References: <88e391a8-78a2-8dbc-a489-fce9c6b922b5@oracle.com> Message-ID: <06EB7239-0A10-4F89-93E2-322D9964C034@oracle.com> hi Coleen, Looks good, just need to update the copyright years here (don?t need to see the webrev): src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/classfile/ClassLoaderDataGraph.java src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/oops/Field.java src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/oops/InstanceKlass.java src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/oops/Method.java src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/oops/Symbol.java src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/utilities/HeapHprofBinWriter.java src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/utilities/ObjectReader.java src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/utilities/soql/JSJavaFactoryImpl.java test/hotspot/jtreg/serviceability/sa/ClhsdbPrintStatics.java test/hotspot/jtreg/serviceability/sa/ClhsdbSource.java Thank you for doing this! cheers > On Jun 22, 2018, at 4:40 PM, coleen.phillimore at oracle.com wrote: > > Summary: Modify SA code to not use SymbolTable and remove it. > > This is to support the concurrent hashtable for SymbolTable. > > open webrev at http://cr.openjdk.java.net/~coleenp/8205534.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8205534 > > Tested with hs-tier1-5. > > Thanks, > Coleen From coleen.phillimore at oracle.com Tue Jun 26 19:33:10 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 26 Jun 2018 15:33:10 -0400 Subject: [12] RFR (S) 8205534: Remove SymbolTable dependency from serviceability agent In-Reply-To: <06EB7239-0A10-4F89-93E2-322D9964C034@oracle.com> References: <88e391a8-78a2-8dbc-a489-fce9c6b922b5@oracle.com> <06EB7239-0A10-4F89-93E2-322D9964C034@oracle.com> Message-ID: <78249220-ea34-a026-1c25-ea7369743bc6@oracle.com> Thank you for reviewing, Gerard! Coleen On 6/26/18 3:31 PM, Gerard Ziemski wrote: > hi Coleen, > > Looks good, just need to update the copyright years here (don?t need to see the webrev): > > src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/classfile/ClassLoaderDataGraph.java > src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/oops/Field.java > src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/oops/InstanceKlass.java > src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/oops/Method.java > src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/oops/Symbol.java > src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/utilities/HeapHprofBinWriter.java > src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/utilities/ObjectReader.java > src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/utilities/soql/JSJavaFactoryImpl.java > test/hotspot/jtreg/serviceability/sa/ClhsdbPrintStatics.java > test/hotspot/jtreg/serviceability/sa/ClhsdbSource.java > > Thank you for doing this! > > > cheers > >> On Jun 22, 2018, at 4:40 PM, coleen.phillimore at oracle.com wrote: >> >> Summary: Modify SA code to not use SymbolTable and remove it. >> >> This is to support the concurrent hashtable for SymbolTable. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8205534.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8205534 >> >> Tested with hs-tier1-5. >> >> Thanks, >> Coleen From serguei.spitsyn at oracle.com Tue Jun 26 19:43:39 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 26 Jun 2018 12:43:39 -0700 Subject: RFR(XS): 8205721: Problem list serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatArrayCorrectnessTest.java Message-ID: Please, review the fix for sub-task: ? https://bugs.openjdk.java.net/browse/JDK-8205721 The test HeapMonitorGCCMSTest.java needs to be problem listed until main bug is fixed: ? https://bugs.openjdk.java.net/browse/JDK-8205541 The patch is: diff -r f9ae777f71ee test/hotspot/jtreg/ProblemList.txt --- a/test/hotspot/jtreg/ProblemList.txt??? Tue Jun 26 12:29:20 2018 -0700 +++ b/test/hotspot/jtreg/ProblemList.txt??? Tue Jun 26 12:38:37 2018 -0700 @@ -82,6 +82,7 @@ ?serviceability/sa/TestRevPtrsForInvokeDynamic.java?? 8191270 generic-all ?serviceability/sa/sadebugd/SADebugDTest.java???????? 8163805 generic-all ?serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorGCCMSTest.java 8205643 generic-all +serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatArrayCorrectnessTest.java 8205541 generic-all ?############################################################################# Thanks, Serguei From serguei.spitsyn at oracle.com Tue Jun 26 20:25:17 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 26 Jun 2018 13:25:17 -0700 Subject: RFR(XS): 8205719: Windows Java_sun_tools_attach_VirtualMachineImpl_enqueue() method should include exitCode in exception message In-Reply-To: <30f1ccc6-46ae-bdd6-7460-e69c88d90e7a@oracle.com> References: <30f1ccc6-46ae-bdd6-7460-e69c88d90e7a@oracle.com> Message-ID: <19220be6-9168-101b-be49-08a6c7000f0a@oracle.com> Hi Chris, It looks good to me. Thanks, Serguei On 6/26/18 12:29, Chris Plummer wrote: > Hello, > > Please review the following: > > https://bugs.openjdk.java.net/browse/JDK-8205719 > http://cr.openjdk.java.net/~cjplummer/8205719/webrev.00/ > > I am adding some debugging code to help track down the root cause of > JDK-8199811. > > thanks, > > Chris From daniel.daugherty at oracle.com Tue Jun 26 20:45:51 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 26 Jun 2018 16:45:51 -0400 Subject: RFR(XS): 8205721: Problem list serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatArrayCorrectnessTest.java In-Reply-To: References: Message-ID: <85584420-9c19-2af9-36cc-f79d755023a9@oracle.com> Thumbs up and the Trivial Change rule applies. Dan On 6/26/18 3:43 PM, serguei.spitsyn at oracle.com wrote: > Please, review the fix for sub-task: > ? https://bugs.openjdk.java.net/browse/JDK-8205721 > > The test HeapMonitorGCCMSTest.java needs to be problem listed until > main bug is fixed: > ? https://bugs.openjdk.java.net/browse/JDK-8205541 > > The patch is: > > diff -r f9ae777f71ee test/hotspot/jtreg/ProblemList.txt > --- a/test/hotspot/jtreg/ProblemList.txt??? Tue Jun 26 12:29:20 2018 > -0700 > +++ b/test/hotspot/jtreg/ProblemList.txt??? Tue Jun 26 12:38:37 2018 > -0700 > @@ -82,6 +82,7 @@ > ?serviceability/sa/TestRevPtrsForInvokeDynamic.java?? 8191270 generic-all > ?serviceability/sa/sadebugd/SADebugDTest.java???????? 8163805 generic-all > ?serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorGCCMSTest.java > 8205643 generic-all > +serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatArrayCorrectnessTest.java > 8205541 generic-all > > ?############################################################################# > > > > Thanks, > Serguei From serguei.spitsyn at oracle.com Tue Jun 26 20:48:01 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 26 Jun 2018 13:48:01 -0700 Subject: RFR(XS): 8205721: Problem list serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatArrayCorrectnessTest.java In-Reply-To: <85584420-9c19-2af9-36cc-f79d755023a9@oracle.com> References: <85584420-9c19-2af9-36cc-f79d755023a9@oracle.com> Message-ID: Thanks a lot, Dan! Serguei On 6/26/18 13:45, Daniel D. Daugherty wrote: > Thumbs up and the Trivial Change rule applies. > > Dan > > > On 6/26/18 3:43 PM, serguei.spitsyn at oracle.com wrote: >> Please, review the fix for sub-task: >> ? https://bugs.openjdk.java.net/browse/JDK-8205721 >> >> The test HeapMonitorGCCMSTest.java needs to be problem listed until >> main bug is fixed: >> ? https://bugs.openjdk.java.net/browse/JDK-8205541 >> >> The patch is: >> >> diff -r f9ae777f71ee test/hotspot/jtreg/ProblemList.txt >> --- a/test/hotspot/jtreg/ProblemList.txt??? Tue Jun 26 12:29:20 2018 >> -0700 >> +++ b/test/hotspot/jtreg/ProblemList.txt??? Tue Jun 26 12:38:37 2018 >> -0700 >> @@ -82,6 +82,7 @@ >> ?serviceability/sa/TestRevPtrsForInvokeDynamic.java?? 8191270 >> generic-all >> ?serviceability/sa/sadebugd/SADebugDTest.java???????? 8163805 >> generic-all >> ?serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorGCCMSTest.java >> 8205643 generic-all >> +serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatArrayCorrectnessTest.java >> 8205541 generic-all >> >> ?############################################################################# >> >> >> >> Thanks, >> Serguei > From serguei.spitsyn at oracle.com Tue Jun 26 21:02:47 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 26 Jun 2018 14:02:47 -0700 Subject: RFR(XS): 8205723: Problem list serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatRateTest.java Message-ID: <9114ef63-1380-7e1f-758e-3e616ac02d09@oracle.com> Please, review the fix for sub-task: ? https://bugs.openjdk.java.net/browse/JDK-8205723 The test HeapMonitorStatRateTest.java needs to be problem listed until main bug is fixed: ? https://bugs.openjdk.java.net/browse/JDK-8205652 The patch is: diff -r fa380b3b2b7d test/hotspot/jtreg/ProblemList.txt --- a/test/hotspot/jtreg/ProblemList.txt??? Tue Jun 26 13:50:59 2018 -0700 +++ b/test/hotspot/jtreg/ProblemList.txt??? Tue Jun 26 13:57:20 2018 -0700 @@ -83,6 +83,7 @@ ?serviceability/sa/sadebugd/SADebugDTest.java???????? 8163805 generic-all ?serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorGCCMSTest.java 8205643 generic-all ?serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatArrayCorrectnessTest.java 8205541 generic-all +serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatRateTest.java 8205652 generic-all ?############################################################################# Thanks, Serguei From poonam.bajaj at oracle.com Tue Jun 26 21:18:50 2018 From: poonam.bajaj at oracle.com (Poonam Parhar) Date: Tue, 26 Jun 2018 14:18:50 -0700 Subject: [12] RFR (S) 8205534: Remove SymbolTable dependency from serviceability agent In-Reply-To: <88e391a8-78a2-8dbc-a489-fce9c6b922b5@oracle.com> References: <88e391a8-78a2-8dbc-a489-fce9c6b922b5@oracle.com> Message-ID: <0974cf5c-171a-401d-0b05-7dee8c8538d7@oracle.com> Hello Coleen, The changes look good to me. One minor thing that I noticed though: At Line 94 in file src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/classfile/ClassLoaderData.java The comment above the function find() is removed. Is that intentional? Thanks, Poonam On 6/22/18 2:40 PM, coleen.phillimore at oracle.com wrote: > Summary: Modify SA code to not use SymbolTable and remove it. > > This is to support the concurrent hashtable for SymbolTable. > > open webrev at http://cr.openjdk.java.net/~coleenp/8205534.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8205534 > > Tested with hs-tier1-5. > > Thanks, > Coleen From coleen.phillimore at oracle.com Tue Jun 26 21:23:03 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 26 Jun 2018 17:23:03 -0400 Subject: [12] RFR (S) 8205534: Remove SymbolTable dependency from serviceability agent In-Reply-To: <0974cf5c-171a-401d-0b05-7dee8c8538d7@oracle.com> References: <88e391a8-78a2-8dbc-a489-fce9c6b922b5@oracle.com> <0974cf5c-171a-401d-0b05-7dee8c8538d7@oracle.com> Message-ID: <964ebca6-0e8c-24b5-c023-118a6cb06554@oracle.com> On 6/26/18 5:18 PM, Poonam Parhar wrote: > Hello Coleen, > > The changes look good to me. One minor thing that I noticed though: > > At Line 94 in file > src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/classfile/ClassLoaderData.java > > The comment above the function find() is removed. Is that intentional? Hi, Poonam, thank you for looking at this and your previous pre-review.? No, the comment was accidently removed and I'll restore it.? Thank you for catching this! Coleen > > Thanks, > Poonam > > On 6/22/18 2:40 PM, coleen.phillimore at oracle.com wrote: >> Summary: Modify SA code to not use SymbolTable and remove it. >> >> This is to support the concurrent hashtable for SymbolTable. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8205534.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8205534 >> >> Tested with hs-tier1-5. >> >> Thanks, >> Coleen > From chris.plummer at oracle.com Tue Jun 26 21:38:05 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 26 Jun 2018 14:38:05 -0700 Subject: RFR(XS): 8205723: Problem list serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatRateTest.java In-Reply-To: <9114ef63-1380-7e1f-758e-3e616ac02d09@oracle.com> References: <9114ef63-1380-7e1f-758e-3e616ac02d09@oracle.com> Message-ID: <85471983-ed61-2ae6-c26d-5bb5c6cfe80a@oracle.com> Looks good. Trivial change, so you can push it now. thanks, Chris On 6/26/18 2:02 PM, serguei.spitsyn at oracle.com wrote: > Please, review the fix for sub-task: > ? https://bugs.openjdk.java.net/browse/JDK-8205723 > > The test HeapMonitorStatRateTest.java needs to be problem listed until > main bug is fixed: > ? https://bugs.openjdk.java.net/browse/JDK-8205652 > > > The patch is: > > diff -r fa380b3b2b7d test/hotspot/jtreg/ProblemList.txt > --- a/test/hotspot/jtreg/ProblemList.txt??? Tue Jun 26 13:50:59 2018 > -0700 > +++ b/test/hotspot/jtreg/ProblemList.txt??? Tue Jun 26 13:57:20 2018 > -0700 > @@ -83,6 +83,7 @@ > ?serviceability/sa/sadebugd/SADebugDTest.java???????? 8163805 generic-all > ?serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorGCCMSTest.java > 8205643 generic-all > ?serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatArrayCorrectnessTest.java > 8205541 generic-all > +serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatRateTest.java > 8205652 generic-all > > ?############################################################################# > > > > Thanks, > Serguei > From daniel.daugherty at oracle.com Tue Jun 26 21:39:29 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 26 Jun 2018 17:39:29 -0400 Subject: RFR(XS): 8205723: Problem list serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatRateTest.java In-Reply-To: <9114ef63-1380-7e1f-758e-3e616ac02d09@oracle.com> References: <9114ef63-1380-7e1f-758e-3e616ac02d09@oracle.com> Message-ID: <52d40858-309d-dcee-631b-af1beffb08ae@oracle.com> Thumbs up and this meets the Trivial Change policy... Dan On 6/26/18 5:02 PM, serguei.spitsyn at oracle.com wrote: > Please, review the fix for sub-task: > ? https://bugs.openjdk.java.net/browse/JDK-8205723 > > The test HeapMonitorStatRateTest.java needs to be problem listed until > main bug is fixed: > ? https://bugs.openjdk.java.net/browse/JDK-8205652 > > > The patch is: > > diff -r fa380b3b2b7d test/hotspot/jtreg/ProblemList.txt > --- a/test/hotspot/jtreg/ProblemList.txt??? Tue Jun 26 13:50:59 2018 > -0700 > +++ b/test/hotspot/jtreg/ProblemList.txt??? Tue Jun 26 13:57:20 2018 > -0700 > @@ -83,6 +83,7 @@ > ?serviceability/sa/sadebugd/SADebugDTest.java???????? 8163805 generic-all > ?serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorGCCMSTest.java > 8205643 generic-all > ?serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatArrayCorrectnessTest.java > 8205541 generic-all > +serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatRateTest.java > 8205652 generic-all > > ?############################################################################# > > > > Thanks, > Serguei > From serguei.spitsyn at oracle.com Tue Jun 26 22:42:26 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 26 Jun 2018 15:42:26 -0700 Subject: RFR(XS): 8205723: Problem list serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatRateTest.java In-Reply-To: <52d40858-309d-dcee-631b-af1beffb08ae@oracle.com> References: <9114ef63-1380-7e1f-758e-3e616ac02d09@oracle.com> <52d40858-309d-dcee-631b-af1beffb08ae@oracle.com> Message-ID: <8b140ebc-b641-fa1b-7b6f-955856c3b539@oracle.com> Thanks a lot for review, Chris and Dan! Serguei On 6/26/18 14:39, Daniel D. Daugherty wrote: > Thumbs up and this meets the Trivial Change policy... > > Dan > > > On 6/26/18 5:02 PM, serguei.spitsyn at oracle.com wrote: >> Please, review the fix for sub-task: >> ? https://bugs.openjdk.java.net/browse/JDK-8205723 >> >> The test HeapMonitorStatRateTest.java needs to be problem listed >> until main bug is fixed: >> ? https://bugs.openjdk.java.net/browse/JDK-8205652 >> >> >> The patch is: >> >> diff -r fa380b3b2b7d test/hotspot/jtreg/ProblemList.txt >> --- a/test/hotspot/jtreg/ProblemList.txt??? Tue Jun 26 13:50:59 2018 >> -0700 >> +++ b/test/hotspot/jtreg/ProblemList.txt??? Tue Jun 26 13:57:20 2018 >> -0700 >> @@ -83,6 +83,7 @@ >> ?serviceability/sa/sadebugd/SADebugDTest.java???????? 8163805 >> generic-all >> ?serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorGCCMSTest.java >> 8205643 generic-all >> ?serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatArrayCorrectnessTest.java >> 8205541 generic-all >> +serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatRateTest.java >> 8205652 generic-all >> >> ?############################################################################# >> >> >> >> Thanks, >> Serguei >> > From david.holmes at oracle.com Wed Jun 27 03:43:40 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 27 Jun 2018 13:43:40 +1000 Subject: RFR(XXXS): 8205648 fix for 8205195 breaks secondary error handling In-Reply-To: <4f5a5d7d-f4e4-657e-dbc2-d17b06c1de91@oracle.com> References: <4f5a5d7d-f4e4-657e-dbc2-d17b06c1de91@oracle.com> Message-ID: Looks good Dan! Thanks, David On 26/06/2018 11:06 PM, Daniel D. Daugherty wrote: > Greetings, > > For this one liner review, I'd love to hear from David H., Thomas St?fe, > and Serguei Spitsyn. > > David Holmes spotted a locking problem with my fix for the following bug: > > ??? JDK-8205195 NestedThreadsListHandleInErrorHandlingTest fails because > ??????????????? hs_err doesn't contain _nested_thread_list_max > ??? https://bugs.openjdk.java.net/browse/JDK-8205195 > > ??? Webrev URL: > http://cr.openjdk.java.net/~dcubed/8205195-webrev/0-for-jdk-jdk/ > > The fix for the issue is a one liner that only grabs the Threads_lock > when the caller does not already own it: > > $ hg diff > diff -r 5698cf4e50f1 src/hotspot/share/utilities/vmError.cpp > --- a/src/hotspot/share/utilities/vmError.cpp Fri Jun 22 12:15:16 2018 > -0400 > +++ b/src/hotspot/share/utilities/vmError.cpp Mon Jun 25 21:20:52 2018 > -0400 > @@ -1703,7 +1703,7 @@ > ?? // from racing with Threads::add() or Threads::remove() as we > ?? // generate the hs_err_pid file. This makes our ErrorHandling tests > ?? // more stable. > - MutexLockerEx ml(Threads_lock, Mutex::_no_safepoint_check_flag); > + MutexLockerEx ml(Threads_lock->owned_by_self() ? NULL : Threads_lock, > Mutex::_no_safepoint_check_flag); > > ?? switch (how) { > ???? case 1: vmassert(str == NULL, "expected null"); break; > > > Figuring out a way to detect the failure mode in order to test the > fix was much more involved than the fix itself. Gory details are > in the bug: > > ??? JDK-8205648 fix for 8205195 breaks secondary error handling > ??? https://bugs.openjdk.java.net/browse/JDK-8205648 > > No webrev since the one liner diff is shown above. This fix was tested > with the test/hotspot/jtreg/runtime/ErrorHandling tests on Linux-X64 > and with a special version of ErrorHandling/SecondaryErrorTest.java > that is documented in JDK-8205648. > > Thanks, in advance, for any comments, questions or suggestions. > > Dan From david.holmes at oracle.com Wed Jun 27 06:32:34 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 27 Jun 2018 16:32:34 +1000 Subject: RFR: JDK-8189429: SA: MacOSX: Replace the deprecated PT_ATTACH with PT_ATTACHEXC In-Reply-To: <8e5f1747-3d1f-7381-549c-965ee3d5ef08@oracle.com> References: <7abea33e-7fb0-fbe0-d463-8999ccdf54da@oracle.com> <0c93044e-7aaf-179b-c80c-45ff4d7e2712@oracle.com> <5ba6916b-35bf-617e-fff8-93f096162047@oracle.com> <8e5f1747-3d1f-7381-549c-965ee3d5ef08@oracle.com> Message-ID: <91f8141a-d550-7a70-c65e-efa9321d8977@oracle.com> Hi Jini, I took a look ... that's about all I can say :) I know that you and Sharath have worked through this in detail over an extended period of time, so I'm okay to add my Reviewed stamp to it. About the only thing I'd suggest, if not already done, is to do a mach5 run only on OSX with --test-repeat 10, to try to check that we are okay on a range of OSX machines. Thanks, David On 26/06/2018 5:25 PM, Jini George wrote: > Ping - Gentle reminder ! > > Thanks, > Jini. > > On 6/25/2018 3:41 PM, Jini George wrote: >> Thank you, Sharath. May I have a Reviewer to take a look at the >> MacosxDebuggerLocal code? >> >> Thanks, >> Jini. >> >> On 6/25/2018 1:52 PM, Sharath Ballal wrote: >>> Hi Jini, >>> >>> Changes in MacosxDebuggerLocal.m looks good. >>> >>> Thanks, >>> Sharath >>> >>> >>> -----Original Message----- >>> From: Jini George >>> Sent: Sunday, June 24, 2018 11:07 PM >>> To: Erik Joelsson; serviceability-dev at openjdk.java.net; >>> build-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net >>> Subject: Re: RFR: JDK-8189429: SA: MacOSX: Replace the deprecated >>> PT_ATTACH with PT_ATTACHEXC >>> >>> Hi Erik, >>> >>> Thank you very much for looking into this. I have addressed your >>> comments. The latest webrev is at: >>> >>> http://cr.openjdk.java.net/~jgeorge/8189429/webrev.06/ >>> >>> Thank you, >>> Jini. >>> >>> >>> On 6/23/2018 3:31 AM, Erik Joelsson wrote: >>>> Hello Jini, >>>> >>>> In general this looks pretty good, but it's also breaking some new >>>> ground as it's adding generation of native source in the java gensrc >>>> step. Mixing native code with the java code that the genrcs targets >>>> and gensrc output directories are meant for seems ok for now, but may >>>> cause trouble in the future. I'm going to accept it for now though. >>>> >>>> In Gensrc-jdk.hotspot.agent.gmk: Please put the new macosx stuff in >>>> its own section (as delimited by the 80x # lines) and put that whole >>>> thing inside a conditional for macosx. Also please break line 47 (for >>>> recipe lines, indent with tab and 4 additional spaces for >>>> continuation [1]). >>>> >>>> In Lib-jdk.hotspot.agent.gmk: I believe adding extra src should be >>>> enough as that will implicitly add the same directories as header dirs >>>> by default. At least that's the intention. >>>> >>>> /Erik >>>> >>>> [1] http://openjdk.java.net/groups/build/doc/code-conventions.html >>>> >>>> On 2018-06-22 11:11, Jini George wrote: >>>>> Hi all, >>>>> >>>>> [Including build-dev also since this includes build related changes]. >>>>> >>>>> Requesting reviews for: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8189429 (SA: MacOSX: Replace >>>>> the deprecated PT_ATTACH with PT_ATTACHEXC) >>>>> >>>>> Webrev: http://cr.openjdk.java.net/~jgeorge/8189429/webrev.04/ >>>>> >>>>> This is the follow-up solution for the temporary restoration of >>>>> PT_ATTACH to fix https://bugs.openjdk.java.net/browse/JDK-8184042 >>>>> (several serviceability/sa tests timed out on MacOS X), which was >>>>> done in Oct 2017. The mails related to this are at: >>>>> >>>>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2017-August >>>>> /021702.html >>>>> >>>>> >>>>> Changes as compared to the patch sent last year >>>>> (http://cr.openjdk.java.net/~jgeorge/8184042/webrev.00/): >>>>> >>>>> * Addressed the review comments which were provided by Poonam, Dan, >>>>> Dmitry. >>>>> * A major change as compared to what was done last year is that the >>>>> MIG generated files have been included as a part of the JDK build >>>>> process. >>>>> * The test case which was provided in the patch last year is no >>>>> longer required since we have ClhsdbAttach.java testing the same >>>>> functionality as a part of the SA testsuite now. >>>>> * Other than that, some minor improvements have been done wrt error >>>>> handling. >>>>> >>>>> The steps for the proposed solution have been provided in JBS. >>>>> >>>>> Testing: ALL the SA tests pass on MacOSX and the other Mach5 >>>>> platforms. >>>>> >>>>> Thanks to Sharath, Robin, Gerard and Dan for looking into the changes >>>>> and providing valuable comments. >>>>> >>>>> Thanks, >>>>> Jini. >>>> From jini.george at oracle.com Wed Jun 27 07:09:58 2018 From: jini.george at oracle.com (Jini George) Date: Wed, 27 Jun 2018 12:39:58 +0530 Subject: RFR: JDK-8189429: SA: MacOSX: Replace the deprecated PT_ATTACH with PT_ATTACHEXC In-Reply-To: <91f8141a-d550-7a70-c65e-efa9321d8977@oracle.com> References: <7abea33e-7fb0-fbe0-d463-8999ccdf54da@oracle.com> <0c93044e-7aaf-179b-c80c-45ff4d7e2712@oracle.com> <5ba6916b-35bf-617e-fff8-93f096162047@oracle.com> <8e5f1747-3d1f-7381-549c-965ee3d5ef08@oracle.com> <91f8141a-d550-7a70-c65e-efa9321d8977@oracle.com> Message-ID: Thank you very much, David. I will do the test-repeat run of the tests (after a temp fix to enable OSX runs on Mach5 (JDK-8199700)). Thanks, Jini. On 6/27/2018 12:02 PM, David Holmes wrote: > Hi Jini, > > I took a look ... that's about all I can say :) > > I know that you and Sharath have worked through this in detail over an > extended period of time, so I'm okay to add my Reviewed stamp to it. > > About the only thing I'd suggest, if not already done, is to do a mach5 > run only on OSX with --test-repeat 10, to try to check that we are okay > on a range of OSX machines. > > Thanks, > David > > On 26/06/2018 5:25 PM, Jini George wrote: >> Ping - Gentle reminder ! >> >> Thanks, >> Jini. >> >> On 6/25/2018 3:41 PM, Jini George wrote: >>> Thank you, Sharath. May I have a Reviewer to take a look at the >>> MacosxDebuggerLocal code? >>> >>> Thanks, >>> Jini. >>> >>> On 6/25/2018 1:52 PM, Sharath Ballal wrote: >>>> Hi Jini, >>>> >>>> Changes in MacosxDebuggerLocal.m looks good. >>>> >>>> Thanks, >>>> Sharath >>>> >>>> >>>> -----Original Message----- >>>> From: Jini George >>>> Sent: Sunday, June 24, 2018 11:07 PM >>>> To: Erik Joelsson; serviceability-dev at openjdk.java.net; >>>> build-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net >>>> Subject: Re: RFR: JDK-8189429: SA: MacOSX: Replace the deprecated >>>> PT_ATTACH with PT_ATTACHEXC >>>> >>>> Hi Erik, >>>> >>>> Thank you very much for looking into this. I have addressed your >>>> comments. The latest webrev is at: >>>> >>>> http://cr.openjdk.java.net/~jgeorge/8189429/webrev.06/ >>>> >>>> Thank you, >>>> Jini. >>>> >>>> >>>> On 6/23/2018 3:31 AM, Erik Joelsson wrote: >>>>> Hello Jini, >>>>> >>>>> In general this looks pretty good, but it's also breaking some new >>>>> ground as it's adding generation of native source in the java gensrc >>>>> step. Mixing native code with the java code that the genrcs targets >>>>> and gensrc output directories are meant for seems ok for now, but may >>>>> cause trouble in the future. I'm going to accept it for now though. >>>>> >>>>> In Gensrc-jdk.hotspot.agent.gmk: Please put the new macosx stuff in >>>>> its own section (as delimited by the 80x # lines) and put that whole >>>>> thing inside a conditional for macosx. Also please break line 47 (for >>>>> recipe lines, indent with tab and 4 additional spaces for >>>>> continuation [1]). >>>>> >>>>> In Lib-jdk.hotspot.agent.gmk: I believe adding extra src should be >>>>> enough as that will implicitly add the same directories as header dirs >>>>> by default. At least that's the intention. >>>>> >>>>> /Erik >>>>> >>>>> [1] http://openjdk.java.net/groups/build/doc/code-conventions.html >>>>> >>>>> On 2018-06-22 11:11, Jini George wrote: >>>>>> Hi all, >>>>>> >>>>>> [Including build-dev also since this includes build related changes]. >>>>>> >>>>>> Requesting reviews for: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8189429 (SA: MacOSX: Replace >>>>>> the deprecated PT_ATTACH with PT_ATTACHEXC) >>>>>> >>>>>> Webrev: http://cr.openjdk.java.net/~jgeorge/8189429/webrev.04/ >>>>>> >>>>>> This is the follow-up solution for the temporary restoration of >>>>>> PT_ATTACH to fix https://bugs.openjdk.java.net/browse/JDK-8184042 >>>>>> (several serviceability/sa tests timed out on MacOS X), which was >>>>>> done in Oct 2017. The mails related to this are at: >>>>>> >>>>>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2017-August >>>>>> /021702.html >>>>>> >>>>>> >>>>>> Changes as compared to the patch sent last year >>>>>> (http://cr.openjdk.java.net/~jgeorge/8184042/webrev.00/): >>>>>> >>>>>> * Addressed the review comments which were provided by Poonam, Dan, >>>>>> Dmitry. >>>>>> * A major change as compared to what was done last year is that the >>>>>> MIG generated files have been included as a part of the JDK build >>>>>> process. >>>>>> * The test case which was provided in the patch last year is no >>>>>> longer required since we have ClhsdbAttach.java testing the same >>>>>> functionality as a part of the SA testsuite now. >>>>>> * Other than that, some minor improvements have been done wrt error >>>>>> handling. >>>>>> >>>>>> The steps for the proposed solution have been provided in JBS. >>>>>> >>>>>> Testing: ALL the SA tests pass on MacOSX and the other Mach5 >>>>>> platforms. >>>>>> >>>>>> Thanks to Sharath, Robin, Gerard and Dan for looking into the changes >>>>>> and providing valuable comments. >>>>>> >>>>>> Thanks, >>>>>> Jini. >>>>> From thomas.stuefe at gmail.com Wed Jun 27 12:13:19 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 27 Jun 2018 14:13:19 +0200 Subject: RFR(s): jcmd VM.classloaders should fold similar loaders In-Reply-To: <9fee006f-0de6-08ee-2e53-5cf683066005@oracle.com> References: <9fee006f-0de6-08ee-2e53-5cf683066005@oracle.com> Message-ID: Hi David, On Mon, Jun 25, 2018 at 7:48 AM, David Holmes wrote: > Hi Thomas, > > > On 23/06/2018 5:10 AM, Thomas St?fe wrote: >> >> Resent with the correct subject, sorry. >> >> ..Thomas >> >> On Fri, Jun 22, 2018 at 9:08 PM, Thomas St?fe >> wrote: >>> >>> Hi all, >>> >>> may I have reviews for this small enhancement to the jcmd >>> VM.classloader diagnostic command: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8205531 >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8205531-vm.classloader-tree-folding/webrev.00/webrev/ >>> >>> >>> VM.classloaders prints a tree of class loaders. This tree can grow a >>> lot and become unwieldy, especially with a lot of similar loaders. One >>> prime example is the DelegatingClassLoader. It would be helpful were >>> all these loaders: >>> >>> 13114: >>> +-- >>> | >>> +-- "platform", >>> jdk.internal.loader.ClassLoaders$PlatformClassLoader >>> | >>> +-- "app", jdk.internal.loader.ClassLoaders$AppClassLoader >>> | >>> +-- test3.internals.InMemoryClassLoader >>> | >>> +-- jdk.internal.reflect.DelegatingClassLoader >>> | >>> +-- jdk.internal.reflect.DelegatingClassLoader >>> | >>> +-- jdk.internal.reflect.DelegatingClassLoader >>> | >>> +-- jdk.internal.reflect.DelegatingClassLoader >>> | >>> +-- jdk.internal.reflect.DelegatingClassLoader >>> | >>> ...... repeat 1495 times >>> >>> folded into one: >>> >>> 13114: >>> +-- >>> | >>> +-- "platform", >>> jdk.internal.loader.ClassLoaders$PlatformClassLoader >>> | >>> +-- "app", jdk.internal.loader.ClassLoaders$AppClassLoader >>> | >>> +-- test3.internals.InMemoryClassLoader >>> | >>> +-- jdk.internal.reflect.DelegatingClassLoader >>> (+ 1499 more) > > > I don't quite understand that. These are different instances aren't they - > potentially uniquely named? So if we gave all loaders a default name (like > we do threads) they would all show up expanded again - right? > This patch will fold loaders loaded by the same parent, having the same class and the same or no name. Example: 25401: +-- | +-- "platform", jdk.internal.loader.ClassLoaders$PlatformClassLoader | | | +-- "app", jdk.internal.loader.ClassLoaders$AppClassLoader | +-- "small-loader", test3.internals.InMemoryClassLoader (+ 1101 more) Note that I compare addresses of _Symbol_. So, loaders which have the _same_ default name still fold, see example above. Thanks, Thomas > typo: > > ! // we we add a > > Not a review as I don't know any of the logic being modified. > > David > > >>> >>> >>> Original idea by Bernd Eckenfels, see >>> >>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2018-May/023824.html >>> >>> Thank you, Thomas From david.holmes at oracle.com Wed Jun 27 12:21:14 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 27 Jun 2018 22:21:14 +1000 Subject: RFR(s): jcmd VM.classloaders should fold similar loaders In-Reply-To: References: <9fee006f-0de6-08ee-2e53-5cf683066005@oracle.com> Message-ID: Hi Thomas, On 27/06/2018 10:13 PM, Thomas St?fe wrote: > Hi David, > > On Mon, Jun 25, 2018 at 7:48 AM, David Holmes wrote: >> Hi Thomas, >> >> >> On 23/06/2018 5:10 AM, Thomas St?fe wrote: >>> >>> Resent with the correct subject, sorry. >>> >>> ..Thomas >>> >>> On Fri, Jun 22, 2018 at 9:08 PM, Thomas St?fe >>> wrote: >>>> >>>> Hi all, >>>> >>>> may I have reviews for this small enhancement to the jcmd >>>> VM.classloader diagnostic command: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8205531 >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8205531-vm.classloader-tree-folding/webrev.00/webrev/ >>>> >>>> >>>> VM.classloaders prints a tree of class loaders. This tree can grow a >>>> lot and become unwieldy, especially with a lot of similar loaders. One >>>> prime example is the DelegatingClassLoader. It would be helpful were >>>> all these loaders: >>>> >>>> 13114: >>>> +-- >>>> | >>>> +-- "platform", >>>> jdk.internal.loader.ClassLoaders$PlatformClassLoader >>>> | >>>> +-- "app", jdk.internal.loader.ClassLoaders$AppClassLoader >>>> | >>>> +-- test3.internals.InMemoryClassLoader >>>> | >>>> +-- jdk.internal.reflect.DelegatingClassLoader >>>> | >>>> +-- jdk.internal.reflect.DelegatingClassLoader >>>> | >>>> +-- jdk.internal.reflect.DelegatingClassLoader >>>> | >>>> +-- jdk.internal.reflect.DelegatingClassLoader >>>> | >>>> +-- jdk.internal.reflect.DelegatingClassLoader >>>> | >>>> ...... repeat 1495 times >>>> >>>> folded into one: >>>> >>>> 13114: >>>> +-- >>>> | >>>> +-- "platform", >>>> jdk.internal.loader.ClassLoaders$PlatformClassLoader >>>> | >>>> +-- "app", jdk.internal.loader.ClassLoaders$AppClassLoader >>>> | >>>> +-- test3.internals.InMemoryClassLoader >>>> | >>>> +-- jdk.internal.reflect.DelegatingClassLoader >>>> (+ 1499 more) >> >> >> I don't quite understand that. These are different instances aren't they - >> potentially uniquely named? So if we gave all loaders a default name (like >> we do threads) they would all show up expanded again - right? >> > > This patch will fold loaders loaded by the same parent, having the > same class and the same or no name. > > Example: > > 25401: > +-- > | > +-- "platform", jdk.internal.loader.ClassLoaders$PlatformClassLoader > | | > | +-- "app", jdk.internal.loader.ClassLoaders$AppClassLoader > | > +-- "small-loader", test3.internals.InMemoryClassLoader (+ 1101 more) > > Note that I compare addresses of _Symbol_. So, loaders which have the > _same_ default name still fold, see example above. Right. So in the case of DelegatingClassloader, today this change will fold them all into one because they are unnamed. If tomorrow we give them a unique name then they will no longer fold. There's no substantive difference between the two cases that I can see so why should they present differently? I guess I'm not sure the name is actually significant here. What happens when some of these similar loaders have distinct child loaders? How will things be grouped and displayed then? Thanks, David > Thanks, Thomas > > >> typo: >> >> ! // we we add a >> >> Not a review as I don't know any of the logic being modified. >> >> David >> >> >>>> >>>> >>>> Original idea by Bernd Eckenfels, see >>>> >>>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2018-May/023824.html >>>> >>>> Thank you, Thomas From thomas.stuefe at gmail.com Wed Jun 27 12:30:29 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 27 Jun 2018 14:30:29 +0200 Subject: RFR(s): jcmd VM.classloaders should fold similar loaders In-Reply-To: References: <9fee006f-0de6-08ee-2e53-5cf683066005@oracle.com> Message-ID: On Wed, Jun 27, 2018 at 2:21 PM, David Holmes wrote: > Hi Thomas, > > > On 27/06/2018 10:13 PM, Thomas St?fe wrote: >> >> Hi David, >> >> On Mon, Jun 25, 2018 at 7:48 AM, David Holmes >> wrote: >>> >>> Hi Thomas, >>> >>> >>> On 23/06/2018 5:10 AM, Thomas St?fe wrote: >>>> >>>> >>>> Resent with the correct subject, sorry. >>>> >>>> ..Thomas >>>> >>>> On Fri, Jun 22, 2018 at 9:08 PM, Thomas St?fe >>>> wrote: >>>>> >>>>> >>>>> Hi all, >>>>> >>>>> may I have reviews for this small enhancement to the jcmd >>>>> VM.classloader diagnostic command: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8205531 >>>>> >>>>> >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8205531-vm.classloader-tree-folding/webrev.00/webrev/ >>>>> >>>>> >>>>> VM.classloaders prints a tree of class loaders. This tree can grow a >>>>> lot and become unwieldy, especially with a lot of similar loaders. One >>>>> prime example is the DelegatingClassLoader. It would be helpful were >>>>> all these loaders: >>>>> >>>>> 13114: >>>>> +-- >>>>> | >>>>> +-- "platform", >>>>> jdk.internal.loader.ClassLoaders$PlatformClassLoader >>>>> | >>>>> +-- "app", >>>>> jdk.internal.loader.ClassLoaders$AppClassLoader >>>>> | >>>>> +-- test3.internals.InMemoryClassLoader >>>>> | >>>>> +-- >>>>> jdk.internal.reflect.DelegatingClassLoader >>>>> | >>>>> +-- >>>>> jdk.internal.reflect.DelegatingClassLoader >>>>> | >>>>> +-- >>>>> jdk.internal.reflect.DelegatingClassLoader >>>>> | >>>>> +-- >>>>> jdk.internal.reflect.DelegatingClassLoader >>>>> | >>>>> +-- >>>>> jdk.internal.reflect.DelegatingClassLoader >>>>> | >>>>> ...... repeat 1495 times >>>>> >>>>> folded into one: >>>>> >>>>> 13114: >>>>> +-- >>>>> | >>>>> +-- "platform", >>>>> jdk.internal.loader.ClassLoaders$PlatformClassLoader >>>>> | >>>>> +-- "app", >>>>> jdk.internal.loader.ClassLoaders$AppClassLoader >>>>> | >>>>> +-- test3.internals.InMemoryClassLoader >>>>> | >>>>> +-- >>>>> jdk.internal.reflect.DelegatingClassLoader >>>>> (+ 1499 more) >>> >>> >>> >>> I don't quite understand that. These are different instances aren't they >>> - >>> potentially uniquely named? So if we gave all loaders a default name >>> (like >>> we do threads) they would all show up expanded again - right? >>> >> >> This patch will fold loaders loaded by the same parent, having the >> same class and the same or no name. >> >> Example: >> >> 25401: >> +-- >> | >> +-- "platform", >> jdk.internal.loader.ClassLoaders$PlatformClassLoader >> | | >> | +-- "app", jdk.internal.loader.ClassLoaders$AppClassLoader >> | >> +-- "small-loader", test3.internals.InMemoryClassLoader (+ 1101 >> more) >> >> Note that I compare addresses of _Symbol_. So, loaders which have the >> _same_ default name still fold, see example above. > > > Right. So in the case of DelegatingClassloader, today this change will fold > them all into one because they are unnamed. Yes. > If tomorrow we give them a > unique name then they will no longer fold. Yes. > There's no substantive difference > between the two cases that I can see so why should they present differently? There is, they are now named :) The whole point of this exercise is to make the output of VM.classloaders better parsable for human brains without omitting information: If class and name are identical, I can mentally expand ["small-loader", test3.internals.InMemoryClassLoader (+ 1101 more)] to 1102 lines of ["small-loader", test3.internals.InMemoryClassLoader]. The moment someone starts giving his classloaders unique names, this makes no sense anymore, since the name carries information I do not want to hide. > I guess I'm not sure the name is actually significant here. Hm. So, you would ignore the name and just fold the same class? And then what, write "various" as name for the 1101 differently named loaders? As I wrote, I did not want to omit information. > > What happens when some of these similar loaders have distinct child loaders? > How will things be grouped and displayed then? No folding is done in that case. Thanks, Thomas > > Thanks, > David > > >> Thanks, Thomas >> >> >>> typo: >>> >>> ! // we we add a >>> >>> Not a review as I don't know any of the logic being modified. >>> >>> David >>> >>> >>>>> >>>>> >>>>> Original idea by Bernd Eckenfels, see >>>>> >>>>> >>>>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2018-May/023824.html >>>>> >>>>> Thank you, Thomas From daniel.daugherty at oracle.com Wed Jun 27 13:03:41 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 27 Jun 2018 09:03:41 -0400 Subject: RFR(XXXS): 8205648 fix for 8205195 breaks secondary error handling In-Reply-To: References: <4f5a5d7d-f4e4-657e-dbc2-d17b06c1de91@oracle.com> Message-ID: <0391b125-3adb-e460-f5aa-fb6b38d68c9e@oracle.com> Thanks! And thanks for catching the bug itself. Dan On 6/26/18 11:43 PM, David Holmes wrote: > Looks good Dan! > > Thanks, > David > > On 26/06/2018 11:06 PM, Daniel D. Daugherty wrote: >> Greetings, >> >> For this one liner review, I'd love to hear from David H., Thomas St?fe, >> and Serguei Spitsyn. >> >> David Holmes spotted a locking problem with my fix for the following >> bug: >> >> ???? JDK-8205195 NestedThreadsListHandleInErrorHandlingTest fails >> because >> ???????????????? hs_err doesn't contain _nested_thread_list_max >> ???? https://bugs.openjdk.java.net/browse/JDK-8205195 >> >> ???? Webrev URL: >> http://cr.openjdk.java.net/~dcubed/8205195-webrev/0-for-jdk-jdk/ >> >> The fix for the issue is a one liner that only grabs the Threads_lock >> when the caller does not already own it: >> >> $ hg diff >> diff -r 5698cf4e50f1 src/hotspot/share/utilities/vmError.cpp >> --- a/src/hotspot/share/utilities/vmError.cpp Fri Jun 22 12:15:16 >> 2018 -0400 >> +++ b/src/hotspot/share/utilities/vmError.cpp Mon Jun 25 21:20:52 >> 2018 -0400 >> @@ -1703,7 +1703,7 @@ >> ??? // from racing with Threads::add() or Threads::remove() as we >> ??? // generate the hs_err_pid file. This makes our ErrorHandling tests >> ??? // more stable. >> - MutexLockerEx ml(Threads_lock, Mutex::_no_safepoint_check_flag); >> + MutexLockerEx ml(Threads_lock->owned_by_self() ? NULL : >> Threads_lock, Mutex::_no_safepoint_check_flag); >> >> ??? switch (how) { >> ????? case 1: vmassert(str == NULL, "expected null"); break; >> >> >> Figuring out a way to detect the failure mode in order to test the >> fix was much more involved than the fix itself. Gory details are >> in the bug: >> >> ???? JDK-8205648 fix for 8205195 breaks secondary error handling >> ???? https://bugs.openjdk.java.net/browse/JDK-8205648 >> >> No webrev since the one liner diff is shown above. This fix was tested >> with the test/hotspot/jtreg/runtime/ErrorHandling tests on Linux-X64 >> and with a special version of ErrorHandling/SecondaryErrorTest.java >> that is documented in JDK-8205648. >> >> Thanks, in advance, for any comments, questions or suggestions. >> >> Dan From mandy.chung at oracle.com Wed Jun 27 15:20:17 2018 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 27 Jun 2018 08:20:17 -0700 Subject: RFR : JDK-8192953 - sun/management/jmxremote/bootstrap/*.sh tests fail with error : revokeall.exe: Permission denied In-Reply-To: References: Message-ID: I would suggest to take the time and replace it with java.nio.file API and remove revokeall.exe sooner rather than later. Mandy On 6/26/18 7:09 AM, Harsha Wardhana B wrote: > Hi All, > > Please find the fix for the bug, > > https://bugs.openjdk.java.net/browse/JDK-8192953 > > having webrev at, > > http://cr.openjdk.java.net/~hb/8192953/webrev.00/ > > The fix grants execute permission for revokeall.exe. The paths in the > shell sciprt had to be converted to cygwin paths (/cygwin/c/... ) from > windows path (C:/...). Using windows path was causing strange behavior > in cygwin. > > revokeall.exe should be removed and the above tests need to be > refactored to use java.nio.Acl* APIs. That plan is in the near future, > and the current fix needs to go in to stop consistent failures in Mach5. > > Please review the above patch and provide feedback if any. > > Thanks > Harsha > From alexey.menkov at oracle.com Wed Jun 27 15:27:19 2018 From: alexey.menkov at oracle.com (Alex Menkov) Date: Wed, 27 Jun 2018 08:27:19 -0700 Subject: RFR: JDK-8205681: [TEST_BUG] vmTestbase/nsk/jvmti/Allocate/alloc001/TestDescription.java fails with exit code 98 Message-ID: Hi all, please review simple fix for https://bugs.openjdk.java.net/browse/JDK-8205681 webrev: http://cr.openjdk.java.net/~amenkov/open_test_env/webrev/ The test was recently open-sourced and path to file in the open repository assumes open repo is cloned into directory named "open". For openjdk-only build root directory can be named differently. The test is now in open repo, so the path can be simplified. --alex From harsha.wardhana.b at oracle.com Wed Jun 27 16:03:44 2018 From: harsha.wardhana.b at oracle.com (Harsha Wardhana B) Date: Wed, 27 Jun 2018 21:33:44 +0530 Subject: RFR : JDK-8192953 - sun/management/jmxremote/bootstrap/*.sh tests fail with error : revokeall.exe: Permission denied In-Reply-To: References: Message-ID: That will be done subsequently and tracked under a different bug. Don't you think pushing this fix is better than quarantining the tests? Harsha On Wednesday 27 June 2018 08:50 PM, mandy chung wrote: > I would suggest to take the time and replace it with java.nio.file API > and remove revokeall.exe sooner rather than later. > > Mandy > > On 6/26/18 7:09 AM, Harsha Wardhana B wrote: >> Hi All, >> >> Please find the fix for the bug, >> >> https://bugs.openjdk.java.net/browse/JDK-8192953 >> >> having webrev at, >> >> http://cr.openjdk.java.net/~hb/8192953/webrev.00/ >> >> The fix grants execute permission for revokeall.exe. The paths in the >> shell sciprt had to be converted to cygwin paths (/cygwin/c/... ) >> from windows path (C:/...). Using windows path was causing strange >> behavior in cygwin. >> >> revokeall.exe should be removed and the above tests need to be >> refactored to use java.nio.Acl* APIs. That plan is in the near >> future, and the current fix needs to go in to stop consistent >> failures in Mach5. >> >> Please review the above patch and provide feedback if any. >> >> Thanks >> Harsha >> From mandy.chung at oracle.com Wed Jun 27 16:22:05 2018 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 27 Jun 2018 09:22:05 -0700 Subject: RFR : JDK-8192953 - sun/management/jmxremote/bootstrap/*.sh tests fail with error : revokeall.exe: Permission denied In-Reply-To: References: Message-ID: <181d45a6-d445-fc11-6026-6a3a59cd8e1c@oracle.com> I think the right thing to do is to bite the bullet and fix the test properly. In addition, this fix does not seem to work if there is no "open" directory. Mandy On 6/27/18 9:03 AM, Harsha Wardhana B wrote: > That will be done subsequently and tracked under a different bug. Don't > you think pushing this fix is better than quarantining the tests? > > Harsha > > On Wednesday 27 June 2018 08:50 PM, mandy chung wrote: >> I would suggest to take the time and replace it with java.nio.file API >> and remove revokeall.exe sooner rather than later. >> >> Mandy >> >> On 6/26/18 7:09 AM, Harsha Wardhana B wrote: >>> Hi All, >>> >>> Please find the fix for the bug, >>> >>> https://bugs.openjdk.java.net/browse/JDK-8192953 >>> >>> having webrev at, >>> >>> http://cr.openjdk.java.net/~hb/8192953/webrev.00/ >>> >>> The fix grants execute permission for revokeall.exe. The paths in the >>> shell sciprt had to be converted to cygwin paths (/cygwin/c/... ) >>> from windows path (C:/...). Using windows path was causing strange >>> behavior in cygwin. >>> >>> revokeall.exe should be removed and the above tests need to be >>> refactored to use java.nio.Acl* APIs. That plan is in the near >>> future, and the current fix needs to go in to stop consistent >>> failures in Mach5. >>> >>> Please review the above patch and provide feedback if any. >>> >>> Thanks >>> Harsha >>> > From harsha.wardhana.b at oracle.com Wed Jun 27 16:28:08 2018 From: harsha.wardhana.b at oracle.com (Harsha Wardhana B) Date: Wed, 27 Jun 2018 21:58:08 +0530 Subject: RFR : JDK-8192953 - sun/management/jmxremote/bootstrap/*.sh tests fail with error : revokeall.exe: Permission denied In-Reply-To: <181d45a6-d445-fc11-6026-6a3a59cd8e1c@oracle.com> References: <181d45a6-d445-fc11-6026-6a3a59cd8e1c@oracle.com> Message-ID: Since the tests are failing in every CI run, we have the option to push this fix or quarantine the tests. Refactoring the tests takes more than a week of effort and tests will have to be quarantined till then. I guess pushing this fix is the right thing to do now. Harsha On Wednesday 27 June 2018 09:52 PM, mandy chung wrote: > I think the right thing to do is to bite the bullet and fix the test > properly. > > In addition, this fix does not seem to work if there is no "open" > directory. > > Mandy > > On 6/27/18 9:03 AM, Harsha Wardhana B wrote: >> That will be done subsequently and tracked under a different bug. >> Don't you think pushing this fix is better than quarantining the tests? >> >> Harsha >> >> On Wednesday 27 June 2018 08:50 PM, mandy chung wrote: >>> I would suggest to take the time and replace it with java.nio.file >>> API and remove revokeall.exe sooner rather than later. >>> >>> Mandy >>> >>> On 6/26/18 7:09 AM, Harsha Wardhana B wrote: >>>> Hi All, >>>> >>>> Please find the fix for the bug, >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8192953 >>>> >>>> having webrev at, >>>> >>>> http://cr.openjdk.java.net/~hb/8192953/webrev.00/ >>>> >>>> The fix grants execute permission for revokeall.exe. The paths in >>>> the shell sciprt had to be converted to cygwin paths (/cygwin/c/... >>>> ) from windows path (C:/...). Using windows path was causing >>>> strange behavior in cygwin. >>>> >>>> revokeall.exe should be removed and the above tests need to be >>>> refactored to use java.nio.Acl* APIs. That plan is in the near >>>> future, and the current fix needs to go in to stop consistent >>>> failures in Mach5. >>>> >>>> Please review the above patch and provide feedback if any. >>>> >>>> Thanks >>>> Harsha >>>> >> From mandy.chung at oracle.com Wed Jun 27 17:08:14 2018 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 27 Jun 2018 10:08:14 -0700 Subject: RFR : JDK-8192953 - sun/management/jmxremote/bootstrap/*.sh tests fail with error : revokeall.exe: Permission denied In-Reply-To: References: <181d45a6-d445-fc11-6026-6a3a59cd8e1c@oracle.com> Message-ID: <12db0189-c6c0-ded8-ee7b-6f487b42b852@oracle.com> What refactoring are you thinking about about? It should be straight-forward to write an utility in java to replace revokeall.exe. As it has been a long-standing testing reliability issue and this is a test-only bug, you have time to fix in 11. Also, your fix does not work if "open" directory does not exist. Mandy On 6/27/18 9:28 AM, Harsha Wardhana B wrote: > Since the tests are failing in every CI run, we have the option to push > this fix or quarantine the tests. Refactoring the tests takes more than > a week of effort and tests will have to be quarantined till then. I > guess pushing this fix is the right thing to do now. > > Harsha > > On Wednesday 27 June 2018 09:52 PM, mandy chung wrote: >> I think the right thing to do is to bite the bullet and fix the test >> properly. >> >> In addition, this fix does not seem to work if there is no "open" >> directory. >> >> Mandy >> >> On 6/27/18 9:03 AM, Harsha Wardhana B wrote: >>> That will be done subsequently and tracked under a different bug. >>> Don't you think pushing this fix is better than quarantining the tests? >>> >>> Harsha >>> >>> On Wednesday 27 June 2018 08:50 PM, mandy chung wrote: >>>> I would suggest to take the time and replace it with java.nio.file >>>> API and remove revokeall.exe sooner rather than later. >>>> >>>> Mandy >>>> >>>> On 6/26/18 7:09 AM, Harsha Wardhana B wrote: >>>>> Hi All, >>>>> >>>>> Please find the fix for the bug, >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8192953 >>>>> >>>>> having webrev at, >>>>> >>>>> http://cr.openjdk.java.net/~hb/8192953/webrev.00/ >>>>> >>>>> The fix grants execute permission for revokeall.exe. The paths in >>>>> the shell sciprt had to be converted to cygwin paths (/cygwin/c/... >>>>> ) from windows path (C:/...). Using windows path was causing >>>>> strange behavior in cygwin. >>>>> >>>>> revokeall.exe should be removed and the above tests need to be >>>>> refactored to use java.nio.Acl* APIs. That plan is in the near >>>>> future, and the current fix needs to go in to stop consistent >>>>> failures in Mach5. >>>>> >>>>> Please review the above patch and provide feedback if any. >>>>> >>>>> Thanks >>>>> Harsha >>>>> >>> > From serguei.spitsyn at oracle.com Wed Jun 27 17:19:30 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 27 Jun 2018 10:19:30 -0700 Subject: RFR: JDK-8205681: [TEST_BUG] vmTestbase/nsk/jvmti/Allocate/alloc001/TestDescription.java fails with exit code 98 In-Reply-To: References: Message-ID: Hi Alex, It looks good. Thanks, Serguei On 6/27/18 08:27, Alex Menkov wrote: > Hi all, > > please review simple fix for > https://bugs.openjdk.java.net/browse/JDK-8205681 > webrev: > http://cr.openjdk.java.net/~amenkov/open_test_env/webrev/ > > The test was recently open-sourced and path to file in the open > repository assumes open repo is cloned into directory named "open". > For openjdk-only build root directory can be named differently. > The test is now in open repo, so the path can be simplified. > > --alex From chris.plummer at oracle.com Wed Jun 27 18:41:04 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 27 Jun 2018 11:41:04 -0700 Subject: RFR(XS): 8205719: Windows Java_sun_tools_attach_VirtualMachineImpl_enqueue() method should include exitCode in exception message In-Reply-To: <19220be6-9168-101b-be49-08a6c7000f0a@oracle.com> References: <30f1ccc6-46ae-bdd6-7460-e69c88d90e7a@oracle.com> <19220be6-9168-101b-be49-08a6c7000f0a@oracle.com> Message-ID: <916cec81-690e-e1fd-0090-c31f7523cefb@oracle.com> Thanks, Serguei! Can I get a second reviewer please? thanks, Chris On 6/26/18 1:25 PM, serguei.spitsyn at oracle.com wrote: > Hi Chris, > > It looks good to me. > > Thanks, > Serguei > > On 6/26/18 12:29, Chris Plummer wrote: >> Hello, >> >> Please review the following: >> >> https://bugs.openjdk.java.net/browse/JDK-8205719 >> http://cr.openjdk.java.net/~cjplummer/8205719/webrev.00/ >> >> I am adding some debugging code to help track down the root cause of >> JDK-8199811. >> >> thanks, >> >> Chris > From chris.plummer at oracle.com Wed Jun 27 19:09:32 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 27 Jun 2018 12:09:32 -0700 Subject: RFR: JDK-8205681: [TEST_BUG] vmTestbase/nsk/jvmti/Allocate/alloc001/TestDescription.java fails with exit code 98 In-Reply-To: References: Message-ID: <14889af4-13c3-8d34-f385-7a363659b2ff@oracle.com> +1 On 6/27/18 10:19 AM, serguei.spitsyn at oracle.com wrote: > Hi Alex, > > It looks good. > > Thanks, > Serguei > > On 6/27/18 08:27, Alex Menkov wrote: >> Hi all, >> >> please review simple fix for >> https://bugs.openjdk.java.net/browse/JDK-8205681 >> webrev: >> http://cr.openjdk.java.net/~amenkov/open_test_env/webrev/ >> >> The test was recently open-sourced and path to file in the open >> repository assumes open repo is cloned into directory named "open". >> For openjdk-only build root directory can be named differently. >> The test is now in open repo, so the path can be simplified. >> >> --alex > From chris.plummer at oracle.com Thu Jun 28 01:16:51 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 27 Jun 2018 18:16:51 -0700 Subject: RFR(XS): 8205719: Windows Java_sun_tools_attach_VirtualMachineImpl_enqueue() method should include exitCode in exception message In-Reply-To: <916cec81-690e-e1fd-0090-c31f7523cefb@oracle.com> References: <30f1ccc6-46ae-bdd6-7460-e69c88d90e7a@oracle.com> <19220be6-9168-101b-be49-08a6c7000f0a@oracle.com> <916cec81-690e-e1fd-0090-c31f7523cefb@oracle.com> Message-ID: <2526bf00-feae-c7a4-e518-14732df30684@oracle.com> Ping! One last request for a second reviewer. I'd really like to get this pushed before the cutoff. It's a p4, so I can't push it to 11 after the cutoff. The changes are trivial. thanks, Chris On 6/27/18 11:41 AM, Chris Plummer wrote: > Thanks, Serguei! > > Can I get a second reviewer please? > > thanks, > > Chris > > On 6/26/18 1:25 PM, serguei.spitsyn at oracle.com wrote: >> Hi Chris, >> >> It looks good to me. >> >> Thanks, >> Serguei >> >> On 6/26/18 12:29, Chris Plummer wrote: >>> Hello, >>> >>> Please review the following: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8205719 >>> http://cr.openjdk.java.net/~cjplummer/8205719/webrev.00/ >>> >>> I am adding some debugging code to help track down the root cause of >>> JDK-8199811. >>> >>> thanks, >>> >>> Chris >> > From daniel.daugherty at oracle.com Thu Jun 28 01:26:51 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 27 Jun 2018 21:26:51 -0400 Subject: RFR(XS): 8205719: Windows Java_sun_tools_attach_VirtualMachineImpl_enqueue() method should include exitCode in exception message In-Reply-To: <30f1ccc6-46ae-bdd6-7460-e69c88d90e7a@oracle.com> References: <30f1ccc6-46ae-bdd6-7460-e69c88d90e7a@oracle.com> Message-ID: On 6/26/18 3:29 PM, Chris Plummer wrote: > Hello, > > Please review the following: > > https://bugs.openjdk.java.net/browse/JDK-8205719 > http://cr.openjdk.java.net/~cjplummer/8205719/webrev.00/ src/jdk.attach/windows/native/libattach/VirtualMachineImpl.c ??? No comments. Thumbs up. Dan > > I am adding some debugging code to help track down the root cause of > JDK-8199811. > > thanks, > > Chris > From chris.plummer at oracle.com Thu Jun 28 02:01:20 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 27 Jun 2018 19:01:20 -0700 Subject: RFR(XS): 8205719: Windows Java_sun_tools_attach_VirtualMachineImpl_enqueue() method should include exitCode in exception message In-Reply-To: References: <30f1ccc6-46ae-bdd6-7460-e69c88d90e7a@oracle.com> Message-ID: On 6/27/18 6:26 PM, Daniel D. Daugherty wrote: > On 6/26/18 3:29 PM, Chris Plummer wrote: >> Hello, >> >> Please review the following: >> >> https://bugs.openjdk.java.net/browse/JDK-8205719 >> http://cr.openjdk.java.net/~cjplummer/8205719/webrev.00/ > > src/jdk.attach/windows/native/libattach/VirtualMachineImpl.c > ??? No comments. > > Thumbs up. Thanks! > > Dan > > >> >> I am adding some debugging code to help track down the root cause of >> JDK-8199811. >> >> thanks, >> >> Chris >> > From harsha.wardhana.b at oracle.com Thu Jun 28 04:24:01 2018 From: harsha.wardhana.b at oracle.com (Harsha Wardhana B) Date: Thu, 28 Jun 2018 09:54:01 +0530 Subject: RFR : JDK-8192953 - sun/management/jmxremote/bootstrap/*.sh tests fail with error : revokeall.exe: Permission denied In-Reply-To: <12db0189-c6c0-ded8-ee7b-6f487b42b852@oracle.com> References: <181d45a6-d445-fc11-6026-6a3a59cd8e1c@oracle.com> <12db0189-c6c0-ded8-ee7b-6f487b42b852@oracle.com> Message-ID: Apart from removing revokeall.exe, all the shell scripts must be converted to Java programs. I have raised JDK-8205972 to track that effort. Harsha On Wednesday 27 June 2018 10:38 PM, mandy chung wrote: > What refactoring are you thinking about about? > > It should be straight-forward to write an utility in java to replace > revokeall.exe.? As it has been a long-standing testing reliability > issue and this is a test-only bug, you have time to fix in 11. > > Also, your fix does not work if "open" directory does not exist. > > Mandy > > On 6/27/18 9:28 AM, Harsha Wardhana B wrote: >> Since the tests are failing in every CI run, we have the option to >> push this fix or quarantine the tests. Refactoring the tests takes >> more than a week of effort and tests will have to be quarantined till >> then. I guess pushing this fix is the right thing to do now. >> >> Harsha >> >> On Wednesday 27 June 2018 09:52 PM, mandy chung wrote: >>> I think the right thing to do is to bite the bullet and fix the test >>> properly. >>> >>> In addition, this fix does not seem to work if there is no "open" >>> directory. >>> >>> Mandy >>> >>> On 6/27/18 9:03 AM, Harsha Wardhana B wrote: >>>> That will be done subsequently and tracked under a different bug. >>>> Don't you think pushing this fix is better than quarantining the >>>> tests? >>>> >>>> Harsha >>>> >>>> On Wednesday 27 June 2018 08:50 PM, mandy chung wrote: >>>>> I would suggest to take the time and replace it with java.nio.file >>>>> API and remove revokeall.exe sooner rather than later. >>>>> >>>>> Mandy >>>>> >>>>> On 6/26/18 7:09 AM, Harsha Wardhana B wrote: >>>>>> Hi All, >>>>>> >>>>>> Please find the fix for the bug, >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8192953 >>>>>> >>>>>> having webrev at, >>>>>> >>>>>> http://cr.openjdk.java.net/~hb/8192953/webrev.00/ >>>>>> >>>>>> The fix grants execute permission for revokeall.exe. The paths in >>>>>> the shell sciprt had to be converted to cygwin paths >>>>>> (/cygwin/c/... ) from windows path (C:/...). Using windows path >>>>>> was causing strange behavior in cygwin. >>>>>> >>>>>> revokeall.exe should be removed and the above tests need to be >>>>>> refactored to use java.nio.Acl* APIs. That plan is in the near >>>>>> future, and the current fix needs to go in to stop consistent >>>>>> failures in Mach5. >>>>>> >>>>>> Please review the above patch and provide feedback if any. >>>>>> >>>>>> Thanks >>>>>> Harsha >>>>>> >>>> >> From david.holmes at oracle.com Thu Jun 28 05:50:18 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 28 Jun 2018 15:50:18 +1000 Subject: RFR : JDK-8192953 - sun/management/jmxremote/bootstrap/*.sh tests fail with error : revokeall.exe: Permission denied In-Reply-To: <12db0189-c6c0-ded8-ee7b-6f487b42b852@oracle.com> References: <181d45a6-d445-fc11-6026-6a3a59cd8e1c@oracle.com> <12db0189-c6c0-ded8-ee7b-6f487b42b852@oracle.com> Message-ID: Hi Mandy, On 28/06/2018 3:08 AM, mandy chung wrote: > What refactoring are you thinking about about? > > It should be straight-forward to write an utility in java to replace > revokeall.exe.? As it has been a long-standing testing reliability issue > and this is a test-only bug, you have time to fix in 11. > > Also, your fix does not work if "open" directory does not exist. There was an updated webrev: http://cr.openjdk.java.net/~hb/8192953/webrev.01/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh.cdiff.html which fixed that - there was actually a duplication of the variable setting that Chris noticed. David > Mandy > > On 6/27/18 9:28 AM, Harsha Wardhana B wrote: >> Since the tests are failing in every CI run, we have the option to >> push this fix or quarantine the tests. Refactoring the tests takes >> more than a week of effort and tests will have to be quarantined till >> then. I guess pushing this fix is the right thing to do now. >> >> Harsha >> >> On Wednesday 27 June 2018 09:52 PM, mandy chung wrote: >>> I think the right thing to do is to bite the bullet and fix the test >>> properly. >>> >>> In addition, this fix does not seem to work if there is no "open" >>> directory. >>> >>> Mandy >>> >>> On 6/27/18 9:03 AM, Harsha Wardhana B wrote: >>>> That will be done subsequently and tracked under a different bug. >>>> Don't you think pushing this fix is better than quarantining the tests? >>>> >>>> Harsha >>>> >>>> On Wednesday 27 June 2018 08:50 PM, mandy chung wrote: >>>>> I would suggest to take the time and replace it with java.nio.file >>>>> API and remove revokeall.exe sooner rather than later. >>>>> >>>>> Mandy >>>>> >>>>> On 6/26/18 7:09 AM, Harsha Wardhana B wrote: >>>>>> Hi All, >>>>>> >>>>>> Please find the fix for the bug, >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8192953 >>>>>> >>>>>> having webrev at, >>>>>> >>>>>> http://cr.openjdk.java.net/~hb/8192953/webrev.00/ >>>>>> >>>>>> The fix grants execute permission for revokeall.exe. The paths in >>>>>> the shell sciprt had to be converted to cygwin paths >>>>>> (/cygwin/c/... ) from windows path (C:/...). Using windows path >>>>>> was causing strange behavior in cygwin. >>>>>> >>>>>> revokeall.exe should be removed and the above tests need to be >>>>>> refactored to use java.nio.Acl* APIs. That plan is in the near >>>>>> future, and the current fix needs to go in to stop consistent >>>>>> failures in Mach5. >>>>>> >>>>>> Please review the above patch and provide feedback if any. >>>>>> >>>>>> Thanks >>>>>> Harsha >>>>>> >>>> >> From david.holmes at oracle.com Thu Jun 28 07:49:42 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 28 Jun 2018 17:49:42 +1000 Subject: RFR: JDK-8205681: [TEST_BUG] vmTestbase/nsk/jvmti/Allocate/alloc001/TestDescription.java fails with exit code 98 In-Reply-To: References: Message-ID: <7bb020a1-da1b-6d86-9187-4d351434b4c8@oracle.com> Looks good to me too! ;-) We should scan for other such occurrences. Thanks, David On 28/06/2018 1:27 AM, Alex Menkov wrote: > Hi all, > > please review simple fix for > https://bugs.openjdk.java.net/browse/JDK-8205681 > webrev: > http://cr.openjdk.java.net/~amenkov/open_test_env/webrev/ > > The test was recently open-sourced and path to file in the open > repository assumes open repo is cloned into directory named "open". > For openjdk-only build root directory can be named differently. > The test is now in open repo, so the path can be simplified. > > --alex From alexey.menkov at oracle.com Thu Jun 28 10:33:37 2018 From: alexey.menkov at oracle.com (Alex Menkov) Date: Thu, 28 Jun 2018 03:33:37 -0700 Subject: RFR: JDK-8205681: [TEST_BUG] vmTestbase/nsk/jvmti/Allocate/alloc001/TestDescription.java fails with exit code 98 In-Reply-To: <7bb020a1-da1b-6d86-9187-4d351434b4c8@oracle.com> References: <7bb020a1-da1b-6d86-9187-4d351434b4c8@oracle.com> Message-ID: I did the scan - haven't detected other issues. --alex On 06/28/2018 00:49, David Holmes wrote: > Looks good to me too! ;-) > > We should scan for other such occurrences. > > Thanks, > David > > On 28/06/2018 1:27 AM, Alex Menkov wrote: >> Hi all, >> >> please review simple fix for >> https://bugs.openjdk.java.net/browse/JDK-8205681 >> webrev: >> http://cr.openjdk.java.net/~amenkov/open_test_env/webrev/ >> >> The test was recently open-sourced and path to file in the open >> repository assumes open repo is cloned into directory named "open". >> For openjdk-only build root directory can be named differently. >> The test is now in open repo, so the path can be simplified. >> >> --alex From yasuenag at gmail.com Thu Jun 28 13:12:28 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Thu, 28 Jun 2018 22:12:28 +0900 Subject: RFR: 8205992: jhsdb cannot attach to Java processes running in Docker containers Message-ID: <5acf45c6-b666-f4f5-2947-8e026701b29a@gmail.com> Hi all, Please review this change. JBS: https://bugs.openjdk.java.net/browse/JDK-8205992 webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8205992/webrev.00/ I tried to attach jhsdb to java process in docker container from container host, but it couldn't. jcmd supports PID namespace in JDK-8193710, but jhsdb hasn't yet. SA gets LWP ID via thread stack and funcs in libthread_db.so, but they returns PIDs in container - they are different from host's PID. So I added the code to scan /proc//task to get all LWP IDs and they are kept in a Map in LinuxDebuggerLocal. Also SA_ALTROOT is set to /proc//root if SA detects debuggee runs in container. It helps SA to parse binaries in container. This change has been pushed to submit repo, and it was failed on OS X (mach5-one-ysuenaga-JDK-8205992-20180628-1015-28963). But I guess it causes JDK-8205906. This change affects to Linux only. Could you review it? Thanks, Yasumasa From gary.adams at oracle.com Thu Jun 28 15:12:22 2018 From: gary.adams at oracle.com (gary.adams at oracle.com) Date: Thu, 28 Jun 2018 11:12:22 -0400 Subject: RFR: JDK-8205508: hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java fails with Prompt is not received during 300200 milliseconds. In-Reply-To: <5B312469.7060206@oracle.com> References: <5B312469.7060206@oracle.com> Message-ID: I'll need a sponsor for the attached patch. It probably should wait for the rdp repos to be forked. I'll file two followup bugs based on the discussions. ??? - one to continue investigation of why the exclude001 test takes so long ??? - one to improve the vmTestbase/nsk tests setting of the jtreg /timeout arg On 6/25/18 1:20 PM, Gary Adams wrote: > The exclude001 test times out on solaris sparc debug builds. > > Basically, this test is all about tracing method calls and introduces > exclude filters to reduce the callbacks to a select set of packages. > The time spent tracing/filtering method callbacks is purely a function > of the number of methods that are processed. On this particularly slow > target platform, more time is needed before issuing a timeout. > > ? Issue: https://bugs.openjdk.java.net/browse/JDK-8205508 > > Proposed fix: > diff --git > a/test/hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.javab/test/hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java > > --- > a/test/hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java > +++ > b/test/hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java > @@ -63,7 +63,7 @@ > ??* nsk.jdb.exclude.exclude001.exclude001a > ??* @run main/othervm PropertyResolvingWrapper > nsk.jdb.exclude.exclude001.exclude001 > ??* -arch=${os.family}-${os.simpleArch} > - * -waittime=5 > + * -waittime=7 > ??* -debugee.vmkind=java > ??* -transport.address=dynamic > ??* -jdb=${test.jdk}/bin/jdb > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- # HG changeset patch # User gadams # Date 1530196299 14400 # Thu Jun 28 10:31:39 2018 -0400 # Node ID 5de4d7b0d1f0b03068adee69b9e552462f388472 # Parent f8c8bfd59e052dedc74b6c030a081094dfd10130 8205508: hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java fails with Prompt is not received during 300200 milliseconds. Reviewed-by: cjplummer, sspitsyn, dcubed diff --git a/test/hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java b/test/hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java --- a/test/hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java +++ b/test/hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java @@ -61,9 +61,9 @@ * @run driver jdk.test.lib.FileInstaller . . * @build nsk.jdb.exclude.exclude001.exclude001 * nsk.jdb.exclude.exclude001.exclude001a - * @run main/othervm PropertyResolvingWrapper nsk.jdb.exclude.exclude001.exclude001 + * @run main/othervm/timeout=420 PropertyResolvingWrapper nsk.jdb.exclude.exclude001.exclude001 * -arch=${os.family}-${os.simpleArch} - * -waittime=5 + * -waittime=7 * -debugee.vmkind=java * -transport.address=dynamic * -jdb=${test.jdk}/bin/jdb From daniel.fuchs at oracle.com Thu Jun 28 15:46:16 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Thu, 28 Jun 2018 17:46:16 +0200 Subject: [11] RFR: 8205653: test/jdk/sun/management/jmxremote/bootstrap/RmiRegistrySslTest.java and RmiSslBootstrapTest.sh fail with handshake_failure In-Reply-To: <3e4af336-6863-4145-9dce-60b08ea64a79@default> References: <3e4af336-6863-4145-9dce-60b08ea64a79@default> Message-ID: <74fabdff-3523-49cc-5ac2-4b766c8bbb30@oracle.com> [ccing serviceability-dev at openjdk.java.net] Hi Siba, This looks good to me - but I'm not a SSL expert. It would be good to get someone from the security team eyeball those changes (Xuelei? Brad?) I added serviceability-dev at openjdk.java.net in cc as this is where reviews for JMX/Monitoring changes happen these days... best regards, -- daniel On 28/06/2018 17:10, Sibabrata Sahoo wrote: > Hi, > > Please review the patch for, > > JBS: https://bugs.openjdk.java.net/browse/JDK-8205653 > > Webrev: http://cr.openjdk.java.net/~ssahoo/8205653/webrev.00/ > > Change: > > The Test has been upgraded to address the following 2 cases, > > 1. Add protocol support for TLSv1.3. The change is done in the config > file named ?management_ssltest11_ok.properties.in?. > 2. Add support for legacy TLS. Now a new config file > ?management_ssltest15_ok.properties.in? hold TLS protocol > ?TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA? instead > of ?SSL_RSA_WITH_RC4_128_SHA,SSL_RSA_WITH_RC4_128_MD5?. > > Previously the Test was using DSA keys which is not compatible with > TLSv1.3. So the keys has been upgraded to use RSA(2048 bit). Hence the > instruction in ?Readme.txt? changed which generates RSA(2048 bit) keys. > > NOTE: Few Test was problem listed which are removed from the list now. > Mach 5 result PASS with multiple try for all 14 Test belongs to > ?open/test/jdk/sun/management/jmxremote? folder. > > Thanks, > > Siba > From xuelei.fan at oracle.com Thu Jun 28 15:57:05 2018 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Thu, 28 Jun 2018 08:57:05 -0700 Subject: [11] RFR: 8205653: test/jdk/sun/management/jmxremote/bootstrap/RmiRegistrySslTest.java and RmiSslBootstrapTest.sh fail with handshake_failure In-Reply-To: <74fabdff-3523-49cc-5ac2-4b766c8bbb30@oracle.com> References: <3e4af336-6863-4145-9dce-60b08ea64a79@default> <74fabdff-3523-49cc-5ac2-4b766c8bbb30@oracle.com> Message-ID: <5764f7f2-f0a7-4bf1-44dc-977953cc6cab@oracle.com> Hi Siba, The change looks fine to me. I would like Serviceability review this change as well. Thanks, Xuelei On 6/28/2018 8:46 AM, Daniel Fuchs wrote: > [ccing serviceability-dev at openjdk.java.net] > > Hi Siba, > > This looks good to me - but I'm not a SSL expert. > It would be good to get someone from the security team eyeball those > changes (Xuelei? Brad?) > > I added serviceability-dev at openjdk.java.net in cc as this is where > reviews for JMX/Monitoring changes happen these days... > > best regards, > > -- daniel > > On 28/06/2018 17:10, Sibabrata Sahoo wrote: >> Hi, >> >> Please review the patch for, >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8205653 >> >> Webrev: http://cr.openjdk.java.net/~ssahoo/8205653/webrev.00/ >> >> Change: >> >> The Test has been upgraded to address the following 2 cases, >> >> ?1. Add protocol support for TLSv1.3. The change is done in the config >> ??? file named ?management_ssltest11_ok.properties.in?. >> ?2. Add support for legacy TLS. Now a new config file >> ??? ?management_ssltest15_ok.properties.in? hold TLS protocol >> ??? ?TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA? instead >> ??? of ?SSL_RSA_WITH_RC4_128_SHA,SSL_RSA_WITH_RC4_128_MD5?. >> >> Previously the Test was using DSA keys which is not compatible with >> TLSv1.3. So the keys has been upgraded to use RSA(2048 bit). Hence the >> instruction in ?Readme.txt? changed which generates RSA(2048 bit) keys. >> >> NOTE: Few Test was problem listed which are removed from the list now. >> Mach 5 result PASS with multiple try for all 14 Test belongs to >> ?open/test/jdk/sun/management/jmxremote? folder. >> >> Thanks, >> >> Siba >> > From serguei.spitsyn at oracle.com Thu Jun 28 16:03:41 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 28 Jun 2018 09:03:41 -0700 Subject: RFR: JDK-8205508: hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java fails with Prompt is not received during 300200 milliseconds. In-Reply-To: References: <5B312469.7060206@oracle.com> Message-ID: <9be0aa5d-1d1b-8575-9f0e-0a72a6bb147c@oracle.com> An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Thu Jun 28 16:07:06 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 28 Jun 2018 09:07:06 -0700 Subject: RFR: JDK-8205508: hotspot/jtreg/vmTestbase/nsk/jdb/exclude/exclude001/exclude001.java fails with Prompt is not received during 300200 milliseconds. In-Reply-To: <9be0aa5d-1d1b-8575-9f0e-0a72a6bb147c@oracle.com> References: <5B312469.7060206@oracle.com> <9be0aa5d-1d1b-8575-9f0e-0a72a6bb147c@oracle.com> Message-ID: <0a7c1222-de14-e54b-8659-8b8bcfda49dc@oracle.com> An HTML attachment was scrubbed... URL: From daniil.x.titov at oracle.com Fri Jun 29 03:57:35 2018 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Thu, 28 Jun 2018 20:57:35 -0700 Subject: RFR 8206086: [Graal] JDI tests fail with com.sun.jdi.ObjectCollectedException Message-ID: Please review the changes that fix 4 JDI tests that fail when running with Graal. The problem here is that in some cases by the time the test decides whether the JDI event should be saved for further analysis the object this event refers to might be already collected in the debuggee VM. The fix ensures that if this happens then ObjectCollectedException is gracefully handled and the event is skipped. Bug: https://bugs.openjdk.java.net/browse/JDK-8206086 Webrev: http://cr.openjdk.java.net/~dtitov/8206086/webrev.001/ Thanks! --Daniil From serguei.spitsyn at oracle.com Fri Jun 29 05:10:02 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 28 Jun 2018 22:10:02 -0700 Subject: RFR 8206086: [Graal] JDI tests fail with com.sun.jdi.ObjectCollectedException In-Reply-To: References: Message-ID: <32102dc6-6110-43d2-2681-ccc443c2a978@oracle.com> Hi Daniil, It looks good to me. Thanks, Serguei On 6/28/18 20:57, Daniil Titov wrote: > Please review the changes that fix 4 JDI tests that fail when running with Graal. > > The problem here is that in some cases by the time the test decides whether the JDI event should be saved for further analysis the object this event refers to might be already collected in the debuggee VM. The fix ensures that if this happens then ObjectCollectedException is gracefully handled and the event is skipped. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8206086 > Webrev: http://cr.openjdk.java.net/~dtitov/8206086/webrev.001/ > > Thanks! > > --Daniil > > From chris.plummer at oracle.com Fri Jun 29 05:53:24 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 28 Jun 2018 22:53:24 -0700 Subject: RFR 8206086: [Graal] JDI tests fail with com.sun.jdi.ObjectCollectedException In-Reply-To: <32102dc6-6110-43d2-2681-ccc443c2a978@oracle.com> References: <32102dc6-6110-43d2-2681-ccc443c2a978@oracle.com> Message-ID: +1 Please add the description of the problem to the CR. thanks, Chris On 6/28/18 10:10 PM, serguei.spitsyn at oracle.com wrote: > Hi Daniil, > > It looks good to me. > > Thanks, > Serguei > > > On 6/28/18 20:57, Daniil Titov wrote: >> Please review the changes that fix 4 JDI tests that fail when running >> with Graal. >> >> The problem here is that in some cases by the time the test decides >> whether the JDI event should be saved for further analysis the object >> this event refers to might be already collected in the debuggee VM. >> The fix ensures that if this happens then ObjectCollectedException is >> gracefully handled and the event is skipped. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8206086 >> Webrev: http://cr.openjdk.java.net/~dtitov/8206086/webrev.001/ >> >> Thanks! >> >> --Daniil >> >> > From Alan.Bateman at oracle.com Fri Jun 29 08:32:38 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 29 Jun 2018 09:32:38 +0100 Subject: [11] RFR: 8205653: test/jdk/sun/management/jmxremote/bootstrap/RmiRegistrySslTest.java and RmiSslBootstrapTest.sh fail with handshake_failure In-Reply-To: <06b1c50c-401c-4c50-9ddf-7876c0638e63@default> References: <3e4af336-6863-4145-9dce-60b08ea64a79@default> <74fabdff-3523-49cc-5ac2-4b766c8bbb30@oracle.com> <5764f7f2-f0a7-4bf1-44dc-977953cc6cab@oracle.com> <06b1c50c-401c-4c50-9ddf-7876c0638e63@default> Message-ID: <687b7dd7-5be5-cfdb-c411-2cf4a7008b12@oracle.com> On 29/06/2018 09:22, Sibabrata Sahoo wrote: > May I get the approval from serviceability-dev at openjdk.java.net. > This a test only change to update the keystores and the list of ciphers/protocols that the test uses. There's nothing serviceability specific here so having a Reviewer from the security area should be okay in the event that don't get a quick review on serviceability-dev list. -Alan From alexey.menkov at oracle.com Fri Jun 29 11:16:15 2018 From: alexey.menkov at oracle.com (Alex Menkov) Date: Fri, 29 Jun 2018 04:16:15 -0700 Subject: RFR 8206086: [Graal] JDI tests fail with com.sun.jdi.ObjectCollectedException In-Reply-To: References: <32102dc6-6110-43d2-2681-ccc443c2a978@oracle.com> Message-ID: +1 --alex On 06/28/2018 22:53, Chris Plummer wrote: > +1 > > Please add the description of the problem to the CR. > > thanks, > > Chris > > On 6/28/18 10:10 PM, serguei.spitsyn at oracle.com wrote: >> Hi Daniil, >> >> It looks good to me. >> >> Thanks, >> Serguei >> >> >> On 6/28/18 20:57, Daniil Titov wrote: >>> Please review the changes that fix 4 JDI tests that fail when running >>> with Graal. >>> >>> The problem here is that in some cases by the time the test decides >>> whether the JDI event should be saved for further analysis the object >>> this event refers to might be already collected in the debuggee VM. >>> The fix ensures that if this happens then ObjectCollectedException is >>> gracefully handled and the event is skipped. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8206086 >>> Webrev: http://cr.openjdk.java.net/~dtitov/8206086/webrev.001/ >>> >>> Thanks! >>> >>> --Daniil >>> >>> >> > From sibabrata.sahoo at oracle.com Fri Jun 29 08:22:20 2018 From: sibabrata.sahoo at oracle.com (Sibabrata Sahoo) Date: Fri, 29 Jun 2018 01:22:20 -0700 (PDT) Subject: [11] RFR: 8205653: test/jdk/sun/management/jmxremote/bootstrap/RmiRegistrySslTest.java and RmiSslBootstrapTest.sh fail with handshake_failure In-Reply-To: <5764f7f2-f0a7-4bf1-44dc-977953cc6cab@oracle.com> References: <3e4af336-6863-4145-9dce-60b08ea64a79@default> <74fabdff-3523-49cc-5ac2-4b766c8bbb30@oracle.com> <5764f7f2-f0a7-4bf1-44dc-977953cc6cab@oracle.com> Message-ID: <06b1c50c-401c-4c50-9ddf-7876c0638e63@default> May I get the approval from serviceability-dev at openjdk.java.net. Thanks, Siba -----Original Message----- From: Xuelei Fan Sent: Thursday, June 28, 2018 9:27 PM To: Daniel Fuchs ; Sibabrata Sahoo ; jmx-dev at openjdk.java.net; security-dev at openjdk.java.net; serviceability-dev at openjdk.java.net serviceability-dev at openjdk.java.net Subject: Re: [11] RFR: 8205653: test/jdk/sun/management/jmxremote/bootstrap/RmiRegistrySslTest.java and RmiSslBootstrapTest.sh fail with handshake_failure Hi Siba, The change looks fine to me. I would like Serviceability review this change as well. Thanks, Xuelei On 6/28/2018 8:46 AM, Daniel Fuchs wrote: > [ccing serviceability-dev at openjdk.java.net] > > Hi Siba, > > This looks good to me - but I'm not a SSL expert. > It would be good to get someone from the security team eyeball those > changes (Xuelei? Brad?) > > I added serviceability-dev at openjdk.java.net in cc as this is where > reviews for JMX/Monitoring changes happen these days... > > best regards, > > -- daniel > > On 28/06/2018 17:10, Sibabrata Sahoo wrote: >> Hi, >> >> Please review the patch for, >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8205653 >> >> Webrev: http://cr.openjdk.java.net/~ssahoo/8205653/webrev.00/ >> >> Change: >> >> The Test has been upgraded to address the following 2 cases, >> >> ?1. Add protocol support for TLSv1.3. The change is done in the >> config >> ??? file named "management_ssltest11_ok.properties.in". >> ?2. Add support for legacy TLS. Now a new config file >> ??? "management_ssltest15_ok.properties.in" hold TLS protocol >> ??? "TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA" >> instead >> ??? of "SSL_RSA_WITH_RC4_128_SHA,SSL_RSA_WITH_RC4_128_MD5". >> >> Previously the Test was using DSA keys which is not compatible with >> TLSv1.3. So the keys has been upgraded to use RSA(2048 bit). Hence >> the instruction in "Readme.txt" changed which generates RSA(2048 bit) keys. >> >> NOTE: Few Test was problem listed which are removed from the list now. >> Mach 5 result PASS with multiple try for all 14 Test belongs to >> "open/test/jdk/sun/management/jmxremote" folder. >> >> Thanks, >> >> Siba >> > From jini.george at oracle.com Fri Jun 29 16:02:02 2018 From: jini.george at oracle.com (Jini George) Date: Fri, 29 Jun 2018 21:32:02 +0530 Subject: [12] RFR (S) 8205534: Remove SymbolTable dependency from serviceability agent In-Reply-To: <88e391a8-78a2-8dbc-a489-fce9c6b922b5@oracle.com> References: <88e391a8-78a2-8dbc-a489-fce9c6b922b5@oracle.com> Message-ID: Hi Coleen, Apologize for the delay. Your changes look good to me overall. A few comments: It might make sense to also remove the corresponding lines in the vmStructs files. Like: File Line vmStructs.cpp 170 typedef RehashableHashtable RehashableSymbolHashtable; vmStructs.cpp 477 static_field(RehashableSymbolHashtable, _seed, juint) \ vmStructs.cpp 1362 declare_type(RehashableSymbolHashtable, BasicHashtable) \ vmStructs.cpp 475 static_field(SymbolTable, _the_table, SymbolTable*) \ vmStructs.cpp 476 static_field(SymbolTable, _shared_table, SymbolCompactHashTable) \ You could also remove the "friend class VMStructs" from the corresponding C++ data types. The test case: test/jdk/sun/tools/jhsdb/AlternateHashingTest.java with the file: test/jdk/sun/tools/jhsdb/LingeredAppWithAltHashing.java were created to test the alternate hashing mechanism of the SymbolTable in SA. Don't know if it makes sense to retain these. One nit: Line 1079 of HeapHprofBinWriter.java: Extra spaces needed. Thanks, Jini. On 6/23/2018 3:10 AM, coleen.phillimore at oracle.com wrote: > Summary: Modify SA code to not use SymbolTable and remove it. > > This is to support the concurrent hashtable for SymbolTable. > > open webrev at http://cr.openjdk.java.net/~coleenp/8205534.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8205534 > > Tested with hs-tier1-5. > > Thanks, > Coleen