From david.holmes at oracle.com Thu Feb 1 01:49:36 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 1 Feb 2018 11:49:36 +1000 Subject: RFR: 8196534: [Testbug] serviceability/dcmd/jvmti/*DcmdTest tests can't tolerate unrelated warnings Message-ID: bug: https://bugs.openjdk.java.net/browse/JDK-8196534 webrev: http://cr.openjdk.java.net/~dholmes/8196534/webrev/ I somehow missed these failures when doing the JDK 11 version change testing. As with the SA tests in JDK-8194067 we need to ignore VM Warnings in stderr. I re-used Sharath's logic from that fix. Will be pushing to jdk/jdk as that is where the version change will go first. Thanks, David From jini.george at oracle.com Thu Feb 1 07:01:48 2018 From: jini.george at oracle.com (Jini George) Date: Thu, 1 Feb 2018 12:31:48 +0530 Subject: RFR: 8196521: serviceability/sa/ClhsdbWhere.java fails to find method 'waitForReferencePendingList' in output In-Reply-To: <71d86c90cba444c6967883078e5c5cc3@NASANEXM01E.na.qualcomm.com> References: <71d86c90cba444c6967883078e5c5cc3@NASANEXM01E.na.qualcomm.com> Message-ID: <4158546c-434b-acd1-c0b1-5e2c7faf680c@oracle.com> Hello Daniel, Your fix looks good to me. You probably could instead add 'remove()' or 'run()' to the list, but I leave it upto you. Thanks, Jini (Not a (R)eviewer). On 2/1/2018 2:39 AM, stewartd.qdt wrote: > Please review this webrev [1] which attempts to fix a test error in > serviceability/sa/ClhsdbWhere.java when it is run under an AArch64 > system (not necessarily exclusive to this system, but it was the system > under test). The bug report [2] provides further details and has the jtr > report that was generated. Essentially the line > ?waitForReferencePendingList? never actually occurs. The test is written > such that it is expecting that line. > > This patch simply removes the line from the set of tested lines it > expects. ?I?m not overly happy with this approach as it actually removes > a test line. However, the test line does not actually appear in the > output (at least on my system) and I?m not sure if there is actually > another way of testing for the intent of this line or if it doesn?t > actually have to appear in the output, depending on the system. Perhaps > the original author could chime in and provide further guidance as to > the intention of the test. > > I am happy to modify the patch as necessary. > > Regards, > > Daniel Stewart > > [1] - http://cr.openjdk.java.net/~dstewart/8196521/webrev.00/ > > [2] - https://bugs.openjdk.java.net/browse/JDK-8196521 > From kirk at kodewerk.com Thu Feb 1 07:15:02 2018 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Thu, 1 Feb 2018 08:15:02 +0100 Subject: RFR (S): 8195115: G1 Old Gen MemoryPool CollectionUsage.used values don't reflect mixed GC results In-Reply-To: References: <3A67BF1B-CE35-4759-B5B4-959C24020A45@amazon.com> <22ceea57-7d27-8350-4457-21f765cb3d0f@oracle.com> <878D6678-84CB-471E-B126-AF14B1BC8EB9@amazon.com> <727f9bcb-9404-df32-917f-70aca6e31cb6@oracle.com> <4BA32239-D645-4B1A-A58E-502258C8FFB7@amazon.com> <997a766f-9f4b-f192-ae87-03f4c8d2666d@oracle.com> <64793ff3-4805-79f9-4e72-6e469cd511dc@oracle.com> <15754d34-f3c7-6431-7abf-05214bd6b9d2@oracle.com> Message-ID: Hi Paul, Is this is targeted for 11? It seems like a positive useful change. My opinion is that if you?re going to break the tool chain.. make sure the change is well worth the pain and get in everything that you want or makes sense or what ever. Just make sure it?s worth the pain. It?s been far too often that this hasn?t been the case. As for this change, bundling values together hides information. This often leaves you either guessing as to what the underlying constraint is or more commonly, one responds to poor performance by expanding the heap size until the underlying supporting data structure is accidently at the a size it needed to be. A poor way of dealing with the issue at hand. Much better would be to understand which resource is stressed and only tune that bit leaving the other stuff alone meaning more information here is better then less. As the moment I calculate tenured size but subtracting young from total. The answer is of course incorrect in that is muxs in other stuff (reserved) but it?s much better to perform analytics on an incorrect tenured than a noisy total heap value. This It would be even better if the breakout was more precise and this work seems to provide a better breakout. As an aside, a number of years ago with I was working on Crays, CRI published a not that their processors could overflow under certain conditions and they wanted to redesign the CPU around the IEEE standard to prevent/correct for this. The resounding response from the Cray users was NO. Reason being, these researchers had years of accumulated data and results and techniques that had been adjusted for the bugs in the Cray CPU. Everyone knew they were there? everyone knew that IEEE was better and a sensible fix to make except when you added in the disruption that would have been caused by the loss of years and years of very valuable historical data? Kind regards, Kirk > On Jan 31, 2018, at 2:30 AM, Hohensee, Paul wrote: > > It?s true that my patch doesn?t completely solve the larger problem, but it fixes the most immediately important part of it, particularly for JDK8 where current expected behavior is entrenched. If we?re going to fix the larger problem, imo we should file another bug/rfe to do it. I?d be happy to fix that one too, once we have a spec. > > What do you think of my suggestions? To summarize: > > - Keep the existing memory pools and add humongous and archive pools. > - Make the archive pool part of the old gen, and generalize it to all collectors. > - Split humongous regions off from the old gen pool into their own pool. The old gen and humongous pools are disjoint region sets. > - Keep the existing ?G1 Young Generation? and ?G1 Old Generation? collectors and their associated memory pools (net of this patch). We add the humongous pool to them. > - Add ?G1 Full? as an alias of the existing ?G1 Old Generation? collector. > - Add the ?G1 Young?, ?G1 Mixed? and ?G1 Concurrent Cycle? collectors. > - Set the G1 old gen memory pool max size to ?Xmx, the archive space max size to whatever it is, and the rest of the G1 memory pool max sizes to -1 == undefined, as now. > > The resulting memory pools: > > ?G1 Eden Space? > ?G1 Survivor Space? > ?G1 Old Gen? > ?G1 Humongous Space? > ?Archive Space? > > The resulting collectors and their memory pools: > > ?G1 Young Generation? (the existing young/mixed collector), ?G1 Old Generation?/?G1 Full?, ?G1 Mixed? > - ?G1 Eden Space? > - ?G1 Survivor Space? > - ?G1 Old Gen? > - ?G1 Humongous Space? > ?G1 Young? > - ?G1 Eden Space? > - ?G1 Survivor Space? > - ?G1 Humongous Space? > ?G1 Concurrent Cycle? > - ?G1 Old Gen? > - ?G1 Humongous Space? > > I?m not religious about the names, but I like my suggestions. :) > > The significant addition to my previous email, and an incompatible change, is splitting humongous regions off from the old gen pool. This means that apps that specifically monitor old gen occupancy will no longer see humongous regions. Monitoring apps that just add up info about all a collector?s pools won?t see a difference. I may be corrected (by Kirk, perhaps), but imo it?s not as bad a compatibility issue as one might think, because the type of app that uses a lot of humongous regions isn?t all that common. E.g., apps that cache data in the heap in the form of large compressed arrays, and apps with large hashmap bucket list arrays. The heaps such apps use are very often large enough to use 32mb regions, hence need really big objects to actually allocate humongous regions. > > Thanks, > > Paul > > On 1/30/18, 5:51 AM, "Erik Helin" wrote: > > On 01/30/2018 03:07 AM, Hohensee, Paul wrote: >> That?s one reviewer who?s ok with a short term patch. Anyone else? And, >> any reviewers for said short term patch? :) > > Well, the patch is not really complete as it is. The problem is the > definitions of the MemoryPoolMXBeans and GarbageCollectorMXBeans, which, > as I tried to hint at in my first email, is a mess for G1. The names and > implementations of these MemoryPoolMXBeans and GarbageCollectionMXBeans > for G1 are very old, G1 has changed a lot since those were implemented > (hence my suggestion for finally fixing this). > > The issue with your patch is that the MemoryPoolMXBean named "G1 Old > Gen" consists of both old and humongous regions (it will also include > archive regions). Old regions can be collected by mixed, concurrent and > full collections. Humongous regions can be collected by young, mixed or > full collections and the concurrent cycle. Archive regions will never be > collected. Your patch will update the pool in the case of a mixed > collection collecting old regions or humongous regions, but misses the > following cases: > - a young collection collecting humongous regions > - a concurrent cycle collecting humongous regions > - a concurrent cycle collecting old regions > > Unfortunately I could not come up with a good way to solve the above > without re-designing the pools. I'm not sure about accepting your patch > as is, since it might cause even more confusion for users compared to > the current (already confusing) situation. > > One idea we have discussed is to implement the re-design but also add a > flag, -XX:+UseG1LegacyPoolsAndBeans (false by default), to allow for a > smoother transition. Would that solution work for you? > > Thanks, > Erik > >> Thanks, >> >> Paul >> >> *From: *mandy chung >> *Organization: *Oracle Corporation >> *Date: *Monday, January 29, 2018 at 1:41 PM >> *To: *"Hohensee, Paul" >> *Cc: *"serviceability-dev at openjdk.java.net" >> , "hotspot-gc-dev at openjdk.java.net" >> >> *Subject: *Re: RFR (S): 8195115: G1 Old Gen MemoryPool >> CollectionUsage.used values don't reflect mixed GC results >> >> I created JDK-8196362 to look into whether it makes sense to provide >> some categorization to differentiate eden space vs the heap space for >> long-lived objects. >> >> W.r.t. to JDK-8195115, I have to defer to GC team to comment on the >> mixed collection update. If they are okay, I have no objection to >> implement a short-term fix and do the proper G1 memory pools as a >> separate patch. >> >> Mandy >> >> On 1/29/18 1:02 PM, Hohensee, Paul wrote: >> >> We don?t use getType, and you guessed correctly: we use the memory >> pool name as an indicator of the specific characteristics of a >> memory pool, in particular eden. >> >> What we want is an indication of long term heap occupancy. We >> calculate it using CollectionUsage for non-eden heap memory pools, >> regardless of collector. We don?t use JMX notification, rather we >> periodically poll CollectionUsage for memory pools with names that >> contain ?Old?, ?Tenured?, or ?Survivor?. We get the memory pools >> from the GarbageCollectorMXBeans (we don?t care what the collector >> names are). For the named memory pools, we sum CollectionUsage.used >> and divide by the sum of CollectionUsage.max to get a long term heap >> occupancy percentage. We don?t want to include eden because it?s >> really just an allocation buffer and not part of the storage for >> long-lived objects. I suppose we could use a negative test instead >> by using all memory pools with names that don?t include ?Eden?. >> >> The bug is that the ?G1 Old Gen? memory pool isn?t being updated >> when the ?G1 Young Generation? collector runs a mixed collection. As >> far as JMX is concerned, that collector only knows about eden and >> the survivor space. The patch adds the old gen to the memory pools >> it knows about and has mixed collections update the old gen?s >> CollectionUsage. >> >> I managed to get a submit repo run to succeed last week and it found >> a problem. I?ve uploaded a new webrev that fixes the failure of the >> jtreg test TestMemoryMXBeansAndPoolsPresence.java due to the young >> gen collector being expected to know only about eden and the >> survivor space. >> >> http://cr.openjdk.java.net/~phh/8195115/webrev.hs.01/ >> >> >> Waiting on the submit repo to come back with a result on it. >> >> Thanks, >> >> Paul >> >> *From: *mandy chung >> >> *Organization: *Oracle Corporation >> *Date: *Monday, January 29, 2018 at 10:52 AM >> *To: *"Hohensee, Paul" >> , Erik Helin >> , David Holmes >> >> *Cc: *"serviceability-dev at openjdk.java.net" >> >> >> , >> "hotspot-gc-dev at openjdk.java.net" >> >> >> >> *Subject: *Re: RFR (S): 8195115: G1 Old Gen MemoryPool >> CollectionUsage.used values don't reflect mixed GC results >> >> On 1/29/18 10:35 AM, mandy chung wrote: >> >> Thanks for the reply Paul. Try to understand a little more on >> the specific from gc-specific memory pool you depend on. >> >> On 1/29/18 8:27 AM, Hohensee, Paul wrote: >> >> A name change would affect Amazon?s heap monitoring, and >> thus I expect it would affect other users as well. >> >> As long as there are gc-specific memory pools, we?re going >> to need to be able to identify them, and right now that?s >> done via name. >> >> >> MemoryPoolMXBean::getType returns "heap" memory type for >> GC-specific memory pools. Are you using this method? Do you >> use the name to build in specific characteristic of a memory >> pool (e.g. eden vs old gen)? >> >> >> >> >> All the mxbeans are identified by name, so that?s a general >> design principle. The only way I can think of to get rid of >> name dependency would be to figure out what abstract metrics >> users want to monitor and implement them for all collectors. >> HeapUsage (instantaneous occupancy) is one, CollectionUsage >> (long-lived occupancy) is another, both of these for the >> entire heap, not just particular memory pools. >> >> >> The sum of HeapUsage and CollectionUsage of all heap memory >> pools was expected to give an incorrect approximation for the >> entire heap usage. Are you seeing issue/bug with the sum result? >> >> >> typo: s/an incorrect approximation/an approximation. >> >> Mandy >> >> >> >> Mandy >> >> >> >> That said, imo there will always be a demand for the ability >> to get collector and memory pool specific details, so I >> don?t see a way to get around providing named entities. >> >> Paul >> >> *From: *mandy chung >> >> *Organization: *Oracle Corporation >> *Date: *Friday, January 26, 2018 at 2:38 PM >> *To: *"Hohensee, Paul" >> , Erik Helin >> , >> David Holmes >> >> *Cc: *"serviceability-dev at openjdk.java.net" >> >> >> , >> "hotspot-gc-dev at openjdk.java.net" >> >> >> >> *Subject: *Re: RFR (S): 8195115: G1 Old Gen MemoryPool >> CollectionUsage.used values don't reflect mixed GC results >> >> On 1/25/18 1:04 PM, Hohensee, Paul wrote: >> >> >>> The JMX API spec doesn?t specify what the memory pool or >> garbage > collector names are, but the current names are >> de-facto part of the > API, so if we change the existing >> ones, imo a CSR should be filed. >> >> The names are implementation details but I can see how an application >> >> might be impacted if they depend on it. CSR approval is not strictly >> >> necessary while I think filing one to document the change would be >> >> good. >> >> Does the name change impact any application you know of? I'm trying to >> >> understand if any improvement to API is needed so that applications >> >> don't need to depend on the names. >> >> >> Mandy >> >> >> >> >> >> >> >> > > From david.holmes at oracle.com Thu Feb 1 07:38:43 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 1 Feb 2018 17:38:43 +1000 Subject: RFR: 8196361: JTReg failure in serviceability/sa/ClhsdbInspect.java In-Reply-To: <3318aed7-f04a-3b92-2660-39cdf13c2d24@oracle.com> References: <6e3441f3c28f4f7387d2174f52283fa7@NASANEXM01E.na.qualcomm.com> <44a71e46-3da2-53c6-7e6b-82658183ae8c@oracle.com> <3318aed7-f04a-3b92-2660-39cdf13c2d24@oracle.com> Message-ID: <55fbc66b-5f0e-4650-c199-f8fdc525c5d7@oracle.com> On 30/01/2018 9:57 PM, Jini George wrote: > Hi Daniel, David, > > Thanks, Daniel, for bringing this up. The intent of the test is to get > the oop address corresponding to a java.lang.ref.ReferenceQueue$Lock, > which can typically be obtained from the stack traces of the > Common-Cleaner or the Finalizer threads. The stack traces which I had > been noticing were typically of the form: > > > "Common-Cleaner" #8 daemon prio=8 tid=0x00007f09c82ac000 nid=0xf6e in > Object.wait() [0x00007f09a18d2000] > ?? java.lang.Thread.State: TIMED_WAITING (on object monitor) > ?? JavaThread state: _thread_blocked > ?- java.lang.Object.wait(long) @bci=0, pc=0x00007f09b7d6480b, > Method*=0x00007f09acc43d60 (Interpreted frame) > ??????? - waiting on <0x000000072e61f6e0> (a > java.lang.ref.ReferenceQueue$Lock) > ?- java.lang.ref.ReferenceQueue.remove(long) @bci=59, line=151, > pc=0x00007f09b7d55243, Method*=0x00007f09acdab9b0 (Interpreted frame) > ??????? - waiting to re-lock in wait() <0x000000072e61f6e0> (a > java.lang.ref.ReferenceQueue$Lock) > ... > > I chose 'waiting to re-lock in wait' since that was what I had been > observing next to the oop address of java.lang.ref.ReferenceQueue$Lock. Actually that output is itself a bug: https://bugs.openjdk.java.net/browse/JDK-8150689 David ----- > But I see how with a timing difference, one could get 'waiting to lock' > as in your case. So, a good way to fix might be to check for the line > containing '(a java.lang.ref.ReferenceQueue$Lock)', getting the oop > address from that line (should be the address appearing immediately > before '(a java.lang.ref.ReferenceQueue$Lock)') and passing that to the > 'inspect' command. > > Thanks much, > Jini. > > On 1/30/2018 3:35 AM, David Holmes wrote: >> Hi Daniel, >> >> Serviceability issues should go to serviceability-dev at openjdk.java.net >> - now cc'd. >> >> On 30/01/2018 7:53 AM, stewartd.qdt wrote: >>> Please review this webrev [1] which attempts to fix a test error in >>> serviceability/sa/ClhsdbInspect.java when it is run under an AArch64 >>> system (not necessarily exclusive to this system, but it was the >>> system under test). The bug report [2] provides further details. >>> Essentially the line "waiting to re-lock in wait" never actually >>> occurs. Instead I have the line "waiting to lock" which occurs for >>> the referenced item of /java/lang/ref/ReferenceQueue$Lock. >>> Unfortunately the test is written such that only the first "waiting >>> to lock" occurrence is seen (for java/lang/Class), which is already >>> accounted for in the test. >> >> I can't tell exactly what the test expects, or why, but it would be >> extremely hard to arrange for "waiting to re-lock in wait" to be seen >> for the ReferenceQueue lock! That requires acquiring the lock >> yourself, issuing a notify() to unblock the wait(), and then issuing >> the jstack command while still holding the lock! >> >> David >> ----- >> >>> I'm not overly happy with this approach as it actually removes a test >>> line. However, the test line does not actually appear in the output >>> (at least on my system) and the test is not currently written to look >>> for the second occurrence of the line "waiting to lock". Perhaps the >>> original author could chime in and provide further guidance as to the >>> intention of the test. >>> >>> I am happy to modify the patch as necessary. >>> >>> Regards, >>> Daniel Stewart >>> >>> >>> [1] -? http://cr.openjdk.java.net/~dstewart/8196361/webrev.00/ >>> [2] - https://bugs.openjdk.java.net/browse/JDK-8196361 >>> From david.holmes at oracle.com Thu Feb 1 07:51:05 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 1 Feb 2018 17:51:05 +1000 Subject: RFR: 8196361: JTReg failure in serviceability/sa/ClhsdbInspect.java In-Reply-To: <24c556061ffb4fde9e87a8806c04c8f7@NASANEXM01E.na.qualcomm.com> References: <6e3441f3c28f4f7387d2174f52283fa7@NASANEXM01E.na.qualcomm.com> <44a71e46-3da2-53c6-7e6b-82658183ae8c@oracle.com> <3318aed7-f04a-3b92-2660-39cdf13c2d24@oracle.com> <24c556061ffb4fde9e87a8806c04c8f7@NASANEXM01E.na.qualcomm.com> Message-ID: <0eca7345-fb57-43e4-6169-c4b3531250e4@oracle.com> Hi Daniel, On 1/02/2018 2:45 AM, stewartd.qdt wrote: > Hi Jini, David, > > Please have a look at the revised webrev: http://cr.openjdk.java.net/~dstewart/8196361/webrev.01/ > > In this webrev I have changed the approach to finding the addresses. This was necessary because in the case of matching for the locks the addresses are before what is matched and in the case of Method the address is after it. The existing code only looked for the addresses after the matched string. I've also tried to align what tokens are being looked for in the lock case. I've taken an approach of breaking the jstack output into lines and then searching each line for it containing what we want. Once found, the line is broken into pieces to find the actual address we want. > > Please let me know if this is an unacceptable approach or any changes you would like to see. I'm not clear on the overall approach as I'm unclear exactly how inspect operates or exactly what the test is trying to verify. One comment on breaking things into lines though: 73 String newline = System.getProperty("line.separator"); 74 String[] lines = jstackOutput.split(newline); As split() takes a regex, I suggest using \R to cover all potential line-breaks, rather than the platform specific line-seperator. We've been recently bitten by the distinction between output that comes from reading a process's stdout/stderr (and for which a newline \n is translated into the platform line-seperator), and output that comes across a socket connection (for which \n is not translated). This could result in failing to parse things correctly on Windows. It's safer/simpler to expect any kind of line-seperator. Thanks, David > Thanks, > Daniel > > > -----Original Message----- > From: Jini George [mailto:jini.george at oracle.com] > Sent: Tuesday, January 30, 2018 6:58 AM > To: David Holmes ; stewartd.qdt > Cc: serviceability-dev ; hotspot-dev at openjdk.java.net > Subject: Re: RFR: 8196361: JTReg failure in serviceability/sa/ClhsdbInspect.java > > Hi Daniel, David, > > Thanks, Daniel, for bringing this up. The intent of the test is to get the oop address corresponding to a java.lang.ref.ReferenceQueue$Lock, > which can typically be obtained from the stack traces of the Common-Cleaner or the Finalizer threads. The stack traces which I had been noticing were typically of the form: > > > "Common-Cleaner" #8 daemon prio=8 tid=0x00007f09c82ac000 nid=0xf6e in > Object.wait() [0x00007f09a18d2000] > java.lang.Thread.State: TIMED_WAITING (on object monitor) > JavaThread state: _thread_blocked > - java.lang.Object.wait(long) @bci=0, pc=0x00007f09b7d6480b, > Method*=0x00007f09acc43d60 (Interpreted frame) > - waiting on <0x000000072e61f6e0> (a > java.lang.ref.ReferenceQueue$Lock) > - java.lang.ref.ReferenceQueue.remove(long) @bci=59, line=151, pc=0x00007f09b7d55243, Method*=0x00007f09acdab9b0 (Interpreted frame) > - waiting to re-lock in wait() <0x000000072e61f6e0> (a > java.lang.ref.ReferenceQueue$Lock) > ... > > I chose 'waiting to re-lock in wait' since that was what I had been observing next to the oop address of java.lang.ref.ReferenceQueue$Lock. > But I see how with a timing difference, one could get 'waiting to lock' > as in your case. So, a good way to fix might be to check for the line containing '(a java.lang.ref.ReferenceQueue$Lock)', getting the oop address from that line (should be the address appearing immediately before '(a java.lang.ref.ReferenceQueue$Lock)') and passing that to the 'inspect' command. > > Thanks much, > Jini. > > On 1/30/2018 3:35 AM, David Holmes wrote: >> Hi Daniel, >> >> Serviceability issues should go to serviceability-dev at openjdk.java.net >> - now cc'd. >> >> On 30/01/2018 7:53 AM, stewartd.qdt wrote: >>> Please review this webrev [1] which attempts to fix a test error in >>> serviceability/sa/ClhsdbInspect.java when it is run under an AArch64 >>> system (not necessarily exclusive to this system, but it was the >>> system under test). The bug report [2] provides further details. >>> Essentially the line "waiting to re-lock in wait" never actually >>> occurs. Instead I have the line "waiting to lock" which occurs for >>> the referenced item of /java/lang/ref/ReferenceQueue$Lock. >>> Unfortunately the test is written such that only the first "waiting to lock" >>> occurrence is seen (for java/lang/Class), which is already accounted >>> for in the test. >> >> I can't tell exactly what the test expects, or why, but it would be >> extremely hard to arrange for "waiting to re-lock in wait" to be seen >> for the ReferenceQueue lock! That requires acquiring the lock >> yourself, issuing a notify() to unblock the wait(), and then issuing >> the jstack command while still holding the lock! >> >> David >> ----- >> >>> I'm not overly happy with this approach as it actually removes a test >>> line. However, the test line does not actually appear in the output >>> (at least on my system) and the test is not currently written to look >>> for the second occurrence of the line "waiting to lock". Perhaps the >>> original author could chime in and provide further guidance as to the >>> intention of the test. >>> >>> I am happy to modify the patch as necessary. >>> >>> Regards, >>> Daniel Stewart >>> >>> >>> [1] -? http://cr.openjdk.java.net/~dstewart/8196361/webrev.00/ >>> [2] - https://bugs.openjdk.java.net/browse/JDK-8196361 >>> From david.holmes at oracle.com Thu Feb 1 07:59:20 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 1 Feb 2018 17:59:20 +1000 Subject: RFR: 8196521: serviceability/sa/ClhsdbWhere.java fails to find method 'waitForReferencePendingList' in output In-Reply-To: <4158546c-434b-acd1-c0b1-5e2c7faf680c@oracle.com> References: <71d86c90cba444c6967883078e5c5cc3@NASANEXM01E.na.qualcomm.com> <4158546c-434b-acd1-c0b1-5e2c7faf680c@oracle.com> Message-ID: <6ab1e27d-cf2a-9397-a970-232fb1ddf523@oracle.com> On 1/02/2018 5:01 PM, Jini George wrote: > Hello Daniel, > > Your fix looks good to me. You probably could instead add 'remove()' or > 'run()' to the list, but I leave it upto you. These tests should only be looking for things that are guaranteed to be seen. If waitForReferencePendingList is not there, how do we know that processPendingReferences will be there? We seem to be assuming a quiescent system. We know the threads that will be present, but the only stack entries we should be looking for are those controlled by the test code itself. David ----- > Thanks, > Jini (Not a (R)eviewer). > > On 2/1/2018 2:39 AM, stewartd.qdt wrote: >> Please review this webrev [1] which attempts to fix a test error in >> serviceability/sa/ClhsdbWhere.java when it is run under an AArch64 >> system (not necessarily exclusive to this system, but it was the >> system under test). The bug report [2] provides further details and >> has the jtr report that was generated. Essentially the line >> ?waitForReferencePendingList? never actually occurs. The test is >> written such that it is expecting that line. >> >> This patch simply removes the line from the set of tested lines it >> expects. ?I?m not overly happy with this approach as it actually >> removes a test line. However, the test line does not actually appear >> in the output (at least on my system) and I?m not sure if there is >> actually another way of testing for the intent of this line or if it >> doesn?t actually have to appear in the output, depending on the >> system. Perhaps the original author could chime in and provide further >> guidance as to the intention of the test. >> >> I am happy to modify the patch as necessary. >> >> Regards, >> >> Daniel Stewart >> >> [1] - http://cr.openjdk.java.net/~dstewart/8196521/webrev.00/ >> >> [2] - https://bugs.openjdk.java.net/browse/JDK-8196521 >> From christoph.langer at sap.com Thu Feb 1 08:11:15 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 1 Feb 2018 08:11:15 +0000 Subject: JDK-8080990: libdt_socket/socket_md.c(202) : warning C4996: 'gethostbyname': Use getaddrinfo() or GetAddrInfoW() In-Reply-To: <5A6A2611.4030003@oracle.com> References: <5A6A2611.4030003@oracle.com> Message-ID: Hi Gary, I was having a look at your changes. I'm wondering what the reason is behind uncommenting WSASendDisconnect in Java_sun_nio_ch_SocketDispatcher_preClose0 of file SocketDispatcher.c? And in dbgsysSocketClose? In socketTransport.c, line: 331 setLastError(0, "gethostbyname: unknown host"); you should change the description text of the error to getaddrinfo instead of gethostbyname. Best regards Christoph -----Original Message----- From: build-dev [mailto:build-dev-bounces at openjdk.java.net] On Behalf Of Gary Adams Sent: Donnerstag, 25. Januar 2018 19:47 To: OpenJDK Serviceability ; OpenJDK Build ; OpenJDK Networking Subject: RFR: JDK-8080990: libdt_socket/socket_md.c(202) : warning C4996: 'gethostbyname': Use getaddrinfo() or GetAddrInfoW() Here's a first pass attempt to remove the -D_WINSOCK_DEPRECATED_NO_WARNINGS build flag and update the winsock deprecated functions. Issue: https://bugs.openjdk.java.net/browse/JDK-8080990 Webrev: http://cr.openjdk.java.net/~gadams/8080990/ These are the initial deprecated functions updated: gethostbyname -> getaddrinfo inet_addr -> inet_pton inet_ntoa -> inet_ntop I'm not sure how to replace the existing WSASendDisconnect calls. It's not clear that it actually provides a graceful disconnect. From gary.adams at oracle.com Thu Feb 1 11:16:49 2018 From: gary.adams at oracle.com (gary.adams at oracle.com) Date: Thu, 1 Feb 2018 06:16:49 -0500 Subject: JDK-8080990: libdt_socket/socket_md.c(202) : warning C4996: 'gethostbyname': Use getaddrinfo() or GetAddrInfoW() In-Reply-To: References: <5A6A2611.4030003@oracle.com> Message-ID: First pass over the code I disabled the compilation flag and then did quick substitution for the easier functions. I commented out the WSASendDisconnect calls so I could see what tests would fail if the function was just removed. I have a replacement now that uses "shutdown(fd,SD_SEND)", but I still have a few more test failures to investigate. I updated the error message text for "getaddrinfo". I'll post a revised webrev after the 4 jshell errors are corrected. On 2/1/18 3:11 AM, Langer, Christoph wrote: > Hi Gary, > > I was having a look at your changes. > > I'm wondering what the reason is behind uncommenting WSASendDisconnect in Java_sun_nio_ch_SocketDispatcher_preClose0 of file SocketDispatcher.c? And in dbgsysSocketClose? > > In socketTransport.c, line: > 331 setLastError(0, "gethostbyname: unknown host"); > you should change the description text of the error to getaddrinfo instead of gethostbyname. > > Best regards > Christoph > > > > -----Original Message----- > From: build-dev [mailto:build-dev-bounces at openjdk.java.net] On Behalf Of Gary Adams > Sent: Donnerstag, 25. Januar 2018 19:47 > To: OpenJDK Serviceability ; OpenJDK Build ; OpenJDK Networking > Subject: RFR: JDK-8080990: libdt_socket/socket_md.c(202) : warning C4996: 'gethostbyname': Use getaddrinfo() or GetAddrInfoW() > > Here's a first pass attempt to remove the -D_WINSOCK_DEPRECATED_NO_WARNINGS > build flag and update the winsock deprecated functions. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8080990 > Webrev: http://cr.openjdk.java.net/~gadams/8080990/ > > These are the initial deprecated functions updated: > gethostbyname -> getaddrinfo > inet_addr -> inet_pton > inet_ntoa -> inet_ntop > > I'm not sure how to replace the existing WSASendDisconnect calls. > It's not clear that it actually provides a graceful disconnect. > > From christoph.langer at sap.com Thu Feb 1 11:59:58 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 1 Feb 2018 11:59:58 +0000 Subject: JDK-8080990: libdt_socket/socket_md.c(202) : warning C4996: 'gethostbyname': Use getaddrinfo() or GetAddrInfoW() In-Reply-To: References: <5A6A2611.4030003@oracle.com> Message-ID: But WSASendDisconnect isn't deprecated, right? So you wanted to get rid of it? I still don't see the reason... -----Original Message----- From: gary.adams at oracle.com [mailto:gary.adams at oracle.com] Sent: Donnerstag, 1. Februar 2018 12:17 To: Langer, Christoph ; OpenJDK Serviceability ; OpenJDK Build ; OpenJDK Networking Subject: Re: JDK-8080990: libdt_socket/socket_md.c(202) : warning C4996: 'gethostbyname': Use getaddrinfo() or GetAddrInfoW() First pass over the code I disabled the compilation flag and then did quick substitution for the easier functions. I commented out the WSASendDisconnect calls so I could see what tests would fail if the function was just removed. I have a replacement now that uses "shutdown(fd,SD_SEND)", but I still have a few more test failures to investigate. I updated the error message text for "getaddrinfo". I'll post a revised webrev after the 4 jshell errors are corrected. On 2/1/18 3:11 AM, Langer, Christoph wrote: > Hi Gary, > > I was having a look at your changes. > > I'm wondering what the reason is behind uncommenting WSASendDisconnect in Java_sun_nio_ch_SocketDispatcher_preClose0 of file SocketDispatcher.c? And in dbgsysSocketClose? > > In socketTransport.c, line: > 331 setLastError(0, "gethostbyname: unknown host"); > you should change the description text of the error to getaddrinfo instead of gethostbyname. > > Best regards > Christoph > > > > -----Original Message----- > From: build-dev [mailto:build-dev-bounces at openjdk.java.net] On Behalf Of Gary Adams > Sent: Donnerstag, 25. Januar 2018 19:47 > To: OpenJDK Serviceability ; OpenJDK Build ; OpenJDK Networking > Subject: RFR: JDK-8080990: libdt_socket/socket_md.c(202) : warning C4996: 'gethostbyname': Use getaddrinfo() or GetAddrInfoW() > > Here's a first pass attempt to remove the -D_WINSOCK_DEPRECATED_NO_WARNINGS > build flag and update the winsock deprecated functions. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8080990 > Webrev: http://cr.openjdk.java.net/~gadams/8080990/ > > These are the initial deprecated functions updated: > gethostbyname -> getaddrinfo > inet_addr -> inet_pton > inet_ntoa -> inet_ntop > > I'm not sure how to replace the existing WSASendDisconnect calls. > It's not clear that it actually provides a graceful disconnect. > > From gary.adams at oracle.com Thu Feb 1 13:21:27 2018 From: gary.adams at oracle.com (Gary Adams) Date: Thu, 01 Feb 2018 08:21:27 -0500 Subject: JDK-8080990: libdt_socket/socket_md.c(202) : warning C4996: 'gethostbyname': Use getaddrinfo() or GetAddrInfoW() In-Reply-To: References: <5A6A2611.4030003@oracle.com> Message-ID: <5A731457.1040502@oracle.com> On 2/1/18, 6:59 AM, Langer, Christoph wrote: > But WSASendDisconnect isn't deprecated, right? So you wanted to get rid of it? I still don't see the reason... vs2013 include/um/winsock2.h has WSASendDisconnect deprecated. .../src/jdk.jdwp.agent/windows/native/libdt_socket/socket_md.c(230) : error C2220: warning treated as error - no 'object' file generated .../src/jdk.jdwp.agent/windows/native/libdt_socket/socket_md.c(230) : warning C4996: 'WSASendDisconnect': Use WSASend() instead or define _WINSOCK_DEPRECATED_NO_WARNINGS to disable deprecated API warnings From yasuenag at gmail.com Thu Feb 1 13:58:30 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Thu, 1 Feb 2018 22:58:30 +0900 Subject: PING: RFR: JDK-8153333: [REDO] STW phases at Concurrent GC should count in PerfCounter In-Reply-To: References: <2f4e0901-1602-6276-e7fd-84e168e7b317@gmail.com> Message-ID: PING: Could you review and sponsor it? >> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ This change has been passed Mach 5 via submit repo: http://java.se.oracle.com:10065/mdash/jobs/mach5-one-ysuenaga-JDK-8153333-20180201-0805-10101 Thanks, Yasumasa On 2017/11/01 22:02, Yasumasa Suenaga wrote: > PING: Could you review and sponsor it? > >> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ > > Also I need JPRT results of this change. > Could you cooperate? > > > Thanks, > > Yasumasa > > > On 2017/09/27 0:08, Yasumasa Suenaga wrote: >> Hi all, >> >> I uploaded new webrev to be adapted to jdk10/hs: >> >> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >> >> I want to check this patch via JPRT, but I cannot access it. >> Could you cooperate? >> >> >> Thanks, >> >> yasumasa >> >> >> On 2017/09/21 7:46, Yasumasa Suenaga wrote: >>> PING: >>> >>> Have you checked this issue? >>> >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/hotspot/ >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/jdk/ >>> >>> >>> Yasumasa >>> >>> >>> On 2017/07/01 23:44, Yasumasa Suenaga wrote: >>>> PING: >>>> >>>> Have you checked this issue? >>>> >>>> >>>> Yasumasa >>>> >>>> >>>> On 2017/06/14 13:22, Yasumasa Suenaga wrote: >>>>> Hi all, >>>>> >>>>> I changed PerfCounter to show CGC STW phase in jstat in JDK-8151674. >>>>> However, it occurred several jtreg test failure, so it was back-outed. >>>>> >>>>> I want to resume to work for this issue. >>>>> >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/hotspot/ >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/jdk/ >>>>> >>>>> These changes are work fine on jtreg test as below: >>>>> >>>>> ?? hotspot/test/serviceability/tmtools/jstat >>>>> ?? jdk/test/sun/tools >>>>> >>>>> >>>>> Since JDK 9, default GC algorithm is set to G1. >>>>> So I think this change is useful to watch GC behavior through jstat. >>>>> >>>>> I cannot access JPRT. Could you help? >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> From stewartd.qdt at qualcommdatacenter.com Thu Feb 1 14:54:48 2018 From: stewartd.qdt at qualcommdatacenter.com (stewartd.qdt) Date: Thu, 1 Feb 2018 14:54:48 +0000 Subject: RFR: 8196361: JTReg failure in serviceability/sa/ClhsdbInspect.java In-Reply-To: <0eca7345-fb57-43e4-6169-c4b3531250e4@oracle.com> References: <6e3441f3c28f4f7387d2174f52283fa7@NASANEXM01E.na.qualcomm.com> <44a71e46-3da2-53c6-7e6b-82658183ae8c@oracle.com> <3318aed7-f04a-3b92-2660-39cdf13c2d24@oracle.com> <24c556061ffb4fde9e87a8806c04c8f7@NASANEXM01E.na.qualcomm.com> <0eca7345-fb57-43e4-6169-c4b3531250e4@oracle.com> Message-ID: <0ee09169513f46509ab002337d9e41df@NASANEXM01E.na.qualcomm.com> David, Thanks for the review. I'll change the split() to look for '\r' instead. I was unaware of the problems with line.separator, and was actually trying to avoid cross-platform issues by using it. But things are always more complicated than they seem! As far as the original intent of the test and how inspect operates, I will defer to the original author. I was just trying to get the same information, but had to change the original search approach because the original approach assumed the addresses came _after_ the string that was being searched. In searching for the name of the actual class instead of just "waiting to lock", the address comes before the string. I assume the classes I am searching for is correct, as these are the classes that actually get found in the original approach and the classes that seem to be looked for by the subsequent OutputAnalyzer. Daniel -----Original Message----- From: David Holmes [mailto:david.holmes at oracle.com] Sent: Thursday, February 1, 2018 2:51 AM To: stewartd.qdt ; Jini George Cc: serviceability-dev ; hotspot-dev at openjdk.java.net Subject: Re: RFR: 8196361: JTReg failure in serviceability/sa/ClhsdbInspect.java Hi Daniel, On 1/02/2018 2:45 AM, stewartd.qdt wrote: > Hi Jini, David, > > Please have a look at the revised webrev: > http://cr.openjdk.java.net/~dstewart/8196361/webrev.01/ > > In this webrev I have changed the approach to finding the addresses. This was necessary because in the case of matching for the locks the addresses are before what is matched and in the case of Method the address is after it. The existing code only looked for the addresses after the matched string. I've also tried to align what tokens are being looked for in the lock case. I've taken an approach of breaking the jstack output into lines and then searching each line for it containing what we want. Once found, the line is broken into pieces to find the actual address we want. > > Please let me know if this is an unacceptable approach or any changes you would like to see. I'm not clear on the overall approach as I'm unclear exactly how inspect operates or exactly what the test is trying to verify. One comment on breaking things into lines though: 73 String newline = System.getProperty("line.separator"); 74 String[] lines = jstackOutput.split(newline); As split() takes a regex, I suggest using \R to cover all potential line-breaks, rather than the platform specific line-seperator. We've been recently bitten by the distinction between output that comes from reading a process's stdout/stderr (and for which a newline \n is translated into the platform line-seperator), and output that comes across a socket connection (for which \n is not translated). This could result in failing to parse things correctly on Windows. It's safer/simpler to expect any kind of line-seperator. Thanks, David > Thanks, > Daniel > > > -----Original Message----- > From: Jini George [mailto:jini.george at oracle.com] > Sent: Tuesday, January 30, 2018 6:58 AM > To: David Holmes ; stewartd.qdt > > Cc: serviceability-dev ; > hotspot-dev at openjdk.java.net > Subject: Re: RFR: 8196361: JTReg failure in > serviceability/sa/ClhsdbInspect.java > > Hi Daniel, David, > > Thanks, Daniel, for bringing this up. The intent of the test is to get > the oop address corresponding to a java.lang.ref.ReferenceQueue$Lock, > which can typically be obtained from the stack traces of the Common-Cleaner or the Finalizer threads. The stack traces which I had been noticing were typically of the form: > > > "Common-Cleaner" #8 daemon prio=8 tid=0x00007f09c82ac000 nid=0xf6e in > Object.wait() [0x00007f09a18d2000] > java.lang.Thread.State: TIMED_WAITING (on object monitor) > JavaThread state: _thread_blocked > - java.lang.Object.wait(long) @bci=0, pc=0x00007f09b7d6480b, > Method*=0x00007f09acc43d60 (Interpreted frame) > - waiting on <0x000000072e61f6e0> (a > java.lang.ref.ReferenceQueue$Lock) > - java.lang.ref.ReferenceQueue.remove(long) @bci=59, line=151, pc=0x00007f09b7d55243, Method*=0x00007f09acdab9b0 (Interpreted frame) > - waiting to re-lock in wait() <0x000000072e61f6e0> (a > java.lang.ref.ReferenceQueue$Lock) > ... > > I chose 'waiting to re-lock in wait' since that was what I had been observing next to the oop address of java.lang.ref.ReferenceQueue$Lock. > But I see how with a timing difference, one could get 'waiting to lock' > as in your case. So, a good way to fix might be to check for the line containing '(a java.lang.ref.ReferenceQueue$Lock)', getting the oop address from that line (should be the address appearing immediately before '(a java.lang.ref.ReferenceQueue$Lock)') and passing that to the 'inspect' command. > > Thanks much, > Jini. > > On 1/30/2018 3:35 AM, David Holmes wrote: >> Hi Daniel, >> >> Serviceability issues should go to >> serviceability-dev at openjdk.java.net >> - now cc'd. >> >> On 30/01/2018 7:53 AM, stewartd.qdt wrote: >>> Please review this webrev [1] which attempts to fix a test error in >>> serviceability/sa/ClhsdbInspect.java when it is run under an AArch64 >>> system (not necessarily exclusive to this system, but it was the >>> system under test). The bug report [2] provides further details. >>> Essentially the line "waiting to re-lock in wait" never actually >>> occurs. Instead I have the line "waiting to lock" which occurs for >>> the referenced item of /java/lang/ref/ReferenceQueue$Lock. >>> Unfortunately the test is written such that only the first "waiting to lock" >>> occurrence is seen (for java/lang/Class), which is already accounted >>> for in the test. >> >> I can't tell exactly what the test expects, or why, but it would be >> extremely hard to arrange for "waiting to re-lock in wait" to be seen >> for the ReferenceQueue lock! That requires acquiring the lock >> yourself, issuing a notify() to unblock the wait(), and then issuing >> the jstack command while still holding the lock! >> >> David >> ----- >> >>> I'm not overly happy with this approach as it actually removes a >>> test line. However, the test line does not actually appear in the >>> output (at least on my system) and the test is not currently written >>> to look for the second occurrence of the line "waiting to lock". >>> Perhaps the original author could chime in and provide further >>> guidance as to the intention of the test. >>> >>> I am happy to modify the patch as necessary. >>> >>> Regards, >>> Daniel Stewart >>> >>> >>> [1] -? http://cr.openjdk.java.net/~dstewart/8196361/webrev.00/ >>> [2] - https://bugs.openjdk.java.net/browse/JDK-8196361 >>> From gary.adams at oracle.com Thu Feb 1 15:27:21 2018 From: gary.adams at oracle.com (Gary Adams) Date: Thu, 01 Feb 2018 10:27:21 -0500 Subject: JDK-8080990: libdt_socket/socket_md.c(202) : warning C4996: 'gethostbyname': Use getaddrinfo() or GetAddrInfoW() In-Reply-To: References: <5A6A2611.4030003@oracle.com> Message-ID: <5A7331D9.9000800@oracle.com> Here's a revised webrev http://cr.openjdk.java.net/~gadams/8080990/webrev.01/index.html Still testing ... Using shutdown() fixed problems reported by the java/nio/channelSocketChannel tests. I also noticed prior use of getaddrinfo for "localhost" was not calling freeaddrinfo. ... On 2/1/18, 6:16 AM, gary.adams at oracle.com wrote: > First pass over the code I disabled the compilation flag and then > did quick substitution for the easier functions. I commented out the > WSASendDisconnect calls so I could see what tests would fail if > the function was just removed. I have a replacement now that uses > "shutdown(fd,SD_SEND)", but I still have a few more test failures to > investigate. > > I updated the error message text for "getaddrinfo". > > I'll post a revised webrev after the 4 jshell errors are corrected. > > > On 2/1/18 3:11 AM, Langer, Christoph wrote: >> Hi Gary, >> >> I was having a look at your changes. >> >> I'm wondering what the reason is behind uncommenting >> WSASendDisconnect in Java_sun_nio_ch_SocketDispatcher_preClose0 of >> file SocketDispatcher.c? And in dbgsysSocketClose? >> >> In socketTransport.c, line: >> 331 setLastError(0, "gethostbyname: unknown host"); >> you should change the description text of the error to getaddrinfo >> instead of gethostbyname. >> >> Best regards >> Christoph >> >> >> >> -----Original Message----- >> From: build-dev [mailto:build-dev-bounces at openjdk.java.net] On Behalf >> Of Gary Adams >> Sent: Donnerstag, 25. Januar 2018 19:47 >> To: OpenJDK Serviceability ; >> OpenJDK Build ; OpenJDK Networking >> >> Subject: RFR: JDK-8080990: libdt_socket/socket_md.c(202) : warning >> C4996: 'gethostbyname': Use getaddrinfo() or GetAddrInfoW() >> >> Here's a first pass attempt to remove the >> -D_WINSOCK_DEPRECATED_NO_WARNINGS >> build flag and update the winsock deprecated functions. >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-8080990 >> Webrev: http://cr.openjdk.java.net/~gadams/8080990/ >> >> These are the initial deprecated functions updated: >> gethostbyname -> getaddrinfo >> inet_addr -> inet_pton >> inet_ntoa -> inet_ntop >> >> I'm not sure how to replace the existing WSASendDisconnect calls. >> It's not clear that it actually provides a graceful disconnect. >> >> > From stewartd.qdt at qualcommdatacenter.com Thu Feb 1 15:50:56 2018 From: stewartd.qdt at qualcommdatacenter.com (stewartd.qdt) Date: Thu, 1 Feb 2018 15:50:56 +0000 Subject: RFR: 8196361: JTReg failure in serviceability/sa/ClhsdbInspect.java In-Reply-To: <0eca7345-fb57-43e4-6169-c4b3531250e4@oracle.com> References: <6e3441f3c28f4f7387d2174f52283fa7@NASANEXM01E.na.qualcomm.com> <44a71e46-3da2-53c6-7e6b-82658183ae8c@oracle.com> <3318aed7-f04a-3b92-2660-39cdf13c2d24@oracle.com> <24c556061ffb4fde9e87a8806c04c8f7@NASANEXM01E.na.qualcomm.com> <0eca7345-fb57-43e4-6169-c4b3531250e4@oracle.com> Message-ID: <4f46f527c17f4d988e4b46e14f93cd4d@NASANEXM01E.na.qualcomm.com> Please have a look at the newest changes at: http://cr.openjdk.java.net/~dstewart/8196361/webrev.02/ The only difference between this and the last changeset is the use of "\\R" instead of whatever is the platform line.separator. Thank you, Daniel -----Original Message----- From: David Holmes [mailto:david.holmes at oracle.com] Sent: Thursday, February 1, 2018 2:51 AM To: stewartd.qdt ; Jini George Cc: serviceability-dev ; hotspot-dev at openjdk.java.net Subject: Re: RFR: 8196361: JTReg failure in serviceability/sa/ClhsdbInspect.java Hi Daniel, On 1/02/2018 2:45 AM, stewartd.qdt wrote: > Hi Jini, David, > > Please have a look at the revised webrev: > http://cr.openjdk.java.net/~dstewart/8196361/webrev.01/ > > In this webrev I have changed the approach to finding the addresses. This was necessary because in the case of matching for the locks the addresses are before what is matched and in the case of Method the address is after it. The existing code only looked for the addresses after the matched string. I've also tried to align what tokens are being looked for in the lock case. I've taken an approach of breaking the jstack output into lines and then searching each line for it containing what we want. Once found, the line is broken into pieces to find the actual address we want. > > Please let me know if this is an unacceptable approach or any changes you would like to see. I'm not clear on the overall approach as I'm unclear exactly how inspect operates or exactly what the test is trying to verify. One comment on breaking things into lines though: 73 String newline = System.getProperty("line.separator"); 74 String[] lines = jstackOutput.split(newline); As split() takes a regex, I suggest using \R to cover all potential line-breaks, rather than the platform specific line-seperator. We've been recently bitten by the distinction between output that comes from reading a process's stdout/stderr (and for which a newline \n is translated into the platform line-seperator), and output that comes across a socket connection (for which \n is not translated). This could result in failing to parse things correctly on Windows. It's safer/simpler to expect any kind of line-seperator. Thanks, David > Thanks, > Daniel > > > -----Original Message----- > From: Jini George [mailto:jini.george at oracle.com] > Sent: Tuesday, January 30, 2018 6:58 AM > To: David Holmes ; stewartd.qdt > > Cc: serviceability-dev ; > hotspot-dev at openjdk.java.net > Subject: Re: RFR: 8196361: JTReg failure in > serviceability/sa/ClhsdbInspect.java > > Hi Daniel, David, > > Thanks, Daniel, for bringing this up. The intent of the test is to get > the oop address corresponding to a java.lang.ref.ReferenceQueue$Lock, > which can typically be obtained from the stack traces of the Common-Cleaner or the Finalizer threads. The stack traces which I had been noticing were typically of the form: > > > "Common-Cleaner" #8 daemon prio=8 tid=0x00007f09c82ac000 nid=0xf6e in > Object.wait() [0x00007f09a18d2000] > java.lang.Thread.State: TIMED_WAITING (on object monitor) > JavaThread state: _thread_blocked > - java.lang.Object.wait(long) @bci=0, pc=0x00007f09b7d6480b, > Method*=0x00007f09acc43d60 (Interpreted frame) > - waiting on <0x000000072e61f6e0> (a > java.lang.ref.ReferenceQueue$Lock) > - java.lang.ref.ReferenceQueue.remove(long) @bci=59, line=151, pc=0x00007f09b7d55243, Method*=0x00007f09acdab9b0 (Interpreted frame) > - waiting to re-lock in wait() <0x000000072e61f6e0> (a > java.lang.ref.ReferenceQueue$Lock) > ... > > I chose 'waiting to re-lock in wait' since that was what I had been observing next to the oop address of java.lang.ref.ReferenceQueue$Lock. > But I see how with a timing difference, one could get 'waiting to lock' > as in your case. So, a good way to fix might be to check for the line containing '(a java.lang.ref.ReferenceQueue$Lock)', getting the oop address from that line (should be the address appearing immediately before '(a java.lang.ref.ReferenceQueue$Lock)') and passing that to the 'inspect' command. > > Thanks much, > Jini. > > On 1/30/2018 3:35 AM, David Holmes wrote: >> Hi Daniel, >> >> Serviceability issues should go to >> serviceability-dev at openjdk.java.net >> - now cc'd. >> >> On 30/01/2018 7:53 AM, stewartd.qdt wrote: >>> Please review this webrev [1] which attempts to fix a test error in >>> serviceability/sa/ClhsdbInspect.java when it is run under an AArch64 >>> system (not necessarily exclusive to this system, but it was the >>> system under test). The bug report [2] provides further details. >>> Essentially the line "waiting to re-lock in wait" never actually >>> occurs. Instead I have the line "waiting to lock" which occurs for >>> the referenced item of /java/lang/ref/ReferenceQueue$Lock. >>> Unfortunately the test is written such that only the first "waiting to lock" >>> occurrence is seen (for java/lang/Class), which is already accounted >>> for in the test. >> >> I can't tell exactly what the test expects, or why, but it would be >> extremely hard to arrange for "waiting to re-lock in wait" to be seen >> for the ReferenceQueue lock! That requires acquiring the lock >> yourself, issuing a notify() to unblock the wait(), and then issuing >> the jstack command while still holding the lock! >> >> David >> ----- >> >>> I'm not overly happy with this approach as it actually removes a >>> test line. However, the test line does not actually appear in the >>> output (at least on my system) and the test is not currently written >>> to look for the second occurrence of the line "waiting to lock". >>> Perhaps the original author could chime in and provide further >>> guidance as to the intention of the test. >>> >>> I am happy to modify the patch as necessary. >>> >>> Regards, >>> Daniel Stewart >>> >>> >>> [1] -? http://cr.openjdk.java.net/~dstewart/8196361/webrev.00/ >>> [2] - https://bugs.openjdk.java.net/browse/JDK-8196361 >>> From serguei.spitsyn at oracle.com Thu Feb 1 19:00:36 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 1 Feb 2018 11:00:36 -0800 Subject: 8161605 : The '!UseSharedSpaces' check is not need in JvmtiManageCapabilities::recompute_always_capabilities In-Reply-To: References: Message-ID: <1c8e3f0e-8aa1-574d-f24f-b78b0dc3e6bd@oracle.com> One more attempt to send it with the correct to-list... Sorry for the noise. Hi Alex, It looks good in general. A couple of comments on the test. We had a convention to avoid having test sub-folders with bug numbers. Could you, please, rename it to some symbolic name? Also we need the test to fail in the native agent when an error is reported with the reportError(). It'd be great if some CDS/AppCDS experts from the Runtime team look at the fix and test so I've included Jiangli and Misha into the to-list. Thanks, Serguei On 1/25/18 16:11, Alex Menkov wrote: > Hi all, > > Please take a look at a fix for > JDK-8161605 : The '!UseSharedSpaces' check is not need in > JvmtiManageCapabilities::recompute_always_capabilities > > The fix moves can_generate_all_class_hook_events capability from > "onload_capabilities" to "always_capabilities" and cleans up > recompute_always_capabilities method. > > jira: https://bugs.openjdk.java.net/browse/JDK-8161605 > webrev: > http://cr.openjdk.java.net/~amenkov/can_generate_class_hook/webrev/ > > --alex From serguei.spitsyn at oracle.com Thu Feb 1 18:56:59 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 1 Feb 2018 10:56:59 -0800 Subject: 8161605 : The '!UseSharedSpaces' check is not need in JvmtiManageCapabilities::recompute_always_capabilities In-Reply-To: References: Message-ID: <94f956e4-e878-292b-78bb-abde26777367@oracle.com> Hi Alex, It looks good in general. A couple of comments on the test. We had a convention to avoid having test sub-folders with bug numbers. Could you, please, rename it to some symbolic name? Also we need the test to fail in the native agent when an error is reported with the reportError(). It'd be great if some CDS/AppCDS experts from the Runtime team look at the fix and test so I've included Jiangli and Misha into the to-list. Thanks, Serguei On 1/25/18 16:11, Alex Menkov wrote: > Hi all, > > Please take a look at a fix for > JDK-8161605 : The '!UseSharedSpaces' check is not need in > JvmtiManageCapabilities::recompute_always_capabilities > > The fix moves can_generate_all_class_hook_events capability from > "onload_capabilities" to "always_capabilities" and cleans up > recompute_always_capabilities method. > > jira: https://bugs.openjdk.java.net/browse/JDK-8161605 > webrev: > http://cr.openjdk.java.net/~amenkov/can_generate_class_hook/webrev/ > > --alex From serguei.spitsyn at oracle.com Thu Feb 1 20:16:00 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 1 Feb 2018 12:16:00 -0800 Subject: RFR: 8196534: [Testbug] serviceability/dcmd/jvmti/*DcmdTest tests can't tolerate unrelated warnings In-Reply-To: References: Message-ID: <10a481dc-b7a2-5e89-92e9-108b15976c76@oracle.com> Hi David, Looks good. Thank you for taking care about it! Thanks, Serguei On 1/31/18 17:49, David Holmes wrote: > bug: https://bugs.openjdk.java.net/browse/JDK-8196534 > webrev: http://cr.openjdk.java.net/~dholmes/8196534/webrev/ > > I somehow missed these failures when doing the JDK 11 version change > testing. As with the SA tests in JDK-8194067 we need to ignore VM > Warnings in stderr. I re-used Sharath's logic from that fix. > > Will be pushing to jdk/jdk as that is where the version change will go > first. > > Thanks, > David From alexey.menkov at oracle.com Fri Feb 2 00:11:36 2018 From: alexey.menkov at oracle.com (Alex Menkov) Date: Thu, 1 Feb 2018 16:11:36 -0800 Subject: 8161605 : The '!UseSharedSpaces' check is not need in JvmtiManageCapabilities::recompute_always_capabilities In-Reply-To: <1c8e3f0e-8aa1-574d-f24f-b78b0dc3e6bd@oracle.com> References: <1c8e3f0e-8aa1-574d-f24f-b78b0dc3e6bd@oracle.com> Message-ID: Hi Serguei, Thanks for the feedback. Updated fix: http://cr.openjdk.java.net/~amenkov/can_generate_class_hook/webrev.02/ --alex On 02/01/2018 11:00, serguei.spitsyn at oracle.com wrote: > One more attempt to send it with the correct to-list... > Sorry for the noise. > > > Hi Alex, > > It looks good in general. > A couple of comments on the test. > > We had a convention to avoid having test sub-folders with bug numbers. > Could you, please, rename it to some symbolic name? > > Also we need the test to fail in the native agent when an error is > reported with the reportError(). > > It'd be great if some CDS/AppCDS experts from the Runtime team look at > the fix and test > so I've included Jiangli and Misha into the to-list. > > Thanks, > Serguei > > > On 1/25/18 16:11, Alex Menkov wrote: >> Hi all, >> >> Please take a look at a fix for >> JDK-8161605 : The '!UseSharedSpaces' check is not need in >> JvmtiManageCapabilities::recompute_always_capabilities >> >> The fix moves can_generate_all_class_hook_events capability from >> "onload_capabilities" to "always_capabilities" and cleans up >> recompute_always_capabilities method. >> >> jira: https://bugs.openjdk.java.net/browse/JDK-8161605 >> webrev: >> http://cr.openjdk.java.net/~amenkov/can_generate_class_hook/webrev/ >> >> --alex > From serguei.spitsyn at oracle.com Fri Feb 2 01:08:28 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 1 Feb 2018 17:08:28 -0800 Subject: 8161605 : The '!UseSharedSpaces' check is not need in JvmtiManageCapabilities::recompute_always_capabilities In-Reply-To: References: <1c8e3f0e-8aa1-574d-f24f-b78b0dc3e6bd@oracle.com> Message-ID: <19a1510e-efc4-e4dd-079a-6c571218ba60@oracle.com> Hi Alex, It looks good. Just a couple of minor comments though. 109 log("onLoad = " + onLoadValue); 110 log("live= " + liveValue); 111 if (onLoadValue != liveValue || onLoadValue < 0) { 112 throw new RuntimeException("Inconsistent can_generate_all_class_hook_events value"); 113 } It'd be nice to make more clear statement about why the test has been failed. A couple of things to cover: ? - the capability is expected to be available in both phases, ??? but now the test would pass if both phases returned false ? - an unexpected difference in returned capability in onload and live phases ? - what does it mean if the printed capability value is negative Something like the following would work: ? log("can_generate_all_class_hook_events capability values returned in ONLOAD and LIVE phases:"); ? log("ONLOAD phase: " + (onLoadValue < 0 ? "Failed to read" : onLoadValue); ? log("LIVE phase: ? " + (liveValue?? < 0 ? "Failed to read" : liveValue); ? if (onLoadValue != 1 || liveValue != 1) { ? ? ? throw new RuntimeException("The can_generate_all_class_hook_events " ? ? ? ? ? ? " expected to be available in ONLOAD and LIVE phases"); ? } It is better to remove the line: 145 //log("CommandLine: " + params.stream().collect(Collectors.joining(" "))); Thanks, Serguei On 2/1/18 16:11, Alex Menkov wrote: > Hi Serguei, > > Thanks for the feedback. > > Updated fix: > http://cr.openjdk.java.net/~amenkov/can_generate_class_hook/webrev.02/ > > --alex > > On 02/01/2018 11:00, serguei.spitsyn at oracle.com wrote: >> One more attempt to send it with the correct to-list... >> Sorry for the noise. >> >> >> Hi Alex, >> >> It looks good in general. >> A couple of comments on the test. >> >> We had a convention to avoid having test sub-folders with bug numbers. >> Could you, please, rename it to some symbolic name? >> >> Also we need the test to fail in the native agent when an error is >> reported with the reportError(). >> >> It'd be great if some CDS/AppCDS experts from the Runtime team look >> at the fix and test >> so I've included Jiangli and Misha into the to-list. >> >> Thanks, >> Serguei >> >> >> On 1/25/18 16:11, Alex Menkov wrote: >>> Hi all, >>> >>> Please take a look at a fix for >>> JDK-8161605 : The '!UseSharedSpaces' check is not need in >>> JvmtiManageCapabilities::recompute_always_capabilities >>> >>> The fix moves can_generate_all_class_hook_events capability from >>> "onload_capabilities" to "always_capabilities" and cleans up >>> recompute_always_capabilities method. >>> >>> jira: https://bugs.openjdk.java.net/browse/JDK-8161605 >>> webrev: >>> http://cr.openjdk.java.net/~amenkov/can_generate_class_hook/webrev/ >>> >>> --alex >> From david.holmes at oracle.com Fri Feb 2 02:01:36 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 2 Feb 2018 12:01:36 +1000 Subject: RFR: 8196361: JTReg failure in serviceability/sa/ClhsdbInspect.java In-Reply-To: <4f46f527c17f4d988e4b46e14f93cd4d@NASANEXM01E.na.qualcomm.com> References: <6e3441f3c28f4f7387d2174f52283fa7@NASANEXM01E.na.qualcomm.com> <44a71e46-3da2-53c6-7e6b-82658183ae8c@oracle.com> <3318aed7-f04a-3b92-2660-39cdf13c2d24@oracle.com> <24c556061ffb4fde9e87a8806c04c8f7@NASANEXM01E.na.qualcomm.com> <0eca7345-fb57-43e4-6169-c4b3531250e4@oracle.com> <4f46f527c17f4d988e4b46e14f93cd4d@NASANEXM01E.na.qualcomm.com> Message-ID: On 2/02/2018 1:50 AM, stewartd.qdt wrote: > Please have a look at the newest changes at: http://cr.openjdk.java.net/~dstewart/8196361/webrev.02/ > > The only difference between this and the last changeset is the use of "\\R" instead of whatever is the platform line.separator. Thanks for that. The overall changes seem reasonable but I'll defer to Jini for final approval. If Jini approves then consider this Reviewed. Thanks, David > Thank you, > Daniel > > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Thursday, February 1, 2018 2:51 AM > To: stewartd.qdt ; Jini George > Cc: serviceability-dev ; hotspot-dev at openjdk.java.net > Subject: Re: RFR: 8196361: JTReg failure in serviceability/sa/ClhsdbInspect.java > > Hi Daniel, > > On 1/02/2018 2:45 AM, stewartd.qdt wrote: >> Hi Jini, David, >> >> Please have a look at the revised webrev: >> http://cr.openjdk.java.net/~dstewart/8196361/webrev.01/ >> >> In this webrev I have changed the approach to finding the addresses. This was necessary because in the case of matching for the locks the addresses are before what is matched and in the case of Method the address is after it. The existing code only looked for the addresses after the matched string. I've also tried to align what tokens are being looked for in the lock case. I've taken an approach of breaking the jstack output into lines and then searching each line for it containing what we want. Once found, the line is broken into pieces to find the actual address we want. >> >> Please let me know if this is an unacceptable approach or any changes you would like to see. > > I'm not clear on the overall approach as I'm unclear exactly how inspect operates or exactly what the test is trying to verify. One comment on breaking things into lines though: > > 73 String newline = System.getProperty("line.separator"); > 74 String[] lines = jstackOutput.split(newline); > > As split() takes a regex, I suggest using \R to cover all potential line-breaks, rather than the platform specific line-seperator. We've been recently bitten by the distinction between output that comes from reading a process's stdout/stderr (and for which a newline \n is translated into the platform line-seperator), and output that comes across a socket connection (for which \n is not translated). This could result in failing to parse things correctly on Windows. It's safer/simpler to expect any kind of line-seperator. > > Thanks, > David > >> Thanks, >> Daniel >> >> >> -----Original Message----- >> From: Jini George [mailto:jini.george at oracle.com] >> Sent: Tuesday, January 30, 2018 6:58 AM >> To: David Holmes ; stewartd.qdt >> >> Cc: serviceability-dev ; >> hotspot-dev at openjdk.java.net >> Subject: Re: RFR: 8196361: JTReg failure in >> serviceability/sa/ClhsdbInspect.java >> >> Hi Daniel, David, >> >> Thanks, Daniel, for bringing this up. The intent of the test is to get >> the oop address corresponding to a java.lang.ref.ReferenceQueue$Lock, >> which can typically be obtained from the stack traces of the Common-Cleaner or the Finalizer threads. The stack traces which I had been noticing were typically of the form: >> >> >> "Common-Cleaner" #8 daemon prio=8 tid=0x00007f09c82ac000 nid=0xf6e in >> Object.wait() [0x00007f09a18d2000] >> java.lang.Thread.State: TIMED_WAITING (on object monitor) >> JavaThread state: _thread_blocked >> - java.lang.Object.wait(long) @bci=0, pc=0x00007f09b7d6480b, >> Method*=0x00007f09acc43d60 (Interpreted frame) >> - waiting on <0x000000072e61f6e0> (a >> java.lang.ref.ReferenceQueue$Lock) >> - java.lang.ref.ReferenceQueue.remove(long) @bci=59, line=151, pc=0x00007f09b7d55243, Method*=0x00007f09acdab9b0 (Interpreted frame) >> - waiting to re-lock in wait() <0x000000072e61f6e0> (a >> java.lang.ref.ReferenceQueue$Lock) >> ... >> >> I chose 'waiting to re-lock in wait' since that was what I had been observing next to the oop address of java.lang.ref.ReferenceQueue$Lock. >> But I see how with a timing difference, one could get 'waiting to lock' >> as in your case. So, a good way to fix might be to check for the line containing '(a java.lang.ref.ReferenceQueue$Lock)', getting the oop address from that line (should be the address appearing immediately before '(a java.lang.ref.ReferenceQueue$Lock)') and passing that to the 'inspect' command. >> >> Thanks much, >> Jini. >> >> On 1/30/2018 3:35 AM, David Holmes wrote: >>> Hi Daniel, >>> >>> Serviceability issues should go to >>> serviceability-dev at openjdk.java.net >>> - now cc'd. >>> >>> On 30/01/2018 7:53 AM, stewartd.qdt wrote: >>>> Please review this webrev [1] which attempts to fix a test error in >>>> serviceability/sa/ClhsdbInspect.java when it is run under an AArch64 >>>> system (not necessarily exclusive to this system, but it was the >>>> system under test). The bug report [2] provides further details. >>>> Essentially the line "waiting to re-lock in wait" never actually >>>> occurs. Instead I have the line "waiting to lock" which occurs for >>>> the referenced item of /java/lang/ref/ReferenceQueue$Lock. >>>> Unfortunately the test is written such that only the first "waiting to lock" >>>> occurrence is seen (for java/lang/Class), which is already accounted >>>> for in the test. >>> >>> I can't tell exactly what the test expects, or why, but it would be >>> extremely hard to arrange for "waiting to re-lock in wait" to be seen >>> for the ReferenceQueue lock! That requires acquiring the lock >>> yourself, issuing a notify() to unblock the wait(), and then issuing >>> the jstack command while still holding the lock! >>> >>> David >>> ----- >>> >>>> I'm not overly happy with this approach as it actually removes a >>>> test line. However, the test line does not actually appear in the >>>> output (at least on my system) and the test is not currently written >>>> to look for the second occurrence of the line "waiting to lock". >>>> Perhaps the original author could chime in and provide further >>>> guidance as to the intention of the test. >>>> >>>> I am happy to modify the patch as necessary. >>>> >>>> Regards, >>>> Daniel Stewart >>>> >>>> >>>> [1] -? http://cr.openjdk.java.net/~dstewart/8196361/webrev.00/ >>>> [2] - https://bugs.openjdk.java.net/browse/JDK-8196361 >>>> From david.holmes at oracle.com Fri Feb 2 02:04:05 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 2 Feb 2018 12:04:05 +1000 Subject: RFR: 8196534: [Testbug] serviceability/dcmd/jvmti/*DcmdTest tests can't tolerate unrelated warnings In-Reply-To: <10a481dc-b7a2-5e89-92e9-108b15976c76@oracle.com> References: <10a481dc-b7a2-5e89-92e9-108b15976c76@oracle.com> Message-ID: <6e47f556-c906-4873-af8c-7e7a49428670@oracle.com> Thanks Serguei. David On 2/02/2018 6:16 AM, serguei.spitsyn at oracle.com wrote: > Hi David, > > Looks good. > Thank you for taking care about it! > > Thanks, > Serguei > > > On 1/31/18 17:49, David Holmes wrote: >> bug: https://bugs.openjdk.java.net/browse/JDK-8196534 >> webrev: http://cr.openjdk.java.net/~dholmes/8196534/webrev/ >> >> I somehow missed these failures when doing the JDK 11 version change >> testing. As with the SA tests in JDK-8194067 we need to ignore VM >> Warnings in stderr. I re-used Sharath's logic from that fix. >> >> Will be pushing to jdk/jdk as that is where the version change will go >> first. >> >> Thanks, >> David > From chris.plummer at oracle.com Fri Feb 2 02:35:47 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 1 Feb 2018 18:35:47 -0800 Subject: RFR: 8196534: [Testbug] serviceability/dcmd/jvmti/*DcmdTest tests can't tolerate unrelated warnings In-Reply-To: <10a481dc-b7a2-5e89-92e9-108b15976c76@oracle.com> References: <10a481dc-b7a2-5e89-92e9-108b15976c76@oracle.com> Message-ID: +1 On 2/1/18 12:16 PM, serguei.spitsyn at oracle.com wrote: > Hi David, > > Looks good. > Thank you for taking care about it! > > Thanks, > Serguei > > > On 1/31/18 17:49, David Holmes wrote: >> bug: https://bugs.openjdk.java.net/browse/JDK-8196534 >> webrev: http://cr.openjdk.java.net/~dholmes/8196534/webrev/ >> >> I somehow missed these failures when doing the JDK 11 version change >> testing. As with the SA tests in JDK-8194067 we need to ignore VM >> Warnings in stderr. I re-used Sharath's logic from that fix. >> >> Will be pushing to jdk/jdk as that is where the version change will >> go first. >> >> Thanks, >> David > From david.holmes at oracle.com Fri Feb 2 02:56:03 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 2 Feb 2018 12:56:03 +1000 Subject: RFR: 8196534: [Testbug] serviceability/dcmd/jvmti/*DcmdTest tests can't tolerate unrelated warnings In-Reply-To: References: <10a481dc-b7a2-5e89-92e9-108b15976c76@oracle.com> Message-ID: <5f900a62-abf1-070d-de34-8c660da46dd4@oracle.com> Thanks Chris. Sorry I already pushed it. David On 2/02/2018 12:35 PM, Chris Plummer wrote: > +1 > > On 2/1/18 12:16 PM, serguei.spitsyn at oracle.com wrote: >> Hi David, >> >> Looks good. >> Thank you for taking care about it! >> >> Thanks, >> Serguei >> >> >> On 1/31/18 17:49, David Holmes wrote: >>> bug: https://bugs.openjdk.java.net/browse/JDK-8196534 >>> webrev: http://cr.openjdk.java.net/~dholmes/8196534/webrev/ >>> >>> I somehow missed these failures when doing the JDK 11 version change >>> testing. As with the SA tests in JDK-8194067 we need to ignore VM >>> Warnings in stderr. I re-used Sharath's logic from that fix. >>> >>> Will be pushing to jdk/jdk as that is where the version change will >>> go first. >>> >>> Thanks, >>> David >> > From jini.george at oracle.com Fri Feb 2 06:19:27 2018 From: jini.george at oracle.com (Jini George) Date: Fri, 2 Feb 2018 11:49:27 +0530 Subject: RFR: 8196361: JTReg failure in serviceability/sa/ClhsdbInspect.java In-Reply-To: References: <6e3441f3c28f4f7387d2174f52283fa7@NASANEXM01E.na.qualcomm.com> <44a71e46-3da2-53c6-7e6b-82658183ae8c@oracle.com> <3318aed7-f04a-3b92-2660-39cdf13c2d24@oracle.com> <24c556061ffb4fde9e87a8806c04c8f7@NASANEXM01E.na.qualcomm.com> <0eca7345-fb57-43e4-6169-c4b3531250e4@oracle.com> <4f46f527c17f4d988e4b46e14f93cd4d@NASANEXM01E.na.qualcomm.com> Message-ID: <1b753efa-6abc-4947-e4f3-f6a29020c082@oracle.com> Hi Daniel, Your changes look good to me overall. Just some nits: * Please do add 2018 to the copyright year. * Since the rest of the file follows 4 spaces for indentation, please keep the indentation to 4 spaces. * Line 81: It would be great if the opening brace is at line 80, so that it would be consistent with the rest of the file. * Line 65: The declaration could be a part of line 79. * Line 51: Please add the 'oop address of a java.lang.Class' to the comment. Thanks! Jini. On 2/2/2018 7:31 AM, David Holmes wrote: > On 2/02/2018 1:50 AM, stewartd.qdt wrote: >> Please have? a look at the newest changes at: >> http://cr.openjdk.java.net/~dstewart/8196361/webrev.02/ >> >> The only difference between this and the last changeset is the use of >> "\\R" instead of whatever is the platform line.separator. > > Thanks for that. > > The overall changes seem reasonable but I'll defer to Jini for final > approval. If Jini approves then consider this Reviewed. > > Thanks, > David > >> Thank you, >> Daniel >> >> -----Original Message----- >> From: David Holmes [mailto:david.holmes at oracle.com] >> Sent: Thursday, February 1, 2018 2:51 AM >> To: stewartd.qdt ; Jini George >> >> Cc: serviceability-dev ; >> hotspot-dev at openjdk.java.net >> Subject: Re: RFR: 8196361: JTReg failure in >> serviceability/sa/ClhsdbInspect.java >> >> Hi Daniel, >> >> On 1/02/2018 2:45 AM, stewartd.qdt wrote: >>> Hi Jini, David, >>> >>> Please have a look at the revised webrev: >>> http://cr.openjdk.java.net/~dstewart/8196361/webrev.01/ >>> >>> In this webrev I have changed the approach to finding the addresses. >>> This was necessary because in the case of matching for the locks the >>> addresses are before what is matched and in the case of Method the >>> address is after it.? The existing code only looked for the addresses >>> after the matched string. I've also tried to align what tokens? are >>> being looked for in the lock case. I've taken an approach of breaking >>> the jstack output into lines and then searching each line for it >>> containing what we want. Once found, the line is broken into pieces >>> to find the actual address we want. >>> >>> Please let me know if this is an unacceptable approach or any changes >>> you would like to see. >> >> I'm not clear on the overall approach as I'm unclear exactly how >> inspect operates or exactly what the test is trying to verify. One >> comment on breaking things into lines though: >> >> ??? 73???????????? String newline = System.getProperty("line.separator"); >> ??? 74???????????? String[] lines = jstackOutput.split(newline); >> >> As split() takes a regex, I suggest using \R to cover all potential >> line-breaks, rather than the platform specific line-seperator. We've >> been recently bitten by the distinction between output that comes from >> reading a process's stdout/stderr (and for which a newline \n is >> translated into the platform line-seperator), and output that comes >> across a socket connection (for which \n is not translated). This >> could result in failing to parse things correctly on Windows. It's >> safer/simpler to expect any kind of line-seperator. >> >> Thanks, >> David >> >>> Thanks, >>> Daniel >>> >>> >>> -----Original Message----- >>> From: Jini George [mailto:jini.george at oracle.com] >>> Sent: Tuesday, January 30, 2018 6:58 AM >>> To: David Holmes ; stewartd.qdt >>> >>> Cc: serviceability-dev ; >>> hotspot-dev at openjdk.java.net >>> Subject: Re: RFR: 8196361: JTReg failure in >>> serviceability/sa/ClhsdbInspect.java >>> >>> Hi Daniel, David, >>> >>> Thanks, Daniel, for bringing this up. The intent of the test is to get >>> the oop address corresponding to a java.lang.ref.ReferenceQueue$Lock, >>> which can typically be obtained from the stack traces of the >>> Common-Cleaner or the Finalizer threads. The stack traces which I had >>> been noticing were typically of the form: >>> >>> >>> "Common-Cleaner" #8 daemon prio=8 tid=0x00007f09c82ac000 nid=0xf6e in >>> Object.wait() [0x00007f09a18d2000] >>> ????? java.lang.Thread.State: TIMED_WAITING (on object monitor) >>> ????? JavaThread state: _thread_blocked >>> ??? - java.lang.Object.wait(long) @bci=0, pc=0x00007f09b7d6480b, >>> Method*=0x00007f09acc43d60 (Interpreted frame) >>> ?????????? - waiting on <0x000000072e61f6e0> (a >>> java.lang.ref.ReferenceQueue$Lock) >>> ??? - java.lang.ref.ReferenceQueue.remove(long) @bci=59, line=151, >>> pc=0x00007f09b7d55243, Method*=0x00007f09acdab9b0 (Interpreted frame) >>> ?????????? - waiting to re-lock in wait() <0x000000072e61f6e0> (a >>> java.lang.ref.ReferenceQueue$Lock) >>> ... >>> >>> I chose 'waiting to re-lock in wait' since that was what I had been >>> observing next to the oop address of java.lang.ref.ReferenceQueue$Lock. >>> But I see how with a timing difference, one could get 'waiting to lock' >>> as in your case. So, a good way to fix might be to check for the line >>> containing '(a java.lang.ref.ReferenceQueue$Lock)', getting the oop >>> address from that line (should be the address appearing immediately >>> before '(a java.lang.ref.ReferenceQueue$Lock)') and passing that to >>> the 'inspect' command. >>> >>> Thanks much, >>> Jini. >>> >>> On 1/30/2018 3:35 AM, David Holmes wrote: >>>> Hi Daniel, >>>> >>>> Serviceability issues should go to >>>> serviceability-dev at openjdk.java.net >>>> - now cc'd. >>>> >>>> On 30/01/2018 7:53 AM, stewartd.qdt wrote: >>>>> Please review this webrev [1] which attempts to fix a test error in >>>>> serviceability/sa/ClhsdbInspect.java when it is run under an AArch64 >>>>> system (not necessarily exclusive to this system, but it was the >>>>> system under test). The bug report [2] provides further details. >>>>> Essentially the line "waiting to re-lock in wait" never actually >>>>> occurs. Instead I have the line "waiting to lock" which occurs for >>>>> the referenced item of /java/lang/ref/ReferenceQueue$Lock. >>>>> Unfortunately the test is written such that only the first "waiting >>>>> to lock" >>>>> occurrence is seen (for java/lang/Class), which is already accounted >>>>> for in the test. >>>> >>>> I can't tell exactly what the test expects, or why, but it would be >>>> extremely hard to arrange for "waiting to re-lock in wait" to be seen >>>> for the ReferenceQueue lock! That requires acquiring the lock >>>> yourself, issuing a notify() to unblock the wait(), and then issuing >>>> the jstack command while still holding the lock! >>>> >>>> David >>>> ----- >>>> >>>>> I'm not overly happy with this approach as it actually removes a >>>>> test line. However, the test line does not actually appear in the >>>>> output (at least on my system) and the test is not currently written >>>>> to look for the second occurrence of the line "waiting to lock". >>>>> Perhaps the original author could chime in and provide further >>>>> guidance as to the intention of the test. >>>>> >>>>> I am happy to modify the patch as necessary. >>>>> >>>>> Regards, >>>>> Daniel Stewart >>>>> >>>>> >>>>> [1] -? http://cr.openjdk.java.net/~dstewart/8196361/webrev.00/ >>>>> [2] - https://bugs.openjdk.java.net/browse/JDK-8196361 >>>>> From christoph.langer at sap.com Fri Feb 2 09:21:17 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 2 Feb 2018 09:21:17 +0000 Subject: JDK-8080990: libdt_socket/socket_md.c(202) : warning C4996: 'gethostbyname': Use getaddrinfo() or GetAddrInfoW() In-Reply-To: <5A7331D9.9000800@oracle.com> References: <5A6A2611.4030003@oracle.com> <5A7331D9.9000800@oracle.com> Message-ID: <9c23cba20f7649459f6a3059f80cb834@sap.com> Hi Gary, > Here's a revised webrev > > http://cr.openjdk.java.net/~gadams/8080990/webrev.01/index.html > > Still testing ... > > Using shutdown() fixed problems reported by the > java/nio/channelSocketChannel tests. The fix looks good. I would think we should rename dbgsysInetAddr to dbgsysPToN and use inet_pton also for the Unix/Linux platforms. It would be the better choice there as well, though we still only support IPv4 in libdt_socket. > > I also noticed prior use of getaddrinfo for "localhost" was not calling > freeaddrinfo. Thanks for spotting/fixing that. Best regards Christoph From sharath.ballal at oracle.com Fri Feb 2 09:48:50 2018 From: sharath.ballal at oracle.com (Sharath Ballal) Date: Fri, 2 Feb 2018 01:48:50 -0800 (PST) Subject: RFR: 8196521: serviceability/sa/ClhsdbWhere.java fails to find method 'waitForReferencePendingList' in output In-Reply-To: <6ab1e27d-cf2a-9397-a970-232fb1ddf523@oracle.com> References: <71d86c90cba444c6967883078e5c5cc3@NASANEXM01E.na.qualcomm.com> <4158546c-434b-acd1-c0b1-5e2c7faf680c@oracle.com> <6ab1e27d-cf2a-9397-a970-232fb1ddf523@oracle.com> Message-ID: <369287e9-a6ab-4d40-a3f3-f81086406bc7@default> Agree with David. It looks like the stack will be different on some machines occasionally depending on the load, machine speed etc. Hence the following lines in the test "java.lang.ref.Reference", "private static void processPendingReferences", "private static native void waitForReferencePendingList", can be replaced by Java Stack Trace for Signal Dispatcher Java Stack Trace for Reference Handler That atlease ensures we are looking for all the threads. Thanks, Sharath (not a Reviewer) -----Original Message----- From: David Holmes Sent: Thursday, February 01, 2018 1:29 PM To: Jini George; stewartd.qdt; serviceability-dev Subject: Re: RFR: 8196521: serviceability/sa/ClhsdbWhere.java fails to find method 'waitForReferencePendingList' in output On 1/02/2018 5:01 PM, Jini George wrote: > Hello Daniel, > > Your fix looks good to me. You probably could instead add 'remove()' > or 'run()' to the list, but I leave it upto you. These tests should only be looking for things that are guaranteed to be seen. If waitForReferencePendingList is not there, how do we know that processPendingReferences will be there? We seem to be assuming a quiescent system. We know the threads that will be present, but the only stack entries we should be looking for are those controlled by the test code itself. David ----- > Thanks, > Jini (Not a (R)eviewer). > > On 2/1/2018 2:39 AM, stewartd.qdt wrote: >> Please review this webrev [1] which attempts to fix a test error in >> serviceability/sa/ClhsdbWhere.java when it is run under an AArch64 >> system (not necessarily exclusive to this system, but it was the >> system under test). The bug report [2] provides further details and >> has the jtr report that was generated. Essentially the line >> "waitForReferencePendingList" never actually occurs. The test is >> written such that it is expecting that line. >> >> This patch simply removes the line from the set of tested lines it >> expects. ?I'm not overly happy with this approach as it actually >> removes a test line. However, the test line does not actually appear >> in the output (at least on my system) and I'm not sure if there is >> actually another way of testing for the intent of this line or if it >> doesn't actually have to appear in the output, depending on the >> system. Perhaps the original author could chime in and provide >> further guidance as to the intention of the test. >> >> I am happy to modify the patch as necessary. >> >> Regards, >> >> Daniel Stewart >> >> [1] - http://cr.openjdk.java.net/~dstewart/8196521/webrev.00/ >> >> [2] - https://bugs.openjdk.java.net/browse/JDK-8196521 >> From stewartd.qdt at qualcommdatacenter.com Fri Feb 2 13:58:19 2018 From: stewartd.qdt at qualcommdatacenter.com (stewartd.qdt) Date: Fri, 2 Feb 2018 13:58:19 +0000 Subject: RFR: 8196361: JTReg failure in serviceability/sa/ClhsdbInspect.java In-Reply-To: <1b753efa-6abc-4947-e4f3-f6a29020c082@oracle.com> References: <6e3441f3c28f4f7387d2174f52283fa7@NASANEXM01E.na.qualcomm.com> <44a71e46-3da2-53c6-7e6b-82658183ae8c@oracle.com> <3318aed7-f04a-3b92-2660-39cdf13c2d24@oracle.com> <24c556061ffb4fde9e87a8806c04c8f7@NASANEXM01E.na.qualcomm.com> <0eca7345-fb57-43e4-6169-c4b3531250e4@oracle.com> <4f46f527c17f4d988e4b46e14f93cd4d@NASANEXM01E.na.qualcomm.com> <1b753efa-6abc-4947-e4f3-f6a29020c082@oracle.com> Message-ID: <4dc3cedf0b1e40b7bceff6672cb3fa1a@NASANEXM01E.na.qualcomm.com> Hi Jini, Thank you for the review. I have made the requested changes and posted them at http://cr.openjdk.java.net/~dstewart/8196361/webrev.03/ Please have a look and review the changes. Thanks, Daniel -----Original Message----- From: Jini George [mailto:jini.george at oracle.com] Sent: Friday, February 2, 2018 1:19 AM To: David Holmes ; stewartd.qdt Cc: serviceability-dev ; hotspot-dev at openjdk.java.net Subject: Re: RFR: 8196361: JTReg failure in serviceability/sa/ClhsdbInspect.java Hi Daniel, Your changes look good to me overall. Just some nits: * Please do add 2018 to the copyright year. * Since the rest of the file follows 4 spaces for indentation, please keep the indentation to 4 spaces. * Line 81: It would be great if the opening brace is at line 80, so that it would be consistent with the rest of the file. * Line 65: The declaration could be a part of line 79. * Line 51: Please add the 'oop address of a java.lang.Class' to the comment. Thanks! Jini. On 2/2/2018 7:31 AM, David Holmes wrote: > On 2/02/2018 1:50 AM, stewartd.qdt wrote: >> Please have? a look at the newest changes at: >> http://cr.openjdk.java.net/~dstewart/8196361/webrev.02/ >> >> The only difference between this and the last changeset is the use of >> "\\R" instead of whatever is the platform line.separator. > > Thanks for that. > > The overall changes seem reasonable but I'll defer to Jini for final > approval. If Jini approves then consider this Reviewed. > > Thanks, > David > >> Thank you, >> Daniel >> >> -----Original Message----- >> From: David Holmes [mailto:david.holmes at oracle.com] >> Sent: Thursday, February 1, 2018 2:51 AM >> To: stewartd.qdt ; Jini George >> >> Cc: serviceability-dev ; >> hotspot-dev at openjdk.java.net >> Subject: Re: RFR: 8196361: JTReg failure in >> serviceability/sa/ClhsdbInspect.java >> >> Hi Daniel, >> >> On 1/02/2018 2:45 AM, stewartd.qdt wrote: >>> Hi Jini, David, >>> >>> Please have a look at the revised webrev: >>> http://cr.openjdk.java.net/~dstewart/8196361/webrev.01/ >>> >>> In this webrev I have changed the approach to finding the addresses. >>> This was necessary because in the case of matching for the locks the >>> addresses are before what is matched and in the case of Method the >>> address is after it.? The existing code only looked for the >>> addresses after the matched string. I've also tried to align what >>> tokens? are being looked for in the lock case. I've taken an >>> approach of breaking the jstack output into lines and then searching >>> each line for it containing what we want. Once found, the line is >>> broken into pieces to find the actual address we want. >>> >>> Please let me know if this is an unacceptable approach or any >>> changes you would like to see. >> >> I'm not clear on the overall approach as I'm unclear exactly how >> inspect operates or exactly what the test is trying to verify. One >> comment on breaking things into lines though: >> >> ??? 73???????????? String newline = >> System.getProperty("line.separator"); >> ??? 74???????????? String[] lines = jstackOutput.split(newline); >> >> As split() takes a regex, I suggest using \R to cover all potential >> line-breaks, rather than the platform specific line-seperator. We've >> been recently bitten by the distinction between output that comes >> from reading a process's stdout/stderr (and for which a newline \n is >> translated into the platform line-seperator), and output that comes >> across a socket connection (for which \n is not translated). This >> could result in failing to parse things correctly on Windows. It's >> safer/simpler to expect any kind of line-seperator. >> >> Thanks, >> David >> >>> Thanks, >>> Daniel >>> >>> >>> -----Original Message----- >>> From: Jini George [mailto:jini.george at oracle.com] >>> Sent: Tuesday, January 30, 2018 6:58 AM >>> To: David Holmes ; stewartd.qdt >>> >>> Cc: serviceability-dev ; >>> hotspot-dev at openjdk.java.net >>> Subject: Re: RFR: 8196361: JTReg failure in >>> serviceability/sa/ClhsdbInspect.java >>> >>> Hi Daniel, David, >>> >>> Thanks, Daniel, for bringing this up. The intent of the test is to >>> get the oop address corresponding to a >>> java.lang.ref.ReferenceQueue$Lock, >>> which can typically be obtained from the stack traces of the >>> Common-Cleaner or the Finalizer threads. The stack traces which I >>> had been noticing were typically of the form: >>> >>> >>> "Common-Cleaner" #8 daemon prio=8 tid=0x00007f09c82ac000 nid=0xf6e >>> in >>> Object.wait() [0x00007f09a18d2000] >>> ????? java.lang.Thread.State: TIMED_WAITING (on object monitor) >>> ????? JavaThread state: _thread_blocked >>> ??? - java.lang.Object.wait(long) @bci=0, pc=0x00007f09b7d6480b, >>> Method*=0x00007f09acc43d60 (Interpreted frame) >>> ?????????? - waiting on <0x000000072e61f6e0> (a >>> java.lang.ref.ReferenceQueue$Lock) >>> ??? - java.lang.ref.ReferenceQueue.remove(long) @bci=59, line=151, >>> pc=0x00007f09b7d55243, Method*=0x00007f09acdab9b0 (Interpreted >>> frame) >>> ?????????? - waiting to re-lock in wait() <0x000000072e61f6e0> (a >>> java.lang.ref.ReferenceQueue$Lock) >>> ... >>> >>> I chose 'waiting to re-lock in wait' since that was what I had been >>> observing next to the oop address of java.lang.ref.ReferenceQueue$Lock. >>> But I see how with a timing difference, one could get 'waiting to lock' >>> as in your case. So, a good way to fix might be to check for the >>> line containing '(a java.lang.ref.ReferenceQueue$Lock)', getting the >>> oop address from that line (should be the address appearing >>> immediately before '(a java.lang.ref.ReferenceQueue$Lock)') and >>> passing that to the 'inspect' command. >>> >>> Thanks much, >>> Jini. >>> >>> On 1/30/2018 3:35 AM, David Holmes wrote: >>>> Hi Daniel, >>>> >>>> Serviceability issues should go to >>>> serviceability-dev at openjdk.java.net >>>> - now cc'd. >>>> >>>> On 30/01/2018 7:53 AM, stewartd.qdt wrote: >>>>> Please review this webrev [1] which attempts to fix a test error >>>>> in serviceability/sa/ClhsdbInspect.java when it is run under an >>>>> AArch64 system (not necessarily exclusive to this system, but it >>>>> was the system under test). The bug report [2] provides further details. >>>>> Essentially the line "waiting to re-lock in wait" never actually >>>>> occurs. Instead I have the line "waiting to lock" which occurs for >>>>> the referenced item of /java/lang/ref/ReferenceQueue$Lock. >>>>> Unfortunately the test is written such that only the first >>>>> "waiting to lock" >>>>> occurrence is seen (for java/lang/Class), which is already >>>>> accounted for in the test. >>>> >>>> I can't tell exactly what the test expects, or why, but it would be >>>> extremely hard to arrange for "waiting to re-lock in wait" to be >>>> seen for the ReferenceQueue lock! That requires acquiring the lock >>>> yourself, issuing a notify() to unblock the wait(), and then >>>> issuing the jstack command while still holding the lock! >>>> >>>> David >>>> ----- >>>> >>>>> I'm not overly happy with this approach as it actually removes a >>>>> test line. However, the test line does not actually appear in the >>>>> output (at least on my system) and the test is not currently >>>>> written to look for the second occurrence of the line "waiting to lock". >>>>> Perhaps the original author could chime in and provide further >>>>> guidance as to the intention of the test. >>>>> >>>>> I am happy to modify the patch as necessary. >>>>> >>>>> Regards, >>>>> Daniel Stewart >>>>> >>>>> >>>>> [1] -? http://cr.openjdk.java.net/~dstewart/8196361/webrev.00/ >>>>> [2] - https://bugs.openjdk.java.net/browse/JDK-8196361 >>>>> From stewartd.qdt at qualcommdatacenter.com Fri Feb 2 14:11:09 2018 From: stewartd.qdt at qualcommdatacenter.com (stewartd.qdt) Date: Fri, 2 Feb 2018 14:11:09 +0000 Subject: RFR: 8196521: serviceability/sa/ClhsdbWhere.java fails to find method 'waitForReferencePendingList' in output In-Reply-To: <369287e9-a6ab-4d40-a3f3-f81086406bc7@default> References: <71d86c90cba444c6967883078e5c5cc3@NASANEXM01E.na.qualcomm.com> <4158546c-434b-acd1-c0b1-5e2c7faf680c@oracle.com> <6ab1e27d-cf2a-9397-a970-232fb1ddf523@oracle.com> <369287e9-a6ab-4d40-a3f3-f81086406bc7@default> Message-ID: <1afda3cf56994421a9919b9e0834492d@NASANEXM01E.na.qualcomm.com> Hi Sharath, Thank you for your comments and insight. I have made the requested changes and the new webrev is at: http://cr.openjdk.java.net/~dstewart/8196521/webrev.01/ Please have a look and let me know if there are any other changes required. This does pass testing on my system. Thanks, Daniel -----Original Message----- From: Sharath Ballal [mailto:sharath.ballal at oracle.com] Sent: Friday, February 2, 2018 4:49 AM To: David Holmes ; Jini Susan George ; stewartd.qdt ; serviceability-dev Subject: RE: RFR: 8196521: serviceability/sa/ClhsdbWhere.java fails to find method 'waitForReferencePendingList' in output Agree with David. It looks like the stack will be different on some machines occasionally depending on the load, machine speed etc. Hence the following lines in the test "java.lang.ref.Reference", "private static void processPendingReferences", "private static native void waitForReferencePendingList", can be replaced by Java Stack Trace for Signal Dispatcher Java Stack Trace for Reference Handler That atlease ensures we are looking for all the threads. Thanks, Sharath (not a Reviewer) -----Original Message----- From: David Holmes Sent: Thursday, February 01, 2018 1:29 PM To: Jini George; stewartd.qdt; serviceability-dev Subject: Re: RFR: 8196521: serviceability/sa/ClhsdbWhere.java fails to find method 'waitForReferencePendingList' in output On 1/02/2018 5:01 PM, Jini George wrote: > Hello Daniel, > > Your fix looks good to me. You probably could instead add 'remove()' > or 'run()' to the list, but I leave it upto you. These tests should only be looking for things that are guaranteed to be seen. If waitForReferencePendingList is not there, how do we know that processPendingReferences will be there? We seem to be assuming a quiescent system. We know the threads that will be present, but the only stack entries we should be looking for are those controlled by the test code itself. David ----- > Thanks, > Jini (Not a (R)eviewer). > > On 2/1/2018 2:39 AM, stewartd.qdt wrote: >> Please review this webrev [1] which attempts to fix a test error in >> serviceability/sa/ClhsdbWhere.java when it is run under an AArch64 >> system (not necessarily exclusive to this system, but it was the >> system under test). The bug report [2] provides further details and >> has the jtr report that was generated. Essentially the line >> "waitForReferencePendingList" never actually occurs. The test is >> written such that it is expecting that line. >> >> This patch simply removes the line from the set of tested lines it >> expects. ?I'm not overly happy with this approach as it actually >> removes a test line. However, the test line does not actually appear >> in the output (at least on my system) and I'm not sure if there is >> actually another way of testing for the intent of this line or if it >> doesn't actually have to appear in the output, depending on the >> system. Perhaps the original author could chime in and provide >> further guidance as to the intention of the test. >> >> I am happy to modify the patch as necessary. >> >> Regards, >> >> Daniel Stewart >> >> [1] - http://cr.openjdk.java.net/~dstewart/8196521/webrev.00/ >> >> [2] - https://bugs.openjdk.java.net/browse/JDK-8196521 >> From stefan.johansson at oracle.com Fri Feb 2 14:38:05 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Fri, 2 Feb 2018 15:38:05 +0100 Subject: PING: RFR: JDK-8153333: [REDO] STW phases at Concurrent GC should count in PerfCounter In-Reply-To: References: <2f4e0901-1602-6276-e7fd-84e168e7b317@gmail.com> Message-ID: <3210723e-c5f9-4277-a97a-be61e10e6b3d@oracle.com> Hi Yasumasa, The changes doesn't apply clean on the latest jdk/hs, can you provide an updated webrev? The testing done by the submit repo doesn't cover the tests you have update so I plan to take the change for a spin and make sure the correct tests are run and verified in Mach 5. Also a question about the change. Why do we need a special flag for CMS? I see that the original bug report refers to the flag as being a way to turn on and off the feature but the current implementation only consider the flag for CMS. Thanks, Stefan On 2018-02-01 14:58, Yasumasa Suenaga wrote: > PING: Could you review and sponsor it? > >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ > > This change has been passed Mach 5 via submit repo: > http://java.se.oracle.com:10065/mdash/jobs/mach5-one-ysuenaga-JDK-8153333-20180201-0805-10101 > > > Thanks, > > Yasumasa > > > On 2017/11/01 22:02, Yasumasa Suenaga wrote: >> PING: Could you review and sponsor it? >> >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >> >> Also I need JPRT results of this change. >> Could you cooperate? >> >> >> Thanks, >> >> Yasumasa >> >> >> On 2017/09/27 0:08, Yasumasa Suenaga wrote: >>> Hi all, >>> >>> I uploaded new webrev to be adapted to jdk10/hs: >>> >>> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>> >>> I want to check this patch via JPRT, but I cannot access it. >>> Could you cooperate? >>> >>> >>> Thanks, >>> >>> yasumasa >>> >>> >>> On 2017/09/21 7:46, Yasumasa Suenaga wrote: >>>> PING: >>>> >>>> Have you checked this issue? >>>> >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/hotspot/ >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/jdk/ >>>> >>>> >>>> Yasumasa >>>> >>>> >>>> On 2017/07/01 23:44, Yasumasa Suenaga wrote: >>>>> PING: >>>>> >>>>> Have you checked this issue? >>>>> >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2017/06/14 13:22, Yasumasa Suenaga wrote: >>>>>> Hi all, >>>>>> >>>>>> I changed PerfCounter to show CGC STW phase in jstat in JDK-8151674. >>>>>> However, it occurred several jtreg test failure, so it was >>>>>> back-outed. >>>>>> >>>>>> I want to resume to work for this issue. >>>>>> >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/hotspot/ >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/jdk/ >>>>>> >>>>>> These changes are work fine on jtreg test as below: >>>>>> >>>>>> ?? hotspot/test/serviceability/tmtools/jstat >>>>>> ?? jdk/test/sun/tools >>>>>> >>>>>> >>>>>> Since JDK 9, default GC algorithm is set to G1. >>>>>> So I think this change is useful to watch GC behavior through jstat. >>>>>> >>>>>> I cannot access JPRT. Could you help? >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> From erik.osterlund at oracle.com Fri Feb 2 16:44:37 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 2 Feb 2018 17:44:37 +0100 Subject: Low-Overhead Heap Profiling In-Reply-To: References: <5ec70351-910a-96bb-eb03-43ca88bd6259@oracle.com> <1508935388.13554.11.camel@oracle.com> <1510146425.3155.11.camel@oracle.com> <1511450419.2477.24.camel@oracle.com> <787d7c33-43bf-328c-9b8f-c0708ae642e9@oracle.com> Message-ID: <5A749575.5050004@oracle.com> Hi JC, Hope I am reviewing the right version of your work. Here goes... src/hotspot/share/gc/shared/collectedHeap.inline.hpp: 159 AllocTracer::send_allocation_outside_tlab(klass, result, size * HeapWordSize, THREAD); 160 161 THREAD->tlab().handle_sample(THREAD, result, size); 162 return result; 163 } Should not call tlab()->X without checking if (UseTLAB) IMO. src/hotspot/share/gc/shared/threadLocalAllocBuffer.cpp: So first of all, there seems to quite a few ends. There is an "end", a "hard end", a "slow path end", and an "actual end". Moreover, it seems like the "hard end" is actually further away than the "actual end". So the "hard end" seems like more of a "really definitely actual end" or something. I don't know about you, but I think it looks kind of messy. In particular, I don't feel like the name "actual end" reflects what it represents, especially when there is another end that is behind the "actual end". 413 HeapWord* ThreadLocalAllocBuffer::hard_end() { 414 // Did a fast TLAB refill occur? 415 if (_slow_path_end != _end) { 416 // Fix up the actual end to be now the end of this TLAB. 417 _slow_path_end = _end; 418 _actual_end = _end; 419 } 420 421 return _actual_end + alignment_reserve(); 422 } I really do not like making getters unexpectedly have these kind of side effects. It is not expected that when you ask for the "hard end", you implicitly update the "slow path end" and "actual end" to new values. src/hotspot/share/prims/jvmti.xml: 10357 10358 10359 Can sample the heap. 10360 If this capability is enabled then the heap sampling methods can be called. 10361 10362 Looks like this capability should not be "since 9" if it gets integrated now. src/hotspot/share/runtime/heapMonitoring.cpp: 448 if (is_alive->do_object_b(value)) { 449 // Update the oop to point to the new object if it is still alive. 450 f->do_oop(&(trace.obj)); 451 452 // Copy the old trace, if it is still live. 453 _allocated_traces->at_put(curr_pos++, trace); 454 455 // Store the live trace in a cache, to be served up on /heapz. 456 _traces_on_last_full_gc->append(trace); 457 458 count++; 459 } else { 460 // If the old trace is no longer live, add it to the list of 461 // recently collected garbage. 462 store_garbage_trace(trace); 463 } In the case where the oop was not live, I would like it to be explicitly cleared. Also I see a lot of concurrent-looking use of the following field: 267 volatile bool _initialized; Please note that the "volatile" qualifier does not help with reordering here. Reordering between volatile and non-volatile fields is completely free for both compiler and hardware, except for windows with MSVC, where volatile semantics is defined to use acquire/release semantics, and the hardware is TSO. But for the general case, I would expect this field to be stored with OrderAccess::release_store and loaded with OrderAccess::load_acquire. Otherwise it is not thread safe. As a kind of meta comment, I wonder if it would make sense to add sampling for non-TLAB allocations. Seems like if someone is rapidly allocating a whole bunch of 1 MB objects that never fit in a TLAB, I might still be interested in seeing that in my traces, and not get surprised that the allocation rate is very high yet not showing up in any profiles. Thanks, /Erik On 2018-01-26 06:45, JC Beyler wrote: > Thanks Robbin for the reviews :) > > The new full webrev is here: > http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.03/ > The incremental webrev is here: > http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.02_03/ > > I inlined my answers: > > On Thu, Jan 25, 2018 at 1:15 AM, Robbin Ehn wrote: >> Hi JC, great to see another revision! >> >> #### >> heapMonitoring.cpp >> >> StackTraceData should not contain the oop for 'safety' reasons. >> When StackTraceData is moved from _allocated_traces: >> L452 store_garbage_trace(trace); >> it contains a dead oop. >> _allocated_traces could instead be a tupel of oop and StackTraceData thus >> dead oops are not kept. > Done I used inheritance to make the copier work regardless but the > idea is the same. > >> You should use the new Access API for loading the oop, something like this: >> RootAccess::load(...) >> I don't think you need to use Access API for clearing the oop, but it would >> look nicer. And you shouldn't probably be using: >> Universe::heap()->is_in_reserved(value) > I am unfamiliar with this but I think I did do it like you wanted me > to (all tests pass so that's a start). I'm not sure how to clear the > oop exactly, is there somewhere that does that, which I can use to do > the same? > > I removed the is_in_reserved, this came from our internal version, I > don't know why it was there but my tests work without so I removed it > :) > > >> The lock: >> L424 MutexLocker mu(HeapMonitorStorage_lock); >> Is not needed as far as I can see. >> weak_oops_do is called in a safepoint, no TLAB allocation can happen and >> JVMTI thread can't access these data-structures. Is there something more to >> this lock that I'm missing? > Since a thread can call the JVMTI getLiveTraces (or any of the other > ones), it can get to the point of trying to copying the > _allocated_traces. I imagine it is possible that this is happening > during a GC or that it can be started and a GC happens afterwards. > Therefore, it seems to me that you want this protected, no? > > >> #### >> You have 6 files without any changes in them (any more): >> g1CollectedHeap.cpp >> psMarkSweep.cpp >> psParallelCompact.cpp >> genCollectedHeap.cpp >> referenceProcessor.cpp >> thread.hpp >> > Done. > >> #### >> I have not looked closely, but is it possible to hide heap sampling in >> AllocTracer ? (with some minor changes to the AllocTracer API) >> > I am imagining that you are saying to move the code that does the > sampling code (change the tlab end, do the call to HeapMonitoring, > etc.) into the AllocTracer code itself? I think that is right and I'll > look if that is possible and prepare a webrev to show what would be > needed to make that happen. > >> #### >> Minor nit, when declaring pointer there is a little mix of having the >> pointer adjacent by type name and data name. (Most hotspot code is by type >> name) >> E.g. >> heapMonitoring.cpp:711 jvmtiStackTrace *trace = .... >> heapMonitoring.cpp:733 Method* m = vfst.method(); >> (not just this file) >> > Done! > >> #### >> HeapMonitorThreadOnOffTest.java:77 >> I would make g_tmp volatile, otherwise the assignment in loop may >> theoretical be skipped. >> > Also done! > > Thanks again! > Jc From parvathi.somashekar at oracle.com Fri Feb 2 17:35:37 2018 From: parvathi.somashekar at oracle.com (Paru Somashekar) Date: Fri, 02 Feb 2018 09:35:37 -0800 Subject: RFR 8193150: Create a jtreg version of the test from JDK-8187143 Message-ID: <5A74A169.5020604@oracle.com> Hi, Please review the fix for JDK-8193150. The fix introduces a new jtreg test, NashornPopFrameTest. It is based on the original test from JDK-8187143 that was provided by the customer. Bug : https://bugs.openjdk.java.net/browse/JDK-8193150 Webrev : http://cr.openjdk.java.net/~psomashe/8193150/webrev/ Here is a brief description of what the test does :- * The debuggee, creates and uses a Nashorn engine to evaluate a simple script. * The debugger tries to set a breakpoint in Nashorn's internal DEBUGGER method. * When the breakpoint is reached, it looks for stack frame whose method's declaring type name starts with (nashorn dynamically generated classes) "jdk.nashorn.internal.scripts.Script$". * It then pops stack frames using the ThreadReference.popFrames() call, up to and including the above stackframe. * The execution of the debuggee application is resumed after the needed frames have been popped. This test is included in the ProblemList as it fails under some circumstances (bug JDK-8187143) . Is always passes with the -Xint flag however always fails with -Xcomp. It fails intermittently with the -Xmixed (default). thanks, Paru. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikhailo.seledtsov at oracle.com Fri Feb 2 19:18:21 2018 From: mikhailo.seledtsov at oracle.com (Mikhailo Seledtsov) Date: Fri, 02 Feb 2018 11:18:21 -0800 Subject: 8161605 : The '!UseSharedSpaces' check is not need in JvmtiManageCapabilities::recompute_always_capabilities In-Reply-To: References: <1c8e3f0e-8aa1-574d-f24f-b78b0dc3e6bd@oracle.com> Message-ID: <5A74B97D.5090901@oracle.com> Hi Alex, Several test-related comments: - CanGenerateAllClassHook.java " 44 * So the test runs 2 java process": s/process/process*es*/ - CanGenerateAllClassHook.java A general comment - please consider using "TestCommon" class and "CDSTestUtils" where possible. There are useful methods for forming ProcessBuilder with parameters specific to AppCDS, and also for checking results. In some cases the test case is so custom that use of these utilities is not applicable. Please take a look at ClassFileLoadHookTest.java as an example. It uses TestCommon.executeAndLog(), TestCommon.checkExec() and more. Other good examples are SharedArchiveFile.java, HelloExtTest.java or other tests under appcds directory. Important to note a condition of "mapping failure". When using CDS/AppCDS with Xshare:on, the mapping of a CDS archive to memory may fail at random (due to ASLR or other conditions). TestCommon.checkExec() handles such conditions propertly, otherwise you will have random/intermittent test failures. - CanGenerateAllClassHook.java 122 private static boolean isWin() { I recommend using jdk.test.lib.Platform, the isWindows() method. The SQE guideline is to use test utilities where possible. FYI, the test utilities reside in: open/test/lib/jdk/test/lib The rest of the tests look good, Thank you, Misha On 2/1/18, 4:11 PM, Alex Menkov wrote: > Hi Serguei, > > Thanks for the feedback. > > Updated fix: > http://cr.openjdk.java.net/~amenkov/can_generate_class_hook/webrev.02/ > > --alex > > On 02/01/2018 11:00, serguei.spitsyn at oracle.com wrote: >> One more attempt to send it with the correct to-list... >> Sorry for the noise. >> >> >> Hi Alex, >> >> It looks good in general. >> A couple of comments on the test. >> >> We had a convention to avoid having test sub-folders with bug numbers. >> Could you, please, rename it to some symbolic name? >> >> Also we need the test to fail in the native agent when an error is >> reported with the reportError(). >> >> It'd be great if some CDS/AppCDS experts from the Runtime team look >> at the fix and test >> so I've included Jiangli and Misha into the to-list. >> >> Thanks, >> Serguei >> >> >> On 1/25/18 16:11, Alex Menkov wrote: >>> Hi all, >>> >>> Please take a look at a fix for >>> JDK-8161605 : The '!UseSharedSpaces' check is not need in >>> JvmtiManageCapabilities::recompute_always_capabilities >>> >>> The fix moves can_generate_all_class_hook_events capability from >>> "onload_capabilities" to "always_capabilities" and cleans up >>> recompute_always_capabilities method. >>> >>> jira: https://bugs.openjdk.java.net/browse/JDK-8161605 >>> webrev: >>> http://cr.openjdk.java.net/~amenkov/can_generate_class_hook/webrev/ >>> >>> --alex >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From hohensee at amazon.com Fri Feb 2 21:14:18 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Fri, 2 Feb 2018 21:14:18 +0000 Subject: RFR (S): 8195115: G1 Old Gen MemoryPool CollectionUsage.used values don't reflect mixed GC results In-Reply-To: <93417506-8e12-6ad3-690a-439e36719652@oracle.com> References: <3A67BF1B-CE35-4759-B5B4-959C24020A45@amazon.com> <22ceea57-7d27-8350-4457-21f765cb3d0f@oracle.com> <878D6678-84CB-471E-B126-AF14B1BC8EB9@amazon.com> <727f9bcb-9404-df32-917f-70aca6e31cb6@oracle.com> <4BA32239-D645-4B1A-A58E-502258C8FFB7@amazon.com> <997a766f-9f4b-f192-ae87-03f4c8d2666d@oracle.com> <64793ff3-4805-79f9-4e72-6e469cd511dc@oracle.com> <15754d34-f3c7-6431-7abf-05214bd6b9d2@oracle.com> <93417506-8e12-6ad3-690a-439e36719652@oracle.com> Message-ID: <99C937B6-F67B-4F4A-ACC8-444B5F83932B@amazon.com> +hotspot-gc-use. I?ve filed a CSR for the current patch, see https://bugs.openjdk.java.net/browse/JDK-8196719. Here?s the argument in favor. It?s possible that there are JDK8 users that rely on current G1 old gen CollectionUsage behavior, but imo it?s unlikely because it?s of little use. Perhaps Kirk and others with operational experience can weigh in. Let?s think about use cases. G1 full GC?s happen rarely and only under severe pressure, so when they do external reaction is pretty much limited to reducing load so the JVM can get back to a usable steady state, or just restarting the JVM. Old gen CollectionUsage is zero until a full GC occurs, after which its value includes both long-lived objects and any transient data that was in eden and the survivor space. That value doesn?t tell you anything about long term old gen occupancy or survivor size because it lumps them all together. So, it isn?t a useful metric, nor will it be after any subsequent full GCs. The only information it provides is on the first zero to non-zero transition, which just tells you that the JVM is/was in trouble. Further, the effect of the runup to a full GC is SLA violations, which are noticed before the full GC happens, so detecting the first full GC is confirmation, not prediction. Conclusion: G1 old gen CollectionUsage is unlikely to be in use in its current form, so changing its definition to something usable is low risk. ?G1 Old Space? is fine, as is ?G1 Archive Space?. Are you ok with the G1 archive space overlapping the G1 old space? Should we add an archive space to the other collectors? If so, how would it be defined and would having it overlap with the old generation as a live prefix be ok? "G1 Young Generation" is the currently young+mixed collector. You?re right, if one is iterating over all collectors, there will be redundancy if we keep the old ones. I?m usually leery of introducing a flag such as UseG1LegacyMXBeans (I changed the name, since all the interfaces are MXBeans, hope that?s ok) which must be either indefinitely maintained, or go through a deprecation cycle. I don?t see a way out of the ?iterate over all collectors? problem without it though. Paul On 1/31/18, 3:42 AM, "Erik Helin" wrote: On 01/31/2018 02:30 AM, Hohensee, Paul wrote: > It?s true that my patch doesn?t completely solve the larger problem, but it fixes the most immediately important part of it, particularly for JDK8 where current expected behavior is entrenched. Yes, your patch fixes part of the problem, but as I said, can potentially lead to more confusion. I'm not sure that doing this behavioral change for a public API in an JDK 8 update release is the right thing. There are likely users that rely on the memory pool "G1 Old Gen" only being updated by a full collection (even though that behavior is not correct), those uses will encounter a new behavior in an update release with your patch. The good thing is that we have very experienced engineers participating in the CSR process that have much more experience than I have in evaluating the impact of behavioral changes such as this one. Would you please file a CSR request for your patch to get their opinion? See https://wiki.openjdk.java.net/display/csr/Main for more details about CSR. On 01/31/2018 02:30 AM, Hohensee, Paul wrote: If we?re going to fix the larger problem, imo we should file another bug/rfe to do it. I?d be happy to fix that one too, once we have a spec. > > What do you think of my suggestions? To summarize: > > - Keep the existing memory pools and add humongous and archive pools. > - Make the archive pool part of the old gen, and generalize it to all collectors. > - Split humongous regions off from the old gen pool into their own pool. The old gen and humongous pools are disjoint region sets. > - Keep the existing ?G1 Young Generation? and ?G1 Old Generation? collectors and their associated memory pools (net of this patch). We add the humongous pool to them. > - Add ?G1 Full? as an alias of the existing ?G1 Old Generation? collector. > - Add the ?G1 Young?, ?G1 Mixed? and ?G1 Concurrent Cycle? collectors. > - Set the G1 old gen memory pool max size to ?Xmx, the archive space max size to whatever it is, and the rest of the G1 memory pool max sizes to -1 == undefined, as now. > > The resulting memory pools: > > ?G1 Eden Space? > ?G1 Survivor Space? > ?G1 Old Gen? > ?G1 Humongous Space? > ?Archive Space? The "Space" suffix is unfortunate, but acceptable. I'm least happy about the "Gen" suffix for the "G1 Old Gen", since G1's old regions differ from a generation in the traditional sense as applied to e.g. Serial, Parallel and CMS. I would be more happy to use a consistent naming scheme in the form of "G1 Old Space" (having only one pool ending "Gen" begs the question how it differs from the others ending in "Space"). Again, we could introduce a flag such as -XX:+UseG1LegacyPoolsAndBeans for those that really wants the old names. "Archive Space" should be named "G1 Archive Space" since it differs in implementation from the other collectors. It would be unfortunate if users thought they could change collector and the "Archive Space" memory pool would keep the same behavior. > The resulting collectors and their memory pools: > > ?G1 Young Generation? (the existing young/mixed collector), ?G1 Old Generation?/?G1 Full?, ?G1 Mixed? > - ?G1 Eden Space? > - ?G1 Survivor Space? > - ?G1 Old Gen? > - ?G1 Humongous Space? > ?G1 Young? > - ?G1 Eden Space? > - ?G1 Survivor Space? > - ?G1 Humongous Space? > ?G1 Concurrent Cycle? > - ?G1 Old Gen? > - ?G1 Humongous Space? > > I?m not religious about the names, but I like my suggestions. :) I think it will be confusing for users to have both "G1 Old Generation" and "G1 Full", particularly for tools iterating over all GarbageCollectorMXBeans. There is no way to indicate that a GarbageCollectorMXBeans is an alias of another GarbageCollectorMXBean (I thought about such a solution as well). I guess I don't follow what the GarbageCollectorMXBean "G1 Young Generation" is meant to represent? > The significant addition to my previous email, and an incompatible change, is splitting humongous regions off from the old gen pool. This means that apps that specifically monitor old gen occupancy will no longer see humongous regions. Monitoring apps that just add up info about all a collector?s pools won?t see a difference. I may be corrected (by Kirk, perhaps), but imo it?s not as bad a compatibility issue as one might think, because the type of app that uses a lot of humongous regions isn?t all that common. E.g., apps that cache data in the heap in the form of large compressed arrays, and apps with large hashmap bucket list arrays. The heaps such apps use are very often large enough to use 32mb regions, hence need really big objects to actually allocate humongous regions. So why not enable backwards compatibility by allowing a user to set the flag -XX:+UseG1LegacyPoolsAndBeans? It is not that cumbersome for us to maintain the current definition of memory pools and collectors. Such a flag allows us to start over and do this right and a user who relies on the current behavior can get that by just setting a flag. Doing such a change in a major release also allows us to clearly highlight the change in the release notes (users are more prepared for larger changes in a major release and that they might have to add flags to keep old behavior). It is not uncommon for memory pools to change in major releases. The perm gen pool was removed in JDK 8, the default pools changed when Parallel Old became default old collector way back in JDK 7 and changed again when G1 became the default collector in JDK 9. Thanks, Erik > Thanks, > > Paul > > On 1/30/18, 5:51 AM, "Erik Helin" wrote: > > On 01/30/2018 03:07 AM, Hohensee, Paul wrote: > > That?s one reviewer who?s ok with a short term patch. Anyone else? And, > > any reviewers for said short term patch? :) > > Well, the patch is not really complete as it is. The problem is the > definitions of the MemoryPoolMXBeans and GarbageCollectorMXBeans, which, > as I tried to hint at in my first email, is a mess for G1. The names and > implementations of these MemoryPoolMXBeans and GarbageCollectionMXBeans > for G1 are very old, G1 has changed a lot since those were implemented > (hence my suggestion for finally fixing this). > > The issue with your patch is that the MemoryPoolMXBean named "G1 Old > Gen" consists of both old and humongous regions (it will also include > archive regions). Old regions can be collected by mixed, concurrent and > full collections. Humongous regions can be collected by young, mixed or > full collections and the concurrent cycle. Archive regions will never be > collected. Your patch will update the pool in the case of a mixed > collection collecting old regions or humongous regions, but misses the > following cases: > - a young collection collecting humongous regions > - a concurrent cycle collecting humongous regions > - a concurrent cycle collecting old regions > > Unfortunately I could not come up with a good way to solve the above > without re-designing the pools. I'm not sure about accepting your patch > as is, since it might cause even more confusion for users compared to > the current (already confusing) situation. > > One idea we have discussed is to implement the re-design but also add a > flag, -XX:+UseG1LegacyPoolsAndBeans (false by default), to allow for a > smoother transition. Would that solution work for you? > > Thanks, > Erik > > > Thanks, > > > > Paul > > > > *From: *mandy chung > > *Organization: *Oracle Corporation > > *Date: *Monday, January 29, 2018 at 1:41 PM > > *To: *"Hohensee, Paul" > > *Cc: *"serviceability-dev at openjdk.java.net" > > , "hotspot-gc-dev at openjdk.java.net" > > > > *Subject: *Re: RFR (S): 8195115: G1 Old Gen MemoryPool > > CollectionUsage.used values don't reflect mixed GC results > > > > I created JDK-8196362 to look into whether it makes sense to provide > > some categorization to differentiate eden space vs the heap space for > > long-lived objects. > > > > W.r.t. to JDK-8195115, I have to defer to GC team to comment on the > > mixed collection update. If they are okay, I have no objection to > > implement a short-term fix and do the proper G1 memory pools as a > > separate patch. > > > > Mandy > > > > On 1/29/18 1:02 PM, Hohensee, Paul wrote: > > > > We don?t use getType, and you guessed correctly: we use the memory > > pool name as an indicator of the specific characteristics of a > > memory pool, in particular eden. > > > > What we want is an indication of long term heap occupancy. We > > calculate it using CollectionUsage for non-eden heap memory pools, > > regardless of collector. We don?t use JMX notification, rather we > > periodically poll CollectionUsage for memory pools with names that > > contain ?Old?, ?Tenured?, or ?Survivor?. We get the memory pools > > from the GarbageCollectorMXBeans (we don?t care what the collector > > names are). For the named memory pools, we sum CollectionUsage.used > > and divide by the sum of CollectionUsage.max to get a long term heap > > occupancy percentage. We don?t want to include eden because it?s > > really just an allocation buffer and not part of the storage for > > long-lived objects. I suppose we could use a negative test instead > > by using all memory pools with names that don?t include ?Eden?. > > > > The bug is that the ?G1 Old Gen? memory pool isn?t being updated > > when the ?G1 Young Generation? collector runs a mixed collection. As > > far as JMX is concerned, that collector only knows about eden and > > the survivor space. The patch adds the old gen to the memory pools > > it knows about and has mixed collections update the old gen?s > > CollectionUsage. > > > > I managed to get a submit repo run to succeed last week and it found > > a problem. I?ve uploaded a new webrev that fixes the failure of the > > jtreg test TestMemoryMXBeansAndPoolsPresence.java due to the young > > gen collector being expected to know only about eden and the > > survivor space. > > > > http://cr.openjdk.java.net/~phh/8195115/webrev.hs.01/ > > > > > > Waiting on the submit repo to come back with a result on it. > > > > Thanks, > > > > Paul > > > > *From: *mandy chung > > > > *Organization: *Oracle Corporation > > *Date: *Monday, January 29, 2018 at 10:52 AM > > *To: *"Hohensee, Paul" > > , Erik Helin > > , David Holmes > > > > *Cc: *"serviceability-dev at openjdk.java.net" > > > > > > , > > "hotspot-gc-dev at openjdk.java.net" > > > > > > > > *Subject: *Re: RFR (S): 8195115: G1 Old Gen MemoryPool > > CollectionUsage.used values don't reflect mixed GC results > > > > On 1/29/18 10:35 AM, mandy chung wrote: > > > > Thanks for the reply Paul. Try to understand a little more on > > the specific from gc-specific memory pool you depend on. > > > > On 1/29/18 8:27 AM, Hohensee, Paul wrote: > > > > A name change would affect Amazon?s heap monitoring, and > > thus I expect it would affect other users as well. > > > > As long as there are gc-specific memory pools, we?re going > > to need to be able to identify them, and right now that?s > > done via name. > > > > > > MemoryPoolMXBean::getType returns "heap" memory type for > > GC-specific memory pools. Are you using this method? Do you > > use the name to build in specific characteristic of a memory > > pool (e.g. eden vs old gen)? > > > > > > > > > > All the mxbeans are identified by name, so that?s a general > > design principle. The only way I can think of to get rid of > > name dependency would be to figure out what abstract metrics > > users want to monitor and implement them for all collectors. > > HeapUsage (instantaneous occupancy) is one, CollectionUsage > > (long-lived occupancy) is another, both of these for the > > entire heap, not just particular memory pools. > > > > > > The sum of HeapUsage and CollectionUsage of all heap memory > > pools was expected to give an incorrect approximation for the > > entire heap usage. Are you seeing issue/bug with the sum result? > > > > > > typo: s/an incorrect approximation/an approximation. > > > > Mandy > > > > > > > > Mandy > > > > > > > > That said, imo there will always be a demand for the ability > > to get collector and memory pool specific details, so I > > don?t see a way to get around providing named entities. > > > > Paul > > > > *From: *mandy chung > > > > *Organization: *Oracle Corporation > > *Date: *Friday, January 26, 2018 at 2:38 PM > > *To: *"Hohensee, Paul" > > , Erik Helin > > , > > David Holmes > > > > *Cc: *"serviceability-dev at openjdk.java.net" > > > > > > , > > "hotspot-gc-dev at openjdk.java.net" > > > > > > > > *Subject: *Re: RFR (S): 8195115: G1 Old Gen MemoryPool > > CollectionUsage.used values don't reflect mixed GC results > > > > On 1/25/18 1:04 PM, Hohensee, Paul wrote: > > > > > > > The JMX API spec doesn?t specify what the memory pool or > > garbage > collector names are, but the current names are > > de-facto part of the > API, so if we change the existing > > ones, imo a CSR should be filed. > > > > The names are implementation details but I can see how an application > > > > might be impacted if they depend on it. CSR approval is not strictly > > > > necessary while I think filing one to document the change would be > > > > good. > > > > Does the name change impact any application you know of? I'm trying to > > > > understand if any improvement to API is needed so that applications > > > > don't need to depend on the names. > > > > > > Mandy > > > > > > > > > > > > > > > > > > From hohensee at amazon.com Fri Feb 2 21:19:18 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Fri, 2 Feb 2018 21:19:18 +0000 Subject: RFR (S): 8195115: G1 Old Gen MemoryPool CollectionUsage.used values don't reflect mixed GC results In-Reply-To: <99C937B6-F67B-4F4A-ACC8-444B5F83932B@amazon.com> References: <3A67BF1B-CE35-4759-B5B4-959C24020A45@amazon.com> <22ceea57-7d27-8350-4457-21f765cb3d0f@oracle.com> <878D6678-84CB-471E-B126-AF14B1BC8EB9@amazon.com> <727f9bcb-9404-df32-917f-70aca6e31cb6@oracle.com> <4BA32239-D645-4B1A-A58E-502258C8FFB7@amazon.com> <997a766f-9f4b-f192-ae87-03f4c8d2666d@oracle.com> <64793ff3-4805-79f9-4e72-6e469cd511dc@oracle.com> <15754d34-f3c7-6431-7abf-05214bd6b9d2@oracle.com> <93417506-8e12-6ad3-690a-439e36719652@oracle.com> <99C937B6-F67B-4F4A-ACC8-444B5F83932B@amazon.com> Message-ID: And, can a get a reviewer or reviewers for the CSR? Thanks, Paul On 2/2/18, 1:14 PM, "Hohensee, Paul" wrote: +hotspot-gc-use. I?ve filed a CSR for the current patch, see https://bugs.openjdk.java.net/browse/JDK-8196719. Here?s the argument in favor. It?s possible that there are JDK8 users that rely on current G1 old gen CollectionUsage behavior, but imo it?s unlikely because it?s of little use. Perhaps Kirk and others with operational experience can weigh in. Let?s think about use cases. G1 full GC?s happen rarely and only under severe pressure, so when they do external reaction is pretty much limited to reducing load so the JVM can get back to a usable steady state, or just restarting the JVM. Old gen CollectionUsage is zero until a full GC occurs, after which its value includes both long-lived objects and any transient data that was in eden and the survivor space. That value doesn?t tell you anything about long term old gen occupancy or survivor size because it lumps them all together. So, it isn?t a useful metric, nor will it be after any subsequent full GCs. The only information it provides is on the first zero to non-zero transition, which just tells you that the JVM is/was in trouble. Further, the effect of the runup to a full GC is SLA violations, which are noticed before the full GC happens, so detecting the first full GC is confirmation, not prediction. Conclusion: G1 old gen CollectionUsage is unlikely to be in use in its current form, so changing its definition to something usable is low risk. ?G1 Old Space? is fine, as is ?G1 Archive Space?. Are you ok with the G1 archive space overlapping the G1 old space? Should we add an archive space to the other collectors? If so, how would it be defined and would having it overlap with the old generation as a live prefix be ok? "G1 Young Generation" is the currently young+mixed collector. You?re right, if one is iterating over all collectors, there will be redundancy if we keep the old ones. I?m usually leery of introducing a flag such as UseG1LegacyMXBeans (I changed the name, since all the interfaces are MXBeans, hope that?s ok) which must be either indefinitely maintained, or go through a deprecation cycle. I don?t see a way out of the ?iterate over all collectors? problem without it though. Paul On 1/31/18, 3:42 AM, "Erik Helin" wrote: On 01/31/2018 02:30 AM, Hohensee, Paul wrote: > It?s true that my patch doesn?t completely solve the larger problem, but it fixes the most immediately important part of it, particularly for JDK8 where current expected behavior is entrenched. Yes, your patch fixes part of the problem, but as I said, can potentially lead to more confusion. I'm not sure that doing this behavioral change for a public API in an JDK 8 update release is the right thing. There are likely users that rely on the memory pool "G1 Old Gen" only being updated by a full collection (even though that behavior is not correct), those uses will encounter a new behavior in an update release with your patch. The good thing is that we have very experienced engineers participating in the CSR process that have much more experience than I have in evaluating the impact of behavioral changes such as this one. Would you please file a CSR request for your patch to get their opinion? See https://wiki.openjdk.java.net/display/csr/Main for more details about CSR. On 01/31/2018 02:30 AM, Hohensee, Paul wrote: If we?re going to fix the larger problem, imo we should file another bug/rfe to do it. I?d be happy to fix that one too, once we have a spec. > > What do you think of my suggestions? To summarize: > > - Keep the existing memory pools and add humongous and archive pools. > - Make the archive pool part of the old gen, and generalize it to all collectors. > - Split humongous regions off from the old gen pool into their own pool. The old gen and humongous pools are disjoint region sets. > - Keep the existing ?G1 Young Generation? and ?G1 Old Generation? collectors and their associated memory pools (net of this patch). We add the humongous pool to them. > - Add ?G1 Full? as an alias of the existing ?G1 Old Generation? collector. > - Add the ?G1 Young?, ?G1 Mixed? and ?G1 Concurrent Cycle? collectors. > - Set the G1 old gen memory pool max size to ?Xmx, the archive space max size to whatever it is, and the rest of the G1 memory pool max sizes to -1 == undefined, as now. > > The resulting memory pools: > > ?G1 Eden Space? > ?G1 Survivor Space? > ?G1 Old Gen? > ?G1 Humongous Space? > ?Archive Space? The "Space" suffix is unfortunate, but acceptable. I'm least happy about the "Gen" suffix for the "G1 Old Gen", since G1's old regions differ from a generation in the traditional sense as applied to e.g. Serial, Parallel and CMS. I would be more happy to use a consistent naming scheme in the form of "G1 Old Space" (having only one pool ending "Gen" begs the question how it differs from the others ending in "Space"). Again, we could introduce a flag such as -XX:+UseG1LegacyPoolsAndBeans for those that really wants the old names. "Archive Space" should be named "G1 Archive Space" since it differs in implementation from the other collectors. It would be unfortunate if users thought they could change collector and the "Archive Space" memory pool would keep the same behavior. > The resulting collectors and their memory pools: > > ?G1 Young Generation? (the existing young/mixed collector), ?G1 Old Generation?/?G1 Full?, ?G1 Mixed? > - ?G1 Eden Space? > - ?G1 Survivor Space? > - ?G1 Old Gen? > - ?G1 Humongous Space? > ?G1 Young? > - ?G1 Eden Space? > - ?G1 Survivor Space? > - ?G1 Humongous Space? > ?G1 Concurrent Cycle? > - ?G1 Old Gen? > - ?G1 Humongous Space? > > I?m not religious about the names, but I like my suggestions. :) I think it will be confusing for users to have both "G1 Old Generation" and "G1 Full", particularly for tools iterating over all GarbageCollectorMXBeans. There is no way to indicate that a GarbageCollectorMXBeans is an alias of another GarbageCollectorMXBean (I thought about such a solution as well). I guess I don't follow what the GarbageCollectorMXBean "G1 Young Generation" is meant to represent? > The significant addition to my previous email, and an incompatible change, is splitting humongous regions off from the old gen pool. This means that apps that specifically monitor old gen occupancy will no longer see humongous regions. Monitoring apps that just add up info about all a collector?s pools won?t see a difference. I may be corrected (by Kirk, perhaps), but imo it?s not as bad a compatibility issue as one might think, because the type of app that uses a lot of humongous regions isn?t all that common. E.g., apps that cache data in the heap in the form of large compressed arrays, and apps with large hashmap bucket list arrays. The heaps such apps use are very often large enough to use 32mb regions, hence need really big objects to actually allocate humongous regions. So why not enable backwards compatibility by allowing a user to set the flag -XX:+UseG1LegacyPoolsAndBeans? It is not that cumbersome for us to maintain the current definition of memory pools and collectors. Such a flag allows us to start over and do this right and a user who relies on the current behavior can get that by just setting a flag. Doing such a change in a major release also allows us to clearly highlight the change in the release notes (users are more prepared for larger changes in a major release and that they might have to add flags to keep old behavior). It is not uncommon for memory pools to change in major releases. The perm gen pool was removed in JDK 8, the default pools changed when Parallel Old became default old collector way back in JDK 7 and changed again when G1 became the default collector in JDK 9. Thanks, Erik > Thanks, > > Paul > > On 1/30/18, 5:51 AM, "Erik Helin" wrote: > > On 01/30/2018 03:07 AM, Hohensee, Paul wrote: > > That?s one reviewer who?s ok with a short term patch. Anyone else? And, > > any reviewers for said short term patch? :) > > Well, the patch is not really complete as it is. The problem is the > definitions of the MemoryPoolMXBeans and GarbageCollectorMXBeans, which, > as I tried to hint at in my first email, is a mess for G1. The names and > implementations of these MemoryPoolMXBeans and GarbageCollectionMXBeans > for G1 are very old, G1 has changed a lot since those were implemented > (hence my suggestion for finally fixing this). > > The issue with your patch is that the MemoryPoolMXBean named "G1 Old > Gen" consists of both old and humongous regions (it will also include > archive regions). Old regions can be collected by mixed, concurrent and > full collections. Humongous regions can be collected by young, mixed or > full collections and the concurrent cycle. Archive regions will never be > collected. Your patch will update the pool in the case of a mixed > collection collecting old regions or humongous regions, but misses the > following cases: > - a young collection collecting humongous regions > - a concurrent cycle collecting humongous regions > - a concurrent cycle collecting old regions > > Unfortunately I could not come up with a good way to solve the above > without re-designing the pools. I'm not sure about accepting your patch > as is, since it might cause even more confusion for users compared to > the current (already confusing) situation. > > One idea we have discussed is to implement the re-design but also add a > flag, -XX:+UseG1LegacyPoolsAndBeans (false by default), to allow for a > smoother transition. Would that solution work for you? > > Thanks, > Erik > > > Thanks, > > > > Paul > > > > *From: *mandy chung > > *Organization: *Oracle Corporation > > *Date: *Monday, January 29, 2018 at 1:41 PM > > *To: *"Hohensee, Paul" > > *Cc: *"serviceability-dev at openjdk.java.net" > > , "hotspot-gc-dev at openjdk.java.net" > > > > *Subject: *Re: RFR (S): 8195115: G1 Old Gen MemoryPool > > CollectionUsage.used values don't reflect mixed GC results > > > > I created JDK-8196362 to look into whether it makes sense to provide > > some categorization to differentiate eden space vs the heap space for > > long-lived objects. > > > > W.r.t. to JDK-8195115, I have to defer to GC team to comment on the > > mixed collection update. If they are okay, I have no objection to > > implement a short-term fix and do the proper G1 memory pools as a > > separate patch. > > > > Mandy > > > > On 1/29/18 1:02 PM, Hohensee, Paul wrote: > > > > We don?t use getType, and you guessed correctly: we use the memory > > pool name as an indicator of the specific characteristics of a > > memory pool, in particular eden. > > > > What we want is an indication of long term heap occupancy. We > > calculate it using CollectionUsage for non-eden heap memory pools, > > regardless of collector. We don?t use JMX notification, rather we > > periodically poll CollectionUsage for memory pools with names that > > contain ?Old?, ?Tenured?, or ?Survivor?. We get the memory pools > > from the GarbageCollectorMXBeans (we don?t care what the collector > > names are). For the named memory pools, we sum CollectionUsage.used > > and divide by the sum of CollectionUsage.max to get a long term heap > > occupancy percentage. We don?t want to include eden because it?s > > really just an allocation buffer and not part of the storage for > > long-lived objects. I suppose we could use a negative test instead > > by using all memory pools with names that don?t include ?Eden?. > > > > The bug is that the ?G1 Old Gen? memory pool isn?t being updated > > when the ?G1 Young Generation? collector runs a mixed collection. As > > far as JMX is concerned, that collector only knows about eden and > > the survivor space. The patch adds the old gen to the memory pools > > it knows about and has mixed collections update the old gen?s > > CollectionUsage. > > > > I managed to get a submit repo run to succeed last week and it found > > a problem. I?ve uploaded a new webrev that fixes the failure of the > > jtreg test TestMemoryMXBeansAndPoolsPresence.java due to the young > > gen collector being expected to know only about eden and the > > survivor space. > > > > http://cr.openjdk.java.net/~phh/8195115/webrev.hs.01/ > > > > > > Waiting on the submit repo to come back with a result on it. > > > > Thanks, > > > > Paul > > > > *From: *mandy chung > > > > *Organization: *Oracle Corporation > > *Date: *Monday, January 29, 2018 at 10:52 AM > > *To: *"Hohensee, Paul" > > , Erik Helin > > , David Holmes > > > > *Cc: *"serviceability-dev at openjdk.java.net" > > > > > > , > > "hotspot-gc-dev at openjdk.java.net" > > > > > > > > *Subject: *Re: RFR (S): 8195115: G1 Old Gen MemoryPool > > CollectionUsage.used values don't reflect mixed GC results > > > > On 1/29/18 10:35 AM, mandy chung wrote: > > > > Thanks for the reply Paul. Try to understand a little more on > > the specific from gc-specific memory pool you depend on. > > > > On 1/29/18 8:27 AM, Hohensee, Paul wrote: > > > > A name change would affect Amazon?s heap monitoring, and > > thus I expect it would affect other users as well. > > > > As long as there are gc-specific memory pools, we?re going > > to need to be able to identify them, and right now that?s > > done via name. > > > > > > MemoryPoolMXBean::getType returns "heap" memory type for > > GC-specific memory pools. Are you using this method? Do you > > use the name to build in specific characteristic of a memory > > pool (e.g. eden vs old gen)? > > > > > > > > > > All the mxbeans are identified by name, so that?s a general > > design principle. The only way I can think of to get rid of > > name dependency would be to figure out what abstract metrics > > users want to monitor and implement them for all collectors. > > HeapUsage (instantaneous occupancy) is one, CollectionUsage > > (long-lived occupancy) is another, both of these for the > > entire heap, not just particular memory pools. > > > > > > The sum of HeapUsage and CollectionUsage of all heap memory > > pools was expected to give an incorrect approximation for the > > entire heap usage. Are you seeing issue/bug with the sum result? > > > > > > typo: s/an incorrect approximation/an approximation. > > > > Mandy > > > > > > > > Mandy > > > > > > > > That said, imo there will always be a demand for the ability > > to get collector and memory pool specific details, so I > > don?t see a way to get around providing named entities. > > > > Paul > > > > *From: *mandy chung > > > > *Organization: *Oracle Corporation > > *Date: *Friday, January 26, 2018 at 2:38 PM > > *To: *"Hohensee, Paul" > > , Erik Helin > > , > > David Holmes > > > > *Cc: *"serviceability-dev at openjdk.java.net" > > > > > > , > > "hotspot-gc-dev at openjdk.java.net" > > > > > > > > *Subject: *Re: RFR (S): 8195115: G1 Old Gen MemoryPool > > CollectionUsage.used values don't reflect mixed GC results > > > > On 1/25/18 1:04 PM, Hohensee, Paul wrote: > > > > > > > The JMX API spec doesn?t specify what the memory pool or > > garbage > collector names are, but the current names are > > de-facto part of the > API, so if we change the existing > > ones, imo a CSR should be filed. > > > > The names are implementation details but I can see how an application > > > > might be impacted if they depend on it. CSR approval is not strictly > > > > necessary while I think filing one to document the change would be > > > > good. > > > > Does the name change impact any application you know of? I'm trying to > > > > understand if any improvement to API is needed so that applications > > > > don't need to depend on the names. > > > > > > Mandy > > > > > > > > > > > > > > > > > > From yasuenag at gmail.com Sat Feb 3 05:40:44 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Sat, 3 Feb 2018 14:40:44 +0900 Subject: PING: RFR: JDK-8153333: [REDO] STW phases at Concurrent GC should count in PerfCounter In-Reply-To: <3210723e-c5f9-4277-a97a-be61e10e6b3d@oracle.com> References: <2f4e0901-1602-6276-e7fd-84e168e7b317@gmail.com> <3210723e-c5f9-4277-a97a-be61e10e6b3d@oracle.com> Message-ID: <89a1a161-874d-4743-e52b-7ab202fd8976@gmail.com> On 2018/02/02 23:38, Stefan Johansson wrote: > Hi Yasumasa, > > The changes doesn't apply clean on the latest jdk/hs, can you provide an updated webrev? I uploaded webrev for jdk-hs: cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.05/ > The testing done by the submit repo doesn't cover the tests you have update so I plan to take the change for a spin and make sure the correct tests are run and verified in Mach 5. I've also tested hotspot/jtreg/:hotspot_serviceability and jdk/:jdk_tools on my laptop. I did not see any errors / failures which are related to this change. > Also a question about the change. Why do we need a special flag for CMS? I see that the original bug report refers to the flag as being a way to turn on and off the feature but the current implementation only consider the flag for CMS. http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-March/016774.html Originally, STW phases (Remark and Cleanup) at G1 are not counted in jstat FGC column. So I think we need not to control the behavior of PerfCounter for G1. Thanks, Yasumasa > Thanks, > Stefan > > On 2018-02-01 14:58, Yasumasa Suenaga wrote: >> PING: Could you review and sponsor it? >> >>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >> >> This change has been passed Mach 5 via submit repo: >> http://java.se.oracle.com:10065/mdash/jobs/mach5-one-ysuenaga-JDK-8153333-20180201-0805-10101 >> >> >> Thanks, >> >> Yasumasa >> >> >> On 2017/11/01 22:02, Yasumasa Suenaga wrote: >>> PING: Could you review and sponsor it? >>> >>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>> >>> Also I need JPRT results of this change. >>> Could you cooperate? >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2017/09/27 0:08, Yasumasa Suenaga wrote: >>>> Hi all, >>>> >>>> I uploaded new webrev to be adapted to jdk10/hs: >>>> >>>> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>>> >>>> I want to check this patch via JPRT, but I cannot access it. >>>> Could you cooperate? >>>> >>>> >>>> Thanks, >>>> >>>> yasumasa >>>> >>>> >>>> On 2017/09/21 7:46, Yasumasa Suenaga wrote: >>>>> PING: >>>>> >>>>> Have you checked this issue? >>>>> >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/hotspot/ >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/jdk/ >>>>> >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2017/07/01 23:44, Yasumasa Suenaga wrote: >>>>>> PING: >>>>>> >>>>>> Have you checked this issue? >>>>>> >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2017/06/14 13:22, Yasumasa Suenaga wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> I changed PerfCounter to show CGC STW phase in jstat in JDK-8151674. >>>>>>> However, it occurred several jtreg test failure, so it was back-outed. >>>>>>> >>>>>>> I want to resume to work for this issue. >>>>>>> >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/hotspot/ >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/jdk/ >>>>>>> >>>>>>> These changes are work fine on jtreg test as below: >>>>>>> >>>>>>> ?? hotspot/test/serviceability/tmtools/jstat >>>>>>> ?? jdk/test/sun/tools >>>>>>> >>>>>>> >>>>>>> Since JDK 9, default GC algorithm is set to G1. >>>>>>> So I think this change is useful to watch GC behavior through jstat. >>>>>>> >>>>>>> I cannot access JPRT. Could you help? >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> > From david.holmes at oracle.com Sat Feb 3 08:57:50 2018 From: david.holmes at oracle.com (David Holmes) Date: Sat, 3 Feb 2018 18:57:50 +1000 Subject: RFR: 8196521: serviceability/sa/ClhsdbWhere.java fails to find method 'waitForReferencePendingList' in output In-Reply-To: <1afda3cf56994421a9919b9e0834492d@NASANEXM01E.na.qualcomm.com> References: <71d86c90cba444c6967883078e5c5cc3@NASANEXM01E.na.qualcomm.com> <4158546c-434b-acd1-c0b1-5e2c7faf680c@oracle.com> <6ab1e27d-cf2a-9397-a970-232fb1ddf523@oracle.com> <369287e9-a6ab-4d40-a3f3-f81086406bc7@default> <1afda3cf56994421a9919b9e0834492d@NASANEXM01E.na.qualcomm.com> Message-ID: Hi Daniel, Sharath, The signal thread won't be found if the test happens to be run with -Xrs, but that seems unlikely. So seems okay. Your sponsor will need to test on all platforms of course. Thanks, David On 3/02/2018 12:11 AM, stewartd.qdt wrote: > Hi Sharath, > > Thank you for your comments and insight. I have made the requested changes and the new webrev is at: http://cr.openjdk.java.net/~dstewart/8196521/webrev.01/ > > Please have a look and let me know if there are any other changes required. This does pass testing on my system. > > Thanks, > Daniel > > -----Original Message----- > From: Sharath Ballal [mailto:sharath.ballal at oracle.com] > Sent: Friday, February 2, 2018 4:49 AM > To: David Holmes ; Jini Susan George ; stewartd.qdt ; serviceability-dev > Subject: RE: RFR: 8196521: serviceability/sa/ClhsdbWhere.java fails to find method 'waitForReferencePendingList' in output > > Agree with David. It looks like the stack will be different on some machines occasionally depending on the load, machine speed etc. > Hence the following lines in the test > "java.lang.ref.Reference", > "private static void processPendingReferences", > "private static native void waitForReferencePendingList", can be replaced by > Java Stack Trace for Signal Dispatcher > Java Stack Trace for Reference Handler > That atlease ensures we are looking for all the threads. > > Thanks, > Sharath (not a Reviewer) > > > -----Original Message----- > From: David Holmes > Sent: Thursday, February 01, 2018 1:29 PM > To: Jini George; stewartd.qdt; serviceability-dev > Subject: Re: RFR: 8196521: serviceability/sa/ClhsdbWhere.java fails to find method 'waitForReferencePendingList' in output > > On 1/02/2018 5:01 PM, Jini George wrote: >> Hello Daniel, >> >> Your fix looks good to me. You probably could instead add 'remove()' >> or 'run()' to the list, but I leave it upto you. > > These tests should only be looking for things that are guaranteed to be seen. If waitForReferencePendingList is not there, how do we know that processPendingReferences will be there? We seem to be assuming a quiescent system. We know the threads that will be present, but the only stack entries we should be looking for are those controlled by the test code itself. > > David > ----- > >> Thanks, >> Jini (Not a (R)eviewer). >> >> On 2/1/2018 2:39 AM, stewartd.qdt wrote: >>> Please review this webrev [1] which attempts to fix a test error in >>> serviceability/sa/ClhsdbWhere.java when it is run under an AArch64 >>> system (not necessarily exclusive to this system, but it was the >>> system under test). The bug report [2] provides further details and >>> has the jtr report that was generated. Essentially the line >>> "waitForReferencePendingList" never actually occurs. The test is >>> written such that it is expecting that line. >>> >>> This patch simply removes the line from the set of tested lines it >>> expects. ?I'm not overly happy with this approach as it actually >>> removes a test line. However, the test line does not actually appear >>> in the output (at least on my system) and I'm not sure if there is >>> actually another way of testing for the intent of this line or if it >>> doesn't actually have to appear in the output, depending on the >>> system. Perhaps the original author could chime in and provide >>> further guidance as to the intention of the test. >>> >>> I am happy to modify the patch as necessary. >>> >>> Regards, >>> >>> Daniel Stewart >>> >>> [1] - http://cr.openjdk.java.net/~dstewart/8196521/webrev.00/ >>> >>> [2] - https://bugs.openjdk.java.net/browse/JDK-8196521 >>> From sharath.ballal at oracle.com Sat Feb 3 09:15:38 2018 From: sharath.ballal at oracle.com (Sharath Ballal) Date: Sat, 3 Feb 2018 01:15:38 -0800 (PST) Subject: RFR: 8196521: serviceability/sa/ClhsdbWhere.java fails to find method 'waitForReferencePendingList' in output In-Reply-To: References: <71d86c90cba444c6967883078e5c5cc3@NASANEXM01E.na.qualcomm.com> <4158546c-434b-acd1-c0b1-5e2c7faf680c@oracle.com> <6ab1e27d-cf2a-9397-a970-232fb1ddf523@oracle.com> <369287e9-a6ab-4d40-a3f3-f81086406bc7@default> <1afda3cf56994421a9919b9e0834492d@NASANEXM01E.na.qualcomm.com> Message-ID: <95afbdd5-f9d4-45bd-b2db-95e68790f030@default> Looks OK Daniel. I will sponsor the change (and run this change through our internal testing). Thanks, Sharath (not a Reviewer) -----Original Message----- From: David Holmes Sent: Saturday, February 03, 2018 2:28 PM To: stewartd.qdt; Sharath Ballal; Jini Susan George; serviceability-dev Subject: Re: RFR: 8196521: serviceability/sa/ClhsdbWhere.java fails to find method 'waitForReferencePendingList' in output Hi Daniel, Sharath, The signal thread won't be found if the test happens to be run with -Xrs, but that seems unlikely. So seems okay. Your sponsor will need to test on all platforms of course. Thanks, David On 3/02/2018 12:11 AM, stewartd.qdt wrote: > Hi Sharath, > > Thank you for your comments and insight. I have made the requested > changes and the new webrev is at: > http://cr.openjdk.java.net/~dstewart/8196521/webrev.01/ > > Please have a look and let me know if there are any other changes required. This does pass testing on my system. > > Thanks, > Daniel > > -----Original Message----- > From: Sharath Ballal [mailto:sharath.ballal at oracle.com] > Sent: Friday, February 2, 2018 4:49 AM > To: David Holmes ; Jini Susan George > ; stewartd.qdt > ; serviceability-dev > > Subject: RE: RFR: 8196521: serviceability/sa/ClhsdbWhere.java fails to > find method 'waitForReferencePendingList' in output > > Agree with David. It looks like the stack will be different on some machines occasionally depending on the load, machine speed etc. > Hence the following lines in the test > "java.lang.ref.Reference", > "private static void processPendingReferences", > "private static native void waitForReferencePendingList", can be replaced by > Java Stack Trace for Signal Dispatcher > Java Stack Trace for Reference Handler That atlease ensures we are > looking for all the threads. > > Thanks, > Sharath (not a Reviewer) > > > -----Original Message----- > From: David Holmes > Sent: Thursday, February 01, 2018 1:29 PM > To: Jini George; stewartd.qdt; serviceability-dev > Subject: Re: RFR: 8196521: serviceability/sa/ClhsdbWhere.java fails to > find method 'waitForReferencePendingList' in output > > On 1/02/2018 5:01 PM, Jini George wrote: >> Hello Daniel, >> >> Your fix looks good to me. You probably could instead add 'remove()' >> or 'run()' to the list, but I leave it upto you. > > These tests should only be looking for things that are guaranteed to be seen. If waitForReferencePendingList is not there, how do we know that processPendingReferences will be there? We seem to be assuming a quiescent system. We know the threads that will be present, but the only stack entries we should be looking for are those controlled by the test code itself. > > David > ----- > >> Thanks, >> Jini (Not a (R)eviewer). >> >> On 2/1/2018 2:39 AM, stewartd.qdt wrote: >>> Please review this webrev [1] which attempts to fix a test error in >>> serviceability/sa/ClhsdbWhere.java when it is run under an AArch64 >>> system (not necessarily exclusive to this system, but it was the >>> system under test). The bug report [2] provides further details and >>> has the jtr report that was generated. Essentially the line >>> "waitForReferencePendingList" never actually occurs. The test is >>> written such that it is expecting that line. >>> >>> This patch simply removes the line from the set of tested lines it >>> expects. ?I'm not overly happy with this approach as it actually >>> removes a test line. However, the test line does not actually appear >>> in the output (at least on my system) and I'm not sure if there is >>> actually another way of testing for the intent of this line or if it >>> doesn't actually have to appear in the output, depending on the >>> system. Perhaps the original author could chime in and provide >>> further guidance as to the intention of the test. >>> >>> I am happy to modify the patch as necessary. >>> >>> Regards, >>> >>> Daniel Stewart >>> >>> [1] - http://cr.openjdk.java.net/~dstewart/8196521/webrev.00/ >>> >>> [2] - https://bugs.openjdk.java.net/browse/JDK-8196521 >>> From Alan.Bateman at oracle.com Sat Feb 3 09:35:41 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 3 Feb 2018 09:35:41 +0000 Subject: Fwd: [PATCH] Fix compiler warnings with newer GCC 7.2.1 In-Reply-To: References: Message-ID: <4c5248f3-e78d-31ba-c892-fd7df1d027af@oracle.com> This was sent to core-libs-dev, I assume it was meant for 2d-dev and serviceability-dev. -------- Forwarded Message -------- Subject: [PATCH] Fix compiler warnings with newer GCC 7.2.1 Date: Fri, 2 Feb 2018 16:12:08 -0800 From: Indu Bhagat To: core-libs-dev at openjdk.java.net This patch fixes a few of the compiler warnings when using gcc 7.2.1. In general there are many other warnings (a majority of which are implicit fallthrough warnings in jdk.crypto.ec package's ecp_*.c files and java.desktop package's splashscreen_gfx_impl.h file). I can post them if there is interest. Most warnings, however, look benign (modulo some warnings about unintialized access that I cannot comment on; they need that the code be thoroughly understood). libfdlibm misleading indentation warnings are already knownhttp://mail.openjdk.java.net/pipermail/jdk9-dev/2016-June/004494.html Thanks ---------------------------------------------- The patch fixes the following warnings : 1. /jdk/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c: In function ?mask_s2u?: /jdk/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c:389:22: warning: left shift of negative value [-Wshift-negative-value] *mask = htonl(-1 << (32 - m)); ^ 2. /jdk/src/java.desktop/share/native/libjavajpeg/jdphuff.c: In function ?decode_mcu_AC_refine?: /jdk/src/java.desktop/share/native/libjavajpeg/jdphuff.c:505:17: warning: left shift of negative value [-Wshift-negative-value] int m1 = (-1) << cinfo->Al; /* -1 in the bit position being coded*/ ^~ 3. /jdk/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-ot-shape-fallback.cc: In function ?void position_mark(const hb_ot_shape_plan_t*, hb_font_t*, hb_buffer_t*, hb_glyph_extents_t&, unsigned int, unsigned int)?: /jdk/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-ot-shape-fallback.cc:223:14: warning: this statement may fall through [-Wimplicit-fallthrough=] } else if (buffer->props.direction == HB_DIRECTION_RTL) { ^~ /jdk/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-ot-shape-fallback.cc:229:5: note: here default: ^~~~~~~ /jdk/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-ot-shape-fallback.cc:261:27: warning: this statement may fall through [-Wimplicit-fallthrough=] base_extents.height -= y_gap; ~~~~~~~~~~~~~~~~~~~~^~~~~~~~ /jdk/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-ot-shape-fallback.cc:264:5: note: here case HB_UNICODE_COMBINING_CLASS_ATTACHED_BELOW_LEFT: ^~~~ /jdk/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-ot-shape-fallback.cc:282:27: warning: this statement may fall through [-Wimplicit-fallthrough=] base_extents.height -= y_gap; ~~~~~~~~~~~~~~~~~~~~^~~~~~~~ /jdk/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-ot-shape-fallback.cc:285:5: note: here case HB_UNICODE_COMBINING_CLASS_ATTACHED_ABOVE: ^~~~ ----------- diff --git a/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-private.hh b/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-private.hh --- a/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-private.hh +++ b/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-private.hh @@ -158,6 +158,9 @@ */ # include # define HB_FALLTHROUGH __fallthrough +#elif __GNUC__ >= 7 + /* GNU fallthrough attribute is available from GCC7 */ +# define HB_FALLTHROUGH __attribute__((fallthrough)) #else # define HB_FALLTHROUGH /* FALLTHROUGH */ #endif diff --git a/src/java.desktop/share/native/libjavajpeg/jdphuff.c b/src/java.desktop/share/native/libjavajpeg/jdphuff.c --- a/src/java.desktop/share/native/libjavajpeg/jdphuff.c +++ b/src/java.desktop/share/native/libjavajpeg/jdphuff.c @@ -501,8 +501,10 @@ { phuff_entropy_ptr entropy = (phuff_entropy_ptr) cinfo->entropy; int Se = cinfo->Se; - int p1 = 1 << cinfo->Al; /* 1 in the bit position being coded */ - int m1 = (-1) << cinfo->Al; /* -1 in the bit position being coded */ + /* 1 in the bit position being coded */ + int p1 = 1 << cinfo->Al; + /* -1 in the bit position being coded */ + int m1 = (int)((unsigned)(~0) << cinfo->Al); register int s, k, r; unsigned int EOBRUN; JBLOCKROW block; diff --git a/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c b/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c --- a/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c +++ b/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c @@ -386,7 +386,7 @@ return instr; } - *mask = htonl(-1 << (32 - m)); + *mask = htonl((uint32_t)(~0) << (32 - m)); return s; } -------------- next part -------------- An HTML attachment was scrubbed... URL: From sharath.ballal at oracle.com Sat Feb 3 10:23:37 2018 From: sharath.ballal at oracle.com (Sharath Ballal) Date: Sat, 3 Feb 2018 02:23:37 -0800 (PST) Subject: RFR: 8196521: serviceability/sa/ClhsdbWhere.java fails to find method 'waitForReferencePendingList' in output In-Reply-To: References: <71d86c90cba444c6967883078e5c5cc3@NASANEXM01E.na.qualcomm.com> <4158546c-434b-acd1-c0b1-5e2c7faf680c@oracle.com> <6ab1e27d-cf2a-9397-a970-232fb1ddf523@oracle.com> <369287e9-a6ab-4d40-a3f3-f81086406bc7@default> <1afda3cf56994421a9919b9e0834492d@NASANEXM01E.na.qualcomm.com> Message-ID: <014fe405-9178-4440-b236-e8d53b9a8736@default> Thanks David. Thanks, Sharath -----Original Message----- From: David Holmes Sent: Saturday, February 03, 2018 2:28 PM To: stewartd.qdt; Sharath Ballal; Jini Susan George; serviceability-dev Subject: Re: RFR: 8196521: serviceability/sa/ClhsdbWhere.java fails to find method 'waitForReferencePendingList' in output Hi Daniel, Sharath, The signal thread won't be found if the test happens to be run with -Xrs, but that seems unlikely. So seems okay. Your sponsor will need to test on all platforms of course. Thanks, David On 3/02/2018 12:11 AM, stewartd.qdt wrote: > Hi Sharath, > > Thank you for your comments and insight. I have made the requested changes and the new webrev is at: http://cr.openjdk.java.net/~dstewart/8196521/webrev.01/ > > Please have a look and let me know if there are any other changes required. This does pass testing on my system. > > Thanks, > Daniel > > -----Original Message----- > From: Sharath Ballal [mailto:sharath.ballal at oracle.com] > Sent: Friday, February 2, 2018 4:49 AM > To: David Holmes ; Jini Susan George ; stewartd.qdt ; serviceability-dev > Subject: RE: RFR: 8196521: serviceability/sa/ClhsdbWhere.java fails to find method 'waitForReferencePendingList' in output > > Agree with David. It looks like the stack will be different on some machines occasionally depending on the load, machine speed etc. > Hence the following lines in the test > "java.lang.ref.Reference", > "private static void processPendingReferences", > "private static native void waitForReferencePendingList", can be replaced by > Java Stack Trace for Signal Dispatcher > Java Stack Trace for Reference Handler > That atlease ensures we are looking for all the threads. > > Thanks, > Sharath (not a Reviewer) > > > -----Original Message----- > From: David Holmes > Sent: Thursday, February 01, 2018 1:29 PM > To: Jini George; stewartd.qdt; serviceability-dev > Subject: Re: RFR: 8196521: serviceability/sa/ClhsdbWhere.java fails to find method 'waitForReferencePendingList' in output > > On 1/02/2018 5:01 PM, Jini George wrote: >> Hello Daniel, >> >> Your fix looks good to me. You probably could instead add 'remove()' >> or 'run()' to the list, but I leave it upto you. > > These tests should only be looking for things that are guaranteed to be seen. If waitForReferencePendingList is not there, how do we know that processPendingReferences will be there? We seem to be assuming a quiescent system. We know the threads that will be present, but the only stack entries we should be looking for are those controlled by the test code itself. > > David > ----- > >> Thanks, >> Jini (Not a (R)eviewer). >> >> On 2/1/2018 2:39 AM, stewartd.qdt wrote: >>> Please review this webrev [1] which attempts to fix a test error in >>> serviceability/sa/ClhsdbWhere.java when it is run under an AArch64 >>> system (not necessarily exclusive to this system, but it was the >>> system under test). The bug report [2] provides further details and >>> has the jtr report that was generated. Essentially the line >>> "waitForReferencePendingList" never actually occurs. The test is >>> written such that it is expecting that line. >>> >>> This patch simply removes the line from the set of tested lines it >>> expects. ?I'm not overly happy with this approach as it actually >>> removes a test line. However, the test line does not actually appear >>> in the output (at least on my system) and I'm not sure if there is >>> actually another way of testing for the intent of this line or if it >>> doesn't actually have to appear in the output, depending on the >>> system. Perhaps the original author could chime in and provide >>> further guidance as to the intention of the test. >>> >>> I am happy to modify the patch as necessary. >>> >>> Regards, >>> >>> Daniel Stewart >>> >>> [1] - http://cr.openjdk.java.net/~dstewart/8196521/webrev.00/ >>> >>> [2] - https://bugs.openjdk.java.net/browse/JDK-8196521 >>> From philip.race at oracle.com Sat Feb 3 13:47:11 2018 From: philip.race at oracle.com (Philip Race) Date: Sat, 03 Feb 2018 05:47:11 -0800 Subject: [OpenJDK 2D-Dev] Fwd: [PATCH] Fix compiler warnings with newer GCC 7.2.1 In-Reply-To: <4c5248f3-e78d-31ba-c892-fd7df1d027af@oracle.com> References: <4c5248f3-e78d-31ba-c892-fd7df1d027af@oracle.com> Message-ID: <5A75BD5F.5080309@oracle.com> jpeg + harfbuzz are imported libraries. There's no point in a source fix in JDK'scopy as it will get overwritten when we upgrade. No way we are tracking such patches to reapply them. If warnings there are an issue either update the makefiles or use disable-warnings-as-errors ... and perhaps report the issue upstream .. -phil. On 2/3/18, 1:35 AM, Alan Bateman wrote: > > This was sent to core-libs-dev, I assume it was meant for 2d-dev and > serviceability-dev. > > -------- Forwarded Message -------- > Subject: [PATCH] Fix compiler warnings with newer GCC 7.2.1 > Date: Fri, 2 Feb 2018 16:12:08 -0800 > From: Indu Bhagat > To: core-libs-dev at openjdk.java.net > > > > This patch fixes a few of the compiler warnings when using gcc 7.2.1. > > In general there are many other warnings (a majority of which are implicit fallthrough warnings in jdk.crypto.ec package's ecp_*.c files and java.desktop package's splashscreen_gfx_impl.h file). > I can post them if there is interest. > Most warnings, however, look benign (modulo some warnings about unintialized access that I cannot comment on; they need that the code be thoroughly understood). > libfdlibm misleading indentation warnings are already knownhttp://mail.openjdk.java.net/pipermail/jdk9-dev/2016-June/004494.html > > Thanks > > ---------------------------------------------- > The patch fixes the following warnings : > > 1./jdk/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c: In function ?mask_s2u?: > /jdk/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c:389:22: warning: left shift of negative value [-Wshift-negative-value] > *mask = htonl(-1<< (32 - m)); > ^ > 2./jdk/src/java.desktop/share/native/libjavajpeg/jdphuff.c: In function ?decode_mcu_AC_refine?: > /jdk/src/java.desktop/share/native/libjavajpeg/jdphuff.c:505:17: warning: left shift of negative value [-Wshift-negative-value] > int m1 = (-1)<< cinfo->Al; /* -1 in the bit position being coded*/ > ^~ > 3./jdk/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-ot-shape-fallback.cc: In function ?void position_mark(const hb_ot_shape_plan_t*, hb_font_t*, hb_buffer_t*, hb_glyph_extents_t&, unsigned int, unsigned int)?: > /jdk/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-ot-shape-fallback.cc:223:14: warning: this statement may fall through [-Wimplicit-fallthrough=] > } else if (buffer->props.direction == HB_DIRECTION_RTL) { > ^~ > /jdk/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-ot-shape-fallback.cc:229:5: note: here > default: > ^~~~~~~ > /jdk/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-ot-shape-fallback.cc:261:27: warning: this statement may fall through [-Wimplicit-fallthrough=] > base_extents.height -= y_gap; > ~~~~~~~~~~~~~~~~~~~~^~~~~~~~ > /jdk/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-ot-shape-fallback.cc:264:5: note: here > case HB_UNICODE_COMBINING_CLASS_ATTACHED_BELOW_LEFT: > ^~~~ > /jdk/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-ot-shape-fallback.cc:282:27: warning: this statement may fall through [-Wimplicit-fallthrough=] > base_extents.height -= y_gap; > ~~~~~~~~~~~~~~~~~~~~^~~~~~~~ > /jdk/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-ot-shape-fallback.cc:285:5: note: here > case HB_UNICODE_COMBINING_CLASS_ATTACHED_ABOVE: > ^~~~ > ----------- > > diff --git a/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-private.hh b/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-private.hh > --- a/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-private.hh > +++ b/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-private.hh > @@ -158,6 +158,9 @@ > */ > # include > # define HB_FALLTHROUGH __fallthrough > +#elif __GNUC__>= 7 > + /* GNU fallthrough attribute is available from GCC7 */ > +# define HB_FALLTHROUGH __attribute__((fallthrough)) > #else > # define HB_FALLTHROUGH /* FALLTHROUGH */ > #endif > diff --git a/src/java.desktop/share/native/libjavajpeg/jdphuff.c b/src/java.desktop/share/native/libjavajpeg/jdphuff.c > --- a/src/java.desktop/share/native/libjavajpeg/jdphuff.c > +++ b/src/java.desktop/share/native/libjavajpeg/jdphuff.c > @@ -501,8 +501,10 @@ > { > phuff_entropy_ptr entropy = (phuff_entropy_ptr) cinfo->entropy; > int Se = cinfo->Se; > - int p1 = 1<< cinfo->Al; /* 1 in the bit position being coded */ > - int m1 = (-1)<< cinfo->Al; /* -1 in the bit position being coded */ > + /* 1 in the bit position being coded */ > + int p1 = 1<< cinfo->Al; > + /* -1 in the bit position being coded */ > + int m1 = (int)((unsigned)(~0)<< cinfo->Al); > register int s, k, r; > unsigned int EOBRUN; > JBLOCKROW block; > diff --git a/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c b/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c > --- a/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c > +++ b/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c > @@ -386,7 +386,7 @@ > return instr; > } > > - *mask = htonl(-1<< (32 - m)); > + *mask = htonl((uint32_t)(~0)<< (32 - m)); > return s; > } > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jini.george at oracle.com Mon Feb 5 03:55:30 2018 From: jini.george at oracle.com (Jini George) Date: Mon, 5 Feb 2018 09:25:30 +0530 Subject: RFR: 8196361: JTReg failure in serviceability/sa/ClhsdbInspect.java In-Reply-To: <4dc3cedf0b1e40b7bceff6672cb3fa1a@NASANEXM01E.na.qualcomm.com> References: <6e3441f3c28f4f7387d2174f52283fa7@NASANEXM01E.na.qualcomm.com> <44a71e46-3da2-53c6-7e6b-82658183ae8c@oracle.com> <3318aed7-f04a-3b92-2660-39cdf13c2d24@oracle.com> <24c556061ffb4fde9e87a8806c04c8f7@NASANEXM01E.na.qualcomm.com> <0eca7345-fb57-43e4-6169-c4b3531250e4@oracle.com> <4f46f527c17f4d988e4b46e14f93cd4d@NASANEXM01E.na.qualcomm.com> <1b753efa-6abc-4947-e4f3-f6a29020c082@oracle.com> <4dc3cedf0b1e40b7bceff6672cb3fa1a@NASANEXM01E.na.qualcomm.com> Message-ID: Your changes look good, Daniel. I can sponsor the changes. Thank you, Jini. On 2/2/2018 7:28 PM, stewartd.qdt wrote: > Hi Jini, > > Thank you for the review. I have made the requested changes and posted them at http://cr.openjdk.java.net/~dstewart/8196361/webrev.03/ > > Please have a look and review the changes. > > Thanks, > Daniel > > > -----Original Message----- > From: Jini George [mailto:jini.george at oracle.com] > Sent: Friday, February 2, 2018 1:19 AM > To: David Holmes ; stewartd.qdt > Cc: serviceability-dev ; hotspot-dev at openjdk.java.net > Subject: Re: RFR: 8196361: JTReg failure in serviceability/sa/ClhsdbInspect.java > > Hi Daniel, > > Your changes look good to me overall. Just some nits: > > * Please do add 2018 to the copyright year. > * Since the rest of the file follows 4 spaces for indentation, please keep the indentation to 4 spaces. > * Line 81: It would be great if the opening brace is at line 80, so that it would be consistent with the rest of the file. > * Line 65: The declaration could be a part of line 79. > * Line 51: Please add the 'oop address of a java.lang.Class' to the comment. > > Thanks! > Jini. > > > On 2/2/2018 7:31 AM, David Holmes wrote: >> On 2/02/2018 1:50 AM, stewartd.qdt wrote: >>> Please have? a look at the newest changes at: >>> http://cr.openjdk.java.net/~dstewart/8196361/webrev.02/ >>> >>> The only difference between this and the last changeset is the use of >>> "\\R" instead of whatever is the platform line.separator. >> >> Thanks for that. >> >> The overall changes seem reasonable but I'll defer to Jini for final >> approval. If Jini approves then consider this Reviewed. >> >> Thanks, >> David >> >>> Thank you, >>> Daniel >>> >>> -----Original Message----- >>> From: David Holmes [mailto:david.holmes at oracle.com] >>> Sent: Thursday, February 1, 2018 2:51 AM >>> To: stewartd.qdt ; Jini George >>> >>> Cc: serviceability-dev ; >>> hotspot-dev at openjdk.java.net >>> Subject: Re: RFR: 8196361: JTReg failure in >>> serviceability/sa/ClhsdbInspect.java >>> >>> Hi Daniel, >>> >>> On 1/02/2018 2:45 AM, stewartd.qdt wrote: >>>> Hi Jini, David, >>>> >>>> Please have a look at the revised webrev: >>>> http://cr.openjdk.java.net/~dstewart/8196361/webrev.01/ >>>> >>>> In this webrev I have changed the approach to finding the addresses. >>>> This was necessary because in the case of matching for the locks the >>>> addresses are before what is matched and in the case of Method the >>>> address is after it.? The existing code only looked for the >>>> addresses after the matched string. I've also tried to align what >>>> tokens? are being looked for in the lock case. I've taken an >>>> approach of breaking the jstack output into lines and then searching >>>> each line for it containing what we want. Once found, the line is >>>> broken into pieces to find the actual address we want. >>>> >>>> Please let me know if this is an unacceptable approach or any >>>> changes you would like to see. >>> >>> I'm not clear on the overall approach as I'm unclear exactly how >>> inspect operates or exactly what the test is trying to verify. One >>> comment on breaking things into lines though: >>> >>> ??? 73???????????? String newline = >>> System.getProperty("line.separator"); >>> ??? 74???????????? String[] lines = jstackOutput.split(newline); >>> >>> As split() takes a regex, I suggest using \R to cover all potential >>> line-breaks, rather than the platform specific line-seperator. We've >>> been recently bitten by the distinction between output that comes >>> from reading a process's stdout/stderr (and for which a newline \n is >>> translated into the platform line-seperator), and output that comes >>> across a socket connection (for which \n is not translated). This >>> could result in failing to parse things correctly on Windows. It's >>> safer/simpler to expect any kind of line-seperator. >>> >>> Thanks, >>> David >>> >>>> Thanks, >>>> Daniel >>>> >>>> >>>> -----Original Message----- >>>> From: Jini George [mailto:jini.george at oracle.com] >>>> Sent: Tuesday, January 30, 2018 6:58 AM >>>> To: David Holmes ; stewartd.qdt >>>> >>>> Cc: serviceability-dev ; >>>> hotspot-dev at openjdk.java.net >>>> Subject: Re: RFR: 8196361: JTReg failure in >>>> serviceability/sa/ClhsdbInspect.java >>>> >>>> Hi Daniel, David, >>>> >>>> Thanks, Daniel, for bringing this up. The intent of the test is to >>>> get the oop address corresponding to a >>>> java.lang.ref.ReferenceQueue$Lock, >>>> which can typically be obtained from the stack traces of the >>>> Common-Cleaner or the Finalizer threads. The stack traces which I >>>> had been noticing were typically of the form: >>>> >>>> >>>> "Common-Cleaner" #8 daemon prio=8 tid=0x00007f09c82ac000 nid=0xf6e >>>> in >>>> Object.wait() [0x00007f09a18d2000] >>>> ????? java.lang.Thread.State: TIMED_WAITING (on object monitor) >>>> ????? JavaThread state: _thread_blocked >>>> ??? - java.lang.Object.wait(long) @bci=0, pc=0x00007f09b7d6480b, >>>> Method*=0x00007f09acc43d60 (Interpreted frame) >>>> ?????????? - waiting on <0x000000072e61f6e0> (a >>>> java.lang.ref.ReferenceQueue$Lock) >>>> ??? - java.lang.ref.ReferenceQueue.remove(long) @bci=59, line=151, >>>> pc=0x00007f09b7d55243, Method*=0x00007f09acdab9b0 (Interpreted >>>> frame) >>>> ?????????? - waiting to re-lock in wait() <0x000000072e61f6e0> (a >>>> java.lang.ref.ReferenceQueue$Lock) >>>> ... >>>> >>>> I chose 'waiting to re-lock in wait' since that was what I had been >>>> observing next to the oop address of java.lang.ref.ReferenceQueue$Lock. >>>> But I see how with a timing difference, one could get 'waiting to lock' >>>> as in your case. So, a good way to fix might be to check for the >>>> line containing '(a java.lang.ref.ReferenceQueue$Lock)', getting the >>>> oop address from that line (should be the address appearing >>>> immediately before '(a java.lang.ref.ReferenceQueue$Lock)') and >>>> passing that to the 'inspect' command. >>>> >>>> Thanks much, >>>> Jini. >>>> >>>> On 1/30/2018 3:35 AM, David Holmes wrote: >>>>> Hi Daniel, >>>>> >>>>> Serviceability issues should go to >>>>> serviceability-dev at openjdk.java.net >>>>> - now cc'd. >>>>> >>>>> On 30/01/2018 7:53 AM, stewartd.qdt wrote: >>>>>> Please review this webrev [1] which attempts to fix a test error >>>>>> in serviceability/sa/ClhsdbInspect.java when it is run under an >>>>>> AArch64 system (not necessarily exclusive to this system, but it >>>>>> was the system under test). The bug report [2] provides further details. >>>>>> Essentially the line "waiting to re-lock in wait" never actually >>>>>> occurs. Instead I have the line "waiting to lock" which occurs for >>>>>> the referenced item of /java/lang/ref/ReferenceQueue$Lock. >>>>>> Unfortunately the test is written such that only the first >>>>>> "waiting to lock" >>>>>> occurrence is seen (for java/lang/Class), which is already >>>>>> accounted for in the test. >>>>> >>>>> I can't tell exactly what the test expects, or why, but it would be >>>>> extremely hard to arrange for "waiting to re-lock in wait" to be >>>>> seen for the ReferenceQueue lock! That requires acquiring the lock >>>>> yourself, issuing a notify() to unblock the wait(), and then >>>>> issuing the jstack command while still holding the lock! >>>>> >>>>> David >>>>> ----- >>>>> >>>>>> I'm not overly happy with this approach as it actually removes a >>>>>> test line. However, the test line does not actually appear in the >>>>>> output (at least on my system) and the test is not currently >>>>>> written to look for the second occurrence of the line "waiting to lock". >>>>>> Perhaps the original author could chime in and provide further >>>>>> guidance as to the intention of the test. >>>>>> >>>>>> I am happy to modify the patch as necessary. >>>>>> >>>>>> Regards, >>>>>> Daniel Stewart >>>>>> >>>>>> >>>>>> [1] -? http://cr.openjdk.java.net/~dstewart/8196361/webrev.00/ >>>>>> [2] - https://bugs.openjdk.java.net/browse/JDK-8196361 >>>>>> From stewartd.qdt at qualcommdatacenter.com Mon Feb 5 13:59:14 2018 From: stewartd.qdt at qualcommdatacenter.com (stewartd.qdt) Date: Mon, 5 Feb 2018 13:59:14 +0000 Subject: RFR: 8196521: serviceability/sa/ClhsdbWhere.java fails to find method 'waitForReferencePendingList' in output In-Reply-To: <95afbdd5-f9d4-45bd-b2db-95e68790f030@default> References: <71d86c90cba444c6967883078e5c5cc3@NASANEXM01E.na.qualcomm.com> <4158546c-434b-acd1-c0b1-5e2c7faf680c@oracle.com> <6ab1e27d-cf2a-9397-a970-232fb1ddf523@oracle.com> <369287e9-a6ab-4d40-a3f3-f81086406bc7@default> <1afda3cf56994421a9919b9e0834492d@NASANEXM01E.na.qualcomm.com> <95afbdd5-f9d4-45bd-b2db-95e68790f030@default> Message-ID: <96e9b3632bab47bda82c26f85f087aee@NASANEXM01E.na.qualcomm.com> Thanks for sponsoring, Sharath. Daniel -----Original Message----- From: Sharath Ballal [mailto:sharath.ballal at oracle.com] Sent: Saturday, February 3, 2018 4:16 AM To: David Holmes ; stewartd.qdt ; Jini Susan George ; serviceability-dev Subject: RE: RFR: 8196521: serviceability/sa/ClhsdbWhere.java fails to find method 'waitForReferencePendingList' in output Looks OK Daniel. I will sponsor the change (and run this change through our internal testing). Thanks, Sharath (not a Reviewer) -----Original Message----- From: David Holmes Sent: Saturday, February 03, 2018 2:28 PM To: stewartd.qdt; Sharath Ballal; Jini Susan George; serviceability-dev Subject: Re: RFR: 8196521: serviceability/sa/ClhsdbWhere.java fails to find method 'waitForReferencePendingList' in output Hi Daniel, Sharath, The signal thread won't be found if the test happens to be run with -Xrs, but that seems unlikely. So seems okay. Your sponsor will need to test on all platforms of course. Thanks, David On 3/02/2018 12:11 AM, stewartd.qdt wrote: > Hi Sharath, > > Thank you for your comments and insight. I have made the requested > changes and the new webrev is at: > http://cr.openjdk.java.net/~dstewart/8196521/webrev.01/ > > Please have a look and let me know if there are any other changes required. This does pass testing on my system. > > Thanks, > Daniel > > -----Original Message----- > From: Sharath Ballal [mailto:sharath.ballal at oracle.com] > Sent: Friday, February 2, 2018 4:49 AM > To: David Holmes ; Jini Susan George > ; stewartd.qdt > ; serviceability-dev > > Subject: RE: RFR: 8196521: serviceability/sa/ClhsdbWhere.java fails to > find method 'waitForReferencePendingList' in output > > Agree with David. It looks like the stack will be different on some machines occasionally depending on the load, machine speed etc. > Hence the following lines in the test > "java.lang.ref.Reference", > "private static void processPendingReferences", > "private static native void waitForReferencePendingList", can be replaced by > Java Stack Trace for Signal Dispatcher > Java Stack Trace for Reference Handler That atlease ensures we are > looking for all the threads. > > Thanks, > Sharath (not a Reviewer) > > > -----Original Message----- > From: David Holmes > Sent: Thursday, February 01, 2018 1:29 PM > To: Jini George; stewartd.qdt; serviceability-dev > Subject: Re: RFR: 8196521: serviceability/sa/ClhsdbWhere.java fails to > find method 'waitForReferencePendingList' in output > > On 1/02/2018 5:01 PM, Jini George wrote: >> Hello Daniel, >> >> Your fix looks good to me. You probably could instead add 'remove()' >> or 'run()' to the list, but I leave it upto you. > > These tests should only be looking for things that are guaranteed to be seen. If waitForReferencePendingList is not there, how do we know that processPendingReferences will be there? We seem to be assuming a quiescent system. We know the threads that will be present, but the only stack entries we should be looking for are those controlled by the test code itself. > > David > ----- > >> Thanks, >> Jini (Not a (R)eviewer). >> >> On 2/1/2018 2:39 AM, stewartd.qdt wrote: >>> Please review this webrev [1] which attempts to fix a test error in >>> serviceability/sa/ClhsdbWhere.java when it is run under an AArch64 >>> system (not necessarily exclusive to this system, but it was the >>> system under test). The bug report [2] provides further details and >>> has the jtr report that was generated. Essentially the line >>> "waitForReferencePendingList" never actually occurs. The test is >>> written such that it is expecting that line. >>> >>> This patch simply removes the line from the set of tested lines it >>> expects. ?I'm not overly happy with this approach as it actually >>> removes a test line. However, the test line does not actually appear >>> in the output (at least on my system) and I'm not sure if there is >>> actually another way of testing for the intent of this line or if it >>> doesn't actually have to appear in the output, depending on the >>> system. Perhaps the original author could chime in and provide >>> further guidance as to the intention of the test. >>> >>> I am happy to modify the patch as necessary. >>> >>> Regards, >>> >>> Daniel Stewart >>> >>> [1] - http://cr.openjdk.java.net/~dstewart/8196521/webrev.00/ >>> >>> [2] - https://bugs.openjdk.java.net/browse/JDK-8196521 >>> From stewartd.qdt at qualcommdatacenter.com Mon Feb 5 13:58:38 2018 From: stewartd.qdt at qualcommdatacenter.com (stewartd.qdt) Date: Mon, 5 Feb 2018 13:58:38 +0000 Subject: RFR: 8196521: serviceability/sa/ClhsdbWhere.java fails to find method 'waitForReferencePendingList' in output In-Reply-To: References: <71d86c90cba444c6967883078e5c5cc3@NASANEXM01E.na.qualcomm.com> <4158546c-434b-acd1-c0b1-5e2c7faf680c@oracle.com> <6ab1e27d-cf2a-9397-a970-232fb1ddf523@oracle.com> <369287e9-a6ab-4d40-a3f3-f81086406bc7@default> <1afda3cf56994421a9919b9e0834492d@NASANEXM01E.na.qualcomm.com> Message-ID: <301fb2af217040278efdce30124c92dd@NASANEXM01E.na.qualcomm.com> Thanks David. -----Original Message----- From: David Holmes [mailto:david.holmes at oracle.com] Sent: Saturday, February 3, 2018 3:58 AM To: stewartd.qdt ; Sharath Ballal ; Jini Susan George ; serviceability-dev Subject: Re: RFR: 8196521: serviceability/sa/ClhsdbWhere.java fails to find method 'waitForReferencePendingList' in output Hi Daniel, Sharath, The signal thread won't be found if the test happens to be run with -Xrs, but that seems unlikely. So seems okay. Your sponsor will need to test on all platforms of course. Thanks, David On 3/02/2018 12:11 AM, stewartd.qdt wrote: > Hi Sharath, > > Thank you for your comments and insight. I have made the requested > changes and the new webrev is at: > http://cr.openjdk.java.net/~dstewart/8196521/webrev.01/ > > Please have a look and let me know if there are any other changes required. This does pass testing on my system. > > Thanks, > Daniel > > -----Original Message----- > From: Sharath Ballal [mailto:sharath.ballal at oracle.com] > Sent: Friday, February 2, 2018 4:49 AM > To: David Holmes ; Jini Susan George > ; stewartd.qdt > ; serviceability-dev > > Subject: RE: RFR: 8196521: serviceability/sa/ClhsdbWhere.java fails to > find method 'waitForReferencePendingList' in output > > Agree with David. It looks like the stack will be different on some machines occasionally depending on the load, machine speed etc. > Hence the following lines in the test > "java.lang.ref.Reference", > "private static void processPendingReferences", > "private static native void waitForReferencePendingList", can be replaced by > Java Stack Trace for Signal Dispatcher > Java Stack Trace for Reference Handler That atlease ensures we are > looking for all the threads. > > Thanks, > Sharath (not a Reviewer) > > > -----Original Message----- > From: David Holmes > Sent: Thursday, February 01, 2018 1:29 PM > To: Jini George; stewartd.qdt; serviceability-dev > Subject: Re: RFR: 8196521: serviceability/sa/ClhsdbWhere.java fails to > find method 'waitForReferencePendingList' in output > > On 1/02/2018 5:01 PM, Jini George wrote: >> Hello Daniel, >> >> Your fix looks good to me. You probably could instead add 'remove()' >> or 'run()' to the list, but I leave it upto you. > > These tests should only be looking for things that are guaranteed to be seen. If waitForReferencePendingList is not there, how do we know that processPendingReferences will be there? We seem to be assuming a quiescent system. We know the threads that will be present, but the only stack entries we should be looking for are those controlled by the test code itself. > > David > ----- > >> Thanks, >> Jini (Not a (R)eviewer). >> >> On 2/1/2018 2:39 AM, stewartd.qdt wrote: >>> Please review this webrev [1] which attempts to fix a test error in >>> serviceability/sa/ClhsdbWhere.java when it is run under an AArch64 >>> system (not necessarily exclusive to this system, but it was the >>> system under test). The bug report [2] provides further details and >>> has the jtr report that was generated. Essentially the line >>> "waitForReferencePendingList" never actually occurs. The test is >>> written such that it is expecting that line. >>> >>> This patch simply removes the line from the set of tested lines it >>> expects. ?I'm not overly happy with this approach as it actually >>> removes a test line. However, the test line does not actually appear >>> in the output (at least on my system) and I'm not sure if there is >>> actually another way of testing for the intent of this line or if it >>> doesn't actually have to appear in the output, depending on the >>> system. Perhaps the original author could chime in and provide >>> further guidance as to the intention of the test. >>> >>> I am happy to modify the patch as necessary. >>> >>> Regards, >>> >>> Daniel Stewart >>> >>> [1] - http://cr.openjdk.java.net/~dstewart/8196521/webrev.00/ >>> >>> [2] - https://bugs.openjdk.java.net/browse/JDK-8196521 >>> From stewartd.qdt at qualcommdatacenter.com Mon Feb 5 13:59:57 2018 From: stewartd.qdt at qualcommdatacenter.com (stewartd.qdt) Date: Mon, 5 Feb 2018 13:59:57 +0000 Subject: RFR: 8196361: JTReg failure in serviceability/sa/ClhsdbInspect.java In-Reply-To: References: <6e3441f3c28f4f7387d2174f52283fa7@NASANEXM01E.na.qualcomm.com> <44a71e46-3da2-53c6-7e6b-82658183ae8c@oracle.com> <3318aed7-f04a-3b92-2660-39cdf13c2d24@oracle.com> <24c556061ffb4fde9e87a8806c04c8f7@NASANEXM01E.na.qualcomm.com> <0eca7345-fb57-43e4-6169-c4b3531250e4@oracle.com> <4f46f527c17f4d988e4b46e14f93cd4d@NASANEXM01E.na.qualcomm.com> <1b753efa-6abc-4947-e4f3-f6a29020c082@oracle.com> <4dc3cedf0b1e40b7bceff6672cb3fa1a@NASANEXM01E.na.qualcomm.com> Message-ID: <16636f3867fb42f6ae26b1b8c2602591@NASANEXM01E.na.qualcomm.com> Thanks Jini. Regards, Daniel -----Original Message----- From: Jini George [mailto:jini.george at oracle.com] Sent: Sunday, February 4, 2018 10:56 PM To: stewartd.qdt ; David Holmes Cc: serviceability-dev ; hotspot-dev at openjdk.java.net Subject: Re: RFR: 8196361: JTReg failure in serviceability/sa/ClhsdbInspect.java Your changes look good, Daniel. I can sponsor the changes. Thank you, Jini. On 2/2/2018 7:28 PM, stewartd.qdt wrote: > Hi Jini, > > Thank you for the review. I have made the requested changes and posted > them at http://cr.openjdk.java.net/~dstewart/8196361/webrev.03/ > > Please have a look and review the changes. > > Thanks, > Daniel > > > -----Original Message----- > From: Jini George [mailto:jini.george at oracle.com] > Sent: Friday, February 2, 2018 1:19 AM > To: David Holmes ; stewartd.qdt > > Cc: serviceability-dev ; > hotspot-dev at openjdk.java.net > Subject: Re: RFR: 8196361: JTReg failure in > serviceability/sa/ClhsdbInspect.java > > Hi Daniel, > > Your changes look good to me overall. Just some nits: > > * Please do add 2018 to the copyright year. > * Since the rest of the file follows 4 spaces for indentation, please keep the indentation to 4 spaces. > * Line 81: It would be great if the opening brace is at line 80, so that it would be consistent with the rest of the file. > * Line 65: The declaration could be a part of line 79. > * Line 51: Please add the 'oop address of a java.lang.Class' to the comment. > > Thanks! > Jini. > > > On 2/2/2018 7:31 AM, David Holmes wrote: >> On 2/02/2018 1:50 AM, stewartd.qdt wrote: >>> Please have? a look at the newest changes at: >>> http://cr.openjdk.java.net/~dstewart/8196361/webrev.02/ >>> >>> The only difference between this and the last changeset is the use >>> of "\\R" instead of whatever is the platform line.separator. >> >> Thanks for that. >> >> The overall changes seem reasonable but I'll defer to Jini for final >> approval. If Jini approves then consider this Reviewed. >> >> Thanks, >> David >> >>> Thank you, >>> Daniel >>> >>> -----Original Message----- >>> From: David Holmes [mailto:david.holmes at oracle.com] >>> Sent: Thursday, February 1, 2018 2:51 AM >>> To: stewartd.qdt ; Jini George >>> >>> Cc: serviceability-dev ; >>> hotspot-dev at openjdk.java.net >>> Subject: Re: RFR: 8196361: JTReg failure in >>> serviceability/sa/ClhsdbInspect.java >>> >>> Hi Daniel, >>> >>> On 1/02/2018 2:45 AM, stewartd.qdt wrote: >>>> Hi Jini, David, >>>> >>>> Please have a look at the revised webrev: >>>> http://cr.openjdk.java.net/~dstewart/8196361/webrev.01/ >>>> >>>> In this webrev I have changed the approach to finding the addresses. >>>> This was necessary because in the case of matching for the locks >>>> the addresses are before what is matched and in the case of Method >>>> the address is after it.? The existing code only looked for the >>>> addresses after the matched string. I've also tried to align what >>>> tokens? are being looked for in the lock case. I've taken an >>>> approach of breaking the jstack output into lines and then >>>> searching each line for it containing what we want. Once found, the >>>> line is broken into pieces to find the actual address we want. >>>> >>>> Please let me know if this is an unacceptable approach or any >>>> changes you would like to see. >>> >>> I'm not clear on the overall approach as I'm unclear exactly how >>> inspect operates or exactly what the test is trying to verify. One >>> comment on breaking things into lines though: >>> >>> ??? 73???????????? String newline = >>> System.getProperty("line.separator"); >>> ??? 74???????????? String[] lines = jstackOutput.split(newline); >>> >>> As split() takes a regex, I suggest using \R to cover all potential >>> line-breaks, rather than the platform specific line-seperator. We've >>> been recently bitten by the distinction between output that comes >>> from reading a process's stdout/stderr (and for which a newline \n >>> is translated into the platform line-seperator), and output that >>> comes across a socket connection (for which \n is not translated). >>> This could result in failing to parse things correctly on Windows. >>> It's safer/simpler to expect any kind of line-seperator. >>> >>> Thanks, >>> David >>> >>>> Thanks, >>>> Daniel >>>> >>>> >>>> -----Original Message----- >>>> From: Jini George [mailto:jini.george at oracle.com] >>>> Sent: Tuesday, January 30, 2018 6:58 AM >>>> To: David Holmes ; stewartd.qdt >>>> >>>> Cc: serviceability-dev ; >>>> hotspot-dev at openjdk.java.net >>>> Subject: Re: RFR: 8196361: JTReg failure in >>>> serviceability/sa/ClhsdbInspect.java >>>> >>>> Hi Daniel, David, >>>> >>>> Thanks, Daniel, for bringing this up. The intent of the test is to >>>> get the oop address corresponding to a >>>> java.lang.ref.ReferenceQueue$Lock, >>>> which can typically be obtained from the stack traces of the >>>> Common-Cleaner or the Finalizer threads. The stack traces which I >>>> had been noticing were typically of the form: >>>> >>>> >>>> "Common-Cleaner" #8 daemon prio=8 tid=0x00007f09c82ac000 nid=0xf6e >>>> in >>>> Object.wait() [0x00007f09a18d2000] >>>> ????? java.lang.Thread.State: TIMED_WAITING (on object monitor) >>>> ????? JavaThread state: _thread_blocked >>>> ??? - java.lang.Object.wait(long) @bci=0, pc=0x00007f09b7d6480b, >>>> Method*=0x00007f09acc43d60 (Interpreted frame) >>>> ?????????? - waiting on <0x000000072e61f6e0> (a >>>> java.lang.ref.ReferenceQueue$Lock) >>>> ??? - java.lang.ref.ReferenceQueue.remove(long) @bci=59, line=151, >>>> pc=0x00007f09b7d55243, Method*=0x00007f09acdab9b0 (Interpreted >>>> frame) >>>> ?????????? - waiting to re-lock in wait() <0x000000072e61f6e0> (a >>>> java.lang.ref.ReferenceQueue$Lock) >>>> ... >>>> >>>> I chose 'waiting to re-lock in wait' since that was what I had been >>>> observing next to the oop address of java.lang.ref.ReferenceQueue$Lock. >>>> But I see how with a timing difference, one could get 'waiting to lock' >>>> as in your case. So, a good way to fix might be to check for the >>>> line containing '(a java.lang.ref.ReferenceQueue$Lock)', getting >>>> the oop address from that line (should be the address appearing >>>> immediately before '(a java.lang.ref.ReferenceQueue$Lock)') and >>>> passing that to the 'inspect' command. >>>> >>>> Thanks much, >>>> Jini. >>>> >>>> On 1/30/2018 3:35 AM, David Holmes wrote: >>>>> Hi Daniel, >>>>> >>>>> Serviceability issues should go to >>>>> serviceability-dev at openjdk.java.net >>>>> - now cc'd. >>>>> >>>>> On 30/01/2018 7:53 AM, stewartd.qdt wrote: >>>>>> Please review this webrev [1] which attempts to fix a test error >>>>>> in serviceability/sa/ClhsdbInspect.java when it is run under an >>>>>> AArch64 system (not necessarily exclusive to this system, but it >>>>>> was the system under test). The bug report [2] provides further details. >>>>>> Essentially the line "waiting to re-lock in wait" never actually >>>>>> occurs. Instead I have the line "waiting to lock" which occurs >>>>>> for the referenced item of /java/lang/ref/ReferenceQueue$Lock. >>>>>> Unfortunately the test is written such that only the first >>>>>> "waiting to lock" >>>>>> occurrence is seen (for java/lang/Class), which is already >>>>>> accounted for in the test. >>>>> >>>>> I can't tell exactly what the test expects, or why, but it would >>>>> be extremely hard to arrange for "waiting to re-lock in wait" to >>>>> be seen for the ReferenceQueue lock! That requires acquiring the >>>>> lock yourself, issuing a notify() to unblock the wait(), and then >>>>> issuing the jstack command while still holding the lock! >>>>> >>>>> David >>>>> ----- >>>>> >>>>>> I'm not overly happy with this approach as it actually removes a >>>>>> test line. However, the test line does not actually appear in the >>>>>> output (at least on my system) and the test is not currently >>>>>> written to look for the second occurrence of the line "waiting to lock". >>>>>> Perhaps the original author could chime in and provide further >>>>>> guidance as to the intention of the test. >>>>>> >>>>>> I am happy to modify the patch as necessary. >>>>>> >>>>>> Regards, >>>>>> Daniel Stewart >>>>>> >>>>>> >>>>>> [1] -? http://cr.openjdk.java.net/~dstewart/8196361/webrev.00/ >>>>>> [2] - https://bugs.openjdk.java.net/browse/JDK-8196361 >>>>>> From stefan.johansson at oracle.com Mon Feb 5 15:33:00 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Mon, 5 Feb 2018 16:33:00 +0100 Subject: PING: RFR: JDK-8153333: [REDO] STW phases at Concurrent GC should count in PerfCounter In-Reply-To: <89a1a161-874d-4743-e52b-7ab202fd8976@gmail.com> References: <2f4e0901-1602-6276-e7fd-84e168e7b317@gmail.com> <3210723e-c5f9-4277-a97a-be61e10e6b3d@oracle.com> <89a1a161-874d-4743-e52b-7ab202fd8976@gmail.com> Message-ID: On 2018-02-03 06:40, Yasumasa Suenaga wrote: > On 2018/02/02 23:38, Stefan Johansson wrote: >> Hi Yasumasa, >> >> The changes doesn't apply clean on the latest jdk/hs, can you provide >> an updated webrev? > > I uploaded webrev for jdk-hs: > ? cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.05/ > Thanks, I've kicked off a testing job now to verify nothing unexpected fails. > >> The testing done by the submit repo doesn't cover the tests you have >> update so I plan to take the change for a spin and make sure the >> correct tests are run and verified in Mach 5. > > I've also tested hotspot/jtreg/:hotspot_serviceability and > jdk/:jdk_tools on my laptop. > I did not see any errors / failures which are related to this change. I also ran some local tests on this and it looks good. > > >> Also a question about the change. Why do we need a special flag for >> CMS? I see that the original bug report refers to the flag as being a >> way to turn on and off the feature but the current implementation >> only consider the flag for CMS. > > http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-March/016774.html > > > Originally, STW phases (Remark and Cleanup) at G1 are not counted in > jstat FGC column. > So I think we need not to control the behavior of PerfCounter for G1. > I agree, for G1 this should not be controlled. Maybe I was a bit unclear, I was wondering why we want to control it for CMS. I think either we should change the behavior without guarding it by a flag or just skip updating CMS (and leave the pauses in FGC). If we do the change for CMS, we should probably also do a CSR, but that should be fairly straight forward. I also found the old review thread where Jon M had the same comment (removing the flag) and it looks like all agreed on that: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-March/017118.html Thanks, Stefan > > Thanks, > > Yasumasa > > >> Thanks, >> Stefan >> >> On 2018-02-01 14:58, Yasumasa Suenaga wrote: >>> PING: Could you review and sponsor it? >>> >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>> >>> This change has been passed Mach 5 via submit repo: >>> http://java.se.oracle.com:10065/mdash/jobs/mach5-one-ysuenaga-JDK-8153333-20180201-0805-10101 >>> >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2017/11/01 22:02, Yasumasa Suenaga wrote: >>>> PING: Could you review and sponsor it? >>>> >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>>> >>>> Also I need JPRT results of this change. >>>> Could you cooperate? >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2017/09/27 0:08, Yasumasa Suenaga wrote: >>>>> Hi all, >>>>> >>>>> I uploaded new webrev to be adapted to jdk10/hs: >>>>> >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>>>> >>>>> I want to check this patch via JPRT, but I cannot access it. >>>>> Could you cooperate? >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> yasumasa >>>>> >>>>> >>>>> On 2017/09/21 7:46, Yasumasa Suenaga wrote: >>>>>> PING: >>>>>> >>>>>> Have you checked this issue? >>>>>> >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/hotspot/ >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/jdk/ >>>>>> >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2017/07/01 23:44, Yasumasa Suenaga wrote: >>>>>>> PING: >>>>>>> >>>>>>> Have you checked this issue? >>>>>>> >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2017/06/14 13:22, Yasumasa Suenaga wrote: >>>>>>>> Hi all, >>>>>>>> >>>>>>>> I changed PerfCounter to show CGC STW phase in jstat in >>>>>>>> JDK-8151674. >>>>>>>> However, it occurred several jtreg test failure, so it was >>>>>>>> back-outed. >>>>>>>> >>>>>>>> I want to resume to work for this issue. >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/hotspot/ >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/jdk/ >>>>>>>> >>>>>>>> These changes are work fine on jtreg test as below: >>>>>>>> >>>>>>>> ?? hotspot/test/serviceability/tmtools/jstat >>>>>>>> ?? jdk/test/sun/tools >>>>>>>> >>>>>>>> >>>>>>>> Since JDK 9, default GC algorithm is set to G1. >>>>>>>> So I think this change is useful to watch GC behavior through >>>>>>>> jstat. >>>>>>>> >>>>>>>> I cannot access JPRT. Could you help? >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >> From egor.ushakov at jetbrains.com Mon Feb 5 15:44:32 2018 From: egor.ushakov at jetbrains.com (Egor Ushakov) Date: Mon, 5 Feb 2018 18:44:32 +0300 Subject: Issues debugging java 9 from jdk 8 Message-ID: <69179b71-90c0-d921-dd8c-3b5800ce8f6d@jetbrains.com> Hi all, in IDEA we faced an issue that when debugging java 9 process memory view does not work: https://youtrack.jetbrains.com/issue/JRE-641 It seems that there's a bug in how VirtualMachineImpl.canGetInstanceInfo checks vm version (it does not pass jdk 9 where minor is 0): if(versionInfo().jdwpMajor <1|| versionInfo().jdwpMinor <6){ returnfalse; } I've found this fixed in jdk 9 inside the fix: http://hg.openjdk.java.net/jdk9/hs/rev/f900d5afd9c8 8142968: Module System implementation Summary: Initial integration of JEP 200, JEP 260, JEP 261, and JEP 282 alanb 17-Mar-16 22:04 We've applied the part of the fix into our jdk 8 fork: https://github.com/JetBrains/jdk8u_jdk/commit/6424e2786e8adc4e012e0b7bd0cfc78ba1ab34dd It seems reasonable to backport at least this part into openjdk 8? What do you think? Maybe some other parts deserve backporting as well. I've attached the patch just in case. -- Egor Ushakov Software Developer JetBrains http://www.jetbrains.com The Drive to Develop -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- Index: src/share/classes/com/sun/tools/jdi/VirtualMachineImpl.java IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- src/share/classes/com/sun/tools/jdi/VirtualMachineImpl.java (revision d93beeae90a6c98a14982dffa4652235c3321307) +++ src/share/classes/com/sun/tools/jdi/VirtualMachineImpl.java (revision 6424e2786e8adc4e012e0b7bd0cfc78ba1ab34dd) @@ -674,20 +674,18 @@ versionInfo().jdwpMinor >= 6; } public boolean canGetInstanceInfo() { - if (versionInfo().jdwpMajor < 1 || - versionInfo().jdwpMinor < 6) { - return false; - } - validateVM(); - return hasNewCapabilities() && - capabilitiesNew().canGetInstanceInfo; - } - public boolean canUseSourceNameFilters() { - if (versionInfo().jdwpMajor < 1 || - versionInfo().jdwpMinor < 6) { + if (versionInfo().jdwpMajor > 1 || + versionInfo().jdwpMinor >= 6) { + validateVM(); + return hasNewCapabilities() && + capabilitiesNew().canGetInstanceInfo; + } else { return false; } - return true; + } + public boolean canUseSourceNameFilters() { + return versionInfo().jdwpMajor > 1 || + versionInfo().jdwpMinor >= 6; } public boolean canForceEarlyReturn() { validateVM(); @@ -703,12 +701,8 @@ capabilitiesNew().canGetSourceDebugExtension; } public boolean canGetClassFileVersion() { - if ( versionInfo().jdwpMajor < 1 && - versionInfo().jdwpMinor < 6) { - return false; - } else { - return true; - } + return versionInfo().jdwpMajor > 1 || + versionInfo().jdwpMinor >= 6; } public boolean canGetConstantPool() { validateVM(); From gary.adams at oracle.com Mon Feb 5 19:15:10 2018 From: gary.adams at oracle.com (Gary Adams) Date: Mon, 05 Feb 2018 14:15:10 -0500 Subject: JDK-8080990: libdt_socket/socket_md.c(202) : warning C4996: 'gethostbyname': Use getaddrinfo() or GetAddrInfoW() In-Reply-To: <5A731457.1040502@oracle.com> References: <5A6A2611.4030003@oracle.com> <5A731457.1040502@oracle.com> Message-ID: <5A78AD3E.9090503@oracle.com> One more to fix to cover the remaining test failures I was seeing. Previously, using inet_addr, there was a single -1 return that needed to be checked. Now that we're using inet_pton, there is a -1 and a 0 error return to be considered. My preference would be to leave the dbgSysInetAddr name unchanged and have the low level check for inet_pton errors to simply return the -1 that was previously checked. This specific issue is about the deprecation of library functions on windows. I'd recommend not upgrading the other platforms at this time until a complete update is done for ipv6 support. I'll post a new webrev when I've retested using the updated inet_pton error checking. ... > Hi Gary, > > > >/ Here's a revised webrev > />/ > />/ http://cr.openjdk.java.net/~gadams/8080990/webrev.01/index.html > />/ > />/ Still testing ... > />/ > />/ Using shutdown() fixed problems reported by the > />/ java/nio/channelSocketChannel tests. > / > The fix looks good. I would think we should rename dbgsysInetAddr to dbgsysPToN and use inet_pton also for the Unix/Linux platforms. It would be the better choice there as well, though we still only support IPv4 in libdt_socket. > > >/ > />/ I also noticed prior use of getaddrinfo for "localhost" was not calling > />/ freeaddrinfo. > / > Thanks for spotting/fixing that. > > Best regards > Christoph > -------------- next part -------------- An HTML attachment was scrubbed... URL: From parvathi.somashekar at oracle.com Mon Feb 5 19:17:58 2018 From: parvathi.somashekar at oracle.com (Paru Somashekar) Date: Mon, 05 Feb 2018 11:17:58 -0800 Subject: RFR 8196308: Update AllLineLocations & ClassesByName to use TestScaffold instead of JDIScaffold Message-ID: <5A78ADE6.7030000@oracle.com> Hi, Please review the fix for JDK-8196308 Bug : https://bugs.openjdk.java.net/browse/JDK-8196308 Webrev : http://cr.openjdk.java.net/~psomashe/8196308/webrev/ The updated tests ran successfully with Mach5. thanks, Paru. From alexey.menkov at oracle.com Mon Feb 5 20:10:22 2018 From: alexey.menkov at oracle.com (Alex Menkov) Date: Mon, 5 Feb 2018 12:10:22 -0800 Subject: 8161605 : The '!UseSharedSpaces' check is not need in JvmtiManageCapabilities::recompute_always_capabilities In-Reply-To: <5A74B97D.5090901@oracle.com> References: <1c8e3f0e-8aa1-574d-f24f-b78b0dc3e6bd@oracle.com> <5A74B97D.5090901@oracle.com> Message-ID: Thank you for the feedback. Updated fix: http://cr.openjdk.java.net/~amenkov/can_generate_class_hook/webrev.03/ --alex On 02/02/2018 11:18, Mikhailo Seledtsov wrote: > Hi Alex, > > ? Several test-related comments: > ? - CanGenerateAllClassHook.java > ???? " 44? * So the test runs 2 java process": s/process/process*es*/ > > ? - CanGenerateAllClassHook.java > ???? A general comment - please consider using "TestCommon" class and > "CDSTestUtils" where possible. > ???? There are useful methods for forming ProcessBuilder with > parameters specific to AppCDS, and also for checking results. > ???? In some cases the test case is so custom that use of these > utilities is not applicable. > ???? Please take a look at ClassFileLoadHookTest.java as an example. > It uses TestCommon.executeAndLog(), > ???? TestCommon.checkExec() and more. Other good examples are > SharedArchiveFile.java, HelloExtTest.java or other tests under > ???? appcds directory. > > ???? Important to note a condition of "mapping failure". When using > CDS/AppCDS > ???? with Xshare:on, the mapping of a CDS archive to memory may fail at > random (due to ASLR or other conditions). > ???? TestCommon.checkExec() handles such conditions propertly, > otherwise you will have random/intermittent test failures. > > ? - CanGenerateAllClassHook.java > > 122 private static boolean isWin() { > > ???? I recommend using jdk.test.lib.Platform, the isWindows() method. > The SQE guideline is to use test utilities where possible. > ???? FYI, the test utilities reside in: open/test/lib/jdk/test/lib > > > The rest of the tests look good, > > Thank you, > Misha > > > On 2/1/18, 4:11 PM, Alex Menkov wrote: >> Hi Serguei, >> >> Thanks for the feedback. >> >> Updated fix: >> http://cr.openjdk.java.net/~amenkov/can_generate_class_hook/webrev.02/ >> >> --alex >> >> On 02/01/2018 11:00, serguei.spitsyn at oracle.com wrote: >>> One more attempt to send it with the correct to-list... >>> Sorry for the noise. >>> >>> >>> Hi Alex, >>> >>> It looks good in general. >>> A couple of comments on the test. >>> >>> We had a convention to avoid having test sub-folders with bug numbers. >>> Could you, please, rename it to some symbolic name? >>> >>> Also we need the test to fail in the native agent when an error is >>> reported with the reportError(). >>> >>> It'd be great if some CDS/AppCDS experts from the Runtime team look >>> at the fix and test >>> so I've included Jiangli and Misha into the to-list. >>> >>> Thanks, >>> Serguei >>> >>> >>> On 1/25/18 16:11, Alex Menkov wrote: >>>> Hi all, >>>> >>>> Please take a look at a fix for >>>> JDK-8161605 : The '!UseSharedSpaces' check is not need in >>>> JvmtiManageCapabilities::recompute_always_capabilities >>>> >>>> The fix moves can_generate_all_class_hook_events capability from >>>> "onload_capabilities" to "always_capabilities" and cleans up >>>> recompute_always_capabilities method. >>>> >>>> jira: https://bugs.openjdk.java.net/browse/JDK-8161605 >>>> webrev: >>>> http://cr.openjdk.java.net/~amenkov/can_generate_class_hook/webrev/ >>>> >>>> --alex >>> From serguei.spitsyn at oracle.com Mon Feb 5 20:29:45 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 5 Feb 2018 12:29:45 -0800 Subject: RFR 8196308: Update AllLineLocations & ClassesByName to use TestScaffold instead of JDIScaffold In-Reply-To: <5A78ADE6.7030000@oracle.com> References: <5A78ADE6.7030000@oracle.com> Message-ID: <23ffb297-fd74-7302-286a-b807330ee091@oracle.com> Hi Paru, 30 * @run build JDIScaffold VMConnection 31 * @run build JDIScaffold VMConnection The JDIScaffold has to be replaced with the TestScaffold in both files. Otherwise, it looks good to me. Thanks, Serguei On 2/5/18 11:17, Paru Somashekar wrote: > Hi, > > Please review the fix for JDK-8196308 > > Bug : https://bugs.openjdk.java.net/browse/JDK-8196308 > Webrev : http://cr.openjdk.java.net/~psomashe/8196308/webrev/ > > > The updated tests ran successfully with Mach5. > > thanks, > Paru. From serguei.spitsyn at oracle.com Mon Feb 5 21:22:50 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 5 Feb 2018 13:22:50 -0800 Subject: 8161605 : The '!UseSharedSpaces' check is not need in JvmtiManageCapabilities::recompute_always_capabilities In-Reply-To: References: <1c8e3f0e-8aa1-574d-f24f-b78b0dc3e6bd@oracle.com> <5A74B97D.5090901@oracle.com> Message-ID: <0608d261-2551-a1a5-ade3-d784aeadcdbe@oracle.com> Hi Alex, Thank you for the update! It looks good to me. Thanks, Serguei On 2/5/18 12:10, Alex Menkov wrote: > Thank you for the feedback. > > Updated fix: > http://cr.openjdk.java.net/~amenkov/can_generate_class_hook/webrev.03/ > > --alex > > On 02/02/2018 11:18, Mikhailo Seledtsov wrote: >> Hi Alex, >> >> ?? Several test-related comments: >> ?? - CanGenerateAllClassHook.java >> ????? " 44? * So the test runs 2 java process": s/process/process*es*/ >> >> ?? - CanGenerateAllClassHook.java >> ????? A general comment - please consider using "TestCommon" class >> and "CDSTestUtils" where possible. >> ????? There are useful methods for forming ProcessBuilder with >> parameters specific to AppCDS, and also for checking results. >> ????? In some cases the test case is so custom that use of these >> utilities is not applicable. >> ????? Please take a look at ClassFileLoadHookTest.java as an >> example.? It uses TestCommon.executeAndLog(), >> ????? TestCommon.checkExec() and more. Other good examples are >> SharedArchiveFile.java, HelloExtTest.java or other tests under >> ????? appcds directory. >> >> ????? Important to note a condition of "mapping failure". When using >> CDS/AppCDS >> ????? with Xshare:on, the mapping of a CDS archive to memory may fail >> at random (due to ASLR or other conditions). >> ????? TestCommon.checkExec() handles such conditions propertly, >> otherwise you will have random/intermittent test failures. >> >> ?? - CanGenerateAllClassHook.java >> >> ???? 122???? private static boolean isWin() { >> >> ????? I recommend using jdk.test.lib.Platform, the isWindows() >> method. The SQE guideline is to use test utilities where possible. >> ????? FYI, the test utilities reside in: open/test/lib/jdk/test/lib >> >> >> The rest of the tests look good, >> >> Thank you, >> Misha >> >> >> On 2/1/18, 4:11 PM, Alex Menkov wrote: >>> Hi Serguei, >>> >>> Thanks for the feedback. >>> >>> Updated fix: >>> http://cr.openjdk.java.net/~amenkov/can_generate_class_hook/webrev.02/ >>> >>> --alex >>> >>> On 02/01/2018 11:00, serguei.spitsyn at oracle.com wrote: >>>> One more attempt to send it with the correct to-list... >>>> Sorry for the noise. >>>> >>>> >>>> Hi Alex, >>>> >>>> It looks good in general. >>>> A couple of comments on the test. >>>> >>>> We had a convention to avoid having test sub-folders with bug numbers. >>>> Could you, please, rename it to some symbolic name? >>>> >>>> Also we need the test to fail in the native agent when an error is >>>> reported with the reportError(). >>>> >>>> It'd be great if some CDS/AppCDS experts from the Runtime team look >>>> at the fix and test >>>> so I've included Jiangli and Misha into the to-list. >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 1/25/18 16:11, Alex Menkov wrote: >>>>> Hi all, >>>>> >>>>> Please take a look at a fix for >>>>> JDK-8161605 : The '!UseSharedSpaces' check is not need in >>>>> JvmtiManageCapabilities::recompute_always_capabilities >>>>> >>>>> The fix moves can_generate_all_class_hook_events capability from >>>>> "onload_capabilities" to "always_capabilities" and cleans up >>>>> recompute_always_capabilities method. >>>>> >>>>> jira: https://bugs.openjdk.java.net/browse/JDK-8161605 >>>>> webrev: >>>>> http://cr.openjdk.java.net/~amenkov/can_generate_class_hook/webrev/ >>>>> >>>>> --alex >>>> From coleen.phillimore at oracle.com Mon Feb 5 21:37:06 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 5 Feb 2018 16:37:06 -0500 Subject: RFR (XS) 8174734: Safepoint sync time did not increase Message-ID: <0ee1d208-f805-72ca-2480-f53efa513435@oracle.com> Summary: allow safepoint time to be zero in the test See bug for more details. open webrev at http://cr.openjdk.java.net/~coleenp/8174734.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8174734 Thanks, Coleen From parvathi.somashekar at oracle.com Mon Feb 5 22:07:24 2018 From: parvathi.somashekar at oracle.com (Paru Somashekar) Date: Mon, 05 Feb 2018 14:07:24 -0800 Subject: RFR 8196308: Update AllLineLocations & ClassesByName to use TestScaffold instead of JDIScaffold In-Reply-To: <23ffb297-fd74-7302-286a-b807330ee091@oracle.com> References: <5A78ADE6.7030000@oracle.com> <23ffb297-fd74-7302-286a-b807330ee091@oracle.com> Message-ID: <5A78D59C.2070803@oracle.com> Thanks Serguei, I have updated the review. thanks, Paru. On 2/5/18, 12:29 PM, serguei.spitsyn at oracle.com wrote: > Hi Paru, > > 30 * @run build JDIScaffold VMConnection > 31 * @run build JDIScaffold VMConnection > > The JDIScaffold has to be replaced with the TestScaffold in both files. > > Otherwise, it looks good to me. > > Thanks, > Serguei > > > On 2/5/18 11:17, Paru Somashekar wrote: >> Hi, >> >> Please review the fix for JDK-8196308 >> >> Bug : https://bugs.openjdk.java.net/browse/JDK-8196308 >> Webrev : http://cr.openjdk.java.net/~psomashe/8196308/webrev/ >> >> >> The updated tests ran successfully with Mach5. >> >> thanks, >> Paru. > From mikhailo.seledtsov at oracle.com Mon Feb 5 22:12:26 2018 From: mikhailo.seledtsov at oracle.com (Mikhailo Seledtsov) Date: Mon, 05 Feb 2018 14:12:26 -0800 Subject: 8161605 : The '!UseSharedSpaces' check is not need in JvmtiManageCapabilities::recompute_always_capabilities In-Reply-To: References: <1c8e3f0e-8aa1-574d-f24f-b78b0dc3e6bd@oracle.com> <5A74B97D.5090901@oracle.com> Message-ID: <5A78D6CA.80407@oracle.com> Looks good to me, Thank you, Misha On 2/5/18, 12:10 PM, Alex Menkov wrote: > Thank you for the feedback. > > Updated fix: > http://cr.openjdk.java.net/~amenkov/can_generate_class_hook/webrev.03/ > > --alex > > On 02/02/2018 11:18, Mikhailo Seledtsov wrote: >> Hi Alex, >> >> Several test-related comments: >> - CanGenerateAllClassHook.java >> " 44 * So the test runs 2 java process": s/process/process*es*/ >> >> - CanGenerateAllClassHook.java >> A general comment - please consider using "TestCommon" class >> and "CDSTestUtils" where possible. >> There are useful methods for forming ProcessBuilder with >> parameters specific to AppCDS, and also for checking results. >> In some cases the test case is so custom that use of these >> utilities is not applicable. >> Please take a look at ClassFileLoadHookTest.java as an >> example. It uses TestCommon.executeAndLog(), >> TestCommon.checkExec() and more. Other good examples are >> SharedArchiveFile.java, HelloExtTest.java or other tests under >> appcds directory. >> >> Important to note a condition of "mapping failure". When using >> CDS/AppCDS >> with Xshare:on, the mapping of a CDS archive to memory may fail >> at random (due to ASLR or other conditions). >> TestCommon.checkExec() handles such conditions propertly, >> otherwise you will have random/intermittent test failures. >> >> - CanGenerateAllClassHook.java >> >> 122 private static boolean isWin() { >> >> I recommend using jdk.test.lib.Platform, the isWindows() >> method. The SQE guideline is to use test utilities where possible. >> FYI, the test utilities reside in: open/test/lib/jdk/test/lib >> >> >> The rest of the tests look good, >> >> Thank you, >> Misha >> >> >> On 2/1/18, 4:11 PM, Alex Menkov wrote: >>> Hi Serguei, >>> >>> Thanks for the feedback. >>> >>> Updated fix: >>> http://cr.openjdk.java.net/~amenkov/can_generate_class_hook/webrev.02/ >>> >>> --alex >>> >>> On 02/01/2018 11:00, serguei.spitsyn at oracle.com wrote: >>>> One more attempt to send it with the correct to-list... >>>> Sorry for the noise. >>>> >>>> >>>> Hi Alex, >>>> >>>> It looks good in general. >>>> A couple of comments on the test. >>>> >>>> We had a convention to avoid having test sub-folders with bug numbers. >>>> Could you, please, rename it to some symbolic name? >>>> >>>> Also we need the test to fail in the native agent when an error is >>>> reported with the reportError(). >>>> >>>> It'd be great if some CDS/AppCDS experts from the Runtime team look >>>> at the fix and test >>>> so I've included Jiangli and Misha into the to-list. >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 1/25/18 16:11, Alex Menkov wrote: >>>>> Hi all, >>>>> >>>>> Please take a look at a fix for >>>>> JDK-8161605 : The '!UseSharedSpaces' check is not need in >>>>> JvmtiManageCapabilities::recompute_always_capabilities >>>>> >>>>> The fix moves can_generate_all_class_hook_events capability from >>>>> "onload_capabilities" to "always_capabilities" and cleans up >>>>> recompute_always_capabilities method. >>>>> >>>>> jira: https://bugs.openjdk.java.net/browse/JDK-8161605 >>>>> webrev: >>>>> http://cr.openjdk.java.net/~amenkov/can_generate_class_hook/webrev/ >>>>> >>>>> --alex >>>> From serguei.spitsyn at oracle.com Mon Feb 5 22:33:24 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 5 Feb 2018 14:33:24 -0800 Subject: RFR 8196308: Update AllLineLocations & ClassesByName to use TestScaffold instead of JDIScaffold In-Reply-To: <5A78D59C.2070803@oracle.com> References: <5A78ADE6.7030000@oracle.com> <23ffb297-fd74-7302-286a-b807330ee091@oracle.com> <5A78D59C.2070803@oracle.com> Message-ID: Looks good. Thank you for the update! Thanks, Serguei On 2/5/18 14:07, Paru Somashekar wrote: > Thanks Serguei, I have updated the review. > > thanks, > Paru. > > On 2/5/18, 12:29 PM, serguei.spitsyn at oracle.com wrote: >> Hi Paru, >> >> ? 30? * @run build JDIScaffold VMConnection >> ? 31? *? @run build JDIScaffold VMConnection >> >> The JDIScaffold? has to be replaced with the TestScaffold in both files. >> >> Otherwise, it looks good to me. >> >> Thanks, >> Serguei >> >> >> On 2/5/18 11:17, Paru Somashekar wrote: >>> Hi, >>> >>> Please review the fix for JDK-8196308 >>> >>> Bug : https://bugs.openjdk.java.net/browse/JDK-8196308 >>> Webrev : http://cr.openjdk.java.net/~psomashe/8196308/webrev/ >>> >>> >>> The updated tests ran successfully with Mach5. >>> >>> thanks, >>> Paru. >> > From jcbeyler at google.com Mon Feb 5 23:04:41 2018 From: jcbeyler at google.com (JC Beyler) Date: Mon, 5 Feb 2018 15:04:41 -0800 Subject: JDK-8171119: Low-Overhead Heap Profiling Message-ID: Hi Erik, (Renaming this to be folded into the newly renamed thread :)) First off, thanks a lot for reviewing the webrev! I appreciate it! I updated the webrev to: http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.05a/ And the incremental one is here: http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.04_05a/ It contains: - The change for since from 9 to 11 for the jvmti.xml - The use of the OrderAccess for initialized - Clearing the oop I also have inlined my answers to your comments. The biggest question will come from the multiple *_end variables. A bit of the logic there is due to handling the slow path refill vs fast path refill and checking that the rug was not pulled underneath the slowpath. I believe that a previous comment was that TlabFastRefill was going to be deprecated. If this is true, we could revert this code a bit and just do a : if TlabFastRefill is enabled, disable this. And then deprecate that when TlabFastRefill is deprecated. This might simplify this webrev and I can work on a follow-up that either: removes TlabFastRefill if Robbin does not have the time to do it or add the support to the assembly side to handle this correctly. What do you think? Now, below, inlined are my answers: On Fri, Feb 2, 2018 at 8:44 AM, Erik ?sterlund wrote: > Hi JC, > > Hope I am reviewing the right version of your work. Here goes... > > src/hotspot/share/gc/shared/collectedHeap.inline.hpp: > > 159 AllocTracer::send_allocation_outside_tlab(klass, result, size * > HeapWordSize, THREAD); > 160 > 161 THREAD->tlab().handle_sample(THREAD, result, size); > 162 return result; > 163 } > > Should not call tlab()->X without checking if (UseTLAB) IMO. > Done! > src/hotspot/share/gc/shared/threadLocalAllocBuffer.cpp: > > So first of all, there seems to quite a few ends. There is an "end", a "hard > end", a "slow path end", and an "actual end". Moreover, it seems like the > "hard end" is actually further away than the "actual end". So the "hard end" > seems like more of a "really definitely actual end" or something. I don't > know about you, but I think it looks kind of messy. In particular, I don't > feel like the name "actual end" reflects what it represents, especially when > there is another end that is behind the "actual end". > > 413 HeapWord* ThreadLocalAllocBuffer::hard_end() { > 414 // Did a fast TLAB refill occur? > 415 if (_slow_path_end != _end) { > 416 // Fix up the actual end to be now the end of this TLAB. > 417 _slow_path_end = _end; > 418 _actual_end = _end; > 419 } > 420 > 421 return _actual_end + alignment_reserve(); > 422 } > > I really do not like making getters unexpectedly have these kind of side > effects. It is not expected that when you ask for the "hard end", you > implicitly update the "slow path end" and "actual end" to new values. > As I said, a lot of this is due to the FastTlabRefill. If I make this not supporting FastTlabRefill, this goes away. The reason the system needs to update itself at the get is that you only know at that get if things have shifted underneath the tlab slow path. I am not sure of really better names (naming is hard!), perhaps we could do these names: - current_tlab_end // Either the allocated tlab end or a sampling point - last_allocation_address // The end of the tlab allocation - last_slowpath_allocated_end // In case a fast refill occurred the end might have changed, this is to remember slow vs fast past refills the hard_end method can be renamed to something like: tlab_end_pointer() // The end of the lab including a bit of alignment reserved bytes Not sure it's better but before updating the webrev, I wanted to try to get input/consensus :) (Note hard_end was always further off than end). > src/hotspot/share/prims/jvmti.xml: > > 10357 > 10358 > 10359 Can sample the heap. > 10360 If this capability is enabled then the heap sampling methods > can be called. > 10361 > 10362 > > Looks like this capability should not be "since 9" if it gets integrated > now. Updated now to 11, crossing my fingers :) > > src/hotspot/share/runtime/heapMonitoring.cpp: > > 448 if (is_alive->do_object_b(value)) { > 449 // Update the oop to point to the new object if it is still > alive. > 450 f->do_oop(&(trace.obj)); > 451 > 452 // Copy the old trace, if it is still live. > 453 _allocated_traces->at_put(curr_pos++, trace); > 454 > 455 // Store the live trace in a cache, to be served up on /heapz. > 456 _traces_on_last_full_gc->append(trace); > 457 > 458 count++; > 459 } else { > 460 // If the old trace is no longer live, add it to the list of > 461 // recently collected garbage. > 462 store_garbage_trace(trace); > 463 } > > In the case where the oop was not live, I would like it to be explicitly > cleared. Done I think how you wanted it. Let me know because I'm not familiar with the RootAccess API. I'm unclear if I'm doing this right or not so reviews of these parts are highly appreciated. Robbin had talked of perhaps later pushing this all into a OopStorage, should I do this now do you think? Or can that wait a second webrev later down the road? + Did I clear it the way you wanted me to or were you thinking of something else? + Final question here, seems like if I were to want to not do the f->do_oop directly on the trace.obj, I'd need to do something like: f->do_oop(&value); ... trace->store_oop(value); to update the oop internally. Is that right/is that one of the advantages of going to the Oopstorage sooner than later? > > Also I see a lot of concurrent-looking use of the following field: > 267 volatile bool _initialized; > > Please note that the "volatile" qualifier does not help with reordering > here. Reordering between volatile and non-volatile fields is completely free > for both compiler and hardware, except for windows with MSVC, where volatile > semantics is defined to use acquire/release semantics, and the hardware is > TSO. But for the general case, I would expect this field to be stored with > OrderAccess::release_store and loaded with OrderAccess::load_acquire. > Otherwise it is not thread safe. Because everything is behind a mutex, I wasn't really worried about this. I have a test that has multiple threads trying to hit this corner case and it passes. However, to be paranoid, I updated it to using the OrderAccess API now, thanks! Let me know what you think there too! > > As a kind of meta comment, I wonder if it would make sense to add sampling > for non-TLAB allocations. Seems like if someone is rapidly allocating a > whole bunch of 1 MB objects that never fit in a TLAB, I might still be > interested in seeing that in my traces, and not get surprised that the > allocation rate is very high yet not showing up in any profiles. > That is handled by the handle_sample where you wanted me to put a UseTlab because you hit that case if the allocation is too big. I double checked by changing the test http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.05a/raw_files/new/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatObjectCorrectnessTest.java to use a smaller Tlab (2048) and made the object bigger and it goes through that and passes. Thanks again for your review and I look forward to your pointers for the questions I now have raised! Jc > Thanks, > /Erik > > > On 2018-01-26 06:45, JC Beyler wrote: >> >> Thanks Robbin for the reviews :) >> >> The new full webrev is here: >> http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.03/ >> The incremental webrev is here: >> http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.02_03/ >> >> I inlined my answers: >> >> On Thu, Jan 25, 2018 at 1:15 AM, Robbin Ehn wrote: >>> >>> Hi JC, great to see another revision! >>> >>> #### >>> heapMonitoring.cpp >>> >>> StackTraceData should not contain the oop for 'safety' reasons. >>> When StackTraceData is moved from _allocated_traces: >>> L452 store_garbage_trace(trace); >>> it contains a dead oop. >>> _allocated_traces could instead be a tupel of oop and StackTraceData thus >>> dead oops are not kept. >> >> Done I used inheritance to make the copier work regardless but the >> idea is the same. >> >>> You should use the new Access API for loading the oop, something like >>> this: >>> RootAccess::load(...) >>> I don't think you need to use Access API for clearing the oop, but it >>> would >>> look nicer. And you shouldn't probably be using: >>> Universe::heap()->is_in_reserved(value) >> >> I am unfamiliar with this but I think I did do it like you wanted me >> to (all tests pass so that's a start). I'm not sure how to clear the >> oop exactly, is there somewhere that does that, which I can use to do >> the same? >> >> I removed the is_in_reserved, this came from our internal version, I >> don't know why it was there but my tests work without so I removed it >> :) >> >> >>> The lock: >>> L424 MutexLocker mu(HeapMonitorStorage_lock); >>> Is not needed as far as I can see. >>> weak_oops_do is called in a safepoint, no TLAB allocation can happen and >>> JVMTI thread can't access these data-structures. Is there something more >>> to >>> this lock that I'm missing? >> >> Since a thread can call the JVMTI getLiveTraces (or any of the other >> ones), it can get to the point of trying to copying the >> _allocated_traces. I imagine it is possible that this is happening >> during a GC or that it can be started and a GC happens afterwards. >> Therefore, it seems to me that you want this protected, no? >> >> >>> #### >>> You have 6 files without any changes in them (any more): >>> g1CollectedHeap.cpp >>> psMarkSweep.cpp >>> psParallelCompact.cpp >>> genCollectedHeap.cpp >>> referenceProcessor.cpp >>> thread.hpp >>> >> Done. >> >>> #### >>> I have not looked closely, but is it possible to hide heap sampling in >>> AllocTracer ? (with some minor changes to the AllocTracer API) >>> >> I am imagining that you are saying to move the code that does the >> sampling code (change the tlab end, do the call to HeapMonitoring, >> etc.) into the AllocTracer code itself? I think that is right and I'll >> look if that is possible and prepare a webrev to show what would be >> needed to make that happen. >> >>> #### >>> Minor nit, when declaring pointer there is a little mix of having the >>> pointer adjacent by type name and data name. (Most hotspot code is by >>> type >>> name) >>> E.g. >>> heapMonitoring.cpp:711 jvmtiStackTrace *trace = .... >>> heapMonitoring.cpp:733 Method* m = vfst.method(); >>> (not just this file) >>> >> Done! >> >>> #### >>> HeapMonitorThreadOnOffTest.java:77 >>> I would make g_tmp volatile, otherwise the assignment in loop may >>> theoretical be skipped. >>> >> Also done! >> >> Thanks again! >> Jc > > From david.holmes at oracle.com Tue Feb 6 04:40:13 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 6 Feb 2018 14:40:13 +1000 Subject: JDK-8171119: Low-Overhead Heap Profiling In-Reply-To: References: Message-ID: <72db2c82-e118-965b-f6b8-f4cd338f5904@oracle.com> Hi Jc, I've just been browsing this and have a few comments/queries src/hotspot/share/prims/jvmtiHeapTransition.hpp In HeapThreadTransition what are the possible entry states? The primary state is presumably _in_native, but what else is expected? You can't transition from arbitrary states to _in_vm Also in that file the include guard has the wrong name: #ifndef SHARE_VM_PRIMS_JVMTIHEAPSAMPLING_HPP etc. --- src/hotspot/share/runtime/heapMonitoring.cpp Can you change StackTraceData from a struct to a class please. StackTraceData::free_data could still be an instance method even if not done in the destructor. I also think the caller should do the delete rather than hiding it in here, as this obscures the lifecycle of the instances. 215 void initialize(int max_storage) { 216 MutexLocker mu(HeapMonitorStorage_lock); 217 allocate_storage(max_storage); 218 memset(&_stats, 0, sizeof(_stats)); 219 } You're using a lock so obviously expect multiple threads to try this, but you aren't checking if initialization has already been done. That check is inside allocate_storage, but you're doing the memset unconditionally. You're inconsistent with accessing/modifying _initialized under the HeapMonitorStorage_lock. The destructor calls free_storage with no lock (obviously) held. It's unclear how this lock-free destructor fits into the concurrent usage expected of this class. I see from your reply to Erik that you've now added some orderAccess usage to be "paranoid". But we don't want paranoid, we want correct. If there is expected to be a lock-free, but potentially concurrent path, then you will need to use OrderAccess appropriately. If everything is actually being done under a lock, then you don't need to use OrderAccess. But you need to be clear on exactly how concurrency comes into play with your code. 397 _allocated_traces = new (ResourceObj::C_HEAP, mtInternal) 398 GrowableArray(128, true); These are allocations that will abort the VM if they fail - correct? If so that seems rather user-unfriendly for a profiling tool! 422 void StackTraceStorage::weak_oops_do(BoolObjectClosure *is_alive, 423 OopClosure *f) { 424 MutexLocker mu(HeapMonitorStorage_lock); IIUC oops-do methods get called at safepoints, but you are then taking a Mutex unconditionally. So any other code that acquires the same mutex must be guaranteed not to hit a safepoint check - so NoSafepointVerifier should be used to check that. 575 void StackTraceStorage::store_garbage_trace(const StackTraceData &trace) { 576 StackTraceData *new_trace = new StackTraceData(); 577 *new_trace = trace; This looks quite odd to me. Shouldn't this just be using a copy-constructor? Thanks, David ----- On 27/01/2018 2:01 PM, JC Beyler wrote: > (Change of subject to contain the bug number :)). > > New webrev is here: > Incremental: > http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.03_04/ > > Full webrev: > http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.04/ > > Inlined are my answers again :). > > On Fri, Jan 26, 2018 at 5:51 AM, Robbin Ehn wrote: >> Hi JC, (dropping compiler on mail thread) >> >> On 01/26/2018 06:45 AM, JC Beyler wrote: >>> >>> Thanks Robbin for the reviews :) >>> >>> The new full webrev is here: >>> http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.03/ >>> The incremental webrev is here: >>> http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.02_03/ >> >> >> Thanks! >> >> I got this compile issue: >> /home/rehn/source/jdk/small/open/src/hotspot/share/gc/shared/threadLocalAllocBuffer.hpp: >> In static member function 'static void >> G1ResManTLABCache::put(ThreadLocalAllocBuffer&, AllocationContext_t)': >> /home/rehn/source/jdk/small/open/src/hotspot/share/gc/shared/threadLocalAllocBuffer.hpp:97:13: >> error: 'HeapWord* ThreadLocalAllocBuffer::hard_end()' is private >> HeapWord* hard_end(); >> ^ >> /home/rehn/source/jdk/small/closed/src/hotspot/share/gc/g1/g1ResManTLABCache.cpp:52:50: >> error: within this context >> size_t remaining = pointer_delta(tlab.hard_end(), tlab.top()); > > This is strange but I'm assuming it is because we are not working on > the same repo? > > I used: > hg clone http://hg.openjdk.java.net/jdk/hs jdkhs-heap > > I'll try a new clone on Monday and see. My new version moved hard_end > back to public so it should work now. > > > >> >>> >>> I inlined my answers: >>> >>> On Thu, Jan 25, 2018 at 1:15 AM, Robbin Ehn wrote: >>>> >>>> Hi JC, great to see another revision! >>>> >>>> #### >>>> heapMonitoring.cpp >>>> >>>> StackTraceData should not contain the oop for 'safety' reasons. >>>> When StackTraceData is moved from _allocated_traces: >>>> L452 store_garbage_trace(trace); >>>> it contains a dead oop. >>>> _allocated_traces could instead be a tupel of oop and StackTraceData thus >>>> dead oops are not kept. >>> >>> >>> Done I used inheritance to make the copier work regardless but the >>> idea is the same. >> >> >> Looks good. >> >>> >>>> >>>> You should use the new Access API for loading the oop, something like >>>> this: >>>> RootAccess::load(...) >>>> I don't think you need to use Access API for clearing the oop, but it >>>> would >>>> look nicer. And you shouldn't probably be using: >>>> Universe::heap()->is_in_reserved(value) >>> >>> >>> I am unfamiliar with this but I think I did do it like you wanted me >>> to (all tests pass so that's a start). I'm not sure how to clear the >>> oop exactly, is there somewhere that does that, which I can use to do >>> the same? >>> >>> I removed the is_in_reserved, this came from our internal version, I >>> don't know why it was there but my tests work without so I removed it >>> :) >> >> >> I talked a bit with GC folks about this today. >> So actually the oop should be in the oopStorage and your should have a >> weakhandle to that oop (at least in the near future). >> But this should work for now... >> In the future when we have the oop in oppStorage you will not get called, so >> you will not know when the oops are dead, either GC must trigger a >> concurrent vm operation (e.g. _no_ safepoint operation) so we can clear dead >> oops or do periodic scanning. >> >> It would be good with some input here from Thomas that knows what you are >> doing and knows GC. > > Fair enough, hopefully Thomas will chime in. Are you saying that this > first version could go in and we can work on a refinement? Or are you > saying I should work on this now at the same time and fix it before > this V1 goes in? (Just so I know :)) > >> >>> >>> >>>> >>>> The lock: >>>> L424 MutexLocker mu(HeapMonitorStorage_lock); >>>> Is not needed as far as I can see. >>>> weak_oops_do is called in a safepoint, no TLAB allocation can happen and >>>> JVMTI thread can't access these data-structures. Is there something more >>>> to >>>> this lock that I'm missing? >>> >>> >>> Since a thread can call the JVMTI getLiveTraces (or any of the other >>> ones), it can get to the point of trying to copying the >>> _allocated_traces. I imagine it is possible that this is happening >>> during a GC or that it can be started and a GC happens afterwards. >>> Therefore, it seems to me that you want this protected, no? > > Looks like it yes, I removed it, ran my tests and they work so the new > webrev no longer has that mutex at all. > >> >> >> A thread calling getLiveTraces will be stopped in the HeapThreadTransition. >> A safepoint don't allow any threads to be in_vm or to be in_java during >> safepoint. Only threads in native can execute, but they will be stopped on >> any transition. If a thread is in_vm the safepoint waits to a transition >> (locking a mutex is also transition to blocked). So if weak_oops is called >> you have an guarantee that no threads are executing inside the VM or >> executing Java code. (not counting GC threads of course) >> This also means that the lock can never be contented when weak_oops is >> called, so it's not harmful. >> >>> >>> >>>> >>>> #### >>>> You have 6 files without any changes in them (any more): >>>> g1CollectedHeap.cpp >>>> psMarkSweep.cpp >>>> psParallelCompact.cpp >>>> genCollectedHeap.cpp >>>> referenceProcessor.cpp >>>> thread.hpp >>>> >>> >>> Done. >>> >>>> #### >>>> I have not looked closely, but is it possible to hide heap sampling in >>>> AllocTracer ? (with some minor changes to the AllocTracer API) >>>> >>> >>> I am imagining that you are saying to move the code that does the >>> sampling code (change the tlab end, do the call to HeapMonitoring, >>> etc.) into the AllocTracer code itself? I think that is right and I'll >>> look if that is possible and prepare a webrev to show what would be >>> needed to make that happen. >> >> >> Sounds good. > > I'll look at this on Monday then! > > Thanks for the reply and have a great weekend! > Jc > >> >>> >>>> #### >>>> Minor nit, when declaring pointer there is a little mix of having the >>>> pointer adjacent by type name and data name. (Most hotspot code is by >>>> type >>>> name) >>>> E.g. >>>> heapMonitoring.cpp:711 jvmtiStackTrace *trace = .... >>>> heapMonitoring.cpp:733 Method* m = vfst.method(); >>>> (not just this file) >>>> >>> >>> Done! >>> >>>> #### >>>> HeapMonitorThreadOnOffTest.java:77 >>>> I would make g_tmp volatile, otherwise the assignment in loop may >>>> theoretical be skipped. >>>> >>> >>> Also done! >> >> >> Looks good, thanks! >> >> /Robbin >> >>> >>> Thanks again! >>> Jc >>> >> From yasuenag at gmail.com Tue Feb 6 05:10:05 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Tue, 6 Feb 2018 14:10:05 +0900 Subject: PING: RFR: JDK-8153333: [REDO] STW phases at Concurrent GC should count in PerfCounter In-Reply-To: References: <2f4e0901-1602-6276-e7fd-84e168e7b317@gmail.com> <3210723e-c5f9-4277-a97a-be61e10e6b3d@oracle.com> <89a1a161-874d-4743-e52b-7ab202fd8976@gmail.com> Message-ID: Hi Stefan, > I agree, for G1 this should not be controlled. Maybe I was a bit unclear, I > was wondering why we want to control it for CMS. I said to remove -XX:EnableConcGCPerfCounter in two years ago. I've missed it :-) http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-March/017125.html So I uploaded new webrev. This change includes copyright year updates. http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.06/ This change passes all tests on submit repo, and :hotspot_serviceability :jdk_tools tests on my laptop. http://java.se.oracle.com:10065/mdash/jobs/mach5-one-ysuenaga-JDK-8153333-20180206-0222-10428 > If we do the change for CMS, we should > probably also do a CSR, but that should be fairly straight forward. What should I do to get CSR approve? Thanks, Yasumasa 2018-02-06 0:33 GMT+09:00 Stefan Johansson : > > > On 2018-02-03 06:40, Yasumasa Suenaga wrote: >> >> On 2018/02/02 23:38, Stefan Johansson wrote: >>> >>> Hi Yasumasa, >>> >>> The changes doesn't apply clean on the latest jdk/hs, can you provide an >>> updated webrev? >> >> >> I uploaded webrev for jdk-hs: >> cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.05/ >> > Thanks, I've kicked off a testing job now to verify nothing unexpected > fails. >> >> >>> The testing done by the submit repo doesn't cover the tests you have >>> update so I plan to take the change for a spin and make sure the correct >>> tests are run and verified in Mach 5. >> >> >> I've also tested hotspot/jtreg/:hotspot_serviceability and jdk/:jdk_tools >> on my laptop. >> I did not see any errors / failures which are related to this change. > > I also ran some local tests on this and it looks good. >> >> >> >>> Also a question about the change. Why do we need a special flag for CMS? >>> I see that the original bug report refers to the flag as being a way to turn >>> on and off the feature but the current implementation only consider the flag >>> for CMS. >> >> >> >> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-March/016774.html >> >> Originally, STW phases (Remark and Cleanup) at G1 are not counted in jstat >> FGC column. >> So I think we need not to control the behavior of PerfCounter for G1. >> > I agree, for G1 this should not be controlled. Maybe I was a bit unclear, I > was wondering why we want to control it for CMS. I think either we should > change the behavior without guarding it by a flag or just skip updating CMS > (and leave the pauses in FGC). If we do the change for CMS, we should > probably also do a CSR, but that should be fairly straight forward. > > I also found the old review thread where Jon M had the same comment > (removing the flag) and it looks like all agreed on that: > http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-March/017118.html > > Thanks, > Stefan > > >> >> Thanks, >> >> Yasumasa >> >> >>> Thanks, >>> Stefan >>> >>> On 2018-02-01 14:58, Yasumasa Suenaga wrote: >>>> >>>> PING: Could you review and sponsor it? >>>> >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>>> >>>> >>>> This change has been passed Mach 5 via submit repo: >>>> >>>> http://java.se.oracle.com:10065/mdash/jobs/mach5-one-ysuenaga-JDK-8153333-20180201-0805-10101 >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2017/11/01 22:02, Yasumasa Suenaga wrote: >>>>> >>>>> PING: Could you review and sponsor it? >>>>> >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>>>> >>>>> >>>>> Also I need JPRT results of this change. >>>>> Could you cooperate? >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2017/09/27 0:08, Yasumasa Suenaga wrote: >>>>>> >>>>>> Hi all, >>>>>> >>>>>> I uploaded new webrev to be adapted to jdk10/hs: >>>>>> >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>>>>> >>>>>> I want to check this patch via JPRT, but I cannot access it. >>>>>> Could you cooperate? >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> yasumasa >>>>>> >>>>>> >>>>>> On 2017/09/21 7:46, Yasumasa Suenaga wrote: >>>>>>> >>>>>>> PING: >>>>>>> >>>>>>> Have you checked this issue? >>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/hotspot/ >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/jdk/ >>>>>>> >>>>>>> >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2017/07/01 23:44, Yasumasa Suenaga wrote: >>>>>>>> >>>>>>>> PING: >>>>>>>> >>>>>>>> Have you checked this issue? >>>>>>>> >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2017/06/14 13:22, Yasumasa Suenaga wrote: >>>>>>>>> >>>>>>>>> Hi all, >>>>>>>>> >>>>>>>>> I changed PerfCounter to show CGC STW phase in jstat in >>>>>>>>> JDK-8151674. >>>>>>>>> However, it occurred several jtreg test failure, so it was >>>>>>>>> back-outed. >>>>>>>>> >>>>>>>>> I want to resume to work for this issue. >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/hotspot/ >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/jdk/ >>>>>>>>> >>>>>>>>> These changes are work fine on jtreg test as below: >>>>>>>>> >>>>>>>>> hotspot/test/serviceability/tmtools/jstat >>>>>>>>> jdk/test/sun/tools >>>>>>>>> >>>>>>>>> >>>>>>>>> Since JDK 9, default GC algorithm is set to G1. >>>>>>>>> So I think this change is useful to watch GC behavior through >>>>>>>>> jstat. >>>>>>>>> >>>>>>>>> I cannot access JPRT. Could you help? >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>> > From david.holmes at oracle.com Tue Feb 6 05:13:57 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 6 Feb 2018 15:13:57 +1000 Subject: RFR (XS) 8174734: Safepoint sync time did not increase In-Reply-To: <0ee1d208-f805-72ca-2480-f53efa513435@oracle.com> References: <0ee1d208-f805-72ca-2480-f53efa513435@oracle.com> Message-ID: Hi Coleen, On 6/02/2018 7:37 AM, coleen.phillimore at oracle.com wrote: > Summary: allow safepoint time to be zero in the test > > See bug for more details. > > open webrev at http://cr.openjdk.java.net/~coleenp/8174734.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8174734 I guess I'm still surprised that 300 thread dumps can take less than a millisecond! There's always more than one thread running. I did some basic benchmarking and dumpAllStacks() from main takes at least 150us on the Linux box I tested on. I just can't see 300 dumps taking less than 1ms ... though I can see them taking < 10ms if we're measuring time using a coarse clock - where do these times come from? That aside this change seem unnecessary: // Careful with these values. ! private static final long MIN_VALUE_FOR_PASS = 0; private static final long MAX_VALUE_FOR_PASS = Long.MAX_VALUE; this is for the minimum number of safepoints that need to be seen, which I think should still be 1. By allowing 0 here (and for the elapsed time), the test could actually fail to do anything related to safepoints and still pass - and that seems wrong. Or the safepoint stat code could be completely broken and we'd never notice. Basically the test just wants to check that we get reasonable looking statistics from the MBean Maybe we need to be measuring the time at a higher resolution than milliseconds - though that would be a non-trivial RFE I expect. ? Thanks, David > Thanks, > Coleen From yasuenag at gmail.com Tue Feb 6 05:30:19 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Tue, 6 Feb 2018 14:30:19 +0900 Subject: PING: RFR: JDK-8153333: [REDO] STW phases at Concurrent GC should count in PerfCounter In-Reply-To: References: <2f4e0901-1602-6276-e7fd-84e168e7b317@gmail.com> <3210723e-c5f9-4277-a97a-be61e10e6b3d@oracle.com> <89a1a161-874d-4743-e52b-7ab202fd8976@gmail.com> Message-ID: Hi Stefan, I've got another report from submit repo, and it includes one error on OS X. Can you share it? http://java.se.oracle.com:10065/mdash/jobs/mach5-one-ysuenaga-JDK-8153333-20180206-0355-10435 I've tested this change (webrev.06) on Linux x64. Thanks, Yasumasa 2018-02-06 14:10 GMT+09:00 Yasumasa Suenaga : > Hi Stefan, > >> I agree, for G1 this should not be controlled. Maybe I was a bit unclear, I >> was wondering why we want to control it for CMS. > > I said to remove -XX:EnableConcGCPerfCounter in two years ago. I've > missed it :-) > > http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-March/017125.html > > So I uploaded new webrev. This change includes copyright year updates. > > http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.06/ > > This change passes all tests on submit repo, and > :hotspot_serviceability :jdk_tools tests on my laptop. > > http://java.se.oracle.com:10065/mdash/jobs/mach5-one-ysuenaga-JDK-8153333-20180206-0222-10428 > > >> If we do the change for CMS, we should >> probably also do a CSR, but that should be fairly straight forward. > > What should I do to get CSR approve? > > > Thanks, > > Yasumasa > > > 2018-02-06 0:33 GMT+09:00 Stefan Johansson : >> >> >> On 2018-02-03 06:40, Yasumasa Suenaga wrote: >>> >>> On 2018/02/02 23:38, Stefan Johansson wrote: >>>> >>>> Hi Yasumasa, >>>> >>>> The changes doesn't apply clean on the latest jdk/hs, can you provide an >>>> updated webrev? >>> >>> >>> I uploaded webrev for jdk-hs: >>> cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.05/ >>> >> Thanks, I've kicked off a testing job now to verify nothing unexpected >> fails. >>> >>> >>>> The testing done by the submit repo doesn't cover the tests you have >>>> update so I plan to take the change for a spin and make sure the correct >>>> tests are run and verified in Mach 5. >>> >>> >>> I've also tested hotspot/jtreg/:hotspot_serviceability and jdk/:jdk_tools >>> on my laptop. >>> I did not see any errors / failures which are related to this change. >> >> I also ran some local tests on this and it looks good. >>> >>> >>> >>>> Also a question about the change. Why do we need a special flag for CMS? >>>> I see that the original bug report refers to the flag as being a way to turn >>>> on and off the feature but the current implementation only consider the flag >>>> for CMS. >>> >>> >>> >>> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-March/016774.html >>> >>> Originally, STW phases (Remark and Cleanup) at G1 are not counted in jstat >>> FGC column. >>> So I think we need not to control the behavior of PerfCounter for G1. >>> >> I agree, for G1 this should not be controlled. Maybe I was a bit unclear, I >> was wondering why we want to control it for CMS. I think either we should >> change the behavior without guarding it by a flag or just skip updating CMS >> (and leave the pauses in FGC). If we do the change for CMS, we should >> probably also do a CSR, but that should be fairly straight forward. >> >> I also found the old review thread where Jon M had the same comment >> (removing the flag) and it looks like all agreed on that: >> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-March/017118.html >> >> Thanks, >> Stefan >> >> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>>> Thanks, >>>> Stefan >>>> >>>> On 2018-02-01 14:58, Yasumasa Suenaga wrote: >>>>> >>>>> PING: Could you review and sponsor it? >>>>> >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>>>> >>>>> >>>>> This change has been passed Mach 5 via submit repo: >>>>> >>>>> http://java.se.oracle.com:10065/mdash/jobs/mach5-one-ysuenaga-JDK-8153333-20180201-0805-10101 >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2017/11/01 22:02, Yasumasa Suenaga wrote: >>>>>> >>>>>> PING: Could you review and sponsor it? >>>>>> >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>>>>> >>>>>> >>>>>> Also I need JPRT results of this change. >>>>>> Could you cooperate? >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2017/09/27 0:08, Yasumasa Suenaga wrote: >>>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> I uploaded new webrev to be adapted to jdk10/hs: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>>>>>> >>>>>>> I want to check this patch via JPRT, but I cannot access it. >>>>>>> Could you cooperate? >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> yasumasa >>>>>>> >>>>>>> >>>>>>> On 2017/09/21 7:46, Yasumasa Suenaga wrote: >>>>>>>> >>>>>>>> PING: >>>>>>>> >>>>>>>> Have you checked this issue? >>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/hotspot/ >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/jdk/ >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2017/07/01 23:44, Yasumasa Suenaga wrote: >>>>>>>>> >>>>>>>>> PING: >>>>>>>>> >>>>>>>>> Have you checked this issue? >>>>>>>>> >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2017/06/14 13:22, Yasumasa Suenaga wrote: >>>>>>>>>> >>>>>>>>>> Hi all, >>>>>>>>>> >>>>>>>>>> I changed PerfCounter to show CGC STW phase in jstat in >>>>>>>>>> JDK-8151674. >>>>>>>>>> However, it occurred several jtreg test failure, so it was >>>>>>>>>> back-outed. >>>>>>>>>> >>>>>>>>>> I want to resume to work for this issue. >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/hotspot/ >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/jdk/ >>>>>>>>>> >>>>>>>>>> These changes are work fine on jtreg test as below: >>>>>>>>>> >>>>>>>>>> hotspot/test/serviceability/tmtools/jstat >>>>>>>>>> jdk/test/sun/tools >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Since JDK 9, default GC algorithm is set to G1. >>>>>>>>>> So I think this change is useful to watch GC behavior through >>>>>>>>>> jstat. >>>>>>>>>> >>>>>>>>>> I cannot access JPRT. Could you help? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>> >> From shafi.s.ahmad at oracle.com Tue Feb 6 05:56:25 2018 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Mon, 5 Feb 2018 21:56:25 -0800 (PST) Subject: [8u] RFR for 'JDK-8195088: [TEST_BUG] StartManagementAgent got unexpected exception' In-Reply-To: <35e118d1-5f59-4947-bad0-4f5fbc032474@default> References: <35e118d1-5f59-4947-bad0-4f5fbc032474@default> Message-ID: <5a466d6e-a899-45bf-982a-dc35e06c61a4@default> Hi, Could someone please review it. Regards, Shafi > -----Original Message----- > From: Shafi Ahmad > Sent: Monday, January 29, 2018 6:02 PM > To: serviceability-dev at openjdk.java.net > Subject: [8u] RFR for 'JDK-8195088: [TEST_BUG] StartManagementAgent got > unexpected exception' > > Hi, > > Please review the trivial code change for the fix of 'JDK-8195088: [TEST_BUG] > StartManagementAgent got unexpected exception' to jdk8u-dev. > > Summary: > This test case is expected to fail but the expected exception message is > different. > This fix > - if (!ex.getMessage().contains("Invalid > com.sun.management.jmxremote.port number")) { > + if (!ex.getMessage().contains("NumberFormatException: For input > string: \"apa\"")) { > > is checking for the correct message. > > This is fixed in jdk10 under bug JDK-8165736 [Error message should be shown > when JVMTI agent cannot be attache] > http://hg.openjdk.java.net/jdk/hs/rev/bc1cffa26561#l10.1 > > Jdk8 bug: https://bugs.openjdk.java.net/browse/JDK-8195088 > webrev link: http://cr.openjdk.java.net/~shshahma/8195088/webrev.00/ > > Regards, > Shafi From christoph.langer at sap.com Tue Feb 6 07:50:21 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Tue, 6 Feb 2018 07:50:21 +0000 Subject: JDK-8080990: libdt_socket/socket_md.c(202) : warning C4996: 'gethostbyname': Use getaddrinfo() or GetAddrInfoW() In-Reply-To: <5A78AD3E.9090503@oracle.com> References: <5A6A2611.4030003@oracle.com> <5A731457.1040502@oracle.com> <5A78AD3E.9090503@oracle.com> Message-ID: Hi Gary, ok, I would agree that a change like globally moving from dbgSysInetAddr to something like dbgsysPToN should at least be handled by a different change. Best regards Christoph From: Gary Adams [mailto:gary.adams at oracle.com] Sent: Montag, 5. Februar 2018 20:15 To: gary.adams at oracle.com Cc: Langer, Christoph ; OpenJDK Serviceability ; OpenJDK Build ; OpenJDK Networking Subject: Re: JDK-8080990: libdt_socket/socket_md.c(202) : warning C4996: 'gethostbyname': Use getaddrinfo() or GetAddrInfoW() One more to fix to cover the remaining test failures I was seeing. Previously, using inet_addr, there was a single -1 return that needed to be checked. Now that we're using inet_pton, there is a -1 and a 0 error return to be considered. My preference would be to leave the dbgSysInetAddr name unchanged and have the low level check for inet_pton errors to simply return the -1 that was previously checked. This specific issue is about the deprecation of library functions on windows. I'd recommend not upgrading the other platforms at this time until a complete update is done for ipv6 support. I'll post a new webrev when I've retested using the updated inet_pton error checking. ... Hi Gary, > Here's a revised webrev > > http://cr.openjdk.java.net/~gadams/8080990/webrev.01/index.html > > Still testing ... > > Using shutdown() fixed problems reported by the > java/nio/channelSocketChannel tests. The fix looks good. I would think we should rename dbgsysInetAddr to dbgsysPToN and use inet_pton also for the Unix/Linux platforms. It would be the better choice there as well, though we still only support IPv4 in libdt_socket. > > I also noticed prior use of getaddrinfo for "localhost" was not calling > freeaddrinfo. Thanks for spotting/fixing that. Best regards Christoph -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefan.johansson at oracle.com Tue Feb 6 12:44:33 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Tue, 6 Feb 2018 13:44:33 +0100 Subject: PING: RFR: JDK-8153333: [REDO] STW phases at Concurrent GC should count in PerfCounter In-Reply-To: References: <2f4e0901-1602-6276-e7fd-84e168e7b317@gmail.com> <3210723e-c5f9-4277-a97a-be61e10e6b3d@oracle.com> <89a1a161-874d-4743-e52b-7ab202fd8976@gmail.com> Message-ID: Hi, On 2018-02-06 06:30, Yasumasa Suenaga wrote: > Hi Stefan, > > I've got another report from submit repo, and it includes one error on OS X. > Can you share it? > > http://java.se.oracle.com:10065/mdash/jobs/mach5-one-ysuenaga-JDK-8153333-20180206-0355-10435 This is a known issue, Robin W is working on a fix as we speak: https://bugs.openjdk.java.net/browse/JDK-8193308 Stefan > > I've tested this change (webrev.06) on Linux x64. > > > Thanks, > > Yasumasa > > > 2018-02-06 14:10 GMT+09:00 Yasumasa Suenaga : >> Hi Stefan, >> >>> I agree, for G1 this should not be controlled. Maybe I was a bit unclear, I >>> was wondering why we want to control it for CMS. >> I said to remove -XX:EnableConcGCPerfCounter in two years ago. I've >> missed it :-) >> >> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-March/017125.html >> >> So I uploaded new webrev. This change includes copyright year updates. >> >> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.06/ >> >> This change passes all tests on submit repo, and >> :hotspot_serviceability :jdk_tools tests on my laptop. >> >> http://java.se.oracle.com:10065/mdash/jobs/mach5-one-ysuenaga-JDK-8153333-20180206-0222-10428 >> >> >>> If we do the change for CMS, we should >>> probably also do a CSR, but that should be fairly straight forward. >> What should I do to get CSR approve? >> >> >> Thanks, >> >> Yasumasa >> >> >> 2018-02-06 0:33 GMT+09:00 Stefan Johansson : >>> >>> On 2018-02-03 06:40, Yasumasa Suenaga wrote: >>>> On 2018/02/02 23:38, Stefan Johansson wrote: >>>>> Hi Yasumasa, >>>>> >>>>> The changes doesn't apply clean on the latest jdk/hs, can you provide an >>>>> updated webrev? >>>> >>>> I uploaded webrev for jdk-hs: >>>> cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.05/ >>>> >>> Thanks, I've kicked off a testing job now to verify nothing unexpected >>> fails. >>>> >>>>> The testing done by the submit repo doesn't cover the tests you have >>>>> update so I plan to take the change for a spin and make sure the correct >>>>> tests are run and verified in Mach 5. >>>> >>>> I've also tested hotspot/jtreg/:hotspot_serviceability and jdk/:jdk_tools >>>> on my laptop. >>>> I did not see any errors / failures which are related to this change. >>> I also ran some local tests on this and it looks good. >>>> >>>> >>>>> Also a question about the change. Why do we need a special flag for CMS? >>>>> I see that the original bug report refers to the flag as being a way to turn >>>>> on and off the feature but the current implementation only consider the flag >>>>> for CMS. >>>> >>>> >>>> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-March/016774.html >>>> >>>> Originally, STW phases (Remark and Cleanup) at G1 are not counted in jstat >>>> FGC column. >>>> So I think we need not to control the behavior of PerfCounter for G1. >>>> >>> I agree, for G1 this should not be controlled. Maybe I was a bit unclear, I >>> was wondering why we want to control it for CMS. I think either we should >>> change the behavior without guarding it by a flag or just skip updating CMS >>> (and leave the pauses in FGC). If we do the change for CMS, we should >>> probably also do a CSR, but that should be fairly straight forward. >>> >>> I also found the old review thread where Jon M had the same comment >>> (removing the flag) and it looks like all agreed on that: >>> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-March/017118.html >>> >>> Thanks, >>> Stefan >>> >>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>>> Thanks, >>>>> Stefan >>>>> >>>>> On 2018-02-01 14:58, Yasumasa Suenaga wrote: >>>>>> PING: Could you review and sponsor it? >>>>>> >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>>>>> >>>>>> This change has been passed Mach 5 via submit repo: >>>>>> >>>>>> http://java.se.oracle.com:10065/mdash/jobs/mach5-one-ysuenaga-JDK-8153333-20180201-0805-10101 >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2017/11/01 22:02, Yasumasa Suenaga wrote: >>>>>>> PING: Could you review and sponsor it? >>>>>>> >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>>>>>> >>>>>>> Also I need JPRT results of this change. >>>>>>> Could you cooperate? >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2017/09/27 0:08, Yasumasa Suenaga wrote: >>>>>>>> Hi all, >>>>>>>> >>>>>>>> I uploaded new webrev to be adapted to jdk10/hs: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>>>>>>> >>>>>>>> I want to check this patch via JPRT, but I cannot access it. >>>>>>>> Could you cooperate? >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2017/09/21 7:46, Yasumasa Suenaga wrote: >>>>>>>>> PING: >>>>>>>>> >>>>>>>>> Have you checked this issue? >>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/hotspot/ >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/jdk/ >>>>>>>>> >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2017/07/01 23:44, Yasumasa Suenaga wrote: >>>>>>>>>> PING: >>>>>>>>>> >>>>>>>>>> Have you checked this issue? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2017/06/14 13:22, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi all, >>>>>>>>>>> >>>>>>>>>>> I changed PerfCounter to show CGC STW phase in jstat in >>>>>>>>>>> JDK-8151674. >>>>>>>>>>> However, it occurred several jtreg test failure, so it was >>>>>>>>>>> back-outed. >>>>>>>>>>> >>>>>>>>>>> I want to resume to work for this issue. >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/hotspot/ >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/jdk/ >>>>>>>>>>> >>>>>>>>>>> These changes are work fine on jtreg test as below: >>>>>>>>>>> >>>>>>>>>>> hotspot/test/serviceability/tmtools/jstat >>>>>>>>>>> jdk/test/sun/tools >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Since JDK 9, default GC algorithm is set to G1. >>>>>>>>>>> So I think this change is useful to watch GC behavior through >>>>>>>>>>> jstat. >>>>>>>>>>> >>>>>>>>>>> I cannot access JPRT. Could you help? >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> From yasuenag at gmail.com Tue Feb 6 12:52:00 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Tue, 6 Feb 2018 21:52:00 +0900 Subject: PING: RFR: JDK-8153333: [REDO] STW phases at Concurrent GC should count in PerfCounter In-Reply-To: References: <2f4e0901-1602-6276-e7fd-84e168e7b317@gmail.com> <3210723e-c5f9-4277-a97a-be61e10e6b3d@oracle.com> <89a1a161-874d-4743-e52b-7ab202fd8976@gmail.com> Message-ID: <759fc62b-a9e0-c186-2681-1129fc61a6a8@gmail.com> Hi Stefan, On 2018/02/06 21:44, Stefan Johansson wrote: > Hi, > > On 2018-02-06 06:30, Yasumasa Suenaga wrote: >> Hi Stefan, >> >> I've got another report from submit repo, and it includes one error on OS X. >> Can you share it? >> >> http://java.se.oracle.com:10065/mdash/jobs/mach5-one-ysuenaga-JDK-8153333-20180206-0355-10435 > This is a known issue, Robin W is working on a fix as we speak: > https://bugs.openjdk.java.net/browse/JDK-8193308 Thanks! So could you review and sponsor for this change? Yasumasa > Stefan > >> >> I've tested this change (webrev.06) on Linux x64. >> >> >> Thanks, >> >> Yasumasa >> >> >> 2018-02-06 14:10 GMT+09:00 Yasumasa Suenaga : >>> Hi Stefan, >>> >>>> I agree, for G1 this should not be controlled. Maybe I was a bit unclear, I >>>> was wondering why we want to control it for CMS. >>> I said to remove -XX:EnableConcGCPerfCounter in two years ago. I've >>> missed it :-) >>> >>> ?? http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-March/017125.html >>> >>> So I uploaded new webrev. This change includes copyright year updates. >>> >>> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.06/ >>> >>> This change passes all tests on submit repo, and >>> :hotspot_serviceability :jdk_tools tests on my laptop. >>> >>> ?? http://java.se.oracle.com:10065/mdash/jobs/mach5-one-ysuenaga-JDK-8153333-20180206-0222-10428 >>> >>> >>>> If we do the change for CMS, we should >>>> probably also do a CSR, but that should be fairly straight forward. >>> What should I do to get CSR approve? >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> 2018-02-06 0:33 GMT+09:00 Stefan Johansson : >>>> >>>> On 2018-02-03 06:40, Yasumasa Suenaga wrote: >>>>> On 2018/02/02 23:38, Stefan Johansson wrote: >>>>>> Hi Yasumasa, >>>>>> >>>>>> The changes doesn't apply clean on the latest jdk/hs, can you provide an >>>>>> updated webrev? >>>>> >>>>> I uploaded webrev for jdk-hs: >>>>> ?? cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.05/ >>>>> >>>> Thanks, I've kicked off a testing job now to verify nothing unexpected >>>> fails. >>>>> >>>>>> The testing done by the submit repo doesn't cover the tests you have >>>>>> update so I plan to take the change for a spin and make sure the correct >>>>>> tests are run and verified in Mach 5. >>>>> >>>>> I've also tested hotspot/jtreg/:hotspot_serviceability and jdk/:jdk_tools >>>>> on my laptop. >>>>> I did not see any errors / failures which are related to this change. >>>> I also ran some local tests on this and it looks good. >>>>> >>>>> >>>>>> Also a question about the change. Why do we need a special flag for CMS? >>>>>> I see that the original bug report refers to the flag as being a way to turn >>>>>> on and off the feature but the current implementation only consider the flag >>>>>> for CMS. >>>>> >>>>> >>>>> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-March/016774.html >>>>> >>>>> Originally, STW phases (Remark and Cleanup) at G1 are not counted in jstat >>>>> FGC column. >>>>> So I think we need not to control the behavior of PerfCounter for G1. >>>>> >>>> I agree, for G1 this should not be controlled. Maybe I was a bit unclear, I >>>> was wondering why we want to control it for CMS. I think either we should >>>> change the behavior without guarding it by a flag or just skip updating CMS >>>> (and leave the pauses in FGC). If we do the change for CMS, we should >>>> probably also do a CSR, but that should be fairly straight forward. >>>> >>>> I also found the old review thread where Jon M had the same comment >>>> (removing the flag) and it looks like all agreed on that: >>>> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-March/017118.html >>>> >>>> Thanks, >>>> Stefan >>>> >>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>>> Thanks, >>>>>> Stefan >>>>>> >>>>>> On 2018-02-01 14:58, Yasumasa Suenaga wrote: >>>>>>> PING: Could you review and sponsor it? >>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>>>>>> >>>>>>> This change has been passed Mach 5 via submit repo: >>>>>>> >>>>>>> http://java.se.oracle.com:10065/mdash/jobs/mach5-one-ysuenaga-JDK-8153333-20180201-0805-10101 >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2017/11/01 22:02, Yasumasa Suenaga wrote: >>>>>>>> PING: Could you review and sponsor it? >>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>>>>>>> >>>>>>>> Also I need JPRT results of this change. >>>>>>>> Could you cooperate? >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2017/09/27 0:08, Yasumasa Suenaga wrote: >>>>>>>>> Hi all, >>>>>>>>> >>>>>>>>> I uploaded new webrev to be adapted to jdk10/hs: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>>>>>>>> >>>>>>>>> I want to check this patch via JPRT, but I cannot access it. >>>>>>>>> Could you cooperate? >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2017/09/21 7:46, Yasumasa Suenaga wrote: >>>>>>>>>> PING: >>>>>>>>>> >>>>>>>>>> Have you checked this issue? >>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/hotspot/ >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/jdk/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2017/07/01 23:44, Yasumasa Suenaga wrote: >>>>>>>>>>> PING: >>>>>>>>>>> >>>>>>>>>>> Have you checked this issue? >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2017/06/14 13:22, Yasumasa Suenaga wrote: >>>>>>>>>>>> Hi all, >>>>>>>>>>>> >>>>>>>>>>>> I changed PerfCounter to show CGC STW phase in jstat in >>>>>>>>>>>> JDK-8151674. >>>>>>>>>>>> However, it occurred several jtreg test failure, so it was >>>>>>>>>>>> back-outed. >>>>>>>>>>>> >>>>>>>>>>>> I want to resume to work for this issue. >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/hotspot/ >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/jdk/ >>>>>>>>>>>> >>>>>>>>>>>> These changes are work fine on jtreg test as below: >>>>>>>>>>>> >>>>>>>>>>>> ??? hotspot/test/serviceability/tmtools/jstat >>>>>>>>>>>> ??? jdk/test/sun/tools >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Since JDK 9, default GC algorithm is set to G1. >>>>>>>>>>>> So I think this change is useful to watch GC behavior through >>>>>>>>>>>> jstat. >>>>>>>>>>>> >>>>>>>>>>>> I cannot access JPRT. Could you help? >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> > From stefan.johansson at oracle.com Tue Feb 6 12:55:25 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Tue, 6 Feb 2018 13:55:25 +0100 Subject: PING: RFR: JDK-8153333: [REDO] STW phases at Concurrent GC should count in PerfCounter In-Reply-To: References: <2f4e0901-1602-6276-e7fd-84e168e7b317@gmail.com> <3210723e-c5f9-4277-a97a-be61e10e6b3d@oracle.com> <89a1a161-874d-4743-e52b-7ab202fd8976@gmail.com> Message-ID: <9799a2ad-6727-cae9-312b-fd9de13c8203@oracle.com> On 2018-02-06 06:10, Yasumasa Suenaga wrote: > Hi Stefan, > >> I agree, for G1 this should not be controlled. Maybe I was a bit unclear, I >> was wondering why we want to control it for CMS. > I said to remove -XX:EnableConcGCPerfCounter in two years ago. I've > missed it :-) > > http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-March/017125.html > > So I uploaded new webrev. This change includes copyright year updates. > > http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.06/ Thanks Yasumasa, This looks good to me, will do some more testing while waiting for a second reviewer and I can sponsor the change once it's ready to go. > > This change passes all tests on submit repo, and > :hotspot_serviceability :jdk_tools tests on my laptop. > > http://java.se.oracle.com:10065/mdash/jobs/mach5-one-ysuenaga-JDK-8153333-20180206-0222-10428 > > >> If we do the change for CMS, we should >> probably also do a CSR, but that should be fairly straight forward. > What should I do to get CSR approve? In the bug system under "More" you can choose "Create CSR" which is the first step. More information can be found on the wiki: https://wiki.openjdk.java.net/display/csr/CSR+FAQs Cheers, Stefan > > Thanks, > > Yasumasa > > > 2018-02-06 0:33 GMT+09:00 Stefan Johansson : >> >> On 2018-02-03 06:40, Yasumasa Suenaga wrote: >>> On 2018/02/02 23:38, Stefan Johansson wrote: >>>> Hi Yasumasa, >>>> >>>> The changes doesn't apply clean on the latest jdk/hs, can you provide an >>>> updated webrev? >>> >>> I uploaded webrev for jdk-hs: >>> cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.05/ >>> >> Thanks, I've kicked off a testing job now to verify nothing unexpected >> fails. >>> >>>> The testing done by the submit repo doesn't cover the tests you have >>>> update so I plan to take the change for a spin and make sure the correct >>>> tests are run and verified in Mach 5. >>> >>> I've also tested hotspot/jtreg/:hotspot_serviceability and jdk/:jdk_tools >>> on my laptop. >>> I did not see any errors / failures which are related to this change. >> I also ran some local tests on this and it looks good. >>> >>> >>>> Also a question about the change. Why do we need a special flag for CMS? >>>> I see that the original bug report refers to the flag as being a way to turn >>>> on and off the feature but the current implementation only consider the flag >>>> for CMS. >>> >>> >>> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-March/016774.html >>> >>> Originally, STW phases (Remark and Cleanup) at G1 are not counted in jstat >>> FGC column. >>> So I think we need not to control the behavior of PerfCounter for G1. >>> >> I agree, for G1 this should not be controlled. Maybe I was a bit unclear, I >> was wondering why we want to control it for CMS. I think either we should >> change the behavior without guarding it by a flag or just skip updating CMS >> (and leave the pauses in FGC). If we do the change for CMS, we should >> probably also do a CSR, but that should be fairly straight forward. >> >> I also found the old review thread where Jon M had the same comment >> (removing the flag) and it looks like all agreed on that: >> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-March/017118.html >> >> Thanks, >> Stefan >> >> >>> Thanks, >>> >>> Yasumasa >>> >>> >>>> Thanks, >>>> Stefan >>>> >>>> On 2018-02-01 14:58, Yasumasa Suenaga wrote: >>>>> PING: Could you review and sponsor it? >>>>> >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>>>> >>>>> This change has been passed Mach 5 via submit repo: >>>>> >>>>> http://java.se.oracle.com:10065/mdash/jobs/mach5-one-ysuenaga-JDK-8153333-20180201-0805-10101 >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2017/11/01 22:02, Yasumasa Suenaga wrote: >>>>>> PING: Could you review and sponsor it? >>>>>> >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>>>>> >>>>>> Also I need JPRT results of this change. >>>>>> Could you cooperate? >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2017/09/27 0:08, Yasumasa Suenaga wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> I uploaded new webrev to be adapted to jdk10/hs: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>>>>>> >>>>>>> I want to check this patch via JPRT, but I cannot access it. >>>>>>> Could you cooperate? >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> yasumasa >>>>>>> >>>>>>> >>>>>>> On 2017/09/21 7:46, Yasumasa Suenaga wrote: >>>>>>>> PING: >>>>>>>> >>>>>>>> Have you checked this issue? >>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/hotspot/ >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/jdk/ >>>>>>>> >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2017/07/01 23:44, Yasumasa Suenaga wrote: >>>>>>>>> PING: >>>>>>>>> >>>>>>>>> Have you checked this issue? >>>>>>>>> >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2017/06/14 13:22, Yasumasa Suenaga wrote: >>>>>>>>>> Hi all, >>>>>>>>>> >>>>>>>>>> I changed PerfCounter to show CGC STW phase in jstat in >>>>>>>>>> JDK-8151674. >>>>>>>>>> However, it occurred several jtreg test failure, so it was >>>>>>>>>> back-outed. >>>>>>>>>> >>>>>>>>>> I want to resume to work for this issue. >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/hotspot/ >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/jdk/ >>>>>>>>>> >>>>>>>>>> These changes are work fine on jtreg test as below: >>>>>>>>>> >>>>>>>>>> hotspot/test/serviceability/tmtools/jstat >>>>>>>>>> jdk/test/sun/tools >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Since JDK 9, default GC algorithm is set to G1. >>>>>>>>>> So I think this change is useful to watch GC behavior through >>>>>>>>>> jstat. >>>>>>>>>> >>>>>>>>>> I cannot access JPRT. Could you help? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> From yasuenag at gmail.com Tue Feb 6 13:33:26 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Tue, 6 Feb 2018 22:33:26 +0900 Subject: PING: RFR: JDK-8153333: [REDO] STW phases at Concurrent GC should count in PerfCounter In-Reply-To: <9799a2ad-6727-cae9-312b-fd9de13c8203@oracle.com> References: <2f4e0901-1602-6276-e7fd-84e168e7b317@gmail.com> <3210723e-c5f9-4277-a97a-be61e10e6b3d@oracle.com> <89a1a161-874d-4743-e52b-7ab202fd8976@gmail.com> <9799a2ad-6727-cae9-312b-fd9de13c8203@oracle.com> Message-ID: Hi Stefan, > This looks good to me, will do some more testing while waiting for a second reviewer and I can sponsor the change once it's ready to go. Thanks! I'm waiting for second reviewer. >> What should I do to get CSR approve? > In the bug system under "More" you can choose "Create CSR" which is the first step. More information can be found on the wiki: > https://wiki.openjdk.java.net/display/csr/CSR+FAQs I filed new CSR: https://bugs.openjdk.java.net/browse/JDK-8196862 Yasumasa On 2018/02/06 21:55, Stefan Johansson wrote: > > > On 2018-02-06 06:10, Yasumasa Suenaga wrote: >> Hi Stefan, >> >>> I agree, for G1 this should not be controlled. Maybe I was a bit unclear, I >>> was wondering why we want to control it for CMS. >> I said to remove -XX:EnableConcGCPerfCounter in two years ago. I've >> missed it :-) >> >> ?? http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-March/017125.html >> >> So I uploaded new webrev. This change includes copyright year updates. >> >> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.06/ > Thanks Yasumasa, > > This looks good to me, will do some more testing while waiting for a second reviewer and I can sponsor the change once it's ready to go. >> >> This change passes all tests on submit repo, and >> :hotspot_serviceability :jdk_tools tests on my laptop. >> >> ?? http://java.se.oracle.com:10065/mdash/jobs/mach5-one-ysuenaga-JDK-8153333-20180206-0222-10428 >> >> >>> If we do the change for CMS, we should >>> probably also do a CSR, but that should be fairly straight forward. >> What should I do to get CSR approve? > In the bug system under "More" you can choose "Create CSR" which is the first step. More information can be found on the wiki: > https://wiki.openjdk.java.net/display/csr/CSR+FAQs > > Cheers, > Stefan > >> >> Thanks, >> >> Yasumasa >> >> >> 2018-02-06 0:33 GMT+09:00 Stefan Johansson : >>> >>> On 2018-02-03 06:40, Yasumasa Suenaga wrote: >>>> On 2018/02/02 23:38, Stefan Johansson wrote: >>>>> Hi Yasumasa, >>>>> >>>>> The changes doesn't apply clean on the latest jdk/hs, can you provide an >>>>> updated webrev? >>>> >>>> I uploaded webrev for jdk-hs: >>>> ?? cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.05/ >>>> >>> Thanks, I've kicked off a testing job now to verify nothing unexpected >>> fails. >>>> >>>>> The testing done by the submit repo doesn't cover the tests you have >>>>> update so I plan to take the change for a spin and make sure the correct >>>>> tests are run and verified in Mach 5. >>>> >>>> I've also tested hotspot/jtreg/:hotspot_serviceability and jdk/:jdk_tools >>>> on my laptop. >>>> I did not see any errors / failures which are related to this change. >>> I also ran some local tests on this and it looks good. >>>> >>>> >>>>> Also a question about the change. Why do we need a special flag for CMS? >>>>> I see that the original bug report refers to the flag as being a way to turn >>>>> on and off the feature but the current implementation only consider the flag >>>>> for CMS. >>>> >>>> >>>> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-March/016774.html >>>> >>>> Originally, STW phases (Remark and Cleanup) at G1 are not counted in jstat >>>> FGC column. >>>> So I think we need not to control the behavior of PerfCounter for G1. >>>> >>> I agree, for G1 this should not be controlled. Maybe I was a bit unclear, I >>> was wondering why we want to control it for CMS. I think either we should >>> change the behavior without guarding it by a flag or just skip updating CMS >>> (and leave the pauses in FGC). If we do the change for CMS, we should >>> probably also do a CSR, but that should be fairly straight forward. >>> >>> I also found the old review thread where Jon M had the same comment >>> (removing the flag) and it looks like all agreed on that: >>> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-March/017118.html >>> >>> Thanks, >>> Stefan >>> >>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>>> Thanks, >>>>> Stefan >>>>> >>>>> On 2018-02-01 14:58, Yasumasa Suenaga wrote: >>>>>> PING: Could you review and sponsor it? >>>>>> >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>>>>> >>>>>> This change has been passed Mach 5 via submit repo: >>>>>> >>>>>> http://java.se.oracle.com:10065/mdash/jobs/mach5-one-ysuenaga-JDK-8153333-20180201-0805-10101 >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2017/11/01 22:02, Yasumasa Suenaga wrote: >>>>>>> PING: Could you review and sponsor it? >>>>>>> >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>>>>>> >>>>>>> Also I need JPRT results of this change. >>>>>>> Could you cooperate? >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2017/09/27 0:08, Yasumasa Suenaga wrote: >>>>>>>> Hi all, >>>>>>>> >>>>>>>> I uploaded new webrev to be adapted to jdk10/hs: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>>>>>>> >>>>>>>> I want to check this patch via JPRT, but I cannot access it. >>>>>>>> Could you cooperate? >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2017/09/21 7:46, Yasumasa Suenaga wrote: >>>>>>>>> PING: >>>>>>>>> >>>>>>>>> Have you checked this issue? >>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/hotspot/ >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/jdk/ >>>>>>>>> >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2017/07/01 23:44, Yasumasa Suenaga wrote: >>>>>>>>>> PING: >>>>>>>>>> >>>>>>>>>> Have you checked this issue? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2017/06/14 13:22, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi all, >>>>>>>>>>> >>>>>>>>>>> I changed PerfCounter to show CGC STW phase in jstat in >>>>>>>>>>> JDK-8151674. >>>>>>>>>>> However, it occurred several jtreg test failure, so it was >>>>>>>>>>> back-outed. >>>>>>>>>>> >>>>>>>>>>> I want to resume to work for this issue. >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/hotspot/ >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/jdk/ >>>>>>>>>>> >>>>>>>>>>> These changes are work fine on jtreg test as below: >>>>>>>>>>> >>>>>>>>>>> ??? hotspot/test/serviceability/tmtools/jstat >>>>>>>>>>> ??? jdk/test/sun/tools >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Since JDK 9, default GC algorithm is set to G1. >>>>>>>>>>> So I think this change is useful to watch GC behavior through >>>>>>>>>>> jstat. >>>>>>>>>>> >>>>>>>>>>> I cannot access JPRT. Could you help? >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> > From indu.bhagat at oracle.com Tue Feb 6 19:00:40 2018 From: indu.bhagat at oracle.com (Indu Bhagat) Date: Tue, 6 Feb 2018 11:00:40 -0800 Subject: [OpenJDK 2D-Dev] Fwd: [PATCH] Fix compiler warnings with newer GCC 7.2.1 In-Reply-To: <5A75BD5F.5080309@oracle.com> References: <4c5248f3-e78d-31ba-c892-fd7df1d027af@oracle.com> <5A75BD5F.5080309@oracle.com> Message-ID: <5eb1e208-bf8c-ec7e-2a75-3afc594e81d3@oracle.com> Thanks. harfbuzz is now fixed upstream. libjpeg has acknowledged the patch for shift-negative-value warnings. Can someone advise on the minor fix in libdt_socket component ? On 02/03/2018 05:47 AM, Philip Race wrote: > jpeg + harfbuzz are imported libraries. There's no point in a source > fix in JDK'scopy as > it will get overwritten when we upgrade. No way we are tracking such > patches to reapply them. > > If warnings there are an issue either update the makefiles or use > disable-warnings-as-errors > ... and perhaps report the issue upstream .. > > -phil. > > On 2/3/18, 1:35 AM, Alan Bateman wrote: >> >> This was sent to core-libs-dev, I assume it was meant for 2d-dev and >> serviceability-dev. >> >> -------- Forwarded Message -------- >> Subject: [PATCH] Fix compiler warnings with newer GCC 7.2.1 >> Date: Fri, 2 Feb 2018 16:12:08 -0800 >> From: Indu Bhagat >> To: core-libs-dev at openjdk.java.net >> >> >> >> This patch fixes a few of the compiler warnings when using gcc 7.2.1. >> >> In general there are many other warnings (a majority of which are implicit fallthrough warnings in jdk.crypto.ec package's ecp_*.c files and java.desktop package's splashscreen_gfx_impl.h file). >> I can post them if there is interest. >> Most warnings, however, look benign (modulo some warnings about unintialized access that I cannot comment on; they need that the code be thoroughly understood). >> libfdlibm misleading indentation warnings are already knownhttp://mail.openjdk.java.net/pipermail/jdk9-dev/2016-June/004494.html >> >> Thanks >> >> ---------------------------------------------- >> The patch fixes the following warnings : >> >> 1. /jdk/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c: In function ?mask_s2u?: >> /jdk/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c:389:22: warning: left shift of negative value [-Wshift-negative-value] >> *mask = htonl(-1 << (32 - m)); >> ^ >> 2. /jdk/src/java.desktop/share/native/libjavajpeg/jdphuff.c: In function ?decode_mcu_AC_refine?: >> /jdk/src/java.desktop/share/native/libjavajpeg/jdphuff.c:505:17: warning: left shift of negative value [-Wshift-negative-value] >> int m1 = (-1) << cinfo->Al; /* -1 in the bit position being coded*/ >> ^~ >> 3. /jdk/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-ot-shape-fallback.cc: In function ?void position_mark(const hb_ot_shape_plan_t*, hb_font_t*, hb_buffer_t*, hb_glyph_extents_t&, unsigned int, unsigned int)?: >> /jdk/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-ot-shape-fallback.cc:223:14: warning: this statement may fall through [-Wimplicit-fallthrough=] >> } else if (buffer->props.direction == HB_DIRECTION_RTL) { >> ^~ >> /jdk/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-ot-shape-fallback.cc:229:5: note: here >> default: >> ^~~~~~~ >> /jdk/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-ot-shape-fallback.cc:261:27: warning: this statement may fall through [-Wimplicit-fallthrough=] >> base_extents.height -= y_gap; >> ~~~~~~~~~~~~~~~~~~~~^~~~~~~~ >> /jdk/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-ot-shape-fallback.cc:264:5: note: here >> case HB_UNICODE_COMBINING_CLASS_ATTACHED_BELOW_LEFT: >> ^~~~ >> /jdk/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-ot-shape-fallback.cc:282:27: warning: this statement may fall through [-Wimplicit-fallthrough=] >> base_extents.height -= y_gap; >> ~~~~~~~~~~~~~~~~~~~~^~~~~~~~ >> /jdk/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-ot-shape-fallback.cc:285:5: note: here >> case HB_UNICODE_COMBINING_CLASS_ATTACHED_ABOVE: >> ^~~~ >> ----------- >> >> diff --git a/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-private.hh b/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-private.hh >> --- a/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-private.hh >> +++ b/src/java.desktop/share/native/libfontmanager/harfbuzz/hb-private.hh >> @@ -158,6 +158,9 @@ >> */ >> # include >> # define HB_FALLTHROUGH __fallthrough >> +#elif __GNUC__ >= 7 >> + /* GNU fallthrough attribute is available from GCC7 */ >> +# define HB_FALLTHROUGH __attribute__((fallthrough)) >> #else >> # define HB_FALLTHROUGH /* FALLTHROUGH */ >> #endif >> diff --git a/src/java.desktop/share/native/libjavajpeg/jdphuff.c b/src/java.desktop/share/native/libjavajpeg/jdphuff.c >> --- a/src/java.desktop/share/native/libjavajpeg/jdphuff.c >> +++ b/src/java.desktop/share/native/libjavajpeg/jdphuff.c >> @@ -501,8 +501,10 @@ >> { >> phuff_entropy_ptr entropy = (phuff_entropy_ptr) cinfo->entropy; >> int Se = cinfo->Se; >> - int p1 = 1 << cinfo->Al; /* 1 in the bit position being coded */ >> - int m1 = (-1) << cinfo->Al; /* -1 in the bit position being coded */ >> + /* 1 in the bit position being coded */ >> + int p1 = 1 << cinfo->Al; >> + /* -1 in the bit position being coded */ >> + int m1 = (int)((unsigned)(~0) << cinfo->Al); >> register int s, k, r; >> unsigned int EOBRUN; >> JBLOCKROW block; >> diff --git a/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c b/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c >> --- a/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c >> +++ b/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c >> @@ -386,7 +386,7 @@ >> return instr; >> } >> >> - *mask = htonl(-1 << (32 - m)); >> + *mask = htonl((uint32_t)(~0) << (32 - m)); >> return s; >> } >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Tue Feb 6 20:06:33 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 6 Feb 2018 12:06:33 -0800 Subject: [OpenJDK 2D-Dev] Fwd: [PATCH] Fix compiler warnings with newer GCC 7.2.1 In-Reply-To: <5eb1e208-bf8c-ec7e-2a75-3afc594e81d3@oracle.com> References: <4c5248f3-e78d-31ba-c892-fd7df1d027af@oracle.com> <5A75BD5F.5080309@oracle.com> <5eb1e208-bf8c-ec7e-2a75-3afc594e81d3@oracle.com> Message-ID: <73005887-dd65-f81e-6b86-7806d4813a69@oracle.com> An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Tue Feb 6 20:56:00 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 6 Feb 2018 12:56:00 -0800 Subject: [OpenJDK 2D-Dev] Fwd: [PATCH] Fix compiler warnings with newer GCC 7.2.1 In-Reply-To: <73005887-dd65-f81e-6b86-7806d4813a69@oracle.com> References: <4c5248f3-e78d-31ba-c892-fd7df1d027af@oracle.com> <5A75BD5F.5080309@oracle.com> <5eb1e208-bf8c-ec7e-2a75-3afc594e81d3@oracle.com> <73005887-dd65-f81e-6b86-7806d4813a69@oracle.com> Message-ID: <8eefd33c-91bf-c2f6-69be-a41dd9cc5bda@oracle.com> An HTML attachment was scrubbed... URL: From coleen.phillimore at oracle.com Tue Feb 6 21:06:50 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 6 Feb 2018 16:06:50 -0500 Subject: RFR (XS) 8174734: Safepoint sync time did not increase In-Reply-To: References: <0ee1d208-f805-72ca-2480-f53efa513435@oracle.com> Message-ID: On 2/6/18 12:13 AM, David Holmes wrote: > Hi Coleen, > > On 6/02/2018 7:37 AM, coleen.phillimore at oracle.com wrote: >> Summary: allow safepoint time to be zero in the test >> >> See bug for more details. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8174734.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8174734 > > I guess I'm still surprised that 300 thread dumps can take less than a > millisecond! There's always more than one thread running. I did some > basic benchmarking and dumpAllStacks() from main takes at least 150us > on the Linux box I tested on. I just can't see 300 dumps taking less > than 1ms ... though I can see them taking < 10ms if we're measuring > time using a coarse clock - where do these times come from? > I think the thread dumps only the actual JavaThread which is not "hidden_from_view".? There are lots of threads but they're all GC and compiler threads when I ran this test. > That aside this change seem unnecessary: > > ????? // Careful with these values. > !???? private static final long MIN_VALUE_FOR_PASS = 0; > ????? private static final long MAX_VALUE_FOR_PASS = Long.MAX_VALUE; This was another one of the failures modes, so we need this change to make this test more reliable. > > this is for the minimum number of safepoints that need to be seen, > which I think should still be 1. By allowing 0 here (and for the > elapsed time), the test could actually fail to do anything related to > safepoints and still pass - and that seems wrong. Or the safepoint > stat code could be completely broken and we'd never notice. Basically > the test just wants to check that we get reasonable looking statistics > from the MBean > > Maybe we need to be measuring the time at a higher resolution than > milliseconds - though that would be a non-trivial RFE I expect. ? > So, looking at and debugging the runtimeService.cpp code, it appears to be doing the thing that it's supposed to be doing.? I agree that it's not a particularly useful test when changing the times to zero, although I traced through and it does exercise the code, and logging makes it non-zero. What you're suggesting would be a lot more work.? I guess my work was to get the test off the ProblemList.txt but if you'd prefer doing more work, I'll reassign it and withdraw this RFR.? I thought getting it running without failure is more worth doing than writing a new test for this feature honestly. thanks, Coleen > Thanks, > David > >> Thanks, >> Coleen From chris.plummer at oracle.com Tue Feb 6 22:27:54 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 6 Feb 2018 14:27:54 -0800 Subject: [8u] RFR for 'JDK-8195088: [TEST_BUG] StartManagementAgent got unexpected exception' In-Reply-To: <5a466d6e-a899-45bf-982a-dc35e06c61a4@default> References: <35e118d1-5f59-4947-bad0-4f5fbc032474@default> <5a466d6e-a899-45bf-982a-dc35e06c61a4@default> Message-ID: Hi Shafi, A simple change with a complex history. It looks like you are backporting just the test modification that was done in JDK 10 for JDK-8165736. And this is needed in 8u because of the JDK-8188865 and JDK-8179643 backports that resulted in a similar change to the error output that JDK-8165736 did. I'm not quite seeing the entire backport picture here, but I wonder why you don't backport all of JDK-8165736. cheers, Chris On 2/5/18 9:56 PM, Shafi Ahmad wrote: > Hi, > > Could someone please review it. > > Regards, > Shafi > >> -----Original Message----- >> From: Shafi Ahmad >> Sent: Monday, January 29, 2018 6:02 PM >> To: serviceability-dev at openjdk.java.net >> Subject: [8u] RFR for 'JDK-8195088: [TEST_BUG] StartManagementAgent got >> unexpected exception' >> >> Hi, >> >> Please review the trivial code change for the fix of 'JDK-8195088: [TEST_BUG] >> StartManagementAgent got unexpected exception' to jdk8u-dev. >> >> Summary: >> This test case is expected to fail but the expected exception message is >> different. >> This fix >> - if (!ex.getMessage().contains("Invalid >> com.sun.management.jmxremote.port number")) { >> + if (!ex.getMessage().contains("NumberFormatException: For input >> string: \"apa\"")) { >> >> is checking for the correct message. >> >> This is fixed in jdk10 under bug JDK-8165736 [Error message should be shown >> when JVMTI agent cannot be attache] >> http://hg.openjdk.java.net/jdk/hs/rev/bc1cffa26561#l10.1 >> >> Jdk8 bug: https://bugs.openjdk.java.net/browse/JDK-8195088 >> webrev link: http://cr.openjdk.java.net/~shshahma/8195088/webrev.00/ >> >> Regards, >> Shafi From jcbeyler at google.com Wed Feb 7 06:43:28 2018 From: jcbeyler at google.com (JC Beyler) Date: Tue, 6 Feb 2018 22:43:28 -0800 Subject: JDK-8171119: Low-Overhead Heap Profiling In-Reply-To: <72db2c82-e118-965b-f6b8-f4cd338f5904@oracle.com> References: <72db2c82-e118-965b-f6b8-f4cd338f5904@oracle.com> Message-ID: Hi David, The new webrev is here: http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.06/ The incremental one is here: http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.05a_06/ Notes: - I've also added two new tests that test the HeapMonitorStatObjectCorrectnessTest but with the ParallelGC, the SerialGC, and CMS. - I still don't know what would be better for the naming and that was addressed in the answer to Erik - I cleaned up a bit more the code here and there Thank you as well for the reviews! I've inlined my answers here as well: On Mon, Feb 5, 2018 at 8:40 PM, David Holmes wrote: > Hi Jc, > > I've just been browsing this and have a few comments/queries > > src/hotspot/share/prims/jvmtiHeapTransition.hpp > > In HeapThreadTransition what are the possible entry states? The primary > state is presumably _in_native, but what else is expected? You can't > transition from arbitrary states to _in_vm > > Also in that file the include guard has the wrong name: > > #ifndef SHARE_VM_PRIMS_JVMTIHEAPSAMPLING_HPP > > etc. > > Done, I fixed both the jvmtiHeapTransition and the heapMonitoring files, they were the only ones I saw that. > --- > > src/hotspot/share/runtime/heapMonitoring.cpp > > Can you change StackTraceData from a struct to a class please. > > StackTraceData::free_data could still be an instance method even if not > done in the destructor. I also think the caller should do the delete rather > than hiding it in here, as this obscures the lifecycle of the instances. > > Done, I cleaned it up to be classes, refactored the decrementation to be in a single static method, which calls delete if need be. I think it still looks similar to what you did not like but perhaps the renaming and the fact it is now more integrated makes it better? The other solution I have is to just inline that code across the board to always have the caller do: a) Decrement the reference count b) check the reference count c) if it is 0, get the trace data, free it; then free the instance But I find that messier so I left to a similar system as before... any better ideas? Also, due to this, it exposes the need to use get_oop_addr at the weak_oops_do stage for the closure. Is there a way around this? This is what I have: oop value = trace.load_oop(); if (is_alive->do_object_b(value)) { // Update the oop to point to the new object if it is still alive. f->do_oop(trace.get_oop_addr()); But I imagine I could/should perhaps do: oop value = trace.load_oop(); if (is_alive->do_object_b(value)) { // Update the oop to point to the new object if it is still alive. f->do_oop(&value); trace.store_oop(&value); Which do you think is better in this case? 215 void initialize(int max_storage) { > 216 MutexLocker mu(HeapMonitorStorage_lock); > 217 allocate_storage(max_storage); > 218 memset(&_stats, 0, sizeof(_stats)); > 219 } > > You're using a lock so obviously expect multiple threads to try this, but > you aren't checking if initialization has already been done. That check is > inside allocate_storage, but you're doing the memset unconditionally. > Yes sorry, I added the _stats later and did not see this. Thanks for catching it. (The truth is that this is also protected by the HeapMonitoring class above that will only call the initialization once at enabling so it is protected by that too, no one could call this multiple times in a row but still, better being consistent). > > You're inconsistent with accessing/modifying _initialized under the > HeapMonitorStorage_lock. The destructor calls free_storage with no lock > (obviously) held. It's unclear how this lock-free destructor fits into the > concurrent usage expected of this class. StackTraceStorage is a singleton created once and then never destroyed during the lifetime of the program. Only if you start/stop the profiling would you call initialize/stop, which do have a mutex hold. The destructor did not have one due to an oversight that it probably should have one for when the program ends to ensure nothing is happening concurrently at that time. I've reviewed the usage of that class and the mutex and fixed it up to be consistent (I believe) > I see from your reply to Erik that you've now added some orderAccess usage > to be "paranoid". But we don't want paranoid, we want correct. If there is > expected to be a lock-free, but potentially concurrent path, then you will > need to use OrderAccess appropriately. If everything is actually being done > under a lock, then you don't need to use OrderAccess. But you need to be > clear on exactly how concurrency comes into play with your code. > That is a fair point and I was not trying to convey that kind of meaning when I said it that way. I apologize for that. I've gone through the code and ensured to my best ability: - StackTraceStorage has no lock-free concurrent path to _initialized; therefore I removed the OrderAccess for it - HeapMonitoring::_enabled did however via the enabled method so I now load/store it via OrderAccess Basically what it amounts to: HeapMonitoring has a mutex that controls enabling/disabling and uses OrderAccess to get the value of the enabled flag. All other public HeapMonitoring methods are going directly to the StackTraceStorage and hit the second mutex there if needed, which protects the internal data. > 397 _allocated_traces = new (ResourceObj::C_HEAP, mtInternal) > 398 GrowableArray(128, true); > > These are allocations that will abort the VM if they fail - correct? If so > that seems rather user-unfriendly for a profiling tool! > I looked around to see what could be done here but failed to see an example of code that is nicer (in the sense would not just fail). Do you have a pointer to where I could inspire myself to not have it fail? In general, on our production code, this is on by default and turned on at start and never turned off, so this case does not arise but I can add code to handle the case if you have an example. > > 422 void StackTraceStorage::weak_oops_do(BoolObjectClosure *is_alive, > 423 OopClosure *f) { > 424 MutexLocker mu(HeapMonitorStorage_lock); > > IIUC oops-do methods get called at safepoints, but you are then taking a > Mutex unconditionally. So any other code that acquires the same mutex must > be guaranteed not to hit a safepoint check - so NoSafepointVerifier should > be used to check that. > Agreed, Robbin pointed me to that and I fixed that in a previous version. > > 575 void StackTraceStorage::store_garbage_trace(const StackTraceData > &trace) { > 576 StackTraceData *new_trace = new StackTraceData(); > 577 *new_trace = trace; > > This looks quite odd to me. Shouldn't this just be using a > copy-constructor? > > Agreed, this got fixed before (I don't remember when). This used to be a C-ism that never got fixed. As soon as I created a StackTraceDataWithOop and one without, this disappeared. > Thanks, > David > Thanks again! Jc > ----- > > > On 27/01/2018 2:01 PM, JC Beyler wrote: > >> (Change of subject to contain the bug number :)). >> >> New webrev is here: >> Incremental: >> http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.03_04/ >> >> Full webrev: >> http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.04/ >> >> Inlined are my answers again :). >> >> On Fri, Jan 26, 2018 at 5:51 AM, Robbin Ehn >> wrote: >> >>> Hi JC, (dropping compiler on mail thread) >>> >>> On 01/26/2018 06:45 AM, JC Beyler wrote: >>> >>>> >>>> Thanks Robbin for the reviews :) >>>> >>>> The new full webrev is here: >>>> http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.03/ >>>> The incremental webrev is here: >>>> http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.02_03/ >>>> >>> >>> >>> Thanks! >>> >>> I got this compile issue: >>> /home/rehn/source/jdk/small/open/src/hotspot/share/gc/shared >>> /threadLocalAllocBuffer.hpp: >>> In static member function 'static void >>> G1ResManTLABCache::put(ThreadLocalAllocBuffer&, AllocationContext_t)': >>> /home/rehn/source/jdk/small/open/src/hotspot/share/gc/shared >>> /threadLocalAllocBuffer.hpp:97:13: >>> error: 'HeapWord* ThreadLocalAllocBuffer::hard_end()' is private >>> HeapWord* hard_end(); >>> ^ >>> /home/rehn/source/jdk/small/closed/src/hotspot/share/gc/g1/g >>> 1ResManTLABCache.cpp:52:50: >>> error: within this context >>> size_t remaining = pointer_delta(tlab.hard_end(), tlab.top()); >>> >> >> This is strange but I'm assuming it is because we are not working on >> the same repo? >> >> I used: >> hg clone http://hg.openjdk.java.net/jdk/hs jdkhs-heap >> >> I'll try a new clone on Monday and see. My new version moved hard_end >> back to public so it should work now. >> >> >> >> >>> >>>> I inlined my answers: >>>> >>>> On Thu, Jan 25, 2018 at 1:15 AM, Robbin Ehn >>>> wrote: >>>> >>>>> >>>>> Hi JC, great to see another revision! >>>>> >>>>> #### >>>>> heapMonitoring.cpp >>>>> >>>>> StackTraceData should not contain the oop for 'safety' reasons. >>>>> When StackTraceData is moved from _allocated_traces: >>>>> L452 store_garbage_trace(trace); >>>>> it contains a dead oop. >>>>> _allocated_traces could instead be a tupel of oop and StackTraceData >>>>> thus >>>>> dead oops are not kept. >>>>> >>>> >>>> >>>> Done I used inheritance to make the copier work regardless but the >>>> idea is the same. >>>> >>> >>> >>> Looks good. >>> >>> >>>> >>>>> You should use the new Access API for loading the oop, something like >>>>> this: >>>>> RootAccess::load(...) >>>>> I don't think you need to use Access API for clearing the oop, but it >>>>> would >>>>> look nicer. And you shouldn't probably be using: >>>>> Universe::heap()->is_in_reserved(value) >>>>> >>>> >>>> >>>> I am unfamiliar with this but I think I did do it like you wanted me >>>> to (all tests pass so that's a start). I'm not sure how to clear the >>>> oop exactly, is there somewhere that does that, which I can use to do >>>> the same? >>>> >>>> I removed the is_in_reserved, this came from our internal version, I >>>> don't know why it was there but my tests work without so I removed it >>>> :) >>>> >>> >>> >>> I talked a bit with GC folks about this today. >>> So actually the oop should be in the oopStorage and your should have a >>> weakhandle to that oop (at least in the near future). >>> But this should work for now... >>> In the future when we have the oop in oppStorage you will not get >>> called, so >>> you will not know when the oops are dead, either GC must trigger a >>> concurrent vm operation (e.g. _no_ safepoint operation) so we can clear >>> dead >>> oops or do periodic scanning. >>> >>> It would be good with some input here from Thomas that knows what you are >>> doing and knows GC. >>> >> >> Fair enough, hopefully Thomas will chime in. Are you saying that this >> first version could go in and we can work on a refinement? Or are you >> saying I should work on this now at the same time and fix it before >> this V1 goes in? (Just so I know :)) >> >> >>> >>>> >>>> >>>>> The lock: >>>>> L424 MutexLocker mu(HeapMonitorStorage_lock); >>>>> Is not needed as far as I can see. >>>>> weak_oops_do is called in a safepoint, no TLAB allocation can happen >>>>> and >>>>> JVMTI thread can't access these data-structures. Is there something >>>>> more >>>>> to >>>>> this lock that I'm missing? >>>>> >>>> >>>> >>>> Since a thread can call the JVMTI getLiveTraces (or any of the other >>>> ones), it can get to the point of trying to copying the >>>> _allocated_traces. I imagine it is possible that this is happening >>>> during a GC or that it can be started and a GC happens afterwards. >>>> Therefore, it seems to me that you want this protected, no? >>>> >>> >> Looks like it yes, I removed it, ran my tests and they work so the new >> webrev no longer has that mutex at all. >> >> >>> >>> A thread calling getLiveTraces will be stopped in the >>> HeapThreadTransition. >>> A safepoint don't allow any threads to be in_vm or to be in_java during >>> safepoint. Only threads in native can execute, but they will be stopped >>> on >>> any transition. If a thread is in_vm the safepoint waits to a transition >>> (locking a mutex is also transition to blocked). So if weak_oops is >>> called >>> you have an guarantee that no threads are executing inside the VM or >>> executing Java code. (not counting GC threads of course) >>> This also means that the lock can never be contented when weak_oops is >>> called, so it's not harmful. >>> >>> >>>> >>>> >>>>> #### >>>>> You have 6 files without any changes in them (any more): >>>>> g1CollectedHeap.cpp >>>>> psMarkSweep.cpp >>>>> psParallelCompact.cpp >>>>> genCollectedHeap.cpp >>>>> referenceProcessor.cpp >>>>> thread.hpp >>>>> >>>>> >>>> Done. >>>> >>>> #### >>>>> I have not looked closely, but is it possible to hide heap sampling in >>>>> AllocTracer ? (with some minor changes to the AllocTracer API) >>>>> >>>>> >>>> I am imagining that you are saying to move the code that does the >>>> sampling code (change the tlab end, do the call to HeapMonitoring, >>>> etc.) into the AllocTracer code itself? I think that is right and I'll >>>> look if that is possible and prepare a webrev to show what would be >>>> needed to make that happen. >>>> >>> >>> >>> Sounds good. >>> >> >> I'll look at this on Monday then! >> >> Thanks for the reply and have a great weekend! >> Jc >> >> >>> >>>> #### >>>>> Minor nit, when declaring pointer there is a little mix of having the >>>>> pointer adjacent by type name and data name. (Most hotspot code is by >>>>> type >>>>> name) >>>>> E.g. >>>>> heapMonitoring.cpp:711 jvmtiStackTrace *trace = .... >>>>> heapMonitoring.cpp:733 Method* m = vfst.method(); >>>>> (not just this file) >>>>> >>>>> >>>> Done! >>>> >>>> #### >>>>> HeapMonitorThreadOnOffTest.java:77 >>>>> I would make g_tmp volatile, otherwise the assignment in loop may >>>>> theoretical be skipped. >>>>> >>>>> >>>> Also done! >>>> >>> >>> >>> Looks good, thanks! >>> >>> /Robbin >>> >>> >>>> Thanks again! >>>> Jc >>>> >>>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From harsha.wardhana.b at oracle.com Wed Feb 7 09:19:45 2018 From: harsha.wardhana.b at oracle.com (Harsha Wardhana B) Date: Wed, 7 Feb 2018 14:49:45 +0530 Subject: RFR: JDK-8187498: Add a -Xmanagement flag as syntactic sugar for -Dcom.sun.management.jmxremote.* properties In-Reply-To: <248276b1-1589-561f-0718-7bcb1fd578c7@oracle.com> References: <0fca13c3-1b8f-947c-9041-bdea87b04ba8@oracle.com> <5A687F75.3050404@oracle.com> <05c2e5af-b9d0-0386-4565-ade1179a1d49@oracle.com> <7a18b9d5-3a59-8132-2c4f-ba5de35bfa1d@oracle.com> <0e019151-ef48-b4f4-d253-b9da62aa07c3@oracle.com> <248276b1-1589-561f-0718-7bcb1fd578c7@oracle.com> Message-ID: <40cf194e-4d74-3a34-5eb2-1acc5a5abafb@oracle.com> Hi All, After internal discussions, below format for management flags was agreed upon. 1. --start-management-agent port=1234,ssl=on??? ??? (space seperator) or 2. --start-management-agent=port=1234,ssl=on??? ??? ('=' seperator) If option 1 is specified, it will be converted to option 2 by the java launcher before it is passed onto VM. With above GNU long format for management options, specifying arguments is mandatory unlike before. --start-management-agent will not be recognized in the current format and hence will not default to --start-management-agent=local=true. Below is the webrev with above changes and corresponding tests. http://cr.openjdk.java.net/~hb/8187498/webrev.01/ Please review and comment. Thanks Harsha On Monday 29 January 2018 03:14 PM, Harsha Wardhana B wrote: > Hi Alan, > > I am not fully aware about Java launcher or how it passes options to > VM. Let me check with some other folks and get back to you. > > Thanks > Harsha > > On Monday 29 January 2018 01:55 PM, Alan Bateman wrote: >> >> >> On 29/01/2018 05:20, Harsha Wardhana B wrote: >>> Hi Mandy,Alan, >>> >>> Thanks for your inputs. >>> If I keep it as launcher option, it may need to know JMX agent flags >>> which may need to be extended in future. >>> I would prefer making it a VM option. I will make the required >>> changes and send out an updated webrev. >> I think Mandy's suggestion is to just transform --management >> so a form that the VM can read. The launcher will need to >> replace the space anyway as the VM only accepts "=". >> >> -Alan > From robin.westberg at oracle.com Wed Feb 7 15:18:05 2018 From: robin.westberg at oracle.com (Robin Westberg) Date: Wed, 7 Feb 2018 16:18:05 +0100 Subject: RFR: 8041626: [Event Request] Shutdown reason Message-ID: Hi all, Please review the following change that adds an event-based tracing event that is generated when the VM shuts down. The intent is to capture shutdowns that occur after the VM has been properly initialized (as initialization problems would most likely mean that the tracing framework hasn?t been properly started either). Issue: https://bugs.openjdk.java.net/browse/JDK-8041626 Webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00/ Testing: hs-tier1,hs-tier2,jdk-tier1,jdk-tier2 Best regards, Robin -------------- next part -------------- An HTML attachment was scrubbed... URL: From kumar.x.srinivasan at oracle.com Wed Feb 7 15:32:57 2018 From: kumar.x.srinivasan at oracle.com (Kumar Srinivasan) Date: Wed, 07 Feb 2018 07:32:57 -0800 Subject: RFR: JDK-8187498: Add a -Xmanagement flag as syntactic sugar for -Dcom.sun.management.jmxremote.* properties In-Reply-To: <40cf194e-4d74-3a34-5eb2-1acc5a5abafb@oracle.com> References: <0fca13c3-1b8f-947c-9041-bdea87b04ba8@oracle.com> <5A687F75.3050404@oracle.com> <05c2e5af-b9d0-0386-4565-ade1179a1d49@oracle.com> <7a18b9d5-3a59-8132-2c4f-ba5de35bfa1d@oracle.com> <0e019151-ef48-b4f4-d253-b9da62aa07c3@oracle.com> <248276b1-1589-561f-0718-7bcb1fd578c7@oracle.com> <40cf194e-4d74-3a34-5eb2-1acc5a5abafb@oracle.com> Message-ID: <5A7B1C29.2040308@oracle.com> Hi Harsha, Changes look reasonable to me, couple of things that must be addressed: 1. Since this is a main-stream launcher change with a documented and supported option, a CSR is required, you have to add and document the option in the help page http://hg.openjdk.java.net/jdk/jdk/file/8cc67294ec56/src/java.base/share/classes/sun/launcher/resources/launcher.properties 2. You also have to create a doc bug so that the doc team will document it in the Tools Reference Guide, and link it to this bug. Does it need a Release note ? probably does, in which case you will have to create Release Note subtask and follow the RN process. 3. Is XmanagementAgentTest.java part of tier1 test suite ? If not, then I think it ought to be in tier1 grouping, perhaps best to park this under jdk/tools/launcher/management ? Kumar > Hi All, > > After internal discussions, below format for management flags was > agreed upon. > > 1. --start-management-agent port=1234,ssl=on (space seperator) > or > 2. --start-management-agent=port=1234,ssl=on ('=' seperator) > > If option 1 is specified, it will be converted to option 2 by the java > launcher before it is passed onto VM. > > With above GNU long format for management options, specifying > arguments is mandatory unlike before. > > --start-management-agent will not be recognized in the current format > and hence will not default to --start-management-agent=local=true. > > Below is the webrev with above changes and corresponding tests. > > http://cr.openjdk.java.net/~hb/8187498/webrev.01/ > > Please review and comment. > > Thanks > Harsha > > On Monday 29 January 2018 03:14 PM, Harsha Wardhana B wrote: >> Hi Alan, >> >> I am not fully aware about Java launcher or how it passes options to >> VM. Let me check with some other folks and get back to you. >> >> Thanks >> Harsha >> >> On Monday 29 January 2018 01:55 PM, Alan Bateman wrote: >>> >>> >>> On 29/01/2018 05:20, Harsha Wardhana B wrote: >>>> Hi Mandy,Alan, >>>> >>>> Thanks for your inputs. >>>> If I keep it as launcher option, it may need to know JMX agent >>>> flags which may need to be extended in future. >>>> I would prefer making it a VM option. I will make the required >>>> changes and send out an updated webrev. >>> I think Mandy's suggestion is to just transform --management >>> so a form that the VM can read. The launcher will need to >>> replace the space anyway as the VM only accepts "=". >>> >>> -Alan >> > From Alan.Bateman at oracle.com Wed Feb 7 15:30:11 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 7 Feb 2018 15:30:11 +0000 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: References: Message-ID: <22b0955b-e1dc-a3b7-ab1b-8e4f0ec49670@oracle.com> On 07/02/2018 15:18, Robin Westberg wrote: > Hi all, > > Please review the following change that adds an event-based tracing > event that is generated when the VM shuts down. The intent is to > capture shutdowns that occur after the VM has been properly > initialized (as initialization problems would most likely mean that > the tracing framework hasn?t been properly started either). > > Issue: https://bugs.openjdk.java.net/browse/JDK-8041626 > Webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00/ > > Testing:?hs-tier1,hs-tier2,jdk-tier1,jdk-tier2 > Can you elaborate a bit on why this isn't in JVM_Halt? Is this partially to help with cases where the shutdown hooks or finalizers run at exit cause issues? -Alan From stewartd.qdt at qualcommdatacenter.com Wed Feb 7 16:42:53 2018 From: stewartd.qdt at qualcommdatacenter.com (stewartd.qdt) Date: Wed, 7 Feb 2018 16:42:53 +0000 Subject: Issue with 8192897: NPE occurs on clhsdb jstack Message-ID: I am getting Null Pointer Exceptions with both ClhsdbJstack.java and ClhsdbFindPC.java. It appears the addition of testing with '-Xcomp` causing a Method to come back as null during the stack walk. I have included the results of the Jstack test (the FindPC is similar) for the Xcomp section below. I have created bug JDK-8196969. The full .jtr is attached there. I am not familiar enough with the Java stack to know if having a null Method pointer is valid or not. Thanks, Daniel hsdb> Command line: ['/home/stewartd/openjdk/OpenJDK/hs/build/linux-aarch64-normal-server-release/images/jdk/bin/java' '-Xcomp' '-cp' '/home/stewartd/openjdk/OpenJDK/hs/JTwork/classes/serviceability/sa/ClhsdbJstack.d:/home/stewartd/openjdk/OpenJDK/hs/JTwork/classes/test/lib' 'jdk.test.lib.apps.LingeredApp' '717a6b55-bcf8-405e-9d5a-afe1212f0fc6.lck' ] Started LingeredApp (-Xcomp) with pid 23731 Starting clhsdb against 23731 Warning! JS Engine can't start, some commands will not be available. hsdb> Deadlock Detection: No deadlocks found. "Common-Cleaner" #22 daemon prio=8 tid=0x0000ffffa0528000 nid=0x5d2d in Object.wait() [0x0000fffe427de000] java.lang.Thread.State: TIMED_WAITING (on object monitor) JavaThread state: _thread_blocked - java.lang.ref.ReferenceQueue.remove(long) @bci=59, line=151, pc=0x0000ffff892af7c4, Method*=0x0000fffe49851a98 (Compiled frame; information may be imprecise) - waiting to lock <0x0000000101c956f0> (a java.lang.ref.ReferenceQueue$Lock) - jdk.internal.ref.CleanerImpl.run() @bci=65, line=148, pc=0x0000ffff892ac8e4, Method*=0x0000fffe49ab23d8 (Compiled frame) - java.lang.Thread.run() @bci=11, line=844, pc=0x0000ffff892a9d30, Method*=0x0000fffe49738ee0 (Compiled frame) - jdk.internal.misc.InnocuousThread.run() @bci=20, line=134, pc=0x0000ffff892a9d30, Method*=0x0000fffe49aba318 (Compiled frame) Locked ownable synchronizers: - None "Signal Dispatcher" #4 daemon prio=9 tid=0x0000ffffa04a6800 nid=0x5d05 runnable [0x0000000000000000] java.lang.Thread.State: RUNNABLE JavaThread state: _thread_blocked Locked ownable synchronizers: - None "Finalizer" #3 daemon prio=8 tid=0x0000ffffa0494000 nid=0x5d04 in Object.wait() [0x0000fffe492ee000] java.lang.Thread.State: WAITING (on object monitor) JavaThread state: _thread_blocked - java.lang.ref.ReferenceQueue.remove(long) @bci=59, line=151, pc=0x0000ffff888e15a0, Method*=0x0000fffe49851a98 (Interpreted frame) - waiting to lock <0x0000000101c09528> (a java.lang.ref.ReferenceQueue$Lock) - java.lang.ref.ReferenceQueue.remove() @bci=2, line=172, pc=0x0000ffff888e12c0, Method*=0x0000fffe49851b40 (Interpreted frame) - java.lang.ref.Finalizer$FinalizerThread.run() @bci=37, line=216, pc=0x0000ffff888e12c0, Method*=0x0000fffe49856cb0 (Interpreted frame) Locked ownable synchronizers: - None "Reference Handler" #2 daemon prio=10 tid=0x0000ffffa0492000 nid=0x5d03 waiting on condition [0x0000fffe494ee000] java.lang.Thread.State: RUNNABLE JavaThread state: _thread_blocked - java.lang.ref.Reference.processPendingReferences() @bci=0, line=166, pc=0x0000ffff888e15a0, Method*=0x0000fffe49733100 (Interpreted frame) - java.lang.ref.Reference.access$000() @bci=0, line=44, pc=0x0000ffff888e15a0, Method*=0x0000fffe49733788 (Interpreted frame) - java.lang.ref.Reference$ReferenceHandler.run() @bci=0, line=138, pc=0x0000ffff888e15a0, Method*=0x0000fffe4984ffa0 (Interpreted frame) Locked ownable synchronizers: - None "main" #1 prio=5 tid=0x0000ffffa0010000 nid=0x5cb5 waiting on condition [0x0000ffffa5d2e000] java.lang.Thread.State: TIMED_WAITING (sleeping) JavaThread state: _thread_blocked Error occurred during stack walking: Locked ownable synchronizers: - None hsdb> ----------System.err:(147/8216)---------- Attaching to process 23463, please wait... javax.script.ScriptException: TypeError: sapkg.runtime.VM.getVM is not a function in sa.js at line number 54 javax.script.ScriptException: TypeError: sapkg.runtime.VM.getVM is not a function in sa.js at line number 54 Attaching to process 23731, please wait... javax.script.ScriptException: TypeError: sapkg.runtime.VM.getVM is not a function in sa.js at line number 54 javax.script.ScriptException: TypeError: sapkg.runtime.VM.getVM is not a function in sa.js at line number 54 java.lang.NullPointerException at jdk.hotspot.agent/sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:83) at jdk.hotspot.agent/sun.jvm.hotspot.CommandProcessor$26.doit(CommandProcessor.java:1073) at jdk.hotspot.agent/sun.jvm.hotspot.CommandProcessor.executeCommand(CommandProcessor.java:1966) at jdk.hotspot.agent/sun.jvm.hotspot.CommandProcessor.executeCommand(CommandProcessor.java:1936) at jdk.hotspot.agent/sun.jvm.hotspot.CommandProcessor.run(CommandProcessor.java:1816) at jdk.hotspot.agent/sun.jvm.hotspot.CLHSDB.run(CLHSDB.java:99) at jdk.hotspot.agent/sun.jvm.hotspot.CLHSDB.main(CLHSDB.java:40) at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runCLHSDB(SALauncher.java:191) at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:439) -------------- next part -------------- An HTML attachment was scrubbed... URL: From gary.adams at oracle.com Wed Feb 7 16:51:07 2018 From: gary.adams at oracle.com (gary.adams at oracle.com) Date: Wed, 7 Feb 2018 11:51:07 -0500 Subject: RFR: JDK-8031445: Attach on windows can fail with java.io.IOException: All pipe instances are busy Message-ID: The IOException that is observed when creating a new named pipe when the pipe already exists and is in use, recommends to retry the operation later. Since we are already using a random number to generate a unique pipe name, it makes sense to simply retry the operation with a new pipe name. Here is a proposed fix. Testing in progress. ? Issue: https://bugs.openjdk.java.net/browse/JDK-8031445 ? Webrev: http://cr.openjdk.java.net/~gadams/8031445/ From chris.hegarty at oracle.com Wed Feb 7 17:31:35 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 7 Feb 2018 17:31:35 +0000 Subject: JDK-8080990: libdt_socket/socket_md.c(202) : warning C4996: 'gethostbyname': Use getaddrinfo() or GetAddrInfoW() In-Reply-To: <5A7331D9.9000800@oracle.com> References: <5A6A2611.4030003@oracle.com> <5A7331D9.9000800@oracle.com> Message-ID: <624B1D02-366B-4FF0-92F4-757ED64D356F@oracle.com> Gary, > http://cr.openjdk.java.net/%7Egadams/8080990/webrev.02/ I think the replacement of WSASendDisconnect with shutdown(SD_SEND) should be fine. I do note that there is another usage of WSASendDisconnect in java.base/windows/native/libnet/net_util_md.c. [ Maybe you want to separate out the changes in java.base ( NIO and NET ) from the serviceability changes? Up to you. ] Curious about the specific hints you have chosen to use. In other areas we have the following: hints.ai_flags = AI_CANONNAME; hints.ai_family = AF_INET; [ Not saying that what you have is incorrect, just questioning if you need to specify the socket type and protocol ] -Chris. From Alan.Bateman at oracle.com Wed Feb 7 17:48:08 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 7 Feb 2018 17:48:08 +0000 Subject: JDK-8080990: libdt_socket/socket_md.c(202) : warning C4996: 'gethostbyname': Use getaddrinfo() or GetAddrInfoW() In-Reply-To: <624B1D02-366B-4FF0-92F4-757ED64D356F@oracle.com> References: <5A6A2611.4030003@oracle.com> <5A7331D9.9000800@oracle.com> <624B1D02-366B-4FF0-92F4-757ED64D356F@oracle.com> Message-ID: On 07/02/2018 17:31, Chris Hegarty wrote: > Gary, > >> http://cr.openjdk.java.net/%7Egadams/8080990/webrev.02/ > I think the replacement of WSASendDisconnect with > shutdown(SD_SEND) should be fine. I do note that there > is another usage of WSASendDisconnect in > java.base/windows/native/libnet/net_util_md.c. > > [ Maybe you want to separate out the changes in java.base > ( NIO and NET ) from the serviceability changes? Up to > you. ] > Is NIO SocketDispatcher change needed in this patch? I would prefer if we could separate this was WSASendDisconnect was the semantics that we need. -Alan From gary.adams at oracle.com Wed Feb 7 17:55:38 2018 From: gary.adams at oracle.com (gary.adams at oracle.com) Date: Wed, 7 Feb 2018 12:55:38 -0500 Subject: JDK-8080990: libdt_socket/socket_md.c(202) : warning C4996: 'gethostbyname': Use getaddrinfo() or GetAddrInfoW() In-Reply-To: <624B1D02-366B-4FF0-92F4-757ED64D356F@oracle.com> References: <5A6A2611.4030003@oracle.com> <5A7331D9.9000800@oracle.com> <624B1D02-366B-4FF0-92F4-757ED64D356F@oracle.com> Message-ID: On 2/7/18 12:31 PM, Chris Hegarty wrote: > Gary, > >> http://cr.openjdk.java.net/%7Egadams/8080990/webrev.02/ > I think the replacement of WSASendDisconnect with > shutdown(SD_SEND) should be fine. I do note that there > is another usage of WSASendDisconnect in > java.base/windows/native/libnet/net_util_md.c. Thanks for the reference. I'm not sure how that one slipped by unnoticed. > > [ Maybe you want to separate out the changes in java.base > ( NIO and NET ) from the serviceability changes? Up to > you. ] Actually, this change? is about removing the compilation flag globally in flags.m4 and any sources that need to be updated to replace the deprecated functions. > > Curious about the specific hints you have chosen to use. > In other areas we have the following: > hints.ai_flags = AI_CANONNAME; > hints.ai_family = AF_INET; > > [ Not saying that what you have is incorrect, just questioning > if you need to specify the socket type and protocol ] One of the benefits of getaddrinfo is that it can return a list of addresses from the name service. In the places we were using gethostbyname we were looking for just one ipv4 address. By adding additional constraints on the supplied hints, it can help reduce the list of returned addresses. > > -Chris. From gary.adams at oracle.com Wed Feb 7 18:02:08 2018 From: gary.adams at oracle.com (gary.adams at oracle.com) Date: Wed, 7 Feb 2018 13:02:08 -0500 Subject: JDK-8080990: libdt_socket/socket_md.c(202) : warning C4996: 'gethostbyname': Use getaddrinfo() or GetAddrInfoW() In-Reply-To: References: <5A6A2611.4030003@oracle.com> <5A7331D9.9000800@oracle.com> <624B1D02-366B-4FF0-92F4-757ED64D356F@oracle.com> Message-ID: On 2/7/18 12:48 PM, Alan Bateman wrote: > On 07/02/2018 17:31, Chris Hegarty wrote: >> Gary, >> >>> http://cr.openjdk.java.net/%7Egadams/8080990/webrev.02/ >> I think the replacement of WSASendDisconnect with >> shutdown(SD_SEND) should be fine. I do note that there >> is another usage of WSASendDisconnect in >> java.base/windows/native/libnet/net_util_md.c. >> >> [ Maybe you want to separate out the changes in java.base >> ( NIO and NET ) from the serviceability changes? Up to >> you. ] >> > Is NIO SocketDispatcher change needed in this patch? I would prefer if > we could separate this was WSASendDisconnect was the semantics that we > need. > > -Alan Yes, WSASendDisconnect is deprecated in vs2013. As far as I know "shutdown(fd, SD_SEND)" prevents further outgoing writes and there was no final payload to send. I have not seen any failures in the java/nio tests. From alexey.menkov at oracle.com Wed Feb 7 18:47:31 2018 From: alexey.menkov at oracle.com (Alex Menkov) Date: Wed, 7 Feb 2018 10:47:31 -0800 Subject: 8196977: add test fo JDK-8161605 Message-ID: <6c66d7ca-7ba9-ffd2-5db3-5c582de39b79@oracle.com> Pushing fix for JDK-8161605, I forgot to add the test. Please review the correction. jira: https://bugs.openjdk.java.net/browse/JDK-8196977 webrev: http://cr.openjdk.java.net/~amenkov/forgotten_test/webrev/ --alex From chris.plummer at oracle.com Wed Feb 7 19:39:16 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 7 Feb 2018 11:39:16 -0800 Subject: RFR: JDK-8031445: Attach on windows can fail with java.io.IOException: All pipe instances are busy In-Reply-To: References: Message-ID: <2f615a45-b8da-912f-2b8c-b631303532c1@oracle.com> Hi Gary, I don't think you intended to include the ProblemList.txt changes in your webrev. I think your changes address the "java.io.IOException: CreateNamedPipe failed" failures if a name collision is the cause. This failure mode was extremely rare (only 3 sightings), and if due to a collision, a single retry should suffice in making it not appear again in our lifetime. However, I don't think this addresses the "java.io.IOException: All pipe instances are busy" issue, which seems to the more common failures mode, although also very rare. Have you looked into its potential cause? thanks, Chris On 2/7/18 8:51 AM, gary.adams at oracle.com wrote: > The IOException that is observed when creating a new named pipe > when the pipe already exists and is in use, recommends to retry > the operation later. Since we are already using a random number > to generate a unique pipe name, it makes sense to simply > retry the operation with a new pipe name. > > Here is a proposed fix. Testing in progress. > > ? Issue: https://bugs.openjdk.java.net/browse/JDK-8031445 > ? Webrev: http://cr.openjdk.java.net/~gadams/8031445/ > > From serguei.spitsyn at oracle.com Wed Feb 7 20:25:36 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 7 Feb 2018 12:25:36 -0800 Subject: RFR 8193150: Create a jtreg version of the test from JDK-8187143 In-Reply-To: <5A74A169.5020604@oracle.com> References: <5A74A169.5020604@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From gary.adams at oracle.com Wed Feb 7 20:34:17 2018 From: gary.adams at oracle.com (gary.adams at oracle.com) Date: Wed, 7 Feb 2018 15:34:17 -0500 Subject: RFR: JDK-8031445: Attach on windows can fail with java.io.IOException: All pipe instances are busy In-Reply-To: <4eb6ff53-0b9a-e4fb-60d1-1a5e0b82c10c@oracle.com> References: <4eb6ff53-0b9a-e4fb-60d1-1a5e0b82c10c@oracle.com> Message-ID: On 2/7/18 3:19 PM, gary.adams at oracle.com wrote: > Hi Gary, > > I don't think you intended to include the ProblemList.txt changes in > your webrev. You are right. I was also looking at JDK-8057732 in the same workspace. I believe there may have been a windows-x86 issue that may no longer be an issue. > > I think your changes address the "java.io.IOException: CreateNamedPipe > failed" failures if a name collision is the cause. This failure mode was > extremely rare (only 3 sightings), and if due to a collision, a single > retry should suffice in making it not appear again in our lifetime. > However, I don't think this addresses the "java.io.IOException: All pipe > instances are busy" issue, which seems to the more common failures mode, > although also very rare. Have you looked into its potential cause? Unfortunately, we no longer have the stack traces from the earlier test failures. The one stack trace we do have comes from this same native call to createNamedPipe. I have not been able to reproduce any of the original reported errors, yet. If this is a question of a heavily loaded system contending for a limit number of named pipes, the retry should address a number of those race conditions. We could also introduce a delay before the retry in case an older process is exiting and not getting enough cycles to complete. Since we're talking about attach operations, I don't think we'll see this issue failing in real life situations. > thanks, > > Chris > > On 2/7/18 8:51 AM,gary.adams at oracle.com > wrote: > >/The IOException that is observed when creating a new named pipe />/when the pipe already exists and is in use, recommends to retry />/the operation later. Since we are already using a random number />/to generate a unique pipe name, it makes sense to simply />/retry the operation with a new pipe name. />//>/Here is a proposed fix. Testing in progress. />//>/? Issue: https://bugs.openjdk.java.net/browse/JDK-8031445 />/? Webrev: http://cr.openjdk.java.net/~gadams/8031445/ > />//>// -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Wed Feb 7 20:39:56 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 7 Feb 2018 12:39:56 -0800 Subject: 8196977: add test fo JDK-8161605 In-Reply-To: <6c66d7ca-7ba9-ffd2-5db3-5c582de39b79@oracle.com> References: <6c66d7ca-7ba9-ffd2-5db3-5c582de39b79@oracle.com> Message-ID: <7f98e984-b23c-38b0-23b3-59fc7f5b7481@oracle.com> Hi Alex, Looks good. As the test was already reviewed should the 'trivial fix' rule apply here? One reviewer would be enough then. Or just add reviewers from the 8161605 to this fix as well. Thanks, Serguei On 2/7/18 10:47, Alex Menkov wrote: > Pushing fix for JDK-8161605, I forgot to add the test. > Please review the correction. > > jira: https://bugs.openjdk.java.net/browse/JDK-8196977 > webrev: http://cr.openjdk.java.net/~amenkov/forgotten_test/webrev/ > > --alex From erik.gahlin at oracle.com Wed Feb 7 20:47:29 2018 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Wed, 7 Feb 2018 21:47:29 +0100 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: References: Message-ID: <5A7B65E1.5050003@oracle.com> Hi Robin, I can sponsor this. I wonder if we should change the name of the event to "Shutdown" instead? It will give us flexibility to add other shutdown related fields in the future. Could you change the label and field to "Status Code" and statusCode. Event fields should have headline-style capitalization and statusCode allows us to add other status related information in the future. Thanks Erik > Hi all, > > Please review the following change that adds an event-based tracing > event that is generated when the VM shuts down. The intent is to > capture shutdowns that occur after the VM has been properly > initialized (as initialization problems would most likely mean that > the tracing framework hasn?t been properly started either). > > Issue: https://bugs.openjdk.java.net/browse/JDK-8041626 > Webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00/ > > Testing: hs-tier1,hs-tier2,jdk-tier1,jdk-tier2 > > Best regards, > Robin -------------- next part -------------- An HTML attachment was scrubbed... URL: From robin.westberg at oracle.com Wed Feb 7 20:52:13 2018 From: robin.westberg at oracle.com (Robin Westberg) Date: Wed, 7 Feb 2018 21:52:13 +0100 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: <22b0955b-e1dc-a3b7-ab1b-8e4f0ec49670@oracle.com> References: <22b0955b-e1dc-a3b7-ab1b-8e4f0ec49670@oracle.com> Message-ID: <9F1116D1-F61C-47AA-8137-21BDCDA84ADC@oracle.com> Hi Alan, > On 7 Feb 2018, at 16:30, Alan Bateman wrote: > > On 07/02/2018 15:18, Robin Westberg wrote: >> Hi all, >> >> Please review the following change that adds an event-based tracing event that is generated when the VM shuts down. The intent is to capture shutdowns that occur after the VM has been properly initialized (as initialization problems would most likely mean that the tracing framework hasn?t been properly started either). >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-8041626 >> Webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00/ >> Testing: hs-tier1,hs-tier2,jdk-tier1,jdk-tier2 >> > Can you elaborate a bit on why this isn't in JVM_Halt? Is this partially to help with cases where the shutdown hooks or finalizers run at exit cause issues? Sure, the main problem I had with that approach is actually that the tracing framework shuts down from a shutdown hook, so events generated in JVM_Halt will be lost. And I couldn?t think of any good way to capture the exit code and proper stack trace from inside that shutdown hook, but I could perhaps explore that option a bit further. Best regards, Robin From chris.plummer at oracle.com Wed Feb 7 21:25:19 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 7 Feb 2018 13:25:19 -0800 Subject: RFR 8193150: Create a jtreg version of the test from JDK-8187143 In-Reply-To: References: <5A74A169.5020604@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From parvathi.somashekar at oracle.com Wed Feb 7 22:04:02 2018 From: parvathi.somashekar at oracle.com (Paru Somashekar) Date: Wed, 07 Feb 2018 14:04:02 -0800 Subject: RFR 8193150: Create a jtreg version of the test from JDK-8187143 In-Reply-To: References: <5A74A169.5020604@oracle.com> Message-ID: <5A7B77D2.3030408@oracle.com> Thanks for the review Serguei. thanks, Paru. On 2/7/18, 12:25 PM, serguei.spitsyn at oracle.com wrote: > Hi Paru, > > It looks good. > Thank you a lot for taking care about this! > > Could we get at least one more review from the Serviceability team on > this new test? > > Thanks, > Serguei > > > On 2/2/18 09:35, Paru Somashekar wrote: >> Hi, >> >> Please review the fix for JDK-8193150. >> >> The fix introduces a new jtreg test, NashornPopFrameTest. It is based >> on the original test from JDK-8187143 >> that was provided >> by the customer. >> >> Bug : https://bugs.openjdk.java.net/browse/JDK-8193150 >> >> Webrev : http://cr.openjdk.java.net/~psomashe/8193150/webrev/ >> >> >> Here is a brief description of what the test does :- >> >> * The debuggee, creates and uses a Nashorn engine to evaluate a >> simple script. >> >> * The debugger tries to set a breakpoint in Nashorn?s internal >> DEBUGGER method. >> * When the breakpoint is reached, it looks for stack frame whose >> method's declaring type name starts with (nashorn dynamically >> generated classes) ?jdk.nashorn.internal.scripts.Script$?. >> * It then pops stack frames using the ThreadReference.popFrames() >> call, up to and including the above stackframe. >> * The execution of the debuggee application is resumed after the >> needed frames have been popped. >> >> This test is included in the ProblemList as it fails under some >> circumstances (bug JDK-8187143) >> . Is always passes >> with the -Xint flag however always fails with -Xcomp. It fails >> intermittently with the -Xmixed (default). >> >> thanks, >> Paru. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hohensee at amazon.com Wed Feb 7 22:17:43 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Wed, 7 Feb 2018 22:17:43 +0000 Subject: RFR (S): 8195115: G1 Old Gen MemoryPool CollectionUsage.used values don't reflect mixed GC results In-Reply-To: References: <3A67BF1B-CE35-4759-B5B4-959C24020A45@amazon.com> <22ceea57-7d27-8350-4457-21f765cb3d0f@oracle.com> <878D6678-84CB-471E-B126-AF14B1BC8EB9@amazon.com> <727f9bcb-9404-df32-917f-70aca6e31cb6@oracle.com> <4BA32239-D645-4B1A-A58E-502258C8FFB7@amazon.com> <997a766f-9f4b-f192-ae87-03f4c8d2666d@oracle.com> <64793ff3-4805-79f9-4e72-6e469cd511dc@oracle.com> <15754d34-f3c7-6431-7abf-05214bd6b9d2@oracle.com> <93417506-8e12-6ad3-690a-439e36719652@oracle.com> <99C937B6-F67B-4F4A-ACC8-444B5F83932B@amazon.com> Message-ID: <3922628B-81DE-4904-AEFE-F4163EB1E655@amazon.com> I?ve filed https://bugs.openjdk.java.net/browse/JDK-8196989 : Revamp G1 JMX MemoryPool and GarbageCollector MXBean definitions and the corresponding CSR https://bugs.openjdk.java.net/browse/JDK-8196991. Would you please comment on the CSR, and on the original CSR https://bugs.openjdk.java.net/browse/JDK-8196719? Thanks, Paul On 2/2/18, 1:20 PM, "hotspot-gc-dev on behalf of Hohensee, Paul" wrote: And, can a get a reviewer or reviewers for the CSR? Thanks, Paul On 2/2/18, 1:14 PM, "Hohensee, Paul" wrote: +hotspot-gc-use. I?ve filed a CSR for the current patch, see https://bugs.openjdk.java.net/browse/JDK-8196719. Here?s the argument in favor. It?s possible that there are JDK8 users that rely on current G1 old gen CollectionUsage behavior, but imo it?s unlikely because it?s of little use. Perhaps Kirk and others with operational experience can weigh in. Let?s think about use cases. G1 full GC?s happen rarely and only under severe pressure, so when they do external reaction is pretty much limited to reducing load so the JVM can get back to a usable steady state, or just restarting the JVM. Old gen CollectionUsage is zero until a full GC occurs, after which its value includes both long-lived objects and any transient data that was in eden and the survivor space. That value doesn?t tell you anything about long term old gen occupancy or survivor size because it lumps them all together. So, it isn?t a useful metric, nor will it be after any subsequent full GCs. The only information it provides is on the first zero to non-zero transition, which just tells you that the JVM is/was in trouble. Further, the effect of the runup to a full GC is SLA violations, which are noticed before the full GC happens, so detecting the first full GC is confirmation, not prediction. Conclusion: G1 old gen CollectionUsage is unlikely to be in use in its current form, so changing its definition to something usable is low risk. ?G1 Old Space? is fine, as is ?G1 Archive Space?. Are you ok with the G1 archive space overlapping the G1 old space? Should we add an archive space to the other collectors? If so, how would it be defined and would having it overlap with the old generation as a live prefix be ok? "G1 Young Generation" is the currently young+mixed collector. You?re right, if one is iterating over all collectors, there will be redundancy if we keep the old ones. I?m usually leery of introducing a flag such as UseG1LegacyMXBeans (I changed the name, since all the interfaces are MXBeans, hope that?s ok) which must be either indefinitely maintained, or go through a deprecation cycle. I don?t see a way out of the ?iterate over all collectors? problem without it though. Paul On 1/31/18, 3:42 AM, "Erik Helin" wrote: On 01/31/2018 02:30 AM, Hohensee, Paul wrote: > It?s true that my patch doesn?t completely solve the larger problem, but it fixes the most immediately important part of it, particularly for JDK8 where current expected behavior is entrenched. Yes, your patch fixes part of the problem, but as I said, can potentially lead to more confusion. I'm not sure that doing this behavioral change for a public API in an JDK 8 update release is the right thing. There are likely users that rely on the memory pool "G1 Old Gen" only being updated by a full collection (even though that behavior is not correct), those uses will encounter a new behavior in an update release with your patch. The good thing is that we have very experienced engineers participating in the CSR process that have much more experience than I have in evaluating the impact of behavioral changes such as this one. Would you please file a CSR request for your patch to get their opinion? See https://wiki.openjdk.java.net/display/csr/Main for more details about CSR. On 01/31/2018 02:30 AM, Hohensee, Paul wrote: If we?re going to fix the larger problem, imo we should file another bug/rfe to do it. I?d be happy to fix that one too, once we have a spec. > > What do you think of my suggestions? To summarize: > > - Keep the existing memory pools and add humongous and archive pools. > - Make the archive pool part of the old gen, and generalize it to all collectors. > - Split humongous regions off from the old gen pool into their own pool. The old gen and humongous pools are disjoint region sets. > - Keep the existing ?G1 Young Generation? and ?G1 Old Generation? collectors and their associated memory pools (net of this patch). We add the humongous pool to them. > - Add ?G1 Full? as an alias of the existing ?G1 Old Generation? collector. > - Add the ?G1 Young?, ?G1 Mixed? and ?G1 Concurrent Cycle? collectors. > - Set the G1 old gen memory pool max size to ?Xmx, the archive space max size to whatever it is, and the rest of the G1 memory pool max sizes to -1 == undefined, as now. > > The resulting memory pools: > > ?G1 Eden Space? > ?G1 Survivor Space? > ?G1 Old Gen? > ?G1 Humongous Space? > ?Archive Space? The "Space" suffix is unfortunate, but acceptable. I'm least happy about the "Gen" suffix for the "G1 Old Gen", since G1's old regions differ from a generation in the traditional sense as applied to e.g. Serial, Parallel and CMS. I would be more happy to use a consistent naming scheme in the form of "G1 Old Space" (having only one pool ending "Gen" begs the question how it differs from the others ending in "Space"). Again, we could introduce a flag such as -XX:+UseG1LegacyPoolsAndBeans for those that really wants the old names. "Archive Space" should be named "G1 Archive Space" since it differs in implementation from the other collectors. It would be unfortunate if users thought they could change collector and the "Archive Space" memory pool would keep the same behavior. > The resulting collectors and their memory pools: > > ?G1 Young Generation? (the existing young/mixed collector), ?G1 Old Generation?/?G1 Full?, ?G1 Mixed? > - ?G1 Eden Space? > - ?G1 Survivor Space? > - ?G1 Old Gen? > - ?G1 Humongous Space? > ?G1 Young? > - ?G1 Eden Space? > - ?G1 Survivor Space? > - ?G1 Humongous Space? > ?G1 Concurrent Cycle? > - ?G1 Old Gen? > - ?G1 Humongous Space? > > I?m not religious about the names, but I like my suggestions. :) I think it will be confusing for users to have both "G1 Old Generation" and "G1 Full", particularly for tools iterating over all GarbageCollectorMXBeans. There is no way to indicate that a GarbageCollectorMXBeans is an alias of another GarbageCollectorMXBean (I thought about such a solution as well). I guess I don't follow what the GarbageCollectorMXBean "G1 Young Generation" is meant to represent? > The significant addition to my previous email, and an incompatible change, is splitting humongous regions off from the old gen pool. This means that apps that specifically monitor old gen occupancy will no longer see humongous regions. Monitoring apps that just add up info about all a collector?s pools won?t see a difference. I may be corrected (by Kirk, perhaps), but imo it?s not as bad a compatibility issue as one might think, because the type of app that uses a lot of humongous regions isn?t all that common. E.g., apps that cache data in the heap in the form of large compressed arrays, and apps with large hashmap bucket list arrays. The heaps such apps use are very often large enough to use 32mb regions, hence need really big objects to actually allocate humongous regions. So why not enable backwards compatibility by allowing a user to set the flag -XX:+UseG1LegacyPoolsAndBeans? It is not that cumbersome for us to maintain the current definition of memory pools and collectors. Such a flag allows us to start over and do this right and a user who relies on the current behavior can get that by just setting a flag. Doing such a change in a major release also allows us to clearly highlight the change in the release notes (users are more prepared for larger changes in a major release and that they might have to add flags to keep old behavior). It is not uncommon for memory pools to change in major releases. The perm gen pool was removed in JDK 8, the default pools changed when Parallel Old became default old collector way back in JDK 7 and changed again when G1 became the default collector in JDK 9. Thanks, Erik > Thanks, > > Paul > > On 1/30/18, 5:51 AM, "Erik Helin" wrote: > > On 01/30/2018 03:07 AM, Hohensee, Paul wrote: > > That?s one reviewer who?s ok with a short term patch. Anyone else? And, > > any reviewers for said short term patch? :) > > Well, the patch is not really complete as it is. The problem is the > definitions of the MemoryPoolMXBeans and GarbageCollectorMXBeans, which, > as I tried to hint at in my first email, is a mess for G1. The names and > implementations of these MemoryPoolMXBeans and GarbageCollectionMXBeans > for G1 are very old, G1 has changed a lot since those were implemented > (hence my suggestion for finally fixing this). > > The issue with your patch is that the MemoryPoolMXBean named "G1 Old > Gen" consists of both old and humongous regions (it will also include > archive regions). Old regions can be collected by mixed, concurrent and > full collections. Humongous regions can be collected by young, mixed or > full collections and the concurrent cycle. Archive regions will never be > collected. Your patch will update the pool in the case of a mixed > collection collecting old regions or humongous regions, but misses the > following cases: > - a young collection collecting humongous regions > - a concurrent cycle collecting humongous regions > - a concurrent cycle collecting old regions > > Unfortunately I could not come up with a good way to solve the above > without re-designing the pools. I'm not sure about accepting your patch > as is, since it might cause even more confusion for users compared to > the current (already confusing) situation. > > One idea we have discussed is to implement the re-design but also add a > flag, -XX:+UseG1LegacyPoolsAndBeans (false by default), to allow for a > smoother transition. Would that solution work for you? > > Thanks, > Erik > > > Thanks, > > > > Paul > > > > *From: *mandy chung > > *Organization: *Oracle Corporation > > *Date: *Monday, January 29, 2018 at 1:41 PM > > *To: *"Hohensee, Paul" > > *Cc: *"serviceability-dev at openjdk.java.net" > > , "hotspot-gc-dev at openjdk.java.net" > > > > *Subject: *Re: RFR (S): 8195115: G1 Old Gen MemoryPool > > CollectionUsage.used values don't reflect mixed GC results > > > > I created JDK-8196362 to look into whether it makes sense to provide > > some categorization to differentiate eden space vs the heap space for > > long-lived objects. > > > > W.r.t. to JDK-8195115, I have to defer to GC team to comment on the > > mixed collection update. If they are okay, I have no objection to > > implement a short-term fix and do the proper G1 memory pools as a > > separate patch. > > > > Mandy > > > > On 1/29/18 1:02 PM, Hohensee, Paul wrote: > > > > We don?t use getType, and you guessed correctly: we use the memory > > pool name as an indicator of the specific characteristics of a > > memory pool, in particular eden. > > > > What we want is an indication of long term heap occupancy. We > > calculate it using CollectionUsage for non-eden heap memory pools, > > regardless of collector. We don?t use JMX notification, rather we > > periodically poll CollectionUsage for memory pools with names that > > contain ?Old?, ?Tenured?, or ?Survivor?. We get the memory pools > > from the GarbageCollectorMXBeans (we don?t care what the collector > > names are). For the named memory pools, we sum CollectionUsage.used > > and divide by the sum of CollectionUsage.max to get a long term heap > > occupancy percentage. We don?t want to include eden because it?s > > really just an allocation buffer and not part of the storage for > > long-lived objects. I suppose we could use a negative test instead > > by using all memory pools with names that don?t include ?Eden?. > > > > The bug is that the ?G1 Old Gen? memory pool isn?t being updated > > when the ?G1 Young Generation? collector runs a mixed collection. As > > far as JMX is concerned, that collector only knows about eden and > > the survivor space. The patch adds the old gen to the memory pools > > it knows about and has mixed collections update the old gen?s > > CollectionUsage. > > > > I managed to get a submit repo run to succeed last week and it found > > a problem. I?ve uploaded a new webrev that fixes the failure of the > > jtreg test TestMemoryMXBeansAndPoolsPresence.java due to the young > > gen collector being expected to know only about eden and the > > survivor space. > > > > http://cr.openjdk.java.net/~phh/8195115/webrev.hs.01/ > > > > > > Waiting on the submit repo to come back with a result on it. > > > > Thanks, > > > > Paul > > > > *From: *mandy chung > > > > *Organization: *Oracle Corporation > > *Date: *Monday, January 29, 2018 at 10:52 AM > > *To: *"Hohensee, Paul" > > , Erik Helin > > , David Holmes > > > > *Cc: *"serviceability-dev at openjdk.java.net" > > > > > > , > > "hotspot-gc-dev at openjdk.java.net" > > > > > > > > *Subject: *Re: RFR (S): 8195115: G1 Old Gen MemoryPool > > CollectionUsage.used values don't reflect mixed GC results > > > > On 1/29/18 10:35 AM, mandy chung wrote: > > > > Thanks for the reply Paul. Try to understand a little more on > > the specific from gc-specific memory pool you depend on. > > > > On 1/29/18 8:27 AM, Hohensee, Paul wrote: > > > > A name change would affect Amazon?s heap monitoring, and > > thus I expect it would affect other users as well. > > > > As long as there are gc-specific memory pools, we?re going > > to need to be able to identify them, and right now that?s > > done via name. > > > > > > MemoryPoolMXBean::getType returns "heap" memory type for > > GC-specific memory pools. Are you using this method? Do you > > use the name to build in specific characteristic of a memory > > pool (e.g. eden vs old gen)? > > > > > > > > > > All the mxbeans are identified by name, so that?s a general > > design principle. The only way I can think of to get rid of > > name dependency would be to figure out what abstract metrics > > users want to monitor and implement them for all collectors. > > HeapUsage (instantaneous occupancy) is one, CollectionUsage > > (long-lived occupancy) is another, both of these for the > > entire heap, not just particular memory pools. > > > > > > The sum of HeapUsage and CollectionUsage of all heap memory > > pools was expected to give an incorrect approximation for the > > entire heap usage. Are you seeing issue/bug with the sum result? > > > > > > typo: s/an incorrect approximation/an approximation. > > > > Mandy > > > > > > > > Mandy > > > > > > > > That said, imo there will always be a demand for the ability > > to get collector and memory pool specific details, so I > > don?t see a way to get around providing named entities. > > > > Paul > > > > *From: *mandy chung > > > > *Organization: *Oracle Corporation > > *Date: *Friday, January 26, 2018 at 2:38 PM > > *To: *"Hohensee, Paul" > > , Erik Helin > > , > > David Holmes > > > > *Cc: *"serviceability-dev at openjdk.java.net" > > > > > > , > > "hotspot-gc-dev at openjdk.java.net" > > > > > > > > *Subject: *Re: RFR (S): 8195115: G1 Old Gen MemoryPool > > CollectionUsage.used values don't reflect mixed GC results > > > > On 1/25/18 1:04 PM, Hohensee, Paul wrote: > > > > > > > The JMX API spec doesn?t specify what the memory pool or > > garbage > collector names are, but the current names are > > de-facto part of the > API, so if we change the existing > > ones, imo a CSR should be filed. > > > > The names are implementation details but I can see how an application > > > > might be impacted if they depend on it. CSR approval is not strictly > > > > necessary while I think filing one to document the change would be > > > > good. > > > > Does the name change impact any application you know of? I'm trying to > > > > understand if any improvement to API is needed so that applications > > > > don't need to depend on the names. > > > > > > Mandy > > > > > > > > > > > > > > > > > > From gary.adams at oracle.com Wed Feb 7 22:24:49 2018 From: gary.adams at oracle.com (gary.adams at oracle.com) Date: Wed, 7 Feb 2018 17:24:49 -0500 Subject: JDK-8080990: libdt_socket/socket_md.c(202) : warning C4996: 'gethostbyname': Use getaddrinfo() or GetAddrInfoW() In-Reply-To: References: <5A6A2611.4030003@oracle.com> <5A7331D9.9000800@oracle.com> <624B1D02-366B-4FF0-92F4-757ED64D356F@oracle.com> Message-ID: <84ca29de-90df-21d1-0130-220cad2b7200@oracle.com> On 2/7/18 12:55 PM, gary.adams at oracle.com wrote: > On 2/7/18 12:31 PM, Chris Hegarty wrote: >> Gary, >> >>> http://cr.openjdk.java.net/%7Egadams/8080990/webrev.02/ >> I think the replacement of WSASendDisconnect with >> shutdown(SD_SEND) should be fine. I do note that there >> is another usage of WSASendDisconnect in >> java.base/windows/native/libnet/net_util_md.c. > Thanks for the reference. > I'm not sure how that one slipped by unnoticed. I found in make/lib/NetworkingLibraries.gmk the BUILD_LIBNET target includes a number of specific disabled warnings. In particular, 4996 which suppresses all deprecated function warnings. I think I'd like to get the current changes pushed to remove the specific WINSOCK_DEPRECATED_NO_WARNINGS and then file another bug to come back and look at selective makefiles that use a blanket 4996 suppression of deprecated warnings. >> >> [ Maybe you want to separate out the changes in java.base >> ( NIO and NET ) from the serviceability changes? Up to >> you. ] > Actually, this change? is about removing the compilation flag globally > in flags.m4 and any sources that need to be updated > to replace the deprecated functions. >> >> Curious about the specific hints you have chosen to use. >> In other areas we have the following: >> ?? hints.ai_flags = AI_CANONNAME; >> ?? hints.ai_family = AF_INET; >> >> [ Not saying that what you have is incorrect, just questioning >> ? if you need to specify the socket type and protocol ] > One of the benefits of getaddrinfo is that it can return a list of > addresses from the name service. In the places we were using > gethostbyname we were looking for just one ipv4 address. > By adding additional constraints on the supplied hints, it can help > reduce the list of returned addresses. > >> >> -Chris. > > From parvathi.somashekar at oracle.com Wed Feb 7 22:30:53 2018 From: parvathi.somashekar at oracle.com (Paru Somashekar) Date: Wed, 07 Feb 2018 14:30:53 -0800 Subject: RFR 8193150: Create a jtreg version of the test from JDK-8187143 In-Reply-To: References: <5A74A169.5020604@oracle.com> Message-ID: <5A7B7E1D.3090304@oracle.com> Thanks for the review Chris, comments inline.. On 2/7/18, 1:25 PM, Chris Plummer wrote: > Hi Paru, > > Thanks for writing this test. It will make figuring out JDK-8187143 > a lot easier. > > Overall the test looks good. My main concern is the lack of comments. > It makes it hard for the reader to understand the flow of the test and > to understand some of the less obvious actions being performed. That > is especially true for this test, which is doing some really bizarre > stuff. Some of this you cover in our RFR summary below, but that info > really needs to be in the test itself, along with additional comments. > For example, what does pauseAtDebugger() do? It looks to me like it > sets a breakpoint on the java script debugger that has a class name > that ends with ScriptRuntime and the method name is DEBUGGER(). But I > only figured that out after staring at the code for a while, and > recalling a conversation we had a few weeks ago. It's also not > described why this is being done. I shall add more comments into the test code to make it easier to understand. However while I understand what is being done ( e.g. adding breakpoint on Nashorn's internal DEBUGGER method) - unfortunately I too am not sure "why" it is being done. I do not have insight on what the author ( bug reporter) was trying to do.. > > Here's another example: > > 126 while (!vmDisconnected) { > 127 try { > 128 Thread.sleep(100); > 129 } catch (InterruptedException ee) { > 130 } > 131 } > > I seem to also recall us discussing the need for this, but can no > longer recall the reason The above loop is there to make the debugger continue to run until it receives a VMdisconnect event either because the Debuggee crashed / got exception / finished. I shall add a comment for this as well. > > Another example is findScriptFrame(). What is the significance of the > frame whose class name starts with > jdk.nashorn.internal.scripts.Script$? I think I understand (it's the > generated java method for the javascript you setup in > ScriptDebuggee.doit()), but I can only figure this out based on > earlier conversations we had and your RFR comments below. I'd expect > the uninformed reader to spend a long time coming the same conclusion. Again, I am not clear on the significance of popping frames until this method which is a generated java method for javascript in the debuggee. I could consult with the author and add those comments as well. > > The following are just a few minor things I noticed: > > Copyright should only have 2018. > > 57 } catch (Exception npe) { > > Probably best to call it "ex" instead of "npe". > > 85 NashornPopFrameTest bbcT = new NashornPopFrameTest(args); > > It's unclear to me where the name "bbcT" comes from. I shall change that to npft or something like that. > > 134 if (failReason != null) { > 135 failure(failReason); > 136 } > > You have two classes that declare "String failReason" which makes it a > bit confusing to track which one the reader is dealing with. Also, the > NashornPopFrameTest version is initialized to non-null, so doesn't > that make the test always fail when the above code is executed? Even though failReason is initialized, it is reset if the expected breakpoint is reached. And when the breakpoint is reached, it checks the Debuggee version of the field, if it is non null, then this field is set to the non null value - else it is set to null. I shall add some comments to make it less confusing. > > Is there a reason why ScriptDebuggee doesn't just put everything in > main() and not have a doit() method? No there isn't a specific reason. I noticed that other tests were doing it - like BreakpointTest and for consistency and clarity, i followed that pattern. thanks, Paru. > > thanks, > > Chris > > On 2/7/18 12:25 PM, serguei.spitsyn at oracle.com > wrote: >> Hi Paru, >> >> It looks good. >> Thank you a lot for taking care about this! >> >> Could we get at least one more review from the Serviceability team on >> this new test? >> >> Thanks, >> Serguei >> >> >> On 2/2/18 09:35, Paru Somashekar wrote: >>> Hi, >>> >>> Please review the fix for JDK-8193150. >>> >>> The fix introduces a new jtreg test, NashornPopFrameTest. It is >>> based on the original test from JDK-8187143 >>> that was provided >>> by the customer. >>> >>> Bug : https://bugs.openjdk.java.net/browse/JDK-8193150 >>> >>> Webrev : http://cr.openjdk.java.net/~psomashe/8193150/webrev/ >>> >>> >>> Here is a brief description of what the test does :- >>> >>> * The debuggee, creates and uses a Nashorn engine to evaluate a >>> simple script. >>> >>> * The debugger tries to set a breakpoint in Nashorn?s internal >>> DEBUGGER method. >>> * When the breakpoint is reached, it looks for stack frame whose >>> method's declaring type name starts with (nashorn dynamically >>> generated classes) ?jdk.nashorn.internal.scripts.Script$?. >>> * It then pops stack frames using the ThreadReference.popFrames() >>> call, up to and including the above stackframe. >>> * The execution of the debuggee application is resumed after the >>> needed frames have been popped. >>> >>> This test is included in the ProblemList as it fails under some >>> circumstances (bug JDK-8187143) >>> . Is always passes >>> with the -Xint flag however always fails with -Xcomp. It fails >>> intermittently with the -Xmixed (default). >>> >>> thanks, >>> Paru. >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Wed Feb 7 22:44:58 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 7 Feb 2018 14:44:58 -0800 Subject: Issues debugging java 9 from jdk 8 In-Reply-To: <69179b71-90c0-d921-dd8c-3b5800ce8f6d@jetbrains.com> References: <69179b71-90c0-d921-dd8c-3b5800ce8f6d@jetbrains.com> Message-ID: An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Wed Feb 7 23:06:37 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 7 Feb 2018 15:06:37 -0800 Subject: RFR 8193150: Create a jtreg version of the test from JDK-8187143 In-Reply-To: <5A7B7E1D.3090304@oracle.com> References: <5A74A169.5020604@oracle.com> <5A7B7E1D.3090304@oracle.com> Message-ID: <58bf290a-2c35-afb2-c195-8cfbc08d3702@oracle.com> An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Wed Feb 7 23:23:44 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 7 Feb 2018 15:23:44 -0800 Subject: RFR 8193150: Create a jtreg version of the test from JDK-8187143 In-Reply-To: <58bf290a-2c35-afb2-c195-8cfbc08d3702@oracle.com> References: <5A74A169.5020604@oracle.com> <5A7B7E1D.3090304@oracle.com> <58bf290a-2c35-afb2-c195-8cfbc08d3702@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From gary.adams at oracle.com Wed Feb 7 23:39:46 2018 From: gary.adams at oracle.com (gary.adams at oracle.com) Date: Wed, 7 Feb 2018 18:39:46 -0500 Subject: JDK-8080990: libdt_socket/socket_md.c(202) : warning C4996: 'gethostbyname': Use getaddrinfo() or GetAddrInfoW() In-Reply-To: <5A78AD3E.9090503@oracle.com> References: <5A6A2611.4030003@oracle.com> <5A731457.1040502@oracle.com> <5A78AD3E.9090503@oracle.com> Message-ID: <0d39f31d-9e09-80f6-d6fd-98346edc9f06@oracle.com> A fresh webrev rebased with latest jdk/jdk repos: ?http://cr.openjdk.java.net/~gadams/8080990/webrev.03/index.html If there are no more comments, I'll check in locally with these reviewers ? clanger,chegar,erikj and cut patches on Thurs AM. On 2/5/18 2:15 PM, Gary Adams wrote: > One more to fix to cover the remaining test failures I was seeing. > > Previously, using inet_addr, there was a single -1 return that needed > to be checked. > Now that we're using inet_pton, there is a -1 and a 0 error return to > be considered. > > My preference would be to leave the dbgSysInetAddr name unchanged and > have the low level > check for inet_pton errors to simply return the -1 that was previously > checked. > > This specific issue is about the deprecation of library functions on > windows. I'd recommend not > upgrading the other platforms at this time until a complete update is > done for ipv6 support. > > I'll post a new webrev when I've retested using the updated inet_pton > error checking. > > ... >> Hi Gary, >> >> >> >/Here's a revised webrev />//>/http://cr.openjdk.java.net/~gadams/8080990/webrev.01/index.html >> />//>/Still testing ... />//>/Using shutdown() fixed problems reported by the />/java/nio/channelSocketChannel tests. / >> The fix looks good. I would think we should rename dbgsysInetAddr to dbgsysPToN and use inet_pton also for the Unix/Linux platforms. It would be the better choice there as well, though we still only support IPv4 in libdt_socket. >> >> >//>/I also noticed prior use of getaddrinfo for "localhost" was not calling />/freeaddrinfo. / >> Thanks for spotting/fixing that. >> >> Best regards >> Christoph >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Thu Feb 8 00:56:47 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 7 Feb 2018 16:56:47 -0800 Subject: RFR 8193150: Create a jtreg version of the test from JDK-8187143 In-Reply-To: References: <5A74A169.5020604@oracle.com> <5A7B7E1D.3090304@oracle.com> <58bf290a-2c35-afb2-c195-8cfbc08d3702@oracle.com> Message-ID: <5ceed45b-90fe-f57c-d5c6-a553ce047d33@oracle.com> An HTML attachment was scrubbed... URL: From parvathi.somashekar at oracle.com Thu Feb 8 02:55:28 2018 From: parvathi.somashekar at oracle.com (Paru Somashekar) Date: Wed, 07 Feb 2018 18:55:28 -0800 Subject: RFR 8193150: Create a jtreg version of the test from JDK-8187143 In-Reply-To: <5ceed45b-90fe-f57c-d5c6-a553ce047d33@oracle.com> References: <5A74A169.5020604@oracle.com> <5A7B7E1D.3090304@oracle.com> <58bf290a-2c35-afb2-c195-8cfbc08d3702@oracle.com> <5ceed45b-90fe-f57c-d5c6-a553ce047d33@oracle.com> Message-ID: <5A7BBC20.9070805@oracle.com> Hi Chris, Serguei, On 2/7/18, 4:56 PM, serguei.spitsyn at oracle.com wrote: > On 2/7/18 16:47, Chris Plummer wrote: >> On 2/7/18 3:23 PM, serguei.spitsyn at oracle.com >> wrote: >>> Hi Chris, >>> >>> >>> On 2/7/18 15:06, Chris Plummer wrote: >>>> Hi Paru, >>>> >>>> On 2/7/18 2:30 PM, Paru Somashekar wrote: >>>>> Thanks for the review Chris, comments inline.. >>>>> On 2/7/18, 1:25 PM, Chris Plummer wrote: >>>>>> Hi Paru, >>>>>> >>>>>> Thanks for writing this test. It will make figuring out >>>>>> JDK-8187143 a >>>>>> lot easier. >>>>>> >>>>>> Overall the test looks good. My main concern is the lack of >>>>>> comments. It makes it hard for the reader to understand the flow >>>>>> of the test and to understand some of the less obvious actions >>>>>> being performed. That is especially true for this test, which is >>>>>> doing some really bizarre stuff. Some of this you cover in our >>>>>> RFR summary below, but that info really needs to be in the test >>>>>> itself, along with additional comments. For example, what does >>>>>> pauseAtDebugger() do? It looks to me like it sets a breakpoint on >>>>>> the java script debugger that has a class name that ends with >>>>>> ScriptRuntime and the method name is DEBUGGER(). But I only >>>>>> figured that out after staring at the code for a while, and >>>>>> recalling a conversation we had a few weeks ago. It's also not >>>>>> described why this is being done. >>>>> I shall add more comments into the test code to make it easier to >>>>> understand. However while I understand what is being done ( e.g. >>>>> adding breakpoint on Nashorn's internal DEBUGGER method) - >>>>> unfortunately I too am not sure "why" it is being done. I do not >>>>> have insight on what the author ( bug reporter) was trying to do.. >>>> That's ok. They "why" is because this is a test case demonstrating >>>> a failure a user ran into. You might want to mention that also, >>>> although the @bug reference might enough. >>> >>> Agreed as this is my understanding too. >>> >>> >>>>>> >>>>>> Here's another example: >>>>>> >>>>>> 126 while (!vmDisconnected) { >>>>>> 127 try { >>>>>> 128 Thread.sleep(100); >>>>>> 129 } catch (InterruptedException ee) { >>>>>> 130 } >>>>>> 131 } >>>>>> >>>>>> I seem to also recall us discussing the need for this, but can no >>>>>> longer recall the reason >>>>> The above loop is there to make the debugger continue to run until >>>>> it receives a VMdisconnect event either because the Debuggee >>>>> crashed / got exception / finished. >>>>> I shall add a comment for this as well. >>>>>> >>>>>> Another example is findScriptFrame(). What is the significance of >>>>>> the frame whose class name starts with >>>>>> jdk.nashorn.internal.scripts.Script$? I think I understand (it's >>>>>> the generated java method for the javascript you setup in >>>>>> ScriptDebuggee.doit()), but I can only figure this out based on >>>>>> earlier conversations we had and your RFR comments below. I'd >>>>>> expect the uninformed reader to spend a long time coming the same >>>>>> conclusion. >>>>> Again, I am not clear on the significance of popping frames until >>>>> this method which is a generated java method for javascript in the >>>>> debuggee. I could consult with the author and add those comments >>>>> as well. >>>> This is just to recreate the situation the customer saw when >>>> running into the bug. We don't need to know the details of why they >>>> were doing what they did, only that it resulted in a bug being >>>> exposed. I'm mostly asking that you add comments that explain what >>>> the test is doing, but not worry so much about explaining the >>>> underlying reasons why the test was written in the first place >>>> (although that might be useful as part of an overall test summary >>>> comment at the top). >>> >>> Right. >>> Thank you for the suggestion! >>> I did not pay attention to it when pre-reviewed. >>> >>>>>> >>>>>> The following are just a few minor things I noticed: >>>>>> >>>>>> Copyright should only have 2018. >>>>>> >>>>>> 57 } catch (Exception npe) { >>>>>> >>>>>> Probably best to call it "ex" instead of "npe". >>>>>> >>>>>> 85 NashornPopFrameTest bbcT = new >>>>>> NashornPopFrameTest(args); >>>>>> >>>>>> It's unclear to me where the name "bbcT" comes from. >>>>> I shall change that to npft or something like that. >>>>>> >>>>>> 134 if (failReason != null) { >>>>>> 135 failure(failReason); >>>>>> 136 } >>>>>> >>>>>> You have two classes that declare "String failReason" which makes >>>>>> it a bit confusing to track which one the reader is dealing with. >>>>>> Also, the NashornPopFrameTest version is initialized to non-null, >>>>>> so doesn't that make the test always fail when the above code is >>>>>> executed? >>>>> Even though failReason is initialized, it is reset if the expected >>>>> breakpoint is reached. And when the breakpoint is reached, it >>>>> checks the Debuggee version of the field, if it is non null, then >>>>> this field is set to the non null value - else it is set to null. >>>>> I shall add some comments to make it less confusing. >>>>> >>>> So in the above check failReason has a double meaning (maybe even >>>> triple). It could be set to its original value, which means the >>>> breakpoint was never reached, or if the breakpoint is reached it is >>>> set to ScriptDebuggee.failReason, which basically represents the >>>> result of having called engine.eval(script). I think it would be >>>> clearer if you just had a static flag to indicate if the breakpoint >>>> was reached and just initialize failReason to null. >>> >>> The static flag does not work as the debuggee is in a different VM >>> process. >> Of course. Rookie mistake on my part. :) > > I knew it but had done the same mistake. :) > > >>>> Then the above becomes something like: >>>> >>>> if (breakpointReached) { >>>> if (failReason != null) { >>>> failure(failReason); >>>> } >>>> } else { >>>> failure("Expected breakpoint in ScriptDebuggee:" + >>>> ScriptDebuggee.BKPT_LINE + " was not reached"); >>>> } >>>> >>>> But then I wonder, why not just rethrow the exception when >>>> engine.eval(script) fails and save yourself from having to fetch >>>> ScriptDebuggee.failReason using the debugger, or is that somehow >>>> part of what is being tested? >>> >>> It is not going to work if I understand things correctly. >>> Please, check my comment above. >>> In order to make it less confusing, I'd suggest to rename failReason >>> to debuggeeFailReason on the debuggee side. >> Yeah, maybe. But then you could also call it debuggeeFailReason on >> the debugger side. That might make more sense. There's no reason for >> ScriptDebuggee to add the "debuggee" prefix to one of its own fields. > > Agreed. > >> I think there's still a need to have cleaner logic for indicating if >> the breakpoint was reached. Right now we initialize failReason to a >> potential failed reason string, then clear it if we hit the break >> point and the debuggee had no previous errors. I think using >> breakpointReached logic like I have above is a cleaner approach. > > Got it, thanks. > Yes, this will be more clear. I shall change the logic as you have suggested and post another patch for review. thanks, Paru. > > Thanks, > Serguei > > >> >> thanks, >> >> Chris >>> >>> >>> Thanks, >>> Serguei >>> >>> >>>>>> >>>>>> Is there a reason why ScriptDebuggee doesn't just put everything >>>>>> in main() and not have a doit() method? >>>>> No there isn't a specific reason. I noticed that other tests were >>>>> doing it - like BreakpointTest and for consistency and clarity, i >>>>> followed that pattern. >>>> Ok. >>>> >>>> thanks, >>>> >>>> Chris >>>>> >>>>> thanks, >>>>> Paru. >>>>> >>>>>> >>>>>> thanks, >>>>>> >>>>>> Chris >>>>>> >>>>>> On 2/7/18 12:25 PM, serguei.spitsyn at oracle.com >>>>>> wrote: >>>>>>> Hi Paru, >>>>>>> >>>>>>> It looks good. >>>>>>> Thank you a lot for taking care about this! >>>>>>> >>>>>>> Could we get at least one more review from the Serviceability >>>>>>> team on this new test? >>>>>>> >>>>>>> Thanks, >>>>>>> Serguei >>>>>>> >>>>>>> >>>>>>> On 2/2/18 09:35, Paru Somashekar wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> Please review the fix for JDK-8193150. >>>>>>>> >>>>>>>> The fix introduces a new jtreg test, NashornPopFrameTest. It is >>>>>>>> based on the original test from JDK-8187143 >>>>>>>> that was >>>>>>>> provided by the customer. >>>>>>>> >>>>>>>> Bug : https://bugs.openjdk.java.net/browse/JDK-8193150 >>>>>>>> >>>>>>>> Webrev : http://cr.openjdk.java.net/~psomashe/8193150/webrev/ >>>>>>>> >>>>>>>> >>>>>>>> Here is a brief description of what the test does :- >>>>>>>> >>>>>>>> * The debuggee, creates and uses a Nashorn engine to evaluate >>>>>>>> a simple script. >>>>>>>> >>>>>>>> * The debugger tries to set a breakpoint in Nashorn?s internal >>>>>>>> DEBUGGER method. >>>>>>>> * When the breakpoint is reached, it looks for stack frame >>>>>>>> whose method's declaring type name starts with (nashorn >>>>>>>> dynamically generated classes) >>>>>>>> ?jdk.nashorn.internal.scripts.Script$?. >>>>>>>> * It then pops stack frames using the >>>>>>>> ThreadReference.popFrames() call, up to and including the above >>>>>>>> stackframe. >>>>>>>> * The execution of the debuggee application is resumed after >>>>>>>> the needed frames have been popped. >>>>>>>> >>>>>>>> This test is included in the ProblemList as it fails under some >>>>>>>> circumstances (bug JDK-8187143) >>>>>>>> . Is always >>>>>>>> passes with the -Xint flag however always fails with -Xcomp. It >>>>>>>> fails intermittently with the -Xmixed (default). >>>>>>>> >>>>>>>> thanks, >>>>>>>> Paru. >>>>>>> >>>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Thu Feb 8 03:28:31 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 8 Feb 2018 13:28:31 +1000 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: References: Message-ID: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> Hi Robin, Adding in hotspot-runtime-dev as all the hotspot changes belong to runtime. I had an initial look through this. To be honest I don't like it. We seem to have to record little bits of information all over the place just so they can be reported by the shutdown event. It seems untidy. :( Can you rename _starting_thread to _main_thread please. The use of "starting" in thread.hpp/cpp is really a naming anomaly. The main thread is the thread that loads the JVM. And that would be consistent with set_exception_in_main_thread. Though why do we care if the main thread exited with an unhandled exception? And if we do care, why do we only care when the shutdown reason is ""No remaining non-daemon Java threads"? I don't like the need to add os::get_abort_exit_code. Do we really need it? What useful information does it convey? It is unfortunate that you need to add beforeHalt to deal with the event mechanism itself being turned off (?) by a shutdown hook. That would seem to potentially lose a lot of event information given hooks can run in arbitrary order and execute arbitrary Java code. And essentially you end up recording the initial reason shutdown started, though potentially it could end up terminating for a different reason. Let's see what others think ... Thanks, David On 8/02/2018 1:18 AM, Robin Westberg wrote: > Hi all, > > Please review the following change that adds an event-based tracing > event that is generated when the VM shuts down. The intent is to capture > shutdowns that occur after the VM has been properly initialized (as > initialization problems would most likely mean that the tracing > framework hasn?t been properly started either). > > Issue: https://bugs.openjdk.java.net/browse/JDK-8041626 > Webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00/ > Testing:?hs-tier1,hs-tier2,jdk-tier1,jdk-tier2 > > Best regards, > Robin From jini.george at oracle.com Thu Feb 8 04:34:26 2018 From: jini.george at oracle.com (Jini George) Date: Thu, 8 Feb 2018 10:04:26 +0530 Subject: Issue with 8192897: NPE occurs on clhsdb jstack In-Reply-To: References: Message-ID: <84512510-d96a-209a-30e5-70111694ebf9@oracle.com> Thanks for this, Daniel. Is this consistently reproducible on the Aarch64 system with the -Xcomp part of the test ? Thanks, Jini On 2/7/2018 10:12 PM, stewartd.qdt wrote: > I am getting Null Pointer Exceptions with both ClhsdbJstack.java and > ClhsdbFindPC.java. It appears the addition of testing with ??-Xcomp` > causing a Method to come back as null during the stack walk. I have > included the results of the Jstack test (the FindPC is similar) for the > Xcomp section below. I have created bug JDK-8196969 > . ?The full .jtr is > attached there. ?I am not familiar enough with the Java stack to know if > having? a null Method pointer is valid or not. > > Thanks, > > Daniel > > hsdb> > > Command line: > ['/home/stewartd/openjdk/OpenJDK/hs/build/linux-aarch64-normal-server-release/images/jdk/bin/java' > '-Xcomp' '-cp' > '/home/stewartd/openjdk/OpenJDK/hs/JTwork/classes/serviceability/sa/ClhsdbJstack.d:/home/stewartd/openjdk/OpenJDK/hs/JTwork/classes/test/lib' > 'jdk.test.lib.apps.LingeredApp' '717a6b55-bcf8-405e-9d5a-afe1212f0fc6.lck' ] > > Started LingeredApp (-Xcomp) with pid 23731 > > Starting clhsdb against 23731 > > Warning! JS Engine can't start, some commands will not be available. > > hsdb> Deadlock Detection: > > No deadlocks found. > > "Common-Cleaner" #22 daemon prio=8 tid=0x0000ffffa0528000 nid=0x5d2d in > Object.wait() [0x0000fffe427de000] > > ?? java.lang.Thread.State: TIMED_WAITING (on object monitor) > > ?? JavaThread state: _thread_blocked > > - java.lang.ref.ReferenceQueue.remove(long) @bci=59, line=151, > pc=0x0000ffff892af7c4, Method*=0x0000fffe49851a98 (Compiled frame; > information may be imprecise) > > ??????????????? - waiting to lock <0x0000000101c956f0> (a > java.lang.ref.ReferenceQueue$Lock) > > - jdk.internal.ref.CleanerImpl.run() @bci=65, line=148, > pc=0x0000ffff892ac8e4, Method*=0x0000fffe49ab23d8 (Compiled frame) > > - java.lang.Thread.run() @bci=11, line=844, pc=0x0000ffff892a9d30, > Method*=0x0000fffe49738ee0 (Compiled frame) > > - jdk.internal.misc.InnocuousThread.run() @bci=20, line=134, > pc=0x0000ffff892a9d30, Method*=0x0000fffe49aba318 (Compiled frame) > > Locked ownable synchronizers: > > ??? - None > > "Signal Dispatcher" #4 daemon prio=9 tid=0x0000ffffa04a6800 nid=0x5d05 > runnable [0x0000000000000000] > > ?? java.lang.Thread.State: RUNNABLE > > ?? JavaThread state: _thread_blocked > > Locked ownable synchronizers: > > ??? - None > > "Finalizer" #3 daemon prio=8 tid=0x0000ffffa0494000 nid=0x5d04 in > Object.wait() [0x0000fffe492ee000] > > ?? java.lang.Thread.State: WAITING (on object monitor) > > ?? JavaThread state: _thread_blocked > > - java.lang.ref.ReferenceQueue.remove(long) @bci=59, line=151, > pc=0x0000ffff888e15a0, Method*=0x0000fffe49851a98 (Interpreted frame) > > ??????????????? - waiting to lock <0x0000000101c09528> (a > java.lang.ref.ReferenceQueue$Lock) > > - java.lang.ref.ReferenceQueue.remove() @bci=2, line=172, > pc=0x0000ffff888e12c0, Method*=0x0000fffe49851b40 (Interpreted frame) > > - java.lang.ref.Finalizer$FinalizerThread.run() @bci=37, line=216, > pc=0x0000ffff888e12c0, Method*=0x0000fffe49856cb0 (Interpreted frame) > > Locked ownable synchronizers: > > ??? - None > > "Reference Handler" #2 daemon prio=10 tid=0x0000ffffa0492000 nid=0x5d03 > waiting on condition [0x0000fffe494ee000] > > ?? java.lang.Thread.State: RUNNABLE > > ?? JavaThread state: _thread_blocked > > - java.lang.ref.Reference.processPendingReferences() @bci=0, line=166, > pc=0x0000ffff888e15a0, Method*=0x0000fffe49733100 (Interpreted frame) > > - java.lang.ref.Reference.access$000() @bci=0, line=44, > pc=0x0000ffff888e15a0, Method*=0x0000fffe49733788 (Interpreted frame) > > - java.lang.ref.Reference$ReferenceHandler.run() @bci=0, line=138, > pc=0x0000ffff888e15a0, Method*=0x0000fffe4984ffa0 (Interpreted frame) > > Locked ownable synchronizers: > > ??? - None > > "main" #1 prio=5 tid=0x0000ffffa0010000 nid=0x5cb5 waiting on condition > [0x0000ffffa5d2e000] > > ?? java.lang.Thread.State: TIMED_WAITING (sleeping) > > ?? JavaThread state: _thread_blocked > > Error occurred during stack walking: > > Locked ownable synchronizers: > > ??? - None > > hsdb> > > ----------System.err:(147/8216)---------- > > Attaching to process 23463, please wait... > > javax.script.ScriptException: TypeError: sapkg.runtime.VM.getVM is not a > function in sa.js at line number 54 > > javax.script.ScriptException: TypeError: sapkg.runtime.VM.getVM is not a > function in sa.js at line number 54 > > Attaching to process 23731, please wait... > > javax.script.ScriptException: TypeError: sapkg.runtime.VM.getVM is not a > function in sa.js at line number 54 > > javax.script.ScriptException: TypeError: sapkg.runtime.VM.getVM is not a > function in sa.js at line number 54 > > java.lang.NullPointerException > > ??????????????? at > jdk.hotspot.agent/sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:83) > > ??????????????? at > jdk.hotspot.agent/sun.jvm.hotspot.CommandProcessor$26.doit(CommandProcessor.java:1073) > > ??????????????? at > jdk.hotspot.agent/sun.jvm.hotspot.CommandProcessor.executeCommand(CommandProcessor.java:1966) > > ??????????????? at > jdk.hotspot.agent/sun.jvm.hotspot.CommandProcessor.executeCommand(CommandProcessor.java:1936) > > ??????????????? at > jdk.hotspot.agent/sun.jvm.hotspot.CommandProcessor.run(CommandProcessor.java:1816) > > ??????????????? at > jdk.hotspot.agent/sun.jvm.hotspot.CLHSDB.run(CLHSDB.java:99) > > ??????????????? at > jdk.hotspot.agent/sun.jvm.hotspot.CLHSDB.main(CLHSDB.java:40) > > ??????????????? at > jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runCLHSDB(SALauncher.java:191) > > ??????????????? at > jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:439) > From chris.plummer at oracle.com Thu Feb 8 00:47:47 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 7 Feb 2018 16:47:47 -0800 Subject: RFR 8193150: Create a jtreg version of the test from JDK-8187143 In-Reply-To: References: <5A74A169.5020604@oracle.com> <5A7B7E1D.3090304@oracle.com> <58bf290a-2c35-afb2-c195-8cfbc08d3702@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From harsha.wardhana.b at oracle.com Thu Feb 8 06:31:42 2018 From: harsha.wardhana.b at oracle.com (Harsha Wardhana B) Date: Thu, 8 Feb 2018 12:01:42 +0530 Subject: RFR: JDK-8187498: Add a -Xmanagement flag as syntactic sugar for -Dcom.sun.management.jmxremote.* properties In-Reply-To: <5A7B1C29.2040308@oracle.com> References: <0fca13c3-1b8f-947c-9041-bdea87b04ba8@oracle.com> <5A687F75.3050404@oracle.com> <05c2e5af-b9d0-0386-4565-ade1179a1d49@oracle.com> <7a18b9d5-3a59-8132-2c4f-ba5de35bfa1d@oracle.com> <0e019151-ef48-b4f4-d253-b9da62aa07c3@oracle.com> <248276b1-1589-561f-0718-7bcb1fd578c7@oracle.com> <40cf194e-4d74-3a34-5eb2-1acc5a5abafb@oracle.com> <5A7B1C29.2040308@oracle.com> Message-ID: <92ff19df-fc8c-3df7-fffb-09c6985b14f5@oracle.com> Hi Kumar, Thanks for your inputs. On Wednesday 07 February 2018 09:02 PM, Kumar Srinivasan wrote: > > Hi Harsha, > > Changes look reasonable to me, couple of things that must be addressed: > > 1. Since this is a main-stream launcher change with a documented and > supported > option, a CSR is required,? you have? to add and document the option > in the help page > http://hg.openjdk.java.net/jdk/jdk/file/8cc67294ec56/src/java.base/share/classes/sun/launcher/resources/launcher.properties > > > 2. You also have to create a doc bug so that the doc team will > document it in the Tools > Reference Guide, and link it to this bug. Does it need a Release note > ? probably does, > in which case you will have to create Release Note subtask and follow > the RN process. Will take care of above items. > > 3. Is XmanagementAgentTest.java part of tier1 test suite ? If not, > then? I think it ought > to be in tier1 grouping, perhaps best to park this under > jdk/tools/launcher/management ? Ok. I will move the test in subsequent webrev. -Harsha > > > Kumar > > >> Hi All, >> >> After internal discussions, below format for management flags was >> agreed upon. >> >> 1. --start-management-agent port=1234,ssl=on??????? (space seperator) >> or >> 2. --start-management-agent=port=1234,ssl=on??????? ('=' seperator) >> >> If option 1 is specified, it will be converted to option 2 by the >> java launcher before it is passed onto VM. >> >> With above GNU long format for management options, specifying >> arguments is mandatory unlike before. >> >> --start-management-agent will not be recognized in the current format >> and hence will not default to --start-management-agent=local=true. >> >> Below is the webrev with above changes and corresponding tests. >> >> http://cr.openjdk.java.net/~hb/8187498/webrev.01/ >> >> Please review and comment. >> >> Thanks >> Harsha >> >> On Monday 29 January 2018 03:14 PM, Harsha Wardhana B wrote: >>> Hi Alan, >>> >>> I am not fully aware about Java launcher or how it passes options to >>> VM. Let me check with some other folks and get back to you. >>> >>> Thanks >>> Harsha >>> >>> On Monday 29 January 2018 01:55 PM, Alan Bateman wrote: >>>> >>>> >>>> On 29/01/2018 05:20, Harsha Wardhana B wrote: >>>>> Hi Mandy,Alan, >>>>> >>>>> Thanks for your inputs. >>>>> If I keep it as launcher option, it may need to know JMX agent >>>>> flags which may need to be extended in future. >>>>> I would prefer making it a VM option. I will make the required >>>>> changes and send out an updated webrev. >>>> I think Mandy's suggestion is to just transform --management >>>> so a form that the VM can read. The launcher will need to >>>> replace the space anyway as the VM only accepts "=". >>>> >>>> -Alan >>> >> > From Alan.Bateman at oracle.com Thu Feb 8 06:46:45 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 8 Feb 2018 06:46:45 +0000 Subject: JDK-8080990: libdt_socket/socket_md.c(202) : warning C4996: 'gethostbyname': Use getaddrinfo() or GetAddrInfoW() In-Reply-To: References: <5A6A2611.4030003@oracle.com> <5A7331D9.9000800@oracle.com> <624B1D02-366B-4FF0-92F4-757ED64D356F@oracle.com> Message-ID: On 07/02/2018 18:02, gary.adams at oracle.com wrote: > Yes, WSASendDisconnect is deprecated in vs2013. > As far as I know "shutdown(fd, SD_SEND)" prevents further outgoing writes > and there was no final payload to send. > > I have not seen any failures in the java/nio tests. > Okay, I guess it should okay but can you fix the formatting before you push this? (8 space indent should be 4). Also we need to fix the bug information as core-svc/debugger and a summary saying it's debugger transport issue is misleading. You've done the right thing of course to fix the larger issue and the bug should reflect that. -Alan From Alan.Bateman at oracle.com Thu Feb 8 07:40:32 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 8 Feb 2018 07:40:32 +0000 Subject: RFR: JDK-8187498: Add a -Xmanagement flag as syntactic sugar for -Dcom.sun.management.jmxremote.* properties In-Reply-To: <40cf194e-4d74-3a34-5eb2-1acc5a5abafb@oracle.com> References: <0fca13c3-1b8f-947c-9041-bdea87b04ba8@oracle.com> <5A687F75.3050404@oracle.com> <05c2e5af-b9d0-0386-4565-ade1179a1d49@oracle.com> <7a18b9d5-3a59-8132-2c4f-ba5de35bfa1d@oracle.com> <0e019151-ef48-b4f4-d253-b9da62aa07c3@oracle.com> <248276b1-1589-561f-0718-7bcb1fd578c7@oracle.com> <40cf194e-4d74-3a34-5eb2-1acc5a5abafb@oracle.com> Message-ID: On 07/02/2018 09:19, Harsha Wardhana B wrote: > Hi All, > > After internal discussions, below format for management flags was > agreed upon. > > 1. --start-management-agent port=1234,ssl=on??? ??? (space seperator) > or > 2. --start-management-agent=port=1234,ssl=on??? ??? ('=' seperator) > > If option 1 is specified, it will be converted to option 2 by the java > launcher before it is passed onto VM. > > With above GNU long format for management options, specifying > arguments is mandatory unlike before. > > --start-management-agent will not be recognized in the current format > and hence will not default to --start-management-agent=local=true. `--start-management-agent ` make sense and it wouldn't be hard to introduce `--start-local-management-agent` if really needed. One point that needs discussion is whether the needs something, maybe an agent name or kind, to make it clear what agent should be started. Today we have a JMX management agent, Oracle JDK downloads additionally have a SNMP management agent. There are several 3rd party exporters. If AgentProvider were promoted to an exported package then it would make this pluggibility a lot cleaner and would mean the --start-management-agent could specify the agent name/kind without needing hacks or forwarding by the JMX agent. -Alan From jini.george at oracle.com Thu Feb 8 08:59:12 2018 From: jini.george at oracle.com (Jini George) Date: Thu, 8 Feb 2018 14:29:12 +0530 Subject: RFR: SA: clhsdb 'printall' throws ClassCastException while printing out the bytecodes In-Reply-To: <9327a930-d576-b625-1eb5-f5345c92f80c@oracle.com> References: <2c7094ce-2310-721c-0f1b-f17e26a8d014@oracle.com> <9327a930-d576-b625-1eb5-f5345c92f80c@oracle.com> Message-ID: Thank you very much, David and Serguei for the review. David, based on your comments, I took a relook at the fix and I have modified it. Since this is an object array klass, the search for the method needs to be done among the methods of the superclass (java.lang.Object). The modified webrev is at: http://cr.openjdk.java.net/~jgeorge/8175384/webrev.02/index.html I have addressed the other comments. Thank you, Jini. On 1/25/2018 8:00 AM, David Holmes wrote: > Hi Jini, > > On 25/01/2018 3:43 AM, Jini George wrote: >> Hello, >> >> Requesting reviews for the issue: >> >> SA: clhsdb 'printall' throws ClassCastException while printing out the >> bytecodes >> >> ID: https://bugs.openjdk.java.net/browse/JDK-8175384 >> Webrev: http://cr.openjdk.java.net/~jgeorge/8175384/webrev.01/ >> >> The issue was seen while printing out the details of the invokevirtual >> bytecode of the java.lang.Enum.values() method. The fix is to avoid a >> ClassCastException by returning the bottom klass in the case of an >> object array klass, while printing out the details. > > I'm a bit confused as to what getFieldOrMethodKlassRefAt is supposed to > be returning - should it really be instanceKlass? It's not at all clear > to me that returning the array component type is the right thing to do. > > That aside a couple of minor nits: > > src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/oops/ConstantPool.java > > !?????? k = (k.isObjArrayKlass())?((ObjArrayKlass)k).getBottomKlass():k; > > No () needed around k.isObjArrayKlass() > Spaces needed around the ? and : operators > > test/hotspot/jtreg/serviceability/sa/ClhsdbLauncher.java > > Copyright year should now be "2017, 2018," > > Thanks, > David > >> Tests run: The SA tests pass with Mach5 and jprt. >> >> Thanks, >> Jini. From stewartd.qdt at qualcommdatacenter.com Thu Feb 8 13:13:46 2018 From: stewartd.qdt at qualcommdatacenter.com (stewartd.qdt) Date: Thu, 8 Feb 2018 13:13:46 +0000 Subject: Issue with 8192897: NPE occurs on clhsdb jstack In-Reply-To: <84512510-d96a-209a-30e5-70111694ebf9@oracle.com> References: <84512510-d96a-209a-30e5-70111694ebf9@oracle.com> Message-ID: <47d1f562fe5e4079863b5413693b6ef6@NASANEXM01E.na.qualcomm.com> Yes, this is consistently reproducible on my system. I'm happy to provide more information if it will help. If I knew more about how the Java stack worked, I could do more of this debugging myself. Is it ever legal to have a Method pointer of null? Daniel -----Original Message----- From: Jini George [mailto:jini.george at oracle.com] Sent: Wednesday, February 7, 2018 11:34 PM To: stewartd.qdt ; serviceability-dev ; David Holmes ; Sharath Ballal ; yasuenag at gmail.com Subject: Re: Issue with 8192897: NPE occurs on clhsdb jstack Thanks for this, Daniel. Is this consistently reproducible on the Aarch64 system with the -Xcomp part of the test ? Thanks, Jini On 2/7/2018 10:12 PM, stewartd.qdt wrote: > I am getting Null Pointer Exceptions with both ClhsdbJstack.java and > ClhsdbFindPC.java. It appears the addition of testing with ?'-Xcomp` > causing a Method to come back as null during the stack walk. I have > included the results of the Jstack test (the FindPC is similar) for > the Xcomp section below. I have created bug JDK-8196969 > . ?The full .jtr is > attached there. ?I am not familiar enough with the Java stack to know > if having? a null Method pointer is valid or not. > > Thanks, > > Daniel > > hsdb> > > Command line: > ['/home/stewartd/openjdk/OpenJDK/hs/build/linux-aarch64-normal-server-release/images/jdk/bin/java' > '-Xcomp' '-cp' > '/home/stewartd/openjdk/OpenJDK/hs/JTwork/classes/serviceability/sa/ClhsdbJstack.d:/home/stewartd/openjdk/OpenJDK/hs/JTwork/classes/test/lib' > 'jdk.test.lib.apps.LingeredApp' > '717a6b55-bcf8-405e-9d5a-afe1212f0fc6.lck' ] > > Started LingeredApp (-Xcomp) with pid 23731 > > Starting clhsdb against 23731 > > Warning! JS Engine can't start, some commands will not be available. > > hsdb> Deadlock Detection: > > No deadlocks found. > > "Common-Cleaner" #22 daemon prio=8 tid=0x0000ffffa0528000 nid=0x5d2d > in > Object.wait() [0x0000fffe427de000] > > ?? java.lang.Thread.State: TIMED_WAITING (on object monitor) > > ?? JavaThread state: _thread_blocked > > - java.lang.ref.ReferenceQueue.remove(long) @bci=59, line=151, > pc=0x0000ffff892af7c4, Method*=0x0000fffe49851a98 (Compiled frame; > information may be imprecise) > > ??????????????? - waiting to lock <0x0000000101c956f0> (a > java.lang.ref.ReferenceQueue$Lock) > > - jdk.internal.ref.CleanerImpl.run() @bci=65, line=148, > pc=0x0000ffff892ac8e4, Method*=0x0000fffe49ab23d8 (Compiled frame) > > - java.lang.Thread.run() @bci=11, line=844, pc=0x0000ffff892a9d30, > Method*=0x0000fffe49738ee0 (Compiled frame) > > - jdk.internal.misc.InnocuousThread.run() @bci=20, line=134, > pc=0x0000ffff892a9d30, Method*=0x0000fffe49aba318 (Compiled frame) > > Locked ownable synchronizers: > > ??? - None > > "Signal Dispatcher" #4 daemon prio=9 tid=0x0000ffffa04a6800 nid=0x5d05 > runnable [0x0000000000000000] > > ?? java.lang.Thread.State: RUNNABLE > > ?? JavaThread state: _thread_blocked > > Locked ownable synchronizers: > > ??? - None > > "Finalizer" #3 daemon prio=8 tid=0x0000ffffa0494000 nid=0x5d04 in > Object.wait() [0x0000fffe492ee000] > > ?? java.lang.Thread.State: WAITING (on object monitor) > > ?? JavaThread state: _thread_blocked > > - java.lang.ref.ReferenceQueue.remove(long) @bci=59, line=151, > pc=0x0000ffff888e15a0, Method*=0x0000fffe49851a98 (Interpreted frame) > > ??????????????? - waiting to lock <0x0000000101c09528> (a > java.lang.ref.ReferenceQueue$Lock) > > - java.lang.ref.ReferenceQueue.remove() @bci=2, line=172, > pc=0x0000ffff888e12c0, Method*=0x0000fffe49851b40 (Interpreted frame) > > - java.lang.ref.Finalizer$FinalizerThread.run() @bci=37, line=216, > pc=0x0000ffff888e12c0, Method*=0x0000fffe49856cb0 (Interpreted frame) > > Locked ownable synchronizers: > > ??? - None > > "Reference Handler" #2 daemon prio=10 tid=0x0000ffffa0492000 > nid=0x5d03 waiting on condition [0x0000fffe494ee000] > > ?? java.lang.Thread.State: RUNNABLE > > ?? JavaThread state: _thread_blocked > > - java.lang.ref.Reference.processPendingReferences() @bci=0, line=166, > pc=0x0000ffff888e15a0, Method*=0x0000fffe49733100 (Interpreted frame) > > - java.lang.ref.Reference.access$000() @bci=0, line=44, > pc=0x0000ffff888e15a0, Method*=0x0000fffe49733788 (Interpreted frame) > > - java.lang.ref.Reference$ReferenceHandler.run() @bci=0, line=138, > pc=0x0000ffff888e15a0, Method*=0x0000fffe4984ffa0 (Interpreted frame) > > Locked ownable synchronizers: > > ??? - None > > "main" #1 prio=5 tid=0x0000ffffa0010000 nid=0x5cb5 waiting on > condition [0x0000ffffa5d2e000] > > ?? java.lang.Thread.State: TIMED_WAITING (sleeping) > > ?? JavaThread state: _thread_blocked > > Error occurred during stack walking: > > Locked ownable synchronizers: > > ??? - None > > hsdb> > > ----------System.err:(147/8216)---------- > > Attaching to process 23463, please wait... > > javax.script.ScriptException: TypeError: sapkg.runtime.VM.getVM is not > a function in sa.js at line number 54 > > javax.script.ScriptException: TypeError: sapkg.runtime.VM.getVM is not > a function in sa.js at line number 54 > > Attaching to process 23731, please wait... > > javax.script.ScriptException: TypeError: sapkg.runtime.VM.getVM is not > a function in sa.js at line number 54 > > javax.script.ScriptException: TypeError: sapkg.runtime.VM.getVM is not > a function in sa.js at line number 54 > > java.lang.NullPointerException > > ??????????????? at > jdk.hotspot.agent/sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java > :83) > > ??????????????? at > jdk.hotspot.agent/sun.jvm.hotspot.CommandProcessor$26.doit(CommandProc > essor.java:1073) > > ??????????????? at > jdk.hotspot.agent/sun.jvm.hotspot.CommandProcessor.executeCommand(Comm > andProcessor.java:1966) > > ??????????????? at > jdk.hotspot.agent/sun.jvm.hotspot.CommandProcessor.executeCommand(Comm > andProcessor.java:1936) > > ??????????????? at > jdk.hotspot.agent/sun.jvm.hotspot.CommandProcessor.run(CommandProcesso > r.java:1816) > > ??????????????? at > jdk.hotspot.agent/sun.jvm.hotspot.CLHSDB.run(CLHSDB.java:99) > > ??????????????? at > jdk.hotspot.agent/sun.jvm.hotspot.CLHSDB.main(CLHSDB.java:40) > > ??????????????? at > jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runCLHSDB(SALauncher.java > :191) > > ??????????????? at > jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:439) > From robin.westberg at oracle.com Thu Feb 8 15:50:50 2018 From: robin.westberg at oracle.com (Robin Westberg) Date: Thu, 8 Feb 2018 16:50:50 +0100 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> References: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> Message-ID: <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> Hi David, > On 8 Feb 2018, at 04:28, David Holmes wrote: > > Hi Robin, > > Adding in hotspot-runtime-dev as all the hotspot changes belong to runtime. Thanks, sorry about that.. > I had an initial look through this. > > To be honest I don't like it. We seem to have to record little bits of information all over the place just so they can be reported by the shutdown event. It seems untidy. :( > > Can you rename _starting_thread to _main_thread please. The use of "starting" in thread.hpp/cpp is really a naming anomaly. The main thread is the thread that loads the JVM. And that would be consistent with set_exception_in_main_thread. > > Though why do we care if the main thread exited with an unhandled exception? And if we do care, why do we only care when the shutdown reason is ""No remaining non-daemon Java threads"? > > I don't like the need to add os::get_abort_exit_code. Do we really need it? What useful information does it convey? Right, almost all the runtime changes are made in order to try to figure out what the process exit code from the launcher will eventually be. For example, the launcher returns 1 if the main thread exited with an unhandled exception, but 0 otherwise. But I actually agree that this information is probably only of marginal use (it could always be captured from wherever Java is launched if someone really wants it), so it is perhaps not worth the trouble. Discussed it a bit with Erik Gahlin, and perhaps the best option here is to simply remove the status code field from the event, that would simplify things and make the code you mention above go away. > It is unfortunate that you need to add beforeHalt to deal with the event mechanism itself being turned off (?) by a shutdown hook. That would seem to potentially lose a lot of event information given hooks can run in arbitrary order and execute arbitrary Java code. And essentially you end up recording the initial reason shutdown started, though potentially it could end up terminating for a different reason. In this case I think it actually conveys useful information if you are trying to diagnose an unexpected shutdown. It should be useful to find the initial request of an orderly shutdown, even if something else happens during the shutdown sequence like a finalizer calling exit (in which case you could possibly end up with two shutdown events, but both may contain interesting information). Best regards, Robin > Let's see what others think ... > > Thanks, > David > > On 8/02/2018 1:18 AM, Robin Westberg wrote: >> Hi all, >> Please review the following change that adds an event-based tracing event that is generated when the VM shuts down. The intent is to capture shutdowns that occur after the VM has been properly initialized (as initialization problems would most likely mean that the tracing framework hasn?t been properly started either). >> Issue: https://bugs.openjdk.java.net/browse/JDK-8041626 >> Webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00/ >> Testing: hs-tier1,hs-tier2,jdk-tier1,jdk-tier2 >> Best regards, >> Robin From parvathi.somashekar at oracle.com Thu Feb 8 22:50:59 2018 From: parvathi.somashekar at oracle.com (Paru Somashekar) Date: Thu, 08 Feb 2018 14:50:59 -0800 Subject: RFR 8193150: Create a jtreg version of the test from JDK-8187143 In-Reply-To: <5A7BBC20.9070805@oracle.com> References: <5A74A169.5020604@oracle.com> <5A7B7E1D.3090304@oracle.com> <58bf290a-2c35-afb2-c195-8cfbc08d3702@oracle.com> <5ceed45b-90fe-f57c-d5c6-a553ce047d33@oracle.com> <5A7BBC20.9070805@oracle.com> Message-ID: <5A7CD453.7090301@oracle.com> I have incorporated all your feedback and created a new webrev at : http://cr.openjdk.java.net/~psomashe/8193150/webrev.01/ - Added comments - modified the logic for failReason and breakpoint reached aspect on the debugger side. thanks, Paru. On 2/7/18, 6:55 PM, Paru Somashekar wrote: > Hi Chris, Serguei, > > On 2/7/18, 4:56 PM, serguei.spitsyn at oracle.com wrote: >> On 2/7/18 16:47, Chris Plummer wrote: >>> On 2/7/18 3:23 PM, serguei.spitsyn at oracle.com >>> wrote: >>>> Hi Chris, >>>> >>>> >>>> On 2/7/18 15:06, Chris Plummer wrote: >>>>> Hi Paru, >>>>> >>>>> On 2/7/18 2:30 PM, Paru Somashekar wrote: >>>>>> Thanks for the review Chris, comments inline.. >>>>>> On 2/7/18, 1:25 PM, Chris Plummer wrote: >>>>>>> Hi Paru, >>>>>>> >>>>>>> Thanks for writing this test. It will make figuring out >>>>>>> JDK-8187143 a >>>>>>> lot easier. >>>>>>> >>>>>>> Overall the test looks good. My main concern is the lack of >>>>>>> comments. It makes it hard for the reader to understand the flow >>>>>>> of the test and to understand some of the less obvious actions >>>>>>> being performed. That is especially true for this test, which is >>>>>>> doing some really bizarre stuff. Some of this you cover in our >>>>>>> RFR summary below, but that info really needs to be in the test >>>>>>> itself, along with additional comments. For example, what does >>>>>>> pauseAtDebugger() do? It looks to me like it sets a breakpoint >>>>>>> on the java script debugger that has a class name that ends with >>>>>>> ScriptRuntime and the method name is DEBUGGER(). But I only >>>>>>> figured that out after staring at the code for a while, and >>>>>>> recalling a conversation we had a few weeks ago. It's also not >>>>>>> described why this is being done. >>>>>> I shall add more comments into the test code to make it easier to >>>>>> understand. However while I understand what is being done ( e.g. >>>>>> adding breakpoint on Nashorn's internal DEBUGGER method) - >>>>>> unfortunately I too am not sure "why" it is being done. I do not >>>>>> have insight on what the author ( bug reporter) was trying to do.. >>>>> That's ok. They "why" is because this is a test case demonstrating >>>>> a failure a user ran into. You might want to mention that also, >>>>> although the @bug reference might enough. >>>> >>>> Agreed as this is my understanding too. >>>> >>>> >>>>>>> >>>>>>> Here's another example: >>>>>>> >>>>>>> 126 while (!vmDisconnected) { >>>>>>> 127 try { >>>>>>> 128 Thread.sleep(100); >>>>>>> 129 } catch (InterruptedException ee) { >>>>>>> 130 } >>>>>>> 131 } >>>>>>> >>>>>>> I seem to also recall us discussing the need for this, but can >>>>>>> no longer recall the reason >>>>>> The above loop is there to make the debugger continue to run >>>>>> until it receives a VMdisconnect event either because the >>>>>> Debuggee crashed / got exception / finished. >>>>>> I shall add a comment for this as well. >>>>>>> >>>>>>> Another example is findScriptFrame(). What is the significance >>>>>>> of the frame whose class name starts with >>>>>>> jdk.nashorn.internal.scripts.Script$? I think I understand (it's >>>>>>> the generated java method for the javascript you setup in >>>>>>> ScriptDebuggee.doit()), but I can only figure this out based on >>>>>>> earlier conversations we had and your RFR comments below. I'd >>>>>>> expect the uninformed reader to spend a long time coming the >>>>>>> same conclusion. >>>>>> Again, I am not clear on the significance of popping frames until >>>>>> this method which is a generated java method for javascript in >>>>>> the debuggee. I could consult with the author and add those >>>>>> comments as well. >>>>> This is just to recreate the situation the customer saw when >>>>> running into the bug. We don't need to know the details of why >>>>> they were doing what they did, only that it resulted in a bug >>>>> being exposed. I'm mostly asking that you add comments that >>>>> explain what the test is doing, but not worry so much about >>>>> explaining the underlying reasons why the test was written in the >>>>> first place (although that might be useful as part of an overall >>>>> test summary comment at the top). >>>> >>>> Right. >>>> Thank you for the suggestion! >>>> I did not pay attention to it when pre-reviewed. >>>> >>>>>>> >>>>>>> The following are just a few minor things I noticed: >>>>>>> >>>>>>> Copyright should only have 2018. >>>>>>> >>>>>>> 57 } catch (Exception npe) { >>>>>>> >>>>>>> Probably best to call it "ex" instead of "npe". >>>>>>> >>>>>>> 85 NashornPopFrameTest bbcT = new >>>>>>> NashornPopFrameTest(args); >>>>>>> >>>>>>> It's unclear to me where the name "bbcT" comes from. >>>>>> I shall change that to npft or something like that. >>>>>>> >>>>>>> 134 if (failReason != null) { >>>>>>> 135 failure(failReason); >>>>>>> 136 } >>>>>>> >>>>>>> You have two classes that declare "String failReason" which >>>>>>> makes it a bit confusing to track which one the reader is >>>>>>> dealing with. Also, the NashornPopFrameTest version is >>>>>>> initialized to non-null, so doesn't that make the test always >>>>>>> fail when the above code is executed? >>>>>> Even though failReason is initialized, it is reset if the >>>>>> expected breakpoint is reached. And when the breakpoint is >>>>>> reached, it checks the Debuggee version of the field, if it is >>>>>> non null, then this field is set to the non null value - else it >>>>>> is set to null. >>>>>> I shall add some comments to make it less confusing. >>>>>> >>>>> So in the above check failReason has a double meaning (maybe even >>>>> triple). It could be set to its original value, which means the >>>>> breakpoint was never reached, or if the breakpoint is reached it >>>>> is set to ScriptDebuggee.failReason, which basically represents >>>>> the result of having called engine.eval(script). I think it would >>>>> be clearer if you just had a static flag to indicate if the >>>>> breakpoint was reached and just initialize failReason to null. >>>> >>>> The static flag does not work as the debuggee is in a different VM >>>> process. >>> Of course. Rookie mistake on my part. :) >> >> I knew it but had done the same mistake. :) >> >> >>>>> Then the above becomes something like: >>>>> >>>>> if (breakpointReached) { >>>>> if (failReason != null) { >>>>> failure(failReason); >>>>> } >>>>> } else { >>>>> failure("Expected breakpoint in ScriptDebuggee:" + >>>>> ScriptDebuggee.BKPT_LINE + " was not reached"); >>>>> } >>>>> >>>>> But then I wonder, why not just rethrow the exception when >>>>> engine.eval(script) fails and save yourself from having to fetch >>>>> ScriptDebuggee.failReason using the debugger, or is that somehow >>>>> part of what is being tested? >>>> >>>> It is not going to work if I understand things correctly. >>>> Please, check my comment above. >>>> In order to make it less confusing, I'd suggest to rename >>>> failReason to debuggeeFailReason on the debuggee side. >>> Yeah, maybe. But then you could also call it debuggeeFailReason on >>> the debugger side. That might make more sense. There's no reason for >>> ScriptDebuggee to add the "debuggee" prefix to one of its own fields. >> >> Agreed. >> >>> I think there's still a need to have cleaner logic for indicating if >>> the breakpoint was reached. Right now we initialize failReason to a >>> potential failed reason string, then clear it if we hit the break >>> point and the debuggee had no previous errors. I think using >>> breakpointReached logic like I have above is a cleaner approach. >> >> Got it, thanks. >> Yes, this will be more clear. > I shall change the logic as you have suggested and post another patch > for review. > > thanks, > Paru. > >> >> Thanks, >> Serguei >> >> >>> >>> thanks, >>> >>> Chris >>>> >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>>>>> >>>>>>> Is there a reason why ScriptDebuggee doesn't just put everything >>>>>>> in main() and not have a doit() method? >>>>>> No there isn't a specific reason. I noticed that other tests were >>>>>> doing it - like BreakpointTest and for consistency and clarity, i >>>>>> followed that pattern. >>>>> Ok. >>>>> >>>>> thanks, >>>>> >>>>> Chris >>>>>> >>>>>> thanks, >>>>>> Paru. >>>>>> >>>>>>> >>>>>>> thanks, >>>>>>> >>>>>>> Chris >>>>>>> >>>>>>> On 2/7/18 12:25 PM, serguei.spitsyn at oracle.com >>>>>>> wrote: >>>>>>>> Hi Paru, >>>>>>>> >>>>>>>> It looks good. >>>>>>>> Thank you a lot for taking care about this! >>>>>>>> >>>>>>>> Could we get at least one more review from the Serviceability >>>>>>>> team on this new test? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Serguei >>>>>>>> >>>>>>>> >>>>>>>> On 2/2/18 09:35, Paru Somashekar wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> Please review the fix for JDK-8193150. >>>>>>>>> >>>>>>>>> The fix introduces a new jtreg test, NashornPopFrameTest. It >>>>>>>>> is based on the original test from JDK-8187143 >>>>>>>>> that was >>>>>>>>> provided by the customer. >>>>>>>>> >>>>>>>>> Bug : https://bugs.openjdk.java.net/browse/JDK-8193150 >>>>>>>>> >>>>>>>>> Webrev : http://cr.openjdk.java.net/~psomashe/8193150/webrev/ >>>>>>>>> >>>>>>>>> >>>>>>>>> Here is a brief description of what the test does :- >>>>>>>>> >>>>>>>>> * The debuggee, creates and uses a Nashorn engine to evaluate >>>>>>>>> a simple script. >>>>>>>>> >>>>>>>>> * The debugger tries to set a breakpoint in Nashorn?s >>>>>>>>> internal DEBUGGER method. >>>>>>>>> * When the breakpoint is reached, it looks for stack frame >>>>>>>>> whose method's declaring type name starts with (nashorn >>>>>>>>> dynamically generated classes) >>>>>>>>> ?jdk.nashorn.internal.scripts.Script$?. >>>>>>>>> * It then pops stack frames using the >>>>>>>>> ThreadReference.popFrames() call, up to and including the >>>>>>>>> above stackframe. >>>>>>>>> * The execution of the debuggee application is resumed after >>>>>>>>> the needed frames have been popped. >>>>>>>>> >>>>>>>>> This test is included in the ProblemList as it fails under >>>>>>>>> some circumstances (bug JDK-8187143) >>>>>>>>> . Is always >>>>>>>>> passes with the -Xint flag however always fails with -Xcomp. >>>>>>>>> It fails intermittently with the -Xmixed (default). >>>>>>>>> >>>>>>>>> thanks, >>>>>>>>> Paru. >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Thu Feb 8 23:36:13 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 8 Feb 2018 15:36:13 -0800 Subject: RFR 8193150: Create a jtreg version of the test from JDK-8187143 In-Reply-To: <5A7CD453.7090301@oracle.com> References: <5A74A169.5020604@oracle.com> <5A7B7E1D.3090304@oracle.com> <58bf290a-2c35-afb2-c195-8cfbc08d3702@oracle.com> <5ceed45b-90fe-f57c-d5c6-a553ce047d33@oracle.com> <5A7BBC20.9070805@oracle.com> <5A7CD453.7090301@oracle.com> Message-ID: <68b04ca1-34ed-6c32-dc1a-6430355f8df0@oracle.com> Hi Paru, Looks good. Thanks for the changes. Chris On 2/8/18 2:50 PM, Paru Somashekar wrote: > > I have incorporated all your feedback and created a new webrev at : > http://cr.openjdk.java.net/~psomashe/8193150/webrev.01/ > > > - Added comments > - modified the logic for failReason and breakpoint reached aspect on > the debugger side. > > thanks, > Paru. > > On 2/7/18, 6:55 PM, Paru Somashekar wrote: >> Hi Chris, Serguei, >> >> On 2/7/18, 4:56 PM, serguei.spitsyn at oracle.com wrote: >>> On 2/7/18 16:47, Chris Plummer wrote: >>>> On 2/7/18 3:23 PM, serguei.spitsyn at oracle.com >>>> wrote: >>>>> Hi Chris, >>>>> >>>>> >>>>> On 2/7/18 15:06, Chris Plummer wrote: >>>>>> Hi Paru, >>>>>> >>>>>> On 2/7/18 2:30 PM, Paru Somashekar wrote: >>>>>>> Thanks for the review Chris, comments inline.. >>>>>>> On 2/7/18, 1:25 PM, Chris Plummer wrote: >>>>>>>> Hi Paru, >>>>>>>> >>>>>>>> Thanks for writing this test. It will make figuring out >>>>>>>> JDK-8187143 >>>>>>>> a lot easier. >>>>>>>> >>>>>>>> Overall the test looks good. My main concern is the lack of >>>>>>>> comments. It makes it hard for the reader to understand the >>>>>>>> flow of the test and to understand some of the less obvious >>>>>>>> actions being performed. That is especially true for this test, >>>>>>>> which is doing some really bizarre stuff. Some of this you >>>>>>>> cover in our RFR summary below, but that info really needs to >>>>>>>> be in the test itself, along with additional comments. For >>>>>>>> example, what does pauseAtDebugger() do? It looks to me like it >>>>>>>> sets a breakpoint on the java script debugger that has a class >>>>>>>> name that ends with ScriptRuntime and the method name is >>>>>>>> DEBUGGER(). But I only figured that out after staring at the >>>>>>>> code for a while, and recalling a conversation we had a few >>>>>>>> weeks ago. It's also not described why this is being done. >>>>>>> I shall add more comments into the test code to make it easier >>>>>>> to understand. However while I understand what is being done ( >>>>>>> e.g. adding breakpoint on Nashorn's internal DEBUGGER method) - >>>>>>> unfortunately I too am not sure "why" it is being done. I do not >>>>>>> have insight on what the author ( bug reporter) was trying to do.. >>>>>> That's ok. They "why" is because this is a test case >>>>>> demonstrating a failure a user ran into. You might want to >>>>>> mention that also, although the @bug reference might enough. >>>>> >>>>> Agreed as this is my understanding too. >>>>> >>>>> >>>>>>>> >>>>>>>> Here's another example: >>>>>>>> >>>>>>>> ?126???????? while (!vmDisconnected) { >>>>>>>> ?127???????????? try { >>>>>>>> ?128???????????????? Thread.sleep(100); >>>>>>>> ?129???????????? } catch (InterruptedException ee) { >>>>>>>> ?130???????????? } >>>>>>>> ?131???????? } >>>>>>>> >>>>>>>> I seem to also recall us discussing the need for this, but can >>>>>>>> no longer recall the reason >>>>>>> The above loop is there to make the debugger continue to run >>>>>>> until it receives a VMdisconnect event either because the >>>>>>> Debuggee crashed / got exception / finished. >>>>>>> I shall add a comment for this as well. >>>>>>>> >>>>>>>> Another example is findScriptFrame(). What is the significance >>>>>>>> of the frame whose class name starts with >>>>>>>> jdk.nashorn.internal.scripts.Script$? I think I understand >>>>>>>> (it's the generated java method for the javascript you setup in >>>>>>>> ScriptDebuggee.doit()), but I can only figure this out based on >>>>>>>> earlier conversations we had and your RFR comments below. I'd >>>>>>>> expect the uninformed reader to spend a long time coming the >>>>>>>> same conclusion. >>>>>>> Again, I am not clear on the significance of popping frames >>>>>>> until this method which is a generated java method for >>>>>>> javascript in the debuggee. I could consult with the author and >>>>>>> add those comments as well. >>>>>> This is just to recreate the situation the customer saw when >>>>>> running into the bug. We don't need to know the details of why >>>>>> they were doing what they did, only that it resulted in a bug >>>>>> being exposed. I'm mostly asking that you add comments that >>>>>> explain what the test is doing, but not worry so much about >>>>>> explaining the underlying reasons why the test was written in the >>>>>> first place (although that might be useful as part of an overall >>>>>> test summary comment at the top). >>>>> >>>>> Right. >>>>> Thank you for the suggestion! >>>>> I did not pay attention to it when pre-reviewed. >>>>> >>>>>>>> >>>>>>>> The following are just a few minor things I noticed: >>>>>>>> >>>>>>>> Copyright should only have 2018. >>>>>>>> >>>>>>>> ? 57???????? } catch (Exception npe) { >>>>>>>> >>>>>>>> Probably best to call it "ex" instead of "npe". >>>>>>>> >>>>>>>> ? 85???????? NashornPopFrameTest bbcT = new >>>>>>>> NashornPopFrameTest(args); >>>>>>>> >>>>>>>> It's unclear to me where the name "bbcT" comes from. >>>>>>> I shall change that to npft or something like that. >>>>>>>> >>>>>>>> ?134???????? if (failReason != null) { >>>>>>>> ?135???????????? failure(failReason); >>>>>>>> ?136???????? } >>>>>>>> >>>>>>>> You have two classes that declare "String failReason" which >>>>>>>> makes it a bit confusing to track which one the reader is >>>>>>>> dealing with. Also, the NashornPopFrameTest version is >>>>>>>> initialized to non-null, so doesn't that make the test always >>>>>>>> fail when the above code is executed? >>>>>>> Even though failReason is initialized, it is reset if the >>>>>>> expected breakpoint is reached. And when the breakpoint is >>>>>>> reached, it checks the Debuggee version of the field, if it is >>>>>>> non null, then this field is set to the non null value - else it >>>>>>> is set to null. >>>>>>> I shall add some comments to make it less confusing. >>>>>>> >>>>>> So in the above check failReason has a double meaning (maybe even >>>>>> triple). It could be set to its original value, which means the >>>>>> breakpoint was never reached, or if the breakpoint is reached it >>>>>> is set to ScriptDebuggee.failReason, which basically represents >>>>>> the result of having called engine.eval(script). I think it would >>>>>> be clearer if you just had a static flag to indicate if the >>>>>> breakpoint was reached and just initialize failReason to null. >>>>> >>>>> The static flag does not work as the debuggee is in a different VM >>>>> process. >>>> Of course. Rookie mistake on my part. :) >>> >>> I knew it but had done the same mistake. :) >>> >>> >>>>>> Then the above becomes something like: >>>>>> >>>>>> ??? if (breakpointReached) { >>>>>> ???????? if (failReason != null) { >>>>>> ????????????? failure(failReason); >>>>>> ???????? } >>>>>> ??? } else { >>>>>> ??? ??? failure("Expected breakpoint in ScriptDebuggee:" + >>>>>> ?????????????? ScriptDebuggee.BKPT_LINE + " was not reached"); >>>>>> ??? } >>>>>> >>>>>> But then I wonder, why not just rethrow the exception when >>>>>> engine.eval(script) fails and save yourself from having to fetch >>>>>> ScriptDebuggee.failReason using the debugger, or is that somehow >>>>>> part of what is being tested? >>>>> >>>>> It is not going to work if I understand things correctly. >>>>> Please, check my comment above. >>>>> In order to make it less confusing, I'd suggest to rename >>>>> failReason to debuggeeFailReason on the debuggee side. >>>> Yeah, maybe. But then you could also call it debuggeeFailReason on >>>> the debugger side. That might make more sense. There's no reason >>>> for ScriptDebuggee to add the "debuggee" prefix to one of its own >>>> fields. >>> >>> Agreed. >>> >>>> I think there's still a need to have cleaner logic for indicating >>>> if the breakpoint was reached. Right now we initialize failReason >>>> to a potential failed reason string, then clear it if we hit the >>>> break point and the debuggee had no previous errors. I think using >>>> breakpointReached logic like I have above is a cleaner approach. >>> >>> Got it, thanks. >>> Yes, this will be more clear. >> I shall change the logic as you have suggested and post another patch >> for review. >> >> thanks, >> Paru. >> >>> >>> Thanks, >>> Serguei >>> >>> >>>> >>>> thanks, >>>> >>>> Chris >>>>> >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>>>>> >>>>>>>> Is there a reason why ScriptDebuggee doesn't just put >>>>>>>> everything in main() and not have a doit() method? >>>>>>> No there isn't a specific reason. I noticed that other tests >>>>>>> were doing it - like BreakpointTest and for consistency and >>>>>>> clarity, i followed that pattern. >>>>>> Ok. >>>>>> >>>>>> thanks, >>>>>> >>>>>> Chris >>>>>>> >>>>>>> thanks, >>>>>>> Paru. >>>>>>> >>>>>>>> >>>>>>>> thanks, >>>>>>>> >>>>>>>> Chris >>>>>>>> >>>>>>>> On 2/7/18 12:25 PM, serguei.spitsyn at oracle.com >>>>>>>> wrote: >>>>>>>>> Hi Paru, >>>>>>>>> >>>>>>>>> It looks good. >>>>>>>>> Thank you a lot for taking care about this! >>>>>>>>> >>>>>>>>> Could we get at least one more review from the Serviceability >>>>>>>>> team on this new test? >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Serguei >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2/2/18 09:35, Paru Somashekar wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> Please review the fix for JDK-8193150. >>>>>>>>>> >>>>>>>>>> The fix introduces a new jtreg test, NashornPopFrameTest. It >>>>>>>>>> is based on the original test from JDK-8187143 >>>>>>>>>> that was >>>>>>>>>> provided by the customer. >>>>>>>>>> >>>>>>>>>> Bug : https://bugs.openjdk.java.net/browse/JDK-8193150 >>>>>>>>>> Webrev : http://cr.openjdk.java.net/~psomashe/8193150/webrev/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Here is a brief description of what the test does :- >>>>>>>>>> >>>>>>>>>> * The debuggee,? creates and uses a Nashorn engine to >>>>>>>>>> evaluate a simple script. >>>>>>>>>> >>>>>>>>>> * The debugger? tries to set a breakpoint in Nashorn?s >>>>>>>>>> internal DEBUGGER method. >>>>>>>>>> * When the breakpoint is reached, it looks for stack frame >>>>>>>>>> whose method's declaring type name starts with (nashorn >>>>>>>>>> dynamically generated classes) >>>>>>>>>> ?jdk.nashorn.internal.scripts.Script$?. >>>>>>>>>> * It then pops stack frames using the >>>>>>>>>> ThreadReference.popFrames() call, up to and including the >>>>>>>>>> above stackframe. >>>>>>>>>> * The execution of the debuggee application is resumed after >>>>>>>>>> the needed frames have been popped. >>>>>>>>>> >>>>>>>>>> This test is included in the ProblemList as it fails under >>>>>>>>>> some circumstances (bug JDK-8187143) >>>>>>>>>> . Is always >>>>>>>>>> passes with the -Xint flag however always fails with -Xcomp. >>>>>>>>>> It fails intermittently with the -Xmixed (default). >>>>>>>>>> >>>>>>>>>> thanks, >>>>>>>>>> Paru. >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From mandy.chung at oracle.com Thu Feb 8 23:37:25 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 8 Feb 2018 15:37:25 -0800 Subject: RFR: JDK-8187498: Add a -Xmanagement flag as syntactic sugar for -Dcom.sun.management.jmxremote.* properties In-Reply-To: <40cf194e-4d74-3a34-5eb2-1acc5a5abafb@oracle.com> References: <0fca13c3-1b8f-947c-9041-bdea87b04ba8@oracle.com> <5A687F75.3050404@oracle.com> <05c2e5af-b9d0-0386-4565-ade1179a1d49@oracle.com> <7a18b9d5-3a59-8132-2c4f-ba5de35bfa1d@oracle.com> <0e019151-ef48-b4f4-d253-b9da62aa07c3@oracle.com> <248276b1-1589-561f-0718-7bcb1fd578c7@oracle.com> <40cf194e-4d74-3a34-5eb2-1acc5a5abafb@oracle.com> Message-ID: <87cb309b-8764-193a-0447-8ecb741d308d@oracle.com> On 2/7/18 1:19 AM, Harsha Wardhana B wrote: > > --start-management-agent will not be recognized in the current format and > hence will not default to --start-management-agent=local=true. `--start-local-management-server` is one option as Alan suggests. Another option is to make `--start-management-agent` to accept an optional argument. VM can accept `--start-management-agent` or `--start-management-agent=port=1234,ssl=on`. It may require more launcher change to support it. > Below is the webrev with above changes and corresponding tests. > > http://cr.openjdk.java.net/~hb/8187498/webrev.01/ arguments.cpp 3216 if (FLAG_SET_CMDLINE(bool, ManagementServer, true) != Flag::SUCCESS) { 3217 return JNI_EINVAL; 3218 } 3219 // management agent in module jdk.management.agent 3220 if (!create_numbered_property("jdk.module.addmods", "jdk.management.agent", addmods_count++)) { 3221 return JNI_ENOMEM; 3222 } - you can consider refactor this to replace this block and line 2988-3994. 3224 jio_fprintf(defaultStream::output_stream(), 3225 "-Xmanagement is not supported in this VM.\n"); -Xmanagement not yet renamed java.c To set the precedence for future VM long form options, I suggest to rename IsManagementOption to IsVMLongFormOption and change the error message not specific to management agent. Agent.java If --start-management-agent is set, -Dcom.sun.management.* takes precedence. Do you really want to do that? The new VM option intends to simplify the command-line to type in. I think it's cleaner and reasonable if --start-management-agent is specified, -Dcom.sun.management.* are ignored (maybe worth emit a warning if set) The implementation uses VMManagement::getVmArguments and scan the VM options for -start-management-agent. The VM is the one invoking jdk.internal.agent.Agent::startAgent. A simpler option is to change Agent::startAgent to take an argument parameter that will be passed with the argument of --start-management-agent or null if not set. Similarly for startRemoteManagementAgent and startLocalAgent_name. parseXmgmtArgs should be renamed as no longer -Xmgmt. Better not to tightly coupled with the option name. 291 if(args.startsWith("--start-management-agent")) { 292 return parseXmgmtArgs(args); 293 } What is this change for? Mandy -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Fri Feb 9 00:49:48 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 8 Feb 2018 16:49:48 -0800 Subject: RFR 8193150: Create a jtreg version of the test from JDK-8187143 In-Reply-To: <68b04ca1-34ed-6c32-dc1a-6430355f8df0@oracle.com> References: <5A74A169.5020604@oracle.com> <5A7B7E1D.3090304@oracle.com> <58bf290a-2c35-afb2-c195-8cfbc08d3702@oracle.com> <5ceed45b-90fe-f57c-d5c6-a553ce047d33@oracle.com> <5A7BBC20.9070805@oracle.com> <5A7CD453.7090301@oracle.com> <68b04ca1-34ed-6c32-dc1a-6430355f8df0@oracle.com> Message-ID: Hi Paru, +1 Thanks, Serguei On 2/8/18 15:36, Chris Plummer wrote: > Hi Paru, > > Looks good. Thanks for the changes. > > Chris > > On 2/8/18 2:50 PM, Paru Somashekar wrote: >> >> I have incorporated all your feedback and created a new webrev at : >> http://cr.openjdk.java.net/~psomashe/8193150/webrev.01/ >> >> >> - Added comments >> - modified the logic for failReason and breakpoint reached aspect on >> the debugger side. >> >> thanks, >> Paru. >> >> On 2/7/18, 6:55 PM, Paru Somashekar wrote: >>> Hi Chris, Serguei, >>> >>> On 2/7/18, 4:56 PM, serguei.spitsyn at oracle.com wrote: >>>> On 2/7/18 16:47, Chris Plummer wrote: >>>>> On 2/7/18 3:23 PM, serguei.spitsyn at oracle.com >>>>> wrote: >>>>>> Hi Chris, >>>>>> >>>>>> >>>>>> On 2/7/18 15:06, Chris Plummer wrote: >>>>>>> Hi Paru, >>>>>>> >>>>>>> On 2/7/18 2:30 PM, Paru Somashekar wrote: >>>>>>>> Thanks for the review Chris, comments inline.. >>>>>>>> On 2/7/18, 1:25 PM, Chris Plummer wrote: >>>>>>>>> Hi Paru, >>>>>>>>> >>>>>>>>> Thanks for writing this test. It will make figuring out >>>>>>>>> JDK-8187143 >>>>>>>>> a lot easier. >>>>>>>>> >>>>>>>>> Overall the test looks good. My main concern is the lack of >>>>>>>>> comments. It makes it hard for the reader to understand the >>>>>>>>> flow of the test and to understand some of the less obvious >>>>>>>>> actions being performed. That is especially true for this >>>>>>>>> test, which is doing some really bizarre stuff. Some of this >>>>>>>>> you cover in our RFR summary below, but that info really needs >>>>>>>>> to be in the test itself, along with additional comments. For >>>>>>>>> example, what does pauseAtDebugger() do? It looks to me like >>>>>>>>> it sets a breakpoint on the java script debugger that has a >>>>>>>>> class name that ends with ScriptRuntime and the method name is >>>>>>>>> DEBUGGER(). But I only figured that out after staring at the >>>>>>>>> code for a while, and recalling a conversation we had a few >>>>>>>>> weeks ago. It's also not described why this is being done. >>>>>>>> I shall add more comments into the test code to make it easier >>>>>>>> to understand. However while I understand what is being done ( >>>>>>>> e.g. adding breakpoint on Nashorn's internal DEBUGGER method) - >>>>>>>> unfortunately I too am not sure "why" it is being done. I do >>>>>>>> not have insight on what the author ( bug reporter) was trying >>>>>>>> to do.. >>>>>>> That's ok. They "why" is because this is a test case >>>>>>> demonstrating a failure a user ran into. You might want to >>>>>>> mention that also, although the @bug reference might enough. >>>>>> >>>>>> Agreed as this is my understanding too. >>>>>> >>>>>> >>>>>>>>> >>>>>>>>> Here's another example: >>>>>>>>> >>>>>>>>> ?126???????? while (!vmDisconnected) { >>>>>>>>> ?127???????????? try { >>>>>>>>> ?128???????????????? Thread.sleep(100); >>>>>>>>> ?129???????????? } catch (InterruptedException ee) { >>>>>>>>> ?130???????????? } >>>>>>>>> ?131???????? } >>>>>>>>> >>>>>>>>> I seem to also recall us discussing the need for this, but can >>>>>>>>> no longer recall the reason >>>>>>>> The above loop is there to make the debugger continue to run >>>>>>>> until it receives a VMdisconnect event either because the >>>>>>>> Debuggee crashed / got exception / finished. >>>>>>>> I shall add a comment for this as well. >>>>>>>>> >>>>>>>>> Another example is findScriptFrame(). What is the significance >>>>>>>>> of the frame whose class name starts with >>>>>>>>> jdk.nashorn.internal.scripts.Script$? I think I understand >>>>>>>>> (it's the generated java method for the javascript you setup >>>>>>>>> in ScriptDebuggee.doit()), but I can only figure this out >>>>>>>>> based on earlier conversations we had and your RFR comments >>>>>>>>> below. I'd expect the uninformed reader to spend a long time >>>>>>>>> coming the same conclusion. >>>>>>>> Again, I am not clear on the significance of popping frames >>>>>>>> until this method which is a generated java method for >>>>>>>> javascript in the debuggee. I could consult with the author and >>>>>>>> add those comments as well. >>>>>>> This is just to recreate the situation the customer saw when >>>>>>> running into the bug. We don't need to know the details of why >>>>>>> they were doing what they did, only that it resulted in a bug >>>>>>> being exposed. I'm mostly asking that you add comments that >>>>>>> explain what the test is doing, but not worry so much about >>>>>>> explaining the underlying reasons why the test was written in >>>>>>> the first place (although that might be useful as part of an >>>>>>> overall test summary comment at the top). >>>>>> >>>>>> Right. >>>>>> Thank you for the suggestion! >>>>>> I did not pay attention to it when pre-reviewed. >>>>>> >>>>>>>>> >>>>>>>>> The following are just a few minor things I noticed: >>>>>>>>> >>>>>>>>> Copyright should only have 2018. >>>>>>>>> >>>>>>>>> ? 57???????? } catch (Exception npe) { >>>>>>>>> >>>>>>>>> Probably best to call it "ex" instead of "npe". >>>>>>>>> >>>>>>>>> ? 85???????? NashornPopFrameTest bbcT = new >>>>>>>>> NashornPopFrameTest(args); >>>>>>>>> >>>>>>>>> It's unclear to me where the name "bbcT" comes from. >>>>>>>> I shall change that to npft or something like that. >>>>>>>>> >>>>>>>>> ?134???????? if (failReason != null) { >>>>>>>>> ?135???????????? failure(failReason); >>>>>>>>> ?136???????? } >>>>>>>>> >>>>>>>>> You have two classes that declare "String failReason" which >>>>>>>>> makes it a bit confusing to track which one the reader is >>>>>>>>> dealing with. Also, the NashornPopFrameTest version is >>>>>>>>> initialized to non-null, so doesn't that make the test always >>>>>>>>> fail when the above code is executed? >>>>>>>> Even though failReason is initialized, it is reset if the >>>>>>>> expected breakpoint is reached. And when the breakpoint is >>>>>>>> reached, it checks the Debuggee version of the field, if it is >>>>>>>> non null, then this field is set to the non null value - else >>>>>>>> it is set to null. >>>>>>>> I shall add some comments to make it less confusing. >>>>>>>> >>>>>>> So in the above check failReason has a double meaning (maybe >>>>>>> even triple). It could be set to its original value, which means >>>>>>> the breakpoint was never reached, or if the breakpoint is >>>>>>> reached it is set to ScriptDebuggee.failReason, which basically >>>>>>> represents the result of having called engine.eval(script). I >>>>>>> think it would be clearer if you just had a static flag to >>>>>>> indicate if the breakpoint was reached and just initialize >>>>>>> failReason to null. >>>>>> >>>>>> The static flag does not work as the debuggee is in a different >>>>>> VM process. >>>>> Of course. Rookie mistake on my part. :) >>>> >>>> I knew it but had done the same mistake. :) >>>> >>>> >>>>>>> Then the above becomes something like: >>>>>>> >>>>>>> ??? if (breakpointReached) { >>>>>>> ???????? if (failReason != null) { >>>>>>> ????????????? failure(failReason); >>>>>>> ???????? } >>>>>>> ??? } else { >>>>>>> ??? ??? failure("Expected breakpoint in ScriptDebuggee:" + >>>>>>> ?????????????? ScriptDebuggee.BKPT_LINE + " was not reached"); >>>>>>> ??? } >>>>>>> >>>>>>> But then I wonder, why not just rethrow the exception when >>>>>>> engine.eval(script) fails and save yourself from having to fetch >>>>>>> ScriptDebuggee.failReason using the debugger, or is that somehow >>>>>>> part of what is being tested? >>>>>> >>>>>> It is not going to work if I understand things correctly. >>>>>> Please, check my comment above. >>>>>> In order to make it less confusing, I'd suggest to rename >>>>>> failReason to debuggeeFailReason on the debuggee side. >>>>> Yeah, maybe. But then you could also call it debuggeeFailReason on >>>>> the debugger side. That might make more sense. There's no reason >>>>> for ScriptDebuggee to add the "debuggee" prefix to one of its own >>>>> fields. >>>> >>>> Agreed. >>>> >>>>> I think there's still a need to have cleaner logic for indicating >>>>> if the breakpoint was reached. Right now we initialize failReason >>>>> to a potential failed reason string, then clear it if we hit the >>>>> break point and the debuggee had no previous errors. I think using >>>>> breakpointReached logic like I have above is a cleaner approach. >>>> >>>> Got it, thanks. >>>> Yes, this will be more clear. >>> I shall change the logic as you have suggested and post another >>> patch for review. >>> >>> thanks, >>> Paru. >>> >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>>> >>>>> thanks, >>>>> >>>>> Chris >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>>> >>>>>>>>> >>>>>>>>> Is there a reason why ScriptDebuggee doesn't just put >>>>>>>>> everything in main() and not have a doit() method? >>>>>>>> No there isn't a specific reason. I noticed that other tests >>>>>>>> were doing it - like BreakpointTest and for consistency and >>>>>>>> clarity, i followed that pattern. >>>>>>> Ok. >>>>>>> >>>>>>> thanks, >>>>>>> >>>>>>> Chris >>>>>>>> >>>>>>>> thanks, >>>>>>>> Paru. >>>>>>>> >>>>>>>>> >>>>>>>>> thanks, >>>>>>>>> >>>>>>>>> Chris >>>>>>>>> >>>>>>>>> On 2/7/18 12:25 PM, serguei.spitsyn at oracle.com >>>>>>>>> wrote: >>>>>>>>>> Hi Paru, >>>>>>>>>> >>>>>>>>>> It looks good. >>>>>>>>>> Thank you a lot for taking care about this! >>>>>>>>>> >>>>>>>>>> Could we get at least one more review from the Serviceability >>>>>>>>>> team on this new test? >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Serguei >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2/2/18 09:35, Paru Somashekar wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> Please review the fix for JDK-8193150. >>>>>>>>>>> >>>>>>>>>>> The fix introduces a new jtreg test, NashornPopFrameTest. It >>>>>>>>>>> is based on the original test from JDK-8187143 >>>>>>>>>>> that was >>>>>>>>>>> provided by the customer. >>>>>>>>>>> >>>>>>>>>>> Bug : https://bugs.openjdk.java.net/browse/JDK-8193150 >>>>>>>>>>> Webrev : >>>>>>>>>>> http://cr.openjdk.java.net/~psomashe/8193150/webrev/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Here is a brief description of what the test does :- >>>>>>>>>>> >>>>>>>>>>> * The debuggee,? creates and uses a Nashorn engine to >>>>>>>>>>> evaluate a simple script. >>>>>>>>>>> >>>>>>>>>>> * The debugger? tries to set a breakpoint in Nashorn?s >>>>>>>>>>> internal DEBUGGER method. >>>>>>>>>>> * When the breakpoint is reached, it looks for stack frame >>>>>>>>>>> whose method's declaring type name starts with (nashorn >>>>>>>>>>> dynamically generated classes) >>>>>>>>>>> ?jdk.nashorn.internal.scripts.Script$?. >>>>>>>>>>> * It then pops stack frames using the >>>>>>>>>>> ThreadReference.popFrames() call, up to and including the >>>>>>>>>>> above stackframe. >>>>>>>>>>> * The execution of the debuggee application is resumed after >>>>>>>>>>> the needed frames have been popped. >>>>>>>>>>> >>>>>>>>>>> This test is included in the ProblemList as it fails under >>>>>>>>>>> some circumstances (bug JDK-8187143) >>>>>>>>>>> . Is >>>>>>>>>>> always passes with the -Xint flag however always fails with >>>>>>>>>>> -Xcomp. It fails intermittently with the -Xmixed (default). >>>>>>>>>>> >>>>>>>>>>> thanks, >>>>>>>>>>> Paru. >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > > From parvathi.somashekar at oracle.com Fri Feb 9 02:16:59 2018 From: parvathi.somashekar at oracle.com (Paru Somashekar) Date: Thu, 08 Feb 2018 18:16:59 -0800 Subject: RFR 8193150: Create a jtreg version of the test from JDK-8187143 In-Reply-To: References: <5A74A169.5020604@oracle.com> <5A7B7E1D.3090304@oracle.com> <58bf290a-2c35-afb2-c195-8cfbc08d3702@oracle.com> <5ceed45b-90fe-f57c-d5c6-a553ce047d33@oracle.com> <5A7BBC20.9070805@oracle.com> <5A7CD453.7090301@oracle.com> <68b04ca1-34ed-6c32-dc1a-6430355f8df0@oracle.com> Message-ID: <5A7D049B.1010109@oracle.com> Thanks Chris and Serguei for the review. thanks, Paru. On 2/8/18, 4:49 PM, serguei.spitsyn at oracle.com wrote: > Hi Paru, > > +1 > > Thanks, > Serguei > > > On 2/8/18 15:36, Chris Plummer wrote: >> Hi Paru, >> >> Looks good. Thanks for the changes. >> >> Chris >> >> On 2/8/18 2:50 PM, Paru Somashekar wrote: >>> >>> I have incorporated all your feedback and created a new webrev at : >>> http://cr.openjdk.java.net/~psomashe/8193150/webrev.01/ >>> >>> >>> - Added comments >>> - modified the logic for failReason and breakpoint reached aspect on >>> the debugger side. >>> >>> thanks, >>> Paru. >>> >>> On 2/7/18, 6:55 PM, Paru Somashekar wrote: >>>> Hi Chris, Serguei, >>>> >>>> On 2/7/18, 4:56 PM, serguei.spitsyn at oracle.com wrote: >>>>> On 2/7/18 16:47, Chris Plummer wrote: >>>>>> On 2/7/18 3:23 PM, serguei.spitsyn at oracle.com >>>>>> wrote: >>>>>>> Hi Chris, >>>>>>> >>>>>>> >>>>>>> On 2/7/18 15:06, Chris Plummer wrote: >>>>>>>> Hi Paru, >>>>>>>> >>>>>>>> On 2/7/18 2:30 PM, Paru Somashekar wrote: >>>>>>>>> Thanks for the review Chris, comments inline.. >>>>>>>>> On 2/7/18, 1:25 PM, Chris Plummer wrote: >>>>>>>>>> Hi Paru, >>>>>>>>>> >>>>>>>>>> Thanks for writing this test. It will make figuring out >>>>>>>>>> JDK-8187143 >>>>>>>>>> a lot easier. >>>>>>>>>> >>>>>>>>>> Overall the test looks good. My main concern is the lack of >>>>>>>>>> comments. It makes it hard for the reader to understand the >>>>>>>>>> flow of the test and to understand some of the less obvious >>>>>>>>>> actions being performed. That is especially true for this >>>>>>>>>> test, which is doing some really bizarre stuff. Some of this >>>>>>>>>> you cover in our RFR summary below, but that info really >>>>>>>>>> needs to be in the test itself, along with additional >>>>>>>>>> comments. For example, what does pauseAtDebugger() do? It >>>>>>>>>> looks to me like it sets a breakpoint on the java script >>>>>>>>>> debugger that has a class name that ends with ScriptRuntime >>>>>>>>>> and the method name is DEBUGGER(). But I only figured that >>>>>>>>>> out after staring at the code for a while, and recalling a >>>>>>>>>> conversation we had a few weeks ago. It's also not described >>>>>>>>>> why this is being done. >>>>>>>>> I shall add more comments into the test code to make it easier >>>>>>>>> to understand. However while I understand what is being done ( >>>>>>>>> e.g. adding breakpoint on Nashorn's internal DEBUGGER method) >>>>>>>>> - unfortunately I too am not sure "why" it is being done. I do >>>>>>>>> not have insight on what the author ( bug reporter) was trying >>>>>>>>> to do.. >>>>>>>> That's ok. They "why" is because this is a test case >>>>>>>> demonstrating a failure a user ran into. You might want to >>>>>>>> mention that also, although the @bug reference might enough. >>>>>>> >>>>>>> Agreed as this is my understanding too. >>>>>>> >>>>>>> >>>>>>>>>> >>>>>>>>>> Here's another example: >>>>>>>>>> >>>>>>>>>> 126 while (!vmDisconnected) { >>>>>>>>>> 127 try { >>>>>>>>>> 128 Thread.sleep(100); >>>>>>>>>> 129 } catch (InterruptedException ee) { >>>>>>>>>> 130 } >>>>>>>>>> 131 } >>>>>>>>>> >>>>>>>>>> I seem to also recall us discussing the need for this, but >>>>>>>>>> can no longer recall the reason >>>>>>>>> The above loop is there to make the debugger continue to run >>>>>>>>> until it receives a VMdisconnect event either because the >>>>>>>>> Debuggee crashed / got exception / finished. >>>>>>>>> I shall add a comment for this as well. >>>>>>>>>> >>>>>>>>>> Another example is findScriptFrame(). What is the >>>>>>>>>> significance of the frame whose class name starts with >>>>>>>>>> jdk.nashorn.internal.scripts.Script$? I think I understand >>>>>>>>>> (it's the generated java method for the javascript you setup >>>>>>>>>> in ScriptDebuggee.doit()), but I can only figure this out >>>>>>>>>> based on earlier conversations we had and your RFR comments >>>>>>>>>> below. I'd expect the uninformed reader to spend a long time >>>>>>>>>> coming the same conclusion. >>>>>>>>> Again, I am not clear on the significance of popping frames >>>>>>>>> until this method which is a generated java method for >>>>>>>>> javascript in the debuggee. I could consult with the author >>>>>>>>> and add those comments as well. >>>>>>>> This is just to recreate the situation the customer saw when >>>>>>>> running into the bug. We don't need to know the details of why >>>>>>>> they were doing what they did, only that it resulted in a bug >>>>>>>> being exposed. I'm mostly asking that you add comments that >>>>>>>> explain what the test is doing, but not worry so much about >>>>>>>> explaining the underlying reasons why the test was written in >>>>>>>> the first place (although that might be useful as part of an >>>>>>>> overall test summary comment at the top). >>>>>>> >>>>>>> Right. >>>>>>> Thank you for the suggestion! >>>>>>> I did not pay attention to it when pre-reviewed. >>>>>>> >>>>>>>>>> >>>>>>>>>> The following are just a few minor things I noticed: >>>>>>>>>> >>>>>>>>>> Copyright should only have 2018. >>>>>>>>>> >>>>>>>>>> 57 } catch (Exception npe) { >>>>>>>>>> >>>>>>>>>> Probably best to call it "ex" instead of "npe". >>>>>>>>>> >>>>>>>>>> 85 NashornPopFrameTest bbcT = new >>>>>>>>>> NashornPopFrameTest(args); >>>>>>>>>> >>>>>>>>>> It's unclear to me where the name "bbcT" comes from. >>>>>>>>> I shall change that to npft or something like that. >>>>>>>>>> >>>>>>>>>> 134 if (failReason != null) { >>>>>>>>>> 135 failure(failReason); >>>>>>>>>> 136 } >>>>>>>>>> >>>>>>>>>> You have two classes that declare "String failReason" which >>>>>>>>>> makes it a bit confusing to track which one the reader is >>>>>>>>>> dealing with. Also, the NashornPopFrameTest version is >>>>>>>>>> initialized to non-null, so doesn't that make the test always >>>>>>>>>> fail when the above code is executed? >>>>>>>>> Even though failReason is initialized, it is reset if the >>>>>>>>> expected breakpoint is reached. And when the breakpoint is >>>>>>>>> reached, it checks the Debuggee version of the field, if it is >>>>>>>>> non null, then this field is set to the non null value - else >>>>>>>>> it is set to null. >>>>>>>>> I shall add some comments to make it less confusing. >>>>>>>>> >>>>>>>> So in the above check failReason has a double meaning (maybe >>>>>>>> even triple). It could be set to its original value, which >>>>>>>> means the breakpoint was never reached, or if the breakpoint is >>>>>>>> reached it is set to ScriptDebuggee.failReason, which basically >>>>>>>> represents the result of having called engine.eval(script). I >>>>>>>> think it would be clearer if you just had a static flag to >>>>>>>> indicate if the breakpoint was reached and just initialize >>>>>>>> failReason to null. >>>>>>> >>>>>>> The static flag does not work as the debuggee is in a different >>>>>>> VM process. >>>>>> Of course. Rookie mistake on my part. :) >>>>> >>>>> I knew it but had done the same mistake. :) >>>>> >>>>> >>>>>>>> Then the above becomes something like: >>>>>>>> >>>>>>>> if (breakpointReached) { >>>>>>>> if (failReason != null) { >>>>>>>> failure(failReason); >>>>>>>> } >>>>>>>> } else { >>>>>>>> failure("Expected breakpoint in ScriptDebuggee:" + >>>>>>>> ScriptDebuggee.BKPT_LINE + " was not reached"); >>>>>>>> } >>>>>>>> >>>>>>>> But then I wonder, why not just rethrow the exception when >>>>>>>> engine.eval(script) fails and save yourself from having to >>>>>>>> fetch ScriptDebuggee.failReason using the debugger, or is that >>>>>>>> somehow part of what is being tested? >>>>>>> >>>>>>> It is not going to work if I understand things correctly. >>>>>>> Please, check my comment above. >>>>>>> In order to make it less confusing, I'd suggest to rename >>>>>>> failReason to debuggeeFailReason on the debuggee side. >>>>>> Yeah, maybe. But then you could also call it debuggeeFailReason >>>>>> on the debugger side. That might make more sense. There's no >>>>>> reason for ScriptDebuggee to add the "debuggee" prefix to one of >>>>>> its own fields. >>>>> >>>>> Agreed. >>>>> >>>>>> I think there's still a need to have cleaner logic for indicating >>>>>> if the breakpoint was reached. Right now we initialize failReason >>>>>> to a potential failed reason string, then clear it if we hit the >>>>>> break point and the debuggee had no previous errors. I think >>>>>> using breakpointReached logic like I have above is a cleaner >>>>>> approach. >>>>> >>>>> Got it, thanks. >>>>> Yes, this will be more clear. >>>> I shall change the logic as you have suggested and post another >>>> patch for review. >>>> >>>> thanks, >>>> Paru. >>>> >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>>> >>>>>> thanks, >>>>>> >>>>>> Chris >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Serguei >>>>>>> >>>>>>> >>>>>>>>>> >>>>>>>>>> Is there a reason why ScriptDebuggee doesn't just put >>>>>>>>>> everything in main() and not have a doit() method? >>>>>>>>> No there isn't a specific reason. I noticed that other tests >>>>>>>>> were doing it - like BreakpointTest and for consistency and >>>>>>>>> clarity, i followed that pattern. >>>>>>>> Ok. >>>>>>>> >>>>>>>> thanks, >>>>>>>> >>>>>>>> Chris >>>>>>>>> >>>>>>>>> thanks, >>>>>>>>> Paru. >>>>>>>>> >>>>>>>>>> >>>>>>>>>> thanks, >>>>>>>>>> >>>>>>>>>> Chris >>>>>>>>>> >>>>>>>>>> On 2/7/18 12:25 PM, serguei.spitsyn at oracle.com >>>>>>>>>> wrote: >>>>>>>>>>> Hi Paru, >>>>>>>>>>> >>>>>>>>>>> It looks good. >>>>>>>>>>> Thank you a lot for taking care about this! >>>>>>>>>>> >>>>>>>>>>> Could we get at least one more review from the >>>>>>>>>>> Serviceability team on this new test? >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Serguei >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2/2/18 09:35, Paru Somashekar wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> Please review the fix for JDK-8193150. >>>>>>>>>>>> >>>>>>>>>>>> The fix introduces a new jtreg test, NashornPopFrameTest. >>>>>>>>>>>> It is based on the original test from JDK-8187143 >>>>>>>>>>>> that was >>>>>>>>>>>> provided by the customer. >>>>>>>>>>>> >>>>>>>>>>>> Bug : https://bugs.openjdk.java.net/browse/JDK-8193150 >>>>>>>>>>>> Webrev : >>>>>>>>>>>> http://cr.openjdk.java.net/~psomashe/8193150/webrev/ >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Here is a brief description of what the test does :- >>>>>>>>>>>> >>>>>>>>>>>> * The debuggee, creates and uses a Nashorn engine to >>>>>>>>>>>> evaluate a simple script. >>>>>>>>>>>> >>>>>>>>>>>> * The debugger tries to set a breakpoint in Nashorn?s >>>>>>>>>>>> internal DEBUGGER method. >>>>>>>>>>>> * When the breakpoint is reached, it looks for stack frame >>>>>>>>>>>> whose method's declaring type name starts with (nashorn >>>>>>>>>>>> dynamically generated classes) >>>>>>>>>>>> ?jdk.nashorn.internal.scripts.Script$?. >>>>>>>>>>>> * It then pops stack frames using the >>>>>>>>>>>> ThreadReference.popFrames() call, up to and including the >>>>>>>>>>>> above stackframe. >>>>>>>>>>>> * The execution of the debuggee application is resumed >>>>>>>>>>>> after the needed frames have been popped. >>>>>>>>>>>> >>>>>>>>>>>> This test is included in the ProblemList as it fails under >>>>>>>>>>>> some circumstances (bug JDK-8187143) >>>>>>>>>>>> . Is >>>>>>>>>>>> always passes with the -Xint flag however always fails with >>>>>>>>>>>> -Xcomp. It fails intermittently with the -Xmixed (default). >>>>>>>>>>>> >>>>>>>>>>>> thanks, >>>>>>>>>>>> Paru. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >> > From david.holmes at oracle.com Fri Feb 9 03:48:07 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 9 Feb 2018 13:48:07 +1000 Subject: RFR: SA: clhsdb 'printall' throws ClassCastException while printing out the bytecodes In-Reply-To: References: <2c7094ce-2310-721c-0f1b-f17e26a8d014@oracle.com> <9327a930-d576-b625-1eb5-f5345c92f80c@oracle.com> Message-ID: <3886ce17-9ee7-2e7b-46cc-f165e58aabf3@oracle.com> Hi Jini, On 8/02/2018 6:59 PM, Jini George wrote: > Thank you very much, David and Serguei for the review. > > David, based on your comments, I took a relook at the fix and I have > modified it. Since this is an object array klass, the search for the > method needs to be done among the methods of the superclass Okay I think that makes sense - the only methods an array has are those of Object. > (java.lang.Object). The modified webrev is at: > > http://cr.openjdk.java.net/~jgeorge/8175384/webrev.02/index.html Changes look good. Please update the bug report with the actual issue/solution. Thanks, David > I have addressed the other comments. > > Thank you, > Jini. > > On 1/25/2018 8:00 AM, David Holmes wrote: >> Hi Jini, >> >> On 25/01/2018 3:43 AM, Jini George wrote: >>> Hello, >>> >>> Requesting reviews for the issue: >>> >>> SA: clhsdb 'printall' throws ClassCastException while printing out >>> the bytecodes >>> >>> ID: https://bugs.openjdk.java.net/browse/JDK-8175384 >>> Webrev: http://cr.openjdk.java.net/~jgeorge/8175384/webrev.01/ >>> >>> The issue was seen while printing out the details of the >>> invokevirtual bytecode of the java.lang.Enum.values() method. The fix >>> is to avoid a ClassCastException by returning the bottom klass in the >>> case of an object array klass, while printing out the details. >> >> I'm a bit confused as to what getFieldOrMethodKlassRefAt is supposed >> to be returning - should it really be instanceKlass? It's not at all >> clear to me that returning the array component type is the right thing >> to do. >> >> That aside a couple of minor nits: >> >> src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/oops/ConstantPool.java >> >> >> !?????? k = (k.isObjArrayKlass())?((ObjArrayKlass)k).getBottomKlass():k; >> >> No () needed around k.isObjArrayKlass() >> Spaces needed around the ? and : operators >> >> test/hotspot/jtreg/serviceability/sa/ClhsdbLauncher.java >> >> Copyright year should now be "2017, 2018," >> >> Thanks, >> David >> >>> Tests run: The SA tests pass with Mach5 and jprt. >>> >>> Thanks, >>> Jini. From jini.george at oracle.com Fri Feb 9 03:55:39 2018 From: jini.george at oracle.com (Jini George) Date: Fri, 9 Feb 2018 09:25:39 +0530 Subject: RFR: SA: clhsdb 'printall' throws ClassCastException while printing out the bytecodes In-Reply-To: <3886ce17-9ee7-2e7b-46cc-f165e58aabf3@oracle.com> References: <2c7094ce-2310-721c-0f1b-f17e26a8d014@oracle.com> <9327a930-d576-b625-1eb5-f5345c92f80c@oracle.com> <3886ce17-9ee7-2e7b-46cc-f165e58aabf3@oracle.com> Message-ID: Thank you, David! - Jini. On 2/9/2018 9:18 AM, David Holmes wrote: > Hi Jini, > > On 8/02/2018 6:59 PM, Jini George wrote: >> Thank you very much, David and Serguei for the review. >> >> David, based on your comments, I took a relook at the fix and I have >> modified it. Since this is an object array klass, the search for the >> method needs to be done among the methods of the superclass > > Okay I think that makes sense - the only methods an array has are those > of Object. > >> (java.lang.Object). The modified webrev is at: >> >> http://cr.openjdk.java.net/~jgeorge/8175384/webrev.02/index.html > > Changes look good. Please update the bug report with the actual > issue/solution. > > Thanks, > David > >> I have addressed the other comments. >> >> Thank you, >> Jini. >> >> On 1/25/2018 8:00 AM, David Holmes wrote: >>> Hi Jini, >>> >>> On 25/01/2018 3:43 AM, Jini George wrote: >>>> Hello, >>>> >>>> Requesting reviews for the issue: >>>> >>>> SA: clhsdb 'printall' throws ClassCastException while printing out >>>> the bytecodes >>>> >>>> ID: https://bugs.openjdk.java.net/browse/JDK-8175384 >>>> Webrev: http://cr.openjdk.java.net/~jgeorge/8175384/webrev.01/ >>>> >>>> The issue was seen while printing out the details of the >>>> invokevirtual bytecode of the java.lang.Enum.values() method. The >>>> fix is to avoid a ClassCastException by returning the bottom klass >>>> in the case of an object array klass, while printing out the details. >>> >>> I'm a bit confused as to what getFieldOrMethodKlassRefAt is supposed >>> to be returning - should it really be instanceKlass? It's not at all >>> clear to me that returning the array component type is the right >>> thing to do. >>> >>> That aside a couple of minor nits: >>> >>> src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/oops/ConstantPool.java >>> >>> >>> !?????? k = (k.isObjArrayKlass())?((ObjArrayKlass)k).getBottomKlass():k; >>> >>> No () needed around k.isObjArrayKlass() >>> Spaces needed around the ? and : operators >>> >>> test/hotspot/jtreg/serviceability/sa/ClhsdbLauncher.java >>> >>> Copyright year should now be "2017, 2018," >>> >>> Thanks, >>> David >>> >>>> Tests run: The SA tests pass with Mach5 and jprt. >>>> >>>> Thanks, >>>> Jini. From david.holmes at oracle.com Fri Feb 9 04:22:01 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 9 Feb 2018 14:22:01 +1000 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> References: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> Message-ID: <5622d8af-d0d6-e77a-f7c1-f2c474f89763@oracle.com> Hi Robin, On 9/02/2018 1:50 AM, Robin Westberg wrote: > Hi David, > >> On 8 Feb 2018, at 04:28, David Holmes wrote: >> >> Hi Robin, >> >> Adding in hotspot-runtime-dev as all the hotspot changes belong to runtime. > > Thanks, sorry about that.. > >> I had an initial look through this. >> >> To be honest I don't like it. We seem to have to record little bits of information all over the place just so they can be reported by the shutdown event. It seems untidy. :( >> >> Can you rename _starting_thread to _main_thread please. The use of "starting" in thread.hpp/cpp is really a naming anomaly. The main thread is the thread that loads the JVM. And that would be consistent with set_exception_in_main_thread. >> >> Though why do we care if the main thread exited with an unhandled exception? And if we do care, why do we only care when the shutdown reason is ""No remaining non-daemon Java threads"? >> >> I don't like the need to add os::get_abort_exit_code. Do we really need it? What useful information does it convey? > > Right, almost all the runtime changes are made in order to try to figure out what the process exit code from the launcher will eventually be. For example, the launcher returns 1 if the main thread exited with an unhandled exception, but 0 otherwise. But I actually agree that this information is probably only of marginal use (it could always be captured from wherever Java is launched if someone really wants it), so it is perhaps not worth the trouble. Yes and that particular example of launcher behaviour is an archaic throwback to C programming - where end of "main" means end of the program - IMHO. I don't think it is of interest - and certainly not worth jumping through so many hoops to make the info available. > Discussed it a bit with Erik Gahlin, and perhaps the best option here is to simply remove the status code field from the event, that would simplify things and make the code you mention above go away. That sounds good to me. :) >> It is unfortunate that you need to add beforeHalt to deal with the event mechanism itself being turned off (?) by a shutdown hook. That would seem to potentially lose a lot of event information given hooks can run in arbitrary order and execute arbitrary Java code. And essentially you end up recording the initial reason shutdown started, though potentially it could end up terminating for a different reason. > > In this case I think it actually conveys useful information if you are trying to diagnose an unexpected shutdown. It should be useful to find the initial request of an orderly shutdown, even if something else happens during the shutdown sequence like a finalizer calling exit (in which case you could possibly end up with two shutdown events, but both may contain interesting information). Yes that is useful. But I find the need to code things the way they are, unfortunate. But given current constraints, so be it. Thanks, David > Best regards, > Robin > >> Let's see what others think ... >> >> Thanks, >> David >> >> On 8/02/2018 1:18 AM, Robin Westberg wrote: >>> Hi all, >>> Please review the following change that adds an event-based tracing event that is generated when the VM shuts down. The intent is to capture shutdowns that occur after the VM has been properly initialized (as initialization problems would most likely mean that the tracing framework hasn?t been properly started either). >>> Issue: https://bugs.openjdk.java.net/browse/JDK-8041626 >>> Webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00/ >>> Testing: hs-tier1,hs-tier2,jdk-tier1,jdk-tier2 >>> Best regards, >>> Robin > From jini.george at oracle.com Fri Feb 9 08:01:11 2018 From: jini.george at oracle.com (Jini George) Date: Fri, 9 Feb 2018 13:31:11 +0530 Subject: RFR: SA: clhsdb 'printall' throws ClassCastException while printing out the bytecodes In-Reply-To: <3bcb7174-6c70-ee18-c974-fe93093c7682@oracle.com> References: <2c7094ce-2310-721c-0f1b-f17e26a8d014@oracle.com> <9327a930-d576-b625-1eb5-f5345c92f80c@oracle.com> <3bcb7174-6c70-ee18-c974-fe93093c7682@oracle.com> Message-ID: Thank you very much, Serguei. - Jini. On 2/9/2018 1:28 PM, serguei.spitsyn at oracle.com wrote: > Hi Jini, > > The update looks good to me. > > Thanks, > Serguei > > > On 2/8/18 00:59, Jini George wrote: >> Thank you very much, David and Serguei for the review. >> >> David, based on your comments, I took a relook at the fix and I have >> modified it. Since this is an object array klass, the search for the >> method needs to be done among the methods of the superclass >> (java.lang.Object). The modified webrev is at: >> >> http://cr.openjdk.java.net/~jgeorge/8175384/webrev.02/index.html >> >> I have addressed the other comments. >> >> Thank you, >> Jini. >> >> On 1/25/2018 8:00 AM, David Holmes wrote: >>> Hi Jini, >>> >>> On 25/01/2018 3:43 AM, Jini George wrote: >>>> Hello, >>>> >>>> Requesting reviews for the issue: >>>> >>>> SA: clhsdb 'printall' throws ClassCastException while printing out >>>> the bytecodes >>>> >>>> ID: https://bugs.openjdk.java.net/browse/JDK-8175384 >>>> Webrev: http://cr.openjdk.java.net/~jgeorge/8175384/webrev.01/ >>>> >>>> The issue was seen while printing out the details of the >>>> invokevirtual bytecode of the java.lang.Enum.values() method. The >>>> fix is to avoid a ClassCastException by returning the bottom klass >>>> in the case of an object array klass, while printing out the details. >>> >>> I'm a bit confused as to what getFieldOrMethodKlassRefAt is supposed >>> to be returning - should it really be instanceKlass? It's not at all >>> clear to me that returning the array component type is the right >>> thing to do. >>> >>> That aside a couple of minor nits: >>> >>> src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/oops/ConstantPool.java >>> >>> >>> !?????? k = (k.isObjArrayKlass())?((ObjArrayKlass)k).getBottomKlass():k; >>> >>> No () needed around k.isObjArrayKlass() >>> Spaces needed around the ? and : operators >>> >>> test/hotspot/jtreg/serviceability/sa/ClhsdbLauncher.java >>> >>> Copyright year should now be "2017, 2018," >>> >>> Thanks, >>> David >>> >>>> Tests run: The SA tests pass with Mach5 and jprt. >>>> >>>> Thanks, >>>> Jini. > From serguei.spitsyn at oracle.com Fri Feb 9 07:58:55 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 8 Feb 2018 23:58:55 -0800 Subject: RFR: SA: clhsdb 'printall' throws ClassCastException while printing out the bytecodes In-Reply-To: References: <2c7094ce-2310-721c-0f1b-f17e26a8d014@oracle.com> <9327a930-d576-b625-1eb5-f5345c92f80c@oracle.com> Message-ID: <3bcb7174-6c70-ee18-c974-fe93093c7682@oracle.com> Hi Jini, The update looks good to me. Thanks, Serguei On 2/8/18 00:59, Jini George wrote: > Thank you very much, David and Serguei for the review. > > David, based on your comments, I took a relook at the fix and I have > modified it. Since this is an object array klass, the search for the > method needs to be done among the methods of the superclass > (java.lang.Object). The modified webrev is at: > > http://cr.openjdk.java.net/~jgeorge/8175384/webrev.02/index.html > > I have addressed the other comments. > > Thank you, > Jini. > > On 1/25/2018 8:00 AM, David Holmes wrote: >> Hi Jini, >> >> On 25/01/2018 3:43 AM, Jini George wrote: >>> Hello, >>> >>> Requesting reviews for the issue: >>> >>> SA: clhsdb 'printall' throws ClassCastException while printing out >>> the bytecodes >>> >>> ID: https://bugs.openjdk.java.net/browse/JDK-8175384 >>> Webrev: http://cr.openjdk.java.net/~jgeorge/8175384/webrev.01/ >>> >>> The issue was seen while printing out the details of the >>> invokevirtual bytecode of the java.lang.Enum.values() method. The >>> fix is to avoid a ClassCastException by returning the bottom klass >>> in the case of an object array klass, while printing out the details. >> >> I'm a bit confused as to what getFieldOrMethodKlassRefAt is supposed >> to be returning - should it really be instanceKlass? It's not at all >> clear to me that returning the array component type is the right >> thing to do. >> >> That aside a couple of minor nits: >> >> src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/oops/ConstantPool.java >> >> >> !?????? k = (k.isObjArrayKlass())?((ObjArrayKlass)k).getBottomKlass():k; >> >> No () needed around k.isObjArrayKlass() >> Spaces needed around the ? and : operators >> >> test/hotspot/jtreg/serviceability/sa/ClhsdbLauncher.java >> >> Copyright year should now be "2017, 2018," >> >> Thanks, >> David >> >>> Tests run: The SA tests pass with Mach5 and jprt. >>> >>> Thanks, >>> Jini. From robin.westberg at oracle.com Fri Feb 9 15:41:20 2018 From: robin.westberg at oracle.com (Robin Westberg) Date: Fri, 9 Feb 2018 16:41:20 +0100 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: <5A7B65E1.5050003@oracle.com> References: <5A7B65E1.5050003@oracle.com> Message-ID: Hi Erik, > On 7 Feb 2018, at 21:47, Erik Gahlin wrote: > > Hi Robin, > > I can sponsor this. > > I wonder if we should change the name of the event to "Shutdown" instead? It will give us flexibility to add other shutdown related fields in the future. Makes sense, I?ll rename it. > Could you change the label and field to "Status Code" and statusCode. Event fields should have headline-style capitalization and statusCode allows us to add other status related information in the future. Removed the status field now, I?ll keep it in mind if it gets added back in the future. Best regards, Robin > > Thanks > Erik > >> Hi all, >> >> Please review the following change that adds an event-based tracing event that is generated when the VM shuts down. The intent is to capture shutdowns that occur after the VM has been properly initialized (as initialization problems would most likely mean that the tracing framework hasn?t been properly started either). >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-8041626 >> Webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00/ >> Testing: hs-tier1,hs-tier2,jdk-tier1,jdk-tier2 >> >> Best regards, >> Robin > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robin.westberg at oracle.com Fri Feb 9 15:41:27 2018 From: robin.westberg at oracle.com (Robin Westberg) Date: Fri, 9 Feb 2018 16:41:27 +0100 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: <5622d8af-d0d6-e77a-f7c1-f2c474f89763@oracle.com> References: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> <5622d8af-d0d6-e77a-f7c1-f2c474f89763@oracle.com> Message-ID: <3EDE8C81-4FBD-4D40-A798-5F71CF7DD2B8@oracle.com> Hi David, > On 9 Feb 2018, at 05:22, David Holmes wrote: > > Hi Robin, > >> Right, almost all the runtime changes are made in order to try to figure out what the process exit code from the launcher will eventually be. For example, the launcher returns 1 if the main thread exited with an unhandled exception, but 0 otherwise. But I actually agree that this information is probably only of marginal use (it could always be captured from wherever Java is launched if someone really wants it), so it is perhaps not worth the trouble. > > Yes and that particular example of launcher behaviour is an archaic throwback to C programming - where end of "main" means end of the program - IMHO. I don't think it is of interest - and certainly not worth jumping through so many hoops to make the info available. > >> Discussed it a bit with Erik Gahlin, and perhaps the best option here is to simply remove the status code field from the event, that would simplify things and make the code you mention above go away. > > That sounds good to me. :) > >>> It is unfortunate that you need to add beforeHalt to deal with the event mechanism itself being turned off (?) by a shutdown hook. That would seem to potentially lose a lot of event information given hooks can run in arbitrary order and execute arbitrary Java code. And essentially you end up recording the initial reason shutdown started, though potentially it could end up terminating for a different reason. >> In this case I think it actually conveys useful information if you are trying to diagnose an unexpected shutdown. It should be useful to find the initial request of an orderly shutdown, even if something else happens during the shutdown sequence like a finalizer calling exit (in which case you could possibly end up with two shutdown events, but both may contain interesting information). > > Yes that is useful. But I find the need to code things the way they are, unfortunate. But given current constraints, so be it. Okay, I?ve removed the code related to the status field, certainly makes the change a bit less intrusive. Updated webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.01/ Incremental: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00-01/ Best regards, Robin > > Thanks, > David > >> Best regards, >> Robin >>> Let's see what others think ... >>> >>> Thanks, >>> David >>> >>> On 8/02/2018 1:18 AM, Robin Westberg wrote: >>>> Hi all, >>>> Please review the following change that adds an event-based tracing event that is generated when the VM shuts down. The intent is to capture shutdowns that occur after the VM has been properly initialized (as initialization problems would most likely mean that the tracing framework hasn?t been properly started either). >>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8041626 >>>> Webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00/ >>>> Testing: hs-tier1,hs-tier2,jdk-tier1,jdk-tier2 >>>> Best regards, >>>> Robin -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Mon Feb 12 07:07:03 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 12 Feb 2018 17:07:03 +1000 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: <3EDE8C81-4FBD-4D40-A798-5F71CF7DD2B8@oracle.com> References: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> <5622d8af-d0d6-e77a-f7c1-f2c474f89763@oracle.com> <3EDE8C81-4FBD-4D40-A798-5F71CF7DD2B8@oracle.com> Message-ID: <0362f486-b446-4023-9914-0469d6274b7a@oracle.com> Hi Robin, On 10/02/2018 1:41 AM, Robin Westberg wrote: > Hi David, > >> On 9 Feb 2018, at 05:22, David Holmes > > wrote: >> >> Hi Robin, >> >>> Right, almost all the runtime changes are made in order to try to >>> figure out what the process exit code from the launcher will >>> eventually be. For example, the launcher returns 1 if the main thread >>> exited with an unhandled exception, but 0 otherwise. But I actually >>> agree that this information is probably only of marginal use (it >>> could always be captured from wherever Java is launched if someone >>> really wants it), so it is perhaps not worth the trouble. >> >> Yes and that particular example of launcher behaviour is an archaic >> throwback to C programming - where end of "main" means end of the >> program - IMHO. I don't think it is of interest - and certainly not >> worth jumping through so many hoops to make the info available. >> >>> Discussed it a bit with Erik Gahlin, and perhaps the best option here >>> is to simply remove the status code field from the event, that would >>> simplify things and make the code you mention above go away. >> >> That sounds good to me. :) >> >>>> It is unfortunate that you need to add beforeHalt to deal with the >>>> event mechanism itself being turned off (?) by a shutdown hook. That >>>> would seem to potentially lose a lot of event information given >>>> hooks can run in arbitrary order and execute arbitrary Java code. >>>> And essentially you end up recording the initial reason shutdown >>>> started, though potentially it could end up terminating for a >>>> different reason. >>> In this case I think it actually conveys useful information if you >>> are trying to diagnose an unexpected shutdown. It should be useful to >>> find the initial request of an orderly shutdown, even if something >>> else happens during the shutdown sequence like a finalizer calling >>> exit (in which case you could possibly end up with two shutdown >>> events, but both may contain interesting information). >> >> Yes that is useful. But I find the need to code things the way they >> are, unfortunate. But given current constraints, so be it. > > Okay, I?ve removed the code related to the status field, certainly makes > the change a bit less intrusive. > > Updated webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.01/ > Incremental: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00-01/ This looks much cleaner/neater to me - thanks. Tool Nit: the git-webrev tool you use has an annoying quirk compared to our own webrev tool - it first reports the number of files changed even when it is reporting on an individual file! I'm used to seeing a first number that represents number of lines changes - and the "1" in this case throws me as I like to quickly look at the size of the change for each file when deciding what order to look at them. :( Thanks, David > Best regards, > Robin > >> >> Thanks, >> David >> >>> Best regards, >>> Robin >>>> Let's see what others think ... >>>> >>>> Thanks, >>>> David >>>> >>>> On 8/02/2018 1:18 AM, Robin Westberg wrote: >>>>> Hi all, >>>>> Please review the following change that adds an event-based tracing >>>>> event that is generated when the VM shuts down. The intent is to >>>>> capture shutdowns that occur after the VM has been properly >>>>> initialized (as initialization problems would most likely mean that >>>>> the tracing framework hasn?t been properly started either). >>>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8041626 >>>>> Webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00/ >>>>> Testing: hs-tier1,hs-tier2,jdk-tier1,jdk-tier2 >>>>> Best regards, >>>>> Robin > From Alan.Bateman at oracle.com Mon Feb 12 08:02:42 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 12 Feb 2018 08:02:42 +0000 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: <0362f486-b446-4023-9914-0469d6274b7a@oracle.com> References: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> <5622d8af-d0d6-e77a-f7c1-f2c474f89763@oracle.com> <3EDE8C81-4FBD-4D40-A798-5F71CF7DD2B8@oracle.com> <0362f486-b446-4023-9914-0469d6274b7a@oracle.com> Message-ID: On 12/02/2018 07:07, David Holmes wrote: > >> Okay, I?ve removed the code related to the status field, certainly >> makes the change a bit less intrusive. >> >> Updated webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.01/ >> Incremental: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00-01/ > > This looks much cleaner/neater to me - thanks. > The updates to Runtime/Shutdown seems okay. -Alan From sharath.ballal at oracle.com Mon Feb 12 09:12:11 2018 From: sharath.ballal at oracle.com (Sharath Ballal) Date: Mon, 12 Feb 2018 01:12:11 -0800 (PST) Subject: RFR: JDK-8196930 - [Testbug] serviceability/sa/ClhsdbFindPC.java fails to find expected output Message-ID: Hello, Requesting reviews for the issue: ID: https://bugs.openjdk.java.net/browse/JDK-8196930 Webrev: http://cr.openjdk.java.net/~sballal/8196930/webrev.00/ Tests run: The SA tests pass with Mach5. Thanks, Sharath -------------- next part -------------- An HTML attachment was scrubbed... URL: From erik.osterlund at oracle.com Mon Feb 12 14:05:04 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 12 Feb 2018 15:05:04 +0100 Subject: JDK-8171119: Low-Overhead Heap Profiling In-Reply-To: References: Message-ID: <5A819F10.8040201@oracle.com> Hi JC, Sorry for the delayed reply. Inlined answers: On 2018-02-06 00:04, JC Beyler wrote: > Hi Erik, > > (Renaming this to be folded into the newly renamed thread :)) > > First off, thanks a lot for reviewing the webrev! I appreciate it! > > I updated the webrev to: > http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.05a/ > > And the incremental one is here: > http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.04_05a/ > > It contains: > - The change for since from 9 to 11 for the jvmti.xml > - The use of the OrderAccess for initialized > - Clearing the oop > > I also have inlined my answers to your comments. The biggest question > will come from the multiple *_end variables. A bit of the logic there > is due to handling the slow path refill vs fast path refill and > checking that the rug was not pulled underneath the slowpath. I > believe that a previous comment was that TlabFastRefill was going to > be deprecated. > > If this is true, we could revert this code a bit and just do a : if > TlabFastRefill is enabled, disable this. And then deprecate that when > TlabFastRefill is deprecated. > > This might simplify this webrev and I can work on a follow-up that > either: removes TlabFastRefill if Robbin does not have the time to do > it or add the support to the assembly side to handle this correctly. > What do you think? I support removing TlabFastRefill, but I think it is good to not depend on that happening first. > Now, below, inlined are my answers: > > On Fri, Feb 2, 2018 at 8:44 AM, Erik ?sterlund > wrote: >> Hi JC, >> >> Hope I am reviewing the right version of your work. Here goes... >> >> src/hotspot/share/gc/shared/collectedHeap.inline.hpp: >> >> 159 AllocTracer::send_allocation_outside_tlab(klass, result, size * >> HeapWordSize, THREAD); >> 160 >> 161 THREAD->tlab().handle_sample(THREAD, result, size); >> 162 return result; >> 163 } >> >> Should not call tlab()->X without checking if (UseTLAB) IMO. >> > Done! More about this later. > >> src/hotspot/share/gc/shared/threadLocalAllocBuffer.cpp: >> >> So first of all, there seems to quite a few ends. There is an "end", a "hard >> end", a "slow path end", and an "actual end". Moreover, it seems like the >> "hard end" is actually further away than the "actual end". So the "hard end" >> seems like more of a "really definitely actual end" or something. I don't >> know about you, but I think it looks kind of messy. In particular, I don't >> feel like the name "actual end" reflects what it represents, especially when >> there is another end that is behind the "actual end". >> >> 413 HeapWord* ThreadLocalAllocBuffer::hard_end() { >> 414 // Did a fast TLAB refill occur? >> 415 if (_slow_path_end != _end) { >> 416 // Fix up the actual end to be now the end of this TLAB. >> 417 _slow_path_end = _end; >> 418 _actual_end = _end; >> 419 } >> 420 >> 421 return _actual_end + alignment_reserve(); >> 422 } >> >> I really do not like making getters unexpectedly have these kind of side >> effects. It is not expected that when you ask for the "hard end", you >> implicitly update the "slow path end" and "actual end" to new values. >> > As I said, a lot of this is due to the FastTlabRefill. If I make this > not supporting FastTlabRefill, this goes away. The reason the system > needs to update itself at the get is that you only know at that get if > things have shifted underneath the tlab slow path. I am not sure of > really better names (naming is hard!), perhaps we could do these > names: > > - current_tlab_end // Either the allocated tlab end or a sampling point > - last_allocation_address // The end of the tlab allocation > - last_slowpath_allocated_end // In case a fast refill occurred the > end might have changed, this is to remember slow vs fast past refills > > the hard_end method can be renamed to something like: > tlab_end_pointer() // The end of the lab including a bit of > alignment reserved bytes Those names sound better to me. Could you please provide a mapping from the old names to the new names so I understand which one is which please? This is my current guess of what you are proposing: end -> current_tlab_end actual_end -> last_allocation_address slow_path_end -> last_slowpath_allocated_end hard_end -> tlab_end_pointer I would prefer this naming: end -> slow_path_end // the end for taking a slow path; either due to sampling or refilling actual_end -> allocation_end // the end for allocations slow_path_end -> last_slow_path_end // last address for slow_path_end (as opposed to allocation_end) hard_end -> reserved_end // the end of the reserved space of the TLAB About setting things in the getter... that still seems like a very unpleasant thing to me. It would be better to inspect the call hierarchy and explicitly update the ends where they need updating, and assert in the getter that they are in sync, rather than implicitly setting various ends as a surprising side effect in a getter. It looks like the call hierarchy is very small. With my new naming convention, reserved_end() would presumably return _allocation_end + alignment_reserve(), and have an assert checking that _allocation_end == _last_slow_path_allocation_end, complaining that this invariant must hold, and that a caller to this function, such as make_parsable(), must first explicitly synchronize the ends as required, to honor that invariant. > Not sure it's better but before updating the webrev, I wanted to try > to get input/consensus :) > > (Note hard_end was always further off than end). > >> src/hotspot/share/prims/jvmti.xml: >> >> 10357 >> 10358 >> 10359 Can sample the heap. >> 10360 If this capability is enabled then the heap sampling methods >> can be called. >> 10361 >> 10362 >> >> Looks like this capability should not be "since 9" if it gets integrated >> now. > Updated now to 11, crossing my fingers :) > > >> src/hotspot/share/runtime/heapMonitoring.cpp: >> >> 448 if (is_alive->do_object_b(value)) { >> 449 // Update the oop to point to the new object if it is still >> alive. >> 450 f->do_oop(&(trace.obj)); >> 451 >> 452 // Copy the old trace, if it is still live. >> 453 _allocated_traces->at_put(curr_pos++, trace); >> 454 >> 455 // Store the live trace in a cache, to be served up on /heapz. >> 456 _traces_on_last_full_gc->append(trace); >> 457 >> 458 count++; >> 459 } else { >> 460 // If the old trace is no longer live, add it to the list of >> 461 // recently collected garbage. >> 462 store_garbage_trace(trace); >> 463 } >> >> In the case where the oop was not live, I would like it to be explicitly >> cleared. > Done I think how you wanted it. Let me know because I'm not familiar > with the RootAccess API. I'm unclear if I'm doing this right or not so > reviews of these parts are highly appreciated. Robbin had talked of > perhaps later pushing this all into a OopStorage, should I do this now > do you think? Or can that wait a second webrev later down the road? I think using handles can and should be done later. You can use the Access API now. I noticed that you are missing an #include "oops/access.inline.hpp" in your heapMonitoring.cpp file. > + Did I clear it the way you wanted me to or were you thinking of > something else? That is precisely how I wanted it to be cleared. Thanks. > + Final question here, seems like if I were to want to not do the > f->do_oop directly on the trace.obj, I'd need to do something like: > > f->do_oop(&value); > ... > trace->store_oop(value); > > to update the oop internally. Is that right/is that one of the > advantages of going to the Oopstorage sooner than later? I think you really want to do the do_oop on the root directly. Is there a particular reason why you would not want to do that? Otherwise, yes - the benefit with using the handle approach is that you do not need to call do_oop explicitly in your code. > >> Also I see a lot of concurrent-looking use of the following field: >> 267 volatile bool _initialized; >> >> Please note that the "volatile" qualifier does not help with reordering >> here. Reordering between volatile and non-volatile fields is completely free >> for both compiler and hardware, except for windows with MSVC, where volatile >> semantics is defined to use acquire/release semantics, and the hardware is >> TSO. But for the general case, I would expect this field to be stored with >> OrderAccess::release_store and loaded with OrderAccess::load_acquire. >> Otherwise it is not thread safe. > Because everything is behind a mutex, I wasn't really worried about > this. I have a test that has multiple threads trying to hit this > corner case and it passes. > > However, to be paranoid, I updated it to using the OrderAccess API > now, thanks! Let me know what you think there too! If it is indeed always supposed to be read and written under a mutex, then I would strongly prefer to have it accessed as a normal non-volatile member, and have an assertion that given lock is held or we are in a safepoint, as we do in many other places. Something like this: assert(HeapMonitorStorage_lock->owned_by_self() || (SafepointSynchronize::is_at_safepoint() && Thread::current()->is_VM_thread()), "this should not be accessed concurrently"); It would be confusing to people reading the code if there are uses of OrderAccess that are actually always protected under a mutex. >> As a kind of meta comment, I wonder if it would make sense to add sampling >> for non-TLAB allocations. Seems like if someone is rapidly allocating a >> whole bunch of 1 MB objects that never fit in a TLAB, I might still be >> interested in seeing that in my traces, and not get surprised that the >> allocation rate is very high yet not showing up in any profiles. >> > That is handled by the handle_sample where you wanted me to put a > UseTlab because you hit that case if the allocation is too big. I see. It was not obvious to me that non-TLAB sampling is done in the TLAB class. That seems like an abstraction crime. What I wanted in my previous comment was that we do not call into the TLAB when we are not using TLABs. If there is sampling logic in the TLAB that is used for something else than TLABs, then it seems like that logic simply does not belong inside of the TLAB. It should be moved out of the TLAB, and instead have the TLAB call this common abstraction that makes sense. Hope I have answered your questions and that my feedback makes sense to you. Thanks, /Erik > I double checked by changing the test > http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.05a/raw_files/new/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatObjectCorrectnessTest.java > > to use a smaller Tlab (2048) and made the object bigger and it goes > through that and passes. > > Thanks again for your review and I look forward to your pointers for > the questions I now have raised! > Jc > > > > > > > > > >> Thanks, >> /Erik >> >> >> On 2018-01-26 06:45, JC Beyler wrote: >>> Thanks Robbin for the reviews :) >>> >>> The new full webrev is here: >>> http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.03/ >>> The incremental webrev is here: >>> http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.02_03/ >>> >>> I inlined my answers: >>> >>> On Thu, Jan 25, 2018 at 1:15 AM, Robbin Ehn wrote: >>>> Hi JC, great to see another revision! >>>> >>>> #### >>>> heapMonitoring.cpp >>>> >>>> StackTraceData should not contain the oop for 'safety' reasons. >>>> When StackTraceData is moved from _allocated_traces: >>>> L452 store_garbage_trace(trace); >>>> it contains a dead oop. >>>> _allocated_traces could instead be a tupel of oop and StackTraceData thus >>>> dead oops are not kept. >>> Done I used inheritance to make the copier work regardless but the >>> idea is the same. >>> >>>> You should use the new Access API for loading the oop, something like >>>> this: >>>> RootAccess::load(...) >>>> I don't think you need to use Access API for clearing the oop, but it >>>> would >>>> look nicer. And you shouldn't probably be using: >>>> Universe::heap()->is_in_reserved(value) >>> I am unfamiliar with this but I think I did do it like you wanted me >>> to (all tests pass so that's a start). I'm not sure how to clear the >>> oop exactly, is there somewhere that does that, which I can use to do >>> the same? >>> >>> I removed the is_in_reserved, this came from our internal version, I >>> don't know why it was there but my tests work without so I removed it >>> :) >>> >>> >>>> The lock: >>>> L424 MutexLocker mu(HeapMonitorStorage_lock); >>>> Is not needed as far as I can see. >>>> weak_oops_do is called in a safepoint, no TLAB allocation can happen and >>>> JVMTI thread can't access these data-structures. Is there something more >>>> to >>>> this lock that I'm missing? >>> Since a thread can call the JVMTI getLiveTraces (or any of the other >>> ones), it can get to the point of trying to copying the >>> _allocated_traces. I imagine it is possible that this is happening >>> during a GC or that it can be started and a GC happens afterwards. >>> Therefore, it seems to me that you want this protected, no? >>> >>> >>>> #### >>>> You have 6 files without any changes in them (any more): >>>> g1CollectedHeap.cpp >>>> psMarkSweep.cpp >>>> psParallelCompact.cpp >>>> genCollectedHeap.cpp >>>> referenceProcessor.cpp >>>> thread.hpp >>>> >>> Done. >>> >>>> #### >>>> I have not looked closely, but is it possible to hide heap sampling in >>>> AllocTracer ? (with some minor changes to the AllocTracer API) >>>> >>> I am imagining that you are saying to move the code that does the >>> sampling code (change the tlab end, do the call to HeapMonitoring, >>> etc.) into the AllocTracer code itself? I think that is right and I'll >>> look if that is possible and prepare a webrev to show what would be >>> needed to make that happen. >>> >>>> #### >>>> Minor nit, when declaring pointer there is a little mix of having the >>>> pointer adjacent by type name and data name. (Most hotspot code is by >>>> type >>>> name) >>>> E.g. >>>> heapMonitoring.cpp:711 jvmtiStackTrace *trace = .... >>>> heapMonitoring.cpp:733 Method* m = vfst.method(); >>>> (not just this file) >>>> >>> Done! >>> >>>> #### >>>> HeapMonitorThreadOnOffTest.java:77 >>>> I would make g_tmp volatile, otherwise the assignment in loop may >>>> theoretical be skipped. >>>> >>> Also done! >>> >>> Thanks again! >>> Jc >> From erik.helin at oracle.com Mon Feb 12 15:39:35 2018 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 12 Feb 2018 16:39:35 +0100 Subject: RFR (S): 8195115: G1 Old Gen MemoryPool CollectionUsage.used values don't reflect mixed GC results In-Reply-To: <3922628B-81DE-4904-AEFE-F4163EB1E655@amazon.com> References: <3A67BF1B-CE35-4759-B5B4-959C24020A45@amazon.com> <22ceea57-7d27-8350-4457-21f765cb3d0f@oracle.com> <878D6678-84CB-471E-B126-AF14B1BC8EB9@amazon.com> <727f9bcb-9404-df32-917f-70aca6e31cb6@oracle.com> <4BA32239-D645-4B1A-A58E-502258C8FFB7@amazon.com> <997a766f-9f4b-f192-ae87-03f4c8d2666d@oracle.com> <64793ff3-4805-79f9-4e72-6e469cd511dc@oracle.com> <15754d34-f3c7-6431-7abf-05214bd6b9d2@oracle.com> <93417506-8e12-6ad3-690a-439e36719652@oracle.com> <99C937B6-F67B-4F4A-ACC8-444B5F83932B@amazon.com> <3922628B-81DE-4904-AEFE-F4163EB1E655@amazon.com> Message-ID: <8444d766-8adb-013f-f75b-b5d4420635df@oracle.com> Hi Paul, sorry, I'm just back from a really busy week with two conferences. I will take a look tomorrow. Thanks, Erik On 02/07/2018 11:17 PM, Hohensee, Paul wrote: > I?ve filed https://bugs.openjdk.java.net/browse/JDK-8196989 : Revamp G1 JMX MemoryPool and GarbageCollector MXBean definitions and the corresponding CSR https://bugs.openjdk.java.net/browse/JDK-8196991. > > Would you please comment on the CSR, and on the original CSR https://bugs.openjdk.java.net/browse/JDK-8196719? > > Thanks, > > Paul > > On 2/2/18, 1:20 PM, "hotspot-gc-dev on behalf of Hohensee, Paul" wrote: > > And, can a get a reviewer or reviewers for the CSR? > > Thanks, > > Paul > > On 2/2/18, 1:14 PM, "Hohensee, Paul" wrote: > > +hotspot-gc-use. > > I?ve filed a CSR for the current patch, see https://bugs.openjdk.java.net/browse/JDK-8196719. Here?s the argument in favor. > > > It?s possible that there are JDK8 users that rely on current G1 old gen CollectionUsage behavior, but imo it?s unlikely because it?s of little use. Perhaps Kirk and others with operational experience can weigh in. > > Let?s think about use cases. G1 full GC?s happen rarely and only under severe pressure, so when they do external reaction is pretty much limited to reducing load so the JVM can get back to a usable steady state, or just restarting the JVM. Old gen CollectionUsage is zero until a full GC occurs, after which its value includes both long-lived objects and any transient data that was in eden and the survivor space. That value doesn?t tell you anything about long term old gen occupancy or survivor size because it lumps them all together. So, it isn?t a useful metric, nor will it be after any subsequent full GCs. The only information it provides is on the first zero to non-zero transition, which just tells you that the JVM is/was in trouble. Further, the effect of the runup to a full GC is SLA violations, which are noticed before the full GC happens, so detecting the first full GC is confirmation, not prediction. > > Conclusion: G1 old gen CollectionUsage is unlikely to be in use in its current form, so changing its definition to something usable is low risk. > > > ?G1 Old Space? is fine, as is ?G1 Archive Space?. Are you ok with the G1 archive space overlapping the G1 old space? Should we add an archive space to the other collectors? If so, how would it be defined and would having it overlap with the old generation as a live prefix be ok? > > "G1 Young Generation" is the currently young+mixed collector. > > You?re right, if one is iterating over all collectors, there will be redundancy if we keep the old ones. I?m usually leery of introducing a flag such as UseG1LegacyMXBeans (I changed the name, since all the interfaces are MXBeans, hope that?s ok) which must be either indefinitely maintained, or go through a deprecation cycle. I don?t see a way out of the ?iterate over all collectors? problem without it though. > > Paul > > On 1/31/18, 3:42 AM, "Erik Helin" wrote: > > On 01/31/2018 02:30 AM, Hohensee, Paul wrote: > > It?s true that my patch doesn?t completely solve the larger problem, but it fixes the most immediately important part of it, particularly for JDK8 where current expected behavior is entrenched. > > Yes, your patch fixes part of the problem, but as I said, can > potentially lead to more confusion. I'm not sure that doing this > behavioral change for a public API in an JDK 8 update release is the > right thing. There are likely users that rely on the memory pool "G1 Old > Gen" only being updated by a full collection (even though that behavior > is not correct), those uses will encounter a new behavior in an update > release with your patch. > > The good thing is that we have very experienced engineers participating > in the CSR process that have much more experience than I have in > evaluating the impact of behavioral changes such as this one. Would you > please file a CSR request for your patch to get their opinion? > > See https://wiki.openjdk.java.net/display/csr/Main for more details > about CSR. > > On 01/31/2018 02:30 AM, Hohensee, Paul wrote: > If we?re going to fix the larger problem, imo we should file another > bug/rfe to do it. I?d be happy to fix that one too, once we have a spec. > > > > What do you think of my suggestions? To summarize: > > > > - Keep the existing memory pools and add humongous and archive pools. > > - Make the archive pool part of the old gen, and generalize it to all collectors. > > - Split humongous regions off from the old gen pool into their own pool. The old gen and humongous pools are disjoint region sets. > > - Keep the existing ?G1 Young Generation? and ?G1 Old Generation? collectors and their associated memory pools (net of this patch). We add the humongous pool to them. > > - Add ?G1 Full? as an alias of the existing ?G1 Old Generation? collector. > > - Add the ?G1 Young?, ?G1 Mixed? and ?G1 Concurrent Cycle? collectors. > > - Set the G1 old gen memory pool max size to ?Xmx, the archive space max size to whatever it is, and the rest of the G1 memory pool max sizes to -1 == undefined, as now. > > > > The resulting memory pools: > > > > ?G1 Eden Space? > > ?G1 Survivor Space? > > ?G1 Old Gen? > > ?G1 Humongous Space? > > ?Archive Space? > > The "Space" suffix is unfortunate, but acceptable. I'm least happy about > the "Gen" suffix for the "G1 Old Gen", since G1's old regions differ > from a generation in the traditional sense as applied to e.g. Serial, > Parallel and CMS. I would be more happy to use a consistent naming > scheme in the form of "G1 Old Space" (having only one pool ending "Gen" > begs the question how it differs from the others ending in "Space"). > Again, we could introduce a flag such as -XX:+UseG1LegacyPoolsAndBeans > for those that really wants the old names. > > "Archive Space" should be named "G1 Archive Space" since it differs in > implementation from the other collectors. It would be unfortunate if > users thought they could change collector and the "Archive Space" memory > pool would keep the same behavior. > > > The resulting collectors and their memory pools: > > > > ?G1 Young Generation? (the existing young/mixed collector), ?G1 Old Generation?/?G1 Full?, ?G1 Mixed? > > - ?G1 Eden Space? > > - ?G1 Survivor Space? > > - ?G1 Old Gen? > > - ?G1 Humongous Space? > > ?G1 Young? > > - ?G1 Eden Space? > > - ?G1 Survivor Space? > > - ?G1 Humongous Space? > > ?G1 Concurrent Cycle? > > - ?G1 Old Gen? > > - ?G1 Humongous Space? > > > > I?m not religious about the names, but I like my suggestions. :) > I think it will be confusing for users to have both "G1 Old Generation" > and "G1 Full", particularly for tools iterating over all > GarbageCollectorMXBeans. There is no way to indicate that a > GarbageCollectorMXBeans is an alias of another GarbageCollectorMXBean (I > thought about such a solution as well). > > I guess I don't follow what the GarbageCollectorMXBean "G1 Young > Generation" is meant to represent? > > > The significant addition to my previous email, and an incompatible change, is splitting humongous regions off from the old gen pool. This means that apps that specifically monitor old gen occupancy will no longer see humongous regions. Monitoring apps that just add up info about all a collector?s pools won?t see a difference. I may be corrected (by Kirk, perhaps), but imo it?s not as bad a compatibility issue as one might think, because the type of app that uses a lot of humongous regions isn?t all that common. E.g., apps that cache data in the heap in the form of large compressed arrays, and apps with large hashmap bucket list arrays. The heaps such apps use are very often large enough to use 32mb regions, hence need really big objects to actually allocate humongous regions. > > So why not enable backwards compatibility by allowing a user to set the > flag -XX:+UseG1LegacyPoolsAndBeans? It is not that cumbersome for us to > maintain the current definition of memory pools and collectors. Such a > flag allows us to start over and do this right and a user who relies on > the current behavior can get that by just setting a flag. Doing such a > change in a major release also allows us to clearly highlight the change > in the release notes (users are more prepared for larger changes in a > major release and that they might have to add flags to keep old behavior). > > It is not uncommon for memory pools to change in major releases. The > perm gen pool was removed in JDK 8, the default pools changed when > Parallel Old became default old collector way back in JDK 7 and changed > again when G1 became the default collector in JDK 9. > > Thanks, > Erik > > > Thanks, > > > > Paul > > > > On 1/30/18, 5:51 AM, "Erik Helin" wrote: > > > > On 01/30/2018 03:07 AM, Hohensee, Paul wrote: > > > That?s one reviewer who?s ok with a short term patch. Anyone else? And, > > > any reviewers for said short term patch? :) > > > > Well, the patch is not really complete as it is. The problem is the > > definitions of the MemoryPoolMXBeans and GarbageCollectorMXBeans, which, > > as I tried to hint at in my first email, is a mess for G1. The names and > > implementations of these MemoryPoolMXBeans and GarbageCollectionMXBeans > > for G1 are very old, G1 has changed a lot since those were implemented > > (hence my suggestion for finally fixing this). > > > > The issue with your patch is that the MemoryPoolMXBean named "G1 Old > > Gen" consists of both old and humongous regions (it will also include > > archive regions). Old regions can be collected by mixed, concurrent and > > full collections. Humongous regions can be collected by young, mixed or > > full collections and the concurrent cycle. Archive regions will never be > > collected. Your patch will update the pool in the case of a mixed > > collection collecting old regions or humongous regions, but misses the > > following cases: > > - a young collection collecting humongous regions > > - a concurrent cycle collecting humongous regions > > - a concurrent cycle collecting old regions > > > > Unfortunately I could not come up with a good way to solve the above > > without re-designing the pools. I'm not sure about accepting your patch > > as is, since it might cause even more confusion for users compared to > > the current (already confusing) situation. > > > > One idea we have discussed is to implement the re-design but also add a > > flag, -XX:+UseG1LegacyPoolsAndBeans (false by default), to allow for a > > smoother transition. Would that solution work for you? > > > > Thanks, > > Erik > > > > > Thanks, > > > > > > Paul > > > > > > *From: *mandy chung > > > *Organization: *Oracle Corporation > > > *Date: *Monday, January 29, 2018 at 1:41 PM > > > *To: *"Hohensee, Paul" > > > *Cc: *"serviceability-dev at openjdk.java.net" > > > , "hotspot-gc-dev at openjdk.java.net" > > > > > > *Subject: *Re: RFR (S): 8195115: G1 Old Gen MemoryPool > > > CollectionUsage.used values don't reflect mixed GC results > > > > > > I created JDK-8196362 to look into whether it makes sense to provide > > > some categorization to differentiate eden space vs the heap space for > > > long-lived objects. > > > > > > W.r.t. to JDK-8195115, I have to defer to GC team to comment on the > > > mixed collection update. If they are okay, I have no objection to > > > implement a short-term fix and do the proper G1 memory pools as a > > > separate patch. > > > > > > Mandy > > > > > > On 1/29/18 1:02 PM, Hohensee, Paul wrote: > > > > > > We don?t use getType, and you guessed correctly: we use the memory > > > pool name as an indicator of the specific characteristics of a > > > memory pool, in particular eden. > > > > > > What we want is an indication of long term heap occupancy. We > > > calculate it using CollectionUsage for non-eden heap memory pools, > > > regardless of collector. We don?t use JMX notification, rather we > > > periodically poll CollectionUsage for memory pools with names that > > > contain ?Old?, ?Tenured?, or ?Survivor?. We get the memory pools > > > from the GarbageCollectorMXBeans (we don?t care what the collector > > > names are). For the named memory pools, we sum CollectionUsage.used > > > and divide by the sum of CollectionUsage.max to get a long term heap > > > occupancy percentage. We don?t want to include eden because it?s > > > really just an allocation buffer and not part of the storage for > > > long-lived objects. I suppose we could use a negative test instead > > > by using all memory pools with names that don?t include ?Eden?. > > > > > > The bug is that the ?G1 Old Gen? memory pool isn?t being updated > > > when the ?G1 Young Generation? collector runs a mixed collection. As > > > far as JMX is concerned, that collector only knows about eden and > > > the survivor space. The patch adds the old gen to the memory pools > > > it knows about and has mixed collections update the old gen?s > > > CollectionUsage. > > > > > > I managed to get a submit repo run to succeed last week and it found > > > a problem. I?ve uploaded a new webrev that fixes the failure of the > > > jtreg test TestMemoryMXBeansAndPoolsPresence.java due to the young > > > gen collector being expected to know only about eden and the > > > survivor space. > > > > > > http://cr.openjdk.java.net/~phh/8195115/webrev.hs.01/ > > > > > > > > > Waiting on the submit repo to come back with a result on it. > > > > > > Thanks, > > > > > > Paul > > > > > > *From: *mandy chung > > > > > > *Organization: *Oracle Corporation > > > *Date: *Monday, January 29, 2018 at 10:52 AM > > > *To: *"Hohensee, Paul" > > > , Erik Helin > > > , David Holmes > > > > > > *Cc: *"serviceability-dev at openjdk.java.net" > > > > > > > > > , > > > "hotspot-gc-dev at openjdk.java.net" > > > > > > > > > > > > *Subject: *Re: RFR (S): 8195115: G1 Old Gen MemoryPool > > > CollectionUsage.used values don't reflect mixed GC results > > > > > > On 1/29/18 10:35 AM, mandy chung wrote: > > > > > > Thanks for the reply Paul. Try to understand a little more on > > > the specific from gc-specific memory pool you depend on. > > > > > > On 1/29/18 8:27 AM, Hohensee, Paul wrote: > > > > > > A name change would affect Amazon?s heap monitoring, and > > > thus I expect it would affect other users as well. > > > > > > As long as there are gc-specific memory pools, we?re going > > > to need to be able to identify them, and right now that?s > > > done via name. > > > > > > > > > MemoryPoolMXBean::getType returns "heap" memory type for > > > GC-specific memory pools. Are you using this method? Do you > > > use the name to build in specific characteristic of a memory > > > pool (e.g. eden vs old gen)? > > > > > > > > > > > > > > > All the mxbeans are identified by name, so that?s a general > > > design principle. The only way I can think of to get rid of > > > name dependency would be to figure out what abstract metrics > > > users want to monitor and implement them for all collectors. > > > HeapUsage (instantaneous occupancy) is one, CollectionUsage > > > (long-lived occupancy) is another, both of these for the > > > entire heap, not just particular memory pools. > > > > > > > > > The sum of HeapUsage and CollectionUsage of all heap memory > > > pools was expected to give an incorrect approximation for the > > > entire heap usage. Are you seeing issue/bug with the sum result? > > > > > > > > > typo: s/an incorrect approximation/an approximation. > > > > > > Mandy > > > > > > > > > > > > Mandy > > > > > > > > > > > > That said, imo there will always be a demand for the ability > > > to get collector and memory pool specific details, so I > > > don?t see a way to get around providing named entities. > > > > > > Paul > > > > > > *From: *mandy chung > > > > > > *Organization: *Oracle Corporation > > > *Date: *Friday, January 26, 2018 at 2:38 PM > > > *To: *"Hohensee, Paul" > > > , Erik Helin > > > , > > > David Holmes > > > > > > *Cc: *"serviceability-dev at openjdk.java.net" > > > > > > > > > , > > > "hotspot-gc-dev at openjdk.java.net" > > > > > > > > > > > > *Subject: *Re: RFR (S): 8195115: G1 Old Gen MemoryPool > > > CollectionUsage.used values don't reflect mixed GC results > > > > > > On 1/25/18 1:04 PM, Hohensee, Paul wrote: > > > > > > > > > > The JMX API spec doesn?t specify what the memory pool or > > > garbage > collector names are, but the current names are > > > de-facto part of the > API, so if we change the existing > > > ones, imo a CSR should be filed. > > > > > > The names are implementation details but I can see how an application > > > > > > might be impacted if they depend on it. CSR approval is not strictly > > > > > > necessary while I think filing one to document the change would be > > > > > > good. > > > > > > Does the name change impact any application you know of? I'm trying to > > > > > > understand if any improvement to API is needed so that applications > > > > > > don't need to depend on the names. > > > > > > > > > Mandy > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From parvathi.somashekar at oracle.com Mon Feb 12 19:25:12 2018 From: parvathi.somashekar at oracle.com (Paru Somashekar) Date: Mon, 12 Feb 2018 11:25:12 -0800 Subject: RFR 8196324: Update FilterMatch & FilterNoMatch to use TestScaffold instead of JDIScaffold Message-ID: <5A81EA18.9080808@oracle.com> Hi, Please review fix for JDK-8196324 The includes the following changes: * Use startToMain method of TestScaffold that automatically calls connect() and waitForVMStart() * Since TestScaffold extends TargetAdapter, it now overrides event callbacks directly in the test class rather than implement an anonymous inner class. * Breakpoint handling is done in breakpointReached callback instead of waitForRequestedEvent and subsequently removing it. Bug : https://bugs.openjdk.java.net/browse/JDK-8196324 Review : http://cr.openjdk.java.net/~psomashe/8196324/webrev/ thanks, Paru. From hohensee at amazon.com Mon Feb 12 22:29:35 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Mon, 12 Feb 2018 22:29:35 +0000 Subject: RFR (S): 8195115: G1 Old Gen MemoryPool CollectionUsage.used values don't reflect mixed GC results In-Reply-To: <8444d766-8adb-013f-f75b-b5d4420635df@oracle.com> References: <3A67BF1B-CE35-4759-B5B4-959C24020A45@amazon.com> <22ceea57-7d27-8350-4457-21f765cb3d0f@oracle.com> <878D6678-84CB-471E-B126-AF14B1BC8EB9@amazon.com> <727f9bcb-9404-df32-917f-70aca6e31cb6@oracle.com> <4BA32239-D645-4B1A-A58E-502258C8FFB7@amazon.com> <997a766f-9f4b-f192-ae87-03f4c8d2666d@oracle.com> <64793ff3-4805-79f9-4e72-6e469cd511dc@oracle.com> <15754d34-f3c7-6431-7abf-05214bd6b9d2@oracle.com> <93417506-8e12-6ad3-690a-439e36719652@oracle.com> <99C937B6-F67B-4F4A-ACC8-444B5F83932B@amazon.com> <3922628B-81DE-4904-AEFE-F4163EB1E655@amazon.com> <8444d766-8adb-013f-f75b-b5d4420635df@oracle.com> Message-ID: <5768E4BF-5556-455B-899B-94A67BF940A0@amazon.com> Thanks, very much appreciated. Paul On 2/12/18, 7:42 AM, "Erik Helin" wrote: Hi Paul, sorry, I'm just back from a really busy week with two conferences. I will take a look tomorrow. Thanks, Erik On 02/07/2018 11:17 PM, Hohensee, Paul wrote: > I?ve filed https://bugs.openjdk.java.net/browse/JDK-8196989 : Revamp G1 JMX MemoryPool and GarbageCollector MXBean definitions and the corresponding CSR https://bugs.openjdk.java.net/browse/JDK-8196991. > > Would you please comment on the CSR, and on the original CSR https://bugs.openjdk.java.net/browse/JDK-8196719? > > Thanks, > > Paul > > On 2/2/18, 1:20 PM, "hotspot-gc-dev on behalf of Hohensee, Paul" wrote: > > And, can a get a reviewer or reviewers for the CSR? > > Thanks, > > Paul > > On 2/2/18, 1:14 PM, "Hohensee, Paul" wrote: > > +hotspot-gc-use. > > I?ve filed a CSR for the current patch, see https://bugs.openjdk.java.net/browse/JDK-8196719. Here?s the argument in favor. > > > It?s possible that there are JDK8 users that rely on current G1 old gen CollectionUsage behavior, but imo it?s unlikely because it?s of little use. Perhaps Kirk and others with operational experience can weigh in. > > Let?s think about use cases. G1 full GC?s happen rarely and only under severe pressure, so when they do external reaction is pretty much limited to reducing load so the JVM can get back to a usable steady state, or just restarting the JVM. Old gen CollectionUsage is zero until a full GC occurs, after which its value includes both long-lived objects and any transient data that was in eden and the survivor space. That value doesn?t tell you anything about long term old gen occupancy or survivor size because it lumps them all together. So, it isn?t a useful metric, nor will it be after any subsequent full GCs. The only information it provides is on the first zero to non-zero transition, which just tells you that the JVM is/was in trouble. Further, the effect of the runup to a full GC is SLA violations, which are noticed before the full GC happens, so detecting the first full GC is confirmation, not prediction. > > Conclusion: G1 old gen CollectionUsage is unlikely to be in use in its current form, so changing its definition to something usable is low risk. > > > ?G1 Old Space? is fine, as is ?G1 Archive Space?. Are you ok with the G1 archive space overlapping the G1 old space? Should we add an archive space to the other collectors? If so, how would it be defined and would having it overlap with the old generation as a live prefix be ok? > > "G1 Young Generation" is the currently young+mixed collector. > > You?re right, if one is iterating over all collectors, there will be redundancy if we keep the old ones. I?m usually leery of introducing a flag such as UseG1LegacyMXBeans (I changed the name, since all the interfaces are MXBeans, hope that?s ok) which must be either indefinitely maintained, or go through a deprecation cycle. I don?t see a way out of the ?iterate over all collectors? problem without it though. > > Paul > > On 1/31/18, 3:42 AM, "Erik Helin" wrote: > > On 01/31/2018 02:30 AM, Hohensee, Paul wrote: > > It?s true that my patch doesn?t completely solve the larger problem, but it fixes the most immediately important part of it, particularly for JDK8 where current expected behavior is entrenched. > > Yes, your patch fixes part of the problem, but as I said, can > potentially lead to more confusion. I'm not sure that doing this > behavioral change for a public API in an JDK 8 update release is the > right thing. There are likely users that rely on the memory pool "G1 Old > Gen" only being updated by a full collection (even though that behavior > is not correct), those uses will encounter a new behavior in an update > release with your patch. > > The good thing is that we have very experienced engineers participating > in the CSR process that have much more experience than I have in > evaluating the impact of behavioral changes such as this one. Would you > please file a CSR request for your patch to get their opinion? > > See https://wiki.openjdk.java.net/display/csr/Main for more details > about CSR. > > On 01/31/2018 02:30 AM, Hohensee, Paul wrote: > If we?re going to fix the larger problem, imo we should file another > bug/rfe to do it. I?d be happy to fix that one too, once we have a spec. > > > > What do you think of my suggestions? To summarize: > > > > - Keep the existing memory pools and add humongous and archive pools. > > - Make the archive pool part of the old gen, and generalize it to all collectors. > > - Split humongous regions off from the old gen pool into their own pool. The old gen and humongous pools are disjoint region sets. > > - Keep the existing ?G1 Young Generation? and ?G1 Old Generation? collectors and their associated memory pools (net of this patch). We add the humongous pool to them. > > - Add ?G1 Full? as an alias of the existing ?G1 Old Generation? collector. > > - Add the ?G1 Young?, ?G1 Mixed? and ?G1 Concurrent Cycle? collectors. > > - Set the G1 old gen memory pool max size to ?Xmx, the archive space max size to whatever it is, and the rest of the G1 memory pool max sizes to -1 == undefined, as now. > > > > The resulting memory pools: > > > > ?G1 Eden Space? > > ?G1 Survivor Space? > > ?G1 Old Gen? > > ?G1 Humongous Space? > > ?Archive Space? > > The "Space" suffix is unfortunate, but acceptable. I'm least happy about > the "Gen" suffix for the "G1 Old Gen", since G1's old regions differ > from a generation in the traditional sense as applied to e.g. Serial, > Parallel and CMS. I would be more happy to use a consistent naming > scheme in the form of "G1 Old Space" (having only one pool ending "Gen" > begs the question how it differs from the others ending in "Space"). > Again, we could introduce a flag such as -XX:+UseG1LegacyPoolsAndBeans > for those that really wants the old names. > > "Archive Space" should be named "G1 Archive Space" since it differs in > implementation from the other collectors. It would be unfortunate if > users thought they could change collector and the "Archive Space" memory > pool would keep the same behavior. > > > The resulting collectors and their memory pools: > > > > ?G1 Young Generation? (the existing young/mixed collector), ?G1 Old Generation?/?G1 Full?, ?G1 Mixed? > > - ?G1 Eden Space? > > - ?G1 Survivor Space? > > - ?G1 Old Gen? > > - ?G1 Humongous Space? > > ?G1 Young? > > - ?G1 Eden Space? > > - ?G1 Survivor Space? > > - ?G1 Humongous Space? > > ?G1 Concurrent Cycle? > > - ?G1 Old Gen? > > - ?G1 Humongous Space? > > > > I?m not religious about the names, but I like my suggestions. :) > I think it will be confusing for users to have both "G1 Old Generation" > and "G1 Full", particularly for tools iterating over all > GarbageCollectorMXBeans. There is no way to indicate that a > GarbageCollectorMXBeans is an alias of another GarbageCollectorMXBean (I > thought about such a solution as well). > > I guess I don't follow what the GarbageCollectorMXBean "G1 Young > Generation" is meant to represent? > > > The significant addition to my previous email, and an incompatible change, is splitting humongous regions off from the old gen pool. This means that apps that specifically monitor old gen occupancy will no longer see humongous regions. Monitoring apps that just add up info about all a collector?s pools won?t see a difference. I may be corrected (by Kirk, perhaps), but imo it?s not as bad a compatibility issue as one might think, because the type of app that uses a lot of humongous regions isn?t all that common. E.g., apps that cache data in the heap in the form of large compressed arrays, and apps with large hashmap bucket list arrays. The heaps such apps use are very often large enough to use 32mb regions, hence need really big objects to actually allocate humongous regions. > > So why not enable backwards compatibility by allowing a user to set the > flag -XX:+UseG1LegacyPoolsAndBeans? It is not that cumbersome for us to > maintain the current definition of memory pools and collectors. Such a > flag allows us to start over and do this right and a user who relies on > the current behavior can get that by just setting a flag. Doing such a > change in a major release also allows us to clearly highlight the change > in the release notes (users are more prepared for larger changes in a > major release and that they might have to add flags to keep old behavior). > > It is not uncommon for memory pools to change in major releases. The > perm gen pool was removed in JDK 8, the default pools changed when > Parallel Old became default old collector way back in JDK 7 and changed > again when G1 became the default collector in JDK 9. > > Thanks, > Erik > > > Thanks, > > > > Paul > > > > On 1/30/18, 5:51 AM, "Erik Helin" wrote: > > > > On 01/30/2018 03:07 AM, Hohensee, Paul wrote: > > > That?s one reviewer who?s ok with a short term patch. Anyone else? And, > > > any reviewers for said short term patch? :) > > > > Well, the patch is not really complete as it is. The problem is the > > definitions of the MemoryPoolMXBeans and GarbageCollectorMXBeans, which, > > as I tried to hint at in my first email, is a mess for G1. The names and > > implementations of these MemoryPoolMXBeans and GarbageCollectionMXBeans > > for G1 are very old, G1 has changed a lot since those were implemented > > (hence my suggestion for finally fixing this). > > > > The issue with your patch is that the MemoryPoolMXBean named "G1 Old > > Gen" consists of both old and humongous regions (it will also include > > archive regions). Old regions can be collected by mixed, concurrent and > > full collections. Humongous regions can be collected by young, mixed or > > full collections and the concurrent cycle. Archive regions will never be > > collected. Your patch will update the pool in the case of a mixed > > collection collecting old regions or humongous regions, but misses the > > following cases: > > - a young collection collecting humongous regions > > - a concurrent cycle collecting humongous regions > > - a concurrent cycle collecting old regions > > > > Unfortunately I could not come up with a good way to solve the above > > without re-designing the pools. I'm not sure about accepting your patch > > as is, since it might cause even more confusion for users compared to > > the current (already confusing) situation. > > > > One idea we have discussed is to implement the re-design but also add a > > flag, -XX:+UseG1LegacyPoolsAndBeans (false by default), to allow for a > > smoother transition. Would that solution work for you? > > > > Thanks, > > Erik > > > > > Thanks, > > > > > > Paul > > > > > > *From: *mandy chung > > > *Organization: *Oracle Corporation > > > *Date: *Monday, January 29, 2018 at 1:41 PM > > > *To: *"Hohensee, Paul" > > > *Cc: *"serviceability-dev at openjdk.java.net" > > > , "hotspot-gc-dev at openjdk.java.net" > > > > > > *Subject: *Re: RFR (S): 8195115: G1 Old Gen MemoryPool > > > CollectionUsage.used values don't reflect mixed GC results > > > > > > I created JDK-8196362 to look into whether it makes sense to provide > > > some categorization to differentiate eden space vs the heap space for > > > long-lived objects. > > > > > > W.r.t. to JDK-8195115, I have to defer to GC team to comment on the > > > mixed collection update. If they are okay, I have no objection to > > > implement a short-term fix and do the proper G1 memory pools as a > > > separate patch. > > > > > > Mandy > > > > > > On 1/29/18 1:02 PM, Hohensee, Paul wrote: > > > > > > We don?t use getType, and you guessed correctly: we use the memory > > > pool name as an indicator of the specific characteristics of a > > > memory pool, in particular eden. > > > > > > What we want is an indication of long term heap occupancy. We > > > calculate it using CollectionUsage for non-eden heap memory pools, > > > regardless of collector. We don?t use JMX notification, rather we > > > periodically poll CollectionUsage for memory pools with names that > > > contain ?Old?, ?Tenured?, or ?Survivor?. We get the memory pools > > > from the GarbageCollectorMXBeans (we don?t care what the collector > > > names are). For the named memory pools, we sum CollectionUsage.used > > > and divide by the sum of CollectionUsage.max to get a long term heap > > > occupancy percentage. We don?t want to include eden because it?s > > > really just an allocation buffer and not part of the storage for > > > long-lived objects. I suppose we could use a negative test instead > > > by using all memory pools with names that don?t include ?Eden?. > > > > > > The bug is that the ?G1 Old Gen? memory pool isn?t being updated > > > when the ?G1 Young Generation? collector runs a mixed collection. As > > > far as JMX is concerned, that collector only knows about eden and > > > the survivor space. The patch adds the old gen to the memory pools > > > it knows about and has mixed collections update the old gen?s > > > CollectionUsage. > > > > > > I managed to get a submit repo run to succeed last week and it found > > > a problem. I?ve uploaded a new webrev that fixes the failure of the > > > jtreg test TestMemoryMXBeansAndPoolsPresence.java due to the young > > > gen collector being expected to know only about eden and the > > > survivor space. > > > > > > http://cr.openjdk.java.net/~phh/8195115/webrev.hs.01/ > > > > > > > > > Waiting on the submit repo to come back with a result on it. > > > > > > Thanks, > > > > > > Paul > > > > > > *From: *mandy chung > > > > > > *Organization: *Oracle Corporation > > > *Date: *Monday, January 29, 2018 at 10:52 AM > > > *To: *"Hohensee, Paul" > > > , Erik Helin > > > , David Holmes > > > > > > *Cc: *"serviceability-dev at openjdk.java.net" > > > > > > > > > , > > > "hotspot-gc-dev at openjdk.java.net" > > > > > > > > > > > > *Subject: *Re: RFR (S): 8195115: G1 Old Gen MemoryPool > > > CollectionUsage.used values don't reflect mixed GC results > > > > > > On 1/29/18 10:35 AM, mandy chung wrote: > > > > > > Thanks for the reply Paul. Try to understand a little more on > > > the specific from gc-specific memory pool you depend on. > > > > > > On 1/29/18 8:27 AM, Hohensee, Paul wrote: > > > > > > A name change would affect Amazon?s heap monitoring, and > > > thus I expect it would affect other users as well. > > > > > > As long as there are gc-specific memory pools, we?re going > > > to need to be able to identify them, and right now that?s > > > done via name. > > > > > > > > > MemoryPoolMXBean::getType returns "heap" memory type for > > > GC-specific memory pools. Are you using this method? Do you > > > use the name to build in specific characteristic of a memory > > > pool (e.g. eden vs old gen)? > > > > > > > > > > > > > > > All the mxbeans are identified by name, so that?s a general > > > design principle. The only way I can think of to get rid of > > > name dependency would be to figure out what abstract metrics > > > users want to monitor and implement them for all collectors. > > > HeapUsage (instantaneous occupancy) is one, CollectionUsage > > > (long-lived occupancy) is another, both of these for the > > > entire heap, not just particular memory pools. > > > > > > > > > The sum of HeapUsage and CollectionUsage of all heap memory > > > pools was expected to give an incorrect approximation for the > > > entire heap usage. Are you seeing issue/bug with the sum result? > > > > > > > > > typo: s/an incorrect approximation/an approximation. > > > > > > Mandy > > > > > > > > > > > > Mandy > > > > > > > > > > > > That said, imo there will always be a demand for the ability > > > to get collector and memory pool specific details, so I > > > don?t see a way to get around providing named entities. > > > > > > Paul > > > > > > *From: *mandy chung > > > > > > *Organization: *Oracle Corporation > > > *Date: *Friday, January 26, 2018 at 2:38 PM > > > *To: *"Hohensee, Paul" > > > , Erik Helin > > > , > > > David Holmes > > > > > > *Cc: *"serviceability-dev at openjdk.java.net" > > > > > > > > > , > > > "hotspot-gc-dev at openjdk.java.net" > > > > > > > > > > > > *Subject: *Re: RFR (S): 8195115: G1 Old Gen MemoryPool > > > CollectionUsage.used values don't reflect mixed GC results > > > > > > On 1/25/18 1:04 PM, Hohensee, Paul wrote: > > > > > > > > > > The JMX API spec doesn?t specify what the memory pool or > > > garbage > collector names are, but the current names are > > > de-facto part of the > API, so if we change the existing > > > ones, imo a CSR should be filed. > > > > > > The names are implementation details but I can see how an application > > > > > > might be impacted if they depend on it. CSR approval is not strictly > > > > > > necessary while I think filing one to document the change would be > > > > > > good. > > > > > > Does the name change impact any application you know of? I'm trying to > > > > > > understand if any improvement to API is needed so that applications > > > > > > don't need to depend on the names. > > > > > > > > > Mandy > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From jcbeyler at google.com Tue Feb 13 05:18:28 2018 From: jcbeyler at google.com (JC Beyler) Date: Mon, 12 Feb 2018 21:18:28 -0800 Subject: JDK-8171119: Low-Overhead Heap Profiling In-Reply-To: <5A819F10.8040201@oracle.com> References: <5A819F10.8040201@oracle.com> Message-ID: Hi Erik, Thanks for your answers, I've now inlined my own answers/comments. I've done a new webrev here: http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.08/ The incremental is here: http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.07_08/ Note to all: - I've been integrating changes from Erin/Serguei/David comments so this webrev incremental is a bit an answer to all comments in one. I apologize for that :) On Mon, Feb 12, 2018 at 6:05 AM, Erik ?sterlund wrote: > Hi JC, > > Sorry for the delayed reply. > > Inlined answers: > > > On 2018-02-06 00:04, JC Beyler wrote: > >> Hi Erik, >> >> (Renaming this to be folded into the newly renamed thread :)) >> >> First off, thanks a lot for reviewing the webrev! I appreciate it! >> >> I updated the webrev to: >> http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.05a/ >> >> And the incremental one is here: >> http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.04_05a/ >> >> It contains: >> - The change for since from 9 to 11 for the jvmti.xml >> - The use of the OrderAccess for initialized >> - Clearing the oop >> >> I also have inlined my answers to your comments. The biggest question >> will come from the multiple *_end variables. A bit of the logic there >> is due to handling the slow path refill vs fast path refill and >> checking that the rug was not pulled underneath the slowpath. I >> believe that a previous comment was that TlabFastRefill was going to >> be deprecated. >> >> If this is true, we could revert this code a bit and just do a : if >> TlabFastRefill is enabled, disable this. And then deprecate that when >> TlabFastRefill is deprecated. >> >> This might simplify this webrev and I can work on a follow-up that >> either: removes TlabFastRefill if Robbin does not have the time to do >> it or add the support to the assembly side to handle this correctly. >> What do you think? >> > > I support removing TlabFastRefill, but I think it is good to not depend on > that happening first. > > I'm slowly pushing on the FastTLABRefill (https://bugs.openjdk.java.net /browse/JDK-8194084), I agree on keeping both separate for now though so that we can think of both differently > Now, below, inlined are my answers: >> >> On Fri, Feb 2, 2018 at 8:44 AM, Erik ?sterlund >> wrote: >> >>> Hi JC, >>> >>> Hope I am reviewing the right version of your work. Here goes... >>> >>> src/hotspot/share/gc/shared/collectedHeap.inline.hpp: >>> >>> 159 AllocTracer::send_allocation_outside_tlab(klass, result, size >>> * >>> HeapWordSize, THREAD); >>> 160 >>> 161 THREAD->tlab().handle_sample(THREAD, result, size); >>> 162 return result; >>> 163 } >>> >>> Should not call tlab()->X without checking if (UseTLAB) IMO. >>> >>> Done! >> > > More about this later. > > > >> src/hotspot/share/gc/shared/threadLocalAllocBuffer.cpp: >>> >>> So first of all, there seems to quite a few ends. There is an "end", a >>> "hard >>> end", a "slow path end", and an "actual end". Moreover, it seems like the >>> "hard end" is actually further away than the "actual end". So the "hard >>> end" >>> seems like more of a "really definitely actual end" or something. I don't >>> know about you, but I think it looks kind of messy. In particular, I >>> don't >>> feel like the name "actual end" reflects what it represents, especially >>> when >>> there is another end that is behind the "actual end". >>> >>> 413 HeapWord* ThreadLocalAllocBuffer::hard_end() { >>> 414 // Did a fast TLAB refill occur? >>> 415 if (_slow_path_end != _end) { >>> 416 // Fix up the actual end to be now the end of this TLAB. >>> 417 _slow_path_end = _end; >>> 418 _actual_end = _end; >>> 419 } >>> 420 >>> 421 return _actual_end + alignment_reserve(); >>> 422 } >>> >>> I really do not like making getters unexpectedly have these kind of side >>> effects. It is not expected that when you ask for the "hard end", you >>> implicitly update the "slow path end" and "actual end" to new values. >>> >>> As I said, a lot of this is due to the FastTlabRefill. If I make this >> not supporting FastTlabRefill, this goes away. The reason the system >> needs to update itself at the get is that you only know at that get if >> things have shifted underneath the tlab slow path. I am not sure of >> really better names (naming is hard!), perhaps we could do these >> names: >> >> - current_tlab_end // Either the allocated tlab end or a sampling >> point >> - last_allocation_address // The end of the tlab allocation >> - last_slowpath_allocated_end // In case a fast refill occurred the >> end might have changed, this is to remember slow vs fast past refills >> >> the hard_end method can be renamed to something like: >> tlab_end_pointer() // The end of the lab including a bit of >> alignment reserved bytes >> > > Those names sound better to me. Could you please provide a mapping from > the old names to the new names so I understand which one is which please? > > This is my current guess of what you are proposing: > > end -> current_tlab_end > actual_end -> last_allocation_address > slow_path_end -> last_slowpath_allocated_end > hard_end -> tlab_end_pointer > > Yes that is correct, that was what I was proposing. > I would prefer this naming: > > end -> slow_path_end // the end for taking a slow path; either due to > sampling or refilling > actual_end -> allocation_end // the end for allocations > slow_path_end -> last_slow_path_end // last address for slow_path_end (as > opposed to allocation_end) > hard_end -> reserved_end // the end of the reserved space of the TLAB > > About setting things in the getter... that still seems like a very > unpleasant thing to me. It would be better to inspect the call hierarchy > and explicitly update the ends where they need updating, and assert in the > getter that they are in sync, rather than implicitly setting various ends > as a surprising side effect in a getter. It looks like the call hierarchy > is very small. With my new naming convention, reserved_end() would > presumably return _allocation_end + alignment_reserve(), and have an assert > checking that _allocation_end == _last_slow_path_allocation_end, > complaining that this invariant must hold, and that a caller to this > function, such as make_parsable(), must first explicitly synchronize the > ends as required, to honor that invariant. > > I've renamed the variables to how you preferred it except for the _end one. I did: current_end last_allocation_address tlab_end_ptr The reason is that the architecture dependent code use the thread.hpp API and it already has tlab included into the name so it becomes tlab_current_end (which is better that tlab_current_tlab_end in my opinion). I also moved the update into a separate method with a TODO that says to remove it when FastTLABRefill is deprecated > > Not sure it's better but before updating the webrev, I wanted to try >> to get input/consensus :) >> >> (Note hard_end was always further off than end). >> >> src/hotspot/share/prims/jvmti.xml: >>> >>> 10357 >>> 10358 >>> 10359 Can sample the heap. >>> 10360 If this capability is enabled then the heap sampling >>> methods >>> can be called. >>> 10361 >>> 10362 >>> >>> Looks like this capability should not be "since 9" if it gets integrated >>> now. >>> >> Updated now to 11, crossing my fingers :) >> >> >> src/hotspot/share/runtime/heapMonitoring.cpp: >>> >>> 448 if (is_alive->do_object_b(value)) { >>> 449 // Update the oop to point to the new object if it is still >>> alive. >>> 450 f->do_oop(&(trace.obj)); >>> 451 >>> 452 // Copy the old trace, if it is still live. >>> 453 _allocated_traces->at_put(curr_pos++, trace); >>> 454 >>> 455 // Store the live trace in a cache, to be served up on >>> /heapz. >>> 456 _traces_on_last_full_gc->append(trace); >>> 457 >>> 458 count++; >>> 459 } else { >>> 460 // If the old trace is no longer live, add it to the list >>> of >>> 461 // recently collected garbage. >>> 462 store_garbage_trace(trace); >>> 463 } >>> >>> In the case where the oop was not live, I would like it to be explicitly >>> cleared. >>> >> Done I think how you wanted it. Let me know because I'm not familiar >> with the RootAccess API. I'm unclear if I'm doing this right or not so >> reviews of these parts are highly appreciated. Robbin had talked of >> perhaps later pushing this all into a OopStorage, should I do this now >> do you think? Or can that wait a second webrev later down the road? >> > > I think using handles can and should be done later. You can use the Access > API now. > I noticed that you are missing an #include "oops/access.inline.hpp" in > your heapMonitoring.cpp file. > > The missing header is there for me so I don't know, I made sure it is present in the latest webrev. Sorry about that. > + Did I clear it the way you wanted me to or were you thinking of >> something else? >> > > That is precisely how I wanted it to be cleared. Thanks. > > + Final question here, seems like if I were to want to not do the >> f->do_oop directly on the trace.obj, I'd need to do something like: >> >> f->do_oop(&value); >> ... >> trace->store_oop(value); >> >> to update the oop internally. Is that right/is that one of the >> advantages of going to the Oopstorage sooner than later? >> > > I think you really want to do the do_oop on the root directly. Is there a > particular reason why you would not want to do that? > Otherwise, yes - the benefit with using the handle approach is that you do > not need to call do_oop explicitly in your code. > > There is no reason except that now we have a load_oop and a get_oop_addr, I was not sure what you would think of that. > >> Also I see a lot of concurrent-looking use of the following field: >>> 267 volatile bool _initialized; >>> >>> Please note that the "volatile" qualifier does not help with reordering >>> here. Reordering between volatile and non-volatile fields is completely >>> free >>> for both compiler and hardware, except for windows with MSVC, where >>> volatile >>> semantics is defined to use acquire/release semantics, and the hardware >>> is >>> TSO. But for the general case, I would expect this field to be stored >>> with >>> OrderAccess::release_store and loaded with OrderAccess::load_acquire. >>> Otherwise it is not thread safe. >>> >> Because everything is behind a mutex, I wasn't really worried about >> this. I have a test that has multiple threads trying to hit this >> corner case and it passes. >> >> However, to be paranoid, I updated it to using the OrderAccess API >> now, thanks! Let me know what you think there too! >> > > If it is indeed always supposed to be read and written under a mutex, then > I would strongly prefer to have it accessed as a normal non-volatile > member, and have an assertion that given lock is held or we are in a > safepoint, as we do in many other places. Something like this: > > assert(HeapMonitorStorage_lock->owned_by_self() || > (SafepointSynchronize::is_at_safepoint() && Thread::current()->is_VM_thread()), > "this should not be accessed concurrently"); > > It would be confusing to people reading the code if there are uses of > OrderAccess that are actually always protected under a mutex. > > Thank you for the exact example to be put in the code! I put it around each access/assignment of the _initialized method and found one case where yes you can touch it and not have the lock. It actually is "ok" because you don't act on the storage until later and only when you really want to modify the storage (see the object_alloc_do_sample method which calls the add_trace method). But, because of this, I'm going to put the OrderAccess here, I'll do some performance numbers later and if there are issues, I might add a "unsafe" read and a "safe" one to make it explicit to the reader. But I don't think it will come to that. > As a kind of meta comment, I wonder if it would make sense to add sampling >>> for non-TLAB allocations. Seems like if someone is rapidly allocating a >>> whole bunch of 1 MB objects that never fit in a TLAB, I might still be >>> interested in seeing that in my traces, and not get surprised that the >>> allocation rate is very high yet not showing up in any profiles. >>> >>> That is handled by the handle_sample where you wanted me to put a >> UseTlab because you hit that case if the allocation is too big. >> > > I see. It was not obvious to me that non-TLAB sampling is done in the TLAB > class. That seems like an abstraction crime. > What I wanted in my previous comment was that we do not call into the TLAB > when we are not using TLABs. If there is sampling logic in the TLAB that is > used for something else than TLABs, then it seems like that logic simply > does not belong inside of the TLAB. It should be moved out of the TLAB, and > instead have the TLAB call this common abstraction that makes sense. > > So in the incremental version: http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.07_08/, this is still a "crime". The reason is that the system has to have the bytes_until_sample on a per-thread level and it made "sense" to have it with the TLAB implementation. Also, I was not sure how people felt about adding something to the thread instance instead. Do you think it fits better at the Thread level? I can see how difficult it is to make it happen there and add some logic there. Let me know what you think. > Hope I have answered your questions and that my feedback makes sense to > you. > > You have and thank you for them, I think we are getting to a cleaner implementation and things are getting better and more readable :) Thanks for your help! Jc > Thanks, > /Erik > > > I double checked by changing the test >> http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.05a/raw_ >> files/new/test/hotspot/jtreg/serviceability/jvmti/HeapMonito >> r/MyPackage/HeapMonitorStatObjectCorrectnessTest.java >> >> to use a smaller Tlab (2048) and made the object bigger and it goes >> through that and passes. >> >> Thanks again for your review and I look forward to your pointers for >> the questions I now have raised! >> Jc >> >> >> >> >> >> >> >> >> >> Thanks, >>> /Erik >>> >>> >>> On 2018-01-26 06:45, JC Beyler wrote: >>> >>>> Thanks Robbin for the reviews :) >>>> >>>> The new full webrev is here: >>>> http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.03/ >>>> The incremental webrev is here: >>>> http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.02_03/ >>>> >>>> I inlined my answers: >>>> >>>> On Thu, Jan 25, 2018 at 1:15 AM, Robbin Ehn >>>> wrote: >>>> >>>>> Hi JC, great to see another revision! >>>>> >>>>> #### >>>>> heapMonitoring.cpp >>>>> >>>>> StackTraceData should not contain the oop for 'safety' reasons. >>>>> When StackTraceData is moved from _allocated_traces: >>>>> L452 store_garbage_trace(trace); >>>>> it contains a dead oop. >>>>> _allocated_traces could instead be a tupel of oop and StackTraceData >>>>> thus >>>>> dead oops are not kept. >>>>> >>>> Done I used inheritance to make the copier work regardless but the >>>> idea is the same. >>>> >>>> You should use the new Access API for loading the oop, something like >>>>> this: >>>>> RootAccess::load(...) >>>>> I don't think you need to use Access API for clearing the oop, but it >>>>> would >>>>> look nicer. And you shouldn't probably be using: >>>>> Universe::heap()->is_in_reserved(value) >>>>> >>>> I am unfamiliar with this but I think I did do it like you wanted me >>>> to (all tests pass so that's a start). I'm not sure how to clear the >>>> oop exactly, is there somewhere that does that, which I can use to do >>>> the same? >>>> >>>> I removed the is_in_reserved, this came from our internal version, I >>>> don't know why it was there but my tests work without so I removed it >>>> :) >>>> >>>> >>>> The lock: >>>>> L424 MutexLocker mu(HeapMonitorStorage_lock); >>>>> Is not needed as far as I can see. >>>>> weak_oops_do is called in a safepoint, no TLAB allocation can happen >>>>> and >>>>> JVMTI thread can't access these data-structures. Is there something >>>>> more >>>>> to >>>>> this lock that I'm missing? >>>>> >>>> Since a thread can call the JVMTI getLiveTraces (or any of the other >>>> ones), it can get to the point of trying to copying the >>>> _allocated_traces. I imagine it is possible that this is happening >>>> during a GC or that it can be started and a GC happens afterwards. >>>> Therefore, it seems to me that you want this protected, no? >>>> >>>> >>>> #### >>>>> You have 6 files without any changes in them (any more): >>>>> g1CollectedHeap.cpp >>>>> psMarkSweep.cpp >>>>> psParallelCompact.cpp >>>>> genCollectedHeap.cpp >>>>> referenceProcessor.cpp >>>>> thread.hpp >>>>> >>>>> Done. >>>> >>>> #### >>>>> I have not looked closely, but is it possible to hide heap sampling in >>>>> AllocTracer ? (with some minor changes to the AllocTracer API) >>>>> >>>>> I am imagining that you are saying to move the code that does the >>>> sampling code (change the tlab end, do the call to HeapMonitoring, >>>> etc.) into the AllocTracer code itself? I think that is right and I'll >>>> look if that is possible and prepare a webrev to show what would be >>>> needed to make that happen. >>>> >>>> #### >>>>> Minor nit, when declaring pointer there is a little mix of having the >>>>> pointer adjacent by type name and data name. (Most hotspot code is by >>>>> type >>>>> name) >>>>> E.g. >>>>> heapMonitoring.cpp:711 jvmtiStackTrace *trace = .... >>>>> heapMonitoring.cpp:733 Method* m = vfst.method(); >>>>> (not just this file) >>>>> >>>>> Done! >>>> >>>> #### >>>>> HeapMonitorThreadOnOffTest.java:77 >>>>> I would make g_tmp volatile, otherwise the assignment in loop may >>>>> theoretical be skipped. >>>>> >>>>> Also done! >>>> >>>> Thanks again! >>>> Jc >>>> >>> >>> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From egor.ushakov at jetbrains.com Tue Feb 13 09:14:14 2018 From: egor.ushakov at jetbrains.com (Egor Ushakov) Date: Tue, 13 Feb 2018 12:14:14 +0300 Subject: Issues debugging java 9 from jdk 8 In-Reply-To: References: <69179b71-90c0-d921-dd8c-3b5800ce8f6d@jetbrains.com> Message-ID: <77f0c027-4df2-baba-8d90-0755df27aec0@jetbrains.com> Thanks Chris! Unfortunately, I'm unable to join the list, also I do not see this forwarded email in the jdk8u-dev archives: http://mail.openjdk.java.net/pipermail/jdk8u-dev/2018-February/thread.html Probably we're not on the list or something. Can someone on the jdk8u-dev mailing list forward it there? Thanks, Egor On 08-Feb-18 01:44, Chris Plummer wrote: > Hi Egor, > > [adding jdk8u-dev, which is where 8u backports are discussed] > > I think major.minor changed from 1.8 to 9.0, although I haven't found > the code to confirm that yet. I'm assuming this because of the > following code: > > ??? public boolean canGetModuleInfo() { > ??????? validateVM(); > ??????? return versionInfo().jdwpMajor >= 9; > ??? } > > Given that, your changes look correct. I'm not an 8u reviewer. You'll > need to get the official ok from someone on the 8u list. > > thanks, > > Chris > > On 2/5/18 7:44 AM, Egor Ushakov wrote: >> >> Hi all, >> >> in IDEA we faced an issue that when debugging java 9 process memory >> view does not work: >> https://youtrack.jetbrains.com/issue/JRE-641 >> >> It seems that there's a bug in how >> VirtualMachineImpl.canGetInstanceInfo checks vm version (it does not >> pass jdk 9 where minor is 0): >> if(versionInfo().jdwpMajor <1|| >> versionInfo().jdwpMinor <6){ >> returnfalse; >> } >> I've found this fixed in jdk 9 inside the fix: >> http://hg.openjdk.java.net/jdk9/hs/rev/f900d5afd9c8 >> 8142968: Module System implementation Summary: Initial integration of >> JEP 200, JEP 260, JEP 261, and JEP 282 alanb 17-Mar-16 22:04 >> >> We've applied the part of the fix into our jdk 8 fork: >> https://github.com/JetBrains/jdk8u_jdk/commit/6424e2786e8adc4e012e0b7bd0cfc78ba1ab34dd >> >> It seems reasonable to backport at least this part into openjdk 8? >> What do you think? >> Maybe some other parts deserve backporting as well. >> I've attached the patch just in case. >> -- >> Egor Ushakov >> Software Developer >> JetBrains >> http://www.jetbrains.com >> The Drive to Develop > > -- Egor Ushakov Software Developer JetBrains http://www.jetbrains.com The Drive to Develop -------------- next part -------------- An HTML attachment was scrubbed... URL: From harsha.wardhana.b at oracle.com Tue Feb 13 09:30:20 2018 From: harsha.wardhana.b at oracle.com (Harsha Wardhana B) Date: Tue, 13 Feb 2018 15:00:20 +0530 Subject: RFR: JDK-8187498: Add a -Xmanagement flag as syntactic sugar for -Dcom.sun.management.jmxremote.* properties In-Reply-To: <87cb309b-8764-193a-0447-8ecb741d308d@oracle.com> References: <0fca13c3-1b8f-947c-9041-bdea87b04ba8@oracle.com> <5A687F75.3050404@oracle.com> <05c2e5af-b9d0-0386-4565-ade1179a1d49@oracle.com> <7a18b9d5-3a59-8132-2c4f-ba5de35bfa1d@oracle.com> <0e019151-ef48-b4f4-d253-b9da62aa07c3@oracle.com> <248276b1-1589-561f-0718-7bcb1fd578c7@oracle.com> <40cf194e-4d74-3a34-5eb2-1acc5a5abafb@oracle.com> <87cb309b-8764-193a-0447-8ecb741d308d@oracle.com> Message-ID: <66ad342e-f41f-c01b-e515-8100d108e6ef@oracle.com> Hi, Please find below the revised webrev. http://cr.openjdk.java.net/~hb/8187498/webrev.02/ On Friday 09 February 2018 05:07 AM, mandy chung wrote: > On 2/7/18 1:19 AM, Harsha Wardhana B wrote: > > > > --start-management-agent will not be recognized in the current format and > > hence will not default to --start-management-agent=local=true. > > `--start-local-management-server` is one option as Alan suggests. > Another option is to make `--start-management-agent` to accept > an optional argument. VM can accept `--start-management-agent` > or `--start-management-agent=port=1234,ssl=on`. It may require > more launcher change to support it. Yes. It requires lot more changes to launcher. Hence optional argument to --start-management-agent was not added in order to keep the launcher impact minimum. > > Below is the webrev with above changes and corresponding tests. > > > > http://cr.openjdk.java.net/~hb/8187498/webrev.01/ > > arguments.cpp > > 3216 if (FLAG_SET_CMDLINE(bool, ManagementServer, true) != Flag::SUCCESS) { > 3217 return JNI_EINVAL; > 3218 } > 3219 // management agent in module jdk.management.agent > 3220 if (!create_numbered_property("jdk.module.addmods", "jdk.management.agent", addmods_count++)) { > 3221 return JNI_ENOMEM; > 3222 } > > - you can consider refactor this to replace this block and line 2988-3994. > > 3224 jio_fprintf(defaultStream::output_stream(), > 3225 "-Xmanagement is not supported in this VM.\n"); > > -Xmanagement not yet renamed > java.c > To set the precedence for future VM long form options, I suggest to > rename IsManagementOption to IsVMLongFormOption and change the error > message not specific to management agent. Incorporated above changes. > Agent.java > If --start-management-agent is set, -Dcom.sun.management.* takes > precedence. Do you really want to do that? The new VM option > intends to simplify the command-line to type in. I think it's > cleaner and reasonable if --start-management-agent is specified, > -Dcom.sun.management.* are ignored (maybe worth emit a warning if set) We can probably document that -D options will be overridden instead of emitting a warning. > The implementation uses VMManagement::getVmArguments and scan > the VM options for -start-management-agent. The VM is the one > invoking jdk.internal.agent.Agent::startAgent. A simpler option > is to change Agent::startAgent to take an argument parameter > that will be passed with the argument of --start-management-agent > or null if not set. > > Similarly for startRemoteManagementAgent and startLocalAgent_name. Implementing this change requires lot of changes to argument parsing in native code and hence it is simpler to handle this at java layer. > parseXmgmtArgs should be renamed as no longer -Xmgmt. Better not > to tightly coupled with the option name. Ok. > 291 if(args.startsWith("--start-management-agent")) { > 292 return parseXmgmtArgs(args); > 293 } > > What is this change for? This is required when management agent is started via attach mechanism. > Mandy -Harsha -------------- next part -------------- An HTML attachment was scrubbed... URL: From leo.korinth at oracle.com Tue Feb 13 11:25:32 2018 From: leo.korinth at oracle.com (Leo Korinth) Date: Tue, 13 Feb 2018 12:25:32 +0100 Subject: RFR: 8196337 Add commit methods that take all event properties as argument In-Reply-To: <6c57498a-d0cc-e7bd-c54d-bfd4b2acf323@oracle.com> References: <6c57498a-d0cc-e7bd-c54d-bfd4b2acf323@oracle.com> Message-ID: <495b2840-6031-593d-e6a8-6398e7c3883c@oracle.com> Hi, I would be happy for reviews! Thanks, Leo On 31/01/18 11:00, Leo Korinth wrote: > Hi, > > I am adding commit methods that take all event properties as argument. > > For instant events (without start and stop times) a static commit method > is created (taking all properties). > > For non-instant events, a non static commit method is created (taking > all properties). Also a static commit method (with additional > startTicks/endTicks) is created. > > Also an extra constructor is created (taking all properties). An auto > commit destructor was considered (that would auto commit if the > constructor with all properties was used) but has not yet been implemented. > > Enhancement: > https://bugs.openjdk.java.net/browse/JDK-8196337 > > Webrev: > http://cr.openjdk.java.net/~lkorinth/8196337/00/ (open) > > Testing: > - hs-tier1, hs-tier2 > > Thanks, > Leo From robin.westberg at oracle.com Tue Feb 13 14:59:06 2018 From: robin.westberg at oracle.com (Robin Westberg) Date: Tue, 13 Feb 2018 15:59:06 +0100 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: <0362f486-b446-4023-9914-0469d6274b7a@oracle.com> References: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> <5622d8af-d0d6-e77a-f7c1-f2c474f89763@oracle.com> <3EDE8C81-4FBD-4D40-A798-5F71CF7DD2B8@oracle.com> <0362f486-b446-4023-9914-0469d6274b7a@oracle.com> Message-ID: <7B06DA71-AB33-480B-B176-A3163A78495B@oracle.com> Hi David, > On 12 Feb 2018, at 08:07, David Holmes wrote: > > Hi Robin, > > On 10/02/2018 1:41 AM, Robin Westberg wrote: >> Hi David, >>> On 9 Feb 2018, at 05:22, David Holmes >> wrote: >>> >>> Hi Robin, >>> >>>> Right, almost all the runtime changes are made in order to try to figure out what the process exit code from the launcher will eventually be. For example, the launcher returns 1 if the main thread exited with an unhandled exception, but 0 otherwise. But I actually agree that this information is probably only of marginal use (it could always be captured from wherever Java is launched if someone really wants it), so it is perhaps not worth the trouble. >>> >>> Yes and that particular example of launcher behaviour is an archaic throwback to C programming - where end of "main" means end of the program - IMHO. I don't think it is of interest - and certainly not worth jumping through so many hoops to make the info available. >>> >>>> Discussed it a bit with Erik Gahlin, and perhaps the best option here is to simply remove the status code field from the event, that would simplify things and make the code you mention above go away. >>> >>> That sounds good to me. :) >>> >>>>> It is unfortunate that you need to add beforeHalt to deal with the event mechanism itself being turned off (?) by a shutdown hook. That would seem to potentially lose a lot of event information given hooks can run in arbitrary order and execute arbitrary Java code. And essentially you end up recording the initial reason shutdown started, though potentially it could end up terminating for a different reason. >>>> In this case I think it actually conveys useful information if you are trying to diagnose an unexpected shutdown. It should be useful to find the initial request of an orderly shutdown, even if something else happens during the shutdown sequence like a finalizer calling exit (in which case you could possibly end up with two shutdown events, but both may contain interesting information). >>> >>> Yes that is useful. But I find the need to code things the way they are, unfortunate. But given current constraints, so be it. >> Okay, I?ve removed the code related to the status field, certainly makes the change a bit less intrusive. >> Updated webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.01/ >> Incremental: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00-01/ > > This looks much cleaner/neater to me - thanks. Thanks for reviewing! > Tool Nit: the git-webrev tool you use has an annoying quirk compared to our own webrev tool - it first reports the number of files changed even when it is reporting on an individual file! I'm used to seeing a first number that represents number of lines changes - and the "1" in this case throws me as I like to quickly look at the size of the change for each file when deciding what order to look at them. :( Good point, I?ll make sure the git-webrev tool is updated to more closely match the output of the standard webrev tool. Best regards, Robin > > Thanks, > David > >> Best regards, >> Robin >>> >>> Thanks, >>> David >>> >>>> Best regards, >>>> Robin >>>>> Let's see what others think ... >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>> On 8/02/2018 1:18 AM, Robin Westberg wrote: >>>>>> Hi all, >>>>>> Please review the following change that adds an event-based tracing event that is generated when the VM shuts down. The intent is to capture shutdowns that occur after the VM has been properly initialized (as initialization problems would most likely mean that the tracing framework hasn?t been properly started either). >>>>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8041626 >>>>>> Webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00/ >>>>>> Testing: hs-tier1,hs-tier2,jdk-tier1,jdk-tier2 >>>>>> Best regards, >>>>>> Robin -------------- next part -------------- An HTML attachment was scrubbed... URL: From robin.westberg at oracle.com Tue Feb 13 14:59:12 2018 From: robin.westberg at oracle.com (Robin Westberg) Date: Tue, 13 Feb 2018 15:59:12 +0100 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: References: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> <5622d8af-d0d6-e77a-f7c1-f2c474f89763@oracle.com> <3EDE8C81-4FBD-4D40-A798-5F71CF7DD2B8@oracle.com> <0362f486-b446-4023-9914-0469d6274b7a@oracle.com> Message-ID: <4730283A-A1B1-46C4-9E04-C3993B62E248@oracle.com> Hi Alan, > On 12 Feb 2018, at 09:02, Alan Bateman wrote: > > > > On 12/02/2018 07:07, David Holmes wrote: >> >>> Okay, I?ve removed the code related to the status field, certainly makes the change a bit less intrusive. >>> >>> Updated webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.01/ >>> Incremental: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00-01/ >> >> This looks much cleaner/neater to me - thanks. >> > The updates to Runtime/Shutdown seems okay. Thanks for reviewing! Best regards, Robin > > -Alan From Roger.Riggs at Oracle.com Tue Feb 13 15:17:54 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 13 Feb 2018 10:17:54 -0500 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: <4730283A-A1B1-46C4-9E04-C3993B62E248@oracle.com> References: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> <5622d8af-d0d6-e77a-f7c1-f2c474f89763@oracle.com> <3EDE8C81-4FBD-4D40-A798-5F71CF7DD2B8@oracle.com> <0362f486-b446-4023-9914-0469d6274b7a@oracle.com> <4730283A-A1B1-46C4-9E04-C3993B62E248@oracle.com> Message-ID: <29aea368-9908-72d9-9dae-107a15a63043@Oracle.com> Hi Robin, It looks like the status argument to BeforeHalt is discarded in JVM_BeforeHalt and is not inserted into the event. That suggests it should be removed all the way back to Shutdown.beforeHalt. Roger On 2/13/2018 9:59 AM, Robin Westberg wrote: > Hi Alan, > >> On 12 Feb 2018, at 09:02, Alan Bateman wrote: >> >> >> >> On 12/02/2018 07:07, David Holmes wrote: >>>> Okay, I?ve removed the code related to the status field, certainly makes the change a bit less intrusive. >>>> >>>> Updated webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.01/ >>>> Incremental: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00-01/ >>> This looks much cleaner/neater to me - thanks. >>> >> The updates to Runtime/Shutdown seems okay. > Thanks for reviewing! > > Best regards, > Robin > >> -Alan From mandy.chung at oracle.com Tue Feb 13 19:45:39 2018 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 13 Feb 2018 11:45:39 -0800 Subject: RFR: JDK-8187498: Add a -Xmanagement flag as syntactic sugar for -Dcom.sun.management.jmxremote.* properties In-Reply-To: <66ad342e-f41f-c01b-e515-8100d108e6ef@oracle.com> References: <0fca13c3-1b8f-947c-9041-bdea87b04ba8@oracle.com> <5A687F75.3050404@oracle.com> <05c2e5af-b9d0-0386-4565-ade1179a1d49@oracle.com> <7a18b9d5-3a59-8132-2c4f-ba5de35bfa1d@oracle.com> <0e019151-ef48-b4f4-d253-b9da62aa07c3@oracle.com> <248276b1-1589-561f-0718-7bcb1fd578c7@oracle.com> <40cf194e-4d74-3a34-5eb2-1acc5a5abafb@oracle.com> <87cb309b-8764-193a-0447-8ecb741d308d@oracle.com> <66ad342e-f41f-c01b-e515-8100d108e6ef@oracle.com> Message-ID: <0a13fe7c-b7b0-e5d9-1361-bfea19073574@oracle.com> On 2/13/18 1:30 AM, Harsha Wardhana B wrote: > Hi, > > Please find below the revised webrev. > > http://cr.openjdk.java.net/~hb/8187498/webrev.02/ > > > On Friday 09 February 2018 05:07 AM, mandy chung wrote: >> On 2/7/18 1:19 AM, Harsha Wardhana B wrote: >> > >> > --start-management-agent will not be recognized in the current format and >> > hence will not default to --start-management-agent=local=true. >> >> `--start-local-management-server` is one option as Alan suggests. >> Another option is to make `--start-management-agent` to accept >> an optional argument. VM can accept `--start-management-agent` >> or `--start-management-agent=port=1234,ssl=on`. It may require >> more launcher change to support it. > Yes. It requires lot more changes to launcher. Hence optional argument > to --start-management-agent was not added in order to keep the > launcher impact minimum. This is just one option for you to consider.?? So the current proposal of the new option to start a remote management server, right? >> Agent.java >> If --start-management-agent is set, -Dcom.sun.management.* takes >> precedence. Do you really want to do that? The new VM option >> intends to simplify the command-line to type in. I think it's >> cleaner and reasonable if --start-management-agent is specified, >> -Dcom.sun.management.* are ignored (maybe worth emit a warning if set) > We can probably document that -D options will be overridden instead of > emitting a warning. What is the behavior when -Dcom.sun.management.jmxremote.port=1234 --start-management-agent port=2345 -Dcom.sun.management.jmxremote.port=3456? What is the value of the system property com.sun.management.jmxremote.port at runtime?? What port number does the management server start with? >> The implementation uses VMManagement::getVmArguments and scan >> the VM options for -start-management-agent. The VM is the one >> invoking jdk.internal.agent.Agent::startAgent. A simpler option >> is to change Agent::startAgent to take an argument parameter >> that will be passed with the argument of --start-management-agent >> or null if not set. >> >> Similarly for startRemoteManagementAgent and startLocalAgent_name. > Implementing this change requires lot of changes to argument parsing > in native code and hence it is simpler to handle this at java layer. Can you describe how --start-management-agent option will be used for jcmd and any other tool that attaches to a running VM to start the agent?? Example command-line will be useful. Mandy -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Tue Feb 13 21:24:22 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 13 Feb 2018 13:24:22 -0800 Subject: RFR: JDK-8196930 - [Testbug] serviceability/sa/ClhsdbFindPC.java fails to find expected output In-Reply-To: References: Message-ID: <5db673de-f9b7-17fb-14cc-5a023613e693@oracle.com> An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Tue Feb 13 22:02:27 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 13 Feb 2018 14:02:27 -0800 Subject: RFR 8196324: Update FilterMatch & FilterNoMatch to use TestScaffold instead of JDIScaffold In-Reply-To: <5A81EA18.9080808@oracle.com> References: <5A81EA18.9080808@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From jcbeyler at google.com Wed Feb 14 00:11:35 2018 From: jcbeyler at google.com (JC Beyler) Date: Tue, 13 Feb 2018 16:11:35 -0800 Subject: JDK-8171119: Low-Overhead Heap Profiling In-Reply-To: References: <5A819F10.8040201@oracle.com> Message-ID: Hi all, Just to show a bit how to solve the one issue Erik was referring to, consider the following webrev: http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.08a/ and incremental is here: http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.08_08a/ This puts the sampling bytes left in the Thread class () and then the code goes to Thread to sample or not. The advantage of this is that it is probably simpler to understand and follow what is going on, there is less of internal tlab magic going on and the outside of tlab allocations go through the thread instance the same way the TLAB allocations do. I think it's cleaner but what do you think? Thanks! Jc On Mon, Feb 12, 2018 at 9:18 PM, JC Beyler wrote: > Hi Erik, > > Thanks for your answers, I've now inlined my own answers/comments. > > I've done a new webrev here: > http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.08/ > > The incremental is here: > http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.07_08/ > > Note to all: > - I've been integrating changes from Erin/Serguei/David comments so this > webrev incremental is a bit an answer to all comments in one. I apologize > for that :) > > > On Mon, Feb 12, 2018 at 6:05 AM, Erik ?sterlund > wrote: > >> Hi JC, >> >> Sorry for the delayed reply. >> >> Inlined answers: >> >> >> On 2018-02-06 00:04, JC Beyler wrote: >> >>> Hi Erik, >>> >>> (Renaming this to be folded into the newly renamed thread :)) >>> >>> First off, thanks a lot for reviewing the webrev! I appreciate it! >>> >>> I updated the webrev to: >>> http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.05a/ >>> >>> And the incremental one is here: >>> http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.04_05a/ >>> >>> It contains: >>> - The change for since from 9 to 11 for the jvmti.xml >>> - The use of the OrderAccess for initialized >>> - Clearing the oop >>> >>> I also have inlined my answers to your comments. The biggest question >>> will come from the multiple *_end variables. A bit of the logic there >>> is due to handling the slow path refill vs fast path refill and >>> checking that the rug was not pulled underneath the slowpath. I >>> believe that a previous comment was that TlabFastRefill was going to >>> be deprecated. >>> >>> If this is true, we could revert this code a bit and just do a : if >>> TlabFastRefill is enabled, disable this. And then deprecate that when >>> TlabFastRefill is deprecated. >>> >>> This might simplify this webrev and I can work on a follow-up that >>> either: removes TlabFastRefill if Robbin does not have the time to do >>> it or add the support to the assembly side to handle this correctly. >>> What do you think? >>> >> >> I support removing TlabFastRefill, but I think it is good to not depend >> on that happening first. >> >> > > I'm slowly pushing on the FastTLABRefill (https://bugs.openjdk.java.net > /browse/JDK-8194084), I agree on keeping both separate for now though so > that we can think of both differently > > > >> Now, below, inlined are my answers: >>> >>> On Fri, Feb 2, 2018 at 8:44 AM, Erik ?sterlund >>> wrote: >>> >>>> Hi JC, >>>> >>>> Hope I am reviewing the right version of your work. Here goes... >>>> >>>> src/hotspot/share/gc/shared/collectedHeap.inline.hpp: >>>> >>>> 159 AllocTracer::send_allocation_outside_tlab(klass, result, >>>> size * >>>> HeapWordSize, THREAD); >>>> 160 >>>> 161 THREAD->tlab().handle_sample(THREAD, result, size); >>>> 162 return result; >>>> 163 } >>>> >>>> Should not call tlab()->X without checking if (UseTLAB) IMO. >>>> >>>> Done! >>> >> >> More about this later. >> >> >> >>> src/hotspot/share/gc/shared/threadLocalAllocBuffer.cpp: >>>> >>>> So first of all, there seems to quite a few ends. There is an "end", a >>>> "hard >>>> end", a "slow path end", and an "actual end". Moreover, it seems like >>>> the >>>> "hard end" is actually further away than the "actual end". So the "hard >>>> end" >>>> seems like more of a "really definitely actual end" or something. I >>>> don't >>>> know about you, but I think it looks kind of messy. In particular, I >>>> don't >>>> feel like the name "actual end" reflects what it represents, especially >>>> when >>>> there is another end that is behind the "actual end". >>>> >>>> 413 HeapWord* ThreadLocalAllocBuffer::hard_end() { >>>> 414 // Did a fast TLAB refill occur? >>>> 415 if (_slow_path_end != _end) { >>>> 416 // Fix up the actual end to be now the end of this TLAB. >>>> 417 _slow_path_end = _end; >>>> 418 _actual_end = _end; >>>> 419 } >>>> 420 >>>> 421 return _actual_end + alignment_reserve(); >>>> 422 } >>>> >>>> I really do not like making getters unexpectedly have these kind of side >>>> effects. It is not expected that when you ask for the "hard end", you >>>> implicitly update the "slow path end" and "actual end" to new values. >>>> >>>> As I said, a lot of this is due to the FastTlabRefill. If I make this >>> not supporting FastTlabRefill, this goes away. The reason the system >>> needs to update itself at the get is that you only know at that get if >>> things have shifted underneath the tlab slow path. I am not sure of >>> really better names (naming is hard!), perhaps we could do these >>> names: >>> >>> - current_tlab_end // Either the allocated tlab end or a sampling >>> point >>> - last_allocation_address // The end of the tlab allocation >>> - last_slowpath_allocated_end // In case a fast refill occurred the >>> end might have changed, this is to remember slow vs fast past refills >>> >>> the hard_end method can be renamed to something like: >>> tlab_end_pointer() // The end of the lab including a bit of >>> alignment reserved bytes >>> >> >> Those names sound better to me. Could you please provide a mapping from >> the old names to the new names so I understand which one is which please? >> >> This is my current guess of what you are proposing: >> >> end -> current_tlab_end >> actual_end -> last_allocation_address >> slow_path_end -> last_slowpath_allocated_end >> hard_end -> tlab_end_pointer >> >> > Yes that is correct, that was what I was proposing. > > >> I would prefer this naming: >> >> end -> slow_path_end // the end for taking a slow path; either due to >> sampling or refilling >> actual_end -> allocation_end // the end for allocations >> slow_path_end -> last_slow_path_end // last address for slow_path_end (as >> opposed to allocation_end) >> hard_end -> reserved_end // the end of the reserved space of the TLAB >> >> About setting things in the getter... that still seems like a very >> unpleasant thing to me. It would be better to inspect the call hierarchy >> and explicitly update the ends where they need updating, and assert in the >> getter that they are in sync, rather than implicitly setting various ends >> as a surprising side effect in a getter. It looks like the call hierarchy >> is very small. With my new naming convention, reserved_end() would >> presumably return _allocation_end + alignment_reserve(), and have an assert >> checking that _allocation_end == _last_slow_path_allocation_end, >> complaining that this invariant must hold, and that a caller to this >> function, such as make_parsable(), must first explicitly synchronize the >> ends as required, to honor that invariant. >> >> > > I've renamed the variables to how you preferred it except for the _end > one. I did: > current_end > last_allocation_address > tlab_end_ptr > > The reason is that the architecture dependent code use the thread.hpp API > and it already has tlab included into the name so it becomes > tlab_current_end (which is better that tlab_current_tlab_end in my opinion). > > I also moved the update into a separate method with a TODO that says to > remove it when FastTLABRefill is deprecated > > > >> >> Not sure it's better but before updating the webrev, I wanted to try >>> to get input/consensus :) >>> >>> (Note hard_end was always further off than end). >>> >>> src/hotspot/share/prims/jvmti.xml: >>>> >>>> 10357 >>>> 10358 >>>> 10359 Can sample the heap. >>>> 10360 If this capability is enabled then the heap sampling >>>> methods >>>> can be called. >>>> 10361 >>>> 10362 >>>> >>>> Looks like this capability should not be "since 9" if it gets integrated >>>> now. >>>> >>> Updated now to 11, crossing my fingers :) >>> >>> >>> src/hotspot/share/runtime/heapMonitoring.cpp: >>>> >>>> 448 if (is_alive->do_object_b(value)) { >>>> 449 // Update the oop to point to the new object if it is >>>> still >>>> alive. >>>> 450 f->do_oop(&(trace.obj)); >>>> 451 >>>> 452 // Copy the old trace, if it is still live. >>>> 453 _allocated_traces->at_put(curr_pos++, trace); >>>> 454 >>>> 455 // Store the live trace in a cache, to be served up on >>>> /heapz. >>>> 456 _traces_on_last_full_gc->append(trace); >>>> 457 >>>> 458 count++; >>>> 459 } else { >>>> 460 // If the old trace is no longer live, add it to the list >>>> of >>>> 461 // recently collected garbage. >>>> 462 store_garbage_trace(trace); >>>> 463 } >>>> >>>> In the case where the oop was not live, I would like it to be explicitly >>>> cleared. >>>> >>> Done I think how you wanted it. Let me know because I'm not familiar >>> with the RootAccess API. I'm unclear if I'm doing this right or not so >>> reviews of these parts are highly appreciated. Robbin had talked of >>> perhaps later pushing this all into a OopStorage, should I do this now >>> do you think? Or can that wait a second webrev later down the road? >>> >> >> I think using handles can and should be done later. You can use the >> Access API now. >> I noticed that you are missing an #include "oops/access.inline.hpp" in >> your heapMonitoring.cpp file. >> >> > The missing header is there for me so I don't know, I made sure it is > present in the latest webrev. Sorry about that. > > > >> + Did I clear it the way you wanted me to or were you thinking of >>> something else? >>> >> >> That is precisely how I wanted it to be cleared. Thanks. >> >> + Final question here, seems like if I were to want to not do the >>> f->do_oop directly on the trace.obj, I'd need to do something like: >>> >>> f->do_oop(&value); >>> ... >>> trace->store_oop(value); >>> >>> to update the oop internally. Is that right/is that one of the >>> advantages of going to the Oopstorage sooner than later? >>> >> >> I think you really want to do the do_oop on the root directly. Is there a >> particular reason why you would not want to do that? >> Otherwise, yes - the benefit with using the handle approach is that you >> do not need to call do_oop explicitly in your code. >> >> > There is no reason except that now we have a load_oop and a get_oop_addr, > I was not sure what you would think of that. > > > > >> >>> Also I see a lot of concurrent-looking use of the following field: >>>> 267 volatile bool _initialized; >>>> >>>> Please note that the "volatile" qualifier does not help with reordering >>>> here. Reordering between volatile and non-volatile fields is completely >>>> free >>>> for both compiler and hardware, except for windows with MSVC, where >>>> volatile >>>> semantics is defined to use acquire/release semantics, and the hardware >>>> is >>>> TSO. But for the general case, I would expect this field to be stored >>>> with >>>> OrderAccess::release_store and loaded with OrderAccess::load_acquire. >>>> Otherwise it is not thread safe. >>>> >>> Because everything is behind a mutex, I wasn't really worried about >>> this. I have a test that has multiple threads trying to hit this >>> corner case and it passes. >>> >>> However, to be paranoid, I updated it to using the OrderAccess API >>> now, thanks! Let me know what you think there too! >>> >> >> If it is indeed always supposed to be read and written under a mutex, >> then I would strongly prefer to have it accessed as a normal non-volatile >> member, and have an assertion that given lock is held or we are in a >> safepoint, as we do in many other places. Something like this: >> >> assert(HeapMonitorStorage_lock->owned_by_self() || >> (SafepointSynchronize::is_at_safepoint() && >> Thread::current()->is_VM_thread()), "this should not be accessed >> concurrently"); >> >> It would be confusing to people reading the code if there are uses of >> OrderAccess that are actually always protected under a mutex. >> >> > Thank you for the exact example to be put in the code! I put it around > each access/assignment of the _initialized method and found one case where > yes you can touch it and not have the lock. It actually is "ok" because you > don't act on the storage until later and only when you really want to > modify the storage (see the object_alloc_do_sample method which calls the > add_trace method). > > But, because of this, I'm going to put the OrderAccess here, I'll do some > performance numbers later and if there are issues, I might add a "unsafe" > read and a "safe" one to make it explicit to the reader. But I don't think > it will come to that. > > > >> As a kind of meta comment, I wonder if it would make sense to add sampling >>>> for non-TLAB allocations. Seems like if someone is rapidly allocating a >>>> whole bunch of 1 MB objects that never fit in a TLAB, I might still be >>>> interested in seeing that in my traces, and not get surprised that the >>>> allocation rate is very high yet not showing up in any profiles. >>>> >>>> That is handled by the handle_sample where you wanted me to put a >>> UseTlab because you hit that case if the allocation is too big. >>> >> >> I see. It was not obvious to me that non-TLAB sampling is done in the >> TLAB class. That seems like an abstraction crime. >> What I wanted in my previous comment was that we do not call into the >> TLAB when we are not using TLABs. If there is sampling logic in the TLAB >> that is used for something else than TLABs, then it seems like that logic >> simply does not belong inside of the TLAB. It should be moved out of the >> TLAB, and instead have the TLAB call this common abstraction that makes >> sense. >> >> > So in the incremental version: > http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.07_08/, this is still > a "crime". The reason is that the system has to have the bytes_until_sample > on a per-thread level and it made "sense" to have it with the TLAB > implementation. Also, I was not sure how people felt about adding something > to the thread instance instead. > > Do you think it fits better at the Thread level? I can see how difficult > it is to make it happen there and add some logic there. Let me know what > you think. > > > >> Hope I have answered your questions and that my feedback makes sense to >> you. >> >> > You have and thank you for them, I think we are getting to a cleaner > implementation and things are getting better and more readable :) > > Thanks for your help! > Jc > > > >> Thanks, >> /Erik >> >> >> I double checked by changing the test >>> http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.05a/raw_ >>> files/new/test/hotspot/jtreg/serviceability/jvmti/HeapMonito >>> r/MyPackage/HeapMonitorStatObjectCorrectnessTest.java >>> >>> to use a smaller Tlab (2048) and made the object bigger and it goes >>> through that and passes. >>> >>> Thanks again for your review and I look forward to your pointers for >>> the questions I now have raised! >>> Jc >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> Thanks, >>>> /Erik >>>> >>>> >>>> On 2018-01-26 06:45, JC Beyler wrote: >>>> >>>>> Thanks Robbin for the reviews :) >>>>> >>>>> The new full webrev is here: >>>>> http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.03/ >>>>> The incremental webrev is here: >>>>> http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.02_03/ >>>>> >>>>> I inlined my answers: >>>>> >>>>> On Thu, Jan 25, 2018 at 1:15 AM, Robbin Ehn >>>>> wrote: >>>>> >>>>>> Hi JC, great to see another revision! >>>>>> >>>>>> #### >>>>>> heapMonitoring.cpp >>>>>> >>>>>> StackTraceData should not contain the oop for 'safety' reasons. >>>>>> When StackTraceData is moved from _allocated_traces: >>>>>> L452 store_garbage_trace(trace); >>>>>> it contains a dead oop. >>>>>> _allocated_traces could instead be a tupel of oop and StackTraceData >>>>>> thus >>>>>> dead oops are not kept. >>>>>> >>>>> Done I used inheritance to make the copier work regardless but the >>>>> idea is the same. >>>>> >>>>> You should use the new Access API for loading the oop, something like >>>>>> this: >>>>>> RootAccess::load(...) >>>>>> I don't think you need to use Access API for clearing the oop, but it >>>>>> would >>>>>> look nicer. And you shouldn't probably be using: >>>>>> Universe::heap()->is_in_reserved(value) >>>>>> >>>>> I am unfamiliar with this but I think I did do it like you wanted me >>>>> to (all tests pass so that's a start). I'm not sure how to clear the >>>>> oop exactly, is there somewhere that does that, which I can use to do >>>>> the same? >>>>> >>>>> I removed the is_in_reserved, this came from our internal version, I >>>>> don't know why it was there but my tests work without so I removed it >>>>> :) >>>>> >>>>> >>>>> The lock: >>>>>> L424 MutexLocker mu(HeapMonitorStorage_lock); >>>>>> Is not needed as far as I can see. >>>>>> weak_oops_do is called in a safepoint, no TLAB allocation can happen >>>>>> and >>>>>> JVMTI thread can't access these data-structures. Is there something >>>>>> more >>>>>> to >>>>>> this lock that I'm missing? >>>>>> >>>>> Since a thread can call the JVMTI getLiveTraces (or any of the other >>>>> ones), it can get to the point of trying to copying the >>>>> _allocated_traces. I imagine it is possible that this is happening >>>>> during a GC or that it can be started and a GC happens afterwards. >>>>> Therefore, it seems to me that you want this protected, no? >>>>> >>>>> >>>>> #### >>>>>> You have 6 files without any changes in them (any more): >>>>>> g1CollectedHeap.cpp >>>>>> psMarkSweep.cpp >>>>>> psParallelCompact.cpp >>>>>> genCollectedHeap.cpp >>>>>> referenceProcessor.cpp >>>>>> thread.hpp >>>>>> >>>>>> Done. >>>>> >>>>> #### >>>>>> I have not looked closely, but is it possible to hide heap sampling in >>>>>> AllocTracer ? (with some minor changes to the AllocTracer API) >>>>>> >>>>>> I am imagining that you are saying to move the code that does the >>>>> sampling code (change the tlab end, do the call to HeapMonitoring, >>>>> etc.) into the AllocTracer code itself? I think that is right and I'll >>>>> look if that is possible and prepare a webrev to show what would be >>>>> needed to make that happen. >>>>> >>>>> #### >>>>>> Minor nit, when declaring pointer there is a little mix of having the >>>>>> pointer adjacent by type name and data name. (Most hotspot code is by >>>>>> type >>>>>> name) >>>>>> E.g. >>>>>> heapMonitoring.cpp:711 jvmtiStackTrace *trace = .... >>>>>> heapMonitoring.cpp:733 Method* m = vfst.method(); >>>>>> (not just this file) >>>>>> >>>>>> Done! >>>>> >>>>> #### >>>>>> HeapMonitorThreadOnOffTest.java:77 >>>>>> I would make g_tmp volatile, otherwise the assignment in loop may >>>>>> theoretical be skipped. >>>>>> >>>>>> Also done! >>>>> >>>>> Thanks again! >>>>> Jc >>>>> >>>> >>>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Wed Feb 14 02:07:47 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 14 Feb 2018 12:07:47 +1000 Subject: RFR: JDK-8196930 - [Testbug] serviceability/sa/ClhsdbFindPC.java fails to find expected output In-Reply-To: References: Message-ID: <527fbc12-4ff6-3946-346e-2a46903a3c30@oracle.com> Hi Sharath, On 12/02/2018 7:12 PM, Sharath Ballal wrote: > Hello, > > Requesting reviews for the issue: > > ID: https://bugs.openjdk.java.net/browse/JDK-8196930 > > Webrev: http://cr.openjdk.java.net/~sballal/8196930/webrev.00/ I've no doubt this makes the test pass, but I always have to wonder with these tests: why was it looking for the thing now removed? what changed (or in what way was the expected output invalid) to make the test fail? Thanks, David > Tests run: The SA tests pass with Mach5. > > Thanks, > > Sharath > From sharath.ballal at oracle.com Wed Feb 14 05:11:11 2018 From: sharath.ballal at oracle.com (Sharath Ballal) Date: Tue, 13 Feb 2018 21:11:11 -0800 (PST) Subject: RFR: JDK-8196930 - [Testbug] serviceability/sa/ClhsdbFindPC.java fails to find expected output In-Reply-To: <527fbc12-4ff6-3946-346e-2a46903a3c30@oracle.com> References: <527fbc12-4ff6-3946-346e-2a46903a3c30@oracle.com> Message-ID: David, When I wrote these tests, I looked for the output generated by these commands. The output I saw was same (across windows, mac, linux) and hence grepped for certain strings from it. Now we are seeing some issues where depending on the hardware we are at different execution point and hence some of the strings are changing. So now I am removing such strings from the test. The main intention of these tests are to check the basic sanity that the SA command is working. Thanks, Sharath -----Original Message----- From: David Holmes Sent: Wednesday, February 14, 2018 7:38 AM To: Sharath Ballal; serviceability-dev Subject: Re: RFR: JDK-8196930 - [Testbug] serviceability/sa/ClhsdbFindPC.java fails to find expected output Hi Sharath, On 12/02/2018 7:12 PM, Sharath Ballal wrote: > Hello, > > Requesting reviews for the issue: > > ID: https://bugs.openjdk.java.net/browse/JDK-8196930 > > Webrev: http://cr.openjdk.java.net/~sballal/8196930/webrev.00/ I've no doubt this makes the test pass, but I always have to wonder with these tests: why was it looking for the thing now removed? what changed (or in what way was the expected output invalid) to make the test fail? Thanks, David > Tests run: The SA tests pass with Mach5. > > Thanks, > > Sharath > From david.holmes at oracle.com Wed Feb 14 05:22:50 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 14 Feb 2018 15:22:50 +1000 Subject: RFR: JDK-8196930 - [Testbug] serviceability/sa/ClhsdbFindPC.java fails to find expected output In-Reply-To: References: <527fbc12-4ff6-3946-346e-2a46903a3c30@oracle.com> Message-ID: On 14/02/2018 3:11 PM, Sharath Ballal wrote: > David, > When I wrote these tests, I looked for the output generated by these commands. The output I saw was same (across windows, mac, linux) and hence grepped for certain strings from it. > Now we are seeing some issues where depending on the hardware we are at different execution point and hence some of the strings are changing. So now I am removing such strings from the test. > The main intention of these tests are to check the basic sanity that the SA command is working. Ok. But based on "In interpreter codelet" I would expect this would fail if run in -Xcomp mode ?? Thanks, David > Thanks, > Sharath > > > -----Original Message----- > From: David Holmes > Sent: Wednesday, February 14, 2018 7:38 AM > To: Sharath Ballal; serviceability-dev > Subject: Re: RFR: JDK-8196930 - [Testbug] serviceability/sa/ClhsdbFindPC.java fails to find expected output > > Hi Sharath, > > On 12/02/2018 7:12 PM, Sharath Ballal wrote: >> Hello, >> >> Requesting reviews for the issue: >> >> ID: https://bugs.openjdk.java.net/browse/JDK-8196930 >> >> Webrev: http://cr.openjdk.java.net/~sballal/8196930/webrev.00/ > > I've no doubt this makes the test pass, but I always have to wonder with these tests: why was it looking for the thing now removed? what changed (or in what way was the expected output invalid) to make the test fail? > > Thanks, > David > >> Tests run: The SA tests pass with Mach5. >> >> Thanks, >> >> Sharath >> From sharath.ballal at oracle.com Wed Feb 14 05:29:28 2018 From: sharath.ballal at oracle.com (Sharath Ballal) Date: Tue, 13 Feb 2018 21:29:28 -0800 (PST) Subject: RFR: JDK-8196930 - [Testbug] serviceability/sa/ClhsdbFindPC.java fails to find expected output In-Reply-To: References: <527fbc12-4ff6-3946-346e-2a46903a3c30@oracle.com> Message-ID: <15041369-ba2a-4efb-aba8-b8ebcb1363b9@default> Thanks, Sharath -----Original Message----- From: David Holmes Sent: Wednesday, February 14, 2018 10:53 AM To: Sharath Ballal; serviceability-dev Subject: Re: RFR: JDK-8196930 - [Testbug] serviceability/sa/ClhsdbFindPC.java fails to find expected output On 14/02/2018 3:11 PM, Sharath Ballal wrote: > David, > When I wrote these tests, I looked for the output generated by these commands. The output I saw was same (across windows, mac, linux) and hence grepped for certain strings from it. > Now we are seeing some issues where depending on the hardware we are at different execution point and hence some of the strings are changing. So now I am removing such strings from the test. > The main intention of these tests are to check the basic sanity that the SA command is working. Ok. But based on "In interpreter codelet" I would expect this would fail if run in -Xcomp mode ?? This test works in both Xcomp and Xint mode. "In interpreter codelet" is being checked for Xint, for Xcomp we are checking "In code in NMethod for jdk/test/lib/apps/LingeredApp.main" Thanks, David > Thanks, > Sharath > > > -----Original Message----- > From: David Holmes > Sent: Wednesday, February 14, 2018 7:38 AM > To: Sharath Ballal; serviceability-dev > Subject: Re: RFR: JDK-8196930 - [Testbug] > serviceability/sa/ClhsdbFindPC.java fails to find expected output > > Hi Sharath, > > On 12/02/2018 7:12 PM, Sharath Ballal wrote: >> Hello, >> >> Requesting reviews for the issue: >> >> ID: https://bugs.openjdk.java.net/browse/JDK-8196930 >> >> Webrev: http://cr.openjdk.java.net/~sballal/8196930/webrev.00/ > > I've no doubt this makes the test pass, but I always have to wonder with these tests: why was it looking for the thing now removed? what changed (or in what way was the expected output invalid) to make the test fail? > > Thanks, > David > >> Tests run: The SA tests pass with Mach5. >> >> Thanks, >> >> Sharath >> From david.holmes at oracle.com Wed Feb 14 05:39:50 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 14 Feb 2018 15:39:50 +1000 Subject: RFR: JDK-8196930 - [Testbug] serviceability/sa/ClhsdbFindPC.java fails to find expected output In-Reply-To: <15041369-ba2a-4efb-aba8-b8ebcb1363b9@default> References: <527fbc12-4ff6-3946-346e-2a46903a3c30@oracle.com> <15041369-ba2a-4efb-aba8-b8ebcb1363b9@default> Message-ID: <544646a2-2ba2-a67c-ffeb-dd42c35c3b13@oracle.com> On 14/02/2018 3:29 PM, Sharath Ballal wrote: > -----Original Message----- > From: David Holmes > Sent: Wednesday, February 14, 2018 10:53 AM > To: Sharath Ballal; serviceability-dev > Subject: Re: RFR: JDK-8196930 - [Testbug] serviceability/sa/ClhsdbFindPC.java fails to find expected output > > On 14/02/2018 3:11 PM, Sharath Ballal wrote: >> David, >> When I wrote these tests, I looked for the output generated by these commands. The output I saw was same (across windows, mac, linux) and hence grepped for certain strings from it. >> Now we are seeing some issues where depending on the hardware we are at different execution point and hence some of the strings are changing. So now I am removing such strings from the test. >> The main intention of these tests are to check the basic sanity that the SA command is working. > >> Ok. But based on "In interpreter codelet" I would expect this would fail if run in -Xcomp mode ?? > This test works in both Xcomp and Xint mode. "In interpreter codelet" is being checked for Xint, for Xcomp we are checking "In code in NMethod for jdk/test/lib/apps/LingeredApp.main" Yep sorry - missed that bit. David > > Thanks, > David > >> Thanks, >> Sharath >> >> >> -----Original Message----- >> From: David Holmes >> Sent: Wednesday, February 14, 2018 7:38 AM >> To: Sharath Ballal; serviceability-dev >> Subject: Re: RFR: JDK-8196930 - [Testbug] >> serviceability/sa/ClhsdbFindPC.java fails to find expected output >> >> Hi Sharath, >> >> On 12/02/2018 7:12 PM, Sharath Ballal wrote: >>> Hello, >>> >>> Requesting reviews for the issue: >>> >>> ID: https://bugs.openjdk.java.net/browse/JDK-8196930 >>> >>> Webrev: http://cr.openjdk.java.net/~sballal/8196930/webrev.00/ >> >> I've no doubt this makes the test pass, but I always have to wonder with these tests: why was it looking for the thing now removed? what changed (or in what way was the expected output invalid) to make the test fail? >> >> Thanks, >> David >> >>> Tests run: The SA tests pass with Mach5. >>> >>> Thanks, >>> >>> Sharath >>> From parvathi.somashekar at oracle.com Wed Feb 14 07:01:18 2018 From: parvathi.somashekar at oracle.com (Paru Somashekar) Date: Tue, 13 Feb 2018 23:01:18 -0800 Subject: RFR 8196324: Update FilterMatch & FilterNoMatch to use TestScaffold instead of JDIScaffold In-Reply-To: References: <5A81EA18.9080808@oracle.com> Message-ID: <5A83DEBE.1010709@oracle.com> Hi Serguei, Thanks very much for the review and I have made all the changes incorporating your feedback. New Webrev at :http://cr.openjdk.java.net/~psomashe/8196324/webrev.01/ thanks, Paru. On 2/13/18, 2:02 PM, serguei.spitsyn at oracle.com wrote: > Hi Paru, > > It looks good in general but I'd like to ask for some cleanup. > > I tell just about the first test (FilterMatch) but the other one needs > the same. > > 132 // This gets called if all filters match. > 133 public void stepCompleted(StepEvent event) { > 134 listenCalled = true; > 135 System.out.println("listen: line#=" + event.location().lineNumber() > 136 + " event=" + event); > 137 // disable the step and then run to completion > 138 StepRequest str= (StepRequest)event.request(); > 139 str.disable(); > 140 eventSet.resume(); > 141 } > I'd suggest to replace "listen:" above with "Agent: StepEvent:" to make > it more consistent with similar tracing in the breakpointReached() below. > > 143 public void breakpointReached(BreakpointEvent event) { > 144 ThreadReference thread = ((BreakpointEvent) event).thread(); > 145 String locStr = "" + ((BreakpointEvent) event).location(); > 146 System.out.println("Agent: BreakpointEvent #" + > 147 " at " + locStr + " in thread: " + thread); > 148 // The bkpt was hit; disable it. > 149 request.disable(); > 150 } > The casts to BreakpointEvent at lines 144, 145 are not needed > as the "event" is already of this type. > Unneeded sign '#' at the line 146. > Also, I'm suggesting to disable the "request" the same way > as it is done in the stepCompleted(): > BreakpointRequest bpr= (BreakpointRequest)event.request(); > bpr.disable(); > > It will help to make the "request" local in the runTests(). > Unneeded empty lines: 52, 54: > 51 EventSet eventSet = null; > 52 > 53 static boolean listenCalled; > 54 > 55 BreakpointRequest request; > > The listenCalled needs to be renamed to stepCompleted. > There is no big need for it to be static as it is similar to the eventSet. > It is better to initialize it with false. > Then this line can be removed: > 114 listenCalled = false; > > I'd suggest to rename the variables "request" and "request1" > to "bpRequest" and "stepRequest" and make the 'bpRequest' to be local > in the runTests() as it is the only place where it has to be used. > > > Thanks, > Serguei > > > On 2/12/18 11:25, Paru Somashekar wrote: >> Hi, >> >> Please review fix for JDK-8196324 >> >> The includes the following changes: >> >> * Use startToMain method of TestScaffold that automatically calls >> connect() and waitForVMStart() >> * Since TestScaffold extends TargetAdapter, it now overrides event >> callbacks directly in the test class rather than implement an >> anonymous inner class. >> * Breakpoint handling is done in breakpointReached callback instead >> of waitForRequestedEvent and subsequently removing it. >> >> Bug : https://bugs.openjdk.java.net/browse/JDK-8196324 >> Review : http://cr.openjdk.java.net/~psomashe/8196324/webrev/ >> >> >> >> thanks, >> Paru. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robbin.ehn at oracle.com Wed Feb 14 08:13:02 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 14 Feb 2018 09:13:02 +0100 Subject: JDK-8171119: Low-Overhead Heap Profiling In-Reply-To: References: <5A819F10.8040201@oracle.com> Message-ID: <88a73c1c-0c11-fe23-b07e-eec5aa799450@oracle.com> Hi JC, Adding Markus Gr?nlund. On 2018-02-14 01:11, JC Beyler wrote: > Hi all, > > Just to show a bit how to solve the one issue Erik was referring to, consider the following webrev: > http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.08a/ > > and incremental is here: > http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.08_08a/ We have long standing issue of adding stuff directly to the thread. Please put tracing stuff into a separate data structure and file. Hoping that this new structure can contain this and thread specific parts of up coming tracing framework. (Markus?) Maybe follow threadSMR, which is contained in: open/src/hotspot/share/runtime/threadSMR.cpp open/src/hotspot/share/runtime/threadSMR.inline.hpp open/src/hotspot/share/runtime/threadSMR.hpp And create threadTracing.XXX or similar ? Thanks, Robbin > > This puts the sampling bytes left in the Thread class () and then the code goes to Thread to sample or not. The advantage of this is that it is probably simpler to understand and follow what is going on, there is less of internal tlab magic going on and the outside of tlab allocations go through the thread instance the same way the TLAB allocations do. > > I think it's cleaner but what do you think? > > Thanks! > Jc > > On Mon, Feb 12, 2018 at 9:18 PM, JC Beyler > wrote: > > Hi Erik, > > Thanks for your answers, I've now inlined my own answers/comments. > > I've done a new webrev here: > http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.08/ > > The incremental is here: > http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.07_08/ > > Note to all: > ? - I've been integrating changes from Erin/Serguei/David comments so this webrev incremental is a bit an answer to all comments in one. I apologize for that :) > > > On Mon, Feb 12, 2018 at 6:05 AM, Erik ?sterlund > wrote: > > Hi JC, > > Sorry for the delayed reply. > > Inlined answers: > > > On 2018-02-06 00:04, JC Beyler wrote: > > Hi Erik, > > (Renaming this to be folded into the newly renamed thread :)) > > First off, thanks a lot for reviewing the webrev! I appreciate it! > > I updated the webrev to: > http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.05a/ > > And the incremental one is here: > http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.04_05a/ > > It contains: > - The change for since from 9 to 11 for the jvmti.xml > - The use of the OrderAccess for initialized > - Clearing the oop > > I also have inlined my answers to your comments. The biggest question > will come from the multiple *_end variables. A bit of the logic there > is due to handling the slow path refill vs fast path refill and > checking that the rug was not pulled underneath the slowpath. I > believe that a previous comment was that TlabFastRefill was going to > be deprecated. > > If this is true, we could revert this code a bit and just do a : if > TlabFastRefill is enabled, disable this. And then deprecate that when > TlabFastRefill is deprecated. > > This might simplify this webrev and I can work on a follow-up that > either: removes TlabFastRefill if Robbin does not have the time to do > it or add the support to the assembly side to handle this correctly. > What do you think? > > > I support removing TlabFastRefill, but I think it is good to not depend on that happening first. > > > > I'm slowly pushing on the FastTLABRefill (https://bugs.openjdk.java.net/browse/JDK-8194084 ), I agree on keeping both separate for now though so that we can think of both differently > > Now, below, inlined are my answers: > > On Fri, Feb 2, 2018 at 8:44 AM, Erik ?sterlund > > wrote: > > Hi JC, > > Hope I am reviewing the right version of your work. Here goes... > > src/hotspot/share/gc/shared/collectedHeap.inline.hpp: > > ? 159? ? ?AllocTracer::send_allocation_outside_tlab(klass, result, size * > HeapWordSize, THREAD); > ? 160 > ? 161? ? ?THREAD->tlab().handle_sample(THREAD, result, size); > ? 162? ? ?return result; > ? 163? ?} > > Should not call tlab()->X without checking if (UseTLAB) IMO. > > Done! > > > More about this later. > > > > src/hotspot/share/gc/shared/threadLocalAllocBuffer.cpp: > > So first of all, there seems to quite a few ends. There is an "end", a "hard > end", a "slow path end", and an "actual end". Moreover, it seems like the > "hard end" is actually further away than the "actual end". So the "hard end" > seems like more of a "really definitely actual end" or something. I don't > know about you, but I think it looks kind of messy. In particular, I don't > feel like the name "actual end" reflects what it represents, especially when > there is another end that is behind the "actual end". > > ? 413 HeapWord* ThreadLocalAllocBuffer::hard_end() { > ? 414? ?// Did a fast TLAB refill occur? > ? 415? ?if (_slow_path_end != _end) { > ? 416? ? ?// Fix up the actual end to be now the end of this TLAB. > ? 417? ? ?_slow_path_end = _end; > ? 418? ? ?_actual_end = _end; > ? 419? ?} > ? 420 > ? 421? ?return _actual_end + alignment_reserve(); > ? 422 } > > I really do not like making getters unexpectedly have these kind of side > effects. It is not expected that when you ask for the "hard end", you > implicitly update the "slow path end" and "actual end" to new values. > > As I said, a lot of this is due to the FastTlabRefill. If I make this > not supporting FastTlabRefill, this goes away. The reason the system > needs to update itself at the get is that you only know at that get if > things have shifted underneath the tlab slow path. I am not sure of > really better names (naming is hard!), perhaps we could do these > names: > > - current_tlab_end? ? ? ?// Either the allocated tlab end or a sampling point > - last_allocation_address? // The end of the tlab allocation > - last_slowpath_allocated_end? // In case a fast refill occurred the > end might have changed, this is to remember slow vs fast past refills > > the hard_end method can be renamed to something like: > tlab_end_pointer()? ? ? ? // The end of the lab including a bit of > alignment reserved bytes > > > Those names sound better to me. Could you please provide a mapping from the old names to the new names so I understand which one is which please? > > This is my current guess of what you are proposing: > > end -> current_tlab_end > actual_end -> last_allocation_address > slow_path_end -> last_slowpath_allocated_end > hard_end -> tlab_end_pointer > > > Yes that is correct, that was what I was proposing. > > I would prefer this naming: > > end -> slow_path_end // the end for taking a slow path; either due to sampling or refilling > actual_end -> allocation_end // the end for allocations > slow_path_end -> last_slow_path_end // last address for slow_path_end (as opposed to allocation_end) > hard_end -> reserved_end // the end of the reserved space of the TLAB > > About setting things in the getter... that still seems like a very unpleasant thing to me. It would be better to inspect the call hierarchy and explicitly update the ends where they need updating, and assert in the getter that they are in sync, rather than implicitly setting various ends as a surprising side effect in a getter. It looks like the call hierarchy is very small. With my new naming convention, reserved_end() would presumably return _allocation_end + alignment_reserve(), and have an assert checking that _allocation_end == _last_slow_path_allocation_end, complaining that this > invariant must hold, and that a caller to this function, such as make_parsable(), must first explicitly synchronize the ends as required, to honor that invariant. > > > > I've renamed the variables to how you preferred it except for the _end one. I did: > current_end > last_allocation_address > tlab_end_ptr > > The reason is that the architecture dependent code use the thread.hpp API and it already has tlab included into the name so it becomes tlab_current_end (which is better that tlab_current_tlab_end in my opinion). > > I also moved the update into a separate method with a TODO that says to remove it when FastTLABRefill is deprecated > > > Not sure it's better but before updating the webrev, I wanted to try > to get input/consensus :) > > (Note hard_end was always further off than end). > > src/hotspot/share/prims/jvmti.xml: > > 10357? ? ? ? > 10358? ? ? ? ? > 10359? ? ? ? ? ?Can sample the heap. > 10360? ? ? ? ? ?If this capability is enabled then the heap sampling methods > can be called. > 10361? ? ? ? ? > 10362? ? ? ? > > Looks like this capability should not be "since 9" if it gets integrated > now. > > Updated now to 11, crossing my fingers :) > > > src/hotspot/share/runtime/heapMonitoring.cpp: > > ? 448? ? ? ?if (is_alive->do_object_b(value)) { > ? 449? ? ? ? ?// Update the oop to point to the new object if it is still > alive. > ? 450? ? ? ? ?f->do_oop(&(trace.obj)); > ? 451 > ? 452? ? ? ? ?// Copy the old trace, if it is still live. > ? 453? ? ? ? ?_allocated_traces->at_put(curr_pos++, trace); > ? 454 > ? 455? ? ? ? ?// Store the live trace in a cache, to be served up on /heapz. > ? 456? ? ? ? ?_traces_on_last_full_gc->append(trace); > ? 457 > ? 458? ? ? ? ?count++; > ? 459? ? ? ?} else { > ? 460? ? ? ? ?// If the old trace is no longer live, add it to the list of > ? 461? ? ? ? ?// recently collected garbage. > ? 462? ? ? ? ?store_garbage_trace(trace); > ? 463? ? ? ?} > > In the case where the oop was not live, I would like it to be explicitly > cleared. > > Done I think how you wanted it. Let me know because I'm not familiar > with the RootAccess API. I'm unclear if I'm doing this right or not so > reviews of these parts are highly appreciated. Robbin had talked of > perhaps later pushing this all into a OopStorage, should I do this now > do you think? Or can that wait a second webrev later down the road? > > > I think using handles can and should be done later. You can use the Access API now. > I noticed that you are missing an #include "oops/access.inline.hpp" in your heapMonitoring.cpp file. > > > The missing header is there for me so I don't know, I made sure it is present in the latest webrev. Sorry about that. > > > + Did I clear it the way you wanted me to or were you thinking of > something else? > > > That is precisely how I wanted it to be cleared. Thanks. > > + Final question here, seems like if I were to want to not do the > f->do_oop directly on the trace.obj, I'd need to do something like: > > ? ? f->do_oop(&value); > ? ? ... > ? ? trace->store_oop(value); > > to update the oop internally. Is that right/is that one of the > advantages of going to the Oopstorage sooner than later? > > > I think you really want to do the do_oop on the root directly. Is there a particular reason why you would not want to do that? > Otherwise, yes - the benefit with using the handle approach is that you do not need to call do_oop explicitly in your code. > > > There is no reason except that now we have a load_oop and a get_oop_addr, I was not sure what you would think of that. > > > > Also I see a lot of concurrent-looking use of the following field: > ? 267? ?volatile bool _initialized; > > Please note that the "volatile" qualifier does not help with reordering > here. Reordering between volatile and non-volatile fields is completely free > for both compiler and hardware, except for windows with MSVC, where volatile > semantics is defined to use acquire/release semantics, and the hardware is > TSO. But for the general case, I would expect this field to be stored with > OrderAccess::release_store and loaded with OrderAccess::load_acquire. > Otherwise it is not thread safe. > > Because everything is behind a mutex, I wasn't really worried about > this. I have a test that has multiple threads trying to hit this > corner case and it passes. > > However, to be paranoid, I updated it to using the OrderAccess API > now, thanks! Let me know what you think there too! > > > If it is indeed always supposed to be read and written under a mutex, then I would strongly prefer to have it accessed as a normal non-volatile member, and have an assertion that given lock is held or we are in a safepoint, as we do in many other places. Something like this: > > assert(HeapMonitorStorage_lock->owned_by_self() || (SafepointSynchronize::is_at_safepoint() && Thread::current()->is_VM_thread()), "this should not be accessed concurrently"); > > It would be confusing to people reading the code if there are uses of OrderAccess that are actually always protected under a mutex. > > > Thank you for the exact example to be put in the code! I put it around each access/assignment of the _initialized method and found one case where yes you can touch it and not have the lock. It actually is "ok" because you don't act on the storage until later and only when you really want to modify the storage (see the object_alloc_do_sample method which calls the add_trace method). > > But, because of this, I'm going to put the OrderAccess here, I'll do some performance numbers later and if there are issues, I might add a "unsafe" read and a "safe" one to make it explicit to the reader. But I don't think it will come to that. > > As a kind of meta comment, I wonder if it would make sense to add sampling > for non-TLAB allocations. Seems like if someone is rapidly allocating a > whole bunch of 1 MB objects that never fit in a TLAB, I might still be > interested in seeing that in my traces, and not get surprised that the > allocation rate is very high yet not showing up in any profiles. > > That is handled by the handle_sample where you wanted me to put a > UseTlab because you hit that case if the allocation is too big. > > > I see. It was not obvious to me that non-TLAB sampling is done in the TLAB class. That seems like an abstraction crime. > What I wanted in my previous comment was that we do not call into the TLAB when we are not using TLABs. If there is sampling logic in the TLAB that is used for something else than TLABs, then it seems like that logic simply does not belong inside of the TLAB. It should be moved out of the TLAB, and instead have the TLAB call this common abstraction that makes sense. > > > So in the incremental version: > http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.07_08/ , this is still a "crime". The reason is that the system has to have the bytes_until_sample on a per-thread level and it made "sense" to have it with the TLAB implementation. Also, I was not sure how people felt about adding something to the thread instance instead. > > Do you think it fits better at the Thread level? I can see how difficult it is to make it happen there and add some logic there. Let me know what you think. > > Hope I have answered your questions and that my feedback makes sense to you. > > > You have and thank you for them, I think we are getting to a cleaner implementation and things are getting better and more readable :) > > Thanks for your help! > Jc > > Thanks, > /Erik > > > I double checked by changing the test > http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.05a/raw_files/new/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatObjectCorrectnessTest.java > > to use a smaller Tlab (2048) and made the object bigger and it goes > through that and passes. > > Thanks again for your review and I look forward to your pointers for > the questions I now have raised! > Jc > > > > > > > > > > Thanks, > /Erik > > > On 2018-01-26 06:45, JC Beyler wrote: > > Thanks Robbin for the reviews :) > > The new full webrev is here: > http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.03/ > The incremental webrev is here: > http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.02_03/ > > I inlined my answers: > > On Thu, Jan 25, 2018 at 1:15 AM, Robbin Ehn > wrote: > > Hi JC, great to see another revision! > > #### > heapMonitoring.cpp > > StackTraceData should not contain the oop for 'safety' reasons. > When StackTraceData is moved from _allocated_traces: > L452 store_garbage_trace(trace); > it contains a dead oop. > _allocated_traces could instead be a tupel of oop and StackTraceData thus > dead oops are not kept. > > Done I used inheritance to make the copier work regardless but the > idea is the same. > > You should use the new Access API for loading the oop, something like > this: > RootAccess::load(...) > I don't think you need to use Access API for clearing the oop, but it > would > look nicer. And you shouldn't probably be using: > Universe::heap()->is_in_reserved(value) > > I am unfamiliar with this but I think I did do it like you wanted me > to (all tests pass so that's a start). I'm not sure how to clear the > oop exactly, is there somewhere that does that, which I can use to do > the same? > > I removed the is_in_reserved, this came from our internal version, I > don't know why it was there but my tests work without so I removed it > :) > > > The lock: > L424? ?MutexLocker mu(HeapMonitorStorage_lock); > Is not needed as far as I can see. > weak_oops_do is called in a safepoint, no TLAB allocation can happen and > JVMTI thread can't access these data-structures. Is there something more > to > this lock that I'm missing? > > Since a thread can call the JVMTI getLiveTraces (or any of the other > ones), it can get to the point of trying to copying the > _allocated_traces. I imagine it is possible that this is happening > during a GC or that it can be started and a GC happens afterwards. > Therefore, it seems to me that you want this protected, no? > > > #### > You have 6 files without any changes in them (any more): > g1CollectedHeap.cpp > psMarkSweep.cpp > psParallelCompact.cpp > genCollectedHeap.cpp > referenceProcessor.cpp > thread.hpp > > Done. > > #### > I have not looked closely, but is it possible to hide heap sampling in > AllocTracer ? (with some minor changes to the AllocTracer API) > > I am imagining that you are saying to move the code that does the > sampling code (change the tlab end, do the call to HeapMonitoring, > etc.) into the AllocTracer code itself? I think that is right and I'll > look if that is possible and prepare a webrev to show what would be > needed to make that happen. > > #### > Minor nit, when declaring pointer there is a little mix of having the > pointer adjacent by type name and data name. (Most hotspot code is by > type > name) > E.g. > heapMonitoring.cpp:711? ? ?jvmtiStackTrace *trace = .... > heapMonitoring.cpp:733? ? ? ? ?Method* m = vfst.method(); > (not just this file) > > Done! > > #### > HeapMonitorThreadOnOffTest.java:77 > I would make g_tmp volatile, otherwise the assignment in loop may > theoretical be skipped. > > Also done! > > Thanks again! > Jc > > > > > From jini.george at oracle.com Wed Feb 14 09:07:32 2018 From: jini.george at oracle.com (Jini George) Date: Wed, 14 Feb 2018 14:37:32 +0530 Subject: RFR: JDK-8175312: SA: clhsdb: Provide an improved heap summary for 'universe' for G1GC Message-ID: Hello, (Including the SVC and GC groups) Requesting reviews for enabling the clhsdb 'universe' command to print out the various G1 regions (along with the type). JIRA ID:https://bugs.openjdk.java.net/browse/JDK-8175312 Webrev: http://cr.openjdk.java.net/~jgeorge/8175312/webrev.00/index.html Modifications have been made to display the regions like: ... Region: 0x00000005c5400000,0x00000005c5600000,0x00000005c5600000:Old Region: 0x00000005c5600000,0x00000005c5800000,0x00000005c5800000:Old Region: 0x00000005c5800000,0x00000005c5a00000,0x00000005c5a00000:Old Region: 0x00000005c5a00000,0x00000005c5c00000,0x00000005c5c00000:Old Region: 0x00000005c5c00000,0x00000005c5c00000,0x00000005c5e00000:Free Region: 0x00000005c5e00000,0x00000005c5e00000,0x00000005c6000000:Free Region: 0x00000005c6000000,0x00000005c6200000,0x00000005c6200000:Old ... The jtreg test at this point does not include any testing for the display of archived or pinned regions. The testing for this will be added once JDK-8174994 is resolved. The SA tests pass with jprt and Mach5. Thanks, Jini. From erik.osterlund at oracle.com Wed Feb 14 10:51:24 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 14 Feb 2018 11:51:24 +0100 Subject: JDK-8171119: Low-Overhead Heap Profiling In-Reply-To: References: <5A819F10.8040201@oracle.com> Message-ID: <5A8414AC.3020209@oracle.com> Hi JC, Comments are inlined below. On 2018-02-13 06:18, JC Beyler wrote: > Hi Erik, > > Thanks for your answers, I've now inlined my own answers/comments. > > I've done a new webrev here: > http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.08/ > > > The incremental is here: > http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.07_08/ > > > Note to all: > - I've been integrating changes from Erin/Serguei/David comments so > this webrev incremental is a bit an answer to all comments in one. I > apologize for that :) > > > On Mon, Feb 12, 2018 at 6:05 AM, Erik ?sterlund > > wrote: > > Hi JC, > > Sorry for the delayed reply. > > Inlined answers: > > > On 2018-02-06 00:04, JC Beyler wrote: > > Hi Erik, > > (Renaming this to be folded into the newly renamed thread :)) > > First off, thanks a lot for reviewing the webrev! I appreciate it! > > I updated the webrev to: > http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.05a/ > > > And the incremental one is here: > http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.04_05a/ > > > It contains: > - The change for since from 9 to 11 for the jvmti.xml > - The use of the OrderAccess for initialized > - Clearing the oop > > I also have inlined my answers to your comments. The biggest > question > will come from the multiple *_end variables. A bit of the > logic there > is due to handling the slow path refill vs fast path refill and > checking that the rug was not pulled underneath the slowpath. I > believe that a previous comment was that TlabFastRefill was > going to > be deprecated. > > If this is true, we could revert this code a bit and just do a > : if > TlabFastRefill is enabled, disable this. And then deprecate > that when > TlabFastRefill is deprecated. > > This might simplify this webrev and I can work on a follow-up that > either: removes TlabFastRefill if Robbin does not have the > time to do > it or add the support to the assembly side to handle this > correctly. > What do you think? > > > I support removing TlabFastRefill, but I think it is good to not > depend on that happening first. > > > > I'm slowly pushing on the FastTLABRefill > (https://bugs.openjdk.java.net/browse/JDK-8194084 > ), I agree on > keeping both separate for now though so that we can think of both > differently > > Now, below, inlined are my answers: > > On Fri, Feb 2, 2018 at 8:44 AM, Erik ?sterlund > > > wrote: > > Hi JC, > > Hope I am reviewing the right version of your work. Here > goes... > > src/hotspot/share/gc/shared/collectedHeap.inline.hpp: > > 159 AllocTracer::send_allocation_outside_tlab(klass, > result, size * > HeapWordSize, THREAD); > 160 > 161 THREAD->tlab().handle_sample(THREAD, result, size); > 162 return result; > 163 } > > Should not call tlab()->X without checking if (UseTLAB) IMO. > > Done! > > > More about this later. > > > > src/hotspot/share/gc/shared/threadLocalAllocBuffer.cpp: > > So first of all, there seems to quite a few ends. There is > an "end", a "hard > end", a "slow path end", and an "actual end". Moreover, it > seems like the > "hard end" is actually further away than the "actual end". > So the "hard end" > seems like more of a "really definitely actual end" or > something. I don't > know about you, but I think it looks kind of messy. In > particular, I don't > feel like the name "actual end" reflects what it > represents, especially when > there is another end that is behind the "actual end". > > 413 HeapWord* ThreadLocalAllocBuffer::hard_end() { > 414 // Did a fast TLAB refill occur? > 415 if (_slow_path_end != _end) { > 416 // Fix up the actual end to be now the end of > this TLAB. > 417 _slow_path_end = _end; > 418 _actual_end = _end; > 419 } > 420 > 421 return _actual_end + alignment_reserve(); > 422 } > > I really do not like making getters unexpectedly have > these kind of side > effects. It is not expected that when you ask for the > "hard end", you > implicitly update the "slow path end" and "actual end" to > new values. > > As I said, a lot of this is due to the FastTlabRefill. If I > make this > not supporting FastTlabRefill, this goes away. The reason the > system > needs to update itself at the get is that you only know at > that get if > things have shifted underneath the tlab slow path. I am not > sure of > really better names (naming is hard!), perhaps we could do these > names: > > - current_tlab_end // Either the allocated tlab end or a > sampling point > - last_allocation_address // The end of the tlab allocation > - last_slowpath_allocated_end // In case a fast refill > occurred the > end might have changed, this is to remember slow vs fast past > refills > > the hard_end method can be renamed to something like: > tlab_end_pointer() // The end of the lab including a bit of > alignment reserved bytes > > > Those names sound better to me. Could you please provide a mapping > from the old names to the new names so I understand which one is > which please? > > This is my current guess of what you are proposing: > > end -> current_tlab_end > actual_end -> last_allocation_address > slow_path_end -> last_slowpath_allocated_end > hard_end -> tlab_end_pointer > > > Yes that is correct, that was what I was proposing. > > I would prefer this naming: > > end -> slow_path_end // the end for taking a slow path; either due > to sampling or refilling > actual_end -> allocation_end // the end for allocations > slow_path_end -> last_slow_path_end // last address for > slow_path_end (as opposed to allocation_end) > hard_end -> reserved_end // the end of the reserved space of the TLAB > > About setting things in the getter... that still seems like a very > unpleasant thing to me. It would be better to inspect the call > hierarchy and explicitly update the ends where they need updating, > and assert in the getter that they are in sync, rather than > implicitly setting various ends as a surprising side effect in a > getter. It looks like the call hierarchy is very small. With my > new naming convention, reserved_end() would presumably return > _allocation_end + alignment_reserve(), and have an assert checking > that _allocation_end == _last_slow_path_allocation_end, > complaining that this invariant must hold, and that a caller to > this function, such as make_parsable(), must first explicitly > synchronize the ends as required, to honor that invariant. > > > > I've renamed the variables to how you preferred it except for the _end > one. I did: > current_end > last_allocation_address > tlab_end_ptr > > The reason is that the architecture dependent code use the thread.hpp > API and it already has tlab included into the name so it becomes > tlab_current_end (which is better that tlab_current_tlab_end in my > opinion). > > I also moved the update into a separate method with a TODO that says > to remove it when FastTLABRefill is deprecated This looks a lot better now. Thanks. Note that the following comment now needs updating accordingly in threadLocalAllocBuffer.hpp: 41 // Heap sampling is performed via the end/actual_end fields. 42 // actual_end contains the real end of the tlab allocation, 43 // whereas end can be set to an arbitrary spot in the tlab to 44 // trip the return and sample the allocation. 45 // slow_path_end is used to track if a fast tlab refill occured 46 // between slowpath calls. There might be other comments too, I have not looked in detail. > > > Not sure it's better but before updating the webrev, I wanted > to try > to get input/consensus :) > > (Note hard_end was always further off than end). > > src/hotspot/share/prims/jvmti.xml: > > 10357 > 10358 > 10359 Can sample the heap. > 10360 If this capability is enabled then the > heap sampling methods > can be called. > 10361 > 10362 > > Looks like this capability should not be "since 9" if it > gets integrated > now. > > Updated now to 11, crossing my fingers :) > > > src/hotspot/share/runtime/heapMonitoring.cpp: > > 448 if (is_alive->do_object_b(value)) { > 449 // Update the oop to point to the new object > if it is still > alive. > 450 f->do_oop(&(trace.obj)); > 451 > 452 // Copy the old trace, if it is still live. > 453 _allocated_traces->at_put(curr_pos++, trace); > 454 > 455 // Store the live trace in a cache, to be > served up on /heapz. > 456 _traces_on_last_full_gc->append(trace); > 457 > 458 count++; > 459 } else { > 460 // If the old trace is no longer live, add > it to the list of > 461 // recently collected garbage. > 462 store_garbage_trace(trace); > 463 } > > In the case where the oop was not live, I would like it to > be explicitly > cleared. > > Done I think how you wanted it. Let me know because I'm not > familiar > with the RootAccess API. I'm unclear if I'm doing this right > or not so > reviews of these parts are highly appreciated. Robbin had > talked of > perhaps later pushing this all into a OopStorage, should I do > this now > do you think? Or can that wait a second webrev later down the > road? > > > I think using handles can and should be done later. You can use > the Access API now. > I noticed that you are missing an #include > "oops/access.inline.hpp" in your heapMonitoring.cpp file. > > > The missing header is there for me so I don't know, I made sure it is > present in the latest webrev. Sorry about that. > > > + Did I clear it the way you wanted me to or were you thinking of > something else? > > > That is precisely how I wanted it to be cleared. Thanks. > > + Final question here, seems like if I were to want to not do the > f->do_oop directly on the trace.obj, I'd need to do something > like: > > f->do_oop(&value); > ... > trace->store_oop(value); > > to update the oop internally. Is that right/is that one of the > advantages of going to the Oopstorage sooner than later? > > > I think you really want to do the do_oop on the root directly. Is > there a particular reason why you would not want to do that? > Otherwise, yes - the benefit with using the handle approach is > that you do not need to call do_oop explicitly in your code. > > > There is no reason except that now we have a load_oop and a > get_oop_addr, I was not sure what you would think of that. > That's fine. > > Also I see a lot of concurrent-looking use of the > following field: > 267 volatile bool _initialized; > > Please note that the "volatile" qualifier does not help > with reordering > here. Reordering between volatile and non-volatile fields > is completely free > for both compiler and hardware, except for windows with > MSVC, where volatile > semantics is defined to use acquire/release semantics, and > the hardware is > TSO. But for the general case, I would expect this field > to be stored with > OrderAccess::release_store and loaded with > OrderAccess::load_acquire. > Otherwise it is not thread safe. > > Because everything is behind a mutex, I wasn't really worried > about > this. I have a test that has multiple threads trying to hit this > corner case and it passes. > > However, to be paranoid, I updated it to using the OrderAccess API > now, thanks! Let me know what you think there too! > > > If it is indeed always supposed to be read and written under a > mutex, then I would strongly prefer to have it accessed as a > normal non-volatile member, and have an assertion that given lock > is held or we are in a safepoint, as we do in many other places. > Something like this: > > assert(HeapMonitorStorage_lock->owned_by_self() || > (SafepointSynchronize::is_at_safepoint() && > Thread::current()->is_VM_thread()), "this should not be accessed > concurrently"); > > It would be confusing to people reading the code if there are uses > of OrderAccess that are actually always protected under a mutex. > > > Thank you for the exact example to be put in the code! I put it around > each access/assignment of the _initialized method and found one case > where yes you can touch it and not have the lock. It actually is "ok" > because you don't act on the storage until later and only when you > really want to modify the storage (see the object_alloc_do_sample > method which calls the add_trace method). > > But, because of this, I'm going to put the OrderAccess here, I'll do > some performance numbers later and if there are issues, I might add a > "unsafe" read and a "safe" one to make it explicit to the reader. But > I don't think it will come to that. Okay. This double return in heapMonitoring.cpp looks wrong: 283 bool initialized() { 284 return OrderAccess::load_acquire(&_initialized) != 0; 285 return _initialized; 286 } Since you said object_alloc_do_sample() is the only place where you do not hold the mutex while reading initialized(), I had a closer look at that. It looks like in its current shape, the lack of a mutex may lead to a memory leak. In particular, it first checks if (initialized()). Let's assume this is now true. It then allocates a bunch of stuff, and checks if the number of frames were over 0. If they were, it calls StackTraceStorage::storage()->add_trace() seemingly hoping that after grabbing the lock in there, initialized() will still return true. But it could now return false and skip doing anything, in which case the allocated stuff will never be freed. So the analysis seems to be that _initialized is only used outside of the mutex in once instance, where it is used to perform double-checked locking, that actually causes a memory leak. I am not proposing how to fix that, just raising the issue. If you still want to perform this double-checked locking somehow, then the use of acquire/release still seems odd. Because the memory ordering restrictions of it never comes into play in this particular case. If it ever did, then the use of destroy_stuff(); release_store(_initialized, 0) would be broken anyway as that would imply that whatever concurrent reader there ever was would after reading _initialized with load_acquire() could *never* read the data that is concurrently destroyed anyway. I would be biased to think that RawAccess::load/store looks like a more appropriate solution, given that the memory leak issue is resolved. I do not know how painful it would be to not perform this double-checked locking. > > As a kind of meta comment, I wonder if it would make sense > to add sampling > for non-TLAB allocations. Seems like if someone is rapidly > allocating a > whole bunch of 1 MB objects that never fit in a TLAB, I > might still be > interested in seeing that in my traces, and not get > surprised that the > allocation rate is very high yet not showing up in any > profiles. > > That is handled by the handle_sample where you wanted me to put a > UseTlab because you hit that case if the allocation is too big. > > > I see. It was not obvious to me that non-TLAB sampling is done in > the TLAB class. That seems like an abstraction crime. > What I wanted in my previous comment was that we do not call into > the TLAB when we are not using TLABs. If there is sampling logic > in the TLAB that is used for something else than TLABs, then it > seems like that logic simply does not belong inside of the TLAB. > It should be moved out of the TLAB, and instead have the TLAB call > this common abstraction that makes sense. > > > So in the incremental version: > http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.07_08/ > , this > is still a "crime". The reason is that the system has to have the > bytes_until_sample on a per-thread level and it made "sense" to have > it with the TLAB implementation. Also, I was not sure how people felt > about adding something to the thread instance instead. > > Do you think it fits better at the Thread level? I can see how > difficult it is to make it happen there and add some logic there. Let > me know what you think. We have an unfortunate situation where everyone that has some fields that are thread local tend to dump them right into Thread, making the size and complexity of Thread grow as it becomes tightly coupled with various unrelated subsystems. It would be desirable to have a separate class for this instead that encapsulates the sampling logic. That class could possibly reside in Thread though as a value object of Thread. > > Hope I have answered your questions and that my feedback makes > sense to you. > > > You have and thank you for them, I think we are getting to a cleaner > implementation and things are getting better and more readable :) Yes it is getting better. Thanks, /Erik > Thanks for your help! > Jc > > Thanks, > /Erik > > > I double checked by changing the test > http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.05a/raw_files/new/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorStatObjectCorrectnessTest.java > > > to use a smaller Tlab (2048) and made the object bigger and it > goes > through that and passes. > > Thanks again for your review and I look forward to your > pointers for > the questions I now have raised! > Jc > > > > > > > > > > Thanks, > /Erik > > > On 2018-01-26 06:45, JC Beyler wrote: > > Thanks Robbin for the reviews :) > > The new full webrev is here: > http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.03/ > The incremental webrev is here: > http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.02_03/ > > > I inlined my answers: > > On Thu, Jan 25, 2018 at 1:15 AM, Robbin Ehn > > > wrote: > > Hi JC, great to see another revision! > > #### > heapMonitoring.cpp > > StackTraceData should not contain the oop for > 'safety' reasons. > When StackTraceData is moved from _allocated_traces: > L452 store_garbage_trace(trace); > it contains a dead oop. > _allocated_traces could instead be a tupel of oop > and StackTraceData thus > dead oops are not kept. > > Done I used inheritance to make the copier work > regardless but the > idea is the same. > > You should use the new Access API for loading the > oop, something like > this: > RootAccess AS_NO_KEEPALIVE>::load(...) > I don't think you need to use Access API for > clearing the oop, but it > would > look nicer. And you shouldn't probably be using: > Universe::heap()->is_in_reserved(value) > > I am unfamiliar with this but I think I did do it like > you wanted me > to (all tests pass so that's a start). I'm not sure > how to clear the > oop exactly, is there somewhere that does that, which > I can use to do > the same? > > I removed the is_in_reserved, this came from our > internal version, I > don't know why it was there but my tests work without > so I removed it > :) > > > The lock: > L424 MutexLocker mu(HeapMonitorStorage_lock); > Is not needed as far as I can see. > weak_oops_do is called in a safepoint, no TLAB > allocation can happen and > JVMTI thread can't access these data-structures. > Is there something more > to > this lock that I'm missing? > > Since a thread can call the JVMTI getLiveTraces (or > any of the other > ones), it can get to the point of trying to copying the > _allocated_traces. I imagine it is possible that this > is happening > during a GC or that it can be started and a GC happens > afterwards. > Therefore, it seems to me that you want this > protected, no? > > > #### > You have 6 files without any changes in them (any > more): > g1CollectedHeap.cpp > psMarkSweep.cpp > psParallelCompact.cpp > genCollectedHeap.cpp > referenceProcessor.cpp > thread.hpp > > Done. > > #### > I have not looked closely, but is it possible to > hide heap sampling in > AllocTracer ? (with some minor changes to the > AllocTracer API) > > I am imagining that you are saying to move the code > that does the > sampling code (change the tlab end, do the call to > HeapMonitoring, > etc.) into the AllocTracer code itself? I think that > is right and I'll > look if that is possible and prepare a webrev to show > what would be > needed to make that happen. > > #### > Minor nit, when declaring pointer there is a > little mix of having the > pointer adjacent by type name and data name. (Most > hotspot code is by > type > name) > E.g. > heapMonitoring.cpp:711 jvmtiStackTrace *trace > = .... > heapMonitoring.cpp:733 Method* m = > vfst.method(); > (not just this file) > > Done! > > #### > HeapMonitorThreadOnOffTest.java:77 > I would make g_tmp volatile, otherwise the > assignment in loop may > theoretical be skipped. > > Also done! > > Thanks again! > Jc > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robin.westberg at oracle.com Wed Feb 14 15:15:58 2018 From: robin.westberg at oracle.com (Robin Westberg) Date: Wed, 14 Feb 2018 16:15:58 +0100 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: <29aea368-9908-72d9-9dae-107a15a63043@Oracle.com> References: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> <5622d8af-d0d6-e77a-f7c1-f2c474f89763@oracle.com> <3EDE8C81-4FBD-4D40-A798-5F71CF7DD2B8@oracle.com> <0362f486-b446-4023-9914-0469d6274b7a@oracle.com> <4730283A-A1B1-46C4-9E04-C3993B62E248@oracle.com> <29aea368-9908-72d9-9dae-107a15a63043@Oracle.com> Message-ID: <0F81FDA5-443F-4E5D-9AA0-895AB48BEA73@oracle.com> Hi Roger, > On 13 Feb 2018, at 16:17, Roger Riggs wrote: > > Hi Robin, > > It looks like the status argument to BeforeHalt is discarded in JVM_BeforeHalt > and is not inserted into the event. > That suggests it should be removed all the way back to Shutdown.beforeHalt. You are right, my thinking was that the interface wouldn?t need to be changed if we decided to revisit the event in the future and add the status code. But that is perhaps unlikely, so can certainly remove the argument for now. Best regards, Robin > > Roger > > > > On 2/13/2018 9:59 AM, Robin Westberg wrote: >> Hi Alan, >> >>> On 12 Feb 2018, at 09:02, Alan Bateman wrote: >>> >>> >>> >>> On 12/02/2018 07:07, David Holmes wrote: >>>>> Okay, I?ve removed the code related to the status field, certainly makes the change a bit less intrusive. >>>>> >>>>> Updated webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.01/ >>>>> Incremental: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00-01/ >>>> This looks much cleaner/neater to me - thanks. >>>> >>> The updates to Runtime/Shutdown seems okay. >> Thanks for reviewing! >> >> Best regards, >> Robin >> >>> -Alan > From harsha.wardhana.b at oracle.com Wed Feb 14 16:29:10 2018 From: harsha.wardhana.b at oracle.com (Harsha Wardhana B) Date: Wed, 14 Feb 2018 21:59:10 +0530 Subject: RFR: JDK-8187498: Add a -Xmanagement flag as syntactic sugar for -Dcom.sun.management.jmxremote.* properties In-Reply-To: <0a13fe7c-b7b0-e5d9-1361-bfea19073574@oracle.com> References: <0fca13c3-1b8f-947c-9041-bdea87b04ba8@oracle.com> <5A687F75.3050404@oracle.com> <05c2e5af-b9d0-0386-4565-ade1179a1d49@oracle.com> <7a18b9d5-3a59-8132-2c4f-ba5de35bfa1d@oracle.com> <0e019151-ef48-b4f4-d253-b9da62aa07c3@oracle.com> <248276b1-1589-561f-0718-7bcb1fd578c7@oracle.com> <40cf194e-4d74-3a34-5eb2-1acc5a5abafb@oracle.com> <87cb309b-8764-193a-0447-8ecb741d308d@oracle.com> <66ad342e-f41f-c01b-e515-8100d108e6ef@oracle.com> <0a13fe7c-b7b0-e5d9-1361-bfea19073574@oracle.com> Message-ID: <0332f3ad-1329-da9a-c6a7-192064fb04bb@oracle.com> Hi, Below is the updated webrev. http://cr.openjdk.java.net/~hb/8187498/webrev.03/ On Wednesday 14 February 2018 01:15 AM, mandy chung wrote: > > > On 2/13/18 1:30 AM, Harsha Wardhana B wrote: >> Hi, >> >> Please find below the revised webrev. >> >> http://cr.openjdk.java.net/~hb/8187498/webrev.02/ >> >> >> On Friday 09 February 2018 05:07 AM, mandy chung wrote: >>> On 2/7/18 1:19 AM, Harsha Wardhana B wrote: >>> > >>> > --start-management-agent will not be recognized in the current format and >>> > hence will not default to --start-management-agent=local=true. >>> >>> `--start-local-management-server` is one option as Alan suggests. >>> Another option is to make `--start-management-agent` to accept >>> an optional argument. VM can accept `--start-management-agent` >>> or `--start-management-agent=port=1234,ssl=on`. It may require >>> more launcher change to support it. >> Yes. It requires lot more changes to launcher. Hence optional >> argument to --start-management-agent was not added in order to keep >> the launcher impact minimum. > > This is just one option for you to consider.?? So the current proposal > of the new option to start a remote management server, right? Not necessarily. --start-management-agent=local=true starts local server and --start-management-agent=port=1234 starts remote management server. >>> Agent.java >>> If --start-management-agent is set, -Dcom.sun.management.* takes >>> precedence. Do you really want to do that? The new VM option >>> intends to simplify the command-line to type in. I think it's >>> cleaner and reasonable if --start-management-agent is specified, >>> -Dcom.sun.management.* are ignored (maybe worth emit a warning if set) >> We can probably document that -D options will be overridden instead >> of emitting a warning. > > What is the behavior when -Dcom.sun.management.jmxremote.port=1234 > --start-management-agent port=2345 > -Dcom.sun.management.jmxremote.port=3456? > > What is the value of the system property > com.sun.management.jmxremote.port at runtime?? What port number does > the management server start with? As said earlier, values set via new flags override values set by -D flags. So 2345 will be the value of com.sun.management.jmxremote.port. Added a test case to validate that. > >>> The implementation uses VMManagement::getVmArguments and scan >>> the VM options for -start-management-agent. The VM is the one >>> invoking jdk.internal.agent.Agent::startAgent. A simpler option >>> is to change Agent::startAgent to take an argument parameter >>> that will be passed with the argument of --start-management-agent >>> or null if not set. >>> >>> Similarly for startRemoteManagementAgent and startLocalAgent_name. >> Implementing this change requires lot of changes to argument parsing >> in native code and hence it is simpler to handle this at java layer. > > Can you describe how --start-management-agent option will be used for > jcmd and any other tool that attaches to a running VM to start the > agent?? Example command-line will be useful. --start-management-agent cannot be used by jcmd (or dynamic attach) as it accepts flags only in -D format or -D flags with com.sun.management removed. That code to parse string passed via jcmd was a mistake and hence I have removed it. > > > Mandy -Harsha -------------- next part -------------- An HTML attachment was scrubbed... URL: From stewartd.qdt at qualcommdatacenter.com Wed Feb 14 18:34:30 2018 From: stewartd.qdt at qualcommdatacenter.com (stewartd.qdt) Date: Wed, 14 Feb 2018 18:34:30 +0000 Subject: RFR: JDK-8196930 - [Testbug] serviceability/sa/ClhsdbFindPC.java fails to find expected output In-Reply-To: <544646a2-2ba2-a67c-ffeb-dd42c35c3b13@oracle.com> References: <527fbc12-4ff6-3946-346e-2a46903a3c30@oracle.com> <15041369-ba2a-4efb-aba8-b8ebcb1363b9@default> <544646a2-2ba2-a67c-ffeb-dd42c35c3b13@oracle.com> Message-ID: <1f5d25b298fa4ec28f4f194dc6a18f6b@NASANEXM01E.na.qualcomm.com> This test actually fails for me for the same reason that ClhsdbJstack.java fails, there is a frame in the stack of the main method that is returning null for the method. See the OpenJDK bug at https://bugs.openjdk.java.net/browse/JDK-8196969 for details of that issue. In this case I get the same problem, that a Method is null in the stack, which then causes a failure at 73 of ClhsdbFindPC.java. Daniel -----Original Message----- From: serviceability-dev [mailto:serviceability-dev-bounces at openjdk.java.net] On Behalf Of David Holmes Sent: Wednesday, February 14, 2018 12:40 AM To: Sharath Ballal ; serviceability-dev Subject: Re: RFR: JDK-8196930 - [Testbug] serviceability/sa/ClhsdbFindPC.java fails to find expected output On 14/02/2018 3:29 PM, Sharath Ballal wrote: > -----Original Message----- > From: David Holmes > Sent: Wednesday, February 14, 2018 10:53 AM > To: Sharath Ballal; serviceability-dev > Subject: Re: RFR: JDK-8196930 - [Testbug] > serviceability/sa/ClhsdbFindPC.java fails to find expected output > > On 14/02/2018 3:11 PM, Sharath Ballal wrote: >> David, >> When I wrote these tests, I looked for the output generated by these commands. The output I saw was same (across windows, mac, linux) and hence grepped for certain strings from it. >> Now we are seeing some issues where depending on the hardware we are at different execution point and hence some of the strings are changing. So now I am removing such strings from the test. >> The main intention of these tests are to check the basic sanity that the SA command is working. > >> Ok. But based on "In interpreter codelet" I would expect this would fail if run in -Xcomp mode ?? > This test works in both Xcomp and Xint mode. "In interpreter codelet" is being checked for Xint, for Xcomp we are checking "In code in NMethod for jdk/test/lib/apps/LingeredApp.main" Yep sorry - missed that bit. David > > Thanks, > David > >> Thanks, >> Sharath >> >> >> -----Original Message----- >> From: David Holmes >> Sent: Wednesday, February 14, 2018 7:38 AM >> To: Sharath Ballal; serviceability-dev >> Subject: Re: RFR: JDK-8196930 - [Testbug] >> serviceability/sa/ClhsdbFindPC.java fails to find expected output >> >> Hi Sharath, >> >> On 12/02/2018 7:12 PM, Sharath Ballal wrote: >>> Hello, >>> >>> Requesting reviews for the issue: >>> >>> ID: https://bugs.openjdk.java.net/browse/JDK-8196930 >>> >>> Webrev: http://cr.openjdk.java.net/~sballal/8196930/webrev.00/ >> >> I've no doubt this makes the test pass, but I always have to wonder with these tests: why was it looking for the thing now removed? what changed (or in what way was the expected output invalid) to make the test fail? >> >> Thanks, >> David >> >>> Tests run: The SA tests pass with Mach5. >>> >>> Thanks, >>> >>> Sharath >>> From serguei.spitsyn at oracle.com Wed Feb 14 22:30:03 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 14 Feb 2018 14:30:03 -0800 Subject: RFR 8196324: Update FilterMatch & FilterNoMatch to use TestScaffold instead of JDIScaffold In-Reply-To: <5A83DEBE.1010709@oracle.com> References: <5A81EA18.9080808@oracle.com> <5A83DEBE.1010709@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Wed Feb 14 22:30:41 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 14 Feb 2018 14:30:41 -0800 Subject: RFR(S): 8196909: gcc 7.2.1 compiler warning in libdt_socket Message-ID: Hello, Please review the following simple fix for a compiler warning: https://bugs.openjdk.java.net/browse/JDK-8196909 diff --git a/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c b/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c --- a/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c +++ b/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c @@ -398,7 +398,7 @@ ??????? return instr; ???? } -??? *mask = htonl(-1 << (32 - m)); +??? *mask = htonl((uint32_t)(~0) << (32 - m)); ???? return s; ?} Tested with all open and closed jdb, jdwp, and jdi test suites on all supported platforms. thanks, Chris From serguei.spitsyn at oracle.com Wed Feb 14 22:35:39 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 14 Feb 2018 14:35:39 -0800 Subject: RFR(S): 8196909: gcc 7.2.1 compiler warning in libdt_socket In-Reply-To: References: Message-ID: Hi Chris, Looks good. Thanks, Serguei On 2/14/18 14:30, Chris Plummer wrote: > Hello, > > Please review the following simple fix for a compiler warning: > > https://bugs.openjdk.java.net/browse/JDK-8196909 > > diff --git > a/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c > b/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c > --- a/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c > +++ b/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c > @@ -398,7 +398,7 @@ > ??????? return instr; > ???? } > > -??? *mask = htonl(-1 << (32 - m)); > +??? *mask = htonl((uint32_t)(~0) << (32 - m)); > ???? return s; > ?} > > > Tested with all open and closed jdb, jdwp, and jdi test suites on all > supported platforms. > > thanks, > > Chris > From parvathi.somashekar at oracle.com Thu Feb 15 00:39:45 2018 From: parvathi.somashekar at oracle.com (Paru Somashekar) Date: Wed, 14 Feb 2018 16:39:45 -0800 Subject: RFR 8196324: Update FilterMatch & FilterNoMatch to use TestScaffold instead of JDIScaffold In-Reply-To: References: <5A81EA18.9080808@oracle.com> <5A83DEBE.1010709@oracle.com> Message-ID: <5A84D6D1.1020400@oracle.com> Thanks Serguei, made the changes you have suggested, and the latest review is at: http://cr.openjdk.java.net/~psomashe/8196324/webrev.02/ thanks, Paru. On 2/14/18, 2:30 PM, serguei.spitsyn at oracle.com wrote: > Hi Paru, > > Thank you for the update! > > Could you, please, rearrange a couple of more places in both files? > > 66 BreakpointEvent bpe = startToMain("HelloWorld"); > 67 ReferenceType referenceType = (ClassType)bpe.location().declaringType(); > 68 > 69 mainThread = bpe.thread(); > 70 // VM has started, but hasn't started running the test program yet. > 71 EventRequestManager requestManager = vm().eventRequestManager(); > 72 > 73 Location loc = findLocation(referenceType, 3); > 74 > 75 BreakpointRequest bpRequest = requestManager.createBreakpointRequest(loc); > > I'd suggest to move the lines 68,69 after the line 75. > Also, the empty lines 72, 74 are not needed. > I'm not sure, the line 70 with the comment is placed correctly or > needed at all. > > This line needs an update of "request1": > 107 //request1.addClassFilter("x"); > > Extra space before '!' and missed space before '{' : > 115 if ( !stepCompleted){ > > > Let's simplify/unify the tracing lines below further: > > 130 System.out.println("Agent: StepEvent: line#=" + event.location().lineNumber() > 131 + " event=" + event); > . . . > 141 System.out.println("Agent: BreakpointEvent " + > 142 " at " + locStr + " in thread: " + thread); > > Something like this would be better: > 130 System.out.println("StepEvent at " + event.location()); > . . . > 141 System.out.println("BreakpointEvent at " + event.location()); > > > The lines 139 and 140 can be removed: > 139 ThreadReference thread = event.thread(); > 140 String locStr = "" + event.location(); > > Thanks, > Serguei > > > > On 2/13/18 23:01, Paru Somashekar wrote: >> Hi Serguei, >> >> Thanks very much for the review and I have made all the changes >> incorporating your feedback. >> >> New Webrev at >> :http://cr.openjdk.java.net/~psomashe/8196324/webrev.01/ >> >> >> thanks, >> Paru. >> >> On 2/13/18, 2:02 PM, serguei.spitsyn at oracle.com >> wrote: >>> Hi Paru, >>> >>> It looks good in general but I'd like to ask for some cleanup. >>> >>> I tell just about the first test (FilterMatch) but the other one >>> needs the same. >>> >>> 132 // This gets called if all filters match. >>> 133 public void stepCompleted(StepEvent event) { >>> 134 listenCalled = true; >>> 135 System.out.println("listen: line#=" + event.location().lineNumber() >>> 136 + " event=" + event); >>> 137 // disable the step and then run to completion >>> 138 StepRequest str= (StepRequest)event.request(); >>> 139 str.disable(); >>> 140 eventSet.resume(); >>> 141 } >>> I'd suggest to replace "listen:" above with "Agent: StepEvent:" to >>> make >>> it more consistent with similar tracing in the breakpointReached() >>> below. >>> >>> 143 public void breakpointReached(BreakpointEvent event) { >>> 144 ThreadReference thread = ((BreakpointEvent) event).thread(); >>> 145 String locStr = "" + ((BreakpointEvent) event).location(); >>> 146 System.out.println("Agent: BreakpointEvent #" + >>> 147 " at " + locStr + " in thread: " + thread); >>> 148 // The bkpt was hit; disable it. >>> 149 request.disable(); >>> 150 } >>> The casts to BreakpointEvent at lines 144, 145 are not needed >>> as the "event" is already of this type. >>> Unneeded sign '#' at the line 146. >>> Also, I'm suggesting to disable the "request" the same way >>> as it is done in the stepCompleted(): >>> BreakpointRequest bpr= (BreakpointRequest)event.request(); >>> bpr.disable(); >>> >>> It will help to make the "request" local in the runTests(). >>> Unneeded empty lines: 52, 54: >>> 51 EventSet eventSet = null; >>> 52 >>> 53 static boolean listenCalled; >>> 54 >>> 55 BreakpointRequest request; >>> >>> The listenCalled needs to be renamed to stepCompleted. >>> There is no big need for it to be static as it is similar to the >>> eventSet. >>> It is better to initialize it with false. >>> Then this line can be removed: >>> 114 listenCalled = false; >>> >>> I'd suggest to rename the variables "request" and "request1" >>> to "bpRequest" and "stepRequest" and make the 'bpRequest' to be local >>> in the runTests() as it is the only place where it has to be used. >>> >>> >>> Thanks, >>> Serguei >>> >>> >>> On 2/12/18 11:25, Paru Somashekar wrote: >>>> Hi, >>>> >>>> Please review fix for JDK-8196324 >>>> >>>> The includes the following changes: >>>> >>>> * Use startToMain method of TestScaffold that automatically calls >>>> connect() and waitForVMStart() >>>> * Since TestScaffold extends TargetAdapter, it now overrides event >>>> callbacks directly in the test class rather than implement an >>>> anonymous inner class. >>>> * Breakpoint handling is done in breakpointReached callback instead >>>> of waitForRequestedEvent and subsequently removing it. >>>> >>>> Bug : https://bugs.openjdk.java.net/browse/JDK-8196324 >>>> Review : http://cr.openjdk.java.net/~psomashe/8196324/webrev/ >>>> >>>> >>>> >>>> thanks, >>>> Paru. >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Thu Feb 15 00:50:42 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 14 Feb 2018 16:50:42 -0800 Subject: RFR 8196324: Update FilterMatch & FilterNoMatch to use TestScaffold instead of JDIScaffold In-Reply-To: <5A84D6D1.1020400@oracle.com> References: <5A81EA18.9080808@oracle.com> <5A83DEBE.1010709@oracle.com> <5A84D6D1.1020400@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From yasuenag at gmail.com Thu Feb 15 01:23:36 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Thu, 15 Feb 2018 10:23:36 +0900 Subject: PING: RFR: JDK-8153333: [REDO] STW phases at Concurrent GC should count in PerfCounter In-Reply-To: References: <2f4e0901-1602-6276-e7fd-84e168e7b317@gmail.com> <3210723e-c5f9-4277-a97a-be61e10e6b3d@oracle.com> <89a1a161-874d-4743-e52b-7ab202fd8976@gmail.com> <9799a2ad-6727-cae9-312b-fd9de13c8203@oracle.com> Message-ID: Hi all, CSR for this issue [1] has been approved. This webrev has been reviewed by Stefan, but we need one more reviewer. Could you review it? http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.07/ Thanks, Yasumasa [1] https://bugs.openjdk.java.net/browse/JDK-8196862 2018-02-06 22:33 GMT+09:00 Yasumasa Suenaga : > Hi Stefan, > >> This looks good to me, will do some more testing while waiting for a >> second reviewer and I can sponsor the change once it's ready to go. > > > Thanks! I'm waiting for second reviewer. > >>> What should I do to get CSR approve? >> >> In the bug system under "More" you can choose "Create CSR" which is the >> first step. More information can be found on the wiki: >> https://wiki.openjdk.java.net/display/csr/CSR+FAQs > > > I filed new CSR: > https://bugs.openjdk.java.net/browse/JDK-8196862 > > > Yasumasa > > > > On 2018/02/06 21:55, Stefan Johansson wrote: >> >> >> >> On 2018-02-06 06:10, Yasumasa Suenaga wrote: >>> >>> Hi Stefan, >>> >>>> I agree, for G1 this should not be controlled. Maybe I was a bit >>>> unclear, I >>>> was wondering why we want to control it for CMS. >>> >>> I said to remove -XX:EnableConcGCPerfCounter in two years ago. I've >>> missed it :-) >>> >>> >>> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-March/017125.html >>> >>> So I uploaded new webrev. This change includes copyright year updates. >>> >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.06/ >> >> Thanks Yasumasa, >> >> This looks good to me, will do some more testing while waiting for a >> second reviewer and I can sponsor the change once it's ready to go. >>> >>> >>> This change passes all tests on submit repo, and >>> :hotspot_serviceability :jdk_tools tests on my laptop. >>> >>> >>> http://java.se.oracle.com:10065/mdash/jobs/mach5-one-ysuenaga-JDK-8153333-20180206-0222-10428 >>> >>> >>>> If we do the change for CMS, we should >>>> probably also do a CSR, but that should be fairly straight forward. >>> >>> What should I do to get CSR approve? >> >> In the bug system under "More" you can choose "Create CSR" which is the >> first step. More information can be found on the wiki: >> https://wiki.openjdk.java.net/display/csr/CSR+FAQs >> >> Cheers, >> Stefan >> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> 2018-02-06 0:33 GMT+09:00 Stefan Johansson : >>>> >>>> >>>> On 2018-02-03 06:40, Yasumasa Suenaga wrote: >>>>> >>>>> On 2018/02/02 23:38, Stefan Johansson wrote: >>>>>> >>>>>> Hi Yasumasa, >>>>>> >>>>>> The changes doesn't apply clean on the latest jdk/hs, can you provide >>>>>> an >>>>>> updated webrev? >>>>> >>>>> >>>>> I uploaded webrev for jdk-hs: >>>>> cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.05/ >>>>> >>>> Thanks, I've kicked off a testing job now to verify nothing unexpected >>>> fails. >>>>> >>>>> >>>>>> The testing done by the submit repo doesn't cover the tests you have >>>>>> update so I plan to take the change for a spin and make sure the >>>>>> correct >>>>>> tests are run and verified in Mach 5. >>>>> >>>>> >>>>> I've also tested hotspot/jtreg/:hotspot_serviceability and >>>>> jdk/:jdk_tools >>>>> on my laptop. >>>>> I did not see any errors / failures which are related to this change. >>>> >>>> I also ran some local tests on this and it looks good. >>>>> >>>>> >>>>> >>>>>> Also a question about the change. Why do we need a special flag for >>>>>> CMS? >>>>>> I see that the original bug report refers to the flag as being a way >>>>>> to turn >>>>>> on and off the feature but the current implementation only consider >>>>>> the flag >>>>>> for CMS. >>>>> >>>>> >>>>> >>>>> >>>>> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-March/016774.html >>>>> >>>>> Originally, STW phases (Remark and Cleanup) at G1 are not counted in >>>>> jstat >>>>> FGC column. >>>>> So I think we need not to control the behavior of PerfCounter for G1. >>>>> >>>> I agree, for G1 this should not be controlled. Maybe I was a bit >>>> unclear, I >>>> was wondering why we want to control it for CMS. I think either we >>>> should >>>> change the behavior without guarding it by a flag or just skip updating >>>> CMS >>>> (and leave the pauses in FGC). If we do the change for CMS, we should >>>> probably also do a CSR, but that should be fairly straight forward. >>>> >>>> I also found the old review thread where Jon M had the same comment >>>> (removing the flag) and it looks like all agreed on that: >>>> >>>> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-March/017118.html >>>> >>>> Thanks, >>>> Stefan >>>> >>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>>> Thanks, >>>>>> Stefan >>>>>> >>>>>> On 2018-02-01 14:58, Yasumasa Suenaga wrote: >>>>>>> >>>>>>> PING: Could you review and sponsor it? >>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>>>>>> >>>>>>> >>>>>>> This change has been passed Mach 5 via submit repo: >>>>>>> >>>>>>> >>>>>>> http://java.se.oracle.com:10065/mdash/jobs/mach5-one-ysuenaga-JDK-8153333-20180201-0805-10101 >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2017/11/01 22:02, Yasumasa Suenaga wrote: >>>>>>>> >>>>>>>> PING: Could you review and sponsor it? >>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>>>>>>> >>>>>>>> >>>>>>>> Also I need JPRT results of this change. >>>>>>>> Could you cooperate? >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2017/09/27 0:08, Yasumasa Suenaga wrote: >>>>>>>>> >>>>>>>>> Hi all, >>>>>>>>> >>>>>>>>> I uploaded new webrev to be adapted to jdk10/hs: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>>>>>>>> >>>>>>>>> I want to check this patch via JPRT, but I cannot access it. >>>>>>>>> Could you cooperate? >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2017/09/21 7:46, Yasumasa Suenaga wrote: >>>>>>>>>> >>>>>>>>>> PING: >>>>>>>>>> >>>>>>>>>> Have you checked this issue? >>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/hotspot/ >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/jdk/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2017/07/01 23:44, Yasumasa Suenaga wrote: >>>>>>>>>>> >>>>>>>>>>> PING: >>>>>>>>>>> >>>>>>>>>>> Have you checked this issue? >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2017/06/14 13:22, Yasumasa Suenaga wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi all, >>>>>>>>>>>> >>>>>>>>>>>> I changed PerfCounter to show CGC STW phase in jstat in >>>>>>>>>>>> JDK-8151674. >>>>>>>>>>>> However, it occurred several jtreg test failure, so it was >>>>>>>>>>>> back-outed. >>>>>>>>>>>> >>>>>>>>>>>> I want to resume to work for this issue. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/hotspot/ >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/jdk/ >>>>>>>>>>>> >>>>>>>>>>>> These changes are work fine on jtreg test as below: >>>>>>>>>>>> >>>>>>>>>>>> hotspot/test/serviceability/tmtools/jstat >>>>>>>>>>>> jdk/test/sun/tools >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Since JDK 9, default GC algorithm is set to G1. >>>>>>>>>>>> So I think this change is useful to watch GC behavior through >>>>>>>>>>>> jstat. >>>>>>>>>>>> >>>>>>>>>>>> I cannot access JPRT. Could you help? >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> >> > From parvathi.somashekar at oracle.com Thu Feb 15 03:01:12 2018 From: parvathi.somashekar at oracle.com (Paru Somashekar) Date: Wed, 14 Feb 2018 19:01:12 -0800 Subject: RFR 8196324: Update FilterMatch & FilterNoMatch to use TestScaffold instead of JDIScaffold In-Reply-To: References: <5A81EA18.9080808@oracle.com> <5A83DEBE.1010709@oracle.com> <5A84D6D1.1020400@oracle.com> Message-ID: <5A84F7F8.9010500@oracle.com> Sound great, thanks Serguei. thanks, Paru. On 2/14/18, 4:50 PM, serguei.spitsyn at oracle.com wrote: > Paru, > > Thank you for the update. > > Just noticed one more thing in both files - extra space before the > parameter: > + addListener (this); > > Otherwise, it looks good. > No need in another webrev. > > Thanks, > Serguei > > > > On 2/14/18 16:39, Paru Somashekar wrote: >> Thanks Serguei, made the changes you have suggested, >> >> and the latest review is at: >> http://cr.openjdk.java.net/~psomashe/8196324/webrev.02/ >> >> >> thanks, >> Paru. >> >> >> On 2/14/18, 2:30 PM, serguei.spitsyn at oracle.com >> wrote: >>> Hi Paru, >>> >>> Thank you for the update! >>> >>> Could you, please, rearrange a couple of more places in both files? >>> >>> 66 BreakpointEvent bpe = startToMain("HelloWorld"); >>> 67 ReferenceType referenceType = (ClassType)bpe.location().declaringType(); >>> 68 >>> 69 mainThread = bpe.thread(); >>> 70 // VM has started, but hasn't started running the test program yet. >>> 71 EventRequestManager requestManager = vm().eventRequestManager(); >>> 72 >>> 73 Location loc = findLocation(referenceType, 3); >>> 74 >>> 75 BreakpointRequest bpRequest = requestManager.createBreakpointRequest(loc); >>> >>> I'd suggest to move the lines 68,69 after the line 75. >>> Also, the empty lines 72, 74 are not needed. >>> I'm not sure, the line 70 with the comment is placed correctly or >>> needed at all. >>> >>> This line needs an update of "request1": >>> 107 //request1.addClassFilter("x"); >>> >>> Extra space before '!' and missed space before '{' : >>> 115 if ( !stepCompleted){ >>> >>> >>> Let's simplify/unify the tracing lines below further: >>> >>> 130 System.out.println("Agent: StepEvent: line#=" + event.location().lineNumber() >>> 131 + " event=" + event); >>> . . . >>> 141 System.out.println("Agent: BreakpointEvent " + >>> 142 " at " + locStr + " in thread: " + thread); >>> >>> Something like this would be better: >>> 130 System.out.println("StepEvent at " + event.location()); >>> . . . >>> 141 System.out.println("BreakpointEvent at " + event.location()); >>> >>> >>> The lines 139 and 140 can be removed: >>> 139 ThreadReference thread = event.thread(); >>> 140 String locStr = "" + event.location(); >>> >>> Thanks, >>> Serguei >>> >>> >>> >>> On 2/13/18 23:01, Paru Somashekar wrote: >>>> Hi Serguei, >>>> >>>> Thanks very much for the review and I have made all the changes >>>> incorporating your feedback. >>>> >>>> New Webrev at >>>> :http://cr.openjdk.java.net/~psomashe/8196324/webrev.01/ >>>> >>>> >>>> thanks, >>>> Paru. >>>> >>>> On 2/13/18, 2:02 PM, serguei.spitsyn at oracle.com >>>> wrote: >>>>> Hi Paru, >>>>> >>>>> It looks good in general but I'd like to ask for some cleanup. >>>>> >>>>> I tell just about the first test (FilterMatch) but the other one >>>>> needs the same. >>>>> >>>>> 132 // This gets called if all filters match. >>>>> 133 public void stepCompleted(StepEvent event) { >>>>> 134 listenCalled = true; >>>>> 135 System.out.println("listen: line#=" + event.location().lineNumber() >>>>> 136 + " event=" + event); >>>>> 137 // disable the step and then run to completion >>>>> 138 StepRequest str= (StepRequest)event.request(); >>>>> 139 str.disable(); >>>>> 140 eventSet.resume(); >>>>> 141 } >>>>> I'd suggest to replace "listen:" above with "Agent: StepEvent:" >>>>> to make >>>>> it more consistent with similar tracing in the >>>>> breakpointReached() below. >>>>> >>>>> 143 public void breakpointReached(BreakpointEvent event) { >>>>> 144 ThreadReference thread = ((BreakpointEvent) event).thread(); >>>>> 145 String locStr = "" + ((BreakpointEvent) event).location(); >>>>> 146 System.out.println("Agent: BreakpointEvent #" + >>>>> 147 " at " + locStr + " in thread: " + thread); >>>>> 148 // The bkpt was hit; disable it. >>>>> 149 request.disable(); >>>>> 150 } >>>>> The casts to BreakpointEvent at lines 144, 145 are not needed >>>>> as the "event" is already of this type. >>>>> Unneeded sign '#' at the line 146. >>>>> Also, I'm suggesting to disable the "request" the same way >>>>> as it is done in the stepCompleted(): >>>>> BreakpointRequest bpr= (BreakpointRequest)event.request(); >>>>> bpr.disable(); >>>>> >>>>> It will help to make the "request" local in the runTests(). >>>>> Unneeded empty lines: 52, 54: >>>>> 51 EventSet eventSet = null; >>>>> 52 >>>>> 53 static boolean listenCalled; >>>>> 54 >>>>> 55 BreakpointRequest request; >>>>> >>>>> The listenCalled needs to be renamed to stepCompleted. >>>>> There is no big need for it to be static as it is similar to the >>>>> eventSet. >>>>> It is better to initialize it with false. >>>>> Then this line can be removed: >>>>> 114 listenCalled = false; >>>>> >>>>> I'd suggest to rename the variables "request" and "request1" >>>>> to "bpRequest" and "stepRequest" and make the 'bpRequest' to be local >>>>> in the runTests() as it is the only place where it has to be used. >>>>> >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>> On 2/12/18 11:25, Paru Somashekar wrote: >>>>>> Hi, >>>>>> >>>>>> Please review fix for JDK-8196324 >>>>>> >>>>>> The includes the following changes: >>>>>> >>>>>> * Use startToMain method of TestScaffold that automatically calls >>>>>> connect() and waitForVMStart() >>>>>> * Since TestScaffold extends TargetAdapter, it now overrides >>>>>> event callbacks directly in the test class rather than implement >>>>>> an anonymous inner class. >>>>>> * Breakpoint handling is done in breakpointReached callback >>>>>> instead of waitForRequestedEvent and subsequently removing it. >>>>>> >>>>>> Bug : https://bugs.openjdk.java.net/browse/JDK-8196324 >>>>>> Review : http://cr.openjdk.java.net/~psomashe/8196324/webrev/ >>>>>> >>>>>> >>>>>> >>>>>> thanks, >>>>>> Paru. >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sharath.ballal at oracle.com Thu Feb 15 03:48:43 2018 From: sharath.ballal at oracle.com (Sharath Ballal) Date: Wed, 14 Feb 2018 19:48:43 -0800 (PST) Subject: RFR: JDK-8196930 - [Testbug] serviceability/sa/ClhsdbFindPC.java fails to find expected output In-Reply-To: <1f5d25b298fa4ec28f4f194dc6a18f6b@NASANEXM01E.na.qualcomm.com> References: <527fbc12-4ff6-3946-346e-2a46903a3c30@oracle.com> <15041369-ba2a-4efb-aba8-b8ebcb1363b9@default> <544646a2-2ba2-a67c-ffeb-dd42c35c3b13@oracle.com> <1f5d25b298fa4ec28f4f194dc6a18f6b@NASANEXM01E.na.qualcomm.com> Message-ID: <97e60981-2f2b-493c-b752-9c30bf0c7e8e@default> Daniel, This is a different problem than getting the null method. Here we are expecting "invoke return entry points" in the output of findpc command but getting ""method entry point". The null method problem seems to be specific to AArch64 system and I am looking at it separately. Thanks, Sharath -----Original Message----- From: stewartd.qdt [mailto:stewartd.qdt at qualcommdatacenter.com] Sent: Thursday, February 15, 2018 12:05 AM To: David Holmes; Sharath Ballal; serviceability-dev Subject: RE: RFR: JDK-8196930 - [Testbug] serviceability/sa/ClhsdbFindPC.java fails to find expected output This test actually fails for me for the same reason that ClhsdbJstack.java fails, there is a frame in the stack of the main method that is returning null for the method. See the OpenJDK bug at https://bugs.openjdk.java.net/browse/JDK-8196969 for details of that issue. In this case I get the same problem, that a Method is null in the stack, which then causes a failure at 73 of ClhsdbFindPC.java. Daniel -----Original Message----- From: serviceability-dev [mailto:serviceability-dev-bounces at openjdk.java.net] On Behalf Of David Holmes Sent: Wednesday, February 14, 2018 12:40 AM To: Sharath Ballal ; serviceability-dev Subject: Re: RFR: JDK-8196930 - [Testbug] serviceability/sa/ClhsdbFindPC.java fails to find expected output On 14/02/2018 3:29 PM, Sharath Ballal wrote: > -----Original Message----- > From: David Holmes > Sent: Wednesday, February 14, 2018 10:53 AM > To: Sharath Ballal; serviceability-dev > Subject: Re: RFR: JDK-8196930 - [Testbug] > serviceability/sa/ClhsdbFindPC.java fails to find expected output > > On 14/02/2018 3:11 PM, Sharath Ballal wrote: >> David, >> When I wrote these tests, I looked for the output generated by these commands. The output I saw was same (across windows, mac, linux) and hence grepped for certain strings from it. >> Now we are seeing some issues where depending on the hardware we are at different execution point and hence some of the strings are changing. So now I am removing such strings from the test. >> The main intention of these tests are to check the basic sanity that the SA command is working. > >> Ok. But based on "In interpreter codelet" I would expect this would fail if run in -Xcomp mode ?? > This test works in both Xcomp and Xint mode. "In interpreter codelet" is being checked for Xint, for Xcomp we are checking "In code in NMethod for jdk/test/lib/apps/LingeredApp.main" Yep sorry - missed that bit. David > > Thanks, > David > >> Thanks, >> Sharath >> >> >> -----Original Message----- >> From: David Holmes >> Sent: Wednesday, February 14, 2018 7:38 AM >> To: Sharath Ballal; serviceability-dev >> Subject: Re: RFR: JDK-8196930 - [Testbug] >> serviceability/sa/ClhsdbFindPC.java fails to find expected output >> >> Hi Sharath, >> >> On 12/02/2018 7:12 PM, Sharath Ballal wrote: >>> Hello, >>> >>> Requesting reviews for the issue: >>> >>> ID: https://bugs.openjdk.java.net/browse/JDK-8196930 >>> >>> Webrev: http://cr.openjdk.java.net/~sballal/8196930/webrev.00/ >> >> I've no doubt this makes the test pass, but I always have to wonder with these tests: why was it looking for the thing now removed? what changed (or in what way was the expected output invalid) to make the test fail? >> >> Thanks, >> David >> >>> Tests run: The SA tests pass with Mach5. >>> >>> Thanks, >>> >>> Sharath >>> From david.holmes at oracle.com Thu Feb 15 04:58:09 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 15 Feb 2018 14:58:09 +1000 Subject: [8u] RFR for 'JDK-8195088: [TEST_BUG] StartManagementAgent got unexpected exception' In-Reply-To: <5a466d6e-a899-45bf-982a-dc35e06c61a4@default> References: <35e118d1-5f59-4947-bad0-4f5fbc032474@default> <5a466d6e-a899-45bf-982a-dc35e06c61a4@default> Message-ID: Reviewed. Thanks, David On 6/02/2018 3:56 PM, Shafi Ahmad wrote: > Hi, > > Could someone please review it. > > Regards, > Shafi > >> -----Original Message----- >> From: Shafi Ahmad >> Sent: Monday, January 29, 2018 6:02 PM >> To: serviceability-dev at openjdk.java.net >> Subject: [8u] RFR for 'JDK-8195088: [TEST_BUG] StartManagementAgent got >> unexpected exception' >> >> Hi, >> >> Please review the trivial code change for the fix of 'JDK-8195088: [TEST_BUG] >> StartManagementAgent got unexpected exception' to jdk8u-dev. >> >> Summary: >> This test case is expected to fail but the expected exception message is >> different. >> This fix >> - if (!ex.getMessage().contains("Invalid >> com.sun.management.jmxremote.port number")) { >> + if (!ex.getMessage().contains("NumberFormatException: For input >> string: \"apa\"")) { >> >> is checking for the correct message. >> >> This is fixed in jdk10 under bug JDK-8165736 [Error message should be shown >> when JVMTI agent cannot be attache] >> http://hg.openjdk.java.net/jdk/hs/rev/bc1cffa26561#l10.1 >> >> Jdk8 bug: https://bugs.openjdk.java.net/browse/JDK-8195088 >> webrev link: http://cr.openjdk.java.net/~shshahma/8195088/webrev.00/ >> >> Regards, >> Shafi From jcbeyler at google.com Thu Feb 15 06:34:48 2018 From: jcbeyler at google.com (JC Beyler) Date: Wed, 14 Feb 2018 22:34:48 -0800 Subject: JDK-8171119: Low-Overhead Heap Profiling In-Reply-To: <5A8414AC.3020209@oracle.com> References: <5A819F10.8040201@oracle.com> <5A8414AC.3020209@oracle.com> Message-ID: Hi Erik, I inlined my answers, which the last one seems to answer Robbin's concerns about the same thing (adding things to Thread). On Wed, Feb 14, 2018 at 2:51 AM, Erik ?sterlund wrote: > Hi JC, > > Comments are inlined below. > > > On 2018-02-13 06:18, JC Beyler wrote: > > Hi Erik, > > Thanks for your answers, I've now inlined my own answers/comments. > > I've done a new webrev here: > http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.08/ > > The incremental is here: > http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.07_08/ > > Note to all: > - I've been integrating changes from Erin/Serguei/David comments so this > webrev incremental is a bit an answer to all comments in one. I apologize > for that :) > > > On Mon, Feb 12, 2018 at 6:05 AM, Erik ?sterlund > wrote: > >> Hi JC, >> >> Sorry for the delayed reply. >> >> Inlined answers: >> >> >> On 2018-02-06 00:04, JC Beyler wrote: >> >>> Hi Erik, >>> >>> (Renaming this to be folded into the newly renamed thread :)) >>> >>> First off, thanks a lot for reviewing the webrev! I appreciate it! >>> >>> I updated the webrev to: >>> http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.05a/ >>> >>> And the incremental one is here: >>> http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.04_05a/ >>> >>> It contains: >>> - The change for since from 9 to 11 for the jvmti.xml >>> - The use of the OrderAccess for initialized >>> - Clearing the oop >>> >>> I also have inlined my answers to your comments. The biggest question >>> will come from the multiple *_end variables. A bit of the logic there >>> is due to handling the slow path refill vs fast path refill and >>> checking that the rug was not pulled underneath the slowpath. I >>> believe that a previous comment was that TlabFastRefill was going to >>> be deprecated. >>> >>> If this is true, we could revert this code a bit and just do a : if >>> TlabFastRefill is enabled, disable this. And then deprecate that when >>> TlabFastRefill is deprecated. >>> >>> This might simplify this webrev and I can work on a follow-up that >>> either: removes TlabFastRefill if Robbin does not have the time to do >>> it or add the support to the assembly side to handle this correctly. >>> What do you think? >>> >> >> I support removing TlabFastRefill, but I think it is good to not depend >> on that happening first. >> >> > > I'm slowly pushing on the FastTLABRefill ( > > https://bugs.openjdk.java.net/browse/JDK-8194084), I agree on keeping > both separate for now though so that we can think of both differently > > > >> Now, below, inlined are my answers: >>> >>> On Fri, Feb 2, 2018 at 8:44 AM, Erik ?sterlund >>> wrote: >>> >>>> Hi JC, >>>> >>>> Hope I am reviewing the right version of your work. Here goes... >>>> >>>> src/hotspot/share/gc/shared/collectedHeap.inline.hpp: >>>> >>>> 159 AllocTracer::send_allocation_outside_tlab(klass, result, >>>> size * >>>> HeapWordSize, THREAD); >>>> 160 >>>> 161 THREAD->tlab().handle_sample(THREAD, result, size); >>>> 162 return result; >>>> 163 } >>>> >>>> Should not call tlab()->X without checking if (UseTLAB) IMO. >>>> >>>> Done! >>> >> >> More about this later. >> >> >> >>> src/hotspot/share/gc/shared/threadLocalAllocBuffer.cpp: >>>> >>>> So first of all, there seems to quite a few ends. There is an "end", a >>>> "hard >>>> end", a "slow path end", and an "actual end". Moreover, it seems like >>>> the >>>> "hard end" is actually further away than the "actual end". So the "hard >>>> end" >>>> seems like more of a "really definitely actual end" or something. I >>>> don't >>>> know about you, but I think it looks kind of messy. In particular, I >>>> don't >>>> feel like the name "actual end" reflects what it represents, especially >>>> when >>>> there is another end that is behind the "actual end". >>>> >>>> 413 HeapWord* ThreadLocalAllocBuffer::hard_end() { >>>> 414 // Did a fast TLAB refill occur? >>>> 415 if (_slow_path_end != _end) { >>>> 416 // Fix up the actual end to be now the end of this TLAB. >>>> 417 _slow_path_end = _end; >>>> 418 _actual_end = _end; >>>> 419 } >>>> 420 >>>> 421 return _actual_end + alignment_reserve(); >>>> 422 } >>>> >>>> I really do not like making getters unexpectedly have these kind of side >>>> effects. It is not expected that when you ask for the "hard end", you >>>> implicitly update the "slow path end" and "actual end" to new values. >>>> >>>> As I said, a lot of this is due to the FastTlabRefill. If I make this >>> not supporting FastTlabRefill, this goes away. The reason the system >>> needs to update itself at the get is that you only know at that get if >>> things have shifted underneath the tlab slow path. I am not sure of >>> really better names (naming is hard!), perhaps we could do these >>> names: >>> >>> - current_tlab_end // Either the allocated tlab end or a sampling >>> point >>> - last_allocation_address // The end of the tlab allocation >>> - last_slowpath_allocated_end // In case a fast refill occurred the >>> end might have changed, this is to remember slow vs fast past refills >>> >>> the hard_end method can be renamed to something like: >>> tlab_end_pointer() // The end of the lab including a bit of >>> alignment reserved bytes >>> >> >> Those names sound better to me. Could you please provide a mapping from >> the old names to the new names so I understand which one is which please? >> >> This is my current guess of what you are proposing: >> >> end -> current_tlab_end >> actual_end -> last_allocation_address >> slow_path_end -> last_slowpath_allocated_end >> hard_end -> tlab_end_pointer >> >> > Yes that is correct, that was what I was proposing. > > >> I would prefer this naming: >> >> end -> slow_path_end // the end for taking a slow path; either due to >> sampling or refilling >> actual_end -> allocation_end // the end for allocations >> slow_path_end -> last_slow_path_end // last address for slow_path_end (as >> opposed to allocation_end) >> hard_end -> reserved_end // the end of the reserved space of the TLAB >> >> About setting things in the getter... that still seems like a very >> unpleasant thing to me. It would be better to inspect the call hierarchy >> and explicitly update the ends where they need updating, and assert in the >> getter that they are in sync, rather than implicitly setting various ends >> as a surprising side effect in a getter. It looks like the call hierarchy >> is very small. With my new naming convention, reserved_end() would >> presumably return _allocation_end + alignment_reserve(), and have an assert >> checking that _allocation_end == _last_slow_path_allocation_end, >> complaining that this invariant must hold, and that a caller to this >> function, such as make_parsable(), must first explicitly synchronize the >> ends as required, to honor that invariant. >> >> > > I've renamed the variables to how you preferred it except for the _end > one. I did: > current_end > last_allocation_address > tlab_end_ptr > > The reason is that the architecture dependent code use the thread.hpp API > and it already has tlab included into the name so it becomes > tlab_current_end (which is better that tlab_current_tlab_end in my opinion). > > I also moved the update into a separate method with a TODO that says to > remove it when FastTLABRefill is deprecated > > > This looks a lot better now. Thanks. > > Note that the following comment now needs updating accordingly in > threadLocalAllocBuffer.hpp: > > 41 // Heap sampling is performed via the end/actual_end fields. 42 // actual_end contains the real end of the tlab allocation, 43 // whereas end can be set to an arbitrary spot in the tlab to 44 // trip the return and sample the allocation. 45 // slow_path_end is used to track if a fast tlab refill occured 46 // between slowpath calls. > > There might be other comments too, I have not looked in detail. > This was the only spot that still had an actual_end, I fixed it now. I'll do a sweep to double check other comments. > > > > > >> >> Not sure it's better but before updating the webrev, I wanted to try >>> to get input/consensus :) >>> >>> (Note hard_end was always further off than end). >>> >>> src/hotspot/share/prims/jvmti.xml: >>>> >>>> 10357 >>>> 10358 >>>> 10359 Can sample the heap. >>>> 10360 If this capability is enabled then the heap sampling >>>> methods >>>> can be called. >>>> 10361 >>>> 10362 >>>> >>>> Looks like this capability should not be "since 9" if it gets integrated >>>> now. >>>> >>> Updated now to 11, crossing my fingers :) >>> >>> >>> src/hotspot/share/runtime/heapMonitoring.cpp: >>>> >>>> 448 if (is_alive->do_object_b(value)) { >>>> 449 // Update the oop to point to the new object if it is >>>> still >>>> alive. >>>> 450 f->do_oop(&(trace.obj)); >>>> 451 >>>> 452 // Copy the old trace, if it is still live. >>>> 453 _allocated_traces->at_put(curr_pos++, trace); >>>> 454 >>>> 455 // Store the live trace in a cache, to be served up on >>>> /heapz. >>>> 456 _traces_on_last_full_gc->append(trace); >>>> 457 >>>> 458 count++; >>>> 459 } else { >>>> 460 // If the old trace is no longer live, add it to the list >>>> of >>>> 461 // recently collected garbage. >>>> 462 store_garbage_trace(trace); >>>> 463 } >>>> >>>> In the case where the oop was not live, I would like it to be explicitly >>>> cleared. >>>> >>> Done I think how you wanted it. Let me know because I'm not familiar >>> with the RootAccess API. I'm unclear if I'm doing this right or not so >>> reviews of these parts are highly appreciated. Robbin had talked of >>> perhaps later pushing this all into a OopStorage, should I do this now >>> do you think? Or can that wait a second webrev later down the road? >>> >> >> I think using handles can and should be done later. You can use the >> Access API now. >> I noticed that you are missing an #include "oops/access.inline.hpp" in >> your heapMonitoring.cpp file. >> >> > The missing header is there for me so I don't know, I made sure it is > present in the latest webrev. Sorry about that. > > > >> + Did I clear it the way you wanted me to or were you thinking of >>> something else? >>> >> >> That is precisely how I wanted it to be cleared. Thanks. >> >> + Final question here, seems like if I were to want to not do the >>> f->do_oop directly on the trace.obj, I'd need to do something like: >>> >>> f->do_oop(&value); >>> ... >>> trace->store_oop(value); >>> >>> to update the oop internally. Is that right/is that one of the >>> advantages of going to the Oopstorage sooner than later? >>> >> >> I think you really want to do the do_oop on the root directly. Is there a >> particular reason why you would not want to do that? >> Otherwise, yes - the benefit with using the handle approach is that you >> do not need to call do_oop explicitly in your code. >> >> > There is no reason except that now we have a load_oop and a get_oop_addr, > I was not sure what you would think of that. > > > That's fine. > > > >> >>> Also I see a lot of concurrent-looking use of the following field: >>>> 267 volatile bool _initialized; >>>> >>>> Please note that the "volatile" qualifier does not help with reordering >>>> here. Reordering between volatile and non-volatile fields is completely >>>> free >>>> for both compiler and hardware, except for windows with MSVC, where >>>> volatile >>>> semantics is defined to use acquire/release semantics, and the hardware >>>> is >>>> TSO. But for the general case, I would expect this field to be stored >>>> with >>>> OrderAccess::release_store and loaded with OrderAccess::load_acquire. >>>> Otherwise it is not thread safe. >>>> >>> Because everything is behind a mutex, I wasn't really worried about >>> this. I have a test that has multiple threads trying to hit this >>> corner case and it passes. >>> >>> However, to be paranoid, I updated it to using the OrderAccess API >>> now, thanks! Let me know what you think there too! >>> >> >> If it is indeed always supposed to be read and written under a mutex, >> then I would strongly prefer to have it accessed as a normal non-volatile >> member, and have an assertion that given lock is held or we are in a >> safepoint, as we do in many other places. Something like this: >> >> assert(HeapMonitorStorage_lock->owned_by_self() || >> (SafepointSynchronize::is_at_safepoint() && >> Thread::current()->is_VM_thread()), "this should not be accessed >> concurrently"); >> >> It would be confusing to people reading the code if there are uses of >> OrderAccess that are actually always protected under a mutex. >> >> > Thank you for the exact example to be put in the code! I put it around > each access/assignment of the _initialized method and found one case where > yes you can touch it and not have the lock. It actually is "ok" because you > don't act on the storage until later and only when you really want to > modify the storage (see the object_alloc_do_sample method which calls the > add_trace method). > > But, because of this, I'm going to put the OrderAccess here, I'll do some > performance numbers later and if there are issues, I might add a "unsafe" > read and a "safe" one to make it explicit to the reader. But I don't think > it will come to that. > > > Okay. This double return in heapMonitoring.cpp looks wrong: > > 283 bool initialized() { > 284 return OrderAccess::load_acquire(&_initialized) != 0; > 285 return _initialized; > 286 } > > Since you said object_alloc_do_sample() is the only place where you do not > hold the mutex while reading initialized(), I had a closer look at that. It > looks like in its current shape, the lack of a mutex may lead to a memory > leak. In particular, it first checks if (initialized()). Let's assume this > is now true. It then allocates a bunch of stuff, and checks if the number > of frames were over 0. If they were, it calls StackTraceStorage::storage()->add_trace() > seemingly hoping that after grabbing the lock in there, initialized() will > still return true. But it could now return false and skip doing anything, > in which case the allocated stuff will never be freed. > I fixed this now by making add_trace return a boolean and checking for that. It will be in the next webrev. Thanks, the truth is that in our implementation the system is always on or off, so this never really occurs :). In this version though, that is not true and it's important to handle so thanks again! > > So the analysis seems to be that _initialized is only used outside of the > mutex in once instance, where it is used to perform double-checked locking, > that actually causes a memory leak. > > I am not proposing how to fix that, just raising the issue. If you still > want to perform this double-checked locking somehow, then the use of > acquire/release still seems odd. Because the memory ordering restrictions > of it never comes into play in this particular case. If it ever did, then > the use of destroy_stuff(); release_store(_initialized, 0) would be broken > anyway as that would imply that whatever concurrent reader there ever was > would after reading _initialized with load_acquire() could *never* read the > data that is concurrently destroyed anyway. I would be biased to think that > RawAccess::load/store looks like a more appropriate solution, > given that the memory leak issue is resolved. I do not know how painful it > would be to not perform this double-checked locking. > So I agree with this entirely. I looked also a bit more and the difference and code really stems from our internal version. In this version however, there are actually a lot of things going on that I did not go entirely through in my head but this comment made me ponder a bit more on it. Since every object_alloc_do_sample is protected by a check to HeapMonitoring::enabled(), there is only a small chance that the call is happening when things have been disabled. So there is no real need to do a first check on the initialized, it is a rare occurence that a call happens to object_alloc_do_sample and the initialized of the storage returns false. (By the way, even if you did call object_alloc_do_sample without looking at HeapMonitoring::enabled(), that would be ok too. You would gather the stacktrace and get nowhere at the add_trace call, which would return false; so though not optimal performance wise, nothing would break). Furthermore, the add_trace is really the moment of no return and we have the mutex lock and then the initialized check. So, in the end, I did two things: I removed that first check and then I removed the OrderAccess for the storage initialized. I think now I have a better grasp and understanding why it was done in our code and why it is not needed here. Thanks for pointing it out :). This now still passes my JTREG tests, especially the threaded one. > > > > > >> As a kind of meta comment, I wonder if it would make sense to add sampling >>>> for non-TLAB allocations. Seems like if someone is rapidly allocating a >>>> whole bunch of 1 MB objects that never fit in a TLAB, I might still be >>>> interested in seeing that in my traces, and not get surprised that the >>>> allocation rate is very high yet not showing up in any profiles. >>>> >>>> That is handled by the handle_sample where you wanted me to put a >>> UseTlab because you hit that case if the allocation is too big. >>> >> >> I see. It was not obvious to me that non-TLAB sampling is done in the >> TLAB class. That seems like an abstraction crime. >> What I wanted in my previous comment was that we do not call into the >> TLAB when we are not using TLABs. If there is sampling logic in the TLAB >> that is used for something else than TLABs, then it seems like that logic >> simply does not belong inside of the TLAB. It should be moved out of the >> TLAB, and instead have the TLAB call this common abstraction that makes >> sense. >> >> > So in the incremental version: > http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.07_08/, this is still > a "crime". The reason is that the system has to have the bytes_until_sample > on a per-thread level and it made "sense" to have it with the TLAB > implementation. Also, I was not sure how people felt about adding something > to the thread instance instead. > > Do you think it fits better at the Thread level? I can see how difficult > it is to make it happen there and add some logic there. Let me know what > you think. > > > We have an unfortunate situation where everyone that has some fields that > are thread local tend to dump them right into Thread, making the size and > complexity of Thread grow as it becomes tightly coupled with various > unrelated subsystems. It would be desirable to have a separate class for > this instead that encapsulates the sampling logic. That class could > possibly reside in Thread though as a value object of Thread. > I imagined that would be the case but was not sure. I will look at the example that Robbin is talking about (ThreadSMR) and will see how to refactor my code to use that. Thanks again for your help, Jc > > > > > >> Hope I have answered your questions and that my feedback makes sense to >> you. >> >> > You have and thank you for them, I think we are getting to a cleaner > implementation and things are getting better and more readable :) > > > Yes it is getting better. > > Thanks, > /Erik > > > Thanks for your help! > Jc > > > >> Thanks, >> /Erik >> >> >> I double checked by changing the test >>> http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.05a/raw_ >>> files/new/test/hotspot/jtreg/serviceability/jvmti/HeapMonito >>> r/MyPackage/HeapMonitorStatObjectCorrectnessTest.java >>> >>> to use a smaller Tlab (2048) and made the object bigger and it goes >>> through that and passes. >>> >>> Thanks again for your review and I look forward to your pointers for >>> the questions I now have raised! >>> Jc >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> Thanks, >>>> /Erik >>>> >>>> >>>> On 2018-01-26 06:45, JC Beyler wrote: >>>> >>>>> Thanks Robbin for the reviews :) >>>>> >>>>> The new full webrev is here: >>>>> http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.03/ >>>>> The incremental webrev is here: >>>>> http://cr.openjdk.java.net/~jcbeyler/8171119/webrev.02_03/ >>>>> >>>>> I inlined my answers: >>>>> >>>>> On Thu, Jan 25, 2018 at 1:15 AM, Robbin Ehn < >>>>> robbin.ehn at oracle.com> wrote: >>>>> >>>>>> Hi JC, great to see another revision! >>>>>> >>>>>> #### >>>>>> heapMonitoring.cpp >>>>>> >>>>>> StackTraceData should not contain the oop for 'safety' reasons. >>>>>> When StackTraceData is moved from _allocated_traces: >>>>>> L452 store_garbage_trace(trace); >>>>>> it contains a dead oop. >>>>>> _allocated_traces could instead be a tupel of oop and StackTraceData >>>>>> thus >>>>>> dead oops are not kept. >>>>>> >>>>> Done I used inheritance to make the copier work regardless but the >>>>> idea is the same. >>>>> >>>>> You should use the new Access API for loading the oop, something like >>>>>> this: >>>>>> RootAccess::load(...) >>>>>> I don't think you need to use Access API for clearing the oop, but it >>>>>> would >>>>>> look nicer. And you shouldn't probably be using: >>>>>> Universe::heap()->is_in_reserved(value) >>>>>> >>>>> I am unfamiliar with this but I think I did do it like you wanted me >>>>> to (all tests pass so that's a start). I'm not sure how to clear the >>>>> oop exactly, is there somewhere that does that, which I can use to do >>>>> the same? >>>>> >>>>> I removed the is_in_reserved, this came from our internal version, I >>>>> don't know why it was there but my tests work without so I removed it >>>>> :) >>>>> >>>>> >>>>> The lock: >>>>>> L424 MutexLocker mu(HeapMonitorStorage_lock); >>>>>> Is not needed as far as I can see. >>>>>> weak_oops_do is called in a safepoint, no TLAB allocation can happen >>>>>> and >>>>>> JVMTI thread can't access these data-structures. Is there something >>>>>> more >>>>>> to >>>>>> this lock that I'm missing? >>>>>> >>>>> Since a thread can call the JVMTI getLiveTraces (or any of the other >>>>> ones), it can get to the point of trying to copying the >>>>> _allocated_traces. I imagine it is possible that this is happening >>>>> during a GC or that it can be started and a GC happens afterwards. >>>>> Therefore, it seems to me that you want this protected, no? >>>>> >>>>> >>>>> #### >>>>>> You have 6 files without any changes in them (any more): >>>>>> g1CollectedHeap.cpp >>>>>> psMarkSweep.cpp >>>>>> psParallelCompact.cpp >>>>>> genCollectedHeap.cpp >>>>>> referenceProcessor.cpp >>>>>> thread.hpp >>>>>> >>>>>> Done. >>>>> >>>>> #### >>>>>> I have not looked closely, but is it possible to hide heap sampling in >>>>>> AllocTracer ? (with some minor changes to the AllocTracer API) >>>>>> >>>>>> I am imagining that you are saying to move the code that does the >>>>> sampling code (change the tlab end, do the call to HeapMonitoring, >>>>> etc.) into the AllocTracer code itself? I think that is right and I'll >>>>> look if that is possible and prepare a webrev to show what would be >>>>> needed to make that happen. >>>>> >>>>> #### >>>>>> Minor nit, when declaring pointer there is a little mix of having the >>>>>> pointer adjacent by type name and data name. (Most hotspot code is by >>>>>> type >>>>>> name) >>>>>> E.g. >>>>>> heapMonitoring.cpp:711 jvmtiStackTrace *trace = .... >>>>>> heapMonitoring.cpp:733 Method* m = vfst.method(); >>>>>> (not just this file) >>>>>> >>>>>> Done! >>>>> >>>>> #### >>>>>> HeapMonitorThreadOnOffTest.java:77 >>>>>> I would make g_tmp volatile, otherwise the assignment in loop may >>>>>> theoretical be skipped. >>>>>> >>>>>> Also done! >>>>> >>>>> Thanks again! >>>>> Jc >>>>> >>>> >>>> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From christoph.langer at sap.com Thu Feb 15 07:47:52 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 15 Feb 2018 07:47:52 +0000 Subject: RFR(S): 8196909: gcc 7.2.1 compiler warning in libdt_socket In-Reply-To: References: Message-ID: <7d53497f82a84536abd2726fd99dcbc9@sap.com> Hi Chris, looks good. Best regards Christoph > -----Original Message----- > From: serviceability-dev [mailto:serviceability-dev- > bounces at openjdk.java.net] On Behalf Of Chris Plummer > Sent: Mittwoch, 14. Februar 2018 23:31 > To: serviceability-dev > Cc: Indu Bhagat > Subject: RFR(S): 8196909: gcc 7.2.1 compiler warning in libdt_socket > > Hello, > > Please review the following simple fix for a compiler warning: > > https://bugs.openjdk.java.net/browse/JDK-8196909 > > diff --git > a/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c > b/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c > --- a/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c > +++ b/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c > @@ -398,7 +398,7 @@ > ??????? return instr; > ???? } > > -??? *mask = htonl(-1 << (32 - m)); > +??? *mask = htonl((uint32_t)(~0) << (32 - m)); > ???? return s; > ?} > > > Tested with all open and closed jdb, jdwp, and jdi test suites on all > supported platforms. > > thanks, > > Chris From robin.westberg at oracle.com Thu Feb 15 13:35:23 2018 From: robin.westberg at oracle.com (Robin Westberg) Date: Thu, 15 Feb 2018 14:35:23 +0100 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: <0F81FDA5-443F-4E5D-9AA0-895AB48BEA73@oracle.com> References: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> <5622d8af-d0d6-e77a-f7c1-f2c474f89763@oracle.com> <3EDE8C81-4FBD-4D40-A798-5F71CF7DD2B8@oracle.com> <0362f486-b446-4023-9914-0469d6274b7a@oracle.com> <4730283A-A1B1-46C4-9E04-C3993B62E248@oracle.com> <29aea368-9908-72d9-9dae-107a15a63043@Oracle.com> <0F81FDA5-443F-4E5D-9AA0-895AB48BEA73@oracle.com> Message-ID: <8CDB2E80-3841-4A02-ADA5-45BB842FEB82@oracle.com> Hi again, Here?s the (hopefully) final version: Full: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.02/ Incremental: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.01-02/ Best regards, Robin > On 14 Feb 2018, at 16:15, Robin Westberg wrote: > > Hi Roger, > >> On 13 Feb 2018, at 16:17, Roger Riggs wrote: >> >> Hi Robin, >> >> It looks like the status argument to BeforeHalt is discarded in JVM_BeforeHalt >> and is not inserted into the event. >> That suggests it should be removed all the way back to Shutdown.beforeHalt. > > You are right, my thinking was that the interface wouldn?t need to be changed if we decided to revisit the event in the future and add the status code. But that is perhaps unlikely, so can certainly remove the argument for now. > > Best regards, > Robin > >> >> Roger >> >> >> >> On 2/13/2018 9:59 AM, Robin Westberg wrote: >>> Hi Alan, >>> >>>> On 12 Feb 2018, at 09:02, Alan Bateman wrote: >>>> >>>> >>>> >>>> On 12/02/2018 07:07, David Holmes wrote: >>>>>> Okay, I?ve removed the code related to the status field, certainly makes the change a bit less intrusive. >>>>>> >>>>>> Updated webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.01/ >>>>>> Incremental: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00-01/ >>>>> This looks much cleaner/neater to me - thanks. >>>>> >>>> The updates to Runtime/Shutdown seems okay. >>> Thanks for reviewing! >>> >>> Best regards, >>> Robin >>> >>>> -Alan >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stewartd.qdt at qualcommdatacenter.com Thu Feb 15 14:29:51 2018 From: stewartd.qdt at qualcommdatacenter.com (stewartd.qdt) Date: Thu, 15 Feb 2018 14:29:51 +0000 Subject: RFR: JDK-8196930 - [Testbug] serviceability/sa/ClhsdbFindPC.java fails to find expected output In-Reply-To: <97e60981-2f2b-493c-b752-9c30bf0c7e8e@default> References: <527fbc12-4ff6-3946-346e-2a46903a3c30@oracle.com> <15041369-ba2a-4efb-aba8-b8ebcb1363b9@default> <544646a2-2ba2-a67c-ffeb-dd42c35c3b13@oracle.com> <1f5d25b298fa4ec28f4f194dc6a18f6b@NASANEXM01E.na.qualcomm.com> <97e60981-2f2b-493c-b752-9c30bf0c7e8e@default> Message-ID: <33204e7e1eb54291a8223d7e613b1b94@NASANEXM01E.na.qualcomm.com> Thanks Sharath. I did realize this patch was solving another issue, but just wanted to make you aware that the null Method issue pops up in this test as well. Please let me know if there is something I can do to help you solve this issue. I don't know if you are able to reproduce the issue or not, so if you need any logs or other info, I'm happy to get it for you. Daniel -----Original Message----- From: Sharath Ballal [mailto:sharath.ballal at oracle.com] Sent: Wednesday, February 14, 2018 10:49 PM To: stewartd.qdt ; David Holmes ; serviceability-dev Subject: RE: RFR: JDK-8196930 - [Testbug] serviceability/sa/ClhsdbFindPC.java fails to find expected output Daniel, This is a different problem than getting the null method. Here we are expecting "invoke return entry points" in the output of findpc command but getting ""method entry point". The null method problem seems to be specific to AArch64 system and I am looking at it separately. Thanks, Sharath -----Original Message----- From: stewartd.qdt [mailto:stewartd.qdt at qualcommdatacenter.com] Sent: Thursday, February 15, 2018 12:05 AM To: David Holmes; Sharath Ballal; serviceability-dev Subject: RE: RFR: JDK-8196930 - [Testbug] serviceability/sa/ClhsdbFindPC.java fails to find expected output This test actually fails for me for the same reason that ClhsdbJstack.java fails, there is a frame in the stack of the main method that is returning null for the method. See the OpenJDK bug at https://bugs.openjdk.java.net/browse/JDK-8196969 for details of that issue. In this case I get the same problem, that a Method is null in the stack, which then causes a failure at 73 of ClhsdbFindPC.java. Daniel -----Original Message----- From: serviceability-dev [mailto:serviceability-dev-bounces at openjdk.java.net] On Behalf Of David Holmes Sent: Wednesday, February 14, 2018 12:40 AM To: Sharath Ballal ; serviceability-dev Subject: Re: RFR: JDK-8196930 - [Testbug] serviceability/sa/ClhsdbFindPC.java fails to find expected output On 14/02/2018 3:29 PM, Sharath Ballal wrote: > -----Original Message----- > From: David Holmes > Sent: Wednesday, February 14, 2018 10:53 AM > To: Sharath Ballal; serviceability-dev > Subject: Re: RFR: JDK-8196930 - [Testbug] > serviceability/sa/ClhsdbFindPC.java fails to find expected output > > On 14/02/2018 3:11 PM, Sharath Ballal wrote: >> David, >> When I wrote these tests, I looked for the output generated by these commands. The output I saw was same (across windows, mac, linux) and hence grepped for certain strings from it. >> Now we are seeing some issues where depending on the hardware we are at different execution point and hence some of the strings are changing. So now I am removing such strings from the test. >> The main intention of these tests are to check the basic sanity that the SA command is working. > >> Ok. But based on "In interpreter codelet" I would expect this would fail if run in -Xcomp mode ?? > This test works in both Xcomp and Xint mode. "In interpreter codelet" is being checked for Xint, for Xcomp we are checking "In code in NMethod for jdk/test/lib/apps/LingeredApp.main" Yep sorry - missed that bit. David > > Thanks, > David > >> Thanks, >> Sharath >> >> >> -----Original Message----- >> From: David Holmes >> Sent: Wednesday, February 14, 2018 7:38 AM >> To: Sharath Ballal; serviceability-dev >> Subject: Re: RFR: JDK-8196930 - [Testbug] >> serviceability/sa/ClhsdbFindPC.java fails to find expected output >> >> Hi Sharath, >> >> On 12/02/2018 7:12 PM, Sharath Ballal wrote: >>> Hello, >>> >>> Requesting reviews for the issue: >>> >>> ID: https://bugs.openjdk.java.net/browse/JDK-8196930 >>> >>> Webrev: http://cr.openjdk.java.net/~sballal/8196930/webrev.00/ >> >> I've no doubt this makes the test pass, but I always have to wonder with these tests: why was it looking for the thing now removed? what changed (or in what way was the expected output invalid) to make the test fail? >> >> Thanks, >> David >> >>> Tests run: The SA tests pass with Mach5. >>> >>> Thanks, >>> >>> Sharath >>> From Roger.Riggs at Oracle.com Thu Feb 15 16:00:57 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 15 Feb 2018 11:00:57 -0500 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: <8CDB2E80-3841-4A02-ADA5-45BB842FEB82@oracle.com> References: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> <5622d8af-d0d6-e77a-f7c1-f2c474f89763@oracle.com> <3EDE8C81-4FBD-4D40-A798-5F71CF7DD2B8@oracle.com> <0362f486-b446-4023-9914-0469d6274b7a@oracle.com> <4730283A-A1B1-46C4-9E04-C3993B62E248@oracle.com> <29aea368-9908-72d9-9dae-107a15a63043@Oracle.com> <0F81FDA5-443F-4E5D-9AA0-895AB48BEA73@oracle.com> <8CDB2E80-3841-4A02-ADA5-45BB842FEB82@oracle.com> Message-ID: <07ea69ac-7b62-992f-5275-97c990906a2b@Oracle.com> Hi Robin, Looks fine to me. (How is this tested?, Normal, exceptional, etc.) Thanks, Roger On 2/15/2018 8:35 AM, Robin Westberg wrote: > Hi again, > > Here?s the (hopefully) final version: > > Full: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.02/ > > Incremental: > http://cr.openjdk.java.net/~rwestberg/8041626/webrev.01-02/ > > > Best regards, > Robin > >> On 14 Feb 2018, at 16:15, Robin Westberg > > wrote: >> >> Hi Roger, >> >>> On 13 Feb 2018, at 16:17, Roger Riggs >> > wrote: >>> >>> Hi Robin, >>> >>> It looks like the status argument to BeforeHalt is discarded in >>> JVM_BeforeHalt >>> and is not inserted into the event. >>> That suggests it should be removed all the way back to >>> Shutdown.beforeHalt. >> >> You are right, my thinking was that the interface wouldn?t need to be >> changed if we decided to revisit the event in the future and add the >> status code. But that is perhaps unlikely, so can certainly remove >> the argument for now. >> >> Best regards, >> Robin >> >>> >>> Roger >>> >>> >>> >>> On 2/13/2018 9:59 AM, Robin Westberg wrote: >>>> Hi Alan, >>>> >>>>> On 12 Feb 2018, at 09:02, Alan Bateman >>>> > wrote: >>>>> >>>>> >>>>> >>>>> On 12/02/2018 07:07, David Holmes wrote: >>>>>>> Okay, I?ve removed the code related to the status field, >>>>>>> certainly makes the change a bit less intrusive. >>>>>>> >>>>>>> Updated webrev: >>>>>>> http://cr.openjdk.java.net/~rwestberg/8041626/webrev.01/ >>>>>>> >>>>>>> Incremental: >>>>>>> http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00-01/ >>>>>>> >>>>>> This looks much cleaner/neater to me - thanks. >>>>>> >>>>> The updates to Runtime/Shutdown seems okay. >>>> Thanks for reviewing! >>>> >>>> Best regards, >>>> Robin >>>> >>>>> -Alan >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Thu Feb 15 17:17:28 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 15 Feb 2018 09:17:28 -0800 Subject: RFR(S): 8196909: gcc 7.2.1 compiler warning in libdt_socket In-Reply-To: <7d53497f82a84536abd2726fd99dcbc9@sap.com> References: <7d53497f82a84536abd2726fd99dcbc9@sap.com> Message-ID: Thanks Christoph and Serguei. Chris On 2/14/18 11:47 PM, Langer, Christoph wrote: > Hi Chris, > > looks good. > > Best regards > Christoph > >> -----Original Message----- >> From: serviceability-dev [mailto:serviceability-dev- >> bounces at openjdk.java.net] On Behalf Of Chris Plummer >> Sent: Mittwoch, 14. Februar 2018 23:31 >> To: serviceability-dev >> Cc: Indu Bhagat >> Subject: RFR(S): 8196909: gcc 7.2.1 compiler warning in libdt_socket >> >> Hello, >> >> Please review the following simple fix for a compiler warning: >> >> https://bugs.openjdk.java.net/browse/JDK-8196909 >> >> diff --git >> a/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c >> b/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c >> --- a/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c >> +++ b/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c >> @@ -398,7 +398,7 @@ >> ??????? return instr; >> ???? } >> >> -??? *mask = htonl(-1 << (32 - m)); >> +??? *mask = htonl((uint32_t)(~0) << (32 - m)); >> ???? return s; >> ?} >> >> >> Tested with all open and closed jdb, jdwp, and jdi test suites on all >> supported platforms. >> >> thanks, >> >> Chris From mandy.chung at oracle.com Thu Feb 15 19:10:34 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 15 Feb 2018 11:10:34 -0800 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: <8CDB2E80-3841-4A02-ADA5-45BB842FEB82@oracle.com> References: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> <5622d8af-d0d6-e77a-f7c1-f2c474f89763@oracle.com> <3EDE8C81-4FBD-4D40-A798-5F71CF7DD2B8@oracle.com> <0362f486-b446-4023-9914-0469d6274b7a@oracle.com> <4730283A-A1B1-46C4-9E04-C3993B62E248@oracle.com> <29aea368-9908-72d9-9dae-107a15a63043@Oracle.com> <0F81FDA5-443F-4E5D-9AA0-895AB48BEA73@oracle.com> <8CDB2E80-3841-4A02-ADA5-45BB842FEB82@oracle.com> Message-ID: Hi Robin, Do you want a shutdown event for every call to Runtime::exit regardless where it is in the shutdown sequence?? or do you expect to get an event of the first thread calling JVM_Halt?? or the first thread starting the shutdown sequence but it may not be the thread halting? Mandy On 2/15/18 5:35 AM, Robin Westberg wrote: > Hi again, > > Here?s the (hopefully) final version: > > Full: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.02/ > > Incremental: > http://cr.openjdk.java.net/~rwestberg/8041626/webrev.01-02/ > > > Best regards, > Robin -------------- next part -------------- An HTML attachment was scrubbed... URL: From parvathi.somashekar at oracle.com Thu Feb 15 23:22:53 2018 From: parvathi.somashekar at oracle.com (Paru Somashekar) Date: Thu, 15 Feb 2018 15:22:53 -0800 Subject: RFR 4916621: update those still using JDIScaffold to use TestScaffold instead Message-ID: <5A86164D.1080805@oracle.com> Hi, Please review fix for JDK-4916621 These are the remaining tests from the list that needed to be updated. Bug : JDK-4916621 Webrev : http://cr.openjdk.java.net/~psomashe/4916621/webrev/ The updated tests ran successfully with Mach5. thanks, Paru. -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Thu Feb 15 23:41:40 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 15 Feb 2018 15:41:40 -0800 Subject: RFR 4916621: update those still using JDIScaffold to use TestScaffold instead In-Reply-To: <5A86164D.1080805@oracle.com> References: <5A86164D.1080805@oracle.com> Message-ID: <3ffe8dd1-66de-5443-2f2c-4981c7891d3a@oracle.com> An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Thu Feb 15 23:46:07 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 15 Feb 2018 15:46:07 -0800 Subject: RFR 4916621: update those still using JDIScaffold to use TestScaffold instead In-Reply-To: <5A86164D.1080805@oracle.com> References: <5A86164D.1080805@oracle.com> Message-ID: <7e900af5-2ca2-4095-75ad-9a4cb6262dd1@oracle.com> An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Fri Feb 16 00:22:13 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 15 Feb 2018 16:22:13 -0800 Subject: RFR 4916621: update those still using JDIScaffold to use TestScaffold instead In-Reply-To: <7e900af5-2ca2-4095-75ad-9a4cb6262dd1@oracle.com> References: <5A86164D.1080805@oracle.com> <7e900af5-2ca2-4095-75ad-9a4cb6262dd1@oracle.com> Message-ID: <2460b9ca-3f07-6ab3-3024-1e1ffd8cb6cf@oracle.com> An HTML attachment was scrubbed... URL: From jeremymanson at google.com Fri Feb 16 00:46:56 2018 From: jeremymanson at google.com (Jeremy Manson) Date: Thu, 15 Feb 2018 16:46:56 -0800 Subject: RFR JDK-8198253: ThreadInfo.from(CompositeData) assigning fields incorrectly in JDK 9 Message-ID: Hi folks, Been a long time! I'd like someone to do a code review. This is in code I wrote a few years ago, and got wrong. At the time, David Holmes, Staffan Larsen, and Mandy Chung reviewed it. It does mean that people using ThreadInfo.from(CompositeData) may be getting the wrong values out for ThreadInfo, so it is definitely worth fixing. The patch below fixes the bug, but is a pretty questionable approach. Better approaches happily considered. Patch: http://cr.openjdk.java.net/~jmanson/8198253/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8198253 Thanks! Jeremy -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeremymanson at google.com Fri Feb 16 00:57:01 2018 From: jeremymanson at google.com (Jeremy Manson) Date: Thu, 15 Feb 2018 16:57:01 -0800 Subject: RFR JDK-8198253: ThreadInfo.from(CompositeData) assigning fields incorrectly in JDK 9 In-Reply-To: References: Message-ID: (I should note that it is also broken at head, not just in JDK 9.) On Thu, Feb 15, 2018 at 4:46 PM, Jeremy Manson wrote: > Hi folks, > > Been a long time! I'd like someone to do a code review. This is in code > I wrote a few years ago, and got wrong. At the time, David Holmes, Staffan > Larsen, and Mandy Chung reviewed it. It does mean that people using > ThreadInfo.from(CompositeData) may be getting the wrong values out for > ThreadInfo, so it is definitely worth fixing. > > The patch below fixes the bug, but is a pretty questionable approach. > Better approaches happily considered. > > Patch: > http://cr.openjdk.java.net/~jmanson/8198253/webrev.00/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8198253 > > Thanks! > > Jeremy > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Fri Feb 16 01:02:27 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 16 Feb 2018 11:02:27 +1000 Subject: RFR JDK-8198253: ThreadInfo.from(CompositeData) assigning fields incorrectly in JDK 9 In-Reply-To: References: Message-ID: <36fbddd0-b77b-4a3e-4a3f-8abc40c45bbc@oracle.com> Hi Jeremy, On 16/02/2018 10:46 AM, Jeremy Manson wrote: > Hi folks, > > Been a long time!? I'd like someone to do a code review.? This is in > code I wrote a few years ago, and got wrong.? At the time, David Holmes, > Staffan Larsen, and Mandy Chung reviewed it.? It does mean that people Was there no bug back then? Do you have a link to the review thread? I don't recall this one. :) Thanks, David > using ThreadInfo.from(CompositeData) may be getting the wrong values out > for ThreadInfo, so it is definitely worth fixing. > > The patch below fixes the bug, but is a pretty questionable approach. > Better approaches happily considered. > > Patch: > http://cr.openjdk.java.net/~jmanson/8198253/webrev.00/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8198253 > > Thanks! > > Jeremy From jeremymanson at google.com Fri Feb 16 01:12:08 2018 From: jeremymanson at google.com (Jeremy Manson) Date: Thu, 15 Feb 2018 17:12:08 -0800 Subject: RFR JDK-8198253: ThreadInfo.from(CompositeData) assigning fields incorrectly in JDK 9 In-Reply-To: <36fbddd0-b77b-4a3e-4a3f-8abc40c45bbc@oracle.com> References: <36fbddd0-b77b-4a3e-4a3f-8abc40c45bbc@oracle.com> Message-ID: Previous bug: https://bugs.openjdk.java.net/browse/JDK-6588467 And the review thread: http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-January/016356.html I don't think the bug would have been obvious to a reviewer (or, indeed, the author of the patch!), because we would have had to think about how ticd.isCurrentVersion worked, and noticed the fact that some of the fields are optional. Jeremy On Thu, Feb 15, 2018 at 5:02 PM, David Holmes wrote: > Hi Jeremy, > > On 16/02/2018 10:46 AM, Jeremy Manson wrote: > >> Hi folks, >> >> Been a long time! I'd like someone to do a code review. This is in code >> I wrote a few years ago, and got wrong. At the time, David Holmes, Staffan >> Larsen, and Mandy Chung reviewed it. It does mean that people >> > > Was there no bug back then? Do you have a link to the review thread? I > don't recall this one. :) > > Thanks, > David > > > using ThreadInfo.from(CompositeData) may be getting the wrong values out >> for ThreadInfo, so it is definitely worth fixing. >> >> The patch below fixes the bug, but is a pretty questionable approach. >> Better approaches happily considered. >> >> Patch: >> http://cr.openjdk.java.net/~jmanson/8198253/webrev.00/ >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8198253 >> >> Thanks! >> >> Jeremy >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeremymanson at google.com Fri Feb 16 01:15:00 2018 From: jeremymanson at google.com (Jeremy Manson) Date: Thu, 15 Feb 2018 17:15:00 -0800 Subject: RFR JDK-8198253: ThreadInfo.from(CompositeData) assigning fields incorrectly in JDK 9 In-Reply-To: References: <36fbddd0-b77b-4a3e-4a3f-8abc40c45bbc@oracle.com> Message-ID: I should also say that I found this because I was tinkering with a fix for JDK-8154176. If there is likely to be a lot of pushback on a fix for that - it will require an API change - it would be good to know. Jeremy On Thu, Feb 15, 2018 at 5:12 PM, Jeremy Manson wrote: > Previous bug: > > https://bugs.openjdk.java.net/browse/JDK-6588467 > > And the review thread: > > http://mail.openjdk.java.net/pipermail/serviceability-dev/ > 2015-January/016356.html > > I don't think the bug would have been obvious to a reviewer (or, indeed, > the author of the patch!), because we would have had to think about how > ticd.isCurrentVersion worked, and noticed the fact that some of the fields > are optional. > > Jeremy > > On Thu, Feb 15, 2018 at 5:02 PM, David Holmes > wrote: > >> Hi Jeremy, >> >> On 16/02/2018 10:46 AM, Jeremy Manson wrote: >> >>> Hi folks, >>> >>> Been a long time! I'd like someone to do a code review. This is in >>> code I wrote a few years ago, and got wrong. At the time, David Holmes, >>> Staffan Larsen, and Mandy Chung reviewed it. It does mean that people >>> >> >> Was there no bug back then? Do you have a link to the review thread? I >> don't recall this one. :) >> >> Thanks, >> David >> >> >> using ThreadInfo.from(CompositeData) may be getting the wrong values out >>> for ThreadInfo, so it is definitely worth fixing. >>> >>> The patch below fixes the bug, but is a pretty questionable approach. >>> Better approaches happily considered. >>> >>> Patch: >>> http://cr.openjdk.java.net/~jmanson/8198253/webrev.00/ >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8198253 >>> >>> Thanks! >>> >>> Jeremy >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Fri Feb 16 02:40:50 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 16 Feb 2018 12:40:50 +1000 Subject: RFR JDK-8198253: ThreadInfo.from(CompositeData) assigning fields incorrectly in JDK 9 In-Reply-To: References: <36fbddd0-b77b-4a3e-4a3f-8abc40c45bbc@oracle.com> Message-ID: <26858050-a49f-b8dc-a28d-790d588a90be@oracle.com> On 16/02/2018 11:12 AM, Jeremy Manson wrote: > Previous bug: > > https://bugs.openjdk.java.net/browse/JDK-6588467 > > And the review thread: > > http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-January/016356.html > > I don't think the bug would have been obvious to a reviewer (or, indeed, > the author of the patch!), because we would have had to think about how > ticd.isCurrentVersion worked, and noticed the fact that some of the > fields are optional. I misunderstood the connection to the old bug and review. This is a pre-existing issue that wasn't noticed last time this code was updated - right? Thanks, David > Jeremy > > On Thu, Feb 15, 2018 at 5:02 PM, David Holmes > wrote: > > Hi Jeremy, > > On 16/02/2018 10:46 AM, Jeremy Manson wrote: > > Hi folks, > > Been a long time!? I'd like someone to do a code review.? This > is in code I wrote a few years ago, and got wrong.? At the time, > David Holmes, Staffan Larsen, and Mandy Chung reviewed it.? It > does mean that people > > > Was there no bug back then? Do you have a link to the review thread? > I don't recall this one. :) > > Thanks, > David > > > using ThreadInfo.from(CompositeData) may be getting the wrong > values out for ThreadInfo, so it is definitely worth fixing. > > The patch below fixes the bug, but is a pretty questionable > approach.? Better approaches happily considered. > > Patch: > http://cr.openjdk.java.net/~jmanson/8198253/webrev.00/ > > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8198253 > > > Thanks! > > Jeremy > > From david.holmes at oracle.com Fri Feb 16 02:44:13 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 16 Feb 2018 12:44:13 +1000 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: <8CDB2E80-3841-4A02-ADA5-45BB842FEB82@oracle.com> References: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> <5622d8af-d0d6-e77a-f7c1-f2c474f89763@oracle.com> <3EDE8C81-4FBD-4D40-A798-5F71CF7DD2B8@oracle.com> <0362f486-b446-4023-9914-0469d6274b7a@oracle.com> <4730283A-A1B1-46C4-9E04-C3993B62E248@oracle.com> <29aea368-9908-72d9-9dae-107a15a63043@Oracle.com> <0F81FDA5-443F-4E5D-9AA0-895AB48BEA73@oracle.com> <8CDB2E80-3841-4A02-ADA5-45BB842FEB82@oracle.com> Message-ID: Looks fine to me. Thanks, David ----- On 15/02/2018 11:35 PM, Robin Westberg wrote: > Hi again, > > Here?s the (hopefully) final version: > > Full: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.02/ > Incremental: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.01-02/ > > Best regards, > Robin > >> On 14 Feb 2018, at 16:15, Robin Westberg wrote: >> >> Hi Roger, >> >>> On 13 Feb 2018, at 16:17, Roger Riggs wrote: >>> >>> Hi Robin, >>> >>> It looks like the status argument to BeforeHalt is discarded in JVM_BeforeHalt >>> and is not inserted into the event. >>> That suggests it should be removed all the way back to Shutdown.beforeHalt. >> >> You are right, my thinking was that the interface wouldn?t need to be changed if we decided to revisit the event in the future and add the status code. But that is perhaps unlikely, so can certainly remove the argument for now. >> >> Best regards, >> Robin >> >>> >>> Roger >>> >>> >>> >>> On 2/13/2018 9:59 AM, Robin Westberg wrote: >>>> Hi Alan, >>>> >>>>> On 12 Feb 2018, at 09:02, Alan Bateman wrote: >>>>> >>>>> >>>>> >>>>> On 12/02/2018 07:07, David Holmes wrote: >>>>>>> Okay, I?ve removed the code related to the status field, certainly makes the change a bit less intrusive. >>>>>>> >>>>>>> Updated webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.01/ >>>>>>> Incremental: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00-01/ >>>>>> This looks much cleaner/neater to me - thanks. >>>>>> >>>>> The updates to Runtime/Shutdown seems okay. >>>> Thanks for reviewing! >>>> >>>> Best regards, >>>> Robin >>>> >>>>> -Alan >>> >> > From sharath.ballal at oracle.com Fri Feb 16 04:23:11 2018 From: sharath.ballal at oracle.com (Sharath Ballal) Date: Thu, 15 Feb 2018 20:23:11 -0800 (PST) Subject: RFR: JDK-8196930 - [Testbug] serviceability/sa/ClhsdbFindPC.java fails to find expected output In-Reply-To: <33204e7e1eb54291a8223d7e613b1b94@NASANEXM01E.na.qualcomm.com> References: <527fbc12-4ff6-3946-346e-2a46903a3c30@oracle.com> <15041369-ba2a-4efb-aba8-b8ebcb1363b9@default> <544646a2-2ba2-a67c-ffeb-dd42c35c3b13@oracle.com> <1f5d25b298fa4ec28f4f194dc6a18f6b@NASANEXM01E.na.qualcomm.com> <97e60981-2f2b-493c-b752-9c30bf0c7e8e@default> <33204e7e1eb54291a8223d7e613b1b94@NASANEXM01E.na.qualcomm.com> Message-ID: Thanks Daniel. I am trying to setup a aarch64 machine and recreate the issue, I will let you know if I need anything. Thanks, Sharath -----Original Message----- From: stewartd.qdt [mailto:stewartd.qdt at qualcommdatacenter.com] Sent: Thursday, February 15, 2018 8:00 PM To: Sharath Ballal; stewartd.qdt; David Holmes; serviceability-dev Subject: RE: RFR: JDK-8196930 - [Testbug] serviceability/sa/ClhsdbFindPC.java fails to find expected output Thanks Sharath. I did realize this patch was solving another issue, but just wanted to make you aware that the null Method issue pops up in this test as well. Please let me know if there is something I can do to help you solve this issue. I don't know if you are able to reproduce the issue or not, so if you need any logs or other info, I'm happy to get it for you. Daniel -----Original Message----- From: Sharath Ballal [mailto:sharath.ballal at oracle.com] Sent: Wednesday, February 14, 2018 10:49 PM To: stewartd.qdt ; David Holmes ; serviceability-dev Subject: RE: RFR: JDK-8196930 - [Testbug] serviceability/sa/ClhsdbFindPC.java fails to find expected output Daniel, This is a different problem than getting the null method. Here we are expecting "invoke return entry points" in the output of findpc command but getting ""method entry point". The null method problem seems to be specific to AArch64 system and I am looking at it separately. Thanks, Sharath -----Original Message----- From: stewartd.qdt [mailto:stewartd.qdt at qualcommdatacenter.com] Sent: Thursday, February 15, 2018 12:05 AM To: David Holmes; Sharath Ballal; serviceability-dev Subject: RE: RFR: JDK-8196930 - [Testbug] serviceability/sa/ClhsdbFindPC.java fails to find expected output This test actually fails for me for the same reason that ClhsdbJstack.java fails, there is a frame in the stack of the main method that is returning null for the method. See the OpenJDK bug at https://bugs.openjdk.java.net/browse/JDK-8196969 for details of that issue. In this case I get the same problem, that a Method is null in the stack, which then causes a failure at 73 of ClhsdbFindPC.java. Daniel -----Original Message----- From: serviceability-dev [mailto:serviceability-dev-bounces at openjdk.java.net] On Behalf Of David Holmes Sent: Wednesday, February 14, 2018 12:40 AM To: Sharath Ballal ; serviceability-dev Subject: Re: RFR: JDK-8196930 - [Testbug] serviceability/sa/ClhsdbFindPC.java fails to find expected output On 14/02/2018 3:29 PM, Sharath Ballal wrote: > -----Original Message----- > From: David Holmes > Sent: Wednesday, February 14, 2018 10:53 AM > To: Sharath Ballal; serviceability-dev > Subject: Re: RFR: JDK-8196930 - [Testbug] > serviceability/sa/ClhsdbFindPC.java fails to find expected output > > On 14/02/2018 3:11 PM, Sharath Ballal wrote: >> David, >> When I wrote these tests, I looked for the output generated by these commands. The output I saw was same (across windows, mac, linux) and hence grepped for certain strings from it. >> Now we are seeing some issues where depending on the hardware we are at different execution point and hence some of the strings are changing. So now I am removing such strings from the test. >> The main intention of these tests are to check the basic sanity that the SA command is working. > >> Ok. But based on "In interpreter codelet" I would expect this would fail if run in -Xcomp mode ?? > This test works in both Xcomp and Xint mode. "In interpreter codelet" is being checked for Xint, for Xcomp we are checking "In code in NMethod for jdk/test/lib/apps/LingeredApp.main" Yep sorry - missed that bit. David > > Thanks, > David > >> Thanks, >> Sharath >> >> >> -----Original Message----- >> From: David Holmes >> Sent: Wednesday, February 14, 2018 7:38 AM >> To: Sharath Ballal; serviceability-dev >> Subject: Re: RFR: JDK-8196930 - [Testbug] >> serviceability/sa/ClhsdbFindPC.java fails to find expected output >> >> Hi Sharath, >> >> On 12/02/2018 7:12 PM, Sharath Ballal wrote: >>> Hello, >>> >>> Requesting reviews for the issue: >>> >>> ID: https://bugs.openjdk.java.net/browse/JDK-8196930 >>> >>> Webrev: http://cr.openjdk.java.net/~sballal/8196930/webrev.00/ >> >> I've no doubt this makes the test pass, but I always have to wonder with these tests: why was it looking for the thing now removed? what changed (or in what way was the expected output invalid) to make the test fail? >> >> Thanks, >> David >> >>> Tests run: The SA tests pass with Mach5. >>> >>> Thanks, >>> >>> Sharath >>> From jeremymanson at google.com Fri Feb 16 05:01:01 2018 From: jeremymanson at google.com (Jeremy Manson) Date: Thu, 15 Feb 2018 21:01:01 -0800 Subject: RFR JDK-8198253: ThreadInfo.from(CompositeData) assigning fields incorrectly in JDK 9 In-Reply-To: <26858050-a49f-b8dc-a28d-790d588a90be@oracle.com> References: <36fbddd0-b77b-4a3e-4a3f-8abc40c45bbc@oracle.com> <26858050-a49f-b8dc-a28d-790d588a90be@oracle.com> Message-ID: On Thu, Feb 15, 2018 at 6:40 PM, David Holmes wrote: > On 16/02/2018 11:12 AM, Jeremy Manson wrote: > >> Previous bug: >> >> https://bugs.openjdk.java.net/browse/JDK-6588467 >> >> And the review thread: >> >> http://mail.openjdk.java.net/pipermail/serviceability-dev/20 >> 15-January/016356.html >> >> I don't think the bug would have been obvious to a reviewer (or, indeed, >> the author of the patch!), because we would have had to think about how >> ticd.isCurrentVersion worked, and noticed the fact that some of the fields >> are optional. >> > > I misunderstood the connection to the old bug and review. This is a > pre-existing issue that wasn't noticed last time this code was updated - > right? > Sort of. The uses of isCurrentVersion() didn't happen to need to exclude optional values before I made my changes. It falls under the category of unstated assumptions. The method could have been called "isCurrentVersionAndDoesntIgnoreOptionalValues". :) Jeremy -------------- next part -------------- An HTML attachment was scrubbed... URL: From parvathi.somashekar at oracle.com Fri Feb 16 07:11:34 2018 From: parvathi.somashekar at oracle.com (Paru Somashekar) Date: Thu, 15 Feb 2018 23:11:34 -0800 Subject: RFR 4916621: update those still using JDIScaffold to use TestScaffold instead In-Reply-To: <3ffe8dd1-66de-5443-2f2c-4981c7891d3a@oracle.com> References: <5A86164D.1080805@oracle.com> <3ffe8dd1-66de-5443-2f2c-4981c7891d3a@oracle.com> Message-ID: <5A868426.30608@oracle.com> Thanks for the review Serguei. thanks, Paru. On 2/15/18, 3:41 PM, serguei.spitsyn at oracle.com wrote: > Hi Paru, > > This looks good. > We need one more reviewer. > > Thanks, > Serguei > > > On 2/15/18 15:22, Paru Somashekar wrote: >> Hi, >> >> Please review fix for JDK-4916621 >> >> These are the remaining tests from the list that needed to be updated. >> >> Bug : JDK-4916621 >> Webrev : http://cr.openjdk.java.net/~psomashe/4916621/webrev/ >> >> >> The updated tests ran successfully with Mach5. >> >> thanks, >> Paru. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From parvathi.somashekar at oracle.com Fri Feb 16 07:25:13 2018 From: parvathi.somashekar at oracle.com (Paru Somashekar) Date: Thu, 15 Feb 2018 23:25:13 -0800 Subject: RFR 4916621: update those still using JDIScaffold to use TestScaffold instead In-Reply-To: <2460b9ca-3f07-6ab3-3024-1e1ffd8cb6cf@oracle.com> References: <5A86164D.1080805@oracle.com> <7e900af5-2ca2-4095-75ad-9a4cb6262dd1@oracle.com> <2460b9ca-3f07-6ab3-3024-1e1ffd8cb6cf@oracle.com> Message-ID: <5A868759.3070204@oracle.com> Hi Chris, Thanks for the review. I have updated the patch that has the change you suggested i.e. we just need startTo("TestVars", "hi", "()V"); in the runTests method and the code to save the value of args is not needed at all. Also, I realized that there was one more class in the list ( ModificationWatchpoints.java) which needed an update to its @run build JDIScaffold comment to the class. So I included that change as well in the review. Note : this class has already been updated to extend TestScaffold. http://cr.openjdk.java.net/~psomashe/4916621/webrev .01 thanks, Paru. On 2/15/18, 4:22 PM, Chris Plummer wrote: > Hi Paru, > > I think instead of moving the following code, which did to avoid > having to save the value of args, can instead replaced with a call to > TestScaffold.startup("TestVars"): > > 147 List argList = new ArrayList(Arrays.asList(args)); > 148 argList.add("TestVars"); > 149 System.out.println("run args: " + argList); > 150 connect((String[])argList.toArray(args)); > 151 waitForVMStart(); > > Other than that it looks good. > > thanks, > > Chris > > On 2/15/18 3:46 PM, serguei.spitsyn at oracle.com > wrote: >> Hi Paru, >> >> This looks good. >> We need one more reviewer. >> >> Thanks, >> Serguei >> >> >> On 2/15/18 15:22, Paru Somashekar wrote: >>> Hi, >>> >>> Please review fix for JDK-4916621 >>> >>> These are the remaining tests from the list that needed to be updated. >>> >>> Bug : JDK-4916621 >>> Webrev : http://cr.openjdk.java.net/~psomashe/4916621/webrev/ >>> >>> >>> The updated tests ran successfully with Mach5. >>> >>> thanks, >>> Paru. >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Fri Feb 16 07:28:52 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 15 Feb 2018 23:28:52 -0800 Subject: RFR 4916621: update those still using JDIScaffold to use TestScaffold instead In-Reply-To: <5A868759.3070204@oracle.com> References: <5A86164D.1080805@oracle.com> <7e900af5-2ca2-4095-75ad-9a4cb6262dd1@oracle.com> <2460b9ca-3f07-6ab3-3024-1e1ffd8cb6cf@oracle.com> <5A868759.3070204@oracle.com> Message-ID: <930d7afb-cc4d-a980-8104-560e9c445564@oracle.com> An HTML attachment was scrubbed... URL: From daniel.fuchs at oracle.com Fri Feb 16 10:51:55 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 16 Feb 2018 10:51:55 +0000 Subject: RFR JDK-8198253: ThreadInfo.from(CompositeData) assigning fields incorrectly in JDK 9 In-Reply-To: References: <36fbddd0-b77b-4a3e-4a3f-8abc40c45bbc@oracle.com> <26858050-a49f-b8dc-a28d-790d588a90be@oracle.com> Message-ID: <9830e444-2d74-8376-c78e-76e57d450991@oracle.com> Hi Jeremy, I'm not sure I understand exactly what is going on there. I wonder if this has anything to do with an issue I noticed some time back when the jvisualvm for JDK 8 and JDK 9 were still out of sync: https://bugs.openjdk.java.net/browse/JDK-8167099 I stumbled on that because at the time I was hacking my developer's build of JDK 9 and trying to make it generate JDK 8 CompositeDatas. But then as far as I could see that bug (JDK-8167099) could not be observed in a regular environment - it was just tickled by my hacks. Now I'm wondering if what you're seeing is more or less related to that issue (it might not). In any case - one possibility if you don't want to create a new CompositeType might be to pass a filtering lambda to isTypeMatched. Something like a BiPredicate that would take a composite type and a field name and tell whether to include it or not in the comparison. best regards, -- daniel On 16/02/2018 05:01, Jeremy Manson wrote: > > > On Thu, Feb 15, 2018 at 6:40 PM, David Holmes > wrote: > > On 16/02/2018 11:12 AM, Jeremy Manson wrote: > > Previous bug: > > https://bugs.openjdk.java.net/browse/JDK-6588467 > > > And the review thread: > > http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-January/016356.html > > > I don't think the bug would have been obvious to a reviewer (or, > indeed, the author of the patch!), because we would have had to > think about how ticd.isCurrentVersion worked, and noticed the > fact that some of the fields are optional. > > > I misunderstood the connection to the old bug and review. This is a > pre-existing issue that wasn't noticed last time this code was > updated - right? > > > Sort of.? The uses of isCurrentVersion() didn't happen to need to > exclude optional values before I made my changes.? It falls under the > category of unstated assumptions.? The method could have been called > "isCurrentVersionAndDoesntIgnoreOptionalValues".? :) > > Jeremy From parvathi.somashekar at oracle.com Fri Feb 16 15:26:51 2018 From: parvathi.somashekar at oracle.com (Paru Somashekar) Date: Fri, 16 Feb 2018 07:26:51 -0800 Subject: RFR 4916621: update those still using JDIScaffold to use TestScaffold instead In-Reply-To: <930d7afb-cc4d-a980-8104-560e9c445564@oracle.com> References: <5A86164D.1080805@oracle.com> <7e900af5-2ca2-4095-75ad-9a4cb6262dd1@oracle.com> <2460b9ca-3f07-6ab3-3024-1e1ffd8cb6cf@oracle.com> <5A868759.3070204@oracle.com> <930d7afb-cc4d-a980-8104-560e9c445564@oracle.com> Message-ID: <5A86F83B.5020100@oracle.com> Thanks Serguei. thanks, Paru. On 2/15/18, 11:28 PM, serguei.spitsyn at oracle.com wrote: > Hi Paru, > > The fix looks better now. > Good catch with the ModificationWatchpoints.hava. > > Thanks, > Serguei > > > On 2/15/18 23:25, Paru Somashekar wrote: >> Hi Chris, >> >> Thanks for the review. I have updated the patch that has the change >> you suggested i.e. we just need startTo("TestVars", "hi", "()V"); in >> the runTests method and the code to save the value of args is not >> needed at all. >> >> Also, I realized that there was one more class in the list ( >> ModificationWatchpoints.java) which needed an update to its @run >> build JDIScaffold comment to the class. So I included that change as >> well in the review. Note : this class has already been updated to >> extend TestScaffold. >> >> http://cr.openjdk.java.net/~psomashe/4916621/webrev >> .01 >> >> >> thanks, >> Paru. >> >> On 2/15/18, 4:22 PM, Chris Plummer wrote: >>> Hi Paru, >>> >>> I think instead of moving the following code, which did to avoid >>> having to save the value of args, can instead replaced with a call >>> to TestScaffold.startup("TestVars"): >>> >>> 147 List argList = new ArrayList(Arrays.asList(args)); >>> 148 argList.add("TestVars"); >>> 149 System.out.println("run args: " + argList); >>> 150 connect((String[])argList.toArray(args)); >>> 151 waitForVMStart(); >>> >>> Other than that it looks good. >>> >>> thanks, >>> >>> Chris >>> >>> On 2/15/18 3:46 PM, serguei.spitsyn at oracle.com >>> wrote: >>>> Hi Paru, >>>> >>>> This looks good. >>>> We need one more reviewer. >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 2/15/18 15:22, Paru Somashekar wrote: >>>>> Hi, >>>>> >>>>> Please review fix for JDK-4916621 >>>>> >>>>> These are the remaining tests from the list that needed to be >>>>> updated. >>>>> >>>>> Bug : JDK-4916621 >>>>> Webrev : http://cr.openjdk.java.net/~psomashe/4916621/webrev/ >>>>> >>>>> >>>>> The updated tests ran successfully with Mach5. >>>>> >>>>> thanks, >>>>> Paru. >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robin.westberg at oracle.com Fri Feb 16 16:00:19 2018 From: robin.westberg at oracle.com (Robin Westberg) Date: Fri, 16 Feb 2018 17:00:19 +0100 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: <07ea69ac-7b62-992f-5275-97c990906a2b@Oracle.com> References: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> <5622d8af-d0d6-e77a-f7c1-f2c474f89763@oracle.com> <3EDE8C81-4FBD-4D40-A798-5F71CF7DD2B8@oracle.com> <0362f486-b446-4023-9914-0469d6274b7a@oracle.com> <4730283A-A1B1-46C4-9E04-C3993B62E248@oracle.com> <29aea368-9908-72d9-9dae-107a15a63043@Oracle.com> <0F81FDA5-443F-4E5D-9AA0-895AB48BEA73@oracle.com> <8CDB2E80-3841-4A02-ADA5-45BB842FEB82@oracle.com> <07ea69ac-7b62-992f-5275-97c990906a2b@Oracle.com> Message-ID: <447A6370-7C4B-46D9-8CAA-40BAE3454819@oracle.com> Hi Roger, > On 15 Feb 2018, at 17:00, Roger Riggs wrote: > > Hi Robin, > > Looks fine to me. Thanks for reviewing! > (How is this tested?, Normal, exceptional, etc.) The tests are part of the (currently closed) JFR tests. Best regards, Robin > > Thanks, Roger > > > On 2/15/2018 8:35 AM, Robin Westberg wrote: >> Hi again, >> >> Here?s the (hopefully) final version: >> >> Full: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.02/ >> Incremental: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.01-02/ >> >> Best regards, >> Robin >> >>> On 14 Feb 2018, at 16:15, Robin Westberg > wrote: >>> >>> Hi Roger, >>> >>>> On 13 Feb 2018, at 16:17, Roger Riggs > wrote: >>>> >>>> Hi Robin, >>>> >>>> It looks like the status argument to BeforeHalt is discarded in JVM_BeforeHalt >>>> and is not inserted into the event. >>>> That suggests it should be removed all the way back to Shutdown.beforeHalt. >>> >>> You are right, my thinking was that the interface wouldn?t need to be changed if we decided to revisit the event in the future and add the status code. But that is perhaps unlikely, so can certainly remove the argument for now. >>> >>> Best regards, >>> Robin >>> >>>> >>>> Roger >>>> >>>> >>>> >>>> On 2/13/2018 9:59 AM, Robin Westberg wrote: >>>>> Hi Alan, >>>>> >>>>>> On 12 Feb 2018, at 09:02, Alan Bateman > wrote: >>>>>> >>>>>> >>>>>> >>>>>> On 12/02/2018 07:07, David Holmes wrote: >>>>>>>> Okay, I?ve removed the code related to the status field, certainly makes the change a bit less intrusive. >>>>>>>> >>>>>>>> Updated webrev: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.01/ >>>>>>>> Incremental: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.00-01/ >>>>>>> This looks much cleaner/neater to me - thanks. >>>>>>> >>>>>> The updates to Runtime/Shutdown seems okay. >>>>> Thanks for reviewing! >>>>> >>>>> Best regards, >>>>> Robin >>>>> >>>>>> -Alan >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robin.westberg at oracle.com Fri Feb 16 16:00:27 2018 From: robin.westberg at oracle.com (Robin Westberg) Date: Fri, 16 Feb 2018 17:00:27 +0100 Subject: RFR: 8041626: [Event Request] Shutdown reason In-Reply-To: References: <02f768aa-1e82-acd9-f203-2157462565ca@oracle.com> <7AFC0F37-DB8A-468D-8F79-D6621CC76C41@oracle.com> <5622d8af-d0d6-e77a-f7c1-f2c474f89763@oracle.com> <3EDE8C81-4FBD-4D40-A798-5F71CF7DD2B8@oracle.com> <0362f486-b446-4023-9914-0469d6274b7a@oracle.com> <4730283A-A1B1-46C4-9E04-C3993B62E248@oracle.com> <29aea368-9908-72d9-9dae-107a15a63043@Oracle.com> <0F81FDA5-443F-4E5D-9AA0-895AB48BEA73@oracle.com> <8CDB2E80-3841-4A02-ADA5-45BB842FEB82@oracle.com> Message-ID: Hi Mandy, > On 15 Feb 2018, at 20:10, mandy chung wrote: > > Hi Robin, > > Do you want a shutdown event for every call to Runtime::exit regardless where it is in the shutdown sequence? or do you expect to get an event of the first thread calling JVM_Halt? or the first thread starting the shutdown sequence but it may not be the thread halting? I was aiming for an event from the first thread starting the shutdown sequence, as I think that?s probably the one you are looking for if you are diagnosing an unexpected shutdown. Any event generated after that point may not make it into the recording as the tracing framework shuts down from a shutdown hook. If finalizers on exit go away, is it still possible that the thread starting the shutdown sequence isn?t the one eventually calling JVM_Halt (in the absence of calls to Runtime.halt)? Best regards, Robin > > Mandy > > On 2/15/18 5:35 AM, Robin Westberg wrote: >> Hi again, >> >> Here?s the (hopefully) final version: >> >> Full: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.02/ >> Incremental: http://cr.openjdk.java.net/~rwestberg/8041626/webrev.01-02/ >> >> Best regards, >> Robin > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Fri Feb 16 18:14:52 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Fri, 16 Feb 2018 10:14:52 -0800 Subject: RFR 4916621: update those still using JDIScaffold to use TestScaffold instead In-Reply-To: <5A868759.3070204@oracle.com> References: <5A86164D.1080805@oracle.com> <7e900af5-2ca2-4095-75ad-9a4cb6262dd1@oracle.com> <2460b9ca-3f07-6ab3-3024-1e1ffd8cb6cf@oracle.com> <5A868759.3070204@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From parvathi.somashekar at oracle.com Fri Feb 16 18:38:15 2018 From: parvathi.somashekar at oracle.com (Paru Somashekar) Date: Fri, 16 Feb 2018 10:38:15 -0800 Subject: RFR 4916621: update those still using JDIScaffold to use TestScaffold instead In-Reply-To: References: <5A86164D.1080805@oracle.com> <7e900af5-2ca2-4095-75ad-9a4cb6262dd1@oracle.com> <2460b9ca-3f07-6ab3-3024-1e1ffd8cb6cf@oracle.com> <5A868759.3070204@oracle.com> Message-ID: <5A872517.2070907@oracle.com> Hi Chris, Firstly, when I moved the code from runTests() method to the main() method of the debugger, I only moved that part where the argument was saved - not the call to connect. Hence connect() was never called twice. Connect() gets called when we do a startTo(..). Hence the saving of the argument did no harm and produced no errors. thanks, Paru. On 2/16/18, 10:14 AM, Chris Plummer wrote: > Hi Paru, > > So it looks like the code I pointed out that you removed was actually > all redundant since it is covered by startTo(). Is that correct? If > so, why did the test not produce any errors? What happens when > connect() is called twice on the same debuggee? > > thanks, > > Chris > > On 2/15/18 11:25 PM, Paru Somashekar wrote: >> Hi Chris, >> >> Thanks for the review. I have updated the patch that has the change >> you suggested i.e. we just need startTo("TestVars", "hi", "()V"); in >> the runTests method and the code to save the value of args is not >> needed at all. >> >> Also, I realized that there was one more class in the list ( >> ModificationWatchpoints.java) which needed an update to its @run >> build JDIScaffold comment to the class. So I included that change as >> well in the review. Note : this class has already been updated to >> extend TestScaffold. >> >> http://cr.openjdk.java.net/~psomashe/4916621/webrev >> .01 >> >> >> thanks, >> Paru. >> >> On 2/15/18, 4:22 PM, Chris Plummer wrote: >>> Hi Paru, >>> >>> I think instead of moving the following code, which did to avoid >>> having to save the value of args, can instead replaced with a call >>> to TestScaffold.startup("TestVars"): >>> >>> 147 List argList = new ArrayList(Arrays.asList(args)); >>> 148 argList.add("TestVars"); >>> 149 System.out.println("run args: " + argList); >>> 150 connect((String[])argList.toArray(args)); >>> 151 waitForVMStart(); >>> >>> Other than that it looks good. >>> >>> thanks, >>> >>> Chris >>> >>> On 2/15/18 3:46 PM, serguei.spitsyn at oracle.com >>> wrote: >>>> Hi Paru, >>>> >>>> This looks good. >>>> We need one more reviewer. >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 2/15/18 15:22, Paru Somashekar wrote: >>>>> Hi, >>>>> >>>>> Please review fix for JDK-4916621 >>>>> >>>>> These are the remaining tests from the list that needed to be >>>>> updated. >>>>> >>>>> Bug : JDK-4916621 >>>>> Webrev : http://cr.openjdk.java.net/~psomashe/4916621/webrev/ >>>>> >>>>> >>>>> The updated tests ran successfully with Mach5. >>>>> >>>>> thanks, >>>>> Paru. >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Fri Feb 16 18:59:05 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Fri, 16 Feb 2018 10:59:05 -0800 Subject: RFR 4916621: update those still using JDIScaffold to use TestScaffold instead In-Reply-To: <5A872517.2070907@oracle.com> References: <5A86164D.1080805@oracle.com> <7e900af5-2ca2-4095-75ad-9a4cb6262dd1@oracle.com> <2460b9ca-3f07-6ab3-3024-1e1ffd8cb6cf@oracle.com> <5A868759.3070204@oracle.com> <5A872517.2070907@oracle.com> Message-ID: <9ef91641-0b8f-c5ca-48a4-ead3d61d4702@oracle.com> An HTML attachment was scrubbed... URL: From parvathi.somashekar at oracle.com Fri Feb 16 19:11:38 2018 From: parvathi.somashekar at oracle.com (Paru Somashekar) Date: Fri, 16 Feb 2018 11:11:38 -0800 Subject: RFR 4916621: update those still using JDIScaffold to use TestScaffold instead In-Reply-To: <9ef91641-0b8f-c5ca-48a4-ead3d61d4702@oracle.com> References: <5A86164D.1080805@oracle.com> <7e900af5-2ca2-4095-75ad-9a4cb6262dd1@oracle.com> <2460b9ca-3f07-6ab3-3024-1e1ffd8cb6cf@oracle.com> <5A868759.3070204@oracle.com> <5A872517.2070907@oracle.com> <9ef91641-0b8f-c5ca-48a4-ead3d61d4702@oracle.com> Message-ID: <5A872CEA.3080400@oracle.com> Thanks Chris, thanks, Paru. On 2/16/18, 10:59 AM, Chris Plummer wrote: > Hi Paru, > > Thanks for the explanation. Your changes look fine. > > Chris > > On 2/16/18 10:38 AM, Paru Somashekar wrote: >> Hi Chris, >> >> Firstly, when I moved the code from runTests() method to the main() >> method of the debugger, I only moved that part where the argument was >> saved - not the call to connect. Hence connect() was never called >> twice. Connect() gets called when we do a startTo(..). Hence the >> saving of the argument did no harm and produced no errors. >> >> thanks, >> Paru. >> >> On 2/16/18, 10:14 AM, Chris Plummer wrote: >>> Hi Paru, >>> >>> So it looks like the code I pointed out that you removed was >>> actually all redundant since it is covered by startTo(). Is that >>> correct? If so, why did the test not produce any errors? What >>> happens when connect() is called twice on the same debuggee? >>> >>> thanks, >>> >>> Chris >>> >>> On 2/15/18 11:25 PM, Paru Somashekar wrote: >>>> Hi Chris, >>>> >>>> Thanks for the review. I have updated the patch that has the change >>>> you suggested i.e. we just need startTo("TestVars", "hi", "()V"); >>>> in the runTests method and the code to save the value of args is >>>> not needed at all. >>>> >>>> Also, I realized that there was one more class in the list ( >>>> ModificationWatchpoints.java) which needed an update to its @run >>>> build JDIScaffold comment to the class. So I included that change >>>> as well in the review. Note : this class has already been updated >>>> to extend TestScaffold. >>>> >>>> http://cr.openjdk.java.net/~psomashe/4916621/webrev >>>> .01 >>>> >>>> >>>> thanks, >>>> Paru. >>>> >>>> On 2/15/18, 4:22 PM, Chris Plummer wrote: >>>>> Hi Paru, >>>>> >>>>> I think instead of moving the following code, which did to avoid >>>>> having to save the value of args, can instead replaced with a call >>>>> to TestScaffold.startup("TestVars"): >>>>> >>>>> 147 List argList = new ArrayList(Arrays.asList(args)); >>>>> 148 argList.add("TestVars"); >>>>> 149 System.out.println("run args: " + argList); >>>>> 150 connect((String[])argList.toArray(args)); >>>>> 151 waitForVMStart(); >>>>> >>>>> Other than that it looks good. >>>>> >>>>> thanks, >>>>> >>>>> Chris >>>>> >>>>> On 2/15/18 3:46 PM, serguei.spitsyn at oracle.com >>>>> wrote: >>>>>> Hi Paru, >>>>>> >>>>>> This looks good. >>>>>> We need one more reviewer. >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>>> >>>>>> On 2/15/18 15:22, Paru Somashekar wrote: >>>>>>> Hi, >>>>>>> >>>>>>> Please review fix for JDK-4916621 >>>>>>> >>>>>>> These are the remaining tests from the list that needed to be >>>>>>> updated. >>>>>>> >>>>>>> Bug : JDK-4916621 >>>>>>> Webrev : http://cr.openjdk.java.net/~psomashe/4916621/webrev/ >>>>>>> >>>>>>> >>>>>>> The updated tests ran successfully with Mach5. >>>>>>> >>>>>>> thanks, >>>>>>> Paru. >>>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hohensee at amazon.com Fri Feb 16 20:25:40 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Fri, 16 Feb 2018 20:25:40 +0000 Subject: RFR (S): 8195115: G1 Old Gen MemoryPool CollectionUsage.used values don't reflect mixed GC results In-Reply-To: <5768E4BF-5556-455B-899B-94A67BF940A0@amazon.com> References: <3A67BF1B-CE35-4759-B5B4-959C24020A45@amazon.com> <22ceea57-7d27-8350-4457-21f765cb3d0f@oracle.com> <878D6678-84CB-471E-B126-AF14B1BC8EB9@amazon.com> <727f9bcb-9404-df32-917f-70aca6e31cb6@oracle.com> <4BA32239-D645-4B1A-A58E-502258C8FFB7@amazon.com> <997a766f-9f4b-f192-ae87-03f4c8d2666d@oracle.com> <64793ff3-4805-79f9-4e72-6e469cd511dc@oracle.com> <15754d34-f3c7-6431-7abf-05214bd6b9d2@oracle.com> <93417506-8e12-6ad3-690a-439e36719652@oracle.com> <99C937B6-F67B-4F4A-ACC8-444B5F83932B@amazon.com> <3922628B-81DE-4904-AEFE-F4163EB1E655@amazon.com> <8444d766-8adb-013f-f75b-b5d4420635df@oracle.com> <5768E4BF-5556-455B-899B-94A67BF940A0@amazon.com> Message-ID: <3E15AAD8-5017-4CCB-B927-6B31FD0D7809@amazon.com> The CSR https://bugs.openjdk.java.net/browse/JDK-8196719 for the original fix has been approved, so I?m back to requesting a code review, please. Bug: https://bugs.openjdk.java.net/browse/JDK-8195115 Webrev: http://cr.openjdk.java.net/~phh/8195115/webrev.hs.01/ Passed a submit repo run, passes its jtreg test, and a JDK8 version is in production use at Amazon. From the original RR: > The bug is that from the JMX point of view, G1?s incremental collector > (misnamed as the ?G1 Young Generation? collector) only affects G1?s > survivor and eden spaces. In fact, mixed collections run by this > collector also affect the G1 old generation. > > This proposed fix is to record, for each of a JMX garbage collector's > memory pools, whether that memory pool is affected by all collections > using that collector. And, for each collection, record whether or not > all the collector's memory pools are affected. After each collection, > for each memory pool, if either all the collector's memory pools were > affected or the memory pool is affected for all collections, record > CollectionUsage for that pool. > > For collectors other than G1 Young Generation, all pools are recorded as > affected by all collections and every collection is recorded as > affecting all the collector?s memory pools. For the G1 Young Generation > collector, the G1 Old Gen pool is recorded as not being affected by all > collections, and non-mixed collections are recorded as not affecting all > memory pools. The result is that for non-mixed collections, > CollectionUsage is recorded after a collection only the G1 Eden Space > and G1 Survivor Space pools, while for mixed collections CollectionUsage > is recorded for G1 Old Gen as well. > > Other than the effect of the fix on G1 Old Gen MemoryPool. > CollectionUsage, the only external behavior change is that > GarbageCollectorMXBean.getMemoryPoolNames will now return 3 pool names > rather than 2. > > With this fix, a collector?s memory pools can be divided into two > disjoint subsets, one that participates in all collections and one that > doesn?t. This is a bit more general than the minimum necessary to fix > G1, but not by much. Because I expect it to apply to other incremental > region-based collectors, I went with the more general solution. I > minimized the amount of code I had to touch by using default parameters > for GCMemoryManager::add_pool and the TraceMemoryManagerStats constructors. Thanks, Paul On 2/12/18, 2:30 PM, "serviceability-dev on behalf of Hohensee, Paul" wrote: Thanks, very much appreciated. Paul On 2/12/18, 7:42 AM, "Erik Helin" wrote: Hi Paul, sorry, I'm just back from a really busy week with two conferences. I will take a look tomorrow. Thanks, Erik On 02/07/2018 11:17 PM, Hohensee, Paul wrote: > I?ve filed https://bugs.openjdk.java.net/browse/JDK-8196989 : Revamp G1 JMX MemoryPool and GarbageCollector MXBean definitions and the corresponding CSR https://bugs.openjdk.java.net/browse/JDK-8196991. > > Would you please comment on the CSR, and on the original CSR https://bugs.openjdk.java.net/browse/JDK-8196719? > > Thanks, > > Paul > > On 2/2/18, 1:20 PM, "hotspot-gc-dev on behalf of Hohensee, Paul" wrote: > > And, can a get a reviewer or reviewers for the CSR? > > Thanks, > > Paul > > On 2/2/18, 1:14 PM, "Hohensee, Paul" wrote: > > +hotspot-gc-use. > > I?ve filed a CSR for the current patch, see https://bugs.openjdk.java.net/browse/JDK-8196719. Here?s the argument in favor. > > > It?s possible that there are JDK8 users that rely on current G1 old gen CollectionUsage behavior, but imo it?s unlikely because it?s of little use. Perhaps Kirk and others with operational experience can weigh in. > > Let?s think about use cases. G1 full GC?s happen rarely and only under severe pressure, so when they do external reaction is pretty much limited to reducing load so the JVM can get back to a usable steady state, or just restarting the JVM. Old gen CollectionUsage is zero until a full GC occurs, after which its value includes both long-lived objects and any transient data that was in eden and the survivor space. That value doesn?t tell you anything about long term old gen occupancy or survivor size because it lumps them all together. So, it isn?t a useful metric, nor will it be after any subsequent full GCs. The only information it provides is on the first zero to non-zero transition, which just tells you that the JVM is/was in trouble. Further, the effect of the runup to a full GC is SLA violations, which are noticed before the full GC happens, so detecting the first full GC is confirmation, not prediction. > > Conclusion: G1 old gen CollectionUsage is unlikely to be in use in its current form, so changing its definition to something usable is low risk. > > > ?G1 Old Space? is fine, as is ?G1 Archive Space?. Are you ok with the G1 archive space overlapping the G1 old space? Should we add an archive space to the other collectors? If so, how would it be defined and would having it overlap with the old generation as a live prefix be ok? > > "G1 Young Generation" is the currently young+mixed collector. > > You?re right, if one is iterating over all collectors, there will be redundancy if we keep the old ones. I?m usually leery of introducing a flag such as UseG1LegacyMXBeans (I changed the name, since all the interfaces are MXBeans, hope that?s ok) which must be either indefinitely maintained, or go through a deprecation cycle. I don?t see a way out of the ?iterate over all collectors? problem without it though. > > Paul > > On 1/31/18, 3:42 AM, "Erik Helin" wrote: > > On 01/31/2018 02:30 AM, Hohensee, Paul wrote: > > It?s true that my patch doesn?t completely solve the larger problem, but it fixes the most immediately important part of it, particularly for JDK8 where current expected behavior is entrenched. > > Yes, your patch fixes part of the problem, but as I said, can > potentially lead to more confusion. I'm not sure that doing this > behavioral change for a public API in an JDK 8 update release is the > right thing. There are likely users that rely on the memory pool "G1 Old > Gen" only being updated by a full collection (even though that behavior > is not correct), those uses will encounter a new behavior in an update > release with your patch. > > The good thing is that we have very experienced engineers participating > in the CSR process that have much more experience than I have in > evaluating the impact of behavioral changes such as this one. Would you > please file a CSR request for your patch to get their opinion? > > See https://wiki.openjdk.java.net/display/csr/Main for more details > about CSR. > > On 01/31/2018 02:30 AM, Hohensee, Paul wrote: > If we?re going to fix the larger problem, imo we should file another > bug/rfe to do it. I?d be happy to fix that one too, once we have a spec. > > > > What do you think of my suggestions? To summarize: > > > > - Keep the existing memory pools and add humongous and archive pools. > > - Make the archive pool part of the old gen, and generalize it to all collectors. > > - Split humongous regions off from the old gen pool into their own pool. The old gen and humongous pools are disjoint region sets. > > - Keep the existing ?G1 Young Generation? and ?G1 Old Generation? collectors and their associated memory pools (net of this patch). We add the humongous pool to them. > > - Add ?G1 Full? as an alias of the existing ?G1 Old Generation? collector. > > - Add the ?G1 Young?, ?G1 Mixed? and ?G1 Concurrent Cycle? collectors. > > - Set the G1 old gen memory pool max size to ?Xmx, the archive space max size to whatever it is, and the rest of the G1 memory pool max sizes to -1 == undefined, as now. > > > > The resulting memory pools: > > > > ?G1 Eden Space? > > ?G1 Survivor Space? > > ?G1 Old Gen? > > ?G1 Humongous Space? > > ?Archive Space? > > The "Space" suffix is unfortunate, but acceptable. I'm least happy about > the "Gen" suffix for the "G1 Old Gen", since G1's old regions differ > from a generation in the traditional sense as applied to e.g. Serial, > Parallel and CMS. I would be more happy to use a consistent naming > scheme in the form of "G1 Old Space" (having only one pool ending "Gen" > begs the question how it differs from the others ending in "Space"). > Again, we could introduce a flag such as -XX:+UseG1LegacyPoolsAndBeans > for those that really wants the old names. > > "Archive Space" should be named "G1 Archive Space" since it differs in > implementation from the other collectors. It would be unfortunate if > users thought they could change collector and the "Archive Space" memory > pool would keep the same behavior. > > > The resulting collectors and their memory pools: > > > > ?G1 Young Generation? (the existing young/mixed collector), ?G1 Old Generation?/?G1 Full?, ?G1 Mixed? > > - ?G1 Eden Space? > > - ?G1 Survivor Space? > > - ?G1 Old Gen? > > - ?G1 Humongous Space? > > ?G1 Young? > > - ?G1 Eden Space? > > - ?G1 Survivor Space? > > - ?G1 Humongous Space? > > ?G1 Concurrent Cycle? > > - ?G1 Old Gen? > > - ?G1 Humongous Space? > > > > I?m not religious about the names, but I like my suggestions. :) > I think it will be confusing for users to have both "G1 Old Generation" > and "G1 Full", particularly for tools iterating over all > GarbageCollectorMXBeans. There is no way to indicate that a > GarbageCollectorMXBeans is an alias of another GarbageCollectorMXBean (I > thought about such a solution as well). > > I guess I don't follow what the GarbageCollectorMXBean "G1 Young > Generation" is meant to represent? > > > The significant addition to my previous email, and an incompatible change, is splitting humongous regions off from the old gen pool. This means that apps that specifically monitor old gen occupancy will no longer see humongous regions. Monitoring apps that just add up info about all a collector?s pools won?t see a difference. I may be corrected (by Kirk, perhaps), but imo it?s not as bad a compatibility issue as one might think, because the type of app that uses a lot of humongous regions isn?t all that common. E.g., apps that cache data in the heap in the form of large compressed arrays, and apps with large hashmap bucket list arrays. The heaps such apps use are very often large enough to use 32mb regions, hence need really big objects to actually allocate humongous regions. > > So why not enable backwards compatibility by allowing a user to set the > flag -XX:+UseG1LegacyPoolsAndBeans? It is not that cumbersome for us to > maintain the current definition of memory pools and collectors. Such a > flag allows us to start over and do this right and a user who relies on > the current behavior can get that by just setting a flag. Doing such a > change in a major release also allows us to clearly highlight the change > in the release notes (users are more prepared for larger changes in a > major release and that they might have to add flags to keep old behavior). > > It is not uncommon for memory pools to change in major releases. The > perm gen pool was removed in JDK 8, the default pools changed when > Parallel Old became default old collector way back in JDK 7 and changed > again when G1 became the default collector in JDK 9. > > Thanks, > Erik > > > Thanks, > > > > Paul > > > > On 1/30/18, 5:51 AM, "Erik Helin" wrote: > > > > On 01/30/2018 03:07 AM, Hohensee, Paul wrote: > > > That?s one reviewer who?s ok with a short term patch. Anyone else? And, > > > any reviewers for said short term patch? :) > > > > Well, the patch is not really complete as it is. The problem is the > > definitions of the MemoryPoolMXBeans and GarbageCollectorMXBeans, which, > > as I tried to hint at in my first email, is a mess for G1. The names and > > implementations of these MemoryPoolMXBeans and GarbageCollectionMXBeans > > for G1 are very old, G1 has changed a lot since those were implemented > > (hence my suggestion for finally fixing this). > > > > The issue with your patch is that the MemoryPoolMXBean named "G1 Old > > Gen" consists of both old and humongous regions (it will also include > > archive regions). Old regions can be collected by mixed, concurrent and > > full collections. Humongous regions can be collected by young, mixed or > > full collections and the concurrent cycle. Archive regions will never be > > collected. Your patch will update the pool in the case of a mixed > > collection collecting old regions or humongous regions, but misses the > > following cases: > > - a young collection collecting humongous regions > > - a concurrent cycle collecting humongous regions > > - a concurrent cycle collecting old regions > > > > Unfortunately I could not come up with a good way to solve the above > > without re-designing the pools. I'm not sure about accepting your patch > > as is, since it might cause even more confusion for users compared to > > the current (already confusing) situation. > > > > One idea we have discussed is to implement the re-design but also add a > > flag, -XX:+UseG1LegacyPoolsAndBeans (false by default), to allow for a > > smoother transition. Would that solution work for you? > > > > Thanks, > > Erik > > > > > Thanks, > > > > > > Paul > > > > > > *From: *mandy chung > > > *Organization: *Oracle Corporation > > > *Date: *Monday, January 29, 2018 at 1:41 PM > > > *To: *"Hohensee, Paul" > > > *Cc: *"serviceability-dev at openjdk.java.net" > > > , "hotspot-gc-dev at openjdk.java.net" > > > > > > *Subject: *Re: RFR (S): 8195115: G1 Old Gen MemoryPool > > > CollectionUsage.used values don't reflect mixed GC results > > > > > > I created JDK-8196362 to look into whether it makes sense to provide > > > some categorization to differentiate eden space vs the heap space for > > > long-lived objects. > > > > > > W.r.t. to JDK-8195115, I have to defer to GC team to comment on the > > > mixed collection update. If they are okay, I have no objection to > > > implement a short-term fix and do the proper G1 memory pools as a > > > separate patch. > > > > > > Mandy > > > > > > On 1/29/18 1:02 PM, Hohensee, Paul wrote: > > > > > > We don?t use getType, and you guessed correctly: we use the memory > > > pool name as an indicator of the specific characteristics of a > > > memory pool, in particular eden. > > > > > > What we want is an indication of long term heap occupancy. We > > > calculate it using CollectionUsage for non-eden heap memory pools, > > > regardless of collector. We don?t use JMX notification, rather we > > > periodically poll CollectionUsage for memory pools with names that > > > contain ?Old?, ?Tenured?, or ?Survivor?. We get the memory pools > > > from the GarbageCollectorMXBeans (we don?t care what the collector > > > names are). For the named memory pools, we sum CollectionUsage.used > > > and divide by the sum of CollectionUsage.max to get a long term heap > > > occupancy percentage. We don?t want to include eden because it?s > > > really just an allocation buffer and not part of the storage for > > > long-lived objects. I suppose we could use a negative test instead > > > by using all memory pools with names that don?t include ?Eden?. > > > > > > The bug is that the ?G1 Old Gen? memory pool isn?t being updated > > > when the ?G1 Young Generation? collector runs a mixed collection. As > > > far as JMX is concerned, that collector only knows about eden and > > > the survivor space. The patch adds the old gen to the memory pools > > > it knows about and has mixed collections update the old gen?s > > > CollectionUsage. > > > > > > I managed to get a submit repo run to succeed last week and it found > > > a problem. I?ve uploaded a new webrev that fixes the failure of the > > > jtreg test TestMemoryMXBeansAndPoolsPresence.java due to the young > > > gen collector being expected to know only about eden and the > > > survivor space. > > > > > > http://cr.openjdk.java.net/~phh/8195115/webrev.hs.01/ > > > > > > > > > Waiting on the submit repo to come back with a result on it. > > > > > > Thanks, > > > > > > Paul > > > > > > *From: *mandy chung > > > > > > *Organization: *Oracle Corporation > > > *Date: *Monday, January 29, 2018 at 10:52 AM > > > *To: *"Hohensee, Paul" > > > , Erik Helin > > > , David Holmes > > > > > > *Cc: *"serviceability-dev at openjdk.java.net" > > > > > > > > > , > > > "hotspot-gc-dev at openjdk.java.net" > > > > > > > > > > > > *Subject: *Re: RFR (S): 8195115: G1 Old Gen MemoryPool > > > CollectionUsage.used values don't reflect mixed GC results > > > > > > On 1/29/18 10:35 AM, mandy chung wrote: > > > > > > Thanks for the reply Paul. Try to understand a little more on > > > the specific from gc-specific memory pool you depend on. > > > > > > On 1/29/18 8:27 AM, Hohensee, Paul wrote: > > > > > > A name change would affect Amazon?s heap monitoring, and > > > thus I expect it would affect other users as well. > > > > > > As long as there are gc-specific memory pools, we?re going > > > to need to be able to identify them, and right now that?s > > > done via name. > > > > > > > > > MemoryPoolMXBean::getType returns "heap" memory type for > > > GC-specific memory pools. Are you using this method? Do you > > > use the name to build in specific characteristic of a memory > > > pool (e.g. eden vs old gen)? > > > > > > > > > > > > > > > All the mxbeans are identified by name, so that?s a general > > > design principle. The only way I can think of to get rid of > > > name dependency would be to figure out what abstract metrics > > > users want to monitor and implement them for all collectors. > > > HeapUsage (instantaneous occupancy) is one, CollectionUsage > > > (long-lived occupancy) is another, both of these for the > > > entire heap, not just particular memory pools. > > > > > > > > > The sum of HeapUsage and CollectionUsage of all heap memory > > > pools was expected to give an incorrect approximation for the > > > entire heap usage. Are you seeing issue/bug with the sum result? > > > > > > > > > typo: s/an incorrect approximation/an approximation. > > > > > > Mandy > > > > > > > > > > > > Mandy > > > > > > > > > > > > That said, imo there will always be a demand for the ability > > > to get collector and memory pool specific details, so I > > > don?t see a way to get around providing named entities. > > > > > > Paul > > > > > > *From: *mandy chung > > > > > > *Organization: *Oracle Corporation > > > *Date: *Friday, January 26, 2018 at 2:38 PM > > > *To: *"Hohensee, Paul" > > > , Erik Helin > > > , > > > David Holmes > > > > > > *Cc: *"serviceability-dev at openjdk.java.net" > > > > > > > > > , > > > "hotspot-gc-dev at openjdk.java.net" > > > > > > > > > > > > *Subject: *Re: RFR (S): 8195115: G1 Old Gen MemoryPool > > > CollectionUsage.used values don't reflect mixed GC results > > > > > > On 1/25/18 1:04 PM, Hohensee, Paul wrote: > > > > > > > > > > The JMX API spec doesn?t specify what the memory pool or > > > garbage > collector names are, but the current names are > > > de-facto part of the > API, so if we change the existing > > > ones, imo a CSR should be filed. > > > > > > The names are implementation details but I can see how an application > > > > > > might be impacted if they depend on it. CSR approval is not strictly > > > > > > necessary while I think filing one to document the change would be > > > > > > good. > > > > > > Does the name change impact any application you know of? I'm trying to > > > > > > understand if any improvement to API is needed so that applications > > > > > > don't need to depend on the names. > > > > > > > > > Mandy > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From mandy.chung at oracle.com Sat Feb 17 02:20:45 2018 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 16 Feb 2018 18:20:45 -0800 Subject: RFR JDK-8198253: ThreadInfo.from(CompositeData) assigning fields incorrectly in JDK 9 In-Reply-To: References: Message-ID: <45a5eee6-023d-e342-d6ed-6797702b8804@oracle.com> Hi Jeremy, lockedMonitors and lockedSynchronizers attribute are not optional if that's the issue you try to resolve. I think the specification should be clarified. ThreadInfo::from supports the three different versions for interoperability: 1. CompositeData for JDK 1.5 ThreadInfo with no lockedMonitors and lockedSynchronizers attribute 2. CompositeData for JDK 6 ThreadInfo with lockedMonitors and lockedSynchronizers attributes 3. CompositeData for JDK 9 ThreadInfo with lockedMonitors and lockedSynchronizers ?attributes and with daemon and priority attribute. JMX client can connect to a running VM in any version and get back a proper ThreadInfo. If ThreadInfo::from is called with a CompositeData containing daemon and priority attribute but lockedMonitors and lockedSynchronizers attributes are absent then the given CompositeData is invalid and IllegalArgumentException should be thrown. Does this help? Mandy On 2/15/18 4:46 PM, Jeremy Manson wrote: > Hi folks, > > Been a long time!? I'd like someone to do a code review. This is in > code I wrote a few years ago, and got wrong.? At the time, David > Holmes, Staffan Larsen, and Mandy Chung reviewed it.? It does mean > that people using ThreadInfo.from(CompositeData) may be getting the > wrong values out for ThreadInfo, so it is definitely worth fixing. > > The patch below fixes the bug, but is a pretty questionable approach.? > Better approaches happily considered. > > Patch: > http://cr.openjdk.java.net/~jmanson/8198253/webrev.00/ > > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8198253 > > Thanks! > > Jeremy -------------- next part -------------- An HTML attachment was scrubbed... URL: From harsha.wardhana.b at oracle.com Tue Feb 20 11:30:05 2018 From: harsha.wardhana.b at oracle.com (Harsha Wardhana B) Date: Tue, 20 Feb 2018 17:00:05 +0530 Subject: RFR : JDK-8196744 : JMX: Not enough JDP packets received before timeout Message-ID: <2eb1146c-f089-716d-8d83-70504cde24f8@oracle.com> Hi All, Please find the fix below for the Jdp test-case. issue: https://bugs.openjdk.java.net/browse/JDK-8196028 webrev : http://cr.openjdk.java.net/~hb/8196028/webrev.00/ Fix details : The test was receiving JDP packets from other VM and hence the multi-cast socket was not timing-out. The default timeout handler was causing test to fail. Added a shutdown method that passes the test in case of timeout. Thanks Harsha From harsha.wardhana.b at oracle.com Tue Feb 20 11:31:08 2018 From: harsha.wardhana.b at oracle.com (Harsha Wardhana B) Date: Tue, 20 Feb 2018 17:01:08 +0530 Subject: RFR: JDK-8187498: Add a -Xmanagement flag as syntactic sugar for -Dcom.sun.management.jmxremote.* properties In-Reply-To: <0332f3ad-1329-da9a-c6a7-192064fb04bb@oracle.com> References: <0fca13c3-1b8f-947c-9041-bdea87b04ba8@oracle.com> <5A687F75.3050404@oracle.com> <05c2e5af-b9d0-0386-4565-ade1179a1d49@oracle.com> <7a18b9d5-3a59-8132-2c4f-ba5de35bfa1d@oracle.com> <0e019151-ef48-b4f4-d253-b9da62aa07c3@oracle.com> <248276b1-1589-561f-0718-7bcb1fd578c7@oracle.com> <40cf194e-4d74-3a34-5eb2-1acc5a5abafb@oracle.com> <87cb309b-8764-193a-0447-8ecb741d308d@oracle.com> <66ad342e-f41f-c01b-e515-8100d108e6ef@oracle.com> <0a13fe7c-b7b0-e5d9-1361-bfea19073574@oracle.com> <0332f3ad-1329-da9a-c6a7-192064fb04bb@oracle.com> Message-ID: <9ac62f66-c14d-951f-1a14-ccf8906ecab4@oracle.com> Ping. Could I please have reviews for below webrev. Thanks Harsha On Wednesday 14 February 2018 09:59 PM, Harsha Wardhana B wrote: > Hi, > > Below is the updated webrev. > > http://cr.openjdk.java.net/~hb/8187498/webrev.03/ > > > On Wednesday 14 February 2018 01:15 AM, mandy chung wrote: >> >> >> On 2/13/18 1:30 AM, Harsha Wardhana B wrote: >>> Hi, >>> >>> Please find below the revised webrev. >>> >>> http://cr.openjdk.java.net/~hb/8187498/webrev.02/ >>> >>> >>> On Friday 09 February 2018 05:07 AM, mandy chung wrote: >>>> On 2/7/18 1:19 AM, Harsha Wardhana B wrote: >>>> > >>>> > --start-management-agent will not be recognized in the current format and >>>> > hence will not default to --start-management-agent=local=true. >>>> >>>> `--start-local-management-server` is one option as Alan suggests. >>>> Another option is to make `--start-management-agent` to accept >>>> an optional argument. VM can accept `--start-management-agent` >>>> or `--start-management-agent=port=1234,ssl=on`. It may require >>>> more launcher change to support it. >>> Yes. It requires lot more changes to launcher. Hence optional >>> argument to --start-management-agent was not added in order to keep >>> the launcher impact minimum. >> >> This is just one option for you to consider.?? So the current >> proposal of the new option to start a remote management server, right? > Not necessarily. --start-management-agent=local=true starts local > server and --start-management-agent=port=1234 starts remote management > server. >>>> Agent.java >>>> If --start-management-agent is set, -Dcom.sun.management.* takes >>>> precedence. Do you really want to do that? The new VM option >>>> intends to simplify the command-line to type in. I think it's >>>> cleaner and reasonable if --start-management-agent is specified, >>>> -Dcom.sun.management.* are ignored (maybe worth emit a warning if set) >>> We can probably document that -D options will be overridden instead >>> of emitting a warning. >> >> What is the behavior when -Dcom.sun.management.jmxremote.port=1234 >> --start-management-agent port=2345 >> -Dcom.sun.management.jmxremote.port=3456? >> >> What is the value of the system property >> com.sun.management.jmxremote.port at runtime?? What port number does >> the management server start with? > As said earlier, values set via new flags override values set by -D > flags. So 2345 will be the value of com.sun.management.jmxremote.port. > Added a test case to validate that. >> >>>> The implementation uses VMManagement::getVmArguments and scan >>>> the VM options for -start-management-agent. The VM is the one >>>> invoking jdk.internal.agent.Agent::startAgent. A simpler option >>>> is to change Agent::startAgent to take an argument parameter >>>> that will be passed with the argument of --start-management-agent >>>> or null if not set. >>>> >>>> Similarly for startRemoteManagementAgent and startLocalAgent_name. >>> Implementing this change requires lot of changes to argument parsing >>> in native code and hence it is simpler to handle this at java layer. >> >> Can you describe how --start-management-agent option will be used for >> jcmd and any other tool that attaches to a running VM to start the >> agent?? Example command-line will be useful. > --start-management-agent cannot be used by jcmd (or dynamic attach) as > it accepts flags only in -D format or -D flags with com.sun.management > removed. That code to parse string passed via jcmd was a mistake and > hence I have removed it. >> >> >> Mandy > -Harsha From kumar.x.srinivasan at oracle.com Tue Feb 20 14:41:58 2018 From: kumar.x.srinivasan at oracle.com (Kumar Srinivasan) Date: Tue, 20 Feb 2018 06:41:58 -0800 Subject: RFR: JDK-8187498: Add a -Xmanagement flag as syntactic sugar for -Dcom.sun.management.jmxremote.* properties In-Reply-To: <9ac62f66-c14d-951f-1a14-ccf8906ecab4@oracle.com> References: <0fca13c3-1b8f-947c-9041-bdea87b04ba8@oracle.com> <5A687F75.3050404@oracle.com> <05c2e5af-b9d0-0386-4565-ade1179a1d49@oracle.com> <7a18b9d5-3a59-8132-2c4f-ba5de35bfa1d@oracle.com> <0e019151-ef48-b4f4-d253-b9da62aa07c3@oracle.com> <248276b1-1589-561f-0718-7bcb1fd578c7@oracle.com> <40cf194e-4d74-3a34-5eb2-1acc5a5abafb@oracle.com> <87cb309b-8764-193a-0447-8ecb741d308d@oracle.com> <66ad342e-f41f-c01b-e515-8100d108e6ef@oracle.com> <0a13fe7c-b7b0-e5d9-1361-bfea19073574@oracle.com> <0332f3ad-1329-da9a-c6a7-192064fb04bb@oracle.com> <9ac62f66-c14d-951f-1a14-ccf8906ecab4@oracle.com> Message-ID: <5A8C33B6.4040206@oracle.com> You are going backwards. You need to have a CSR approved first. Kumar > Ping. Could I please have reviews for below webrev. > > Thanks > Harsha > > On Wednesday 14 February 2018 09:59 PM, Harsha Wardhana B wrote: >> Hi, >> >> Below is the updated webrev. >> >> http://cr.openjdk.java.net/~hb/8187498/webrev.03/ >> >> >> On Wednesday 14 February 2018 01:15 AM, mandy chung wrote: >>> >>> >>> On 2/13/18 1:30 AM, Harsha Wardhana B wrote: >>>> Hi, >>>> >>>> Please find below the revised webrev. >>>> >>>> http://cr.openjdk.java.net/~hb/8187498/webrev.02/ >>>> >>>> >>>> On Friday 09 February 2018 05:07 AM, mandy chung wrote: >>>>> On 2/7/18 1:19 AM, Harsha Wardhana B wrote: >>>>> > > --start-management-agent will not be recognized in the current >>>>> format and >>>>> > hence will not default to --start-management-agent=local=true. >>>>> >>>>> `--start-local-management-server` is one option as Alan suggests. >>>>> Another option is to make `--start-management-agent` to accept >>>>> an optional argument. VM can accept `--start-management-agent` >>>>> or `--start-management-agent=port=1234,ssl=on`. It may require >>>>> more launcher change to support it. >>>> Yes. It requires lot more changes to launcher. Hence optional >>>> argument to --start-management-agent was not added in order to keep >>>> the launcher impact minimum. >>> >>> This is just one option for you to consider. So the current >>> proposal of the new option to start a remote management server, right? >> Not necessarily. --start-management-agent=local=true starts local >> server and --start-management-agent=port=1234 starts remote >> management server. >>>>> Agent.java >>>>> If --start-management-agent is set, -Dcom.sun.management.* takes >>>>> precedence. Do you really want to do that? The new VM option >>>>> intends to simplify the command-line to type in. I think it's >>>>> cleaner and reasonable if --start-management-agent is specified, >>>>> -Dcom.sun.management.* are ignored (maybe worth emit a warning >>>>> if set) >>>> We can probably document that -D options will be overridden instead >>>> of emitting a warning. >>> >>> What is the behavior when -Dcom.sun.management.jmxremote.port=1234 >>> --start-management-agent port=2345 >>> -Dcom.sun.management.jmxremote.port=3456? >>> >>> What is the value of the system property >>> com.sun.management.jmxremote.port at runtime? What port number does >>> the management server start with? >> As said earlier, values set via new flags override values set by -D >> flags. So 2345 will be the value of >> com.sun.management.jmxremote.port. Added a test case to validate that. >>> >>>>> The implementation uses VMManagement::getVmArguments and scan >>>>> the VM options for -start-management-agent. The VM is the one >>>>> invoking jdk.internal.agent.Agent::startAgent. A simpler option >>>>> is to change Agent::startAgent to take an argument parameter >>>>> that will be passed with the argument of --start-management-agent >>>>> or null if not set. >>>>> >>>>> Similarly for startRemoteManagementAgent and startLocalAgent_name. >>>> Implementing this change requires lot of changes to argument >>>> parsing in native code and hence it is simpler to handle this at >>>> java layer. >>> >>> Can you describe how --start-management-agent option will be used >>> for jcmd and any other tool that attaches to a running VM to start >>> the agent? Example command-line will be useful. >> --start-management-agent cannot be used by jcmd (or dynamic attach) >> as it accepts flags only in -D format or -D flags with >> com.sun.management removed. That code to parse string passed via jcmd >> was a mistake and hence I have removed it. >>> >>> >>> Mandy >> -Harsha > From erik.gahlin at oracle.com Tue Feb 20 15:14:42 2018 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Tue, 20 Feb 2018 16:14:42 +0100 Subject: RFR : JDK-8196744 : JMX: Not enough JDP packets received before timeout In-Reply-To: <2eb1146c-f089-716d-8d83-70504cde24f8@oracle.com> References: <2eb1146c-f089-716d-8d83-70504cde24f8@oracle.com> Message-ID: <5A8C3B62.5050302@oracle.com> Looks OK. Erik > Hi All, > > Please find the fix below for the Jdp test-case. > > issue: https://bugs.openjdk.java.net/browse/JDK-8196028 > webrev : http://cr.openjdk.java.net/~hb/8196028/webrev.00/ > > Fix details : The test was receiving JDP packets from other VM and > hence the multi-cast socket was not timing-out. The default timeout > handler was causing test to fail. Added a shutdown method that passes > the test in case of timeout. > > Thanks > Harsha From harsha.wardhana.b at oracle.com Tue Feb 20 15:29:34 2018 From: harsha.wardhana.b at oracle.com (Harsha Wardhana B) Date: Tue, 20 Feb 2018 20:59:34 +0530 Subject: RFR : JDK-8196028 : JMX: Not enough JDP packets received before timeout In-Reply-To: <5A8C3B62.5050302@oracle.com> References: <2eb1146c-f089-716d-8d83-70504cde24f8@oracle.com> <5A8C3B62.5050302@oracle.com> Message-ID: <53155029-32ab-3fe4-b00c-ffa3267e3e64@oracle.com> [Correcting bug Id in the subject] Thanks Erik. Harsha On Tuesday 20 February 2018 08:44 PM, Erik Gahlin wrote: > Looks OK. > > Erik > >> Hi All, >> >> Please find the fix below for the Jdp test-case. >> >> issue: https://bugs.openjdk.java.net/browse/JDK-8196028 >> webrev : http://cr.openjdk.java.net/~hb/8196028/webrev.00/ >> >> Fix details : The test was receiving JDP packets from other VM and >> hence the multi-cast socket was not timing-out. The default timeout >> handler was causing test to fail. Added a shutdown method that passes >> the test in case of timeout. >> >> Thanks >> Harsha > From karen.kinnear at oracle.com Tue Feb 20 15:54:28 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Tue, 20 Feb 2018 10:54:28 -0500 Subject: JVMTI retransformation and addition of private methods Message-ID: <386B2730-5732-4937-B334-411D8E458022@oracle.com> Folks, As part of the Valhalla EG discussions for JVMTI changes for nestmates (thank you Serguei and David), IBM brought up a request that we update the JVMTI documentation to reflect that we allow addition of private methods. Is there a reason we do not document this? I?m inviting those who were involved at the time - please include others if needed. thanks, Karen From daniil.x.titov at oracle.com Tue Feb 20 17:24:17 2018 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Tue, 20 Feb 2018 09:24:17 -0800 Subject: RFR 8170541: serviceability/jdwp/AllModulesCommandTest.java fails intermittently on Windows and Solaris Message-ID: Please review the changes that fix intermittent failure of serviceability/jdwp/AllModulesCommandTest.java test. The problem here is that for a large data the JDWP agent (socketTransport_writePacket() method in src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c ) sends 2 packets and in some cases only the first packet is received at the time when the test reads the reply from the JDWP agent. Since the test does not check that all data is received in the first packet the correlation between commands and replies became broken (the unread second packet is read by the next command and the reply for the next command is read by the next after next command and so on). Bug: https://bugs.openjdk.java.net/browse/JDK-8170541 Webrev: http://cr.openjdk.java.net/~dtitov/8170541/webrev.01 The tests ran successfully with Mach5. Best regards, Daniil From mandy.chung at oracle.com Tue Feb 20 20:21:01 2018 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 20 Feb 2018 12:21:01 -0800 Subject: RFR: JDK-8187498: Add a -Xmanagement flag as syntactic sugar for -Dcom.sun.management.jmxremote.* properties In-Reply-To: <5A8C33B6.4040206@oracle.com> References: <0fca13c3-1b8f-947c-9041-bdea87b04ba8@oracle.com> <5A687F75.3050404@oracle.com> <05c2e5af-b9d0-0386-4565-ade1179a1d49@oracle.com> <7a18b9d5-3a59-8132-2c4f-ba5de35bfa1d@oracle.com> <0e019151-ef48-b4f4-d253-b9da62aa07c3@oracle.com> <248276b1-1589-561f-0718-7bcb1fd578c7@oracle.com> <40cf194e-4d74-3a34-5eb2-1acc5a5abafb@oracle.com> <87cb309b-8764-193a-0447-8ecb741d308d@oracle.com> <66ad342e-f41f-c01b-e515-8100d108e6ef@oracle.com> <0a13fe7c-b7b0-e5d9-1361-bfea19073574@oracle.com> <0332f3ad-1329-da9a-c6a7-192064fb04bb@oracle.com> <9ac62f66-c14d-951f-1a14-ccf8906ecab4@oracle.com> <5A8C33B6.4040206@oracle.com> Message-ID: The code review and CSR review can be in parallel. For this case, I agree with Kumar to have CSR written that would help the code review. Please specify the behavior and its relationship with jcmd and other relevant diagnosability tools. On 2/20/18 6:41 AM, Kumar Srinivasan wrote: > >>>> What is the behavior when -Dcom.sun.management.jmxremote.port=1234 >>>> --start-management-agent port=2345 >>>> -Dcom.sun.management.jmxremote.port=3456? >>>> >>>> What is the value of the system property >>>> com.sun.management.jmxremote.port at runtime?? What port number >>>> does the management server start with? >>> As said earlier, values set via new flags override values set by -D >>> flags. So 2345 will be the value of >>> com.sun.management.jmxremote.port. Added a test case to validate that. VM options are the last one wins if same option specified multiple times.? In this case, it could cause confusion (the last -D option sets the value to 3456 but it's set to a different value). Why not taking the simplest approach - when --start-management-agent is set, it does not accept mixing the old way (i.e. does not accept the management properties to be set via -D)??? This RFE is to make the command-line simpler and ease-of-use.? I don't see any downside to migrate entirely to the new form. Mandy -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Tue Feb 20 23:00:01 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 20 Feb 2018 15:00:01 -0800 Subject: RFR 8170541: serviceability/jdwp/AllModulesCommandTest.java fails intermittently on Windows and Solaris In-Reply-To: References: Message-ID: <86bf7cb3-50a2-ddeb-0f88-73b8b2b4b2cf@oracle.com> An HTML attachment was scrubbed... URL: From daniil.x.titov at oracle.com Wed Feb 21 00:14:57 2018 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Tue, 20 Feb 2018 16:14:57 -0800 Subject: RFR 8170541: serviceability/jdwp/AllModulesCommandTest.java fails intermittently on Windows and Solaris In-Reply-To: <86bf7cb3-50a2-ddeb-0f88-73b8b2b4b2cf@oracle.com> References: <86bf7cb3-50a2-ddeb-0f88-73b8b2b4b2cf@oracle.com> Message-ID: <18D0CADB-CDB9-4F5D-B77A-DE5A5F6A5C19@oracle.com> Hi Serguei, A new version of the webrev that has these strings reformatted is at http://cr.openjdk.java.net/~dtitov/8170541/webrev.02/ Thank you! Best regards, Daniil From: "serguei.spitsyn at oracle.com" Date: Tuesday, February 20, 2018 at 3:00 PM To: Daniil Titov , "serviceability-dev at openjdk.java.net" Subject: Re: RFR 8170541: serviceability/jdwp/AllModulesCommandTest.java fails intermittently on Windows and Solaris Hi Daniil, Interesting issue... Thank you for finding to the root cause so quickly! The fix looks good. Could I ask you to reformat these lines to make the L54 shorter ?: ? 54???????????????? System.out.println("[" + getClass().getName() + "] Only " + bytesRead + " bytes of " + dataLength + ? 55???????????????????????? " were read in the first packet. Reading the rest..."); Thanks, Serguei On 2/20/18 09:24, Daniil Titov wrote: Please review the changes that fix intermittent failure of serviceability/jdwp/AllModulesCommandTest.java test. The problem here is that for a large data the JDWP agent (socketTransport_writePacket() method in src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c ) sends 2 packets and in some cases only the first packet is received at the time when the test reads the reply from the JDWP agent. Since the test does not check that all data is received in the first packet the correlation between commands and replies became broken (the unread second packet is read by the next command and the reply for the next command is read by the next after next command and so on). Bug: https://bugs.openjdk.java.net/browse/JDK-8170541 Webrev: http://cr.openjdk.java.net/~dtitov/8170541/webrev.01 The tests ran successfully with Mach5. Best regards, Daniil -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeremymanson at google.com Wed Feb 21 00:46:31 2018 From: jeremymanson at google.com (Jeremy Manson) Date: Tue, 20 Feb 2018 16:46:31 -0800 Subject: RFR JDK-8198253: ThreadInfo.from(CompositeData) assigning fields incorrectly in JDK 9 In-Reply-To: <45a5eee6-023d-e342-d6ed-6797702b8804@oracle.com> References: <45a5eee6-023d-e342-d6ed-6797702b8804@oracle.com> Message-ID: (I dropped serviceability-dev from this thread by mistake! Oops!) Okay. Here's the revised patch. LMK if that's what you had in mind. http://cr.openjdk.java.net/~jmanson/8198253/webrev.01/ On Fri, Feb 16, 2018 at 6:20 PM, mandy chung wrote: > Hi Jeremy, > > lockedMonitors and lockedSynchronizers attribute are not optional if that's > the issue you try to resolve. I think the specification should be clarified. > > ThreadInfo::from supports the three different versions for interoperability: > 1. CompositeData for JDK 1.5 ThreadInfo with no lockedMonitors and > lockedSynchronizers attribute > 2. CompositeData for JDK 6 ThreadInfo with lockedMonitors and lockedSynchronizers > attributes > 3. CompositeData for JDK 9 ThreadInfo with lockedMonitors and lockedSynchronizers > attributes and with daemon and priority attribute. > > JMX client can connect to a running VM in any version and get back a > proper ThreadInfo. > > If ThreadInfo::from is called with a CompositeData containing daemon and > priority attribute but lockedMonitors and lockedSynchronizers attributes are > absent then the given CompositeData is invalid and IllegalArgumentException > should be thrown. > > Does this help? > > Mandy > > > On 2/15/18 4:46 PM, Jeremy Manson wrote: > > Hi folks, > > Been a long time! I'd like someone to do a code review. This is in code > I wrote a few years ago, and got wrong. At the time, David Holmes, Staffan > Larsen, and Mandy Chung reviewed it. It does mean that people using > ThreadInfo.from(CompositeData) may be getting the wrong values out for > ThreadInfo, so it is definitely worth fixing. > > The patch below fixes the bug, but is a pretty questionable approach. > Better approaches happily considered. > > Patch: > http://cr.openjdk.java.net/~jmanson/8198253/webrev.00/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8198253 > > Thanks! > > Jeremy > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mandy.chung at oracle.com Wed Feb 21 02:41:05 2018 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 20 Feb 2018 18:41:05 -0800 Subject: RFR JDK-8198253: ThreadInfo.from(CompositeData) assigning fields incorrectly in JDK 9 In-Reply-To: References: <45a5eee6-023d-e342-d6ed-6797702b8804@oracle.com> Message-ID: <9110e27c-e492-b089-7b02-36459915c779@oracle.com> Hi Jeremy, Another approach is to change ThreadInfoCompositeData::validateCompositeData to validate the given CompositeData.?? I also revised ThreadInfoCompositeData to return the default value of the attributes, if not present. This is the patch: http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/webrev.00/ What do you think? Mandy On 2/20/18 4:46 PM, Jeremy Manson wrote: > (I dropped serviceability-dev from this thread by mistake! Oops!) > > Okay. Here's the revised patch.? LMK if that's what you had in mind. > > http://cr.openjdk.java.net/~jmanson/8198253/webrev.01/ > > > On Fri, Feb 16, 2018 at 6:20 PM, mandy chung > wrote: > > Hi Jeremy, > > lockedMonitors and lockedSynchronizers attribute are not optional if that's > the issue you try to resolve. I think the specification should be clarified. > > ThreadInfo::from supports the three different versions for interoperability: > 1. CompositeData for JDK 1.5 ThreadInfo with no lockedMonitors and > lockedSynchronizers attribute > 2. CompositeData for JDK 6 ThreadInfo with lockedMonitors and lockedSynchronizers > attributes > 3. CompositeData for JDK 9 ThreadInfo with lockedMonitors and lockedSynchronizers > ?attributes and with daemon and priority attribute. > > JMX client can connect to a running VM in any version and get back a > proper ThreadInfo. > > If ThreadInfo::from is called with a CompositeData containing daemon and > priority attribute but lockedMonitors and lockedSynchronizers attributes are > absent then the given CompositeData is invalid and IllegalArgumentException > should be thrown. > > Does this help? > > Mandy > > > On 2/15/18 4:46 PM, Jeremy Manson wrote: >> Hi folks, >> >> Been a long time!? I'd like someone to do a code review.? This is >> in code I wrote a few years ago, and got wrong.? At the time, >> David Holmes, Staffan Larsen, and Mandy Chung reviewed it.? It >> does mean that people using ThreadInfo.from(CompositeData) may be >> getting the wrong values out for ThreadInfo, so it is definitely >> worth fixing. >> >> The patch below fixes the bug, but is a pretty questionable >> approach.? Better approaches happily considered. >> >> Patch: >> http://cr.openjdk.java.net/~jmanson/8198253/webrev.00/ >> >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8198253 >> >> >> Thanks! >> >> Jeremy > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Wed Feb 21 03:31:22 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 20 Feb 2018 19:31:22 -0800 Subject: RFR 8170541: serviceability/jdwp/AllModulesCommandTest.java fails intermittently on Windows and Solaris In-Reply-To: <18D0CADB-CDB9-4F5D-B77A-DE5A5F6A5C19@oracle.com> References: <86bf7cb3-50a2-ddeb-0f88-73b8b2b4b2cf@oracle.com> <18D0CADB-CDB9-4F5D-B77A-DE5A5F6A5C19@oracle.com> Message-ID: <9043dc28-5a69-296d-f127-0b449e20d8e3@oracle.com> An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Wed Feb 21 03:52:55 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 21 Feb 2018 13:52:55 +1000 Subject: JVMTI retransformation and addition of private methods In-Reply-To: <386B2730-5732-4937-B334-411D8E458022@oracle.com> References: <386B2730-5732-4937-B334-411D8E458022@oracle.com> Message-ID: <6ddd571e-11a9-aa64-5ba6-dd98df9a792a@oracle.com> Hi Karen, On 21/02/2018 1:54 AM, Karen Kinnear wrote: > Folks, > > As part of the Valhalla EG discussions for JVMTI changes for nestmates (thank you Serguei and David), > IBM brought up a request that we update the JVMTI documentation to reflect that we allow addition > of private methods. > > Is there a reason we do not document this? I?m inviting those who were involved at the time - please include > others if needed. This issue is tracked by: https://bugs.openjdk.java.net/browse/JDK-8192936 "RI does not follow the JVMTI RedefineClasses spec that is too strict in the definition" As I wrote there ... It is not at all clear how JDK-6404550 morphed into "Permit the adding or deleting of private final/static methods with redefine" - nor why those changes failed to make any change to the spec itself. It is also unclear whether the add/delete is restricted to final/static methods or any private method? I can see that the intent was to only allow something that would not perturb the vtable for existing instances. -- David > thanks, > Karen > From david.holmes at oracle.com Wed Feb 21 04:02:48 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 21 Feb 2018 14:02:48 +1000 Subject: RFR 8170541: serviceability/jdwp/AllModulesCommandTest.java fails intermittently on Windows and Solaris In-Reply-To: <18D0CADB-CDB9-4F5D-B77A-DE5A5F6A5C19@oracle.com> References: <86bf7cb3-50a2-ddeb-0f88-73b8b2b4b2cf@oracle.com> <18D0CADB-CDB9-4F5D-B77A-DE5A5F6A5C19@oracle.com> Message-ID: <4ba6d0b1-0277-3e5e-1aa9-49e1db636586@oracle.com> Hi Daniil, Good find on this! What does the actual spec say about the length of things and how they may be split across multiple packets? Are we guaranteed that at most two packets will be involved? What about for other things eg: 68 protected byte[] readJdwpString(DataInputStream ds) throws IOException { 69 byte[] str = null; 70 int len = ds.readInt(); 71 if (len > 0) { 72 str = new byte[len]; 73 ds.read(str, 0, len); 74 } might we get a short-read of the string if it is split across multiple packets? I'm wondering if all these reads should be loops, ensuring we read the expected amount of data. One further comment - not sure why we need the print out for when we do read multiple packets? That would seem to be a debugging aid. Thanks, David On 21/02/2018 10:14 AM, Daniil Titov wrote: > Hi Serguei, > > A new version of the webrev that has these strings reformatted is at > http://cr.openjdk.java.net/~dtitov/8170541/webrev.02/ > > Thank you! > > Best regards, > > Daniil > > *From: *"serguei.spitsyn at oracle.com" > *Date: *Tuesday, February 20, 2018 at 3:00 PM > *To: *Daniil Titov , > "serviceability-dev at openjdk.java.net" > *Subject: *Re: RFR 8170541: > serviceability/jdwp/AllModulesCommandTest.java fails intermittently on > Windows and Solaris > > Hi Daniil, > > Interesting issue... > Thank you for finding to the root cause so quickly! > > The fix looks good. > Could I ask you to reformat these lines to make the L54 shorter ?: > > ? 54???????????????? System.out.println("[" + getClass().getName() + "] > Only " + bytesRead + " bytes of " + dataLength + > > ? 55???????????????????????? " were read in the first packet. Reading > the rest..."); > > Thanks, > Serguei > > > On 2/20/18 09:24, Daniil Titov wrote: > > Please review the changes that fix intermittent failure of > serviceability/jdwp/AllModulesCommandTest.java test. > > The problem here is that for a large data the JDWP agent > (socketTransport_writePacket() method in > src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c ) > sends 2 packets and in some cases only the first packet is received > at the time when the test reads the reply from the JDWP agent. Since > the test does not check that all data is received in the first > packet the correlation between commands and replies became broken > (the unread second packet is read by the next command and the reply > for the next command is read by the next after next command and so on). > > Bug: https://bugs.openjdk.java.net/browse/JDK-8170541 > > Webrev: http://cr.openjdk.java.net/~dtitov/8170541/webrev.01 > > The tests ran successfully with Mach5. > > Best regards, > > Daniil > > > From daniil.x.titov at oracle.com Wed Feb 21 04:20:28 2018 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Tue, 20 Feb 2018 20:20:28 -0800 Subject: RFR 8170541: serviceability/jdwp/AllModulesCommandTest.java fails intermittently on Windows and Solaris In-Reply-To: <9043dc28-5a69-296d-f127-0b449e20d8e3@oracle.com> References: <86bf7cb3-50a2-ddeb-0f88-73b8b2b4b2cf@oracle.com> <18D0CADB-CDB9-4F5D-B77A-DE5A5F6A5C19@oracle.com> <9043dc28-5a69-296d-f127-0b449e20d8e3@oracle.com> Message-ID: <2D1CFB29-BBE3-4375-9527-94CF005CD7D9@oracle.com> Hi Serguei, It is correct. ?I was able to reproduce it locally and on Mach5. Before the fix the failure rate on Mach5 was about 15%. ?After the fix I run it on Mach5 400 times and no any failure was reported. Best regards, Daniil From: "serguei.spitsyn at oracle.com" Date: Tuesday, February 20, 2018 at 7:31 PM To: Daniil Titov , "serviceability-dev at openjdk.java.net" Subject: Re: RFR 8170541: serviceability/jdwp/AllModulesCommandTest.java fails intermittently on Windows and Solaris Hi Daniil, It looks good to me. It seems, you have been able to reproduce this failure, and with this fix the test does not fail anymore. Is that right? Just wanted to be sure. Thanks, Serguei On 2/20/18 16:14, Daniil Titov wrote: Hi Serguei, A new version of the webrev that has these strings reformatted is at http://cr.openjdk.java.net/~dtitov/8170541/webrev.02/ Thank you! Best regards, Daniil From: "serguei.spitsyn at oracle.com" Date: Tuesday, February 20, 2018 at 3:00 PM To: Daniil Titov , "serviceability-dev at openjdk.java.net" Subject: Re: RFR 8170541: serviceability/jdwp/AllModulesCommandTest.java fails intermittently on Windows and Solaris Hi Daniil, Interesting issue... Thank you for finding to the root cause so quickly! The fix looks good. Could I ask you to reformat these lines to make the L54 shorter ?: 54 System.out.println("[" + getClass().getName() + "] Only " + bytesRead + " bytes of " + dataLength + 55 " were read in the first packet. Reading the rest..."); Thanks, Serguei On 2/20/18 09:24, Daniil Titov wrote: Please review the changes that fix intermittent failure of serviceability/jdwp/AllModulesCommandTest.java test. The problem here is that for a large data the JDWP agent (socketTransport_writePacket() method in src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c ) sends 2 packets and in some cases only the first packet is received at the time when the test reads the reply from the JDWP agent. Since the test does not check that all data is received in the first packet the correlation between commands and replies became broken (the unread second packet is read by the next command and the reply for the next command is read by the next after next command and so on). Bug: https://bugs.openjdk.java.net/browse/JDK-8170541 Webrev: http://cr.openjdk.java.net/~dtitov/8170541/webrev.01 The tests ran successfully with Mach5. Best regards, Daniil -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Wed Feb 21 04:31:08 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 20 Feb 2018 20:31:08 -0800 Subject: RFR : JDK-8196744 : JMX: Not enough JDP packets received before timeout In-Reply-To: <2eb1146c-f089-716d-8d83-70504cde24f8@oracle.com> References: <2eb1146c-f089-716d-8d83-70504cde24f8@oracle.com> Message-ID: <4a870972-15c1-695f-2912-6ca5fe92ea61@oracle.com> Hi Harsha, Not a review, but just a request that you add the explanation of the problem to the CR so we have a record of it. Also, the copyright needs to be updated. thanks, Chris On 2/20/18 3:30 AM, Harsha Wardhana B wrote: > Hi All, > > Please find the fix below for the Jdp test-case. > > issue: https://bugs.openjdk.java.net/browse/JDK-8196028 > webrev : http://cr.openjdk.java.net/~hb/8196028/webrev.00/ > > Fix details : The test was receiving JDP packets from other VM and > hence the multi-cast socket was not timing-out. The default timeout > handler was causing test to fail. Added a shutdown method that passes > the test in case of timeout. > > Thanks > Harsha From jeremymanson at google.com Wed Feb 21 05:48:25 2018 From: jeremymanson at google.com (Jeremy Manson) Date: Tue, 20 Feb 2018 21:48:25 -0800 Subject: RFR JDK-8198253: ThreadInfo.from(CompositeData) assigning fields incorrectly in JDK 9 In-Reply-To: <9110e27c-e492-b089-7b02-36459915c779@oracle.com> References: <45a5eee6-023d-e342-d6ed-6797702b8804@oracle.com> <9110e27c-e492-b089-7b02-36459915c779@oracle.com> Message-ID: I think that's a much better approach (I didn't notice the validate method :) ) I think you may want to grab my test changes, or make some similar ones. Presumably, the tests do not pass with your change. I think the API wording change is very slightly confusing. * A {@code CompositeData} does not contain this attribute > * when representing a {@code ThreadInfo} of JDK 6 or older > version. > * This attribute will be set to {@link > Thread#NORM_PRIORITY}. I'd probably say "In such cases" at the beginning of the second sentence. Also, is there a rule about when you say JDK 6 and when you say Java 6? Jeremy On Tue, Feb 20, 2018 at 6:41 PM, mandy chung wrote: > Hi Jeremy, > > Another approach is to change ThreadInfoCompositeData:: > validateCompositeData > to validate the given CompositeData. I also revised > ThreadInfoCompositeData to > return the default value of the attributes, if not present. > > This is the patch: > http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/webrev.00/ > > What do you think? > Mandy > > > On 2/20/18 4:46 PM, Jeremy Manson wrote: > > (I dropped serviceability-dev from this thread by mistake! Oops!) > > Okay. Here's the revised patch. LMK if that's what you had in mind. > > http://cr.openjdk.java.net/~jmanson/8198253/webrev.01/ > > On Fri, Feb 16, 2018 at 6:20 PM, mandy chung > wrote: > >> Hi Jeremy, >> >> lockedMonitors and lockedSynchronizers attribute are not optional if that's >> the issue you try to resolve. I think the specification should be clarified. >> >> ThreadInfo::from supports the three different versions for interoperability: >> 1. CompositeData for JDK 1.5 ThreadInfo with no lockedMonitors and >> lockedSynchronizers attribute >> 2. CompositeData for JDK 6 ThreadInfo with lockedMonitors and lockedSynchronizers >> attributes >> 3. CompositeData for JDK 9 ThreadInfo with lockedMonitors and lockedSynchronizers >> attributes and with daemon and priority attribute. >> >> JMX client can connect to a running VM in any version and get back a >> proper ThreadInfo. >> >> If ThreadInfo::from is called with a CompositeData containing daemon and >> priority attribute but lockedMonitors and lockedSynchronizers attributes are >> absent then the given CompositeData is invalid and IllegalArgumentException >> should be thrown. >> >> Does this help? >> >> Mandy >> >> >> On 2/15/18 4:46 PM, Jeremy Manson wrote: >> >> Hi folks, >> >> Been a long time! I'd like someone to do a code review. This is in code >> I wrote a few years ago, and got wrong. At the time, David Holmes, Staffan >> Larsen, and Mandy Chung reviewed it. It does mean that people using >> ThreadInfo.from(CompositeData) may be getting the wrong values out for >> ThreadInfo, so it is definitely worth fixing. >> >> The patch below fixes the bug, but is a pretty questionable approach. >> Better approaches happily considered. >> >> Patch: >> http://cr.openjdk.java.net/~jmanson/8198253/webrev.00/ >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8198253 >> >> Thanks! >> >> Jeremy >> >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From harsha.wardhana.b at oracle.com Wed Feb 21 05:55:58 2018 From: harsha.wardhana.b at oracle.com (Harsha Wardhana B) Date: Wed, 21 Feb 2018 11:25:58 +0530 Subject: RFR: JDK-8187498: Add a -Xmanagement flag as syntactic sugar for -Dcom.sun.management.jmxremote.* properties In-Reply-To: References: <0fca13c3-1b8f-947c-9041-bdea87b04ba8@oracle.com> <5A687F75.3050404@oracle.com> <05c2e5af-b9d0-0386-4565-ade1179a1d49@oracle.com> <7a18b9d5-3a59-8132-2c4f-ba5de35bfa1d@oracle.com> <0e019151-ef48-b4f4-d253-b9da62aa07c3@oracle.com> <248276b1-1589-561f-0718-7bcb1fd578c7@oracle.com> <40cf194e-4d74-3a34-5eb2-1acc5a5abafb@oracle.com> <87cb309b-8764-193a-0447-8ecb741d308d@oracle.com> <66ad342e-f41f-c01b-e515-8100d108e6ef@oracle.com> <0a13fe7c-b7b0-e5d9-1361-bfea19073574@oracle.com> <0332f3ad-1329-da9a-c6a7-192064fb04bb@oracle.com> <9ac62f66-c14d-951f-1a14-ccf8906ecab4@oracle.com> <5A8C33B6.4040206@oracle.com> Message-ID: <7bf813ed-9390-f9e1-b5df-32a4585c2be3@oracle.com> On Wednesday 21 February 2018 01:51 AM, mandy chung wrote: > The code review and CSR review can be in parallel. For this case, > I agree with Kumar to have CSR written that would help the code > review. Please specify the behavior and its relationship with > jcmd and other relevant diagnosability tools. ok. > > On 2/20/18 6:41 AM, Kumar Srinivasan wrote: >> >>>>> What is the behavior when -Dcom.sun.management.jmxremote.port=1234 >>>>> --start-management-agent port=2345 >>>>> -Dcom.sun.management.jmxremote.port=3456? >>>>> >>>>> What is the value of the system property >>>>> com.sun.management.jmxremote.port at runtime?? What port number >>>>> does the management server start with? >>>> As said earlier, values set via new flags override values set by -D >>>> flags. So 2345 will be the value of >>>> com.sun.management.jmxremote.port. Added a test case to validate that. > > VM options are the last one wins if same option specified multiple > times.? In this case, it could cause confusion (the last -D option > sets the value to 3456 but it's set to a different value). > > Why not taking the simplest approach - when --start-management-agent > is set, it does not accept mixing the old way (i.e. does not accept > the management properties to be set via -D)??? This RFE is to make the > command-line simpler and ease-of-use.? I don't see any downside to > migrate entirely to the new form. We cannot get rid of specifying options via -D. We have plenty of -D flags but very few have short-hand alternative via --start-management-agent. If management properties are specified by --start-management-agent, the options specified by -D are anyway overwritten if specified. > > Mandy Harsha From serguei.spitsyn at oracle.com Wed Feb 21 06:16:53 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 20 Feb 2018 22:16:53 -0800 Subject: RFR 8170541: serviceability/jdwp/AllModulesCommandTest.java fails intermittently on Windows and Solaris In-Reply-To: <4ba6d0b1-0277-3e5e-1aa9-49e1db636586@oracle.com> References: <86bf7cb3-50a2-ddeb-0f88-73b8b2b4b2cf@oracle.com> <18D0CADB-CDB9-4F5D-B77A-DE5A5F6A5C19@oracle.com> <4ba6d0b1-0277-3e5e-1aa9-49e1db636586@oracle.com> Message-ID: Hi David, On 2/20/18 20:02, David Holmes wrote: > Hi Daniil, > > Good find on this! > > What does the actual spec say about the length of things and how they > may be split across multiple packets? Are we guaranteed that at most > two packets will be involved? What about for other things eg: > > ?68???? protected byte[] readJdwpString(DataInputStream ds) throws > IOException { > ? 69???????? byte[] str = null; > ? 70???????? int len = ds.readInt(); > ? 71???????? if (len > 0) { > ? 72???????????? str = new byte[len]; > ? 73???????????? ds.read(str, 0, len); > ? 74???????? } > > might we get a short-read of the string if it is split across multiple > packets? Nice catch! Even though this fix is enough to resolve this problem now, there is a chance, it can fail in the future when more modules are added to the platform. > I'm wondering if all these reads should be loops, ensuring we read the > expected amount of data. > > One further comment - not sure why we need the print out for when we > do read multiple packets? > That would seem to be a debugging aid. Yes, it helps to understand what happens. Many tests have a lack of tracing which makes it harder to debug and understand failures. Thanks, Serguei > > Thanks, > David > > On 21/02/2018 10:14 AM, Daniil Titov wrote: >> Hi Serguei, >> >> A new version of the webrev that has these strings reformatted is at >> http://cr.openjdk.java.net/~dtitov/8170541/webrev.02/ >> >> Thank you! >> >> Best regards, >> >> Daniil >> >> *From: *"serguei.spitsyn at oracle.com" >> *Date: *Tuesday, February 20, 2018 at 3:00 PM >> *To: *Daniil Titov , >> "serviceability-dev at openjdk.java.net" >> >> *Subject: *Re: RFR 8170541: >> serviceability/jdwp/AllModulesCommandTest.java fails intermittently >> on Windows and Solaris >> >> Hi Daniil, >> >> Interesting issue... >> Thank you for finding to the root cause so quickly! >> >> The fix looks good. >> Could I ask you to reformat these lines to make the L54 shorter ?: >> >> ?? 54???????????????? System.out.println("[" + getClass().getName() + >> "] Only " + bytesRead + " bytes of " + dataLength + >> >> ?? 55???????????????????????? " were read in the first packet. >> Reading the rest..."); >> >> Thanks, >> Serguei >> >> >> On 2/20/18 09:24, Daniil Titov wrote: >> >> ??? Please review the changes that fix intermittent failure of >> ??? serviceability/jdwp/AllModulesCommandTest.java test. >> >> ??? The problem here is that for a large data the JDWP agent >> ??? (socketTransport_writePacket() method in >> src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c ) >> ??? sends 2 packets and in some cases only the first packet is received >> ??? at the time when the test reads the reply from the JDWP agent. Since >> ??? the test does not check that all data is received in the first >> ??? packet the correlation between commands and replies became broken >> ??? (the unread second packet is read by the next command and the reply >> ??? for the next command is read by the next after next command and >> so on). >> >> ??? Bug: https://bugs.openjdk.java.net/browse/JDK-8170541 >> >> ??? Webrev: http://cr.openjdk.java.net/~dtitov/8170541/webrev.01 >> >> ??? The tests ran successfully with Mach5. >> >> ??? Best regards, >> >> ??? Daniil >> >> >> From serguei.spitsyn at oracle.com Wed Feb 21 06:50:33 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 20 Feb 2018 22:50:33 -0800 Subject: JVMTI retransformation and addition of private methods In-Reply-To: <6ddd571e-11a9-aa64-5ba6-dd98df9a792a@oracle.com> References: <386B2730-5732-4937-B334-411D8E458022@oracle.com> <6ddd571e-11a9-aa64-5ba6-dd98df9a792a@oracle.com> Message-ID: <2bd8bf1c-0328-9604-2661-634f8b1a59f3@oracle.com> An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Wed Feb 21 07:01:15 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 21 Feb 2018 17:01:15 +1000 Subject: JVMTI retransformation and addition of private methods In-Reply-To: <2bd8bf1c-0328-9604-2661-634f8b1a59f3@oracle.com> References: <386B2730-5732-4937-B334-411D8E458022@oracle.com> <6ddd571e-11a9-aa64-5ba6-dd98df9a792a@oracle.com> <2bd8bf1c-0328-9604-2661-634f8b1a59f3@oracle.com> Message-ID: On 21/02/2018 4:50 PM, serguei.spitsyn at oracle.com wrote: > Hi Karen and David, > > > On 2/20/18 19:52, David Holmes wrote: >> Hi Karen, >> >> On 21/02/2018 1:54 AM, Karen Kinnear wrote: >>> Folks, >>> >>> As part of the Valhalla EG discussions for JVMTI changes for >>> nestmates (thank you Serguei and David), >>> IBM brought up a request that we update the JVMTI documentation to >>> reflect that we allow addition >>> of private methods. >>> >>> Is there a reason we do not document this? I?m inviting those who >>> were involved at the time - please include >>> others if needed. > > I support documenting this in the JVMTI spec and had a plan to fix it in 11. > However, it is not clear to me yet if we have a consensus on it. I would like to see a detailed analysis of the implications of allowing this. I _think_ it is safe but ... >> This issue is tracked by: >> >> https://bugs.openjdk.java.net/browse/JDK-8192936 >> >> "RI does not follow the JVMTI RedefineClasses spec that is too strict >> in the definition" > > Yes, this is the one. > Thank you, David, for posting the link. > > >> As I wrote there ... It is not at all clear how JDK-6404550 morphed >> into "Permit the adding or deleting of private final/static methods >> with redefine" - nor why those changes failed to make any change to >> the spec itself. It is also unclear whether the add/delete is >> restricted to final/static methods or any private method? I can see >> that the intent was to only allow something that would not perturb the >> vtable for existing instances. > > I agree, there is a confusion somewhere. > Is it possible, the JDK-6404550 in JIRA is a different bug than the one > in the Bugtraq system? > > The JDK-6404550 in JIRA has a different synopsis: > https://bugs.openjdk.java.net/browse/JDK-6404550 > ???? Cannot implement late attach in NetBeans Profiler due to missing > functionality in JVMTI Digging deeper ... to fix the problem described in that bug they augmented JVM TI to allow private method redefinition as an alternate to the "native rebinding" technique that had been used previously. See the final comment in: https://bugs.openjdk.java.net/browse/JDK-6341303 "JVMTI Spec: Need a way how to rebind Object.wait and Thread.sleep with late attach" which was closed as a duplicate. David ----- > > Thanks, > Serguei > > >> -- >> David >> >> >>> thanks, >>> Karen >>> > From serguei.spitsyn at ORACLE.COM Wed Feb 21 07:45:56 2018 From: serguei.spitsyn at ORACLE.COM (serguei.spitsyn@oracle.com) Date: Tue, 20 Feb 2018 23:45:56 -0800 Subject: JVMTI retransformation and addition of private methods In-Reply-To: References: <386B2730-5732-4937-B334-411D8E458022@oracle.com> <6ddd571e-11a9-aa64-5ba6-dd98df9a792a@oracle.com> <2bd8bf1c-0328-9604-2661-634f8b1a59f3@oracle.com> Message-ID: <43f6c290-fe2f-f7e2-6e83-21cb11fbf0fe@oracle.com> On 2/20/18 23:01, David Holmes wrote: > On 21/02/2018 4:50 PM, serguei.spitsyn at oracle.com wrote: >> Hi Karen and David, >> >> >> On 2/20/18 19:52, David Holmes wrote: >>> Hi Karen, >>> >>> On 21/02/2018 1:54 AM, Karen Kinnear wrote: >>>> Folks, >>>> >>>> As part of the Valhalla EG discussions for JVMTI changes for >>>> nestmates (thank you Serguei and David), >>>> IBM brought up a request that we update the JVMTI documentation to >>>> reflect that we allow addition >>>> of private methods. >>>> >>>> Is there a reason we do not document this? I?m inviting those who >>>> were involved at the time - please include >>>> others if needed. >> >> I support documenting this in the JVMTI spec and had a plan to fix it >> in 11. >> However, it is not clear to me yet if we have a consensus on it. > > I would like to see a detailed analysis of the implications of > allowing this. I _think_ it is safe but ... Valid concern. Also, I'd love to collect more details on the initial motivation to relax the JVMTI spec. Most likely we had no CCC/CSR filed on this change. >>> This issue is tracked by: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8192936 >>> >>> "RI does not follow the JVMTI RedefineClasses spec that is too >>> strict in the definition" >> >> Yes, this is the one. >> Thank you, David, for posting the link. >> >> >>> As I wrote there ... It is not at all clear how JDK-6404550 morphed >>> into "Permit the adding or deleting of private final/static methods >>> with redefine" - nor why those changes failed to make any change to >>> the spec itself. It is also unclear whether the add/delete is >>> restricted to final/static methods or any private method? I can see >>> that the intent was to only allow something that would not perturb >>> the vtable for existing instances. >> >> I agree, there is a confusion somewhere. >> Is it possible, the JDK-6404550 in JIRA is a different bug than the >> one in the Bugtraq system? >> >> The JDK-6404550 in JIRA has a different synopsis: >> https://bugs.openjdk.java.net/browse/JDK-6404550 >> ????? Cannot implement late attach in NetBeans Profiler due to >> missing functionality in JVMTI > > Digging deeper ... to fix the problem described in that bug they > augmented JVM TI to allow private method redefinition as an alternate > to the "native rebinding" technique that had been used previously. See > the final comment in: > > https://bugs.openjdk.java.net/browse/JDK-6341303 > > "JVMTI Spec: Need a way how to rebind Object.wait and Thread.sleep > with late attach" > > which was closed as a duplicate. Thank you for the point. This explains it. It seems, the bug synopsis was changed at some moment. Thanks, Serguei > > David > ----- > >> >> Thanks, >> Serguei >> >> >>> -- >>> David >>> >>> >>>> thanks, >>>> Karen >>>> >> From yasuenag at gmail.com Wed Feb 21 12:14:31 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Wed, 21 Feb 2018 21:14:31 +0900 Subject: PING: RFR: JDK-8153333: [REDO] STW phases at Concurrent GC should count in PerfCounter In-Reply-To: References: <2f4e0901-1602-6276-e7fd-84e168e7b317@gmail.com> <3210723e-c5f9-4277-a97a-be61e10e6b3d@oracle.com> <89a1a161-874d-4743-e52b-7ab202fd8976@gmail.com> <9799a2ad-6727-cae9-312b-fd9de13c8203@oracle.com> Message-ID: <9183dfd3-522f-fd73-a681-5ea957f1d717@gmail.com> PING: Could you review it? > http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.07/ JBS: https://bugs.openjdk.java.net/browse/JDK-8153333 CSR: https://bugs.openjdk.java.net/browse/JDK-8196862 Yasumasa On 2018/02/15 10:23, Yasumasa Suenaga wrote: > Hi all, > > CSR for this issue [1] has been approved. > This webrev has been reviewed by Stefan, but we need one more > reviewer. Could you review it? > > http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.07/ > > > Thanks, > > Yasumasa > > > [1] https://bugs.openjdk.java.net/browse/JDK-8196862 > > > > 2018-02-06 22:33 GMT+09:00 Yasumasa Suenaga : >> Hi Stefan, >> >>> This looks good to me, will do some more testing while waiting for a >>> second reviewer and I can sponsor the change once it's ready to go. >> >> >> Thanks! I'm waiting for second reviewer. >> >>>> What should I do to get CSR approve? >>> >>> In the bug system under "More" you can choose "Create CSR" which is the >>> first step. More information can be found on the wiki: >>> https://wiki.openjdk.java.net/display/csr/CSR+FAQs >> >> >> I filed new CSR: >> https://bugs.openjdk.java.net/browse/JDK-8196862 >> >> >> Yasumasa >> >> >> >> On 2018/02/06 21:55, Stefan Johansson wrote: >>> >>> >>> >>> On 2018-02-06 06:10, Yasumasa Suenaga wrote: >>>> >>>> Hi Stefan, >>>> >>>>> I agree, for G1 this should not be controlled. Maybe I was a bit >>>>> unclear, I >>>>> was wondering why we want to control it for CMS. >>>> >>>> I said to remove -XX:EnableConcGCPerfCounter in two years ago. I've >>>> missed it :-) >>>> >>>> >>>> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-March/017125.html >>>> >>>> So I uploaded new webrev. This change includes copyright year updates. >>>> >>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.06/ >>> >>> Thanks Yasumasa, >>> >>> This looks good to me, will do some more testing while waiting for a >>> second reviewer and I can sponsor the change once it's ready to go. >>>> >>>> >>>> This change passes all tests on submit repo, and >>>> :hotspot_serviceability :jdk_tools tests on my laptop. >>>> >>>> >>>> http://java.se.oracle.com:10065/mdash/jobs/mach5-one-ysuenaga-JDK-8153333-20180206-0222-10428 >>>> >>>> >>>>> If we do the change for CMS, we should >>>>> probably also do a CSR, but that should be fairly straight forward. >>>> >>>> What should I do to get CSR approve? >>> >>> In the bug system under "More" you can choose "Create CSR" which is the >>> first step. More information can be found on the wiki: >>> https://wiki.openjdk.java.net/display/csr/CSR+FAQs >>> >>> Cheers, >>> Stefan >>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> 2018-02-06 0:33 GMT+09:00 Stefan Johansson : >>>>> >>>>> >>>>> On 2018-02-03 06:40, Yasumasa Suenaga wrote: >>>>>> >>>>>> On 2018/02/02 23:38, Stefan Johansson wrote: >>>>>>> >>>>>>> Hi Yasumasa, >>>>>>> >>>>>>> The changes doesn't apply clean on the latest jdk/hs, can you provide >>>>>>> an >>>>>>> updated webrev? >>>>>> >>>>>> >>>>>> I uploaded webrev for jdk-hs: >>>>>> cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.05/ >>>>>> >>>>> Thanks, I've kicked off a testing job now to verify nothing unexpected >>>>> fails. >>>>>> >>>>>> >>>>>>> The testing done by the submit repo doesn't cover the tests you have >>>>>>> update so I plan to take the change for a spin and make sure the >>>>>>> correct >>>>>>> tests are run and verified in Mach 5. >>>>>> >>>>>> >>>>>> I've also tested hotspot/jtreg/:hotspot_serviceability and >>>>>> jdk/:jdk_tools >>>>>> on my laptop. >>>>>> I did not see any errors / failures which are related to this change. >>>>> >>>>> I also ran some local tests on this and it looks good. >>>>>> >>>>>> >>>>>> >>>>>>> Also a question about the change. Why do we need a special flag for >>>>>>> CMS? >>>>>>> I see that the original bug report refers to the flag as being a way >>>>>>> to turn >>>>>>> on and off the feature but the current implementation only consider >>>>>>> the flag >>>>>>> for CMS. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-March/016774.html >>>>>> >>>>>> Originally, STW phases (Remark and Cleanup) at G1 are not counted in >>>>>> jstat >>>>>> FGC column. >>>>>> So I think we need not to control the behavior of PerfCounter for G1. >>>>>> >>>>> I agree, for G1 this should not be controlled. Maybe I was a bit >>>>> unclear, I >>>>> was wondering why we want to control it for CMS. I think either we >>>>> should >>>>> change the behavior without guarding it by a flag or just skip updating >>>>> CMS >>>>> (and leave the pauses in FGC). If we do the change for CMS, we should >>>>> probably also do a CSR, but that should be fairly straight forward. >>>>> >>>>> I also found the old review thread where Jon M had the same comment >>>>> (removing the flag) and it looks like all agreed on that: >>>>> >>>>> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2016-March/017118.html >>>>> >>>>> Thanks, >>>>> Stefan >>>>> >>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>>> Thanks, >>>>>>> Stefan >>>>>>> >>>>>>> On 2018-02-01 14:58, Yasumasa Suenaga wrote: >>>>>>>> >>>>>>>> PING: Could you review and sponsor it? >>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>>>>>>> >>>>>>>> >>>>>>>> This change has been passed Mach 5 via submit repo: >>>>>>>> >>>>>>>> >>>>>>>> http://java.se.oracle.com:10065/mdash/jobs/mach5-one-ysuenaga-JDK-8153333-20180201-0805-10101 >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2017/11/01 22:02, Yasumasa Suenaga wrote: >>>>>>>>> >>>>>>>>> PING: Could you review and sponsor it? >>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>>>>>>>> >>>>>>>>> >>>>>>>>> Also I need JPRT results of this change. >>>>>>>>> Could you cooperate? >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2017/09/27 0:08, Yasumasa Suenaga wrote: >>>>>>>>>> >>>>>>>>>> Hi all, >>>>>>>>>> >>>>>>>>>> I uploaded new webrev to be adapted to jdk10/hs: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.04/ >>>>>>>>>> >>>>>>>>>> I want to check this patch via JPRT, but I cannot access it. >>>>>>>>>> Could you cooperate? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2017/09/21 7:46, Yasumasa Suenaga wrote: >>>>>>>>>>> >>>>>>>>>>> PING: >>>>>>>>>>> >>>>>>>>>>> Have you checked this issue? >>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/hotspot/ >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/jdk/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2017/07/01 23:44, Yasumasa Suenaga wrote: >>>>>>>>>>>> >>>>>>>>>>>> PING: >>>>>>>>>>>> >>>>>>>>>>>> Have you checked this issue? >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 2017/06/14 13:22, Yasumasa Suenaga wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Hi all, >>>>>>>>>>>>> >>>>>>>>>>>>> I changed PerfCounter to show CGC STW phase in jstat in >>>>>>>>>>>>> JDK-8151674. >>>>>>>>>>>>> However, it occurred several jtreg test failure, so it was >>>>>>>>>>>>> back-outed. >>>>>>>>>>>>> >>>>>>>>>>>>> I want to resume to work for this issue. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/hotspot/ >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8153333/webrev.03/jdk/ >>>>>>>>>>>>> >>>>>>>>>>>>> These changes are work fine on jtreg test as below: >>>>>>>>>>>>> >>>>>>>>>>>>> hotspot/test/serviceability/tmtools/jstat >>>>>>>>>>>>> jdk/test/sun/tools >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Since JDK 9, default GC algorithm is set to G1. >>>>>>>>>>>>> So I think this change is useful to watch GC behavior through >>>>>>>>>>>>> jstat. >>>>>>>>>>>>> >>>>>>>>>>>>> I cannot access JPRT. Could you help? >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> >>> >> From Roger.Riggs at Oracle.com Wed Feb 21 15:03:40 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 21 Feb 2018 10:03:40 -0500 Subject: RFR: JDK-8187498: Add a -Xmanagement flag as syntactic sugar for -Dcom.sun.management.jmxremote.* properties In-Reply-To: <7bf813ed-9390-f9e1-b5df-32a4585c2be3@oracle.com> References: <0fca13c3-1b8f-947c-9041-bdea87b04ba8@oracle.com> <5A687F75.3050404@oracle.com> <05c2e5af-b9d0-0386-4565-ade1179a1d49@oracle.com> <7a18b9d5-3a59-8132-2c4f-ba5de35bfa1d@oracle.com> <0e019151-ef48-b4f4-d253-b9da62aa07c3@oracle.com> <248276b1-1589-561f-0718-7bcb1fd578c7@oracle.com> <40cf194e-4d74-3a34-5eb2-1acc5a5abafb@oracle.com> <87cb309b-8764-193a-0447-8ecb741d308d@oracle.com> <66ad342e-f41f-c01b-e515-8100d108e6ef@oracle.com> <0a13fe7c-b7b0-e5d9-1361-bfea19073574@oracle.com> <0332f3ad-1329-da9a-c6a7-192064fb04bb@oracle.com> <9ac62f66-c14d-951f-1a14-ccf8906ecab4@oracle.com> <5A8C33B6.4040206@oracle.com> <7bf813ed-9390-f9e1-b5df-32a4585c2be3@oracle.com> Message-ID: <31d45eaa-b5cb-7cd7-5311-6eaa70afe914@Oracle.com> Hi, I'm a bit leary of command line arguments being special cased and the corresponding custom mappings to system properties.?? The convenience is fine but we need to keep the handling out of native code so it is easier to maintain.? We don't have a Java API for processing (VM) command line arguments so they are being shoehorned into properties. $.02, Roger On 2/21/2018 12:55 AM, Harsha Wardhana B wrote: > > > On Wednesday 21 February 2018 01:51 AM, mandy chung wrote: >> The code review and CSR review can be in parallel.? For this case, >> I agree with Kumar to have CSR written that would help the code >> review. Please specify the behavior and its relationship with >> jcmd and other relevant diagnosability tools. > ok. >> >> On 2/20/18 6:41 AM, Kumar Srinivasan wrote: >>> >>>>>> What is the behavior when >>>>>> -Dcom.sun.management.jmxremote.port=1234 --start-management-agent >>>>>> port=2345 -Dcom.sun.management.jmxremote.port=3456? >>>>>> >>>>>> What is the value of the system property >>>>>> com.sun.management.jmxremote.port at runtime?? What port number >>>>>> does the management server start with? >>>>> As said earlier, values set via new flags override values set by >>>>> -D flags. So 2345 will be the value of >>>>> com.sun.management.jmxremote.port. Added a test case to validate >>>>> that. >> >> VM options are the last one wins if same option specified multiple >> times.? In this case, it could cause confusion (the last -D option >> sets the value to 3456 but it's set to a different value). >> >> Why not taking the simplest approach - when --start-management-agent >> is set, it does not accept mixing the old way (i.e. does not accept >> the management properties to be set via -D)??? This RFE is to make >> the command-line simpler and ease-of-use.? I don't see any downside >> to migrate entirely to the new form. > We cannot get rid of specifying options via -D. We have plenty of -D > flags but very few have short-hand alternative via > --start-management-agent. If management properties are specified by > --start-management-agent, the options specified by -D are anyway > overwritten if specified. >> >> Mandy > Harsha From andrew_m_leonard at uk.ibm.com Wed Feb 21 15:29:02 2018 From: andrew_m_leonard at uk.ibm.com (Andrew Leonard) Date: Wed, 21 Feb 2018 15:29:02 +0000 Subject: Remove unused import proposal: in JdpController.java Message-ID: Hi, I would like to find a sponsor please for this simple tidy up of JdpController.java ? It contains an unused "import sun.management.VMManagement;" which is for a sun specific java.management class which this class used to use prior to jdk9. Thanks Andrew diff --git a/src/jdk.management.agent/share/classes/sun/management/jdp/JdpController.java b/src/jdk.management.agent/share/classes/sun/management/jdp/JdpController.java --- a/src/jdk.management.agent/share/classes/sun/management/jdp/JdpController.java +++ b/src/jdk.management.agent/share/classes/sun/management/jdp/JdpController.java @@ -34,7 +34,6 @@ import java.lang.reflect.Field; import java.lang.reflect.Method; import java.lang.UnsupportedOperationException; -import sun.management.VMManagement; /** * JdpController is responsible to create and manage a broadcast loop. Andrew Leonard Java Runtimes Development IBM Hursley IBM United Kingdom Ltd Phone internal: 245913, external: 01962 815913 internet email: andrew_m_leonard at uk.ibm.com Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.daugherty at oracle.com Wed Feb 21 15:40:42 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 21 Feb 2018 10:40:42 -0500 Subject: JVMTI retransformation and addition of private methods In-Reply-To: <43f6c290-fe2f-f7e2-6e83-21cb11fbf0fe@oracle.com> References: <386B2730-5732-4937-B334-411D8E458022@oracle.com> <6ddd571e-11a9-aa64-5ba6-dd98df9a792a@oracle.com> <2bd8bf1c-0328-9604-2661-634f8b1a59f3@oracle.com> <43f6c290-fe2f-f7e2-6e83-21cb11fbf0fe@oracle.com> Message-ID: On 2/21/18 2:45 AM, serguei.spitsyn at oracle.com wrote: > On 2/20/18 23:01, David Holmes wrote: >> On 21/02/2018 4:50 PM, serguei.spitsyn at oracle.com wrote: >>> Hi Karen and David, >>> >>> >>> On 2/20/18 19:52, David Holmes wrote: >>>> Hi Karen, >>>> >>>> On 21/02/2018 1:54 AM, Karen Kinnear wrote: >>>>> Folks, >>>>> >>>>> As part of the Valhalla EG discussions for JVMTI changes for >>>>> nestmates (thank you Serguei and David), >>>>> IBM brought up a request that we update the JVMTI documentation to >>>>> reflect that we allow addition >>>>> of private methods. >>>>> >>>>> Is there a reason we do not document this? I?m inviting those who >>>>> were involved at the time - please include >>>>> others if needed. >>> >>> I support documenting this in the JVMTI spec and had a plan to fix >>> it in 11. >>> However, it is not clear to me yet if we have a consensus on it. >> >> I would like to see a detailed analysis of the implications of >> allowing this. I _think_ it is safe but ... > > Valid concern. > Also, I'd love to collect more details on the initial motivation to > relax the JVMTI spec. > Most likely we had no CCC/CSR filed on this change. > > >>>> This issue is tracked by: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8192936 >>>> >>>> "RI does not follow the JVMTI RedefineClasses spec that is too >>>> strict in the definition" >>> >>> Yes, this is the one. >>> Thank you, David, for posting the link. >>> >>> >>>> As I wrote there ... It is not at all clear how JDK-6404550 morphed >>>> into "Permit the adding or deleting of private final/static methods >>>> with redefine" - nor why those changes failed to make any change to >>>> the spec itself. It is also unclear whether the add/delete is >>>> restricted to final/static methods or any private method? I can see >>>> that the intent was to only allow something that would not perturb >>>> the vtable for existing instances. >>> >>> I agree, there is a confusion somewhere. >>> Is it possible, the JDK-6404550 in JIRA is a different bug than the >>> one in the Bugtraq system? >>> >>> The JDK-6404550 in JIRA has a different synopsis: >>> https://bugs.openjdk.java.net/browse/JDK-6404550 >>> ????? Cannot implement late attach in NetBeans Profiler due to >>> missing functionality in JVMTI >> >> Digging deeper ... to fix the problem described in that bug they >> augmented JVM TI to allow private method redefinition as an alternate >> to the "native rebinding" technique that had been used previously. >> See the final comment in: >> >> https://bugs.openjdk.java.net/browse/JDK-6341303 >> >> "JVMTI Spec: Need a way how to rebind Object.wait and Thread.sleep >> with late attach" >> >> which was closed as a duplicate. > > Thank you for the point. > This explains it. > It seems, the bug synopsis was changed at some moment. The synopsis for 6404550 has never changed. Here's the subject line when it was created on 2006.03.27: > CR 6404550 *HOT* Created P1 hotspot/jvmti Cannot implement late attach in NetBeans Profiler due to missing functionality in JVMTI I think the confusion arises over comments like this in 6341303: > rfield Robert Field > > added a comment - 2006-05-04 11:54 > BT2:EVALUATION > > This can now be accomplished with Java programming language > instrumentation, via: > > ?????6404550: missing late attach (JVM TI redefine) functionality > ???????????Permit the adding or deleting of private final/static > methods with redefine > > Closing this bug as a duplicate. That's just Robert's style for an sccs delta comment: D 1.65.2.3 06/04/25 23:36:35 rfield 140 139???? 00023/00013/03263 MRs: COMMENTS: 6404550: missing late attach (JVM TI redefine) functionality ??????????? Add/delete private methods, continued: changes per review Back in the ancient past we tried to include some brief info about the change in the delta comment. This was one of many deltas associated with 6404550. Please see the attached email that I sent on 2012.12.17 about the history behind this issue... (sent to Karen, Mikael V, and Serguei) It seems I forwarded that same email to Coleen, Markus G and Serguei back on 2014.05.20. Since Markus is on that thread, it must have had something to do with research about JFR... I need to do a detailed read thru my e-mail archive for 6404550 to see if I can spot some clues about why we didn't do a spec update. Dan > > Thanks, > Serguei > >> >> David >> ----- >> >>> >>> Thanks, >>> Serguei >>> >>> >>>> -- >>>> David >>>> >>>> >>>>> thanks, >>>>> Karen >>>>> >>> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An embedded message was scrubbed... From: "Daniel D. Daugherty" Subject: Re: adding/deleting methods in JVM/TI RedefineClasses()/RetransformClasses() Date: Mon, 17 Dec 2012 13:03:14 -0700 Size: 6818 URL: From mandy.chung at oracle.com Wed Feb 21 16:21:23 2018 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 21 Feb 2018 08:21:23 -0800 Subject: RFR: JDK-8187498: Add a -Xmanagement flag as syntactic sugar for -Dcom.sun.management.jmxremote.* properties In-Reply-To: <7bf813ed-9390-f9e1-b5df-32a4585c2be3@oracle.com> References: <0fca13c3-1b8f-947c-9041-bdea87b04ba8@oracle.com> <5A687F75.3050404@oracle.com> <05c2e5af-b9d0-0386-4565-ade1179a1d49@oracle.com> <7a18b9d5-3a59-8132-2c4f-ba5de35bfa1d@oracle.com> <0e019151-ef48-b4f4-d253-b9da62aa07c3@oracle.com> <248276b1-1589-561f-0718-7bcb1fd578c7@oracle.com> <40cf194e-4d74-3a34-5eb2-1acc5a5abafb@oracle.com> <87cb309b-8764-193a-0447-8ecb741d308d@oracle.com> <66ad342e-f41f-c01b-e515-8100d108e6ef@oracle.com> <0a13fe7c-b7b0-e5d9-1361-bfea19073574@oracle.com> <0332f3ad-1329-da9a-c6a7-192064fb04bb@oracle.com> <9ac62f66-c14d-951f-1a14-ccf8906ecab4@oracle.com> <5A8C33B6.4040206@oracle.com> <7bf813ed-9390-f9e1-b5df-32a4585c2be3@oracle.com> Message-ID: On 2/20/18 9:55 PM, Harsha Wardhana B wrote: > > We cannot get rid of specifying options via -D. We have plenty of -D > flags but very few have short-hand alternative via > --start-management-agent. If management properties are specified by > --start-management-agent, the options specified by -D are anyway > overwritten if specified. I have to see the specification of this feature and give further feedback.?? If the new option does not accept all management properties, what is the benefit of this new option? Mandy -------------- next part -------------- An HTML attachment was scrubbed... URL: From mandy.chung at oracle.com Wed Feb 21 16:53:04 2018 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 21 Feb 2018 08:53:04 -0800 Subject: RFR JDK-8198253: ThreadInfo.from(CompositeData) assigning fields incorrectly in JDK 9 In-Reply-To: References: <45a5eee6-023d-e342-d6ed-6797702b8804@oracle.com> <9110e27c-e492-b089-7b02-36459915c779@oracle.com> Message-ID: <715a0bd4-10aa-ee19-1cdf-0a721005b689@oracle.com> On 2/20/18 9:48 PM, Jeremy Manson wrote: > I think that's a much better approach (I didn't notice the validate > method :) ) > > I think you may want to grab my test changes, or make some similar > ones.? Presumably, the tests do not pass with your change. I ran your test and it passed before posting it. > > I think the API wording change is very slightly confusing. > > ? ? * ? ? ? A {@code CompositeData} does not contain this attribute > ? ? ? * ? ? ? when representing a {@code ThreadInfo} of JDK 6 or > older version. > ? ? ? * ? ? ? This attribute will be set to {@link > Thread#NORM_PRIORITY}. > > > I'd probably say "In such cases" at the beginning of the second > sentence.? Also, is there a rule about when you say JDK 6 and when you > say Java 6? Java SE 6 should be the proper terminology be used in this case.? I actually try to see if we can use @since instead.? I will take any pass on the spec and how it can tie with @since in the getter methods. I will look at it later today. Mandy > > Jeremy > > On Tue, Feb 20, 2018 at 6:41 PM, mandy chung > wrote: > > Hi Jeremy, > > Another approach is to change > ThreadInfoCompositeData::validateCompositeData > to validate the given CompositeData.?? I also revised > ThreadInfoCompositeData to > return the default value of the attributes, if not present. > > This is the patch: > http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/webrev.00/ > > > What do you think? > Mandy > > > On 2/20/18 4:46 PM, Jeremy Manson wrote: >> (I dropped serviceability-dev from this thread by mistake!? Oops!) >> >> Okay. Here's the revised patch.? LMK if that's what you had in mind. >> >> http://cr.openjdk.java.net/~jmanson/8198253/webrev.01/ >> >> >> On Fri, Feb 16, 2018 at 6:20 PM, mandy chung >> > wrote: >> >> Hi Jeremy, >> >> lockedMonitors and lockedSynchronizers attribute are not optional if that's >> the issue you try to resolve. I think the specification should be clarified. >> >> ThreadInfo::from supports the three different versions for interoperability: >> 1. CompositeData for JDK 1.5 ThreadInfo with no lockedMonitors and >> lockedSynchronizers attribute >> 2. CompositeData for JDK 6 ThreadInfo with lockedMonitors and lockedSynchronizers >> attributes >> 3. CompositeData for JDK 9 ThreadInfo with lockedMonitors and lockedSynchronizers >> ?attributes and with daemon and priority attribute. >> >> JMX client can connect to a running VM in any version and get back a >> proper ThreadInfo. >> >> If ThreadInfo::from is called with a CompositeData containing daemon and >> priority attribute but lockedMonitors and lockedSynchronizers attributes are >> absent then the given CompositeData is invalid and IllegalArgumentException >> should be thrown. >> >> Does this help? >> >> Mandy >> >> >> On 2/15/18 4:46 PM, Jeremy Manson wrote: >>> Hi folks, >>> >>> Been a long time!? I'd like someone to do a code review.? >>> This is in code I wrote a few years ago, and got wrong.? At >>> the time, David Holmes, Staffan Larsen, and Mandy Chung >>> reviewed it.? It does mean that people using >>> ThreadInfo.from(CompositeData) may be getting the wrong >>> values out for ThreadInfo, so it is definitely worth fixing. >>> >>> The patch below fixes the bug, but is a pretty questionable >>> approach.? Better approaches happily considered. >>> >>> Patch: >>> http://cr.openjdk.java.net/~jmanson/8198253/webrev.00/ >>> >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8198253 >>> >>> >>> Thanks! >>> >>> Jeremy >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrb at google.com Wed Feb 21 17:20:24 2018 From: martinrb at google.com (Martin Buchholz) Date: Wed, 21 Feb 2018 09:20:24 -0800 Subject: Remove unused import proposal: in JdpController.java In-Reply-To: References: Message-ID: I've done my fair share of import cleanup, but friction is high enough that it should be done in bulk. If you can succeed in loading jdk sources into intellij, then you can get intellij to find all the unused imports for you, e.g. in the jdk core libraries. On Wed, Feb 21, 2018 at 7:29 AM, Andrew Leonard wrote: > Hi, > I would like to find a sponsor please for this simple tidy up of > JdpController.java ? > It contains an unused "import sun.management.VMManagement;" which is for a > sun specific java.management class which this class used to use prior to > jdk9. > Thanks > Andrew > > diff --git a/src/jdk.management.agent/share/classes/sun/management/jdp/JdpController.java > b/src/jdk.management.agent/share/classes/sun/management/ > jdp/JdpController.java > --- a/src/jdk.management.agent/share/classes/sun/management/ > jdp/JdpController.java > +++ b/src/jdk.management.agent/share/classes/sun/management/ > jdp/JdpController.java > @@ -34,7 +34,6 @@ > import java.lang.reflect.Field; > import java.lang.reflect.Method; > import java.lang.UnsupportedOperationException; > -import sun.management.VMManagement; > > /** > * JdpController is responsible to create and manage a broadcast loop. > > > > > > Andrew Leonard > Java Runtimes Development > IBM Hursley > IBM United Kingdom Ltd > Phone internal: 245913, external: 01962 815913 > internet email: andrew_m_leonard at uk.ibm.com > > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > -------------- next part -------------- An HTML attachment was scrubbed... URL: From karen.kinnear at oracle.com Wed Feb 21 17:55:19 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Wed, 21 Feb 2018 12:55:19 -0500 Subject: JVMTI retransformation and addition of private methods In-Reply-To: References: <386B2730-5732-4937-B334-411D8E458022@oracle.com> <6ddd571e-11a9-aa64-5ba6-dd98df9a792a@oracle.com> <2bd8bf1c-0328-9604-2661-634f8b1a59f3@oracle.com> <43f6c290-fe2f-f7e2-6e83-21cb11fbf0fe@oracle.com> Message-ID: Dan, Thank you for all the background digging. This is really helpful. Serguei - do you know what tests exist for this behavior? The way I read the source code - we currently allow ADD and DELETE for PRIVATE OR STATIC OR FINAL methods. Did I read that correctly? With the current implementation, I am not sure if deletion works for private methods - do we have a test for that? Or could you add one as part of this exercise? Today we create a vtable entry for private methods (my misunderstanding ~ 2006ish). After discussions with David I no longer believe we need those. Today, klassVtable::adjust_method_entries has an assertion assert(!old_method->is_deleted(), ?vtable methods may not be deleted?) I may have read the code incorrectly - but I would expect to hit this assertion if you had a private method you were deleting that was not final and not static. option 1) we could explicitly tighten the restrictions to match what we have implemented option 2) we could make this work by changing klassVtable.cpp::update_inherited_vtable handling of private fields to be done the way it handles final fields. option 3) I read the code incorrectly? thanks, Karen > On Feb 21, 2018, at 10:40 AM, Daniel D. Daugherty wrote: > > On 2/21/18 2:45 AM, serguei.spitsyn at oracle.com wrote: >> On 2/20/18 23:01, David Holmes wrote: >>> On 21/02/2018 4:50 PM, serguei.spitsyn at oracle.com wrote: >>>> Hi Karen and David, >>>> >>>> >>>> On 2/20/18 19:52, David Holmes wrote: >>>>> Hi Karen, >>>>> >>>>> On 21/02/2018 1:54 AM, Karen Kinnear wrote: >>>>>> Folks, >>>>>> >>>>>> As part of the Valhalla EG discussions for JVMTI changes for nestmates (thank you Serguei and David), >>>>>> IBM brought up a request that we update the JVMTI documentation to reflect that we allow addition >>>>>> of private methods. >>>>>> >>>>>> Is there a reason we do not document this? I?m inviting those who were involved at the time - please include >>>>>> others if needed. >>>> >>>> I support documenting this in the JVMTI spec and had a plan to fix it in 11. >>>> However, it is not clear to me yet if we have a consensus on it. >>> >>> I would like to see a detailed analysis of the implications of allowing this. I _think_ it is safe but ... >> >> Valid concern. >> Also, I'd love to collect more details on the initial motivation to relax the JVMTI spec. >> Most likely we had no CCC/CSR filed on this change. >> >> >>>>> This issue is tracked by: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8192936 >>>>> >>>>> "RI does not follow the JVMTI RedefineClasses spec that is too strict in the definition" >>>> >>>> Yes, this is the one. >>>> Thank you, David, for posting the link. >>>> >>>> >>>>> As I wrote there ... It is not at all clear how JDK-6404550 morphed into "Permit the adding or deleting of private final/static methods with redefine" - nor why those changes failed to make any change to the spec itself. It is also unclear whether the add/delete is restricted to final/static methods or any private method? I can see that the intent was to only allow something that would not perturb the vtable for existing instances. >>>> >>>> I agree, there is a confusion somewhere. >>>> Is it possible, the JDK-6404550 in JIRA is a different bug than the one in the Bugtraq system? >>>> >>>> The JDK-6404550 in JIRA has a different synopsis: >>>> https://bugs.openjdk.java.net/browse/JDK-6404550 >>>> Cannot implement late attach in NetBeans Profiler due to missing functionality in JVMTI >>> >>> Digging deeper ... to fix the problem described in that bug they augmented JVM TI to allow private method redefinition as an alternate to the "native rebinding" technique that had been used previously. See the final comment in: >>> >>> https://bugs.openjdk.java.net/browse/JDK-6341303 >>> >>> "JVMTI Spec: Need a way how to rebind Object.wait and Thread.sleep with late attach" >>> >>> which was closed as a duplicate. >> >> Thank you for the point. >> This explains it. >> It seems, the bug synopsis was changed at some moment. > > The synopsis for 6404550 has never changed. Here's the subject line when > it was created on 2006.03.27: > > > CR 6404550 *HOT* Created P1 hotspot/jvmti Cannot implement late attach in NetBeans Profiler due to missing functionality in JVMTI > > I think the confusion arises over comments like this in 6341303: > >> Robert Field added a comment - 2006-05-04 11:54 >> BT2:EVALUATION >> >> This can now be accomplished with Java programming language instrumentation, via: >> >> 6404550: missing late attach (JVM TI redefine) functionality >> Permit the adding or deleting of private final/static methods with redefine >> >> Closing this bug as a duplicate. > > That's just Robert's style for an sccs delta comment: > > D 1.65.2.3 06/04/25 23:36:35 rfield 140 139 00023/00013/03263 > MRs: > COMMENTS: > 6404550: missing late attach (JVM TI redefine) functionality > Add/delete private methods, continued: changes per review > > Back in the ancient past we tried to include some brief > info about the change in the delta comment. This was one of many > deltas associated with 6404550. > > Please see the attached email that I sent on 2012.12.17 about the > history behind this issue... (sent to Karen, Mikael V, and Serguei) > > It seems I forwarded that same email to Coleen, Markus G and Serguei > back on 2014.05.20. Since Markus is on that thread, it must have had > something to do with research about JFR... > > I need to do a detailed read thru my e-mail archive for 6404550 to > see if I can spot some clues about why we didn't do a spec update. > > Dan > > >> >> Thanks, >> Serguei >> >>> >>> David >>> ----- >>> >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>>> -- >>>>> David >>>>> >>>>> >>>>>> thanks, >>>>>> Karen >>>>>> >>>> >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.daugherty at oracle.com Wed Feb 21 18:02:07 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 21 Feb 2018 13:02:07 -0500 Subject: JVMTI retransformation and addition of private methods In-Reply-To: References: <386B2730-5732-4937-B334-411D8E458022@oracle.com> <6ddd571e-11a9-aa64-5ba6-dd98df9a792a@oracle.com> <2bd8bf1c-0328-9604-2661-634f8b1a59f3@oracle.com> <43f6c290-fe2f-f7e2-6e83-21cb11fbf0fe@oracle.com> Message-ID: <324be7ad-8d8c-4350-7a5a-28cc4dfcc827@oracle.com> On 2/21/18 12:55 PM, Karen Kinnear wrote: > Dan, > > Thank you for all the background digging. This is really helpful. > > Serguei - do you know what tests exist for this behavior? I mentioned a test that I found in the email that I attached to my previous reply: > Subject: > Re: adding/deleting methods in JVM/TI > RedefineClasses()/RetransformClasses() > > From: > "Daniel D. Daugherty" > Date: > 12/17/12, 3:03 PM > > To: > Karen Kinnear > CC: > Mikael Vidstedt , Serguei Spitsyn > > > > Looks like I added a JLI/JPLIS test that exercises this "feature" > by calling the newly added method via reflection: > > ??? $ ls > /work/shared/bug_hunt/jdk8/exp/test/java/lang/instrument/RedefineMethodAdd* > ??? RedefineMethodAddInvoke.sh > ??? RedefineMethodAddInvokeAgent.java > ??? RedefineMethodAddInvokeApp.java > ??? RedefineMethodAddInvokeTarget.java > ??? RedefineMethodAddInvokeTarget_1.java > ??? RedefineMethodAddInvokeTarget_2.java > > This test was added for the following bug: > > ??? 6667089 3/3 multiple redefinitions of a class break reflection > > All these little pieces of information keep coming back at me... > > Dan I haven't checked to see if that test is still there or if any additional tests have been added. Dan > > The way I read the source code - we currently allow ADD and DELETE for > PRIVATE OR STATIC OR FINAL methods. Did I read that correctly? > > With the current implementation, I am not sure if deletion works for > private methods - do we > have a test for that? Or could you add one as part of this exercise? > > Today we create a vtable entry for private methods (my > misunderstanding ~ 2006ish). After discussions > with David I no longer believe we need those. > Today, klassVtable::adjust_method_entries has an assertion > ? assert(!old_method->is_deleted(), ?vtable methods may not be deleted?) > > I may have read the code incorrectly - but I would expect to hit this > assertion if you had a private > method you were deleting that was not final and not static. > > option 1) we could explicitly tighten the restrictions to match what > we have implemented > option 2) we could make this work by changing > klassVtable.cpp::update_inherited_vtable > ? handling of private fields to be done the way it handles final fields. > option 3) I read the code incorrectly? > > thanks, > Karen > >> On Feb 21, 2018, at 10:40 AM, Daniel D. Daugherty >> > wrote: >> >> On 2/21/18 2:45 AM, serguei.spitsyn at oracle.com wrote: >>> On 2/20/18 23:01, David Holmes wrote: >>>> On 21/02/2018 4:50 PM, serguei.spitsyn at oracle.com wrote: >>>>> Hi Karen and David, >>>>> >>>>> >>>>> On 2/20/18 19:52, David Holmes wrote: >>>>>> Hi Karen, >>>>>> >>>>>> On 21/02/2018 1:54 AM, Karen Kinnear wrote: >>>>>>> Folks, >>>>>>> >>>>>>> As part of the Valhalla EG discussions for JVMTI changes for >>>>>>> nestmates (thank you Serguei and David), >>>>>>> IBM brought up a request that we update the JVMTI documentation >>>>>>> to reflect that we allow addition >>>>>>> of private methods. >>>>>>> >>>>>>> Is there a reason we do not document this? I?m inviting those >>>>>>> who were involved at the time - please include >>>>>>> others if needed. >>>>> >>>>> I support documenting this in the JVMTI spec and had a plan to fix >>>>> it in 11. >>>>> However, it is not clear to me yet if we have a consensus on it. >>>> >>>> I would like to see a detailed analysis of the implications of >>>> allowing this. I _think_ it is safe but ... >>> >>> Valid concern. >>> Also, I'd love to collect more details on the initial motivation to >>> relax the JVMTI spec. >>> Most likely we had no CCC/CSR filed on this change. >>> >>> >>>>>> This issue is tracked by: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8192936 >>>>>> >>>>>> "RI does not follow the JVMTI RedefineClasses spec that is too >>>>>> strict in the definition" >>>>> >>>>> Yes, this is the one. >>>>> Thank you, David, for posting the link. >>>>> >>>>> >>>>>> As I wrote there ... It is not at all clear how JDK-6404550 >>>>>> morphed into "Permit the adding or deleting of private >>>>>> final/static methods with redefine" - nor why those changes >>>>>> failed to make any change to the spec itself. It is also unclear >>>>>> whether the add/delete is restricted to final/static methods or >>>>>> any private method? I can see that the intent was to only allow >>>>>> something that would not perturb the vtable for existing instances. >>>>> >>>>> I agree, there is a confusion somewhere. >>>>> Is it possible, the JDK-6404550 in JIRA is a different bug than >>>>> the one in the Bugtraq system? >>>>> >>>>> The JDK-6404550 in JIRA has a different synopsis: >>>>> https://bugs.openjdk.java.net/browse/JDK-6404550 >>>>> ????? Cannot implement late attach in NetBeans Profiler due to >>>>> missing functionality in JVMTI >>>> >>>> Digging deeper ... to fix the problem described in that bug they >>>> augmented JVM TI to allow private method redefinition as an >>>> alternate to the "native rebinding" technique that had been used >>>> previously. See the final comment in: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-6341303 >>>> >>>> "JVMTI Spec: Need a way how to rebind Object.wait and Thread.sleep >>>> with late attach" >>>> >>>> which was closed as a duplicate. >>> >>> Thank you for the point. >>> This explains it. >>> It seems, the bug synopsis was changed at some moment. >> >> The synopsis for 6404550 has never changed. Here's the subject line when >> it was created on 2006.03.27: >> >> > CR 6404550 *HOT* Created P1 hotspot/jvmti Cannot implement late >> attach in NetBeans Profiler due to missing functionality in JVMTI >> >> I think the confusion arises over comments like this in 6341303: >> >>> rfield Robert Field >>> >>> added a comment - 2006-05-04 11:54 >>> BT2:EVALUATION >>> >>> This can now be accomplished with Java programming language >>> instrumentation, via: >>> >>> ?????6404550: missing late attach (JVM TI redefine) functionality >>> ???????????Permit the adding or deleting of private final/static >>> methods with redefine >>> >>> Closing this bug as a duplicate. >> >> That's just Robert's style for an sccs delta comment: >> >> D 1.65.2.3 06/04/25 23:36:35 rfield 140 139 00023/00013/03263 >> MRs: >> COMMENTS: >> 6404550: missing late attach (JVM TI redefine) functionality >> ??????????? Add/delete private methods, continued: changes per review >> >> Back in the ancient past we tried to include some brief >> info about the change in the delta comment. This was one of many >> deltas associated with 6404550. >> >> Please see the attached email that I sent on 2012.12.17 about the >> history behind this issue... (sent to Karen, Mikael V, and Serguei) >> >> It seems I forwarded that same email to Coleen, Markus G and Serguei >> back on 2014.05.20. Since Markus is on that thread, it must have had >> something to do with research about JFR... >> >> I need to do a detailed read thru my e-mail archive for 6404550 to >> see if I can spot some clues about why we didn't do a spec update. >> >> Dan >> >> >>> >>> Thanks, >>> Serguei >>> >>>> >>>> David >>>> ----- >>>> >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>>> -- >>>>>> David >>>>>> >>>>>> >>>>>>> thanks, >>>>>>> Karen >>>>>>> >>>>> >>> >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Wed Feb 21 20:08:03 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 21 Feb 2018 12:08:03 -0800 Subject: JVMTI retransformation and addition of private methods In-Reply-To: <324be7ad-8d8c-4350-7a5a-28cc4dfcc827@oracle.com> References: <386B2730-5732-4937-B334-411D8E458022@oracle.com> <6ddd571e-11a9-aa64-5ba6-dd98df9a792a@oracle.com> <2bd8bf1c-0328-9604-2661-634f8b1a59f3@oracle.com> <43f6c290-fe2f-f7e2-6e83-21cb11fbf0fe@oracle.com> <324be7ad-8d8c-4350-7a5a-28cc4dfcc827@oracle.com> Message-ID: <61b00767-0cbc-49d6-c80e-8876829d67c5@oracle.com> An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Wed Feb 21 21:44:31 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 21 Feb 2018 13:44:31 -0800 Subject: JVMTI retransformation and addition of private methods In-Reply-To: References: <386B2730-5732-4937-B334-411D8E458022@oracle.com> <6ddd571e-11a9-aa64-5ba6-dd98df9a792a@oracle.com> <2bd8bf1c-0328-9604-2661-634f8b1a59f3@oracle.com> <43f6c290-fe2f-f7e2-6e83-21cb11fbf0fe@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From jeremymanson at google.com Wed Feb 21 22:40:46 2018 From: jeremymanson at google.com (Jeremy Manson) Date: Wed, 21 Feb 2018 14:40:46 -0800 Subject: Thread Native ID Access Message-ID: Hey folks, I mentioned earlier in the thread about the ThreadInfo.from() bug that I found this because I was looking at fixing JDK-8154176, which proposes introducing native thread ids to Thread and ThreadInfo. https://bugs.openjdk.java.net/browse/JDK-8154176 I have a prototype for it. I have a couple of questions, though: 0) Does anyone object to this being done or my doing it? I see that it already has an owner. 1) Should the ID be a long? The Hotspot thread dump prints it out as 0x%x, which is an unsigned hexadecimal integer. The type hiding behind it is platform-dependent, though: a pid_t on Linux, an unsigned long on Windows, a thread_t on Solaris. I could make it a String instead... Jeremy -------------- next part -------------- An HTML attachment was scrubbed... URL: From mandy.chung at oracle.com Wed Feb 21 23:04:36 2018 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 21 Feb 2018 15:04:36 -0800 Subject: Thread Native ID Access In-Reply-To: References: Message-ID: <7080d7ef-a8a4-02d2-db93-cc334462054e@oracle.com> I'm not comfortable for ThreadInfo to expose the implementation details.? What should we specify w.r.t. Java Thread mapping to native thread which is platform dependent.? Also how does it relate to the future fibers [1]? Another alternative is for JDK specific diagnostic tools to do that mapping for you for example jcmd, rather than exposing it in the API.? I assume is that this is more about troubleshooting than on-going VM monitoring. Mandy [1] http://openjdk.java.net/projects/loom/ On 2/21/18 2:40 PM, Jeremy Manson wrote: > Hey folks, > > I mentioned earlier in the thread about the ThreadInfo.from() bug that > I found this because I was looking at fixing JDK-8154176, which > proposes introducing native thread ids to Thread and ThreadInfo. > > https://bugs.openjdk.java.net/browse/JDK-8154176 > > I have a prototype for it.? I have a couple of questions, though: > > 0) Does anyone object to this being done or my doing it?? I see that > it already has an owner. > > 1) Should the ID be a long?? The Hotspot thread dump prints it out as > 0x%x, which is an unsigned hexadecimal integer.? The type hiding > behind it is platform-dependent, though: a pid_t on Linux, an unsigned > long on Windows, a thread_t on Solaris.? I could make it a String > instead... > > Jeremy -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrb at google.com Wed Feb 21 23:46:29 2018 From: martinrb at google.com (Martin Buchholz) Date: Wed, 21 Feb 2018 15:46:29 -0800 Subject: Thread Native ID Access In-Reply-To: References: Message-ID: On Wed, Feb 21, 2018 at 2:40 PM, Jeremy Manson wrote: > > 1) Should the ID be a long? The Hotspot thread dump prints it out as > 0x%x, which is an unsigned hexadecimal integer. The type hiding behind it > is platform-dependent, though: a pid_t on Linux, an unsigned long on > Windows, a thread_t on Solaris. I could make it a String instead... > This is very similar to the question of what to do for e.g. Process#pid https://docs.oracle.com/javase/9/docs/api/java/lang/Process.html#pid-- Note the Posixy naming of the method and its long return type. When I was thinking about this 10 years ago, I was expecting us to represent it as a String inside the JVM, which would be more flexible, but it hasn't caused any trouble so far. Since thread ids are even more system dependent, the case for String is stronger. It's at least plausible that there are no native threads on the platform -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeremymanson at google.com Wed Feb 21 23:46:35 2018 From: jeremymanson at google.com (Jeremy Manson) Date: Wed, 21 Feb 2018 15:46:35 -0800 Subject: Thread Native ID Access In-Reply-To: <7080d7ef-a8a4-02d2-db93-cc334462054e@oracle.com> References: <7080d7ef-a8a4-02d2-db93-cc334462054e@oracle.com> Message-ID: Agreed that it is platform-dependent. That said, so is PID, and you can now get that from java.lang.Process. And also, there is very little available in SIGQUIT thread dumps that you can't get programmatically, and this is one of the big ticket things (we also have a patch to export the list of VM threads; maybe that would be useful to expose somewhere). Although the nid is useful for troubleshooting, it's more often used for ongoing VM monitoring, because you often want to map from Java Thread to some native resource, and the only way to do it is via native thread id. For examples of this sort of thing, on Linux, look in /proc//task/. There's a lot of useful information in there, but how to map it to Java threads is non-obvious. At Google, we added an interface to export the information, but I would rather relieve ourselves of the (relatively minor) technical debt and contribute to the community. I'm generally a little skeptical that adding to a JDK-specific diagnostic tool is the right solution. At Google, at least, it can be very, very difficult to apply those tools at scale / integrate them into existing monitoring tooling. When the JDK has functionality that is only available via one of these commands, we usually end up having to figure out what the command is doing and reimplement it. So, for example, we ended up with our own parser for hsperfdata. We end up relying on undocumented interfaces that can go away at the next JDK revision, and that's not good. I have no problem with some other, documented interface for it that doesn't go through Thread/ThreadInfo, but we (at least) will end up needing something that isn't tied to a particular tool. As far as the interaction of a feature like this with Fibers (or other alternate threading approaches, like Green threads): I wouldn't be worried about that. A fiber executes on an os thread. The only thing that might happen is that the os thread might change over time, or that multiple fibers might multiplex on the same os thread. If you are worried about the value changing, all you have to do is not to cache it in the Thread object. You wouldn't want to do that anyway, because it would be a waste of space for something that's only needed occasionally. Is there another way of doing the right thing here? Jeremy On Wed, Feb 21, 2018 at 3:04 PM, mandy chung wrote: > I'm not comfortable for ThreadInfo to expose the implementation details. > What should we specify w.r.t. Java Thread mapping to native thread which is > platform dependent. Also how does it relate to the future fibers [1]? > > Another alternative is for JDK specific diagnostic tools to do that > mapping for you for example jcmd, rather than exposing it in the API. I > assume is that this is more about troubleshooting than on-going VM > monitoring. > > Mandy > [1] http://openjdk.java.net/projects/loom/ > > > On 2/21/18 2:40 PM, Jeremy Manson wrote: > > Hey folks, > > I mentioned earlier in the thread about the ThreadInfo.from() bug that I > found this because I was looking at fixing JDK-8154176, which proposes > introducing native thread ids to Thread and ThreadInfo. > > https://bugs.openjdk.java.net/browse/JDK-8154176 > > I have a prototype for it. I have a couple of questions, though: > > 0) Does anyone object to this being done or my doing it? I see that it > already has an owner. > > 1) Should the ID be a long? The Hotspot thread dump prints it out as > 0x%x, which is an unsigned hexadecimal integer. The type hiding behind it > is platform-dependent, though: a pid_t on Linux, an unsigned long on > Windows, a thread_t on Solaris. I could make it a String instead... > > Jeremy > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From karen.kinnear at oracle.com Thu Feb 22 00:46:42 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Wed, 21 Feb 2018 19:46:42 -0500 Subject: JVMTI retransformation and addition of private methods In-Reply-To: References: <386B2730-5732-4937-B334-411D8E458022@oracle.com> <6ddd571e-11a9-aa64-5ba6-dd98df9a792a@oracle.com> <2bd8bf1c-0328-9604-2661-634f8b1a59f3@oracle.com> <43f6c290-fe2f-f7e2-6e83-21cb11fbf0fe@oracle.com> Message-ID: Sergeui, You were right - I read the sources incorrectly. Would still help to understand both the motivation and the reason to not add to the spec. Robert - do you remember why we didn?t add this to the specification? (6404550) > On Feb 21, 2018, at 4:44 PM, serguei.spitsyn at oracle.com wrote: > > Hi Karen, > > Thank you for sorting this out! > > > On 2/21/18 09:55, Karen Kinnear wrote: >> Dan, >> >> Thank you for all the background digging. This is really helpful. >> >> Serguei - do you know what tests exist for this behavior? > > Dan already replied (thanks, Dan!) > There are two tests in the open/test/jdk/java/lang/instrument: > RedefineMethodAddInvoke* > RedefineMethodDelInvoke* > > >> The way I read the source code - we currently allow ADD and DELETE for >> PRIVATE OR STATIC OR FINAL methods. Did I read that correctly? > The above does not look correct to me. > We have the same check for both ADD and DELETE method: > if ((old_flags & JVM_ACC_PRIVATE) == 0 > // hack: private should be treated as final, but alas > || (old_flags & (JVM_ACC_FINAL|JVM_ACC_STATIC)) == 0 > ) { > // deleted methods must be private > return JVMTI_ERROR_UNSUPPORTED_REDEFINITION_METHOD_DELETED; > } > > I read it as we allow ADD and DELETE for PRIVATE && (STATIC || FINAL) methods. > (Rephrase: We allow PRIVATE FINAL or PRIVATE STATIC methods.) > As private should always be treated final then we can simplify the above to: > We allow and and delete PRIVATE INSTANCE or PRIVATE STATIC methods > which is equal to just "PRIVATE methods?. Sergeui - you are right - I misread this today. And I played with the tests a bit - thanks Dan - and it matches the way you read this. So today, private methods that are not marked as final in the source code can not be added - I tried that variation by modifying the RedefineMethodAddInvokeTarget_1.java and changing private final void myMethod1() to private void myMethod1() and got an UnsupportedOperationException. So - private methods are not marked as ACC_FINAL today so I think the simplification doesn?t apply, so we are left with the ability to ADD or DELETE PRIVATE && (STATIC || FINAL) methods - at least that is what we support today. > >> With the current implementation, I am not sure if deletion works for private methods - do we >> have a test for that? Or could you add one as part of this exercise? > > Yes, we have one j.l.instrument test: RedefineMethodDelInvoke.sh. I tried the test. And it works because of the requirement that FINAL or STATIC are set, which therefore means no vtable entry. > > >> Today we create a vtable entry for private methods (my misunderstanding ~ 2006ish). After discussions >> with David I no longer believe we need those. >> Today, klassVtable::adjust_method_entries has an assertion >> assert(!old_method->is_deleted(), ?vtable methods may not be deleted?) >> >> I may have read the code incorrectly - but I would expect to hit this assertion if you had a private >> method you were deleting that was not final and not static. >> >> option 1) we could explicitly tighten the restrictions to match what we have implemented >> option 2) we could make this work by changing klassVtable.cpp::update_inherited_vtable >> handling of private fields to be done the way it handles final fields. >> option 3) I read the code incorrectly? > > If we create a vtable entry for private methods then we should hit the assert above. > If we no longer need this then we have no problem. We do create a vtable entry for private methods; however if FINAL or STATIC is set, then we do not create a vtable entry. That is why we don?t ever get here. thanks, Karen > > Thanks, > Serguei > >> thanks, >> Karen >> >>> On Feb 21, 2018, at 10:40 AM, Daniel D. Daugherty > wrote: >>> >>> On 2/21/18 2:45 AM, serguei.spitsyn at oracle.com wrote: >>>> On 2/20/18 23:01, David Holmes wrote: >>>>> On 21/02/2018 4:50 PM, serguei.spitsyn at oracle.com wrote: >>>>>> Hi Karen and David, >>>>>> >>>>>> >>>>>> On 2/20/18 19:52, David Holmes wrote: >>>>>>> Hi Karen, >>>>>>> >>>>>>> On 21/02/2018 1:54 AM, Karen Kinnear wrote: >>>>>>>> Folks, >>>>>>>> >>>>>>>> As part of the Valhalla EG discussions for JVMTI changes for nestmates (thank you Serguei and David), >>>>>>>> IBM brought up a request that we update the JVMTI documentation to reflect that we allow addition >>>>>>>> of private methods. >>>>>>>> >>>>>>>> Is there a reason we do not document this? I?m inviting those who were involved at the time - please include >>>>>>>> others if needed. >>>>>> >>>>>> I support documenting this in the JVMTI spec and had a plan to fix it in 11. >>>>>> However, it is not clear to me yet if we have a consensus on it. >>>>> >>>>> I would like to see a detailed analysis of the implications of allowing this. I _think_ it is safe but ... >>>> >>>> Valid concern. >>>> Also, I'd love to collect more details on the initial motivation to relax the JVMTI spec. >>>> Most likely we had no CCC/CSR filed on this change. >>>> >>>> >>>>>>> This issue is tracked by: >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8192936 >>>>>>> >>>>>>> "RI does not follow the JVMTI RedefineClasses spec that is too strict in the definition" >>>>>> >>>>>> Yes, this is the one. >>>>>> Thank you, David, for posting the link. >>>>>> >>>>>> >>>>>>> As I wrote there ... It is not at all clear how JDK-6404550 morphed into "Permit the adding or deleting of private final/static methods with redefine" - nor why those changes failed to make any change to the spec itself. It is also unclear whether the add/delete is restricted to final/static methods or any private method? I can see that the intent was to only allow something that would not perturb the vtable for existing instances. >>>>>> >>>>>> I agree, there is a confusion somewhere. >>>>>> Is it possible, the JDK-6404550 in JIRA is a different bug than the one in the Bugtraq system? >>>>>> >>>>>> The JDK-6404550 in JIRA has a different synopsis: >>>>>> https://bugs.openjdk.java.net/browse/JDK-6404550 >>>>>> Cannot implement late attach in NetBeans Profiler due to missing functionality in JVMTI >>>>> >>>>> Digging deeper ... to fix the problem described in that bug they augmented JVM TI to allow private method redefinition as an alternate to the "native rebinding" technique that had been used previously. See the final comment in: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-6341303 >>>>> >>>>> "JVMTI Spec: Need a way how to rebind Object.wait and Thread.sleep with late attach" >>>>> >>>>> which was closed as a duplicate. >>>> >>>> Thank you for the point. >>>> This explains it. >>>> It seems, the bug synopsis was changed at some moment. >>> >>> The synopsis for 6404550 has never changed. Here's the subject line when >>> it was created on 2006.03.27: >>> >>> > CR 6404550 *HOT* Created P1 hotspot/jvmti Cannot implement late attach in NetBeans Profiler due to missing functionality in JVMTI >>> >>> I think the confusion arises over comments like this in 6341303: >>> >>>> Robert Field added a comment - 2006-05-04 11:54 >>>> BT2:EVALUATION >>>> >>>> This can now be accomplished with Java programming language instrumentation, via: >>>> >>>> 6404550: missing late attach (JVM TI redefine) functionality >>>> Permit the adding or deleting of private final/static methods with redefine >>>> >>>> Closing this bug as a duplicate. >>> >>> That's just Robert's style for an sccs delta comment: >>> >>> D 1.65.2.3 06/04/25 23:36:35 rfield 140 139 00023/00013/03263 >>> MRs: >>> COMMENTS: >>> 6404550: missing late attach (JVM TI redefine) functionality >>> Add/delete private methods, continued: changes per review >>> >>> Back in the ancient past we tried to include some brief >>> info about the change in the delta comment. This was one of many >>> deltas associated with 6404550. >>> >>> Please see the attached email that I sent on 2012.12.17 about the >>> history behind this issue... (sent to Karen, Mikael V, and Serguei) >>> >>> It seems I forwarded that same email to Coleen, Markus G and Serguei >>> back on 2014.05.20. Since Markus is on that thread, it must have had >>> something to do with research about JFR... >>> >>> I need to do a detailed read thru my e-mail archive for 6404550 to >>> see if I can spot some clues about why we didn't do a spec update. >>> >>> Dan >>> >>> >>>> >>>> Thanks, >>>> Serguei >>>> >>>>> >>>>> David >>>>> ----- >>>>> >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>>> >>>>>>> -- >>>>>>> David >>>>>>> >>>>>>> >>>>>>>> thanks, >>>>>>>> Karen >>>>>>>> >>>>>> >>>> >>>> >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Thu Feb 22 00:59:30 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 22 Feb 2018 10:59:30 +1000 Subject: JVMTI retransformation and addition of private methods In-Reply-To: References: <386B2730-5732-4937-B334-411D8E458022@oracle.com> <6ddd571e-11a9-aa64-5ba6-dd98df9a792a@oracle.com> <2bd8bf1c-0328-9604-2661-634f8b1a59f3@oracle.com> <43f6c290-fe2f-f7e2-6e83-21cb11fbf0fe@oracle.com> Message-ID: <38c3949e-c9ac-8419-a5c3-5fac876a5125@oracle.com> On 22/02/2018 7:44 AM, serguei.spitsyn at oracle.com wrote: > Hi Karen, > > Thank you for sorting this out! > > > On 2/21/18 09:55, Karen Kinnear wrote: >> Dan, >> >> Thank you for all the background digging. This is really helpful. >> >> Serguei - do you know what tests exist for this behavior? > > Dan already replied (thanks, Dan!) > There are two tests in the open/test/jdk/java/lang/instrument: > RedefineMethodAddInvoke* > RedefineMethodDelInvoke* > > >> The way I read the source code - we currently allow ADD and DELETE for >> PRIVATE OR STATIC OR FINAL methods. Did I read that correctly? > The above does not look correct to me. > We have the same check for both ADD and DELETE method: > ????? if ((old_flags & JVM_ACC_PRIVATE) == 0 > ?????????? // hack: private should be treated as final, but alas > ????????? || (old_flags & (JVM_ACC_FINAL|JVM_ACC_STATIC)) == 0 > ???????? ) { > ??????? // deleted methods must be private > ??????? return JVMTI_ERROR_UNSUPPORTED_REDEFINITION_METHOD_DELETED; > ????? } > > I read it as we allow ADD and DELETE for PRIVATE && (STATIC || FINAL) > methods. > (Rephrase: We allow PRIVATE FINAL or PRIVATE STATIC methods.) > As private should always be treated final then we can simplify the above to: > ? We allow and and delete PRIVATE INSTANCE or PRIVATE STATIC methods > ? which is equal to just "PRIVATE methods". I read it as "PRIVATE" or "FINAL STATIC". David ----- >> With the current implementation, I am not sure if deletion works for >> private methods - do we >> have a test for that? Or could you add one as part of this exercise? > > Yes, we have one j.l.instrument test: RedefineMethodDelInvoke.sh. > > >> Today we create a vtable entry for private methods (my >> misunderstanding ~ 2006ish). After discussions >> with David I no longer believe we need those. >> Today, klassVtable::adjust_method_entries has an assertion >> ? assert(!old_method->is_deleted(), ?vtable methods may not be deleted?) >> >> I may have read the code incorrectly - but I would expect to hit this >> assertion if you had a private >> method you were deleting that was not final and not static. >> >> option 1) we could explicitly tighten the restrictions to match what >> we have implemented >> option 2) we could make this work by changing >> klassVtable.cpp::update_inherited_vtable >> ? handling of private fields to be done the way it handles final fields. >> option 3) I read the code incorrectly? > > If we create a vtable entry for private methods then we should hit the > assert above. > If we no longer need this then we have no problem. > > Thanks, > Serguei > >> thanks, >> Karen >> >>> On Feb 21, 2018, at 10:40 AM, Daniel D. Daugherty >>> > wrote: >>> >>> On 2/21/18 2:45 AM, serguei.spitsyn at oracle.com wrote: >>>> On 2/20/18 23:01, David Holmes wrote: >>>>> On 21/02/2018 4:50 PM, serguei.spitsyn at oracle.com wrote: >>>>>> Hi Karen and David, >>>>>> >>>>>> >>>>>> On 2/20/18 19:52, David Holmes wrote: >>>>>>> Hi Karen, >>>>>>> >>>>>>> On 21/02/2018 1:54 AM, Karen Kinnear wrote: >>>>>>>> Folks, >>>>>>>> >>>>>>>> As part of the Valhalla EG discussions for JVMTI changes for >>>>>>>> nestmates (thank you Serguei and David), >>>>>>>> IBM brought up a request that we update the JVMTI documentation >>>>>>>> to reflect that we allow addition >>>>>>>> of private methods. >>>>>>>> >>>>>>>> Is there a reason we do not document this? I?m inviting those >>>>>>>> who were involved at the time - please include >>>>>>>> others if needed. >>>>>> >>>>>> I support documenting this in the JVMTI spec and had a plan to fix >>>>>> it in 11. >>>>>> However, it is not clear to me yet if we have a consensus on it. >>>>> >>>>> I would like to see a detailed analysis of the implications of >>>>> allowing this. I _think_ it is safe but ... >>>> >>>> Valid concern. >>>> Also, I'd love to collect more details on the initial motivation to >>>> relax the JVMTI spec. >>>> Most likely we had no CCC/CSR filed on this change. >>>> >>>> >>>>>>> This issue is tracked by: >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8192936 >>>>>>> >>>>>>> "RI does not follow the JVMTI RedefineClasses spec that is too >>>>>>> strict in the definition" >>>>>> >>>>>> Yes, this is the one. >>>>>> Thank you, David, for posting the link. >>>>>> >>>>>> >>>>>>> As I wrote there ... It is not at all clear how JDK-6404550 >>>>>>> morphed into "Permit the adding or deleting of private >>>>>>> final/static methods with redefine" - nor why those changes >>>>>>> failed to make any change to the spec itself. It is also unclear >>>>>>> whether the add/delete is restricted to final/static methods or >>>>>>> any private method? I can see that the intent was to only allow >>>>>>> something that would not perturb the vtable for existing instances. >>>>>> >>>>>> I agree, there is a confusion somewhere. >>>>>> Is it possible, the JDK-6404550 in JIRA is a different bug than >>>>>> the one in the Bugtraq system? >>>>>> >>>>>> The JDK-6404550 in JIRA has a different synopsis: >>>>>> https://bugs.openjdk.java.net/browse/JDK-6404550 >>>>>> ????? Cannot implement late attach in NetBeans Profiler due to >>>>>> missing functionality in JVMTI >>>>> >>>>> Digging deeper ... to fix the problem described in that bug they >>>>> augmented JVM TI to allow private method redefinition as an >>>>> alternate to the "native rebinding" technique that had been used >>>>> previously. See the final comment in: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-6341303 >>>>> >>>>> "JVMTI Spec: Need a way how to rebind Object.wait and Thread.sleep >>>>> with late attach" >>>>> >>>>> which was closed as a duplicate. >>>> >>>> Thank you for the point. >>>> This explains it. >>>> It seems, the bug synopsis was changed at some moment. >>> >>> The synopsis for 6404550 has never changed. Here's the subject line when >>> it was created on 2006.03.27: >>> >>> > CR 6404550 *HOT* Created P1 hotspot/jvmti Cannot implement late >>> attach in NetBeans Profiler due to missing functionality in JVMTI >>> >>> I think the confusion arises over comments like this in 6341303: >>> >>>> rfield Robert Field >>>> >>>> added a comment - 2006-05-04 11:54 >>>> BT2:EVALUATION >>>> >>>> This can now be accomplished with Java programming language >>>> instrumentation, via: >>>> >>>> ?????6404550: missing late attach (JVM TI redefine) functionality >>>> ???????????Permit the adding or deleting of private final/static >>>> methods with redefine >>>> >>>> Closing this bug as a duplicate. >>> >>> That's just Robert's style for an sccs delta comment: >>> >>> D 1.65.2.3 06/04/25 23:36:35 rfield 140 139 00023/00013/03263 >>> MRs: >>> COMMENTS: >>> 6404550: missing late attach (JVM TI redefine) functionality >>> ??????????? Add/delete private methods, continued: changes per review >>> >>> Back in the ancient past we tried to include some brief >>> info about the change in the delta comment. This was one of many >>> deltas associated with 6404550. >>> >>> Please see the attached email that I sent on 2012.12.17 about the >>> history behind this issue... (sent to Karen, Mikael V, and Serguei) >>> >>> It seems I forwarded that same email to Coleen, Markus G and Serguei >>> back on 2014.05.20. Since Markus is on that thread, it must have had >>> something to do with research about JFR... >>> >>> I need to do a detailed read thru my e-mail archive for 6404550 to >>> see if I can spot some clues about why we didn't do a spec update. >>> >>> Dan >>> >>> >>>> >>>> Thanks, >>>> Serguei >>>> >>>>> >>>>> David >>>>> ----- >>>>> >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>>> >>>>>>> -- >>>>>>> David >>>>>>> >>>>>>> >>>>>>>> thanks, >>>>>>>> Karen >>>>>>>> >>>>>> >>>> >>>> >>> >>> >> > From serguei.spitsyn at oracle.com Thu Feb 22 01:00:15 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 21 Feb 2018 17:00:15 -0800 Subject: JVMTI retransformation and addition of private methods In-Reply-To: References: <386B2730-5732-4937-B334-411D8E458022@oracle.com> <6ddd571e-11a9-aa64-5ba6-dd98df9a792a@oracle.com> <2bd8bf1c-0328-9604-2661-634f8b1a59f3@oracle.com> <43f6c290-fe2f-f7e2-6e83-21cb11fbf0fe@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Thu Feb 22 01:10:47 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 22 Feb 2018 11:10:47 +1000 Subject: JVMTI retransformation and addition of private methods In-Reply-To: <38c3949e-c9ac-8419-a5c3-5fac876a5125@oracle.com> References: <386B2730-5732-4937-B334-411D8E458022@oracle.com> <6ddd571e-11a9-aa64-5ba6-dd98df9a792a@oracle.com> <2bd8bf1c-0328-9604-2661-634f8b1a59f3@oracle.com> <43f6c290-fe2f-f7e2-6e83-21cb11fbf0fe@oracle.com> <38c3949e-c9ac-8419-a5c3-5fac876a5125@oracle.com> Message-ID: Correction ... On 22/02/2018 10:59 AM, David Holmes wrote: > On 22/02/2018 7:44 AM, serguei.spitsyn at oracle.com wrote: >> Hi Karen, >> >> Thank you for sorting this out! >> >> >> On 2/21/18 09:55, Karen Kinnear wrote: >>> Dan, >>> >>> Thank you for all the background digging. This is really helpful. >>> >>> Serguei - do you know what tests exist for this behavior? >> >> Dan already replied (thanks, Dan!) >> There are two tests in the open/test/jdk/java/lang/instrument: >> RedefineMethodAddInvoke* >> RedefineMethodDelInvoke* >> >> >>> The way I read the source code - we currently allow ADD and DELETE for >>> PRIVATE OR STATIC OR FINAL methods. Did I read that correctly? >> The above does not look correct to me. >> We have the same check for both ADD and DELETE method: >> ?????? if ((old_flags & JVM_ACC_PRIVATE) == 0 >> ??????????? // hack: private should be treated as final, but alas >> ?????????? || (old_flags & (JVM_ACC_FINAL|JVM_ACC_STATIC)) == 0 >> ????????? ) { >> ???????? // deleted methods must be private >> ???????? return JVMTI_ERROR_UNSUPPORTED_REDEFINITION_METHOD_DELETED; >> ?????? } >> >> I read it as we allow ADD and DELETE for PRIVATE && (STATIC || FINAL) >> methods. >> (Rephrase: We allow PRIVATE FINAL or PRIVATE STATIC methods.) >> As private should always be treated final then we can simplify the >> above to: >> ?? We allow and and delete PRIVATE INSTANCE or PRIVATE STATIC methods >> ?? which is equal to just "PRIVATE methods". > > I read it as "PRIVATE" or "FINAL STATIC". Nope I read it wrong - Serguei is right. Which is somewhat strange as the underlying problem being addressed seemed to require retransformation of Object.wait and Thread.sleep - the former is a public final instance method; the latter a public static method. Neither are private ... ??? David > David > ----- > >>> With the current implementation, I am not sure if deletion works for >>> private methods - do we >>> have a test for that? Or could you add one as part of this exercise? >> >> Yes, we have one j.l.instrument test: RedefineMethodDelInvoke.sh. >> >> >>> Today we create a vtable entry for private methods (my >>> misunderstanding ~ 2006ish). After discussions >>> with David I no longer believe we need those. >>> Today, klassVtable::adjust_method_entries has an assertion >>> ? assert(!old_method->is_deleted(), ?vtable methods may not be deleted?) >>> >>> I may have read the code incorrectly - but I would expect to hit this >>> assertion if you had a private >>> method you were deleting that was not final and not static. >>> >>> option 1) we could explicitly tighten the restrictions to match what >>> we have implemented >>> option 2) we could make this work by changing >>> klassVtable.cpp::update_inherited_vtable >>> ? handling of private fields to be done the way it handles final fields. >>> option 3) I read the code incorrectly? >> >> If we create a vtable entry for private methods then we should hit the >> assert above. >> If we no longer need this then we have no problem. >> >> Thanks, >> Serguei >> >>> thanks, >>> Karen >>> >>>> On Feb 21, 2018, at 10:40 AM, Daniel D. Daugherty >>>> > >>>> wrote: >>>> >>>> On 2/21/18 2:45 AM, serguei.spitsyn at oracle.com wrote: >>>>> On 2/20/18 23:01, David Holmes wrote: >>>>>> On 21/02/2018 4:50 PM, serguei.spitsyn at oracle.com wrote: >>>>>>> Hi Karen and David, >>>>>>> >>>>>>> >>>>>>> On 2/20/18 19:52, David Holmes wrote: >>>>>>>> Hi Karen, >>>>>>>> >>>>>>>> On 21/02/2018 1:54 AM, Karen Kinnear wrote: >>>>>>>>> Folks, >>>>>>>>> >>>>>>>>> As part of the Valhalla EG discussions for JVMTI changes for >>>>>>>>> nestmates (thank you Serguei and David), >>>>>>>>> IBM brought up a request that we update the JVMTI documentation >>>>>>>>> to reflect that we allow addition >>>>>>>>> of private methods. >>>>>>>>> >>>>>>>>> Is there a reason we do not document this? I?m inviting those >>>>>>>>> who were involved at the time - please include >>>>>>>>> others if needed. >>>>>>> >>>>>>> I support documenting this in the JVMTI spec and had a plan to >>>>>>> fix it in 11. >>>>>>> However, it is not clear to me yet if we have a consensus on it. >>>>>> >>>>>> I would like to see a detailed analysis of the implications of >>>>>> allowing this. I _think_ it is safe but ... >>>>> >>>>> Valid concern. >>>>> Also, I'd love to collect more details on the initial motivation to >>>>> relax the JVMTI spec. >>>>> Most likely we had no CCC/CSR filed on this change. >>>>> >>>>> >>>>>>>> This issue is tracked by: >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8192936 >>>>>>>> >>>>>>>> "RI does not follow the JVMTI RedefineClasses spec that is too >>>>>>>> strict in the definition" >>>>>>> >>>>>>> Yes, this is the one. >>>>>>> Thank you, David, for posting the link. >>>>>>> >>>>>>> >>>>>>>> As I wrote there ... It is not at all clear how JDK-6404550 >>>>>>>> morphed into "Permit the adding or deleting of private >>>>>>>> final/static methods with redefine" - nor why those changes >>>>>>>> failed to make any change to the spec itself. It is also unclear >>>>>>>> whether the add/delete is restricted to final/static methods or >>>>>>>> any private method? I can see that the intent was to only allow >>>>>>>> something that would not perturb the vtable for existing instances. >>>>>>> >>>>>>> I agree, there is a confusion somewhere. >>>>>>> Is it possible, the JDK-6404550 in JIRA is a different bug than >>>>>>> the one in the Bugtraq system? >>>>>>> >>>>>>> The JDK-6404550 in JIRA has a different synopsis: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-6404550 >>>>>>> ????? Cannot implement late attach in NetBeans Profiler due to >>>>>>> missing functionality in JVMTI >>>>>> >>>>>> Digging deeper ... to fix the problem described in that bug they >>>>>> augmented JVM TI to allow private method redefinition as an >>>>>> alternate to the "native rebinding" technique that had been used >>>>>> previously. See the final comment in: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-6341303 >>>>>> >>>>>> "JVMTI Spec: Need a way how to rebind Object.wait and Thread.sleep >>>>>> with late attach" >>>>>> >>>>>> which was closed as a duplicate. >>>>> >>>>> Thank you for the point. >>>>> This explains it. >>>>> It seems, the bug synopsis was changed at some moment. >>>> >>>> The synopsis for 6404550 has never changed. Here's the subject line >>>> when >>>> it was created on 2006.03.27: >>>> >>>> > CR 6404550 *HOT* Created P1 hotspot/jvmti Cannot implement late >>>> attach in NetBeans Profiler due to missing functionality in JVMTI >>>> >>>> I think the confusion arises over comments like this in 6341303: >>>> >>>>> rfield Robert Field >>>>> >>>>> added a comment - 2006-05-04 11:54 >>>>> BT2:EVALUATION >>>>> >>>>> This can now be accomplished with Java programming language >>>>> instrumentation, via: >>>>> >>>>> ?????6404550: missing late attach (JVM TI redefine) functionality >>>>> ???????????Permit the adding or deleting of private final/static >>>>> methods with redefine >>>>> >>>>> Closing this bug as a duplicate. >>>> >>>> That's just Robert's style for an sccs delta comment: >>>> >>>> D 1.65.2.3 06/04/25 23:36:35 rfield 140 139 00023/00013/03263 >>>> MRs: >>>> COMMENTS: >>>> 6404550: missing late attach (JVM TI redefine) functionality >>>> ??????????? Add/delete private methods, continued: changes per review >>>> >>>> Back in the ancient past we tried to include some brief >>>> info about the change in the delta comment. This was one of many >>>> deltas associated with 6404550. >>>> >>>> Please see the attached email that I sent on 2012.12.17 about the >>>> history behind this issue... (sent to Karen, Mikael V, and Serguei) >>>> >>>> It seems I forwarded that same email to Coleen, Markus G and Serguei >>>> back on 2014.05.20. Since Markus is on that thread, it must have had >>>> something to do with research about JFR... >>>> >>>> I need to do a detailed read thru my e-mail archive for 6404550 to >>>> see if I can spot some clues about why we didn't do a spec update. >>>> >>>> Dan >>>> >>>> >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>>> >>>>>> David >>>>>> ----- >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Serguei >>>>>>> >>>>>>> >>>>>>>> -- >>>>>>>> David >>>>>>>> >>>>>>>> >>>>>>>>> thanks, >>>>>>>>> Karen >>>>>>>>> >>>>>>> >>>>> >>>>> >>>> >>>> >>> >> From serguei.spitsyn at oracle.com Thu Feb 22 01:13:18 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 21 Feb 2018 17:13:18 -0800 Subject: JVMTI retransformation and addition of private methods In-Reply-To: <38c3949e-c9ac-8419-a5c3-5fac876a5125@oracle.com> References: <386B2730-5732-4937-B334-411D8E458022@oracle.com> <6ddd571e-11a9-aa64-5ba6-dd98df9a792a@oracle.com> <2bd8bf1c-0328-9604-2661-634f8b1a59f3@oracle.com> <43f6c290-fe2f-f7e2-6e83-21cb11fbf0fe@oracle.com> <38c3949e-c9ac-8419-a5c3-5fac876a5125@oracle.com> Message-ID: On 2/21/18 16:59, David Holmes wrote: > On 22/02/2018 7:44 AM, serguei.spitsyn at oracle.com wrote: >> Hi Karen, >> >> Thank you for sorting this out! >> >> >> On 2/21/18 09:55, Karen Kinnear wrote: >>> Dan, >>> >>> Thank you for all the background digging. This is really helpful. >>> >>> Serguei - do you know what tests exist for this behavior? >> >> Dan already replied (thanks, Dan!) >> There are two tests in the open/test/jdk/java/lang/instrument: >> RedefineMethodAddInvoke* >> RedefineMethodDelInvoke* >> >> >>> The way I read the source code - we currently allow ADD and DELETE for >>> PRIVATE OR STATIC OR FINAL methods. Did I read that correctly? >> The above does not look correct to me. >> We have the same check for both ADD and DELETE method: >> ?????? if ((old_flags & JVM_ACC_PRIVATE) == 0 >> ??????????? // hack: private should be treated as final, but alas >> ?????????? || (old_flags & (JVM_ACC_FINAL|JVM_ACC_STATIC)) == 0 >> ????????? ) { >> ???????? // deleted methods must be private >> ???????? return JVMTI_ERROR_UNSUPPORTED_REDEFINITION_METHOD_DELETED; >> ?????? } >> >> I read it as we allow ADD and DELETE for PRIVATE && (STATIC || FINAL) >> methods. >> (Rephrase: We allow PRIVATE FINAL or PRIVATE STATIC methods.) >> As private should always be treated final then we can simplify the >> above to: >> ?? We allow and and delete PRIVATE INSTANCE or PRIVATE STATIC methods >> ?? which is equal to just "PRIVATE methods". > > I read it as "PRIVATE" or "FINAL STATIC". There are multiple negations above, so this needs to be interpreted in a right context: ? Return error if deleted method is !PRIVATE or (!FINAL and !STATIC) After inversion: ? Allow to delete if method is PRIVATE and (FINAL or STATIC) I feel myself stupid when reading this code. :) Thanks, Serguei > David > ----- > >>> With the current implementation, I am not sure if deletion works for >>> private methods - do we >>> have a test for that? Or could you add one as part of this exercise? >> >> Yes, we have one j.l.instrument test: RedefineMethodDelInvoke.sh. >> >> >>> Today we create a vtable entry for private methods (my >>> misunderstanding ~ 2006ish). After discussions >>> with David I no longer believe we need those. >>> Today, klassVtable::adjust_method_entries has an assertion >>> ? assert(!old_method->is_deleted(), ?vtable methods may not be >>> deleted?) >>> >>> I may have read the code incorrectly - but I would expect to hit >>> this assertion if you had a private >>> method you were deleting that was not final and not static. >>> >>> option 1) we could explicitly tighten the restrictions to match what >>> we have implemented >>> option 2) we could make this work by changing >>> klassVtable.cpp::update_inherited_vtable >>> ? handling of private fields to be done the way it handles final >>> fields. >>> option 3) I read the code incorrectly? >> >> If we create a vtable entry for private methods then we should hit >> the assert above. >> If we no longer need this then we have no problem. >> >> Thanks, >> Serguei >> >>> thanks, >>> Karen >>> >>>> On Feb 21, 2018, at 10:40 AM, Daniel D. Daugherty >>>> > >>>> wrote: >>>> >>>> On 2/21/18 2:45 AM, serguei.spitsyn at oracle.com wrote: >>>>> On 2/20/18 23:01, David Holmes wrote: >>>>>> On 21/02/2018 4:50 PM, serguei.spitsyn at oracle.com wrote: >>>>>>> Hi Karen and David, >>>>>>> >>>>>>> >>>>>>> On 2/20/18 19:52, David Holmes wrote: >>>>>>>> Hi Karen, >>>>>>>> >>>>>>>> On 21/02/2018 1:54 AM, Karen Kinnear wrote: >>>>>>>>> Folks, >>>>>>>>> >>>>>>>>> As part of the Valhalla EG discussions for JVMTI changes for >>>>>>>>> nestmates (thank you Serguei and David), >>>>>>>>> IBM brought up a request that we update the JVMTI >>>>>>>>> documentation to reflect that we allow addition >>>>>>>>> of private methods. >>>>>>>>> >>>>>>>>> Is there a reason we do not document this? I?m inviting those >>>>>>>>> who were involved at the time - please include >>>>>>>>> others if needed. >>>>>>> >>>>>>> I support documenting this in the JVMTI spec and had a plan to >>>>>>> fix it in 11. >>>>>>> However, it is not clear to me yet if we have a consensus on it. >>>>>> >>>>>> I would like to see a detailed analysis of the implications of >>>>>> allowing this. I _think_ it is safe but ... >>>>> >>>>> Valid concern. >>>>> Also, I'd love to collect more details on the initial motivation >>>>> to relax the JVMTI spec. >>>>> Most likely we had no CCC/CSR filed on this change. >>>>> >>>>> >>>>>>>> This issue is tracked by: >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8192936 >>>>>>>> >>>>>>>> "RI does not follow the JVMTI RedefineClasses spec that is too >>>>>>>> strict in the definition" >>>>>>> >>>>>>> Yes, this is the one. >>>>>>> Thank you, David, for posting the link. >>>>>>> >>>>>>> >>>>>>>> As I wrote there ... It is not at all clear how JDK-6404550 >>>>>>>> morphed into "Permit the adding or deleting of private >>>>>>>> final/static methods with redefine" - nor why those changes >>>>>>>> failed to make any change to the spec itself. It is also >>>>>>>> unclear whether the add/delete is restricted to final/static >>>>>>>> methods or any private method? I can see that the intent was to >>>>>>>> only allow something that would not perturb the vtable for >>>>>>>> existing instances. >>>>>>> >>>>>>> I agree, there is a confusion somewhere. >>>>>>> Is it possible, the JDK-6404550 in JIRA is a different bug than >>>>>>> the one in the Bugtraq system? >>>>>>> >>>>>>> The JDK-6404550 in JIRA has a different synopsis: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-6404550 >>>>>>> ????? Cannot implement late attach in NetBeans Profiler due to >>>>>>> missing functionality in JVMTI >>>>>> >>>>>> Digging deeper ... to fix the problem described in that bug they >>>>>> augmented JVM TI to allow private method redefinition as an >>>>>> alternate to the "native rebinding" technique that had been used >>>>>> previously. See the final comment in: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-6341303 >>>>>> >>>>>> "JVMTI Spec: Need a way how to rebind Object.wait and >>>>>> Thread.sleep with late attach" >>>>>> >>>>>> which was closed as a duplicate. >>>>> >>>>> Thank you for the point. >>>>> This explains it. >>>>> It seems, the bug synopsis was changed at some moment. >>>> >>>> The synopsis for 6404550 has never changed. Here's the subject line >>>> when >>>> it was created on 2006.03.27: >>>> >>>> > CR 6404550 *HOT* Created P1 hotspot/jvmti Cannot implement late >>>> attach in NetBeans Profiler due to missing functionality in JVMTI >>>> >>>> I think the confusion arises over comments like this in 6341303: >>>> >>>>> rfield Robert Field >>>>> >>>>> added a comment - 2006-05-04 11:54 >>>>> BT2:EVALUATION >>>>> >>>>> This can now be accomplished with Java programming language >>>>> instrumentation, via: >>>>> >>>>> ?????6404550: missing late attach (JVM TI redefine) functionality >>>>> ???????????Permit the adding or deleting of private final/static >>>>> methods with redefine >>>>> >>>>> Closing this bug as a duplicate. >>>> >>>> That's just Robert's style for an sccs delta comment: >>>> >>>> D 1.65.2.3 06/04/25 23:36:35 rfield 140 139 00023/00013/03263 >>>> MRs: >>>> COMMENTS: >>>> 6404550: missing late attach (JVM TI redefine) functionality >>>> ??????????? Add/delete private methods, continued: changes per review >>>> >>>> Back in the ancient past we tried to include some brief >>>> info about the change in the delta comment. This was one of many >>>> deltas associated with 6404550. >>>> >>>> Please see the attached email that I sent on 2012.12.17 about the >>>> history behind this issue... (sent to Karen, Mikael V, and Serguei) >>>> >>>> It seems I forwarded that same email to Coleen, Markus G and Serguei >>>> back on 2014.05.20. Since Markus is on that thread, it must have had >>>> something to do with research about JFR... >>>> >>>> I need to do a detailed read thru my e-mail archive for 6404550 to >>>> see if I can spot some clues about why we didn't do a spec update. >>>> >>>> Dan >>>> >>>> >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>>> >>>>>> David >>>>>> ----- >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Serguei >>>>>>> >>>>>>> >>>>>>>> -- >>>>>>>> David >>>>>>>> >>>>>>>> >>>>>>>>> thanks, >>>>>>>>> Karen >>>>>>>>> >>>>>>> >>>>> >>>>> >>>> >>>> >>> >> From mandy.chung at oracle.com Thu Feb 22 01:13:03 2018 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 21 Feb 2018 17:13:03 -0800 Subject: Thread Native ID Access In-Reply-To: References: <7080d7ef-a8a4-02d2-db93-cc334462054e@oracle.com> Message-ID: JDK 5 did very minimal for monitoring of native OS resources due to its scope and also its platform-dependent nature. I don't have a good advice how and where to expose them via Java SE API without further investigation. A couple of things come to mind: - Should native threads be exposed in Process API? - jdk.management module defines JDK extension to java.lang.management API that might be a good place to list VM threads. - other than VM threads and Java to native OS threads mapping, any other items on your list for monitoring? Mandy [1] http://download.java.net/java/jdk10/docs/api/jdk.management-summary.html On 2/21/18 3:46 PM, Jeremy Manson wrote: > Agreed that it is platform-dependent. That said, so is PID, and you > can now get that from java.lang.Process. And also, there is very > little available in SIGQUIT thread dumps that you can't get > programmatically, and this is one of the big ticket things (we also > have a patch to export the list of VM threads; maybe that would be > useful to expose somewhere). > > Although the nid is useful for troubleshooting, it's more often used > for ongoing VM monitoring, because you often want to map from Java > Thread to some native resource, and the only way to do it is via > native thread id. For examples of this sort of thing, on Linux, look > in /proc//task/. There's a lot of useful information in > there, but how to map it to Java threads is non-obvious. > > At Google, we added an interface to export the information, but I > would rather relieve ourselves of the (relatively minor) technical > debt and contribute to the community. > > I'm generally a little skeptical that adding to a JDK-specific > diagnostic tool is the right solution. At Google, at least, it can > be very, very difficult to apply those tools at scale / integrate > them into existing monitoring tooling. When the JDK has > functionality that is only available via one of these commands, we > usually end up having to figure out what the command is doing and > reimplement it. So, for example, we ended up with our own parser for > hsperfdata. We end up relying on undocumented interfaces that can go > away at the next JDK revision, and that's not good. > > I have no problem with some other, documented interface for it that > doesn't go through Thread/ThreadInfo, but we (at least) will end up > needing something that isn't tied to a particular tool. > > As far as the interaction of a feature like this with Fibers (or > other alternate threading approaches, like Green threads): I wouldn't > be worried about that. A fiber executes on an os thread. The only > thing that might happen is that the os thread might change over time, > or that multiple fibers might multiplex on the same os thread. If > you are worried about the value changing, all you have to do is not > to cache it in the Thread object. You wouldn't want to do that > anyway, because it would be a waste of space for something that's > only needed occasionally. > > Is there another way of doing the right thing here? > > Jeremy > > > > On Wed, Feb 21, 2018 at 3:04 PM, mandy chung > wrote: > > I'm not comfortable for ThreadInfo to expose the implementation > details. What should we specify w.r.t. Java Thread mapping to native > thread which is platform dependent. Also how does it relate to the > future fibers [1]? > > Another alternative is for JDK specific diagnostic tools to do that > mapping for you for example jcmd, rather than exposing it in the API. > I assume is that this is more about troubleshooting than on-going VM > monitoring. > > Mandy [1] http://openjdk.java.net/projects/loom/ > > > > On 2/21/18 2:40 PM, Jeremy Manson wrote: >> Hey folks, >> >> I mentioned earlier in the thread about the ThreadInfo.from() bug >> that I found this because I was looking at fixing JDK-8154176, >> which proposes introducing native thread ids to Thread and >> ThreadInfo. >> >> https://bugs.openjdk.java.net/browse/JDK-8154176 >> >> >> I have a prototype for it. I have a couple of questions, though: >> >> 0) Does anyone object to this being done or my doing it? I see >> that it already has an owner. >> >> 1) Should the ID be a long? The Hotspot thread dump prints it out >> as 0x%x, which is an unsigned hexadecimal integer. The type hiding >> behind it is platform-dependent, though: a pid_t on Linux, an >> unsigned long on Windows, a thread_t on Solaris. I could make it a >> String instead... >> >> Jeremy > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From harsha.wardhana.b at oracle.com Thu Feb 22 03:56:56 2018 From: harsha.wardhana.b at oracle.com (Harsha Wardhana B) Date: Thu, 22 Feb 2018 09:26:56 +0530 Subject: RFR: JDK-8187498: Add a -Xmanagement flag as syntactic sugar for -Dcom.sun.management.jmxremote.* properties In-Reply-To: References: <0fca13c3-1b8f-947c-9041-bdea87b04ba8@oracle.com> <5A687F75.3050404@oracle.com> <05c2e5af-b9d0-0386-4565-ade1179a1d49@oracle.com> <7a18b9d5-3a59-8132-2c4f-ba5de35bfa1d@oracle.com> <0e019151-ef48-b4f4-d253-b9da62aa07c3@oracle.com> <248276b1-1589-561f-0718-7bcb1fd578c7@oracle.com> <40cf194e-4d74-3a34-5eb2-1acc5a5abafb@oracle.com> <87cb309b-8764-193a-0447-8ecb741d308d@oracle.com> <66ad342e-f41f-c01b-e515-8100d108e6ef@oracle.com> <0a13fe7c-b7b0-e5d9-1361-bfea19073574@oracle.com> <0332f3ad-1329-da9a-c6a7-192064fb04bb@oracle.com> <9ac62f66-c14d-951f-1a14-ccf8906ecab4@oracle.com> <5A8C33B6.4040206@oracle.com> <7bf813ed-9390-f9e1-b5df-32a4585c2be3@oracle.com> Message-ID: On Wednesday 21 February 2018 09:51 PM, mandy chung wrote: > > > On 2/20/18 9:55 PM, Harsha Wardhana B wrote: >> >> We cannot get rid of specifying options via -D. We have plenty of -D >> flags but very few have short-hand alternative via >> --start-management-agent. If management properties are specified by >> --start-management-agent, the options specified by -D are anyway >> overwritten if specified. > > I have to see the specification of this feature and give further > feedback.?? If the new option does not accept all management > properties, what is the benefit of this new option? The new options makes it easy to start and configure default agent. As of now, users needs to be aware of -D flags which are long and cumbersome. Hence only most frequently used -D flags are given the short-hand alternative to --start-management-agent. Harsha > > Mandy From david.holmes at oracle.com Thu Feb 22 05:21:26 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 22 Feb 2018 15:21:26 +1000 Subject: Thread Native ID Access In-Reply-To: References: Message-ID: <2efaf7ce-a9f5-2be0-4778-dcff12932bc1@oracle.com> Hi Jeremy, I think "thread id" is too platform specific to be exposed in a core Java API like Thread - even as a non-descript String. Maybe through a platform specific MXBean API? For things like /proc//task/ is there a way to say in a way that means "current thread"? That may be a partial solution - or even whole if you could then read back the actual id? David On 22/02/2018 8:40 AM, Jeremy Manson wrote: > Hey folks, > > I mentioned earlier in the thread about the ThreadInfo.from() bug that I > found this because I was looking at fixing JDK-8154176, which proposes > introducing native thread ids to Thread and ThreadInfo. > > https://bugs.openjdk.java.net/browse/JDK-8154176 > > I have a prototype for it.? I have a couple of questions, though: > > 0) Does anyone object to this being done or my doing it?? I see that it > already has an owner. > > 1) Should the ID be a long?? The Hotspot thread dump prints it out as > 0x%x, which is an unsigned hexadecimal integer.? The type hiding behind > it is platform-dependent, though: a pid_t on Linux, an unsigned long on > Windows, a thread_t on Solaris.? I could make it a String instead... > > Jeremy From thomas.stuefe at gmail.com Thu Feb 22 07:16:37 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 22 Feb 2018 08:16:37 +0100 Subject: Thread Native ID Access In-Reply-To: References: Message-ID: Hi Jeremy, On Wed, Feb 21, 2018 at 11:40 PM, Jeremy Manson wrote: > Hey folks, > > I mentioned earlier in the thread about the ThreadInfo.from() bug that I > found this because I was looking at fixing JDK-8154176, which proposes > introducing native thread ids to Thread and ThreadInfo. > > https://bugs.openjdk.java.net/browse/JDK-8154176 > > I have a prototype for it. I have a couple of questions, though: > > 0) Does anyone object to this being done or my doing it? I see that it > already has an owner. > > 1) Should the ID be a long? The Hotspot thread dump prints it out as > 0x%x, which is an unsigned hexadecimal integer. The type hiding behind it > is platform-dependent, though: a pid_t on Linux, an unsigned long on > Windows, a thread_t on Solaris. I could make it a String instead... > > Just to chime in on this, I would be fine with it. We (SAP) maintain AIX, linux ppc and s390. Internally we also have a (closed) port for hpux and AS/400. On all these platforms we have thread ids which can be expressed as numericals. Neither do I know a contemporary platform which does not have that, does anyone? And even if they exist, seeing how reluctant we are to allow new platforms (e.g. see the discussions about letting in the BSD people into the main line), I cannot see these platforms being porting targets soon? But please make it 64bit. At least AIX has 64bit thread ids. That also gives porters a back door which want or have to hide the real thread id by just returning e.g. &Thread (probably not that, but something thread specific which can be represented by a pointer on 64bit platforms). Kind Regards, Thomas Jeremy > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Thu Feb 22 04:12:14 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 22 Feb 2018 14:12:14 +1000 Subject: Remove unused import proposal: in JdpController.java In-Reply-To: References: Message-ID: <21787cec-e1db-6441-911f-beb769029cc5@oracle.com> Hi Andrew, I've filed: https://bugs.openjdk.java.net/browse/JDK-8198539 to clean up your leftover import from JDK-8183123 :) (which was fixed in 10 not 9). If someone from serviceability doesn't pick this up I may be able to. David On 22/02/2018 1:29 AM, Andrew Leonard wrote: > Hi, > I would like to find a sponsor please for this simple tidy up of > JdpController.java ? > It contains an unused "import sun.management.VMManagement;" which is for > a sun specific java.management class which this class used to use prior > to jdk9. > Thanks > Andrew > > diff --git > a/src/jdk.management.agent/share/classes/sun/management/jdp/JdpController.java > b/src/jdk.management.agent/share/classes/sun/management/jdp/JdpController.java > > --- > a/src/jdk.management.agent/share/classes/sun/management/jdp/JdpController.java > > +++ > b/src/jdk.management.agent/share/classes/sun/management/jdp/JdpController.java > > @@ -34,7 +34,6 @@ > ?import java.lang.reflect.Field; > ?import java.lang.reflect.Method; > ?import java.lang.UnsupportedOperationException; > -import sun.management.VMManagement; > > ?/** > ? * JdpController is responsible to create and manage a broadcast loop. > > > > > > Andrew Leonard > Java Runtimes Development > IBM Hursley > IBM United Kingdom Ltd > Phone internal: 245913, external: 01962 815913 > internet email: andrew_m_leonard at uk.ibm.com > > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From christoph.langer at sap.com Thu Feb 22 08:26:49 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 22 Feb 2018 08:26:49 +0000 Subject: Remove unused import proposal: in JdpController.java In-Reply-To: <21787cec-e1db-6441-911f-beb769029cc5@oracle.com> References: <21787cec-e1db-6441-911f-beb769029cc5@oracle.com> Message-ID: Hi Andrew, I'll take this on as I have a similar include cleanup change in my queue for java/util/jar/Attributes.java. Will post a separate review request... Best regards Christoph > -----Original Message----- > From: serviceability-dev [mailto:serviceability-dev- > bounces at openjdk.java.net] On Behalf Of David Holmes > Sent: Donnerstag, 22. Februar 2018 05:12 > To: Andrew Leonard ; serviceability- > dev at openjdk.java.net > Subject: Re: Remove unused import proposal: in JdpController.java > > Hi Andrew, > > I've filed: > > https://bugs.openjdk.java.net/browse/JDK-8198539 > > to clean up your leftover import from JDK-8183123 :) (which was fixed in > 10 not 9). > > If someone from serviceability doesn't pick this up I may be able to. > > David > > On 22/02/2018 1:29 AM, Andrew Leonard wrote: > > Hi, > > I would like to find a sponsor please for this simple tidy up of > > JdpController.java ? > > It contains an unused "import sun.management.VMManagement;" which > is for > > a sun specific java.management class which this class used to use prior > > to jdk9. > > Thanks > > Andrew > > > > diff --git > > > a/src/jdk.management.agent/share/classes/sun/management/jdp/JdpCont > roller.java > > > b/src/jdk.management.agent/share/classes/sun/management/jdp/JdpCont > roller.java > > > > --- > > > a/src/jdk.management.agent/share/classes/sun/management/jdp/JdpCont > roller.java > > > > +++ > > > b/src/jdk.management.agent/share/classes/sun/management/jdp/JdpCont > roller.java > > > > @@ -34,7 +34,6 @@ > > ?import java.lang.reflect.Field; > > ?import java.lang.reflect.Method; > > ?import java.lang.UnsupportedOperationException; > > -import sun.management.VMManagement; > > > > ?/** > > ? * JdpController is responsible to create and manage a broadcast loop. > > > > > > > > > > > > Andrew Leonard > > Java Runtimes Development > > IBM Hursley > > IBM United Kingdom Ltd > > Phone internal: 245913, external: 01962 815913 > > internet email: andrew_m_leonard at uk.ibm.com > > > > > > Unless stated otherwise above: > > IBM United Kingdom Limited - Registered in England and Wales with > number > > 741598. > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 > 3AU From christoph.langer at sap.com Thu Feb 22 08:42:39 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 22 Feb 2018 08:42:39 +0000 Subject: RFR (XS): 8198539: Cleanup of unused imports in java/util/jar/Attributes.java (java.base) and JdpController.java (jdk.management.agent) Message-ID: <58519ccf88ac4517b6186af613e385fd@sap.com> Hi, please review a simple import cleanup fix for java/util/jar/Attributes.java and sun/management/jdp/JdpController.java. Bug: https://bugs.openjdk.java.net/browse/JDK-8198539 Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8198539.0/ Thanks and best regards Christoph -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.stuefe at gmail.com Thu Feb 22 08:54:16 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 22 Feb 2018 09:54:16 +0100 Subject: RFR (XS): 8198539: Cleanup of unused imports in java/util/jar/Attributes.java (java.base) and JdpController.java (jdk.management.agent) In-Reply-To: <58519ccf88ac4517b6186af613e385fd@sap.com> References: <58519ccf88ac4517b6186af613e385fd@sap.com> Message-ID: Hi Christoph, Looks fine. .. Thomas On Feb 22, 2018 09:42, "Langer, Christoph" wrote: > Hi, > > please review a simple import cleanup fix for > java/util/jar/Attributes.java and sun/management/jdp/JdpController.java. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8198539 > Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8198539.0/ > > Thanks and best regards > Christoph > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew_m_leonard at uk.ibm.com Thu Feb 22 09:02:37 2018 From: andrew_m_leonard at uk.ibm.com (Andrew Leonard) Date: Thu, 22 Feb 2018 09:02:37 +0000 Subject: Remove unused import proposal: in JdpController.java In-Reply-To: References: <21787cec-e1db-6441-911f-beb769029cc5@oracle.com> Message-ID: Thank you Christoph Andrew Leonard Java Runtimes Development IBM Hursley IBM United Kingdom Ltd Phone internal: 245913, external: 01962 815913 internet email: andrew_m_leonard at uk.ibm.com From: "Langer, Christoph" To: David Holmes , Andrew Leonard , "serviceability-dev at openjdk.java.net" Date: 22/02/2018 08:26 Subject: RE: Remove unused import proposal: in JdpController.java Hi Andrew, I'll take this on as I have a similar include cleanup change in my queue for java/util/jar/Attributes.java. Will post a separate review request... Best regards > -----Original Message----- > From: serviceability-dev [mailto:serviceability-dev- > bounces at openjdk.java.net] On Behalf Of David Holmes > Sent: Donnerstag, 22. Februar 2018 05:12 > To: Andrew Leonard ; serviceability- > dev at openjdk.java.net > Subject: Re: Remove unused import proposal: in JdpController.java > > Hi Andrew, > > I've filed: > > https://urldefense.proofpoint.com/v2/url?u=https-3A__bugs.openjdk.java.net_browse_JDK-2D8198539&d=DwIFAw&c=jf_iaSHvJObTbx-siA1ZOg&r=NaV8Iy8Ld-vjpXZFDdTbgGlRTghGHnwM75wUPd5_NUQ&m=ViGiFidumq45NMcWmohOp_xM59zUHYwEnA94bbvXpvA&s=tp8_Q0Fm7qSARlXkA1Qew-9-vQ_rXuKic8vN8-SXlf0&e= > > to clean up your leftover import from JDK-8183123 :) (which was fixed in > 10 not 9). > > If someone from serviceability doesn't pick this up I may be able to. > > David > > On 22/02/2018 1:29 AM, Andrew Leonard wrote: > > Hi, > > I would like to find a sponsor please for this simple tidy up of > > JdpController.java ? > > It contains an unused "import sun.management.VMManagement;" which > is for > > a sun specific java.management class which this class used to use prior > > to jdk9. > > Thanks > > Andrew > > > > diff --git > > > a/src/jdk.management.agent/share/classes/sun/management/jdp/JdpCont > roller.java > > > b/src/jdk.management.agent/share/classes/sun/management/jdp/JdpCont > roller.java > > > > --- > > > a/src/jdk.management.agent/share/classes/sun/management/jdp/JdpCont > roller.java > > > > +++ > > > b/src/jdk.management.agent/share/classes/sun/management/jdp/JdpCont > roller.java > > > > @@ -34,7 +34,6 @@ > > import java.lang.reflect.Field; > > import java.lang.reflect.Method; > > import java.lang.UnsupportedOperationException; > > -import sun.management.VMManagement; > > > > /** > > * JdpController is responsible to create and manage a broadcast loop. > > > > > > > > > > > > Andrew Leonard > > Java Runtimes Development > > IBM Hursley > > IBM United Kingdom Ltd > > Phone internal: 245913, external: 01962 815913 > > internet email: andrew_m_leonard at uk.ibm.com > > > > > > Unless stated otherwise above: > > IBM United Kingdom Limited - Registered in England and Wales with > number > > 741598. > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 > 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew_m_leonard at uk.ibm.com Thu Feb 22 09:03:17 2018 From: andrew_m_leonard at uk.ibm.com (Andrew Leonard) Date: Thu, 22 Feb 2018 09:03:17 +0000 Subject: RFR (XS): 8198539: Cleanup of unused imports in java/util/jar/Attributes.java (java.base) and JdpController.java (jdk.management.agent) In-Reply-To: <58519ccf88ac4517b6186af613e385fd@sap.com> References: <58519ccf88ac4517b6186af613e385fd@sap.com> Message-ID: looks good Cheers Andrew Leonard Java Runtimes Development IBM Hursley IBM United Kingdom Ltd Phone internal: 245913, external: 01962 815913 internet email: andrew_m_leonard at uk.ibm.com From: "Langer, Christoph" To: OpenJDK Dev list , "serviceability-dev at openjdk.java.net" Cc: "andrew_m_leonard at uk.ibm.com" Date: 22/02/2018 08:42 Subject: RFR (XS): 8198539: Cleanup of unused imports in java/util/jar/Attributes.java (java.base) and JdpController.java (jdk.management.agent) Hi, please review a simple import cleanup fix for java/util/jar/Attributes.java and sun/management/jdp/JdpController.java. Bug: https://bugs.openjdk.java.net/browse/JDK-8198539 Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8198539.0/ Thanks and best regards Christoph Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -------------- next part -------------- An HTML attachment was scrubbed... URL: From christoph.langer at sap.com Thu Feb 22 10:21:04 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 22 Feb 2018 10:21:04 +0000 Subject: 8u: RFR(S): 8197943: Unable to use JDWP API in JDK 8 to debug JDK 9 VM Message-ID: Hi JDK 8 reviewers, I'd like to propose a fix for a backport of JDI changes that came with modularization. Egor has brought this up in 8u-dev: http://mail.openjdk.java.net/pipermail/jdk8u-dev/2018-February/007230.html The fix is straightforward and is used by Egor in JetBrains, as well as we at SAP have this already patched for a while in our JDK8 port. Bug: https://bugs.openjdk.java.net/browse/JDK-8197943 Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8197943.0/ The build went through fine and jtreg tests are running. @Stephen: In the bug I read that you wanted to backport this. I hope this matches your intentions. Otherwise I'd step back and wait for your proposal... Best regards Christoph -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Thu Feb 22 15:24:48 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 22 Feb 2018 15:24:48 +0000 Subject: Thread Native ID Access In-Reply-To: References: Message-ID: <00cead79-3914-4ad7-5233-849d13257981@oracle.com> On 21/02/2018 22:40, Jeremy Manson wrote: > Hey folks, > > I mentioned earlier in the thread about the ThreadInfo.from() bug that > I found this because I was looking at fixing JDK-8154176, which > proposes introducing native thread ids to Thread and ThreadInfo. > > https://bugs.openjdk.java.net/browse/JDK-8154176 > > I have a prototype for it.? I have a couple of questions, though: > > 0) Does anyone object to this being done or my doing it?? I see that > it already has an owner. > > 1) Should the ID be a long?? The Hotspot thread dump prints it out as > 0x%x, which is an unsigned hexadecimal integer.? The type hiding > behind it is platform-dependent, though: a pid_t on Linux, an unsigned > long on Windows, a thread_t on Solaris.? I could make it a String > instead... > Mandy mentioned fibers in one of the replies and I think we do need to be cautious about exposing native thread IDs in the Java SE APIs until we have a better sense how some of these APIs will work. I expect there will be a Thread object per fiber but I'm less sure on ThreadMXBean. It may be that ThreadMXBean's dumpAllThreads returns the ThreadInfos for just the kernels threads, the equivalent of today where it only returns the ThreadInfos for the started threads, but once the project is further along then maybe a different choice might emerge. If there is pressing need then could we just expose it it in the JDK-specific com.sun.management.ThreadMXBean API instead? -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeremymanson at google.com Thu Feb 22 18:00:41 2018 From: jeremymanson at google.com (Jeremy Manson) Date: Thu, 22 Feb 2018 10:00:41 -0800 Subject: Thread Native ID Access In-Reply-To: <00cead79-3914-4ad7-5233-849d13257981@oracle.com> References: <00cead79-3914-4ad7-5233-849d13257981@oracle.com> Message-ID: There isn't a pressing need. I wrote this patch almost 5 years ago. We can do what we've always done and keep the patch locally. It's just more work for us to keep forward porting it, and loss of potential benefit to the community. I can certainly understand not wanting this in the high-value real estate in java.lang.Thread. I am reasonably happy just to add it to ThreadInfo, or, if we think even that's too much, just to com.sun.management.ThreadMXBean. Since Alan cc'd loom-dev, I'll reiterate what I said before: IMO, for the purposes of this feature, it doesn't matter how fibers map to Java Threads, or how Java Threads map to OS threads. As long as a Java Thread is running on some OS thread (and nothing that we do in the JVM will change that), this proposed API can return whatever OS thread the Java Thread currently maps to. David asked the following question: For things like /proc//task/ is there a way to say in a > way that means "current thread"? That may be a partial solution - or even > whole if you could then read back the actual id? > Linux 3.17 and greater has /proc/thread-self. But I'm not sure that's a solution - if I want to use this to generate a list of threads with Java names and OS-level state (which often differs from JVM thread state), then I'm not sure how I would do that. Mandy says this: > - other than VM threads and Java to native OS threads mapping, any other > items on your list for monitoring? > > There is a bunch of stuff - the ongoing thread with JC Beyler about our heap profiler is the current thing on the front burner. The only thing that is relevant to thread monitoring is the aforementioned list of VM threads, but that's not a point change to ThreadInfo. For some of the other stuff, we need to do more due diligence on our end or wait and see what you are doing before pushing it. For example, we have programmatic access to JFR event based GC history, but we're waiting to see how the open-sourcing goes on that before pushing. We have TSan support, which we've mentioned in various forums before. We let people register native callbacks on allocation sites, but that's likely to get folded into the heap profiling work. We let monitoring tools look at the code cache directly to avoid JVMTI overhead, but that's pretty dangerous, so I'm not sure that it's a great idea as-is. We have a bunch of extra entries in perfdata, but we should probably review them to make sure we still care about all of them. We also have some bug fixes and performance improvements, and we should probably just send those in. Jeremy On Thu, Feb 22, 2018 at 7:24 AM, Alan Bateman wrote: > On 21/02/2018 22:40, Jeremy Manson wrote: > > Hey folks, > > I mentioned earlier in the thread about the ThreadInfo.from() bug that I > found this because I was looking at fixing JDK-8154176, which proposes > introducing native thread ids to Thread and ThreadInfo. > > https://bugs.openjdk.java.net/browse/JDK-8154176 > > I have a prototype for it. I have a couple of questions, though: > > 0) Does anyone object to this being done or my doing it? I see that it > already has an owner. > > 1) Should the ID be a long? The Hotspot thread dump prints it out as > 0x%x, which is an unsigned hexadecimal integer. The type hiding behind it > is platform-dependent, though: a pid_t on Linux, an unsigned long on > Windows, a thread_t on Solaris. I could make it a String instead... > > Mandy mentioned fibers in one of the replies and I think we do need to be > cautious about exposing native thread IDs in the Java SE APIs until we have > a better sense how some of these APIs will work. I expect there will be a > Thread object per fiber but I'm less sure on ThreadMXBean. It may be that > ThreadMXBean's dumpAllThreads returns the ThreadInfos for just the kernels > threads, the equivalent of today where it only returns the ThreadInfos for > the started threads, but once the project is further along then maybe a > different choice might emerge. > > If there is pressing need then could we just expose it it in the > JDK-specific com.sun.management.ThreadMXBean API instead? > > -Alan > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ecki at zusammenkunft.net Thu Feb 22 19:07:31 2018 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Thu, 22 Feb 2018 20:07:31 +0100 Subject: Thread Native ID Access In-Reply-To: References: <00cead79-3914-4ad7-5233-849d13257981@oracle.com> Message-ID: <5a8f14f3.ac82df0a.6097c.7509@mx.google.com> Hello, one Option would be to use nid as tid on platforms where the datatype is compatible. Thread#getId() is ?positive Long?. That should work(?) at least on Windows and Linux, it will actually reduce some Overhead and make the identifier more useful for Debugging without adding a new API. With the thread-name Counter not being the tid there are way too many different identifiers anyway. Gruss Bernd -- http://bernd.eckenfels.net Von: Jeremy Manson Gesendet: Donnerstag, 22. Februar 2018 20:01 An: Alan Bateman Cc: serviceability-dev at openjdk.java.net; loom-dev at openjdk.java.net Betreff: Re: Thread Native ID Access There isn't a pressing need.? I wrote this patch almost 5 years ago.? We can do what we've always done and keep the patch locally.? It's just more work for us to keep forward porting it, and loss of potential benefit to the community. I can certainly understand not wanting this in the high-value real estate in java.lang.Thread.? I am reasonably happy just to add it to?ThreadInfo, or, if we think even that's too much,?just to com.sun.management.ThreadMXBean. Since Alan cc'd loom-dev, I'll reiterate what I said before: IMO, for the purposes of this feature, it doesn't matter how fibers map to Java Threads, or how Java Threads map to OS threads.? As long as a Java Thread is running on some OS thread (and nothing that we do in the JVM will change that), this proposed API can return whatever OS thread the Java Thread currently maps to. David asked the following question: For things like? /proc//task/ is there a way to say in a way that means "current thread"? That may be a partial solution - or even whole if you could then read back the actual id? Linux 3.17 and greater has /proc/thread-self.? But I'm not sure that's a solution - if I want to use this to generate a list of threads with Java names and OS-level state (which often differs from JVM thread state), then I'm not sure how I would do that. Mandy says this: - other than VM threads and Java to native OS threads mapping, any other items on your list for monitoring? There is a bunch of stuff - the ongoing thread with JC Beyler about our heap profiler is the current thing on the front burner.? The only thing that is relevant to thread monitoring is the aforementioned list of VM threads, but that's not a point change to ThreadInfo. For some of the other stuff, we need to do more due diligence on our end or wait and see what you are doing before pushing it.? For example, we have programmatic access to JFR event based GC history, but we're waiting to see how the open-sourcing goes on that before pushing.? We have TSan support, which we've mentioned in various forums before.? We let people register native callbacks on allocation sites, but that's likely to get folded into the heap profiling work.? We let monitoring tools look at the code cache directly to avoid JVMTI overhead, but that's pretty dangerous, so I'm not sure that it's a great idea as-is.? We have a bunch of extra entries in perfdata, but we should probably review them to make sure we still care about all of them. We also have some bug fixes and performance improvements, and we should probably just send those in. Jeremy On Thu, Feb 22, 2018 at 7:24 AM, Alan Bateman wrote: On 21/02/2018 22:40, Jeremy Manson wrote: Hey folks, I mentioned earlier in the thread about the ThreadInfo.from() bug that I found this because I was looking at fixing?JDK-8154176, which proposes introducing native thread ids to Thread and ThreadInfo. https://bugs.openjdk.java.net/browse/JDK-8154176 I have a prototype for it.? I have a couple of questions, though: 0) Does anyone object to this being done or my doing it?? I see that it already has an owner. 1) Should the ID be a long?? The Hotspot thread dump prints it out as 0x%x, which is an unsigned hexadecimal integer.? The type hiding behind it is platform-dependent, though: a pid_t on Linux, an unsigned long on Windows, a thread_t on Solaris.? I could make it a String instead... Mandy mentioned fibers in one of the replies and I think we do need to be cautious about exposing native thread IDs in the Java SE APIs until we have a better sense how some of these APIs will work. I expect there will be a Thread object per fiber but I'm less sure on ThreadMXBean. It may be that ThreadMXBean's dumpAllThreads returns the ThreadInfos for just the kernels threads, the equivalent of today where it only returns the ThreadInfos for the started threads, but once the project is further along then maybe a different choice might emerge. If there is pressing need then could we just expose it it in the JDK-specific com.sun.management.ThreadMXBean API instead? -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeremymanson at google.com Thu Feb 22 19:15:15 2018 From: jeremymanson at google.com (Jeremy Manson) Date: Thu, 22 Feb 2018 11:15:15 -0800 Subject: Thread Native ID Access In-Reply-To: <5a8f14f3.ac82df0a.6097c.7509@mx.google.com> References: <00cead79-3914-4ad7-5233-849d13257981@oracle.com> <5a8f14f3.ac82df0a.6097c.7509@mx.google.com> Message-ID: Bernd, The thread ID needs to be unique, so that approach might actually cause problems with fibers, depending on their implementation. Jeremy On Thu, Feb 22, 2018 at 11:07 AM, Bernd Eckenfels wrote: > Hello, > > > > one Option would be to use nid as tid on platforms where the datatype is > compatible. Thread#getId() is ?positive Long?. That should work(?) at least > on Windows and Linux, it will actually reduce some Overhead and make the > identifier more useful for Debugging without adding a new API. With the > thread-name Counter not being the tid there are way too many different > identifiers anyway. > > > > Gruss > > Bernd > > -- > http://bernd.eckenfels.net > > > > *Von: *Jeremy Manson > *Gesendet: *Donnerstag, 22. Februar 2018 20:01 > *An: *Alan Bateman > *Cc: *serviceability-dev at openjdk.java.net; loom-dev at openjdk.java.net > *Betreff: *Re: Thread Native ID Access > > > > There isn't a pressing need. I wrote this patch almost 5 years ago. We > can do what we've always done and keep the patch locally. It's just more > work for us to keep forward porting it, and loss of potential benefit to > the community. > > > > I can certainly understand not wanting this in the high-value real estate > in java.lang.Thread. I am reasonably happy just to add it to ThreadInfo, > or, if we think even that's too much, just to com.sun.management. > ThreadMXBean. > > > > Since Alan cc'd loom-dev, I'll reiterate what I said before: IMO, for the > purposes of this feature, it doesn't matter how fibers map to Java Threads, > or how Java Threads map to OS threads. As long as a Java Thread is running > on some OS thread (and nothing that we do in the JVM will change that), > this proposed API can return whatever OS thread the Java Thread currently > maps to. > > > > David asked the following question: > > > > For things like /proc//task/ is there a way to say in a > way that means "current thread"? That may be a partial solution - or even > whole if you could then read back the actual id? > > > > Linux 3.17 and greater has /proc/thread-self. But I'm not sure that's a > solution - if I want to use this to generate a list of threads with Java > names and OS-level state (which often differs from JVM thread state), then > I'm not sure how I would do that. > > > > Mandy says this: > > - other than VM threads and Java to native OS threads mapping, any other > > items on your list for monitoring? > > > > There is a bunch of stuff - the ongoing thread with JC Beyler about our > heap profiler is the current thing on the front burner. The only thing > that is relevant to thread monitoring is the aforementioned list of VM > threads, but that's not a point change to ThreadInfo. > > > > For some of the other stuff, we need to do more due diligence on our end > or wait and see what you are doing before pushing it. For example, we have > programmatic access to JFR event based GC history, but we're waiting to see > how the open-sourcing goes on that before pushing. We have TSan support, > which we've mentioned in various forums before. We let people register > native callbacks on allocation sites, but that's likely to get folded into > the heap profiling work. We let monitoring tools look at the code cache > directly to avoid JVMTI overhead, but that's pretty dangerous, so I'm not > sure that it's a great idea as-is. We have a bunch of extra entries in > perfdata, but we should probably review them to make sure we still care > about all of them. > > > > We also have some bug fixes and performance improvements, and we should > probably just send those in. > > > > Jeremy > > > > On Thu, Feb 22, 2018 at 7:24 AM, Alan Bateman > wrote: > > On 21/02/2018 22:40, Jeremy Manson wrote: > > Hey folks, > > > > I mentioned earlier in the thread about the ThreadInfo.from() bug that I > found this because I was looking at fixing JDK-8154176, which proposes > introducing native thread ids to Thread and ThreadInfo. > > > > https://bugs.openjdk.java.net/browse/JDK-8154176 > > > > I have a prototype for it. I have a couple of questions, though: > > > > 0) Does anyone object to this being done or my doing it? I see that it > already has an owner. > > > > 1) Should the ID be a long? The Hotspot thread dump prints it out as > 0x%x, which is an unsigned hexadecimal integer. The type hiding behind it > is platform-dependent, though: a pid_t on Linux, an unsigned long on > Windows, a thread_t on Solaris. I could make it a String instead... > > > > Mandy mentioned fibers in one of the replies and I think we do need to be > cautious about exposing native thread IDs in the Java SE APIs until we have > a better sense how some of these APIs will work. I expect there will be a > Thread object per fiber but I'm less sure on ThreadMXBean. It may be that > ThreadMXBean's dumpAllThreads returns the ThreadInfos for just the kernels > threads, the equivalent of today where it only returns the ThreadInfos for > the started threads, but once the project is further along then maybe a > different choice might emerge. > > If there is pressing need then could we just expose it it in the > JDK-specific com.sun.management.ThreadMXBean API instead? > > -Alan > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ecki at zusammenkunft.net Thu Feb 22 19:19:01 2018 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Thu, 22 Feb 2018 20:19:01 +0100 Subject: Thread Native ID Access In-Reply-To: References: <00cead79-3914-4ad7-5233-849d13257981@oracle.com> <5a8f14f3.ac82df0a.6097c.7509@mx.google.com> Message-ID: <5a8f17a5.3886df0a.d31d9.5e8a@mx.google.com> Agreed, when those hit the Code they will require a different Change. (Which does not make it less useful for the current Code). Probably the ID is from the days of green threads anyway. ?? -- http://bernd.eckenfels.net Von: Jeremy Manson Gesendet: Donnerstag, 22. Februar 2018 20:15 An: Bernd Eckenfels Cc: Alan Bateman; serviceability-dev at openjdk.java.net; loom-dev at openjdk.java.net Betreff: Re: Thread Native ID Access Bernd, The thread ID needs to be unique, so that approach might actually cause problems with fibers, depending on their implementation. Jeremy On Thu, Feb 22, 2018 at 11:07 AM, Bernd Eckenfels wrote: Hello, ? one Option would be to use nid as tid on platforms where the datatype is compatible. Thread#getId() is ?positive Long?. That should work(?) at least on Windows and Linux, it will actually reduce some Overhead and make the identifier more useful for Debugging without adding a new API. With the thread-name Counter not being the tid there are way too many different identifiers anyway. ? Gruss Bernd -- http://bernd.eckenfels.net ? Von: Jeremy Manson Gesendet: Donnerstag, 22. Februar 2018 20:01 An: Alan Bateman Cc: serviceability-dev at openjdk.java.net; loom-dev at openjdk.java.net Betreff: Re: Thread Native ID Access ? There isn't a pressing need.? I wrote this patch almost 5 years ago.? We can do what we've always done and keep the patch locally.? It's just more work for us to keep forward porting it, and loss of potential benefit to the community. ? I can certainly understand not wanting this in the high-value real estate in java.lang.Thread.? I am reasonably happy just to add it to?ThreadInfo, or, if we think even that's too much,?just to com.sun.management.ThreadMXBean. ? Since Alan cc'd loom-dev, I'll reiterate what I said before: IMO, for the purposes of this feature, it doesn't matter how fibers map to Java Threads, or how Java Threads map to OS threads.? As long as a Java Thread is running on some OS thread (and nothing that we do in the JVM will change that), this proposed API can return whatever OS thread the Java Thread currently maps to. ? David asked the following question: ? For things like? /proc//task/ is there a way to say in a way that means "current thread"? That may be a partial solution - or even whole if you could then read back the actual id? ? Linux 3.17 and greater has /proc/thread-self.? But I'm not sure that's a solution - if I want to use this to generate a list of threads with Java names and OS-level state (which often differs from JVM thread state), then I'm not sure how I would do that. ? Mandy says this: - other than VM threads and Java to native OS threads mapping, any other ? items on your list for monitoring? ? There is a bunch of stuff - the ongoing thread with JC Beyler about our heap profiler is the current thing on the front burner.? The only thing that is relevant to thread monitoring is the aforementioned list of VM threads, but that's not a point change to ThreadInfo. ? For some of the other stuff, we need to do more due diligence on our end or wait and see what you are doing before pushing it.? For example, we have programmatic access to JFR event based GC history, but we're waiting to see how the open-sourcing goes on that before pushing.? We have TSan support, which we've mentioned in various forums before.? We let people register native callbacks on allocation sites, but that's likely to get folded into the heap profiling work.? We let monitoring tools look at the code cache directly to avoid JVMTI overhead, but that's pretty dangerous, so I'm not sure that it's a great idea as-is.? We have a bunch of extra entries in perfdata, but we should probably review them to make sure we still care about all of them. ? We also have some bug fixes and performance improvements, and we should probably just send those in. ? Jeremy ? On Thu, Feb 22, 2018 at 7:24 AM, Alan Bateman wrote: On 21/02/2018 22:40, Jeremy Manson wrote: Hey folks, ? I mentioned earlier in the thread about the ThreadInfo.from() bug that I found this because I was looking at fixing?JDK-8154176, which proposes introducing native thread ids to Thread and ThreadInfo. ? https://bugs.openjdk.java.net/browse/JDK-8154176 ? I have a prototype for it.? I have a couple of questions, though: ? 0) Does anyone object to this being done or my doing it?? I see that it already has an owner. ? 1) Should the ID be a long?? The Hotspot thread dump prints it out as 0x%x, which is an unsigned hexadecimal integer.? The type hiding behind it is platform-dependent, though: a pid_t on Linux, an unsigned long on Windows, a thread_t on Solaris.? I could make it a String instead... ? Mandy mentioned fibers in one of the replies and I think we do need to be cautious about exposing native thread IDs in the Java SE APIs until we have a better sense how some of these APIs will work. I expect there will be a Thread object per fiber but I'm less sure on ThreadMXBean. It may be that ThreadMXBean's dumpAllThreads returns the ThreadInfos for just the kernels threads, the equivalent of today where it only returns the ThreadInfos for the started threads, but once the project is further along then maybe a different choice might emerge. If there is pressing need then could we just expose it it in the JDK-specific com.sun.management.ThreadMXBean API instead? -Alan ? ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From parvathi.somashekar at oracle.com Thu Feb 22 19:41:23 2018 From: parvathi.somashekar at oracle.com (Paru Somashekar) Date: Thu, 22 Feb 2018 11:41:23 -0800 Subject: RFR 8198533 Remove obsolete JDIScaffold class from repo Message-ID: <5A8F1CE3.5080308@oracle.com> Hi Please review fix for JDK-8198533 While removing obsolete JDIScaffold class, found one more class : NativeInstanceFilter, that needed to be updated to use TestScaffold. Bug : JDK-8198533 Webrev : http://cr.openjdk.java.net/~psomashe/8198533/webrev/ thanks, Paru. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hohensee at amazon.com Thu Feb 22 20:54:47 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Thu, 22 Feb 2018 20:54:47 +0000 Subject: RFR (S): 8195115: G1 Old Gen MemoryPool CollectionUsage.used values don't reflect mixed GC results In-Reply-To: <3E15AAD8-5017-4CCB-B927-6B31FD0D7809@amazon.com> References: <3A67BF1B-CE35-4759-B5B4-959C24020A45@amazon.com> <22ceea57-7d27-8350-4457-21f765cb3d0f@oracle.com> <878D6678-84CB-471E-B126-AF14B1BC8EB9@amazon.com> <727f9bcb-9404-df32-917f-70aca6e31cb6@oracle.com> <4BA32239-D645-4B1A-A58E-502258C8FFB7@amazon.com> <997a766f-9f4b-f192-ae87-03f4c8d2666d@oracle.com> <64793ff3-4805-79f9-4e72-6e469cd511dc@oracle.com> <15754d34-f3c7-6431-7abf-05214bd6b9d2@oracle.com> <93417506-8e12-6ad3-690a-439e36719652@oracle.com> <99C937B6-F67B-4F4A-ACC8-444B5F83932B@amazon.com> <3922628B-81DE-4904-AEFE-F4163EB1E655@amazon.com> <8444d766-8adb-013f-f75b-b5d4420635df@oracle.com> <5768E4BF-5556-455B-899B-94A67BF940A0@amazon.com> <3E15AAD8-5017-4CCB-B927-6B31FD0D7809@amazon.com> Message-ID: <22216AD4-9A78-4427-9AB8-629EC685C296@amazon.com> Ping for a review please. Thanks, Paul On 2/16/18, 12:26 PM, "serviceability-dev on behalf of Hohensee, Paul" wrote: The CSR https://bugs.openjdk.java.net/browse/JDK-8196719 for the original fix has been approved, so I?m back to requesting a code review, please. Bug: https://bugs.openjdk.java.net/browse/JDK-8195115 Webrev: http://cr.openjdk.java.net/~phh/8195115/webrev.hs.01/ Passed a submit repo run, passes its jtreg test, and a JDK8 version is in production use at Amazon. From the original RR: > The bug is that from the JMX point of view, G1?s incremental collector > (misnamed as the ?G1 Young Generation? collector) only affects G1?s > survivor and eden spaces. In fact, mixed collections run by this > collector also affect the G1 old generation. > > This proposed fix is to record, for each of a JMX garbage collector's > memory pools, whether that memory pool is affected by all collections > using that collector. And, for each collection, record whether or not > all the collector's memory pools are affected. After each collection, > for each memory pool, if either all the collector's memory pools were > affected or the memory pool is affected for all collections, record > CollectionUsage for that pool. > > For collectors other than G1 Young Generation, all pools are recorded as > affected by all collections and every collection is recorded as > affecting all the collector?s memory pools. For the G1 Young Generation > collector, the G1 Old Gen pool is recorded as not being affected by all > collections, and non-mixed collections are recorded as not affecting all > memory pools. The result is that for non-mixed collections, > CollectionUsage is recorded after a collection only the G1 Eden Space > and G1 Survivor Space pools, while for mixed collections CollectionUsage > is recorded for G1 Old Gen as well. > > Other than the effect of the fix on G1 Old Gen MemoryPool. > CollectionUsage, the only external behavior change is that > GarbageCollectorMXBean.getMemoryPoolNames will now return 3 pool names > rather than 2. > > With this fix, a collector?s memory pools can be divided into two > disjoint subsets, one that participates in all collections and one that > doesn?t. This is a bit more general than the minimum necessary to fix > G1, but not by much. Because I expect it to apply to other incremental > region-based collectors, I went with the more general solution. I > minimized the amount of code I had to touch by using default parameters > for GCMemoryManager::add_pool and the TraceMemoryManagerStats constructors. From chris.plummer at oracle.com Thu Feb 22 21:21:29 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 22 Feb 2018 13:21:29 -0800 Subject: RFR 8198533 Remove obsolete JDIScaffold class from repo In-Reply-To: <5A8F1CE3.5080308@oracle.com> References: <5A8F1CE3.5080308@oracle.com> Message-ID: <2baefa06-79ab-63c5-bbf1-6d1c9ae7e3bf@oracle.com> An HTML attachment was scrubbed... URL: From parvathi.somashekar at oracle.com Thu Feb 22 21:27:47 2018 From: parvathi.somashekar at oracle.com (Paru Somashekar) Date: Thu, 22 Feb 2018 13:27:47 -0800 Subject: RFR 8198533 Remove obsolete JDIScaffold class from repo In-Reply-To: <2baefa06-79ab-63c5-bbf1-6d1c9ae7e3bf@oracle.com> References: <5A8F1CE3.5080308@oracle.com> <2baefa06-79ab-63c5-bbf1-6d1c9ae7e3bf@oracle.com> Message-ID: <5A8F35D3.5000508@oracle.com> Thanks Chris. thanks, Paru. On 2/22/18, 1:21 PM, Chris Plummer wrote: > Hi Paru, > > Looks good. > > Chris > > On 2/22/18 11:41 AM, Paru Somashekar wrote: >> Hi >> >> Please review fix for JDK-8198533 >> >> While removing obsolete JDIScaffold class, found one more class : >> NativeInstanceFilter, that needed to be updated to use TestScaffold. >> >> Bug : JDK-8198533 >> Webrev : http://cr.openjdk.java.net/~psomashe/8198533/webrev/ >> >> >> thanks, >> Paru. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Thu Feb 22 22:22:30 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 22 Feb 2018 14:22:30 -0800 Subject: RFR 8198533 Remove obsolete JDIScaffold class from repo In-Reply-To: <2baefa06-79ab-63c5-bbf1-6d1c9ae7e3bf@oracle.com> References: <5A8F1CE3.5080308@oracle.com> <2baefa06-79ab-63c5-bbf1-6d1c9ae7e3bf@oracle.com> Message-ID: <9a66a0bf-e882-3595-e03e-61222723cd6c@oracle.com> An HTML attachment was scrubbed... URL: From parvathi.somashekar at oracle.com Thu Feb 22 22:36:03 2018 From: parvathi.somashekar at oracle.com (Paru Somashekar) Date: Thu, 22 Feb 2018 14:36:03 -0800 Subject: RFR 8198533 Remove obsolete JDIScaffold class from repo In-Reply-To: <9a66a0bf-e882-3595-e03e-61222723cd6c@oracle.com> References: <5A8F1CE3.5080308@oracle.com> <2baefa06-79ab-63c5-bbf1-6d1c9ae7e3bf@oracle.com> <9a66a0bf-e882-3595-e03e-61222723cd6c@oracle.com> Message-ID: <5A8F45D3.7080301@oracle.com> Thanks Serguei. thanks, Paru. On 2/22/18, 2:22 PM, serguei.spitsyn at oracle.com wrote: > Hi Paru, > > +1 > > Thanks, > Serguei > > On 2/22/18 13:21, Chris Plummer wrote: >> Hi Paru, >> >> Looks good. >> >> Chris >> >> On 2/22/18 11:41 AM, Paru Somashekar wrote: >>> Hi >>> >>> Please review fix for JDK-8198533 >>> >>> While removing obsolete JDIScaffold class, found one more class : >>> NativeInstanceFilter, that needed to be updated to use TestScaffold. >>> >>> Bug : JDK-8198533 >>> Webrev : http://cr.openjdk.java.net/~psomashe/8198533/webrev/ >>> >>> >>> thanks, >>> Paru. >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Fri Feb 23 01:03:55 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 22 Feb 2018 17:03:55 -0800 Subject: RFR(S): 8196992: Resolve disabled warnings for libdt_socket Message-ID: Hello, Please review the following: https://bugs.openjdk.java.net/browse/JDK-8196992 diff --git a/make/lib/Lib-jdk.jdwp.agent.gmk b/make/lib/Lib-jdk.jdwp.agent.gmk --- a/make/lib/Lib-jdk.jdwp.agent.gmk +++ b/make/lib/Lib-jdk.jdwp.agent.gmk @@ -43,7 +43,6 @@ ???? OPTIMIZATION := LOW, \ ???? CFLAGS := $(CFLAGS_JDKLIB) -DUSE_MMAP \ ???????? $(LIBDT_SOCKET_CPPFLAGS), \ -??? DISABLED_WARNINGS_gcc := shift-negative-value, \ ???? MAPFILE := $(TOPDIR)/make/mapfiles/libdt_socket/mapfile-vers, \ ???? LDFLAGS := $(LDFLAGS_JDKLIB) \ ???????? $(call SET_SHARED_LIBRARY_ORIGIN), \ This change is undoing the makefile change done as part of JDK-8196985. The only warning that was turning up in libdt_socket code before JDK-8196985 was done has already been fixed by JDK-8196909. Thus no warnings need to be fixed. After removing the above makefile code, I tested by building with the new toolchain. As a first test I undid the socketTransport.cpp fix from JDK-8196909 to verify that the new toolchain exposed the warning. Then I reverted socketTransport.cpp back to tip sources and saw no warnings with the new toolchain. thanks, Chris From chris.plummer at oracle.com Fri Feb 23 01:16:57 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 22 Feb 2018 17:16:57 -0800 Subject: RFR(XS): 8198585: add asserts to verify that ServiceUtil::visible_oop is not needed Message-ID: Hello, Please review the following: https://bugs.openjdk.java.net/browse/JDK-8198585 http://cr.openjdk.java.net/~cjplummer/8198585/webrev.00/webrev/ Before removing ServiceUtil::visible_oop(), I want to make sure it really isn't needed. Supposedly it should never return false, thus negating the need for its existence. This change adds asserts whenever false is returned. If it makes it all the way through promotion testing, then I'll delete the ServiceUtil::visible_oop() code and the references to it. I tested by running all jdk and hotspot tier1-3 tests. thanks, Chris From mandy.chung at oracle.com Fri Feb 23 02:50:43 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 22 Feb 2018 18:50:43 -0800 Subject: RFR JDK-8198253: ThreadInfo.from(CompositeData) assigning fields incorrectly in JDK 9 In-Reply-To: References: <45a5eee6-023d-e342-d6ed-6797702b8804@oracle.com> <9110e27c-e492-b089-7b02-36459915c779@oracle.com> Message-ID: Hi Jeremy, I have revised the spec of ThreadInfo::from to include "since" column: http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/api/java/lang/management/ThreadInfo.html#from(javax.management.openmbean.CompositeData) I caught a few other issues like mix-n-match older version of StackTraceElement. If you don't mind, I can take this issue and fix a couple of the interop issues. thanks Mandy -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Fri Feb 23 04:49:12 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 23 Feb 2018 14:49:12 +1000 Subject: Thread Native ID Access In-Reply-To: <5a8f17a5.3886df0a.d31d9.5e8a@mx.google.com> References: <00cead79-3914-4ad7-5233-849d13257981@oracle.com> <5a8f14f3.ac82df0a.6097c.7509@mx.google.com> <5a8f17a5.3886df0a.d31d9.5e8a@mx.google.com> Message-ID: <0a2a65b5-b1ac-3967-6a3c-02fd899fde43@oracle.com> On 23/02/2018 5:19 AM, Bernd Eckenfels wrote: > Agreed, when those hit the Code they will require a different Change. > (Which does not make it less useful for the current Code). Probably the > ID is from the days of green threads anyway. ?? No it was added in Java SE 5 as a way to uniquely identify Threads. I don't think this request has any impact on Fibers at all. At any given time a piece of executing Java code is executing on a current Thread, and that current Thread must be running on a native thread (regardless of mapping) and the native thread has an id. The only use for exposing such an id is to allow you to correlate information obtained from inside the VM, with information observed outside, possibly by external tools. David ----- > -- > http://bernd.eckenfels.net > > *Von: *Jeremy Manson > *Gesendet: *Donnerstag, 22. Februar 2018 20:15 > *An: *Bernd Eckenfels > *Cc: *Alan Bateman ; > serviceability-dev at openjdk.java.net > ; loom-dev at openjdk.java.net > > *Betreff: *Re: Thread Native ID Access > > Bernd, > > The thread ID needs to be unique, so that approach might actually cause > problems with fibers, depending on their implementation. > > Jeremy > > On Thu, Feb 22, 2018 at 11:07 AM, Bernd Eckenfels > > wrote: > > Hello, > > one Option would be to use nid as tid on platforms where the > datatype is compatible. Thread#getId() is ?positive Long?. That > should work(?) at least on Windows and Linux, it will actually > reduce some Overhead and make the identifier more useful for > Debugging without adding a new API. With the thread-name Counter not > being the tid there are way too many different identifiers anyway. > > Gruss > > Bernd > > -- > http://bernd.eckenfels.net > > *Von: *Jeremy Manson > *Gesendet: *Donnerstag, 22. Februar 2018 20:01 > *An: *Alan Bateman > *Cc: *serviceability-dev at openjdk.java.net > ; > loom-dev at openjdk.java.net > *Betreff: *Re: Thread Native ID Access > > There isn't a pressing need.? I wrote this patch almost 5 years > ago.? We can do what we've always done and keep the patch locally. > It's just more work for us to keep forward porting it, and loss of > potential benefit to the community. > > I can certainly understand not wanting this in the high-value real > estate in java.lang.Thread.? I am reasonably happy just to add it to > ThreadInfo, or, if we think even that's too much,?just to > com.sun.management.ThreadMXBean. > > Since Alan cc'd loom-dev, I'll reiterate what I said before: IMO, > for the purposes of this feature, it doesn't matter how fibers map > to Java Threads, or how Java Threads map to OS threads.? As long as > a Java Thread is running on some OS thread (and nothing that we do > in the JVM will change that), this proposed API can return whatever > OS thread the Java Thread currently maps to. > > David asked the following question: > > For things like? /proc//task/ is there a way to say > in a way that means "current thread"? That may be a > partial solution - or even whole if you could then read back the > actual id? > > Linux 3.17 and greater has /proc/thread-self.? But I'm not sure > that's a solution - if I want to use this to generate a list of > threads with Java names and OS-level state (which often differs from > JVM thread state), then I'm not sure how I would do that. > > Mandy says this: > > - other than VM threads and Java to native OS threads mapping, > any other > > ? items on your list for monitoring? > > There is a bunch of stuff - the ongoing thread with JC Beyler about > our heap profiler is the current thing on the front burner.? The > only thing that is relevant to thread monitoring is the > aforementioned list of VM threads, but that's not a point change to > ThreadInfo. > > For some of the other stuff, we need to do more due diligence on our > end or wait and see what you are doing before pushing it.? For > example, we have programmatic access to JFR event based GC history, > but we're waiting to see how the open-sourcing goes on that before > pushing.? We have TSan support, which we've mentioned in various > forums before.? We let people register native callbacks on > allocation sites, but that's likely to get folded into the heap > profiling work.? We let monitoring tools look at the code cache > directly to avoid JVMTI overhead, but that's pretty dangerous, so > I'm not sure that it's a great idea as-is.? We have a bunch of extra > entries in perfdata, but we should probably review them to make sure > we still care about all of them. > > We also have some bug fixes and performance improvements, and we > should probably just send those in. > > Jeremy > > On Thu, Feb 22, 2018 at 7:24 AM, Alan Bateman > > wrote: > > On 21/02/2018 22:40, Jeremy Manson wrote: > > Hey folks, > > I mentioned earlier in the thread about the > ThreadInfo.from() bug that I found this because I was > looking at fixing JDK-8154176, which proposes introducing > native thread ids to Thread and ThreadInfo. > > https://bugs.openjdk.java.net/browse/JDK-8154176 > > I have a prototype for it.? I have a couple of questions, > though: > > 0) Does anyone object to this being done or my doing it?? I > see that it already has an owner. > > 1) Should the ID be a long?? The Hotspot thread dump prints > it out as 0x%x, which is an unsigned hexadecimal integer. > The type hiding behind it is platform-dependent, though: a > pid_t on Linux, an unsigned long on Windows, a thread_t on > Solaris.? I could make it a String instead... > > Mandy mentioned fibers in one of the replies and I think we do > need to be cautious about exposing native thread IDs in the Java > SE APIs until we have a better sense how some of these APIs will > work. I expect there will be a Thread object per fiber but I'm > less sure on ThreadMXBean. It may be that ThreadMXBean's > dumpAllThreads returns the ThreadInfos for just the kernels > threads, the equivalent of today where it only returns the > ThreadInfos for the started threads, but once the project is > further along then maybe a different choice might emerge. > > If there is pressing need then could we just expose it it in the > JDK-specific com.sun.management.ThreadMXBean API instead? > > -Alan > From john.r.rose at oracle.com Fri Feb 23 05:04:01 2018 From: john.r.rose at oracle.com (John Rose) Date: Thu, 22 Feb 2018 21:04:01 -0800 Subject: Thread Native ID Access In-Reply-To: <0a2a65b5-b1ac-3967-6a3c-02fd899fde43@oracle.com> References: <00cead79-3914-4ad7-5233-849d13257981@oracle.com> <5a8f14f3.ac82df0a.6097c.7509@mx.google.com> <5a8f17a5.3886df0a.d31d9.5e8a@mx.google.com> <0a2a65b5-b1ac-3967-6a3c-02fd899fde43@oracle.com> Message-ID: <7A1B1422-6D9C-4421-BC02-13DCB97449B9@oracle.com> On Feb 22, 2018, at 8:49 PM, David Holmes wrote: > > I don't think this request has any impact on Fibers at all. At any given time a piece of executing Java code is executing on a current Thread, and that current Thread must be running on a native thread (regardless of mapping) and the native thread has an id. The only use for exposing such an id is to allow you to correlate information obtained from inside the VM, with information observed outside, possibly by external tools. We may or may not bind fiber identity to the legacy java.lang.Thread type. This affects the question of whether the following code can return false: boolean testThreadShift(Runnable r) { Thread t0 = Thread.current(); r.run(); Thread t1 = Thread.current(); return t0 == t1; } This method can return false if (a) Thread.current() reports the identity of the OS thread (not the fiber currently running), and (b) the runnable includes a blocking operation which causes the current fiber to reschedule on a different OS thread before it returns. Having this method return false is, perhaps, surprising enough to cause us to inject fiber identity into java.lang.Thread, so that a java.lang.Thread identity is 1-1 with a fiber, rather than an OS "dinosaur" thread. (You can only pick one!) The alternative is to keep a 1-1 identity between OS threads and java.lang.Thread instances, making a new type which carries fiber identity that multiplexes in a scheduler-dependent way over OS threads. It's an interesting trade-off, with many compatibility concerns. I think we are leaning toward injecting fiber identity into Thread, although this makes me shudder to think of the overheads we will buy with this one move (ThreadLocal hash tables). So, now if you reify the OS thread identity, you could have the same issue again. Can this routine return false? boolean testNTIDShift(Runnable r) { long t0 = NativeThreadID.current(); r.run(); long t1 = NativeThreadID.current(); return t0 == t1; } I think, if you are digging down all the way to the native thread ID, this routine *must* be allowed to return false under a Loom JVM. There's no credible way to assign unique int64_t IDs to fiber objects in the Java heap, as if they were OS objects. So my question is, do the proposed use cases for NTID allow for testNTIDShift to return false? If not, then it's pretty easy to say that we don't want to supply NativeThreadID.current() for general use, just to people implementing schedulers. ? John -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Fri Feb 23 05:16:54 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 23 Feb 2018 15:16:54 +1000 Subject: Thread Native ID Access In-Reply-To: <7A1B1422-6D9C-4421-BC02-13DCB97449B9@oracle.com> References: <00cead79-3914-4ad7-5233-849d13257981@oracle.com> <5a8f14f3.ac82df0a.6097c.7509@mx.google.com> <5a8f17a5.3886df0a.d31d9.5e8a@mx.google.com> <0a2a65b5-b1ac-3967-6a3c-02fd899fde43@oracle.com> <7A1B1422-6D9C-4421-BC02-13DCB97449B9@oracle.com> Message-ID: On 23/02/2018 3:04 PM, John Rose wrote: > On Feb 22, 2018, at 8:49 PM, David Holmes > wrote: >> >> I don't think this request has any impact on Fibers at all. At any >> given time a piece of executing Java code is executing on a current >> Thread, and that current Thread must be running on a native thread >> (regardless of mapping) and the native thread has an id. The only use >> for exposing such an id is to allow you to correlate information >> obtained from inside the VM, with information observed outside, >> possibly by external tools. > > We may or may not bind fiber identity to the legacy java.lang.Thread type. I'm not assuming that exposing native thread ids implies anything about the binding between those entities and anything higher-level. I have no issue with a Fiber/Continuation/Task/whatever running on different java.lang.Threads over its lifetime. For that matter I don't think it matters if java.lang.Threads could execute on different native threads (they don't but they could). David ----- > This affects the question of whether the following code can return false: > > boolean testThreadShift(Runnable r) { > ? Thread t0 = Thread.current(); > ? r.run(); > ? Thread t1 = Thread.current(); > ? return t0 == t1; > } > > This method can return false if (a) Thread.current() reports the identity > of the OS thread (not the fiber currently running), and (b) the runnable > includes a blocking operation which causes the current fiber to reschedule > on a different OS thread before it returns. > > Having this method return false is, perhaps, surprising enough to cause > us to inject fiber identity into java.lang.Thread, so that a > java.lang.Thread > identity is 1-1 with a fiber, rather than an OS "dinosaur" thread. ?(You can > only pick one!) ?The alternative is to keep a 1-1 identity between OS > threads > and java.lang.Thread instances, making a new type which carries fiber > identity that multiplexes in a scheduler-dependent way over OS threads. > > It's an interesting trade-off, with many compatibility concerns. > I think we are leaning toward injecting fiber identity into Thread, > although this makes me shudder to think of the overheads we > will buy with this one move (ThreadLocal hash tables). > > So, now if you reify the OS thread identity, you could have the same > issue again. ?Can this routine return false? > > boolean testNTIDShift(Runnable r) { > ? long t0 = NativeThreadID.current(); > ? r.run(); > ? long t1 = NativeThreadID.current(); > ? return t0 == t1; > } > > I think, if you are digging down all the way to the native thread ID, > this routine *must* be allowed to return false under a Loom JVM. > There's no credible way to assign unique int64_t IDs to fiber objects > in the Java heap, as if they were OS objects. > > So my question is, do the proposed use cases for NTID allow > for testNTIDShift to return false? ?If not, then it's pretty easy to say > that we don't want to supply NativeThreadID.current() for general > use, just to people implementing schedulers. > > ? John From jeremymanson at google.com Fri Feb 23 05:40:47 2018 From: jeremymanson at google.com (Jeremy Manson) Date: Thu, 22 Feb 2018 21:40:47 -0800 Subject: Thread Native ID Access In-Reply-To: References: <00cead79-3914-4ad7-5233-849d13257981@oracle.com> <5a8f14f3.ac82df0a.6097c.7509@mx.google.com> <5a8f17a5.3886df0a.d31d9.5e8a@mx.google.com> <0a2a65b5-b1ac-3967-6a3c-02fd899fde43@oracle.com> <7A1B1422-6D9C-4421-BC02-13DCB97449B9@oracle.com> Message-ID: I think we're all pretty much agreed that fibers imply native tids can change. I think that, as long as we document that, we're fine. An API where there is probably a worse implementation problem is ThreadMXBean#getThreadCpuTime . Right now, the implementation assumes a 1:1 mapping between OS thread and Java thread. Removing that assumption implies you have to do something like track cpu time every time the scheduler switches you out, which is going to make your switches expensive. Jeremy On Thu, Feb 22, 2018 at 9:16 PM, David Holmes wrote: > On 23/02/2018 3:04 PM, John Rose wrote: > >> On Feb 22, 2018, at 8:49 PM, David Holmes > > wrote: >> >>> >>> I don't think this request has any impact on Fibers at all. At any given >>> time a piece of executing Java code is executing on a current Thread, and >>> that current Thread must be running on a native thread (regardless of >>> mapping) and the native thread has an id. The only use for exposing such an >>> id is to allow you to correlate information obtained from inside the VM, >>> with information observed outside, possibly by external tools. >>> >> >> We may or may not bind fiber identity to the legacy java.lang.Thread type. >> > > I'm not assuming that exposing native thread ids implies anything about > the binding between those entities and anything higher-level. I have no > issue with a Fiber/Continuation/Task/whatever running on different > java.lang.Threads over its lifetime. For that matter I don't think it > matters if java.lang.Threads could execute on different native threads > (they don't but they could). > > David > ----- > > > This affects the question of whether the following code can return false: >> >> boolean testThreadShift(Runnable r) { >> Thread t0 = Thread.current(); >> r.run(); >> Thread t1 = Thread.current(); >> return t0 == t1; >> } >> >> This method can return false if (a) Thread.current() reports the identity >> of the OS thread (not the fiber currently running), and (b) the runnable >> includes a blocking operation which causes the current fiber to reschedule >> on a different OS thread before it returns. >> >> Having this method return false is, perhaps, surprising enough to cause >> us to inject fiber identity into java.lang.Thread, so that a >> java.lang.Thread >> identity is 1-1 with a fiber, rather than an OS "dinosaur" thread. (You >> can >> only pick one!) The alternative is to keep a 1-1 identity between OS >> threads >> and java.lang.Thread instances, making a new type which carries fiber >> identity that multiplexes in a scheduler-dependent way over OS threads. >> >> It's an interesting trade-off, with many compatibility concerns. >> I think we are leaning toward injecting fiber identity into Thread, >> although this makes me shudder to think of the overheads we >> will buy with this one move (ThreadLocal hash tables). >> >> So, now if you reify the OS thread identity, you could have the same >> issue again. Can this routine return false? >> >> boolean testNTIDShift(Runnable r) { >> long t0 = NativeThreadID.current(); >> r.run(); >> long t1 = NativeThreadID.current(); >> return t0 == t1; >> } >> >> I think, if you are digging down all the way to the native thread ID, >> this routine *must* be allowed to return false under a Loom JVM. >> There's no credible way to assign unique int64_t IDs to fiber objects >> in the Java heap, as if they were OS objects. >> >> So my question is, do the proposed use cases for NTID allow >> for testNTIDShift to return false? If not, then it's pretty easy to say >> that we don't want to supply NativeThreadID.current() for general >> use, just to people implementing schedulers. >> >> ? John >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeremymanson at google.com Fri Feb 23 05:43:02 2018 From: jeremymanson at google.com (Jeremy Manson) Date: Thu, 22 Feb 2018 21:43:02 -0800 Subject: RFR JDK-8198253: ThreadInfo.from(CompositeData) assigning fields incorrectly in JDK 9 In-Reply-To: References: <45a5eee6-023d-e342-d6ed-6797702b8804@oracle.com> <9110e27c-e492-b089-7b02-36459915c779@oracle.com> Message-ID: I don't mind at all! I'm happy to have someone who knows the code and wants to deal with all of that CompositeData/CompositeType stuff looking at it. I'll keep an eye out for updates. Jeremy On Thu, Feb 22, 2018 at 6:50 PM, mandy chung wrote: > Hi Jeremy, > > I have revised the spec of ThreadInfo::from to include "since" column: > http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/api/java/lang/management/ThreadInfo.html#from(javax.management.openmbean.CompositeData) > > I caught a few other issues like mix-n-match older version of > StackTraceElement. If you don't mind, I can take this issue > and fix a couple of the interop issues. > > thanks > Mandy > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From christoph.langer at sap.com Fri Feb 23 08:48:46 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 23 Feb 2018 08:48:46 +0000 Subject: RFR(S): 8196992: Resolve disabled warnings for libdt_socket In-Reply-To: References: Message-ID: Looks good, Chris. > -----Original Message----- > From: build-dev [mailto:build-dev-bounces at openjdk.java.net] On Behalf Of > Chris Plummer > Sent: Freitag, 23. Februar 2018 02:04 > To: build-dev at openjdk.java.net build-dev ; > serviceability-dev > Subject: RFR(S): 8196992: Resolve disabled warnings for libdt_socket > > Hello, > > Please review the following: > > https://bugs.openjdk.java.net/browse/JDK-8196992 > > diff --git a/make/lib/Lib-jdk.jdwp.agent.gmk > b/make/lib/Lib-jdk.jdwp.agent.gmk > --- a/make/lib/Lib-jdk.jdwp.agent.gmk > +++ b/make/lib/Lib-jdk.jdwp.agent.gmk > @@ -43,7 +43,6 @@ > ???? OPTIMIZATION := LOW, \ > ???? CFLAGS := $(CFLAGS_JDKLIB) -DUSE_MMAP \ > ???????? $(LIBDT_SOCKET_CPPFLAGS), \ > -??? DISABLED_WARNINGS_gcc := shift-negative-value, \ > ???? MAPFILE := $(TOPDIR)/make/mapfiles/libdt_socket/mapfile-vers, \ > ???? LDFLAGS := $(LDFLAGS_JDKLIB) \ > ???????? $(call SET_SHARED_LIBRARY_ORIGIN), \ > > This change is undoing the makefile change done as part of JDK-8196985. > The only warning that was turning up in libdt_socket code before > JDK-8196985 was done has already been fixed by JDK-8196909. Thus no > warnings need to be fixed. > > After removing the above makefile code, I tested by building with the > new toolchain. As a first test I undid the socketTransport.cpp fix from > JDK-8196909 to verify that the new toolchain exposed the warning. Then I > reverted socketTransport.cpp back to tip sources and saw no warnings > with the new toolchain. > > thanks, > > Chris From christoph.langer at sap.com Fri Feb 23 09:05:06 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 23 Feb 2018 09:05:06 +0000 Subject: RFR (XS): 8198539: Cleanup of unused imports in java/util/jar/Attributes.java (java.base) and JdpController.java (jdk.management.agent) In-Reply-To: References: <58519ccf88ac4517b6186af613e385fd@sap.com> Message-ID: Thanks, Thomas. From: Thomas St?fe [mailto:thomas.stuefe at gmail.com] Sent: Donnerstag, 22. Februar 2018 09:54 To: Langer, Christoph Cc: Java Core Libs ; serviceability-dev at openjdk.java.net; andrew_m_leonard at uk.ibm.com Subject: Re: RFR (XS): 8198539: Cleanup of unused imports in java/util/jar/Attributes.java (java.base) and JdpController.java (jdk.management.agent) Hi Christoph, Looks fine. .. Thomas On Feb 22, 2018 09:42, "Langer, Christoph" > wrote: Hi, please review a simple import cleanup fix for java/util/jar/Attributes.java and sun/management/jdp/JdpController.java. Bug: https://bugs.openjdk.java.net/browse/JDK-8198539 Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8198539.0/ Thanks and best regards Christoph -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefan.karlsson at oracle.com Fri Feb 23 09:58:36 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Fri, 23 Feb 2018 10:58:36 +0100 Subject: RFR(XS): 8198585: add asserts to verify that ServiceUtil::visible_oop is not needed In-Reply-To: References: Message-ID: <64329c64-9b59-9aa7-976b-bff2d5254f2f@oracle.com> Looks good. StefanK On 2018-02-23 02:16, Chris Plummer wrote: > Hello, > > Please review the following: > > https://bugs.openjdk.java.net/browse/JDK-8198585 > http://cr.openjdk.java.net/~cjplummer/8198585/webrev.00/webrev/ > > Before removing ServiceUtil::visible_oop(), I want to make sure it > really isn't needed. Supposedly it should never return false, thus > negating the need for its existence. This change adds asserts whenever > false is returned. If it makes it all the way through promotion > testing, then I'll delete the ServiceUtil::visible_oop() code and the > references to it. > > I tested by running all jdk and hotspot tier1-3 tests. > > thanks, > > Chris From serguei.spitsyn at oracle.com Fri Feb 23 10:15:52 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 23 Feb 2018 02:15:52 -0800 Subject: RFR(S): 8196992: Resolve disabled warnings for libdt_socket In-Reply-To: References: Message-ID: <1253244c-6fcb-19c4-e645-bc7838219428@oracle.com> Chris, This looks good to me too. Thanks, Serguei On 2/23/18 00:48, Langer, Christoph wrote: > Looks good, Chris. > >> -----Original Message----- >> From: build-dev [mailto:build-dev-bounces at openjdk.java.net] On Behalf Of >> Chris Plummer >> Sent: Freitag, 23. Februar 2018 02:04 >> To: build-dev at openjdk.java.net build-dev ; >> serviceability-dev >> Subject: RFR(S): 8196992: Resolve disabled warnings for libdt_socket >> >> Hello, >> >> Please review the following: >> >> https://bugs.openjdk.java.net/browse/JDK-8196992 >> >> diff --git a/make/lib/Lib-jdk.jdwp.agent.gmk >> b/make/lib/Lib-jdk.jdwp.agent.gmk >> --- a/make/lib/Lib-jdk.jdwp.agent.gmk >> +++ b/make/lib/Lib-jdk.jdwp.agent.gmk >> @@ -43,7 +43,6 @@ >> ???? OPTIMIZATION := LOW, \ >> ???? CFLAGS := $(CFLAGS_JDKLIB) -DUSE_MMAP \ >> ???????? $(LIBDT_SOCKET_CPPFLAGS), \ >> -??? DISABLED_WARNINGS_gcc := shift-negative-value, \ >> ???? MAPFILE := $(TOPDIR)/make/mapfiles/libdt_socket/mapfile-vers, \ >> ???? LDFLAGS := $(LDFLAGS_JDKLIB) \ >> ???????? $(call SET_SHARED_LIBRARY_ORIGIN), \ >> >> This change is undoing the makefile change done as part of JDK-8196985. >> The only warning that was turning up in libdt_socket code before >> JDK-8196985 was done has already been fixed by JDK-8196909. Thus no >> warnings need to be fixed. >> >> After removing the above makefile code, I tested by building with the >> new toolchain. As a first test I undid the socketTransport.cpp fix from >> JDK-8196909 to verify that the new toolchain exposed the warning. Then I >> reverted socketTransport.cpp back to tip sources and saw no warnings >> with the new toolchain. >> >> thanks, >> >> Chris From serguei.spitsyn at oracle.com Fri Feb 23 10:14:40 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 23 Feb 2018 02:14:40 -0800 Subject: RFR(XS): 8198585: add asserts to verify that ServiceUtil::visible_oop is not needed In-Reply-To: <64329c64-9b59-9aa7-976b-bff2d5254f2f@oracle.com> References: <64329c64-9b59-9aa7-976b-bff2d5254f2f@oracle.com> Message-ID: Hi Chris, +1 Thanks, Serguei On 2/23/18 01:58, Stefan Karlsson wrote: > Looks good. > > StefanK > > On 2018-02-23 02:16, Chris Plummer wrote: >> Hello, >> >> Please review the following: >> >> https://bugs.openjdk.java.net/browse/JDK-8198585 >> http://cr.openjdk.java.net/~cjplummer/8198585/webrev.00/webrev/ >> >> Before removing ServiceUtil::visible_oop(), I want to make sure it >> really isn't needed. Supposedly it should never return false, thus >> negating the need for its existence. This change adds asserts >> whenever false is returned. If it makes it all the way through >> promotion testing, then I'll delete the ServiceUtil::visible_oop() >> code and the references to it. >> >> I tested by running all jdk and hotspot tier1-3 tests. >> >> thanks, >> >> Chris > > From Alan.Bateman at oracle.com Fri Feb 23 10:57:40 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 23 Feb 2018 10:57:40 +0000 Subject: Thread Native ID Access In-Reply-To: <7A1B1422-6D9C-4421-BC02-13DCB97449B9@oracle.com> References: <00cead79-3914-4ad7-5233-849d13257981@oracle.com> <5a8f14f3.ac82df0a.6097c.7509@mx.google.com> <5a8f17a5.3886df0a.d31d9.5e8a@mx.google.com> <0a2a65b5-b1ac-3967-6a3c-02fd899fde43@oracle.com> <7A1B1422-6D9C-4421-BC02-13DCB97449B9@oracle.com> Message-ID: <5ae327b5-8c89-2cd8-14ed-80525d5b1fa1@oracle.com> On 23/02/2018 05:04, John Rose wrote: > We may or may not bind fiber identity to the legacy java.lang.Thread type. > This affects the question of whether the following code can return false: > > boolean testThreadShift(Runnable r) { > ? Thread t0 = Thread.current(); > ? r.run(); > ? Thread t1 = Thread.current(); > ? return t0 == t1; > } This is an area that I have been spending time recently. A big benefit of a Thread object per fiber, and the above always returning true, is that things that are thread local today (ThreadLocal, thread context class loader, uncaught exception handler, ...) become fiber local without too much effort. It means we have Thread objects that aren't tied to a kernel thread or don't have thread meta data in the VM (the lifecycle of Threads means we do this today anyway). Lots of details of course but it's a direction that potentially allows a lot of existing code to be executed in a fiber without changes. In the java.lang.management API, ThreadMXBean is more tied to kernel threads due to methods that reveal thread CPU counters. A possible direction is to continue with a ThreadMXBean per dinosaur thread, in which case Jeremy's proposal to expose the native thread identifier might be okay. That direction would require sorting out a details with the thread identifier (Thread::getId) as it can be used today to link a Thread to a ThreadMXBean but that shouldn't be too hard. -Alan. From magnus.ihse.bursie at oracle.com Fri Feb 23 12:49:47 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Fri, 23 Feb 2018 13:49:47 +0100 Subject: RFR(S): 8196992: Resolve disabled warnings for libdt_socket In-Reply-To: References: Message-ID: <04b2eccf-02c3-82b7-7042-e12375a1e5b2@oracle.com> On 2018-02-23 02:03, Chris Plummer wrote: > Hello, > > Please review the following: > > https://bugs.openjdk.java.net/browse/JDK-8196992 > > diff --git a/make/lib/Lib-jdk.jdwp.agent.gmk > b/make/lib/Lib-jdk.jdwp.agent.gmk > --- a/make/lib/Lib-jdk.jdwp.agent.gmk > +++ b/make/lib/Lib-jdk.jdwp.agent.gmk > @@ -43,7 +43,6 @@ > ???? OPTIMIZATION := LOW, \ > ???? CFLAGS := $(CFLAGS_JDKLIB) -DUSE_MMAP \ > ???????? $(LIBDT_SOCKET_CPPFLAGS), \ > -??? DISABLED_WARNINGS_gcc := shift-negative-value, \ > ???? MAPFILE := $(TOPDIR)/make/mapfiles/libdt_socket/mapfile-vers, \ > ???? LDFLAGS := $(LDFLAGS_JDKLIB) \ > ???????? $(call SET_SHARED_LIBRARY_ORIGIN), \ Looks good. Thanks for fixing these code quality issues! /Magnus > > This change is undoing the makefile change done as part of > JDK-8196985. The only warning that was turning up in libdt_socket code > before JDK-8196985 was done has already been fixed by JDK-8196909. > Thus no warnings need to be fixed. > > After removing the above makefile code, I tested by building with the > new toolchain. As a first test I undid the socketTransport.cpp fix > from JDK-8196909 to verify that the new toolchain exposed the warning. > Then I reverted socketTransport.cpp back to tip sources and saw no > warnings with the new toolchain. > > thanks, > > Chris > From daniel.daugherty at oracle.com Fri Feb 23 14:45:31 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 23 Feb 2018 09:45:31 -0500 Subject: RFR(XS): 8198585: add asserts to verify that ServiceUtil::visible_oop is not needed In-Reply-To: References: Message-ID: <22b8c46b-4608-6e34-607d-6031e2549173@oracle.com> Instead of "assert(false, ...", I recommend "fatal(...". This will cause a failure in all build configs including 'release' bits. Dan On 2/22/18 8:16 PM, Chris Plummer wrote: > Hello, > > Please review the following: > > https://bugs.openjdk.java.net/browse/JDK-8198585 > http://cr.openjdk.java.net/~cjplummer/8198585/webrev.00/webrev/ > > Before removing ServiceUtil::visible_oop(), I want to make sure it > really isn't needed. Supposedly it should never return false, thus > negating the need for its existence. This change adds asserts whenever > false is returned. If it makes it all the way through promotion > testing, then I'll delete the ServiceUtil::visible_oop() code and the > references to it. > > I tested by running all jdk and hotspot tier1-3 tests. > > thanks, > > Chris From chris.plummer at oracle.com Fri Feb 23 17:12:32 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Fri, 23 Feb 2018 09:12:32 -0800 Subject: RFR(XS): 8198585: add asserts to verify that ServiceUtil::visible_oop is not needed In-Reply-To: <22b8c46b-4608-6e34-607d-6031e2549173@oracle.com> References: <22b8c46b-4608-6e34-607d-6031e2549173@oracle.com> Message-ID: Ok. I'll make that change. thanks, Chris On 2/23/18 6:45 AM, Daniel D. Daugherty wrote: > Instead of "assert(false, ...", I recommend "fatal(...". > This will cause a failure in all build configs including 'release' bits. > > Dan > > > On 2/22/18 8:16 PM, Chris Plummer wrote: >> Hello, >> >> Please review the following: >> >> https://bugs.openjdk.java.net/browse/JDK-8198585 >> http://cr.openjdk.java.net/~cjplummer/8198585/webrev.00/webrev/ >> >> Before removing ServiceUtil::visible_oop(), I want to make sure it >> really isn't needed. Supposedly it should never return false, thus >> negating the need for its existence. This change adds asserts >> whenever false is returned. If it makes it all the way through >> promotion testing, then I'll delete the ServiceUtil::visible_oop() >> code and the references to it. >> >> I tested by running all jdk and hotspot tier1-3 tests. >> >> thanks, >> >> Chris > From hohensee at amazon.com Fri Feb 23 21:07:39 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Fri, 23 Feb 2018 21:07:39 +0000 Subject: Thread Native ID Access In-Reply-To: References: <00cead79-3914-4ad7-5233-849d13257981@oracle.com> <5a8f14f3.ac82df0a.6097c.7509@mx.google.com> <5a8f17a5.3886df0a.d31d9.5e8a@mx.google.com> <0a2a65b5-b1ac-3967-6a3c-02fd899fde43@oracle.com> <7A1B1422-6D9C-4421-BC02-13DCB97449B9@oracle.com> Message-ID: Here?s a concrete proposal on which people may want to comment. We could add a method to ThreadInfo, vis., long getPlatformThreadId() Returns the platform-specific thread ID of the thread associated with this ThreadInfo. The ID has no meaning to the Java runtime and is not guaranteed to have any meaning at all. I.e., it may be a constant value for all threads, or a native address, and may not be the same across multiple invocations. Returns: the platform-specific ID of the associated thread. For a fiber, a reasonable value would be the OS thread id for the OS thread that?s running the fiber at ThreadInfo construction time. If, say, the platform used a string or some other larger-than-long value, it could be a native address or some other value that mapped to one. If we don?t want to add a new method to ThreadInfo, we could define a com.sun.management.ThreadInfo to go with com.sun.management.ThreadMXBean. Parenthetically, to me a fiber is enough different from a thread to warrant a separate MXBean. I.e., a thread is a species of fiber, but not vica versa. So, we shouldn?t gate changes to ThreadMXBean on having a thorough understanding of how to support fibers in JMX. We could also add to com.sun.management.ThreadMXBean the methods getThreadId(long id), getThreadId(long [] ids), getPlatformThreadId(long id), and getPlatformThreadId(long [] ids). These presumably would be fast compared to creating ThreadInfo(s). We don?t want to add Thread.getPlatformId(), so the way to get the current thread?s platform id would be getThreadInfo( Thread.currentThread().getId() ).getPlatformThreadId() which is cumbersome and slow. Instead, by analogy to ThreadMXBean.getCurrentThreadCpuTime(), we could add to com.sun.management.ThreadMXBean the methods getCurrentThreadId() and getCurrentPlatformThreadId(). All this may seem somewhat overkill, but it?s no more so than how we currently create ThreadInfo objects and the various ThreadCpu/UserTime method definitions. Thanks, Paul From: serviceability-dev on behalf of Jeremy Manson Date: Thursday, February 22, 2018 at 9:42 PM To: David Holmes Cc: "serviceability-dev at openjdk.java.net" , "loom-dev at openjdk.java.net" , John Rose Subject: Re: Thread Native ID Access I think we're all pretty much agreed that fibers imply native tids can change. I think that, as long as we document that, we're fine. An API where there is probably a worse implementation problem is ThreadMXBean#getThreadCpuTime. Right now, the implementation assumes a 1:1 mapping between OS thread and Java thread. Removing that assumption implies you have to do something like track cpu time every time the scheduler switches you out, which is going to make your switches expensive. Jeremy On Thu, Feb 22, 2018 at 9:16 PM, David Holmes > wrote: On 23/02/2018 3:04 PM, John Rose wrote: On Feb 22, 2018, at 8:49 PM, David Holmes >> wrote: I don't think this request has any impact on Fibers at all. At any given time a piece of executing Java code is executing on a current Thread, and that current Thread must be running on a native thread (regardless of mapping) and the native thread has an id. The only use for exposing such an id is to allow you to correlate information obtained from inside the VM, with information observed outside, possibly by external tools. We may or may not bind fiber identity to the legacy java.lang.Thread type. I'm not assuming that exposing native thread ids implies anything about the binding between those entities and anything higher-level. I have no issue with a Fiber/Continuation/Task/whatever running on different java.lang.Threads over its lifetime. For that matter I don't think it matters if java.lang.Threads could execute on different native threads (they don't but they could). David ----- This affects the question of whether the following code can return false: boolean testThreadShift(Runnable r) { Thread t0 = Thread.current(); r.run(); Thread t1 = Thread.current(); return t0 == t1; } This method can return false if (a) Thread.current() reports the identity of the OS thread (not the fiber currently running), and (b) the runnable includes a blocking operation which causes the current fiber to reschedule on a different OS thread before it returns. Having this method return false is, perhaps, surprising enough to cause us to inject fiber identity into java.lang.Thread, so that a java.lang.Thread identity is 1-1 with a fiber, rather than an OS "dinosaur" thread. (You can only pick one!) The alternative is to keep a 1-1 identity between OS threads and java.lang.Thread instances, making a new type which carries fiber identity that multiplexes in a scheduler-dependent way over OS threads. It's an interesting trade-off, with many compatibility concerns. I think we are leaning toward injecting fiber identity into Thread, although this makes me shudder to think of the overheads we will buy with this one move (ThreadLocal hash tables). So, now if you reify the OS thread identity, you could have the same issue again. Can this routine return false? boolean testNTIDShift(Runnable r) { long t0 = NativeThreadID.current(); r.run(); long t1 = NativeThreadID.current(); return t0 == t1; } I think, if you are digging down all the way to the native thread ID, this routine *must* be allowed to return false under a Loom JVM. There's no credible way to assign unique int64_t IDs to fiber objects in the Java heap, as if they were OS objects. So my question is, do the proposed use cases for NTID allow for testNTIDShift to return false? If not, then it's pretty easy to say that we don't want to supply NativeThreadID.current() for general use, just to people implementing schedulers. ? John -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeremymanson at google.com Fri Feb 23 23:11:49 2018 From: jeremymanson at google.com (Jeremy Manson) Date: Fri, 23 Feb 2018 15:11:49 -0800 Subject: Thread Native ID Access In-Reply-To: References: <00cead79-3914-4ad7-5233-849d13257981@oracle.com> <5a8f14f3.ac82df0a.6097c.7509@mx.google.com> <5a8f17a5.3886df0a.d31d9.5e8a@mx.google.com> <0a2a65b5-b1ac-3967-6a3c-02fd899fde43@oracle.com> <7A1B1422-6D9C-4421-BC02-13DCB97449B9@oracle.com> Message-ID: Thanks, Paul. This is approximately what I was envisioning when Mandy and David said they didn't really want to expose it in java.lang.Thread. If folks are generally amenable to something like this, I can probably find the spare time to cobble it together. Jeremy On Fri, Feb 23, 2018 at 1:07 PM, Hohensee, Paul wrote: > Here?s a concrete proposal on which people may want to comment. > > > > We could add a method to ThreadInfo, vis., > > > > long getPlatformThreadId() > > > > Returns the platform-specific thread ID of the thread associated with this > ThreadInfo. The ID has no meaning to the Java runtime and is not guaranteed > to have any meaning at all. > > I.e., it may be a constant value for all threads, or a native address, and > may not be the same across multiple invocations. > > > > Returns: > > > > the platform-specific ID of the associated thread. > > > > For a fiber, a reasonable value would be the OS thread id for the OS > thread that?s running the fiber at ThreadInfo construction time. If, say, > the platform used a string or some other larger-than-long value, it could > be a native address or some other value that mapped to one. > > > > If we don?t want to add a new method to ThreadInfo, we could define a > com.sun.management.ThreadInfo to go with com.sun.management.ThreadMXBean. > > > > Parenthetically, to me a fiber is enough different from a thread to > warrant a separate MXBean. I.e., a thread is a species of fiber, but not > vica versa. So, we shouldn?t gate changes to ThreadMXBean on having a > thorough understanding of how to support fibers in JMX. > > > > We could also add to com.sun.management.ThreadMXBean the methods > getThreadId(long id), getThreadId(long [] ids), getPlatformThreadId(long > id), and getPlatformThreadId(long [] ids). These presumably would be fast > compared to creating ThreadInfo(s). > > > > We don?t want to add Thread.getPlatformId(), so the way to get the current > thread?s platform id would be > > > > getThreadInfo( Thread.currentThread().getId() ).getPlatformThreadId() > > > > which is cumbersome and slow. Instead, by analogy to ThreadMXBean.getCurrentThreadCpuTime(), > we could add to com.sun.management.ThreadMXBean the methods > getCurrentThreadId() and getCurrentPlatformThreadId(). > > > > All this may seem somewhat overkill, but it?s no more so than how we > currently create ThreadInfo objects and the various ThreadCpu/UserTime > method definitions. > > > > Thanks, > > > > Paul > > > > *From: *serviceability-dev > on behalf of Jeremy Manson > *Date: *Thursday, February 22, 2018 at 9:42 PM > *To: *David Holmes > *Cc: *"serviceability-dev at openjdk.java.net" java.net>, "loom-dev at openjdk.java.net" , John > Rose > *Subject: *Re: Thread Native ID Access > > > > I think we're all pretty much agreed that fibers imply native tids can > change. I think that, as long as we document that, we're fine. > > > > An API where there is probably a worse implementation problem is > ThreadMXBean#getThreadCpuTime > > . Right now, the implementation assumes a 1:1 mapping between OS thread > and Java thread. Removing that assumption implies you have to do something > like track cpu time every time the scheduler switches you out, which is > going to make your switches expensive. > > > > Jeremy > > > > On Thu, Feb 22, 2018 at 9:16 PM, David Holmes > wrote: > > On 23/02/2018 3:04 PM, John Rose wrote: > > On Feb 22, 2018, at 8:49 PM, David Holmes > wrote: > > > I don't think this request has any impact on Fibers at all. At any given > time a piece of executing Java code is executing on a current Thread, and > that current Thread must be running on a native thread (regardless of > mapping) and the native thread has an id. The only use for exposing such an > id is to allow you to correlate information obtained from inside the VM, > with information observed outside, possibly by external tools. > > > We may or may not bind fiber identity to the legacy java.lang.Thread type. > > > I'm not assuming that exposing native thread ids implies anything about > the binding between those entities and anything higher-level. I have no > issue with a Fiber/Continuation/Task/whatever running on different > java.lang.Threads over its lifetime. For that matter I don't think it > matters if java.lang.Threads could execute on different native threads > (they don't but they could). > > David > ----- > > > > This affects the question of whether the following code can return false: > > boolean testThreadShift(Runnable r) { > Thread t0 = Thread.current(); > r.run(); > Thread t1 = Thread.current(); > return t0 == t1; > } > > This method can return false if (a) Thread.current() reports the identity > of the OS thread (not the fiber currently running), and (b) the runnable > includes a blocking operation which causes the current fiber to reschedule > on a different OS thread before it returns. > > Having this method return false is, perhaps, surprising enough to cause > us to inject fiber identity into java.lang.Thread, so that a > java.lang.Thread > identity is 1-1 with a fiber, rather than an OS "dinosaur" thread. (You > can > only pick one!) The alternative is to keep a 1-1 identity between OS > threads > and java.lang.Thread instances, making a new type which carries fiber > identity that multiplexes in a scheduler-dependent way over OS threads. > > It's an interesting trade-off, with many compatibility concerns. > I think we are leaning toward injecting fiber identity into Thread, > although this makes me shudder to think of the overheads we > will buy with this one move (ThreadLocal hash tables). > > So, now if you reify the OS thread identity, you could have the same > issue again. Can this routine return false? > > boolean testNTIDShift(Runnable r) { > long t0 = NativeThreadID.current(); > r.run(); > long t1 = NativeThreadID.current(); > return t0 == t1; > } > > I think, if you are digging down all the way to the native thread ID, > this routine *must* be allowed to return false under a Loom JVM. > There's no credible way to assign unique int64_t IDs to fiber objects > in the Java heap, as if they were OS objects. > > So my question is, do the proposed use cases for NTID allow > for testNTIDShift to return false? If not, then it's pretty easy to say > that we don't want to supply NativeThreadID.current() for general > use, just to people implementing schedulers. > > ? John > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hohensee at amazon.com Fri Feb 23 23:44:41 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Fri, 23 Feb 2018 23:44:41 +0000 Subject: Thread Native ID Access In-Reply-To: References: <00cead79-3914-4ad7-5233-849d13257981@oracle.com> <5a8f14f3.ac82df0a.6097c.7509@mx.google.com> <5a8f17a5.3886df0a.d31d9.5e8a@mx.google.com> <0a2a65b5-b1ac-3967-6a3c-02fd899fde43@oracle.com> <7A1B1422-6D9C-4421-BC02-13DCB97449B9@oracle.com> Message-ID: <39B8BA33-EE67-47D2-96C8-22D2AAB11AC7@amazon.com> I am. :) The process is to file an RFE and then a CSR associated with it. You file the CSR from the RFE page, not separately. See https://wiki.openjdk.java.net/display/csr/Main https://wiki.openjdk.java.net/display/csr/Fields+of+a+CSR+Request https://wiki.openjdk.java.net/display/csr/CSR+FAQs You need a review of the CSR by a domain expert before you can finalize it for committee review. If you go with com.sun.management versions of ThreadInfo and ThreadMXBean methods, there?s no change in the java.lang.management specification, which is important because the j.l.m interface must be supported by all implementations, which is a higher bar than for c.s.m changes. Paul From: Jeremy Manson Date: Friday, February 23, 2018 at 3:12 PM To: "Hohensee, Paul" Cc: David Holmes , "serviceability-dev at openjdk.java.net" , "loom-dev at openjdk.java.net" , John Rose Subject: Re: Thread Native ID Access Thanks, Paul. This is approximately what I was envisioning when Mandy and David said they didn't really want to expose it in java.lang.Thread. If folks are generally amenable to something like this, I can probably find the spare time to cobble it together. Jeremy On Fri, Feb 23, 2018 at 1:07 PM, Hohensee, Paul > wrote: Here?s a concrete proposal on which people may want to comment. We could add a method to ThreadInfo, vis., long getPlatformThreadId() Returns the platform-specific thread ID of the thread associated with this ThreadInfo. The ID has no meaning to the Java runtime and is not guaranteed to have any meaning at all. I.e., it may be a constant value for all threads, or a native address, and may not be the same across multiple invocations. Returns: the platform-specific ID of the associated thread. For a fiber, a reasonable value would be the OS thread id for the OS thread that?s running the fiber at ThreadInfo construction time. If, say, the platform used a string or some other larger-than-long value, it could be a native address or some other value that mapped to one. If we don?t want to add a new method to ThreadInfo, we could define a com.sun.management.ThreadInfo to go with com.sun.management.ThreadMXBean. Parenthetically, to me a fiber is enough different from a thread to warrant a separate MXBean. I.e., a thread is a species of fiber, but not vica versa. So, we shouldn?t gate changes to ThreadMXBean on having a thorough understanding of how to support fibers in JMX. We could also add to com.sun.management.ThreadMXBean the methods getThreadId(long id), getThreadId(long [] ids), getPlatformThreadId(long id), and getPlatformThreadId(long [] ids). These presumably would be fast compared to creating ThreadInfo(s). We don?t want to add Thread.getPlatformId(), so the way to get the current thread?s platform id would be getThreadInfo( Thread.currentThread().getId() ).getPlatformThreadId() which is cumbersome and slow. Instead, by analogy to ThreadMXBean.getCurrentThreadCpuTime(), we could add to com.sun.management.ThreadMXBean the methods getCurrentThreadId() and getCurrentPlatformThreadId(). All this may seem somewhat overkill, but it?s no more so than how we currently create ThreadInfo objects and the various ThreadCpu/UserTime method definitions. Thanks, Paul From: serviceability-dev > on behalf of Jeremy Manson > Date: Thursday, February 22, 2018 at 9:42 PM To: David Holmes > Cc: "serviceability-dev at openjdk.java.net" >, "loom-dev at openjdk.java.net" >, John Rose > Subject: Re: Thread Native ID Access I think we're all pretty much agreed that fibers imply native tids can change. I think that, as long as we document that, we're fine. An API where there is probably a worse implementation problem is ThreadMXBean#getThreadCpuTime. Right now, the implementation assumes a 1:1 mapping between OS thread and Java thread. Removing that assumption implies you have to do something like track cpu time every time the scheduler switches you out, which is going to make your switches expensive. Jeremy On Thu, Feb 22, 2018 at 9:16 PM, David Holmes > wrote: On 23/02/2018 3:04 PM, John Rose wrote: On Feb 22, 2018, at 8:49 PM, David Holmes >> wrote: I don't think this request has any impact on Fibers at all. At any given time a piece of executing Java code is executing on a current Thread, and that current Thread must be running on a native thread (regardless of mapping) and the native thread has an id. The only use for exposing such an id is to allow you to correlate information obtained from inside the VM, with information observed outside, possibly by external tools. We may or may not bind fiber identity to the legacy java.lang.Thread type. I'm not assuming that exposing native thread ids implies anything about the binding between those entities and anything higher-level. I have no issue with a Fiber/Continuation/Task/whatever running on different java.lang.Threads over its lifetime. For that matter I don't think it matters if java.lang.Threads could execute on different native threads (they don't but they could). David ----- This affects the question of whether the following code can return false: boolean testThreadShift(Runnable r) { Thread t0 = Thread.current(); r.run(); Thread t1 = Thread.current(); return t0 == t1; } This method can return false if (a) Thread.current() reports the identity of the OS thread (not the fiber currently running), and (b) the runnable includes a blocking operation which causes the current fiber to reschedule on a different OS thread before it returns. Having this method return false is, perhaps, surprising enough to cause us to inject fiber identity into java.lang.Thread, so that a java.lang.Thread identity is 1-1 with a fiber, rather than an OS "dinosaur" thread. (You can only pick one!) The alternative is to keep a 1-1 identity between OS threads and java.lang.Thread instances, making a new type which carries fiber identity that multiplexes in a scheduler-dependent way over OS threads. It's an interesting trade-off, with many compatibility concerns. I think we are leaning toward injecting fiber identity into Thread, although this makes me shudder to think of the overheads we will buy with this one move (ThreadLocal hash tables). So, now if you reify the OS thread identity, you could have the same issue again. Can this routine return false? boolean testNTIDShift(Runnable r) { long t0 = NativeThreadID.current(); r.run(); long t1 = NativeThreadID.current(); return t0 == t1; } I think, if you are digging down all the way to the native thread ID, this routine *must* be allowed to return false under a Loom JVM. There's no credible way to assign unique int64_t IDs to fiber objects in the Java heap, as if they were OS objects. So my question is, do the proposed use cases for NTID allow for testNTIDShift to return false? If not, then it's pretty easy to say that we don't want to supply NativeThreadID.current() for general use, just to people implementing schedulers. ? John -------------- next part -------------- An HTML attachment was scrubbed... URL: From mandy.chung at oracle.com Sat Feb 24 00:50:39 2018 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 23 Feb 2018 16:50:39 -0800 Subject: RFR JDK-8198253: ThreadInfo.from(CompositeData) assigning fields incorrectly in JDK 9 In-Reply-To: References: <45a5eee6-023d-e342-d6ed-6797702b8804@oracle.com> <9110e27c-e492-b089-7b02-36459915c779@oracle.com> Message-ID: <2e7cf7ef-7a60-cd52-e5ab-a18e4b03f7b3@oracle.com> Webrev at: http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/webrev.01 This patch updates the spec to clarify what attributes are required since which release. There is a spec bug that "classLoaderName" added in JDK 9 is missing in the CompositeData for StackTraceElement but the implementation is correct. I will file a CSR for this update. This patch ensures that CompositeData for ThreadInfo of version N must have the attributes defined since <= N. ThreadInfo::from also makes sure 'stackTrace' and 'lockedMonitors' attributes of version N while it can include additional attributes which has been the current behavior. JDK-8139587 intended to support older versions of StackTraceElement which does not seem a complete solution. I reverted the fix for JDK-8139587 (mostly) and removed TypeVersionMapper. The fix constructs the CompositeType for ThreadInfo and MonitorInfo of different versions and used them for validation. Minor cleanup: the static final variables are renamed to all capitals. CompatibilityTest.java test is missing the copyright header. thanks Mandy [1] http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/api/java/lang/management/ThreadInfo.html#from(javax.management.openmbean.CompositeData) -------------- next part -------------- An HTML attachment was scrubbed... URL: From harsha.wardhana.b at oracle.com Mon Feb 26 05:12:42 2018 From: harsha.wardhana.b at oracle.com (Harsha Wardhana B) Date: Mon, 26 Feb 2018 10:42:42 +0530 Subject: RFR : JDK-8196744 : JMX: Not enough JDP packets received before timeout In-Reply-To: <4a870972-15c1-695f-2912-6ca5fe92ea61@oracle.com> References: <2eb1146c-f089-716d-8d83-70504cde24f8@oracle.com> <4a870972-15c1-695f-2912-6ca5fe92ea61@oracle.com> Message-ID: <1dcedaa6-51ed-eb38-e490-3ef3cf1b0817@oracle.com> Hello All, Requesting for review from one more reviewer. Thanks Harsha On Wednesday 21 February 2018 10:01 AM, Chris Plummer wrote: > Hi Harsha, > > Not a review, but just a request that you add the explanation of the > problem to the CR so we have a record of it. Also, the copyright needs > to be updated. > > thanks, > > Chris > > On 2/20/18 3:30 AM, Harsha Wardhana B wrote: >> Hi All, >> >> Please find the fix below for the Jdp test-case. >> >> issue: https://bugs.openjdk.java.net/browse/JDK-8196028 >> webrev : http://cr.openjdk.java.net/~hb/8196028/webrev.00/ >> >> Fix details : The test was receiving JDP packets from other VM and >> hence the multi-cast socket was not timing-out. The default timeout >> handler was causing test to fail. Added a shutdown method that passes >> the test in case of timeout. >> >> Thanks >> Harsha > > From volker.simonis at gmail.com Mon Feb 26 10:28:29 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Mon, 26 Feb 2018 11:28:29 +0100 Subject: 8u: RFR(S): 8197943: Unable to use JDWP API in JDK 8 to debug JDK 9 VM In-Reply-To: References: Message-ID: Hi Christoph, I think the new code is wrong for "jdwpMajor == 0", which was correctly handled before. But I'm not sure if that is relevant at all nowadays and taking into account that this is a verbatim downport from 9 I don't think we have to do better in 8u. Otherwise looks good from my side. Reviewed for 8u. Thank you and best regards, Volker On Thu, Feb 22, 2018 at 11:21 AM, Langer, Christoph wrote: > Hi JDK 8 reviewers, > > > > I?d like to propose a fix for a backport of JDI changes that came with > modularization. > > > > Egor has brought this up in 8u-dev: > http://mail.openjdk.java.net/pipermail/jdk8u-dev/2018-February/007230.html > > > > The fix is straightforward and is used by Egor in JetBrains, as well as we > at SAP have this already patched for a while in our JDK8 port. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8197943 > > Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8197943.0/ > > > > The build went through fine and jtreg tests are running. > > > > @Stephen: In the bug I read that you wanted to backport this. I hope this > matches your intentions. Otherwise I?d step back and wait for your proposal? > > > > Best regards > > Christoph > > From daniel.fuchs at oracle.com Mon Feb 26 11:35:07 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Mon, 26 Feb 2018 11:35:07 +0000 Subject: RFR JDK-8198253: ThreadInfo.from(CompositeData) assigning fields incorrectly in JDK 9 In-Reply-To: <2e7cf7ef-7a60-cd52-e5ab-a18e4b03f7b3@oracle.com> References: <45a5eee6-023d-e342-d6ed-6797702b8804@oracle.com> <9110e27c-e492-b089-7b02-36459915c779@oracle.com> <2e7cf7ef-7a60-cd52-e5ab-a18e4b03f7b3@oracle.com> Message-ID: Hi Mandy, Nice to see this fixed. In ThreadInfoCompositeData::initV6CompositeType 425 if (name.equals(STACK_TRACE)) { 426 ot = new ArrayType<>(1, StackTraceElementCompositeData.v5CompositeType()); 427 } else if (name.equals(LOCKED_MONITORS)) { 428 ot = new ArrayType<>(1, MonitorInfoCompositeData.v5CompositeType()); This is probably OK for StackTraceElementCompositeData, but is confusing for MonitorInfoCompositeData. I thought MonitorInfo was added in 6? so why would it have a v5CompositeType? I'd suggest renaming v5 to v6 in MonitorInfoCompositeData, and also adding a comment when calling StackTraceElementCompositeData.v5CompositeType() to say that nothing changed in StackTraceElementCompositeData between 6 and 5. best regards, -- daniel On 24/02/2018 00:50, mandy chung wrote: > Webrev at: > http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/webrev.01 > > This patch updates the spec to clarify what attributes are required > since which release. There is a spec bug that "classLoaderName" > added in JDK 9 is missing in the CompositeData for StackTraceElement > but the implementation is correct. I will file a CSR for this update. > > This patch ensures that CompositeData for ThreadInfo of version N > must have the attributes defined since <= N. > ThreadInfo::from also makes sure 'stackTrace' and 'lockedMonitors' > attributes of version N while it can include additional attributes > which has been the current behavior. > > JDK-8139587 intended to support older versions of StackTraceElement > which does not seem a complete solution. I reverted the fix for > JDK-8139587 (mostly) and removed TypeVersionMapper. The fix constructs > the CompositeType for ThreadInfo and MonitorInfo of different > versions and used them for validation. Minor cleanup: the static > final variables are renamed to all capitals. CompatibilityTest.java > test is missing the copyright header. > > thanks > Mandy > [1]http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/api/java/lang/management/ThreadInfo.html#from(javax.management.openmbean.CompositeData) > From erik.helin at oracle.com Mon Feb 26 16:46:28 2018 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 26 Feb 2018 17:46:28 +0100 Subject: RFR (S): 8195115: G1 Old Gen MemoryPool CollectionUsage.used values don't reflect mixed GC results In-Reply-To: <22216AD4-9A78-4427-9AB8-629EC685C296@amazon.com> References: <3A67BF1B-CE35-4759-B5B4-959C24020A45@amazon.com> <878D6678-84CB-471E-B126-AF14B1BC8EB9@amazon.com> <727f9bcb-9404-df32-917f-70aca6e31cb6@oracle.com> <4BA32239-D645-4B1A-A58E-502258C8FFB7@amazon.com> <997a766f-9f4b-f192-ae87-03f4c8d2666d@oracle.com> <64793ff3-4805-79f9-4e72-6e469cd511dc@oracle.com> <15754d34-f3c7-6431-7abf-05214bd6b9d2@oracle.com> <93417506-8e12-6ad3-690a-439e36719652@oracle.com> <99C937B6-F67B-4F4A-ACC8-444B5F83932B@amazon.com> <3922628B-81DE-4904-AEFE-F4163EB1E655@amazon.com> <8444d766-8adb-013f-f75b-b5d4420635df@oracle.com> <5768E4BF-5556-455B-899B-94A67BF940A0@amazon.com> <3E15AAD8-5017-4CCB-B927-6B31FD0D7809@amazon.com> <22216AD4-9A78-4427-9AB8-629EC685C296@amazon.com> Message-ID: <9f972398-115f-06ad-1e0c-513abceb097a@oracle.com> Hi Paul, a couple of comments on the patch: - memoryService.hpp: + 150 bool countCollection, + 151 bool allMemoryPoolsAffected = true); There is no need to use a default value for the parameter allMemoryPoolsAffected here. Skipping the default value also allows you to put allMemoryPoolsAffected to TraceMemoryManager::initialize in the same relative position as for the constructor parameter (this will make the code more uniform and easier to follow). - memoryManager.cpp Instead of adding a default parameter, maybe add a new method? Something like GCMemoryManager::add_not_always_affected_pool() (I couldn't come up with a shorter name at the moment). - TestMixedOldGenCollectionUsage.java The test is too strict about how and when collections should occur. Tests written this way often become very brittle, they might e.g. fail to finish a concurrent mark on time on a very slow, single core, machine. It is better to either force collections by using the WhiteBox API or make the test more lenient. Thanks, Erik On 02/22/2018 09:54 PM, Hohensee, Paul wrote: > Ping for a review please. > > Thanks, > > Paul > > On 2/16/18, 12:26 PM, "serviceability-dev on behalf of Hohensee, Paul" wrote: > > The CSR https://bugs.openjdk.java.net/browse/JDK-8196719 for the original fix has been approved, so I?m back to requesting a code review, please. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8195115 > Webrev: http://cr.openjdk.java.net/~phh/8195115/webrev.hs.01/ > > Passed a submit repo run, passes its jtreg test, and a JDK8 version is in production use at Amazon. > > From the original RR: > > > The bug is that from the JMX point of view, G1?s incremental collector > > (misnamed as the ?G1 Young Generation? collector) only affects G1?s > > survivor and eden spaces. In fact, mixed collections run by this > > collector also affect the G1 old generation. > > > > This proposed fix is to record, for each of a JMX garbage collector's > > memory pools, whether that memory pool is affected by all collections > > using that collector. And, for each collection, record whether or not > > all the collector's memory pools are affected. After each collection, > > for each memory pool, if either all the collector's memory pools were > > affected or the memory pool is affected for all collections, record > > CollectionUsage for that pool. > > > > For collectors other than G1 Young Generation, all pools are recorded as > > affected by all collections and every collection is recorded as > > affecting all the collector?s memory pools. For the G1 Young Generation > > collector, the G1 Old Gen pool is recorded as not being affected by all > > collections, and non-mixed collections are recorded as not affecting all > > memory pools. The result is that for non-mixed collections, > > CollectionUsage is recorded after a collection only the G1 Eden Space > > and G1 Survivor Space pools, while for mixed collections CollectionUsage > > is recorded for G1 Old Gen as well. > > > > Other than the effect of the fix on G1 Old Gen MemoryPool. > > CollectionUsage, the only external behavior change is that > > GarbageCollectorMXBean.getMemoryPoolNames will now return 3 pool names > > rather than 2. > > > > With this fix, a collector?s memory pools can be divided into two > > disjoint subsets, one that participates in all collections and one that > > doesn?t. This is a bit more general than the minimum necessary to fix > > G1, but not by much. Because I expect it to apply to other incremental > > region-based collectors, I went with the more general solution. I > > minimized the amount of code I had to touch by using default parameters > > for GCMemoryManager::add_pool and the TraceMemoryManagerStats constructors. > > > From mandy.chung at oracle.com Mon Feb 26 17:21:34 2018 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 26 Feb 2018 09:21:34 -0800 Subject: RFR JDK-8198253: ThreadInfo.from(CompositeData) assigning fields incorrectly in JDK 9 In-Reply-To: References: <45a5eee6-023d-e342-d6ed-6797702b8804@oracle.com> <9110e27c-e492-b089-7b02-36459915c779@oracle.com> <2e7cf7ef-7a60-cd52-e5ab-a18e4b03f7b3@oracle.com> Message-ID: <979b8d09-1767-d39e-eda6-61b78a104146@oracle.com> On 2/26/18 3:35 AM, Daniel Fuchs wrote: > Hi Mandy, > > Nice to see this fixed. > > In ThreadInfoCompositeData::initV6CompositeType > > ?425???????????????????? if (name.equals(STACK_TRACE)) { > ?426???????????????????????? ot = new ArrayType<>(1, > StackTraceElementCompositeData.v5CompositeType()); > ?427???????????????????? } else if (name.equals(LOCKED_MONITORS)) { > ?428???????????????????????? ot = new ArrayType<>(1, > MonitorInfoCompositeData.v5CompositeType()); > > This is probably OK for StackTraceElementCompositeData, but > is confusing for MonitorInfoCompositeData. I thought > MonitorInfo was added in 6? so why would it have a > v5CompositeType? > Good catch!? I certainly don't like numbering them explicitly.? I will see if I can come up with some simple way to support future changes. > I'd suggest renaming v5 to v6 in MonitorInfoCompositeData, and also > adding a comment when calling > StackTraceElementCompositeData.v5CompositeType() > to say that nothing changed in StackTraceElementCompositeData > between 6 and 5. > Will fix this.?? I will file a CSR for you to review. thanks Mandy > best regards, > > -- daniel > > > > On 24/02/2018 00:50, mandy chung wrote: >> Webrev at: >> http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/webrev.01 >> >> This patch updates the spec to clarify what attributes are required >> since which release.? There is a spec bug that "classLoaderName" >> added in JDK 9 is missing in the CompositeData for StackTraceElement >> but the implementation is correct.? I will file a CSR for this update. >> >> This patch ensures that CompositeData for ThreadInfo of version N >> must have the attributes defined since <= N. >> ThreadInfo::from also makes sure 'stackTrace' and 'lockedMonitors' >> attributes of version N while it can include additional attributes >> which has been the current behavior. >> >> JDK-8139587 intended to support older versions of StackTraceElement >> which does not seem a complete solution.? I reverted the fix for >> JDK-8139587 (mostly) and removed TypeVersionMapper.? The fix constructs >> the CompositeType for ThreadInfo and MonitorInfo of different >> versions and used them for validation.? Minor cleanup: the static >> final variables are renamed to all capitals. CompatibilityTest.java >> test is missing the copyright header. >> >> thanks >> Mandy >> [1]http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/api/java/lang/management/ThreadInfo.html#from(javax.management.openmbean.CompositeData) >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Mon Feb 26 19:22:55 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 26 Feb 2018 11:22:55 -0800 Subject: 8u: RFR(S): 8197943: Unable to use JDWP API in JDK 8 to debug JDK 9 VM In-Reply-To: References: Message-ID: <678d140a-bc3c-f2c3-769b-95c35fc9e9ca@oracle.com> I'm not sure the old code was doing anything useful by essentially checking for jdwpMajor == 0. When was it ever zero? Chris On 2/26/18 2:28 AM, Volker Simonis wrote: > Hi Christoph, > > I think the new code is wrong for "jdwpMajor == 0", which was > correctly handled before. > > But I'm not sure if that is relevant at all nowadays and taking into > account that this is a verbatim downport from 9 I don't think we have > to do better in 8u. > > Otherwise looks good from my side. Reviewed for 8u. > > Thank you and best regards, > Volker > > > On Thu, Feb 22, 2018 at 11:21 AM, Langer, Christoph > wrote: >> Hi JDK 8 reviewers, >> >> >> >> I?d like to propose a fix for a backport of JDI changes that came with >> modularization. >> >> >> >> Egor has brought this up in 8u-dev: >> http://mail.openjdk.java.net/pipermail/jdk8u-dev/2018-February/007230.html >> >> >> >> The fix is straightforward and is used by Egor in JetBrains, as well as we >> at SAP have this already patched for a while in our JDK8 port. >> >> >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8197943 >> >> Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8197943.0/ >> >> >> >> The build went through fine and jtreg tests are running. >> >> >> >> @Stephen: In the bug I read that you wanted to backport this. I hope this >> matches your intentions. Otherwise I?d step back and wait for your proposal? >> >> >> >> Best regards >> >> Christoph >> >> From daniil.x.titov at oracle.com Mon Feb 26 19:51:45 2018 From: daniil.x.titov at oracle.com (daniil.x.titov at oracle.com) Date: Mon, 26 Feb 2018 11:51:45 -0800 Subject: RFR 8170541: serviceability/jdwp/AllModulesCommandTest.java fails intermittently on Windows and Solaris In-Reply-To: References: <86bf7cb3-50a2-ddeb-0f88-73b8b2b4b2cf@oracle.com> <18D0CADB-CDB9-4F5D-B77A-DE5A5F6A5C19@oracle.com> <4ba6d0b1-0277-3e5e-1aa9-49e1db636586@oracle.com> Message-ID: Hi David and Sergei, On 2/20/18 10:16 PM, serguei.spitsyn at oracle.com wrote: > Hi David, > > > On 2/20/18 20:02, David Holmes wrote: >> Hi Daniil, >> >> Good find on this! >> >> What does the actual spec say about the length of things and how they >> may be split across multiple packets? Are we guaranteed that at most >> two packets will be involved? The JDWP spec (https://docs.oracle.com/javase/9/docs/specs/jdwp/jdwp-spec.html) says nothing about splitting JDWP reply packets at all but the implementation limits the max number of the sent packets to two packets max. The implementation is dated back to the initial load that happened in 2007 and the information about the related Jira issue is missing. open/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c 836 ?? data = packet->type.cmd.data; 837 ?? /* Do one send for short packets, two for longer ones */ 838 ?? if (data_len <= MAX_DATA_SIZE) { 839? ????? memcpy(header + JDWP_HEADER_SIZE, data, data_len); 840 ?????? if (send_fully(socketFD, (char *)&header, JDWP_HEADER_SIZE + data_len) != 841? ????????? JDWP_HEADER_SIZE + data_len) { 842 ?????????? RETURN_IO_ERROR("send failed"); 843? ????? } 844 ?? } else { 845? ????? memcpy(header + JDWP_HEADER_SIZE, data, MAX_DATA_SIZE); 846 ?????? if (send_fully(socketFD, (char *)&header, JDWP_HEADER_SIZE + MAX_DATA_SIZE) != 847 ?????????? JDWP_HEADER_SIZE + MAX_DATA_SIZE) { 848 ?????????? RETURN_IO_ERROR("send failed"); 849 ?????? } 850 ? ???? /* Send the remaining data bytes right out of the data area. */ 851 ?????? if (send_fully(socketFD, (char *)data + MAX_DATA_SIZE, 852 ????????????????????? data_len - MAX_DATA_SIZE) != data_len - MAX_DATA_SIZE) { 853 ?????????? RETURN_IO_ERROR("send failed"); 854 ?????? } 855 ?? } >> What about for other things eg: >> >> ?68???? protected byte[] readJdwpString(DataInputStream ds) throws >> IOException { >> ? 69???????? byte[] str = null; >> ? 70???????? int len = ds.readInt(); >> ? 71???????? if (len > 0) { >> ? 72???????????? str = new byte[len]; >> ? 73???????????? ds.read(str, 0, len); >> ? 74???????? } >> >> might we get a short-read of the string if it is split across >> multiple packets? > This and all other reads happen not directly from the socket input stream but rather from the? DataInputStream object that is constructed in? JdwpReply.initFromStream(InputStream) method. With the proposed fix we do ensure that the created DataInputStream object contains data from both packets in cases when the reply was split in two packets. > Nice catch! > Even though this fix is enough to resolve this problem now, there is a > chance, > it can fail in the future when more modules are added to the platform. > > >> I'm wondering if all these reads should be loops, ensuring we read >> the expected amount of data. >> Since the implementation of the socket transport limits the max number of packets the reply might be split in to two packets I don't think we really need it here. >> One further comment - not sure why we need the print out for when we >> do read multiple packets? >> That would seem to be a debugging aid. > > Yes, it helps to understand what happens. > Many tests have a lack of tracing which makes it harder to debug and > understand failures. That is correct.? This additional tracing was added to help to understand the possible failures in the future. > > > Thanks, > Serguei > >> >> Thanks, >> David >> Thanks, Daniil >> On 21/02/2018 10:14 AM, Daniil Titov wrote: >>> Hi Serguei, >>> >>> A new version of the webrev that has these strings reformatted is at >>> http://cr.openjdk.java.net/~dtitov/8170541/webrev.02/ >>> >>> Thank you! >>> >>> Best regards, >>> >>> Daniil >>> >>> *From: *"serguei.spitsyn at oracle.com" >>> *Date: *Tuesday, February 20, 2018 at 3:00 PM >>> *To: *Daniil Titov , >>> "serviceability-dev at openjdk.java.net" >>> >>> *Subject: *Re: RFR 8170541: >>> serviceability/jdwp/AllModulesCommandTest.java fails intermittently >>> on Windows and Solaris >>> >>> Hi Daniil, >>> >>> Interesting issue... >>> Thank you for finding to the root cause so quickly! >>> >>> The fix looks good. >>> Could I ask you to reformat these lines to make the L54 shorter ?: >>> >>> ?? 54???????????????? System.out.println("[" + getClass().getName() >>> + "] Only " + bytesRead + " bytes of " + dataLength + >>> >>> ?? 55???????????????????????? " were read in the first packet. >>> Reading the rest..."); >>> >>> Thanks, >>> Serguei >>> >>> >>> On 2/20/18 09:24, Daniil Titov wrote: >>> >>> ??? Please review the changes that fix intermittent failure of >>> ??? serviceability/jdwp/AllModulesCommandTest.java test. >>> >>> ??? The problem here is that for a large data the JDWP agent >>> ??? (socketTransport_writePacket() method in >>> src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c ) >>> ??? sends 2 packets and in some cases only the first packet is received >>> ??? at the time when the test reads the reply from the JDWP agent. >>> Since >>> ??? the test does not check that all data is received in the first >>> ??? packet the correlation between commands and replies became broken >>> ??? (the unread second packet is read by the next command and the reply >>> ??? for the next command is read by the next after next command and >>> so on). >>> >>> ??? Bug: https://bugs.openjdk.java.net/browse/JDK-8170541 >>> >>> ??? Webrev: http://cr.openjdk.java.net/~dtitov/8170541/webrev.01 >>> >>> ??? The tests ran successfully with Mach5. >>> >>> ??? Best regards, >>> >>> ??? Daniil >>> >>> >>> > From chris.plummer at oracle.com Mon Feb 26 20:16:16 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 26 Feb 2018 12:16:16 -0800 Subject: RFR 8170541: serviceability/jdwp/AllModulesCommandTest.java fails intermittently on Windows and Solaris In-Reply-To: References: <86bf7cb3-50a2-ddeb-0f88-73b8b2b4b2cf@oracle.com> <18D0CADB-CDB9-4F5D-B77A-DE5A5F6A5C19@oracle.com> <4ba6d0b1-0277-3e5e-1aa9-49e1db636586@oracle.com> Message-ID: <1c404611-6021-96d2-5f5f-1f48735ab5e2@oracle.com> On 2/26/18 11:51 AM, daniil.x.titov at oracle.com wrote: > Hi David and Sergei, > > On 2/20/18 10:16 PM, serguei.spitsyn at oracle.com wrote: >> Hi David, >> >> >> On 2/20/18 20:02, David Holmes wrote: >>> Hi Daniil, >>> >>> Good find on this! >>> >>> What does the actual spec say about the length of things and how >>> they may be split across multiple packets? Are we guaranteed that at >>> most two packets will be involved? > > The JDWP spec > (https://docs.oracle.com/javase/9/docs/specs/jdwp/jdwp-spec.html) says > nothing about splitting JDWP reply packets at all but the > implementation limits the max number of the sent packets to two > packets max. The implementation is dated back to the initial load that > happened in 2007 and the information about the related Jira issue is > missing. > > open/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c > > 836 ?? data = packet->type.cmd.data; > 837 ?? /* Do one send for short packets, two for longer ones */ > 838 ?? if (data_len <= MAX_DATA_SIZE) { > 839? ????? memcpy(header + JDWP_HEADER_SIZE, data, data_len); > 840 ?????? if (send_fully(socketFD, (char *)&header, JDWP_HEADER_SIZE > + data_len) != > 841? ????????? JDWP_HEADER_SIZE + data_len) { > 842 ?????????? RETURN_IO_ERROR("send failed"); > 843? ????? } > 844 ?? } else { > 845? ????? memcpy(header + JDWP_HEADER_SIZE, data, MAX_DATA_SIZE); > 846 ?????? if (send_fully(socketFD, (char *)&header, JDWP_HEADER_SIZE > + MAX_DATA_SIZE) != > 847 ?????????? JDWP_HEADER_SIZE + MAX_DATA_SIZE) { > 848 ?????????? RETURN_IO_ERROR("send failed"); > 849 ?????? } > 850 ? ???? /* Send the remaining data bytes right out of the data > area. */ > 851 ?????? if (send_fully(socketFD, (char *)data + MAX_DATA_SIZE, > 852 ????????????????????? data_len - MAX_DATA_SIZE) != data_len - > MAX_DATA_SIZE) { > 853 ?????????? RETURN_IO_ERROR("send failed"); > 854 ?????? } > 855 ?? } > Curious. First packet is limited to MAX_DATA_SIZE, 2nd packet has no size limit. What's the point then of splitting it then? Is there a desire to get the header transmitted in a smaller packet. Chris >>> What about for other things eg: >>> >>> ?68???? protected byte[] readJdwpString(DataInputStream ds) throws >>> IOException { >>> ? 69???????? byte[] str = null; >>> ? 70???????? int len = ds.readInt(); >>> ? 71???????? if (len > 0) { >>> ? 72???????????? str = new byte[len]; >>> ? 73???????????? ds.read(str, 0, len); >>> ? 74???????? } >>> >>> might we get a short-read of the string if it is split across >>> multiple packets? >> > This and all other reads happen not directly from the socket input > stream but rather from the? DataInputStream object that is constructed > in? JdwpReply.initFromStream(InputStream) method. With the proposed > fix we do ensure that the created DataInputStream object contains data > from both packets in cases when the reply was split in two packets. > >> Nice catch! >> Even though this fix is enough to resolve this problem now, there is >> a chance, >> it can fail in the future when more modules are added to the platform. >> >> >>> I'm wondering if all these reads should be loops, ensuring we read >>> the expected amount of data. >>> > Since the implementation of the socket transport limits the max number > of packets the reply might be split in to two packets I don't think we > really need it here. >>> One further comment - not sure why we need the print out for when we >>> do read multiple packets? >>> That would seem to be a debugging aid. >> >> Yes, it helps to understand what happens. >> Many tests have a lack of tracing which makes it harder to debug and >> understand failures. > That is correct.? This additional tracing was added to help to > understand the possible failures in the future. >> >> >> Thanks, >> Serguei >> >>> >>> Thanks, >>> David >>> > Thanks, > Daniil > >>> On 21/02/2018 10:14 AM, Daniil Titov wrote: >>>> Hi Serguei, >>>> >>>> A new version of the webrev that has these strings reformatted is >>>> at http://cr.openjdk.java.net/~dtitov/8170541/webrev.02/ >>>> >>>> Thank you! >>>> >>>> Best regards, >>>> >>>> Daniil >>>> >>>> *From: *"serguei.spitsyn at oracle.com" >>>> *Date: *Tuesday, February 20, 2018 at 3:00 PM >>>> *To: *Daniil Titov , >>>> "serviceability-dev at openjdk.java.net" >>>> >>>> *Subject: *Re: RFR 8170541: >>>> serviceability/jdwp/AllModulesCommandTest.java fails intermittently >>>> on Windows and Solaris >>>> >>>> Hi Daniil, >>>> >>>> Interesting issue... >>>> Thank you for finding to the root cause so quickly! >>>> >>>> The fix looks good. >>>> Could I ask you to reformat these lines to make the L54 shorter ?: >>>> >>>> ?? 54???????????????? System.out.println("[" + getClass().getName() >>>> + "] Only " + bytesRead + " bytes of " + dataLength + >>>> >>>> ?? 55???????????????????????? " were read in the first packet. >>>> Reading the rest..."); >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 2/20/18 09:24, Daniil Titov wrote: >>>> >>>> ??? Please review the changes that fix intermittent failure of >>>> ??? serviceability/jdwp/AllModulesCommandTest.java test. >>>> >>>> ??? The problem here is that for a large data the JDWP agent >>>> ??? (socketTransport_writePacket() method in >>>> src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c ) >>>> ??? sends 2 packets and in some cases only the first packet is >>>> received >>>> ??? at the time when the test reads the reply from the JDWP agent. >>>> Since >>>> ??? the test does not check that all data is received in the first >>>> ??? packet the correlation between commands and replies became broken >>>> ??? (the unread second packet is read by the next command and the >>>> reply >>>> ??? for the next command is read by the next after next command and >>>> so on). >>>> >>>> ??? Bug: https://bugs.openjdk.java.net/browse/JDK-8170541 >>>> >>>> ??? Webrev: http://cr.openjdk.java.net/~dtitov/8170541/webrev.01 >>>> >>>> ??? The tests ran successfully with Mach5. >>>> >>>> ??? Best regards, >>>> >>>> ??? Daniil >>>> >>>> >>>> >> > From Stephen.Fitch at oracle.com Mon Feb 26 20:31:12 2018 From: Stephen.Fitch at oracle.com (Stephen Fitch) Date: Mon, 26 Feb 2018 12:31:12 -0800 Subject: 8u: RFR(S): 8197943: Unable to use JDWP API in JDK 8 to debug JDK 9 VM In-Reply-To: <678d140a-bc3c-f2c3-769b-95c35fc9e9ca@oracle.com> References: <678d140a-bc3c-f2c3-769b-95c35fc9e9ca@oracle.com> Message-ID: <08762177-1fe5-ea67-f485-05824698a26a@oracle.com> Happy to see this get done (ahead of when I can get to it) feel free to take the JBS backport back into your name Christoph. s. On 2/26/18 11:22 AM, Chris Plummer wrote: > I'm not sure the old code was doing anything useful by essentially checking > for jdwpMajor == 0. When was it ever zero? > > Chris > > On 2/26/18 2:28 AM, Volker Simonis wrote: >> Hi Christoph, >> >> I think the new code is wrong for "jdwpMajor == 0", which was >> correctly handled before. >> >> But I'm not sure if that is relevant at all nowadays and taking into >> account that this is a verbatim downport from 9 I don't think we have >> to do better in 8u. >> >> Otherwise looks good from my side. Reviewed for 8u. >> >> Thank you and best regards, >> Volker >> >> >> On Thu, Feb 22, 2018 at 11:21 AM, Langer, Christoph >> wrote: >>> Hi JDK 8 reviewers, >>> >>> >>> >>> I?d like to propose a fix for a backport of JDI changes that came with >>> modularization. >>> >>> >>> >>> Egor has brought this up in 8u-dev: >>> http://mail.openjdk.java.net/pipermail/jdk8u-dev/2018-February/007230.html >>> >>> >>> >>> The fix is straightforward and is used by Egor in JetBrains, as well as we >>> at SAP have this already patched for a while in our JDK8 port. >>> >>> >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8197943 >>> >>> Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8197943.0/ >>> >>> >>> >>> The build went through fine and jtreg tests are running. >>> >>> >>> >>> @Stephen: In the bug I read that you wanted to backport this. I hope this >>> matches your intentions. Otherwise I?d step back and wait for your proposal? From alexey.menkov at oracle.com Mon Feb 26 22:43:23 2018 From: alexey.menkov at oracle.com (Alex Menkov) Date: Mon, 26 Feb 2018 14:43:23 -0800 Subject: RFR: JDK-8193369: post_field_access does not work for some functions, possibly related to fast_getfield Message-ID: <1fca6b67-c0d1-db03-52ed-f2c6bcc29a5b@oracle.com> Hi all, Please review a fix for JDK-8193369: post_field_access does not work for some functions, possibly related to fast_getfield The fix disables "fast" command generation when FieldAccess or FieldModification notifications are requested. jira: https://bugs.openjdk.java.net/browse/JDK-8193369 webrev: http://cr.openjdk.java.net/~amenkov/fast_field_access/webrev/ --alex From daniil.x.titov at oracle.com Mon Feb 26 23:00:29 2018 From: daniil.x.titov at oracle.com (daniil.x.titov at oracle.com) Date: Mon, 26 Feb 2018 15:00:29 -0800 Subject: RFR 8170541: serviceability/jdwp/AllModulesCommandTest.java fails intermittently on Windows and Solaris In-Reply-To: <1c404611-6021-96d2-5f5f-1f48735ab5e2@oracle.com> References: <86bf7cb3-50a2-ddeb-0f88-73b8b2b4b2cf@oracle.com> <18D0CADB-CDB9-4F5D-B77A-DE5A5F6A5C19@oracle.com> <4ba6d0b1-0277-3e5e-1aa9-49e1db636586@oracle.com> <1c404611-6021-96d2-5f5f-1f48735ab5e2@oracle.com> Message-ID: On 2/26/18 12:16 PM, Chris Plummer wrote: > On 2/26/18 11:51 AM, daniil.x.titov at oracle.com wrote: >> Hi David and Sergei, >> >> On 2/20/18 10:16 PM, serguei.spitsyn at oracle.com wrote: >>> Hi David, >>> >>> >>> On 2/20/18 20:02, David Holmes wrote: >>>> Hi Daniil, >>>> >>>> Good find on this! >>>> >>>> What does the actual spec say about the length of things and how >>>> they may be split across multiple packets? Are we guaranteed that >>>> at most two packets will be involved? >> >> The JDWP spec >> (https://docs.oracle.com/javase/9/docs/specs/jdwp/jdwp-spec.html) >> says nothing about splitting JDWP reply packets at all but the >> implementation limits the max number of the sent packets to two >> packets max. The implementation is dated back to the initial load >> that happened in 2007 and the information about the related Jira >> issue is missing. >> >> open/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c >> >> 836 ?? data = packet->type.cmd.data; >> 837 ?? /* Do one send for short packets, two for longer ones */ >> 838 ?? if (data_len <= MAX_DATA_SIZE) { >> 839? ????? memcpy(header + JDWP_HEADER_SIZE, data, data_len); >> 840 ?????? if (send_fully(socketFD, (char *)&header, JDWP_HEADER_SIZE >> + data_len) != >> 841? ????????? JDWP_HEADER_SIZE + data_len) { >> 842 ?????????? RETURN_IO_ERROR("send failed"); >> 843? ????? } >> 844 ?? } else { >> 845? ????? memcpy(header + JDWP_HEADER_SIZE, data, MAX_DATA_SIZE); >> 846 ?????? if (send_fully(socketFD, (char *)&header, JDWP_HEADER_SIZE >> + MAX_DATA_SIZE) != >> 847 ?????????? JDWP_HEADER_SIZE + MAX_DATA_SIZE) { >> 848 ?????????? RETURN_IO_ERROR("send failed"); >> 849 ?????? } >> 850 ? ???? /* Send the remaining data bytes right out of the data >> area. */ >> 851 ?????? if (send_fully(socketFD, (char *)data + MAX_DATA_SIZE, >> 852 ????????????????????? data_len - MAX_DATA_SIZE) != data_len - >> MAX_DATA_SIZE) { >> 853 ?????????? RETURN_IO_ERROR("send failed"); >> 854 ?????? } >> 855 ?? } >> > Curious. First packet is limited to MAX_DATA_SIZE, 2nd packet has no > size limit. What's the point then of splitting it then? Is there a > desire to get the header transmitted in a smaller packet. > > Chris It looks as the goal was to somehow improve the responsiveness in case of the large data but I am not sure about this. I could not locate any traces in Jira related to this implementation. Probably Serguei has some info what is the history behind this design. >>>> >>>> ?68???? protected byte[] readJdwpString(DataInputStream ds) throws >>>> IOException { >>>> ? 69???????? byte[] str = null; >>>> ? 70???????? int len = ds.readInt(); >>>> ? 71???????? if (len > 0) { >>>> ? 72???????????? str = new byte[len]; >>>> ? 73???????????? ds.read(str, 0, len); >>>> ? 74???????? } >>>> >>>> might we get a short-read of the string if it is split across >>>> multiple packets? >>> >> This and all other reads happen not directly from the socket input >> stream but rather from the? DataInputStream object that is >> constructed in? JdwpReply.initFromStream(InputStream) method. With >> the proposed fix we do ensure that the created DataInputStream object >> contains data from both packets in cases when the reply was split in >> two packets. >> >>> Nice catch! >>> Even though this fix is enough to resolve this problem now, there is >>> a chance, >>> it can fail in the future when more modules are added to the platform. >>> >>> >>>> I'm wondering if all these reads should be loops, ensuring we read >>>> the expected amount of data. >>>> >> Since the implementation of the socket transport limits the max >> number of packets the reply might be split in to two packets I don't >> think we really need it here. >>>> One further comment - not sure why we need the print out for when >>>> we do read multiple packets? >>>> That would seem to be a debugging aid. >>> >>> Yes, it helps to understand what happens. >>> Many tests have a lack of tracing which makes it harder to debug and >>> understand failures. >> That is correct.? This additional tracing was added to help to >> understand the possible failures in the future. >>> >>> >>> Thanks, >>> Serguei >>> >>>> >>>> Thanks, >>>> David >>>> >> Thanks, >> Daniil >> >>>> On 21/02/2018 10:14 AM, Daniil Titov wrote: >>>>> Hi Serguei, >>>>> >>>>> A new version of the webrev that has these strings reformatted is >>>>> at http://cr.openjdk.java.net/~dtitov/8170541/webrev.02/ >>>>> >>>>> Thank you! >>>>> >>>>> Best regards, >>>>> >>>>> Daniil >>>>> >>>>> *From: *"serguei.spitsyn at oracle.com" >>>>> *Date: *Tuesday, February 20, 2018 at 3:00 PM >>>>> *To: *Daniil Titov , >>>>> "serviceability-dev at openjdk.java.net" >>>>> >>>>> *Subject: *Re: RFR 8170541: >>>>> serviceability/jdwp/AllModulesCommandTest.java fails >>>>> intermittently on Windows and Solaris >>>>> >>>>> Hi Daniil, >>>>> >>>>> Interesting issue... >>>>> Thank you for finding to the root cause so quickly! >>>>> >>>>> The fix looks good. >>>>> Could I ask you to reformat these lines to make the L54 shorter ?: >>>>> >>>>> ?? 54???????????????? System.out.println("[" + >>>>> getClass().getName() + "] Only " + bytesRead + " bytes of " + >>>>> dataLength + >>>>> >>>>> ?? 55???????????????????????? " were read in the first packet. >>>>> Reading the rest..."); >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>> On 2/20/18 09:24, Daniil Titov wrote: >>>>> >>>>> ??? Please review the changes that fix intermittent failure of >>>>> ??? serviceability/jdwp/AllModulesCommandTest.java test. >>>>> >>>>> ??? The problem here is that for a large data the JDWP agent >>>>> ??? (socketTransport_writePacket() method in >>>>> src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c ) >>>>> ??? sends 2 packets and in some cases only the first packet is >>>>> received >>>>> ??? at the time when the test reads the reply from the JDWP agent. >>>>> Since >>>>> ??? the test does not check that all data is received in the first >>>>> ??? packet the correlation between commands and replies became broken >>>>> ??? (the unread second packet is read by the next command and the >>>>> reply >>>>> ??? for the next command is read by the next after next command >>>>> and so on). >>>>> >>>>> ??? Bug: https://bugs.openjdk.java.net/browse/JDK-8170541 >>>>> >>>>> ??? Webrev: http://cr.openjdk.java.net/~dtitov/8170541/webrev.01 >>>>> >>>>> ??? The tests ran successfully with Mach5. >>>>> >>>>> ??? Best regards, >>>>> >>>>> ??? Daniil >>>>> >>>>> >>>>> >>> >> > > From chris.plummer at oracle.com Mon Feb 26 23:06:26 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 26 Feb 2018 15:06:26 -0800 Subject: RFR 8170541: serviceability/jdwp/AllModulesCommandTest.java fails intermittently on Windows and Solaris In-Reply-To: References: <86bf7cb3-50a2-ddeb-0f88-73b8b2b4b2cf@oracle.com> <18D0CADB-CDB9-4F5D-B77A-DE5A5F6A5C19@oracle.com> <4ba6d0b1-0277-3e5e-1aa9-49e1db636586@oracle.com> <1c404611-6021-96d2-5f5f-1f48735ab5e2@oracle.com> Message-ID: <07e6e62e-5713-fd57-34f6-ad439e8da209@oracle.com> On 2/26/18 3:00 PM, daniil.x.titov at oracle.com wrote: > > > On 2/26/18 12:16 PM, Chris Plummer wrote: >> On 2/26/18 11:51 AM, daniil.x.titov at oracle.com wrote: >>> Hi David and Sergei, >>> >>> On 2/20/18 10:16 PM, serguei.spitsyn at oracle.com wrote: >>>> Hi David, >>>> >>>> >>>> On 2/20/18 20:02, David Holmes wrote: >>>>> Hi Daniil, >>>>> >>>>> Good find on this! >>>>> >>>>> What does the actual spec say about the length of things and how >>>>> they may be split across multiple packets? Are we guaranteed that >>>>> at most two packets will be involved? >>> >>> The JDWP spec >>> (https://docs.oracle.com/javase/9/docs/specs/jdwp/jdwp-spec.html) >>> says nothing about splitting JDWP reply packets at all but the >>> implementation limits the max number of the sent packets to two >>> packets max. The implementation is dated back to the initial load >>> that happened in 2007 and the information about the related Jira >>> issue is missing. >>> >>> open/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c >>> >>> 836 ?? data = packet->type.cmd.data; >>> 837 ?? /* Do one send for short packets, two for longer ones */ >>> 838 ?? if (data_len <= MAX_DATA_SIZE) { >>> 839? ????? memcpy(header + JDWP_HEADER_SIZE, data, data_len); >>> 840 ?????? if (send_fully(socketFD, (char *)&header, >>> JDWP_HEADER_SIZE + data_len) != >>> 841? ????????? JDWP_HEADER_SIZE + data_len) { >>> 842 ?????????? RETURN_IO_ERROR("send failed"); >>> 843? ????? } >>> 844 ?? } else { >>> 845? ????? memcpy(header + JDWP_HEADER_SIZE, data, MAX_DATA_SIZE); >>> 846 ?????? if (send_fully(socketFD, (char *)&header, >>> JDWP_HEADER_SIZE + MAX_DATA_SIZE) != >>> 847 ?????????? JDWP_HEADER_SIZE + MAX_DATA_SIZE) { >>> 848 ?????????? RETURN_IO_ERROR("send failed"); >>> 849 ?????? } >>> 850 ? ???? /* Send the remaining data bytes right out of the data >>> area. */ >>> 851 ?????? if (send_fully(socketFD, (char *)data + MAX_DATA_SIZE, >>> 852 ????????????????????? data_len - MAX_DATA_SIZE) != data_len - >>> MAX_DATA_SIZE) { >>> 853 ?????????? RETURN_IO_ERROR("send failed"); >>> 854 ?????? } >>> 855 ?? } >>> >> Curious. First packet is limited to MAX_DATA_SIZE, 2nd packet has no >> size limit. What's the point then of splitting it then? Is there a >> desire to get the header transmitted in a smaller packet. >> >> Chris > > It looks as the goal was to somehow improve the responsiveness in case > of the large data but I am not sure about this. I could not locate any > traces in Jira related to this implementation. I was thinking it might be something like that too. Get the header across the wire quickly. Maybe the user just wants the header (with size info) initially, and will allocate a large buffer for the rest if necessary. Chris > Probably Serguei has some info what is the history behind this design. >>>>> >>>>> ?68???? protected byte[] readJdwpString(DataInputStream ds) throws >>>>> IOException { >>>>> ? 69???????? byte[] str = null; >>>>> ? 70???????? int len = ds.readInt(); >>>>> ? 71???????? if (len > 0) { >>>>> ? 72???????????? str = new byte[len]; >>>>> ? 73???????????? ds.read(str, 0, len); >>>>> ? 74???????? } >>>>> >>>>> might we get a short-read of the string if it is split across >>>>> multiple packets? >>>> >>> This and all other reads happen not directly from the socket input >>> stream but rather from the? DataInputStream object that is >>> constructed in? JdwpReply.initFromStream(InputStream) method. With >>> the proposed fix we do ensure that the created DataInputStream >>> object contains data from both packets in cases when the reply was >>> split in two packets. >>> >>>> Nice catch! >>>> Even though this fix is enough to resolve this problem now, there >>>> is a chance, >>>> it can fail in the future when more modules are added to the platform. >>>> >>>> >>>>> I'm wondering if all these reads should be loops, ensuring we read >>>>> the expected amount of data. >>>>> >>> Since the implementation of the socket transport limits the max >>> number of packets the reply might be split in to two packets I don't >>> think we really need it here. >>>>> One further comment - not sure why we need the print out for when >>>>> we do read multiple packets? >>>>> That would seem to be a debugging aid. >>>> >>>> Yes, it helps to understand what happens. >>>> Many tests have a lack of tracing which makes it harder to debug >>>> and understand failures. >>> That is correct.? This additional tracing was added to help to >>> understand the possible failures in the future. >>>> >>>> >>>> Thanks, >>>> Serguei >>>> >>>>> >>>>> Thanks, >>>>> David >>>>> >>> Thanks, >>> Daniil >>> >>>>> On 21/02/2018 10:14 AM, Daniil Titov wrote: >>>>>> Hi Serguei, >>>>>> >>>>>> A new version of the webrev that has these strings reformatted is >>>>>> at http://cr.openjdk.java.net/~dtitov/8170541/webrev.02/ >>>>>> >>>>>> Thank you! >>>>>> >>>>>> Best regards, >>>>>> >>>>>> Daniil >>>>>> >>>>>> *From: *"serguei.spitsyn at oracle.com" >>>>>> *Date: *Tuesday, February 20, 2018 at 3:00 PM >>>>>> *To: *Daniil Titov , >>>>>> "serviceability-dev at openjdk.java.net" >>>>>> >>>>>> *Subject: *Re: RFR 8170541: >>>>>> serviceability/jdwp/AllModulesCommandTest.java fails >>>>>> intermittently on Windows and Solaris >>>>>> >>>>>> Hi Daniil, >>>>>> >>>>>> Interesting issue... >>>>>> Thank you for finding to the root cause so quickly! >>>>>> >>>>>> The fix looks good. >>>>>> Could I ask you to reformat these lines to make the L54 shorter ?: >>>>>> >>>>>> ?? 54???????????????? System.out.println("[" + >>>>>> getClass().getName() + "] Only " + bytesRead + " bytes of " + >>>>>> dataLength + >>>>>> >>>>>> ?? 55???????????????????????? " were read in the first packet. >>>>>> Reading the rest..."); >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>>> >>>>>> On 2/20/18 09:24, Daniil Titov wrote: >>>>>> >>>>>> ??? Please review the changes that fix intermittent failure of >>>>>> ??? serviceability/jdwp/AllModulesCommandTest.java test. >>>>>> >>>>>> ??? The problem here is that for a large data the JDWP agent >>>>>> ??? (socketTransport_writePacket() method in >>>>>> src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c ) >>>>>> ??? sends 2 packets and in some cases only the first packet is >>>>>> received >>>>>> ??? at the time when the test reads the reply from the JDWP >>>>>> agent. Since >>>>>> ??? the test does not check that all data is received in the first >>>>>> ??? packet the correlation between commands and replies became >>>>>> broken >>>>>> ??? (the unread second packet is read by the next command and the >>>>>> reply >>>>>> ??? for the next command is read by the next after next command >>>>>> and so on). >>>>>> >>>>>> ??? Bug: https://bugs.openjdk.java.net/browse/JDK-8170541 >>>>>> >>>>>> ??? Webrev: http://cr.openjdk.java.net/~dtitov/8170541/webrev.01 >>>>>> >>>>>> ??? The tests ran successfully with Mach5. >>>>>> >>>>>> ??? Best regards, >>>>>> >>>>>> ??? Daniil >>>>>> >>>>>> >>>>>> >>>> >>> >> >> > From serguei.spitsyn at oracle.com Mon Feb 26 23:29:56 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 26 Feb 2018 15:29:56 -0800 Subject: RFR 8170541: serviceability/jdwp/AllModulesCommandTest.java fails intermittently on Windows and Solaris In-Reply-To: <07e6e62e-5713-fd57-34f6-ad439e8da209@oracle.com> References: <86bf7cb3-50a2-ddeb-0f88-73b8b2b4b2cf@oracle.com> <18D0CADB-CDB9-4F5D-B77A-DE5A5F6A5C19@oracle.com> <4ba6d0b1-0277-3e5e-1aa9-49e1db636586@oracle.com> <1c404611-6021-96d2-5f5f-1f48735ab5e2@oracle.com> <07e6e62e-5713-fd57-34f6-ad439e8da209@oracle.com> Message-ID: On 2/26/18 15:06, Chris Plummer wrote: > On 2/26/18 3:00 PM, daniil.x.titov at oracle.com wrote: >> >> >> On 2/26/18 12:16 PM, Chris Plummer wrote: >>> On 2/26/18 11:51 AM, daniil.x.titov at oracle.com wrote: >>>> Hi David and Sergei, >>>> >>>> On 2/20/18 10:16 PM, serguei.spitsyn at oracle.com wrote: >>>>> Hi David, >>>>> >>>>> >>>>> On 2/20/18 20:02, David Holmes wrote: >>>>>> Hi Daniil, >>>>>> >>>>>> Good find on this! >>>>>> >>>>>> What does the actual spec say about the length of things and how >>>>>> they may be split across multiple packets? Are we guaranteed that >>>>>> at most two packets will be involved? >>>> >>>> The JDWP spec >>>> (https://docs.oracle.com/javase/9/docs/specs/jdwp/jdwp-spec.html) >>>> says nothing about splitting JDWP reply packets at all but the >>>> implementation limits the max number of the sent packets to two >>>> packets max. The implementation is dated back to the initial load >>>> that happened in 2007 and the information about the related Jira >>>> issue is missing. >>>> >>>> open/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c >>>> >>>> 836 ?? data = packet->type.cmd.data; >>>> 837 ?? /* Do one send for short packets, two for longer ones */ >>>> 838 ?? if (data_len <= MAX_DATA_SIZE) { >>>> 839? ????? memcpy(header + JDWP_HEADER_SIZE, data, data_len); >>>> 840 ?????? if (send_fully(socketFD, (char *)&header, >>>> JDWP_HEADER_SIZE + data_len) != >>>> 841? ????????? JDWP_HEADER_SIZE + data_len) { >>>> 842 ?????????? RETURN_IO_ERROR("send failed"); >>>> 843? ????? } >>>> 844 ?? } else { >>>> 845? ????? memcpy(header + JDWP_HEADER_SIZE, data, MAX_DATA_SIZE); >>>> 846 ?????? if (send_fully(socketFD, (char *)&header, >>>> JDWP_HEADER_SIZE + MAX_DATA_SIZE) != >>>> 847 ?????????? JDWP_HEADER_SIZE + MAX_DATA_SIZE) { >>>> 848 ?????????? RETURN_IO_ERROR("send failed"); >>>> 849 ?????? } >>>> 850 ? ???? /* Send the remaining data bytes right out of the data >>>> area. */ >>>> 851 ?????? if (send_fully(socketFD, (char *)data + MAX_DATA_SIZE, >>>> 852 ????????????????????? data_len - MAX_DATA_SIZE) != data_len - >>>> MAX_DATA_SIZE) { >>>> 853 ?????????? RETURN_IO_ERROR("send failed"); >>>> 854 ?????? } >>>> 855 ?? } >>>> >>> Curious. First packet is limited to MAX_DATA_SIZE, 2nd packet has no >>> size limit. What's the point then of splitting it then? Is there a >>> desire to get the header transmitted in a smaller packet. >>> >>> Chris >> >> It looks as the goal was to somehow improve the responsiveness in >> case of the large data but I am not sure about this. I could not >> locate any traces in Jira related to this implementation. > I was thinking it might be something like that too. Get the header > across the wire quickly. Maybe the user just wants the header (with > size info) initially, and will allocate a large buffer for the rest if > necessary. It was my guess too. At least, it is the best explanation for this design that looks reasonable to me. > Chris >> Probably Serguei has some info what is the history behind this design. I don't know the history here. This was implemented in very early days, most likely, before JDK 1.5 or even 1.4.2. Thanks, Serguei >>>>>> >>>>>> ?68???? protected byte[] readJdwpString(DataInputStream ds) >>>>>> throws IOException { >>>>>> ? 69???????? byte[] str = null; >>>>>> ? 70???????? int len = ds.readInt(); >>>>>> ? 71???????? if (len > 0) { >>>>>> ? 72???????????? str = new byte[len]; >>>>>> ? 73???????????? ds.read(str, 0, len); >>>>>> ? 74???????? } >>>>>> >>>>>> might we get a short-read of the string if it is split across >>>>>> multiple packets? >>>>> >>>> This and all other reads happen not directly from the socket input >>>> stream but rather from the? DataInputStream object that is >>>> constructed in JdwpReply.initFromStream(InputStream) method. With >>>> the proposed fix we do ensure that the created DataInputStream >>>> object contains data from both packets in cases when the reply was >>>> split in two packets. >>>> >>>>> Nice catch! >>>>> Even though this fix is enough to resolve this problem now, there >>>>> is a chance, >>>>> it can fail in the future when more modules are added to the >>>>> platform. >>>>> >>>>> >>>>>> I'm wondering if all these reads should be loops, ensuring we >>>>>> read the expected amount of data. >>>>>> >>>> Since the implementation of the socket transport limits the max >>>> number of packets the reply might be split in to two packets I >>>> don't think we really need it here. >>>>>> One further comment - not sure why we need the print out for when >>>>>> we do read multiple packets? >>>>>> That would seem to be a debugging aid. >>>>> >>>>> Yes, it helps to understand what happens. >>>>> Many tests have a lack of tracing which makes it harder to debug >>>>> and understand failures. >>>> That is correct.? This additional tracing was added to help to >>>> understand the possible failures in the future. >>>>> >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>> Thanks, >>>> Daniil >>>> >>>>>> On 21/02/2018 10:14 AM, Daniil Titov wrote: >>>>>>> Hi Serguei, >>>>>>> >>>>>>> A new version of the webrev that has these strings reformatted >>>>>>> is at http://cr.openjdk.java.net/~dtitov/8170541/webrev.02/ >>>>>>> >>>>>>> Thank you! >>>>>>> >>>>>>> Best regards, >>>>>>> >>>>>>> Daniil >>>>>>> >>>>>>> *From: *"serguei.spitsyn at oracle.com" >>>>>>> *Date: *Tuesday, February 20, 2018 at 3:00 PM >>>>>>> *To: *Daniil Titov , >>>>>>> "serviceability-dev at openjdk.java.net" >>>>>>> >>>>>>> *Subject: *Re: RFR 8170541: >>>>>>> serviceability/jdwp/AllModulesCommandTest.java fails >>>>>>> intermittently on Windows and Solaris >>>>>>> >>>>>>> Hi Daniil, >>>>>>> >>>>>>> Interesting issue... >>>>>>> Thank you for finding to the root cause so quickly! >>>>>>> >>>>>>> The fix looks good. >>>>>>> Could I ask you to reformat these lines to make the L54 shorter ?: >>>>>>> >>>>>>> ?? 54???????????????? System.out.println("[" + >>>>>>> getClass().getName() + "] Only " + bytesRead + " bytes of " + >>>>>>> dataLength + >>>>>>> >>>>>>> ?? 55???????????????????????? " were read in the first packet. >>>>>>> Reading the rest..."); >>>>>>> >>>>>>> Thanks, >>>>>>> Serguei >>>>>>> >>>>>>> >>>>>>> On 2/20/18 09:24, Daniil Titov wrote: >>>>>>> >>>>>>> ??? Please review the changes that fix intermittent failure of >>>>>>> ??? serviceability/jdwp/AllModulesCommandTest.java test. >>>>>>> >>>>>>> ??? The problem here is that for a large data the JDWP agent >>>>>>> ??? (socketTransport_writePacket() method in >>>>>>> src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c ) >>>>>>> ??? sends 2 packets and in some cases only the first packet is >>>>>>> received >>>>>>> ??? at the time when the test reads the reply from the JDWP >>>>>>> agent. Since >>>>>>> ??? the test does not check that all data is received in the first >>>>>>> ??? packet the correlation between commands and replies became >>>>>>> broken >>>>>>> ??? (the unread second packet is read by the next command and >>>>>>> the reply >>>>>>> ??? for the next command is read by the next after next command >>>>>>> and so on). >>>>>>> >>>>>>> ??? Bug: https://bugs.openjdk.java.net/browse/JDK-8170541 >>>>>>> >>>>>>> ??? Webrev: http://cr.openjdk.java.net/~dtitov/8170541/webrev.01 >>>>>>> >>>>>>> ??? The tests ran successfully with Mach5. >>>>>>> >>>>>>> ??? Best regards, >>>>>>> >>>>>>> ??? Daniil >>>>>>> >>>>>>> >>>>>>> >>>>> >>>> >>> >>> >> > > From jeremymanson at google.com Tue Feb 27 00:23:28 2018 From: jeremymanson at google.com (Jeremy Manson) Date: Mon, 26 Feb 2018 16:23:28 -0800 Subject: RFR JDK-8198253: ThreadInfo.from(CompositeData) assigning fields incorrectly in JDK 9 In-Reply-To: <2e7cf7ef-7a60-cd52-e5ab-a18e4b03f7b3@oracle.com> References: <45a5eee6-023d-e342-d6ed-6797702b8804@oracle.com> <9110e27c-e492-b089-7b02-36459915c779@oracle.com> <2e7cf7ef-7a60-cd52-e5ab-a18e4b03f7b3@oracle.com> Message-ID: Hi Mandy, Thanks for taking this on! I'm happy to see that you are happy to do cleanups I was too timid to do (like adding the Factory in the tests). I note a few places in the test code where static initializers can throw RuntimeExceptions. When I ran the tests, and a static initializer threw a RuntimeException, I didn't see it reflected in the output, so I had to add print statements. Was I just doing it wrong, or is that a feature of jtreg? Just a couple of random nits: > 267 * @throws IllegalArgumentException if the given CompositeData does not > 268 * contain all attributes representing ThreadInfo of any release. Could this be misinterpreted? If you have everything in V5, and only one of the V6 attributes, it contains all of the attributes representing the ThreadInfo of V5. Maybe you want to say "does not contain exactly the attributes present in ThreadInfo for a particular release"? For signatures like: > static CompositeType initV5CompositeType(CompositeType ctype) { I probably would change the name of ctype to indicate when you are reading the method that the parameter is always supposed to be the the CompositeType for ThreadInfo. On Fri, Feb 23, 2018 at 4:50 PM, mandy chung wrote: > Webrev at: > http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/webrev.01 > > This patch updates the spec to clarify what attributes are required > since which release. There is a spec bug that "classLoaderName" > added in JDK 9 is missing in the CompositeData for StackTraceElement > but the implementation is correct. I will file a CSR for this update. > > This patch ensures that CompositeData for ThreadInfo of version N > must have the attributes defined since <= N. > ThreadInfo::from also makes sure 'stackTrace' and 'lockedMonitors' > attributes of version N while it can include additional attributes > which has been the current behavior. > > JDK-8139587 intended to support older versions of StackTraceElement > which does not seem a complete solution. I reverted the fix for > JDK-8139587 (mostly) and removed TypeVersionMapper. The fix constructs > the CompositeType for ThreadInfo and MonitorInfo of different > versions and used them for validation. Minor cleanup: the static > final variables are renamed to all capitals. CompatibilityTest.java > test is missing the copyright header. > > thanks > Mandy > [1] http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/api/java/lang/management/ThreadInfo.html#from(javax.management.openmbean.CompositeData) > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeremymanson at google.com Tue Feb 27 00:38:37 2018 From: jeremymanson at google.com (Jeremy Manson) Date: Mon, 26 Feb 2018 16:38:37 -0800 Subject: Thread Native ID Access In-Reply-To: <39B8BA33-EE67-47D2-96C8-22D2AAB11AC7@amazon.com> References: <00cead79-3914-4ad7-5233-849d13257981@oracle.com> <5a8f14f3.ac82df0a.6097c.7509@mx.google.com> <5a8f17a5.3886df0a.d31d9.5e8a@mx.google.com> <0a2a65b5-b1ac-3967-6a3c-02fd899fde43@oracle.com> <7A1B1422-6D9C-4421-BC02-13DCB97449B9@oracle.com> <39B8BA33-EE67-47D2-96C8-22D2AAB11AC7@amazon.com> Message-ID: Okay, I'll find time at some point to tinker with it. I'm going to wait until Mandy lands the fix to JDK-8198253, though. Jeremy On Fri, Feb 23, 2018 at 3:44 PM, Hohensee, Paul wrote: > I am. :) > > > > The process is to file an RFE and then a CSR associated with it. You file > the CSR from the RFE page, not separately. See > > > > https://wiki.openjdk.java.net/display/csr/Main > > https://wiki.openjdk.java.net/display/csr/Fields+of+a+CSR+Request > > https://wiki.openjdk.java.net/display/csr/CSR+FAQs > > > > You need a review of the CSR by a domain expert before you can finalize it > for committee review. If you go with com.sun.management versions of > ThreadInfo and ThreadMXBean methods, there?s no change in the > java.lang.management specification, which is important because the j.l.m > interface must be supported by all implementations, which is a higher bar > than for c.s.m changes. > > > > Paul > > > > *From: *Jeremy Manson > *Date: *Friday, February 23, 2018 at 3:12 PM > *To: *"Hohensee, Paul" > *Cc: *David Holmes , "serviceability-dev at openjdk. > java.net" , " > loom-dev at openjdk.java.net" , John Rose < > john.r.rose at oracle.com> > > *Subject: *Re: Thread Native ID Access > > > > Thanks, Paul. This is approximately what I was envisioning when Mandy and > David said they didn't really want to expose it in java.lang.Thread. > > > > If folks are generally amenable to something like this, I can probably > find the spare time to cobble it together. > > > > Jeremy > > > > On Fri, Feb 23, 2018 at 1:07 PM, Hohensee, Paul > wrote: > > Here?s a concrete proposal on which people may want to comment. > > > > We could add a method to ThreadInfo, vis., > > > > long getPlatformThreadId() > > > > Returns the platform-specific thread ID of the thread associated with this > ThreadInfo. The ID has no meaning to the Java runtime and is not guaranteed > to have any meaning at all. > > I.e., it may be a constant value for all threads, or a native address, and > may not be the same across multiple invocations. > > > > Returns: > > > > the platform-specific ID of the associated thread. > > > > For a fiber, a reasonable value would be the OS thread id for the OS > thread that?s running the fiber at ThreadInfo construction time. If, say, > the platform used a string or some other larger-than-long value, it could > be a native address or some other value that mapped to one. > > > > If we don?t want to add a new method to ThreadInfo, we could define a > com.sun.management.ThreadInfo to go with com.sun.management.ThreadMXBean. > > > > Parenthetically, to me a fiber is enough different from a thread to > warrant a separate MXBean. I.e., a thread is a species of fiber, but not > vica versa. So, we shouldn?t gate changes to ThreadMXBean on having a > thorough understanding of how to support fibers in JMX. > > > > We could also add to com.sun.management.ThreadMXBean the methods > getThreadId(long id), getThreadId(long [] ids), getPlatformThreadId(long > id), and getPlatformThreadId(long [] ids). These presumably would be fast > compared to creating ThreadInfo(s). > > > > We don?t want to add Thread.getPlatformId(), so the way to get the current > thread?s platform id would be > > > > getThreadInfo( Thread.currentThread().getId() ).getPlatformThreadId() > > > > which is cumbersome and slow. Instead, by analogy to ThreadMXBean.getCurrentThreadCpuTime(), > we could add to com.sun.management.ThreadMXBean the methods > getCurrentThreadId() and getCurrentPlatformThreadId(). > > > > All this may seem somewhat overkill, but it?s no more so than how we > currently create ThreadInfo objects and the various ThreadCpu/UserTime > method definitions. > > > > Thanks, > > > > Paul > > > > *From: *serviceability-dev > on behalf of Jeremy Manson > *Date: *Thursday, February 22, 2018 at 9:42 PM > *To: *David Holmes > *Cc: *"serviceability-dev at openjdk.java.net" java.net>, "loom-dev at openjdk.java.net" , John > Rose > *Subject: *Re: Thread Native ID Access > > > > I think we're all pretty much agreed that fibers imply native tids can > change. I think that, as long as we document that, we're fine. > > > > An API where there is probably a worse implementation problem is > ThreadMXBean#getThreadCpuTime > > . Right now, the implementation assumes a 1:1 mapping between OS thread > and Java thread. Removing that assumption implies you have to do something > like track cpu time every time the scheduler switches you out, which is > going to make your switches expensive. > > > > Jeremy > > > > On Thu, Feb 22, 2018 at 9:16 PM, David Holmes > wrote: > > On 23/02/2018 3:04 PM, John Rose wrote: > > On Feb 22, 2018, at 8:49 PM, David Holmes > wrote: > > > I don't think this request has any impact on Fibers at all. At any given > time a piece of executing Java code is executing on a current Thread, and > that current Thread must be running on a native thread (regardless of > mapping) and the native thread has an id. The only use for exposing such an > id is to allow you to correlate information obtained from inside the VM, > with information observed outside, possibly by external tools. > > > We may or may not bind fiber identity to the legacy java.lang.Thread type. > > > I'm not assuming that exposing native thread ids implies anything about > the binding between those entities and anything higher-level. I have no > issue with a Fiber/Continuation/Task/whatever running on different > java.lang.Threads over its lifetime. For that matter I don't think it > matters if java.lang.Threads could execute on different native threads > (they don't but they could). > > David > ----- > > > > This affects the question of whether the following code can return false: > > boolean testThreadShift(Runnable r) { > Thread t0 = Thread.current(); > r.run(); > Thread t1 = Thread.current(); > return t0 == t1; > } > > This method can return false if (a) Thread.current() reports the identity > of the OS thread (not the fiber currently running), and (b) the runnable > includes a blocking operation which causes the current fiber to reschedule > on a different OS thread before it returns. > > Having this method return false is, perhaps, surprising enough to cause > us to inject fiber identity into java.lang.Thread, so that a > java.lang.Thread > identity is 1-1 with a fiber, rather than an OS "dinosaur" thread. (You > can > only pick one!) The alternative is to keep a 1-1 identity between OS > threads > and java.lang.Thread instances, making a new type which carries fiber > identity that multiplexes in a scheduler-dependent way over OS threads. > > It's an interesting trade-off, with many compatibility concerns. > I think we are leaning toward injecting fiber identity into Thread, > although this makes me shudder to think of the overheads we > will buy with this one move (ThreadLocal hash tables). > > So, now if you reify the OS thread identity, you could have the same > issue again. Can this routine return false? > > boolean testNTIDShift(Runnable r) { > long t0 = NativeThreadID.current(); > r.run(); > long t1 = NativeThreadID.current(); > return t0 == t1; > } > > I think, if you are digging down all the way to the native thread ID, > this routine *must* be allowed to return false under a Loom JVM. > There's no credible way to assign unique int64_t IDs to fiber objects > in the Java heap, as if they were OS objects. > > So my question is, do the proposed use cases for NTID allow > for testNTIDShift to return false? If not, then it's pretty easy to say > that we don't want to supply NativeThreadID.current() for general > use, just to people implementing schedulers. > > ? John > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mandy.chung at oracle.com Tue Feb 27 00:53:23 2018 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 26 Feb 2018 16:53:23 -0800 Subject: RFR JDK-8198253: ThreadInfo.from(CompositeData) assigning fields incorrectly in JDK 9 In-Reply-To: References: <45a5eee6-023d-e342-d6ed-6797702b8804@oracle.com> <9110e27c-e492-b089-7b02-36459915c779@oracle.com> <2e7cf7ef-7a60-cd52-e5ab-a18e4b03f7b3@oracle.com> Message-ID: <6fbda4d8-ad57-0b1a-9f72-e736d349fd74@oracle.com> On 2/26/18 4:23 PM, Jeremy Manson wrote: > Hi Mandy, > > Thanks for taking this on!? I'm happy to see that you are happy to do > cleanups I was too timid to do (like adding the Factory in the tests). > > I note a few places in the test code where static initializers can > throw RuntimeExceptions.? When I ran the tests, and a static > initializer threw a RuntimeException, I didn't see it reflected in the > output, so I had to add print statements.? Was I just doing it wrong, > or is that a feature of jtreg? > Do you see ExceptionInInitializerError and its cause should be RuntimeException?? ExceptionInInitializerError is thrown when fails. > Just a couple of random nits: > > >? 267 ? ? ?* @throws IllegalArgumentException if the given > CompositeData does not > > ?268 ? ? ?* contain all attributes representing ThreadInfo of any > release. > > Could this be misinterpreted?? If you have everything in V5, and only > one of the V6 attributes, it contains all of the attributes > representing the ThreadInfo of V5.? Maybe you want to say "does not > contain exactly the attributes present in ThreadInfo for a particular > release"? > The javadoc has this statement: ?? A CompositeData representing a ThreadInfo of version N must contain all the attributes defined since N or earlier unless specified otherwise. What about: @throws IAE if the given CompositeData does not contain all {@linkplain #attrs ThreadInfo attributes} for a? release. I will add a link to the table. > For signatures like: > > > static CompositeType initV5CompositeType(CompositeType ctype) { > > I probably would change the name of ctype to indicate when you are > reading the method that the parameter is always supposed to be the the > CompositeType for ThreadInfo. > > Let me think about it and send another webrev.? I am tempted to improve and provide an automated way for the versioning support. Mandy > > > On Fri, Feb 23, 2018 at 4:50 PM, mandy chung > wrote: > > Webrev at: > http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/webrev.01 > > > This patch updates the spec to clarify what attributes are required > since which release. There is a spec bug that "classLoaderName" > added in JDK 9 is missing in the CompositeData for StackTraceElement > but the implementation is correct. I will file a CSR for this update. > > This patch ensures that CompositeData for ThreadInfo of version N > must have the attributes defined since <= N. > ThreadInfo::from also makes sure 'stackTrace' and 'lockedMonitors' > attributes of version N while it can include additional attributes > which has been the current behavior. > > JDK-8139587 intended to support older versions of StackTraceElement > which does not seem a complete solution. I reverted the fix for > JDK-8139587 (mostly) and removed TypeVersionMapper. The fix constructs > the CompositeType for ThreadInfo and MonitorInfo of different > versions and used them for validation. Minor cleanup: the static > final variables are renamed to all capitals. CompatibilityTest.java > test is missing the copyright header. > > thanks > Mandy > [1]http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/api/java/lang/management/ThreadInfo.html#from(javax.management.openmbean.CompositeData) > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mandy.chung at oracle.com Tue Feb 27 04:20:42 2018 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 26 Feb 2018 20:20:42 -0800 Subject: RFR JDK-8198253: ThreadInfo.from(CompositeData) assigning fields incorrectly in JDK 9 In-Reply-To: <2e7cf7ef-7a60-cd52-e5ab-a18e4b03f7b3@oracle.com> References: <45a5eee6-023d-e342-d6ed-6797702b8804@oracle.com> <9110e27c-e492-b089-7b02-36459915c779@oracle.com> <2e7cf7ef-7a60-cd52-e5ab-a18e4b03f7b3@oracle.com> Message-ID: <93e8e891-5f21-a3f3-832b-5d6e4f8e4cc8@oracle.com> CSR: https://bugs.openjdk.java.net/browse/JDK-8198732 Can you please review? Mandy On 2/23/18 4:50 PM, mandy chung wrote: > Webrev at: > http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/webrev.01 > > This patch updates the spec to clarify what attributes are required > since which release. There is a spec bug that "classLoaderName" > added in JDK 9 is missing in the CompositeData for StackTraceElement > but the implementation is correct. I will file a CSR for this update. > > This patch ensures that CompositeData for ThreadInfo of version N > must have the attributes defined since <= N. > ThreadInfo::from also makes sure 'stackTrace' and 'lockedMonitors' > attributes of version N while it can include additional attributes > which has been the current behavior. > > JDK-8139587 intended to support older versions of StackTraceElement > which does not seem a complete solution. I reverted the fix for > JDK-8139587 (mostly) and removed TypeVersionMapper. The fix constructs > the CompositeType for ThreadInfo and MonitorInfo of different > versions and used them for validation. Minor cleanup: the static > final variables are renamed to all capitals. CompatibilityTest.java > test is missing the copyright header. > > thanks > Mandy > [1]http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/api/java/lang/management/ThreadInfo.html#from(javax.management.openmbean.CompositeData) -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Tue Feb 27 04:20:05 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 27 Feb 2018 14:20:05 +1000 Subject: RFR 8170541: serviceability/jdwp/AllModulesCommandTest.java fails intermittently on Windows and Solaris In-Reply-To: References: <86bf7cb3-50a2-ddeb-0f88-73b8b2b4b2cf@oracle.com> <18D0CADB-CDB9-4F5D-B77A-DE5A5F6A5C19@oracle.com> <4ba6d0b1-0277-3e5e-1aa9-49e1db636586@oracle.com> <1c404611-6021-96d2-5f5f-1f48735ab5e2@oracle.com> <07e6e62e-5713-fd57-34f6-ad439e8da209@oracle.com> Message-ID: <1b51738d-4eee-f07a-4342-a814bd30eacd@oracle.com> The two-step send came in with: https://bugs.openjdk.java.net/browse/JDK-6401245 "Small JDWP packets with the socket transport causes slow debugging on linux 2.6.15 kernel and newer" David ----- On 27/02/2018 9:29 AM, serguei.spitsyn at oracle.com wrote: > On 2/26/18 15:06, Chris Plummer wrote: >> On 2/26/18 3:00 PM, daniil.x.titov at oracle.com wrote: >>> >>> >>> On 2/26/18 12:16 PM, Chris Plummer wrote: >>>> On 2/26/18 11:51 AM, daniil.x.titov at oracle.com wrote: >>>>> Hi David and Sergei, >>>>> >>>>> On 2/20/18 10:16 PM, serguei.spitsyn at oracle.com wrote: >>>>>> Hi David, >>>>>> >>>>>> >>>>>> On 2/20/18 20:02, David Holmes wrote: >>>>>>> Hi Daniil, >>>>>>> >>>>>>> Good find on this! >>>>>>> >>>>>>> What does the actual spec say about the length of things and how >>>>>>> they may be split across multiple packets? Are we guaranteed that >>>>>>> at most two packets will be involved? >>>>> >>>>> The JDWP spec >>>>> (https://docs.oracle.com/javase/9/docs/specs/jdwp/jdwp-spec.html) >>>>> says nothing about splitting JDWP reply packets at all but the >>>>> implementation limits the max number of the sent packets to two >>>>> packets max. The implementation is dated back to the initial load >>>>> that happened in 2007 and the information about the related Jira >>>>> issue is missing. >>>>> >>>>> open/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c >>>>> >>>>> 836 ?? data = packet->type.cmd.data; >>>>> 837 ?? /* Do one send for short packets, two for longer ones */ >>>>> 838 ?? if (data_len <= MAX_DATA_SIZE) { >>>>> 839? ????? memcpy(header + JDWP_HEADER_SIZE, data, data_len); >>>>> 840 ?????? if (send_fully(socketFD, (char *)&header, >>>>> JDWP_HEADER_SIZE + data_len) != >>>>> 841? ????????? JDWP_HEADER_SIZE + data_len) { >>>>> 842 ?????????? RETURN_IO_ERROR("send failed"); >>>>> 843? ????? } >>>>> 844 ?? } else { >>>>> 845? ????? memcpy(header + JDWP_HEADER_SIZE, data, MAX_DATA_SIZE); >>>>> 846 ?????? if (send_fully(socketFD, (char *)&header, >>>>> JDWP_HEADER_SIZE + MAX_DATA_SIZE) != >>>>> 847 ?????????? JDWP_HEADER_SIZE + MAX_DATA_SIZE) { >>>>> 848 ?????????? RETURN_IO_ERROR("send failed"); >>>>> 849 ?????? } >>>>> 850 ? ???? /* Send the remaining data bytes right out of the data >>>>> area. */ >>>>> 851 ?????? if (send_fully(socketFD, (char *)data + MAX_DATA_SIZE, >>>>> 852 ????????????????????? data_len - MAX_DATA_SIZE) != data_len - >>>>> MAX_DATA_SIZE) { >>>>> 853 ?????????? RETURN_IO_ERROR("send failed"); >>>>> 854 ?????? } >>>>> 855 ?? } >>>>> >>>> Curious. First packet is limited to MAX_DATA_SIZE, 2nd packet has no >>>> size limit. What's the point then of splitting it then? Is there a >>>> desire to get the header transmitted in a smaller packet. >>>> >>>> Chris >>> >>> It looks as the goal was to somehow improve the responsiveness in >>> case of the large data but I am not sure about this. I could not >>> locate any traces in Jira related to this implementation. >> I was thinking it might be something like that too. Get the header >> across the wire quickly. Maybe the user just wants the header (with >> size info) initially, and will allocate a large buffer for the rest if >> necessary. > > It was my guess too. > At least, it is the best explanation for this design that looks > reasonable to me. > > >> Chris >>> Probably Serguei has some info what is the history behind this design. > > I don't know the history here. > This was implemented in very early days, most likely, before JDK 1.5 or > even 1.4.2. > > Thanks, > Serguei > >>>>>>> >>>>>>> ?68???? protected byte[] readJdwpString(DataInputStream ds) >>>>>>> throws IOException { >>>>>>> ? 69???????? byte[] str = null; >>>>>>> ? 70???????? int len = ds.readInt(); >>>>>>> ? 71???????? if (len > 0) { >>>>>>> ? 72???????????? str = new byte[len]; >>>>>>> ? 73???????????? ds.read(str, 0, len); >>>>>>> ? 74???????? } >>>>>>> >>>>>>> might we get a short-read of the string if it is split across >>>>>>> multiple packets? >>>>>> >>>>> This and all other reads happen not directly from the socket input >>>>> stream but rather from the? DataInputStream object that is >>>>> constructed in JdwpReply.initFromStream(InputStream) method. With >>>>> the proposed fix we do ensure that the created DataInputStream >>>>> object contains data from both packets in cases when the reply was >>>>> split in two packets. >>>>> >>>>>> Nice catch! >>>>>> Even though this fix is enough to resolve this problem now, there >>>>>> is a chance, >>>>>> it can fail in the future when more modules are added to the >>>>>> platform. >>>>>> >>>>>> >>>>>>> I'm wondering if all these reads should be loops, ensuring we >>>>>>> read the expected amount of data. >>>>>>> >>>>> Since the implementation of the socket transport limits the max >>>>> number of packets the reply might be split in to two packets I >>>>> don't think we really need it here. >>>>>>> One further comment - not sure why we need the print out for when >>>>>>> we do read multiple packets? >>>>>>> That would seem to be a debugging aid. >>>>>> >>>>>> Yes, it helps to understand what happens. >>>>>> Many tests have a lack of tracing which makes it harder to debug >>>>>> and understand failures. >>>>> That is correct.? This additional tracing was added to help to >>>>> understand the possible failures in the future. >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> >>>>> Thanks, >>>>> Daniil >>>>> >>>>>>> On 21/02/2018 10:14 AM, Daniil Titov wrote: >>>>>>>> Hi Serguei, >>>>>>>> >>>>>>>> A new version of the webrev that has these strings reformatted >>>>>>>> is at http://cr.openjdk.java.net/~dtitov/8170541/webrev.02/ >>>>>>>> >>>>>>>> Thank you! >>>>>>>> >>>>>>>> Best regards, >>>>>>>> >>>>>>>> Daniil >>>>>>>> >>>>>>>> *From: *"serguei.spitsyn at oracle.com" >>>>>>>> *Date: *Tuesday, February 20, 2018 at 3:00 PM >>>>>>>> *To: *Daniil Titov , >>>>>>>> "serviceability-dev at openjdk.java.net" >>>>>>>> >>>>>>>> *Subject: *Re: RFR 8170541: >>>>>>>> serviceability/jdwp/AllModulesCommandTest.java fails >>>>>>>> intermittently on Windows and Solaris >>>>>>>> >>>>>>>> Hi Daniil, >>>>>>>> >>>>>>>> Interesting issue... >>>>>>>> Thank you for finding to the root cause so quickly! >>>>>>>> >>>>>>>> The fix looks good. >>>>>>>> Could I ask you to reformat these lines to make the L54 shorter ?: >>>>>>>> >>>>>>>> ?? 54???????????????? System.out.println("[" + >>>>>>>> getClass().getName() + "] Only " + bytesRead + " bytes of " + >>>>>>>> dataLength + >>>>>>>> >>>>>>>> ?? 55???????????????????????? " were read in the first packet. >>>>>>>> Reading the rest..."); >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Serguei >>>>>>>> >>>>>>>> >>>>>>>> On 2/20/18 09:24, Daniil Titov wrote: >>>>>>>> >>>>>>>> ??? Please review the changes that fix intermittent failure of >>>>>>>> ??? serviceability/jdwp/AllModulesCommandTest.java test. >>>>>>>> >>>>>>>> ??? The problem here is that for a large data the JDWP agent >>>>>>>> ??? (socketTransport_writePacket() method in >>>>>>>> src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c ) >>>>>>>> ??? sends 2 packets and in some cases only the first packet is >>>>>>>> received >>>>>>>> ??? at the time when the test reads the reply from the JDWP >>>>>>>> agent. Since >>>>>>>> ??? the test does not check that all data is received in the first >>>>>>>> ??? packet the correlation between commands and replies became >>>>>>>> broken >>>>>>>> ??? (the unread second packet is read by the next command and >>>>>>>> the reply >>>>>>>> ??? for the next command is read by the next after next command >>>>>>>> and so on). >>>>>>>> >>>>>>>> ??? Bug: https://bugs.openjdk.java.net/browse/JDK-8170541 >>>>>>>> >>>>>>>> ??? Webrev: http://cr.openjdk.java.net/~dtitov/8170541/webrev.01 >>>>>>>> >>>>>>>> ??? The tests ran successfully with Mach5. >>>>>>>> >>>>>>>> ??? Best regards, >>>>>>>> >>>>>>>> ??? Daniil >>>>>>>> >>>>>>>> >>>>>>>> >>>>>> >>>>> >>>> >>>> >>> >> >> > From jini.george at oracle.com Tue Feb 27 07:37:26 2018 From: jini.george at oracle.com (Jini George) Date: Tue, 27 Feb 2018 13:07:26 +0530 Subject: RFR: JDK-8175312: SA: clhsdb: Provide an improved heap summary for 'universe' for G1GC In-Reply-To: References: Message-ID: <38d71740-0b66-3ce8-26ed-a0f2b9f9e91c@oracle.com> Gentle reminder! Thanks, Jini. On 2/14/2018 2:37 PM, Jini George wrote: > Hello, > > (Including the SVC and GC groups) > > Requesting reviews for enabling the clhsdb 'universe' command to print > out the various G1 regions (along with the type). > > JIRA ID:https://bugs.openjdk.java.net/browse/JDK-8175312 > Webrev: http://cr.openjdk.java.net/~jgeorge/8175312/webrev.00/index.html > > Modifications have been made to display the regions like: > > ... > Region: 0x00000005c5400000,0x00000005c5600000,0x00000005c5600000:Old > Region: 0x00000005c5600000,0x00000005c5800000,0x00000005c5800000:Old > Region: 0x00000005c5800000,0x00000005c5a00000,0x00000005c5a00000:Old > Region: 0x00000005c5a00000,0x00000005c5c00000,0x00000005c5c00000:Old > Region: 0x00000005c5c00000,0x00000005c5c00000,0x00000005c5e00000:Free > Region: 0x00000005c5e00000,0x00000005c5e00000,0x00000005c6000000:Free > Region: 0x00000005c6000000,0x00000005c6200000,0x00000005c6200000:Old > ... > > The jtreg test at this point does not include any testing for the > display of archived or pinned regions. The testing for this will be > added once JDK-8174994 is resolved. > > The SA tests pass with jprt and Mach5. > > Thanks, > Jini. From stefan.johansson at oracle.com Tue Feb 27 10:00:51 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Tue, 27 Feb 2018 11:00:51 +0100 Subject: RFR: JDK-8175312: SA: clhsdb: Provide an improved heap summary for 'universe' for G1GC In-Reply-To: <38d71740-0b66-3ce8-26ed-a0f2b9f9e91c@oracle.com> References: <38d71740-0b66-3ce8-26ed-a0f2b9f9e91c@oracle.com> Message-ID: <5e8c582e-b32f-daf7-0e0c-1e6606ceaf3a@oracle.com> Hi Jini, On 2018-02-27 08:37, Jini George wrote: > Gentle reminder! > > Thanks, > Jini. > > On 2/14/2018 2:37 PM, Jini George wrote: >> Hello, >> >> (Including the SVC and GC groups) >> >> Requesting reviews for enabling the clhsdb 'universe' command to >> print out the various G1 regions (along with the type). >> >> JIRA ID:https://bugs.openjdk.java.net/browse/JDK-8175312 >> Webrev: http://cr.openjdk.java.net/~jgeorge/8175312/webrev.00/index.html >> It looks like a file is missing, did you forget to add it to the changeset? --- open/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/g1/G1CollectedHeap.java:36: error: cannot find symbol import sun.jvm.hotspot.gc.shared.PrintRegionClosure; --- Otherwise the change looks good, but I would like to see the output live. For a big heap this will print a lot of data, just wondering if the universe command is the correct choice for this kind of output. I like having the possibility to print all regions, so I want the change but maybe it should be a different command and 'universe' just prints a little more than before. Something like our logging heap-summary at shutdown: garbage-first heap?? total 16384K, used 3072K [0x00000000ff000000, 0x0000000100000000) ?region size 1024K, 4 young (4096K), 0 survivors (0K) Metaspace?????? used 6731K, capacity 6825K, committed 7040K, reserved 1056768K ?class space??? used 559K, capacity 594K, committed 640K, reserved 1048576K Thanks, Stefan >> Modifications have been made to display the regions like: >> >> ... >> Region: 0x00000005c5400000,0x00000005c5600000,0x00000005c5600000:Old >> Region: 0x00000005c5600000,0x00000005c5800000,0x00000005c5800000:Old >> Region: 0x00000005c5800000,0x00000005c5a00000,0x00000005c5a00000:Old >> Region: 0x00000005c5a00000,0x00000005c5c00000,0x00000005c5c00000:Old >> Region: 0x00000005c5c00000,0x00000005c5c00000,0x00000005c5e00000:Free >> Region: 0x00000005c5e00000,0x00000005c5e00000,0x00000005c6000000:Free >> Region: 0x00000005c6000000,0x00000005c6200000,0x00000005c6200000:Old >> ... >> >> The jtreg test at this point does not include any testing for the >> display of archived or pinned regions. The testing for this will be >> added once JDK-8174994 is resolved. >> >> The SA tests pass with jprt and Mach5. >> >> Thanks, >> Jini. From jeremymanson at google.com Tue Feb 27 17:44:18 2018 From: jeremymanson at google.com (Jeremy Manson) Date: Tue, 27 Feb 2018 09:44:18 -0800 Subject: RFR JDK-8198253: ThreadInfo.from(CompositeData) assigning fields incorrectly in JDK 9 In-Reply-To: <6fbda4d8-ad57-0b1a-9f72-e736d349fd74@oracle.com> References: <45a5eee6-023d-e342-d6ed-6797702b8804@oracle.com> <9110e27c-e492-b089-7b02-36459915c779@oracle.com> <2e7cf7ef-7a60-cd52-e5ab-a18e4b03f7b3@oracle.com> <6fbda4d8-ad57-0b1a-9f72-e736d349fd74@oracle.com> Message-ID: On Mon, Feb 26, 2018 at 4:53 PM, mandy chung wrote: > > > On 2/26/18 4:23 PM, Jeremy Manson wrote: > > Hi Mandy, > > Thanks for taking this on! I'm happy to see that you are happy to do > cleanups I was too timid to do (like adding the Factory in the tests). > > I note a few places in the test code where static initializers can throw > RuntimeExceptions. When I ran the tests, and a static initializer threw a > RuntimeException, I didn't see it reflected in the output, so I had to add > print statements. Was I just doing it wrong, or is that a feature of jtreg? > > > Do you see ExceptionInInitializerError and its cause should be > RuntimeException? ExceptionInInitializerError is thrown when > fails. > There was no relevant output. > > > > Just a couple of random nits: > > > 267 * @throws IllegalArgumentException if the given CompositeData > does not > > 268 * contain all attributes representing ThreadInfo of any > release. > > Could this be misinterpreted? If you have everything in V5, and only one > of the V6 attributes, it contains all of the attributes representing the > ThreadInfo of V5. Maybe you want to say "does not contain exactly the > attributes present in ThreadInfo for a particular release"? > > > The javadoc has this statement: > A CompositeData representing a ThreadInfo of version N must contain all > the attributes defined since N or earlier unless specified otherwise. > > What about: > @throws IAE if the given CompositeData does not contain all {@linkplain > #attrs ThreadInfo attributes} for a release. > > I will add a link to the table. > That phrasing has the same problem. It needs to be *exactly* the ThreadInfo attributes for a release. > > > For signatures like: > > > static CompositeType initV5CompositeType(CompositeType ctype) { > > I probably would change the name of ctype to indicate when you are reading > the method that the parameter is always supposed to be the the > CompositeType for ThreadInfo. > > > > Let me think about it and send another webrev. I am tempted to improve > and provide an automated way for the versioning support. > > Mandy > > > > On Fri, Feb 23, 2018 at 4:50 PM, mandy chung > wrote: > >> Webrev at: >> http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/webrev.01 >> >> This patch updates the spec to clarify what attributes are required >> since which release. There is a spec bug that "classLoaderName" >> added in JDK 9 is missing in the CompositeData for StackTraceElement >> but the implementation is correct. I will file a CSR for this update. >> >> This patch ensures that CompositeData for ThreadInfo of version N >> must have the attributes defined since <= N. >> ThreadInfo::from also makes sure 'stackTrace' and 'lockedMonitors' >> attributes of version N while it can include additional attributes >> which has been the current behavior. >> >> JDK-8139587 intended to support older versions of StackTraceElement >> which does not seem a complete solution. I reverted the fix for >> JDK-8139587 (mostly) and removed TypeVersionMapper. The fix constructs >> the CompositeType for ThreadInfo and MonitorInfo of different >> versions and used them for validation. Minor cleanup: the static >> final variables are renamed to all capitals. CompatibilityTest.java >> test is missing the copyright header. >> >> thanks >> Mandy >> [1] http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/api/java/lang/management/ThreadInfo.html#from(javax.management.openmbean.CompositeData) >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeremymanson at google.com Tue Feb 27 17:56:11 2018 From: jeremymanson at google.com (Jeremy Manson) Date: Tue, 27 Feb 2018 09:56:11 -0800 Subject: RFR JDK-8198253: ThreadInfo.from(CompositeData) assigning fields incorrectly in JDK 9 In-Reply-To: <93e8e891-5f21-a3f3-832b-5d6e4f8e4cc8@oracle.com> References: <45a5eee6-023d-e342-d6ed-6797702b8804@oracle.com> <9110e27c-e492-b089-7b02-36459915c779@oracle.com> <2e7cf7ef-7a60-cd52-e5ab-a18e4b03f7b3@oracle.com> <93e8e891-5f21-a3f3-832b-5d6e4f8e4cc8@oracle.com> Message-ID: Comment on new doc wording: * A {@code CompositeData} representing a {@code ThreadInfo} of * version N must contain all the attributes defined * since N or earlier unless specified otherwise. Wouldn't "all of the attributes defined since N or earlier" just mean "all of the attributes"? "Since" is basically the same as "after". Would "must contain all of the attributes for every version up to and including N" work? Jeremy On Mon, Feb 26, 2018 at 8:20 PM, mandy chung wrote: > CSR: https://bugs.openjdk.java.net/browse/JDK-8198732 > > Can you please review? > > Mandy > > On 2/23/18 4:50 PM, mandy chung wrote: > > Webrev at: > http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/webrev.01 > > This patch updates the spec to clarify what attributes are required > since which release. There is a spec bug that "classLoaderName" > added in JDK 9 is missing in the CompositeData for StackTraceElement > but the implementation is correct. I will file a CSR for this update. > > This patch ensures that CompositeData for ThreadInfo of version N > must have the attributes defined since <= N. > ThreadInfo::from also makes sure 'stackTrace' and 'lockedMonitors' > attributes of version N while it can include additional attributes > which has been the current behavior. > > JDK-8139587 intended to support older versions of StackTraceElement > which does not seem a complete solution. I reverted the fix for > JDK-8139587 (mostly) and removed TypeVersionMapper. The fix constructs > the CompositeType for ThreadInfo and MonitorInfo of different > versions and used them for validation. Minor cleanup: the static > final variables are renamed to all capitals. CompatibilityTest.java > test is missing the copyright header. > > thanks > Mandy > [1] http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/api/java/lang/management/ThreadInfo.html#from(javax.management.openmbean.CompositeData) > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mandy.chung at oracle.com Tue Feb 27 18:55:22 2018 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 27 Feb 2018 10:55:22 -0800 Subject: RFR JDK-8198253: ThreadInfo.from(CompositeData) assigning fields incorrectly in JDK 9 In-Reply-To: References: <45a5eee6-023d-e342-d6ed-6797702b8804@oracle.com> <9110e27c-e492-b089-7b02-36459915c779@oracle.com> <2e7cf7ef-7a60-cd52-e5ab-a18e4b03f7b3@oracle.com> <93e8e891-5f21-a3f3-832b-5d6e4f8e4cc8@oracle.com> Message-ID: <38ad0751-a4a6-9010-5cdb-5941e21b2421@oracle.com> Good point, Jeremy. I notice some strange-ness when I wrote it but wasn't able to pin point the error. Daniel also suggests to clarify MonitorInfo as well. Does this version look better? * Returns a {@code ThreadInfo} object represented by the * given {@code CompositeData}. * * A {@code CompositeData} representing a {@code ThreadInfo} of * version N must contain all of the attributes defined * in version ≤ N unless specified otherwise. * Same rule applies transitively to attributes whose type or * component type is {@code CompositeType}. *

* A {@code CompositeData} representing {@code ThreadInfo} of version * N contains {@code "stackTrace"} attribute representing * an array of {@code StackTraceElement} of version N. * The {@code "lockedMonitors"} attribute represents * an array of {@link MonitorInfo} of version N * which implies that its {@code "lockedStackFrame"} attribute also * represents {@code StackTraceElement} of the same version, N. * Otherwise, this method will throw {@code IllegalArgumentException}. Mandy On 2/27/18 9:56 AM, Jeremy Manson wrote: > Comment on new doc wording: > > > * A {@code CompositeData} representing a {@code ThreadInfo} of > * version N must contain all the attributes defined > * since N or earlier unless specified otherwise. > > Wouldn't "all of the attributes defined since N or earlier" just mean > "all of the attributes"?? "Since" is basically the same as "after".? > Would "must contain all of the attributes for every version up to and > including N" work? > > Jeremy > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mandy.chung at oracle.com Tue Feb 27 20:31:32 2018 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 27 Feb 2018 12:31:32 -0800 Subject: RFR JDK-8198253: ThreadInfo.from(CompositeData) assigning fields incorrectly in JDK 9 In-Reply-To: <38ad0751-a4a6-9010-5cdb-5941e21b2421@oracle.com> References: <45a5eee6-023d-e342-d6ed-6797702b8804@oracle.com> <9110e27c-e492-b089-7b02-36459915c779@oracle.com> <2e7cf7ef-7a60-cd52-e5ab-a18e4b03f7b3@oracle.com> <93e8e891-5f21-a3f3-832b-5d6e4f8e4cc8@oracle.com> <38ad0751-a4a6-9010-5cdb-5941e21b2421@oracle.com> Message-ID: I made further edits to the javadoc and I am happy with this version (move out the attributes for StackTraceElement as a separate table). Specdiff: http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/specdiff/overview-summary.html javadoc: http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/api/java/lang/management/ Webrev: http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/webrev.02/ Mandy On 2/27/18 10:55 AM, mandy chung wrote: > Good point, Jeremy. I notice some strange-ness when I wrote it but > wasn't able to pin point the error. Daniel also suggests to clarify > MonitorInfo as well. > > Does this version look better? > > > * Returns a {@code ThreadInfo} object represented by the > * given {@code CompositeData}. > * > * A {@code CompositeData} representing a {@code ThreadInfo} of > * version N must contain all of the attributes defined > * in version ≤ N unless specified otherwise. > * Same rule applies transitively to attributes whose type or > * component type is {@code CompositeType}. > *

> * A {@code CompositeData} representing {@code ThreadInfo} of version > * N contains {@code "stackTrace"} attribute representing > * an array of {@code StackTraceElement} of version N. > * The {@code "lockedMonitors"} attribute represents > * an array of {@link MonitorInfo} of version N > * which implies that its {@code "lockedStackFrame"} attribute also > * represents {@code StackTraceElement} of the same version, N. > * Otherwise, this method will throw {@code IllegalArgumentException}. > > Mandy -------------- next part -------------- An HTML attachment was scrubbed... URL: From mandy.chung at oracle.com Tue Feb 27 23:46:24 2018 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 27 Feb 2018 15:46:24 -0800 Subject: RFR JDK-8198253: ThreadInfo.from(CompositeData) assigning fields incorrectly in JDK 9 In-Reply-To: References: <45a5eee6-023d-e342-d6ed-6797702b8804@oracle.com> <9110e27c-e492-b089-7b02-36459915c779@oracle.com> <2e7cf7ef-7a60-cd52-e5ab-a18e4b03f7b3@oracle.com> <6fbda4d8-ad57-0b1a-9f72-e736d349fd74@oracle.com> Message-ID: On 2/27/18 9:44 AM, Jeremy Manson wrote: > > > On Mon, Feb 26, 2018 at 4:53 PM, mandy chung > wrote: > > > > On 2/26/18 4:23 PM, Jeremy Manson wrote: >> Hi Mandy, >> >> Thanks for taking this on!? I'm happy to see that you are happy >> to do cleanups I was too timid to do (like adding the Factory in >> the tests). >> >> I note a few places in the test code where static initializers >> can throw RuntimeExceptions.? When I ran the tests, and a static >> initializer threw a RuntimeException, I didn't see it reflected >> in the output, so I had to add print statements.? Was I just >> doing it wrong, or is that a feature of jtreg? >> > > Do you see ExceptionInInitializerError and its cause should be > RuntimeException? ExceptionInInitializerError is thrown when > fails. > > > There was no relevant output. I don't see the issue.? I add "throw new RuntimeException("abc")" and force the exception.? I got this output: ----------System.err:(17/1008)---------- java.lang.ExceptionInInitializerError ??????? at ThreadInfoCompositeData.createGoodCompositeData(ThreadInfoCompositeData.java:72) ??????? at ThreadInfoCompositeData.main(ThreadInfoCompositeData.java:55) ??????? at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ??????? at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ??????? at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ??????? at java.base/java.lang.reflect.Method.invoke(Method.java:564) ??????? at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:115) ??????? at java.base/java.lang.Thread.run(Thread.java:841) Caused by: java.lang.RuntimeException: abc ??????? at ThreadInfoCompositeData$Factory.(ThreadInfoCompositeData.java:317) ??????? ... 8 more Mandy -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniil.x.titov at oracle.com Wed Feb 28 05:08:33 2018 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Tue, 27 Feb 2018 21:08:33 -0800 Subject: RFR 8170541: serviceability/jdwp/AllModulesCommandTest.java fails intermittently on Windows and Solaris In-Reply-To: <1b51738d-4eee-f07a-4342-a814bd30eacd@oracle.com> References: <86bf7cb3-50a2-ddeb-0f88-73b8b2b4b2cf@oracle.com> <18D0CADB-CDB9-4F5D-B77A-DE5A5F6A5C19@oracle.com> <4ba6d0b1-0277-3e5e-1aa9-49e1db636586@oracle.com> <1c404611-6021-96d2-5f5f-1f48735ab5e2@oracle.com> <07e6e62e-5713-fd57-34f6-ad439e8da209@oracle.com> <1b51738d-4eee-f07a-4342-a814bd30eacd@oracle.com> Message-ID: Thank you, David! Best regards, Daniil ?On 2/26/18, 8:20 PM, "David Holmes" wrote: The two-step send came in with: https://bugs.openjdk.java.net/browse/JDK-6401245 "Small JDWP packets with the socket transport causes slow debugging on linux 2.6.15 kernel and newer" David ----- On 27/02/2018 9:29 AM, serguei.spitsyn at oracle.com wrote: > On 2/26/18 15:06, Chris Plummer wrote: >> On 2/26/18 3:00 PM, daniil.x.titov at oracle.com wrote: >>> >>> >>> On 2/26/18 12:16 PM, Chris Plummer wrote: >>>> On 2/26/18 11:51 AM, daniil.x.titov at oracle.com wrote: >>>>> Hi David and Sergei, >>>>> >>>>> On 2/20/18 10:16 PM, serguei.spitsyn at oracle.com wrote: >>>>>> Hi David, >>>>>> >>>>>> >>>>>> On 2/20/18 20:02, David Holmes wrote: >>>>>>> Hi Daniil, >>>>>>> >>>>>>> Good find on this! >>>>>>> >>>>>>> What does the actual spec say about the length of things and how >>>>>>> they may be split across multiple packets? Are we guaranteed that >>>>>>> at most two packets will be involved? >>>>> >>>>> The JDWP spec >>>>> (https://docs.oracle.com/javase/9/docs/specs/jdwp/jdwp-spec.html) >>>>> says nothing about splitting JDWP reply packets at all but the >>>>> implementation limits the max number of the sent packets to two >>>>> packets max. The implementation is dated back to the initial load >>>>> that happened in 2007 and the information about the related Jira >>>>> issue is missing. >>>>> >>>>> open/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c >>>>> >>>>> 836 data = packet->type.cmd.data; >>>>> 837 /* Do one send for short packets, two for longer ones */ >>>>> 838 if (data_len <= MAX_DATA_SIZE) { >>>>> 839 memcpy(header + JDWP_HEADER_SIZE, data, data_len); >>>>> 840 if (send_fully(socketFD, (char *)&header, >>>>> JDWP_HEADER_SIZE + data_len) != >>>>> 841 JDWP_HEADER_SIZE + data_len) { >>>>> 842 RETURN_IO_ERROR("send failed"); >>>>> 843 } >>>>> 844 } else { >>>>> 845 memcpy(header + JDWP_HEADER_SIZE, data, MAX_DATA_SIZE); >>>>> 846 if (send_fully(socketFD, (char *)&header, >>>>> JDWP_HEADER_SIZE + MAX_DATA_SIZE) != >>>>> 847 JDWP_HEADER_SIZE + MAX_DATA_SIZE) { >>>>> 848 RETURN_IO_ERROR("send failed"); >>>>> 849 } >>>>> 850 /* Send the remaining data bytes right out of the data >>>>> area. */ >>>>> 851 if (send_fully(socketFD, (char *)data + MAX_DATA_SIZE, >>>>> 852 data_len - MAX_DATA_SIZE) != data_len - >>>>> MAX_DATA_SIZE) { >>>>> 853 RETURN_IO_ERROR("send failed"); >>>>> 854 } >>>>> 855 } >>>>> >>>> Curious. First packet is limited to MAX_DATA_SIZE, 2nd packet has no >>>> size limit. What's the point then of splitting it then? Is there a >>>> desire to get the header transmitted in a smaller packet. >>>> >>>> Chris >>> >>> It looks as the goal was to somehow improve the responsiveness in >>> case of the large data but I am not sure about this. I could not >>> locate any traces in Jira related to this implementation. >> I was thinking it might be something like that too. Get the header >> across the wire quickly. Maybe the user just wants the header (with >> size info) initially, and will allocate a large buffer for the rest if >> necessary. > > It was my guess too. > At least, it is the best explanation for this design that looks > reasonable to me. > > >> Chris >>> Probably Serguei has some info what is the history behind this design. > > I don't know the history here. > This was implemented in very early days, most likely, before JDK 1.5 or > even 1.4.2. > > Thanks, > Serguei > >>>>>>> >>>>>>> 68 protected byte[] readJdwpString(DataInputStream ds) >>>>>>> throws IOException { >>>>>>> 69 byte[] str = null; >>>>>>> 70 int len = ds.readInt(); >>>>>>> 71 if (len > 0) { >>>>>>> 72 str = new byte[len]; >>>>>>> 73 ds.read(str, 0, len); >>>>>>> 74 } >>>>>>> >>>>>>> might we get a short-read of the string if it is split across >>>>>>> multiple packets? >>>>>> >>>>> This and all other reads happen not directly from the socket input >>>>> stream but rather from the DataInputStream object that is >>>>> constructed in JdwpReply.initFromStream(InputStream) method. With >>>>> the proposed fix we do ensure that the created DataInputStream >>>>> object contains data from both packets in cases when the reply was >>>>> split in two packets. >>>>> >>>>>> Nice catch! >>>>>> Even though this fix is enough to resolve this problem now, there >>>>>> is a chance, >>>>>> it can fail in the future when more modules are added to the >>>>>> platform. >>>>>> >>>>>> >>>>>>> I'm wondering if all these reads should be loops, ensuring we >>>>>>> read the expected amount of data. >>>>>>> >>>>> Since the implementation of the socket transport limits the max >>>>> number of packets the reply might be split in to two packets I >>>>> don't think we really need it here. >>>>>>> One further comment - not sure why we need the print out for when >>>>>>> we do read multiple packets? >>>>>>> That would seem to be a debugging aid. >>>>>> >>>>>> Yes, it helps to understand what happens. >>>>>> Many tests have a lack of tracing which makes it harder to debug >>>>>> and understand failures. >>>>> That is correct. This additional tracing was added to help to >>>>> understand the possible failures in the future. >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> >>>>> Thanks, >>>>> Daniil >>>>> >>>>>>> On 21/02/2018 10:14 AM, Daniil Titov wrote: >>>>>>>> Hi Serguei, >>>>>>>> >>>>>>>> A new version of the webrev that has these strings reformatted >>>>>>>> is at http://cr.openjdk.java.net/~dtitov/8170541/webrev.02/ >>>>>>>> >>>>>>>> Thank you! >>>>>>>> >>>>>>>> Best regards, >>>>>>>> >>>>>>>> Daniil >>>>>>>> >>>>>>>> *From: *"serguei.spitsyn at oracle.com" >>>>>>>> *Date: *Tuesday, February 20, 2018 at 3:00 PM >>>>>>>> *To: *Daniil Titov , >>>>>>>> "serviceability-dev at openjdk.java.net" >>>>>>>> >>>>>>>> *Subject: *Re: RFR 8170541: >>>>>>>> serviceability/jdwp/AllModulesCommandTest.java fails >>>>>>>> intermittently on Windows and Solaris >>>>>>>> >>>>>>>> Hi Daniil, >>>>>>>> >>>>>>>> Interesting issue... >>>>>>>> Thank you for finding to the root cause so quickly! >>>>>>>> >>>>>>>> The fix looks good. >>>>>>>> Could I ask you to reformat these lines to make the L54 shorter ?: >>>>>>>> >>>>>>>> 54 System.out.println("[" + >>>>>>>> getClass().getName() + "] Only " + bytesRead + " bytes of " + >>>>>>>> dataLength + >>>>>>>> >>>>>>>> 55 " were read in the first packet. >>>>>>>> Reading the rest..."); >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Serguei >>>>>>>> >>>>>>>> >>>>>>>> On 2/20/18 09:24, Daniil Titov wrote: >>>>>>>> >>>>>>>> Please review the changes that fix intermittent failure of >>>>>>>> serviceability/jdwp/AllModulesCommandTest.java test. >>>>>>>> >>>>>>>> The problem here is that for a large data the JDWP agent >>>>>>>> (socketTransport_writePacket() method in >>>>>>>> src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c ) >>>>>>>> sends 2 packets and in some cases only the first packet is >>>>>>>> received >>>>>>>> at the time when the test reads the reply from the JDWP >>>>>>>> agent. Since >>>>>>>> the test does not check that all data is received in the first >>>>>>>> packet the correlation between commands and replies became >>>>>>>> broken >>>>>>>> (the unread second packet is read by the next command and >>>>>>>> the reply >>>>>>>> for the next command is read by the next after next command >>>>>>>> and so on). >>>>>>>> >>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8170541 >>>>>>>> >>>>>>>> Webrev: http://cr.openjdk.java.net/~dtitov/8170541/webrev.01 >>>>>>>> >>>>>>>> The tests ran successfully with Mach5. >>>>>>>> >>>>>>>> Best regards, >>>>>>>> >>>>>>>> Daniil >>>>>>>> >>>>>>>> >>>>>>>> >>>>>> >>>>> >>>> >>>> >>> >> >> > From jeremymanson at google.com Wed Feb 28 06:04:32 2018 From: jeremymanson at google.com (Jeremy Manson) Date: Tue, 27 Feb 2018 22:04:32 -0800 Subject: RFR JDK-8198253: ThreadInfo.from(CompositeData) assigning fields incorrectly in JDK 9 In-Reply-To: References: <45a5eee6-023d-e342-d6ed-6797702b8804@oracle.com> <9110e27c-e492-b089-7b02-36459915c779@oracle.com> <2e7cf7ef-7a60-cd52-e5ab-a18e4b03f7b3@oracle.com> <6fbda4d8-ad57-0b1a-9f72-e736d349fd74@oracle.com> Message-ID: I'm probably just holding it wrong. :) Jeremy On Tue, Feb 27, 2018 at 3:46 PM, mandy chung wrote: > > > On 2/27/18 9:44 AM, Jeremy Manson wrote: > > > > On Mon, Feb 26, 2018 at 4:53 PM, mandy chung > wrote: > >> >> >> On 2/26/18 4:23 PM, Jeremy Manson wrote: >> >> Hi Mandy, >> >> Thanks for taking this on! I'm happy to see that you are happy to do >> cleanups I was too timid to do (like adding the Factory in the tests). >> >> I note a few places in the test code where static initializers can throw >> RuntimeExceptions. When I ran the tests, and a static initializer threw a >> RuntimeException, I didn't see it reflected in the output, so I had to add >> print statements. Was I just doing it wrong, or is that a feature of jtreg? >> >> >> Do you see ExceptionInInitializerError and its cause should be >> RuntimeException? ExceptionInInitializerError is thrown when >> fails. >> > > There was no relevant output. > > > > I don't see the issue. I add "throw new RuntimeException("abc")" and > force the exception. I got this output: > > ----------System.err:(17/1008)---------- > java.lang.ExceptionInInitializerError > at ThreadInfoCompositeData.createGoodCompositeData( > ThreadInfoCompositeData.java:72) > at ThreadInfoCompositeData.main(ThreadInfoCompositeData.java:55) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke( > NativeMethodAccessorImpl.java:62) > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl. > invoke(DelegatingMethodAccessorImpl.java:43) > at java.base/java.lang.reflect.Method.invoke(Method.java:564) > at com.sun.javatest.regtest.agent.MainWrapper$MainThread. > run(MainWrapper.java:115) > at java.base/java.lang.Thread.run(Thread.java:841) > Caused by: java.lang.RuntimeException: abc > at ThreadInfoCompositeData$Factory.( > ThreadInfoCompositeData.java:317) > ... 8 more > > Mandy > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeremymanson at google.com Wed Feb 28 06:05:09 2018 From: jeremymanson at google.com (Jeremy Manson) Date: Tue, 27 Feb 2018 22:05:09 -0800 Subject: RFR JDK-8198253: ThreadInfo.from(CompositeData) assigning fields incorrectly in JDK 9 In-Reply-To: <38ad0751-a4a6-9010-5cdb-5941e21b2421@oracle.com> References: <45a5eee6-023d-e342-d6ed-6797702b8804@oracle.com> <9110e27c-e492-b089-7b02-36459915c779@oracle.com> <2e7cf7ef-7a60-cd52-e5ab-a18e4b03f7b3@oracle.com> <93e8e891-5f21-a3f3-832b-5d6e4f8e4cc8@oracle.com> <38ad0751-a4a6-9010-5cdb-5941e21b2421@oracle.com> Message-ID: Yup, that's better. I'd probably say "The same rule" instead of "Same rule". Jeremy On Tue, Feb 27, 2018 at 10:55 AM, mandy chung wrote: > Good point, Jeremy. I notice some strange-ness when I wrote it but > wasn't able to pin point the error. Daniel also suggests to clarify > MonitorInfo as well. > > Does this version look better? > > > * Returns a {@code ThreadInfo} object represented by the > * given {@code CompositeData}. > * > * A {@code CompositeData} representing a {@code ThreadInfo} of > * version N must contain all of the attributes defined > * in version ≤ N unless specified otherwise. > * Same rule applies transitively to attributes whose type or > * component type is {@code CompositeType}. > *

> * A {@code CompositeData} representing {@code ThreadInfo} of version > * N contains {@code "stackTrace"} attribute representing > * an array of {@code StackTraceElement} of version N. > * The {@code "lockedMonitors"} attribute represents > * an array of {@link MonitorInfo} of version N > * which implies that its {@code "lockedStackFrame"} attribute also > * represents {@code StackTraceElement} of the same version, N. > * Otherwise, this method will throw {@code IllegalArgumentException}. > > > Mandy > > On 2/27/18 9:56 AM, Jeremy Manson wrote: > > Comment on new doc wording: > > > * A {@code CompositeData} representing a {@code ThreadInfo} of > * version N must contain all the attributes defined > * since N or earlier unless specified otherwise. > > Wouldn't "all of the attributes defined since N or earlier" just mean "all > of the attributes"? "Since" is basically the same as "after". Would "must > contain all of the attributes for every version up to and including N" work? > > Jeremy > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mandy.chung at oracle.com Wed Feb 28 06:16:32 2018 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 27 Feb 2018 22:16:32 -0800 Subject: RFR JDK-8198253: ThreadInfo.from(CompositeData) assigning fields incorrectly in JDK 9 In-Reply-To: References: <45a5eee6-023d-e342-d6ed-6797702b8804@oracle.com> <9110e27c-e492-b089-7b02-36459915c779@oracle.com> <2e7cf7ef-7a60-cd52-e5ab-a18e4b03f7b3@oracle.com> <93e8e891-5f21-a3f3-832b-5d6e4f8e4cc8@oracle.com> <38ad0751-a4a6-9010-5cdb-5941e21b2421@oracle.com> Message-ID: On 2/27/18 10:05 PM, Jeremy Manson wrote: > Yup, that's better.? I'd probably say "The same rule" instead of "Same > rule". Thanks. Will fix it before I push. Mandy > > Jeremy > > On Tue, Feb 27, 2018 at 10:55 AM, mandy chung > wrote: > > Good point, Jeremy. I notice some strange-ness when I wrote it but > wasn't able to pin point the error. Daniel also suggests to clarify > MonitorInfo as well. > > Does this version look better? > > > * Returns a {@code ThreadInfo} object represented by the > * given {@code CompositeData}. > * > * A {@code CompositeData} representing a {@code ThreadInfo} of > * version N must contain all of the attributes defined > * in version ≤ N unless specified otherwise. > * Same rule applies transitively to attributes whose type or > * component type is {@code CompositeType}. > *

> * A {@code CompositeData} representing {@code ThreadInfo} of version > * N contains {@code "stackTrace"} attribute representing > * an array of {@code StackTraceElement} of version N. > * The {@code "lockedMonitors"} attribute represents > * an array of {@link MonitorInfo} of version N > * which implies that its {@code "lockedStackFrame"} attribute also > * represents {@code StackTraceElement} of the same version, N. > * Otherwise, this method will throw {@code IllegalArgumentException}. > > > Mandy > > On 2/27/18 9:56 AM, Jeremy Manson wrote: >> Comment on new doc wording: >> >> >> * A {@code CompositeData} representing a {@code ThreadInfo} of >> * version N must contain all the attributes defined >> * since N or earlier unless specified otherwise. >> >> Wouldn't "all of the attributes defined since N or earlier" just >> mean "all of the attributes"? "Since" is basically the same as >> "after".? Would "must contain all of the attributes for every >> version up to and including N" work? >> >> Jeremy >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Wed Feb 28 07:08:50 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 27 Feb 2018 23:08:50 -0800 Subject: RFR: JDK-8193369: post_field_access does not work for some functions, possibly related to fast_getfield In-Reply-To: <1fca6b67-c0d1-db03-52ed-f2c6bcc29a5b@oracle.com> References: <1fca6b67-c0d1-db03-52ed-f2c6bcc29a5b@oracle.com> Message-ID: <91aadc35-125a-bf74-6cf5-672dc77ffb22@oracle.com> Hi Alex, Thank you for taking care about this! The fix looks good to me. Some comments on the test. http://cr.openjdk.java.net/~amenkov/fast_field_access/webrev/test/hotspot/jtreg/serviceability/jvmti/FieldAccessWatch/FieldAccessWatch.java.html There are some commented lines in the TestResult class. A cleanup is needed to delete them. I guess, it is already in your plan. The empty line #135 is not needed. An empty line is needed after the L99. Probably, the intention was to spell "startTest" insted of "initTest" below: 119 if (!startTest(result)) { 120 throw new RuntimeException("initTest failed"); 121 } I wonder if this sleep is really needed: ??? 124 Thread.sleep(500); The "action.apply()" is executed synchronously, is not it? I'm thinking if moving the test() to native side would simplify things. An Exception can be thrown from native if the test failed or just a boolean status returned. http://cr.openjdk.java.net/~amenkov/fast_field_access/webrev/test/hotspot/jtreg/serviceability/jvmti/FieldAccessWatch/libFieldAccessWatch.c.html I'd suggest to rename currentTestResults to testResultObject, so it will be in line with testResultClass. One concern is that that the reportError() does not cause the test to fail and does not break the execution. Would it better to throw an exception with the same message as was printed? You can find some tests with examples in the serviceability/jvmti folder: static const char *EXC_CNAME = "java/lang/Exception"; static jint throw_exc(JNIEnv *env, char *msg) { ??? jclass exc_class = JNI_ENV_PTR(env)->FindClass(JNI_ENV_ARG(env, EXC_CNAME)); ??? if (exc_class == NULL) { ??????? printf("throw_exc: Error in FindClass(env, %s)\n", EXC_CNAME); ??????? return -1; ??? } ??? return JNI_ENV_PTR(env)->ThrowNew(JNI_ENV_ARG(env, exc_class), msg); } It seems, the function tagAndWatch() adds some complexity to the code. Is all this really needed? Could you, please, add some comments. It does not seem this functions tags anything. 168 (*jvmti)->Deallocate(jvmti, (unsigned char*)sig); The sig needs to be cleared after deallocation as it is used and checked in a loop. Missed initializations: 68 char *name; 142 jfieldID* klassFields; 143 jint fieldCount; Thanks, Serguei On 2/26/18 14:43, Alex Menkov wrote: > Hi all, > > Please review a fix for > JDK-8193369: post_field_access does not work for some functions, > possibly related to fast_getfield > > The fix disables "fast" command generation when FieldAccess or > FieldModification notifications are requested. > > jira: https://bugs.openjdk.java.net/browse/JDK-8193369 > webrev: http://cr.openjdk.java.net/~amenkov/fast_field_access/webrev/ > > --alex From jini.george at oracle.com Wed Feb 28 07:26:00 2018 From: jini.george at oracle.com (Jini George) Date: Wed, 28 Feb 2018 12:56:00 +0530 Subject: RFR: JDK-8175312: SA: clhsdb: Provide an improved heap summary for 'universe' for G1GC In-Reply-To: <5e8c582e-b32f-daf7-0e0c-1e6606ceaf3a@oracle.com> References: <38d71740-0b66-3ce8-26ed-a0f2b9f9e91c@oracle.com> <5e8c582e-b32f-daf7-0e0c-1e6606ceaf3a@oracle.com> Message-ID: Thank you very much, Stefan. My answers inline. On 2/27/2018 3:30 PM, Stefan Johansson wrote: > Hi Jini, >>> JIRA ID:https://bugs.openjdk.java.net/browse/JDK-8175312 >>> Webrev: http://cr.openjdk.java.net/~jgeorge/8175312/webrev.00/index.html >>> > It looks like a file is missing, did you forget to add it to the changeset? Indeed, I had missed that! I added the missing file in the following webrev: http://cr.openjdk.java.net/~jgeorge/8175312/webrev.01/ > --- > open/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/g1/G1CollectedHeap.java:36: > error: cannot find symbol > import sun.jvm.hotspot.gc.shared.PrintRegionClosure; > --- > > Otherwise the change looks good, but I would like to see the output > live. For a big heap this will print a lot of data, just wondering if > the universe command is the correct choice for this kind of output. I > like having the possibility to print all regions, so I want the change > but maybe it should be a different command and 'universe' just prints a > little more than before. Something like our logging heap-summary at > shutdown: > garbage-first heap?? total 16384K, used 3072K [0x00000000ff000000, > 0x0000000100000000) > ?region size 1024K, 4 young (4096K), 0 survivors (0K) > Metaspace?????? used 6731K, capacity 6825K, committed 7040K, reserved > 1056768K > ?class space??? used 559K, capacity 594K, committed 640K, reserved > 1048576K Ok, will add this, and could probably have the region details displayed under a new command called "g1regiondetails", or some such, and send out a new webrev. Thanks, Jini. > > Thanks, > Stefan >>> Modifications have been made to display the regions like: >>> >>> ... >>> Region: 0x00000005c5400000,0x00000005c5600000,0x00000005c5600000:Old >>> Region: 0x00000005c5600000,0x00000005c5800000,0x00000005c5800000:Old >>> Region: 0x00000005c5800000,0x00000005c5a00000,0x00000005c5a00000:Old >>> Region: 0x00000005c5a00000,0x00000005c5c00000,0x00000005c5c00000:Old >>> Region: 0x00000005c5c00000,0x00000005c5c00000,0x00000005c5e00000:Free >>> Region: 0x00000005c5e00000,0x00000005c5e00000,0x00000005c6000000:Free >>> Region: 0x00000005c6000000,0x00000005c6200000,0x00000005c6200000:Old >>> ... >>> >>> The jtreg test at this point does not include any testing for the >>> display of archived or pinned regions. The testing for this will be >>> added once JDK-8174994 is resolved. >>> >>> The SA tests pass with jprt and Mach5. >>> >>> Thanks, >>> Jini. > From daniel.fuchs at oracle.com Wed Feb 28 11:07:04 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 28 Feb 2018 11:07:04 +0000 Subject: RFR JDK-8198253: ThreadInfo.from(CompositeData) assigning fields incorrectly in JDK 9 In-Reply-To: References: <45a5eee6-023d-e342-d6ed-6797702b8804@oracle.com> <9110e27c-e492-b089-7b02-36459915c779@oracle.com> <2e7cf7ef-7a60-cd52-e5ab-a18e4b03f7b3@oracle.com> <93e8e891-5f21-a3f3-832b-5d6e4f8e4cc8@oracle.com> <38ad0751-a4a6-9010-5cdb-5941e21b2421@oracle.com> Message-ID: Hi Mandy, This looks very good. In the API documentation of ThreadInfo::from, below the table that lists the attributes of StackTraceElement, I wonder if the following text should be added for completeness: ```A CompositeData representing a MonitorInfo of version N must contain a lockedStackTrace attribute that is either null if stackDepth < 0, or is a CompositeData representing a StackTraceElement of version N.``` What do you think? best regards, -- danie On 27/02/2018 20:31, mandy chung wrote: > I made further edits to the javadoc and I am happy with this version > (move out the attributes for StackTraceElement as a separate table). > > Specdiff: > http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/specdiff/overview-summary.html > > javadoc: > http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/api/java/lang/management/ > > Webrev: > http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/webrev.02/ > > Mandy > > On 2/27/18 10:55 AM, mandy chung wrote: >> Good point, Jeremy. I notice some strange-ness when I wrote it but >> wasn't able to pin point the error. Daniel also suggests to clarify >> MonitorInfo as well. >> >> Does this version look better? >> >> >> * Returns a {@code ThreadInfo} object represented by the >> * given {@code CompositeData}. >> * >> * A {@code CompositeData} representing a {@code ThreadInfo} of >> * version N must contain all of the attributes defined >> * in version ≤ N unless specified otherwise. >> * Same rule applies transitively to attributes whose type or >> * component type is {@code CompositeType}. >> *

>> * A {@code CompositeData} representing {@code ThreadInfo} of version >> * N contains {@code "stackTrace"} attribute representing >> * an array of {@code StackTraceElement} of version N. >> * The {@code "lockedMonitors"} attribute represents >> * an array of {@link MonitorInfo} of version N >> * which implies that its {@code "lockedStackFrame"} attribute also >> * represents {@code StackTraceElement} of the same version, N. >> * Otherwise, this method will throw {@code IllegalArgumentException}. >> >> Mandy > From mandy.chung at oracle.com Wed Feb 28 18:08:47 2018 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 28 Feb 2018 10:08:47 -0800 Subject: RFR JDK-8198253: ThreadInfo.from(CompositeData) assigning fields incorrectly in JDK 9 In-Reply-To: References: <45a5eee6-023d-e342-d6ed-6797702b8804@oracle.com> <9110e27c-e492-b089-7b02-36459915c779@oracle.com> <2e7cf7ef-7a60-cd52-e5ab-a18e4b03f7b3@oracle.com> <93e8e891-5f21-a3f3-832b-5d6e4f8e4cc8@oracle.com> <38ad0751-a4a6-9010-5cdb-5941e21b2421@oracle.com> Message-ID: <15609ce4-3cc4-2812-3c08-78d4de337a69@oracle.com> Hi Daniel, The from method validates the CompositeType regardless of the attribute value is null or not. In addition, the CompositeData also? I updated the test to make sure that's the case. I tweak the wording s/composite data/composite type/ to avoid the ambiguity. * The {@code "lockedStackFrame"} attribute in * {@link MonitorInfo#from(CompositeData) MonitorInfo}'s composite type * must represent {@code StackTraceElement} of the same version N. Mandy On 2/28/18 3:07 AM, Daniel Fuchs wrote: > Hi Mandy, > > This looks very good. > > In the API documentation of ThreadInfo::from, below the table > that lists the attributes of StackTraceElement, I wonder if the > following text should be added for completeness: > > ```A CompositeData representing a MonitorInfo of version N must > contain a lockedStackTrace attribute that is either null if > stackDepth? < 0, or is a CompositeData representing a > StackTraceElement of version N.``` > > What do you think? > > best regards, > > -- danie > > On 27/02/2018 20:31, mandy chung wrote: >> I made further edits to the javadoc and I am happy with this version >> (move out the attributes for StackTraceElement as a separate table). >> >> Specdiff: >> http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/specdiff/overview-summary.html >> >> >> javadoc: >> http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/api/java/lang/management/ >> >> >> Webrev: >> http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/webrev.02/ >> >> Mandy >> >> On 2/27/18 10:55 AM, mandy chung wrote: >>> Good point, Jeremy.? I notice some strange-ness when I wrote it but >>> wasn't able to pin point the error.? Daniel also suggests to clarify >>> MonitorInfo as well. >>> >>> Does this version look better? >>> >>> ????? * Returns a {@code ThreadInfo} object represented by the >>> ????? * given {@code CompositeData}. >>> ????? * >>> ????? * A {@code CompositeData} representing a {@code ThreadInfo} of >>> ????? * version N must contain all of the attributes defined >>> ????? * in version ≤ N unless specified otherwise. >>> ????? * Same rule applies transitively to attributes whose type or >>> ????? * component type is {@code CompositeType}. >>> ????? *

>>> ????? * A {@code CompositeData} representing {@code ThreadInfo} of >>> version >>> ????? * N contains {@code "stackTrace"} attribute representing >>> ????? * an array of {@code StackTraceElement} of version N. >>> ????? * The {@code "lockedMonitors"} attribute represents >>> ????? * an array of {@link MonitorInfo} of version N >>> ????? * which implies that its {@code "lockedStackFrame"} attribute >>> also >>> ????? * represents {@code StackTraceElement} of the same version, >>> N. >>> ????? * Otherwise, this method will throw {@code >>> IllegalArgumentException}. >>> >>> Mandy >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.fuchs at oracle.com Wed Feb 28 18:46:34 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 28 Feb 2018 18:46:34 +0000 Subject: RFR JDK-8198253: ThreadInfo.from(CompositeData) assigning fields incorrectly in JDK 9 In-Reply-To: <15609ce4-3cc4-2812-3c08-78d4de337a69@oracle.com> References: <45a5eee6-023d-e342-d6ed-6797702b8804@oracle.com> <9110e27c-e492-b089-7b02-36459915c779@oracle.com> <2e7cf7ef-7a60-cd52-e5ab-a18e4b03f7b3@oracle.com> <93e8e891-5f21-a3f3-832b-5d6e4f8e4cc8@oracle.com> <38ad0751-a4a6-9010-5cdb-5941e21b2421@oracle.com> <15609ce4-3cc4-2812-3c08-78d4de337a69@oracle.com> Message-ID: On 28/02/18 18:08, mandy chung wrote: > Hi Daniel, > > The from method validates the CompositeType regardless of the attribute > value is null or not. In addition, the CompositeData also? I updated the > test to make sure that's the case. > > I tweak the wording s/composite data/composite type/ to avoid the ambiguity. > > * The {@code "lockedStackFrame"} attribute in > * {@link MonitorInfo#from(CompositeData) MonitorInfo}'s composite type > * must represent {@code StackTraceElement} of the same version N. Looks fine! -- daniel > > Mandy > > On 2/28/18 3:07 AM, Daniel Fuchs wrote: >> Hi Mandy, >> >> This looks very good. >> >> In the API documentation of ThreadInfo::from, below the table >> that lists the attributes of StackTraceElement, I wonder if the >> following text should be added for completeness: >> >> ```A CompositeData representing a MonitorInfo of version N must >> contain a lockedStackTrace attribute that is either null if >> stackDepth? < 0, or is a CompositeData representing a >> StackTraceElement of version N.``` >> >> What do you think? >> >> best regards, >> >> -- danie >> >> On 27/02/2018 20:31, mandy chung wrote: >>> I made further edits to the javadoc and I am happy with this version >>> (move out the attributes for StackTraceElement as a separate table). >>> >>> Specdiff: >>> http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/specdiff/overview-summary.html >>> >>> >>> javadoc: >>> http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/api/java/lang/management/ >>> >>> >>> Webrev: >>> http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8198253/webrev.02/ >>> >>> Mandy >>> >>> On 2/27/18 10:55 AM, mandy chung wrote: >>>> Good point, Jeremy.? I notice some strange-ness when I wrote it but >>>> wasn't able to pin point the error.? Daniel also suggests to clarify >>>> MonitorInfo as well. >>>> >>>> Does this version look better? >>>> >>>> ????? * Returns a {@code ThreadInfo} object represented by the >>>> ????? * given {@code CompositeData}. >>>> ????? * >>>> ????? * A {@code CompositeData} representing a {@code ThreadInfo} of >>>> ????? * version N must contain all of the attributes defined >>>> ????? * in version ≤ N unless specified otherwise. >>>> ????? * Same rule applies transitively to attributes whose type or >>>> ????? * component type is {@code CompositeType}. >>>> ????? *

>>>> ????? * A {@code CompositeData} representing {@code ThreadInfo} of >>>> version >>>> ????? * N contains {@code "stackTrace"} attribute representing >>>> ????? * an array of {@code StackTraceElement} of version N. >>>> ????? * The {@code "lockedMonitors"} attribute represents >>>> ????? * an array of {@link MonitorInfo} of version N >>>> ????? * which implies that its {@code "lockedStackFrame"} attribute >>>> also >>>> ????? * represents {@code StackTraceElement} of the same version, >>>> N. >>>> ????? * Otherwise, this method will throw {@code >>>> IllegalArgumentException}. >>>> >>>> Mandy >>> >> >