From daniil.x.titov at oracle.com Sat Dec 1 03:08:53 2018 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Fri, 30 Nov 2018 19:08:53 -0800 Subject: RFR 8214572: nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the thread when the top frame executes JVMCI code Message-ID: <4E0EFD3E-F725-4469-BA20-7532D78BA479@oracle.com> Please review the change for nsk/jvmti/unit/ForceEarlyReturn/earlyretbase test. The problem here is that before doing an early force return the test doesn't check that the test thread is suspended at a right place where the top frame executes the test method rather than JVMCI code triggered by invocation counter overflow. That results in the early return happens for a wrong method and the test fails. The fix changes the test to do resume/suspend in the loop until the target method is executed in the top frame or the loop counter exceeds the limit. There is another problem with this test that requires changes on VM side and is currently covered by JDK-8195635:" [Graal] nsk/jvmti/unit/ForceEarlyReturn/earlyretbase crashes with assertion "compilation level out of bounds"". The test will have to be removed from test/hotspot/jtreg/ProblemList-graal.txt only after both these issues are fixed. Bug: https://bugs.openjdk.java.net/browse/JDK-8214572 Webrev: http://cr.openjdk.java.net/~dtitov/8214572/webrev.01/ Thanks, Daniil From jcbeyler at google.com Sat Dec 1 03:46:53 2018 From: jcbeyler at google.com (JC Beyler) Date: Fri, 30 Nov 2018 19:46:53 -0800 Subject: RFR 8214572: nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the thread when the top frame executes JVMCI code In-Reply-To: <4E0EFD3E-F725-4469-BA20-7532D78BA479@oracle.com> References: <4E0EFD3E-F725-4469-BA20-7532D78BA479@oracle.com> Message-ID: Hi Daniil, The webrev looks good but I have a few comments and questions (if you don't mind :-)): Comments: - You say that normally the test will be removed from the problem list once the two fixes are done but in this webrev, you've already removed it (I can't see the other case so I can't see if it is resolved :-)) - now that we are in C++ for the tests, could we not declare the variables at their first use instead of doing the pedantic top of the block C style? - I feel that this sort of "wait until we are not in JVMCI frames" might happen a lot, maybe we could move that code into a helper method (+ it cleans the method a bit to just concentrate on the rest) and then if needed we can make it public to other tests? Questions because I'm not familiar with JVMCI consequences so not really comments on the webrev but so that I know: - Is it normaly that you can suspend when you are in a JVMCI frame? will/is there not a better way that we could detect that we are in a JVMCI frame? Is it always safe to suspend a JVMCI frame? Thanks! Jc On Fri, Nov 30, 2018 at 7:11 PM Daniil Titov wrote: > Please review the change for nsk/jvmti/unit/ForceEarlyReturn/earlyretbase > test. The problem here is that before doing an early force return the test > doesn't check that the test thread is suspended at a right place where the > top frame executes the test method rather than JVMCI code triggered by > invocation counter overflow. That results in the early return happens for a > wrong method and the test fails. > > The fix changes the test to do resume/suspend in the loop until the target > method is executed in the top frame or the loop counter exceeds the limit. > > There is another problem with this test that requires changes on VM side > and is currently covered by JDK-8195635:" [Graal] > nsk/jvmti/unit/ForceEarlyReturn/earlyretbase crashes with assertion > "compilation level out of bounds"". The test will have to be removed from > test/hotspot/jtreg/ProblemList-graal.txt only after both these issues are > fixed. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214572 > Webrev: http://cr.openjdk.java.net/~dtitov/8214572/webrev.01/ > > Thanks, > Daniil > > > > > > > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From gary.adams at oracle.com Sat Dec 1 11:10:39 2018 From: gary.adams at oracle.com (gary.adams at oracle.com) Date: Sat, 1 Dec 2018 06:10:39 -0500 Subject: RFR: JDK-821430: .attach_pid files may remain in the process cwd In-Reply-To: References: <5C0012B9.2060603@oracle.com> <5C0039E1.3010700@oracle.com> <95b85891-5acb-6a16-b8a8-4bfdaa142516@oracle.com> Message-ID: <84625284-712a-52a0-9024-2ff28c7ec4e2@oracle.com> Updated webrev: ? http://cr.openjdk.java.net/~gadams/8214300/webrev.01/ On 11/30/18 3:10 PM, Chris Plummer wrote: > ...and I've been waiting for the webrev update to give it a thumbs up. > > Chris > > On 11/30/18 1:11 AM, gary.adams at oracle.com wrote: >> I have not refreshed the webrev, yet. >> Waiting to see if there are any additional comments >> before updating the webrev. >> >> The spaces and the assignments you noticed >> have been fixed. >> >> On 11/29/18 5:27 PM, JC Beyler wrote: >>> Hi Gary, >>> >>> Somehow I still see the same webrev? Has it been updated and my >>> browser does not want to update it? >>> >>> Thanks! >>> Jc >>> >>> On Thu, Nov 29, 2018 at 11:10 AM Gary Adams >> > wrote: >>> >>> Sorry, coding before coffee this morning. >>> >>> Fixed. >>> >>> On 11/29/18, 11:54 AM, JC Beyler wrote: >>>> Hi Gary, >>>> >>>> Is there a reason why only >>>> http://cr.openjdk.java.net/~gadams/8214300/webrev.00/src/jdk.attach/linux/classes/sun/tools/attach/VirtualMachineImpl.java.udiff.html >>>> >>>> actually has the assignment after the call ? >>>> f = f.getCanonicalFile()? >>>> >>>> (Nit, there seems to be a space missing before the =). >>>> >>>> In the other two files, you just call the method but don't >>>> update the variable f making the call useless, no? (The patch >>>> in the bug has the assigments for all the cases), >>>> Jc >>>> >>>> >>>> On Thu, Nov 29, 2018 at 8:23 AM Gary Adams >>>> > wrote: >>>> >>>> If a process exits during an attempt to attach to it, the >>>> .attach_pid file >>>> will not be removed properly, if the path used included >>>> symbolic link >>>> traversal, >>>> which is typically done for "/proc//cwd/". Using >>>> getCanonicalFile() >>>> before the >>>> initial file is created should prevent this edge case for >>>> left over files. >>>> >>>> ? ?Webrev: >>>> http://cr.openjdk.java.net/~gadams/8214300/webrev.00/index.html >>>> >>>> ? ?Issue: https://bugs.openjdk.java.net/browse/JDK-8214300 >>>> >>>> Testing is in progress for every attach test I can find, >>>> but there is no >>>> explicit test available for the case where the attachee >>>> exits while the >>>> attacher >>>> is making the initial connection. >>>> >>>> >>>> >>>> -- >>>> >>>> Thanks, >>>> Jc >>> >>> >>> >>> -- >>> >>> Thanks, >>> Jc >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladimir.kozlov at oracle.com Sun Dec 2 00:15:21 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Sat, 1 Dec 2018 16:15:21 -0800 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> Message-ID: <5dfadb93-a8a8-54bc-3d6d-1cdd795fe252@oracle.com> Much better. Why you need to include shenandoahBarrierSetC2.hpp in /opto/classes.cpp ? Why not include only shenandoahSupport.hpp when new nodes are defined? I think it is fine to not use #ifdef in loopopts.cpp when you check is_ShenandoahBarrier(). And you don't do that in other files. Code in opto/macro.cpp is ugly but it is only the place so we can live with it I think. Thanks, Vladimir On 11/30/18 1:00 PM, Roman Kennke wrote: > Hi all, > > here comes round 4 of Shenandoah upstreaming review: > > This includes fixes for the issues that Per brought up: > - Verify and gracefully reject dangerous flags combinations that > disables required barriers > - Revisited @requires filters in tests > - Trim unused code from Shenandoah's SA impl > - Move ShenandoahGCTracer to gc/shenandoah > - Fix ordering of GC names in various files > - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W > > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ > > Thanks everybody for taking time to review this! > Roman > >> Hello all, >> >> Thanks so far for all the reviews and support! >> >> I forgot to update the 'round' yesterday. We are in round 3 now :-) >> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >> >> Things we've changed today: >> - We moved shenandoah-specific code out of .ad files into our own .ad >> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >> requires an addition in build machinery though, see >> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >> - Improved zero-disabling and build-code-simplification as suggested by >> Magnus and Per >> - Cleaned up some leftovers in C2 >> - Improved C2 loop opts code by introducing another APIs in >> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now. >> - We would all very much prefer to keep ShenandoahXYZNode names, as >> noted earlier. This stuff is Shenandoah-specific, so let's just call it >> that. >> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >> - Rebased on jdk-12+22 >> >> - Question: let us know if you need separate RFE for the new BSC2 APIs? >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >> >> Thanks, >> Roman >> >>> Alright, we fixed: >>> - The minor issues that Kim reported in shared-gc >>> - A lot of fixes in shared-tests according to Leonid's review >>> - Disabled SA heapdumping similar to ZGC as Per suggested >>> >>> Some notes: >>> Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>> correct. The @requires there means to exclude runs with both CMS and >>> ExplicitGCInvokesConcurrent at the same time, because that would be >>> (expectedly) failing. It can run CMS, default GC and any other GC just >>> fine. Adding the same clause for Shenandoah means the same, and filters >>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>> made the condition a bit clearer by avoiding triple-negation. >>> >>> See: >>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>> >>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>> >>> we filter them for Shenandoah now. I'm wondering: how do you get past >>> those with ZGC? >>> >>> See: >>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>> >>> (Note to Leonid and tests reviewers: I'll add those related filters in >>> next round). >>> >>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>> better. I can tell that we're not done with C2 yet. Can you look over >>> the code and see what is ok, and especially what is not ok, so that we >>> can focus our efforts on the relevant parts? >>> >>> Updated set of webrevs: >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>> >>> Thanks, >>> Roman >>> >>> >>>> Hi, >>>> >>>> This is the first round of changes for including Shenandoah GC into >>>> mainline. >>>> I divided the review into parts that roughly correspond to the mailing lists >>>> that would normally review it, and I divided it into 'shared' code >>>> changes and >>>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>>> eventually >>>> push them as single 'combined' changeset, once reviewed. >>>> >>>> JEP: >>>> ? https://openjdk.java.net/jeps/189 >>>> Bug entry: >>>> >>>> ?https://bugs.openjdk.java.net/browse/JDK-8214259 >>>> >>>> Webrevs: >>>> ? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>> >>>> For those who want to see the full change, have a look at the >>>> shenandoah-complete >>>> >>>> directory, >>>> it contains the full combined webrev. Alternatively, there is the file >>>> shenandoah-master.patch >>>> , >>>> which is what I intend to commit (and which should be equivalent to the >>>> 'shenandoah-complete' webrev). >>>> >>>> Sections to review (at this point) are the following: >>>> ?*) shenandoah-gc >>>> >>>> ??? - Actual Shenandoah implementation, almost completely residing in >>>> gc/shenandoah >>>> >>>> ?*) shared-gc >>>> >>>> ??? - This is mostly boilerplate that is common to any GC >>>> ??? - referenceProcessor.cpp has a little change to make one assert not >>>> fail (next to CMS and G1) >>>> ??? - taskqueue.hpp has some small adjustments to enable subclassing >>>> >>>> ?*) shared-serviceability >>>> >>>> ??? - The usual code to support another GC >>>> >>>> ?*) shared-runtime >>>> >>>> ??? - A number of friends declarations to allow Shenandoah iterators to >>>> hook up with, >>>> ????? e.g. ClassLoaderData, CodeCache, etc >>>> ??? - Warning and disabling JFR LeakProfiler >>>> ??? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>> Shenandoah C2 optimizations >>>> ??? - Locks initialization in mutexLocker.cpp as usual >>>> ??? - VM operations defines for Shenandoah's VM ops >>>> ??? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>> Shenandoah's logging >>>> ??? - The usual macros in macro.hpp >>>> >>>> ?*) shared-build >>>> >>>> ??? - Add shenandoah feature, enabled by default, as agreed with >>>> Vladimir K. beforehand >>>> ??? - Some flags for shenandoah-enabled compilation to get >>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>> ????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>> Shenandoah's barriers >>>> ??? - --param inline-unit-growth=1000 settings for 2 shenandoah source >>>> files, which is >>>> ????? useful to get the whole marking loop inlined (observed significant >>>> regression if we >>>> ????? don't) >>>> >>>> ?*) shared-tests >>>> >>>> ??? - Test infrastructure to support Shenandoah >>>> ??? - Shenandoah test groups >>>> ??? - Exclude Shenandoah in various tests that can be run with selected GC >>>> ??? - Enable/add configure for Shenandoah for tests that make sense to >>>> run with it >>>> >>>> ?*) shenandoah-tests >>>> >>>> ??? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >>>> ??? - A couple of tests configurations have been added, e.g. >>>> TestGCBasherWithShenandoah.java >>>> >>>> I intentionally left out shared-compiler for now, because we have some >>>> work left to do >>>> there, but if you click around you'll find the patch anyway, in case you >>>> want to take >>>> a peek at it. >>>> >>>> We have regular builds on: >>>> ? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>> ? - {Windows} x {x86_64}, >>>> ? - {MacOS X} x {x86_64} >>>> >>>> This also routinely passes: >>>> ? - the new Shenandoah tests >>>> ? - jcstress with/without aggressive Shenandoah verification >>>> ? - specjvm2008 with/without aggressive Shenandoah verification >>>> >>>> >>>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>>> the credit for being the original inventor of Shenandoah, Aleksey >>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>> teams for tirelessly helping with and reviewing all the GC interface and >>>> related changes, and of course the many early adopters for reporting >>>> bugs and success stories and feature requests: we wouldn't be here >>>> without any of you! >>>> >>>> Best regards, >>>> Roman >>>> >>> >> > From rkennke at redhat.com Sun Dec 2 09:57:35 2018 From: rkennke at redhat.com (Roman Kennke) Date: Sun, 2 Dec 2018 10:57:35 +0100 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <5dfadb93-a8a8-54bc-3d6d-1cdd795fe252@oracle.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <5dfadb93-a8a8-54bc-3d6d-1cdd795fe252@oracle.com> Message-ID: <5aecf696-a125-597c-c02b-5d4da8d453a0@redhat.com> Hi Vladimir, > Much better. Thanks! > Why you need to include shenandoahBarrierSetC2.hpp in /opto/classes.cpp > ? Why not include only shenandoahSupport.hpp when new nodes are defined? For no particular reason, I guess. I'll fix it in next round. > I think it is fine to not use #ifdef in loopopts.cpp when you check > is_ShenandoahBarrier(). And you don't do that in other files. > > Code in opto/macro.cpp is ugly but it is only the place so we can live > with it I think. Ok, thanks! We'll see if we can improve macro.cpp. Thanks, Roman > Thanks, > Vladimir > > On 11/30/18 1:00 PM, Roman Kennke wrote: >> Hi all, >> >> here comes round 4 of Shenandoah upstreaming review: >> >> This includes fixes for the issues that Per brought up: >> - Verify and gracefully reject dangerous flags combinations that >> disables required barriers >> - Revisited @requires filters in tests >> - Trim unused code from Shenandoah's SA impl >> - Move ShenandoahGCTracer to gc/shenandoah >> - Fix ordering of GC names in various files >> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >> >> Thanks everybody for taking time to review this! >> Roman >> >>> Hello all, >>> >>> Thanks so far for all the reviews and support! >>> >>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >>> >>> Things we've changed today: >>> - We moved shenandoah-specific code out of .ad files into our own .ad >>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>> requires an addition in build machinery though, see >>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>> - Improved zero-disabling and build-code-simplification as suggested by >>> Magnus and Per >>> - Cleaned up some leftovers in C2 >>> - Improved C2 loop opts code by introducing another APIs in >>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards >>> now. >>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>> that. >>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>> - Rebased on jdk-12+22 >>> >>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>> >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>> >>> Thanks, >>> Roman >>> >>>> Alright, we fixed: >>>> - The minor issues that Kim reported in shared-gc >>>> - A lot of fixes in shared-tests according to Leonid's review >>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>> >>>> Some notes: >>>> Leonid:? test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>> correct. The @requires there means to exclude runs with both CMS and >>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>> fine. Adding the same clause for Shenandoah means the same, and filters >>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>> made the condition a bit clearer by avoiding triple-negation. >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>> >>>> >>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>> >>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>> those with ZGC? >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>> >>>> >>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>> next round). >>>> >>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>> better. I can tell that we're not done with C2 yet. Can you look over >>>> the code and see what is ok, and especially what is not ok, so that we >>>> can focus our efforts on the relevant parts? >>>> >>>> Updated set of webrevs: >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>> >>>> Thanks, >>>> Roman >>>> >>>> >>>>> Hi, >>>>> >>>>> This is the first round of changes for including Shenandoah GC into >>>>> mainline. >>>>> I divided the review into parts that roughly correspond to the >>>>> mailing lists >>>>> that would normally review it, and I divided it into 'shared' code >>>>> changes and >>>>> 'shenandoah' code changes (actually, mostly additions). The intend >>>>> is to >>>>> eventually >>>>> push them as single 'combined' changeset, once reviewed. >>>>> >>>>> JEP: >>>>> ?? https://openjdk.java.net/jeps/189 >>>>> Bug entry: >>>>> >>>>> ??https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>> >>>>> Webrevs: >>>>> ?? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>> >>>>> For those who want to see the full change, have a look at the >>>>> shenandoah-complete >>>>> >>>>> >>>>> directory, >>>>> it contains the full combined webrev. Alternatively, there is the file >>>>> shenandoah-master.patch >>>>> , >>>>> >>>>> which is what I intend to commit (and which should be equivalent to >>>>> the >>>>> 'shenandoah-complete' webrev). >>>>> >>>>> Sections to review (at this point) are the following: >>>>> ??*) shenandoah-gc >>>>> >>>>> >>>>> ???? - Actual Shenandoah implementation, almost completely residing in >>>>> gc/shenandoah >>>>> >>>>> ??*) shared-gc >>>>> >>>>> >>>>> ???? - This is mostly boilerplate that is common to any GC >>>>> ???? - referenceProcessor.cpp has a little change to make one >>>>> assert not >>>>> fail (next to CMS and G1) >>>>> ???? - taskqueue.hpp has some small adjustments to enable subclassing >>>>> >>>>> ??*) shared-serviceability >>>>> >>>>> >>>>> ???? - The usual code to support another GC >>>>> >>>>> ??*) shared-runtime >>>>> >>>>> >>>>> ???? - A number of friends declarations to allow Shenandoah >>>>> iterators to >>>>> hook up with, >>>>> ?????? e.g. ClassLoaderData, CodeCache, etc >>>>> ???? - Warning and disabling JFR LeakProfiler >>>>> ???? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>> Shenandoah C2 optimizations >>>>> ???? - Locks initialization in mutexLocker.cpp as usual >>>>> ???? - VM operations defines for Shenandoah's VM ops >>>>> ???? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>> Shenandoah's logging >>>>> ???? - The usual macros in macro.hpp >>>>> >>>>> ??*) shared-build >>>>> >>>>> >>>>> ???? - Add shenandoah feature, enabled by default, as agreed with >>>>> Vladimir K. beforehand >>>>> ???? - Some flags for shenandoah-enabled compilation to get >>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>> ?????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>> Shenandoah's barriers >>>>> ???? - --param inline-unit-growth=1000 settings for 2 shenandoah >>>>> source >>>>> files, which is >>>>> ?????? useful to get the whole marking loop inlined (observed >>>>> significant >>>>> regression if we >>>>> ?????? don't) >>>>> >>>>> ??*) shared-tests >>>>> >>>>> >>>>> ???? - Test infrastructure to support Shenandoah >>>>> ???? - Shenandoah test groups >>>>> ???? - Exclude Shenandoah in various tests that can be run with >>>>> selected GC >>>>> ???? - Enable/add configure for Shenandoah for tests that make >>>>> sense to >>>>> run with it >>>>> >>>>> ??*) shenandoah-tests >>>>> >>>>> >>>>> ???? - Shenandoah specific tests, most reside in gc/shenandoah >>>>> subdirectory >>>>> ???? - A couple of tests configurations have been added, e.g. >>>>> TestGCBasherWithShenandoah.java >>>>> >>>>> I intentionally left out shared-compiler for now, because we have some >>>>> work left to do >>>>> there, but if you click around you'll find the patch anyway, in >>>>> case you >>>>> want to take >>>>> a peek at it. >>>>> >>>>> We have regular builds on: >>>>> ?? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>> ?? - {Windows} x {x86_64}, >>>>> ?? - {MacOS X} x {x86_64} >>>>> >>>>> This also routinely passes: >>>>> ?? - the new Shenandoah tests >>>>> ?? - jcstress with/without aggressive Shenandoah verification >>>>> ?? - specjvm2008 with/without aggressive Shenandoah verification >>>>> >>>>> >>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she >>>>> deserves >>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>> teams for tirelessly helping with and reviewing all the GC >>>>> interface and >>>>> related changes, and of course the many early adopters for reporting >>>>> bugs and success stories and feature requests: we wouldn't be here >>>>> without any of you! >>>>> >>>>> Best regards, >>>>> Roman >>>>> >>>> >>> >> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Sun Dec 2 16:55:38 2018 From: rkennke at redhat.com (Roman Kennke) Date: Sun, 2 Dec 2018 17:55:38 +0100 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <5dfadb93-a8a8-54bc-3d6d-1cdd795fe252@oracle.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <5dfadb93-a8a8-54bc-3d6d-1cdd795fe252@oracle.com> Message-ID: <99cf5fe7-1fb1-66cf-8e9c-d8879d003b2d@redhat.com> Hello Vladimir, > Why you need to include shenandoahBarrierSetC2.hpp in /opto/classes.cpp > ? Why not include only shenandoahSupport.hpp when new nodes are defined? I discussed this with my team. shenandoahBarrierSetC2.hpp is supposed to be the entry point for external C2 code. No C2 code is supposed to include shenandoahSupport.hpp, this is just an implementation detail of shenandoahBarrierSetC2.hpp. This seems symmetrical to how ZGC includes zBarrierSetC2.hpp from external C2 code. Is that ok to leave as it is? Cheers, Roman > > I think it is fine to not use #ifdef in loopopts.cpp when you check > is_ShenandoahBarrier(). And you don't do that in other files. > > Code in opto/macro.cpp is ugly but it is only the place so we can live > with it I think. > > Thanks, > Vladimir > > On 11/30/18 1:00 PM, Roman Kennke wrote: >> Hi all, >> >> here comes round 4 of Shenandoah upstreaming review: >> >> This includes fixes for the issues that Per brought up: >> - Verify and gracefully reject dangerous flags combinations that >> disables required barriers >> - Revisited @requires filters in tests >> - Trim unused code from Shenandoah's SA impl >> - Move ShenandoahGCTracer to gc/shenandoah >> - Fix ordering of GC names in various files >> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >> >> Thanks everybody for taking time to review this! >> Roman >> >>> Hello all, >>> >>> Thanks so far for all the reviews and support! >>> >>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >>> >>> Things we've changed today: >>> - We moved shenandoah-specific code out of .ad files into our own .ad >>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>> requires an addition in build machinery though, see >>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>> - Improved zero-disabling and build-code-simplification as suggested by >>> Magnus and Per >>> - Cleaned up some leftovers in C2 >>> - Improved C2 loop opts code by introducing another APIs in >>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards >>> now. >>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>> that. >>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>> - Rebased on jdk-12+22 >>> >>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>> >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>> >>> Thanks, >>> Roman >>> >>>> Alright, we fixed: >>>> - The minor issues that Kim reported in shared-gc >>>> - A lot of fixes in shared-tests according to Leonid's review >>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>> >>>> Some notes: >>>> Leonid:? test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>> correct. The @requires there means to exclude runs with both CMS and >>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>> fine. Adding the same clause for Shenandoah means the same, and filters >>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>> made the condition a bit clearer by avoiding triple-negation. >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>> >>>> >>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>> >>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>> those with ZGC? >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>> >>>> >>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>> next round). >>>> >>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>> better. I can tell that we're not done with C2 yet. Can you look over >>>> the code and see what is ok, and especially what is not ok, so that we >>>> can focus our efforts on the relevant parts? >>>> >>>> Updated set of webrevs: >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>> >>>> Thanks, >>>> Roman >>>> >>>> >>>>> Hi, >>>>> >>>>> This is the first round of changes for including Shenandoah GC into >>>>> mainline. >>>>> I divided the review into parts that roughly correspond to the >>>>> mailing lists >>>>> that would normally review it, and I divided it into 'shared' code >>>>> changes and >>>>> 'shenandoah' code changes (actually, mostly additions). The intend >>>>> is to >>>>> eventually >>>>> push them as single 'combined' changeset, once reviewed. >>>>> >>>>> JEP: >>>>> ?? https://openjdk.java.net/jeps/189 >>>>> Bug entry: >>>>> >>>>> ??https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>> >>>>> Webrevs: >>>>> ?? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>> >>>>> For those who want to see the full change, have a look at the >>>>> shenandoah-complete >>>>> >>>>> >>>>> directory, >>>>> it contains the full combined webrev. Alternatively, there is the file >>>>> shenandoah-master.patch >>>>> , >>>>> >>>>> which is what I intend to commit (and which should be equivalent to >>>>> the >>>>> 'shenandoah-complete' webrev). >>>>> >>>>> Sections to review (at this point) are the following: >>>>> ??*) shenandoah-gc >>>>> >>>>> >>>>> ???? - Actual Shenandoah implementation, almost completely residing in >>>>> gc/shenandoah >>>>> >>>>> ??*) shared-gc >>>>> >>>>> >>>>> ???? - This is mostly boilerplate that is common to any GC >>>>> ???? - referenceProcessor.cpp has a little change to make one >>>>> assert not >>>>> fail (next to CMS and G1) >>>>> ???? - taskqueue.hpp has some small adjustments to enable subclassing >>>>> >>>>> ??*) shared-serviceability >>>>> >>>>> >>>>> ???? - The usual code to support another GC >>>>> >>>>> ??*) shared-runtime >>>>> >>>>> >>>>> ???? - A number of friends declarations to allow Shenandoah >>>>> iterators to >>>>> hook up with, >>>>> ?????? e.g. ClassLoaderData, CodeCache, etc >>>>> ???? - Warning and disabling JFR LeakProfiler >>>>> ???? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>> Shenandoah C2 optimizations >>>>> ???? - Locks initialization in mutexLocker.cpp as usual >>>>> ???? - VM operations defines for Shenandoah's VM ops >>>>> ???? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>> Shenandoah's logging >>>>> ???? - The usual macros in macro.hpp >>>>> >>>>> ??*) shared-build >>>>> >>>>> >>>>> ???? - Add shenandoah feature, enabled by default, as agreed with >>>>> Vladimir K. beforehand >>>>> ???? - Some flags for shenandoah-enabled compilation to get >>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>> ?????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>> Shenandoah's barriers >>>>> ???? - --param inline-unit-growth=1000 settings for 2 shenandoah >>>>> source >>>>> files, which is >>>>> ?????? useful to get the whole marking loop inlined (observed >>>>> significant >>>>> regression if we >>>>> ?????? don't) >>>>> >>>>> ??*) shared-tests >>>>> >>>>> >>>>> ???? - Test infrastructure to support Shenandoah >>>>> ???? - Shenandoah test groups >>>>> ???? - Exclude Shenandoah in various tests that can be run with >>>>> selected GC >>>>> ???? - Enable/add configure for Shenandoah for tests that make >>>>> sense to >>>>> run with it >>>>> >>>>> ??*) shenandoah-tests >>>>> >>>>> >>>>> ???? - Shenandoah specific tests, most reside in gc/shenandoah >>>>> subdirectory >>>>> ???? - A couple of tests configurations have been added, e.g. >>>>> TestGCBasherWithShenandoah.java >>>>> >>>>> I intentionally left out shared-compiler for now, because we have some >>>>> work left to do >>>>> there, but if you click around you'll find the patch anyway, in >>>>> case you >>>>> want to take >>>>> a peek at it. >>>>> >>>>> We have regular builds on: >>>>> ?? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>> ?? - {Windows} x {x86_64}, >>>>> ?? - {MacOS X} x {x86_64} >>>>> >>>>> This also routinely passes: >>>>> ?? - the new Shenandoah tests >>>>> ?? - jcstress with/without aggressive Shenandoah verification >>>>> ?? - specjvm2008 with/without aggressive Shenandoah verification >>>>> >>>>> >>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she >>>>> deserves >>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>> teams for tirelessly helping with and reviewing all the GC >>>>> interface and >>>>> related changes, and of course the many early adopters for reporting >>>>> bugs and success stories and feature requests: we wouldn't be here >>>>> without any of you! >>>>> >>>>> Best regards, >>>>> Roman >>>>> >>>> >>> >> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From vladimir.kozlov at oracle.com Sun Dec 2 18:10:16 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Sun, 2 Dec 2018 10:10:16 -0800 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <99cf5fe7-1fb1-66cf-8e9c-d8879d003b2d@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <5dfadb93-a8a8-54bc-3d6d-1cdd795fe252@oracle.com> <99cf5fe7-1fb1-66cf-8e9c-d8879d003b2d@redhat.com> Message-ID: On 12/2/18 8:55 AM, Roman Kennke wrote: > Hello Vladimir, > >> Why you need to include shenandoahBarrierSetC2.hpp in /opto/classes.cpp >> ? Why not include only shenandoahSupport.hpp when new nodes are defined? > > I discussed this with my team. shenandoahBarrierSetC2.hpp is supposed to > be the entry point for external C2 code. No C2 code is supposed to > include shenandoahSupport.hpp, this is just an implementation detail of > shenandoahBarrierSetC2.hpp. This seems symmetrical to how ZGC includes > zBarrierSetC2.hpp from external C2 code. Is that ok to leave as it is? Yes, it is fine. Vladimir > > Cheers, > Roman > > >> >> I think it is fine to not use #ifdef in loopopts.cpp when you check >> is_ShenandoahBarrier(). And you don't do that in other files. >> >> Code in opto/macro.cpp is ugly but it is only the place so we can live >> with it I think. >> >> Thanks, >> Vladimir >> >> On 11/30/18 1:00 PM, Roman Kennke wrote: >>> Hi all, >>> >>> here comes round 4 of Shenandoah upstreaming review: >>> >>> This includes fixes for the issues that Per brought up: >>> - Verify and gracefully reject dangerous flags combinations that >>> disables required barriers >>> - Revisited @requires filters in tests >>> - Trim unused code from Shenandoah's SA impl >>> - Move ShenandoahGCTracer to gc/shenandoah >>> - Fix ordering of GC names in various files >>> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >>> >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >>> >>> Thanks everybody for taking time to review this! >>> Roman >>> >>>> Hello all, >>>> >>>> Thanks so far for all the reviews and support! >>>> >>>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>>> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >>>> >>>> Things we've changed today: >>>> - We moved shenandoah-specific code out of .ad files into our own .ad >>>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>>> requires an addition in build machinery though, see >>>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>>> - Improved zero-disabling and build-code-simplification as suggested by >>>> Magnus and Per >>>> - Cleaned up some leftovers in C2 >>>> - Improved C2 loop opts code by introducing another APIs in >>>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards >>>> now. >>>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>>> that. >>>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>>> - Rebased on jdk-12+22 >>>> >>>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>>> >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>>> >>>> Thanks, >>>> Roman >>>> >>>>> Alright, we fixed: >>>>> - The minor issues that Kim reported in shared-gc >>>>> - A lot of fixes in shared-tests according to Leonid's review >>>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>>> >>>>> Some notes: >>>>> Leonid:? test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>>> correct. The @requires there means to exclude runs with both CMS and >>>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>>> fine. Adding the same clause for Shenandoah means the same, and filters >>>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>>> made the condition a bit clearer by avoiding triple-negation. >>>>> >>>>> See: >>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>>> >>>>> >>>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>>> >>>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>>> those with ZGC? >>>>> >>>>> See: >>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>>> >>>>> >>>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>>> next round). >>>>> >>>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>>> better. I can tell that we're not done with C2 yet. Can you look over >>>>> the code and see what is ok, and especially what is not ok, so that we >>>>> can focus our efforts on the relevant parts? >>>>> >>>>> Updated set of webrevs: >>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>>> >>>>> Thanks, >>>>> Roman >>>>> >>>>> >>>>>> Hi, >>>>>> >>>>>> This is the first round of changes for including Shenandoah GC into >>>>>> mainline. >>>>>> I divided the review into parts that roughly correspond to the >>>>>> mailing lists >>>>>> that would normally review it, and I divided it into 'shared' code >>>>>> changes and >>>>>> 'shenandoah' code changes (actually, mostly additions). The intend >>>>>> is to >>>>>> eventually >>>>>> push them as single 'combined' changeset, once reviewed. >>>>>> >>>>>> JEP: >>>>>> ?? https://openjdk.java.net/jeps/189 >>>>>> Bug entry: >>>>>> >>>>>> ??https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>>> >>>>>> Webrevs: >>>>>> ?? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>>> >>>>>> For those who want to see the full change, have a look at the >>>>>> shenandoah-complete >>>>>> >>>>>> >>>>>> directory, >>>>>> it contains the full combined webrev. Alternatively, there is the file >>>>>> shenandoah-master.patch >>>>>> , >>>>>> >>>>>> which is what I intend to commit (and which should be equivalent to >>>>>> the >>>>>> 'shenandoah-complete' webrev). >>>>>> >>>>>> Sections to review (at this point) are the following: >>>>>> ??*) shenandoah-gc >>>>>> >>>>>> >>>>>> ???? - Actual Shenandoah implementation, almost completely residing in >>>>>> gc/shenandoah >>>>>> >>>>>> ??*) shared-gc >>>>>> >>>>>> >>>>>> ???? - This is mostly boilerplate that is common to any GC >>>>>> ???? - referenceProcessor.cpp has a little change to make one >>>>>> assert not >>>>>> fail (next to CMS and G1) >>>>>> ???? - taskqueue.hpp has some small adjustments to enable subclassing >>>>>> >>>>>> ??*) shared-serviceability >>>>>> >>>>>> >>>>>> ???? - The usual code to support another GC >>>>>> >>>>>> ??*) shared-runtime >>>>>> >>>>>> >>>>>> ???? - A number of friends declarations to allow Shenandoah >>>>>> iterators to >>>>>> hook up with, >>>>>> ?????? e.g. ClassLoaderData, CodeCache, etc >>>>>> ???? - Warning and disabling JFR LeakProfiler >>>>>> ???? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>>> Shenandoah C2 optimizations >>>>>> ???? - Locks initialization in mutexLocker.cpp as usual >>>>>> ???? - VM operations defines for Shenandoah's VM ops >>>>>> ???? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>>> Shenandoah's logging >>>>>> ???? - The usual macros in macro.hpp >>>>>> >>>>>> ??*) shared-build >>>>>> >>>>>> >>>>>> ???? - Add shenandoah feature, enabled by default, as agreed with >>>>>> Vladimir K. beforehand >>>>>> ???? - Some flags for shenandoah-enabled compilation to get >>>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>>> ?????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>>> Shenandoah's barriers >>>>>> ???? - --param inline-unit-growth=1000 settings for 2 shenandoah >>>>>> source >>>>>> files, which is >>>>>> ?????? useful to get the whole marking loop inlined (observed >>>>>> significant >>>>>> regression if we >>>>>> ?????? don't) >>>>>> >>>>>> ??*) shared-tests >>>>>> >>>>>> >>>>>> ???? - Test infrastructure to support Shenandoah >>>>>> ???? - Shenandoah test groups >>>>>> ???? - Exclude Shenandoah in various tests that can be run with >>>>>> selected GC >>>>>> ???? - Enable/add configure for Shenandoah for tests that make >>>>>> sense to >>>>>> run with it >>>>>> >>>>>> ??*) shenandoah-tests >>>>>> >>>>>> >>>>>> ???? - Shenandoah specific tests, most reside in gc/shenandoah >>>>>> subdirectory >>>>>> ???? - A couple of tests configurations have been added, e.g. >>>>>> TestGCBasherWithShenandoah.java >>>>>> >>>>>> I intentionally left out shared-compiler for now, because we have some >>>>>> work left to do >>>>>> there, but if you click around you'll find the patch anyway, in >>>>>> case you >>>>>> want to take >>>>>> a peek at it. >>>>>> >>>>>> We have regular builds on: >>>>>> ?? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>>> ?? - {Windows} x {x86_64}, >>>>>> ?? - {MacOS X} x {x86_64} >>>>>> >>>>>> This also routinely passes: >>>>>> ?? - the new Shenandoah tests >>>>>> ?? - jcstress with/without aggressive Shenandoah verification >>>>>> ?? - specjvm2008 with/without aggressive Shenandoah verification >>>>>> >>>>>> >>>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she >>>>>> deserves >>>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>>> teams for tirelessly helping with and reviewing all the GC >>>>>> interface and >>>>>> related changes, and of course the many early adopters for reporting >>>>>> bugs and success stories and feature requests: we wouldn't be here >>>>>> without any of you! >>>>>> >>>>>> Best regards, >>>>>> Roman >>>>>> >>>>> >>>> >>> > From rkennke at redhat.com Sun Dec 2 18:15:15 2018 From: rkennke at redhat.com (Roman Kennke) Date: Sun, 2 Dec 2018 19:15:15 +0100 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <5dfadb93-a8a8-54bc-3d6d-1cdd795fe252@oracle.com> <99cf5fe7-1fb1-66cf-8e9c-d8879d003b2d@redhat.com> Message-ID: <75392bec-8157-2fb2-51d0-f50fd51bbc6b@redhat.com> >> Hello Vladimir, >> >>> Why you need to include shenandoahBarrierSetC2.hpp in /opto/classes.cpp >>> ? Why not include only shenandoahSupport.hpp when new nodes are defined? >> >> I discussed this with my team. shenandoahBarrierSetC2.hpp is supposed to >> be the entry point for external C2 code. No C2 code is supposed to >> include shenandoahSupport.hpp, this is just an implementation detail of >> shenandoahBarrierSetC2.hpp. This seems symmetrical to how ZGC includes >> zBarrierSetC2.hpp from external C2 code. Is that ok to leave as it is? > > Yes, it is fine. Ok great! Thanks! Can I consider the shared-compiler part reviewed by you then? Roman > > Vladimir > >> >> Cheers, >> Roman >> >> >>> >>> I think it is fine to not use #ifdef in loopopts.cpp when you check >>> is_ShenandoahBarrier(). And you don't do that in other files. >>> >>> Code in opto/macro.cpp is ugly but it is only the place so we can live >>> with it I think. >>> >>> Thanks, >>> Vladimir >>> >>> On 11/30/18 1:00 PM, Roman Kennke wrote: >>>> Hi all, >>>> >>>> here comes round 4 of Shenandoah upstreaming review: >>>> >>>> This includes fixes for the issues that Per brought up: >>>> - Verify and gracefully reject dangerous flags combinations that >>>> disables required barriers >>>> - Revisited @requires filters in tests >>>> - Trim unused code from Shenandoah's SA impl >>>> - Move ShenandoahGCTracer to gc/shenandoah >>>> - Fix ordering of GC names in various files >>>> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >>>> >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >>>> >>>> Thanks everybody for taking time to review this! >>>> Roman >>>> >>>>> Hello all, >>>>> >>>>> Thanks so far for all the reviews and support! >>>>> >>>>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>>>> Also, I fixed the numbering of my webrevs to match the >>>>> review-round. ;-) >>>>> >>>>> Things we've changed today: >>>>> - We moved shenandoah-specific code out of .ad files into our own .ad >>>>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>>>> requires an addition in build machinery though, see >>>>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>>>> - Improved zero-disabling and build-code-simplification as >>>>> suggested by >>>>> Magnus and Per >>>>> - Cleaned up some leftovers in C2 >>>>> - Improved C2 loop opts code by introducing another APIs in >>>>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>>>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards >>>>> now. >>>>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>>>> noted earlier. This stuff is Shenandoah-specific, so let's just >>>>> call it >>>>> that. >>>>> - Rehashed Shenandoah tests (formatting, naming, directory layout, >>>>> etc) >>>>> - Rebased on jdk-12+22 >>>>> >>>>> - Question: let us know if you need separate RFE for the new BSC2 >>>>> APIs? >>>>> >>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>>>> >>>>> Thanks, >>>>> Roman >>>>> >>>>>> Alright, we fixed: >>>>>> - The minor issues that Kim reported in shared-gc >>>>>> - A lot of fixes in shared-tests according to Leonid's review >>>>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>>>> >>>>>> Some notes: >>>>>> Leonid:? test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>>>> correct. The @requires there means to exclude runs with both CMS and >>>>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>>>> (expectedly) failing. It can run CMS, default GC and any other GC >>>>>> just >>>>>> fine. Adding the same clause for Shenandoah means the same, and >>>>>> filters >>>>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>>>> made the condition a bit clearer by avoiding triple-negation. >>>>>> >>>>>> See: >>>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>>>> >>>>>> >>>>>> >>>>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>>>> >>>>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>>>> those with ZGC? >>>>>> >>>>>> See: >>>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>>>> >>>>>> >>>>>> >>>>>> (Note to Leonid and tests reviewers: I'll add those related >>>>>> filters in >>>>>> next round). >>>>>> >>>>>> Vladimir: Roland integrated a bunch of changes to make loop* code >>>>>> look >>>>>> better. I can tell that we're not done with C2 yet. Can you look over >>>>>> the code and see what is ok, and especially what is not ok, so >>>>>> that we >>>>>> can focus our efforts on the relevant parts? >>>>>> >>>>>> Updated set of webrevs: >>>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>>>> >>>>>> Thanks, >>>>>> Roman >>>>>> >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> This is the first round of changes for including Shenandoah GC into >>>>>>> mainline. >>>>>>> I divided the review into parts that roughly correspond to the >>>>>>> mailing lists >>>>>>> that would normally review it, and I divided it into 'shared' code >>>>>>> changes and >>>>>>> 'shenandoah' code changes (actually, mostly additions). The intend >>>>>>> is to >>>>>>> eventually >>>>>>> push them as single 'combined' changeset, once reviewed. >>>>>>> >>>>>>> JEP: >>>>>>> ??? https://openjdk.java.net/jeps/189 >>>>>>> Bug entry: >>>>>>> >>>>>>> ???https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>>>> >>>>>>> Webrevs: >>>>>>> ??? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>>>> >>>>>>> For those who want to see the full change, have a look at the >>>>>>> shenandoah-complete >>>>>>> >>>>>>> >>>>>>> >>>>>>> directory, >>>>>>> it contains the full combined webrev. Alternatively, there is the >>>>>>> file >>>>>>> shenandoah-master.patch >>>>>>> , >>>>>>> >>>>>>> >>>>>>> which is what I intend to commit (and which should be equivalent to >>>>>>> the >>>>>>> 'shenandoah-complete' webrev). >>>>>>> >>>>>>> Sections to review (at this point) are the following: >>>>>>> ???*) shenandoah-gc >>>>>>> >>>>>>> >>>>>>> >>>>>>> ????? - Actual Shenandoah implementation, almost completely >>>>>>> residing in >>>>>>> gc/shenandoah >>>>>>> >>>>>>> ???*) shared-gc >>>>>>> >>>>>>> >>>>>>> >>>>>>> ????? - This is mostly boilerplate that is common to any GC >>>>>>> ????? - referenceProcessor.cpp has a little change to make one >>>>>>> assert not >>>>>>> fail (next to CMS and G1) >>>>>>> ????? - taskqueue.hpp has some small adjustments to enable >>>>>>> subclassing >>>>>>> >>>>>>> ???*) shared-serviceability >>>>>>> >>>>>>> >>>>>>> >>>>>>> ????? - The usual code to support another GC >>>>>>> >>>>>>> ???*) shared-runtime >>>>>>> >>>>>>> >>>>>>> >>>>>>> ????? - A number of friends declarations to allow Shenandoah >>>>>>> iterators to >>>>>>> hook up with, >>>>>>> ??????? e.g. ClassLoaderData, CodeCache, etc >>>>>>> ????? - Warning and disabling JFR LeakProfiler >>>>>>> ????? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>>>> Shenandoah C2 optimizations >>>>>>> ????? - Locks initialization in mutexLocker.cpp as usual >>>>>>> ????? - VM operations defines for Shenandoah's VM ops >>>>>>> ????? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>>>> Shenandoah's logging >>>>>>> ????? - The usual macros in macro.hpp >>>>>>> >>>>>>> ???*) shared-build >>>>>>> >>>>>>> >>>>>>> >>>>>>> ????? - Add shenandoah feature, enabled by default, as agreed with >>>>>>> Vladimir K. beforehand >>>>>>> ????? - Some flags for shenandoah-enabled compilation to get >>>>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>>>> ??????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>>>> Shenandoah's barriers >>>>>>> ????? - --param inline-unit-growth=1000 settings for 2 shenandoah >>>>>>> source >>>>>>> files, which is >>>>>>> ??????? useful to get the whole marking loop inlined (observed >>>>>>> significant >>>>>>> regression if we >>>>>>> ??????? don't) >>>>>>> >>>>>>> ???*) shared-tests >>>>>>> >>>>>>> >>>>>>> >>>>>>> ????? - Test infrastructure to support Shenandoah >>>>>>> ????? - Shenandoah test groups >>>>>>> ????? - Exclude Shenandoah in various tests that can be run with >>>>>>> selected GC >>>>>>> ????? - Enable/add configure for Shenandoah for tests that make >>>>>>> sense to >>>>>>> run with it >>>>>>> >>>>>>> ???*) shenandoah-tests >>>>>>> >>>>>>> >>>>>>> >>>>>>> ????? - Shenandoah specific tests, most reside in gc/shenandoah >>>>>>> subdirectory >>>>>>> ????? - A couple of tests configurations have been added, e.g. >>>>>>> TestGCBasherWithShenandoah.java >>>>>>> >>>>>>> I intentionally left out shared-compiler for now, because we have >>>>>>> some >>>>>>> work left to do >>>>>>> there, but if you click around you'll find the patch anyway, in >>>>>>> case you >>>>>>> want to take >>>>>>> a peek at it. >>>>>>> >>>>>>> We have regular builds on: >>>>>>> ??? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>>>> ??? - {Windows} x {x86_64}, >>>>>>> ??? - {MacOS X} x {x86_64} >>>>>>> >>>>>>> This also routinely passes: >>>>>>> ??? - the new Shenandoah tests >>>>>>> ??? - jcstress with/without aggressive Shenandoah verification >>>>>>> ??? - specjvm2008 with/without aggressive Shenandoah verification >>>>>>> >>>>>>> >>>>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she >>>>>>> deserves >>>>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>>>> advice and support, my collegues in Oracle's GC, runtime and >>>>>>> compiler >>>>>>> teams for tirelessly helping with and reviewing all the GC >>>>>>> interface and >>>>>>> related changes, and of course the many early adopters for reporting >>>>>>> bugs and success stories and feature requests: we wouldn't be here >>>>>>> without any of you! >>>>>>> >>>>>>> Best regards, >>>>>>> Roman >>>>>>> >>>>>> >>>>> >>>> >> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From vladimir.kozlov at oracle.com Sun Dec 2 20:37:31 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Sun, 2 Dec 2018 12:37:31 -0800 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <75392bec-8157-2fb2-51d0-f50fd51bbc6b@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <5dfadb93-a8a8-54bc-3d6d-1cdd795fe252@oracle.com> <99cf5fe7-1fb1-66cf-8e9c-d8879d003b2d@redhat.com> <75392bec-8157-2fb2-51d0-f50fd51bbc6b@redhat.com> Message-ID: Yes, compiler changes reviewed. Vladimir On 12/2/18 10:15 AM, Roman Kennke wrote: >>> Hello Vladimir, >>> >>>> Why you need to include shenandoahBarrierSetC2.hpp in /opto/classes.cpp >>>> ? Why not include only shenandoahSupport.hpp when new nodes are defined? >>> >>> I discussed this with my team. shenandoahBarrierSetC2.hpp is supposed to >>> be the entry point for external C2 code. No C2 code is supposed to >>> include shenandoahSupport.hpp, this is just an implementation detail of >>> shenandoahBarrierSetC2.hpp. This seems symmetrical to how ZGC includes >>> zBarrierSetC2.hpp from external C2 code. Is that ok to leave as it is? >> >> Yes, it is fine. > > Ok great! Thanks! > > Can I consider the shared-compiler part reviewed by you then? > > Roman > > >> >> Vladimir >> >>> >>> Cheers, >>> Roman >>> >>> >>>> >>>> I think it is fine to not use #ifdef in loopopts.cpp when you check >>>> is_ShenandoahBarrier(). And you don't do that in other files. >>>> >>>> Code in opto/macro.cpp is ugly but it is only the place so we can live >>>> with it I think. >>>> >>>> Thanks, >>>> Vladimir >>>> >>>> On 11/30/18 1:00 PM, Roman Kennke wrote: >>>>> Hi all, >>>>> >>>>> here comes round 4 of Shenandoah upstreaming review: >>>>> >>>>> This includes fixes for the issues that Per brought up: >>>>> - Verify and gracefully reject dangerous flags combinations that >>>>> disables required barriers >>>>> - Revisited @requires filters in tests >>>>> - Trim unused code from Shenandoah's SA impl >>>>> - Move ShenandoahGCTracer to gc/shenandoah >>>>> - Fix ordering of GC names in various files >>>>> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >>>>> >>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >>>>> >>>>> Thanks everybody for taking time to review this! >>>>> Roman >>>>> >>>>>> Hello all, >>>>>> >>>>>> Thanks so far for all the reviews and support! >>>>>> >>>>>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>>>>> Also, I fixed the numbering of my webrevs to match the >>>>>> review-round. ;-) >>>>>> >>>>>> Things we've changed today: >>>>>> - We moved shenandoah-specific code out of .ad files into our own .ad >>>>>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>>>>> requires an addition in build machinery though, see >>>>>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>>>>> - Improved zero-disabling and build-code-simplification as >>>>>> suggested by >>>>>> Magnus and Per >>>>>> - Cleaned up some leftovers in C2 >>>>>> - Improved C2 loop opts code by introducing another APIs in >>>>>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>>>>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards >>>>>> now. >>>>>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>>>>> noted earlier. This stuff is Shenandoah-specific, so let's just >>>>>> call it >>>>>> that. >>>>>> - Rehashed Shenandoah tests (formatting, naming, directory layout, >>>>>> etc) >>>>>> - Rebased on jdk-12+22 >>>>>> >>>>>> - Question: let us know if you need separate RFE for the new BSC2 >>>>>> APIs? >>>>>> >>>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>>>>> >>>>>> Thanks, >>>>>> Roman >>>>>> >>>>>>> Alright, we fixed: >>>>>>> - The minor issues that Kim reported in shared-gc >>>>>>> - A lot of fixes in shared-tests according to Leonid's review >>>>>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>>>>> >>>>>>> Some notes: >>>>>>> Leonid:? test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>>>>> correct. The @requires there means to exclude runs with both CMS and >>>>>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>>>>> (expectedly) failing. It can run CMS, default GC and any other GC >>>>>>> just >>>>>>> fine. Adding the same clause for Shenandoah means the same, and >>>>>>> filters >>>>>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>>>>> made the condition a bit clearer by avoiding triple-negation. >>>>>>> >>>>>>> See: >>>>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>>>>> >>>>>>> >>>>>>> >>>>>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>>>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>>>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>>>>> >>>>>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>>>>> those with ZGC? >>>>>>> >>>>>>> See: >>>>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>>>>> >>>>>>> >>>>>>> >>>>>>> (Note to Leonid and tests reviewers: I'll add those related >>>>>>> filters in >>>>>>> next round). >>>>>>> >>>>>>> Vladimir: Roland integrated a bunch of changes to make loop* code >>>>>>> look >>>>>>> better. I can tell that we're not done with C2 yet. Can you look over >>>>>>> the code and see what is ok, and especially what is not ok, so >>>>>>> that we >>>>>>> can focus our efforts on the relevant parts? >>>>>>> >>>>>>> Updated set of webrevs: >>>>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>>>>> >>>>>>> Thanks, >>>>>>> Roman >>>>>>> >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> This is the first round of changes for including Shenandoah GC into >>>>>>>> mainline. >>>>>>>> I divided the review into parts that roughly correspond to the >>>>>>>> mailing lists >>>>>>>> that would normally review it, and I divided it into 'shared' code >>>>>>>> changes and >>>>>>>> 'shenandoah' code changes (actually, mostly additions). The intend >>>>>>>> is to >>>>>>>> eventually >>>>>>>> push them as single 'combined' changeset, once reviewed. >>>>>>>> >>>>>>>> JEP: >>>>>>>> ??? https://openjdk.java.net/jeps/189 >>>>>>>> Bug entry: >>>>>>>> >>>>>>>> ???https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>>>>> >>>>>>>> Webrevs: >>>>>>>> ??? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>>>>> >>>>>>>> For those who want to see the full change, have a look at the >>>>>>>> shenandoah-complete >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> directory, >>>>>>>> it contains the full combined webrev. Alternatively, there is the >>>>>>>> file >>>>>>>> shenandoah-master.patch >>>>>>>> , >>>>>>>> >>>>>>>> >>>>>>>> which is what I intend to commit (and which should be equivalent to >>>>>>>> the >>>>>>>> 'shenandoah-complete' webrev). >>>>>>>> >>>>>>>> Sections to review (at this point) are the following: >>>>>>>> ???*) shenandoah-gc >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ????? - Actual Shenandoah implementation, almost completely >>>>>>>> residing in >>>>>>>> gc/shenandoah >>>>>>>> >>>>>>>> ???*) shared-gc >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ????? - This is mostly boilerplate that is common to any GC >>>>>>>> ????? - referenceProcessor.cpp has a little change to make one >>>>>>>> assert not >>>>>>>> fail (next to CMS and G1) >>>>>>>> ????? - taskqueue.hpp has some small adjustments to enable >>>>>>>> subclassing >>>>>>>> >>>>>>>> ???*) shared-serviceability >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ????? - The usual code to support another GC >>>>>>>> >>>>>>>> ???*) shared-runtime >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ????? - A number of friends declarations to allow Shenandoah >>>>>>>> iterators to >>>>>>>> hook up with, >>>>>>>> ??????? e.g. ClassLoaderData, CodeCache, etc >>>>>>>> ????? - Warning and disabling JFR LeakProfiler >>>>>>>> ????? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>>>>> Shenandoah C2 optimizations >>>>>>>> ????? - Locks initialization in mutexLocker.cpp as usual >>>>>>>> ????? - VM operations defines for Shenandoah's VM ops >>>>>>>> ????? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>>>>> Shenandoah's logging >>>>>>>> ????? - The usual macros in macro.hpp >>>>>>>> >>>>>>>> ???*) shared-build >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ????? - Add shenandoah feature, enabled by default, as agreed with >>>>>>>> Vladimir K. beforehand >>>>>>>> ????? - Some flags for shenandoah-enabled compilation to get >>>>>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>>>>> ??????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>>>>> Shenandoah's barriers >>>>>>>> ????? - --param inline-unit-growth=1000 settings for 2 shenandoah >>>>>>>> source >>>>>>>> files, which is >>>>>>>> ??????? useful to get the whole marking loop inlined (observed >>>>>>>> significant >>>>>>>> regression if we >>>>>>>> ??????? don't) >>>>>>>> >>>>>>>> ???*) shared-tests >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ????? - Test infrastructure to support Shenandoah >>>>>>>> ????? - Shenandoah test groups >>>>>>>> ????? - Exclude Shenandoah in various tests that can be run with >>>>>>>> selected GC >>>>>>>> ????? - Enable/add configure for Shenandoah for tests that make >>>>>>>> sense to >>>>>>>> run with it >>>>>>>> >>>>>>>> ???*) shenandoah-tests >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ????? - Shenandoah specific tests, most reside in gc/shenandoah >>>>>>>> subdirectory >>>>>>>> ????? - A couple of tests configurations have been added, e.g. >>>>>>>> TestGCBasherWithShenandoah.java >>>>>>>> >>>>>>>> I intentionally left out shared-compiler for now, because we have >>>>>>>> some >>>>>>>> work left to do >>>>>>>> there, but if you click around you'll find the patch anyway, in >>>>>>>> case you >>>>>>>> want to take >>>>>>>> a peek at it. >>>>>>>> >>>>>>>> We have regular builds on: >>>>>>>> ??? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>>>>> ??? - {Windows} x {x86_64}, >>>>>>>> ??? - {MacOS X} x {x86_64} >>>>>>>> >>>>>>>> This also routinely passes: >>>>>>>> ??? - the new Shenandoah tests >>>>>>>> ??? - jcstress with/without aggressive Shenandoah verification >>>>>>>> ??? - specjvm2008 with/without aggressive Shenandoah verification >>>>>>>> >>>>>>>> >>>>>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she >>>>>>>> deserves >>>>>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>>>>> advice and support, my collegues in Oracle's GC, runtime and >>>>>>>> compiler >>>>>>>> teams for tirelessly helping with and reviewing all the GC >>>>>>>> interface and >>>>>>>> related changes, and of course the many early adopters for reporting >>>>>>>> bugs and success stories and feature requests: we wouldn't be here >>>>>>>> without any of you! >>>>>>>> >>>>>>>> Best regards, >>>>>>>> Roman >>>>>>>> >>>>>>> >>>>>> >>>>> >>> > From david.holmes at oracle.com Mon Dec 3 00:38:12 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 3 Dec 2018 10:38:12 +1000 Subject: RFR (round 3), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <2828302a-3e42-d35b-864d-1bbadba7a233@oracle.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <2b31b813-3178-4712-7a72-b3ec62394c71@redhat.com> <13c3fa3b-dae3-0042-ce7a-27fb7b4ee4ca@redhat.com> <2828302a-3e42-d35b-864d-1bbadba7a233@oracle.com> Message-ID: On 30/11/2018 8:40 pm, Per Liden wrote: > Hi, > > On 11/30/18 10:55 AM, Roman Kennke wrote: >>>> Hi Per, >>>> >>>> Thanks for taking time to look at this! >>>> >>>> We will take care of all the issues you mentioned. >>>> >>>> Regarding the flags, I think they are useful as they are now. >>>> Shenandoah >>>> can be run in 'passive' mode, which doesn't involve any concurrent GC >>>> (iow, it runs kindof like Parallel GC). In this mode we can selectively >>>> turn on/off different kinds of barriers. This is useful: >>>> - for debugging. if we see a crash and we suspect it's caused by a >>>> certain type of barrier, we can turn on/off those barriers to narrow >>>> down >>>> - for performance experiments: passive w/o any barriers give a good >>>> baseline against which we can measure impact of types of barriers. >>>> >>>> Debugging may or may not be useful in develop mode (some bugs only show >>>> up in product build), performance work quite definitely is not >>>> useful in >>>> develop builds, and therefore I think it makes sense to keep them as >>>> diagnostic, because that is what they are: diagnostic flags. >>>> >>>> Makes sense? >>> >>> I can see how these flags can be useful. But I think you might be in >>> violating-spec-territory here, if you're allowing users to turn on flags >>> that crash the VM. If they are safe to use in 'passive' mode, then I >>> think there should be code in shenandoahArguments.cpp that >>> rejects/corrects flag combinations that are unstable. >> >> Let us see if we can do that. >> >> If you think this violates the spec, then please point to the part that >> says diagnostic (!!) options must pass the TCK and/or not crash the JVM? >> I mean, it's called diagnostic for a reason. It seems counter-useful to >> bury diagnostic flags that we would be using for diagnostics. >> >>> I think the correct way to think about this is: We should pass the TCK, >>> regardless of which features are enabled/disabled (unless the VM barfs >>> at start-up because the chosen combination of flags are incompatible or >>> isn't supported in the current environment, etc). >>> >>> CC:ing Mikael here, who might be able to shed some light on what's ok >>> and not ok here. >> >> Yeah, again, where is it said that diagnostic flags must even pass the >> TCK? That is totally new to me. > > I'm not really the right person to speak authoritatively about this. > However, from what I understand it doesn't matter what kind of flag it > is. If a user can enable it in a production/shipped build I think it > must pass the TCK. That cannot be true in a strict sense ref: ./share/runtime/globals.hpp: diagnostic(ccstr, AbortVMOnException, NULL, which would obviously fail the TCK. However I think there is a distinction between a flag that causes the VM to abruptly terminate versus a flag that allows the VM to run in a manner that violates the spec. David ----- > > But let's have someone who knows more about the rules around that weight > in on this. > > cheers, > Per From jini.george at oracle.com Mon Dec 3 04:36:08 2018 From: jini.george at oracle.com (Jini George) Date: Mon, 3 Dec 2018 10:06:08 +0530 Subject: RFR: (S): JDK-8213323: sa/TestJmapCoreMetaspace.java and sa/TestJmapCore.java fail with ZGC In-Reply-To: <2b263b92-af0d-6f25-e92b-7eb814c4ead8@oracle.com> References: <783746d6-07d2-4a5d-afaa-9d8d3ac08a61@oracle.com> <0732e52d-8f73-fc2d-3147-987a84039adf@oracle.com> <2b263b92-af0d-6f25-e92b-7eb814c4ead8@oracle.com> Message-ID: <9dce665e-2c1e-b5bc-c40a-6bf5b478ca82@oracle.com> Gentle reminder ! Thanks, Jini On 11/29/2018 12:03 PM, Jini George wrote: > Thank you very much, Chris. I have modified HeapHprofBinWriter.java > slightly so that the hprof file does not dumped if this is ZGC. In the > test, we check for the presence of the hprof file and throw an exception > if the hprof file does not exist and only if this is not ZGC. The tests > are not excluded for zgc now. > > The modified webrev is at: > > http://cr.openjdk.java.net/~jgeorge/8213323/webrev.02/index.html > > Thanks! > Jini. > > > On 11/27/2018 2:51 AM, Chris Plummer wrote: >> Hi Jini, >> >> The tool changes look fine for disabling zgc support, but with the >> test changes you are no longer testing what happens if you use jmap >> with zgc. What would the tests do if you made no test changes? If they >> still fail ungracefully, could they be fixed by catching the >> RuntimeException you are now throwing? Maybe you could test that this >> only happens when using zgc. >> >> thanks, >> >> Chris >> >> On 11/21/18 7:49 PM, Jini George wrote: >>> Thanks a bunch, JC ! Could have a Reviewer also take a look at this >>> please ? >>> >>> - Jini. >>> >>> On 11/21/2018 11:05 PM, JC Beyler wrote: >>>> Hi Jini, >>>> >>>> Looks good to me, thanks for fixing the nits :) >>>> >>>> Thanks, >>>> Jc >>>> >>>> >>>> On Wed, Nov 21, 2018 at 9:29 AM Jini George >>> > wrote: >>>> >>>> ??? The modified webrev with the RuntimeException and the nit >>>> removed is at: >>>> >>>> ??? http://cr.openjdk.java.net/~jgeorge/8213323/webrev.01/ >>>> >>>> ??? Thanks! >>>> ??? Jini. >>>> >>>> ??? On 11/12/2018 12:41 PM, Jini George wrote: >>>> ???? > Thank you very much, JC, for looking into this. >>>> ???? > >>>> ???? > >>>> http://mail.openjdk.java.net/pipermail/zgc-dev/2018-August/000455.html >>>> ???? > >>>> ???? > The link above provides an explanation of why it is difficult to >>>> ??? support >>>> ???? > ZGC with SA where there is an iteration over the heap. And >>>> ??? currently, I >>>> ???? > am unsure as to whether we will devise a way to overcome this >>>> ??? issue. We >>>> ???? > currently have the following JBS entry for tracking this. >>>> ???? > >>>> ???? > https://bugs.openjdk.java.net/browse/JDK-8207843 (SA: Add >>>> support >>>> ??? for >>>> ???? > Object Histogram with ZGC) >>>> ???? > >>>> ???? > I have added a comment in the ER above to keep track of >>>> enabling the >>>> ???? > tests if this is resolved. >>>> ???? > >>>> ???? > I agree with your comment on throwing a RuntimeException. >>>> Doing this >>>> ???? > avoids the misleading "heap written to" message. I will modify >>>> ??? this and >>>> ???? > remove the ';' you pointed out and post another webrev. >>>> ???? > >>>> ???? > Thank you, >>>> ???? > Jini. >>>> ???? > >>>> ???? > >>>> ???? > >>>> ???? > On 11/9/2018 9:36 PM, JC Beyler wrote: >>>> ???? >> Hi Jini, >>>> ???? >> >>>> ???? >> The webrev looks good to me as well except for a few >>>> ??? questions/comments: >>>> ???? >> >>>> ???? >> I have a generic question that is related to the webrev: >>>> ???? >> ?? - Are there plans at some point for Jmap to support ZGC >>>> heaps in >>>> ???? >> the future ? Or is this infeasible? >>>> ???? >> ?? ? ? ? I ask because if a lot of tests are disabled for >>>> ZGC for a >>>> ???? >> limited amount of time (in order to provide time for future >>>> ??? support) >>>> ???? >> there should be a means to scrub the tests at a later date >>>> to see >>>> ???? >> which are now supported, no? >>>> ???? >> >>>> ???? >> ?? - In >>>> ???? >> >>>> http://cr.openjdk.java.net/~jgeorge/8213323/webrev.00/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/utilities/HeapHprofBinWriter.java.html: >>>> >>>> >>>> ???? >> >>>> ???? >> >>>> ???? >> ??397? ? ? ? ?if (vm.getUniverse().heap() instanceof >>>> ??? ZCollectedHeap) { >>>> ???? >> ??398? ? ? ? ? ? ?System.err.println("WARNING: Operation not >>>> ??? supported >>>> ???? >> for ZGC."); >>>> ???? >> ??399? ? ? ? ? ? ?return; >>>> ???? >> ??400? ? ? ? ?}; >>>> ???? >> >>>> ???? >> ?? ? ?- 1 nit is the ';'? after the closing '}' >>>> ???? >> ?? ? ?- Should the code throw a RuntimeException instead? Just >>>> ???? >> printing an error seems "weak" for me when this really won't >>>> ??? work. Of >>>> ???? >> course, that means tracking the callers now of write and >>>> probably >>>> ???? >> adding exception handling there and I'm not sure that is >>>> something >>>> ???? >> that is warranted here but thought I would ask :) >>>> ???? >> >>>> ???? >> Thanks! >>>> ???? >> Jc >>>> ???? >> >>>> ???? >> >>>> ???? >> >>>> ???? >> Thanks, >>>> ???? >> Jc >>>> ???? >> >>>> ???? >> On Fri, Nov 9, 2018 at 1:47 AM gary.adams at oracle.com >>>> ??? >>>> ???? >> > >>>> ??? >>>> ???? >> >> >>>> ??? wrote: >>>> ???? >> >>>> ???? >> ??? Looks OK to me. >>>> ???? >> >>>> ???? >> ??? On 11/9/18 1:40 AM, Jini George wrote: >>>> ???? >> ???? > Hello! >>>> ???? >> ???? > >>>> ???? >> ???? > Requesting reviews for a small change for disabling the >>>> ??? testing of >>>> ???? >> ???? > TestJmapCoreMetaspace.java and TestJmapCore.java with >>>> ??? ZGC. The >>>> ???? >> ??? change >>>> ???? >> ???? > also includes skipping of creating the heapdump file >>>> with >>>> ??? jmap if >>>> ???? >> ??? the >>>> ???? >> ???? > GC being used is ZGC. >>>> ???? >> ???? > >>>> ???? >> ???? > Webrev: >>>> ??? http://cr.openjdk.java.net/~jgeorge/8213323/webrev.00/ >>>> ???? >> ???? > Bug ID: https://bugs.openjdk.java.net/browse/JDK-8213323 >>>> ???? >> ???? > >>>> ???? >> ???? > Thanks, >>>> ???? >> ???? > Jini. >>>> ???? >> >>>> ???? >> >>>> ???? >> >>>> ???? >> >>>> ???? >> -- >>>> ???? >> >>>> ???? >> Thanks, >>>> ???? >> Jc >>>> >>>> >>>> >>>> -- >>>> >>>> Thanks, >>>> Jc >> >> From john.r.rose at oracle.com Mon Dec 3 04:52:43 2018 From: john.r.rose at oracle.com (John Rose) Date: Sun, 2 Dec 2018 20:52:43 -0800 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> Message-ID: <8769251B-C42F-4DDF-9E72-8628D09C3464@oracle.com> On Nov 30, 2018, at 3:22 PM, coleen.phillimore at oracle.com wrote: > > Hi, I looked at the runtime changes, which are very few, since you've upstreamed most of the runtime dependent changes already. > They look good to me. "since you've upstreamed most of the runtime dependent changes already" Yes. That's what winning looks like! ? John From david.holmes at oracle.com Mon Dec 3 07:14:46 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 3 Dec 2018 17:14:46 +1000 Subject: RFR 8214148: [TESTBUG] serviceability/tmtools/jstack/WaitNotifyThreadTest.java is not doing what is expected In-Reply-To: References: <57a4cb4c-87e5-bee4-192a-4f41d19e4ea5@oracle.com> <18c5ccc0-4058-b4bd-9d2a-3fbfafd5069b@oracle.com> Message-ID: <1f51ea19-9d1a-916a-e1f0-428a305796f5@oracle.com> Hi Patricio, On 1/12/2018 2:31 am, Patricio Chilano wrote: > Hi David, > > On 11/29/18 8:05 PM, David Holmes wrote: >> Hi Patricio, >> >> This seems very complicated and I'm not quite seeing how it all goes >> together. The check for waiting to re-lock now seems to dominant the >> test and obscure the original checks. >> >> I'm not sure this is worthwhile in the context of this test. It might >> be much simpler to just get rid of the existing "waiting to re-lock" >> check which will not be seen and then if we really want to check that >> case add a much simpler form that just checks for that. > Ok, I actually had similar thoughts while I was adding the extra code. > Here is the new webrev: > > http://cr.openjdk.java.net/~pchilanomate/8214148.02/webrev/ > > I removed the check for the "waiting to re-lock" case. Good - the more I look at this test the more I see the "waiting to re-lock" case is just not relevant to it. The addition of the code to wait until the target thread is in the right state seems good. I'm unclear on some of the changes to the checking code in analyzeThreadStackWaiting. It seems you removed the code that watched for finding the wrong monitor and replaced that with a call to assertMonitorInfo. But the latter is passed the address you got from the MonitorInfo in the first place so the check of the address is never going to fail. ?? That said you can't be waiting on two monitors so I don't see how we can ever have the wrong one ?? A few minor style nits: Pre-existing: 54 //Notify the waiting thread, so it stops waiting and sleeps Please add a space after // 106 // Start athread that just waits s/athread/a thread/ 145 throw new RuntimeException(OBJECT_WAIT 146 + " method has to contain one lock record but it contains " + mi.getLocks().size()); Indentation is wrong - the '+' should align with the O in OBJECT. Break into three lines (at second + if needed) New: 154 if(mi.getLocks().size() == 1){ Space after "if", and space before { 157 else{ Space before { 158 throw new RuntimeException(RUN_METHOD + " method has to contain one lock record but it contains " 159 + mi.getLocks().size()); Incorrect indentation - '+' should align with R in RUN > I'm not sure if > it's okay to keep the change to > serviceability/tmtools/jstack/utils/DefaultFormat.java then. It doesn't > really affect this test, but it is needed for jstack to detect the locks > that appear in the stack report with the message "waiting to re-lock in > wait()". I'd probably revert that change at this stage. Thanks, David > Thanks, > Patricio >> To me the simplest way to see the "re-lock in wait" case is to just: >> >> synchronized(obj) { >> ?? obj.notifyAll(); >> ?? <= take stack dump here => >> } >> >> Cheers, >> David >> >> On 30/11/2018 5:52 am, Daniel D. Daugherty wrote: >>> Adding serviceability-dev at ... since the 'tmtools' including 'jstack' >>> are owned by the Serviceability team. >>> >>> Dan >>> >>> >>> On 11/28/18 4:21 PM, Patricio Chilano wrote: >>>> Hi all, >>>> >>>> Could you review this?fix for test >>>> serviceability/tmtools/jstack/WaitNotifyThreadTest.java? >>>> >>>> On one hand the test was not properly checking what it was intended >>>> to check, since as mentioned in JBS the logic for checking the >>>> method name was wrong. Also since there was only one monitor in the >>>> test, the "for" loop with the message "waiting to re-lock in wait()" >>>> was never actually reached. >>>> >>>> Additionally, with change 8150689 the message?"waiting to re-lock in >>>> wait()" is now shown in the frame where the relocking is actually >>>> taking place, so the logic for checking that should change. >>>> >>>> I fixed the first issues and added logic to check for the "waiting >>>> to re-lock in wait()" case. I used the Thread.State attribute to >>>> check desire states are reached before getting the thread dump >>>> reports through jstack. I run the test in mach5 several times for >>>> all platforms and they all passed. >>>> >>>> Webrev URL: http://cr.openjdk.java.net/~pchilanomate/8214148.01/webrev >>>> Bug URL: https://bugs.openjdk.java.net/browse/JDK-8214148 >>>> >>>> >>>> Thanks, >>>> Patricio >>> > From jini.george at oracle.com Mon Dec 3 07:57:36 2018 From: jini.george at oracle.com (Jini George) Date: Mon, 3 Dec 2018 13:27:36 +0530 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> Message-ID: <1da44101-954c-c560-c332-af82bac2abec@oracle.com> Hi Roman, A few comments on the SA changes: ==> Could you please add the following lines in src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/HSDB.java from line 1120 onwards to avoid the "[Unknown generation]" message with hsdb while displaying the Stack Memory for a mutator thread ? else if (collHeap instanceof ShenandoahHeap) { ShenandoahHeap heap = (ShenandoahHeap) collHeap; anno = "ShenandoahHeap "; bad = false; } ==> src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/HeapSummary.java The printGCAlgorithm() method would need to be updated to read in the UseShenandoahGC flag to avoid the default "Mark Sweep Compact GC" being displayed with jhsdb jmap -heap. ==> src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shared/GCName.java Could you please add "Shenandoah" to the GCName enum list ? ==> src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shared/GCCause.java Could you please update the GCCause enum values to include these: _shenandoah_stop_vm, _shenandoah_allocation_failure_evac, _shenandoah_concurrent_gc, _shenandoah_traversal_gc, _shenandoah_upgrade_to_full_gc, ==> share/classes/sun/jvm/hotspot/runtime/VMOps.java It would be good to add 'ShenandoahOperation' to the VMOps enum (though it is probably not in sync now). Thank you, Jini. On 12/1/2018 2:30 AM, Roman Kennke wrote: > Hi all, > > here comes round 4 of Shenandoah upstreaming review: > > This includes fixes for the issues that Per brought up: > - Verify and gracefully reject dangerous flags combinations that > disables required barriers > - Revisited @requires filters in tests > - Trim unused code from Shenandoah's SA impl > - Move ShenandoahGCTracer to gc/shenandoah > - Fix ordering of GC names in various files > - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W > > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ > > Thanks everybody for taking time to review this! > Roman > >> Hello all, >> >> Thanks so far for all the reviews and support! >> >> I forgot to update the 'round' yesterday. We are in round 3 now :-) >> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >> >> Things we've changed today: >> - We moved shenandoah-specific code out of .ad files into our own .ad >> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >> requires an addition in build machinery though, see >> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >> - Improved zero-disabling and build-code-simplification as suggested by >> Magnus and Per >> - Cleaned up some leftovers in C2 >> - Improved C2 loop opts code by introducing another APIs in >> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now. >> - We would all very much prefer to keep ShenandoahXYZNode names, as >> noted earlier. This stuff is Shenandoah-specific, so let's just call it >> that. >> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >> - Rebased on jdk-12+22 >> >> - Question: let us know if you need separate RFE for the new BSC2 APIs? >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >> >> Thanks, >> Roman >> >>> Alright, we fixed: >>> - The minor issues that Kim reported in shared-gc >>> - A lot of fixes in shared-tests according to Leonid's review >>> - Disabled SA heapdumping similar to ZGC as Per suggested >>> >>> Some notes: >>> Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>> correct. The @requires there means to exclude runs with both CMS and >>> ExplicitGCInvokesConcurrent at the same time, because that would be >>> (expectedly) failing. It can run CMS, default GC and any other GC just >>> fine. Adding the same clause for Shenandoah means the same, and filters >>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>> made the condition a bit clearer by avoiding triple-negation. >>> >>> See: >>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>> >>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>> >>> we filter them for Shenandoah now. I'm wondering: how do you get past >>> those with ZGC? >>> >>> See: >>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>> >>> (Note to Leonid and tests reviewers: I'll add those related filters in >>> next round). >>> >>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>> better. I can tell that we're not done with C2 yet. Can you look over >>> the code and see what is ok, and especially what is not ok, so that we >>> can focus our efforts on the relevant parts? >>> >>> Updated set of webrevs: >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>> >>> Thanks, >>> Roman >>> >>> >>>> Hi, >>>> >>>> This is the first round of changes for including Shenandoah GC into >>>> mainline. >>>> I divided the review into parts that roughly correspond to the mailing lists >>>> that would normally review it, and I divided it into 'shared' code >>>> changes and >>>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>>> eventually >>>> push them as single 'combined' changeset, once reviewed. >>>> >>>> JEP: >>>> ? https://openjdk.java.net/jeps/189 >>>> Bug entry: >>>> >>>> ?https://bugs.openjdk.java.net/browse/JDK-8214259 >>>> >>>> Webrevs: >>>> ? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>> >>>> For those who want to see the full change, have a look at the >>>> shenandoah-complete >>>> >>>> directory, >>>> it contains the full combined webrev. Alternatively, there is the file >>>> shenandoah-master.patch >>>> , >>>> which is what I intend to commit (and which should be equivalent to the >>>> 'shenandoah-complete' webrev). >>>> >>>> Sections to review (at this point) are the following: >>>> ?*) shenandoah-gc >>>> >>>> ??? - Actual Shenandoah implementation, almost completely residing in >>>> gc/shenandoah >>>> >>>> ?*) shared-gc >>>> >>>> ??? - This is mostly boilerplate that is common to any GC >>>> ??? - referenceProcessor.cpp has a little change to make one assert not >>>> fail (next to CMS and G1) >>>> ??? - taskqueue.hpp has some small adjustments to enable subclassing >>>> >>>> ?*) shared-serviceability >>>> >>>> ??? - The usual code to support another GC >>>> >>>> ?*) shared-runtime >>>> >>>> ??? - A number of friends declarations to allow Shenandoah iterators to >>>> hook up with, >>>> ????? e.g. ClassLoaderData, CodeCache, etc >>>> ??? - Warning and disabling JFR LeakProfiler >>>> ??? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>> Shenandoah C2 optimizations >>>> ??? - Locks initialization in mutexLocker.cpp as usual >>>> ??? - VM operations defines for Shenandoah's VM ops >>>> ??? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>> Shenandoah's logging >>>> ??? - The usual macros in macro.hpp >>>> >>>> ?*) shared-build >>>> >>>> ??? - Add shenandoah feature, enabled by default, as agreed with >>>> Vladimir K. beforehand >>>> ??? - Some flags for shenandoah-enabled compilation to get >>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>> ????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>> Shenandoah's barriers >>>> ??? - --param inline-unit-growth=1000 settings for 2 shenandoah source >>>> files, which is >>>> ????? useful to get the whole marking loop inlined (observed significant >>>> regression if we >>>> ????? don't) >>>> >>>> ?*) shared-tests >>>> >>>> ??? - Test infrastructure to support Shenandoah >>>> ??? - Shenandoah test groups >>>> ??? - Exclude Shenandoah in various tests that can be run with selected GC >>>> ??? - Enable/add configure for Shenandoah for tests that make sense to >>>> run with it >>>> >>>> ?*) shenandoah-tests >>>> >>>> ??? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >>>> ??? - A couple of tests configurations have been added, e.g. >>>> TestGCBasherWithShenandoah.java >>>> >>>> I intentionally left out shared-compiler for now, because we have some >>>> work left to do >>>> there, but if you click around you'll find the patch anyway, in case you >>>> want to take >>>> a peek at it. >>>> >>>> We have regular builds on: >>>> ? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>> ? - {Windows} x {x86_64}, >>>> ? - {MacOS X} x {x86_64} >>>> >>>> This also routinely passes: >>>> ? - the new Shenandoah tests >>>> ? - jcstress with/without aggressive Shenandoah verification >>>> ? - specjvm2008 with/without aggressive Shenandoah verification >>>> >>>> >>>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>>> the credit for being the original inventor of Shenandoah, Aleksey >>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>> teams for tirelessly helping with and reviewing all the GC interface and >>>> related changes, and of course the many early adopters for reporting >>>> bugs and success stories and feature requests: we wouldn't be here >>>> without any of you! >>>> >>>> Best regards, >>>> Roman >>>> >>> >> > From per.liden at oracle.com Mon Dec 3 09:05:52 2018 From: per.liden at oracle.com (Per Liden) Date: Mon, 3 Dec 2018 10:05:52 +0100 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <8d5b0cd0-d791-3c77-7302-eea97012897e@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <566619f4-87f5-a605-ca49-8b9083fc17eb@oracle.com> <20c6ccb0-4161-90e9-a6b7-f1aff4c05bef@redhat.com> <169dbd79-2ae1-8c9a-67b2-bc3a9f0f01a7@oracle.com> <8d5b0cd0-d791-3c77-7302-eea97012897e@redhat.com> Message-ID: Hi, On 11/28/18 3:24 PM, Roman Kennke wrote: >>> Hi Per, >>> >>>> Hi Roman, >>>> >>>> On 11/26/18 10:39 PM, Roman Kennke wrote: >>>> [...] >>>>> ???*) shared-serviceability >>>>> >>>>> >>>>> >>>>> ????? - The usual code to support another GC >>>> >>>> Just had a quick look at the SA part. I was thinking you'd have the same >>>> problem as ZGC here, with regards to parsing the heap and potentially >>>> reading garbage when you step on a Klass* which had been unloaded? >>> >>> Possible. I am myself not very familiar with SA. I guess it depends on >>> how SA does it: if it iterates objects via CH::object_iterate() (e.g. >>> same entry point as, e.g., heap-dumping code), then we should be fine. >>> We're kicking off a traversal rather than straight scan there. If >>> however SA somehow makes a raw scan itself, then we'd have the problem >>> you describe. >> >> The SA does a raw scan itself, which is the root of the problem. >> ObejctHeap.iterateLiveRegions() will locate the first object in a region >> by doing >> >> ? OopHandle handle = bottom.addOffsetToAsOopHandle(0); >> >> and to get the next object it does >> >> ? handle.addOffsetToAsOopHandle(obj.getObjectSize()); >> >> and you'll crash. So I'm afraid this will not work for Shenandoah either. > > Alright. I'll 'disable' it like you did with ZGC then. Thanks for > pointing it out. > > I'm wondering: this would crash with G1 and > +ClassUnloadingWithConcurrentMark too, then? That's a very good point and I think you're actually right. Would be good if someone from the G1 camp could confirm. cheers, Per From david.griffiths at gmail.com Mon Dec 3 09:35:58 2018 From: david.griffiths at gmail.com (David Griffiths) Date: Mon, 3 Dec 2018 09:35:58 +0000 Subject: Suggested improvement to X86Frame.getInterpreterFrameBCI In-Reply-To: References: <0f590762-2c34-0ceb-0937-be4ef39d30b2@oracle.com> <11798f90-4e4e-55e3-3226-35c26df0a202@oracle.com> Message-ID: Ok great, I will submit new patch then. Thanks for reviewing! Cheers, David On Fri, 30 Nov 2018 at 17:06, JC Beyler wrote: > Hi both, > > The webrev looks good to me but I could see gains of just adding a new > constructor instead of doing a new + set. > > LinuxAMD64JavaThreadPDAccess.java would become consistent with the rest of > the code: > + } else if (VM.getVM().getInterpreter().contains(guesser.getPC())) { > + // pass the value of R13 which contains the bcp for the top level > frame > + return new X86Frame(guesser.getSP(), guesser.getFP(), > guesser.getPC(), > + context.getRegisterAsAddress(AMD64ThreadContext.R13)); > } else { > > - And for X86Frame.java, that means there is no set method (there isn't a > single one yet there so we are consistent there). > - Finally, instead of just r13 internally to the Frame, we could just call > it the bcp since that is what you are saying it is and then adapt the > getInterpreterFrameBCI a bit because a bcp local variable is there :-) > > But these are nits :), > Jc > > On Fri, Nov 30, 2018 at 6:21 AM Jini George > wrote: > >> Your patch looks good to me, David. I can sponsor this for you if we get >> one more review. >> >> Thanks, >> Jini. >> >> On 11/22/2018 5:42 PM, David Griffiths wrote: >> > Thanks Jini, please find patch for Java 9 attached (I don't have author >> > access to the bug itself). >> > >> > Cheers, >> > >> > David >> > >> > On Thu, 22 Nov 2018 at 09:02, Jini George > > > wrote: >> > >> > Thank you very much for working on the fix for this issue, David. It >> > would be great if you can send in a complete patch for the review >> (With >> > a first cut look, there seems to be missing pieces). >> > >> > I have created a bug for this: >> > >> > https://bugs.openjdk.java.net/browse/JDK-8214226 >> > >> > Thank you, >> > Jini >> > >> > On 11/22/2018 12:50 AM, David Griffiths wrote: >> > > PS: should have added a new X86Frame constructor really, may have >> > just >> > > been put off because there is already a four address constructor >> so >> > > would have had to add dummy argument or something. >> > > >> > > On Wed, 21 Nov 2018 at 19:15, David Griffiths >> > >> > > > > >> wrote: >> > > >> > > Hi, thanks, apart from adding a setter for R13 in X86Frame, >> the >> > > other half of the fix is this: >> > > >> > > public Frame getCurrentFrameGuess(JavaThread thread, >> > Address >> > > addr) { >> > > ThreadProxy t = getThreadProxy(addr); >> > > AMD64ThreadContext context = (AMD64ThreadContext) >> > t.getContext(); >> > > AMD64CurrentFrameGuess guesser = new >> > > AMD64CurrentFrameGuess(context, thread); >> > > if (!guesser.run(GUESS_SCAN_RANGE)) { >> > > return null; >> > > } >> > > if (guesser.getPC() == null) { >> > > return new X86Frame(guesser.getSP(), guesser.getFP()); >> > > } else if >> > (VM.getVM().getInterpreter().contains(guesser.getPC())) { >> > > // pass the value of R13 which contains the bcp for >> > the top >> > > level frame >> > > Address r13 = >> > > context.getRegisterAsAddress(AMD64ThreadContext.R13); >> > > X86Frame frame = new X86Frame(guesser.getSP(), >> > > guesser.getFP(), guesser.getPC()); >> > > frame.setR13(r13); >> > > return frame; >> > > } else { >> > > return new X86Frame(guesser.getSP(), guesser.getFP(), >> > > guesser.getPC()); >> > > } >> > > } >> > > >> > > (the whole "if pc in interpreter" block is new) >> > > >> > > Overhead likely to be low as this is only used in diagnostic >> > code. >> > > Can't think of any risk but I'm not an expert on this code. >> > > >> > > Cheers, >> > > >> > > David >> > > >> > > On Wed, 21 Nov 2018 at 19:01, JC Beyler > > >> > > >> >> wrote: >> > > >> > > Hi David, >> > > >> > > I think the easiest would be to see whole change to >> > understand >> > > the repercussions of the change. I would imagine that any >> > change >> > > that helps stacktraces being more precise is a good >> thing but >> > > there are questions that arise every time: >> > > - What is the overhead of adding this? >> > > - Does this add any risk of failure? >> > > >> > > I'd imagine that the change is relatively small and >> should be >> > > easy to assess this but seeing it would make things >> easier to >> > > determine. >> > > >> > > That being said, I'm not a reviewer for OpenJDK so this >> is >> > > really just my 2cents, >> > > Jc >> > > >> > > On Wed, Nov 21, 2018 at 9:17 AM David Griffiths >> > > > > > david.griffiths at gmail.com >> > >> >> > > wrote: >> > > >> > > Hi, I'm new to this mailing list and working on a >> project >> > > that makes use of the SA classes to get stack traces >> > from a >> > > paused in flight JVM (we can't use JDWP). I have >> observed >> > > that if the top frame is in the interpreter it >> > reports the >> > > BCI and line number incorrectly. This is because >> > > X86Frame.getInterpreterFrameBCI uses the value stored >> > on the >> > > stack rather than the actual live value stored in >> R13. >> > > >> > > I have a patch for this which lets >> > > LinuxAMD64JavaThreadPDAccess.getCurrentFrameGuess >> > pass the >> > > R13 value to X86Frame so that the latter can then do: >> > > >> > > public int getInterpreterFrameBCI() { >> > > Address bcp = >> > > addressOfInterpreterFrameBCX().getAddressAt(0); >> > > // If we are in the top level frame then R13 may >> > have >> > > been set for us which contains >> > > // the BCP. If so then let it take priority. If >> > we are >> > > in a top level interpreter frame, >> > > // the BCP is live in R13 (on x86) and not saved >> > in the >> > > BCX stack slot. >> > > if (r13 != null) { >> > > bcp = r13; >> > > } >> > > Address methodHandle = >> > > addressOfInterpreterFrameMethod().getAddressAt(0); >> > > >> > > and this fixes the problem. >> > > >> > > Does this sound like a good idea and if so should I >> > submit a >> > > patch? >> > > >> > > Cheers, >> > > >> > > David >> > > >> > > >> > > >> > > -- >> > > >> > > Thanks, >> > > Jc >> > > >> > >> > > > -- > > Thanks, > Jc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From per.liden at oracle.com Mon Dec 3 10:28:45 2018 From: per.liden at oracle.com (Per Liden) Date: Mon, 3 Dec 2018 11:28:45 +0100 Subject: RFR: 8214484: ZGC: Exclude SA tests ClhsdbJhisto and TestHeapDumpFor* In-Reply-To: <3e46ddc12dca834dfae438b58732ec9795c4387e.camel@oracle.com> References: <5cf23e3c-37a2-da8a-b32f-a7c1b57036fe@oracle.com> <3e46ddc12dca834dfae438b58732ec9795c4387e.camel@oracle.com> Message-ID: Thanks for reviewing, Thomas! /Per On 11/30/18 1:53 PM, Thomas Schatzl wrote: > Hi, > > On Thu, 2018-11-29 at 16:07 +0100, Per Liden wrote: >> There's no support for SA heap walking when using ZGC. Hence the >> tests >> for this should not execute when ZGC is enabled. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8214484 >> Webrev: http://cr.openjdk.java.net/~pliden/8214484/webrev.0 >> > > looks good after the explanation of the subtleties of jtreg > attributes. > > Thanks, > Thomas > > From rkennke at redhat.com Mon Dec 3 12:30:19 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 3 Dec 2018 13:30:19 +0100 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <1da44101-954c-c560-c332-af82bac2abec@oracle.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <1da44101-954c-c560-c332-af82bac2abec@oracle.com> Message-ID: <479eaac8-73a9-4a07-0e7e-2c8bdb672bf4@redhat.com> Hi Jini, Thanks for your suggestions. I've added this to Shenandoah's dev: http://cr.openjdk.java.net/~rkennke/shenandoah-sa/webrev.01/ and it will show up in next round of webrevs. Thanks, Roman > A few comments on the SA changes: > > ==> Could you please add the following lines in > src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/HSDB.java from line > 1120 onwards to avoid the "[Unknown generation]" message with hsdb while > displaying the Stack Memory for a mutator thread ? > > else if (collHeap instanceof ShenandoahHeap) { > ?? ShenandoahHeap heap = (ShenandoahHeap) collHeap; > ?? anno = "ShenandoahHeap "; > ?? bad = false; > } > > ==> > src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/HeapSummary.java > > The printGCAlgorithm() method would need to be updated to read in the > UseShenandoahGC flag to avoid the default "Mark Sweep Compact GC" being > displayed with jhsdb jmap -heap. > > ==> > src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shared/GCName.java > > Could you please add "Shenandoah" to the GCName enum list ? > > ==> > src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shared/GCCause.java > > Could you please update the GCCause enum values to include these: > > ??? _shenandoah_stop_vm, > ??? _shenandoah_allocation_failure_evac, > ??? _shenandoah_concurrent_gc, > ??? _shenandoah_traversal_gc, > ??? _shenandoah_upgrade_to_full_gc, > > ==> share/classes/sun/jvm/hotspot/runtime/VMOps.java > > It would be good to add 'ShenandoahOperation' to the VMOps enum (though > it is probably not in sync now). > > Thank you, > Jini. > > On 12/1/2018 2:30 AM, Roman Kennke wrote: >> Hi all, >> >> here comes round 4 of Shenandoah upstreaming review: >> >> This includes fixes for the issues that Per brought up: >> - Verify and gracefully reject dangerous flags combinations that >> disables required barriers >> - Revisited @requires filters in tests >> - Trim unused code from Shenandoah's SA impl >> - Move ShenandoahGCTracer to gc/shenandoah >> - Fix ordering of GC names in various files >> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >> >> Thanks everybody for taking time to review this! >> Roman >> >>> Hello all, >>> >>> Thanks so far for all the reviews and support! >>> >>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >>> >>> Things we've changed today: >>> - We moved shenandoah-specific code out of .ad files into our own .ad >>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>> requires an addition in build machinery though, see >>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>> - Improved zero-disabling and build-code-simplification as suggested by >>> Magnus and Per >>> - Cleaned up some leftovers in C2 >>> - Improved C2 loop opts code by introducing another APIs in >>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards >>> now. >>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>> that. >>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>> - Rebased on jdk-12+22 >>> >>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>> >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>> >>> Thanks, >>> Roman >>> >>>> Alright, we fixed: >>>> - The minor issues that Kim reported in shared-gc >>>> - A lot of fixes in shared-tests according to Leonid's review >>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>> >>>> Some notes: >>>> Leonid:? test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>> correct. The @requires there means to exclude runs with both CMS and >>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>> fine. Adding the same clause for Shenandoah means the same, and filters >>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>> made the condition a bit clearer by avoiding triple-negation. >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>> >>>> >>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>> >>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>> those with ZGC? >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>> >>>> >>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>> next round). >>>> >>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>> better. I can tell that we're not done with C2 yet. Can you look over >>>> the code and see what is ok, and especially what is not ok, so that we >>>> can focus our efforts on the relevant parts? >>>> >>>> Updated set of webrevs: >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>> >>>> Thanks, >>>> Roman >>>> >>>> >>>>> Hi, >>>>> >>>>> This is the first round of changes for including Shenandoah GC into >>>>> mainline. >>>>> I divided the review into parts that roughly correspond to the >>>>> mailing lists >>>>> that would normally review it, and I divided it into 'shared' code >>>>> changes and >>>>> 'shenandoah' code changes (actually, mostly additions). The intend >>>>> is to >>>>> eventually >>>>> push them as single 'combined' changeset, once reviewed. >>>>> >>>>> JEP: >>>>> ?? https://openjdk.java.net/jeps/189 >>>>> Bug entry: >>>>> >>>>> ??https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>> >>>>> Webrevs: >>>>> ?? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>> >>>>> For those who want to see the full change, have a look at the >>>>> shenandoah-complete >>>>> >>>>> >>>>> directory, >>>>> it contains the full combined webrev. Alternatively, there is the file >>>>> shenandoah-master.patch >>>>> , >>>>> >>>>> which is what I intend to commit (and which should be equivalent to >>>>> the >>>>> 'shenandoah-complete' webrev). >>>>> >>>>> Sections to review (at this point) are the following: >>>>> ??*) shenandoah-gc >>>>> >>>>> >>>>> ???? - Actual Shenandoah implementation, almost completely residing in >>>>> gc/shenandoah >>>>> >>>>> ??*) shared-gc >>>>> >>>>> >>>>> ???? - This is mostly boilerplate that is common to any GC >>>>> ???? - referenceProcessor.cpp has a little change to make one >>>>> assert not >>>>> fail (next to CMS and G1) >>>>> ???? - taskqueue.hpp has some small adjustments to enable subclassing >>>>> >>>>> ??*) shared-serviceability >>>>> >>>>> >>>>> ???? - The usual code to support another GC >>>>> >>>>> ??*) shared-runtime >>>>> >>>>> >>>>> ???? - A number of friends declarations to allow Shenandoah >>>>> iterators to >>>>> hook up with, >>>>> ?????? e.g. ClassLoaderData, CodeCache, etc >>>>> ???? - Warning and disabling JFR LeakProfiler >>>>> ???? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>> Shenandoah C2 optimizations >>>>> ???? - Locks initialization in mutexLocker.cpp as usual >>>>> ???? - VM operations defines for Shenandoah's VM ops >>>>> ???? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>> Shenandoah's logging >>>>> ???? - The usual macros in macro.hpp >>>>> >>>>> ??*) shared-build >>>>> >>>>> >>>>> ???? - Add shenandoah feature, enabled by default, as agreed with >>>>> Vladimir K. beforehand >>>>> ???? - Some flags for shenandoah-enabled compilation to get >>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>> ?????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>> Shenandoah's barriers >>>>> ???? - --param inline-unit-growth=1000 settings for 2 shenandoah >>>>> source >>>>> files, which is >>>>> ?????? useful to get the whole marking loop inlined (observed >>>>> significant >>>>> regression if we >>>>> ?????? don't) >>>>> >>>>> ??*) shared-tests >>>>> >>>>> >>>>> ???? - Test infrastructure to support Shenandoah >>>>> ???? - Shenandoah test groups >>>>> ???? - Exclude Shenandoah in various tests that can be run with >>>>> selected GC >>>>> ???? - Enable/add configure for Shenandoah for tests that make >>>>> sense to >>>>> run with it >>>>> >>>>> ??*) shenandoah-tests >>>>> >>>>> >>>>> ???? - Shenandoah specific tests, most reside in gc/shenandoah >>>>> subdirectory >>>>> ???? - A couple of tests configurations have been added, e.g. >>>>> TestGCBasherWithShenandoah.java >>>>> >>>>> I intentionally left out shared-compiler for now, because we have some >>>>> work left to do >>>>> there, but if you click around you'll find the patch anyway, in >>>>> case you >>>>> want to take >>>>> a peek at it. >>>>> >>>>> We have regular builds on: >>>>> ?? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>> ?? - {Windows} x {x86_64}, >>>>> ?? - {MacOS X} x {x86_64} >>>>> >>>>> This also routinely passes: >>>>> ?? - the new Shenandoah tests >>>>> ?? - jcstress with/without aggressive Shenandoah verification >>>>> ?? - specjvm2008 with/without aggressive Shenandoah verification >>>>> >>>>> >>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she >>>>> deserves >>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>> teams for tirelessly helping with and reviewing all the GC >>>>> interface and >>>>> related changes, and of course the many early adopters for reporting >>>>> bugs and success stories and feature requests: we wouldn't be here >>>>> without any of you! >>>>> >>>>> Best regards, >>>>> Roman >>>>> >>>> >>> >> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From per.liden at oracle.com Mon Dec 3 12:45:31 2018 From: per.liden at oracle.com (Per Liden) Date: Mon, 3 Dec 2018 13:45:31 +0100 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> Message-ID: <66f45e27-9e75-42ca-2a9b-c77166d38533@oracle.com> Hi Roman, On 11/30/18 10:00 PM, Roman Kennke wrote: > Hi all, > > here comes round 4 of Shenandoah upstreaming review: > > This includes fixes for the issues that Per brought up: > - Verify and gracefully reject dangerous flags combinations that > disables required barriers > - Revisited @requires filters in tests > - Trim unused code from Shenandoah's SA impl > - Move ShenandoahGCTracer to gc/shenandoah > - Fix ordering of GC names in various files > - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W Thanks for fixing. Looks good to me, except it looks like you missed adjusting the macro order in the following files: src/hotspot/share/gc/shared/gc_globals.hpp src/hotspot/share/gc/shared/vmStructs_gc.hpp cheers, Per > > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ > > Thanks everybody for taking time to review this! > Roman > >> Hello all, >> >> Thanks so far for all the reviews and support! >> >> I forgot to update the 'round' yesterday. We are in round 3 now :-) >> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >> >> Things we've changed today: >> - We moved shenandoah-specific code out of .ad files into our own .ad >> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >> requires an addition in build machinery though, see >> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >> - Improved zero-disabling and build-code-simplification as suggested by >> Magnus and Per >> - Cleaned up some leftovers in C2 >> - Improved C2 loop opts code by introducing another APIs in >> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now. >> - We would all very much prefer to keep ShenandoahXYZNode names, as >> noted earlier. This stuff is Shenandoah-specific, so let's just call it >> that. >> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >> - Rebased on jdk-12+22 >> >> - Question: let us know if you need separate RFE for the new BSC2 APIs? >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >> >> Thanks, >> Roman >> >>> Alright, we fixed: >>> - The minor issues that Kim reported in shared-gc >>> - A lot of fixes in shared-tests according to Leonid's review >>> - Disabled SA heapdumping similar to ZGC as Per suggested >>> >>> Some notes: >>> Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>> correct. The @requires there means to exclude runs with both CMS and >>> ExplicitGCInvokesConcurrent at the same time, because that would be >>> (expectedly) failing. It can run CMS, default GC and any other GC just >>> fine. Adding the same clause for Shenandoah means the same, and filters >>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>> made the condition a bit clearer by avoiding triple-negation. >>> >>> See: >>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>> >>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>> >>> we filter them for Shenandoah now. I'm wondering: how do you get past >>> those with ZGC? >>> >>> See: >>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>> >>> (Note to Leonid and tests reviewers: I'll add those related filters in >>> next round). >>> >>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>> better. I can tell that we're not done with C2 yet. Can you look over >>> the code and see what is ok, and especially what is not ok, so that we >>> can focus our efforts on the relevant parts? >>> >>> Updated set of webrevs: >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>> >>> Thanks, >>> Roman >>> >>> >>>> Hi, >>>> >>>> This is the first round of changes for including Shenandoah GC into >>>> mainline. >>>> I divided the review into parts that roughly correspond to the mailing lists >>>> that would normally review it, and I divided it into 'shared' code >>>> changes and >>>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>>> eventually >>>> push them as single 'combined' changeset, once reviewed. >>>> >>>> JEP: >>>> ? https://openjdk.java.net/jeps/189 >>>> Bug entry: >>>> >>>> ?https://bugs.openjdk.java.net/browse/JDK-8214259 >>>> >>>> Webrevs: >>>> ? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>> >>>> For those who want to see the full change, have a look at the >>>> shenandoah-complete >>>> >>>> directory, >>>> it contains the full combined webrev. Alternatively, there is the file >>>> shenandoah-master.patch >>>> , >>>> which is what I intend to commit (and which should be equivalent to the >>>> 'shenandoah-complete' webrev). >>>> >>>> Sections to review (at this point) are the following: >>>> ?*) shenandoah-gc >>>> >>>> ??? - Actual Shenandoah implementation, almost completely residing in >>>> gc/shenandoah >>>> >>>> ?*) shared-gc >>>> >>>> ??? - This is mostly boilerplate that is common to any GC >>>> ??? - referenceProcessor.cpp has a little change to make one assert not >>>> fail (next to CMS and G1) >>>> ??? - taskqueue.hpp has some small adjustments to enable subclassing >>>> >>>> ?*) shared-serviceability >>>> >>>> ??? - The usual code to support another GC >>>> >>>> ?*) shared-runtime >>>> >>>> ??? - A number of friends declarations to allow Shenandoah iterators to >>>> hook up with, >>>> ????? e.g. ClassLoaderData, CodeCache, etc >>>> ??? - Warning and disabling JFR LeakProfiler >>>> ??? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>> Shenandoah C2 optimizations >>>> ??? - Locks initialization in mutexLocker.cpp as usual >>>> ??? - VM operations defines for Shenandoah's VM ops >>>> ??? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>> Shenandoah's logging >>>> ??? - The usual macros in macro.hpp >>>> >>>> ?*) shared-build >>>> >>>> ??? - Add shenandoah feature, enabled by default, as agreed with >>>> Vladimir K. beforehand >>>> ??? - Some flags for shenandoah-enabled compilation to get >>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>> ????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>> Shenandoah's barriers >>>> ??? - --param inline-unit-growth=1000 settings for 2 shenandoah source >>>> files, which is >>>> ????? useful to get the whole marking loop inlined (observed significant >>>> regression if we >>>> ????? don't) >>>> >>>> ?*) shared-tests >>>> >>>> ??? - Test infrastructure to support Shenandoah >>>> ??? - Shenandoah test groups >>>> ??? - Exclude Shenandoah in various tests that can be run with selected GC >>>> ??? - Enable/add configure for Shenandoah for tests that make sense to >>>> run with it >>>> >>>> ?*) shenandoah-tests >>>> >>>> ??? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >>>> ??? - A couple of tests configurations have been added, e.g. >>>> TestGCBasherWithShenandoah.java >>>> >>>> I intentionally left out shared-compiler for now, because we have some >>>> work left to do >>>> there, but if you click around you'll find the patch anyway, in case you >>>> want to take >>>> a peek at it. >>>> >>>> We have regular builds on: >>>> ? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>> ? - {Windows} x {x86_64}, >>>> ? - {MacOS X} x {x86_64} >>>> >>>> This also routinely passes: >>>> ? - the new Shenandoah tests >>>> ? - jcstress with/without aggressive Shenandoah verification >>>> ? - specjvm2008 with/without aggressive Shenandoah verification >>>> >>>> >>>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>>> the credit for being the original inventor of Shenandoah, Aleksey >>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>> teams for tirelessly helping with and reviewing all the GC interface and >>>> related changes, and of course the many early adopters for reporting >>>> bugs and success stories and feature requests: we wouldn't be here >>>> without any of you! >>>> >>>> Best regards, >>>> Roman >>>> >>> >> > From rkennke at redhat.com Mon Dec 3 13:24:19 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 3 Dec 2018 14:24:19 +0100 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <66f45e27-9e75-42ca-2a9b-c77166d38533@oracle.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <66f45e27-9e75-42ca-2a9b-c77166d38533@oracle.com> Message-ID: <11a96945-7b8e-f30c-0e29-cd532f6addef@redhat.com> Hi Per, Thanks for looking again. I've fixed the ordering in shenandoah-dev: http://cr.openjdk.java.net/~rkennke/fix-macro-order/webrev.00/ and it will apear in the next round of webrevs. Thanks, Roman > Hi Roman, > > On 11/30/18 10:00 PM, Roman Kennke wrote: >> Hi all, >> >> here comes round 4 of Shenandoah upstreaming review: >> >> This includes fixes for the issues that Per brought up: >> - Verify and gracefully reject dangerous flags combinations that >> disables required barriers >> - Revisited @requires filters in tests >> - Trim unused code from Shenandoah's SA impl >> - Move ShenandoahGCTracer to gc/shenandoah >> - Fix ordering of GC names in various files >> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W > > Thanks for fixing. Looks good to me, except it looks like you missed > adjusting the macro order in the following files: > ?src/hotspot/share/gc/shared/gc_globals.hpp > ?src/hotspot/share/gc/shared/vmStructs_gc.hpp > > cheers, > Per > >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >> >> Thanks everybody for taking time to review this! >> Roman >> >>> Hello all, >>> >>> Thanks so far for all the reviews and support! >>> >>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >>> >>> Things we've changed today: >>> - We moved shenandoah-specific code out of .ad files into our own .ad >>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>> requires an addition in build machinery though, see >>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>> - Improved zero-disabling and build-code-simplification as suggested by >>> Magnus and Per >>> - Cleaned up some leftovers in C2 >>> - Improved C2 loop opts code by introducing another APIs in >>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards >>> now. >>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>> that. >>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>> - Rebased on jdk-12+22 >>> >>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>> >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>> >>> Thanks, >>> Roman >>> >>>> Alright, we fixed: >>>> - The minor issues that Kim reported in shared-gc >>>> - A lot of fixes in shared-tests according to Leonid's review >>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>> >>>> Some notes: >>>> Leonid:? test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>> correct. The @requires there means to exclude runs with both CMS and >>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>> fine. Adding the same clause for Shenandoah means the same, and filters >>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>> made the condition a bit clearer by avoiding triple-negation. >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>> >>>> >>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>> >>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>> those with ZGC? >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>> >>>> >>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>> next round). >>>> >>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>> better. I can tell that we're not done with C2 yet. Can you look over >>>> the code and see what is ok, and especially what is not ok, so that we >>>> can focus our efforts on the relevant parts? >>>> >>>> Updated set of webrevs: >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>> >>>> Thanks, >>>> Roman >>>> >>>> >>>>> Hi, >>>>> >>>>> This is the first round of changes for including Shenandoah GC into >>>>> mainline. >>>>> I divided the review into parts that roughly correspond to the >>>>> mailing lists >>>>> that would normally review it, and I divided it into 'shared' code >>>>> changes and >>>>> 'shenandoah' code changes (actually, mostly additions). The intend >>>>> is to >>>>> eventually >>>>> push them as single 'combined' changeset, once reviewed. >>>>> >>>>> JEP: >>>>> ?? https://openjdk.java.net/jeps/189 >>>>> Bug entry: >>>>> >>>>> ??https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>> >>>>> Webrevs: >>>>> ?? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>> >>>>> For those who want to see the full change, have a look at the >>>>> shenandoah-complete >>>>> >>>>> >>>>> directory, >>>>> it contains the full combined webrev. Alternatively, there is the file >>>>> shenandoah-master.patch >>>>> , >>>>> >>>>> which is what I intend to commit (and which should be equivalent to >>>>> the >>>>> 'shenandoah-complete' webrev). >>>>> >>>>> Sections to review (at this point) are the following: >>>>> ??*) shenandoah-gc >>>>> >>>>> >>>>> ???? - Actual Shenandoah implementation, almost completely residing in >>>>> gc/shenandoah >>>>> >>>>> ??*) shared-gc >>>>> >>>>> >>>>> ???? - This is mostly boilerplate that is common to any GC >>>>> ???? - referenceProcessor.cpp has a little change to make one >>>>> assert not >>>>> fail (next to CMS and G1) >>>>> ???? - taskqueue.hpp has some small adjustments to enable subclassing >>>>> >>>>> ??*) shared-serviceability >>>>> >>>>> >>>>> ???? - The usual code to support another GC >>>>> >>>>> ??*) shared-runtime >>>>> >>>>> >>>>> ???? - A number of friends declarations to allow Shenandoah >>>>> iterators to >>>>> hook up with, >>>>> ?????? e.g. ClassLoaderData, CodeCache, etc >>>>> ???? - Warning and disabling JFR LeakProfiler >>>>> ???? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>> Shenandoah C2 optimizations >>>>> ???? - Locks initialization in mutexLocker.cpp as usual >>>>> ???? - VM operations defines for Shenandoah's VM ops >>>>> ???? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>> Shenandoah's logging >>>>> ???? - The usual macros in macro.hpp >>>>> >>>>> ??*) shared-build >>>>> >>>>> >>>>> ???? - Add shenandoah feature, enabled by default, as agreed with >>>>> Vladimir K. beforehand >>>>> ???? - Some flags for shenandoah-enabled compilation to get >>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>> ?????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>> Shenandoah's barriers >>>>> ???? - --param inline-unit-growth=1000 settings for 2 shenandoah >>>>> source >>>>> files, which is >>>>> ?????? useful to get the whole marking loop inlined (observed >>>>> significant >>>>> regression if we >>>>> ?????? don't) >>>>> >>>>> ??*) shared-tests >>>>> >>>>> >>>>> ???? - Test infrastructure to support Shenandoah >>>>> ???? - Shenandoah test groups >>>>> ???? - Exclude Shenandoah in various tests that can be run with >>>>> selected GC >>>>> ???? - Enable/add configure for Shenandoah for tests that make >>>>> sense to >>>>> run with it >>>>> >>>>> ??*) shenandoah-tests >>>>> >>>>> >>>>> ???? - Shenandoah specific tests, most reside in gc/shenandoah >>>>> subdirectory >>>>> ???? - A couple of tests configurations have been added, e.g. >>>>> TestGCBasherWithShenandoah.java >>>>> >>>>> I intentionally left out shared-compiler for now, because we have some >>>>> work left to do >>>>> there, but if you click around you'll find the patch anyway, in >>>>> case you >>>>> want to take >>>>> a peek at it. >>>>> >>>>> We have regular builds on: >>>>> ?? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>> ?? - {Windows} x {x86_64}, >>>>> ?? - {MacOS X} x {x86_64} >>>>> >>>>> This also routinely passes: >>>>> ?? - the new Shenandoah tests >>>>> ?? - jcstress with/without aggressive Shenandoah verification >>>>> ?? - specjvm2008 with/without aggressive Shenandoah verification >>>>> >>>>> >>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she >>>>> deserves >>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>> teams for tirelessly helping with and reviewing all the GC >>>>> interface and >>>>> related changes, and of course the many early adopters for reporting >>>>> bugs and success stories and feature requests: we wouldn't be here >>>>> without any of you! >>>>> >>>>> Best regards, >>>>> Roman >>>>> >>>> >>> >> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From patricio.chilano.mateo at oracle.com Mon Dec 3 16:48:53 2018 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Mon, 3 Dec 2018 11:48:53 -0500 Subject: RFR 8214148: [TESTBUG] serviceability/tmtools/jstack/WaitNotifyThreadTest.java is not doing what is expected In-Reply-To: <1f51ea19-9d1a-916a-e1f0-428a305796f5@oracle.com> References: <57a4cb4c-87e5-bee4-192a-4f41d19e4ea5@oracle.com> <18c5ccc0-4058-b4bd-9d2a-3fbfafd5069b@oracle.com> <1f51ea19-9d1a-916a-e1f0-428a305796f5@oracle.com> Message-ID: <3b553c1c-eedf-4ecd-41a7-ff9d120fa51a@oracle.com> Hi David, On 12/3/18 2:14 AM, David Holmes wrote: > Hi Patricio, > > On 1/12/2018 2:31 am, Patricio Chilano wrote: >> Hi David, >> >> On 11/29/18 8:05 PM, David Holmes wrote: >>> Hi Patricio, >>> >>> This seems very complicated and I'm not quite seeing how it all goes >>> together. The check for waiting to re-lock now seems to dominant the >>> test and obscure the original checks. >>> >>> I'm not sure this is worthwhile in the context of this test. It >>> might be much simpler to just get rid of the existing "waiting to >>> re-lock" check which will not be seen and then if we really want to >>> check that case add a much simpler form that just checks for that. >> Ok, I actually had similar thoughts while I was adding the extra >> code. Here is the new webrev: >> >> http://cr.openjdk.java.net/~pchilanomate/8214148.02/webrev/ >> >> I removed the check for the "waiting to re-lock" case. > > Good - the more I look at this test the more I see the "waiting to > re-lock" case is just not relevant to it. > > The addition of the code to wait until the target thread is in the > right state seems good. > > I'm unclear on some of the changes to the checking code in > analyzeThreadStackWaiting. It seems you removed the code that watched > for finding the wrong monitor and replaced that with a call to > assertMonitorInfo. But the latter is passed the address you got from > the MonitorInfo in the first place so the check of the address is > never going to fail. ?? I just replaced that "if-else" with assertMonitorInfo() because the same check "monInfo.getType().equals("waiting on") && compareMonitorClass(monInfo)" will be done in assertMonitorInfo() and the error case is also doing the same, so the code gets simplified. Yes, the extra address check will never fail. > That said you can't be waiting on two monitors so I don't see how we > can ever have the wrong one ?? I'm not sure why this test is checking for the monitor address in assertMonitorInfo(). The only case where I see it could fail is for the RUN_METHOD case if the address is null, but that has a separate check before assertMonitorInfo(). Maybe at some point the test had more monitors, because there was also that "for" loop checking for the "waiting to re-lock in wait()" case. I can remove the test "monInfo.getMonitorAddress().equals(monitorAddress)" in assertMonitorInfo() but I don't think it hurts to keep it. > A few minor style nits: > > Pre-existing: > > 54???????????? //Notify the waiting thread, so it stops waiting and > sleeps > > Please add a space after // > > ?106???????? // Start athread that just waits > > s/athread/a thread/ > > ?145???????????????????? throw new RuntimeException(OBJECT_WAIT > ?146???????????????????????????? + " method has to contain one lock > record but it contains " + mi.getLocks().size()); > > Indentation is wrong - the '+' should align with the O in OBJECT. > Break into three lines (at second + if needed) > > New: > > 154???????????????? if(mi.getLocks().size() == 1){ > > Space after "if", and space before { > > ?157???????????????? else{ > > Space before { > > 158???????????????????? throw new RuntimeException(RUN_METHOD + " > method has to contain one lock record but it contains " > ?159???????????????????????????????????????????????????????????? + > mi.getLocks().size()); > > Incorrect indentation - '+' should align with R in RUN Done! >> I'm not sure if it's okay to keep the change to >> serviceability/tmtools/jstack/utils/DefaultFormat.java then. It >> doesn't really affect this test, but it is needed for jstack to >> detect the locks that appear in the stack report with the message >> "waiting to re-lock in wait()". > > I'd probably revert that change at this stage. Done! Here is the new webrev: http://cr.openjdk.java.net/~pchilanomate/8214148.03/webrev/ Thanks, Patricio > Thanks, > David > >> Thanks, >> Patricio >>> To me the simplest way to see the "re-lock in wait" case is to just: >>> >>> synchronized(obj) { >>> ?? obj.notifyAll(); >>> ?? <= take stack dump here => >>> } >>> >>> Cheers, >>> David >>> >>> On 30/11/2018 5:52 am, Daniel D. Daugherty wrote: >>>> Adding serviceability-dev at ... since the 'tmtools' including 'jstack' >>>> are owned by the Serviceability team. >>>> >>>> Dan >>>> >>>> >>>> On 11/28/18 4:21 PM, Patricio Chilano wrote: >>>>> Hi all, >>>>> >>>>> Could you review this?fix for test >>>>> serviceability/tmtools/jstack/WaitNotifyThreadTest.java? >>>>> >>>>> On one hand the test was not properly checking what it was >>>>> intended to check, since as mentioned in JBS the logic for >>>>> checking the method name was wrong. Also since there was only one >>>>> monitor in the test, the "for" loop with the message "waiting to >>>>> re-lock in wait()" was never actually reached. >>>>> >>>>> Additionally, with change 8150689 the message?"waiting to re-lock >>>>> in wait()" is now shown in the frame where the relocking is >>>>> actually taking place, so the logic for checking that should change. >>>>> >>>>> I fixed the first issues and added logic to check for the "waiting >>>>> to re-lock in wait()" case. I used the Thread.State attribute to >>>>> check desire states are reached before getting the thread dump >>>>> reports through jstack. I run the test in mach5 several times for >>>>> all platforms and they all passed. >>>>> >>>>> Webrev URL: >>>>> http://cr.openjdk.java.net/~pchilanomate/8214148.01/webrev >>>>> Bug URL: https://bugs.openjdk.java.net/browse/JDK-8214148 >>>>> >>>>> >>>>> Thanks, >>>>> Patricio >>>> >> From rkennke at redhat.com Mon Dec 3 19:27:04 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 3 Dec 2018 20:27:04 +0100 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> Message-ID: Round 5 of Shenandoah review includes: - A fix for the @requires tag in TestFullGCCountTest.java. It should be correct now. We believe the CMS @requires was also not quite right and fixed it the same. It reads now: Don't run this test if: - Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is true, as set by harness - Actual GC set by harness is Shenandoah *and* ExplicitGCInvokesConcurrent is not set false by harness (it's true by default in Shenandoah, so this needs to be double-inverteed). The @requires for CMS was wrong before (we think), because it would also filter defaultGC + ExplicitGCInvokesConcurrent. - Sorting of macros was fixed, as was pointed out by Per - Some stuff was added to SA, as suggested by Jini - Rebased on most current jdk/jdk code Webrevs: http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/ I also need reviews from GC reviewers for the CSR: https://bugs.openjdk.java.net/browse/JDK-8214349 I already got reviews for: [x] shared-runtime (coleenp) [x] shared-compiler (kvn) I got reviews for shared-build, but an earlier version, so maybe makes sense to look over this again. Erik J, Magnus? I still need approvals for: [ ] shared-build (kvn, erikj, ihse, pliden) [ ] shared-gc (pliden, kbarrett) [ ] shared-serviceability (jgeorge, pliden) [ ] shared-tests (lmesnik, pliden) [ ] shenandoah-gc [ ] shenandoah-tests Thanks for your patience and ongoing support! Cheers, Roman > Hi all, > > here comes round 4 of Shenandoah upstreaming review: > > This includes fixes for the issues that Per brought up: > - Verify and gracefully reject dangerous flags combinations that > disables required barriers > - Revisited @requires filters in tests > - Trim unused code from Shenandoah's SA impl > - Move ShenandoahGCTracer to gc/shenandoah > - Fix ordering of GC names in various files > - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W > > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ > > Thanks everybody for taking time to review this! > Roman > >> Hello all, >> >> Thanks so far for all the reviews and support! >> >> I forgot to update the 'round' yesterday. We are in round 3 now :-) >> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >> >> Things we've changed today: >> - We moved shenandoah-specific code out of .ad files into our own .ad >> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >> requires an addition in build machinery though, see >> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >> - Improved zero-disabling and build-code-simplification as suggested by >> Magnus and Per >> - Cleaned up some leftovers in C2 >> - Improved C2 loop opts code by introducing another APIs in >> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now. >> - We would all very much prefer to keep ShenandoahXYZNode names, as >> noted earlier. This stuff is Shenandoah-specific, so let's just call it >> that. >> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >> - Rebased on jdk-12+22 >> >> - Question: let us know if you need separate RFE for the new BSC2 APIs? >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >> >> Thanks, >> Roman >> >>> Alright, we fixed: >>> - The minor issues that Kim reported in shared-gc >>> - A lot of fixes in shared-tests according to Leonid's review >>> - Disabled SA heapdumping similar to ZGC as Per suggested >>> >>> Some notes: >>> Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>> correct. The @requires there means to exclude runs with both CMS and >>> ExplicitGCInvokesConcurrent at the same time, because that would be >>> (expectedly) failing. It can run CMS, default GC and any other GC just >>> fine. Adding the same clause for Shenandoah means the same, and filters >>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>> made the condition a bit clearer by avoiding triple-negation. >>> >>> See: >>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>> >>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>> >>> we filter them for Shenandoah now. I'm wondering: how do you get past >>> those with ZGC? >>> >>> See: >>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>> >>> (Note to Leonid and tests reviewers: I'll add those related filters in >>> next round). >>> >>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>> better. I can tell that we're not done with C2 yet. Can you look over >>> the code and see what is ok, and especially what is not ok, so that we >>> can focus our efforts on the relevant parts? >>> >>> Updated set of webrevs: >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>> >>> Thanks, >>> Roman >>> >>> >>>> Hi, >>>> >>>> This is the first round of changes for including Shenandoah GC into >>>> mainline. >>>> I divided the review into parts that roughly correspond to the mailing lists >>>> that would normally review it, and I divided it into 'shared' code >>>> changes and >>>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>>> eventually >>>> push them as single 'combined' changeset, once reviewed. >>>> >>>> JEP: >>>> ? https://openjdk.java.net/jeps/189 >>>> Bug entry: >>>> >>>> ?https://bugs.openjdk.java.net/browse/JDK-8214259 >>>> >>>> Webrevs: >>>> ? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>> >>>> For those who want to see the full change, have a look at the >>>> shenandoah-complete >>>> >>>> directory, >>>> it contains the full combined webrev. Alternatively, there is the file >>>> shenandoah-master.patch >>>> , >>>> which is what I intend to commit (and which should be equivalent to the >>>> 'shenandoah-complete' webrev). >>>> >>>> Sections to review (at this point) are the following: >>>> ?*) shenandoah-gc >>>> >>>> ??? - Actual Shenandoah implementation, almost completely residing in >>>> gc/shenandoah >>>> >>>> ?*) shared-gc >>>> >>>> ??? - This is mostly boilerplate that is common to any GC >>>> ??? - referenceProcessor.cpp has a little change to make one assert not >>>> fail (next to CMS and G1) >>>> ??? - taskqueue.hpp has some small adjustments to enable subclassing >>>> >>>> ?*) shared-serviceability >>>> >>>> ??? - The usual code to support another GC >>>> >>>> ?*) shared-runtime >>>> >>>> ??? - A number of friends declarations to allow Shenandoah iterators to >>>> hook up with, >>>> ????? e.g. ClassLoaderData, CodeCache, etc >>>> ??? - Warning and disabling JFR LeakProfiler >>>> ??? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>> Shenandoah C2 optimizations >>>> ??? - Locks initialization in mutexLocker.cpp as usual >>>> ??? - VM operations defines for Shenandoah's VM ops >>>> ??? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>> Shenandoah's logging >>>> ??? - The usual macros in macro.hpp >>>> >>>> ?*) shared-build >>>> >>>> ??? - Add shenandoah feature, enabled by default, as agreed with >>>> Vladimir K. beforehand >>>> ??? - Some flags for shenandoah-enabled compilation to get >>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>> ????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>> Shenandoah's barriers >>>> ??? - --param inline-unit-growth=1000 settings for 2 shenandoah source >>>> files, which is >>>> ????? useful to get the whole marking loop inlined (observed significant >>>> regression if we >>>> ????? don't) >>>> >>>> ?*) shared-tests >>>> >>>> ??? - Test infrastructure to support Shenandoah >>>> ??? - Shenandoah test groups >>>> ??? - Exclude Shenandoah in various tests that can be run with selected GC >>>> ??? - Enable/add configure for Shenandoah for tests that make sense to >>>> run with it >>>> >>>> ?*) shenandoah-tests >>>> >>>> ??? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >>>> ??? - A couple of tests configurations have been added, e.g. >>>> TestGCBasherWithShenandoah.java >>>> >>>> I intentionally left out shared-compiler for now, because we have some >>>> work left to do >>>> there, but if you click around you'll find the patch anyway, in case you >>>> want to take >>>> a peek at it. >>>> >>>> We have regular builds on: >>>> ? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>> ? - {Windows} x {x86_64}, >>>> ? - {MacOS X} x {x86_64} >>>> >>>> This also routinely passes: >>>> ? - the new Shenandoah tests >>>> ? - jcstress with/without aggressive Shenandoah verification >>>> ? - specjvm2008 with/without aggressive Shenandoah verification >>>> >>>> >>>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>>> the credit for being the original inventor of Shenandoah, Aleksey >>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>> teams for tirelessly helping with and reviewing all the GC interface and >>>> related changes, and of course the many early adopters for reporting >>>> bugs and success stories and feature requests: we wouldn't be here >>>> without any of you! >>>> >>>> Best regards, >>>> Roman >>>> >>> >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From dean.long at oracle.com Mon Dec 3 19:53:36 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Mon, 3 Dec 2018 11:53:36 -0800 Subject: RFR 8214572: nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the thread when the top frame executes JVMCI code In-Reply-To: References: <4E0EFD3E-F725-4469-BA20-7532D78BA479@oracle.com> Message-ID: On 11/30/18 7:46 PM, JC Beyler wrote: > Questions because I'm not familiar with JVMCI consequences so not > really comments on the webrev but so that I know: > ? - Is it normaly that you can suspend when you are in a JVMCI frame? Yes, because it's just Java code, and we allow all Java code to be suspended, even Graal and JVMCI code. > will/is there not a better way that we could detect that we are in a > JVMCI frame? We could check the threads's _adjusting_comp_level flag for this particular case, if we decided that we don't want to be able to debug JVMCI Java code. > Is it always safe to suspend a JVMCI frame? That's a good question.? If it was grabbing any locks, then suspending it could cause problems for other threads calling into JVMCI. Another solution would be to do adjusting_comp_level() in a separate thread.? So I think there are at least 3 possible solutions: 1) Allow JVMCI adjusting_comp_level call to be suspended/debugged 2) Don't allow it to be suspended/debugged, ??? a) by running in a separate thread, or ??? b) don't suspend when _adjusting_comp_level flag is set We could introduce a concept of "system Java" code, which, just like Unix kernel code that is not debuggable without a kernel debugger, would not normally be debuggable without setting a special flag. CCing graal-dev alias. dl From chris.plummer at oracle.com Mon Dec 3 20:09:06 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 3 Dec 2018 12:09:06 -0800 Subject: RFR: JDK-821430: .attach_pid files may remain in the process cwd In-Reply-To: <84625284-712a-52a0-9024-2ff28c7ec4e2@oracle.com> References: <5C0012B9.2060603@oracle.com> <5C0039E1.3010700@oracle.com> <95b85891-5acb-6a16-b8a8-4bfdaa142516@oracle.com> <84625284-712a-52a0-9024-2ff28c7ec4e2@oracle.com> Message-ID: <6a3cce17-3e6a-fb0f-c267-52eab9a29d0b@oracle.com> An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Mon Dec 3 20:13:13 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 3 Dec 2018 12:13:13 -0800 Subject: RFR: JDK-821430: .attach_pid files may remain in the process cwd In-Reply-To: <6a3cce17-3e6a-fb0f-c267-52eab9a29d0b@oracle.com> References: <5C0012B9.2060603@oracle.com> <5C0039E1.3010700@oracle.com> <95b85891-5acb-6a16-b8a8-4bfdaa142516@oracle.com> <84625284-712a-52a0-9024-2ff28c7ec4e2@oracle.com> <6a3cce17-3e6a-fb0f-c267-52eab9a29d0b@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Tue Dec 4 00:14:29 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 3 Dec 2018 16:14:29 -0800 Subject: RFR 8214572: nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the thread when the top frame executes JVMCI code In-Reply-To: <4E0EFD3E-F725-4469-BA20-7532D78BA479@oracle.com> References: <4E0EFD3E-F725-4469-BA20-7532D78BA479@oracle.com> Message-ID: <567a8f54-c9f1-7936-65c3-02d1c660dc70@oracle.com> An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Tue Dec 4 03:59:34 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 3 Dec 2018 19:59:34 -0800 Subject: RFR: (S): JDK-8213323: sa/TestJmapCoreMetaspace.java and sa/TestJmapCore.java fail with ZGC In-Reply-To: <9dce665e-2c1e-b5bc-c40a-6bf5b478ca82@oracle.com> References: <783746d6-07d2-4a5d-afaa-9d8d3ac08a61@oracle.com> <0732e52d-8f73-fc2d-3147-987a84039adf@oracle.com> <2b263b92-af0d-6f25-e92b-7eb814c4ead8@oracle.com> <9dce665e-2c1e-b5bc-c40a-6bf5b478ca82@oracle.com> Message-ID: <3535aaa3-0267-51c7-7431-b5d170e17972@oracle.com> Looks good. Chris On 12/2/18 8:36 PM, Jini George wrote: > Gentle reminder ! > > Thanks, > Jini > > On 11/29/2018 12:03 PM, Jini George wrote: >> Thank you very much, Chris. I have modified HeapHprofBinWriter.java >> slightly so that the hprof file does not dumped if this is ZGC. In >> the test, we check for the presence of the hprof file and throw an >> exception if the hprof file does not exist and only if this is not >> ZGC. The tests are not excluded for zgc now. >> >> The modified webrev is at: >> >> http://cr.openjdk.java.net/~jgeorge/8213323/webrev.02/index.html >> >> Thanks! >> Jini. >> >> >> On 11/27/2018 2:51 AM, Chris Plummer wrote: >>> Hi Jini, >>> >>> The tool changes look fine for disabling zgc support, but with the >>> test changes you are no longer testing what happens if you use jmap >>> with zgc. What would the tests do if you made no test changes? If >>> they still fail ungracefully, could they be fixed by catching the >>> RuntimeException you are now throwing? Maybe you could test that >>> this only happens when using zgc. >>> >>> thanks, >>> >>> Chris >>> >>> On 11/21/18 7:49 PM, Jini George wrote: >>>> Thanks a bunch, JC ! Could have a Reviewer also take a look at this >>>> please ? >>>> >>>> - Jini. >>>> >>>> On 11/21/2018 11:05 PM, JC Beyler wrote: >>>>> Hi Jini, >>>>> >>>>> Looks good to me, thanks for fixing the nits :) >>>>> >>>>> Thanks, >>>>> Jc >>>>> >>>>> >>>>> On Wed, Nov 21, 2018 at 9:29 AM Jini George >>>>> > wrote: >>>>> >>>>> ??? The modified webrev with the RuntimeException and the nit >>>>> removed is at: >>>>> >>>>> ??? http://cr.openjdk.java.net/~jgeorge/8213323/webrev.01/ >>>>> >>>>> ??? Thanks! >>>>> ??? Jini. >>>>> >>>>> ??? On 11/12/2018 12:41 PM, Jini George wrote: >>>>> ???? > Thank you very much, JC, for looking into this. >>>>> ???? > >>>>> ???? > >>>>> http://mail.openjdk.java.net/pipermail/zgc-dev/2018-August/000455.html >>>>> >>>>> ???? > >>>>> ???? > The link above provides an explanation of why it is >>>>> difficult to >>>>> ??? support >>>>> ???? > ZGC with SA where there is an iteration over the heap. And >>>>> ??? currently, I >>>>> ???? > am unsure as to whether we will devise a way to overcome this >>>>> ??? issue. We >>>>> ???? > currently have the following JBS entry for tracking this. >>>>> ???? > >>>>> ???? > https://bugs.openjdk.java.net/browse/JDK-8207843 (SA: Add >>>>> support >>>>> ??? for >>>>> ???? > Object Histogram with ZGC) >>>>> ???? > >>>>> ???? > I have added a comment in the ER above to keep track of >>>>> enabling the >>>>> ???? > tests if this is resolved. >>>>> ???? > >>>>> ???? > I agree with your comment on throwing a RuntimeException. >>>>> Doing this >>>>> ???? > avoids the misleading "heap written to" message. I will modify >>>>> ??? this and >>>>> ???? > remove the ';' you pointed out and post another webrev. >>>>> ???? > >>>>> ???? > Thank you, >>>>> ???? > Jini. >>>>> ???? > >>>>> ???? > >>>>> ???? > >>>>> ???? > On 11/9/2018 9:36 PM, JC Beyler wrote: >>>>> ???? >> Hi Jini, >>>>> ???? >> >>>>> ???? >> The webrev looks good to me as well except for a few >>>>> ??? questions/comments: >>>>> ???? >> >>>>> ???? >> I have a generic question that is related to the webrev: >>>>> ???? >> ?? - Are there plans at some point for Jmap to support ZGC >>>>> heaps in >>>>> ???? >> the future ? Or is this infeasible? >>>>> ???? >> ?? ? ? ? I ask because if a lot of tests are disabled for >>>>> ZGC for a >>>>> ???? >> limited amount of time (in order to provide time for future >>>>> ??? support) >>>>> ???? >> there should be a means to scrub the tests at a later date >>>>> to see >>>>> ???? >> which are now supported, no? >>>>> ???? >> >>>>> ???? >> ?? - In >>>>> ???? >> >>>>> http://cr.openjdk.java.net/~jgeorge/8213323/webrev.00/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/utilities/HeapHprofBinWriter.java.html: >>>>> >>>>> >>>>> ???? >> >>>>> ???? >> >>>>> ???? >> ??397? ? ? ? ?if (vm.getUniverse().heap() instanceof >>>>> ??? ZCollectedHeap) { >>>>> ???? >> ??398 ?System.err.println("WARNING: Operation not >>>>> ??? supported >>>>> ???? >> for ZGC."); >>>>> ???? >> ??399? ? ? ? ? ? ?return; >>>>> ???? >> ??400? ? ? ? ?}; >>>>> ???? >> >>>>> ???? >> ?? ? ?- 1 nit is the ';'? after the closing '}' >>>>> ???? >> ?? ? ?- Should the code throw a RuntimeException instead? >>>>> Just >>>>> ???? >> printing an error seems "weak" for me when this really won't >>>>> ??? work. Of >>>>> ???? >> course, that means tracking the callers now of write and >>>>> probably >>>>> ???? >> adding exception handling there and I'm not sure that is >>>>> something >>>>> ???? >> that is warranted here but thought I would ask :) >>>>> ???? >> >>>>> ???? >> Thanks! >>>>> ???? >> Jc >>>>> ???? >> >>>>> ???? >> >>>>> ???? >> >>>>> ???? >> Thanks, >>>>> ???? >> Jc >>>>> ???? >> >>>>> ???? >> On Fri, Nov 9, 2018 at 1:47 AM gary.adams at oracle.com >>>>> ??? >>>>> ???? >> > >>>>> ??? >>>>> ???? >> >>>> >> >>>>> ??? wrote: >>>>> ???? >> >>>>> ???? >> ??? Looks OK to me. >>>>> ???? >> >>>>> ???? >> ??? On 11/9/18 1:40 AM, Jini George wrote: >>>>> ???? >> ???? > Hello! >>>>> ???? >> ???? > >>>>> ???? >> ???? > Requesting reviews for a small change for disabling >>>>> the >>>>> ??? testing of >>>>> ???? >> ???? > TestJmapCoreMetaspace.java and TestJmapCore.java with >>>>> ??? ZGC. The >>>>> ???? >> ??? change >>>>> ???? >> ???? > also includes skipping of creating the heapdump >>>>> file with >>>>> ??? jmap if >>>>> ???? >> ??? the >>>>> ???? >> ???? > GC being used is ZGC. >>>>> ???? >> ???? > >>>>> ???? >> ???? > Webrev: >>>>> ??? http://cr.openjdk.java.net/~jgeorge/8213323/webrev.00/ >>>>> ???? >> ???? > Bug ID: >>>>> https://bugs.openjdk.java.net/browse/JDK-8213323 >>>>> ???? >> ???? > >>>>> ???? >> ???? > Thanks, >>>>> ???? >> ???? > Jini. >>>>> ???? >> >>>>> ???? >> >>>>> ???? >> >>>>> ???? >> >>>>> ???? >> -- >>>>> ???? >> >>>>> ???? >> Thanks, >>>>> ???? >> Jc >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Thanks, >>>>> Jc >>> >>> From david.holmes at oracle.com Tue Dec 4 04:06:51 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 4 Dec 2018 14:06:51 +1000 Subject: JDK-8176828: jtools do not list VM process launched with the debugger option suspend=y In-Reply-To: <5C012F30.1040005@oracle.com> References: <5BFEC34B.8050904@oracle.com> <8f71447f-5285-43fe-6f4b-094a0989bd36@oracle.com> <5BFEF214.1050405@oracle.com> <6637a974-b849-114e-3802-483e1b4d8ba5@oracle.com> <3ce00a95-2f30-761c-1ad3-08aa6fe3423e@oracle.com> <07ae6022-e820-70eb-4ea2-dea06795bd18@oracle.com> <68089bf3-1ca0-fb84-844a-ac0137bc3c62@oracle.com> <5C012F30.1040005@oracle.com> Message-ID: <4cd2879c-fe2a-fe55-7275-d72f4fb3d3df@oracle.com> Hi Gary, Somewhere in this I got the mistaken impression that it is the VMThread that blocks waiting for the debugger to attach, but it is actually the main thread executing create_vm. That simplifies things and addresses the concern I had about jcmds that needed VM ops. So IIUC the only reason we can't attach is because PerfMemory is not "accessible" so the patch below addresses that by making it accessible before the main thread would block. I've looked at the initialization flow and moving the set_accessible call seems reasonable to me. Thanks, David On 30/11/2018 10:38 pm, Gary Adams wrote: > Here's a smaller change. I'll run through a bunch of serviceability > tests to make sure > this early accessible setting does not have any unexpected side effects.. > > diff --git a/src/hotspot/share/runtime/thread.cpp > b/src/hotspot/share/runtime/thread.cpp > --- a/src/hotspot/share/runtime/thread.cpp > +++ b/src/hotspot/share/runtime/thread.cpp > @@ -3902,6 +3902,9 @@ > ?? // may be attached late and JVMTI must track phases of VM execution > ?? JvmtiExport::enter_live_phase(); > > +? // Make perfmemory accessible > +? PerfMemory::set_accessible(true); > + > ?? // Notify JVMTI agents that VM initialization is complete - nop if > no agents. > ?? JvmtiExport::post_vm_initialized(); > > On 11/30/18, 5:03 AM, gary.adams at oracle.com wrote: >> It turns out that almost all the diagnostic commands provide >> meaningful output even though the vm thread is not totally >> initialized. >> >> We could declare the perfmemory initialized a little earlier ... >> >> diff --git a/src/hotspot/share/runtime/thread.cpp >> b/src/hotspot/share/runtime/thread.cpp >> --- a/src/hotspot/share/runtime/thread.cpp >> +++ b/src/hotspot/share/runtime/thread.cpp >> @@ -3902,6 +3902,9 @@ >> ?? // may be attached late and JVMTI must track phases of VM execution >> ?? JvmtiExport::enter_live_phase(); >> >> +? // Make perfmemory accessible >> +? create_vm_timer.end(); >> + >> ?? // Notify JVMTI agents that VM initialization is complete - nop if >> no agents. >> ?? JvmtiExport::post_vm_initialized(); >> >> @@ -3951,7 +3954,6 @@ >> ???? } >> ?? } >> >> -? create_vm_timer.end(); >> ?#ifdef ASSERT >> ?? _vm_complete = true; >> ?#endif >> >> This would be the "jcmd -l" output when the post_vm_initialized() >> would block on the "suspend=y" processing >> >> 23732 jdk.jcmd/sun.tools.jcmd.JCmd -l >> 22526 Unknown >> >> ... when the debugger attaches "jdb -attach 5900" >> >> 23771 jdk.jdi/com.sun.tools.example.debug.tty.TTY -attach 5900 >> 23884 jdk.jcmd/sun.tools.jcmd.JCmd -l >> 22526 Unknown >> >> ... when the debugger starts with the "run" command >> >> 23922 jdk.jcmd/sun.tools.jcmd.JCmd -l >> 23771 jdk.jdi/com.sun.tools.example.debug.tty.TTY -attach 5900 >> 22526 my >> >> We could also pull out the perfmemory set accessible step >> and set it explicitly. >> >> ... >> >> On 11/28/18 11:10 PM, David Holmes wrote: >>> I've been lurking on this ... >>> >>> On 29/11/2018 9:41 am, gary.adams at oracle.com wrote: >>>> On 11/28/18 4:12 PM, Chris Plummer wrote: >>>>> On 11/28/18 11:52 AM, Gary Adams wrote: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8156537 >>>>>> ? - Added a fix for module naming >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8176533 >>>>>> ? - Fixed a regression from JDK-816537 that prevented "jcmd >>>>>> " >>>>>> ???? from sending to >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8176828 >>>>>> ? - Requests that "jcmd -l" should list the >>>>>> ???? for the VM that has not completed initialization. >>>>>> ???? This is not a regression, because jdk8 jcmd did >>>>>> ???? not display the uninitialized VM either. >>>>>> >>>>>> It appears you can send "jcmd " to >>>>>> the process that has not completed vm initialization. >>>>> So what happens when you do this? >>>> The commands work, albeit the vm thread is blocked waiting for the >>>> debugger connection. >>> >>> Define "work". If the VM is partially initialized there is very >>> little that can be examined. If the VMThread is blocked on the >>> debugger connection then you can't do anything that requires a VM >>> operation (which includes safepoints - though at this stage if there >>> are no JavaThreads we're safe anyway). >>> >>> Thanks, >>> David >>> >>>>> >>>>> Chris >>>>>> >>>>>> On 11/28/18, 2:29 PM, Chris Plummer wrote: >>>>>>> Sorry, meant to say "before JDK-8176533". >>>>>>> >>>>>>> Chris >>>>>>> >>>>>>> On 11/28/18 10:11 AM, Chris Plummer wrote: >>>>>>>> Hi Gary, >>>>>>>> >>>>>>>> This seems reasonable. I just want clarification on one thing >>>>>>>> first. Before JDK-8176828, it listed the suspend=y process, but >>>>>>>> could you do anything with it? I'm assuming no, and therefore it >>>>>>>> makes sense not to list it at all. >>>>>>>> >>>>>>>> thanks, >>>>>>>> >>>>>>>> Chris >>>>>>>> >>>>>>>> On 11/28/18 8:33 AM, Gary Adams wrote: >>>>>>>>> I'd like to close JDK-8176828 as will not fix. >>>>>>>>> ? https://bugs.openjdk.java.net/browse/JDK-8176828 >>>>>>>>> >>>>>>>>> This bug was originally thought to be associated with >>>>>>>>> a regression fix in JDK-8176533, but I believe there >>>>>>>>> is simply a misunderstanding of how the "suspend=y" >>>>>>>>> behavior is supported for the jdwp agent library. >>>>>>>>> >>>>>>>>> The "suspend=y" setting is designed to maximize the number >>>>>>>>> of events that can been seen by an attaching debugger. >>>>>>>>> This calls for the process to be waiting for the debugger >>>>>>>>> connection >>>>>>>>> before the VM has been completely initialized. >>>>>>>>> >>>>>>>>> In particular, the thread.cpp create_vm() function causes the >>>>>>>>> debugInit.c initialize() function to start the transport >>>>>>>>> connection >>>>>>>>> and wait for an external debugger to attach. Once the dt_socket >>>>>>>>> connection >>>>>>>>> is established the create_vm() function will complete. It is the >>>>>>>>> processing of the create_vm_timer.end() that results in the >>>>>>>>> management.cpp record_vm_startup_time() and the >>>>>>>>> PerfMemory::set_accessible(true) to be set. >>>>>>>>> >>>>>>>>> Without the accessible field being set an external program can >>>>>>>>> not use >>>>>>>>> the PerfMemory. That is why "jcmd -l" passes over processes which >>>>>>>>> do not have an initialized VM running. >>>>>>>>> >>>>>>>>> ===== >>>>>>>>> src/hotspot/share/runtime/: >>>>>>>>> src/jdk.jdwp.agent/share/native/libjdwp/: >>>>>>>>> >>>>>>>>> ? thread.cpp:3587:jint Threads::create_vm(JavaVMInitArgs* args, >>>>>>>>> bool* canTryAgain) { >>>>>>>>> ? ... >>>>>>>>> ? thread.cpp:3610:? create_vm_timer.start(); >>>>>>>>> ? ... >>>>>>>>> ? thread.cpp:3907: JvmtiExport::post_vm_initialized(); >>>>>>>>> ? ... >>>>>>>>> ??? debugInit.c:432:cbEarlyVMInit(jvmtiEnv *jvmti_env, JNIEnv >>>>>>>>> *env, jthread thread) >>>>>>>>> ??? ... >>>>>>>>> ??? debugInit.c:103:static void initialize(JNIEnv *env, jthread >>>>>>>>> thread, EventIndex triggering_ei); >>>>>>>>> ??? ... >>>>>>>>> ??? debugInit.c:732: (void)bagEnumerateOver(transports, >>>>>>>>> startTransport, &arg); >>>>>>>>> ??? ... >>>>>>>>> ??? debugInit.c:751:??? transport_waitForConnection(); >>>>>>>>> ? ... >>>>>>>>> ? thread.cpp:3955:? create_vm_timer.end(); >>>>>>>>> >>>>>>>>> ==== >>>>>>>>> src/hotspot/share/services/management.cpp: >>>>>>>>> >>>>>>>>> ?? 203? void Management::record_vm_startup_time(jlong begin, >>>>>>>>> jlong duration) { >>>>>>>>> ?? 204??? // if the performance counter is not initialized, >>>>>>>>> ?? 205??? // then vm initialization failed; simply return. >>>>>>>>> ?? 206??? if (_begin_vm_creation_time == NULL) return; >>>>>>>>> ?? 207 >>>>>>>>> ?? 208 _begin_vm_creation_time->set_value(begin); >>>>>>>>> ?? 209??? _end_vm_creation_time->set_value(begin + duration); >>>>>>>>> ?? 210??? PerfMemory::set_accessible(true); >>>>>>>>> ?? 211? } >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>> >> > From jini.george at oracle.com Tue Dec 4 04:30:01 2018 From: jini.george at oracle.com (Jini George) Date: Tue, 4 Dec 2018 10:00:01 +0530 Subject: RFR: (S): JDK-8213323: sa/TestJmapCoreMetaspace.java and sa/TestJmapCore.java fail with ZGC In-Reply-To: <3535aaa3-0267-51c7-7431-b5d170e17972@oracle.com> References: <783746d6-07d2-4a5d-afaa-9d8d3ac08a61@oracle.com> <0732e52d-8f73-fc2d-3147-987a84039adf@oracle.com> <2b263b92-af0d-6f25-e92b-7eb814c4ead8@oracle.com> <9dce665e-2c1e-b5bc-c40a-6bf5b478ca82@oracle.com> <3535aaa3-0267-51c7-7431-b5d170e17972@oracle.com> Message-ID: Thank you, Chris! - Jini On 12/4/2018 9:29 AM, Chris Plummer wrote: > Looks good. > > Chris > > On 12/2/18 8:36 PM, Jini George wrote: >> Gentle reminder ! >> >> Thanks, >> Jini >> >> On 11/29/2018 12:03 PM, Jini George wrote: >>> Thank you very much, Chris. I have modified HeapHprofBinWriter.java >>> slightly so that the hprof file does not dumped if this is ZGC. In >>> the test, we check for the presence of the hprof file and throw an >>> exception if the hprof file does not exist and only if this is not >>> ZGC. The tests are not excluded for zgc now. >>> >>> The modified webrev is at: >>> >>> http://cr.openjdk.java.net/~jgeorge/8213323/webrev.02/index.html >>> >>> Thanks! >>> Jini. >>> >>> >>> On 11/27/2018 2:51 AM, Chris Plummer wrote: >>>> Hi Jini, >>>> >>>> The tool changes look fine for disabling zgc support, but with the >>>> test changes you are no longer testing what happens if you use jmap >>>> with zgc. What would the tests do if you made no test changes? If >>>> they still fail ungracefully, could they be fixed by catching the >>>> RuntimeException you are now throwing? Maybe you could test that >>>> this only happens when using zgc. >>>> >>>> thanks, >>>> >>>> Chris >>>> >>>> On 11/21/18 7:49 PM, Jini George wrote: >>>>> Thanks a bunch, JC ! Could have a Reviewer also take a look at this >>>>> please ? >>>>> >>>>> - Jini. >>>>> >>>>> On 11/21/2018 11:05 PM, JC Beyler wrote: >>>>>> Hi Jini, >>>>>> >>>>>> Looks good to me, thanks for fixing the nits :) >>>>>> >>>>>> Thanks, >>>>>> Jc >>>>>> >>>>>> >>>>>> On Wed, Nov 21, 2018 at 9:29 AM Jini George >>>>>> > wrote: >>>>>> >>>>>> ??? The modified webrev with the RuntimeException and the nit >>>>>> removed is at: >>>>>> >>>>>> ??? http://cr.openjdk.java.net/~jgeorge/8213323/webrev.01/ >>>>>> >>>>>> ??? Thanks! >>>>>> ??? Jini. >>>>>> >>>>>> ??? On 11/12/2018 12:41 PM, Jini George wrote: >>>>>> ???? > Thank you very much, JC, for looking into this. >>>>>> ???? > >>>>>> ???? > >>>>>> http://mail.openjdk.java.net/pipermail/zgc-dev/2018-August/000455.html >>>>>> >>>>>> ???? > >>>>>> ???? > The link above provides an explanation of why it is >>>>>> difficult to >>>>>> ??? support >>>>>> ???? > ZGC with SA where there is an iteration over the heap. And >>>>>> ??? currently, I >>>>>> ???? > am unsure as to whether we will devise a way to overcome this >>>>>> ??? issue. We >>>>>> ???? > currently have the following JBS entry for tracking this. >>>>>> ???? > >>>>>> ???? > https://bugs.openjdk.java.net/browse/JDK-8207843 (SA: Add >>>>>> support >>>>>> ??? for >>>>>> ???? > Object Histogram with ZGC) >>>>>> ???? > >>>>>> ???? > I have added a comment in the ER above to keep track of >>>>>> enabling the >>>>>> ???? > tests if this is resolved. >>>>>> ???? > >>>>>> ???? > I agree with your comment on throwing a RuntimeException. >>>>>> Doing this >>>>>> ???? > avoids the misleading "heap written to" message. I will modify >>>>>> ??? this and >>>>>> ???? > remove the ';' you pointed out and post another webrev. >>>>>> ???? > >>>>>> ???? > Thank you, >>>>>> ???? > Jini. >>>>>> ???? > >>>>>> ???? > >>>>>> ???? > >>>>>> ???? > On 11/9/2018 9:36 PM, JC Beyler wrote: >>>>>> ???? >> Hi Jini, >>>>>> ???? >> >>>>>> ???? >> The webrev looks good to me as well except for a few >>>>>> ??? questions/comments: >>>>>> ???? >> >>>>>> ???? >> I have a generic question that is related to the webrev: >>>>>> ???? >> ?? - Are there plans at some point for Jmap to support ZGC >>>>>> heaps in >>>>>> ???? >> the future ? Or is this infeasible? >>>>>> ???? >> ?? ? ? ? I ask because if a lot of tests are disabled for >>>>>> ZGC for a >>>>>> ???? >> limited amount of time (in order to provide time for future >>>>>> ??? support) >>>>>> ???? >> there should be a means to scrub the tests at a later date >>>>>> to see >>>>>> ???? >> which are now supported, no? >>>>>> ???? >> >>>>>> ???? >> ?? - In >>>>>> ???? >> >>>>>> http://cr.openjdk.java.net/~jgeorge/8213323/webrev.00/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/utilities/HeapHprofBinWriter.java.html: >>>>>> >>>>>> >>>>>> ???? >> >>>>>> ???? >> >>>>>> ???? >> ??397? ? ? ? ?if (vm.getUniverse().heap() instanceof >>>>>> ??? ZCollectedHeap) { >>>>>> ???? >> ??398 ?System.err.println("WARNING: Operation not >>>>>> ??? supported >>>>>> ???? >> for ZGC."); >>>>>> ???? >> ??399? ? ? ? ? ? ?return; >>>>>> ???? >> ??400? ? ? ? ?}; >>>>>> ???? >> >>>>>> ???? >> ?? ? ?- 1 nit is the ';'? after the closing '}' >>>>>> ???? >> ?? ? ?- Should the code throw a RuntimeException instead? >>>>>> Just >>>>>> ???? >> printing an error seems "weak" for me when this really won't >>>>>> ??? work. Of >>>>>> ???? >> course, that means tracking the callers now of write and >>>>>> probably >>>>>> ???? >> adding exception handling there and I'm not sure that is >>>>>> something >>>>>> ???? >> that is warranted here but thought I would ask :) >>>>>> ???? >> >>>>>> ???? >> Thanks! >>>>>> ???? >> Jc >>>>>> ???? >> >>>>>> ???? >> >>>>>> ???? >> >>>>>> ???? >> Thanks, >>>>>> ???? >> Jc >>>>>> ???? >> >>>>>> ???? >> On Fri, Nov 9, 2018 at 1:47 AM gary.adams at oracle.com >>>>>> ??? >>>>>> ???? >> > >>>>>> ??? >>>>>> ???? >> >>>>> >> >>>>>> ??? wrote: >>>>>> ???? >> >>>>>> ???? >> ??? Looks OK to me. >>>>>> ???? >> >>>>>> ???? >> ??? On 11/9/18 1:40 AM, Jini George wrote: >>>>>> ???? >> ???? > Hello! >>>>>> ???? >> ???? > >>>>>> ???? >> ???? > Requesting reviews for a small change for disabling >>>>>> the >>>>>> ??? testing of >>>>>> ???? >> ???? > TestJmapCoreMetaspace.java and TestJmapCore.java with >>>>>> ??? ZGC. The >>>>>> ???? >> ??? change >>>>>> ???? >> ???? > also includes skipping of creating the heapdump >>>>>> file with >>>>>> ??? jmap if >>>>>> ???? >> ??? the >>>>>> ???? >> ???? > GC being used is ZGC. >>>>>> ???? >> ???? > >>>>>> ???? >> ???? > Webrev: >>>>>> ??? http://cr.openjdk.java.net/~jgeorge/8213323/webrev.00/ >>>>>> ???? >> ???? > Bug ID: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8213323 >>>>>> ???? >> ???? > >>>>>> ???? >> ???? > Thanks, >>>>>> ???? >> ???? > Jini. >>>>>> ???? >> >>>>>> ???? >> >>>>>> ???? >> >>>>>> ???? >> >>>>>> ???? >> -- >>>>>> ???? >> >>>>>> ???? >> Thanks, >>>>>> ???? >> Jc >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Thanks, >>>>>> Jc >>>> >>>> > From david.holmes at oracle.com Tue Dec 4 04:30:50 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 4 Dec 2018 14:30:50 +1000 Subject: RFR 8214148: [TESTBUG] serviceability/tmtools/jstack/WaitNotifyThreadTest.java is not doing what is expected In-Reply-To: <3b553c1c-eedf-4ecd-41a7-ff9d120fa51a@oracle.com> References: <57a4cb4c-87e5-bee4-192a-4f41d19e4ea5@oracle.com> <18c5ccc0-4058-b4bd-9d2a-3fbfafd5069b@oracle.com> <1f51ea19-9d1a-916a-e1f0-428a305796f5@oracle.com> <3b553c1c-eedf-4ecd-41a7-ff9d120fa51a@oracle.com> Message-ID: <7967f3a4-3aa9-79f7-30d8-7b7eae53aa7c@oracle.com> Looks good! Thanks, David On 4/12/2018 2:48 am, Patricio Chilano wrote: > Hi David, > > On 12/3/18 2:14 AM, David Holmes wrote: >> Hi Patricio, >> >> On 1/12/2018 2:31 am, Patricio Chilano wrote: >>> Hi David, >>> >>> On 11/29/18 8:05 PM, David Holmes wrote: >>>> Hi Patricio, >>>> >>>> This seems very complicated and I'm not quite seeing how it all goes >>>> together. The check for waiting to re-lock now seems to dominant the >>>> test and obscure the original checks. >>>> >>>> I'm not sure this is worthwhile in the context of this test. It >>>> might be much simpler to just get rid of the existing "waiting to >>>> re-lock" check which will not be seen and then if we really want to >>>> check that case add a much simpler form that just checks for that. >>> Ok, I actually had similar thoughts while I was adding the extra >>> code. Here is the new webrev: >>> >>> http://cr.openjdk.java.net/~pchilanomate/8214148.02/webrev/ >>> >>> I removed the check for the "waiting to re-lock" case. >> >> Good - the more I look at this test the more I see the "waiting to >> re-lock" case is just not relevant to it. >> >> The addition of the code to wait until the target thread is in the >> right state seems good. >> >> I'm unclear on some of the changes to the checking code in >> analyzeThreadStackWaiting. It seems you removed the code that watched >> for finding the wrong monitor and replaced that with a call to >> assertMonitorInfo. But the latter is passed the address you got from >> the MonitorInfo in the first place so the check of the address is >> never going to fail. ?? > I just replaced that "if-else" with assertMonitorInfo() because the same > check "monInfo.getType().equals("waiting on") && > compareMonitorClass(monInfo)" will be done in assertMonitorInfo() and > the error case is also doing the same, so the code gets simplified. Yes, > the extra address check will never fail. > >> That said you can't be waiting on two monitors so I don't see how we >> can ever have the wrong one ?? > I'm not sure why this test is checking for the monitor address in > assertMonitorInfo(). The only case where I see it could fail is for the > RUN_METHOD case if the address is null, but that has a separate check > before assertMonitorInfo(). Maybe at some point the test had more > monitors, because there was also that "for" loop checking for the > "waiting to re-lock in wait()" case. I can remove the test > "monInfo.getMonitorAddress().equals(monitorAddress)" in > assertMonitorInfo() but I don't think it hurts to keep it. > >> A few minor style nits: >> >> Pre-existing: >> >> 54???????????? //Notify the waiting thread, so it stops waiting and >> sleeps >> >> Please add a space after // >> >> ?106???????? // Start athread that just waits >> >> s/athread/a thread/ >> >> ?145???????????????????? throw new RuntimeException(OBJECT_WAIT >> ?146???????????????????????????? + " method has to contain one lock >> record but it contains " + mi.getLocks().size()); >> >> Indentation is wrong - the '+' should align with the O in OBJECT. >> Break into three lines (at second + if needed) >> >> New: >> >> 154???????????????? if(mi.getLocks().size() == 1){ >> >> Space after "if", and space before { >> >> ?157???????????????? else{ >> >> Space before { >> >> 158???????????????????? throw new RuntimeException(RUN_METHOD + " >> method has to contain one lock record but it contains " >> ?159???????????????????????????????????????????????????????????? + >> mi.getLocks().size()); >> >> Incorrect indentation - '+' should align with R in RUN > Done! >>> I'm not sure if it's okay to keep the change to >>> serviceability/tmtools/jstack/utils/DefaultFormat.java then. It >>> doesn't really affect this test, but it is needed for jstack to >>> detect the locks that appear in the stack report with the message >>> "waiting to re-lock in wait()". >> >> I'd probably revert that change at this stage. > Done! > > Here is the new webrev: > > http://cr.openjdk.java.net/~pchilanomate/8214148.03/webrev/ > > > Thanks, > Patricio >> Thanks, >> David >> >>> Thanks, >>> Patricio >>>> To me the simplest way to see the "re-lock in wait" case is to just: >>>> >>>> synchronized(obj) { >>>> ?? obj.notifyAll(); >>>> ?? <= take stack dump here => >>>> } >>>> >>>> Cheers, >>>> David >>>> >>>> On 30/11/2018 5:52 am, Daniel D. Daugherty wrote: >>>>> Adding serviceability-dev at ... since the 'tmtools' including 'jstack' >>>>> are owned by the Serviceability team. >>>>> >>>>> Dan >>>>> >>>>> >>>>> On 11/28/18 4:21 PM, Patricio Chilano wrote: >>>>>> Hi all, >>>>>> >>>>>> Could you review this?fix for test >>>>>> serviceability/tmtools/jstack/WaitNotifyThreadTest.java? >>>>>> >>>>>> On one hand the test was not properly checking what it was >>>>>> intended to check, since as mentioned in JBS the logic for >>>>>> checking the method name was wrong. Also since there was only one >>>>>> monitor in the test, the "for" loop with the message "waiting to >>>>>> re-lock in wait()" was never actually reached. >>>>>> >>>>>> Additionally, with change 8150689 the message?"waiting to re-lock >>>>>> in wait()" is now shown in the frame where the relocking is >>>>>> actually taking place, so the logic for checking that should change. >>>>>> >>>>>> I fixed the first issues and added logic to check for the "waiting >>>>>> to re-lock in wait()" case. I used the Thread.State attribute to >>>>>> check desire states are reached before getting the thread dump >>>>>> reports through jstack. I run the test in mach5 several times for >>>>>> all platforms and they all passed. >>>>>> >>>>>> Webrev URL: >>>>>> http://cr.openjdk.java.net/~pchilanomate/8214148.01/webrev >>>>>> Bug URL: https://bugs.openjdk.java.net/browse/JDK-8214148 >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Patricio >>>>> >>> > From david.holmes at oracle.com Tue Dec 4 04:45:47 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 4 Dec 2018 14:45:47 +1000 Subject: RFR 8214572: nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the thread when the top frame executes JVMCI code In-Reply-To: References: <4E0EFD3E-F725-4469-BA20-7532D78BA479@oracle.com> Message-ID: <1a500fc6-9918-2b19-a0fe-bc299e58a015@oracle.com> On 4/12/2018 5:53 am, dean.long at oracle.com wrote: > On 11/30/18 7:46 PM, JC Beyler wrote: >> Questions because I'm not familiar with JVMCI consequences so not >> really comments on the webrev but so that I know: >> ? - Is it normaly that you can suspend when you are in a JVMCI frame? > > Yes, because it's just Java code, and we allow all Java code to be > suspended, even Graal and JVMCI code. A choice which can be argued for and against. On the one hand it is nice to be able to try to debug JVMCI code, and on the other this injects execution of Java code into places which to date could not execute Java code and so can "shift" debugging actions from the application/test code, to the JVMCI code. Arguably the application/test code may need to have been more specific about its intent (ie verifying that the debuggee is suspended in the expected frame) and has just "been lucky" but nevertheless the use of JVMCI may disrupt existing code using these facilities. >> will/is there not a better way that we could detect that we are in a >> JVMCI frame? > > We could check the threads's _adjusting_comp_level flag for this > particular case, if we decided that we don't want to be able to debug > JVMCI Java code. > >> Is it always safe to suspend a JVMCI frame? > > That's a good question.? If it was grabbing any locks, then suspending > it could cause problems for other threads calling into JVMCI. > > Another solution would be to do adjusting_comp_level() in a separate > thread.? So I think there are at least 3 possible solutions: > > 1) Allow JVMCI adjusting_comp_level call to be suspended/debugged > 2) Don't allow it to be suspended/debugged, > ??? a) by running in a separate thread, or > ??? b) don't suspend when _adjusting_comp_level flag is set > > We could introduce a concept of "system Java" code, which, just like > Unix kernel code that is not debuggable without a kernel debugger, would > not normally be debuggable without setting a special flag. That may be something to consider in the future (albeit something that should IMHO have been considered well in the past!) but I think it's out of scope for this particular issue if we want to address this in 12. There's certainly a need, again IMHO, for a broader discussion as to how VM services written in Java should interact with other platform services intended for use with application and library code. I don't know if JVMCI/Graal explicitly hide themselves from agents and retransformation/redefinition/ClassFileLoadHook, or even basic things like event generation (where JVMCI may now generate many more events than previously encountered). > CCing graal-dev alias. As a non-subscriber my reply will get held for moderation. Cheers, David ----- > dl From serguei.spitsyn at oracle.com Tue Dec 4 06:32:03 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 3 Dec 2018 22:32:03 -0800 Subject: RFR 8214572: nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the thread when the top frame executes JVMCI code In-Reply-To: <1a500fc6-9918-2b19-a0fe-bc299e58a015@oracle.com> References: <4E0EFD3E-F725-4469-BA20-7532D78BA479@oracle.com> <1a500fc6-9918-2b19-a0fe-bc299e58a015@oracle.com> Message-ID: Hi David and Dean, One option is to add a command line option (disabled by default) to enable debugging/profiling of the Graal compiler. This will help to avoid all these Graal related issues, simplify the development and stabilize the tests. Not sure the Graal developer will like this proposal though. :) Also, it is not very clear what level of complexity we add with this. For instance, we will have to identify all spots where new checks have to be added. Thanks, Serguei On 12/3/18 20:45, David Holmes wrote: > On 4/12/2018 5:53 am, dean.long at oracle.com wrote: >> On 11/30/18 7:46 PM, JC Beyler wrote: >>> Questions because I'm not familiar with JVMCI consequences so not >>> really comments on the webrev but so that I know: >>> ? - Is it normaly that you can suspend when you are in a JVMCI frame? >> >> Yes, because it's just Java code, and we allow all Java code to be >> suspended, even Graal and JVMCI code. > > A choice which can be argued for and against. On the one hand it is > nice to be able to try to debug JVMCI code, and on the other this > injects execution of Java code into places which to date could not > execute Java code and so can "shift" debugging actions from the > application/test code, to the JVMCI code. Arguably the > application/test code may need to have been more specific about its > intent (ie verifying that the debuggee is suspended in the expected > frame) and has just "been lucky" but nevertheless the use of JVMCI may > disrupt existing code using these facilities. > >>> will/is there not a better way that we could detect that we are in a >>> JVMCI frame? >> >> We could check the threads's _adjusting_comp_level flag for this >> particular case, if we decided that we don't want to be able to debug >> JVMCI Java code. >> >>> Is it always safe to suspend a JVMCI frame? >> >> That's a good question.? If it was grabbing any locks, then >> suspending it could cause problems for other threads calling into JVMCI. >> >> Another solution would be to do adjusting_comp_level() in a separate >> thread.? So I think there are at least 3 possible solutions: >> >> 1) Allow JVMCI adjusting_comp_level call to be suspended/debugged >> 2) Don't allow it to be suspended/debugged, >> ???? a) by running in a separate thread, or >> ???? b) don't suspend when _adjusting_comp_level flag is set >> >> We could introduce a concept of "system Java" code, which, just like >> Unix kernel code that is not debuggable without a kernel debugger, >> would not normally be debuggable without setting a special flag. > > That may be something to consider in the future (albeit something that > should IMHO have been considered well in the past!) but I think it's > out of scope for this particular issue if we want to address this in > 12. There's certainly a need, again IMHO, for a broader discussion as > to how VM services written in Java should interact with other platform > services intended for use with application and library code. I don't > know if JVMCI/Graal explicitly hide themselves from agents and > retransformation/redefinition/ClassFileLoadHook, or even basic things > like event generation (where JVMCI may now generate many more events > than previously encountered). > >> CCing graal-dev alias. > > As a non-subscriber my reply will get held for moderation. > > Cheers, > David > ----- > >> dl From rkennke at redhat.com Tue Dec 4 07:10:28 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 4 Dec 2018 08:10:28 +0100 Subject: RFR (round 5), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> Message-ID: <8c267450-9ca8-79a6-0e4a-39b3b68af2e3@redhat.com> Round 5 of Shenandoah review includes: - A fix for the @requires tag in TestFullGCCountTest.java. It should be correct now. We believe the CMS @requires was also not quite right and fixed it the same. It reads now: Don't run this test if: - Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is true, as set by harness - Actual GC set by harness is Shenandoah *and* ExplicitGCInvokesConcurrent is not set false by harness (it's true by default in Shenandoah, so this needs to be double-inverteed). The @requires for CMS was wrong before (we think), because it would also filter defaultGC + ExplicitGCInvokesConcurrent. - Sorting of macros was fixed, as was pointed out by Per - Some stuff was added to SA, as suggested by Jini - Rebased on most current jdk/jdk code Webrevs: http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/ I also need reviews from GC reviewers for the CSR: https://bugs.openjdk.java.net/browse/JDK-8214349 I already got reviews for: [x] shared-runtime (coleenp) [x] shared-compiler (kvn) I got reviews for shared-build, but an earlier version, so maybe makes sense to look over this again. Erik J, Magnus? I still need approvals for: [ ] shared-build (kvn, erikj, ihse, pliden) [ ] shared-gc (pliden, kbarrett) [ ] shared-serviceability (jgeorge, pliden) [ ] shared-tests (lmesnik, pliden) [ ] shenandoah-gc [ ] shenandoah-tests Thanks for your patience and ongoing support! Cheers, Roman > Hi all, > > here comes round 4 of Shenandoah upstreaming review: > > This includes fixes for the issues that Per brought up: > - Verify and gracefully reject dangerous flags combinations that > disables required barriers > - Revisited @requires filters in tests > - Trim unused code from Shenandoah's SA impl > - Move ShenandoahGCTracer to gc/shenandoah > - Fix ordering of GC names in various files > - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W > > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ > > Thanks everybody for taking time to review this! > Roman > >> Hello all, >> >> Thanks so far for all the reviews and support! >> >> I forgot to update the 'round' yesterday. We are in round 3 now :-) >> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >> >> Things we've changed today: >> - We moved shenandoah-specific code out of .ad files into our own .ad >> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >> requires an addition in build machinery though, see >> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >> - Improved zero-disabling and build-code-simplification as suggested by >> Magnus and Per >> - Cleaned up some leftovers in C2 >> - Improved C2 loop opts code by introducing another APIs in >> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now. >> - We would all very much prefer to keep ShenandoahXYZNode names, as >> noted earlier. This stuff is Shenandoah-specific, so let's just call it >> that. >> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >> - Rebased on jdk-12+22 >> >> - Question: let us know if you need separate RFE for the new BSC2 APIs? >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >> >> Thanks, >> Roman >> >>> Alright, we fixed: >>> - The minor issues that Kim reported in shared-gc >>> - A lot of fixes in shared-tests according to Leonid's review >>> - Disabled SA heapdumping similar to ZGC as Per suggested >>> >>> Some notes: >>> Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>> correct. The @requires there means to exclude runs with both CMS and >>> ExplicitGCInvokesConcurrent at the same time, because that would be >>> (expectedly) failing. It can run CMS, default GC and any other GC just >>> fine. Adding the same clause for Shenandoah means the same, and filters >>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>> made the condition a bit clearer by avoiding triple-negation. >>> >>> See: >>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>> >>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>> >>> we filter them for Shenandoah now. I'm wondering: how do you get past >>> those with ZGC? >>> >>> See: >>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>> >>> (Note to Leonid and tests reviewers: I'll add those related filters in >>> next round). >>> >>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>> better. I can tell that we're not done with C2 yet. Can you look over >>> the code and see what is ok, and especially what is not ok, so that we >>> can focus our efforts on the relevant parts? >>> >>> Updated set of webrevs: >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>> >>> Thanks, >>> Roman >>> >>> >>>> Hi, >>>> >>>> This is the first round of changes for including Shenandoah GC into >>>> mainline. >>>> I divided the review into parts that roughly correspond to the mailing lists >>>> that would normally review it, and I divided it into 'shared' code >>>> changes and >>>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>>> eventually >>>> push them as single 'combined' changeset, once reviewed. >>>> >>>> JEP: >>>> ? https://openjdk.java.net/jeps/189 >>>> Bug entry: >>>> >>>> ?https://bugs.openjdk.java.net/browse/JDK-8214259 >>>> >>>> Webrevs: >>>> ? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>> >>>> For those who want to see the full change, have a look at the >>>> shenandoah-complete >>>> >>>> directory, >>>> it contains the full combined webrev. Alternatively, there is the file >>>> shenandoah-master.patch >>>> , >>>> which is what I intend to commit (and which should be equivalent to the >>>> 'shenandoah-complete' webrev). >>>> >>>> Sections to review (at this point) are the following: >>>> ?*) shenandoah-gc >>>> >>>> ??? - Actual Shenandoah implementation, almost completely residing in >>>> gc/shenandoah >>>> >>>> ?*) shared-gc >>>> >>>> ??? - This is mostly boilerplate that is common to any GC >>>> ??? - referenceProcessor.cpp has a little change to make one assert not >>>> fail (next to CMS and G1) >>>> ??? - taskqueue.hpp has some small adjustments to enable subclassing >>>> >>>> ?*) shared-serviceability >>>> >>>> ??? - The usual code to support another GC >>>> >>>> ?*) shared-runtime >>>> >>>> ??? - A number of friends declarations to allow Shenandoah iterators to >>>> hook up with, >>>> ????? e.g. ClassLoaderData, CodeCache, etc >>>> ??? - Warning and disabling JFR LeakProfiler >>>> ??? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>> Shenandoah C2 optimizations >>>> ??? - Locks initialization in mutexLocker.cpp as usual >>>> ??? - VM operations defines for Shenandoah's VM ops >>>> ??? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>> Shenandoah's logging >>>> ??? - The usual macros in macro.hpp >>>> >>>> ?*) shared-build >>>> >>>> ??? - Add shenandoah feature, enabled by default, as agreed with >>>> Vladimir K. beforehand >>>> ??? - Some flags for shenandoah-enabled compilation to get >>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>> ????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>> Shenandoah's barriers >>>> ??? - --param inline-unit-growth=1000 settings for 2 shenandoah source >>>> files, which is >>>> ????? useful to get the whole marking loop inlined (observed significant >>>> regression if we >>>> ????? don't) >>>> >>>> ?*) shared-tests >>>> >>>> ??? - Test infrastructure to support Shenandoah >>>> ??? - Shenandoah test groups >>>> ??? - Exclude Shenandoah in various tests that can be run with selected GC >>>> ??? - Enable/add configure for Shenandoah for tests that make sense to >>>> run with it >>>> >>>> ?*) shenandoah-tests >>>> >>>> ??? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >>>> ??? - A couple of tests configurations have been added, e.g. >>>> TestGCBasherWithShenandoah.java >>>> >>>> I intentionally left out shared-compiler for now, because we have some >>>> work left to do >>>> there, but if you click around you'll find the patch anyway, in case you >>>> want to take >>>> a peek at it. >>>> >>>> We have regular builds on: >>>> ? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>> ? - {Windows} x {x86_64}, >>>> ? - {MacOS X} x {x86_64} >>>> >>>> This also routinely passes: >>>> ? - the new Shenandoah tests >>>> ? - jcstress with/without aggressive Shenandoah verification >>>> ? - specjvm2008 with/without aggressive Shenandoah verification >>>> >>>> >>>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>>> the credit for being the original inventor of Shenandoah, Aleksey >>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>> teams for tirelessly helping with and reviewing all the GC interface and >>>> related changes, and of course the many early adopters for reporting >>>> bugs and success stories and feature requests: we wouldn't be here >>>> without any of you! >>>> >>>> Best regards, >>>> Roman >>>> >>> >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From magnus.ihse.bursie at oracle.com Tue Dec 4 07:14:25 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 4 Dec 2018 08:14:25 +0100 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> Message-ID: > 3 dec. 2018 kl. 20:27 skrev Roman Kennke : > > Round 5 of Shenandoah review includes: > - A fix for the @requires tag in TestFullGCCountTest.java. It should be > correct now. We believe the CMS @requires was also not quite right and > fixed it the same. > > It reads now: Don't run this test if: > - Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is > true, as set by harness > - Actual GC set by harness is Shenandoah *and* > ExplicitGCInvokesConcurrent is not set false by harness (it's true by > default in Shenandoah, so this needs to be double-inverteed). > > The @requires for CMS was wrong before (we think), because it would also > filter defaultGC + ExplicitGCInvokesConcurrent. > > - Sorting of macros was fixed, as was pointed out by Per > - Some stuff was added to SA, as suggested by Jini > - Rebased on most current jdk/jdk code > > Webrevs: > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/ > > I also need reviews from GC reviewers for the CSR: > https://bugs.openjdk.java.net/browse/JDK-8214349 > > I already got reviews for: > [x] shared-runtime (coleenp) > [x] shared-compiler (kvn) > > I got reviews for shared-build, but an earlier version, so maybe makes > sense to look over this again. Erik J, Magnus? Build changes look good. /Magnus > > I still need approvals for: > [ ] shared-build (kvn, erikj, ihse, pliden) > [ ] shared-gc (pliden, kbarrett) > [ ] shared-serviceability (jgeorge, pliden) > [ ] shared-tests (lmesnik, pliden) > [ ] shenandoah-gc > [ ] shenandoah-tests > > > Thanks for your patience and ongoing support! > > Cheers, > Roman > >> Hi all, >> >> here comes round 4 of Shenandoah upstreaming review: >> >> This includes fixes for the issues that Per brought up: >> - Verify and gracefully reject dangerous flags combinations that >> disables required barriers >> - Revisited @requires filters in tests >> - Trim unused code from Shenandoah's SA impl >> - Move ShenandoahGCTracer to gc/shenandoah >> - Fix ordering of GC names in various files >> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >> >> Thanks everybody for taking time to review this! >> Roman >> >>> Hello all, >>> >>> Thanks so far for all the reviews and support! >>> >>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >>> >>> Things we've changed today: >>> - We moved shenandoah-specific code out of .ad files into our own .ad >>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>> requires an addition in build machinery though, see >>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>> - Improved zero-disabling and build-code-simplification as suggested by >>> Magnus and Per >>> - Cleaned up some leftovers in C2 >>> - Improved C2 loop opts code by introducing another APIs in >>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now. >>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>> that. >>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>> - Rebased on jdk-12+22 >>> >>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>> >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>> >>> Thanks, >>> Roman >>> >>>> Alright, we fixed: >>>> - The minor issues that Kim reported in shared-gc >>>> - A lot of fixes in shared-tests according to Leonid's review >>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>> >>>> Some notes: >>>> Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>> correct. The @requires there means to exclude runs with both CMS and >>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>> fine. Adding the same clause for Shenandoah means the same, and filters >>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>> made the condition a bit clearer by avoiding triple-negation. >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>> >>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>> >>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>> those with ZGC? >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>> >>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>> next round). >>>> >>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>> better. I can tell that we're not done with C2 yet. Can you look over >>>> the code and see what is ok, and especially what is not ok, so that we >>>> can focus our efforts on the relevant parts? >>>> >>>> Updated set of webrevs: >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>> >>>> Thanks, >>>> Roman >>>> >>>> >>>>> Hi, >>>>> >>>>> This is the first round of changes for including Shenandoah GC into >>>>> mainline. >>>>> I divided the review into parts that roughly correspond to the mailing lists >>>>> that would normally review it, and I divided it into 'shared' code >>>>> changes and >>>>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>>>> eventually >>>>> push them as single 'combined' changeset, once reviewed. >>>>> >>>>> JEP: >>>>> https://openjdk.java.net/jeps/189 >>>>> Bug entry: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>> >>>>> Webrevs: >>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>> >>>>> For those who want to see the full change, have a look at the >>>>> shenandoah-complete >>>>> >>>>> directory, >>>>> it contains the full combined webrev. Alternatively, there is the file >>>>> shenandoah-master.patch >>>>> , >>>>> which is what I intend to commit (and which should be equivalent to the >>>>> 'shenandoah-complete' webrev). >>>>> >>>>> Sections to review (at this point) are the following: >>>>> *) shenandoah-gc >>>>> >>>>> - Actual Shenandoah implementation, almost completely residing in >>>>> gc/shenandoah >>>>> >>>>> *) shared-gc >>>>> >>>>> - This is mostly boilerplate that is common to any GC >>>>> - referenceProcessor.cpp has a little change to make one assert not >>>>> fail (next to CMS and G1) >>>>> - taskqueue.hpp has some small adjustments to enable subclassing >>>>> >>>>> *) shared-serviceability >>>>> >>>>> - The usual code to support another GC >>>>> >>>>> *) shared-runtime >>>>> >>>>> - A number of friends declarations to allow Shenandoah iterators to >>>>> hook up with, >>>>> e.g. ClassLoaderData, CodeCache, etc >>>>> - Warning and disabling JFR LeakProfiler >>>>> - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>> Shenandoah C2 optimizations >>>>> - Locks initialization in mutexLocker.cpp as usual >>>>> - VM operations defines for Shenandoah's VM ops >>>>> - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>> Shenandoah's logging >>>>> - The usual macros in macro.hpp >>>>> >>>>> *) shared-build >>>>> >>>>> - Add shenandoah feature, enabled by default, as agreed with >>>>> Vladimir K. beforehand >>>>> - Some flags for shenandoah-enabled compilation to get >>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>> and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>> Shenandoah's barriers >>>>> - --param inline-unit-growth=1000 settings for 2 shenandoah source >>>>> files, which is >>>>> useful to get the whole marking loop inlined (observed significant >>>>> regression if we >>>>> don't) >>>>> >>>>> *) shared-tests >>>>> >>>>> - Test infrastructure to support Shenandoah >>>>> - Shenandoah test groups >>>>> - Exclude Shenandoah in various tests that can be run with selected GC >>>>> - Enable/add configure for Shenandoah for tests that make sense to >>>>> run with it >>>>> >>>>> *) shenandoah-tests >>>>> >>>>> - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >>>>> - A couple of tests configurations have been added, e.g. >>>>> TestGCBasherWithShenandoah.java >>>>> >>>>> I intentionally left out shared-compiler for now, because we have some >>>>> work left to do >>>>> there, but if you click around you'll find the patch anyway, in case you >>>>> want to take >>>>> a peek at it. >>>>> >>>>> We have regular builds on: >>>>> - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>> - {Windows} x {x86_64}, >>>>> - {MacOS X} x {x86_64} >>>>> >>>>> This also routinely passes: >>>>> - the new Shenandoah tests >>>>> - jcstress with/without aggressive Shenandoah verification >>>>> - specjvm2008 with/without aggressive Shenandoah verification >>>>> >>>>> >>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>> teams for tirelessly helping with and reviewing all the GC interface and >>>>> related changes, and of course the many early adopters for reporting >>>>> bugs and success stories and feature requests: we wouldn't be here >>>>> without any of you! >>>>> >>>>> Best regards, >>>>> Roman > From jini.george at oracle.com Tue Dec 4 07:23:55 2018 From: jini.george at oracle.com (Jini George) Date: Tue, 4 Dec 2018 12:53:55 +0530 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> Message-ID: Hi Roman, Thank you for making the changes. The SA portion looks good to me. One nit though: In src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/HeapSummary.java, in printGCAlgorithm(), does displaying the nbr of Parallel GC threads not make sense for Shenandoah (like it is for G1, ZGC, etc)? Thank you, Jini. On 12/4/2018 12:57 AM, Roman Kennke wrote: > Round 5 of Shenandoah review includes: > - A fix for the @requires tag in TestFullGCCountTest.java. It should be > correct now. We believe the CMS @requires was also not quite right and > fixed it the same. > > It reads now: Don't run this test if: > - Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is > true, as set by harness > - Actual GC set by harness is Shenandoah *and* > ExplicitGCInvokesConcurrent is not set false by harness (it's true by > default in Shenandoah, so this needs to be double-inverteed). > > The @requires for CMS was wrong before (we think), because it would also > filter defaultGC + ExplicitGCInvokesConcurrent. > > - Sorting of macros was fixed, as was pointed out by Per > - Some stuff was added to SA, as suggested by Jini > - Rebased on most current jdk/jdk code > > Webrevs: > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/ > > I also need reviews from GC reviewers for the CSR: > https://bugs.openjdk.java.net/browse/JDK-8214349 > > I already got reviews for: > [x] shared-runtime (coleenp) > [x] shared-compiler (kvn) > > I got reviews for shared-build, but an earlier version, so maybe makes > sense to look over this again. Erik J, Magnus? > > I still need approvals for: > [ ] shared-build (kvn, erikj, ihse, pliden) > [ ] shared-gc (pliden, kbarrett) > [ ] shared-serviceability (jgeorge, pliden) > [ ] shared-tests (lmesnik, pliden) > [ ] shenandoah-gc > [ ] shenandoah-tests > > > Thanks for your patience and ongoing support! > > Cheers, > Roman > >> Hi all, >> >> here comes round 4 of Shenandoah upstreaming review: >> >> This includes fixes for the issues that Per brought up: >> - Verify and gracefully reject dangerous flags combinations that >> disables required barriers >> - Revisited @requires filters in tests >> - Trim unused code from Shenandoah's SA impl >> - Move ShenandoahGCTracer to gc/shenandoah >> - Fix ordering of GC names in various files >> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >> >> Thanks everybody for taking time to review this! >> Roman >> >>> Hello all, >>> >>> Thanks so far for all the reviews and support! >>> >>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >>> >>> Things we've changed today: >>> - We moved shenandoah-specific code out of .ad files into our own .ad >>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>> requires an addition in build machinery though, see >>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>> - Improved zero-disabling and build-code-simplification as suggested by >>> Magnus and Per >>> - Cleaned up some leftovers in C2 >>> - Improved C2 loop opts code by introducing another APIs in >>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now. >>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>> that. >>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>> - Rebased on jdk-12+22 >>> >>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>> >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>> >>> Thanks, >>> Roman >>> >>>> Alright, we fixed: >>>> - The minor issues that Kim reported in shared-gc >>>> - A lot of fixes in shared-tests according to Leonid's review >>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>> >>>> Some notes: >>>> Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>> correct. The @requires there means to exclude runs with both CMS and >>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>> fine. Adding the same clause for Shenandoah means the same, and filters >>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>> made the condition a bit clearer by avoiding triple-negation. >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>> >>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>> >>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>> those with ZGC? >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>> >>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>> next round). >>>> >>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>> better. I can tell that we're not done with C2 yet. Can you look over >>>> the code and see what is ok, and especially what is not ok, so that we >>>> can focus our efforts on the relevant parts? >>>> >>>> Updated set of webrevs: >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>> >>>> Thanks, >>>> Roman >>>> >>>> >>>>> Hi, >>>>> >>>>> This is the first round of changes for including Shenandoah GC into >>>>> mainline. >>>>> I divided the review into parts that roughly correspond to the mailing lists >>>>> that would normally review it, and I divided it into 'shared' code >>>>> changes and >>>>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>>>> eventually >>>>> push them as single 'combined' changeset, once reviewed. >>>>> >>>>> JEP: >>>>> ? https://openjdk.java.net/jeps/189 >>>>> Bug entry: >>>>> >>>>> ?https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>> >>>>> Webrevs: >>>>> ? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>> >>>>> For those who want to see the full change, have a look at the >>>>> shenandoah-complete >>>>> >>>>> directory, >>>>> it contains the full combined webrev. Alternatively, there is the file >>>>> shenandoah-master.patch >>>>> , >>>>> which is what I intend to commit (and which should be equivalent to the >>>>> 'shenandoah-complete' webrev). >>>>> >>>>> Sections to review (at this point) are the following: >>>>> ?*) shenandoah-gc >>>>> >>>>> ??? - Actual Shenandoah implementation, almost completely residing in >>>>> gc/shenandoah >>>>> >>>>> ?*) shared-gc >>>>> >>>>> ??? - This is mostly boilerplate that is common to any GC >>>>> ??? - referenceProcessor.cpp has a little change to make one assert not >>>>> fail (next to CMS and G1) >>>>> ??? - taskqueue.hpp has some small adjustments to enable subclassing >>>>> >>>>> ?*) shared-serviceability >>>>> >>>>> ??? - The usual code to support another GC >>>>> >>>>> ?*) shared-runtime >>>>> >>>>> ??? - A number of friends declarations to allow Shenandoah iterators to >>>>> hook up with, >>>>> ????? e.g. ClassLoaderData, CodeCache, etc >>>>> ??? - Warning and disabling JFR LeakProfiler >>>>> ??? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>> Shenandoah C2 optimizations >>>>> ??? - Locks initialization in mutexLocker.cpp as usual >>>>> ??? - VM operations defines for Shenandoah's VM ops >>>>> ??? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>> Shenandoah's logging >>>>> ??? - The usual macros in macro.hpp >>>>> >>>>> ?*) shared-build >>>>> >>>>> ??? - Add shenandoah feature, enabled by default, as agreed with >>>>> Vladimir K. beforehand >>>>> ??? - Some flags for shenandoah-enabled compilation to get >>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>> ????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>> Shenandoah's barriers >>>>> ??? - --param inline-unit-growth=1000 settings for 2 shenandoah source >>>>> files, which is >>>>> ????? useful to get the whole marking loop inlined (observed significant >>>>> regression if we >>>>> ????? don't) >>>>> >>>>> ?*) shared-tests >>>>> >>>>> ??? - Test infrastructure to support Shenandoah >>>>> ??? - Shenandoah test groups >>>>> ??? - Exclude Shenandoah in various tests that can be run with selected GC >>>>> ??? - Enable/add configure for Shenandoah for tests that make sense to >>>>> run with it >>>>> >>>>> ?*) shenandoah-tests >>>>> >>>>> ??? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >>>>> ??? - A couple of tests configurations have been added, e.g. >>>>> TestGCBasherWithShenandoah.java >>>>> >>>>> I intentionally left out shared-compiler for now, because we have some >>>>> work left to do >>>>> there, but if you click around you'll find the patch anyway, in case you >>>>> want to take >>>>> a peek at it. >>>>> >>>>> We have regular builds on: >>>>> ? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>> ? - {Windows} x {x86_64}, >>>>> ? - {MacOS X} x {x86_64} >>>>> >>>>> This also routinely passes: >>>>> ? - the new Shenandoah tests >>>>> ? - jcstress with/without aggressive Shenandoah verification >>>>> ? - specjvm2008 with/without aggressive Shenandoah verification >>>>> >>>>> >>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>> teams for tirelessly helping with and reviewing all the GC interface and >>>>> related changes, and of course the many early adopters for reporting >>>>> bugs and success stories and feature requests: we wouldn't be here >>>>> without any of you! >>>>> >>>>> Best regards, >>>>> Roman >>>>> >>>> >>> >> > From david.holmes at oracle.com Tue Dec 4 07:32:20 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 4 Dec 2018 17:32:20 +1000 Subject: RFR 8214572: nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the thread when the top frame executes JVMCI code In-Reply-To: References: <4E0EFD3E-F725-4469-BA20-7532D78BA479@oracle.com> <1a500fc6-9918-2b19-a0fe-bc299e58a015@oracle.com> Message-ID: <25e1102b-cce0-206d-b4bd-c11d72a6d4d9@oracle.com> On 4/12/2018 4:32 pm, serguei.spitsyn at oracle.com wrote: > Hi David and Dean, > > One option is to add a command line option (disabled by default) > to enable debugging/profiling of the Graal compiler. > This will help to avoid all these Graal related issues, > simplify the development and stabilize the tests. It would simply tests, which could just disable it. But things would still need to work correctly if enabled - we can't have the current problems of trying to do an early return, or popframe, from the wrong frame. > Not sure the Graal developer will like this proposal though. :) > Also, it is not very clear what level of complexity we add with this. > For instance, we will have to identify all spots where new checks have > to be added. Yes identifying exactly where you needed to check for this is non-trivial I think. And this all needs a broader discussion before choosing this kind of approach. Cheers, David > > Thanks, > Serguei > > > On 12/3/18 20:45, David Holmes wrote: >> On 4/12/2018 5:53 am, dean.long at oracle.com wrote: >>> On 11/30/18 7:46 PM, JC Beyler wrote: >>>> Questions because I'm not familiar with JVMCI consequences so not >>>> really comments on the webrev but so that I know: >>>> ? - Is it normaly that you can suspend when you are in a JVMCI frame? >>> >>> Yes, because it's just Java code, and we allow all Java code to be >>> suspended, even Graal and JVMCI code. >> >> A choice which can be argued for and against. On the one hand it is >> nice to be able to try to debug JVMCI code, and on the other this >> injects execution of Java code into places which to date could not >> execute Java code and so can "shift" debugging actions from the >> application/test code, to the JVMCI code. Arguably the >> application/test code may need to have been more specific about its >> intent (ie verifying that the debuggee is suspended in the expected >> frame) and has just "been lucky" but nevertheless the use of JVMCI may >> disrupt existing code using these facilities. >> >>>> will/is there not a better way that we could detect that we are in a >>>> JVMCI frame? >>> >>> We could check the threads's _adjusting_comp_level flag for this >>> particular case, if we decided that we don't want to be able to debug >>> JVMCI Java code. >>> >>>> Is it always safe to suspend a JVMCI frame? >>> >>> That's a good question.? If it was grabbing any locks, then >>> suspending it could cause problems for other threads calling into JVMCI. >>> >>> Another solution would be to do adjusting_comp_level() in a separate >>> thread.? So I think there are at least 3 possible solutions: >>> >>> 1) Allow JVMCI adjusting_comp_level call to be suspended/debugged >>> 2) Don't allow it to be suspended/debugged, >>> ???? a) by running in a separate thread, or >>> ???? b) don't suspend when _adjusting_comp_level flag is set >>> >>> We could introduce a concept of "system Java" code, which, just like >>> Unix kernel code that is not debuggable without a kernel debugger, >>> would not normally be debuggable without setting a special flag. >> >> That may be something to consider in the future (albeit something that >> should IMHO have been considered well in the past!) but I think it's >> out of scope for this particular issue if we want to address this in >> 12. There's certainly a need, again IMHO, for a broader discussion as >> to how VM services written in Java should interact with other platform >> services intended for use with application and library code. I don't >> know if JVMCI/Graal explicitly hide themselves from agents and >> retransformation/redefinition/ClassFileLoadHook, or even basic things >> like event generation (where JVMCI may now generate many more events >> than previously encountered). >> >>> CCing graal-dev alias. >> >> As a non-subscriber my reply will get held for moderation. >> >> Cheers, >> David >> ----- >> >>> dl > From serguei.spitsyn at oracle.com Tue Dec 4 07:45:31 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 3 Dec 2018 23:45:31 -0800 Subject: RFR 8214572: nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the thread when the top frame executes JVMCI code In-Reply-To: <25e1102b-cce0-206d-b4bd-c11d72a6d4d9@oracle.com> References: <4E0EFD3E-F725-4469-BA20-7532D78BA479@oracle.com> <1a500fc6-9918-2b19-a0fe-bc299e58a015@oracle.com> <25e1102b-cce0-206d-b4bd-c11d72a6d4d9@oracle.com> Message-ID: <4d7e4c67-6a88-f0c5-a3b7-51901826f7bd@oracle.com> Agreed on both statements below. Thanks, Serguei On 12/3/18 23:32, David Holmes wrote: > On 4/12/2018 4:32 pm, serguei.spitsyn at oracle.com wrote: >> Hi David and Dean, >> >> One option is to add a command line option (disabled by default) >> to enable debugging/profiling of the Graal compiler. >> This will help to avoid all these Graal related issues, >> simplify the development and stabilize the tests. > > It would simply tests, which could just disable it. But things would > still need to work correctly if enabled - we can't have the current > problems of trying to do an early return, or popframe, from the wrong > frame. > >> Not sure the Graal developer will like this proposal though. :) >> Also, it is not very clear what level of complexity we add with this. >> For instance, we will have to identify all spots where new checks >> have to be added. > > Yes identifying exactly where you needed to check for this is > non-trivial I think. > > And this all needs a broader discussion before choosing this kind of > approach. > > Cheers, > David > >> >> Thanks, >> Serguei >> >> >> On 12/3/18 20:45, David Holmes wrote: >>> On 4/12/2018 5:53 am, dean.long at oracle.com wrote: >>>> On 11/30/18 7:46 PM, JC Beyler wrote: >>>>> Questions because I'm not familiar with JVMCI consequences so not >>>>> really comments on the webrev but so that I know: >>>>> ? - Is it normaly that you can suspend when you are in a JVMCI frame? >>>> >>>> Yes, because it's just Java code, and we allow all Java code to be >>>> suspended, even Graal and JVMCI code. >>> >>> A choice which can be argued for and against. On the one hand it is >>> nice to be able to try to debug JVMCI code, and on the other this >>> injects execution of Java code into places which to date could not >>> execute Java code and so can "shift" debugging actions from the >>> application/test code, to the JVMCI code. Arguably the >>> application/test code may need to have been more specific about its >>> intent (ie verifying that the debuggee is suspended in the expected >>> frame) and has just "been lucky" but nevertheless the use of JVMCI >>> may disrupt existing code using these facilities. >>> >>>>> will/is there not a better way that we could detect that we are in >>>>> a JVMCI frame? >>>> >>>> We could check the threads's _adjusting_comp_level flag for this >>>> particular case, if we decided that we don't want to be able to >>>> debug JVMCI Java code. >>>> >>>>> Is it always safe to suspend a JVMCI frame? >>>> >>>> That's a good question.? If it was grabbing any locks, then >>>> suspending it could cause problems for other threads calling into >>>> JVMCI. >>>> >>>> Another solution would be to do adjusting_comp_level() in a >>>> separate thread.? So I think there are at least 3 possible solutions: >>>> >>>> 1) Allow JVMCI adjusting_comp_level call to be suspended/debugged >>>> 2) Don't allow it to be suspended/debugged, >>>> ???? a) by running in a separate thread, or >>>> ???? b) don't suspend when _adjusting_comp_level flag is set >>>> >>>> We could introduce a concept of "system Java" code, which, just >>>> like Unix kernel code that is not debuggable without a kernel >>>> debugger, would not normally be debuggable without setting a >>>> special flag. >>> >>> That may be something to consider in the future (albeit something >>> that should IMHO have been considered well in the past!) but I think >>> it's out of scope for this particular issue if we want to address >>> this in 12. There's certainly a need, again IMHO, for a broader >>> discussion as to how VM services written in Java should interact >>> with other platform services intended for use with application and >>> library code. I don't know if JVMCI/Graal explicitly hide themselves >>> from agents and retransformation/redefinition/ClassFileLoadHook, or >>> even basic things like event generation (where JVMCI may now >>> generate many more events than previously encountered). >>> >>>> CCing graal-dev alias. >>> >>> As a non-subscriber my reply will get held for moderation. >>> >>> Cheers, >>> David >>> ----- >>> >>>> dl >> From rkennke at redhat.com Tue Dec 4 08:00:55 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 4 Dec 2018 09:00:55 +0100 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> Message-ID: <56886a20-3543-3006-8d11-164786a1e596@redhat.com> Hi Jini, > Thank you for making the changes. The SA portion looks good to me. Thank you! > One > nit though: > > In > src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/HeapSummary.java, > in printGCAlgorithm(), does displaying the nbr of Parallel GC threads > not make sense for Shenandoah (like it is for G1, ZGC, etc)? I suppose it does. I will add it. Thanks, Roman > Thank you, > Jini. > > > On 12/4/2018 12:57 AM, Roman Kennke wrote: >> Round 5 of Shenandoah review includes: >> - A fix for the @requires tag in TestFullGCCountTest.java. It should be >> correct now. We believe the CMS @requires was also not quite right and >> fixed it the same. >> >> It reads now: Don't run this test if: >> ? - Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is >> true, as set by harness >> ? - Actual GC set by harness is Shenandoah *and* >> ExplicitGCInvokesConcurrent is not set false by harness (it's true by >> default in Shenandoah, so this needs to be double-inverteed). >> >> The @requires for CMS was wrong before (we think), because it would also >> filter defaultGC + ExplicitGCInvokesConcurrent. >> >> - Sorting of macros was fixed, as was pointed out by Per >> - Some stuff was added to SA, as suggested by Jini >> - Rebased on most current jdk/jdk code >> >> Webrevs: >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/ >> >> I also need reviews from GC reviewers for the CSR: >> https://bugs.openjdk.java.net/browse/JDK-8214349 >> >> I already got reviews for: >> [x] shared-runtime (coleenp) >> [x] shared-compiler (kvn) >> >> I got reviews for shared-build, but an earlier version, so maybe makes >> sense to look over this again. Erik J, Magnus? >> >> I still need approvals for: >> [ ] shared-build????????? (kvn, erikj, ihse, pliden) >> [ ] shared-gc???????????? (pliden, kbarrett) >> [ ] shared-serviceability (jgeorge, pliden) >> [ ] shared-tests????????? (lmesnik, pliden) >> [ ] shenandoah-gc >> [ ] shenandoah-tests >> >> >> Thanks for your patience and ongoing support! >> >> Cheers, >> Roman >> >>> Hi all, >>> >>> here comes round 4 of Shenandoah upstreaming review: >>> >>> This includes fixes for the issues that Per brought up: >>> - Verify and gracefully reject dangerous flags combinations that >>> disables required barriers >>> - Revisited @requires filters in tests >>> - Trim unused code from Shenandoah's SA impl >>> - Move ShenandoahGCTracer to gc/shenandoah >>> - Fix ordering of GC names in various files >>> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >>> >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >>> >>> Thanks everybody for taking time to review this! >>> Roman >>> >>>> Hello all, >>>> >>>> Thanks so far for all the reviews and support! >>>> >>>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>>> Also, I fixed the numbering of my webrevs to match the review-round. >>>> ;-) >>>> >>>> Things we've changed today: >>>> - We moved shenandoah-specific code out of .ad files into our own .ad >>>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>>> requires an addition in build machinery though, see >>>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>>> - Improved zero-disabling and build-code-simplification as suggested by >>>> Magnus and Per >>>> - Cleaned up some leftovers in C2 >>>> - Improved C2 loop opts code by introducing another APIs in >>>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC >>>> guards now. >>>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>>> that. >>>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>>> - Rebased on jdk-12+22 >>>> >>>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>>> >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>>> >>>> Thanks, >>>> Roman >>>> >>>>> Alright, we fixed: >>>>> - The minor issues that Kim reported in shared-gc >>>>> - A lot of fixes in shared-tests according to Leonid's review >>>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>>> >>>>> Some notes: >>>>> Leonid:? test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>>> correct. The @requires there means to exclude runs with both CMS and >>>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>>> fine. Adding the same clause for Shenandoah means the same, and >>>>> filters >>>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>>> made the condition a bit clearer by avoiding triple-negation. >>>>> >>>>> See: >>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>>> >>>>> >>>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>>> >>>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>>> those with ZGC? >>>>> >>>>> See: >>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>>> >>>>> >>>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>>> next round). >>>>> >>>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>>> better. I can tell that we're not done with C2 yet. Can you look over >>>>> the code and see what is ok, and especially what is not ok, so that we >>>>> can focus our efforts on the relevant parts? >>>>> >>>>> Updated set of webrevs: >>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>>> >>>>> Thanks, >>>>> Roman >>>>> >>>>> >>>>>> Hi, >>>>>> >>>>>> This is the first round of changes for including Shenandoah GC into >>>>>> mainline. >>>>>> I divided the review into parts that roughly correspond to the >>>>>> mailing lists >>>>>> that would normally review it, and I divided it into 'shared' code >>>>>> changes and >>>>>> 'shenandoah' code changes (actually, mostly additions). The intend >>>>>> is to >>>>>> eventually >>>>>> push them as single 'combined' changeset, once reviewed. >>>>>> >>>>>> JEP: >>>>>> ?? https://openjdk.java.net/jeps/189 >>>>>> Bug entry: >>>>>> >>>>>> ??https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>>> >>>>>> Webrevs: >>>>>> ?? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>>> >>>>>> For those who want to see the full change, have a look at the >>>>>> shenandoah-complete >>>>>> >>>>>> >>>>>> directory, >>>>>> it contains the full combined webrev. Alternatively, there is the >>>>>> file >>>>>> shenandoah-master.patch >>>>>> , >>>>>> >>>>>> which is what I intend to commit (and which should be equivalent >>>>>> to the >>>>>> 'shenandoah-complete' webrev). >>>>>> >>>>>> Sections to review (at this point) are the following: >>>>>> ??*) shenandoah-gc >>>>>> >>>>>> >>>>>> ???? - Actual Shenandoah implementation, almost completely >>>>>> residing in >>>>>> gc/shenandoah >>>>>> >>>>>> ??*) shared-gc >>>>>> >>>>>> >>>>>> ???? - This is mostly boilerplate that is common to any GC >>>>>> ???? - referenceProcessor.cpp has a little change to make one >>>>>> assert not >>>>>> fail (next to CMS and G1) >>>>>> ???? - taskqueue.hpp has some small adjustments to enable subclassing >>>>>> >>>>>> ??*) shared-serviceability >>>>>> >>>>>> >>>>>> ???? - The usual code to support another GC >>>>>> >>>>>> ??*) shared-runtime >>>>>> >>>>>> >>>>>> ???? - A number of friends declarations to allow Shenandoah >>>>>> iterators to >>>>>> hook up with, >>>>>> ?????? e.g. ClassLoaderData, CodeCache, etc >>>>>> ???? - Warning and disabling JFR LeakProfiler >>>>>> ???? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>>> Shenandoah C2 optimizations >>>>>> ???? - Locks initialization in mutexLocker.cpp as usual >>>>>> ???? - VM operations defines for Shenandoah's VM ops >>>>>> ???? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>>> Shenandoah's logging >>>>>> ???? - The usual macros in macro.hpp >>>>>> >>>>>> ??*) shared-build >>>>>> >>>>>> >>>>>> ???? - Add shenandoah feature, enabled by default, as agreed with >>>>>> Vladimir K. beforehand >>>>>> ???? - Some flags for shenandoah-enabled compilation to get >>>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>>> ?????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>>> Shenandoah's barriers >>>>>> ???? - --param inline-unit-growth=1000 settings for 2 shenandoah >>>>>> source >>>>>> files, which is >>>>>> ?????? useful to get the whole marking loop inlined (observed >>>>>> significant >>>>>> regression if we >>>>>> ?????? don't) >>>>>> >>>>>> ??*) shared-tests >>>>>> >>>>>> >>>>>> ???? - Test infrastructure to support Shenandoah >>>>>> ???? - Shenandoah test groups >>>>>> ???? - Exclude Shenandoah in various tests that can be run with >>>>>> selected GC >>>>>> ???? - Enable/add configure for Shenandoah for tests that make >>>>>> sense to >>>>>> run with it >>>>>> >>>>>> ??*) shenandoah-tests >>>>>> >>>>>> >>>>>> ???? - Shenandoah specific tests, most reside in gc/shenandoah >>>>>> subdirectory >>>>>> ???? - A couple of tests configurations have been added, e.g. >>>>>> TestGCBasherWithShenandoah.java >>>>>> >>>>>> I intentionally left out shared-compiler for now, because we have >>>>>> some >>>>>> work left to do >>>>>> there, but if you click around you'll find the patch anyway, in >>>>>> case you >>>>>> want to take >>>>>> a peek at it. >>>>>> >>>>>> We have regular builds on: >>>>>> ?? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>>> ?? - {Windows} x {x86_64}, >>>>>> ?? - {MacOS X} x {x86_64} >>>>>> >>>>>> This also routinely passes: >>>>>> ?? - the new Shenandoah tests >>>>>> ?? - jcstress with/without aggressive Shenandoah verification >>>>>> ?? - specjvm2008 with/without aggressive Shenandoah verification >>>>>> >>>>>> >>>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she >>>>>> deserves >>>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>>> teams for tirelessly helping with and reviewing all the GC >>>>>> interface and >>>>>> related changes, and of course the many early adopters for reporting >>>>>> bugs and success stories and feature requests: we wouldn't be here >>>>>> without any of you! >>>>>> >>>>>> Best regards, >>>>>> Roman >>>>>> >>>>> >>>> >>> >> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Tue Dec 4 08:01:23 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 4 Dec 2018 09:01:23 +0100 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> Message-ID: Hi Magnus, >> I got reviews for shared-build, but an earlier version, so maybe makes >> sense to look over this again. Erik J, Magnus? > > Build changes look good. Thanks, Magnus! Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From per.liden at oracle.com Tue Dec 4 09:35:26 2018 From: per.liden at oracle.com (Per Liden) Date: Tue, 4 Dec 2018 10:35:26 +0100 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> Message-ID: Hi Roman, On 12/3/18 8:27 PM, Roman Kennke wrote: > Round 5 of Shenandoah review includes: > - A fix for the @requires tag in TestFullGCCountTest.java. It should be > correct now. We believe the CMS @requires was also not quite right and > fixed it the same. > > It reads now: Don't run this test if: > - Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is > true, as set by harness > - Actual GC set by harness is Shenandoah *and* > ExplicitGCInvokesConcurrent is not set false by harness (it's true by > default in Shenandoah, so this needs to be double-inverteed). > > The @requires for CMS was wrong before (we think), because it would also > filter defaultGC + ExplicitGCInvokesConcurrent. > > - Sorting of macros was fixed, as was pointed out by Per > - Some stuff was added to SA, as suggested by Jini > - Rebased on most current jdk/jdk code > > Webrevs: > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/ > > I also need reviews from GC reviewers for the CSR: > https://bugs.openjdk.java.net/browse/JDK-8214349 > > I already got reviews for: > [x] shared-runtime (coleenp) > [x] shared-compiler (kvn) > > I got reviews for shared-build, but an earlier version, so maybe makes > sense to look over this again. Erik J, Magnus? > > I still need approvals for: > [ ] shared-build (kvn, erikj, ihse, pliden) > [ ] shared-gc (pliden, kbarrett) > [ ] shared-serviceability (jgeorge, pliden) > [ ] shared-tests (lmesnik, pliden) The above parts look good to me. Reviewed. Just one tiny nit (and I don't need to see a new webrev for this): In src/hotspot/share/gc/shared/gcCause.cpp you have this: + case _shenandoah_allocation_failure_evac: + return "Allocation Failure During Evac"; + + case _shenandoah_stop_vm: + return "Stopping VM"; + + case _shenandoah_concurrent_gc: + return "Shenandoah Concurrent GC"; + + case _shenandoah_traversal_gc: + return "Shenandoah Traversal GC"; + + case _shenandoah_upgrade_to_full_gc: + return "Shenandoah Upgrade To Full GC"; + And in src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shared/GCCause.java you have this: + _shenandoah_stop_vm ("Stop VM"), + _shenandoah_allocation_failure_evac ("Allocation Failure During Evacuation"), + _shenandoah_concurrent_gc ("Concurrent GC"), + _shenandoah_traversal_gc ("Traversal GC"), + _shenandoah_upgrade_to_full_gc ("Upgrade to Full GC"), It would be good to have the exact same strings in both places. There are currently small differences in all of them. "Evac" vs "Evacuation", "Stop" vs "Stopping", "Shenandoah" vs "", etc. May I also suggest that you skip "Shenandoah" in things like "Shenandoah Concurrent GC" as I kind of think it's implied by the context. But I also know that CMS/G1 isn't consistent on this point. An alternative would be to add "Shenandoah" to all of the strings to keep things consistent, but I'm not sure I like that better. You decide. > [ ] shenandoah-gc > [ ] shenandoah-tests I haven't looked very much on these parts, and I didn't plan to do so in detail right now. I think it's fine of the folks that have been working on the Shenandoah code reviewed this. cheers, Per > > > Thanks for your patience and ongoing support! > > Cheers, > Roman > >> Hi all, >> >> here comes round 4 of Shenandoah upstreaming review: >> >> This includes fixes for the issues that Per brought up: >> - Verify and gracefully reject dangerous flags combinations that >> disables required barriers >> - Revisited @requires filters in tests >> - Trim unused code from Shenandoah's SA impl >> - Move ShenandoahGCTracer to gc/shenandoah >> - Fix ordering of GC names in various files >> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >> >> Thanks everybody for taking time to review this! >> Roman >> >>> Hello all, >>> >>> Thanks so far for all the reviews and support! >>> >>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >>> >>> Things we've changed today: >>> - We moved shenandoah-specific code out of .ad files into our own .ad >>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>> requires an addition in build machinery though, see >>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>> - Improved zero-disabling and build-code-simplification as suggested by >>> Magnus and Per >>> - Cleaned up some leftovers in C2 >>> - Improved C2 loop opts code by introducing another APIs in >>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now. >>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>> that. >>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>> - Rebased on jdk-12+22 >>> >>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>> >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>> >>> Thanks, >>> Roman >>> >>>> Alright, we fixed: >>>> - The minor issues that Kim reported in shared-gc >>>> - A lot of fixes in shared-tests according to Leonid's review >>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>> >>>> Some notes: >>>> Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>> correct. The @requires there means to exclude runs with both CMS and >>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>> fine. Adding the same clause for Shenandoah means the same, and filters >>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>> made the condition a bit clearer by avoiding triple-negation. >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>> >>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>> >>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>> those with ZGC? >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>> >>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>> next round). >>>> >>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>> better. I can tell that we're not done with C2 yet. Can you look over >>>> the code and see what is ok, and especially what is not ok, so that we >>>> can focus our efforts on the relevant parts? >>>> >>>> Updated set of webrevs: >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>> >>>> Thanks, >>>> Roman >>>> >>>> >>>>> Hi, >>>>> >>>>> This is the first round of changes for including Shenandoah GC into >>>>> mainline. >>>>> I divided the review into parts that roughly correspond to the mailing lists >>>>> that would normally review it, and I divided it into 'shared' code >>>>> changes and >>>>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>>>> eventually >>>>> push them as single 'combined' changeset, once reviewed. >>>>> >>>>> JEP: >>>>> ? https://openjdk.java.net/jeps/189 >>>>> Bug entry: >>>>> >>>>> ?https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>> >>>>> Webrevs: >>>>> ? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>> >>>>> For those who want to see the full change, have a look at the >>>>> shenandoah-complete >>>>> >>>>> directory, >>>>> it contains the full combined webrev. Alternatively, there is the file >>>>> shenandoah-master.patch >>>>> , >>>>> which is what I intend to commit (and which should be equivalent to the >>>>> 'shenandoah-complete' webrev). >>>>> >>>>> Sections to review (at this point) are the following: >>>>> ?*) shenandoah-gc >>>>> >>>>> ??? - Actual Shenandoah implementation, almost completely residing in >>>>> gc/shenandoah >>>>> >>>>> ?*) shared-gc >>>>> >>>>> ??? - This is mostly boilerplate that is common to any GC >>>>> ??? - referenceProcessor.cpp has a little change to make one assert not >>>>> fail (next to CMS and G1) >>>>> ??? - taskqueue.hpp has some small adjustments to enable subclassing >>>>> >>>>> ?*) shared-serviceability >>>>> >>>>> ??? - The usual code to support another GC >>>>> >>>>> ?*) shared-runtime >>>>> >>>>> ??? - A number of friends declarations to allow Shenandoah iterators to >>>>> hook up with, >>>>> ????? e.g. ClassLoaderData, CodeCache, etc >>>>> ??? - Warning and disabling JFR LeakProfiler >>>>> ??? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>> Shenandoah C2 optimizations >>>>> ??? - Locks initialization in mutexLocker.cpp as usual >>>>> ??? - VM operations defines for Shenandoah's VM ops >>>>> ??? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>> Shenandoah's logging >>>>> ??? - The usual macros in macro.hpp >>>>> >>>>> ?*) shared-build >>>>> >>>>> ??? - Add shenandoah feature, enabled by default, as agreed with >>>>> Vladimir K. beforehand >>>>> ??? - Some flags for shenandoah-enabled compilation to get >>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>> ????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>> Shenandoah's barriers >>>>> ??? - --param inline-unit-growth=1000 settings for 2 shenandoah source >>>>> files, which is >>>>> ????? useful to get the whole marking loop inlined (observed significant >>>>> regression if we >>>>> ????? don't) >>>>> >>>>> ?*) shared-tests >>>>> >>>>> ??? - Test infrastructure to support Shenandoah >>>>> ??? - Shenandoah test groups >>>>> ??? - Exclude Shenandoah in various tests that can be run with selected GC >>>>> ??? - Enable/add configure for Shenandoah for tests that make sense to >>>>> run with it >>>>> >>>>> ?*) shenandoah-tests >>>>> >>>>> ??? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >>>>> ??? - A couple of tests configurations have been added, e.g. >>>>> TestGCBasherWithShenandoah.java >>>>> >>>>> I intentionally left out shared-compiler for now, because we have some >>>>> work left to do >>>>> there, but if you click around you'll find the patch anyway, in case you >>>>> want to take >>>>> a peek at it. >>>>> >>>>> We have regular builds on: >>>>> ? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>> ? - {Windows} x {x86_64}, >>>>> ? - {MacOS X} x {x86_64} >>>>> >>>>> This also routinely passes: >>>>> ? - the new Shenandoah tests >>>>> ? - jcstress with/without aggressive Shenandoah verification >>>>> ? - specjvm2008 with/without aggressive Shenandoah verification >>>>> >>>>> >>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>> teams for tirelessly helping with and reviewing all the GC interface and >>>>> related changes, and of course the many early adopters for reporting >>>>> bugs and success stories and feature requests: we wouldn't be here >>>>> without any of you! >>>>> >>>>> Best regards, >>>>> Roman >>>>> >>>> >>> >> > From rkennke at redhat.com Tue Dec 4 10:53:26 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 4 Dec 2018 11:53:26 +0100 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> Message-ID: <91b933b1-9b42-090e-5f0f-07454aeb456e@redhat.com> Hi Per, >> Round 5 of Shenandoah review includes: >> - A fix for the @requires tag in TestFullGCCountTest.java. It should be >> correct now. We believe the CMS @requires was also not quite right and >> fixed it the same. >> >> It reads now: Don't run this test if: >> ? - Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is >> true, as set by harness >> ? - Actual GC set by harness is Shenandoah *and* >> ExplicitGCInvokesConcurrent is not set false by harness (it's true by >> default in Shenandoah, so this needs to be double-inverteed). >> >> The @requires for CMS was wrong before (we think), because it would also >> filter defaultGC + ExplicitGCInvokesConcurrent. >> >> - Sorting of macros was fixed, as was pointed out by Per >> - Some stuff was added to SA, as suggested by Jini >> - Rebased on most current jdk/jdk code >> >> Webrevs: >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/ >> >> I also need reviews from GC reviewers for the CSR: >> https://bugs.openjdk.java.net/browse/JDK-8214349 >> >> I already got reviews for: >> [x] shared-runtime (coleenp) >> [x] shared-compiler (kvn) >> >> I got reviews for shared-build, but an earlier version, so maybe makes >> sense to look over this again. Erik J, Magnus? >> >> I still need approvals for: >> [ ] shared-build????????? (kvn, erikj, ihse, pliden) >> [ ] shared-gc???????????? (pliden, kbarrett) >> [ ] shared-serviceability (jgeorge, pliden) >> [ ] shared-tests????????? (lmesnik, pliden) > > The above parts look good to me. Reviewed. Great! Thanks! > Just one tiny nit (and I don't need to see a new webrev for this): > > In src/hotspot/share/gc/shared/gcCause.cpp you have this: > > +??? case _shenandoah_allocation_failure_evac: > +????? return "Allocation Failure During Evac"; > + > +??? case _shenandoah_stop_vm: > +????? return "Stopping VM"; > + > +??? case _shenandoah_concurrent_gc: > +????? return "Shenandoah Concurrent GC"; > + > +??? case _shenandoah_traversal_gc: > +????? return "Shenandoah Traversal GC"; > + > +??? case _shenandoah_upgrade_to_full_gc: > +????? return "Shenandoah Upgrade To Full GC"; > + > > And in > src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shared/GCCause.java > you have this: > > +? _shenandoah_stop_vm ("Stop VM"), > +? _shenandoah_allocation_failure_evac ("Allocation Failure During > Evacuation"), > +? _shenandoah_concurrent_gc ("Concurrent GC"), > +? _shenandoah_traversal_gc ("Traversal GC"), > +? _shenandoah_upgrade_to_full_gc ("Upgrade to Full GC"), > > It would be good to have the exact same strings in both places. There > are currently small differences in all of them. "Evac" vs "Evacuation", > "Stop" vs "Stopping", "Shenandoah" vs "", etc. > > May I also suggest that you skip "Shenandoah" in things like "Shenandoah > Concurrent GC" as I kind of think it's implied by the context. But I > also know that CMS/G1 isn't consistent on this point. An alternative > would be to add "Shenandoah" to all of the strings to keep things > consistent, but I'm not sure I like that better. You decide. I'm addressing it here: http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-December/008572.html >> [ ] shenandoah-gc >> [ ] shenandoah-tests > > I haven't looked very much on these parts, and I didn't plan to do so in > detail right now. I think it's fine of the folks that have been working > on the Shenandoah code reviewed this. Yes, I think so too (except maybe stuff like shenandoah_globals.hpp, but that was already reviewed I think). Thanks for reviewing, Per! Roman > cheers, > Per > >> >> >> Thanks for your patience and ongoing support! >> >> Cheers, >> Roman >> >>> Hi all, >>> >>> here comes round 4 of Shenandoah upstreaming review: >>> >>> This includes fixes for the issues that Per brought up: >>> - Verify and gracefully reject dangerous flags combinations that >>> disables required barriers >>> - Revisited @requires filters in tests >>> - Trim unused code from Shenandoah's SA impl >>> - Move ShenandoahGCTracer to gc/shenandoah >>> - Fix ordering of GC names in various files >>> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >>> >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >>> >>> Thanks everybody for taking time to review this! >>> Roman >>> >>>> Hello all, >>>> >>>> Thanks so far for all the reviews and support! >>>> >>>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>>> Also, I fixed the numbering of my webrevs to match the review-round. >>>> ;-) >>>> >>>> Things we've changed today: >>>> - We moved shenandoah-specific code out of .ad files into our own .ad >>>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>>> requires an addition in build machinery though, see >>>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>>> - Improved zero-disabling and build-code-simplification as suggested by >>>> Magnus and Per >>>> - Cleaned up some leftovers in C2 >>>> - Improved C2 loop opts code by introducing another APIs in >>>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC >>>> guards now. >>>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>>> that. >>>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>>> - Rebased on jdk-12+22 >>>> >>>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>>> >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>>> >>>> Thanks, >>>> Roman >>>> >>>>> Alright, we fixed: >>>>> - The minor issues that Kim reported in shared-gc >>>>> - A lot of fixes in shared-tests according to Leonid's review >>>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>>> >>>>> Some notes: >>>>> Leonid:? test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>>> correct. The @requires there means to exclude runs with both CMS and >>>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>>> fine. Adding the same clause for Shenandoah means the same, and >>>>> filters >>>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>>> made the condition a bit clearer by avoiding triple-negation. >>>>> >>>>> See: >>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>>> >>>>> >>>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>>> >>>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>>> those with ZGC? >>>>> >>>>> See: >>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>>> >>>>> >>>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>>> next round). >>>>> >>>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>>> better. I can tell that we're not done with C2 yet. Can you look over >>>>> the code and see what is ok, and especially what is not ok, so that we >>>>> can focus our efforts on the relevant parts? >>>>> >>>>> Updated set of webrevs: >>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>>> >>>>> Thanks, >>>>> Roman >>>>> >>>>> >>>>>> Hi, >>>>>> >>>>>> This is the first round of changes for including Shenandoah GC into >>>>>> mainline. >>>>>> I divided the review into parts that roughly correspond to the >>>>>> mailing lists >>>>>> that would normally review it, and I divided it into 'shared' code >>>>>> changes and >>>>>> 'shenandoah' code changes (actually, mostly additions). The intend >>>>>> is to >>>>>> eventually >>>>>> push them as single 'combined' changeset, once reviewed. >>>>>> >>>>>> JEP: >>>>>> ?? https://openjdk.java.net/jeps/189 >>>>>> Bug entry: >>>>>> >>>>>> ??https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>>> >>>>>> Webrevs: >>>>>> ?? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>>> >>>>>> For those who want to see the full change, have a look at the >>>>>> shenandoah-complete >>>>>> >>>>>> >>>>>> directory, >>>>>> it contains the full combined webrev. Alternatively, there is the >>>>>> file >>>>>> shenandoah-master.patch >>>>>> , >>>>>> >>>>>> which is what I intend to commit (and which should be equivalent >>>>>> to the >>>>>> 'shenandoah-complete' webrev). >>>>>> >>>>>> Sections to review (at this point) are the following: >>>>>> ??*) shenandoah-gc >>>>>> >>>>>> >>>>>> ???? - Actual Shenandoah implementation, almost completely >>>>>> residing in >>>>>> gc/shenandoah >>>>>> >>>>>> ??*) shared-gc >>>>>> >>>>>> >>>>>> ???? - This is mostly boilerplate that is common to any GC >>>>>> ???? - referenceProcessor.cpp has a little change to make one >>>>>> assert not >>>>>> fail (next to CMS and G1) >>>>>> ???? - taskqueue.hpp has some small adjustments to enable subclassing >>>>>> >>>>>> ??*) shared-serviceability >>>>>> >>>>>> >>>>>> ???? - The usual code to support another GC >>>>>> >>>>>> ??*) shared-runtime >>>>>> >>>>>> >>>>>> ???? - A number of friends declarations to allow Shenandoah >>>>>> iterators to >>>>>> hook up with, >>>>>> ?????? e.g. ClassLoaderData, CodeCache, etc >>>>>> ???? - Warning and disabling JFR LeakProfiler >>>>>> ???? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>>> Shenandoah C2 optimizations >>>>>> ???? - Locks initialization in mutexLocker.cpp as usual >>>>>> ???? - VM operations defines for Shenandoah's VM ops >>>>>> ???? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>>> Shenandoah's logging >>>>>> ???? - The usual macros in macro.hpp >>>>>> >>>>>> ??*) shared-build >>>>>> >>>>>> >>>>>> ???? - Add shenandoah feature, enabled by default, as agreed with >>>>>> Vladimir K. beforehand >>>>>> ???? - Some flags for shenandoah-enabled compilation to get >>>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>>> ?????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>>> Shenandoah's barriers >>>>>> ???? - --param inline-unit-growth=1000 settings for 2 shenandoah >>>>>> source >>>>>> files, which is >>>>>> ?????? useful to get the whole marking loop inlined (observed >>>>>> significant >>>>>> regression if we >>>>>> ?????? don't) >>>>>> >>>>>> ??*) shared-tests >>>>>> >>>>>> >>>>>> ???? - Test infrastructure to support Shenandoah >>>>>> ???? - Shenandoah test groups >>>>>> ???? - Exclude Shenandoah in various tests that can be run with >>>>>> selected GC >>>>>> ???? - Enable/add configure for Shenandoah for tests that make >>>>>> sense to >>>>>> run with it >>>>>> >>>>>> ??*) shenandoah-tests >>>>>> >>>>>> >>>>>> ???? - Shenandoah specific tests, most reside in gc/shenandoah >>>>>> subdirectory >>>>>> ???? - A couple of tests configurations have been added, e.g. >>>>>> TestGCBasherWithShenandoah.java >>>>>> >>>>>> I intentionally left out shared-compiler for now, because we have >>>>>> some >>>>>> work left to do >>>>>> there, but if you click around you'll find the patch anyway, in >>>>>> case you >>>>>> want to take >>>>>> a peek at it. >>>>>> >>>>>> We have regular builds on: >>>>>> ?? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>>> ?? - {Windows} x {x86_64}, >>>>>> ?? - {MacOS X} x {x86_64} >>>>>> >>>>>> This also routinely passes: >>>>>> ?? - the new Shenandoah tests >>>>>> ?? - jcstress with/without aggressive Shenandoah verification >>>>>> ?? - specjvm2008 with/without aggressive Shenandoah verification >>>>>> >>>>>> >>>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she >>>>>> deserves >>>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>>> teams for tirelessly helping with and reviewing all the GC >>>>>> interface and >>>>>> related changes, and of course the many early adopters for reporting >>>>>> bugs and success stories and feature requests: we wouldn't be here >>>>>> without any of you! >>>>>> >>>>>> Best regards, >>>>>> Roman >>>>>> >>>>> >>>> >>> >> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From stooke at redhat.com Tue Dec 4 14:57:41 2018 From: stooke at redhat.com (Simon Tooke) Date: Tue, 4 Dec 2018 09:57:41 -0500 Subject: RFR: (XS) 8214061: Buffer written into itself Message-ID: In one fatal error code path, snprintf() is given its output buffer asan input string.? src/jdk.jdwp.agent/share/native/libjdwp/debugInit.c line 645:? ????(void)snprintf(buf, sizeof(buf), "JDWP %s", buf); Proposed fix is of course: ??? (void)snprintf(buf, sizeof(buf), "JDWP %s", msg); This was found by compiling with GCC 8.1 Bug: https://bugs.openjdk.java.net/browse/JDK-8214061 webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/stooke/JDK-8214061/01/webrev/ Thanks, -Simon Tooke From patricio.chilano.mateo at oracle.com Tue Dec 4 15:02:35 2018 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Tue, 4 Dec 2018 10:02:35 -0500 Subject: RFR 8214148: [TESTBUG] serviceability/tmtools/jstack/WaitNotifyThreadTest.java is not doing what is expected In-Reply-To: <7967f3a4-3aa9-79f7-30d8-7b7eae53aa7c@oracle.com> References: <57a4cb4c-87e5-bee4-192a-4f41d19e4ea5@oracle.com> <18c5ccc0-4058-b4bd-9d2a-3fbfafd5069b@oracle.com> <1f51ea19-9d1a-916a-e1f0-428a305796f5@oracle.com> <3b553c1c-eedf-4ecd-41a7-ff9d120fa51a@oracle.com> <7967f3a4-3aa9-79f7-30d8-7b7eae53aa7c@oracle.com> Message-ID: <6db4d7ee-a5ca-d7a8-fbdb-e9edb0e8aa21@oracle.com> Thanks David! Patricio On 12/3/18 11:30 PM, David Holmes wrote: > Looks good! > > Thanks, > David > > On 4/12/2018 2:48 am, Patricio Chilano wrote: >> Hi David, >> >> On 12/3/18 2:14 AM, David Holmes wrote: >>> Hi Patricio, >>> >>> On 1/12/2018 2:31 am, Patricio Chilano wrote: >>>> Hi David, >>>> >>>> On 11/29/18 8:05 PM, David Holmes wrote: >>>>> Hi Patricio, >>>>> >>>>> This seems very complicated and I'm not quite seeing how it all >>>>> goes together. The check for waiting to re-lock now seems to >>>>> dominant the test and obscure the original checks. >>>>> >>>>> I'm not sure this is worthwhile in the context of this test. It >>>>> might be much simpler to just get rid of the existing "waiting to >>>>> re-lock" check which will not be seen and then if we really want >>>>> to check that case add a much simpler form that just checks for that. >>>> Ok, I actually had similar thoughts while I was adding the extra >>>> code. Here is the new webrev: >>>> >>>> http://cr.openjdk.java.net/~pchilanomate/8214148.02/webrev/ >>>> >>>> I removed the check for the "waiting to re-lock" case. >>> >>> Good - the more I look at this test the more I see the "waiting to >>> re-lock" case is just not relevant to it. >>> >>> The addition of the code to wait until the target thread is in the >>> right state seems good. >>> >>> I'm unclear on some of the changes to the checking code in >>> analyzeThreadStackWaiting. It seems you removed the code that >>> watched for finding the wrong monitor and replaced that with a call >>> to assertMonitorInfo. But the latter is passed the address you got >>> from the MonitorInfo in the first place so the check of the address >>> is never going to fail. ?? >> I just replaced that "if-else" with assertMonitorInfo() because the >> same check "monInfo.getType().equals("waiting on") && >> compareMonitorClass(monInfo)" will be done in assertMonitorInfo() and >> the error case is also doing the same, so the code gets simplified. >> Yes, the extra address check will never fail. >> >>> That said you can't be waiting on two monitors so I don't see how we >>> can ever have the wrong one ?? >> I'm not sure why this test is checking for the monitor address in >> assertMonitorInfo(). The only case where I see it could fail is for >> the RUN_METHOD case if the address is null, but that has a separate >> check before assertMonitorInfo(). Maybe at some point the test had >> more monitors, because there was also that "for" loop checking for >> the "waiting to re-lock in wait()" case. I can remove the test >> "monInfo.getMonitorAddress().equals(monitorAddress)" in >> assertMonitorInfo() but I don't think it hurts to keep it. >> >>> A few minor style nits: >>> >>> Pre-existing: >>> >>> 54???????????? //Notify the waiting thread, so it stops waiting and >>> sleeps >>> >>> Please add a space after // >>> >>> ?106???????? // Start athread that just waits >>> >>> s/athread/a thread/ >>> >>> ?145???????????????????? throw new RuntimeException(OBJECT_WAIT >>> ?146???????????????????????????? + " method has to contain one lock >>> record but it contains " + mi.getLocks().size()); >>> >>> Indentation is wrong - the '+' should align with the O in OBJECT. >>> Break into three lines (at second + if needed) >>> >>> New: >>> >>> 154???????????????? if(mi.getLocks().size() == 1){ >>> >>> Space after "if", and space before { >>> >>> ?157???????????????? else{ >>> >>> Space before { >>> >>> 158???????????????????? throw new RuntimeException(RUN_METHOD + " >>> method has to contain one lock record but it contains " >>> ?159???????????????????????????????????????????????????????????? + >>> mi.getLocks().size()); >>> >>> Incorrect indentation - '+' should align with R in RUN >> Done! >>>> I'm not sure if it's okay to keep the change to >>>> serviceability/tmtools/jstack/utils/DefaultFormat.java then. It >>>> doesn't really affect this test, but it is needed for jstack to >>>> detect the locks that appear in the stack report with the message >>>> "waiting to re-lock in wait()". >>> >>> I'd probably revert that change at this stage. >> Done! >> >> Here is the new webrev: >> >> http://cr.openjdk.java.net/~pchilanomate/8214148.03/webrev/ >> >> >> Thanks, >> Patricio >>> Thanks, >>> David >>> >>>> Thanks, >>>> Patricio >>>>> To me the simplest way to see the "re-lock in wait" case is to just: >>>>> >>>>> synchronized(obj) { >>>>> ?? obj.notifyAll(); >>>>> ?? <= take stack dump here => >>>>> } >>>>> >>>>> Cheers, >>>>> David >>>>> >>>>> On 30/11/2018 5:52 am, Daniel D. Daugherty wrote: >>>>>> Adding serviceability-dev at ... since the 'tmtools' including 'jstack' >>>>>> are owned by the Serviceability team. >>>>>> >>>>>> Dan >>>>>> >>>>>> >>>>>> On 11/28/18 4:21 PM, Patricio Chilano wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> Could you review this?fix for test >>>>>>> serviceability/tmtools/jstack/WaitNotifyThreadTest.java? >>>>>>> >>>>>>> On one hand the test was not properly checking what it was >>>>>>> intended to check, since as mentioned in JBS the logic for >>>>>>> checking the method name was wrong. Also since there was only >>>>>>> one monitor in the test, the "for" loop with the message >>>>>>> "waiting to re-lock in wait()" was never actually reached. >>>>>>> >>>>>>> Additionally, with change 8150689 the message?"waiting to >>>>>>> re-lock in wait()" is now shown in the frame where the relocking >>>>>>> is actually taking place, so the logic for checking that should >>>>>>> change. >>>>>>> >>>>>>> I fixed the first issues and added logic to check for the >>>>>>> "waiting to re-lock in wait()" case. I used the Thread.State >>>>>>> attribute to check desire states are reached before getting the >>>>>>> thread dump reports through jstack. I run the test in mach5 >>>>>>> several times for all platforms and they all passed. >>>>>>> >>>>>>> Webrev URL: >>>>>>> http://cr.openjdk.java.net/~pchilanomate/8214148.01/webrev >>>>>>> Bug URL: https://bugs.openjdk.java.net/browse/JDK-8214148 >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Patricio >>>>>> >>>> >> From daniel.daugherty at oracle.com Tue Dec 4 15:08:54 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 4 Dec 2018 10:08:54 -0500 Subject: RFR: (XS) 8214061: Buffer written into itself In-Reply-To: References: Message-ID: <7ea8d17d-0771-7211-cd39-965406d1ddbc@oracle.com> On 12/4/18 9:57 AM, Simon Tooke wrote: > In one fatal error code path, snprintf() is given its output buffer asan > input string. > > src/jdk.jdwp.agent/share/native/libjdwp/debugInit.c line 645: > > ????(void)snprintf(buf, sizeof(buf), "JDWP %s", buf); > > Proposed fix is of course: > > ??? (void)snprintf(buf, sizeof(buf), "JDWP %s", msg); > > This was found by compiling with GCC 8.1 > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214061 > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/stooke/JDK-8214061/01/webrev/ src/jdk.jdwp.agent/share/native/libjdwp/debugInit.c ??? No comments. Thumbs up. Dan > > > Thanks, > -Simon Tooke > From sgehwolf at redhat.com Tue Dec 4 16:02:49 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Tue, 04 Dec 2018 17:02:49 +0100 Subject: RFR: (XS) 8214061: Buffer written into itself In-Reply-To: <7ea8d17d-0771-7211-cd39-965406d1ddbc@oracle.com> References: <7ea8d17d-0771-7211-cd39-965406d1ddbc@oracle.com> Message-ID: Hi, On Tue, 2018-12-04 at 10:08 -0500, Daniel D. Daugherty wrote: > > On 12/4/18 9:57 AM, Simon Tooke wrote: > > In one fatal error code path, snprintf() is given its output buffer asan > > input string. > > > > src/jdk.jdwp.agent/share/native/libjdwp/debugInit.c line 645: > > > > (void)snprintf(buf, sizeof(buf), "JDWP %s", buf); > > > > Proposed fix is of course: > > > > (void)snprintf(buf, sizeof(buf), "JDWP %s", msg); > > > > This was found by compiling with GCC 8.1 > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214061 > > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/stooke/JDK-8214061/01/webrev/ > > src/jdk.jdwp.agent/share/native/libjdwp/debugInit.c > No comments. > > Thumbs up. Can this be considered trivial enough to only require one reviewer? FWIW, this looks good to me too, but I'm not a Reviewer. Thanks, Severin From daniel.daugherty at oracle.com Tue Dec 4 16:42:56 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 4 Dec 2018 11:42:56 -0500 Subject: RFR: (XS) 8214061: Buffer written into itself In-Reply-To: References: <7ea8d17d-0771-7211-cd39-965406d1ddbc@oracle.com> Message-ID: <6f8b0747-515f-52d4-443d-ea046d53a2b3@oracle.com> On 12/4/18 11:02 AM, Severin Gehwolf wrote: > Hi, > > On Tue, 2018-12-04 at 10:08 -0500, Daniel D. Daugherty wrote: >> On 12/4/18 9:57 AM, Simon Tooke wrote: >>> In one fatal error code path, snprintf() is given its output buffer asan >>> input string. >>> >>> src/jdk.jdwp.agent/share/native/libjdwp/debugInit.c line 645: >>> >>> (void)snprintf(buf, sizeof(buf), "JDWP %s", buf); >>> >>> Proposed fix is of course: >>> >>> (void)snprintf(buf, sizeof(buf), "JDWP %s", msg); >>> >>> This was found by compiling with GCC 8.1 >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8214061 >>> webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/stooke/JDK-8214061/01/webrev/ >> src/jdk.jdwp.agent/share/native/libjdwp/debugInit.c >> No comments. >> >> Thumbs up. > Can this be considered trivial enough to only require one reviewer? Yes. This is trivial so only one (R)eviewer is needed. > FWIW, this looks good to me too, but I'm not a Reviewer. That's okay. You can still be listed as a (r)eviewer. So now it has 2 folks... :-) Dan > > Thanks, > Severin > From sgehwolf at redhat.com Tue Dec 4 16:56:18 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Tue, 04 Dec 2018 17:56:18 +0100 Subject: RFR: (XS) 8214061: Buffer written into itself In-Reply-To: <6f8b0747-515f-52d4-443d-ea046d53a2b3@oracle.com> References: <7ea8d17d-0771-7211-cd39-965406d1ddbc@oracle.com> <6f8b0747-515f-52d4-443d-ea046d53a2b3@oracle.com> Message-ID: On Tue, 2018-12-04 at 11:42 -0500, Daniel D. Daugherty wrote: > On 12/4/18 11:02 AM, Severin Gehwolf wrote: > > Hi, > > > > On Tue, 2018-12-04 at 10:08 -0500, Daniel D. Daugherty wrote: > > > On 12/4/18 9:57 AM, Simon Tooke wrote: > > > > In one fatal error code path, snprintf() is given its output buffer asan > > > > input string. > > > > > > > > src/jdk.jdwp.agent/share/native/libjdwp/debugInit.c line 645: > > > > > > > > (void)snprintf(buf, sizeof(buf), "JDWP %s", buf); > > > > > > > > Proposed fix is of course: > > > > > > > > (void)snprintf(buf, sizeof(buf), "JDWP %s", msg); > > > > > > > > This was found by compiling with GCC 8.1 > > > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214061 > > > > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/stooke/JDK-8214061/01/webrev/ > > > > > > src/jdk.jdwp.agent/share/native/libjdwp/debugInit.c > > > No comments. > > > > > > Thumbs up. > > > > Can this be considered trivial enough to only require one reviewer? > > Yes. This is trivial so only one (R)eviewer is needed. > > > > FWIW, this looks good to me too, but I'm not a Reviewer. > > That's okay. You can still be listed as a (r)eviewer. So now it > has 2 folks... :-) OK. Thanks, Dan. Cheers, Severin From serguei.spitsyn at oracle.com Tue Dec 4 16:56:32 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 4 Dec 2018 08:56:32 -0800 Subject: RFR: (XS) 8214061: Buffer written into itself In-Reply-To: <6f8b0747-515f-52d4-443d-ea046d53a2b3@oracle.com> References: <7ea8d17d-0771-7211-cd39-965406d1ddbc@oracle.com> <6f8b0747-515f-52d4-443d-ea046d53a2b3@oracle.com> Message-ID: Hi Simon, Nice catch! Looks good. Copyright comment needs an update. Thanks, Serguei On 12/4/18 08:42, Daniel D. Daugherty wrote: > On 12/4/18 11:02 AM, Severin Gehwolf wrote: >> Hi, >> >> On Tue, 2018-12-04 at 10:08 -0500, Daniel D. Daugherty wrote: >>> On 12/4/18 9:57 AM, Simon Tooke wrote: >>>> In one fatal error code path, snprintf() is given its output buffer >>>> asan >>>> input string. >>>> >>>> src/jdk.jdwp.agent/share/native/libjdwp/debugInit.c line 645: >>>> >>>> ????? (void)snprintf(buf, sizeof(buf), "JDWP %s", buf); >>>> >>>> Proposed fix is of course: >>>> >>>> ????? (void)snprintf(buf, sizeof(buf), "JDWP %s", msg); >>>> >>>> This was found by compiling with GCC 8.1 >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8214061 >>>> webrev: >>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/stooke/JDK-8214061/01/webrev/ >>> src/jdk.jdwp.agent/share/native/libjdwp/debugInit.c >>> ????? No comments. >>> >>> Thumbs up. >> Can this be considered trivial enough to only require one reviewer? > > Yes. This is trivial so only one (R)eviewer is needed. > > >> FWIW, this looks good to me too, but I'm not a Reviewer. > > That's okay. You can still be listed as a (r)eviewer. So now it > has 2 folks... :-) > > Dan > > >> >> Thanks, >> Severin >> > From daniil.x.titov at oracle.com Tue Dec 4 18:24:17 2018 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Tue, 04 Dec 2018 10:24:17 -0800 Subject: RFR 8214572: nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the thread when the top frame executes JVMCI code In-Reply-To: <567a8f54-c9f1-7936-65c3-02d1c660dc70@oracle.com> References: <4E0EFD3E-F725-4469-BA20-7532D78BA479@oracle.com> <567a8f54-c9f1-7936-65c3-02d1c660dc70@oracle.com> Message-ID: <73D62FE8-DBD5-40AF-88B8-8397E0DAE2CA@oracle.com> Hi Serguei and JC, Thank you for reviewing this change. And many thanks to David and Dean for answering JVMCI questions. Please review a new version of the fix that moves the most of the new code in a helper method ( as JC suggested) and corrects error messages. I also excluded the changes in test/hotspot/jtreg/ProblemList-graal.txt from this webrev. Bug: https://bugs.openjdk.java.net/browse/JDK-8214572 Webrev: http://cr.openjdk.java.net/~dtitov/8214572/webrev.02/ Thanks, --Daniil From: "serguei.spitsyn at oracle.com" Date: Monday, December 3, 2018 at 4:14 PM To: Daniil Titov , serviceability-dev Subject: Re: RFR 8214572: nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the thread when the top frame executes JVMCI code Hi Daniil, It looks good in general. I have two comments though. -vmTestbase/nsk/jvmti/unit/ForceEarlyReturn/earlyretbase/TestDescription.java?????? 8195635?? generic-all It is not a good idea to remove the test from the ProblemList before the 8195635 is fixed. 148???? if(method == midActiveMethod) { 149???????? printf("<<<<<<<< SuspendThread() is successfully done\n"); 150???? } else { 151???????? printf("Warning: method \"activeMethod\" was missed\n"); 152???????? errCode = STATUS_FAILED; 153???? } I'd suggest to tweak the error message to something like: "Failed in the suspThread: was not able to suspend thread with the activeMethod() on top\n"); Thanks, Serguei From: JC Beyler Date: Friday, November 30, 2018 at 7:47 PM To: Cc: Subject: Re: RFR 8214572: nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the thread when the top frame executes JVMCI code Hi Daniil, The webrev looks good but I have a few comments and questions (if you don't mind :-)): Comments: - You say that normally the test will be removed from the problem list once the two fixes are done but in this webrev, you've already removed it (I can't see the other case so I can't see if it is resolved :-)) - now that we are in C++ for the tests, could we not declare the variables at their first use instead of doing the pedantic top of the block C style? - I feel that this sort of "wait until we are not in JVMCI frames" might happen a lot, maybe we could move that code into a helper method (+ it cleans the method a bit to just concentrate on the rest) and then if needed we can make it public to other tests? Questions because I'm not familiar with JVMCI consequences so not really comments on the webrev but so that I know: - Is it normaly that you can suspend when you are in a JVMCI frame? will/is there not a better way that we could detect that we are in a JVMCI frame? Is it always safe to suspend a JVMCI frame? Thanks! Jc On 11/30/18 19:08, Daniil Titov wrote: Please review the change for nsk/jvmti/unit/ForceEarlyReturn/earlyretbase test. The problem here is that before doing an early force return the test doesn't check that the test thread is suspended at a right place where the top frame executes the test method rather than JVMCI code triggered by invocation counter overflow. That results in the early return happens for a wrong method and the test fails. The fix changes the test to do resume/suspend in the loop until the target method is executed in the top frame or the loop counter exceeds the limit. There is another problem with this test that requires changes on VM side and is currently covered by JDK-8195635:" [Graal] nsk/jvmti/unit/ForceEarlyReturn/earlyretbase crashes with assertion "compilation level out of bounds"".? The test will have to be removed from test/hotspot/jtreg/ProblemList-graal.txt only after both these issues are fixed. Bug: https://bugs.openjdk.java.net/browse/JDK-8214572 Webrev: http://cr.openjdk.java.net/~dtitov/8214572/webrev.01/ Thanks, Daniil -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Tue Dec 4 19:40:37 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 4 Dec 2018 11:40:37 -0800 Subject: RFR 8214572: nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the thread when the top frame executes JVMCI code In-Reply-To: <73D62FE8-DBD5-40AF-88B8-8397E0DAE2CA@oracle.com> References: <4E0EFD3E-F725-4469-BA20-7532D78BA479@oracle.com> <567a8f54-c9f1-7936-65c3-02d1c660dc70@oracle.com> <73D62FE8-DBD5-40AF-88B8-8397E0DAE2CA@oracle.com> Message-ID: <633bf1e4-5423-76df-32bb-0f4470e905e0@oracle.com> Hi Daniil, It looks good in general. Thank you for the update! I have some minor comment though. http://cr.openjdk.java.net/%7Edtitov/8214572/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_tools.h.udiff.html +/** +* This method suspends the thread while ensuring the top frame executes the test method +* rather then JVMCI code triggered by invocation counter overflow. +*/ +int suspendThreadAtMethod(jvmtiEnv *jvmti, jclass cls, jobject thread, jmethodID method); The comment above is not precise as it tells nothing about top frame. I like this one from implementation: + // We need to ensure that the thread is suspended at the right place + // when the top frame belongs to the test rather then to JVMCI code. So, the can be rephrased to something like: + // This method makes the thread to be suspended at the right place + // when the top frame belongs to the test rather then to JVMCI code. No need in another webrev if you fix the comment. Thanks, Serguei On 12/4/18 10:24 AM, Daniil Titov wrote: > > Hi Serguei and JC, > > Thank you for reviewing this change. And many thanks to David and Dean > for answering JVMCI questions. > > Please review a new version of the fix that moves the most of the new > code in a helper method ( as JC suggested) and corrects error > messages. I also excluded the changes in > test/hotspot/jtreg/ProblemList-graal.txt from this webrev. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214572 > > Webrev:http://cr.openjdk.java.net/~dtitov/8214572/webrev.02/ > > > Thanks, > > --Daniil > > *From: *"serguei.spitsyn at oracle.com" > *Date: *Monday, December 3, 2018 at 4:14 PM > *To: *Daniil Titov , serviceability-dev > > *Subject: *Re: RFR 8214572: > nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the > thread when the top frame executes JVMCI code > > Hi Daniil, > > It looks good in general. > > I have two comments though. > > -vmTestbase/nsk/jvmti/unit/ForceEarlyReturn/earlyretbase/TestDescription.java?????? > 8195635?? generic-all > > ? It is not a good idea to remove the test from the ProblemList before > the 8195635 is fixed. > > 148???? if(method == midActiveMethod) { > 149???????? printf("<<<<<<<< SuspendThread() is successfully done\n"); > 150???? } else { > 151???????? printf("Warning: method \"activeMethod\" was missed\n"); > 152???????? errCode = STATUS_FAILED; > 153???? } > > ?I'd suggest to tweak the error message to something like: > ?? "Failed in the suspThread: was not able to suspend thread with the > activeMethod() on top\n"); > > > Thanks, > Serguei > > *From: *JC Beyler > *Date: *Friday, November 30, 2018 at 7:47 PM > *To: * > *Cc: * > *Subject: *Re: RFR 8214572: > nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the > thread when the top frame executes JVMCI code > > Hi Daniil, > > The webrev looks good but I have a few comments and questions (if you > don't mind :-)): > > Comments: > > ? - You say that normally the test will be removed from the problem > list once the two fixes are done but in this webrev, you've already > removed it (I can't see the other case so I can't see if it is > resolved :-)) > > ? - now that we are in C++ for the tests, could we not declare the > variables at their first use instead of doing the pedantic top of the > block C style? > > ? - I feel that this sort of "wait until we are not in JVMCI frames" > might happen a lot, maybe we could move that code into a helper method > (+ it cleans the method a bit to just concentrate on the rest) and > then if needed we can make it public to other tests? > > Questions because I'm not familiar with JVMCI consequences so not > really comments on the webrev but so that I know: > > ? - Is it normaly that you can suspend when you are in a JVMCI frame? > will/is there not a better way that we could detect that we are in a > JVMCI frame? Is it always safe to suspend a JVMCI frame? > > Thanks! > > Jc > > > > > On 11/30/18 19:08, Daniil Titov wrote: > > Please review the change for nsk/jvmti/unit/ForceEarlyReturn/earlyretbase test. The problem here is that before doing an early force return the test doesn't check that the test thread is suspended at a right place where the top frame executes the test method rather than JVMCI code triggered by invocation counter overflow. That results in the early return happens for a wrong method and the test fails. > > The fix changes the test to do resume/suspend in the loop until the target method is executed in the top frame or the loop counter exceeds the limit. > > There is another problem with this test that requires changes on VM side and is currently covered by JDK-8195635:" [Graal] nsk/jvmti/unit/ForceEarlyReturn/earlyretbase crashes with assertion "compilation level out of bounds"".? The test will have to be removed from test/hotspot/jtreg/ProblemList-graal.txt only after both these issues are fixed. > > Bug:https://bugs.openjdk.java.net/browse/JDK-8214572 > > Webrev:http://cr.openjdk.java.net/~dtitov/8214572/webrev.01/ > > > Thanks, > > Daniil > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From leonid.mesnik at oracle.com Tue Dec 4 20:03:14 2018 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Tue, 4 Dec 2018 12:03:14 -0800 Subject: RFR (round 5), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <8c267450-9ca8-79a6-0e4a-39b3b68af2e3@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <8c267450-9ca8-79a6-0e4a-39b3b68af2e3@redhat.com> Message-ID: <5788143E-E206-4795-B7B8-B47FDDF3B54F@oracle.com> Hi The shared tests changes looks good for me. Thank you for fixing and testing different combinations. Leonid > On Dec 3, 2018, at 11:10 PM, Roman Kennke wrote: > > Round 5 of Shenandoah review includes: > - A fix for the @requires tag in TestFullGCCountTest.java. It should be > correct now. We believe the CMS @requires was also not quite right and > fixed it the same. > > It reads now: Don't run this test if: > - Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is > true, as set by harness > - Actual GC set by harness is Shenandoah *and* > ExplicitGCInvokesConcurrent is not set false by harness (it's true by > default in Shenandoah, so this needs to be double-inverteed). > > The @requires for CMS was wrong before (we think), because it would also > filter defaultGC + ExplicitGCInvokesConcurrent. > > - Sorting of macros was fixed, as was pointed out by Per > - Some stuff was added to SA, as suggested by Jini > - Rebased on most current jdk/jdk code > > Webrevs: > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/ > > I also need reviews from GC reviewers for the CSR: > https://bugs.openjdk.java.net/browse/JDK-8214349 > > I already got reviews for: > [x] shared-runtime (coleenp) > [x] shared-compiler (kvn) > > I got reviews for shared-build, but an earlier version, so maybe makes > sense to look over this again. Erik J, Magnus? > > I still need approvals for: > [ ] shared-build (kvn, erikj, ihse, pliden) > [ ] shared-gc (pliden, kbarrett) > [ ] shared-serviceability (jgeorge, pliden) > [ ] shared-tests (lmesnik, pliden) > [ ] shenandoah-gc > [ ] shenandoah-tests > > > Thanks for your patience and ongoing support! > > Cheers, > Roman > > >> Hi all, >> >> here comes round 4 of Shenandoah upstreaming review: >> >> This includes fixes for the issues that Per brought up: >> - Verify and gracefully reject dangerous flags combinations that >> disables required barriers >> - Revisited @requires filters in tests >> - Trim unused code from Shenandoah's SA impl >> - Move ShenandoahGCTracer to gc/shenandoah >> - Fix ordering of GC names in various files >> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >> >> Thanks everybody for taking time to review this! >> Roman >> >>> Hello all, >>> >>> Thanks so far for all the reviews and support! >>> >>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >>> >>> Things we've changed today: >>> - We moved shenandoah-specific code out of .ad files into our own .ad >>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>> requires an addition in build machinery though, see >>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>> - Improved zero-disabling and build-code-simplification as suggested by >>> Magnus and Per >>> - Cleaned up some leftovers in C2 >>> - Improved C2 loop opts code by introducing another APIs in >>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now. >>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>> that. >>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>> - Rebased on jdk-12+22 >>> >>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>> >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>> >>> Thanks, >>> Roman >>> >>>> Alright, we fixed: >>>> - The minor issues that Kim reported in shared-gc >>>> - A lot of fixes in shared-tests according to Leonid's review >>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>> >>>> Some notes: >>>> Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>> correct. The @requires there means to exclude runs with both CMS and >>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>> fine. Adding the same clause for Shenandoah means the same, and filters >>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>> made the condition a bit clearer by avoiding triple-negation. >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>> >>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>> >>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>> those with ZGC? >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>> >>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>> next round). >>>> >>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>> better. I can tell that we're not done with C2 yet. Can you look over >>>> the code and see what is ok, and especially what is not ok, so that we >>>> can focus our efforts on the relevant parts? >>>> >>>> Updated set of webrevs: >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>> >>>> Thanks, >>>> Roman >>>> >>>> >>>>> Hi, >>>>> >>>>> This is the first round of changes for including Shenandoah GC into >>>>> mainline. >>>>> I divided the review into parts that roughly correspond to the mailing lists >>>>> that would normally review it, and I divided it into 'shared' code >>>>> changes and >>>>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>>>> eventually >>>>> push them as single 'combined' changeset, once reviewed. >>>>> >>>>> JEP: >>>>> https://openjdk.java.net/jeps/189 >>>>> Bug entry: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>> >>>>> Webrevs: >>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>> >>>>> For those who want to see the full change, have a look at the >>>>> shenandoah-complete >>>>> >>>>> directory, >>>>> it contains the full combined webrev. Alternatively, there is the file >>>>> shenandoah-master.patch >>>>> , >>>>> which is what I intend to commit (and which should be equivalent to the >>>>> 'shenandoah-complete' webrev). >>>>> >>>>> Sections to review (at this point) are the following: >>>>> *) shenandoah-gc >>>>> >>>>> - Actual Shenandoah implementation, almost completely residing in >>>>> gc/shenandoah >>>>> >>>>> *) shared-gc >>>>> >>>>> - This is mostly boilerplate that is common to any GC >>>>> - referenceProcessor.cpp has a little change to make one assert not >>>>> fail (next to CMS and G1) >>>>> - taskqueue.hpp has some small adjustments to enable subclassing >>>>> >>>>> *) shared-serviceability >>>>> >>>>> - The usual code to support another GC >>>>> >>>>> *) shared-runtime >>>>> >>>>> - A number of friends declarations to allow Shenandoah iterators to >>>>> hook up with, >>>>> e.g. ClassLoaderData, CodeCache, etc >>>>> - Warning and disabling JFR LeakProfiler >>>>> - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>> Shenandoah C2 optimizations >>>>> - Locks initialization in mutexLocker.cpp as usual >>>>> - VM operations defines for Shenandoah's VM ops >>>>> - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>> Shenandoah's logging >>>>> - The usual macros in macro.hpp >>>>> >>>>> *) shared-build >>>>> >>>>> - Add shenandoah feature, enabled by default, as agreed with >>>>> Vladimir K. beforehand >>>>> - Some flags for shenandoah-enabled compilation to get >>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>> and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>> Shenandoah's barriers >>>>> - --param inline-unit-growth=1000 settings for 2 shenandoah source >>>>> files, which is >>>>> useful to get the whole marking loop inlined (observed significant >>>>> regression if we >>>>> don't) >>>>> >>>>> *) shared-tests >>>>> >>>>> - Test infrastructure to support Shenandoah >>>>> - Shenandoah test groups >>>>> - Exclude Shenandoah in various tests that can be run with selected GC >>>>> - Enable/add configure for Shenandoah for tests that make sense to >>>>> run with it >>>>> >>>>> *) shenandoah-tests >>>>> >>>>> - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >>>>> - A couple of tests configurations have been added, e.g. >>>>> TestGCBasherWithShenandoah.java >>>>> >>>>> I intentionally left out shared-compiler for now, because we have some >>>>> work left to do >>>>> there, but if you click around you'll find the patch anyway, in case you >>>>> want to take >>>>> a peek at it. >>>>> >>>>> We have regular builds on: >>>>> - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>> - {Windows} x {x86_64}, >>>>> - {MacOS X} x {x86_64} >>>>> >>>>> This also routinely passes: >>>>> - the new Shenandoah tests >>>>> - jcstress with/without aggressive Shenandoah verification >>>>> - specjvm2008 with/without aggressive Shenandoah verification >>>>> >>>>> >>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>> teams for tirelessly helping with and reviewing all the GC interface and >>>>> related changes, and of course the many early adopters for reporting >>>>> bugs and success stories and feature requests: we wouldn't be here >>>>> without any of you! >>>>> >>>>> Best regards, >>>>> Roman >>>>> >>>> >>> >> > From rkennke at redhat.com Tue Dec 4 20:37:49 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 4 Dec 2018 21:37:49 +0100 Subject: RFR (round 5), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <5788143E-E206-4795-B7B8-B47FDDF3B54F@oracle.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <8c267450-9ca8-79a6-0e4a-39b3b68af2e3@redhat.com> <5788143E-E206-4795-B7B8-B47FDDF3B54F@oracle.com> Message-ID: Thanks, Leonid, for reviewing! Roman > Hi > > The shared tests changes looks good for me. Thank you for fixing and testing different combinations. > > Leonid > >> On Dec 3, 2018, at 11:10 PM, Roman Kennke wrote: >> >> Round 5 of Shenandoah review includes: >> - A fix for the @requires tag in TestFullGCCountTest.java. It should be >> correct now. We believe the CMS @requires was also not quite right and >> fixed it the same. >> >> It reads now: Don't run this test if: >> - Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is >> true, as set by harness >> - Actual GC set by harness is Shenandoah *and* >> ExplicitGCInvokesConcurrent is not set false by harness (it's true by >> default in Shenandoah, so this needs to be double-inverteed). >> >> The @requires for CMS was wrong before (we think), because it would also >> filter defaultGC + ExplicitGCInvokesConcurrent. >> >> - Sorting of macros was fixed, as was pointed out by Per >> - Some stuff was added to SA, as suggested by Jini >> - Rebased on most current jdk/jdk code >> >> Webrevs: >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/ >> >> I also need reviews from GC reviewers for the CSR: >> https://bugs.openjdk.java.net/browse/JDK-8214349 >> >> I already got reviews for: >> [x] shared-runtime (coleenp) >> [x] shared-compiler (kvn) >> >> I got reviews for shared-build, but an earlier version, so maybe makes >> sense to look over this again. Erik J, Magnus? >> >> I still need approvals for: >> [ ] shared-build (kvn, erikj, ihse, pliden) >> [ ] shared-gc (pliden, kbarrett) >> [ ] shared-serviceability (jgeorge, pliden) >> [ ] shared-tests (lmesnik, pliden) >> [ ] shenandoah-gc >> [ ] shenandoah-tests >> >> >> Thanks for your patience and ongoing support! >> >> Cheers, >> Roman >> >> >>> Hi all, >>> >>> here comes round 4 of Shenandoah upstreaming review: >>> >>> This includes fixes for the issues that Per brought up: >>> - Verify and gracefully reject dangerous flags combinations that >>> disables required barriers >>> - Revisited @requires filters in tests >>> - Trim unused code from Shenandoah's SA impl >>> - Move ShenandoahGCTracer to gc/shenandoah >>> - Fix ordering of GC names in various files >>> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >>> >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >>> >>> Thanks everybody for taking time to review this! >>> Roman >>> >>>> Hello all, >>>> >>>> Thanks so far for all the reviews and support! >>>> >>>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>>> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >>>> >>>> Things we've changed today: >>>> - We moved shenandoah-specific code out of .ad files into our own .ad >>>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>>> requires an addition in build machinery though, see >>>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>>> - Improved zero-disabling and build-code-simplification as suggested by >>>> Magnus and Per >>>> - Cleaned up some leftovers in C2 >>>> - Improved C2 loop opts code by introducing another APIs in >>>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now. >>>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>>> that. >>>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>>> - Rebased on jdk-12+22 >>>> >>>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>>> >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>>> >>>> Thanks, >>>> Roman >>>> >>>>> Alright, we fixed: >>>>> - The minor issues that Kim reported in shared-gc >>>>> - A lot of fixes in shared-tests according to Leonid's review >>>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>>> >>>>> Some notes: >>>>> Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>>> correct. The @requires there means to exclude runs with both CMS and >>>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>>> fine. Adding the same clause for Shenandoah means the same, and filters >>>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>>> made the condition a bit clearer by avoiding triple-negation. >>>>> >>>>> See: >>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>>> >>>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>>> >>>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>>> those with ZGC? >>>>> >>>>> See: >>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>>> >>>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>>> next round). >>>>> >>>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>>> better. I can tell that we're not done with C2 yet. Can you look over >>>>> the code and see what is ok, and especially what is not ok, so that we >>>>> can focus our efforts on the relevant parts? >>>>> >>>>> Updated set of webrevs: >>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>>> >>>>> Thanks, >>>>> Roman >>>>> >>>>> >>>>>> Hi, >>>>>> >>>>>> This is the first round of changes for including Shenandoah GC into >>>>>> mainline. >>>>>> I divided the review into parts that roughly correspond to the mailing lists >>>>>> that would normally review it, and I divided it into 'shared' code >>>>>> changes and >>>>>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>>>>> eventually >>>>>> push them as single 'combined' changeset, once reviewed. >>>>>> >>>>>> JEP: >>>>>> https://openjdk.java.net/jeps/189 >>>>>> Bug entry: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>>> >>>>>> Webrevs: >>>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>>> >>>>>> For those who want to see the full change, have a look at the >>>>>> shenandoah-complete >>>>>> >>>>>> directory, >>>>>> it contains the full combined webrev. Alternatively, there is the file >>>>>> shenandoah-master.patch >>>>>> , >>>>>> which is what I intend to commit (and which should be equivalent to the >>>>>> 'shenandoah-complete' webrev). >>>>>> >>>>>> Sections to review (at this point) are the following: >>>>>> *) shenandoah-gc >>>>>> >>>>>> - Actual Shenandoah implementation, almost completely residing in >>>>>> gc/shenandoah >>>>>> >>>>>> *) shared-gc >>>>>> >>>>>> - This is mostly boilerplate that is common to any GC >>>>>> - referenceProcessor.cpp has a little change to make one assert not >>>>>> fail (next to CMS and G1) >>>>>> - taskqueue.hpp has some small adjustments to enable subclassing >>>>>> >>>>>> *) shared-serviceability >>>>>> >>>>>> - The usual code to support another GC >>>>>> >>>>>> *) shared-runtime >>>>>> >>>>>> - A number of friends declarations to allow Shenandoah iterators to >>>>>> hook up with, >>>>>> e.g. ClassLoaderData, CodeCache, etc >>>>>> - Warning and disabling JFR LeakProfiler >>>>>> - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>>> Shenandoah C2 optimizations >>>>>> - Locks initialization in mutexLocker.cpp as usual >>>>>> - VM operations defines for Shenandoah's VM ops >>>>>> - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>>> Shenandoah's logging >>>>>> - The usual macros in macro.hpp >>>>>> >>>>>> *) shared-build >>>>>> >>>>>> - Add shenandoah feature, enabled by default, as agreed with >>>>>> Vladimir K. beforehand >>>>>> - Some flags for shenandoah-enabled compilation to get >>>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>>> and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>>> Shenandoah's barriers >>>>>> - --param inline-unit-growth=1000 settings for 2 shenandoah source >>>>>> files, which is >>>>>> useful to get the whole marking loop inlined (observed significant >>>>>> regression if we >>>>>> don't) >>>>>> >>>>>> *) shared-tests >>>>>> >>>>>> - Test infrastructure to support Shenandoah >>>>>> - Shenandoah test groups >>>>>> - Exclude Shenandoah in various tests that can be run with selected GC >>>>>> - Enable/add configure for Shenandoah for tests that make sense to >>>>>> run with it >>>>>> >>>>>> *) shenandoah-tests >>>>>> >>>>>> - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >>>>>> - A couple of tests configurations have been added, e.g. >>>>>> TestGCBasherWithShenandoah.java >>>>>> >>>>>> I intentionally left out shared-compiler for now, because we have some >>>>>> work left to do >>>>>> there, but if you click around you'll find the patch anyway, in case you >>>>>> want to take >>>>>> a peek at it. >>>>>> >>>>>> We have regular builds on: >>>>>> - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>>> - {Windows} x {x86_64}, >>>>>> - {MacOS X} x {x86_64} >>>>>> >>>>>> This also routinely passes: >>>>>> - the new Shenandoah tests >>>>>> - jcstress with/without aggressive Shenandoah verification >>>>>> - specjvm2008 with/without aggressive Shenandoah verification >>>>>> >>>>>> >>>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>>> teams for tirelessly helping with and reviewing all the GC interface and >>>>>> related changes, and of course the many early adopters for reporting >>>>>> bugs and success stories and feature requests: we wouldn't be here >>>>>> without any of you! >>>>>> >>>>>> Best regards, >>>>>> Roman >>>>>> >>>>> >>>> >>> >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From jcbeyler at google.com Tue Dec 4 21:15:52 2018 From: jcbeyler at google.com (JC Beyler) Date: Tue, 4 Dec 2018 13:15:52 -0800 Subject: RFR 8214572: nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the thread when the top frame executes JVMCI code In-Reply-To: <633bf1e4-5423-76df-32bb-0f4470e905e0@oracle.com> References: <4E0EFD3E-F725-4469-BA20-7532D78BA479@oracle.com> <567a8f54-c9f1-7936-65c3-02d1c660dc70@oracle.com> <73D62FE8-DBD5-40AF-88B8-8397E0DAE2CA@oracle.com> <633bf1e4-5423-76df-32bb-0f4470e905e0@oracle.com> Message-ID: Hi Daniil, Looks good to me! Potential Nit: Any reason the comment above the loop in http://cr.openjdk.java.net/~dtitov/8214572/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_tools.cpp.udiff.html is of a different indentation? Thanks, Jc On Tue, Dec 4, 2018 at 11:40 AM wrote: > Hi Daniil, > > It looks good in general. > Thank you for the update! > > I have some minor comment though. > > > http://cr.openjdk.java.net/%7Edtitov/8214572/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_tools.h.udiff.html > > +/**+* This method suspends the thread while ensuring the top frame executes the test method+* rather then JVMCI code triggered by invocation counter overflow.+*/+int suspendThreadAtMethod(jvmtiEnv *jvmti, jclass cls, jobject thread, jmethodID method); > > > The comment above is not precise as it tells nothing about top frame. > > I like this one from implementation: > > + // We need to ensure that the thread is suspended at the right place+ // when the top frame belongs to the test rather then to JVMCI code. > > > So, the can be rephrased to something like: > > + // This method makes the thread to be suspended at the right place+ // when the top frame belongs to the test rather then to JVMCI code. > > > > No need in another webrev if you fix the comment. > > Thanks, > Serguei > > > On 12/4/18 10:24 AM, Daniil Titov wrote: > > Hi Serguei and JC, > > > > Thank you for reviewing this change. And many thanks to David and Dean for > answering JVMCI questions. > > > > Please review a new version of the fix that moves the most of the new code > in a helper method ( as JC suggested) and corrects error messages. I also > excluded the changes in test/hotspot/jtreg/ProblemList-graal.txt from this > webrev. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214572 > > Webrev: http://cr.openjdk.java.net/~dtitov/8214572/webrev.02/ > > > > Thanks, > > --Daniil > > > > *From: *"serguei.spitsyn at oracle.com" > > *Date: *Monday, December 3, 2018 at 4:14 PM > *To: *Daniil Titov , > serviceability-dev > > *Subject: *Re: RFR 8214572: nsk/jvmti/unit/ForceEarlyReturn/earlyretbase > should not suspend the thread when the top frame executes JVMCI code > > > > Hi Daniil, > > It looks good in general. > > I have two comments though. > > -vmTestbase/nsk/jvmti/unit/ForceEarlyReturn/earlyretbase/TestDescription.java 8195635 generic-all > > It is not a good idea to remove the test from the ProblemList before the > 8195635 is fixed. > > 148 if(method == midActiveMethod) { > > 149 printf("<<<<<<<< SuspendThread() is successfully done\n"); > > 150 } else { > > 151 printf("Warning: method \"activeMethod\" was missed\n"); > > 152 errCode = STATUS_FAILED; > > 153 } > > I'd suggest to tweak the error message to something like: > "Failed in the suspThread: was not able to suspend thread with the > activeMethod() on top\n"); > > > Thanks, > Serguei > > *From: *JC Beyler > *Date: *Friday, November 30, 2018 at 7:47 PM > *To: * > *Cc: * > > *Subject: *Re: RFR 8214572: nsk/jvmti/unit/ForceEarlyReturn/earlyretbase > should not suspend the thread when the top frame executes JVMCI code > > > > Hi Daniil, > > > > The webrev looks good but I have a few comments and questions (if you > don't mind :-)): > > > > Comments: > > - You say that normally the test will be removed from the problem list > once the two fixes are done but in this webrev, you've already removed it > (I can't see the other case so I can't see if it is resolved :-)) > > - now that we are in C++ for the tests, could we not declare the > variables at their first use instead of doing the pedantic top of the block > C style? > > - I feel that this sort of "wait until we are not in JVMCI frames" might > happen a lot, maybe we could move that code into a helper method (+ it > cleans the method a bit to just concentrate on the rest) and then if needed > we can make it public to other tests? > > > > Questions because I'm not familiar with JVMCI consequences so not really > comments on the webrev but so that I know: > > - Is it normaly that you can suspend when you are in a JVMCI frame? > will/is there not a better way that we could detect that we are in a JVMCI > frame? Is it always safe to suspend a JVMCI frame? > > > > Thanks! > > Jc > > > > > > > On 11/30/18 19:08, Daniil Titov wrote: > > Please review the change for nsk/jvmti/unit/ForceEarlyReturn/earlyretbase test. The problem here is that before doing an early force return the test doesn't check that the test thread is suspended at a right place where the top frame executes the test method rather than JVMCI code triggered by invocation counter overflow. That results in the early return happens for a wrong method and the test fails. > > > > The fix changes the test to do resume/suspend in the loop until the target method is executed in the top frame or the loop counter exceeds the limit. > > > > There is another problem with this test that requires changes on VM side and is currently covered by JDK-8195635:" [Graal] nsk/jvmti/unit/ForceEarlyReturn/earlyretbase crashes with assertion "compilation level out of bounds"". The test will have to be removed from test/hotspot/jtreg/ProblemList-graal.txt only after both these issues are fixed. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214572 > > Webrev: http://cr.openjdk.java.net/~dtitov/8214572/webrev.01/ > > > > Thanks, > > Daniil > > > > > > > > > > > > > > > > > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniil.x.titov at oracle.com Tue Dec 4 21:46:08 2018 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Tue, 04 Dec 2018 13:46:08 -0800 Subject: RFR 8214572: nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the thread when the top frame executes JVMCI code In-Reply-To: References: <4E0EFD3E-F725-4469-BA20-7532D78BA479@oracle.com> <567a8f54-c9f1-7936-65c3-02d1c660dc70@oracle.com> <73D62FE8-DBD5-40AF-88B8-8397E0DAE2CA@oracle.com> <633bf1e4-5423-76df-32bb-0f4470e905e0@oracle.com> Message-ID: Thank you, JC and Serguei, I will correct both comments before pushing the change. Best regards, Daniil From: JC Beyler Date: Tuesday, December 4, 2018 at 1:16 PM To: Cc: , , David Holmes , Subject: Re: RFR 8214572: nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the thread when the top frame executes JVMCI code Hi Daniil, Looks good to me! Potential Nit: Any reason the comment above the loop in http://cr.openjdk.java.net/~dtitov/8214572/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_tools.cpp.udiff.html is of a different indentation? Thanks, Jc On Tue, Dec 4, 2018 at 11:40 AM wrote: Hi Daniil, It looks good in general. Thank you for the update! I have some minor comment though. http://cr.openjdk.java.net/%7Edtitov/8214572/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_tools.h.udiff.html +/** +* This method suspends the thread while ensuring the top frame executes the test method +* rather then JVMCI code triggered by invocation counter overflow. +*/ +int suspendThreadAtMethod(jvmtiEnv *jvmti, jclass cls, jobject thread, jmethodID method); The comment above is not precise as it tells nothing about top frame. I like this one from implementation: +??????? // We need to ensure that the thread is suspended at the right place +??????? // when the top frame belongs to the test rather then to JVMCI code. So, the can be rephrased to something like: +??????? // This method makes the thread to be suspended at the right place +??????? // when the top frame belongs to the test rather then to JVMCI code. No need in another webrev if you fix the comment. Thanks, Serguei On 12/4/18 10:24 AM, Daniil Titov wrote: Hi Serguei and JC, Thank you for reviewing this change. And many thanks to David and Dean for answering JVMCI questions. Please review a new version of the fix that moves the most of the new code in a helper method ( as JC suggested) and corrects error messages. I also excluded the changes in test/hotspot/jtreg/ProblemList-graal.txt from this webrev. Bug: https://bugs.openjdk.java.net/browse/JDK-8214572 Webrev: http://cr.openjdk.java.net/~dtitov/8214572/webrev.02/ Thanks, --Daniil From: "serguei.spitsyn at oracle.com" Date: Monday, December 3, 2018 at 4:14 PM To: Daniil Titov , serviceability-dev Subject: Re: RFR 8214572: nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the thread when the top frame executes JVMCI code Hi Daniil, It looks good in general. I have two comments though. -vmTestbase/nsk/jvmti/unit/ForceEarlyReturn/earlyretbase/TestDescription.java 8195635 generic-all It is not a good idea to remove the test from the ProblemList before the 8195635 is fixed. 148 if(method == midActiveMethod) { 149 printf("<<<<<<<< SuspendThread() is successfully done\n"); 150 } else { 151 printf("Warning: method \"activeMethod\" was missed\n"); 152 errCode = STATUS_FAILED; 153 } I'd suggest to tweak the error message to something like: "Failed in the suspThread: was not able to suspend thread with the activeMethod() on top\n"); Thanks, Serguei From: JC Beyler Date: Friday, November 30, 2018 at 7:47 PM To: Cc: Subject: Re: RFR 8214572: nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the thread when the top frame executes JVMCI code Hi Daniil, The webrev looks good but I have a few comments and questions (if you don't mind :-)): Comments: - You say that normally the test will be removed from the problem list once the two fixes are done but in this webrev, you've already removed it (I can't see the other case so I can't see if it is resolved :-)) - now that we are in C++ for the tests, could we not declare the variables at their first use instead of doing the pedantic top of the block C style? - I feel that this sort of "wait until we are not in JVMCI frames" might happen a lot, maybe we could move that code into a helper method (+ it cleans the method a bit to just concentrate on the rest) and then if needed we can make it public to other tests? Questions because I'm not familiar with JVMCI consequences so not really comments on the webrev but so that I know: - Is it normaly that you can suspend when you are in a JVMCI frame? will/is there not a better way that we could detect that we are in a JVMCI frame? Is it always safe to suspend a JVMCI frame? Thanks! Jc On 11/30/18 19:08, Daniil Titov wrote: Please review the change for nsk/jvmti/unit/ForceEarlyReturn/earlyretbase test. The problem here is that before doing an early force return the test doesn't check that the test thread is suspended at a right place where the top frame executes the test method rather than JVMCI code triggered by invocation counter overflow. That results in the early return happens for a wrong method and the test fails. The fix changes the test to do resume/suspend in the loop until the target method is executed in the top frame or the loop counter exceeds the limit. There is another problem with this test that requires changes on VM side and is currently covered by JDK-8195635:" [Graal] nsk/jvmti/unit/ForceEarlyReturn/earlyretbase crashes with assertion "compilation level out of bounds"". The test will have to be removed from test/hotspot/jtreg/ProblemList-graal.txt only after both these issues are fixed. Bug: https://bugs.openjdk.java.net/browse/JDK-8214572 Webrev: http://cr.openjdk.java.net/~dtitov/8214572/webrev.01/ Thanks, Daniil -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Wed Dec 5 00:54:00 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 5 Dec 2018 10:54:00 +1000 Subject: RFR 8214572: nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the thread when the top frame executes JVMCI code In-Reply-To: <633bf1e4-5423-76df-32bb-0f4470e905e0@oracle.com> References: <4E0EFD3E-F725-4469-BA20-7532D78BA479@oracle.com> <567a8f54-c9f1-7936-65c3-02d1c660dc70@oracle.com> <73D62FE8-DBD5-40AF-88B8-8397E0DAE2CA@oracle.com> <633bf1e4-5423-76df-32bb-0f4470e905e0@oracle.com> Message-ID: Hi everyone, I'd actually argue that the comment not refer just to JVMCI but more generally: + // when the top frame belongs to the test rather than to incidental Java code (classloading, JVMCI, etc) Also note typo: then -> than Cheers, David On 5/12/2018 5:40 am, serguei.spitsyn at oracle.com wrote: > Hi Daniil, > > It looks good in general. > Thank you for the update! > > I have some minor comment though. > > http://cr.openjdk.java.net/%7Edtitov/8214572/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_tools.h.udiff.html > > +/** > +* This method suspends the thread while ensuring the top frame executes > the test method > +* rather then JVMCI code triggered by invocation counter overflow. > +*/ > +int suspendThreadAtMethod(jvmtiEnv *jvmti, jclass cls, jobject thread, > jmethodID method); > > > The comment above is not precise as it tells nothing about top frame. > > I like this one from implementation: > > + // We need to ensure that the thread is suspended at the right place > + // when the top frame belongs to the test rather then to JVMCI code. > > > So, the can be rephrased to something like: > > + // This method makes the thread to be suspended at the right place > + // when the top frame belongs to the test rather then to JVMCI code. > > > > No need in another webrev if you fix the comment. > > Thanks, > Serguei > > > On 12/4/18 10:24 AM, Daniil Titov wrote: >> >> Hi Serguei and JC, >> >> Thank you for reviewing this change. And many thanks to David and Dean >> for answering JVMCI questions. >> >> Please review a new version of the fix that moves the most of the new >> code in a helper method ( as JC suggested) and corrects error >> messages. I also excluded the changes in >> test/hotspot/jtreg/ProblemList-graal.txt from this webrev. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8214572 >> >> Webrev:http://cr.openjdk.java.net/~dtitov/8214572/webrev.02/ >> >> >> Thanks, >> >> --Daniil >> >> *From: *"serguei.spitsyn at oracle.com" >> *Date: *Monday, December 3, 2018 at 4:14 PM >> *To: *Daniil Titov , serviceability-dev >> >> *Subject: *Re: RFR 8214572: >> nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the >> thread when the top frame executes JVMCI code >> >> Hi Daniil, >> >> It looks good in general. >> >> I have two comments though. >> >> -vmTestbase/nsk/jvmti/unit/ForceEarlyReturn/earlyretbase/TestDescription.java >> 8195635?? generic-all >> >> ? It is not a good idea to remove the test from the ProblemList before >> the 8195635 is fixed. >> >> 148???? if(method == midActiveMethod) { >> 149???????? printf("<<<<<<<< SuspendThread() is successfully done\n"); >> 150???? } else { >> 151???????? printf("Warning: method \"activeMethod\" was missed\n"); >> 152???????? errCode = STATUS_FAILED; >> 153???? } >> >> ?I'd suggest to tweak the error message to something like: >> ?? "Failed in the suspThread: was not able to suspend thread with the >> activeMethod() on top\n"); >> >> >> Thanks, >> Serguei >> >> *From: *JC Beyler >> *Date: *Friday, November 30, 2018 at 7:47 PM >> *To: * >> *Cc: * >> *Subject: *Re: RFR 8214572: >> nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the >> thread when the top frame executes JVMCI code >> >> Hi Daniil, >> >> The webrev looks good but I have a few comments and questions (if you >> don't mind :-)): >> >> Comments: >> >> ? - You say that normally the test will be removed from the problem >> list once the two fixes are done but in this webrev, you've already >> removed it (I can't see the other case so I can't see if it is >> resolved :-)) >> >> ? - now that we are in C++ for the tests, could we not declare the >> variables at their first use instead of doing the pedantic top of the >> block C style? >> >> ? - I feel that this sort of "wait until we are not in JVMCI frames" >> might happen a lot, maybe we could move that code into a helper method >> (+ it cleans the method a bit to just concentrate on the rest) and >> then if needed we can make it public to other tests? >> >> Questions because I'm not familiar with JVMCI consequences so not >> really comments on the webrev but so that I know: >> >> ? - Is it normaly that you can suspend when you are in a JVMCI frame? >> will/is there not a better way that we could detect that we are in a >> JVMCI frame? Is it always safe to suspend a JVMCI frame? >> >> Thanks! >> >> Jc >> >> >> >> >> On 11/30/18 19:08, Daniil Titov wrote: >> >> Please review the change for nsk/jvmti/unit/ForceEarlyReturn/earlyretbase test. The problem here is that before doing an early force return the test doesn't check that the test thread is suspended at a right place where the top frame executes the test method rather than JVMCI code triggered by invocation counter overflow. That results in the early return happens for a wrong method and the test fails. >> >> The fix changes the test to do resume/suspend in the loop until the target method is executed in the top frame or the loop counter exceeds the limit. >> >> There is another problem with this test that requires changes on VM side and is currently covered by JDK-8195635:" [Graal] nsk/jvmti/unit/ForceEarlyReturn/earlyretbase crashes with assertion "compilation level out of bounds"".? The test will have to be removed from test/hotspot/jtreg/ProblemList-graal.txt only after both these issues are fixed. >> >> Bug:https://bugs.openjdk.java.net/browse/JDK-8214572 >> >> Webrev:http://cr.openjdk.java.net/~dtitov/8214572/webrev.01/ >> >> Thanks, >> >> Daniil >> >> >> > From serguei.spitsyn at oracle.com Wed Dec 5 00:55:55 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 4 Dec 2018 16:55:55 -0800 Subject: RFR 8214572: nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the thread when the top frame executes JVMCI code In-Reply-To: References: <4E0EFD3E-F725-4469-BA20-7532D78BA479@oracle.com> <567a8f54-c9f1-7936-65c3-02d1c660dc70@oracle.com> <73D62FE8-DBD5-40AF-88B8-8397E0DAE2CA@oracle.com> <633bf1e4-5423-76df-32bb-0f4470e905e0@oracle.com> Message-ID: On 12/4/18 4:54 PM, David Holmes wrote: > Hi everyone, > > I'd actually argue that the comment not refer just to JVMCI but more > generally: > > +???????? // when the top frame belongs to the test rather than to > incidental Java code (classloading, JVMCI, etc) Reasonable. > Also note typo: then -> than Nice catch! Thanks, Serguei > > Cheers, > David > > On 5/12/2018 5:40 am, serguei.spitsyn at oracle.com wrote: >> Hi Daniil, >> >> It looks good in general. >> Thank you for the update! >> >> I have some minor comment though. >> >> http://cr.openjdk.java.net/%7Edtitov/8214572/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_tools.h.udiff.html >> >> >> +/** >> +* This method suspends the thread while ensuring the top frame >> executes the test method >> +* rather then JVMCI code triggered by invocation counter overflow. >> +*/ >> +int suspendThreadAtMethod(jvmtiEnv *jvmti, jclass cls, jobject >> thread, jmethodID method); >> >> >> The comment above is not precise as it tells nothing about top frame. >> >> I like this one from implementation: >> >> + // We need to ensure that the thread is suspended at the right place >> + // when the top frame belongs to the test rather then to JVMCI code. >> >> >> So, the can be rephrased to something like: >> >> + // This method makes the thread to be suspended at the right place >> + // when the top frame belongs to the test rather then to JVMCI code. >> >> >> >> No need in another webrev if you fix the comment. >> >> Thanks, >> Serguei >> >> >> On 12/4/18 10:24 AM, Daniil Titov wrote: >>> >>> Hi Serguei and JC, >>> >>> Thank you for reviewing this change. And many thanks to David and >>> Dean for answering JVMCI questions. >>> >>> Please review a new version of the fix that moves the most of the >>> new code in a helper method ( as JC suggested) and corrects error >>> messages. I also excluded the changes in >>> test/hotspot/jtreg/ProblemList-graal.txt from this webrev. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8214572 >>> >>> Webrev:http://cr.openjdk.java.net/~dtitov/8214572/webrev.02/ >>> >>> >>> Thanks, >>> >>> --Daniil >>> >>> *From: *"serguei.spitsyn at oracle.com" >>> *Date: *Monday, December 3, 2018 at 4:14 PM >>> *To: *Daniil Titov , serviceability-dev >>> >>> *Subject: *Re: RFR 8214572: >>> nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the >>> thread when the top frame executes JVMCI code >>> >>> Hi Daniil, >>> >>> It looks good in general. >>> >>> I have two comments though. >>> >>> -vmTestbase/nsk/jvmti/unit/ForceEarlyReturn/earlyretbase/TestDescription.java >>> 8195635?? generic-all >>> >>> ? It is not a good idea to remove the test from the ProblemList >>> before the 8195635 is fixed. >>> >>> 148???? if(method == midActiveMethod) { >>> ? 149???????? printf("<<<<<<<< SuspendThread() is successfully >>> done\n"); >>> 150???? } else { >>> 151???????? printf("Warning: method \"activeMethod\" was missed\n"); >>> 152???????? errCode = STATUS_FAILED; >>> 153???? } >>> >>> ?I'd suggest to tweak the error message to something like: >>> ?? "Failed in the suspThread: was not able to suspend thread with >>> the activeMethod() on top\n"); >>> >>> >>> Thanks, >>> Serguei >>> >>> *From: *JC Beyler >>> *Date: *Friday, November 30, 2018 at 7:47 PM >>> *To: * >>> *Cc: * >>> *Subject: *Re: RFR 8214572: >>> nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the >>> thread when the top frame executes JVMCI code >>> >>> Hi Daniil, >>> >>> The webrev looks good but I have a few comments and questions (if >>> you don't mind :-)): >>> >>> Comments: >>> >>> ? - You say that normally the test will be removed from the problem >>> list once the two fixes are done but in this webrev, you've already >>> removed it (I can't see the other case so I can't see if it is >>> resolved :-)) >>> >>> ? - now that we are in C++ for the tests, could we not declare the >>> variables at their first use instead of doing the pedantic top of >>> the block C style? >>> >>> ? - I feel that this sort of "wait until we are not in JVMCI frames" >>> might happen a lot, maybe we could move that code into a helper >>> method (+ it cleans the method a bit to just concentrate on the >>> rest) and then if needed we can make it public to other tests? >>> >>> Questions because I'm not familiar with JVMCI consequences so not >>> really comments on the webrev but so that I know: >>> >>> ? - Is it normaly that you can suspend when you are in a JVMCI >>> frame? will/is there not a better way that we could detect that we >>> are in a JVMCI frame? Is it always safe to suspend a JVMCI frame? >>> >>> Thanks! >>> >>> Jc >>> >>> >>> >>> >>> On 11/30/18 19:08, Daniil Titov wrote: >>> >>> ??? Please review the change for >>> nsk/jvmti/unit/ForceEarlyReturn/earlyretbase test. The problem here >>> is that before doing an early force return the test doesn't check >>> that the test thread is suspended at a right place where the top >>> frame executes the test method rather than JVMCI code triggered by >>> invocation counter overflow. That results in the early return >>> happens for a wrong method and the test fails. >>> >>> ??? The fix changes the test to do resume/suspend in the loop until >>> the target method is executed in the top frame or the loop counter >>> exceeds the limit. >>> >>> ??? There is another problem with this test that requires changes on >>> VM side and is currently covered by JDK-8195635:" [Graal] >>> nsk/jvmti/unit/ForceEarlyReturn/earlyretbase crashes with assertion >>> "compilation level out of bounds"".? The test will have to be >>> removed from test/hotspot/jtreg/ProblemList-graal.txt only after >>> both these issues are fixed. >>> >>> ??? Bug:https://bugs.openjdk.java.net/browse/JDK-8214572 >>> >>> Webrev:http://cr.openjdk.java.net/~dtitov/8214572/webrev.01/ >>> >>> ??? Thanks, >>> >>> ??? Daniil >>> >>> >>> >> From daniil.x.titov at oracle.com Wed Dec 5 01:15:00 2018 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Tue, 04 Dec 2018 17:15:00 -0800 Subject: RFR 8214572: nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the thread when the top frame executes JVMCI code In-Reply-To: References: <4E0EFD3E-F725-4469-BA20-7532D78BA479@oracle.com> <567a8f54-c9f1-7936-65c3-02d1c660dc70@oracle.com> <73D62FE8-DBD5-40AF-88B8-8397E0DAE2CA@oracle.com> <633bf1e4-5423-76df-32bb-0f4470e905e0@oracle.com> Message-ID: <9F03405A-1D91-49F1-9C03-626750AB034B@oracle.com> Thank you, David and Serguei, Please review an updated version of the patch with the corrected comment. Bug: https://bugs.openjdk.java.net/browse/JDK-8214572 Webrev: http://cr.openjdk.java.net/~dtitov/8214572/webrev.03 Best regards, Daniil ?On 12/4/18, 4:55 PM, "serguei.spitsyn at oracle.com" wrote: On 12/4/18 4:54 PM, David Holmes wrote: > Hi everyone, > > I'd actually argue that the comment not refer just to JVMCI but more > generally: > > + // when the top frame belongs to the test rather than to > incidental Java code (classloading, JVMCI, etc) Reasonable. > Also note typo: then -> than Nice catch! Thanks, Serguei > > Cheers, > David > > On 5/12/2018 5:40 am, serguei.spitsyn at oracle.com wrote: >> Hi Daniil, >> >> It looks good in general. >> Thank you for the update! >> >> I have some minor comment though. >> >> http://cr.openjdk.java.net/%7Edtitov/8214572/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_tools.h.udiff.html >> >> >> +/** >> +* This method suspends the thread while ensuring the top frame >> executes the test method >> +* rather then JVMCI code triggered by invocation counter overflow. >> +*/ >> +int suspendThreadAtMethod(jvmtiEnv *jvmti, jclass cls, jobject >> thread, jmethodID method); >> >> >> The comment above is not precise as it tells nothing about top frame. >> >> I like this one from implementation: >> >> + // We need to ensure that the thread is suspended at the right place >> + // when the top frame belongs to the test rather then to JVMCI code. >> >> >> So, the can be rephrased to something like: >> >> + // This method makes the thread to be suspended at the right place >> + // when the top frame belongs to the test rather then to JVMCI code. >> >> >> >> No need in another webrev if you fix the comment. >> >> Thanks, >> Serguei >> >> >> On 12/4/18 10:24 AM, Daniil Titov wrote: >>> >>> Hi Serguei and JC, >>> >>> Thank you for reviewing this change. And many thanks to David and >>> Dean for answering JVMCI questions. >>> >>> Please review a new version of the fix that moves the most of the >>> new code in a helper method ( as JC suggested) and corrects error >>> messages. I also excluded the changes in >>> test/hotspot/jtreg/ProblemList-graal.txt from this webrev. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8214572 >>> >>> Webrev:http://cr.openjdk.java.net/~dtitov/8214572/webrev.02/ >>> >>> >>> Thanks, >>> >>> --Daniil >>> >>> *From: *"serguei.spitsyn at oracle.com" >>> *Date: *Monday, December 3, 2018 at 4:14 PM >>> *To: *Daniil Titov , serviceability-dev >>> >>> *Subject: *Re: RFR 8214572: >>> nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the >>> thread when the top frame executes JVMCI code >>> >>> Hi Daniil, >>> >>> It looks good in general. >>> >>> I have two comments though. >>> >>> -vmTestbase/nsk/jvmti/unit/ForceEarlyReturn/earlyretbase/TestDescription.java >>> 8195635 generic-all >>> >>> It is not a good idea to remove the test from the ProblemList >>> before the 8195635 is fixed. >>> >>> 148 if(method == midActiveMethod) { >>> 149 printf("<<<<<<<< SuspendThread() is successfully >>> done\n"); >>> 150 } else { >>> 151 printf("Warning: method \"activeMethod\" was missed\n"); >>> 152 errCode = STATUS_FAILED; >>> 153 } >>> >>> I'd suggest to tweak the error message to something like: >>> "Failed in the suspThread: was not able to suspend thread with >>> the activeMethod() on top\n"); >>> >>> >>> Thanks, >>> Serguei >>> >>> *From: *JC Beyler >>> *Date: *Friday, November 30, 2018 at 7:47 PM >>> *To: * >>> *Cc: * >>> *Subject: *Re: RFR 8214572: >>> nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the >>> thread when the top frame executes JVMCI code >>> >>> Hi Daniil, >>> >>> The webrev looks good but I have a few comments and questions (if >>> you don't mind :-)): >>> >>> Comments: >>> >>> - You say that normally the test will be removed from the problem >>> list once the two fixes are done but in this webrev, you've already >>> removed it (I can't see the other case so I can't see if it is >>> resolved :-)) >>> >>> - now that we are in C++ for the tests, could we not declare the >>> variables at their first use instead of doing the pedantic top of >>> the block C style? >>> >>> - I feel that this sort of "wait until we are not in JVMCI frames" >>> might happen a lot, maybe we could move that code into a helper >>> method (+ it cleans the method a bit to just concentrate on the >>> rest) and then if needed we can make it public to other tests? >>> >>> Questions because I'm not familiar with JVMCI consequences so not >>> really comments on the webrev but so that I know: >>> >>> - Is it normaly that you can suspend when you are in a JVMCI >>> frame? will/is there not a better way that we could detect that we >>> are in a JVMCI frame? Is it always safe to suspend a JVMCI frame? >>> >>> Thanks! >>> >>> Jc >>> >>> >>> >>> >>> On 11/30/18 19:08, Daniil Titov wrote: >>> >>> Please review the change for >>> nsk/jvmti/unit/ForceEarlyReturn/earlyretbase test. The problem here >>> is that before doing an early force return the test doesn't check >>> that the test thread is suspended at a right place where the top >>> frame executes the test method rather than JVMCI code triggered by >>> invocation counter overflow. That results in the early return >>> happens for a wrong method and the test fails. >>> >>> The fix changes the test to do resume/suspend in the loop until >>> the target method is executed in the top frame or the loop counter >>> exceeds the limit. >>> >>> There is another problem with this test that requires changes on >>> VM side and is currently covered by JDK-8195635:" [Graal] >>> nsk/jvmti/unit/ForceEarlyReturn/earlyretbase crashes with assertion >>> "compilation level out of bounds"". The test will have to be >>> removed from test/hotspot/jtreg/ProblemList-graal.txt only after >>> both these issues are fixed. >>> >>> Bug:https://bugs.openjdk.java.net/browse/JDK-8214572 >>> >>> Webrev:http://cr.openjdk.java.net/~dtitov/8214572/webrev.01/ >>> >>> Thanks, >>> >>> Daniil >>> >>> >>> >> From david.holmes at oracle.com Wed Dec 5 01:29:42 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 5 Dec 2018 11:29:42 +1000 Subject: RFR 8214572: nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the thread when the top frame executes JVMCI code In-Reply-To: <9F03405A-1D91-49F1-9C03-626750AB034B@oracle.com> References: <4E0EFD3E-F725-4469-BA20-7532D78BA479@oracle.com> <567a8f54-c9f1-7936-65c3-02d1c660dc70@oracle.com> <73D62FE8-DBD5-40AF-88B8-8397E0DAE2CA@oracle.com> <633bf1e4-5423-76df-32bb-0f4470e905e0@oracle.com> <9F03405A-1D91-49F1-9C03-626750AB034B@oracle.com> Message-ID: <9cea96f1-79f0-54ac-028d-a1c0d387bac3@oracle.com> Thanks Daniil, that seems fine to me. I have some reservations about whether looping up to 100ms will necessarily be enough for compilation to complete, but we can adjust in the future as needed. Cheers, David On 5/12/2018 11:15 am, Daniil Titov wrote: > Thank you, David and Serguei, > > Please review an updated version of the patch with the corrected comment. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214572 > Webrev: http://cr.openjdk.java.net/~dtitov/8214572/webrev.03 > > Best regards, > Daniil > > ?On 12/4/18, 4:55 PM, "serguei.spitsyn at oracle.com" wrote: > > > > On 12/4/18 4:54 PM, David Holmes wrote: > > Hi everyone, > > > > I'd actually argue that the comment not refer just to JVMCI but more > > generally: > > > > + // when the top frame belongs to the test rather than to > > incidental Java code (classloading, JVMCI, etc) > > Reasonable. > > > Also note typo: then -> than > > Nice catch! > > Thanks, > Serguei > > > > > Cheers, > > David > > > > On 5/12/2018 5:40 am, serguei.spitsyn at oracle.com wrote: > >> Hi Daniil, > >> > >> It looks good in general. > >> Thank you for the update! > >> > >> I have some minor comment though. > >> > >> http://cr.openjdk.java.net/%7Edtitov/8214572/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_tools.h.udiff.html > >> > >> > >> +/** > >> +* This method suspends the thread while ensuring the top frame > >> executes the test method > >> +* rather then JVMCI code triggered by invocation counter overflow. > >> +*/ > >> +int suspendThreadAtMethod(jvmtiEnv *jvmti, jclass cls, jobject > >> thread, jmethodID method); > >> > >> > >> The comment above is not precise as it tells nothing about top frame. > >> > >> I like this one from implementation: > >> > >> + // We need to ensure that the thread is suspended at the right place > >> + // when the top frame belongs to the test rather then to JVMCI code. > >> > >> > >> So, the can be rephrased to something like: > >> > >> + // This method makes the thread to be suspended at the right place > >> + // when the top frame belongs to the test rather then to JVMCI code. > >> > >> > >> > >> No need in another webrev if you fix the comment. > >> > >> Thanks, > >> Serguei > >> > >> > >> On 12/4/18 10:24 AM, Daniil Titov wrote: > >>> > >>> Hi Serguei and JC, > >>> > >>> Thank you for reviewing this change. And many thanks to David and > >>> Dean for answering JVMCI questions. > >>> > >>> Please review a new version of the fix that moves the most of the > >>> new code in a helper method ( as JC suggested) and corrects error > >>> messages. I also excluded the changes in > >>> test/hotspot/jtreg/ProblemList-graal.txt from this webrev. > >>> > >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8214572 > >>> > >>> Webrev:http://cr.openjdk.java.net/~dtitov/8214572/webrev.02/ > >>> > >>> > >>> Thanks, > >>> > >>> --Daniil > >>> > >>> *From: *"serguei.spitsyn at oracle.com" > >>> *Date: *Monday, December 3, 2018 at 4:14 PM > >>> *To: *Daniil Titov , serviceability-dev > >>> > >>> *Subject: *Re: RFR 8214572: > >>> nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the > >>> thread when the top frame executes JVMCI code > >>> > >>> Hi Daniil, > >>> > >>> It looks good in general. > >>> > >>> I have two comments though. > >>> > >>> -vmTestbase/nsk/jvmti/unit/ForceEarlyReturn/earlyretbase/TestDescription.java > >>> 8195635 generic-all > >>> > >>> It is not a good idea to remove the test from the ProblemList > >>> before the 8195635 is fixed. > >>> > >>> 148 if(method == midActiveMethod) { > >>> 149 printf("<<<<<<<< SuspendThread() is successfully > >>> done\n"); > >>> 150 } else { > >>> 151 printf("Warning: method \"activeMethod\" was missed\n"); > >>> 152 errCode = STATUS_FAILED; > >>> 153 } > >>> > >>> I'd suggest to tweak the error message to something like: > >>> "Failed in the suspThread: was not able to suspend thread with > >>> the activeMethod() on top\n"); > >>> > >>> > >>> Thanks, > >>> Serguei > >>> > >>> *From: *JC Beyler > >>> *Date: *Friday, November 30, 2018 at 7:47 PM > >>> *To: * > >>> *Cc: * > >>> *Subject: *Re: RFR 8214572: > >>> nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the > >>> thread when the top frame executes JVMCI code > >>> > >>> Hi Daniil, > >>> > >>> The webrev looks good but I have a few comments and questions (if > >>> you don't mind :-)): > >>> > >>> Comments: > >>> > >>> - You say that normally the test will be removed from the problem > >>> list once the two fixes are done but in this webrev, you've already > >>> removed it (I can't see the other case so I can't see if it is > >>> resolved :-)) > >>> > >>> - now that we are in C++ for the tests, could we not declare the > >>> variables at their first use instead of doing the pedantic top of > >>> the block C style? > >>> > >>> - I feel that this sort of "wait until we are not in JVMCI frames" > >>> might happen a lot, maybe we could move that code into a helper > >>> method (+ it cleans the method a bit to just concentrate on the > >>> rest) and then if needed we can make it public to other tests? > >>> > >>> Questions because I'm not familiar with JVMCI consequences so not > >>> really comments on the webrev but so that I know: > >>> > >>> - Is it normaly that you can suspend when you are in a JVMCI > >>> frame? will/is there not a better way that we could detect that we > >>> are in a JVMCI frame? Is it always safe to suspend a JVMCI frame? > >>> > >>> Thanks! > >>> > >>> Jc > >>> > >>> > >>> > >>> > >>> On 11/30/18 19:08, Daniil Titov wrote: > >>> > >>> Please review the change for > >>> nsk/jvmti/unit/ForceEarlyReturn/earlyretbase test. The problem here > >>> is that before doing an early force return the test doesn't check > >>> that the test thread is suspended at a right place where the top > >>> frame executes the test method rather than JVMCI code triggered by > >>> invocation counter overflow. That results in the early return > >>> happens for a wrong method and the test fails. > >>> > >>> The fix changes the test to do resume/suspend in the loop until > >>> the target method is executed in the top frame or the loop counter > >>> exceeds the limit. > >>> > >>> There is another problem with this test that requires changes on > >>> VM side and is currently covered by JDK-8195635:" [Graal] > >>> nsk/jvmti/unit/ForceEarlyReturn/earlyretbase crashes with assertion > >>> "compilation level out of bounds"". The test will have to be > >>> removed from test/hotspot/jtreg/ProblemList-graal.txt only after > >>> both these issues are fixed. > >>> > >>> Bug:https://bugs.openjdk.java.net/browse/JDK-8214572 > >>> > >>> Webrev:http://cr.openjdk.java.net/~dtitov/8214572/webrev.01/ > >>> > >>> Thanks, > >>> > >>> Daniil > >>> > >>> > >>> > >> > > > > From serguei.spitsyn at oracle.com Wed Dec 5 03:49:16 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 4 Dec 2018 19:49:16 -0800 Subject: RFR 8214572: nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the thread when the top frame executes JVMCI code In-Reply-To: <9F03405A-1D91-49F1-9C03-626750AB034B@oracle.com> References: <4E0EFD3E-F725-4469-BA20-7532D78BA479@oracle.com> <567a8f54-c9f1-7936-65c3-02d1c660dc70@oracle.com> <73D62FE8-DBD5-40AF-88B8-8397E0DAE2CA@oracle.com> <633bf1e4-5423-76df-32bb-0f4470e905e0@oracle.com> <9F03405A-1D91-49F1-9C03-626750AB034B@oracle.com> Message-ID: <235b8139-ed24-95ec-b8ca-25b28e395bc7@oracle.com> An HTML attachment was scrubbed... URL: From daniil.x.titov at oracle.com Wed Dec 5 05:31:52 2018 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Tue, 04 Dec 2018 21:31:52 -0800 Subject: RFR 8214572: nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the thread when the top frame executes JVMCI code In-Reply-To: <235b8139-ed24-95ec-b8ca-25b28e395bc7@oracle.com> References: <4E0EFD3E-F725-4469-BA20-7532D78BA479@oracle.com> <567a8f54-c9f1-7936-65c3-02d1c660dc70@oracle.com> <73D62FE8-DBD5-40AF-88B8-8397E0DAE2CA@oracle.com> <633bf1e4-5423-76df-32bb-0f4470e905e0@oracle.com> <9F03405A-1D91-49F1-9C03-626750AB034B@oracle.com> <235b8139-ed24-95ec-b8ca-25b28e395bc7@oracle.com> Message-ID: Thank you, Serguei, I will correct it before pushing ?the changes in the repository. Best regards, Daniil From: "serguei.spitsyn at oracle.com" Date: Tuesday, December 4, 2018 at 7:49 PM To: Daniil Titov , David Holmes , serviceability-dev , JC Beyler Subject: Re: RFR 8214572: nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the thread when the top frame executes JVMCI code http://cr.openjdk.java.net/~dtitov/8214572/webrev.03/test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_tools.cpp.udiff.html +??? // frame belongs to the test rather then to incidental Java code (classloading, The same typo but in another comment: then => than No need in another webrev. Thanks, Serguei On 12/4/18 17:15, Daniil Titov wrote: Thank you, David and Serguei, Please review an updated version of the patch with the corrected comment. Bug: https://bugs.openjdk.java.net/browse/JDK-8214572 Webrev: http://cr.openjdk.java.net/~dtitov/8214572/webrev.03 Best regards, Daniil ?On 12/4/18, 4:55 PM, "serguei.spitsyn at oracle.com" wrote: ??? ???? ????On 12/4/18 4:54 PM, David Holmes wrote: ??? > Hi everyone, ??? > ??? > I'd actually argue that the comment not refer just to JVMCI but more ????> generally: ??? > ??? > +???????? // when the top frame belongs to the test rather than to ????> incidental Java code (classloading, JVMCI, etc) ??? ????Reasonable. ??? ????> Also note typo: then -> than ??? ????Nice catch! ??? ????Thanks, ??? Serguei ??? ????> ??? > Cheers, ??? > David ??? > ??? > On 5/12/2018 5:40 am, serguei.spitsyn at oracle.com wrote: ??? >> Hi Daniil, ??? >> ??? >> It looks good in general. ??? >> Thank you for the update! ??? >> ??? >> I have some minor comment though. ??? >> ??? >> http://cr.openjdk.java.net/%7Edtitov/8214572/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_tools.h.udiff.html ????>> ??? >> ??? >> +/** ??? >> +* This method suspends the thread while ensuring the top frame ????>> executes the test method ??? >> +* rather then JVMCI code triggered by invocation counter overflow. ??? >> +*/ ??? >> +int suspendThreadAtMethod(jvmtiEnv *jvmti, jclass cls, jobject ????>> thread, jmethodID method); ??? >> ??? >> ??? >> The comment above is not precise as it tells nothing about top frame. ??? >> ??? >> I like this one from implementation: ??? >> ??? >> + // We need to ensure that the thread is suspended at the right place ??? >> + // when the top frame belongs to the test rather then to JVMCI code. ??? >> ??? >> ??? >> So, the can be rephrased to something like: ??? >> ??? >> + // This method makes the thread to be suspended at the right place ??? >> + // when the top frame belongs to the test rather then to JVMCI code. ??? >> ??? >> ??? >> ??? >> No need in another webrev if you fix the comment. ??? >> ??? >> Thanks, ??? >> Serguei ??? >> ??? >> ??? >> On 12/4/18 10:24 AM, Daniil Titov wrote: ??? >>> ??? >>> Hi Serguei and JC, ??? >>> ??? >>> Thank you for reviewing this change. And many thanks to David and ????>>> Dean for answering JVMCI questions. ??? >>> ??? >>> Please review a new version of the fix that moves the most of the ????>>> new code in a helper method ( as JC suggested) and corrects error ????>>> messages. I also excluded the changes in ????>>> test/hotspot/jtreg/ProblemList-graal.txt from this webrev. ??? >>> ??? >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8214572 ??? >>> ??? >>> Webrev:http://cr.openjdk.java.net/~dtitov/8214572/webrev.02/ ????>>> ??? >>> ??? >>> Thanks, ??? >>> ??? >>> --Daniil ??? >>> ??? >>> *From: *"serguei.spitsyn at oracle.com" ??? >>> *Date: *Monday, December 3, 2018 at 4:14 PM ??? >>> *To: *Daniil Titov , serviceability-dev ????>>> ??? >>> *Subject: *Re: RFR 8214572: ????>>> nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the ????>>> thread when the top frame executes JVMCI code ??? >>> ??? >>> Hi Daniil, ??? >>> ??? >>> It looks good in general. ??? >>> ??? >>> I have two comments though. ??? >>> ??? >>> -vmTestbase/nsk/jvmti/unit/ForceEarlyReturn/earlyretbase/TestDescription.java ????>>> 8195635?? generic-all ??? >>> ??? >>>?? It is not a good idea to remove the test from the ProblemList ????>>> before the 8195635 is fixed. ??? >>> ??? >>> 148???? if(method == midActiveMethod) { ??? >>>?? 149???????? printf("<<<<<<<< SuspendThread() is successfully ????>>> done\n"); ??? >>> 150???? } else { ??? >>> 151???????? printf("Warning: method \"activeMethod\" was missed\n"); ??? >>> 152???????? errCode = STATUS_FAILED; ??? >>> 153???? } ??? >>> ??? >>>? I'd suggest to tweak the error message to something like: ??? >>>??? "Failed in the suspThread: was not able to suspend thread with ????>>> the activeMethod() on top\n"); ??? >>> ??? >>> ??? >>> Thanks, ??? >>> Serguei ??? >>> ??? >>> *From: *JC Beyler ??? >>> *Date: *Friday, November 30, 2018 at 7:47 PM ??? >>> *To: * ??? >>> *Cc: * ??? >>> *Subject: *Re: RFR 8214572: ????>>> nsk/jvmti/unit/ForceEarlyReturn/earlyretbase should not suspend the ????>>> thread when the top frame executes JVMCI code ??? >>> ??? >>> Hi Daniil, ??? >>> ??? >>> The webrev looks good but I have a few comments and questions (if ????>>> you don't mind :-)): ??? >>> ??? >>> Comments: ??? >>> ??? >>>?? - You say that normally the test will be removed from the problem ????>>> list once the two fixes are done but in this webrev, you've already ????>>> removed it (I can't see the other case so I can't see if it is ????>>> resolved :-)) ??? >>> ??? >>>?? - now that we are in C++ for the tests, could we not declare the ????>>> variables at their first use instead of doing the pedantic top of ????>>> the block C style? ??? >>> ??? >>>?? - I feel that this sort of "wait until we are not in JVMCI frames" ????>>> might happen a lot, maybe we could move that code into a helper ????>>> method (+ it cleans the method a bit to just concentrate on the ????>>> rest) and then if needed we can make it public to other tests? ??? >>> ??? >>> Questions because I'm not familiar with JVMCI consequences so not ????>>> really comments on the webrev but so that I know: ??? >>> ??? >>>?? - Is it normaly that you can suspend when you are in a JVMCI ????>>> frame? will/is there not a better way that we could detect that we ????>>> are in a JVMCI frame? Is it always safe to suspend a JVMCI frame? ??? >>> ??? >>> Thanks! ??? >>> ??? >>> Jc ??? >>> ??? >>> ??? >>> ??? >>> ??? >>> On 11/30/18 19:08, Daniil Titov wrote: ??? >>> ??? >>>???? Please review the change for ????>>> nsk/jvmti/unit/ForceEarlyReturn/earlyretbase test. The problem here ????>>> is that before doing an early force return the test doesn't check ????>>> that the test thread is suspended at a right place where the top ????>>> frame executes the test method rather than JVMCI code triggered by ????>>> invocation counter overflow. That results in the early return ????>>> happens for a wrong method and the test fails. ??? >>> ??? >>>???? The fix changes the test to do resume/suspend in the loop until ????>>> the target method is executed in the top frame or the loop counter ????>>> exceeds the limit. ??? >>> ??? >>>???? There is another problem with this test that requires changes on ????>>> VM side and is currently covered by JDK-8195635:" [Graal] ????>>> nsk/jvmti/unit/ForceEarlyReturn/earlyretbase crashes with assertion ????>>> "compilation level out of bounds"".? The test will have to be ????>>> removed from test/hotspot/jtreg/ProblemList-graal.txt only after ????>>> both these issues are fixed. ??? >>> ??? >>>???? Bug:https://bugs.openjdk.java.net/browse/JDK-8214572 ??? >>> ??? >>> Webrev:http://cr.openjdk.java.net/~dtitov/8214572/webrev.01/ ????>>> ??? >>>???? Thanks, ??? >>> ??? >>>???? Daniil ??? >>> ??? >>> ??? >>> ??? >> ??? ???? -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.griffiths at gmail.com Wed Dec 5 11:09:54 2018 From: david.griffiths at gmail.com (David Griffiths) Date: Wed, 5 Dec 2018 11:09:54 +0000 Subject: Suggested improvement to X86Frame.getInterpreterFrameBCI In-Reply-To: References: <0f590762-2c34-0ceb-0937-be4ef39d30b2@oracle.com> <11798f90-4e4e-55e3-3226-35c26df0a202@oracle.com> Message-ID: Hi, thanks for reviewing. I have made the changes you suggested and also tidied up the constructors a bit (there was already a 4x Address constructor), hope that's ok. Cheers, David On Fri, 30 Nov 2018 at 17:06, JC Beyler wrote: > Hi both, > > The webrev looks good to me but I could see gains of just adding a new > constructor instead of doing a new + set. > > LinuxAMD64JavaThreadPDAccess.java would become consistent with the rest of > the code: > + } else if (VM.getVM().getInterpreter().contains(guesser.getPC())) { > + // pass the value of R13 which contains the bcp for the top level > frame > + return new X86Frame(guesser.getSP(), guesser.getFP(), > guesser.getPC(), > + context.getRegisterAsAddress(AMD64ThreadContext.R13)); > } else { > > - And for X86Frame.java, that means there is no set method (there isn't a > single one yet there so we are consistent there). > - Finally, instead of just r13 internally to the Frame, we could just call > it the bcp since that is what you are saying it is and then adapt the > getInterpreterFrameBCI a bit because a bcp local variable is there :-) > > But these are nits :), > Jc > > On Fri, Nov 30, 2018 at 6:21 AM Jini George > wrote: > >> Your patch looks good to me, David. I can sponsor this for you if we get >> one more review. >> >> Thanks, >> Jini. >> >> On 11/22/2018 5:42 PM, David Griffiths wrote: >> > Thanks Jini, please find patch for Java 9 attached (I don't have author >> > access to the bug itself). >> > >> > Cheers, >> > >> > David >> > >> > On Thu, 22 Nov 2018 at 09:02, Jini George > > > wrote: >> > >> > Thank you very much for working on the fix for this issue, David. It >> > would be great if you can send in a complete patch for the review >> (With >> > a first cut look, there seems to be missing pieces). >> > >> > I have created a bug for this: >> > >> > https://bugs.openjdk.java.net/browse/JDK-8214226 >> > >> > Thank you, >> > Jini >> > >> > On 11/22/2018 12:50 AM, David Griffiths wrote: >> > > PS: should have added a new X86Frame constructor really, may have >> > just >> > > been put off because there is already a four address constructor >> so >> > > would have had to add dummy argument or something. >> > > >> > > On Wed, 21 Nov 2018 at 19:15, David Griffiths >> > >> > > > > >> wrote: >> > > >> > > Hi, thanks, apart from adding a setter for R13 in X86Frame, >> the >> > > other half of the fix is this: >> > > >> > > public Frame getCurrentFrameGuess(JavaThread thread, >> > Address >> > > addr) { >> > > ThreadProxy t = getThreadProxy(addr); >> > > AMD64ThreadContext context = (AMD64ThreadContext) >> > t.getContext(); >> > > AMD64CurrentFrameGuess guesser = new >> > > AMD64CurrentFrameGuess(context, thread); >> > > if (!guesser.run(GUESS_SCAN_RANGE)) { >> > > return null; >> > > } >> > > if (guesser.getPC() == null) { >> > > return new X86Frame(guesser.getSP(), guesser.getFP()); >> > > } else if >> > (VM.getVM().getInterpreter().contains(guesser.getPC())) { >> > > // pass the value of R13 which contains the bcp for >> > the top >> > > level frame >> > > Address r13 = >> > > context.getRegisterAsAddress(AMD64ThreadContext.R13); >> > > X86Frame frame = new X86Frame(guesser.getSP(), >> > > guesser.getFP(), guesser.getPC()); >> > > frame.setR13(r13); >> > > return frame; >> > > } else { >> > > return new X86Frame(guesser.getSP(), guesser.getFP(), >> > > guesser.getPC()); >> > > } >> > > } >> > > >> > > (the whole "if pc in interpreter" block is new) >> > > >> > > Overhead likely to be low as this is only used in diagnostic >> > code. >> > > Can't think of any risk but I'm not an expert on this code. >> > > >> > > Cheers, >> > > >> > > David >> > > >> > > On Wed, 21 Nov 2018 at 19:01, JC Beyler > > >> > > >> >> wrote: >> > > >> > > Hi David, >> > > >> > > I think the easiest would be to see whole change to >> > understand >> > > the repercussions of the change. I would imagine that any >> > change >> > > that helps stacktraces being more precise is a good >> thing but >> > > there are questions that arise every time: >> > > - What is the overhead of adding this? >> > > - Does this add any risk of failure? >> > > >> > > I'd imagine that the change is relatively small and >> should be >> > > easy to assess this but seeing it would make things >> easier to >> > > determine. >> > > >> > > That being said, I'm not a reviewer for OpenJDK so this >> is >> > > really just my 2cents, >> > > Jc >> > > >> > > On Wed, Nov 21, 2018 at 9:17 AM David Griffiths >> > > > > > david.griffiths at gmail.com >> > >> >> > > wrote: >> > > >> > > Hi, I'm new to this mailing list and working on a >> project >> > > that makes use of the SA classes to get stack traces >> > from a >> > > paused in flight JVM (we can't use JDWP). I have >> observed >> > > that if the top frame is in the interpreter it >> > reports the >> > > BCI and line number incorrectly. This is because >> > > X86Frame.getInterpreterFrameBCI uses the value stored >> > on the >> > > stack rather than the actual live value stored in >> R13. >> > > >> > > I have a patch for this which lets >> > > LinuxAMD64JavaThreadPDAccess.getCurrentFrameGuess >> > pass the >> > > R13 value to X86Frame so that the latter can then do: >> > > >> > > public int getInterpreterFrameBCI() { >> > > Address bcp = >> > > addressOfInterpreterFrameBCX().getAddressAt(0); >> > > // If we are in the top level frame then R13 may >> > have >> > > been set for us which contains >> > > // the BCP. If so then let it take priority. If >> > we are >> > > in a top level interpreter frame, >> > > // the BCP is live in R13 (on x86) and not saved >> > in the >> > > BCX stack slot. >> > > if (r13 != null) { >> > > bcp = r13; >> > > } >> > > Address methodHandle = >> > > addressOfInterpreterFrameMethod().getAddressAt(0); >> > > >> > > and this fixes the problem. >> > > >> > > Does this sound like a good idea and if so should I >> > submit a >> > > patch? >> > > >> > > Cheers, >> > > >> > > David >> > > >> > > >> > > >> > > -- >> > > >> > > Thanks, >> > > Jc >> > > >> > >> > > > -- > > Thanks, > Jc > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: revised_interpreter_frame.patch Type: application/octet-stream Size: 4644 bytes Desc: not available URL: From ralf.schmelter at sap.com Wed Dec 5 14:56:46 2018 From: ralf.schmelter at sap.com (Schmelter, Ralf) Date: Wed, 5 Dec 2018 14:56:46 +0000 Subject: RFR (M) 8214892: Delayed starting of debugging via jcmd Message-ID: <01fdfc8e58a246c18d5909fc7df313b7@sap.com> Hi All, Please review the fix for the bug https://bugs.openjdk.java.net/browse/JDK-8214892. The webref is at http://cr.openjdk.java.net/~rrich/webrevs/schmelter/8214892/webrev.01/ . This change allows debugging to be started delayed, triggered by a jcmd name VM.start_java_debugging. It works more or less like the onthrow and onuncaught options, but instead of triggering by exception we trigger it via the jcmd. This is useful when we want to trigger debugging when we see errors not related to exceptions (e.g. if a server becomes unresponsive it could be taken out of the routing and debugging is triggered to find the cause). Best regards, Ralf From chris.plummer at oracle.com Wed Dec 5 20:45:23 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 5 Dec 2018 12:45:23 -0800 Subject: RFR (M) 8214892: Delayed starting of debugging via jcmd In-Reply-To: <01fdfc8e58a246c18d5909fc7df313b7@sap.com> References: <01fdfc8e58a246c18d5909fc7df313b7@sap.com> Message-ID: <940d8a26-d5b3-0d5b-ccb8-a71d4d17efdc@oracle.com> Hi Ralf, Overall looks good, but I do have a few suggestions and questions: I think I prefer passing EI_VM_INIT for triggering_ei, but if you prefer to keep it as is, I suggest a comment to clarify why it might be out of range. The help for "oncmd" says "debug triggered by jcmd?". I think your use of "cmd" should be more consistent. It's a dcmd providing the service, and jcmd is just one way to get at the dcmd. I'm not sure of the best way to convey this simply in the option name and help, but it should be consistent. "ondcmd" is probably most accurate, but "onjcmd" is probably what is most meaningful to users. I don't think it makes sense to use "oncmd" if the help is referring to jcmd. Your error handling in debugInit_startDebuggingViaCommand() is a bit inconsistent. For !allowStartViaCmd you immediately return the error. For !allowStartViaCmd you set the error, and then need to do error checks later before eventually returning the error. It think you could just return the error right away and remove the error checking code that comes later. It's not clear to me why you want the name and address of the first transport. Why is the first one necessarily the one you want? 1403???????? bagEnumerateOver(transports, getFirstTransport, &spec); 1404 1405???????? if ((spec != NULL) && (transport_name != NULL) && (address != NULL)) { 1406???????????? *transport_name = spec->name; 1407???????????? *address = spec->address; 1408???????? } thanks, Chris On 12/5/18 6:56 AM, Schmelter, Ralf wrote: > Hi All, > > Please review the fix for the bug https://bugs.openjdk.java.net/browse/JDK-8214892. The webref is at http://cr.openjdk.java.net/~rrich/webrevs/schmelter/8214892/webrev.01/ . > > This change allows debugging to be started delayed, triggered by a jcmd name VM.start_java_debugging. It works more or less like the onthrow and onuncaught options, but instead of triggering by exception we trigger it via the jcmd. This is useful when we want to trigger debugging when we see errors not related to exceptions (e.g. if a server becomes unresponsive it could be taken out of the routing and debugging is triggered to find the cause). > > Best regards, > Ralf From patrick at reini.net Thu Dec 6 07:58:26 2018 From: patrick at reini.net (Patrick Reinhart) Date: Thu, 06 Dec 2018 08:58:26 +0100 Subject: JDK-8171311 Current state Message-ID: <09715124f23fca815e2a27f7fbd2558b@reini.net> Hi everybody, I am already done some contributions within the core libs of the JDK and wanted to ask if I could help in bringing this JEP forward. Looking into the it the last actions where made mid year- Is there any work being done here in the mean time? -Patrick From Alan.Bateman at oracle.com Thu Dec 6 08:30:46 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 6 Dec 2018 08:30:46 +0000 Subject: JDK-8171311 Current state In-Reply-To: <09715124f23fca815e2a27f7fbd2558b@reini.net> References: <09715124f23fca815e2a27f7fbd2558b@reini.net> Message-ID: <322750d5-478c-5e4a-dedb-56d3c4ae1e1b@oracle.com> On 06/12/2018 07:58, Patrick Reinhart wrote: > Hi everybody, > > I am already done some contributions within the core libs of the JDK > and wanted to ask if I could help in bringing this JEP forward. > Looking into the it the last actions where made mid year- Is there any > work being done here in the mean time? I'm not aware of any current activity on this in OpenJDK. One thing about the JEP is that it didn't make the case clear as to why the adapter needed to be in the JDK. There are also several existing JMX adapters that support REST and it would have been useful to evaluate those and maybe explore what the pain points and issues are with deploying these solutions. It might be that the -Dcom.sun.management mechanism to start the JMX agent needs to be improved, maybe it should be integrated with -javaagent, maybe the pluggability of the JMX agent just needs to be improved. -Alan From gary.adams at oracle.com Thu Dec 6 13:27:45 2018 From: gary.adams at oracle.com (Gary Adams) Date: Thu, 06 Dec 2018 08:27:45 -0500 Subject: RFR: JDK-8210106: sun/tools/jps/TestJps.java timed out Message-ID: <5C0923D1.8080705@oracle.com> On a local linux-x64-debug build this test consistently runs in less than 40 seconds. On the mach5 testing machines there was a large fluctuation in the time to complete this test. Since the test runs jps with different combinations of arguments, the total test time is dependent on the number of processes running java programs. Since the mach5 test infrastructure runs java programs I have seen a 3X in the amount of output the test produces compared to local test runs. I've run the test several hundred times through mach5 on the slower platforms and then examined the test logs using a 3X setting from the default 120 second jtreg timeout. The slowest reported elapse time from the test logs showed 280 seconds to complete. To improve the reliability of the test to complete, I'd like to increase the timeout to 360 seconds. Issue: https://bugs.openjdk.java.net/browse/JDK-8210106 Proposed fix: diff --git a/test/jdk/sun/tools/jps/TestJps.java b/test/jdk/sun/tools/jps/TestJps.java --- a/test/jdk/sun/tools/jps/TestJps.java +++ b/test/jdk/sun/tools/jps/TestJps.java @@ -27,7 +27,7 @@ * @modules jdk.jartool/sun.tools.jar * @build LingeredAppForJps * @build LingeredApp - * @run main/othervm TestJps + * @run main/othervm/timeout=360 TestJps */ From coleen.phillimore at oracle.com Thu Dec 6 13:47:53 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 6 Dec 2018 08:47:53 -0500 Subject: RFR (round 5), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <8c267450-9ca8-79a6-0e4a-39b3b68af2e3@redhat.com> <5788143E-E206-4795-B7B8-B47FDDF3B54F@oracle.com> Message-ID: <496b3ac5-defc-92b6-7339-64c88c2e2dfa@oracle.com> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/shenandoah-gc/src/hotspot/share/gc/shenandoah/vm_operations_shenandoah.cpp.html Can you rename these to shenandoahVMOperations.hpp/cpp to match the newly agreed upon naming convention for this? See 8214791: Consistently name gc files containing VM operations [Was: Re: RFR (S): 8214791: Rename vm_operations_g1* files to g1VMOperations*] thanks, Coleen On 12/4/18 3:37 PM, Roman Kennke wrote: > Thanks, Leonid, for reviewing! > > Roman > > >> Hi >> >> The shared tests changes looks good for me. Thank you for fixing and testing different combinations. >> >> Leonid >> >>> On Dec 3, 2018, at 11:10 PM, Roman Kennke wrote: >>> >>> Round 5 of Shenandoah review includes: >>> - A fix for the @requires tag in TestFullGCCountTest.java. It should be >>> correct now. We believe the CMS @requires was also not quite right and >>> fixed it the same. >>> >>> It reads now: Don't run this test if: >>> - Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is >>> true, as set by harness >>> - Actual GC set by harness is Shenandoah *and* >>> ExplicitGCInvokesConcurrent is not set false by harness (it's true by >>> default in Shenandoah, so this needs to be double-inverteed). >>> >>> The @requires for CMS was wrong before (we think), because it would also >>> filter defaultGC + ExplicitGCInvokesConcurrent. >>> >>> - Sorting of macros was fixed, as was pointed out by Per >>> - Some stuff was added to SA, as suggested by Jini >>> - Rebased on most current jdk/jdk code >>> >>> Webrevs: >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/ >>> >>> I also need reviews from GC reviewers for the CSR: >>> https://bugs.openjdk.java.net/browse/JDK-8214349 >>> >>> I already got reviews for: >>> [x] shared-runtime (coleenp) >>> [x] shared-compiler (kvn) >>> >>> I got reviews for shared-build, but an earlier version, so maybe makes >>> sense to look over this again. Erik J, Magnus? >>> >>> I still need approvals for: >>> [ ] shared-build (kvn, erikj, ihse, pliden) >>> [ ] shared-gc (pliden, kbarrett) >>> [ ] shared-serviceability (jgeorge, pliden) >>> [ ] shared-tests (lmesnik, pliden) >>> [ ] shenandoah-gc >>> [ ] shenandoah-tests >>> >>> >>> Thanks for your patience and ongoing support! >>> >>> Cheers, >>> Roman >>> >>> >>>> Hi all, >>>> >>>> here comes round 4 of Shenandoah upstreaming review: >>>> >>>> This includes fixes for the issues that Per brought up: >>>> - Verify and gracefully reject dangerous flags combinations that >>>> disables required barriers >>>> - Revisited @requires filters in tests >>>> - Trim unused code from Shenandoah's SA impl >>>> - Move ShenandoahGCTracer to gc/shenandoah >>>> - Fix ordering of GC names in various files >>>> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >>>> >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >>>> >>>> Thanks everybody for taking time to review this! >>>> Roman >>>> >>>>> Hello all, >>>>> >>>>> Thanks so far for all the reviews and support! >>>>> >>>>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>>>> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >>>>> >>>>> Things we've changed today: >>>>> - We moved shenandoah-specific code out of .ad files into our own .ad >>>>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>>>> requires an addition in build machinery though, see >>>>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>>>> - Improved zero-disabling and build-code-simplification as suggested by >>>>> Magnus and Per >>>>> - Cleaned up some leftovers in C2 >>>>> - Improved C2 loop opts code by introducing another APIs in >>>>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>>>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now. >>>>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>>>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>>>> that. >>>>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>>>> - Rebased on jdk-12+22 >>>>> >>>>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>>>> >>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>>>> >>>>> Thanks, >>>>> Roman >>>>> >>>>>> Alright, we fixed: >>>>>> - The minor issues that Kim reported in shared-gc >>>>>> - A lot of fixes in shared-tests according to Leonid's review >>>>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>>>> >>>>>> Some notes: >>>>>> Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>>>> correct. The @requires there means to exclude runs with both CMS and >>>>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>>>> fine. Adding the same clause for Shenandoah means the same, and filters >>>>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>>>> made the condition a bit clearer by avoiding triple-negation. >>>>>> >>>>>> See: >>>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>>>> >>>>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>>>> >>>>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>>>> those with ZGC? >>>>>> >>>>>> See: >>>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>>>> >>>>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>>>> next round). >>>>>> >>>>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>>>> better. I can tell that we're not done with C2 yet. Can you look over >>>>>> the code and see what is ok, and especially what is not ok, so that we >>>>>> can focus our efforts on the relevant parts? >>>>>> >>>>>> Updated set of webrevs: >>>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>>>> >>>>>> Thanks, >>>>>> Roman >>>>>> >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> This is the first round of changes for including Shenandoah GC into >>>>>>> mainline. >>>>>>> I divided the review into parts that roughly correspond to the mailing lists >>>>>>> that would normally review it, and I divided it into 'shared' code >>>>>>> changes and >>>>>>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>>>>>> eventually >>>>>>> push them as single 'combined' changeset, once reviewed. >>>>>>> >>>>>>> JEP: >>>>>>> https://openjdk.java.net/jeps/189 >>>>>>> Bug entry: >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>>>> >>>>>>> Webrevs: >>>>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>>>> >>>>>>> For those who want to see the full change, have a look at the >>>>>>> shenandoah-complete >>>>>>> >>>>>>> directory, >>>>>>> it contains the full combined webrev. Alternatively, there is the file >>>>>>> shenandoah-master.patch >>>>>>> , >>>>>>> which is what I intend to commit (and which should be equivalent to the >>>>>>> 'shenandoah-complete' webrev). >>>>>>> >>>>>>> Sections to review (at this point) are the following: >>>>>>> *) shenandoah-gc >>>>>>> >>>>>>> - Actual Shenandoah implementation, almost completely residing in >>>>>>> gc/shenandoah >>>>>>> >>>>>>> *) shared-gc >>>>>>> >>>>>>> - This is mostly boilerplate that is common to any GC >>>>>>> - referenceProcessor.cpp has a little change to make one assert not >>>>>>> fail (next to CMS and G1) >>>>>>> - taskqueue.hpp has some small adjustments to enable subclassing >>>>>>> >>>>>>> *) shared-serviceability >>>>>>> >>>>>>> - The usual code to support another GC >>>>>>> >>>>>>> *) shared-runtime >>>>>>> >>>>>>> - A number of friends declarations to allow Shenandoah iterators to >>>>>>> hook up with, >>>>>>> e.g. ClassLoaderData, CodeCache, etc >>>>>>> - Warning and disabling JFR LeakProfiler >>>>>>> - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>>>> Shenandoah C2 optimizations >>>>>>> - Locks initialization in mutexLocker.cpp as usual >>>>>>> - VM operations defines for Shenandoah's VM ops >>>>>>> - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>>>> Shenandoah's logging >>>>>>> - The usual macros in macro.hpp >>>>>>> >>>>>>> *) shared-build >>>>>>> >>>>>>> - Add shenandoah feature, enabled by default, as agreed with >>>>>>> Vladimir K. beforehand >>>>>>> - Some flags for shenandoah-enabled compilation to get >>>>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>>>> and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>>>> Shenandoah's barriers >>>>>>> - --param inline-unit-growth=1000 settings for 2 shenandoah source >>>>>>> files, which is >>>>>>> useful to get the whole marking loop inlined (observed significant >>>>>>> regression if we >>>>>>> don't) >>>>>>> >>>>>>> *) shared-tests >>>>>>> >>>>>>> - Test infrastructure to support Shenandoah >>>>>>> - Shenandoah test groups >>>>>>> - Exclude Shenandoah in various tests that can be run with selected GC >>>>>>> - Enable/add configure for Shenandoah for tests that make sense to >>>>>>> run with it >>>>>>> >>>>>>> *) shenandoah-tests >>>>>>> >>>>>>> - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >>>>>>> - A couple of tests configurations have been added, e.g. >>>>>>> TestGCBasherWithShenandoah.java >>>>>>> >>>>>>> I intentionally left out shared-compiler for now, because we have some >>>>>>> work left to do >>>>>>> there, but if you click around you'll find the patch anyway, in case you >>>>>>> want to take >>>>>>> a peek at it. >>>>>>> >>>>>>> We have regular builds on: >>>>>>> - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>>>> - {Windows} x {x86_64}, >>>>>>> - {MacOS X} x {x86_64} >>>>>>> >>>>>>> This also routinely passes: >>>>>>> - the new Shenandoah tests >>>>>>> - jcstress with/without aggressive Shenandoah verification >>>>>>> - specjvm2008 with/without aggressive Shenandoah verification >>>>>>> >>>>>>> >>>>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>>>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>>>> teams for tirelessly helping with and reviewing all the GC interface and >>>>>>> related changes, and of course the many early adopters for reporting >>>>>>> bugs and success stories and feature requests: we wouldn't be here >>>>>>> without any of you! >>>>>>> >>>>>>> Best regards, >>>>>>> Roman >>>>>>> From rkennke at redhat.com Thu Dec 6 14:36:24 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 6 Dec 2018 15:36:24 +0100 Subject: RFR (round 5), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <496b3ac5-defc-92b6-7339-64c88c2e2dfa@oracle.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <8c267450-9ca8-79a6-0e4a-39b3b68af2e3@redhat.com> <5788143E-E206-4795-B7B8-B47FDDF3B54F@oracle.com> <496b3ac5-defc-92b6-7339-64c88c2e2dfa@oracle.com> Message-ID: <73e720e4-251c-aa06-418b-3fc8409e7cf6@redhat.com> Hi Coleen, > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/shenandoah-gc/src/hotspot/share/gc/shenandoah/vm_operations_shenandoah.cpp.html > > > Can you rename these to shenandoahVMOperations.hpp/cpp to match the > newly agreed upon naming convention for this? > > See 8214791: Consistently name gc files containing VM operations [Was: > Re: RFR (S): 8214791: Rename vm_operations_g1* files to g1VMOperations*] Doing so: http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-December/008595.html I will integrate this in next round of webrevs. I expect the next will be the final round of webrevs. I got positive reviews for all shared-* parts, I'm waiting for shenandoah-* reviews from Shenandoah devs, plus Zhengyu+Thomas' TaskQueue stuff to arrive in upstream jdk. The CSR for Shenandoah flags has been approved, and the JEP should be moved to targeted JDK12 ~tomorrow. I intend/expect to push Shenandoah during the next couple of days, unless somebody speaks up until then :-) Thanks, Roman > > thanks, > Coleen > > On 12/4/18 3:37 PM, Roman Kennke wrote: >> Thanks, Leonid, for reviewing! >> >> Roman >> >> >>> Hi >>> >>> The shared tests changes looks good for me. Thank you for fixing and >>> testing different combinations. >>> >>> Leonid >>> >>>> On Dec 3, 2018, at 11:10 PM, Roman Kennke wrote: >>>> >>>> Round 5 of Shenandoah review includes: >>>> - A fix for the @requires tag in TestFullGCCountTest.java. It should be >>>> correct now. We believe the CMS @requires was also not quite right and >>>> fixed it the same. >>>> >>>> It reads now: Don't run this test if: >>>> - Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is >>>> true, as set by harness >>>> - Actual GC set by harness is Shenandoah *and* >>>> ExplicitGCInvokesConcurrent is not set false by harness (it's true by >>>> default in Shenandoah, so this needs to be double-inverteed). >>>> >>>> The @requires for CMS was wrong before (we think), because it would >>>> also >>>> filter defaultGC + ExplicitGCInvokesConcurrent. >>>> >>>> - Sorting of macros was fixed, as was pointed out by Per >>>> - Some stuff was added to SA, as suggested by Jini >>>> - Rebased on most current jdk/jdk code >>>> >>>> Webrevs: >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/ >>>> >>>> I also need reviews from GC reviewers for the CSR: >>>> https://bugs.openjdk.java.net/browse/JDK-8214349 >>>> >>>> I already got reviews for: >>>> [x] shared-runtime (coleenp) >>>> [x] shared-compiler (kvn) >>>> >>>> I got reviews for shared-build, but an earlier version, so maybe makes >>>> sense to look over this again. Erik J, Magnus? >>>> >>>> I still need approvals for: >>>> [ ] shared-build????????? (kvn, erikj, ihse, pliden) >>>> [ ] shared-gc???????????? (pliden, kbarrett) >>>> [ ] shared-serviceability (jgeorge, pliden) >>>> [ ] shared-tests????????? (lmesnik, pliden) >>>> [ ] shenandoah-gc >>>> [ ] shenandoah-tests >>>> >>>> >>>> Thanks for your patience and ongoing support! >>>> >>>> Cheers, >>>> Roman >>>> >>>> >>>>> Hi all, >>>>> >>>>> here comes round 4 of Shenandoah upstreaming review: >>>>> >>>>> This includes fixes for the issues that Per brought up: >>>>> - Verify and gracefully reject dangerous flags combinations that >>>>> disables required barriers >>>>> - Revisited @requires filters in tests >>>>> - Trim unused code from Shenandoah's SA impl >>>>> - Move ShenandoahGCTracer to gc/shenandoah >>>>> - Fix ordering of GC names in various files >>>>> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >>>>> >>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >>>>> >>>>> Thanks everybody for taking time to review this! >>>>> Roman >>>>> >>>>>> Hello all, >>>>>> >>>>>> Thanks so far for all the reviews and support! >>>>>> >>>>>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>>>>> Also, I fixed the numbering of my webrevs to match the >>>>>> review-round. ;-) >>>>>> >>>>>> Things we've changed today: >>>>>> - We moved shenandoah-specific code out of .ad files into our own .ad >>>>>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>>>>> requires an addition in build machinery though, see >>>>>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>>>>> - Improved zero-disabling and build-code-simplification as >>>>>> suggested by >>>>>> Magnus and Per >>>>>> - Cleaned up some leftovers in C2 >>>>>> - Improved C2 loop opts code by introducing another APIs in >>>>>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>>>>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC >>>>>> guards now. >>>>>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>>>>> noted earlier. This stuff is Shenandoah-specific, so let's just >>>>>> call it >>>>>> that. >>>>>> - Rehashed Shenandoah tests (formatting, naming, directory layout, >>>>>> etc) >>>>>> - Rebased on jdk-12+22 >>>>>> >>>>>> - Question: let us know if you need separate RFE for the new BSC2 >>>>>> APIs? >>>>>> >>>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>>>>> >>>>>> Thanks, >>>>>> Roman >>>>>> >>>>>>> Alright, we fixed: >>>>>>> - The minor issues that Kim reported in shared-gc >>>>>>> - A lot of fixes in shared-tests according to Leonid's review >>>>>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>>>>> >>>>>>> Some notes: >>>>>>> Leonid:? test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>>>>> correct. The @requires there means to exclude runs with both CMS and >>>>>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>>>>> (expectedly) failing. It can run CMS, default GC and any other GC >>>>>>> just >>>>>>> fine. Adding the same clause for Shenandoah means the same, and >>>>>>> filters >>>>>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>>>>> made the condition a bit clearer by avoiding triple-negation. >>>>>>> >>>>>>> See: >>>>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>>>>> >>>>>>> >>>>>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>>>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>>>>> - >>>>>>> test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>>>>> >>>>>>> we filter them for Shenandoah now. I'm wondering: how do you get >>>>>>> past >>>>>>> those with ZGC? >>>>>>> >>>>>>> See: >>>>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>>>>> >>>>>>> >>>>>>> (Note to Leonid and tests reviewers: I'll add those related >>>>>>> filters in >>>>>>> next round). >>>>>>> >>>>>>> Vladimir: Roland integrated a bunch of changes to make loop* code >>>>>>> look >>>>>>> better. I can tell that we're not done with C2 yet. Can you look >>>>>>> over >>>>>>> the code and see what is ok, and especially what is not ok, so >>>>>>> that we >>>>>>> can focus our efforts on the relevant parts? >>>>>>> >>>>>>> Updated set of webrevs: >>>>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>>>>> >>>>>>> Thanks, >>>>>>> Roman >>>>>>> >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> This is the first round of changes for including Shenandoah GC into >>>>>>>> mainline. >>>>>>>> I divided the review into parts that roughly correspond to the >>>>>>>> mailing lists >>>>>>>> that would normally review it, and I divided it into 'shared' code >>>>>>>> changes and >>>>>>>> 'shenandoah' code changes (actually, mostly additions). The >>>>>>>> intend is to >>>>>>>> eventually >>>>>>>> push them as single 'combined' changeset, once reviewed. >>>>>>>> >>>>>>>> JEP: >>>>>>>> ?? https://openjdk.java.net/jeps/189 >>>>>>>> Bug entry: >>>>>>>> >>>>>>>> ? https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>>>>> >>>>>>>> Webrevs: >>>>>>>> ?? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>>>>> >>>>>>>> For those who want to see the full change, have a look at the >>>>>>>> shenandoah-complete >>>>>>>> >>>>>>>> >>>>>>>> directory, >>>>>>>> it contains the full combined webrev. Alternatively, there is >>>>>>>> the file >>>>>>>> shenandoah-master.patch >>>>>>>> , >>>>>>>> >>>>>>>> which is what I intend to commit (and which should be equivalent >>>>>>>> to the >>>>>>>> 'shenandoah-complete' webrev). >>>>>>>> >>>>>>>> Sections to review (at this point) are the following: >>>>>>>> ? *) shenandoah-gc >>>>>>>> >>>>>>>> >>>>>>>> ???? - Actual Shenandoah implementation, almost completely >>>>>>>> residing in >>>>>>>> gc/shenandoah >>>>>>>> >>>>>>>> ? *) shared-gc >>>>>>>> >>>>>>>> >>>>>>>> ???? - This is mostly boilerplate that is common to any GC >>>>>>>> ???? - referenceProcessor.cpp has a little change to make one >>>>>>>> assert not >>>>>>>> fail (next to CMS and G1) >>>>>>>> ???? - taskqueue.hpp has some small adjustments to enable >>>>>>>> subclassing >>>>>>>> >>>>>>>> ? *) shared-serviceability >>>>>>>> >>>>>>>> >>>>>>>> ???? - The usual code to support another GC >>>>>>>> >>>>>>>> ? *) shared-runtime >>>>>>>> >>>>>>>> >>>>>>>> ???? - A number of friends declarations to allow Shenandoah >>>>>>>> iterators to >>>>>>>> hook up with, >>>>>>>> ?????? e.g. ClassLoaderData, CodeCache, etc >>>>>>>> ???? - Warning and disabling JFR LeakProfiler >>>>>>>> ???? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>>>>> Shenandoah C2 optimizations >>>>>>>> ???? - Locks initialization in mutexLocker.cpp as usual >>>>>>>> ???? - VM operations defines for Shenandoah's VM ops >>>>>>>> ???? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>>>>> Shenandoah's logging >>>>>>>> ???? - The usual macros in macro.hpp >>>>>>>> >>>>>>>> ? *) shared-build >>>>>>>> >>>>>>>> >>>>>>>> ???? - Add shenandoah feature, enabled by default, as agreed with >>>>>>>> Vladimir K. beforehand >>>>>>>> ???? - Some flags for shenandoah-enabled compilation to get >>>>>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>>>>> ?????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>>>>> Shenandoah's barriers >>>>>>>> ???? - --param inline-unit-growth=1000 settings for 2 shenandoah >>>>>>>> source >>>>>>>> files, which is >>>>>>>> ?????? useful to get the whole marking loop inlined (observed >>>>>>>> significant >>>>>>>> regression if we >>>>>>>> ?????? don't) >>>>>>>> >>>>>>>> ? *) shared-tests >>>>>>>> >>>>>>>> >>>>>>>> ???? - Test infrastructure to support Shenandoah >>>>>>>> ???? - Shenandoah test groups >>>>>>>> ???? - Exclude Shenandoah in various tests that can be run with >>>>>>>> selected GC >>>>>>>> ???? - Enable/add configure for Shenandoah for tests that make >>>>>>>> sense to >>>>>>>> run with it >>>>>>>> >>>>>>>> ? *) shenandoah-tests >>>>>>>> >>>>>>>> >>>>>>>> ???? - Shenandoah specific tests, most reside in gc/shenandoah >>>>>>>> subdirectory >>>>>>>> ???? - A couple of tests configurations have been added, e.g. >>>>>>>> TestGCBasherWithShenandoah.java >>>>>>>> >>>>>>>> I intentionally left out shared-compiler for now, because we >>>>>>>> have some >>>>>>>> work left to do >>>>>>>> there, but if you click around you'll find the patch anyway, in >>>>>>>> case you >>>>>>>> want to take >>>>>>>> a peek at it. >>>>>>>> >>>>>>>> We have regular builds on: >>>>>>>> ?? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>>>>> ?? - {Windows} x {x86_64}, >>>>>>>> ?? - {MacOS X} x {x86_64} >>>>>>>> >>>>>>>> This also routinely passes: >>>>>>>> ?? - the new Shenandoah tests >>>>>>>> ?? - jcstress with/without aggressive Shenandoah verification >>>>>>>> ?? - specjvm2008 with/without aggressive Shenandoah verification >>>>>>>> >>>>>>>> >>>>>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she >>>>>>>> deserves >>>>>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>>>>> contributions, everybody else in Red Hat's OpenJDK team for >>>>>>>> testing, >>>>>>>> advice and support, my collegues in Oracle's GC, runtime and >>>>>>>> compiler >>>>>>>> teams for tirelessly helping with and reviewing all the GC >>>>>>>> interface and >>>>>>>> related changes, and of course the many early adopters for >>>>>>>> reporting >>>>>>>> bugs and success stories and feature requests: we wouldn't be here >>>>>>>> without any of you! >>>>>>>> >>>>>>>> Best regards, >>>>>>>> Roman >>>>>>>> > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From ralf.schmelter at sap.com Thu Dec 6 14:54:24 2018 From: ralf.schmelter at sap.com (Schmelter, Ralf) Date: Thu, 6 Dec 2018 14:54:24 +0000 Subject: RFR (M) 8214892: Delayed starting of debugging via jcmd In-Reply-To: <940d8a26-d5b3-0d5b-ccb8-a71d4d17efdc@oracle.com> References: <01fdfc8e58a246c18d5909fc7df313b7@sap.com> <940d8a26-d5b3-0d5b-ccb8-a71d4d17efdc@oracle.com> Message-ID: <6283122f12f041caa56f768f8005920b@sap.com> Hi Chris, > I think I prefer passing EI_VM_INIT for triggering_ei, but if you prefer > to keep it as is, I suggest a comment to clarify why it might be out of > range. Actually, I used EI_VM_INIT for the longest time and only changed it recently, because I thought that code could assume that e.g. no classes have been loaded yet when getting the INIT_VM event. But since the JVMTI spec does not guarantees this in any way (it allows other events to be send before a VM_INIT), I just will change it back to use EI_VM_INIT for the initialize call. Regarding the name of the option, I agree that onjcmd, while not technically fully accurate, makes most sense for the common user. > ... It think you could just return the error right away and remove > the error checking code that comes later. I've changed the code to just return the error directly. > It's not clear to me why you want the name and address of the first > transport. Why is the first one necessarily the one you want? Since currently the bag of transports must always have a size of 1, getting the first or the last transport is the same. But the callback function used to iterate the bag has to return a boolean value. I've decided to return JNI_FALSE, which would mean the iteration stops at the first entry. I've updated the webrev with your and Goetz Lindenmeier's suggestions: http://cr.openjdk.java.net/~rrich/webrevs/schmelter/8214892/webrev.02/ Best regards, Ralf From daniel.daugherty at oracle.com Thu Dec 6 14:57:32 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 6 Dec 2018 09:57:32 -0500 Subject: RFR: JDK-8210106: sun/tools/jps/TestJps.java timed out In-Reply-To: <5C0923D1.8080705@oracle.com> References: <5C0923D1.8080705@oracle.com> Message-ID: <795c5546-a3c7-e411-96c3-455b848c3701@oracle.com> On 12/6/18 8:27 AM, Gary Adams wrote: > On a local linux-x64-debug build this test consistently runs in less > than 40 seconds. > On the mach5 testing machines there was a large fluctuation in the > time to complete this test. > > Since the test runs jps with different combinations of arguments, the > total > test time is dependent on the number of processes running java programs. > Since the mach5 test infrastructure runs java programs I have seen a 3X > in the amount of output the test produces compared to local test > runs. > > I've run the test several hundred times through mach5 on the slower > platforms > and then examined the test logs using a 3X setting from the default > 120 second > jtreg timeout. The slowest reported elapse time from the test logs showed > 280 seconds to complete. > > To improve the reliability of the test to complete, I'd like to > increase the > timeout to 360 seconds. > > ? Issue: https://bugs.openjdk.java.net/browse/JDK-8210106 > > Proposed fix: > > diff --git a/test/jdk/sun/tools/jps/TestJps.java > b/test/jdk/sun/tools/jps/TestJps.java > --- a/test/jdk/sun/tools/jps/TestJps.java > +++ b/test/jdk/sun/tools/jps/TestJps.java > @@ -27,7 +27,7 @@ > ? * @modules jdk.jartool/sun.tools.jar > ? * @build LingeredAppForJps > ? * @build LingeredApp > - * @run main/othervm TestJps > + * @run main/othervm/timeout=360 TestJps > ? */ > Thumbs up! And trivial. Dan From gary.adams at oracle.com Thu Dec 6 16:37:18 2018 From: gary.adams at oracle.com (Gary Adams) Date: Thu, 06 Dec 2018 11:37:18 -0500 Subject: JDK-8160380: com/sun/jdi/GenericsTest.java timeout Message-ID: <5C09503E.5010208@oracle.com> I'd like to close this issue as "cannot reproduce". Issue: https://bugs.openjdk.java.net/browse/JDK-8160380 The original issue was reported 2+ years ago and was looking for someone to examine the core files to see if a regression could be identified. The core files were not preserved, so they can not be examined. Local test runs and mach5 test runs were done for the full set of test/jdk/com/sun/jdi tests in context. No failures have been replicated and the GenericsTest in particular runs in under 2 seconds. There is nothing specific in the test itself that is likely to have a scaling issue. From chris.plummer at oracle.com Thu Dec 6 17:32:34 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 6 Dec 2018 09:32:34 -0800 Subject: RFR (M) 8214892: Delayed starting of debugging via jcmd In-Reply-To: <6283122f12f041caa56f768f8005920b@sap.com> References: <01fdfc8e58a246c18d5909fc7df313b7@sap.com> <940d8a26-d5b3-0d5b-ccb8-a71d4d17efdc@oracle.com> <6283122f12f041caa56f768f8005920b@sap.com> Message-ID: <9b3a023b-7ba6-ab9b-d7e9-3f57e78347a7@oracle.com> Hi Ralf, I don't think the comments from Goetz made it to the list. However, with the current changes it all looks good to me, except there are some indentation issues. Indent tab stops are should be increments of 2 for hotspot code. You have a few cases of using 4 in the dcmd files. For the jdwp agent, pre-existing code uses tab stops of 4, so should remain that way, and it looks like you've correctly done this. thanks, Chris On 12/6/18 6:54 AM, Schmelter, Ralf wrote: > Hi Chris, > >> I think I prefer passing EI_VM_INIT for triggering_ei, but if you prefer >> to keep it as is, I suggest a comment to clarify why it might be out of >> range. > Actually, I used EI_VM_INIT for the longest time and only changed it recently, because I thought that code could assume that e.g. no classes have been loaded yet when getting the INIT_VM event. But since the JVMTI spec does not guarantees this in any way (it allows other events to be send before a VM_INIT), I just will change it back to use EI_VM_INIT for the initialize call. > > Regarding the name of the option, I agree that onjcmd, while not technically fully accurate, makes most sense for the common user. > >> ... It think you could just return the error right away and remove >> the error checking code that comes later. > I've changed the code to just return the error directly. > >> It's not clear to me why you want the name and address of the first >> transport. Why is the first one necessarily the one you want? > Since currently the bag of transports must always have a size of 1, getting the first or the last transport is the same. But the callback function used to iterate the bag has to return a boolean value. I've decided to return JNI_FALSE, which would mean the iteration stops at the first entry. > > > I've updated the webrev with your and Goetz Lindenmeier's suggestions: http://cr.openjdk.java.net/~rrich/webrevs/schmelter/8214892/webrev.02/ > > Best regards, > Ralf From jcbeyler at google.com Thu Dec 6 17:41:53 2018 From: jcbeyler at google.com (JC Beyler) Date: Thu, 6 Dec 2018 09:41:53 -0800 Subject: RFR (XS) 8214531: HeapMonitorEventOnOffTest.java test fails with "Statistics should be null to begin with" In-Reply-To: References: <7be7bd58-8efe-9edf-bcd4-48a1ee06ff40@oracle.com> Message-ID: Hi all, Anyway I could get a second review on this please? :) Webrev: http://cr.openjdk.java.net/~jcbeyler/8214531/webrev.01/ Bug: https://bugs.openjdk.java.net/browse/JDK-8214531 Thanks! Jc On Fri, Nov 30, 2018 at 1:53 PM JC Beyler wrote: > Hi Serguei, thanks! > > Done here then: > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8214531/webrev.01/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8214531 > > I've sent it to the submit repo while waiting for a second review :) > > Have a great friday! > Jc > > On Fri, Nov 30, 2018 at 12:22 PM serguei.spitsyn at oracle.com < > serguei.spitsyn at oracle.com> wrote: > >> Hi Jc, >> >> Looks better. >> >> Thanks, >> Serguei >> >> >> >> On 11/30/18 11:48, JC Beyler wrote: >> >> Hi Serguei, >> >> Technically "allocateAndCheckFrames" does enable it internally and the >> comment helps understand that we are testing "sampling on - off - on", no? >> >> I find that without the comments, you now have to understand what allocateAndCheckFrames >> does implicitly. >> >> We could change it to this: >> // By calling allocateAndCheckFrames(), we enabling the notifications >> and check if allocations get sampled again >> >> Does that look better? >> >> Thanks! >> Jc >> >> On Fri, Nov 30, 2018 at 11:12 AM serguei.spitsyn at oracle.com < >> serguei.spitsyn at oracle.com> wrote: >> >>> Hi Jc, >>> >>> It looks good in general. >>> I wonder if this comment is still needed: >>> // Enabling the notification system should start events again. >>> >>> Thanks, >>> Serguei >>> >>> >>> On 11/30/18 09:08, JC Beyler wrote: >>> >>> Hi all, >>> >>> Tiny webrev that removes enabling the sampling system for the >>> HeapMonitorEventOnOffTest before calling the allocateAndCheckFrames method >>> (the allocateAndCheckFrames method enables it internally when no flag >>> is passed to it). >>> >>> Webrev: http://cr.openjdk.java.net/~jcbeyler/8214531/webrev.00/ >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8214531 >>> >>> Thanks, >>> Jc >>> >>> >>> >> >> -- >> >> Thanks, >> Jc >> >> >> > > -- > > Thanks, > Jc > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From hohensee at amazon.com Thu Dec 6 18:54:49 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Thu, 6 Dec 2018 18:54:49 +0000 Subject: RFR (XS) 8214531: HeapMonitorEventOnOffTest.java test fails with "Statistics should be null to begin with" In-Reply-To: References: <7be7bd58-8efe-9edf-bcd4-48a1ee06ff40@oracle.com> Message-ID: <9143DF4F-E68D-41D6-8C5B-429F806AC5A5@amazon.com> Lgtm. Paul From: serviceability-dev on behalf of JC Beyler Date: Thursday, December 6, 2018 at 9:42 AM To: "serguei.spitsyn at oracle.com" Cc: "serviceability-dev at openjdk.java.net" Subject: Re: RFR (XS) 8214531: HeapMonitorEventOnOffTest.java test fails with "Statistics should be null to begin with" Hi all, Anyway I could get a second review on this please? :) Webrev: http://cr.openjdk.java.net/~jcbeyler/8214531/webrev.01/ Bug: https://bugs.openjdk.java.net/browse/JDK-8214531 Thanks! Jc On Fri, Nov 30, 2018 at 1:53 PM JC Beyler > wrote: Hi Serguei, thanks! Done here then: Webrev: http://cr.openjdk.java.net/~jcbeyler/8214531/webrev.01/ Bug: https://bugs.openjdk.java.net/browse/JDK-8214531 I've sent it to the submit repo while waiting for a second review :) Have a great friday! Jc On Fri, Nov 30, 2018 at 12:22 PM serguei.spitsyn at oracle.com > wrote: Hi Jc, Looks better. Thanks, Serguei On 11/30/18 11:48, JC Beyler wrote: Hi Serguei, Technically "allocateAndCheckFrames" does enable it internally and the comment helps understand that we are testing "sampling on - off - on", no? I find that without the comments, you now have to understand what allocateAndCheckFrames does implicitly. We could change it to this: // By calling allocateAndCheckFrames(), we enabling the notifications and check if allocations get sampled again Does that look better? Thanks! Jc On Fri, Nov 30, 2018 at 11:12 AM serguei.spitsyn at oracle.com > wrote: Hi Jc, It looks good in general. I wonder if this comment is still needed: // Enabling the notification system should start events again. Thanks, Serguei On 11/30/18 09:08, JC Beyler wrote: Hi all, Tiny webrev that removes enabling the sampling system for the HeapMonitorEventOnOffTest before calling the allocateAndCheckFrames method (the allocateAndCheckFrames method enables it internally when no flag is passed to it). Webrev: http://cr.openjdk.java.net/~jcbeyler/8214531/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8214531 Thanks, Jc -- Thanks, Jc -- Thanks, Jc -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Thu Dec 6 21:18:41 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 6 Dec 2018 13:18:41 -0800 Subject: JDK-8160380: com/sun/jdi/GenericsTest.java timeout In-Reply-To: <5C09503E.5010208@oracle.com> References: <5C09503E.5010208@oracle.com> Message-ID: Ok by me. Chris On 12/6/18 8:37 AM, Gary Adams wrote: > I'd like to close this issue as "cannot reproduce". > > ? Issue: https://bugs.openjdk.java.net/browse/JDK-8160380 > > The original issue was reported 2+ years ago and was > looking for someone to examine the core files to see > if a regression could be identified. > > The core files were not preserved, so they can not be examined. > > Local test runs and mach5 test runs were done for the full set of > test/jdk/com/sun/jdi tests in context. No failures have been > replicated and the GenericsTest in particular runs in under 2 > seconds. There is nothing specific in the test itself that > is likely to have a scaling issue. From ali.ince at gmail.com Thu Dec 6 22:49:04 2018 From: ali.ince at gmail.com (=?utf-8?B?QWxpIMSwbmNl?=) Date: Thu, 6 Dec 2018 22:49:04 +0000 Subject: [PATCH] JDK-8214122 Prevent name mangling of jdwpTransport_OnLoad in Windows 32-bit DLL name decoration In-Reply-To: References: <2DD922C6-EFB1-4891-90F0-3378DEA91B1C@gmail.com> <07840796-f1c4-e077-7b0b-cd5c1a880ec1@oracle.com> <00e58330-26f0-4401-f911-24fda0127bcc@oracle.com> <34d2b504-c4eb-6df3-3bd1-e23b9c22ea6a@oracle.com> <01b80c3f-8f71-26ec-304e-a7b245924bd5@oracle.com> <17e7873f-f2f4-dc02-82a6-2d7eb1565ca2@oracle.com> <5e032661-b70c-c6ec-2700-120589300c9e@oracle.com> Message-ID: <5A54BA77-B240-472A-A4B7-535C3A30A20F@gmail.com> Hi Magnus, Alexey, I believe we won?t be able to get further opinions from serviceability-dev. Andrew Luo suggested using a similar mechanism as is used for jvm.dll by using symbol name files mapped by platform (files are under make/hotspot/symbols and interestingly windows is almost the only platform for which a symbol file doesn?t exist). Another issue, again opened against AdoptOpenJDK 32-bit builds is somehow related with the same problem - but this time it is jimage.dll depending on zip.dll expecting the ZIP_InflateFully function to be decorated with JNICALL - whereas it was removed earlier from the implementation and the runtime image just fails with access violation just because jimage source code still declared it with JNICALL (https://github.com/AdoptOpenJDK/openjdk-build/issues/763 ). I?ve also searched for GetProcAddress calls across the source code - and I think it?s important to work on the consistency of JNICALL usages across the whole source code. Another noteworthy thing I?ve noticed is there are still JNICALL modifiers for example in https://github.com/AdoptOpenJDK/openjdk-jdk11u/blob/master/src/jdk.crypto.cryptoki/windows/native/libj2pkcs11/j2secmod_md.c#L49 - which I guess will also be reported as broken since these functions are again name decorated. If we?re still determined to remove JNICALL, what about just removing __stdcall from JNICALL declaration at https://github.com/AdoptOpenJDK/openjdk-jdk11u/blob/master/src/java.base/windows/native/include/jni_md.h#L31 ? Wouldn?t that be a more consistent move? Regards, Ali > On 22 Nov 2018, at 17:29, Magnus Ihse Bursie wrote: > > I think we are in complete agreement. What's missing is some expert opinion from serviceability-dev if there is any problem with changing the signature. I'd preferably have that. > > If no one knows, I'd say, change it, and see if we still pass the relevant tests, and if so, ship it. > > /Magnus > > 22 nov. 2018 kl. 18:04 skrev Alexey Ivanov >: > >> >> On 21/11/2018 12:16, Magnus Ihse Bursie wrote: >>> >>> On 2018-11-20 16:49, Alexey Ivanov wrote: >>> >>>> Hi Ali, Magnus, >>>> >>>> I absolutely agree this change has to be reviewed by someone from serviceability. >>>> >>>> There are several options: >>>> >>>> 1. Add -export:jdwpTransport_OnLoad to LDFLAGS for Windows >>>> http://mail.openjdk.java.net/pipermail/build-dev/2018-November/023935.html >>>> so it partially reverts the changes from >>>> https://bugs.openjdk.java.net/browse/JDK-8200178 >>>> >>>> 2. Remove JNICALL (__stdcall) modifier, eventually changing the calling convention to __cdecl. >>>> http://mail.openjdk.java.net/pipermail/build-dev/2018-November/023969.html >>>> >>>> 3. Use inline /export option via #pragma: >>>> #pragma comment(linker, "/export:jdwpTransport_OnLoad= _jdwpTransport_OnLoad at 16") >>>> as referenced in >>>> https://docs.microsoft.com/en-ie/cpp/cpp/dllexport-dllimport?view=vs-2017 >>>> https://docs.microsoft.com/en-ie/cpp/build/reference/exports?view=vs-2017 >>>> >>>> The third options still needs to be tested to make sure it does not break other platforms builds. >>> I'm not fond of either solution 1 or 3. I really think the signature should be made correctly at the point of definition in the source code. >> >> That is why I proposed removing JNICALL (__stdcall) from the function declaration as we had done for libzip, libjimage [1] and mlib_image [2]. >> >> Microsoft recommends using __stdcall for DLLs, at the same time it decorates the function name making it harder to export it by its plain name. >> >> >> By removing JNICALL / __stdcall, we make the function use __cdecl calling convention. The difference between them is that with __stdcall the callee cleans the stack whereas with __cdecl the caller cleans the stack. >> >> It shouldn't be a problem as long as all function declarations use the same calling convention, which, I believe, is the case here. >> >>> We've spent some considerable effort of getting rid of the /export flags for Windows (and map files for unix), and I don't want to see them creep back in. Note that option 3 is just option 1 in disguise. The only single good thing about it is that it allows you to put the compiler option next to the signature declaration in the source code. >> >> At least in this case, the option will be near the function body definition. It will be easier to update it if the signature changes. >> >> If we are to use __stdcall, it seems to be the only way to export the undecorated name. >> >>> The same goes for def files, as suggested by Ali. It's just yet another version of option 1 in disguise/map files. >> >> If option 2 is rejected, I'm for using option 3. If option 3 cannot be used too, I'm for option 1. >> I think we should not fall back to def files in any case. And Makefile will have to include the reference to the def file, so it's even worse than option 1. >> >> >> Regards, >> Alexey >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8201226 >> [2] https://bugs.openjdk.java.net/browse/JDK-8202476 >>> >>> /Magnus >>> >>>> >>>> >>>> Regards, >>>> Alexey >>>> >>>> On 19/11/2018 12:19, Magnus Ihse Bursie wrote: >>>>> Hi Ali, >>>>> >>>>> From a build perspective this change looks OK. I'm not aware of the finer details on the OnLoad mechanism, like what calling convention is to be used. So maybe this is a no-go from that view. >>>>> >>>>> I'm cc:ing servicability so they can have a look at it. >>>>> >>>>> /Magnus >>>>> >>>>> On 2018-11-18 13:07, Ali ?nce wrote: >>>>>> Hi Alexey, >>>>>> >>>>>> Below is a new patch content that removes JNICALL modifiers from the exported functions of interest. This results in exported functions not to be name decorated and use __cdecl calling convention. >>>>>> >>>>>> Do you have any more suggestions, or would you please guide me on next steps? >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Ali >>>>>> >>>>>> # HG changeset patch >>>>>> # User Ali Ince >>>>>> # Date 1542542415 0 >>>>>> # Sun Nov 18 12:00:15 2018 +0000 >>>>>> # Node ID a41df44e13e1b761f4b3f05a17ed18953067ae39 >>>>>> # Parent 16d5ec7dbbb49ef3f969e34be870e3f917ea89ff >>>>>> Remove JNICALL modifiers to prevent name decoration on 32-bit windows builds >>>>>> >>>>>> diff -r 16d5ec7dbbb4 -r a41df44e13e1 src/jdk.jdi/share/native/libdt_shmem/shmemBack.c >>>>>> --- a/src/jdk.jdi/share/native/libdt_shmem/shmemBack.c Tue Aug 14 14:28:23 2018 +0200 >>>>>> +++ b/src/jdk.jdi/share/native/libdt_shmem/shmemBack.c Sun Nov 18 12:00:15 2018 +0000 >>>>>> @@ -339,7 +339,7 @@ >>>>>> return JDWPTRANSPORT_ERROR_NONE; >>>>>> } >>>>>> >>>>>> -JNIEXPORT jint JNICALL >>>>>> +JNIEXPORT jint >>>>>> jdwpTransport_OnLoad(JavaVM *vm, jdwpTransportCallback* cbTablePtr, >>>>>> jint version, jdwpTransportEnv** result) >>>>>> { >>>>>> diff -r 16d5ec7dbbb4 -r a41df44e13e1 src/jdk.jdwp.agent/share/native/include/jdwpTransport.h >>>>>> --- a/src/jdk.jdwp.agent/share/native/include/jdwpTransport.h Tue Aug 14 14:28:23 2018 +0200 >>>>>> +++ b/src/jdk.jdwp.agent/share/native/include/jdwpTransport.h Sun Nov 18 12:00:15 2018 +0000 >>>>>> @@ -140,7 +140,7 @@ >>>>>> void (*free)(void *buffer); /* Call this for all deallocations */ >>>>>> } jdwpTransportCallback; >>>>>> >>>>>> -typedef jint (JNICALL *jdwpTransport_OnLoad_t)(JavaVM *jvm, >>>>>> +typedef jint (*jdwpTransport_OnLoad_t)(JavaVM *jvm, >>>>>> jdwpTransportCallback *callback, >>>>>> jint version, >>>>>> jdwpTransportEnv** env); >>>>>> diff -r 16d5ec7dbbb4 -r a41df44e13e1 src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c >>>>>> --- a/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c Tue Aug 14 14:28:23 2018 +0200 >>>>>> +++ b/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c Sun Nov 18 12:00:15 2018 +0000 >>>>>> @@ -1019,7 +1019,7 @@ >>>>>> return JDWPTRANSPORT_ERROR_NONE; >>>>>> } >>>>>> >>>>>> -JNIEXPORT jint JNICALL >>>>>> +JNIEXPORT jint >>>>>> jdwpTransport_OnLoad(JavaVM *vm, jdwpTransportCallback* cbTablePtr, >>>>>> jint version, jdwpTransportEnv** env) >>>>>> { >>>>>> >>>>>> >>>>>> >>>>>>> On 16 Nov 2018, at 23:03, Alexey Ivanov wrote: >>>>>>> >>>>>>> Hi Ali, >>>>>>> >>>>>>> It's exactly what I referred to. >>>>>>> >>>>>>> Yes, it does change the calling convention. >>>>>>> Yet it's not a (big) problem because both parties will use the same calling convention. >>>>>>> >>>>>>> Using a DLL from another build will not be possible. But it's not a good idea to do so anyway. >>>>>>> >>>>>>> >>>>>>> Mapfiles and export options have been removed by https://bugs.openjdk.java.net/browse/JDK-8200178 to simplify managing exported functions. So that to add or remove an exported function, you don't have modify makefiles and/or mapfiles. You just edit the source files. >>>>>>> >>>>>>> Regards, >>>>>>> Alexey >>>>>>> >>>>>>> On 16/11/2018 22:42, Ali ?nce wrote: >>>>>>>> Hi Alexey, >>>>>>>> >>>>>>>> Thanks for your reply. >>>>>>>> >>>>>>>> I will definitely give it a try though I?m not sure if that?ll be a breaking change. This is because JNICALL modifier is defined to be __stdcall on windows which is both specified on jdwpTransport_OnLoad exported functions both for dt_socket.dll and dt_shmem.dll. >>>>>>>> >>>>>>>> The __stdcall is ignored on x64 platforms and also name decoration is not applied (https://docs.microsoft.com/en-gb/cpp/build/reference/decorated-names?view=vs-2017 ). I believe that?s why we don?t experience any problems on x64 builds. >>>>>>>> >>>>>>>> Removing JNICALL (thus __stdcall) modifier (apparently only applies to x86 builds) will result in the calling convention to be changed, and thus will change how parameters are ordered and also the stack cleanup rules. I think this may result in problems in programs that are designed to load shared libraries and its exported functions at runtime (not bound by link time which probably won?t cause any issues - assuming that they use the shared function signature) - as in https://github.com/AdoptOpenJDK/openjdk-jdk11u/blob/5f01925b80ed851b133ee26fbcb07026ac04149e/src/jdk.jdwp.agent/share/native/libjdwp/transport.c#L99 . >>>>>>>> >>>>>>>> Any thoughts? >>>>>>>> >>>>>>>> Regards, >>>>>>>> >>>>>>>> Ali >>>>>>>> >>>>>>>>> On 15 Nov 2018, at 22:14, Alexey Ivanov wrote: >>>>>>>>> >>>>>>>>> Hi Ali, >>>>>>>>> >>>>>>>>> Can the issue be resolved by using different call modifiers on the affected functions? >>>>>>>>> >>>>>>>>> Some build / link issues were resolved by adding / removing JNICALL modifier, see >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8201226 >>>>>>>>> http://mail.openjdk.java.net/pipermail/build-dev/2018-April/021553.html >>>>>>>>> >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8202476 >>>>>>>>> http://mail.openjdk.java.net/pipermail/build-dev/2018-May/021913.html >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Alexey >>>>>>>>> >>>>>>>>> On 15/11/2018 21:43, Ali ?nce wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> An issue (https://github.com/AdoptOpenJDK/openjdk-build/issues/709 ) raised against AdoptOpenJDK jdk11 32-bit windows builds led us to the name decoration problem on built 32-bit dlls - specifically dt_socket.dll and dt_shmem.dll. Whereas 64-bit MSVC builds don?t apply any name decorations on exported functions, 32-bit builds apply them - which requires either def files or /export flags to be specified on the linker arguments. >>>>>>>>>> >>>>>>>>>> Although the expected linker arguments were present on previous jdk makefiles, they were removed in jdk11 makefiles. >>>>>>>>>> >>>>>>>>>> Below is the proposed patch, which can also be browsed at https://github.com/AdoptOpenJDK/openjdk-jdk11u/pull/1 . >>>>>>>>>> >>>>>>>>>> Would you please have a look and let me know of any points I may have missed (I don?t have any background information about why the export flags were removed in jdk11)? >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> >>>>>>>>>> Ali >>>>>>>>>> >>>>>>>>>> diff --git a/make/lib/Lib-jdk.jdi.gmk b/make/lib/Lib-jdk.jdi.gmk >>>>>>>>>> index 197b95c2e2..ac6ebf5591 100644 >>>>>>>>>> --- a/make/lib/Lib-jdk.jdi.gmk >>>>>>>>>> +++ b/make/lib/Lib-jdk.jdi.gmk >>>>>>>>>> @@ -37,6 +37,7 @@ ifeq ($(OPENJDK_TARGET_OS), windows) >>>>>>>>>> jdk.jdwp.agent:include \ >>>>>>>>>> jdk.jdwp.agent:libjdwp/export, \ >>>>>>>>>> LDFLAGS := $(LDFLAGS_JDKLIB), \ >>>>>>>>>> + LDFLAGS_windows := -export:jdwpTransport_OnLoad, \ >>>>>>>>>> LIBS := $(JDKLIB_LIBS), \ >>>>>>>>>> )) >>>>>>>>>> >>>>>>>>>> diff --git a/make/lib/Lib-jdk.jdwp.agent.gmk b/make/lib/Lib-jdk.jdwp.agent.gmk >>>>>>>>>> index 0bc93e0d35..0382e55325 100644 >>>>>>>>>> --- a/make/lib/Lib-jdk.jdwp.agent.gmk >>>>>>>>>> +++ b/make/lib/Lib-jdk.jdwp.agent.gmk >>>>>>>>>> @@ -37,6 +37,7 @@ $(eval $(call SetupJdkLibrary, BUILD_LIBDT_SOCKET, \ >>>>>>>>>> libjdwp/export, \ >>>>>>>>>> LDFLAGS := $(LDFLAGS_JDKLIB) \ >>>>>>>>>> $(call SET_SHARED_LIBRARY_ORIGIN), \ >>>>>>>>>> + LDFLAGS_windows := -export:jdwpTransport_OnLoad, \ >>>>>>>>>> LIBS_linux := -lpthread, \ >>>>>>>>>> LIBS_solaris := -lnsl -lsocket, \ >>>>>>>>>> LIBS_windows := $(JDKLIB_LIBS) ws2_32.lib, \ >>>>> >>>> >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Fri Dec 7 00:52:31 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 7 Dec 2018 10:52:31 +1000 Subject: RFR: JDK-8210106: sun/tools/jps/TestJps.java timed out In-Reply-To: <5C0923D1.8080705@oracle.com> References: <5C0923D1.8080705@oracle.com> Message-ID: <6697b4a2-b0d7-8a52-7549-23d6433114cc@oracle.com> Hi Gary, On 6/12/2018 11:27 pm, Gary Adams wrote: > On a local linux-x64-debug build this test consistently runs in less > than 40 seconds. > On the mach5 testing machines there was a large fluctuation in the time > to complete this test. > > Since the test runs jps with different combinations of arguments, the total > test time is dependent on the number of processes running java programs. > Since the mach5 test infrastructure runs java programs I have seen a 3X > in the amount of output the test produces compared to local test > runs. > > I've run the test several hundred times through mach5 on the slower > platforms > and then examined the test logs using a 3X setting from the default 120 > second > jtreg timeout. The slowest reported elapse time from the test logs showed > 280 seconds to complete. > > To improve the reliability of the test to complete, I'd like to increase > the > timeout to 360 seconds. > > ? Issue: https://bugs.openjdk.java.net/browse/JDK-8210106 > > Proposed fix: > > diff --git a/test/jdk/sun/tools/jps/TestJps.java > b/test/jdk/sun/tools/jps/TestJps.java > --- a/test/jdk/sun/tools/jps/TestJps.java > +++ b/test/jdk/sun/tools/jps/TestJps.java > @@ -27,7 +27,7 @@ > ? * @modules jdk.jartool/sun.tools.jar > ? * @build LingeredAppForJps > ? * @build LingeredApp > - * @run main/othervm TestJps > + * @run main/othervm/timeout=360 TestJps > ? */ Doesn't that then get scaled by the timeout factor resulting in a much longer timeout than the 360 seconds you intended? For other timeout adjustments the needed time has been divided by the timeout factor to get the actual intended timeout. Cheers, David From patrick at reini.net Fri Dec 7 08:36:33 2018 From: patrick at reini.net (Patrick Reinhart) Date: Fri, 07 Dec 2018 09:36:33 +0100 Subject: JDK-8171311 Current state In-Reply-To: <322750d5-478c-5e4a-dedb-56d3c4ae1e1b@oracle.com> References: <09715124f23fca815e2a27f7fbd2558b@reini.net> <322750d5-478c-5e4a-dedb-56d3c4ae1e1b@oracle.com> Message-ID: <8ab1c51aa8756672117dd97030d29f45@reini.net> It's a bit disturbing that just at the time of my question this JEP has been closed (without any further comment why) I think that it still would be worth while looking into supporting a REST based implementation in favour of the existing RMI based solution just by the fact of the troubles just one can have with firewalls. When doing a quick search for JMX REST adapters, I did not found that many: - Jolokia (seems to be active) [1] - Apache ESME - JMX REST API (inactive) [2] - MX4J (inactive) [3] - OpenDMK (not found anymore) -Patrick > I'm not aware of any current activity on this in OpenJDK. One thing > about the JEP is that it didn't make the case clear as to why the > adapter needed to be in the JDK. There are also several existing JMX > adapters that support REST and it would have been useful to evaluate > those and maybe explore what the pain points and issues are with > deploying these solutions. It might be that the -Dcom.sun.management > mechanism to start the JMX agent needs to be improved, maybe it should > be integrated with -javaagent, maybe the pluggability of the JMX agent > just needs to be improved. > > -Alan [1] https://jolokia.org/features-nb.html [2] https://esme.apache.org/docs/apis/jmx-rest-api.html [3] http://mx4j.sourceforge.net From christoph.langer at sap.com Fri Dec 7 08:41:09 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 7 Dec 2018 08:41:09 +0000 Subject: RFR (M) 8214892: Delayed starting of debugging via jcmd In-Reply-To: <6283122f12f041caa56f768f8005920b@sap.com> References: <01fdfc8e58a246c18d5909fc7df313b7@sap.com> <940d8a26-d5b3-0d5b-ccb8-a71d4d17efdc@oracle.com> <6283122f12f041caa56f768f8005920b@sap.com> Message-ID: Hi Ralf, thanks for doing this change. Overall, it looks very good to me and seems to be a nice feature. I have thought about a more generic name for the option rather than "onjcmd". What about "standby=y". That would indicate that the debugging agent is on standby and can be triggered by whatever means. What do others think? Here are some minor comments, mostly about the wording of text messages: src/hotspot/share/services/diagnosticCommand.cpp, line 1097: I'd prefer if you write out either "Debugging has been started." or "Debugging is already active.". I think this would make it more clear for the user of the feature what actually happened. src/hotspot/share/services/diagnosticCommand.hpp, line 878: "Starts up the Java debugging if enabled in the jdwp agentlib via the onjcmd=y option." A better wording would be: "Starts up the Java debugging if the jdwp agentlib was enabled with the option onjcmd=y." src/jdk.jdwp.agent/share/native/libjdwp/debugInit.c: line 876: replace "debug triggered by jcmd?" with " start debug via jcmd" (or "start debug on request" if we opt to change the option name from onjcmd to something more general) line 1300: suspendOnInit = JNI_FALSE; Is it really necessary/desired to set suspendOnInit to false? We could still honor suspend=y|n. The suspension will of course happen at the time debug activation is triggered. Another question: Do we need a CSR for this? Best regards Christoph > -----Original Message----- > From: serviceability-dev On > Behalf Of Schmelter, Ralf > Sent: Donnerstag, 6. Dezember 2018 15:54 > To: Chris Plummer ; serviceability- > dev at openjdk.java.net > Subject: [CAUTION] RE: RFR (M) 8214892: Delayed starting of debugging via > jcmd > > Hi Chris, > > > I think I prefer passing EI_VM_INIT for triggering_ei, but if you prefer > > to keep it as is, I suggest a comment to clarify why it might be out of > > range. > > Actually, I used EI_VM_INIT for the longest time and only changed it > recently, because I thought that code could assume that e.g. no classes have > been loaded yet when getting the INIT_VM event. But since the JVMTI spec > does not guarantees this in any way (it allows other events to be send before > a VM_INIT), I just will change it back to use EI_VM_INIT for the initialize call. > > Regarding the name of the option, I agree that onjcmd, while not technically > fully accurate, makes most sense for the common user. > > > ... It think you could just return the error right away and remove > > the error checking code that comes later. > > I've changed the code to just return the error directly. > > > It's not clear to me why you want the name and address of the first > > transport. Why is the first one necessarily the one you want? > > Since currently the bag of transports must always have a size of 1, getting the > first or the last transport is the same. But the callback function used to iterate > the bag has to return a boolean value. I've decided to return JNI_FALSE, > which would mean the iteration stops at the first entry. > > > I've updated the webrev with your and Goetz Lindenmeier's suggestions: > http://cr.openjdk.java.net/~rrich/webrevs/schmelter/8214892/webrev.02/ > > Best regards, > Ralf From Alan.Bateman at oracle.com Fri Dec 7 09:22:20 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 7 Dec 2018 09:22:20 +0000 Subject: JDK-8171311 Current state In-Reply-To: <8ab1c51aa8756672117dd97030d29f45@reini.net> References: <09715124f23fca815e2a27f7fbd2558b@reini.net> <322750d5-478c-5e4a-dedb-56d3c4ae1e1b@oracle.com> <8ab1c51aa8756672117dd97030d29f45@reini.net> Message-ID: On 07/12/2018 08:36, Patrick Reinhart wrote: > It's a bit disturbing that just at the time of my question this JEP > has been closed (without any further comment why) I suspect your inquiry prompted Raghavan to close it as there isn't (to my knowledge anyway) anyone actively working on it. I agree a comment is needed when closing issues. > > I think that it still would be worth while looking into supporting > a REST based implementation in favour of the existing RMI based > solution just by the fact of the troubles just one can have with > firewalls. Right, and I think there is some interest. In addition to the REST adapters that you found then I think some of the app servers have support too. The big question for features like this is whether it is something that the JDK? has to include or not (the batteries included vs. batteries available discussion).? If you look at Harsha's prototype (linked from the JEP) then you'll you see it can be mostly developed in its own project, the only JDK piece is integrating it with the JMX agent and existing -Dcom.sun.management options for starting the JMX agent. I think this is an area that could be improved to make it easier to deploy JMX adapters that aren't in the JDK. -Alan From raghavan.puranam at oracle.com Fri Dec 7 09:59:55 2018 From: raghavan.puranam at oracle.com (Raghavan Puranam) Date: Fri, 7 Dec 2018 01:59:55 -0800 (PST) Subject: JDK-8171311 Current state In-Reply-To: References: <09715124f23fca815e2a27f7fbd2558b@reini.net> <322750d5-478c-5e4a-dedb-56d3c4ae1e1b@oracle.com> <8ab1c51aa8756672117dd97030d29f45@reini.net> Message-ID: My apologies Patrick...I should have added the comment first before closing. I have added it now. Regards, Raga -----Original Message----- From: Alan Bateman Sent: Friday, December 7, 2018 2:52 PM To: Patrick Reinhart Cc: serviceability-dev at openjdk.java.net Subject: Re: JDK-8171311 Current state On 07/12/2018 08:36, Patrick Reinhart wrote: > It's a bit disturbing that just at the time of my question this JEP > has been closed (without any further comment why) I suspect your inquiry prompted Raghavan to close it as there isn't (to my knowledge anyway) anyone actively working on it. I agree a comment is needed when closing issues. > > I think that it still would be worth while looking into supporting > a REST based implementation in favour of the existing RMI based > solution just by the fact of the troubles just one can have with > firewalls. Right, and I think there is some interest. In addition to the REST adapters that you found then I think some of the app servers have support too. The big question for features like this is whether it is something that the JDK? has to include or not (the batteries included vs. batteries available discussion).? If you look at Harsha's prototype (linked from the JEP) then you'll you see it can be mostly developed in its own project, the only JDK piece is integrating it with the JMX agent and existing -Dcom.sun.management options for starting the JMX agent. I think this is an area that could be improved to make it easier to deploy JMX adapters that aren't in the JDK. -Alan From shade at redhat.com Fri Dec 7 10:26:26 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 7 Dec 2018 11:26:26 +0100 Subject: RFR (round 5), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <8c267450-9ca8-79a6-0e4a-39b3b68af2e3@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <8c267450-9ca8-79a6-0e4a-39b3b68af2e3@redhat.com> Message-ID: <9fca6745-8c43-a15b-a967-2ce1be84e0be@redhat.com> On 12/4/18 8:10 AM, Roman Kennke wrote: > Webrevs: > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/ > > [ ] shenandoah-gc > [ ] shenandoah-tests These two parts look good. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From gary.adams at oracle.com Fri Dec 7 12:58:53 2018 From: gary.adams at oracle.com (Gary Adams) Date: Fri, 07 Dec 2018 07:58:53 -0500 Subject: RFR: JDK-8210106: sun/tools/jps/TestJps.java timed out In-Reply-To: <6697b4a2-b0d7-8a52-7549-23d6433114cc@oracle.com> References: <5C0923D1.8080705@oracle.com> <6697b4a2-b0d7-8a52-7549-23d6433114cc@oracle.com> Message-ID: <5C0A6E8D.7010108@oracle.com> On 12/6/18, 7:52 PM, David Holmes wrote: > Hi Gary, > > On 6/12/2018 11:27 pm, Gary Adams wrote: >> On a local linux-x64-debug build this test consistently runs in less >> than 40 seconds. >> On the mach5 testing machines there was a large fluctuation in the >> time to complete this test. >> >> Since the test runs jps with different combinations of arguments, the >> total >> test time is dependent on the number of processes running java programs. >> Since the mach5 test infrastructure runs java programs I have seen a 3X >> in the amount of output the test produces compared to local test >> runs. >> >> I've run the test several hundred times through mach5 on the slower >> platforms >> and then examined the test logs using a 3X setting from the default >> 120 second >> jtreg timeout. The slowest reported elapse time from the test logs >> showed >> 280 seconds to complete. >> >> To improve the reliability of the test to complete, I'd like to >> increase the >> timeout to 360 seconds. >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-8210106 >> >> Proposed fix: >> >> diff --git a/test/jdk/sun/tools/jps/TestJps.java >> b/test/jdk/sun/tools/jps/TestJps.java >> --- a/test/jdk/sun/tools/jps/TestJps.java >> +++ b/test/jdk/sun/tools/jps/TestJps.java >> @@ -27,7 +27,7 @@ >> * @modules jdk.jartool/sun.tools.jar >> * @build LingeredAppForJps >> * @build LingeredApp >> - * @run main/othervm TestJps >> + * @run main/othervm/timeout=360 TestJps >> */ > > Doesn't that then get scaled by the timeout factor resulting in a much > longer timeout than the 360 seconds you intended? > > For other timeout adjustments the needed time has been divided by the > timeout factor to get the actual intended timeout. This bug was filed fairly recently in Aug 2018. At that time the timeout and timeout factor were not sufficient to avoid the test failing. The mach5 timeout factors were adjusted recently, so this test may no longer be an issue. If that is true, then we could simply close this bug as "cannot reproduce". An argument could be made that the change in timeout factor may be responsible for fixing a lot more of the intermittent bugs and that they should be closed in a similar manner. Historically, we could say this particular bug should have had timeouts reassessed when the infrastructure switched from jprt to mach5 testing where there were more visible Java processes running. Using a higher explicit timeout will not make the test run any longer than it needs. It will simply allow the test to not be terminated sooner in a hung test scenario. What is your preference for this particular issue: - increase the explicit timeout - close as cannot reproduce attributed to the timeout factor adjustments What would you recommend going forward for other similar issues: - determine a new explicit timeout - close if no timeout failures have been observed since the timeout factor was raised > > Cheers, > David > From rkennke at redhat.com Fri Dec 7 13:00:51 2018 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 7 Dec 2018 14:00:51 +0100 Subject: RFR (round 6), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <8c267450-9ca8-79a6-0e4a-39b3b68af2e3@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <8c267450-9ca8-79a6-0e4a-39b3b68af2e3@redhat.com> Message-ID: Here comes round 6, possibly/hopefully the last round of webrevs for upstreaming Shenandoah: http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/06/ It incorporates: - renames vm_operations_shenandoah* to shenandoahVMOperations*, as suggested by Coleen - reshuffles gcCause in shared-gc SA as suggested by Per - print number of threads in SA, as suggested by Jini - fixed a problem in webrev generation that did not track move of CriticalNativeStress.java and CriticalNativeArgs.java The CSR has been approved, the JEP moved to target jdk12, and I got positive reviews for all parts. I intend to push this early next week, unless somebody stops me. If Zhengyu's and Thomas' TaskQueue change goes in by then, I'll incorporate it. Thanks everybody for reviewing and reviewing and reviewing again ;-) Cheers, Roman > Round 5 of Shenandoah review includes: > - A fix for the @requires tag in TestFullGCCountTest.java. It should be > correct now. We believe the CMS @requires was also not quite right and > fixed it the same. > > It reads now: Don't run this test if: > - Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is > true, as set by harness > - Actual GC set by harness is Shenandoah *and* > ExplicitGCInvokesConcurrent is not set false by harness (it's true by > default in Shenandoah, so this needs to be double-inverteed). > > The @requires for CMS was wrong before (we think), because it would also > filter defaultGC + ExplicitGCInvokesConcurrent. > > - Sorting of macros was fixed, as was pointed out by Per > - Some stuff was added to SA, as suggested by Jini > - Rebased on most current jdk/jdk code > > Webrevs: > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/ > > I also need reviews from GC reviewers for the CSR: > https://bugs.openjdk.java.net/browse/JDK-8214349 > > I already got reviews for: > [x] shared-runtime (coleenp) > [x] shared-compiler (kvn) > > I got reviews for shared-build, but an earlier version, so maybe makes > sense to look over this again. Erik J, Magnus? > > I still need approvals for: > [ ] shared-build (kvn, erikj, ihse, pliden) > [ ] shared-gc (pliden, kbarrett) > [ ] shared-serviceability (jgeorge, pliden) > [ ] shared-tests (lmesnik, pliden) > [ ] shenandoah-gc > [ ] shenandoah-tests > > > Thanks for your patience and ongoing support! > > Cheers, > Roman > > >> Hi all, >> >> here comes round 4 of Shenandoah upstreaming review: >> >> This includes fixes for the issues that Per brought up: >> - Verify and gracefully reject dangerous flags combinations that >> disables required barriers >> - Revisited @requires filters in tests >> - Trim unused code from Shenandoah's SA impl >> - Move ShenandoahGCTracer to gc/shenandoah >> - Fix ordering of GC names in various files >> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >> >> Thanks everybody for taking time to review this! >> Roman >> >>> Hello all, >>> >>> Thanks so far for all the reviews and support! >>> >>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >>> >>> Things we've changed today: >>> - We moved shenandoah-specific code out of .ad files into our own .ad >>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>> requires an addition in build machinery though, see >>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>> - Improved zero-disabling and build-code-simplification as suggested by >>> Magnus and Per >>> - Cleaned up some leftovers in C2 >>> - Improved C2 loop opts code by introducing another APIs in >>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now. >>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>> that. >>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>> - Rebased on jdk-12+22 >>> >>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>> >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>> >>> Thanks, >>> Roman >>> >>>> Alright, we fixed: >>>> - The minor issues that Kim reported in shared-gc >>>> - A lot of fixes in shared-tests according to Leonid's review >>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>> >>>> Some notes: >>>> Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>> correct. The @requires there means to exclude runs with both CMS and >>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>> fine. Adding the same clause for Shenandoah means the same, and filters >>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>> made the condition a bit clearer by avoiding triple-negation. >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>> >>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>> >>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>> those with ZGC? >>>> >>>> See: >>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>> >>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>> next round). >>>> >>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>> better. I can tell that we're not done with C2 yet. Can you look over >>>> the code and see what is ok, and especially what is not ok, so that we >>>> can focus our efforts on the relevant parts? >>>> >>>> Updated set of webrevs: >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>> >>>> Thanks, >>>> Roman >>>> >>>> >>>>> Hi, >>>>> >>>>> This is the first round of changes for including Shenandoah GC into >>>>> mainline. >>>>> I divided the review into parts that roughly correspond to the mailing lists >>>>> that would normally review it, and I divided it into 'shared' code >>>>> changes and >>>>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>>>> eventually >>>>> push them as single 'combined' changeset, once reviewed. >>>>> >>>>> JEP: >>>>> ? https://openjdk.java.net/jeps/189 >>>>> Bug entry: >>>>> >>>>> ?https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>> >>>>> Webrevs: >>>>> ? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>> >>>>> For those who want to see the full change, have a look at the >>>>> shenandoah-complete >>>>> >>>>> directory, >>>>> it contains the full combined webrev. Alternatively, there is the file >>>>> shenandoah-master.patch >>>>> , >>>>> which is what I intend to commit (and which should be equivalent to the >>>>> 'shenandoah-complete' webrev). >>>>> >>>>> Sections to review (at this point) are the following: >>>>> ?*) shenandoah-gc >>>>> >>>>> ??? - Actual Shenandoah implementation, almost completely residing in >>>>> gc/shenandoah >>>>> >>>>> ?*) shared-gc >>>>> >>>>> ??? - This is mostly boilerplate that is common to any GC >>>>> ??? - referenceProcessor.cpp has a little change to make one assert not >>>>> fail (next to CMS and G1) >>>>> ??? - taskqueue.hpp has some small adjustments to enable subclassing >>>>> >>>>> ?*) shared-serviceability >>>>> >>>>> ??? - The usual code to support another GC >>>>> >>>>> ?*) shared-runtime >>>>> >>>>> ??? - A number of friends declarations to allow Shenandoah iterators to >>>>> hook up with, >>>>> ????? e.g. ClassLoaderData, CodeCache, etc >>>>> ??? - Warning and disabling JFR LeakProfiler >>>>> ??? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>> Shenandoah C2 optimizations >>>>> ??? - Locks initialization in mutexLocker.cpp as usual >>>>> ??? - VM operations defines for Shenandoah's VM ops >>>>> ??? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>> Shenandoah's logging >>>>> ??? - The usual macros in macro.hpp >>>>> >>>>> ?*) shared-build >>>>> >>>>> ??? - Add shenandoah feature, enabled by default, as agreed with >>>>> Vladimir K. beforehand >>>>> ??? - Some flags for shenandoah-enabled compilation to get >>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>> ????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>> Shenandoah's barriers >>>>> ??? - --param inline-unit-growth=1000 settings for 2 shenandoah source >>>>> files, which is >>>>> ????? useful to get the whole marking loop inlined (observed significant >>>>> regression if we >>>>> ????? don't) >>>>> >>>>> ?*) shared-tests >>>>> >>>>> ??? - Test infrastructure to support Shenandoah >>>>> ??? - Shenandoah test groups >>>>> ??? - Exclude Shenandoah in various tests that can be run with selected GC >>>>> ??? - Enable/add configure for Shenandoah for tests that make sense to >>>>> run with it >>>>> >>>>> ?*) shenandoah-tests >>>>> >>>>> ??? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >>>>> ??? - A couple of tests configurations have been added, e.g. >>>>> TestGCBasherWithShenandoah.java >>>>> >>>>> I intentionally left out shared-compiler for now, because we have some >>>>> work left to do >>>>> there, but if you click around you'll find the patch anyway, in case you >>>>> want to take >>>>> a peek at it. >>>>> >>>>> We have regular builds on: >>>>> ? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>> ? - {Windows} x {x86_64}, >>>>> ? - {MacOS X} x {x86_64} >>>>> >>>>> This also routinely passes: >>>>> ? - the new Shenandoah tests >>>>> ? - jcstress with/without aggressive Shenandoah verification >>>>> ? - specjvm2008 with/without aggressive Shenandoah verification >>>>> >>>>> >>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>> teams for tirelessly helping with and reviewing all the GC interface and >>>>> related changes, and of course the many early adopters for reporting >>>>> bugs and success stories and feature requests: we wouldn't be here >>>>> without any of you! >>>>> >>>>> Best regards, >>>>> Roman >>>>> >>>> >>> >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From patrick at reini.net Fri Dec 7 14:11:52 2018 From: patrick at reini.net (Patrick Reinhart) Date: Fri, 07 Dec 2018 15:11:52 +0100 Subject: JDK-8171311 Current state In-Reply-To: References: <09715124f23fca815e2a27f7fbd2558b@reini.net> <322750d5-478c-5e4a-dedb-56d3c4ae1e1b@oracle.com> <8ab1c51aa8756672117dd97030d29f45@reini.net> Message-ID: <653a5ea4a8a19d059fc1b33863cb40aa@reini.net> Thanks, I just found that JEP searching for an simple way to attach to a non application server VM avoiding the hassle for setting up Firewall Rules for RMI and that JEP was the first in the list followed by the Jolokia that seems not jet ready for Java 11... I will look into the Jolokia library and will try to find out, what the exact issues with Java 11 are. Besides that it would really make sense to see if there would be a better way for starting the JMX services as Alan pointed out. -Patrick On 2018-12-07 10:59, Raghavan Puranam wrote: > My apologies Patrick...I should have added the comment first before > closing. I have added it now. > > Regards, > Raga > > -----Original Message----- > From: Alan Bateman > Sent: Friday, December 7, 2018 2:52 PM > To: Patrick Reinhart > Cc: serviceability-dev at openjdk.java.net > Subject: Re: JDK-8171311 Current state > > On 07/12/2018 08:36, Patrick Reinhart wrote: >> It's a bit disturbing that just at the time of my question this JEP >> has been closed (without any further comment why) > I suspect your inquiry prompted Raghavan to close it as there isn't (to > my knowledge anyway) anyone actively working on it. I agree a comment > is > needed when closing issues. > >> >> I think that it still would be worth while looking into supporting >> a REST based implementation in favour of the existing RMI based >> solution just by the fact of the troubles just one can have with >> firewalls. > Right, and I think there is some interest. In addition to the REST > adapters that you found then I think some of the app servers have > support too. The big question for features like this is whether it is > something that the JDK? has to include or not (the batteries included > vs. batteries available discussion).? If you look at Harsha's prototype > (linked from the JEP) then you'll you see it can be mostly developed in > its own project, the only JDK piece is integrating it with the JMX > agent > and existing -Dcom.sun.management options for starting the JMX agent. I > think this is an area that could be improved to make it easier to > deploy > JMX adapters that aren't in the JDK. > > -Alan From daniel.daugherty at oracle.com Fri Dec 7 15:12:37 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 7 Dec 2018 10:12:37 -0500 Subject: RFR: JDK-8210106: sun/tools/jps/TestJps.java timed out In-Reply-To: <5C0A6E8D.7010108@oracle.com> References: <5C0923D1.8080705@oracle.com> <6697b4a2-b0d7-8a52-7549-23d6433114cc@oracle.com> <5C0A6E8D.7010108@oracle.com> Message-ID: On 12/7/18 7:58 AM, Gary Adams wrote: > On 12/6/18, 7:52 PM, David Holmes wrote: >> Hi Gary, >> >> On 6/12/2018 11:27 pm, Gary Adams wrote: >>> On a local linux-x64-debug build this test consistently runs in less >>> than 40 seconds. >>> On the mach5 testing machines there was a large fluctuation in the >>> time to complete this test. >>> >>> Since the test runs jps with different combinations of arguments, >>> the total >>> test time is dependent on the number of processes running java >>> programs. >>> Since the mach5 test infrastructure runs java programs I have seen a 3X >>> in the amount of output the test produces compared to local test >>> runs. >>> >>> I've run the test several hundred times through mach5 on the slower >>> platforms >>> and then examined the test logs using a 3X setting from the default >>> 120 second >>> jtreg timeout. The slowest reported elapse time from the test logs >>> showed >>> 280 seconds to complete. >>> >>> To improve the reliability of the test to complete, I'd like to >>> increase the >>> timeout to 360 seconds. >>> >>> ?? Issue: https://bugs.openjdk.java.net/browse/JDK-8210106 >>> >>> Proposed fix: >>> >>> diff --git a/test/jdk/sun/tools/jps/TestJps.java >>> b/test/jdk/sun/tools/jps/TestJps.java >>> --- a/test/jdk/sun/tools/jps/TestJps.java >>> +++ b/test/jdk/sun/tools/jps/TestJps.java >>> @@ -27,7 +27,7 @@ >>> ?? * @modules jdk.jartool/sun.tools.jar >>> ?? * @build LingeredAppForJps >>> ?? * @build LingeredApp >>> - * @run main/othervm TestJps >>> + * @run main/othervm/timeout=360 TestJps >>> ?? */ >> >> Doesn't that then get scaled by the timeout factor resulting in a >> much longer timeout than the 360 seconds you intended? >> >> For other timeout adjustments the needed time has been divided by the >> timeout factor to get the actual intended timeout. > > This bug was filed fairly recently in Aug 2018. > At that time the timeout and timeout factor were not sufficient > to avoid the test failing. > > The mach5 timeout factors were adjusted recently, so this test may > no longer be an issue. > > If that is true, then we could simply close this bug as "cannot > reproduce". > An argument could be made that the change in timeout factor may be > responsible for fixing a lot more of the intermittent bugs and that they > should be closed in a similar manner. > > Historically, we could say this particular bug should have had timeouts > reassessed when the infrastructure switched from jprt to mach5 testing > where there were more visible Java processes running. > > Using a higher explicit timeout will not make the test run any longer > than it needs. It will simply allow the test to not be terminated sooner > in a hung test scenario. > > What is your preference for this particular issue: > ?? - increase the explicit timeout > ?? - close as cannot reproduce attributed to the timeout factor > adjustments > > What would you recommend going forward for other similar issues: > ?? - determine a new explicit timeout > ?? - close if no timeout failures have been observed since the timeout > factor > ????? was raised General guidance that I've been giving folks is that you bump the default timeout when you see timeouts with 'release' bits with a timeout factor of '1'. The default timeout factor in Mach5 is 4 so we run both 'release' and 'fastdebug' bits with that timeout factor. That means our total timeout value for 'release' bits is probably a little long and our total timeout value for 'fastdebug' bits is probably just right for a concurrency factor of 12. On my personal servers I use the following timeout factors: ??????? release) ??????????? TIMEOUT_FACTOR="4" ??????????? ;; ??????? fastdebug) ??????????? TIMEOUT_FACTOR="6" ??????????? ;; ??????? slowdebug) ??????????? TIMEOUT_FACTOR="12" ??????????? ;; However, I run with a concurrency factor of 16. I rarely see timeout failures. All that is well and good, but a 'jps' test is a different beast since it is affected by factors outside of most tests. Since the number of java processes running on the system can affect the test, I'm okay with using a default timeout of '360' for this test as a guard against increases in load or... Gary saw a longest time value of 280 seconds in his testing with fastdebug bits. The default timeout value is 120 seconds with a default timeout factor of 4 in Mach5 gives a total timeout of 480 seconds (8 minutes). With Gary's new default timeout of 360 seconds, we'll have a total timeout value of 1440 seconds (24 minutes). Before the recent changes to Mach5, the default timeout factor was 10 so we had a total timeout value of 1200 seconds (20 minutes). Gary's change only bumps the total timeout value by 4 minutes from what we had back in August... when this bug was filed... unfortunately, the log for that sighting is gone so all we know is that the test timed out after 20 minutes > Timeout refired 1200 times We know that JTREG timeout handling kicks in at timeout expiration, but without the log we don't know how much longer than 20 minutes the test ran before JTREG killed it. 24 minutes might do it, but... Dan > > >> >> Cheers, >> David >> > > From erik.gahlin at oracle.com Fri Dec 7 16:44:23 2018 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Fri, 7 Dec 2018 17:44:23 +0100 Subject: JDK-8171311 Current state In-Reply-To: <653a5ea4a8a19d059fc1b33863cb40aa@reini.net> References: <09715124f23fca815e2a27f7fbd2558b@reini.net> <322750d5-478c-5e4a-dedb-56d3c4ae1e1b@oracle.com> <8ab1c51aa8756672117dd97030d29f45@reini.net> <653a5ea4a8a19d059fc1b33863cb40aa@reini.net> Message-ID: <5C0AA367.9090303@oracle.com> The reason to put this into the JDK is to standardize the protocol. If you want to build a client today, you must build one for every adapter because they have different ways to represent URLs etc. The JDK is in unique position to set the standard, since the implementation comes by default. Erik > Thanks, > > I just found that JEP searching for an simple way to attach to a non > application server VM avoiding the hassle for setting up Firewall > Rules for RMI and that JEP was the first in the list followed by the > Jolokia that seems not jet ready for Java 11... > > I will look into the Jolokia library and will try to find out, what > the exact issues with Java 11 are. > > Besides that it would really make sense to see if there would be a > better way for starting the JMX services as Alan pointed out. > > -Patrick > > > > > On 2018-12-07 10:59, Raghavan Puranam wrote: >> My apologies Patrick...I should have added the comment first before >> closing. I have added it now. >> >> Regards, >> Raga >> >> -----Original Message----- >> From: Alan Bateman >> Sent: Friday, December 7, 2018 2:52 PM >> To: Patrick Reinhart >> Cc: serviceability-dev at openjdk.java.net >> Subject: Re: JDK-8171311 Current state >> >> On 07/12/2018 08:36, Patrick Reinhart wrote: >>> It's a bit disturbing that just at the time of my question this JEP >>> has been closed (without any further comment why) >> I suspect your inquiry prompted Raghavan to close it as there isn't (to >> my knowledge anyway) anyone actively working on it. I agree a comment is >> needed when closing issues. >> >>> >>> I think that it still would be worth while looking into supporting >>> a REST based implementation in favour of the existing RMI based >>> solution just by the fact of the troubles just one can have with >>> firewalls. >> Right, and I think there is some interest. In addition to the REST >> adapters that you found then I think some of the app servers have >> support too. The big question for features like this is whether it is >> something that the JDK has to include or not (the batteries included >> vs. batteries available discussion). If you look at Harsha's prototype >> (linked from the JEP) then you'll you see it can be mostly developed in >> its own project, the only JDK piece is integrating it with the JMX agent >> and existing -Dcom.sun.management options for starting the JMX agent. I >> think this is an area that could be improved to make it easier to deploy >> JMX adapters that aren't in the JDK. >> >> -Alan From jini.george at oracle.com Fri Dec 7 19:22:41 2018 From: jini.george at oracle.com (Jini George) Date: Sat, 8 Dec 2018 00:52:41 +0530 Subject: RFR: (S): JDK-8200613: SA: jstack throws UnmappedAddressException with a CDS core file In-Reply-To: <798a4d93-1638-8785-885f-fac0b21c2417@oracle.com> References: <864997e5-2581-7ac9-44f7-76cc31c9ad93@oracle.com> <60d2e6db-c87b-9658-57cb-b03ca881a624@oracle.com> <9eb9041d-f984-0df4-a21e-4645966a62b8@oracle.com> <42d6d604-8df5-47c4-a748-6aa25bfc9a7e@oracle.com> <2dbef041-c75e-07f6-fafc-d18f2a95f455@oracle.com> <798a4d93-1638-8785-885f-fac0b21c2417@oracle.com> Message-ID: <2c6a2d67-5d36-07f1-4563-c1cd2ecdf9a3@oracle.com> I have the revised webrev here: http://cr.openjdk.java.net/~jgeorge/8200613/webrev.02/index.html The extra changes here are to: * Introduce a new option DumpPrivateMappingsInCore to control the dumping of the file backed private regions into the corefile. * Close the modules file before dumping core in os::abort(). Currently, there is a small bug (https://bugs.openjdk.java.net/browse/JDK-8215026) which prevents the closure of the image file in unmapping the entire file. I plan to take up the unmapping of NIO MapMode.PRIVATE files as a separate task (https://bugs.openjdk.java.net/browse/JDK-8215027) since this seems a bit involved. Thanks a bunch, Jini. On 11/12/2018 10:26 AM, Jini George wrote: > Thank you very much, Chris, Kevin and Ioi for your comments! > > I will send another webrev with this change enabled under an opt-out > flag, as you suggest, and would look at unmapping the JDK modules file > and if possible, the NIO mapped files too in the signal handler. > > Thanks a bunch, > Jini. > > On 11/9/2018 11:53 PM, Ioi Lam wrote: >> Hi Jini, >> >> Thanks for investigating the size expansion issue. >> >> I agree that the size increase is worth it. Even when not using SA, if >> we open the core file inside GDB, we cannot read certain sections in >> the CDS archive (such as the RO section and strings sections). That >> would make debugging difficult. So I am in favor of this change. >> >> For the JDK modules file, maybe we can unmap it in the signal handler, >> before going ahead with the core dump? I think it's hardly needed for >> debugging purposes. (Perhaps we can also do the same for the NIO >> mapped files?) >> >> A opt-flag as suggested by Kevin is a good idea. >> >> Thanks >> >> - Ioi >> >> On 11/9/18 3:29 AM, Kevin Walls wrote: >>> Hi Jini, >>> >>> Looks good to me.? It might be a significant increase in size of >>> _some_ core files, but so many core files we see are much larger, in >>> gigabytes++ of course, so the CDS data size should not be such a >>> significant increase on (I think) most files. >>> >>> The flexibiity of always having the CDS data there is very >>> significant.? A core file should ideally be usable, without >>> additionally requiring the CDS archive from the machine.? That >>> additional human round-trip upload request on every transmitted core >>> that needs investigating, seems like a less efficient route...). >>> >>> Is there an opt-out?? It's conditional on UseSharedSpaces but could >>> there be a flag to disable, in case we see crashes with gigabytes of >>> private mappings that we really don't want to retain (the user would >>> have to know to set a flag, to disable the new coredump filter ahead >>> of time). >>> >>> Thanks! >>> Kevin >>> >>> >>> On 29/10/2018 06:02, Jini George wrote: >>>> Thank you very much, Ioi, for looking into this, and the >>>> clarification offline. My bad, I had missed the earlier mail from >>>> you. :-( My responses below. >>>> >>>> Yes, I had tested this on MacOS. The issue does not exist on MacOS >>>> since the file backed private mmap()-ed regions get dumped into the >>>> MacOS corefiles by default. >>>> >>>> The corefile sizes on Linux do increase due to this change. And the >>>> increase would also include any file mapped using NIO with >>>> MapMode.PRIVATE. The typical corefile size increase with this change >>>> would include the following components at a high level: >>>> >>>> * Any NIO file mapping with MapMode.PRIVATE. >>>> * Any file mmap()-ed by any native library with MAP_PRIVATE. >>>> * The read only CDS regions (ro and od): Of the order of a few MB. >>>> * The shared strings CDS region. (typically less than 1 MB). >>>> * 2 MB per native shared library (regions with ---p permissions >>>> mapped by the dynamic linker for better alignment and for keeping >>>> libraries efficiently shareable). >>>> * The JDK 'modules' file. (About 140 MB). >>>> >>>> So, without including any NIO mapping, I typically see around >>>> 250-300 MB increase in the corefile sizes. I agree that the size >>>> increase could be a cause for concern, but for FWIW, these privately >>>> mapped files get dumped into the corefile for MacOS too. And the >>>> corefile sizes for the same program on MacOS are way larger (of the >>>> order of a few GB as against about 300 MB on Linux (without the >>>> change)). >>>> >>>> The advantage of fixing this by modifying the coredump_filter v/s >>>> doing it in SA (by reading in more sections of the shared archive >>>> file) is that this would benefit other debuggers like gdb also. (And >>>> reduces the dependence on having the shared archive file being >>>> available at the time of debugging). If folks still think this is a >>>> cause for concern, I could make modifications to fix this by reading >>>> in the regions from the shared archive file in the SA code. I also >>>> wonder if it is worth taking a relook at the mapping types of the >>>> various CDS regions also. >>>> >>>> Thank you, >>>> Jini. >>>> >>>> On 10/22/2018 10:27 AM, Ioi Lam wrote: >>>>> Hi Jini, >>>>> >>>>> Did you see my earlier reply? I might have sent it out during the >>>>> mail server outage days :-( >>>>> >>>>> http://openjdk.5641.n7.nabble.com/RFR-S-JDK-8200613-SA-jstack-throws-UnmappedAddressException-with-a-CDS-core-file-td352681.html#a353026 >>>>> >>>>> >>>>> Here was my reply again: >>>>> >>>>>> Hi Jini, >>>>>> >>>>>> The changes looks good to me. >>>>>> >>>>>> Have you tested this on MacOS? CDS heap support is also enabled on >>>>>> MacOS. See macros.hpp: >>>>>> >>>>>> ???? #if INCLUDE_CDS && INCLUDE_G1GC && defined(_LP64) && >>>>>> !defined(_WINDOWS) >>>>>> ???? #define INCLUDE_CDS_JAVA_HEAP 1 >>>>>> >>>>>> Also, besides CDS, do we know how often other files will be mmaped >>>>>> with >>>>>> MAP_PRIVATE? Will users get huge core files because CDS is >>>>>> enabled? In >>>>>> JDK 12, CDS will be enabled by default (see JDK-8202951), so all >>>>>> users >>>>>> will be affected by the following line: >>>>>> >>>>>> ?? if (UseSharedSpaces) { >>>>>> ???? set_coredump_filter(FILE_BACKED_PVT_BIT); >>>>>> ?? } >>>>>> >>>>>> Maybe you can run an big app such as Eclipse, trigger a core dump, >>>>>> and >>>>>> compare the size of the core file before/after this change? >>>>>> >>>>>> Thanks >>>>>> >>>>>> - Ioi >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> Thanks >>>>> >>>>> - Ioi >>>>> >>>>> >>>>> On 10/21/18 8:58 PM, Jini George wrote: >>>>>> Gentle reminder! >>>>>> >>>>>> Thanks, >>>>>> - Jini >>>>>> >>>>>> On 10/9/2018 11:31 AM, Jini George wrote: >>>>>>> Hello! >>>>>>> >>>>>>> [Including runtime-dev since the changes are in runtime code] >>>>>>> >>>>>>> Requesting reviews for: >>>>>>> >>>>>>> Webrev: http://cr.openjdk.java.net/~jgeorge/8200613/webrev.01/ >>>>>>> BugID: https://bugs.openjdk.java.net/browse/JDK-8200613 >>>>>>> >>>>>>> Issue: jhsdb jstack would throw an UnmappedAddressException with >>>>>>> a core file generated from a CDS enabled java process. This is >>>>>>> seen only with Linux and with G1GC, while trying to read in data >>>>>>> from the shared strings region (the closed archive heap space). >>>>>>> This region (which is a file backed private memory region) is not >>>>>>> dumped into the corefile for Linux. This, being a heap region >>>>>>> (and therefore being a read-write region) is also not read in >>>>>>> from the classes.jsa file in SA since only the read only regions >>>>>>> are read in while processing the core file. (The expectation >>>>>>> being that the read write regions are in the core file). >>>>>>> >>>>>>> Proposed solution: The proposed solution is to have the >>>>>>> coredump_filter value corresponding to the CDS process to include >>>>>>> bit 2 (file-backed private memory), so that the file-backed >>>>>>> private memory region also gets dumped into the corefile. The >>>>>>> proposed fix is in src/hotspot/os/linux/os_linux.cpp. >>>>>>> >>>>>>> Thanks, >>>>>>> Jini. >>>>>>> >>>>> >>> >> From alexey.ivanov at oracle.com Fri Dec 7 20:24:29 2018 From: alexey.ivanov at oracle.com (Alexey Ivanov) Date: Fri, 7 Dec 2018 20:24:29 +0000 Subject: [PATCH] JDK-8214122 Prevent name mangling of jdwpTransport_OnLoad in Windows 32-bit DLL name decoration In-Reply-To: <5A54BA77-B240-472A-A4B7-535C3A30A20F@gmail.com> References: <2DD922C6-EFB1-4891-90F0-3378DEA91B1C@gmail.com> <07840796-f1c4-e077-7b0b-cd5c1a880ec1@oracle.com> <00e58330-26f0-4401-f911-24fda0127bcc@oracle.com> <34d2b504-c4eb-6df3-3bd1-e23b9c22ea6a@oracle.com> <01b80c3f-8f71-26ec-304e-a7b245924bd5@oracle.com> <17e7873f-f2f4-dc02-82a6-2d7eb1565ca2@oracle.com> <5e032661-b70c-c6ec-2700-120589300c9e@oracle.com> <5A54BA77-B240-472A-A4B7-535C3A30A20F@gmail.com> Message-ID: <9d060b4d-bc2f-d402-d767-e5962df45aa6@oracle.com> Hi Ali, On 06/12/2018 22:49, Ali ?nce wrote: > Hi Magnus, Alexey, > > I believe we won?t be able to get further opinions from > serviceability-dev. Unfortunately, no one has replied as of now. Have you found any issues with jdwpTransport_OnLoad after removing JNICALL? > Andrew Luo suggested using a similar mechanism as is used for jvm.dll > by using symbol name files mapped by platform (files are under > make/hotspot/symbols and interestingly windows is almost the only > platform for which a symbol file doesn?t exist). Andrew says the .def files are auto-generated for Windows. There's a set of shared functions. JVM cannot change the calling convention, jvm.dll is the public interface to JVM. > Another issue, again opened against AdoptOpenJDK 32-bit builds is > somehow related with the same problem - but this time it is jimage.dll > depending on zip.dll expecting the ZIP_InflateFully function to be > decorated with JNICALL - whereas it was removed earlier from the > implementation and the runtime image just fails with access violation > just because jimage source code still declared it with JNICALL > (https://github.com/AdoptOpenJDK/openjdk-build/issues/763). The usage of ZIP_InflateFully from zip.dll by jimage.dll was overlooked during work on https://bugs.openjdk.java.net/browse/JDK-8201226. I can take care of it. (I could not build 32 bit Windows. It seem to have overcome the problem by adding --disable-warnings-as-errors option to configure.) However, the report says the resulting image crashes in 64 bit windows too which shouldn't be affected by JNICALL mismatch. > I?ve also searched for GetProcAddress calls across the source code - > and I think it?s important to work on the consistency of JNICALL > usages across the whole source code. There are many places where libraries are loaded dynamically or a function may be unavailable on older versions of the platform. Have you identified any other place where functions from JDK DLLs are looked up by names? > Another noteworthy thing I?ve noticed is there are still JNICALL > modifiers for example in > https://github.com/AdoptOpenJDK/openjdk-jdk11u/blob/master/src/jdk.crypto.cryptoki/windows/native/libj2pkcs11/j2secmod_md.c#L49?- > which I guess will also be reported as broken since these functions > are again name decorated. This is a JNI method. It should use JNICALL modifier. If it wasn't found, the class sun.security.pkcs11.Secmod would fail to load. I guess JVM handles name mangling somehow for native method implementation. Such functions were never explicitly exported using mapfiles or /export options on Windows, at least in the client area. For Linux and Solaris, adding or removing a native method required editing mapfiles. > If we?re still determined to remove JNICALL, what about just removing > __stdcall from JNICALL declaration at > https://github.com/AdoptOpenJDK/openjdk-jdk11u/blob/master/src/java.base/windows/native/include/jni_md.h#L31???Wouldn?t > that be a more consistent move? We can't do that. Implementation of Java native methods must use __stdcall calling convention. > > Regards, > > Ali > >> On 22 Nov 2018, at 17:29, Magnus Ihse Bursie >> > > wrote: >> >> I think we are in complete agreement. What's missing is some expert >> opinion from serviceability-dev if there is any problem with changing >> the signature. I'd preferably have that. >> >> If no one knows, I'd say, change it, and see if we still pass the >> relevant tests, and if so, ship it. >> >> /Magnus >> >> 22 nov. 2018 kl. 18:04 skrev Alexey Ivanov > >: >> >>> >>> On 21/11/2018 12:16, Magnus Ihse Bursie wrote: >>>> >>>> On 2018-11-20 16:49, Alexey Ivanov wrote: >>>> >>>>> Hi Ali, Magnus, >>>>> >>>>> I absolutely agree this change has to be reviewed by someone from >>>>> serviceability. >>>>> >>>>> There are several options: >>>>> >>>>> 1. Add -export:jdwpTransport_OnLoad to LDFLAGS for Windows >>>>> http://mail.openjdk.java.net/pipermail/build-dev/2018-November/023935.html >>>>> so it partially reverts the changes from >>>>> https://bugs.openjdk.java.net/browse/JDK-8200178 >>>>> >>>>> 2. Remove JNICALL (__stdcall) modifier, eventually changing the >>>>> calling convention to __cdecl. >>>>> http://mail.openjdk.java.net/pipermail/build-dev/2018-November/023969.html >>>>> >>>>> 3. Use inline /export option via #pragma: >>>>> #pragma comment(linker, "/export:jdwpTransport_OnLoad= >>>>> _jdwpTransport_OnLoad at 16") >>>>> as referenced in >>>>> https://docs.microsoft.com/en-ie/cpp/cpp/dllexport-dllimport?view=vs-2017 >>>>> https://docs.microsoft.com/en-ie/cpp/build/reference/exports?view=vs-2017 >>>>> >>>>> The third options still needs to be tested to make sure it does >>>>> not break other platforms builds. >>>> I'm not fond of either solution 1 or 3. I really think the >>>> signature should be made correctly at the point of definition in >>>> the source code. >>> >>> That is why I proposed removing JNICALL (__stdcall) from the >>> function declaration as we had done for libzip, libjimage [1] and >>> mlib_image [2]. >>> >>> Microsoft recommends using __stdcall for DLLs, at the same time it >>> decorates the function name making it harder to export it by its >>> plain name. >>> >>> >>> By removing JNICALL / __stdcall, we make the function use __cdecl >>> calling convention. The difference between them is that with >>> __stdcall the callee cleans the stack whereas with __cdecl the >>> caller cleans the stack. >>> >>> It shouldn't be a problem as long as all function declarations use >>> the same calling convention, which, I believe, is the case here. >>> >>>> We've spent some considerable effort of getting rid of the /export >>>> flags for Windows (and map files for unix), and I don't want to see >>>> them creep back in. Note that option 3 is just option 1 in >>>> disguise. The only single good thing about it is that it allows you >>>> to put the compiler option next to the signature declaration in the >>>> source code. >>> >>> At least in this case, the option will be near the function body >>> definition. It will be easier to update it if the signature changes. >>> >>> If we are to use __stdcall, it seems to be the only way to export >>> the undecorated name. >>> >>>> The same goes for def files, as suggested by Ali. It's just yet >>>> another version of option 1 in disguise/map files. >>> >>> If option 2 is rejected, I'm for using option 3. If option 3 cannot >>> be used too, I'm for option 1. >>> I think we should not fall back to def files in any case. And >>> Makefile will have to include the reference to the def file, so it's >>> even worse than option 1. >>> >>> >>> Regards, >>> Alexey >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8201226 >>> [2] https://bugs.openjdk.java.net/browse/JDK-8202476 >>>> >>>> /Magnus >>>> >>>>> >>>>> >>>>> Regards, >>>>> Alexey >>>>> >>>>> On 19/11/2018 12:19, Magnus Ihse Bursie wrote: >>>>>> Hi Ali, >>>>>> >>>>>> From a build perspective this change looks OK. I'm not aware of >>>>>> the finer details on the OnLoad mechanism, like what calling >>>>>> convention is to be used. So maybe this is a no-go from that view. >>>>>> >>>>>> I'm cc:ing servicability so they can have a look at it. >>>>>> >>>>>> /Magnus >>>>>> >>>>>> On 2018-11-18 13:07, Ali ?nce wrote: >>>>>>> Hi Alexey, >>>>>>> >>>>>>> Below is a new patch content that removes JNICALL modifiers from the exported functions of interest. This results in exported functions not to be name decorated and use __cdecl calling convention. >>>>>>> >>>>>>> Do you have any more suggestions, or would you please guide me on next steps? >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Ali >>>>>>> >>>>>>> # HG changeset patch >>>>>>> # User Ali Ince >>>>>>> # Date 1542542415 0 >>>>>>> # Sun Nov 18 12:00:15 2018 +0000 >>>>>>> # Node ID a41df44e13e1b761f4b3f05a17ed18953067ae39 >>>>>>> # Parent 16d5ec7dbbb49ef3f969e34be870e3f917ea89ff >>>>>>> Remove JNICALL modifiers to prevent name decoration on 32-bit windows builds >>>>>>> >>>>>>> diff -r 16d5ec7dbbb4 -r a41df44e13e1 src/jdk.jdi/share/native/libdt_shmem/shmemBack.c >>>>>>> --- a/src/jdk.jdi/share/native/libdt_shmem/shmemBack.c Tue Aug 14 14:28:23 2018 +0200 >>>>>>> +++ b/src/jdk.jdi/share/native/libdt_shmem/shmemBack.c Sun Nov 18 12:00:15 2018 +0000 >>>>>>> @@ -339,7 +339,7 @@ >>>>>>> return JDWPTRANSPORT_ERROR_NONE; >>>>>>> } >>>>>>> >>>>>>> -JNIEXPORT jint JNICALL >>>>>>> +JNIEXPORT jint >>>>>>> jdwpTransport_OnLoad(JavaVM *vm, jdwpTransportCallback* cbTablePtr, >>>>>>> jint version, jdwpTransportEnv** result) >>>>>>> { >>>>>>> diff -r 16d5ec7dbbb4 -r a41df44e13e1 src/jdk.jdwp.agent/share/native/include/jdwpTransport.h >>>>>>> --- a/src/jdk.jdwp.agent/share/native/include/jdwpTransport.h Tue Aug 14 14:28:23 2018 +0200 >>>>>>> +++ b/src/jdk.jdwp.agent/share/native/include/jdwpTransport.h Sun Nov 18 12:00:15 2018 +0000 >>>>>>> @@ -140,7 +140,7 @@ >>>>>>> void (*free)(void *buffer); /* Call this for all deallocations */ >>>>>>> } jdwpTransportCallback; >>>>>>> >>>>>>> -typedef jint (JNICALL *jdwpTransport_OnLoad_t)(JavaVM *jvm, >>>>>>> +typedef jint (*jdwpTransport_OnLoad_t)(JavaVM *jvm, >>>>>>> jdwpTransportCallback *callback, >>>>>>> jint version, >>>>>>> jdwpTransportEnv** env); >>>>>>> diff -r 16d5ec7dbbb4 -r a41df44e13e1 src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c >>>>>>> --- a/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c Tue Aug 14 14:28:23 2018 +0200 >>>>>>> +++ b/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c Sun Nov 18 12:00:15 2018 +0000 >>>>>>> @@ -1019,7 +1019,7 @@ >>>>>>> return JDWPTRANSPORT_ERROR_NONE; >>>>>>> } >>>>>>> >>>>>>> -JNIEXPORT jint JNICALL >>>>>>> +JNIEXPORT jint >>>>>>> jdwpTransport_OnLoad(JavaVM *vm, jdwpTransportCallback* cbTablePtr, >>>>>>> jint version, jdwpTransportEnv** env) >>>>>>> { >>>>>>> >>>>>>> >>>>>>> >>>>>>>> On 16 Nov 2018, at 23:03, Alexey Ivanov wrote: >>>>>>>> >>>>>>>> Hi Ali, >>>>>>>> >>>>>>>> It's exactly what I referred to. >>>>>>>> >>>>>>>> Yes, it does change the calling convention. >>>>>>>> Yet it's not a (big) problem because both parties will use the same calling convention. >>>>>>>> >>>>>>>> Using a DLL from another build will not be possible. But it's not a good idea to do so anyway. >>>>>>>> >>>>>>>> >>>>>>>> Mapfiles and export options have been removed byhttps://bugs.openjdk.java.net/browse/JDK-8200178 to simplify managing exported functions. So that to add or remove an exported function, you don't have modify makefiles and/or mapfiles. You just edit the source files. >>>>>>>> >>>>>>>> Regards, >>>>>>>> Alexey >>>>>>>> >>>>>>>> On 16/11/2018 22:42, Ali ?nce wrote: >>>>>>>>> Hi Alexey, >>>>>>>>> >>>>>>>>> Thanks for your reply. >>>>>>>>> >>>>>>>>> I will definitely give it a try though I?m not sure if that?ll be a breaking change. This is because JNICALL modifier is defined to be __stdcall on windows which is both specified on jdwpTransport_OnLoad exported functions both for dt_socket.dll and dt_shmem.dll. >>>>>>>>> >>>>>>>>> The __stdcall is ignored on x64 platforms and also name decoration is not applied (https://docs.microsoft.com/en-gb/cpp/build/reference/decorated-names?view=vs-2017). I believe that?s why we don?t experience any problems on x64 builds. >>>>>>>>> >>>>>>>>> Removing JNICALL (thus __stdcall) modifier (apparently only applies to x86 builds) will result in the calling convention to be changed, and thus will change how parameters are ordered and also the stack cleanup rules. I think this may result in problems in programs that are designed to load shared libraries and its exported functions at runtime (not bound by link time which probably won?t cause any issues - assuming that they use the shared function signature) - as inhttps://github.com/AdoptOpenJDK/openjdk-jdk11u/blob/5f01925b80ed851b133ee26fbcb07026ac04149e/src/jdk.jdwp.agent/share/native/libjdwp/transport.c#L99. >>>>>>>>> >>>>>>>>> Any thoughts? >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> >>>>>>>>> Ali >>>>>>>>> >>>>>>>>>> On 15 Nov 2018, at 22:14, Alexey Ivanov wrote: >>>>>>>>>> >>>>>>>>>> Hi Ali, >>>>>>>>>> >>>>>>>>>> Can the issue be resolved by using different call modifiers on the affected functions? >>>>>>>>>> >>>>>>>>>> Some build / link issues were resolved by adding / removing JNICALL modifier, see >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8201226 >>>>>>>>>> http://mail.openjdk.java.net/pipermail/build-dev/2018-April/021553.html >>>>>>>>>> >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8202476 >>>>>>>>>> http://mail.openjdk.java.net/pipermail/build-dev/2018-May/021913.html >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> Alexey >>>>>>>>>> >>>>>>>>>> On 15/11/2018 21:43, Ali ?nce wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> An issue (https://github.com/AdoptOpenJDK/openjdk-build/issues/709) raised against AdoptOpenJDK jdk11 32-bit windows builds led us to the name decoration problem on built 32-bit dlls - specifically dt_socket.dll and dt_shmem.dll. Whereas 64-bit MSVC builds don?t apply any name decorations on exported functions, 32-bit builds apply them - which requires either def files or /export flags to be specified on the linker arguments. >>>>>>>>>>> >>>>>>>>>>> Although the expected linker arguments were present on previous jdk makefiles, they were removed in jdk11 makefiles. >>>>>>>>>>> >>>>>>>>>>> Below is the proposed patch, which can also be browsed athttps://github.com/AdoptOpenJDK/openjdk-jdk11u/pull/1. >>>>>>>>>>> >>>>>>>>>>> Would you please have a look and let me know of any points I may have missed (I don?t have any background information about why the export flags were removed in jdk11)? >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> >>>>>>>>>>> Ali >>>>>>>>>>> >>>>>>>>>>> diff --git a/make/lib/Lib-jdk.jdi.gmk b/make/lib/Lib-jdk.jdi.gmk >>>>>>>>>>> index 197b95c2e2..ac6ebf5591 100644 >>>>>>>>>>> --- a/make/lib/Lib-jdk.jdi.gmk >>>>>>>>>>> +++ b/make/lib/Lib-jdk.jdi.gmk >>>>>>>>>>> @@ -37,6 +37,7 @@ ifeq ($(OPENJDK_TARGET_OS), windows) >>>>>>>>>>> jdk.jdwp.agent:include \ >>>>>>>>>>> jdk.jdwp.agent:libjdwp/export, \ >>>>>>>>>>> LDFLAGS := $(LDFLAGS_JDKLIB), \ >>>>>>>>>>> + LDFLAGS_windows := -export:jdwpTransport_OnLoad, \ >>>>>>>>>>> LIBS := $(JDKLIB_LIBS), \ >>>>>>>>>>> )) >>>>>>>>>>> >>>>>>>>>>> diff --git a/make/lib/Lib-jdk.jdwp.agent.gmk b/make/lib/Lib-jdk.jdwp.agent.gmk >>>>>>>>>>> index 0bc93e0d35..0382e55325 100644 >>>>>>>>>>> --- a/make/lib/Lib-jdk.jdwp.agent.gmk >>>>>>>>>>> +++ b/make/lib/Lib-jdk.jdwp.agent.gmk >>>>>>>>>>> @@ -37,6 +37,7 @@ $(eval $(call SetupJdkLibrary, BUILD_LIBDT_SOCKET, \ >>>>>>>>>>> libjdwp/export, \ >>>>>>>>>>> LDFLAGS := $(LDFLAGS_JDKLIB) \ >>>>>>>>>>> $(call SET_SHARED_LIBRARY_ORIGIN), \ >>>>>>>>>>> + LDFLAGS_windows := -export:jdwpTransport_OnLoad, \ >>>>>>>>>>> LIBS_linux := -lpthread, \ >>>>>>>>>>> LIBS_solaris := -lnsl -lsocket, \ >>>>>>>>>>> LIBS_windows := $(JDKLIB_LIBS) ws2_32.lib, \ >>>>>> >>>>> >>>> >>> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From coleen.phillimore at oracle.com Fri Dec 7 21:57:43 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 7 Dec 2018 16:57:43 -0500 Subject: RFR (T) 8215034: Remove old HOTSWAP conditionals Message-ID: I was in the area and found these few remaining conditionals. Tested with tier1 on Oracle platforms. open webrev at http://cr.openjdk.java.net/~coleenp/8215034.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8215034 Thanks, Coleen From daniel.daugherty at oracle.com Fri Dec 7 22:04:35 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 7 Dec 2018 17:04:35 -0500 Subject: RFR (T) 8215034: Remove old HOTSWAP conditionals In-Reply-To: References: Message-ID: <5b1e8f41-df32-4a1a-e7d3-e8d744d3698b@oracle.com> On 12/7/18 4:57 PM, coleen.phillimore at oracle.com wrote: > I was in the area and found these few remaining conditionals. > > Tested with tier1 on Oracle platforms. > > open webrev at http://cr.openjdk.java.net/~coleenp/8215034.01/webrev src/hotspot/share/code/codeCache.cpp ??? No comments. src/hotspot/share/code/codeCache.hpp ??? No comments. src/hotspot/share/interpreter/templateInterpreter.hpp ??? No comments. src/hotspot/share/utilities/globalDefinitions.hpp ??? No comments. Thumbs up! Agree this is 'trivial'. Dan > bug link https://bugs.openjdk.java.net/browse/JDK-8215034 > > Thanks, > Coleen From jiangli.zhou at oracle.com Fri Dec 7 22:06:14 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Fri, 7 Dec 2018 14:06:14 -0800 Subject: RFR (T) 8215034: Remove old HOTSWAP conditionals In-Reply-To: References: Message-ID: Looks good and trivial. Thanks, Jiangli On 12/7/18 1:57 PM, coleen.phillimore at oracle.com wrote: > I was in the area and found these few remaining conditionals. > > Tested with tier1 on Oracle platforms. > > open webrev at http://cr.openjdk.java.net/~coleenp/8215034.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8215034 > > Thanks, > Coleen From coleen.phillimore at oracle.com Fri Dec 7 22:50:49 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 7 Dec 2018 17:50:49 -0500 Subject: RFR (T) 8215034: Remove old HOTSWAP conditionals In-Reply-To: References: Message-ID: <4546215b-4b24-ebb8-ebdb-dd90588509c5@oracle.com> Thanks Dan, Jiangli, and Serguei! Coleen On 12/7/18 5:50 PM, serguei.spitsyn at oracle.com wrote: > Hi Coleen, > > +1 > > Thanks, > Serguei > > On 12/7/18 2:06 PM, Jiangli Zhou wrote: >> Looks good and trivial. >> >> Thanks, >> >> Jiangli >> >> >> On 12/7/18 1:57 PM, coleen.phillimore at oracle.com wrote: >>> I was in the area and found these few remaining conditionals. >>> >>> Tested with tier1 on Oracle platforms. >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8215034.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8215034 >>> >>> Thanks, >>> Coleen >> > From serguei.spitsyn at oracle.com Fri Dec 7 22:50:12 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 7 Dec 2018 14:50:12 -0800 Subject: RFR (T) 8215034: Remove old HOTSWAP conditionals In-Reply-To: References: Message-ID: Hi Coleen, +1 Thanks, Serguei On 12/7/18 2:06 PM, Jiangli Zhou wrote: > Looks good and trivial. > > Thanks, > > Jiangli > > > On 12/7/18 1:57 PM, coleen.phillimore at oracle.com wrote: >> I was in the area and found these few remaining conditionals. >> >> Tested with tier1 on Oracle platforms. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8215034.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8215034 >> >> Thanks, >> Coleen > From rwestrel at redhat.com Fri Dec 7 10:40:06 2018 From: rwestrel at redhat.com (Roland Westrelin) Date: Fri, 07 Dec 2018 11:40:06 +0100 Subject: RFR (round 5), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <8c267450-9ca8-79a6-0e4a-39b3b68af2e3@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <8c267450-9ca8-79a6-0e4a-39b3b68af2e3@redhat.com> Message-ID: <871s6tiprt.fsf@redhat.com> > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/ C2 changes look good to me. Roland. From leonid.mesnik at oracle.com Sat Dec 8 05:53:06 2018 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Fri, 7 Dec 2018 21:53:06 -0800 Subject: RFR(XS): 8215042: Move serviceability/sa tests from tier1 to tier3. Message-ID: <1BA254CB-E62C-4E94-ACA3-129414EF0E19@oracle.com> Hi Could you please review following fix which moves SA tests from tier1 to tier3. There are some bugs which cause intermittent failures of any test. SA tests fail intermittently are not stable enough for tier1. However failures are not very frequent. Also I don't think that putting all test in Problemlist.txt is very good idea because it left SA without any testing at all. So now all SA tests which are included in hotspot_tier3_runtime group. webrev: http://cr.openjdk.java.net/~lmesnik/8215042/webrev.00/ bug: https://bugs.openjdk.java.net/browse/JDK-8215042 Leonid -------------- next part -------------- An HTML attachment was scrubbed... URL: From jcbeyler at google.com Sat Dec 8 06:07:18 2018 From: jcbeyler at google.com (JC Beyler) Date: Fri, 7 Dec 2018 22:07:18 -0800 Subject: RFR(XS): 8215042: Move serviceability/sa tests from tier1 to tier3. In-Reply-To: <1BA254CB-E62C-4E94-ACA3-129414EF0E19@oracle.com> References: <1BA254CB-E62C-4E94-ACA3-129414EF0E19@oracle.com> Message-ID: Hi Leonid, I cannot comment on whether it is a good idea to put the tests in tier3 but the webrev does look good to achieve that :) So LGTM as far as it seems to do what you intended :) Thanks, Jc On Fri, Dec 7, 2018 at 9:53 PM Leonid Mesnik wrote: > Hi > > Could you please review following fix which moves SA tests from tier1 to > tier3. There are some bugs which cause intermittent failures of any test. > SA tests fail intermittently are not stable enough for tier1. > However failures are not very frequent. Also I don't think that putting > all test in Problemlist.txt is very good idea because it left SA without > any testing at all. > So now all SA tests which are included in hotspot_tier3_runtime group. > > webrev: http://cr.openjdk.java.net/~lmesnik/8215042/webrev.00/ > bug: https://bugs.openjdk.java.net/browse/JDK-8215042 > > Leonid > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From gary.adams at oracle.com Sat Dec 8 11:05:59 2018 From: gary.adams at oracle.com (gary.adams at oracle.com) Date: Sat, 8 Dec 2018 06:05:59 -0500 Subject: RFR(XS): 8215042: Move serviceability/sa tests from tier1 to tier3. In-Reply-To: <1BA254CB-E62C-4E94-ACA3-129414EF0E19@oracle.com> References: <1BA254CB-E62C-4E94-ACA3-129414EF0E19@oracle.com> Message-ID: Looks OK to me. On 12/8/18 12:53 AM, Leonid Mesnik wrote: > Hi > > Could you please review following fix which moves SA tests from tier1 > to tier3. There are some bugs which cause intermittent failures of any > test. SA tests fail intermittently are not stable enough for tier1. > However failures are not very frequent. Also I don't think that > putting all test in Problemlist.txt is very good idea because it left > SA without any testing at all. > So now all SA tests which are included in hotspot_tier3_runtime group. > > webrev: http://cr.openjdk.java.net/~lmesnik/8215042/webrev.00/ > > bug: https://bugs.openjdk.java.net/browse/JDK-8215042 > > Leonid -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Sat Dec 8 11:18:34 2018 From: david.holmes at oracle.com (David Holmes) Date: Sat, 8 Dec 2018 21:18:34 +1000 Subject: RFR(XS): 8215042: Move serviceability/sa tests from tier1 to tier3. In-Reply-To: <1BA254CB-E62C-4E94-ACA3-129414EF0E19@oracle.com> References: <1BA254CB-E62C-4E94-ACA3-129414EF0E19@oracle.com> Message-ID: <44995fad-d870-027c-66bb-9990d91fb723@oracle.com> Hi Leonid, My concern here, if we care about keeping the SA operational, is that in tier3 these tests will not be covered by the jdk/submit testing process. David On 8/12/2018 3:53 pm, Leonid Mesnik wrote: > Hi > > Could you please review following fix which moves SA tests from tier1 to > tier3. There are some bugs which cause intermittent failures of any > test. SA tests fail intermittently are not stable enough for tier1. > However failures are not very frequent. Also I don't think that putting > all test in Problemlist.txt is very good idea because it left SA without > any testing at all. > So now all SA tests which are included in hotspot_tier3_runtime group. > > webrev: http://cr.openjdk.java.net/~lmesnik/8215042/webrev.00/ > bug: https://bugs.openjdk.java.net/browse/JDK-8215042 > > Leonid From rkennke at redhat.com Sat Dec 8 11:33:02 2018 From: rkennke at redhat.com (Roman Kennke) Date: Sat, 8 Dec 2018 12:33:02 +0100 Subject: RFR (round 6), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <8c267450-9ca8-79a6-0e4a-39b3b68af2e3@redhat.com> Message-ID: <09d71e87-9fa1-619b-0cbd-259434e81822@redhat.com> Zhengyu's and Thomas' TaskQueue stuff landed in jdk/jdk and we integrated it into shenandoah/jdk and in the master patch and webrevs. I'm updating the webrevs in-place in: http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/06/ This is going to be the final webrev that I intend to push on Monday, unless somebody stops me or we face serious merge conflicts by then. I've just pushed this to jdk/submit for testing. I will also do more testing locally over the weekend. Thanks everybody for reviewing and helping and being patient! Cheers, Roman > Here comes round 6, possibly/hopefully the last round of webrevs for > upstreaming Shenandoah: > > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/06/ > > It incorporates: > - renames vm_operations_shenandoah* to shenandoahVMOperations*, as > suggested by Coleen > - reshuffles gcCause in shared-gc SA as suggested by Per > - print number of threads in SA, as suggested by Jini > - fixed a problem in webrev generation that did not track move of > CriticalNativeStress.java and CriticalNativeArgs.java > > The CSR has been approved, the JEP moved to target jdk12, and I got > positive reviews for all parts. I intend to push this early next week, > unless somebody stops me. If Zhengyu's and Thomas' TaskQueue change goes > in by then, I'll incorporate it. > > Thanks everybody for reviewing and reviewing and reviewing again ;-) > > Cheers, > Roman > >> Round 5 of Shenandoah review includes: >> - A fix for the @requires tag in TestFullGCCountTest.java. It should be >> correct now. We believe the CMS @requires was also not quite right and >> fixed it the same. >> >> It reads now: Don't run this test if: >> - Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is >> true, as set by harness >> - Actual GC set by harness is Shenandoah *and* >> ExplicitGCInvokesConcurrent is not set false by harness (it's true by >> default in Shenandoah, so this needs to be double-inverteed). >> >> The @requires for CMS was wrong before (we think), because it would also >> filter defaultGC + ExplicitGCInvokesConcurrent. >> >> - Sorting of macros was fixed, as was pointed out by Per >> - Some stuff was added to SA, as suggested by Jini >> - Rebased on most current jdk/jdk code >> >> Webrevs: >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/ >> >> I also need reviews from GC reviewers for the CSR: >> https://bugs.openjdk.java.net/browse/JDK-8214349 >> >> I already got reviews for: >> [x] shared-runtime (coleenp) >> [x] shared-compiler (kvn) >> >> I got reviews for shared-build, but an earlier version, so maybe makes >> sense to look over this again. Erik J, Magnus? >> >> I still need approvals for: >> [ ] shared-build (kvn, erikj, ihse, pliden) >> [ ] shared-gc (pliden, kbarrett) >> [ ] shared-serviceability (jgeorge, pliden) >> [ ] shared-tests (lmesnik, pliden) >> [ ] shenandoah-gc >> [ ] shenandoah-tests >> >> >> Thanks for your patience and ongoing support! >> >> Cheers, >> Roman >> >> >>> Hi all, >>> >>> here comes round 4 of Shenandoah upstreaming review: >>> >>> This includes fixes for the issues that Per brought up: >>> - Verify and gracefully reject dangerous flags combinations that >>> disables required barriers >>> - Revisited @requires filters in tests >>> - Trim unused code from Shenandoah's SA impl >>> - Move ShenandoahGCTracer to gc/shenandoah >>> - Fix ordering of GC names in various files >>> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >>> >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >>> >>> Thanks everybody for taking time to review this! >>> Roman >>> >>>> Hello all, >>>> >>>> Thanks so far for all the reviews and support! >>>> >>>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>>> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >>>> >>>> Things we've changed today: >>>> - We moved shenandoah-specific code out of .ad files into our own .ad >>>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>>> requires an addition in build machinery though, see >>>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>>> - Improved zero-disabling and build-code-simplification as suggested by >>>> Magnus and Per >>>> - Cleaned up some leftovers in C2 >>>> - Improved C2 loop opts code by introducing another APIs in >>>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now. >>>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>>> that. >>>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>>> - Rebased on jdk-12+22 >>>> >>>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>>> >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>>> >>>> Thanks, >>>> Roman >>>> >>>>> Alright, we fixed: >>>>> - The minor issues that Kim reported in shared-gc >>>>> - A lot of fixes in shared-tests according to Leonid's review >>>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>>> >>>>> Some notes: >>>>> Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>>> correct. The @requires there means to exclude runs with both CMS and >>>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>>> fine. Adding the same clause for Shenandoah means the same, and filters >>>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>>> made the condition a bit clearer by avoiding triple-negation. >>>>> >>>>> See: >>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>>> >>>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>>> >>>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>>> those with ZGC? >>>>> >>>>> See: >>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>>> >>>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>>> next round). >>>>> >>>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>>> better. I can tell that we're not done with C2 yet. Can you look over >>>>> the code and see what is ok, and especially what is not ok, so that we >>>>> can focus our efforts on the relevant parts? >>>>> >>>>> Updated set of webrevs: >>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>>> >>>>> Thanks, >>>>> Roman >>>>> >>>>> >>>>>> Hi, >>>>>> >>>>>> This is the first round of changes for including Shenandoah GC into >>>>>> mainline. >>>>>> I divided the review into parts that roughly correspond to the mailing lists >>>>>> that would normally review it, and I divided it into 'shared' code >>>>>> changes and >>>>>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>>>>> eventually >>>>>> push them as single 'combined' changeset, once reviewed. >>>>>> >>>>>> JEP: >>>>>> ? https://openjdk.java.net/jeps/189 >>>>>> Bug entry: >>>>>> >>>>>> ?https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>>> >>>>>> Webrevs: >>>>>> ? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>>> >>>>>> For those who want to see the full change, have a look at the >>>>>> shenandoah-complete >>>>>> >>>>>> directory, >>>>>> it contains the full combined webrev. Alternatively, there is the file >>>>>> shenandoah-master.patch >>>>>> , >>>>>> which is what I intend to commit (and which should be equivalent to the >>>>>> 'shenandoah-complete' webrev). >>>>>> >>>>>> Sections to review (at this point) are the following: >>>>>> ?*) shenandoah-gc >>>>>> >>>>>> ??? - Actual Shenandoah implementation, almost completely residing in >>>>>> gc/shenandoah >>>>>> >>>>>> ?*) shared-gc >>>>>> >>>>>> ??? - This is mostly boilerplate that is common to any GC >>>>>> ??? - referenceProcessor.cpp has a little change to make one assert not >>>>>> fail (next to CMS and G1) >>>>>> ??? - taskqueue.hpp has some small adjustments to enable subclassing >>>>>> >>>>>> ?*) shared-serviceability >>>>>> >>>>>> ??? - The usual code to support another GC >>>>>> >>>>>> ?*) shared-runtime >>>>>> >>>>>> ??? - A number of friends declarations to allow Shenandoah iterators to >>>>>> hook up with, >>>>>> ????? e.g. ClassLoaderData, CodeCache, etc >>>>>> ??? - Warning and disabling JFR LeakProfiler >>>>>> ??? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>>> Shenandoah C2 optimizations >>>>>> ??? - Locks initialization in mutexLocker.cpp as usual >>>>>> ??? - VM operations defines for Shenandoah's VM ops >>>>>> ??? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>>> Shenandoah's logging >>>>>> ??? - The usual macros in macro.hpp >>>>>> >>>>>> ?*) shared-build >>>>>> >>>>>> ??? - Add shenandoah feature, enabled by default, as agreed with >>>>>> Vladimir K. beforehand >>>>>> ??? - Some flags for shenandoah-enabled compilation to get >>>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>>> ????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>>> Shenandoah's barriers >>>>>> ??? - --param inline-unit-growth=1000 settings for 2 shenandoah source >>>>>> files, which is >>>>>> ????? useful to get the whole marking loop inlined (observed significant >>>>>> regression if we >>>>>> ????? don't) >>>>>> >>>>>> ?*) shared-tests >>>>>> >>>>>> ??? - Test infrastructure to support Shenandoah >>>>>> ??? - Shenandoah test groups >>>>>> ??? - Exclude Shenandoah in various tests that can be run with selected GC >>>>>> ??? - Enable/add configure for Shenandoah for tests that make sense to >>>>>> run with it >>>>>> >>>>>> ?*) shenandoah-tests >>>>>> >>>>>> ??? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >>>>>> ??? - A couple of tests configurations have been added, e.g. >>>>>> TestGCBasherWithShenandoah.java >>>>>> >>>>>> I intentionally left out shared-compiler for now, because we have some >>>>>> work left to do >>>>>> there, but if you click around you'll find the patch anyway, in case you >>>>>> want to take >>>>>> a peek at it. >>>>>> >>>>>> We have regular builds on: >>>>>> ? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>>> ? - {Windows} x {x86_64}, >>>>>> ? - {MacOS X} x {x86_64} >>>>>> >>>>>> This also routinely passes: >>>>>> ? - the new Shenandoah tests >>>>>> ? - jcstress with/without aggressive Shenandoah verification >>>>>> ? - specjvm2008 with/without aggressive Shenandoah verification >>>>>> >>>>>> >>>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>>> teams for tirelessly helping with and reviewing all the GC interface and >>>>>> related changes, and of course the many early adopters for reporting >>>>>> bugs and success stories and feature requests: we wouldn't be here >>>>>> without any of you! >>>>>> >>>>>> Best regards, >>>>>> Roman >>>>>> >>>>> >>>> >>> >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Sat Dec 8 13:57:53 2018 From: rkennke at redhat.com (Roman Kennke) Date: Sat, 8 Dec 2018 14:57:53 +0100 Subject: RFR (round 6), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <09d71e87-9fa1-619b-0cbd-259434e81822@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <8c267450-9ca8-79a6-0e4a-39b3b68af2e3@redhat.com> <09d71e87-9fa1-619b-0cbd-259434e81822@redhat.com> Message-ID: Anybody has an idea what's up with the test below? As far as I can tell, it passes when I run locally. Thanks, Roman mach5-one-rkennke-JDK-8214259-20181208-1149-13785: FAILED, Failed tests: 1 Build Details: 2018-12-08-1147316.roman.source 1 Failed Test Test Tier Platform Keywords Description Task compiler/aot/calls/fromAot/AotInvokeDynamic2AotTest.java tier1 linux-x64-debug othervm driver Exception: java.lang.AssertionError: duplicate classes for name Ljava/lang/Object;, fingerprints old: ..., new: ..., klass pointers old: meta{HotSpotType}, new: meta{HotSpotType} task Mach5 Tasks Results Summary FAILED: 0 EXECUTED_WITH_FAILURE: 1 NA: 0 KILLED: 0 PASSED: 75 UNABLE_TO_RUN: 0 Test 1 Executed with failure tier1-debug-open_test_hotspot_jtreg_tier1_compiler_not_xcomp-linux-x64-debug-37 Results: total: 73, passed: 72, failed: 1, skipped: 0 Am 08.12.18 um 12:33 schrieb Roman Kennke: > Zhengyu's and Thomas' TaskQueue stuff landed in jdk/jdk and we > integrated it into shenandoah/jdk and in the master patch and webrevs. > I'm updating the webrevs in-place in: > > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/06/ > > This is going to be the final webrev that I intend to push on Monday, > unless somebody stops me or we face serious merge conflicts by then. > > I've just pushed this to jdk/submit for testing. I will also do more > testing locally over the weekend. > > Thanks everybody for reviewing and helping and being patient! > > Cheers, > Roman > >> Here comes round 6, possibly/hopefully the last round of webrevs for >> upstreaming Shenandoah: >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/06/ >> >> It incorporates: >> - renames vm_operations_shenandoah* to shenandoahVMOperations*, as >> suggested by Coleen >> - reshuffles gcCause in shared-gc SA as suggested by Per >> - print number of threads in SA, as suggested by Jini >> - fixed a problem in webrev generation that did not track move of >> CriticalNativeStress.java and CriticalNativeArgs.java >> >> The CSR has been approved, the JEP moved to target jdk12, and I got >> positive reviews for all parts. I intend to push this early next week, >> unless somebody stops me. If Zhengyu's and Thomas' TaskQueue change goes >> in by then, I'll incorporate it. >> >> Thanks everybody for reviewing and reviewing and reviewing again ;-) >> >> Cheers, >> Roman >> >>> Round 5 of Shenandoah review includes: >>> - A fix for the @requires tag in TestFullGCCountTest.java. It should be >>> correct now. We believe the CMS @requires was also not quite right and >>> fixed it the same. >>> >>> It reads now: Don't run this test if: >>> - Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is >>> true, as set by harness >>> - Actual GC set by harness is Shenandoah *and* >>> ExplicitGCInvokesConcurrent is not set false by harness (it's true by >>> default in Shenandoah, so this needs to be double-inverteed). >>> >>> The @requires for CMS was wrong before (we think), because it would also >>> filter defaultGC + ExplicitGCInvokesConcurrent. >>> >>> - Sorting of macros was fixed, as was pointed out by Per >>> - Some stuff was added to SA, as suggested by Jini >>> - Rebased on most current jdk/jdk code >>> >>> Webrevs: >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/ >>> >>> I also need reviews from GC reviewers for the CSR: >>> https://bugs.openjdk.java.net/browse/JDK-8214349 >>> >>> I already got reviews for: >>> [x] shared-runtime (coleenp) >>> [x] shared-compiler (kvn) >>> >>> I got reviews for shared-build, but an earlier version, so maybe makes >>> sense to look over this again. Erik J, Magnus? >>> >>> I still need approvals for: >>> [ ] shared-build (kvn, erikj, ihse, pliden) >>> [ ] shared-gc (pliden, kbarrett) >>> [ ] shared-serviceability (jgeorge, pliden) >>> [ ] shared-tests (lmesnik, pliden) >>> [ ] shenandoah-gc >>> [ ] shenandoah-tests >>> >>> >>> Thanks for your patience and ongoing support! >>> >>> Cheers, >>> Roman >>> >>> >>>> Hi all, >>>> >>>> here comes round 4 of Shenandoah upstreaming review: >>>> >>>> This includes fixes for the issues that Per brought up: >>>> - Verify and gracefully reject dangerous flags combinations that >>>> disables required barriers >>>> - Revisited @requires filters in tests >>>> - Trim unused code from Shenandoah's SA impl >>>> - Move ShenandoahGCTracer to gc/shenandoah >>>> - Fix ordering of GC names in various files >>>> - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W >>>> >>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ >>>> >>>> Thanks everybody for taking time to review this! >>>> Roman >>>> >>>>> Hello all, >>>>> >>>>> Thanks so far for all the reviews and support! >>>>> >>>>> I forgot to update the 'round' yesterday. We are in round 3 now :-) >>>>> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >>>>> >>>>> Things we've changed today: >>>>> - We moved shenandoah-specific code out of .ad files into our own .ad >>>>> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >>>>> requires an addition in build machinery though, see >>>>> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >>>>> - Improved zero-disabling and build-code-simplification as suggested by >>>>> Magnus and Per >>>>> - Cleaned up some leftovers in C2 >>>>> - Improved C2 loop opts code by introducing another APIs in >>>>> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >>>>> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now. >>>>> - We would all very much prefer to keep ShenandoahXYZNode names, as >>>>> noted earlier. This stuff is Shenandoah-specific, so let's just call it >>>>> that. >>>>> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >>>>> - Rebased on jdk-12+22 >>>>> >>>>> - Question: let us know if you need separate RFE for the new BSC2 APIs? >>>>> >>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >>>>> >>>>> Thanks, >>>>> Roman >>>>> >>>>>> Alright, we fixed: >>>>>> - The minor issues that Kim reported in shared-gc >>>>>> - A lot of fixes in shared-tests according to Leonid's review >>>>>> - Disabled SA heapdumping similar to ZGC as Per suggested >>>>>> >>>>>> Some notes: >>>>>> Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>>>>> correct. The @requires there means to exclude runs with both CMS and >>>>>> ExplicitGCInvokesConcurrent at the same time, because that would be >>>>>> (expectedly) failing. It can run CMS, default GC and any other GC just >>>>>> fine. Adding the same clause for Shenandoah means the same, and filters >>>>>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>>>>> made the condition a bit clearer by avoiding triple-negation. >>>>>> >>>>>> See: >>>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>>>>> >>>>>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>>>>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>>>>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>>>>> >>>>>> we filter them for Shenandoah now. I'm wondering: how do you get past >>>>>> those with ZGC? >>>>>> >>>>>> See: >>>>>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>>>>> >>>>>> (Note to Leonid and tests reviewers: I'll add those related filters in >>>>>> next round). >>>>>> >>>>>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>>>>> better. I can tell that we're not done with C2 yet. Can you look over >>>>>> the code and see what is ok, and especially what is not ok, so that we >>>>>> can focus our efforts on the relevant parts? >>>>>> >>>>>> Updated set of webrevs: >>>>>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>>>>> >>>>>> Thanks, >>>>>> Roman >>>>>> >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> This is the first round of changes for including Shenandoah GC into >>>>>>> mainline. >>>>>>> I divided the review into parts that roughly correspond to the mailing lists >>>>>>> that would normally review it, and I divided it into 'shared' code >>>>>>> changes and >>>>>>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>>>>>> eventually >>>>>>> push them as single 'combined' changeset, once reviewed. >>>>>>> >>>>>>> JEP: >>>>>>> ? https://openjdk.java.net/jeps/189 >>>>>>> Bug entry: >>>>>>> >>>>>>> ?https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>>>> >>>>>>> Webrevs: >>>>>>> ? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>>>> >>>>>>> For those who want to see the full change, have a look at the >>>>>>> shenandoah-complete >>>>>>> >>>>>>> directory, >>>>>>> it contains the full combined webrev. Alternatively, there is the file >>>>>>> shenandoah-master.patch >>>>>>> , >>>>>>> which is what I intend to commit (and which should be equivalent to the >>>>>>> 'shenandoah-complete' webrev). >>>>>>> >>>>>>> Sections to review (at this point) are the following: >>>>>>> ?*) shenandoah-gc >>>>>>> >>>>>>> ??? - Actual Shenandoah implementation, almost completely residing in >>>>>>> gc/shenandoah >>>>>>> >>>>>>> ?*) shared-gc >>>>>>> >>>>>>> ??? - This is mostly boilerplate that is common to any GC >>>>>>> ??? - referenceProcessor.cpp has a little change to make one assert not >>>>>>> fail (next to CMS and G1) >>>>>>> ??? - taskqueue.hpp has some small adjustments to enable subclassing >>>>>>> >>>>>>> ?*) shared-serviceability >>>>>>> >>>>>>> ??? - The usual code to support another GC >>>>>>> >>>>>>> ?*) shared-runtime >>>>>>> >>>>>>> ??? - A number of friends declarations to allow Shenandoah iterators to >>>>>>> hook up with, >>>>>>> ????? e.g. ClassLoaderData, CodeCache, etc >>>>>>> ??? - Warning and disabling JFR LeakProfiler >>>>>>> ??? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>>>> Shenandoah C2 optimizations >>>>>>> ??? - Locks initialization in mutexLocker.cpp as usual >>>>>>> ??? - VM operations defines for Shenandoah's VM ops >>>>>>> ??? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>>>> Shenandoah's logging >>>>>>> ??? - The usual macros in macro.hpp >>>>>>> >>>>>>> ?*) shared-build >>>>>>> >>>>>>> ??? - Add shenandoah feature, enabled by default, as agreed with >>>>>>> Vladimir K. beforehand >>>>>>> ??? - Some flags for shenandoah-enabled compilation to get >>>>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>>>> ????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>>>> Shenandoah's barriers >>>>>>> ??? - --param inline-unit-growth=1000 settings for 2 shenandoah source >>>>>>> files, which is >>>>>>> ????? useful to get the whole marking loop inlined (observed significant >>>>>>> regression if we >>>>>>> ????? don't) >>>>>>> >>>>>>> ?*) shared-tests >>>>>>> >>>>>>> ??? - Test infrastructure to support Shenandoah >>>>>>> ??? - Shenandoah test groups >>>>>>> ??? - Exclude Shenandoah in various tests that can be run with selected GC >>>>>>> ??? - Enable/add configure for Shenandoah for tests that make sense to >>>>>>> run with it >>>>>>> >>>>>>> ?*) shenandoah-tests >>>>>>> >>>>>>> ??? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >>>>>>> ??? - A couple of tests configurations have been added, e.g. >>>>>>> TestGCBasherWithShenandoah.java >>>>>>> >>>>>>> I intentionally left out shared-compiler for now, because we have some >>>>>>> work left to do >>>>>>> there, but if you click around you'll find the patch anyway, in case you >>>>>>> want to take >>>>>>> a peek at it. >>>>>>> >>>>>>> We have regular builds on: >>>>>>> ? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>>>> ? - {Windows} x {x86_64}, >>>>>>> ? - {MacOS X} x {x86_64} >>>>>>> >>>>>>> This also routinely passes: >>>>>>> ? - the new Shenandoah tests >>>>>>> ? - jcstress with/without aggressive Shenandoah verification >>>>>>> ? - specjvm2008 with/without aggressive Shenandoah verification >>>>>>> >>>>>>> >>>>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>>>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>>>> teams for tirelessly helping with and reviewing all the GC interface and >>>>>>> related changes, and of course the many early adopters for reporting >>>>>>> bugs and success stories and feature requests: we wouldn't be here >>>>>>> without any of you! >>>>>>> >>>>>>> Best regards, >>>>>>> Roman >>>>>>> >>>>>> >>>>> >>>> >>> >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From kim.barrett at oracle.com Sat Dec 8 18:30:18 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 8 Dec 2018 13:30:18 -0500 Subject: RFR (round 6), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> <8c267450-9ca8-79a6-0e4a-39b3b68af2e3@redhat.com> <09d71e87-9fa1-619b-0cbd-259434e81822@redhat.com> Message-ID: <0A259B61-A4CA-4E2E-8FFF-E855E9285D6E@oracle.com> > On Dec 8, 2018, at 8:57 AM, Roman Kennke wrote: > > Anybody has an idea what's up with the test below? As far as I can tell, > it passes when I run locally. It appears to be an instance of https://bugs.openjdk.java.net/browse/JDK-8201248. > Thanks, > Roman > > mach5-one-rkennke-JDK-8214259-20181208-1149-13785: FAILED, Failed tests: 1 > > Build Details: 2018-12-08-1147316.roman.source > 1 Failed Test > Test Tier Platform Keywords Description Task > compiler/aot/calls/fromAot/AotInvokeDynamic2AotTest.java tier1 > linux-x64-debug othervm driver Exception: java.lang.AssertionError: > duplicate classes for name Ljava/lang/Object;, fingerprints old: ..., > new: ..., klass pointers old: meta{HotSpotType}, new: > meta{HotSpotType} task > Mach5 Tasks Results Summary > > FAILED: 0 > EXECUTED_WITH_FAILURE: 1 > NA: 0 > KILLED: 0 > PASSED: 75 > UNABLE_TO_RUN: 0 > Test > > 1 Executed with failure > > tier1-debug-open_test_hotspot_jtreg_tier1_compiler_not_xcomp-linux-x64-debug-37 > Results: total: 73, passed: 72, failed: 1, skipped: 0 From patricio.chilano.mateo at oracle.com Sat Dec 8 23:04:23 2018 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Sat, 8 Dec 2018 18:04:23 -0500 Subject: RFR 8215050: [TESTBUG] serviceability/tmtools/jstack/WaitNotifyThreadTest.java fails when run with flag -Xcomp Message-ID: Hi, Could you review this small fix for test serviceability/tmtools/jstack/WaitNotifyThreadTest.java ? After change 8214148 the test fails if the flag -Xcomp is used as explained in the bug details. The proposed change is to identified this special case and set the monitor address to match the one expected. http://cr.openjdk.java.net/~pchilanomate/8215050.02/webrev/ https://bugs.openjdk.java.net/browse/JDK-8215050 Run tier-4 where the test was failing and passed (the test was failing on tier6 also because it uses -Xcomp too). Currently running tiers1-3. Thanks, Patricio From david.holmes at oracle.com Sat Dec 8 23:21:57 2018 From: david.holmes at oracle.com (David Holmes) Date: Sun, 9 Dec 2018 09:21:57 +1000 Subject: RFR 8215050: [TESTBUG] serviceability/tmtools/jstack/WaitNotifyThreadTest.java fails when run with flag -Xcomp In-Reply-To: References: Message-ID: Hi Patricio, On 9/12/2018 9:04 am, Patricio Chilano wrote: > Hi, > > Could you review this small fix for test > serviceability/tmtools/jstack/WaitNotifyThreadTest.java ? > After change 8214148 the test fails if the flag -Xcomp is used as > explained in the bug details. The proposed change is to identified this > special case and set the monitor address to match the one expected. > > http://cr.openjdk.java.net/~pchilanomate/8215050.02/webrev/ > https://bugs.openjdk.java.net/browse/JDK-8215050 > > Run tier-4 where the test was failing and passed (the test was failing > on tier6 also because it uses -Xcomp too). Currently running tiers1-3. Change seems fine. The lack of address must be something relatively new as I don't see it with Xcomp in JDK 9. Thanks, David > Thanks, > Patricio From patricio.chilano.mateo at oracle.com Sun Dec 9 00:42:23 2018 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Sat, 8 Dec 2018 19:42:23 -0500 Subject: RFR 8215050: [TESTBUG] serviceability/tmtools/jstack/WaitNotifyThreadTest.java fails when run with flag -Xcomp In-Reply-To: References: Message-ID: Hi David, On 12/8/18 6:21 PM, David Holmes wrote: > Hi Patricio, > > On 9/12/2018 9:04 am, Patricio Chilano wrote: >> Hi, >> >> Could you review this small fix for test >> serviceability/tmtools/jstack/WaitNotifyThreadTest.java ? >> After change 8214148 the test fails if the flag -Xcomp is used as >> explained in the bug details. The proposed change is to identified >> this special case and set the monitor address to match the one expected. >> >> http://cr.openjdk.java.net/~pchilanomate/8215050.02/webrev/ >> https://bugs.openjdk.java.net/browse/JDK-8215050 >> >> Run tier-4 where the test was failing and passed (the test was >> failing on tier6 also because it uses -Xcomp too). Currently running >> tiers1-3. > > Change seems fine. The lack of address must be something relatively > new as I don't see it with Xcomp in JDK 9. Thanks for the review! Thanks, Patricio > Thanks, > David > >> Thanks, >> Patricio From jini.george at oracle.com Mon Dec 10 03:19:26 2018 From: jini.george at oracle.com (Jini George) Date: Mon, 10 Dec 2018 08:49:26 +0530 Subject: RFR(XS): 8215042: Move serviceability/sa tests from tier1 to tier3. In-Reply-To: <44995fad-d870-027c-66bb-9990d91fb723@oracle.com> References: <1BA254CB-E62C-4E94-ACA3-129414EF0E19@oracle.com> <44995fad-d870-027c-66bb-9990d91fb723@oracle.com> Message-ID: <5c9d711a-4a77-b649-1385-b4e6e2060bd8@oracle.com> Hi Leonid, I agree with David. I am also concerned about us not detecting SA breakages (which could happen along with hotspot changes) soon enough. (Which was the primary reason to get these tests in). Thank you, Jini. On 12/8/2018 4:48 PM, David Holmes wrote: > Hi Leonid, > > My concern here, if we care about keeping the SA operational, is that in > tier3 these tests will not be covered by the jdk/submit testing process. > > David > > On 8/12/2018 3:53 pm, Leonid Mesnik wrote: >> Hi >> >> Could you please review following fix which moves SA tests from tier1 >> to tier3. There are some bugs which cause intermittent failures of any >> test. SA tests fail intermittently are not stable enough for tier1. >> However failures are not very frequent. Also I don't think that >> putting all test in Problemlist.txt is very good idea because it left >> SA without any testing at all. >> So now all SA tests which are included in hotspot_tier3_runtime group. >> >> webrev: http://cr.openjdk.java.net/~lmesnik/8215042/webrev.00/ >> bug: https://bugs.openjdk.java.net/browse/JDK-8215042 >> >> Leonid From zanglin5 at jd.com Mon Dec 10 06:24:01 2018 From: zanglin5 at jd.com (=?gb2312?B?6rDB1Q==?=) Date: Mon, 10 Dec 2018 06:24:01 +0000 Subject: optimize KlassInfoTable size to power of 2 Message-ID: <140bac18565c48be88313f7cc1d24bf8@jd.com> Dear All, I am investigating JMap utility for large heap (~200GB) and found that the current KlassInfoTable?s _num_buckets size(20011) may not be optimal. My observation is that when ?jmap histo? iterating objects , the object?s klass is identified and then hash idx in KlassInfoTable?s buckets[] is calculated by mod of _num_bucktes, which would issue a heavy instruction idiv on x86 platform. It means for every object scanned, a idiv instruction is issued, which lag the performance espically when there are large number of objects in heap. Hence if the _num_buckets can be changed to a pow of 2, (e.g. 65536) the idiv can be replaced with a faster instruction such as shl (left bit shift), And I have prepared a patch for this change. My question is that why 20011 is used now? is there any special reason? And is there any potential problem if I change the value to 65536, or 32768? Thanks! BRs, Lin -------------- next part -------------- An HTML attachment was scrubbed... URL: From ioi.lam at oracle.com Mon Dec 10 06:31:46 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Sun, 9 Dec 2018 22:31:46 -0800 Subject: RFR: (S): JDK-8200613: SA: jstack throws UnmappedAddressException with a CDS core file In-Reply-To: <2c6a2d67-5d36-07f1-4563-c1cd2ecdf9a3@oracle.com> References: <864997e5-2581-7ac9-44f7-76cc31c9ad93@oracle.com> <60d2e6db-c87b-9658-57cb-b03ca881a624@oracle.com> <9eb9041d-f984-0df4-a21e-4645966a62b8@oracle.com> <42d6d604-8df5-47c4-a748-6aa25bfc9a7e@oracle.com> <2dbef041-c75e-07f6-fafc-d18f2a95f455@oracle.com> <798a4d93-1638-8785-885f-fac0b21c2417@oracle.com> <2c6a2d67-5d36-07f1-4563-c1cd2ecdf9a3@oracle.com> Message-ID: <2e0d7e6a-631f-6947-d49a-c57c4fe64691@oracle.com> Hi Jini, These changes look good to me. Thanks! - Ioi On 12/7/18 11:22 AM, Jini George wrote: > I have the revised webrev here: > > http://cr.openjdk.java.net/~jgeorge/8200613/webrev.02/index.html > > The extra changes here are to: > > * Introduce a new option DumpPrivateMappingsInCore to control the > dumping of the file backed private regions into the corefile. > > * Close the modules file before dumping core in os::abort(). > Currently, there is a small bug > (https://bugs.openjdk.java.net/browse/JDK-8215026) which prevents the > closure of the image file in unmapping the entire file. > > I plan to take up the unmapping of NIO MapMode.PRIVATE files as a > separate task (https://bugs.openjdk.java.net/browse/JDK-8215027) since > this seems a bit involved. > > Thanks a bunch, > Jini. > > On 11/12/2018 10:26 AM, Jini George wrote: >> Thank you very much, Chris, Kevin and Ioi for your comments! >> >> I will send another webrev with this change enabled under an opt-out >> flag, as you suggest, and would look at unmapping the JDK modules >> file and if possible, the NIO mapped files too in the signal handler. >> >> Thanks a bunch, >> Jini. >> >> On 11/9/2018 11:53 PM, Ioi Lam wrote: >>> Hi Jini, >>> >>> Thanks for investigating the size expansion issue. >>> >>> I agree that the size increase is worth it. Even when not using SA, >>> if we open the core file inside GDB, we cannot read certain sections >>> in the CDS archive (such as the RO section and strings sections). >>> That would make debugging difficult. So I am in favor of this change. >>> >>> For the JDK modules file, maybe we can unmap it in the signal >>> handler, before going ahead with the core dump? I think it's hardly >>> needed for debugging purposes. (Perhaps we can also do the same for >>> the NIO mapped files?) >>> >>> A opt-flag as suggested by Kevin is a good idea. >>> >>> Thanks >>> >>> - Ioi >>> >>> On 11/9/18 3:29 AM, Kevin Walls wrote: >>>> Hi Jini, >>>> >>>> Looks good to me.? It might be a significant increase in size of >>>> _some_ core files, but so many core files we see are much larger, >>>> in gigabytes++ of course, so the CDS data size should not be such a >>>> significant increase on (I think) most files. >>>> >>>> The flexibiity of always having the CDS data there is very >>>> significant.? A core file should ideally be usable, without >>>> additionally requiring the CDS archive from the machine. That >>>> additional human round-trip upload request on every transmitted >>>> core that needs investigating, seems like a less efficient route...). >>>> >>>> Is there an opt-out?? It's conditional on UseSharedSpaces but could >>>> there be a flag to disable, in case we see crashes with gigabytes >>>> of private mappings that we really don't want to retain (the user >>>> would have to know to set a flag, to disable the new coredump >>>> filter ahead of time). >>>> >>>> Thanks! >>>> Kevin >>>> >>>> >>>> On 29/10/2018 06:02, Jini George wrote: >>>>> Thank you very much, Ioi, for looking into this, and the >>>>> clarification offline. My bad, I had missed the earlier mail from >>>>> you. :-( My responses below. >>>>> >>>>> Yes, I had tested this on MacOS. The issue does not exist on MacOS >>>>> since the file backed private mmap()-ed regions get dumped into >>>>> the MacOS corefiles by default. >>>>> >>>>> The corefile sizes on Linux do increase due to this change. And >>>>> the increase would also include any file mapped using NIO with >>>>> MapMode.PRIVATE. The typical corefile size increase with this >>>>> change would include the following components at a high level: >>>>> >>>>> * Any NIO file mapping with MapMode.PRIVATE. >>>>> * Any file mmap()-ed by any native library with MAP_PRIVATE. >>>>> * The read only CDS regions (ro and od): Of the order of a few MB. >>>>> * The shared strings CDS region. (typically less than 1 MB). >>>>> * 2 MB per native shared library (regions with ---p permissions >>>>> mapped by the dynamic linker for better alignment and for keeping >>>>> libraries efficiently shareable). >>>>> * The JDK 'modules' file. (About 140 MB). >>>>> >>>>> So, without including any NIO mapping, I typically see around >>>>> 250-300 MB increase in the corefile sizes. I agree that the size >>>>> increase could be a cause for concern, but for FWIW, these >>>>> privately mapped files get dumped into the corefile for MacOS too. >>>>> And the corefile sizes for the same program on MacOS are way >>>>> larger (of the order of a few GB as against about 300 MB on Linux >>>>> (without the change)). >>>>> >>>>> The advantage of fixing this by modifying the coredump_filter v/s >>>>> doing it in SA (by reading in more sections of the shared archive >>>>> file) is that this would benefit other debuggers like gdb also. >>>>> (And reduces the dependence on having the shared archive file >>>>> being available at the time of debugging). If folks still think >>>>> this is a cause for concern, I could make modifications to fix >>>>> this by reading in the regions from the shared archive file in the >>>>> SA code. I also wonder if it is worth taking a relook at the >>>>> mapping types of the various CDS regions also. >>>>> >>>>> Thank you, >>>>> Jini. >>>>> >>>>> On 10/22/2018 10:27 AM, Ioi Lam wrote: >>>>>> Hi Jini, >>>>>> >>>>>> Did you see my earlier reply? I might have sent it out during the >>>>>> mail server outage days :-( >>>>>> >>>>>> http://openjdk.5641.n7.nabble.com/RFR-S-JDK-8200613-SA-jstack-throws-UnmappedAddressException-with-a-CDS-core-file-td352681.html#a353026 >>>>>> >>>>>> >>>>>> Here was my reply again: >>>>>> >>>>>>> Hi Jini, >>>>>>> >>>>>>> The changes looks good to me. >>>>>>> >>>>>>> Have you tested this on MacOS? CDS heap support is also enabled on >>>>>>> MacOS. See macros.hpp: >>>>>>> >>>>>>> ???? #if INCLUDE_CDS && INCLUDE_G1GC && defined(_LP64) && >>>>>>> !defined(_WINDOWS) >>>>>>> ???? #define INCLUDE_CDS_JAVA_HEAP 1 >>>>>>> >>>>>>> Also, besides CDS, do we know how often other files will be >>>>>>> mmaped with >>>>>>> MAP_PRIVATE? Will users get huge core files because CDS is >>>>>>> enabled? In >>>>>>> JDK 12, CDS will be enabled by default (see JDK-8202951), so all >>>>>>> users >>>>>>> will be affected by the following line: >>>>>>> >>>>>>> ?? if (UseSharedSpaces) { >>>>>>> ???? set_coredump_filter(FILE_BACKED_PVT_BIT); >>>>>>> ?? } >>>>>>> >>>>>>> Maybe you can run an big app such as Eclipse, trigger a core >>>>>>> dump, and >>>>>>> compare the size of the core file before/after this change? >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> - Ioi >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Thanks >>>>>> >>>>>> - Ioi >>>>>> >>>>>> >>>>>> On 10/21/18 8:58 PM, Jini George wrote: >>>>>>> Gentle reminder! >>>>>>> >>>>>>> Thanks, >>>>>>> - Jini >>>>>>> >>>>>>> On 10/9/2018 11:31 AM, Jini George wrote: >>>>>>>> Hello! >>>>>>>> >>>>>>>> [Including runtime-dev since the changes are in runtime code] >>>>>>>> >>>>>>>> Requesting reviews for: >>>>>>>> >>>>>>>> Webrev: http://cr.openjdk.java.net/~jgeorge/8200613/webrev.01/ >>>>>>>> BugID: https://bugs.openjdk.java.net/browse/JDK-8200613 >>>>>>>> >>>>>>>> Issue: jhsdb jstack would throw an UnmappedAddressException >>>>>>>> with a core file generated from a CDS enabled java process. >>>>>>>> This is seen only with Linux and with G1GC, while trying to >>>>>>>> read in data from the shared strings region (the closed archive >>>>>>>> heap space). This region (which is a file backed private memory >>>>>>>> region) is not dumped into the corefile for Linux. This, being >>>>>>>> a heap region (and therefore being a read-write region) is also >>>>>>>> not read in from the classes.jsa file in SA since only the read >>>>>>>> only regions are read in while processing the core file. (The >>>>>>>> expectation being that the read write regions are in the core >>>>>>>> file). >>>>>>>> >>>>>>>> Proposed solution: The proposed solution is to have the >>>>>>>> coredump_filter value corresponding to the CDS process to >>>>>>>> include bit 2 (file-backed private memory), so that the >>>>>>>> file-backed private memory region also gets dumped into the >>>>>>>> corefile. The proposed fix is in >>>>>>>> src/hotspot/os/linux/os_linux.cpp. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Jini. >>>>>>>> >>>>>> >>>> >>> From jini.george at oracle.com Mon Dec 10 06:44:09 2018 From: jini.george at oracle.com (Jini George) Date: Mon, 10 Dec 2018 12:14:09 +0530 Subject: RFR: (S): JDK-8200613: SA: jstack throws UnmappedAddressException with a CDS core file In-Reply-To: <2e0d7e6a-631f-6947-d49a-c57c4fe64691@oracle.com> References: <864997e5-2581-7ac9-44f7-76cc31c9ad93@oracle.com> <60d2e6db-c87b-9658-57cb-b03ca881a624@oracle.com> <9eb9041d-f984-0df4-a21e-4645966a62b8@oracle.com> <42d6d604-8df5-47c4-a748-6aa25bfc9a7e@oracle.com> <2dbef041-c75e-07f6-fafc-d18f2a95f455@oracle.com> <798a4d93-1638-8785-885f-fac0b21c2417@oracle.com> <2c6a2d67-5d36-07f1-4563-c1cd2ecdf9a3@oracle.com> <2e0d7e6a-631f-6947-d49a-c57c4fe64691@oracle.com> Message-ID: Thank you very much, Ioi! - Jini. On 12/10/2018 12:01 PM, Ioi Lam wrote: > Hi Jini, > > These changes look good to me. Thanks! > > - Ioi > > > On 12/7/18 11:22 AM, Jini George wrote: >> I have the revised webrev here: >> >> http://cr.openjdk.java.net/~jgeorge/8200613/webrev.02/index.html >> >> The extra changes here are to: >> >> * Introduce a new option DumpPrivateMappingsInCore to control the >> dumping of the file backed private regions into the corefile. >> >> * Close the modules file before dumping core in os::abort(). >> Currently, there is a small bug >> (https://bugs.openjdk.java.net/browse/JDK-8215026) which prevents the >> closure of the image file in unmapping the entire file. >> >> I plan to take up the unmapping of NIO MapMode.PRIVATE files as a >> separate task (https://bugs.openjdk.java.net/browse/JDK-8215027) since >> this seems a bit involved. >> >> Thanks a bunch, >> Jini. >> >> On 11/12/2018 10:26 AM, Jini George wrote: >>> Thank you very much, Chris, Kevin and Ioi for your comments! >>> >>> I will send another webrev with this change enabled under an opt-out >>> flag, as you suggest, and would look at unmapping the JDK modules >>> file and if possible, the NIO mapped files too in the signal handler. >>> >>> Thanks a bunch, >>> Jini. >>> >>> On 11/9/2018 11:53 PM, Ioi Lam wrote: >>>> Hi Jini, >>>> >>>> Thanks for investigating the size expansion issue. >>>> >>>> I agree that the size increase is worth it. Even when not using SA, >>>> if we open the core file inside GDB, we cannot read certain sections >>>> in the CDS archive (such as the RO section and strings sections). >>>> That would make debugging difficult. So I am in favor of this change. >>>> >>>> For the JDK modules file, maybe we can unmap it in the signal >>>> handler, before going ahead with the core dump? I think it's hardly >>>> needed for debugging purposes. (Perhaps we can also do the same for >>>> the NIO mapped files?) >>>> >>>> A opt-flag as suggested by Kevin is a good idea. >>>> >>>> Thanks >>>> >>>> - Ioi >>>> >>>> On 11/9/18 3:29 AM, Kevin Walls wrote: >>>>> Hi Jini, >>>>> >>>>> Looks good to me.? It might be a significant increase in size of >>>>> _some_ core files, but so many core files we see are much larger, >>>>> in gigabytes++ of course, so the CDS data size should not be such a >>>>> significant increase on (I think) most files. >>>>> >>>>> The flexibiity of always having the CDS data there is very >>>>> significant.? A core file should ideally be usable, without >>>>> additionally requiring the CDS archive from the machine. That >>>>> additional human round-trip upload request on every transmitted >>>>> core that needs investigating, seems like a less efficient route...). >>>>> >>>>> Is there an opt-out?? It's conditional on UseSharedSpaces but could >>>>> there be a flag to disable, in case we see crashes with gigabytes >>>>> of private mappings that we really don't want to retain (the user >>>>> would have to know to set a flag, to disable the new coredump >>>>> filter ahead of time). >>>>> >>>>> Thanks! >>>>> Kevin >>>>> >>>>> >>>>> On 29/10/2018 06:02, Jini George wrote: >>>>>> Thank you very much, Ioi, for looking into this, and the >>>>>> clarification offline. My bad, I had missed the earlier mail from >>>>>> you. :-( My responses below. >>>>>> >>>>>> Yes, I had tested this on MacOS. The issue does not exist on MacOS >>>>>> since the file backed private mmap()-ed regions get dumped into >>>>>> the MacOS corefiles by default. >>>>>> >>>>>> The corefile sizes on Linux do increase due to this change. And >>>>>> the increase would also include any file mapped using NIO with >>>>>> MapMode.PRIVATE. The typical corefile size increase with this >>>>>> change would include the following components at a high level: >>>>>> >>>>>> * Any NIO file mapping with MapMode.PRIVATE. >>>>>> * Any file mmap()-ed by any native library with MAP_PRIVATE. >>>>>> * The read only CDS regions (ro and od): Of the order of a few MB. >>>>>> * The shared strings CDS region. (typically less than 1 MB). >>>>>> * 2 MB per native shared library (regions with ---p permissions >>>>>> mapped by the dynamic linker for better alignment and for keeping >>>>>> libraries efficiently shareable). >>>>>> * The JDK 'modules' file. (About 140 MB). >>>>>> >>>>>> So, without including any NIO mapping, I typically see around >>>>>> 250-300 MB increase in the corefile sizes. I agree that the size >>>>>> increase could be a cause for concern, but for FWIW, these >>>>>> privately mapped files get dumped into the corefile for MacOS too. >>>>>> And the corefile sizes for the same program on MacOS are way >>>>>> larger (of the order of a few GB as against about 300 MB on Linux >>>>>> (without the change)). >>>>>> >>>>>> The advantage of fixing this by modifying the coredump_filter v/s >>>>>> doing it in SA (by reading in more sections of the shared archive >>>>>> file) is that this would benefit other debuggers like gdb also. >>>>>> (And reduces the dependence on having the shared archive file >>>>>> being available at the time of debugging). If folks still think >>>>>> this is a cause for concern, I could make modifications to fix >>>>>> this by reading in the regions from the shared archive file in the >>>>>> SA code. I also wonder if it is worth taking a relook at the >>>>>> mapping types of the various CDS regions also. >>>>>> >>>>>> Thank you, >>>>>> Jini. >>>>>> >>>>>> On 10/22/2018 10:27 AM, Ioi Lam wrote: >>>>>>> Hi Jini, >>>>>>> >>>>>>> Did you see my earlier reply? I might have sent it out during the >>>>>>> mail server outage days :-( >>>>>>> >>>>>>> http://openjdk.5641.n7.nabble.com/RFR-S-JDK-8200613-SA-jstack-throws-UnmappedAddressException-with-a-CDS-core-file-td352681.html#a353026 >>>>>>> >>>>>>> >>>>>>> Here was my reply again: >>>>>>> >>>>>>>> Hi Jini, >>>>>>>> >>>>>>>> The changes looks good to me. >>>>>>>> >>>>>>>> Have you tested this on MacOS? CDS heap support is also enabled on >>>>>>>> MacOS. See macros.hpp: >>>>>>>> >>>>>>>> ???? #if INCLUDE_CDS && INCLUDE_G1GC && defined(_LP64) && >>>>>>>> !defined(_WINDOWS) >>>>>>>> ???? #define INCLUDE_CDS_JAVA_HEAP 1 >>>>>>>> >>>>>>>> Also, besides CDS, do we know how often other files will be >>>>>>>> mmaped with >>>>>>>> MAP_PRIVATE? Will users get huge core files because CDS is >>>>>>>> enabled? In >>>>>>>> JDK 12, CDS will be enabled by default (see JDK-8202951), so all >>>>>>>> users >>>>>>>> will be affected by the following line: >>>>>>>> >>>>>>>> ?? if (UseSharedSpaces) { >>>>>>>> ???? set_coredump_filter(FILE_BACKED_PVT_BIT); >>>>>>>> ?? } >>>>>>>> >>>>>>>> Maybe you can run an big app such as Eclipse, trigger a core >>>>>>>> dump, and >>>>>>>> compare the size of the core file before/after this change? >>>>>>>> >>>>>>>> Thanks >>>>>>>> >>>>>>>> - Ioi >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> - Ioi >>>>>>> >>>>>>> >>>>>>> On 10/21/18 8:58 PM, Jini George wrote: >>>>>>>> Gentle reminder! >>>>>>>> >>>>>>>> Thanks, >>>>>>>> - Jini >>>>>>>> >>>>>>>> On 10/9/2018 11:31 AM, Jini George wrote: >>>>>>>>> Hello! >>>>>>>>> >>>>>>>>> [Including runtime-dev since the changes are in runtime code] >>>>>>>>> >>>>>>>>> Requesting reviews for: >>>>>>>>> >>>>>>>>> Webrev: http://cr.openjdk.java.net/~jgeorge/8200613/webrev.01/ >>>>>>>>> BugID: https://bugs.openjdk.java.net/browse/JDK-8200613 >>>>>>>>> >>>>>>>>> Issue: jhsdb jstack would throw an UnmappedAddressException >>>>>>>>> with a core file generated from a CDS enabled java process. >>>>>>>>> This is seen only with Linux and with G1GC, while trying to >>>>>>>>> read in data from the shared strings region (the closed archive >>>>>>>>> heap space). This region (which is a file backed private memory >>>>>>>>> region) is not dumped into the corefile for Linux. This, being >>>>>>>>> a heap region (and therefore being a read-write region) is also >>>>>>>>> not read in from the classes.jsa file in SA since only the read >>>>>>>>> only regions are read in while processing the core file. (The >>>>>>>>> expectation being that the read write regions are in the core >>>>>>>>> file). >>>>>>>>> >>>>>>>>> Proposed solution: The proposed solution is to have the >>>>>>>>> coredump_filter value corresponding to the CDS process to >>>>>>>>> include bit 2 (file-backed private memory), so that the >>>>>>>>> file-backed private memory region also gets dumped into the >>>>>>>>> corefile. The proposed fix is in >>>>>>>>> src/hotspot/os/linux/os_linux.cpp. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Jini. >>>>>>>>> >>>>>>> >>>>> >>>> > From leonid.mesnik at oracle.com Mon Dec 10 07:21:17 2018 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Sun, 9 Dec 2018 23:21:17 -0800 Subject: RFR(XS): 8215042: Move serviceability/sa tests from tier1 to tier3. In-Reply-To: <5c9d711a-4a77-b649-1385-b4e6e2060bd8@oracle.com> References: <1BA254CB-E62C-4E94-ACA3-129414EF0E19@oracle.com> <44995fad-d870-027c-66bb-9990d91fb723@oracle.com> <5c9d711a-4a77-b649-1385-b4e6e2060bd8@oracle.com> Message-ID: David, Jini I understand your concerns. But the original idea of tiered testing is that tier1 failures are treated as urgent issues and to resolve. [1] Here is list of test failures for 1000 runs of tier1 tests in Mach5. (I am not able to provide a link here) Please note that all SA tests are excluded on Solaris and MacosX already. 1 compiler/aot/calls/fromAot/AotInvokeSpecial2AotTest.java 2 serviceability/sa/ClhsdbFindPC.java 3 serviceability/sa/TestPrintMdo.java 4 serviceability/sa/ClhsdbJstack.java 5 serviceability/sa/ClhsdbJdis.java 6 compiler/c2/Test8004741.java 7 runtime/handshake/HandshakeWalkSuspendExitTest.java 8 runtime/handshake/HandshakeWalkSuspendExitTest.java 9 compiler/aot/calls/fromAot/AotInvokeVirtual2AotTest.java 10 runtime/handshake/HandshakeWalkExitTest.java 11 runtime/handshake/HandshakeWalkSuspendExitTest.java 12 serviceability/sa/ClhsdbRegionDetailsScanOopsForG1.java 13 serviceability/sa/ClhsdbRegionDetailsScanOopsForG1.java 14 compiler/aot/calls/fromAot/AotInvokeVirtual2AotTest.java The failures in of 'runtime/handshake/' are relatively caused by https://bugs.openjdk.java.net/browse/JDK-8214174 but should be also fixed/excluded. SA tests are also unstable and there are no plans to fix them soon. So it means that we are going to have tier1 tests unstable for a long time. The possible way to make tier1 more stable would be to run only some very basic sanity SA tests in tier1. Might be to develop new sanity test which have some failover for existing SA bugs. Leonid [1] http://mail.openjdk.java.net/pipermail/jdk9-dev/2015-March/001991.html > On Dec 9, 2018, at 7:19 PM, Jini George wrote: > > Hi Leonid, > > I agree with David. I am also concerned about us not detecting SA breakages (which could happen along with hotspot changes) soon enough. (Which was the primary reason to get these tests in). > > Thank you, > Jini. > > On 12/8/2018 4:48 PM, David Holmes wrote: >> Hi Leonid, >> My concern here, if we care about keeping the SA operational, is that in tier3 these tests will not be covered by the jdk/submit testing process. >> David >> On 8/12/2018 3:53 pm, Leonid Mesnik wrote: >>> Hi >>> >>> Could you please review following fix which moves SA tests from tier1 to tier3. There are some bugs which cause intermittent failures of any test. SA tests fail intermittently are not stable enough for tier1. >>> However failures are not very frequent. Also I don't think that putting all test in Problemlist.txt is very good idea because it left SA without any testing at all. >>> So now all SA tests which are included in hotspot_tier3_runtime group. >>> >>> webrev: http://cr.openjdk.java.net/~lmesnik/8215042/webrev.00/ >>> bug: https://bugs.openjdk.java.net/browse/JDK-8215042 >>> >>> Leonid -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Mon Dec 10 07:29:16 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 10 Dec 2018 17:29:16 +1000 Subject: RFR(XS): 8215042: Move serviceability/sa tests from tier1 to tier3. In-Reply-To: References: <1BA254CB-E62C-4E94-ACA3-129414EF0E19@oracle.com> <44995fad-d870-027c-66bb-9990d91fb723@oracle.com> <5c9d711a-4a77-b649-1385-b4e6e2060bd8@oracle.com> Message-ID: <8c006fa4-d8bd-36ca-174e-05b240de6a74@oracle.com> Hi Leonid, If there are specific unstable SA tests then they should be problem-listed and/or excluded from tier1 and put in a later tier. Lets not remove all SA testing in tier1 just because of a handful of issues. Thanks, David On 10/12/2018 5:21 pm, Leonid Mesnik wrote: > David, Jini > > I understand your concerns. But the original idea of tiered testing is > that tier1 failures are treated as urgent issues and to resolve. [1] > > Here is list of test failures for 1000 runs of tier1 tests in Mach5. (I > am not able to provide a link here) Please note that all SA tests are > excluded on Solaris and MacosX already. > > 1 compiler/aot/calls/fromAot/AotInvokeSpecial2AotTest.java > 2 serviceability/sa/ClhsdbFindPC.java > 3 serviceability/sa/TestPrintMdo.java > 4 serviceability/sa/ClhsdbJstack.java > 5 serviceability/sa/ClhsdbJdis.java > 6 compiler/c2/Test8004741.java > 7 runtime/handshake/HandshakeWalkSuspendExitTest.java > 8 runtime/handshake/HandshakeWalkSuspendExitTest.java > 9 compiler/aot/calls/fromAot/AotInvokeVirtual2AotTest.java > 10 runtime/handshake/HandshakeWalkExitTest.java > 11 runtime/handshake/HandshakeWalkSuspendExitTest.java > 12 serviceability/sa/ClhsdbRegionDetailsScanOopsForG1.java > 13 serviceability/sa/ClhsdbRegionDetailsScanOopsForG1.java > 14 compiler/aot/calls/fromAot/AotInvokeVirtual2AotTest.java > > The failures in of 'runtime/handshake/' are relatively caused by > https://bugs.openjdk.java.net/browse/JDK-8214174?but should be also > fixed/excluded. SA tests are also unstable and there are no plans to fix > them soon. > So it means that we are going to have tier1 tests unstable for a long time. > > The possible way to make tier1 more stable would be to run only some > very basic sanity SA tests in tier1. ?Might be to develop new sanity > test which have some failover for existing SA bugs. > > Leonid > > [1] http://mail.openjdk.java.net/pipermail/jdk9-dev/2015-March/001991.html > >> On Dec 9, 2018, at 7:19 PM, Jini George > > wrote: >> >> Hi Leonid, >> >> I agree with David. I am also concerned about us not detecting SA >> breakages (which could happen along with hotspot changes) soon enough. >> (Which was the primary reason to get these tests in). >> >> Thank you, >> Jini. >> >> On 12/8/2018 4:48 PM, David Holmes wrote: >>> Hi Leonid, >>> My concern here, if we care about keeping the SA operational, is that >>> in tier3 these tests will not be covered by the jdk/submit testing >>> process. >>> David >>> On 8/12/2018 3:53 pm, Leonid Mesnik wrote: >>>> Hi >>>> >>>> Could you please review following fix which moves SA tests from >>>> tier1 to tier3. There are some bugs which cause intermittent >>>> failures of any test. SA tests fail intermittently are not stable >>>> enough for tier1. >>>> However failures are not very frequent. Also I don't think that >>>> putting all test in Problemlist.txt is very good idea because it >>>> left SA without any testing at all. >>>> So now all SA tests which are included in hotspot_tier3_runtime group. >>>> >>>> webrev: http://cr.openjdk.java.net/~lmesnik/8215042/webrev.00/ >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8215042 >>>> >>>> Leonid > From jini.george at oracle.com Mon Dec 10 07:38:06 2018 From: jini.george at oracle.com (Jini George) Date: Mon, 10 Dec 2018 13:08:06 +0530 Subject: RFR(XS): 8215042: Move serviceability/sa tests from tier1 to tier3. In-Reply-To: References: <1BA254CB-E62C-4E94-ACA3-129414EF0E19@oracle.com> <44995fad-d870-027c-66bb-9990d91fb723@oracle.com> <5c9d711a-4a77-b649-1385-b4e6e2060bd8@oracle.com> Message-ID: Hi Leonid, Looks like all the SA failures here are all due to https://bugs.openjdk.java.net/browse/JDK-8202884. Do let me know if I am mistaken. We will work on fixing that issue faster. Thanks, Jini. On 12/10/2018 12:51 PM, Leonid Mesnik wrote: > David, Jini > > I understand your concerns. But the original idea of tiered testing is > that tier1 failures are treated as urgent issues and to resolve. [1] > > Here is list of test failures for 1000 runs of tier1 tests in Mach5. (I > am not able to provide a link here) Please note that all SA tests are > excluded on Solaris and MacosX already. > > 1 compiler/aot/calls/fromAot/AotInvokeSpecial2AotTest.java > 2 serviceability/sa/ClhsdbFindPC.java > 3 serviceability/sa/TestPrintMdo.java > 4 serviceability/sa/ClhsdbJstack.java > 5 serviceability/sa/ClhsdbJdis.java > 6 compiler/c2/Test8004741.java > 7 runtime/handshake/HandshakeWalkSuspendExitTest.java > 8 runtime/handshake/HandshakeWalkSuspendExitTest.java > 9 compiler/aot/calls/fromAot/AotInvokeVirtual2AotTest.java > 10 runtime/handshake/HandshakeWalkExitTest.java > 11 runtime/handshake/HandshakeWalkSuspendExitTest.java > 12 serviceability/sa/ClhsdbRegionDetailsScanOopsForG1.java > 13 serviceability/sa/ClhsdbRegionDetailsScanOopsForG1.java > 14 compiler/aot/calls/fromAot/AotInvokeVirtual2AotTest.java > > The failures in of 'runtime/handshake/' are relatively caused by > https://bugs.openjdk.java.net/browse/JDK-8214174?but should be also > fixed/excluded. SA tests are also unstable and there are no plans to fix > them soon. > So it means that we are going to have tier1 tests unstable for a long time. > > The possible way to make tier1 more stable would be to run only some > very basic sanity SA tests in tier1. ?Might be to develop new sanity > test which have some failover for existing SA bugs. > > Leonid > > [1] http://mail.openjdk.java.net/pipermail/jdk9-dev/2015-March/001991.html > >> On Dec 9, 2018, at 7:19 PM, Jini George > > wrote: >> >> Hi Leonid, >> >> I agree with David. I am also concerned about us not detecting SA >> breakages (which could happen along with hotspot changes) soon enough. >> (Which was the primary reason to get these tests in). >> >> Thank you, >> Jini. >> >> On 12/8/2018 4:48 PM, David Holmes wrote: >>> Hi Leonid, >>> My concern here, if we care about keeping the SA operational, is that >>> in tier3 these tests will not be covered by the jdk/submit testing >>> process. >>> David >>> On 8/12/2018 3:53 pm, Leonid Mesnik wrote: >>>> Hi >>>> >>>> Could you please review following fix which moves SA tests from >>>> tier1 to tier3. There are some bugs which cause intermittent >>>> failures of any test. SA tests fail intermittently are not stable >>>> enough for tier1. >>>> However failures are not very frequent. Also I don't think that >>>> putting all test in Problemlist.txt is very good idea because it >>>> left SA without any testing at all. >>>> So now all SA tests which are included in hotspot_tier3_runtime group. >>>> >>>> webrev: http://cr.openjdk.java.net/~lmesnik/8215042/webrev.00/ >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8215042 >>>> >>>> Leonid > From leonid.mesnik at oracle.com Mon Dec 10 07:42:52 2018 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Sun, 9 Dec 2018 23:42:52 -0800 Subject: RFR(XS): 8215042: Move serviceability/sa tests from tier1 to tier3. In-Reply-To: References: <1BA254CB-E62C-4E94-ACA3-129414EF0E19@oracle.com> <44995fad-d870-027c-66bb-9990d91fb723@oracle.com> <5c9d711a-4a77-b649-1385-b4e6e2060bd8@oracle.com> Message-ID: <7CF3E99B-C715-4652-90B2-77B46A14AC0D@oracle.com> Hi Here is summary of such "non test-specific" bugs: Following bugs affect all (or mostly all) tests: JDK-8202884 SA: Attach/detach might fail on Linux if debugee application create/destroy threads during attaching JDK-8204994 SA might fail to attach to process with "Windbg Error: WaitForEvent failed" JDK-8197591 Tests failing with App waiting timeout JDK-8203364 Some serviceability/sa/ tests intermittently fail with java.io.IOException: LingeredApp terminated with non-zero exit code 3 and the similar bug for Solaris-SPARC JDK-8193639 tests failing intermittently with Error attaching to process: Can't create thread_db agent! SA might fail connecting to any process because of JDK-8202884 , JDK-8204994 . Also all tests relying on LingeredApp might affected by JDK-8197591 , JDK-8203364 . You have seen a lot of failures caused by https://bugs.openjdk.java.net/browse/JDK-8202884 because it is a Linux-specific bug which is often reproduced on multi-core host. Leonid > On Dec 9, 2018, at 11:38 PM, Jini George wrote: > > Hi Leonid, > > Looks like all the SA failures here are all due to https://bugs.openjdk.java.net/browse/JDK-8202884. Do let me know if I am mistaken. We will work on fixing that issue faster. > > Thanks, > Jini. > > On 12/10/2018 12:51 PM, Leonid Mesnik wrote: >> David, Jini >> I understand your concerns. But the original idea of tiered testing is that tier1 failures are treated as urgent issues and to resolve. [1] >> Here is list of test failures for 1000 runs of tier1 tests in Mach5. (I am not able to provide a link here) Please note that all SA tests are excluded on Solaris and MacosX already. >> 1 compiler/aot/calls/fromAot/AotInvokeSpecial2AotTest.java >> 2 serviceability/sa/ClhsdbFindPC.java >> 3 serviceability/sa/TestPrintMdo.java >> 4 serviceability/sa/ClhsdbJstack.java >> 5 serviceability/sa/ClhsdbJdis.java >> 6 compiler/c2/Test8004741.java >> 7 runtime/handshake/HandshakeWalkSuspendExitTest.java >> 8 runtime/handshake/HandshakeWalkSuspendExitTest.java >> 9 compiler/aot/calls/fromAot/AotInvokeVirtual2AotTest.java >> 10 runtime/handshake/HandshakeWalkExitTest.java >> 11 runtime/handshake/HandshakeWalkSuspendExitTest.java >> 12 serviceability/sa/ClhsdbRegionDetailsScanOopsForG1.java >> 13 serviceability/sa/ClhsdbRegionDetailsScanOopsForG1.java >> 14 compiler/aot/calls/fromAot/AotInvokeVirtual2AotTest.java >> The failures in of 'runtime/handshake/' are relatively caused by https://bugs.openjdk.java.net/browse/JDK-8214174 but should be also fixed/excluded. SA tests are also unstable and there are no plans to fix them soon. >> So it means that we are going to have tier1 tests unstable for a long time. >> The possible way to make tier1 more stable would be to run only some very basic sanity SA tests in tier1. Might be to develop new sanity test which have some failover for existing SA bugs. >> Leonid >> [1] http://mail.openjdk.java.net/pipermail/jdk9-dev/2015-March/001991.html >>> On Dec 9, 2018, at 7:19 PM, Jini George > wrote: >>> >>> Hi Leonid, >>> >>> I agree with David. I am also concerned about us not detecting SA breakages (which could happen along with hotspot changes) soon enough. (Which was the primary reason to get these tests in). >>> >>> Thank you, >>> Jini. >>> >>> On 12/8/2018 4:48 PM, David Holmes wrote: >>>> Hi Leonid, >>>> My concern here, if we care about keeping the SA operational, is that in tier3 these tests will not be covered by the jdk/submit testing process. >>>> David >>>> On 8/12/2018 3:53 pm, Leonid Mesnik wrote: >>>>> Hi >>>>> >>>>> Could you please review following fix which moves SA tests from tier1 to tier3. There are some bugs which cause intermittent failures of any test. SA tests fail intermittently are not stable enough for tier1. >>>>> However failures are not very frequent. Also I don't think that putting all test in Problemlist.txt is very good idea because it left SA without any testing at all. >>>>> So now all SA tests which are included in hotspot_tier3_runtime group. >>>>> >>>>> webrev: http://cr.openjdk.java.net/~lmesnik/8215042/webrev.00/ >>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8215042 >>>>> >>>>> Leonid -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Mon Dec 10 07:55:16 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 10 Dec 2018 17:55:16 +1000 Subject: RFR: JDK-8210106: sun/tools/jps/TestJps.java timed out In-Reply-To: References: <5C0923D1.8080705@oracle.com> <6697b4a2-b0d7-8a52-7549-23d6433114cc@oracle.com> <5C0A6E8D.7010108@oracle.com> Message-ID: <41d1ffce-202a-2c65-61b4-27815e347a28@oracle.com> Hi Dan, > Gary saw a longest time value of 280 seconds in his testing with > fastdebug bits. The default timeout value is 120 seconds with a > default timeout factor of 4 in Mach5 gives a total timeout of > 480 seconds (8 minutes). With Gary's new default timeout of > 360 seconds, we'll have a total timeout value of 1440 seconds > (24 minutes). So based on that it seems Gary's observations are not relevant to what has been seen in mach5 because at 280 seconds we should never have been timing out! I guess these changes won't hurt. Thanks, David On 8/12/2018 1:12 am, Daniel D. Daugherty wrote: > On 12/7/18 7:58 AM, Gary Adams wrote: >> On 12/6/18, 7:52 PM, David Holmes wrote: >>> Hi Gary, >>> >>> On 6/12/2018 11:27 pm, Gary Adams wrote: >>>> On a local linux-x64-debug build this test consistently runs in less >>>> than 40 seconds. >>>> On the mach5 testing machines there was a large fluctuation in the >>>> time to complete this test. >>>> >>>> Since the test runs jps with different combinations of arguments, >>>> the total >>>> test time is dependent on the number of processes running java >>>> programs. >>>> Since the mach5 test infrastructure runs java programs I have seen a 3X >>>> in the amount of output the test produces compared to local test >>>> runs. >>>> >>>> I've run the test several hundred times through mach5 on the slower >>>> platforms >>>> and then examined the test logs using a 3X setting from the default >>>> 120 second >>>> jtreg timeout. The slowest reported elapse time from the test logs >>>> showed >>>> 280 seconds to complete. >>>> >>>> To improve the reliability of the test to complete, I'd like to >>>> increase the >>>> timeout to 360 seconds. >>>> >>>> ?? Issue: https://bugs.openjdk.java.net/browse/JDK-8210106 >>>> >>>> Proposed fix: >>>> >>>> diff --git a/test/jdk/sun/tools/jps/TestJps.java >>>> b/test/jdk/sun/tools/jps/TestJps.java >>>> --- a/test/jdk/sun/tools/jps/TestJps.java >>>> +++ b/test/jdk/sun/tools/jps/TestJps.java >>>> @@ -27,7 +27,7 @@ >>>> ?? * @modules jdk.jartool/sun.tools.jar >>>> ?? * @build LingeredAppForJps >>>> ?? * @build LingeredApp >>>> - * @run main/othervm TestJps >>>> + * @run main/othervm/timeout=360 TestJps >>>> ?? */ >>> >>> Doesn't that then get scaled by the timeout factor resulting in a >>> much longer timeout than the 360 seconds you intended? >>> >>> For other timeout adjustments the needed time has been divided by the >>> timeout factor to get the actual intended timeout. >> >> This bug was filed fairly recently in Aug 2018. >> At that time the timeout and timeout factor were not sufficient >> to avoid the test failing. >> >> The mach5 timeout factors were adjusted recently, so this test may >> no longer be an issue. >> >> If that is true, then we could simply close this bug as "cannot >> reproduce". >> An argument could be made that the change in timeout factor may be >> responsible for fixing a lot more of the intermittent bugs and that they >> should be closed in a similar manner. >> >> Historically, we could say this particular bug should have had timeouts >> reassessed when the infrastructure switched from jprt to mach5 testing >> where there were more visible Java processes running. >> >> Using a higher explicit timeout will not make the test run any longer >> than it needs. It will simply allow the test to not be terminated sooner >> in a hung test scenario. >> >> What is your preference for this particular issue: >> ?? - increase the explicit timeout >> ?? - close as cannot reproduce attributed to the timeout factor >> adjustments >> >> What would you recommend going forward for other similar issues: >> ?? - determine a new explicit timeout >> ?? - close if no timeout failures have been observed since the timeout >> factor >> ????? was raised > > General guidance that I've been giving folks is that you bump the > default timeout when you see timeouts with 'release' bits with > a timeout factor of '1'. The default timeout factor in Mach5 is 4 > so we run both 'release' and 'fastdebug' bits with that timeout > factor. That means our total timeout value for 'release' bits is > probably a little long and our total timeout value for 'fastdebug' > bits is probably just right for a concurrency factor of 12. > > On my personal servers I use the following timeout factors: > > ??????? release) > ??????????? TIMEOUT_FACTOR="4" > ??????????? ;; > ??????? fastdebug) > ??????????? TIMEOUT_FACTOR="6" > ??????????? ;; > ??????? slowdebug) > ??????????? TIMEOUT_FACTOR="12" > ??????????? ;; > > However, I run with a concurrency factor of 16. I rarely see > timeout failures. > > All that is well and good, but a 'jps' test is a different > beast since it is affected by factors outside of most tests. > Since the number of java processes running on the system can > affect the test, I'm okay with using a default timeout of '360' > for this test as a guard against increases in load or... > > Gary saw a longest time value of 280 seconds in his testing with > fastdebug bits. The default timeout value is 120 seconds with a > default timeout factor of 4 in Mach5 gives a total timeout of > 480 seconds (8 minutes). With Gary's new default timeout of > 360 seconds, we'll have a total timeout value of 1440 seconds > (24 minutes). > > Before the recent changes to Mach5, the default timeout factor > was 10 so we had a total timeout value of 1200 seconds (20 > minutes). Gary's change only bumps the total timeout value > by 4 minutes from what we had back in August... when this bug > was filed... unfortunately, the log for that sighting is gone > so all we know is that the test timed out after 20 minutes > > > Timeout refired 1200 times > > We know that JTREG timeout handling kicks in at timeout expiration, > but without the log we don't know how much longer than 20 minutes > the test ran before JTREG killed it. 24 minutes might do it, but... > > Dan > > >> >> >>> >>> Cheers, >>> David >>> >> >> > From zanglin5 at jd.com Mon Dec 10 08:18:42 2018 From: zanglin5 at jd.com (=?gb2312?B?6rDB1Q==?=) Date: Mon, 10 Dec 2018 08:18:42 +0000 Subject: optimize KlassInfoTable size to power of 2 Message-ID: Our preliminary measurement show the patch introduced about ~5% speed up for jmap ?Chisto when iterate ~13GB committed heap. ( with ?CXX:+UseG1GC ?CXmx180g configured) Just FYI. Cheers, Lin From: ???? Sent: Monday, December 10, 2018 2:24 PM To: serviceability-dev at openjdk.java.net Subject: optimize KlassInfoTable size to power of 2 Dear All, I am investigating JMap utility for large heap (~200GB) and found that the current KlassInfoTable??s _num_buckets size(20011) may not be optimal. My observation is that when ??jmap histo?? iterating objects , the object??s klass is identified and then hash idx in KlassInfoTable??s buckets[] is calculated by mod of _num_bucktes, which would issue a heavy instruction idiv on x86 platform. It means for every object scanned, a idiv instruction is issued, which lag the performance espically when there are large number of objects in heap. Hence if the _num_buckets can be changed to a pow of 2, (e.g. 65536) the idiv can be replaced with a faster instruction such as shl (left bit shift), And I have prepared a patch for this change. My question is that why 20011 is used now? is there any special reason? And is there any potential problem if I change the value to 65536, or 32768? Thanks! BRs, Lin -------------- next part -------------- An HTML attachment was scrubbed... URL: From kevin.walls at oracle.com Mon Dec 10 09:39:43 2018 From: kevin.walls at oracle.com (Kevin Walls) Date: Mon, 10 Dec 2018 09:39:43 +0000 Subject: RFR: (S): JDK-8200613: SA: jstack throws UnmappedAddressException with a CDS core file In-Reply-To: References: <864997e5-2581-7ac9-44f7-76cc31c9ad93@oracle.com> <60d2e6db-c87b-9658-57cb-b03ca881a624@oracle.com> <9eb9041d-f984-0df4-a21e-4645966a62b8@oracle.com> <42d6d604-8df5-47c4-a748-6aa25bfc9a7e@oracle.com> <2dbef041-c75e-07f6-fafc-d18f2a95f455@oracle.com> <798a4d93-1638-8785-885f-fac0b21c2417@oracle.com> <2c6a2d67-5d36-07f1-4563-c1cd2ecdf9a3@oracle.com> <2e0d7e6a-631f-6947-d49a-c57c4fe64691@oracle.com> Message-ID: <0b90b17b-67b3-fa39-b08d-8bdff77916ca@oracle.com> Thanks for adding the flag Jini, all looks good to me too, Thanks Kevin On 10/12/2018 06:44, Jini George wrote: > Thank you very much, Ioi! > > - Jini. > > On 12/10/2018 12:01 PM, Ioi Lam wrote: >> Hi Jini, >> >> These changes look good to me. Thanks! >> >> - Ioi >> >> >> On 12/7/18 11:22 AM, Jini George wrote: >>> I have the revised webrev here: >>> >>> http://cr.openjdk.java.net/~jgeorge/8200613/webrev.02/index.html >>> >>> The extra changes here are to: >>> >>> * Introduce a new option DumpPrivateMappingsInCore to control the >>> dumping of the file backed private regions into the corefile. >>> >>> * Close the modules file before dumping core in os::abort(). >>> Currently, there is a small bug >>> (https://bugs.openjdk.java.net/browse/JDK-8215026) which prevents >>> the closure of the image file in unmapping the entire file. >>> >>> I plan to take up the unmapping of NIO MapMode.PRIVATE files as a >>> separate task (https://bugs.openjdk.java.net/browse/JDK-8215027) >>> since this seems a bit involved. >>> >>> Thanks a bunch, >>> Jini. >>> >>> On 11/12/2018 10:26 AM, Jini George wrote: >>>> Thank you very much, Chris, Kevin and Ioi for your comments! >>>> >>>> I will send another webrev with this change enabled under an >>>> opt-out flag, as you suggest, and would look at unmapping the JDK >>>> modules file and if possible, the NIO mapped files too in the >>>> signal handler. >>>> >>>> Thanks a bunch, >>>> Jini. >>>> >>>> On 11/9/2018 11:53 PM, Ioi Lam wrote: >>>>> Hi Jini, >>>>> >>>>> Thanks for investigating the size expansion issue. >>>>> >>>>> I agree that the size increase is worth it. Even when not using >>>>> SA, if we open the core file inside GDB, we cannot read certain >>>>> sections in the CDS archive (such as the RO section and strings >>>>> sections). That would make debugging difficult. So I am in favor >>>>> of this change. >>>>> >>>>> For the JDK modules file, maybe we can unmap it in the signal >>>>> handler, before going ahead with the core dump? I think it's >>>>> hardly needed for debugging purposes. (Perhaps we can also do the >>>>> same for the NIO mapped files?) >>>>> >>>>> A opt-flag as suggested by Kevin is a good idea. >>>>> >>>>> Thanks >>>>> >>>>> - Ioi >>>>> >>>>> On 11/9/18 3:29 AM, Kevin Walls wrote: >>>>>> Hi Jini, >>>>>> >>>>>> Looks good to me.? It might be a significant increase in size of >>>>>> _some_ core files, but so many core files we see are much larger, >>>>>> in gigabytes++ of course, so the CDS data size should not be such >>>>>> a significant increase on (I think) most files. >>>>>> >>>>>> The flexibiity of always having the CDS data there is very >>>>>> significant.? A core file should ideally be usable, without >>>>>> additionally requiring the CDS archive from the machine. That >>>>>> additional human round-trip upload request on every transmitted >>>>>> core that needs investigating, seems like a less efficient >>>>>> route...). >>>>>> >>>>>> Is there an opt-out?? It's conditional on UseSharedSpaces but >>>>>> could there be a flag to disable, in case we see crashes with >>>>>> gigabytes of private mappings that we really don't want to retain >>>>>> (the user would have to know to set a flag, to disable the new >>>>>> coredump filter ahead of time). >>>>>> >>>>>> Thanks! >>>>>> Kevin >>>>>> >>>>>> >>>>>> On 29/10/2018 06:02, Jini George wrote: >>>>>>> Thank you very much, Ioi, for looking into this, and the >>>>>>> clarification offline. My bad, I had missed the earlier mail >>>>>>> from you. :-( My responses below. >>>>>>> >>>>>>> Yes, I had tested this on MacOS. The issue does not exist on >>>>>>> MacOS since the file backed private mmap()-ed regions get dumped >>>>>>> into the MacOS corefiles by default. >>>>>>> >>>>>>> The corefile sizes on Linux do increase due to this change. And >>>>>>> the increase would also include any file mapped using NIO with >>>>>>> MapMode.PRIVATE. The typical corefile size increase with this >>>>>>> change would include the following components at a high level: >>>>>>> >>>>>>> * Any NIO file mapping with MapMode.PRIVATE. >>>>>>> * Any file mmap()-ed by any native library with MAP_PRIVATE. >>>>>>> * The read only CDS regions (ro and od): Of the order of a few MB. >>>>>>> * The shared strings CDS region. (typically less than 1 MB). >>>>>>> * 2 MB per native shared library (regions with ---p permissions >>>>>>> mapped by the dynamic linker for better alignment and for >>>>>>> keeping libraries efficiently shareable). >>>>>>> * The JDK 'modules' file. (About 140 MB). >>>>>>> >>>>>>> So, without including any NIO mapping, I typically see around >>>>>>> 250-300 MB increase in the corefile sizes. I agree that the size >>>>>>> increase could be a cause for concern, but for FWIW, these >>>>>>> privately mapped files get dumped into the corefile for MacOS >>>>>>> too. And the corefile sizes for the same program on MacOS are >>>>>>> way larger (of the order of a few GB as against about 300 MB on >>>>>>> Linux (without the change)). >>>>>>> >>>>>>> The advantage of fixing this by modifying the coredump_filter >>>>>>> v/s doing it in SA (by reading in more sections of the shared >>>>>>> archive file) is that this would benefit other debuggers like >>>>>>> gdb also. (And reduces the dependence on having the shared >>>>>>> archive file being available at the time of debugging). If folks >>>>>>> still think this is a cause for concern, I could make >>>>>>> modifications to fix this by reading in the regions from the >>>>>>> shared archive file in the SA code. I also wonder if it is worth >>>>>>> taking a relook at the mapping types of the various CDS regions >>>>>>> also. >>>>>>> >>>>>>> Thank you, >>>>>>> Jini. >>>>>>> >>>>>>> On 10/22/2018 10:27 AM, Ioi Lam wrote: >>>>>>>> Hi Jini, >>>>>>>> >>>>>>>> Did you see my earlier reply? I might have sent it out during >>>>>>>> the mail server outage days :-( >>>>>>>> >>>>>>>> http://openjdk.5641.n7.nabble.com/RFR-S-JDK-8200613-SA-jstack-throws-UnmappedAddressException-with-a-CDS-core-file-td352681.html#a353026 >>>>>>>> >>>>>>>> >>>>>>>> Here was my reply again: >>>>>>>> >>>>>>>>> Hi Jini, >>>>>>>>> >>>>>>>>> The changes looks good to me. >>>>>>>>> >>>>>>>>> Have you tested this on MacOS? CDS heap support is also >>>>>>>>> enabled on >>>>>>>>> MacOS. See macros.hpp: >>>>>>>>> >>>>>>>>> ???? #if INCLUDE_CDS && INCLUDE_G1GC && defined(_LP64) && >>>>>>>>> !defined(_WINDOWS) >>>>>>>>> ???? #define INCLUDE_CDS_JAVA_HEAP 1 >>>>>>>>> >>>>>>>>> Also, besides CDS, do we know how often other files will be >>>>>>>>> mmaped with >>>>>>>>> MAP_PRIVATE? Will users get huge core files because CDS is >>>>>>>>> enabled? In >>>>>>>>> JDK 12, CDS will be enabled by default (see JDK-8202951), so >>>>>>>>> all users >>>>>>>>> will be affected by the following line: >>>>>>>>> >>>>>>>>> ?? if (UseSharedSpaces) { >>>>>>>>> ???? set_coredump_filter(FILE_BACKED_PVT_BIT); >>>>>>>>> ?? } >>>>>>>>> >>>>>>>>> Maybe you can run an big app such as Eclipse, trigger a core >>>>>>>>> dump, and >>>>>>>>> compare the size of the core file before/after this change? >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> >>>>>>>>> - Ioi >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Thanks >>>>>>>> >>>>>>>> - Ioi >>>>>>>> >>>>>>>> >>>>>>>> On 10/21/18 8:58 PM, Jini George wrote: >>>>>>>>> Gentle reminder! >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> - Jini >>>>>>>>> >>>>>>>>> On 10/9/2018 11:31 AM, Jini George wrote: >>>>>>>>>> Hello! >>>>>>>>>> >>>>>>>>>> [Including runtime-dev since the changes are in runtime code] >>>>>>>>>> >>>>>>>>>> Requesting reviews for: >>>>>>>>>> >>>>>>>>>> Webrev: http://cr.openjdk.java.net/~jgeorge/8200613/webrev.01/ >>>>>>>>>> BugID: https://bugs.openjdk.java.net/browse/JDK-8200613 >>>>>>>>>> >>>>>>>>>> Issue: jhsdb jstack would throw an UnmappedAddressException >>>>>>>>>> with a core file generated from a CDS enabled java process. >>>>>>>>>> This is seen only with Linux and with G1GC, while trying to >>>>>>>>>> read in data from the shared strings region (the closed >>>>>>>>>> archive heap space). This region (which is a file backed >>>>>>>>>> private memory region) is not dumped into the corefile for >>>>>>>>>> Linux. This, being a heap region (and therefore being a >>>>>>>>>> read-write region) is also not read in from the classes.jsa >>>>>>>>>> file in SA since only the read only regions are read in while >>>>>>>>>> processing the core file. (The expectation being that the >>>>>>>>>> read write regions are in the core file). >>>>>>>>>> >>>>>>>>>> Proposed solution: The proposed solution is to have the >>>>>>>>>> coredump_filter value corresponding to the CDS process to >>>>>>>>>> include bit 2 (file-backed private memory), so that the >>>>>>>>>> file-backed private memory region also gets dumped into the >>>>>>>>>> corefile. The proposed fix is in >>>>>>>>>> src/hotspot/os/linux/os_linux.cpp. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Jini. >>>>>>>>>> >>>>>>>> >>>>>> >>>>> >> From magnus.ihse.bursie at oracle.com Mon Dec 10 10:45:46 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Mon, 10 Dec 2018 11:45:46 +0100 Subject: [PATCH] JDK-8214122 Prevent name mangling of jdwpTransport_OnLoad in Windows 32-bit DLL name decoration In-Reply-To: <9d060b4d-bc2f-d402-d767-e5962df45aa6@oracle.com> References: <2DD922C6-EFB1-4891-90F0-3378DEA91B1C@gmail.com> <07840796-f1c4-e077-7b0b-cd5c1a880ec1@oracle.com> <00e58330-26f0-4401-f911-24fda0127bcc@oracle.com> <34d2b504-c4eb-6df3-3bd1-e23b9c22ea6a@oracle.com> <01b80c3f-8f71-26ec-304e-a7b245924bd5@oracle.com> <17e7873f-f2f4-dc02-82a6-2d7eb1565ca2@oracle.com> <5e032661-b70c-c6ec-2700-120589300c9e@oracle.com> <5A54BA77-B240-472A-A4B7-535C3A30A20F@gmail.com> <9d060b4d-bc2f-d402-d767-e5962df45aa6@oracle.com> Message-ID: <4daf4b17-dc29-8814-24a9-9b08cda2b5dc@oracle.com> On 2018-12-07 21:24, Alexey Ivanov wrote: > Hi Ali, > > On 06/12/2018 22:49, Ali ?nce wrote: >> Hi Magnus, Alexey, >> >> I believe we won?t be able to get further opinions from >> serviceability-dev. > > Unfortunately, no one has replied as of now. > Have you found any issues with jdwpTransport_OnLoad after removing > JNICALL? > >> Andrew Luo suggested using a similar mechanism as is used for jvm.dll >> by using symbol name files mapped by platform (files are under >> make/hotspot/symbols and interestingly windows is almost the only >> platform for which a symbol file doesn?t exist). > > Andrew says the .def files are auto-generated for Windows. There's a > set of shared functions. > JVM cannot change the calling convention, jvm.dll is the public > interface to JVM. > >> Another issue, again opened against AdoptOpenJDK 32-bit builds is >> somehow related with the same problem - but this time it is >> jimage.dll depending on zip.dll expecting the ZIP_InflateFully >> function to be decorated with JNICALL - whereas it was removed >> earlier from the implementation and the runtime image just fails with >> access violation just because jimage source code still declared it >> with JNICALL (https://github.com/AdoptOpenJDK/openjdk-build/issues/763). > > The usage of ZIP_InflateFully from zip.dll by jimage.dll was > overlooked during work on > https://bugs.openjdk.java.net/browse/JDK-8201226. > > I can take care of it. It might be worthwhile to scan the entire code base for JNICALL and verify that we have no more mismatches. In general, JNICALL should be preserved on all officially supported, exported interfaces. It need not be present on interfaces used only internally, and my current thinking is that it would be better if it were removed on all internal interfaces. But at the very least, it should be consistently specificed where exported and imported. (Any misses here is probably due to duplicate declarations, instead of properly including header files, so that's the correct way to resolve any problems found...) /Magnus > (I could not build 32 bit Windows. It seem to have overcome the > problem by adding --disable-warnings-as-errors option to configure.) > > However, the report says the resulting image crashes in 64 bit windows > too which shouldn't be affected by JNICALL mismatch. > >> I?ve also searched for GetProcAddress calls across the source code - >> and I think it?s important to work on the consistency of JNICALL >> usages across the whole source code. > > There are many places where libraries are loaded dynamically or a > function may be unavailable on older versions of the platform. > Have you identified any other place where functions from JDK DLLs are > looked up by names? > >> Another noteworthy thing I?ve noticed is there are still JNICALL >> modifiers for example in >> https://github.com/AdoptOpenJDK/openjdk-jdk11u/blob/master/src/jdk.crypto.cryptoki/windows/native/libj2pkcs11/j2secmod_md.c#L49 - >> which I guess will also be reported as broken since these functions >> are again name decorated. > > This is a JNI method. It should use JNICALL modifier. If it wasn't > found, the class sun.security.pkcs11.Secmod would fail to load. I > guess JVM handles name mangling somehow for native method implementation. > > Such functions were never explicitly exported using mapfiles or > /export options on Windows, at least in the client area. For Linux and > Solaris, adding or removing a native method required editing mapfiles. > >> If we?re still determined to remove JNICALL, what about just removing >> __stdcall from JNICALL declaration at >> https://github.com/AdoptOpenJDK/openjdk-jdk11u/blob/master/src/java.base/windows/native/include/jni_md.h#L31 ? Wouldn?t >> that be a more consistent move? > > We can't do that. > Implementation of Java native methods must use __stdcall calling > convention. > >> >> Regards, >> >> Ali >> >>> On 22 Nov 2018, at 17:29, Magnus Ihse Bursie >>> >> > wrote: >>> >>> I think we are in complete agreement. What's missing is some expert >>> opinion from serviceability-dev if there is any problem with >>> changing the signature. I'd preferably have that. >>> >>> If no one knows, I'd say, change it, and see if we still pass the >>> relevant tests, and if so, ship it. >>> >>> /Magnus >>> >>> 22 nov. 2018 kl. 18:04 skrev Alexey Ivanov >> >: >>> >>>> >>>> On 21/11/2018 12:16, Magnus Ihse Bursie wrote: >>>>> >>>>> On 2018-11-20 16:49, Alexey Ivanov wrote: >>>>> >>>>>> Hi Ali, Magnus, >>>>>> >>>>>> I absolutely agree this change has to be reviewed by someone from >>>>>> serviceability. >>>>>> >>>>>> There are several options: >>>>>> >>>>>> 1. Add -export:jdwpTransport_OnLoad to LDFLAGS for Windows >>>>>> http://mail.openjdk.java.net/pipermail/build-dev/2018-November/023935.html >>>>>> so it partially reverts the changes from >>>>>> https://bugs.openjdk.java.net/browse/JDK-8200178 >>>>>> >>>>>> 2. Remove JNICALL (__stdcall) modifier, eventually changing the >>>>>> calling convention to __cdecl. >>>>>> http://mail.openjdk.java.net/pipermail/build-dev/2018-November/023969.html >>>>>> >>>>>> 3. Use inline /export option via #pragma: >>>>>> #pragma comment(linker, "/export:jdwpTransport_OnLoad= >>>>>> _jdwpTransport_OnLoad at 16") >>>>>> as referenced in >>>>>> https://docs.microsoft.com/en-ie/cpp/cpp/dllexport-dllimport?view=vs-2017 >>>>>> https://docs.microsoft.com/en-ie/cpp/build/reference/exports?view=vs-2017 >>>>>> >>>>>> The third options still needs to be tested to make sure it does >>>>>> not break other platforms builds. >>>>> I'm not fond of either solution 1 or 3. I really think the >>>>> signature should be made correctly at the point of definition in >>>>> the source code. >>>> >>>> That is why I proposed removing JNICALL (__stdcall) from the >>>> function declaration as we had done for libzip, libjimage [1] and >>>> mlib_image [2]. >>>> >>>> Microsoft recommends using __stdcall for DLLs, at the same time it >>>> decorates the function name making it harder to export it by its >>>> plain name. >>>> >>>> >>>> By removing JNICALL / __stdcall, we make the function use __cdecl >>>> calling convention. The difference between them is that with >>>> __stdcall the callee cleans the stack whereas with __cdecl the >>>> caller cleans the stack. >>>> >>>> It shouldn't be a problem as long as all function declarations use >>>> the same calling convention, which, I believe, is the case here. >>>> >>>>> We've spent some considerable effort of getting rid of the /export >>>>> flags for Windows (and map files for unix), and I don't want to >>>>> see them creep back in. Note that option 3 is just option 1 in >>>>> disguise. The only single good thing about it is that it allows >>>>> you to put the compiler option next to the signature declaration >>>>> in the source code. >>>> >>>> At least in this case, the option will be near the function body >>>> definition. It will be easier to update it if the signature changes. >>>> >>>> If we are to use __stdcall, it seems to be the only way to export >>>> the undecorated name. >>>> >>>>> The same goes for def files, as suggested by Ali. It's just yet >>>>> another version of option 1 in disguise/map files. >>>> >>>> If option 2 is rejected, I'm for using option 3. If option 3 cannot >>>> be used too, I'm for option 1. >>>> I think we should not fall back to def files in any case. And >>>> Makefile will have to include the reference to the def file, so >>>> it's even worse than option 1. >>>> >>>> >>>> Regards, >>>> Alexey >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8201226 >>>> [2] https://bugs.openjdk.java.net/browse/JDK-8202476 >>>>> >>>>> /Magnus >>>>> >>>>>> >>>>>> >>>>>> Regards, >>>>>> Alexey >>>>>> >>>>>> On 19/11/2018 12:19, Magnus Ihse Bursie wrote: >>>>>>> Hi Ali, >>>>>>> >>>>>>> From a build perspective this change looks OK. I'm not aware of >>>>>>> the finer details on the OnLoad mechanism, like what calling >>>>>>> convention is to be used. So maybe this is a no-go from that view. >>>>>>> >>>>>>> I'm cc:ing servicability so they can have a look at it. >>>>>>> >>>>>>> /Magnus >>>>>>> >>>>>>> On 2018-11-18 13:07, Ali ?nce wrote: >>>>>>>> Hi Alexey, >>>>>>>> >>>>>>>> Below is a new patch content that removes JNICALL modifiers from the exported functions of interest. This results in exported functions not to be name decorated and use __cdecl calling convention. >>>>>>>> >>>>>>>> Do you have any more suggestions, or would you please guide me on next steps? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Ali >>>>>>>> >>>>>>>> # HG changeset patch >>>>>>>> # User Ali Ince >>>>>>>> # Date 1542542415 0 >>>>>>>> # Sun Nov 18 12:00:15 2018 +0000 >>>>>>>> # Node ID a41df44e13e1b761f4b3f05a17ed18953067ae39 >>>>>>>> # Parent 16d5ec7dbbb49ef3f969e34be870e3f917ea89ff >>>>>>>> Remove JNICALL modifiers to prevent name decoration on 32-bit windows builds >>>>>>>> >>>>>>>> diff -r 16d5ec7dbbb4 -r a41df44e13e1 src/jdk.jdi/share/native/libdt_shmem/shmemBack.c >>>>>>>> --- a/src/jdk.jdi/share/native/libdt_shmem/shmemBack.c Tue Aug 14 14:28:23 2018 +0200 >>>>>>>> +++ b/src/jdk.jdi/share/native/libdt_shmem/shmemBack.c Sun Nov 18 12:00:15 2018 +0000 >>>>>>>> @@ -339,7 +339,7 @@ >>>>>>>> return JDWPTRANSPORT_ERROR_NONE; >>>>>>>> } >>>>>>>> >>>>>>>> -JNIEXPORT jint JNICALL >>>>>>>> +JNIEXPORT jint >>>>>>>> jdwpTransport_OnLoad(JavaVM *vm, jdwpTransportCallback* cbTablePtr, >>>>>>>> jint version, jdwpTransportEnv** result) >>>>>>>> { >>>>>>>> diff -r 16d5ec7dbbb4 -r a41df44e13e1 src/jdk.jdwp.agent/share/native/include/jdwpTransport.h >>>>>>>> --- a/src/jdk.jdwp.agent/share/native/include/jdwpTransport.h Tue Aug 14 14:28:23 2018 +0200 >>>>>>>> +++ b/src/jdk.jdwp.agent/share/native/include/jdwpTransport.h Sun Nov 18 12:00:15 2018 +0000 >>>>>>>> @@ -140,7 +140,7 @@ >>>>>>>> void (*free)(void *buffer); /* Call this for all deallocations */ >>>>>>>> } jdwpTransportCallback; >>>>>>>> >>>>>>>> -typedef jint (JNICALL *jdwpTransport_OnLoad_t)(JavaVM *jvm, >>>>>>>> +typedef jint (*jdwpTransport_OnLoad_t)(JavaVM *jvm, >>>>>>>> jdwpTransportCallback *callback, >>>>>>>> jint version, >>>>>>>> jdwpTransportEnv** env); >>>>>>>> diff -r 16d5ec7dbbb4 -r a41df44e13e1 src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c >>>>>>>> --- a/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c Tue Aug 14 14:28:23 2018 +0200 >>>>>>>> +++ b/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c Sun Nov 18 12:00:15 2018 +0000 >>>>>>>> @@ -1019,7 +1019,7 @@ >>>>>>>> return JDWPTRANSPORT_ERROR_NONE; >>>>>>>> } >>>>>>>> >>>>>>>> -JNIEXPORT jint JNICALL >>>>>>>> +JNIEXPORT jint >>>>>>>> jdwpTransport_OnLoad(JavaVM *vm, jdwpTransportCallback* cbTablePtr, >>>>>>>> jint version, jdwpTransportEnv** env) >>>>>>>> { >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> On 16 Nov 2018, at 23:03, Alexey Ivanov wrote: >>>>>>>>> >>>>>>>>> Hi Ali, >>>>>>>>> >>>>>>>>> It's exactly what I referred to. >>>>>>>>> >>>>>>>>> Yes, it does change the calling convention. >>>>>>>>> Yet it's not a (big) problem because both parties will use the same calling convention. >>>>>>>>> >>>>>>>>> Using a DLL from another build will not be possible. But it's not a good idea to do so anyway. >>>>>>>>> >>>>>>>>> >>>>>>>>> Mapfiles and export options have been removed byhttps://bugs.openjdk.java.net/browse/JDK-8200178 to simplify managing exported functions. So that to add or remove an exported function, you don't have modify makefiles and/or mapfiles. You just edit the source files. >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Alexey >>>>>>>>> >>>>>>>>> On 16/11/2018 22:42, Ali ?nce wrote: >>>>>>>>>> Hi Alexey, >>>>>>>>>> >>>>>>>>>> Thanks for your reply. >>>>>>>>>> >>>>>>>>>> I will definitely give it a try though I?m not sure if that?ll be a breaking change. This is because JNICALL modifier is defined to be __stdcall on windows which is both specified on jdwpTransport_OnLoad exported functions both for dt_socket.dll and dt_shmem.dll. >>>>>>>>>> >>>>>>>>>> The __stdcall is ignored on x64 platforms and also name decoration is not applied (https://docs.microsoft.com/en-gb/cpp/build/reference/decorated-names?view=vs-2017). I believe that?s why we don?t experience any problems on x64 builds. >>>>>>>>>> >>>>>>>>>> Removing JNICALL (thus __stdcall) modifier (apparently only applies to x86 builds) will result in the calling convention to be changed, and thus will change how parameters are ordered and also the stack cleanup rules. I think this may result in problems in programs that are designed to load shared libraries and its exported functions at runtime (not bound by link time which probably won?t cause any issues - assuming that they use the shared function signature) - as inhttps://github.com/AdoptOpenJDK/openjdk-jdk11u/blob/5f01925b80ed851b133ee26fbcb07026ac04149e/src/jdk.jdwp.agent/share/native/libjdwp/transport.c#L99. >>>>>>>>>> >>>>>>>>>> Any thoughts? >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> >>>>>>>>>> Ali >>>>>>>>>> >>>>>>>>>>> On 15 Nov 2018, at 22:14, Alexey Ivanov wrote: >>>>>>>>>>> >>>>>>>>>>> Hi Ali, >>>>>>>>>>> >>>>>>>>>>> Can the issue be resolved by using different call modifiers on the affected functions? >>>>>>>>>>> >>>>>>>>>>> Some build / link issues were resolved by adding / removing JNICALL modifier, see >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8201226 >>>>>>>>>>> http://mail.openjdk.java.net/pipermail/build-dev/2018-April/021553.html >>>>>>>>>>> >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8202476 >>>>>>>>>>> http://mail.openjdk.java.net/pipermail/build-dev/2018-May/021913.html >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> Alexey >>>>>>>>>>> >>>>>>>>>>> On 15/11/2018 21:43, Ali ?nce wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> An issue (https://github.com/AdoptOpenJDK/openjdk-build/issues/709) raised against AdoptOpenJDK jdk11 32-bit windows builds led us to the name decoration problem on built 32-bit dlls - specifically dt_socket.dll and dt_shmem.dll. Whereas 64-bit MSVC builds don?t apply any name decorations on exported functions, 32-bit builds apply them - which requires either def files or /export flags to be specified on the linker arguments. >>>>>>>>>>>> >>>>>>>>>>>> Although the expected linker arguments were present on previous jdk makefiles, they were removed in jdk11 makefiles. >>>>>>>>>>>> >>>>>>>>>>>> Below is the proposed patch, which can also be browsed athttps://github.com/AdoptOpenJDK/openjdk-jdk11u/pull/1. >>>>>>>>>>>> >>>>>>>>>>>> Would you please have a look and let me know of any points I may have missed (I don?t have any background information about why the export flags were removed in jdk11)? >>>>>>>>>>>> >>>>>>>>>>>> Regards, >>>>>>>>>>>> >>>>>>>>>>>> Ali >>>>>>>>>>>> >>>>>>>>>>>> diff --git a/make/lib/Lib-jdk.jdi.gmk b/make/lib/Lib-jdk.jdi.gmk >>>>>>>>>>>> index 197b95c2e2..ac6ebf5591 100644 >>>>>>>>>>>> --- a/make/lib/Lib-jdk.jdi.gmk >>>>>>>>>>>> +++ b/make/lib/Lib-jdk.jdi.gmk >>>>>>>>>>>> @@ -37,6 +37,7 @@ ifeq ($(OPENJDK_TARGET_OS), windows) >>>>>>>>>>>> jdk.jdwp.agent:include \ >>>>>>>>>>>> jdk.jdwp.agent:libjdwp/export, \ >>>>>>>>>>>> LDFLAGS := $(LDFLAGS_JDKLIB), \ >>>>>>>>>>>> + LDFLAGS_windows := -export:jdwpTransport_OnLoad, \ >>>>>>>>>>>> LIBS := $(JDKLIB_LIBS), \ >>>>>>>>>>>> )) >>>>>>>>>>>> >>>>>>>>>>>> diff --git a/make/lib/Lib-jdk.jdwp.agent.gmk b/make/lib/Lib-jdk.jdwp.agent.gmk >>>>>>>>>>>> index 0bc93e0d35..0382e55325 100644 >>>>>>>>>>>> --- a/make/lib/Lib-jdk.jdwp.agent.gmk >>>>>>>>>>>> +++ b/make/lib/Lib-jdk.jdwp.agent.gmk >>>>>>>>>>>> @@ -37,6 +37,7 @@ $(eval $(call SetupJdkLibrary, BUILD_LIBDT_SOCKET, \ >>>>>>>>>>>> libjdwp/export, \ >>>>>>>>>>>> LDFLAGS := $(LDFLAGS_JDKLIB) \ >>>>>>>>>>>> $(call SET_SHARED_LIBRARY_ORIGIN), \ >>>>>>>>>>>> + LDFLAGS_windows := -export:jdwpTransport_OnLoad, \ >>>>>>>>>>>> LIBS_linux := -lpthread, \ >>>>>>>>>>>> LIBS_solaris := -lnsl -lsocket, \ >>>>>>>>>>>> LIBS_windows := $(JDKLIB_LIBS) ws2_32.lib, \ >>>>>>> >>>>>> >>>>> >>>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jini.george at oracle.com Mon Dec 10 11:51:11 2018 From: jini.george at oracle.com (Jini George) Date: Mon, 10 Dec 2018 17:21:11 +0530 Subject: RFR: (S): JDK-8200613: SA: jstack throws UnmappedAddressException with a CDS core file In-Reply-To: <0b90b17b-67b3-fa39-b08d-8bdff77916ca@oracle.com> References: <864997e5-2581-7ac9-44f7-76cc31c9ad93@oracle.com> <60d2e6db-c87b-9658-57cb-b03ca881a624@oracle.com> <9eb9041d-f984-0df4-a21e-4645966a62b8@oracle.com> <42d6d604-8df5-47c4-a748-6aa25bfc9a7e@oracle.com> <2dbef041-c75e-07f6-fafc-d18f2a95f455@oracle.com> <798a4d93-1638-8785-885f-fac0b21c2417@oracle.com> <2c6a2d67-5d36-07f1-4563-c1cd2ecdf9a3@oracle.com> <2e0d7e6a-631f-6947-d49a-c57c4fe64691@oracle.com> <0b90b17b-67b3-fa39-b08d-8bdff77916ca@oracle.com> Message-ID: <94b02b28-c1ff-622d-01f6-da8a5f6a7c81@oracle.com> Thank you very much, Kevin! - Jini. On 12/10/2018 3:09 PM, Kevin Walls wrote: > Thanks for adding the flag Jini, all looks good to me too, > > Thanks > Kevin > > > On 10/12/2018 06:44, Jini George wrote: >> Thank you very much, Ioi! >> >> - Jini. >> >> On 12/10/2018 12:01 PM, Ioi Lam wrote: >>> Hi Jini, >>> >>> These changes look good to me. Thanks! >>> >>> - Ioi >>> >>> >>> On 12/7/18 11:22 AM, Jini George wrote: >>>> I have the revised webrev here: >>>> >>>> http://cr.openjdk.java.net/~jgeorge/8200613/webrev.02/index.html >>>> >>>> The extra changes here are to: >>>> >>>> * Introduce a new option DumpPrivateMappingsInCore to control the >>>> dumping of the file backed private regions into the corefile. >>>> >>>> * Close the modules file before dumping core in os::abort(). >>>> Currently, there is a small bug >>>> (https://bugs.openjdk.java.net/browse/JDK-8215026) which prevents >>>> the closure of the image file in unmapping the entire file. >>>> >>>> I plan to take up the unmapping of NIO MapMode.PRIVATE files as a >>>> separate task (https://bugs.openjdk.java.net/browse/JDK-8215027) >>>> since this seems a bit involved. >>>> >>>> Thanks a bunch, >>>> Jini. >>>> >>>> On 11/12/2018 10:26 AM, Jini George wrote: >>>>> Thank you very much, Chris, Kevin and Ioi for your comments! >>>>> >>>>> I will send another webrev with this change enabled under an >>>>> opt-out flag, as you suggest, and would look at unmapping the JDK >>>>> modules file and if possible, the NIO mapped files too in the >>>>> signal handler. >>>>> >>>>> Thanks a bunch, >>>>> Jini. >>>>> >>>>> On 11/9/2018 11:53 PM, Ioi Lam wrote: >>>>>> Hi Jini, >>>>>> >>>>>> Thanks for investigating the size expansion issue. >>>>>> >>>>>> I agree that the size increase is worth it. Even when not using >>>>>> SA, if we open the core file inside GDB, we cannot read certain >>>>>> sections in the CDS archive (such as the RO section and strings >>>>>> sections). That would make debugging difficult. So I am in favor >>>>>> of this change. >>>>>> >>>>>> For the JDK modules file, maybe we can unmap it in the signal >>>>>> handler, before going ahead with the core dump? I think it's >>>>>> hardly needed for debugging purposes. (Perhaps we can also do the >>>>>> same for the NIO mapped files?) >>>>>> >>>>>> A opt-flag as suggested by Kevin is a good idea. >>>>>> >>>>>> Thanks >>>>>> >>>>>> - Ioi >>>>>> >>>>>> On 11/9/18 3:29 AM, Kevin Walls wrote: >>>>>>> Hi Jini, >>>>>>> >>>>>>> Looks good to me.? It might be a significant increase in size of >>>>>>> _some_ core files, but so many core files we see are much larger, >>>>>>> in gigabytes++ of course, so the CDS data size should not be such >>>>>>> a significant increase on (I think) most files. >>>>>>> >>>>>>> The flexibiity of always having the CDS data there is very >>>>>>> significant.? A core file should ideally be usable, without >>>>>>> additionally requiring the CDS archive from the machine. That >>>>>>> additional human round-trip upload request on every transmitted >>>>>>> core that needs investigating, seems like a less efficient >>>>>>> route...). >>>>>>> >>>>>>> Is there an opt-out?? It's conditional on UseSharedSpaces but >>>>>>> could there be a flag to disable, in case we see crashes with >>>>>>> gigabytes of private mappings that we really don't want to retain >>>>>>> (the user would have to know to set a flag, to disable the new >>>>>>> coredump filter ahead of time). >>>>>>> >>>>>>> Thanks! >>>>>>> Kevin >>>>>>> >>>>>>> >>>>>>> On 29/10/2018 06:02, Jini George wrote: >>>>>>>> Thank you very much, Ioi, for looking into this, and the >>>>>>>> clarification offline. My bad, I had missed the earlier mail >>>>>>>> from you. :-( My responses below. >>>>>>>> >>>>>>>> Yes, I had tested this on MacOS. The issue does not exist on >>>>>>>> MacOS since the file backed private mmap()-ed regions get dumped >>>>>>>> into the MacOS corefiles by default. >>>>>>>> >>>>>>>> The corefile sizes on Linux do increase due to this change. And >>>>>>>> the increase would also include any file mapped using NIO with >>>>>>>> MapMode.PRIVATE. The typical corefile size increase with this >>>>>>>> change would include the following components at a high level: >>>>>>>> >>>>>>>> * Any NIO file mapping with MapMode.PRIVATE. >>>>>>>> * Any file mmap()-ed by any native library with MAP_PRIVATE. >>>>>>>> * The read only CDS regions (ro and od): Of the order of a few MB. >>>>>>>> * The shared strings CDS region. (typically less than 1 MB). >>>>>>>> * 2 MB per native shared library (regions with ---p permissions >>>>>>>> mapped by the dynamic linker for better alignment and for >>>>>>>> keeping libraries efficiently shareable). >>>>>>>> * The JDK 'modules' file. (About 140 MB). >>>>>>>> >>>>>>>> So, without including any NIO mapping, I typically see around >>>>>>>> 250-300 MB increase in the corefile sizes. I agree that the size >>>>>>>> increase could be a cause for concern, but for FWIW, these >>>>>>>> privately mapped files get dumped into the corefile for MacOS >>>>>>>> too. And the corefile sizes for the same program on MacOS are >>>>>>>> way larger (of the order of a few GB as against about 300 MB on >>>>>>>> Linux (without the change)). >>>>>>>> >>>>>>>> The advantage of fixing this by modifying the coredump_filter >>>>>>>> v/s doing it in SA (by reading in more sections of the shared >>>>>>>> archive file) is that this would benefit other debuggers like >>>>>>>> gdb also. (And reduces the dependence on having the shared >>>>>>>> archive file being available at the time of debugging). If folks >>>>>>>> still think this is a cause for concern, I could make >>>>>>>> modifications to fix this by reading in the regions from the >>>>>>>> shared archive file in the SA code. I also wonder if it is worth >>>>>>>> taking a relook at the mapping types of the various CDS regions >>>>>>>> also. >>>>>>>> >>>>>>>> Thank you, >>>>>>>> Jini. >>>>>>>> >>>>>>>> On 10/22/2018 10:27 AM, Ioi Lam wrote: >>>>>>>>> Hi Jini, >>>>>>>>> >>>>>>>>> Did you see my earlier reply? I might have sent it out during >>>>>>>>> the mail server outage days :-( >>>>>>>>> >>>>>>>>> http://openjdk.5641.n7.nabble.com/RFR-S-JDK-8200613-SA-jstack-throws-UnmappedAddressException-with-a-CDS-core-file-td352681.html#a353026 >>>>>>>>> >>>>>>>>> >>>>>>>>> Here was my reply again: >>>>>>>>> >>>>>>>>>> Hi Jini, >>>>>>>>>> >>>>>>>>>> The changes looks good to me. >>>>>>>>>> >>>>>>>>>> Have you tested this on MacOS? CDS heap support is also >>>>>>>>>> enabled on >>>>>>>>>> MacOS. See macros.hpp: >>>>>>>>>> >>>>>>>>>> ???? #if INCLUDE_CDS && INCLUDE_G1GC && defined(_LP64) && >>>>>>>>>> !defined(_WINDOWS) >>>>>>>>>> ???? #define INCLUDE_CDS_JAVA_HEAP 1 >>>>>>>>>> >>>>>>>>>> Also, besides CDS, do we know how often other files will be >>>>>>>>>> mmaped with >>>>>>>>>> MAP_PRIVATE? Will users get huge core files because CDS is >>>>>>>>>> enabled? In >>>>>>>>>> JDK 12, CDS will be enabled by default (see JDK-8202951), so >>>>>>>>>> all users >>>>>>>>>> will be affected by the following line: >>>>>>>>>> >>>>>>>>>> ?? if (UseSharedSpaces) { >>>>>>>>>> ???? set_coredump_filter(FILE_BACKED_PVT_BIT); >>>>>>>>>> ?? } >>>>>>>>>> >>>>>>>>>> Maybe you can run an big app such as Eclipse, trigger a core >>>>>>>>>> dump, and >>>>>>>>>> compare the size of the core file before/after this change? >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> >>>>>>>>>> - Ioi >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> >>>>>>>>> - Ioi >>>>>>>>> >>>>>>>>> >>>>>>>>> On 10/21/18 8:58 PM, Jini George wrote: >>>>>>>>>> Gentle reminder! >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> - Jini >>>>>>>>>> >>>>>>>>>> On 10/9/2018 11:31 AM, Jini George wrote: >>>>>>>>>>> Hello! >>>>>>>>>>> >>>>>>>>>>> [Including runtime-dev since the changes are in runtime code] >>>>>>>>>>> >>>>>>>>>>> Requesting reviews for: >>>>>>>>>>> >>>>>>>>>>> Webrev: http://cr.openjdk.java.net/~jgeorge/8200613/webrev.01/ >>>>>>>>>>> BugID: https://bugs.openjdk.java.net/browse/JDK-8200613 >>>>>>>>>>> >>>>>>>>>>> Issue: jhsdb jstack would throw an UnmappedAddressException >>>>>>>>>>> with a core file generated from a CDS enabled java process. >>>>>>>>>>> This is seen only with Linux and with G1GC, while trying to >>>>>>>>>>> read in data from the shared strings region (the closed >>>>>>>>>>> archive heap space). This region (which is a file backed >>>>>>>>>>> private memory region) is not dumped into the corefile for >>>>>>>>>>> Linux. This, being a heap region (and therefore being a >>>>>>>>>>> read-write region) is also not read in from the classes.jsa >>>>>>>>>>> file in SA since only the read only regions are read in while >>>>>>>>>>> processing the core file. (The expectation being that the >>>>>>>>>>> read write regions are in the core file). >>>>>>>>>>> >>>>>>>>>>> Proposed solution: The proposed solution is to have the >>>>>>>>>>> coredump_filter value corresponding to the CDS process to >>>>>>>>>>> include bit 2 (file-backed private memory), so that the >>>>>>>>>>> file-backed private memory region also gets dumped into the >>>>>>>>>>> corefile. The proposed fix is in >>>>>>>>>>> src/hotspot/os/linux/os_linux.cpp. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Jini. >>>>>>>>>>> >>>>>>>>> >>>>>>> >>>>>> >>> > From aph at redhat.com Mon Dec 10 12:29:39 2018 From: aph at redhat.com (Andrew Haley) Date: Mon, 10 Dec 2018 12:29:39 +0000 Subject: optimize KlassInfoTable size to power of 2 In-Reply-To: <140bac18565c48be88313f7cc1d24bf8@jd.com> References: <140bac18565c48be88313f7cc1d24bf8@jd.com> Message-ID: On 12/10/18 6:24 AM, ?? wrote: > My observation is that when ?jmap histo? iterating objects , the object?s klass > is identified and then hash idx in KlassInfoTable?s buckets[] is calculated by mod > of _num_bucktes, which would issue a heavy instruction idiv on x86 platform. It > means for every object scanned, a idiv instruction is issued, which lag the performance espically when > there are large number of objects in heap. I'm surprised that GCC uses idiv in this case: it has an optimization for constant modulo which doesn't use division. There is sometimes an advantage for using hash tables of prime size. Is it that the size of the table is not a constant that GCC can see? -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From merkel05 at gmail.com Mon Dec 10 12:57:39 2018 From: merkel05 at gmail.com (Sergei Ustimenko) Date: Mon, 10 Dec 2018 13:57:39 +0100 Subject: Fwd: [PATCH] JDK-8025886: Replace [[ and == bash extensions in tests In-Reply-To: References: Message-ID: Hi, I've recently been working on bug https://bugs.openjdk.java.net/browse/JDK-8214052 that is mainly about cleaning up the [[ and == bash extensions from the tests. I've discovered, that there's a leftover after hg forest consolidation. GeneratePropertyPassword.sh script still uses beforementioned bash extensions. That has been fixed within the scope of this bug: https://bugs.openjdk.java.net/browse/JDK-8025886 Though fix didn't get to the consolidated forest. The fix is trivial and just reapplies the patch from the JBS' comments. Change has been tested with `dash` and works fine. Please find the patch inlined below. That would also be great if anyone could sponsor it. diff --git a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh --- a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh +++ b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh @@ -34,12 +34,13 @@ OS=`uname -s` UMASK=`umask` -if [[ $OS == CYGWIN_NT* ]] ; then +case $OS in +CYGWIN_NT*) OS="Windows_NT" if [ -z "$SystemRoot" ] ; then - SystemRoot=`cygpath $SYSTEMROOT` + SystemRoot=$SYSTEMROOT fi -fi +esac case $OS in SunOS | Linux | Darwin | AIX ) Thanks, Sergei -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.schmelter at sap.com Mon Dec 10 13:54:48 2018 From: ralf.schmelter at sap.com (Schmelter, Ralf) Date: Mon, 10 Dec 2018 13:54:48 +0000 Subject: RFR (M) 8214892: Delayed starting of debugging via jcmd In-Reply-To: References: <01fdfc8e58a246c18d5909fc7df313b7@sap.com> <940d8a26-d5b3-0d5b-ccb8-a71d4d17efdc@oracle.com> <6283122f12f041caa56f768f8005920b@sap.com> Message-ID: <87976683317e41028e146fcc000e5638@sap.com> Hi Christoph, > I have thought about a more generic name for the option rather than > "onjcmd". What about "standby=y". That would indicate that the > debugging agent is on standby and can be triggered by whatever > means. What do others think? I'm not sure making the name more generic is the right move, when it will probably be enabled via the jcmd in >95% of the cases. > I'd prefer if you write out either "Debugging has been started." > or "Debugging is already active.". I think this would make it more > clear for the user of the feature what actually happened. OK, I've changed that. > A better wording would be: > "Starts up the Java debugging if the jdwp agentlib was enabled with the option onjcmd=y." OK. > replace "debug triggered by jcmd?" with " start debug via jcmd" Ok. > Is it really necessary/desired to set suspendOnInit to false? We > could still honor suspend=y|n. The suspension will of course > happen at the time debug activation is triggered. I don't think it would make sense to support suspend=y. In makes sense when starting the debugging directly (so no, or at least not much) Java code can be executed and you can debug from the beginning. And it also makes sense for the onthrow/onuncaught, since you can to see the exceptions in the debugger. But when triggered from the outside, you have no natural event to wait for and no state to preserve. The only effect you would see is that the jcmd would not return, since the initialize method would be blocked until a debugger has been connected. I've updated the webrev: http://cr.openjdk.java.net/~rrich/webrevs/schmelter/8214892/webrev.03/ Best regards, Ralf Schmelter -----Original Message----- From: Langer, Christoph Sent: Freitag, 7. Dezember 2018 09:41 To: Schmelter, Ralf ; Chris Plummer ; serviceability-dev at openjdk.java.net Subject: RE: RFR (M) 8214892: Delayed starting of debugging via jcmd Hi Ralf, thanks for doing this change. Overall, it looks very good to me and seems to be a nice feature. I have thought about a more generic name for the option rather than "onjcmd". What about "standby=y". That would indicate that the debugging agent is on standby and can be triggered by whatever means. What do others think? Here are some minor comments, mostly about the wording of text messages: src/hotspot/share/services/diagnosticCommand.cpp, line 1097: I'd prefer if you write out either "Debugging has been started." or "Debugging is already active.". I think this would make it more clear for the user of the feature what actually happened. src/hotspot/share/services/diagnosticCommand.hpp, line 878: "Starts up the Java debugging if enabled in the jdwp agentlib via the onjcmd=y option." A better wording would be: "Starts up the Java debugging if the jdwp agentlib was enabled with the option onjcmd=y." src/jdk.jdwp.agent/share/native/libjdwp/debugInit.c: line 876: replace "debug triggered by jcmd?" with " start debug via jcmd" (or "start debug on request" if we opt to change the option name from onjcmd to something more general) line 1300: suspendOnInit = JNI_FALSE; Is it really necessary/desired to set suspendOnInit to false? We could still honor suspend=y|n. The suspension will of course happen at the time debug activation is triggered. Another question: Do we need a CSR for this? Best regards Christoph > -----Original Message----- > From: serviceability-dev On > Behalf Of Schmelter, Ralf > Sent: Donnerstag, 6. Dezember 2018 15:54 > To: Chris Plummer ; serviceability- > dev at openjdk.java.net > Subject: [CAUTION] RE: RFR (M) 8214892: Delayed starting of debugging via > jcmd > > Hi Chris, > > > I think I prefer passing EI_VM_INIT for triggering_ei, but if you prefer > > to keep it as is, I suggest a comment to clarify why it might be out of > > range. > > Actually, I used EI_VM_INIT for the longest time and only changed it > recently, because I thought that code could assume that e.g. no classes have > been loaded yet when getting the INIT_VM event. But since the JVMTI spec > does not guarantees this in any way (it allows other events to be send before > a VM_INIT), I just will change it back to use EI_VM_INIT for the initialize call. > > Regarding the name of the option, I agree that onjcmd, while not technically > fully accurate, makes most sense for the common user. > > > ... It think you could just return the error right away and remove > > the error checking code that comes later. > > I've changed the code to just return the error directly. > > > It's not clear to me why you want the name and address of the first > > transport. Why is the first one necessarily the one you want? > > Since currently the bag of transports must always have a size of 1, getting the > first or the last transport is the same. But the callback function used to iterate > the bag has to return a boolean value. I've decided to return JNI_FALSE, > which would mean the iteration stops at the first entry. > > > I've updated the webrev with your and Goetz Lindenmeier's suggestions: > http://cr.openjdk.java.net/~rrich/webrevs/schmelter/8214892/webrev.02/ > > Best regards, > Ralf From zanglin5 at jd.com Mon Dec 10 14:46:34 2018 From: zanglin5 at jd.com (=?gb2312?B?6rDB1Q==?=) Date: Mon, 10 Dec 2018 14:46:34 +0000 Subject: optimize KlassInfoTable size to power of 2 In-Reply-To: References: <140bac18565c48be88313f7cc1d24bf8@jd.com> Message-ID: <7fc37fd7cf24439eb3a6fff132fc7387@jd.com> Hi Andrew, Thanks for your comments. I think the reason that idiv is issued is that the constant _num_buckets is not used directly for the hash idx calculation. It is the _size which is initialized to be _num_buckets. And I am also puzzling about why the _size is necessary since there is no modification on it after initialization. I can try use _num_buckets directly to calculate idx and see what is the instruction issued, and also do some measurement for comparing. Would you like to give more details about the benefit of using hash table with prime size? Thanks! Cheers, Lin -----Original Message----- From: Andrew Haley [mailto:aph at redhat.com] Sent: Monday, December 10, 2018 8:30 PM To: ?? ; serviceability-dev at openjdk.java.net Subject: Re: optimize KlassInfoTable size to power of 2 On 12/10/18 6:24 AM, ?? wrote: > My observation is that when ?jmap histo? iterating objects , the object?s klass > is identified and then hash idx in KlassInfoTable?s buckets[] is > calculated by mod of _num_bucktes, which would issue a heavy > instruction idiv on x86 platform. It means for every object scanned, a > idiv instruction is issued, which lag the performance espically when there are large number of objects in heap. I'm surprised that GCC uses idiv in this case: it has an optimization for constant modulo which doesn't use division. There is sometimes an advantage for using hash tables of prime size. Is it that the size of the table is not a constant that GCC can see? -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From magnus.ihse.bursie at oracle.com Mon Dec 10 15:11:25 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Mon, 10 Dec 2018 16:11:25 +0100 Subject: [PATCH] Windows 32-bit DLL name decoration In-Reply-To: <530c2651-dc96-e04a-0b0e-d1f04d8e135a@oracle.com> References: <2DD922C6-EFB1-4891-90F0-3378DEA91B1C@gmail.com> <07840796-f1c4-e077-7b0b-cd5c1a880ec1@oracle.com> <00e58330-26f0-4401-f911-24fda0127bcc@oracle.com> <7083B6EB-588E-4296-9A03-D9CC9B688936@gmail.com> <813f4b3d-0b4a-1e28-845f-a6422a22c695@redhat.com> <530c2651-dc96-e04a-0b0e-d1f04d8e135a@oracle.com> Message-ID: On 2018-12-10 16:02, Alexey Ivanov wrote: > > > On 10/12/2018 10:41, Magnus Ihse Bursie wrote: >> >> >> On 2018-12-07 22:18, Simon Tooke wrote: >>> Looking at the code, it seems (to me) the "correct" thing to do is to is >>> to create a Windows-specific version of dbgsysBuildFunName() in >>> linker_md.c, and have it decorate the function name as appropriate for >>> 32-bit windows. Note that in transport.c, you'd need to pass in the >>> parameter stack size (the bit after the @), but it could at least behave >>> properly on 32 vs 64 bit Windows. >>> >>> Notice this approach is already used for the library name, via >>> dbgsysBuildLibName() >>> >>> If the function is never intended to be called by Java via JNI, then it >>> should never have been declared JNICALL in the first place - but that >>> ship has sailed. >>> >>> I don't think changing the decorated exported name to undecorated is a >>> good idea, as it obfuscates the calling convention used. >> Good analysis, Simon. I agree. >> >> I have now looked more into the situation. In fact, it looks a lot >> like JDWP has been broken on Windows 32-bit for a long time, but we >> have patched around the issue in the JDK by using /export. This is >> the spec we're dealing with: >> https://docs.oracle.com/javase/10/docs/specs/jdwp/jdwp-transport.html. >> I quote: >> >>> The transport library must export an/onload/function with the >>> following prototype: >>> >>> |JNIEXPORT jint JNICALL jdwpTransport_OnLoad(JavaVM *jvm, >>> jdwpTransportCallback *callback, jint version, jdwpTransportEnv** env);| >>> >>> This function will be called by the JDWP (or other agent) when the >>> library is loaded. >>> >>> >> >> Nothing here says anything that the function should be exported using >> anything other than the normal _stdcall (implied by JNICALL) name >> mangling rules. However, JDWP has not been working according to the >> spec and looking for the correct symbol, and while we could have >> noticed that in the JDK, we didn't, because someone (long ago) added >> /export:jdwpTransport_OnLoad as a workaround to the affected >> libraries, instead of fixing JDWP. > > This means that we cannot change the calling convention: it must stay > as it is now. > > I think JDWP has always been working according to the spec. Using > __stdcall is the recommended calling convention for Win32 and for DLLs > in particular. Usually function names are exported undecorated in DLL. > (If my memory serves me well, older Microsoft tools exported |extern > "C" __stdcall| functions undecorated.) So then the question becomes: what does the spec mean? That the __stdcall convention should be used, or that the function name should be explicitly exported under a non __stdcall-convention name? Neither behavior seems clearly written out, so this is left to an implicit interpretation of what that "usually" means..? > > I believe this ? exporting the undecorated function names ? allows for > interoperability between 32 bit and 64 bit in cases where you load a > DLL and dynamically look up a function in it. > > There were too many /export options to export Win32 functions > undecorated for this one to be an accident rather than the intention. How do you mean? > >> Now, given that we've shipped code that didn't adhere to the specs >> for so long, we can probably not break that behavior. Instead, I >> propose we amend dbgsysBuildFunName() so that on Windows 32-bit, it >> looks for both the "jdwpTransport_OnLoad", and the symbol as mangled >> by _stdcall rules. I also propose that if both symbols are present, >> the _stdcall named one should take precedence, since that's according >> to the spec (and the other is just a fallback to maintain backwards >> compatibility). > > Since removing JNICALL is not an option, there are only two options: > > 1. Add |/export| option to the Makefile or pragma-comment to the > source file; > 2. Lookup the decorated name |_jdwpTransport_OnLoad at 16| for Win32 with > fallback to undecorated one. Yes. I think the correct solution here is 2. > > > I just wonder how it's possible to implement a generic > |dbgsysBuildFunName| for an arbitrary function without knowing the > size of function parameters. Could it be the reason why most DLLs > export __stdcall functions undecorated? (I assume you mean dbgsysFindLibraryEntry; there is a dbgsysBuildFunName as well but it appears to be dead code.) The dbgsysFindLibraryEntry does not need to work with an arbitrary function. It's implemented in jdk.jdwp.agent, for the sole reason of locating the jdwpTransport_OnLoad function. In general, I assume this to hold true. If you don't know the signature of the function you want to call, you're screwed anyway, regardless of calling convention. /Magnus > > Regards, > Alexey > >> >> /Magnus >>> -Simon Tooke >>> >>> >>> On 12/7/2018 2:09 PM, Alexey Ivanov wrote: >>>> Hi Andrew, >>>> >>>> Sorry for my belated reply. >>>> Yes, it's also an option? however, this solution seems to be >>>> overcomplicated to export one function or two. The calling convention >>>> of exported functions for JVM cannot be changed, they can be called >>>> from third-party software. >>>> >>>> If the function is used internally-only, its calling convention can be >>>> changed as long as all components are updated. >>>> >>>> Thank you for the pointer to another approach used to handle name >>>> decorations of __stdcall functions. It looks we have to work out a >>>> common approach to this problem. >>>> >>>> Regards, >>>> Alexey >>>> >>>> On 27/11/2018 18:49, Andrew Luo wrote: >>>>> By the way, in hotspot we are generating a .def file dynamically >>>>> while keeping the JNICALL calling convention (for symbols such as >>>>> JNI_CreateJavaVM) I believe (just by looking through the code in >>>>> http://hg.openjdk.java.net/jdk/jdk/file/44fe5fab538a/make/hotspot/lib/JvmMapfile.gmk, >>>>> unless this code is inactive - someone who knows this area better >>>>> than me might want to comment...). Shouldn't the same approach be >>>>> taken here as well? >>>>> >>>>> Thanks >>>>> Andrew >>>>> >>>>> -----Original Message----- >>>>> From: core-libs-dev On >>>>> Behalf Of Ali Ince >>>>> Sent: Monday, November 19, 2018 2:08 PM >>>>> To: Alexey Ivanov; build-dev >>>>> ; core-libs >>>>> Subject: Re: [PATCH] Windows 32-bit DLL name decoration >>>>> >>>>> Hi Alexey, >>>>> >>>>> I don?t have an account on JBS as I?m not an author yet, my OCA has >>>>> just been processed. Would it be possible for someone with an author >>>>> status to create an issue? >>>>> >>>>> Regards, >>>>> >>>>> Ali >>>>> >>>>>> On 19 Nov 2018, at 12:12, Alexey Ivanov >>>>>> wrote: >>>>>> >>>>>> Hi Ali, >>>>>> >>>>>> The fix looks good to me provided it resolves your problem. >>>>>> I am not a reviewer so you'll have to get OK from reviewers, likely >>>>>> from build-dev and from core-libs. >>>>>> >>>>>> Have you submitted the issue in JBS? >>>>>> >>>>>> You have to sign OCA to be able to contribute to OpenJDK: >>>>>> https://openjdk.java.net/contribute/ >>>>>> and also >>>>>> https://openjdk.java.net/sponsor/ >>>>>> >>>>>> Regards, >>>>>> Alexey >>>>>> >>>>>> On 18/11/2018 12:07, Ali ?nce wrote: >>>>>>> Hi Alexey, >>>>>>> >>>>>>> Below is a new patch content that removes JNICALL modifiers from >>>>>>> the exported functions of interest. This results in exported >>>>>>> functions not to be name decorated and use __cdecl calling convention. >>>>>>> >>>>>>> Do you have any more suggestions, or would you please guide me on >>>>>>> next steps? >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Ali >>>>>>> >>>>>>> # HG changeset patch >>>>>>> # User Ali Ince >>>>>>> # Date 1542542415 0 >>>>>>> # Sun Nov 18 12:00:15 2018 +0000 >>>>>>> # Node ID a41df44e13e1b761f4b3f05a17ed18953067ae39 >>>>>>> # Parent 16d5ec7dbbb49ef3f969e34be870e3f917ea89ff >>>>>>> Remove JNICALL modifiers to prevent name decoration on 32-bit windows >>>>>>> builds >>>>>>> >>>>>>> diff -r 16d5ec7dbbb4 -r a41df44e13e1 >>>>>>> src/jdk.jdi/share/native/libdt_shmem/shmemBack.c >>>>>>> --- a/src/jdk.jdi/share/native/libdt_shmem/shmemBack.c Tue Aug >>>>>>> 14 14:28:23 2018 +0200 >>>>>>> +++ b/src/jdk.jdi/share/native/libdt_shmem/shmemBack.c Sun Nov >>>>>>> 18 12:00:15 2018 +0000 >>>>>>> @@ -339,7 +339,7 @@ >>>>>>> return JDWPTRANSPORT_ERROR_NONE; } -JNIEXPORT jint JNICALL >>>>>>> +JNIEXPORT jint >>>>>>> jdwpTransport_OnLoad(JavaVM *vm, jdwpTransportCallback* cbTablePtr, >>>>>>> jint version, jdwpTransportEnv** result) { >>>>>>> diff -r 16d5ec7dbbb4 -r a41df44e13e1 >>>>>>> src/jdk.jdwp.agent/share/native/include/jdwpTransport.h >>>>>>> --- a/src/jdk.jdwp.agent/share/native/include/jdwpTransport.h >>>>>>> Tue Aug 14 14:28:23 2018 +0200 >>>>>>> +++ b/src/jdk.jdwp.agent/share/native/include/jdwpTransport.h >>>>>>> Sun Nov 18 12:00:15 2018 +0000 >>>>>>> @@ -140,7 +140,7 @@ >>>>>>> void (*free)(void *buffer); /* Call this for all >>>>>>> deallocations */ >>>>>>> } jdwpTransportCallback; >>>>>>> -typedef jint (JNICALL *jdwpTransport_OnLoad_t)(JavaVM *jvm, >>>>>>> +typedef jint (*jdwpTransport_OnLoad_t)(JavaVM *jvm, >>>>>>> >>>>>>> jdwpTransportCallback *callback, >>>>>>> jint version, >>>>>>> jdwpTransportEnv** >>>>>>> env); diff -r 16d5ec7dbbb4 -r a41df44e13e1 >>>>>>> src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c >>>>>>> --- >>>>>>> a/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c >>>>>>> Tue Aug 14 14:28:23 2018 +0200 >>>>>>> +++ >>>>>>> b/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c >>>>>>> Sun Nov 18 12:00:15 2018 +0000 >>>>>>> @@ -1019,7 +1019,7 @@ >>>>>>> return JDWPTRANSPORT_ERROR_NONE; } -JNIEXPORT jint JNICALL >>>>>>> +JNIEXPORT jint >>>>>>> jdwpTransport_OnLoad(JavaVM *vm, jdwpTransportCallback* cbTablePtr, >>>>>>> jint version, jdwpTransportEnv** env) { >>>>>>> >>>>>>> >>>>>>> >>>>>>>> On 16 Nov 2018, at 23:03, Alexey Ivanov >>>>>>>> wrote: >>>>>>>> >>>>>>>> Hi Ali, >>>>>>>> >>>>>>>> It's exactly what I referred to. >>>>>>>> >>>>>>>> Yes, it does change the calling convention. >>>>>>>> Yet it's not a (big) problem because both parties will use the >>>>>>>> same calling convention. >>>>>>>> >>>>>>>> Using a DLL from another build will not be possible. But it's not >>>>>>>> a good idea to do so anyway. >>>>>>>> >>>>>>>> >>>>>>>> Mapfiles and export options have been removed by >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8200178 to simplify >>>>>>>> managing exported functions. So that to add or remove an exported >>>>>>>> function, you don't have modify makefiles and/or mapfiles. You >>>>>>>> just edit the source files. >>>>>>>> >>>>>>>> Regards, >>>>>>>> Alexey >>>>>>>> >>>>>>>> On 16/11/2018 22:42, Ali ?nce wrote: >>>>>>>>> Hi Alexey, >>>>>>>>> >>>>>>>>> Thanks for your reply. >>>>>>>>> >>>>>>>>> I will definitely give it a try though I?m not sure if that?ll be >>>>>>>>> a breaking change. This is because JNICALL modifier is defined to >>>>>>>>> be __stdcall on windows which is both specified on >>>>>>>>> jdwpTransport_OnLoad exported functions both for dt_socket.dll >>>>>>>>> and dt_shmem.dll. >>>>>>>>> >>>>>>>>> The __stdcall is ignored on x64 platforms and also name >>>>>>>>> decoration is not applied >>>>>>>>> (https://docs.microsoft.com/en-gb/cpp/build/reference/decorated-names?view=vs-2017). >>>>>>>>> I believe that?s why we don?t experience any problems on x64 builds. >>>>>>>>> >>>>>>>>> Removing JNICALL (thus __stdcall) modifier (apparently only >>>>>>>>> applies to x86 builds) will result in the calling convention to >>>>>>>>> be changed, and thus will change how parameters are ordered and >>>>>>>>> also the stack cleanup rules. I think this may result in problems >>>>>>>>> in programs that are designed to load shared libraries and its >>>>>>>>> exported functions at runtime (not bound by link time which >>>>>>>>> probably won?t cause any issues - assuming that they use the >>>>>>>>> shared function signature) - as in >>>>>>>>> https://github.com/AdoptOpenJDK/openjdk-jdk11u/blob/5f01925b80ed851b133ee26fbcb07026ac04149e/src/jdk.jdwp.agent/share/native/libjdwp/transport.c#L99. >>>>>>>>> >>>>>>>>> Any thoughts? >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> >>>>>>>>> Ali >>>>>>>>> >>>>>>>>>> On 15 Nov 2018, at 22:14, Alexey Ivanov >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> Hi Ali, >>>>>>>>>> >>>>>>>>>> Can the issue be resolved by using different call modifiers on >>>>>>>>>> the affected functions? >>>>>>>>>> >>>>>>>>>> Some build / link issues were resolved by adding / removing >>>>>>>>>> JNICALL modifier, see >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8201226 >>>>>>>>>> http://mail.openjdk.java.net/pipermail/build-dev/2018-April/021553 >>>>>>>>>> .html >>>>>>>>>> >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8202476 >>>>>>>>>> http://mail.openjdk.java.net/pipermail/build-dev/2018-May/021913.h >>>>>>>>>> tml >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> Alexey >>>>>>>>>> >>>>>>>>>> On 15/11/2018 21:43, Ali ?nce wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> An issue >>>>>>>>>>> (https://github.com/AdoptOpenJDK/openjdk-build/issues/709) >>>>>>>>>>> raised against AdoptOpenJDK jdk11 32-bit windows builds led us >>>>>>>>>>> to the name decoration problem on built 32-bit dlls - >>>>>>>>>>> specifically dt_socket.dll and dt_shmem.dll. Whereas 64-bit >>>>>>>>>>> MSVC builds don?t apply any name decorations on exported >>>>>>>>>>> functions, 32-bit builds apply them - which requires either def >>>>>>>>>>> files or /export flags to be specified on the linker arguments. >>>>>>>>>>> >>>>>>>>>>> Although the expected linker arguments were present on previous >>>>>>>>>>> jdk makefiles, they were removed in jdk11 makefiles. >>>>>>>>>>> >>>>>>>>>>> Below is the proposed patch, which can also be browsed at >>>>>>>>>>> https://github.com/AdoptOpenJDK/openjdk-jdk11u/pull/1. >>>>>>>>>>> >>>>>>>>>>> Would you please have a look and let me know of any points I >>>>>>>>>>> may have missed (I don?t have any background information about >>>>>>>>>>> why the export flags were removed in jdk11)? >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> >>>>>>>>>>> Ali >>>>>>>>>>> >>>>>>>>>>> diff --git a/make/lib/Lib-jdk.jdi.gmk b/make/lib/Lib-jdk.jdi.gmk >>>>>>>>>>> index 197b95c2e2..ac6ebf5591 100644 >>>>>>>>>>> --- a/make/lib/Lib-jdk.jdi.gmk >>>>>>>>>>> +++ b/make/lib/Lib-jdk.jdi.gmk >>>>>>>>>>> @@ -37,6 +37,7 @@ ifeq ($(OPENJDK_TARGET_OS), windows) >>>>>>>>>>> jdk.jdwp.agent:include \ >>>>>>>>>>> jdk.jdwp.agent:libjdwp/export, \ >>>>>>>>>>> LDFLAGS := $(LDFLAGS_JDKLIB), \ >>>>>>>>>>> + LDFLAGS_windows := -export:jdwpTransport_OnLoad, \ >>>>>>>>>>> LIBS := $(JDKLIB_LIBS), \ >>>>>>>>>>> )) >>>>>>>>>>> >>>>>>>>>>> diff --git a/make/lib/Lib-jdk.jdwp.agent.gmk >>>>>>>>>>> b/make/lib/Lib-jdk.jdwp.agent.gmk index 0bc93e0d35..0382e55325 >>>>>>>>>>> 100644 >>>>>>>>>>> --- a/make/lib/Lib-jdk.jdwp.agent.gmk >>>>>>>>>>> +++ b/make/lib/Lib-jdk.jdwp.agent.gmk >>>>>>>>>>> @@ -37,6 +37,7 @@ $(eval $(call SetupJdkLibrary, >>>>>>>>>>> BUILD_LIBDT_SOCKET, \ >>>>>>>>>>> libjdwp/export, \ >>>>>>>>>>> LDFLAGS := $(LDFLAGS_JDKLIB) \ >>>>>>>>>>> $(call SET_SHARED_LIBRARY_ORIGIN), \ >>>>>>>>>>> + LDFLAGS_windows := -export:jdwpTransport_OnLoad, \ >>>>>>>>>>> LIBS_linux := -lpthread, \ >>>>>>>>>>> LIBS_solaris := -lnsl -lsocket, \ >>>>>>>>>>> LIBS_windows := $(JDKLIB_LIBS) ws2_32.lib, \ >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aph at redhat.com Mon Dec 10 15:19:49 2018 From: aph at redhat.com (Andrew Haley) Date: Mon, 10 Dec 2018 15:19:49 +0000 Subject: optimize KlassInfoTable size to power of 2 In-Reply-To: <7fc37fd7cf24439eb3a6fff132fc7387@jd.com> References: <140bac18565c48be88313f7cc1d24bf8@jd.com> <7fc37fd7cf24439eb3a6fff132fc7387@jd.com> Message-ID: <4ea1c92f-4e72-7d38-8c7a-91e2892e0c3b@redhat.com> On 12/10/18 2:46 PM, ?? wrote: > Would you like to give more details about the benefit of using hash table with prime > size? Really? OK. Imagine that your hash table is an exact power of two in size. Reducing the index mod any power of two means using only the lower n bits of the hash value. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From daniel.daugherty at oracle.com Mon Dec 10 16:24:41 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 10 Dec 2018 11:24:41 -0500 Subject: RFR: JDK-8210106: sun/tools/jps/TestJps.java timed out In-Reply-To: <41d1ffce-202a-2c65-61b4-27815e347a28@oracle.com> References: <5C0923D1.8080705@oracle.com> <6697b4a2-b0d7-8a52-7549-23d6433114cc@oracle.com> <5C0A6E8D.7010108@oracle.com> <41d1ffce-202a-2c65-61b4-27815e347a28@oracle.com> Message-ID: On 12/10/18 2:55 AM, David Holmes wrote: > Hi Dan, > >> Gary saw a longest time value of 280 seconds in his testing with >> fastdebug bits. The default timeout value is 120 seconds with a >> default timeout factor of 4 in Mach5 gives a total timeout of >> 480 seconds (8 minutes). With Gary's new default timeout of >> 360 seconds, we'll have a total timeout value of 1440 seconds >> (24 minutes). > > So based on that it seems Gary's observations are not relevant to what > has been seen in mach5 because at 280 seconds we should never have > been timing out! Correct. Gary's test runs never ran into the same state as the end-of-August run where the test ran into the 20 minute timeout. Since those logs are now gone, we cannot discern any more information about what happened back then... Dan > > I guess these changes won't hurt. > > Thanks, > David > > On 8/12/2018 1:12 am, Daniel D. Daugherty wrote: >> On 12/7/18 7:58 AM, Gary Adams wrote: >>> On 12/6/18, 7:52 PM, David Holmes wrote: >>>> Hi Gary, >>>> >>>> On 6/12/2018 11:27 pm, Gary Adams wrote: >>>>> On a local linux-x64-debug build this test consistently runs in >>>>> less than 40 seconds. >>>>> On the mach5 testing machines there was a large fluctuation in the >>>>> time to complete this test. >>>>> >>>>> Since the test runs jps with different combinations of arguments, >>>>> the total >>>>> test time is dependent on the number of processes running java >>>>> programs. >>>>> Since the mach5 test infrastructure runs java programs I have seen >>>>> a 3X >>>>> in the amount of output the test produces compared to local test >>>>> runs. >>>>> >>>>> I've run the test several hundred times through mach5 on the >>>>> slower platforms >>>>> and then examined the test logs using a 3X setting from the >>>>> default 120 second >>>>> jtreg timeout. The slowest reported elapse time from the test logs >>>>> showed >>>>> 280 seconds to complete. >>>>> >>>>> To improve the reliability of the test to complete, I'd like to >>>>> increase the >>>>> timeout to 360 seconds. >>>>> >>>>> ?? Issue: https://bugs.openjdk.java.net/browse/JDK-8210106 >>>>> >>>>> Proposed fix: >>>>> >>>>> diff --git a/test/jdk/sun/tools/jps/TestJps.java >>>>> b/test/jdk/sun/tools/jps/TestJps.java >>>>> --- a/test/jdk/sun/tools/jps/TestJps.java >>>>> +++ b/test/jdk/sun/tools/jps/TestJps.java >>>>> @@ -27,7 +27,7 @@ >>>>> ?? * @modules jdk.jartool/sun.tools.jar >>>>> ?? * @build LingeredAppForJps >>>>> ?? * @build LingeredApp >>>>> - * @run main/othervm TestJps >>>>> + * @run main/othervm/timeout=360 TestJps >>>>> ?? */ >>>> >>>> Doesn't that then get scaled by the timeout factor resulting in a >>>> much longer timeout than the 360 seconds you intended? >>>> >>>> For other timeout adjustments the needed time has been divided by >>>> the timeout factor to get the actual intended timeout. >>> >>> This bug was filed fairly recently in Aug 2018. >>> At that time the timeout and timeout factor were not sufficient >>> to avoid the test failing. >>> >>> The mach5 timeout factors were adjusted recently, so this test may >>> no longer be an issue. >>> >>> If that is true, then we could simply close this bug as "cannot >>> reproduce". >>> An argument could be made that the change in timeout factor may be >>> responsible for fixing a lot more of the intermittent bugs and that >>> they >>> should be closed in a similar manner. >>> >>> Historically, we could say this particular bug should have had timeouts >>> reassessed when the infrastructure switched from jprt to mach5 testing >>> where there were more visible Java processes running. >>> >>> Using a higher explicit timeout will not make the test run any longer >>> than it needs. It will simply allow the test to not be terminated >>> sooner >>> in a hung test scenario. >>> >>> What is your preference for this particular issue: >>> ?? - increase the explicit timeout >>> ?? - close as cannot reproduce attributed to the timeout factor >>> adjustments >>> >>> What would you recommend going forward for other similar issues: >>> ?? - determine a new explicit timeout >>> ?? - close if no timeout failures have been observed since the >>> timeout factor >>> ????? was raised >> >> General guidance that I've been giving folks is that you bump the >> default timeout when you see timeouts with 'release' bits with >> a timeout factor of '1'. The default timeout factor in Mach5 is 4 >> so we run both 'release' and 'fastdebug' bits with that timeout >> factor. That means our total timeout value for 'release' bits is >> probably a little long and our total timeout value for 'fastdebug' >> bits is probably just right for a concurrency factor of 12. >> >> On my personal servers I use the following timeout factors: >> >> ???????? release) >> ???????????? TIMEOUT_FACTOR="4" >> ???????????? ;; >> ???????? fastdebug) >> ???????????? TIMEOUT_FACTOR="6" >> ???????????? ;; >> ???????? slowdebug) >> ???????????? TIMEOUT_FACTOR="12" >> ???????????? ;; >> >> However, I run with a concurrency factor of 16. I rarely see >> timeout failures. >> >> All that is well and good, but a 'jps' test is a different >> beast since it is affected by factors outside of most tests. >> Since the number of java processes running on the system can >> affect the test, I'm okay with using a default timeout of '360' >> for this test as a guard against increases in load or... >> >> Gary saw a longest time value of 280 seconds in his testing with >> fastdebug bits. The default timeout value is 120 seconds with a >> default timeout factor of 4 in Mach5 gives a total timeout of >> 480 seconds (8 minutes). With Gary's new default timeout of >> 360 seconds, we'll have a total timeout value of 1440 seconds >> (24 minutes). >> >> Before the recent changes to Mach5, the default timeout factor >> was 10 so we had a total timeout value of 1200 seconds (20 >> minutes). Gary's change only bumps the total timeout value >> by 4 minutes from what we had back in August... when this bug >> was filed... unfortunately, the log for that sighting is gone >> so all we know is that the test timed out after 20 minutes >> >> ?> Timeout refired 1200 times >> >> We know that JTREG timeout handling kicks in at timeout expiration, >> but without the log we don't know how much longer than 20 minutes >> the test ran before JTREG killed it. 24 minutes might do it, but... >> >> Dan >> >> >>> >>> >>>> >>>> Cheers, >>>> David >>>> >>> >>> >> From martinrb at google.com Mon Dec 10 17:32:35 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 10 Dec 2018 09:32:35 -0800 Subject: [PATCH] JDK-8025886: Replace [[ and == bash extensions in tests In-Reply-To: References: Message-ID: I would not try to remove all bash-isms from openjdk. Instead I would find instances of /bin/sh that need to be changed to /bin/bash. (Ubuntu's use of /bin/sh -> /bin/dash is technically correct, but caused much suffering https://bugs.launchpad.net/ubuntu/+source/dash/+bug/61463 ) -------------- next part -------------- An HTML attachment was scrubbed... URL: From jcbeyler at google.com Mon Dec 10 21:02:32 2018 From: jcbeyler at google.com (JC Beyler) Date: Mon, 10 Dec 2018 13:02:32 -0800 Subject: RFR (L) 8215161: Remove spaces in assignments for vmTestbase/[a-j] Message-ID: Hi all, Could I get a review that adds a space around comparisons for the vmTestbase? This is the first of two webrevs to handle this. Webrev: http://cr.openjdk.java.net/~jcbeyler/8215161/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8215161 Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From jcbeyler at google.com Mon Dec 10 21:02:35 2018 From: jcbeyler at google.com (JC Beyler) Date: Mon, 10 Dec 2018 13:02:35 -0800 Subject: RFR(L) 8212960: Remove spaces in assignments for remaining vmTestbase Message-ID: Hi all, Could I get a review that adds a space around comparisons for the vmTestbase? This is the second of two webrevs to handle this. Webrev: http://cr.openjdk.java.net/~jcbeyler/8215160/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8215160 Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From ali.ince at gmail.com Mon Dec 10 21:05:28 2018 From: ali.ince at gmail.com (=?utf-8?B?QWxpIMSwbmNl?=) Date: Mon, 10 Dec 2018 21:05:28 +0000 Subject: [PATCH] Windows 32-bit DLL name decoration In-Reply-To: References: <2DD922C6-EFB1-4891-90F0-3378DEA91B1C@gmail.com> <07840796-f1c4-e077-7b0b-cd5c1a880ec1@oracle.com> <00e58330-26f0-4401-f911-24fda0127bcc@oracle.com> <7083B6EB-588E-4296-9A03-D9CC9B688936@gmail.com> <813f4b3d-0b4a-1e28-845f-a6422a22c695@redhat.com> <530c2651-dc96-e04a-0b0e-d1f04d8e135a@oracle.com> Message-ID: <6700A382-6D68-4565-A3C5-3FA1BF2029C4@gmail.com> > On 10 Dec 2018, at 15:11, Magnus Ihse Bursie wrote: > > > > On 2018-12-10 16:02, Alexey Ivanov wrote: >> >> >> On 10/12/2018 10:41, Magnus Ihse Bursie wrote: >>> >>> >>> On 2018-12-07 22:18, Simon Tooke wrote: >>>> Looking at the code, it seems (to me) the "correct" thing to do is to is >>>> to create a Windows-specific version of dbgsysBuildFunName() in >>>> linker_md.c, and have it decorate the function name as appropriate for >>>> 32-bit windows. Note that in transport.c, you'd need to pass in the >>>> parameter stack size (the bit after the @), but it could at least behave >>>> properly on 32 vs 64 bit Windows. >>>> >>>> Notice this approach is already used for the library name, via >>>> dbgsysBuildLibName() >>>> >>>> If the function is never intended to be called by Java via JNI, then it >>>> should never have been declared JNICALL in the first place - but that >>>> ship has sailed. >>>> >>>> I don't think changing the decorated exported name to undecorated is a >>>> good idea, as it obfuscates the calling convention used. >>>> >>> Good analysis, Simon. I agree. >>> >>> I have now looked more into the situation. In fact, it looks a lot like JDWP has been broken on Windows 32-bit for a long time, but we have patched around the issue in the JDK by using /export. This is the spec we're dealing with: https://docs.oracle.com/javase/10/docs/specs/jdwp/jdwp-transport.html. I quote: >>> >>>> The transport library must export an onload function with the following prototype: >>>> >>>> JNIEXPORT jint JNICALL >>>> jdwpTransport_OnLoad(JavaVM *jvm, >>>> jdwpTransportCallback *callback, >>>> jint version, >>>> jdwpTransportEnv** env); >>>> >>>> This function will be called by the JDWP (or other agent) when the library is loaded. >>>> >>>> >>> >>> Nothing here says anything that the function should be exported using anything other than the normal _stdcall (implied by JNICALL) name mangling rules. However, JDWP has not been working according to the spec and looking for the correct symbol, and while we could have noticed that in the JDK, we didn't, because someone (long ago) added /export:jdwpTransport_OnLoad as a workaround to the affected libraries, instead of fixing JDWP. >> >> This means that we cannot change the calling convention: it must stay as it is now. >> >> I think JDWP has always been working according to the spec. Using __stdcall is the recommended calling convention for Win32 and for DLLs in particular. Usually function names are exported undecorated in DLL. (If my memory serves me well, older Microsoft tools exported extern "C" __stdcall functions undecorated.) > So then the question becomes: what does the spec mean? That the __stdcall convention should be used, or that the function name should be explicitly exported under a non __stdcall-convention name? Neither behavior seems clearly written out, so this is left to an implicit interpretation of what that "usually" means..? A couple of MSFT quotes on name decoration from https://docs.microsoft.com/en-gb/cpp/build/reference/decorated-names?view=vs-2017 says; > Normally, you don't have to know the decorated name to write code that compiles and links successfully. Decorated names are an implementation detail internal to the compiler and linker. > Name decoration is also important when linking to code written in other programming languages or using other compilers. Different compilers use different name decoration conventions. When your executable links to code written in another language, special care must be taken to match the exported and imported names and calling conventions. And we should also keep in mind that MSFT itself doesn?t use name mangling in the core windows API. I still see this |jdwpTransport_OnLoad| not much different than |JNI_CreateJavaVM| in the context of it?s availability and accessibility from outer world. >> >> I believe this ? exporting the undecorated function names ? allows for interoperability between 32 bit and 64 bit in cases where you load a DLL and dynamically look up a function in it. >> >> There were too many /export options to export Win32 functions undecorated for this one to be an accident rather than the intention. > How do you mean? >> >>> Now, given that we've shipped code that didn't adhere to the specs for so long, we can probably not break that behavior. Instead, I propose we amend dbgsysBuildFunName() so that on Windows 32-bit, it looks for both the "jdwpTransport_OnLoad", and the symbol as mangled by _stdcall rules. I also propose that if both symbols are present, the _stdcall named one should take precedence, since that's according to the spec (and the other is just a fallback to maintain backwards compatibility). >> >> Since removing JNICALL is not an option, there are only two options: >> >> 1. Add /export option to the Makefile or pragma-comment to the source file; >> 2. Lookup the decorated name _jdwpTransport_OnLoad at 16 for Win32 with fallback to undecorated one. > Yes. > > I think the correct solution here is 2. Does keeping name mangling in-place gives anyone any value? It would probably only add the overhead of making the callers compute the stack size of the parameters. And I couldn?t find any sign or history of changes but name mangling, theoretically, it could change over versions of compilers. The pragma-comment approach could also be overly simplified as shown https://stackoverflow.com/a/41910450, which will leave us with a single line addition to the function body as; #pragma EXPORT which is from my point of view a simpler (and maintainable) approach than adding a decorated name lookup logic. Regards, Ali >> >> >> I just wonder how it's possible to implement a generic dbgsysBuildFunName for an arbitrary function without knowing the size of function parameters. Could it be the reason why most DLLs export __stdcall functions undecorated? > (I assume you mean dbgsysFindLibraryEntry; there is a dbgsysBuildFunName as well but it appears to be dead code.) > > The dbgsysFindLibraryEntry does not need to work with an arbitrary function. It's implemented in jdk.jdwp.agent, for the sole reason of locating the jdwpTransport_OnLoad function. > > In general, I assume this to hold true. If you don't know the signature of the function you want to call, you're screwed anyway, regardless of calling convention. > > /Magnus > >> >> Regards, >> Alexey >> >>> >>> /Magnus >>>> -Simon Tooke >>>> >>>> >>>> On 12/7/2018 2:09 PM, Alexey Ivanov wrote: >>>> >>>>> Hi Andrew, >>>>> >>>>> Sorry for my belated reply. >>>>> Yes, it's also an option? however, this solution seems to be >>>>> overcomplicated to export one function or two. The calling convention >>>>> of exported functions for JVM cannot be changed, they can be called >>>>> from third-party software. >>>>> >>>>> If the function is used internally-only, its calling convention can be >>>>> changed as long as all components are updated. >>>>> >>>>> Thank you for the pointer to another approach used to handle name >>>>> decorations of __stdcall functions. It looks we have to work out a >>>>> common approach to this problem. >>>>> >>>>> Regards, >>>>> Alexey >>>>> >>>>> On 27/11/2018 18:49, Andrew Luo wrote: >>>>> >>>>>> By the way, in hotspot we are generating a .def file dynamically >>>>>> while keeping the JNICALL calling convention (for symbols such as >>>>>> JNI_CreateJavaVM) I believe (just by looking through the code in >>>>>> >>>>>> http://hg.openjdk.java.net/jdk/jdk/file/44fe5fab538a/make/hotspot/lib/JvmMapfile.gmk >>>>>> , >>>>>> unless this code is inactive - someone who knows this area better >>>>>> than me might want to comment...). Shouldn't the same approach be >>>>>> taken here as well? >>>>>> >>>>>> Thanks >>>>>> Andrew >>>>>> >>>>>> -----Original Message----- >>>>>> From: core-libs-dev >>>>>> >>>>>> On >>>>>> Behalf Of Ali Ince >>>>>> Sent: Monday, November 19, 2018 2:08 PM >>>>>> To: Alexey Ivanov >>>>>> >>>>>> ; build-dev >>>>>> >>>>>> ; core-libs >>>>>> >>>>>> Subject: Re: [PATCH] Windows 32-bit DLL name decoration >>>>>> >>>>>> Hi Alexey, >>>>>> >>>>>> I don?t have an account on JBS as I?m not an author yet, my OCA has >>>>>> just been processed. Would it be possible for someone with an author >>>>>> status to create an issue? >>>>>> >>>>>> Regards, >>>>>> >>>>>> Ali >>>>>> >>>>>> >>>>>>> On 19 Nov 2018, at 12:12, Alexey Ivanov >>>>>>> >>>>>>> wrote: >>>>>>> >>>>>>> Hi Ali, >>>>>>> >>>>>>> The fix looks good to me provided it resolves your problem. >>>>>>> I am not a reviewer so you'll have to get OK from reviewers, likely >>>>>>> from build-dev and from core-libs. >>>>>>> >>>>>>> Have you submitted the issue in JBS? >>>>>>> >>>>>>> You have to sign OCA to be able to contribute to OpenJDK: >>>>>>> >>>>>>> https://openjdk.java.net/contribute/ >>>>>>> >>>>>>> and also >>>>>>> >>>>>>> https://openjdk.java.net/sponsor/ >>>>>>> >>>>>>> >>>>>>> Regards, >>>>>>> Alexey >>>>>>> >>>>>>> On 18/11/2018 12:07, Ali ?nce wrote: >>>>>>> >>>>>>>> Hi Alexey, >>>>>>>> >>>>>>>> Below is a new patch content that removes JNICALL modifiers from >>>>>>>> the exported functions of interest. This results in exported >>>>>>>> functions not to be name decorated and use __cdecl calling convention. >>>>>>>> >>>>>>>> Do you have any more suggestions, or would you please guide me on >>>>>>>> next steps? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Ali >>>>>>>> >>>>>>>> # HG changeset patch >>>>>>>> # User Ali Ince >>>>>>>> >>>>>>>> >>>>>>>> # Date 1542542415 0 >>>>>>>> # Sun Nov 18 12:00:15 2018 +0000 >>>>>>>> # Node ID a41df44e13e1b761f4b3f05a17ed18953067ae39 >>>>>>>> # Parent 16d5ec7dbbb49ef3f969e34be870e3f917ea89ff >>>>>>>> Remove JNICALL modifiers to prevent name decoration on 32-bit windows >>>>>>>> builds >>>>>>>> >>>>>>>> diff -r 16d5ec7dbbb4 -r a41df44e13e1 >>>>>>>> src/jdk.jdi/share/native/libdt_shmem/shmemBack.c >>>>>>>> --- a/src/jdk.jdi/share/native/libdt_shmem/shmemBack.c Tue Aug >>>>>>>> 14 14:28:23 2018 +0200 >>>>>>>> +++ b/src/jdk.jdi/share/native/libdt_shmem/shmemBack.c Sun Nov >>>>>>>> 18 12:00:15 2018 +0000 >>>>>>>> @@ -339,7 +339,7 @@ >>>>>>>> return JDWPTRANSPORT_ERROR_NONE; } -JNIEXPORT jint JNICALL >>>>>>>> +JNIEXPORT jint >>>>>>>> jdwpTransport_OnLoad(JavaVM *vm, jdwpTransportCallback* cbTablePtr, >>>>>>>> jint version, jdwpTransportEnv** result) { >>>>>>>> diff -r 16d5ec7dbbb4 -r a41df44e13e1 >>>>>>>> src/jdk.jdwp.agent/share/native/include/jdwpTransport.h >>>>>>>> --- a/src/jdk.jdwp.agent/share/native/include/jdwpTransport.h >>>>>>>> Tue Aug 14 14:28:23 2018 +0200 >>>>>>>> +++ b/src/jdk.jdwp.agent/share/native/include/jdwpTransport.h >>>>>>>> Sun Nov 18 12:00:15 2018 +0000 >>>>>>>> @@ -140,7 +140,7 @@ >>>>>>>> void (*free)(void *buffer); /* Call this for all >>>>>>>> deallocations */ >>>>>>>> } jdwpTransportCallback; >>>>>>>> -typedef jint (JNICALL *jdwpTransport_OnLoad_t)(JavaVM *jvm, >>>>>>>> +typedef jint (*jdwpTransport_OnLoad_t)(JavaVM *jvm, >>>>>>>> >>>>>>>> jdwpTransportCallback *callback, >>>>>>>> jint version, >>>>>>>> jdwpTransportEnv** >>>>>>>> env); diff -r 16d5ec7dbbb4 -r a41df44e13e1 >>>>>>>> src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c >>>>>>>> --- >>>>>>>> a/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c >>>>>>>> Tue Aug 14 14:28:23 2018 +0200 >>>>>>>> +++ >>>>>>>> b/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c >>>>>>>> Sun Nov 18 12:00:15 2018 +0000 >>>>>>>> @@ -1019,7 +1019,7 @@ >>>>>>>> return JDWPTRANSPORT_ERROR_NONE; } -JNIEXPORT jint JNICALL >>>>>>>> +JNIEXPORT jint >>>>>>>> jdwpTransport_OnLoad(JavaVM *vm, jdwpTransportCallback* cbTablePtr, >>>>>>>> jint version, jdwpTransportEnv** env) { >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> On 16 Nov 2018, at 23:03, Alexey Ivanov >>>>>>>>> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Hi Ali, >>>>>>>>> >>>>>>>>> It's exactly what I referred to. >>>>>>>>> >>>>>>>>> Yes, it does change the calling convention. >>>>>>>>> Yet it's not a (big) problem because both parties will use the >>>>>>>>> same calling convention. >>>>>>>>> >>>>>>>>> Using a DLL from another build will not be possible. But it's not >>>>>>>>> a good idea to do so anyway. >>>>>>>>> >>>>>>>>> >>>>>>>>> Mapfiles and export options have been removed by >>>>>>>>> >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8200178 >>>>>>>>> to simplify >>>>>>>>> managing exported functions. So that to add or remove an exported >>>>>>>>> function, you don't have modify makefiles and/or mapfiles. You >>>>>>>>> just edit the source files. >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Alexey >>>>>>>>> >>>>>>>>> On 16/11/2018 22:42, Ali ?nce wrote: >>>>>>>>> >>>>>>>>>> Hi Alexey, >>>>>>>>>> >>>>>>>>>> Thanks for your reply. >>>>>>>>>> >>>>>>>>>> I will definitely give it a try though I?m not sure if that?ll be >>>>>>>>>> a breaking change. This is because JNICALL modifier is defined to >>>>>>>>>> be __stdcall on windows which is both specified on >>>>>>>>>> jdwpTransport_OnLoad exported functions both for dt_socket.dll >>>>>>>>>> and dt_shmem.dll. >>>>>>>>>> >>>>>>>>>> The __stdcall is ignored on x64 platforms and also name >>>>>>>>>> decoration is not applied >>>>>>>>>> ( >>>>>>>>>> https://docs.microsoft.com/en-gb/cpp/build/reference/decorated-names?view=vs-2017 >>>>>>>>>> ). >>>>>>>>>> I believe that?s why we don?t experience any problems on x64 builds. >>>>>>>>>> >>>>>>>>>> Removing JNICALL (thus __stdcall) modifier (apparently only >>>>>>>>>> applies to x86 builds) will result in the calling convention to >>>>>>>>>> be changed, and thus will change how parameters are ordered and >>>>>>>>>> also the stack cleanup rules. I think this may result in problems >>>>>>>>>> in programs that are designed to load shared libraries and its >>>>>>>>>> exported functions at runtime (not bound by link time which >>>>>>>>>> probably won?t cause any issues - assuming that they use the >>>>>>>>>> shared function signature) - as in >>>>>>>>>> >>>>>>>>>> https://github.com/AdoptOpenJDK/openjdk-jdk11u/blob/5f01925b80ed851b133ee26fbcb07026ac04149e/src/jdk.jdwp.agent/share/native/libjdwp/transport.c#L99 >>>>>>>>>> . >>>>>>>>>> >>>>>>>>>> Any thoughts? >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> >>>>>>>>>> Ali >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On 15 Nov 2018, at 22:14, Alexey Ivanov >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> Hi Ali, >>>>>>>>>>> >>>>>>>>>>> Can the issue be resolved by using different call modifiers on >>>>>>>>>>> the affected functions? >>>>>>>>>>> >>>>>>>>>>> Some build / link issues were resolved by adding / removing >>>>>>>>>>> JNICALL modifier, see >>>>>>>>>>> >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8201226 >>>>>>>>>>> http://mail.openjdk.java.net/pipermail/build-dev/2018-April/021553 >>>>>>>>>>> >>>>>>>>>>> .html >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8202476 >>>>>>>>>>> http://mail.openjdk.java.net/pipermail/build-dev/2018-May/021913.h >>>>>>>>>>> >>>>>>>>>>> tml >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> Alexey >>>>>>>>>>> >>>>>>>>>>> On 15/11/2018 21:43, Ali ?nce wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> An issue >>>>>>>>>>>> ( >>>>>>>>>>>> https://github.com/AdoptOpenJDK/openjdk-build/issues/709 >>>>>>>>>>>> ) >>>>>>>>>>>> raised against AdoptOpenJDK jdk11 32-bit windows builds led us >>>>>>>>>>>> to the name decoration problem on built 32-bit dlls - >>>>>>>>>>>> specifically dt_socket.dll and dt_shmem.dll. Whereas 64-bit >>>>>>>>>>>> MSVC builds don?t apply any name decorations on exported >>>>>>>>>>>> functions, 32-bit builds apply them - which requires either def >>>>>>>>>>>> files or /export flags to be specified on the linker arguments. >>>>>>>>>>>> >>>>>>>>>>>> Although the expected linker arguments were present on previous >>>>>>>>>>>> jdk makefiles, they were removed in jdk11 makefiles. >>>>>>>>>>>> >>>>>>>>>>>> Below is the proposed patch, which can also be browsed at >>>>>>>>>>>> >>>>>>>>>>>> https://github.com/AdoptOpenJDK/openjdk-jdk11u/pull/1 >>>>>>>>>>>> . >>>>>>>>>>>> >>>>>>>>>>>> Would you please have a look and let me know of any points I >>>>>>>>>>>> may have missed (I don?t have any background information about >>>>>>>>>>>> why the export flags were removed in jdk11)? >>>>>>>>>>>> >>>>>>>>>>>> Regards, >>>>>>>>>>>> >>>>>>>>>>>> Ali >>>>>>>>>>>> >>>>>>>>>>>> diff --git a/make/lib/Lib-jdk.jdi.gmk b/make/lib/Lib-jdk.jdi.gmk >>>>>>>>>>>> index 197b95c2e2..ac6ebf5591 100644 >>>>>>>>>>>> --- a/make/lib/Lib-jdk.jdi.gmk >>>>>>>>>>>> +++ b/make/lib/Lib-jdk.jdi.gmk >>>>>>>>>>>> @@ -37,6 +37,7 @@ ifeq ($(OPENJDK_TARGET_OS), windows) >>>>>>>>>>>> jdk.jdwp.agent:include \ >>>>>>>>>>>> jdk.jdwp.agent:libjdwp/export, \ >>>>>>>>>>>> LDFLAGS := $(LDFLAGS_JDKLIB), \ >>>>>>>>>>>> + LDFLAGS_windows := -export:jdwpTransport_OnLoad, \ >>>>>>>>>>>> LIBS := $(JDKLIB_LIBS), \ >>>>>>>>>>>> )) >>>>>>>>>>>> >>>>>>>>>>>> diff --git a/make/lib/Lib-jdk.jdwp.agent.gmk >>>>>>>>>>>> b/make/lib/Lib-jdk.jdwp.agent.gmk index 0bc93e0d35..0382e55325 >>>>>>>>>>>> 100644 >>>>>>>>>>>> --- a/make/lib/Lib-jdk.jdwp.agent.gmk >>>>>>>>>>>> +++ b/make/lib/Lib-jdk.jdwp.agent.gmk >>>>>>>>>>>> @@ -37,6 +37,7 @@ $(eval $(call SetupJdkLibrary, >>>>>>>>>>>> BUILD_LIBDT_SOCKET, \ >>>>>>>>>>>> libjdwp/export, \ >>>>>>>>>>>> LDFLAGS := $(LDFLAGS_JDKLIB) \ >>>>>>>>>>>> $(call SET_SHARED_LIBRARY_ORIGIN), \ >>>>>>>>>>>> + LDFLAGS_windows := -export:jdwpTransport_OnLoad, \ >>>>>>>>>>>> LIBS_linux := -lpthread, \ >>>>>>>>>>>> LIBS_solaris := -lnsl -lsocket, \ >>>>>>>>>>>> LIBS_windows := $(JDKLIB_LIBS) ws2_32.lib, \ >>>>>>>>>>>> >>> >> > From merkel05 at gmail.com Mon Dec 10 21:12:21 2018 From: merkel05 at gmail.com (Sergei Ustimenko) Date: Mon, 10 Dec 2018 22:12:21 +0100 Subject: [PATCH] JDK-8025886: Replace [[ and == bash extensions in tests In-Reply-To: References: Message-ID: Hi Martin, That sounds good! I've counted all the sh-shebangs and it appears that there are at least 66 of them inside the test/ directory, where only 12 bashes. I've also ran the search in order to identify all the occurrences that use either [[ or == and found only three of them that use "==". That one for example: http://hg.openjdk.java.net/jdk/sandbox/file/f94276ccc9fc/test/hotspot/jtreg/vmTestbase/jit/tiered/tieredTest.sh#l63 of course `dash` reports failure in that case. So I'm quite hesitant in that case and not really sure what to do. I haven't also found any existent JBS ticket for such /bin/sh => /bin/bash a replacement. So any advise in this case would be appreciated! Regards, Sergei On Mon, 10 Dec 2018 at 18:32, Martin Buchholz wrote: > I would not try to remove all bash-isms from openjdk. Instead I would > find instances of /bin/sh that need to be changed to /bin/bash. > > (Ubuntu's use of /bin/sh -> /bin/dash is technically correct, but caused > much suffering > https://bugs.launchpad.net/ubuntu/+source/dash/+bug/61463 > ) > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ali.ince at gmail.com Mon Dec 10 21:13:57 2018 From: ali.ince at gmail.com (=?utf-8?B?QWxpIMSwbmNl?=) Date: Mon, 10 Dec 2018 21:13:57 +0000 Subject: [PATCH] JDK-8214122 Prevent name mangling of jdwpTransport_OnLoad in Windows 32-bit DLL name decoration In-Reply-To: <9d060b4d-bc2f-d402-d767-e5962df45aa6@oracle.com> References: <2DD922C6-EFB1-4891-90F0-3378DEA91B1C@gmail.com> <07840796-f1c4-e077-7b0b-cd5c1a880ec1@oracle.com> <00e58330-26f0-4401-f911-24fda0127bcc@oracle.com> <34d2b504-c4eb-6df3-3bd1-e23b9c22ea6a@oracle.com> <01b80c3f-8f71-26ec-304e-a7b245924bd5@oracle.com> <17e7873f-f2f4-dc02-82a6-2d7eb1565ca2@oracle.com> <5e032661-b70c-c6ec-2700-120589300c9e@oracle.com> <5A54BA77-B240-472A-A4B7-535C3A30A20F@gmail.com> <9d060b4d-bc2f-d402-d767-e5962df45aa6@oracle.com> Message-ID: <998A7FEB-9846-40F5-8880-1812302F002F@gmail.com> Hi Alexey, I?ve searched for |GetProcAddress| usages across source code and couldn?t find (hopefully tbh) other occurrences of such mismatches. Regards, Ali > On 7 Dec 2018, at 20:24, Alexey Ivanov wrote: > > Hi Ali, > > On 06/12/2018 22:49, Ali ?nce wrote: >> Hi Magnus, Alexey, >> >> I believe we won?t be able to get further opinions from serviceability-dev. > > Unfortunately, no one has replied as of now. > Have you found any issues with jdwpTransport_OnLoad after removing JNICALL? > >> Andrew Luo suggested using a similar mechanism as is used for jvm.dll by using symbol name files mapped by platform (files are under make/hotspot/symbols and interestingly windows is almost the only platform for which a symbol file doesn?t exist). > > Andrew says the .def files are auto-generated for Windows. There's a set of shared functions. > JVM cannot change the calling convention, jvm.dll is the public interface to JVM. > >> Another issue, again opened against AdoptOpenJDK 32-bit builds is somehow related with the same problem - but this time it is jimage.dll depending on zip.dll expecting the ZIP_InflateFully function to be decorated with JNICALL - whereas it was removed earlier from the implementation and the runtime image just fails with access violation just because jimage source code still declared it with JNICALL (https://github.com/AdoptOpenJDK/openjdk-build/issues/763 ). > > The usage of ZIP_InflateFully from zip.dll by jimage.dll was overlooked during work on https://bugs.openjdk.java.net/browse/JDK-8201226 . > > I can take care of it. > (I could not build 32 bit Windows. It seem to have overcome the problem by adding --disable-warnings-as-errors option to configure.) > > However, the report says the resulting image crashes in 64 bit windows too which shouldn't be affected by JNICALL mismatch. > >> I?ve also searched for GetProcAddress calls across the source code - and I think it?s important to work on the consistency of JNICALL usages across the whole source code. > > There are many places where libraries are loaded dynamically or a function may be unavailable on older versions of the platform. > Have you identified any other place where functions from JDK DLLs are looked up by names? > >> Another noteworthy thing I?ve noticed is there are still JNICALL modifiers for example in https://github.com/AdoptOpenJDK/openjdk-jdk11u/blob/master/src/jdk.crypto.cryptoki/windows/native/libj2pkcs11/j2secmod_md.c#L49 - which I guess will also be reported as broken since these functions are again name decorated. > > This is a JNI method. It should use JNICALL modifier. If it wasn't found, the class sun.security.pkcs11.Secmod would fail to load. I guess JVM handles name mangling somehow for native method implementation. > > Such functions were never explicitly exported using mapfiles or /export options on Windows, at least in the client area. For Linux and Solaris, adding or removing a native method required editing mapfiles. > >> If we?re still determined to remove JNICALL, what about just removing __stdcall from JNICALL declaration at https://github.com/AdoptOpenJDK/openjdk-jdk11u/blob/master/src/java.base/windows/native/include/jni_md.h#L31 ? Wouldn?t that be a more consistent move? > > We can't do that. > Implementation of Java native methods must use __stdcall calling convention. > >> >> Regards, >> >> Ali >> >>> On 22 Nov 2018, at 17:29, Magnus Ihse Bursie > wrote: >>> >>> I think we are in complete agreement. What's missing is some expert opinion from serviceability-dev if there is any problem with changing the signature. I'd preferably have that. >>> >>> If no one knows, I'd say, change it, and see if we still pass the relevant tests, and if so, ship it. >>> >>> /Magnus >>> >>> 22 nov. 2018 kl. 18:04 skrev Alexey Ivanov >: >>> >>>> >>>> On 21/11/2018 12:16, Magnus Ihse Bursie wrote: >>>>> >>>>> On 2018-11-20 16:49, Alexey Ivanov wrote: >>>>> >>>>>> Hi Ali, Magnus, >>>>>> >>>>>> I absolutely agree this change has to be reviewed by someone from serviceability. >>>>>> >>>>>> There are several options: >>>>>> >>>>>> 1. Add -export:jdwpTransport_OnLoad to LDFLAGS for Windows >>>>>> http://mail.openjdk.java.net/pipermail/build-dev/2018-November/023935.html >>>>>> so it partially reverts the changes from >>>>>> https://bugs.openjdk.java.net/browse/JDK-8200178 >>>>>> >>>>>> 2. Remove JNICALL (__stdcall) modifier, eventually changing the calling convention to __cdecl. >>>>>> http://mail.openjdk.java.net/pipermail/build-dev/2018-November/023969.html >>>>>> >>>>>> 3. Use inline /export option via #pragma: >>>>>> #pragma comment(linker, "/export:jdwpTransport_OnLoad= _jdwpTransport_OnLoad at 16") >>>>>> as referenced in >>>>>> https://docs.microsoft.com/en-ie/cpp/cpp/dllexport-dllimport?view=vs-2017 >>>>>> https://docs.microsoft.com/en-ie/cpp/build/reference/exports?view=vs-2017 >>>>>> >>>>>> The third options still needs to be tested to make sure it does not break other platforms builds. >>>>> I'm not fond of either solution 1 or 3. I really think the signature should be made correctly at the point of definition in the source code. >>>> >>>> That is why I proposed removing JNICALL (__stdcall) from the function declaration as we had done for libzip, libjimage [1] and mlib_image [2]. >>>> >>>> Microsoft recommends using __stdcall for DLLs, at the same time it decorates the function name making it harder to export it by its plain name. >>>> >>>> >>>> By removing JNICALL / __stdcall, we make the function use __cdecl calling convention. The difference between them is that with __stdcall the callee cleans the stack whereas with __cdecl the caller cleans the stack. >>>> >>>> It shouldn't be a problem as long as all function declarations use the same calling convention, which, I believe, is the case here. >>>> >>>>> We've spent some considerable effort of getting rid of the /export flags for Windows (and map files for unix), and I don't want to see them creep back in. Note that option 3 is just option 1 in disguise. The only single good thing about it is that it allows you to put the compiler option next to the signature declaration in the source code. >>>> >>>> At least in this case, the option will be near the function body definition. It will be easier to update it if the signature changes. >>>> >>>> If we are to use __stdcall, it seems to be the only way to export the undecorated name. >>>> >>>>> The same goes for def files, as suggested by Ali. It's just yet another version of option 1 in disguise/map files. >>>> >>>> If option 2 is rejected, I'm for using option 3. If option 3 cannot be used too, I'm for option 1. >>>> I think we should not fall back to def files in any case. And Makefile will have to include the reference to the def file, so it's even worse than option 1. >>>> >>>> >>>> Regards, >>>> Alexey >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8201226 >>>> [2] https://bugs.openjdk.java.net/browse/JDK-8202476 >>>>> >>>>> /Magnus >>>>> >>>>>> >>>>>> >>>>>> Regards, >>>>>> Alexey >>>>>> >>>>>> On 19/11/2018 12:19, Magnus Ihse Bursie wrote: >>>>>>> Hi Ali, >>>>>>> >>>>>>> From a build perspective this change looks OK. I'm not aware of the finer details on the OnLoad mechanism, like what calling convention is to be used. So maybe this is a no-go from that view. >>>>>>> >>>>>>> I'm cc:ing servicability so they can have a look at it. >>>>>>> >>>>>>> /Magnus >>>>>>> >>>>>>> On 2018-11-18 13:07, Ali ?nce wrote: >>>>>>>> Hi Alexey, >>>>>>>> >>>>>>>> Below is a new patch content that removes JNICALL modifiers from the exported functions of interest. This results in exported functions not to be name decorated and use __cdecl calling convention. >>>>>>>> >>>>>>>> Do you have any more suggestions, or would you please guide me on next steps? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Ali >>>>>>>> >>>>>>>> # HG changeset patch >>>>>>>> # User Ali Ince >>>>>>>> # Date 1542542415 0 >>>>>>>> # Sun Nov 18 12:00:15 2018 +0000 >>>>>>>> # Node ID a41df44e13e1b761f4b3f05a17ed18953067ae39 >>>>>>>> # Parent 16d5ec7dbbb49ef3f969e34be870e3f917ea89ff >>>>>>>> Remove JNICALL modifiers to prevent name decoration on 32-bit windows builds >>>>>>>> >>>>>>>> diff -r 16d5ec7dbbb4 -r a41df44e13e1 src/jdk.jdi/share/native/libdt_shmem/shmemBack.c >>>>>>>> --- a/src/jdk.jdi/share/native/libdt_shmem/shmemBack.c Tue Aug 14 14:28:23 2018 +0200 >>>>>>>> +++ b/src/jdk.jdi/share/native/libdt_shmem/shmemBack.c Sun Nov 18 12:00:15 2018 +0000 >>>>>>>> @@ -339,7 +339,7 @@ >>>>>>>> return JDWPTRANSPORT_ERROR_NONE; >>>>>>>> } >>>>>>>> >>>>>>>> -JNIEXPORT jint JNICALL >>>>>>>> +JNIEXPORT jint >>>>>>>> jdwpTransport_OnLoad(JavaVM *vm, jdwpTransportCallback* cbTablePtr, >>>>>>>> jint version, jdwpTransportEnv** result) >>>>>>>> { >>>>>>>> diff -r 16d5ec7dbbb4 -r a41df44e13e1 src/jdk.jdwp.agent/share/native/include/jdwpTransport.h >>>>>>>> --- a/src/jdk.jdwp.agent/share/native/include/jdwpTransport.h Tue Aug 14 14:28:23 2018 +0200 >>>>>>>> +++ b/src/jdk.jdwp.agent/share/native/include/jdwpTransport.h Sun Nov 18 12:00:15 2018 +0000 >>>>>>>> @@ -140,7 +140,7 @@ >>>>>>>> void (*free)(void *buffer); /* Call this for all deallocations */ >>>>>>>> } jdwpTransportCallback; >>>>>>>> >>>>>>>> -typedef jint (JNICALL *jdwpTransport_OnLoad_t)(JavaVM *jvm, >>>>>>>> +typedef jint (*jdwpTransport_OnLoad_t)(JavaVM *jvm, >>>>>>>> jdwpTransportCallback *callback, >>>>>>>> jint version, >>>>>>>> jdwpTransportEnv** env); >>>>>>>> diff -r 16d5ec7dbbb4 -r a41df44e13e1 src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c >>>>>>>> --- a/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c Tue Aug 14 14:28:23 2018 +0200 >>>>>>>> +++ b/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c Sun Nov 18 12:00:15 2018 +0000 >>>>>>>> @@ -1019,7 +1019,7 @@ >>>>>>>> return JDWPTRANSPORT_ERROR_NONE; >>>>>>>> } >>>>>>>> >>>>>>>> -JNIEXPORT jint JNICALL >>>>>>>> +JNIEXPORT jint >>>>>>>> jdwpTransport_OnLoad(JavaVM *vm, jdwpTransportCallback* cbTablePtr, >>>>>>>> jint version, jdwpTransportEnv** env) >>>>>>>> { >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> On 16 Nov 2018, at 23:03, Alexey Ivanov wrote: >>>>>>>>> >>>>>>>>> Hi Ali, >>>>>>>>> >>>>>>>>> It's exactly what I referred to. >>>>>>>>> >>>>>>>>> Yes, it does change the calling convention. >>>>>>>>> Yet it's not a (big) problem because both parties will use the same calling convention. >>>>>>>>> >>>>>>>>> Using a DLL from another build will not be possible. But it's not a good idea to do so anyway. >>>>>>>>> >>>>>>>>> >>>>>>>>> Mapfiles and export options have been removed by https://bugs.openjdk.java.net/browse/JDK-8200178 to simplify managing exported functions. So that to add or remove an exported function, you don't have modify makefiles and/or mapfiles. You just edit the source files. >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Alexey >>>>>>>>> >>>>>>>>> On 16/11/2018 22:42, Ali ?nce wrote: >>>>>>>>>> Hi Alexey, >>>>>>>>>> >>>>>>>>>> Thanks for your reply. >>>>>>>>>> >>>>>>>>>> I will definitely give it a try though I?m not sure if that?ll be a breaking change. This is because JNICALL modifier is defined to be __stdcall on windows which is both specified on jdwpTransport_OnLoad exported functions both for dt_socket.dll and dt_shmem.dll. >>>>>>>>>> >>>>>>>>>> The __stdcall is ignored on x64 platforms and also name decoration is not applied (https://docs.microsoft.com/en-gb/cpp/build/reference/decorated-names?view=vs-2017 ). I believe that?s why we don?t experience any problems on x64 builds. >>>>>>>>>> >>>>>>>>>> Removing JNICALL (thus __stdcall) modifier (apparently only applies to x86 builds) will result in the calling convention to be changed, and thus will change how parameters are ordered and also the stack cleanup rules. I think this may result in problems in programs that are designed to load shared libraries and its exported functions at runtime (not bound by link time which probably won?t cause any issues - assuming that they use the shared function signature) - as in https://github.com/AdoptOpenJDK/openjdk-jdk11u/blob/5f01925b80ed851b133ee26fbcb07026ac04149e/src/jdk.jdwp.agent/share/native/libjdwp/transport.c#L99 . >>>>>>>>>> >>>>>>>>>> Any thoughts? >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> >>>>>>>>>> Ali >>>>>>>>>> >>>>>>>>>>> On 15 Nov 2018, at 22:14, Alexey Ivanov wrote: >>>>>>>>>>> >>>>>>>>>>> Hi Ali, >>>>>>>>>>> >>>>>>>>>>> Can the issue be resolved by using different call modifiers on the affected functions? >>>>>>>>>>> >>>>>>>>>>> Some build / link issues were resolved by adding / removing JNICALL modifier, see >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8201226 >>>>>>>>>>> http://mail.openjdk.java.net/pipermail/build-dev/2018-April/021553.html >>>>>>>>>>> >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8202476 >>>>>>>>>>> http://mail.openjdk.java.net/pipermail/build-dev/2018-May/021913.html >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> Alexey >>>>>>>>>>> >>>>>>>>>>> On 15/11/2018 21:43, Ali ?nce wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> An issue (https://github.com/AdoptOpenJDK/openjdk-build/issues/709 ) raised against AdoptOpenJDK jdk11 32-bit windows builds led us to the name decoration problem on built 32-bit dlls - specifically dt_socket.dll and dt_shmem.dll. Whereas 64-bit MSVC builds don?t apply any name decorations on exported functions, 32-bit builds apply them - which requires either def files or /export flags to be specified on the linker arguments. >>>>>>>>>>>> >>>>>>>>>>>> Although the expected linker arguments were present on previous jdk makefiles, they were removed in jdk11 makefiles. >>>>>>>>>>>> >>>>>>>>>>>> Below is the proposed patch, which can also be browsed at https://github.com/AdoptOpenJDK/openjdk-jdk11u/pull/1 . >>>>>>>>>>>> >>>>>>>>>>>> Would you please have a look and let me know of any points I may have missed (I don?t have any background information about why the export flags were removed in jdk11)? >>>>>>>>>>>> >>>>>>>>>>>> Regards, >>>>>>>>>>>> >>>>>>>>>>>> Ali >>>>>>>>>>>> >>>>>>>>>>>> diff --git a/make/lib/Lib-jdk.jdi.gmk b/make/lib/Lib-jdk.jdi.gmk >>>>>>>>>>>> index 197b95c2e2..ac6ebf5591 100644 >>>>>>>>>>>> --- a/make/lib/Lib-jdk.jdi.gmk >>>>>>>>>>>> +++ b/make/lib/Lib-jdk.jdi.gmk >>>>>>>>>>>> @@ -37,6 +37,7 @@ ifeq ($(OPENJDK_TARGET_OS), windows) >>>>>>>>>>>> jdk.jdwp.agent:include \ >>>>>>>>>>>> jdk.jdwp.agent:libjdwp/export, \ >>>>>>>>>>>> LDFLAGS := $(LDFLAGS_JDKLIB), \ >>>>>>>>>>>> + LDFLAGS_windows := -export:jdwpTransport_OnLoad, \ >>>>>>>>>>>> LIBS := $(JDKLIB_LIBS), \ >>>>>>>>>>>> )) >>>>>>>>>>>> >>>>>>>>>>>> diff --git a/make/lib/Lib-jdk.jdwp.agent.gmk b/make/lib/Lib-jdk.jdwp.agent.gmk >>>>>>>>>>>> index 0bc93e0d35..0382e55325 100644 >>>>>>>>>>>> --- a/make/lib/Lib-jdk.jdwp.agent.gmk >>>>>>>>>>>> +++ b/make/lib/Lib-jdk.jdwp.agent.gmk >>>>>>>>>>>> @@ -37,6 +37,7 @@ $(eval $(call SetupJdkLibrary, BUILD_LIBDT_SOCKET, \ >>>>>>>>>>>> libjdwp/export, \ >>>>>>>>>>>> LDFLAGS := $(LDFLAGS_JDKLIB) \ >>>>>>>>>>>> $(call SET_SHARED_LIBRARY_ORIGIN), \ >>>>>>>>>>>> + LDFLAGS_windows := -export:jdwpTransport_OnLoad, \ >>>>>>>>>>>> LIBS_linux := -lpthread, \ >>>>>>>>>>>> LIBS_solaris := -lnsl -lsocket, \ >>>>>>>>>>>> LIBS_windows := $(JDKLIB_LIBS) ws2_32.lib, \ >>>>>>> >>>>>> >>>>> >>>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrb at google.com Mon Dec 10 21:23:01 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 10 Dec 2018 13:23:01 -0800 Subject: RFR (L) 8215161: Remove spaces in assignments for vmTestbase/[a-j] In-Reply-To: References: Message-ID: The bug title says Add spaces but the description says Remove spaces On Mon, Dec 10, 2018 at 1:04 PM JC Beyler wrote: > Hi all, > > Could I get a review that adds a space around comparisons for the > vmTestbase? This is the first of two webrevs to handle this. > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8215161/webrev.00/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8215161 > > Thanks, > Jc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrb at google.com Mon Dec 10 21:27:36 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 10 Dec 2018 13:27:36 -0800 Subject: [PATCH] JDK-8025886: Replace [[ and == bash extensions in tests In-Reply-To: References: Message-ID: I don't know if there's an official policy on how ultra-portable tests are supposed to be. In practice, you probably won't be able to build openjdk on a system without bash. On Mon, Dec 10, 2018 at 1:12 PM Sergei Ustimenko wrote: > Hi Martin, > > That sounds good! > > I've counted all the sh-shebangs and it appears that > there are at least 66 of them inside the test/ directory, > where only 12 bashes. > > I've also ran the search in order to identify all the > occurrences that use either [[ or == and found only > three of them that use "==". That one for example: > > http://hg.openjdk.java.net/jdk/sandbox/file/f94276ccc9fc/test/hotspot/jtreg/vmTestbase/jit/tiered/tieredTest.sh#l63 > of course `dash` reports failure in that case. > > So I'm quite hesitant in that case and not really sure > what to do. I haven't also found any existent JBS ticket > for such /bin/sh => /bin/bash a replacement. > > So any advise in this case would be appreciated! > > Regards, > Sergei > > On Mon, 10 Dec 2018 at 18:32, Martin Buchholz wrote: > >> I would not try to remove all bash-isms from openjdk. Instead I would >> find instances of /bin/sh that need to be changed to /bin/bash. >> >> (Ubuntu's use of /bin/sh -> /bin/dash is technically correct, but caused >> much suffering >> https://bugs.launchpad.net/ubuntu/+source/dash/+bug/61463 >> ) >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jcbeyler at google.com Mon Dec 10 21:28:02 2018 From: jcbeyler at google.com (JC Beyler) Date: Mon, 10 Dec 2018 13:28:02 -0800 Subject: RFR (L) 8215161: Remove spaces in assignments for vmTestbase/[a-j] In-Reply-To: References: Message-ID: Hi Martin, I know, Claes noticed it too and I modified the title in the bug and the webrevs but the RFR is out. Should I send a new email thread for 's/Remove/Add' ? Jc On Mon, Dec 10, 2018 at 1:23 PM Martin Buchholz wrote: > The bug title says > Add spaces > but the description says > Remove spaces > > On Mon, Dec 10, 2018 at 1:04 PM JC Beyler wrote: > >> Hi all, >> >> Could I get a review that adds a space around comparisons for the >> vmTestbase? This is the first of two webrevs to handle this. >> >> Webrev: http://cr.openjdk.java.net/~jcbeyler/8215161/webrev.00/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8215161 >> >> Thanks, >> Jc >> > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrb at google.com Mon Dec 10 21:31:51 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 10 Dec 2018 13:31:51 -0800 Subject: RFR (L) 8215161: Remove spaces in assignments for vmTestbase/[a-j] In-Reply-To: References: Message-ID: On Mon, Dec 10, 2018 at 1:28 PM JC Beyler wrote: > > I know, Claes noticed it too and I modified the title in the bug and the > webrevs but the RFR is out. > You should modify the bug Description to agree with the bug title. -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Mon Dec 10 21:39:20 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 10 Dec 2018 13:39:20 -0800 Subject: RFR (L) 8215161: Remove spaces in assignments for vmTestbase/[a-j] In-Reply-To: References: Message-ID: An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Mon Dec 10 21:44:17 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 10 Dec 2018 13:44:17 -0800 Subject: RFR(L) 8212960: Remove spaces in assignments for remaining vmTestbase In-Reply-To: References: Message-ID: <24e7f37c-828a-c137-c743-568bef0083cd@oracle.com> An HTML attachment was scrubbed... URL: From jcbeyler at google.com Mon Dec 10 21:46:32 2018 From: jcbeyler at google.com (JC Beyler) Date: Mon, 10 Dec 2018 13:46:32 -0800 Subject: RFR (L) 8215160: Normalize spaces for remaining vmTestbase tests Message-ID: Hi all, Let's try this again; my apologies for the spam. Could I get a review that normalizes spaces around comparisons and the ternary operator? This is the second of two webrevs to handle this. Webrev: http://cr.openjdk.java.net/~jcbeyler/8215160/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8215160 Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From jcbeyler at google.com Mon Dec 10 21:48:02 2018 From: jcbeyler at google.com (JC Beyler) Date: Mon, 10 Dec 2018 13:48:02 -0800 Subject: RFR (L) 8215161: Normalize spaces for vmTestbase/[a-j] Message-ID: Hi all, Let's try this again; my apologies for the spam. Could I get a review that normalizes spaces around comparisons and the ternary operator? This is the first of two webrevs to handle this. Webrev: http://cr.openjdk.java.net/~jcbeyler/8215161/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8215161 Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrb at google.com Mon Dec 10 21:49:41 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 10 Dec 2018 13:49:41 -0800 Subject: RFR (L) 8215161: Remove spaces in assignments for vmTestbase/[a-j] In-Reply-To: References: Message-ID: Looks good to me. Make sure to "test" using hg diff -wbB On Mon, Dec 10, 2018 at 1:04 PM JC Beyler wrote: > Hi all, > > Could I get a review that adds a space around comparisons for the > vmTestbase? This is the first of two webrevs to handle this. > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8215161/webrev.00/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8215161 > > Thanks, > Jc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Mon Dec 10 21:54:05 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 10 Dec 2018 13:54:05 -0800 Subject: RFR (L) 8215160: Normalize spaces for remaining vmTestbase tests In-Reply-To: References: Message-ID: <7de505f5-4407-7961-1b71-0def2559f0bb@oracle.com> An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Mon Dec 10 21:54:29 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 10 Dec 2018 13:54:29 -0800 Subject: RFR (L) 8215161: Normalize spaces for vmTestbase/[a-j] In-Reply-To: References: Message-ID: An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Mon Dec 10 21:56:28 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Dec 2018 07:56:28 +1000 Subject: [PATCH] JDK-8025886: Replace [[ and == bash extensions in tests In-Reply-To: References: Message-ID: <240ce837-623f-e869-7583-dda59e30f63f@oracle.com> On 11/12/2018 7:27 am, Martin Buchholz wrote: > I don't know if there's an official policy on how ultra-portable tests > are supposed?to be.? In practice, you probably won't be able to build > openjdk on a system without bash. configure must be run with bash; so no bash, no build. David > On Mon, Dec 10, 2018 at 1:12 PM Sergei Ustimenko > wrote: > > Hi Martin, > > That sounds good! > > I've counted all the sh-shebangs and it appears that > there are at least 66 of them inside the test/ directory, > where only 12 bashes. > > I've also ran the search? in order to identify all the > occurrences that use either [[ or == and found only > three of them that use "==". That one for example: > http://hg.openjdk.java.net/jdk/sandbox/file/f94276ccc9fc/test/hotspot/jtreg/vmTestbase/jit/tiered/tieredTest.sh#l63 > of course `dash` reports failure in that case. > > So I'm quite hesitant in that case and not really sure > what to do. I haven't also found any existent JBS ticket > for such /bin/sh => /bin/bash a replacement. > > So any advise in this case?would be appreciated! > > Regards, > Sergei > > On Mon, 10 Dec 2018 at 18:32, Martin Buchholz > wrote: > > I would not try to remove all bash-isms from openjdk. Instead I > would find instances of /bin/sh that need to be changed to > /bin/bash. > > (Ubuntu's use of /bin/sh -> /bin/dash is technically correct, > but caused much suffering > https://bugs.launchpad.net/ubuntu/+source/dash/+bug/61463 > ) > From martinrb at google.com Mon Dec 10 22:03:52 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 10 Dec 2018 14:03:52 -0800 Subject: RFR (L) 8215161: Normalize spaces for vmTestbase/[a-j] In-Reply-To: References: Message-ID: Looks good to me. On Mon, Dec 10, 2018 at 1:49 PM JC Beyler wrote: > Hi all, > > Let's try this again; my apologies for the spam. > > Could I get a review that normalizes spaces around comparisons and the > ternary operator? This is the first of two webrevs to handle this. > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8215161/webrev.00/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8215161 > > Thanks, > Jc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From merkel05 at gmail.com Mon Dec 10 22:06:33 2018 From: merkel05 at gmail.com (Sergei Ustimenko) Date: Mon, 10 Dec 2018 23:06:33 +0100 Subject: [PATCH] JDK-8025886: Replace [[ and == bash extensions in tests In-Reply-To: References: Message-ID: Martin, >you probably won't be able to build openjdk on a system without bash Most certainly not, it appears I really wonder now if there's any policy/rule of thumb to not to use /bin/sh. I don't feel really comfortable if there would be such a policy. I've checked build docs and it seems that troubleshooting section could be a bit enhanced, though I guess that is given, that user knows how to change a /bin/sh symlink. So I don't know if it worth to do this. Regards, Sergei On Mon, 10 Dec 2018 at 22:27, Martin Buchholz wrote: > I don't know if there's an official policy on how ultra-portable tests are > supposed to be. In practice, you probably won't be able to build openjdk > on a system without bash. > > On Mon, Dec 10, 2018 at 1:12 PM Sergei Ustimenko > wrote: > >> Hi Martin, >> >> That sounds good! >> >> I've counted all the sh-shebangs and it appears that >> there are at least 66 of them inside the test/ directory, >> where only 12 bashes. >> >> I've also ran the search in order to identify all the >> occurrences that use either [[ or == and found only >> three of them that use "==". That one for example: >> >> http://hg.openjdk.java.net/jdk/sandbox/file/f94276ccc9fc/test/hotspot/jtreg/vmTestbase/jit/tiered/tieredTest.sh#l63 >> of course `dash` reports failure in that case. >> >> So I'm quite hesitant in that case and not really sure >> what to do. I haven't also found any existent JBS ticket >> for such /bin/sh => /bin/bash a replacement. >> >> So any advise in this case would be appreciated! >> >> Regards, >> Sergei >> >> On Mon, 10 Dec 2018 at 18:32, Martin Buchholz >> wrote: >> >>> I would not try to remove all bash-isms from openjdk. Instead I would >>> find instances of /bin/sh that need to be changed to /bin/bash. >>> >>> (Ubuntu's use of /bin/sh -> /bin/dash is technically correct, but caused >>> much suffering >>> https://bugs.launchpad.net/ubuntu/+source/dash/+bug/61463 >>> ) >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexey.menkov at oracle.com Mon Dec 10 22:28:55 2018 From: alexey.menkov at oracle.com (Alex Menkov) Date: Mon, 10 Dec 2018 14:28:55 -0800 Subject: RFR (L) 8215160: Normalize spaces for remaining vmTestbase tests In-Reply-To: <7de505f5-4407-7961-1b71-0def2559f0bb@oracle.com> References: <7de505f5-4407-7961-1b71-0def2559f0bb@oracle.com> Message-ID: <63e7d134-1c24-ac13-ab4e-57e75df72b4e@oracle.com> +1 One minor request. Could you please fix nsk/jvmti/NativeMethodBind/nativemethbind002/nativemethbind002.cpp replacing if (!(methNam == NULL)) and if (!(methSig == NULL)) with if (methNam != NULL) and if (methSig != NULL) No need for new round of the review. --alex On 12/10/2018 13:54, serguei.spitsyn at oracle.com wrote: > Hi Jc, > > LGTM > > Thank you for the re-post! > Serguei > > > On 12/10/18 13:46, JC Beyler wrote: >> Hi all, >> >> Let's try this again; my apologies for the spam. >> >> Could I get a review that normalizes spaces around comparisons and the >> ternary operator? This is the second of two webrevs to handle this. >> >> Webrev: http://cr.openjdk.java.net/~jcbeyler/8215160/webrev.00/ >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8215160 >> >> Thanks, >> Jc >> > From alexey.menkov at oracle.com Tue Dec 11 00:41:05 2018 From: alexey.menkov at oracle.com (Alex Menkov) Date: Mon, 10 Dec 2018 16:41:05 -0800 Subject: RFR (L) 8215160: Normalize spaces for remaining vmTestbase tests In-Reply-To: <63e7d134-1c24-ac13-ab4e-57e75df72b4e@oracle.com> References: <7de505f5-4407-7961-1b71-0def2559f0bb@oracle.com> <63e7d134-1c24-ac13-ab4e-57e75df72b4e@oracle.com> Message-ID: <06aafed7-d05f-3207-f378-62797506637f@oracle.com> Sorry, it was review for 8215161 Some minor notes for this webrev: - nsk/jvmti/scenarios/jni_interception/JI05/ji05t001/ji05t001.cpp for some reason there are missed spaces before "?" in statements like (indx == 0)? "A" : "B" see lines 190, 227, 243 Also there is some inconsistency with spaces around "%" operator - in nsk/stress/jni/libjnistress001.cpp and nsk/stress/jni/libjnistress003.cpp there are no spaces like + if (allocs%printperiod == 0) { in other files the spaces are added like (nsk/stress/jni/libjnistress005.cpp) + if (Exceptcalls % 1000 == 0) --alex On 12/10/2018 14:28, Alex Menkov wrote: > +1 > > One minor request. > Could you please fix > nsk/jvmti/NativeMethodBind/nativemethbind002/nativemethbind002.cpp > replacing > > if (!(methNam == NULL)) > and > if (!(methSig == NULL)) > > with > if (methNam != NULL) > and > if (methSig != NULL) > > No need for new round of the review. > > --alex > > > On 12/10/2018 13:54, serguei.spitsyn at oracle.com wrote: >> Hi Jc, >> >> LGTM >> >> Thank you for the re-post! >> Serguei >> >> >> On 12/10/18 13:46, JC Beyler wrote: >>> Hi all, >>> >>> Let's try this again; my apologies for the spam. >>> >>> Could I get a review that normalizes spaces around comparisons and >>> the ternary operator? This is the second of two webrevs to handle this. >>> >>> Webrev: http://cr.openjdk.java.net/~jcbeyler/8215160/webrev.00/ >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8215160 >>> >>> Thanks, >>> Jc >>> >> From jcbeyler at google.com Tue Dec 11 04:39:01 2018 From: jcbeyler at google.com (JC Beyler) Date: Mon, 10 Dec 2018 20:39:01 -0800 Subject: RFR (L) 8215160: Normalize spaces for remaining vmTestbase tests In-Reply-To: <06aafed7-d05f-3207-f378-62797506637f@oracle.com> References: <7de505f5-4407-7961-1b71-0def2559f0bb@oracle.com> <63e7d134-1c24-ac13-ab4e-57e75df72b4e@oracle.com> <06aafed7-d05f-3207-f378-62797506637f@oracle.com> Message-ID: Hi Alexey, Thanks for the review! Here is the fixed version: Webrev: http://cr.openjdk.java.net/~jcbeyler/8215160/webrev.01/ Bug: https://bugs.openjdk.java.net/browse/JDK-8215160 Thanks again, Jc On Mon, Dec 10, 2018 at 4:41 PM Alex Menkov wrote: > Sorry, it was review for 8215161 > > Some minor notes for this webrev: > - nsk/jvmti/scenarios/jni_interception/JI05/ji05t001/ji05t001.cpp > for some reason there are missed spaces before "?" in statements like > (indx == 0)? "A" : "B" > see lines 190, 227, 243 > > Also there is some inconsistency with spaces around "%" operator - > in nsk/stress/jni/libjnistress001.cpp and > nsk/stress/jni/libjnistress003.cpp > there are no spaces like > + if (allocs%printperiod == 0) { > > in other files the spaces are added > like (nsk/stress/jni/libjnistress005.cpp) > + if (Exceptcalls % 1000 == 0) > > --alex > > On 12/10/2018 14:28, Alex Menkov wrote: > > +1 > > > > One minor request. > > Could you please fix > > nsk/jvmti/NativeMethodBind/nativemethbind002/nativemethbind002.cpp > > replacing > > > > if (!(methNam == NULL)) > > and > > if (!(methSig == NULL)) > > > > with > > if (methNam != NULL) > > and > > if (methSig != NULL) > > > > No need for new round of the review. > > > > --alex > > > > > > On 12/10/2018 13:54, serguei.spitsyn at oracle.com wrote: > >> Hi Jc, > >> > >> LGTM > >> > >> Thank you for the re-post! > >> Serguei > >> > >> > >> On 12/10/18 13:46, JC Beyler wrote: > >>> Hi all, > >>> > >>> Let's try this again; my apologies for the spam. > >>> > >>> Could I get a review that normalizes spaces around comparisons and > >>> the ternary operator? This is the second of two webrevs to handle this. > >>> > >>> Webrev: http://cr.openjdk.java.net/~jcbeyler/8215160/webrev.00/ > >>> > >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8215160 > >>> > >>> Thanks, > >>> Jc > >>> > >> > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Tue Dec 11 04:50:50 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Dec 2018 14:50:50 +1000 Subject: RFR 8215050: [TESTBUG] serviceability/tmtools/jstack/WaitNotifyThreadTest.java fails when run with flag -Xcomp In-Reply-To: References: Message-ID: <8a619bbc-01d8-ac68-09a7-95bc7ccfa027@oracle.com> This is still failing in tier4: Error: incorrect monitor info: locked, a java.lang.Object, 0x000000078653eeb0 Expected: locked, a java.lang.Object, no object reference available Looks like the presence/absence of an address is contingent on other things ?? "MyWaitingThread" #13 prio=5 os_prio=0 cpu=0.27ms elapsed=2.63s tid=0x00007f522c009000 nid=0x62d0 in Object.wait() [0x00007f526537d000] java.lang.Thread.State: WAITING (on object monitor) Thread: 0x00007f522c009000 [0x62d0] State: _at_safepoint _has_called_back 0 _at_poll_safepoint 0 JavaThread state: _thread_blocked at java.lang.Object.wait(java.base at 12-internal/Native Method) - waiting on at java.lang.Object.wait(java.base at 12-internal/Object.java:328) at WaitNotifyThreadTest$WaitThread.run(WaitNotifyThreadTest.java:80) - locked <0x000000078653eeb0> (a java.lang.Object) Or maybe there is a bug in the stack printing code thats printing the information? Filed: https://bugs.openjdk.java.net/browse/JDK-8215199 David ----- On 9/12/2018 9:21 am, David Holmes wrote: > Hi Patricio, > > On 9/12/2018 9:04 am, Patricio Chilano wrote: >> Hi, >> >> Could you review this small fix for test >> serviceability/tmtools/jstack/WaitNotifyThreadTest.java ? >> After change 8214148 the test fails if the flag -Xcomp is used as >> explained in the bug details. The proposed change is to identified >> this special case and set the monitor address to match the one expected. >> >> http://cr.openjdk.java.net/~pchilanomate/8215050.02/webrev/ >> https://bugs.openjdk.java.net/browse/JDK-8215050 >> >> Run tier-4 where the test was failing and passed (the test was failing >> on tier6 also because it uses -Xcomp too). Currently running tiers1-3. > > Change seems fine. The lack of address must be something relatively new > as I don't see it with Xcomp in JDK 9. > > Thanks, > David > >> Thanks, >> Patricio From david.holmes at oracle.com Tue Dec 11 04:54:29 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Dec 2018 14:54:29 +1000 Subject: RFR 8215050: [TESTBUG] serviceability/tmtools/jstack/WaitNotifyThreadTest.java fails when run with flag -Xcomp In-Reply-To: <8a619bbc-01d8-ac68-09a7-95bc7ccfa027@oracle.com> References: <8a619bbc-01d8-ac68-09a7-95bc7ccfa027@oracle.com> Message-ID: <30d309df-d7b6-c4b2-c543-71e57190d0d8@oracle.com> Sorry please ignore - "off by one error" checking the CI results. This bug was fixed in 1097 and I was looking at 1096. David On 11/12/2018 2:50 pm, David Holmes wrote: > This is still failing in tier4: > > Error: incorrect monitor info: locked, a java.lang.Object, > 0x000000078653eeb0 > Expected: locked, a java.lang.Object, no object reference available > > Looks like the presence/absence of an address is contingent on other > things ?? > > "MyWaitingThread" #13 prio=5 os_prio=0 cpu=0.27ms elapsed=2.63s > tid=0x00007f522c009000 nid=0x62d0 in Object.wait()? [0x00007f526537d000] > ?? java.lang.Thread.State: WAITING (on object monitor) > Thread: 0x00007f522c009000? [0x62d0] State: _at_safepoint > _has_called_back 0 _at_poll_safepoint 0 > ?? JavaThread state: _thread_blocked > ????at java.lang.Object.wait(java.base at 12-internal/Native Method) > ????- waiting on > ????at java.lang.Object.wait(java.base at 12-internal/Object.java:328) > ????at WaitNotifyThreadTest$WaitThread.run(WaitNotifyThreadTest.java:80) > ????- locked <0x000000078653eeb0> (a java.lang.Object) > > Or maybe there is a bug in the stack printing code thats printing the > information? > > Filed: https://bugs.openjdk.java.net/browse/JDK-8215199 > > David > ----- > > On 9/12/2018 9:21 am, David Holmes wrote: >> Hi Patricio, >> >> On 9/12/2018 9:04 am, Patricio Chilano wrote: >>> Hi, >>> >>> Could you review this small fix for test >>> serviceability/tmtools/jstack/WaitNotifyThreadTest.java ? >>> After change 8214148 the test fails if the flag -Xcomp is used as >>> explained in the bug details. The proposed change is to identified >>> this special case and set the monitor address to match the one expected. >>> >>> http://cr.openjdk.java.net/~pchilanomate/8215050.02/webrev/ >>> https://bugs.openjdk.java.net/browse/JDK-8215050 >>> >>> Run tier-4 where the test was failing and passed (the test was >>> failing on tier6 also because it uses -Xcomp too). Currently running >>> tiers1-3. >> >> Change seems fine. The lack of address must be something relatively >> new as I don't see it with Xcomp in JDK 9. >> >> Thanks, >> David >> >>> Thanks, >>> Patricio From jini.george at oracle.com Tue Dec 11 07:37:29 2018 From: jini.george at oracle.com (Jini George) Date: Tue, 11 Dec 2018 13:07:29 +0530 Subject: RFR: (S): JDK-8200613: SA: jstack throws UnmappedAddressException with a CDS core file In-Reply-To: References: <864997e5-2581-7ac9-44f7-76cc31c9ad93@oracle.com> <60d2e6db-c87b-9658-57cb-b03ca881a624@oracle.com> <9eb9041d-f984-0df4-a21e-4645966a62b8@oracle.com> <42d6d604-8df5-47c4-a748-6aa25bfc9a7e@oracle.com> <2dbef041-c75e-07f6-fafc-d18f2a95f455@oracle.com> <798a4d93-1638-8785-885f-fac0b21c2417@oracle.com> <2c6a2d67-5d36-07f1-4563-c1cd2ecdf9a3@oracle.com> Message-ID: <536d23dc-e9bf-86d9-0c86-fb15732576ea@oracle.com> Thank you very much, Coleen. I have converted the flag into a diagnostic flag. The revised webrev is at: http://cr.openjdk.java.net/~jgeorge/8200613/webrev.03/ A plain diff between this patch and the earlier one is at: http://cr.openjdk.java.net/~jgeorge/8200613/incremental_diff I will withdraw the associated CSR. Thanks! Jini. On 12/10/2018 11:55 PM, coleen.phillimore at oracle.com wrote: > > I see.? In this case, the flag should be diagnostic and not require a > CSR.? I suppose some documentation should be added so that sustaining > will know about the option in this case (Hi Kevin!) > > thanks, > Coleen > > > On 12/10/18 1:38 AM, Ioi Lam wrote: >> Hi Coleen, >> >> I was one of the people who suggested the DumpPrivateMappingsInCore >> flag. It's enabled by default, so by default all the contents of >> mmap'ed files with MAP_PRIVATE will be saved to the core files. >> >> The worry is, there may be some extreme cases where the JVM has mapped >> very large files (with NIO or JNI, etc). For example, you could have a >> 100GB in-memory database. For those cases, if the user is experiencing >> crashes, but they are unable to get a core dump (because it would be >> too large), they can try running with -XX:-DumpPrivateMappingsInCore. >> >> Thanks >> >> - Ioi >> >> >> On 12/7/18 12:03 PM, coleen.phillimore at oracle.com wrote: >>> >>> Hi Jini,? We were just talking about this new option.? If someone >>> gets a crash, I don't think they're going to run their application >>> again with -XX:-DumpPrivateMappingsInCore in the case of the core >>> file being too large.? So I don't know how generally useful this >>> option is. I think it would be better to not add it and set the bit >>> to include the mappings unconditionally. >>> >>> How much larger is this core file (we had trouble understanding from >>> the bug)?? If you need the mappings to understand and use the SA >>> tools on it, we want to have them. >>> >>> Thanks, >>> Coleen >>> >>> >>> On 12/7/18 2:22 PM, Jini George wrote: >>>> I have the revised webrev here: >>>> >>>> http://cr.openjdk.java.net/~jgeorge/8200613/webrev.02/index.html >>>> >>>> The extra changes here are to: >>>> >>>> * Introduce a new option DumpPrivateMappingsInCore to control the >>>> dumping of the file backed private regions into the corefile. >>>> >>>> * Close the modules file before dumping core in os::abort(). >>>> Currently, there is a small bug >>>> (https://bugs.openjdk.java.net/browse/JDK-8215026) which prevents >>>> the closure of the image file in unmapping the entire file. >>>> >>>> I plan to take up the unmapping of NIO MapMode.PRIVATE files as a >>>> separate task (https://bugs.openjdk.java.net/browse/JDK-8215027) >>>> since this seems a bit involved. >>>> >>>> Thanks a bunch, >>>> Jini. >>>> >>>> On 11/12/2018 10:26 AM, Jini George wrote: >>>>> Thank you very much, Chris, Kevin and Ioi for your comments! >>>>> >>>>> I will send another webrev with this change enabled under an >>>>> opt-out flag, as you suggest, and would look at unmapping the JDK >>>>> modules file and if possible, the NIO mapped files too in the >>>>> signal handler. >>>>> >>>>> Thanks a bunch, >>>>> Jini. >>>>> >>>>> On 11/9/2018 11:53 PM, Ioi Lam wrote: >>>>>> Hi Jini, >>>>>> >>>>>> Thanks for investigating the size expansion issue. >>>>>> >>>>>> I agree that the size increase is worth it. Even when not using >>>>>> SA, if we open the core file inside GDB, we cannot read certain >>>>>> sections in the CDS archive (such as the RO section and strings >>>>>> sections). That would make debugging difficult. So I am in favor >>>>>> of this change. >>>>>> >>>>>> For the JDK modules file, maybe we can unmap it in the signal >>>>>> handler, before going ahead with the core dump? I think it's >>>>>> hardly needed for debugging purposes. (Perhaps we can also do the >>>>>> same for the NIO mapped files?) >>>>>> >>>>>> A opt-flag as suggested by Kevin is a good idea. >>>>>> >>>>>> Thanks >>>>>> >>>>>> - Ioi >>>>>> >>>>>> On 11/9/18 3:29 AM, Kevin Walls wrote: >>>>>>> Hi Jini, >>>>>>> >>>>>>> Looks good to me.? It might be a significant increase in size of >>>>>>> _some_ core files, but so many core files we see are much larger, >>>>>>> in gigabytes++ of course, so the CDS data size should not be such >>>>>>> a significant increase on (I think) most files. >>>>>>> >>>>>>> The flexibiity of always having the CDS data there is very >>>>>>> significant.? A core file should ideally be usable, without >>>>>>> additionally requiring the CDS archive from the machine. That >>>>>>> additional human round-trip upload request on every transmitted >>>>>>> core that needs investigating, seems like a less efficient >>>>>>> route...). >>>>>>> >>>>>>> Is there an opt-out?? It's conditional on UseSharedSpaces but >>>>>>> could there be a flag to disable, in case we see crashes with >>>>>>> gigabytes of private mappings that we really don't want to retain >>>>>>> (the user would have to know to set a flag, to disable the new >>>>>>> coredump filter ahead of time). >>>>>>> >>>>>>> Thanks! >>>>>>> Kevin >>>>>>> >>>>>>> >>>>>>> On 29/10/2018 06:02, Jini George wrote: >>>>>>>> Thank you very much, Ioi, for looking into this, and the >>>>>>>> clarification offline. My bad, I had missed the earlier mail >>>>>>>> from you. :-( My responses below. >>>>>>>> >>>>>>>> Yes, I had tested this on MacOS. The issue does not exist on >>>>>>>> MacOS since the file backed private mmap()-ed regions get dumped >>>>>>>> into the MacOS corefiles by default. >>>>>>>> >>>>>>>> The corefile sizes on Linux do increase due to this change. And >>>>>>>> the increase would also include any file mapped using NIO with >>>>>>>> MapMode.PRIVATE. The typical corefile size increase with this >>>>>>>> change would include the following components at a high level: >>>>>>>> >>>>>>>> * Any NIO file mapping with MapMode.PRIVATE. >>>>>>>> * Any file mmap()-ed by any native library with MAP_PRIVATE. >>>>>>>> * The read only CDS regions (ro and od): Of the order of a few MB. >>>>>>>> * The shared strings CDS region. (typically less than 1 MB). >>>>>>>> * 2 MB per native shared library (regions with ---p permissions >>>>>>>> mapped by the dynamic linker for better alignment and for >>>>>>>> keeping libraries efficiently shareable). >>>>>>>> * The JDK 'modules' file. (About 140 MB). >>>>>>>> >>>>>>>> So, without including any NIO mapping, I typically see around >>>>>>>> 250-300 MB increase in the corefile sizes. I agree that the size >>>>>>>> increase could be a cause for concern, but for FWIW, these >>>>>>>> privately mapped files get dumped into the corefile for MacOS >>>>>>>> too. And the corefile sizes for the same program on MacOS are >>>>>>>> way larger (of the order of a few GB as against about 300 MB on >>>>>>>> Linux (without the change)). >>>>>>>> >>>>>>>> The advantage of fixing this by modifying the coredump_filter >>>>>>>> v/s doing it in SA (by reading in more sections of the shared >>>>>>>> archive file) is that this would benefit other debuggers like >>>>>>>> gdb also. (And reduces the dependence on having the shared >>>>>>>> archive file being available at the time of debugging). If folks >>>>>>>> still think this is a cause for concern, I could make >>>>>>>> modifications to fix this by reading in the regions from the >>>>>>>> shared archive file in the SA code. I also wonder if it is worth >>>>>>>> taking a relook at the mapping types of the various CDS regions >>>>>>>> also. >>>>>>>> >>>>>>>> Thank you, >>>>>>>> Jini. >>>>>>>> >>>>>>>> On 10/22/2018 10:27 AM, Ioi Lam wrote: >>>>>>>>> Hi Jini, >>>>>>>>> >>>>>>>>> Did you see my earlier reply? I might have sent it out during >>>>>>>>> the mail server outage days :-( >>>>>>>>> >>>>>>>>> http://openjdk.5641.n7.nabble.com/RFR-S-JDK-8200613-SA-jstack-throws-UnmappedAddressException-with-a-CDS-core-file-td352681.html#a353026 >>>>>>>>> >>>>>>>>> >>>>>>>>> Here was my reply again: >>>>>>>>> >>>>>>>>>> Hi Jini, >>>>>>>>>> >>>>>>>>>> The changes looks good to me. >>>>>>>>>> >>>>>>>>>> Have you tested this on MacOS? CDS heap support is also >>>>>>>>>> enabled on >>>>>>>>>> MacOS. See macros.hpp: >>>>>>>>>> >>>>>>>>>> ???? #if INCLUDE_CDS && INCLUDE_G1GC && defined(_LP64) && >>>>>>>>>> !defined(_WINDOWS) >>>>>>>>>> ???? #define INCLUDE_CDS_JAVA_HEAP 1 >>>>>>>>>> >>>>>>>>>> Also, besides CDS, do we know how often other files will be >>>>>>>>>> mmaped with >>>>>>>>>> MAP_PRIVATE? Will users get huge core files because CDS is >>>>>>>>>> enabled? In >>>>>>>>>> JDK 12, CDS will be enabled by default (see JDK-8202951), so >>>>>>>>>> all users >>>>>>>>>> will be affected by the following line: >>>>>>>>>> >>>>>>>>>> ?? if (UseSharedSpaces) { >>>>>>>>>> ???? set_coredump_filter(FILE_BACKED_PVT_BIT); >>>>>>>>>> ?? } >>>>>>>>>> >>>>>>>>>> Maybe you can run an big app such as Eclipse, trigger a core >>>>>>>>>> dump, and >>>>>>>>>> compare the size of the core file before/after this change? >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> >>>>>>>>>> - Ioi >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> >>>>>>>>> - Ioi >>>>>>>>> >>>>>>>>> >>>>>>>>> On 10/21/18 8:58 PM, Jini George wrote: >>>>>>>>>> Gentle reminder! >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> - Jini >>>>>>>>>> >>>>>>>>>> On 10/9/2018 11:31 AM, Jini George wrote: >>>>>>>>>>> Hello! >>>>>>>>>>> >>>>>>>>>>> [Including runtime-dev since the changes are in runtime code] >>>>>>>>>>> >>>>>>>>>>> Requesting reviews for: >>>>>>>>>>> >>>>>>>>>>> Webrev: http://cr.openjdk.java.net/~jgeorge/8200613/webrev.01/ >>>>>>>>>>> BugID: https://bugs.openjdk.java.net/browse/JDK-8200613 >>>>>>>>>>> >>>>>>>>>>> Issue: jhsdb jstack would throw an UnmappedAddressException >>>>>>>>>>> with a core file generated from a CDS enabled java process. >>>>>>>>>>> This is seen only with Linux and with G1GC, while trying to >>>>>>>>>>> read in data from the shared strings region (the closed >>>>>>>>>>> archive heap space). This region (which is a file backed >>>>>>>>>>> private memory region) is not dumped into the corefile for >>>>>>>>>>> Linux. This, being a heap region (and therefore being a >>>>>>>>>>> read-write region) is also not read in from the classes.jsa >>>>>>>>>>> file in SA since only the read only regions are read in while >>>>>>>>>>> processing the core file. (The expectation being that the >>>>>>>>>>> read write regions are in the core file). >>>>>>>>>>> >>>>>>>>>>> Proposed solution: The proposed solution is to have the >>>>>>>>>>> coredump_filter value corresponding to the CDS process to >>>>>>>>>>> include bit 2 (file-backed private memory), so that the >>>>>>>>>>> file-backed private memory region also gets dumped into the >>>>>>>>>>> corefile. The proposed fix is in >>>>>>>>>>> src/hotspot/os/linux/os_linux.cpp. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Jini. >>>>>>>>>>> >>>>>>>>> >>>>>>> >>>>>> >>> >> > From zanglin5 at jd.com Tue Dec 11 09:53:38 2018 From: zanglin5 at jd.com (=?gb2312?B?6rDB1Q==?=) Date: Tue, 11 Dec 2018 09:53:38 +0000 Subject: optimize KlassInfoTable size to power of 2 In-Reply-To: <4ea1c92f-4e72-7d38-8c7a-91e2892e0c3b@redhat.com> References: <140bac18565c48be88313f7cc1d24bf8@jd.com> <7fc37fd7cf24439eb3a6fff132fc7387@jd.com> <4ea1c92f-4e72-7d38-8c7a-91e2892e0c3b@redhat.com> Message-ID: <1ab2d28b40024ba6b299756612fa3760@jd.com> Hi Andrew, I understand your concern, thanks for point it out. I think the reason of not using power of 2 is because it more or less balanced the hash table entries, which may potentially cause more hash conflict and hence lag the entry lookup speed. However, in this case for KlassInfoTable, the hash value is the subtraction of klass address with a reference address, and then 2 bits shift right. Due to the address is limited by the heap size ,I believe that several highest bits of the hash value are same. So the mod by power of 2 in this case may not hit performance, that maybe one reason why I observed speedup in my experiment. And I just did quick test, that using the constant _num_buckets for modulo, gcc issues several instructions instead of idiv, and the speed up at about 12%, while using 65536 for _number_buckets speedup at 20%. BRs, Lin From christoph.langer at sap.com Tue Dec 11 10:28:15 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Tue, 11 Dec 2018 10:28:15 +0000 Subject: RFR (M) 8214892: Delayed starting of debugging via jcmd In-Reply-To: <87976683317e41028e146fcc000e5638@sap.com> References: <01fdfc8e58a246c18d5909fc7df313b7@sap.com> <940d8a26-d5b3-0d5b-ccb8-a71d4d17efdc@oracle.com> <6283122f12f041caa56f768f8005920b@sap.com> <87976683317e41028e146fcc000e5638@sap.com> Message-ID: <4167d2a331044ab59c51a6e50c5167ac@sap.com> Hi, @Ralf: Thanks for updating the patch. I'm good with it now ?? As there are no other opinions about the naming of the jdwp agent property, I'm fine with "onjcmd". It is the main use case. @all: With me and Chris as reviewers, I think we are good to push. I'll do this tomorrow if I don't hear objections until then. Thanks and best regards Christoph > -----Original Message----- > From: Schmelter, Ralf > Sent: Monday, December 10, 2018 2:55 PM > To: Langer, Christoph ; Chris Plummer > ; serviceability-dev at openjdk.java.net > Subject: RE: RFR (M) 8214892: Delayed starting of debugging via jcmd > > Hi Christoph, > > > I have thought about a more generic name for the option rather than > > "onjcmd". What about "standby=y". That would indicate that the > > debugging agent is on standby and can be triggered by whatever > > means. What do others think? > > I'm not sure making the name more generic is the right move, when it will > probably be enabled via the jcmd in >95% of the cases. > > > I'd prefer if you write out either "Debugging has been started." > > or "Debugging is already active.". I think this would make it more > > clear for the user of the feature what actually happened. > > OK, I've changed that. > > > A better wording would be: > > "Starts up the Java debugging if the jdwp agentlib was enabled with the > option onjcmd=y." > > OK. > > > replace "debug triggered by jcmd?" with " start debug via jcmd" > > Ok. > > > Is it really necessary/desired to set suspendOnInit to false? We > > could still honor suspend=y|n. The suspension will of course > > happen at the time debug activation is triggered. > > I don't think it would make sense to support suspend=y. In makes sense > when starting the debugging directly (so no, or at least not much) Java code > can be executed and you can debug from the beginning. And it also makes > sense for the onthrow/onuncaught, since you can to see the exceptions in > the debugger. But when triggered from the outside, you have no natural > event to wait for and no state to preserve. The only effect you would see is > that the jcmd would not return, since the initialize method would be blocked > until a debugger has been connected. > > I've updated the webrev: > http://cr.openjdk.java.net/~rrich/webrevs/schmelter/8214892/webrev.03/ > > Best regards, > Ralf Schmelter > > > -----Original Message----- > From: Langer, Christoph > Sent: Freitag, 7. Dezember 2018 09:41 > To: Schmelter, Ralf ; Chris Plummer > ; serviceability-dev at openjdk.java.net > Subject: RE: RFR (M) 8214892: Delayed starting of debugging via jcmd > > Hi Ralf, > > thanks for doing this change. Overall, it looks very good to me and seems to > be a nice feature. > > I have thought about a more generic name for the option rather than > "onjcmd". What about "standby=y". That would indicate that the debugging > agent is on standby and can be triggered by whatever means. What do > others think? > > Here are some minor comments, mostly about the wording of text > messages: > > src/hotspot/share/services/diagnosticCommand.cpp, line 1097: > I'd prefer if you write out either "Debugging has been started." or > "Debugging is already active.". I think this would make it more clear for the > user of the feature what actually happened. > > src/hotspot/share/services/diagnosticCommand.hpp, line 878: > "Starts up the Java debugging if enabled in the jdwp agentlib via the > onjcmd=y option." > A better wording would be: > "Starts up the Java debugging if the jdwp agentlib was enabled with the > option onjcmd=y." > > src/jdk.jdwp.agent/share/native/libjdwp/debugInit.c: > line 876: > replace "debug triggered by jcmd?" with " start debug via jcmd" (or "start > debug on request" if we opt to change the option name from onjcmd to > something more general) > line 1300: > suspendOnInit = JNI_FALSE; > Is it really necessary/desired to set suspendOnInit to false? We could still > honor suspend=y|n. The suspension will of course happen at the time debug > activation is triggered. > > Another question: Do we need a CSR for this? > > Best regards > Christoph > > > -----Original Message----- > > From: serviceability-dev > On > > Behalf Of Schmelter, Ralf > > Sent: Donnerstag, 6. Dezember 2018 15:54 > > To: Chris Plummer ; serviceability- > > dev at openjdk.java.net > > Subject: [CAUTION] RE: RFR (M) 8214892: Delayed starting of debugging via > > jcmd > > > > Hi Chris, > > > > > I think I prefer passing EI_VM_INIT for triggering_ei, but if you prefer > > > to keep it as is, I suggest a comment to clarify why it might be out of > > > range. > > > > Actually, I used EI_VM_INIT for the longest time and only changed it > > recently, because I thought that code could assume that e.g. no classes > have > > been loaded yet when getting the INIT_VM event. But since the JVMTI > spec > > does not guarantees this in any way (it allows other events to be send > before > > a VM_INIT), I just will change it back to use EI_VM_INIT for the initialize > call. > > > > Regarding the name of the option, I agree that onjcmd, while not > technically > > fully accurate, makes most sense for the common user. > > > > > ... It think you could just return the error right away and remove > > > the error checking code that comes later. > > > > I've changed the code to just return the error directly. > > > > > It's not clear to me why you want the name and address of the first > > > transport. Why is the first one necessarily the one you want? > > > > Since currently the bag of transports must always have a size of 1, getting > the > > first or the last transport is the same. But the callback function used to > iterate > > the bag has to return a boolean value. I've decided to return JNI_FALSE, > > which would mean the iteration stops at the first entry. > > > > > > I've updated the webrev with your and Goetz Lindenmeier's suggestions: > > > http://cr.openjdk.java.net/~rrich/webrevs/schmelter/8214892/webrev.02/ > > > > Best regards, > > Ralf From aph at redhat.com Tue Dec 11 11:05:41 2018 From: aph at redhat.com (Andrew Haley) Date: Tue, 11 Dec 2018 11:05:41 +0000 Subject: optimize KlassInfoTable size to power of 2 In-Reply-To: <1ab2d28b40024ba6b299756612fa3760@jd.com> References: <140bac18565c48be88313f7cc1d24bf8@jd.com> <7fc37fd7cf24439eb3a6fff132fc7387@jd.com> <4ea1c92f-4e72-7d38-8c7a-91e2892e0c3b@redhat.com> <1ab2d28b40024ba6b299756612fa3760@jd.com> Message-ID: <7a1e5d87-94fd-9be9-8aab-5dd0f1a3fff9@redhat.com> On 12/11/18 9:53 AM, ?? wrote: > And I just did quick test, that using the constant _num_buckets for modulo, gcc issues several instructions instead > of idiv, and the speed up at about 12%, while using 65536 for _number_buckets speedup at 20%. OK. So GCC's constant optimization is an improvement over what we have already. There is something that I do not understand. You said that GCC didn't know that _num_buckets was constant. In that case, how did GCC know not to use a divide instruction when you tried 65536? In any case, if we really do care so much about this, I would have thought that the best solution would be to use 65537 as the table size because there is a nice way to calculate n % 65537:: unsigned mod_m(unsigned n) { unsigned tmp = n % 65536; tmp -= n / 65536; if (tmp >= 65537) // overflow tmp += 65537; return tmp; } It's very difficult to prove that using a non-prime table size won't impact the performance on some systems; using only a few bits of the address isn't worth the risk, IMO. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Tue Dec 11 11:10:13 2018 From: aph at redhat.com (Andrew Haley) Date: Tue, 11 Dec 2018 11:10:13 +0000 Subject: optimize KlassInfoTable size to power of 2 In-Reply-To: <7a1e5d87-94fd-9be9-8aab-5dd0f1a3fff9@redhat.com> References: <140bac18565c48be88313f7cc1d24bf8@jd.com> <7fc37fd7cf24439eb3a6fff132fc7387@jd.com> <4ea1c92f-4e72-7d38-8c7a-91e2892e0c3b@redhat.com> <1ab2d28b40024ba6b299756612fa3760@jd.com> <7a1e5d87-94fd-9be9-8aab-5dd0f1a3fff9@redhat.com> Message-ID: On 12/11/18 11:05 AM, Andrew Haley wrote: > unsigned mod_m(unsigned n) { > unsigned tmp = n % 65536; > tmp -= n / 65536; > if (tmp >= 65537) // overflow > tmp += 65537; > return tmp; > } NB: this assumes that unsigned int is uint64_t. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From zanglin5 at jd.com Tue Dec 11 11:36:03 2018 From: zanglin5 at jd.com (=?gb2312?B?6rDB1Q==?=) Date: Tue, 11 Dec 2018 11:36:03 +0000 Subject: optimize KlassInfoTable size to power of 2 In-Reply-To: References: <140bac18565c48be88313f7cc1d24bf8@jd.com> <7fc37fd7cf24439eb3a6fff132fc7387@jd.com> <4ea1c92f-4e72-7d38-8c7a-91e2892e0c3b@redhat.com> <1ab2d28b40024ba6b299756612fa3760@jd.com> <7a1e5d87-94fd-9be9-8aab-5dd0f1a3fff9@redhat.com> Message-ID: <8a44110225004b51a921c55a7806e7fc@jd.com> >There is something that I do not understand. You said that GCC didn't know that _num_buckets was constant. In > that case, how did GCC know not to use a divide instruction when you tried 65536? The original code doesn't use _num_buckets directly when calculating the hash idx, it use a non-constant variable _size, a member of KlassInfoTable, which is assigned to _num_buckets. I guess it is the reason GCC didn't do the constant propagation in this case. And _size is not changed during the life of KlassInfoTable, it only set to zero at de-construction. In my experiment, I use _num_buckets directly instead of _size to calculate hash idx. So GCC provides the optimizations. I think your suggestion of using 65537 is valuable, And I agree that it is not easy to prove that non-prime for hash is better. So how about make the change to use constant _number_buckets directly for calculating hash idx, and leave the prime not changed. In my case, speed up JMap histo helps at avoid getting killed by a timer, especially when heap is large at ~200GB. Do you think it is reasonable to have a patch? Thanks, Lin -----Original Message----- From: Andrew Haley [mailto:aph at redhat.com] Sent: Tuesday, December 11, 2018 7:10 PM To: ?? ; serviceability-dev at openjdk.java.net Subject: Re: optimize KlassInfoTable size to power of 2 On 12/11/18 11:05 AM, Andrew Haley wrote: > unsigned mod_m(unsigned n) { > unsigned tmp = n % 65536; > tmp -= n / 65536; > if (tmp >= 65537) // overflow > tmp += 65537; > return tmp; > } NB: this assumes that unsigned int is uint64_t. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From zanglin5 at jd.com Tue Dec 11 11:44:18 2018 From: zanglin5 at jd.com (=?gb2312?B?6rDB1Q==?=) Date: Tue, 11 Dec 2018 11:44:18 +0000 Subject: optimize KlassInfoTable size to power of 2 References: <140bac18565c48be88313f7cc1d24bf8@jd.com> <7fc37fd7cf24439eb3a6fff132fc7387@jd.com> <4ea1c92f-4e72-7d38-8c7a-91e2892e0c3b@redhat.com> <1ab2d28b40024ba6b299756612fa3760@jd.com> <7a1e5d87-94fd-9be9-8aab-5dd0f1a3fff9@redhat.com> Message-ID: <56b9099e33584796af8a508980ff4690@jd.com> Oh, to make it clear, when testing with 65536, I made the change to use hash & 65535 instead of hash % 65536. BRs, Lin -----Original Message----- From: ?? Sent: Tuesday, December 11, 2018 7:36 PM To: 'Andrew Haley' ; serviceability-dev at openjdk.java.net Subject: RE: optimize KlassInfoTable size to power of 2 >There is something that I do not understand. You said that GCC didn't >know that _num_buckets was constant. In that case, how did GCC know not to use a divide instruction when you tried 65536? The original code doesn't use _num_buckets directly when calculating the hash idx, it use a non-constant variable _size, a member of KlassInfoTable, which is assigned to _num_buckets. I guess it is the reason GCC didn't do the constant propagation in this case. And _size is not changed during the life of KlassInfoTable, it only set to zero at de-construction. In my experiment, I use _num_buckets directly instead of _size to calculate hash idx. So GCC provides the optimizations. I think your suggestion of using 65537 is valuable, And I agree that it is not easy to prove that non-prime for hash is better. So how about make the change to use constant _number_buckets directly for calculating hash idx, and leave the prime not changed. In my case, speed up JMap histo helps at avoid getting killed by a timer, especially when heap is large at ~200GB. Do you think it is reasonable to have a patch? Thanks, Lin -----Original Message----- From: Andrew Haley [mailto:aph at redhat.com] Sent: Tuesday, December 11, 2018 7:10 PM To: ?? ; serviceability-dev at openjdk.java.net Subject: Re: optimize KlassInfoTable size to power of 2 On 12/11/18 11:05 AM, Andrew Haley wrote: > unsigned mod_m(unsigned n) { > unsigned tmp = n % 65536; > tmp -= n / 65536; > if (tmp >= 65537) // overflow > tmp += 65537; > return tmp; > } NB: this assumes that unsigned int is uint64_t. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From alexey.ivanov at oracle.com Tue Dec 11 15:05:37 2018 From: alexey.ivanov at oracle.com (Alexey Ivanov) Date: Tue, 11 Dec 2018 15:05:37 +0000 Subject: [PATCH] Windows 32-bit DLL name decoration In-Reply-To: References: <2DD922C6-EFB1-4891-90F0-3378DEA91B1C@gmail.com> <07840796-f1c4-e077-7b0b-cd5c1a880ec1@oracle.com> <00e58330-26f0-4401-f911-24fda0127bcc@oracle.com> <7083B6EB-588E-4296-9A03-D9CC9B688936@gmail.com> <813f4b3d-0b4a-1e28-845f-a6422a22c695@redhat.com> <530c2651-dc96-e04a-0b0e-d1f04d8e135a@oracle.com> Message-ID: Hi everyone, I came up with the following patch: http://cr.openjdk.java.net/~aivanov/8214122/webrev.00/ It specifically addresses the problem in JDK-8214122 where on 32 bit Windows jdwpTransport_OnLoad can exported with its plain and __stdcall-mangled name. I used conditional compilation so that for other platforms the code remains as it is now. jshell starts successfully with this fix; an IDE debugger works as well. Regards, Alexey https://bugs.openjdk.java.net/browse/JDK-8214122 On 10/12/2018 15:11, Magnus Ihse Bursie wrote: >> Since removing JNICALL is not an option, there are only two options: >> >> 1. Add |/export| option to the Makefile or pragma-comment to the >> source file; >> 2. Lookup the decorated name |_jdwpTransport_OnLoad at 16| for Win32 >> with fallback to undecorated one. > Yes. > > I think the correct solution here is 2. From stooke at redhat.com Tue Dec 11 16:43:19 2018 From: stooke at redhat.com (Simon Tooke) Date: Tue, 11 Dec 2018 11:43:19 -0500 Subject: [PATCH] Windows 32-bit DLL name decoration In-Reply-To: References: <2DD922C6-EFB1-4891-90F0-3378DEA91B1C@gmail.com> <07840796-f1c4-e077-7b0b-cd5c1a880ec1@oracle.com> <00e58330-26f0-4401-f911-24fda0127bcc@oracle.com> <7083B6EB-588E-4296-9A03-D9CC9B688936@gmail.com> <813f4b3d-0b4a-1e28-845f-a6422a22c695@redhat.com> <530c2651-dc96-e04a-0b0e-d1f04d8e135a@oracle.com> Message-ID: <77cc6547-a145-e7b9-5f29-c66df76bc50a@redhat.com> On 2018-12-11 10:05 a.m., Alexey Ivanov wrote: > Hi everyone, > > I came up with the following patch: > http://cr.openjdk.java.net/~aivanov/8214122/webrev.00/ > > It specifically addresses the problem in JDK-8214122 where on 32 bit > Windows jdwpTransport_OnLoad can exported with its plain and > __stdcall-mangled name. I used conditional compilation so that for > other platforms the code remains as it is now. > > jshell starts successfully with this fix; an IDE debugger works as well. > I am not a reviewer, but this patch only works for the specific case under discussion; the '@16' refers to the reserved stack size in the Pascal calling convention.? So, the patch only works for 16 bytes of parameters.? To be generic, the routine needs to have the stack size passed in by the caller.? If a generic fix isn't desired (and I hope it is), I'd prefer to see the caller simply pass the decorated or undecorated name depending on the Win32/64 defines. #if defined(_WIN32) && !defined(_WIN64) onLoad = (jdwpTransport_OnLoad_t) dbgsysFindLibraryEntry(handle, "_jdwpTransport_OnLoad at 16"); #else onLoad = (jdwpTransport_OnLoad_t) dbgsysFindLibraryEntry(handle, "jdwpTransport_OnLoad"); #endif Thanks, -Simon > > Regards, > Alexey > > https://bugs.openjdk.java.net/browse/JDK-8214122 > > On 10/12/2018 15:11, Magnus Ihse Bursie wrote: >>> Since removing JNICALL is not an option, there are only two options: >>> >>> 1. Add |/export| option to the Makefile or pragma-comment to the >>> source file; >>> 2. Lookup the decorated name |_jdwpTransport_OnLoad at 16| for Win32 >>> with fallback to undecorated one. >> Yes. >> >> I think the correct solution here is 2. > From alexey.ivanov at oracle.com Tue Dec 11 17:16:19 2018 From: alexey.ivanov at oracle.com (Alexey Ivanov) Date: Tue, 11 Dec 2018 17:16:19 +0000 Subject: [PATCH] Windows 32-bit DLL name decoration In-Reply-To: <77cc6547-a145-e7b9-5f29-c66df76bc50a@redhat.com> References: <2DD922C6-EFB1-4891-90F0-3378DEA91B1C@gmail.com> <07840796-f1c4-e077-7b0b-cd5c1a880ec1@oracle.com> <00e58330-26f0-4401-f911-24fda0127bcc@oracle.com> <7083B6EB-588E-4296-9A03-D9CC9B688936@gmail.com> <813f4b3d-0b4a-1e28-845f-a6422a22c695@redhat.com> <530c2651-dc96-e04a-0b0e-d1f04d8e135a@oracle.com> <77cc6547-a145-e7b9-5f29-c66df76bc50a@redhat.com> Message-ID: Hi Simon, Thank you for your comments. The updated webrev: http://cr.openjdk.java.net/~aivanov/8214122/webrev.01/ Indeed, it looks much cleaner. Regards, Alexey On 11/12/2018 16:43, Simon Tooke wrote: > On 2018-12-11 10:05 a.m., Alexey Ivanov wrote: >> Hi everyone, >> >> I came up with the following patch: >> http://cr.openjdk.java.net/~aivanov/8214122/webrev.00/ >> >> It specifically addresses the problem in JDK-8214122 where on 32 bit >> Windows jdwpTransport_OnLoad can exported with its plain and >> __stdcall-mangled name. I used conditional compilation so that for >> other platforms the code remains as it is now. >> >> jshell starts successfully with this fix; an IDE debugger works as well. >> > I am not a reviewer, but this patch only works for the specific case > under discussion; the '@16' refers to the reserved stack size in the > Pascal calling convention.? So, the patch only works for 16 bytes of > parameters.? To be generic, the routine needs to have the stack size > passed in by the caller.? If a generic fix isn't desired (and I hope it > is), I'd prefer to see the caller simply pass the decorated or > undecorated name depending on the Win32/64 defines. > > #if defined(_WIN32) && !defined(_WIN64) onLoad = > (jdwpTransport_OnLoad_t) dbgsysFindLibraryEntry(handle, > "_jdwpTransport_OnLoad at 16"); #else onLoad = (jdwpTransport_OnLoad_t) > dbgsysFindLibraryEntry(handle, "jdwpTransport_OnLoad"); #endif > > > Thanks, > -Simon > >> Regards, >> Alexey >> >> https://bugs.openjdk.java.net/browse/JDK-8214122 >> >> On 10/12/2018 15:11, Magnus Ihse Bursie wrote: >>>> Since removing JNICALL is not an option, there are only two options: >>>> >>>> 1. Add |/export| option to the Makefile or pragma-comment to the >>>> source file; >>>> 2. Lookup the decorated name |_jdwpTransport_OnLoad at 16| for Win32 >>>> with fallback to undecorated one. >>> Yes. >>> >>> I think the correct solution here is 2. From bob.vandette at oracle.com Tue Dec 11 17:22:26 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Tue, 11 Dec 2018 12:22:26 -0500 Subject: [PATCH] Windows 32-bit DLL name decoration In-Reply-To: <77cc6547-a145-e7b9-5f29-c66df76bc50a@redhat.com> References: <2DD922C6-EFB1-4891-90F0-3378DEA91B1C@gmail.com> <07840796-f1c4-e077-7b0b-cd5c1a880ec1@oracle.com> <00e58330-26f0-4401-f911-24fda0127bcc@oracle.com> <7083B6EB-588E-4296-9A03-D9CC9B688936@gmail.com> <813f4b3d-0b4a-1e28-845f-a6422a22c695@redhat.com> <530c2651-dc96-e04a-0b0e-d1f04d8e135a@oracle.com> <77cc6547-a145-e7b9-5f29-c66df76bc50a@redhat.com> Message-ID: <2A193B76-87CF-4F16-89F1-7BA4257B13FF@oracle.com> Hotspot has had support for decorated and non-decorated names for the JNI_OnLoad functions. Perhaps you should just follow the same procedure for the debug library. #define JNI_ONLOAD_SYMBOLS {"_JNI_OnLoad at 8", "JNI_OnLoad"} #define JNI_ONUNLOAD_SYMBOLS {"_JNI_OnUnload at 8", "JNI_OnUnload"} #define JVM_ONLOAD_SYMBOLS {"_JVM_OnLoad at 12", "JVM_OnLoad"} #define AGENT_ONLOAD_SYMBOLS {"_Agent_OnLoad at 12", "Agent_OnLoad"} #define AGENT_ONUNLOAD_SYMBOLS {"_Agent_OnUnload at 4", "Agent_OnUnload"} #define AGENT_ONATTACH_SYMBOLS {"_Agent_OnAttach at 12", ?Agent_OnAttach?} Bob. > On Dec 11, 2018, at 11:43 AM, Simon Tooke wrote: > > On 2018-12-11 10:05 a.m., Alexey Ivanov wrote: >> Hi everyone, >> >> I came up with the following patch: >> http://cr.openjdk.java.net/~aivanov/8214122/webrev.00/ >> >> It specifically addresses the problem in JDK-8214122 where on 32 bit >> Windows jdwpTransport_OnLoad can exported with its plain and >> __stdcall-mangled name. I used conditional compilation so that for >> other platforms the code remains as it is now. >> >> jshell starts successfully with this fix; an IDE debugger works as well. >> > I am not a reviewer, but this patch only works for the specific case > under discussion; the '@16' refers to the reserved stack size in the > Pascal calling convention. So, the patch only works for 16 bytes of > parameters. To be generic, the routine needs to have the stack size > passed in by the caller. If a generic fix isn't desired (and I hope it > is), I'd prefer to see the caller simply pass the decorated or > undecorated name depending on the Win32/64 defines. > > #if defined(_WIN32) && !defined(_WIN64) onLoad = > (jdwpTransport_OnLoad_t) dbgsysFindLibraryEntry(handle, > "_jdwpTransport_OnLoad at 16"); #else onLoad = (jdwpTransport_OnLoad_t) > dbgsysFindLibraryEntry(handle, "jdwpTransport_OnLoad"); #endif > > > Thanks, > -Simon > >> >> Regards, >> Alexey >> >> https://bugs.openjdk.java.net/browse/JDK-8214122 >> >> On 10/12/2018 15:11, Magnus Ihse Bursie wrote: >>>> Since removing JNICALL is not an option, there are only two options: >>>> >>>> 1. Add |/export| option to the Makefile or pragma-comment to the >>>> source file; >>>> 2. Lookup the decorated name |_jdwpTransport_OnLoad at 16| for Win32 >>>> with fallback to undecorated one. >>> Yes. >>> >>> I think the correct solution here is 2. >> > From jini.george at oracle.com Tue Dec 11 17:29:31 2018 From: jini.george at oracle.com (Jini George) Date: Tue, 11 Dec 2018 22:59:31 +0530 Subject: RFR: JDK-8202884: SA: Attach/detach might fail on Linux if debugee application create/destroy threads during attaching Message-ID: Hello ! Requesting reviews for: https://bugs.openjdk.java.net/browse/JDK-8202884 Webrev: http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/index.html Details: For attaching to the threads in a process, we first go ahead and do a ptrace attach to the main thread. Later, we use the libthread_db library (or, in the case of being within a container, iterate through the /proc//task files) to discover the threads of the process, and add them to the threads list (within SA) for this process. Once, we have discovered all the threads and added these to the list of threads, we then invoke ptrace attach individually on all these threads to attach to these. When we deal with an application where the threads are exiting continuously, some of these threads might not exist by the time we try to ptrace attach to these threads. The proposed fix includes the following modifications to solve this. 1. Check the state of the threads in the thread_db callback routine, and skip if the state of the thread is TD_THR_UNKNOWN or TD_THR_ZOMBIE. SA does not try to ptrace attach to these threads and does not include these threads in the threads list. 2. While ptrace attaching to the thread, if ptrace(PTRACE_ATTACH, ...) fails with either ESCRH or EPERM, check the state of the thread by checking if the /proc//status file corresponding to that thread exists and if so, reading in the 'State:' line of that file. Skip attaching to this thread and delete this thread from the SA list of threads, if the thread is dead (State: X) or is a zombie (State: Z). From the /proc man page, "Current state of the process. One of "R (running)", "S (sleeping)", "D (disk sleep)", "T (stopped)", "T (tracing stop)", "Z (zombie)", or "X (dead)"." 3. If waitpid() on the thread is a failure, again skip this thread (delete this from SA's list of threads) instead of bailing out if the thread has exited or terminated. Testing: 1. Tested by attaching and detaching multiple times to a test program spawning numerous short lived threads. 2. The SA tests (under test/hotspot/jtreg/serviceability/sa) passed with 100 repeats on Mach5. 3. No new failures and no occurrences of JDK-8202884 seen with testing the SA tests (tiers 1 to 5) on Mach5. More details in the bug comments section. Thank you, Jini. From jcbeyler at google.com Tue Dec 11 18:26:56 2018 From: jcbeyler at google.com (JC Beyler) Date: Tue, 11 Dec 2018 10:26:56 -0800 Subject: optimize KlassInfoTable size to power of 2 In-Reply-To: <56b9099e33584796af8a508980ff4690@jd.com> References: <140bac18565c48be88313f7cc1d24bf8@jd.com> <7fc37fd7cf24439eb3a6fff132fc7387@jd.com> <4ea1c92f-4e72-7d38-8c7a-91e2892e0c3b@redhat.com> <1ab2d28b40024ba6b299756612fa3760@jd.com> <7a1e5d87-94fd-9be9-8aab-5dd0f1a3fff9@redhat.com> <56b9099e33584796af8a508980ff4690@jd.com> Message-ID: Hi all, I was playing with this yesterday because I was curious to see the different code generation patterns so I had the webrev ready to go: Webrev: http://cr.openjdk.java.net/~jcbeyler/8215228/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8215228 @Lin: check but I'd imagine this is what you had originally done but you also changed the size Thanks, Jc On Tue, Dec 11, 2018 at 3:44 AM ?? wrote: > > Oh, to make it clear, when testing with 65536, I made the change to use > hash & 65535 > instead of hash % 65536. > > > BRs, > Lin > > > > -----Original Message----- > From: ?? > Sent: Tuesday, December 11, 2018 7:36 PM > To: 'Andrew Haley' ; serviceability-dev at openjdk.java.net > Subject: RE: optimize KlassInfoTable size to power of 2 > > >There is something that I do not understand. You said that GCC didn't > >know that _num_buckets was constant. In that case, how did GCC know not > to use a divide instruction when you tried 65536? > > The original code doesn't use _num_buckets directly when calculating the > hash idx, it use a non-constant variable _size, a member of KlassInfoTable, > which is assigned to _num_buckets. I guess it is the reason GCC didn't do > the constant propagation in this case. And _size is not changed during the > life of KlassInfoTable, it only set to zero at de-construction. > > In my experiment, I use _num_buckets directly instead of _size to > calculate hash idx. So GCC provides the optimizations. > > I think your suggestion of using 65537 is valuable, And I agree that it is > not easy to prove that non-prime for hash is better. So how about make the > change to use constant _number_buckets directly for calculating hash idx, > and leave the prime not changed. In my case, speed up JMap histo helps at > avoid getting killed by a timer, especially when heap is large at ~200GB. > > Do you think it is reasonable to have a patch? > > Thanks, > Lin > > -----Original Message----- > From: Andrew Haley [mailto:aph at redhat.com] > Sent: Tuesday, December 11, 2018 7:10 PM > To: ?? ; serviceability-dev at openjdk.java.net > Subject: Re: optimize KlassInfoTable size to power of 2 > > On 12/11/18 11:05 AM, Andrew Haley wrote: > > unsigned mod_m(unsigned n) { > > unsigned tmp = n % 65536; > > tmp -= n / 65536; > > if (tmp >= 65537) // overflow > > tmp += 65537; > > return tmp; > > } > > NB: this assumes that unsigned int is uint64_t. > > -- > Andrew Haley > Java Platform Lead Engineer > Red Hat UK Ltd. > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From coleen.phillimore at oracle.com Tue Dec 11 19:28:22 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 11 Dec 2018 14:28:22 -0500 Subject: RFR: (S): JDK-8200613: SA: jstack throws UnmappedAddressException with a CDS core file In-Reply-To: <536d23dc-e9bf-86d9-0c86-fb15732576ea@oracle.com> References: <864997e5-2581-7ac9-44f7-76cc31c9ad93@oracle.com> <60d2e6db-c87b-9658-57cb-b03ca881a624@oracle.com> <9eb9041d-f984-0df4-a21e-4645966a62b8@oracle.com> <42d6d604-8df5-47c4-a748-6aa25bfc9a7e@oracle.com> <2dbef041-c75e-07f6-fafc-d18f2a95f455@oracle.com> <798a4d93-1638-8785-885f-fac0b21c2417@oracle.com> <2c6a2d67-5d36-07f1-4563-c1cd2ecdf9a3@oracle.com> <536d23dc-e9bf-86d9-0c86-fb15732576ea@oracle.com> Message-ID: <7db96561-7845-bee5-e276-558fb49b94c3@oracle.com> On 12/11/18 2:37 AM, Jini George wrote: > Thank you very much, Coleen. I have converted the flag into a > diagnostic flag. The revised webrev is at: > > http://cr.openjdk.java.net/~jgeorge/8200613/webrev.03/ Looks great! > > A plain diff between this patch and the earlier one is at: > > http://cr.openjdk.java.net/~jgeorge/8200613/incremental_diff > > I will withdraw the associated CSR. Yes, please withdraw it.? Thank you! Coleen > > Thanks! > Jini. > > On 12/10/2018 11:55 PM, coleen.phillimore at oracle.com wrote: >> >> I see.? In this case, the flag should be diagnostic and not require a >> CSR.? I suppose some documentation should be added so that sustaining >> will know about the option in this case (Hi Kevin!) >> >> thanks, >> Coleen >> >> >> On 12/10/18 1:38 AM, Ioi Lam wrote: >>> Hi Coleen, >>> >>> I was one of the people who suggested the DumpPrivateMappingsInCore >>> flag. It's enabled by default, so by default all the contents of >>> mmap'ed files with MAP_PRIVATE will be saved to the core files. >>> >>> The worry is, there may be some extreme cases where the JVM has >>> mapped very large files (with NIO or JNI, etc). For example, you >>> could have a 100GB in-memory database. For those cases, if the user >>> is experiencing crashes, but they are unable to get a core dump >>> (because it would be too large), they can try running with >>> -XX:-DumpPrivateMappingsInCore. >>> >>> Thanks >>> >>> - Ioi >>> >>> >>> On 12/7/18 12:03 PM, coleen.phillimore at oracle.com wrote: >>>> >>>> Hi Jini,? We were just talking about this new option.? If someone >>>> gets a crash, I don't think they're going to run their application >>>> again with -XX:-DumpPrivateMappingsInCore in the case of the core >>>> file being too large.? So I don't know how generally useful this >>>> option is. I think it would be better to not add it and set the bit >>>> to include the mappings unconditionally. >>>> >>>> How much larger is this core file (we had trouble understanding >>>> from the bug)?? If you need the mappings to understand and use the >>>> SA tools on it, we want to have them. >>>> >>>> Thanks, >>>> Coleen >>>> >>>> >>>> On 12/7/18 2:22 PM, Jini George wrote: >>>>> I have the revised webrev here: >>>>> >>>>> http://cr.openjdk.java.net/~jgeorge/8200613/webrev.02/index.html >>>>> >>>>> The extra changes here are to: >>>>> >>>>> * Introduce a new option DumpPrivateMappingsInCore to control the >>>>> dumping of the file backed private regions into the corefile. >>>>> >>>>> * Close the modules file before dumping core in os::abort(). >>>>> Currently, there is a small bug >>>>> (https://bugs.openjdk.java.net/browse/JDK-8215026) which prevents >>>>> the closure of the image file in unmapping the entire file. >>>>> >>>>> I plan to take up the unmapping of NIO MapMode.PRIVATE files as a >>>>> separate task (https://bugs.openjdk.java.net/browse/JDK-8215027) >>>>> since this seems a bit involved. >>>>> >>>>> Thanks a bunch, >>>>> Jini. >>>>> >>>>> On 11/12/2018 10:26 AM, Jini George wrote: >>>>>> Thank you very much, Chris, Kevin and Ioi for your comments! >>>>>> >>>>>> I will send another webrev with this change enabled under an >>>>>> opt-out flag, as you suggest, and would look at unmapping the JDK >>>>>> modules file and if possible, the NIO mapped files too in the >>>>>> signal handler. >>>>>> >>>>>> Thanks a bunch, >>>>>> Jini. >>>>>> >>>>>> On 11/9/2018 11:53 PM, Ioi Lam wrote: >>>>>>> Hi Jini, >>>>>>> >>>>>>> Thanks for investigating the size expansion issue. >>>>>>> >>>>>>> I agree that the size increase is worth it. Even when not using >>>>>>> SA, if we open the core file inside GDB, we cannot read certain >>>>>>> sections in the CDS archive (such as the RO section and strings >>>>>>> sections). That would make debugging difficult. So I am in favor >>>>>>> of this change. >>>>>>> >>>>>>> For the JDK modules file, maybe we can unmap it in the signal >>>>>>> handler, before going ahead with the core dump? I think it's >>>>>>> hardly needed for debugging purposes. (Perhaps we can also do >>>>>>> the same for the NIO mapped files?) >>>>>>> >>>>>>> A opt-flag as suggested by Kevin is a good idea. >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> - Ioi >>>>>>> >>>>>>> On 11/9/18 3:29 AM, Kevin Walls wrote: >>>>>>>> Hi Jini, >>>>>>>> >>>>>>>> Looks good to me.? It might be a significant increase in size >>>>>>>> of _some_ core files, but so many core files we see are much >>>>>>>> larger, in gigabytes++ of course, so the CDS data size should >>>>>>>> not be such a significant increase on (I think) most files. >>>>>>>> >>>>>>>> The flexibiity of always having the CDS data there is very >>>>>>>> significant.? A core file should ideally be usable, without >>>>>>>> additionally requiring the CDS archive from the machine. That >>>>>>>> additional human round-trip upload request on every transmitted >>>>>>>> core that needs investigating, seems like a less efficient >>>>>>>> route...). >>>>>>>> >>>>>>>> Is there an opt-out?? It's conditional on UseSharedSpaces but >>>>>>>> could there be a flag to disable, in case we see crashes with >>>>>>>> gigabytes of private mappings that we really don't want to >>>>>>>> retain (the user would have to know to set a flag, to disable >>>>>>>> the new coredump filter ahead of time). >>>>>>>> >>>>>>>> Thanks! >>>>>>>> Kevin >>>>>>>> >>>>>>>> >>>>>>>> On 29/10/2018 06:02, Jini George wrote: >>>>>>>>> Thank you very much, Ioi, for looking into this, and the >>>>>>>>> clarification offline. My bad, I had missed the earlier mail >>>>>>>>> from you. :-( My responses below. >>>>>>>>> >>>>>>>>> Yes, I had tested this on MacOS. The issue does not exist on >>>>>>>>> MacOS since the file backed private mmap()-ed regions get >>>>>>>>> dumped into the MacOS corefiles by default. >>>>>>>>> >>>>>>>>> The corefile sizes on Linux do increase due to this change. >>>>>>>>> And the increase would also include any file mapped using NIO >>>>>>>>> with MapMode.PRIVATE. The typical corefile size increase with >>>>>>>>> this change would include the following components at a high >>>>>>>>> level: >>>>>>>>> >>>>>>>>> * Any NIO file mapping with MapMode.PRIVATE. >>>>>>>>> * Any file mmap()-ed by any native library with MAP_PRIVATE. >>>>>>>>> * The read only CDS regions (ro and od): Of the order of a few >>>>>>>>> MB. >>>>>>>>> * The shared strings CDS region. (typically less than 1 MB). >>>>>>>>> * 2 MB per native shared library (regions with ---p >>>>>>>>> permissions mapped by the dynamic linker for better alignment >>>>>>>>> and for keeping libraries efficiently shareable). >>>>>>>>> * The JDK 'modules' file. (About 140 MB). >>>>>>>>> >>>>>>>>> So, without including any NIO mapping, I typically see around >>>>>>>>> 250-300 MB increase in the corefile sizes. I agree that the >>>>>>>>> size increase could be a cause for concern, but for FWIW, >>>>>>>>> these privately mapped files get dumped into the corefile for >>>>>>>>> MacOS too. And the corefile sizes for the same program on >>>>>>>>> MacOS are way larger (of the order of a few GB as against >>>>>>>>> about 300 MB on Linux (without the change)). >>>>>>>>> >>>>>>>>> The advantage of fixing this by modifying the coredump_filter >>>>>>>>> v/s doing it in SA (by reading in more sections of the shared >>>>>>>>> archive file) is that this would benefit other debuggers like >>>>>>>>> gdb also. (And reduces the dependence on having the shared >>>>>>>>> archive file being available at the time of debugging). If >>>>>>>>> folks still think this is a cause for concern, I could make >>>>>>>>> modifications to fix this by reading in the regions from the >>>>>>>>> shared archive file in the SA code. I also wonder if it is >>>>>>>>> worth taking a relook at the mapping types of the various CDS >>>>>>>>> regions also. >>>>>>>>> >>>>>>>>> Thank you, >>>>>>>>> Jini. >>>>>>>>> >>>>>>>>> On 10/22/2018 10:27 AM, Ioi Lam wrote: >>>>>>>>>> Hi Jini, >>>>>>>>>> >>>>>>>>>> Did you see my earlier reply? I might have sent it out during >>>>>>>>>> the mail server outage days :-( >>>>>>>>>> >>>>>>>>>> http://openjdk.5641.n7.nabble.com/RFR-S-JDK-8200613-SA-jstack-throws-UnmappedAddressException-with-a-CDS-core-file-td352681.html#a353026 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Here was my reply again: >>>>>>>>>> >>>>>>>>>>> Hi Jini, >>>>>>>>>>> >>>>>>>>>>> The changes looks good to me. >>>>>>>>>>> >>>>>>>>>>> Have you tested this on MacOS? CDS heap support is also >>>>>>>>>>> enabled on >>>>>>>>>>> MacOS. See macros.hpp: >>>>>>>>>>> >>>>>>>>>>> ???? #if INCLUDE_CDS && INCLUDE_G1GC && defined(_LP64) && >>>>>>>>>>> !defined(_WINDOWS) >>>>>>>>>>> ???? #define INCLUDE_CDS_JAVA_HEAP 1 >>>>>>>>>>> >>>>>>>>>>> Also, besides CDS, do we know how often other files will be >>>>>>>>>>> mmaped with >>>>>>>>>>> MAP_PRIVATE? Will users get huge core files because CDS is >>>>>>>>>>> enabled? In >>>>>>>>>>> JDK 12, CDS will be enabled by default (see JDK-8202951), so >>>>>>>>>>> all users >>>>>>>>>>> will be affected by the following line: >>>>>>>>>>> >>>>>>>>>>> ?? if (UseSharedSpaces) { >>>>>>>>>>> ???? set_coredump_filter(FILE_BACKED_PVT_BIT); >>>>>>>>>>> ?? } >>>>>>>>>>> >>>>>>>>>>> Maybe you can run an big app such as Eclipse, trigger a core >>>>>>>>>>> dump, and >>>>>>>>>>> compare the size of the core file before/after this change? >>>>>>>>>>> >>>>>>>>>>> Thanks >>>>>>>>>>> >>>>>>>>>>> - Ioi >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> >>>>>>>>>> - Ioi >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 10/21/18 8:58 PM, Jini George wrote: >>>>>>>>>>> Gentle reminder! >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> - Jini >>>>>>>>>>> >>>>>>>>>>> On 10/9/2018 11:31 AM, Jini George wrote: >>>>>>>>>>>> Hello! >>>>>>>>>>>> >>>>>>>>>>>> [Including runtime-dev since the changes are in runtime code] >>>>>>>>>>>> >>>>>>>>>>>> Requesting reviews for: >>>>>>>>>>>> >>>>>>>>>>>> Webrev: http://cr.openjdk.java.net/~jgeorge/8200613/webrev.01/ >>>>>>>>>>>> BugID: https://bugs.openjdk.java.net/browse/JDK-8200613 >>>>>>>>>>>> >>>>>>>>>>>> Issue: jhsdb jstack would throw an UnmappedAddressException >>>>>>>>>>>> with a core file generated from a CDS enabled java process. >>>>>>>>>>>> This is seen only with Linux and with G1GC, while trying to >>>>>>>>>>>> read in data from the shared strings region (the closed >>>>>>>>>>>> archive heap space). This region (which is a file backed >>>>>>>>>>>> private memory region) is not dumped into the corefile for >>>>>>>>>>>> Linux. This, being a heap region (and therefore being a >>>>>>>>>>>> read-write region) is also not read in from the classes.jsa >>>>>>>>>>>> file in SA since only the read only regions are read in >>>>>>>>>>>> while processing the core file. (The expectation being that >>>>>>>>>>>> the read write regions are in the core file). >>>>>>>>>>>> >>>>>>>>>>>> Proposed solution: The proposed solution is to have the >>>>>>>>>>>> coredump_filter value corresponding to the CDS process to >>>>>>>>>>>> include bit 2 (file-backed private memory), so that the >>>>>>>>>>>> file-backed private memory region also gets dumped into the >>>>>>>>>>>> corefile. The proposed fix is in >>>>>>>>>>>> src/hotspot/os/linux/os_linux.cpp. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Jini. >>>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>> >>>> >>> >> From alexey.menkov at oracle.com Tue Dec 11 20:21:24 2018 From: alexey.menkov at oracle.com (Alex Menkov) Date: Tue, 11 Dec 2018 12:21:24 -0800 Subject: RFR (L) 8215160: Normalize spaces for remaining vmTestbase tests In-Reply-To: References: <7de505f5-4407-7961-1b71-0def2559f0bb@oracle.com> <63e7d134-1c24-ac13-ab4e-57e75df72b4e@oracle.com> <06aafed7-d05f-3207-f378-62797506637f@oracle.com> Message-ID: <4e5dc177-22cd-a98d-eff4-140d9112b60d@oracle.com> Hi Jc, Thanks for the update. LGTM. --alex On 12/10/2018 20:39, JC Beyler wrote: > Hi Alexey, > > Thanks for the review! > > Here is the fixed version: > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8215160/webrev.01/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8215160 > > Thanks again, > Jc > > > On Mon, Dec 10, 2018 at 4:41 PM Alex Menkov > wrote: > > Sorry, it was review for 8215161 > > Some minor notes for this webrev: > - nsk/jvmti/scenarios/jni_interception/JI05/ji05t001/ji05t001.cpp > for some reason there are missed spaces before "?" in statements like > ? (indx == 0)? "A" : "B" > see lines 190, 227, 243 > > Also there is some inconsistency with spaces around "%" operator - > in nsk/stress/jni/libjnistress001.cpp and > nsk/stress/jni/libjnistress003.cpp > there are no spaces like > +? ? if (allocs%printperiod == 0) { > > in other files the spaces are added > like (nsk/stress/jni/libjnistress005.cpp) > +? ? ? ? if (Exceptcalls % 1000 == 0) > > --alex > > On 12/10/2018 14:28, Alex Menkov wrote: > > +1 > > > > One minor request. > > Could you please fix > > nsk/jvmti/NativeMethodBind/nativemethbind002/nativemethbind002.cpp > > replacing > > > > if (!(methNam == NULL)) > > and > > if (!(methSig == NULL)) > > > > with > > if (methNam != NULL) > > and > > if (methSig != NULL) > > > > No need for new round of the review. > > > > --alex > > > > > > On 12/10/2018 13:54, serguei.spitsyn at oracle.com > wrote: > >> Hi Jc, > >> > >> LGTM > >> > >> Thank you for the re-post! > >> Serguei > >> > >> > >> On 12/10/18 13:46, JC Beyler wrote: > >>> Hi all, > >>> > >>> Let's try this again; my apologies for the spam. > >>> > >>> Could I get a review that normalizes spaces around comparisons and > >>> the ternary operator? This is the second of two webrevs to > handle this. > >>> > >>> Webrev: http://cr.openjdk.java.net/~jcbeyler/8215160/webrev.00/ > >>> > >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8215160 > >>> > >>> Thanks, > >>> Jc > >>> > >> > > > > -- > > Thanks, > Jc From jcbeyler at google.com Tue Dec 11 20:46:21 2018 From: jcbeyler at google.com (JC Beyler) Date: Tue, 11 Dec 2018 12:46:21 -0800 Subject: RFR (L) 8215160: Normalize spaces for remaining vmTestbase tests In-Reply-To: <4e5dc177-22cd-a98d-eff4-140d9112b60d@oracle.com> References: <7de505f5-4407-7961-1b71-0def2559f0bb@oracle.com> <63e7d134-1c24-ac13-ab4e-57e75df72b4e@oracle.com> <06aafed7-d05f-3207-f378-62797506637f@oracle.com> <4e5dc177-22cd-a98d-eff4-140d9112b60d@oracle.com> Message-ID: Thanks Alex for the review! Tested & pushed, Jc On Tue, Dec 11, 2018 at 12:21 PM Alex Menkov wrote: > Hi Jc, > > Thanks for the update. > LGTM. > > --alex > > On 12/10/2018 20:39, JC Beyler wrote: > > Hi Alexey, > > > > Thanks for the review! > > > > Here is the fixed version: > > > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8215160/webrev.01/ > > Bug: https://bugs.openjdk.java.net/browse/JDK-8215160 > > > > Thanks again, > > Jc > > > > > > On Mon, Dec 10, 2018 at 4:41 PM Alex Menkov > > wrote: > > > > Sorry, it was review for 8215161 > > > > Some minor notes for this webrev: > > - nsk/jvmti/scenarios/jni_interception/JI05/ji05t001/ji05t001.cpp > > for some reason there are missed spaces before "?" in statements like > > (indx == 0)? "A" : "B" > > see lines 190, 227, 243 > > > > Also there is some inconsistency with spaces around "%" operator - > > in nsk/stress/jni/libjnistress001.cpp and > > nsk/stress/jni/libjnistress003.cpp > > there are no spaces like > > + if (allocs%printperiod == 0) { > > > > in other files the spaces are added > > like (nsk/stress/jni/libjnistress005.cpp) > > + if (Exceptcalls % 1000 == 0) > > > > --alex > > > > On 12/10/2018 14:28, Alex Menkov wrote: > > > +1 > > > > > > One minor request. > > > Could you please fix > > > nsk/jvmti/NativeMethodBind/nativemethbind002/nativemethbind002.cpp > > > replacing > > > > > > if (!(methNam == NULL)) > > > and > > > if (!(methSig == NULL)) > > > > > > with > > > if (methNam != NULL) > > > and > > > if (methSig != NULL) > > > > > > No need for new round of the review. > > > > > > --alex > > > > > > > > > On 12/10/2018 13:54, serguei.spitsyn at oracle.com > > wrote: > > >> Hi Jc, > > >> > > >> LGTM > > >> > > >> Thank you for the re-post! > > >> Serguei > > >> > > >> > > >> On 12/10/18 13:46, JC Beyler wrote: > > >>> Hi all, > > >>> > > >>> Let's try this again; my apologies for the spam. > > >>> > > >>> Could I get a review that normalizes spaces around comparisons > and > > >>> the ternary operator? This is the second of two webrevs to > > handle this. > > >>> > > >>> Webrev: http://cr.openjdk.java.net/~jcbeyler/8215160/webrev.00/ > > >>> > > >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8215160 > > >>> > > >>> Thanks, > > >>> Jc > > >>> > > >> > > > > > > > > -- > > > > Thanks, > > Jc > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Tue Dec 11 23:11:56 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 11 Dec 2018 15:11:56 -0800 Subject: RFR (M) 8201655: Add thread-enabled support for the Heap Sampling In-Reply-To: References: <0e62b2cc-c28e-5cc3-59df-5ce5171d4475@oracle.com> Message-ID: <102a9f1b-8f07-06f4-7749-4d04eccd3a91@oracle.com> Hi Jc, Alex will take a look at the test update. Thanks, Serguei On 11/12/18 9:53 AM, JC Beyler wrote: > Hi Serguei, > > Thanks for the update and thanks for testing mach5. Serguei sent me > that the testing passed mach5 testing, could I get another review to > be able to push it? > > Thanks! > Jc > > On Tue, Nov 6, 2018 at 10:41 PM serguei.spitsyn at oracle.com > > wrote: > > Hi Jc, > > Thank you for the update! > It looks good. > It is great that testing on your side is Okay. > > I'll submit a mach5 job soon (today or tomorrow). > > Thanks, > Serguei > > > On 11/6/18 20:03, JC Beyler wrote: >> Hi Serguei, >> >> You are right, I should have reverted the memAllocator.cpp file, >> sorry about that. >> >> Here is the new webrev: >> http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.04/ >> >> >> I think we are good by testing standards, like I >> said?HeapMonitorThreadTest.java tests multiple threads. I did >> test an example with a thousand threads and I get the samples >> from 1000 threads so it seems to work there too. >> >> Per thread is tested via the new >> HeapMonitorThreadDisabledTest.java so I think we are good there too. >> >> I would recommend a mach-5 testing just in case for this one if >> you can, it will be better to have that reinsurance. >> >> Thanks for your help, >> Jc >> >> On Tue, Nov 6, 2018 at 4:29 PM > > wrote: >> >> Hi Jc, >> >> Not sure, I understand a motivation for this change: >> >> - if (JvmtiExport::should_post_sampled_object_alloc()) { >> + { >> >> Also, I'm not sure this is still needed: >> >> +#include "prims/jvmtiEventController.inline.hpp" >> +#include "prims/jvmtiThreadState.inline.hpp" >> >> I expected you'd just revert all the changes in the >> memAllocator.cpp. >> >> Also, it is up to you to make a decision if these >> performance-related fix is needed or not. >> >> But it needs to be well tested so that both global+thread >> event management works correctly. >> >> Thanks, >> Serguei >> >> >> On 11/6/18 9:42 AM, JC Beyler wrote: >>> Hi Serguei, >>> >>> Yes exactly it was an optimization. When using a 512k >>> sampling rate, I don't see a no real difference (the >>> overhead is anyway low for that sampling rate), I imagine >>> there would be a difference if trying to sample every >>> allocation or with a low sampling interval. But because you >>> are right and it is an optimization of the system and not a >>> functional need, I've reverted it and now the webrev is >>> updated here: >>> >>> Webrev: >>> http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.03/ >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8201655 >>> >>> The incremental webrev is here: >>> http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.02_03/ >>> >>> >>> Let me know what you think, >>> Jc >>> >>> On Mon, Nov 5, 2018 at 6:51 PM serguei.spitsyn at oracle.com >>> >>> >> > wrote: >>> >>> Hi Jc, >>> >>> Okay, I see your point: the change in memAllocator.cpp >>> is for performance. >>> Do you have any measurements showing a performance >>> difference? >>> Also, do you need me to submit a mach5 test run? >>> >>> Thanks, >>> Serguei >>> >>> >>> On 11/5/18 15:14, JC Beyler wrote: >>>> Hi Serguei, >>>> >>>> First off, thanks as always for looking at this :-) Let >>>> me inline my answers: >>>> >>>> I actually "struggled" with this part of the change. My >>>> change is correct semantically and if you care about >>>> performance for when sampling a given thread. >>>> Your change will work semantically but the performance >>>> is the same as the global sampling. >>>> >>>> What I mean by working semantically is that that the >>>> tests and the code will work. However, this means that >>>> all threads will be doing the sampling work but when >>>> the code will post the event here: >>>> ? ?-> >>>> http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.02/src/hotspot/share/prims/jvmtiExport.cpp.udiff.html >>>> >>>> >>>> (which is why your suggestion works, the change in >>>> jvmtiExport basically ensures only the threads >>>> requested are posting events) >>>> >>>> The code will check that we actually only post for >>>> threads we care about. The code above ensures that only >>>> threads that were requested to be sampling are the ones >>>> that are sampling internally. >>>> >>>> Note: I REALLY prefer your suggestion for two reasons: >>>> ? - We do not change the runtime/GC code at all, it >>>> remains "simple" >>>> ? - The overhead in the general case goes away and this >>>> is a NOP for my actual use-case from a performance >>>> point of view (sampling every thread) >>>> >>>> But: >>>> ? - Then sampling per thread really is just telling the >>>> system don't pollute the callbacks, though internally >>>> you are doing all the work anyway. >>>> >>>> Let me know which you prefer :) >>>> >>>> >>>> Also, do you see that enabling the sampling events globally still works? >>>> >>>> >>>> Yes, otherwise HeapMonitorThreadTest.java would fail >>>> since it checks that. >>>> >>>> http://cr.openjdk.java.net/%7Ejcbeyler/8201655/webrev.02/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.c.frames.html >>>> >>>> A couple of places where err is declared as int instead of jvmtiError: >>>> 714 int err; >>>> 742 int err; Should not be silent in a case of >>>> JVMTI error: 744 err = (*jvmti)->GetThreadInfo(jvmti, thread, &info); >>>> 745 if (err != JVMTI_ERROR_NONE) { >>>> 746 return; >>>> >>>> >>>> >>>> Done and done, I added a fprintf on stderr saying the >>>> GetThreadInfo failed and the test is ignoring the add >>>> count. >>>> >>>> Thanks again for looking and let me know what you think, >>>> Jc >>>> >>>> On Mon, Nov 5, 2018 at 2:25 PM >>>> serguei.spitsyn at oracle.com >>>> >>>> >>> > wrote: >>>> >>>> Hi Jc, >>>> >>>> It looks good in general but I have some comments >>>> below. >>>> >>>> >>>> http://cr.openjdk.java.net/%7Ejcbeyler/8201655/webrev.02/src/hotspot/share/gc/shared/memAllocator.cpp.udiff.html >>>> >>>> +static bool thread_enabled_for_one_jvmti_env() { >>>> + JavaThread *thread = JavaThread::current(); >>>> + JvmtiThreadState *state = >>>> thread->jvmti_thread_state(); >>>> + if (state == NULL) { >>>> + return false; >>>> + } >>>> + >>>> + JvmtiEnvThreadStateIterator it(state); >>>> + for (JvmtiEnvThreadState* ets = it.first(); ets >>>> != NULL; ets = it.next(ets)) { >>>> + if >>>> (ets->is_enabled(JVMTI_EVENT_SAMPLED_OBJECT_ALLOC)) { >>>> + return true; >>>> + } >>>> + } >>>> + >>>> + return false; >>>> +} >>>> + >>>> void MemAllocator::Allocation::notify_allocation_jvmti_sampler() { >>>> // support for JVMTI VMObjectAlloc event (no-op if not enabled) >>>> JvmtiExport::vm_object_alloc_event_collector(obj()); >>>> >>>> if (!JvmtiExport::should_post_sampled_object_alloc()) { >>>> // Sampling disabled >>>> return; >>>> } >>>> >>>> + // Sampling is enabled for at least one thread, >>>> is it this one? >>>> + if (!thread_enabled_for_one_jvmti_env()) { >>>> + return; >>>> + } >>>> + I don't think you need this change as this >>>> condition already does it: if (!JvmtiExport::should_post_sampled_object_alloc()) { >>>> // Sampling disabled >>>> return; >>>> } >>>> >>>> Please, look at the following line in the jvmtiEventController.cpp: >>>> JvmtiExport::set_should_post_sampled_object_alloc((any_env_thread_enabled & SAMPLED_OBJECT_ALLOC_BIT) != 0); >>>> >>>> I hope, testing will prove my suggestion is correct. >>>> Also, do you see that enabling the sampling events globally still works? >>>> >>>> >>>> http://cr.openjdk.java.net/%7Ejcbeyler/8201655/webrev.02/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.c.frames.html >>>> >>>> A couple of places where err is declared as int instead of jvmtiError: >>>> 714 int err; >>>> 742 int err; Should not be silent in a case of >>>> JVMTI error: 744 err = (*jvmti)->GetThreadInfo(jvmti, thread, &info); >>>> 745 if (err != JVMTI_ERROR_NONE) { >>>> 746 return; >>>> >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 10/26/18 10:48, JC Beyler wrote: >>>>> Hi all, >>>>> >>>>> When working on the heap sampling, I had promised >>>>> to do the per thread event so here it is! >>>>> >>>>> Could I get a review for this: >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.02/ >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8201655 >>>>> >>>>> I was thinking of adding GC-dev for the >>>>> memAllocator change once I get favorable reviews >>>>> for the rest of the change. >>>>> >>>>> I've done a bit of performance testing and on the >>>>> Dacapo benchmark I see no change in performance >>>>> when turned off (logical, any code change is >>>>> behind a flag check already in place) and when >>>>> turned on it is comparable to the current performance. >>>>> >>>>> (More information is: I see a very slight >>>>> degradation if we are doing 512k sampling but no >>>>> degradation at 2MB). >>>>> >>>>> Thanks, >>>>> Jc >>>> >>>> >>>> >>>> -- >>>> >>>> Thanks, >>>> Jc >>> >>> >>> >>> -- >>> >>> Thanks, >>> Jc >> >> >> >> -- >> >> Thanks, >> Jc > > > > -- > > Thanks, > Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexey.menkov at oracle.com Wed Dec 12 00:42:28 2018 From: alexey.menkov at oracle.com (Alex Menkov) Date: Tue, 11 Dec 2018 16:42:28 -0800 Subject: RFR (M) 8201655: Add thread-enabled support for the Heap Sampling In-Reply-To: <102a9f1b-8f07-06f4-7749-4d04eccd3a91@oracle.com> References: <0e62b2cc-c28e-5cc3-59df-5ce5171d4475@oracle.com> <102a9f1b-8f07-06f4-7749-4d04eccd3a91@oracle.com> Message-ID: <1b8ab7a1-ec08-4397-020e-24463688112b@oracle.com> Hi Jc, The fix looks good. The only note is checkThreadSamplesOnlyFrom native function implementation - the cycle looks confusing. As far as I got the function should check that thread_stats contains only 1 thread and name of the thread is the same as name of the specified thread. And for error analysis it would be great to provide good error description. So I'd make it like if (thread_stats.number_threads != 1) { fprintf(stderr, "Wrong thread number: %d (expected 1)\n", thread_stats.number_threads); return FALSE; } if (strcmp(expected_name, thread_stats.threads[i]) != 0) { fprintf(stderr, "Unexpected thread name: '%s' (expected '%s')\n", thread_stats.threads[i], expected_name); return FALSE; } return TRUE; --alex On 12/11/2018 15:11, serguei.spitsyn at oracle.com wrote: > Hi Jc, > > Alex will take a look at the test update. > > Thanks, > Serguei > > On 11/12/18 9:53 AM, JC Beyler wrote: >> Hi Serguei, >> >> Thanks for the update and thanks for testing mach5. Serguei sent me >> that the testing passed mach5 testing, could I get another review to >> be able to push it? >> >> Thanks! >> Jc >> >> On Tue, Nov 6, 2018 at 10:41 PM serguei.spitsyn at oracle.com >> > > wrote: >> >> Hi Jc, >> >> Thank you for the update! >> It looks good. >> It is great that testing on your side is Okay. >> >> I'll submit a mach5 job soon (today or tomorrow). >> >> Thanks, >> Serguei >> >> >> On 11/6/18 20:03, JC Beyler wrote: >>> Hi Serguei, >>> >>> You are right, I should have reverted the memAllocator.cpp file, >>> sorry about that. >>> >>> Here is the new webrev: >>> http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.04/ >>> >>> >>> I think we are good by testing standards, like I >>> said?HeapMonitorThreadTest.java tests multiple threads. I did >>> test an example with a thousand threads and I get the samples >>> from 1000 threads so it seems to work there too. >>> >>> Per thread is tested via the new >>> HeapMonitorThreadDisabledTest.java so I think we are good there too. >>> >>> I would recommend a mach-5 testing just in case for this one if >>> you can, it will be better to have that reinsurance. >>> >>> Thanks for your help, >>> Jc >>> >>> On Tue, Nov 6, 2018 at 4:29 PM >> > wrote: >>> >>> Hi Jc, >>> >>> Not sure, I understand a motivation for this change: >>> >>> - if (JvmtiExport::should_post_sampled_object_alloc()) { >>> + { >>> >>> Also, I'm not sure this is still needed: >>> >>> +#include "prims/jvmtiEventController.inline.hpp" >>> +#include "prims/jvmtiThreadState.inline.hpp" >>> >>> I expected you'd just revert all the changes in the >>> memAllocator.cpp. >>> >>> Also, it is up to you to make a decision if these >>> performance-related fix is needed or not. >>> >>> But it needs to be well tested so that both global+thread >>> event management works correctly. >>> >>> Thanks, >>> Serguei >>> >>> >>> On 11/6/18 9:42 AM, JC Beyler wrote: >>>> Hi Serguei, >>>> >>>> Yes exactly it was an optimization. When using a 512k >>>> sampling rate, I don't see a no real difference (the >>>> overhead is anyway low for that sampling rate), I imagine >>>> there would be a difference if trying to sample every >>>> allocation or with a low sampling interval. But because you >>>> are right and it is an optimization of the system and not a >>>> functional need, I've reverted it and now the webrev is >>>> updated here: >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.03/ >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8201655 >>>> >>>> The incremental webrev is here: >>>> http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.02_03/ >>>> >>>> >>>> Let me know what you think, >>>> Jc >>>> >>>> On Mon, Nov 5, 2018 at 6:51 PM serguei.spitsyn at oracle.com >>>> >>>> >>> > wrote: >>>> >>>> Hi Jc, >>>> >>>> Okay, I see your point: the change in memAllocator.cpp >>>> is for performance. >>>> Do you have any measurements showing a performance >>>> difference? >>>> Also, do you need me to submit a mach5 test run? >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 11/5/18 15:14, JC Beyler wrote: >>>>> Hi Serguei, >>>>> >>>>> First off, thanks as always for looking at this :-) Let >>>>> me inline my answers: >>>>> >>>>> I actually "struggled" with this part of the change. My >>>>> change is correct semantically and if you care about >>>>> performance for when sampling a given thread. >>>>> Your change will work semantically but the performance >>>>> is the same as the global sampling. >>>>> >>>>> What I mean by working semantically is that that the >>>>> tests and the code will work. However, this means that >>>>> all threads will be doing the sampling work but when >>>>> the code will post the event here: >>>>> ? ?-> >>>>> http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.02/src/hotspot/share/prims/jvmtiExport.cpp.udiff.html >>>>> >>>>> >>>>> (which is why your suggestion works, the change in >>>>> jvmtiExport basically ensures only the threads >>>>> requested are posting events) >>>>> >>>>> The code will check that we actually only post for >>>>> threads we care about. The code above ensures that only >>>>> threads that were requested to be sampling are the ones >>>>> that are sampling internally. >>>>> >>>>> Note: I REALLY prefer your suggestion for two reasons: >>>>> ? - We do not change the runtime/GC code at all, it >>>>> remains "simple" >>>>> ? - The overhead in the general case goes away and this >>>>> is a NOP for my actual use-case from a performance >>>>> point of view (sampling every thread) >>>>> >>>>> But: >>>>> ? - Then sampling per thread really is just telling the >>>>> system don't pollute the callbacks, though internally >>>>> you are doing all the work anyway. >>>>> >>>>> Let me know which you prefer :) >>>>> >>>>> >>>>> Also, do you see that enabling the sampling events globally still works? >>>>> >>>>> >>>>> Yes, otherwise HeapMonitorThreadTest.java would fail >>>>> since it checks that. >>>>> >>>>> http://cr.openjdk.java.net/%7Ejcbeyler/8201655/webrev.02/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.c.frames.html >>>>> >>>>> A couple of places where err is declared as int instead of jvmtiError: >>>>> 714 int err; >>>>> 742 int err; Should not be silent in a case of >>>>> JVMTI error: 744 err = (*jvmti)->GetThreadInfo(jvmti, thread, &info); >>>>> 745 if (err != JVMTI_ERROR_NONE) { >>>>> 746 return; >>>>> >>>>> >>>>> >>>>> Done and done, I added a fprintf on stderr saying the >>>>> GetThreadInfo failed and the test is ignoring the add >>>>> count. >>>>> >>>>> Thanks again for looking and let me know what you think, >>>>> Jc >>>>> >>>>> On Mon, Nov 5, 2018 at 2:25 PM >>>>> serguei.spitsyn at oracle.com >>>>> >>>>> >>>> > wrote: >>>>> >>>>> Hi Jc, >>>>> >>>>> It looks good in general but I have some comments >>>>> below. >>>>> >>>>> >>>>> http://cr.openjdk.java.net/%7Ejcbeyler/8201655/webrev.02/src/hotspot/share/gc/shared/memAllocator.cpp.udiff.html >>>>> >>>>> +static bool thread_enabled_for_one_jvmti_env() { >>>>> + JavaThread *thread = JavaThread::current(); >>>>> + JvmtiThreadState *state = >>>>> thread->jvmti_thread_state(); >>>>> + if (state == NULL) { >>>>> + return false; >>>>> + } >>>>> + >>>>> + JvmtiEnvThreadStateIterator it(state); >>>>> + for (JvmtiEnvThreadState* ets = it.first(); ets >>>>> != NULL; ets = it.next(ets)) { >>>>> + if >>>>> (ets->is_enabled(JVMTI_EVENT_SAMPLED_OBJECT_ALLOC)) { >>>>> + return true; >>>>> + } >>>>> + } >>>>> + >>>>> + return false; >>>>> +} >>>>> + >>>>> void MemAllocator::Allocation::notify_allocation_jvmti_sampler() { >>>>> // support for JVMTI VMObjectAlloc event (no-op if not enabled) >>>>> JvmtiExport::vm_object_alloc_event_collector(obj()); >>>>> >>>>> if (!JvmtiExport::should_post_sampled_object_alloc()) { >>>>> // Sampling disabled >>>>> return; >>>>> } >>>>> >>>>> + // Sampling is enabled for at least one thread, >>>>> is it this one? >>>>> + if (!thread_enabled_for_one_jvmti_env()) { >>>>> + return; >>>>> + } >>>>> + I don't think you need this change as this >>>>> condition already does it: if (!JvmtiExport::should_post_sampled_object_alloc()) { >>>>> // Sampling disabled >>>>> return; >>>>> } >>>>> >>>>> Please, look at the following line in the jvmtiEventController.cpp: >>>>> JvmtiExport::set_should_post_sampled_object_alloc((any_env_thread_enabled & SAMPLED_OBJECT_ALLOC_BIT) != 0); >>>>> >>>>> I hope, testing will prove my suggestion is correct. >>>>> Also, do you see that enabling the sampling events globally still works? >>>>> >>>>> >>>>> http://cr.openjdk.java.net/%7Ejcbeyler/8201655/webrev.02/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.c.frames.html >>>>> >>>>> A couple of places where err is declared as int instead of jvmtiError: >>>>> 714 int err; >>>>> 742 int err; Should not be silent in a case of >>>>> JVMTI error: 744 err = (*jvmti)->GetThreadInfo(jvmti, thread, &info); >>>>> 745 if (err != JVMTI_ERROR_NONE) { >>>>> 746 return; >>>>> >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>> On 10/26/18 10:48, JC Beyler wrote: >>>>>> Hi all, >>>>>> >>>>>> When working on the heap sampling, I had promised >>>>>> to do the per thread event so here it is! >>>>>> >>>>>> Could I get a review for this: >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.02/ >>>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8201655 >>>>>> >>>>>> I was thinking of adding GC-dev for the >>>>>> memAllocator change once I get favorable reviews >>>>>> for the rest of the change. >>>>>> >>>>>> I've done a bit of performance testing and on the >>>>>> Dacapo benchmark I see no change in performance >>>>>> when turned off (logical, any code change is >>>>>> behind a flag check already in place) and when >>>>>> turned on it is comparable to the current performance. >>>>>> >>>>>> (More information is: I see a very slight >>>>>> degradation if we are doing 512k sampling but no >>>>>> degradation at 2MB). >>>>>> >>>>>> Thanks, >>>>>> Jc >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Thanks, >>>>> Jc >>>> >>>> >>>> >>>> -- >>>> >>>> Thanks, >>>> Jc >>> >>> >>> >>> -- >>> >>> Thanks, >>> Jc >> >> >> >> -- >> >> Thanks, >> Jc > From serguei.spitsyn at oracle.com Wed Dec 12 01:01:47 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 11 Dec 2018 17:01:47 -0800 Subject: RFR (M) 8201655: Add thread-enabled support for the Heap Sampling In-Reply-To: <1b8ab7a1-ec08-4397-020e-24463688112b@oracle.com> References: <0e62b2cc-c28e-5cc3-59df-5ce5171d4475@oracle.com> <102a9f1b-8f07-06f4-7749-4d04eccd3a91@oracle.com> <1b8ab7a1-ec08-4397-020e-24463688112b@oracle.com> Message-ID: Hi Alex, Nice catch! It is about the following fragment: 726 for (i = 0; i < thread_stats.number_threads; i++) { 727 if (strcmp(expected_name, thread_stats.threads[i])) { 728 return FALSE; 729 } else { 730 found_thread = TRUE; 731 } 732 } 733 return found_thread; 734 } Also, I'd also use 'count' in place of 'number'. It is to avoid association with thread identification number. Thanks, Serguei On 12/11/18 4:42 PM, Alex Menkov wrote: > Hi Jc, > > The fix looks good. > The only note is checkThreadSamplesOnlyFrom native function > implementation - the cycle looks confusing. > As far as I got the function should check that thread_stats contains > only 1 thread and name of the thread is the same as name of the > specified thread. > And for error analysis it would be great to provide good error > description. > So I'd make it like > > if (thread_stats.number_threads != 1) { > ? fprintf(stderr, "Wrong thread number: %d (expected 1)\n", > thread_stats.number_threads); > ? return FALSE; > } > if (strcmp(expected_name, thread_stats.threads[i]) != 0) { > ? fprintf(stderr, "Unexpected thread name: '%s' (expected '%s')\n", > thread_stats.threads[i], expected_name); > ? return FALSE; > } > return TRUE; > > --alex > > On 12/11/2018 15:11, serguei.spitsyn at oracle.com wrote: >> Hi Jc, >> >> Alex will take a look at the test update. >> >> Thanks, >> Serguei >> >> On 11/12/18 9:53 AM, JC Beyler wrote: >>> Hi Serguei, >>> >>> Thanks for the update and thanks for testing mach5. Serguei sent me >>> that the testing passed mach5 testing, could I get another review to >>> be able to push it? >>> >>> Thanks! >>> Jc >>> >>> On Tue, Nov 6, 2018 at 10:41 PM serguei.spitsyn at oracle.com >>> >> > wrote: >>> >>> ??? Hi Jc, >>> >>> ??? Thank you for the update! >>> ??? It looks good. >>> ??? It is great that testing on your side is Okay. >>> >>> ??? I'll submit a mach5 job soon (today or tomorrow). >>> >>> ??? Thanks, >>> ??? Serguei >>> >>> >>> ??? On 11/6/18 20:03, JC Beyler wrote: >>>> ??? Hi Serguei, >>>> >>>> ??? You are right, I should have reverted the memAllocator.cpp file, >>>> ??? sorry about that. >>>> >>>> ??? Here is the new webrev: >>>> ??? http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.04/ >>>> >>>> >>>> ??? I think we are good by testing standards, like I >>>> ??? said?HeapMonitorThreadTest.java tests multiple threads. I did >>>> ??? test an example with a thousand threads and I get the samples >>>> ??? from 1000 threads so it seems to work there too. >>>> >>>> ??? Per thread is tested via the new >>>> ??? HeapMonitorThreadDisabledTest.java so I think we are good there >>>> too. >>>> >>>> ??? I would recommend a mach-5 testing just in case for this one if >>>> ??? you can, it will be better to have that reinsurance. >>>> >>>> ??? Thanks for your help, >>>> ??? Jc >>>> >>>> ??? On Tue, Nov 6, 2018 at 4:29 PM >>> ??? > wrote: >>>> >>>> ??????? Hi Jc, >>>> >>>> ??????? Not sure, I understand a motivation for this change: >>>> >>>> ??????? - if (JvmtiExport::should_post_sampled_object_alloc()) { >>>> ??????? + { >>>> >>>> ??????? Also, I'm not sure this is still needed: >>>> >>>> ??????? +#include "prims/jvmtiEventController.inline.hpp" >>>> ??????? +#include "prims/jvmtiThreadState.inline.hpp" >>>> >>>> ??????? I expected you'd just revert all the changes in the >>>> ??????? memAllocator.cpp. >>>> >>>> ??????? Also, it is up to you to make a decision if these >>>> ??????? performance-related fix is needed or not. >>>> >>>> ??????? But it needs to be well tested so that both global+thread >>>> ??????? event management works correctly. >>>> >>>> ??????? Thanks, >>>> ??????? Serguei >>>> >>>> >>>> ??????? On 11/6/18 9:42 AM, JC Beyler wrote: >>>>> ??????? Hi Serguei, >>>>> >>>>> ??????? Yes exactly it was an optimization. When using a 512k >>>>> ??????? sampling rate, I don't see a no real difference (the >>>>> ??????? overhead is anyway low for that sampling rate), I imagine >>>>> ??????? there would be a difference if trying to sample every >>>>> ??????? allocation or with a low sampling interval. But because you >>>>> ??????? are right and it is an optimization of the system and not a >>>>> ??????? functional need, I've reverted it and now the webrev is >>>>> ??????? updated here: >>>>> >>>>> ??????? Webrev: >>>>> http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.03/ >>>>> >>>>> ??????? Bug: https://bugs.openjdk.java.net/browse/JDK-8201655 >>>>> >>>>> ??????? The incremental webrev is here: >>>>> http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.02_03/ >>>>> >>>>> >>>>> ??????? Let me know what you think, >>>>> ??????? Jc >>>>> >>>>> ??????? On Mon, Nov 5, 2018 at 6:51 PM serguei.spitsyn at oracle.com >>>>> ??????? >>>>> ??????? >>>> ??????? > wrote: >>>>> >>>>> ??????????? Hi Jc, >>>>> >>>>> ??????????? Okay, I see your point: the change in memAllocator.cpp >>>>> ??????????? is for performance. >>>>> ??????????? Do you have any measurements showing a performance >>>>> ??????????? difference? >>>>> ??????????? Also, do you need me to submit a mach5 test run? >>>>> >>>>> ??????????? Thanks, >>>>> ??????????? Serguei >>>>> >>>>> >>>>> ??????????? On 11/5/18 15:14, JC Beyler wrote: >>>>>> ??????????? Hi Serguei, >>>>>> >>>>>> ??????????? First off, thanks as always for looking at this :-) Let >>>>>> ??????????? me inline my answers: >>>>>> >>>>>> ??????????? I actually "struggled" with this part of the change. My >>>>>> ??????????? change is correct semantically and if you care about >>>>>> ??????????? performance for when sampling a given thread. >>>>>> ??????????? Your change will work semantically but the performance >>>>>> ??????????? is the same as the global sampling. >>>>>> >>>>>> ??????????? What I mean by working semantically is that that the >>>>>> ??????????? tests and the code will work. However, this means that >>>>>> ??????????? all threads will be doing the sampling work but when >>>>>> ??????????? the code will post the event here: >>>>>> ??????????? ? ?-> >>>>>> http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.02/src/hotspot/share/prims/jvmtiExport.cpp.udiff.html >>>>>> >>>>>> >>>>>> ??????????? (which is why your suggestion works, the change in >>>>>> ??????????? jvmtiExport basically ensures only the threads >>>>>> ??????????? requested are posting events) >>>>>> >>>>>> ??????????? The code will check that we actually only post for >>>>>> ??????????? threads we care about. The code above ensures that only >>>>>> ??????????? threads that were requested to be sampling are the ones >>>>>> ??????????? that are sampling internally. >>>>>> >>>>>> ??????????? Note: I REALLY prefer your suggestion for two reasons: >>>>>> ??????????? ? - We do not change the runtime/GC code at all, it >>>>>> ??????????? remains "simple" >>>>>> ??????????? ? - The overhead in the general case goes away and this >>>>>> ??????????? is a NOP for my actual use-case from a performance >>>>>> ??????????? point of view (sampling every thread) >>>>>> >>>>>> ??????????? But: >>>>>> ??????????? ? - Then sampling per thread really is just telling the >>>>>> ??????????? system don't pollute the callbacks, though internally >>>>>> ??????????? you are doing all the work anyway. >>>>>> >>>>>> ??????????? Let me know which you prefer :) >>>>>> >>>>>> >>>>>> ????????????????? Also, do you see that enabling the sampling >>>>>> events globally still works? >>>>>> >>>>>> >>>>>> ??????????? Yes, otherwise HeapMonitorThreadTest.java would fail >>>>>> ??????????? since it checks that. >>>>>> >>>>>> http://cr.openjdk.java.net/%7Ejcbeyler/8201655/webrev.02/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.c.frames.html >>>>>> >>>>>> ????????????????? A couple of places where err is declared as int >>>>>> instead of jvmtiError: >>>>>> ???????????????????? 714?? int err; >>>>>> ??????????????? 742 int err; Should not be silent in a case of >>>>>> ??????????????? JVMTI error:?? 744?? err = >>>>>> (*jvmti)->GetThreadInfo(jvmti, thread, &info); >>>>>> ????????????????? 745?? if (err != JVMTI_ERROR_NONE) { >>>>>> ??????????????? 746 return; >>>>>> >>>>>> >>>>>> >>>>>> ??????????? Done and done, I added a fprintf on stderr saying the >>>>>> ??????????? GetThreadInfo failed and the test is ignoring the add >>>>>> ??????????? count. >>>>>> >>>>>> ??????????? Thanks again for looking and let me know what you think, >>>>>> ??????????? Jc >>>>>> >>>>>> ??????????? On Mon, Nov 5, 2018 at 2:25 PM >>>>>> ??????????? serguei.spitsyn at oracle.com >>>>>> ??????????? >>>>>> ??????????? >>>>> > wrote: >>>>>> >>>>>> ??????????????? Hi Jc, >>>>>> >>>>>> ??????????????? It looks good in general but I have some comments >>>>>> ??????????????? below. >>>>>> >>>>>> >>>>>> http://cr.openjdk.java.net/%7Ejcbeyler/8201655/webrev.02/src/hotspot/share/gc/shared/memAllocator.cpp.udiff.html >>>>>> >>>>>> ??????????????? +static bool thread_enabled_for_one_jvmti_env() { >>>>>> ??????????????? + JavaThread *thread = JavaThread::current(); >>>>>> ??????????????? + JvmtiThreadState *state = >>>>>> ??????????????? thread->jvmti_thread_state(); >>>>>> ??????????????? + if (state == NULL) { >>>>>> ??????????????? + return false; >>>>>> ??????????????? + } >>>>>> ??????????????? + >>>>>> ??????????????? + JvmtiEnvThreadStateIterator it(state); >>>>>> ??????????????? + for (JvmtiEnvThreadState* ets = it.first(); ets >>>>>> ??????????????? != NULL; ets = it.next(ets)) { >>>>>> ??????????????? + if >>>>>> (ets->is_enabled(JVMTI_EVENT_SAMPLED_OBJECT_ALLOC)) { >>>>>> ??????????????? + return true; >>>>>> ??????????????? + } >>>>>> ??????????????? + } >>>>>> ??????????????? + >>>>>> ??????????????? + return false; >>>>>> ??????????????? +} >>>>>> ??????????????? + >>>>>> ????????????????? void >>>>>> MemAllocator::Allocation::notify_allocation_jvmti_sampler() { >>>>>> ??????????????????? // support for JVMTI VMObjectAlloc event >>>>>> (no-op if not enabled) >>>>>> JvmtiExport::vm_object_alloc_event_collector(obj()); >>>>>> ????????????????? ??????????????????? if >>>>>> (!JvmtiExport::should_post_sampled_object_alloc()) { >>>>>> ????????????????????? // Sampling disabled >>>>>> ????????????????????? return; >>>>>> ??????????????????? } >>>>>> ????????????????? ??????????????? + // Sampling is enabled for at >>>>>> least one thread, >>>>>> ??????????????? is it this one? >>>>>> ??????????????? + if (!thread_enabled_for_one_jvmti_env()) { >>>>>> ??????????????? + return; >>>>>> ??????????????? + } >>>>>> ??????????????? + I don't think you need this change as this >>>>>> ??????????????? condition already does it:???? if >>>>>> (!JvmtiExport::should_post_sampled_object_alloc()) { >>>>>> ????????????????????? // Sampling disabled >>>>>> ????????????????????? return; >>>>>> ??????????????????? } >>>>>> >>>>>> ????????????????? Please, look at the following line in the >>>>>> jvmtiEventController.cpp: >>>>>> JvmtiExport::set_should_post_sampled_object_alloc((any_env_thread_enabled >>>>>> & SAMPLED_OBJECT_ALLOC_BIT) != 0); >>>>>> >>>>>> ????????????????? I hope, testing will prove my suggestion is >>>>>> correct. >>>>>> ????????????????? Also, do you see that enabling the sampling >>>>>> events globally still works? >>>>>> >>>>>> >>>>>> http://cr.openjdk.java.net/%7Ejcbeyler/8201655/webrev.02/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.c.frames.html >>>>>> >>>>>> ????????????????? A couple of places where err is declared as int >>>>>> instead of jvmtiError: >>>>>> ???????????????????? 714?? int err; >>>>>> ??????????????? 742 int err; Should not be silent in a case of >>>>>> ??????????????? JVMTI error:?? 744?? err = >>>>>> (*jvmti)->GetThreadInfo(jvmti, thread, &info); >>>>>> ????????????????? 745?? if (err != JVMTI_ERROR_NONE) { >>>>>> ??????????????? 746 return; >>>>>> >>>>>> >>>>>> ??????????????? Thanks, >>>>>> ??????????????? Serguei >>>>>> >>>>>> >>>>>> ??????????????? On 10/26/18 10:48, JC Beyler wrote: >>>>>>> ??????????????? Hi all, >>>>>>> >>>>>>> ??????????????? When working on the heap sampling, I had promised >>>>>>> ??????????????? to do the per thread event so here it is! >>>>>>> >>>>>>> ??????????????? Could I get a review for this: >>>>>>> ??????????????? Webrev: >>>>>>> http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.02/ >>>>>>> >>>>>>> ??????????????? Bug: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8201655 >>>>>>> >>>>>>> ??????????????? I was thinking of adding GC-dev for the >>>>>>> ??????????????? memAllocator change once I get favorable reviews >>>>>>> ??????????????? for the rest of the change. >>>>>>> >>>>>>> ??????????????? I've done a bit of performance testing and on the >>>>>>> ??????????????? Dacapo benchmark I see no change in performance >>>>>>> ??????????????? when turned off (logical, any code change is >>>>>>> ??????????????? behind a flag check already in place) and when >>>>>>> ??????????????? turned on it is comparable to the current >>>>>>> performance. >>>>>>> >>>>>>> ??????????????? (More information is: I see a very slight >>>>>>> ??????????????? degradation if we are doing 512k sampling but no >>>>>>> ??????????????? degradation at 2MB). >>>>>>> >>>>>>> ??????????????? Thanks, >>>>>>> ??????????????? Jc >>>>>> >>>>>> >>>>>> >>>>>> ??????????? -- >>>>>> ??????????? Thanks, >>>>>> ??????????? Jc >>>>> >>>>> >>>>> >>>>> ??????? -- >>>>> ??????? Thanks, >>>>> ??????? Jc >>>> >>>> >>>> >>>> ??? -- >>>> ??? Thanks, >>>> ??? Jc >>> >>> >>> >>> -- >>> >>> Thanks, >>> Jc >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From zanglin5 at jd.com Wed Dec 12 02:29:13 2018 From: zanglin5 at jd.com (=?utf-8?B?6Ien55Cz?=) Date: Wed, 12 Dec 2018 02:29:13 +0000 Subject: optimize KlassInfoTable size to power of 2 In-Reply-To: References: <140bac18565c48be88313f7cc1d24bf8@jd.com> <7fc37fd7cf24439eb3a6fff132fc7387@jd.com> <4ea1c92f-4e72-7d38-8c7a-91e2892e0c3b@redhat.com> <1ab2d28b40024ba6b299756612fa3760@jd.com> <7a1e5d87-94fd-9be9-8aab-5dd0f1a3fff9@redhat.com> <56b9099e33584796af8a508980ff4690@jd.com> Message-ID: <436fddb22d3b4fb9aec315f3150cff41@jd.com> Dear JC, That?s exactly modification I made ? Thanks a lot for making the patch! BRs, Lin From: JC Beyler [mailto:jcbeyler at google.com] Sent: Wednesday, December 12, 2018 2:27 AM To: ?? Cc: Andrew Haley ; serviceability-dev at openjdk.java.net Subject: Re: optimize KlassInfoTable size to power of 2 Hi all, I was playing with this yesterday because I was curious to see the different code generation patterns so I had the webrev ready to go: Webrev: http://cr.openjdk.java.net/~jcbeyler/8215228/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8215228 @Lin: check but I'd imagine this is what you had originally done but you also changed the size Thanks, Jc On Tue, Dec 11, 2018 at 3:44 AM ?? > wrote: Oh, to make it clear, when testing with 65536, I made the change to use hash & 65535 instead of hash % 65536. BRs, Lin -----Original Message----- From: ?? Sent: Tuesday, December 11, 2018 7:36 PM To: 'Andrew Haley' >; serviceability-dev at openjdk.java.net Subject: RE: optimize KlassInfoTable size to power of 2 >There is something that I do not understand. You said that GCC didn't >know that _num_buckets was constant. In that case, how did GCC know not to use a divide instruction when you tried 65536? The original code doesn't use _num_buckets directly when calculating the hash idx, it use a non-constant variable _size, a member of KlassInfoTable, which is assigned to _num_buckets. I guess it is the reason GCC didn't do the constant propagation in this case. And _size is not changed during the life of KlassInfoTable, it only set to zero at de-construction. In my experiment, I use _num_buckets directly instead of _size to calculate hash idx. So GCC provides the optimizations. I think your suggestion of using 65537 is valuable, And I agree that it is not easy to prove that non-prime for hash is better. So how about make the change to use constant _number_buckets directly for calculating hash idx, and leave the prime not changed. In my case, speed up JMap histo helps at avoid getting killed by a timer, especially when heap is large at ~200GB. Do you think it is reasonable to have a patch? Thanks, Lin -----Original Message----- From: Andrew Haley [mailto:aph at redhat.com] Sent: Tuesday, December 11, 2018 7:10 PM To: ?? >; serviceability-dev at openjdk.java.net Subject: Re: optimize KlassInfoTable size to power of 2 On 12/11/18 11:05 AM, Andrew Haley wrote: > unsigned mod_m(unsigned n) { > unsigned tmp = n % 65536; > tmp -= n / 65536; > if (tmp >= 65537) // overflow > tmp += 65537; > return tmp; > } NB: this assumes that unsigned int is uint64_t. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From jini.george at oracle.com Wed Dec 12 04:47:38 2018 From: jini.george at oracle.com (Jini George) Date: Wed, 12 Dec 2018 10:17:38 +0530 Subject: RFR: (S): JDK-8200613: SA: jstack throws UnmappedAddressException with a CDS core file In-Reply-To: <7db96561-7845-bee5-e276-558fb49b94c3@oracle.com> References: <864997e5-2581-7ac9-44f7-76cc31c9ad93@oracle.com> <60d2e6db-c87b-9658-57cb-b03ca881a624@oracle.com> <9eb9041d-f984-0df4-a21e-4645966a62b8@oracle.com> <42d6d604-8df5-47c4-a748-6aa25bfc9a7e@oracle.com> <2dbef041-c75e-07f6-fafc-d18f2a95f455@oracle.com> <798a4d93-1638-8785-885f-fac0b21c2417@oracle.com> <2c6a2d67-5d36-07f1-4563-c1cd2ecdf9a3@oracle.com> <536d23dc-e9bf-86d9-0c86-fb15732576ea@oracle.com> <7db96561-7845-bee5-e276-558fb49b94c3@oracle.com> Message-ID: Thank you, Coleen! -Jini. On 12/12/2018 12:58 AM, coleen.phillimore at oracle.com wrote: > > > On 12/11/18 2:37 AM, Jini George wrote: >> Thank you very much, Coleen. I have converted the flag into a >> diagnostic flag. The revised webrev is at: >> >> http://cr.openjdk.java.net/~jgeorge/8200613/webrev.03/ > > Looks great! >> >> A plain diff between this patch and the earlier one is at: >> >> http://cr.openjdk.java.net/~jgeorge/8200613/incremental_diff >> >> I will withdraw the associated CSR. > > Yes, please withdraw it.? Thank you! > > Coleen > >> >> Thanks! >> Jini. >> >> On 12/10/2018 11:55 PM, coleen.phillimore at oracle.com wrote: >>> >>> I see.? In this case, the flag should be diagnostic and not require a >>> CSR.? I suppose some documentation should be added so that sustaining >>> will know about the option in this case (Hi Kevin!) >>> >>> thanks, >>> Coleen >>> >>> >>> On 12/10/18 1:38 AM, Ioi Lam wrote: >>>> Hi Coleen, >>>> >>>> I was one of the people who suggested the DumpPrivateMappingsInCore >>>> flag. It's enabled by default, so by default all the contents of >>>> mmap'ed files with MAP_PRIVATE will be saved to the core files. >>>> >>>> The worry is, there may be some extreme cases where the JVM has >>>> mapped very large files (with NIO or JNI, etc). For example, you >>>> could have a 100GB in-memory database. For those cases, if the user >>>> is experiencing crashes, but they are unable to get a core dump >>>> (because it would be too large), they can try running with >>>> -XX:-DumpPrivateMappingsInCore. >>>> >>>> Thanks >>>> >>>> - Ioi >>>> >>>> >>>> On 12/7/18 12:03 PM, coleen.phillimore at oracle.com wrote: >>>>> >>>>> Hi Jini,? We were just talking about this new option.? If someone >>>>> gets a crash, I don't think they're going to run their application >>>>> again with -XX:-DumpPrivateMappingsInCore in the case of the core >>>>> file being too large.? So I don't know how generally useful this >>>>> option is. I think it would be better to not add it and set the bit >>>>> to include the mappings unconditionally. >>>>> >>>>> How much larger is this core file (we had trouble understanding >>>>> from the bug)?? If you need the mappings to understand and use the >>>>> SA tools on it, we want to have them. >>>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>> >>>>> On 12/7/18 2:22 PM, Jini George wrote: >>>>>> I have the revised webrev here: >>>>>> >>>>>> http://cr.openjdk.java.net/~jgeorge/8200613/webrev.02/index.html >>>>>> >>>>>> The extra changes here are to: >>>>>> >>>>>> * Introduce a new option DumpPrivateMappingsInCore to control the >>>>>> dumping of the file backed private regions into the corefile. >>>>>> >>>>>> * Close the modules file before dumping core in os::abort(). >>>>>> Currently, there is a small bug >>>>>> (https://bugs.openjdk.java.net/browse/JDK-8215026) which prevents >>>>>> the closure of the image file in unmapping the entire file. >>>>>> >>>>>> I plan to take up the unmapping of NIO MapMode.PRIVATE files as a >>>>>> separate task (https://bugs.openjdk.java.net/browse/JDK-8215027) >>>>>> since this seems a bit involved. >>>>>> >>>>>> Thanks a bunch, >>>>>> Jini. >>>>>> >>>>>> On 11/12/2018 10:26 AM, Jini George wrote: >>>>>>> Thank you very much, Chris, Kevin and Ioi for your comments! >>>>>>> >>>>>>> I will send another webrev with this change enabled under an >>>>>>> opt-out flag, as you suggest, and would look at unmapping the JDK >>>>>>> modules file and if possible, the NIO mapped files too in the >>>>>>> signal handler. >>>>>>> >>>>>>> Thanks a bunch, >>>>>>> Jini. >>>>>>> >>>>>>> On 11/9/2018 11:53 PM, Ioi Lam wrote: >>>>>>>> Hi Jini, >>>>>>>> >>>>>>>> Thanks for investigating the size expansion issue. >>>>>>>> >>>>>>>> I agree that the size increase is worth it. Even when not using >>>>>>>> SA, if we open the core file inside GDB, we cannot read certain >>>>>>>> sections in the CDS archive (such as the RO section and strings >>>>>>>> sections). That would make debugging difficult. So I am in favor >>>>>>>> of this change. >>>>>>>> >>>>>>>> For the JDK modules file, maybe we can unmap it in the signal >>>>>>>> handler, before going ahead with the core dump? I think it's >>>>>>>> hardly needed for debugging purposes. (Perhaps we can also do >>>>>>>> the same for the NIO mapped files?) >>>>>>>> >>>>>>>> A opt-flag as suggested by Kevin is a good idea. >>>>>>>> >>>>>>>> Thanks >>>>>>>> >>>>>>>> - Ioi >>>>>>>> >>>>>>>> On 11/9/18 3:29 AM, Kevin Walls wrote: >>>>>>>>> Hi Jini, >>>>>>>>> >>>>>>>>> Looks good to me.? It might be a significant increase in size >>>>>>>>> of _some_ core files, but so many core files we see are much >>>>>>>>> larger, in gigabytes++ of course, so the CDS data size should >>>>>>>>> not be such a significant increase on (I think) most files. >>>>>>>>> >>>>>>>>> The flexibiity of always having the CDS data there is very >>>>>>>>> significant.? A core file should ideally be usable, without >>>>>>>>> additionally requiring the CDS archive from the machine. That >>>>>>>>> additional human round-trip upload request on every transmitted >>>>>>>>> core that needs investigating, seems like a less efficient >>>>>>>>> route...). >>>>>>>>> >>>>>>>>> Is there an opt-out?? It's conditional on UseSharedSpaces but >>>>>>>>> could there be a flag to disable, in case we see crashes with >>>>>>>>> gigabytes of private mappings that we really don't want to >>>>>>>>> retain (the user would have to know to set a flag, to disable >>>>>>>>> the new coredump filter ahead of time). >>>>>>>>> >>>>>>>>> Thanks! >>>>>>>>> Kevin >>>>>>>>> >>>>>>>>> >>>>>>>>> On 29/10/2018 06:02, Jini George wrote: >>>>>>>>>> Thank you very much, Ioi, for looking into this, and the >>>>>>>>>> clarification offline. My bad, I had missed the earlier mail >>>>>>>>>> from you. :-( My responses below. >>>>>>>>>> >>>>>>>>>> Yes, I had tested this on MacOS. The issue does not exist on >>>>>>>>>> MacOS since the file backed private mmap()-ed regions get >>>>>>>>>> dumped into the MacOS corefiles by default. >>>>>>>>>> >>>>>>>>>> The corefile sizes on Linux do increase due to this change. >>>>>>>>>> And the increase would also include any file mapped using NIO >>>>>>>>>> with MapMode.PRIVATE. The typical corefile size increase with >>>>>>>>>> this change would include the following components at a high >>>>>>>>>> level: >>>>>>>>>> >>>>>>>>>> * Any NIO file mapping with MapMode.PRIVATE. >>>>>>>>>> * Any file mmap()-ed by any native library with MAP_PRIVATE. >>>>>>>>>> * The read only CDS regions (ro and od): Of the order of a few >>>>>>>>>> MB. >>>>>>>>>> * The shared strings CDS region. (typically less than 1 MB). >>>>>>>>>> * 2 MB per native shared library (regions with ---p >>>>>>>>>> permissions mapped by the dynamic linker for better alignment >>>>>>>>>> and for keeping libraries efficiently shareable). >>>>>>>>>> * The JDK 'modules' file. (About 140 MB). >>>>>>>>>> >>>>>>>>>> So, without including any NIO mapping, I typically see around >>>>>>>>>> 250-300 MB increase in the corefile sizes. I agree that the >>>>>>>>>> size increase could be a cause for concern, but for FWIW, >>>>>>>>>> these privately mapped files get dumped into the corefile for >>>>>>>>>> MacOS too. And the corefile sizes for the same program on >>>>>>>>>> MacOS are way larger (of the order of a few GB as against >>>>>>>>>> about 300 MB on Linux (without the change)). >>>>>>>>>> >>>>>>>>>> The advantage of fixing this by modifying the coredump_filter >>>>>>>>>> v/s doing it in SA (by reading in more sections of the shared >>>>>>>>>> archive file) is that this would benefit other debuggers like >>>>>>>>>> gdb also. (And reduces the dependence on having the shared >>>>>>>>>> archive file being available at the time of debugging). If >>>>>>>>>> folks still think this is a cause for concern, I could make >>>>>>>>>> modifications to fix this by reading in the regions from the >>>>>>>>>> shared archive file in the SA code. I also wonder if it is >>>>>>>>>> worth taking a relook at the mapping types of the various CDS >>>>>>>>>> regions also. >>>>>>>>>> >>>>>>>>>> Thank you, >>>>>>>>>> Jini. >>>>>>>>>> >>>>>>>>>> On 10/22/2018 10:27 AM, Ioi Lam wrote: >>>>>>>>>>> Hi Jini, >>>>>>>>>>> >>>>>>>>>>> Did you see my earlier reply? I might have sent it out during >>>>>>>>>>> the mail server outage days :-( >>>>>>>>>>> >>>>>>>>>>> http://openjdk.5641.n7.nabble.com/RFR-S-JDK-8200613-SA-jstack-throws-UnmappedAddressException-with-a-CDS-core-file-td352681.html#a353026 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Here was my reply again: >>>>>>>>>>> >>>>>>>>>>>> Hi Jini, >>>>>>>>>>>> >>>>>>>>>>>> The changes looks good to me. >>>>>>>>>>>> >>>>>>>>>>>> Have you tested this on MacOS? CDS heap support is also >>>>>>>>>>>> enabled on >>>>>>>>>>>> MacOS. See macros.hpp: >>>>>>>>>>>> >>>>>>>>>>>> ???? #if INCLUDE_CDS && INCLUDE_G1GC && defined(_LP64) && >>>>>>>>>>>> !defined(_WINDOWS) >>>>>>>>>>>> ???? #define INCLUDE_CDS_JAVA_HEAP 1 >>>>>>>>>>>> >>>>>>>>>>>> Also, besides CDS, do we know how often other files will be >>>>>>>>>>>> mmaped with >>>>>>>>>>>> MAP_PRIVATE? Will users get huge core files because CDS is >>>>>>>>>>>> enabled? In >>>>>>>>>>>> JDK 12, CDS will be enabled by default (see JDK-8202951), so >>>>>>>>>>>> all users >>>>>>>>>>>> will be affected by the following line: >>>>>>>>>>>> >>>>>>>>>>>> ?? if (UseSharedSpaces) { >>>>>>>>>>>> ???? set_coredump_filter(FILE_BACKED_PVT_BIT); >>>>>>>>>>>> ?? } >>>>>>>>>>>> >>>>>>>>>>>> Maybe you can run an big app such as Eclipse, trigger a core >>>>>>>>>>>> dump, and >>>>>>>>>>>> compare the size of the core file before/after this change? >>>>>>>>>>>> >>>>>>>>>>>> Thanks >>>>>>>>>>>> >>>>>>>>>>>> - Ioi >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks >>>>>>>>>>> >>>>>>>>>>> - Ioi >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 10/21/18 8:58 PM, Jini George wrote: >>>>>>>>>>>> Gentle reminder! >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> - Jini >>>>>>>>>>>> >>>>>>>>>>>> On 10/9/2018 11:31 AM, Jini George wrote: >>>>>>>>>>>>> Hello! >>>>>>>>>>>>> >>>>>>>>>>>>> [Including runtime-dev since the changes are in runtime code] >>>>>>>>>>>>> >>>>>>>>>>>>> Requesting reviews for: >>>>>>>>>>>>> >>>>>>>>>>>>> Webrev: http://cr.openjdk.java.net/~jgeorge/8200613/webrev.01/ >>>>>>>>>>>>> BugID: https://bugs.openjdk.java.net/browse/JDK-8200613 >>>>>>>>>>>>> >>>>>>>>>>>>> Issue: jhsdb jstack would throw an UnmappedAddressException >>>>>>>>>>>>> with a core file generated from a CDS enabled java process. >>>>>>>>>>>>> This is seen only with Linux and with G1GC, while trying to >>>>>>>>>>>>> read in data from the shared strings region (the closed >>>>>>>>>>>>> archive heap space). This region (which is a file backed >>>>>>>>>>>>> private memory region) is not dumped into the corefile for >>>>>>>>>>>>> Linux. This, being a heap region (and therefore being a >>>>>>>>>>>>> read-write region) is also not read in from the classes.jsa >>>>>>>>>>>>> file in SA since only the read only regions are read in >>>>>>>>>>>>> while processing the core file. (The expectation being that >>>>>>>>>>>>> the read write regions are in the core file). >>>>>>>>>>>>> >>>>>>>>>>>>> Proposed solution: The proposed solution is to have the >>>>>>>>>>>>> coredump_filter value corresponding to the CDS process to >>>>>>>>>>>>> include bit 2 (file-backed private memory), so that the >>>>>>>>>>>>> file-backed private memory region also gets dumped into the >>>>>>>>>>>>> corefile. The proposed fix is in >>>>>>>>>>>>> src/hotspot/os/linux/os_linux.cpp. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Jini. >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>> >>>> >>> > From magnus.ihse.bursie at oracle.com Wed Dec 12 11:14:22 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 12 Dec 2018 12:14:22 +0100 Subject: [PATCH] Windows 32-bit DLL name decoration In-Reply-To: <2A193B76-87CF-4F16-89F1-7BA4257B13FF@oracle.com> References: <2DD922C6-EFB1-4891-90F0-3378DEA91B1C@gmail.com> <07840796-f1c4-e077-7b0b-cd5c1a880ec1@oracle.com> <00e58330-26f0-4401-f911-24fda0127bcc@oracle.com> <7083B6EB-588E-4296-9A03-D9CC9B688936@gmail.com> <813f4b3d-0b4a-1e28-845f-a6422a22c695@redhat.com> <530c2651-dc96-e04a-0b0e-d1f04d8e135a@oracle.com> <77cc6547-a145-e7b9-5f29-c66df76bc50a@redhat.com> <2A193B76-87CF-4F16-89F1-7BA4257B13FF@oracle.com> Message-ID: <67081751-6ee0-c4c1-26a9-f6432b27db9a@oracle.com> On 2018-12-11 18:22, Bob Vandette wrote: > Hotspot has had support for decorated and non-decorated names for the JNI_OnLoad > functions. Perhaps you should just follow the same procedure for the debug > library. > > #define JNI_ONLOAD_SYMBOLS {"_JNI_OnLoad at 8", "JNI_OnLoad"} > #define JNI_ONUNLOAD_SYMBOLS {"_JNI_OnUnload at 8", "JNI_OnUnload"} > #define JVM_ONLOAD_SYMBOLS {"_JVM_OnLoad at 12", "JVM_OnLoad"} > #define AGENT_ONLOAD_SYMBOLS {"_Agent_OnLoad at 12", "Agent_OnLoad"} > #define AGENT_ONUNLOAD_SYMBOLS {"_Agent_OnUnload at 4", "Agent_OnUnload"} > #define AGENT_ONATTACH_SYMBOLS {"_Agent_OnAttach at 12", ?Agent_OnAttach?} Yes, that sounds mostly reasonable. The latest iteration of the patch is doing just this. I searched the code for "jdwpTransport_On" to see of there was any corresponding OnUnload function (or other), but could not find any. But if there are other *_OnEvent() functions in the JDK code, they should be fixed too to handle decorated names. To the best of my knowledge, jdwpTransport_OnLoad was the only such *_OnEvent function that did not handle decorated names. I searched the code base for the pattern '[a-zA-Z]+_On[A-Z][a-zA-Z]*' and skimmed the result, but could not find anything else apart from those listed by you above, and the jdwpTransport_OnLoad. Any knowledge of additional such functions but with different names must come from the component teams. /Magnus > > Bob. > > >> On Dec 11, 2018, at 11:43 AM, Simon Tooke wrote: >> >> On 2018-12-11 10:05 a.m., Alexey Ivanov wrote: >>> Hi everyone, >>> >>> I came up with the following patch: >>> http://cr.openjdk.java.net/~aivanov/8214122/webrev.00/ >>> >>> It specifically addresses the problem in JDK-8214122 where on 32 bit >>> Windows jdwpTransport_OnLoad can exported with its plain and >>> __stdcall-mangled name. I used conditional compilation so that for >>> other platforms the code remains as it is now. >>> >>> jshell starts successfully with this fix; an IDE debugger works as well. >>> >> I am not a reviewer, but this patch only works for the specific case >> under discussion; the '@16' refers to the reserved stack size in the >> Pascal calling convention. So, the patch only works for 16 bytes of >> parameters. To be generic, the routine needs to have the stack size >> passed in by the caller. If a generic fix isn't desired (and I hope it >> is), I'd prefer to see the caller simply pass the decorated or >> undecorated name depending on the Win32/64 defines. >> >> #if defined(_WIN32) && !defined(_WIN64) onLoad = >> (jdwpTransport_OnLoad_t) dbgsysFindLibraryEntry(handle, >> "_jdwpTransport_OnLoad at 16"); #else onLoad = (jdwpTransport_OnLoad_t) >> dbgsysFindLibraryEntry(handle, "jdwpTransport_OnLoad"); #endif >> >> >> Thanks, >> -Simon >> >>> Regards, >>> Alexey >>> >>> https://bugs.openjdk.java.net/browse/JDK-8214122 >>> >>> On 10/12/2018 15:11, Magnus Ihse Bursie wrote: >>>>> Since removing JNICALL is not an option, there are only two options: >>>>> >>>>> 1. Add |/export| option to the Makefile or pragma-comment to the >>>>> source file; >>>>> 2. Lookup the decorated name |_jdwpTransport_OnLoad at 16| for Win32 >>>>> with fallback to undecorated one. >>>> Yes. >>>> >>>> I think the correct solution here is 2. From magnus.ihse.bursie at oracle.com Wed Dec 12 11:19:15 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 12 Dec 2018 12:19:15 +0100 Subject: [PATCH] Windows 32-bit DLL name decoration In-Reply-To: References: <2DD922C6-EFB1-4891-90F0-3378DEA91B1C@gmail.com> <07840796-f1c4-e077-7b0b-cd5c1a880ec1@oracle.com> <00e58330-26f0-4401-f911-24fda0127bcc@oracle.com> <7083B6EB-588E-4296-9A03-D9CC9B688936@gmail.com> <813f4b3d-0b4a-1e28-845f-a6422a22c695@redhat.com> <530c2651-dc96-e04a-0b0e-d1f04d8e135a@oracle.com> <77cc6547-a145-e7b9-5f29-c66df76bc50a@redhat.com> Message-ID: On 2018-12-11 18:16, Alexey Ivanov wrote: > Hi Simon, > > Thank you for your comments. > > The updated webrev: > http://cr.openjdk.java.net/~aivanov/8214122/webrev.01/ > > Indeed, it looks much cleaner. Yes! This seems like the correct fix. Ship it! :) /Magnus > > Regards, > Alexey > > On 11/12/2018 16:43, Simon Tooke wrote: >> On 2018-12-11 10:05 a.m., Alexey Ivanov wrote: >>> Hi everyone, >>> >>> I came up with the following patch: >>> http://cr.openjdk.java.net/~aivanov/8214122/webrev.00/ >>> >>> It specifically addresses the problem in JDK-8214122 where on 32 bit >>> Windows jdwpTransport_OnLoad can exported with its plain and >>> __stdcall-mangled name. I used conditional compilation so that for >>> other platforms the code remains as it is now. >>> >>> jshell starts successfully with this fix; an IDE debugger works as >>> well. >>> >> I am not a reviewer, but this patch only works for the specific case >> under discussion; the '@16' refers to the reserved stack size in the >> Pascal calling convention. So, the patch only works for 16 bytes of >> parameters. To be generic, the routine needs to have the stack size >> passed in by the caller. If a generic fix isn't desired (and I hope it >> is), I'd prefer to see the caller simply pass the decorated or >> undecorated name depending on the Win32/64 defines. >> >> #if defined(_WIN32) && !defined(_WIN64) onLoad = >> (jdwpTransport_OnLoad_t) dbgsysFindLibraryEntry(handle, >> "_jdwpTransport_OnLoad at 16"); #else onLoad = >> (jdwpTransport_OnLoad_t) >> dbgsysFindLibraryEntry(handle, "jdwpTransport_OnLoad"); #endif >> >> >> Thanks, >> -Simon >> >>> Regards, >>> Alexey >>> >>> https://bugs.openjdk.java.net/browse/JDK-8214122 >>> >>> On 10/12/2018 15:11, Magnus Ihse Bursie wrote: >>>>> Since removing JNICALL is not an option, there are only two options: >>>>> >>>>> 1. Add |/export| option to the Makefile or pragma-comment to the >>>>> source file; >>>>> 2. Lookup the decorated name |_jdwpTransport_OnLoad at 16| for Win32 >>>>> with fallback to undecorated one. >>>> Yes. >>>> >>>> I think the correct solution here is 2. > From Alan.Bateman at oracle.com Wed Dec 12 11:35:28 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 12 Dec 2018 11:35:28 +0000 Subject: [PATCH] Windows 32-bit DLL name decoration In-Reply-To: <67081751-6ee0-c4c1-26a9-f6432b27db9a@oracle.com> References: <2DD922C6-EFB1-4891-90F0-3378DEA91B1C@gmail.com> <07840796-f1c4-e077-7b0b-cd5c1a880ec1@oracle.com> <00e58330-26f0-4401-f911-24fda0127bcc@oracle.com> <7083B6EB-588E-4296-9A03-D9CC9B688936@gmail.com> <813f4b3d-0b4a-1e28-845f-a6422a22c695@redhat.com> <530c2651-dc96-e04a-0b0e-d1f04d8e135a@oracle.com> <77cc6547-a145-e7b9-5f29-c66df76bc50a@redhat.com> <2A193B76-87CF-4F16-89F1-7BA4257B13FF@oracle.com> <67081751-6ee0-c4c1-26a9-f6432b27db9a@oracle.com> Message-ID: <56caf2e5-aa21-de91-1c03-5e5076bc55e4@oracle.com> On 12/12/2018 11:14, Magnus Ihse Bursie wrote: > : > > I searched the code for "jdwpTransport_On" to see of there was any > corresponding OnUnload function (or other), but could not find any. That's right, an unload wasn't needed when that SPI was originally added but could be added in the event that some future debugger agent need it. The SPI is a supported/documented JDK-specific mechanism, its "spec" is hosted here: https://docs.oracle.com/en/java/javase/11/docs/specs/jdwp/jdwp-transport.html I see this thread is touching on the functions exported by libjli. This library is part of the launcher and shouldn't be used directly by anything outside of the JDK. However we have to be careful because JavaFX `javapackager` tool has been using it. There's a JEP to bring a similar tool, jpackage in the current proposal, that will supersede it but in the mean time we need to be careful not to break it. A second issue is that the libjli is listed in the property list (Info.plist) on macOS. This came from Apple in 7u4 and periodically things show up that are using it,? e.g. that recent breakage with Eclipse that was never fully diagnosed but we think it was using Info.plist. -Alan From christoph.langer at sap.com Wed Dec 12 11:58:08 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Wed, 12 Dec 2018 11:58:08 +0000 Subject: RFR (M) 8214892: Delayed starting of debugging via jcmd In-Reply-To: <4167d2a331044ab59c51a6e50c5167ac@sap.com> References: <01fdfc8e58a246c18d5909fc7df313b7@sap.com> <940d8a26-d5b3-0d5b-ccb8-a71d4d17efdc@oracle.com> <6283122f12f041caa56f768f8005920b@sap.com> <87976683317e41028e146fcc000e5638@sap.com> <4167d2a331044ab59c51a6e50c5167ac@sap.com> Message-ID: <595e6087b590411b930a32e8f0a0d3b0@sap.com> Hi, pushed: http://hg.openjdk.java.net/jdk/jdk/rev/21dfea980e23 Best regards Christoph > -----Original Message----- > From: Langer, Christoph > Sent: Dienstag, 11. Dezember 2018 11:28 > To: Schmelter, Ralf ; Chris Plummer > ; serviceability-dev at openjdk.java.net > Cc: Lindenmaier, Goetz > Subject: RE: RFR (M) 8214892: Delayed starting of debugging via jcmd > > Hi, > > @Ralf: Thanks for updating the patch. I'm good with it now ?? As there are no > other opinions about the naming of the jdwp agent property, I'm fine with > "onjcmd". It is the main use case. > > @all: With me and Chris as reviewers, I think we are good to push. I'll do this > tomorrow if I don't hear objections until then. > > Thanks and best regards > Christoph > > > -----Original Message----- > > From: Schmelter, Ralf > > Sent: Monday, December 10, 2018 2:55 PM > > To: Langer, Christoph ; Chris Plummer > > ; serviceability-dev at openjdk.java.net > > Subject: RE: RFR (M) 8214892: Delayed starting of debugging via jcmd > > > > Hi Christoph, > > > > > I have thought about a more generic name for the option rather than > > > "onjcmd". What about "standby=y". That would indicate that the > > > debugging agent is on standby and can be triggered by whatever > > > means. What do others think? > > > > I'm not sure making the name more generic is the right move, when it will > > probably be enabled via the jcmd in >95% of the cases. > > > > > I'd prefer if you write out either "Debugging has been started." > > > or "Debugging is already active.". I think this would make it more > > > clear for the user of the feature what actually happened. > > > > OK, I've changed that. > > > > > A better wording would be: > > > "Starts up the Java debugging if the jdwp agentlib was enabled with the > > option onjcmd=y." > > > > OK. > > > > > replace "debug triggered by jcmd?" with " start debug via jcmd" > > > > Ok. > > > > > Is it really necessary/desired to set suspendOnInit to false? We > > > could still honor suspend=y|n. The suspension will of course > > > happen at the time debug activation is triggered. > > > > I don't think it would make sense to support suspend=y. In makes sense > > when starting the debugging directly (so no, or at least not much) Java code > > can be executed and you can debug from the beginning. And it also makes > > sense for the onthrow/onuncaught, since you can to see the exceptions in > > the debugger. But when triggered from the outside, you have no natural > > event to wait for and no state to preserve. The only effect you would see is > > that the jcmd would not return, since the initialize method would be > blocked > > until a debugger has been connected. > > > > I've updated the webrev: > > > http://cr.openjdk.java.net/~rrich/webrevs/schmelter/8214892/webrev.03/ > > > > Best regards, > > Ralf Schmelter > > > > > > -----Original Message----- > > From: Langer, Christoph > > Sent: Freitag, 7. Dezember 2018 09:41 > > To: Schmelter, Ralf ; Chris Plummer > > ; serviceability-dev at openjdk.java.net > > Subject: RE: RFR (M) 8214892: Delayed starting of debugging via jcmd > > > > Hi Ralf, > > > > thanks for doing this change. Overall, it looks very good to me and seems to > > be a nice feature. > > > > I have thought about a more generic name for the option rather than > > "onjcmd". What about "standby=y". That would indicate that the > debugging > > agent is on standby and can be triggered by whatever means. What do > > others think? > > > > Here are some minor comments, mostly about the wording of text > > messages: > > > > src/hotspot/share/services/diagnosticCommand.cpp, line 1097: > > I'd prefer if you write out either "Debugging has been started." or > > "Debugging is already active.". I think this would make it more clear for the > > user of the feature what actually happened. > > > > src/hotspot/share/services/diagnosticCommand.hpp, line 878: > > "Starts up the Java debugging if enabled in the jdwp agentlib via the > > onjcmd=y option." > > A better wording would be: > > "Starts up the Java debugging if the jdwp agentlib was enabled with the > > option onjcmd=y." > > > > src/jdk.jdwp.agent/share/native/libjdwp/debugInit.c: > > line 876: > > replace "debug triggered by jcmd?" with " start debug via jcmd" (or "start > > debug on request" if we opt to change the option name from onjcmd to > > something more general) > > line 1300: > > suspendOnInit = JNI_FALSE; > > Is it really necessary/desired to set suspendOnInit to false? We could still > > honor suspend=y|n. The suspension will of course happen at the time > debug > > activation is triggered. > > > > Another question: Do we need a CSR for this? > > > > Best regards > > Christoph > > > > > -----Original Message----- > > > From: serviceability-dev > > On > > > Behalf Of Schmelter, Ralf > > > Sent: Donnerstag, 6. Dezember 2018 15:54 > > > To: Chris Plummer ; serviceability- > > > dev at openjdk.java.net > > > Subject: [CAUTION] RE: RFR (M) 8214892: Delayed starting of debugging > via > > > jcmd > > > > > > Hi Chris, > > > > > > > I think I prefer passing EI_VM_INIT for triggering_ei, but if you prefer > > > > to keep it as is, I suggest a comment to clarify why it might be out of > > > > range. > > > > > > Actually, I used EI_VM_INIT for the longest time and only changed it > > > recently, because I thought that code could assume that e.g. no classes > > have > > > been loaded yet when getting the INIT_VM event. But since the JVMTI > > spec > > > does not guarantees this in any way (it allows other events to be send > > before > > > a VM_INIT), I just will change it back to use EI_VM_INIT for the initialize > > call. > > > > > > Regarding the name of the option, I agree that onjcmd, while not > > technically > > > fully accurate, makes most sense for the common user. > > > > > > > ... It think you could just return the error right away and remove > > > > the error checking code that comes later. > > > > > > I've changed the code to just return the error directly. > > > > > > > It's not clear to me why you want the name and address of the first > > > > transport. Why is the first one necessarily the one you want? > > > > > > Since currently the bag of transports must always have a size of 1, getting > > the > > > first or the last transport is the same. But the callback function used to > > iterate > > > the bag has to return a boolean value. I've decided to return JNI_FALSE, > > > which would mean the iteration stops at the first entry. > > > > > > > > > I've updated the webrev with your and Goetz Lindenmeier's suggestions: > > > > > > http://cr.openjdk.java.net/~rrich/webrevs/schmelter/8214892/webrev.02/ > > > > > > Best regards, > > > Ralf From jini.george at oracle.com Wed Dec 12 12:15:54 2018 From: jini.george at oracle.com (Jini George) Date: Wed, 12 Dec 2018 17:45:54 +0530 Subject: RFR: JDK-8202884: SA: Attach/detach might fail on Linux if debugee application create/destroy threads during attaching In-Reply-To: References: Message-ID: Thank you very much for looking into this, JC! I have a revised webrev addressing your comments at: http://cr.openjdk.java.net/~jgeorge/8202884/webrev.02/index.html Requesting one more review for this. My comments inline: On 12/12/2018 2:53 AM, JC Beyler wrote: > Hi Jini, > > I saw a few nits: > http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/src/jdk.hotspot.agent/linux/native/libsaproc/libproc_impl.h.udiff.html > ? -> The comments are in the third person normally it seems so it would > become (I also removed the s from threads): > > +// deletes a thread from the thread list Done. > > http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/src/jdk.hotspot.agent/linux/native/libsaproc/libproc_impl.c.udiff.html > ? -> You added two empty lines it seems that could be removed Done. > > > http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/src/jdk.hotspot.agent/linux/native/libsaproc/ps_proc.c.udiff.html > ? -> Is there a real reason to have both enums? We could have a single > enum it seems and not lose too much You are right. I have done away with the WAITPID* enum. > ? -> you have a switch " > ? ? ? ?switch (errno) {" > ? ? ? ? -> Where really you could simplify the reading by moving the > EINTR case outside with its continue > ? ? ? ? -> The switch could then remain as it was (though you move > print_debug to print_error) > ? ? ? ? -> and just return in each cases I have changed this to: 206 } else { 207 switch (errno) { 208 case EINTR: 209 continue; 210 break; 211 case ECHILD: 212 print_debug("waitpid() failed. Child process pid (%d) does not exist \n", pid); 213 return ATTACH_THREAD_DEAD; 214 case EINVAL: 215 print_error("waitpid() failed. Invalid options argument.\n"); 216 return ATTACH_FAIL; 217 default: 218 print_error("waitpid() failed. Unexpected error %d\n", errno); 219 return ATTACH_FAIL; 220 } 221 } // else > > ? ?->?if (strncmp (buf, "State:", 6) == 0) { > ? ? ? -> You use sizeof("State:") right below; perhaps you could just > use "? const char const state[] = "State:";" and use sizeof(state) and > for the string, it seems less error prone > > ? -> A minor "bug" is here: > +? ? ? state = buf + sizeof ("State:"); > ? ? ? ? -> You did a strncmp above but that only assures the start of > the string is "State:", technically the character after the ':' is the > but it could only be that; sizeof("State:") is 7 and not 6. So you miss > one character when you are skipping spaces > ? ? ? ? -> It was probably ok because you always had at least one > space, ie "State: " Thanks! I have made some changes here to use a const char string and a variable to store the calculated length using strlen(). And I am using isspace() now to skip spaces since tabs could also be used as a delimiter. > ? -> Extra space here before the '(': "sizeof (buf)" Done. > > Finally your return sequence for that method could be simplified to: > > +? if (!found_state) { > +? ? print_error(" Could not find the State: string in the status file > for pid %d\n", pid); > +? } > +? fclose (fp); > +? return !found_state; I have modified this to: 257 if (!found_state) { 258 // Assuming the thread exists. 259 print_error("Could not find the 'State:' string in the /proc/%d/status file\n", pid); 260 } 261 fclose (fp); 262 return false; Thank you, Jini. > > Thanks! > Jc > > On Tue, Dec 11, 2018 at 9:30 AM Jini George > wrote: > > Hello ! > > Requesting reviews for: > > https://bugs.openjdk.java.net/browse/JDK-8202884 > Webrev: http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/index.html > > Details: > For attaching to the threads in a process, we first go ahead and do a > ptrace attach to the main thread. Later, we use the libthread_db > library > (or, in the case of being within a container, iterate through the > /proc//task files) to discover the threads of the process, and add > them to the threads list (within SA) for this process. Once, we have > discovered all the threads and added these to the list of threads, we > then invoke ptrace attach individually on all these threads to > attach to > these. When we deal with an application where the threads are exiting > continuously, some of these threads might not exist by the time we try > to ptrace attach to these threads. The proposed fix includes the > following modifications to solve this. > ? 1. Check the state of the threads in the thread_db callback routine, > and skip if the state of the thread is TD_THR_UNKNOWN or TD_THR_ZOMBIE. > SA does not try to ptrace attach to these threads and does not include > these threads in the threads list. > ? 2. While ptrace attaching to the thread, if ptrace(PTRACE_ATTACH, > ...) > fails with either ESCRH or EPERM, check the state of the thread by > checking if the /proc//status file corresponding to that thread > exists and if so, reading in the 'State:' line of that file. Skip > attaching to this thread and delete this thread from the SA list of > threads, if the thread is dead (State: X) or is a zombie (State: Z). > ?From the /proc man page, "Current state of the process. One of "R > (running)", "S (sleeping)", "D (disk sleep)", "T (stopped)", "T > (tracing > stop)", "Z (zombie)", or "X (dead)"." > ? 3. If waitpid() on the thread is a failure, again skip this thread > (delete this from SA's list of threads) instead of bailing out if the > thread has exited or terminated. > > Testing: > 1. Tested by attaching and detaching multiple times to a test program > spawning numerous short lived threads. > 2. The SA tests (under test/hotspot/jtreg/serviceability/sa) passed > with > 100 repeats on Mach5. > 3. No new failures and no occurrences of JDK-8202884 seen with testing > the SA tests (tiers 1 to 5) on Mach5. > > More details in the bug comments section. > > Thank you, > Jini. > > > > -- > > Thanks, > Jc From magnus.ihse.bursie at oracle.com Wed Dec 12 12:58:27 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 12 Dec 2018 13:58:27 +0100 Subject: [PATCH] Windows 32-bit DLL name decoration In-Reply-To: <56caf2e5-aa21-de91-1c03-5e5076bc55e4@oracle.com> References: <2DD922C6-EFB1-4891-90F0-3378DEA91B1C@gmail.com> <07840796-f1c4-e077-7b0b-cd5c1a880ec1@oracle.com> <00e58330-26f0-4401-f911-24fda0127bcc@oracle.com> <7083B6EB-588E-4296-9A03-D9CC9B688936@gmail.com> <813f4b3d-0b4a-1e28-845f-a6422a22c695@redhat.com> <530c2651-dc96-e04a-0b0e-d1f04d8e135a@oracle.com> <77cc6547-a145-e7b9-5f29-c66df76bc50a@redhat.com> <2A193B76-87CF-4F16-89F1-7BA4257B13FF@oracle.com> <67081751-6ee0-c4c1-26a9-f6432b27db9a@oracle.com> <56caf2e5-aa21-de91-1c03-5e5076bc55e4@oracle.com> Message-ID: On 2018-12-12 12:35, Alan Bateman wrote: > On 12/12/2018 11:14, Magnus Ihse Bursie wrote: >> : >> >> I searched the code for "jdwpTransport_On" to see of there was any >> corresponding OnUnload function (or other), but could not find any. > That's right, an unload wasn't needed when that SPI was originally > added but could be added in the event that some future debugger agent > need it. The SPI is a supported/documented JDK-specific mechanism, its > "spec" is hosted here: > > https://docs.oracle.com/en/java/javase/11/docs/specs/jdwp/jdwp-transport.html > ... yet in all that time, we have not fully supported the spec on Windows 32. :-( (We never discovered this because of lack of testing, I presume, and that our internal usage empoyed a questonable workaround.) > > I see this thread is touching on the functions exported by libjli. > This library is part of the launcher and shouldn't be used directly by > anything outside of the JDK. However we have to be careful because > JavaFX `javapackager` tool has been using it. There's a JEP to bring a > similar tool, jpackage in the current proposal, that will supersede it > but in the mean time we need to be careful not to break it. A second > issue is that the libjli is listed in the property list (Info.plist) > on macOS. This came from Apple in 7u4 and periodically things show up > that are using it, e.g. that recent breakage with Eclipse that was > never fully diagnosed but we think it was using Info.plist. The latest patch, as suggested, will not modify JLI, but instead fix the broken behaviour of JDWP on Windows 32. /Magnus From hohensee at amazon.com Wed Dec 12 14:07:35 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Wed, 12 Dec 2018 14:07:35 +0000 Subject: optimize KlassInfoTable size to power of 2 In-Reply-To: <436fddb22d3b4fb9aec315f3150cff41@jd.com> References: <140bac18565c48be88313f7cc1d24bf8@jd.com> <7fc37fd7cf24439eb3a6fff132fc7387@jd.com> <4ea1c92f-4e72-7d38-8c7a-91e2892e0c3b@redhat.com> <1ab2d28b40024ba6b299756612fa3760@jd.com> <7a1e5d87-94fd-9be9-8aab-5dd0f1a3fff9@redhat.com> <56b9099e33584796af8a508980ff4690@jd.com> <436fddb22d3b4fb9aec315f3150cff41@jd.com> Message-ID: <0045EBD6-F21D-457B-865E-3796BD86FFF9@amazon.com> Lgtm, JC. Thanks, Paul From: serviceability-dev on behalf of ?? Date: Tuesday, December 11, 2018 at 6:31 PM To: JC Beyler Cc: "serviceability-dev at openjdk.java.net" Subject: RE: optimize KlassInfoTable size to power of 2 Dear JC, That?s exactly modification I made ? Thanks a lot for making the patch! BRs, Lin From: JC Beyler [mailto:jcbeyler at google.com] Sent: Wednesday, December 12, 2018 2:27 AM To: ?? Cc: Andrew Haley ; serviceability-dev at openjdk.java.net Subject: Re: optimize KlassInfoTable size to power of 2 Hi all, I was playing with this yesterday because I was curious to see the different code generation patterns so I had the webrev ready to go: Webrev: http://cr.openjdk.java.net/~jcbeyler/8215228/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8215228 @Lin: check but I'd imagine this is what you had originally done but you also changed the size Thanks, Jc On Tue, Dec 11, 2018 at 3:44 AM ?? > wrote: Oh, to make it clear, when testing with 65536, I made the change to use hash & 65535 instead of hash % 65536. BRs, Lin -----Original Message----- From: ?? Sent: Tuesday, December 11, 2018 7:36 PM To: 'Andrew Haley' >; serviceability-dev at openjdk.java.net Subject: RE: optimize KlassInfoTable size to power of 2 >There is something that I do not understand. You said that GCC didn't >know that _num_buckets was constant. In that case, how did GCC know not to use a divide instruction when you tried 65536? The original code doesn't use _num_buckets directly when calculating the hash idx, it use a non-constant variable _size, a member of KlassInfoTable, which is assigned to _num_buckets. I guess it is the reason GCC didn't do the constant propagation in this case. And _size is not changed during the life of KlassInfoTable, it only set to zero at de-construction. In my experiment, I use _num_buckets directly instead of _size to calculate hash idx. So GCC provides the optimizations. I think your suggestion of using 65537 is valuable, And I agree that it is not easy to prove that non-prime for hash is better. So how about make the change to use constant _number_buckets directly for calculating hash idx, and leave the prime not changed. In my case, speed up JMap histo helps at avoid getting killed by a timer, especially when heap is large at ~200GB. Do you think it is reasonable to have a patch? Thanks, Lin -----Original Message----- From: Andrew Haley [mailto:aph at redhat.com] Sent: Tuesday, December 11, 2018 7:10 PM To: ?? >; serviceability-dev at openjdk.java.net Subject: Re: optimize KlassInfoTable size to power of 2 On 12/11/18 11:05 AM, Andrew Haley wrote: > unsigned mod_m(unsigned n) { > unsigned tmp = n % 65536; > tmp -= n / 65536; > if (tmp >= 65537) // overflow > tmp += 65537; > return tmp; > } NB: this assumes that unsigned int is uint64_t. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From sgehwolf at redhat.com Wed Dec 12 14:09:23 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Wed, 12 Dec 2018 15:09:23 +0100 Subject: [8u] RFR: 8059038: Create new launcher for SA tools Message-ID: Hi, Can I get a review of this small 8u enhancement, please? It adds two new launchers for the serviceability agent, one CLI version and one GUI version: $ /bin/clhsdb $ /bin/hsdb The enhancement request has been approved here: http://mail.openjdk.java.net/pipermail/jdk8u-dev/2018-December/008257.html During that discussion it has been suggested to use separate launchers for GUI and CLI. So this is the revised two-launcher-approach: webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8059038/02/ bug: https://bugs.openjdk.java.net/browse/JDK-8059038 Note: The initial version of this patch[1] had one launcher "jhsdb" with sub-commands "hsdb" and "clhsdb" delegating to the old launcher classes. The patch has two simple tests verifying that the launchers work. Thoughts? Thanks, Severin [1] http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8059038/01/ From alexey.ivanov at oracle.com Wed Dec 12 15:34:18 2018 From: alexey.ivanov at oracle.com (Alexey Ivanov) Date: Wed, 12 Dec 2018 15:34:18 +0000 Subject: [PATCH] Windows 32-bit DLL name decoration In-Reply-To: References: <2DD922C6-EFB1-4891-90F0-3378DEA91B1C@gmail.com> <07840796-f1c4-e077-7b0b-cd5c1a880ec1@oracle.com> <00e58330-26f0-4401-f911-24fda0127bcc@oracle.com> <7083B6EB-588E-4296-9A03-D9CC9B688936@gmail.com> <813f4b3d-0b4a-1e28-845f-a6422a22c695@redhat.com> <530c2651-dc96-e04a-0b0e-d1f04d8e135a@oracle.com> <77cc6547-a145-e7b9-5f29-c66df76bc50a@redhat.com> <2A193B76-87CF-4F16-89F1-7BA4257B13FF@oracle.com> <67081751-6ee0-c4c1-26a9-f6432b27db9a@oracle.com> <56caf2e5-aa21-de91-1c03-5e5076bc55e4@oracle.com> Message-ID: On 12/12/2018 12:58, Magnus Ihse Bursie wrote: > > > On 2018-12-12 12:35, Alan Bateman wrote: >> On 12/12/2018 11:14, Magnus Ihse Bursie wrote: >>> : >>> >>> I searched the code for "jdwpTransport_On" to see of there was any >>> corresponding OnUnload function (or other), but could not find any. >> That's right, an unload wasn't needed when that SPI was originally >> added but could be added in the event that some future debugger agent >> need it. The SPI is a supported/documented JDK-specific mechanism, >> its "spec" is hosted here: >> >> https://docs.oracle.com/en/java/javase/11/docs/specs/jdwp/jdwp-transport.html >> > ... yet in all that time, we have not fully supported the spec on > Windows 32. :-( (We never discovered this because of lack of testing, > I presume, and that our internal usage empoyed a questonable workaround.) The spec does not specify whether the mangled name should be exported or not (for Windows 32 bit). I looked through JNI specification and have found no mention of JNIEXPORT or JNICALL. https://docs.oracle.com/en/java/javase/11/docs/specs/jni/index.html This example uses extern "C" modifier but neither JNIEXPORT or JNICALL: https://docs.oracle.com/en/java/javase/11/docs/specs/jni/design.html#native-method-arguments > >> >> I see this thread is touching on the functions exported by libjli. >> This library is part of the launcher and shouldn't be used directly >> by anything outside of the JDK. However we have to be careful because >> JavaFX `javapackager` tool has been using it. There's a JEP to bring >> a similar tool, jpackage in the current proposal, that will supersede >> it but in the mean time we need to be careful not to break it. A >> second issue is that the libjli is listed in the property list >> (Info.plist) on macOS. This came from Apple in 7u4 and periodically >> things show up that are using it, e.g. that recent breakage with >> Eclipse that was never fully diagnosed but we think it was using >> Info.plist. > The latest patch, as suggested, will not modify JLI, but instead fix > the broken behaviour of JDWP on Windows 32. Yes, that's right. However, I believe the previous versions did not modify libjli. Regards, Alexey > > /Magnus From alexey.ivanov at oracle.com Wed Dec 12 15:43:38 2018 From: alexey.ivanov at oracle.com (Alexey Ivanov) Date: Wed, 12 Dec 2018 15:43:38 +0000 Subject: [PATCH] Windows 32-bit DLL name decoration In-Reply-To: <2A193B76-87CF-4F16-89F1-7BA4257B13FF@oracle.com> References: <2DD922C6-EFB1-4891-90F0-3378DEA91B1C@gmail.com> <07840796-f1c4-e077-7b0b-cd5c1a880ec1@oracle.com> <00e58330-26f0-4401-f911-24fda0127bcc@oracle.com> <7083B6EB-588E-4296-9A03-D9CC9B688936@gmail.com> <813f4b3d-0b4a-1e28-845f-a6422a22c695@redhat.com> <530c2651-dc96-e04a-0b0e-d1f04d8e135a@oracle.com> <77cc6547-a145-e7b9-5f29-c66df76bc50a@redhat.com> <2A193B76-87CF-4F16-89F1-7BA4257B13FF@oracle.com> Message-ID: Hi Bob, Thank you for the pointers on how the same situation is handled in Hotspot for *_OnLoad* functions. At this time, we're dealing with one symbol only. If other symbols are added in the future, we should definitely use this approach. Regards, Alexey On 11/12/2018 17:22, Bob Vandette wrote: > Hotspot has had support for decorated and non-decorated names for the JNI_OnLoad > functions. Perhaps you should just follow the same procedure for the debug > library. > > #define JNI_ONLOAD_SYMBOLS {"_JNI_OnLoad at 8", "JNI_OnLoad"} > #define JNI_ONUNLOAD_SYMBOLS {"_JNI_OnUnload at 8", "JNI_OnUnload"} > #define JVM_ONLOAD_SYMBOLS {"_JVM_OnLoad at 12", "JVM_OnLoad"} > #define AGENT_ONLOAD_SYMBOLS {"_Agent_OnLoad at 12", "Agent_OnLoad"} > #define AGENT_ONUNLOAD_SYMBOLS {"_Agent_OnUnload at 4", "Agent_OnUnload"} > #define AGENT_ONATTACH_SYMBOLS {"_Agent_OnAttach at 12", ?Agent_OnAttach?} > > Bob. > > >> On Dec 11, 2018, at 11:43 AM, Simon Tooke wrote: >> >> On 2018-12-11 10:05 a.m., Alexey Ivanov wrote: >>> Hi everyone, >>> >>> I came up with the following patch: >>> http://cr.openjdk.java.net/~aivanov/8214122/webrev.00/ >>> >>> It specifically addresses the problem in JDK-8214122 where on 32 bit >>> Windows jdwpTransport_OnLoad can exported with its plain and >>> __stdcall-mangled name. I used conditional compilation so that for >>> other platforms the code remains as it is now. >>> >>> jshell starts successfully with this fix; an IDE debugger works as well. >>> >> I am not a reviewer, but this patch only works for the specific case >> under discussion; the '@16' refers to the reserved stack size in the >> Pascal calling convention. So, the patch only works for 16 bytes of >> parameters. To be generic, the routine needs to have the stack size >> passed in by the caller. If a generic fix isn't desired (and I hope it >> is), I'd prefer to see the caller simply pass the decorated or >> undecorated name depending on the Win32/64 defines. >> >> #if defined(_WIN32) && !defined(_WIN64) onLoad = >> (jdwpTransport_OnLoad_t) dbgsysFindLibraryEntry(handle, >> "_jdwpTransport_OnLoad at 16"); #else onLoad = (jdwpTransport_OnLoad_t) >> dbgsysFindLibraryEntry(handle, "jdwpTransport_OnLoad"); #endif >> >> >> Thanks, >> -Simon >> >>> Regards, >>> Alexey >>> >>> https://bugs.openjdk.java.net/browse/JDK-8214122 >>> >>> On 10/12/2018 15:11, Magnus Ihse Bursie wrote: >>>>> Since removing JNICALL is not an option, there are only two options: >>>>> >>>>> 1. Add |/export| option to the Makefile or pragma-comment to the >>>>> source file; >>>>> 2. Lookup the decorated name |_jdwpTransport_OnLoad at 16| for Win32 >>>>> with fallback to undecorated one. >>>> Yes. >>>> >>>> I think the correct solution here is 2. From alexey.ivanov at oracle.com Wed Dec 12 16:02:33 2018 From: alexey.ivanov at oracle.com (Alexey Ivanov) Date: Wed, 12 Dec 2018 16:02:33 +0000 Subject: [PATCH] Windows 32-bit DLL name decoration In-Reply-To: References: <2DD922C6-EFB1-4891-90F0-3378DEA91B1C@gmail.com> <07840796-f1c4-e077-7b0b-cd5c1a880ec1@oracle.com> <00e58330-26f0-4401-f911-24fda0127bcc@oracle.com> <7083B6EB-588E-4296-9A03-D9CC9B688936@gmail.com> <813f4b3d-0b4a-1e28-845f-a6422a22c695@redhat.com> <530c2651-dc96-e04a-0b0e-d1f04d8e135a@oracle.com> <77cc6547-a145-e7b9-5f29-c66df76bc50a@redhat.com> Message-ID: Hi Simon, I'd like to clarify some points? Please see my comments inline: On 11/12/2018 17:16, Alexey Ivanov wrote: > Hi Simon, > > Thank you for your comments. > > The updated webrev: > http://cr.openjdk.java.net/~aivanov/8214122/webrev.01/ > > Indeed, it looks much cleaner. > > Regards, > Alexey > > On 11/12/2018 16:43, Simon Tooke wrote: >> On 2018-12-11 10:05 a.m., Alexey Ivanov wrote: >>> Hi everyone, >>> >>> I came up with the following patch: >>> http://cr.openjdk.java.net/~aivanov/8214122/webrev.00/ >>> >>> It specifically addresses the problem in JDK-8214122 where on 32 bit >>> Windows jdwpTransport_OnLoad can exported with its plain and >>> __stdcall-mangled name. I used conditional compilation so that for >>> other platforms the code remains as it is now. >>> >>> jshell starts successfully with this fix; an IDE debugger works as >>> well. >>> >> I am not a reviewer, but this patch only works for the specific case >> under discussion; the '@16' refers to the reserved stack size in the >> Pascal calling convention.? So, the patch only works for 16 bytes of >> parameters.? To be generic, the routine needs to have the stack size >> passed in by the caller.? If a generic fix isn't desired (and I hope it >> is), I'd prefer to see the caller simply pass the decorated or >> undecorated name depending on the Win32/64 defines. At this time, it's only one function. So the current approach works well enough, although it will not scale. If other similar functions are added in the future, _OnUnload for example, we should implement approach used in Hotspot as suggested by Bob: http://mail.openjdk.java.net/pipermail/build-dev/2018-December/024373.html >> >> >> ???? #if defined(_WIN32) && !defined(_WIN64) onLoad = >> ???? (jdwpTransport_OnLoad_t) dbgsysFindLibraryEntry(handle, >> ???? "_jdwpTransport_OnLoad at 16"); #else onLoad = >> (jdwpTransport_OnLoad_t) >> ???? dbgsysFindLibraryEntry(handle, "jdwpTransport_OnLoad"); #endif I believe falling back to undecorated name is the right thing in the case where the decorated name isn't found. Only the undecorated name has been looked up before. If a (third-party) JDWP transport DLL exports the undecorated name only, it will fail to load. With fallback in place, it will load successfully. Does it sound reasonable? Regards, Alexey >> >> >> Thanks, >> -Simon >> >>> Regards, >>> Alexey >>> >>> https://bugs.openjdk.java.net/browse/JDK-8214122 >>> >>> On 10/12/2018 15:11, Magnus Ihse Bursie wrote: >>>>> Since removing JNICALL is not an option, there are only two options: >>>>> >>>>> 1. Add |/export| option to the Makefile or pragma-comment to the >>>>> source file; >>>>> 2. Lookup the decorated name |_jdwpTransport_OnLoad at 16| for Win32 >>>>> with fallback to undecorated one. >>>> Yes. >>>> >>>> I think the correct solution here is 2. > From alexey.ivanov at oracle.com Wed Dec 12 16:10:57 2018 From: alexey.ivanov at oracle.com (Alexey Ivanov) Date: Wed, 12 Dec 2018 16:10:57 +0000 Subject: [PATCH] JDK-8214122 Prevent name mangling of jdwpTransport_OnLoad in Windows 32-bit DLL name decoration In-Reply-To: <4daf4b17-dc29-8814-24a9-9b08cda2b5dc@oracle.com> References: <2DD922C6-EFB1-4891-90F0-3378DEA91B1C@gmail.com> <07840796-f1c4-e077-7b0b-cd5c1a880ec1@oracle.com> <00e58330-26f0-4401-f911-24fda0127bcc@oracle.com> <34d2b504-c4eb-6df3-3bd1-e23b9c22ea6a@oracle.com> <01b80c3f-8f71-26ec-304e-a7b245924bd5@oracle.com> <17e7873f-f2f4-dc02-82a6-2d7eb1565ca2@oracle.com> <5e032661-b70c-c6ec-2700-120589300c9e@oracle.com> <5A54BA77-B240-472A-A4B7-535C3A30A20F@gmail.com> <9d060b4d-bc2f-d402-d767-e5962df45aa6@oracle.com> <4daf4b17-dc29-8814-24a9-9b08cda2b5dc@oracle.com> Message-ID: <6345d99a-de96-2659-b1e2-b9a137847b68@oracle.com> Ali has scanned the code base, he says, ?couldn?t find (hopefully tbh) other occurrences of such mismatches.? On 10/12/2018 10:45, Magnus Ihse Bursie wrote: > It might be worthwhile to scan the entire code base for JNICALL and > verify that we have no more mismatches. In general, JNICALL should be > preserved on all officially supported, exported interfaces. It need > not be present on interfaces used only internally, and my current > thinking is that it would be better if it were removed on all internal > interfaces. But at the very least, it should be consistently > specificed where exported and imported. (Any misses here is probably > due to duplicate declarations, instead of properly including header > files, so that's the correct way to resolve any problems found...) -- Regards, Alexey From alexey.ivanov at oracle.com Wed Dec 12 16:13:30 2018 From: alexey.ivanov at oracle.com (Alexey Ivanov) Date: Wed, 12 Dec 2018 16:13:30 +0000 Subject: [PATCH] JDK-8214122 Prevent name mangling of jdwpTransport_OnLoad in Windows 32-bit DLL name decoration In-Reply-To: <998A7FEB-9846-40F5-8880-1812302F002F@gmail.com> References: <2DD922C6-EFB1-4891-90F0-3378DEA91B1C@gmail.com> <07840796-f1c4-e077-7b0b-cd5c1a880ec1@oracle.com> <00e58330-26f0-4401-f911-24fda0127bcc@oracle.com> <34d2b504-c4eb-6df3-3bd1-e23b9c22ea6a@oracle.com> <01b80c3f-8f71-26ec-304e-a7b245924bd5@oracle.com> <17e7873f-f2f4-dc02-82a6-2d7eb1565ca2@oracle.com> <5e032661-b70c-c6ec-2700-120589300c9e@oracle.com> <5A54BA77-B240-472A-A4B7-535C3A30A20F@gmail.com> <9d060b4d-bc2f-d402-d767-e5962df45aa6@oracle.com> <998A7FEB-9846-40F5-8880-1812302F002F@gmail.com> Message-ID: <49872f2d-229e-6da9-6d35-2db161f3f589@oracle.com> Thank you, Ali, for doing this! On 10/12/2018 21:13, Ali ?nce wrote: > Hi Alexey, > > I?ve searched for |GetProcAddress| usages across source code and > couldn?t find (hopefully tbh) other occurrences of such mismatches. > > Regards, > > Ali > >> From alexey.ivanov at oracle.com Wed Dec 12 16:14:48 2018 From: alexey.ivanov at oracle.com (Alexey Ivanov) Date: Wed, 12 Dec 2018 16:14:48 +0000 Subject: [PATCH] JDK-8214122 Prevent name mangling of jdwpTransport_OnLoad in Windows 32-bit DLL name decoration In-Reply-To: <6345d99a-de96-2659-b1e2-b9a137847b68@oracle.com> References: <2DD922C6-EFB1-4891-90F0-3378DEA91B1C@gmail.com> <07840796-f1c4-e077-7b0b-cd5c1a880ec1@oracle.com> <00e58330-26f0-4401-f911-24fda0127bcc@oracle.com> <34d2b504-c4eb-6df3-3bd1-e23b9c22ea6a@oracle.com> <01b80c3f-8f71-26ec-304e-a7b245924bd5@oracle.com> <17e7873f-f2f4-dc02-82a6-2d7eb1565ca2@oracle.com> <5e032661-b70c-c6ec-2700-120589300c9e@oracle.com> <5A54BA77-B240-472A-A4B7-535C3A30A20F@gmail.com> <9d060b4d-bc2f-d402-d767-e5962df45aa6@oracle.com> <4daf4b17-dc29-8814-24a9-9b08cda2b5dc@oracle.com> <6345d99a-de96-2659-b1e2-b9a137847b68@oracle.com> Message-ID: <7189cb83-7a6e-4cf7-dc77-a91a18919e4e@oracle.com> Forgot to add the link: On 12/12/2018 16:10, Alexey Ivanov wrote: > Ali has scanned the code base, he says, ?couldn?t find (hopefully tbh) > other occurrences of such mismatches.? http://mail.openjdk.java.net/pipermail/build-dev/2018-December/024358.html > > On 10/12/2018 10:45, Magnus Ihse Bursie wrote: >> It might be worthwhile to scan the entire code base for JNICALL and >> verify that we have no more mismatches. In general, JNICALL should be >> preserved on all officially supported, exported interfaces. It need >> not be present on interfaces used only internally, and my current >> thinking is that it would be better if it were removed on all >> internal interfaces. But at the very least, it should be consistently >> specificed where exported and imported. (Any misses here is probably >> due to duplicate declarations, instead of properly including header >> files, so that's the correct way to resolve any problems found...) > -- Regards, Alexey From gnu.andrew at redhat.com Wed Dec 12 16:18:18 2018 From: gnu.andrew at redhat.com (Andrew Hughes) Date: Wed, 12 Dec 2018 16:18:18 +0000 Subject: [8u] RFR: 8059038: Create new launcher for SA tools In-Reply-To: References: Message-ID: On Wed, 12 Dec 2018 at 14:10, Severin Gehwolf wrote: > > Hi, > > Can I get a review of this small 8u enhancement, please? It adds two > new launchers for the serviceability agent, one CLI version and one GUI > version: > > $ /bin/clhsdb > $ /bin/hsdb > > The enhancement request has been approved here: > http://mail.openjdk.java.net/pipermail/jdk8u-dev/2018-December/008257.html > > During that discussion it has been suggested to use separate launchers > for GUI and CLI. So this is the revised two-launcher-approach: > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8059038/02/ > bug: https://bugs.openjdk.java.net/browse/JDK-8059038 > > Note: The initial version of this patch[1] had one launcher "jhsdb" > with sub-commands "hsdb" and "clhsdb" delegating to the old launcher > classes. > > The patch has two simple tests verifying that the launchers work. > Thoughts? > > Thanks, > Severin > > [1] http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8059038/01/ > The patch itself looks ok. But it looks completely different from the changesets in 9: https://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/bf17c0a1c746 https://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/24a8cbde76d8 Can you explain a little what's going on here? Thanks, -- Andrew :) Senior Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) Web Site: http://fuseyism.com Twitter: https://twitter.com/gnu_andrew_java PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 From alexey.ivanov at oracle.com Wed Dec 12 16:38:27 2018 From: alexey.ivanov at oracle.com (Alexey Ivanov) Date: Wed, 12 Dec 2018 16:38:27 +0000 Subject: [12] RFR for JDK-8214122: JDWP is broken on 32 bit Windows: transport library missing onLoad entry (was: [PATCH] Windows 32-bit DLL name decoration) In-Reply-To: References: <2DD922C6-EFB1-4891-90F0-3378DEA91B1C@gmail.com> <07840796-f1c4-e077-7b0b-cd5c1a880ec1@oracle.com> <00e58330-26f0-4401-f911-24fda0127bcc@oracle.com> <7083B6EB-588E-4296-9A03-D9CC9B688936@gmail.com> <813f4b3d-0b4a-1e28-845f-a6422a22c695@redhat.com> <530c2651-dc96-e04a-0b0e-d1f04d8e135a@oracle.com> <77cc6547-a145-e7b9-5f29-c66df76bc50a@redhat.com> Message-ID: Hi all, I have updated the summary of JDK-8214122 and the subject accordingly. Could you please review the following fix? https://bugs.openjdk.java.net/browse/JDK-8214122 webrev: http://cr.openjdk.java.net/~aivanov/8214122/webrev.01/ *Root cause* jdwpTransport_OnLoad is exported as _jdwpTransport_OnLoad at 16 on 32 bit Windows according to the name decorations [1] for __stdcall [2] calling conventions. *Fix* On 32 bit Windows, look up the decorated name, _jdwpTransport_OnLoad at 16, first; if not found, look up the undecorated name, jdwpTransport_OnLoad. Regards, Alexey [1] https://docs.microsoft.com/en-us/cpp/build/reference/decorated-names?view=vs-2017#FormatC [2] https://docs.microsoft.com/en-ie/cpp/cpp/stdcall?view=vs-2017 On 12/12/2018 11:19, Magnus Ihse Bursie wrote: > > > On 2018-12-11 18:16, Alexey Ivanov wrote: >> Hi Simon, >> >> Thank you for your comments. >> >> The updated webrev: >> http://cr.openjdk.java.net/~aivanov/8214122/webrev.01/ >> >> Indeed, it looks much cleaner. > Yes! This seems like the correct fix. > > Ship it! :) > > /Magnus > >> >> Regards, >> Alexey >> >> On 11/12/2018 16:43, Simon Tooke wrote: >>> On 2018-12-11 10:05 a.m., Alexey Ivanov wrote: >>>> Hi everyone, >>>> >>>> I came up with the following patch: >>>> http://cr.openjdk.java.net/~aivanov/8214122/webrev.00/ >>>> >>>> It specifically addresses the problem in JDK-8214122 where on 32 bit >>>> Windows jdwpTransport_OnLoad can exported with its plain and >>>> __stdcall-mangled name. I used conditional compilation so that for >>>> other platforms the code remains as it is now. >>>> >>>> jshell starts successfully with this fix; an IDE debugger works as >>>> well. >>>> >>> I am not a reviewer, but this patch only works for the specific case >>> under discussion; the '@16' refers to the reserved stack size in the >>> Pascal calling convention.? So, the patch only works for 16 bytes of >>> parameters.? To be generic, the routine needs to have the stack size >>> passed in by the caller.? If a generic fix isn't desired (and I hope it >>> is), I'd prefer to see the caller simply pass the decorated or >>> undecorated name depending on the Win32/64 defines. >>> >>> ???? #if defined(_WIN32) && !defined(_WIN64) onLoad = >>> ???? (jdwpTransport_OnLoad_t) dbgsysFindLibraryEntry(handle, >>> ???? "_jdwpTransport_OnLoad at 16"); #else onLoad = >>> (jdwpTransport_OnLoad_t) >>> ???? dbgsysFindLibraryEntry(handle, "jdwpTransport_OnLoad"); #endif >>> >>> >>> Thanks, >>> -Simon >>> >>>> Regards, >>>> Alexey >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8214122 >>>> >>>> On 10/12/2018 15:11, Magnus Ihse Bursie wrote: >>>>>> Since removing JNICALL is not an option, there are only two options: >>>>>> >>>>>> 1. Add |/export| option to the Makefile or pragma-comment to the >>>>>> source file; >>>>>> 2. Lookup the decorated name |_jdwpTransport_OnLoad at 16| for Win32 >>>>>> with fallback to undecorated one. >>>>> Yes. >>>>> >>>>> I think the correct solution here is 2. >> > From magnus.ihse.bursie at oracle.com Wed Dec 12 16:43:18 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 12 Dec 2018 17:43:18 +0100 Subject: [12] RFR for JDK-8214122: JDWP is broken on 32 bit Windows: transport library missing onLoad entry In-Reply-To: References: <2DD922C6-EFB1-4891-90F0-3378DEA91B1C@gmail.com> <07840796-f1c4-e077-7b0b-cd5c1a880ec1@oracle.com> <00e58330-26f0-4401-f911-24fda0127bcc@oracle.com> <7083B6EB-588E-4296-9A03-D9CC9B688936@gmail.com> <813f4b3d-0b4a-1e28-845f-a6422a22c695@redhat.com> <530c2651-dc96-e04a-0b0e-d1f04d8e135a@oracle.com> <77cc6547-a145-e7b9-5f29-c66df76bc50a@redhat.com> Message-ID: <4c9e06c6-b11d-483d-8411-bc8761fda150@oracle.com> On 2018-12-12 17:38, Alexey Ivanov wrote: > Hi all, > > I have updated the summary of JDK-8214122 and the subject accordingly. > > Could you please review the following fix? > > https://bugs.openjdk.java.net/browse/JDK-8214122 > webrev: http://cr.openjdk.java.net/~aivanov/8214122/webrev.01/ Looks good to me. /Magnus > > *Root cause* > jdwpTransport_OnLoad is exported as _jdwpTransport_OnLoad at 16 on 32 bit > Windows according to the name decorations [1] for __stdcall [2] > calling conventions. > > *Fix* > On 32 bit Windows, look up the decorated name, > _jdwpTransport_OnLoad at 16, first; if not found, look up the undecorated > name, jdwpTransport_OnLoad. > > > Regards, > Alexey > > [1] > https://docs.microsoft.com/en-us/cpp/build/reference/decorated-names?view=vs-2017#FormatC > [2] https://docs.microsoft.com/en-ie/cpp/cpp/stdcall?view=vs-2017 > > On 12/12/2018 11:19, Magnus Ihse Bursie wrote: >> >> >> On 2018-12-11 18:16, Alexey Ivanov wrote: >>> Hi Simon, >>> >>> Thank you for your comments. >>> >>> The updated webrev: >>> http://cr.openjdk.java.net/~aivanov/8214122/webrev.01/ >>> >>> Indeed, it looks much cleaner. >> Yes! This seems like the correct fix. >> >> Ship it! :) >> >> /Magnus >> >>> >>> Regards, >>> Alexey >>> >>> On 11/12/2018 16:43, Simon Tooke wrote: >>>> On 2018-12-11 10:05 a.m., Alexey Ivanov wrote: >>>>> Hi everyone, >>>>> >>>>> I came up with the following patch: >>>>> http://cr.openjdk.java.net/~aivanov/8214122/webrev.00/ >>>>> >>>>> It specifically addresses the problem in JDK-8214122 where on 32 bit >>>>> Windows jdwpTransport_OnLoad can exported with its plain and >>>>> __stdcall-mangled name. I used conditional compilation so that for >>>>> other platforms the code remains as it is now. >>>>> >>>>> jshell starts successfully with this fix; an IDE debugger works as >>>>> well. >>>>> >>>> I am not a reviewer, but this patch only works for the specific case >>>> under discussion; the '@16' refers to the reserved stack size in the >>>> Pascal calling convention. So, the patch only works for 16 bytes of >>>> parameters. To be generic, the routine needs to have the stack size >>>> passed in by the caller. If a generic fix isn't desired (and I >>>> hope it >>>> is), I'd prefer to see the caller simply pass the decorated or >>>> undecorated name depending on the Win32/64 defines. >>>> >>>> #if defined(_WIN32) && !defined(_WIN64) onLoad = >>>> (jdwpTransport_OnLoad_t) dbgsysFindLibraryEntry(handle, >>>> "_jdwpTransport_OnLoad at 16"); #else onLoad = >>>> (jdwpTransport_OnLoad_t) >>>> dbgsysFindLibraryEntry(handle, "jdwpTransport_OnLoad"); #endif >>>> >>>> >>>> Thanks, >>>> -Simon >>>> >>>>> Regards, >>>>> Alexey >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8214122 >>>>> >>>>> On 10/12/2018 15:11, Magnus Ihse Bursie wrote: >>>>>>> Since removing JNICALL is not an option, there are only two >>>>>>> options: >>>>>>> >>>>>>> 1. Add |/export| option to the Makefile or pragma-comment to the >>>>>>> source file; >>>>>>> 2. Lookup the decorated name |_jdwpTransport_OnLoad at 16| for Win32 >>>>>>> with fallback to undecorated one. >>>>>> Yes. >>>>>> >>>>>> I think the correct solution here is 2. >>> >> > From magnus.ihse.bursie at oracle.com Wed Dec 12 16:52:15 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 12 Dec 2018 17:52:15 +0100 Subject: [PATCH] Windows 32-bit DLL name decoration In-Reply-To: References: <2DD922C6-EFB1-4891-90F0-3378DEA91B1C@gmail.com> <00e58330-26f0-4401-f911-24fda0127bcc@oracle.com> <7083B6EB-588E-4296-9A03-D9CC9B688936@gmail.com> <813f4b3d-0b4a-1e28-845f-a6422a22c695@redhat.com> <530c2651-dc96-e04a-0b0e-d1f04d8e135a@oracle.com> <77cc6547-a145-e7b9-5f29-c66df76bc50a@redhat.com> <2A193B76-87CF-4F16-89F1-7BA4257B13FF@oracle.com> <67081751-6ee0-c4c1-26a9-f6432b27db9a@oracle.com> <56caf2e5-aa21-de91-1c03-5e5076bc55e4@oracle.com> Message-ID: <4b5b40b0-7206-1852-179e-3bb582b4f8c2@oracle.com> On 2018-12-12 16:34, Alexey Ivanov wrote: > > > On 12/12/2018 12:58, Magnus Ihse Bursie wrote: >> >> >> On 2018-12-12 12:35, Alan Bateman wrote: >>> On 12/12/2018 11:14, Magnus Ihse Bursie wrote: >>>> : >>>> >>>> I searched the code for "jdwpTransport_On" to see of there was any >>>> corresponding OnUnload function (or other), but could not find any. >>> That's right, an unload wasn't needed when that SPI was originally >>> added but could be added in the event that some future debugger >>> agent need it. The SPI is a supported/documented JDK-specific >>> mechanism, its "spec" is hosted here: >>> >>> https://docs.oracle.com/en/java/javase/11/docs/specs/jdwp/jdwp-transport.html >>> >> ... yet in all that time, we have not fully supported the spec on >> Windows 32. :-( (We never discovered this because of lack of testing, >> I presume, and that our internal usage empoyed a questonable >> workaround.) > > The spec does not specify whether the mangled name should be exported > or not (for Windows 32 bit). > > I looked through JNI specification and have found no mention of > JNIEXPORT or JNICALL. > https://docs.oracle.com/en/java/javase/11/docs/specs/jni/index.html > > This example uses extern "C" modifier but neither JNIEXPORT or JNICALL: > https://docs.oracle.com/en/java/javase/11/docs/specs/jni/design.html#native-method-arguments > I'm not sure if it's worth continuing this discussion, but at the very least the specs have been not very clear on this. The links you cite point to the JNI specification, not JDWP. I don't bother go looking for the exact place in the JNI spec, but I assume they state that you must implement the header file as generated by javac (formerly javah). These do look like this: * DO NOT EDIT THIS FILE - it is machine generated */ #include /* Header for class sun_nio_ch_FileKey */ #ifndef _Included_sun_nio_ch_FileKey #define _Included_sun_nio_ch_FileKey #ifdef __cplusplus extern "C" { #endif /* * Class: sun_nio_ch_FileKey * Method: init * Signature: (Ljava/io/FileDescriptor;)V */ JNIEXPORT void JNICALL Java_sun_nio_ch_FileKey_init (JNIEnv *, jobject, jobject); So at least implicitly, they say that you need to use JNIEXPORT and JNICALL. And there is no /export thingy here. And no instructions in the specs to do a specific export; if it were, we would have needed it for all the hundereds of JNI functions in the JDK. So obviously the JNI code reads decorated names on Windows 32. Then again, it is not clear that the spec for JNI should apply to JDWP. But, if they state that you must use the JNICALL, and this will, unless other actions are taken, such as using /export, by default create a decorated name, I think it's very clear that *if* this indeed was intended to be the formal interface, it *should* have been explicitly written in the spec. /Magnus > >> >>> >>> I see this thread is touching on the functions exported by libjli. >>> This library is part of the launcher and shouldn't be used directly >>> by anything outside of the JDK. However we have to be careful >>> because JavaFX `javapackager` tool has been using it. There's a JEP >>> to bring a similar tool, jpackage in the current proposal, that will >>> supersede it but in the mean time we need to be careful not to break >>> it. A second issue is that the libjli is listed in the property list >>> (Info.plist) on macOS. This came from Apple in 7u4 and periodically >>> things show up that are using it, e.g. that recent breakage with >>> Eclipse that was never fully diagnosed but we think it was using >>> Info.plist. >> The latest patch, as suggested, will not modify JLI, but instead fix >> the broken behaviour of JDWP on Windows 32. > > Yes, that's right. > However, I believe the previous versions did not modify libjli. > > Regards, > Alexey > >> >> /Magnus > From gary.adams at oracle.com Wed Dec 12 16:54:37 2018 From: gary.adams at oracle.com (Gary Adams) Date: Wed, 12 Dec 2018 11:54:37 -0500 Subject: RFR: JDK-8051349: nsk/jvmti/scenarios/sampling/SP06/sp06t003 fails in nightly Message-ID: <5C113D4D.7070503@oracle.com> After some testing with nsk verbose messages enabled, it is clear that this test is failing in checkThreads when the location did not match between the call to GetStackTrace and GetFrameLocation. For the tests that are run when the threads have not been suspended, there is no guarantee the locations will match. Issue: https://bugs.openjdk.java.net/browse/JDK-8051349 Webrev: http://cr.openjdk.java.net/~gadams/8051349/webrev.00/ From alexey.ivanov at oracle.com Wed Dec 12 17:01:23 2018 From: alexey.ivanov at oracle.com (Alexey Ivanov) Date: Wed, 12 Dec 2018 17:01:23 +0000 Subject: [PATCH] Windows 32-bit DLL name decoration In-Reply-To: <4b5b40b0-7206-1852-179e-3bb582b4f8c2@oracle.com> References: <2DD922C6-EFB1-4891-90F0-3378DEA91B1C@gmail.com> <00e58330-26f0-4401-f911-24fda0127bcc@oracle.com> <7083B6EB-588E-4296-9A03-D9CC9B688936@gmail.com> <813f4b3d-0b4a-1e28-845f-a6422a22c695@redhat.com> <530c2651-dc96-e04a-0b0e-d1f04d8e135a@oracle.com> <77cc6547-a145-e7b9-5f29-c66df76bc50a@redhat.com> <2A193B76-87CF-4F16-89F1-7BA4257B13FF@oracle.com> <67081751-6ee0-c4c1-26a9-f6432b27db9a@oracle.com> <56caf2e5-aa21-de91-1c03-5e5076bc55e4@oracle.com> <4b5b40b0-7206-1852-179e-3bb582b4f8c2@oracle.com> Message-ID: On 12/12/2018 16:52, Magnus Ihse Bursie wrote: > On 2018-12-12 16:34, Alexey Ivanov wrote: >> >> >> On 12/12/2018 12:58, Magnus Ihse Bursie wrote: >>> >>> >>> On 2018-12-12 12:35, Alan Bateman wrote: >>>> On 12/12/2018 11:14, Magnus Ihse Bursie wrote: >>>>> : >>>>> >>>>> I searched the code for "jdwpTransport_On" to see of there was any >>>>> corresponding OnUnload function (or other), but could not find any. >>>> That's right, an unload wasn't needed when that SPI was originally >>>> added but could be added in the event that some future debugger >>>> agent need it. The SPI is a supported/documented JDK-specific >>>> mechanism, its "spec" is hosted here: >>>> >>>> https://docs.oracle.com/en/java/javase/11/docs/specs/jdwp/jdwp-transport.html >>>> >>> ... yet in all that time, we have not fully supported the spec on >>> Windows 32. :-( (We never discovered this because of lack of >>> testing, I presume, and that our internal usage empoyed a >>> questonable workaround.) >> >> The spec does not specify whether the mangled name should be exported >> or not (for Windows 32 bit). >> >> I looked through JNI specification and have found no mention of >> JNIEXPORT or JNICALL. >> https://docs.oracle.com/en/java/javase/11/docs/specs/jni/index.html >> >> This example uses extern "C" modifier but neither JNIEXPORT or JNICALL: >> https://docs.oracle.com/en/java/javase/11/docs/specs/jni/design.html#native-method-arguments >> > I'm not sure if it's worth continuing this discussion, but at the very > least the specs have been not very clear on this. I absolutely agree it's not worth it. It does not solve the problem at hand; it does not solve any problem at all. So let's leave it here. -- Alexey > The links you cite point to the JNI specification, not JDWP. I don't > bother go looking for the exact place in the JNI spec, but I assume > they state that you must implement the header file as generated by > javac (formerly javah). These do look like this: > > * DO NOT EDIT THIS FILE - it is machine generated */ > #include > /* Header for class sun_nio_ch_FileKey */ > > #ifndef _Included_sun_nio_ch_FileKey > #define _Included_sun_nio_ch_FileKey > #ifdef __cplusplus > extern "C" { > #endif > /* > ?* Class:???? sun_nio_ch_FileKey > ?* Method:??? init > ?* Signature: (Ljava/io/FileDescriptor;)V > ?*/ > JNIEXPORT void JNICALL Java_sun_nio_ch_FileKey_init > ? (JNIEnv *, jobject, jobject); > > So at least implicitly, they say that you need to use JNIEXPORT and > JNICALL. And there is no /export thingy here. And no instructions in > the specs to do a specific export; if it were, we would have needed it > for all the hundereds of JNI functions in the JDK. So obviously the > JNI code reads decorated names on Windows 32. > > Then again, it is not clear that the spec for JNI should apply to > JDWP. But, if they state that you must use the JNICALL, and this will, > unless other actions are taken, such as using /export, by default > create a decorated name, I think it's very clear that *if* this indeed > was intended to be the formal interface, it *should* have been > explicitly written in the spec. > > /Magnus > > >> >>> >>>> >>>> I see this thread is touching on the functions exported by libjli. >>>> This library is part of the launcher and shouldn't be used directly >>>> by anything outside of the JDK. However we have to be careful >>>> because JavaFX `javapackager` tool has been using it. There's a JEP >>>> to bring a similar tool, jpackage in the current proposal, that >>>> will supersede it but in the mean time we need to be careful not to >>>> break it. A second issue is that the libjli is listed in the >>>> property list (Info.plist) on macOS. This came from Apple in 7u4 >>>> and periodically things show up that are using it, e.g. that recent >>>> breakage with Eclipse that was never fully diagnosed but we think >>>> it was using Info.plist. >>> The latest patch, as suggested, will not modify JLI, but instead fix >>> the broken behaviour of JDWP on Windows 32. >> >> Yes, that's right. >> However, I believe the previous versions did not modify libjli. >> >> Regards, >> Alexey >> >>> >>> /Magnus >> > From jcbeyler at google.com Wed Dec 12 17:08:33 2018 From: jcbeyler at google.com (JC Beyler) Date: Wed, 12 Dec 2018 09:08:33 -0800 Subject: Suggested improvement to X86Frame.getInterpreterFrameBCI In-Reply-To: References: <0f590762-2c34-0ceb-0937-be4ef39d30b2@oracle.com> <11798f90-4e4e-55e3-3226-35c26df0a202@oracle.com> Message-ID: Hi David, Thanks for the changes! Your latest patch looks good to me (I think a follow-up could/might be to go to a builder system for the frame because the various constructors are a bit confusing but that might be out of scope of this :-)), Jc On Wed, Dec 5, 2018 at 3:10 AM David Griffiths wrote: > Hi, thanks for reviewing. I have made the changes you suggested and also > tidied up the constructors a bit (there was already a 4x Address > constructor), hope that's ok. > > Cheers, > > David > > On Fri, 30 Nov 2018 at 17:06, JC Beyler wrote: > >> Hi both, >> >> The webrev looks good to me but I could see gains of just adding a new >> constructor instead of doing a new + set. >> >> LinuxAMD64JavaThreadPDAccess.java would become consistent with the rest >> of the code: >> + } else if (VM.getVM().getInterpreter().contains(guesser.getPC())) { >> + // pass the value of R13 which contains the bcp for the top level >> frame >> + return new X86Frame(guesser.getSP(), guesser.getFP(), >> guesser.getPC(), >> + context.getRegisterAsAddress(AMD64ThreadContext.R13)); >> } else { >> >> - And for X86Frame.java, that means there is no set method (there isn't a >> single one yet there so we are consistent there). >> - Finally, instead of just r13 internally to the Frame, we could just >> call it the bcp since that is what you are saying it is and then adapt the >> getInterpreterFrameBCI a bit because a bcp local variable is there :-) >> >> But these are nits :), >> Jc >> >> On Fri, Nov 30, 2018 at 6:21 AM Jini George >> wrote: >> >>> Your patch looks good to me, David. I can sponsor this for you if we get >>> one more review. >>> >>> Thanks, >>> Jini. >>> >>> On 11/22/2018 5:42 PM, David Griffiths wrote: >>> > Thanks Jini, please find patch for Java 9 attached (I don't have >>> author >>> > access to the bug itself). >>> > >>> > Cheers, >>> > >>> > David >>> > >>> > On Thu, 22 Nov 2018 at 09:02, Jini George >> > > wrote: >>> > >>> > Thank you very much for working on the fix for this issue, David. >>> It >>> > would be great if you can send in a complete patch for the review >>> (With >>> > a first cut look, there seems to be missing pieces). >>> > >>> > I have created a bug for this: >>> > >>> > https://bugs.openjdk.java.net/browse/JDK-8214226 >>> > >>> > Thank you, >>> > Jini >>> > >>> > On 11/22/2018 12:50 AM, David Griffiths wrote: >>> > > PS: should have added a new X86Frame constructor really, may >>> have >>> > just >>> > > been put off because there is already a four address >>> constructor so >>> > > would have had to add dummy argument or something. >>> > > >>> > > On Wed, 21 Nov 2018 at 19:15, David Griffiths >>> > >>> > > >> > >> wrote: >>> > > >>> > > Hi, thanks, apart from adding a setter for R13 in X86Frame, >>> the >>> > > other half of the fix is this: >>> > > >>> > > public Frame getCurrentFrameGuess(JavaThread thread, >>> > Address >>> > > addr) { >>> > > ThreadProxy t = getThreadProxy(addr); >>> > > AMD64ThreadContext context = (AMD64ThreadContext) >>> > t.getContext(); >>> > > AMD64CurrentFrameGuess guesser = new >>> > > AMD64CurrentFrameGuess(context, thread); >>> > > if (!guesser.run(GUESS_SCAN_RANGE)) { >>> > > return null; >>> > > } >>> > > if (guesser.getPC() == null) { >>> > > return new X86Frame(guesser.getSP(), >>> guesser.getFP()); >>> > > } else if >>> > (VM.getVM().getInterpreter().contains(guesser.getPC())) { >>> > > // pass the value of R13 which contains the bcp for >>> > the top >>> > > level frame >>> > > Address r13 = >>> > > context.getRegisterAsAddress(AMD64ThreadContext.R13); >>> > > X86Frame frame = new X86Frame(guesser.getSP(), >>> > > guesser.getFP(), guesser.getPC()); >>> > > frame.setR13(r13); >>> > > return frame; >>> > > } else { >>> > > return new X86Frame(guesser.getSP(), guesser.getFP(), >>> > > guesser.getPC()); >>> > > } >>> > > } >>> > > >>> > > (the whole "if pc in interpreter" block is new) >>> > > >>> > > Overhead likely to be low as this is only used in diagnostic >>> > code. >>> > > Can't think of any risk but I'm not an expert on this code. >>> > > >>> > > Cheers, >>> > > >>> > > David >>> > > >>> > > On Wed, 21 Nov 2018 at 19:01, JC Beyler < >>> jcbeyler at google.com >>> > >>> > > >> >>> wrote: >>> > > >>> > > Hi David, >>> > > >>> > > I think the easiest would be to see whole change to >>> > understand >>> > > the repercussions of the change. I would imagine that >>> any >>> > change >>> > > that helps stacktraces being more precise is a good >>> thing but >>> > > there are questions that arise every time: >>> > > - What is the overhead of adding this? >>> > > - Does this add any risk of failure? >>> > > >>> > > I'd imagine that the change is relatively small and >>> should be >>> > > easy to assess this but seeing it would make things >>> easier to >>> > > determine. >>> > > >>> > > That being said, I'm not a reviewer for OpenJDK so this >>> is >>> > > really just my 2cents, >>> > > Jc >>> > > >>> > > On Wed, Nov 21, 2018 at 9:17 AM David Griffiths >>> > > >> > >> david.griffiths at gmail.com >>> > >> >>> > > wrote: >>> > > >>> > > Hi, I'm new to this mailing list and working on a >>> project >>> > > that makes use of the SA classes to get stack traces >>> > from a >>> > > paused in flight JVM (we can't use JDWP). I have >>> observed >>> > > that if the top frame is in the interpreter it >>> > reports the >>> > > BCI and line number incorrectly. This is because >>> > > X86Frame.getInterpreterFrameBCI uses the value >>> stored >>> > on the >>> > > stack rather than the actual live value stored in >>> R13. >>> > > >>> > > I have a patch for this which lets >>> > > LinuxAMD64JavaThreadPDAccess.getCurrentFrameGuess >>> > pass the >>> > > R13 value to X86Frame so that the latter can then >>> do: >>> > > >>> > > public int getInterpreterFrameBCI() { >>> > > Address bcp = >>> > > addressOfInterpreterFrameBCX().getAddressAt(0); >>> > > // If we are in the top level frame then R13 >>> may >>> > have >>> > > been set for us which contains >>> > > // the BCP. If so then let it take priority. If >>> > we are >>> > > in a top level interpreter frame, >>> > > // the BCP is live in R13 (on x86) and not >>> saved >>> > in the >>> > > BCX stack slot. >>> > > if (r13 != null) { >>> > > bcp = r13; >>> > > } >>> > > Address methodHandle = >>> > > addressOfInterpreterFrameMethod().getAddressAt(0); >>> > > >>> > > and this fixes the problem. >>> > > >>> > > Does this sound like a good idea and if so should I >>> > submit a >>> > > patch? >>> > > >>> > > Cheers, >>> > > >>> > > David >>> > > >>> > > >>> > > >>> > > -- >>> > > >>> > > Thanks, >>> > > Jc >>> > > >>> > >>> >> >> >> -- >> >> Thanks, >> Jc >> > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From jcbeyler at google.com Wed Dec 12 17:17:51 2018 From: jcbeyler at google.com (JC Beyler) Date: Wed, 12 Dec 2018 09:17:51 -0800 Subject: RFR: JDK-8202884: SA: Attach/detach might fail on Linux if debugee application create/destroy threads during attaching In-Reply-To: References: Message-ID: Hi Jini, Should your return not be return !found_state instead of false here: 257 if (!found_state) { 258 // Assuming the thread exists. 259 print_error("Could not find the 'State:' string in the /proc/%d/status file\n", pid); 260 } 261 fclose (fp); 262 return false; In your webrev.00 it was the case but now, you always return the process does exist even if you have not found it. cf: http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/src/jdk.hotspot.agent/linux/native/libsaproc/ps_proc.c.udiff.html vs http://cr.openjdk.java.net/~jgeorge/8202884/webrev.02/src/jdk.hotspot.agent/linux/native/libsaproc/ps_proc.c.udiff.html Tiny nit: no need to check *space if you are using isspace(*space) right after :) Apart from the return question, the webrev looks good to me :-) Jc On Wed, Dec 12, 2018 at 4:15 AM Jini George wrote: > Thank you very much for looking into this, JC! > > I have a revised webrev addressing your comments at: > > http://cr.openjdk.java.net/~jgeorge/8202884/webrev.02/index.html > > Requesting one more review for this. My comments inline: > > On 12/12/2018 2:53 AM, JC Beyler wrote: > > Hi Jini, > > > > I saw a few nits: > > > http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/src/jdk.hotspot.agent/linux/native/libsaproc/libproc_impl.h.udiff.html > > -> The comments are in the third person normally it seems so it would > > become (I also removed the s from threads): > > > > +// deletes a thread from the thread list > Done. > > > > > http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/src/jdk.hotspot.agent/linux/native/libsaproc/libproc_impl.c.udiff.html > > -> You added two empty lines it seems that could be removed > Done. > > > > > > > http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/src/jdk.hotspot.agent/linux/native/libsaproc/ps_proc.c.udiff.html > > -> Is there a real reason to have both enums? We could have a single > > enum it seems and not lose too much > > You are right. I have done away with the WAITPID* enum. > > > -> you have a switch " > > switch (errno) {" > > -> Where really you could simplify the reading by moving the > > EINTR case outside with its continue > > -> The switch could then remain as it was (though you move > > print_debug to print_error) > > -> and just return in each cases > I have changed this to: > > 206 } else { > 207 switch (errno) { > 208 case EINTR: > 209 continue; > 210 break; > 211 case ECHILD: > 212 print_debug("waitpid() failed. Child process pid (%d) does > not exist \n", pid); > 213 return ATTACH_THREAD_DEAD; > 214 case EINVAL: > 215 print_error("waitpid() failed. Invalid options argument.\n"); > 216 return ATTACH_FAIL; > 217 default: > 218 print_error("waitpid() failed. Unexpected error %d\n", > errno); > 219 return ATTACH_FAIL; > 220 } > 221 } // else > > > > > > -> if (strncmp (buf, "State:", 6) == 0) { > > -> You use sizeof("State:") right below; perhaps you could just > > use " const char const state[] = "State:";" and use sizeof(state) and > > for the string, it seems less error prone > > > > -> A minor "bug" is here: > > + state = buf + sizeof ("State:"); > > -> You did a strncmp above but that only assures the start of > > the string is "State:", technically the character after the ':' is the > > but it could only be that; sizeof("State:") is 7 and not 6. So you miss > > one character when you are skipping spaces > > -> It was probably ok because you always had at least one > > space, ie "State: " > > Thanks! I have made some changes here to use a const char string and a > variable to store the calculated length using strlen(). And I am using > isspace() now to skip spaces since tabs could also be used as a delimiter. > > > -> Extra space here before the '(': "sizeof (buf)" > Done. > > > > Finally your return sequence for that method could be simplified to: > > > > + if (!found_state) { > > + print_error(" Could not find the State: string in the status file > > for pid %d\n", pid); > > + } > > + fclose (fp); > > + return !found_state; > > I have modified this to: > > 257 if (!found_state) { > 258 // Assuming the thread exists. > 259 print_error("Could not find the 'State:' string in the > /proc/%d/status file\n", pid); > 260 } > 261 fclose (fp); > 262 return false; > > Thank you, > Jini. > > > > > Thanks! > > Jc > > > > On Tue, Dec 11, 2018 at 9:30 AM Jini George > > wrote: > > > > Hello ! > > > > Requesting reviews for: > > > > https://bugs.openjdk.java.net/browse/JDK-8202884 > > Webrev: > http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/index.html > > > > Details: > > For attaching to the threads in a process, we first go ahead and do a > > ptrace attach to the main thread. Later, we use the libthread_db > > library > > (or, in the case of being within a container, iterate through the > > /proc//task files) to discover the threads of the process, and > add > > them to the threads list (within SA) for this process. Once, we have > > discovered all the threads and added these to the list of threads, we > > then invoke ptrace attach individually on all these threads to > > attach to > > these. When we deal with an application where the threads are exiting > > continuously, some of these threads might not exist by the time we > try > > to ptrace attach to these threads. The proposed fix includes the > > following modifications to solve this. > > 1. Check the state of the threads in the thread_db callback > routine, > > and skip if the state of the thread is TD_THR_UNKNOWN or > TD_THR_ZOMBIE. > > SA does not try to ptrace attach to these threads and does not > include > > these threads in the threads list. > > 2. While ptrace attaching to the thread, if ptrace(PTRACE_ATTACH, > > ...) > > fails with either ESCRH or EPERM, check the state of the thread by > > checking if the /proc//status file corresponding to that thread > > exists and if so, reading in the 'State:' line of that file. Skip > > attaching to this thread and delete this thread from the SA list of > > threads, if the thread is dead (State: X) or is a zombie (State: Z). > > From the /proc man page, "Current state of the process. One of "R > > (running)", "S (sleeping)", "D (disk sleep)", "T (stopped)", "T > > (tracing > > stop)", "Z (zombie)", or "X (dead)"." > > 3. If waitpid() on the thread is a failure, again skip this thread > > (delete this from SA's list of threads) instead of bailing out if the > > thread has exited or terminated. > > > > Testing: > > 1. Tested by attaching and detaching multiple times to a test program > > spawning numerous short lived threads. > > 2. The SA tests (under test/hotspot/jtreg/serviceability/sa) passed > > with > > 100 repeats on Mach5. > > 3. No new failures and no occurrences of JDK-8202884 seen with > testing > > the SA tests (tiers 1 to 5) on Mach5. > > > > More details in the bug comments section. > > > > Thank you, > > Jini. > > > > > > > > -- > > > > Thanks, > > Jc > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From jcbeyler at google.com Wed Dec 12 17:25:55 2018 From: jcbeyler at google.com (JC Beyler) Date: Wed, 12 Dec 2018 09:25:55 -0800 Subject: RFR: JDK-8051349: nsk/jvmti/scenarios/sampling/SP06/sp06t003 fails in nightly In-Reply-To: <5C113D4D.7070503@oracle.com> References: <5C113D4D.7070503@oracle.com> Message-ID: Hi Gary, Looks good to me, especially since the SP02t003 does the same thing. Anyway we could do the same code as that test so that one day when we want to merge them with a common method, it will be easier to see that that specific code is identical (or really close from what I can tell): if ((suspended == NSK_TRUE) && (frameStack[j].location != qLocation)) { Anyhow, it looks good to me. Thanks, Jc On Wed, Dec 12, 2018 at 8:55 AM Gary Adams wrote: > After some testing with nsk verbose messages enabled, > it is clear that this test is failing in checkThreads when the > location did not match between the call to GetStackTrace > and GetFrameLocation. For the tests that are run when the threads > have not been suspended, there is no guarantee the locations > will match. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8051349 > Webrev: http://cr.openjdk.java.net/~gadams/8051349/webrev.00/ > > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From sgehwolf at redhat.com Wed Dec 12 17:28:24 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Wed, 12 Dec 2018 18:28:24 +0100 Subject: [8u] RFR: 8059038: Create new launcher for SA tools In-Reply-To: References: Message-ID: <975dca9cb3fdea8f2a799c52f7d3e5edd9d79306.camel@redhat.com> On Wed, 2018-12-12 at 16:18 +0000, Andrew Hughes wrote: > On Wed, 12 Dec 2018 at 14:10, Severin Gehwolf wrote: > > > > Hi, > > > > Can I get a review of this small 8u enhancement, please? It adds two > > new launchers for the serviceability agent, one CLI version and one GUI > > version: > > > > $ /bin/clhsdb > > $ /bin/hsdb > > > > The enhancement request has been approved here: > > http://mail.openjdk.java.net/pipermail/jdk8u-dev/2018-December/008257.html > > > > During that discussion it has been suggested to use separate launchers > > for GUI and CLI. So this is the revised two-launcher-approach: > > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8059038/02/ > > bug: https://bugs.openjdk.java.net/browse/JDK-8059038 > > > > Note: The initial version of this patch[1] had one launcher "jhsdb" > > with sub-commands "hsdb" and "clhsdb" delegating to the old launcher > > classes. > > > > The patch has two simple tests verifying that the launchers work. > > Thoughts? > > > > Thanks, > > Severin > > > > [1] http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8059038/01/ > > > > The patch itself looks ok. But it looks completely different from the > changesets in 9: > > https://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/bf17c0a1c746 > https://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/24a8cbde76d8 > > Can you explain a little what's going on here? Sure. The original patch for 9 added a new class, SALauncher, which launches many of the original stand-alone commands from 8. jstack from JDK 8, for example, became "jhsdb jstack" in 9+ via this class[I]. On the JDK 8 enhancement request approval thread a suggestion was made for the backport to JDK 8 to mimic the stand-alone launcher approach JDK 8 uses: "jstack" over "jhsdb jstack". So "clhsdb" (JDK 8) will be a separate launcher over "jhsdb clhsdb" in JDK 9+. The first webrev[1] above, mimic'ed JDK 9+, but not quite since it didn't change the arguments and options. jhsdb clhsdb --pid (JDK 9+) vs. jhsdb clhsdb (in [1]). That's basically the reason why the JDK 8 backport looks this different. Of course we can also go with the original version[1], but so far I've heard two +1's for the second version... Hence, I'm proposing that version. Thanks, Severin [I] https://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/24a8cbde76d8#l2.213 From daniil.x.titov at oracle.com Wed Dec 12 17:31:30 2018 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Wed, 12 Dec 2018 09:31:30 -0800 Subject: JDK-8051349: nsk/jvmti/scenarios/sampling/SP06/sp06t003 fails in nightly In-Reply-To: <59728937-0AD4-48AD-A2E7-0F96A6C11309@oracle.com> References: <59728937-0AD4-48AD-A2E7-0F96A6C11309@oracle.com> Message-ID: Hi Gary, The fix looks good to me. Thanks, Daniil ?On 12/12/18, 8:55 AM, "serviceability-dev on behalf of Gary Adams" wrote: After some testing with nsk verbose messages enabled, it is clear that this test is failing in checkThreads when the location did not match between the call to GetStackTrace and GetFrameLocation. For the tests that are run when the threads have not been suspended, there is no guarantee the locations will match. Issue: https://bugs.openjdk.java.net/browse/JDK-8051349 Webrev: http://cr.openjdk.java.net/~gadams/8051349/webrev.00/ From jini.george at oracle.com Wed Dec 12 18:27:39 2018 From: jini.george at oracle.com (Jini George) Date: Wed, 12 Dec 2018 23:57:39 +0530 Subject: RFR: JDK-8202884: SA: Attach/detach might fail on Linux if debugee application create/destroy threads during attaching In-Reply-To: References: Message-ID: On 12/12/2018 10:47 PM, JC Beyler wrote: > Hi Jini, > > Should your return not be return !found_state instead of false here: > > 257? ?if (!found_state) { > 258? ? ?// Assuming the thread exists. > 259? ? ?print_error("Could not find the 'State:' string in the > /proc/%d/status file\n", pid); > 260? ?} > 261? ?fclose (fp); > 262? ?return false; > > In your webrev.00 it was the case but now, you always return the process > does exist even if you have not found it. I referred to the gdb sources to check what is done under this scenario, and in gdb, it is assumed that if the line beginning with 'State:' is not found, the thread is alive. But to be frank, I don't know under what circumstances will we ever encounter such a scenario. Let me know if you don't agree with this. > cf: > http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/src/jdk.hotspot.agent/linux/native/libsaproc/ps_proc.c.udiff.html > vs > http://cr.openjdk.java.net/~jgeorge/8202884/webrev.02/src/jdk.hotspot.agent/linux/native/libsaproc/ps_proc.c.udiff.html > > Tiny nit: no need to check *space if you are using isspace(*space) right > after :) Will change this. Thanks! > > Apart from the return question, the webrev looks good to me :-) > Jc Thank you again! Jini > > > On Wed, Dec 12, 2018 at 4:15 AM Jini George > wrote: > > Thank you very much for looking into this, JC! > > I have a revised webrev addressing your comments at: > > http://cr.openjdk.java.net/~jgeorge/8202884/webrev.02/index.html > > Requesting one more review for this. My comments inline: > > On 12/12/2018 2:53 AM, JC Beyler wrote: > > Hi Jini, > > > > I saw a few nits: > > > http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/src/jdk.hotspot.agent/linux/native/libsaproc/libproc_impl.h.udiff.html > >? ? -> The comments are in the third person normally it seems so > it would > > become (I also removed the s from threads): > > > > +// deletes a thread from the thread list > Done. > > > > > http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/src/jdk.hotspot.agent/linux/native/libsaproc/libproc_impl.c.udiff.html > >? ? -> You added two empty lines it seems that could be removed > Done. > > > > > > > http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/src/jdk.hotspot.agent/linux/native/libsaproc/ps_proc.c.udiff.html > >? ? -> Is there a real reason to have both enums? We could have a > single > > enum it seems and not lose too much > > You are right. I have done away with the WAITPID* enum. > > >? ? -> you have a switch " > >? ? ? ? ?switch (errno) {" > >? ? ? ? ? -> Where really you could simplify the reading by moving > the > > EINTR case outside with its continue > >? ? ? ? ? -> The switch could then remain as it was (though you move > > print_debug to print_error) > >? ? ? ? ? -> and just return in each cases > I have changed this to: > > 206? ? ?} else { > 207? ? ? ?switch (errno) { > 208? ? ? ? ?case EINTR: > 209? ? ? ? ? ?continue; > 210? ? ? ? ? ?break; > 211? ? ? ? ?case ECHILD: > 212? ? ? ? ? ?print_debug("waitpid() failed. Child process pid (%d) > does > not exist \n", pid); > 213? ? ? ? ? ?return ATTACH_THREAD_DEAD; > 214? ? ? ? ?case EINVAL: > 215? ? ? ? ? ?print_error("waitpid() failed. Invalid options > argument.\n"); > 216? ? ? ? ? ?return ATTACH_FAIL; > 217? ? ? ? ?default: > 218? ? ? ? ? ?print_error("waitpid() failed. Unexpected error %d\n", > errno); > 219? ? ? ? ? ?return ATTACH_FAIL; > 220? ? ? ?} > 221? ? ?} // else > > > > > >? ? ?->?if (strncmp (buf, "State:", 6) == 0) { > >? ? ? ? -> You use sizeof("State:") right below; perhaps you could > just > > use "? const char const state[] = "State:";" and use > sizeof(state) and > > for the string, it seems less error prone > > > >? ? -> A minor "bug" is here: > > +? ? ? state = buf + sizeof ("State:"); > >? ? ? ? ? -> You did a strncmp above but that only assures the > start of > > the string is "State:", technically the character after the ':' > is the > > but it could only be that; sizeof("State:") is 7 and not 6. So > you miss > > one character when you are skipping spaces > >? ? ? ? ? -> It was probably ok because you always had at least one > > space, ie "State: " > > Thanks! I have made some changes here to use a const char string and a > variable to store the calculated length using strlen(). And I am using > isspace() now to skip spaces since tabs could also be used as a > delimiter. > > >? ? -> Extra space here before the '(': "sizeof (buf)" > Done. > > > > Finally your return sequence for that method could be simplified to: > > > > +? if (!found_state) { > > +? ? print_error(" Could not find the State: string in the status > file > > for pid %d\n", pid); > > +? } > > +? fclose (fp); > > +? return !found_state; > > I have modified this to: > > 257? ?if (!found_state) { > 258? ? ?// Assuming the thread exists. > 259? ? ?print_error("Could not find the 'State:' string in the > /proc/%d/status file\n", pid); > 260? ?} > 261? ?fclose (fp); > 262? ?return false; > > Thank you, > Jini. > > > > > Thanks! > > Jc > > > > On Tue, Dec 11, 2018 at 9:30 AM Jini George > > > >> > wrote: > > > >? ? ?Hello ! > > > >? ? ?Requesting reviews for: > > > > https://bugs.openjdk.java.net/browse/JDK-8202884 > >? ? ?Webrev: > http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/index.html > > > >? ? ?Details: > >? ? ?For attaching to the threads in a process, we first go ahead > and do a > >? ? ?ptrace attach to the main thread. Later, we use the libthread_db > >? ? ?library > >? ? ?(or, in the case of being within a container, iterate through the > >? ? ?/proc//task files) to discover the threads of the > process, and add > >? ? ?them to the threads list (within SA) for this process. Once, > we have > >? ? ?discovered all the threads and added these to the list of > threads, we > >? ? ?then invoke ptrace attach individually on all these threads to > >? ? ?attach to > >? ? ?these. When we deal with an application where the threads are > exiting > >? ? ?continuously, some of these threads might not exist by the > time we try > >? ? ?to ptrace attach to these threads. The proposed fix includes the > >? ? ?following modifications to solve this. > >? ? ? ? 1. Check the state of the threads in the thread_db > callback routine, > >? ? ?and skip if the state of the thread is TD_THR_UNKNOWN or > TD_THR_ZOMBIE. > >? ? ?SA does not try to ptrace attach to these threads and does > not include > >? ? ?these threads in the threads list. > >? ? ? ? 2. While ptrace attaching to the thread, if > ptrace(PTRACE_ATTACH, > >? ? ?...) > >? ? ?fails with either ESCRH or EPERM, check the state of the > thread by > >? ? ?checking if the /proc//status file corresponding to that > thread > >? ? ?exists and if so, reading in the 'State:' line of that file. Skip > >? ? ?attaching to this thread and delete this thread from the SA > list of > >? ? ?threads, if the thread is dead (State: X) or is a zombie > (State: Z). > >? ? ? ?From the /proc man page, "Current state of the process. One > of "R > >? ? ?(running)", "S (sleeping)", "D (disk sleep)", "T (stopped)", "T > >? ? ?(tracing > >? ? ?stop)", "Z (zombie)", or "X (dead)"." > >? ? ? ? 3. If waitpid() on the thread is a failure, again skip > this thread > >? ? ?(delete this from SA's list of threads) instead of bailing > out if the > >? ? ?thread has exited or terminated. > > > >? ? ?Testing: > >? ? ?1. Tested by attaching and detaching multiple times to a test > program > >? ? ?spawning numerous short lived threads. > >? ? ?2. The SA tests (under test/hotspot/jtreg/serviceability/sa) > passed > >? ? ?with > >? ? ?100 repeats on Mach5. > >? ? ?3. No new failures and no occurrences of JDK-8202884 seen > with testing > >? ? ?the SA tests (tiers 1 to 5) on Mach5. > > > >? ? ?More details in the bug comments section. > > > >? ? ?Thank you, > >? ? ?Jini. > > > > > > > > -- > > > > Thanks, > > Jc > > > > -- > > Thanks, > Jc From jcbeyler at google.com Wed Dec 12 18:39:22 2018 From: jcbeyler at google.com (JC Beyler) Date: Wed, 12 Dec 2018 10:39:22 -0800 Subject: RFR: JDK-8202884: SA: Attach/detach might fail on Linux if debugee application create/destroy threads during attaching In-Reply-To: References: Message-ID: Hi Jini, Fair enough, thanks for the explanation. It makes sense to me. I imagine that it is a conservative approach, ie can't find the information then assume still live. Perhaps a small comment above the 'X'/'Z' test saying that the threads are considered to be dead then? And for the return perhaps say: the thread is considered live if the state was not 'X'/'Z' or not found? But those are really nits and no need to send another webrev for me! Thanks! Jc On Wed, Dec 12, 2018 at 10:27 AM Jini George wrote: > > On 12/12/2018 10:47 PM, JC Beyler wrote: > > Hi Jini, > > > > Should your return not be return !found_state instead of false here: > > > > 257 if (!found_state) { > > 258 // Assuming the thread exists. > > 259 print_error("Could not find the 'State:' string in the > > /proc/%d/status file\n", pid); > > 260 } > > 261 fclose (fp); > > 262 return false; > > > > In your webrev.00 it was the case but now, you always return the process > > does exist even if you have not found it. > > I referred to the gdb sources to check what is done under this scenario, > and in gdb, it is assumed that if the line beginning with 'State:' is > not found, the thread is alive. But to be frank, I don't know under what > circumstances will we ever encounter such a scenario. Let me know if you > don't agree with this. > > > cf: > > > http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/src/jdk.hotspot.agent/linux/native/libsaproc/ps_proc.c.udiff.html > > vs > > > http://cr.openjdk.java.net/~jgeorge/8202884/webrev.02/src/jdk.hotspot.agent/linux/native/libsaproc/ps_proc.c.udiff.html > > > > Tiny nit: no need to check *space if you are using isspace(*space) right > > after :) > > Will change this. Thanks! > > > > > Apart from the return question, the webrev looks good to me :-) > > Jc > > Thank you again! > Jini > > > > > > > On Wed, Dec 12, 2018 at 4:15 AM Jini George > > wrote: > > > > Thank you very much for looking into this, JC! > > > > I have a revised webrev addressing your comments at: > > > > http://cr.openjdk.java.net/~jgeorge/8202884/webrev.02/index.html > > > > Requesting one more review for this. My comments inline: > > > > On 12/12/2018 2:53 AM, JC Beyler wrote: > > > Hi Jini, > > > > > > I saw a few nits: > > > > > > http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/src/jdk.hotspot.agent/linux/native/libsaproc/libproc_impl.h.udiff.html > > > -> The comments are in the third person normally it seems so > > it would > > > become (I also removed the s from threads): > > > > > > +// deletes a thread from the thread list > > Done. > > > > > > > > > http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/src/jdk.hotspot.agent/linux/native/libsaproc/libproc_impl.c.udiff.html > > > -> You added two empty lines it seems that could be removed > > Done. > > > > > > > > > > > > http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/src/jdk.hotspot.agent/linux/native/libsaproc/ps_proc.c.udiff.html > > > -> Is there a real reason to have both enums? We could have a > > single > > > enum it seems and not lose too much > > > > You are right. I have done away with the WAITPID* enum. > > > > > -> you have a switch " > > > switch (errno) {" > > > -> Where really you could simplify the reading by moving > > the > > > EINTR case outside with its continue > > > -> The switch could then remain as it was (though you > move > > > print_debug to print_error) > > > -> and just return in each cases > > I have changed this to: > > > > 206 } else { > > 207 switch (errno) { > > 208 case EINTR: > > 209 continue; > > 210 break; > > 211 case ECHILD: > > 212 print_debug("waitpid() failed. Child process pid (%d) > > does > > not exist \n", pid); > > 213 return ATTACH_THREAD_DEAD; > > 214 case EINVAL: > > 215 print_error("waitpid() failed. Invalid options > > argument.\n"); > > 216 return ATTACH_FAIL; > > 217 default: > > 218 print_error("waitpid() failed. Unexpected error %d\n", > > errno); > > 219 return ATTACH_FAIL; > > 220 } > > 221 } // else > > > > > > > > > > -> if (strncmp (buf, "State:", 6) == 0) { > > > -> You use sizeof("State:") right below; perhaps you could > > just > > > use " const char const state[] = "State:";" and use > > sizeof(state) and > > > for the string, it seems less error prone > > > > > > -> A minor "bug" is here: > > > + state = buf + sizeof ("State:"); > > > -> You did a strncmp above but that only assures the > > start of > > > the string is "State:", technically the character after the ':' > > is the > > > but it could only be that; sizeof("State:") is 7 and not 6. So > > you miss > > > one character when you are skipping spaces > > > -> It was probably ok because you always had at least one > > > space, ie "State: " > > > > Thanks! I have made some changes here to use a const char string and > a > > variable to store the calculated length using strlen(). And I am > using > > isspace() now to skip spaces since tabs could also be used as a > > delimiter. > > > > > -> Extra space here before the '(': "sizeof (buf)" > > Done. > > > > > > Finally your return sequence for that method could be simplified > to: > > > > > > + if (!found_state) { > > > + print_error(" Could not find the State: string in the status > > file > > > for pid %d\n", pid); > > > + } > > > + fclose (fp); > > > + return !found_state; > > > > I have modified this to: > > > > 257 if (!found_state) { > > 258 // Assuming the thread exists. > > 259 print_error("Could not find the 'State:' string in the > > /proc/%d/status file\n", pid); > > 260 } > > 261 fclose (fp); > > 262 return false; > > > > Thank you, > > Jini. > > > > > > > > Thanks! > > > Jc > > > > > > On Tue, Dec 11, 2018 at 9:30 AM Jini George > > > > > >> > > wrote: > > > > > > Hello ! > > > > > > Requesting reviews for: > > > > > > https://bugs.openjdk.java.net/browse/JDK-8202884 > > > Webrev: > > http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/index.html > > > > > > Details: > > > For attaching to the threads in a process, we first go ahead > > and do a > > > ptrace attach to the main thread. Later, we use the > libthread_db > > > library > > > (or, in the case of being within a container, iterate through > the > > > /proc//task files) to discover the threads of the > > process, and add > > > them to the threads list (within SA) for this process. Once, > > we have > > > discovered all the threads and added these to the list of > > threads, we > > > then invoke ptrace attach individually on all these threads to > > > attach to > > > these. When we deal with an application where the threads are > > exiting > > > continuously, some of these threads might not exist by the > > time we try > > > to ptrace attach to these threads. The proposed fix includes > the > > > following modifications to solve this. > > > 1. Check the state of the threads in the thread_db > > callback routine, > > > and skip if the state of the thread is TD_THR_UNKNOWN or > > TD_THR_ZOMBIE. > > > SA does not try to ptrace attach to these threads and does > > not include > > > these threads in the threads list. > > > 2. While ptrace attaching to the thread, if > > ptrace(PTRACE_ATTACH, > > > ...) > > > fails with either ESCRH or EPERM, check the state of the > > thread by > > > checking if the /proc//status file corresponding to that > > thread > > > exists and if so, reading in the 'State:' line of that file. > Skip > > > attaching to this thread and delete this thread from the SA > > list of > > > threads, if the thread is dead (State: X) or is a zombie > > (State: Z). > > > From the /proc man page, "Current state of the process. One > > of "R > > > (running)", "S (sleeping)", "D (disk sleep)", "T (stopped)", > "T > > > (tracing > > > stop)", "Z (zombie)", or "X (dead)"." > > > 3. If waitpid() on the thread is a failure, again skip > > this thread > > > (delete this from SA's list of threads) instead of bailing > > out if the > > > thread has exited or terminated. > > > > > > Testing: > > > 1. Tested by attaching and detaching multiple times to a test > > program > > > spawning numerous short lived threads. > > > 2. The SA tests (under test/hotspot/jtreg/serviceability/sa) > > passed > > > with > > > 100 repeats on Mach5. > > > 3. No new failures and no occurrences of JDK-8202884 seen > > with testing > > > the SA tests (tiers 1 to 5) on Mach5. > > > > > > More details in the bug comments section. > > > > > > Thank you, > > > Jini. > > > > > > > > > > > > -- > > > > > > Thanks, > > > Jc > > > > > > > > -- > > > > Thanks, > > Jc > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From jcbeyler at google.com Wed Dec 12 19:25:46 2018 From: jcbeyler at google.com (JC Beyler) Date: Wed, 12 Dec 2018 11:25:46 -0800 Subject: RFR (M) 8201655: Add thread-enabled support for the Heap Sampling In-Reply-To: References: <0e62b2cc-c28e-5cc3-59df-5ce5171d4475@oracle.com> <102a9f1b-8f07-06f4-7749-4d04eccd3a91@oracle.com> <1b8ab7a1-ec08-4397-020e-24463688112b@oracle.com> Message-ID: Thanks both for the review, I fixed both issues and here is the new webrev, let me know what you think: Webrev: http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.06/ Bug: https://bugs.openjdk.java.net/browse/JDK-8201655 Thanks! Jc On Tue, Dec 11, 2018 at 5:01 PM wrote: > Hi Alex, > > Nice catch! > > It is about the following fragment: > > 726 for (i = 0; i < thread_stats.number_threads; i++) { 727 if (strcmp(expected_name, thread_stats.threads[i])) { 728 return FALSE; > 729 } else { 730 found_thread = TRUE; > 731 } > 732 } 733 return found_thread; 734 } > > Also, I'd also use 'count' in place of 'number'. > It is to avoid association with thread identification number. > > Thanks, > Serguei > > > On 12/11/18 4:42 PM, Alex Menkov wrote: > > Hi Jc, > > The fix looks good. > The only note is checkThreadSamplesOnlyFrom native function implementation > - the cycle looks confusing. > As far as I got the function should check that thread_stats contains only > 1 thread and name of the thread is the same as name of the specified > thread. > And for error analysis it would be great to provide good error > description. > So I'd make it like > > if (thread_stats.number_threads != 1) { > fprintf(stderr, "Wrong thread number: %d (expected 1)\n", > thread_stats.number_threads); > return FALSE; > } > if (strcmp(expected_name, thread_stats.threads[i]) != 0) { > fprintf(stderr, "Unexpected thread name: '%s' (expected '%s')\n", > thread_stats.threads[i], expected_name); > return FALSE; > } > return TRUE; > > --alex > > On 12/11/2018 15:11, serguei.spitsyn at oracle.com wrote: > > Hi Jc, > > Alex will take a look at the test update. > > Thanks, > Serguei > > On 11/12/18 9:53 AM, JC Beyler wrote: > > Hi Serguei, > > Thanks for the update and thanks for testing mach5. Serguei sent me that > the testing passed mach5 testing, could I get another review to be able to > push it? > > Thanks! > Jc > > On Tue, Nov 6, 2018 at 10:41 PM serguei.spitsyn at oracle.com > < > serguei.spitsyn at oracle.com > > wrote: > > Hi Jc, > > Thank you for the update! > It looks good. > It is great that testing on your side is Okay. > > I'll submit a mach5 job soon (today or tomorrow). > > Thanks, > Serguei > > > On 11/6/18 20:03, JC Beyler wrote: > > Hi Serguei, > > You are right, I should have reverted the memAllocator.cpp file, > sorry about that. > > Here is the new webrev: > http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.04/ > > > > I think we are good by testing standards, like I > said HeapMonitorThreadTest.java tests multiple threads. I did > test an example with a thousand threads and I get the samples > from 1000 threads so it seems to work there too. > > Per thread is tested via the new > HeapMonitorThreadDisabledTest.java so I think we are good there too. > > I would recommend a mach-5 testing just in case for this one if > you can, it will be better to have that reinsurance. > > Thanks for your help, > Jc > > On Tue, Nov 6, 2018 at 4:29 PM > > wrote: > > Hi Jc, > > Not sure, I understand a motivation for this change: > > - if (JvmtiExport::should_post_sampled_object_alloc()) { > + { > > Also, I'm not sure this is still needed: > > +#include "prims/jvmtiEventController.inline.hpp" > +#include "prims/jvmtiThreadState.inline.hpp" > > I expected you'd just revert all the changes in the > memAllocator.cpp. > > Also, it is up to you to make a decision if these > performance-related fix is needed or not. > > But it needs to be well tested so that both global+thread > event management works correctly. > > Thanks, > Serguei > > > On 11/6/18 9:42 AM, JC Beyler wrote: > > Hi Serguei, > > Yes exactly it was an optimization. When using a 512k > sampling rate, I don't see a no real difference (the > overhead is anyway low for that sampling rate), I imagine > there would be a difference if trying to sample every > allocation or with a low sampling interval. But because you > are right and it is an optimization of the system and not a > functional need, I've reverted it and now the webrev is > updated here: > > Webrev: > http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.03/ > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8201655 > > The incremental webrev is here: > http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.02_03/ > > > > Let me know what you think, > Jc > > On Mon, Nov 5, 2018 at 6:51 PM serguei.spitsyn at oracle.com > > > > wrote: > > Hi Jc, > > Okay, I see your point: the change in memAllocator.cpp > is for performance. > Do you have any measurements showing a performance > difference? > Also, do you need me to submit a mach5 test run? > > Thanks, > Serguei > > > On 11/5/18 15:14, JC Beyler wrote: > > Hi Serguei, > > First off, thanks as always for looking at this :-) Let > me inline my answers: > > I actually "struggled" with this part of the change. My > change is correct semantically and if you care about > performance for when sampling a given thread. > Your change will work semantically but the performance > is the same as the global sampling. > > What I mean by working semantically is that that the > tests and the code will work. However, this means that > all threads will be doing the sampling work but when > the code will post the event here: > -> > > http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.02/src/hotspot/share/prims/jvmtiExport.cpp.udiff.html > > > > > (which is why your suggestion works, the change in > jvmtiExport basically ensures only the threads > requested are posting events) > > The code will check that we actually only post for > threads we care about. The code above ensures that only > threads that were requested to be sampling are the ones > that are sampling internally. > > Note: I REALLY prefer your suggestion for two reasons: > - We do not change the runtime/GC code at all, it > remains "simple" > - The overhead in the general case goes away and this > is a NOP for my actual use-case from a performance > point of view (sampling every thread) > > But: > - Then sampling per thread really is just telling the > system don't pollute the callbacks, though internally > you are doing all the work anyway. > > Let me know which you prefer :) > > > Also, do you see that enabling the sampling events > globally still works? > > > Yes, otherwise HeapMonitorThreadTest.java would fail > since it checks that. > > > http://cr.openjdk.java.net/%7Ejcbeyler/8201655/webrev.02/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.c.frames.html > > A couple of places where err is declared as int instead > of jvmtiError: > 714 int err; > 742 int err; Should not be silent in a case of > JVMTI error: 744 err = (*jvmti)->GetThreadInfo(jvmti, > thread, &info); > 745 if (err != JVMTI_ERROR_NONE) { > 746 return; > > > > Done and done, I added a fprintf on stderr saying the > GetThreadInfo failed and the test is ignoring the add > count. > > Thanks again for looking and let me know what you think, > Jc > > On Mon, Nov 5, 2018 at 2:25 PM > serguei.spitsyn at oracle.com > > > > > wrote: > > Hi Jc, > > It looks good in general but I have some comments > below. > > > > http://cr.openjdk.java.net/%7Ejcbeyler/8201655/webrev.02/src/hotspot/share/gc/shared/memAllocator.cpp.udiff.html > > +static bool thread_enabled_for_one_jvmti_env() { > + JavaThread *thread = JavaThread::current(); > + JvmtiThreadState *state = > thread->jvmti_thread_state(); > + if (state == NULL) { > + return false; > + } > + > + JvmtiEnvThreadStateIterator it(state); > + for (JvmtiEnvThreadState* ets = it.first(); ets > != NULL; ets = it.next(ets)) { > + if > (ets->is_enabled(JVMTI_EVENT_SAMPLED_OBJECT_ALLOC)) { > + return true; > + } > + } > + > + return false; > +} > + > void > MemAllocator::Allocation::notify_allocation_jvmti_sampler() { > // support for JVMTI VMObjectAlloc event (no-op if not > enabled) > JvmtiExport::vm_object_alloc_event_collector(obj()); > if > (!JvmtiExport::should_post_sampled_object_alloc()) { > // Sampling disabled > return; > } > + // Sampling is enabled for at least > one thread, > is it this one? > + if (!thread_enabled_for_one_jvmti_env()) { > + return; > + } > + I don't think you need this change as this > condition already does it: if > (!JvmtiExport::should_post_sampled_object_alloc()) { > // Sampling disabled > return; > } > > Please, look at the following line in the > jvmtiEventController.cpp: > > JvmtiExport::set_should_post_sampled_object_alloc((any_env_thread_enabled & > SAMPLED_OBJECT_ALLOC_BIT) != 0); > > I hope, testing will prove my suggestion is correct. > Also, do you see that enabling the sampling events > globally still works? > > > > http://cr.openjdk.java.net/%7Ejcbeyler/8201655/webrev.02/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.c.frames.html > > A couple of places where err is declared as int instead > of jvmtiError: > 714 int err; > 742 int err; Should not be silent in a case of > JVMTI error: 744 err = (*jvmti)->GetThreadInfo(jvmti, > thread, &info); > 745 if (err != JVMTI_ERROR_NONE) { > 746 return; > > > Thanks, > Serguei > > > On 10/26/18 10:48, JC Beyler wrote: > > Hi all, > > When working on the heap sampling, I had promised > to do the per thread event so here it is! > > Could I get a review for this: > Webrev: > http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.02/ > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8201655 > > I was thinking of adding GC-dev for the > memAllocator change once I get favorable reviews > for the rest of the change. > > I've done a bit of performance testing and on the > Dacapo benchmark I see no change in performance > when turned off (logical, any code change is > behind a flag check already in place) and when > turned on it is comparable to the current performance. > > (More information is: I see a very slight > degradation if we are doing 512k sampling but no > degradation at 2MB). > > Thanks, > Jc > > > > > -- > Thanks, > Jc > > > > > -- > Thanks, > Jc > > > > > -- > Thanks, > Jc > > > > > -- > > Thanks, > Jc > > > > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Wed Dec 12 19:45:52 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 12 Dec 2018 11:45:52 -0800 Subject: RFR: JDK-8051349: nsk/jvmti/scenarios/sampling/SP06/sp06t003 fails in nightly In-Reply-To: References: <5C113D4D.7070503@oracle.com> Message-ID: <90d7b4eb-0c0c-63b3-1934-8f8a39061e05@oracle.com> An HTML attachment was scrubbed... URL: From alexey.menkov at oracle.com Wed Dec 12 20:15:01 2018 From: alexey.menkov at oracle.com (Alex Menkov) Date: Wed, 12 Dec 2018 12:15:01 -0800 Subject: RFR (M) 8201655: Add thread-enabled support for the Heap Sampling In-Reply-To: References: <0e62b2cc-c28e-5cc3-59df-5ce5171d4475@oracle.com> <102a9f1b-8f07-06f4-7749-4d04eccd3a91@oracle.com> <1b8ab7a1-ec08-4397-020e-24463688112b@oracle.com> Message-ID: <0bf7779b-2bc6-49fc-94b7-28d37d6b91d7@oracle.com> Looks good to me. --alex On 12/12/2018 11:25, JC Beyler wrote: > Thanks both for the review, I fixed both issues and here is the new > webrev, let me know what you think: > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.06/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8201655 > > Thanks! > Jc > > On Tue, Dec 11, 2018 at 5:01 PM > wrote: > > Hi Alex, > > Nice catch! > > It is about the following fragment: > > 726 for (i = 0; i < thread_stats.number_threads; i++) { > 727 if (strcmp(expected_name, thread_stats.threads[i])) { > 728 return FALSE; > 729 } else { > 730 found_thread = TRUE; > 731 } > 732 } > 733 return found_thread; > 734 } > > Also, I'd also use 'count' in place of 'number'. > It is to avoid association with thread identification number. > > Thanks, > Serguei > > > On 12/11/18 4:42 PM, Alex Menkov wrote: >> Hi Jc, >> >> The fix looks good. >> The only note is checkThreadSamplesOnlyFrom native function >> implementation - the cycle looks confusing. >> As far as I got the function should check that thread_stats >> contains only 1 thread and name of the thread is the same as name >> of the specified thread. >> And for error analysis it would be great to provide good error >> description. >> So I'd make it like >> >> if (thread_stats.number_threads != 1) { >> ? fprintf(stderr, "Wrong thread number: %d (expected 1)\n", >> thread_stats.number_threads); >> ? return FALSE; >> } >> if (strcmp(expected_name, thread_stats.threads[i]) != 0) { >> ? fprintf(stderr, "Unexpected thread name: '%s' (expected >> '%s')\n", thread_stats.threads[i], expected_name); >> ? return FALSE; >> } >> return TRUE; >> >> --alex >> >> On 12/11/2018 15:11, serguei.spitsyn at oracle.com >> wrote: >>> Hi Jc, >>> >>> Alex will take a look at the test update. >>> >>> Thanks, >>> Serguei >>> >>> On 11/12/18 9:53 AM, JC Beyler wrote: >>>> Hi Serguei, >>>> >>>> Thanks for the update and thanks for testing mach5. Serguei sent >>>> me that the testing passed mach5 testing, could I get another >>>> review to be able to push it? >>>> >>>> Thanks! >>>> Jc >>>> >>>> On Tue, Nov 6, 2018 at 10:41 PM serguei.spitsyn at oracle.com >>>> >>>> >>>> >>> >>>> >>>> > wrote: >>>> >>>> ??? Hi Jc, >>>> >>>> ??? Thank you for the update! >>>> ??? It looks good. >>>> ??? It is great that testing on your side is Okay. >>>> >>>> ??? I'll submit a mach5 job soon (today or tomorrow). >>>> >>>> ??? Thanks, >>>> ??? Serguei >>>> >>>> >>>> ??? On 11/6/18 20:03, JC Beyler wrote: >>>>> ??? Hi Serguei, >>>>> >>>>> ??? You are right, I should have reverted the memAllocator.cpp >>>>> file, >>>>> ??? sorry about that. >>>>> >>>>> ??? Here is the new webrev: >>>>> http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.04/ >>>>> >>>>> >>>>> >>>>> ??? I think we are good by testing standards, like I >>>>> ??? said?HeapMonitorThreadTest.java tests multiple threads. I did >>>>> ??? test an example with a thousand threads and I get the samples >>>>> ??? from 1000 threads so it seems to work there too. >>>>> >>>>> ??? Per thread is tested via the new >>>>> ??? HeapMonitorThreadDisabledTest.java so I think we are good >>>>> there too. >>>>> >>>>> ??? I would recommend a mach-5 testing just in case for this >>>>> one if >>>>> ??? you can, it will be better to have that reinsurance. >>>>> >>>>> ??? Thanks for your help, >>>>> ??? Jc >>>>> >>>>> ??? On Tue, Nov 6, 2018 at 4:29 PM >>>> >>>>> >>>>> > wrote: >>>>> >>>>> ??????? Hi Jc, >>>>> >>>>> ??????? Not sure, I understand a motivation for this change: >>>>> >>>>> ??????? - if (JvmtiExport::should_post_sampled_object_alloc()) { >>>>> ??????? + { >>>>> >>>>> ??????? Also, I'm not sure this is still needed: >>>>> >>>>> ??????? +#include "prims/jvmtiEventController.inline.hpp" >>>>> ??????? +#include "prims/jvmtiThreadState.inline.hpp" >>>>> >>>>> ??????? I expected you'd just revert all the changes in the >>>>> ??????? memAllocator.cpp. >>>>> >>>>> ??????? Also, it is up to you to make a decision if these >>>>> ??????? performance-related fix is needed or not. >>>>> >>>>> ??????? But it needs to be well tested so that both global+thread >>>>> ??????? event management works correctly. >>>>> >>>>> ??????? Thanks, >>>>> ??????? Serguei >>>>> >>>>> >>>>> ??????? On 11/6/18 9:42 AM, JC Beyler wrote: >>>>>> ??????? Hi Serguei, >>>>>> >>>>>> ??????? Yes exactly it was an optimization. When using a 512k >>>>>> ??????? sampling rate, I don't see a no real difference (the >>>>>> ??????? overhead is anyway low for that sampling rate), I imagine >>>>>> ??????? there would be a difference if trying to sample every >>>>>> ??????? allocation or with a low sampling interval. But >>>>>> because you >>>>>> ??????? are right and it is an optimization of the system and >>>>>> not a >>>>>> ??????? functional need, I've reverted it and now the webrev is >>>>>> ??????? updated here: >>>>>> >>>>>> ??????? Webrev: >>>>>> http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.03/ >>>>>> >>>>>> >>>>>> ??????? Bug: https://bugs.openjdk.java.net/browse/JDK-8201655 >>>>>> >>>>>> ??????? The incremental webrev is here: >>>>>> http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.02_03/ >>>>>> >>>>>> >>>>>> >>>>>> ??????? Let me know what you think, >>>>>> ??????? Jc >>>>>> >>>>>> ??????? On Mon, Nov 5, 2018 at 6:51 PM >>>>>> serguei.spitsyn at oracle.com >>>>>> >>>>>> >>>>>> ??????? >>>>> >>>>>> >>>>>> > wrote: >>>>>> >>>>>> ??????????? Hi Jc, >>>>>> >>>>>> ??????????? Okay, I see your point: the change in >>>>>> memAllocator.cpp >>>>>> ??????????? is for performance. >>>>>> ??????????? Do you have any measurements showing a performance >>>>>> ??????????? difference? >>>>>> ??????????? Also, do you need me to submit a mach5 test run? >>>>>> >>>>>> ??????????? Thanks, >>>>>> ??????????? Serguei >>>>>> >>>>>> >>>>>> ??????????? On 11/5/18 15:14, JC Beyler wrote: >>>>>>> ??????????? Hi Serguei, >>>>>>> >>>>>>> ??????????? First off, thanks as always for looking at this >>>>>>> :-) Let >>>>>>> ??????????? me inline my answers: >>>>>>> >>>>>>> ??????????? I actually "struggled" with this part of the >>>>>>> change. My >>>>>>> ??????????? change is correct semantically and if you care about >>>>>>> ??????????? performance for when sampling a given thread. >>>>>>> ??????????? Your change will work semantically but the >>>>>>> performance >>>>>>> ??????????? is the same as the global sampling. >>>>>>> >>>>>>> ??????????? What I mean by working semantically is that that the >>>>>>> ??????????? tests and the code will work. However, this means >>>>>>> that >>>>>>> ??????????? all threads will be doing the sampling work but when >>>>>>> ??????????? the code will post the event here: >>>>>>> ??????????? ? ?-> >>>>>>> http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.02/src/hotspot/share/prims/jvmtiExport.cpp.udiff.html >>>>>>> >>>>>>> >>>>>>> >>>>>>> ??????????? (which is why your suggestion works, the change in >>>>>>> ??????????? jvmtiExport basically ensures only the threads >>>>>>> ??????????? requested are posting events) >>>>>>> >>>>>>> ??????????? The code will check that we actually only post for >>>>>>> ??????????? threads we care about. The code above ensures >>>>>>> that only >>>>>>> ??????????? threads that were requested to be sampling are >>>>>>> the ones >>>>>>> ??????????? that are sampling internally. >>>>>>> >>>>>>> ??????????? Note: I REALLY prefer your suggestion for two >>>>>>> reasons: >>>>>>> ??????????? ? - We do not change the runtime/GC code at all, it >>>>>>> ??????????? remains "simple" >>>>>>> ??????????? ? - The overhead in the general case goes away >>>>>>> and this >>>>>>> ??????????? is a NOP for my actual use-case from a performance >>>>>>> ??????????? point of view (sampling every thread) >>>>>>> >>>>>>> ??????????? But: >>>>>>> ??????????? ? - Then sampling per thread really is just >>>>>>> telling the >>>>>>> ??????????? system don't pollute the callbacks, though >>>>>>> internally >>>>>>> ??????????? you are doing all the work anyway. >>>>>>> >>>>>>> ??????????? Let me know which you prefer :) >>>>>>> >>>>>>> >>>>>>> ????????????????? Also, do you see that enabling the sampling >>>>>>> events globally still works? >>>>>>> >>>>>>> >>>>>>> ??????????? Yes, otherwise HeapMonitorThreadTest.java would fail >>>>>>> ??????????? since it checks that. >>>>>>> >>>>>>> http://cr.openjdk.java.net/%7Ejcbeyler/8201655/webrev.02/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.c.frames.html >>>>>>> >>>>>>> ????????????????? A couple of places where err is declared as >>>>>>> int instead of jvmtiError: >>>>>>> ???????????????????? 714?? int err; >>>>>>> ??????????????? 742 int err; Should not be silent in a case of >>>>>>> ??????????????? JVMTI error:?? 744?? err = >>>>>>> (*jvmti)->GetThreadInfo(jvmti, thread, &info); >>>>>>> ????????????????? 745?? if (err != JVMTI_ERROR_NONE) { >>>>>>> ??????????????? 746 return; >>>>>>> >>>>>>> >>>>>>> >>>>>>> ??????????? Done and done, I added a fprintf on stderr saying >>>>>>> the >>>>>>> ??????????? GetThreadInfo failed and the test is ignoring the >>>>>>> add >>>>>>> ??????????? count. >>>>>>> >>>>>>> ??????????? Thanks again for looking and let me know what you >>>>>>> think, >>>>>>> ??????????? Jc >>>>>>> >>>>>>> ??????????? On Mon, Nov 5, 2018 at 2:25 PM >>>>>>> serguei.spitsyn at oracle.com >>>>>>> >>>>>>> >>>>>>> ??????????? >>>>>> >>>>>>> >>>>>>> > wrote: >>>>>>> >>>>>>> ??????????????? Hi Jc, >>>>>>> >>>>>>> ??????????????? It looks good in general but I have some >>>>>>> comments >>>>>>> ??????????????? below. >>>>>>> >>>>>>> >>>>>>> http://cr.openjdk.java.net/%7Ejcbeyler/8201655/webrev.02/src/hotspot/share/gc/shared/memAllocator.cpp.udiff.html >>>>>>> >>>>>>> ??????????????? +static bool >>>>>>> thread_enabled_for_one_jvmti_env() { >>>>>>> ??????????????? + JavaThread *thread = JavaThread::current(); >>>>>>> ??????????????? + JvmtiThreadState *state = >>>>>>> ??????????????? thread->jvmti_thread_state(); >>>>>>> ??????????????? + if (state == NULL) { >>>>>>> ??????????????? + return false; >>>>>>> ??????????????? + } >>>>>>> ??????????????? + >>>>>>> ??????????????? + JvmtiEnvThreadStateIterator it(state); >>>>>>> ??????????????? + for (JvmtiEnvThreadState* ets = it.first(); >>>>>>> ets >>>>>>> ??????????????? != NULL; ets = it.next(ets)) { >>>>>>> ??????????????? + if >>>>>>> (ets->is_enabled(JVMTI_EVENT_SAMPLED_OBJECT_ALLOC)) { >>>>>>> ??????????????? + return true; >>>>>>> ??????????????? + } >>>>>>> ??????????????? + } >>>>>>> ??????????????? + >>>>>>> ??????????????? + return false; >>>>>>> ??????????????? +} >>>>>>> ??????????????? + >>>>>>> ????????????????? void >>>>>>> MemAllocator::Allocation::notify_allocation_jvmti_sampler() { >>>>>>> ??????????????????? // support for JVMTI VMObjectAlloc event >>>>>>> (no-op if not enabled) >>>>>>> JvmtiExport::vm_object_alloc_event_collector(obj()); >>>>>>> ????????????????? ??????????????????? if >>>>>>> (!JvmtiExport::should_post_sampled_object_alloc()) { >>>>>>> ????????????????????? // Sampling disabled >>>>>>> ????????????????????? return; >>>>>>> ??????????????????? } >>>>>>> ????????????????? ??????????????? + // Sampling is enabled >>>>>>> for at least one thread, >>>>>>> ??????????????? is it this one? >>>>>>> ??????????????? + if (!thread_enabled_for_one_jvmti_env()) { >>>>>>> ??????????????? + return; >>>>>>> ??????????????? + } >>>>>>> ??????????????? + I don't think you need this change as this >>>>>>> ??????????????? condition already does it:???? if >>>>>>> (!JvmtiExport::should_post_sampled_object_alloc()) { >>>>>>> ????????????????????? // Sampling disabled >>>>>>> ????????????????????? return; >>>>>>> ??????????????????? } >>>>>>> >>>>>>> ????????????????? Please, look at the following line in the >>>>>>> jvmtiEventController.cpp: >>>>>>> JvmtiExport::set_should_post_sampled_object_alloc((any_env_thread_enabled >>>>>>> & SAMPLED_OBJECT_ALLOC_BIT) != 0); >>>>>>> >>>>>>> ????????????????? I hope, testing will prove my suggestion is >>>>>>> correct. >>>>>>> ????????????????? Also, do you see that enabling the sampling >>>>>>> events globally still works? >>>>>>> >>>>>>> >>>>>>> http://cr.openjdk.java.net/%7Ejcbeyler/8201655/webrev.02/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.c.frames.html >>>>>>> >>>>>>> ????????????????? A couple of places where err is declared as >>>>>>> int instead of jvmtiError: >>>>>>> ???????????????????? 714?? int err; >>>>>>> ??????????????? 742 int err; Should not be silent in a case of >>>>>>> ??????????????? JVMTI error:?? 744?? err = >>>>>>> (*jvmti)->GetThreadInfo(jvmti, thread, &info); >>>>>>> ????????????????? 745?? if (err != JVMTI_ERROR_NONE) { >>>>>>> ??????????????? 746 return; >>>>>>> >>>>>>> >>>>>>> ??????????????? Thanks, >>>>>>> ??????????????? Serguei >>>>>>> >>>>>>> >>>>>>> ??????????????? On 10/26/18 10:48, JC Beyler wrote: >>>>>>>> ??????????????? Hi all, >>>>>>>> >>>>>>>> ??????????????? When working on the heap sampling, I had >>>>>>>> promised >>>>>>>> ??????????????? to do the per thread event so here it is! >>>>>>>> >>>>>>>> ??????????????? Could I get a review for this: >>>>>>>> ??????????????? Webrev: >>>>>>>> http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.02/ >>>>>>>> >>>>>>>> >>>>>>>> ??????????????? Bug: >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8201655 >>>>>>>> >>>>>>>> ??????????????? I was thinking of adding GC-dev for the >>>>>>>> ??????????????? memAllocator change once I get favorable >>>>>>>> reviews >>>>>>>> ??????????????? for the rest of the change. >>>>>>>> >>>>>>>> ??????????????? I've done a bit of performance testing and >>>>>>>> on the >>>>>>>> ??????????????? Dacapo benchmark I see no change in performance >>>>>>>> ??????????????? when turned off (logical, any code change is >>>>>>>> ??????????????? behind a flag check already in place) and when >>>>>>>> ??????????????? turned on it is comparable to the current >>>>>>>> performance. >>>>>>>> >>>>>>>> ??????????????? (More information is: I see a very slight >>>>>>>> ??????????????? degradation if we are doing 512k sampling >>>>>>>> but no >>>>>>>> ??????????????? degradation at 2MB). >>>>>>>> >>>>>>>> ??????????????? Thanks, >>>>>>>> ??????????????? Jc >>>>>>> >>>>>>> >>>>>>> >>>>>>> ??????????? -- >>>>>>> ??????????? Thanks, >>>>>>> ??????????? Jc >>>>>> >>>>>> >>>>>> >>>>>> ??????? -- >>>>>> ??????? Thanks, >>>>>> ??????? Jc >>>>> >>>>> >>>>> >>>>> ??? -- >>>>> ??? Thanks, >>>>> ??? Jc >>>> >>>> >>>> >>>> -- >>>> >>>> Thanks, >>>> Jc >>> > > > > -- > > Thanks, > Jc From serguei.spitsyn at oracle.com Wed Dec 12 20:17:52 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 12 Dec 2018 12:17:52 -0800 Subject: RFR (M) 8201655: Add thread-enabled support for the Heap Sampling In-Reply-To: <0bf7779b-2bc6-49fc-94b7-28d37d6b91d7@oracle.com> References: <0e62b2cc-c28e-5cc3-59df-5ce5171d4475@oracle.com> <102a9f1b-8f07-06f4-7749-4d04eccd3a91@oracle.com> <1b8ab7a1-ec08-4397-020e-24463688112b@oracle.com> <0bf7779b-2bc6-49fc-94b7-28d37d6b91d7@oracle.com> Message-ID: <3f49676e-acc7-3d03-ebb0-14e8e2fbe708@oracle.com> LGTM++ Thanks, Serguei On 12/12/18 12:15, Alex Menkov wrote: > Looks good to me. > > --alex > > On 12/12/2018 11:25, JC Beyler wrote: >> Thanks both for the review, I fixed both issues and here is the new >> webrev, let me know what you think: >> >> Webrev: http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.06/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8201655 >> >> Thanks! >> Jc >> >> On Tue, Dec 11, 2018 at 5:01 PM > > wrote: >> >> ??? Hi Alex, >> >> ??? Nice catch! >> >> ??? It is about the following fragment: >> >> ????? 726?? for (i = 0; i < thread_stats.number_threads; i++) { >> ??? 727 if (strcmp(expected_name, thread_stats.threads[i])) { >> ??? 728 return FALSE; >> ????? 729???? } else { >> ??? 730 found_thread = TRUE; >> ????? 731???? } >> ????? 732?? } >> ??? 733 return found_thread; >> ??? 734 } >> >> ??? Also, I'd also use 'count' in place of 'number'. >> ??? It is to avoid association with thread identification number. >> >> ??? Thanks, >> ??? Serguei >> >> >> ??? On 12/11/18 4:42 PM, Alex Menkov wrote: >>> ??? Hi Jc, >>> >>> ??? The fix looks good. >>> ??? The only note is checkThreadSamplesOnlyFrom native function >>> ??? implementation - the cycle looks confusing. >>> ??? As far as I got the function should check that thread_stats >>> ??? contains only 1 thread and name of the thread is the same as name >>> ??? of the specified thread. >>> ??? And for error analysis it would be great to provide good error >>> ??? description. >>> ??? So I'd make it like >>> >>> ??? if (thread_stats.number_threads != 1) { >>> ??? ? fprintf(stderr, "Wrong thread number: %d (expected 1)\n", >>> ??? thread_stats.number_threads); >>> ??? ? return FALSE; >>> ??? } >>> ??? if (strcmp(expected_name, thread_stats.threads[i]) != 0) { >>> ??? ? fprintf(stderr, "Unexpected thread name: '%s' (expected >>> ??? '%s')\n", thread_stats.threads[i], expected_name); >>> ??? ? return FALSE; >>> ??? } >>> ??? return TRUE; >>> >>> ??? --alex >>> >>> ??? On 12/11/2018 15:11, serguei.spitsyn at oracle.com >>> ??? wrote: >>>> ??? Hi Jc, >>>> >>>> ??? Alex will take a look at the test update. >>>> >>>> ??? Thanks, >>>> ??? Serguei >>>> >>>> ??? On 11/12/18 9:53 AM, JC Beyler wrote: >>>>> ??? Hi Serguei, >>>>> >>>>> ??? Thanks for the update and thanks for testing mach5. Serguei sent >>>>> ??? me that the testing passed mach5 testing, could I get another >>>>> ??? review to be able to push it? >>>>> >>>>> ??? Thanks! >>>>> ??? Jc >>>>> >>>>> ??? On Tue, Nov 6, 2018 at 10:41 PM serguei.spitsyn at oracle.com >>>>> ??? >>>>> ??? >>>>> ??? >>>> ??? >>>>> ??? >>>>> ??? > wrote: >>>>> >>>>> ??? ??? Hi Jc, >>>>> >>>>> ??? ??? Thank you for the update! >>>>> ??? ??? It looks good. >>>>> ??? ??? It is great that testing on your side is Okay. >>>>> >>>>> ??? ??? I'll submit a mach5 job soon (today or tomorrow). >>>>> >>>>> ??? ??? Thanks, >>>>> ??? ??? Serguei >>>>> >>>>> >>>>> ??? ??? On 11/6/18 20:03, JC Beyler wrote: >>>>>> ??? ??? Hi Serguei, >>>>>> >>>>>> ??? ??? You are right, I should have reverted the memAllocator.cpp >>>>>> ??? file, >>>>>> ??? ??? sorry about that. >>>>>> >>>>>> ??? ??? Here is the new webrev: >>>>>> http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.04/ >>>>>> >>>>>> >>>>>> >>>>>> ??? ??? I think we are good by testing standards, like I >>>>>> ??? ??? said?HeapMonitorThreadTest.java tests multiple threads. I >>>>>> did >>>>>> ??? ??? test an example with a thousand threads and I get the >>>>>> samples >>>>>> ??? ??? from 1000 threads so it seems to work there too. >>>>>> >>>>>> ??? ??? Per thread is tested via the new >>>>>> ??? ??? HeapMonitorThreadDisabledTest.java so I think we are good >>>>>> ??? there too. >>>>>> >>>>>> ??? ??? I would recommend a mach-5 testing just in case for this >>>>>> ??? one if >>>>>> ??? ??? you can, it will be better to have that reinsurance. >>>>>> >>>>>> ??? ??? Thanks for your help, >>>>>> ??? ??? Jc >>>>>> >>>>>> ??? ??? On Tue, Nov 6, 2018 at 4:29 PM >>>>> ??? >>>>>> ??? >>>>>> ??? > wrote: >>>>>> >>>>>> ??? ??????? Hi Jc, >>>>>> >>>>>> ??? ??????? Not sure, I understand a motivation for this change: >>>>>> >>>>>> ??? ??????? - if (JvmtiExport::should_post_sampled_object_alloc()) { >>>>>> ??? ??????? + { >>>>>> >>>>>> ??? ??????? Also, I'm not sure this is still needed: >>>>>> >>>>>> ??? ??????? +#include "prims/jvmtiEventController.inline.hpp" >>>>>> ??? ??????? +#include "prims/jvmtiThreadState.inline.hpp" >>>>>> >>>>>> ??? ??????? I expected you'd just revert all the changes in the >>>>>> ??? ??????? memAllocator.cpp. >>>>>> >>>>>> ??? ??????? Also, it is up to you to make a decision if these >>>>>> ??? ??????? performance-related fix is needed or not. >>>>>> >>>>>> ??? ??????? But it needs to be well tested so that both >>>>>> global+thread >>>>>> ??? ??????? event management works correctly. >>>>>> >>>>>> ??? ??????? Thanks, >>>>>> ??? ??????? Serguei >>>>>> >>>>>> >>>>>> ??? ??????? On 11/6/18 9:42 AM, JC Beyler wrote: >>>>>>> ??? ??????? Hi Serguei, >>>>>>> >>>>>>> ??? ??????? Yes exactly it was an optimization. When using a 512k >>>>>>> ??? ??????? sampling rate, I don't see a no real difference (the >>>>>>> ??? ??????? overhead is anyway low for that sampling rate), I >>>>>>> imagine >>>>>>> ??? ??????? there would be a difference if trying to sample every >>>>>>> ??? ??????? allocation or with a low sampling interval. But >>>>>>> ??? because you >>>>>>> ??? ??????? are right and it is an optimization of the system and >>>>>>> ??? not a >>>>>>> ??? ??????? functional need, I've reverted it and now the webrev is >>>>>>> ??? ??????? updated here: >>>>>>> >>>>>>> ??? ??????? Webrev: >>>>>>> http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.03/ >>>>>>> >>>>>>> >>>>>>> ??? ??????? Bug: https://bugs.openjdk.java.net/browse/JDK-8201655 >>>>>>> >>>>>>> ??? ??????? The incremental webrev is here: >>>>>>> http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.02_03/ >>>>>>> >>>>>>> >>>>>>> >>>>>>> ??? ??????? Let me know what you think, >>>>>>> ??? ??????? Jc >>>>>>> >>>>>>> ??? ??????? On Mon, Nov 5, 2018 at 6:51 PM >>>>>>> ??? serguei.spitsyn at oracle.com >>>>>>> ??? >>>>>>> ??? >>>>>>> ??? ??????? >>>>>> ??? >>>>>>> ??? >>>>>>> ??? > wrote: >>>>>>> >>>>>>> ??? ??????????? Hi Jc, >>>>>>> >>>>>>> ??? ??????????? Okay, I see your point: the change in >>>>>>> ??? memAllocator.cpp >>>>>>> ??? ??????????? is for performance. >>>>>>> ??? ??????????? Do you have any measurements showing a performance >>>>>>> ??? ??????????? difference? >>>>>>> ??? ??????????? Also, do you need me to submit a mach5 test run? >>>>>>> >>>>>>> ??? ??????????? Thanks, >>>>>>> ??? ??????????? Serguei >>>>>>> >>>>>>> >>>>>>> ??? ??????????? On 11/5/18 15:14, JC Beyler wrote: >>>>>>>> ??? ??????????? Hi Serguei, >>>>>>>> >>>>>>>> ??? ??????????? First off, thanks as always for looking at this >>>>>>>> ??? :-) Let >>>>>>>> ??? ??????????? me inline my answers: >>>>>>>> >>>>>>>> ??? ??????????? I actually "struggled" with this part of the >>>>>>>> ??? change. My >>>>>>>> ??? ??????????? change is correct semantically and if you care >>>>>>>> about >>>>>>>> ??? ??????????? performance for when sampling a given thread. >>>>>>>> ??? ??????????? Your change will work semantically but the >>>>>>>> ??? performance >>>>>>>> ??? ??????????? is the same as the global sampling. >>>>>>>> >>>>>>>> ??? ??????????? What I mean by working semantically is that >>>>>>>> that the >>>>>>>> ??? ??????????? tests and the code will work. However, this means >>>>>>>> ??? that >>>>>>>> ??? ??????????? all threads will be doing the sampling work but >>>>>>>> when >>>>>>>> ??? ??????????? the code will post the event here: >>>>>>>> ??? ??????????? ? ?-> >>>>>>>> http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.02/src/hotspot/share/prims/jvmtiExport.cpp.udiff.html >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ??? ??????????? (which is why your suggestion works, the change in >>>>>>>> ??? ??????????? jvmtiExport basically ensures only the threads >>>>>>>> ??? ??????????? requested are posting events) >>>>>>>> >>>>>>>> ??? ??????????? The code will check that we actually only post for >>>>>>>> ??? ??????????? threads we care about. The code above ensures >>>>>>>> ??? that only >>>>>>>> ??? ??????????? threads that were requested to be sampling are >>>>>>>> ??? the ones >>>>>>>> ??? ??????????? that are sampling internally. >>>>>>>> >>>>>>>> ??? ??????????? Note: I REALLY prefer your suggestion for two >>>>>>>> ??? reasons: >>>>>>>> ??? ??????????? ? - We do not change the runtime/GC code at >>>>>>>> all, it >>>>>>>> ??? ??????????? remains "simple" >>>>>>>> ??? ??????????? ? - The overhead in the general case goes away >>>>>>>> ??? and this >>>>>>>> ??? ??????????? is a NOP for my actual use-case from a performance >>>>>>>> ??? ??????????? point of view (sampling every thread) >>>>>>>> >>>>>>>> ??? ??????????? But: >>>>>>>> ??? ??????????? ? - Then sampling per thread really is just >>>>>>>> ??? telling the >>>>>>>> ??? ??????????? system don't pollute the callbacks, though >>>>>>>> ??? internally >>>>>>>> ??? ??????????? you are doing all the work anyway. >>>>>>>> >>>>>>>> ??? ??????????? Let me know which you prefer :) >>>>>>>> >>>>>>>> >>>>>>>> ??? ????????????????? Also, do you see that enabling the sampling >>>>>>>> ??? events globally still works? >>>>>>>> >>>>>>>> >>>>>>>> ??? ??????????? Yes, otherwise HeapMonitorThreadTest.java would >>>>>>>> fail >>>>>>>> ??? ??????????? since it checks that. >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/%7Ejcbeyler/8201655/webrev.02/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.c.frames.html >>>>>>>> >>>>>>>> ??? ????????????????? A couple of places where err is declared as >>>>>>>> ??? int instead of jvmtiError: >>>>>>>> ??? ???????????????????? 714?? int err; >>>>>>>> ??? ??????????????? 742 int err; Should not be silent in a case of >>>>>>>> ??? ??????????????? JVMTI error:?? 744?? err = >>>>>>>> ??? (*jvmti)->GetThreadInfo(jvmti, thread, &info); >>>>>>>> ??? ????????????????? 745?? if (err != JVMTI_ERROR_NONE) { >>>>>>>> ??? ??????????????? 746 return; >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ??? ??????????? Done and done, I added a fprintf on stderr saying >>>>>>>> ??? the >>>>>>>> ??? ??????????? GetThreadInfo failed and the test is ignoring the >>>>>>>> ??? add >>>>>>>> ??? ??????????? count. >>>>>>>> >>>>>>>> ??? ??????????? Thanks again for looking and let me know what you >>>>>>>> ??? think, >>>>>>>> ??? ??????????? Jc >>>>>>>> >>>>>>>> ??? ??????????? On Mon, Nov 5, 2018 at 2:25 PM >>>>>>>> ??? serguei.spitsyn at oracle.com >>>>>>>> ??? >>>>>>>> ??? >>>>>>>> ??? ??????????? >>>>>>> ??? >>>>>>>> ??? >>>>>>>> ??? > wrote: >>>>>>>> >>>>>>>> ??? ??????????????? Hi Jc, >>>>>>>> >>>>>>>> ??? ??????????????? It looks good in general but I have some >>>>>>>> ??? comments >>>>>>>> ??? ??????????????? below. >>>>>>>> >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/%7Ejcbeyler/8201655/webrev.02/src/hotspot/share/gc/shared/memAllocator.cpp.udiff.html >>>>>>>> >>>>>>>> ??? ??????????????? +static bool >>>>>>>> ??? thread_enabled_for_one_jvmti_env() { >>>>>>>> ??? ??????????????? + JavaThread *thread = JavaThread::current(); >>>>>>>> ??? ??????????????? + JvmtiThreadState *state = >>>>>>>> ??? ??????????????? thread->jvmti_thread_state(); >>>>>>>> ??? ??????????????? + if (state == NULL) { >>>>>>>> ??? ??????????????? + return false; >>>>>>>> ??? ??????????????? + } >>>>>>>> ??? ??????????????? + >>>>>>>> ??? ??????????????? + JvmtiEnvThreadStateIterator it(state); >>>>>>>> ??? ??????????????? + for (JvmtiEnvThreadState* ets = it.first(); >>>>>>>> ??? ets >>>>>>>> ??? ??????????????? != NULL; ets = it.next(ets)) { >>>>>>>> ??? ??????????????? + if >>>>>>>> (ets->is_enabled(JVMTI_EVENT_SAMPLED_OBJECT_ALLOC)) { >>>>>>>> ??? ??????????????? + return true; >>>>>>>> ??? ??????????????? + } >>>>>>>> ??? ??????????????? + } >>>>>>>> ??? ??????????????? + >>>>>>>> ??? ??????????????? + return false; >>>>>>>> ??? ??????????????? +} >>>>>>>> ??? ??????????????? + >>>>>>>> ??? ????????????????? void >>>>>>>> MemAllocator::Allocation::notify_allocation_jvmti_sampler() { >>>>>>>> ??? ??????????????????? // support for JVMTI VMObjectAlloc event >>>>>>>> ??? (no-op if not enabled) >>>>>>>> JvmtiExport::vm_object_alloc_event_collector(obj()); >>>>>>>> ??? ????????????????? ??????????????????? if >>>>>>>> (!JvmtiExport::should_post_sampled_object_alloc()) { >>>>>>>> ??? ????????????????????? // Sampling disabled >>>>>>>> ??? ????????????????????? return; >>>>>>>> ??? ??????????????????? } >>>>>>>> ??? ????????????????? ??????????????? + // Sampling is enabled >>>>>>>> ??? for at least one thread, >>>>>>>> ??? ??????????????? is it this one? >>>>>>>> ??? ??????????????? + if (!thread_enabled_for_one_jvmti_env()) { >>>>>>>> ??? ??????????????? + return; >>>>>>>> ??? ??????????????? + } >>>>>>>> ??? ??????????????? + I don't think you need this change as this >>>>>>>> ??? ??????????????? condition already does it: if >>>>>>>> (!JvmtiExport::should_post_sampled_object_alloc()) { >>>>>>>> ??? ????????????????????? // Sampling disabled >>>>>>>> ??? ????????????????????? return; >>>>>>>> ??? ??????????????????? } >>>>>>>> >>>>>>>> ??? ????????????????? Please, look at the following line in the >>>>>>>> ??? jvmtiEventController.cpp: >>>>>>>> JvmtiExport::set_should_post_sampled_object_alloc((any_env_thread_enabled >>>>>>>> ??? & SAMPLED_OBJECT_ALLOC_BIT) != 0); >>>>>>>> >>>>>>>> ??? ????????????????? I hope, testing will prove my suggestion is >>>>>>>> ??? correct. >>>>>>>> ??? ????????????????? Also, do you see that enabling the sampling >>>>>>>> ??? events globally still works? >>>>>>>> >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/%7Ejcbeyler/8201655/webrev.02/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/libHeapMonitorTest.c.frames.html >>>>>>>> >>>>>>>> ??? ????????????????? A couple of places where err is declared as >>>>>>>> ??? int instead of jvmtiError: >>>>>>>> ??? ???????????????????? 714?? int err; >>>>>>>> ??? ??????????????? 742 int err; Should not be silent in a case of >>>>>>>> ??? ??????????????? JVMTI error:?? 744?? err = >>>>>>>> ??? (*jvmti)->GetThreadInfo(jvmti, thread, &info); >>>>>>>> ??? ????????????????? 745?? if (err != JVMTI_ERROR_NONE) { >>>>>>>> ??? ??????????????? 746 return; >>>>>>>> >>>>>>>> >>>>>>>> ??? ??????????????? Thanks, >>>>>>>> ??? ??????????????? Serguei >>>>>>>> >>>>>>>> >>>>>>>> ??? ??????????????? On 10/26/18 10:48, JC Beyler wrote: >>>>>>>>> ??? ??????????????? Hi all, >>>>>>>>> >>>>>>>>> ??? ??????????????? When working on the heap sampling, I had >>>>>>>>> ??? promised >>>>>>>>> ??? ??????????????? to do the per thread event so here it is! >>>>>>>>> >>>>>>>>> ??? ??????????????? Could I get a review for this: >>>>>>>>> ??? ??????????????? Webrev: >>>>>>>>> http://cr.openjdk.java.net/~jcbeyler/8201655/webrev.02/ >>>>>>>>> >>>>>>>>> >>>>>>>>> ??? ??????????????? Bug: >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8201655 >>>>>>>>> >>>>>>>>> ??? ??????????????? I was thinking of adding GC-dev for the >>>>>>>>> ??? ??????????????? memAllocator change once I get favorable >>>>>>>>> ??? reviews >>>>>>>>> ??? ??????????????? for the rest of the change. >>>>>>>>> >>>>>>>>> ??? ??????????????? I've done a bit of performance testing and >>>>>>>>> ??? on the >>>>>>>>> ??? ??????????????? Dacapo benchmark I see no change in >>>>>>>>> performance >>>>>>>>> ??? ??????????????? when turned off (logical, any code change is >>>>>>>>> ??? ??????????????? behind a flag check already in place) and >>>>>>>>> when >>>>>>>>> ??? ??????????????? turned on it is comparable to the current >>>>>>>>> ??? performance. >>>>>>>>> >>>>>>>>> ??? ??????????????? (More information is: I see a very slight >>>>>>>>> ??? ??????????????? degradation if we are doing 512k sampling >>>>>>>>> ??? but no >>>>>>>>> ??? ??????????????? degradation at 2MB). >>>>>>>>> >>>>>>>>> ??? ??????????????? Thanks, >>>>>>>>> ??? ??????????????? Jc >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ??? ??????????? -- >>>>>>>> ??? ??????????? Thanks, >>>>>>>> ??? ??????????? Jc >>>>>>> >>>>>>> >>>>>>> >>>>>>> ??? ??????? -- >>>>>>> ??? ??????? Thanks, >>>>>>> ??? ??????? Jc >>>>>> >>>>>> >>>>>> >>>>>> ??? ??? -- >>>>>> ??? ??? Thanks, >>>>>> ??? ??? Jc >>>>> >>>>> >>>>> >>>>> ??? -- >>>>> ??? Thanks, >>>>> ??? Jc >>>> >> >> >> >> -- >> >> Thanks, >> Jc From serguei.spitsyn at oracle.com Wed Dec 12 21:28:10 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 12 Dec 2018 13:28:10 -0800 Subject: RFR: JDK-8051349: nsk/jvmti/scenarios/sampling/SP06/sp06t003 fails in nightly In-Reply-To: References: <5C113D4D.7070503@oracle.com> Message-ID: <3c71201a-4083-b25c-cc7b-84367c341720@oracle.com> An HTML attachment was scrubbed... URL: From yasuenag at gmail.com Thu Dec 13 00:45:07 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Thu, 13 Dec 2018 09:45:07 +0900 Subject: RFR: JDK-8202884: SA: Attach/detach might fail on Linux if debugee application create/destroy threads during attaching In-Reply-To: References: Message-ID: Hi Jini, I have a comment for your webrev.02 . You added process_doesnt_exist() to check process / thread liveness from /proc/, but it is not enough. Information of threads (LWP) will be stored in /proc//task/. So you should check /proc//task/status for threads. Thanks, Yasumasa On 2018/12/12 21:15, Jini George wrote: > Thank you very much for looking into this, JC! > > I have a revised webrev addressing your comments at: > > http://cr.openjdk.java.net/~jgeorge/8202884/webrev.02/index.html > > Requesting one more review for this. My comments inline: > > On 12/12/2018 2:53 AM, JC Beyler wrote: >> Hi Jini, >> >> I saw a few nits: >> http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/src/jdk.hotspot.agent/linux/native/libsaproc/libproc_impl.h.udiff.html >> ? -> The comments are in the third person normally it seems so it would >> become (I also removed the s from threads): >> >> +// deletes a thread from the thread list > Done. >> >> http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/src/jdk.hotspot.agent/linux/native/libsaproc/libproc_impl.c.udiff.html >> ? -> You added two empty lines it seems that could be removed > Done. >> >> >> http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/src/jdk.hotspot.agent/linux/native/libsaproc/ps_proc.c.udiff.html >> ? -> Is there a real reason to have both enums? We could have a single >> enum it seems and not lose too much > > You are right. I have done away with the WAITPID* enum. > >> ? -> you have a switch " >> ? ? ? ?switch (errno) {" >> ? ? ? ? -> Where really you could simplify the reading by moving the >> EINTR case outside with its continue >> ? ? ? ? -> The switch could then remain as it was (though you move >> print_debug to print_error) >> ? ? ? ? -> and just return in each cases > I have changed this to: > > 206 } else { > 207 switch (errno) { > 208 case EINTR: > 209 continue; > 210 break; > 211 case ECHILD: > 212 print_debug("waitpid() failed. Child process pid (%d) does > not exist \n", pid); > 213 return ATTACH_THREAD_DEAD; > 214 case EINVAL: > 215 print_error("waitpid() failed. Invalid options argument.\n"); > 216 return ATTACH_FAIL; > 217 default: > 218 print_error("waitpid() failed. Unexpected error %d\n", errno); > 219 return ATTACH_FAIL; > 220 } > 221 } // else > > >> >> ? ?->?if (strncmp (buf, "State:", 6) == 0) { >> ? ? ? -> You use sizeof("State:") right below; perhaps you could just >> use "? const char const state[] = "State:";" and use sizeof(state) and >> for the string, it seems less error prone >> >> ? -> A minor "bug" is here: >> +? ? ? state = buf + sizeof ("State:"); >> ? ? ? ? -> You did a strncmp above but that only assures the start of >> the string is "State:", technically the character after the ':' is the >> but it could only be that; sizeof("State:") is 7 and not 6. So you miss >> one character when you are skipping spaces >> ? ? ? ? -> It was probably ok because you always had at least one >> space, ie "State: " > > Thanks! I have made some changes here to use a const char string and a > variable to store the calculated length using strlen(). And I am using > isspace() now to skip spaces since tabs could also be used as a delimiter. > >> ? -> Extra space here before the '(': "sizeof (buf)" > Done. >> >> Finally your return sequence for that method could be simplified to: >> >> +? if (!found_state) { >> +? ? print_error(" Could not find the State: string in the status file >> for pid %d\n", pid); >> +? } >> +? fclose (fp); >> +? return !found_state; > > I have modified this to: > > 257 if (!found_state) { > 258 // Assuming the thread exists. > 259 print_error("Could not find the 'State:' string in the > /proc/%d/status file\n", pid); > 260 } > 261 fclose (fp); > 262 return false; > > Thank you, > Jini. > >> >> Thanks! >> Jc >> >> On Tue, Dec 11, 2018 at 9:30 AM Jini George > > wrote: >> >> Hello ! >> >> Requesting reviews for: >> >> https://bugs.openjdk.java.net/browse/JDK-8202884 >> Webrev: http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/index.html >> >> Details: >> For attaching to the threads in a process, we first go ahead and do a >> ptrace attach to the main thread. Later, we use the libthread_db >> library >> (or, in the case of being within a container, iterate through the >> /proc//task files) to discover the threads of the process, and add >> them to the threads list (within SA) for this process. Once, we have >> discovered all the threads and added these to the list of threads, we >> then invoke ptrace attach individually on all these threads to >> attach to >> these. When we deal with an application where the threads are exiting >> continuously, some of these threads might not exist by the time we try >> to ptrace attach to these threads. The proposed fix includes the >> following modifications to solve this. >> ? 1. Check the state of the threads in the thread_db callback routine, >> and skip if the state of the thread is TD_THR_UNKNOWN or TD_THR_ZOMBIE. >> SA does not try to ptrace attach to these threads and does not include >> these threads in the threads list. >> ? 2. While ptrace attaching to the thread, if ptrace(PTRACE_ATTACH, >> ...) >> fails with either ESCRH or EPERM, check the state of the thread by >> checking if the /proc//status file corresponding to that thread >> exists and if so, reading in the 'State:' line of that file. Skip >> attaching to this thread and delete this thread from the SA list of >> threads, if the thread is dead (State: X) or is a zombie (State: Z). >> ?From the /proc man page, "Current state of the process. One of "R >> (running)", "S (sleeping)", "D (disk sleep)", "T (stopped)", "T >> (tracing >> stop)", "Z (zombie)", or "X (dead)"." >> ? 3. If waitpid() on the thread is a failure, again skip this thread >> (delete this from SA's list of threads) instead of bailing out if the >> thread has exited or terminated. >> >> Testing: >> 1. Tested by attaching and detaching multiple times to a test program >> spawning numerous short lived threads. >> 2. The SA tests (under test/hotspot/jtreg/serviceability/sa) passed >> with >> 100 repeats on Mach5. >> 3. No new failures and no occurrences of JDK-8202884 seen with testing >> the SA tests (tiers 1 to 5) on Mach5. >> >> More details in the bug comments section. >> >> Thank you, >> Jini. >> >> >> >> -- >> >> Thanks, >> Jc > From zanglin5 at jd.com Thu Dec 13 01:37:06 2018 From: zanglin5 at jd.com (=?utf-8?B?6Ien55Cz?=) Date: Thu, 13 Dec 2018 01:37:06 +0000 Subject: optimize KlassInfoTable size to power of 2 In-Reply-To: <0045EBD6-F21D-457B-865E-3796BD86FFF9@amazon.com> References: <140bac18565c48be88313f7cc1d24bf8@jd.com> <7fc37fd7cf24439eb3a6fff132fc7387@jd.com> <4ea1c92f-4e72-7d38-8c7a-91e2892e0c3b@redhat.com> <1ab2d28b40024ba6b299756612fa3760@jd.com> <7a1e5d87-94fd-9be9-8aab-5dd0f1a3fff9@redhat.com> <56b9099e33584796af8a508980ff4690@jd.com> <436fddb22d3b4fb9aec315f3150cff41@jd.com> <0045EBD6-F21D-457B-865E-3796BD86FFF9@amazon.com> Message-ID: <0c787ea18bf24ef9baec5fbb0b0dfc18@jd.com> Thanks Paul for looking into this! Hi All, Can I get more review about this: Webrev: http://cr.openjdk.java.net/~jcbeyler/8215228/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8215228 Thanks. BRs, Lin From: Hohensee, Paul [mailto:hohensee at amazon.com] Sent: Wednesday, December 12, 2018 10:08 PM To: ?? ; JC Beyler Cc: serviceability-dev at openjdk.java.net Subject: Re: optimize KlassInfoTable size to power of 2 Lgtm, JC. Thanks, Paul From: serviceability-dev > on behalf of ?? > Date: Tuesday, December 11, 2018 at 6:31 PM To: JC Beyler > Cc: "serviceability-dev at openjdk.java.net" > Subject: RE: optimize KlassInfoTable size to power of 2 Dear JC, That?s exactly modification I made ? Thanks a lot for making the patch! BRs, Lin From: JC Beyler [mailto:jcbeyler at google.com] Sent: Wednesday, December 12, 2018 2:27 AM To: ?? > Cc: Andrew Haley >; serviceability-dev at openjdk.java.net Subject: Re: optimize KlassInfoTable size to power of 2 Hi all, I was playing with this yesterday because I was curious to see the different code generation patterns so I had the webrev ready to go: Webrev: http://cr.openjdk.java.net/~jcbeyler/8215228/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8215228 @Lin: check but I'd imagine this is what you had originally done but you also changed the size Thanks, Jc On Tue, Dec 11, 2018 at 3:44 AM ?? > wrote: Oh, to make it clear, when testing with 65536, I made the change to use hash & 65535 instead of hash % 65536. BRs, Lin -----Original Message----- From: ?? Sent: Tuesday, December 11, 2018 7:36 PM To: 'Andrew Haley' >; serviceability-dev at openjdk.java.net Subject: RE: optimize KlassInfoTable size to power of 2 >There is something that I do not understand. You said that GCC didn't >know that _num_buckets was constant. In that case, how did GCC know not to use a divide instruction when you tried 65536? The original code doesn't use _num_buckets directly when calculating the hash idx, it use a non-constant variable _size, a member of KlassInfoTable, which is assigned to _num_buckets. I guess it is the reason GCC didn't do the constant propagation in this case. And _size is not changed during the life of KlassInfoTable, it only set to zero at de-construction. In my experiment, I use _num_buckets directly instead of _size to calculate hash idx. So GCC provides the optimizations. I think your suggestion of using 65537 is valuable, And I agree that it is not easy to prove that non-prime for hash is better. So how about make the change to use constant _number_buckets directly for calculating hash idx, and leave the prime not changed. In my case, speed up JMap histo helps at avoid getting killed by a timer, especially when heap is large at ~200GB. Do you think it is reasonable to have a patch? Thanks, Lin -----Original Message----- From: Andrew Haley [mailto:aph at redhat.com] Sent: Tuesday, December 11, 2018 7:10 PM To: ?? >; serviceability-dev at openjdk.java.net Subject: Re: optimize KlassInfoTable size to power of 2 On 12/11/18 11:05 AM, Andrew Haley wrote: > unsigned mod_m(unsigned n) { > unsigned tmp = n % 65536; > tmp -= n / 65536; > if (tmp >= 65537) // overflow > tmp += 65537; > return tmp; > } NB: this assumes that unsigned int is uint64_t. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From jini.george at oracle.com Thu Dec 13 03:14:09 2018 From: jini.george at oracle.com (Jini George) Date: Thu, 13 Dec 2018 08:44:09 +0530 Subject: RFR: JDK-8202884: SA: Attach/detach might fail on Linux if debugee application create/destroy threads during attaching In-Reply-To: References: Message-ID: <817e84e9-7a57-0000-97ed-85fae11b5093@oracle.com> Thank you very much for looking into this, Yasumasa! The 'pid' used in process_doesnt_exist() is actually the lwpid of the thread to be attached to. From Pgrab(), we call ptrace_attach() first for the pid at line 448. In which case, we end up calling process_doesnt_exist() through ptrace_attach() with the pid. In this case, we completely error out if the 'pid' doesn't exist. Then we go on to discover the lwpids of this process through libpthread_db or by going through the /proc//task/ in case of the process running in a container, and we then invoke ptrace_attach() again on all these newly discovered lwpids at line 503. (we have already attached to the main thread (where the pid and the lwpid are the same). This time when process_doesnt_exist() gets called inside ptrace_attach(), we are dealing with the lwpids. And we would not error out if the thread is missing or is in an 'exiting' state when we try to attach. From the proc man page, /proc//task//* and /proc//* files would have the same content for the same lwpid. ============= < Man Page Snippet > ================================ /proc/[pid]/task (since Linux 2.6.0-test6) This is a directory that contains one subdirectory for each thread in the process. The name of each subdirectory is the numerical thread ID ([tid]) of the thread (see gettid(2)). Within each of these subdirectories, there is a set of files with the same names and contents as under the /proc/[pid] directories. ============= < Man Page Snippet End> ============================= Let me know if you are not Ok with this. Going forward, we should remove the libpthread_db dependency for the threads discovery and only depend on /proc//task/s for this (https://bugs.openjdk.java.net/browse/JDK-8181313). Thank you, Jini. On 12/13/2018 6:15 AM, Yasumasa Suenaga wrote: > Hi Jini, > > I have a comment for your webrev.02 . > > > You added process_doesnt_exist() to check process / thread liveness from > /proc/, but it is not enough. > Information of threads (LWP) will be stored in /proc//task/. > So you should check /proc//task/status for threads. > > > Thanks, > > Yasumasa > > > On 2018/12/12 21:15, Jini George wrote: >> Thank you very much for looking into this, JC! >> >> I have a revised webrev addressing your comments at: >> >> http://cr.openjdk.java.net/~jgeorge/8202884/webrev.02/index.html >> >> Requesting one more review for this. My comments inline: >> >> On 12/12/2018 2:53 AM, JC Beyler wrote: >>> Hi Jini, >>> >>> I saw a few nits: >>> http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/src/jdk.hotspot.agent/linux/native/libsaproc/libproc_impl.h.udiff.html >>> >>> ? ? -> The comments are in the third person normally it seems so it >>> would >>> become (I also removed the s from threads): >>> >>> +// deletes a thread from the thread list >> Done. >>> >>> http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/src/jdk.hotspot.agent/linux/native/libsaproc/libproc_impl.c.udiff.html >>> >>> ? ? -> You added two empty lines it seems that could be removed >> Done. >>> >>> >>> http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/src/jdk.hotspot.agent/linux/native/libsaproc/ps_proc.c.udiff.html >>> >>> ? ? -> Is there a real reason to have both enums? We could have a single >>> enum it seems and not lose too much >> >> You are right. I have done away with the WAITPID* enum. >> >>> ? ? -> you have a switch " >>> ? ? ? ? ?switch (errno) {" >>> ? ? ? ? ? -> Where really you could simplify the reading by moving the >>> EINTR case outside with its continue >>> ? ? ? ? ? -> The switch could then remain as it was (though you move >>> print_debug to print_error) >>> ? ? ? ? ? -> and just return in each cases >> I have changed this to: >> >> 206???? } else { >> 207?????? switch (errno) { >> 208???????? case EINTR: >> 209?????????? continue; >> 210?????????? break; >> 211???????? case ECHILD: >> 212?????????? print_debug("waitpid() failed. Child process pid (%d) does >> not exist \n", pid); >> 213?????????? return ATTACH_THREAD_DEAD; >> 214???????? case EINVAL: >> 215?????????? print_error("waitpid() failed. Invalid options >> argument.\n"); >> 216?????????? return ATTACH_FAIL; >> 217???????? default: >> 218?????????? print_error("waitpid() failed. Unexpected error %d\n", >> errno); >> 219?????????? return ATTACH_FAIL; >> 220?????? } >> 221???? } // else >> >> >>> >>> ? ? ?->?if (strncmp (buf, "State:", 6) == 0) { >>> ? ? ? ? -> You use sizeof("State:") right below; perhaps you could just >>> use "? const char const state[] = "State:";" and use sizeof(state) and >>> for the string, it seems less error prone >>> >>> ? ? -> A minor "bug" is here: >>> +? ? ? state = buf + sizeof ("State:"); >>> ? ? ? ? ? -> You did a strncmp above but that only assures the start of >>> the string is "State:", technically the character after the ':' is the >>> but it could only be that; sizeof("State:") is 7 and not 6. So you miss >>> one character when you are skipping spaces >>> ? ? ? ? ? -> It was probably ok because you always had at least one >>> space, ie "State: " >> >> Thanks! I have made some changes here to use a const char string and a >> variable to store the calculated length using strlen(). And I am using >> isspace() now to skip spaces since tabs could also be used as a >> delimiter. >> >>> ? ? -> Extra space here before the '(': "sizeof (buf)" >> Done. >>> >>> Finally your return sequence for that method could be simplified to: >>> >>> +? if (!found_state) { >>> +? ? print_error(" Could not find the State: string in the status file >>> for pid %d\n", pid); >>> +? } >>> +? fclose (fp); >>> +? return !found_state; >> >> I have modified this to: >> >> 257?? if (!found_state) { >> 258???? // Assuming the thread exists. >> 259???? print_error("Could not find the 'State:' string in the >> /proc/%d/status file\n", pid); >> 260?? } >> 261?? fclose (fp); >> 262?? return false; >> >> Thank you, >> Jini. >> >>> >>> Thanks! >>> Jc >>> >>> On Tue, Dec 11, 2018 at 9:30 AM Jini George >> > wrote: >>> >>> ???? Hello ! >>> >>> ???? Requesting reviews for: >>> >>> ???? https://bugs.openjdk.java.net/browse/JDK-8202884 >>> ???? Webrev: >>> http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/index.html >>> >>> ???? Details: >>> ???? For attaching to the threads in a process, we first go ahead and >>> do a >>> ???? ptrace attach to the main thread. Later, we use the libthread_db >>> ???? library >>> ???? (or, in the case of being within a container, iterate through the >>> ???? /proc//task files) to discover the threads of the process, >>> and add >>> ???? them to the threads list (within SA) for this process. Once, we >>> have >>> ???? discovered all the threads and added these to the list of >>> threads, we >>> ???? then invoke ptrace attach individually on all these threads to >>> ???? attach to >>> ???? these. When we deal with an application where the threads are >>> exiting >>> ???? continuously, some of these threads might not exist by the time >>> we try >>> ???? to ptrace attach to these threads. The proposed fix includes the >>> ???? following modifications to solve this. >>> ????? ? 1. Check the state of the threads in the thread_db callback >>> routine, >>> ???? and skip if the state of the thread is TD_THR_UNKNOWN or >>> TD_THR_ZOMBIE. >>> ???? SA does not try to ptrace attach to these threads and does not >>> include >>> ???? these threads in the threads list. >>> ????? ? 2. While ptrace attaching to the thread, if >>> ptrace(PTRACE_ATTACH, >>> ???? ...) >>> ???? fails with either ESCRH or EPERM, check the state of the thread by >>> ???? checking if the /proc//status file corresponding to that >>> thread >>> ???? exists and if so, reading in the 'State:' line of that file. Skip >>> ???? attaching to this thread and delete this thread from the SA list of >>> ???? threads, if the thread is dead (State: X) or is a zombie (State: >>> Z). >>> ????? ?From the /proc man page, "Current state of the process. One of "R >>> ???? (running)", "S (sleeping)", "D (disk sleep)", "T (stopped)", "T >>> ???? (tracing >>> ???? stop)", "Z (zombie)", or "X (dead)"." >>> ????? ? 3. If waitpid() on the thread is a failure, again skip this >>> thread >>> ???? (delete this from SA's list of threads) instead of bailing out >>> if the >>> ???? thread has exited or terminated. >>> >>> ???? Testing: >>> ???? 1. Tested by attaching and detaching multiple times to a test >>> program >>> ???? spawning numerous short lived threads. >>> ???? 2. The SA tests (under test/hotspot/jtreg/serviceability/sa) passed >>> ???? with >>> ???? 100 repeats on Mach5. >>> ???? 3. No new failures and no occurrences of JDK-8202884 seen with >>> testing >>> ???? the SA tests (tiers 1 to 5) on Mach5. >>> >>> ???? More details in the bug comments section. >>> >>> ???? Thank you, >>> ???? Jini. >>> >>> >>> >>> -- >>> >>> Thanks, >>> Jc >> From jini.george at oracle.com Thu Dec 13 03:41:22 2018 From: jini.george at oracle.com (Jini George) Date: Thu, 13 Dec 2018 09:11:22 +0530 Subject: RFR: JDK-8202884: SA: Attach/detach might fail on Linux if debugee application create/destroy threads during attaching In-Reply-To: References: Message-ID: <6539e8f8-1a78-b039-406d-1fe7c3f2e65f@oracle.com> Thanks, JC! Will add the comments. Thanks, - Jini. On 12/13/2018 12:09 AM, JC Beyler wrote: > Hi Jini, > > Fair enough, thanks for the explanation. It makes sense to me. I imagine > that it is a conservative approach, ie can't find the information then > assume still live. > > Perhaps a small comment above the 'X'/'Z' test saying that the threads > are considered to be dead then? > And for the return perhaps say: the thread is considered live if the > state was not 'X'/'Z' or not found? > > But those are really nits and no need to send another webrev for me! > > Thanks! > Jc > > > > On Wed, Dec 12, 2018 at 10:27 AM Jini George > wrote: > > > On 12/12/2018 10:47 PM, JC Beyler wrote: > > Hi Jini, > > > > Should your return not be return !found_state instead of false here: > > > > 257? ?if (!found_state) { > > 258? ? ?// Assuming the thread exists. > > 259? ? ?print_error("Could not find the 'State:' string in the > > /proc/%d/status file\n", pid); > > 260? ?} > > 261? ?fclose (fp); > > 262? ?return false; > > > > In your webrev.00 it was the case but now, you always return the > process > > does exist even if you have not found it. > > I referred to the gdb sources to check what is done under this > scenario, > and in gdb, it is assumed that if the line beginning with 'State:' is > not found, the thread is alive. But to be frank, I don't know under > what > circumstances will we ever encounter such a scenario. Let me know if > you > don't agree with this. > > > cf: > > > http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/src/jdk.hotspot.agent/linux/native/libsaproc/ps_proc.c.udiff.html > > vs > > > http://cr.openjdk.java.net/~jgeorge/8202884/webrev.02/src/jdk.hotspot.agent/linux/native/libsaproc/ps_proc.c.udiff.html > > > > Tiny nit: no need to check *space if you are using > isspace(*space) right > > after :) > > Will change this. Thanks! > > > > > Apart from the return question, the webrev looks good to me :-) > > Jc > > Thank you again! > Jini > > > > > > > On Wed, Dec 12, 2018 at 4:15 AM Jini George > > > >> > wrote: > > > >? ? ?Thank you very much for looking into this, JC! > > > >? ? ?I have a revised webrev addressing your comments at: > > > > http://cr.openjdk.java.net/~jgeorge/8202884/webrev.02/index.html > > > >? ? ?Requesting one more review for this. My comments inline: > > > >? ? ?On 12/12/2018 2:53 AM, JC Beyler wrote: > >? ? ? > Hi Jini, > >? ? ? > > >? ? ? > I saw a few nits: > >? ? ? > > > > http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/src/jdk.hotspot.agent/linux/native/libsaproc/libproc_impl.h.udiff.html > >? ? ? >? ? -> The comments are in the third person normally it > seems so > >? ? ?it would > >? ? ? > become (I also removed the s from threads): > >? ? ? > > >? ? ? > +// deletes a thread from the thread list > >? ? ?Done. > >? ? ? > > >? ? ? > > > > http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/src/jdk.hotspot.agent/linux/native/libsaproc/libproc_impl.c.udiff.html > >? ? ? >? ? -> You added two empty lines it seems that could be removed > >? ? ?Done. > >? ? ? > > >? ? ? > > >? ? ? > > > > http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/src/jdk.hotspot.agent/linux/native/libsaproc/ps_proc.c.udiff.html > >? ? ? >? ? -> Is there a real reason to have both enums? We could > have a > >? ? ?single > >? ? ? > enum it seems and not lose too much > > > >? ? ?You are right. I have done away with the WAITPID* enum. > > > >? ? ? >? ? -> you have a switch " > >? ? ? >? ? ? ? ?switch (errno) {" > >? ? ? >? ? ? ? ? -> Where really you could simplify the reading by > moving > >? ? ?the > >? ? ? > EINTR case outside with its continue > >? ? ? >? ? ? ? ? -> The switch could then remain as it was (though > you move > >? ? ? > print_debug to print_error) > >? ? ? >? ? ? ? ? -> and just return in each cases > >? ? ?I have changed this to: > > > >? ? ?206? ? ?} else { > >? ? ?207? ? ? ?switch (errno) { > >? ? ?208? ? ? ? ?case EINTR: > >? ? ?209? ? ? ? ? ?continue; > >? ? ?210? ? ? ? ? ?break; > >? ? ?211? ? ? ? ?case ECHILD: > >? ? ?212? ? ? ? ? ?print_debug("waitpid() failed. Child process > pid (%d) > >? ? ?does > >? ? ?not exist \n", pid); > >? ? ?213? ? ? ? ? ?return ATTACH_THREAD_DEAD; > >? ? ?214? ? ? ? ?case EINVAL: > >? ? ?215? ? ? ? ? ?print_error("waitpid() failed. Invalid options > >? ? ?argument.\n"); > >? ? ?216? ? ? ? ? ?return ATTACH_FAIL; > >? ? ?217? ? ? ? ?default: > >? ? ?218? ? ? ? ? ?print_error("waitpid() failed. Unexpected error > %d\n", > >? ? ?errno); > >? ? ?219? ? ? ? ? ?return ATTACH_FAIL; > >? ? ?220? ? ? ?} > >? ? ?221? ? ?} // else > > > > > >? ? ? > > >? ? ? >? ? ?->?if (strncmp (buf, "State:", 6) == 0) { > >? ? ? >? ? ? ? -> You use sizeof("State:") right below; perhaps > you could > >? ? ?just > >? ? ? > use "? const char const state[] = "State:";" and use > >? ? ?sizeof(state) and > >? ? ? > for the string, it seems less error prone > >? ? ? > > >? ? ? >? ? -> A minor "bug" is here: > >? ? ? > +? ? ? state = buf + sizeof ("State:"); > >? ? ? >? ? ? ? ? -> You did a strncmp above but that only assures the > >? ? ?start of > >? ? ? > the string is "State:", technically the character after > the ':' > >? ? ?is the > >? ? ? > but it could only be that; sizeof("State:") is 7 and not 6. So > >? ? ?you miss > >? ? ? > one character when you are skipping spaces > >? ? ? >? ? ? ? ? -> It was probably ok because you always had at > least one > >? ? ? > space, ie "State: " > > > >? ? ?Thanks! I have made some changes here to use a const char > string and a > >? ? ?variable to store the calculated length using strlen(). And I > am using > >? ? ?isspace() now to skip spaces since tabs could also be used as a > >? ? ?delimiter. > > > >? ? ? >? ? -> Extra space here before the '(': "sizeof (buf)" > >? ? ?Done. > >? ? ? > > >? ? ? > Finally your return sequence for that method could be > simplified to: > >? ? ? > > >? ? ? > +? if (!found_state) { > >? ? ? > +? ? print_error(" Could not find the State: string in the > status > >? ? ?file > >? ? ? > for pid %d\n", pid); > >? ? ? > +? } > >? ? ? > +? fclose (fp); > >? ? ? > +? return !found_state; > > > >? ? ?I have modified this to: > > > >? ? ?257? ?if (!found_state) { > >? ? ?258? ? ?// Assuming the thread exists. > >? ? ?259? ? ?print_error("Could not find the 'State:' string in the > >? ? ?/proc/%d/status file\n", pid); > >? ? ?260? ?} > >? ? ?261? ?fclose (fp); > >? ? ?262? ?return false; > > > >? ? ?Thank you, > >? ? ?Jini. > > > >? ? ? > > >? ? ? > Thanks! > >? ? ? > Jc > >? ? ? > > >? ? ? > On Tue, Dec 11, 2018 at 9:30 AM Jini George > >? ? ? > > > >? ? ? > >>> > >? ? ?wrote: > >? ? ? > > >? ? ? >? ? ?Hello ! > >? ? ? > > >? ? ? >? ? ?Requesting reviews for: > >? ? ? > > >? ? ? > https://bugs.openjdk.java.net/browse/JDK-8202884 > >? ? ? >? ? ?Webrev: > > http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/index.html > >? ? ? > > >? ? ? >? ? ?Details: > >? ? ? >? ? ?For attaching to the threads in a process, we first go > ahead > >? ? ?and do a > >? ? ? >? ? ?ptrace attach to the main thread. Later, we use the > libthread_db > >? ? ? >? ? ?library > >? ? ? >? ? ?(or, in the case of being within a container, iterate > through the > >? ? ? >? ? ?/proc//task files) to discover the threads of the > >? ? ?process, and add > >? ? ? >? ? ?them to the threads list (within SA) for this process. > Once, > >? ? ?we have > >? ? ? >? ? ?discovered all the threads and added these to the list of > >? ? ?threads, we > >? ? ? >? ? ?then invoke ptrace attach individually on all these > threads to > >? ? ? >? ? ?attach to > >? ? ? >? ? ?these. When we deal with an application where the > threads are > >? ? ?exiting > >? ? ? >? ? ?continuously, some of these threads might not exist by the > >? ? ?time we try > >? ? ? >? ? ?to ptrace attach to these threads. The proposed fix > includes the > >? ? ? >? ? ?following modifications to solve this. > >? ? ? >? ? ? ? 1. Check the state of the threads in the thread_db > >? ? ?callback routine, > >? ? ? >? ? ?and skip if the state of the thread is TD_THR_UNKNOWN or > >? ? ?TD_THR_ZOMBIE. > >? ? ? >? ? ?SA does not try to ptrace attach to these threads and does > >? ? ?not include > >? ? ? >? ? ?these threads in the threads list. > >? ? ? >? ? ? ? 2. While ptrace attaching to the thread, if > >? ? ?ptrace(PTRACE_ATTACH, > >? ? ? >? ? ?...) > >? ? ? >? ? ?fails with either ESCRH or EPERM, check the state of the > >? ? ?thread by > >? ? ? >? ? ?checking if the /proc//status file corresponding > to that > >? ? ?thread > >? ? ? >? ? ?exists and if so, reading in the 'State:' line of that > file. Skip > >? ? ? >? ? ?attaching to this thread and delete this thread from > the SA > >? ? ?list of > >? ? ? >? ? ?threads, if the thread is dead (State: X) or is a zombie > >? ? ?(State: Z). > >? ? ? >? ? ? ?From the /proc man page, "Current state of the > process. One > >? ? ?of "R > >? ? ? >? ? ?(running)", "S (sleeping)", "D (disk sleep)", "T > (stopped)", "T > >? ? ? >? ? ?(tracing > >? ? ? >? ? ?stop)", "Z (zombie)", or "X (dead)"." > >? ? ? >? ? ? ? 3. If waitpid() on the thread is a failure, again skip > >? ? ?this thread > >? ? ? >? ? ?(delete this from SA's list of threads) instead of bailing > >? ? ?out if the > >? ? ? >? ? ?thread has exited or terminated. > >? ? ? > > >? ? ? >? ? ?Testing: > >? ? ? >? ? ?1. Tested by attaching and detaching multiple times to > a test > >? ? ?program > >? ? ? >? ? ?spawning numerous short lived threads. > >? ? ? >? ? ?2. The SA tests (under > test/hotspot/jtreg/serviceability/sa) > >? ? ?passed > >? ? ? >? ? ?with > >? ? ? >? ? ?100 repeats on Mach5. > >? ? ? >? ? ?3. No new failures and no occurrences of JDK-8202884 seen > >? ? ?with testing > >? ? ? >? ? ?the SA tests (tiers 1 to 5) on Mach5. > >? ? ? > > >? ? ? >? ? ?More details in the bug comments section. > >? ? ? > > >? ? ? >? ? ?Thank you, > >? ? ? >? ? ?Jini. > >? ? ? > > >? ? ? > > >? ? ? > > >? ? ? > -- > >? ? ? > > >? ? ? > Thanks, > >? ? ? > Jc > > > > > > > > -- > > > > Thanks, > > Jc > > > > -- > > Thanks, > Jc From jcbeyler at google.com Thu Dec 13 04:44:57 2018 From: jcbeyler at google.com (JC Beyler) Date: Wed, 12 Dec 2018 20:44:57 -0800 Subject: RFR (XS) 8215329: Modify ZGC requirement for HeapMonitorThreadTest.java Message-ID: Hi all, When working on another webrev, I saw this problem: Webrev: http://cr.openjdk.java.net/~jcbeyler/8215329/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8215329 (Basically, from what I understood from an email from Per Liden: - @requires !vm.gc.Z -> ZGC is built in the JDK - @requires vm.gc != "Z" -> ZGC is being used for the runtime ) Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From yasuenag at gmail.com Thu Dec 13 05:05:16 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Thu, 13 Dec 2018 14:05:16 +0900 Subject: RFR: JDK-8202884: SA: Attach/detach might fail on Linux if debugee application create/destroy threads during attaching In-Reply-To: <817e84e9-7a57-0000-97ed-85fae11b5093@oracle.com> References: <817e84e9-7a57-0000-97ed-85fae11b5093@oracle.com> Message-ID: Hi Jini, 2018?12?13?(?) 12:14 Jini George : > > Thank you very much for looking into this, Yasumasa! > > The 'pid' used in process_doesnt_exist() is actually the lwpid of the > thread to be attached to. > > From Pgrab(), we call ptrace_attach() first for the pid at line 448. In > which case, we end up calling process_doesnt_exist() through > ptrace_attach() with the pid. In this case, we completely error out if > the 'pid' doesn't exist. Then we go on to discover the lwpids of this > process through libpthread_db or by going through the > /proc//task/ in case of the process running in a container, > and we then invoke ptrace_attach() again on all these newly discovered > lwpids at line 503. (we have already attached to the main thread (where > the pid and the lwpid are the same). This time when > process_doesnt_exist() gets called inside ptrace_attach(), we are > dealing with the lwpids. And we would not error out if the thread is > missing or is in an 'exiting' state when we try to attach. Ok, it seems good. I think zombie thread(s) will not exist in thread_info list at this point because they will be removed at thread_db_callback(). > From the proc man page, /proc//task//* and > /proc//* files would have the same content for > the same lwpid. > > ============= < Man Page Snippet > ================================ > /proc/[pid]/task (since Linux 2.6.0-test6) > This is a directory that contains one subdirectory for > each thread in the process. The name of each subdirectory is the > numerical thread ID ([tid]) of the thread (see gettid(2)). Within each > of these subdirectories, there is a set of files with the same names and > contents as under the /proc/[pid] directories. > ============= < Man Page Snippet End> ============================= > > Let me know if you are not Ok with this. > > Going forward, we should remove the libpthread_db dependency for the > threads discovery and only depend on /proc//task/s for this > (https://bugs.openjdk.java.net/browse/JDK-8181313). It's great news! I will help you :-) Thanks, Yasumasa > Thank you, > Jini. > > On 12/13/2018 6:15 AM, Yasumasa Suenaga wrote: > > Hi Jini, > > > > I have a comment for your webrev.02 . > > > > > > You added process_doesnt_exist() to check process / thread liveness from > > /proc/, but it is not enough. > > Information of threads (LWP) will be stored in /proc//task/. > > So you should check /proc//task/status for threads. > > > > > > Thanks, > > > > Yasumasa > > > > > > On 2018/12/12 21:15, Jini George wrote: > >> Thank you very much for looking into this, JC! > >> > >> I have a revised webrev addressing your comments at: > >> > >> http://cr.openjdk.java.net/~jgeorge/8202884/webrev.02/index.html > >> > >> Requesting one more review for this. My comments inline: > >> > >> On 12/12/2018 2:53 AM, JC Beyler wrote: > >>> Hi Jini, > >>> > >>> I saw a few nits: > >>> http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/src/jdk.hotspot.agent/linux/native/libsaproc/libproc_impl.h.udiff.html > >>> > >>> ? -> The comments are in the third person normally it seems so it > >>> would > >>> become (I also removed the s from threads): > >>> > >>> +// deletes a thread from the thread list > >> Done. > >>> > >>> http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/src/jdk.hotspot.agent/linux/native/libsaproc/libproc_impl.c.udiff.html > >>> > >>> ? -> You added two empty lines it seems that could be removed > >> Done. > >>> > >>> > >>> http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/src/jdk.hotspot.agent/linux/native/libsaproc/ps_proc.c.udiff.html > >>> > >>> ? -> Is there a real reason to have both enums? We could have a single > >>> enum it seems and not lose too much > >> > >> You are right. I have done away with the WAITPID* enum. > >> > >>> ? -> you have a switch " > >>> ? ? ? ?switch (errno) {" > >>> ? ? ? ? -> Where really you could simplify the reading by moving the > >>> EINTR case outside with its continue > >>> ? ? ? ? -> The switch could then remain as it was (though you move > >>> print_debug to print_error) > >>> ? ? ? ? -> and just return in each cases > >> I have changed this to: > >> > >> 206 } else { > >> 207 switch (errno) { > >> 208 case EINTR: > >> 209 continue; > >> 210 break; > >> 211 case ECHILD: > >> 212 print_debug("waitpid() failed. Child process pid (%d) does > >> not exist \n", pid); > >> 213 return ATTACH_THREAD_DEAD; > >> 214 case EINVAL: > >> 215 print_error("waitpid() failed. Invalid options > >> argument.\n"); > >> 216 return ATTACH_FAIL; > >> 217 default: > >> 218 print_error("waitpid() failed. Unexpected error %d\n", > >> errno); > >> 219 return ATTACH_FAIL; > >> 220 } > >> 221 } // else > >> > >> > >>> > >>> ? ?->?if (strncmp (buf, "State:", 6) == 0) { > >>> ? ? ? -> You use sizeof("State:") right below; perhaps you could just > >>> use "? const char const state[] = "State:";" and use sizeof(state) and > >>> for the string, it seems less error prone > >>> > >>> ? -> A minor "bug" is here: > >>> +? ? ? state = buf + sizeof ("State:"); > >>> ? ? ? ? -> You did a strncmp above but that only assures the start of > >>> the string is "State:", technically the character after the ':' is the > >>> but it could only be that; sizeof("State:") is 7 and not 6. So you miss > >>> one character when you are skipping spaces > >>> ? ? ? ? -> It was probably ok because you always had at least one > >>> space, ie "State: " > >> > >> Thanks! I have made some changes here to use a const char string and a > >> variable to store the calculated length using strlen(). And I am using > >> isspace() now to skip spaces since tabs could also be used as a > >> delimiter. > >> > >>> ? -> Extra space here before the '(': "sizeof (buf)" > >> Done. > >>> > >>> Finally your return sequence for that method could be simplified to: > >>> > >>> +? if (!found_state) { > >>> +? ? print_error(" Could not find the State: string in the status file > >>> for pid %d\n", pid); > >>> +? } > >>> +? fclose (fp); > >>> +? return !found_state; > >> > >> I have modified this to: > >> > >> 257 if (!found_state) { > >> 258 // Assuming the thread exists. > >> 259 print_error("Could not find the 'State:' string in the > >> /proc/%d/status file\n", pid); > >> 260 } > >> 261 fclose (fp); > >> 262 return false; > >> > >> Thank you, > >> Jini. > >> > >>> > >>> Thanks! > >>> Jc > >>> > >>> On Tue, Dec 11, 2018 at 9:30 AM Jini George >>> > wrote: > >>> > >>> Hello ! > >>> > >>> Requesting reviews for: > >>> > >>> https://bugs.openjdk.java.net/browse/JDK-8202884 > >>> Webrev: > >>> http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/index.html > >>> > >>> Details: > >>> For attaching to the threads in a process, we first go ahead and > >>> do a > >>> ptrace attach to the main thread. Later, we use the libthread_db > >>> library > >>> (or, in the case of being within a container, iterate through the > >>> /proc//task files) to discover the threads of the process, > >>> and add > >>> them to the threads list (within SA) for this process. Once, we > >>> have > >>> discovered all the threads and added these to the list of > >>> threads, we > >>> then invoke ptrace attach individually on all these threads to > >>> attach to > >>> these. When we deal with an application where the threads are > >>> exiting > >>> continuously, some of these threads might not exist by the time > >>> we try > >>> to ptrace attach to these threads. The proposed fix includes the > >>> following modifications to solve this. > >>> ? 1. Check the state of the threads in the thread_db callback > >>> routine, > >>> and skip if the state of the thread is TD_THR_UNKNOWN or > >>> TD_THR_ZOMBIE. > >>> SA does not try to ptrace attach to these threads and does not > >>> include > >>> these threads in the threads list. > >>> ? 2. While ptrace attaching to the thread, if > >>> ptrace(PTRACE_ATTACH, > >>> ...) > >>> fails with either ESCRH or EPERM, check the state of the thread by > >>> checking if the /proc//status file corresponding to that > >>> thread > >>> exists and if so, reading in the 'State:' line of that file. Skip > >>> attaching to this thread and delete this thread from the SA list of > >>> threads, if the thread is dead (State: X) or is a zombie (State: > >>> Z). > >>> ?From the /proc man page, "Current state of the process. One of "R > >>> (running)", "S (sleeping)", "D (disk sleep)", "T (stopped)", "T > >>> (tracing > >>> stop)", "Z (zombie)", or "X (dead)"." > >>> ? 3. If waitpid() on the thread is a failure, again skip this > >>> thread > >>> (delete this from SA's list of threads) instead of bailing out > >>> if the > >>> thread has exited or terminated. > >>> > >>> Testing: > >>> 1. Tested by attaching and detaching multiple times to a test > >>> program > >>> spawning numerous short lived threads. > >>> 2. The SA tests (under test/hotspot/jtreg/serviceability/sa) passed > >>> with > >>> 100 repeats on Mach5. > >>> 3. No new failures and no occurrences of JDK-8202884 seen with > >>> testing > >>> the SA tests (tiers 1 to 5) on Mach5. > >>> > >>> More details in the bug comments section. > >>> > >>> Thank you, > >>> Jini. > >>> > >>> > >>> > >>> -- > >>> > >>> Thanks, > >>> Jc > >> From yasuenag at gmail.com Thu Dec 13 05:12:31 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Thu, 13 Dec 2018 14:12:31 +0900 Subject: RFR: JDK-8202884: SA: Attach/detach might fail on Linux if debugee application create/destroy threads during attaching In-Reply-To: References: <817e84e9-7a57-0000-97ed-85fae11b5093@oracle.com> Message-ID: Your webrev.02 looks good to me! Yasumasa 2018?12?13?(?) 14:05 Yasumasa Suenaga : > > Hi Jini, > > 2018?12?13?(?) 12:14 Jini George : > > > > Thank you very much for looking into this, Yasumasa! > > > > The 'pid' used in process_doesnt_exist() is actually the lwpid of the > > thread to be attached to. > > > > From Pgrab(), we call ptrace_attach() first for the pid at line 448. In > > which case, we end up calling process_doesnt_exist() through > > ptrace_attach() with the pid. In this case, we completely error out if > > the 'pid' doesn't exist. Then we go on to discover the lwpids of this > > process through libpthread_db or by going through the > > /proc//task/ in case of the process running in a container, > > and we then invoke ptrace_attach() again on all these newly discovered > > lwpids at line 503. (we have already attached to the main thread (where > > the pid and the lwpid are the same). This time when > > process_doesnt_exist() gets called inside ptrace_attach(), we are > > dealing with the lwpids. And we would not error out if the thread is > > missing or is in an 'exiting' state when we try to attach. > > Ok, it seems good. > > I think zombie thread(s) will not exist in thread_info list at this point > because they will be removed at thread_db_callback(). > > > > From the proc man page, /proc//task//* and > > /proc//* files would have the same content for > > the same lwpid. > > > > ============= < Man Page Snippet > ================================ > > /proc/[pid]/task (since Linux 2.6.0-test6) > > This is a directory that contains one subdirectory for > > each thread in the process. The name of each subdirectory is the > > numerical thread ID ([tid]) of the thread (see gettid(2)). Within each > > of these subdirectories, there is a set of files with the same names and > > contents as under the /proc/[pid] directories. > > ============= < Man Page Snippet End> ============================= > > > > Let me know if you are not Ok with this. > > > > Going forward, we should remove the libpthread_db dependency for the > > threads discovery and only depend on /proc//task/s for this > > (https://bugs.openjdk.java.net/browse/JDK-8181313). > > It's great news! > I will help you :-) > > > Thanks, > > Yasumasa > > > > Thank you, > > Jini. > > > > On 12/13/2018 6:15 AM, Yasumasa Suenaga wrote: > > > Hi Jini, > > > > > > I have a comment for your webrev.02 . > > > > > > > > > You added process_doesnt_exist() to check process / thread liveness from > > > /proc/, but it is not enough. > > > Information of threads (LWP) will be stored in /proc//task/. > > > So you should check /proc//task/status for threads. > > > > > > > > > Thanks, > > > > > > Yasumasa > > > > > > > > > On 2018/12/12 21:15, Jini George wrote: > > >> Thank you very much for looking into this, JC! > > >> > > >> I have a revised webrev addressing your comments at: > > >> > > >> http://cr.openjdk.java.net/~jgeorge/8202884/webrev.02/index.html > > >> > > >> Requesting one more review for this. My comments inline: > > >> > > >> On 12/12/2018 2:53 AM, JC Beyler wrote: > > >>> Hi Jini, > > >>> > > >>> I saw a few nits: > > >>> http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/src/jdk.hotspot.agent/linux/native/libsaproc/libproc_impl.h.udiff.html > > >>> > > >>> ? -> The comments are in the third person normally it seems so it > > >>> would > > >>> become (I also removed the s from threads): > > >>> > > >>> +// deletes a thread from the thread list > > >> Done. > > >>> > > >>> http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/src/jdk.hotspot.agent/linux/native/libsaproc/libproc_impl.c.udiff.html > > >>> > > >>> ? -> You added two empty lines it seems that could be removed > > >> Done. > > >>> > > >>> > > >>> http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/src/jdk.hotspot.agent/linux/native/libsaproc/ps_proc.c.udiff.html > > >>> > > >>> ? -> Is there a real reason to have both enums? We could have a single > > >>> enum it seems and not lose too much > > >> > > >> You are right. I have done away with the WAITPID* enum. > > >> > > >>> ? -> you have a switch " > > >>> ? ? ? ?switch (errno) {" > > >>> ? ? ? ? -> Where really you could simplify the reading by moving the > > >>> EINTR case outside with its continue > > >>> ? ? ? ? -> The switch could then remain as it was (though you move > > >>> print_debug to print_error) > > >>> ? ? ? ? -> and just return in each cases > > >> I have changed this to: > > >> > > >> 206 } else { > > >> 207 switch (errno) { > > >> 208 case EINTR: > > >> 209 continue; > > >> 210 break; > > >> 211 case ECHILD: > > >> 212 print_debug("waitpid() failed. Child process pid (%d) does > > >> not exist \n", pid); > > >> 213 return ATTACH_THREAD_DEAD; > > >> 214 case EINVAL: > > >> 215 print_error("waitpid() failed. Invalid options > > >> argument.\n"); > > >> 216 return ATTACH_FAIL; > > >> 217 default: > > >> 218 print_error("waitpid() failed. Unexpected error %d\n", > > >> errno); > > >> 219 return ATTACH_FAIL; > > >> 220 } > > >> 221 } // else > > >> > > >> > > >>> > > >>> ? ?->?if (strncmp (buf, "State:", 6) == 0) { > > >>> ? ? ? -> You use sizeof("State:") right below; perhaps you could just > > >>> use "? const char const state[] = "State:";" and use sizeof(state) and > > >>> for the string, it seems less error prone > > >>> > > >>> ? -> A minor "bug" is here: > > >>> +? ? ? state = buf + sizeof ("State:"); > > >>> ? ? ? ? -> You did a strncmp above but that only assures the start of > > >>> the string is "State:", technically the character after the ':' is the > > >>> but it could only be that; sizeof("State:") is 7 and not 6. So you miss > > >>> one character when you are skipping spaces > > >>> ? ? ? ? -> It was probably ok because you always had at least one > > >>> space, ie "State: " > > >> > > >> Thanks! I have made some changes here to use a const char string and a > > >> variable to store the calculated length using strlen(). And I am using > > >> isspace() now to skip spaces since tabs could also be used as a > > >> delimiter. > > >> > > >>> ? -> Extra space here before the '(': "sizeof (buf)" > > >> Done. > > >>> > > >>> Finally your return sequence for that method could be simplified to: > > >>> > > >>> +? if (!found_state) { > > >>> +? ? print_error(" Could not find the State: string in the status file > > >>> for pid %d\n", pid); > > >>> +? } > > >>> +? fclose (fp); > > >>> +? return !found_state; > > >> > > >> I have modified this to: > > >> > > >> 257 if (!found_state) { > > >> 258 // Assuming the thread exists. > > >> 259 print_error("Could not find the 'State:' string in the > > >> /proc/%d/status file\n", pid); > > >> 260 } > > >> 261 fclose (fp); > > >> 262 return false; > > >> > > >> Thank you, > > >> Jini. > > >> > > >>> > > >>> Thanks! > > >>> Jc > > >>> > > >>> On Tue, Dec 11, 2018 at 9:30 AM Jini George > >>> > wrote: > > >>> > > >>> Hello ! > > >>> > > >>> Requesting reviews for: > > >>> > > >>> https://bugs.openjdk.java.net/browse/JDK-8202884 > > >>> Webrev: > > >>> http://cr.openjdk.java.net/~jgeorge/8202884/webrev.00/index.html > > >>> > > >>> Details: > > >>> For attaching to the threads in a process, we first go ahead and > > >>> do a > > >>> ptrace attach to the main thread. Later, we use the libthread_db > > >>> library > > >>> (or, in the case of being within a container, iterate through the > > >>> /proc//task files) to discover the threads of the process, > > >>> and add > > >>> them to the threads list (within SA) for this process. Once, we > > >>> have > > >>> discovered all the threads and added these to the list of > > >>> threads, we > > >>> then invoke ptrace attach individually on all these threads to > > >>> attach to > > >>> these. When we deal with an application where the threads are > > >>> exiting > > >>> continuously, some of these threads might not exist by the time > > >>> we try > > >>> to ptrace attach to these threads. The proposed fix includes the > > >>> following modifications to solve this. > > >>> ? 1. Check the state of the threads in the thread_db callback > > >>> routine, > > >>> and skip if the state of the thread is TD_THR_UNKNOWN or > > >>> TD_THR_ZOMBIE. > > >>> SA does not try to ptrace attach to these threads and does not > > >>> include > > >>> these threads in the threads list. > > >>> ? 2. While ptrace attaching to the thread, if > > >>> ptrace(PTRACE_ATTACH, > > >>> ...) > > >>> fails with either ESCRH or EPERM, check the state of the thread by > > >>> checking if the /proc//status file corresponding to that > > >>> thread > > >>> exists and if so, reading in the 'State:' line of that file. Skip > > >>> attaching to this thread and delete this thread from the SA list of > > >>> threads, if the thread is dead (State: X) or is a zombie (State: > > >>> Z). > > >>> ?From the /proc man page, "Current state of the process. One of "R > > >>> (running)", "S (sleeping)", "D (disk sleep)", "T (stopped)", "T > > >>> (tracing > > >>> stop)", "Z (zombie)", or "X (dead)"." > > >>> ? 3. If waitpid() on the thread is a failure, again skip this > > >>> thread > > >>> (delete this from SA's list of threads) instead of bailing out > > >>> if the > > >>> thread has exited or terminated. > > >>> > > >>> Testing: > > >>> 1. Tested by attaching and detaching multiple times to a test > > >>> program > > >>> spawning numerous short lived threads. > > >>> 2. The SA tests (under test/hotspot/jtreg/serviceability/sa) passed > > >>> with > > >>> 100 repeats on Mach5. > > >>> 3. No new failures and no occurrences of JDK-8202884 seen with > > >>> testing > > >>> the SA tests (tiers 1 to 5) on Mach5. > > >>> > > >>> More details in the bug comments section. > > >>> > > >>> Thank you, > > >>> Jini. > > >>> > > >>> > > >>> > > >>> -- > > >>> > > >>> Thanks, > > >>> Jc > > >> From sharath.ballal at oracle.com Thu Dec 13 06:41:04 2018 From: sharath.ballal at oracle.com (Sharath Ballal) Date: Wed, 12 Dec 2018 22:41:04 -0800 (PST) Subject: RFR: JDK-8211923- [Testbug] serviceability/sa/ClhsdbFindPC.java ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1 Message-ID: Hi, Pls review the test fix in serviceability/sa/ClhsdbFindPC.java Jira: https://bugs.openjdk.java.net/browse/JDK-8211923 Webrev: http://cr.openjdk.java.net/~sballal/8211923/webrev.00/ I have done the following modifications to the testcase: 1. If jstack output does not contain "LingeredApp.main", fail the test. 2. The test is testing both -Xcomp and -Xint options. I have changed the order of the test and made -Xint first. Currently the failure is in -Xcomp. Changing the order will tell us (in future failures) if the interpretted mode passed or not. SA tests passed in Mach5 runs. Thanks, Sharath -------------- next part -------------- An HTML attachment was scrubbed... URL: From jini.george at oracle.com Thu Dec 13 07:36:44 2018 From: jini.george at oracle.com (Jini George) Date: Thu, 13 Dec 2018 13:06:44 +0530 Subject: RFR: JDK-8202884: SA: Attach/detach might fail on Linux if debugee application create/destroy threads during attaching In-Reply-To: References: <817e84e9-7a57-0000-97ed-85fae11b5093@oracle.com> Message-ID: <7886c60e-6b12-4db4-16d4-2e3128f53253@oracle.com> > >> >> Going forward, we should remove the libpthread_db dependency for the >> threads discovery and only depend on /proc//task/s for this >> (https://bugs.openjdk.java.net/browse/JDK-8181313). > > It's great news! > I will help you :-) > Sure -- and thank you for the review! -Jini. From goetz.lindenmaier at sap.com Thu Dec 13 07:50:43 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 13 Dec 2018 07:50:43 +0000 Subject: 8215534: [testbug] some jfr test don't check @requires vm.hasJFR Message-ID: Hi, These tests lack @requires vm.hasJFR, thus they are failing on AIX. http://cr.openjdk.java.net/~goetz/wr18/8215334-JFR_requires/01/ Please review. I will push this to jdk12 as it is a testbug if I miss the RDP deadline. Best regards, Goetz. From sundararajan.athijegannathan at oracle.com Thu Dec 13 08:46:36 2018 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Thu, 13 Dec 2018 14:16:36 +0530 Subject: 8215534: [testbug] some jfr test don't check @requires vm.hasJFR In-Reply-To: References: Message-ID: <5C121C6C.7010204@oracle.com> Looks good. PS. I just checked that there are other tests with the same requires clause. -Sundar On 13/12/18, 1:20 PM, Lindenmaier, Goetz wrote: > Hi, > > These tests lack @requires vm.hasJFR, thus they are failing on AIX. > http://cr.openjdk.java.net/~goetz/wr18/8215334-JFR_requires/01/ > > Please review. > I will push this to jdk12 as it is a testbug if I miss the RDP deadline. > > Best regards, > Goetz. > From goetz.lindenmaier at sap.com Thu Dec 13 08:38:49 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 13 Dec 2018 08:38:49 +0000 Subject: 8215534: [testbug] some jfr test don't check @requires vm.hasJFR In-Reply-To: <5C121C6C.7010204@oracle.com> References: <5C121C6C.7010204@oracle.com> Message-ID: <6d1b7054a36a47379aa2c5c8515bfc02@sap.com> Thanks Sundar! Best regards Goetz > -----Original Message----- > From: Sundararajan Athijegannathan > > Sent: Thursday, December 13, 2018 9:47 AM > To: Lindenmaier, Goetz > Cc: security-dev at openjdk.java.net; serviceability-dev (serviceability- > dev at openjdk.java.net) > Subject: Re: 8215534: [testbug] some jfr test don't check @requires > vm.hasJFR > > Looks good. > > PS. I just checked that there are other tests with the same requires clause. > > -Sundar > > On 13/12/18, 1:20 PM, Lindenmaier, Goetz wrote: > > Hi, > > > > These tests lack @requires vm.hasJFR, thus they are failing on AIX. > > http://cr.openjdk.java.net/~goetz/wr18/8215334-JFR_requires/01/ > > > > Please review. > > I will push this to jdk12 as it is a testbug if I miss the RDP deadline. > > > > Best regards, > > Goetz. > > From per.liden at oracle.com Thu Dec 13 09:31:35 2018 From: per.liden at oracle.com (Per Liden) Date: Thu, 13 Dec 2018 10:31:35 +0100 Subject: RFR (XS) 8215329: Modify ZGC requirement for HeapMonitorThreadTest.java In-Reply-To: References: Message-ID: <53d8feb7-1616-c5fb-7973-5a22d7eef713@oracle.com> Hi JC, What's the reason to exclude ZGC from this test to begin with? From what I can tell, it's because the test is using a slightly too small heap, or are there some other reason? I ran it a few times using various heap sizes and the test passes with ZGC when using anything above 612M. So if we instead just dump the heap size a bit, then we get test coverage with ZGC too. I picked 768M here to have some headroom in case the exact limit is run-to-run dependent. diff --git a/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorThreadTest.java b/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorThreadTest.java --- a/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorThreadTest.java +++ b/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorThreadTest.java @@ -29,8 +29,7 @@ * @build Frame HeapMonitor ThreadInformation * @summary Verifies the JVMTI Heap Monitor Thread information sanity. * @compile HeapMonitorThreadTest.java - * @run main/othervm/native -Xmx512m -agentlib:HeapMonitorTest MyPackage.HeapMonitorThreadTest - * @requires !vm.gc.Z + * @run main/othervm/native -Xmx768m -agentlib:HeapMonitorTest MyPackage.HeapMonitorThreadTest */ import java.util.List; cheers, Per On 12/13/2018 05:44 AM, JC Beyler wrote: > Hi all, > > When working on another webrev, I saw this problem: > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8215329/webrev.00/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8215329 > > (Basically, from what I understood from an email from Per Liden: > - @requires !vm.gc.Z -> ZGC is built in the JDK > - @requires vm.gc != "Z" -> ZGC is being used for the runtime > ) > > Thanks, > Jc From aph at redhat.com Thu Dec 13 09:57:05 2018 From: aph at redhat.com (Andrew Haley) Date: Thu, 13 Dec 2018 09:57:05 +0000 Subject: optimize KlassInfoTable size to power of 2 In-Reply-To: <0c787ea18bf24ef9baec5fbb0b0dfc18@jd.com> References: <140bac18565c48be88313f7cc1d24bf8@jd.com> <7fc37fd7cf24439eb3a6fff132fc7387@jd.com> <4ea1c92f-4e72-7d38-8c7a-91e2892e0c3b@redhat.com> <1ab2d28b40024ba6b299756612fa3760@jd.com> <7a1e5d87-94fd-9be9-8aab-5dd0f1a3fff9@redhat.com> <56b9099e33584796af8a508980ff4690@jd.com> <436fddb22d3b4fb9aec315f3150cff41@jd.com> <0045EBD6-F21D-457B-865E-3796BD86FFF9@amazon.com> <0c787ea18bf24ef9baec5fbb0b0dfc18@jd.com> Message-ID: On 12/13/18 1:37 AM, ?? wrote: > Can I get more review about this: > Webrev: http://cr.openjdk.java.net/~jcbeyler/8215228/webrev.00/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8215228 It's fine. Please change the commit comment to something like "Use a constant hash table size in order to enable compiler optimization." -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From per.liden at oracle.com Thu Dec 13 11:21:55 2018 From: per.liden at oracle.com (Per Liden) Date: Thu, 13 Dec 2018 12:21:55 +0100 Subject: RFR (XS) 8215329: Modify ZGC requirement for HeapMonitorThreadTest.java In-Reply-To: <53d8feb7-1616-c5fb-7973-5a22d7eef713@oracle.com> References: <53d8feb7-1616-c5fb-7973-5a22d7eef713@oracle.com> Message-ID: Hi again, I ran this test some more and managed to get an OOME even with a 768M heap. I'm getting a bit suspicious that something else is wrong here. Let me dig into this some more and see if I can understand what the real issue is. cheers, Per On 12/13/2018 10:31 AM, Per Liden wrote: > Hi JC, > > What's the reason to exclude ZGC from this test to begin with? From what > I can tell, it's because the test is using a slightly too small heap, or > are there some other reason? I ran it a few times using various heap > sizes and the test passes with ZGC when using anything above 612M. So if > we instead just dump the heap size a bit, then we get test coverage with > ZGC too. I picked 768M here to have some headroom in case the exact > limit is run-to-run dependent. > > diff --git > a/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorThreadTest.java > b/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorThreadTest.java > > --- > a/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorThreadTest.java > > +++ > b/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorThreadTest.java > > @@ -29,8 +29,7 @@ > * @build Frame HeapMonitor ThreadInformation > * @summary Verifies the JVMTI Heap Monitor Thread information sanity. > * @compile HeapMonitorThreadTest.java > - * @run main/othervm/native -Xmx512m -agentlib:HeapMonitorTest > MyPackage.HeapMonitorThreadTest > - * @requires !vm.gc.Z > + * @run main/othervm/native -Xmx768m -agentlib:HeapMonitorTest > MyPackage.HeapMonitorThreadTest > */ > > import java.util.List; > > cheers, > Per > > On 12/13/2018 05:44 AM, JC Beyler wrote: >> Hi all, >> >> When working on another webrev, I saw this problem: >> >> Webrev: http://cr.openjdk.java.net/~jcbeyler/8215329/webrev.00/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8215329 >> >> (Basically, from what I understood from an email from Per Liden: >> - @requires !vm.gc.Z -> ZGC is built in the JDK >> - @requires vm.gc != "Z" -> ZGC is being used for the runtime >> ) >> >> Thanks, >> Jc From erik.gahlin at oracle.com Thu Dec 13 11:34:07 2018 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Thu, 13 Dec 2018 12:34:07 +0100 Subject: 8215534: [testbug] some jfr test don't check @requires vm.hasJFR In-Reply-To: References: Message-ID: <5C1243AF.6010100@oracle.com> Looks good. Erik > Hi, > > These tests lack @requires vm.hasJFR, thus they are failing on AIX. > http://cr.openjdk.java.net/~goetz/wr18/8215334-JFR_requires/01/ > > Please review. > I will push this to jdk12 as it is a testbug if I miss the RDP deadline. > > Best regards, > Goetz. > From goetz.lindenmaier at sap.com Thu Dec 13 11:57:37 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 13 Dec 2018 11:57:37 +0000 Subject: 8215534: [testbug] some jfr test don't check @requires vm.hasJFR In-Reply-To: <5C1243AF.6010100@oracle.com> References: <5C1243AF.6010100@oracle.com> Message-ID: <3072253043ce4b218da8bf1b2173acea@sap.com> Thanks Erik! Pushed. Best regards, Goetz > -----Original Message----- > From: serviceability-dev On > Behalf Of Erik Gahlin > Sent: Thursday, December 13, 2018 12:34 PM > To: security-dev at openjdk.java.net; serviceability-dev (serviceability- > dev at openjdk.java.net) > Subject: Re: 8215534: [testbug] some jfr test don't check @requires > vm.hasJFR > > Looks good. > > Erik > > > Hi, > > > > These tests lack @requires vm.hasJFR, thus they are failing on AIX. > > http://cr.openjdk.java.net/~goetz/wr18/8215334-JFR_requires/01/ > > > > Please review. > > I will push this to jdk12 as it is a testbug if I miss the RDP deadline. > > > > Best regards, > > Goetz. > > From per.liden at oracle.com Thu Dec 13 12:47:17 2018 From: per.liden at oracle.com (Per Liden) Date: Thu, 13 Dec 2018 13:47:17 +0100 Subject: RFR (XS) 8215329: Modify ZGC requirement for HeapMonitorThreadTest.java In-Reply-To: References: <53d8feb7-1616-c5fb-7973-5a22d7eef713@oracle.com> Message-ID: <36e27b1d-defb-549e-84c1-cbc40d3cd70a@oracle.com> Hi, me again ;) I think I've found the root cause of this. There's a tiny race in the ZGC allocation path, which can lead to pre-mature OOME being thrown. It's not trivial to fix, so I suggest you go ahead with your original patch (Looks good btw), and I'll file a separate bug to fix the ZGC issue (and update this test to run with ZGC again). cheers, Per On 12/13/2018 12:21 PM, Per Liden wrote: > Hi again, > > I ran this test some more and managed to get an OOME even with a 768M > heap. I'm getting a bit suspicious that something else is wrong here. > Let me dig into this some more and see if I can understand what the real > issue is. > > cheers, > Per > > On 12/13/2018 10:31 AM, Per Liden wrote: >> Hi JC, >> >> What's the reason to exclude ZGC from this test to begin with? From >> what I can tell, it's because the test is using a slightly too small >> heap, or are there some other reason? I ran it a few times using >> various heap sizes and the test passes with ZGC when using anything >> above 612M. So if we instead just dump the heap size a bit, then we >> get test coverage with ZGC too. I picked 768M here to have some >> headroom in case the exact limit is run-to-run dependent. >> >> diff --git >> a/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorThreadTest.java >> b/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorThreadTest.java >> >> --- >> a/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorThreadTest.java >> >> +++ >> b/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorThreadTest.java >> >> @@ -29,8 +29,7 @@ >> * @build Frame HeapMonitor ThreadInformation >> * @summary Verifies the JVMTI Heap Monitor Thread information sanity. >> * @compile HeapMonitorThreadTest.java >> - * @run main/othervm/native -Xmx512m -agentlib:HeapMonitorTest >> MyPackage.HeapMonitorThreadTest >> - * @requires !vm.gc.Z >> + * @run main/othervm/native -Xmx768m -agentlib:HeapMonitorTest >> MyPackage.HeapMonitorThreadTest >> */ >> >> import java.util.List; >> >> cheers, >> Per >> >> On 12/13/2018 05:44 AM, JC Beyler wrote: >>> Hi all, >>> >>> When working on another webrev, I saw this problem: >>> >>> Webrev: http://cr.openjdk.java.net/~jcbeyler/8215329/webrev.00/ >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8215329 >>> >>> (Basically, from what I understood from an email from Per Liden: >>> - @requires !vm.gc.Z -> ZGC is built in the JDK >>> - @requires vm.gc != "Z" -> ZGC is being used for the runtime >>> ) >>> >>> Thanks, >>> Jc From gary.adams at oracle.com Thu Dec 13 13:25:16 2018 From: gary.adams at oracle.com (Gary Adams) Date: Thu, 13 Dec 2018 08:25:16 -0500 Subject: RFR: JDK-8051349: nsk/jvmti/scenarios/sampling/SP06/sp06t003 fails in nightly In-Reply-To: <5C113D4D.7070503@oracle.com> References: <5C113D4D.7070503@oracle.com> Message-ID: <5C125DBC.6090307@oracle.com> While testing I ran into another of the related failures that were associated with the original bug. The following fake exception stacktrace is for failure analysis. nsk.share.Fake_Exception_for_RULE_Creation: (sp02t003.cpp:313) jvmti->GetFrameLocation(threadsDesc[i].thread, j,&qMethod,&qLocation) at nsk_lvcomplain(nsk_tools.cpp:172) # ERROR: sp02t003.cpp, 313: jvmti->GetFrameLocation(threadsDesc[i].thread, j,&qMethod,&qLocation) # jvmti error: code=31, name=JVMTI_ERROR_NO_MORE_FRAMES - sp02t003.cpp, 310: 3 frame: method: 0x7f225042b2d8, location: 16 # ERROR: sp02t003.cpp, 313: jvmti->GetFrameLocation(threadsDesc[i].thread, j,&qMethod,&qLocation) # jvmti error: code=31, name=JVMTI_ERROR_NO_MORE_FRAMES # ERROR: sp02t003.cpp, 350: No expected method frame for not suspended thread #4 (threadRunningInterrupted) In this particular failure, the GetFrameLocation call failed, because the frame was no longer accessible. If the threads are not suspended, the GetFrameLocation may fail, or if it passes, the qMethod and qLocation could belong to another frame. I've updated the webrev to allow the call to GetFrameLocation, but to only report an error if the threads are suspended. Similarly, the checks to compare qMethod and qLocation will be skipped, if the threads are not suspended. And the final comparison that the method was found will only be an error if the threads are suspended. Webrev: http://cr.openjdk.java.net/~gadams/8051349/webrev.01/ On 12/12/18, 11:54 AM, Gary Adams wrote: > After some testing with nsk verbose messages enabled, > it is clear that this test is failing in checkThreads when the > location did not match between the call to GetStackTrace > and GetFrameLocation. For the tests that are run when the threads > have not been suspended, there is no guarantee the locations > will match. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8051349 > Webrev: http://cr.openjdk.java.net/~gadams/8051349/webrev.00/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kevin.walls at oracle.com Thu Dec 13 14:01:45 2018 From: kevin.walls at oracle.com (Kevin Walls) Date: Thu, 13 Dec 2018 14:01:45 +0000 Subject: [8u] RFR: 8059038: Create new launcher for SA tools In-Reply-To: <975dca9cb3fdea8f2a799c52f7d3e5edd9d79306.camel@redhat.com> References: <975dca9cb3fdea8f2a799c52f7d3e5edd9d79306.camel@redhat.com> Message-ID: Hi Severin, Looks good to me. This could have merited a distinct bug ID, I think you could argue either way on that, they are clearly strongly related - in 9, this bug ID creates a new launcher for all SA tools, here in 8 we're creating the missing launchers, within the existing 8u launcher system. Thanks Kevin On 12/12/2018 17:28, Severin Gehwolf wrote: > On Wed, 2018-12-12 at 16:18 +0000, Andrew Hughes wrote: >> On Wed, 12 Dec 2018 at 14:10, Severin Gehwolf wrote: >>> Hi, >>> >>> Can I get a review of this small 8u enhancement, please? It adds two >>> new launchers for the serviceability agent, one CLI version and one GUI >>> version: >>> >>> $ /bin/clhsdb >>> $ /bin/hsdb >>> >>> The enhancement request has been approved here: >>> http://mail.openjdk.java.net/pipermail/jdk8u-dev/2018-December/008257.html >>> >>> During that discussion it has been suggested to use separate launchers >>> for GUI and CLI. So this is the revised two-launcher-approach: >>> webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8059038/02/ >>> bug: https://bugs.openjdk.java.net/browse/JDK-8059038 >>> >>> Note: The initial version of this patch[1] had one launcher "jhsdb" >>> with sub-commands "hsdb" and "clhsdb" delegating to the old launcher >>> classes. >>> >>> The patch has two simple tests verifying that the launchers work. >>> Thoughts? >>> >>> Thanks, >>> Severin >>> >>> [1] http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8059038/01/ >>> >> The patch itself looks ok. But it looks completely different from the >> changesets in 9: >> >> https://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/bf17c0a1c746 >> https://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/24a8cbde76d8 >> >> Can you explain a little what's going on here? > Sure. The original patch for 9 added a new class, SALauncher, which > launches many of the original stand-alone commands from 8. jstack from > JDK 8, for example, became "jhsdb jstack" in 9+ via this class[I]. > > On the JDK 8 enhancement request approval thread a suggestion was made > for the backport to JDK 8 to mimic the stand-alone launcher approach > JDK 8 uses: "jstack" over "jhsdb jstack". So "clhsdb" (JDK 8) will be a > separate launcher over "jhsdb clhsdb" in JDK 9+. > > The first webrev[1] above, mimic'ed JDK 9+, but not quite since it > didn't change the arguments and options. jhsdb clhsdb --pid (JDK > 9+) vs. jhsdb clhsdb (in [1]). > > That's basically the reason why the JDK 8 backport looks this > different. Of course we can also go with the original version[1], but > so far I've heard two +1's for the second version... Hence, I'm > proposing that version. > > Thanks, > Severin > > [I] https://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/24a8cbde76d8#l2.213 > From gnu.andrew at redhat.com Thu Dec 13 15:49:03 2018 From: gnu.andrew at redhat.com (Andrew Hughes) Date: Thu, 13 Dec 2018 15:49:03 +0000 Subject: [8u] RFR: 8059038: Create new launcher for SA tools In-Reply-To: <975dca9cb3fdea8f2a799c52f7d3e5edd9d79306.camel@redhat.com> References: <975dca9cb3fdea8f2a799c52f7d3e5edd9d79306.camel@redhat.com> Message-ID: On Wed, 12 Dec 2018 at 17:28, Severin Gehwolf wrote: > > On Wed, 2018-12-12 at 16:18 +0000, Andrew Hughes wrote: > > On Wed, 12 Dec 2018 at 14:10, Severin Gehwolf wrote: > > > > > > Hi, > > > > > > Can I get a review of this small 8u enhancement, please? It adds two > > > new launchers for the serviceability agent, one CLI version and one GUI > > > version: > > > > > > $ /bin/clhsdb > > > $ /bin/hsdb > > > > > > The enhancement request has been approved here: > > > http://mail.openjdk.java.net/pipermail/jdk8u-dev/2018-December/008257.html > > > > > > During that discussion it has been suggested to use separate launchers > > > for GUI and CLI. So this is the revised two-launcher-approach: > > > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8059038/02/ > > > bug: https://bugs.openjdk.java.net/browse/JDK-8059038 > > > > > > Note: The initial version of this patch[1] had one launcher "jhsdb" > > > with sub-commands "hsdb" and "clhsdb" delegating to the old launcher > > > classes. > > > > > > The patch has two simple tests verifying that the launchers work. > > > Thoughts? > > > > > > Thanks, > > > Severin > > > > > > [1] http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8059038/01/ > > > > > > > The patch itself looks ok. But it looks completely different from the > > changesets in 9: > > > > https://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/bf17c0a1c746 > > https://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/24a8cbde76d8 > > > > Can you explain a little what's going on here? > > Sure. The original patch for 9 added a new class, SALauncher, which > launches many of the original stand-alone commands from 8. jstack from > JDK 8, for example, became "jhsdb jstack" in 9+ via this class[I]. > > On the JDK 8 enhancement request approval thread a suggestion was made > for the backport to JDK 8 to mimic the stand-alone launcher approach > JDK 8 uses: "jstack" over "jhsdb jstack". So "clhsdb" (JDK 8) will be a > separate launcher over "jhsdb clhsdb" in JDK 9+. > > The first webrev[1] above, mimic'ed JDK 9+, but not quite since it > didn't change the arguments and options. jhsdb clhsdb --pid (JDK > 9+) vs. jhsdb clhsdb (in [1]). > > That's basically the reason why the JDK 8 backport looks this > different. Of course we can also go with the original version[1], but > so far I've heard two +1's for the second version... Hence, I'm > proposing that version. > > Thanks, > Severin > > [I] https://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/24a8cbde76d8#l2.213 > Ok, I presume that's not a change you're also going to make in OpenJDK 12? I'm fine with them being linked by the same bug ID, as they resolve the same underlying issue, but this really needs to be clear in the summary text, as, to the uninformed, they look like completely different patches. -- Andrew :) Senior Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) Web Site: http://fuseyism.com Twitter: https://twitter.com/gnu_andrew_java PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 From chris.plummer at oracle.com Thu Dec 13 16:51:11 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 13 Dec 2018 08:51:11 -0800 Subject: RFR: JDK-8051349: nsk/jvmti/scenarios/sampling/SP06/sp06t003 fails in nightly In-Reply-To: <5C125DBC.6090307@oracle.com> References: <5C113D4D.7070503@oracle.com> <5C125DBC.6090307@oracle.com> Message-ID: <196f4645-b082-65f8-ff2d-0434a03a34ca@oracle.com> An HTML attachment was scrubbed... URL: From jcbeyler at google.com Thu Dec 13 16:59:57 2018 From: jcbeyler at google.com (JC Beyler) Date: Thu, 13 Dec 2018 08:59:57 -0800 Subject: RFR (XS) 8215329: Modify ZGC requirement for HeapMonitorThreadTest.java In-Reply-To: <36e27b1d-defb-549e-84c1-cbc40d3cd70a@oracle.com> References: <53d8feb7-1616-c5fb-7973-5a22d7eef713@oracle.com> <36e27b1d-defb-549e-84c1-cbc40d3cd70a@oracle.com> Message-ID: Hi Per, Thanks for the messages and review :-). I believe that actually what happened was that when JDK11 was close to release both ZGC and HeapMonitoring tried to get in. In a last effort, we turned off this test for ZGC as it was showing test failures for ZGC. It's a bit fuzzy to be honest (I was also on paternity leave and Serguei & Jeremy were helping out here). But anyway, what seems to be true now is that you found a bug (yeah I guess?) and we can remove the @requires once you fix your race. Therefore, could I please get another review to update the @requires to be correct then until then? (@Per: if you want, I can update the test once it's done; either assign a JBS bug to me or send me an email) Thanks, Jc On Thu, Dec 13, 2018 at 4:47 AM Per Liden wrote: > Hi, me again ;) > > I think I've found the root cause of this. There's a tiny race in the > ZGC allocation path, which can lead to pre-mature OOME being thrown. > It's not trivial to fix, so I suggest you go ahead with your original > patch (Looks good btw), and I'll file a separate bug to fix the ZGC > issue (and update this test to run with ZGC again). > > cheers, > Per > > On 12/13/2018 12:21 PM, Per Liden wrote: > > Hi again, > > > > I ran this test some more and managed to get an OOME even with a 768M > > heap. I'm getting a bit suspicious that something else is wrong here. > > Let me dig into this some more and see if I can understand what the real > > issue is. > > > > cheers, > > Per > > > > On 12/13/2018 10:31 AM, Per Liden wrote: > >> Hi JC, > >> > >> What's the reason to exclude ZGC from this test to begin with? From > >> what I can tell, it's because the test is using a slightly too small > >> heap, or are there some other reason? I ran it a few times using > >> various heap sizes and the test passes with ZGC when using anything > >> above 612M. So if we instead just dump the heap size a bit, then we > >> get test coverage with ZGC too. I picked 768M here to have some > >> headroom in case the exact limit is run-to-run dependent. > >> > >> diff --git > >> > a/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorThreadTest.java > > >> > b/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorThreadTest.java > > >> > >> --- > >> > a/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorThreadTest.java > > >> > >> +++ > >> > b/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorThreadTest.java > > >> > >> @@ -29,8 +29,7 @@ > >> * @build Frame HeapMonitor ThreadInformation > >> * @summary Verifies the JVMTI Heap Monitor Thread information sanity. > >> * @compile HeapMonitorThreadTest.java > >> - * @run main/othervm/native -Xmx512m -agentlib:HeapMonitorTest > >> MyPackage.HeapMonitorThreadTest > >> - * @requires !vm.gc.Z > >> + * @run main/othervm/native -Xmx768m -agentlib:HeapMonitorTest > >> MyPackage.HeapMonitorThreadTest > >> */ > >> > >> import java.util.List; > >> > >> cheers, > >> Per > >> > >> On 12/13/2018 05:44 AM, JC Beyler wrote: > >>> Hi all, > >>> > >>> When working on another webrev, I saw this problem: > >>> > >>> Webrev: http://cr.openjdk.java.net/~jcbeyler/8215329/webrev.00/ > >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8215329 > >>> > >>> (Basically, from what I understood from an email from Per Liden: > >>> - @requires !vm.gc.Z -> ZGC is built in the JDK > >>> - @requires vm.gc != "Z" -> ZGC is being used for the runtime > >>> ) > >>> > >>> Thanks, > >>> Jc > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From jcbeyler at google.com Thu Dec 13 17:11:28 2018 From: jcbeyler at google.com (JC Beyler) Date: Thu, 13 Dec 2018 09:11:28 -0800 Subject: RFR: JDK-8211923- [Testbug] serviceability/sa/ClhsdbFindPC.java ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1 In-Reply-To: References: Message-ID: Hi Sharath, It looks like this bug is a potential duplicate of : https://bugs.openjdk.java.net/browse/JDK-8196969 and https://bugs.openjdk.java.net/browse/JDK-8145741 Should we not wait until the underlying bug is fixed before fixing this? Although I believe that those bugs would still show up with your fix here and maybe getting the interpreter first would be good information to have. Since the test runs the application and tries to attach to it, I don't see anything wrong with trying to modify the order of execution though and getting a good messaged runtime exception instead of a more obscure index out of bounds is a step in the right direction in my opinion. So LGTM, Jc On Wed, Dec 12, 2018 at 10:43 PM Sharath Ballal wrote: > Hi, > > Pls review the test fix in serviceability/sa/ClhsdbFindPC.java > > Jira: https://bugs.openjdk.java.net/browse/JDK-8211923 > > Webrev: http://cr.openjdk.java.net/~sballal/8211923/webrev.00/ > > > > I have done the following modifications to the testcase: > 1. If jstack output does not contain "LingeredApp.main", fail the test. > 2. The test is testing both -Xcomp and -Xint options. I have changed the > order of the test and made -Xint first. Currently the failure is in -Xcomp. > Changing the order will tell us (in future failures) if the interpretted > mode passed or not. > > > > SA tests passed in Mach5 runs. > > > > Thanks, > > Sharath > > > > > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From sgehwolf at redhat.com Thu Dec 13 17:19:32 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Thu, 13 Dec 2018 18:19:32 +0100 Subject: [8u] RFR: 8059038: Create new launcher for SA tools In-Reply-To: References: <975dca9cb3fdea8f2a799c52f7d3e5edd9d79306.camel@redhat.com> Message-ID: On Thu, 2018-12-13 at 15:49 +0000, Andrew Hughes wrote: > Ok, I presume that's not a change you're also going to make in > OpenJDK 12? Yes. I don't intend to change this in JDK 12. JDK 8 and JDK 9+ are not compatible in this regard. That's the case before and after this patch. > I'm fine with them being linked by the same bug ID, as they resolve the same > underlying issue, but this really needs to be clear in the summary > text, as, to the > uninformed, they look like completely different patches. OK will do. Can I consider this reviewed? Thanks, Severin From alexey.menkov at oracle.com Thu Dec 13 17:40:27 2018 From: alexey.menkov at oracle.com (Alex Menkov) Date: Thu, 13 Dec 2018 09:40:27 -0800 Subject: RFR (XS) 8215329: Modify ZGC requirement for HeapMonitorThreadTest.java In-Reply-To: References: <53d8feb7-1616-c5fb-7973-5a22d7eef713@oracle.com> <36e27b1d-defb-549e-84c1-cbc40d3cd70a@oracle.com> Message-ID: +1 --alex On 12/13/2018 08:59, JC Beyler wrote: > Hi Per, > > Thanks for the messages and review :-). I believe that actually what > happened was that when JDK11 was close to release both ZGC and > HeapMonitoring tried to get in. In a last effort, we turned off this > test for ZGC as it was showing test failures for ZGC. It's a bit fuzzy > to be honest (I was also on paternity leave and Serguei & Jeremy were > helping out here). But anyway, what seems to be true now is that you > found a bug (yeah I guess?) and we can remove the?@requires once you fix > your race. > > Therefore, could I please get another review to update the?@requires to > be correct then until then? > > (@Per: if you want, I can update the test once it's done; either assign > a JBS bug to me or send me an email) > > Thanks, > Jc > > On Thu, Dec 13, 2018 at 4:47 AM Per Liden > wrote: > > Hi, me again ;) > > I think I've found the root cause of this. There's a tiny race in the > ZGC allocation path, which can lead to pre-mature OOME being thrown. > It's not trivial to fix, so I suggest you go ahead with your original > patch (Looks good btw), and I'll file a separate bug to fix the ZGC > issue (and update this test to run with ZGC again). > > cheers, > Per > > On 12/13/2018 12:21 PM, Per Liden wrote: > > Hi again, > > > > I ran this test some more and managed to get an OOME even with a > 768M > > heap. I'm getting a bit suspicious that something else is wrong > here. > > Let me dig into this some more and see if I can understand what > the real > > issue is. > > > > cheers, > > Per > > > > On 12/13/2018 10:31 AM, Per Liden wrote: > >> Hi JC, > >> > >> What's the reason to exclude ZGC from this test to begin with? From > >> what I can tell, it's because the test is using a slightly too > small > >> heap, or are there some other reason? I ran it a few times using > >> various heap sizes and the test passes with ZGC when using anything > >> above 612M. So if we instead just dump the heap size a bit, then we > >> get test coverage with ZGC too. I picked 768M here to have some > >> headroom in case the exact limit is run-to-run dependent. > >> > >> diff --git > >> > a/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorThreadTest.java > > >> > b/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorThreadTest.java > > >> > >> --- > >> > a/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorThreadTest.java > > >> > >> +++ > >> > b/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorThreadTest.java > > >> > >> @@ -29,8 +29,7 @@ > >>? ? * @build Frame HeapMonitor ThreadInformation > >>? ? * @summary Verifies the JVMTI Heap Monitor Thread information > sanity. > >>? ? * @compile HeapMonitorThreadTest.java > >> - * @run main/othervm/native -Xmx512m -agentlib:HeapMonitorTest > >> MyPackage.HeapMonitorThreadTest > >> - * @requires !vm.gc.Z > >> + * @run main/othervm/native -Xmx768m -agentlib:HeapMonitorTest > >> MyPackage.HeapMonitorThreadTest > >>? ? */ > >> > >>? ?import java.util.List; > >> > >> cheers, > >> Per > >> > >> On 12/13/2018 05:44 AM, JC Beyler wrote: > >>> Hi all, > >>> > >>> When working on another webrev, I saw this problem: > >>> > >>> Webrev: http://cr.openjdk.java.net/~jcbeyler/8215329/webrev.00/ > >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8215329 > >>> > >>> (Basically, from what I understood from an email from Per Liden: > >>>? ? ?- @requires !vm.gc.Z -> ZGC is built in the JDK > >>>? ? ?- @requires vm.gc != "Z" -> ZGC is being used for the runtime > >>> ) > >>> > >>> Thanks, > >>> Jc > > > > -- > > Thanks, > Jc From serguei.spitsyn at oracle.com Thu Dec 13 18:25:09 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 13 Dec 2018 10:25:09 -0800 Subject: RFR (XS) 8215329: Modify ZGC requirement for HeapMonitorThreadTest.java In-Reply-To: References: <53d8feb7-1616-c5fb-7973-5a22d7eef713@oracle.com> <36e27b1d-defb-549e-84c1-cbc40d3cd70a@oracle.com> Message-ID: <9abfcd5a-8266-c539-d8fc-79400de23066@oracle.com> +1 Thanks, Serguei On 12/13/18 09:40, Alex Menkov wrote: > +1 > > --alex > > On 12/13/2018 08:59, JC Beyler wrote: >> Hi Per, >> >> Thanks for the messages and review :-). I believe that actually what >> happened was that when JDK11 was close to release both ZGC and >> HeapMonitoring tried to get in. In a last effort, we turned off this >> test for ZGC as it was showing test failures for ZGC. It's a bit >> fuzzy to be honest (I was also on paternity leave and Serguei & >> Jeremy were helping out here). But anyway, what seems to be true now >> is that you found a bug (yeah I guess?) and we can remove >> the?@requires once you fix your race. >> >> Therefore, could I please get another review to update the?@requires >> to be correct then until then? >> >> (@Per: if you want, I can update the test once it's done; either >> assign a JBS bug to me or send me an email) >> >> Thanks, >> Jc >> >> On Thu, Dec 13, 2018 at 4:47 AM Per Liden > > wrote: >> >> ??? Hi, me again ;) >> >> ??? I think I've found the root cause of this. There's a tiny race in >> the >> ??? ZGC allocation path, which can lead to pre-mature OOME being thrown. >> ??? It's not trivial to fix, so I suggest you go ahead with your >> original >> ??? patch (Looks good btw), and I'll file a separate bug to fix the ZGC >> ??? issue (and update this test to run with ZGC again). >> >> ??? cheers, >> ??? Per >> >> ??? On 12/13/2018 12:21 PM, Per Liden wrote: >> ???? > Hi again, >> ???? > >> ???? > I ran this test some more and managed to get an OOME even with a >> ??? 768M >> ???? > heap. I'm getting a bit suspicious that something else is wrong >> ??? here. >> ???? > Let me dig into this some more and see if I can understand what >> ??? the real >> ???? > issue is. >> ???? > >> ???? > cheers, >> ???? > Per >> ???? > >> ???? > On 12/13/2018 10:31 AM, Per Liden wrote: >> ???? >> Hi JC, >> ???? >> >> ???? >> What's the reason to exclude ZGC from this test to begin >> with? From >> ???? >> what I can tell, it's because the test is using a slightly too >> ??? small >> ???? >> heap, or are there some other reason? I ran it a few times using >> ???? >> various heap sizes and the test passes with ZGC when using >> anything >> ???? >> above 612M. So if we instead just dump the heap size a bit, >> then we >> ???? >> get test coverage with ZGC too. I picked 768M here to have some >> ???? >> headroom in case the exact limit is run-to-run dependent. >> ???? >> >> ???? >> diff --git >> ???? >> >> a/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorThreadTest.java >> >> ???? >> >> b/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorThreadTest.java >> >> ???? >> >> ???? >> --- >> ???? >> >> a/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorThreadTest.java >> >> ???? >> >> ???? >> +++ >> ???? >> >> b/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorThreadTest.java >> >> ???? >> >> ???? >> @@ -29,8 +29,7 @@ >> ???? >>? ? * @build Frame HeapMonitor ThreadInformation >> ???? >>? ? * @summary Verifies the JVMTI Heap Monitor Thread information >> ??? sanity. >> ???? >>? ? * @compile HeapMonitorThreadTest.java >> ???? >> - * @run main/othervm/native -Xmx512m -agentlib:HeapMonitorTest >> ???? >> MyPackage.HeapMonitorThreadTest >> ???? >> - * @requires !vm.gc.Z >> ???? >> + * @run main/othervm/native -Xmx768m -agentlib:HeapMonitorTest >> ???? >> MyPackage.HeapMonitorThreadTest >> ???? >>? ? */ >> ???? >> >> ???? >>? ?import java.util.List; >> ???? >> >> ???? >> cheers, >> ???? >> Per >> ???? >> >> ???? >> On 12/13/2018 05:44 AM, JC Beyler wrote: >> ???? >>> Hi all, >> ???? >>> >> ???? >>> When working on another webrev, I saw this problem: >> ???? >>> >> ???? >>> Webrev: http://cr.openjdk.java.net/~jcbeyler/8215329/webrev.00/ >> ???? >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8215329 >> ???? >>> >> ???? >>> (Basically, from what I understood from an email from Per >> Liden: >> ???? >>>? ? ?- @requires !vm.gc.Z -> ZGC is built in the JDK >> ???? >>>? ? ?- @requires vm.gc != "Z" -> ZGC is being used for the >> runtime >> ???? >>> ) >> ???? >>> >> ???? >>> Thanks, >> ???? >>> Jc >> >> >> >> -- >> >> Thanks, >> Jc From jini.george at oracle.com Fri Dec 14 06:41:20 2018 From: jini.george at oracle.com (Jini George) Date: Fri, 14 Dec 2018 12:11:20 +0530 Subject: RFR: JDK-8211923- [Testbug] serviceability/sa/ClhsdbFindPC.java ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1 In-Reply-To: References: Message-ID: <3b94a4d4-f9a7-46d7-d817-ae6cdca93cdf@oracle.com> Hi Sharath, Since this is a duplicate of https://bugs.openjdk.java.net/browse/JDK-8196969, I am not sure if trying to confirm that this occurs only with -Xcomp will help. (I think we are already mostly convinced about that -- as per the comments in JDK-8196969). I guess we should try to fix the actual issue of the getMethod() returning NULL. We could probably have a subtask under this for putting in extra diagnostics in the code to check why getMethod() would return NULL. But if you think we need to do any test case improvements for this, we could -- but probably as a separate bug or a subtask. Thanks, Jini. On 12/13/2018 12:11 PM, Sharath Ballal wrote: > Hi, > > Pls review the test fix in serviceability/sa/ClhsdbFindPC.java > > Jira: https://bugs.openjdk.java.net/browse/JDK-8211923 > > Webrev: http://cr.openjdk.java.net/~sballal/8211923/webrev.00/ > > I have done the following modifications to the testcase: > 1. If jstack output does not contain "LingeredApp.main", fail the test. > 2. The test is testing both -Xcomp and -Xint options. I have changed the > order of the test and made -Xint first. Currently the failure is in > -Xcomp. Changing the order will tell us (in future failures) if the > interpretted mode passed or not. > > SA tests passed in Mach5 runs. > > Thanks, > > Sharath > From jini.george at oracle.com Fri Dec 14 06:47:34 2018 From: jini.george at oracle.com (Jini George) Date: Fri, 14 Dec 2018 12:17:34 +0530 Subject: RFR: JDK-8211923- [Testbug] serviceability/sa/ClhsdbFindPC.java ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1 In-Reply-To: <3b94a4d4-f9a7-46d7-d817-ae6cdca93cdf@oracle.com> References: <3b94a4d4-f9a7-46d7-d817-ae6cdca93cdf@oracle.com> Message-ID: Sorry -- missed adding this. If we are convinced after the extra diagnostics that getMethod() returning NULL is a valid scenario (could SA be attaching at an unstable moment?), maybe we could improve the test case(s) then to handle this situation gracefully. Thank you, Jini. On 12/14/2018 12:11 PM, Jini George wrote: > Hi Sharath, > > Since this is a duplicate of > https://bugs.openjdk.java.net/browse/JDK-8196969, I am not sure if > trying to confirm that this occurs only with -Xcomp will help. (I think > we are already mostly convinced about that -- as per the comments in > JDK-8196969). I guess we should try to fix the actual issue of the > getMethod() returning NULL. We could probably have a subtask under this > for putting in extra diagnostics in the code to check why getMethod() > would return NULL. > > But if you think we need to do any test case improvements for this, we > could -- but probably as a separate bug or a subtask. > > Thanks, > Jini. > > > On 12/13/2018 12:11 PM, Sharath Ballal wrote: >> Hi, >> >> Pls review the test fix in serviceability/sa/ClhsdbFindPC.java >> >> Jira: https://bugs.openjdk.java.net/browse/JDK-8211923 >> >> Webrev: http://cr.openjdk.java.net/~sballal/8211923/webrev.00/ >> >> I have done the following modifications to the testcase: >> 1. If jstack output does not contain "LingeredApp.main", fail the test. >> 2. The test is testing both -Xcomp and -Xint options. I have changed >> the order of the test and made -Xint first. Currently the failure is >> in -Xcomp. Changing the order will tell us (in future failures) if the >> interpretted mode passed or not. >> >> SA tests passed in Mach5 runs. >> >> Thanks, >> >> Sharath >> From sharath.ballal at oracle.com Fri Dec 14 07:09:03 2018 From: sharath.ballal at oracle.com (Sharath Ballal) Date: Thu, 13 Dec 2018 23:09:03 -0800 (PST) Subject: RFR: JDK-8211923- [Testbug] serviceability/sa/ClhsdbFindPC.java ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1 In-Reply-To: References: <3b94a4d4-f9a7-46d7-d817-ae6cdca93cdf@oracle.com> Message-ID: Thanks Jc and Jini for your comments. For now I won't proceed with these changes. Once the other bug is root caused and If there is a need to add changes to the testcase, I will do it as a separate bug. Thanks, Sharath -----Original Message----- From: Jini George Sent: Friday, December 14, 2018 12:18 PM To: Sharath Ballal; serviceability-dev Subject: Re: RFR: JDK-8211923- [Testbug] serviceability/sa/ClhsdbFindPC.java ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1 Sorry -- missed adding this. If we are convinced after the extra diagnostics that getMethod() returning NULL is a valid scenario (could SA be attaching at an unstable moment?), maybe we could improve the test case(s) then to handle this situation gracefully. Thank you, Jini. On 12/14/2018 12:11 PM, Jini George wrote: > Hi Sharath, > > Since this is a duplicate of > https://bugs.openjdk.java.net/browse/JDK-8196969, I am not sure if > trying to confirm that this occurs only with -Xcomp will help. (I > think we are already mostly convinced about that -- as per the > comments in JDK-8196969). I guess we should try to fix the actual > issue of the > getMethod() returning NULL. We could probably have a subtask under > this for putting in extra diagnostics in the code to check why > getMethod() would return NULL. > > But if you think we need to do any test case improvements for this, we > could -- but probably as a separate bug or a subtask. > > Thanks, > Jini. > > > On 12/13/2018 12:11 PM, Sharath Ballal wrote: >> Hi, >> >> Pls review the test fix in serviceability/sa/ClhsdbFindPC.java >> >> Jira: https://bugs.openjdk.java.net/browse/JDK-8211923 >> >> Webrev: http://cr.openjdk.java.net/~sballal/8211923/webrev.00/ >> >> I have done the following modifications to the testcase: >> 1. If jstack output does not contain "LingeredApp.main", fail the test. >> 2. The test is testing both -Xcomp and -Xint options. I have changed >> the order of the test and made -Xint first. Currently the failure is >> in -Xcomp. Changing the order will tell us (in future failures) if >> the interpretted mode passed or not. >> >> SA tests passed in Mach5 runs. >> >> Thanks, >> >> Sharath >> From gary.adams at oracle.com Fri Dec 14 13:28:23 2018 From: gary.adams at oracle.com (Gary Adams) Date: Fri, 14 Dec 2018 08:28:23 -0500 Subject: RFR: JDK-8051349: nsk/jvmti/scenarios/sampling/SP06/sp06t003 fails in nightly In-Reply-To: <196f4645-b082-65f8-ff2d-0434a03a34ca@oracle.com> References: <5C113D4D.7070503@oracle.com> <5C125DBC.6090307@oracle.com> <196f4645-b082-65f8-ff2d-0434a03a34ca@oracle.com> Message-ID: <5C13AFF7.2050201@oracle.com> On 12/13/18, 11:51 AM, Chris Plummer wrote: > Hi Gary, > > Unfortunately GetStackTrace() returns the top frame as the first frame > in the array. Thus if the thread is not suspended and later you call > GetFrameLocation() for some depth, there's no way to make sure the > frame at that depth is the same frame at that depth in the array > returned by GetStackTrace() unless you first suspend the frame before > calling GetFrameLocation(). You would also need to call > GetFrameCount() while suspended, compare that with the number of > frames returned by GetStackTrace(), and make any needed depth > adjustments. But since I think the intent is to not suspend the thread > here, it probably does not make sense to do that, and instead accept > that there might be some errors as you have done. I'm not totally comfortable with these attempted operations and then bypassing the test failure if the threads were not suspended. Would it be better to only perform check thread operations on the suspended threads and to totally skip the operations that can not be safely performed when the thread is not suspended. > > One improvement I would like to see for your fix is to only ignore > JVMTI_ERROR_NO_MORE_FRAMES rather than ignore all failures when > suspended == NSK_TRUE. The current layering of the macros and verify routines do not lend themselves to this sort of selective error checking. I'd probably file a follow up bug to address specific error checking. > > Also, I don't think you want the suspended check here: > > 348 /* check if expected method frame found */ > 349 if ((suspended == NSK_TRUE) && (found <= 0)) { > > The check for finding the method is: > > 341 if (frameStack[j].method == threadsDesc[i].method) { > > Since frameStack[] is returned by GetStackTrace(), it is not impacted > when not suspended, and the expected method should always be in > frameStack[] somewhere. The issue is only with using > GetFrameLocation() to correlate what is in the result of GetStackTrace(). Agreed. The early continuation was bypassing the found method checking. BTW, another failure has been detected in sp06t001. This time the threads are suspended, but I believe there is a race between thread start and the call to interrupt() the thread. I think there may be some confusion over which thread is invoking the interrupt() call. It is running on the main thread from the code after a call to start the thread, but the thread may not have run when the interrupt is requested. public class sp06t001 extends DebugeeClass { ... // create threads list threads = new sp06t001Thread[] { new sp06t001ThreadRunning("threadRunning", log), new sp06t001ThreadEntering("threadEntering", log), new sp06t001ThreadWaiting("threadWaiting", log), new sp06t001ThreadSleeping("threadSleeping", log), new sp06t001ThreadRunningInterrupted("threadRunningInterrupted", log), new sp06t001ThreadRunningNative("threadRunningNative", log) }; // run threads log.display("Starting tested threads"); try { synchronized (endingMonitor) { // start threads (except first one) for (int i = 0; i < threads.length; i++) { threads[i].start(); if (!threads[i].checkReady()) { throw new Failure("Unable to prepare thread ..." ... class sp06t001ThreadRunningInterrupted extends sp06t001Thread { ... public boolean checkReady() { // interrupt thread on wait() synchronized (waitingMonitor) { interrupt(); } return true; } > > thanks, > > Chris > > On 12/13/18 5:25 AM, Gary Adams wrote: >> While testing I ran into another of the related failures that were >> associated with the original bug. >> The following fake exception stacktrace is for failure analysis. >> nsk.share.Fake_Exception_for_RULE_Creation: (sp02t003.cpp:313) jvmti->GetFrameLocation(threadsDesc[i].thread, j,&qMethod,&qLocation) >> at nsk_lvcomplain(nsk_tools.cpp:172) >> # ERROR: sp02t003.cpp, 313: jvmti->GetFrameLocation(threadsDesc[i].thread, j,&qMethod,&qLocation) >> # jvmti error: code=31, name=JVMTI_ERROR_NO_MORE_FRAMES >> - sp02t003.cpp, 310: 3 frame: method: 0x7f225042b2d8, location: 16 >> # ERROR: sp02t003.cpp, 313: jvmti->GetFrameLocation(threadsDesc[i].thread, j,&qMethod,&qLocation) >> # jvmti error: code=31, name=JVMTI_ERROR_NO_MORE_FRAMES >> # ERROR: sp02t003.cpp, 350: No expected method frame for not suspended thread #4 (threadRunningInterrupted) >> In this particular failure, the GetFrameLocation call failed, because >> the frame was no longer >> accessible. >> >> If the threads are not suspended, the GetFrameLocation may fail, >> or if it passes, the qMethod and qLocation could belong to another >> frame. >> >> I've updated the webrev to allow the call to GetFrameLocation, but to >> only >> report an error if the threads are suspended. Similarly, the checks >> to compare >> qMethod and qLocation will be skipped, if the threads are not suspended. >> And the final comparison that the method was found will only be an error >> if the threads are suspended. >> >> Webrev: http://cr.openjdk.java.net/~gadams/8051349/webrev.01/ >> >> On 12/12/18, 11:54 AM, Gary Adams wrote: >>> After some testing with nsk verbose messages enabled, >>> it is clear that this test is failing in checkThreads when the >>> location did not match between the call to GetStackTrace >>> and GetFrameLocation. For the tests that are run when the threads >>> have not been suspended, there is no guarantee the locations >>> will match. >>> >>> Issue: https://bugs.openjdk.java.net/browse/JDK-8051349 >>> Webrev: http://cr.openjdk.java.net/~gadams/8051349/webrev.00/ >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jcbeyler at google.com Fri Dec 14 18:52:02 2018 From: jcbeyler at google.com (JC Beyler) Date: Fri, 14 Dec 2018 10:52:02 -0800 Subject: RFR (XS) 8215329: Modify ZGC requirement for HeapMonitorThreadTest.java In-Reply-To: <9abfcd5a-8266-c539-d8fc-79400de23066@oracle.com> References: <53d8feb7-1616-c5fb-7973-5a22d7eef713@oracle.com> <36e27b1d-defb-549e-84c1-cbc40d3cd70a@oracle.com> <9abfcd5a-8266-c539-d8fc-79400de23066@oracle.com> Message-ID: Thanks all, tested & pushed. @Per: let me know when you resolve your bug and if you want me to do anything from my side On Thu, Dec 13, 2018 at 10:25 AM serguei.spitsyn at oracle.com < serguei.spitsyn at oracle.com> wrote: > +1 > > Thanks, > Serguei > > > On 12/13/18 09:40, Alex Menkov wrote: > > +1 > > > > --alex > > > > On 12/13/2018 08:59, JC Beyler wrote: > >> Hi Per, > >> > >> Thanks for the messages and review :-). I believe that actually what > >> happened was that when JDK11 was close to release both ZGC and > >> HeapMonitoring tried to get in. In a last effort, we turned off this > >> test for ZGC as it was showing test failures for ZGC. It's a bit > >> fuzzy to be honest (I was also on paternity leave and Serguei & > >> Jeremy were helping out here). But anyway, what seems to be true now > >> is that you found a bug (yeah I guess?) and we can remove > >> the @requires once you fix your race. > >> > >> Therefore, could I please get another review to update the @requires > >> to be correct then until then? > >> > >> (@Per: if you want, I can update the test once it's done; either > >> assign a JBS bug to me or send me an email) > >> > >> Thanks, > >> Jc > >> > >> On Thu, Dec 13, 2018 at 4:47 AM Per Liden >> > wrote: > >> > >> Hi, me again ;) > >> > >> I think I've found the root cause of this. There's a tiny race in > >> the > >> ZGC allocation path, which can lead to pre-mature OOME being thrown. > >> It's not trivial to fix, so I suggest you go ahead with your > >> original > >> patch (Looks good btw), and I'll file a separate bug to fix the ZGC > >> issue (and update this test to run with ZGC again). > >> > >> cheers, > >> Per > >> > >> On 12/13/2018 12:21 PM, Per Liden wrote: > >> > Hi again, > >> > > >> > I ran this test some more and managed to get an OOME even with a > >> 768M > >> > heap. I'm getting a bit suspicious that something else is wrong > >> here. > >> > Let me dig into this some more and see if I can understand what > >> the real > >> > issue is. > >> > > >> > cheers, > >> > Per > >> > > >> > On 12/13/2018 10:31 AM, Per Liden wrote: > >> >> Hi JC, > >> >> > >> >> What's the reason to exclude ZGC from this test to begin > >> with? From > >> >> what I can tell, it's because the test is using a slightly too > >> small > >> >> heap, or are there some other reason? I ran it a few times using > >> >> various heap sizes and the test passes with ZGC when using > >> anything > >> >> above 612M. So if we instead just dump the heap size a bit, > >> then we > >> >> get test coverage with ZGC too. I picked 768M here to have some > >> >> headroom in case the exact limit is run-to-run dependent. > >> >> > >> >> diff --git > >> >> > >> > a/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorThreadTest.java > >> > >> >> > >> > b/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorThreadTest.java > >> > >> >> > >> >> --- > >> >> > >> > a/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorThreadTest.java > >> > >> >> > >> >> +++ > >> >> > >> > b/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorThreadTest.java > >> > >> >> > >> >> @@ -29,8 +29,7 @@ > >> >> * @build Frame HeapMonitor ThreadInformation > >> >> * @summary Verifies the JVMTI Heap Monitor Thread information > >> sanity. > >> >> * @compile HeapMonitorThreadTest.java > >> >> - * @run main/othervm/native -Xmx512m -agentlib:HeapMonitorTest > >> >> MyPackage.HeapMonitorThreadTest > >> >> - * @requires !vm.gc.Z > >> >> + * @run main/othervm/native -Xmx768m -agentlib:HeapMonitorTest > >> >> MyPackage.HeapMonitorThreadTest > >> >> */ > >> >> > >> >> import java.util.List; > >> >> > >> >> cheers, > >> >> Per > >> >> > >> >> On 12/13/2018 05:44 AM, JC Beyler wrote: > >> >>> Hi all, > >> >>> > >> >>> When working on another webrev, I saw this problem: > >> >>> > >> >>> Webrev: > http://cr.openjdk.java.net/~jcbeyler/8215329/webrev.00/ > >> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8215329 > >> >>> > >> >>> (Basically, from what I understood from an email from Per > >> Liden: > >> >>> - @requires !vm.gc.Z -> ZGC is built in the JDK > >> >>> - @requires vm.gc != "Z" -> ZGC is being used for the > >> runtime > >> >>> ) > >> >>> > >> >>> Thanks, > >> >>> Jc > >> > >> > >> > >> -- > >> > >> Thanks, > >> Jc > > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From jcbeyler at google.com Fri Dec 14 18:56:01 2018 From: jcbeyler at google.com (JC Beyler) Date: Fri, 14 Dec 2018 10:56:01 -0800 Subject: optimize KlassInfoTable size to power of 2 In-Reply-To: References: <140bac18565c48be88313f7cc1d24bf8@jd.com> <7fc37fd7cf24439eb3a6fff132fc7387@jd.com> <4ea1c92f-4e72-7d38-8c7a-91e2892e0c3b@redhat.com> <1ab2d28b40024ba6b299756612fa3760@jd.com> <7a1e5d87-94fd-9be9-8aab-5dd0f1a3fff9@redhat.com> <56b9099e33584796af8a508980ff4690@jd.com> <436fddb22d3b4fb9aec315f3150cff41@jd.com> <0045EBD6-F21D-457B-865E-3796BD86FFF9@amazon.com> <0c787ea18bf24ef9baec5fbb0b0dfc18@jd.com> Message-ID: Hi Andrew, Done for the change of the commit message and JBS title. I've pushed it on the submit repo and will wait for the results before pushing, Jc On Thu, Dec 13, 2018 at 1:57 AM Andrew Haley wrote: > On 12/13/18 1:37 AM, ?? wrote: > > Can I get more review about this: > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8215228/webrev.00/ > > Bug: https://bugs.openjdk.java.net/browse/JDK-8215228 > > It's fine. Please change the commit comment to something like > "Use a constant hash table size in order to enable compiler optimization." > > -- > Andrew Haley > Java Platform Lead Engineer > Red Hat UK Ltd. > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Fri Dec 14 21:43:43 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Fri, 14 Dec 2018 13:43:43 -0800 Subject: RFR: JDK-8051349: nsk/jvmti/scenarios/sampling/SP06/sp06t003 fails in nightly In-Reply-To: <5C13AFF7.2050201@oracle.com> References: <5C113D4D.7070503@oracle.com> <5C125DBC.6090307@oracle.com> <196f4645-b082-65f8-ff2d-0434a03a34ca@oracle.com> <5C13AFF7.2050201@oracle.com> Message-ID: <412971f0-a8e7-fab5-5dce-827aa9343e74@oracle.com> An HTML attachment was scrubbed... URL: From gary.adams at oracle.com Sat Dec 15 13:15:08 2018 From: gary.adams at oracle.com (gary.adams at oracle.com) Date: Sat, 15 Dec 2018 08:15:08 -0500 Subject: RFR: JDK-8051349: nsk/jvmti/scenarios/sampling/SP06/sp06t003 fails in nightly In-Reply-To: <412971f0-a8e7-fab5-5dce-827aa9343e74@oracle.com> References: <5C113D4D.7070503@oracle.com> <5C125DBC.6090307@oracle.com> <196f4645-b082-65f8-ff2d-0434a03a34ca@oracle.com> <5C13AFF7.2050201@oracle.com> <412971f0-a8e7-fab5-5dce-827aa9343e74@oracle.com> Message-ID: On 12/14/18 4:43 PM, Chris Plummer wrote: > On 12/14/18 5:28 AM, Gary Adams wrote: > > BTW, I forgot to ask if you can fix the typo in the following comment: > > ?345???????????? /* check frame equalaty */ Yes, I fixed that one on Fri. I usually save the typos for the final webrev. >> On 12/13/18, 11:51 AM, Chris Plummer wrote: >>> Hi Gary, >>> >>> Unfortunately GetStackTrace() returns the top frame as the first >>> frame in the array. Thus if the thread is not suspended and later >>> you call GetFrameLocation() for some depth, there's no way to make >>> sure the frame at that depth is the same frame at that depth in the >>> array returned by GetStackTrace() unless you first suspend the frame >>> before calling GetFrameLocation(). You would also need to call >>> GetFrameCount() while suspended, compare that with the number of >>> frames returned by GetStackTrace(), and make any needed depth >>> adjustments. But since I think the intent is to not suspend the >>> thread here, it probably does not make sense to do that, and instead >>> accept that there might be some errors as you have done. >> I'm not totally comfortable with these attempted operations and then >> bypassing the test failure if the threads were not suspended. >> Would it be better to only perform check? thread operations on the >> suspended threads and to totally skip the operations that can not >> be safely performed when the thread is not suspended. > I was starting to think along those lines. What's the point of testing > something if you know it can fail and you can't distinguish between > expected and unexpected failure. The other option is to suspend like I > describe above to make sure you can accurately verify the stack trace. > I'm just not sure if it makes sense to do that. Why is the test run > unsuspended in the first place? The test runs several separate threads and each is presenting a different scenario. It probably is good to try the various functions about stack trace, frame counts and locations on suspended and resumed threads. It is just a bit harder to determine what a failed condition would be for the running threads. >>> >>> One improvement I would like to see for your fix is to only ignore >>> JVMTI_ERROR_NO_MORE_FRAMES rather than ignore all failures when >>> suspended == NSK_TRUE. >> The current layering of the macros and verify routines do not lend >> themselves >> to this sort of selective error checking. I'd probably file a follow >> up bug to >> address specific error checking. > Can't you just bypass the use of NSK_VERIFY? Isn't JC getting rid of > it anyway and using his JNI exception mechanism? In that case you > would need to use the version that does not fail if there was an > exception. I do not know what set of error returns would be acceptable or how to express that in the new mechanism. That's why I suggested addressing it in a follow up bug. It simply would narrow the range of error returns allowed. >>> >>> Also, I don't think you want the suspended check here: >>> >>> ?348???????? /* check if expected method frame found */ >>> ?349???????? if ((suspended == NSK_TRUE) && (found <= 0)) { >>> >>> The check for finding the method is: >>> >>> ?341???????????? if (frameStack[j].method == threadsDesc[i].method) { >>> >>> Since frameStack[] is returned by GetStackTrace(), it is not >>> impacted when not suspended, and the expected method should always >>> be in frameStack[] somewhere. The issue is only with using >>> GetFrameLocation() to correlate what is in the result of >>> GetStackTrace(). >> Agreed. The early continuation was bypassing the found method checking. >> >> BTW, another failure has been detected in sp06t001. This time the >> threads are suspended, >> but I believe there is a race between thread start and the call to >> interrupt() the thread. >> I think there may be some confusion over which thread is invoking the >> interrupt() call. >> It is running on the main thread from the code after a call to start >> the thread, but the >> thread may not have run when the interrupt is requested. >> >> public class sp06t001 extends DebugeeClass { >> ... >> >> ??????? // create threads list >> ??????? threads = new sp06t001Thread[] { >> ??????????? new sp06t001ThreadRunning("threadRunning", log), >> ??????????? new sp06t001ThreadEntering("threadEntering", log), >> ??????????? new sp06t001ThreadWaiting("threadWaiting", log), >> ??????????? new sp06t001ThreadSleeping("threadSleeping", log), >> ??????????? new >> sp06t001ThreadRunningInterrupted("threadRunningInterrupted", log), >> ??????????? new sp06t001ThreadRunningNative("threadRunningNative", log) >> ??????? }; >> >> ??????? // run threads >> ??????? log.display("Starting tested threads"); >> ??????? try { >> ??????????? synchronized (endingMonitor) { >> ??????????????? // start threads (except first one) >> ??????????????? for (int i = 0; i < threads.length; i++) { >> ??????????????????? threads[i].start(); >> ??????????????????? if (!threads[i].checkReady()) { >> ??????????????????????? throw new Failure("Unable to prepare thread ..." >> ... >> >> class sp06t001ThreadRunningInterrupted extends sp06t001Thread { >> ... >> ??? public boolean checkReady() { >> ??????? // interrupt thread on wait() >> ??????? synchronized (waitingMonitor) { >> ??????????? interrupt(); >> ??????? } >> ??????? return true; >> ??? } > Certainly checkReady() could be called before the target thread has > started the wait(). Then the thread does the wait() and probably never > exits (or maybe the early interrupted is resulting in some other > unexpected behavior). If we get to the wait then the framecount would be fine. The debuggee class launches 6 threads. The agent is synced with the debuggee starting. I don't see the native agent synchronizing on the 6 started threads, when the threads are suspended and checked. As far as I can tell the call to interrupted() just sets a flag and it is up to the thread itself to decide how to interpret the isInterrupted() state. > > I think you need synchronize between the two threads. What if the > target thread does a synchronized (waitingMonitor), sets a flag, and > then a wait(). The main thread sleeps until the flag is set, does a > synchronized(waitingMonitor), and then the interrupt(). Since the test is already using a waiting monitor, I do not want to disturb how it is being used in setting up the interrupted thread test case. The test is also using a "volatile boolean threadReady" for signalling conditions in the test. On Fri I was experimenting with another boolean flag to delay the checkReady until the testMethod was at the waiting monitor, e.g. ready to be interrupted. It seems to work but I was waiting for enough testruns before posting a revised webrev. > > Chris >>> >>> thanks, >>> >>> Chris >>> >>> On 12/13/18 5:25 AM, Gary Adams wrote: >>>> While testing I ran into another of the related failures that were >>>> associated with the original bug. >>>> The following fake exception stacktrace is for failure analysis. >>>> nsk.share.Fake_Exception_for_RULE_Creation: (sp02t003.cpp:313) jvmti->GetFrameLocation(threadsDesc[i].thread, j, &qMethod, &qLocation) >>>> at nsk_lvcomplain(nsk_tools.cpp:172) >>>> # ERROR: sp02t003.cpp, 313: jvmti->GetFrameLocation(threadsDesc[i].thread, j, &qMethod, &qLocation) >>>> # jvmti error: code=31, name=JVMTI_ERROR_NO_MORE_FRAMES >>>> - sp02t003.cpp, 310: 3 frame: method: 0x7f225042b2d8, location: 16 >>>> # ERROR: sp02t003.cpp, 313: jvmti->GetFrameLocation(threadsDesc[i].thread, j, &qMethod, &qLocation) >>>> # jvmti error: code=31, name=JVMTI_ERROR_NO_MORE_FRAMES >>>> # ERROR: sp02t003.cpp, 350: No expected method frame for not suspended thread #4 (threadRunningInterrupted) >>>> In this particular failure, the GetFrameLocation call failed, >>>> because the frame was no longer >>>> accessible. >>>> >>>> If the threads are not suspended, the GetFrameLocation may fail, >>>> or if it passes, the qMethod and qLocation could belong to another >>>> frame. >>>> >>>> I've updated the webrev to allow the call to GetFrameLocation, but >>>> to only >>>> report an error if the threads are suspended. Similarly, the checks >>>> to compare >>>> qMethod and qLocation will be skipped, if the threads are not >>>> suspended. >>>> And the final comparison that the method was found will only be an >>>> error >>>> if the threads are suspended. >>>> >>>> ? Webrev: http://cr.openjdk.java.net/~gadams/8051349/webrev.01/ >>>> >>>> On 12/12/18, 11:54 AM, Gary Adams wrote: >>>>> After some testing with nsk verbose messages enabled, >>>>> it is clear that this test is failing in checkThreads when the >>>>> location did not match between the call to GetStackTrace >>>>> and GetFrameLocation. For the tests that are run when the threads >>>>> have not been suspended, there is no guarantee the locations >>>>> will match. >>>>> >>>>> ? Issue: https://bugs.openjdk.java.net/browse/JDK-8051349 >>>>> ? Webrev: http://cr.openjdk.java.net/~gadams/8051349/webrev.00/ >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From per.liden at oracle.com Sat Dec 15 14:37:57 2018 From: per.liden at oracle.com (Per Liden) Date: Sat, 15 Dec 2018 15:37:57 +0100 Subject: RFR (XS) 8215329: Modify ZGC requirement for HeapMonitorThreadTest.java In-Reply-To: References: <53d8feb7-1616-c5fb-7973-5a22d7eef713@oracle.com> <36e27b1d-defb-549e-84c1-cbc40d3cd70a@oracle.com> <9abfcd5a-8266-c539-d8fc-79400de23066@oracle.com> Message-ID: <79179951-346d-f3ac-747c-122668d14612@oracle.com> Just a follow up. After some further debugging, the real cause for this issue turned out to be a bug in JNI IsSameObject(). Fix currently out for review on hotspot-runtime-dev. https://bugs.openjdk.java.net/browse/JDK-8215451 cheers, Per On 12/14/2018 07:52 PM, JC Beyler wrote: > Thanks all, tested & pushed. > > @Per: let me know when you resolve your bug and if you want me to do > anything from my side > > On Thu, Dec 13, 2018 at 10:25 AM serguei.spitsyn at oracle.com > > wrote: > > +1 > > Thanks, > Serguei > > > On 12/13/18 09:40, Alex Menkov wrote: > > +1 > > > > --alex > > > > On 12/13/2018 08:59, JC Beyler wrote: > >> Hi Per, > >> > >> Thanks for the messages and review :-). I believe that actually > what > >> happened was that when JDK11 was close to release both ZGC and > >> HeapMonitoring tried to get in. In a last effort, we turned off > this > >> test for ZGC as it was showing test failures for ZGC. It's a bit > >> fuzzy to be honest (I was also on paternity leave and Serguei & > >> Jeremy were helping out here). But anyway, what seems to be true > now > >> is that you found a bug (yeah I guess?) and we can remove > >> the @requires once you fix your race. > >> > >> Therefore, could I please get another review to update > the @requires > >> to be correct then until then? > >> > >> (@Per: if you want, I can update the test once it's done; either > >> assign a JBS bug to me or send me an email) > >> > >> Thanks, > >> Jc > >> > >> On Thu, Dec 13, 2018 at 4:47 AM Per Liden > >> >> wrote: > >> > >> Hi, me again ;) > >> > >> I think I've found the root cause of this. There's a tiny > race in > >> the > >> ZGC allocation path, which can lead to pre-mature OOME being > thrown. > >> It's not trivial to fix, so I suggest you go ahead with your > >> original > >> patch (Looks good btw), and I'll file a separate bug to fix > the ZGC > >> issue (and update this test to run with ZGC again). > >> > >> cheers, > >> Per > >> > >> On 12/13/2018 12:21 PM, Per Liden wrote: > >> > Hi again, > >> > > >> > I ran this test some more and managed to get an OOME even > with a > >> 768M > >> > heap. I'm getting a bit suspicious that something else is > wrong > >> here. > >> > Let me dig into this some more and see if I can > understand what > >> the real > >> > issue is. > >> > > >> > cheers, > >> > Per > >> > > >> > On 12/13/2018 10:31 AM, Per Liden wrote: > >> >> Hi JC, > >> >> > >> >> What's the reason to exclude ZGC from this test to begin > >> with? From > >> >> what I can tell, it's because the test is using a > slightly too > >> small > >> >> heap, or are there some other reason? I ran it a few > times using > >> >> various heap sizes and the test passes with ZGC when using > >> anything > >> >> above 612M. So if we instead just dump the heap size a bit, > >> then we > >> >> get test coverage with ZGC too. I picked 768M here to > have some > >> >> headroom in case the exact limit is run-to-run dependent. > >> >> > >> >> diff --git > >> >> > >> > a/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorThreadTest.java > >> > >> >> > >> > b/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorThreadTest.java > >> > >> >> > >> >> --- > >> >> > >> > a/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorThreadTest.java > >> > >> >> > >> >> +++ > >> >> > >> > b/test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorThreadTest.java > >> > >> >> > >> >> @@ -29,8 +29,7 @@ > >> >> * @build Frame HeapMonitor ThreadInformation > >> >> * @summary Verifies the JVMTI Heap Monitor Thread > information > >> sanity. > >> >> * @compile HeapMonitorThreadTest.java > >> >> - * @run main/othervm/native -Xmx512m > -agentlib:HeapMonitorTest > >> >> MyPackage.HeapMonitorThreadTest > >> >> - * @requires !vm.gc.Z > >> >> + * @run main/othervm/native -Xmx768m > -agentlib:HeapMonitorTest > >> >> MyPackage.HeapMonitorThreadTest > >> >> */ > >> >> > >> >> import java.util.List; > >> >> > >> >> cheers, > >> >> Per > >> >> > >> >> On 12/13/2018 05:44 AM, JC Beyler wrote: > >> >>> Hi all, > >> >>> > >> >>> When working on another webrev, I saw this problem: > >> >>> > >> >>> Webrev: > http://cr.openjdk.java.net/~jcbeyler/8215329/webrev.00/ > >> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8215329 > >> >>> > >> >>> (Basically, from what I understood from an email from Per > >> Liden: > >> >>> - @requires !vm.gc.Z -> ZGC is built in the JDK > >> >>> - @requires vm.gc != "Z" -> ZGC is being used for the > >> runtime > >> >>> ) > >> >>> > >> >>> Thanks, > >> >>> Jc > >> > >> > >> > >> -- > >> > >> Thanks, > >> Jc > > > > -- > > Thanks, > Jc From gary.adams at oracle.com Sat Dec 15 20:24:03 2018 From: gary.adams at oracle.com (gary.adams at oracle.com) Date: Sat, 15 Dec 2018 15:24:03 -0500 Subject: RFR: JDK-8051349: nsk/jvmti/scenarios/sampling/SP06/sp06t003 fails in nightly In-Reply-To: References: <5C113D4D.7070503@oracle.com> <5C125DBC.6090307@oracle.com> <196f4645-b082-65f8-ff2d-0434a03a34ca@oracle.com> <5C13AFF7.2050201@oracle.com> <412971f0-a8e7-fab5-5dce-827aa9343e74@oracle.com> Message-ID: <48289bc7-1d4a-6c61-b207-6a3214ff7e4a@oracle.com> Updated webrev: ? http://cr.openjdk.java.net/~gadams/8051349/webrev.02/index.html On 12/15/18 8:15 AM, gary.adams at oracle.com wrote: > On 12/14/18 4:43 PM, Chris Plummer wrote: >> On 12/14/18 5:28 AM, Gary Adams wrote: >> >> BTW, I forgot to ask if you can fix the typo in the following comment: >> >> ?345???????????? /* check frame equalaty */ > > Yes, I fixed that one on Fri. > I usually save the typos for the final webrev. > >>> On 12/13/18, 11:51 AM, Chris Plummer wrote: >>>> Hi Gary, >>>> >>>> Unfortunately GetStackTrace() returns the top frame as the first >>>> frame in the array. Thus if the thread is not suspended and later >>>> you call GetFrameLocation() for some depth, there's no way to make >>>> sure the frame at that depth is the same frame at that depth in the >>>> array returned by GetStackTrace() unless you first suspend the >>>> frame before calling GetFrameLocation(). You would also need to >>>> call GetFrameCount() while suspended, compare that with the number >>>> of frames returned by GetStackTrace(), and make any needed depth >>>> adjustments. But since I think the intent is to not suspend the >>>> thread here, it probably does not make sense to do that, and >>>> instead accept that there might be some errors as you have done. >>> I'm not totally comfortable with these attempted operations and then >>> bypassing the test failure if the threads were not suspended. >>> Would it be better to only perform check? thread operations on the >>> suspended threads and to totally skip the operations that can not >>> be safely performed when the thread is not suspended. >> I was starting to think along those lines. What's the point of >> testing something if you know it can fail and you can't distinguish >> between expected and unexpected failure. The other option is to >> suspend like I describe above to make sure you can accurately verify >> the stack trace. I'm just not sure if it makes sense to do that. Why >> is the test run unsuspended in the first place? > The test runs several separate threads and each is presenting > a different scenario. It probably is good to try the various functions > about stack trace, frame counts and locations on suspended and resumed > threads. It is just a bit harder to determine what a failed condition > would be for the running threads. > >>>> >>>> One improvement I would like to see for your fix is to only ignore >>>> JVMTI_ERROR_NO_MORE_FRAMES rather than ignore all failures when >>>> suspended == NSK_TRUE. >>> The current layering of the macros and verify routines do not lend >>> themselves >>> to this sort of selective error checking. I'd probably file a follow >>> up bug to >>> address specific error checking. >> Can't you just bypass the use of NSK_VERIFY? Isn't JC getting rid of >> it anyway and using his JNI exception mechanism? In that case you >> would need to use the version that does not fail if there was an >> exception. > I do not know what set of error returns would be acceptable > or how to express that in the new mechanism. That's why I suggested > addressing it in a follow up bug. It simply would narrow the range > of error returns allowed. >>>> >>>> Also, I don't think you want the suspended check here: >>>> >>>> ?348???????? /* check if expected method frame found */ >>>> ?349???????? if ((suspended == NSK_TRUE) && (found <= 0)) { >>>> >>>> The check for finding the method is: >>>> >>>> ?341???????????? if (frameStack[j].method == threadsDesc[i].method) { >>>> >>>> Since frameStack[] is returned by GetStackTrace(), it is not >>>> impacted when not suspended, and the expected method should always >>>> be in frameStack[] somewhere. The issue is only with using >>>> GetFrameLocation() to correlate what is in the result of >>>> GetStackTrace(). >>> Agreed. The early continuation was bypassing the found method checking. >>> >>> BTW, another failure has been detected in sp06t001. This time the >>> threads are suspended, >>> but I believe there is a race between thread start and the call to >>> interrupt() the thread. >>> I think there may be some confusion over which thread is invoking >>> the interrupt() call. >>> It is running on the main thread from the code after a call to start >>> the thread, but the >>> thread may not have run when the interrupt is requested. >>> >>> public class sp06t001 extends DebugeeClass { >>> ... >>> >>> ??????? // create threads list >>> ??????? threads = new sp06t001Thread[] { >>> ??????????? new sp06t001ThreadRunning("threadRunning", log), >>> ??????????? new sp06t001ThreadEntering("threadEntering", log), >>> ??????????? new sp06t001ThreadWaiting("threadWaiting", log), >>> ??????????? new sp06t001ThreadSleeping("threadSleeping", log), >>> ??????????? new >>> sp06t001ThreadRunningInterrupted("threadRunningInterrupted", log), >>> ??????????? new sp06t001ThreadRunningNative("threadRunningNative", log) >>> ??????? }; >>> >>> ??????? // run threads >>> ??????? log.display("Starting tested threads"); >>> ??????? try { >>> ??????????? synchronized (endingMonitor) { >>> ??????????????? // start threads (except first one) >>> ??????????????? for (int i = 0; i < threads.length; i++) { >>> ??????????????????? threads[i].start(); >>> ??????????????????? if (!threads[i].checkReady()) { >>> ??????????????????????? throw new Failure("Unable to prepare thread ..." >>> ... >>> >>> class sp06t001ThreadRunningInterrupted extends sp06t001Thread { >>> ... >>> ??? public boolean checkReady() { >>> ??????? // interrupt thread on wait() >>> ??????? synchronized (waitingMonitor) { >>> ??????????? interrupt(); >>> ??????? } >>> ??????? return true; >>> ??? } >> Certainly checkReady() could be called before the target thread has >> started the wait(). Then the thread does the wait() and probably >> never exits (or maybe the early interrupted is resulting in some >> other unexpected behavior). > If we get to the wait then the framecount would be fine. > > The debuggee class launches 6 threads. The agent is synced with the > debuggee > starting. I don't see the native agent synchronizing on the 6 started > threads, > when the threads are suspended and checked. > > As far as I can tell the call to interrupted() just sets a flag and it > is up to the thread > itself to decide how to interpret the isInterrupted() state. >> >> I think you need synchronize between the two threads. What if the >> target thread does a synchronized (waitingMonitor), sets a flag, and >> then a wait(). The main thread sleeps until the flag is set, does a >> synchronized(waitingMonitor), and then the interrupt(). > Since the test is already using a waiting monitor, I do not want to > disturb how it is being used in setting up the interrupted thread test > case. > The test is also using a "volatile boolean threadReady" for signalling > conditions in the test. > > On Fri I was experimenting with another boolean flag to delay the > checkReady until the testMethod was at the waiting monitor, > e.g. ready to be interrupted. It seems to work but I was waiting > for enough testruns before posting a revised webrev. > >> >> Chris >>>> >>>> thanks, >>>> >>>> Chris >>>> >>>> On 12/13/18 5:25 AM, Gary Adams wrote: >>>>> While testing I ran into another of the related failures that were >>>>> associated with the original bug. >>>>> The following fake exception stacktrace is for failure analysis. >>>>> nsk.share.Fake_Exception_for_RULE_Creation: (sp02t003.cpp:313) jvmti->GetFrameLocation(threadsDesc[i].thread, j, &qMethod, &qLocation) >>>>> at nsk_lvcomplain(nsk_tools.cpp:172) >>>>> # ERROR: sp02t003.cpp, 313: jvmti->GetFrameLocation(threadsDesc[i].thread, j, &qMethod, &qLocation) >>>>> # jvmti error: code=31, name=JVMTI_ERROR_NO_MORE_FRAMES >>>>> - sp02t003.cpp, 310: 3 frame: method: 0x7f225042b2d8, location: 16 >>>>> # ERROR: sp02t003.cpp, 313: jvmti->GetFrameLocation(threadsDesc[i].thread, j, &qMethod, &qLocation) >>>>> # jvmti error: code=31, name=JVMTI_ERROR_NO_MORE_FRAMES >>>>> # ERROR: sp02t003.cpp, 350: No expected method frame for not suspended thread #4 (threadRunningInterrupted) >>>>> In this particular failure, the GetFrameLocation call failed, >>>>> because the frame was no longer >>>>> accessible. >>>>> >>>>> If the threads are not suspended, the GetFrameLocation may fail, >>>>> or if it passes, the qMethod and qLocation could belong to another >>>>> frame. >>>>> >>>>> I've updated the webrev to allow the call to GetFrameLocation, but >>>>> to only >>>>> report an error if the threads are suspended. Similarly, the >>>>> checks to compare >>>>> qMethod and qLocation will be skipped, if the threads are not >>>>> suspended. >>>>> And the final comparison that the method was found will only be an >>>>> error >>>>> if the threads are suspended. >>>>> >>>>> ? Webrev: http://cr.openjdk.java.net/~gadams/8051349/webrev.01/ >>>>> >>>>> On 12/12/18, 11:54 AM, Gary Adams wrote: >>>>>> After some testing with nsk verbose messages enabled, >>>>>> it is clear that this test is failing in checkThreads when the >>>>>> location did not match between the call to GetStackTrace >>>>>> and GetFrameLocation. For the tests that are run when the threads >>>>>> have not been suspended, there is no guarantee the locations >>>>>> will match. >>>>>> >>>>>> ? Issue: https://bugs.openjdk.java.net/browse/JDK-8051349 >>>>>> ? Webrev: http://cr.openjdk.java.net/~gadams/8051349/webrev.00/ >>>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Sun Dec 16 06:50:27 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Sat, 15 Dec 2018 22:50:27 -0800 Subject: RFR: JDK-8051349: nsk/jvmti/scenarios/sampling/SP06/sp06t003 fails in nightly In-Reply-To: <48289bc7-1d4a-6c61-b207-6a3214ff7e4a@oracle.com> References: <5C113D4D.7070503@oracle.com> <5C125DBC.6090307@oracle.com> <196f4645-b082-65f8-ff2d-0434a03a34ca@oracle.com> <5C13AFF7.2050201@oracle.com> <412971f0-a8e7-fab5-5dce-827aa9343e74@oracle.com> <48289bc7-1d4a-6c61-b207-6a3214ff7e4a@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Mon Dec 17 02:01:19 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 17 Dec 2018 12:01:19 +1000 Subject: RFR: JDK-8051349: nsk/jvmti/scenarios/sampling/SP06/sp06t003 fails in nightly In-Reply-To: <5C13AFF7.2050201@oracle.com> References: <5C113D4D.7070503@oracle.com> <5C125DBC.6090307@oracle.com> <196f4645-b082-65f8-ff2d-0434a03a34ca@oracle.com> <5C13AFF7.2050201@oracle.com> Message-ID: On 14/12/2018 11:28 pm, Gary Adams wrote: > On 12/13/18, 11:51 AM, Chris Plummer wrote: > BTW, another failure has been detected in sp06t001. This time the > threads are suspended, > but I believe there is a race between thread start and the call to > interrupt() the thread. > I think there may be some confusion over which thread is invoking the > interrupt() call. > It is running on the main thread from the code after a call to start the > thread, but the > thread may not have run when the interrupt is requested. > > public class sp06t001 extends DebugeeClass { > ... > > ??????? // create threads list > ??????? threads = new sp06t001Thread[] { > ??????????? new sp06t001ThreadRunning("threadRunning", log), > ??????????? new sp06t001ThreadEntering("threadEntering", log), > ??????????? new sp06t001ThreadWaiting("threadWaiting", log), > ??????????? new sp06t001ThreadSleeping("threadSleeping", log), > ??????????? new > sp06t001ThreadRunningInterrupted("threadRunningInterrupted", log), > ??????????? new sp06t001ThreadRunningNative("threadRunningNative", log) > ??????? }; > > ??????? // run threads > ??????? log.display("Starting tested threads"); > ??????? try { > ??????????? synchronized (endingMonitor) { > ??????????????? // start threads (except first one) > ??????????????? for (int i = 0; i < threads.length; i++) { > ??????????????????? threads[i].start(); > ??????????????????? if (!threads[i].checkReady()) { > ??????????????????????? throw new Failure("Unable to prepare thread ..." > ... > > class sp06t001ThreadRunningInterrupted extends sp06t001Thread { > ... > ??? public boolean checkReady() { > ??????? // interrupt thread on wait() > ??????? synchronized (waitingMonitor) { > ??????????? interrupt(); > ??????? } > ??????? return true; > ??? } checkReady is broken. The synchronization is completely pointless as it doesn't ensure the target got to the wait() as the calling thread could call checkReady before the target has run sufficient to acquire the waitingMonitor monitor. That said as long as the interrupt() occurs after the thread has been started it will cause the wait() to terminate. Though the wait() itself could return early (in theory) due to spurious wakeup. The coordination/synchronization in these tests is really flawed. David ----- >> >> thanks, >> >> Chris >> >> On 12/13/18 5:25 AM, Gary Adams wrote: >>> While testing I ran into another of the related failures that were >>> associated with the original bug. >>> The following fake exception stacktrace is for failure analysis. >>> nsk.share.Fake_Exception_for_RULE_Creation: (sp02t003.cpp:313) jvmti->GetFrameLocation(threadsDesc[i].thread, j, &qMethod, &qLocation) >>> at nsk_lvcomplain(nsk_tools.cpp:172) >>> # ERROR: sp02t003.cpp, 313: jvmti->GetFrameLocation(threadsDesc[i].thread, j, &qMethod, &qLocation) >>> # jvmti error: code=31, name=JVMTI_ERROR_NO_MORE_FRAMES >>> - sp02t003.cpp, 310: 3 frame: method: 0x7f225042b2d8, location: 16 >>> # ERROR: sp02t003.cpp, 313: jvmti->GetFrameLocation(threadsDesc[i].thread, j, &qMethod, &qLocation) >>> # jvmti error: code=31, name=JVMTI_ERROR_NO_MORE_FRAMES >>> # ERROR: sp02t003.cpp, 350: No expected method frame for not suspended thread #4 (threadRunningInterrupted) >>> In this particular failure, the GetFrameLocation call failed, because >>> the frame was no longer >>> accessible. >>> >>> If the threads are not suspended, the GetFrameLocation may fail, >>> or if it passes, the qMethod and qLocation could belong to another >>> frame. >>> >>> I've updated the webrev to allow the call to GetFrameLocation, but to >>> only >>> report an error if the threads are suspended. Similarly, the checks >>> to compare >>> qMethod and qLocation will be skipped, if the threads are not suspended. >>> And the final comparison that the method was found will only be an error >>> if the threads are suspended. >>> >>> ? Webrev: http://cr.openjdk.java.net/~gadams/8051349/webrev.01/ >>> >>> On 12/12/18, 11:54 AM, Gary Adams wrote: >>>> After some testing with nsk verbose messages enabled, >>>> it is clear that this test is failing in checkThreads when the >>>> location did not match between the call to GetStackTrace >>>> and GetFrameLocation. For the tests that are run when the threads >>>> have not been suspended, there is no guarantee the locations >>>> will match. >>>> >>>> ? Issue: https://bugs.openjdk.java.net/browse/JDK-8051349 >>>> ? Webrev: http://cr.openjdk.java.net/~gadams/8051349/webrev.00/ >>>> >>> >> > From matthias.baesken at sap.com Mon Dec 17 08:59:18 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Mon, 17 Dec 2018 08:59:18 +0000 Subject: RFR: [XS] 8215411: some GetByteArrayElements calls miss corresponding Release Message-ID: Hello, please review the following change. I noticed that we miss at some places (for example in case of early returns) where GetByteArrayElements is used, the corresponding ReleaseByteArrayElements call. In VirtualMachineImpl.c I also removed a check for isCopy (is the returned byte array a copy ?) because from what I read at https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html the ReleaseByteArrayElements routine should always be called. bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8215411 http://cr.openjdk.java.net/~mbaesken/webrevs/8215411.0/ Thanks, Matthias -------------- next part -------------- An HTML attachment was scrubbed... URL: From jcbeyler at google.com Mon Dec 17 17:25:41 2018 From: jcbeyler at google.com (JC Beyler) Date: Mon, 17 Dec 2018 09:25:41 -0800 Subject: RFR: [XS] 8215411: some GetByteArrayElements calls miss corresponding Release In-Reply-To: References: Message-ID: Hi Matthias, Your webrev looks good to me (not a reviewer though ;-)). I saw a few things about the isCopy updates but that is out of scope of this webrev, I'll be creating a separate one in a bit :-) Thanks, Jc On Mon, Dec 17, 2018 at 1:00 AM Baesken, Matthias wrote: > Hello, please review the following change. > I noticed that we miss at some places (for example in case of early > returns) > where GetByteArrayElements is used, the corresponding > ReleaseByteArrayElements call. > > In VirtualMachineImpl.c I also removed a check for isCopy (is the > returned byte array a copy ?) > because from what I read at > > > https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html > > the ReleaseByteArrayElements routine should always be called. > > > bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8215411 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8215411.0/ > > > Thanks, Matthias > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From jcbeyler at google.com Mon Dec 17 17:42:24 2018 From: jcbeyler at google.com (JC Beyler) Date: Mon, 17 Dec 2018 09:42:24 -0800 Subject: RFR (XS) 8215495: Always set isCopy Message-ID: Hi all, Could I get a review for this webrev: Webrev: http://cr.openjdk.java.net/~jcbeyler/8215495/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8215495 Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.daugherty at oracle.com Mon Dec 17 17:52:42 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 17 Dec 2018 12:52:42 -0500 Subject: RFR (XS) 8215495: Always set isCopy In-Reply-To: References: Message-ID: Adding hotspot-runtime-dev at ... since this is JNI. Dan On 12/17/18 12:42 PM, JC Beyler wrote: > Hi all, > > Could I get a review for this webrev: > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8215495/webrev.00/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8215495 > > Thanks, > Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Mon Dec 17 21:29:39 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 18 Dec 2018 07:29:39 +1000 Subject: RFR (XS) 8215495: Always set isCopy In-Reply-To: References: Message-ID: <3cf5ffda-6f26-54c9-2ea3-4977bc2e6859@oracle.com> Hi Jc, On 18/12/2018 3:42 am, JC Beyler wrote: > Hi all, > > Could I get a review for this webrev: > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8215495/webrev.00/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8215495 isCopy only has to be set if the method executes successfully "If isCopy is not NULL, then *isCopy is set to JNI_TRUE if a copy is made; or it is set to JNI_FALSE if no copy is made." You can only make (or not) a copy if the operation actually succeeds. So before checking isCopy the caller must check for NULL and/or a pending exception. I see no bug here. David ----- > Thanks, > Jc From jcbeyler at google.com Mon Dec 17 22:12:45 2018 From: jcbeyler at google.com (JC Beyler) Date: Mon, 17 Dec 2018 14:12:45 -0800 Subject: RFR (XS) 8215495: Always set isCopy In-Reply-To: <3cf5ffda-6f26-54c9-2ea3-4977bc2e6859@oracle.com> References: <3cf5ffda-6f26-54c9-2ea3-4977bc2e6859@oracle.com> Message-ID: Hi David, I understand the rationale behind the "If the method does return NULL, then isCopy's value is undefined". But what about the DEFINE_GETSCALARARRAYELEMENTS case? It does this: if (len == 0) { \ /* Empty array: legal but useless, can't return NULL. \ * Return a pointer to something useless. \ * Avoid asserts in typeArrayOop. */ \ result = (ElementType*)get_bad_address(); \ Should we at least then put isCopy to JNI_FALSE in that case since it does not return NULL and no exception is raised, Thanks, Jc On Mon, Dec 17, 2018 at 1:29 PM David Holmes wrote: > Hi Jc, > > On 18/12/2018 3:42 am, JC Beyler wrote: > > Hi all, > > > > Could I get a review for this webrev: > > > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8215495/webrev.00/ > > Bug: https://bugs.openjdk.java.net/browse/JDK-8215495 > > isCopy only has to be set if the method executes successfully > > "If isCopy is not NULL, then *isCopy is set to JNI_TRUE if a copy is > made; or it is set to JNI_FALSE if no copy is made." > > You can only make (or not) a copy if the operation actually succeeds. So > before checking isCopy the caller must check for NULL and/or a pending > exception. > > I see no bug here. > > David > ----- > > > Thanks, > > Jc > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Mon Dec 17 22:22:40 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 17 Dec 2018 14:22:40 -0800 Subject: RFR: JDK-8051349: nsk/jvmti/scenarios/sampling/SP06/sp06t003 fails in nightly In-Reply-To: <48289bc7-1d4a-6c61-b207-6a3214ff7e4a@oracle.com> References: <5C113D4D.7070503@oracle.com> <5C125DBC.6090307@oracle.com> <196f4645-b082-65f8-ff2d-0434a03a34ca@oracle.com> <5C13AFF7.2050201@oracle.com> <412971f0-a8e7-fab5-5dce-827aa9343e74@oracle.com> <48289bc7-1d4a-6c61-b207-6a3214ff7e4a@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Mon Dec 17 22:25:01 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 18 Dec 2018 08:25:01 +1000 Subject: RFR: [XS] 8215411: some GetByteArrayElements calls miss corresponding Release In-Reply-To: References: Message-ID: Hi Matthias, On 17/12/2018 6:59 pm, Baesken, Matthias wrote: > Hello, please review the following change. > I noticed that we miss at some places (for example in case of early returns) > where GetByteArrayElements is used, the corresponding ReleaseByteArrayElements call. > > In VirtualMachineImpl.c I also removed a check for isCopy (is the returned byte array a copy ?) > because from what I read at > > https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html > > the ReleaseByteArrayElements routine should always be called. That's not clear to me. My reading is that you only need to release if you have changes you need to ensure are written back and that is only needed if a copy was made. That said, the change seem semantically correct and harmless in this case. --- src/jdk.hotspot.agent/windows/native/libsaproc/sawindbg.cpp AFAICS there are still multiple exit paths via CHECK_EXCEPTION_(0) and THROW_NEW_DEBUGGER_EXCEPTION_ that won't release the buffer. Also you change seem wrong to me because it will commit the changes in the buffer even if we "abort" here: 735 if (bytesRead != numBytes) { 736 return 0; 737 } so it seems to me if you really want to release on all paths then the error paths should use a mode of JNI_ABORT and only the success path should use mode 0. --- src/java.base/macosx/native/libosxsecurity/KeystoreImpl.m This change seems okay, though again potentially not necessary - as we never commit any changes. Thanks, David ----- > > bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8215411 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8215411.0/ > > > Thanks, Matthias > From david.holmes at oracle.com Mon Dec 17 22:30:03 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 18 Dec 2018 08:30:03 +1000 Subject: RFR: JDK-8051349: nsk/jvmti/scenarios/sampling/SP06/sp06t003 fails in nightly In-Reply-To: References: <5C113D4D.7070503@oracle.com> <5C125DBC.6090307@oracle.com> <196f4645-b082-65f8-ff2d-0434a03a34ca@oracle.com> <5C13AFF7.2050201@oracle.com> <412971f0-a8e7-fab5-5dce-827aa9343e74@oracle.com> <48289bc7-1d4a-6c61-b207-6a3214ff7e4a@oracle.com> Message-ID: On 18/12/2018 8:22 am, Chris Plummer wrote: > I think you need to set interruptReady inside the synchronized block. > Otherwise checkReady() can still get to the interrupt() call before > testedMethod() gets to the wait(). That is not an issue. You can interrupt at any time independent of owning the monitor. The interrupt remains pending until the thread explicitly clears it, or an API, like wait() sees it, throws IE and clears it. David > Chris > > On 12/15/18 12:24 PM, gary.adams at oracle.com wrote: >> Updated webrev: >> >> http://cr.openjdk.java.net/~gadams/8051349/webrev.02/index.html >> >> On 12/15/18 8:15 AM, gary.adams at oracle.com wrote: >>> On 12/14/18 4:43 PM, Chris Plummer wrote: >>>> On 12/14/18 5:28 AM, Gary Adams wrote: >>>> >>>> BTW, I forgot to ask if you can fix the typo in the following comment: >>>> >>>> ?345???????????? /* check frame equalaty */ >>> >>> Yes, I fixed that one on Fri. >>> I usually save the typos for the final webrev. >>> >>>>> On 12/13/18, 11:51 AM, Chris Plummer wrote: >>>>>> Hi Gary, >>>>>> >>>>>> Unfortunately GetStackTrace() returns the top frame as the first >>>>>> frame in the array. Thus if the thread is not suspended and later >>>>>> you call GetFrameLocation() for some depth, there's no way to make >>>>>> sure the frame at that depth is the same frame at that depth in >>>>>> the array returned by GetStackTrace() unless you first suspend the >>>>>> frame before calling GetFrameLocation(). You would also need to >>>>>> call GetFrameCount() while suspended, compare that with the number >>>>>> of frames returned by GetStackTrace(), and make any needed depth >>>>>> adjustments. But since I think the intent is to not suspend the >>>>>> thread here, it probably does not make sense to do that, and >>>>>> instead accept that there might be some errors as you have done. >>>>> I'm not totally comfortable with these attempted operations and then >>>>> bypassing the test failure if the threads were not suspended. >>>>> Would it be better to only perform check? thread operations on the >>>>> suspended threads and to totally skip the operations that can not >>>>> be safely performed when the thread is not suspended. >>>> I was starting to think along those lines. What's the point of >>>> testing something if you know it can fail and you can't distinguish >>>> between expected and unexpected failure. The other option is to >>>> suspend like I describe above to make sure you can accurately verify >>>> the stack trace. I'm just not sure if it makes sense to do that. Why >>>> is the test run unsuspended in the first place? >>> The test runs several separate threads and each is presenting >>> a different scenario. It probably is good to try the various functions >>> about stack trace, frame counts and locations on suspended and resumed >>> threads. It is just a bit harder to determine what a failed condition >>> would be for the running threads. >>> >>>>>> >>>>>> One improvement I would like to see for your fix is to only ignore >>>>>> JVMTI_ERROR_NO_MORE_FRAMES rather than ignore all failures when >>>>>> suspended == NSK_TRUE. >>>>> The current layering of the macros and verify routines do not lend >>>>> themselves >>>>> to this sort of selective error checking. I'd probably file a >>>>> follow up bug to >>>>> address specific error checking. >>>> Can't you just bypass the use of NSK_VERIFY? Isn't JC getting rid of >>>> it anyway and using his JNI exception mechanism? In that case you >>>> would need to use the version that does not fail if there was an >>>> exception. >>> I do not know what set of error returns would be acceptable >>> or how to express that in the new mechanism. That's why I suggested >>> addressing it in a follow up bug. It simply would narrow the range >>> of error returns allowed. >>>>>> >>>>>> Also, I don't think you want the suspended check here: >>>>>> >>>>>> ?348???????? /* check if expected method frame found */ >>>>>> ?349???????? if ((suspended == NSK_TRUE) && (found <= 0)) { >>>>>> >>>>>> The check for finding the method is: >>>>>> >>>>>> ?341???????????? if (frameStack[j].method == threadsDesc[i].method) { >>>>>> >>>>>> Since frameStack[] is returned by GetStackTrace(), it is not >>>>>> impacted when not suspended, and the expected method should always >>>>>> be in frameStack[] somewhere. The issue is only with using >>>>>> GetFrameLocation() to correlate what is in the result of >>>>>> GetStackTrace(). >>>>> Agreed. The early continuation was bypassing the found method checking. >>>>> >>>>> BTW, another failure has been detected in sp06t001. This time the >>>>> threads are suspended, >>>>> but I believe there is a race between thread start and the call to >>>>> interrupt() the thread. >>>>> I think there may be some confusion over which thread is invoking >>>>> the interrupt() call. >>>>> It is running on the main thread from the code after a call to >>>>> start the thread, but the >>>>> thread may not have run when the interrupt is requested. >>>>> >>>>> public class sp06t001 extends DebugeeClass { >>>>> ... >>>>> >>>>> ??????? // create threads list >>>>> ??????? threads = new sp06t001Thread[] { >>>>> ??????????? new sp06t001ThreadRunning("threadRunning", log), >>>>> ??????????? new sp06t001ThreadEntering("threadEntering", log), >>>>> ??????????? new sp06t001ThreadWaiting("threadWaiting", log), >>>>> ??????????? new sp06t001ThreadSleeping("threadSleeping", log), >>>>> ??????????? new >>>>> sp06t001ThreadRunningInterrupted("threadRunningInterrupted", log), >>>>> ??????????? new sp06t001ThreadRunningNative("threadRunningNative", log) >>>>> ??????? }; >>>>> >>>>> ??????? // run threads >>>>> ??????? log.display("Starting tested threads"); >>>>> ??????? try { >>>>> ??????????? synchronized (endingMonitor) { >>>>> ??????????????? // start threads (except first one) >>>>> ??????????????? for (int i = 0; i < threads.length; i++) { >>>>> ??????????????????? threads[i].start(); >>>>> ??????????????????? if (!threads[i].checkReady()) { >>>>> ??????????????????????? throw new Failure("Unable to prepare thread >>>>> ..." >>>>> ... >>>>> >>>>> class sp06t001ThreadRunningInterrupted extends sp06t001Thread { >>>>> ... >>>>> ??? public boolean checkReady() { >>>>> ??????? // interrupt thread on wait() >>>>> ??????? synchronized (waitingMonitor) { >>>>> ??????????? interrupt(); >>>>> ??????? } >>>>> ??????? return true; >>>>> ??? } >>>> Certainly checkReady() could be called before the target thread has >>>> started the wait(). Then the thread does the wait() and probably >>>> never exits (or maybe the early interrupted is resulting in some >>>> other unexpected behavior). >>> If we get to the wait then the framecount would be fine. >>> >>> The debuggee class launches 6 threads. The agent is synced with the >>> debuggee >>> starting. I don't see the native agent synchronizing on the 6 started >>> threads, >>> when the threads are suspended and checked. >>> >>> As far as I can tell the call to interrupted() just sets a flag and >>> it is up to the thread >>> itself to decide how to interpret the isInterrupted() state. >>>> >>>> I think you need synchronize between the two threads. What if the >>>> target thread does a synchronized (waitingMonitor), sets a flag, and >>>> then a wait(). The main thread sleeps until the flag is set, does a >>>> synchronized(waitingMonitor), and then the interrupt(). >>> Since the test is already using a waiting monitor, I do not want to >>> disturb how it is being used in setting up the interrupted thread >>> test case. >>> The test is also using a "volatile boolean threadReady" for signalling >>> conditions in the test. >>> >>> On Fri I was experimenting with another boolean flag to delay the >>> checkReady until the testMethod was at the waiting monitor, >>> e.g. ready to be interrupted. It seems to work but I was waiting >>> for enough testruns before posting a revised webrev. >>> >>>> >>>> Chris >>>>>> >>>>>> thanks, >>>>>> >>>>>> Chris >>>>>> >>>>>> On 12/13/18 5:25 AM, Gary Adams wrote: >>>>>>> While testing I ran into another of the related failures that were >>>>>>> associated with the original bug. >>>>>>> The following fake exception stacktrace is for failure analysis. >>>>>>> nsk.share.Fake_Exception_for_RULE_Creation: (sp02t003.cpp:313) jvmti->GetFrameLocation(threadsDesc[i].thread, j, &qMethod, &qLocation) >>>>>>> at nsk_lvcomplain(nsk_tools.cpp:172) >>>>>>> # ERROR: sp02t003.cpp, 313: jvmti->GetFrameLocation(threadsDesc[i].thread, j, &qMethod, &qLocation) >>>>>>> # jvmti error: code=31, name=JVMTI_ERROR_NO_MORE_FRAMES >>>>>>> - sp02t003.cpp, 310: 3 frame: method: 0x7f225042b2d8, location: 16 >>>>>>> # ERROR: sp02t003.cpp, 313: jvmti->GetFrameLocation(threadsDesc[i].thread, j, &qMethod, &qLocation) >>>>>>> # jvmti error: code=31, name=JVMTI_ERROR_NO_MORE_FRAMES >>>>>>> # ERROR: sp02t003.cpp, 350: No expected method frame for not suspended thread #4 (threadRunningInterrupted) >>>>>>> In this particular failure, the GetFrameLocation call failed, >>>>>>> because the frame was no longer >>>>>>> accessible. >>>>>>> >>>>>>> If the threads are not suspended, the GetFrameLocation may fail, >>>>>>> or if it passes, the qMethod and qLocation could belong to another >>>>>>> frame. >>>>>>> >>>>>>> I've updated the webrev to allow the call to GetFrameLocation, >>>>>>> but to only >>>>>>> report an error if the threads are suspended. Similarly, the >>>>>>> checks to compare >>>>>>> qMethod and qLocation will be skipped, if the threads are not >>>>>>> suspended. >>>>>>> And the final comparison that the method was found will only be >>>>>>> an error >>>>>>> if the threads are suspended. >>>>>>> >>>>>>> ? Webrev: http://cr.openjdk.java.net/~gadams/8051349/webrev.01/ >>>>>>> >>>>>>> On 12/12/18, 11:54 AM, Gary Adams wrote: >>>>>>>> After some testing with nsk verbose messages enabled, >>>>>>>> it is clear that this test is failing in checkThreads when the >>>>>>>> location did not match between the call to GetStackTrace >>>>>>>> and GetFrameLocation. For the tests that are run when the threads >>>>>>>> have not been suspended, there is no guarantee the locations >>>>>>>> will match. >>>>>>>> >>>>>>>> ? Issue: https://bugs.openjdk.java.net/browse/JDK-8051349 >>>>>>>> ? Webrev: http://cr.openjdk.java.net/~gadams/8051349/webrev.00/ >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From david.holmes at oracle.com Mon Dec 17 22:39:34 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 18 Dec 2018 08:39:34 +1000 Subject: RFR (XS) 8215495: Always set isCopy In-Reply-To: References: <3cf5ffda-6f26-54c9-2ea3-4977bc2e6859@oracle.com> Message-ID: <6d5ccd78-37f6-5f3e-c651-6543ea2fa0d9@oracle.com> Hi Jc, On 18/12/2018 8:12 am, JC Beyler wrote: > Hi David, > > I understand the rationale behind the "If the method does return NULL, > then isCopy's value is undefined". But what about > the?DEFINE_GETSCALARARRAYELEMENTS case? > > It does this: > ? if (len == 0) { \ > ? ? /* Empty array: legal but useless, can't return NULL. \ > ? ? ?* Return a pointer to something useless. \ > ? ? ?* Avoid asserts in typeArrayOop. */ \ > ? ? result = (ElementType*)get_bad_address(); \ > > Should we at least then put isCopy to JNI_FALSE in that case since it > does not return NULL and no exception is raised, Sure - it's an odd corner case, but better not to leave isCopy potentially uninitialized. Thanks, David > Thanks, > Jc > > > On Mon, Dec 17, 2018 at 1:29 PM David Holmes > wrote: > > Hi Jc, > > On 18/12/2018 3:42 am, JC Beyler wrote: > > Hi all, > > > > Could I get a review for this webrev: > > > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8215495/webrev.00/ > > Bug: https://bugs.openjdk.java.net/browse/JDK-8215495 > > isCopy only has to be set if the method executes successfully > > "If isCopy is not NULL, then *isCopy is set to JNI_TRUE if a copy is > made; or it is set to JNI_FALSE if no copy is made." > > You can only make (or not) a copy if the operation actually > succeeds. So > before checking isCopy the caller must check for NULL and/or a pending > exception. > > I see no bug here. > > David > ----- > > > Thanks, > > Jc > > > > -- > > Thanks, > Jc From jcbeyler at google.com Mon Dec 17 22:53:56 2018 From: jcbeyler at google.com (JC Beyler) Date: Mon, 17 Dec 2018 14:53:56 -0800 Subject: RFR (XS) 8215495: Always set isCopy In-Reply-To: <6d5ccd78-37f6-5f3e-c651-6543ea2fa0d9@oracle.com> References: <3cf5ffda-6f26-54c9-2ea3-4977bc2e6859@oracle.com> <6d5ccd78-37f6-5f3e-c651-6543ea2fa0d9@oracle.com> Message-ID: Sounds good to me: For the odd corner case: Webrev: http://cr.openjdk.java.net/~jcbeyler/8215495/webrev.01/ Bug: https://bugs.openjdk.java.net/browse/JDK-8215495 Thanks! Jc On Mon, Dec 17, 2018 at 2:39 PM David Holmes wrote: > Hi Jc, > > On 18/12/2018 8:12 am, JC Beyler wrote: > > Hi David, > > > > I understand the rationale behind the "If the method does return NULL, > > then isCopy's value is undefined". But what about > > the DEFINE_GETSCALARARRAYELEMENTS case? > > > > It does this: > > if (len == 0) { \ > > /* Empty array: legal but useless, can't return NULL. \ > > * Return a pointer to something useless. \ > > * Avoid asserts in typeArrayOop. */ \ > > result = (ElementType*)get_bad_address(); \ > > > > Should we at least then put isCopy to JNI_FALSE in that case since it > > does not return NULL and no exception is raised, > > Sure - it's an odd corner case, but better not to leave isCopy > potentially uninitialized. > > Thanks, > David > > > Thanks, > > Jc > > > > > > On Mon, Dec 17, 2018 at 1:29 PM David Holmes > > wrote: > > > > Hi Jc, > > > > On 18/12/2018 3:42 am, JC Beyler wrote: > > > Hi all, > > > > > > Could I get a review for this webrev: > > > > > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8215495/webrev.00/ > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8215495 > > > > isCopy only has to be set if the method executes successfully > > > > "If isCopy is not NULL, then *isCopy is set to JNI_TRUE if a copy is > > made; or it is set to JNI_FALSE if no copy is made." > > > > You can only make (or not) a copy if the operation actually > > succeeds. So > > before checking isCopy the caller must check for NULL and/or a > pending > > exception. > > > > I see no bug here. > > > > David > > ----- > > > > > Thanks, > > > Jc > > > > > > > > -- > > > > Thanks, > > Jc > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Mon Dec 17 23:38:43 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 17 Dec 2018 15:38:43 -0800 Subject: RFR: JDK-8051349: nsk/jvmti/scenarios/sampling/SP06/sp06t003 fails in nightly In-Reply-To: References: <5C113D4D.7070503@oracle.com> <5C125DBC.6090307@oracle.com> <196f4645-b082-65f8-ff2d-0434a03a34ca@oracle.com> <5C13AFF7.2050201@oracle.com> <412971f0-a8e7-fab5-5dce-827aa9343e74@oracle.com> <48289bc7-1d4a-6c61-b207-6a3214ff7e4a@oracle.com> Message-ID: On 12/17/18 2:30 PM, David Holmes wrote: > On 18/12/2018 8:22 am, Chris Plummer wrote: >> I think you need to set interruptReady inside the synchronized block. >> Otherwise checkReady() can still get to the interrupt() call before >> testedMethod() gets to the wait(). > > That is not an issue. You can interrupt at any time independent of > owning the monitor. The interrupt remains pending until the thread > explicitly clears it, or an API, like wait() sees it, throws IE and > clears it. Ok. In that case LGTM. Chris > > David > >> Chris >> >> On 12/15/18 12:24 PM, gary.adams at oracle.com wrote: >>> Updated webrev: >>> >>> http://cr.openjdk.java.net/~gadams/8051349/webrev.02/index.html >>> >>> On 12/15/18 8:15 AM, gary.adams at oracle.com wrote: >>>> On 12/14/18 4:43 PM, Chris Plummer wrote: >>>>> On 12/14/18 5:28 AM, Gary Adams wrote: >>>>> >>>>> BTW, I forgot to ask if you can fix the typo in the following >>>>> comment: >>>>> >>>>> ?345???????????? /* check frame equalaty */ >>>> >>>> Yes, I fixed that one on Fri. >>>> I usually save the typos for the final webrev. >>>> >>>>>> On 12/13/18, 11:51 AM, Chris Plummer wrote: >>>>>>> Hi Gary, >>>>>>> >>>>>>> Unfortunately GetStackTrace() returns the top frame as the first >>>>>>> frame in the array. Thus if the thread is not suspended and >>>>>>> later you call GetFrameLocation() for some depth, there's no way >>>>>>> to make sure the frame at that depth is the same frame at that >>>>>>> depth in the array returned by GetStackTrace() unless you first >>>>>>> suspend the frame before calling GetFrameLocation(). You would >>>>>>> also need to call GetFrameCount() while suspended, compare that >>>>>>> with the number of frames returned by GetStackTrace(), and make >>>>>>> any needed depth adjustments. But since I think the intent is to >>>>>>> not suspend the thread here, it probably does not make sense to >>>>>>> do that, and instead accept that there might be some errors as >>>>>>> you have done. >>>>>> I'm not totally comfortable with these attempted operations and then >>>>>> bypassing the test failure if the threads were not suspended. >>>>>> Would it be better to only perform check? thread operations on the >>>>>> suspended threads and to totally skip the operations that can not >>>>>> be safely performed when the thread is not suspended. >>>>> I was starting to think along those lines. What's the point of >>>>> testing something if you know it can fail and you can't >>>>> distinguish between expected and unexpected failure. The other >>>>> option is to suspend like I describe above to make sure you can >>>>> accurately verify the stack trace. I'm just not sure if it makes >>>>> sense to do that. Why is the test run unsuspended in the first place? >>>> The test runs several separate threads and each is presenting >>>> a different scenario. It probably is good to try the various functions >>>> about stack trace, frame counts and locations on suspended and resumed >>>> threads. It is just a bit harder to determine what a failed condition >>>> would be for the running threads. >>>> >>>>>>> >>>>>>> One improvement I would like to see for your fix is to only >>>>>>> ignore JVMTI_ERROR_NO_MORE_FRAMES rather than ignore all >>>>>>> failures when suspended == NSK_TRUE. >>>>>> The current layering of the macros and verify routines do not >>>>>> lend themselves >>>>>> to this sort of selective error checking. I'd probably file a >>>>>> follow up bug to >>>>>> address specific error checking. >>>>> Can't you just bypass the use of NSK_VERIFY? Isn't JC getting rid >>>>> of it anyway and using his JNI exception mechanism? In that case >>>>> you would need to use the version that does not fail if there was >>>>> an exception. >>>> I do not know what set of error returns would be acceptable >>>> or how to express that in the new mechanism. That's why I suggested >>>> addressing it in a follow up bug. It simply would narrow the range >>>> of error returns allowed. >>>>>>> >>>>>>> Also, I don't think you want the suspended check here: >>>>>>> >>>>>>> ?348???????? /* check if expected method frame found */ >>>>>>> ?349???????? if ((suspended == NSK_TRUE) && (found <= 0)) { >>>>>>> >>>>>>> The check for finding the method is: >>>>>>> >>>>>>> ?341???????????? if (frameStack[j].method == >>>>>>> threadsDesc[i].method) { >>>>>>> >>>>>>> Since frameStack[] is returned by GetStackTrace(), it is not >>>>>>> impacted when not suspended, and the expected method should >>>>>>> always be in frameStack[] somewhere. The issue is only with >>>>>>> using GetFrameLocation() to correlate what is in the result of >>>>>>> GetStackTrace(). >>>>>> Agreed. The early continuation was bypassing the found method >>>>>> checking. >>>>>> >>>>>> BTW, another failure has been detected in sp06t001. This time the >>>>>> threads are suspended, >>>>>> but I believe there is a race between thread start and the call >>>>>> to interrupt() the thread. >>>>>> I think there may be some confusion over which thread is invoking >>>>>> the interrupt() call. >>>>>> It is running on the main thread from the code after a call to >>>>>> start the thread, but the >>>>>> thread may not have run when the interrupt is requested. >>>>>> >>>>>> public class sp06t001 extends DebugeeClass { >>>>>> ... >>>>>> >>>>>> ??????? // create threads list >>>>>> ??????? threads = new sp06t001Thread[] { >>>>>> ??????????? new sp06t001ThreadRunning("threadRunning", log), >>>>>> ??????????? new sp06t001ThreadEntering("threadEntering", log), >>>>>> ??????????? new sp06t001ThreadWaiting("threadWaiting", log), >>>>>> ??????????? new sp06t001ThreadSleeping("threadSleeping", log), >>>>>> ??????????? new >>>>>> sp06t001ThreadRunningInterrupted("threadRunningInterrupted", log), >>>>>> ??????????? new >>>>>> sp06t001ThreadRunningNative("threadRunningNative", log) >>>>>> ??????? }; >>>>>> >>>>>> ??????? // run threads >>>>>> ??????? log.display("Starting tested threads"); >>>>>> ??????? try { >>>>>> ??????????? synchronized (endingMonitor) { >>>>>> ??????????????? // start threads (except first one) >>>>>> ??????????????? for (int i = 0; i < threads.length; i++) { >>>>>> ??????????????????? threads[i].start(); >>>>>> ??????????????????? if (!threads[i].checkReady()) { >>>>>> ??????????????????????? throw new Failure("Unable to prepare >>>>>> thread ..." >>>>>> ... >>>>>> >>>>>> class sp06t001ThreadRunningInterrupted extends sp06t001Thread { >>>>>> ... >>>>>> ??? public boolean checkReady() { >>>>>> ??????? // interrupt thread on wait() >>>>>> ??????? synchronized (waitingMonitor) { >>>>>> ??????????? interrupt(); >>>>>> ??????? } >>>>>> ??????? return true; >>>>>> ??? } >>>>> Certainly checkReady() could be called before the target thread >>>>> has started the wait(). Then the thread does the wait() and >>>>> probably never exits (or maybe the early interrupted is resulting >>>>> in some other unexpected behavior). >>>> If we get to the wait then the framecount would be fine. >>>> >>>> The debuggee class launches 6 threads. The agent is synced with the >>>> debuggee >>>> starting. I don't see the native agent synchronizing on the 6 >>>> started threads, >>>> when the threads are suspended and checked. >>>> >>>> As far as I can tell the call to interrupted() just sets a flag and >>>> it is up to the thread >>>> itself to decide how to interpret the isInterrupted() state. >>>>> >>>>> I think you need synchronize between the two threads. What if the >>>>> target thread does a synchronized (waitingMonitor), sets a flag, >>>>> and then a wait(). The main thread sleeps until the flag is set, >>>>> does a synchronized(waitingMonitor), and then the interrupt(). >>>> Since the test is already using a waiting monitor, I do not want to >>>> disturb how it is being used in setting up the interrupted thread >>>> test case. >>>> The test is also using a "volatile boolean threadReady" for signalling >>>> conditions in the test. >>>> >>>> On Fri I was experimenting with another boolean flag to delay the >>>> checkReady until the testMethod was at the waiting monitor, >>>> e.g. ready to be interrupted. It seems to work but I was waiting >>>> for enough testruns before posting a revised webrev. >>>> >>>>> >>>>> Chris >>>>>>> >>>>>>> thanks, >>>>>>> >>>>>>> Chris >>>>>>> >>>>>>> On 12/13/18 5:25 AM, Gary Adams wrote: >>>>>>>> While testing I ran into another of the related failures that were >>>>>>>> associated with the original bug. >>>>>>>> The following fake exception stacktrace is for failure analysis. >>>>>>>> nsk.share.Fake_Exception_for_RULE_Creation: (sp02t003.cpp:313) >>>>>>>> jvmti->GetFrameLocation(threadsDesc[i].thread, j, &qMethod, >>>>>>>> &qLocation) >>>>>>>> ????at nsk_lvcomplain(nsk_tools.cpp:172) >>>>>>>> # ERROR: sp02t003.cpp, 313: >>>>>>>> jvmti->GetFrameLocation(threadsDesc[i].thread, j, &qMethod, >>>>>>>> &qLocation) >>>>>>>> #?? jvmti error: code=31, name=JVMTI_ERROR_NO_MORE_FRAMES >>>>>>>> - sp02t003.cpp, 310:?????? 3 frame: method: 0x7f225042b2d8, >>>>>>>> location: 16 >>>>>>>> # ERROR: sp02t003.cpp, 313: >>>>>>>> jvmti->GetFrameLocation(threadsDesc[i].thread, j, &qMethod, >>>>>>>> &qLocation) >>>>>>>> #?? jvmti error: code=31, name=JVMTI_ERROR_NO_MORE_FRAMES >>>>>>>> # ERROR: sp02t003.cpp, 350: No expected method frame for not >>>>>>>> suspended thread #4 (threadRunningInterrupted) >>>>>>>> In this particular failure, the GetFrameLocation call failed, >>>>>>>> because the frame was no longer >>>>>>>> accessible. >>>>>>>> >>>>>>>> If the threads are not suspended, the GetFrameLocation may fail, >>>>>>>> or if it passes, the qMethod and qLocation could belong to another >>>>>>>> frame. >>>>>>>> >>>>>>>> I've updated the webrev to allow the call to GetFrameLocation, >>>>>>>> but to only >>>>>>>> report an error if the threads are suspended. Similarly, the >>>>>>>> checks to compare >>>>>>>> qMethod and qLocation will be skipped, if the threads are not >>>>>>>> suspended. >>>>>>>> And the final comparison that the method was found will only be >>>>>>>> an error >>>>>>>> if the threads are suspended. >>>>>>>> >>>>>>>> ? Webrev: http://cr.openjdk.java.net/~gadams/8051349/webrev.01/ >>>>>>>> >>>>>>>> On 12/12/18, 11:54 AM, Gary Adams wrote: >>>>>>>>> After some testing with nsk verbose messages enabled, >>>>>>>>> it is clear that this test is failing in checkThreads when the >>>>>>>>> location did not match between the call to GetStackTrace >>>>>>>>> and GetFrameLocation. For the tests that are run when the threads >>>>>>>>> have not been suspended, there is no guarantee the locations >>>>>>>>> will match. >>>>>>>>> >>>>>>>>> ? Issue: https://bugs.openjdk.java.net/browse/JDK-8051349 >>>>>>>>> ? Webrev: http://cr.openjdk.java.net/~gadams/8051349/webrev.00/ >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> From david.holmes at oracle.com Tue Dec 18 00:19:51 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 18 Dec 2018 10:19:51 +1000 Subject: RFR: [XS] 8215411: some GetByteArrayElements calls miss corresponding Release In-Reply-To: References: Message-ID: <164fa449-28f6-e61b-5a6d-d3182a4c7e32@oracle.com> Correction ... On 18/12/2018 8:25 am, David Holmes wrote: > Hi Matthias, > > On 17/12/2018 6:59 pm, Baesken, Matthias wrote: >> Hello, please review the following change. >> I noticed that we miss at some places (for example in case of early >> returns) >> where GetByteArrayElements is used,? the corresponding >> ReleaseByteArrayElements? call. >> >> In VirtualMachineImpl.c? I also removed a check for isCopy (is the >> returned byte array a copy ?) >> because from what I read at >> >> https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html >> >> >> the ReleaseByteArrayElements? routine should always be called. > > That's not clear to me. My reading is that you only need to release if > you have changes you need to ensure are written back and that is only > needed if a copy was made. Jc pointed out to me that if a copy is made you may need to release to avoid a memory leak. But that is where the mode flags come in - and again only relevant if a copy was made. But as per: https://developer.android.com/training/articles/perf-jni if a copy is not made and pinning is used (as with the "critical" variants) then you also need to release to un-pin. So code should call release, with an appropriate mode, based on whether isCopy was true**, and whether changed data should be written back. ** mode is ignored if not working on a copy so you can just set it assuming a copy was made. Note that the hotspot implementation always makes a copy for GextXXXArrayElements, and the ReleaseXXXArrayElements knows this and so will always free the buffer that is passed in (other than for mode JNI_COMMIT of course). Cheers, David ----- > That said, the change seem semantically correct and harmless in this case. > --- > > src/jdk.hotspot.agent/windows/native/libsaproc/sawindbg.cpp > > AFAICS there are still multiple exit paths via CHECK_EXCEPTION_(0) and > THROW_NEW_DEBUGGER_EXCEPTION_ that won't release the buffer. > > Also you change seem wrong to me because it will commit the changes in > the buffer even if we "abort" here: > > 735?? if (bytesRead != numBytes) { > ?736????? return 0; > ?737?? } > > so it seems to me if you really want to release on all paths then the > error paths should use a mode of JNI_ABORT and only the success path > should use mode 0. > > --- > > ?src/java.base/macosx/native/libosxsecurity/KeystoreImpl.m > > This change seems okay, though again potentially not necessary - as we > never commit any changes. > > Thanks, > David > ----- > >> >> bug/webrev : >> >> https://bugs.openjdk.java.net/browse/JDK-8215411 >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8215411.0/ >> >> >> Thanks, Matthias >> From david.holmes at oracle.com Tue Dec 18 00:25:23 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 18 Dec 2018 10:25:23 +1000 Subject: RFR (XS) 8215495: Always set isCopy In-Reply-To: References: <3cf5ffda-6f26-54c9-2ea3-4977bc2e6859@oracle.com> <6d5ccd78-37f6-5f3e-c651-6543ea2fa0d9@oracle.com> Message-ID: <499e2b8d-fa8d-2e8b-24ff-0a42500c91a7@oracle.com> LGTM. Thanks, David On 18/12/2018 8:53 am, JC Beyler wrote: > Sounds good to me: > > For the odd corner case: > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8215495/webrev.01/ > > Bug: https://bugs.openjdk.java.net/browse/JDK-8215495 > > Thanks! > Jc > > On Mon, Dec 17, 2018 at 2:39 PM David Holmes > wrote: > > Hi Jc, > > On 18/12/2018 8:12 am, JC Beyler wrote: > > Hi David, > > > > I understand the rationale behind the "If the method does return > NULL, > > then isCopy's value is undefined". But what about > > the?DEFINE_GETSCALARARRAYELEMENTS case? > > > > It does this: > >? ? if (len == 0) { \ > >? ? ? /* Empty array: legal but useless, can't return NULL. \ > >? ? ? ?* Return a pointer to something useless. \ > >? ? ? ?* Avoid asserts in typeArrayOop. */ \ > >? ? ? result = (ElementType*)get_bad_address(); \ > > > > Should we at least then put isCopy to JNI_FALSE in that case > since it > > does not return NULL and no exception is raised, > > Sure - it's an odd corner case, but better not to leave isCopy > potentially uninitialized. > > Thanks, > David > > > Thanks, > > Jc > > > > > > On Mon, Dec 17, 2018 at 1:29 PM David Holmes > > > >> wrote: > > > >? ? ?Hi Jc, > > > >? ? ?On 18/12/2018 3:42 am, JC Beyler wrote: > >? ? ? > Hi all, > >? ? ? > > >? ? ? > Could I get a review for this webrev: > >? ? ? > > >? ? ? > Webrev: > http://cr.openjdk.java.net/~jcbeyler/8215495/webrev.00/ > >? ? ? > Bug: https://bugs.openjdk.java.net/browse/JDK-8215495 > > > >? ? ?isCopy only has to be set if the method executes successfully > > > >? ? ?"If isCopy is not NULL, then *isCopy is set to JNI_TRUE if a > copy is > >? ? ?made; or it is set to JNI_FALSE if no copy is made." > > > >? ? ?You can only make (or not) a copy if the operation actually > >? ? ?succeeds. So > >? ? ?before checking isCopy the caller must check for NULL and/or > a pending > >? ? ?exception. > > > >? ? ?I see no bug here. > > > >? ? ?David > >? ? ?----- > > > >? ? ? > Thanks, > >? ? ? > Jc > > > > > > > > -- > > > > Thanks, > > Jc > > > > -- > > Thanks, > Jc From matthias.baesken at sap.com Tue Dec 18 08:52:46 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 18 Dec 2018 08:52:46 +0000 Subject: RFR: [XS] 8215411: some GetByteArrayElements calls miss corresponding Release In-Reply-To: <164fa449-28f6-e61b-5a6d-d3182a4c7e32@oracle.com> References: <164fa449-28f6-e61b-5a6d-d3182a4c7e32@oracle.com> Message-ID: Hi JC / David, thanks for the input . I'm not really sure what you want me to change David, am I right that I can keep the changes to src/java.base/macosx/native/libosxsecurity/KeystoreImpl.m src/jdk.attach/windows/native/libattach/VirtualMachineImpl.c but adjust src/jdk.hotspot.agent/windows/native/libsaproc/sawindbg.cpp to also handle theis potential early return 723 IDebugDataSpaces* ptrIDebugDataSpaces = (IDebugDataSpaces*) env->GetLongField(obj, 724 ptrIDebugDataSpaces_ID); 725 CHECK_EXCEPTION_(0); ? > > Also you change seem wrong to me because it will commit the changes in > > the buffer even if we "abort" here: > > > > 735 if (bytesRead != numBytes) { > > 736 return 0; > > 737 } > > The spec says : https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html >ReleaseArrayElements Routines >void ReleaseArrayElements(JNIEnv *env, ArrayType array, NativeType *elems, jint mode); > >A family of functions that informs the VM that the native code no longer needs access to elems. The elems argument is a pointer derived from array using the corresponding > GetArrayElements() function. If necessary, this function copies back all changes made to elems to the original array. So shouldn't I tell the VM , that the native code no longer needs access to bytePtr before returning here 735 if (bytesRead != numBytes) { 736 return 0; 737 } ? Best regards, Matthias > -----Original Message----- > From: David Holmes > Sent: Dienstag, 18. Dezember 2018 01:20 > To: Baesken, Matthias ; 'hotspot- > dev at openjdk.java.net' ; serviceability- > dev at openjdk.java.net; security-dev at openjdk.java.net > Subject: Re: RFR: [XS] 8215411: some GetByteArrayElements calls miss > corresponding Release > > Correction ... > > On 18/12/2018 8:25 am, David Holmes wrote: > > Hi Matthias, > > > > On 17/12/2018 6:59 pm, Baesken, Matthias wrote: > >> Hello, please review the following change. > >> I noticed that we miss at some places (for example in case of early > >> returns) > >> where GetByteArrayElements is used,? the corresponding > >> ReleaseByteArrayElements? call. > >> > >> In VirtualMachineImpl.c? I also removed a check for isCopy (is the > >> returned byte array a copy ?) > >> because from what I read at > >> > >> > https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/function > s.html > >> > >> > >> the ReleaseByteArrayElements? routine should always be called. > > > > That's not clear to me. My reading is that you only need to release if > > you have changes you need to ensure are written back and that is only > > needed if a copy was made. > > Jc pointed out to me that if a copy is made you may need to release to > avoid a memory leak. But that is where the mode flags come in - and > again only relevant if a copy was made. > > But as per: > > https://developer.android.com/training/articles/perf-jni > > if a copy is not made and pinning is used (as with the "critical" > variants) then you also need to release to un-pin. > > So code should call release, with an appropriate mode, based on whether > isCopy was true**, and whether changed data should be written back. > > ** mode is ignored if not working on a copy so you can just set it > assuming a copy was made. > > Note that the hotspot implementation always makes a copy for > GextXXXArrayElements, and the ReleaseXXXArrayElements knows this and > so > will always free the buffer that is passed in (other than for mode > JNI_COMMIT of course). > > Cheers, > David > ----- > > > That said, the change seem semantically correct and harmless in this case. > > --- > > > > src/jdk.hotspot.agent/windows/native/libsaproc/sawindbg.cpp > > > > AFAICS there are still multiple exit paths via CHECK_EXCEPTION_(0) and > > THROW_NEW_DEBUGGER_EXCEPTION_ that won't release the buffer. > > > > Also you change seem wrong to me because it will commit the changes in > > the buffer even if we "abort" here: > > > > 735?? if (bytesRead != numBytes) { > > ?736????? return 0; > > ?737?? } > > > > so it seems to me if you really want to release on all paths then the > > error paths should use a mode of JNI_ABORT and only the success path > > should use mode 0. > > > > --- > > > > ?src/java.base/macosx/native/libosxsecurity/KeystoreImpl.m > > > > This change seems okay, though again potentially not necessary - as we > > never commit any changes. > > > > Thanks, > > David > > ----- > > > >> > >> bug/webrev : > >> > >> https://bugs.openjdk.java.net/browse/JDK-8215411 > >> > >> http://cr.openjdk.java.net/~mbaesken/webrevs/8215411.0/ > >> > >> > >> Thanks, Matthias > >> From zanglin5 at jd.com Tue Dec 18 08:57:28 2018 From: zanglin5 at jd.com (=?gb2312?B?6rDB1Q==?=) Date: Tue, 18 Dec 2018 08:57:28 +0000 Subject: [PATCH] 8214535: Extend JMap to support parallel and incremental heap scanning Message-ID: Hi All, I am preparing the patch for https://bugs.openjdk.java.net/browse/JDK-8214535, and want to get your suggestions. To make the patch easier to review. I plan to make 3 patches as following: 1. one patch for enabling file dump of "jmap -histo".(http://cr.openjdk.java.net/~xiaofeya/JDK-8214535/file_dump/webrev.00/webrev/) 2. one patch for incremental dump intermediate data to file of "jmap -histo".(http://cr.openjdk.java.net/~xiaofeya/JDK-8214535/incremental/webrev.00/) 3. one patch for parallel iterating heap of "jmap -histo". (WIP) And the patches for item 1 & 2 are ready. patch for 3 is WIP. May I ask your help to review these patches? Thanks. BRs, Lin -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Tue Dec 18 09:03:48 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 18 Dec 2018 19:03:48 +1000 Subject: RFR: [XS] 8215411: some GetByteArrayElements calls miss corresponding Release In-Reply-To: References: <164fa449-28f6-e61b-5a6d-d3182a4c7e32@oracle.com> Message-ID: <7c2f5dc2-9bbe-29f9-8be0-2613267f60dc@oracle.com> Hi Matthias, On 18/12/2018 6:52 pm, Baesken, Matthias wrote: > Hi JC / David, thanks for the input . > > I'm not really sure what you want me to change David, am I right that I can keep the changes to > > src/java.base/macosx/native/libosxsecurity/KeystoreImpl.m > src/jdk.attach/windows/native/libattach/VirtualMachineImpl.c Yes. > but adjust > > src/jdk.hotspot.agent/windows/native/libsaproc/sawindbg.cpp > > to also handle theis potential early return > > 723 IDebugDataSpaces* ptrIDebugDataSpaces = (IDebugDataSpaces*) env->GetLongField(obj, > 724 ptrIDebugDataSpaces_ID); > 725 CHECK_EXCEPTION_(0); > > ? And I think: 730 THROW_NEW_DEBUGGER_EXCEPTION_("Windbg Error: ReadVirtual failed!", 0); as I assume that won't actually return normally. >>> Also you change seem wrong to me because it will commit the changes in >>> the buffer even if we "abort" here: >>> >>> 735 if (bytesRead != numBytes) { >>> 736 return 0; >>> 737 } >>> > > The spec says : > > https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html > >> ReleaseArrayElements Routines >> void ReleaseArrayElements(JNIEnv *env, ArrayType array, NativeType *elems, jint mode); >> >> A family of functions that informs the VM that the native code no longer needs access to elems. The elems argument is a pointer derived from array using the corresponding >> GetArrayElements() function. If necessary, this function copies back all changes made to elems to the original array. > > So shouldn't I tell the VM , that the native code no longer needs access to bytePtr before returning here > > 735 if (bytesRead != numBytes) { > 736 return 0; > 737 } > > ? There are two aspects to release: 1. the actual release (or "I don't need this any more") 2. committing any changes made back to the original array This code does: 728 if (ptrIDebugDataSpaces->ReadVirtual((ULONG64) address, (PVOID) bytePtr, 729 (ULONG)numBytes, &bytesRead) != S_OK) { which writes into the array. It then checks if we wrote what we expected: 735 if (bytesRead != numBytes) { 736 return 0; 737 } If we didn't read what was expected what should happen to the original array? Should it be left untouched or updated with the unexpected input? I would say left untouched and that is what the original code did. If everything succeeds you should do the release with mode 0; but if taking an error exit the release should use mode JNI_ABORT so no changes are written back. Cheers, David > Best regards, Matthias > > >> -----Original Message----- >> From: David Holmes >> Sent: Dienstag, 18. Dezember 2018 01:20 >> To: Baesken, Matthias ; 'hotspot- >> dev at openjdk.java.net' ; serviceability- >> dev at openjdk.java.net; security-dev at openjdk.java.net >> Subject: Re: RFR: [XS] 8215411: some GetByteArrayElements calls miss >> corresponding Release >> >> Correction ... >> >> On 18/12/2018 8:25 am, David Holmes wrote: >>> Hi Matthias, >>> >>> On 17/12/2018 6:59 pm, Baesken, Matthias wrote: >>>> Hello, please review the following change. >>>> I noticed that we miss at some places (for example in case of early >>>> returns) >>>> where GetByteArrayElements is used,? the corresponding >>>> ReleaseByteArrayElements? call. >>>> >>>> In VirtualMachineImpl.c? I also removed a check for isCopy (is the >>>> returned byte array a copy ?) >>>> because from what I read at >>>> >>>> >> https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/function >> s.html >>>> >>>> >>>> the ReleaseByteArrayElements? routine should always be called. >>> >>> That's not clear to me. My reading is that you only need to release if >>> you have changes you need to ensure are written back and that is only >>> needed if a copy was made. >> >> Jc pointed out to me that if a copy is made you may need to release to >> avoid a memory leak. But that is where the mode flags come in - and >> again only relevant if a copy was made. >> >> But as per: >> >> https://developer.android.com/training/articles/perf-jni >> >> if a copy is not made and pinning is used (as with the "critical" >> variants) then you also need to release to un-pin. >> >> So code should call release, with an appropriate mode, based on whether >> isCopy was true**, and whether changed data should be written back. >> >> ** mode is ignored if not working on a copy so you can just set it >> assuming a copy was made. >> >> Note that the hotspot implementation always makes a copy for >> GextXXXArrayElements, and the ReleaseXXXArrayElements knows this and >> so >> will always free the buffer that is passed in (other than for mode >> JNI_COMMIT of course). >> >> Cheers, >> David >> ----- >> >>> That said, the change seem semantically correct and harmless in this case. >>> --- >>> >>> src/jdk.hotspot.agent/windows/native/libsaproc/sawindbg.cpp >>> >>> AFAICS there are still multiple exit paths via CHECK_EXCEPTION_(0) and >>> THROW_NEW_DEBUGGER_EXCEPTION_ that won't release the buffer. >>> >>> Also you change seem wrong to me because it will commit the changes in >>> the buffer even if we "abort" here: >>> >>> 735?? if (bytesRead != numBytes) { >>> ?736????? return 0; >>> ?737?? } >>> >>> so it seems to me if you really want to release on all paths then the >>> error paths should use a mode of JNI_ABORT and only the success path >>> should use mode 0. >>> >>> --- >>> >>> ?src/java.base/macosx/native/libosxsecurity/KeystoreImpl.m >>> >>> This change seems okay, though again potentially not necessary - as we >>> never commit any changes. >>> >>> Thanks, >>> David >>> ----- >>> >>>> >>>> bug/webrev : >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8215411 >>>> >>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8215411.0/ >>>> >>>> >>>> Thanks, Matthias >>>> From jcbeyler at google.com Tue Dec 18 16:56:10 2018 From: jcbeyler at google.com (JC Beyler) Date: Tue, 18 Dec 2018 08:56:10 -0800 Subject: [PATCH] 8214535: Extend JMap to support parallel and incremental heap scanning In-Reply-To: References: Message-ID: Hi Lin, Would it not be easier to make 3 different JBS items that are all enhancements? Or perhaps 3 subtasks to the original JDK-8214535? Then you could send out a request for review for the first two and we can talk about the third. I think all three parts can be considered separately (and it seems you do too). Thanks, Jc On Tue, Dec 18, 2018 at 12:58 AM ?? wrote: > Hi All, > > I am preparing the patch for > https://bugs.openjdk.java.net/browse/JDK-8214535, and want to get your > suggestions. > > > To make the patch easier to review. I plan to make 3 patches as > following: > > 1. one patch for enabling file dump of "jmap -histo".( > http://cr.openjdk.java.net/~xiaofeya/JDK-8214535/file_dump/webrev.00/webrev/ > ) > > 2. one patch for incremental dump intermediate data to file of "jmap > -histo".( > http://cr.openjdk.java.net/~xiaofeya/JDK-8214535/incremental/webrev.00/) > > 3. one patch for parallel iterating heap of "jmap -histo". (WIP) > > > And the patches for item 1 & 2 are ready. patch for 3 is WIP. > > May I ask your help to review these patches? > > Thanks. > > > BRs, > > Lin > > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthias.baesken at sap.com Tue Dec 18 17:19:21 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 18 Dec 2018 17:19:21 +0000 Subject: RFR: [XS] 8215411: some GetByteArrayElements calls miss corresponding Release In-Reply-To: <7c2f5dc2-9bbe-29f9-8be0-2613267f60dc@oracle.com> References: <164fa449-28f6-e61b-5a6d-d3182a4c7e32@oracle.com> <7c2f5dc2-9bbe-29f9-8be0-2613267f60dc@oracle.com> Message-ID: Hello, here is an updated webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8215411.1/ sawindbg.cpp has been adjusted . The exception cases mentioned now also call env->ReleaseByteArrayElements . Best regards, Matthias > -----Original Message----- > From: David Holmes > Sent: Dienstag, 18. Dezember 2018 10:04 > To: Baesken, Matthias ; 'hotspot- > dev at openjdk.java.net' ; serviceability- > dev at openjdk.java.net; security-dev at openjdk.java.net; JC Beyler > > Subject: Re: RFR: [XS] 8215411: some GetByteArrayElements calls miss > corresponding Release > > Hi Matthias, > > On 18/12/2018 6:52 pm, Baesken, Matthias wrote: > > Hi JC / David, thanks for the input . > > > > I'm not really sure what you want me to change David, am I right that I can > keep the changes to > > > > src/java.base/macosx/native/libosxsecurity/KeystoreImpl.m > > src/jdk.attach/windows/native/libattach/VirtualMachineImpl.c > > Yes. > > > but adjust > > > > src/jdk.hotspot.agent/windows/native/libsaproc/sawindbg.cpp > > > > to also handle theis potential early return > > > > 723 IDebugDataSpaces* ptrIDebugDataSpaces = (IDebugDataSpaces*) > env->GetLongField(obj, > > 724 ptrIDebugDataSpaces_ID); > > 725 CHECK_EXCEPTION_(0); > > > > ? > > And I think: > > 730 THROW_NEW_DEBUGGER_EXCEPTION_("Windbg Error: ReadVirtual > failed!", 0); > > as I assume that won't actually return normally. > > >>> Also you change seem wrong to me because it will commit the changes > in > >>> the buffer even if we "abort" here: > >>> > >>> 735 if (bytesRead != numBytes) { > >>> 736 return 0; > >>> 737 } > >>> > > > > The spec says : > > > > > https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/function > s.html > > > >> ReleaseArrayElements Routines > >> void ReleaseArrayElements(JNIEnv *env, ArrayType > array, NativeType *elems, jint mode); > >> > >> A family of functions that informs the VM that the native code no longer > needs access to elems. The elems argument is a pointer derived from array > using the corresponding > >> GetArrayElements() function. If necessary, this function > copies back all changes made to elems to the original array. > > > > So shouldn't I tell the VM , that the native code no longer needs access to > bytePtr before returning here > > > > 735 if (bytesRead != numBytes) { > > 736 return 0; > > 737 } > > > > ? > > There are two aspects to release: > 1. the actual release (or "I don't need this any more") > 2. committing any changes made back to the original array > > This code does: > > 728 if (ptrIDebugDataSpaces->ReadVirtual((ULONG64) address, (PVOID) > bytePtr, > 729 (ULONG)numBytes, &bytesRead) != > S_OK) { > > which writes into the array. It then checks if we wrote what we expected: > > 735 if (bytesRead != numBytes) { > 736 return 0; > 737 } > > If we didn't read what was expected what should happen to the original > array? Should it be left untouched or updated with the unexpected input? > I would say left untouched and that is what the original code did. > > If everything succeeds you should do the release with mode 0; but if > taking an error exit the release should use mode JNI_ABORT so no changes > are written back. > > Cheers, > David > > > Best regards, Matthias > > > > > >> -----Original Message----- > >> From: David Holmes > >> Sent: Dienstag, 18. Dezember 2018 01:20 > >> To: Baesken, Matthias ; 'hotspot- > >> dev at openjdk.java.net' ; serviceability- > >> dev at openjdk.java.net; security-dev at openjdk.java.net > >> Subject: Re: RFR: [XS] 8215411: some GetByteArrayElements calls miss > >> corresponding Release > >> > >> Correction ... > >> > >> On 18/12/2018 8:25 am, David Holmes wrote: > >>> Hi Matthias, > >>> > >>> On 17/12/2018 6:59 pm, Baesken, Matthias wrote: > >>>> Hello, please review the following change. > >>>> I noticed that we miss at some places (for example in case of early > >>>> returns) > >>>> where GetByteArrayElements is used,? the corresponding > >>>> ReleaseByteArrayElements? call. > >>>> > >>>> In VirtualMachineImpl.c? I also removed a check for isCopy (is the > >>>> returned byte array a copy ?) > >>>> because from what I read at > >>>> > >>>> > >> > https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/function > >> s.html > >>>> > >>>> > >>>> the ReleaseByteArrayElements? routine should always be called. > >>> > >>> That's not clear to me. My reading is that you only need to release if > >>> you have changes you need to ensure are written back and that is only > >>> needed if a copy was made. > >> > >> Jc pointed out to me that if a copy is made you may need to release to > >> avoid a memory leak. But that is where the mode flags come in - and > >> again only relevant if a copy was made. > >> > >> But as per: > >> > >> https://developer.android.com/training/articles/perf-jni > >> > >> if a copy is not made and pinning is used (as with the "critical" > >> variants) then you also need to release to un-pin. > >> > >> So code should call release, with an appropriate mode, based on whether > >> isCopy was true**, and whether changed data should be written back. > >> > >> ** mode is ignored if not working on a copy so you can just set it > >> assuming a copy was made. > >> > >> Note that the hotspot implementation always makes a copy for > >> GextXXXArrayElements, and the ReleaseXXXArrayElements knows this > and > >> so > >> will always free the buffer that is passed in (other than for mode > >> JNI_COMMIT of course). > >> > >> Cheers, > >> David > >> ----- > >> > >>> That said, the change seem semantically correct and harmless in this > case. > >>> --- > >>> > >>> src/jdk.hotspot.agent/windows/native/libsaproc/sawindbg.cpp > >>> > >>> AFAICS there are still multiple exit paths via CHECK_EXCEPTION_(0) and > >>> THROW_NEW_DEBUGGER_EXCEPTION_ that won't release the buffer. > >>> > >>> Also you change seem wrong to me because it will commit the changes > in > >>> the buffer even if we "abort" here: > >>> > >>> 735?? if (bytesRead != numBytes) { > >>> ?736????? return 0; > >>> ?737?? } > >>> > >>> so it seems to me if you really want to release on all paths then the > >>> error paths should use a mode of JNI_ABORT and only the success path > >>> should use mode 0. > >>> > >>> --- > >>> > >>> ?src/java.base/macosx/native/libosxsecurity/KeystoreImpl.m > >>> > >>> This change seems okay, though again potentially not necessary - as we > >>> never commit any changes. > >>> > >>> Thanks, > >>> David > >>> ----- > >>> > >>>> > >>>> bug/webrev : > >>>> > >>>> https://bugs.openjdk.java.net/browse/JDK-8215411 > >>>> > >>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8215411.0/ > >>>> > >>>> > >>>> Thanks, Matthias > >>>> From alexey.menkov at oracle.com Tue Dec 18 21:37:11 2018 From: alexey.menkov at oracle.com (Alex Menkov) Date: Tue, 18 Dec 2018 13:37:11 -0800 Subject: RFR JDK-8215425: vmTestbase/nsk/jvmti/PopFrame should provide more detailed output Message-ID: <421a0f5e-e8cb-0439-2849-9d10fe377d14@oracle.com> Hi all, please review the fix for https://bugs.openjdk.java.net/browse/JDK-8215425 webrev: http://cr.openjdk.java.net/~amenkov/popframe_cleanup/webrev.01/ The fix - turns on detailed output for the tests and cleaned related stuff; - for popframe002 deletes output from run() method as it caused unexpected MethodExit event; - removes "test case 2" in popframe004 test as it's never executed. --alex From serguei.spitsyn at oracle.com Wed Dec 19 00:49:54 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 18 Dec 2018 16:49:54 -0800 Subject: RFR JDK-8215425: vmTestbase/nsk/jvmti/PopFrame should provide more detailed output In-Reply-To: <421a0f5e-e8cb-0439-2849-9d10fe377d14@oracle.com> References: <421a0f5e-e8cb-0439-2849-9d10fe377d14@oracle.com> Message-ID: <283679a8-acb8-fc08-41c2-56bf951c6632@oracle.com> Hi Alex, A couple of minor comments. http://cr.openjdk.java.net/~amenkov/popframe_cleanup/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/PopFrame/popframe002.java.frames.html http://cr.openjdk.java.net/~amenkov/popframe_cleanup/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/PopFrame/popframe004.java.frames.html ? While you are at these files, could you, fix several originally ugly indented comments? ? Could you, also, fix the incorrect spacing around '=' in the popframe004.java: 95 retValue=doPopFrame(true, popFrameClsThr); ? Could you give an idea about the motivation to remove the following fragment ?: 167 if (popframe004.popFdone) { // popping has been done 168 if (DEBUG_MODE) 169 out.println("popFrameCls (" + this + 170 "): enter activeMethod() after popping"); 171 // test case #2: popping from the current thread 172 if (!popframe004.popF2done) { 173 popframe004.popF2done = true; 174 if (DEBUG_MODE) { 175 out.println("popFrameCls (" + this + 176 "): going to pop a frame from the current thread..."); 177 retVal = doPopFrame(3, popFrameClsThr); 178 } else 179 retVal = doPopFrame(2, popFrameClsThr); 180 if (retVal != PASSED) 181 popframe004.totRes = FAILED; 182 } 183 if (DEBUG_MODE) 184 out.println("popFrameCls (" + this + 185 "): leaving activeMethod()..."); 186 return; 187 } Otherwise, it looks good. Thanks, Serguei On 12/18/18 1:37 PM, Alex Menkov wrote: > Hi all, > > please review the fix for > https://bugs.openjdk.java.net/browse/JDK-8215425 > webrev: > http://cr.openjdk.java.net/~amenkov/popframe_cleanup/webrev.01/ > > The fix > - turns on detailed output for the tests and cleaned related stuff; > - for popframe002 deletes output from run() method as it caused > unexpected MethodExit event; > - removes "test case 2" in popframe004 test as it's never executed. > > --alex -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexey.menkov at oracle.com Wed Dec 19 02:01:23 2018 From: alexey.menkov at oracle.com (Alex Menkov) Date: Tue, 18 Dec 2018 18:01:23 -0800 Subject: RFR JDK-8215425: vmTestbase/nsk/jvmti/PopFrame should provide more detailed output In-Reply-To: <283679a8-acb8-fc08-41c2-56bf951c6632@oracle.com> References: <421a0f5e-e8cb-0439-2849-9d10fe377d14@oracle.com> <283679a8-acb8-fc08-41c2-56bf951c6632@oracle.com> Message-ID: <7c45d02e-e32c-2fa5-89b2-78c87ef6efeb@oracle.com> Hi Serguei, Thank you for the review. Updated webrev: http://cr.openjdk.java.net/~amenkov/popframe_cleanup/webrev.02/ On 12/18/2018 16:49, serguei.spitsyn at oracle.com wrote: > Hi Alex, > > A couple of minor comments. > > http://cr.openjdk.java.net/~amenkov/popframe_cleanup/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/PopFrame/popframe002.java.frames.html > http://cr.openjdk.java.net/~amenkov/popframe_cleanup/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/PopFrame/popframe004.java.frames.html > > ? While you are at these files, could you, fix several originally ugly > indented comments? Done. > > ? Could you, also, fix the incorrect spacing around '=' in the > popframe004.java: > > 95 retValue=doPopFrame(true, popFrameClsThr); Done. Also fixed comment in popframe004/TestDescription.java (to be in sync with comment change in popframe004.java) > > > ? Could you give an idea about the motivation to remove the following > fragment ?: This is "test case 2" in popframe004 which I mentioned in the review request. The code is never executed (if it is, this means the test has already failed) and I don't have an idea what other case can be tested here. --alex > > 167 if (popframe004.popFdone) { // popping has been done > 168 if (DEBUG_MODE) > 169 out.println("popFrameCls (" + this + > 170 "): enter activeMethod() after popping"); > 171 // test case #2: popping from the current thread > 172 if (!popframe004.popF2done) { > 173 popframe004.popF2done = true; > 174 if (DEBUG_MODE) { > 175 out.println("popFrameCls (" + this + > 176 "): going to pop a frame from the current thread..."); > 177 retVal = doPopFrame(3, popFrameClsThr); > 178 } else > 179 retVal = doPopFrame(2, popFrameClsThr); > 180 if (retVal != PASSED) > 181 popframe004.totRes = FAILED; > 182 } > 183 if (DEBUG_MODE) > 184 out.println("popFrameCls (" + this + > 185 "): leaving activeMethod()..."); > 186 return; > 187 } > > > Otherwise, it looks good. > > Thanks, > Serguei > > > On 12/18/18 1:37 PM, Alex Menkov wrote: >> Hi all, >> >> please review the fix for >> https://bugs.openjdk.java.net/browse/JDK-8215425 >> webrev: >> http://cr.openjdk.java.net/~amenkov/popframe_cleanup/webrev.01/ >> >> The fix >> - turns on detailed output for the tests and cleaned related stuff; >> - for popframe002 deletes output from run() method as it caused >> unexpected MethodExit event; >> - removes "test case 2" in popframe004 test as it's never executed. >> >> --alex > From serguei.spitsyn at oracle.com Wed Dec 19 02:09:40 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 18 Dec 2018 18:09:40 -0800 Subject: RFR JDK-8215425: vmTestbase/nsk/jvmti/PopFrame should provide more detailed output In-Reply-To: <7c45d02e-e32c-2fa5-89b2-78c87ef6efeb@oracle.com> References: <421a0f5e-e8cb-0439-2849-9d10fe377d14@oracle.com> <283679a8-acb8-fc08-41c2-56bf951c6632@oracle.com> <7c45d02e-e32c-2fa5-89b2-78c87ef6efeb@oracle.com> Message-ID: Alex, Thank you for the update! It looks good. There is another instance with incorrect spacing: 121 totRes=doPopFrame(false, Thread.currentThread()); No need in new webrev if you fix the above. Thanks, Serguei On 12/18/18 6:01 PM, Alex Menkov wrote: > Hi Serguei, > > Thank you for the review. > Updated webrev: > http://cr.openjdk.java.net/~amenkov/popframe_cleanup/webrev.02/ > > On 12/18/2018 16:49, serguei.spitsyn at oracle.com wrote: >> Hi Alex, >> >> A couple of minor comments. >> >> http://cr.openjdk.java.net/~amenkov/popframe_cleanup/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/PopFrame/popframe002.java.frames.html >> >> http://cr.openjdk.java.net/~amenkov/popframe_cleanup/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/PopFrame/popframe004.java.frames.html >> >> >> ?? While you are at these files, could you, fix several originally >> ugly indented comments? > > Done. > >> >> ?? Could you, also, fix the incorrect spacing around '=' in the >> popframe004.java: >> >> 95 retValue=doPopFrame(true, popFrameClsThr); > > Done. > > Also fixed comment in popframe004/TestDescription.java (to be in sync > with comment change in popframe004.java) > >> >> >> ?? Could you give an idea about the motivation to remove the >> following fragment ?: > > This is "test case 2" in popframe004 which I mentioned in the review > request. > The code is never executed (if it is, this means the test has already > failed) and I don't have an idea what other case can be tested here. > > --alex > >> >> 167 if (popframe004.popFdone) { // popping has been done >> 168 if (DEBUG_MODE) >> 169 out.println("popFrameCls (" + this + >> 170 "): enter activeMethod() after popping"); >> 171 // test case #2: popping from the current thread >> 172 if (!popframe004.popF2done) { >> 173 popframe004.popF2done = true; >> 174 if (DEBUG_MODE) { >> 175 out.println("popFrameCls (" + this + >> 176 "): going to pop a frame from the current thread..."); >> 177 retVal = doPopFrame(3, popFrameClsThr); >> 178 } else >> 179 retVal = doPopFrame(2, popFrameClsThr); >> 180 if (retVal != PASSED) >> 181 popframe004.totRes = FAILED; >> 182 } >> 183 if (DEBUG_MODE) >> 184 out.println("popFrameCls (" + this + >> 185 "): leaving activeMethod()..."); >> 186 return; >> 187 } >> >> >> Otherwise, it looks good. >> >> Thanks, >> Serguei >> >> >> On 12/18/18 1:37 PM, Alex Menkov wrote: >>> Hi all, >>> >>> please review the fix for >>> https://bugs.openjdk.java.net/browse/JDK-8215425 >>> webrev: >>> http://cr.openjdk.java.net/~amenkov/popframe_cleanup/webrev.01/ >>> >>> The fix >>> - turns on detailed output for the tests and cleaned related stuff; >>> - for popframe002 deletes output from run() method as it caused >>> unexpected MethodExit event; >>> - removes "test case 2" in popframe004 test as it's never executed. >>> >>> --alex >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From zanglin5 at jd.com Wed Dec 19 02:56:51 2018 From: zanglin5 at jd.com (=?utf-8?B?6Ien55Cz?=) Date: Wed, 19 Dec 2018 02:56:51 +0000 Subject: [PATCH] 8214535: Extend JMap to support parallel and incremental heap scanning In-Reply-To: References: Message-ID: Dear JC, Thanks for your comments, I totally like the idea to have 3 subtasks. It seems that I don?t have access of JBS at present. May I ask your help for that? I think the 3 subtask could be: 1. Add dump to file support for jmap ?histo 2. Add incremental dump for jmap ?histo 3. Add parallel heap iteration for jmap ?histo. Is it reasonable ? Thanks! BRs, Lin From: JC Beyler [mailto:jcbeyler at google.com] Sent: Wednesday, December 19, 2018 12:56 AM To: ?? Cc: serviceability-dev at openjdk.java.net Subject: Re: [PATCH] 8214535: Extend JMap to support parallel and incremental heap scanning Hi Lin, Would it not be easier to make 3 different JBS items that are all enhancements? Or perhaps 3 subtasks to the original JDK-8214535? Then you could send out a request for review for the first two and we can talk about the third. I think all three parts can be considered separately (and it seems you do too). Thanks, Jc On Tue, Dec 18, 2018 at 12:58 AM ?? > wrote: Hi All, I am preparing the patch for https://bugs.openjdk.java.net/browse/JDK-8214535, and want to get your suggestions. To make the patch easier to review. I plan to make 3 patches as following: 1. one patch for enabling file dump of "jmap -histo".(http://cr.openjdk.java.net/~xiaofeya/JDK-8214535/file_dump/webrev.00/webrev/) 2. one patch for incremental dump intermediate data to file of "jmap -histo".(http://cr.openjdk.java.net/~xiaofeya/JDK-8214535/incremental/webrev.00/) 3. one patch for parallel iterating heap of "jmap -histo". (WIP) And the patches for item 1 & 2 are ready. patch for 3 is WIP. May I ask your help to review these patches? Thanks. BRs, Lin -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From jcbeyler at google.com Wed Dec 19 03:18:12 2018 From: jcbeyler at google.com (JC Beyler) Date: Tue, 18 Dec 2018 19:18:12 -0800 Subject: [PATCH] 8214535: Extend JMap to support parallel and incremental heap scanning In-Reply-To: References: Message-ID: Hi Lin, Done: > > 1. Add dump to file support for jmap ?histo > https://bugs.openjdk.java.net/browse/JDK-8215622 > 2. Add incremental dump for jmap ?histo > https://bugs.openjdk.java.net/browse/JDK-8215623 > 3. Add parallel heap iteration for jmap ?histo. > https://bugs.openjdk.java.net/browse/JDK-8215624 > Is it reasonable ? > Sounded reasonable to me :-) Jc > Thanks! > > > > BRs, > > Lin > > *From:* JC Beyler [mailto:jcbeyler at google.com] > *Sent:* Wednesday, December 19, 2018 12:56 AM > *To:* ?? > *Cc:* serviceability-dev at openjdk.java.net > *Subject:* Re: [PATCH] 8214535: Extend JMap to support parallel and > incremental heap scanning > > > > Hi Lin, > > > > Would it not be easier to make 3 different JBS items that are all > enhancements? Or perhaps 3 subtasks to the original JDK-8214535? Then you > could send out a request for review for the first two and we can talk about > the third. > > > > I think all three parts can be considered separately (and it seems you do > too). > > > > Thanks, > > Jc > > > > On Tue, Dec 18, 2018 at 12:58 AM ?? wrote: > > Hi All, > > I am preparing the patch for > https://bugs.openjdk.java.net/browse/JDK-8214535, and want to get your > suggestions. > > > > To make the patch easier to review. I plan to make 3 patches as > following: > > 1. one patch for enabling file dump of "jmap -histo".( > http://cr.openjdk.java.net/~xiaofeya/JDK-8214535/file_dump/webrev.00/webrev/ > ) > > 2. one patch for incremental dump intermediate data to file of "jmap > -histo".( > http://cr.openjdk.java.net/~xiaofeya/JDK-8214535/incremental/webrev.00/) > > 3. one patch for parallel iterating heap of "jmap -histo". (WIP) > > > > And the patches for item 1 & 2 are ready. patch for 3 is WIP. > > May I ask your help to review these patches? > > Thanks. > > > > BRs, > > Lin > > > > > -- > > > > Thanks, > > Jc > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From zanglin5 at jd.com Wed Dec 19 03:21:11 2018 From: zanglin5 at jd.com (=?utf-8?B?6Ien55Cz?=) Date: Wed, 19 Dec 2018 03:21:11 +0000 Subject: [PATCH] 8214535: Extend JMap to support parallel and incremental heap scanning In-Reply-To: References: Message-ID: <3f2376611e4c4cf6b6d96dda9263351b@jd.com> Dear Jc, Thanks a lot! I will update the webrev and ask for review separately ? BRs, Lin From: JC Beyler [mailto:jcbeyler at google.com] Sent: Wednesday, December 19, 2018 11:18 AM To: ?? Cc: serviceability-dev at openjdk.java.net Subject: Re: [PATCH] 8214535: Extend JMap to support parallel and incremental heap scanning Hi Lin, Done: 1. Add dump to file support for jmap ?histo https://bugs.openjdk.java.net/browse/JDK-8215622 2. Add incremental dump for jmap ?histo https://bugs.openjdk.java.net/browse/JDK-8215623 3. Add parallel heap iteration for jmap ?histo. https://bugs.openjdk.java.net/browse/JDK-8215624 Is it reasonable ? Sounded reasonable to me :-) Jc Thanks! BRs, Lin From: JC Beyler [mailto:jcbeyler at google.com] Sent: Wednesday, December 19, 2018 12:56 AM To: ?? > Cc: serviceability-dev at openjdk.java.net Subject: Re: [PATCH] 8214535: Extend JMap to support parallel and incremental heap scanning Hi Lin, Would it not be easier to make 3 different JBS items that are all enhancements? Or perhaps 3 subtasks to the original JDK-8214535? Then you could send out a request for review for the first two and we can talk about the third. I think all three parts can be considered separately (and it seems you do too). Thanks, Jc On Tue, Dec 18, 2018 at 12:58 AM ?? > wrote: Hi All, I am preparing the patch for https://bugs.openjdk.java.net/browse/JDK-8214535, and want to get your suggestions. To make the patch easier to review. I plan to make 3 patches as following: 1. one patch for enabling file dump of "jmap -histo".(http://cr.openjdk.java.net/~xiaofeya/JDK-8214535/file_dump/webrev.00/webrev/) 2. one patch for incremental dump intermediate data to file of "jmap -histo".(http://cr.openjdk.java.net/~xiaofeya/JDK-8214535/incremental/webrev.00/) 3. one patch for parallel iterating heap of "jmap -histo". (WIP) And the patches for item 1 & 2 are ready. patch for 3 is WIP. May I ask your help to review these patches? Thanks. BRs, Lin -- Thanks, Jc -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From jcbeyler at google.com Wed Dec 19 03:35:21 2018 From: jcbeyler at google.com (JC Beyler) Date: Tue, 18 Dec 2018 19:35:21 -0800 Subject: RFR JDK-8215425: vmTestbase/nsk/jvmti/PopFrame should provide more detailed output In-Reply-To: References: <421a0f5e-e8cb-0439-2849-9d10fe377d14@oracle.com> <283679a8-acb8-fc08-41c2-56bf951c6632@oracle.com> <7c45d02e-e32c-2fa5-89b2-78c87ef6efeb@oracle.com> Message-ID: Hi Alex, Looks good to me as well :) Nice job! Jc On Tue, Dec 18, 2018 at 6:10 PM wrote: > Alex, > > Thank you for the update! > > It looks good. > There is another instance with incorrect spacing: > > 121 totRes=doPopFrame(false, Thread.currentThread()); > > > No need in new webrev if you fix the above. > > Thanks, > Serguei > > > On 12/18/18 6:01 PM, Alex Menkov wrote: > > Hi Serguei, > > Thank you for the review. > Updated webrev: > http://cr.openjdk.java.net/~amenkov/popframe_cleanup/webrev.02/ > > On 12/18/2018 16:49, serguei.spitsyn at oracle.com wrote: > > Hi Alex, > > A couple of minor comments. > > > http://cr.openjdk.java.net/~amenkov/popframe_cleanup/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/PopFrame/popframe002.java.frames.html > > http://cr.openjdk.java.net/~amenkov/popframe_cleanup/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/PopFrame/popframe004.java.frames.html > > While you are at these files, could you, fix several originally ugly > indented comments? > > > Done. > > > Could you, also, fix the incorrect spacing around '=' in the > popframe004.java: > > 95 retValue=doPopFrame(true, popFrameClsThr); > > > Done. > > Also fixed comment in popframe004/TestDescription.java (to be in sync with > comment change in popframe004.java) > > > > Could you give an idea about the motivation to remove the following > fragment ?: > > > This is "test case 2" in popframe004 which I mentioned in the review > request. > The code is never executed (if it is, this means the test has already > failed) and I don't have an idea what other case can be tested here. > > --alex > > > 167 if (popframe004.popFdone) { // popping has been done > 168 if (DEBUG_MODE) > 169 out.println("popFrameCls (" + this + > 170 "): enter activeMethod() after popping"); > 171 // test case #2: popping from the current thread > 172 if (!popframe004.popF2done) { > 173 popframe004.popF2done = true; > 174 if (DEBUG_MODE) { > 175 out.println("popFrameCls (" + this + > 176 "): going to pop a frame from the current thread..."); > 177 retVal = doPopFrame(3, popFrameClsThr); > 178 } else > 179 retVal = doPopFrame(2, popFrameClsThr); > 180 if (retVal != PASSED) > 181 popframe004.totRes = FAILED; > 182 } > 183 if (DEBUG_MODE) > 184 out.println("popFrameCls (" + this + > 185 "): leaving activeMethod()..."); > 186 return; > 187 } > > > Otherwise, it looks good. > > Thanks, > Serguei > > > On 12/18/18 1:37 PM, Alex Menkov wrote: > > Hi all, > > please review the fix for > https://bugs.openjdk.java.net/browse/JDK-8215425 > webrev: > http://cr.openjdk.java.net/~amenkov/popframe_cleanup/webrev.01/ > > The fix > - turns on detailed output for the tests and cleaned related stuff; > - for popframe002 deletes output from run() method as it caused unexpected > MethodExit event; > - removes "test case 2" in popframe004 test as it's never executed. > > --alex > > > > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From jini.george at oracle.com Wed Dec 19 04:52:47 2018 From: jini.george at oracle.com (Jini George) Date: Wed, 19 Dec 2018 10:22:47 +0530 Subject: RFR: JDK-8215568: Refactor SA clhsdb tests to use ClhsdbLauncher Message-ID: <92017316-89fd-a3bf-3b2e-5c76c443c62a@oracle.com> Hello! Requesting reviews for: http://cr.openjdk.java.net/~jgeorge/8215568/webrev.00/ BugID: https://bugs.openjdk.java.net/browse/JDK-8215568 No new failures with the SA tests (hs-tiers 1-5) with these changes. The changes here include: * Refactoring the SA tests which test clhsdb commands to use ClhsdbLauncher for uniformity and ease of maintainence. * Testing for regular expressions with shouldMatch rather than shouldContain. * Minor cleanups. Thank you, Jini. From david.holmes at oracle.com Wed Dec 19 06:20:49 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 19 Dec 2018 16:20:49 +1000 Subject: RFR: [XS] 8215411: some GetByteArrayElements calls miss corresponding Release In-Reply-To: References: <164fa449-28f6-e61b-5a6d-d3182a4c7e32@oracle.com> <7c2f5dc2-9bbe-29f9-8be0-2613267f60dc@oracle.com> Message-ID: <0e0a64cf-0047-417e-f0f6-351e60e3704e@oracle.com> On 19/12/2018 3:19 am, Baesken, Matthias wrote: > Hello, here is an updated webrev : > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8215411.1/ > > sawindbg.cpp has been adjusted . > The exception cases mentioned now also call env->ReleaseByteArrayElements . Looks good - thanks. One minor style nit: 725 if(env->ExceptionOccurred()) { Please add a space after "if" (yes I see the macros have this wrong too). No need for updated webrev. Thanks, David > Best regards, Matthias > >> -----Original Message----- >> From: David Holmes >> Sent: Dienstag, 18. Dezember 2018 10:04 >> To: Baesken, Matthias ; 'hotspot- >> dev at openjdk.java.net' ; serviceability- >> dev at openjdk.java.net; security-dev at openjdk.java.net; JC Beyler >> >> Subject: Re: RFR: [XS] 8215411: some GetByteArrayElements calls miss >> corresponding Release >> >> Hi Matthias, >> >> On 18/12/2018 6:52 pm, Baesken, Matthias wrote: >>> Hi JC / David, thanks for the input . >>> >>> I'm not really sure what you want me to change David, am I right that I can >> keep the changes to >>> >>> src/java.base/macosx/native/libosxsecurity/KeystoreImpl.m >>> src/jdk.attach/windows/native/libattach/VirtualMachineImpl.c >> >> Yes. >> >>> but adjust >>> >>> src/jdk.hotspot.agent/windows/native/libsaproc/sawindbg.cpp >>> >>> to also handle theis potential early return >>> >>> 723 IDebugDataSpaces* ptrIDebugDataSpaces = (IDebugDataSpaces*) >> env->GetLongField(obj, >>> 724 ptrIDebugDataSpaces_ID); >>> 725 CHECK_EXCEPTION_(0); >>> >>> ? >> >> And I think: >> >> 730 THROW_NEW_DEBUGGER_EXCEPTION_("Windbg Error: ReadVirtual >> failed!", 0); >> >> as I assume that won't actually return normally. >> >>>>> Also you change seem wrong to me because it will commit the changes >> in >>>>> the buffer even if we "abort" here: >>>>> >>>>> 735 if (bytesRead != numBytes) { >>>>> 736 return 0; >>>>> 737 } >>>>> >>> >>> The spec says : >>> >>> >> https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/function >> s.html >>> >>>> ReleaseArrayElements Routines >>>> void ReleaseArrayElements(JNIEnv *env, ArrayType >> array, NativeType *elems, jint mode); >>>> >>>> A family of functions that informs the VM that the native code no longer >> needs access to elems. The elems argument is a pointer derived from array >> using the corresponding >>>> GetArrayElements() function. If necessary, this function >> copies back all changes made to elems to the original array. >>> >>> So shouldn't I tell the VM , that the native code no longer needs access to >> bytePtr before returning here >>> >>> 735 if (bytesRead != numBytes) { >>> 736 return 0; >>> 737 } >>> >>> ? >> >> There are two aspects to release: >> 1. the actual release (or "I don't need this any more") >> 2. committing any changes made back to the original array >> >> This code does: >> >> 728 if (ptrIDebugDataSpaces->ReadVirtual((ULONG64) address, (PVOID) >> bytePtr, >> 729 (ULONG)numBytes, &bytesRead) != >> S_OK) { >> >> which writes into the array. It then checks if we wrote what we expected: >> >> 735 if (bytesRead != numBytes) { >> 736 return 0; >> 737 } >> >> If we didn't read what was expected what should happen to the original >> array? Should it be left untouched or updated with the unexpected input? >> I would say left untouched and that is what the original code did. >> >> If everything succeeds you should do the release with mode 0; but if >> taking an error exit the release should use mode JNI_ABORT so no changes >> are written back. >> >> Cheers, >> David >> >>> Best regards, Matthias >>> >>> >>>> -----Original Message----- >>>> From: David Holmes >>>> Sent: Dienstag, 18. Dezember 2018 01:20 >>>> To: Baesken, Matthias ; 'hotspot- >>>> dev at openjdk.java.net' ; serviceability- >>>> dev at openjdk.java.net; security-dev at openjdk.java.net >>>> Subject: Re: RFR: [XS] 8215411: some GetByteArrayElements calls miss >>>> corresponding Release >>>> >>>> Correction ... >>>> >>>> On 18/12/2018 8:25 am, David Holmes wrote: >>>>> Hi Matthias, >>>>> >>>>> On 17/12/2018 6:59 pm, Baesken, Matthias wrote: >>>>>> Hello, please review the following change. >>>>>> I noticed that we miss at some places (for example in case of early >>>>>> returns) >>>>>> where GetByteArrayElements is used,? the corresponding >>>>>> ReleaseByteArrayElements? call. >>>>>> >>>>>> In VirtualMachineImpl.c? I also removed a check for isCopy (is the >>>>>> returned byte array a copy ?) >>>>>> because from what I read at >>>>>> >>>>>> >>>> >> https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/function >>>> s.html >>>>>> >>>>>> >>>>>> the ReleaseByteArrayElements? routine should always be called. >>>>> >>>>> That's not clear to me. My reading is that you only need to release if >>>>> you have changes you need to ensure are written back and that is only >>>>> needed if a copy was made. >>>> >>>> Jc pointed out to me that if a copy is made you may need to release to >>>> avoid a memory leak. But that is where the mode flags come in - and >>>> again only relevant if a copy was made. >>>> >>>> But as per: >>>> >>>> https://developer.android.com/training/articles/perf-jni >>>> >>>> if a copy is not made and pinning is used (as with the "critical" >>>> variants) then you also need to release to un-pin. >>>> >>>> So code should call release, with an appropriate mode, based on whether >>>> isCopy was true**, and whether changed data should be written back. >>>> >>>> ** mode is ignored if not working on a copy so you can just set it >>>> assuming a copy was made. >>>> >>>> Note that the hotspot implementation always makes a copy for >>>> GextXXXArrayElements, and the ReleaseXXXArrayElements knows this >> and >>>> so >>>> will always free the buffer that is passed in (other than for mode >>>> JNI_COMMIT of course). >>>> >>>> Cheers, >>>> David >>>> ----- >>>> >>>>> That said, the change seem semantically correct and harmless in this >> case. >>>>> --- >>>>> >>>>> src/jdk.hotspot.agent/windows/native/libsaproc/sawindbg.cpp >>>>> >>>>> AFAICS there are still multiple exit paths via CHECK_EXCEPTION_(0) and >>>>> THROW_NEW_DEBUGGER_EXCEPTION_ that won't release the buffer. >>>>> >>>>> Also you change seem wrong to me because it will commit the changes >> in >>>>> the buffer even if we "abort" here: >>>>> >>>>> 735?? if (bytesRead != numBytes) { >>>>> ?736????? return 0; >>>>> ?737?? } >>>>> >>>>> so it seems to me if you really want to release on all paths then the >>>>> error paths should use a mode of JNI_ABORT and only the success path >>>>> should use mode 0. >>>>> >>>>> --- >>>>> >>>>> ?src/java.base/macosx/native/libosxsecurity/KeystoreImpl.m >>>>> >>>>> This change seems okay, though again potentially not necessary - as we >>>>> never commit any changes. >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>>> >>>>>> bug/webrev : >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8215411 >>>>>> >>>>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8215411.0/ >>>>>> >>>>>> >>>>>> Thanks, Matthias >>>>>> From gnu.andrew at redhat.com Wed Dec 19 06:46:28 2018 From: gnu.andrew at redhat.com (Andrew Hughes) Date: Wed, 19 Dec 2018 06:46:28 +0000 Subject: [8u] RFR: 8059038: Create new launcher for SA tools In-Reply-To: References: <975dca9cb3fdea8f2a799c52f7d3e5edd9d79306.camel@redhat.com> Message-ID: On Thu, 13 Dec 2018 at 17:19, Severin Gehwolf wrote: > > On Thu, 2018-12-13 at 15:49 +0000, Andrew Hughes wrote: > > Ok, I presume that's not a change you're also going to make in > > OpenJDK 12? > > Yes. I don't intend to change this in JDK 12. JDK 8 and JDK 9+ are not > compatible in this regard. That's the case before and after this patch. > > > I'm fine with them being linked by the same bug ID, as they resolve the same > > underlying issue, but this really needs to be clear in the summary > > text, as, to the > > uninformed, they look like completely different patches. > > OK will do. Can I consider this reviewed? > > Thanks, > Severin > I'd like to see an updated webrev with a more detailed commit message first. Thanks, -- Andrew :) Senior Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) Web Site: http://fuseyism.com Twitter: https://twitter.com/gnu_andrew_java PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 From sgehwolf at redhat.com Wed Dec 19 12:14:59 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Wed, 19 Dec 2018 13:14:59 +0100 Subject: [8u] RFR: 8059038: Create new launcher for SA tools In-Reply-To: References: <975dca9cb3fdea8f2a799c52f7d3e5edd9d79306.camel@redhat.com> Message-ID: On Wed, 2018-12-19 at 06:46 +0000, Andrew Hughes wrote: > On Thu, 13 Dec 2018 at 17:19, Severin Gehwolf > wrote: > > > > On Thu, 2018-12-13 at 15:49 +0000, Andrew Hughes wrote: > > > Ok, I presume that's not a change you're also going to make in > > > OpenJDK 12? > > > > Yes. I don't intend to change this in JDK 12. JDK 8 and JDK 9+ are > > not > > compatible in this regard. That's the case before and after this > > patch. > > > > > I'm fine with them being linked by the same bug ID, as they > > > resolve the same > > > underlying issue, but this really needs to be clear in the > > > summary > > > text, as, to the > > > uninformed, they look like completely different patches. > > > > OK will do. Can I consider this reviewed? > > > > Thanks, > > Severin > > > > I'd like to see an updated webrev with a more detailed commit message > first. Here you go: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8059038/03/ Thanks, Severin From harold.seigel at oracle.com Wed Dec 19 13:22:35 2018 From: harold.seigel at oracle.com (Harold David Seigel) Date: Wed, 19 Dec 2018 08:22:35 -0500 Subject: RFR 8215398: -Xlog option usage => Invalid decorator '\temp\app_cds.log'. Message-ID: <71efe071-62c1-b1cc-c98f-953c6790eae1@oracle.com> Hi, Please review this small change to fix JDK-8215398.? The fix works by not treating ':' as a delimiter in a -Xlog... option string if it is following by a '\' and preceded by either a single character or the text 'file='.? The fix is for Windows only. Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8215398/webrev/index.html JBS Bug:? https://bugs.openjdk.java.net/browse/JDK-8215398 The fix was regression tested by running Mach5 tiers 1 and 2 tests and builds on Linux-x64, Windows, and Mac OS X, running tiers 3-5 tests on Linux-x64, running JCK-12 Lang and VM tests on Linux-x64, and by hand on Windows. Thanks, Harol From alexey.ivanov at oracle.com Wed Dec 19 14:23:50 2018 From: alexey.ivanov at oracle.com (Alexey Ivanov) Date: Wed, 19 Dec 2018 14:23:50 +0000 Subject: [12] RFR for JDK-8214122: JDWP is broken on 32 bit Windows: transport library missing onLoad entry In-Reply-To: <4c9e06c6-b11d-483d-8411-bc8761fda150@oracle.com> References: <2DD922C6-EFB1-4891-90F0-3378DEA91B1C@gmail.com> <07840796-f1c4-e077-7b0b-cd5c1a880ec1@oracle.com> <00e58330-26f0-4401-f911-24fda0127bcc@oracle.com> <7083B6EB-588E-4296-9A03-D9CC9B688936@gmail.com> <813f4b3d-0b4a-1e28-845f-a6422a22c695@redhat.com> <530c2651-dc96-e04a-0b0e-d1f04d8e135a@oracle.com> <77cc6547-a145-e7b9-5f29-c66df76bc50a@redhat.com> <4c9e06c6-b11d-483d-8411-bc8761fda150@oracle.com> Message-ID: <99b6b1ef-2636-c8ae-acc0-bf792129091a@oracle.com> Any volunteers from core-libs and serviceability to review? Regards, Alexey On 12/12/2018 16:43, Magnus Ihse Bursie wrote: > > On 2018-12-12 17:38, Alexey Ivanov wrote: >> Hi all, >> >> I have updated the summary of JDK-8214122 and the subject accordingly. >> >> Could you please review the following fix? >> >> https://bugs.openjdk.java.net/browse/JDK-8214122 >> webrev: http://cr.openjdk.java.net/~aivanov/8214122/webrev.01/ > Looks good to me. > > /Magnus >> >> *Root cause* >> jdwpTransport_OnLoad is exported as _jdwpTransport_OnLoad at 16 on 32 >> bit Windows according to the name decorations [1] for __stdcall [2] >> calling conventions. >> >> *Fix* >> On 32 bit Windows, look up the decorated name, >> _jdwpTransport_OnLoad at 16, first; if not found, look up the >> undecorated name, jdwpTransport_OnLoad. >> >> >> Regards, >> Alexey >> >> [1] >> https://docs.microsoft.com/en-us/cpp/build/reference/decorated-names?view=vs-2017#FormatC >> [2] https://docs.microsoft.com/en-ie/cpp/cpp/stdcall?view=vs-2017 >> >> On 12/12/2018 11:19, Magnus Ihse Bursie wrote: >>> >>> >>> On 2018-12-11 18:16, Alexey Ivanov wrote: >>>> Hi Simon, >>>> >>>> Thank you for your comments. >>>> >>>> The updated webrev: >>>> http://cr.openjdk.java.net/~aivanov/8214122/webrev.01/ >>>> >>>> Indeed, it looks much cleaner. >>> Yes! This seems like the correct fix. >>> >>> Ship it! :) >>> >>> /Magnus >>> >>>> >>>> Regards, >>>> Alexey >>>> >>>> On 11/12/2018 16:43, Simon Tooke wrote: >>>>> On 2018-12-11 10:05 a.m., Alexey Ivanov wrote: >>>>>> Hi everyone, >>>>>> >>>>>> I came up with the following patch: >>>>>> http://cr.openjdk.java.net/~aivanov/8214122/webrev.00/ >>>>>> >>>>>> It specifically addresses the problem in JDK-8214122 where on 32 bit >>>>>> Windows jdwpTransport_OnLoad can exported with its plain and >>>>>> __stdcall-mangled name. I used conditional compilation so that for >>>>>> other platforms the code remains as it is now. >>>>>> >>>>>> jshell starts successfully with this fix; an IDE debugger works >>>>>> as well. >>>>>> >>>>> I am not a reviewer, but this patch only works for the specific case >>>>> under discussion; the '@16' refers to the reserved stack size in the >>>>> Pascal calling convention.? So, the patch only works for 16 bytes of >>>>> parameters.? To be generic, the routine needs to have the stack size >>>>> passed in by the caller.? If a generic fix isn't desired (and I >>>>> hope it >>>>> is), I'd prefer to see the caller simply pass the decorated or >>>>> undecorated name depending on the Win32/64 defines. >>>>> >>>>> ???? #if defined(_WIN32) && !defined(_WIN64) onLoad = >>>>> ???? (jdwpTransport_OnLoad_t) dbgsysFindLibraryEntry(handle, >>>>> ???? "_jdwpTransport_OnLoad at 16"); #else onLoad = >>>>> (jdwpTransport_OnLoad_t) >>>>> ???? dbgsysFindLibraryEntry(handle, "jdwpTransport_OnLoad"); #endif >>>>> >>>>> >>>>> Thanks, >>>>> -Simon >>>>> >>>>>> Regards, >>>>>> Alexey >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8214122 >>>>>> >>>>>> On 10/12/2018 15:11, Magnus Ihse Bursie wrote: >>>>>>>> Since removing JNICALL is not an option, there are only two >>>>>>>> options: >>>>>>>> >>>>>>>> 1. Add |/export| option to the Makefile or pragma-comment to the >>>>>>>> source file; >>>>>>>> 2. Lookup the decorated name |_jdwpTransport_OnLoad at 16| for Win32 >>>>>>>> with fallback to undecorated one. >>>>>>> Yes. >>>>>>> >>>>>>> I think the correct solution here is 2. >>>> >>> >> > From daniel.daugherty at oracle.com Wed Dec 19 15:05:42 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 19 Dec 2018 10:05:42 -0500 Subject: [12] RFR for JDK-8214122: JDWP is broken on 32 bit Windows: transport library missing onLoad entry In-Reply-To: <99b6b1ef-2636-c8ae-acc0-bf792129091a@oracle.com> References: <2DD922C6-EFB1-4891-90F0-3378DEA91B1C@gmail.com> <00e58330-26f0-4401-f911-24fda0127bcc@oracle.com> <7083B6EB-588E-4296-9A03-D9CC9B688936@gmail.com> <813f4b3d-0b4a-1e28-845f-a6422a22c695@redhat.com> <530c2651-dc96-e04a-0b0e-d1f04d8e135a@oracle.com> <77cc6547-a145-e7b9-5f29-c66df76bc50a@redhat.com> <4c9e06c6-b11d-483d-8411-bc8761fda150@oracle.com> <99b6b1ef-2636-c8ae-acc0-bf792129091a@oracle.com> Message-ID: <8d177aef-2735-52e5-f46d-3f946d4fa284@oracle.com> On 12/19/18 9:23 AM, Alexey Ivanov wrote: > Any volunteers from core-libs and serviceability to review? How about former Serviceability Team members? :-) Alan B. and I both used to be on the Serviceability Team... And Alan B. is a current member of Core Libs... > webrev: http://cr.openjdk.java.net/~aivanov/8214122/webrev.01/ src/jdk.jdwp.agent/share/native/libjdwp/transport.c ??? No comments. The code appears to do what is described in this (very long) review thread. I don't believe we (Oracle) have any Win32 test setups for the jdk/jdk repo so this isn't a change that someone from Oracle can do additional testing on. Thumbs up! Dan > > Regards, > Alexey > > On 12/12/2018 16:43, Magnus Ihse Bursie wrote: >> >> On 2018-12-12 17:38, Alexey Ivanov wrote: >>> Hi all, >>> >>> I have updated the summary of JDK-8214122 and the subject accordingly. >>> >>> Could you please review the following fix? >>> >>> https://bugs.openjdk.java.net/browse/JDK-8214122 >>> webrev: http://cr.openjdk.java.net/~aivanov/8214122/webrev.01/ >> Looks good to me. >> >> /Magnus >>> >>> *Root cause* >>> jdwpTransport_OnLoad is exported as _jdwpTransport_OnLoad at 16 on 32 >>> bit Windows according to the name decorations [1] for __stdcall [2] >>> calling conventions. >>> >>> *Fix* >>> On 32 bit Windows, look up the decorated name, >>> _jdwpTransport_OnLoad at 16, first; if not found, look up the >>> undecorated name, jdwpTransport_OnLoad. >>> >>> >>> Regards, >>> Alexey >>> >>> [1] >>> https://docs.microsoft.com/en-us/cpp/build/reference/decorated-names?view=vs-2017#FormatC >>> [2] https://docs.microsoft.com/en-ie/cpp/cpp/stdcall?view=vs-2017 >>> >>> On 12/12/2018 11:19, Magnus Ihse Bursie wrote: >>>> >>>> >>>> On 2018-12-11 18:16, Alexey Ivanov wrote: >>>>> Hi Simon, >>>>> >>>>> Thank you for your comments. >>>>> >>>>> The updated webrev: >>>>> http://cr.openjdk.java.net/~aivanov/8214122/webrev.01/ >>>>> >>>>> Indeed, it looks much cleaner. >>>> Yes! This seems like the correct fix. >>>> >>>> Ship it! :) >>>> >>>> /Magnus >>>> >>>>> >>>>> Regards, >>>>> Alexey >>>>> >>>>> On 11/12/2018 16:43, Simon Tooke wrote: >>>>>> On 2018-12-11 10:05 a.m., Alexey Ivanov wrote: >>>>>>> Hi everyone, >>>>>>> >>>>>>> I came up with the following patch: >>>>>>> http://cr.openjdk.java.net/~aivanov/8214122/webrev.00/ >>>>>>> >>>>>>> It specifically addresses the problem in JDK-8214122 where on 32 >>>>>>> bit >>>>>>> Windows jdwpTransport_OnLoad can exported with its plain and >>>>>>> __stdcall-mangled name. I used conditional compilation so that for >>>>>>> other platforms the code remains as it is now. >>>>>>> >>>>>>> jshell starts successfully with this fix; an IDE debugger works >>>>>>> as well. >>>>>>> >>>>>> I am not a reviewer, but this patch only works for the specific case >>>>>> under discussion; the '@16' refers to the reserved stack size in the >>>>>> Pascal calling convention.? So, the patch only works for 16 bytes of >>>>>> parameters.? To be generic, the routine needs to have the stack size >>>>>> passed in by the caller.? If a generic fix isn't desired (and I >>>>>> hope it >>>>>> is), I'd prefer to see the caller simply pass the decorated or >>>>>> undecorated name depending on the Win32/64 defines. >>>>>> >>>>>> ???? #if defined(_WIN32) && !defined(_WIN64) onLoad = >>>>>> ???? (jdwpTransport_OnLoad_t) dbgsysFindLibraryEntry(handle, >>>>>> ???? "_jdwpTransport_OnLoad at 16"); #else onLoad = >>>>>> (jdwpTransport_OnLoad_t) >>>>>> ???? dbgsysFindLibraryEntry(handle, "jdwpTransport_OnLoad"); #endif >>>>>> >>>>>> >>>>>> Thanks, >>>>>> -Simon >>>>>> >>>>>>> Regards, >>>>>>> Alexey >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8214122 >>>>>>> >>>>>>> On 10/12/2018 15:11, Magnus Ihse Bursie wrote: >>>>>>>>> Since removing JNICALL is not an option, there are only two >>>>>>>>> options: >>>>>>>>> >>>>>>>>> 1. Add |/export| option to the Makefile or pragma-comment to the >>>>>>>>> source file; >>>>>>>>> 2. Lookup the decorated name |_jdwpTransport_OnLoad at 16| for Win32 >>>>>>>>> with fallback to undecorated one. >>>>>>>> Yes. >>>>>>>> >>>>>>>> I think the correct solution here is 2. >>>>> >>>> >>> >> > > From alexey.ivanov at oracle.com Wed Dec 19 15:28:19 2018 From: alexey.ivanov at oracle.com (Alexey Ivanov) Date: Wed, 19 Dec 2018 15:28:19 +0000 Subject: [12] RFR for JDK-8214122: JDWP is broken on 32 bit Windows: transport library missing onLoad entry In-Reply-To: <8d177aef-2735-52e5-f46d-3f946d4fa284@oracle.com> References: <2DD922C6-EFB1-4891-90F0-3378DEA91B1C@gmail.com> <00e58330-26f0-4401-f911-24fda0127bcc@oracle.com> <7083B6EB-588E-4296-9A03-D9CC9B688936@gmail.com> <813f4b3d-0b4a-1e28-845f-a6422a22c695@redhat.com> <530c2651-dc96-e04a-0b0e-d1f04d8e135a@oracle.com> <77cc6547-a145-e7b9-5f29-c66df76bc50a@redhat.com> <4c9e06c6-b11d-483d-8411-bc8761fda150@oracle.com> <99b6b1ef-2636-c8ae-acc0-bf792129091a@oracle.com> <8d177aef-2735-52e5-f46d-3f946d4fa284@oracle.com> Message-ID: <10f87cd8-86bf-e1d4-ae33-7e1c34a1c633@oracle.com> Thank you, Daniel, for your quick review. On 19/12/2018 15:05, Daniel D. Daugherty wrote: > On 12/19/18 9:23 AM, Alexey Ivanov wrote: >> Any volunteers from core-libs and serviceability to review? > > How about former Serviceability Team members? :-) Alan B. and I > both used to be on the Serviceability Team... And Alan B. is a > current member of Core Libs... That's fine, I guess. :-) > > > webrev: http://cr.openjdk.java.net/~aivanov/8214122/webrev.01/ > > src/jdk.jdwp.agent/share/native/libjdwp/transport.c > ??? No comments. > > The code appears to do what is described in this (very long) review > thread. I don't believe we (Oracle) have any Win32 test setups for > the jdk/jdk repo so this isn't a change that someone from Oracle can > do additional testing on. That's true. I ran all the builds with tier3 testing, just in case. I also verified locally that jshell.exe and debugger start successfully in Win32 build. -- Alexey > > Thumbs up! > > Dan > > >> >> Regards, >> Alexey >> >> On 12/12/2018 16:43, Magnus Ihse Bursie wrote: >>> >>> On 2018-12-12 17:38, Alexey Ivanov wrote: >>>> Hi all, >>>> >>>> I have updated the summary of JDK-8214122 and the subject accordingly. >>>> >>>> Could you please review the following fix? >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8214122 >>>> webrev: http://cr.openjdk.java.net/~aivanov/8214122/webrev.01/ >>> Looks good to me. >>> >>> /Magnus >>>> >>>> *Root cause* >>>> jdwpTransport_OnLoad is exported as _jdwpTransport_OnLoad at 16 on 32 >>>> bit Windows according to the name decorations [1] for __stdcall [2] >>>> calling conventions. >>>> >>>> *Fix* >>>> On 32 bit Windows, look up the decorated name, >>>> _jdwpTransport_OnLoad at 16, first; if not found, look up the >>>> undecorated name, jdwpTransport_OnLoad. >>>> >>>> >>>> Regards, >>>> Alexey >>>> >>>> [1] >>>> https://docs.microsoft.com/en-us/cpp/build/reference/decorated-names?view=vs-2017#FormatC >>>> [2] https://docs.microsoft.com/en-ie/cpp/cpp/stdcall?view=vs-2017 >>>> >>>> On 12/12/2018 11:19, Magnus Ihse Bursie wrote: >>>>> >>>>> >>>>> On 2018-12-11 18:16, Alexey Ivanov wrote: >>>>>> Hi Simon, >>>>>> >>>>>> Thank you for your comments. >>>>>> >>>>>> The updated webrev: >>>>>> http://cr.openjdk.java.net/~aivanov/8214122/webrev.01/ >>>>>> >>>>>> Indeed, it looks much cleaner. >>>>> Yes! This seems like the correct fix. >>>>> >>>>> Ship it! :) >>>>> >>>>> /Magnus >>>>> >>>>>> >>>>>> Regards, >>>>>> Alexey >>>>>> >>>>>> On 11/12/2018 16:43, Simon Tooke wrote: >>>>>>> On 2018-12-11 10:05 a.m., Alexey Ivanov wrote: >>>>>>>> Hi everyone, >>>>>>>> >>>>>>>> I came up with the following patch: >>>>>>>> http://cr.openjdk.java.net/~aivanov/8214122/webrev.00/ >>>>>>>> >>>>>>>> It specifically addresses the problem in JDK-8214122 where on >>>>>>>> 32 bit >>>>>>>> Windows jdwpTransport_OnLoad can exported with its plain and >>>>>>>> __stdcall-mangled name. I used conditional compilation so that for >>>>>>>> other platforms the code remains as it is now. >>>>>>>> >>>>>>>> jshell starts successfully with this fix; an IDE debugger works >>>>>>>> as well. >>>>>>>> >>>>>>> I am not a reviewer, but this patch only works for the specific >>>>>>> case >>>>>>> under discussion; the '@16' refers to the reserved stack size in >>>>>>> the >>>>>>> Pascal calling convention.? So, the patch only works for 16 >>>>>>> bytes of >>>>>>> parameters.? To be generic, the routine needs to have the stack >>>>>>> size >>>>>>> passed in by the caller.? If a generic fix isn't desired (and I >>>>>>> hope it >>>>>>> is), I'd prefer to see the caller simply pass the decorated or >>>>>>> undecorated name depending on the Win32/64 defines. >>>>>>> >>>>>>> ???? #if defined(_WIN32) && !defined(_WIN64) onLoad = >>>>>>> ???? (jdwpTransport_OnLoad_t) dbgsysFindLibraryEntry(handle, >>>>>>> ???? "_jdwpTransport_OnLoad at 16"); #else onLoad = >>>>>>> (jdwpTransport_OnLoad_t) >>>>>>> ???? dbgsysFindLibraryEntry(handle, "jdwpTransport_OnLoad"); #endif >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> -Simon >>>>>>> >>>>>>>> Regards, >>>>>>>> Alexey >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8214122 >>>>>>>> >>>>>>>> On 10/12/2018 15:11, Magnus Ihse Bursie wrote: >>>>>>>>>> Since removing JNICALL is not an option, there are only two >>>>>>>>>> options: >>>>>>>>>> >>>>>>>>>> 1. Add |/export| option to the Makefile or pragma-comment to the >>>>>>>>>> source file; >>>>>>>>>> 2. Lookup the decorated name |_jdwpTransport_OnLoad at 16| for >>>>>>>>>> Win32 >>>>>>>>>> with fallback to undecorated one. >>>>>>>>> Yes. >>>>>>>>> >>>>>>>>> I think the correct solution here is 2. >>>>>> >>>>> >>>> >>> >> >> > From jcbeyler at google.com Wed Dec 19 15:42:00 2018 From: jcbeyler at google.com (JC Beyler) Date: Wed, 19 Dec 2018 07:42:00 -0800 Subject: RFR (XS) 8215495: Always set isCopy In-Reply-To: <499e2b8d-fa8d-2e8b-24ff-0a42500c91a7@oracle.com> References: <3cf5ffda-6f26-54c9-2ea3-4977bc2e6859@oracle.com> <6d5ccd78-37f6-5f3e-c651-6543ea2fa0d9@oracle.com> <499e2b8d-fa8d-2e8b-24ff-0a42500c91a7@oracle.com> Message-ID: Hi all, Could I get a second review for this please? :) Webrev: http://cr.openjdk.java.net/~jcbeyler/8215495/webrev.01/ Bug: https://bugs.openjdk.java.net/browse/JDK-8215495 Thanks! Jc On Mon, Dec 17, 2018 at 4:25 PM David Holmes wrote: > LGTM. > > Thanks, > David > > On 18/12/2018 8:53 am, JC Beyler wrote: > > Sounds good to me: > > > > For the odd corner case: > > > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8215495/webrev.01/ > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8215495 > > > > Thanks! > > Jc > > > > On Mon, Dec 17, 2018 at 2:39 PM David Holmes > > wrote: > > > > Hi Jc, > > > > On 18/12/2018 8:12 am, JC Beyler wrote: > > > Hi David, > > > > > > I understand the rationale behind the "If the method does return > > NULL, > > > then isCopy's value is undefined". But what about > > > the DEFINE_GETSCALARARRAYELEMENTS case? > > > > > > It does this: > > > if (len == 0) { \ > > > /* Empty array: legal but useless, can't return NULL. \ > > > * Return a pointer to something useless. \ > > > * Avoid asserts in typeArrayOop. */ \ > > > result = (ElementType*)get_bad_address(); \ > > > > > > Should we at least then put isCopy to JNI_FALSE in that case > > since it > > > does not return NULL and no exception is raised, > > > > Sure - it's an odd corner case, but better not to leave isCopy > > potentially uninitialized. > > > > Thanks, > > David > > > > > Thanks, > > > Jc > > > > > > > > > On Mon, Dec 17, 2018 at 1:29 PM David Holmes > > > > > > >> wrote: > > > > > > Hi Jc, > > > > > > On 18/12/2018 3:42 am, JC Beyler wrote: > > > > Hi all, > > > > > > > > Could I get a review for this webrev: > > > > > > > > Webrev: > > http://cr.openjdk.java.net/~jcbeyler/8215495/webrev.00/ > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8215495 > > > > > > isCopy only has to be set if the method executes successfully > > > > > > "If isCopy is not NULL, then *isCopy is set to JNI_TRUE if a > > copy is > > > made; or it is set to JNI_FALSE if no copy is made." > > > > > > You can only make (or not) a copy if the operation actually > > > succeeds. So > > > before checking isCopy the caller must check for NULL and/or > > a pending > > > exception. > > > > > > I see no bug here. > > > > > > David > > > ----- > > > > > > > Thanks, > > > > Jc > > > > > > > > > > > > -- > > > > > > Thanks, > > > Jc > > > > > > > > -- > > > > Thanks, > > Jc > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From gnu.andrew at redhat.com Wed Dec 19 16:56:09 2018 From: gnu.andrew at redhat.com (Andrew Hughes) Date: Wed, 19 Dec 2018 16:56:09 +0000 Subject: [8u] RFR: 8059038: Create new launcher for SA tools In-Reply-To: References: <975dca9cb3fdea8f2a799c52f7d3e5edd9d79306.camel@redhat.com> Message-ID: On Wed, 19 Dec 2018 at 12:15, Severin Gehwolf wrote: > > On Wed, 2018-12-19 at 06:46 +0000, Andrew Hughes wrote: > > On Thu, 13 Dec 2018 at 17:19, Severin Gehwolf > > wrote: > > > > > > On Thu, 2018-12-13 at 15:49 +0000, Andrew Hughes wrote: > > > > Ok, I presume that's not a change you're also going to make in > > > > OpenJDK 12? > > > > > > Yes. I don't intend to change this in JDK 12. JDK 8 and JDK 9+ are > > > not > > > compatible in this regard. That's the case before and after this > > > patch. > > > > > > > I'm fine with them being linked by the same bug ID, as they > > > > resolve the same > > > > underlying issue, but this really needs to be clear in the > > > > summary > > > > text, as, to the > > > > uninformed, they look like completely different patches. > > > > > > OK will do. Can I consider this reviewed? > > > > > > Thanks, > > > Severin > > > > > > > I'd like to see an updated webrev with a more detailed commit message > > first. > > Here you go: > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8059038/03/ > > Thanks, > Severin > I was hoping for a note on why it differs from 9 :-) -- Andrew :) Senior Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) Web Site: http://fuseyism.com Twitter: https://twitter.com/gnu_andrew_java PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 From sgehwolf at redhat.com Wed Dec 19 16:59:42 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Wed, 19 Dec 2018 17:59:42 +0100 Subject: [8u] RFR: 8059038: Create new launcher for SA tools In-Reply-To: References: <975dca9cb3fdea8f2a799c52f7d3e5edd9d79306.camel@redhat.com> Message-ID: On Wed, 2018-12-19 at 16:56 +0000, Andrew Hughes wrote: > On Wed, 19 Dec 2018 at 12:15, Severin Gehwolf wrote: > > > > On Wed, 2018-12-19 at 06:46 +0000, Andrew Hughes wrote: > > > On Thu, 13 Dec 2018 at 17:19, Severin Gehwolf > > > wrote: > > > > > > > > On Thu, 2018-12-13 at 15:49 +0000, Andrew Hughes wrote: > > > > > Ok, I presume that's not a change you're also going to make in > > > > > OpenJDK 12? > > > > > > > > Yes. I don't intend to change this in JDK 12. JDK 8 and JDK 9+ are > > > > not > > > > compatible in this regard. That's the case before and after this > > > > patch. > > > > > > > > > I'm fine with them being linked by the same bug ID, as they > > > > > resolve the same > > > > > underlying issue, but this really needs to be clear in the > > > > > summary > > > > > text, as, to the > > > > > uninformed, they look like completely different patches. > > > > > > > > OK will do. Can I consider this reviewed? > > > > > > > > Thanks, > > > > Severin > > > > > > > > > > I'd like to see an updated webrev with a more detailed commit message > > > first. > > > > Here you go: > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8059038/03/ > > > > Thanks, > > Severin > > > > I was hoping for a note on why it differs from 9 :-) Any suggestions? Perhaps this? "Summary: Create separate launchers for clhsdb and hsdb since JDK 8 uses separate launchers for SA tools" Thanks, Severin From gnu.andrew at redhat.com Wed Dec 19 17:04:13 2018 From: gnu.andrew at redhat.com (Andrew Hughes) Date: Wed, 19 Dec 2018 17:04:13 +0000 Subject: [8u] RFR: 8059038: Create new launcher for SA tools In-Reply-To: References: <975dca9cb3fdea8f2a799c52f7d3e5edd9d79306.camel@redhat.com> Message-ID: On Wed, 19 Dec 2018 at 16:59, Severin Gehwolf wrote: > > On Wed, 2018-12-19 at 16:56 +0000, Andrew Hughes wrote: > > On Wed, 19 Dec 2018 at 12:15, Severin Gehwolf wrote: > > > > > > On Wed, 2018-12-19 at 06:46 +0000, Andrew Hughes wrote: > > > > On Thu, 13 Dec 2018 at 17:19, Severin Gehwolf > > > > wrote: > > > > > > > > > > On Thu, 2018-12-13 at 15:49 +0000, Andrew Hughes wrote: > > > > > > Ok, I presume that's not a change you're also going to make in > > > > > > OpenJDK 12? > > > > > > > > > > Yes. I don't intend to change this in JDK 12. JDK 8 and JDK 9+ are > > > > > not > > > > > compatible in this regard. That's the case before and after this > > > > > patch. > > > > > > > > > > > I'm fine with them being linked by the same bug ID, as they > > > > > > resolve the same > > > > > > underlying issue, but this really needs to be clear in the > > > > > > summary > > > > > > text, as, to the > > > > > > uninformed, they look like completely different patches. > > > > > > > > > > OK will do. Can I consider this reviewed? > > > > > > > > > > Thanks, > > > > > Severin > > > > > > > > > > > > > I'd like to see an updated webrev with a more detailed commit message > > > > first. > > > > > > Here you go: > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8059038/03/ > > > > > > Thanks, > > > Severin > > > > > > > I was hoping for a note on why it differs from 9 :-) > > Any suggestions? Perhaps this? > > "Summary: Create separate launchers for clhsdb and hsdb since JDK 8 uses separate launchers for SA tools" > > Thanks, > Severin > Sounds fine to me. I'm just trying to avoid others having the confusion I had with the differences between these. Consider it reviewed with that message. Thanks, -- Andrew :) Senior Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) Web Site: http://fuseyism.com Twitter: https://twitter.com/gnu_andrew_java PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 From jcbeyler at google.com Wed Dec 19 18:12:18 2018 From: jcbeyler at google.com (JC Beyler) Date: Wed, 19 Dec 2018 10:12:18 -0800 Subject: RFR: JDK-8215568: Refactor SA clhsdb tests to use ClhsdbLauncher In-Reply-To: <92017316-89fd-a3bf-3b2e-5c76c443c62a@oracle.com> References: <92017316-89fd-a3bf-3b2e-5c76c443c62a@oracle.com> Message-ID: Hi Jini, I saw this potential issue with one of the test conversions: http://cr.openjdk.java.net/~jgeorge/8215568/webrev.00/test/hotspot/jtreg/serviceability/sa/TestPrintMdo.java.udiff.html - It seems like there is a missing "unexpected" strings check here no? - The original test was doing - - if (line.contains("missing reason for ")) { - unexpected = new RuntimeException("Unexpected msg: missing reason for "); - break; - } whereas the new test is not seemingly (though http://cr.openjdk.java.net/~jgeorge/8215568/webrev.00/test/hotspot/jtreg/serviceability/sa/TestClhsdbJstackLock.java.udiff.html does do it so I think this is an oversight?). - Also interesting is that the original test was trying to find one of X: - if (line.contains("VirtualCallData") || - line.contains("CounterData") || - line.contains("ReceiverTypeData")) { - knownProfileDataTypeFound = true; - } whereas you are now wanting to find all of them. Is that normal/wanted? The rest looked good to me, though I wish there were a way to not have to change all the strings to be regex friendly but I fail to see how to do that without writing a runCmdWithoutRegexMatcher, which seems overkill :-) Jc On Tue, Dec 18, 2018 at 8:55 PM Jini George wrote: > Hello! > > Requesting reviews for: > > http://cr.openjdk.java.net/~jgeorge/8215568/webrev.00/ > > BugID: https://bugs.openjdk.java.net/browse/JDK-8215568 > > No new failures with the SA tests (hs-tiers 1-5) with these changes. The > changes here include: > > * Refactoring the SA tests which test clhsdb commands to use > ClhsdbLauncher for uniformity and ease of maintainence. > * Testing for regular expressions with shouldMatch rather than > shouldContain. > * Minor cleanups. > > Thank you, > Jini. > > > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From merkel05 at gmail.com Wed Dec 19 19:03:43 2018 From: merkel05 at gmail.com (Sergei Ustimenko) Date: Wed, 19 Dec 2018 20:03:43 +0100 Subject: [PATCH] JDK-8025886: Replace [[ and == bash extensions in tests In-Reply-To: References: Message-ID: HI Martin, As you've suggested I've simply added bash's shebang. It wouldn't add any problem since, as David have mentioned, no bash - no build. I've also quickly checked for similar cases and found one. An updated patch is below. diff --git a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh --- a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh +++ b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh @@ -1,3 +1,5 @@ +#!/bin/bash + # # Copyright (c) 2003, 2018, Oracle and/or its affiliates. All rights reserved. # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. Regards, Sergei On Mon, 10 Dec 2018 at 22:27, Martin Buchholz wrote: > I don't know if there's an official policy on how ultra-portable tests are > supposed to be. In practice, you probably won't be able to build openjdk > on a system without bash. > > On Mon, Dec 10, 2018 at 1:12 PM Sergei Ustimenko > wrote: > >> Hi Martin, >> >> That sounds good! >> >> I've counted all the sh-shebangs and it appears that >> there are at least 66 of them inside the test/ directory, >> where only 12 bashes. >> >> I've also ran the search in order to identify all the >> occurrences that use either [[ or == and found only >> three of them that use "==". That one for example: >> >> http://hg.openjdk.java.net/jdk/sandbox/file/f94276ccc9fc/test/hotspot/jtreg/vmTestbase/jit/tiered/tieredTest.sh#l63 >> of course `dash` reports failure in that case. >> >> So I'm quite hesitant in that case and not really sure >> what to do. I haven't also found any existent JBS ticket >> for such /bin/sh => /bin/bash a replacement. >> >> So any advise in this case would be appreciated! >> >> Regards, >> Sergei >> >> On Mon, 10 Dec 2018 at 18:32, Martin Buchholz >> wrote: >> >>> I would not try to remove all bash-isms from openjdk. Instead I would >>> find instances of /bin/sh that need to be changed to /bin/bash. >>> >>> (Ubuntu's use of /bin/sh -> /bin/dash is technically correct, but caused >>> much suffering >>> https://bugs.launchpad.net/ubuntu/+source/dash/+bug/61463 >>> ) >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From martinrb at google.com Wed Dec 19 19:22:41 2018 From: martinrb at google.com (Martin Buchholz) Date: Wed, 19 Dec 2018 11:22:41 -0800 Subject: [PATCH] JDK-8025886: Replace [[ and == bash extensions in tests In-Reply-To: References: Message-ID: Did we really have shell scripts without a shebang line? Yeah, let's fix ! On Wed, Dec 19, 2018 at 11:03 AM Sergei Ustimenko wrote: > HI Martin, > > As you've suggested I've simply added bash's shebang. > It wouldn't add any problem since, as David have mentioned, > no bash - no build. I've also quickly checked for similar cases > and found one. > > An updated patch is below. > > diff --git > a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > --- > a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > +++ > b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > @@ -1,3 +1,5 @@ > +#!/bin/bash > + > # > # Copyright (c) 2003, 2018, Oracle and/or its affiliates. All rights > reserved. > # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > > > Regards, > Sergei > > On Mon, 10 Dec 2018 at 22:27, Martin Buchholz wrote: > >> I don't know if there's an official policy on how ultra-portable tests >> are supposed to be. In practice, you probably won't be able to build >> openjdk on a system without bash. >> >> On Mon, Dec 10, 2018 at 1:12 PM Sergei Ustimenko >> wrote: >> >>> Hi Martin, >>> >>> That sounds good! >>> >>> I've counted all the sh-shebangs and it appears that >>> there are at least 66 of them inside the test/ directory, >>> where only 12 bashes. >>> >>> I've also ran the search in order to identify all the >>> occurrences that use either [[ or == and found only >>> three of them that use "==". That one for example: >>> >>> http://hg.openjdk.java.net/jdk/sandbox/file/f94276ccc9fc/test/hotspot/jtreg/vmTestbase/jit/tiered/tieredTest.sh#l63 >>> of course `dash` reports failure in that case. >>> >>> So I'm quite hesitant in that case and not really sure >>> what to do. I haven't also found any existent JBS ticket >>> for such /bin/sh => /bin/bash a replacement. >>> >>> So any advise in this case would be appreciated! >>> >>> Regards, >>> Sergei >>> >>> On Mon, 10 Dec 2018 at 18:32, Martin Buchholz >>> wrote: >>> >>>> I would not try to remove all bash-isms from openjdk. Instead I would >>>> find instances of /bin/sh that need to be changed to /bin/bash. >>>> >>>> (Ubuntu's use of /bin/sh -> /bin/dash is technically correct, but >>>> caused much suffering >>>> https://bugs.launchpad.net/ubuntu/+source/dash/+bug/61463 >>>> ) >>>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From yumin.qi at gmail.com Wed Dec 19 19:38:39 2018 From: yumin.qi at gmail.com (yumin qi) Date: Wed, 19 Dec 2018 11:38:39 -0800 Subject: RFR (XS) 8215495: Always set isCopy In-Reply-To: References: <3cf5ffda-6f26-54c9-2ea3-4977bc2e6859@oracle.com> <6d5ccd78-37f6-5f3e-c651-6543ea2fa0d9@oracle.com> <499e2b8d-fa8d-2e8b-24ff-0a42500c91a7@oracle.com> Message-ID: Looks good to me. Thanks Yumin On Wed, Dec 19, 2018 at 7:42 AM JC Beyler wrote: > Hi all, > > Could I get a second review for this please? :) > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8215495/webrev.01/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8215495 > > Thanks! > Jc > > On Mon, Dec 17, 2018 at 4:25 PM David Holmes > wrote: > > > LGTM. > > > > Thanks, > > David > > > > On 18/12/2018 8:53 am, JC Beyler wrote: > > > Sounds good to me: > > > > > > For the odd corner case: > > > > > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8215495/webrev.01/ > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8215495 > > > > > > Thanks! > > > Jc > > > > > > On Mon, Dec 17, 2018 at 2:39 PM David Holmes > > > wrote: > > > > > > Hi Jc, > > > > > > On 18/12/2018 8:12 am, JC Beyler wrote: > > > > Hi David, > > > > > > > > I understand the rationale behind the "If the method does return > > > NULL, > > > > then isCopy's value is undefined". But what about > > > > the DEFINE_GETSCALARARRAYELEMENTS case? > > > > > > > > It does this: > > > > if (len == 0) { \ > > > > /* Empty array: legal but useless, can't return NULL. \ > > > > * Return a pointer to something useless. \ > > > > * Avoid asserts in typeArrayOop. */ \ > > > > result = (ElementType*)get_bad_address(); \ > > > > > > > > Should we at least then put isCopy to JNI_FALSE in that case > > > since it > > > > does not return NULL and no exception is raised, > > > > > > Sure - it's an odd corner case, but better not to leave isCopy > > > potentially uninitialized. > > > > > > Thanks, > > > David > > > > > > > Thanks, > > > > Jc > > > > > > > > > > > > On Mon, Dec 17, 2018 at 1:29 PM David Holmes > > > > > > > > > >> wrote: > > > > > > > > Hi Jc, > > > > > > > > On 18/12/2018 3:42 am, JC Beyler wrote: > > > > > Hi all, > > > > > > > > > > Could I get a review for this webrev: > > > > > > > > > > Webrev: > > > http://cr.openjdk.java.net/~jcbeyler/8215495/webrev.00/ > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8215495 > > > > > > > > isCopy only has to be set if the method executes > successfully > > > > > > > > "If isCopy is not NULL, then *isCopy is set to JNI_TRUE if a > > > copy is > > > > made; or it is set to JNI_FALSE if no copy is made." > > > > > > > > You can only make (or not) a copy if the operation actually > > > > succeeds. So > > > > before checking isCopy the caller must check for NULL and/or > > > a pending > > > > exception. > > > > > > > > I see no bug here. > > > > > > > > David > > > > ----- > > > > > > > > > Thanks, > > > > > Jc > > > > > > > > > > > > > > > > -- > > > > > > > > Thanks, > > > > Jc > > > > > > > > > > > > -- > > > > > > Thanks, > > > Jc > > > > > -- > > Thanks, > Jc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From merkel05 at gmail.com Wed Dec 19 20:04:52 2018 From: merkel05 at gmail.com (Sergei Ustimenko) Date: Wed, 19 Dec 2018 21:04:52 +0100 Subject: [PATCH] JDK-8025886: Replace [[ and == bash extensions in tests In-Reply-To: References: Message-ID: Quickfix on my previous mail: I've found no scripts with the same problem, except this one in the patch. >Yeah, let's fix ! Perfect! I will just need some help with sponsorship to push this. I've signed the OCA, so no problems from my side. Regards, Sergei On Wed, 19 Dec 2018 at 20:22, Martin Buchholz wrote: > Did we really have shell scripts without a shebang line? > Yeah, let's fix ! > > On Wed, Dec 19, 2018 at 11:03 AM Sergei Ustimenko > wrote: > >> HI Martin, >> >> As you've suggested I've simply added bash's shebang. >> It wouldn't add any problem since, as David have mentioned, >> no bash - no build. I've also quickly checked for similar cases >> and found one. >> >> An updated patch is below. >> >> diff --git >> a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh >> b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh >> --- >> a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh >> +++ >> b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh >> @@ -1,3 +1,5 @@ >> +#!/bin/bash >> + >> # >> # Copyright (c) 2003, 2018, Oracle and/or its affiliates. All rights >> reserved. >> # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >> >> >> Regards, >> Sergei >> >> On Mon, 10 Dec 2018 at 22:27, Martin Buchholz >> wrote: >> >>> I don't know if there's an official policy on how ultra-portable tests >>> are supposed to be. In practice, you probably won't be able to build >>> openjdk on a system without bash. >>> >>> On Mon, Dec 10, 2018 at 1:12 PM Sergei Ustimenko >>> wrote: >>> >>>> Hi Martin, >>>> >>>> That sounds good! >>>> >>>> I've counted all the sh-shebangs and it appears that >>>> there are at least 66 of them inside the test/ directory, >>>> where only 12 bashes. >>>> >>>> I've also ran the search in order to identify all the >>>> occurrences that use either [[ or == and found only >>>> three of them that use "==". That one for example: >>>> >>>> http://hg.openjdk.java.net/jdk/sandbox/file/f94276ccc9fc/test/hotspot/jtreg/vmTestbase/jit/tiered/tieredTest.sh#l63 >>>> of course `dash` reports failure in that case. >>>> >>>> So I'm quite hesitant in that case and not really sure >>>> what to do. I haven't also found any existent JBS ticket >>>> for such /bin/sh => /bin/bash a replacement. >>>> >>>> So any advise in this case would be appreciated! >>>> >>>> Regards, >>>> Sergei >>>> >>>> On Mon, 10 Dec 2018 at 18:32, Martin Buchholz >>>> wrote: >>>> >>>>> I would not try to remove all bash-isms from openjdk. Instead I would >>>>> find instances of /bin/sh that need to be changed to /bin/bash. >>>>> >>>>> (Ubuntu's use of /bin/sh -> /bin/dash is technically correct, but >>>>> caused much suffering >>>>> https://bugs.launchpad.net/ubuntu/+source/dash/+bug/61463 >>>>> ) >>>>> >>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Thu Dec 20 06:44:01 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 20 Dec 2018 16:44:01 +1000 Subject: RFR JDK-8215425: vmTestbase/nsk/jvmti/PopFrame should provide more detailed output In-Reply-To: References: <421a0f5e-e8cb-0439-2849-9d10fe377d14@oracle.com> <283679a8-acb8-fc08-41c2-56bf951c6632@oracle.com> <7c45d02e-e32c-2fa5-89b2-78c87ef6efeb@oracle.com> Message-ID: <63d7c1e7-ca48-1300-7e6e-bff9267b07cc@oracle.com> Hi Alexey, The popframe004 test is now failing in tier 4 - [CI] jdk13-jdk.26 Please file a bug and investigate. Thanks, David On 19/12/2018 1:35 pm, JC Beyler wrote: > Hi Alex, > > Looks good to me as well :) > > Nice job! > Jc > > On Tue, Dec 18, 2018 at 6:10 PM > wrote: > > Alex, > > Thank you for the update! > > It looks good. > There is another instance with incorrect spacing: > > 121 totRes=doPopFrame(false, Thread.currentThread()); > > > No need in new webrev if you fix the above. > > Thanks, > Serguei > > > On 12/18/18 6:01 PM, Alex Menkov wrote: >> Hi Serguei, >> >> Thank you for the review. >> Updated webrev: >> http://cr.openjdk.java.net/~amenkov/popframe_cleanup/webrev.02/ >> >> On 12/18/2018 16:49, serguei.spitsyn at oracle.com >> wrote: >>> Hi Alex, >>> >>> A couple of minor comments. >>> >>> http://cr.openjdk.java.net/~amenkov/popframe_cleanup/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/PopFrame/popframe002.java.frames.html >>> >>> http://cr.openjdk.java.net/~amenkov/popframe_cleanup/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/PopFrame/popframe004.java.frames.html >>> >>> >>> ?? While you are at these files, could you, fix several >>> originally ugly indented comments? >> >> Done. >> >>> >>> ?? Could you, also, fix the incorrect spacing around '=' in the >>> popframe004.java: >>> >>> 95 retValue=doPopFrame(true, popFrameClsThr); >> >> Done. >> >> Also fixed comment in popframe004/TestDescription.java (to be in >> sync with comment change in popframe004.java) >> >>> >>> >>> ?? Could you give an idea about the motivation to remove the >>> following fragment ?: >> >> This is "test case 2" in popframe004 which I mentioned in the >> review request. >> The code is never executed (if it is, this means the test has >> already failed) and I don't have an idea what other case can be >> tested here. >> >> --alex >> >>> >>> 167 if (popframe004.popFdone) { // popping has been done >>> 168 if (DEBUG_MODE) >>> 169 out.println("popFrameCls (" + this + >>> 170 "): enter activeMethod() after popping"); >>> 171 // test case #2: popping from the current thread >>> 172 if (!popframe004.popF2done) { >>> 173 popframe004.popF2done = true; >>> 174 if (DEBUG_MODE) { >>> 175 out.println("popFrameCls (" + this + >>> 176 "): going to pop a frame from the current thread..."); >>> 177 retVal = doPopFrame(3, popFrameClsThr); >>> 178 } else >>> 179 retVal = doPopFrame(2, popFrameClsThr); >>> 180 if (retVal != PASSED) >>> 181 popframe004.totRes = FAILED; >>> 182 } >>> 183 if (DEBUG_MODE) >>> 184 out.println("popFrameCls (" + this + >>> 185 "): leaving activeMethod()..."); >>> 186 return; >>> 187 } >>> >>> >>> Otherwise, it looks good. >>> >>> Thanks, >>> Serguei >>> >>> >>> On 12/18/18 1:37 PM, Alex Menkov wrote: >>>> Hi all, >>>> >>>> please review the fix for >>>> https://bugs.openjdk.java.net/browse/JDK-8215425 >>>> webrev: >>>> http://cr.openjdk.java.net/~amenkov/popframe_cleanup/webrev.01/ >>>> >>>> The fix >>>> - turns on detailed output for the tests and cleaned related stuff; >>>> - for popframe002 deletes output from run() method as it caused >>>> unexpected MethodExit event; >>>> - removes "test case 2" in popframe004 test as it's never executed. >>>> >>>> --alex >>> > > > > -- > > Thanks, > Jc From david.holmes at oracle.com Thu Dec 20 07:07:06 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 20 Dec 2018 17:07:06 +1000 Subject: RFR 8215398: -Xlog option usage => Invalid decorator '\temp\app_cds.log'. In-Reply-To: <71efe071-62c1-b1cc-c98f-953c6790eae1@oracle.com> References: <71efe071-62c1-b1cc-c98f-953c6790eae1@oracle.com> Message-ID: <37e19362-0065-eb00-9bfb-7dcd67f55318@oracle.com> Hi Harold, On 19/12/2018 11:22 pm, Harold David Seigel wrote: > Hi, > > Please review this small change to fix JDK-8215398.? The fix works by > not treating ':' as a delimiter in a -Xlog... option string if it is > following by a '\' and preceded by either a single character or the text > 'file='.? The fix is for Windows only. > > Open Webrev: > http://cr.openjdk.java.net/~hseigel/bug_8215398/webrev/index.html I think I can follow the fix. But I'm a bit concerned about the test. AFAICS the test thought it was already testing this case - albeit with the path quoted: 42 // Ensure log files can be specified with full path. 43 // On windows, this means that the file name will contain 44 // a colon ('C:\log.txt' for example), which is used to 45 // separate -Xlog: options (-Xlog:tags:filename:decorators). 46 // Try to log to a file in our current directory, using its absolute path. 47 String baseName = "test file.log"; 48 Path filePath = Paths.get(baseName).toAbsolutePath(); 49 String fileName = filePath.toString(); 50 File file = filePath.toFile(); ... 65 String[] validOutputs = new String[] { 66 quote + fileName + quote, 67 "file=" + quote + fileName + quote, 68 quote + fileName + quote + ":", 69 quote + fileName + quote + "::" 70 }; But even quoted if I specify the drive designator in the path, it fails! Here's a local attempt at this: D:\ade> apps\Java\jdk-11\fastdebug\bin\java -Xlog:safepoint=trace:'d:\safepointtrace.txt' -version [0.014s][error][logging] Invalid decorator '\safepointtrace.txt''. Invalid -Xlog option '-Xlog:safepoint=trace:'d:\safepointtrace.txt'', see error log for details. Error: Could not create the Java Virtual Machine. Error: A fatal exception has occurred. Program will exit. So AFAICS the test can't possibly have been testing what it thought it was testing! ??? I'm also not sure about just adding some unquoted variants to the existing TestQuotedLogOutputs without renaming the test and updating the @summary Thanks, David > JBS Bug:? https://bugs.openjdk.java.net/browse/JDK-8215398 > > The fix was regression tested by running Mach5 tiers 1 and 2 tests and > builds on Linux-x64, Windows, and Mac OS X, running tiers 3-5 tests on > Linux-x64, running JCK-12 Lang and VM tests on Linux-x64, and by hand on > Windows. > > Thanks, Harol > From gary.adams at oracle.com Thu Dec 20 12:52:33 2018 From: gary.adams at oracle.com (Gary Adams) Date: Thu, 20 Dec 2018 07:52:33 -0500 Subject: RFR: JDK-8215571: jdb does not include jdk.* in the default class filter Message-ID: <5C1B9091.7020108@oracle.com> This should be a trivial update. The default "excludes" filter for jdb should have been updated when the jdk.* packages were first introduced. diff --git a/src/jdk.jdi/share/classes/com/sun/tools/example/debug/tty/Env.java b/src/jdk.jdi/share/classes/com/sun/tools/example/debug/tty/Env.java --- a/src/jdk.jdi/share/classes/com/sun/tools/example/debug/tty/Env.java +++ b/src/jdk.jdi/share/classes/com/sun/tools/example/debug/tty/Env.java @@ -1,5 +1,5 @@ /* - * Copyright (c) 1998, 2011, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1998, 2018, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -108,7 +108,7 @@ static private List excludes() { if (excludes == null) { - setExcludes("java.*, javax.*, sun.*, com.sun.*"); + setExcludes("java.*, javax.*, sun.*, com.sun.*, jdk.*"); } return excludes; } From Alan.Bateman at oracle.com Thu Dec 20 12:55:45 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 20 Dec 2018 12:55:45 +0000 Subject: RFR: JDK-8215571: jdb does not include jdk.* in the default class filter In-Reply-To: <5C1B9091.7020108@oracle.com> References: <5C1B9091.7020108@oracle.com> Message-ID: On 20/12/2018 12:52, Gary Adams wrote: > This should be a trivial update. > > The default "excludes" filter for jdb > should have been updated when the jdk.* packages were first introduced. This looks okay to me. One thing to mention is that produce a more accurate exclude filter from the packages of the modules in the run-time image. That would allow it work with libraries on the class path with javax.* APIs for example. -Alan From gary.adams at oracle.com Thu Dec 20 13:18:46 2018 From: gary.adams at oracle.com (Gary Adams) Date: Thu, 20 Dec 2018 08:18:46 -0500 Subject: JDK-8211343: nsk_jvmti_parseoptions should handle multiple suboptions Message-ID: <5C1B96B6.1040002@oracle.com> During some earlier jvmti test debugging, I noticed that it was not possible to add a quick argument to the current tests and rerun the test. e.g. expand "waittime=5" to "waittime=5,verbose". I tracked down the options parsing in jvmti_tools.cpp and saw some comments that only a single option could be parsed. So I filed this bug to revisit the issue for the next release: Issue: https://bugs.openjdk.java.net/browse/JDK-8211343 I think the option parsing should be ripped out and redone, but for now I think a simple tweak to use strsep(), might go a long way to solving the multiple option support. I just started testing, but wanted to know if anyone else has been down this rabbit hole before. diff --git a/test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_tools.cpp b/test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_tools.cpp --- a/test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_tools.cpp +++ b/test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_tools.cpp @@ -196,22 +196,14 @@ } -/** - * - * The current option will not perform more than one - * single option which given, this is due to places explained - * in this question. - * - **/ - /* - * This whole play can be reduced with simple StringTokenizer (strtok). - * + * Parse a comma or space separated list of options. */ int nsk_jvmti_parseOptions(const char options[]) { size_t len; const char* opt; + char *str = strdup(options); int success = NSK_TRUE; context.options.string = NULL; @@ -232,7 +224,7 @@ context.options.string[len] = '\0'; context.options.string[len+1] = '\0'; - for (opt = context.options.string; ;) { + while ((opt = strsep(&str, " ,")) != NULL) { const char* opt_end; const char* val_sep; int opt_len=0; From chris.plummer at oracle.com Thu Dec 20 15:18:03 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 20 Dec 2018 07:18:03 -0800 Subject: RFR: JDK-8215571: jdb does not include jdk.* in the default class filter In-Reply-To: <5C1B9091.7020108@oracle.com> References: <5C1B9091.7020108@oracle.com> Message-ID: Looks good. I've been using it for a while now with no issues. Chris On 12/20/18 4:52 AM, Gary Adams wrote: > This should be a trivial update. > > The default "excludes" filter for jdb > should have been updated when the jdk.* packages were first introduced. > > > diff --git > a/src/jdk.jdi/share/classes/com/sun/tools/example/debug/tty/Env.java > b/src/jdk.jdi/share/classes/com/sun/tools/example/debug/tty/Env.java > --- a/src/jdk.jdi/share/classes/com/sun/tools/example/debug/tty/Env.java > +++ b/src/jdk.jdi/share/classes/com/sun/tools/example/debug/tty/Env.java > @@ -1,5 +1,5 @@ > ?/* > - * Copyright (c) 1998, 2011, Oracle and/or its affiliates. All rights > reserved. > + * Copyright (c) 1998, 2018, Oracle and/or its affiliates. All rights > reserved. > ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > ? * > ? * This code is free software; you can redistribute it and/or modify it > @@ -108,7 +108,7 @@ > > ???? static private List excludes() { > ???????? if (excludes == null) { > -??????????? setExcludes("java.*, javax.*, sun.*, com.sun.*"); > +??????????? setExcludes("java.*, javax.*, sun.*, com.sun.*, jdk.*"); > ???????? } > ???????? return excludes; > ???? } > From jcbeyler at google.com Thu Dec 20 15:54:35 2018 From: jcbeyler at google.com (JC Beyler) Date: Thu, 20 Dec 2018 07:54:35 -0800 Subject: JDK-8211343: nsk_jvmti_parseoptions should handle multiple suboptions In-Reply-To: <5C1B96B6.1040002@oracle.com> References: <5C1B96B6.1040002@oracle.com> Message-ID: Hi Gary, I had seen that too and forgot to file it! So thanks! I think the comment you removed is interesting, I'm not sure what it means exactly and I've tried re-reading a few times but the use of "in this question" is weird :-) Anyway, the webrev looks good except perhaps for the use of strsep, which is BSD, instead of strtok, which is C89/C99/Posix. Thanks for doing this! Jc On Thu, Dec 20, 2018 at 5:17 AM Gary Adams wrote: > During some earlier jvmti test debugging, I noticed that it was not > possible to > add a quick argument to the current tests and rerun the test. e.g. > expand "waittime=5" to > "waittime=5,verbose". I tracked down the options parsing in > jvmti_tools.cpp and saw some > comments that only a single option could be parsed. > > So I filed this bug to revisit the issue for the next release: > > Issue: https://bugs.openjdk.java.net/browse/JDK-8211343 > > I think the option parsing should be ripped out and redone, > but for now I think a simple tweak to use strsep(), might go a long way > to solving the multiple option support. I just started testing, > but wanted to know if anyone else has been down this rabbit hole > before. > > > diff --git > a/test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_tools.cpp > b/test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_tools.cpp > --- a/test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_tools.cpp > +++ b/test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_tools.cpp > @@ -196,22 +196,14 @@ > } > > > -/** > - * > - * The current option will not perform more than one > - * single option which given, this is due to places explained > - * in this question. > - * > - **/ > - > /* > - * This whole play can be reduced with simple StringTokenizer (strtok). > - * > + * Parse a comma or space separated list of options. > */ > > int nsk_jvmti_parseOptions(const char options[]) { > size_t len; > const char* opt; > + char *str = strdup(options); > int success = NSK_TRUE; > > context.options.string = NULL; > @@ -232,7 +224,7 @@ > context.options.string[len] = '\0'; > context.options.string[len+1] = '\0'; > > - for (opt = context.options.string; ;) { > + while ((opt = strsep(&str, " ,")) != NULL) { > const char* opt_end; > const char* val_sep; > int opt_len=0; > > -- Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From gary.adams at oracle.com Thu Dec 20 16:33:17 2018 From: gary.adams at oracle.com (Gary Adams) Date: Thu, 20 Dec 2018 11:33:17 -0500 Subject: JDK-8211343: nsk_jvmti_parseoptions should handle multiple suboptions In-Reply-To: References: <5C1B96B6.1040002@oracle.com> Message-ID: <5C1BC44D.9030305@oracle.com> I prototyped with strsep, because strtok is not safe and did not want to deal with the strtok_r versus strtok_s (windows) platform variants. ... #include #include int main(int argc, char **argv){ char *str = strdup("waittime=5,verbose foo bar baz=11"); char *name; char *value; while ((name = strsep (&str, " ,")) != NULL) { value = index(name, '='); if (value == NULL) { printf ("%s\n", name); } else { *value++ = '\0'; printf ("%s=%s\n", name, value); } } ... ./main waittime=5 verbose foo bar baz=11 On 12/20/18, 10:54 AM, JC Beyler wrote: > Hi Gary, > > I had seen that too and forgot to file it! So thanks! > > I think the comment you removed is interesting, I'm not sure what it > means exactly and I've tried re-reading a few times but the use of "in > this question" is weird :-) > > Anyway, the webrev looks good except perhaps for the use of strsep, > which is BSD, instead of strtok, which is C89/C99/Posix. > > Thanks for doing this! > Jc > > On Thu, Dec 20, 2018 at 5:17 AM Gary Adams > wrote: > > During some earlier jvmti test debugging, I noticed that it was not > possible to > add a quick argument to the current tests and rerun the test. e.g. > expand "waittime=5" to > "waittime=5,verbose". I tracked down the options parsing in > jvmti_tools.cpp and saw some > comments that only a single option could be parsed. > > So I filed this bug to revisit the issue for the next release: > > Issue: https://bugs.openjdk.java.net/browse/JDK-8211343 > > I think the option parsing should be ripped out and redone, > but for now I think a simple tweak to use strsep(), might go a > long way > to solving the multiple option support. I just started testing, > but wanted to know if anyone else has been down this rabbit hole > before. > > > diff --git > a/test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_tools.cpp > b/test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_tools.cpp > --- a/test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_tools.cpp > +++ b/test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_tools.cpp > @@ -196,22 +196,14 @@ > } > > > -/** > - * > - * The current option will not perform more than one > - * single option which given, this is due to places explained > - * in this question. > - * > - **/ > - > /* > - * This whole play can be reduced with simple StringTokenizer > (strtok). > - * > + * Parse a comma or space separated list of options. > */ > > int nsk_jvmti_parseOptions(const char options[]) { > size_t len; > const char* opt; > + char *str = strdup(options); > int success = NSK_TRUE; > > context.options.string = NULL; > @@ -232,7 +224,7 @@ > context.options.string[len] = '\0'; > context.options.string[len+1] = '\0'; > > - for (opt = context.options.string; ;) { > + while ((opt = strsep(&str, " ,")) != NULL) { > const char* opt_end; > const char* val_sep; > int opt_len=0; > > > > -- > > Thanks, > Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Thu Dec 20 17:10:35 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 20 Dec 2018 09:10:35 -0800 Subject: JDK-8211343: nsk_jvmti_parseoptions should handle multiple suboptions In-Reply-To: <5C1BC44D.9030305@oracle.com> References: <5C1B96B6.1040002@oracle.com> <5C1BC44D.9030305@oracle.com> Message-ID: <55bef8f2-08e5-1e43-0960-485b1ffb064e@oracle.com> An HTML attachment was scrubbed... URL: From harold.seigel at oracle.com Thu Dec 20 21:30:55 2018 From: harold.seigel at oracle.com (Harold David Seigel) Date: Thu, 20 Dec 2018 16:30:55 -0500 Subject: RFR 8215398: -Xlog option usage => Invalid decorator '\temp\app_cds.log'. In-Reply-To: <37e19362-0065-eb00-9bfb-7dcd67f55318@oracle.com> References: <71efe071-62c1-b1cc-c98f-953c6790eae1@oracle.com> <37e19362-0065-eb00-9bfb-7dcd67f55318@oracle.com> Message-ID: <3b0ff6e7-0fa6-d631-e203-c6070c1e66f0@oracle.com> Hi David, Thanks for looking at this! Please review this updated webrev.? The fix is the same but the webrev contains a new test instead of modifying an existing test: http://cr.openjdk.java.net/~hseigel/bug_8215398.2/webrev/ The logging implementation does not handle single quotes as expected.? For example, if the TestQuotedLogOutputs.java test is changed to use single quotes instead of double quotes, it will fail, even on Linux, because the single quotes are included as part of the file name.? They are not stripped off.? That is why "java -Xlog:safepoint=trace:'d:\safepointtrace.txt' -version" fails. Perhaps a new bug is needed to change logging's handling of single quotes?? It's a bit surprising that this issue hasn't already been reported. Thanks, Harold On 12/20/2018 2:07 AM, David Holmes wrote: > Hi Harold, > > On 19/12/2018 11:22 pm, Harold David Seigel wrote: >> Hi, >> >> Please review this small change to fix JDK-8215398.? The fix works by >> not treating ':' as a delimiter in a -Xlog... option string if it is >> following by a '\' and preceded by either a single character or the >> text 'file='.? The fix is for Windows only. >> >> Open Webrev: >> http://cr.openjdk.java.net/~hseigel/bug_8215398/webrev/index.html > > I think I can follow the fix. > > But I'm a bit concerned about the test. AFAICS the test thought it was > already testing this case - albeit with the path quoted: > > ? 42???????? // Ensure log files can be specified with full path. > ? 43???????? // On windows, this means that the file name will contain > ? 44???????? // a colon ('C:\log.txt' for example), which is used to > ? 45???????? // separate -Xlog: options (-Xlog:tags:filename:decorators). > ? 46???????? // Try to log to a file in our current directory, using > its absolute path. > ? 47???????? String baseName = "test file.log"; > ? 48???????? Path filePath = Paths.get(baseName).toAbsolutePath(); > ? 49???????? String fileName = filePath.toString(); > ? 50???????? File file = filePath.toFile(); > ... > ? 65???????? String[] validOutputs = new String[] { > ? 66???????????? quote + fileName + quote, > ? 67???????????? "file=" + quote + fileName + quote, > ? 68???????????? quote + fileName + quote + ":", > ? 69???????????? quote + fileName + quote + "::" > ? 70???????? }; > > But even quoted if I specify the drive designator in the path, it > fails! Here's a local attempt at this: > > D:\ade> apps\Java\jdk-11\fastdebug\bin\java > -Xlog:safepoint=trace:'d:\safepointtrace.txt' -version > [0.014s][error][logging] Invalid decorator '\safepointtrace.txt''. > Invalid -Xlog option '-Xlog:safepoint=trace:'d:\safepointtrace.txt'', > see error log for details. > Error: Could not create the Java Virtual Machine. > Error: A fatal exception has occurred. Program will exit. > > So AFAICS the test can't possibly have been testing what it thought it > was testing! ??? > > I'm also not sure about just adding some unquoted variants to the > existing TestQuotedLogOutputs without renaming the test and updating > the @summary > > Thanks, > David > >> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8215398 >> >> The fix was regression tested by running Mach5 tiers 1 and 2 tests >> and builds on Linux-x64, Windows, and Mac OS X, running tiers 3-5 >> tests on Linux-x64, running JCK-12 Lang and VM tests on Linux-x64, >> and by hand on Windows. >> >> Thanks, Harol >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexey.menkov at oracle.com Fri Dec 21 01:04:34 2018 From: alexey.menkov at oracle.com (Alex Menkov) Date: Thu, 20 Dec 2018 17:04:34 -0800 Subject: RFR JDK-8215716: PopFrame() was unexpectedly done Message-ID: Hi all, please review a small fix for https://bugs.openjdk.java.net/browse/JDK-8215716 webrev: http://cr.openjdk.java.net/~amenkov/popframe004/webrev.01/ The failure is caused by race condition - main test thread calls PopFrame expecting that test thread is in the activeMethod method. But logging (System.out.println call) is performed after main frame is notified that test thread is ready to be popped. More general --alex From david.holmes at oracle.com Fri Dec 21 01:09:16 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 21 Dec 2018 11:09:16 +1000 Subject: RFR JDK-8215716: PopFrame() was unexpectedly done In-Reply-To: References: Message-ID: <3233b246-7a90-9547-081b-435b8e1f79f2@oracle.com> Looks good. Thanks, David On 21/12/2018 11:04 am, Alex Menkov wrote: > Hi all, > > please review a small fix for > https://bugs.openjdk.java.net/browse/JDK-8215716 > webrev: > http://cr.openjdk.java.net/~amenkov/popframe004/webrev.01/ > > The failure is caused by race condition - main test thread calls > PopFrame expecting that test thread is in the activeMethod method. > But logging (System.out.println call) is performed after main frame is > notified that test thread is ready to be popped. > > More general > > --alex From serguei.spitsyn at oracle.com Fri Dec 21 01:17:51 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 20 Dec 2018 17:17:51 -0800 Subject: RFR JDK-8215716: PopFrame() was unexpectedly done In-Reply-To: <3233b246-7a90-9547-081b-435b8e1f79f2@oracle.com> References: <3233b246-7a90-9547-081b-435b8e1f79f2@oracle.com> Message-ID: <4af8268f-8eff-c69e-b425-77aaa1aa90f1@oracle.com> Hi Alex, LG++ Thanks, Serguei On 12/20/18 5:09 PM, David Holmes wrote: > Looks good. > > Thanks, > David > > On 21/12/2018 11:04 am, Alex Menkov wrote: >> Hi all, >> >> please review a small fix for >> https://bugs.openjdk.java.net/browse/JDK-8215716 >> webrev: >> http://cr.openjdk.java.net/~amenkov/popframe004/webrev.01/ >> >> The failure is caused by race condition - main test thread calls >> PopFrame expecting that test thread is in the activeMethod method. >> But logging (System.out.println call) is performed after main frame >> is notified that test thread is ready to be popped. >> >> More general >> >> --alex From serguei.spitsyn at oracle.com Fri Dec 21 01:40:41 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 20 Dec 2018 17:40:41 -0800 Subject: RFR 8215398: -Xlog option usage => Invalid decorator '\temp\app_cds.log'. In-Reply-To: <3b0ff6e7-0fa6-d631-e203-c6070c1e66f0@oracle.com> References: <71efe071-62c1-b1cc-c98f-953c6790eae1@oracle.com> <37e19362-0065-eb00-9bfb-7dcd67f55318@oracle.com> <3b0ff6e7-0fa6-d631-e203-c6070c1e66f0@oracle.com> Message-ID: <949fde0a-e386-5377-905f-f1ac336bfdb2@oracle.com> Hi Harold, It looks good to me. Thanks, Serguei On 12/20/18 1:30 PM, Harold David Seigel wrote: > Hi David, > > Thanks for looking at this! > > Please review this updated webrev.? The fix is the same but the webrev > contains a new test instead of modifying an existing test: > > ?? http://cr.openjdk.java.net/~hseigel/bug_8215398.2/webrev/ > > The logging implementation does not handle single quotes as expected.? > For example, if the TestQuotedLogOutputs.java test is changed to use > single quotes instead of double quotes, it will fail, even on Linux, > because the single quotes are included as part of the file name.? They > are not stripped off.? That is why "java > -Xlog:safepoint=trace:'d:\safepointtrace.txt' -version" fails. > > Perhaps a new bug is needed to change logging's handling of single > quotes?? It's a bit surprising that this issue hasn't already been > reported. > > Thanks, Harold > > On 12/20/2018 2:07 AM, David Holmes wrote: >> Hi Harold, >> >> On 19/12/2018 11:22 pm, Harold David Seigel wrote: >>> Hi, >>> >>> Please review this small change to fix JDK-8215398.? The fix works >>> by not treating ':' as a delimiter in a -Xlog... option string if it >>> is following by a '\' and preceded by either a single character or >>> the text 'file='.? The fix is for Windows only. >>> >>> Open Webrev: >>> http://cr.openjdk.java.net/~hseigel/bug_8215398/webrev/index.html >> >> I think I can follow the fix. >> >> But I'm a bit concerned about the test. AFAICS the test thought it >> was already testing this case - albeit with the path quoted: >> >> ? 42???????? // Ensure log files can be specified with full path. >> ? 43???????? // On windows, this means that the file name will contain >> ? 44???????? // a colon ('C:\log.txt' for example), which is used to >> ? 45???????? // separate -Xlog: options >> (-Xlog:tags:filename:decorators). >> ? 46???????? // Try to log to a file in our current directory, using >> its absolute path. >> ? 47???????? String baseName = "test file.log"; >> ? 48???????? Path filePath = Paths.get(baseName).toAbsolutePath(); >> ? 49???????? String fileName = filePath.toString(); >> ? 50???????? File file = filePath.toFile(); >> ... >> ? 65???????? String[] validOutputs = new String[] { >> ? 66???????????? quote + fileName + quote, >> ? 67???????????? "file=" + quote + fileName + quote, >> ? 68???????????? quote + fileName + quote + ":", >> ? 69???????????? quote + fileName + quote + "::" >> ? 70???????? }; >> >> But even quoted if I specify the drive designator in the path, it >> fails! Here's a local attempt at this: >> >> D:\ade> apps\Java\jdk-11\fastdebug\bin\java >> -Xlog:safepoint=trace:'d:\safepointtrace.txt' -version >> [0.014s][error][logging] Invalid decorator '\safepointtrace.txt''. >> Invalid -Xlog option '-Xlog:safepoint=trace:'d:\safepointtrace.txt'', >> see error log for details. >> Error: Could not create the Java Virtual Machine. >> Error: A fatal exception has occurred. Program will exit. >> >> So AFAICS the test can't possibly have been testing what it thought >> it was testing! ??? >> >> I'm also not sure about just adding some unquoted variants to the >> existing TestQuotedLogOutputs without renaming the test and updating >> the @summary >> >> Thanks, >> David >> >>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8215398 >>> >>> The fix was regression tested by running Mach5 tiers 1 and 2 tests >>> and builds on Linux-x64, Windows, and Mac OS X, running tiers 3-5 >>> tests on Linux-x64, running JCK-12 Lang and VM tests on Linux-x64, >>> and by hand on Windows. >>> >>> Thanks, Harol >>> > From david.holmes at oracle.com Fri Dec 21 02:28:33 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 21 Dec 2018 12:28:33 +1000 Subject: RFR 8215398: -Xlog option usage => Invalid decorator '\temp\app_cds.log'. In-Reply-To: <3b0ff6e7-0fa6-d631-e203-c6070c1e66f0@oracle.com> References: <71efe071-62c1-b1cc-c98f-953c6790eae1@oracle.com> <37e19362-0065-eb00-9bfb-7dcd67f55318@oracle.com> <3b0ff6e7-0fa6-d631-e203-c6070c1e66f0@oracle.com> Message-ID: On 21/12/2018 7:30 am, Harold David Seigel wrote: > Hi David, > > Thanks for looking at this! > > Please review this updated webrev.? The fix is the same but the webrev > contains a new test instead of modifying an existing test: > > http://cr.openjdk.java.net/~hseigel/bug_8215398.2/webrev/ I think renaming the existing test would be better as the only difference is the use of quotes. But if you want a new test please add: @bug 8215398 > The logging implementation does not handle single quotes as expected. > For example, if the TestQuotedLogOutputs.java test is changed to use > single quotes instead of double quotes, it will fail, even on Linux, > because the single quotes are included as part of the file name.? They > are not stripped off.? That is why "java > -Xlog:safepoint=trace:'d:\safepointtrace.txt' -version" fails. It fails with double-quotes too! AFAICS it always fails in a Windows command shell. Maybe the test only passes in cygwin shell? That said I don't understand your comment. I expect the quotes to delimit the filename in case the filename has spaces in the path component. If they aren't stripped off it wouldn't be tokenizing at the internal ":", so ... ??? Thanks, David > Perhaps a new bug is needed to change logging's handling of single > quotes?? It's a bit surprising that this issue hasn't already been reported. > > Thanks, Harold > > On 12/20/2018 2:07 AM, David Holmes wrote: >> Hi Harold, >> >> On 19/12/2018 11:22 pm, Harold David Seigel wrote: >>> Hi, >>> >>> Please review this small change to fix JDK-8215398.? The fix works by >>> not treating ':' as a delimiter in a -Xlog... option string if it is >>> following by a '\' and preceded by either a single character or the >>> text 'file='.? The fix is for Windows only. >>> >>> Open Webrev: >>> http://cr.openjdk.java.net/~hseigel/bug_8215398/webrev/index.html >> >> I think I can follow the fix. >> >> But I'm a bit concerned about the test. AFAICS the test thought it was >> already testing this case - albeit with the path quoted: >> >> ? 42???????? // Ensure log files can be specified with full path. >> ? 43???????? // On windows, this means that the file name will contain >> ? 44???????? // a colon ('C:\log.txt' for example), which is used to >> ? 45???????? // separate -Xlog: options (-Xlog:tags:filename:decorators). >> ? 46???????? // Try to log to a file in our current directory, using >> its absolute path. >> ? 47???????? String baseName = "test file.log"; >> ? 48???????? Path filePath = Paths.get(baseName).toAbsolutePath(); >> ? 49???????? String fileName = filePath.toString(); >> ? 50???????? File file = filePath.toFile(); >> ... >> ? 65???????? String[] validOutputs = new String[] { >> ? 66???????????? quote + fileName + quote, >> ? 67???????????? "file=" + quote + fileName + quote, >> ? 68???????????? quote + fileName + quote + ":", >> ? 69???????????? quote + fileName + quote + "::" >> ? 70???????? }; >> >> But even quoted if I specify the drive designator in the path, it >> fails! Here's a local attempt at this: >> >> D:\ade> apps\Java\jdk-11\fastdebug\bin\java >> -Xlog:safepoint=trace:'d:\safepointtrace.txt' -version >> [0.014s][error][logging] Invalid decorator '\safepointtrace.txt''. >> Invalid -Xlog option '-Xlog:safepoint=trace:'d:\safepointtrace.txt'', >> see error log for details. >> Error: Could not create the Java Virtual Machine. >> Error: A fatal exception has occurred. Program will exit. >> >> So AFAICS the test can't possibly have been testing what it thought it >> was testing! ??? >> >> I'm also not sure about just adding some unquoted variants to the >> existing TestQuotedLogOutputs without renaming the test and updating >> the @summary >> >> Thanks, >> David >> >>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8215398 >>> >>> The fix was regression tested by running Mach5 tiers 1 and 2 tests >>> and builds on Linux-x64, Windows, and Mac OS X, running tiers 3-5 >>> tests on Linux-x64, running JCK-12 Lang and VM tests on Linux-x64, >>> and by hand on Windows. >>> >>> Thanks, Harol >>> > From zanglin5 at jd.com Fri Dec 21 07:01:19 2018 From: zanglin5 at jd.com (=?gb2312?B?6rDB1Q==?=) Date: Fri, 21 Dec 2018 07:01:19 +0000 Subject: [RFR]8215622: Add dump to file support for jmap histo Message-ID: <336c32cab24f42e388ccab4910c0d3b6@jd.com> Hi All, May I ask your help to review this patch for enhance jmap ?Chisto. It add the ??file=?? arguments that allow jmap ?Chisto outputs data to file directly. This patch is also part of the enhancement described in https://bugs.openjdk.java.net/browse/JDK-8214535. Webrev: http://cr.openjdk.java.net/~xiaofeya/8215622/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8215622 Thanks. BRs, Lin -------------- next part -------------- An HTML attachment was scrubbed... URL: From zanglin5 at jd.com Fri Dec 21 07:11:39 2018 From: zanglin5 at jd.com (=?gb2312?B?6rDB1Q==?=) Date: Fri, 21 Dec 2018 07:11:39 +0000 Subject: [RFR]8215623: Add incremental dump for jmap histo Message-ID: <66546343ae324ab28bb54951ad774a89@jd.com> Hi All, May I ask your help to review this patch for enhance jmap ?Chisto. It adds the ??incremental?? arguments that allow jmap ?Chisto to incrementally save the intermediate data into a temp file. The intermediate data is dumped incrementally and write to a rolling file, which limit the size of the temp file to be small. This is useful for user to get intermediate results when jmap/jvm process is killed accidentally. Especially when the heap is large. This patch is also part of the enhancement described in https://bugs.openjdk.java.net/browse/JDK-8214535. Webrev: http://cr.openjdk.java.net/~xiaofeya/8215623/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8215623 Thanks. BRs, Lin -------------- next part -------------- An HTML attachment was scrubbed... URL: From gary.adams at oracle.com Fri Dec 21 13:33:33 2018 From: gary.adams at oracle.com (Gary Adams) Date: Fri, 21 Dec 2018 08:33:33 -0500 Subject: JDK-8211343: nsk_jvmti_parseoptions should handle multiple suboptions In-Reply-To: <55bef8f2-08e5-1e43-0960-485b1ffb064e@oracle.com> References: <5C1B96B6.1040002@oracle.com> <5C1BC44D.9030305@oracle.com> <55bef8f2-08e5-1e43-0960-485b1ffb064e@oracle.com> Message-ID: <5C1CEBAD.1080601@oracle.com> My intention is to establish a more robust argument parsing and then see how much code and comments can be replaced. Since strsep is BSD based and not available on windows, an alternative is to use strpbrk which is on windows and posix. #include #include char* token(char **s, char *delim) { char *p; char *start = *s; if (s == NULL || *s == NULL) { return NULL; } p = strpbrk(*s, delim); if (p != NULL) { /* Advance to next token. */ *p = '\0'; *s = p + 1; } else { /* End of tokens. */ *s = NULL; } return start; } int main(int argc, char **argv){ char *str = strdup("waittime=5,verbose foo bar= = =rab baz=11"); char *name; char *value; while ((name = token(&str, " ,")) != NULL) { value = index(name, '='); if (value == NULL) { printf ("%s\n", name); } else { *value++ = '\0'; printf ("%s=%s\n", name, value); } } } ... ./main waittime=5 verbose foo bar= = =rab baz=11 On 12/20/18, 12:10 PM, Chris Plummer wrote: > Wow, the existing comments for this function take a lot of work to > translate. You basically need to understand the code in order to > understand the comment. Kind of backwards. Below I've included all the > existing code and comments, with my translation of the comments and > also additional annotations. > > But before getting into that, I think the proposed fix is just working > around all the bugs elsewhere in the function by making opt point to a > string that just contains the current option we are working on, rather > than attempting (poorly and incorrectly) to point to the next option > within the options string. Although tokenizing the string in this way > is probably the better approach, it would be nice to at the same time > clean up all the other errors and comments in this code. > > /** > * > * The current option will not perform more than one > * single option which given, this is due to places explained > * in this question. > * > **/ > > # This current implementation will not parse more than one option. The > # reason is explained in comments below. > > /* > * This whole play can be reduced with simple StringTokenizer (strtok). > * > */ > > # This function can be simplified by using strok(). > > int nsk_jvmti_parseOptions(const char options[]) { > size_t len; > const char* opt; > int success = NSK_TRUE; > > context.options.string = NULL; > context.options.count = 0; > context.waittime = 2; > > if (options == NULL) > return NSK_TRUE; > > len = strlen(options); > context.options.string = (char*)malloc(len + 2); > > if (context.options.string == NULL) { > nsk_complain("nsk_jvmti_parseOptions(): out of memory\n"); > return NSK_FALSE; > } > strncpy(context.options.string, options, len); > context.options.string[len] = '\0'; > context.options.string[len+1] = '\0'; > > for (opt = context.options.string; ;) { > const char* opt_end; > const char* val_sep; > int opt_len=0; > int val_len=0; > int exit=1; > > while (*opt != '\0' && isOptSep(*opt)) opt++; > if (*opt == '\0') break; > > val_sep = NULL; > /* > This should break when the first option it encounters > other wise > */ > # This should break at the end of the first option, before the option > value is specified > # if there is an option value. > for (opt_end = opt, opt_len=0; !(*opt_end == '\0' || > isOptSep(*opt_end)); opt_end++,opt_len++) { > if (*opt_end == NSK_JVMTI_OPTION_VAL_SEP) { > val_sep = opt_end; > exit=0; > break; > } > } > > if (exit == 1) break; > > /* now scan for the search for the option value end. > > */ > # Now scan for the end of the option value. > [Chris: This is a bug because it assumes that there is a value. If we > stopped in the above > loop due to finding the option separator (which also seems to be > broken), then we start > reading the next option as the option value.] > exit =1; > opt_end++; > val_sep++; > /** > * I was expecting this jvmti_parseOptions(), > * should be for multiple options as well. > * If this break is not there then It will expects > * to have. so a space should be sufficient as well. > */ > # I was expecting that nsk_jvmti_parseOptions() would parse multiple > options. > [Chris: I have no idea what the rest of the comment means. Due to the > bug above > with handling an option with no value, the code below doesn't do what > was expected. > The commented out part of it tried to do the right thing by searching > for the '-' for the > next option, whichis wrong because options don't begin with a '\'. > They are separated > by a comma, although the code elsewhere wants them separated by a > space or a `~`.] > for (val_len=0; !(*opt_end == '\0' || isOptSep(*opt_end)); > opt_end++,val_len++) { > //if (*opt_end == NSK_JVMTI_OPTION_START) { > // break; > //} > } > > if (!add_option(opt, opt_len, val_sep, val_len)) { > success = NSK_FALSE; > break; > } > opt_end++; > opt = opt_end; > } > > > On 12/20/18 8:33 AM, Gary Adams wrote: >> I prototyped with strsep, because strtok is not safe >> and did not want to deal with the strtok_r versus >> strtok_s (windows) platform variants. >> >> ... >> #include >> #include >> >> int main(int argc, char **argv){ >> char *str = strdup("waittime=5,verbose foo bar baz=11"); >> char *name; >> char *value; >> >> while ((name = strsep (&str, " ,")) != NULL) { >> value = index(name, '='); >> >> if (value == NULL) { >> printf ("%s\n", name); >> } else { >> *value++ = '\0'; >> printf ("%s=%s\n", name, value); >> } >> } >> >> ... >> >> ./main >> waittime=5 >> verbose >> foo >> bar >> baz=11 >> >> >> On 12/20/18, 10:54 AM, JC Beyler wrote: >>> Hi Gary, >>> >>> I had seen that too and forgot to file it! So thanks! >>> >>> I think the comment you removed is interesting, I'm not sure what it >>> means exactly and I've tried re-reading a few times but the use of >>> "in this question" is weird :-) >>> >>> Anyway, the webrev looks good except perhaps for the use of strsep, >>> which is BSD, instead of strtok, which is C89/C99/Posix. >>> >>> Thanks for doing this! >>> Jc >>> >>> On Thu, Dec 20, 2018 at 5:17 AM Gary Adams >> > wrote: >>> >>> During some earlier jvmti test debugging, I noticed that it was not >>> possible to >>> add a quick argument to the current tests and rerun the test. e.g. >>> expand "waittime=5" to >>> "waittime=5,verbose". I tracked down the options parsing in >>> jvmti_tools.cpp and saw some >>> comments that only a single option could be parsed. >>> >>> So I filed this bug to revisit the issue for the next release: >>> >>> Issue: https://bugs.openjdk.java.net/browse/JDK-8211343 >>> >>> I think the option parsing should be ripped out and redone, >>> but for now I think a simple tweak to use strsep(), might go a >>> long way >>> to solving the multiple option support. I just started testing, >>> but wanted to know if anyone else has been down this rabbit hole >>> before. >>> >>> >>> diff --git >>> a/test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_tools.cpp >>> b/test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_tools.cpp >>> --- a/test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_tools.cpp >>> +++ b/test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_tools.cpp >>> @@ -196,22 +196,14 @@ >>> } >>> >>> >>> -/** >>> - * >>> - * The current option will not perform more than one >>> - * single option which given, this is due to places explained >>> - * in this question. >>> - * >>> - **/ >>> - >>> /* >>> - * This whole play can be reduced with simple StringTokenizer >>> (strtok). >>> - * >>> + * Parse a comma or space separated list of options. >>> */ >>> >>> int nsk_jvmti_parseOptions(const char options[]) { >>> size_t len; >>> const char* opt; >>> + char *str = strdup(options); >>> int success = NSK_TRUE; >>> >>> context.options.string = NULL; >>> @@ -232,7 +224,7 @@ >>> context.options.string[len] = '\0'; >>> context.options.string[len+1] = '\0'; >>> >>> - for (opt = context.options.string; ;) { >>> + while ((opt = strsep(&str, " ,")) != NULL) { >>> const char* opt_end; >>> const char* val_sep; >>> int opt_len=0; >>> >>> >>> >>> -- >>> >>> Thanks, >>> Jc >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From harold.seigel at oracle.com Fri Dec 21 15:50:49 2018 From: harold.seigel at oracle.com (Harold David Seigel) Date: Fri, 21 Dec 2018 10:50:49 -0500 Subject: RFR 8215398: -Xlog option usage => Invalid decorator '\temp\app_cds.log'. In-Reply-To: <949fde0a-e386-5377-905f-f1ac336bfdb2@oracle.com> References: <71efe071-62c1-b1cc-c98f-953c6790eae1@oracle.com> <37e19362-0065-eb00-9bfb-7dcd67f55318@oracle.com> <3b0ff6e7-0fa6-d631-e203-c6070c1e66f0@oracle.com> <949fde0a-e386-5377-905f-f1ac336bfdb2@oracle.com> Message-ID: Thanks Serguei! Harold On 12/20/2018 8:40 PM, serguei.spitsyn at oracle.com wrote: > Hi Harold, > > It looks good to me. > > Thanks, > Serguei > > On 12/20/18 1:30 PM, Harold David Seigel wrote: >> Hi David, >> >> Thanks for looking at this! >> >> Please review this updated webrev.? The fix is the same but the >> webrev contains a new test instead of modifying an existing test: >> >> ?? http://cr.openjdk.java.net/~hseigel/bug_8215398.2/webrev/ >> >> The logging implementation does not handle single quotes as >> expected.? For example, if the TestQuotedLogOutputs.java test is >> changed to use single quotes instead of double quotes, it will fail, >> even on Linux, because the single quotes are included as part of the >> file name.? They are not stripped off.? That is why "java >> -Xlog:safepoint=trace:'d:\safepointtrace.txt' -version" fails. >> >> Perhaps a new bug is needed to change logging's handling of single >> quotes?? It's a bit surprising that this issue hasn't already been >> reported. >> >> Thanks, Harold >> >> On 12/20/2018 2:07 AM, David Holmes wrote: >>> Hi Harold, >>> >>> On 19/12/2018 11:22 pm, Harold David Seigel wrote: >>>> Hi, >>>> >>>> Please review this small change to fix JDK-8215398.? The fix works >>>> by not treating ':' as a delimiter in a -Xlog... option string if >>>> it is following by a '\' and preceded by either a single character >>>> or the text 'file='.? The fix is for Windows only. >>>> >>>> Open Webrev: >>>> http://cr.openjdk.java.net/~hseigel/bug_8215398/webrev/index.html >>> >>> I think I can follow the fix. >>> >>> But I'm a bit concerned about the test. AFAICS the test thought it >>> was already testing this case - albeit with the path quoted: >>> >>> ? 42???????? // Ensure log files can be specified with full path. >>> ? 43???????? // On windows, this means that the file name will contain >>> ? 44???????? // a colon ('C:\log.txt' for example), which is used to >>> ? 45???????? // separate -Xlog: options >>> (-Xlog:tags:filename:decorators). >>> ? 46???????? // Try to log to a file in our current directory, using >>> its absolute path. >>> ? 47???????? String baseName = "test file.log"; >>> ? 48???????? Path filePath = Paths.get(baseName).toAbsolutePath(); >>> ? 49???????? String fileName = filePath.toString(); >>> ? 50???????? File file = filePath.toFile(); >>> ... >>> ? 65???????? String[] validOutputs = new String[] { >>> ? 66???????????? quote + fileName + quote, >>> ? 67???????????? "file=" + quote + fileName + quote, >>> ? 68???????????? quote + fileName + quote + ":", >>> ? 69???????????? quote + fileName + quote + "::" >>> ? 70???????? }; >>> >>> But even quoted if I specify the drive designator in the path, it >>> fails! Here's a local attempt at this: >>> >>> D:\ade> apps\Java\jdk-11\fastdebug\bin\java >>> -Xlog:safepoint=trace:'d:\safepointtrace.txt' -version >>> [0.014s][error][logging] Invalid decorator '\safepointtrace.txt''. >>> Invalid -Xlog option >>> '-Xlog:safepoint=trace:'d:\safepointtrace.txt'', see error log for >>> details. >>> Error: Could not create the Java Virtual Machine. >>> Error: A fatal exception has occurred. Program will exit. >>> >>> So AFAICS the test can't possibly have been testing what it thought >>> it was testing! ??? >>> >>> I'm also not sure about just adding some unquoted variants to the >>> existing TestQuotedLogOutputs without renaming the test and updating >>> the @summary >>> >>> Thanks, >>> David >>> >>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8215398 >>>> >>>> The fix was regression tested by running Mach5 tiers 1 and 2 tests >>>> and builds on Linux-x64, Windows, and Mac OS X, running tiers 3-5 >>>> tests on Linux-x64, running JCK-12 Lang and VM tests on Linux-x64, >>>> and by hand on Windows. >>>> >>>> Thanks, Harol >>>> >> > From harold.seigel at oracle.com Fri Dec 21 18:32:30 2018 From: harold.seigel at oracle.com (Harold David Seigel) Date: Fri, 21 Dec 2018 10:32:30 -0800 (PST) Subject: RFR 8215398: -Xlog option usage => Invalid decorator '\temp\app_cds.log'. In-Reply-To: References: <71efe071-62c1-b1cc-c98f-953c6790eae1@oracle.com> <37e19362-0065-eb00-9bfb-7dcd67f55318@oracle.com> <3b0ff6e7-0fa6-d631-e203-c6070c1e66f0@oracle.com> Message-ID: <1a8d1c28-bc6c-da60-11bb-a57b48ee90b9@oracle.com> Hi David, Here is the latest webrev.? The only difference between this and the previous webrev is the addition of "@bug 8215398" to the test: http://cr.openjdk.java.net/~hseigel/bug_8215398.3/webrev/index.html The point that I'm trying to make about logging's handling of single quotes vs. double quotes can perhaps best be shown on Linux: If I execute the following commands (on Linux): java -Xlog:safepoint=trace:\"dquotes.log\" -version java -Xlog:safepoint=trace:\'squotes.log\' -version and then list out the current directory: > ls 'squotes.log'? dquotes.log it shows that logging stripped off the double quotes, but not the single quotes. With my fix, these commands succeed on Windows even when run from the Windows shell: java -Xlog:safepoint=trace:d:\safepointtrace.txt -version java -Xlog:safepoint=trace:"d:\safepointtrace.txt" -version java -Xlog:safepoint=trace:"d:\safepoint trace.txt" -version This one continues to fail because of the single quote issue described above: java -Xlog:safepoint=trace:'d:\safepointtrace.txt' -version I hope we can move forward with this change. Thanks, Harold On 12/20/2018 9:28 PM, David Holmes wrote: > On 21/12/2018 7:30 am, Harold David Seigel wrote: >> Hi David, >> >> Thanks for looking at this! >> >> Please review this updated webrev.? The fix is the same but the >> webrev contains a new test instead of modifying an existing test: >> >> ??? http://cr.openjdk.java.net/~hseigel/bug_8215398.2/webrev/ > > I think renaming the existing test would be better as the only > difference is the use of quotes. But if you want a new test please add: > > @bug 8215398 > >> The logging implementation does not handle single quotes as >> expected.? For example, if the TestQuotedLogOutputs.java test is >> changed to use single quotes instead of double quotes, it will fail, >> even on Linux, because the single quotes are included as part of the >> file name.? They are not stripped off.? That is why "java >> -Xlog:safepoint=trace:'d:\safepointtrace.txt' -version" fails. > > It fails with double-quotes too! AFAICS it always fails in a Windows > command shell. Maybe the test only passes in cygwin shell? > > That said I don't understand your comment. I expect the quotes to > delimit the filename in case the filename has spaces in the path > component. If they aren't stripped off it wouldn't be tokenizing at > the internal ":", so ... ??? > > Thanks, > David > >> Perhaps a new bug is needed to change logging's handling of single >> quotes?? It's a bit surprising that this issue hasn't already been >> reported. >> >> Thanks, Harold >> >> On 12/20/2018 2:07 AM, David Holmes wrote: >>> Hi Harold, >>> >>> On 19/12/2018 11:22 pm, Harold David Seigel wrote: >>>> Hi, >>>> >>>> Please review this small change to fix JDK-8215398.? The fix works >>>> by not treating ':' as a delimiter in a -Xlog... option string if >>>> it is following by a '\' and preceded by either a single character >>>> or the text 'file='.? The fix is for Windows only. >>>> >>>> Open Webrev: >>>> http://cr.openjdk.java.net/~hseigel/bug_8215398/webrev/index.html >>> >>> I think I can follow the fix. >>> >>> But I'm a bit concerned about the test. AFAICS the test thought it >>> was already testing this case - albeit with the path quoted: >>> >>> ? 42???????? // Ensure log files can be specified with full path. >>> ? 43???????? // On windows, this means that the file name will contain >>> ? 44???????? // a colon ('C:\log.txt' for example), which is used to >>> ? 45???????? // separate -Xlog: options >>> (-Xlog:tags:filename:decorators). >>> ? 46???????? // Try to log to a file in our current directory, using >>> its absolute path. >>> ? 47???????? String baseName = "test file.log"; >>> ? 48???????? Path filePath = Paths.get(baseName).toAbsolutePath(); >>> ? 49???????? String fileName = filePath.toString(); >>> ? 50???????? File file = filePath.toFile(); >>> ... >>> ? 65???????? String[] validOutputs = new String[] { >>> ? 66???????????? quote + fileName + quote, >>> ? 67???????????? "file=" + quote + fileName + quote, >>> ? 68???????????? quote + fileName + quote + ":", >>> ? 69???????????? quote + fileName + quote + "::" >>> ? 70???????? }; >>> >>> But even quoted if I specify the drive designator in the path, it >>> fails! Here's a local attempt at this: >>> >>> D:\ade> apps\Java\jdk-11\fastdebug\bin\java >>> -Xlog:safepoint=trace:'d:\safepointtrace.txt' -version >>> [0.014s][error][logging] Invalid decorator '\safepointtrace.txt''. >>> Invalid -Xlog option >>> '-Xlog:safepoint=trace:'d:\safepointtrace.txt'', see error log for >>> details. >>> Error: Could not create the Java Virtual Machine. >>> Error: A fatal exception has occurred. Program will exit. >>> >>> So AFAICS the test can't possibly have been testing what it thought >>> it was testing! ??? >>> >>> I'm also not sure about just adding some unquoted variants to the >>> existing TestQuotedLogOutputs without renaming the test and updating >>> the @summary >>> >>> Thanks, >>> David >>> >>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8215398 >>>> >>>> The fix was regression tested by running Mach5 tiers 1 and 2 tests >>>> and builds on Linux-x64, Windows, and Mac OS X, running tiers 3-5 >>>> tests on Linux-x64, running JCK-12 Lang and VM tests on Linux-x64, >>>> and by hand on Windows. >>>> >>>> Thanks, Harol >>>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From gary.adams at oracle.com Fri Dec 21 18:52:18 2018 From: gary.adams at oracle.com (Gary Adams) Date: Fri, 21 Dec 2018 13:52:18 -0500 Subject: RFR: JDK-8211343: nsk_jvmti_parseoptions should handle multiple suboptions Message-ID: <5C1D3662.4070505@oracle.com> Here is a first pass at a replacement parser for jvmti test options. Webrev: http://cr.openjdk.java.net/~gadams/8211343/webrev.00/ Issue: https://bugs.openjdk.java.net/browse/JDK-8211343 Testing is in progress. Need to check out more platforms. On local linux testing one jvmti test failed SetNativeMethodPrefix001 which passed an empty options string. ... From chris.plummer at oracle.com Fri Dec 21 19:31:55 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Fri, 21 Dec 2018 11:31:55 -0800 Subject: RFR: JDK-8211343: nsk_jvmti_parseoptions should handle multiple suboptions In-Reply-To: <5C1D3662.4070505@oracle.com> References: <5C1D3662.4070505@oracle.com> Message-ID: <7635f7d3-7cc9-d257-b0f5-c3f693581d07@oracle.com> Hi Gary, This is much better. Just one question and a couple of minor comments: The original code had the following: ?225???? context.options.string = (char*)malloc(len + 2); ... ?231???? strncpy(context.options.string, options, len); ?232???? context.options.string[len] = '\0'; ?233???? context.options.string[len+1] = '\0'; Do you know why it was placing two NULLs at the end (and the first was unnecessary since strncpy already did that)? ?243???????????? *value++ = '\0'; This should be rewritten in a manner that make it clearer what it does. At first glance you have to ask yourself whether it is setting *value or *(value++). Then hopefully you eventually clue in that they are both the same since "value++" evaluates to "value". I think the following would be better: ????????????? *value = '\0'; ????????????? value++; ?235???? /* Create a temporay copy of the options string to be tokenized. */ "temporay" typo. thanks, Chris On 12/21/18 10:52 AM, Gary Adams wrote: > Here is a first pass at a replacement parser for jvmti test options. > > ? Webrev: http://cr.openjdk.java.net/~gadams/8211343/webrev.00/ > ? Issue: https://bugs.openjdk.java.net/browse/JDK-8211343 > > Testing is in progress. Need to check out more platforms. > On local linux testing one jvmti test failed SetNativeMethodPrefix001 > which passed an empty options string. > ... From serguei.spitsyn at oracle.com Fri Dec 21 20:33:45 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 21 Dec 2018 12:33:45 -0800 Subject: RFR: JDK-8211343: nsk_jvmti_parseoptions should handle multiple suboptions In-Reply-To: <7635f7d3-7cc9-d257-b0f5-c3f693581d07@oracle.com> References: <5C1D3662.4070505@oracle.com> <7635f7d3-7cc9-d257-b0f5-c3f693581d07@oracle.com> Message-ID: An HTML attachment was scrubbed... URL: From gary.adams at oracle.com Fri Dec 21 20:48:38 2018 From: gary.adams at oracle.com (gary.adams at oracle.com) Date: Fri, 21 Dec 2018 15:48:38 -0500 Subject: RFR: JDK-8211343: nsk_jvmti_parseoptions should handle multiple suboptions In-Reply-To: <7635f7d3-7cc9-d257-b0f5-c3f693581d07@oracle.com> References: <5C1D3662.4070505@oracle.com> <7635f7d3-7cc9-d257-b0f5-c3f693581d07@oracle.com> Message-ID: On 12/21/18 2:31 PM, Chris Plummer wrote: > Hi Gary, > > This is much better. Just one question and a couple of minor comments: > > The original code had the following: > > ?225???? context.options.string = (char*)malloc(len + 2); > ... > ?231???? strncpy(context.options.string, options, len); > ?232???? context.options.string[len] = '\0'; > ?233???? context.options.string[len+1] = '\0'; > > Do you know why it was placing two NULLs at the end (and the first was > unnecessary since strncpy already did that)? I choose to believe the author was confused. > > ?243???????????? *value++ = '\0'; > > This should be rewritten in a manner that make it clearer what it > does. At first glance you have to ask yourself whether it is setting > *value or *(value++). Then hopefully you eventually clue in that they > are both the same since "value++" evaluates to "value". I think the > following would be better: > > ????????????? *value = '\0'; > ????????????? value++; I believe it is a common idiom for C programs, to set a null and increment a pointer. If you think it is confusing I can change it. > > ?235???? /* Create a temporay copy of the options string to be > tokenized. */ > > "temporay" typo. Done. I did run into solaris and windows build failures, because the index() prototype is in strings.h on other platforms. I'll also try substituting strchr() so we can just include string.h I've been checking other places in the jdk sources and noticed checks for _MSC_VER, _WIN32, defined(__linix__), and NEED_BSD_STRINGS when including strings.h I'll post a fresh webrev when I get clean builds. ... > > thanks, > > Chris > > On 12/21/18 10:52 AM, Gary Adams wrote: >> Here is a first pass at a replacement parser for jvmti test options. >> >> ? Webrev: http://cr.openjdk.java.net/~gadams/8211343/webrev.00/ >> ? Issue: https://bugs.openjdk.java.net/browse/JDK-8211343 >> >> Testing is in progress. Need to check out more platforms. >> On local linux testing one jvmti test failed SetNativeMethodPrefix001 >> which passed an empty options string. >> ... > > > From gary.adams at oracle.com Fri Dec 21 20:55:27 2018 From: gary.adams at oracle.com (gary.adams at oracle.com) Date: Fri, 21 Dec 2018 15:55:27 -0500 Subject: RFR: JDK-8211343: nsk_jvmti_parseoptions should handle multiple suboptions In-Reply-To: References: <5C1D3662.4070505@oracle.com> <7635f7d3-7cc9-d257-b0f5-c3f693581d07@oracle.com> Message-ID: <45ff0fa6-312e-0eab-1399-33db44c4b57e@oracle.com> On 12/21/18 3:33 PM, serguei.spitsyn at oracle.com wrote: > Hi Gary, > > Looks good in general. > Thank you for taking care about this! > > A couple of comments. > > It seems, with your fixes the macro NSK_JVMTI_OPTION_VAL_SEP is not > needed anymore. > Also, I wonder if it makes sense to continue using: > 238 value = index(name, NSK_JVMTI_OPTION_VAL_SEP); This is one of the places where I feel having the symbolic constant actually makes the code harder to read. The constant is not used anywhere else. I'm testing now with strchr because of strings.h issues on other platforms. > > > Also, a cleanup is needed to remove the lines 241 and 244 from the > fragment: > 240 if (value == NULL) { > 241 /* debug: printf ("%s\n", name); */ > 242 } else { > 243 *value++ = '\0'; > 244 /* debug: printf ("%s=%s\n", name, value); */ > 245 } > Then it makes sense to refactor it: > if (value != NULL) { > *value++ = '\0'; > } Yes, that is the plan. Until all the testing is done, it helps to have some quick debugging available. ... > Thanks, > Serguei > > > On 12/21/18 11:31, Chris Plummer wrote: >> Hi Gary, >> >> This is much better. Just one question and a couple of minor comments: >> >> The original code had the following: >> >> ?225???? context.options.string = (char*)malloc(len + 2); >> ... >> ?231???? strncpy(context.options.string, options, len); >> ?232???? context.options.string[len] = '\0'; >> ?233???? context.options.string[len+1] = '\0'; >> >> Do you know why it was placing two NULLs at the end (and the first >> was unnecessary since strncpy already did that)? >> >> ?243???????????? *value++ = '\0'; >> >> This should be rewritten in a manner that make it clearer what it >> does. At first glance you have to ask yourself whether it is setting >> *value or *(value++). Then hopefully you eventually clue in that they >> are both the same since "value++" evaluates to "value". I think the >> following would be better: >> >> ????????????? *value = '\0'; >> ????????????? value++; >> >> ?235???? /* Create a temporay copy of the options string to be >> tokenized. */ >> >> "temporay" typo. >> >> thanks, >> >> Chris >> >> On 12/21/18 10:52 AM, Gary Adams wrote: >>> Here is a first pass at a replacement parser for jvmti test options. >>> >>> ? Webrev: http://cr.openjdk.java.net/~gadams/8211343/webrev.00/ >>> ? Issue: https://bugs.openjdk.java.net/browse/JDK-8211343 >>> >>> Testing is in progress. Need to check out more platforms. >>> On local linux testing one jvmti test failed SetNativeMethodPrefix001 >>> which passed an empty options string. >>> ... >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Fri Dec 21 22:08:08 2018 From: david.holmes at oracle.com (David Holmes) Date: Sat, 22 Dec 2018 08:08:08 +1000 Subject: RFR 8215398: -Xlog option usage => Invalid decorator '\temp\app_cds.log'. In-Reply-To: <1a8d1c28-bc6c-da60-11bb-a57b48ee90b9@oracle.com> References: <71efe071-62c1-b1cc-c98f-953c6790eae1@oracle.com> <37e19362-0065-eb00-9bfb-7dcd67f55318@oracle.com> <3b0ff6e7-0fa6-d631-e203-c6070c1e66f0@oracle.com> <1a8d1c28-bc6c-da60-11bb-a57b48ee90b9@oracle.com> Message-ID: <2602b588-9d9e-253e-a875-046846141364@oracle.com> Hi Harold, On 22/12/2018 4:32 am, Harold David Seigel wrote: > Hi David, > > Here is the latest webrev.? The only difference between this and the > previous webrev is the addition of "@bug 8215398" to the test: > > http://cr.openjdk.java.net/~hseigel/bug_8215398.3/webrev/index.html Okay. > > The point that I'm trying to make about logging's handling of single > quotes vs. double quotes can perhaps best be shown on Linux: > > If I execute the following commands (on Linux): > > java -Xlog:safepoint=trace:\"dquotes.log\" -version > > java -Xlog:safepoint=trace:\'squotes.log\' -version > > and then list out the current directory: > > > ls > 'squotes.log'? dquotes.log > > it shows that logging stripped off the double quotes, but not the single > quotes. I see - that is very odd behaviour. > With my fix, these commands succeed on Windows even when run from the > Windows shell: > > java -Xlog:safepoint=trace:d:\safepointtrace.txt -version > java -Xlog:safepoint=trace:"d:\safepointtrace.txt" -version > java -Xlog:safepoint=trace:"d:\safepoint trace.txt" -version > > This one continues to fail because of the single quote issue described > above: > > java -Xlog:safepoint=trace:'d:\safepointtrace.txt' -version > > I hope we can move forward with this change. I don't have an issue with your fix. I just don't understand how the existing test actually works when the test cases seems to fail at the command-line for me. Thanks, David > Thanks, Harold > > > On 12/20/2018 9:28 PM, David Holmes wrote: >> On 21/12/2018 7:30 am, Harold David Seigel wrote: >>> Hi David, >>> >>> Thanks for looking at this! >>> >>> Please review this updated webrev.? The fix is the same but the >>> webrev contains a new test instead of modifying an existing test: >>> >>> http://cr.openjdk.java.net/~hseigel/bug_8215398.2/webrev/ >> >> I think renaming the existing test would be better as the only >> difference is the use of quotes. But if you want a new test please add: >> >> @bug 8215398 >> >>> The logging implementation does not handle single quotes as >>> expected.? For example, if the TestQuotedLogOutputs.java test is >>> changed to use single quotes instead of double quotes, it will fail, >>> even on Linux, because the single quotes are included as part of the >>> file name.? They are not stripped off.? That is why "java >>> -Xlog:safepoint=trace:'d:\safepointtrace.txt' -version" fails. >> >> It fails with double-quotes too! AFAICS it always fails in a Windows >> command shell. Maybe the test only passes in cygwin shell? >> >> That said I don't understand your comment. I expect the quotes to >> delimit the filename in case the filename has spaces in the path >> component. If they aren't stripped off it wouldn't be tokenizing at >> the internal ":", so ... ??? >> >> Thanks, >> David >> >>> Perhaps a new bug is needed to change logging's handling of single >>> quotes?? It's a bit surprising that this issue hasn't already been >>> reported. >>> >>> Thanks, Harold >>> >>> On 12/20/2018 2:07 AM, David Holmes wrote: >>>> Hi Harold, >>>> >>>> On 19/12/2018 11:22 pm, Harold David Seigel wrote: >>>>> Hi, >>>>> >>>>> Please review this small change to fix JDK-8215398.? The fix works >>>>> by not treating ':' as a delimiter in a -Xlog... option string if >>>>> it is following by a '\' and preceded by either a single character >>>>> or the text 'file='.? The fix is for Windows only. >>>>> >>>>> Open Webrev: >>>>> http://cr.openjdk.java.net/~hseigel/bug_8215398/webrev/index.html >>>> >>>> I think I can follow the fix. >>>> >>>> But I'm a bit concerned about the test. AFAICS the test thought it >>>> was already testing this case - albeit with the path quoted: >>>> >>>> ? 42???????? // Ensure log files can be specified with full path. >>>> ? 43???????? // On windows, this means that the file name will contain >>>> ? 44???????? // a colon ('C:\log.txt' for example), which is used to >>>> ? 45???????? // separate -Xlog: options >>>> (-Xlog:tags:filename:decorators). >>>> ? 46???????? // Try to log to a file in our current directory, using >>>> its absolute path. >>>> ? 47???????? String baseName = "test file.log"; >>>> ? 48???????? Path filePath = Paths.get(baseName).toAbsolutePath(); >>>> ? 49???????? String fileName = filePath.toString(); >>>> ? 50???????? File file = filePath.toFile(); >>>> ... >>>> ? 65???????? String[] validOutputs = new String[] { >>>> ? 66???????????? quote + fileName + quote, >>>> ? 67???????????? "file=" + quote + fileName + quote, >>>> ? 68???????????? quote + fileName + quote + ":", >>>> ? 69???????????? quote + fileName + quote + "::" >>>> ? 70???????? }; >>>> >>>> But even quoted if I specify the drive designator in the path, it >>>> fails! Here's a local attempt at this: >>>> >>>> D:\ade> apps\Java\jdk-11\fastdebug\bin\java >>>> -Xlog:safepoint=trace:'d:\safepointtrace.txt' -version >>>> [0.014s][error][logging] Invalid decorator '\safepointtrace.txt''. >>>> Invalid -Xlog option >>>> '-Xlog:safepoint=trace:'d:\safepointtrace.txt'', see error log for >>>> details. >>>> Error: Could not create the Java Virtual Machine. >>>> Error: A fatal exception has occurred. Program will exit. >>>> >>>> So AFAICS the test can't possibly have been testing what it thought >>>> it was testing! ??? >>>> >>>> I'm also not sure about just adding some unquoted variants to the >>>> existing TestQuotedLogOutputs without renaming the test and updating >>>> the @summary >>>> >>>> Thanks, >>>> David >>>> >>>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8215398 >>>>> >>>>> The fix was regression tested by running Mach5 tiers 1 and 2 tests >>>>> and builds on Linux-x64, Windows, and Mac OS X, running tiers 3-5 >>>>> tests on Linux-x64, running JCK-12 Lang and VM tests on Linux-x64, >>>>> and by hand on Windows. >>>>> >>>>> Thanks, Harol >>>>> >>> > From serguei.spitsyn at oracle.com Fri Dec 21 22:26:15 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 21 Dec 2018 14:26:15 -0800 Subject: RFR: JDK-8211343: nsk_jvmti_parseoptions should handle multiple suboptions In-Reply-To: <45ff0fa6-312e-0eab-1399-33db44c4b57e@oracle.com> References: <5C1D3662.4070505@oracle.com> <7635f7d3-7cc9-d257-b0f5-c3f693581d07@oracle.com> <45ff0fa6-312e-0eab-1399-33db44c4b57e@oracle.com> Message-ID: <452e7040-0a67-7979-018a-7dc8738ac090@oracle.com> On 12/21/18 12:55 PM, gary.adams at oracle.com wrote: > On 12/21/18 3:33 PM, serguei.spitsyn at oracle.com wrote: >> Hi Gary, >> >> Looks good in general. >> Thank you for taking care about this! >> >> A couple of comments. >> >> It seems, with your fixes the macro NSK_JVMTI_OPTION_VAL_SEP is not >> needed anymore. >> Also, I wonder if it makes sense to continue using: >> 238 value = index(name, NSK_JVMTI_OPTION_VAL_SEP); > This is one of the places where I feel having the symbolic constant > actually makes the code harder to read. > > The constant is not used anywhere else. > Kind of agree with you. > I'm testing now with strchr because of strings.h issues on other > platforms. > >> >> >> Also, a cleanup is needed to remove the lines 241 and 244 from the >> fragment: >> 240 if (value == NULL) { >> 241 /* debug: printf ("%s\n", name); */ >> 242 } else { >> 243 *value++ = '\0'; >> 244 /* debug: printf ("%s=%s\n", name, value); */ >> 245 } >> Then it makes sense to refactor it: >> if (value != NULL) { >> *value++ = '\0'; >> } > > Yes, that is the plan. > > Until all the testing is done, it helps to have some quick debugging > available. > Okay. Thanks, Serguei > ... > >> Thanks, >> Serguei >> >> >> On 12/21/18 11:31, Chris Plummer wrote: >>> Hi Gary, >>> >>> This is much better. Just one question and a couple of minor comments: >>> >>> The original code had the following: >>> >>> ?225???? context.options.string = (char*)malloc(len + 2); >>> ... >>> ?231???? strncpy(context.options.string, options, len); >>> ?232???? context.options.string[len] = '\0'; >>> ?233???? context.options.string[len+1] = '\0'; >>> >>> Do you know why it was placing two NULLs at the end (and the first >>> was unnecessary since strncpy already did that)? >>> >>> ?243???????????? *value++ = '\0'; >>> >>> This should be rewritten in a manner that make it clearer what it >>> does. At first glance you have to ask yourself whether it is setting >>> *value or *(value++). Then hopefully you eventually clue in that >>> they are both the same since "value++" evaluates to "value". I think >>> the following would be better: >>> >>> ????????????? *value = '\0'; >>> ????????????? value++; >>> >>> ?235???? /* Create a temporay copy of the options string to be >>> tokenized. */ >>> >>> "temporay" typo. >>> >>> thanks, >>> >>> Chris >>> >>> On 12/21/18 10:52 AM, Gary Adams wrote: >>>> Here is a first pass at a replacement parser for jvmti test options. >>>> >>>> ? Webrev: http://cr.openjdk.java.net/~gadams/8211343/webrev.00/ >>>> ? Issue: https://bugs.openjdk.java.net/browse/JDK-8211343 >>>> >>>> Testing is in progress. Need to check out more platforms. >>>> On local linux testing one jvmti test failed SetNativeMethodPrefix001 >>>> which passed an empty options string. >>>> ... >>> >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ecki at zusammenkunft.net Fri Dec 21 23:31:10 2018 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Sat, 22 Dec 2018 00:31:10 +0100 Subject: RFR 8215398: -Xlog option usage => Invalid decorator'\temp\app_cds.log'. In-Reply-To: <2602b588-9d9e-253e-a875-046846141364@oracle.com> References: <71efe071-62c1-b1cc-c98f-953c6790eae1@oracle.com> <37e19362-0065-eb00-9bfb-7dcd67f55318@oracle.com> <3b0ff6e7-0fa6-d631-e203-c6070c1e66f0@oracle.com> <1a8d1c28-bc6c-da60-11bb-a57b48ee90b9@oracle.com> <2602b588-9d9e-253e-a875-046846141364@oracle.com> Message-ID: <5c1d77bd.1c69fb81.695ed.d412@mx.google.com> Hello, David: > I don't have an issue with your fix. I just don't understand how the > existing test actually works when the test cases seems to fail at the > command-line for me. I think the difference between commandlie and testcase is, that the command line Interpreter might do some additional Interpretation of quotes and the process builder might transparently wrap Arguments in additional quotes. If I recall correctly on Windows the launcher does some quite Interpretation, not on *nix. Maybe ist best to compare the command line of the running process to be sure all intended cases are tested? (But for Windows ist not that easy to pass those quotes as theyare removed on muiltiple Levels ? I think tripple quotes might pass through), so supporting single quotes and also raw drive letter (even when seem uncanonical interpretation) seems like a good idea.) On Linux I would rather expect to generate filenames with single quotes if it is docmented to look for double quotes only. (I I pass a quoted single Quote to a command it will be part of the file name on most unix Tools) Gruss Bernd -- http://bernd.eckenfels.net Von: David Holmes Gesendet: Freitag, 21. Dezember 2018 23:38 An: Harold David Seigel; hotspot-runtime-dev at openjdk.java.net; serviceability-dev at openjdk.java.net Betreff: Re: RFR 8215398: -Xlog option usage => Invalid decorator'\temp\app_cds.log'. Hi Harold, On 22/12/2018 4:32 am, Harold David Seigel wrote: > Hi David, > > Here is the latest webrev.? The only difference between this and the > previous webrev is the addition of "@bug 8215398" to the test: > > http://cr.openjdk.java.net/~hseigel/bug_8215398.3/webrev/index.html Okay. > > The point that I'm trying to make about logging's handling of single > quotes vs. double quotes can perhaps best be shown on Linux: > > If I execute the following commands (on Linux): > > java -Xlog:safepoint=trace:\"dquotes.log\" -version > > java -Xlog:safepoint=trace:\'squotes.log\' -version > > and then list out the current directory: > > > ls > 'squotes.log'? dquotes.log > > it shows that logging stripped off the double quotes, but not the single > quotes. I see - that is very odd behaviour. > With my fix, these commands succeed on Windows even when run from the > Windows shell: > > java -Xlog:safepoint=trace:d:\safepointtrace.txt -version > java -Xlog:safepoint=trace:"d:\safepointtrace.txt" -version > java -Xlog:safepoint=trace:"d:\safepoint trace.txt" -version > > This one continues to fail because of the single quote issue described > above: > > java -Xlog:safepoint=trace:'d:\safepointtrace.txt' -version > > I hope we can move forward with this change. Thanks, David > Thanks, Harold > > > On 12/20/2018 9:28 PM, David Holmes wrote: >> On 21/12/2018 7:30 am, Harold David Seigel wrote: >>> Hi David, >>> >>> Thanks for looking at this! >>> >>> Please review this updated webrev.? The fix is the same but the >>> webrev contains a new test instead of modifying an existing test: >>> >>> http://cr.openjdk.java.net/~hseigel/bug_8215398.2/webrev/ >> >> I think renaming the existing test would be better as the only >> difference is the use of quotes. But if you want a new test please add: >> >> @bug 8215398 >> >>> The logging implementation does not handle single quotes as >>> expected.? For example, if the TestQuotedLogOutputs.java test is >>> changed to use single quotes instead of double quotes, it will fail, >>> even on Linux, because the single quotes are included as part of the >>> file name.? They are not stripped off.? That is why "java >>> -Xlog:safepoint=trace:'d:\safepointtrace.txt' -version" fails. >> >> It fails with double-quotes too! AFAICS it always fails in a Windows >> command shell. Maybe the test only passes in cygwin shell? >> >> That said I don't understand your comment. I expect the quotes to >> delimit the filename in case the filename has spaces in the path >> component. If they aren't stripped off it wouldn't be tokenizing at >> the internal ":", so ... ??? >> >> Thanks, >> David >> >>> Perhaps a new bug is needed to change logging's handling of single >>> quotes?? It's a bit surprising that this issue hasn't already been >>> reported. >>> >>> Thanks, Harold >>> >>> On 12/20/2018 2:07 AM, David Holmes wrote: >>>> Hi Harold, >>>> >>>> On 19/12/2018 11:22 pm, Harold David Seigel wrote: >>>>> Hi, >>>>> >>>>> Please review this small change to fix JDK-8215398.? The fix works >>>>> by not treating ':' as a delimiter in a -Xlog... option string if >>>>> it is following by a '\' and preceded by either a single character >>>>> or the text 'file='.? The fix is for Windows only. >>>>> >>>>> Open Webrev: >>>>> http://cr.openjdk.java.net/~hseigel/bug_8215398/webrev/index.html >>>> >>>> I think I can follow the fix. >>>> >>>> But I'm a bit concerned about the test. AFAICS the test thought it >>>> was already testing this case - albeit with the path quoted: >>>> >>>> ? 42???????? // Ensure log files can be specified with full path. >>>> ? 43???????? // On windows, this means that the file name will contain >>>> ? 44???????? // a colon ('C:\log.txt' for example), which is used to >>>> ? 45???????? // separate -Xlog: options >>>> (-Xlog:tags:filename:decorators). >>>> ? 46???????? // Try to log to a file in our current directory, using >>>> its absolute path. >>>> ? 47???????? String baseName = "test file.log"; >>>> ? 48???????? Path filePath = Paths.get(baseName).toAbsolutePath(); >>>> ? 49???????? String fileName = filePath.toString(); >>>> ? 50???????? File file = filePath.toFile(); >>>> ... >>>> ? 65???????? String[] validOutputs = new String[] { >>>> ? 66???????????? quote + fileName + quote, >>>> ? 67???????????? "file=" + quote + fileName + quote, >>>> ? 68???????????? quote + fileName + quote + ":", >>>> ? 69???????????? quote + fileName + quote + "::" >>>> ? 70???????? }; >>>> >>>> But even quoted if I specify the drive designator in the path, it >>>> fails! Here's a local attempt at this: >>>> >>>> D:\ade> apps\Java\jdk-11\fastdebug\bin\java >>>> -Xlog:safepoint=trace:'d:\safepointtrace.txt' -version >>>> [0.014s][error][logging] Invalid decorator '\safepointtrace.txt''. >>>> Invalid -Xlog option >>>> '-Xlog:safepoint=trace:'d:\safepointtrace.txt'', see error log for >>>> details. >>>> Error: Could not create the Java Virtual Machine. >>>> Error: A fatal exception has occurred. Program will exit. >>>> >>>> So AFAICS the test can't possibly have been testing what it thought >>>> it was testing! ??? >>>> >>>> I'm also not sure about just adding some unquoted variants to the >>>> existing TestQuotedLogOutputs without renaming the test and updating >>>> the @summary >>>> >>>> Thanks, >>>> David >>>> >>>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8215398 >>>>> >>>>> The fix was regression tested by running Mach5 tiers 1 and 2 tests >>>>> and builds on Linux-x64, Windows, and Mac OS X, running tiers 3-5 >>>>> tests on Linux-x64, running JCK-12 Lang and VM tests on Linux-x64, >>>>> and by hand on Windows. >>>>> >>>>> Thanks, Harol >>>>> >>> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniil.x.titov at oracle.com Sat Dec 22 02:04:43 2018 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Fri, 21 Dec 2018 18:04:43 -0800 Subject: RFR 8205654: serviceability/dcmd/framework/HelpTest.java timed out In-Reply-To: <2a75f4de-6a77-615e-1648-0fcca59c6ea2@oracle.com> References: <8550b971-eede-c9ce-6eb9-4ec1671dbce3@oracle.com> <3C6EC1EF-672D-4577-9EEF-2CF73BDED256@oracle.com> <2a75f4de-6a77-615e-1648-0fcca59c6ea2@oracle.com> Message-ID: <0B661A5C-E59D-481C-BC3B-0951621F1F51@oracle.com> Hi David and Serguei, Please review a new version of the fix that for Linux platform uses the proc filesystem to retrieve the main class name for the running Java process. Webrev: http://cr.openjdk.java.net/~dtitov/8205654/webrev.02/ Bug: https://bugs.openjdk.java.net/browse/JDK-8205654 Thanks, Daniil ?On 11/29/18, 4:52 PM, "David Holmes" wrote: Hi Daniil, On 30/11/2018 7:30 am, Daniil Titov wrote: > Thank you, David! > > The proposed fix didn't help. It still hangs at some occasions. Additional tracing showed that when jcmd is invoked with the main class name it iterates over all running Java processes and temporary attaches to them to retrieve the main class name. It hangs while trying to attach to one of the running Java processes. There are numerous Java processes running at the host machine some associated with the test framework itself and another with the tests running in parallel. It is not clear what exact is this particular process since the jcmd hangs before retrieving the process' main class name, but after all tests terminated the process with this id is no longer running. I have to revoke this review since more investigation is required. That sounds like an unsolvable problem for the test. You can't control other Java processes on the machine, and searching by name requires asking each of them in turn. How do we get the list of Java processes in the first place? Perhaps we need to do some /proc//cmdline peeking? Cheers, David > > Best regards, > Daniil > > > > ?On 11/11/18, 1:35 PM, "David Holmes" wrote: > > Hi Daniil, > > I took a quick look at this one ... two minor comments > > The static class names could just be "Process" as they will acquire the > enclosing class name as part of their own name anyway. As it is this > gets repeated eg: > > HelpTest$HelpTestProcess > InvalidCommandTest$InvalidCommandTestProcess > > TestJavaProcess.java: > > 39 public static void main(String argv[]) { > > Nit: Should be "String[] argv" in Java style > > Thanks, > David > > On 10/11/2018 3:18 PM, Daniil Titov wrote: > > Please review the change that fixes serviceability/dcmd/framework/* tests from a time out. The fix for JDK-8166642 made serviceability/dcmd/framework/* tests non-concurrent to ensure that they don't interact with each other and there are no multiple tests running simultaneously since all they do share the common main class name com.sun.javatest.regtest.agent.MainWrapper. However, it looks like the tests from other directories still might run in parallel with these tests and they also have com.sun.javatest.regtest.agent.MainWrapper as a main class. > > > > The fix ensures that each serviceability/dcmd/framework/* test uses a Java process with a unique main class name when connecting to this process with jcmd and the main class name. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8205654 > > Webrev: http://cr.openjdk.java.net/~dtitov/8205654/webrev.001/ > > > > Best regards, > > Daniil > > > > > > > From david.holmes at oracle.com Sat Dec 22 03:47:03 2018 From: david.holmes at oracle.com (David Holmes) Date: Sat, 22 Dec 2018 13:47:03 +1000 Subject: RFR 8215398: -Xlog option usage => Invalid decorator'\temp\app_cds.log'. In-Reply-To: <5c1d77bd.1c69fb81.695ed.d412@mx.google.com> References: <71efe071-62c1-b1cc-c98f-953c6790eae1@oracle.com> <37e19362-0065-eb00-9bfb-7dcd67f55318@oracle.com> <3b0ff6e7-0fa6-d631-e203-c6070c1e66f0@oracle.com> <1a8d1c28-bc6c-da60-11bb-a57b48ee90b9@oracle.com> <2602b588-9d9e-253e-a875-046846141364@oracle.com> <5c1d77bd.1c69fb81.695ed.d412@mx.google.com> Message-ID: <9d1ca342-b9b2-32da-8eca-8504a79bd0b4@oracle.com> On 22/12/2018 9:31 am, Bernd Eckenfels wrote: > Hello, > > David: >> I don't have an issue with your fix. I just don't understand how the >> existing test actually works when the test cases seems to fail at the >> command-line for me. > > I think the difference between commandlie and testcase is, that the command line Interpreter might do some additional Interpretation of quotes and the process builder might transparently wrap Arguments in additional quotes. If I recall correctly on Windows the launcher does some quite Interpretation, not on *nix. Maybe ist best to compare the command line of the running process to be sure all intended cases are tested? > > (But for Windows ist not that easy to pass those quotes as theyare removed on muiltiple Levels ? I think tripple quotes might pass through), so supporting single quotes and also raw drive letter (even when seem uncanonical interpretation) seems like a good idea.) I've no idea what's happening. All I know is that syntax that is supposed to work (not altered by this bug) is not actually working. It will be interesting to see what happens after this patch is applied. Thanks, David > On Linux I would rather expect to generate filenames with single quotes if it is docmented to look for double quotes only. (I I pass a quoted single Quote to a command it will be part of the file name on most unix Tools) > > Gruss > Bernd > From merkel05 at gmail.com Sat Dec 22 16:09:52 2018 From: merkel05 at gmail.com (Sergei Ustimenko) Date: Sat, 22 Dec 2018 17:09:52 +0100 Subject: [PATCH] JDK-8025886: Replace [[ and == bash extensions in tests In-Reply-To: References: Message-ID: Hi, Could anyone please review and sponsor this small patch to add a shebang line to the test script? diff --git a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh --- a/test/jdk/sun/management/jmxremote/bootstrap/ GeneratePropertyPassword.sh +++ b/test/jdk/sun/management/jmxremote/bootstrap/ GeneratePropertyPassword.sh @@ -1,3 +1,5 @@ +#!/bin/bash + # # Copyright (c) 2003, 2018, Oracle and/or its affiliates. All rights reserved. # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. Thanks, Sergei On Wed, Dec 19, 2018, 21:04 Sergei Ustimenko Quickfix on my previous mail: I've found no scripts > with the same problem, except this one in the patch. > > >Yeah, let's fix ! > > Perfect! > > I will just need some help with sponsorship to push this. > I've signed the OCA, so no problems from my side. > > Regards, > Sergei > > On Wed, 19 Dec 2018 at 20:22, Martin Buchholz wrote: > >> Did we really have shell scripts without a shebang line? >> Yeah, let's fix ! >> >> On Wed, Dec 19, 2018 at 11:03 AM Sergei Ustimenko >> wrote: >> >>> HI Martin, >>> >>> As you've suggested I've simply added bash's shebang. >>> It wouldn't add any problem since, as David have mentioned, >>> no bash - no build. I've also quickly checked for similar cases >>> and found one. >>> >>> An updated patch is below. >>> >>> diff --git a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh >>> b/test/jdk/sun/management/jmxremote/bootstrap/ >>> GeneratePropertyPassword.sh >>> --- a/test/jdk/sun/management/jmxremote/bootstrap/ >>> GeneratePropertyPassword.sh >>> +++ b/test/jdk/sun/management/jmxremote/bootstrap/ >>> GeneratePropertyPassword.sh >>> @@ -1,3 +1,5 @@ >>> +#!/bin/bash >>> + >>> # >>> # Copyright (c) 2003, 2018, Oracle and/or its affiliates. All rights >>> reserved. >>> # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >>> >>> >>> Regards, >>> Sergei >>> >>> On Mon, 10 Dec 2018 at 22:27, Martin Buchholz >>> wrote: >>> >>>> I don't know if there's an official policy on how ultra-portable tests >>>> are supposed to be. In practice, you probably won't be able to build >>>> openjdk on a system without bash. >>>> >>>> On Mon, Dec 10, 2018 at 1:12 PM Sergei Ustimenko >>>> wrote: >>>> >>>>> Hi Martin, >>>>> >>>>> That sounds good! >>>>> >>>>> I've counted all the sh-shebangs and it appears that >>>>> there are at least 66 of them inside the test/ directory, >>>>> where only 12 bashes. >>>>> >>>>> I've also ran the search in order to identify all the >>>>> occurrences that use either [[ or == and found only >>>>> three of them that use "==". That one for example: >>>>> >>>>> http://hg.openjdk.java.net/jdk/sandbox/file/f94276ccc9fc/test/hotspot/jtreg/vmTestbase/jit/tiered/tieredTest.sh#l63 >>>>> of course `dash` reports failure in that case. >>>>> >>>>> So I'm quite hesitant in that case and not really sure >>>>> what to do. I haven't also found any existent JBS ticket >>>>> for such /bin/sh => /bin/bash a replacement. >>>>> >>>>> So any advise in this case would be appreciated! >>>>> >>>>> Regards, >>>>> Sergei >>>>> >>>>> On Mon, 10 Dec 2018 at 18:32, Martin Buchholz >>>>> wrote: >>>>> >>>>>> I would not try to remove all bash-isms from openjdk. Instead I >>>>>> would find instances of /bin/sh that need to be changed to /bin/bash. >>>>>> >>>>>> (Ubuntu's use of /bin/sh -> /bin/dash is technically correct, but >>>>>> caused much suffering >>>>>> https://bugs.launchpad.net/ubuntu/+source/dash/+bug/61463 >>>>>> ) >>>>>> >>>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From merkel05 at gmail.com Fri Dec 28 16:39:33 2018 From: merkel05 at gmail.com (Sergei Ustimenko) Date: Fri, 28 Dec 2018 17:39:33 +0100 Subject: [PATCH] JDK-8025886: Replace [[ and == bash extensions in tests In-Reply-To: References: Message-ID: Hi, Could anyone please help with review? Regards, Sergei On Sat, 22 Dec 2018 at 17:09, Sergei Ustimenko wrote: > Hi, > > Could anyone please review and sponsor > this small patch to add a shebang line to > the test script? > > diff --git a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > --- a/test/jdk/sun/management/jmxremote/bootstrap/ > GeneratePropertyPassword.sh > +++ b/test/jdk/sun/management/jmxremote/bootstrap/ > GeneratePropertyPassword.sh > @@ -1,3 +1,5 @@ > +#!/bin/bash > + > # > # Copyright (c) 2003, 2018, Oracle and/or its affiliates. All rights > reserved. > # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > > Thanks, > Sergei > > On Wed, Dec 19, 2018, 21:04 Sergei Ustimenko >> Quickfix on my previous mail: I've found no scripts >> with the same problem, except this one in the patch. >> >> >Yeah, let's fix ! >> >> Perfect! >> >> I will just need some help with sponsorship to push this. >> I've signed the OCA, so no problems from my side. >> >> Regards, >> Sergei >> >> On Wed, 19 Dec 2018 at 20:22, Martin Buchholz >> wrote: >> >>> Did we really have shell scripts without a shebang line? >>> Yeah, let's fix ! >>> >>> On Wed, Dec 19, 2018 at 11:03 AM Sergei Ustimenko >>> wrote: >>> >>>> HI Martin, >>>> >>>> As you've suggested I've simply added bash's shebang. >>>> It wouldn't add any problem since, as David have mentioned, >>>> no bash - no build. I've also quickly checked for similar cases >>>> and found one. >>>> >>>> An updated patch is below. >>>> >>>> diff --git a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh >>>> b/test/jdk/sun/management/jmxremote/bootstrap/ >>>> GeneratePropertyPassword.sh >>>> --- a/test/jdk/sun/management/jmxremote/bootstrap/ >>>> GeneratePropertyPassword.sh >>>> +++ b/test/jdk/sun/management/jmxremote/bootstrap/ >>>> GeneratePropertyPassword.sh >>>> @@ -1,3 +1,5 @@ >>>> +#!/bin/bash >>>> + >>>> # >>>> # Copyright (c) 2003, 2018, Oracle and/or its affiliates. All rights >>>> reserved. >>>> # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >>>> >>>> >>>> Regards, >>>> Sergei >>>> >>>> On Mon, 10 Dec 2018 at 22:27, Martin Buchholz >>>> wrote: >>>> >>>>> I don't know if there's an official policy on how ultra-portable tests >>>>> are supposed to be. In practice, you probably won't be able to build >>>>> openjdk on a system without bash. >>>>> >>>>> On Mon, Dec 10, 2018 at 1:12 PM Sergei Ustimenko >>>>> wrote: >>>>> >>>>>> Hi Martin, >>>>>> >>>>>> That sounds good! >>>>>> >>>>>> I've counted all the sh-shebangs and it appears that >>>>>> there are at least 66 of them inside the test/ directory, >>>>>> where only 12 bashes. >>>>>> >>>>>> I've also ran the search in order to identify all the >>>>>> occurrences that use either [[ or == and found only >>>>>> three of them that use "==". That one for example: >>>>>> >>>>>> http://hg.openjdk.java.net/jdk/sandbox/file/f94276ccc9fc/test/hotspot/jtreg/vmTestbase/jit/tiered/tieredTest.sh#l63 >>>>>> of course `dash` reports failure in that case. >>>>>> >>>>>> So I'm quite hesitant in that case and not really sure >>>>>> what to do. I haven't also found any existent JBS ticket >>>>>> for such /bin/sh => /bin/bash a replacement. >>>>>> >>>>>> So any advise in this case would be appreciated! >>>>>> >>>>>> Regards, >>>>>> Sergei >>>>>> >>>>>> On Mon, 10 Dec 2018 at 18:32, Martin Buchholz >>>>>> wrote: >>>>>> >>>>>>> I would not try to remove all bash-isms from openjdk. Instead I >>>>>>> would find instances of /bin/sh that need to be changed to /bin/bash >>>>>>> . >>>>>>> >>>>>>> (Ubuntu's use of /bin/sh -> /bin/dash is technically correct, but >>>>>>> caused much suffering >>>>>>> https://bugs.launchpad.net/ubuntu/+source/dash/+bug/61463 >>>>>>> ) >>>>>>> >>>>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From hohensee at amazon.com Fri Dec 28 18:05:38 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Fri, 28 Dec 2018 18:05:38 +0000 Subject: RFR(XS): 8215966: GeneratePropertyPassword.sh uses bash syntax (was Re: [PATCH] JDK-8025886: Replace [[ and == bash extensions in tests) Message-ID: <69B2A60F-4C0F-4FFD-BED1-2EAF8385F7A8@amazon.com> I?ll sponsor it. I?ve filed JDK-8215966 for it: it?s a simple forward port. Bug: https://bugs.openjdk.java.net/browse/JDK-8215966 Webrev: http://cr.openjdk.java.net/~phh/8215966/webrev.00/ While I agree with Martin that we might as well specify bash, imo in this specific case it?s better to do a forward port since the code already exists in JDK9. The webrev therefore dups the JDK-8025886 patch. If we want to make scripts explicitly depend on bash, I?d prefer to see a separate RFE for it. Lgtm, so that?s one review. May we have another please? Thanks, Paul From: serviceability-dev on behalf of Sergei Ustimenko Date: Friday, December 28, 2018 at 8:40 AM To: serviceability-dev Subject: Re: [PATCH] JDK-8025886: Replace [[ and == bash extensions in tests Hi, Could anyone please help with review? Regards, Sergei On Sat, 22 Dec 2018 at 17:09, Sergei Ustimenko > wrote: Hi, Could anyone please review and sponsor this small patch to add a shebang line to the test script? diff --git a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh --- a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh +++ b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh @@ -1,3 +1,5 @@ +#!/bin/bash + # # Copyright (c) 2003, 2018, Oracle and/or its affiliates. All rights reserved. # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. Thanks, Sergei On Wed, Dec 19, 2018, 21:04 Sergei Ustimenko wrote: Quickfix on my previous mail: I've found no scripts with the same problem, except this one in the patch. >Yeah, let's fix ! Perfect! I will just need some help with sponsorship to push this. I've signed the OCA, so no problems from my side. Regards, Sergei On Wed, 19 Dec 2018 at 20:22, Martin Buchholz > wrote: Did we really have shell scripts without a shebang line? Yeah, let's fix ! On Wed, Dec 19, 2018 at 11:03 AM Sergei Ustimenko > wrote: HI Martin, As you've suggested I've simply added bash's shebang. It wouldn't add any problem since, as David have mentioned, no bash - no build. I've also quickly checked for similar cases and found one. An updated patch is below. diff --git a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh --- a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh +++ b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh @@ -1,3 +1,5 @@ +#!/bin/bash + # # Copyright (c) 2003, 2018, Oracle and/or its affiliates. All rights reserved. # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. Regards, Sergei On Mon, 10 Dec 2018 at 22:27, Martin Buchholz > wrote: I don't know if there's an official policy on how ultra-portable tests are supposed to be. In practice, you probably won't be able to build openjdk on a system without bash. On Mon, Dec 10, 2018 at 1:12 PM Sergei Ustimenko > wrote: Hi Martin, That sounds good! I've counted all the sh-shebangs and it appears that there are at least 66 of them inside the test/ directory, where only 12 bashes. I've also ran the search in order to identify all the occurrences that use either [[ or == and found only three of them that use "==". That one for example: http://hg.openjdk.java.net/jdk/sandbox/file/f94276ccc9fc/test/hotspot/jtreg/vmTestbase/jit/tiered/tieredTest.sh#l63 of course `dash` reports failure in that case. So I'm quite hesitant in that case and not really sure what to do. I haven't also found any existent JBS ticket for such /bin/sh => /bin/bash a replacement. So any advise in this case would be appreciated! Regards, Sergei On Mon, 10 Dec 2018 at 18:32, Martin Buchholz > wrote: I would not try to remove all bash-isms from openjdk. Instead I would find instances of /bin/sh that need to be changed to /bin/bash. (Ubuntu's use of /bin/sh -> /bin/dash is technically correct, but caused much suffering https://bugs.launchpad.net/ubuntu/+source/dash/+bug/61463 ) -------------- next part -------------- An HTML attachment was scrubbed... URL: From hohensee at amazon.com Fri Dec 28 18:20:34 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Fri, 28 Dec 2018 18:20:34 +0000 Subject: RFR (XS) 8215495: Always set isCopy In-Reply-To: References: Message-ID: Lgtm. Thanks, Paul From: serviceability-dev on behalf of JC Beyler Date: Monday, December 17, 2018 at 9:43 AM To: "serviceability-dev at openjdk.java.net" Subject: RFR (XS) 8215495: Always set isCopy Hi all, Could I get a review for this webrev: Webrev: http://cr.openjdk.java.net/~jcbeyler/8215495/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8215495 Thanks, Jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From merkel05 at gmail.com Fri Dec 28 18:54:22 2018 From: merkel05 at gmail.com (Sergei Ustimenko) Date: Fri, 28 Dec 2018 19:54:22 +0100 Subject: RFR(XS): 8215966: GeneratePropertyPassword.sh uses bash syntax (was Re: [PATCH] JDK-8025886: Replace [[ and == bash extensions in tests) In-Reply-To: <69B2A60F-4C0F-4FFD-BED1-2EAF8385F7A8@amazon.com> References: <69B2A60F-4C0F-4FFD-BED1-2EAF8385F7A8@amazon.com> Message-ID: Hi Paul, Thanks for taking care of this and for your time! Agree, initial patch was mainly about forward porting lost changes as you've just mentioned. Regarding filling another RFE as I don't have a JBS account it would be a bit of a trouble for me I guess. I could try my best at https://bugreport.java.com/bugreport/ though I guess that's not the right place for it. So if I may borrow a bit more of your time, could you please help me with filling the RFE? I guess missing shebang is not critical, but might lead to a nasty situation. Thanks, Sergei On Fri, 28 Dec 2018 at 19:05, Hohensee, Paul wrote: > I?ll sponsor it. I?ve filed JDK-8215966 > for it: it?s a simple > forward port. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8215966 > > Webrev: http://cr.openjdk.java.net/~phh/8215966/webrev.00/ > > > > While I agree with Martin that we might as well specify bash, imo in this > specific case it?s better to do a forward port since the code already > exists in JDK9. The webrev therefore dups the JDK-8025886 > patch. If we want to > make scripts explicitly depend on bash, I?d prefer to see a separate RFE > for it. > > > > Lgtm, so that?s one review. May we have another please? > > > Thanks, > > > > Paul > > > > *From: *serviceability-dev > on behalf of Sergei Ustimenko > *Date: *Friday, December 28, 2018 at 8:40 AM > *To: *serviceability-dev > *Subject: *Re: [PATCH] JDK-8025886: Replace [[ and == bash extensions in > tests > > > > Hi, > > > > Could anyone please help with review? > > > > Regards, > > Sergei > > > > On Sat, 22 Dec 2018 at 17:09, Sergei Ustimenko wrote: > > Hi, > > > > Could anyone please review and sponsor > > this small patch to add a shebang line to > > the test script? > > > > diff --git > a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > --- > a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > +++ > b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > @@ -1,3 +1,5 @@ > +#!/bin/bash > + > # > # Copyright (c) 2003, 2018, Oracle and/or its affiliates. All rights > reserved. > # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > > > > Thanks, > > Sergei > > > > On Wed, Dec 19, 2018, 21:04 Sergei Ustimenko > Quickfix on my previous mail: I've found no scripts > > with the same problem, except this one in the patch. > > > > >Yeah, let's fix ! > > > > Perfect! > > > > I will just need some help with sponsorship to push this. > > I've signed the OCA, so no problems from my side. > > > > Regards, > > Sergei > > > > On Wed, 19 Dec 2018 at 20:22, Martin Buchholz wrote: > > Did we really have shell scripts without a shebang line? > > Yeah, let's fix ! > > > > On Wed, Dec 19, 2018 at 11:03 AM Sergei Ustimenko > wrote: > > HI Martin, > > > > As you've suggested I've simply added bash's shebang. > > It wouldn't add any problem since, as David have mentioned, > > no bash - no build. I've also quickly checked for similar cases > > and found one. > > > > An updated patch is below. > > > > diff --git > a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > --- > a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > +++ > b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > @@ -1,3 +1,5 @@ > +#!/bin/bash > + > # > # Copyright (c) 2003, 2018, Oracle and/or its affiliates. All rights > reserved. > # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > > > > > > Regards, > > Sergei > > > > On Mon, 10 Dec 2018 at 22:27, Martin Buchholz wrote: > > I don't know if there's an official policy on how ultra-portable tests are > supposed to be. In practice, you probably won't be able to build openjdk > on a system without bash. > > > > On Mon, Dec 10, 2018 at 1:12 PM Sergei Ustimenko > wrote: > > Hi Martin, > > > > That sounds good! > > > > I've counted all the sh-shebangs and it appears that > > there are at least 66 of them inside the test/ directory, > > where only 12 bashes. > > > > I've also ran the search in order to identify all the > > occurrences that use either [[ or == and found only > > three of them that use "==". That one for example: > > > http://hg.openjdk.java.net/jdk/sandbox/file/f94276ccc9fc/test/hotspot/jtreg/vmTestbase/jit/tiered/tieredTest.sh#l63 > > of course `dash` reports failure in that case. > > > > So I'm quite hesitant in that case and not really sure > > what to do. I haven't also found any existent JBS ticket > > for such /bin/sh => /bin/bash a replacement. > > > > So any advise in this case would be appreciated! > > > > Regards, > > Sergei > > > > On Mon, 10 Dec 2018 at 18:32, Martin Buchholz wrote: > > I would not try to remove all bash-isms from openjdk. Instead I would > find instances of /bin/sh that need to be changed to /bin/bash. > > > > (Ubuntu's use of /bin/sh -> /bin/dash is technically correct, but caused > much suffering > > https://bugs.launchpad.net/ubuntu/+source/dash/+bug/61463 > > ) > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hohensee at amazon.com Fri Dec 28 19:01:08 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Fri, 28 Dec 2018 19:01:08 +0000 Subject: RFR(XS): 8215966: GeneratePropertyPassword.sh uses bash syntax (was Re: [PATCH] JDK-8025886: Replace [[ and == bash extensions in tests) In-Reply-To: References: <69B2A60F-4C0F-4FFD-BED1-2EAF8385F7A8@amazon.com> Message-ID: <0BD9F34E-AC93-42CB-BCFA-856FC4EE81ED@amazon.com> Send a patch directly to me (hohensee at amazon.com) and I?ll file an RFE and post a webrev so you can ask for a review. Thanks, Paul From: Sergei Ustimenko Date: Friday, December 28, 2018 at 10:55 AM To: "Hohensee, Paul" Cc: serviceability-dev , Martin Buchholz Subject: Re: RFR(XS): 8215966: GeneratePropertyPassword.sh uses bash syntax (was Re: [PATCH] JDK-8025886: Replace [[ and == bash extensions in tests) Hi Paul, Thanks for taking care of this and for your time! Agree, initial patch was mainly about forward porting lost changes as you've just mentioned. Regarding filling another RFE as I don't have a JBS account it would be a bit of a trouble for me I guess. I could try my best at https://bugreport.java.com/bugreport/ though I guess that's not the right place for it. So if I may borrow a bit more of your time, could you please help me with filling the RFE? I guess missing shebang is not critical, but might lead to a nasty situation. Thanks, Sergei On Fri, 28 Dec 2018 at 19:05, Hohensee, Paul > wrote: I?ll sponsor it. I?ve filed JDK-8215966 for it: it?s a simple forward port. Bug: https://bugs.openjdk.java.net/browse/JDK-8215966 Webrev: http://cr.openjdk.java.net/~phh/8215966/webrev.00/ While I agree with Martin that we might as well specify bash, imo in this specific case it?s better to do a forward port since the code already exists in JDK9. The webrev therefore dups the JDK-8025886 patch. If we want to make scripts explicitly depend on bash, I?d prefer to see a separate RFE for it. Lgtm, so that?s one review. May we have another please? Thanks, Paul From: serviceability-dev > on behalf of Sergei Ustimenko > Date: Friday, December 28, 2018 at 8:40 AM To: serviceability-dev > Subject: Re: [PATCH] JDK-8025886: Replace [[ and == bash extensions in tests Hi, Could anyone please help with review? Regards, Sergei On Sat, 22 Dec 2018 at 17:09, Sergei Ustimenko > wrote: Hi, Could anyone please review and sponsor this small patch to add a shebang line to the test script? diff --git a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh --- a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh +++ b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh @@ -1,3 +1,5 @@ +#!/bin/bash + # # Copyright (c) 2003, 2018, Oracle and/or its affiliates. All rights reserved. # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. Thanks, Sergei On Wed, Dec 19, 2018, 21:04 Sergei Ustimenko wrote: Quickfix on my previous mail: I've found no scripts with the same problem, except this one in the patch. >Yeah, let's fix ! Perfect! I will just need some help with sponsorship to push this. I've signed the OCA, so no problems from my side. Regards, Sergei On Wed, 19 Dec 2018 at 20:22, Martin Buchholz > wrote: Did we really have shell scripts without a shebang line? Yeah, let's fix ! On Wed, Dec 19, 2018 at 11:03 AM Sergei Ustimenko > wrote: HI Martin, As you've suggested I've simply added bash's shebang. It wouldn't add any problem since, as David have mentioned, no bash - no build. I've also quickly checked for similar cases and found one. An updated patch is below. diff --git a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh --- a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh +++ b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh @@ -1,3 +1,5 @@ +#!/bin/bash + # # Copyright (c) 2003, 2018, Oracle and/or its affiliates. All rights reserved. # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. Regards, Sergei On Mon, 10 Dec 2018 at 22:27, Martin Buchholz > wrote: I don't know if there's an official policy on how ultra-portable tests are supposed to be. In practice, you probably won't be able to build openjdk on a system without bash. On Mon, Dec 10, 2018 at 1:12 PM Sergei Ustimenko > wrote: Hi Martin, That sounds good! I've counted all the sh-shebangs and it appears that there are at least 66 of them inside the test/ directory, where only 12 bashes. I've also ran the search in order to identify all the occurrences that use either [[ or == and found only three of them that use "==". That one for example: http://hg.openjdk.java.net/jdk/sandbox/file/f94276ccc9fc/test/hotspot/jtreg/vmTestbase/jit/tiered/tieredTest.sh#l63 of course `dash` reports failure in that case. So I'm quite hesitant in that case and not really sure what to do. I haven't also found any existent JBS ticket for such /bin/sh => /bin/bash a replacement. So any advise in this case would be appreciated! Regards, Sergei On Mon, 10 Dec 2018 at 18:32, Martin Buchholz > wrote: I would not try to remove all bash-isms from openjdk. Instead I would find instances of /bin/sh that need to be changed to /bin/bash. (Ubuntu's use of /bin/sh -> /bin/dash is technically correct, but caused much suffering https://bugs.launchpad.net/ubuntu/+source/dash/+bug/61463 ) -------------- next part -------------- An HTML attachment was scrubbed... URL: From hohensee at amazon.com Fri Dec 28 19:53:38 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Fri, 28 Dec 2018 19:53:38 +0000 Subject: [RFR]8215622: Add dump to file support for jmap histo In-Reply-To: <336c32cab24f42e388ccab4910c0d3b6@jd.com> References: <336c32cab24f42e388ccab4910c0d3b6@jd.com> Message-ID: Update the copyright dates to 2019 please, since this won?t get pushed until then. In JMap.java, I?d emulate dump() and use String filename = null; String subopts[] = options.split(","); for (String subopt : subopts){ if (subopt.isEmpty() || subopt.equals("all")) { // pass } else if (subopt.equals("live")) { liveopt = "-live"; } else if (subopt.startsWith("file=")) { // file= - check that is specified if (subopt.length() > 5) { filename = subopt.substring(5); } } else { usage(1); } } // get the canonical path - important to avoid just passing // a "heap.bin" and having the dump created in the target VM // working directory rather than the directory where jmap // is executed. filename = new File(filename).getCanonicalPath(); // inspectHeap is not the same as jcmd GC.class_histogram executeCommandForPid(pid, "inspectheap", filename, liveopt); I.e., use an enhanced for loop to scan the array, and duplicate dump()?s executeCommandForPid() argument order, as well as dump()?s ?file=<>? check (the code that starts with ?if (subopt.startsWith?) and canonicalization. Actually, better to factor the latter out into its own method and use it from both histo() and dump(). The argument checking code in heap_inspection() in attachListener.cpp can be simplified along the lines of dump_heap(). I.e., you don?t need to loop over the argument list. To match up with dump_heap()?s info messages, the info message string at the end should be ?Heap inspection file created: %s?. Thanks, Paul From: serviceability-dev on behalf of ?? Date: Thursday, December 20, 2018 at 11:03 PM To: "serviceability-dev at openjdk.java.net" Subject: [RFR]8215622: Add dump to file support for jmap histo Hi All, May I ask your help to review this patch for enhance jmap ?histo. It add the ?file=? arguments that allow jmap ?histo outputs data to file directly. This patch is also part of the enhancement described in https://bugs.openjdk.java.net/browse/JDK-8214535. Webrev: http://cr.openjdk.java.net/~xiaofeya/8215622/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8215622 Thanks. BRs, Lin -------------- next part -------------- An HTML attachment was scrubbed... URL: From merkel05 at gmail.com Fri Dec 28 20:14:53 2018 From: merkel05 at gmail.com (Sergei Ustimenko) Date: Fri, 28 Dec 2018 21:14:53 +0100 Subject: RFR(XS): 8215966: GeneratePropertyPassword.sh uses bash syntax (was Re: [PATCH] JDK-8025886: Replace [[ and == bash extensions in tests) In-Reply-To: <0BD9F34E-AC93-42CB-BCFA-856FC4EE81ED@amazon.com> References: <69B2A60F-4C0F-4FFD-BED1-2EAF8385F7A8@amazon.com> <0BD9F34E-AC93-42CB-BCFA-856FC4EE81ED@amazon.com> Message-ID: Hi, I've checked if there are scripts (*.sh) that miss shebang and found some (~170 in test/). As an example those: ... test/jdk/java/lang/instrument/StressGetObjectSizeTest.sh test/jdk/java/lang/instrument/RedefineClassWithNativeMethod.sh test/jdk/java/lang/instrument/RedefineMethodWithAnnotations.sh test/jdk/java/io/File/GetXSpace.sh test/jdk/java/rmi/activation/Activatable/extLoadedImpl/ext.sh test/jdk/java/util/Formatter/Basic.sh test/jdk/java/util/zip/3GBZipFiles.sh test/jdk/tools/launcher/6842838/Test6842838.sh test/jdk/tools/jjs/common.sh ... test/jdk/sun/rmi/rmic/covariantReturns/run.sh test/jdk/sun/rmi/rmic/manifestClassPath/Util.sh test/jdk/sun/rmi/rmic/oldjavacRemoved/sunToolsJavacMain.sh test/jdk/sun/management/jmxremote/RunTest.sh test/jdk/sun/management/jmxremote/bootstrap/RmiBootstrapTest.sh test/jdk/sun/management/jmxremote/bootstrap/RmiSslBootstrapTest.sh test/jdk/sun/management/jmxremote/bootstrap/RmiSslNoKeyStoreTest.sh test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh test/jdk/sun/net/sdp/sanity.sh test/jdk/sun/jvmstat/testlibrary/utils.sh ... Some of them are tests others are utility scripts. I wonder what we can do in that case? Can we fill an RFE? Relying on default here might lead to a nasty situations in future, as it was with Ubuntu sh -> dash. As a small remark all the scripts that miss shebang are located in the test/ dir. Regards, Sergei On Fri, 28 Dec 2018 at 20:01, Hohensee, Paul wrote: > Send a patch directly to me (hohensee at amazon.com) and I?ll file an RFE > and post a webrev so you can ask for a review. > > > > Thanks, > > > > Paul > > > > *From: *Sergei Ustimenko > *Date: *Friday, December 28, 2018 at 10:55 AM > *To: *"Hohensee, Paul" > *Cc: *serviceability-dev , Martin > Buchholz > *Subject: *Re: RFR(XS): 8215966: GeneratePropertyPassword.sh uses bash > syntax (was Re: [PATCH] JDK-8025886: Replace [[ and == bash extensions in > tests) > > > > Hi Paul, > > > > Thanks for taking care of this and for your time! > > > > Agree, initial patch was mainly about forward porting > > lost changes as you've just mentioned. > > > > Regarding filling another RFE as I don't have a JBS account it > > would be a bit of a trouble for me I guess. I could try my best at > > https://bugreport.java.com/bugreport/ though I guess that's > > not the right place for it. So if I may borrow a bit more of your > > time, could you please help me with filling the RFE? I guess missing > > shebang is not critical, but might lead to a nasty situation. > > > > Thanks, > > Sergei > > > > On Fri, 28 Dec 2018 at 19:05, Hohensee, Paul wrote: > > I?ll sponsor it. I?ve filed JDK-8215966 > for it: it?s a simple > forward port. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8215966 > > Webrev: http://cr.openjdk.java.net/~phh/8215966/webrev.00/ > > > > While I agree with Martin that we might as well specify bash, imo in this > specific case it?s better to do a forward port since the code already > exists in JDK9. The webrev therefore dups the JDK-8025886 > patch. If we want to > make scripts explicitly depend on bash, I?d prefer to see a separate RFE > for it. > > > > Lgtm, so that?s one review. May we have another please? > > > Thanks, > > > > Paul > > > > *From: *serviceability-dev > on behalf of Sergei Ustimenko > *Date: *Friday, December 28, 2018 at 8:40 AM > *To: *serviceability-dev > *Subject: *Re: [PATCH] JDK-8025886: Replace [[ and == bash extensions in > tests > > > > Hi, > > > > Could anyone please help with review? > > > > Regards, > > Sergei > > > > On Sat, 22 Dec 2018 at 17:09, Sergei Ustimenko wrote: > > Hi, > > > > Could anyone please review and sponsor > > this small patch to add a shebang line to > > the test script? > > > > diff --git > a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > --- > a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > +++ > b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > @@ -1,3 +1,5 @@ > +#!/bin/bash > + > # > # Copyright (c) 2003, 2018, Oracle and/or its affiliates. All rights > reserved. > # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > > > > Thanks, > > Sergei > > > > On Wed, Dec 19, 2018, 21:04 Sergei Ustimenko > Quickfix on my previous mail: I've found no scripts > > with the same problem, except this one in the patch. > > > > >Yeah, let's fix ! > > > > Perfect! > > > > I will just need some help with sponsorship to push this. > > I've signed the OCA, so no problems from my side. > > > > Regards, > > Sergei > > > > On Wed, 19 Dec 2018 at 20:22, Martin Buchholz wrote: > > Did we really have shell scripts without a shebang line? > > Yeah, let's fix ! > > > > On Wed, Dec 19, 2018 at 11:03 AM Sergei Ustimenko > wrote: > > HI Martin, > > > > As you've suggested I've simply added bash's shebang. > > It wouldn't add any problem since, as David have mentioned, > > no bash - no build. I've also quickly checked for similar cases > > and found one. > > > > An updated patch is below. > > > > diff --git > a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > --- > a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > +++ > b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > @@ -1,3 +1,5 @@ > +#!/bin/bash > + > # > # Copyright (c) 2003, 2018, Oracle and/or its affiliates. All rights > reserved. > # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > > > > > > Regards, > > Sergei > > > > On Mon, 10 Dec 2018 at 22:27, Martin Buchholz wrote: > > I don't know if there's an official policy on how ultra-portable tests are > supposed to be. In practice, you probably won't be able to build openjdk > on a system without bash. > > > > On Mon, Dec 10, 2018 at 1:12 PM Sergei Ustimenko > wrote: > > Hi Martin, > > > > That sounds good! > > > > I've counted all the sh-shebangs and it appears that > > there are at least 66 of them inside the test/ directory, > > where only 12 bashes. > > > > I've also ran the search in order to identify all the > > occurrences that use either [[ or == and found only > > three of them that use "==". That one for example: > > > http://hg.openjdk.java.net/jdk/sandbox/file/f94276ccc9fc/test/hotspot/jtreg/vmTestbase/jit/tiered/tieredTest.sh#l63 > > of course `dash` reports failure in that case. > > > > So I'm quite hesitant in that case and not really sure > > what to do. I haven't also found any existent JBS ticket > > for such /bin/sh => /bin/bash a replacement. > > > > So any advise in this case would be appreciated! > > > > Regards, > > Sergei > > > > On Mon, 10 Dec 2018 at 18:32, Martin Buchholz wrote: > > I would not try to remove all bash-isms from openjdk. Instead I would > find instances of /bin/sh that need to be changed to /bin/bash. > > > > (Ubuntu's use of /bin/sh -> /bin/dash is technically correct, but caused > much suffering > > https://bugs.launchpad.net/ubuntu/+source/dash/+bug/61463 > > ) > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Fri Dec 28 21:43:25 2018 From: david.holmes at oracle.com (David Holmes) Date: Sat, 29 Dec 2018 07:43:25 +1000 Subject: RFR(XS): 8215966: GeneratePropertyPassword.sh uses bash syntax (was Re: [PATCH] JDK-8025886: Replace [[ and == bash extensions in tests) In-Reply-To: <69B2A60F-4C0F-4FFD-BED1-2EAF8385F7A8@amazon.com> References: <69B2A60F-4C0F-4FFD-BED1-2EAF8385F7A8@amazon.com> Message-ID: Hi Paul, On 29/12/2018 4:05 am, Hohensee, Paul wrote: > I?ll sponsor it. I?ve filed JDK-8215966 > for it: it?s a simple > forward port. To be clear, it isn't that this change didn't make it into post 9, the change was reverted by JDK-8192953 in June 2018. > Bug: https://bugs.openjdk.java.net/browse/JDK-8215966 > > Webrev: http://cr.openjdk.java.net/~phh/8215966/webrev.00/ > > While I agree with Martin that we might as well specify bash, imo in > this specific case it?s better to do a forward port since the code > already exists in JDK9. The webrev therefore dups the JDK-8025886 > patch. If we want to > make scripts explicitly depend on bash, I?d prefer to see a separate RFE > for it. The patch looks good to me. Reviewed. > Lgtm, so that?s one review. May we have another please? Point of order: the change proposed is not the change that Sergei requested a review of. This is new code proposed by you, so you can't review your own contribution. But I've Reviewed it, and Sergei is okay with it too so that's a "r"eview and so it is good to go. Thanks, David > > Thanks, > > Paul > > *From: *serviceability-dev > on behalf of Sergei Ustimenko > *Date: *Friday, December 28, 2018 at 8:40 AM > *To: *serviceability-dev > *Subject: *Re: [PATCH] JDK-8025886: Replace [[ and == bash extensions in > tests > > Hi, > > Could anyone please help with review? > > Regards, > > Sergei > > On Sat, 22 Dec 2018 at 17:09, Sergei Ustimenko > wrote: > > Hi, > > Could anyone please review and sponsor > > this small patch to add a shebang line to > > the test script? > > diff --git > a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > --- > a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > +++ > b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > @@ -1,3 +1,5 @@ > +#!/bin/bash > + > ?# > ?# Copyright (c) 2003, 2018, Oracle and/or its affiliates. All > rights reserved. > ?# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > > Thanks, > > Sergei > > On Wed, Dec 19, 2018, 21:04 Sergei Ustimenko wrote: > > Quickfix on my previous mail: I've found no scripts > > with the same problem, except this one in the patch. > > >Yeah, let's fix ! > > Perfect! > > I will just need some help with sponsorship to push this. > > I've signed the OCA, so no problems from my side. > > Regards, > > Sergei > > On Wed, 19 Dec 2018 at 20:22, Martin Buchholz > > wrote: > > Did we really have shell scripts without a shebang line? > > Yeah, let's fix ! > > On Wed, Dec 19, 2018 at 11:03 AM Sergei Ustimenko > > wrote: > > HI Martin, > > As you've suggested I've simply added bash's shebang. > > It wouldn't add any problem since, as David have mentioned, > > no bash - no build. I've also quickly checked for > similar cases > > and found one. > > An updated patch is below. > > diff --git > a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > --- > a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > +++ > b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > @@ -1,3 +1,5 @@ > +#!/bin/bash > + > ?# > ?# Copyright (c) 2003, 2018, Oracle and/or its > affiliates. All rights reserved. > ?# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS > FILE HEADER. > > Regards, > > Sergei > > On Mon, 10 Dec 2018 at 22:27, Martin Buchholz > > wrote: > > I don't know if there's an official policy on how > ultra-portable tests are supposed?to be.? In > practice, you probably won't be able to build > openjdk on a system without bash. > > On Mon, Dec 10, 2018 at 1:12 PM Sergei Ustimenko > > wrote: > > Hi Martin, > > That sounds good! > > I've counted all the sh-shebangs and it appears that > > there are at least 66 of them inside the test/ > directory, > > where only 12 bashes. > > I've also ran the search? in order to identify > all the > > occurrences that use either [[ or == and found only > > three of them that use "==". That one for example: > > http://hg.openjdk.java.net/jdk/sandbox/file/f94276ccc9fc/test/hotspot/jtreg/vmTestbase/jit/tiered/tieredTest.sh#l63 > > of course `dash` reports failure in that case. > > So I'm quite hesitant in that case and not > really sure > > what to do. I haven't also found any existent > JBS ticket > > for such /bin/sh => /bin/bash a replacement. > > So any advise in this case?would be appreciated! > > Regards, > > Sergei > > On Mon, 10 Dec 2018 at 18:32, Martin Buchholz > > wrote: > > I would not try to remove all bash-isms from > openjdk. Instead I would find instances of > /bin/sh that need to be changed to /bin/bash. > > (Ubuntu's use of /bin/sh -> /bin/dash is > technically correct, but caused much suffering > > https://bugs.launchpad.net/ubuntu/+source/dash/+bug/61463 > > ) > From hohensee at amazon.com Fri Dec 28 23:20:42 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Fri, 28 Dec 2018 23:20:42 +0000 Subject: RFR(XS): 8215966: GeneratePropertyPassword.sh uses bash syntax (was Re: [PATCH] JDK-8025886: Replace [[ and == bash extensions in tests) In-Reply-To: References: <69B2A60F-4C0F-4FFD-BED1-2EAF8385F7A8@amazon.com> Message-ID: <186DA688-7AAA-49C9-9CAF-BE82593BBAA7@amazon.com> The change is Sergei's original proposal (not included in this message), but I'll take what I can get. :) Pushed. Thanks, Paul ?On 12/28/18, 1:44 PM, "David Holmes" wrote: Hi Paul, On 29/12/2018 4:05 am, Hohensee, Paul wrote: > I?ll sponsor it. I?ve filed JDK-8215966 > for it: it?s a simple > forward port. To be clear, it isn't that this change didn't make it into post 9, the change was reverted by JDK-8192953 in June 2018. > Bug: https://bugs.openjdk.java.net/browse/JDK-8215966 > > Webrev: http://cr.openjdk.java.net/~phh/8215966/webrev.00/ > > While I agree with Martin that we might as well specify bash, imo in > this specific case it?s better to do a forward port since the code > already exists in JDK9. The webrev therefore dups the JDK-8025886 > patch. If we want to > make scripts explicitly depend on bash, I?d prefer to see a separate RFE > for it. The patch looks good to me. Reviewed. > Lgtm, so that?s one review. May we have another please? Point of order: the change proposed is not the change that Sergei requested a review of. This is new code proposed by you, so you can't review your own contribution. But I've Reviewed it, and Sergei is okay with it too so that's a "r"eview and so it is good to go. Thanks, David > > Thanks, > > Paul > > *From: *serviceability-dev > on behalf of Sergei Ustimenko > *Date: *Friday, December 28, 2018 at 8:40 AM > *To: *serviceability-dev > *Subject: *Re: [PATCH] JDK-8025886: Replace [[ and == bash extensions in > tests > > Hi, > > Could anyone please help with review? > > Regards, > > Sergei > > On Sat, 22 Dec 2018 at 17:09, Sergei Ustimenko > wrote: > > Hi, > > Could anyone please review and sponsor > > this small patch to add a shebang line to > > the test script? > > diff --git > a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > --- > a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > +++ > b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > @@ -1,3 +1,5 @@ > +#!/bin/bash > + > # > # Copyright (c) 2003, 2018, Oracle and/or its affiliates. All > rights reserved. > # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > > Thanks, > > Sergei > > On Wed, Dec 19, 2018, 21:04 Sergei Ustimenko wrote: > > Quickfix on my previous mail: I've found no scripts > > with the same problem, except this one in the patch. > > >Yeah, let's fix ! > > Perfect! > > I will just need some help with sponsorship to push this. > > I've signed the OCA, so no problems from my side. > > Regards, > > Sergei > > On Wed, 19 Dec 2018 at 20:22, Martin Buchholz > > wrote: > > Did we really have shell scripts without a shebang line? > > Yeah, let's fix ! > > On Wed, Dec 19, 2018 at 11:03 AM Sergei Ustimenko > > wrote: > > HI Martin, > > As you've suggested I've simply added bash's shebang. > > It wouldn't add any problem since, as David have mentioned, > > no bash - no build. I've also quickly checked for > similar cases > > and found one. > > An updated patch is below. > > diff --git > a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > --- > a/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > +++ > b/test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh > @@ -1,3 +1,5 @@ > +#!/bin/bash > + > # > # Copyright (c) 2003, 2018, Oracle and/or its > affiliates. All rights reserved. > # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS > FILE HEADER. > > Regards, > > Sergei > > On Mon, 10 Dec 2018 at 22:27, Martin Buchholz > > wrote: > > I don't know if there's an official policy on how > ultra-portable tests are supposed to be. In > practice, you probably won't be able to build > openjdk on a system without bash. > > On Mon, Dec 10, 2018 at 1:12 PM Sergei Ustimenko > > wrote: > > Hi Martin, > > That sounds good! > > I've counted all the sh-shebangs and it appears that > > there are at least 66 of them inside the test/ > directory, > > where only 12 bashes. > > I've also ran the search in order to identify > all the > > occurrences that use either [[ or == and found only > > three of them that use "==". That one for example: > > http://hg.openjdk.java.net/jdk/sandbox/file/f94276ccc9fc/test/hotspot/jtreg/vmTestbase/jit/tiered/tieredTest.sh#l63 > > of course `dash` reports failure in that case. > > So I'm quite hesitant in that case and not > really sure > > what to do. I haven't also found any existent > JBS ticket > > for such /bin/sh => /bin/bash a replacement. > > So any advise in this case would be appreciated! > > Regards, > > Sergei > > On Mon, 10 Dec 2018 at 18:32, Martin Buchholz > > wrote: > > I would not try to remove all bash-isms from > openjdk. Instead I would find instances of > /bin/sh that need to be changed to /bin/bash. > > (Ubuntu's use of /bin/sh -> /bin/dash is > technically correct, but caused much suffering > > https://bugs.launchpad.net/ubuntu/+source/dash/+bug/61463 > > ) > From goetz.lindenmaier at sap.com Mon Dec 31 13:48:44 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 31 Dec 2018 13:48:44 +0000 Subject: RFR(S): 8215975: [testbug] Adapt nsk tests to the PPC, S390 and AIX platforms. Message-ID: Hi, Some of the nsk tests are not properly configured for the above platforms: nsk/jvmti/RetransformClasses/retransform003/TestDriver.java: nsk/jvmti/SetNativeMethodPrefix/SetNativeMethodPrefix002/TestDriver.java The tests use the path to native libraries, which is named "LIBPATH" on AIX. nsk/share/jdi/ArgumentHandler.java Add ppc/s390/aix which don't have a shared memory connector. This disables a row of failing tests. Please review this change. I would like to push it to jdk12, as it is a mere test fix. http://cr.openjdk.java.net/~goetz/wr18/8215975-nskTests/01/index.html Best regards, Goetz. From gary.adams at oracle.com Mon Dec 31 14:16:42 2018 From: gary.adams at oracle.com (gary.adams at oracle.com) Date: Mon, 31 Dec 2018 09:16:42 -0500 Subject: RFR(S): 8215975: [testbug] Adapt nsk tests to the PPC, S390 and AIX platforms. In-Reply-To: References: Message-ID: <101ecbcd-76c9-2311-04e4-871ca4f3764a@oracle.com> Would it make sense to add a method to test/lib/jdk/test/lib/Platform.java similar to sharedLibraryExt() to cover the envName setting? Are the other places DYLD_LIBRARY_PATH is set also need to be updated for AIX? On 12/31/18 8:48 AM, Lindenmaier, Goetz wrote: > Hi, > > Some of the nsk tests are not properly configured for the > above platforms: > > nsk/jvmti/RetransformClasses/retransform003/TestDriver.java: > nsk/jvmti/SetNativeMethodPrefix/SetNativeMethodPrefix002/TestDriver.java > The tests use the path to native libraries, which is named "LIBPATH" on AIX. > > nsk/share/jdi/ArgumentHandler.java > Add ppc/s390/aix which don't have a shared memory connector. This disables a row of failing tests. > > Please review this change. I would like to push it to jdk12, as it is a mere test fix. > http://cr.openjdk.java.net/~goetz/wr18/8215975-nskTests/01/index.html > > Best regards, > Goetz. From goetz.lindenmaier at sap.com Mon Dec 31 15:19:03 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 31 Dec 2018 15:19:03 +0000 Subject: RFR(S): 8215975: [testbug] Adapt nsk tests to the PPC, S390 and AIX platforms. In-Reply-To: <101ecbcd-76c9-2311-04e4-871ca4f3764a@oracle.com> References: <101ecbcd-76c9-2311-04e4-871ca4f3764a@oracle.com> Message-ID: <0176fe1fda7c495582435d2b2a48dbb6@sap.com> Hi Gary, > Would it make sense to add a method to test/lib/jdk/test/lib/Platform.java > similar to sharedLibraryExt() to cover the envName setting? Good point. I'll post a follow up change next year ... and believe me, that's not too far in the future ?? I'd like to keep this out of this test fix, as I want to downport it to 11 at some point and thus keep it small so that it will apply cleanly. > Are the other places DYLD_LIBRARY_PATH is set also need to be > updated for AIX? Alloc001 might need it. The other vmTestbase tests are all passing, so I didn't touch them. Webrev with alloc001: http://cr.openjdk.java.net/~goetz/wr18/8215975-nskTests/02/ Best regards, Goetz. > > > On 12/31/18 8:48 AM, Lindenmaier, Goetz wrote: > > Hi, > > > > Some of the nsk tests are not properly configured for the > > above platforms: > > > > nsk/jvmti/RetransformClasses/retransform003/TestDriver.java: > > > nsk/jvmti/SetNativeMethodPrefix/SetNativeMethodPrefix002/TestDriver.ja > va > > The tests use the path to native libraries, which is named "LIBPATH" on > AIX. > > > > nsk/share/jdi/ArgumentHandler.java > > Add ppc/s390/aix which don't have a shared memory connector. This > disables a row of failing tests. > > > > Please review this change. I would like to push it to jdk12, as it is a mere > test fix. > > http://cr.openjdk.java.net/~goetz/wr18/8215975-nskTests/01/index.html > > > > Best regards, > > Goetz. > From gary.adams at oracle.com Mon Dec 31 15:59:25 2018 From: gary.adams at oracle.com (gary.adams at oracle.com) Date: Mon, 31 Dec 2018 10:59:25 -0500 Subject: RFR(S): 8215975: [testbug] Adapt nsk tests to the PPC, S390 and AIX platforms. In-Reply-To: <0176fe1fda7c495582435d2b2a48dbb6@sap.com> References: <101ecbcd-76c9-2311-04e4-871ca4f3764a@oracle.com> <0176fe1fda7c495582435d2b2a48dbb6@sap.com> Message-ID: <750630a7-63e8-495a-7c9b-54ff50af3c40@oracle.com> Here are few more DYLD_LIBRARY_PATH locations that would be worth checking ./jdk/vm/JniInvocationTest.java:50: env.compute("DYLD_LIBRARY_PATH", (k, v) -> (k == null) ? libdir : v + ":" + libdir); ./jdk/tools/launcher/ExecutionEnvironment.java:66:??????????? ? "DYLD_LIBRARY_PATH" ./jdk/tools/launcher/JliLaunchTest.java:56:??????????? String LD_LIBRARY_PATH = Platform.isOSX() ? "DYLD_LIBRARY_PATH" : "LD_LIBRARY_PATH"; ./jdk/sun/security/krb5/auto/BasicProc.java:311: p.env("DYLD_LIBRARY_PATH", lib.substring(0, pos)); ./jdk/sun/security/krb5/auto/ReplayCacheTestProc.java:402: .env("DYLD_LIBRARY_PATH", libDir) ./jdk/sun/security/krb5/auto/KDC.java:1725: "DYLD_LIBRARY_PATH", nativePath + "/lib", ./jdk/sun/security/krb5/auto/KDC.java:1818: "DYLD_LIBRARY_PATH", nativePath + "/lib", ./hotspot/jtreg/vmTestbase/ExecDriver.java:122: name = Platform.isOSX() ? "DYLD_LIBRARY_PATH" : "LD_LIBRARY_PATH"; ./hotspot/jtreg/runtime/signal/SigTestDriver.java:72: (Platform.isOSX() ? "DYLD_LIBRARY_PATH" : "LD_LIBRARY_PATH"); On 12/31/18 10:19 AM, Lindenmaier, Goetz wrote: > Hi Gary, > >> Would it make sense to add a method to test/lib/jdk/test/lib/Platform.java >> similar to sharedLibraryExt() to cover the envName setting? > Good point. I'll post a follow up change next year ... and believe me, > that's not too far in the future ?? > I'd like to keep this out of this test fix, as I want to downport it to > 11 at some point and thus keep it small so that it will apply cleanly. > >> Are the other places DYLD_LIBRARY_PATH is set also need to be >> updated for AIX? > Alloc001 might need it. The other vmTestbase tests are all passing, > so I didn't touch them. > Webrev with alloc001: > http://cr.openjdk.java.net/~goetz/wr18/8215975-nskTests/02/ > > Best regards, > Goetz. > > > >> >> On 12/31/18 8:48 AM, Lindenmaier, Goetz wrote: >>> Hi, >>> >>> Some of the nsk tests are not properly configured for the >>> above platforms: >>> >>> nsk/jvmti/RetransformClasses/retransform003/TestDriver.java: >>> >> nsk/jvmti/SetNativeMethodPrefix/SetNativeMethodPrefix002/TestDriver.ja >> va >>> The tests use the path to native libraries, which is named "LIBPATH" on >> AIX. >>> nsk/share/jdi/ArgumentHandler.java >>> Add ppc/s390/aix which don't have a shared memory connector. This >> disables a row of failing tests. >>> Please review this change. I would like to push it to jdk12, as it is a mere >> test fix. >>> http://cr.openjdk.java.net/~goetz/wr18/8215975-nskTests/01/index.html >>> >>> Best regards, >>> Goetz. From gary.adams at oracle.com Mon Dec 31 18:06:02 2018 From: gary.adams at oracle.com (gary.adams at oracle.com) Date: Mon, 31 Dec 2018 13:06:02 -0500 Subject: RFR: JDK-8211343: nsk_jvmti_parseoptions should handle multiple suboptions In-Reply-To: <5C1D3662.4070505@oracle.com> References: <5C1D3662.4070505@oracle.com> Message-ID: Here's a revised webrev. ? Webrev: http://bussund0416.us.oracle.com/export/users/gradams/work/webrevs/8211343/webrev.01/ Updates in this round of changes : ? - replaced index() with strchr() to avoid platform dependent issues with strings.h include ? - removed NSK_JVMTI_OPTION_VAL_SEP ? - removed temporary debugging print statements ? - removed empty options string from SetNativeMethodPrefix001 ? - added free for temporary strdup buffer ? - updated copyright for 2019 On 12/21/18 1:52 PM, Gary Adams wrote: > Here is a first pass at a replacement parser for jvmti test options. > > ? Webrev: http://cr.openjdk.java.net/~gadams/8211343/webrev.00/ > ? Issue: https://bugs.openjdk.java.net/browse/JDK-8211343 > > Testing is in progress. Need to check out more platforms. > On local linux testing one jvmti test failed SetNativeMethodPrefix001 > which passed an empty options string. > ... From chris.plummer at oracle.com Mon Dec 31 19:25:30 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 31 Dec 2018 11:25:30 -0800 Subject: RFR: JDK-8211343: nsk_jvmti_parseoptions should handle multiple suboptions In-Reply-To: References: <5C1D3662.4070505@oracle.com> Message-ID: Looks good. Are empty options strings still allowed after your changes? Chris On 12/31/18 10:06 AM, gary.adams at oracle.com wrote: > Here's a revised webrev. > > ? Webrev: > http://bussund0416.us.oracle.com/export/users/gradams/work/webrevs/8211343/webrev.01/ > > Updates in this round of changes : > ? - replaced index() with strchr() to avoid platform dependent issues > with strings.h include > ? - removed NSK_JVMTI_OPTION_VAL_SEP > ? - removed temporary debugging print statements > ? - removed empty options string from SetNativeMethodPrefix001 > ? - added free for temporary strdup buffer > ? - updated copyright for 2019 > > On 12/21/18 1:52 PM, Gary Adams wrote: >> Here is a first pass at a replacement parser for jvmti test options. >> >> ? Webrev: http://cr.openjdk.java.net/~gadams/8211343/webrev.00/ >> ? Issue: https://bugs.openjdk.java.net/browse/JDK-8211343 >> >> Testing is in progress. Need to check out more platforms. >> On local linux testing one jvmti test failed SetNativeMethodPrefix001 >> which passed an empty options string. >> ... > > From gary.adams at oracle.com Mon Dec 31 20:30:43 2018 From: gary.adams at oracle.com (gary.adams at oracle.com) Date: Mon, 31 Dec 2018 15:30:43 -0500 Subject: RFR: JDK-8211343: nsk_jvmti_parseoptions should handle multiple suboptions In-Reply-To: References: <5C1D3662.4070505@oracle.com> Message-ID: The one test that failed is changed in this webrev. When it failed, it was the check in add_option that does not allow an "empty" option name. On 12/31/18 2:25 PM, Chris Plummer wrote: > Looks good. Are empty options strings still allowed after your changes? > > Chris > > On 12/31/18 10:06 AM, gary.adams at oracle.com wrote: >> Here's a revised webrev. >> >> ? Webrev: >> http://bussund0416.us.oracle.com/export/users/gradams/work/webrevs/8211343/webrev.01/ >> >> Updates in this round of changes : >> ? - replaced index() with strchr() to avoid platform dependent issues >> with strings.h include >> ? - removed NSK_JVMTI_OPTION_VAL_SEP >> ? - removed temporary debugging print statements >> ? - removed empty options string from SetNativeMethodPrefix001 >> ? - added free for temporary strdup buffer >> ? - updated copyright for 2019 >> >> On 12/21/18 1:52 PM, Gary Adams wrote: >>> Here is a first pass at a replacement parser for jvmti test options. >>> >>> ? Webrev: http://cr.openjdk.java.net/~gadams/8211343/webrev.00/ >>> ? Issue: https://bugs.openjdk.java.net/browse/JDK-8211343 >>> >>> Testing is in progress. Need to check out more platforms. >>> On local linux testing one jvmti test failed SetNativeMethodPrefix001 >>> which passed an empty options string. >>> ... >> >> > > From hohensee at amazon.com Mon Dec 31 20:56:49 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Mon, 31 Dec 2018 20:56:49 +0000 Subject: [RFR]8215623: Add incremental dump for jmap histo In-Reply-To: <66546343ae324ab28bb54951ad774a89@jd.com> References: <66546343ae324ab28bb54951ad774a89@jd.com> Message-ID: <32BE694F-58A4-4BC0-88A9-9295FE9411F6@amazon.com> As for 8215622, update the copyright dates to 2019 please, since this won?t get pushed until then. You might generalize the implementation so that all inspections are done incrementally, and parameterize RecordInstanceClosure with the incremental threshold. ?incremental? could become ?chunkcount=?, where defaults to infinity (max value of size_t). I?d not use a default file name when ?chunkcount? is specified, I?d just write to whatever the output stream is. ?chunkcount? is then independent of ?file?. I?d add another histo argument for the maximum file size, call it ?maxfilesize?, and parameterize RecordInstanceClosure with it. Default would be infinity (max value of size_t). If you want to make it easy to use your 8k and 5mb chunkcount and maxfilesize combination, you could redefine your original ?incremental? argument as syntactic sugar for ?chunkcount=8k,maxfilesize=5m?. INCREMENTAL_THRESHOLD and MAX_INCREMENTAL_FILESIZE become DEFAULT_CHUNKSIZE and MAX_FILE_SIZE, are both set to max size_t, and should be defined as ?const int? within RecordInstanceClosure. Thanks, Paul From: serviceability-dev on behalf of ?? Date: Thursday, December 20, 2018 at 11:13 PM To: "serviceability-dev at openjdk.java.net" Subject: [RFR]8215623: Add incremental dump for jmap histo Hi All, May I ask your help to review this patch for enhance jmap ?histo. It adds the ?incremental? arguments that allow jmap ?histo to incrementally save the intermediate data into a temp file. The intermediate data is dumped incrementally and write to a rolling file, which limit the size of the temp file to be small. This is useful for user to get intermediate results when jmap/jvm process is killed accidentally. Especially when the heap is large. This patch is also part of the enhancement described in https://bugs.openjdk.java.net/browse/JDK-8214535. Webrev: http://cr.openjdk.java.net/~xiaofeya/8215623/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8215623 Thanks. BRs, Lin -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.plummer at oracle.com Mon Dec 31 21:25:34 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 31 Dec 2018 13:25:34 -0800 Subject: RFR: JDK-8211343: nsk_jvmti_parseoptions should handle multiple suboptions In-Reply-To: References: <5C1D3662.4070505@oracle.com> Message-ID: <45b3cee6-0922-3f14-df73-91b2cea63bac@oracle.com> Ok. Chris On 12/31/18 12:30 PM, gary.adams at oracle.com wrote: > The one test that failed is changed in this webrev. > > When it failed, it was the check in add_option > that does not allow an "empty" option name. > > > On 12/31/18 2:25 PM, Chris Plummer wrote: >> Looks good. Are empty options strings still allowed after your changes? >> >> Chris >> >> On 12/31/18 10:06 AM, gary.adams at oracle.com wrote: >>> Here's a revised webrev. >>> >>> ? Webrev: >>> http://bussund0416.us.oracle.com/export/users/gradams/work/webrevs/8211343/webrev.01/ >>> >>> Updates in this round of changes : >>> ? - replaced index() with strchr() to avoid platform dependent >>> issues with strings.h include >>> ? - removed NSK_JVMTI_OPTION_VAL_SEP >>> ? - removed temporary debugging print statements >>> ? - removed empty options string from SetNativeMethodPrefix001 >>> ? - added free for temporary strdup buffer >>> ? - updated copyright for 2019 >>> >>> On 12/21/18 1:52 PM, Gary Adams wrote: >>>> Here is a first pass at a replacement parser for jvmti test options. >>>> >>>> ? Webrev: http://cr.openjdk.java.net/~gadams/8211343/webrev.00/ >>>> ? Issue: https://bugs.openjdk.java.net/browse/JDK-8211343 >>>> >>>> Testing is in progress. Need to check out more platforms. >>>> On local linux testing one jvmti test failed SetNativeMethodPrefix001 >>>> which passed an empty options string. >>>> ... >>> >>> >> >> >